2024-09-10T15:47:27.290 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-10T15:47:27.308 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-10T15:47:27.394 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/skanta-2024-09-10_15:30:27-rados-wip-bharath7-testing-2024-09-10-1409-distro-default-smithi/7899267 branch: wip-bharath7-testing-2024-09-10-1409 description: rados/singleton/{all/lost-unfound mon_election/classic msgr-failures/none msgr/async-v2only objectstore/{bluestore-options/write$/{write_v1} bluestore/bluestore-stupid} rados supported-random-distro$/{centos_latest}} email: skanta@redhat.com first_in_suite: false job_id: '7899267' kernel: kdb: 1 sha1: distro last_in_suite: false machine_type: smithi name: skanta-2024-09-10_15:30:27-rados-wip-bharath7-testing-2024-09-10-1409-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 3 size: 10 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-bharath7-testing-2024-09-10-1409 ceph: conf: global: mon election default strategy: 1 ms bind msgr1: false ms bind msgr2: true ms type: async mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon scrub interval: 300 osd: bdev async discard: true bdev enable discard: true bluestore allocator: stupid bluestore block size: 96636764160 bluestore fsck on mount: true bluestore write v2: false bluestore zero block detection: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd debug verify cached snaps: true osd debug verify missing on start: true osd failsafe full ratio: 0.95 osd mclock override recovery settings: true osd mclock profile: high_recovery_ops osd objectstore: bluestore osd op queue: debug_random osd op queue cut off: debug_random flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) sha1: bad1f558c327094897e4a32e99dd5cb4ae6ccd46 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: bad1f558c327094897e4a32e99dd5cb4ae6ccd46 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-bharath7-testing-2024-09-10-1409 sha1: bad1f558c327094897e4a32e99dd5cb4ae6ccd46 owner: scheduled_skanta@teuthology priority: 99 repo: https://github.com/ceph/ceph-ci.git roles: - - mon.a - mon.b - mon.c - mgr.x - osd.0 - osd.1 - osd.2 seed: 5245 sha1: bad1f558c327094897e4a32e99dd5cb4ae6ccd46 sleep_before_teardown: 0 subset: 111/120000 suite: rados suite_branch: wip-bharath7-testing-2024-09-10-1409 suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_bad1f558c327094897e4a32e99dd5cb4ae6ccd46/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci.git suite_sha1: bad1f558c327094897e4a32e99dd5cb4ae6ccd46 targets: smithi115.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAbn0gFLrlXEllx0KRe2j0SykP79/6lDKSBUs3JykdjnpNbpBuhOftUWXzM7lVw8JebkeQkBHLUS6R+mREJfCbI= tasks: - install: null - ceph: log-ignorelist: - objects unfound and apparently lost - overall HEALTH_ - \(OSDMAP_FLAGS\) - \(OSD_ - \(PG_ - \(OBJECT_ - \(SLOW_OPS\) - slow request - \(POOL_APP_NOT_ENABLED\) pre-mgr-commands: - sudo ceph config set mgr mgr_pool false --force - lost_unfound: null teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-10_15:30:27 tube: smithi user: skanta verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-10T15:47:27.394 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_bad1f558c327094897e4a32e99dd5cb4ae6ccd46/qa; will attempt to use it 2024-09-10T15:47:27.395 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_bad1f558c327094897e4a32e99dd5cb4ae6ccd46/qa/tasks 2024-09-10T15:47:27.395 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-10T15:47:27.396 INFO:teuthology.task.internal:Checking packages... 2024-09-10T15:47:27.417 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'bad1f558c327094897e4a32e99dd5cb4ae6ccd46' 2024-09-10T15:47:27.417 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-10T15:47:27.417 INFO:teuthology.packaging:ref: None 2024-09-10T15:47:27.417 INFO:teuthology.packaging:tag: None 2024-09-10T15:47:27.418 INFO:teuthology.packaging:branch: wip-bharath7-testing-2024-09-10-1409 2024-09-10T15:47:27.418 INFO:teuthology.packaging:sha1: bad1f558c327094897e4a32e99dd5cb4ae6ccd46 2024-09-10T15:47:27.418 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-bharath7-testing-2024-09-10-1409 2024-09-10T15:47:27.565 INFO:teuthology.task.internal:Found packages for ceph version 19.3.0-4858.gbad1f558 2024-09-10T15:47:27.566 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-10T15:47:27.575 INFO:teuthology.task.internal:no buildpackages task found 2024-09-10T15:47:27.575 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-10T15:47:27.616 INFO:teuthology.task.internal:Saving configuration 2024-09-10T15:47:27.628 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-10T15:47:27.641 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-10T15:47:27.661 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi115.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/skanta-2024-09-10_15:30:27-rados-wip-bharath7-testing-2024-09-10-1409-distro-default-smithi/7899267', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-10 15:42:23.976625', 'locked_by': 'scheduled_skanta@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAbn0gFLrlXEllx0KRe2j0SykP79/6lDKSBUs3JykdjnpNbpBuhOftUWXzM7lVw8JebkeQkBHLUS6R+mREJfCbI='} 2024-09-10T15:47:27.661 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-10T15:47:27.673 INFO:teuthology.task.internal:roles: ubuntu@smithi115.front.sepia.ceph.com - ['mon.a', 'mon.b', 'mon.c', 'mgr.x', 'osd.0', 'osd.1', 'osd.2'] 2024-09-10T15:47:27.673 INFO:teuthology.run_tasks:Running task console_log... 2024-09-10T15:47:27.725 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f9d5fa67be0>, signals=[15]) 2024-09-10T15:47:27.725 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-10T15:47:27.737 INFO:teuthology.task.internal:Opening connections... 2024-09-10T15:47:27.737 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi115.front.sepia.ceph.com 2024-09-10T15:47:27.739 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi115.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T15:47:27.813 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-10T15:47:27.819 DEBUG:teuthology.orchestra.run.smithi115:> uname -m 2024-09-10T15:47:27.895 INFO:teuthology.orchestra.run.smithi115.stdout:x86_64 2024-09-10T15:47:27.895 DEBUG:teuthology.orchestra.run.smithi115:> cat /etc/os-release 2024-09-10T15:47:27.953 INFO:teuthology.orchestra.run.smithi115.stdout:NAME="CentOS Stream" 2024-09-10T15:47:27.954 INFO:teuthology.orchestra.run.smithi115.stdout:VERSION="9" 2024-09-10T15:47:27.954 INFO:teuthology.orchestra.run.smithi115.stdout:ID="centos" 2024-09-10T15:47:27.954 INFO:teuthology.orchestra.run.smithi115.stdout:ID_LIKE="rhel fedora" 2024-09-10T15:47:27.954 INFO:teuthology.orchestra.run.smithi115.stdout:VERSION_ID="9" 2024-09-10T15:47:27.954 INFO:teuthology.orchestra.run.smithi115.stdout:PLATFORM_ID="platform:el9" 2024-09-10T15:47:27.954 INFO:teuthology.orchestra.run.smithi115.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-10T15:47:27.954 INFO:teuthology.orchestra.run.smithi115.stdout:ANSI_COLOR="0;31" 2024-09-10T15:47:27.954 INFO:teuthology.orchestra.run.smithi115.stdout:LOGO="fedora-logo-icon" 2024-09-10T15:47:27.954 INFO:teuthology.orchestra.run.smithi115.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-10T15:47:27.954 INFO:teuthology.orchestra.run.smithi115.stdout:HOME_URL="https://centos.org/" 2024-09-10T15:47:27.954 INFO:teuthology.orchestra.run.smithi115.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-10T15:47:27.954 INFO:teuthology.orchestra.run.smithi115.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-10T15:47:27.954 INFO:teuthology.orchestra.run.smithi115.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-10T15:47:27.955 INFO:teuthology.lock.ops:Updating smithi115.front.sepia.ceph.com on lock server 2024-09-10T15:47:27.978 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-10T15:47:27.987 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-10T15:47:28.010 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-10T15:47:28.011 DEBUG:teuthology.orchestra.run.smithi115:> test '!' -e /home/ubuntu/cephtest 2024-09-10T15:47:28.028 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-10T15:47:28.037 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-10T15:47:28.037 DEBUG:teuthology.orchestra.run.smithi115:> test -z $(ls -A /var/lib/ceph) 2024-09-10T15:47:28.112 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-10T15:47:28.144 INFO:teuthology.run_tasks:Running task kernel... 2024-09-10T15:47:28.161 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-09-10T15:47:28.161 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-09-10T15:47:28.161 DEBUG:teuthology.orchestra.run.smithi115:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-10T15:47:28.177 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T15:47:28.178 DEBUG:teuthology.orchestra.run.smithi115:> uname -r 2024-09-10T15:47:28.235 INFO:teuthology.orchestra.run.smithi115.stdout:5.14.0-503.el9.x86_64 2024-09-10T15:47:28.235 INFO:teuthology.task.kernel:Running kernel on smithi115: 5.14.0-503.el9.x86_64 2024-09-10T15:47:28.235 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum install -y kernel 2024-09-10T15:47:31.787 INFO:teuthology.orchestra.run.smithi115.stdout:CentOS Stream 9 - BaseOS 4.5 MB/s | 8.2 MB 00:01 2024-09-10T15:47:33.804 INFO:teuthology.orchestra.run.smithi115.stdout:CentOS Stream 9 - AppStream 22 MB/s | 20 MB 00:00 2024-09-10T15:47:38.238 INFO:teuthology.orchestra.run.smithi115.stdout:CentOS Stream 9 - CRB 10 MB/s | 6.5 MB 00:00 2024-09-10T15:47:39.880 INFO:teuthology.orchestra.run.smithi115.stdout:CentOS Stream 9 - Extras packages 45 kB/s | 19 kB 00:00 2024-09-10T15:47:40.876 INFO:teuthology.orchestra.run.smithi115.stdout:Extra Packages for Enterprise Linux 26 MB/s | 23 MB 00:00 2024-09-10T15:47:46.306 INFO:teuthology.orchestra.run.smithi115.stdout:lab-extras 33 kB/s | 1.7 kB 00:00 2024-09-10T15:47:48.329 INFO:teuthology.orchestra.run.smithi115.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-10T15:47:48.329 INFO:teuthology.orchestra.run.smithi115.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-10T15:47:48.412 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2024-09-10T15:47:48.416 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-10T15:47:48.416 INFO:teuthology.orchestra.run.smithi115.stdout: Package Architecture Version Repository Size 2024-09-10T15:47:48.416 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-10T15:47:48.416 INFO:teuthology.orchestra.run.smithi115.stdout:Installing: 2024-09-10T15:47:48.416 INFO:teuthology.orchestra.run.smithi115.stdout: kernel x86_64 5.14.0-505.el9 baseos 23 k 2024-09-10T15:47:48.417 INFO:teuthology.orchestra.run.smithi115.stdout:Installing dependencies: 2024-09-10T15:47:48.417 INFO:teuthology.orchestra.run.smithi115.stdout: kernel-core x86_64 5.14.0-505.el9 baseos 16 M 2024-09-10T15:47:48.417 INFO:teuthology.orchestra.run.smithi115.stdout: kernel-modules x86_64 5.14.0-505.el9 baseos 35 M 2024-09-10T15:47:48.417 INFO:teuthology.orchestra.run.smithi115.stdout: kernel-modules-core x86_64 5.14.0-505.el9 baseos 29 M 2024-09-10T15:47:48.417 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T15:47:48.417 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction Summary 2024-09-10T15:47:48.417 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-10T15:47:48.417 INFO:teuthology.orchestra.run.smithi115.stdout:Install 4 Packages 2024-09-10T15:47:48.417 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T15:47:48.418 INFO:teuthology.orchestra.run.smithi115.stdout:Total download size: 79 M 2024-09-10T15:47:48.418 INFO:teuthology.orchestra.run.smithi115.stdout:Installed size: 126 M 2024-09-10T15:47:48.418 INFO:teuthology.orchestra.run.smithi115.stdout:Downloading Packages: 2024-09-10T15:47:48.801 INFO:teuthology.orchestra.run.smithi115.stdout:(1/4): kernel-5.14.0-505.el9.x86_64.rpm 182 kB/s | 23 kB 00:00 2024-09-10T15:47:55.340 INFO:teuthology.orchestra.run.smithi115.stdout:(2/4): kernel-core-5.14.0-505.el9.x86_64.rpm 2.3 MB/s | 16 MB 00:06 2024-09-10T15:47:57.956 INFO:teuthology.orchestra.run.smithi115.stdout:(3/4): kernel-modules-core-5.14.0-505.el9.x86_6 3.1 MB/s | 29 MB 00:09 2024-09-10T15:48:03.089 INFO:teuthology.orchestra.run.smithi115.stdout:(4/4): kernel-modules-5.14.0-505.el9.x86_64.rpm 2.4 MB/s | 35 MB 00:14 2024-09-10T15:48:03.089 INFO:teuthology.orchestra.run.smithi115.stdout:-------------------------------------------------------------------------------- 2024-09-10T15:48:03.089 INFO:teuthology.orchestra.run.smithi115.stdout:Total 5.4 MB/s | 79 MB 00:14 2024-09-10T15:48:03.772 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction check 2024-09-10T15:48:04.122 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction check succeeded. 2024-09-10T15:48:04.123 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction test 2024-09-10T15:48:05.095 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction test succeeded. 2024-09-10T15:48:05.095 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction 2024-09-10T15:48:07.529 INFO:teuthology.orchestra.run.smithi115.stdout: Preparing : 1/1 2024-09-10T15:48:08.230 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : kernel-modules-core-5.14.0-505.el9.x86_64 1/4 2024-09-10T15:48:08.339 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-10T15:48:11.146 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-10T15:48:11.276 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-10T15:48:16.128 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-10T15:48:16.226 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : kernel-5.14.0-505.el9.x86_64 4/4 2024-09-10T15:48:21.086 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-10T15:48:59.355 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 4/4 2024-09-10T15:48:59.396 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 4/4 2024-09-10T15:48:59.705 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: kernel-5.14.0-505.el9.x86_64 4/4 2024-09-10T15:48:59.705 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : kernel-5.14.0-505.el9.x86_64 1/4 2024-09-10T15:48:59.705 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-10T15:48:59.705 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-10T15:49:00.551 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-10T15:49:00.551 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T15:49:00.551 INFO:teuthology.orchestra.run.smithi115.stdout:Installed: 2024-09-10T15:49:00.551 INFO:teuthology.orchestra.run.smithi115.stdout: kernel-5.14.0-505.el9.x86_64 2024-09-10T15:49:00.551 INFO:teuthology.orchestra.run.smithi115.stdout: kernel-core-5.14.0-505.el9.x86_64 2024-09-10T15:49:00.551 INFO:teuthology.orchestra.run.smithi115.stdout: kernel-modules-5.14.0-505.el9.x86_64 2024-09-10T15:49:00.551 INFO:teuthology.orchestra.run.smithi115.stdout: kernel-modules-core-5.14.0-505.el9.x86_64 2024-09-10T15:49:00.551 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T15:49:00.551 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2024-09-10T15:49:00.919 DEBUG:teuthology.orchestra.run.smithi115:> rpm -q kernel | sort -rV | head -n 1 2024-09-10T15:49:00.998 INFO:teuthology.orchestra.run.smithi115.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-10T15:49:00.998 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-10T15:49:00.999 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-505.el9.x86_64 2024-09-10T15:49:00.999 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-10T15:49:00.999 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-10T15:49:00.999 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-09-10T15:49:00.999 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi115.front.sepia.ceph.com, path=None, version=distro) 2024-09-10T15:49:00.999 DEBUG:teuthology.orchestra.run.smithi115:> rpm -q kernel | sort -rV | head -n 1 2024-09-10T15:49:01.058 INFO:teuthology.orchestra.run.smithi115.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-10T15:49:01.059 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-10T15:49:01.059 DEBUG:teuthology.orchestra.run.smithi115:> sudo rpm -qi grub2-tools 2024-09-10T15:49:01.091 INFO:teuthology.orchestra.run.smithi115.stdout:Name : grub2-tools 2024-09-10T15:49:01.092 INFO:teuthology.orchestra.run.smithi115.stdout:Epoch : 1 2024-09-10T15:49:01.092 INFO:teuthology.orchestra.run.smithi115.stdout:Version : 2.06 2024-09-10T15:49:01.092 INFO:teuthology.orchestra.run.smithi115.stdout:Release : 82.el9 2024-09-10T15:49:01.092 INFO:teuthology.orchestra.run.smithi115.stdout:Architecture: x86_64 2024-09-10T15:49:01.092 INFO:teuthology.orchestra.run.smithi115.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-10T15:49:01.092 INFO:teuthology.orchestra.run.smithi115.stdout:Group : Unspecified 2024-09-10T15:49:01.092 INFO:teuthology.orchestra.run.smithi115.stdout:Size : 8274670 2024-09-10T15:49:01.092 INFO:teuthology.orchestra.run.smithi115.stdout:License : GPLv3+ 2024-09-10T15:49:01.092 INFO:teuthology.orchestra.run.smithi115.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-10T15:49:01.092 INFO:teuthology.orchestra.run.smithi115.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-10T15:49:01.092 INFO:teuthology.orchestra.run.smithi115.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-10T15:49:01.092 INFO:teuthology.orchestra.run.smithi115.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-10T15:49:01.092 INFO:teuthology.orchestra.run.smithi115.stdout:Packager : builder@centos.org 2024-09-10T15:49:01.093 INFO:teuthology.orchestra.run.smithi115.stdout:Vendor : CentOS 2024-09-10T15:49:01.093 INFO:teuthology.orchestra.run.smithi115.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-10T15:49:01.093 INFO:teuthology.orchestra.run.smithi115.stdout:Summary : Support tools for GRUB. 2024-09-10T15:49:01.093 INFO:teuthology.orchestra.run.smithi115.stdout:Description : 2024-09-10T15:49:01.093 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T15:49:01.093 INFO:teuthology.orchestra.run.smithi115.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-10T15:49:01.093 INFO:teuthology.orchestra.run.smithi115.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-10T15:49:01.093 INFO:teuthology.orchestra.run.smithi115.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-10T15:49:01.093 INFO:teuthology.orchestra.run.smithi115.stdout:hardware devices. 2024-09-10T15:49:01.093 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T15:49:01.094 INFO:teuthology.orchestra.run.smithi115.stdout:This subpackage provides tools for support of all platforms. 2024-09-10T15:49:01.095 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-10T15:49:01.095 INFO:teuthology.task.kernel:Updating grub on smithi115 to boot 5.14.0-505.el9.x86_64 2024-09-10T15:49:01.095 DEBUG:teuthology.orchestra.run.smithi115:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-10T15:49:01.755 INFO:teuthology.orchestra.run.smithi115.stderr:Generating grub configuration file ... 2024-09-10T15:49:03.058 INFO:teuthology.orchestra.run.smithi115.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-10T15:49:03.089 INFO:teuthology.orchestra.run.smithi115.stderr:done 2024-09-10T15:49:03.092 DEBUG:teuthology.orchestra.run.smithi115:> mktemp 2024-09-10T15:49:03.109 INFO:teuthology.orchestra.run.smithi115.stdout:/tmp/tmp.bqfxQzzWrs 2024-09-10T15:49:03.109 DEBUG:teuthology.orchestra.run.smithi115:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.bqfxQzzWrs 2024-09-10T15:49:03.176 DEBUG:teuthology.orchestra.run.smithi115:> sudo chmod 0666 /tmp/tmp.bqfxQzzWrs 2024-09-10T15:49:03.313 DEBUG:teuthology.orchestra.remote:smithi115:/tmp/tmp.bqfxQzzWrs is 6KB 2024-09-10T15:49:03.362 DEBUG:teuthology.orchestra.run.smithi115:> rm -fr /tmp/tmp.bqfxQzzWrs 2024-09-10T15:49:03.378 DEBUG:teuthology.orchestra.run.smithi115:> sudo /bin/ls /boot/loader/entries || true 2024-09-10T15:49:03.440 INFO:teuthology.orchestra.run.smithi115.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-10T15:49:03.440 INFO:teuthology.orchestra.run.smithi115.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-10T15:49:03.440 INFO:teuthology.orchestra.run.smithi115.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-10T15:49:03.440 INFO:teuthology.orchestra.run.smithi115.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64.conf 2024-09-10T15:49:03.441 DEBUG:teuthology.orchestra.run.smithi115:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64 2024-09-10T15:49:03.597 DEBUG:teuthology.orchestra.run.smithi115:> sudo shutdown -r now 2024-09-10T15:49:33.629 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-09-10T15:49:33.630 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi115.front.sepia.ceph.com' 2024-09-10T15:49:33.631 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi115.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T15:50:08.193 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.115 2024-09-10T15:50:17.201 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi115.front.sepia.ceph.com' 2024-09-10T15:50:17.202 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi115.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T15:50:35.581 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.115 2024-09-10T15:50:47.593 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi115.front.sepia.ceph.com' 2024-09-10T15:50:47.594 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi115.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T15:50:48.226 DEBUG:teuthology.orchestra.run.smithi115:> true 2024-09-10T15:50:48.692 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi115.front.sepia.ceph.com' 2024-09-10T15:50:48.692 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.14.0-505.el9.x86_64"... 2024-09-10T15:50:48.692 DEBUG:teuthology.orchestra.run.smithi115:> uname -r 2024-09-10T15:50:48.707 INFO:teuthology.orchestra.run.smithi115.stdout:5.14.0-505.el9.x86_64 2024-09-10T15:50:48.707 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-505.el9.x86_64 vs 5.14.0-505.el9.x86_64 2024-09-10T15:50:48.707 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-10T15:50:48.707 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-10T15:50:49.709 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-09-10T15:50:49.709 DEBUG:teuthology.orchestra.run.smithi115:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-10T15:50:49.800 INFO:teuthology.orchestra.run.smithi115.stdout:ttyS1 2024-09-10T15:50:49.825 DEBUG:teuthology.parallel:result is None 2024-09-10T15:50:49.826 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-10T15:50:49.834 INFO:teuthology.task.internal:Creating test directory... 2024-09-10T15:50:49.834 DEBUG:teuthology.orchestra.run.smithi115:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-10T15:50:49.850 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-10T15:50:49.871 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-10T15:50:49.904 INFO:teuthology.task.internal:Creating archive directory... 2024-09-10T15:50:49.905 DEBUG:teuthology.orchestra.run.smithi115:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-10T15:50:49.932 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-10T15:50:49.981 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-10T15:50:49.981 DEBUG:teuthology.orchestra.run.smithi115:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-10T15:50:50.013 INFO:teuthology.orchestra.run.smithi115.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-10T15:50:50.023 INFO:teuthology.orchestra.run.smithi115.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-10T15:50:50.025 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-10T15:50:50.032 INFO:teuthology.task.internal:Configuring sudo... 2024-09-10T15:50:50.033 DEBUG:teuthology.orchestra.run.smithi115:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-10T15:50:50.091 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-10T15:50:50.100 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-10T15:50:50.100 DEBUG:teuthology.orchestra.run.smithi115:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-10T15:50:50.146 DEBUG:teuthology.orchestra.run.smithi115:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-10T15:50:50.231 DEBUG:teuthology.orchestra.run.smithi115:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-10T15:50:50.280 DEBUG:teuthology.orchestra.run.smithi115:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-10T15:50:50.361 DEBUG:teuthology.orchestra.run.smithi115:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-10T15:50:50.401 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-09-10T15:50:50.402 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-10T15:50:50.467 DEBUG:teuthology.orchestra.run.smithi115:> sudo service rsyslog restart 2024-09-10T15:50:50.557 INFO:teuthology.orchestra.run.smithi115.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-10T15:50:51.029 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-10T15:50:51.039 INFO:teuthology.task.internal:Starting timer... 2024-09-10T15:50:51.039 INFO:teuthology.run_tasks:Running task pcp... 2024-09-10T15:50:51.072 INFO:teuthology.run_tasks:Running task selinux... 2024-09-10T15:50:51.104 DEBUG:teuthology.orchestra.run.smithi115:> sudo service auditd rotate 2024-09-10T15:50:51.149 INFO:teuthology.orchestra.run.smithi115.stdout:Rotating logs: 2024-09-10T15:50:51.151 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-10T15:50:51.151 DEBUG:teuthology.orchestra.run.smithi115:> /usr/sbin/getenforce 2024-09-10T15:50:51.216 INFO:teuthology.orchestra.run.smithi115.stdout:Permissive 2024-09-10T15:50:51.216 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi115.front.sepia.ceph.com': 'permissive'} 2024-09-10T15:50:51.216 DEBUG:teuthology.orchestra.run.smithi115:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-10T15:50:51.287 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T15:50:51.287 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-10T15:50:51.288 DEBUG:teuthology.orchestra.run.smithi115:> sudo /usr/sbin/setenforce permissive 2024-09-10T15:50:51.375 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-10T15:50:51.384 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-10T15:50:51.395 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-10T15:50:51.396 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi115.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-10T15:54:33.051 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi115.front.sepia.ceph.com')] 2024-09-10T15:54:33.053 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi115.front.sepia.ceph.com' 2024-09-10T15:54:33.054 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi115.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T15:54:33.130 DEBUG:teuthology.orchestra.run.smithi115:> true 2024-09-10T15:54:33.211 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi115.front.sepia.ceph.com' 2024-09-10T15:54:33.211 INFO:teuthology.run_tasks:Running task clock... 2024-09-10T15:54:33.222 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-10T15:54:33.222 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-10T15:54:33.223 DEBUG:teuthology.orchestra.run.smithi115:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-10T15:54:33.280 INFO:teuthology.orchestra.run.smithi115.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-10T15:54:33.294 INFO:teuthology.orchestra.run.smithi115.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-10T15:54:33.318 INFO:teuthology.orchestra.run.smithi115.stderr:sudo: ntpd: command not found 2024-09-10T15:54:33.329 INFO:teuthology.orchestra.run.smithi115.stdout:506 Cannot talk to daemon 2024-09-10T15:54:33.343 INFO:teuthology.orchestra.run.smithi115.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-10T15:54:33.357 INFO:teuthology.orchestra.run.smithi115.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-10T15:54:33.411 INFO:teuthology.orchestra.run.smithi115.stderr:bash: line 1: ntpq: command not found 2024-09-10T15:54:33.414 INFO:teuthology.orchestra.run.smithi115.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-10T15:54:33.414 INFO:teuthology.orchestra.run.smithi115.stdout:=============================================================================== 2024-09-10T15:54:33.414 INFO:teuthology.orchestra.run.smithi115.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T15:54:33.414 INFO:teuthology.orchestra.run.smithi115.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T15:54:33.414 INFO:teuthology.orchestra.run.smithi115.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T15:54:33.414 INFO:teuthology.orchestra.run.smithi115.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T15:54:33.416 INFO:teuthology.run_tasks:Running task install... 2024-09-10T15:54:33.423 DEBUG:teuthology.task.install:project ceph 2024-09-10T15:54:33.424 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'bad1f558c327094897e4a32e99dd5cb4ae6ccd46'}} 2024-09-10T15:54:33.424 DEBUG:teuthology.task.install:config {'flavor': 'default', 'sha1': 'bad1f558c327094897e4a32e99dd5cb4ae6ccd46'} 2024-09-10T15:54:33.424 INFO:teuthology.task.install:Using flavor: default 2024-09-10T15:54:33.429 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-10T15:54:33.429 INFO:teuthology.task.install:extra packages: [] 2024-09-10T15:54:33.430 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'bad1f558c327094897e4a32e99dd5cb4ae6ccd46', 'tag': None, 'wait_for_package': False} 2024-09-10T15:54:33.430 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=bad1f558c327094897e4a32e99dd5cb4ae6ccd46 2024-09-10T15:54:33.579 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-bharath7-testing-2024-09-10-1409/bad1f558c327094897e4a32e99dd5cb4ae6ccd46/centos/9/flavors/default/ 2024-09-10T15:54:33.579 INFO:teuthology.task.install.rpm:Package version is 19.3.0-4858.gbad1f558 2024-09-10T15:54:33.691 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-bharath7-testing-2024-09-10-1409/bad1f558c327094897e4a32e99dd5cb4ae6ccd46/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-bharath7-testing-2024-09-10-1409/bad1f558c327094897e4a32e99dd5cb4ae6ccd46/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-bharath7-testing-2024-09-10-1409/bad1f558c327094897e4a32e99dd5cb4ae6ccd46/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-10T15:54:33.691 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-09-10T15:54:33.691 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-10T15:54:33.717 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-09-10T15:54:33.718 DEBUG:teuthology.orchestra.run.smithi115:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/bad1f558c327094897e4a32e99dd5cb4ae6ccd46/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-10T15:54:33.784 DEBUG:teuthology.orchestra.run.smithi115:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-10T15:54:33.861 DEBUG:teuthology.orchestra.run.smithi115:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-10T15:54:33.926 INFO:teuthology.orchestra.run.smithi115.stdout:check_obsoletes = 1 2024-09-10T15:54:33.928 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum clean all 2024-09-10T15:54:34.156 INFO:teuthology.orchestra.run.smithi115.stdout:45 files removed 2024-09-10T15:54:34.183 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum -y install ceph-radosgw 2024-09-10T15:54:34.767 INFO:teuthology.orchestra.run.smithi115.stdout:ceph packages for x86_64 279 kB/s | 84 kB 00:00 2024-09-10T15:54:35.051 INFO:teuthology.orchestra.run.smithi115.stdout:ceph noarch packages 101 kB/s | 24 kB 00:00 2024-09-10T15:54:35.326 INFO:teuthology.orchestra.run.smithi115.stdout:ceph source packages 8.1 kB/s | 1.9 kB 00:00 2024-09-10T15:54:36.902 INFO:teuthology.orchestra.run.smithi115.stdout:CentOS Stream 9 - BaseOS 5.4 MB/s | 8.2 MB 00:01 2024-09-10T15:54:41.515 INFO:teuthology.orchestra.run.smithi115.stdout:CentOS Stream 9 - AppStream 5.8 MB/s | 20 MB 00:03 2024-09-10T15:54:47.036 INFO:teuthology.orchestra.run.smithi115.stdout:CentOS Stream 9 - CRB 3.9 MB/s | 6.5 MB 00:01 2024-09-10T15:54:48.895 INFO:teuthology.orchestra.run.smithi115.stdout:CentOS Stream 9 - Extras packages 36 kB/s | 19 kB 00:00 2024-09-10T15:54:49.665 INFO:teuthology.orchestra.run.smithi115.stdout:Extra Packages for Enterprise Linux 35 MB/s | 23 MB 00:00 2024-09-10T15:54:55.162 INFO:teuthology.orchestra.run.smithi115.stdout:lab-extras 32 kB/s | 1.7 kB 00:00 2024-09-10T15:54:57.589 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2024-09-10T15:54:57.593 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-10T15:54:57.593 INFO:teuthology.orchestra.run.smithi115.stdout: Package Arch Version Repository Size 2024-09-10T15:54:57.593 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-10T15:54:57.593 INFO:teuthology.orchestra.run.smithi115.stdout:Installing: 2024-09-10T15:54:57.594 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-radosgw x86_64 2:19.3.0-4858.gbad1f558.el9 ceph 11 M 2024-09-10T15:54:57.594 INFO:teuthology.orchestra.run.smithi115.stdout:Upgrading: 2024-09-10T15:54:57.594 INFO:teuthology.orchestra.run.smithi115.stdout: librados2 x86_64 2:19.3.0-4858.gbad1f558.el9 ceph 3.5 M 2024-09-10T15:54:57.594 INFO:teuthology.orchestra.run.smithi115.stdout: librbd1 x86_64 2:19.3.0-4858.gbad1f558.el9 ceph 3.2 M 2024-09-10T15:54:57.594 INFO:teuthology.orchestra.run.smithi115.stdout:Installing dependencies: 2024-09-10T15:54:57.594 INFO:teuthology.orchestra.run.smithi115.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-10T15:54:57.594 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-base x86_64 2:19.3.0-4858.gbad1f558.el9 ceph 5.5 M 2024-09-10T15:54:57.594 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-common x86_64 2:19.3.0-4858.gbad1f558.el9 ceph 22 M 2024-09-10T15:54:57.594 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-selinux x86_64 2:19.3.0-4858.gbad1f558.el9 ceph 25 k 2024-09-10T15:54:57.594 INFO:teuthology.orchestra.run.smithi115.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-09-10T15:54:57.594 INFO:teuthology.orchestra.run.smithi115.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-10T15:54:57.594 INFO:teuthology.orchestra.run.smithi115.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-10T15:54:57.594 INFO:teuthology.orchestra.run.smithi115.stdout: libcephfs2 x86_64 2:19.3.0-4858.gbad1f558.el9 ceph 755 k 2024-09-10T15:54:57.595 INFO:teuthology.orchestra.run.smithi115.stdout: libnbd x86_64 1.20.2-2.el9 appstream 171 k 2024-09-10T15:54:57.595 INFO:teuthology.orchestra.run.smithi115.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-10T15:54:57.595 INFO:teuthology.orchestra.run.smithi115.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-10T15:54:57.595 INFO:teuthology.orchestra.run.smithi115.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-10T15:54:57.595 INFO:teuthology.orchestra.run.smithi115.stdout: libradosstriper1 x86_64 2:19.3.0-4858.gbad1f558.el9 ceph 509 k 2024-09-10T15:54:57.595 INFO:teuthology.orchestra.run.smithi115.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-10T15:54:57.595 INFO:teuthology.orchestra.run.smithi115.stdout: librgw2 x86_64 2:19.3.0-4858.gbad1f558.el9 ceph 5.5 M 2024-09-10T15:54:57.595 INFO:teuthology.orchestra.run.smithi115.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-10T15:54:57.595 INFO:teuthology.orchestra.run.smithi115.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-10T15:54:57.595 INFO:teuthology.orchestra.run.smithi115.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-10T15:54:57.595 INFO:teuthology.orchestra.run.smithi115.stdout: python3-ceph-argparse x86_64 2:19.3.0-4858.gbad1f558.el9 ceph 45 k 2024-09-10T15:54:57.595 INFO:teuthology.orchestra.run.smithi115.stdout: python3-ceph-common x86_64 2:19.3.0-4858.gbad1f558.el9 ceph 143 k 2024-09-10T15:54:57.596 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cephfs x86_64 2:19.3.0-4858.gbad1f558.el9 ceph 161 k 2024-09-10T15:54:57.596 INFO:teuthology.orchestra.run.smithi115.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-10T15:54:57.596 INFO:teuthology.orchestra.run.smithi115.stdout: python3-rados x86_64 2:19.3.0-4858.gbad1f558.el9 ceph 321 k 2024-09-10T15:54:57.596 INFO:teuthology.orchestra.run.smithi115.stdout: python3-rbd x86_64 2:19.3.0-4858.gbad1f558.el9 ceph 306 k 2024-09-10T15:54:57.596 INFO:teuthology.orchestra.run.smithi115.stdout: python3-rgw x86_64 2:19.3.0-4858.gbad1f558.el9 ceph 99 k 2024-09-10T15:54:57.596 INFO:teuthology.orchestra.run.smithi115.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-10T15:54:57.596 INFO:teuthology.orchestra.run.smithi115.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-10T15:54:57.596 INFO:teuthology.orchestra.run.smithi115.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-10T15:54:57.596 INFO:teuthology.orchestra.run.smithi115.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-10T15:54:57.596 INFO:teuthology.orchestra.run.smithi115.stdout:Installing weak dependencies: 2024-09-10T15:54:57.596 INFO:teuthology.orchestra.run.smithi115.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-10T15:54:57.597 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T15:54:57.597 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction Summary 2024-09-10T15:54:57.597 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-10T15:54:57.597 INFO:teuthology.orchestra.run.smithi115.stdout:Install 31 Packages 2024-09-10T15:54:57.597 INFO:teuthology.orchestra.run.smithi115.stdout:Upgrade 2 Packages 2024-09-10T15:54:57.597 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T15:54:57.597 INFO:teuthology.orchestra.run.smithi115.stdout:Total download size: 62 M 2024-09-10T15:54:57.598 INFO:teuthology.orchestra.run.smithi115.stdout:Downloading Packages: 2024-09-10T15:54:58.313 INFO:teuthology.orchestra.run.smithi115.stdout:(1/33): ceph-base-19.3.0-4858.gbad1f558.el9.x86 9.1 MB/s | 5.5 MB 00:00 2024-09-10T15:54:58.355 INFO:teuthology.orchestra.run.smithi115.stdout:(2/33): ceph-selinux-19.3.0-4858.gbad1f558.el9. 606 kB/s | 25 kB 00:00 2024-09-10T15:54:58.506 INFO:teuthology.orchestra.run.smithi115.stdout:(3/33): ceph-radosgw-19.3.0-4858.gbad1f558.el9. 14 MB/s | 11 MB 00:00 2024-09-10T15:54:58.548 INFO:teuthology.orchestra.run.smithi115.stdout:(4/33): libcephfs2-19.3.0-4858.gbad1f558.el9.x8 3.8 MB/s | 755 kB 00:00 2024-09-10T15:54:58.581 INFO:teuthology.orchestra.run.smithi115.stdout:(5/33): libradosstriper1-19.3.0-4858.gbad1f558. 6.6 MB/s | 509 kB 00:00 2024-09-10T15:54:58.640 INFO:teuthology.orchestra.run.smithi115.stdout:(6/33): python3-ceph-argparse-19.3.0-4858.gbad1 776 kB/s | 45 kB 00:00 2024-09-10T15:54:58.698 INFO:teuthology.orchestra.run.smithi115.stdout:(7/33): python3-ceph-common-19.3.0-4858.gbad1f5 2.4 MB/s | 143 kB 00:00 2024-09-10T15:54:58.757 INFO:teuthology.orchestra.run.smithi115.stdout:(8/33): python3-cephfs-19.3.0-4858.gbad1f558.el 2.7 MB/s | 161 kB 00:00 2024-09-10T15:54:58.866 INFO:teuthology.orchestra.run.smithi115.stdout:(9/33): librgw2-19.3.0-4858.gbad1f558.el9.x86_6 17 MB/s | 5.5 MB 00:00 2024-09-10T15:54:58.891 INFO:teuthology.orchestra.run.smithi115.stdout:(10/33): python3-rados-19.3.0-4858.gbad1f558.el 2.3 MB/s | 321 kB 00:00 2024-09-10T15:54:58.916 INFO:teuthology.orchestra.run.smithi115.stdout:(11/33): python3-rbd-19.3.0-4858.gbad1f558.el9. 6.0 MB/s | 306 kB 00:00 2024-09-10T15:54:59.125 INFO:teuthology.orchestra.run.smithi115.stdout:(12/33): ceph-common-19.3.0-4858.gbad1f558.el9. 15 MB/s | 22 MB 00:01 2024-09-10T15:54:59.150 INFO:teuthology.orchestra.run.smithi115.stdout:(13/33): python3-rgw-19.3.0-4858.gbad1f558.el9. 383 kB/s | 99 kB 00:00 2024-09-10T15:54:59.250 INFO:teuthology.orchestra.run.smithi115.stdout:(14/33): boost-program-options-1.75.0-8.el9.x86 320 kB/s | 107 kB 00:00 2024-09-10T15:54:59.275 INFO:teuthology.orchestra.run.smithi115.stdout:(15/33): libnbd-1.20.2-2.el9.x86_64.rpm 1.1 MB/s | 171 kB 00:00 2024-09-10T15:54:59.302 INFO:teuthology.orchestra.run.smithi115.stdout:(16/33): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.0 MB/s | 160 kB 00:00 2024-09-10T15:54:59.326 INFO:teuthology.orchestra.run.smithi115.stdout:(17/33): librabbitmq-0.11.0-7.el9.x86_64.rpm 599 kB/s | 45 kB 00:00 2024-09-10T15:54:59.368 INFO:teuthology.orchestra.run.smithi115.stdout:(18/33): python3-prettytable-0.7.2-27.el9.noarc 1.0 MB/s | 42 kB 00:00 2024-09-10T15:54:59.393 INFO:teuthology.orchestra.run.smithi115.stdout:(19/33): librdkafka-1.6.1-102.el9.x86_64.rpm 5.5 MB/s | 662 kB 00:00 2024-09-10T15:54:59.427 INFO:teuthology.orchestra.run.smithi115.stdout:(20/33): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.8 MB/s | 292 kB 00:00 2024-09-10T15:54:59.452 INFO:teuthology.orchestra.run.smithi115.stdout:(21/33): qatlib-service-24.02.0-1.el9.x86_64.rp 613 kB/s | 36 kB 00:00 2024-09-10T15:54:59.477 INFO:teuthology.orchestra.run.smithi115.stdout:(22/33): qatlib-24.02.0-1.el9.x86_64.rpm 2.0 MB/s | 221 kB 00:00 2024-09-10T15:54:59.502 INFO:teuthology.orchestra.run.smithi115.stdout:(23/33): qatzip-libs-1.2.0-1.el9.x86_64.rpm 628 kB/s | 47 kB 00:00 2024-09-10T15:54:59.561 INFO:teuthology.orchestra.run.smithi115.stdout:(24/33): gperftools-libs-2.9.1-2.el9.x86_64.rpm 2.8 MB/s | 309 kB 00:00 2024-09-10T15:54:59.586 INFO:teuthology.orchestra.run.smithi115.stdout:(25/33): libarrow-doc-9.0.0-11.el9.noarch.rpm 312 kB/s | 26 kB 00:00 2024-09-10T15:54:59.611 INFO:teuthology.orchestra.run.smithi115.stdout:(26/33): liboath-2.6.7-2.el9.x86_64.rpm 968 kB/s | 49 kB 00:00 2024-09-10T15:54:59.637 INFO:teuthology.orchestra.run.smithi115.stdout:(27/33): libunwind-1.6.2-1.el9.x86_64.rpm 1.3 MB/s | 67 kB 00:00 2024-09-10T15:54:59.679 INFO:teuthology.orchestra.run.smithi115.stdout:(28/33): re2-20211101-20.el9.x86_64.rpm 4.5 MB/s | 191 kB 00:00 2024-09-10T15:54:59.712 INFO:teuthology.orchestra.run.smithi115.stdout:(29/33): parquet-libs-9.0.0-11.el9.x86_64.rpm 8.2 MB/s | 839 kB 00:00 2024-09-10T15:54:59.771 INFO:teuthology.orchestra.run.smithi115.stdout:(30/33): thrift-0.15.0-2.el9.x86_64.rpm 17 MB/s | 1.6 MB 00:00 2024-09-10T15:54:59.846 INFO:teuthology.orchestra.run.smithi115.stdout:(31/33): libarrow-9.0.0-11.el9.x86_64.rpm 13 MB/s | 4.4 MB 00:00 2024-09-10T15:54:59.941 INFO:teuthology.orchestra.run.smithi115.stdout:(32/33): librados2-19.3.0-4858.gbad1f558.el9.x8 15 MB/s | 3.5 MB 00:00 2024-09-10T15:55:00.057 INFO:teuthology.orchestra.run.smithi115.stdout:(33/33): librbd1-19.3.0-4858.gbad1f558.el9.x86_ 11 MB/s | 3.2 MB 00:00 2024-09-10T15:55:00.061 INFO:teuthology.orchestra.run.smithi115.stdout:-------------------------------------------------------------------------------- 2024-09-10T15:55:00.062 INFO:teuthology.orchestra.run.smithi115.stdout:Total 25 MB/s | 62 MB 00:02 2024-09-10T15:55:00.175 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction check 2024-09-10T15:55:00.211 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction check succeeded. 2024-09-10T15:55:00.211 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction test 2024-09-10T15:55:00.934 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction test succeeded. 2024-09-10T15:55:00.934 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction 2024-09-10T15:55:02.452 INFO:teuthology.orchestra.run.smithi115.stdout: Preparing : 1/1 2024-09-10T15:55:02.525 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/35 2024-09-10T15:55:02.553 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/35 2024-09-10T15:55:02.587 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/35 2024-09-10T15:55:02.618 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 4/35 2024-09-10T15:55:02.640 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 4/35 2024-09-10T15:55:02.829 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 5/35 2024-09-10T15:55:02.890 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 5/35 2024-09-10T15:55:03.257 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/35 2024-09-10T15:55:03.355 INFO:teuthology.orchestra.run.smithi115.stdout: Upgrading : librados2-2:19.3.0-4858.gbad1f558.el9.x86_64 7/35 2024-09-10T15:55:03.448 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: librados2-2:19.3.0-4858.gbad1f558.el9.x86_64 7/35 2024-09-10T15:55:03.537 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-rados-2:19.3.0-4858.gbad1f558.el9.x86_64 8/35 2024-09-10T15:55:03.565 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : libcephfs2-2:19.3.0-4858.gbad1f558.el9.x86_64 9/35 2024-09-10T15:55:03.628 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: libcephfs2-2:19.3.0-4858.gbad1f558.el9.x86_64 9/35 2024-09-10T15:55:03.665 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : liboath-2.6.7-2.el9.x86_64 10/35 2024-09-10T15:55:03.709 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-ceph-argparse-2:19.3.0-4858.gbad1f558.el9. 11/35 2024-09-10T15:55:03.779 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-cephfs-2:19.3.0-4858.gbad1f558.el9.x86_64 12/35 2024-09-10T15:55:03.801 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : libradosstriper1-2:19.3.0-4858.gbad1f558.el9.x86_6 13/35 2024-09-10T15:55:03.865 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: libradosstriper1-2:19.3.0-4858.gbad1f558.el9.x86_6 13/35 2024-09-10T15:55:03.899 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 14/35 2024-09-10T15:55:04.010 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : re2-1:20211101-20.el9.x86_64 15/35 2024-09-10T15:55:04.042 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 16/35 2024-09-10T15:55:04.092 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 17/35 2024-09-10T15:55:04.141 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 18/35 2024-09-10T15:55:04.170 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 19/35 2024-09-10T15:55:04.211 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 20/35 2024-09-10T15:55:04.256 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 21/35 2024-09-10T15:55:04.577 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : libnbd-1.20.2-2.el9.x86_64 22/35 2024-09-10T15:55:04.625 INFO:teuthology.orchestra.run.smithi115.stdout: Upgrading : librbd1-2:19.3.0-4858.gbad1f558.el9.x86_64 23/35 2024-09-10T15:55:04.715 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: librbd1-2:19.3.0-4858.gbad1f558.el9.x86_64 23/35 2024-09-10T15:55:04.746 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-rbd-2:19.3.0-4858.gbad1f558.el9.x86_64 24/35 2024-09-10T15:55:04.782 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 25/35 2024-09-10T15:55:05.274 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 26/35 2024-09-10T15:55:05.310 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : librgw2-2:19.3.0-4858.gbad1f558.el9.x86_64 27/35 2024-09-10T15:55:05.382 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: librgw2-2:19.3.0-4858.gbad1f558.el9.x86_64 27/35 2024-09-10T15:55:05.451 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-rgw-2:19.3.0-4858.gbad1f558.el9.x86_64 28/35 2024-09-10T15:55:05.488 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-ceph-common-2:19.3.0-4858.gbad1f558.el9.x8 29/35 2024-09-10T15:55:07.851 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-common-2:19.3.0-4858.gbad1f558.el9.x86_64 30/35 2024-09-10T15:55:07.884 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-common-2:19.3.0-4858.gbad1f558.el9.x86_64 30/35 2024-09-10T15:55:08.449 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-common-2:19.3.0-4858.gbad1f558.el9.x86_64 30/35 2024-09-10T15:55:08.496 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-base-2:19.3.0-4858.gbad1f558.el9.x86_64 31/35 2024-09-10T15:55:08.570 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-base-2:19.3.0-4858.gbad1f558.el9.x86_64 31/35 2024-09-10T15:55:08.570 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-10T15:55:08.570 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-10T15:55:08.570 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T15:55:08.630 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-selinux-2:19.3.0-4858.gbad1f558.el9.x86_64 32/35 2024-09-10T15:55:18.875 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-selinux-2:19.3.0-4858.gbad1f558.el9.x86_64 32/35 2024-09-10T15:55:18.875 INFO:teuthology.orchestra.run.smithi115.stdout:skipping the directory /sys 2024-09-10T15:55:18.876 INFO:teuthology.orchestra.run.smithi115.stdout:skipping the directory /proc 2024-09-10T15:55:18.876 INFO:teuthology.orchestra.run.smithi115.stdout:skipping the directory /mnt 2024-09-10T15:55:18.876 INFO:teuthology.orchestra.run.smithi115.stdout:skipping the directory /var/tmp 2024-09-10T15:55:18.876 INFO:teuthology.orchestra.run.smithi115.stdout:skipping the directory /home 2024-09-10T15:55:18.876 INFO:teuthology.orchestra.run.smithi115.stdout:skipping the directory /root 2024-09-10T15:55:18.876 INFO:teuthology.orchestra.run.smithi115.stdout:skipping the directory /tmp 2024-09-10T15:55:18.876 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T15:55:19.890 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-radosgw-2:19.3.0-4858.gbad1f558.el9.x86_64 33/35 2024-09-10T15:55:19.916 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-radosgw-2:19.3.0-4858.gbad1f558.el9.x86_64 33/35 2024-09-10T15:55:19.916 INFO:teuthology.orchestra.run.smithi115.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T15:55:19.916 INFO:teuthology.orchestra.run.smithi115.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-10T15:55:19.916 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-10T15:55:19.916 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-10T15:55:19.916 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T15:55:19.918 INFO:teuthology.orchestra.run.smithi115.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 34/35 2024-09-10T15:55:20.009 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 34/35 2024-09-10T15:55:20.009 INFO:teuthology.orchestra.run.smithi115.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 35/35 2024-09-10T15:55:21.129 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 35/35 2024-09-10T15:55:21.129 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-base-2:19.3.0-4858.gbad1f558.el9.x86_64 1/35 2024-09-10T15:55:21.129 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-common-2:19.3.0-4858.gbad1f558.el9.x86_64 2/35 2024-09-10T15:55:21.130 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-radosgw-2:19.3.0-4858.gbad1f558.el9.x86_64 3/35 2024-09-10T15:55:21.130 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-selinux-2:19.3.0-4858.gbad1f558.el9.x86_64 4/35 2024-09-10T15:55:21.130 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libcephfs2-2:19.3.0-4858.gbad1f558.el9.x86_64 5/35 2024-09-10T15:55:21.130 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libradosstriper1-2:19.3.0-4858.gbad1f558.el9.x86_6 6/35 2024-09-10T15:55:21.130 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : librgw2-2:19.3.0-4858.gbad1f558.el9.x86_64 7/35 2024-09-10T15:55:21.130 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-ceph-argparse-2:19.3.0-4858.gbad1f558.el9. 8/35 2024-09-10T15:55:21.130 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-ceph-common-2:19.3.0-4858.gbad1f558.el9.x8 9/35 2024-09-10T15:55:21.130 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-cephfs-2:19.3.0-4858.gbad1f558.el9.x86_64 10/35 2024-09-10T15:55:21.131 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-rados-2:19.3.0-4858.gbad1f558.el9.x86_64 11/35 2024-09-10T15:55:21.131 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-rbd-2:19.3.0-4858.gbad1f558.el9.x86_64 12/35 2024-09-10T15:55:21.131 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-rgw-2:19.3.0-4858.gbad1f558.el9.x86_64 13/35 2024-09-10T15:55:21.131 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/35 2024-09-10T15:55:21.131 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libnbd-1.20.2-2.el9.x86_64 15/35 2024-09-10T15:55:21.131 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 16/35 2024-09-10T15:55:21.131 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 17/35 2024-09-10T15:55:21.131 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 18/35 2024-09-10T15:55:21.131 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 19/35 2024-09-10T15:55:21.131 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 20/35 2024-09-10T15:55:21.131 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 21/35 2024-09-10T15:55:21.132 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 22/35 2024-09-10T15:55:21.132 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 23/35 2024-09-10T15:55:21.132 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 24/35 2024-09-10T15:55:21.132 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 25/35 2024-09-10T15:55:21.132 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 26/35 2024-09-10T15:55:21.132 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 27/35 2024-09-10T15:55:21.132 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 28/35 2024-09-10T15:55:21.132 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 29/35 2024-09-10T15:55:21.132 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : re2-1:20211101-20.el9.x86_64 30/35 2024-09-10T15:55:21.133 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 31/35 2024-09-10T15:55:21.134 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : librados2-2:19.3.0-4858.gbad1f558.el9.x86_64 32/35 2024-09-10T15:55:21.134 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 33/35 2024-09-10T15:55:21.134 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : librbd1-2:19.3.0-4858.gbad1f558.el9.x86_64 34/35 2024-09-10T15:55:22.138 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 35/35 2024-09-10T15:55:22.138 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T15:55:22.138 INFO:teuthology.orchestra.run.smithi115.stdout:Upgraded: 2024-09-10T15:55:22.138 INFO:teuthology.orchestra.run.smithi115.stdout: librados2-2:19.3.0-4858.gbad1f558.el9.x86_64 2024-09-10T15:55:22.138 INFO:teuthology.orchestra.run.smithi115.stdout: librbd1-2:19.3.0-4858.gbad1f558.el9.x86_64 2024-09-10T15:55:22.138 INFO:teuthology.orchestra.run.smithi115.stdout:Installed: 2024-09-10T15:55:22.138 INFO:teuthology.orchestra.run.smithi115.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-10T15:55:22.139 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-base-2:19.3.0-4858.gbad1f558.el9.x86_64 2024-09-10T15:55:22.139 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-common-2:19.3.0-4858.gbad1f558.el9.x86_64 2024-09-10T15:55:22.139 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-radosgw-2:19.3.0-4858.gbad1f558.el9.x86_64 2024-09-10T15:55:22.139 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-selinux-2:19.3.0-4858.gbad1f558.el9.x86_64 2024-09-10T15:55:22.139 INFO:teuthology.orchestra.run.smithi115.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-09-10T15:55:22.139 INFO:teuthology.orchestra.run.smithi115.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-10T15:55:22.139 INFO:teuthology.orchestra.run.smithi115.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-10T15:55:22.139 INFO:teuthology.orchestra.run.smithi115.stdout: libcephfs2-2:19.3.0-4858.gbad1f558.el9.x86_64 2024-09-10T15:55:22.139 INFO:teuthology.orchestra.run.smithi115.stdout: libnbd-1.20.2-2.el9.x86_64 2024-09-10T15:55:22.139 INFO:teuthology.orchestra.run.smithi115.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-10T15:55:22.139 INFO:teuthology.orchestra.run.smithi115.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-10T15:55:22.139 INFO:teuthology.orchestra.run.smithi115.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-10T15:55:22.139 INFO:teuthology.orchestra.run.smithi115.stdout: libradosstriper1-2:19.3.0-4858.gbad1f558.el9.x86_64 2024-09-10T15:55:22.140 INFO:teuthology.orchestra.run.smithi115.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-10T15:55:22.140 INFO:teuthology.orchestra.run.smithi115.stdout: librgw2-2:19.3.0-4858.gbad1f558.el9.x86_64 2024-09-10T15:55:22.140 INFO:teuthology.orchestra.run.smithi115.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-10T15:55:22.140 INFO:teuthology.orchestra.run.smithi115.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-10T15:55:22.140 INFO:teuthology.orchestra.run.smithi115.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-10T15:55:22.140 INFO:teuthology.orchestra.run.smithi115.stdout: python3-ceph-argparse-2:19.3.0-4858.gbad1f558.el9.x86_64 2024-09-10T15:55:22.140 INFO:teuthology.orchestra.run.smithi115.stdout: python3-ceph-common-2:19.3.0-4858.gbad1f558.el9.x86_64 2024-09-10T15:55:22.140 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cephfs-2:19.3.0-4858.gbad1f558.el9.x86_64 2024-09-10T15:55:22.140 INFO:teuthology.orchestra.run.smithi115.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-10T15:55:22.140 INFO:teuthology.orchestra.run.smithi115.stdout: python3-rados-2:19.3.0-4858.gbad1f558.el9.x86_64 2024-09-10T15:55:22.140 INFO:teuthology.orchestra.run.smithi115.stdout: python3-rbd-2:19.3.0-4858.gbad1f558.el9.x86_64 2024-09-10T15:55:22.140 INFO:teuthology.orchestra.run.smithi115.stdout: python3-rgw-2:19.3.0-4858.gbad1f558.el9.x86_64 2024-09-10T15:55:22.140 INFO:teuthology.orchestra.run.smithi115.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-10T15:55:22.140 INFO:teuthology.orchestra.run.smithi115.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-10T15:55:22.141 INFO:teuthology.orchestra.run.smithi115.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-10T15:55:22.141 INFO:teuthology.orchestra.run.smithi115.stdout: re2-1:20211101-20.el9.x86_64 2024-09-10T15:55:22.141 INFO:teuthology.orchestra.run.smithi115.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-10T15:55:22.141 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T15:55:22.141 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2024-09-10T15:55:22.491 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum -y install ceph-test 2024-09-10T15:55:23.053 INFO:teuthology.orchestra.run.smithi115.stdout:Last metadata expiration check: 0:00:28 ago on Tue 10 Sep 2024 03:54:55 PM UTC. 2024-09-10T15:55:23.184 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2024-09-10T15:55:23.185 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-10T15:55:23.185 INFO:teuthology.orchestra.run.smithi115.stdout: Package Arch Version Repository Size 2024-09-10T15:55:23.185 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-10T15:55:23.185 INFO:teuthology.orchestra.run.smithi115.stdout:Installing: 2024-09-10T15:55:23.185 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-test x86_64 2:19.3.0-4858.gbad1f558.el9 ceph 50 M 2024-09-10T15:55:23.185 INFO:teuthology.orchestra.run.smithi115.stdout:Installing dependencies: 2024-09-10T15:55:23.186 INFO:teuthology.orchestra.run.smithi115.stdout: libcephsqlite x86_64 2:19.3.0-4858.gbad1f558.el9 ceph 161 k 2024-09-10T15:55:23.186 INFO:teuthology.orchestra.run.smithi115.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-10T15:55:23.186 INFO:teuthology.orchestra.run.smithi115.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-10T15:55:23.186 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T15:55:23.186 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction Summary 2024-09-10T15:55:23.186 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-10T15:55:23.186 INFO:teuthology.orchestra.run.smithi115.stdout:Install 4 Packages 2024-09-10T15:55:23.186 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T15:55:23.187 INFO:teuthology.orchestra.run.smithi115.stdout:Total download size: 50 M 2024-09-10T15:55:23.187 INFO:teuthology.orchestra.run.smithi115.stdout:Installed size: 215 M 2024-09-10T15:55:23.187 INFO:teuthology.orchestra.run.smithi115.stdout:Downloading Packages: 2024-09-10T15:55:23.356 INFO:teuthology.orchestra.run.smithi115.stdout:(1/4): socat-1.7.4.1-6.el9.x86_64.rpm 2.0 MB/s | 304 kB 00:00 2024-09-10T15:55:23.423 INFO:teuthology.orchestra.run.smithi115.stdout:(2/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 956 kB/s | 64 kB 00:00 2024-09-10T15:55:23.506 INFO:teuthology.orchestra.run.smithi115.stdout:(3/4): libcephsqlite-19.3.0-4858.gbad1f558.el9. 545 kB/s | 161 kB 00:00 2024-09-10T15:55:25.948 INFO:teuthology.orchestra.run.smithi115.stdout:(4/4): ceph-test-19.3.0-4858.gbad1f558.el9.x86_ 18 MB/s | 50 MB 00:02 2024-09-10T15:55:25.950 INFO:teuthology.orchestra.run.smithi115.stdout:-------------------------------------------------------------------------------- 2024-09-10T15:55:25.951 INFO:teuthology.orchestra.run.smithi115.stdout:Total 18 MB/s | 50 MB 00:02 2024-09-10T15:55:25.979 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction check 2024-09-10T15:55:25.999 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction check succeeded. 2024-09-10T15:55:25.999 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction test 2024-09-10T15:55:26.375 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction test succeeded. 2024-09-10T15:55:26.375 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction 2024-09-10T15:55:26.910 INFO:teuthology.orchestra.run.smithi115.stdout: Preparing : 1/1 2024-09-10T15:55:26.991 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-09-10T15:55:27.030 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 2/4 2024-09-10T15:55:27.067 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : libcephsqlite-2:19.3.0-4858.gbad1f558.el9.x86_64 3/4 2024-09-10T15:55:31.915 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: libcephsqlite-2:19.3.0-4858.gbad1f558.el9.x86_64 3/4 2024-09-10T15:55:32.006 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-test-2:19.3.0-4858.gbad1f558.el9.x86_64 4/4 2024-09-10T15:55:34.513 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-test-2:19.3.0-4858.gbad1f558.el9.x86_64 4/4 2024-09-10T15:55:34.513 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-test-2:19.3.0-4858.gbad1f558.el9.x86_64 1/4 2024-09-10T15:55:34.513 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libcephsqlite-2:19.3.0-4858.gbad1f558.el9.x86_64 2/4 2024-09-10T15:55:34.513 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 3/4 2024-09-10T15:55:34.830 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-09-10T15:55:34.830 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T15:55:34.830 INFO:teuthology.orchestra.run.smithi115.stdout:Installed: 2024-09-10T15:55:34.830 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-test-2:19.3.0-4858.gbad1f558.el9.x86_64 2024-09-10T15:55:34.830 INFO:teuthology.orchestra.run.smithi115.stdout: libcephsqlite-2:19.3.0-4858.gbad1f558.el9.x86_64 2024-09-10T15:55:34.830 INFO:teuthology.orchestra.run.smithi115.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-10T15:55:34.830 INFO:teuthology.orchestra.run.smithi115.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-10T15:55:34.831 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T15:55:34.831 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2024-09-10T15:55:35.140 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum -y install ceph 2024-09-10T15:55:35.710 INFO:teuthology.orchestra.run.smithi115.stdout:Last metadata expiration check: 0:00:40 ago on Tue 10 Sep 2024 03:54:55 PM UTC. 2024-09-10T15:55:35.846 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2024-09-10T15:55:35.848 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-10T15:55:35.848 INFO:teuthology.orchestra.run.smithi115.stdout: Package Arch Version Repository Size 2024-09-10T15:55:35.848 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-10T15:55:35.848 INFO:teuthology.orchestra.run.smithi115.stdout:Installing: 2024-09-10T15:55:35.849 INFO:teuthology.orchestra.run.smithi115.stdout: ceph x86_64 2:19.3.0-4858.gbad1f558.el9 ceph 6.6 k 2024-09-10T15:55:35.849 INFO:teuthology.orchestra.run.smithi115.stdout:Installing dependencies: 2024-09-10T15:55:35.849 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mds x86_64 2:19.3.0-4858.gbad1f558.el9 ceph 2.4 M 2024-09-10T15:55:35.849 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mgr x86_64 2:19.3.0-4858.gbad1f558.el9 ceph 1.5 M 2024-09-10T15:55:35.849 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mgr-modules-core noarch 2:19.3.0-4858.gbad1f558.el9 ceph-noarch 291 k 2024-09-10T15:55:35.849 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mon x86_64 2:19.3.0-4858.gbad1f558.el9 ceph 4.8 M 2024-09-10T15:55:35.849 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-osd x86_64 2:19.3.0-4858.gbad1f558.el9 ceph 17 M 2024-09-10T15:55:35.849 INFO:teuthology.orchestra.run.smithi115.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-10T15:55:35.849 INFO:teuthology.orchestra.run.smithi115.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-10T15:55:35.849 INFO:teuthology.orchestra.run.smithi115.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-10T15:55:35.849 INFO:teuthology.orchestra.run.smithi115.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-10T15:55:35.849 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-10T15:55:35.849 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-10T15:55:35.850 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-10T15:55:35.850 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-10T15:55:35.850 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-10T15:55:35.850 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-10T15:55:35.850 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jaraco-collections 2024-09-10T15:55:35.850 INFO:teuthology.orchestra.run.smithi115.stdout: noarch 3.0.0-8.el9 epel 23 k 2024-09-10T15:55:35.850 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-10T15:55:35.850 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-10T15:55:35.850 INFO:teuthology.orchestra.run.smithi115.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-10T15:55:35.850 INFO:teuthology.orchestra.run.smithi115.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-10T15:55:35.850 INFO:teuthology.orchestra.run.smithi115.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-10T15:55:35.850 INFO:teuthology.orchestra.run.smithi115.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-10T15:55:35.850 INFO:teuthology.orchestra.run.smithi115.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-10T15:55:35.851 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-10T15:55:35.851 INFO:teuthology.orchestra.run.smithi115.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-10T15:55:35.851 INFO:teuthology.orchestra.run.smithi115.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-10T15:55:35.851 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-10T15:55:35.851 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-10T15:55:35.851 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-10T15:55:35.851 INFO:teuthology.orchestra.run.smithi115.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-10T15:55:35.851 INFO:teuthology.orchestra.run.smithi115.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-10T15:55:35.851 INFO:teuthology.orchestra.run.smithi115.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-10T15:55:35.851 INFO:teuthology.orchestra.run.smithi115.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-10T15:55:35.851 INFO:teuthology.orchestra.run.smithi115.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-10T15:55:35.851 INFO:teuthology.orchestra.run.smithi115.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-10T15:55:35.851 INFO:teuthology.orchestra.run.smithi115.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-10T15:55:35.851 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T15:55:35.852 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction Summary 2024-09-10T15:55:35.852 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-10T15:55:35.852 INFO:teuthology.orchestra.run.smithi115.stdout:Install 37 Packages 2024-09-10T15:55:35.852 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T15:55:35.852 INFO:teuthology.orchestra.run.smithi115.stdout:Total download size: 31 M 2024-09-10T15:55:35.852 INFO:teuthology.orchestra.run.smithi115.stdout:Installed size: 109 M 2024-09-10T15:55:35.853 INFO:teuthology.orchestra.run.smithi115.stdout:Downloading Packages: 2024-09-10T15:55:36.691 INFO:teuthology.orchestra.run.smithi115.stdout:(1/37): ceph-19.3.0-4858.gbad1f558.el9.x86_64.r 54 kB/s | 6.6 kB 00:00 2024-09-10T15:55:36.883 INFO:teuthology.orchestra.run.smithi115.stdout:(2/37): ceph-mgr-19.3.0-4858.gbad1f558.el9.x86_ 4.9 MB/s | 1.5 MB 00:00 2024-09-10T15:55:36.933 INFO:teuthology.orchestra.run.smithi115.stdout:(3/37): ceph-mds-19.3.0-4858.gbad1f558.el9.x86_ 6.6 MB/s | 2.4 MB 00:00 2024-09-10T15:55:37.000 INFO:teuthology.orchestra.run.smithi115.stdout:(4/37): ceph-mgr-modules-core-19.3.0-4858.gbad1 4.3 MB/s | 291 kB 00:00 2024-09-10T15:55:37.100 INFO:teuthology.orchestra.run.smithi115.stdout:(5/37): ceph-mon-19.3.0-4858.gbad1f558.el9.x86_ 12 MB/s | 4.8 MB 00:00 2024-09-10T15:55:37.300 INFO:teuthology.orchestra.run.smithi115.stdout:(6/37): python3-cffi-1.14.5-5.el9.x86_64.rpm 844 kB/s | 253 kB 00:00 2024-09-10T15:55:37.359 INFO:teuthology.orchestra.run.smithi115.stdout:(7/37): python3-ply-3.11-14.el9.noarch.rpm 1.8 MB/s | 106 kB 00:00 2024-09-10T15:55:37.526 INFO:teuthology.orchestra.run.smithi115.stdout:(8/37): ceph-osd-19.3.0-4858.gbad1f558.el9.x86_ 27 MB/s | 17 MB 00:00 2024-09-10T15:55:37.551 INFO:teuthology.orchestra.run.smithi115.stdout:(9/37): python3-pycparser-2.20-6.el9.noarch.rpm 705 kB/s | 135 kB 00:00 2024-09-10T15:55:37.585 INFO:teuthology.orchestra.run.smithi115.stdout:(10/37): python3-cryptography-36.0.1-4.el9.x86_ 2.5 MB/s | 1.2 MB 00:00 2024-09-10T15:55:37.610 INFO:teuthology.orchestra.run.smithi115.stdout:(11/37): python3-urllib3-1.26.5-6.el9.noarch.rp 3.6 MB/s | 215 kB 00:00 2024-09-10T15:55:37.643 INFO:teuthology.orchestra.run.smithi115.stdout:(12/37): lua-5.4.4-4.el9.x86_64.rpm 3.1 MB/s | 188 kB 00:00 2024-09-10T15:55:37.669 INFO:teuthology.orchestra.run.smithi115.stdout:(13/37): python3-mako-1.1.4-6.el9.noarch.rpm 2.9 MB/s | 172 kB 00:00 2024-09-10T15:55:37.694 INFO:teuthology.orchestra.run.smithi115.stdout:(14/37): python3-markupsafe-1.1.1-12.el9.x86_64 690 kB/s | 35 kB 00:00 2024-09-10T15:55:37.728 INFO:teuthology.orchestra.run.smithi115.stdout:(15/37): python3-packaging-20.9-5.el9.noarch.rp 1.3 MB/s | 77 kB 00:00 2024-09-10T15:55:37.753 INFO:teuthology.orchestra.run.smithi115.stdout:(16/37): python3-requests-2.25.1-8.el9.noarch.r 550 kB/s | 125 kB 00:00 2024-09-10T15:55:37.778 INFO:teuthology.orchestra.run.smithi115.stdout:(17/37): python3-pytz-2021.1-5.el9.noarch.rpm 607 kB/s | 51 kB 00:00 2024-09-10T15:55:37.804 INFO:teuthology.orchestra.run.smithi115.stdout:(18/37): python3-toml-0.10.2-6.el9.noarch.rpm 551 kB/s | 42 kB 00:00 2024-09-10T15:55:37.846 INFO:teuthology.orchestra.run.smithi115.stdout:(19/37): luarocks-3.9.2-5.el9.noarch.rpm 2.2 MB/s | 151 kB 00:00 2024-09-10T15:55:37.871 INFO:teuthology.orchestra.run.smithi115.stdout:(20/37): lua-devel-5.4.4-4.el9.x86_64.rpm 190 kB/s | 22 kB 00:00 2024-09-10T15:55:37.888 INFO:teuthology.orchestra.run.smithi115.stdout:(21/37): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 519 kB/s | 43 kB 00:00 2024-09-10T15:55:37.922 INFO:teuthology.orchestra.run.smithi115.stdout:(22/37): python3-cheroot-8.6.0-4.el9.noarch.rpm 2.3 MB/s | 172 kB 00:00 2024-09-10T15:55:37.939 INFO:teuthology.orchestra.run.smithi115.stdout:(23/37): python3-jaraco-8.2.1-3.el9.noarch.rpm 250 kB/s | 11 kB 00:00 2024-09-10T15:55:37.955 INFO:teuthology.orchestra.run.smithi115.stdout:(24/37): python3-jaraco-classes-3.2.1-5.el9.noa 556 kB/s | 18 kB 00:00 2024-09-10T15:55:37.972 INFO:teuthology.orchestra.run.smithi115.stdout:(25/37): python3-jaraco-collections-3.0.0-8.el9 697 kB/s | 23 kB 00:00 2024-09-10T15:55:37.989 INFO:teuthology.orchestra.run.smithi115.stdout:(26/37): python3-jaraco-functools-3.5.0-2.el9.n 579 kB/s | 19 kB 00:00 2024-09-10T15:55:38.006 INFO:teuthology.orchestra.run.smithi115.stdout:(27/37): python3-jaraco-text-3.2.0-6.el9.noarch 586 kB/s | 20 kB 00:00 2024-09-10T15:55:38.031 INFO:teuthology.orchestra.run.smithi115.stdout:(28/37): python3-logutils-0.3.5-21.el9.noarch.r 1.1 MB/s | 46 kB 00:00 2024-09-10T15:55:38.057 INFO:teuthology.orchestra.run.smithi115.stdout:(29/37): python3-more-itertools-8.12.0-2.el9.no 1.5 MB/s | 79 kB 00:00 2024-09-10T15:55:38.082 INFO:teuthology.orchestra.run.smithi115.stdout:(30/37): python3-cherrypy-18.6.1-2.el9.noarch.r 1.7 MB/s | 358 kB 00:00 2024-09-10T15:55:38.108 INFO:teuthology.orchestra.run.smithi115.stdout:(31/37): python3-portend-3.1.0-2.el9.noarch.rpm 327 kB/s | 16 kB 00:00 2024-09-10T15:55:38.133 INFO:teuthology.orchestra.run.smithi115.stdout:(32/37): python3-pyOpenSSL-21.0.0-1.el9.noarch. 1.8 MB/s | 90 kB 00:00 2024-09-10T15:55:38.158 INFO:teuthology.orchestra.run.smithi115.stdout:(33/37): python3-tempora-5.0.0-2.el9.noarch.rpm 715 kB/s | 36 kB 00:00 2024-09-10T15:55:38.192 INFO:teuthology.orchestra.run.smithi115.stdout:(34/37): python3-pecan-1.4.2-3.el9.noarch.rpm 1.7 MB/s | 272 kB 00:00 2024-09-10T15:55:38.217 INFO:teuthology.orchestra.run.smithi115.stdout:(35/37): python3-zc-lockfile-2.0-10.el9.noarch. 797 kB/s | 20 kB 00:00 2024-09-10T15:55:38.242 INFO:teuthology.orchestra.run.smithi115.stdout:(36/37): python3-webob-1.8.8-2.el9.noarch.rpm 2.7 MB/s | 230 kB 00:00 2024-09-10T15:55:38.276 INFO:teuthology.orchestra.run.smithi115.stdout:(37/37): python3-werkzeug-2.0.3-3.el9.1.noarch. 3.6 MB/s | 427 kB 00:00 2024-09-10T15:55:38.281 INFO:teuthology.orchestra.run.smithi115.stdout:-------------------------------------------------------------------------------- 2024-09-10T15:55:38.281 INFO:teuthology.orchestra.run.smithi115.stdout:Total 13 MB/s | 31 MB 00:02 2024-09-10T15:55:38.436 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction check 2024-09-10T15:55:38.475 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction check succeeded. 2024-09-10T15:55:38.475 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction test 2024-09-10T15:55:38.923 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction test succeeded. 2024-09-10T15:55:38.923 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction 2024-09-10T15:55:40.320 INFO:teuthology.orchestra.run.smithi115.stdout: Preparing : 1/1 2024-09-10T15:55:40.378 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/37 2024-09-10T15:55:40.411 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/37 2024-09-10T15:55:40.439 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : lua-5.4.4-4.el9.x86_64 3/37 2024-09-10T15:55:40.511 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 4/37 2024-09-10T15:55:40.572 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : luarocks-3.9.2-5.el9.noarch 5/37 2024-09-10T15:55:40.604 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 6/37 2024-09-10T15:55:40.692 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 7/37 2024-09-10T15:55:40.779 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 8/37 2024-09-10T15:55:40.827 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 9/37 2024-09-10T15:55:40.862 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 10/37 2024-09-10T15:55:40.892 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 11/37 2024-09-10T15:55:40.927 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 12/37 2024-09-10T15:55:40.968 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 13/37 2024-09-10T15:55:41.018 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 14/37 2024-09-10T15:55:41.061 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 15/37 2024-09-10T15:55:41.095 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 16/37 2024-09-10T15:55:41.144 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 17/37 2024-09-10T15:55:41.184 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-packaging-20.9-5.el9.noarch 18/37 2024-09-10T15:55:41.266 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 19/37 2024-09-10T15:55:41.481 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 20/37 2024-09-10T15:55:41.559 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 21/37 2024-09-10T15:55:41.612 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 22/37 2024-09-10T15:55:41.651 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 23/37 2024-09-10T15:55:41.708 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-ply-3.11-14.el9.noarch 24/37 2024-09-10T15:55:41.772 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 25/37 2024-09-10T15:55:41.969 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 26/37 2024-09-10T15:55:42.013 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 27/37 2024-09-10T15:55:42.091 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 28/37 2024-09-10T15:55:42.180 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 29/37 2024-09-10T15:55:42.233 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 30/37 2024-09-10T15:55:42.360 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 31/37 2024-09-10T15:55:42.533 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-mgr-modules-core-2:19.3.0-4858.gbad1f558.el9. 32/37 2024-09-10T15:55:42.558 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-mgr-2:19.3.0-4858.gbad1f558.el9.x86_64 33/37 2024-09-10T15:55:42.585 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-mgr-2:19.3.0-4858.gbad1f558.el9.x86_64 33/37 2024-09-10T15:55:42.585 INFO:teuthology.orchestra.run.smithi115.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T15:55:42.585 INFO:teuthology.orchestra.run.smithi115.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-10T15:55:42.585 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-10T15:55:42.586 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-10T15:55:42.586 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T15:55:44.059 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-osd-2:19.3.0-4858.gbad1f558.el9.x86_64 34/37 2024-09-10T15:55:44.087 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-osd-2:19.3.0-4858.gbad1f558.el9.x86_64 34/37 2024-09-10T15:55:44.087 INFO:teuthology.orchestra.run.smithi115.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T15:55:44.087 INFO:teuthology.orchestra.run.smithi115.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-10T15:55:44.087 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-10T15:55:44.087 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-10T15:55:44.087 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T15:55:44.543 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-mon-2:19.3.0-4858.gbad1f558.el9.x86_64 35/37 2024-09-10T15:55:44.567 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-mon-2:19.3.0-4858.gbad1f558.el9.x86_64 35/37 2024-09-10T15:55:44.567 INFO:teuthology.orchestra.run.smithi115.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T15:55:44.567 INFO:teuthology.orchestra.run.smithi115.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-10T15:55:44.567 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-10T15:55:44.567 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-10T15:55:44.568 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T15:55:44.794 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-mds-2:19.3.0-4858.gbad1f558.el9.x86_64 36/37 2024-09-10T15:55:44.817 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-mds-2:19.3.0-4858.gbad1f558.el9.x86_64 36/37 2024-09-10T15:55:44.818 INFO:teuthology.orchestra.run.smithi115.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T15:55:44.818 INFO:teuthology.orchestra.run.smithi115.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-10T15:55:44.818 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-10T15:55:44.818 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-10T15:55:44.818 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T15:55:44.882 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-2:19.3.0-4858.gbad1f558.el9.x86_64 37/37 2024-09-10T15:55:46.685 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-2:19.3.0-4858.gbad1f558.el9.x86_64 37/37 2024-09-10T15:55:46.685 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-2:19.3.0-4858.gbad1f558.el9.x86_64 1/37 2024-09-10T15:55:46.685 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-mds-2:19.3.0-4858.gbad1f558.el9.x86_64 2/37 2024-09-10T15:55:46.686 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-mgr-2:19.3.0-4858.gbad1f558.el9.x86_64 3/37 2024-09-10T15:55:46.686 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-mon-2:19.3.0-4858.gbad1f558.el9.x86_64 4/37 2024-09-10T15:55:46.686 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-osd-2:19.3.0-4858.gbad1f558.el9.x86_64 5/37 2024-09-10T15:55:46.686 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-mgr-modules-core-2:19.3.0-4858.gbad1f558.el9. 6/37 2024-09-10T15:55:46.686 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/37 2024-09-10T15:55:46.686 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/37 2024-09-10T15:55:46.686 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/37 2024-09-10T15:55:46.686 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/37 2024-09-10T15:55:46.686 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/37 2024-09-10T15:55:46.686 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 12/37 2024-09-10T15:55:46.686 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : lua-5.4.4-4.el9.x86_64 13/37 2024-09-10T15:55:46.686 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 14/37 2024-09-10T15:55:46.687 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 15/37 2024-09-10T15:55:46.687 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 16/37 2024-09-10T15:55:46.687 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 17/37 2024-09-10T15:55:46.687 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 18/37 2024-09-10T15:55:46.687 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 19/37 2024-09-10T15:55:46.687 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 20/37 2024-09-10T15:55:46.687 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 21/37 2024-09-10T15:55:46.687 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 22/37 2024-09-10T15:55:46.687 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 23/37 2024-09-10T15:55:46.687 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 24/37 2024-09-10T15:55:46.687 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 25/37 2024-09-10T15:55:46.687 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 26/37 2024-09-10T15:55:46.687 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 27/37 2024-09-10T15:55:46.688 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 28/37 2024-09-10T15:55:46.688 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 29/37 2024-09-10T15:55:46.688 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 30/37 2024-09-10T15:55:46.688 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 31/37 2024-09-10T15:55:46.688 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 32/37 2024-09-10T15:55:46.688 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 33/37 2024-09-10T15:55:46.688 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 34/37 2024-09-10T15:55:46.688 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 35/37 2024-09-10T15:55:46.688 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 36/37 2024-09-10T15:55:47.688 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 37/37 2024-09-10T15:55:47.689 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T15:55:47.689 INFO:teuthology.orchestra.run.smithi115.stdout:Installed: 2024-09-10T15:55:47.689 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-2:19.3.0-4858.gbad1f558.el9.x86_64 2024-09-10T15:55:47.689 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mds-2:19.3.0-4858.gbad1f558.el9.x86_64 2024-09-10T15:55:47.689 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mgr-2:19.3.0-4858.gbad1f558.el9.x86_64 2024-09-10T15:55:47.689 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mgr-modules-core-2:19.3.0-4858.gbad1f558.el9.noarch 2024-09-10T15:55:47.689 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mon-2:19.3.0-4858.gbad1f558.el9.x86_64 2024-09-10T15:55:47.689 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-osd-2:19.3.0-4858.gbad1f558.el9.x86_64 2024-09-10T15:55:47.689 INFO:teuthology.orchestra.run.smithi115.stdout: lua-5.4.4-4.el9.x86_64 2024-09-10T15:55:47.689 INFO:teuthology.orchestra.run.smithi115.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-10T15:55:47.689 INFO:teuthology.orchestra.run.smithi115.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-10T15:55:47.690 INFO:teuthology.orchestra.run.smithi115.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-10T15:55:47.690 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-10T15:55:47.690 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-10T15:55:47.690 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-10T15:55:47.690 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-10T15:55:47.690 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-10T15:55:47.690 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-10T15:55:47.690 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-10T15:55:47.690 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-10T15:55:47.690 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-10T15:55:47.690 INFO:teuthology.orchestra.run.smithi115.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-10T15:55:47.690 INFO:teuthology.orchestra.run.smithi115.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-10T15:55:47.690 INFO:teuthology.orchestra.run.smithi115.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-10T15:55:47.691 INFO:teuthology.orchestra.run.smithi115.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-10T15:55:47.691 INFO:teuthology.orchestra.run.smithi115.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-10T15:55:47.691 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-10T15:55:47.691 INFO:teuthology.orchestra.run.smithi115.stdout: python3-ply-3.11-14.el9.noarch 2024-09-10T15:55:47.691 INFO:teuthology.orchestra.run.smithi115.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-10T15:55:47.691 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-10T15:55:47.691 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-10T15:55:47.691 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-10T15:55:47.691 INFO:teuthology.orchestra.run.smithi115.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-10T15:55:47.691 INFO:teuthology.orchestra.run.smithi115.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-10T15:55:47.691 INFO:teuthology.orchestra.run.smithi115.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-10T15:55:47.691 INFO:teuthology.orchestra.run.smithi115.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-10T15:55:47.691 INFO:teuthology.orchestra.run.smithi115.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-10T15:55:47.692 INFO:teuthology.orchestra.run.smithi115.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-10T15:55:47.692 INFO:teuthology.orchestra.run.smithi115.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-10T15:55:47.692 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T15:55:47.692 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2024-09-10T15:55:48.036 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum -y install ceph-base 2024-09-10T15:55:48.601 INFO:teuthology.orchestra.run.smithi115.stdout:Last metadata expiration check: 0:00:53 ago on Tue 10 Sep 2024 03:54:55 PM UTC. 2024-09-10T15:55:48.691 INFO:teuthology.orchestra.run.smithi115.stdout:Package ceph-base-2:19.3.0-4858.gbad1f558.el9.x86_64 is already installed. 2024-09-10T15:55:48.734 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2024-09-10T15:55:48.735 INFO:teuthology.orchestra.run.smithi115.stdout:Nothing to do. 2024-09-10T15:55:48.735 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2024-09-10T15:55:48.810 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum -y install cephadm 2024-09-10T15:55:49.361 INFO:teuthology.orchestra.run.smithi115.stdout:Last metadata expiration check: 0:00:54 ago on Tue 10 Sep 2024 03:54:55 PM UTC. 2024-09-10T15:55:49.492 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2024-09-10T15:55:49.493 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-10T15:55:49.493 INFO:teuthology.orchestra.run.smithi115.stdout: Package Arch Version Repository Size 2024-09-10T15:55:49.493 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-10T15:55:49.493 INFO:teuthology.orchestra.run.smithi115.stdout:Installing: 2024-09-10T15:55:49.493 INFO:teuthology.orchestra.run.smithi115.stdout: cephadm noarch 2:19.3.0-4858.gbad1f558.el9 ceph-noarch 784 k 2024-09-10T15:55:49.493 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T15:55:49.493 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction Summary 2024-09-10T15:55:49.493 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-10T15:55:49.493 INFO:teuthology.orchestra.run.smithi115.stdout:Install 1 Package 2024-09-10T15:55:49.493 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T15:55:49.493 INFO:teuthology.orchestra.run.smithi115.stdout:Total download size: 784 k 2024-09-10T15:55:49.494 INFO:teuthology.orchestra.run.smithi115.stdout:Installed size: 790 k 2024-09-10T15:55:49.494 INFO:teuthology.orchestra.run.smithi115.stdout:Downloading Packages: 2024-09-10T15:55:49.767 INFO:teuthology.orchestra.run.smithi115.stdout:cephadm-19.3.0-4858.gbad1f558.el9.noarch.rpm 2.8 MB/s | 784 kB 00:00 2024-09-10T15:55:49.768 INFO:teuthology.orchestra.run.smithi115.stdout:-------------------------------------------------------------------------------- 2024-09-10T15:55:49.768 INFO:teuthology.orchestra.run.smithi115.stdout:Total 2.8 MB/s | 784 kB 00:00 2024-09-10T15:55:49.768 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction check 2024-09-10T15:55:49.773 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction check succeeded. 2024-09-10T15:55:49.774 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction test 2024-09-10T15:55:49.780 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction test succeeded. 2024-09-10T15:55:49.780 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction 2024-09-10T15:55:49.851 INFO:teuthology.orchestra.run.smithi115.stdout: Preparing : 1/1 2024-09-10T15:55:50.296 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: cephadm-2:19.3.0-4858.gbad1f558.el9.noarch 1/1 2024-09-10T15:55:50.406 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : cephadm-2:19.3.0-4858.gbad1f558.el9.noarch 1/1 2024-09-10T15:55:50.754 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: cephadm-2:19.3.0-4858.gbad1f558.el9.noarch 1/1 2024-09-10T15:55:51.011 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : cephadm-2:19.3.0-4858.gbad1f558.el9.noarch 1/1 2024-09-10T15:55:51.011 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T15:55:51.011 INFO:teuthology.orchestra.run.smithi115.stdout:Installed: 2024-09-10T15:55:51.011 INFO:teuthology.orchestra.run.smithi115.stdout: cephadm-2:19.3.0-4858.gbad1f558.el9.noarch 2024-09-10T15:55:51.011 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T15:55:51.011 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2024-09-10T15:55:51.221 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum -y install ceph-immutable-object-cache 2024-09-10T15:55:51.796 INFO:teuthology.orchestra.run.smithi115.stdout:Last metadata expiration check: 0:00:56 ago on Tue 10 Sep 2024 03:54:55 PM UTC. 2024-09-10T15:55:51.932 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2024-09-10T15:55:51.933 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-10T15:55:51.933 INFO:teuthology.orchestra.run.smithi115.stdout: Package Arch Version Repo Size 2024-09-10T15:55:51.933 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-10T15:55:51.933 INFO:teuthology.orchestra.run.smithi115.stdout:Installing: 2024-09-10T15:55:51.933 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-immutable-object-cache x86_64 2:19.3.0-4858.gbad1f558.el9 ceph 145 k 2024-09-10T15:55:51.933 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T15:55:51.934 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction Summary 2024-09-10T15:55:51.934 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-10T15:55:51.934 INFO:teuthology.orchestra.run.smithi115.stdout:Install 1 Package 2024-09-10T15:55:51.934 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T15:55:51.934 INFO:teuthology.orchestra.run.smithi115.stdout:Total download size: 145 k 2024-09-10T15:55:51.934 INFO:teuthology.orchestra.run.smithi115.stdout:Installed size: 432 k 2024-09-10T15:55:51.934 INFO:teuthology.orchestra.run.smithi115.stdout:Downloading Packages: 2024-09-10T15:55:52.124 INFO:teuthology.orchestra.run.smithi115.stdout:ceph-immutable-object-cache-19.3.0-4858.gbad1f5 764 kB/s | 145 kB 00:00 2024-09-10T15:55:52.125 INFO:teuthology.orchestra.run.smithi115.stdout:-------------------------------------------------------------------------------- 2024-09-10T15:55:52.125 INFO:teuthology.orchestra.run.smithi115.stdout:Total 758 kB/s | 145 kB 00:00 2024-09-10T15:55:52.126 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction check 2024-09-10T15:55:52.133 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction check succeeded. 2024-09-10T15:55:52.133 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction test 2024-09-10T15:55:52.191 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction test succeeded. 2024-09-10T15:55:52.191 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction 2024-09-10T15:55:52.340 INFO:teuthology.orchestra.run.smithi115.stdout: Preparing : 1/1 2024-09-10T15:55:52.408 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-immutable-object-cache-2:19.3.0-4858.gbad1f558. 1/1 2024-09-10T15:55:52.432 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.3.0-4858.gbad1f558. 1/1 2024-09-10T15:55:52.433 INFO:teuthology.orchestra.run.smithi115.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T15:55:52.433 INFO:teuthology.orchestra.run.smithi115.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-10T15:55:52.433 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T15:55:53.342 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-immutable-object-cache-2:19.3.0-4858.gbad1f558. 1/1 2024-09-10T15:55:53.342 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T15:55:53.343 INFO:teuthology.orchestra.run.smithi115.stdout:Installed: 2024-09-10T15:55:53.343 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-immutable-object-cache-2:19.3.0-4858.gbad1f558.el9.x86_64 2024-09-10T15:55:53.343 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T15:55:53.343 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2024-09-10T15:55:53.558 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum -y install ceph-mgr 2024-09-10T15:55:54.135 INFO:teuthology.orchestra.run.smithi115.stdout:Last metadata expiration check: 0:00:59 ago on Tue 10 Sep 2024 03:54:55 PM UTC. 2024-09-10T15:55:54.227 INFO:teuthology.orchestra.run.smithi115.stdout:Package ceph-mgr-2:19.3.0-4858.gbad1f558.el9.x86_64 is already installed. 2024-09-10T15:55:54.269 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2024-09-10T15:55:54.269 INFO:teuthology.orchestra.run.smithi115.stdout:Nothing to do. 2024-09-10T15:55:54.269 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2024-09-10T15:55:54.351 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum -y install ceph-mgr-dashboard 2024-09-10T15:55:54.905 INFO:teuthology.orchestra.run.smithi115.stdout:Last metadata expiration check: 0:00:59 ago on Tue 10 Sep 2024 03:54:55 PM UTC. 2024-09-10T15:55:55.037 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2024-09-10T15:55:55.038 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-10T15:55:55.038 INFO:teuthology.orchestra.run.smithi115.stdout: Package Arch Version Repository Size 2024-09-10T15:55:55.038 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-10T15:55:55.038 INFO:teuthology.orchestra.run.smithi115.stdout:Installing: 2024-09-10T15:55:55.038 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mgr-dashboard noarch 2:19.3.0-4858.gbad1f558.el9 ceph-noarch 14 M 2024-09-10T15:55:55.038 INFO:teuthology.orchestra.run.smithi115.stdout:Installing dependencies: 2024-09-10T15:55:55.038 INFO:teuthology.orchestra.run.smithi115.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-10T15:55:55.038 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-grafana-dashboards noarch 2:19.3.0-4858.gbad1f558.el9 ceph-noarch 34 k 2024-09-10T15:55:55.038 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-prometheus-alerts noarch 2:19.3.0-4858.gbad1f558.el9 ceph-noarch 16 k 2024-09-10T15:55:55.038 INFO:teuthology.orchestra.run.smithi115.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-10T15:55:55.039 INFO:teuthology.orchestra.run.smithi115.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-10T15:55:55.039 INFO:teuthology.orchestra.run.smithi115.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-10T15:55:55.039 INFO:teuthology.orchestra.run.smithi115.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-10T15:55:55.039 INFO:teuthology.orchestra.run.smithi115.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-10T15:55:55.039 INFO:teuthology.orchestra.run.smithi115.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-10T15:55:55.039 INFO:teuthology.orchestra.run.smithi115.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-10T15:55:55.039 INFO:teuthology.orchestra.run.smithi115.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-10T15:55:55.039 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T15:55:55.039 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction Summary 2024-09-10T15:55:55.039 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-10T15:55:55.039 INFO:teuthology.orchestra.run.smithi115.stdout:Install 12 Packages 2024-09-10T15:55:55.039 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T15:55:55.040 INFO:teuthology.orchestra.run.smithi115.stdout:Total download size: 19 M 2024-09-10T15:55:55.040 INFO:teuthology.orchestra.run.smithi115.stdout:Installed size: 159 M 2024-09-10T15:55:55.040 INFO:teuthology.orchestra.run.smithi115.stdout:Downloading Packages: 2024-09-10T15:55:55.553 INFO:teuthology.orchestra.run.smithi115.stdout:(1/12): ceph-grafana-dashboards-19.3.0-4858.gba 218 kB/s | 34 kB 00:00 2024-09-10T15:55:55.580 INFO:teuthology.orchestra.run.smithi115.stdout:(2/12): ceph-prometheus-alerts-19.3.0-4858.gbad 89 kB/s | 16 kB 00:00 2024-09-10T15:55:56.063 INFO:teuthology.orchestra.run.smithi115.stdout:(3/12): ceph-mgr-dashboard-19.3.0-4858.gbad1f55 21 MB/s | 14 MB 00:00 2024-09-10T15:55:56.222 INFO:teuthology.orchestra.run.smithi115.stdout:(4/12): protobuf-compiler-3.14.0-13.el9.x86_64. 5.3 MB/s | 863 kB 00:00 2024-09-10T15:55:56.272 INFO:teuthology.orchestra.run.smithi115.stdout:(5/12): abseil-cpp-20211102.0-3.el9.x86_64.rpm 11 MB/s | 548 kB 00:00 2024-09-10T15:55:56.297 INFO:teuthology.orchestra.run.smithi115.stdout:(6/12): grpc-data-1.46.7-10.el9.noarch.rpm 785 kB/s | 19 kB 00:00 2024-09-10T15:55:56.364 INFO:teuthology.orchestra.run.smithi115.stdout:(7/12): python3-grpcio-1.46.7-10.el9.x86_64.rpm 31 MB/s | 2.0 MB 00:00 2024-09-10T15:55:56.390 INFO:teuthology.orchestra.run.smithi115.stdout:(8/12): python3-grpcio-tools-1.46.7-10.el9.x86_ 5.6 MB/s | 144 kB 00:00 2024-09-10T15:55:56.415 INFO:teuthology.orchestra.run.smithi115.stdout:(9/12): python3-repoze-lru-0.7-16.el9.noarch.rp 1.2 MB/s | 31 kB 00:00 2024-09-10T15:55:56.440 INFO:teuthology.orchestra.run.smithi115.stdout:(10/12): python3-routes-2.5.1-5.el9.noarch.rpm 7.4 MB/s | 188 kB 00:00 2024-09-10T15:55:56.574 INFO:teuthology.orchestra.run.smithi115.stdout:(11/12): python3-protobuf-3.14.0-13.el9.noarch. 271 kB/s | 269 kB 00:00 2024-09-10T15:55:57.490 INFO:teuthology.orchestra.run.smithi115.stdout:(12/12): protobuf-3.14.0-13.el9.x86_64.rpm 533 kB/s | 1.0 MB 00:01 2024-09-10T15:55:57.495 INFO:teuthology.orchestra.run.smithi115.stdout:-------------------------------------------------------------------------------- 2024-09-10T15:55:57.495 INFO:teuthology.orchestra.run.smithi115.stdout:Total 7.8 MB/s | 19 MB 00:02 2024-09-10T15:55:57.547 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction check 2024-09-10T15:55:57.567 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction check succeeded. 2024-09-10T15:55:57.567 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction test 2024-09-10T15:55:57.842 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction test succeeded. 2024-09-10T15:55:57.842 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction 2024-09-10T15:55:58.436 INFO:teuthology.orchestra.run.smithi115.stdout: Preparing : 1/1 2024-09-10T15:55:58.508 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 1/12 2024-09-10T15:55:58.551 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 2/12 2024-09-10T15:55:58.597 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/12 2024-09-10T15:55:58.628 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/12 2024-09-10T15:55:58.742 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 5/12 2024-09-10T15:55:58.870 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 6/12 2024-09-10T15:55:58.965 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 7/12 2024-09-10T15:55:59.016 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 8/12 2024-09-10T15:55:59.044 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 9/12 2024-09-10T15:55:59.077 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-prometheus-alerts-2:19.3.0-4858.gbad1f558.el9 10/12 2024-09-10T15:56:02.238 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-grafana-dashboards-2:19.3.0-4858.gbad1f558.el 11/12 2024-09-10T15:56:02.290 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-mgr-dashboard-2:19.3.0-4858.gbad1f558.el9.noa 12/12 2024-09-10T15:56:04.290 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.3.0-4858.gbad1f558.el9.noa 12/12 2024-09-10T15:56:04.291 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-grafana-dashboards-2:19.3.0-4858.gbad1f558.el 1/12 2024-09-10T15:56:04.291 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-mgr-dashboard-2:19.3.0-4858.gbad1f558.el9.noa 2/12 2024-09-10T15:56:04.291 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-prometheus-alerts-2:19.3.0-4858.gbad1f558.el9 3/12 2024-09-10T15:56:04.291 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 4/12 2024-09-10T15:56:04.291 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 5/12 2024-09-10T15:56:04.291 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 6/12 2024-09-10T15:56:04.291 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 7/12 2024-09-10T15:56:04.291 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 8/12 2024-09-10T15:56:04.291 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 9/12 2024-09-10T15:56:04.291 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 10/12 2024-09-10T15:56:04.291 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 11/12 2024-09-10T15:56:04.642 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 12/12 2024-09-10T15:56:04.643 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T15:56:04.643 INFO:teuthology.orchestra.run.smithi115.stdout:Installed: 2024-09-10T15:56:04.643 INFO:teuthology.orchestra.run.smithi115.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-10T15:56:04.643 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-grafana-dashboards-2:19.3.0-4858.gbad1f558.el9.noarch 2024-09-10T15:56:04.643 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mgr-dashboard-2:19.3.0-4858.gbad1f558.el9.noarch 2024-09-10T15:56:04.643 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-prometheus-alerts-2:19.3.0-4858.gbad1f558.el9.noarch 2024-09-10T15:56:04.643 INFO:teuthology.orchestra.run.smithi115.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-10T15:56:04.643 INFO:teuthology.orchestra.run.smithi115.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-10T15:56:04.643 INFO:teuthology.orchestra.run.smithi115.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-10T15:56:04.643 INFO:teuthology.orchestra.run.smithi115.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-10T15:56:04.643 INFO:teuthology.orchestra.run.smithi115.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-10T15:56:04.643 INFO:teuthology.orchestra.run.smithi115.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-10T15:56:04.643 INFO:teuthology.orchestra.run.smithi115.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-10T15:56:04.644 INFO:teuthology.orchestra.run.smithi115.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-10T15:56:04.644 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T15:56:04.644 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2024-09-10T15:56:04.916 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum -y install ceph-mgr-diskprediction-local 2024-09-10T15:56:05.478 INFO:teuthology.orchestra.run.smithi115.stdout:Last metadata expiration check: 0:01:10 ago on Tue 10 Sep 2024 03:54:55 PM UTC. 2024-09-10T15:56:05.612 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2024-09-10T15:56:05.613 INFO:teuthology.orchestra.run.smithi115.stdout:======================================================================================= 2024-09-10T15:56:05.613 INFO:teuthology.orchestra.run.smithi115.stdout: Package Arch Version Repository Size 2024-09-10T15:56:05.613 INFO:teuthology.orchestra.run.smithi115.stdout:======================================================================================= 2024-09-10T15:56:05.613 INFO:teuthology.orchestra.run.smithi115.stdout:Installing: 2024-09-10T15:56:05.613 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mgr-diskprediction-local noarch 2:19.3.0-4858.gbad1f558.el9 ceph-noarch 7.4 M 2024-09-10T15:56:05.613 INFO:teuthology.orchestra.run.smithi115.stdout:Installing dependencies: 2024-09-10T15:56:05.613 INFO:teuthology.orchestra.run.smithi115.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-10T15:56:05.613 INFO:teuthology.orchestra.run.smithi115.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-10T15:56:05.613 INFO:teuthology.orchestra.run.smithi115.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-10T15:56:05.613 INFO:teuthology.orchestra.run.smithi115.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-10T15:56:05.614 INFO:teuthology.orchestra.run.smithi115.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-10T15:56:05.614 INFO:teuthology.orchestra.run.smithi115.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-10T15:56:05.614 INFO:teuthology.orchestra.run.smithi115.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-10T15:56:05.614 INFO:teuthology.orchestra.run.smithi115.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-10T15:56:05.614 INFO:teuthology.orchestra.run.smithi115.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-10T15:56:05.614 INFO:teuthology.orchestra.run.smithi115.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-10T15:56:05.614 INFO:teuthology.orchestra.run.smithi115.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-10T15:56:05.614 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T15:56:05.614 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction Summary 2024-09-10T15:56:05.614 INFO:teuthology.orchestra.run.smithi115.stdout:======================================================================================= 2024-09-10T15:56:05.614 INFO:teuthology.orchestra.run.smithi115.stdout:Install 12 Packages 2024-09-10T15:56:05.614 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T15:56:05.615 INFO:teuthology.orchestra.run.smithi115.stdout:Total download size: 38 M 2024-09-10T15:56:05.615 INFO:teuthology.orchestra.run.smithi115.stdout:Installed size: 212 M 2024-09-10T15:56:05.615 INFO:teuthology.orchestra.run.smithi115.stdout:Downloading Packages: 2024-09-10T15:56:05.881 INFO:teuthology.orchestra.run.smithi115.stdout:(1/12): libquadmath-11.5.0-2.el9.x86_64.rpm 1.3 MB/s | 191 kB 00:00 2024-09-10T15:56:05.923 INFO:teuthology.orchestra.run.smithi115.stdout:(2/12): libgfortran-11.5.0-2.el9.x86_64.rpm 4.2 MB/s | 801 kB 00:00 2024-09-10T15:56:05.948 INFO:teuthology.orchestra.run.smithi115.stdout:(3/12): flexiblas-3.0.4-8.el9.x86_64.rpm 488 kB/s | 33 kB 00:00 2024-09-10T15:56:05.990 INFO:teuthology.orchestra.run.smithi115.stdout:(4/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 436 kB/s | 18 kB 00:00 2024-09-10T15:56:06.040 INFO:teuthology.orchestra.run.smithi115.stdout:(5/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 26 MB/s | 3.0 MB 00:00 2024-09-10T15:56:06.065 INFO:teuthology.orchestra.run.smithi115.stdout:(6/12): openblas-0.3.26-2.el9.x86_64.rpm 511 kB/s | 38 kB 00:00 2024-09-10T15:56:06.160 INFO:teuthology.orchestra.run.smithi115.stdout:(7/12): openblas-openmp-0.3.26-2.el9.x86_64.rpm 42 MB/s | 4.9 MB 00:00 2024-09-10T15:56:06.183 INFO:teuthology.orchestra.run.smithi115.stdout:(8/12): python3-devel-3.9.19-8.el9.x86_64.rpm 2.0 MB/s | 245 kB 00:00 2024-09-10T15:56:06.283 INFO:teuthology.orchestra.run.smithi115.stdout:(9/12): ceph-mgr-diskprediction-local-19.3.0-48 14 MB/s | 7.4 MB 00:00 2024-09-10T15:56:06.317 INFO:teuthology.orchestra.run.smithi115.stdout:(10/12): python3-numpy-f2py-1.20.1-5.el9.x86_64 1.8 MB/s | 250 kB 00:00 2024-09-10T15:56:06.384 INFO:teuthology.orchestra.run.smithi115.stdout:(11/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 23 MB/s | 5.1 MB 00:00 2024-09-10T15:56:06.725 INFO:teuthology.orchestra.run.smithi115.stdout:(12/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 37 MB/s | 16 MB 00:00 2024-09-10T15:56:06.726 INFO:teuthology.orchestra.run.smithi115.stdout:-------------------------------------------------------------------------------- 2024-09-10T15:56:06.726 INFO:teuthology.orchestra.run.smithi115.stdout:Total 34 MB/s | 38 MB 00:01 2024-09-10T15:56:07.078 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction check 2024-09-10T15:56:07.103 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction check succeeded. 2024-09-10T15:56:07.103 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction test 2024-09-10T15:56:07.527 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction test succeeded. 2024-09-10T15:56:07.527 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction 2024-09-10T15:56:08.283 INFO:teuthology.orchestra.run.smithi115.stdout: Preparing : 1/1 2024-09-10T15:56:08.344 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-09-10T15:56:08.381 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 2/12 2024-09-10T15:56:08.484 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 3/12 2024-09-10T15:56:08.516 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 4/12 2024-09-10T15:56:08.701 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : openblas-0.3.26-2.el9.x86_64 5/12 2024-09-10T15:56:08.732 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 6/12 2024-09-10T15:56:08.812 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-09-10T15:56:09.496 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-09-10T15:56:09.612 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-09-10T15:56:10.879 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-09-10T15:56:11.907 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-09-10T15:56:11.940 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-mgr-diskprediction-local-2:19.3.0-4858.gbad1f 12/12 2024-09-10T15:56:14.577 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.3.0-4858.gbad1f 12/12 2024-09-10T15:56:14.577 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.3.0-4858.gbad1f 1/12 2024-09-10T15:56:14.577 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 2/12 2024-09-10T15:56:14.578 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 3/12 2024-09-10T15:56:14.578 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-09-10T15:56:14.578 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-09-10T15:56:14.578 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-09-10T15:56:14.578 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 7/12 2024-09-10T15:56:14.578 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 8/12 2024-09-10T15:56:14.578 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 9/12 2024-09-10T15:56:14.578 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-09-10T15:56:14.578 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-09-10T15:56:14.984 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-09-10T15:56:14.984 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T15:56:14.984 INFO:teuthology.orchestra.run.smithi115.stdout:Installed: 2024-09-10T15:56:14.984 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mgr-diskprediction-local-2:19.3.0-4858.gbad1f558.el9.noarch 2024-09-10T15:56:14.984 INFO:teuthology.orchestra.run.smithi115.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-10T15:56:14.984 INFO:teuthology.orchestra.run.smithi115.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-10T15:56:14.984 INFO:teuthology.orchestra.run.smithi115.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-10T15:56:14.985 INFO:teuthology.orchestra.run.smithi115.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-10T15:56:14.985 INFO:teuthology.orchestra.run.smithi115.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-10T15:56:14.985 INFO:teuthology.orchestra.run.smithi115.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-10T15:56:14.985 INFO:teuthology.orchestra.run.smithi115.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-10T15:56:14.985 INFO:teuthology.orchestra.run.smithi115.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-10T15:56:14.985 INFO:teuthology.orchestra.run.smithi115.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-10T15:56:14.985 INFO:teuthology.orchestra.run.smithi115.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-10T15:56:14.985 INFO:teuthology.orchestra.run.smithi115.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-10T15:56:14.985 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T15:56:14.985 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2024-09-10T15:56:15.358 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum -y install ceph-mgr-rook 2024-09-10T15:56:15.934 INFO:teuthology.orchestra.run.smithi115.stdout:Last metadata expiration check: 0:01:20 ago on Tue 10 Sep 2024 03:54:55 PM UTC. 2024-09-10T15:56:16.069 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2024-09-10T15:56:16.070 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-10T15:56:16.070 INFO:teuthology.orchestra.run.smithi115.stdout: Package Arch Version Repository Size 2024-09-10T15:56:16.070 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-10T15:56:16.070 INFO:teuthology.orchestra.run.smithi115.stdout:Installing: 2024-09-10T15:56:16.070 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mgr-rook noarch 2:19.3.0-4858.gbad1f558.el9 ceph-noarch 49 k 2024-09-10T15:56:16.070 INFO:teuthology.orchestra.run.smithi115.stdout:Installing dependencies: 2024-09-10T15:56:16.070 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-10T15:56:16.071 INFO:teuthology.orchestra.run.smithi115.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-10T15:56:16.071 INFO:teuthology.orchestra.run.smithi115.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-10T15:56:16.071 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-10T15:56:16.071 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-10T15:56:16.071 INFO:teuthology.orchestra.run.smithi115.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-10T15:56:16.071 INFO:teuthology.orchestra.run.smithi115.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-10T15:56:16.071 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-10T15:56:16.071 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-10T15:56:16.071 INFO:teuthology.orchestra.run.smithi115.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-10T15:56:16.071 INFO:teuthology.orchestra.run.smithi115.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-10T15:56:16.071 INFO:teuthology.orchestra.run.smithi115.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-10T15:56:16.071 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T15:56:16.071 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction Summary 2024-09-10T15:56:16.072 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-10T15:56:16.072 INFO:teuthology.orchestra.run.smithi115.stdout:Install 13 Packages 2024-09-10T15:56:16.072 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T15:56:16.072 INFO:teuthology.orchestra.run.smithi115.stdout:Total download size: 2.2 M 2024-09-10T15:56:16.072 INFO:teuthology.orchestra.run.smithi115.stdout:Installed size: 26 M 2024-09-10T15:56:16.072 INFO:teuthology.orchestra.run.smithi115.stdout:Downloading Packages: 2024-09-10T15:56:16.606 INFO:teuthology.orchestra.run.smithi115.stdout:(1/13): python3-jsonpatch-1.21-16.el9.noarch.rp 250 kB/s | 26 kB 00:00 2024-09-10T15:56:16.631 INFO:teuthology.orchestra.run.smithi115.stdout:(2/13): python3-jsonpointer-2.0-4.el9.noarch.rp 150 kB/s | 19 kB 00:00 2024-09-10T15:56:16.657 INFO:teuthology.orchestra.run.smithi115.stdout:(3/13): ceph-mgr-rook-19.3.0-4858.gbad1f558.el9 322 kB/s | 49 kB 00:00 2024-09-10T15:56:16.707 INFO:teuthology.orchestra.run.smithi115.stdout:(4/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 2.2 MB/s | 222 kB 00:00 2024-09-10T15:56:16.749 INFO:teuthology.orchestra.run.smithi115.stdout:(5/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.3 MB/s | 159 kB 00:00 2024-09-10T15:56:16.783 INFO:teuthology.orchestra.run.smithi115.stdout:(6/13): python3-requests-oauthlib-1.3.0-12.el9. 712 kB/s | 54 kB 00:00 2024-09-10T15:56:16.808 INFO:teuthology.orchestra.run.smithi115.stdout:(7/13): python3-pyasn1-modules-0.4.8-6.el9.noar 1.8 MB/s | 279 kB 00:00 2024-09-10T15:56:16.842 INFO:teuthology.orchestra.run.smithi115.stdout:(8/13): python3-cachetools-4.2.4-1.el9.noarch.r 350 kB/s | 32 kB 00:00 2024-09-10T15:56:16.875 INFO:teuthology.orchestra.run.smithi115.stdout:(9/13): python3-certifi-2023.05.07-4.el9.noarch 156 kB/s | 14 kB 00:00 2024-09-10T15:56:16.901 INFO:teuthology.orchestra.run.smithi115.stdout:(10/13): python3-rsa-4.9-2.el9.noarch.rpm 2.5 MB/s | 59 kB 00:00 2024-09-10T15:56:16.926 INFO:teuthology.orchestra.run.smithi115.stdout:(11/13): python3-google-auth-2.34.0-1.el9.noarc 1.9 MB/s | 220 kB 00:00 2024-09-10T15:56:16.951 INFO:teuthology.orchestra.run.smithi115.stdout:(12/13): python3-websocket-client-1.2.3-2.el9.n 1.7 MB/s | 90 kB 00:00 2024-09-10T15:56:16.984 INFO:teuthology.orchestra.run.smithi115.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 7.6 MB/s | 1.0 MB 00:00 2024-09-10T15:56:16.987 INFO:teuthology.orchestra.run.smithi115.stdout:-------------------------------------------------------------------------------- 2024-09-10T15:56:16.987 INFO:teuthology.orchestra.run.smithi115.stdout:Total 2.4 MB/s | 2.2 MB 00:00 2024-09-10T15:56:17.053 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction check 2024-09-10T15:56:17.068 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction check succeeded. 2024-09-10T15:56:17.068 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction test 2024-09-10T15:56:17.156 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction test succeeded. 2024-09-10T15:56:17.158 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction 2024-09-10T15:56:17.667 INFO:teuthology.orchestra.run.smithi115.stdout: Preparing : 1/1 2024-09-10T15:56:17.833 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-09-10T15:56:17.895 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-09-10T15:56:17.958 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-09-10T15:56:18.004 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-09-10T15:56:18.044 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 5/13 2024-09-10T15:56:18.147 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-09-10T15:56:18.256 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 7/13 2024-09-10T15:56:18.314 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-09-10T15:56:19.026 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-09-10T15:56:19.078 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-09-10T15:56:19.111 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-09-10T15:56:19.159 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-09-10T15:56:19.181 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-mgr-rook-2:19.3.0-4858.gbad1f558.el9.noarch 13/13 2024-09-10T15:56:19.958 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-mgr-rook-2:19.3.0-4858.gbad1f558.el9.noarch 13/13 2024-09-10T15:56:19.958 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-mgr-rook-2:19.3.0-4858.gbad1f558.el9.noarch 1/13 2024-09-10T15:56:19.958 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-09-10T15:56:19.958 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-09-10T15:56:19.958 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-09-10T15:56:19.959 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-09-10T15:56:19.959 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-09-10T15:56:19.959 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-09-10T15:56:19.959 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-09-10T15:56:19.959 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 9/13 2024-09-10T15:56:19.959 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 10/13 2024-09-10T15:56:19.959 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-09-10T15:56:19.959 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-09-10T15:56:20.314 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-09-10T15:56:20.314 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T15:56:20.314 INFO:teuthology.orchestra.run.smithi115.stdout:Installed: 2024-09-10T15:56:20.314 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mgr-rook-2:19.3.0-4858.gbad1f558.el9.noarch 2024-09-10T15:56:20.314 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-10T15:56:20.314 INFO:teuthology.orchestra.run.smithi115.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-10T15:56:20.314 INFO:teuthology.orchestra.run.smithi115.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-10T15:56:20.314 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-10T15:56:20.314 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-10T15:56:20.314 INFO:teuthology.orchestra.run.smithi115.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-10T15:56:20.315 INFO:teuthology.orchestra.run.smithi115.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-10T15:56:20.315 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-10T15:56:20.315 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-10T15:56:20.315 INFO:teuthology.orchestra.run.smithi115.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-10T15:56:20.315 INFO:teuthology.orchestra.run.smithi115.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-10T15:56:20.315 INFO:teuthology.orchestra.run.smithi115.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-10T15:56:20.315 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T15:56:20.315 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2024-09-10T15:56:20.549 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum -y install ceph-mgr-cephadm 2024-09-10T15:56:21.113 INFO:teuthology.orchestra.run.smithi115.stdout:Last metadata expiration check: 0:01:26 ago on Tue 10 Sep 2024 03:54:55 PM UTC. 2024-09-10T15:56:21.247 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2024-09-10T15:56:21.247 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-10T15:56:21.248 INFO:teuthology.orchestra.run.smithi115.stdout: Package Arch Version Repository Size 2024-09-10T15:56:21.248 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-10T15:56:21.248 INFO:teuthology.orchestra.run.smithi115.stdout:Installing: 2024-09-10T15:56:21.248 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mgr-cephadm noarch 2:19.3.0-4858.gbad1f558.el9 ceph-noarch 154 k 2024-09-10T15:56:21.248 INFO:teuthology.orchestra.run.smithi115.stdout:Installing dependencies: 2024-09-10T15:56:21.248 INFO:teuthology.orchestra.run.smithi115.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-10T15:56:21.248 INFO:teuthology.orchestra.run.smithi115.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-10T15:56:21.248 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-10T15:56:21.248 INFO:teuthology.orchestra.run.smithi115.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-10T15:56:21.248 INFO:teuthology.orchestra.run.smithi115.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-10T15:56:21.248 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T15:56:21.249 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction Summary 2024-09-10T15:56:21.249 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-10T15:56:21.249 INFO:teuthology.orchestra.run.smithi115.stdout:Install 6 Packages 2024-09-10T15:56:21.249 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T15:56:21.249 INFO:teuthology.orchestra.run.smithi115.stdout:Total download size: 7.0 M 2024-09-10T15:56:21.250 INFO:teuthology.orchestra.run.smithi115.stdout:Installed size: 33 M 2024-09-10T15:56:21.250 INFO:teuthology.orchestra.run.smithi115.stdout:Downloading Packages: 2024-09-10T15:56:21.429 INFO:teuthology.orchestra.run.smithi115.stdout:(1/6): python3-jinja2-2.11.3-6.el9.noarch.rpm 1.9 MB/s | 248 kB 00:00 2024-09-10T15:56:21.487 INFO:teuthology.orchestra.run.smithi115.stdout:(2/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 9.2 MB/s | 548 kB 00:00 2024-09-10T15:56:21.512 INFO:teuthology.orchestra.run.smithi115.stdout:(3/6): python3-natsort-7.1.1-5.el9.noarch.rpm 2.3 MB/s | 58 kB 00:00 2024-09-10T15:56:21.538 INFO:teuthology.orchestra.run.smithi115.stdout:(4/6): python3-typing-extensions-4.12.2-2.el9.n 2.9 MB/s | 74 kB 00:00 2024-09-10T15:56:21.613 INFO:teuthology.orchestra.run.smithi115.stdout:(5/6): ceph-mgr-cephadm-19.3.0-4858.gbad1f558.e 494 kB/s | 154 kB 00:00 2024-09-10T15:56:22.905 INFO:teuthology.orchestra.run.smithi115.stdout:(6/6): python3-babel-2.9.1-2.el9.noarch.rpm 3.7 MB/s | 6.0 MB 00:01 2024-09-10T15:56:22.906 INFO:teuthology.orchestra.run.smithi115.stdout:-------------------------------------------------------------------------------- 2024-09-10T15:56:22.907 INFO:teuthology.orchestra.run.smithi115.stdout:Total 4.2 MB/s | 7.0 MB 00:01 2024-09-10T15:56:22.977 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction check 2024-09-10T15:56:22.988 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction check succeeded. 2024-09-10T15:56:22.989 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction test 2024-09-10T15:56:23.090 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction test succeeded. 2024-09-10T15:56:23.091 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction 2024-09-10T15:56:23.398 INFO:teuthology.orchestra.run.smithi115.stdout: Preparing : 1/1 2024-09-10T15:56:23.578 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 1/6 2024-09-10T15:56:23.631 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-09-10T15:56:24.155 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-09-10T15:56:24.250 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-09-10T15:56:24.318 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 5/6 2024-09-10T15:56:24.352 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-mgr-cephadm-2:19.3.0-4858.gbad1f558.el9.noarch 6/6 2024-09-10T15:56:25.136 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.3.0-4858.gbad1f558.el9.noarch 6/6 2024-09-10T15:56:25.137 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-mgr-cephadm-2:19.3.0-4858.gbad1f558.el9.noarch 1/6 2024-09-10T15:56:25.137 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-09-10T15:56:25.137 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 3/6 2024-09-10T15:56:25.137 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-09-10T15:56:25.137 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-09-10T15:56:25.427 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 6/6 2024-09-10T15:56:25.427 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T15:56:25.427 INFO:teuthology.orchestra.run.smithi115.stdout:Installed: 2024-09-10T15:56:25.427 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mgr-cephadm-2:19.3.0-4858.gbad1f558.el9.noarch 2024-09-10T15:56:25.427 INFO:teuthology.orchestra.run.smithi115.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-10T15:56:25.427 INFO:teuthology.orchestra.run.smithi115.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-10T15:56:25.427 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-10T15:56:25.427 INFO:teuthology.orchestra.run.smithi115.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-10T15:56:25.427 INFO:teuthology.orchestra.run.smithi115.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-10T15:56:25.427 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T15:56:25.427 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2024-09-10T15:56:25.704 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum -y install ceph-fuse 2024-09-10T15:56:26.261 INFO:teuthology.orchestra.run.smithi115.stdout:Last metadata expiration check: 0:01:31 ago on Tue 10 Sep 2024 03:54:55 PM UTC. 2024-09-10T15:56:26.392 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2024-09-10T15:56:26.393 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-10T15:56:26.393 INFO:teuthology.orchestra.run.smithi115.stdout: Package Arch Version Repository Size 2024-09-10T15:56:26.393 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-10T15:56:26.393 INFO:teuthology.orchestra.run.smithi115.stdout:Installing: 2024-09-10T15:56:26.393 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-fuse x86_64 2:19.3.0-4858.gbad1f558.el9 ceph 901 k 2024-09-10T15:56:26.393 INFO:teuthology.orchestra.run.smithi115.stdout:Installing dependencies: 2024-09-10T15:56:26.393 INFO:teuthology.orchestra.run.smithi115.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-10T15:56:26.393 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T15:56:26.394 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction Summary 2024-09-10T15:56:26.394 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-10T15:56:26.394 INFO:teuthology.orchestra.run.smithi115.stdout:Install 2 Packages 2024-09-10T15:56:26.394 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T15:56:26.394 INFO:teuthology.orchestra.run.smithi115.stdout:Total download size: 981 k 2024-09-10T15:56:26.394 INFO:teuthology.orchestra.run.smithi115.stdout:Installed size: 2.9 M 2024-09-10T15:56:26.394 INFO:teuthology.orchestra.run.smithi115.stdout:Downloading Packages: 2024-09-10T15:56:27.232 INFO:teuthology.orchestra.run.smithi115.stdout:(1/2): ceph-fuse-19.3.0-4858.gbad1f558.el9.x86_ 1.5 MB/s | 901 kB 00:00 2024-09-10T15:56:27.293 INFO:teuthology.orchestra.run.smithi115.stdout:(2/2): fuse-2.9.9-16.el9.x86_64.rpm 124 kB/s | 80 kB 00:00 2024-09-10T15:56:27.293 INFO:teuthology.orchestra.run.smithi115.stdout:-------------------------------------------------------------------------------- 2024-09-10T15:56:27.293 INFO:teuthology.orchestra.run.smithi115.stdout:Total 1.1 MB/s | 981 kB 00:00 2024-09-10T15:56:27.305 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction check 2024-09-10T15:56:27.330 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction check succeeded. 2024-09-10T15:56:27.330 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction test 2024-09-10T15:56:27.413 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction test succeeded. 2024-09-10T15:56:27.414 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction 2024-09-10T15:56:27.622 INFO:teuthology.orchestra.run.smithi115.stdout: Preparing : 1/1 2024-09-10T15:56:27.768 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : fuse-2.9.9-16.el9.x86_64 1/2 2024-09-10T15:56:27.839 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-fuse-2:19.3.0-4858.gbad1f558.el9.x86_64 2/2 2024-09-10T15:56:28.528 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-fuse-2:19.3.0-4858.gbad1f558.el9.x86_64 2/2 2024-09-10T15:56:28.528 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-fuse-2:19.3.0-4858.gbad1f558.el9.x86_64 1/2 2024-09-10T15:56:28.822 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 2/2 2024-09-10T15:56:28.822 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T15:56:28.822 INFO:teuthology.orchestra.run.smithi115.stdout:Installed: 2024-09-10T15:56:28.822 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-fuse-2:19.3.0-4858.gbad1f558.el9.x86_64 fuse-2.9.9-16.el9.x86_64 2024-09-10T15:56:28.823 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T15:56:28.823 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2024-09-10T15:56:29.122 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum -y install ceph-volume 2024-09-10T15:56:29.704 INFO:teuthology.orchestra.run.smithi115.stdout:Last metadata expiration check: 0:01:34 ago on Tue 10 Sep 2024 03:54:55 PM UTC. 2024-09-10T15:56:29.839 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2024-09-10T15:56:29.840 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-10T15:56:29.840 INFO:teuthology.orchestra.run.smithi115.stdout: Package Arch Version Repository Size 2024-09-10T15:56:29.840 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-10T15:56:29.840 INFO:teuthology.orchestra.run.smithi115.stdout:Installing: 2024-09-10T15:56:29.840 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-volume noarch 2:19.3.0-4858.gbad1f558.el9 ceph-noarch 285 k 2024-09-10T15:56:29.840 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T15:56:29.840 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction Summary 2024-09-10T15:56:29.840 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-10T15:56:29.840 INFO:teuthology.orchestra.run.smithi115.stdout:Install 1 Package 2024-09-10T15:56:29.841 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T15:56:29.841 INFO:teuthology.orchestra.run.smithi115.stdout:Total download size: 285 k 2024-09-10T15:56:29.841 INFO:teuthology.orchestra.run.smithi115.stdout:Installed size: 1.3 M 2024-09-10T15:56:29.841 INFO:teuthology.orchestra.run.smithi115.stdout:Downloading Packages: 2024-09-10T15:56:30.319 INFO:teuthology.orchestra.run.smithi115.stdout:ceph-volume-19.3.0-4858.gbad1f558.el9.noarch.rp 596 kB/s | 285 kB 00:00 2024-09-10T15:56:30.320 INFO:teuthology.orchestra.run.smithi115.stdout:-------------------------------------------------------------------------------- 2024-09-10T15:56:30.320 INFO:teuthology.orchestra.run.smithi115.stdout:Total 594 kB/s | 285 kB 00:00 2024-09-10T15:56:30.320 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction check 2024-09-10T15:56:30.329 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction check succeeded. 2024-09-10T15:56:30.329 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction test 2024-09-10T15:56:30.358 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction test succeeded. 2024-09-10T15:56:30.358 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction 2024-09-10T15:56:30.584 INFO:teuthology.orchestra.run.smithi115.stdout: Preparing : 1/1 2024-09-10T15:56:30.655 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-volume-2:19.3.0-4858.gbad1f558.el9.noarch 1/1 2024-09-10T15:56:30.671 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-volume-2:19.3.0-4858.gbad1f558.el9.noarch 1/1 2024-09-10T15:56:30.671 INFO:teuthology.orchestra.run.smithi115.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T15:56:30.671 INFO:teuthology.orchestra.run.smithi115.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-10T15:56:30.671 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T15:56:31.727 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-volume-2:19.3.0-4858.gbad1f558.el9.noarch 1/1 2024-09-10T15:56:31.727 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T15:56:31.727 INFO:teuthology.orchestra.run.smithi115.stdout:Installed: 2024-09-10T15:56:31.727 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-volume-2:19.3.0-4858.gbad1f558.el9.noarch 2024-09-10T15:56:31.727 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T15:56:31.727 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2024-09-10T15:56:32.034 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum -y install librados-devel 2024-09-10T15:56:32.647 INFO:teuthology.orchestra.run.smithi115.stdout:Last metadata expiration check: 0:01:37 ago on Tue 10 Sep 2024 03:54:55 PM UTC. 2024-09-10T15:56:32.788 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2024-09-10T15:56:32.789 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-10T15:56:32.789 INFO:teuthology.orchestra.run.smithi115.stdout: Package Arch Version Repo Size 2024-09-10T15:56:32.789 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-10T15:56:32.789 INFO:teuthology.orchestra.run.smithi115.stdout:Installing: 2024-09-10T15:56:32.789 INFO:teuthology.orchestra.run.smithi115.stdout: librados-devel x86_64 2:19.3.0-4858.gbad1f558.el9 ceph 126 k 2024-09-10T15:56:32.789 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T15:56:32.789 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction Summary 2024-09-10T15:56:32.789 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-10T15:56:32.789 INFO:teuthology.orchestra.run.smithi115.stdout:Install 1 Package 2024-09-10T15:56:32.789 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T15:56:32.789 INFO:teuthology.orchestra.run.smithi115.stdout:Total download size: 126 k 2024-09-10T15:56:32.790 INFO:teuthology.orchestra.run.smithi115.stdout:Installed size: 456 k 2024-09-10T15:56:32.790 INFO:teuthology.orchestra.run.smithi115.stdout:Downloading Packages: 2024-09-10T15:56:33.009 INFO:teuthology.orchestra.run.smithi115.stdout:librados-devel-19.3.0-4858.gbad1f558.el9.x86_64 577 kB/s | 126 kB 00:00 2024-09-10T15:56:33.009 INFO:teuthology.orchestra.run.smithi115.stdout:-------------------------------------------------------------------------------- 2024-09-10T15:56:33.010 INFO:teuthology.orchestra.run.smithi115.stdout:Total 573 kB/s | 126 kB 00:00 2024-09-10T15:56:33.010 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction check 2024-09-10T15:56:33.015 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction check succeeded. 2024-09-10T15:56:33.016 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction test 2024-09-10T15:56:33.085 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction test succeeded. 2024-09-10T15:56:33.086 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction 2024-09-10T15:56:33.242 INFO:teuthology.orchestra.run.smithi115.stdout: Preparing : 1/1 2024-09-10T15:56:33.341 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : librados-devel-2:19.3.0-4858.gbad1f558.el9.x86_64 1/1 2024-09-10T15:56:33.822 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: librados-devel-2:19.3.0-4858.gbad1f558.el9.x86_64 1/1 2024-09-10T15:56:34.076 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : librados-devel-2:19.3.0-4858.gbad1f558.el9.x86_64 1/1 2024-09-10T15:56:34.077 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T15:56:34.077 INFO:teuthology.orchestra.run.smithi115.stdout:Installed: 2024-09-10T15:56:34.077 INFO:teuthology.orchestra.run.smithi115.stdout: librados-devel-2:19.3.0-4858.gbad1f558.el9.x86_64 2024-09-10T15:56:34.077 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T15:56:34.077 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2024-09-10T15:56:34.329 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum -y install libcephfs2 2024-09-10T15:56:34.896 INFO:teuthology.orchestra.run.smithi115.stdout:Last metadata expiration check: 0:01:39 ago on Tue 10 Sep 2024 03:54:55 PM UTC. 2024-09-10T15:56:34.987 INFO:teuthology.orchestra.run.smithi115.stdout:Package libcephfs2-2:19.3.0-4858.gbad1f558.el9.x86_64 is already installed. 2024-09-10T15:56:35.031 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2024-09-10T15:56:35.032 INFO:teuthology.orchestra.run.smithi115.stdout:Nothing to do. 2024-09-10T15:56:35.032 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2024-09-10T15:56:35.103 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum -y install libcephfs-devel 2024-09-10T15:56:35.668 INFO:teuthology.orchestra.run.smithi115.stdout:Last metadata expiration check: 0:01:40 ago on Tue 10 Sep 2024 03:54:55 PM UTC. 2024-09-10T15:56:35.803 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2024-09-10T15:56:35.804 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-10T15:56:35.804 INFO:teuthology.orchestra.run.smithi115.stdout: Package Arch Version Repo Size 2024-09-10T15:56:35.804 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-10T15:56:35.804 INFO:teuthology.orchestra.run.smithi115.stdout:Installing: 2024-09-10T15:56:35.804 INFO:teuthology.orchestra.run.smithi115.stdout: libcephfs-devel x86_64 2:19.3.0-4858.gbad1f558.el9 ceph 32 k 2024-09-10T15:56:35.804 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T15:56:35.804 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction Summary 2024-09-10T15:56:35.804 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-10T15:56:35.805 INFO:teuthology.orchestra.run.smithi115.stdout:Install 1 Package 2024-09-10T15:56:35.805 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T15:56:35.805 INFO:teuthology.orchestra.run.smithi115.stdout:Total download size: 32 k 2024-09-10T15:56:35.805 INFO:teuthology.orchestra.run.smithi115.stdout:Installed size: 142 k 2024-09-10T15:56:35.805 INFO:teuthology.orchestra.run.smithi115.stdout:Downloading Packages: 2024-09-10T15:56:35.971 INFO:teuthology.orchestra.run.smithi115.stdout:libcephfs-devel-19.3.0-4858.gbad1f558.el9.x86_6 191 kB/s | 32 kB 00:00 2024-09-10T15:56:35.972 INFO:teuthology.orchestra.run.smithi115.stdout:-------------------------------------------------------------------------------- 2024-09-10T15:56:35.972 INFO:teuthology.orchestra.run.smithi115.stdout:Total 189 kB/s | 32 kB 00:00 2024-09-10T15:56:35.972 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction check 2024-09-10T15:56:35.976 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction check succeeded. 2024-09-10T15:56:35.976 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction test 2024-09-10T15:56:35.998 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction test succeeded. 2024-09-10T15:56:35.998 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction 2024-09-10T15:56:36.087 INFO:teuthology.orchestra.run.smithi115.stdout: Preparing : 1/1 2024-09-10T15:56:36.195 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : libcephfs-devel-2:19.3.0-4858.gbad1f558.el9.x86_64 1/1 2024-09-10T15:56:36.563 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: libcephfs-devel-2:19.3.0-4858.gbad1f558.el9.x86_64 1/1 2024-09-10T15:56:36.818 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libcephfs-devel-2:19.3.0-4858.gbad1f558.el9.x86_64 1/1 2024-09-10T15:56:36.818 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T15:56:36.818 INFO:teuthology.orchestra.run.smithi115.stdout:Installed: 2024-09-10T15:56:36.818 INFO:teuthology.orchestra.run.smithi115.stdout: libcephfs-devel-2:19.3.0-4858.gbad1f558.el9.x86_64 2024-09-10T15:56:36.818 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T15:56:36.818 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2024-09-10T15:56:37.012 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum -y install librados2 2024-09-10T15:56:37.576 INFO:teuthology.orchestra.run.smithi115.stdout:Last metadata expiration check: 0:01:42 ago on Tue 10 Sep 2024 03:54:55 PM UTC. 2024-09-10T15:56:37.663 INFO:teuthology.orchestra.run.smithi115.stdout:Package librados2-2:19.3.0-4858.gbad1f558.el9.x86_64 is already installed. 2024-09-10T15:56:37.706 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2024-09-10T15:56:37.707 INFO:teuthology.orchestra.run.smithi115.stdout:Nothing to do. 2024-09-10T15:56:37.708 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2024-09-10T15:56:37.783 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum -y install librbd1 2024-09-10T15:56:38.343 INFO:teuthology.orchestra.run.smithi115.stdout:Last metadata expiration check: 0:01:43 ago on Tue 10 Sep 2024 03:54:55 PM UTC. 2024-09-10T15:56:38.433 INFO:teuthology.orchestra.run.smithi115.stdout:Package librbd1-2:19.3.0-4858.gbad1f558.el9.x86_64 is already installed. 2024-09-10T15:56:38.476 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2024-09-10T15:56:38.477 INFO:teuthology.orchestra.run.smithi115.stdout:Nothing to do. 2024-09-10T15:56:38.477 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2024-09-10T15:56:38.547 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum -y install python3-rados 2024-09-10T15:56:39.108 INFO:teuthology.orchestra.run.smithi115.stdout:Last metadata expiration check: 0:01:44 ago on Tue 10 Sep 2024 03:54:55 PM UTC. 2024-09-10T15:56:39.197 INFO:teuthology.orchestra.run.smithi115.stdout:Package python3-rados-2:19.3.0-4858.gbad1f558.el9.x86_64 is already installed. 2024-09-10T15:56:39.240 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2024-09-10T15:56:39.241 INFO:teuthology.orchestra.run.smithi115.stdout:Nothing to do. 2024-09-10T15:56:39.241 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2024-09-10T15:56:39.314 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum -y install python3-rgw 2024-09-10T15:56:39.873 INFO:teuthology.orchestra.run.smithi115.stdout:Last metadata expiration check: 0:01:44 ago on Tue 10 Sep 2024 03:54:55 PM UTC. 2024-09-10T15:56:39.965 INFO:teuthology.orchestra.run.smithi115.stdout:Package python3-rgw-2:19.3.0-4858.gbad1f558.el9.x86_64 is already installed. 2024-09-10T15:56:40.006 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2024-09-10T15:56:40.007 INFO:teuthology.orchestra.run.smithi115.stdout:Nothing to do. 2024-09-10T15:56:40.007 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2024-09-10T15:56:40.082 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum -y install python3-cephfs 2024-09-10T15:56:40.650 INFO:teuthology.orchestra.run.smithi115.stdout:Last metadata expiration check: 0:01:45 ago on Tue 10 Sep 2024 03:54:55 PM UTC. 2024-09-10T15:56:40.741 INFO:teuthology.orchestra.run.smithi115.stdout:Package python3-cephfs-2:19.3.0-4858.gbad1f558.el9.x86_64 is already installed. 2024-09-10T15:56:40.785 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2024-09-10T15:56:40.785 INFO:teuthology.orchestra.run.smithi115.stdout:Nothing to do. 2024-09-10T15:56:40.785 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2024-09-10T15:56:40.867 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum -y install python3-rbd 2024-09-10T15:56:41.427 INFO:teuthology.orchestra.run.smithi115.stdout:Last metadata expiration check: 0:01:46 ago on Tue 10 Sep 2024 03:54:55 PM UTC. 2024-09-10T15:56:41.517 INFO:teuthology.orchestra.run.smithi115.stdout:Package python3-rbd-2:19.3.0-4858.gbad1f558.el9.x86_64 is already installed. 2024-09-10T15:56:41.560 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2024-09-10T15:56:41.561 INFO:teuthology.orchestra.run.smithi115.stdout:Nothing to do. 2024-09-10T15:56:41.561 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2024-09-10T15:56:41.642 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum -y install rbd-fuse 2024-09-10T15:56:42.204 INFO:teuthology.orchestra.run.smithi115.stdout:Last metadata expiration check: 0:01:47 ago on Tue 10 Sep 2024 03:54:55 PM UTC. 2024-09-10T15:56:42.339 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2024-09-10T15:56:42.340 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-10T15:56:42.340 INFO:teuthology.orchestra.run.smithi115.stdout: Package Architecture Version Repository Size 2024-09-10T15:56:42.340 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-10T15:56:42.340 INFO:teuthology.orchestra.run.smithi115.stdout:Installing: 2024-09-10T15:56:42.340 INFO:teuthology.orchestra.run.smithi115.stdout: rbd-fuse x86_64 2:19.3.0-4858.gbad1f558.el9 ceph 85 k 2024-09-10T15:56:42.340 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T15:56:42.340 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction Summary 2024-09-10T15:56:42.340 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-10T15:56:42.340 INFO:teuthology.orchestra.run.smithi115.stdout:Install 1 Package 2024-09-10T15:56:42.341 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T15:56:42.341 INFO:teuthology.orchestra.run.smithi115.stdout:Total download size: 85 k 2024-09-10T15:56:42.341 INFO:teuthology.orchestra.run.smithi115.stdout:Installed size: 231 k 2024-09-10T15:56:42.341 INFO:teuthology.orchestra.run.smithi115.stdout:Downloading Packages: 2024-09-10T15:56:42.541 INFO:teuthology.orchestra.run.smithi115.stdout:rbd-fuse-19.3.0-4858.gbad1f558.el9.x86_64.rpm 425 kB/s | 85 kB 00:00 2024-09-10T15:56:42.541 INFO:teuthology.orchestra.run.smithi115.stdout:-------------------------------------------------------------------------------- 2024-09-10T15:56:42.542 INFO:teuthology.orchestra.run.smithi115.stdout:Total 423 kB/s | 85 kB 00:00 2024-09-10T15:56:42.542 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction check 2024-09-10T15:56:42.549 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction check succeeded. 2024-09-10T15:56:42.549 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction test 2024-09-10T15:56:42.612 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction test succeeded. 2024-09-10T15:56:42.613 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction 2024-09-10T15:56:42.754 INFO:teuthology.orchestra.run.smithi115.stdout: Preparing : 1/1 2024-09-10T15:56:42.856 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : rbd-fuse-2:19.3.0-4858.gbad1f558.el9.x86_64 1/1 2024-09-10T15:56:43.248 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: rbd-fuse-2:19.3.0-4858.gbad1f558.el9.x86_64 1/1 2024-09-10T15:56:43.507 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : rbd-fuse-2:19.3.0-4858.gbad1f558.el9.x86_64 1/1 2024-09-10T15:56:43.507 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T15:56:43.508 INFO:teuthology.orchestra.run.smithi115.stdout:Installed: 2024-09-10T15:56:43.508 INFO:teuthology.orchestra.run.smithi115.stdout: rbd-fuse-2:19.3.0-4858.gbad1f558.el9.x86_64 2024-09-10T15:56:43.508 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T15:56:43.508 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2024-09-10T15:56:43.793 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum -y install rbd-mirror 2024-09-10T15:56:44.370 INFO:teuthology.orchestra.run.smithi115.stdout:Last metadata expiration check: 0:01:49 ago on Tue 10 Sep 2024 03:54:55 PM UTC. 2024-09-10T15:56:44.507 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2024-09-10T15:56:44.508 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-10T15:56:44.508 INFO:teuthology.orchestra.run.smithi115.stdout: Package Arch Version Repo Size 2024-09-10T15:56:44.508 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-10T15:56:44.508 INFO:teuthology.orchestra.run.smithi115.stdout:Installing: 2024-09-10T15:56:44.508 INFO:teuthology.orchestra.run.smithi115.stdout: rbd-mirror x86_64 2:19.3.0-4858.gbad1f558.el9 ceph 3.2 M 2024-09-10T15:56:44.508 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T15:56:44.508 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction Summary 2024-09-10T15:56:44.508 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-10T15:56:44.509 INFO:teuthology.orchestra.run.smithi115.stdout:Install 1 Package 2024-09-10T15:56:44.509 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T15:56:44.509 INFO:teuthology.orchestra.run.smithi115.stdout:Total download size: 3.2 M 2024-09-10T15:56:44.509 INFO:teuthology.orchestra.run.smithi115.stdout:Installed size: 14 M 2024-09-10T15:56:44.509 INFO:teuthology.orchestra.run.smithi115.stdout:Downloading Packages: 2024-09-10T15:56:44.844 INFO:teuthology.orchestra.run.smithi115.stdout:rbd-mirror-19.3.0-4858.gbad1f558.el9.x86_64.rpm 9.5 MB/s | 3.2 MB 00:00 2024-09-10T15:56:44.845 INFO:teuthology.orchestra.run.smithi115.stdout:-------------------------------------------------------------------------------- 2024-09-10T15:56:44.845 INFO:teuthology.orchestra.run.smithi115.stdout:Total 9.4 MB/s | 3.2 MB 00:00 2024-09-10T15:56:44.846 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction check 2024-09-10T15:56:44.854 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction check succeeded. 2024-09-10T15:56:44.855 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction test 2024-09-10T15:56:44.941 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction test succeeded. 2024-09-10T15:56:44.941 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction 2024-09-10T15:56:45.385 INFO:teuthology.orchestra.run.smithi115.stdout: Preparing : 1/1 2024-09-10T15:56:45.445 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : rbd-mirror-2:19.3.0-4858.gbad1f558.el9.x86_64 1/1 2024-09-10T15:56:45.470 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: rbd-mirror-2:19.3.0-4858.gbad1f558.el9.x86_64 1/1 2024-09-10T15:56:45.471 INFO:teuthology.orchestra.run.smithi115.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T15:56:45.471 INFO:teuthology.orchestra.run.smithi115.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-10T15:56:45.471 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-10T15:56:45.471 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-10T15:56:45.471 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T15:56:46.574 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : rbd-mirror-2:19.3.0-4858.gbad1f558.el9.x86_64 1/1 2024-09-10T15:56:46.575 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T15:56:46.575 INFO:teuthology.orchestra.run.smithi115.stdout:Installed: 2024-09-10T15:56:46.575 INFO:teuthology.orchestra.run.smithi115.stdout: rbd-mirror-2:19.3.0-4858.gbad1f558.el9.x86_64 2024-09-10T15:56:46.575 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T15:56:46.575 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2024-09-10T15:56:46.807 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum -y install rbd-nbd 2024-09-10T15:56:47.386 INFO:teuthology.orchestra.run.smithi115.stdout:Last metadata expiration check: 0:01:52 ago on Tue 10 Sep 2024 03:54:55 PM UTC. 2024-09-10T15:56:47.518 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2024-09-10T15:56:47.519 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-10T15:56:47.519 INFO:teuthology.orchestra.run.smithi115.stdout: Package Architecture Version Repository Size 2024-09-10T15:56:47.519 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-10T15:56:47.520 INFO:teuthology.orchestra.run.smithi115.stdout:Installing: 2024-09-10T15:56:47.520 INFO:teuthology.orchestra.run.smithi115.stdout: rbd-nbd x86_64 2:19.3.0-4858.gbad1f558.el9 ceph 170 k 2024-09-10T15:56:47.520 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T15:56:47.520 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction Summary 2024-09-10T15:56:47.520 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-10T15:56:47.520 INFO:teuthology.orchestra.run.smithi115.stdout:Install 1 Package 2024-09-10T15:56:47.520 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T15:56:47.520 INFO:teuthology.orchestra.run.smithi115.stdout:Total download size: 170 k 2024-09-10T15:56:47.520 INFO:teuthology.orchestra.run.smithi115.stdout:Installed size: 490 k 2024-09-10T15:56:47.520 INFO:teuthology.orchestra.run.smithi115.stdout:Downloading Packages: 2024-09-10T15:56:47.707 INFO:teuthology.orchestra.run.smithi115.stdout:rbd-nbd-19.3.0-4858.gbad1f558.el9.x86_64.rpm 913 kB/s | 170 kB 00:00 2024-09-10T15:56:47.708 INFO:teuthology.orchestra.run.smithi115.stdout:-------------------------------------------------------------------------------- 2024-09-10T15:56:47.708 INFO:teuthology.orchestra.run.smithi115.stdout:Total 906 kB/s | 170 kB 00:00 2024-09-10T15:56:47.708 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction check 2024-09-10T15:56:47.715 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction check succeeded. 2024-09-10T15:56:47.715 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction test 2024-09-10T15:56:47.779 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction test succeeded. 2024-09-10T15:56:47.780 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction 2024-09-10T15:56:47.937 INFO:teuthology.orchestra.run.smithi115.stdout: Preparing : 1/1 2024-09-10T15:56:48.045 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : rbd-nbd-2:19.3.0-4858.gbad1f558.el9.x86_64 1/1 2024-09-10T15:56:48.464 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: rbd-nbd-2:19.3.0-4858.gbad1f558.el9.x86_64 1/1 2024-09-10T15:56:48.714 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : rbd-nbd-2:19.3.0-4858.gbad1f558.el9.x86_64 1/1 2024-09-10T15:56:48.715 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T15:56:48.715 INFO:teuthology.orchestra.run.smithi115.stdout:Installed: 2024-09-10T15:56:48.715 INFO:teuthology.orchestra.run.smithi115.stdout: rbd-nbd-2:19.3.0-4858.gbad1f558.el9.x86_64 2024-09-10T15:56:48.715 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T15:56:48.715 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2024-09-10T15:56:48.955 DEBUG:teuthology.parallel:result is None 2024-09-10T15:56:48.956 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=bad1f558c327094897e4a32e99dd5cb4ae6ccd46 2024-09-10T15:56:49.103 DEBUG:teuthology.orchestra.run.smithi115:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-10T15:56:49.124 INFO:teuthology.orchestra.run.smithi115.stdout:19.3.0-4858.gbad1f558.el9 2024-09-10T15:56:49.125 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-4858.gbad1f558.el9 2024-09-10T15:56:49.125 INFO:teuthology.task.install:The correct ceph version 19.3.0-4858.gbad1f558 is installed. 2024-09-10T15:56:49.127 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-10T15:56:49.127 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-09-10T15:56:49.127 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-10T15:56:49.189 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-10T15:56:49.190 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-09-10T15:56:49.190 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd of=/usr/bin/daemon-helper 2024-09-10T15:56:49.254 DEBUG:teuthology.orchestra.run.smithi115:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-10T15:56:49.318 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-10T15:56:49.319 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-09-10T15:56:49.319 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-10T15:56:49.382 DEBUG:teuthology.orchestra.run.smithi115:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-10T15:56:49.447 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-10T15:56:49.448 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-09-10T15:56:49.448 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd of=/usr/bin/stdin-killer 2024-09-10T15:56:49.511 DEBUG:teuthology.orchestra.run.smithi115:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-10T15:56:49.576 INFO:teuthology.run_tasks:Running task ceph... 2024-09-10T15:56:49.661 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-09-10T15:56:49.661 DEBUG:teuthology.orchestra.run.smithi115:> sudo chmod 777 /var/log/ceph 2024-09-10T15:56:49.707 INFO:tasks.ceph:Disabling ceph logrotate... 2024-09-10T15:56:49.707 DEBUG:teuthology.orchestra.run.smithi115:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-10T15:56:49.731 INFO:tasks.ceph:Creating extra log directories... 2024-09-10T15:56:49.731 DEBUG:teuthology.orchestra.run.smithi115:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-10T15:56:49.803 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-09-10T15:56:49.803 INFO:tasks.ceph:config {'conf': {'global': {'mon election default strategy': 1, 'ms bind msgr1': False, 'ms bind msgr2': True, 'ms type': 'async'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon scrub interval': 300}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'stupid', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'bluestore write v2': False, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd mclock override recovery settings': True, 'osd mclock profile': 'high_recovery_ops', 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random'}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(OBJECT_', '\\(SLOW_OPS\\)', 'slow request', '\\(POOL_APP_NOT_ENABLED\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-09-10T15:56:49.804 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/skanta-2024-09-10_15:30:27-rados-wip-bharath7-testing-2024-09-10-1409-distro-default-smithi/7899267', 'branch': 'wip-bharath7-testing-2024-09-10-1409', 'description': 'rados/singleton/{all/lost-unfound mon_election/classic msgr-failures/none msgr/async-v2only objectstore/{bluestore-options/write$/{write_v1} bluestore/bluestore-stupid} rados supported-random-distro$/{centos_latest}}', 'email': 'skanta@redhat.com', 'first_in_suite': False, 'job_id': '7899267', 'kernel': {'kdb': 1, 'sha1': 'distro'}, 'last_in_suite': False, 'machine_type': 'smithi', 'name': 'skanta-2024-09-10_15:30:27-rados-wip-bharath7-testing-2024-09-10-1409-distro-default-smithi', 'no_nested_subset': False, 'openstack': [{'volumes': {'count': 3, 'size': 10}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'wip-bharath7-testing-2024-09-10-1409'}, 'ceph': {'conf': {'global': {'mon election default strategy': 1, 'ms bind msgr1': False, 'ms bind msgr2': True, 'ms type': 'async'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon scrub interval': 300}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'stupid', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'bluestore write v2': False, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd mclock override recovery settings': True, 'osd mclock profile': 'high_recovery_ops', 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random'}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)'], 'sha1': 'bad1f558c327094897e4a32e99dd5cb4ae6ccd46'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': 'bad1f558c327094897e4a32e99dd5cb4ae6ccd46'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-bharath7-testing-2024-09-10-1409', 'sha1': 'bad1f558c327094897e4a32e99dd5cb4ae6ccd46'}}, 'owner': 'scheduled_skanta@teuthology', 'priority': 99, 'repo': 'https://github.com/ceph/ceph-ci.git', 'roles': [['mon.a', 'mon.b', 'mon.c', 'mgr.x', 'osd.0', 'osd.1', 'osd.2']], 'seed': 5245, 'sha1': 'bad1f558c327094897e4a32e99dd5cb4ae6ccd46', 'sleep_before_teardown': 0, 'subset': '111/120000', 'suite': 'rados', 'suite_branch': 'wip-bharath7-testing-2024-09-10-1409', 'suite_path': '/home/teuthworker/src/github.com_ceph_ceph-c_bad1f558c327094897e4a32e99dd5cb4ae6ccd46/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/ceph/ceph-ci.git', 'suite_sha1': 'bad1f558c327094897e4a32e99dd5cb4ae6ccd46', 'targets': {'smithi115.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAbn0gFLrlXEllx0KRe2j0SykP79/6lDKSBUs3JykdjnpNbpBuhOftUWXzM7lVw8JebkeQkBHLUS6R+mREJfCbI='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': None}, {'ceph': {'log-ignorelist': ['objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(OBJECT_', '\\(SLOW_OPS\\)', 'slow request', '\\(POOL_APP_NOT_ENABLED\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)'], 'pre-mgr-commands': ['sudo ceph config set mgr mgr_pool false --force'], 'conf': {'global': {'mon election default strategy': 1, 'ms bind msgr1': False, 'ms bind msgr2': True, 'ms type': 'async'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon scrub interval': 300}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'stupid', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'bluestore write v2': False, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd mclock override recovery settings': True, 'osd mclock profile': 'high_recovery_ops', 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random'}}, 'flavor': 'default', 'fs': 'xfs', 'sha1': 'bad1f558c327094897e4a32e99dd5cb4ae6ccd46', 'cluster': 'ceph'}}, {'lost_unfound': None}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': '3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a', 'timestamp': '2024-09-10_15:30:27', 'tube': 'smithi', 'user': 'skanta', 'verbose': True, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856'} 2024-09-10T15:56:49.804 DEBUG:teuthology.orchestra.run.smithi115:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-10T15:56:49.851 DEBUG:teuthology.orchestra.run.smithi115:> sudo install -d -m0777 -- /var/run/ceph 2024-09-10T15:56:49.916 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-09-10T15:56:49.916 DEBUG:teuthology.orchestra.run.smithi115:> dd if=/scratch_devs of=/dev/stdout 2024-09-10T15:56:49.972 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-10T15:56:49.972 DEBUG:teuthology.orchestra.run.smithi115:> stat /dev/vg_nvme/lv_1 2024-09-10T15:56:50.028 INFO:teuthology.orchestra.run.smithi115.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-10T15:56:50.028 INFO:teuthology.orchestra.run.smithi115.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T15:56:50.028 INFO:teuthology.orchestra.run.smithi115.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-09-10T15:56:50.028 INFO:teuthology.orchestra.run.smithi115.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T15:56:50.028 INFO:teuthology.orchestra.run.smithi115.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T15:56:50.028 INFO:teuthology.orchestra.run.smithi115.stdout:Access: 2024-09-10 15:56:45.736793892 +0000 2024-09-10T15:56:50.028 INFO:teuthology.orchestra.run.smithi115.stdout:Modify: 2024-09-10 15:54:13.385817541 +0000 2024-09-10T15:56:50.028 INFO:teuthology.orchestra.run.smithi115.stdout:Change: 2024-09-10 15:54:13.385817541 +0000 2024-09-10T15:56:50.028 INFO:teuthology.orchestra.run.smithi115.stdout: Birth: 2024-09-10 15:54:13.385817541 +0000 2024-09-10T15:56:50.029 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-10T15:56:50.091 INFO:teuthology.orchestra.run.smithi115.stderr:1+0 records in 2024-09-10T15:56:50.091 INFO:teuthology.orchestra.run.smithi115.stderr:1+0 records out 2024-09-10T15:56:50.091 INFO:teuthology.orchestra.run.smithi115.stderr:512 bytes copied, 0.000202015 s, 2.5 MB/s 2024-09-10T15:56:50.092 DEBUG:teuthology.orchestra.run.smithi115:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-10T15:56:50.148 DEBUG:teuthology.orchestra.run.smithi115:> stat /dev/vg_nvme/lv_2 2024-09-10T15:56:50.203 INFO:teuthology.orchestra.run.smithi115.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-10T15:56:50.204 INFO:teuthology.orchestra.run.smithi115.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T15:56:50.204 INFO:teuthology.orchestra.run.smithi115.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-09-10T15:56:50.204 INFO:teuthology.orchestra.run.smithi115.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T15:56:50.204 INFO:teuthology.orchestra.run.smithi115.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T15:56:50.204 INFO:teuthology.orchestra.run.smithi115.stdout:Access: 2024-09-10 15:56:45.736793892 +0000 2024-09-10T15:56:50.204 INFO:teuthology.orchestra.run.smithi115.stdout:Modify: 2024-09-10 15:54:13.722811985 +0000 2024-09-10T15:56:50.204 INFO:teuthology.orchestra.run.smithi115.stdout:Change: 2024-09-10 15:54:13.722811985 +0000 2024-09-10T15:56:50.204 INFO:teuthology.orchestra.run.smithi115.stdout: Birth: 2024-09-10 15:54:13.722811985 +0000 2024-09-10T15:56:50.204 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-10T15:56:50.266 INFO:teuthology.orchestra.run.smithi115.stderr:1+0 records in 2024-09-10T15:56:50.266 INFO:teuthology.orchestra.run.smithi115.stderr:1+0 records out 2024-09-10T15:56:50.267 INFO:teuthology.orchestra.run.smithi115.stderr:512 bytes copied, 0.000207305 s, 2.5 MB/s 2024-09-10T15:56:50.268 DEBUG:teuthology.orchestra.run.smithi115:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-10T15:56:50.324 DEBUG:teuthology.orchestra.run.smithi115:> stat /dev/vg_nvme/lv_3 2024-09-10T15:56:50.380 INFO:teuthology.orchestra.run.smithi115.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-10T15:56:50.380 INFO:teuthology.orchestra.run.smithi115.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T15:56:50.380 INFO:teuthology.orchestra.run.smithi115.stdout:Device: 5h/5d Inode: 705 Links: 1 2024-09-10T15:56:50.380 INFO:teuthology.orchestra.run.smithi115.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T15:56:50.380 INFO:teuthology.orchestra.run.smithi115.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T15:56:50.380 INFO:teuthology.orchestra.run.smithi115.stdout:Access: 2024-09-10 15:56:45.737793874 +0000 2024-09-10T15:56:50.380 INFO:teuthology.orchestra.run.smithi115.stdout:Modify: 2024-09-10 15:54:14.054806511 +0000 2024-09-10T15:56:50.380 INFO:teuthology.orchestra.run.smithi115.stdout:Change: 2024-09-10 15:54:14.054806511 +0000 2024-09-10T15:56:50.380 INFO:teuthology.orchestra.run.smithi115.stdout: Birth: 2024-09-10 15:54:14.054806511 +0000 2024-09-10T15:56:50.381 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-10T15:56:50.443 INFO:teuthology.orchestra.run.smithi115.stderr:1+0 records in 2024-09-10T15:56:50.443 INFO:teuthology.orchestra.run.smithi115.stderr:1+0 records out 2024-09-10T15:56:50.443 INFO:teuthology.orchestra.run.smithi115.stderr:512 bytes copied, 0.000241065 s, 2.1 MB/s 2024-09-10T15:56:50.444 DEBUG:teuthology.orchestra.run.smithi115:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-10T15:56:50.500 DEBUG:teuthology.orchestra.run.smithi115:> stat /dev/vg_nvme/lv_4 2024-09-10T15:56:50.556 INFO:teuthology.orchestra.run.smithi115.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-10T15:56:50.556 INFO:teuthology.orchestra.run.smithi115.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T15:56:50.556 INFO:teuthology.orchestra.run.smithi115.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-09-10T15:56:50.556 INFO:teuthology.orchestra.run.smithi115.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T15:56:50.556 INFO:teuthology.orchestra.run.smithi115.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T15:56:50.556 INFO:teuthology.orchestra.run.smithi115.stdout:Access: 2024-09-10 15:56:45.737793874 +0000 2024-09-10T15:56:50.556 INFO:teuthology.orchestra.run.smithi115.stdout:Modify: 2024-09-10 15:54:14.381801120 +0000 2024-09-10T15:56:50.556 INFO:teuthology.orchestra.run.smithi115.stdout:Change: 2024-09-10 15:54:14.381801120 +0000 2024-09-10T15:56:50.556 INFO:teuthology.orchestra.run.smithi115.stdout: Birth: 2024-09-10 15:54:14.381801120 +0000 2024-09-10T15:56:50.557 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-10T15:56:50.619 INFO:teuthology.orchestra.run.smithi115.stderr:1+0 records in 2024-09-10T15:56:50.620 INFO:teuthology.orchestra.run.smithi115.stderr:1+0 records out 2024-09-10T15:56:50.620 INFO:teuthology.orchestra.run.smithi115.stderr:512 bytes copied, 0.000206559 s, 2.5 MB/s 2024-09-10T15:56:50.621 DEBUG:teuthology.orchestra.run.smithi115:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-10T15:56:50.677 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2024-09-10T15:56:50.678 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi115.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'}} 2024-09-10T15:56:50.678 INFO:tasks.ceph:Generating config... 2024-09-10T15:56:50.679 INFO:tasks.ceph:[global] mon election default strategy = 1 2024-09-10T15:56:50.679 INFO:tasks.ceph:[global] ms bind msgr1 = False 2024-09-10T15:56:50.679 INFO:tasks.ceph:[global] ms bind msgr2 = True 2024-09-10T15:56:50.679 INFO:tasks.ceph:[global] ms type = async 2024-09-10T15:56:50.679 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-09-10T15:56:50.679 INFO:tasks.ceph:[mgr] debug ms = 1 2024-09-10T15:56:50.679 INFO:tasks.ceph:[mon] debug mon = 20 2024-09-10T15:56:50.679 INFO:tasks.ceph:[mon] debug ms = 1 2024-09-10T15:56:50.680 INFO:tasks.ceph:[mon] debug paxos = 20 2024-09-10T15:56:50.680 INFO:tasks.ceph:[mon] mon scrub interval = 300 2024-09-10T15:56:50.680 INFO:tasks.ceph:[osd] bdev async discard = True 2024-09-10T15:56:50.680 INFO:tasks.ceph:[osd] bdev enable discard = True 2024-09-10T15:56:50.680 INFO:tasks.ceph:[osd] bluestore allocator = stupid 2024-09-10T15:56:50.680 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-09-10T15:56:50.680 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-09-10T15:56:50.680 INFO:tasks.ceph:[osd] bluestore write v2 = False 2024-09-10T15:56:50.680 INFO:tasks.ceph:[osd] bluestore zero block detection = True 2024-09-10T15:56:50.680 INFO:tasks.ceph:[osd] debug bluefs = 20 2024-09-10T15:56:50.680 INFO:tasks.ceph:[osd] debug bluestore = 20 2024-09-10T15:56:50.680 INFO:tasks.ceph:[osd] debug ms = 1 2024-09-10T15:56:50.680 INFO:tasks.ceph:[osd] debug osd = 20 2024-09-10T15:56:50.681 INFO:tasks.ceph:[osd] debug rocksdb = 10 2024-09-10T15:56:50.681 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-09-10T15:56:50.681 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-09-10T15:56:50.681 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-09-10T15:56:50.681 INFO:tasks.ceph:[osd] osd debug verify cached snaps = True 2024-09-10T15:56:50.681 INFO:tasks.ceph:[osd] osd debug verify missing on start = True 2024-09-10T15:56:50.681 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-09-10T15:56:50.681 INFO:tasks.ceph:[osd] osd mclock override recovery settings = True 2024-09-10T15:56:50.681 INFO:tasks.ceph:[osd] osd mclock profile = high_recovery_ops 2024-09-10T15:56:50.681 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-09-10T15:56:50.681 INFO:tasks.ceph:[osd] osd op queue = debug_random 2024-09-10T15:56:50.681 INFO:tasks.ceph:[osd] osd op queue cut off = debug_random 2024-09-10T15:56:50.681 INFO:tasks.ceph:Setting up mon.a... 2024-09-10T15:56:50.682 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-09-10T15:56:50.773 INFO:teuthology.orchestra.run.smithi115.stdout:creating /etc/ceph/ceph.keyring 2024-09-10T15:56:50.775 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-09-10T15:56:50.817 DEBUG:teuthology.orchestra.run.smithi115:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-10T15:56:50.882 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.115'), ('mon.b', '[v2:172.21.15.115:3301,v1:172.21.15.115:6790]'), ('mon.c', '[v2:172.21.15.115:3302,v1:172.21.15.115:6791]')] 2024-09-10T15:56:50.882 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.115,[v2:172.21.15.115:3301,v1:172.21.15.115:6790],[v2:172.21.15.115:3302,v1:172.21.15.115:6791]', 'mon election default strategy': 1, 'ms bind msgr1': False, 'ms bind msgr2': True, 'ms type': 'async'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'stupid', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'bluestore write v2': False, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd mclock override recovery settings': True, 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random'}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon scrub interval': 300}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok'}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}} 2024-09-10T15:56:50.883 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-09-10T15:56:50.883 DEBUG:teuthology.orchestra.run.smithi115:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-09-10T15:56:50.938 DEBUG:teuthology.orchestra.run.smithi115:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.115 --addv b '[v2:172.21.15.115:3301,v1:172.21.15.115:6790]' --addv c '[v2:172.21.15.115:3302,v1:172.21.15.115:6791]' --print /home/ubuntu/cephtest/ceph.monmap 2024-09-10T15:56:51.025 INFO:teuthology.orchestra.run.smithi115.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-09-10T15:56:51.025 INFO:teuthology.orchestra.run.smithi115.stdout:monmaptool: generated fsid 7effee29-df05-4351-9741-ee203564ed81 2024-09-10T15:56:51.025 INFO:teuthology.orchestra.run.smithi115.stdout:setting min_mon_release = quincy 2024-09-10T15:56:51.025 INFO:teuthology.orchestra.run.smithi115.stdout:epoch 0 2024-09-10T15:56:51.025 INFO:teuthology.orchestra.run.smithi115.stdout:fsid 7effee29-df05-4351-9741-ee203564ed81 2024-09-10T15:56:51.026 INFO:teuthology.orchestra.run.smithi115.stdout:last_changed 2024-09-10T15:56:51.024669+0000 2024-09-10T15:56:51.026 INFO:teuthology.orchestra.run.smithi115.stdout:created 2024-09-10T15:56:51.024669+0000 2024-09-10T15:56:51.026 INFO:teuthology.orchestra.run.smithi115.stdout:min_mon_release 17 (quincy) 2024-09-10T15:56:51.026 INFO:teuthology.orchestra.run.smithi115.stdout:election_strategy: 1 2024-09-10T15:56:51.026 INFO:teuthology.orchestra.run.smithi115.stdout:0: [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] mon.a 2024-09-10T15:56:51.026 INFO:teuthology.orchestra.run.smithi115.stdout:1: [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] mon.b 2024-09-10T15:56:51.026 INFO:teuthology.orchestra.run.smithi115.stdout:2: [v2:172.21.15.115:3302/0,v1:172.21.15.115:6791/0] mon.c 2024-09-10T15:56:51.026 INFO:teuthology.orchestra.run.smithi115.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-09-10T15:56:51.027 DEBUG:teuthology.orchestra.run.smithi115:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-09-10T15:56:51.042 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 7effee29-df05-4351-9741-ee203564ed81... 2024-09-10T15:56:51.043 DEBUG:teuthology.orchestra.run.smithi115:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-10T15:56:51.125 INFO:teuthology.orchestra.run.smithi115.stdout:[global] 2024-09-10T15:56:51.125 INFO:teuthology.orchestra.run.smithi115.stdout: chdir = "" 2024-09-10T15:56:51.125 INFO:teuthology.orchestra.run.smithi115.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-10T15:56:51.125 INFO:teuthology.orchestra.run.smithi115.stdout: auth supported = cephx 2024-09-10T15:56:51.125 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T15:56:51.125 INFO:teuthology.orchestra.run.smithi115.stdout: filestore xattr use omap = true 2024-09-10T15:56:51.125 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T15:56:51.125 INFO:teuthology.orchestra.run.smithi115.stdout: mon clock drift allowed = 1.000 2024-09-10T15:56:51.125 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T15:56:51.125 INFO:teuthology.orchestra.run.smithi115.stdout: osd crush chooseleaf type = 0 2024-09-10T15:56:51.126 INFO:teuthology.orchestra.run.smithi115.stdout: auth debug = true 2024-09-10T15:56:51.126 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T15:56:51.126 INFO:teuthology.orchestra.run.smithi115.stdout: ms die on old message = true 2024-09-10T15:56:51.126 INFO:teuthology.orchestra.run.smithi115.stdout: ms die on bug = true 2024-09-10T15:56:51.126 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T15:56:51.126 INFO:teuthology.orchestra.run.smithi115.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-10T15:56:51.126 INFO:teuthology.orchestra.run.smithi115.stdout: mon pg warn max object skew = 0 2024-09-10T15:56:51.126 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T15:56:51.126 INFO:teuthology.orchestra.run.smithi115.stdout: # disable pg_autoscaler by default for new pools 2024-09-10T15:56:51.126 INFO:teuthology.orchestra.run.smithi115.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-10T15:56:51.126 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T15:56:51.126 INFO:teuthology.orchestra.run.smithi115.stdout: osd pool default size = 2 2024-09-10T15:56:51.126 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T15:56:51.126 INFO:teuthology.orchestra.run.smithi115.stdout: mon osd allow primary affinity = true 2024-09-10T15:56:51.127 INFO:teuthology.orchestra.run.smithi115.stdout: mon osd allow pg remap = true 2024-09-10T15:56:51.127 INFO:teuthology.orchestra.run.smithi115.stdout: mon warn on legacy crush tunables = false 2024-09-10T15:56:51.127 INFO:teuthology.orchestra.run.smithi115.stdout: mon warn on crush straw calc version zero = false 2024-09-10T15:56:51.127 INFO:teuthology.orchestra.run.smithi115.stdout: mon warn on no sortbitwise = false 2024-09-10T15:56:51.127 INFO:teuthology.orchestra.run.smithi115.stdout: mon warn on osd down out interval zero = false 2024-09-10T15:56:51.127 INFO:teuthology.orchestra.run.smithi115.stdout: mon warn on too few osds = false 2024-09-10T15:56:51.127 INFO:teuthology.orchestra.run.smithi115.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-10T15:56:51.127 INFO:teuthology.orchestra.run.smithi115.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-10T15:56:51.127 INFO:teuthology.orchestra.run.smithi115.stdout: mon_allow_pool_size_one = true 2024-09-10T15:56:51.127 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T15:56:51.127 INFO:teuthology.orchestra.run.smithi115.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-10T15:56:51.127 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T15:56:51.127 INFO:teuthology.orchestra.run.smithi115.stdout: osd default data pool replay window = 5 2024-09-10T15:56:51.127 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T15:56:51.128 INFO:teuthology.orchestra.run.smithi115.stdout: mon allow pool delete = true 2024-09-10T15:56:51.128 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T15:56:51.128 INFO:teuthology.orchestra.run.smithi115.stdout: mon cluster log file level = debug 2024-09-10T15:56:51.128 INFO:teuthology.orchestra.run.smithi115.stdout: debug asserts on shutdown = true 2024-09-10T15:56:51.128 INFO:teuthology.orchestra.run.smithi115.stdout: mon health detail to clog = false 2024-09-10T15:56:51.128 INFO:teuthology.orchestra.run.smithi115.stdout: mon host = "172.21.15.115,[v2:172.21.15.115:3301,v1:172.21.15.115:6790],[v2:172.21.15.115:3302,v1:172.21.15.115:6791]" 2024-09-10T15:56:51.128 INFO:teuthology.orchestra.run.smithi115.stdout: mon election default strategy = 1 2024-09-10T15:56:51.128 INFO:teuthology.orchestra.run.smithi115.stdout: ms bind msgr1 = False 2024-09-10T15:56:51.128 INFO:teuthology.orchestra.run.smithi115.stdout: ms bind msgr2 = True 2024-09-10T15:56:51.128 INFO:teuthology.orchestra.run.smithi115.stdout: ms type = async 2024-09-10T15:56:51.128 INFO:teuthology.orchestra.run.smithi115.stdout: fsid = 7effee29-df05-4351-9741-ee203564ed81 2024-09-10T15:56:51.128 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T15:56:51.128 INFO:teuthology.orchestra.run.smithi115.stdout:[osd] 2024-09-10T15:56:51.129 INFO:teuthology.orchestra.run.smithi115.stdout: osd journal size = 100 2024-09-10T15:56:51.129 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T15:56:51.129 INFO:teuthology.orchestra.run.smithi115.stdout: osd scrub load threshold = 5.0 2024-09-10T15:56:51.129 INFO:teuthology.orchestra.run.smithi115.stdout: osd scrub max interval = 600 2024-09-10T15:56:51.129 INFO:teuthology.orchestra.run.smithi115.stdout: osd mclock profile = high_recovery_ops 2024-09-10T15:56:51.129 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T15:56:51.129 INFO:teuthology.orchestra.run.smithi115.stdout: osd recover clone overlap = true 2024-09-10T15:56:51.129 INFO:teuthology.orchestra.run.smithi115.stdout: osd recovery max chunk = 1048576 2024-09-10T15:56:51.129 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T15:56:51.129 INFO:teuthology.orchestra.run.smithi115.stdout: osd debug shutdown = true 2024-09-10T15:56:51.129 INFO:teuthology.orchestra.run.smithi115.stdout: osd debug op order = true 2024-09-10T15:56:51.129 INFO:teuthology.orchestra.run.smithi115.stdout: osd debug verify stray on activate = true 2024-09-10T15:56:51.129 INFO:teuthology.orchestra.run.smithi115.stdout: osd debug trim objects = true 2024-09-10T15:56:51.129 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T15:56:51.129 INFO:teuthology.orchestra.run.smithi115.stdout: osd open classes on start = true 2024-09-10T15:56:51.130 INFO:teuthology.orchestra.run.smithi115.stdout: osd debug pg log writeout = true 2024-09-10T15:56:51.130 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T15:56:51.130 INFO:teuthology.orchestra.run.smithi115.stdout: osd deep scrub update digest min age = 30 2024-09-10T15:56:51.130 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T15:56:51.130 INFO:teuthology.orchestra.run.smithi115.stdout: osd map max advance = 10 2024-09-10T15:56:51.130 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T15:56:51.130 INFO:teuthology.orchestra.run.smithi115.stdout: journal zero on create = true 2024-09-10T15:56:51.130 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T15:56:51.130 INFO:teuthology.orchestra.run.smithi115.stdout: filestore ondisk finisher threads = 3 2024-09-10T15:56:51.130 INFO:teuthology.orchestra.run.smithi115.stdout: filestore apply finisher threads = 3 2024-09-10T15:56:51.130 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T15:56:51.130 INFO:teuthology.orchestra.run.smithi115.stdout: bdev debug aio = true 2024-09-10T15:56:51.130 INFO:teuthology.orchestra.run.smithi115.stdout: osd debug misdirected ops = true 2024-09-10T15:56:51.130 INFO:teuthology.orchestra.run.smithi115.stdout: bdev async discard = True 2024-09-10T15:56:51.131 INFO:teuthology.orchestra.run.smithi115.stdout: bdev enable discard = True 2024-09-10T15:56:51.131 INFO:teuthology.orchestra.run.smithi115.stdout: bluestore allocator = stupid 2024-09-10T15:56:51.131 INFO:teuthology.orchestra.run.smithi115.stdout: bluestore block size = 96636764160 2024-09-10T15:56:51.131 INFO:teuthology.orchestra.run.smithi115.stdout: bluestore fsck on mount = True 2024-09-10T15:56:51.131 INFO:teuthology.orchestra.run.smithi115.stdout: bluestore write v2 = False 2024-09-10T15:56:51.131 INFO:teuthology.orchestra.run.smithi115.stdout: bluestore zero block detection = True 2024-09-10T15:56:51.131 INFO:teuthology.orchestra.run.smithi115.stdout: debug bluefs = 20 2024-09-10T15:56:51.131 INFO:teuthology.orchestra.run.smithi115.stdout: debug bluestore = 20 2024-09-10T15:56:51.131 INFO:teuthology.orchestra.run.smithi115.stdout: debug ms = 1 2024-09-10T15:56:51.131 INFO:teuthology.orchestra.run.smithi115.stdout: debug osd = 20 2024-09-10T15:56:51.131 INFO:teuthology.orchestra.run.smithi115.stdout: debug rocksdb = 10 2024-09-10T15:56:51.131 INFO:teuthology.orchestra.run.smithi115.stdout: mon osd backfillfull_ratio = 0.85 2024-09-10T15:56:51.131 INFO:teuthology.orchestra.run.smithi115.stdout: mon osd full ratio = 0.9 2024-09-10T15:56:51.131 INFO:teuthology.orchestra.run.smithi115.stdout: mon osd nearfull ratio = 0.8 2024-09-10T15:56:51.131 INFO:teuthology.orchestra.run.smithi115.stdout: osd debug verify cached snaps = True 2024-09-10T15:56:51.132 INFO:teuthology.orchestra.run.smithi115.stdout: osd debug verify missing on start = True 2024-09-10T15:56:51.132 INFO:teuthology.orchestra.run.smithi115.stdout: osd failsafe full ratio = 0.95 2024-09-10T15:56:51.132 INFO:teuthology.orchestra.run.smithi115.stdout: osd mclock override recovery settings = True 2024-09-10T15:56:51.132 INFO:teuthology.orchestra.run.smithi115.stdout: osd objectstore = bluestore 2024-09-10T15:56:51.132 INFO:teuthology.orchestra.run.smithi115.stdout: osd op queue = debug_random 2024-09-10T15:56:51.132 INFO:teuthology.orchestra.run.smithi115.stdout: osd op queue cut off = debug_random 2024-09-10T15:56:51.132 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T15:56:51.132 INFO:teuthology.orchestra.run.smithi115.stdout:[mgr] 2024-09-10T15:56:51.132 INFO:teuthology.orchestra.run.smithi115.stdout: debug ms = 1 2024-09-10T15:56:51.132 INFO:teuthology.orchestra.run.smithi115.stdout: debug mgr = 20 2024-09-10T15:56:51.132 INFO:teuthology.orchestra.run.smithi115.stdout: debug mon = 20 2024-09-10T15:56:51.132 INFO:teuthology.orchestra.run.smithi115.stdout: debug auth = 20 2024-09-10T15:56:51.132 INFO:teuthology.orchestra.run.smithi115.stdout: mon reweight min pgs per osd = 4 2024-09-10T15:56:51.132 INFO:teuthology.orchestra.run.smithi115.stdout: mon reweight min bytes per osd = 10 2024-09-10T15:56:51.132 INFO:teuthology.orchestra.run.smithi115.stdout: mgr/telemetry/nag = false 2024-09-10T15:56:51.133 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T15:56:51.133 INFO:teuthology.orchestra.run.smithi115.stdout:[mon] 2024-09-10T15:56:51.133 INFO:teuthology.orchestra.run.smithi115.stdout: debug ms = 1 2024-09-10T15:56:51.133 INFO:teuthology.orchestra.run.smithi115.stdout: debug mon = 20 2024-09-10T15:56:51.133 INFO:teuthology.orchestra.run.smithi115.stdout: debug paxos = 20 2024-09-10T15:56:51.133 INFO:teuthology.orchestra.run.smithi115.stdout: debug auth = 20 2024-09-10T15:56:51.133 INFO:teuthology.orchestra.run.smithi115.stdout: mon data avail warn = 5 2024-09-10T15:56:51.133 INFO:teuthology.orchestra.run.smithi115.stdout: mon mgr mkfs grace = 240 2024-09-10T15:56:51.133 INFO:teuthology.orchestra.run.smithi115.stdout: mon reweight min pgs per osd = 4 2024-09-10T15:56:51.133 INFO:teuthology.orchestra.run.smithi115.stdout: mon osd reporter subtree level = osd 2024-09-10T15:56:51.133 INFO:teuthology.orchestra.run.smithi115.stdout: mon osd prime pg temp = true 2024-09-10T15:56:51.133 INFO:teuthology.orchestra.run.smithi115.stdout: mon reweight min bytes per osd = 10 2024-09-10T15:56:51.133 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T15:56:51.133 INFO:teuthology.orchestra.run.smithi115.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-10T15:56:51.134 INFO:teuthology.orchestra.run.smithi115.stdout: auth mon ticket ttl = 660 # 11m 2024-09-10T15:56:51.134 INFO:teuthology.orchestra.run.smithi115.stdout: auth service ticket ttl = 240 # 4m 2024-09-10T15:56:51.134 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T15:56:51.134 INFO:teuthology.orchestra.run.smithi115.stdout: # don't complain about insecure global_id in the test suite 2024-09-10T15:56:51.134 INFO:teuthology.orchestra.run.smithi115.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-10T15:56:51.134 INFO:teuthology.orchestra.run.smithi115.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-10T15:56:51.134 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T15:56:51.134 INFO:teuthology.orchestra.run.smithi115.stdout: # 1m isn't quite enough 2024-09-10T15:56:51.134 INFO:teuthology.orchestra.run.smithi115.stdout: mon_down_mkfs_grace = 2m 2024-09-10T15:56:51.134 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T15:56:51.134 INFO:teuthology.orchestra.run.smithi115.stdout: mon_warn_on_filestore_osds = false 2024-09-10T15:56:51.134 INFO:teuthology.orchestra.run.smithi115.stdout: mon scrub interval = 300 2024-09-10T15:56:51.134 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T15:56:51.134 INFO:teuthology.orchestra.run.smithi115.stdout:[client] 2024-09-10T15:56:51.134 INFO:teuthology.orchestra.run.smithi115.stdout: rgw cache enabled = true 2024-09-10T15:56:51.135 INFO:teuthology.orchestra.run.smithi115.stdout: rgw enable ops log = true 2024-09-10T15:56:51.135 INFO:teuthology.orchestra.run.smithi115.stdout: rgw enable usage log = true 2024-09-10T15:56:51.135 INFO:teuthology.orchestra.run.smithi115.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-10T15:56:51.135 INFO:teuthology.orchestra.run.smithi115.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-10T15:56:51.135 INFO:teuthology.orchestra.run.smithi115.stdout:[mon.a] 2024-09-10T15:56:51.135 INFO:teuthology.orchestra.run.smithi115.stdout:[mon.b] 2024-09-10T15:56:51.135 INFO:teuthology.orchestra.run.smithi115.stdout:[mon.c] 2024-09-10T15:56:51.136 INFO:tasks.ceph:Creating admin key on mon.a... 2024-09-10T15:56:51.136 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-09-10T15:56:51.231 INFO:tasks.ceph:Copying monmap to all nodes... 2024-09-10T15:56:51.231 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-09-10T15:56:51.231 DEBUG:teuthology.orchestra.run.smithi115:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-09-10T15:56:51.247 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-09-10T15:56:51.247 DEBUG:teuthology.orchestra.run.smithi115:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-09-10T15:56:51.303 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi115.front.sepia.ceph.com 2024-09-10T15:56:51.303 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-09-10T15:56:51.303 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-10T15:56:51.303 DEBUG:teuthology.orchestra.run.smithi115:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-10T15:56:51.378 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-09-10T15:56:51.378 DEBUG:teuthology.orchestra.run.smithi115:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-10T15:56:51.433 INFO:tasks.ceph:Setting up mon nodes... 2024-09-10T15:56:51.434 INFO:tasks.ceph:Setting up mgr nodes... 2024-09-10T15:56:51.434 DEBUG:teuthology.orchestra.run.smithi115:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-09-10T15:56:51.524 INFO:teuthology.orchestra.run.smithi115.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-09-10T15:56:51.527 INFO:tasks.ceph:Setting up mds nodes... 2024-09-10T15:56:51.527 INFO:tasks.ceph_client:Setting up client nodes... 2024-09-10T15:56:51.527 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-09-10T15:56:51.527 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi115.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'}} 2024-09-10T15:56:51.528 DEBUG:teuthology.orchestra.run.smithi115:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-09-10T15:56:51.552 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2024-09-10T15:56:51.553 INFO:tasks.ceph:role: osd.0 2024-09-10T15:56:51.553 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi115.front.sepia.ceph.com 2024-09-10T15:56:51.553 DEBUG:teuthology.orchestra.run.smithi115:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-10T15:56:51.624 INFO:teuthology.orchestra.run.smithi115.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-10T15:56:51.624 INFO:teuthology.orchestra.run.smithi115.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-10T15:56:51.624 INFO:teuthology.orchestra.run.smithi115.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-10T15:56:51.624 INFO:teuthology.orchestra.run.smithi115.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-10T15:56:51.624 INFO:teuthology.orchestra.run.smithi115.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-10T15:56:51.624 INFO:teuthology.orchestra.run.smithi115.stdout: = sunit=0 swidth=0 blks 2024-09-10T15:56:51.625 INFO:teuthology.orchestra.run.smithi115.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-10T15:56:51.625 INFO:teuthology.orchestra.run.smithi115.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-10T15:56:51.625 INFO:teuthology.orchestra.run.smithi115.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-10T15:56:51.625 INFO:teuthology.orchestra.run.smithi115.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-10T15:56:52.136 INFO:teuthology.orchestra.run.smithi115.stdout:Discarding blocks...Done. 2024-09-10T15:56:52.138 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi115.front.sepia.ceph.com -o noatime 2024-09-10T15:56:52.138 DEBUG:teuthology.orchestra.run.smithi115:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-09-10T15:56:52.283 DEBUG:teuthology.orchestra.run.smithi115:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-09-10T15:56:52.315 DEBUG:teuthology.orchestra.run.smithi115:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-09-10T15:56:52.381 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2024-09-10T15:56:52.381 INFO:tasks.ceph:role: osd.1 2024-09-10T15:56:52.382 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi115.front.sepia.ceph.com 2024-09-10T15:56:52.382 DEBUG:teuthology.orchestra.run.smithi115:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-10T15:56:52.455 INFO:teuthology.orchestra.run.smithi115.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-10T15:56:52.455 INFO:teuthology.orchestra.run.smithi115.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-10T15:56:52.455 INFO:teuthology.orchestra.run.smithi115.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-10T15:56:52.455 INFO:teuthology.orchestra.run.smithi115.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-10T15:56:52.456 INFO:teuthology.orchestra.run.smithi115.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-10T15:56:52.456 INFO:teuthology.orchestra.run.smithi115.stdout: = sunit=0 swidth=0 blks 2024-09-10T15:56:52.456 INFO:teuthology.orchestra.run.smithi115.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-10T15:56:52.456 INFO:teuthology.orchestra.run.smithi115.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-10T15:56:52.456 INFO:teuthology.orchestra.run.smithi115.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-10T15:56:52.456 INFO:teuthology.orchestra.run.smithi115.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-10T15:56:52.903 INFO:teuthology.orchestra.run.smithi115.stdout:Discarding blocks...Done. 2024-09-10T15:56:52.905 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi115.front.sepia.ceph.com -o noatime 2024-09-10T15:56:52.905 DEBUG:teuthology.orchestra.run.smithi115:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-09-10T15:56:52.999 DEBUG:teuthology.orchestra.run.smithi115:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-09-10T15:56:53.030 DEBUG:teuthology.orchestra.run.smithi115:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-09-10T15:56:53.095 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2024-09-10T15:56:53.096 INFO:tasks.ceph:role: osd.2 2024-09-10T15:56:53.096 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi115.front.sepia.ceph.com 2024-09-10T15:56:53.096 DEBUG:teuthology.orchestra.run.smithi115:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-10T15:56:53.168 INFO:teuthology.orchestra.run.smithi115.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-10T15:56:53.168 INFO:teuthology.orchestra.run.smithi115.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-10T15:56:53.168 INFO:teuthology.orchestra.run.smithi115.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-10T15:56:53.168 INFO:teuthology.orchestra.run.smithi115.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-10T15:56:53.168 INFO:teuthology.orchestra.run.smithi115.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-10T15:56:53.168 INFO:teuthology.orchestra.run.smithi115.stdout: = sunit=0 swidth=0 blks 2024-09-10T15:56:53.168 INFO:teuthology.orchestra.run.smithi115.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-10T15:56:53.168 INFO:teuthology.orchestra.run.smithi115.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-10T15:56:53.168 INFO:teuthology.orchestra.run.smithi115.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-10T15:56:53.169 INFO:teuthology.orchestra.run.smithi115.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-10T15:56:53.663 INFO:teuthology.orchestra.run.smithi115.stdout:Discarding blocks...Done. 2024-09-10T15:56:53.665 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi115.front.sepia.ceph.com -o noatime 2024-09-10T15:56:53.666 DEBUG:teuthology.orchestra.run.smithi115:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-09-10T15:56:53.807 DEBUG:teuthology.orchestra.run.smithi115:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-09-10T15:56:53.837 DEBUG:teuthology.orchestra.run.smithi115:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-10T15:56:53.975 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-10T15:56:53.973+0000 7f72fafbd640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-09-10T15:56:53.980 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-10T15:56:53.978+0000 7f72fafbd640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-09-10T15:56:53.980 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-10T15:56:53.978+0000 7f72fafbd640 -1 bdev(0x5600931f9800 /var/lib/ceph/osd/ceph-0/block) open stat got: (1) Operation not permitted 2024-09-10T15:56:53.980 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-10T15:56:53.978+0000 7f72fafbd640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-09-10T15:56:57.136 DEBUG:teuthology.orchestra.run.smithi115:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-09-10T15:56:57.166 DEBUG:teuthology.orchestra.run.smithi115:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-10T15:56:57.257 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-10T15:56:57.255+0000 7fc1a19c1640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-09-10T15:56:57.257 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-10T15:56:57.256+0000 7fc1a19c1640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-09-10T15:56:57.258 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-10T15:56:57.256+0000 7fc1a19c1640 -1 bdev(0x560d03287800 /var/lib/ceph/osd/ceph-1/block) open stat got: (1) Operation not permitted 2024-09-10T15:56:57.258 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-10T15:56:57.256+0000 7fc1a19c1640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-09-10T15:57:00.418 DEBUG:teuthology.orchestra.run.smithi115:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-09-10T15:57:00.448 DEBUG:teuthology.orchestra.run.smithi115:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-10T15:57:00.539 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-10T15:57:00.537+0000 7f1020e19640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-09-10T15:57:00.539 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-10T15:57:00.537+0000 7f1020e19640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-09-10T15:57:00.539 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-10T15:57:00.537+0000 7f1020e19640 -1 bdev(0x55daff3e7800 /var/lib/ceph/osd/ceph-2/block) open stat got: (1) Operation not permitted 2024-09-10T15:57:00.539 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-10T15:57:00.538+0000 7f1020e19640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-09-10T15:57:03.452 DEBUG:teuthology.orchestra.run.smithi115:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-09-10T15:57:03.480 INFO:tasks.ceph:Reading keys from all nodes... 2024-09-10T15:57:03.480 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-09-10T15:57:03.480 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-09-10T15:57:03.547 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-09-10T15:57:03.547 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-09-10T15:57:03.611 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-09-10T15:57:03.611 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-09-10T15:57:03.676 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-09-10T15:57:03.676 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-09-10T15:57:03.741 INFO:tasks.ceph:Adding keys to all mons... 2024-09-10T15:57:03.741 DEBUG:teuthology.orchestra.run.smithi115:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-10T15:57:03.804 INFO:teuthology.orchestra.run.smithi115.stdout:[mgr.x] 2024-09-10T15:57:03.805 INFO:teuthology.orchestra.run.smithi115.stdout: key = AQBDbOBmH2I1HxAAKCQowVdoOU/ys+h4IVJMuQ== 2024-09-10T15:57:03.805 INFO:teuthology.orchestra.run.smithi115.stdout:[osd.0] 2024-09-10T15:57:03.805 INFO:teuthology.orchestra.run.smithi115.stdout: key = AQBFbOBmnBwcOhAA8McaLjFrXniQYx+1K276Sw== 2024-09-10T15:57:03.805 INFO:teuthology.orchestra.run.smithi115.stdout:[osd.1] 2024-09-10T15:57:03.805 INFO:teuthology.orchestra.run.smithi115.stdout: key = AQBJbOBmqKlODxAA6UZOGRmp4eBpxS+jS7pOSQ== 2024-09-10T15:57:03.805 INFO:teuthology.orchestra.run.smithi115.stdout:[osd.2] 2024-09-10T15:57:03.805 INFO:teuthology.orchestra.run.smithi115.stdout: key = AQBMbOBmzzQZIBAAAa9oAMJT3f/5mnCjUHQebQ== 2024-09-10T15:57:03.807 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-10T15:57:03.897 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-10T15:57:03.948 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-10T15:57:03.999 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-10T15:57:04.049 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-09-10T15:57:04.049 DEBUG:teuthology.orchestra.run.smithi115:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-09-10T15:57:04.073 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-10T15:57:04.168 DEBUG:teuthology.orchestra.run.smithi115:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-09-10T15:57:04.193 DEBUG:teuthology.orchestra.run.smithi115:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-09-10T15:57:04.258 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-10T15:57:04.356 DEBUG:teuthology.orchestra.run.smithi115:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-09-10T15:57:04.381 DEBUG:teuthology.orchestra.run.smithi115:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-09-10T15:57:04.446 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-10T15:57:04.543 DEBUG:teuthology.orchestra.run.smithi115:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-09-10T15:57:04.567 DEBUG:teuthology.orchestra.run.smithi115:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-10T15:57:04.622 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-09-10T15:57:04.623 INFO:tasks.ceph.mon.a:Restarting daemon 2024-09-10T15:57:04.623 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-09-10T15:57:04.665 INFO:tasks.ceph.mon.a:Started 2024-09-10T15:57:04.665 INFO:tasks.ceph.mon.b:Restarting daemon 2024-09-10T15:57:04.665 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-09-10T15:57:04.668 INFO:tasks.ceph.mon.b:Started 2024-09-10T15:57:04.668 INFO:tasks.ceph.mon.c:Restarting daemon 2024-09-10T15:57:04.668 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-09-10T15:57:04.671 INFO:tasks.ceph.mon.c:Started 2024-09-10T15:57:04.671 DEBUG:teuthology.orchestra.run.smithi115:> sudo ceph config set mgr mgr_pool false --force 2024-09-10T15:57:15.265 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-09-10T15:57:15.266 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-09-10T15:57:15.266 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-09-10T15:57:15.268 INFO:tasks.ceph.mgr.x:Started 2024-09-10T15:57:15.268 DEBUG:tasks.ceph:set 0 configs 2024-09-10T15:57:15.269 INFO:tasks.ceph:Setting crush tunables to default 2024-09-10T15:57:15.269 DEBUG:teuthology.orchestra.run.smithi115:> sudo ceph --cluster ceph osd crush tunables default 2024-09-10T15:57:15.444 INFO:teuthology.orchestra.run.smithi115.stderr:adjusted tunables profile to default 2024-09-10T15:57:15.456 INFO:tasks.ceph:check_enable_crimson: False 2024-09-10T15:57:15.456 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-09-10T15:57:15.456 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-09-10T15:57:15.456 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-09-10T15:57:15.481 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-09-10T15:57:15.481 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-09-10T15:57:15.546 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-09-10T15:57:15.547 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-09-10T15:57:15.611 DEBUG:teuthology.orchestra.run.smithi115:> sudo ceph --cluster ceph osd new 859508e0-e15d-4011-a39f-65e7aa04a129 0 2024-09-10T15:57:15.836 INFO:teuthology.orchestra.run.smithi115.stdout:0 2024-09-10T15:57:15.847 DEBUG:teuthology.orchestra.run.smithi115:> sudo ceph --cluster ceph osd new 66daed2f-a693-4a13-8d8c-024794a032e3 1 2024-09-10T15:57:16.028 INFO:teuthology.orchestra.run.smithi115.stdout:1 2024-09-10T15:57:16.039 DEBUG:teuthology.orchestra.run.smithi115:> sudo ceph --cluster ceph osd new 6a2ab153-d99e-4a5d-b79c-32b7f8ea9ffb 2 2024-09-10T15:57:16.216 INFO:teuthology.orchestra.run.smithi115.stdout:2 2024-09-10T15:57:16.227 INFO:tasks.ceph.osd.0:Restarting daemon 2024-09-10T15:57:16.227 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-09-10T15:57:16.230 INFO:tasks.ceph.osd.0:Started 2024-09-10T15:57:16.230 INFO:tasks.ceph.osd.1:Restarting daemon 2024-09-10T15:57:16.231 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-09-10T15:57:16.233 INFO:tasks.ceph.osd.1:Started 2024-09-10T15:57:16.233 INFO:tasks.ceph.osd.2:Restarting daemon 2024-09-10T15:57:16.233 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-09-10T15:57:16.235 INFO:tasks.ceph.osd.2:Started 2024-09-10T15:57:16.235 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T15:57:16.409 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T15:57:16.409 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":5,"fsid":"7effee29-df05-4351-9741-ee203564ed81","created":"2024-09-10T15:57:14.967498+0000","modified":"2024-09-10T15:57:16.214215+0000","last_up_change":"0.000000","last_in_change":"2024-09-10T15:57:16.214215+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"859508e0-e15d-4011-a39f-65e7aa04a129","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"66daed2f-a693-4a13-8d8c-024794a032e3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"6a2ab153-d99e-4a5d-b79c-32b7f8ea9ffb","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T15:57:16.421 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-09-10T15:57:16.421 INFO:tasks.ceph:Waiting for OSDs to come up 2024-09-10T15:57:16.723 DEBUG:teuthology.orchestra.run.smithi115:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-10T15:57:16.884 INFO:teuthology.misc.health.smithi115.stdout: 2024-09-10T15:57:16.884 INFO:teuthology.misc.health.smithi115.stdout:{"epoch":5,"fsid":"7effee29-df05-4351-9741-ee203564ed81","created":"2024-09-10T15:57:14.967498+0000","modified":"2024-09-10T15:57:16.214215+0000","last_up_change":"0.000000","last_in_change":"2024-09-10T15:57:16.214215+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"859508e0-e15d-4011-a39f-65e7aa04a129","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"66daed2f-a693-4a13-8d8c-024794a032e3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"6a2ab153-d99e-4a5d-b79c-32b7f8ea9ffb","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T15:57:16.894 DEBUG:teuthology.misc:0 of 3 OSDs are up 2024-09-10T15:57:17.737 INFO:tasks.ceph.osd.2.smithi115.stderr:2024-09-10T15:57:17.735+0000 7efe1d49c640 -1 Falling back to public interface 2024-09-10T15:57:17.971 INFO:tasks.ceph.osd.0.smithi115.stderr:2024-09-10T15:57:17.969+0000 7f1e27a1f640 -1 Falling back to public interface 2024-09-10T15:57:18.258 INFO:tasks.ceph.osd.1.smithi115.stderr:2024-09-10T15:57:18.256+0000 7f45c45a2640 -1 Falling back to public interface 2024-09-10T15:57:20.530 INFO:tasks.ceph.mgr.x.smithi115.stderr:2024-09-10T15:57:20.528+0000 7fca55346640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-09-10T15:57:20.530 INFO:tasks.ceph.mgr.x.smithi115.stderr:2024-09-10T15:57:20.528+0000 7fca55346640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-09-10T15:57:21.802 INFO:tasks.ceph.osd.2.smithi115.stderr:2024-09-10T15:57:21.800+0000 7efe1d49c640 -1 osd.2 0 log_to_monitors true 2024-09-10T15:57:22.053 INFO:tasks.ceph.osd.0.smithi115.stderr:2024-09-10T15:57:22.052+0000 7f1e27a1f640 -1 osd.0 0 log_to_monitors true 2024-09-10T15:57:22.560 INFO:tasks.ceph.osd.1.smithi115.stderr:2024-09-10T15:57:22.559+0000 7f45c45a2640 -1 osd.1 0 log_to_monitors true 2024-09-10T15:57:23.197 DEBUG:teuthology.orchestra.run.smithi115:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-10T15:57:23.482 INFO:teuthology.misc.health.smithi115.stdout: 2024-09-10T15:57:23.482 INFO:teuthology.misc.health.smithi115.stdout:{"epoch":6,"fsid":"7effee29-df05-4351-9741-ee203564ed81","created":"2024-09-10T15:57:14.967498+0000","modified":"2024-09-10T15:57:22.527872+0000","last_up_change":"0.000000","last_in_change":"2024-09-10T15:57:16.214215+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":3,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"859508e0-e15d-4011-a39f-65e7aa04a129","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"66daed2f-a693-4a13-8d8c-024794a032e3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"6a2ab153-d99e-4a5d-b79c-32b7f8ea9ffb","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T15:57:23.493 DEBUG:teuthology.misc:0 of 3 OSDs are up 2024-09-10T15:57:24.839 INFO:tasks.ceph.osd.2.smithi115.stderr:2024-09-10T15:57:24.837+0000 7efe19c3f640 -1 osd.2 0 waiting for initial osdmap 2024-09-10T15:57:24.847 INFO:tasks.ceph.osd.2.smithi115.stderr:2024-09-10T15:57:24.845+0000 7efe1124a640 -1 osd.2 8 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T15:57:24.912 INFO:tasks.ceph.osd.0.smithi115.stderr:2024-09-10T15:57:24.911+0000 7f1e233ec640 -1 osd.0 0 waiting for initial osdmap 2024-09-10T15:57:24.924 INFO:tasks.ceph.osd.0.smithi115.stderr:2024-09-10T15:57:24.921+0000 7f1e1b20b640 -1 osd.0 8 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T15:57:25.393 INFO:tasks.ceph.osd.1.smithi115.stderr:2024-09-10T15:57:25.392+0000 7f45c0531640 -1 osd.1 0 waiting for initial osdmap 2024-09-10T15:57:25.398 INFO:tasks.ceph.osd.1.smithi115.stderr:2024-09-10T15:57:25.397+0000 7f45b8350640 -1 osd.1 8 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T15:57:29.796 DEBUG:teuthology.orchestra.run.smithi115:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-10T15:57:30.083 INFO:teuthology.misc.health.smithi115.stdout: 2024-09-10T15:57:30.084 INFO:teuthology.misc.health.smithi115.stdout:{"epoch":10,"fsid":"7effee29-df05-4351-9741-ee203564ed81","created":"2024-09-10T15:57:14.967498+0000","modified":"2024-09-10T15:57:26.844403+0000","last_up_change":"2024-09-10T15:57:25.842984+0000","last_in_change":"2024-09-10T15:57:16.214215+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"859508e0-e15d-4011-a39f-65e7aa04a129","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":855194165}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6805","nonce":855194165}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6807","nonce":855194165}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":855194165}]},"public_addr":"172.21.15.115:6804/855194165","cluster_addr":"172.21.15.115:6805/855194165","heartbeat_back_addr":"172.21.15.115:6807/855194165","heartbeat_front_addr":"172.21.15.115:6806/855194165","state":["exists","up"]},{"osd":1,"uuid":"66daed2f-a693-4a13-8d8c-024794a032e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":2071140438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6809","nonce":2071140438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6811","nonce":2071140438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":2071140438}]},"public_addr":"172.21.15.115:6808/2071140438","cluster_addr":"172.21.15.115:6809/2071140438","heartbeat_back_addr":"172.21.15.115:6811/2071140438","heartbeat_front_addr":"172.21.15.115:6810/2071140438","state":["exists","up"]},{"osd":2,"uuid":"6a2ab153-d99e-4a5d-b79c-32b7f8ea9ffb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":267302962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6801","nonce":267302962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6803","nonce":267302962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":267302962}]},"public_addr":"172.21.15.115:6800/267302962","cluster_addr":"172.21.15.115:6801/267302962","heartbeat_back_addr":"172.21.15.115:6803/267302962","heartbeat_front_addr":"172.21.15.115:6802/267302962","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T15:57:23.007077+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T15:57:23.605294+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T15:57:22.852045+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T15:57:30.094 DEBUG:teuthology.misc:3 of 3 OSDs are up 2024-09-10T15:57:30.094 INFO:tasks.ceph:Creating RBD pool 2024-09-10T15:57:30.095 DEBUG:teuthology.orchestra.run.smithi115:> sudo ceph --cluster ceph osd pool create rbd 8 2024-09-10T15:57:30.860 INFO:teuthology.orchestra.run.smithi115.stderr:pool 'rbd' created 2024-09-10T15:57:30.874 DEBUG:teuthology.orchestra.run.smithi115:> rbd --cluster ceph pool init rbd 2024-09-10T15:57:33.897 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-09-10T15:57:33.898 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config log 1 --format=json 2024-09-10T15:57:33.899 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-09-10T15:57:34.192 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T15:57:34.204 INFO:teuthology.orchestra.run.smithi115.stdout:[{"version":5,"timestamp":"2024-09-10T15:57:25.393714+0000","name":"","changes":[{"name":"osd.1/osd_mclock_max_capacity_iops_ssd","new_value":"8832.955682"}]}] 2024-09-10T15:57:34.204 INFO:tasks.ceph_manager:config epoch is 5 2024-09-10T15:57:34.205 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-10T15:57:34.205 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-09-10T15:57:34.205 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-09-10T15:57:34.534 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T15:57:34.545 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":4,"flags":0,"active_gid":4107,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":3301897468}]},"active_addr":"172.21.15.115:6812/3301897468","active_change":"2024-09-10T15:57:19.519022+0000","active_mgr_features":4540719139924082687,"available":true,"standbys":[],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:1.26.1","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.17","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"oauth2-proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:latest","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB metrics exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.115:0","nonce":3005075743}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.115:0","nonce":2433121524}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.115:0","nonce":3231732937}]}]} 2024-09-10T15:57:34.550 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-09-10T15:57:34.550 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-10T15:57:34.550 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T15:57:34.837 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T15:57:34.837 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":14,"fsid":"7effee29-df05-4351-9741-ee203564ed81","created":"2024-09-10T15:57:14.967498+0000","modified":"2024-09-10T15:57:33.873157+0000","last_up_change":"2024-09-10T15:57:25.842984+0000","last_in_change":"2024-09-10T15:57:16.214215+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2024-09-10T15:57:30.381324+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":14,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"859508e0-e15d-4011-a39f-65e7aa04a129","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":11,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":855194165}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6805","nonce":855194165}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6807","nonce":855194165}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":855194165}]},"public_addr":"172.21.15.115:6804/855194165","cluster_addr":"172.21.15.115:6805/855194165","heartbeat_back_addr":"172.21.15.115:6807/855194165","heartbeat_front_addr":"172.21.15.115:6806/855194165","state":["exists","up"]},{"osd":1,"uuid":"66daed2f-a693-4a13-8d8c-024794a032e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":11,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":2071140438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6809","nonce":2071140438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6811","nonce":2071140438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":2071140438}]},"public_addr":"172.21.15.115:6808/2071140438","cluster_addr":"172.21.15.115:6809/2071140438","heartbeat_back_addr":"172.21.15.115:6811/2071140438","heartbeat_front_addr":"172.21.15.115:6810/2071140438","state":["exists","up"]},{"osd":2,"uuid":"6a2ab153-d99e-4a5d-b79c-32b7f8ea9ffb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":11,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":267302962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6801","nonce":267302962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6803","nonce":267302962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":267302962}]},"public_addr":"172.21.15.115:6800/267302962","cluster_addr":"172.21.15.115:6801/267302962","heartbeat_back_addr":"172.21.15.115:6803/267302962","heartbeat_front_addr":"172.21.15.115:6802/267302962","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T15:57:23.007077+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T15:57:23.605294+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T15:57:22.852045+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":1,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":1,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T15:57:34.849 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-10T15:57:34.849 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T15:57:35.134 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T15:57:35.134 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":14,"fsid":"7effee29-df05-4351-9741-ee203564ed81","created":"2024-09-10T15:57:14.967498+0000","modified":"2024-09-10T15:57:33.873157+0000","last_up_change":"2024-09-10T15:57:25.842984+0000","last_in_change":"2024-09-10T15:57:16.214215+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2024-09-10T15:57:30.381324+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":14,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"859508e0-e15d-4011-a39f-65e7aa04a129","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":11,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":855194165}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6805","nonce":855194165}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6807","nonce":855194165}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":855194165}]},"public_addr":"172.21.15.115:6804/855194165","cluster_addr":"172.21.15.115:6805/855194165","heartbeat_back_addr":"172.21.15.115:6807/855194165","heartbeat_front_addr":"172.21.15.115:6806/855194165","state":["exists","up"]},{"osd":1,"uuid":"66daed2f-a693-4a13-8d8c-024794a032e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":11,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":2071140438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6809","nonce":2071140438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6811","nonce":2071140438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":2071140438}]},"public_addr":"172.21.15.115:6808/2071140438","cluster_addr":"172.21.15.115:6809/2071140438","heartbeat_back_addr":"172.21.15.115:6811/2071140438","heartbeat_front_addr":"172.21.15.115:6810/2071140438","state":["exists","up"]},{"osd":2,"uuid":"6a2ab153-d99e-4a5d-b79c-32b7f8ea9ffb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":11,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":267302962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6801","nonce":267302962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6803","nonce":267302962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":267302962}]},"public_addr":"172.21.15.115:6800/267302962","cluster_addr":"172.21.15.115:6801/267302962","heartbeat_back_addr":"172.21.15.115:6803/267302962","heartbeat_front_addr":"172.21.15.115:6802/267302962","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T15:57:23.007077+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T15:57:23.605294+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T15:57:22.852045+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":1,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":1,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T15:57:35.147 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-10T15:57:35.147 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-10T15:57:35.148 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-10T15:57:35.283 INFO:teuthology.orchestra.run.smithi115.stdout:38654705669 2024-09-10T15:57:35.284 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-10T15:57:35.303 INFO:teuthology.orchestra.run.smithi115.stdout:38654705667 2024-09-10T15:57:35.303 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-10T15:57:35.304 INFO:teuthology.orchestra.run.smithi115.stdout:38654705669 2024-09-10T15:57:35.304 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-10T15:57:35.594 INFO:teuthology.orchestra.run.smithi115.stdout:38654705668 2024-09-10T15:57:35.608 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705669 got 38654705668 for osd.0 2024-09-10T15:57:35.645 INFO:teuthology.orchestra.run.smithi115.stdout:38654705666 2024-09-10T15:57:35.652 INFO:teuthology.orchestra.run.smithi115.stdout:38654705668 2024-09-10T15:57:35.658 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705667 got 38654705666 for osd.1 2024-09-10T15:57:35.665 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705669 got 38654705668 for osd.2 2024-09-10T15:57:36.609 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-10T15:57:36.659 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-10T15:57:36.666 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-10T15:57:36.921 INFO:teuthology.orchestra.run.smithi115.stdout:38654705669 2024-09-10T15:57:36.935 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705669 got 38654705669 for osd.0 2024-09-10T15:57:36.935 DEBUG:teuthology.parallel:result is None 2024-09-10T15:57:36.968 INFO:teuthology.orchestra.run.smithi115.stdout:38654705667 2024-09-10T15:57:36.980 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705667 got 38654705667 for osd.1 2024-09-10T15:57:36.981 DEBUG:teuthology.parallel:result is None 2024-09-10T15:57:37.004 INFO:teuthology.orchestra.run.smithi115.stdout:38654705669 2024-09-10T15:57:37.016 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705669 got 38654705669 for osd.2 2024-09-10T15:57:37.016 DEBUG:teuthology.parallel:result is None 2024-09-10T15:57:37.017 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-10T15:57:37.017 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:57:37.337 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T15:57:37.338 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T15:57:37.350 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":17,"stamp":"2024-09-10T15:57:35.525370+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":81464,"kb_used_data":368,"kb_used_omap":4,"kb_used_meta":81019,"kb_avail":283034056,"statfs":{"total":289910292480,"available":289826873344,"internally_reserved":0,"allocated":376832,"data_stored":86909,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4785,"internal_metadata":82963791},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":20,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-10T15:57:33.885508+0000","last_change":"2024-09-10T15:57:33.885818+0000","last_active":"2024-09-10T15:57:33.885508+0000","last_peered":"2024-09-10T15:57:33.885508+0000","last_clean":"2024-09-10T15:57:33.885508+0000","last_became_active":"2024-09-10T15:57:31.879578+0000","last_became_peered":"2024-09-10T15:57:31.879578+0000","last_unstale":"2024-09-10T15:57:33.885508+0000","last_undegraded":"2024-09-10T15:57:33.885508+0000","last_fullsized":"2024-09-10T15:57:33.885508+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:35:11.827649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.6","version":"0'0","reported_seq":20,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-10T15:57:33.884758+0000","last_change":"2024-09-10T15:57:33.885317+0000","last_active":"2024-09-10T15:57:33.884758+0000","last_peered":"2024-09-10T15:57:33.884758+0000","last_clean":"2024-09-10T15:57:33.884758+0000","last_became_active":"2024-09-10T15:57:31.880838+0000","last_became_peered":"2024-09-10T15:57:31.880838+0000","last_unstale":"2024-09-10T15:57:33.884758+0000","last_undegraded":"2024-09-10T15:57:33.884758+0000","last_fullsized":"2024-09-10T15:57:33.884758+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:11:25.004601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.5","version":"0'0","reported_seq":20,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-10T15:57:34.056950+0000","last_change":"2024-09-10T15:57:34.057084+0000","last_active":"2024-09-10T15:57:34.056950+0000","last_peered":"2024-09-10T15:57:34.056950+0000","last_clean":"2024-09-10T15:57:34.056950+0000","last_became_active":"2024-09-10T15:57:31.874099+0000","last_became_peered":"2024-09-10T15:57:31.874099+0000","last_unstale":"2024-09-10T15:57:34.056950+0000","last_undegraded":"2024-09-10T15:57:34.056950+0000","last_fullsized":"2024-09-10T15:57:34.056950+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:44:41.891253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.4","version":"0'0","reported_seq":20,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-10T15:57:33.884702+0000","last_change":"2024-09-10T15:57:33.885087+0000","last_active":"2024-09-10T15:57:33.884702+0000","last_peered":"2024-09-10T15:57:33.884702+0000","last_clean":"2024-09-10T15:57:33.884702+0000","last_became_active":"2024-09-10T15:57:31.880825+0000","last_became_peered":"2024-09-10T15:57:31.880825+0000","last_unstale":"2024-09-10T15:57:33.884702+0000","last_undegraded":"2024-09-10T15:57:33.884702+0000","last_fullsized":"2024-09-10T15:57:33.884702+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:46:50.469478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.3","version":"12'1","reported_seq":21,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-10T15:57:33.886150+0000","last_change":"2024-09-10T15:57:33.886400+0000","last_active":"2024-09-10T15:57:33.886150+0000","last_peered":"2024-09-10T15:57:33.886150+0000","last_clean":"2024-09-10T15:57:33.886150+0000","last_became_active":"2024-09-10T15:57:31.879568+0000","last_became_peered":"2024-09-10T15:57:31.879568+0000","last_unstale":"2024-09-10T15:57:33.886150+0000","last_undegraded":"2024-09-10T15:57:33.886150+0000","last_fullsized":"2024-09-10T15:57:33.886150+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:57:35.391909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.2","version":"14'2","reported_seq":22,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-10T15:57:33.885956+0000","last_change":"2024-09-10T15:57:33.885956+0000","last_active":"2024-09-10T15:57:33.885956+0000","last_peered":"2024-09-10T15:57:33.885956+0000","last_clean":"2024-09-10T15:57:33.885956+0000","last_became_active":"2024-09-10T15:57:31.879977+0000","last_became_peered":"2024-09-10T15:57:31.879977+0000","last_unstale":"2024-09-10T15:57:33.885956+0000","last_undegraded":"2024-09-10T15:57:33.885956+0000","last_fullsized":"2024-09-10T15:57:33.885956+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:19:02.673743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.1","version":"0'0","reported_seq":20,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-10T15:57:34.056905+0000","last_change":"2024-09-10T15:57:34.057087+0000","last_active":"2024-09-10T15:57:34.056905+0000","last_peered":"2024-09-10T15:57:34.056905+0000","last_clean":"2024-09-10T15:57:34.056905+0000","last_became_active":"2024-09-10T15:57:31.871233+0000","last_became_peered":"2024-09-10T15:57:31.871233+0000","last_unstale":"2024-09-10T15:57:34.056905+0000","last_undegraded":"2024-09-10T15:57:34.056905+0000","last_fullsized":"2024-09-10T15:57:34.056905+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:59:40.072090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"0'0","reported_seq":20,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-10T15:57:33.885567+0000","last_change":"2024-09-10T15:57:33.885856+0000","last_active":"2024-09-10T15:57:33.885567+0000","last_peered":"2024-09-10T15:57:33.885567+0000","last_clean":"2024-09-10T15:57:33.885567+0000","last_became_active":"2024-09-10T15:57:31.880722+0000","last_became_peered":"2024-09-10T15:57:31.880722+0000","last_unstale":"2024-09-10T15:57:33.885567+0000","last_undegraded":"2024-09-10T15:57:33.885567+0000","last_fullsized":"2024-09-10T15:57:33.885567+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:58:36.899441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":9,"seq":38654705669,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27152,"kb_used_data":120,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344688,"statfs":{"total":96636764160,"available":96608960512,"internally_reserved":0,"allocated":122880,"data_stored":28957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705667,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27160,"kb_used_data":124,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344680,"statfs":{"total":96636764160,"available":96608952320,"internally_reserved":0,"allocated":126976,"data_stored":28976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705669,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27152,"kb_used_data":124,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344688,"statfs":{"total":96636764160,"available":96608960512,"internally_reserved":0,"allocated":126976,"data_stored":28976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:57:37.350 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:57:37.633 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T15:57:37.633 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T15:57:37.645 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":18,"stamp":"2024-09-10T15:57:37.525644+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":81464,"kb_used_data":368,"kb_used_omap":4,"kb_used_meta":81019,"kb_avail":283034056,"statfs":{"total":289910292480,"available":289826873344,"internally_reserved":0,"allocated":376832,"data_stored":86909,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4785,"internal_metadata":82963791},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.000274"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":20,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-10T15:57:33.885508+0000","last_change":"2024-09-10T15:57:33.885818+0000","last_active":"2024-09-10T15:57:33.885508+0000","last_peered":"2024-09-10T15:57:33.885508+0000","last_clean":"2024-09-10T15:57:33.885508+0000","last_became_active":"2024-09-10T15:57:31.879578+0000","last_became_peered":"2024-09-10T15:57:31.879578+0000","last_unstale":"2024-09-10T15:57:33.885508+0000","last_undegraded":"2024-09-10T15:57:33.885508+0000","last_fullsized":"2024-09-10T15:57:33.885508+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:35:11.827649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.6","version":"0'0","reported_seq":20,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-10T15:57:33.884758+0000","last_change":"2024-09-10T15:57:33.885317+0000","last_active":"2024-09-10T15:57:33.884758+0000","last_peered":"2024-09-10T15:57:33.884758+0000","last_clean":"2024-09-10T15:57:33.884758+0000","last_became_active":"2024-09-10T15:57:31.880838+0000","last_became_peered":"2024-09-10T15:57:31.880838+0000","last_unstale":"2024-09-10T15:57:33.884758+0000","last_undegraded":"2024-09-10T15:57:33.884758+0000","last_fullsized":"2024-09-10T15:57:33.884758+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:11:25.004601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.5","version":"0'0","reported_seq":20,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-10T15:57:34.056950+0000","last_change":"2024-09-10T15:57:34.057084+0000","last_active":"2024-09-10T15:57:34.056950+0000","last_peered":"2024-09-10T15:57:34.056950+0000","last_clean":"2024-09-10T15:57:34.056950+0000","last_became_active":"2024-09-10T15:57:31.874099+0000","last_became_peered":"2024-09-10T15:57:31.874099+0000","last_unstale":"2024-09-10T15:57:34.056950+0000","last_undegraded":"2024-09-10T15:57:34.056950+0000","last_fullsized":"2024-09-10T15:57:34.056950+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:44:41.891253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.4","version":"0'0","reported_seq":20,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-10T15:57:33.884702+0000","last_change":"2024-09-10T15:57:33.885087+0000","last_active":"2024-09-10T15:57:33.884702+0000","last_peered":"2024-09-10T15:57:33.884702+0000","last_clean":"2024-09-10T15:57:33.884702+0000","last_became_active":"2024-09-10T15:57:31.880825+0000","last_became_peered":"2024-09-10T15:57:31.880825+0000","last_unstale":"2024-09-10T15:57:33.884702+0000","last_undegraded":"2024-09-10T15:57:33.884702+0000","last_fullsized":"2024-09-10T15:57:33.884702+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:46:50.469478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.3","version":"12'1","reported_seq":21,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-10T15:57:33.886150+0000","last_change":"2024-09-10T15:57:33.886400+0000","last_active":"2024-09-10T15:57:33.886150+0000","last_peered":"2024-09-10T15:57:33.886150+0000","last_clean":"2024-09-10T15:57:33.886150+0000","last_became_active":"2024-09-10T15:57:31.879568+0000","last_became_peered":"2024-09-10T15:57:31.879568+0000","last_unstale":"2024-09-10T15:57:33.886150+0000","last_undegraded":"2024-09-10T15:57:33.886150+0000","last_fullsized":"2024-09-10T15:57:33.886150+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:57:35.391909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.2","version":"14'2","reported_seq":22,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-10T15:57:33.885956+0000","last_change":"2024-09-10T15:57:33.885956+0000","last_active":"2024-09-10T15:57:33.885956+0000","last_peered":"2024-09-10T15:57:33.885956+0000","last_clean":"2024-09-10T15:57:33.885956+0000","last_became_active":"2024-09-10T15:57:31.879977+0000","last_became_peered":"2024-09-10T15:57:31.879977+0000","last_unstale":"2024-09-10T15:57:33.885956+0000","last_undegraded":"2024-09-10T15:57:33.885956+0000","last_fullsized":"2024-09-10T15:57:33.885956+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:19:02.673743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.1","version":"0'0","reported_seq":20,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-10T15:57:34.056905+0000","last_change":"2024-09-10T15:57:34.057087+0000","last_active":"2024-09-10T15:57:34.056905+0000","last_peered":"2024-09-10T15:57:34.056905+0000","last_clean":"2024-09-10T15:57:34.056905+0000","last_became_active":"2024-09-10T15:57:31.871233+0000","last_became_peered":"2024-09-10T15:57:31.871233+0000","last_unstale":"2024-09-10T15:57:34.056905+0000","last_undegraded":"2024-09-10T15:57:34.056905+0000","last_fullsized":"2024-09-10T15:57:34.056905+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:59:40.072090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"0'0","reported_seq":20,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-10T15:57:33.885567+0000","last_change":"2024-09-10T15:57:33.885856+0000","last_active":"2024-09-10T15:57:33.885567+0000","last_peered":"2024-09-10T15:57:33.885567+0000","last_clean":"2024-09-10T15:57:33.885567+0000","last_became_active":"2024-09-10T15:57:31.880722+0000","last_became_peered":"2024-09-10T15:57:31.880722+0000","last_unstale":"2024-09-10T15:57:33.885567+0000","last_undegraded":"2024-09-10T15:57:33.885567+0000","last_fullsized":"2024-09-10T15:57:33.885567+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:58:36.899441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":9,"seq":38654705670,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27152,"kb_used_data":120,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344688,"statfs":{"total":96636764160,"available":96608960512,"internally_reserved":0,"allocated":122880,"data_stored":28957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705668,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27160,"kb_used_data":124,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344680,"statfs":{"total":96636764160,"available":96608952320,"internally_reserved":0,"allocated":126976,"data_stored":28976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705670,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27152,"kb_used_data":124,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344688,"statfs":{"total":96636764160,"available":96608960512,"internally_reserved":0,"allocated":126976,"data_stored":28976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:57:37.646 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-10T15:57:37.646 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-10T15:57:37.647 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-09-10T15:57:37.647 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-10T15:57:37.965 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T15:57:37.965 INFO:teuthology.orchestra.run.smithi115.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-10T15:57:37.978 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-09-10T15:57:37.978 INFO:teuthology.run_tasks:Running task lost_unfound... 2024-09-10T15:57:37.990 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T15:57:38.275 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T15:57:38.275 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":14,"fsid":"7effee29-df05-4351-9741-ee203564ed81","created":"2024-09-10T15:57:14.967498+0000","modified":"2024-09-10T15:57:33.873157+0000","last_up_change":"2024-09-10T15:57:25.842984+0000","last_in_change":"2024-09-10T15:57:16.214215+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2024-09-10T15:57:30.381324+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":14,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"859508e0-e15d-4011-a39f-65e7aa04a129","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":11,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":855194165}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6805","nonce":855194165}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6807","nonce":855194165}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":855194165}]},"public_addr":"172.21.15.115:6804/855194165","cluster_addr":"172.21.15.115:6805/855194165","heartbeat_back_addr":"172.21.15.115:6807/855194165","heartbeat_front_addr":"172.21.15.115:6806/855194165","state":["exists","up"]},{"osd":1,"uuid":"66daed2f-a693-4a13-8d8c-024794a032e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":11,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":2071140438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6809","nonce":2071140438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6811","nonce":2071140438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":2071140438}]},"public_addr":"172.21.15.115:6808/2071140438","cluster_addr":"172.21.15.115:6809/2071140438","heartbeat_back_addr":"172.21.15.115:6811/2071140438","heartbeat_front_addr":"172.21.15.115:6810/2071140438","state":["exists","up"]},{"osd":2,"uuid":"6a2ab153-d99e-4a5d-b79c-32b7f8ea9ffb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":11,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":267302962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6801","nonce":267302962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6803","nonce":267302962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":267302962}]},"public_addr":"172.21.15.115:6800/267302962","cluster_addr":"172.21.15.115:6801/267302962","heartbeat_back_addr":"172.21.15.115:6803/267302962","heartbeat_front_addr":"172.21.15.115:6802/267302962","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T15:57:23.007077+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T15:57:23.605294+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T15:57:22.852045+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":1,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":1,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T15:57:38.288 INFO:tasks.lost_unfound.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2024-09-10T15:57:30.381324+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '14', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 14, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-10T15:57:38.288 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2024-09-10T15:57:38.575 INFO:teuthology.orchestra.run.smithi115.stdout:pg_num: 8 2024-09-10T15:57:38.588 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump 2024-09-10T15:57:38.873 INFO:teuthology.orchestra.run.smithi115.stdout:epoch 14 2024-09-10T15:57:38.873 INFO:teuthology.orchestra.run.smithi115.stdout:fsid 7effee29-df05-4351-9741-ee203564ed81 2024-09-10T15:57:38.873 INFO:teuthology.orchestra.run.smithi115.stdout:created 2024-09-10T15:57:14.967498+0000 2024-09-10T15:57:38.874 INFO:teuthology.orchestra.run.smithi115.stdout:modified 2024-09-10T15:57:33.873157+0000 2024-09-10T15:57:38.874 INFO:teuthology.orchestra.run.smithi115.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-09-10T15:57:38.874 INFO:teuthology.orchestra.run.smithi115.stdout:crush_version 5 2024-09-10T15:57:38.874 INFO:teuthology.orchestra.run.smithi115.stdout:full_ratio 0.95 2024-09-10T15:57:38.874 INFO:teuthology.orchestra.run.smithi115.stdout:backfillfull_ratio 0.9 2024-09-10T15:57:38.874 INFO:teuthology.orchestra.run.smithi115.stdout:nearfull_ratio 0.85 2024-09-10T15:57:38.874 INFO:teuthology.orchestra.run.smithi115.stdout:require_min_compat_client luminous 2024-09-10T15:57:38.874 INFO:teuthology.orchestra.run.smithi115.stdout:min_compat_client jewel 2024-09-10T15:57:38.874 INFO:teuthology.orchestra.run.smithi115.stdout:require_osd_release squid 2024-09-10T15:57:38.874 INFO:teuthology.orchestra.run.smithi115.stdout:stretch_mode_enabled false 2024-09-10T15:57:38.874 INFO:teuthology.orchestra.run.smithi115.stdout:pool 1 'rbd' replicated size 2 min_size 1 crush_rule 0 object_hash rjenkins pg_num 8 pgp_num 8 autoscale_mode off last_change 14 flags hashpspool,selfmanaged_snaps stripe_width 0 application rbd read_balance_score 1.88 2024-09-10T15:57:38.874 INFO:teuthology.orchestra.run.smithi115.stdout: removed_snaps_queue [2~1] 2024-09-10T15:57:38.875 INFO:teuthology.orchestra.run.smithi115.stdout:max_osd 3 2024-09-10T15:57:38.875 INFO:teuthology.orchestra.run.smithi115.stdout:osd.0 up in weight 1 up_from 9 up_thru 11 down_at 0 last_clean_interval [0,0) v2:172.21.15.115:6804/855194165 v2:172.21.15.115:6805/855194165 exists,up 859508e0-e15d-4011-a39f-65e7aa04a129 2024-09-10T15:57:38.875 INFO:teuthology.orchestra.run.smithi115.stdout:osd.1 up in weight 1 up_from 9 up_thru 11 down_at 0 last_clean_interval [0,0) v2:172.21.15.115:6808/2071140438 v2:172.21.15.115:6809/2071140438 exists,up 66daed2f-a693-4a13-8d8c-024794a032e3 2024-09-10T15:57:38.875 INFO:teuthology.orchestra.run.smithi115.stdout:osd.2 up in weight 1 up_from 9 up_thru 11 down_at 0 last_clean_interval [0,0) v2:172.21.15.115:6800/267302962 v2:172.21.15.115:6801/267302962 exists,up 6a2ab153-d99e-4a5d-b79c-32b7f8ea9ffb 2024-09-10T15:57:38.886 INFO:tasks.lost_unfound.ceph_manager:['osd.0 up in weight 1 up_from 9 up_thru 11 down_at 0 last_clean_interval [0,0) v2:172.21.15.115:6804/855194165 v2:172.21.15.115:6805/855194165 exists,up 859508e0-e15d-4011-a39f-65e7aa04a129', 'osd.1 up in weight 1 up_from 9 up_thru 11 down_at 0 last_clean_interval [0,0) v2:172.21.15.115:6808/2071140438 v2:172.21.15.115:6809/2071140438 exists,up 66daed2f-a693-4a13-8d8c-024794a032e3', 'osd.2 up in weight 1 up_from 9 up_thru 11 down_at 0 last_clean_interval [0,0) v2:172.21.15.115:6800/267302962 v2:172.21.15.115:6801/267302962 exists,up 6a2ab153-d99e-4a5d-b79c-32b7f8ea9ffb'] 2024-09-10T15:57:38.886 INFO:tasks.lost_unfound.ceph_manager:waiting for clean 2024-09-10T15:57:38.886 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:57:39.165 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T15:57:39.166 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T15:57:39.177 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":18,"stamp":"2024-09-10T15:57:37.525644+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":81464,"kb_used_data":368,"kb_used_omap":4,"kb_used_meta":81019,"kb_avail":283034056,"statfs":{"total":289910292480,"available":289826873344,"internally_reserved":0,"allocated":376832,"data_stored":86909,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4785,"internal_metadata":82963791},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.000274"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":20,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-10T15:57:33.885508+0000","last_change":"2024-09-10T15:57:33.885818+0000","last_active":"2024-09-10T15:57:33.885508+0000","last_peered":"2024-09-10T15:57:33.885508+0000","last_clean":"2024-09-10T15:57:33.885508+0000","last_became_active":"2024-09-10T15:57:31.879578+0000","last_became_peered":"2024-09-10T15:57:31.879578+0000","last_unstale":"2024-09-10T15:57:33.885508+0000","last_undegraded":"2024-09-10T15:57:33.885508+0000","last_fullsized":"2024-09-10T15:57:33.885508+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:35:11.827649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.6","version":"0'0","reported_seq":20,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-10T15:57:33.884758+0000","last_change":"2024-09-10T15:57:33.885317+0000","last_active":"2024-09-10T15:57:33.884758+0000","last_peered":"2024-09-10T15:57:33.884758+0000","last_clean":"2024-09-10T15:57:33.884758+0000","last_became_active":"2024-09-10T15:57:31.880838+0000","last_became_peered":"2024-09-10T15:57:31.880838+0000","last_unstale":"2024-09-10T15:57:33.884758+0000","last_undegraded":"2024-09-10T15:57:33.884758+0000","last_fullsized":"2024-09-10T15:57:33.884758+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:11:25.004601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.5","version":"0'0","reported_seq":20,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-10T15:57:34.056950+0000","last_change":"2024-09-10T15:57:34.057084+0000","last_active":"2024-09-10T15:57:34.056950+0000","last_peered":"2024-09-10T15:57:34.056950+0000","last_clean":"2024-09-10T15:57:34.056950+0000","last_became_active":"2024-09-10T15:57:31.874099+0000","last_became_peered":"2024-09-10T15:57:31.874099+0000","last_unstale":"2024-09-10T15:57:34.056950+0000","last_undegraded":"2024-09-10T15:57:34.056950+0000","last_fullsized":"2024-09-10T15:57:34.056950+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:44:41.891253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.4","version":"0'0","reported_seq":20,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-10T15:57:33.884702+0000","last_change":"2024-09-10T15:57:33.885087+0000","last_active":"2024-09-10T15:57:33.884702+0000","last_peered":"2024-09-10T15:57:33.884702+0000","last_clean":"2024-09-10T15:57:33.884702+0000","last_became_active":"2024-09-10T15:57:31.880825+0000","last_became_peered":"2024-09-10T15:57:31.880825+0000","last_unstale":"2024-09-10T15:57:33.884702+0000","last_undegraded":"2024-09-10T15:57:33.884702+0000","last_fullsized":"2024-09-10T15:57:33.884702+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:46:50.469478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.3","version":"12'1","reported_seq":21,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-10T15:57:33.886150+0000","last_change":"2024-09-10T15:57:33.886400+0000","last_active":"2024-09-10T15:57:33.886150+0000","last_peered":"2024-09-10T15:57:33.886150+0000","last_clean":"2024-09-10T15:57:33.886150+0000","last_became_active":"2024-09-10T15:57:31.879568+0000","last_became_peered":"2024-09-10T15:57:31.879568+0000","last_unstale":"2024-09-10T15:57:33.886150+0000","last_undegraded":"2024-09-10T15:57:33.886150+0000","last_fullsized":"2024-09-10T15:57:33.886150+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:57:35.391909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.2","version":"14'2","reported_seq":22,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-10T15:57:33.885956+0000","last_change":"2024-09-10T15:57:33.885956+0000","last_active":"2024-09-10T15:57:33.885956+0000","last_peered":"2024-09-10T15:57:33.885956+0000","last_clean":"2024-09-10T15:57:33.885956+0000","last_became_active":"2024-09-10T15:57:31.879977+0000","last_became_peered":"2024-09-10T15:57:31.879977+0000","last_unstale":"2024-09-10T15:57:33.885956+0000","last_undegraded":"2024-09-10T15:57:33.885956+0000","last_fullsized":"2024-09-10T15:57:33.885956+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:19:02.673743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.1","version":"0'0","reported_seq":20,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-10T15:57:34.056905+0000","last_change":"2024-09-10T15:57:34.057087+0000","last_active":"2024-09-10T15:57:34.056905+0000","last_peered":"2024-09-10T15:57:34.056905+0000","last_clean":"2024-09-10T15:57:34.056905+0000","last_became_active":"2024-09-10T15:57:31.871233+0000","last_became_peered":"2024-09-10T15:57:31.871233+0000","last_unstale":"2024-09-10T15:57:34.056905+0000","last_undegraded":"2024-09-10T15:57:34.056905+0000","last_fullsized":"2024-09-10T15:57:34.056905+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:59:40.072090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"0'0","reported_seq":20,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-10T15:57:33.885567+0000","last_change":"2024-09-10T15:57:33.885856+0000","last_active":"2024-09-10T15:57:33.885567+0000","last_peered":"2024-09-10T15:57:33.885567+0000","last_clean":"2024-09-10T15:57:33.885567+0000","last_became_active":"2024-09-10T15:57:31.880722+0000","last_became_peered":"2024-09-10T15:57:31.880722+0000","last_unstale":"2024-09-10T15:57:33.885567+0000","last_undegraded":"2024-09-10T15:57:33.885567+0000","last_fullsized":"2024-09-10T15:57:33.885567+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:58:36.899441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":9,"seq":38654705670,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27152,"kb_used_data":120,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344688,"statfs":{"total":96636764160,"available":96608960512,"internally_reserved":0,"allocated":122880,"data_stored":28957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705668,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27160,"kb_used_data":124,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344680,"statfs":{"total":96636764160,"available":96608952320,"internally_reserved":0,"allocated":126976,"data_stored":28976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705670,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27152,"kb_used_data":124,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344688,"statfs":{"total":96636764160,"available":96608960512,"internally_reserved":0,"allocated":126976,"data_stored":28976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:57:39.178 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:57:39.456 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T15:57:39.457 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T15:57:39.468 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":18,"stamp":"2024-09-10T15:57:37.525644+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":81464,"kb_used_data":368,"kb_used_omap":4,"kb_used_meta":81019,"kb_avail":283034056,"statfs":{"total":289910292480,"available":289826873344,"internally_reserved":0,"allocated":376832,"data_stored":86909,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4785,"internal_metadata":82963791},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.000274"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":20,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-10T15:57:33.885508+0000","last_change":"2024-09-10T15:57:33.885818+0000","last_active":"2024-09-10T15:57:33.885508+0000","last_peered":"2024-09-10T15:57:33.885508+0000","last_clean":"2024-09-10T15:57:33.885508+0000","last_became_active":"2024-09-10T15:57:31.879578+0000","last_became_peered":"2024-09-10T15:57:31.879578+0000","last_unstale":"2024-09-10T15:57:33.885508+0000","last_undegraded":"2024-09-10T15:57:33.885508+0000","last_fullsized":"2024-09-10T15:57:33.885508+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:35:11.827649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.6","version":"0'0","reported_seq":20,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-10T15:57:33.884758+0000","last_change":"2024-09-10T15:57:33.885317+0000","last_active":"2024-09-10T15:57:33.884758+0000","last_peered":"2024-09-10T15:57:33.884758+0000","last_clean":"2024-09-10T15:57:33.884758+0000","last_became_active":"2024-09-10T15:57:31.880838+0000","last_became_peered":"2024-09-10T15:57:31.880838+0000","last_unstale":"2024-09-10T15:57:33.884758+0000","last_undegraded":"2024-09-10T15:57:33.884758+0000","last_fullsized":"2024-09-10T15:57:33.884758+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:11:25.004601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.5","version":"0'0","reported_seq":20,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-10T15:57:34.056950+0000","last_change":"2024-09-10T15:57:34.057084+0000","last_active":"2024-09-10T15:57:34.056950+0000","last_peered":"2024-09-10T15:57:34.056950+0000","last_clean":"2024-09-10T15:57:34.056950+0000","last_became_active":"2024-09-10T15:57:31.874099+0000","last_became_peered":"2024-09-10T15:57:31.874099+0000","last_unstale":"2024-09-10T15:57:34.056950+0000","last_undegraded":"2024-09-10T15:57:34.056950+0000","last_fullsized":"2024-09-10T15:57:34.056950+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:44:41.891253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.4","version":"0'0","reported_seq":20,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-10T15:57:33.884702+0000","last_change":"2024-09-10T15:57:33.885087+0000","last_active":"2024-09-10T15:57:33.884702+0000","last_peered":"2024-09-10T15:57:33.884702+0000","last_clean":"2024-09-10T15:57:33.884702+0000","last_became_active":"2024-09-10T15:57:31.880825+0000","last_became_peered":"2024-09-10T15:57:31.880825+0000","last_unstale":"2024-09-10T15:57:33.884702+0000","last_undegraded":"2024-09-10T15:57:33.884702+0000","last_fullsized":"2024-09-10T15:57:33.884702+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:46:50.469478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.3","version":"12'1","reported_seq":21,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-10T15:57:33.886150+0000","last_change":"2024-09-10T15:57:33.886400+0000","last_active":"2024-09-10T15:57:33.886150+0000","last_peered":"2024-09-10T15:57:33.886150+0000","last_clean":"2024-09-10T15:57:33.886150+0000","last_became_active":"2024-09-10T15:57:31.879568+0000","last_became_peered":"2024-09-10T15:57:31.879568+0000","last_unstale":"2024-09-10T15:57:33.886150+0000","last_undegraded":"2024-09-10T15:57:33.886150+0000","last_fullsized":"2024-09-10T15:57:33.886150+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:57:35.391909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.2","version":"14'2","reported_seq":22,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-10T15:57:33.885956+0000","last_change":"2024-09-10T15:57:33.885956+0000","last_active":"2024-09-10T15:57:33.885956+0000","last_peered":"2024-09-10T15:57:33.885956+0000","last_clean":"2024-09-10T15:57:33.885956+0000","last_became_active":"2024-09-10T15:57:31.879977+0000","last_became_peered":"2024-09-10T15:57:31.879977+0000","last_unstale":"2024-09-10T15:57:33.885956+0000","last_undegraded":"2024-09-10T15:57:33.885956+0000","last_fullsized":"2024-09-10T15:57:33.885956+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:19:02.673743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.1","version":"0'0","reported_seq":20,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-10T15:57:34.056905+0000","last_change":"2024-09-10T15:57:34.057087+0000","last_active":"2024-09-10T15:57:34.056905+0000","last_peered":"2024-09-10T15:57:34.056905+0000","last_clean":"2024-09-10T15:57:34.056905+0000","last_became_active":"2024-09-10T15:57:31.871233+0000","last_became_peered":"2024-09-10T15:57:31.871233+0000","last_unstale":"2024-09-10T15:57:34.056905+0000","last_undegraded":"2024-09-10T15:57:34.056905+0000","last_fullsized":"2024-09-10T15:57:34.056905+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:59:40.072090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"0'0","reported_seq":20,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-10T15:57:33.885567+0000","last_change":"2024-09-10T15:57:33.885856+0000","last_active":"2024-09-10T15:57:33.885567+0000","last_peered":"2024-09-10T15:57:33.885567+0000","last_clean":"2024-09-10T15:57:33.885567+0000","last_became_active":"2024-09-10T15:57:31.880722+0000","last_became_peered":"2024-09-10T15:57:31.880722+0000","last_unstale":"2024-09-10T15:57:33.885567+0000","last_undegraded":"2024-09-10T15:57:33.885567+0000","last_fullsized":"2024-09-10T15:57:33.885567+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:58:36.899441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":9,"seq":38654705670,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27152,"kb_used_data":120,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344688,"statfs":{"total":96636764160,"available":96608960512,"internally_reserved":0,"allocated":122880,"data_stored":28957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705668,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27160,"kb_used_data":124,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344680,"statfs":{"total":96636764160,"available":96608952320,"internally_reserved":0,"allocated":126976,"data_stored":28976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705670,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27152,"kb_used_data":124,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344688,"statfs":{"total":96636764160,"available":96608960512,"internally_reserved":0,"allocated":126976,"data_stored":28976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:57:39.469 INFO:tasks.lost_unfound.ceph_manager:clean! 2024-09-10T15:57:39.469 INFO:tasks.lost_unfound.ceph_manager:creating pool_name unfound_pool 2024-09-10T15:57:39.469 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create unfound_pool 16 2024-09-10T15:57:40.751 INFO:teuthology.orchestra.run.smithi115.stderr:pool 'unfound_pool' created 2024-09-10T15:57:40.772 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool application enable unfound_pool rados --yes-i-really-mean-it || true 2024-09-10T15:57:41.758 INFO:teuthology.orchestra.run.smithi115.stderr:enabled application 'rados' on pool 'unfound_pool' 2024-09-10T15:57:42.782 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-09-10T15:57:42.782 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T15:57:42.819 INFO:tasks.ceph.osd.2:Stopped 2024-09-10T15:57:42.819 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd down 2 2024-09-10T15:57:43.165 INFO:tasks.ceph.mon.a.smithi115.stderr:2024-09-10T15:57:43.164+0000 7fa206bdb640 -1 mon.a@0(leader).osd e18 definitely_dead 0 2024-09-10T15:57:43.762 INFO:teuthology.orchestra.run.smithi115.stderr:marked down osd.2. 2024-09-10T15:57:43.776 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd out 2 2024-09-10T15:57:44.074 INFO:tasks.ceph.mon.a.smithi115.stderr:2024-09-10T15:57:44.072+0000 7fa206bdb640 -1 mon.a@0(leader).osd e19 definitely_dead 0 2024-09-10T15:57:44.757 INFO:teuthology.orchestra.run.smithi115.stderr:marked out osd.2. 2024-09-10T15:57:44.776 INFO:tasks.util.rados:rados -p unfound_pool put dummy /etc/fstab 2024-09-10T15:57:44.776 DEBUG:teuthology.orchestra.run.smithi115:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool put dummy /etc/fstab 2024-09-10T15:57:44.831 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-10T15:57:44.832 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-10T15:57:44.967 INFO:teuthology.orchestra.run.smithi115.stdout:38654705672 2024-09-10T15:57:44.968 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-10T15:57:44.968 INFO:teuthology.orchestra.run.smithi115.stdout:38654705670 2024-09-10T15:57:44.969 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-10T15:57:45.294 INFO:teuthology.orchestra.run.smithi115.stdout:38654705669 2024-09-10T15:57:45.296 INFO:teuthology.orchestra.run.smithi115.stdout:38654705671 2024-09-10T15:57:45.308 INFO:tasks.lost_unfound.ceph_manager:need seq 38654705670 got 38654705669 for osd.1 2024-09-10T15:57:45.310 INFO:tasks.lost_unfound.ceph_manager:need seq 38654705672 got 38654705671 for osd.0 2024-09-10T15:57:46.308 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-10T15:57:46.311 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-10T15:57:46.612 INFO:teuthology.orchestra.run.smithi115.stdout:38654705672 2024-09-10T15:57:46.613 INFO:teuthology.orchestra.run.smithi115.stdout:38654705670 2024-09-10T15:57:46.626 INFO:tasks.lost_unfound.ceph_manager:need seq 38654705672 got 38654705672 for osd.0 2024-09-10T15:57:46.626 DEBUG:teuthology.parallel:result is None 2024-09-10T15:57:46.627 INFO:tasks.lost_unfound.ceph_manager:need seq 38654705670 got 38654705670 for osd.1 2024-09-10T15:57:46.627 DEBUG:teuthology.parallel:result is None 2024-09-10T15:57:46.627 INFO:tasks.lost_unfound.ceph_manager:waiting for recovery to complete 2024-09-10T15:57:46.627 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:57:46.909 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T15:57:46.909 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T15:57:46.921 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":29,"stamp":"2024-09-10T15:57:45.759948+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":36,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54404,"kb_used_data":340,"kb_used_omap":3,"kb_used_meta":54012,"kb_avail":188689276,"statfs":{"total":193273528320,"available":193217818624,"internally_reserved":0,"allocated":348160,"data_stored":90315,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3190,"internal_metadata":55309194},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.006171"},"pg_stats":[{"pgid":"2.f","version":"0'0","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T15:57:44.765847+0000","last_change":"2024-09-10T15:57:41.762176+0000","last_active":"2024-09-10T15:57:44.765847+0000","last_peered":"2024-09-10T15:57:44.765847+0000","last_clean":"2024-09-10T15:57:44.765847+0000","last_became_active":"2024-09-10T15:57:41.760663+0000","last_became_peered":"2024-09-10T15:57:41.760663+0000","last_unstale":"2024-09-10T15:57:44.765847+0000","last_undegraded":"2024-09-10T15:57:44.765847+0000","last_fullsized":"2024-09-10T15:57:44.765847+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:40:37.854184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"peering","last_fresh":"2024-09-10T15:57:44.765540+0000","last_change":"2024-09-10T15:57:43.760533+0000","last_active":"2024-09-10T15:57:43.760533+0000","last_peered":"2024-09-10T15:57:41.764794+0000","last_clean":"2024-09-10T15:57:40.749557+0000","last_became_active":"2024-09-10T15:57:41.764602+0000","last_became_peered":"2024-09-10T15:57:41.764602+0000","last_unstale":"2024-09-10T15:57:44.765540+0000","last_undegraded":"2024-09-10T15:57:44.765540+0000","last_fullsized":"2024-09-10T15:57:44.765540+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":15,"reported_epoch":19,"state":"peering","last_fresh":"2024-09-10T15:57:43.760963+0000","last_change":"2024-09-10T15:57:43.760963+0000","last_active":"2024-09-10T15:57:43.760963+0000","last_peered":"2024-09-10T15:57:43.258110+0000","last_clean":"2024-09-10T15:57:43.258110+0000","last_became_active":"2024-09-10T15:57:41.761703+0000","last_became_peered":"2024-09-10T15:57:41.761703+0000","last_unstale":"2024-09-10T15:57:43.760963+0000","last_undegraded":"2024-09-10T15:57:43.760963+0000","last_fullsized":"2024-09-10T15:57:43.760963+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"peering","last_fresh":"2024-09-10T15:57:44.764624+0000","last_change":"2024-09-10T15:57:43.760175+0000","last_active":"2024-09-10T15:57:43.760175+0000","last_peered":"2024-09-10T15:57:41.765363+0000","last_clean":"2024-09-10T15:57:40.749557+0000","last_became_active":"2024-09-10T15:57:41.765168+0000","last_became_peered":"2024-09-10T15:57:41.765168+0000","last_unstale":"2024-09-10T15:57:44.764624+0000","last_undegraded":"2024-09-10T15:57:44.764624+0000","last_fullsized":"2024-09-10T15:57:44.764624+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":15,"reported_epoch":19,"state":"peering","last_fresh":"2024-09-10T15:57:43.759943+0000","last_change":"2024-09-10T15:57:43.759943+0000","last_active":"2024-09-10T15:57:43.759943+0000","last_peered":"2024-09-10T15:57:42.760642+0000","last_clean":"2024-09-10T15:57:42.760642+0000","last_became_active":"2024-09-10T15:57:41.762101+0000","last_became_peered":"2024-09-10T15:57:41.762101+0000","last_unstale":"2024-09-10T15:57:43.759943+0000","last_undegraded":"2024-09-10T15:57:43.759943+0000","last_fullsized":"2024-09-10T15:57:43.759943+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T15:57:44.763488+0000","last_change":"2024-09-10T15:57:41.766809+0000","last_active":"2024-09-10T15:57:44.763488+0000","last_peered":"2024-09-10T15:57:44.763488+0000","last_clean":"2024-09-10T15:57:44.763488+0000","last_became_active":"2024-09-10T15:57:41.766519+0000","last_became_peered":"2024-09-10T15:57:41.766519+0000","last_unstale":"2024-09-10T15:57:44.763488+0000","last_undegraded":"2024-09-10T15:57:44.763488+0000","last_fullsized":"2024-09-10T15:57:44.763488+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:19:07.895620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T15:57:44.764235+0000","last_change":"2024-09-10T15:57:41.766524+0000","last_active":"2024-09-10T15:57:44.764235+0000","last_peered":"2024-09-10T15:57:44.764235+0000","last_clean":"2024-09-10T15:57:44.764235+0000","last_became_active":"2024-09-10T15:57:41.766212+0000","last_became_peered":"2024-09-10T15:57:41.766212+0000","last_unstale":"2024-09-10T15:57:44.764235+0000","last_undegraded":"2024-09-10T15:57:44.764235+0000","last_fullsized":"2024-09-10T15:57:44.764235+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:45:52.388962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":15,"reported_epoch":19,"state":"peering","last_fresh":"2024-09-10T15:57:43.760833+0000","last_change":"2024-09-10T15:57:43.760833+0000","last_active":"2024-09-10T15:57:43.760833+0000","last_peered":"2024-09-10T15:57:42.760798+0000","last_clean":"2024-09-10T15:57:42.760798+0000","last_became_active":"2024-09-10T15:57:41.761869+0000","last_became_peered":"2024-09-10T15:57:41.761869+0000","last_unstale":"2024-09-10T15:57:43.760833+0000","last_undegraded":"2024-09-10T15:57:43.760833+0000","last_fullsized":"2024-09-10T15:57:43.760833+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":32,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T15:57:44.763896+0000","last_change":"2024-09-10T15:57:33.885856+0000","last_active":"2024-09-10T15:57:44.763896+0000","last_peered":"2024-09-10T15:57:44.763896+0000","last_clean":"2024-09-10T15:57:44.763896+0000","last_became_active":"2024-09-10T15:57:31.880722+0000","last_became_peered":"2024-09-10T15:57:31.880722+0000","last_unstale":"2024-09-10T15:57:44.763896+0000","last_undegraded":"2024-09-10T15:57:44.763896+0000","last_fullsized":"2024-09-10T15:57:44.763896+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:58:36.899441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":15,"reported_epoch":19,"state":"peering","last_fresh":"2024-09-10T15:57:43.760764+0000","last_change":"2024-09-10T15:57:43.760764+0000","last_active":"2024-09-10T15:57:43.760764+0000","last_peered":"2024-09-10T15:57:43.257970+0000","last_clean":"2024-09-10T15:57:43.257970+0000","last_became_active":"2024-09-10T15:57:41.761686+0000","last_became_peered":"2024-09-10T15:57:41.761686+0000","last_unstale":"2024-09-10T15:57:43.760764+0000","last_undegraded":"2024-09-10T15:57:43.760764+0000","last_fullsized":"2024-09-10T15:57:43.760764+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":23,"reported_epoch":20,"state":"peering","last_fresh":"2024-09-10T15:57:44.764420+0000","last_change":"2024-09-10T15:57:43.761472+0000","last_active":"2024-09-10T15:57:43.761472+0000","last_peered":"2024-09-10T15:57:34.057087+0000","last_clean":"2024-09-10T15:57:34.057087+0000","last_became_active":"2024-09-10T15:57:31.871233+0000","last_became_peered":"2024-09-10T15:57:31.871233+0000","last_unstale":"2024-09-10T15:57:44.764420+0000","last_undegraded":"2024-09-10T15:57:44.764420+0000","last_fullsized":"2024-09-10T15:57:44.764420+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T15:57:44.764182+0000","last_change":"2024-09-10T15:57:41.763096+0000","last_active":"2024-09-10T15:57:44.764182+0000","last_peered":"2024-09-10T15:57:44.764182+0000","last_clean":"2024-09-10T15:57:44.764182+0000","last_became_active":"2024-09-10T15:57:41.760684+0000","last_became_peered":"2024-09-10T15:57:41.760684+0000","last_unstale":"2024-09-10T15:57:44.764182+0000","last_undegraded":"2024-09-10T15:57:44.764182+0000","last_fullsized":"2024-09-10T15:57:44.764182+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:05:02.296984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":34,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T15:57:44.765507+0000","last_change":"2024-09-10T15:57:33.885956+0000","last_active":"2024-09-10T15:57:44.765507+0000","last_peered":"2024-09-10T15:57:44.765507+0000","last_clean":"2024-09-10T15:57:44.765507+0000","last_became_active":"2024-09-10T15:57:31.879977+0000","last_became_peered":"2024-09-10T15:57:31.879977+0000","last_unstale":"2024-09-10T15:57:44.765507+0000","last_undegraded":"2024-09-10T15:57:44.765507+0000","last_fullsized":"2024-09-10T15:57:44.765507+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:19:02.673743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"peering","last_fresh":"2024-09-10T15:57:44.764108+0000","last_change":"2024-09-10T15:57:43.759828+0000","last_active":"2024-09-10T15:57:43.759828+0000","last_peered":"2024-09-10T15:57:41.762953+0000","last_clean":"2024-09-10T15:57:40.749557+0000","last_became_active":"2024-09-10T15:57:41.762752+0000","last_became_peered":"2024-09-10T15:57:41.762752+0000","last_unstale":"2024-09-10T15:57:44.764108+0000","last_undegraded":"2024-09-10T15:57:44.764108+0000","last_fullsized":"2024-09-10T15:57:44.764108+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":31,"reported_epoch":19,"state":"peering","last_fresh":"2024-09-10T15:57:43.761000+0000","last_change":"2024-09-10T15:57:43.761000+0000","last_active":"2024-09-10T15:57:43.761000+0000","last_peered":"2024-09-10T15:57:43.257847+0000","last_clean":"2024-09-10T15:57:43.257847+0000","last_became_active":"2024-09-10T15:57:31.879568+0000","last_became_peered":"2024-09-10T15:57:31.879568+0000","last_unstale":"2024-09-10T15:57:43.761000+0000","last_undegraded":"2024-09-10T15:57:43.761000+0000","last_fullsized":"2024-09-10T15:57:43.761000+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"peering","last_fresh":"2024-09-10T15:57:44.763658+0000","last_change":"2024-09-10T15:57:43.761079+0000","last_active":"2024-09-10T15:57:43.761079+0000","last_peered":"2024-09-10T15:57:41.764654+0000","last_clean":"2024-09-10T15:57:40.749557+0000","last_became_active":"2024-09-10T15:57:41.764460+0000","last_became_peered":"2024-09-10T15:57:41.764460+0000","last_unstale":"2024-09-10T15:57:44.763658+0000","last_undegraded":"2024-09-10T15:57:44.763658+0000","last_fullsized":"2024-09-10T15:57:44.763658+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T15:57:44.764399+0000","last_change":"2024-09-10T15:57:41.767200+0000","last_active":"2024-09-10T15:57:44.764399+0000","last_peered":"2024-09-10T15:57:44.764399+0000","last_clean":"2024-09-10T15:57:44.764399+0000","last_became_active":"2024-09-10T15:57:41.766595+0000","last_became_peered":"2024-09-10T15:57:41.766595+0000","last_unstale":"2024-09-10T15:57:44.764399+0000","last_undegraded":"2024-09-10T15:57:44.764399+0000","last_fullsized":"2024-09-10T15:57:44.764399+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:49:33.626681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":32,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T15:57:44.764090+0000","last_change":"2024-09-10T15:57:33.885087+0000","last_active":"2024-09-10T15:57:44.764090+0000","last_peered":"2024-09-10T15:57:44.764090+0000","last_clean":"2024-09-10T15:57:44.764090+0000","last_became_active":"2024-09-10T15:57:31.880825+0000","last_became_peered":"2024-09-10T15:57:31.880825+0000","last_unstale":"2024-09-10T15:57:44.764090+0000","last_undegraded":"2024-09-10T15:57:44.764090+0000","last_fullsized":"2024-09-10T15:57:44.764090+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:46:50.469478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T15:57:44.764169+0000","last_change":"2024-09-10T15:57:41.766037+0000","last_active":"2024-09-10T15:57:44.764169+0000","last_peered":"2024-09-10T15:57:44.764169+0000","last_clean":"2024-09-10T15:57:44.764169+0000","last_became_active":"2024-09-10T15:57:41.765714+0000","last_became_peered":"2024-09-10T15:57:41.765714+0000","last_unstale":"2024-09-10T15:57:44.764169+0000","last_undegraded":"2024-09-10T15:57:44.764169+0000","last_fullsized":"2024-09-10T15:57:44.764169+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:54:41.289541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":23,"reported_epoch":20,"state":"peering","last_fresh":"2024-09-10T15:57:44.765984+0000","last_change":"2024-09-10T15:57:43.760004+0000","last_active":"2024-09-10T15:57:43.760004+0000","last_peered":"2024-09-10T15:57:34.057084+0000","last_clean":"2024-09-10T15:57:34.057084+0000","last_became_active":"2024-09-10T15:57:31.874099+0000","last_became_peered":"2024-09-10T15:57:31.874099+0000","last_unstale":"2024-09-10T15:57:44.765984+0000","last_undegraded":"2024-09-10T15:57:44.765984+0000","last_fullsized":"2024-09-10T15:57:44.765984+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T15:57:44.765060+0000","last_change":"2024-09-10T15:57:41.766737+0000","last_active":"2024-09-10T15:57:44.765060+0000","last_peered":"2024-09-10T15:57:44.765060+0000","last_clean":"2024-09-10T15:57:44.765060+0000","last_became_active":"2024-09-10T15:57:41.766403+0000","last_became_peered":"2024-09-10T15:57:41.766403+0000","last_unstale":"2024-09-10T15:57:44.765060+0000","last_undegraded":"2024-09-10T15:57:44.765060+0000","last_fullsized":"2024-09-10T15:57:44.765060+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:17:45.980711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":32,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T15:57:44.764010+0000","last_change":"2024-09-10T15:57:33.885317+0000","last_active":"2024-09-10T15:57:44.764010+0000","last_peered":"2024-09-10T15:57:44.764010+0000","last_clean":"2024-09-10T15:57:44.764010+0000","last_became_active":"2024-09-10T15:57:31.880838+0000","last_became_peered":"2024-09-10T15:57:31.880838+0000","last_unstale":"2024-09-10T15:57:44.764010+0000","last_undegraded":"2024-09-10T15:57:44.764010+0000","last_fullsized":"2024-09-10T15:57:44.764010+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:11:25.004601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":30,"reported_epoch":19,"state":"peering","last_fresh":"2024-09-10T15:57:43.760863+0000","last_change":"2024-09-10T15:57:43.760863+0000","last_active":"2024-09-10T15:57:43.760863+0000","last_peered":"2024-09-10T15:57:43.258029+0000","last_clean":"2024-09-10T15:57:43.258029+0000","last_became_active":"2024-09-10T15:57:31.879578+0000","last_became_peered":"2024-09-10T15:57:31.879578+0000","last_unstale":"2024-09-10T15:57:43.760863+0000","last_undegraded":"2024-09-10T15:57:43.760863+0000","last_fullsized":"2024-09-10T15:57:43.760863+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"20'1","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T15:57:44.763691+0000","last_change":"2024-09-10T15:57:41.766568+0000","last_active":"2024-09-10T15:57:44.763691+0000","last_peered":"2024-09-10T15:57:44.763691+0000","last_clean":"2024-09-10T15:57:44.763691+0000","last_became_active":"2024-09-10T15:57:41.766163+0000","last_became_peered":"2024-09-10T15:57:41.766163+0000","last_unstale":"2024-09-10T15:57:44.763691+0000","last_undegraded":"2024-09-10T15:57:44.763691+0000","last_fullsized":"2024-09-10T15:57:44.763691+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:33:49.375071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705670,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27192,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344648,"statfs":{"total":96636764160,"available":96608919552,"internally_reserved":0,"allocated":167936,"data_stored":43530,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705672,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27212,"kb_used_data":176,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344628,"statfs":{"total":96636764160,"available":96608899072,"internally_reserved":0,"allocated":180224,"data_stored":46785,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:57:46.923 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:57:47.205 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T15:57:47.206 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T15:57:47.217 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":29,"stamp":"2024-09-10T15:57:45.759948+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":36,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54404,"kb_used_data":340,"kb_used_omap":3,"kb_used_meta":54012,"kb_avail":188689276,"statfs":{"total":193273528320,"available":193217818624,"internally_reserved":0,"allocated":348160,"data_stored":90315,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3190,"internal_metadata":55309194},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.006171"},"pg_stats":[{"pgid":"2.f","version":"0'0","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T15:57:44.765847+0000","last_change":"2024-09-10T15:57:41.762176+0000","last_active":"2024-09-10T15:57:44.765847+0000","last_peered":"2024-09-10T15:57:44.765847+0000","last_clean":"2024-09-10T15:57:44.765847+0000","last_became_active":"2024-09-10T15:57:41.760663+0000","last_became_peered":"2024-09-10T15:57:41.760663+0000","last_unstale":"2024-09-10T15:57:44.765847+0000","last_undegraded":"2024-09-10T15:57:44.765847+0000","last_fullsized":"2024-09-10T15:57:44.765847+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:40:37.854184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"peering","last_fresh":"2024-09-10T15:57:44.765540+0000","last_change":"2024-09-10T15:57:43.760533+0000","last_active":"2024-09-10T15:57:43.760533+0000","last_peered":"2024-09-10T15:57:41.764794+0000","last_clean":"2024-09-10T15:57:40.749557+0000","last_became_active":"2024-09-10T15:57:41.764602+0000","last_became_peered":"2024-09-10T15:57:41.764602+0000","last_unstale":"2024-09-10T15:57:44.765540+0000","last_undegraded":"2024-09-10T15:57:44.765540+0000","last_fullsized":"2024-09-10T15:57:44.765540+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":15,"reported_epoch":19,"state":"peering","last_fresh":"2024-09-10T15:57:43.760963+0000","last_change":"2024-09-10T15:57:43.760963+0000","last_active":"2024-09-10T15:57:43.760963+0000","last_peered":"2024-09-10T15:57:43.258110+0000","last_clean":"2024-09-10T15:57:43.258110+0000","last_became_active":"2024-09-10T15:57:41.761703+0000","last_became_peered":"2024-09-10T15:57:41.761703+0000","last_unstale":"2024-09-10T15:57:43.760963+0000","last_undegraded":"2024-09-10T15:57:43.760963+0000","last_fullsized":"2024-09-10T15:57:43.760963+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"peering","last_fresh":"2024-09-10T15:57:44.764624+0000","last_change":"2024-09-10T15:57:43.760175+0000","last_active":"2024-09-10T15:57:43.760175+0000","last_peered":"2024-09-10T15:57:41.765363+0000","last_clean":"2024-09-10T15:57:40.749557+0000","last_became_active":"2024-09-10T15:57:41.765168+0000","last_became_peered":"2024-09-10T15:57:41.765168+0000","last_unstale":"2024-09-10T15:57:44.764624+0000","last_undegraded":"2024-09-10T15:57:44.764624+0000","last_fullsized":"2024-09-10T15:57:44.764624+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":15,"reported_epoch":19,"state":"peering","last_fresh":"2024-09-10T15:57:43.759943+0000","last_change":"2024-09-10T15:57:43.759943+0000","last_active":"2024-09-10T15:57:43.759943+0000","last_peered":"2024-09-10T15:57:42.760642+0000","last_clean":"2024-09-10T15:57:42.760642+0000","last_became_active":"2024-09-10T15:57:41.762101+0000","last_became_peered":"2024-09-10T15:57:41.762101+0000","last_unstale":"2024-09-10T15:57:43.759943+0000","last_undegraded":"2024-09-10T15:57:43.759943+0000","last_fullsized":"2024-09-10T15:57:43.759943+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T15:57:44.763488+0000","last_change":"2024-09-10T15:57:41.766809+0000","last_active":"2024-09-10T15:57:44.763488+0000","last_peered":"2024-09-10T15:57:44.763488+0000","last_clean":"2024-09-10T15:57:44.763488+0000","last_became_active":"2024-09-10T15:57:41.766519+0000","last_became_peered":"2024-09-10T15:57:41.766519+0000","last_unstale":"2024-09-10T15:57:44.763488+0000","last_undegraded":"2024-09-10T15:57:44.763488+0000","last_fullsized":"2024-09-10T15:57:44.763488+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:19:07.895620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T15:57:44.764235+0000","last_change":"2024-09-10T15:57:41.766524+0000","last_active":"2024-09-10T15:57:44.764235+0000","last_peered":"2024-09-10T15:57:44.764235+0000","last_clean":"2024-09-10T15:57:44.764235+0000","last_became_active":"2024-09-10T15:57:41.766212+0000","last_became_peered":"2024-09-10T15:57:41.766212+0000","last_unstale":"2024-09-10T15:57:44.764235+0000","last_undegraded":"2024-09-10T15:57:44.764235+0000","last_fullsized":"2024-09-10T15:57:44.764235+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:45:52.388962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":15,"reported_epoch":19,"state":"peering","last_fresh":"2024-09-10T15:57:43.760833+0000","last_change":"2024-09-10T15:57:43.760833+0000","last_active":"2024-09-10T15:57:43.760833+0000","last_peered":"2024-09-10T15:57:42.760798+0000","last_clean":"2024-09-10T15:57:42.760798+0000","last_became_active":"2024-09-10T15:57:41.761869+0000","last_became_peered":"2024-09-10T15:57:41.761869+0000","last_unstale":"2024-09-10T15:57:43.760833+0000","last_undegraded":"2024-09-10T15:57:43.760833+0000","last_fullsized":"2024-09-10T15:57:43.760833+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":32,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T15:57:44.763896+0000","last_change":"2024-09-10T15:57:33.885856+0000","last_active":"2024-09-10T15:57:44.763896+0000","last_peered":"2024-09-10T15:57:44.763896+0000","last_clean":"2024-09-10T15:57:44.763896+0000","last_became_active":"2024-09-10T15:57:31.880722+0000","last_became_peered":"2024-09-10T15:57:31.880722+0000","last_unstale":"2024-09-10T15:57:44.763896+0000","last_undegraded":"2024-09-10T15:57:44.763896+0000","last_fullsized":"2024-09-10T15:57:44.763896+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:58:36.899441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":15,"reported_epoch":19,"state":"peering","last_fresh":"2024-09-10T15:57:43.760764+0000","last_change":"2024-09-10T15:57:43.760764+0000","last_active":"2024-09-10T15:57:43.760764+0000","last_peered":"2024-09-10T15:57:43.257970+0000","last_clean":"2024-09-10T15:57:43.257970+0000","last_became_active":"2024-09-10T15:57:41.761686+0000","last_became_peered":"2024-09-10T15:57:41.761686+0000","last_unstale":"2024-09-10T15:57:43.760764+0000","last_undegraded":"2024-09-10T15:57:43.760764+0000","last_fullsized":"2024-09-10T15:57:43.760764+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":23,"reported_epoch":20,"state":"peering","last_fresh":"2024-09-10T15:57:44.764420+0000","last_change":"2024-09-10T15:57:43.761472+0000","last_active":"2024-09-10T15:57:43.761472+0000","last_peered":"2024-09-10T15:57:34.057087+0000","last_clean":"2024-09-10T15:57:34.057087+0000","last_became_active":"2024-09-10T15:57:31.871233+0000","last_became_peered":"2024-09-10T15:57:31.871233+0000","last_unstale":"2024-09-10T15:57:44.764420+0000","last_undegraded":"2024-09-10T15:57:44.764420+0000","last_fullsized":"2024-09-10T15:57:44.764420+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T15:57:44.764182+0000","last_change":"2024-09-10T15:57:41.763096+0000","last_active":"2024-09-10T15:57:44.764182+0000","last_peered":"2024-09-10T15:57:44.764182+0000","last_clean":"2024-09-10T15:57:44.764182+0000","last_became_active":"2024-09-10T15:57:41.760684+0000","last_became_peered":"2024-09-10T15:57:41.760684+0000","last_unstale":"2024-09-10T15:57:44.764182+0000","last_undegraded":"2024-09-10T15:57:44.764182+0000","last_fullsized":"2024-09-10T15:57:44.764182+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:05:02.296984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":34,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T15:57:44.765507+0000","last_change":"2024-09-10T15:57:33.885956+0000","last_active":"2024-09-10T15:57:44.765507+0000","last_peered":"2024-09-10T15:57:44.765507+0000","last_clean":"2024-09-10T15:57:44.765507+0000","last_became_active":"2024-09-10T15:57:31.879977+0000","last_became_peered":"2024-09-10T15:57:31.879977+0000","last_unstale":"2024-09-10T15:57:44.765507+0000","last_undegraded":"2024-09-10T15:57:44.765507+0000","last_fullsized":"2024-09-10T15:57:44.765507+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:19:02.673743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"peering","last_fresh":"2024-09-10T15:57:44.764108+0000","last_change":"2024-09-10T15:57:43.759828+0000","last_active":"2024-09-10T15:57:43.759828+0000","last_peered":"2024-09-10T15:57:41.762953+0000","last_clean":"2024-09-10T15:57:40.749557+0000","last_became_active":"2024-09-10T15:57:41.762752+0000","last_became_peered":"2024-09-10T15:57:41.762752+0000","last_unstale":"2024-09-10T15:57:44.764108+0000","last_undegraded":"2024-09-10T15:57:44.764108+0000","last_fullsized":"2024-09-10T15:57:44.764108+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":31,"reported_epoch":19,"state":"peering","last_fresh":"2024-09-10T15:57:43.761000+0000","last_change":"2024-09-10T15:57:43.761000+0000","last_active":"2024-09-10T15:57:43.761000+0000","last_peered":"2024-09-10T15:57:43.257847+0000","last_clean":"2024-09-10T15:57:43.257847+0000","last_became_active":"2024-09-10T15:57:31.879568+0000","last_became_peered":"2024-09-10T15:57:31.879568+0000","last_unstale":"2024-09-10T15:57:43.761000+0000","last_undegraded":"2024-09-10T15:57:43.761000+0000","last_fullsized":"2024-09-10T15:57:43.761000+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"peering","last_fresh":"2024-09-10T15:57:44.763658+0000","last_change":"2024-09-10T15:57:43.761079+0000","last_active":"2024-09-10T15:57:43.761079+0000","last_peered":"2024-09-10T15:57:41.764654+0000","last_clean":"2024-09-10T15:57:40.749557+0000","last_became_active":"2024-09-10T15:57:41.764460+0000","last_became_peered":"2024-09-10T15:57:41.764460+0000","last_unstale":"2024-09-10T15:57:44.763658+0000","last_undegraded":"2024-09-10T15:57:44.763658+0000","last_fullsized":"2024-09-10T15:57:44.763658+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T15:57:44.764399+0000","last_change":"2024-09-10T15:57:41.767200+0000","last_active":"2024-09-10T15:57:44.764399+0000","last_peered":"2024-09-10T15:57:44.764399+0000","last_clean":"2024-09-10T15:57:44.764399+0000","last_became_active":"2024-09-10T15:57:41.766595+0000","last_became_peered":"2024-09-10T15:57:41.766595+0000","last_unstale":"2024-09-10T15:57:44.764399+0000","last_undegraded":"2024-09-10T15:57:44.764399+0000","last_fullsized":"2024-09-10T15:57:44.764399+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:49:33.626681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":32,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T15:57:44.764090+0000","last_change":"2024-09-10T15:57:33.885087+0000","last_active":"2024-09-10T15:57:44.764090+0000","last_peered":"2024-09-10T15:57:44.764090+0000","last_clean":"2024-09-10T15:57:44.764090+0000","last_became_active":"2024-09-10T15:57:31.880825+0000","last_became_peered":"2024-09-10T15:57:31.880825+0000","last_unstale":"2024-09-10T15:57:44.764090+0000","last_undegraded":"2024-09-10T15:57:44.764090+0000","last_fullsized":"2024-09-10T15:57:44.764090+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:46:50.469478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T15:57:44.764169+0000","last_change":"2024-09-10T15:57:41.766037+0000","last_active":"2024-09-10T15:57:44.764169+0000","last_peered":"2024-09-10T15:57:44.764169+0000","last_clean":"2024-09-10T15:57:44.764169+0000","last_became_active":"2024-09-10T15:57:41.765714+0000","last_became_peered":"2024-09-10T15:57:41.765714+0000","last_unstale":"2024-09-10T15:57:44.764169+0000","last_undegraded":"2024-09-10T15:57:44.764169+0000","last_fullsized":"2024-09-10T15:57:44.764169+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:54:41.289541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":23,"reported_epoch":20,"state":"peering","last_fresh":"2024-09-10T15:57:44.765984+0000","last_change":"2024-09-10T15:57:43.760004+0000","last_active":"2024-09-10T15:57:43.760004+0000","last_peered":"2024-09-10T15:57:34.057084+0000","last_clean":"2024-09-10T15:57:34.057084+0000","last_became_active":"2024-09-10T15:57:31.874099+0000","last_became_peered":"2024-09-10T15:57:31.874099+0000","last_unstale":"2024-09-10T15:57:44.765984+0000","last_undegraded":"2024-09-10T15:57:44.765984+0000","last_fullsized":"2024-09-10T15:57:44.765984+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T15:57:44.765060+0000","last_change":"2024-09-10T15:57:41.766737+0000","last_active":"2024-09-10T15:57:44.765060+0000","last_peered":"2024-09-10T15:57:44.765060+0000","last_clean":"2024-09-10T15:57:44.765060+0000","last_became_active":"2024-09-10T15:57:41.766403+0000","last_became_peered":"2024-09-10T15:57:41.766403+0000","last_unstale":"2024-09-10T15:57:44.765060+0000","last_undegraded":"2024-09-10T15:57:44.765060+0000","last_fullsized":"2024-09-10T15:57:44.765060+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:17:45.980711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":32,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T15:57:44.764010+0000","last_change":"2024-09-10T15:57:33.885317+0000","last_active":"2024-09-10T15:57:44.764010+0000","last_peered":"2024-09-10T15:57:44.764010+0000","last_clean":"2024-09-10T15:57:44.764010+0000","last_became_active":"2024-09-10T15:57:31.880838+0000","last_became_peered":"2024-09-10T15:57:31.880838+0000","last_unstale":"2024-09-10T15:57:44.764010+0000","last_undegraded":"2024-09-10T15:57:44.764010+0000","last_fullsized":"2024-09-10T15:57:44.764010+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:11:25.004601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":30,"reported_epoch":19,"state":"peering","last_fresh":"2024-09-10T15:57:43.760863+0000","last_change":"2024-09-10T15:57:43.760863+0000","last_active":"2024-09-10T15:57:43.760863+0000","last_peered":"2024-09-10T15:57:43.258029+0000","last_clean":"2024-09-10T15:57:43.258029+0000","last_became_active":"2024-09-10T15:57:31.879578+0000","last_became_peered":"2024-09-10T15:57:31.879578+0000","last_unstale":"2024-09-10T15:57:43.760863+0000","last_undegraded":"2024-09-10T15:57:43.760863+0000","last_fullsized":"2024-09-10T15:57:43.760863+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"20'1","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T15:57:44.763691+0000","last_change":"2024-09-10T15:57:41.766568+0000","last_active":"2024-09-10T15:57:44.763691+0000","last_peered":"2024-09-10T15:57:44.763691+0000","last_clean":"2024-09-10T15:57:44.763691+0000","last_became_active":"2024-09-10T15:57:41.766163+0000","last_became_peered":"2024-09-10T15:57:41.766163+0000","last_unstale":"2024-09-10T15:57:44.763691+0000","last_undegraded":"2024-09-10T15:57:44.763691+0000","last_fullsized":"2024-09-10T15:57:44.763691+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:33:49.375071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705670,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27192,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344648,"statfs":{"total":96636764160,"available":96608919552,"internally_reserved":0,"allocated":167936,"data_stored":43530,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705672,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27212,"kb_used_data":176,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344628,"statfs":{"total":96636764160,"available":96608899072,"internally_reserved":0,"allocated":180224,"data_stored":46785,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:57:47.219 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:57:47.499 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T15:57:47.500 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T15:57:47.511 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":29,"stamp":"2024-09-10T15:57:45.759948+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":36,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54404,"kb_used_data":340,"kb_used_omap":3,"kb_used_meta":54012,"kb_avail":188689276,"statfs":{"total":193273528320,"available":193217818624,"internally_reserved":0,"allocated":348160,"data_stored":90315,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3190,"internal_metadata":55309194},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.006171"},"pg_stats":[{"pgid":"2.f","version":"0'0","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T15:57:44.765847+0000","last_change":"2024-09-10T15:57:41.762176+0000","last_active":"2024-09-10T15:57:44.765847+0000","last_peered":"2024-09-10T15:57:44.765847+0000","last_clean":"2024-09-10T15:57:44.765847+0000","last_became_active":"2024-09-10T15:57:41.760663+0000","last_became_peered":"2024-09-10T15:57:41.760663+0000","last_unstale":"2024-09-10T15:57:44.765847+0000","last_undegraded":"2024-09-10T15:57:44.765847+0000","last_fullsized":"2024-09-10T15:57:44.765847+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:40:37.854184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"peering","last_fresh":"2024-09-10T15:57:44.765540+0000","last_change":"2024-09-10T15:57:43.760533+0000","last_active":"2024-09-10T15:57:43.760533+0000","last_peered":"2024-09-10T15:57:41.764794+0000","last_clean":"2024-09-10T15:57:40.749557+0000","last_became_active":"2024-09-10T15:57:41.764602+0000","last_became_peered":"2024-09-10T15:57:41.764602+0000","last_unstale":"2024-09-10T15:57:44.765540+0000","last_undegraded":"2024-09-10T15:57:44.765540+0000","last_fullsized":"2024-09-10T15:57:44.765540+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":15,"reported_epoch":19,"state":"peering","last_fresh":"2024-09-10T15:57:43.760963+0000","last_change":"2024-09-10T15:57:43.760963+0000","last_active":"2024-09-10T15:57:43.760963+0000","last_peered":"2024-09-10T15:57:43.258110+0000","last_clean":"2024-09-10T15:57:43.258110+0000","last_became_active":"2024-09-10T15:57:41.761703+0000","last_became_peered":"2024-09-10T15:57:41.761703+0000","last_unstale":"2024-09-10T15:57:43.760963+0000","last_undegraded":"2024-09-10T15:57:43.760963+0000","last_fullsized":"2024-09-10T15:57:43.760963+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"peering","last_fresh":"2024-09-10T15:57:44.764624+0000","last_change":"2024-09-10T15:57:43.760175+0000","last_active":"2024-09-10T15:57:43.760175+0000","last_peered":"2024-09-10T15:57:41.765363+0000","last_clean":"2024-09-10T15:57:40.749557+0000","last_became_active":"2024-09-10T15:57:41.765168+0000","last_became_peered":"2024-09-10T15:57:41.765168+0000","last_unstale":"2024-09-10T15:57:44.764624+0000","last_undegraded":"2024-09-10T15:57:44.764624+0000","last_fullsized":"2024-09-10T15:57:44.764624+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":15,"reported_epoch":19,"state":"peering","last_fresh":"2024-09-10T15:57:43.759943+0000","last_change":"2024-09-10T15:57:43.759943+0000","last_active":"2024-09-10T15:57:43.759943+0000","last_peered":"2024-09-10T15:57:42.760642+0000","last_clean":"2024-09-10T15:57:42.760642+0000","last_became_active":"2024-09-10T15:57:41.762101+0000","last_became_peered":"2024-09-10T15:57:41.762101+0000","last_unstale":"2024-09-10T15:57:43.759943+0000","last_undegraded":"2024-09-10T15:57:43.759943+0000","last_fullsized":"2024-09-10T15:57:43.759943+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T15:57:44.763488+0000","last_change":"2024-09-10T15:57:41.766809+0000","last_active":"2024-09-10T15:57:44.763488+0000","last_peered":"2024-09-10T15:57:44.763488+0000","last_clean":"2024-09-10T15:57:44.763488+0000","last_became_active":"2024-09-10T15:57:41.766519+0000","last_became_peered":"2024-09-10T15:57:41.766519+0000","last_unstale":"2024-09-10T15:57:44.763488+0000","last_undegraded":"2024-09-10T15:57:44.763488+0000","last_fullsized":"2024-09-10T15:57:44.763488+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:19:07.895620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T15:57:44.764235+0000","last_change":"2024-09-10T15:57:41.766524+0000","last_active":"2024-09-10T15:57:44.764235+0000","last_peered":"2024-09-10T15:57:44.764235+0000","last_clean":"2024-09-10T15:57:44.764235+0000","last_became_active":"2024-09-10T15:57:41.766212+0000","last_became_peered":"2024-09-10T15:57:41.766212+0000","last_unstale":"2024-09-10T15:57:44.764235+0000","last_undegraded":"2024-09-10T15:57:44.764235+0000","last_fullsized":"2024-09-10T15:57:44.764235+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:45:52.388962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":15,"reported_epoch":19,"state":"peering","last_fresh":"2024-09-10T15:57:43.760833+0000","last_change":"2024-09-10T15:57:43.760833+0000","last_active":"2024-09-10T15:57:43.760833+0000","last_peered":"2024-09-10T15:57:42.760798+0000","last_clean":"2024-09-10T15:57:42.760798+0000","last_became_active":"2024-09-10T15:57:41.761869+0000","last_became_peered":"2024-09-10T15:57:41.761869+0000","last_unstale":"2024-09-10T15:57:43.760833+0000","last_undegraded":"2024-09-10T15:57:43.760833+0000","last_fullsized":"2024-09-10T15:57:43.760833+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":32,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T15:57:44.763896+0000","last_change":"2024-09-10T15:57:33.885856+0000","last_active":"2024-09-10T15:57:44.763896+0000","last_peered":"2024-09-10T15:57:44.763896+0000","last_clean":"2024-09-10T15:57:44.763896+0000","last_became_active":"2024-09-10T15:57:31.880722+0000","last_became_peered":"2024-09-10T15:57:31.880722+0000","last_unstale":"2024-09-10T15:57:44.763896+0000","last_undegraded":"2024-09-10T15:57:44.763896+0000","last_fullsized":"2024-09-10T15:57:44.763896+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:58:36.899441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":15,"reported_epoch":19,"state":"peering","last_fresh":"2024-09-10T15:57:43.760764+0000","last_change":"2024-09-10T15:57:43.760764+0000","last_active":"2024-09-10T15:57:43.760764+0000","last_peered":"2024-09-10T15:57:43.257970+0000","last_clean":"2024-09-10T15:57:43.257970+0000","last_became_active":"2024-09-10T15:57:41.761686+0000","last_became_peered":"2024-09-10T15:57:41.761686+0000","last_unstale":"2024-09-10T15:57:43.760764+0000","last_undegraded":"2024-09-10T15:57:43.760764+0000","last_fullsized":"2024-09-10T15:57:43.760764+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":23,"reported_epoch":20,"state":"peering","last_fresh":"2024-09-10T15:57:44.764420+0000","last_change":"2024-09-10T15:57:43.761472+0000","last_active":"2024-09-10T15:57:43.761472+0000","last_peered":"2024-09-10T15:57:34.057087+0000","last_clean":"2024-09-10T15:57:34.057087+0000","last_became_active":"2024-09-10T15:57:31.871233+0000","last_became_peered":"2024-09-10T15:57:31.871233+0000","last_unstale":"2024-09-10T15:57:44.764420+0000","last_undegraded":"2024-09-10T15:57:44.764420+0000","last_fullsized":"2024-09-10T15:57:44.764420+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T15:57:44.764182+0000","last_change":"2024-09-10T15:57:41.763096+0000","last_active":"2024-09-10T15:57:44.764182+0000","last_peered":"2024-09-10T15:57:44.764182+0000","last_clean":"2024-09-10T15:57:44.764182+0000","last_became_active":"2024-09-10T15:57:41.760684+0000","last_became_peered":"2024-09-10T15:57:41.760684+0000","last_unstale":"2024-09-10T15:57:44.764182+0000","last_undegraded":"2024-09-10T15:57:44.764182+0000","last_fullsized":"2024-09-10T15:57:44.764182+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:05:02.296984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":34,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T15:57:44.765507+0000","last_change":"2024-09-10T15:57:33.885956+0000","last_active":"2024-09-10T15:57:44.765507+0000","last_peered":"2024-09-10T15:57:44.765507+0000","last_clean":"2024-09-10T15:57:44.765507+0000","last_became_active":"2024-09-10T15:57:31.879977+0000","last_became_peered":"2024-09-10T15:57:31.879977+0000","last_unstale":"2024-09-10T15:57:44.765507+0000","last_undegraded":"2024-09-10T15:57:44.765507+0000","last_fullsized":"2024-09-10T15:57:44.765507+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:19:02.673743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"peering","last_fresh":"2024-09-10T15:57:44.764108+0000","last_change":"2024-09-10T15:57:43.759828+0000","last_active":"2024-09-10T15:57:43.759828+0000","last_peered":"2024-09-10T15:57:41.762953+0000","last_clean":"2024-09-10T15:57:40.749557+0000","last_became_active":"2024-09-10T15:57:41.762752+0000","last_became_peered":"2024-09-10T15:57:41.762752+0000","last_unstale":"2024-09-10T15:57:44.764108+0000","last_undegraded":"2024-09-10T15:57:44.764108+0000","last_fullsized":"2024-09-10T15:57:44.764108+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":31,"reported_epoch":19,"state":"peering","last_fresh":"2024-09-10T15:57:43.761000+0000","last_change":"2024-09-10T15:57:43.761000+0000","last_active":"2024-09-10T15:57:43.761000+0000","last_peered":"2024-09-10T15:57:43.257847+0000","last_clean":"2024-09-10T15:57:43.257847+0000","last_became_active":"2024-09-10T15:57:31.879568+0000","last_became_peered":"2024-09-10T15:57:31.879568+0000","last_unstale":"2024-09-10T15:57:43.761000+0000","last_undegraded":"2024-09-10T15:57:43.761000+0000","last_fullsized":"2024-09-10T15:57:43.761000+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"peering","last_fresh":"2024-09-10T15:57:44.763658+0000","last_change":"2024-09-10T15:57:43.761079+0000","last_active":"2024-09-10T15:57:43.761079+0000","last_peered":"2024-09-10T15:57:41.764654+0000","last_clean":"2024-09-10T15:57:40.749557+0000","last_became_active":"2024-09-10T15:57:41.764460+0000","last_became_peered":"2024-09-10T15:57:41.764460+0000","last_unstale":"2024-09-10T15:57:44.763658+0000","last_undegraded":"2024-09-10T15:57:44.763658+0000","last_fullsized":"2024-09-10T15:57:44.763658+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T15:57:44.764399+0000","last_change":"2024-09-10T15:57:41.767200+0000","last_active":"2024-09-10T15:57:44.764399+0000","last_peered":"2024-09-10T15:57:44.764399+0000","last_clean":"2024-09-10T15:57:44.764399+0000","last_became_active":"2024-09-10T15:57:41.766595+0000","last_became_peered":"2024-09-10T15:57:41.766595+0000","last_unstale":"2024-09-10T15:57:44.764399+0000","last_undegraded":"2024-09-10T15:57:44.764399+0000","last_fullsized":"2024-09-10T15:57:44.764399+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:49:33.626681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":32,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T15:57:44.764090+0000","last_change":"2024-09-10T15:57:33.885087+0000","last_active":"2024-09-10T15:57:44.764090+0000","last_peered":"2024-09-10T15:57:44.764090+0000","last_clean":"2024-09-10T15:57:44.764090+0000","last_became_active":"2024-09-10T15:57:31.880825+0000","last_became_peered":"2024-09-10T15:57:31.880825+0000","last_unstale":"2024-09-10T15:57:44.764090+0000","last_undegraded":"2024-09-10T15:57:44.764090+0000","last_fullsized":"2024-09-10T15:57:44.764090+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:46:50.469478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T15:57:44.764169+0000","last_change":"2024-09-10T15:57:41.766037+0000","last_active":"2024-09-10T15:57:44.764169+0000","last_peered":"2024-09-10T15:57:44.764169+0000","last_clean":"2024-09-10T15:57:44.764169+0000","last_became_active":"2024-09-10T15:57:41.765714+0000","last_became_peered":"2024-09-10T15:57:41.765714+0000","last_unstale":"2024-09-10T15:57:44.764169+0000","last_undegraded":"2024-09-10T15:57:44.764169+0000","last_fullsized":"2024-09-10T15:57:44.764169+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:54:41.289541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":23,"reported_epoch":20,"state":"peering","last_fresh":"2024-09-10T15:57:44.765984+0000","last_change":"2024-09-10T15:57:43.760004+0000","last_active":"2024-09-10T15:57:43.760004+0000","last_peered":"2024-09-10T15:57:34.057084+0000","last_clean":"2024-09-10T15:57:34.057084+0000","last_became_active":"2024-09-10T15:57:31.874099+0000","last_became_peered":"2024-09-10T15:57:31.874099+0000","last_unstale":"2024-09-10T15:57:44.765984+0000","last_undegraded":"2024-09-10T15:57:44.765984+0000","last_fullsized":"2024-09-10T15:57:44.765984+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T15:57:44.765060+0000","last_change":"2024-09-10T15:57:41.766737+0000","last_active":"2024-09-10T15:57:44.765060+0000","last_peered":"2024-09-10T15:57:44.765060+0000","last_clean":"2024-09-10T15:57:44.765060+0000","last_became_active":"2024-09-10T15:57:41.766403+0000","last_became_peered":"2024-09-10T15:57:41.766403+0000","last_unstale":"2024-09-10T15:57:44.765060+0000","last_undegraded":"2024-09-10T15:57:44.765060+0000","last_fullsized":"2024-09-10T15:57:44.765060+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:17:45.980711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":32,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T15:57:44.764010+0000","last_change":"2024-09-10T15:57:33.885317+0000","last_active":"2024-09-10T15:57:44.764010+0000","last_peered":"2024-09-10T15:57:44.764010+0000","last_clean":"2024-09-10T15:57:44.764010+0000","last_became_active":"2024-09-10T15:57:31.880838+0000","last_became_peered":"2024-09-10T15:57:31.880838+0000","last_unstale":"2024-09-10T15:57:44.764010+0000","last_undegraded":"2024-09-10T15:57:44.764010+0000","last_fullsized":"2024-09-10T15:57:44.764010+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:11:25.004601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":30,"reported_epoch":19,"state":"peering","last_fresh":"2024-09-10T15:57:43.760863+0000","last_change":"2024-09-10T15:57:43.760863+0000","last_active":"2024-09-10T15:57:43.760863+0000","last_peered":"2024-09-10T15:57:43.258029+0000","last_clean":"2024-09-10T15:57:43.258029+0000","last_became_active":"2024-09-10T15:57:31.879578+0000","last_became_peered":"2024-09-10T15:57:31.879578+0000","last_unstale":"2024-09-10T15:57:43.760863+0000","last_undegraded":"2024-09-10T15:57:43.760863+0000","last_fullsized":"2024-09-10T15:57:43.760863+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"20'1","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T15:57:44.763691+0000","last_change":"2024-09-10T15:57:41.766568+0000","last_active":"2024-09-10T15:57:44.763691+0000","last_peered":"2024-09-10T15:57:44.763691+0000","last_clean":"2024-09-10T15:57:44.763691+0000","last_became_active":"2024-09-10T15:57:41.766163+0000","last_became_peered":"2024-09-10T15:57:41.766163+0000","last_unstale":"2024-09-10T15:57:44.763691+0000","last_undegraded":"2024-09-10T15:57:44.763691+0000","last_fullsized":"2024-09-10T15:57:44.763691+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:33:49.375071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705670,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27192,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344648,"statfs":{"total":96636764160,"available":96608919552,"internally_reserved":0,"allocated":167936,"data_stored":43530,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705672,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27212,"kb_used_data":176,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344628,"statfs":{"total":96636764160,"available":96608899072,"internally_reserved":0,"allocated":180224,"data_stored":46785,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:57:50.513 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:57:50.807 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T15:57:50.808 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T15:57:50.820 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":31,"stamp":"2024-09-10T15:57:49.527338+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":36,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54420,"kb_used_data":360,"kb_used_omap":3,"kb_used_meta":54012,"kb_avail":188689260,"statfs":{"total":193273528320,"available":193217802240,"internally_reserved":0,"allocated":368640,"data_stored":96284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3190,"internal_metadata":55309194},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.000896"},"pg_stats":[{"pgid":"2.f","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T15:57:45.762221+0000","last_change":"2024-09-10T15:57:41.762176+0000","last_active":"2024-09-10T15:57:45.762221+0000","last_peered":"2024-09-10T15:57:45.762221+0000","last_clean":"2024-09-10T15:57:45.762221+0000","last_became_active":"2024-09-10T15:57:41.760663+0000","last_became_peered":"2024-09-10T15:57:41.760663+0000","last_unstale":"2024-09-10T15:57:45.762221+0000","last_undegraded":"2024-09-10T15:57:45.762221+0000","last_fullsized":"2024-09-10T15:57:45.762221+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:40:37.854184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T15:57:45.766481+0000","last_change":"2024-09-10T15:57:45.766403+0000","last_active":"2024-09-10T15:57:45.766481+0000","last_peered":"2024-09-10T15:57:45.766481+0000","last_clean":"2024-09-10T15:57:45.766481+0000","last_became_active":"2024-09-10T15:57:45.766064+0000","last_became_peered":"2024-09-10T15:57:45.766064+0000","last_unstale":"2024-09-10T15:57:45.766481+0000","last_undegraded":"2024-09-10T15:57:45.766481+0000","last_fullsized":"2024-09-10T15:57:45.766481+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:40:18.422160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":24,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T15:57:45.768925+0000","last_change":"2024-09-10T15:57:45.768826+0000","last_active":"2024-09-10T15:57:45.768925+0000","last_peered":"2024-09-10T15:57:45.768925+0000","last_clean":"2024-09-10T15:57:45.768925+0000","last_became_active":"2024-09-10T15:57:45.768609+0000","last_became_peered":"2024-09-10T15:57:45.768609+0000","last_unstale":"2024-09-10T15:57:45.768925+0000","last_undegraded":"2024-09-10T15:57:45.768925+0000","last_fullsized":"2024-09-10T15:57:45.768925+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:06:30.271227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T15:57:45.766450+0000","last_change":"2024-09-10T15:57:45.766366+0000","last_active":"2024-09-10T15:57:45.766450+0000","last_peered":"2024-09-10T15:57:45.766450+0000","last_clean":"2024-09-10T15:57:45.766450+0000","last_became_active":"2024-09-10T15:57:45.765994+0000","last_became_peered":"2024-09-10T15:57:45.765994+0000","last_unstale":"2024-09-10T15:57:45.766450+0000","last_undegraded":"2024-09-10T15:57:45.766450+0000","last_fullsized":"2024-09-10T15:57:45.766450+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:57:15.624323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":24,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T15:57:45.766320+0000","last_change":"2024-09-10T15:57:45.766256+0000","last_active":"2024-09-10T15:57:45.766320+0000","last_peered":"2024-09-10T15:57:45.766320+0000","last_clean":"2024-09-10T15:57:45.766320+0000","last_became_active":"2024-09-10T15:57:45.765975+0000","last_became_peered":"2024-09-10T15:57:45.765975+0000","last_unstale":"2024-09-10T15:57:45.766320+0000","last_undegraded":"2024-09-10T15:57:45.766320+0000","last_fullsized":"2024-09-10T15:57:45.766320+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:10:04.584960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T15:57:45.762608+0000","last_change":"2024-09-10T15:57:41.766809+0000","last_active":"2024-09-10T15:57:45.762608+0000","last_peered":"2024-09-10T15:57:45.762608+0000","last_clean":"2024-09-10T15:57:45.762608+0000","last_became_active":"2024-09-10T15:57:41.766519+0000","last_became_peered":"2024-09-10T15:57:41.766519+0000","last_unstale":"2024-09-10T15:57:45.762608+0000","last_undegraded":"2024-09-10T15:57:45.762608+0000","last_fullsized":"2024-09-10T15:57:45.762608+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:19:07.895620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T15:57:45.762536+0000","last_change":"2024-09-10T15:57:41.766524+0000","last_active":"2024-09-10T15:57:45.762536+0000","last_peered":"2024-09-10T15:57:45.762536+0000","last_clean":"2024-09-10T15:57:45.762536+0000","last_became_active":"2024-09-10T15:57:41.766212+0000","last_became_peered":"2024-09-10T15:57:41.766212+0000","last_unstale":"2024-09-10T15:57:45.762536+0000","last_undegraded":"2024-09-10T15:57:45.762536+0000","last_fullsized":"2024-09-10T15:57:45.762536+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:45:52.388962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":24,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T15:57:45.765242+0000","last_change":"2024-09-10T15:57:45.765149+0000","last_active":"2024-09-10T15:57:45.765242+0000","last_peered":"2024-09-10T15:57:45.765242+0000","last_clean":"2024-09-10T15:57:45.765242+0000","last_became_active":"2024-09-10T15:57:45.764811+0000","last_became_peered":"2024-09-10T15:57:45.764811+0000","last_unstale":"2024-09-10T15:57:45.765242+0000","last_undegraded":"2024-09-10T15:57:45.765242+0000","last_fullsized":"2024-09-10T15:57:45.765242+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:50:56.381675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":34,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T15:57:45.762410+0000","last_change":"2024-09-10T15:57:33.885856+0000","last_active":"2024-09-10T15:57:45.762410+0000","last_peered":"2024-09-10T15:57:45.762410+0000","last_clean":"2024-09-10T15:57:45.762410+0000","last_became_active":"2024-09-10T15:57:31.880722+0000","last_became_peered":"2024-09-10T15:57:31.880722+0000","last_unstale":"2024-09-10T15:57:45.762410+0000","last_undegraded":"2024-09-10T15:57:45.762410+0000","last_fullsized":"2024-09-10T15:57:45.762410+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:58:36.899441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":24,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T15:57:45.768416+0000","last_change":"2024-09-10T15:57:45.768321+0000","last_active":"2024-09-10T15:57:45.768416+0000","last_peered":"2024-09-10T15:57:45.768416+0000","last_clean":"2024-09-10T15:57:45.768416+0000","last_became_active":"2024-09-10T15:57:45.768039+0000","last_became_peered":"2024-09-10T15:57:45.768039+0000","last_unstale":"2024-09-10T15:57:45.768416+0000","last_undegraded":"2024-09-10T15:57:45.768416+0000","last_fullsized":"2024-09-10T15:57:45.768416+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:56:47.532336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":32,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T15:57:45.765248+0000","last_change":"2024-09-10T15:57:45.765164+0000","last_active":"2024-09-10T15:57:45.765248+0000","last_peered":"2024-09-10T15:57:45.765248+0000","last_clean":"2024-09-10T15:57:45.765248+0000","last_became_active":"2024-09-10T15:57:45.764875+0000","last_became_peered":"2024-09-10T15:57:45.764875+0000","last_unstale":"2024-09-10T15:57:45.765248+0000","last_undegraded":"2024-09-10T15:57:45.765248+0000","last_fullsized":"2024-09-10T15:57:45.765248+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:42:35.732263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T15:57:45.761820+0000","last_change":"2024-09-10T15:57:41.763096+0000","last_active":"2024-09-10T15:57:45.761820+0000","last_peered":"2024-09-10T15:57:45.761820+0000","last_clean":"2024-09-10T15:57:45.761820+0000","last_became_active":"2024-09-10T15:57:41.760684+0000","last_became_peered":"2024-09-10T15:57:41.760684+0000","last_unstale":"2024-09-10T15:57:45.761820+0000","last_undegraded":"2024-09-10T15:57:45.761820+0000","last_fullsized":"2024-09-10T15:57:45.761820+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:05:02.296984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":36,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T15:57:45.761911+0000","last_change":"2024-09-10T15:57:33.885956+0000","last_active":"2024-09-10T15:57:45.761911+0000","last_peered":"2024-09-10T15:57:45.761911+0000","last_clean":"2024-09-10T15:57:45.761911+0000","last_became_active":"2024-09-10T15:57:31.879977+0000","last_became_peered":"2024-09-10T15:57:31.879977+0000","last_unstale":"2024-09-10T15:57:45.761911+0000","last_undegraded":"2024-09-10T15:57:45.761911+0000","last_fullsized":"2024-09-10T15:57:45.761911+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:19:02.673743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T15:57:45.768399+0000","last_change":"2024-09-10T15:57:45.768312+0000","last_active":"2024-09-10T15:57:45.768399+0000","last_peered":"2024-09-10T15:57:45.768399+0000","last_clean":"2024-09-10T15:57:45.768399+0000","last_became_active":"2024-09-10T15:57:45.768054+0000","last_became_peered":"2024-09-10T15:57:45.768054+0000","last_unstale":"2024-09-10T15:57:45.768399+0000","last_undegraded":"2024-09-10T15:57:45.768399+0000","last_fullsized":"2024-09-10T15:57:45.768399+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:48:45.280303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":44,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T15:57:45.773843+0000","last_change":"2024-09-10T15:57:45.773747+0000","last_active":"2024-09-10T15:57:45.773843+0000","last_peered":"2024-09-10T15:57:45.773843+0000","last_clean":"2024-09-10T15:57:45.773843+0000","last_became_active":"2024-09-10T15:57:45.770708+0000","last_became_peered":"2024-09-10T15:57:45.770708+0000","last_unstale":"2024-09-10T15:57:45.773843+0000","last_undegraded":"2024-09-10T15:57:45.773843+0000","last_fullsized":"2024-09-10T15:57:45.773843+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:20:09.863752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T15:57:45.765371+0000","last_change":"2024-09-10T15:57:45.765296+0000","last_active":"2024-09-10T15:57:45.765371+0000","last_peered":"2024-09-10T15:57:45.765371+0000","last_clean":"2024-09-10T15:57:45.765371+0000","last_became_active":"2024-09-10T15:57:45.765034+0000","last_became_peered":"2024-09-10T15:57:45.765034+0000","last_unstale":"2024-09-10T15:57:45.765371+0000","last_undegraded":"2024-09-10T15:57:45.765371+0000","last_fullsized":"2024-09-10T15:57:45.765371+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:13:59.862554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T15:57:45.762969+0000","last_change":"2024-09-10T15:57:41.767200+0000","last_active":"2024-09-10T15:57:45.762969+0000","last_peered":"2024-09-10T15:57:45.762969+0000","last_clean":"2024-09-10T15:57:45.762969+0000","last_became_active":"2024-09-10T15:57:41.766595+0000","last_became_peered":"2024-09-10T15:57:41.766595+0000","last_unstale":"2024-09-10T15:57:45.762969+0000","last_undegraded":"2024-09-10T15:57:45.762969+0000","last_fullsized":"2024-09-10T15:57:45.762969+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:49:33.626681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":34,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T15:57:45.762936+0000","last_change":"2024-09-10T15:57:33.885087+0000","last_active":"2024-09-10T15:57:45.762936+0000","last_peered":"2024-09-10T15:57:45.762936+0000","last_clean":"2024-09-10T15:57:45.762936+0000","last_became_active":"2024-09-10T15:57:31.880825+0000","last_became_peered":"2024-09-10T15:57:31.880825+0000","last_unstale":"2024-09-10T15:57:45.762936+0000","last_undegraded":"2024-09-10T15:57:45.762936+0000","last_fullsized":"2024-09-10T15:57:45.762936+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:46:50.469478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T15:57:45.762716+0000","last_change":"2024-09-10T15:57:41.766037+0000","last_active":"2024-09-10T15:57:45.762716+0000","last_peered":"2024-09-10T15:57:45.762716+0000","last_clean":"2024-09-10T15:57:45.762716+0000","last_became_active":"2024-09-10T15:57:41.765714+0000","last_became_peered":"2024-09-10T15:57:41.765714+0000","last_unstale":"2024-09-10T15:57:45.762716+0000","last_undegraded":"2024-09-10T15:57:45.762716+0000","last_fullsized":"2024-09-10T15:57:45.762716+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:54:41.289541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":32,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T15:57:45.766323+0000","last_change":"2024-09-10T15:57:45.766219+0000","last_active":"2024-09-10T15:57:45.766323+0000","last_peered":"2024-09-10T15:57:45.766323+0000","last_clean":"2024-09-10T15:57:45.766323+0000","last_became_active":"2024-09-10T15:57:45.765954+0000","last_became_peered":"2024-09-10T15:57:45.765954+0000","last_unstale":"2024-09-10T15:57:45.766323+0000","last_undegraded":"2024-09-10T15:57:45.766323+0000","last_fullsized":"2024-09-10T15:57:45.766323+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:16:27.482500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T15:57:45.763286+0000","last_change":"2024-09-10T15:57:41.766737+0000","last_active":"2024-09-10T15:57:45.763286+0000","last_peered":"2024-09-10T15:57:45.763286+0000","last_clean":"2024-09-10T15:57:45.763286+0000","last_became_active":"2024-09-10T15:57:41.766403+0000","last_became_peered":"2024-09-10T15:57:41.766403+0000","last_unstale":"2024-09-10T15:57:45.763286+0000","last_undegraded":"2024-09-10T15:57:45.763286+0000","last_fullsized":"2024-09-10T15:57:45.763286+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:17:45.980711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":34,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T15:57:45.762809+0000","last_change":"2024-09-10T15:57:33.885317+0000","last_active":"2024-09-10T15:57:45.762809+0000","last_peered":"2024-09-10T15:57:45.762809+0000","last_clean":"2024-09-10T15:57:45.762809+0000","last_became_active":"2024-09-10T15:57:31.880838+0000","last_became_peered":"2024-09-10T15:57:31.880838+0000","last_unstale":"2024-09-10T15:57:45.762809+0000","last_undegraded":"2024-09-10T15:57:45.762809+0000","last_fullsized":"2024-09-10T15:57:45.762809+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:11:25.004601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":39,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T15:57:45.768471+0000","last_change":"2024-09-10T15:57:45.768383+0000","last_active":"2024-09-10T15:57:45.768471+0000","last_peered":"2024-09-10T15:57:45.768471+0000","last_clean":"2024-09-10T15:57:45.768471+0000","last_became_active":"2024-09-10T15:57:45.768105+0000","last_became_peered":"2024-09-10T15:57:45.768105+0000","last_unstale":"2024-09-10T15:57:45.768471+0000","last_undegraded":"2024-09-10T15:57:45.768471+0000","last_fullsized":"2024-09-10T15:57:45.768471+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:39:43.515492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"20'1","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T15:57:45.762962+0000","last_change":"2024-09-10T15:57:41.766568+0000","last_active":"2024-09-10T15:57:45.762962+0000","last_peered":"2024-09-10T15:57:45.762962+0000","last_clean":"2024-09-10T15:57:45.762962+0000","last_became_active":"2024-09-10T15:57:41.766163+0000","last_became_peered":"2024-09-10T15:57:41.766163+0000","last_unstale":"2024-09-10T15:57:45.762962+0000","last_undegraded":"2024-09-10T15:57:45.762962+0000","last_fullsized":"2024-09-10T15:57:45.762962+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:33:49.375071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705671,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27208,"kb_used_data":176,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344632,"statfs":{"total":96636764160,"available":96608903168,"internally_reserved":0,"allocated":180224,"data_stored":46785,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705673,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27212,"kb_used_data":184,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344628,"statfs":{"total":96636764160,"available":96608899072,"internally_reserved":0,"allocated":188416,"data_stored":49499,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:57:50.821 INFO:tasks.lost_unfound.ceph_manager:recovered! 2024-09-10T15:57:50.821 INFO:tasks.util.rados:rados -p unfound_pool put existing_1 /etc/fstab 2024-09-10T15:57:50.821 DEBUG:teuthology.orchestra.run.smithi115:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool put existing_1 /etc/fstab 2024-09-10T15:57:50.871 INFO:tasks.util.rados:rados -p unfound_pool put existed_1 /etc/fstab 2024-09-10T15:57:50.871 DEBUG:teuthology.orchestra.run.smithi115:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool put existed_1 /etc/fstab 2024-09-10T15:57:50.921 INFO:tasks.util.rados:rados -p unfound_pool rm existed_1 2024-09-10T15:57:50.922 DEBUG:teuthology.orchestra.run.smithi115:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool rm existed_1 2024-09-10T15:57:50.971 INFO:tasks.util.rados:rados -p unfound_pool put existing_2 /etc/fstab 2024-09-10T15:57:50.971 DEBUG:teuthology.orchestra.run.smithi115:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool put existing_2 /etc/fstab 2024-09-10T15:57:51.021 INFO:tasks.util.rados:rados -p unfound_pool put existed_2 /etc/fstab 2024-09-10T15:57:51.021 DEBUG:teuthology.orchestra.run.smithi115:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool put existed_2 /etc/fstab 2024-09-10T15:57:51.071 INFO:tasks.util.rados:rados -p unfound_pool rm existed_2 2024-09-10T15:57:51.072 DEBUG:teuthology.orchestra.run.smithi115:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool rm existed_2 2024-09-10T15:57:51.120 INFO:tasks.util.rados:rados -p unfound_pool put existing_3 /etc/fstab 2024-09-10T15:57:51.121 DEBUG:teuthology.orchestra.run.smithi115:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool put existing_3 /etc/fstab 2024-09-10T15:57:51.170 INFO:tasks.util.rados:rados -p unfound_pool put existed_3 /etc/fstab 2024-09-10T15:57:51.171 DEBUG:teuthology.orchestra.run.smithi115:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool put existed_3 /etc/fstab 2024-09-10T15:57:51.220 INFO:tasks.util.rados:rados -p unfound_pool rm existed_3 2024-09-10T15:57:51.221 DEBUG:teuthology.orchestra.run.smithi115:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool rm existed_3 2024-09-10T15:57:51.269 INFO:tasks.util.rados:rados -p unfound_pool put existing_4 /etc/fstab 2024-09-10T15:57:51.269 DEBUG:teuthology.orchestra.run.smithi115:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool put existing_4 /etc/fstab 2024-09-10T15:57:51.320 INFO:tasks.util.rados:rados -p unfound_pool put existed_4 /etc/fstab 2024-09-10T15:57:51.320 DEBUG:teuthology.orchestra.run.smithi115:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool put existed_4 /etc/fstab 2024-09-10T15:57:51.371 INFO:tasks.util.rados:rados -p unfound_pool rm existed_4 2024-09-10T15:57:51.371 DEBUG:teuthology.orchestra.run.smithi115:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool rm existed_4 2024-09-10T15:57:51.420 INFO:tasks.util.rados:rados -p unfound_pool put existing_5 /etc/fstab 2024-09-10T15:57:51.420 DEBUG:teuthology.orchestra.run.smithi115:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool put existing_5 /etc/fstab 2024-09-10T15:57:51.470 INFO:tasks.util.rados:rados -p unfound_pool put existed_5 /etc/fstab 2024-09-10T15:57:51.470 DEBUG:teuthology.orchestra.run.smithi115:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool put existed_5 /etc/fstab 2024-09-10T15:57:51.520 INFO:tasks.util.rados:rados -p unfound_pool rm existed_5 2024-09-10T15:57:51.520 DEBUG:teuthology.orchestra.run.smithi115:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool rm existed_5 2024-09-10T15:57:51.569 INFO:tasks.util.rados:rados -p unfound_pool put existing_6 /etc/fstab 2024-09-10T15:57:51.569 DEBUG:teuthology.orchestra.run.smithi115:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool put existing_6 /etc/fstab 2024-09-10T15:57:51.619 INFO:tasks.util.rados:rados -p unfound_pool put existed_6 /etc/fstab 2024-09-10T15:57:51.619 DEBUG:teuthology.orchestra.run.smithi115:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool put existed_6 /etc/fstab 2024-09-10T15:57:51.669 INFO:tasks.util.rados:rados -p unfound_pool rm existed_6 2024-09-10T15:57:51.670 DEBUG:teuthology.orchestra.run.smithi115:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool rm existed_6 2024-09-10T15:57:51.718 INFO:tasks.util.rados:rados -p unfound_pool put existing_7 /etc/fstab 2024-09-10T15:57:51.718 DEBUG:teuthology.orchestra.run.smithi115:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool put existing_7 /etc/fstab 2024-09-10T15:57:51.768 INFO:tasks.util.rados:rados -p unfound_pool put existed_7 /etc/fstab 2024-09-10T15:57:51.768 DEBUG:teuthology.orchestra.run.smithi115:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool put existed_7 /etc/fstab 2024-09-10T15:57:51.818 INFO:tasks.util.rados:rados -p unfound_pool rm existed_7 2024-09-10T15:57:51.818 DEBUG:teuthology.orchestra.run.smithi115:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool rm existed_7 2024-09-10T15:57:51.866 INFO:tasks.util.rados:rados -p unfound_pool put existing_8 /etc/fstab 2024-09-10T15:57:51.866 DEBUG:teuthology.orchestra.run.smithi115:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool put existing_8 /etc/fstab 2024-09-10T15:57:51.916 INFO:tasks.util.rados:rados -p unfound_pool put existed_8 /etc/fstab 2024-09-10T15:57:51.917 DEBUG:teuthology.orchestra.run.smithi115:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool put existed_8 /etc/fstab 2024-09-10T15:57:51.967 INFO:tasks.util.rados:rados -p unfound_pool rm existed_8 2024-09-10T15:57:51.967 DEBUG:teuthology.orchestra.run.smithi115:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool rm existed_8 2024-09-10T15:57:52.016 INFO:tasks.util.rados:rados -p unfound_pool put existing_9 /etc/fstab 2024-09-10T15:57:52.017 DEBUG:teuthology.orchestra.run.smithi115:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool put existing_9 /etc/fstab 2024-09-10T15:57:52.068 INFO:tasks.util.rados:rados -p unfound_pool put existed_9 /etc/fstab 2024-09-10T15:57:52.068 DEBUG:teuthology.orchestra.run.smithi115:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool put existed_9 /etc/fstab 2024-09-10T15:57:52.121 INFO:tasks.util.rados:rados -p unfound_pool rm existed_9 2024-09-10T15:57:52.122 DEBUG:teuthology.orchestra.run.smithi115:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool rm existed_9 2024-09-10T15:57:52.170 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 injectargs '--osd-recovery-delay-start 1000 --osd-min-pg-log-entries 100000000' 2024-09-10T15:57:52.290 INFO:teuthology.orchestra.run.smithi115.stdout:{} 2024-09-10T15:57:52.290 INFO:teuthology.orchestra.run.smithi115.stderr:osd_min_pg_log_entries = '' (not observed, change may require restart) osd_recovery_delay_start = '' 2024-09-10T15:57:52.302 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-09-10T15:57:52.302 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T15:57:52.334 INFO:tasks.ceph.osd.0:Stopped 2024-09-10T15:57:52.335 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd down 0 2024-09-10T15:57:52.666 INFO:tasks.ceph.mon.a.smithi115.stderr:2024-09-10T15:57:52.665+0000 7fa206bdb640 -1 mon.a@0(leader).osd e21 definitely_dead 0 2024-09-10T15:57:52.933 INFO:teuthology.orchestra.run.smithi115.stderr:marked down osd.0. 2024-09-10T15:57:52.949 INFO:tasks.util.rados:rados -p unfound_pool put new_1 /etc/fstab 2024-09-10T15:57:52.949 DEBUG:teuthology.orchestra.run.smithi115:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool put new_1 /etc/fstab 2024-09-10T15:57:53.955 INFO:tasks.util.rados:rados -p unfound_pool put existed_1 /etc/fstab 2024-09-10T15:57:53.956 DEBUG:teuthology.orchestra.run.smithi115:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool put existed_1 /etc/fstab 2024-09-10T15:57:54.011 INFO:tasks.util.rados:rados -p unfound_pool put existing_1 /etc/fstab 2024-09-10T15:57:54.011 DEBUG:teuthology.orchestra.run.smithi115:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool put existing_1 /etc/fstab 2024-09-10T15:57:54.064 INFO:tasks.util.rados:rados -p unfound_pool put new_2 /etc/fstab 2024-09-10T15:57:54.064 DEBUG:teuthology.orchestra.run.smithi115:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool put new_2 /etc/fstab 2024-09-10T15:57:54.114 INFO:tasks.util.rados:rados -p unfound_pool put existed_2 /etc/fstab 2024-09-10T15:57:54.114 DEBUG:teuthology.orchestra.run.smithi115:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool put existed_2 /etc/fstab 2024-09-10T15:57:54.164 INFO:tasks.util.rados:rados -p unfound_pool put existing_2 /etc/fstab 2024-09-10T15:57:54.164 DEBUG:teuthology.orchestra.run.smithi115:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool put existing_2 /etc/fstab 2024-09-10T15:57:54.214 INFO:tasks.util.rados:rados -p unfound_pool put new_3 /etc/fstab 2024-09-10T15:57:54.214 DEBUG:teuthology.orchestra.run.smithi115:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool put new_3 /etc/fstab 2024-09-10T15:57:54.265 INFO:tasks.util.rados:rados -p unfound_pool put existed_3 /etc/fstab 2024-09-10T15:57:54.265 DEBUG:teuthology.orchestra.run.smithi115:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool put existed_3 /etc/fstab 2024-09-10T15:57:54.315 INFO:tasks.util.rados:rados -p unfound_pool put existing_3 /etc/fstab 2024-09-10T15:57:54.315 DEBUG:teuthology.orchestra.run.smithi115:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool put existing_3 /etc/fstab 2024-09-10T15:57:54.365 INFO:tasks.util.rados:rados -p unfound_pool put new_4 /etc/fstab 2024-09-10T15:57:54.365 DEBUG:teuthology.orchestra.run.smithi115:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool put new_4 /etc/fstab 2024-09-10T15:57:54.415 INFO:tasks.util.rados:rados -p unfound_pool put existed_4 /etc/fstab 2024-09-10T15:57:54.415 DEBUG:teuthology.orchestra.run.smithi115:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool put existed_4 /etc/fstab 2024-09-10T15:57:54.465 INFO:tasks.util.rados:rados -p unfound_pool put existing_4 /etc/fstab 2024-09-10T15:57:54.465 DEBUG:teuthology.orchestra.run.smithi115:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool put existing_4 /etc/fstab 2024-09-10T15:57:54.515 INFO:tasks.util.rados:rados -p unfound_pool put new_5 /etc/fstab 2024-09-10T15:57:54.515 DEBUG:teuthology.orchestra.run.smithi115:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool put new_5 /etc/fstab 2024-09-10T15:57:54.565 INFO:tasks.util.rados:rados -p unfound_pool put existed_5 /etc/fstab 2024-09-10T15:57:54.565 DEBUG:teuthology.orchestra.run.smithi115:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool put existed_5 /etc/fstab 2024-09-10T15:57:54.615 INFO:tasks.util.rados:rados -p unfound_pool put existing_5 /etc/fstab 2024-09-10T15:57:54.615 DEBUG:teuthology.orchestra.run.smithi115:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool put existing_5 /etc/fstab 2024-09-10T15:57:54.665 INFO:tasks.util.rados:rados -p unfound_pool put new_6 /etc/fstab 2024-09-10T15:57:54.665 DEBUG:teuthology.orchestra.run.smithi115:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool put new_6 /etc/fstab 2024-09-10T15:57:54.715 INFO:tasks.util.rados:rados -p unfound_pool put existed_6 /etc/fstab 2024-09-10T15:57:54.715 DEBUG:teuthology.orchestra.run.smithi115:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool put existed_6 /etc/fstab 2024-09-10T15:57:54.765 INFO:tasks.util.rados:rados -p unfound_pool put existing_6 /etc/fstab 2024-09-10T15:57:54.765 DEBUG:teuthology.orchestra.run.smithi115:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool put existing_6 /etc/fstab 2024-09-10T15:57:54.814 INFO:tasks.util.rados:rados -p unfound_pool put new_7 /etc/fstab 2024-09-10T15:57:54.815 DEBUG:teuthology.orchestra.run.smithi115:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool put new_7 /etc/fstab 2024-09-10T15:57:54.864 INFO:tasks.util.rados:rados -p unfound_pool put existed_7 /etc/fstab 2024-09-10T15:57:54.864 DEBUG:teuthology.orchestra.run.smithi115:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool put existed_7 /etc/fstab 2024-09-10T15:57:54.915 INFO:tasks.util.rados:rados -p unfound_pool put existing_7 /etc/fstab 2024-09-10T15:57:54.916 DEBUG:teuthology.orchestra.run.smithi115:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool put existing_7 /etc/fstab 2024-09-10T15:57:54.966 INFO:tasks.util.rados:rados -p unfound_pool put new_8 /etc/fstab 2024-09-10T15:57:54.966 DEBUG:teuthology.orchestra.run.smithi115:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool put new_8 /etc/fstab 2024-09-10T15:57:55.016 INFO:tasks.util.rados:rados -p unfound_pool put existed_8 /etc/fstab 2024-09-10T15:57:55.017 DEBUG:teuthology.orchestra.run.smithi115:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool put existed_8 /etc/fstab 2024-09-10T15:57:55.066 INFO:tasks.util.rados:rados -p unfound_pool put existing_8 /etc/fstab 2024-09-10T15:57:55.066 DEBUG:teuthology.orchestra.run.smithi115:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool put existing_8 /etc/fstab 2024-09-10T15:57:55.116 INFO:tasks.util.rados:rados -p unfound_pool put new_9 /etc/fstab 2024-09-10T15:57:55.116 DEBUG:teuthology.orchestra.run.smithi115:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool put new_9 /etc/fstab 2024-09-10T15:57:55.166 INFO:tasks.util.rados:rados -p unfound_pool put existed_9 /etc/fstab 2024-09-10T15:57:55.166 DEBUG:teuthology.orchestra.run.smithi115:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool put existed_9 /etc/fstab 2024-09-10T15:57:55.216 INFO:tasks.util.rados:rados -p unfound_pool put existing_9 /etc/fstab 2024-09-10T15:57:55.216 DEBUG:teuthology.orchestra.run.smithi115:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool put existing_9 /etc/fstab 2024-09-10T15:57:55.268 INFO:tasks.lost_unfound:osd.0 command_args is foo 2024-09-10T15:57:55.268 INFO:tasks.lost_unfound:() 2024-09-10T15:57:55.268 INFO:tasks.ceph.osd.0:Restarting daemon 2024-09-10T15:57:55.268 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 --osd-recovery-delay-start 1000 2024-09-10T15:57:55.270 INFO:tasks.ceph.osd.0:Started 2024-09-10T15:57:55.270 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-10T15:57:55.368 INFO:teuthology.orchestra.run.smithi115.stderr:no valid command found; 10 closest matches: 2024-09-10T15:57:55.368 INFO:teuthology.orchestra.run.smithi115.stderr:0 2024-09-10T15:57:55.368 INFO:teuthology.orchestra.run.smithi115.stderr:1 2024-09-10T15:57:55.368 INFO:teuthology.orchestra.run.smithi115.stderr:2 2024-09-10T15:57:55.368 INFO:teuthology.orchestra.run.smithi115.stderr:abort 2024-09-10T15:57:55.368 INFO:teuthology.orchestra.run.smithi115.stderr:assert 2024-09-10T15:57:55.368 INFO:teuthology.orchestra.run.smithi115.stderr:config diff 2024-09-10T15:57:55.369 INFO:teuthology.orchestra.run.smithi115.stderr:config diff get 2024-09-10T15:57:55.369 INFO:teuthology.orchestra.run.smithi115.stderr:config get 2024-09-10T15:57:55.369 INFO:teuthology.orchestra.run.smithi115.stderr:config help [] 2024-09-10T15:57:55.369 INFO:teuthology.orchestra.run.smithi115.stderr:config set ... 2024-09-10T15:57:55.369 INFO:teuthology.orchestra.run.smithi115.stderr:admin_socket: invalid command 2024-09-10T15:57:55.370 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-10T15:57:55.371 INFO:tasks.lost_unfound.ceph_manager:waiting on admin_socket for osd-0, ['dump_ops_in_flight'] 2024-09-10T15:57:56.947 INFO:tasks.ceph.osd.0.smithi115.stderr:2024-09-10T15:57:56.946+0000 7fe229502640 -1 Falling back to public interface 2024-09-10T15:58:00.371 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-10T15:58:00.482 INFO:teuthology.orchestra.run.smithi115.stderr:no valid command found; 10 closest matches: 2024-09-10T15:58:00.482 INFO:teuthology.orchestra.run.smithi115.stderr:0 2024-09-10T15:58:00.482 INFO:teuthology.orchestra.run.smithi115.stderr:1 2024-09-10T15:58:00.482 INFO:teuthology.orchestra.run.smithi115.stderr:2 2024-09-10T15:58:00.482 INFO:teuthology.orchestra.run.smithi115.stderr:abort 2024-09-10T15:58:00.482 INFO:teuthology.orchestra.run.smithi115.stderr:assert 2024-09-10T15:58:00.482 INFO:teuthology.orchestra.run.smithi115.stderr:bluefs debug_inject_read_zeros 2024-09-10T15:58:00.482 INFO:teuthology.orchestra.run.smithi115.stderr:bluefs files list 2024-09-10T15:58:00.482 INFO:teuthology.orchestra.run.smithi115.stderr:bluefs stats 2024-09-10T15:58:00.482 INFO:teuthology.orchestra.run.smithi115.stderr:bluestore bluefs device info [] 2024-09-10T15:58:00.482 INFO:teuthology.orchestra.run.smithi115.stderr:config diff 2024-09-10T15:58:00.482 INFO:teuthology.orchestra.run.smithi115.stderr:admin_socket: invalid command 2024-09-10T15:58:00.485 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-10T15:58:00.485 INFO:tasks.lost_unfound.ceph_manager:waiting on admin_socket for osd-0, ['dump_ops_in_flight'] 2024-09-10T15:58:00.756 INFO:tasks.ceph.osd.0.smithi115.stderr:2024-09-10T15:58:00.755+0000 7fe229502640 -1 osd.0 21 log_to_monitors true 2024-09-10T15:58:01.007 INFO:tasks.ceph.osd.0.smithi115.stderr:2024-09-10T15:58:01.005+0000 7fe2212b8640 -1 osd.0 21 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T15:58:05.487 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-10T15:58:05.602 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-10T15:58:05.602 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-10T15:58:05.602 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-09-10T15:58:05.602 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-10T15:58:05.611 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 0 2024-09-10T15:58:05.903 INFO:tasks.ceph.mon.a.smithi115.stderr:2024-09-10T15:58:05.901+0000 7fa206bdb640 -1 mon.a@0(leader).osd e26 definitely_dead 0 2024-09-10T15:58:05.903 INFO:teuthology.orchestra.run.smithi115.stderr:osd.0 is already in. 2024-09-10T15:58:05.915 INFO:tasks.lost_unfound.ceph_manager:waiting for osd.0 to be up 2024-09-10T15:58:05.915 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T15:58:06.203 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T15:58:06.203 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":26,"fsid":"7effee29-df05-4351-9741-ee203564ed81","created":"2024-09-10T15:57:14.967498+0000","modified":"2024-09-10T15:58:03.009245+0000","last_up_change":"2024-09-10T15:58:02.001654+0000","last_in_change":"2024-09-10T15:57:44.754961+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2024-09-10T15:57:30.381324+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":14,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8700000047683716,"score_stable":1.8700000047683716,"optimal_score":0.67000001668930054,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"unfound_pool","create_time":"2024-09-10T15:57:39.758755+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":16,"pg_placement_num":16,"pg_placement_num_target":16,"pg_num_target":16,"pg_num_pending":16,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rados":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8700000047683716,"score_stable":1.8700000047683716,"optimal_score":0.67000001668930054,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"859508e0-e15d-4011-a39f-65e7aa04a129","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":21,"up_from":25,"up_thru":25,"down_at":22,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":1567796493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6801","nonce":1567796493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6803","nonce":1567796493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":1567796493}]},"public_addr":"172.21.15.115:6800/1567796493","cluster_addr":"172.21.15.115:6801/1567796493","heartbeat_back_addr":"172.21.15.115:6803/1567796493","heartbeat_front_addr":"172.21.15.115:6802/1567796493","state":["exists","up"]},{"osd":1,"uuid":"66daed2f-a693-4a13-8d8c-024794a032e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":2071140438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6809","nonce":2071140438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6811","nonce":2071140438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":2071140438}]},"public_addr":"172.21.15.115:6808/2071140438","cluster_addr":"172.21.15.115:6809/2071140438","heartbeat_back_addr":"172.21.15.115:6811/2071140438","heartbeat_front_addr":"172.21.15.115:6810/2071140438","state":["exists","up"]},{"osd":2,"uuid":"6a2ab153-d99e-4a5d-b79c-32b7f8ea9ffb","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":16,"down_at":19,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":267302962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6801","nonce":267302962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6803","nonce":267302962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":267302962}]},"public_addr":"172.21.15.115:6800/267302962","cluster_addr":"172.21.15.115:6801/267302962","heartbeat_back_addr":"172.21.15.115:6803/267302962","heartbeat_front_addr":"172.21.15.115:6802/267302962","state":["exists"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T15:57:23.007077+0000","dead_epoch":22},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T15:57:23.605294+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":65536,"last_purged_snaps_scrub":"2024-09-10T15:57:22.852045+0000","dead_epoch":19}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T15:58:06.216 INFO:tasks.lost_unfound.ceph_manager:osd.0 is up 2024-09-10T15:58:06.217 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-10T15:58:06.217 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-10T15:58:06.349 INFO:teuthology.orchestra.run.smithi115.stdout:38654705675 2024-09-10T15:58:06.350 INFO:teuthology.orchestra.run.smithi115.stdout:107374182403 2024-09-10T15:58:06.351 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-10T15:58:06.351 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-10T15:58:06.663 INFO:teuthology.orchestra.run.smithi115.stdout:38654705674 2024-09-10T15:58:06.665 INFO:teuthology.orchestra.run.smithi115.stdout:1 2024-09-10T15:58:06.676 INFO:tasks.lost_unfound.ceph_manager:need seq 38654705675 got 38654705674 for osd.1 2024-09-10T15:58:06.678 INFO:tasks.lost_unfound.ceph_manager:need seq 107374182403 got 1 for osd.0 2024-09-10T15:58:07.677 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-10T15:58:07.678 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-10T15:58:07.985 INFO:teuthology.orchestra.run.smithi115.stdout:1 2024-09-10T15:58:07.989 INFO:teuthology.orchestra.run.smithi115.stdout:38654705674 2024-09-10T15:58:07.999 INFO:tasks.lost_unfound.ceph_manager:need seq 107374182403 got 1 for osd.0 2024-09-10T15:58:08.001 INFO:tasks.lost_unfound.ceph_manager:need seq 38654705675 got 38654705674 for osd.1 2024-09-10T15:58:09.000 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-10T15:58:09.002 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-10T15:58:09.306 INFO:teuthology.orchestra.run.smithi115.stdout:107374182403 2024-09-10T15:58:09.307 INFO:teuthology.orchestra.run.smithi115.stdout:38654705676 2024-09-10T15:58:09.320 INFO:tasks.lost_unfound.ceph_manager:need seq 107374182403 got 107374182403 for osd.0 2024-09-10T15:58:09.320 DEBUG:teuthology.parallel:result is None 2024-09-10T15:58:09.321 INFO:tasks.lost_unfound.ceph_manager:need seq 38654705675 got 38654705676 for osd.1 2024-09-10T15:58:09.321 DEBUG:teuthology.parallel:result is None 2024-09-10T15:58:09.321 INFO:tasks.lost_unfound.ceph_manager:waiting till active 2024-09-10T15:58:09.321 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:58:09.644 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T15:58:09.644 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T15:58:09.655 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":46,"stamp":"2024-09-10T15:58:09.531489+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":13543,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":7,"num_objects_missing":7,"num_objects_degraded":27,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57,"num_write_kb":48,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":58,"ondisk_log_size":58,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":48,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":55108,"kb_used_data":592,"kb_used_omap":5,"kb_used_meta":54458,"kb_avail":188688572,"statfs":{"total":193273528320,"available":193217097728,"internally_reserved":0,"allocated":606208,"data_stored":147324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6033,"internal_metadata":55765103},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":7,"num_objects_missing":7,"num_objects_degraded":-3,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001738"},"pg_stats":[{"pgid":"2.f","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T15:58:03.027753+0000","last_change":"2024-09-10T15:58:03.027639+0000","last_active":"2024-09-10T15:58:03.027753+0000","last_peered":"2024-09-10T15:58:03.027753+0000","last_clean":"2024-09-10T15:58:03.027753+0000","last_became_active":"2024-09-10T15:58:03.027136+0000","last_became_peered":"2024-09-10T15:58:03.027136+0000","last_unstale":"2024-09-10T15:58:03.027753+0000","last_undegraded":"2024-09-10T15:58:03.027753+0000","last_fullsized":"2024-09-10T15:58:03.027753+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:33:39.786286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"23'4","reported_seq":31,"reported_epoch":26,"state":"active+recovering+degraded","last_fresh":"2024-09-10T15:58:03.027798+0000","last_change":"2024-09-10T15:58:03.030407+0000","last_active":"2024-09-10T15:58:03.027798+0000","last_peered":"2024-09-10T15:58:03.027798+0000","last_clean":"2024-09-10T15:57:51.316978+0000","last_became_active":"2024-09-10T15:58:03.027280+0000","last_became_peered":"2024-09-10T15:58:03.027280+0000","last_unstale":"2024-09-10T15:58:03.027798+0000","last_undegraded":"2024-09-10T15:58:03.016868+0000","last_fullsized":"2024-09-10T15:58:03.027798+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["1"],"object_location_counts":[{"shards":"1","objects":2}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"23'3","reported_seq":44,"reported_epoch":26,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-10T15:58:03.025220+0000","last_change":"2024-09-10T15:58:03.025375+0000","last_active":"2024-09-10T15:58:03.025220+0000","last_peered":"2024-09-10T15:58:03.025220+0000","last_clean":"2024-09-10T15:57:51.863524+0000","last_became_active":"2024-09-10T15:58:03.025141+0000","last_became_peered":"2024-09-10T15:58:03.025141+0000","last_unstale":"2024-09-10T15:58:03.025220+0000","last_undegraded":"2024-09-10T15:58:03.018336+0000","last_fullsized":"2024-09-10T15:58:03.025220+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1"],"object_location_counts":[{"shards":"1","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":31,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T15:58:03.027825+0000","last_change":"2024-09-10T15:58:03.027714+0000","last_active":"2024-09-10T15:58:03.027825+0000","last_peered":"2024-09-10T15:58:03.027825+0000","last_clean":"2024-09-10T15:58:03.027825+0000","last_became_active":"2024-09-10T15:58:03.027213+0000","last_became_peered":"2024-09-10T15:58:03.027213+0000","last_unstale":"2024-09-10T15:58:03.027825+0000","last_undegraded":"2024-09-10T15:58:03.027825+0000","last_fullsized":"2024-09-10T15:58:03.027825+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:19:43.095225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.b","version":"23'5","reported_seq":36,"reported_epoch":26,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-10T15:58:03.027992+0000","last_change":"2024-09-10T15:58:03.028197+0000","last_active":"2024-09-10T15:58:03.027992+0000","last_peered":"2024-09-10T15:58:03.027992+0000","last_clean":"2024-09-10T15:57:51.217636+0000","last_became_active":"2024-09-10T15:58:03.027868+0000","last_became_peered":"2024-09-10T15:58:03.027868+0000","last_unstale":"2024-09-10T15:58:03.027992+0000","last_undegraded":"2024-09-10T15:58:03.016073+0000","last_fullsized":"2024-09-10T15:58:03.027992+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["1"],"object_location_counts":[{"shards":"1","objects":2}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"23'3","reported_seq":39,"reported_epoch":26,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-10T15:58:03.023953+0000","last_change":"2024-09-10T15:58:03.024125+0000","last_active":"2024-09-10T15:58:03.023953+0000","last_peered":"2024-09-10T15:58:03.023953+0000","last_clean":"2024-09-10T15:57:52.065335+0000","last_became_active":"2024-09-10T15:58:03.023842+0000","last_became_peered":"2024-09-10T15:58:03.023842+0000","last_unstale":"2024-09-10T15:58:03.023953+0000","last_undegraded":"2024-09-10T15:58:03.018461+0000","last_fullsized":"2024-09-10T15:58:03.023953+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1"],"object_location_counts":[{"shards":"1","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"23'4","reported_seq":40,"reported_epoch":26,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-10T15:58:03.025721+0000","last_change":"2024-09-10T15:58:03.025959+0000","last_active":"2024-09-10T15:58:03.025721+0000","last_peered":"2024-09-10T15:58:03.025721+0000","last_clean":"2024-09-10T15:57:51.467206+0000","last_became_active":"2024-09-10T15:58:03.025635+0000","last_became_peered":"2024-09-10T15:58:03.025635+0000","last_unstale":"2024-09-10T15:58:03.025721+0000","last_undegraded":"2024-09-10T15:58:03.017964+0000","last_fullsized":"2024-09-10T15:58:03.025721+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1"],"object_location_counts":[{"shards":"1","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"23'1","reported_seq":33,"reported_epoch":26,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-10T15:58:03.027747+0000","last_change":"2024-09-10T15:58:03.027996+0000","last_active":"2024-09-10T15:58:03.027747+0000","last_peered":"2024-09-10T15:58:03.027747+0000","last_clean":"2024-09-10T15:57:42.760798+0000","last_became_active":"2024-09-10T15:58:03.027657+0000","last_became_peered":"2024-09-10T15:58:03.027657+0000","last_unstale":"2024-09-10T15:58:03.027747+0000","last_undegraded":"2024-09-10T15:58:03.015299+0000","last_fullsized":"2024-09-10T15:58:03.027747+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["1"],"object_location_counts":[{"shards":"1","objects":1}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":53,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T15:58:03.026256+0000","last_change":"2024-09-10T15:58:03.026143+0000","last_active":"2024-09-10T15:58:03.026256+0000","last_peered":"2024-09-10T15:58:03.026256+0000","last_clean":"2024-09-10T15:58:03.026256+0000","last_became_active":"2024-09-10T15:58:03.025848+0000","last_became_peered":"2024-09-10T15:58:03.025848+0000","last_unstale":"2024-09-10T15:58:03.026256+0000","last_undegraded":"2024-09-10T15:58:03.026256+0000","last_fullsized":"2024-09-10T15:58:03.026256+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:42:39.226551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"23'5","reported_seq":47,"reported_epoch":26,"state":"active+recovering+degraded","last_fresh":"2024-09-10T15:58:03.023548+0000","last_change":"2024-09-10T15:58:03.029934+0000","last_active":"2024-09-10T15:58:03.023548+0000","last_peered":"2024-09-10T15:58:03.023548+0000","last_clean":"2024-09-10T15:57:51.913440+0000","last_became_active":"2024-09-10T15:58:03.023287+0000","last_became_peered":"2024-09-10T15:58:03.023287+0000","last_unstale":"2024-09-10T15:58:03.023548+0000","last_undegraded":"2024-09-10T15:58:03.017490+0000","last_fullsized":"2024-09-10T15:58:03.023548+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1"],"object_location_counts":[{"shards":"1","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":44,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T15:58:03.027622+0000","last_change":"2024-09-10T15:58:03.027525+0000","last_active":"2024-09-10T15:58:03.027622+0000","last_peered":"2024-09-10T15:58:03.027622+0000","last_clean":"2024-09-10T15:58:03.027622+0000","last_became_active":"2024-09-10T15:58:03.027092+0000","last_became_peered":"2024-09-10T15:58:03.027092+0000","last_unstale":"2024-09-10T15:58:03.027622+0000","last_undegraded":"2024-09-10T15:58:03.027622+0000","last_fullsized":"2024-09-10T15:58:03.027622+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:34:23.953238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"23'6","reported_seq":32,"reported_epoch":26,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-10T15:58:03.027784+0000","last_change":"2024-09-10T15:58:03.027986+0000","last_active":"2024-09-10T15:58:03.027784+0000","last_peered":"2024-09-10T15:58:03.027784+0000","last_clean":"2024-09-10T15:57:52.118434+0000","last_became_active":"2024-09-10T15:58:03.027331+0000","last_became_peered":"2024-09-10T15:58:03.027331+0000","last_unstale":"2024-09-10T15:58:03.027784+0000","last_undegraded":"2024-09-10T15:58:03.015784+0000","last_fullsized":"2024-09-10T15:58:03.027784+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["1"],"object_location_counts":[{"shards":"1","objects":2}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":38,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T15:58:03.027775+0000","last_change":"2024-09-10T15:58:03.027636+0000","last_active":"2024-09-10T15:58:03.027775+0000","last_peered":"2024-09-10T15:58:03.027775+0000","last_clean":"2024-09-10T15:58:03.027775+0000","last_became_active":"2024-09-10T15:58:03.026747+0000","last_became_peered":"2024-09-10T15:58:03.026747+0000","last_unstale":"2024-09-10T15:58:03.027775+0000","last_undegraded":"2024-09-10T15:58:03.027775+0000","last_fullsized":"2024-09-10T15:58:03.027775+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:52:32.797217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"23'3","reported_seq":39,"reported_epoch":26,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-10T15:58:03.025351+0000","last_change":"2024-09-10T15:58:03.025507+0000","last_active":"2024-09-10T15:58:03.025351+0000","last_peered":"2024-09-10T15:58:03.025351+0000","last_clean":"2024-09-10T15:57:51.167715+0000","last_became_active":"2024-09-10T15:58:03.025264+0000","last_became_peered":"2024-09-10T15:58:03.025264+0000","last_unstale":"2024-09-10T15:58:03.025351+0000","last_undegraded":"2024-09-10T15:58:03.017405+0000","last_fullsized":"2024-09-10T15:58:03.025351+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1"],"object_location_counts":[{"shards":"1","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":63,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T15:58:03.024549+0000","last_change":"2024-09-10T15:58:03.024456+0000","last_active":"2024-09-10T15:58:03.024549+0000","last_peered":"2024-09-10T15:58:03.024549+0000","last_clean":"2024-09-10T15:58:03.024549+0000","last_became_active":"2024-09-10T15:58:03.023460+0000","last_became_peered":"2024-09-10T15:58:03.023460+0000","last_unstale":"2024-09-10T15:58:03.024549+0000","last_undegraded":"2024-09-10T15:58:03.024549+0000","last_fullsized":"2024-09-10T15:58:03.024549+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:10:33.965706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"23'4","reported_seq":40,"reported_epoch":26,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-10T15:58:03.026056+0000","last_change":"2024-09-10T15:58:03.026253+0000","last_active":"2024-09-10T15:58:03.026056+0000","last_peered":"2024-09-10T15:58:03.026056+0000","last_clean":"2024-09-10T15:57:51.566307+0000","last_became_active":"2024-09-10T15:58:03.025964+0000","last_became_peered":"2024-09-10T15:58:03.025964+0000","last_unstale":"2024-09-10T15:58:03.026056+0000","last_undegraded":"2024-09-10T15:58:03.017365+0000","last_fullsized":"2024-09-10T15:58:03.026056+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1"],"object_location_counts":[{"shards":"1","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"23'1","reported_seq":36,"reported_epoch":26,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-10T15:58:03.024687+0000","last_change":"2024-09-10T15:58:03.025036+0000","last_active":"2024-09-10T15:58:03.024687+0000","last_peered":"2024-09-10T15:58:03.024687+0000","last_clean":"2024-09-10T15:57:45.763054+0000","last_became_active":"2024-09-10T15:58:03.024601+0000","last_became_peered":"2024-09-10T15:58:03.024601+0000","last_unstale":"2024-09-10T15:58:03.024687+0000","last_undegraded":"2024-09-10T15:58:03.016630+0000","last_fullsized":"2024-09-10T15:58:03.024687+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1"],"object_location_counts":[{"shards":"1","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":53,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T15:58:03.024182+0000","last_change":"2024-09-10T15:58:03.024085+0000","last_active":"2024-09-10T15:58:03.024182+0000","last_peered":"2024-09-10T15:58:03.024182+0000","last_clean":"2024-09-10T15:58:03.024182+0000","last_became_active":"2024-09-10T15:58:03.023798+0000","last_became_peered":"2024-09-10T15:58:03.023798+0000","last_unstale":"2024-09-10T15:58:03.024182+0000","last_undegraded":"2024-09-10T15:58:03.024182+0000","last_fullsized":"2024-09-10T15:58:03.024182+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:25:01.305018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"23'5","reported_seq":41,"reported_epoch":26,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-10T15:58:03.025978+0000","last_change":"2024-09-10T15:58:03.026157+0000","last_active":"2024-09-10T15:58:03.025978+0000","last_peered":"2024-09-10T15:58:03.025978+0000","last_clean":"2024-09-10T15:57:52.013178+0000","last_became_active":"2024-09-10T15:58:03.025859+0000","last_became_peered":"2024-09-10T15:58:03.025859+0000","last_unstale":"2024-09-10T15:58:03.025978+0000","last_undegraded":"2024-09-10T15:58:03.018531+0000","last_fullsized":"2024-09-10T15:58:03.025978+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1"],"object_location_counts":[{"shards":"1","objects":3}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":44,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T15:58:03.028262+0000","last_change":"2024-09-10T15:58:03.028176+0000","last_active":"2024-09-10T15:58:03.028262+0000","last_peered":"2024-09-10T15:58:03.028262+0000","last_clean":"2024-09-10T15:58:03.028262+0000","last_became_active":"2024-09-10T15:58:03.027679+0000","last_became_peered":"2024-09-10T15:58:03.027679+0000","last_unstale":"2024-09-10T15:58:03.028262+0000","last_undegraded":"2024-09-10T15:58:03.028262+0000","last_fullsized":"2024-09-10T15:58:03.028262+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:21:42.319913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"23'2","reported_seq":37,"reported_epoch":26,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-10T15:58:03.025854+0000","last_change":"2024-09-10T15:58:03.026023+0000","last_active":"2024-09-10T15:58:03.025854+0000","last_peered":"2024-09-10T15:58:03.025854+0000","last_clean":"2024-09-10T15:57:45.763338+0000","last_became_active":"2024-09-10T15:58:03.025751+0000","last_became_peered":"2024-09-10T15:58:03.025751+0000","last_unstale":"2024-09-10T15:58:03.025854+0000","last_undegraded":"2024-09-10T15:58:03.018509+0000","last_fullsized":"2024-09-10T15:58:03.025854+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1"],"object_location_counts":[{"shards":"1","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":53,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T15:58:03.026248+0000","last_change":"2024-09-10T15:58:03.026140+0000","last_active":"2024-09-10T15:58:03.026248+0000","last_peered":"2024-09-10T15:58:03.026248+0000","last_clean":"2024-09-10T15:58:03.026248+0000","last_became_active":"2024-09-10T15:58:03.025682+0000","last_became_peered":"2024-09-10T15:58:03.025682+0000","last_unstale":"2024-09-10T15:58:03.026248+0000","last_undegraded":"2024-09-10T15:58:03.026248+0000","last_fullsized":"2024-09-10T15:58:03.026248+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:10:02.706288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":58,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T15:58:03.026292+0000","last_change":"2024-09-10T15:58:03.026204+0000","last_active":"2024-09-10T15:58:03.026292+0000","last_peered":"2024-09-10T15:58:03.026292+0000","last_clean":"2024-09-10T15:58:03.026292+0000","last_became_active":"2024-09-10T15:58:03.025771+0000","last_became_peered":"2024-09-10T15:58:03.025771+0000","last_unstale":"2024-09-10T15:58:03.026292+0000","last_undegraded":"2024-09-10T15:58:03.026292+0000","last_fullsized":"2024-09-10T15:58:03.026292+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:07:13.108700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"23'9","reported_seq":45,"reported_epoch":26,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-10T15:58:03.024440+0000","last_change":"2024-09-10T15:58:03.024607+0000","last_active":"2024-09-10T15:58:03.024440+0000","last_peered":"2024-09-10T15:58:03.024440+0000","last_clean":"2024-09-10T15:57:51.765160+0000","last_became_active":"2024-09-10T15:58:03.024338+0000","last_became_peered":"2024-09-10T15:58:03.024338+0000","last_unstale":"2024-09-10T15:58:03.024440+0000","last_undegraded":"2024-09-10T15:58:03.018244+0000","last_fullsized":"2024-09-10T15:58:03.024440+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1932,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1"],"object_location_counts":[{"shards":"0,1","objects":1},{"shards":"1","objects":3}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":13524,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":7,"num_objects_missing":7,"num_objects_degraded":27,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":55,"num_write_kb":46,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":155648,"data_stored":18354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":55,"ondisk_log_size":55,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705676,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27364,"kb_used_data":332,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344476,"statfs":{"total":96636764160,"available":96608743424,"internally_reserved":0,"allocated":339968,"data_stored":78009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182403,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27744,"kb_used_data":260,"kb_used_omap":4,"kb_used_meta":27451,"kb_avail":94344096,"statfs":{"total":96636764160,"available":96608354304,"internally_reserved":0,"allocated":266240,"data_stored":69315,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":28110506},"hb_peers":[1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":4830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":13524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:58:09.656 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T15:58:10.005 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T15:58:10.005 INFO:teuthology.orchestra.run.smithi115.stdout:{"fsid":"7effee29-df05-4351-9741-ee203564ed81","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 27/60 objects degraded (45.000%), 14 pgs degraded","count":14},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":54,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":26,"num_osds":3,"num_up_osds":2,"osd_up_since":1725983882,"num_in_osds":2,"osd_in_since":1725983864,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+recovery_wait+degraded","count":12},{"state_name":"active+clean","count":10},{"state_name":"active+recovering+degraded","count":2}],"num_pgs":24,"num_pools":2,"num_objects":30,"data_bytes":13543,"bytes_used":56430592,"bytes_avail":193217097728,"bytes_total":193273528320,"degraded_objects":27,"degraded_total":60,"degraded_ratio":0.45000000000000001},"fsmap":{"epoch":1,"btime":"2024-09-10T15:57:14:967120+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":1,"modified":"2024-09-10T15:57:14.968460+0000","services":{}},"progress_events":{}} 2024-09-10T15:58:10.017 INFO:tasks.lost_unfound.ceph_manager:{'fsid': '7effee29-df05-4351-9741-ee203564ed81', 'health': {'status': 'HEALTH_WARN', 'checks': {'PG_DEGRADED': {'severity': 'HEALTH_WARN', 'summary': {'message': 'Degraded data redundancy: 27/60 objects degraded (45.000%), 14 pgs degraded', 'count': 14}, 'muted': False}}, 'mutes': []}, 'election_epoch': 10, 'quorum': [0, 1, 2], 'quorum_names': ['a', 'b', 'c'], 'quorum_age': 54, 'monmap': {'epoch': 1, 'min_mon_release_name': 'squid', 'num_mons': 3}, 'osdmap': {'epoch': 26, 'num_osds': 3, 'num_up_osds': 2, 'osd_up_since': 1725983882, 'num_in_osds': 2, 'osd_in_since': 1725983864, 'num_remapped_pgs': 0}, 'pgmap': {'pgs_by_state': [{'state_name': 'active+recovery_wait+degraded', 'count': 12}, {'state_name': 'active+clean', 'count': 10}, {'state_name': 'active+recovering+degraded', 'count': 2}], 'num_pgs': 24, 'num_pools': 2, 'num_objects': 30, 'data_bytes': 13543, 'bytes_used': 56430592, 'bytes_avail': 193217097728, 'bytes_total': 193273528320, 'degraded_objects': 27, 'degraded_total': 60, 'degraded_ratio': 0.45}, 'fsmap': {'epoch': 1, 'btime': '2024-09-10T15:57:14:967120+0000', 'by_rank': [], 'up:standby': 0}, 'mgrmap': {'available': True, 'num_standbys': 0, 'modules': ['iostat', 'nfs', 'restful'], 'services': {}}, 'servicemap': {'epoch': 1, 'modified': '2024-09-10T15:57:14.968460+0000', 'services': {}}, 'progress_events': {}} 2024-09-10T15:58:10.018 INFO:tasks.lost_unfound.ceph_manager:active! 2024-09-10T15:58:10.018 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-09-10T15:58:10.018 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T15:58:10.055 INFO:tasks.ceph.osd.1:Stopped 2024-09-10T15:58:10.055 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd down 1 2024-09-10T15:58:10.388 INFO:tasks.ceph.mon.a.smithi115.stderr:2024-09-10T15:58:10.387+0000 7fa206bdb640 -1 mon.a@0(leader).osd e26 definitely_dead 0 2024-09-10T15:58:11.047 INFO:teuthology.orchestra.run.smithi115.stderr:marked down osd.1. 2024-09-10T15:58:11.065 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd out 1 2024-09-10T15:58:11.357 INFO:tasks.ceph.mon.a.smithi115.stderr:2024-09-10T15:58:11.355+0000 7fa206bdb640 -1 mon.a@0(leader).osd e27 definitely_dead 0 2024-09-10T15:58:12.052 INFO:teuthology.orchestra.run.smithi115.stderr:marked out osd.1. 2024-09-10T15:58:12.069 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd lost 1 --yes-i-really-mean-it 2024-09-10T15:58:13.074 INFO:tasks.ceph.osd.0.smithi115.stderr:2024-09-10T15:58:13.071+0000 7fe20f27a640 -1 log_channel(cluster) log [ERR] : 2.1 has 2 objects unfound and apparently lost 2024-09-10T15:58:13.074 INFO:tasks.ceph.osd.0.smithi115.stderr:2024-09-10T15:58:13.071+0000 7fe20a270640 -1 log_channel(cluster) log [ERR] : 2.3 has 2 objects unfound and apparently lost 2024-09-10T15:58:13.074 INFO:tasks.ceph.osd.0.smithi115.stderr:2024-09-10T15:58:13.071+0000 7fe20fa7b640 -1 log_channel(cluster) log [ERR] : 2.8 has 1 objects unfound and apparently lost 2024-09-10T15:58:13.074 INFO:tasks.ceph.osd.0.smithi115.stderr:2024-09-10T15:58:13.071+0000 7fe20aa71640 -1 log_channel(cluster) log [ERR] : 2.a has 2 objects unfound and apparently lost 2024-09-10T15:58:13.074 INFO:tasks.ceph.osd.0.smithi115.stderr:2024-09-10T15:58:13.071+0000 7fe20f27a640 -1 log_channel(cluster) log [ERR] : 2.9 has 2 objects unfound and apparently lost 2024-09-10T15:58:13.075 INFO:tasks.ceph.osd.0.smithi115.stderr:2024-09-10T15:58:13.071+0000 7fe20da77640 -1 log_channel(cluster) log [ERR] : 2.4 has 3 objects unfound and apparently lost 2024-09-10T15:58:13.075 INFO:tasks.ceph.osd.0.smithi115.stderr:2024-09-10T15:58:13.071+0000 7fe20ba73640 -1 log_channel(cluster) log [ERR] : 2.0 has 2 objects unfound and apparently lost 2024-09-10T15:58:13.075 INFO:tasks.ceph.osd.0.smithi115.stderr:2024-09-10T15:58:13.071+0000 7fe20aa71640 -1 log_channel(cluster) log [ERR] : 2.2 has 2 objects unfound and apparently lost 2024-09-10T15:58:13.075 INFO:tasks.ceph.osd.0.smithi115.stderr:2024-09-10T15:58:13.072+0000 7fe20a270640 -1 log_channel(cluster) log [ERR] : 2.b has 2 objects unfound and apparently lost 2024-09-10T15:58:13.075 INFO:tasks.ceph.osd.0.smithi115.stderr:2024-09-10T15:58:13.072+0000 7fe20d276640 -1 log_channel(cluster) log [ERR] : 2.d has 1 objects unfound and apparently lost 2024-09-10T15:58:13.075 INFO:tasks.ceph.osd.0.smithi115.stderr:2024-09-10T15:58:13.072+0000 7fe20926e640 -1 log_channel(cluster) log [ERR] : 2.5 has 2 objects unfound and apparently lost 2024-09-10T15:58:13.075 INFO:teuthology.orchestra.run.smithi115.stderr:marked osd lost in epoch 27 2024-09-10T15:58:13.076 INFO:tasks.ceph.osd.0.smithi115.stderr:2024-09-10T15:58:13.072+0000 7fe208a6d640 -1 log_channel(cluster) log [ERR] : 2.e has 2 objects unfound and apparently lost 2024-09-10T15:58:13.076 INFO:tasks.ceph.osd.0.smithi115.stderr:2024-09-10T15:58:13.072+0000 7fe20ca75640 -1 log_channel(cluster) log [ERR] : 2.6 has 3 objects unfound and apparently lost 2024-09-10T15:58:13.076 INFO:tasks.ceph.osd.0.smithi115.stderr:2024-09-10T15:58:13.072+0000 7fe20826c640 -1 log_channel(cluster) log [ERR] : 2.7 has 1 objects unfound and apparently lost 2024-09-10T15:58:13.089 INFO:tasks.ceph.osd.2:Restarting daemon 2024-09-10T15:58:13.089 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-09-10T15:58:13.092 INFO:tasks.ceph.osd.2:Started 2024-09-10T15:58:13.092 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-10T15:58:13.193 INFO:teuthology.orchestra.run.smithi115.stderr:no valid command found; 10 closest matches: 2024-09-10T15:58:13.193 INFO:teuthology.orchestra.run.smithi115.stderr:0 2024-09-10T15:58:13.196 INFO:teuthology.orchestra.run.smithi115.stderr:1 2024-09-10T15:58:13.196 INFO:teuthology.orchestra.run.smithi115.stderr:2 2024-09-10T15:58:13.196 INFO:teuthology.orchestra.run.smithi115.stderr:abort 2024-09-10T15:58:13.196 INFO:teuthology.orchestra.run.smithi115.stderr:assert 2024-09-10T15:58:13.196 INFO:teuthology.orchestra.run.smithi115.stderr:config diff 2024-09-10T15:58:13.196 INFO:teuthology.orchestra.run.smithi115.stderr:config diff get 2024-09-10T15:58:13.196 INFO:teuthology.orchestra.run.smithi115.stderr:config get 2024-09-10T15:58:13.197 INFO:teuthology.orchestra.run.smithi115.stderr:config help [] 2024-09-10T15:58:13.197 INFO:teuthology.orchestra.run.smithi115.stderr:config set ... 2024-09-10T15:58:13.197 INFO:teuthology.orchestra.run.smithi115.stderr:admin_socket: invalid command 2024-09-10T15:58:13.197 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-10T15:58:13.198 INFO:tasks.lost_unfound.ceph_manager:waiting on admin_socket for osd-2, ['dump_ops_in_flight'] 2024-09-10T15:58:14.248 INFO:tasks.ceph.osd.2.smithi115.stderr:2024-09-10T15:58:14.246+0000 7f589ca0e640 -1 Falling back to public interface 2024-09-10T15:58:18.198 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-10T15:58:18.292 INFO:tasks.ceph.osd.2.smithi115.stderr:2024-09-10T15:58:18.290+0000 7f589ca0e640 -1 osd.2 18 log_to_monitors true 2024-09-10T15:58:18.316 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-10T15:58:18.317 INFO:teuthology.orchestra.run.smithi115.stderr:no valid command found; 10 closest matches: 2024-09-10T15:58:18.317 INFO:teuthology.orchestra.run.smithi115.stderr:0 2024-09-10T15:58:18.317 INFO:teuthology.orchestra.run.smithi115.stderr:1 2024-09-10T15:58:18.317 INFO:teuthology.orchestra.run.smithi115.stderr:2 2024-09-10T15:58:18.317 INFO:teuthology.orchestra.run.smithi115.stderr:abort 2024-09-10T15:58:18.317 INFO:teuthology.orchestra.run.smithi115.stderr:assert 2024-09-10T15:58:18.317 INFO:teuthology.orchestra.run.smithi115.stderr:bluefs debug_inject_read_zeros 2024-09-10T15:58:18.318 INFO:teuthology.orchestra.run.smithi115.stderr:bluefs files list 2024-09-10T15:58:18.318 INFO:teuthology.orchestra.run.smithi115.stderr:bluefs stats 2024-09-10T15:58:18.318 INFO:teuthology.orchestra.run.smithi115.stderr:bluestore allocator dump block 2024-09-10T15:58:18.318 INFO:teuthology.orchestra.run.smithi115.stderr:bluestore allocator fragmentation block 2024-09-10T15:58:18.318 INFO:teuthology.orchestra.run.smithi115.stderr:admin_socket: invalid command 2024-09-10T15:58:18.318 INFO:tasks.lost_unfound.ceph_manager:waiting on admin_socket for osd-2, ['dump_ops_in_flight'] 2024-09-10T15:58:19.141 INFO:tasks.ceph.osd.0.smithi115.stderr:2024-09-10T15:58:19.138+0000 7fe20fa7b640 -1 log_channel(cluster) log [ERR] : 2.0 has 2 objects unfound and apparently lost 2024-09-10T15:58:19.141 INFO:tasks.ceph.osd.0.smithi115.stderr:2024-09-10T15:58:19.138+0000 7fe20b272640 -1 log_channel(cluster) log [ERR] : 2.1 has 2 objects unfound and apparently lost 2024-09-10T15:58:19.141 INFO:tasks.ceph.osd.0.smithi115.stderr:2024-09-10T15:58:19.138+0000 7fe20ea79640 -1 log_channel(cluster) log [ERR] : 2.a has 2 objects unfound and apparently lost 2024-09-10T15:58:19.141 INFO:tasks.ceph.osd.0.smithi115.stderr:2024-09-10T15:58:19.138+0000 7fe20fa7b640 -1 log_channel(cluster) log [ERR] : 2.8 has 1 objects unfound and apparently lost 2024-09-10T15:58:19.141 INFO:tasks.ceph.osd.0.smithi115.stderr:2024-09-10T15:58:19.138+0000 7fe20b272640 -1 log_channel(cluster) log [ERR] : 2.9 has 2 objects unfound and apparently lost 2024-09-10T15:58:19.141 INFO:tasks.ceph.osd.0.smithi115.stderr:2024-09-10T15:58:19.138+0000 7fe20da77640 -1 log_channel(cluster) log [ERR] : 2.4 has 3 objects unfound and apparently lost 2024-09-10T15:58:19.141 INFO:tasks.ceph.osd.0.smithi115.stderr:2024-09-10T15:58:19.138+0000 7fe20e278640 -1 log_channel(cluster) log [ERR] : 2.3 has 2 objects unfound and apparently lost 2024-09-10T15:58:19.141 INFO:tasks.ceph.osd.0.smithi115.stderr:2024-09-10T15:58:19.138+0000 7fe20ea79640 -1 log_channel(cluster) log [ERR] : 2.2 has 2 objects unfound and apparently lost 2024-09-10T15:58:19.141 INFO:tasks.ceph.osd.0.smithi115.stderr:2024-09-10T15:58:19.138+0000 7fe20a270640 -1 log_channel(cluster) log [ERR] : 2.b has 2 objects unfound and apparently lost 2024-09-10T15:58:19.141 INFO:tasks.ceph.osd.0.smithi115.stderr:2024-09-10T15:58:19.138+0000 7fe20d276640 -1 log_channel(cluster) log [ERR] : 2.d has 1 objects unfound and apparently lost 2024-09-10T15:58:19.141 INFO:tasks.ceph.osd.0.smithi115.stderr:2024-09-10T15:58:19.139+0000 7fe20926e640 -1 log_channel(cluster) log [ERR] : 2.5 has 2 objects unfound and apparently lost 2024-09-10T15:58:19.142 INFO:tasks.ceph.osd.0.smithi115.stderr:2024-09-10T15:58:19.139+0000 7fe20826c640 -1 log_channel(cluster) log [ERR] : 2.7 has 1 objects unfound and apparently lost 2024-09-10T15:58:19.142 INFO:tasks.ceph.osd.0.smithi115.stderr:2024-09-10T15:58:19.139+0000 7fe20ca75640 -1 log_channel(cluster) log [ERR] : 2.e has 2 objects unfound and apparently lost 2024-09-10T15:58:19.142 INFO:tasks.ceph.osd.0.smithi115.stderr:2024-09-10T15:58:19.139+0000 7fe208a6d640 -1 log_channel(cluster) log [ERR] : 2.6 has 3 objects unfound and apparently lost 2024-09-10T15:58:19.144 INFO:tasks.ceph.osd.2.smithi115.stderr:2024-09-10T15:58:19.143+0000 7f5894200640 -1 osd.2 18 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T15:58:20.142 INFO:tasks.ceph.osd.0.smithi115.stderr:2024-09-10T15:58:20.140+0000 7fe20ba73640 -1 log_channel(cluster) log [ERR] : 2.8 has 1 objects unfound and apparently lost 2024-09-10T15:58:20.142 INFO:tasks.ceph.osd.0.smithi115.stderr:2024-09-10T15:58:20.140+0000 7fe20b272640 -1 log_channel(cluster) log [ERR] : 2.1 has 2 objects unfound and apparently lost 2024-09-10T15:58:20.143 INFO:tasks.ceph.osd.0.smithi115.stderr:2024-09-10T15:58:20.140+0000 7fe20f27a640 -1 log_channel(cluster) log [ERR] : 2.9 has 2 objects unfound and apparently lost 2024-09-10T15:58:20.143 INFO:tasks.ceph.osd.0.smithi115.stderr:2024-09-10T15:58:20.140+0000 7fe20da77640 -1 log_channel(cluster) log [ERR] : 2.4 has 3 objects unfound and apparently lost 2024-09-10T15:58:20.143 INFO:tasks.ceph.osd.0.smithi115.stderr:2024-09-10T15:58:20.140+0000 7fe20a270640 -1 log_channel(cluster) log [ERR] : 2.3 has 2 objects unfound and apparently lost 2024-09-10T15:58:20.144 INFO:tasks.ceph.osd.0.smithi115.stderr:2024-09-10T15:58:20.141+0000 7fe20d276640 -1 log_channel(cluster) log [ERR] : 2.d has 1 objects unfound and apparently lost 2024-09-10T15:58:20.144 INFO:tasks.ceph.osd.0.smithi115.stderr:2024-09-10T15:58:20.141+0000 7fe20a270640 -1 log_channel(cluster) log [ERR] : 2.b has 2 objects unfound and apparently lost 2024-09-10T15:58:20.144 INFO:tasks.ceph.osd.0.smithi115.stderr:2024-09-10T15:58:20.141+0000 7fe20d276640 -1 log_channel(cluster) log [ERR] : 2.5 has 2 objects unfound and apparently lost 2024-09-10T15:58:20.144 INFO:tasks.ceph.osd.0.smithi115.stderr:2024-09-10T15:58:20.141+0000 7fe20aa71640 -1 log_channel(cluster) log [ERR] : 2.a has 2 objects unfound and apparently lost 2024-09-10T15:58:20.144 INFO:tasks.ceph.osd.0.smithi115.stderr:2024-09-10T15:58:20.141+0000 7fe20ea79640 -1 log_channel(cluster) log [ERR] : 2.2 has 2 objects unfound and apparently lost 2024-09-10T15:58:20.145 INFO:tasks.ceph.osd.0.smithi115.stderr:2024-09-10T15:58:20.141+0000 7fe20826c640 -1 log_channel(cluster) log [ERR] : 2.7 has 1 objects unfound and apparently lost 2024-09-10T15:58:20.145 INFO:tasks.ceph.osd.0.smithi115.stderr:2024-09-10T15:58:20.142+0000 7fe20fa7b640 -1 log_channel(cluster) log [ERR] : 2.0 has 2 objects unfound and apparently lost 2024-09-10T15:58:20.145 INFO:tasks.ceph.osd.0.smithi115.stderr:2024-09-10T15:58:20.142+0000 7fe208a6d640 -1 log_channel(cluster) log [ERR] : 2.6 has 3 objects unfound and apparently lost 2024-09-10T15:58:20.145 INFO:tasks.ceph.osd.0.smithi115.stderr:2024-09-10T15:58:20.142+0000 7fe20ca75640 -1 log_channel(cluster) log [ERR] : 2.e has 2 objects unfound and apparently lost 2024-09-10T15:58:21.153 INFO:tasks.ceph.osd.0.smithi115.stderr:2024-09-10T15:58:21.150+0000 7fe20aa71640 -1 log_channel(cluster) log [ERR] : 2.a has 2 objects unfound and apparently lost 2024-09-10T15:58:21.153 INFO:tasks.ceph.osd.0.smithi115.stderr:2024-09-10T15:58:21.150+0000 7fe20aa71640 -1 log_channel(cluster) log [ERR] : 2.2 has 2 objects unfound and apparently lost 2024-09-10T15:58:21.154 INFO:tasks.ceph.osd.0.smithi115.stderr:2024-09-10T15:58:21.151+0000 7fe20c274640 -1 log_channel(cluster) log [ERR] : 2.7 has 1 objects unfound and apparently lost 2024-09-10T15:58:21.155 INFO:tasks.ceph.osd.0.smithi115.stderr:2024-09-10T15:58:21.152+0000 7fe20da77640 -1 log_channel(cluster) log [ERR] : 2.4 has 3 objects unfound and apparently lost 2024-09-10T15:58:21.156 INFO:tasks.ceph.osd.0.smithi115.stderr:2024-09-10T15:58:21.152+0000 7fe20d276640 -1 log_channel(cluster) log [ERR] : 2.5 has 2 objects unfound and apparently lost 2024-09-10T15:58:21.157 INFO:tasks.ceph.osd.0.smithi115.stderr:2024-09-10T15:58:21.154+0000 7fe20ca75640 -1 log_channel(cluster) log [ERR] : 2.6 has 3 objects unfound and apparently lost 2024-09-10T15:58:21.157 INFO:tasks.ceph.osd.0.smithi115.stderr:2024-09-10T15:58:21.154+0000 7fe20b272640 -1 log_channel(cluster) log [ERR] : 2.9 has 2 objects unfound and apparently lost 2024-09-10T15:58:22.153 INFO:tasks.ceph.osd.0.smithi115.stderr:2024-09-10T15:58:22.151+0000 7fe20aa71640 -1 log_channel(cluster) log [ERR] : 2.a has 2 objects unfound and apparently lost 2024-09-10T15:58:22.153 INFO:tasks.ceph.osd.0.smithi115.stderr:2024-09-10T15:58:22.151+0000 7fe20826c640 -1 log_channel(cluster) log [ERR] : 2.7 has 1 objects unfound and apparently lost 2024-09-10T15:58:22.154 INFO:tasks.ceph.osd.0.smithi115.stderr:2024-09-10T15:58:22.152+0000 7fe20ea79640 -1 log_channel(cluster) log [ERR] : 2.2 has 2 objects unfound and apparently lost 2024-09-10T15:58:22.154 INFO:tasks.ceph.osd.0.smithi115.stderr:2024-09-10T15:58:22.152+0000 7fe20f27a640 -1 log_channel(cluster) log [ERR] : 2.9 has 2 objects unfound and apparently lost 2024-09-10T15:58:22.155 INFO:tasks.ceph.osd.0.smithi115.stderr:2024-09-10T15:58:22.152+0000 7fe20ca75640 -1 log_channel(cluster) log [ERR] : 2.6 has 3 objects unfound and apparently lost 2024-09-10T15:58:22.155 INFO:tasks.ceph.osd.0.smithi115.stderr:2024-09-10T15:58:22.152+0000 7fe20da77640 -1 log_channel(cluster) log [ERR] : 2.4 has 3 objects unfound and apparently lost 2024-09-10T15:58:22.155 INFO:tasks.ceph.osd.0.smithi115.stderr:2024-09-10T15:58:22.153+0000 7fe20d276640 -1 log_channel(cluster) log [ERR] : 2.5 has 2 objects unfound and apparently lost 2024-09-10T15:58:23.320 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-10T15:58:23.435 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-10T15:58:23.435 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-10T15:58:23.435 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-09-10T15:58:23.435 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-10T15:58:23.444 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 2 2024-09-10T15:58:23.737 INFO:tasks.ceph.mon.a.smithi115.stderr:2024-09-10T15:58:23.736+0000 7fa206bdb640 -1 mon.a@0(leader).osd e33 definitely_dead 0 2024-09-10T15:58:24.173 INFO:teuthology.orchestra.run.smithi115.stderr:marked in osd.2. 2024-09-10T15:58:24.193 INFO:tasks.lost_unfound.ceph_manager:waiting for osd.2 to be up 2024-09-10T15:58:24.193 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T15:58:24.478 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T15:58:24.479 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":34,"fsid":"7effee29-df05-4351-9741-ee203564ed81","created":"2024-09-10T15:57:14.967498+0000","modified":"2024-09-10T15:58:24.166686+0000","last_up_change":"2024-09-10T15:58:20.137127+0000","last_in_change":"2024-09-10T15:58:24.166686+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":7,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2024-09-10T15:57:30.381324+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":14,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.9900000095367432,"score_stable":1.4900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":1,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"unfound_pool","create_time":"2024-09-10T15:57:39.758755+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":16,"pg_placement_num":16,"pg_placement_num_target":16,"pg_num_target":16,"pg_num_pending":16,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rados":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.9900000095367432,"score_stable":1.8700000047683716,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"859508e0-e15d-4011-a39f-65e7aa04a129","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":21,"up_from":25,"up_thru":32,"down_at":22,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":1567796493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6801","nonce":1567796493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6803","nonce":1567796493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":1567796493}]},"public_addr":"172.21.15.115:6800/1567796493","cluster_addr":"172.21.15.115:6801/1567796493","heartbeat_back_addr":"172.21.15.115:6803/1567796493","heartbeat_front_addr":"172.21.15.115:6802/1567796493","state":["exists","up"]},{"osd":1,"uuid":"66daed2f-a693-4a13-8d8c-024794a032e3","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":25,"down_at":27,"lost_at":27,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":2071140438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6809","nonce":2071140438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6811","nonce":2071140438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":2071140438}]},"public_addr":"172.21.15.115:6808/2071140438","cluster_addr":"172.21.15.115:6809/2071140438","heartbeat_back_addr":"172.21.15.115:6811/2071140438","heartbeat_front_addr":"172.21.15.115:6810/2071140438","state":["exists"]},{"osd":2,"uuid":"6a2ab153-d99e-4a5d-b79c-32b7f8ea9ffb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":18,"up_from":31,"up_thru":16,"down_at":19,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":2928349610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6805","nonce":2928349610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6807","nonce":2928349610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":2928349610}]},"public_addr":"172.21.15.115:6804/2928349610","cluster_addr":"172.21.15.115:6805/2928349610","heartbeat_back_addr":"172.21.15.115:6807/2928349610","heartbeat_front_addr":"172.21.15.115:6806/2928349610","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T15:57:23.007077+0000","dead_epoch":22},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":65536,"last_purged_snaps_scrub":"2024-09-10T15:57:23.605294+0000","dead_epoch":27},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T15:57:22.852045+0000","dead_epoch":19}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[{"pgid":"1.1","osds":[0,2]},{"pgid":"1.3","osds":[0,2]},{"pgid":"1.5","osds":[0,2]},{"pgid":"1.7","osds":[0,2]},{"pgid":"2.0","osds":[0,2]},{"pgid":"2.1","osds":[0,2]},{"pgid":"2.3","osds":[0,2]},{"pgid":"2.c","osds":[0,2]},{"pgid":"2.d","osds":[0,2]},{"pgid":"2.e","osds":[0,2]}],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T15:58:24.491 INFO:tasks.lost_unfound.ceph_manager:osd.2 is up 2024-09-10T15:58:24.491 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-10T15:58:24.492 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-10T15:58:24.627 INFO:teuthology.orchestra.run.smithi115.stdout:133143986179 2024-09-10T15:58:24.627 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-10T15:58:24.628 INFO:teuthology.orchestra.run.smithi115.stdout:107374182407 2024-09-10T15:58:24.628 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-10T15:58:24.969 INFO:teuthology.orchestra.run.smithi115.stdout:107374182406 2024-09-10T15:58:24.970 INFO:teuthology.orchestra.run.smithi115.stdout:1 2024-09-10T15:58:24.982 INFO:tasks.lost_unfound.ceph_manager:need seq 107374182407 got 107374182406 for osd.0 2024-09-10T15:58:24.983 INFO:tasks.lost_unfound.ceph_manager:need seq 133143986179 got 1 for osd.2 2024-09-10T15:58:25.763 INFO:tasks.ceph.osd.0.smithi115.stderr:2024-09-10T15:58:25.761+0000 7fe20fa7b640 -1 log_channel(cluster) log [ERR] : 2.8 has 1 objects unfound and apparently lost 2024-09-10T15:58:25.764 INFO:tasks.ceph.osd.0.smithi115.stderr:2024-09-10T15:58:25.761+0000 7fe20da77640 -1 log_channel(cluster) log [ERR] : 2.4 has 3 objects unfound and apparently lost 2024-09-10T15:58:25.764 INFO:tasks.ceph.osd.0.smithi115.stderr:2024-09-10T15:58:25.761+0000 7fe20aa71640 -1 log_channel(cluster) log [ERR] : 2.a has 2 objects unfound and apparently lost 2024-09-10T15:58:25.764 INFO:tasks.ceph.osd.0.smithi115.stderr:2024-09-10T15:58:25.762+0000 7fe20a270640 -1 log_channel(cluster) log [ERR] : 2.b has 2 objects unfound and apparently lost 2024-09-10T15:58:25.764 INFO:tasks.ceph.osd.0.smithi115.stderr:2024-09-10T15:58:25.762+0000 7fe20ea79640 -1 log_channel(cluster) log [ERR] : 2.2 has 2 objects unfound and apparently lost 2024-09-10T15:58:25.765 INFO:tasks.ceph.osd.0.smithi115.stderr:2024-09-10T15:58:25.762+0000 7fe20926e640 -1 log_channel(cluster) log [ERR] : 2.5 has 2 objects unfound and apparently lost 2024-09-10T15:58:25.765 INFO:tasks.ceph.osd.0.smithi115.stderr:2024-09-10T15:58:25.762+0000 7fe20826c640 -1 log_channel(cluster) log [ERR] : 2.7 has 1 objects unfound and apparently lost 2024-09-10T15:58:25.765 INFO:tasks.ceph.osd.0.smithi115.stderr:2024-09-10T15:58:25.762+0000 7fe208a6d640 -1 log_channel(cluster) log [ERR] : 2.6 has 3 objects unfound and apparently lost 2024-09-10T15:58:25.765 INFO:tasks.ceph.osd.0.smithi115.stderr:2024-09-10T15:58:25.762+0000 7fe20f27a640 -1 log_channel(cluster) log [ERR] : 2.9 has 2 objects unfound and apparently lost 2024-09-10T15:58:25.983 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-10T15:58:25.984 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-10T15:58:26.292 INFO:teuthology.orchestra.run.smithi115.stdout:107374182407 2024-09-10T15:58:26.293 INFO:teuthology.orchestra.run.smithi115.stdout:133143986179 2024-09-10T15:58:26.305 INFO:tasks.lost_unfound.ceph_manager:need seq 107374182407 got 107374182407 for osd.0 2024-09-10T15:58:26.306 DEBUG:teuthology.parallel:result is None 2024-09-10T15:58:26.306 INFO:tasks.lost_unfound.ceph_manager:need seq 133143986179 got 133143986179 for osd.2 2024-09-10T15:58:26.307 DEBUG:teuthology.parallel:result is None 2024-09-10T15:58:26.307 INFO:tasks.lost_unfound.ceph_manager:waiting till active 2024-09-10T15:58:26.307 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:58:26.628 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T15:58:26.629 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T15:58:26.641 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":64,"stamp":"2024-09-10T15:58:25.760924+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":13543,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":27,"num_objects_missing":27,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57,"num_write_kb":48,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":58,"ondisk_log_size":58,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":36,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":55616,"kb_used_data":596,"kb_used_omap":8,"kb_used_meta":54967,"kb_avail":188688064,"statfs":{"total":193273528320,"available":193216577536,"internally_reserved":0,"allocated":610304,"data_stored":180063,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8876,"internal_metadata":56286548},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-57,"num_objects_misplaced":0,"num_objects_unfound":-27,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.226700"},"pg_stats":[{"pgid":"2.f","version":"0'0","reported_seq":41,"reported_epoch":33,"state":"peering","last_fresh":"2024-09-10T15:58:22.152465+0000","last_change":"2024-09-10T15:58:24.181223+0000","last_active":"2024-09-10T15:58:24.181223+0000","last_peered":"2024-09-10T15:58:22.152465+0000","last_clean":"2024-09-10T15:58:03.029267+0000","last_became_active":"2024-09-10T15:58:12.055196+0000","last_became_peered":"2024-09-10T15:58:12.055196+0000","last_unstale":"2024-09-10T15:58:22.152465+0000","last_undegraded":"2024-09-10T15:58:22.152465+0000","last_fullsized":"2024-09-10T15:58:12.051834+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"23'4","reported_seq":51,"reported_epoch":33,"state":"remapped+peering","last_fresh":"2024-09-10T15:58:22.161009+0000","last_change":"2024-09-10T15:58:24.180851+0000","last_active":"2024-09-10T15:58:24.180851+0000","last_peered":"2024-09-10T15:58:22.161009+0000","last_clean":"2024-09-10T15:57:51.316978+0000","last_became_active":"2024-09-10T15:58:22.160292+0000","last_became_peered":"2024-09-10T15:58:22.160292+0000","last_unstale":"2024-09-10T15:58:22.161009+0000","last_undegraded":"2024-09-10T15:58:22.152813+0000","last_fullsized":"2024-09-10T15:58:22.161009+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":2,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"23'3","reported_seq":62,"reported_epoch":33,"state":"remapped+peering","last_fresh":"2024-09-10T15:58:22.160597+0000","last_change":"2024-09-10T15:58:24.177584+0000","last_active":"2024-09-10T15:58:24.177584+0000","last_peered":"2024-09-10T15:58:22.160597+0000","last_clean":"2024-09-10T15:57:51.863524+0000","last_became_active":"2024-09-10T15:58:22.160148+0000","last_became_peered":"2024-09-10T15:58:22.160148+0000","last_unstale":"2024-09-10T15:58:22.160597+0000","last_undegraded":"2024-09-10T15:58:22.154075+0000","last_fullsized":"2024-09-10T15:58:22.160597+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":2,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":55,"reported_epoch":33,"state":"remapped+peering","last_fresh":"2024-09-10T15:58:22.165163+0000","last_change":"2024-09-10T15:58:24.178892+0000","last_active":"2024-09-10T15:58:24.178892+0000","last_peered":"2024-09-10T15:58:22.165163+0000","last_clean":"2024-09-10T15:58:22.165163+0000","last_became_active":"2024-09-10T15:58:22.160059+0000","last_became_peered":"2024-09-10T15:58:22.160059+0000","last_unstale":"2024-09-10T15:58:22.165163+0000","last_undegraded":"2024-09-10T15:58:22.165163+0000","last_fullsized":"2024-09-10T15:58:22.165163+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":2,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.b","version":"23'5","reported_seq":57,"reported_epoch":34,"state":"peering","last_fresh":"2024-09-10T15:58:24.178447+0000","last_change":"2024-09-10T15:58:24.178447+0000","last_active":"2024-09-10T15:58:24.178447+0000","last_peered":"2024-09-10T15:58:22.160914+0000","last_clean":"2024-09-10T15:57:51.217636+0000","last_became_active":"2024-09-10T15:58:22.160370+0000","last_became_peered":"2024-09-10T15:58:22.160370+0000","last_unstale":"2024-09-10T15:58:24.178447+0000","last_undegraded":"2024-09-10T15:58:24.178447+0000","last_fullsized":"2024-09-10T15:58:24.178447+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"23'3","reported_seq":54,"reported_epoch":33,"state":"peering","last_fresh":"2024-09-10T15:58:22.152225+0000","last_change":"2024-09-10T15:58:24.176200+0000","last_active":"2024-09-10T15:58:24.176200+0000","last_peered":"2024-09-10T15:58:22.152225+0000","last_clean":"2024-09-10T15:57:52.065335+0000","last_became_active":"2024-09-10T15:58:12.054653+0000","last_became_peered":"2024-09-10T15:58:12.054653+0000","last_unstale":"2024-09-10T15:58:22.152225+0000","last_undegraded":"2024-09-10T15:58:12.050760+0000","last_fullsized":"2024-09-10T15:58:12.050644+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"23'4","reported_seq":56,"reported_epoch":33,"state":"peering","last_fresh":"2024-09-10T15:58:22.153277+0000","last_change":"2024-09-10T15:58:24.177087+0000","last_active":"2024-09-10T15:58:24.177087+0000","last_peered":"2024-09-10T15:58:22.153277+0000","last_clean":"2024-09-10T15:57:51.467206+0000","last_became_active":"2024-09-10T15:58:12.053464+0000","last_became_peered":"2024-09-10T15:58:12.053464+0000","last_unstale":"2024-09-10T15:58:22.153277+0000","last_undegraded":"2024-09-10T15:58:12.050229+0000","last_fullsized":"2024-09-10T15:58:12.050113+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"23'1","reported_seq":54,"reported_epoch":34,"state":"peering","last_fresh":"2024-09-10T15:58:24.177260+0000","last_change":"2024-09-10T15:58:24.177260+0000","last_active":"2024-09-10T15:58:24.177260+0000","last_peered":"2024-09-10T15:58:22.162152+0000","last_clean":"2024-09-10T15:57:42.760798+0000","last_became_active":"2024-09-10T15:58:22.161364+0000","last_became_peered":"2024-09-10T15:58:22.161364+0000","last_unstale":"2024-09-10T15:58:24.177260+0000","last_undegraded":"2024-09-10T15:58:24.177260+0000","last_fullsized":"2024-09-10T15:58:24.177260+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":33,"state":"peering","last_fresh":"2024-09-10T15:58:22.151716+0000","last_change":"2024-09-10T15:58:24.175554+0000","last_active":"2024-09-10T15:58:24.175554+0000","last_peered":"2024-09-10T15:58:22.151716+0000","last_clean":"2024-09-10T15:57:45.762460+0000","last_became_active":"2024-09-10T15:58:12.051196+0000","last_became_peered":"2024-09-10T15:58:12.051196+0000","last_unstale":"2024-09-10T15:58:22.151716+0000","last_undegraded":"2024-09-10T15:58:22.151716+0000","last_fullsized":"2024-09-10T15:58:12.049713+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"23'5","reported_seq":64,"reported_epoch":33,"state":"remapped+peering","last_fresh":"2024-09-10T15:58:22.161167+0000","last_change":"2024-09-10T15:58:24.176955+0000","last_active":"2024-09-10T15:58:24.176955+0000","last_peered":"2024-09-10T15:58:22.161167+0000","last_clean":"2024-09-10T15:57:51.913440+0000","last_became_active":"2024-09-10T15:58:22.160855+0000","last_became_peered":"2024-09-10T15:58:22.160855+0000","last_unstale":"2024-09-10T15:58:22.161167+0000","last_undegraded":"2024-09-10T15:58:22.152110+0000","last_fullsized":"2024-09-10T15:58:22.161167+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":2,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":68,"reported_epoch":33,"state":"remapped+peering","last_fresh":"2024-09-10T15:58:22.162612+0000","last_change":"2024-09-10T15:58:24.175951+0000","last_active":"2024-09-10T15:58:24.175951+0000","last_peered":"2024-09-10T15:58:22.162612+0000","last_clean":"2024-09-10T15:58:22.162612+0000","last_became_active":"2024-09-10T15:58:22.156823+0000","last_became_peered":"2024-09-10T15:58:22.156823+0000","last_unstale":"2024-09-10T15:58:22.162612+0000","last_undegraded":"2024-09-10T15:58:22.162612+0000","last_fullsized":"2024-09-10T15:58:22.162612+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":2,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"23'6","reported_seq":49,"reported_epoch":33,"state":"peering","last_fresh":"2024-09-10T15:58:22.153130+0000","last_change":"2024-09-10T15:58:24.176775+0000","last_active":"2024-09-10T15:58:24.176775+0000","last_peered":"2024-09-10T15:58:22.153130+0000","last_clean":"2024-09-10T15:57:52.118434+0000","last_became_active":"2024-09-10T15:58:12.054950+0000","last_became_peered":"2024-09-10T15:58:12.054950+0000","last_unstale":"2024-09-10T15:58:22.153130+0000","last_undegraded":"2024-09-10T15:58:12.051089+0000","last_fullsized":"2024-09-10T15:58:12.050975+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":56,"reported_epoch":33,"state":"peering","last_fresh":"2024-09-10T15:58:22.151966+0000","last_change":"2024-09-10T15:58:24.175614+0000","last_active":"2024-09-10T15:58:24.175614+0000","last_peered":"2024-09-10T15:58:22.151966+0000","last_clean":"2024-09-10T15:58:03.029025+0000","last_became_active":"2024-09-10T15:58:12.054074+0000","last_became_peered":"2024-09-10T15:58:12.054074+0000","last_unstale":"2024-09-10T15:58:22.151966+0000","last_undegraded":"2024-09-10T15:58:12.050434+0000","last_fullsized":"2024-09-10T15:58:12.049818+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"23'3","reported_seq":58,"reported_epoch":33,"state":"remapped+peering","last_fresh":"2024-09-10T15:58:22.160847+0000","last_change":"2024-09-10T15:58:24.176633+0000","last_active":"2024-09-10T15:58:24.176633+0000","last_peered":"2024-09-10T15:58:22.160847+0000","last_clean":"2024-09-10T15:57:51.167715+0000","last_became_active":"2024-09-10T15:58:22.160286+0000","last_became_peered":"2024-09-10T15:58:22.160286+0000","last_unstale":"2024-09-10T15:58:22.160847+0000","last_undegraded":"2024-09-10T15:58:22.152148+0000","last_fullsized":"2024-09-10T15:58:22.160847+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":2,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":33,"state":"remapped+peering","last_fresh":"2024-09-10T15:58:22.165339+0000","last_change":"2024-09-10T15:58:24.176673+0000","last_active":"2024-09-10T15:58:24.176673+0000","last_peered":"2024-09-10T15:58:22.165339+0000","last_clean":"2024-09-10T15:58:22.165339+0000","last_became_active":"2024-09-10T15:58:22.161190+0000","last_became_peered":"2024-09-10T15:58:22.161190+0000","last_unstale":"2024-09-10T15:58:22.165339+0000","last_undegraded":"2024-09-10T15:58:22.165339+0000","last_fullsized":"2024-09-10T15:58:22.165339+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":2,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"23'4","reported_seq":58,"reported_epoch":33,"state":"remapped+peering","last_fresh":"2024-09-10T15:58:22.161803+0000","last_change":"2024-09-10T15:58:24.175747+0000","last_active":"2024-09-10T15:58:24.175747+0000","last_peered":"2024-09-10T15:58:22.161803+0000","last_clean":"2024-09-10T15:57:51.566307+0000","last_became_active":"2024-09-10T15:58:22.161497+0000","last_became_peered":"2024-09-10T15:58:22.161497+0000","last_unstale":"2024-09-10T15:58:22.161803+0000","last_undegraded":"2024-09-10T15:58:22.153382+0000","last_fullsized":"2024-09-10T15:58:22.161803+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":2,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"23'1","reported_seq":51,"reported_epoch":33,"state":"peering","last_fresh":"2024-09-10T15:58:22.152263+0000","last_change":"2024-09-10T15:58:24.179459+0000","last_active":"2024-09-10T15:58:24.179459+0000","last_peered":"2024-09-10T15:58:22.152263+0000","last_clean":"2024-09-10T15:57:45.763054+0000","last_became_active":"2024-09-10T15:58:12.054930+0000","last_became_peered":"2024-09-10T15:58:12.054930+0000","last_unstale":"2024-09-10T15:58:22.152263+0000","last_undegraded":"2024-09-10T15:58:12.051743+0000","last_fullsized":"2024-09-10T15:58:12.051629+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":33,"state":"peering","last_fresh":"2024-09-10T15:58:22.152887+0000","last_change":"2024-09-10T15:58:24.178502+0000","last_active":"2024-09-10T15:58:24.178502+0000","last_peered":"2024-09-10T15:58:22.152887+0000","last_clean":"2024-09-10T15:57:45.762982+0000","last_became_active":"2024-09-10T15:58:12.053258+0000","last_became_peered":"2024-09-10T15:58:12.053258+0000","last_unstale":"2024-09-10T15:58:22.152887+0000","last_undegraded":"2024-09-10T15:58:22.152887+0000","last_fullsized":"2024-09-10T15:58:12.049966+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"23'5","reported_seq":56,"reported_epoch":33,"state":"peering","last_fresh":"2024-09-10T15:58:22.153288+0000","last_change":"2024-09-10T15:58:24.179032+0000","last_active":"2024-09-10T15:58:24.179032+0000","last_peered":"2024-09-10T15:58:22.153288+0000","last_clean":"2024-09-10T15:57:52.013178+0000","last_became_active":"2024-09-10T15:58:12.055065+0000","last_became_peered":"2024-09-10T15:58:12.055065+0000","last_unstale":"2024-09-10T15:58:22.153288+0000","last_undegraded":"2024-09-10T15:58:12.051534+0000","last_fullsized":"2024-09-10T15:58:12.051377+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":3,"num_objects_missing":3,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":68,"reported_epoch":33,"state":"remapped+peering","last_fresh":"2024-09-10T15:58:22.165277+0000","last_change":"2024-09-10T15:58:24.177614+0000","last_active":"2024-09-10T15:58:24.177614+0000","last_peered":"2024-09-10T15:58:22.165277+0000","last_clean":"2024-09-10T15:58:22.165277+0000","last_became_active":"2024-09-10T15:58:22.160068+0000","last_became_peered":"2024-09-10T15:58:22.160068+0000","last_unstale":"2024-09-10T15:58:22.165277+0000","last_undegraded":"2024-09-10T15:58:22.165277+0000","last_fullsized":"2024-09-10T15:58:22.165277+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":2,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"23'2","reported_seq":52,"reported_epoch":33,"state":"peering","last_fresh":"2024-09-10T15:58:22.154094+0000","last_change":"2024-09-10T15:58:24.179084+0000","last_active":"2024-09-10T15:58:24.179084+0000","last_peered":"2024-09-10T15:58:22.154094+0000","last_clean":"2024-09-10T15:57:45.763338+0000","last_became_active":"2024-09-10T15:58:12.054884+0000","last_became_peered":"2024-09-10T15:58:12.054884+0000","last_unstale":"2024-09-10T15:58:22.154094+0000","last_undegraded":"2024-09-10T15:58:12.051953+0000","last_fullsized":"2024-09-10T15:58:12.051832+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":33,"state":"peering","last_fresh":"2024-09-10T15:58:22.152421+0000","last_change":"2024-09-10T15:58:24.177485+0000","last_active":"2024-09-10T15:58:24.177485+0000","last_peered":"2024-09-10T15:58:22.152421+0000","last_clean":"2024-09-10T15:57:45.762845+0000","last_became_active":"2024-09-10T15:58:12.053318+0000","last_became_peered":"2024-09-10T15:58:12.053318+0000","last_unstale":"2024-09-10T15:58:22.152421+0000","last_undegraded":"2024-09-10T15:58:22.152421+0000","last_fullsized":"2024-09-10T15:58:12.050277+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":33,"state":"remapped+peering","last_fresh":"2024-09-10T15:58:22.165251+0000","last_change":"2024-09-10T15:58:24.180778+0000","last_active":"2024-09-10T15:58:24.180778+0000","last_peered":"2024-09-10T15:58:22.165251+0000","last_clean":"2024-09-10T15:58:22.165251+0000","last_became_active":"2024-09-10T15:58:22.160854+0000","last_became_peered":"2024-09-10T15:58:22.160854+0000","last_unstale":"2024-09-10T15:58:22.165251+0000","last_undegraded":"2024-09-10T15:58:22.165251+0000","last_fullsized":"2024-09-10T15:58:22.165251+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":2,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"23'9","reported_seq":60,"reported_epoch":33,"state":"peering","last_fresh":"2024-09-10T15:58:22.153320+0000","last_change":"2024-09-10T15:58:24.177295+0000","last_active":"2024-09-10T15:58:24.177295+0000","last_peered":"2024-09-10T15:58:22.153320+0000","last_clean":"2024-09-10T15:57:51.765160+0000","last_became_active":"2024-09-10T15:58:12.054738+0000","last_became_peered":"2024-09-10T15:58:12.054738+0000","last_unstale":"2024-09-10T15:58:22.153320+0000","last_undegraded":"2024-09-10T15:58:12.051645+0000","last_fullsized":"2024-09-10T15:58:12.051197+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1932,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":3,"num_objects_missing":3,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":13524,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":27,"num_objects_missing":27,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":55,"num_write_kb":46,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":155648,"data_stored":18354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":55,"ondisk_log_size":55,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986179,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27824,"kb_used_data":280,"kb_used_omap":4,"kb_used_meta":27515,"kb_avail":94344016,"statfs":{"total":96636764160,"available":96608272384,"internally_reserved":0,"allocated":286720,"data_stored":89277,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":28176042},"hb_peers":[0],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182407,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27792,"kb_used_data":316,"kb_used_omap":4,"kb_used_meta":27451,"kb_avail":94344048,"statfs":{"total":96636764160,"available":96608305152,"internally_reserved":0,"allocated":323584,"data_stored":90786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":28110506},"hb_peers":[2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":4830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":13524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:58:26.642 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T15:58:26.990 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T15:58:26.990 INFO:teuthology.orchestra.run.smithi115.stdout:{"fsid":"7effee29-df05-4351-9741-ee203564ed81","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":71,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":36,"num_osds":3,"num_up_osds":2,"osd_up_since":1725983900,"num_in_osds":2,"osd_in_since":1725983904,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"peering","count":14},{"state_name":"remapped+peering","count":10}],"num_pgs":24,"num_pools":2,"num_objects":30,"data_bytes":13543,"bytes_used":56950784,"bytes_avail":193216577536,"bytes_total":193273528320,"inactive_pgs_ratio":1},"fsmap":{"epoch":1,"btime":"2024-09-10T15:57:14:967120+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":2,"modified":"2024-09-10T15:58:21.534287+0000","services":{"osd":{"daemons":{"summary":"","0":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"1":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"2":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-10T15:58:27.003 INFO:tasks.lost_unfound.ceph_manager:{'fsid': '7effee29-df05-4351-9741-ee203564ed81', 'health': {'status': 'HEALTH_OK', 'checks': {}, 'mutes': []}, 'election_epoch': 10, 'quorum': [0, 1, 2], 'quorum_names': ['a', 'b', 'c'], 'quorum_age': 71, 'monmap': {'epoch': 1, 'min_mon_release_name': 'squid', 'num_mons': 3}, 'osdmap': {'epoch': 36, 'num_osds': 3, 'num_up_osds': 2, 'osd_up_since': 1725983900, 'num_in_osds': 2, 'osd_in_since': 1725983904, 'num_remapped_pgs': 0}, 'pgmap': {'pgs_by_state': [{'state_name': 'peering', 'count': 14}, {'state_name': 'remapped+peering', 'count': 10}], 'num_pgs': 24, 'num_pools': 2, 'num_objects': 30, 'data_bytes': 13543, 'bytes_used': 56950784, 'bytes_avail': 193216577536, 'bytes_total': 193273528320, 'inactive_pgs_ratio': 1}, 'fsmap': {'epoch': 1, 'btime': '2024-09-10T15:57:14:967120+0000', 'by_rank': [], 'up:standby': 0}, 'mgrmap': {'available': True, 'num_standbys': 0, 'modules': ['iostat', 'nfs', 'restful'], 'services': {}}, 'servicemap': {'epoch': 2, 'modified': '2024-09-10T15:58:21.534287+0000', 'services': {'osd': {'daemons': {'summary': '', '0': {'start_epoch': 0, 'start_stamp': '0.000000', 'gid': 0, 'addr': '(unrecognized address family 0)/0', 'metadata': {}, 'task_status': {}}, '1': {'start_epoch': 0, 'start_stamp': '0.000000', 'gid': 0, 'addr': '(unrecognized address family 0)/0', 'metadata': {}, 'task_status': {}}, '2': {'start_epoch': 0, 'start_stamp': '0.000000', 'gid': 0, 'addr': '(unrecognized address family 0)/0', 'metadata': {}, 'task_status': {}}}}}}, 'progress_events': {}} 2024-09-10T15:58:29.762 INFO:tasks.ceph.mon.a.smithi115.stderr:2024-09-10T15:58:29.760+0000 7fa2093e0640 -1 log_channel(cluster) log [ERR] : Health check failed: Possible data damage: 5 pgs recovery_unfound (PG_DAMAGED) 2024-09-10T15:58:30.004 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:58:30.297 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T15:58:30.297 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T15:58:30.309 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":66,"stamp":"2024-09-10T15:58:29.536029+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":13543,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":27,"num_objects_missing":27,"num_objects_degraded":54,"num_objects_misplaced":0,"num_objects_unfound":27,"num_objects_dirty":30,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57,"num_write_kb":48,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":58,"ondisk_log_size":58,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":48,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":55656,"kb_used_data":628,"kb_used_omap":8,"kb_used_meta":54967,"kb_avail":188688024,"statfs":{"total":193273528320,"available":193216536576,"internally_reserved":0,"allocated":643072,"data_stored":191941,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8876,"internal_metadata":56286548},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-3,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001228"},"pg_stats":[{"pgid":"2.f","version":"0'0","reported_seq":52,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T15:58:25.763387+0000","last_change":"2024-09-10T15:58:24.769328+0000","last_active":"2024-09-10T15:58:25.763387+0000","last_peered":"2024-09-10T15:58:25.763387+0000","last_clean":"2024-09-10T15:58:25.763387+0000","last_became_active":"2024-09-10T15:58:24.769050+0000","last_became_peered":"2024-09-10T15:58:24.769050+0000","last_unstale":"2024-09-10T15:58:25.763387+0000","last_undegraded":"2024-09-10T15:58:25.763387+0000","last_fullsized":"2024-09-10T15:58:25.763387+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:08:36.167566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"23'4","reported_seq":57,"reported_epoch":36,"state":"active+recovery_unfound","last_fresh":"2024-09-10T15:58:25.770690+0000","last_change":"2024-09-10T15:58:25.770880+0000","last_active":"2024-09-10T15:58:25.770690+0000","last_peered":"2024-09-10T15:58:25.770690+0000","last_clean":"2024-09-10T15:57:51.316978+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T15:58:25.770690+0000","last_undegraded":"2024-09-10T15:58:25.764660+0000","last_fullsized":"2024-09-10T15:58:25.770690+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"23'3","reported_seq":69,"reported_epoch":36,"state":"active+recovery_unfound+degraded","last_fresh":"2024-09-10T15:58:25.769914+0000","last_change":"2024-09-10T15:58:29.145692+0000","last_active":"2024-09-10T15:58:25.769914+0000","last_peered":"2024-09-10T15:58:25.769914+0000","last_clean":"2024-09-10T15:57:51.863524+0000","last_became_active":"2024-09-10T15:58:25.767861+0000","last_became_peered":"2024-09-10T15:58:25.767861+0000","last_unstale":"2024-09-10T15:58:25.769914+0000","last_undegraded":"2024-09-10T15:58:25.769914+0000","last_fullsized":"2024-09-10T15:58:25.769914+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":1,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":61,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T15:58:25.768463+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T15:58:25.768463+0000","last_peered":"2024-09-10T15:58:25.768463+0000","last_clean":"2024-09-10T15:58:25.768463+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T15:58:25.768463+0000","last_undegraded":"2024-09-10T15:58:25.768463+0000","last_fullsized":"2024-09-10T15:58:25.768463+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"23'5","reported_seq":65,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-10T15:58:25.762992+0000","last_change":"2024-09-10T15:58:24.766952+0000","last_active":"2024-09-10T15:58:25.762992+0000","last_peered":"2024-09-10T15:58:25.762992+0000","last_clean":"2024-09-10T15:57:51.217636+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T15:58:25.762992+0000","last_undegraded":"2024-09-10T15:58:24.759175+0000","last_fullsized":"2024-09-10T15:58:25.762992+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"23'3","reported_seq":62,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-10T15:58:25.762768+0000","last_change":"2024-09-10T15:58:24.768439+0000","last_active":"2024-09-10T15:58:25.762768+0000","last_peered":"2024-09-10T15:58:25.762768+0000","last_clean":"2024-09-10T15:57:52.065335+0000","last_became_active":"2024-09-10T15:58:24.768198+0000","last_became_peered":"2024-09-10T15:58:24.768198+0000","last_unstale":"2024-09-10T15:58:25.762768+0000","last_undegraded":"2024-09-10T15:58:24.758149+0000","last_fullsized":"2024-09-10T15:58:25.762768+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"23'4","reported_seq":64,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-10T15:58:25.763467+0000","last_change":"2024-09-10T15:58:24.769216+0000","last_active":"2024-09-10T15:58:25.763467+0000","last_peered":"2024-09-10T15:58:25.763467+0000","last_clean":"2024-09-10T15:57:51.467206+0000","last_became_active":"2024-09-10T15:58:24.768972+0000","last_became_peered":"2024-09-10T15:58:24.768972+0000","last_unstale":"2024-09-10T15:58:25.763467+0000","last_undegraded":"2024-09-10T15:58:24.758006+0000","last_fullsized":"2024-09-10T15:58:25.763467+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"23'1","reported_seq":63,"reported_epoch":36,"state":"active+recovering","last_fresh":"2024-09-10T15:58:25.762598+0000","last_change":"2024-09-10T15:58:25.762598+0000","last_active":"2024-09-10T15:58:25.762598+0000","last_peered":"2024-09-10T15:58:25.762598+0000","last_clean":"2024-09-10T15:57:42.760798+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T15:58:25.762598+0000","last_undegraded":"2024-09-10T15:58:25.762598+0000","last_fullsized":"2024-09-10T15:58:25.762598+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":1,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T15:58:25.762375+0000","last_change":"2024-09-10T15:58:24.767197+0000","last_active":"2024-09-10T15:58:25.762375+0000","last_peered":"2024-09-10T15:58:25.762375+0000","last_clean":"2024-09-10T15:58:25.762375+0000","last_became_active":"2024-09-10T15:58:24.766691+0000","last_became_peered":"2024-09-10T15:58:24.766691+0000","last_unstale":"2024-09-10T15:58:25.762375+0000","last_undegraded":"2024-09-10T15:58:25.762375+0000","last_fullsized":"2024-09-10T15:58:25.762375+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:51:00.161051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"23'5","reported_seq":71,"reported_epoch":36,"state":"active+recovery_unfound+degraded","last_fresh":"2024-09-10T15:58:25.771896+0000","last_change":"2024-09-10T15:58:29.145649+0000","last_active":"2024-09-10T15:58:25.771896+0000","last_peered":"2024-09-10T15:58:25.771896+0000","last_clean":"2024-09-10T15:57:51.913440+0000","last_became_active":"2024-09-10T15:58:25.769634+0000","last_became_peered":"2024-09-10T15:58:25.769634+0000","last_unstale":"2024-09-10T15:58:25.771896+0000","last_undegraded":"2024-09-10T15:58:25.771896+0000","last_fullsized":"2024-09-10T15:58:25.771896+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":74,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T15:58:25.768681+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T15:58:25.768681+0000","last_peered":"2024-09-10T15:58:25.768681+0000","last_clean":"2024-09-10T15:58:25.768681+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T15:58:25.768681+0000","last_undegraded":"2024-09-10T15:58:25.768681+0000","last_fullsized":"2024-09-10T15:58:25.768681+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"23'6","reported_seq":57,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-10T15:58:25.762729+0000","last_change":"2024-09-10T15:58:24.768933+0000","last_active":"2024-09-10T15:58:25.762729+0000","last_peered":"2024-09-10T15:58:25.762729+0000","last_clean":"2024-09-10T15:57:52.118434+0000","last_became_active":"2024-09-10T15:58:24.768693+0000","last_became_peered":"2024-09-10T15:58:24.768693+0000","last_unstale":"2024-09-10T15:58:25.762729+0000","last_undegraded":"2024-09-10T15:58:24.758580+0000","last_fullsized":"2024-09-10T15:58:25.762729+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":67,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-10T15:58:25.762469+0000","last_change":"2024-09-10T15:58:24.768453+0000","last_active":"2024-09-10T15:58:25.762469+0000","last_peered":"2024-09-10T15:58:25.762469+0000","last_clean":"2024-09-10T15:58:03.029025+0000","last_became_active":"2024-09-10T15:58:24.766316+0000","last_became_peered":"2024-09-10T15:58:24.766316+0000","last_unstale":"2024-09-10T15:58:25.762469+0000","last_undegraded":"2024-09-10T15:58:24.757744+0000","last_fullsized":"2024-09-10T15:58:25.762469+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0"],"object_location_counts":[{"shards":"0","objects":1}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"23'3","reported_seq":64,"reported_epoch":36,"state":"active+recovery_unfound","last_fresh":"2024-09-10T15:58:25.771204+0000","last_change":"2024-09-10T15:58:25.771431+0000","last_active":"2024-09-10T15:58:25.771204+0000","last_peered":"2024-09-10T15:58:25.771204+0000","last_clean":"2024-09-10T15:57:51.167715+0000","last_became_active":"2024-09-10T15:58:25.769424+0000","last_became_peered":"2024-09-10T15:58:25.769424+0000","last_unstale":"2024-09-10T15:58:25.771204+0000","last_undegraded":"2024-09-10T15:58:25.764218+0000","last_fullsized":"2024-09-10T15:58:25.771204+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":92,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T15:58:25.770150+0000","last_change":"2024-09-10T15:58:25.768489+0000","last_active":"2024-09-10T15:58:25.770150+0000","last_peered":"2024-09-10T15:58:25.770150+0000","last_clean":"2024-09-10T15:58:25.770150+0000","last_became_active":"2024-09-10T15:58:25.768117+0000","last_became_peered":"2024-09-10T15:58:25.768117+0000","last_unstale":"2024-09-10T15:58:25.770150+0000","last_undegraded":"2024-09-10T15:58:25.770150+0000","last_fullsized":"2024-09-10T15:58:25.770150+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:56:47.213564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"23'4","reported_seq":64,"reported_epoch":36,"state":"active+recovery_unfound","last_fresh":"2024-09-10T15:58:25.772197+0000","last_change":"2024-09-10T15:58:25.772373+0000","last_active":"2024-09-10T15:58:25.772197+0000","last_peered":"2024-09-10T15:58:25.772197+0000","last_clean":"2024-09-10T15:57:51.566307+0000","last_became_active":"2024-09-10T15:58:25.769762+0000","last_became_peered":"2024-09-10T15:58:25.769762+0000","last_unstale":"2024-09-10T15:58:25.772197+0000","last_undegraded":"2024-09-10T15:58:25.764657+0000","last_fullsized":"2024-09-10T15:58:25.772197+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"23'1","reported_seq":59,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-10T15:58:25.763270+0000","last_change":"2024-09-10T15:58:24.768825+0000","last_active":"2024-09-10T15:58:25.763270+0000","last_peered":"2024-09-10T15:58:25.763270+0000","last_clean":"2024-09-10T15:57:45.763054+0000","last_became_active":"2024-09-10T15:58:24.768609+0000","last_became_peered":"2024-09-10T15:58:24.768609+0000","last_unstale":"2024-09-10T15:58:25.763270+0000","last_undegraded":"2024-09-10T15:58:24.757704+0000","last_fullsized":"2024-09-10T15:58:25.763270+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":1,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T15:58:25.762555+0000","last_change":"2024-09-10T15:58:24.767677+0000","last_active":"2024-09-10T15:58:25.762555+0000","last_peered":"2024-09-10T15:58:25.762555+0000","last_clean":"2024-09-10T15:58:25.762555+0000","last_became_active":"2024-09-10T15:58:24.767409+0000","last_became_peered":"2024-09-10T15:58:24.767409+0000","last_unstale":"2024-09-10T15:58:25.762555+0000","last_undegraded":"2024-09-10T15:58:25.762555+0000","last_fullsized":"2024-09-10T15:58:25.762555+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:35:35.763396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"23'5","reported_seq":64,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-10T15:58:25.763204+0000","last_change":"2024-09-10T15:58:24.769497+0000","last_active":"2024-09-10T15:58:25.763204+0000","last_peered":"2024-09-10T15:58:25.763204+0000","last_clean":"2024-09-10T15:57:52.013178+0000","last_became_active":"2024-09-10T15:58:24.769268+0000","last_became_peered":"2024-09-10T15:58:24.769268+0000","last_unstale":"2024-09-10T15:58:25.763204+0000","last_undegraded":"2024-09-10T15:58:24.759230+0000","last_fullsized":"2024-09-10T15:58:25.763204+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":3,"num_objects_missing":3,"num_objects_degraded":6,"num_objects_misplaced":0,"num_objects_unfound":3,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":3}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":74,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T15:58:25.769812+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T15:58:25.769812+0000","last_peered":"2024-09-10T15:58:25.769812+0000","last_clean":"2024-09-10T15:58:25.769812+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T15:58:25.769812+0000","last_undegraded":"2024-09-10T15:58:25.769812+0000","last_fullsized":"2024-09-10T15:58:25.769812+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"23'2","reported_seq":60,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-10T15:58:25.763128+0000","last_change":"2024-09-10T15:58:24.769323+0000","last_active":"2024-09-10T15:58:25.763128+0000","last_peered":"2024-09-10T15:58:25.763128+0000","last_clean":"2024-09-10T15:57:45.763338+0000","last_became_active":"2024-09-10T15:58:24.769074+0000","last_became_peered":"2024-09-10T15:58:24.769074+0000","last_unstale":"2024-09-10T15:58:25.763128+0000","last_undegraded":"2024-09-10T15:58:24.759853+0000","last_fullsized":"2024-09-10T15:58:25.763128+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T15:58:25.762992+0000","last_change":"2024-09-10T15:58:24.767867+0000","last_active":"2024-09-10T15:58:25.762992+0000","last_peered":"2024-09-10T15:58:25.762992+0000","last_clean":"2024-09-10T15:58:25.762992+0000","last_became_active":"2024-09-10T15:58:24.767615+0000","last_became_peered":"2024-09-10T15:58:24.767615+0000","last_unstale":"2024-09-10T15:58:25.762992+0000","last_undegraded":"2024-09-10T15:58:25.762992+0000","last_fullsized":"2024-09-10T15:58:25.762992+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:11:00.908668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":87,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T15:58:25.770511+0000","last_change":"2024-09-10T15:58:25.768847+0000","last_active":"2024-09-10T15:58:25.770511+0000","last_peered":"2024-09-10T15:58:25.770511+0000","last_clean":"2024-09-10T15:58:25.770511+0000","last_became_active":"2024-09-10T15:58:25.768040+0000","last_became_peered":"2024-09-10T15:58:25.768040+0000","last_unstale":"2024-09-10T15:58:25.770511+0000","last_undegraded":"2024-09-10T15:58:25.770511+0000","last_fullsized":"2024-09-10T15:58:25.770511+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:39:30.402776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"23'9","reported_seq":68,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-10T15:58:25.762584+0000","last_change":"2024-09-10T15:58:24.769859+0000","last_active":"2024-09-10T15:58:25.762584+0000","last_peered":"2024-09-10T15:58:25.762584+0000","last_clean":"2024-09-10T15:57:51.765160+0000","last_became_active":"2024-09-10T15:58:24.769616+0000","last_became_peered":"2024-09-10T15:58:24.769616+0000","last_unstale":"2024-09-10T15:58:25.762584+0000","last_undegraded":"2024-09-10T15:58:24.756607+0000","last_fullsized":"2024-09-10T15:58:25.762584+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1932,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":3,"num_objects_missing":3,"num_objects_degraded":7,"num_objects_misplaced":0,"num_objects_unfound":3,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0","objects":1},{"shards":"0,2","objects":3}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":13524,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":27,"num_objects_missing":27,"num_objects_degraded":53,"num_objects_misplaced":0,"num_objects_unfound":27,"num_objects_dirty":28,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":55,"num_write_kb":46,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":155648,"data_stored":18354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":55,"ondisk_log_size":55,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986180,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27840,"kb_used_data":296,"kb_used_omap":4,"kb_used_meta":27515,"kb_avail":94344000,"statfs":{"total":96636764160,"available":96608256000,"internally_reserved":0,"allocated":303104,"data_stored":94899,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":28176042},"hb_peers":[0],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182408,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27816,"kb_used_data":332,"kb_used_omap":4,"kb_used_meta":27451,"kb_avail":94344024,"statfs":{"total":96636764160,"available":96608280576,"internally_reserved":0,"allocated":339968,"data_stored":97042,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":28110506},"hb_peers":[2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":4830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":13524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:58:30.310 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T15:58:30.659 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T15:58:30.659 INFO:teuthology.orchestra.run.smithi115.stdout:{"fsid":"7effee29-df05-4351-9741-ee203564ed81","health":{"status":"HEALTH_ERR","checks":{"OBJECT_UNFOUND":{"severity":"HEALTH_WARN","summary":{"message":"27/30 objects unfound (90.000%)","count":27},"muted":false},"PG_DAMAGED":{"severity":"HEALTH_ERR","summary":{"message":"Possible data damage: 5 pgs recovery_unfound","count":5},"muted":false},"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 54/60 objects degraded (90.000%), 11 pgs degraded","count":11},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":74,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":36,"num_osds":3,"num_up_osds":2,"osd_up_since":1725983900,"num_in_osds":2,"osd_in_since":1725983904,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":9},{"state_name":"active+recovery_wait+degraded","count":9},{"state_name":"active+recovery_unfound","count":3},{"state_name":"active+recovery_unfound+degraded","count":2},{"state_name":"active+recovering","count":1}],"num_pgs":24,"num_pools":2,"num_objects":30,"data_bytes":13543,"bytes_used":56991744,"bytes_avail":193216536576,"bytes_total":193273528320,"degraded_objects":54,"degraded_total":60,"degraded_ratio":0.90000000000000002,"unfound_objects":27,"unfound_total":30,"unfound_ratio":0.90000000000000002},"fsmap":{"epoch":1,"btime":"2024-09-10T15:57:14:967120+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":3,"modified":"2024-09-10T15:58:27.535668+0000","services":{"osd":{"daemons":{"summary":"","0":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"2":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-10T15:58:30.672 INFO:tasks.lost_unfound.ceph_manager:{'fsid': '7effee29-df05-4351-9741-ee203564ed81', 'health': {'status': 'HEALTH_ERR', 'checks': {'OBJECT_UNFOUND': {'severity': 'HEALTH_WARN', 'summary': {'message': '27/30 objects unfound (90.000%)', 'count': 27}, 'muted': False}, 'PG_DAMAGED': {'severity': 'HEALTH_ERR', 'summary': {'message': 'Possible data damage: 5 pgs recovery_unfound', 'count': 5}, 'muted': False}, 'PG_DEGRADED': {'severity': 'HEALTH_WARN', 'summary': {'message': 'Degraded data redundancy: 54/60 objects degraded (90.000%), 11 pgs degraded', 'count': 11}, 'muted': False}}, 'mutes': []}, 'election_epoch': 10, 'quorum': [0, 1, 2], 'quorum_names': ['a', 'b', 'c'], 'quorum_age': 74, 'monmap': {'epoch': 1, 'min_mon_release_name': 'squid', 'num_mons': 3}, 'osdmap': {'epoch': 36, 'num_osds': 3, 'num_up_osds': 2, 'osd_up_since': 1725983900, 'num_in_osds': 2, 'osd_in_since': 1725983904, 'num_remapped_pgs': 0}, 'pgmap': {'pgs_by_state': [{'state_name': 'active+clean', 'count': 9}, {'state_name': 'active+recovery_wait+degraded', 'count': 9}, {'state_name': 'active+recovery_unfound', 'count': 3}, {'state_name': 'active+recovery_unfound+degraded', 'count': 2}, {'state_name': 'active+recovering', 'count': 1}], 'num_pgs': 24, 'num_pools': 2, 'num_objects': 30, 'data_bytes': 13543, 'bytes_used': 56991744, 'bytes_avail': 193216536576, 'bytes_total': 193273528320, 'degraded_objects': 54, 'degraded_total': 60, 'degraded_ratio': 0.9, 'unfound_objects': 27, 'unfound_total': 30, 'unfound_ratio': 0.9}, 'fsmap': {'epoch': 1, 'btime': '2024-09-10T15:57:14:967120+0000', 'by_rank': [], 'up:standby': 0}, 'mgrmap': {'available': True, 'num_standbys': 0, 'modules': ['iostat', 'nfs', 'restful'], 'services': {}}, 'servicemap': {'epoch': 3, 'modified': '2024-09-10T15:58:27.535668+0000', 'services': {'osd': {'daemons': {'summary': '', '0': {'start_epoch': 0, 'start_stamp': '0.000000', 'gid': 0, 'addr': '(unrecognized address family 0)/0', 'metadata': {}, 'task_status': {}}, '2': {'start_epoch': 0, 'start_stamp': '0.000000', 'gid': 0, 'addr': '(unrecognized address family 0)/0', 'metadata': {}, 'task_status': {}}}}}}, 'progress_events': {}} 2024-09-10T15:58:30.672 INFO:tasks.lost_unfound.ceph_manager:active! 2024-09-10T15:58:30.672 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-10T15:58:30.673 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-10T15:58:30.806 INFO:teuthology.orchestra.run.smithi115.stdout:133143986181 2024-09-10T15:58:30.807 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-10T15:58:30.807 INFO:teuthology.orchestra.run.smithi115.stdout:107374182409 2024-09-10T15:58:30.808 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-10T15:58:31.166 INFO:teuthology.orchestra.run.smithi115.stdout:133143986180 2024-09-10T15:58:31.166 INFO:teuthology.orchestra.run.smithi115.stdout:107374182408 2024-09-10T15:58:31.179 INFO:tasks.lost_unfound.ceph_manager:need seq 133143986181 got 133143986180 for osd.2 2024-09-10T15:58:31.180 INFO:tasks.lost_unfound.ceph_manager:need seq 107374182409 got 107374182408 for osd.0 2024-09-10T15:58:32.181 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-10T15:58:32.181 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-10T15:58:32.488 INFO:teuthology.orchestra.run.smithi115.stdout:107374182410 2024-09-10T15:58:32.489 INFO:teuthology.orchestra.run.smithi115.stdout:133143986181 2024-09-10T15:58:32.502 INFO:tasks.lost_unfound.ceph_manager:need seq 107374182409 got 107374182410 for osd.0 2024-09-10T15:58:32.502 DEBUG:teuthology.parallel:result is None 2024-09-10T15:58:32.503 INFO:tasks.lost_unfound.ceph_manager:need seq 133143986181 got 133143986181 for osd.2 2024-09-10T15:58:32.503 DEBUG:teuthology.parallel:result is None 2024-09-10T15:58:32.503 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T15:58:32.896 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T15:58:32.896 INFO:teuthology.orchestra.run.smithi115.stdout:{"fsid":"7effee29-df05-4351-9741-ee203564ed81","health":{"status":"HEALTH_ERR","checks":{"OBJECT_UNFOUND":{"severity":"HEALTH_WARN","summary":{"message":"27/30 objects unfound (90.000%)","count":27},"muted":false},"PG_DAMAGED":{"severity":"HEALTH_ERR","summary":{"message":"Possible data damage: 5 pgs recovery_unfound","count":5},"muted":false},"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 54/60 objects degraded (90.000%), 11 pgs degraded","count":11},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":76,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":36,"num_osds":3,"num_up_osds":2,"osd_up_since":1725983900,"num_in_osds":2,"osd_in_since":1725983904,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":9},{"state_name":"active+recovery_wait+degraded","count":9},{"state_name":"active+recovery_unfound","count":3},{"state_name":"active+recovery_unfound+degraded","count":2},{"state_name":"active+recovering","count":1}],"num_pgs":24,"num_pools":2,"num_objects":30,"data_bytes":13543,"bytes_used":56991744,"bytes_avail":193216536576,"bytes_total":193273528320,"degraded_objects":54,"degraded_total":60,"degraded_ratio":0.90000000000000002,"unfound_objects":27,"unfound_total":30,"unfound_ratio":0.90000000000000002},"fsmap":{"epoch":1,"btime":"2024-09-10T15:57:14:967120+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":3,"modified":"2024-09-10T15:58:27.535668+0000","services":{"osd":{"daemons":{"summary":"","0":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"2":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-10T15:58:32.908 INFO:tasks.lost_unfound.ceph_manager:{'fsid': '7effee29-df05-4351-9741-ee203564ed81', 'health': {'status': 'HEALTH_ERR', 'checks': {'OBJECT_UNFOUND': {'severity': 'HEALTH_WARN', 'summary': {'message': '27/30 objects unfound (90.000%)', 'count': 27}, 'muted': False}, 'PG_DAMAGED': {'severity': 'HEALTH_ERR', 'summary': {'message': 'Possible data damage: 5 pgs recovery_unfound', 'count': 5}, 'muted': False}, 'PG_DEGRADED': {'severity': 'HEALTH_WARN', 'summary': {'message': 'Degraded data redundancy: 54/60 objects degraded (90.000%), 11 pgs degraded', 'count': 11}, 'muted': False}}, 'mutes': []}, 'election_epoch': 10, 'quorum': [0, 1, 2], 'quorum_names': ['a', 'b', 'c'], 'quorum_age': 76, 'monmap': {'epoch': 1, 'min_mon_release_name': 'squid', 'num_mons': 3}, 'osdmap': {'epoch': 36, 'num_osds': 3, 'num_up_osds': 2, 'osd_up_since': 1725983900, 'num_in_osds': 2, 'osd_in_since': 1725983904, 'num_remapped_pgs': 0}, 'pgmap': {'pgs_by_state': [{'state_name': 'active+clean', 'count': 9}, {'state_name': 'active+recovery_wait+degraded', 'count': 9}, {'state_name': 'active+recovery_unfound', 'count': 3}, {'state_name': 'active+recovery_unfound+degraded', 'count': 2}, {'state_name': 'active+recovering', 'count': 1}], 'num_pgs': 24, 'num_pools': 2, 'num_objects': 30, 'data_bytes': 13543, 'bytes_used': 56991744, 'bytes_avail': 193216536576, 'bytes_total': 193273528320, 'degraded_objects': 54, 'degraded_total': 60, 'degraded_ratio': 0.9, 'unfound_objects': 27, 'unfound_total': 30, 'unfound_ratio': 0.9}, 'fsmap': {'epoch': 1, 'btime': '2024-09-10T15:57:14:967120+0000', 'by_rank': [], 'up:standby': 0}, 'mgrmap': {'available': True, 'num_standbys': 0, 'modules': ['iostat', 'nfs', 'restful'], 'services': {}}, 'servicemap': {'epoch': 3, 'modified': '2024-09-10T15:58:27.535668+0000', 'services': {'osd': {'daemons': {'summary': '', '0': {'start_epoch': 0, 'start_stamp': '0.000000', 'gid': 0, 'addr': '(unrecognized address family 0)/0', 'metadata': {}, 'task_status': {}}, '2': {'start_epoch': 0, 'start_stamp': '0.000000', 'gid': 0, 'addr': '(unrecognized address family 0)/0', 'metadata': {}, 'task_status': {}}}}}}, 'progress_events': {}} 2024-09-10T15:58:32.909 INFO:tasks.lost_unfound:there are 27 unfound objects 2024-09-10T15:58:32.909 DEBUG:teuthology.orchestra.run.smithi115:> /bin/sh -c 'adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --no-log-to-stderr --name client.admin -b 4096 -p unfound_pool -t 20 bench 240 write' 2024-09-10T15:58:32.960 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout:hints = 1 2024-09-10T15:58:32.961 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout:Maintaining 20 concurrent writes of 4096 bytes to objects of size 4096 for up to 240 seconds or 0 objects 2024-09-10T15:58:32.961 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout:Object prefix: benchmark_data_smithi115_43363 2024-09-10T15:58:32.961 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-10T15:58:32.961 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 0 0 0 0 0 0 - 0 2024-09-10T15:58:33.961 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 1 20 2913 2893 11.3001 11.3008 0.00514402 0.00686969 2024-09-10T15:58:34.961 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 2 20 5757 5737 11.2043 11.1094 0.00535294 0.00695883 2024-09-10T15:58:35.961 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 3 20 8748 8728 11.3638 11.6836 0.00498033 0.00686358 2024-09-10T15:58:36.961 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 4 20 11654 11634 11.3605 11.3516 0.00751254 0.00686826 2024-09-10T15:58:37.961 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 5 20 14539 14519 11.3421 11.2695 0.006059 0.00688132 2024-09-10T15:58:38.961 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 6 20 17311 17291 11.2563 10.8281 0.00830368 0.00693416 2024-09-10T15:58:39.962 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 7 20 19956 19936 11.124 10.332 0.00510254 0.00701565 2024-09-10T15:58:40.961 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 8 19 22508 22489 10.9799 9.97266 0.00812477 0.00710808 2024-09-10T15:58:41.961 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 9 20 24955 24935 10.8215 9.55469 0.00679599 0.00721275 2024-09-10T15:58:42.913 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:58:42.961 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 10 20 27297 27277 10.6541 9.14844 0.0118939 0.00732644 2024-09-10T15:58:43.454 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T15:58:43.454 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T15:58:43.477 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":72,"stamp":"2024-09-10T15:58:41.538744+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":85292263,"num_objects":20850,"num_object_clones":0,"num_object_copies":41700,"num_objects_missing_on_primary":27,"num_objects_missing":27,"num_objects_degraded":54,"num_objects_misplaced":0,"num_objects_unfound":27,"num_objects_dirty":20850,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20877,"num_write_kb":83328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":20878,"ondisk_log_size":20878,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":48,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":266808,"kb_used_data":149336,"kb_used_omap":8,"kb_used_meta":117431,"kb_avail":188476872,"statfs":{"total":193273528320,"available":193000316928,"internally_reserved":0,"allocated":152920064,"data_stored":152463447,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8876,"internal_metadata":120249684},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[8,9,11,10,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":85278720,"num_objects":20820,"num_object_clones":0,"num_object_copies":41640,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20820,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20820,"num_write_kb":83280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002715"},"pg_stats":[{"pgid":"2.f","version":"36'1399","reported_seq":1451,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T15:58:40.994090+0000","last_change":"2024-09-10T15:58:24.769328+0000","last_active":"2024-09-10T15:58:40.994090+0000","last_peered":"2024-09-10T15:58:40.994090+0000","last_clean":"2024-09-10T15:58:40.994090+0000","last_became_active":"2024-09-10T15:58:24.769050+0000","last_became_peered":"2024-09-10T15:58:24.769050+0000","last_unstale":"2024-09-10T15:58:40.994090+0000","last_undegraded":"2024-09-10T15:58:40.994090+0000","last_fullsized":"2024-09-10T15:58:40.994090+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":1399,"log_dups_size":0,"ondisk_log_size":1399,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:08:36.167566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5730304,"num_objects":1399,"num_object_clones":0,"num_object_copies":2798,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1399,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1399,"num_write_kb":5596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"36'1109","reported_seq":1162,"reported_epoch":36,"state":"active+recovery_unfound+degraded","last_fresh":"2024-09-10T15:58:39.135597+0000","last_change":"2024-09-10T15:58:32.982361+0000","last_active":"2024-09-10T15:58:39.135597+0000","last_peered":"2024-09-10T15:58:39.135597+0000","last_clean":"2024-09-10T15:57:51.316978+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T15:58:39.135597+0000","last_undegraded":"2024-09-10T15:58:25.770880+0000","last_fullsized":"2024-09-10T15:58:39.135597+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":1109,"log_dups_size":0,"ondisk_log_size":1109,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4527046,"num_objects":1107,"num_object_clones":0,"num_object_copies":2214,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":1107,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1109,"num_write_kb":4424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":1107}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"36'1119","reported_seq":1185,"reported_epoch":36,"state":"active+recovery_unfound+degraded","last_fresh":"2024-09-10T15:58:39.127610+0000","last_change":"2024-09-10T15:58:29.145692+0000","last_active":"2024-09-10T15:58:39.127610+0000","last_peered":"2024-09-10T15:58:39.127610+0000","last_clean":"2024-09-10T15:57:51.863524+0000","last_became_active":"2024-09-10T15:58:25.767861+0000","last_became_peered":"2024-09-10T15:58:25.767861+0000","last_unstale":"2024-09-10T15:58:39.127610+0000","last_undegraded":"2024-09-10T15:58:25.769914+0000","last_fullsized":"2024-09-10T15:58:39.127610+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":1119,"log_dups_size":0,"ondisk_log_size":1119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4571619,"num_objects":1117,"num_object_clones":0,"num_object_copies":2234,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":1,"num_objects_dirty":1117,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1119,"num_write_kb":4466,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":1117}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"36'1144","reported_seq":1205,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T15:58:39.141370+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T15:58:39.141370+0000","last_peered":"2024-09-10T15:58:39.141370+0000","last_clean":"2024-09-10T15:58:39.141370+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T15:58:39.141370+0000","last_undegraded":"2024-09-10T15:58:39.141370+0000","last_fullsized":"2024-09-10T15:58:39.141370+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":1144,"log_dups_size":0,"ondisk_log_size":1144,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4685824,"num_objects":1144,"num_object_clones":0,"num_object_copies":2288,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1144,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1144,"num_write_kb":4576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"36'1393","reported_seq":1451,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-10T15:58:41.015399+0000","last_change":"2024-09-10T15:58:24.766952+0000","last_active":"2024-09-10T15:58:41.015399+0000","last_peered":"2024-09-10T15:58:41.015399+0000","last_clean":"2024-09-10T15:57:51.217636+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T15:58:41.015399+0000","last_undegraded":"2024-09-10T15:58:24.759175+0000","last_fullsized":"2024-09-10T15:58:41.015399+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":1393,"log_dups_size":0,"ondisk_log_size":1393,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5686214,"num_objects":1390,"num_object_clones":0,"num_object_copies":2780,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":1390,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1393,"num_write_kb":5556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":1390}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"36'1436","reported_seq":1495,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-10T15:58:40.961501+0000","last_change":"2024-09-10T15:58:24.768439+0000","last_active":"2024-09-10T15:58:40.961501+0000","last_peered":"2024-09-10T15:58:40.961501+0000","last_clean":"2024-09-10T15:57:52.065335+0000","last_became_active":"2024-09-10T15:58:24.768198+0000","last_became_peered":"2024-09-10T15:58:24.768198+0000","last_unstale":"2024-09-10T15:58:40.961501+0000","last_undegraded":"2024-09-10T15:58:24.758149+0000","last_fullsized":"2024-09-10T15:58:40.961501+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":1436,"log_dups_size":0,"ondisk_log_size":1436,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5870534,"num_objects":1435,"num_object_clones":0,"num_object_copies":2870,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":1435,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1436,"num_write_kb":5735,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":1435}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"36'1394","reported_seq":1454,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-10T15:58:41.003813+0000","last_change":"2024-09-10T15:58:24.769216+0000","last_active":"2024-09-10T15:58:41.003813+0000","last_peered":"2024-09-10T15:58:41.003813+0000","last_clean":"2024-09-10T15:57:51.467206+0000","last_became_active":"2024-09-10T15:58:24.768972+0000","last_became_peered":"2024-09-10T15:58:24.768972+0000","last_unstale":"2024-09-10T15:58:41.003813+0000","last_undegraded":"2024-09-10T15:58:24.758006+0000","last_fullsized":"2024-09-10T15:58:41.003813+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":1394,"log_dups_size":0,"ondisk_log_size":1394,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5694406,"num_objects":1392,"num_object_clones":0,"num_object_copies":2784,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":1392,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1394,"num_write_kb":5564,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":1392}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"36'1435","reported_seq":1497,"reported_epoch":36,"state":"active+recovering","last_fresh":"2024-09-10T15:58:41.013549+0000","last_change":"2024-09-10T15:58:25.762598+0000","last_active":"2024-09-10T15:58:41.013549+0000","last_peered":"2024-09-10T15:58:41.013549+0000","last_clean":"2024-09-10T15:57:42.760798+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T15:58:41.013549+0000","last_undegraded":"2024-09-10T15:58:41.013549+0000","last_fullsized":"2024-09-10T15:58:41.013549+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":1435,"log_dups_size":0,"ondisk_log_size":1435,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5874147,"num_objects":1435,"num_object_clones":0,"num_object_copies":2870,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":1,"num_objects_dirty":1435,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1435,"num_write_kb":5737,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":1435}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T15:58:25.762375+0000","last_change":"2024-09-10T15:58:24.767197+0000","last_active":"2024-09-10T15:58:25.762375+0000","last_peered":"2024-09-10T15:58:25.762375+0000","last_clean":"2024-09-10T15:58:25.762375+0000","last_became_active":"2024-09-10T15:58:24.766691+0000","last_became_peered":"2024-09-10T15:58:24.766691+0000","last_unstale":"2024-09-10T15:58:25.762375+0000","last_undegraded":"2024-09-10T15:58:25.762375+0000","last_fullsized":"2024-09-10T15:58:25.762375+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:51:00.161051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"36'1099","reported_seq":1165,"reported_epoch":36,"state":"active+recovery_unfound+degraded","last_fresh":"2024-09-10T15:58:39.148579+0000","last_change":"2024-09-10T15:58:29.145649+0000","last_active":"2024-09-10T15:58:39.148579+0000","last_peered":"2024-09-10T15:58:39.148579+0000","last_clean":"2024-09-10T15:57:51.913440+0000","last_became_active":"2024-09-10T15:58:25.769634+0000","last_became_peered":"2024-09-10T15:58:25.769634+0000","last_unstale":"2024-09-10T15:58:39.148579+0000","last_undegraded":"2024-09-10T15:58:25.771896+0000","last_fullsized":"2024-09-10T15:58:39.148579+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":1099,"log_dups_size":0,"ondisk_log_size":1099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4481990,"num_objects":1096,"num_object_clones":0,"num_object_copies":2192,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":1096,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1099,"num_write_kb":4380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":1096}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":74,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T15:58:25.768681+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T15:58:25.768681+0000","last_peered":"2024-09-10T15:58:25.768681+0000","last_clean":"2024-09-10T15:58:25.768681+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T15:58:25.768681+0000","last_undegraded":"2024-09-10T15:58:25.768681+0000","last_fullsized":"2024-09-10T15:58:25.768681+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"36'1420","reported_seq":1471,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-10T15:58:41.008189+0000","last_change":"2024-09-10T15:58:24.768933+0000","last_active":"2024-09-10T15:58:41.008189+0000","last_peered":"2024-09-10T15:58:41.008189+0000","last_clean":"2024-09-10T15:57:52.118434+0000","last_became_active":"2024-09-10T15:58:24.768693+0000","last_became_peered":"2024-09-10T15:58:24.768693+0000","last_unstale":"2024-09-10T15:58:41.008189+0000","last_undegraded":"2024-09-10T15:58:24.758580+0000","last_fullsized":"2024-09-10T15:58:41.008189+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":1420,"log_dups_size":0,"ondisk_log_size":1420,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5792710,"num_objects":1416,"num_object_clones":0,"num_object_copies":2832,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":1416,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1420,"num_write_kb":5660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":1416}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":67,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-10T15:58:25.762469+0000","last_change":"2024-09-10T15:58:24.768453+0000","last_active":"2024-09-10T15:58:25.762469+0000","last_peered":"2024-09-10T15:58:25.762469+0000","last_clean":"2024-09-10T15:58:03.029025+0000","last_became_active":"2024-09-10T15:58:24.766316+0000","last_became_peered":"2024-09-10T15:58:24.766316+0000","last_unstale":"2024-09-10T15:58:25.762469+0000","last_undegraded":"2024-09-10T15:58:24.757744+0000","last_fullsized":"2024-09-10T15:58:25.762469+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0"],"object_location_counts":[{"shards":"0","objects":1}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"36'1118","reported_seq":1176,"reported_epoch":36,"state":"active+recovery_unfound+degraded","last_fresh":"2024-09-10T15:58:39.144434+0000","last_change":"2024-09-10T15:58:32.966802+0000","last_active":"2024-09-10T15:58:39.144434+0000","last_peered":"2024-09-10T15:58:39.144434+0000","last_clean":"2024-09-10T15:57:51.167715+0000","last_became_active":"2024-09-10T15:58:25.769424+0000","last_became_peered":"2024-09-10T15:58:25.769424+0000","last_unstale":"2024-09-10T15:58:39.144434+0000","last_undegraded":"2024-09-10T15:58:25.771431+0000","last_fullsized":"2024-09-10T15:58:39.144434+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":1118,"log_dups_size":0,"ondisk_log_size":1118,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4568006,"num_objects":1117,"num_object_clones":0,"num_object_copies":2234,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":1117,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1118,"num_write_kb":4463,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":1117}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":92,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T15:58:25.770150+0000","last_change":"2024-09-10T15:58:25.768489+0000","last_active":"2024-09-10T15:58:25.770150+0000","last_peered":"2024-09-10T15:58:25.770150+0000","last_clean":"2024-09-10T15:58:25.770150+0000","last_became_active":"2024-09-10T15:58:25.768117+0000","last_became_peered":"2024-09-10T15:58:25.768117+0000","last_unstale":"2024-09-10T15:58:25.770150+0000","last_undegraded":"2024-09-10T15:58:25.770150+0000","last_fullsized":"2024-09-10T15:58:25.770150+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:56:47.213564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"36'1138","reported_seq":1198,"reported_epoch":36,"state":"active+recovery_unfound+degraded","last_fresh":"2024-09-10T15:58:39.144204+0000","last_change":"2024-09-10T15:58:32.967084+0000","last_active":"2024-09-10T15:58:39.144204+0000","last_peered":"2024-09-10T15:58:39.144204+0000","last_clean":"2024-09-10T15:57:51.566307+0000","last_became_active":"2024-09-10T15:58:25.769762+0000","last_became_peered":"2024-09-10T15:58:25.769762+0000","last_unstale":"2024-09-10T15:58:39.144204+0000","last_undegraded":"2024-09-10T15:58:25.772373+0000","last_fullsized":"2024-09-10T15:58:39.144204+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":1138,"log_dups_size":0,"ondisk_log_size":1138,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4645830,"num_objects":1136,"num_object_clones":0,"num_object_copies":2272,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":1136,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1138,"num_write_kb":4539,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":1136}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"36'1425","reported_seq":1483,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-10T15:58:41.000853+0000","last_change":"2024-09-10T15:58:24.768825+0000","last_active":"2024-09-10T15:58:41.000853+0000","last_peered":"2024-09-10T15:58:41.000853+0000","last_clean":"2024-09-10T15:57:45.763054+0000","last_became_active":"2024-09-10T15:58:24.768609+0000","last_became_peered":"2024-09-10T15:58:24.768609+0000","last_unstale":"2024-09-10T15:58:41.000853+0000","last_undegraded":"2024-09-10T15:58:24.757704+0000","last_fullsized":"2024-09-10T15:58:41.000853+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":1425,"log_dups_size":0,"ondisk_log_size":1425,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5833187,"num_objects":1425,"num_object_clones":0,"num_object_copies":2850,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":1,"num_objects_dirty":1425,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1425,"num_write_kb":5697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":1425}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T15:58:25.762555+0000","last_change":"2024-09-10T15:58:24.767677+0000","last_active":"2024-09-10T15:58:25.762555+0000","last_peered":"2024-09-10T15:58:25.762555+0000","last_clean":"2024-09-10T15:58:25.762555+0000","last_became_active":"2024-09-10T15:58:24.767409+0000","last_became_peered":"2024-09-10T15:58:24.767409+0000","last_unstale":"2024-09-10T15:58:25.762555+0000","last_undegraded":"2024-09-10T15:58:25.762555+0000","last_fullsized":"2024-09-10T15:58:25.762555+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:35:35.763396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"36'1334","reported_seq":1393,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-10T15:58:41.011953+0000","last_change":"2024-09-10T15:58:24.769497+0000","last_active":"2024-09-10T15:58:41.011953+0000","last_peered":"2024-09-10T15:58:41.011953+0000","last_clean":"2024-09-10T15:57:52.013178+0000","last_became_active":"2024-09-10T15:58:24.769268+0000","last_became_peered":"2024-09-10T15:58:24.769268+0000","last_unstale":"2024-09-10T15:58:41.011953+0000","last_undegraded":"2024-09-10T15:58:24.759230+0000","last_fullsized":"2024-09-10T15:58:41.011953+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":1334,"log_dups_size":0,"ondisk_log_size":1334,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5445033,"num_objects":1332,"num_object_clones":0,"num_object_copies":2664,"num_objects_missing_on_primary":3,"num_objects_missing":3,"num_objects_degraded":6,"num_objects_misplaced":0,"num_objects_unfound":3,"num_objects_dirty":1332,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1334,"num_write_kb":5320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":1332}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":74,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T15:58:25.769812+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T15:58:25.769812+0000","last_peered":"2024-09-10T15:58:25.769812+0000","last_clean":"2024-09-10T15:58:25.769812+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T15:58:25.769812+0000","last_undegraded":"2024-09-10T15:58:25.769812+0000","last_fullsized":"2024-09-10T15:58:25.769812+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"36'1478","reported_seq":1536,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-10T15:58:41.011773+0000","last_change":"2024-09-10T15:58:24.769323+0000","last_active":"2024-09-10T15:58:41.011773+0000","last_peered":"2024-09-10T15:58:41.011773+0000","last_clean":"2024-09-10T15:57:45.763338+0000","last_became_active":"2024-09-10T15:58:24.769074+0000","last_became_peered":"2024-09-10T15:58:24.769074+0000","last_unstale":"2024-09-10T15:58:41.011773+0000","last_undegraded":"2024-09-10T15:58:24.759853+0000","last_fullsized":"2024-09-10T15:58:41.011773+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":1478,"log_dups_size":0,"ondisk_log_size":1478,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6046662,"num_objects":1478,"num_object_clones":0,"num_object_copies":2956,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":1478,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1478,"num_write_kb":5906,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":1478}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T15:58:25.762992+0000","last_change":"2024-09-10T15:58:24.767867+0000","last_active":"2024-09-10T15:58:25.762992+0000","last_peered":"2024-09-10T15:58:25.762992+0000","last_clean":"2024-09-10T15:58:25.762992+0000","last_became_active":"2024-09-10T15:58:24.767615+0000","last_became_peered":"2024-09-10T15:58:24.767615+0000","last_unstale":"2024-09-10T15:58:25.762992+0000","last_undegraded":"2024-09-10T15:58:25.762992+0000","last_fullsized":"2024-09-10T15:58:25.762992+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:11:00.908668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":87,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T15:58:25.770511+0000","last_change":"2024-09-10T15:58:25.768847+0000","last_active":"2024-09-10T15:58:25.770511+0000","last_peered":"2024-09-10T15:58:25.770511+0000","last_clean":"2024-09-10T15:58:25.770511+0000","last_became_active":"2024-09-10T15:58:25.768040+0000","last_became_peered":"2024-09-10T15:58:25.768040+0000","last_unstale":"2024-09-10T15:58:25.770511+0000","last_undegraded":"2024-09-10T15:58:25.770511+0000","last_fullsized":"2024-09-10T15:58:25.770511+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:39:30.402776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"36'1434","reported_seq":1493,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-10T15:58:40.997535+0000","last_change":"2024-09-10T15:58:24.769859+0000","last_active":"2024-09-10T15:58:40.997535+0000","last_peered":"2024-09-10T15:58:40.997535+0000","last_clean":"2024-09-10T15:57:51.765160+0000","last_became_active":"2024-09-10T15:58:24.769616+0000","last_became_peered":"2024-09-10T15:58:24.769616+0000","last_unstale":"2024-09-10T15:58:40.997535+0000","last_undegraded":"2024-09-10T15:58:24.756607+0000","last_fullsized":"2024-09-10T15:58:40.997535+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":1434,"log_dups_size":0,"ondisk_log_size":1434,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5838732,"num_objects":1429,"num_object_clones":0,"num_object_copies":2858,"num_objects_missing_on_primary":3,"num_objects_missing":3,"num_objects_degraded":7,"num_objects_misplaced":0,"num_objects_unfound":3,"num_objects_dirty":1429,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1434,"num_write_kb":5707,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0","objects":1},{"shards":"0,2","objects":1428}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":85292244,"num_objects":20848,"num_object_clones":0,"num_object_copies":41696,"num_objects_missing_on_primary":27,"num_objects_missing":27,"num_objects_degraded":53,"num_objects_misplaced":0,"num_objects_unfound":27,"num_objects_dirty":20848,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20875,"num_write_kb":83326,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":165957632,"data_stored":165820338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":20875,"ondisk_log_size":20875,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986183,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":106232,"kb_used_data":60648,"kb_used_omap":4,"kb_used_meta":45563,"kb_avail":94265608,"statfs":{"total":96636764160,"available":96527982592,"internally_reserved":0,"allocated":62103552,"data_stored":61895347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":46657194},"hb_peers":[0],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,6,6,2],"upper_bound":16},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182412,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":160576,"kb_used_data":88688,"kb_used_omap":4,"kb_used_meta":71867,"kb_avail":94211264,"statfs":{"total":96636764160,"available":96472334336,"internally_reserved":0,"allocated":90816512,"data_stored":90568100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":73592490},"hb_peers":[2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[6,3,5,8,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":92852224,"data_stored":92816094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":13524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":72990720,"data_stored":72990720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:58:43.479 INFO:tasks.lost_unfound:no unfound in 2.f 2024-09-10T15:58:43.479 INFO:tasks.lost_unfound:listing missing/lost in 2.e state active+recovery_unfound+degraded 2024-09-10T15:58:43.479 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- pg 2.e list_unfound '{}' 2024-09-10T15:58:43.698 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-10T15:58:43.698 INFO:teuthology.orchestra.run.smithi115.stdout: "offset": { 2024-09-10T15:58:43.698 INFO:teuthology.orchestra.run.smithi115.stdout: "oid": "", 2024-09-10T15:58:43.698 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "", 2024-09-10T15:58:43.698 INFO:teuthology.orchestra.run.smithi115.stdout: "snapid": 0, 2024-09-10T15:58:43.698 INFO:teuthology.orchestra.run.smithi115.stdout: "hash": 0, 2024-09-10T15:58:43.698 INFO:teuthology.orchestra.run.smithi115.stdout: "max": 0, 2024-09-10T15:58:43.698 INFO:teuthology.orchestra.run.smithi115.stdout: "pool": -9223372036854775808, 2024-09-10T15:58:43.698 INFO:teuthology.orchestra.run.smithi115.stdout: "namespace": "" 2024-09-10T15:58:43.699 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-10T15:58:43.699 INFO:teuthology.orchestra.run.smithi115.stdout: "num_missing": 2, 2024-09-10T15:58:43.699 INFO:teuthology.orchestra.run.smithi115.stdout: "num_unfound": 2, 2024-09-10T15:58:43.699 INFO:teuthology.orchestra.run.smithi115.stdout: "objects": [ 2024-09-10T15:58:43.699 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-10T15:58:43.699 INFO:teuthology.orchestra.run.smithi115.stdout: "oid": { 2024-09-10T15:58:43.699 INFO:teuthology.orchestra.run.smithi115.stdout: "oid": "existing_4", 2024-09-10T15:58:43.699 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "", 2024-09-10T15:58:43.699 INFO:teuthology.orchestra.run.smithi115.stdout: "snapid": -2, 2024-09-10T15:58:43.699 INFO:teuthology.orchestra.run.smithi115.stdout: "hash": 1224592654, 2024-09-10T15:58:43.699 INFO:teuthology.orchestra.run.smithi115.stdout: "max": 0, 2024-09-10T15:58:43.699 INFO:teuthology.orchestra.run.smithi115.stdout: "pool": 2, 2024-09-10T15:58:43.699 INFO:teuthology.orchestra.run.smithi115.stdout: "namespace": "" 2024-09-10T15:58:43.699 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-10T15:58:43.700 INFO:teuthology.orchestra.run.smithi115.stdout: "need": "23'3", 2024-09-10T15:58:43.700 INFO:teuthology.orchestra.run.smithi115.stdout: "have": "21'1", 2024-09-10T15:58:43.700 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": "none", 2024-09-10T15:58:43.700 INFO:teuthology.orchestra.run.smithi115.stdout: "clean_regions": "clean_offsets: [(483, 18446744073709551132)], clean_omap: true, new_object: false", 2024-09-10T15:58:43.700 INFO:teuthology.orchestra.run.smithi115.stdout: "locations": [] 2024-09-10T15:58:43.700 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-10T15:58:43.700 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-10T15:58:43.700 INFO:teuthology.orchestra.run.smithi115.stdout: "oid": { 2024-09-10T15:58:43.700 INFO:teuthology.orchestra.run.smithi115.stdout: "oid": "existing_6", 2024-09-10T15:58:43.700 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "", 2024-09-10T15:58:43.700 INFO:teuthology.orchestra.run.smithi115.stdout: "snapid": -2, 2024-09-10T15:58:43.700 INFO:teuthology.orchestra.run.smithi115.stdout: "hash": 3101443614, 2024-09-10T15:58:43.701 INFO:teuthology.orchestra.run.smithi115.stdout: "max": 0, 2024-09-10T15:58:43.701 INFO:teuthology.orchestra.run.smithi115.stdout: "pool": 2, 2024-09-10T15:58:43.701 INFO:teuthology.orchestra.run.smithi115.stdout: "namespace": "" 2024-09-10T15:58:43.701 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-10T15:58:43.701 INFO:teuthology.orchestra.run.smithi115.stdout: "need": "23'4", 2024-09-10T15:58:43.701 INFO:teuthology.orchestra.run.smithi115.stdout: "have": "21'2", 2024-09-10T15:58:43.701 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": "none", 2024-09-10T15:58:43.701 INFO:teuthology.orchestra.run.smithi115.stdout: "clean_regions": "clean_offsets: [(483, 18446744073709551132)], clean_omap: true, new_object: false", 2024-09-10T15:58:43.701 INFO:teuthology.orchestra.run.smithi115.stdout: "locations": [] 2024-09-10T15:58:43.701 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-10T15:58:43.701 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-09-10T15:58:43.701 INFO:teuthology.orchestra.run.smithi115.stdout: "state": "NotRecovering", 2024-09-10T15:58:43.701 INFO:teuthology.orchestra.run.smithi115.stdout: "available_might_have_unfound": true, 2024-09-10T15:58:43.702 INFO:teuthology.orchestra.run.smithi115.stdout: "might_have_unfound": [ 2024-09-10T15:58:43.702 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-10T15:58:43.702 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "1", 2024-09-10T15:58:43.702 INFO:teuthology.orchestra.run.smithi115.stdout: "status": "osd is down" 2024-09-10T15:58:43.702 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-10T15:58:43.702 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-09-10T15:58:43.702 INFO:teuthology.orchestra.run.smithi115.stdout: "more": false 2024-09-10T15:58:43.702 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-10T15:58:43.723 INFO:tasks.lost_unfound:reverting unfound in 2.e on osd.2 2024-09-10T15:58:43.724 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg 2.e mark_unfound_lost revert 2024-09-10T15:58:43.954 INFO:teuthology.orchestra.run.smithi115.stderr:pg has 2 objects unfound and apparently lost marking 2024-09-10T15:58:43.961 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 11 20 29257 29237 10.3815 7.65625 0.0121243 0.00752055 2024-09-10T15:58:43.977 INFO:tasks.lost_unfound:listing missing/lost in 2.d state active+recovery_unfound+degraded 2024-09-10T15:58:43.977 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- pg 2.d list_unfound '{}' 2024-09-10T15:58:44.192 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-10T15:58:44.192 INFO:teuthology.orchestra.run.smithi115.stdout: "offset": { 2024-09-10T15:58:44.192 INFO:teuthology.orchestra.run.smithi115.stdout: "oid": "", 2024-09-10T15:58:44.192 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "", 2024-09-10T15:58:44.192 INFO:teuthology.orchestra.run.smithi115.stdout: "snapid": 0, 2024-09-10T15:58:44.192 INFO:teuthology.orchestra.run.smithi115.stdout: "hash": 0, 2024-09-10T15:58:44.192 INFO:teuthology.orchestra.run.smithi115.stdout: "max": 0, 2024-09-10T15:58:44.192 INFO:teuthology.orchestra.run.smithi115.stdout: "pool": -9223372036854775808, 2024-09-10T15:58:44.192 INFO:teuthology.orchestra.run.smithi115.stdout: "namespace": "" 2024-09-10T15:58:44.193 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-10T15:58:44.193 INFO:teuthology.orchestra.run.smithi115.stdout: "num_missing": 1, 2024-09-10T15:58:44.193 INFO:teuthology.orchestra.run.smithi115.stdout: "num_unfound": 1, 2024-09-10T15:58:44.193 INFO:teuthology.orchestra.run.smithi115.stdout: "objects": [ 2024-09-10T15:58:44.193 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-10T15:58:44.193 INFO:teuthology.orchestra.run.smithi115.stdout: "oid": { 2024-09-10T15:58:44.193 INFO:teuthology.orchestra.run.smithi115.stdout: "oid": "existed_7", 2024-09-10T15:58:44.193 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "", 2024-09-10T15:58:44.193 INFO:teuthology.orchestra.run.smithi115.stdout: "snapid": -2, 2024-09-10T15:58:44.193 INFO:teuthology.orchestra.run.smithi115.stdout: "hash": 2358579165, 2024-09-10T15:58:44.193 INFO:teuthology.orchestra.run.smithi115.stdout: "max": 0, 2024-09-10T15:58:44.193 INFO:teuthology.orchestra.run.smithi115.stdout: "pool": 2, 2024-09-10T15:58:44.193 INFO:teuthology.orchestra.run.smithi115.stdout: "namespace": "" 2024-09-10T15:58:44.194 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-10T15:58:44.194 INFO:teuthology.orchestra.run.smithi115.stdout: "need": "23'3", 2024-09-10T15:58:44.194 INFO:teuthology.orchestra.run.smithi115.stdout: "have": "0'0", 2024-09-10T15:58:44.194 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": "none", 2024-09-10T15:58:44.194 INFO:teuthology.orchestra.run.smithi115.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2024-09-10T15:58:44.194 INFO:teuthology.orchestra.run.smithi115.stdout: "locations": [] 2024-09-10T15:58:44.194 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-10T15:58:44.194 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-09-10T15:58:44.194 INFO:teuthology.orchestra.run.smithi115.stdout: "state": "NotRecovering", 2024-09-10T15:58:44.194 INFO:teuthology.orchestra.run.smithi115.stdout: "available_might_have_unfound": true, 2024-09-10T15:58:44.194 INFO:teuthology.orchestra.run.smithi115.stdout: "might_have_unfound": [ 2024-09-10T15:58:44.194 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-10T15:58:44.194 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "1", 2024-09-10T15:58:44.195 INFO:teuthology.orchestra.run.smithi115.stdout: "status": "osd is down" 2024-09-10T15:58:44.195 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-10T15:58:44.195 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-09-10T15:58:44.195 INFO:teuthology.orchestra.run.smithi115.stdout: "more": false 2024-09-10T15:58:44.195 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-10T15:58:44.214 INFO:tasks.lost_unfound:reverting unfound in 2.d on osd.2 2024-09-10T15:58:44.214 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg 2.d mark_unfound_lost revert 2024-09-10T15:58:44.441 INFO:teuthology.orchestra.run.smithi115.stderr:pg has 1 objects unfound and apparently lost marking 2024-09-10T15:58:44.467 INFO:tasks.lost_unfound:no unfound in 2.c 2024-09-10T15:58:44.468 INFO:tasks.lost_unfound:listing missing/lost in 2.b state active+recovery_wait+degraded 2024-09-10T15:58:44.468 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- pg 2.b list_unfound '{}' 2024-09-10T15:58:44.681 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-10T15:58:44.682 INFO:teuthology.orchestra.run.smithi115.stdout: "offset": { 2024-09-10T15:58:44.682 INFO:teuthology.orchestra.run.smithi115.stdout: "oid": "", 2024-09-10T15:58:44.682 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "", 2024-09-10T15:58:44.682 INFO:teuthology.orchestra.run.smithi115.stdout: "snapid": 0, 2024-09-10T15:58:44.682 INFO:teuthology.orchestra.run.smithi115.stdout: "hash": 0, 2024-09-10T15:58:44.682 INFO:teuthology.orchestra.run.smithi115.stdout: "max": 0, 2024-09-10T15:58:44.682 INFO:teuthology.orchestra.run.smithi115.stdout: "pool": -9223372036854775808, 2024-09-10T15:58:44.682 INFO:teuthology.orchestra.run.smithi115.stdout: "namespace": "" 2024-09-10T15:58:44.682 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-10T15:58:44.682 INFO:teuthology.orchestra.run.smithi115.stdout: "num_missing": 2, 2024-09-10T15:58:44.682 INFO:teuthology.orchestra.run.smithi115.stdout: "num_unfound": 2, 2024-09-10T15:58:44.683 INFO:teuthology.orchestra.run.smithi115.stdout: "objects": [ 2024-09-10T15:58:44.683 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-10T15:58:44.683 INFO:teuthology.orchestra.run.smithi115.stdout: "oid": { 2024-09-10T15:58:44.683 INFO:teuthology.orchestra.run.smithi115.stdout: "oid": "existed_3", 2024-09-10T15:58:44.683 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "", 2024-09-10T15:58:44.683 INFO:teuthology.orchestra.run.smithi115.stdout: "snapid": -2, 2024-09-10T15:58:44.683 INFO:teuthology.orchestra.run.smithi115.stdout: "hash": 943960171, 2024-09-10T15:58:44.683 INFO:teuthology.orchestra.run.smithi115.stdout: "max": 0, 2024-09-10T15:58:44.683 INFO:teuthology.orchestra.run.smithi115.stdout: "pool": 2, 2024-09-10T15:58:44.683 INFO:teuthology.orchestra.run.smithi115.stdout: "namespace": "" 2024-09-10T15:58:44.683 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-10T15:58:44.683 INFO:teuthology.orchestra.run.smithi115.stdout: "need": "23'5", 2024-09-10T15:58:44.683 INFO:teuthology.orchestra.run.smithi115.stdout: "have": "0'0", 2024-09-10T15:58:44.684 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": "none", 2024-09-10T15:58:44.684 INFO:teuthology.orchestra.run.smithi115.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2024-09-10T15:58:44.684 INFO:teuthology.orchestra.run.smithi115.stdout: "locations": [] 2024-09-10T15:58:44.684 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-10T15:58:44.684 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-10T15:58:44.684 INFO:teuthology.orchestra.run.smithi115.stdout: "oid": { 2024-09-10T15:58:44.684 INFO:teuthology.orchestra.run.smithi115.stdout: "oid": "existing_1", 2024-09-10T15:58:44.684 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "", 2024-09-10T15:58:44.684 INFO:teuthology.orchestra.run.smithi115.stdout: "snapid": -2, 2024-09-10T15:58:44.684 INFO:teuthology.orchestra.run.smithi115.stdout: "hash": 2885884443, 2024-09-10T15:58:44.684 INFO:teuthology.orchestra.run.smithi115.stdout: "max": 0, 2024-09-10T15:58:44.684 INFO:teuthology.orchestra.run.smithi115.stdout: "pool": 2, 2024-09-10T15:58:44.684 INFO:teuthology.orchestra.run.smithi115.stdout: "namespace": "" 2024-09-10T15:58:44.684 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-10T15:58:44.685 INFO:teuthology.orchestra.run.smithi115.stdout: "need": "23'4", 2024-09-10T15:58:44.685 INFO:teuthology.orchestra.run.smithi115.stdout: "have": "21'1", 2024-09-10T15:58:44.685 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": "none", 2024-09-10T15:58:44.685 INFO:teuthology.orchestra.run.smithi115.stdout: "clean_regions": "clean_offsets: [(483, 18446744073709551132)], clean_omap: true, new_object: false", 2024-09-10T15:58:44.685 INFO:teuthology.orchestra.run.smithi115.stdout: "locations": [] 2024-09-10T15:58:44.685 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-10T15:58:44.685 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-09-10T15:58:44.685 INFO:teuthology.orchestra.run.smithi115.stdout: "state": "Active", 2024-09-10T15:58:44.685 INFO:teuthology.orchestra.run.smithi115.stdout: "available_might_have_unfound": true, 2024-09-10T15:58:44.685 INFO:teuthology.orchestra.run.smithi115.stdout: "might_have_unfound": [ 2024-09-10T15:58:44.685 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-10T15:58:44.685 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "1", 2024-09-10T15:58:44.685 INFO:teuthology.orchestra.run.smithi115.stdout: "status": "osd is down" 2024-09-10T15:58:44.685 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-10T15:58:44.686 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-09-10T15:58:44.686 INFO:teuthology.orchestra.run.smithi115.stdout: "more": false 2024-09-10T15:58:44.686 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-10T15:58:44.699 INFO:tasks.lost_unfound:reverting unfound in 2.b on osd.0 2024-09-10T15:58:44.699 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg 2.b mark_unfound_lost revert 2024-09-10T15:58:44.926 INFO:teuthology.orchestra.run.smithi115.stderr:pg has 2 objects unfound and apparently lost marking 2024-09-10T15:58:44.948 INFO:tasks.lost_unfound:listing missing/lost in 2.a state active+recovery_wait+degraded 2024-09-10T15:58:44.948 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- pg 2.a list_unfound '{}' 2024-09-10T15:58:44.962 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 12 20 30947 30927 10.066 6.60156 0.00966883 0.00775619 2024-09-10T15:58:45.164 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-10T15:58:45.164 INFO:teuthology.orchestra.run.smithi115.stdout: "offset": { 2024-09-10T15:58:45.164 INFO:teuthology.orchestra.run.smithi115.stdout: "oid": "", 2024-09-10T15:58:45.164 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "", 2024-09-10T15:58:45.164 INFO:teuthology.orchestra.run.smithi115.stdout: "snapid": 0, 2024-09-10T15:58:45.164 INFO:teuthology.orchestra.run.smithi115.stdout: "hash": 0, 2024-09-10T15:58:45.165 INFO:teuthology.orchestra.run.smithi115.stdout: "max": 0, 2024-09-10T15:58:45.165 INFO:teuthology.orchestra.run.smithi115.stdout: "pool": -9223372036854775808, 2024-09-10T15:58:45.165 INFO:teuthology.orchestra.run.smithi115.stdout: "namespace": "" 2024-09-10T15:58:45.165 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-10T15:58:45.165 INFO:teuthology.orchestra.run.smithi115.stdout: "num_missing": 2, 2024-09-10T15:58:45.165 INFO:teuthology.orchestra.run.smithi115.stdout: "num_unfound": 2, 2024-09-10T15:58:45.165 INFO:teuthology.orchestra.run.smithi115.stdout: "objects": [ 2024-09-10T15:58:45.165 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-10T15:58:45.165 INFO:teuthology.orchestra.run.smithi115.stdout: "oid": { 2024-09-10T15:58:45.165 INFO:teuthology.orchestra.run.smithi115.stdout: "oid": "existing_9", 2024-09-10T15:58:45.165 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "", 2024-09-10T15:58:45.165 INFO:teuthology.orchestra.run.smithi115.stdout: "snapid": -2, 2024-09-10T15:58:45.165 INFO:teuthology.orchestra.run.smithi115.stdout: "hash": 1217469130, 2024-09-10T15:58:45.166 INFO:teuthology.orchestra.run.smithi115.stdout: "max": 0, 2024-09-10T15:58:45.166 INFO:teuthology.orchestra.run.smithi115.stdout: "pool": 2, 2024-09-10T15:58:45.166 INFO:teuthology.orchestra.run.smithi115.stdout: "namespace": "" 2024-09-10T15:58:45.166 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-10T15:58:45.166 INFO:teuthology.orchestra.run.smithi115.stdout: "need": "23'3", 2024-09-10T15:58:45.166 INFO:teuthology.orchestra.run.smithi115.stdout: "have": "21'1", 2024-09-10T15:58:45.166 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": "none", 2024-09-10T15:58:45.166 INFO:teuthology.orchestra.run.smithi115.stdout: "clean_regions": "clean_offsets: [(483, 18446744073709551132)], clean_omap: true, new_object: false", 2024-09-10T15:58:45.166 INFO:teuthology.orchestra.run.smithi115.stdout: "locations": [] 2024-09-10T15:58:45.166 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-10T15:58:45.166 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-10T15:58:45.166 INFO:teuthology.orchestra.run.smithi115.stdout: "oid": { 2024-09-10T15:58:45.166 INFO:teuthology.orchestra.run.smithi115.stdout: "oid": "new_9", 2024-09-10T15:58:45.167 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "", 2024-09-10T15:58:45.167 INFO:teuthology.orchestra.run.smithi115.stdout: "snapid": -2, 2024-09-10T15:58:45.167 INFO:teuthology.orchestra.run.smithi115.stdout: "hash": 170713514, 2024-09-10T15:58:45.167 INFO:teuthology.orchestra.run.smithi115.stdout: "max": 0, 2024-09-10T15:58:45.167 INFO:teuthology.orchestra.run.smithi115.stdout: "pool": 2, 2024-09-10T15:58:45.167 INFO:teuthology.orchestra.run.smithi115.stdout: "namespace": "" 2024-09-10T15:58:45.167 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-10T15:58:45.167 INFO:teuthology.orchestra.run.smithi115.stdout: "need": "23'2", 2024-09-10T15:58:45.167 INFO:teuthology.orchestra.run.smithi115.stdout: "have": "0'0", 2024-09-10T15:58:45.167 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": "none", 2024-09-10T15:58:45.167 INFO:teuthology.orchestra.run.smithi115.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2024-09-10T15:58:45.167 INFO:teuthology.orchestra.run.smithi115.stdout: "locations": [] 2024-09-10T15:58:45.167 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-10T15:58:45.168 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-09-10T15:58:45.168 INFO:teuthology.orchestra.run.smithi115.stdout: "state": "Active", 2024-09-10T15:58:45.168 INFO:teuthology.orchestra.run.smithi115.stdout: "available_might_have_unfound": true, 2024-09-10T15:58:45.168 INFO:teuthology.orchestra.run.smithi115.stdout: "might_have_unfound": [ 2024-09-10T15:58:45.168 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-10T15:58:45.168 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "1", 2024-09-10T15:58:45.168 INFO:teuthology.orchestra.run.smithi115.stdout: "status": "osd is down" 2024-09-10T15:58:45.168 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-10T15:58:45.168 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-09-10T15:58:45.168 INFO:teuthology.orchestra.run.smithi115.stdout: "more": false 2024-09-10T15:58:45.168 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-10T15:58:45.189 INFO:tasks.lost_unfound:reverting unfound in 2.a on osd.0 2024-09-10T15:58:45.190 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg 2.a mark_unfound_lost revert 2024-09-10T15:58:45.454 INFO:teuthology.orchestra.run.smithi115.stderr:pg has 2 objects unfound and apparently lost marking 2024-09-10T15:58:45.475 INFO:tasks.lost_unfound:listing missing/lost in 2.9 state active+recovery_wait+degraded 2024-09-10T15:58:45.476 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- pg 2.9 list_unfound '{}' 2024-09-10T15:58:45.700 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-10T15:58:45.700 INFO:teuthology.orchestra.run.smithi115.stdout: "offset": { 2024-09-10T15:58:45.700 INFO:teuthology.orchestra.run.smithi115.stdout: "oid": "", 2024-09-10T15:58:45.700 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "", 2024-09-10T15:58:45.700 INFO:teuthology.orchestra.run.smithi115.stdout: "snapid": 0, 2024-09-10T15:58:45.700 INFO:teuthology.orchestra.run.smithi115.stdout: "hash": 0, 2024-09-10T15:58:45.700 INFO:teuthology.orchestra.run.smithi115.stdout: "max": 0, 2024-09-10T15:58:45.700 INFO:teuthology.orchestra.run.smithi115.stdout: "pool": -9223372036854775808, 2024-09-10T15:58:45.700 INFO:teuthology.orchestra.run.smithi115.stdout: "namespace": "" 2024-09-10T15:58:45.700 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-10T15:58:45.700 INFO:teuthology.orchestra.run.smithi115.stdout: "num_missing": 2, 2024-09-10T15:58:45.701 INFO:teuthology.orchestra.run.smithi115.stdout: "num_unfound": 2, 2024-09-10T15:58:45.701 INFO:teuthology.orchestra.run.smithi115.stdout: "objects": [ 2024-09-10T15:58:45.701 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-10T15:58:45.701 INFO:teuthology.orchestra.run.smithi115.stdout: "oid": { 2024-09-10T15:58:45.701 INFO:teuthology.orchestra.run.smithi115.stdout: "oid": "existing_5", 2024-09-10T15:58:45.701 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "", 2024-09-10T15:58:45.701 INFO:teuthology.orchestra.run.smithi115.stdout: "snapid": -2, 2024-09-10T15:58:45.701 INFO:teuthology.orchestra.run.smithi115.stdout: "hash": 2998549577, 2024-09-10T15:58:45.701 INFO:teuthology.orchestra.run.smithi115.stdout: "max": 0, 2024-09-10T15:58:45.701 INFO:teuthology.orchestra.run.smithi115.stdout: "pool": 2, 2024-09-10T15:58:45.701 INFO:teuthology.orchestra.run.smithi115.stdout: "namespace": "" 2024-09-10T15:58:45.701 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-10T15:58:45.701 INFO:teuthology.orchestra.run.smithi115.stdout: "need": "23'4", 2024-09-10T15:58:45.701 INFO:teuthology.orchestra.run.smithi115.stdout: "have": "21'2", 2024-09-10T15:58:45.702 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": "none", 2024-09-10T15:58:45.702 INFO:teuthology.orchestra.run.smithi115.stdout: "clean_regions": "clean_offsets: [(483, 18446744073709551132)], clean_omap: true, new_object: false", 2024-09-10T15:58:45.702 INFO:teuthology.orchestra.run.smithi115.stdout: "locations": [] 2024-09-10T15:58:45.702 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-10T15:58:45.702 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-10T15:58:45.702 INFO:teuthology.orchestra.run.smithi115.stdout: "oid": { 2024-09-10T15:58:45.702 INFO:teuthology.orchestra.run.smithi115.stdout: "oid": "existing_2", 2024-09-10T15:58:45.702 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "", 2024-09-10T15:58:45.702 INFO:teuthology.orchestra.run.smithi115.stdout: "snapid": -2, 2024-09-10T15:58:45.702 INFO:teuthology.orchestra.run.smithi115.stdout: "hash": 4032093161, 2024-09-10T15:58:45.702 INFO:teuthology.orchestra.run.smithi115.stdout: "max": 0, 2024-09-10T15:58:45.702 INFO:teuthology.orchestra.run.smithi115.stdout: "pool": 2, 2024-09-10T15:58:45.702 INFO:teuthology.orchestra.run.smithi115.stdout: "namespace": "" 2024-09-10T15:58:45.702 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-10T15:58:45.703 INFO:teuthology.orchestra.run.smithi115.stdout: "need": "23'3", 2024-09-10T15:58:45.703 INFO:teuthology.orchestra.run.smithi115.stdout: "have": "21'1", 2024-09-10T15:58:45.703 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": "none", 2024-09-10T15:58:45.703 INFO:teuthology.orchestra.run.smithi115.stdout: "clean_regions": "clean_offsets: [(483, 18446744073709551132)], clean_omap: true, new_object: false", 2024-09-10T15:58:45.703 INFO:teuthology.orchestra.run.smithi115.stdout: "locations": [] 2024-09-10T15:58:45.703 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-10T15:58:45.703 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-09-10T15:58:45.703 INFO:teuthology.orchestra.run.smithi115.stdout: "state": "Active", 2024-09-10T15:58:45.703 INFO:teuthology.orchestra.run.smithi115.stdout: "available_might_have_unfound": true, 2024-09-10T15:58:45.703 INFO:teuthology.orchestra.run.smithi115.stdout: "might_have_unfound": [ 2024-09-10T15:58:45.703 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-10T15:58:45.703 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "1", 2024-09-10T15:58:45.703 INFO:teuthology.orchestra.run.smithi115.stdout: "status": "osd is down" 2024-09-10T15:58:45.704 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-10T15:58:45.704 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-09-10T15:58:45.704 INFO:teuthology.orchestra.run.smithi115.stdout: "more": false 2024-09-10T15:58:45.704 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-10T15:58:45.717 INFO:tasks.lost_unfound:reverting unfound in 2.9 on osd.0 2024-09-10T15:58:45.717 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg 2.9 mark_unfound_lost revert 2024-09-10T15:58:45.872 INFO:tasks.ceph.mon.a.smithi115.stderr:2024-09-10T15:58:45.870+0000 7fa2093e0640 -1 log_channel(cluster) log [ERR] : Health check update: Possible data damage: 4 pgs recovery_unfound (PG_DAMAGED) 2024-09-10T15:58:45.960 INFO:teuthology.orchestra.run.smithi115.stderr:pg has 2 objects unfound and apparently lost marking 2024-09-10T15:58:45.962 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 13 20 32645 32625 9.80188 6.63281 0.00716251 0.00796369 2024-09-10T15:58:45.985 INFO:tasks.lost_unfound:listing missing/lost in 2.8 state active+recovering 2024-09-10T15:58:45.986 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- pg 2.8 list_unfound '{}' 2024-09-10T15:58:46.206 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-10T15:58:46.206 INFO:teuthology.orchestra.run.smithi115.stdout: "offset": { 2024-09-10T15:58:46.206 INFO:teuthology.orchestra.run.smithi115.stdout: "oid": "", 2024-09-10T15:58:46.207 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "", 2024-09-10T15:58:46.207 INFO:teuthology.orchestra.run.smithi115.stdout: "snapid": 0, 2024-09-10T15:58:46.207 INFO:teuthology.orchestra.run.smithi115.stdout: "hash": 0, 2024-09-10T15:58:46.207 INFO:teuthology.orchestra.run.smithi115.stdout: "max": 0, 2024-09-10T15:58:46.207 INFO:teuthology.orchestra.run.smithi115.stdout: "pool": -9223372036854775808, 2024-09-10T15:58:46.207 INFO:teuthology.orchestra.run.smithi115.stdout: "namespace": "" 2024-09-10T15:58:46.207 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-10T15:58:46.207 INFO:teuthology.orchestra.run.smithi115.stdout: "num_missing": 1, 2024-09-10T15:58:46.207 INFO:teuthology.orchestra.run.smithi115.stdout: "num_unfound": 1, 2024-09-10T15:58:46.207 INFO:teuthology.orchestra.run.smithi115.stdout: "objects": [ 2024-09-10T15:58:46.207 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-10T15:58:46.207 INFO:teuthology.orchestra.run.smithi115.stdout: "oid": { 2024-09-10T15:58:46.208 INFO:teuthology.orchestra.run.smithi115.stdout: "oid": "new_8", 2024-09-10T15:58:46.208 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "", 2024-09-10T15:58:46.208 INFO:teuthology.orchestra.run.smithi115.stdout: "snapid": -2, 2024-09-10T15:58:46.208 INFO:teuthology.orchestra.run.smithi115.stdout: "hash": 2730327960, 2024-09-10T15:58:46.208 INFO:teuthology.orchestra.run.smithi115.stdout: "max": 0, 2024-09-10T15:58:46.208 INFO:teuthology.orchestra.run.smithi115.stdout: "pool": 2, 2024-09-10T15:58:46.208 INFO:teuthology.orchestra.run.smithi115.stdout: "namespace": "" 2024-09-10T15:58:46.208 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-10T15:58:46.208 INFO:teuthology.orchestra.run.smithi115.stdout: "need": "23'1", 2024-09-10T15:58:46.208 INFO:teuthology.orchestra.run.smithi115.stdout: "have": "0'0", 2024-09-10T15:58:46.208 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": "none", 2024-09-10T15:58:46.208 INFO:teuthology.orchestra.run.smithi115.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2024-09-10T15:58:46.208 INFO:teuthology.orchestra.run.smithi115.stdout: "locations": [] 2024-09-10T15:58:46.208 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-10T15:58:46.209 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-09-10T15:58:46.209 INFO:teuthology.orchestra.run.smithi115.stdout: "state": "Active", 2024-09-10T15:58:46.209 INFO:teuthology.orchestra.run.smithi115.stdout: "available_might_have_unfound": true, 2024-09-10T15:58:46.209 INFO:teuthology.orchestra.run.smithi115.stdout: "might_have_unfound": [ 2024-09-10T15:58:46.209 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-10T15:58:46.209 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "1", 2024-09-10T15:58:46.209 INFO:teuthology.orchestra.run.smithi115.stdout: "status": "osd is down" 2024-09-10T15:58:46.209 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-10T15:58:46.209 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-09-10T15:58:46.209 INFO:teuthology.orchestra.run.smithi115.stdout: "more": false 2024-09-10T15:58:46.209 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-10T15:58:46.228 INFO:tasks.lost_unfound:reverting unfound in 2.8 on osd.0 2024-09-10T15:58:46.229 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg 2.8 mark_unfound_lost revert 2024-09-10T15:58:46.469 INFO:teuthology.orchestra.run.smithi115.stderr:pg has 1 objects unfound and apparently lost marking 2024-09-10T15:58:46.491 INFO:tasks.lost_unfound:no unfound in 1.0 2024-09-10T15:58:46.491 INFO:tasks.lost_unfound:listing missing/lost in 2.3 state active+recovery_unfound+degraded 2024-09-10T15:58:46.491 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- pg 2.3 list_unfound '{}' 2024-09-10T15:58:46.706 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-10T15:58:46.706 INFO:teuthology.orchestra.run.smithi115.stdout: "offset": { 2024-09-10T15:58:46.706 INFO:teuthology.orchestra.run.smithi115.stdout: "oid": "", 2024-09-10T15:58:46.706 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "", 2024-09-10T15:58:46.706 INFO:teuthology.orchestra.run.smithi115.stdout: "snapid": 0, 2024-09-10T15:58:46.707 INFO:teuthology.orchestra.run.smithi115.stdout: "hash": 0, 2024-09-10T15:58:46.707 INFO:teuthology.orchestra.run.smithi115.stdout: "max": 0, 2024-09-10T15:58:46.707 INFO:teuthology.orchestra.run.smithi115.stdout: "pool": -9223372036854775808, 2024-09-10T15:58:46.707 INFO:teuthology.orchestra.run.smithi115.stdout: "namespace": "" 2024-09-10T15:58:46.707 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-10T15:58:46.707 INFO:teuthology.orchestra.run.smithi115.stdout: "num_missing": 2, 2024-09-10T15:58:46.707 INFO:teuthology.orchestra.run.smithi115.stdout: "num_unfound": 2, 2024-09-10T15:58:46.707 INFO:teuthology.orchestra.run.smithi115.stdout: "objects": [ 2024-09-10T15:58:46.707 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-10T15:58:46.707 INFO:teuthology.orchestra.run.smithi115.stdout: "oid": { 2024-09-10T15:58:46.707 INFO:teuthology.orchestra.run.smithi115.stdout: "oid": "existing_8", 2024-09-10T15:58:46.707 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "", 2024-09-10T15:58:46.707 INFO:teuthology.orchestra.run.smithi115.stdout: "snapid": -2, 2024-09-10T15:58:46.708 INFO:teuthology.orchestra.run.smithi115.stdout: "hash": 1901787523, 2024-09-10T15:58:46.708 INFO:teuthology.orchestra.run.smithi115.stdout: "max": 0, 2024-09-10T15:58:46.708 INFO:teuthology.orchestra.run.smithi115.stdout: "pool": 2, 2024-09-10T15:58:46.708 INFO:teuthology.orchestra.run.smithi115.stdout: "namespace": "" 2024-09-10T15:58:46.708 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-10T15:58:46.708 INFO:teuthology.orchestra.run.smithi115.stdout: "need": "23'5", 2024-09-10T15:58:46.708 INFO:teuthology.orchestra.run.smithi115.stdout: "have": "21'3", 2024-09-10T15:58:46.708 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": "none", 2024-09-10T15:58:46.708 INFO:teuthology.orchestra.run.smithi115.stdout: "clean_regions": "clean_offsets: [(483, 18446744073709551132)], clean_omap: true, new_object: false", 2024-09-10T15:58:46.708 INFO:teuthology.orchestra.run.smithi115.stdout: "locations": [] 2024-09-10T15:58:46.708 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-10T15:58:46.708 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-10T15:58:46.708 INFO:teuthology.orchestra.run.smithi115.stdout: "oid": { 2024-09-10T15:58:46.709 INFO:teuthology.orchestra.run.smithi115.stdout: "oid": "existed_1", 2024-09-10T15:58:46.709 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "", 2024-09-10T15:58:46.709 INFO:teuthology.orchestra.run.smithi115.stdout: "snapid": -2, 2024-09-10T15:58:46.709 INFO:teuthology.orchestra.run.smithi115.stdout: "hash": 3122379363, 2024-09-10T15:58:46.709 INFO:teuthology.orchestra.run.smithi115.stdout: "max": 0, 2024-09-10T15:58:46.709 INFO:teuthology.orchestra.run.smithi115.stdout: "pool": 2, 2024-09-10T15:58:46.709 INFO:teuthology.orchestra.run.smithi115.stdout: "namespace": "" 2024-09-10T15:58:46.709 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-10T15:58:46.709 INFO:teuthology.orchestra.run.smithi115.stdout: "need": "23'4", 2024-09-10T15:58:46.709 INFO:teuthology.orchestra.run.smithi115.stdout: "have": "0'0", 2024-09-10T15:58:46.709 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": "none", 2024-09-10T15:58:46.709 INFO:teuthology.orchestra.run.smithi115.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2024-09-10T15:58:46.709 INFO:teuthology.orchestra.run.smithi115.stdout: "locations": [] 2024-09-10T15:58:46.709 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-10T15:58:46.710 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-09-10T15:58:46.710 INFO:teuthology.orchestra.run.smithi115.stdout: "state": "NotRecovering", 2024-09-10T15:58:46.710 INFO:teuthology.orchestra.run.smithi115.stdout: "available_might_have_unfound": true, 2024-09-10T15:58:46.710 INFO:teuthology.orchestra.run.smithi115.stdout: "might_have_unfound": [ 2024-09-10T15:58:46.710 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-10T15:58:46.710 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "1", 2024-09-10T15:58:46.710 INFO:teuthology.orchestra.run.smithi115.stdout: "status": "osd is down" 2024-09-10T15:58:46.710 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-10T15:58:46.710 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-09-10T15:58:46.710 INFO:teuthology.orchestra.run.smithi115.stdout: "more": false 2024-09-10T15:58:46.710 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-10T15:58:46.726 INFO:tasks.lost_unfound:reverting unfound in 2.3 on osd.2 2024-09-10T15:58:46.726 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg 2.3 mark_unfound_lost revert 2024-09-10T15:58:46.957 INFO:teuthology.orchestra.run.smithi115.stderr:pg has 2 objects unfound and apparently lost marking 2024-09-10T15:58:46.962 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 14 20 34364 34344 9.58135 6.71484 0.0099063 0.00814752 2024-09-10T15:58:46.976 INFO:tasks.lost_unfound:no unfound in 1.1 2024-09-10T15:58:46.976 INFO:tasks.lost_unfound:listing missing/lost in 2.2 state active+recovery_wait+degraded 2024-09-10T15:58:46.976 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- pg 2.2 list_unfound '{}' 2024-09-10T15:58:47.193 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-10T15:58:47.193 INFO:teuthology.orchestra.run.smithi115.stdout: "offset": { 2024-09-10T15:58:47.193 INFO:teuthology.orchestra.run.smithi115.stdout: "oid": "", 2024-09-10T15:58:47.193 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "", 2024-09-10T15:58:47.193 INFO:teuthology.orchestra.run.smithi115.stdout: "snapid": 0, 2024-09-10T15:58:47.193 INFO:teuthology.orchestra.run.smithi115.stdout: "hash": 0, 2024-09-10T15:58:47.193 INFO:teuthology.orchestra.run.smithi115.stdout: "max": 0, 2024-09-10T15:58:47.193 INFO:teuthology.orchestra.run.smithi115.stdout: "pool": -9223372036854775808, 2024-09-10T15:58:47.193 INFO:teuthology.orchestra.run.smithi115.stdout: "namespace": "" 2024-09-10T15:58:47.194 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-10T15:58:47.194 INFO:teuthology.orchestra.run.smithi115.stdout: "num_missing": 2, 2024-09-10T15:58:47.194 INFO:teuthology.orchestra.run.smithi115.stdout: "num_unfound": 2, 2024-09-10T15:58:47.194 INFO:teuthology.orchestra.run.smithi115.stdout: "objects": [ 2024-09-10T15:58:47.194 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-10T15:58:47.194 INFO:teuthology.orchestra.run.smithi115.stdout: "oid": { 2024-09-10T15:58:47.194 INFO:teuthology.orchestra.run.smithi115.stdout: "oid": "existed_6", 2024-09-10T15:58:47.194 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "", 2024-09-10T15:58:47.194 INFO:teuthology.orchestra.run.smithi115.stdout: "snapid": -2, 2024-09-10T15:58:47.194 INFO:teuthology.orchestra.run.smithi115.stdout: "hash": 1708673474, 2024-09-10T15:58:47.194 INFO:teuthology.orchestra.run.smithi115.stdout: "max": 0, 2024-09-10T15:58:47.194 INFO:teuthology.orchestra.run.smithi115.stdout: "pool": 2, 2024-09-10T15:58:47.194 INFO:teuthology.orchestra.run.smithi115.stdout: "namespace": "" 2024-09-10T15:58:47.194 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-10T15:58:47.195 INFO:teuthology.orchestra.run.smithi115.stdout: "need": "23'5", 2024-09-10T15:58:47.195 INFO:teuthology.orchestra.run.smithi115.stdout: "have": "0'0", 2024-09-10T15:58:47.195 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": "none", 2024-09-10T15:58:47.195 INFO:teuthology.orchestra.run.smithi115.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2024-09-10T15:58:47.195 INFO:teuthology.orchestra.run.smithi115.stdout: "locations": [] 2024-09-10T15:58:47.195 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-10T15:58:47.195 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-10T15:58:47.195 INFO:teuthology.orchestra.run.smithi115.stdout: "oid": { 2024-09-10T15:58:47.195 INFO:teuthology.orchestra.run.smithi115.stdout: "oid": "existed_9", 2024-09-10T15:58:47.195 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "", 2024-09-10T15:58:47.195 INFO:teuthology.orchestra.run.smithi115.stdout: "snapid": -2, 2024-09-10T15:58:47.195 INFO:teuthology.orchestra.run.smithi115.stdout: "hash": 831402034, 2024-09-10T15:58:47.195 INFO:teuthology.orchestra.run.smithi115.stdout: "max": 0, 2024-09-10T15:58:47.195 INFO:teuthology.orchestra.run.smithi115.stdout: "pool": 2, 2024-09-10T15:58:47.196 INFO:teuthology.orchestra.run.smithi115.stdout: "namespace": "" 2024-09-10T15:58:47.196 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-10T15:58:47.196 INFO:teuthology.orchestra.run.smithi115.stdout: "need": "23'6", 2024-09-10T15:58:47.196 INFO:teuthology.orchestra.run.smithi115.stdout: "have": "0'0", 2024-09-10T15:58:47.196 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": "none", 2024-09-10T15:58:47.196 INFO:teuthology.orchestra.run.smithi115.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2024-09-10T15:58:47.196 INFO:teuthology.orchestra.run.smithi115.stdout: "locations": [] 2024-09-10T15:58:47.196 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-10T15:58:47.196 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-09-10T15:58:47.196 INFO:teuthology.orchestra.run.smithi115.stdout: "state": "Active", 2024-09-10T15:58:47.196 INFO:teuthology.orchestra.run.smithi115.stdout: "available_might_have_unfound": true, 2024-09-10T15:58:47.196 INFO:teuthology.orchestra.run.smithi115.stdout: "might_have_unfound": [ 2024-09-10T15:58:47.196 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-10T15:58:47.196 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "1", 2024-09-10T15:58:47.197 INFO:teuthology.orchestra.run.smithi115.stdout: "status": "osd is down" 2024-09-10T15:58:47.197 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-10T15:58:47.197 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-09-10T15:58:47.197 INFO:teuthology.orchestra.run.smithi115.stdout: "more": false 2024-09-10T15:58:47.197 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-10T15:58:47.212 INFO:tasks.lost_unfound:reverting unfound in 2.2 on osd.0 2024-09-10T15:58:47.213 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg 2.2 mark_unfound_lost revert 2024-09-10T15:58:47.441 INFO:teuthology.orchestra.run.smithi115.stderr:pg has 2 objects unfound and apparently lost marking 2024-09-10T15:58:47.463 INFO:tasks.lost_unfound:no unfound in 1.2 2024-09-10T15:58:47.463 INFO:tasks.lost_unfound:listing missing/lost in 2.1 state active+recovery_unfound+degraded 2024-09-10T15:58:47.463 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- pg 2.1 list_unfound '{}' 2024-09-10T15:58:47.665 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-10T15:58:47.665 INFO:teuthology.orchestra.run.smithi115.stdout: "offset": { 2024-09-10T15:58:47.665 INFO:teuthology.orchestra.run.smithi115.stdout: "oid": "", 2024-09-10T15:58:47.665 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "", 2024-09-10T15:58:47.665 INFO:teuthology.orchestra.run.smithi115.stdout: "snapid": 0, 2024-09-10T15:58:47.665 INFO:teuthology.orchestra.run.smithi115.stdout: "hash": 0, 2024-09-10T15:58:47.666 INFO:teuthology.orchestra.run.smithi115.stdout: "max": 0, 2024-09-10T15:58:47.666 INFO:teuthology.orchestra.run.smithi115.stdout: "pool": -9223372036854775808, 2024-09-10T15:58:47.666 INFO:teuthology.orchestra.run.smithi115.stdout: "namespace": "" 2024-09-10T15:58:47.666 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-10T15:58:47.666 INFO:teuthology.orchestra.run.smithi115.stdout: "num_missing": 2, 2024-09-10T15:58:47.666 INFO:teuthology.orchestra.run.smithi115.stdout: "num_unfound": 2, 2024-09-10T15:58:47.666 INFO:teuthology.orchestra.run.smithi115.stdout: "objects": [ 2024-09-10T15:58:47.666 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-10T15:58:47.666 INFO:teuthology.orchestra.run.smithi115.stdout: "oid": { 2024-09-10T15:58:47.666 INFO:teuthology.orchestra.run.smithi115.stdout: "oid": "existing_3", 2024-09-10T15:58:47.666 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "", 2024-09-10T15:58:47.666 INFO:teuthology.orchestra.run.smithi115.stdout: "snapid": -2, 2024-09-10T15:58:47.666 INFO:teuthology.orchestra.run.smithi115.stdout: "hash": 459768081, 2024-09-10T15:58:47.666 INFO:teuthology.orchestra.run.smithi115.stdout: "max": 0, 2024-09-10T15:58:47.667 INFO:teuthology.orchestra.run.smithi115.stdout: "pool": 2, 2024-09-10T15:58:47.667 INFO:teuthology.orchestra.run.smithi115.stdout: "namespace": "" 2024-09-10T15:58:47.667 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-10T15:58:47.667 INFO:teuthology.orchestra.run.smithi115.stdout: "need": "23'3", 2024-09-10T15:58:47.667 INFO:teuthology.orchestra.run.smithi115.stdout: "have": "21'1", 2024-09-10T15:58:47.667 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": "none", 2024-09-10T15:58:47.667 INFO:teuthology.orchestra.run.smithi115.stdout: "clean_regions": "clean_offsets: [(483, 18446744073709551132)], clean_omap: true, new_object: false", 2024-09-10T15:58:47.667 INFO:teuthology.orchestra.run.smithi115.stdout: "locations": [] 2024-09-10T15:58:47.667 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-10T15:58:47.667 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-10T15:58:47.667 INFO:teuthology.orchestra.run.smithi115.stdout: "oid": { 2024-09-10T15:58:47.667 INFO:teuthology.orchestra.run.smithi115.stdout: "oid": "new_3", 2024-09-10T15:58:47.667 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "", 2024-09-10T15:58:47.668 INFO:teuthology.orchestra.run.smithi115.stdout: "snapid": -2, 2024-09-10T15:58:47.668 INFO:teuthology.orchestra.run.smithi115.stdout: "hash": 480221105, 2024-09-10T15:58:47.668 INFO:teuthology.orchestra.run.smithi115.stdout: "max": 0, 2024-09-10T15:58:47.668 INFO:teuthology.orchestra.run.smithi115.stdout: "pool": 2, 2024-09-10T15:58:47.668 INFO:teuthology.orchestra.run.smithi115.stdout: "namespace": "" 2024-09-10T15:58:47.668 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-10T15:58:47.668 INFO:teuthology.orchestra.run.smithi115.stdout: "need": "23'2", 2024-09-10T15:58:47.668 INFO:teuthology.orchestra.run.smithi115.stdout: "have": "0'0", 2024-09-10T15:58:47.668 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": "none", 2024-09-10T15:58:47.668 INFO:teuthology.orchestra.run.smithi115.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2024-09-10T15:58:47.668 INFO:teuthology.orchestra.run.smithi115.stdout: "locations": [] 2024-09-10T15:58:47.668 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-10T15:58:47.668 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-09-10T15:58:47.669 INFO:teuthology.orchestra.run.smithi115.stdout: "state": "NotRecovering", 2024-09-10T15:58:47.669 INFO:teuthology.orchestra.run.smithi115.stdout: "available_might_have_unfound": true, 2024-09-10T15:58:47.669 INFO:teuthology.orchestra.run.smithi115.stdout: "might_have_unfound": [ 2024-09-10T15:58:47.669 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-10T15:58:47.669 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "1", 2024-09-10T15:58:47.669 INFO:teuthology.orchestra.run.smithi115.stdout: "status": "osd is down" 2024-09-10T15:58:47.669 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-10T15:58:47.669 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-09-10T15:58:47.669 INFO:teuthology.orchestra.run.smithi115.stdout: "more": false 2024-09-10T15:58:47.669 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-10T15:58:47.683 INFO:tasks.lost_unfound:reverting unfound in 2.1 on osd.2 2024-09-10T15:58:47.684 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg 2.1 mark_unfound_lost revert 2024-09-10T15:58:47.914 INFO:teuthology.orchestra.run.smithi115.stderr:pg has 2 objects unfound and apparently lost marking 2024-09-10T15:58:47.934 INFO:tasks.lost_unfound:no unfound in 1.3 2024-09-10T15:58:47.934 INFO:tasks.lost_unfound:listing missing/lost in 2.0 state active+recovery_unfound+degraded 2024-09-10T15:58:47.935 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- pg 2.0 list_unfound '{}' 2024-09-10T15:58:47.962 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 15 20 35902 35882 9.34309 6.00781 0.00682336 0.008355 2024-09-10T15:58:48.153 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-10T15:58:48.154 INFO:teuthology.orchestra.run.smithi115.stdout: "offset": { 2024-09-10T15:58:48.154 INFO:teuthology.orchestra.run.smithi115.stdout: "oid": "", 2024-09-10T15:58:48.154 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "", 2024-09-10T15:58:48.154 INFO:teuthology.orchestra.run.smithi115.stdout: "snapid": 0, 2024-09-10T15:58:48.154 INFO:teuthology.orchestra.run.smithi115.stdout: "hash": 0, 2024-09-10T15:58:48.154 INFO:teuthology.orchestra.run.smithi115.stdout: "max": 0, 2024-09-10T15:58:48.154 INFO:teuthology.orchestra.run.smithi115.stdout: "pool": -9223372036854775808, 2024-09-10T15:58:48.154 INFO:teuthology.orchestra.run.smithi115.stdout: "namespace": "" 2024-09-10T15:58:48.154 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-10T15:58:48.154 INFO:teuthology.orchestra.run.smithi115.stdout: "num_missing": 2, 2024-09-10T15:58:48.154 INFO:teuthology.orchestra.run.smithi115.stdout: "num_unfound": 2, 2024-09-10T15:58:48.154 INFO:teuthology.orchestra.run.smithi115.stdout: "objects": [ 2024-09-10T15:58:48.154 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-10T15:58:48.155 INFO:teuthology.orchestra.run.smithi115.stdout: "oid": { 2024-09-10T15:58:48.155 INFO:teuthology.orchestra.run.smithi115.stdout: "oid": "new_5", 2024-09-10T15:58:48.155 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "", 2024-09-10T15:58:48.155 INFO:teuthology.orchestra.run.smithi115.stdout: "snapid": -2, 2024-09-10T15:58:48.155 INFO:teuthology.orchestra.run.smithi115.stdout: "hash": 4248945696, 2024-09-10T15:58:48.155 INFO:teuthology.orchestra.run.smithi115.stdout: "max": 0, 2024-09-10T15:58:48.155 INFO:teuthology.orchestra.run.smithi115.stdout: "pool": 2, 2024-09-10T15:58:48.155 INFO:teuthology.orchestra.run.smithi115.stdout: "namespace": "" 2024-09-10T15:58:48.155 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-10T15:58:48.155 INFO:teuthology.orchestra.run.smithi115.stdout: "need": "23'3", 2024-09-10T15:58:48.155 INFO:teuthology.orchestra.run.smithi115.stdout: "have": "0'0", 2024-09-10T15:58:48.155 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": "none", 2024-09-10T15:58:48.155 INFO:teuthology.orchestra.run.smithi115.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2024-09-10T15:58:48.155 INFO:teuthology.orchestra.run.smithi115.stdout: "locations": [] 2024-09-10T15:58:48.156 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-10T15:58:48.156 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-10T15:58:48.156 INFO:teuthology.orchestra.run.smithi115.stdout: "oid": { 2024-09-10T15:58:48.156 INFO:teuthology.orchestra.run.smithi115.stdout: "oid": "existed_5", 2024-09-10T15:58:48.156 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "", 2024-09-10T15:58:48.156 INFO:teuthology.orchestra.run.smithi115.stdout: "snapid": -2, 2024-09-10T15:58:48.156 INFO:teuthology.orchestra.run.smithi115.stdout: "hash": 4212789968, 2024-09-10T15:58:48.156 INFO:teuthology.orchestra.run.smithi115.stdout: "max": 0, 2024-09-10T15:58:48.156 INFO:teuthology.orchestra.run.smithi115.stdout: "pool": 2, 2024-09-10T15:58:48.156 INFO:teuthology.orchestra.run.smithi115.stdout: "namespace": "" 2024-09-10T15:58:48.156 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-10T15:58:48.156 INFO:teuthology.orchestra.run.smithi115.stdout: "need": "23'4", 2024-09-10T15:58:48.157 INFO:teuthology.orchestra.run.smithi115.stdout: "have": "0'0", 2024-09-10T15:58:48.157 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": "none", 2024-09-10T15:58:48.157 INFO:teuthology.orchestra.run.smithi115.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2024-09-10T15:58:48.157 INFO:teuthology.orchestra.run.smithi115.stdout: "locations": [] 2024-09-10T15:58:48.157 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-10T15:58:48.157 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-09-10T15:58:48.157 INFO:teuthology.orchestra.run.smithi115.stdout: "state": "NotRecovering", 2024-09-10T15:58:48.157 INFO:teuthology.orchestra.run.smithi115.stdout: "available_might_have_unfound": true, 2024-09-10T15:58:48.157 INFO:teuthology.orchestra.run.smithi115.stdout: "might_have_unfound": [ 2024-09-10T15:58:48.157 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-10T15:58:48.157 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "1", 2024-09-10T15:58:48.157 INFO:teuthology.orchestra.run.smithi115.stdout: "status": "osd is down" 2024-09-10T15:58:48.157 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-10T15:58:48.158 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-09-10T15:58:48.158 INFO:teuthology.orchestra.run.smithi115.stdout: "more": false 2024-09-10T15:58:48.158 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-10T15:58:48.179 INFO:tasks.lost_unfound:reverting unfound in 2.0 on osd.2 2024-09-10T15:58:48.179 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg 2.0 mark_unfound_lost revert 2024-09-10T15:58:48.461 INFO:teuthology.orchestra.run.smithi115.stderr:pg has 2 objects unfound and apparently lost marking 2024-09-10T15:58:48.484 INFO:tasks.lost_unfound:listing missing/lost in 2.7 state active+recovery_wait+degraded 2024-09-10T15:58:48.485 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- pg 2.7 list_unfound '{}' 2024-09-10T15:58:48.690 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-10T15:58:48.690 INFO:teuthology.orchestra.run.smithi115.stdout: "offset": { 2024-09-10T15:58:48.690 INFO:teuthology.orchestra.run.smithi115.stdout: "oid": "", 2024-09-10T15:58:48.690 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "", 2024-09-10T15:58:48.690 INFO:teuthology.orchestra.run.smithi115.stdout: "snapid": 0, 2024-09-10T15:58:48.691 INFO:teuthology.orchestra.run.smithi115.stdout: "hash": 0, 2024-09-10T15:58:48.691 INFO:teuthology.orchestra.run.smithi115.stdout: "max": 0, 2024-09-10T15:58:48.691 INFO:teuthology.orchestra.run.smithi115.stdout: "pool": -9223372036854775808, 2024-09-10T15:58:48.691 INFO:teuthology.orchestra.run.smithi115.stdout: "namespace": "" 2024-09-10T15:58:48.691 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-10T15:58:48.691 INFO:teuthology.orchestra.run.smithi115.stdout: "num_missing": 1, 2024-09-10T15:58:48.691 INFO:teuthology.orchestra.run.smithi115.stdout: "num_unfound": 1, 2024-09-10T15:58:48.691 INFO:teuthology.orchestra.run.smithi115.stdout: "objects": [ 2024-09-10T15:58:48.691 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-10T15:58:48.691 INFO:teuthology.orchestra.run.smithi115.stdout: "oid": { 2024-09-10T15:58:48.691 INFO:teuthology.orchestra.run.smithi115.stdout: "oid": "new_7", 2024-09-10T15:58:48.691 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "", 2024-09-10T15:58:48.692 INFO:teuthology.orchestra.run.smithi115.stdout: "snapid": -2, 2024-09-10T15:58:48.692 INFO:teuthology.orchestra.run.smithi115.stdout: "hash": 3761017527, 2024-09-10T15:58:48.692 INFO:teuthology.orchestra.run.smithi115.stdout: "max": 0, 2024-09-10T15:58:48.692 INFO:teuthology.orchestra.run.smithi115.stdout: "pool": 2, 2024-09-10T15:58:48.692 INFO:teuthology.orchestra.run.smithi115.stdout: "namespace": "" 2024-09-10T15:58:48.692 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-10T15:58:48.692 INFO:teuthology.orchestra.run.smithi115.stdout: "need": "23'1", 2024-09-10T15:58:48.692 INFO:teuthology.orchestra.run.smithi115.stdout: "have": "0'0", 2024-09-10T15:58:48.692 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": "none", 2024-09-10T15:58:48.692 INFO:teuthology.orchestra.run.smithi115.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2024-09-10T15:58:48.692 INFO:teuthology.orchestra.run.smithi115.stdout: "locations": [] 2024-09-10T15:58:48.692 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-10T15:58:48.692 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-09-10T15:58:48.693 INFO:teuthology.orchestra.run.smithi115.stdout: "state": "Active", 2024-09-10T15:58:48.693 INFO:teuthology.orchestra.run.smithi115.stdout: "available_might_have_unfound": true, 2024-09-10T15:58:48.693 INFO:teuthology.orchestra.run.smithi115.stdout: "might_have_unfound": [ 2024-09-10T15:58:48.693 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-10T15:58:48.693 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "1", 2024-09-10T15:58:48.693 INFO:teuthology.orchestra.run.smithi115.stdout: "status": "osd is down" 2024-09-10T15:58:48.693 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-10T15:58:48.693 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-09-10T15:58:48.693 INFO:teuthology.orchestra.run.smithi115.stdout: "more": false 2024-09-10T15:58:48.693 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-10T15:58:48.720 INFO:tasks.lost_unfound:reverting unfound in 2.7 on osd.0 2024-09-10T15:58:48.720 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg 2.7 mark_unfound_lost revert 2024-09-10T15:58:48.962 INFO:teuthology.orchestra.run.smithi115.stderr:pg has 1 objects unfound and apparently lost marking 2024-09-10T15:58:48.963 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 16 20 37418 37398 9.12914 5.92188 0.0156815 0.00855215 2024-09-10T15:58:48.986 INFO:tasks.lost_unfound:no unfound in 1.4 2024-09-10T15:58:48.986 INFO:tasks.lost_unfound:listing missing/lost in 2.6 state active+recovery_wait+degraded 2024-09-10T15:58:48.986 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- pg 2.6 list_unfound '{}' 2024-09-10T15:58:49.192 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-10T15:58:49.192 INFO:teuthology.orchestra.run.smithi115.stdout: "offset": { 2024-09-10T15:58:49.193 INFO:teuthology.orchestra.run.smithi115.stdout: "oid": "", 2024-09-10T15:58:49.193 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "", 2024-09-10T15:58:49.193 INFO:teuthology.orchestra.run.smithi115.stdout: "snapid": 0, 2024-09-10T15:58:49.193 INFO:teuthology.orchestra.run.smithi115.stdout: "hash": 0, 2024-09-10T15:58:49.193 INFO:teuthology.orchestra.run.smithi115.stdout: "max": 0, 2024-09-10T15:58:49.193 INFO:teuthology.orchestra.run.smithi115.stdout: "pool": -9223372036854775808, 2024-09-10T15:58:49.193 INFO:teuthology.orchestra.run.smithi115.stdout: "namespace": "" 2024-09-10T15:58:49.193 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-10T15:58:49.193 INFO:teuthology.orchestra.run.smithi115.stdout: "num_missing": 3, 2024-09-10T15:58:49.193 INFO:teuthology.orchestra.run.smithi115.stdout: "num_unfound": 3, 2024-09-10T15:58:49.193 INFO:teuthology.orchestra.run.smithi115.stdout: "objects": [ 2024-09-10T15:58:49.193 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-10T15:58:49.193 INFO:teuthology.orchestra.run.smithi115.stdout: "oid": { 2024-09-10T15:58:49.194 INFO:teuthology.orchestra.run.smithi115.stdout: "oid": "existed_8", 2024-09-10T15:58:49.194 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "", 2024-09-10T15:58:49.194 INFO:teuthology.orchestra.run.smithi115.stdout: "snapid": -2, 2024-09-10T15:58:49.194 INFO:teuthology.orchestra.run.smithi115.stdout: "hash": 1861424966, 2024-09-10T15:58:49.194 INFO:teuthology.orchestra.run.smithi115.stdout: "max": 0, 2024-09-10T15:58:49.194 INFO:teuthology.orchestra.run.smithi115.stdout: "pool": 2, 2024-09-10T15:58:49.194 INFO:teuthology.orchestra.run.smithi115.stdout: "namespace": "" 2024-09-10T15:58:49.194 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-10T15:58:49.194 INFO:teuthology.orchestra.run.smithi115.stdout: "need": "23'5", 2024-09-10T15:58:49.194 INFO:teuthology.orchestra.run.smithi115.stdout: "have": "0'0", 2024-09-10T15:58:49.194 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": "none", 2024-09-10T15:58:49.194 INFO:teuthology.orchestra.run.smithi115.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2024-09-10T15:58:49.194 INFO:teuthology.orchestra.run.smithi115.stdout: "locations": [] 2024-09-10T15:58:49.194 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-10T15:58:49.195 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-10T15:58:49.195 INFO:teuthology.orchestra.run.smithi115.stdout: "oid": { 2024-09-10T15:58:49.195 INFO:teuthology.orchestra.run.smithi115.stdout: "oid": "new_6", 2024-09-10T15:58:49.195 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "", 2024-09-10T15:58:49.195 INFO:teuthology.orchestra.run.smithi115.stdout: "snapid": -2, 2024-09-10T15:58:49.195 INFO:teuthology.orchestra.run.smithi115.stdout: "hash": 3910131350, 2024-09-10T15:58:49.195 INFO:teuthology.orchestra.run.smithi115.stdout: "max": 0, 2024-09-10T15:58:49.195 INFO:teuthology.orchestra.run.smithi115.stdout: "pool": 2, 2024-09-10T15:58:49.195 INFO:teuthology.orchestra.run.smithi115.stdout: "namespace": "" 2024-09-10T15:58:49.195 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-10T15:58:49.195 INFO:teuthology.orchestra.run.smithi115.stdout: "need": "23'4", 2024-09-10T15:58:49.195 INFO:teuthology.orchestra.run.smithi115.stdout: "have": "0'0", 2024-09-10T15:58:49.195 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": "none", 2024-09-10T15:58:49.196 INFO:teuthology.orchestra.run.smithi115.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2024-09-10T15:58:49.196 INFO:teuthology.orchestra.run.smithi115.stdout: "locations": [] 2024-09-10T15:58:49.196 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-10T15:58:49.196 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-10T15:58:49.196 INFO:teuthology.orchestra.run.smithi115.stdout: "oid": { 2024-09-10T15:58:49.196 INFO:teuthology.orchestra.run.smithi115.stdout: "oid": "new_4", 2024-09-10T15:58:49.196 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "", 2024-09-10T15:58:49.196 INFO:teuthology.orchestra.run.smithi115.stdout: "snapid": -2, 2024-09-10T15:58:49.196 INFO:teuthology.orchestra.run.smithi115.stdout: "hash": 2787839990, 2024-09-10T15:58:49.196 INFO:teuthology.orchestra.run.smithi115.stdout: "max": 0, 2024-09-10T15:58:49.196 INFO:teuthology.orchestra.run.smithi115.stdout: "pool": 2, 2024-09-10T15:58:49.196 INFO:teuthology.orchestra.run.smithi115.stdout: "namespace": "" 2024-09-10T15:58:49.196 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-10T15:58:49.197 INFO:teuthology.orchestra.run.smithi115.stdout: "need": "23'3", 2024-09-10T15:58:49.197 INFO:teuthology.orchestra.run.smithi115.stdout: "have": "0'0", 2024-09-10T15:58:49.197 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": "none", 2024-09-10T15:58:49.197 INFO:teuthology.orchestra.run.smithi115.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2024-09-10T15:58:49.197 INFO:teuthology.orchestra.run.smithi115.stdout: "locations": [] 2024-09-10T15:58:49.197 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-10T15:58:49.197 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-09-10T15:58:49.197 INFO:teuthology.orchestra.run.smithi115.stdout: "state": "Active", 2024-09-10T15:58:49.197 INFO:teuthology.orchestra.run.smithi115.stdout: "available_might_have_unfound": true, 2024-09-10T15:58:49.197 INFO:teuthology.orchestra.run.smithi115.stdout: "might_have_unfound": [ 2024-09-10T15:58:49.197 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-10T15:58:49.197 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "1", 2024-09-10T15:58:49.197 INFO:teuthology.orchestra.run.smithi115.stdout: "status": "osd is down" 2024-09-10T15:58:49.198 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-10T15:58:49.198 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-09-10T15:58:49.198 INFO:teuthology.orchestra.run.smithi115.stdout: "more": false 2024-09-10T15:58:49.198 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-10T15:58:49.217 INFO:tasks.lost_unfound:reverting unfound in 2.6 on osd.0 2024-09-10T15:58:49.217 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg 2.6 mark_unfound_lost revert 2024-09-10T15:58:49.436 INFO:teuthology.orchestra.run.smithi115.stderr:pg has 3 objects unfound and apparently lost marking 2024-09-10T15:58:49.459 INFO:tasks.lost_unfound:no unfound in 1.5 2024-09-10T15:58:49.459 INFO:tasks.lost_unfound:listing missing/lost in 2.5 state active+recovery_wait+degraded 2024-09-10T15:58:49.459 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- pg 2.5 list_unfound '{}' 2024-09-10T15:58:49.666 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-10T15:58:49.666 INFO:teuthology.orchestra.run.smithi115.stdout: "offset": { 2024-09-10T15:58:49.666 INFO:teuthology.orchestra.run.smithi115.stdout: "oid": "", 2024-09-10T15:58:49.666 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "", 2024-09-10T15:58:49.666 INFO:teuthology.orchestra.run.smithi115.stdout: "snapid": 0, 2024-09-10T15:58:49.667 INFO:teuthology.orchestra.run.smithi115.stdout: "hash": 0, 2024-09-10T15:58:49.667 INFO:teuthology.orchestra.run.smithi115.stdout: "max": 0, 2024-09-10T15:58:49.667 INFO:teuthology.orchestra.run.smithi115.stdout: "pool": -9223372036854775808, 2024-09-10T15:58:49.667 INFO:teuthology.orchestra.run.smithi115.stdout: "namespace": "" 2024-09-10T15:58:49.667 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-10T15:58:49.667 INFO:teuthology.orchestra.run.smithi115.stdout: "num_missing": 2, 2024-09-10T15:58:49.667 INFO:teuthology.orchestra.run.smithi115.stdout: "num_unfound": 2, 2024-09-10T15:58:49.667 INFO:teuthology.orchestra.run.smithi115.stdout: "objects": [ 2024-09-10T15:58:49.667 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-10T15:58:49.667 INFO:teuthology.orchestra.run.smithi115.stdout: "oid": { 2024-09-10T15:58:49.667 INFO:teuthology.orchestra.run.smithi115.stdout: "oid": "new_1", 2024-09-10T15:58:49.667 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "", 2024-09-10T15:58:49.667 INFO:teuthology.orchestra.run.smithi115.stdout: "snapid": -2, 2024-09-10T15:58:49.667 INFO:teuthology.orchestra.run.smithi115.stdout: "hash": 26281365, 2024-09-10T15:58:49.668 INFO:teuthology.orchestra.run.smithi115.stdout: "max": 0, 2024-09-10T15:58:49.668 INFO:teuthology.orchestra.run.smithi115.stdout: "pool": 2, 2024-09-10T15:58:49.668 INFO:teuthology.orchestra.run.smithi115.stdout: "namespace": "" 2024-09-10T15:58:49.668 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-10T15:58:49.668 INFO:teuthology.orchestra.run.smithi115.stdout: "need": "23'1", 2024-09-10T15:58:49.668 INFO:teuthology.orchestra.run.smithi115.stdout: "have": "0'0", 2024-09-10T15:58:49.668 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": "none", 2024-09-10T15:58:49.668 INFO:teuthology.orchestra.run.smithi115.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2024-09-10T15:58:49.668 INFO:teuthology.orchestra.run.smithi115.stdout: "locations": [] 2024-09-10T15:58:49.668 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-10T15:58:49.668 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-10T15:58:49.668 INFO:teuthology.orchestra.run.smithi115.stdout: "oid": { 2024-09-10T15:58:49.669 INFO:teuthology.orchestra.run.smithi115.stdout: "oid": "new_2", 2024-09-10T15:58:49.669 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "", 2024-09-10T15:58:49.669 INFO:teuthology.orchestra.run.smithi115.stdout: "snapid": -2, 2024-09-10T15:58:49.669 INFO:teuthology.orchestra.run.smithi115.stdout: "hash": 3196340981, 2024-09-10T15:58:49.669 INFO:teuthology.orchestra.run.smithi115.stdout: "max": 0, 2024-09-10T15:58:49.669 INFO:teuthology.orchestra.run.smithi115.stdout: "pool": 2, 2024-09-10T15:58:49.669 INFO:teuthology.orchestra.run.smithi115.stdout: "namespace": "" 2024-09-10T15:58:49.669 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-10T15:58:49.669 INFO:teuthology.orchestra.run.smithi115.stdout: "need": "23'2", 2024-09-10T15:58:49.669 INFO:teuthology.orchestra.run.smithi115.stdout: "have": "0'0", 2024-09-10T15:58:49.669 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": "none", 2024-09-10T15:58:49.669 INFO:teuthology.orchestra.run.smithi115.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2024-09-10T15:58:49.669 INFO:teuthology.orchestra.run.smithi115.stdout: "locations": [] 2024-09-10T15:58:49.669 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-10T15:58:49.670 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-09-10T15:58:49.670 INFO:teuthology.orchestra.run.smithi115.stdout: "state": "Active", 2024-09-10T15:58:49.670 INFO:teuthology.orchestra.run.smithi115.stdout: "available_might_have_unfound": true, 2024-09-10T15:58:49.670 INFO:teuthology.orchestra.run.smithi115.stdout: "might_have_unfound": [ 2024-09-10T15:58:49.670 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-10T15:58:49.670 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "1", 2024-09-10T15:58:49.670 INFO:teuthology.orchestra.run.smithi115.stdout: "status": "osd is down" 2024-09-10T15:58:49.670 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-10T15:58:49.670 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-09-10T15:58:49.670 INFO:teuthology.orchestra.run.smithi115.stdout: "more": false 2024-09-10T15:58:49.670 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-10T15:58:49.689 INFO:tasks.lost_unfound:reverting unfound in 2.5 on osd.0 2024-09-10T15:58:49.689 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg 2.5 mark_unfound_lost revert 2024-09-10T15:58:49.925 INFO:teuthology.orchestra.run.smithi115.stderr:pg has 2 objects unfound and apparently lost marking 2024-09-10T15:58:49.950 INFO:tasks.lost_unfound:no unfound in 1.6 2024-09-10T15:58:49.950 INFO:tasks.lost_unfound:no unfound in 1.7 2024-09-10T15:58:49.950 INFO:tasks.lost_unfound:listing missing/lost in 2.4 state active+recovery_wait+degraded 2024-09-10T15:58:49.950 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- pg 2.4 list_unfound '{}' 2024-09-10T15:58:49.962 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 17 20 38863 38843 8.92415 5.64453 0.0100659 0.00874912 2024-09-10T15:58:50.160 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-10T15:58:50.161 INFO:teuthology.orchestra.run.smithi115.stdout: "offset": { 2024-09-10T15:58:50.161 INFO:teuthology.orchestra.run.smithi115.stdout: "oid": "", 2024-09-10T15:58:50.161 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "", 2024-09-10T15:58:50.161 INFO:teuthology.orchestra.run.smithi115.stdout: "snapid": 0, 2024-09-10T15:58:50.161 INFO:teuthology.orchestra.run.smithi115.stdout: "hash": 0, 2024-09-10T15:58:50.161 INFO:teuthology.orchestra.run.smithi115.stdout: "max": 0, 2024-09-10T15:58:50.161 INFO:teuthology.orchestra.run.smithi115.stdout: "pool": -9223372036854775808, 2024-09-10T15:58:50.161 INFO:teuthology.orchestra.run.smithi115.stdout: "namespace": "" 2024-09-10T15:58:50.162 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-10T15:58:50.162 INFO:teuthology.orchestra.run.smithi115.stdout: "num_missing": 4, 2024-09-10T15:58:50.162 INFO:teuthology.orchestra.run.smithi115.stdout: "num_unfound": 3, 2024-09-10T15:58:50.162 INFO:teuthology.orchestra.run.smithi115.stdout: "objects": [ 2024-09-10T15:58:50.162 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-10T15:58:50.162 INFO:teuthology.orchestra.run.smithi115.stdout: "oid": { 2024-09-10T15:58:50.162 INFO:teuthology.orchestra.run.smithi115.stdout: "oid": "existed_4", 2024-09-10T15:58:50.162 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "", 2024-09-10T15:58:50.162 INFO:teuthology.orchestra.run.smithi115.stdout: "snapid": -2, 2024-09-10T15:58:50.162 INFO:teuthology.orchestra.run.smithi115.stdout: "hash": 873891716, 2024-09-10T15:58:50.162 INFO:teuthology.orchestra.run.smithi115.stdout: "max": 0, 2024-09-10T15:58:50.163 INFO:teuthology.orchestra.run.smithi115.stdout: "pool": 2, 2024-09-10T15:58:50.163 INFO:teuthology.orchestra.run.smithi115.stdout: "namespace": "" 2024-09-10T15:58:50.163 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-10T15:58:50.163 INFO:teuthology.orchestra.run.smithi115.stdout: "need": "23'8", 2024-09-10T15:58:50.163 INFO:teuthology.orchestra.run.smithi115.stdout: "have": "0'0", 2024-09-10T15:58:50.163 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": "none", 2024-09-10T15:58:50.163 INFO:teuthology.orchestra.run.smithi115.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2024-09-10T15:58:50.163 INFO:teuthology.orchestra.run.smithi115.stdout: "locations": [] 2024-09-10T15:58:50.163 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-10T15:58:50.163 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-10T15:58:50.164 INFO:teuthology.orchestra.run.smithi115.stdout: "oid": { 2024-09-10T15:58:50.164 INFO:teuthology.orchestra.run.smithi115.stdout: "oid": "existing_7", 2024-09-10T15:58:50.164 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "", 2024-09-10T15:58:50.164 INFO:teuthology.orchestra.run.smithi115.stdout: "snapid": -2, 2024-09-10T15:58:50.164 INFO:teuthology.orchestra.run.smithi115.stdout: "hash": 3996155460, 2024-09-10T15:58:50.164 INFO:teuthology.orchestra.run.smithi115.stdout: "max": 0, 2024-09-10T15:58:50.164 INFO:teuthology.orchestra.run.smithi115.stdout: "pool": 2, 2024-09-10T15:58:50.164 INFO:teuthology.orchestra.run.smithi115.stdout: "namespace": "" 2024-09-10T15:58:50.164 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-10T15:58:50.164 INFO:teuthology.orchestra.run.smithi115.stdout: "need": "23'9", 2024-09-10T15:58:50.164 INFO:teuthology.orchestra.run.smithi115.stdout: "have": "21'6", 2024-09-10T15:58:50.165 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": "none", 2024-09-10T15:58:50.165 INFO:teuthology.orchestra.run.smithi115.stdout: "clean_regions": "clean_offsets: [(483, 18446744073709551132)], clean_omap: true, new_object: false", 2024-09-10T15:58:50.165 INFO:teuthology.orchestra.run.smithi115.stdout: "locations": [] 2024-09-10T15:58:50.165 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-10T15:58:50.165 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-10T15:58:50.165 INFO:teuthology.orchestra.run.smithi115.stdout: "oid": { 2024-09-10T15:58:50.165 INFO:teuthology.orchestra.run.smithi115.stdout: "oid": "existed_2", 2024-09-10T15:58:50.165 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "", 2024-09-10T15:58:50.165 INFO:teuthology.orchestra.run.smithi115.stdout: "snapid": -2, 2024-09-10T15:58:50.165 INFO:teuthology.orchestra.run.smithi115.stdout: "hash": 1767667508, 2024-09-10T15:58:50.166 INFO:teuthology.orchestra.run.smithi115.stdout: "max": 0, 2024-09-10T15:58:50.166 INFO:teuthology.orchestra.run.smithi115.stdout: "pool": 2, 2024-09-10T15:58:50.166 INFO:teuthology.orchestra.run.smithi115.stdout: "namespace": "" 2024-09-10T15:58:50.166 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-10T15:58:50.166 INFO:teuthology.orchestra.run.smithi115.stdout: "need": "23'7", 2024-09-10T15:58:50.166 INFO:teuthology.orchestra.run.smithi115.stdout: "have": "0'0", 2024-09-10T15:58:50.166 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": "none", 2024-09-10T15:58:50.166 INFO:teuthology.orchestra.run.smithi115.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2024-09-10T15:58:50.166 INFO:teuthology.orchestra.run.smithi115.stdout: "locations": [] 2024-09-10T15:58:50.166 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-10T15:58:50.166 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-09-10T15:58:50.167 INFO:teuthology.orchestra.run.smithi115.stdout: "state": "Active", 2024-09-10T15:58:50.167 INFO:teuthology.orchestra.run.smithi115.stdout: "available_might_have_unfound": true, 2024-09-10T15:58:50.167 INFO:teuthology.orchestra.run.smithi115.stdout: "might_have_unfound": [ 2024-09-10T15:58:50.167 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-10T15:58:50.167 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "1", 2024-09-10T15:58:50.167 INFO:teuthology.orchestra.run.smithi115.stdout: "status": "osd is down" 2024-09-10T15:58:50.167 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-10T15:58:50.167 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-09-10T15:58:50.167 INFO:teuthology.orchestra.run.smithi115.stdout: "more": false 2024-09-10T15:58:50.167 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-10T15:58:50.187 INFO:tasks.lost_unfound:reverting unfound in 2.4 on osd.0 2024-09-10T15:58:50.187 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg 2.4 mark_unfound_lost revert 2024-09-10T15:58:50.451 INFO:teuthology.orchestra.run.smithi115.stderr:pg has 3 objects unfound and apparently lost marking 2024-09-10T15:58:50.477 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 debug kick_recovery_wq 5 2024-09-10T15:58:50.695 INFO:teuthology.orchestra.run.smithi115.stderr:kicking recovery queue. set osd_recovery_delay_start to 5 2024-09-10T15:58:50.714 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 debug kick_recovery_wq 5 2024-09-10T15:58:50.928 INFO:teuthology.orchestra.run.smithi115.stderr:kicking recovery queue. set osd_recovery_delay_start to 5 2024-09-10T15:58:50.952 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-10T15:58:50.953 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-10T15:58:50.963 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 18 20 40398 40378 8.76146 5.99609 0.0130237 0.00891016 2024-09-10T15:58:51.206 INFO:teuthology.orchestra.run.smithi115.stdout:133143986186 2024-09-10T15:58:51.227 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-10T15:58:51.228 INFO:teuthology.orchestra.run.smithi115.stdout:107374182415 2024-09-10T15:58:51.228 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-10T15:58:51.754 INFO:teuthology.orchestra.run.smithi115.stdout:133143986185 2024-09-10T15:58:51.763 INFO:teuthology.orchestra.run.smithi115.stdout:107374182413 2024-09-10T15:58:51.777 INFO:tasks.lost_unfound.ceph_manager:need seq 133143986186 got 133143986185 for osd.2 2024-09-10T15:58:51.787 INFO:tasks.lost_unfound.ceph_manager:need seq 107374182415 got 107374182413 for osd.0 2024-09-10T15:58:51.963 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 19 20 41747 41727 8.57766 5.26953 0.00687819 0.00910284 2024-09-10T15:58:52.777 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-10T15:58:52.789 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-10T15:58:52.963 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout:2024-09-10T15:58:52.962736+0000 min lat: 0.00225581 max lat: 0.0802208 avg lat: 0.00921465 2024-09-10T15:58:52.963 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-10T15:58:52.963 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 20 20 43402 43382 8.472 6.46484 0.00574581 0.00921465 2024-09-10T15:58:53.329 INFO:teuthology.orchestra.run.smithi115.stdout:133143986186 2024-09-10T15:58:53.353 INFO:tasks.lost_unfound.ceph_manager:need seq 133143986186 got 133143986186 for osd.2 2024-09-10T15:58:53.353 DEBUG:teuthology.parallel:result is None 2024-09-10T15:58:53.369 INFO:teuthology.orchestra.run.smithi115.stdout:107374182415 2024-09-10T15:58:53.391 INFO:tasks.lost_unfound.ceph_manager:need seq 107374182415 got 107374182415 for osd.0 2024-09-10T15:58:53.392 DEBUG:teuthology.parallel:result is None 2024-09-10T15:58:53.392 INFO:tasks.lost_unfound.ceph_manager:waiting for recovery to complete 2024-09-10T15:58:53.392 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:58:53.949 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T15:58:53.951 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T15:58:53.963 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 21 20 44767 44747 8.32246 5.33203 0.00974579 0.00937935 2024-09-10T15:58:53.969 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":78,"stamp":"2024-09-10T15:58:53.541530+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":166583160,"num_objects":40692,"num_object_clones":0,"num_object_copies":81384,"num_objects_missing_on_primary":18,"num_objects_missing":18,"num_objects_degraded":36,"num_objects_misplaced":0,"num_objects_unfound":5,"num_objects_dirty":40692,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":29895,"num_write_kb":119460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":40752,"ondisk_log_size":40752,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":48,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":530896,"kb_used_data":319024,"kb_used_omap":8,"kb_used_meta":211831,"kb_avail":188212784,"statfs":{"total":193273528320,"available":192729890816,"internally_reserved":0,"allocated":326680576,"data_stored":326209507,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8876,"internal_metadata":216915284},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,5,3,11,5],"upper_bound":32},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":81290897,"num_objects":19842,"num_object_clones":0,"num_object_copies":39684,"num_objects_missing_on_primary":-9,"num_objects_missing":-9,"num_objects_degraded":-18,"num_objects_misplaced":0,"num_objects_unfound":-22,"num_objects_dirty":19842,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9018,"num_write_kb":36132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002785"},"pg_stats":[{"pgid":"2.f","version":"36'2522","reported_seq":2574,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T15:58:51.168341+0000","last_change":"2024-09-10T15:58:24.769328+0000","last_active":"2024-09-10T15:58:51.168341+0000","last_peered":"2024-09-10T15:58:51.168341+0000","last_clean":"2024-09-10T15:58:51.168341+0000","last_became_active":"2024-09-10T15:58:24.769050+0000","last_became_peered":"2024-09-10T15:58:24.769050+0000","last_unstale":"2024-09-10T15:58:51.168341+0000","last_undegraded":"2024-09-10T15:58:51.168341+0000","last_fullsized":"2024-09-10T15:58:51.168341+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":2522,"log_dups_size":0,"ondisk_log_size":2522,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:08:36.167566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10330112,"num_objects":2522,"num_object_clones":0,"num_object_copies":5044,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2522,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2522,"num_write_kb":10088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"36'2590","reported_seq":2681,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T15:58:51.148185+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T15:58:51.148185+0000","last_peered":"2024-09-10T15:58:51.148185+0000","last_clean":"2024-09-10T15:58:51.148185+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T15:58:51.148185+0000","last_undegraded":"2024-09-10T15:58:51.148185+0000","last_fullsized":"2024-09-10T15:58:51.148185+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":2590,"log_dups_size":0,"ondisk_log_size":2590,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10585030,"num_objects":2586,"num_object_clones":0,"num_object_copies":5172,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2586,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":695,"num_write_kb":2780,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"36'2509","reported_seq":2612,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T15:58:51.155598+0000","last_change":"2024-09-10T15:58:45.933955+0000","last_active":"2024-09-10T15:58:51.155598+0000","last_peered":"2024-09-10T15:58:51.155598+0000","last_clean":"2024-09-10T15:58:51.155598+0000","last_became_active":"2024-09-10T15:58:25.767861+0000","last_became_peered":"2024-09-10T15:58:25.767861+0000","last_unstale":"2024-09-10T15:58:51.155598+0000","last_undegraded":"2024-09-10T15:58:51.155598+0000","last_fullsized":"2024-09-10T15:58:51.155598+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":2509,"log_dups_size":0,"ondisk_log_size":2509,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:22:24.003676+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10260480,"num_objects":2505,"num_object_clones":0,"num_object_copies":5010,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2505,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":513,"num_write_kb":2052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"36'2519","reported_seq":2580,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T15:58:51.146267+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T15:58:51.146267+0000","last_peered":"2024-09-10T15:58:51.146267+0000","last_clean":"2024-09-10T15:58:51.146267+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T15:58:51.146267+0000","last_undegraded":"2024-09-10T15:58:51.146267+0000","last_fullsized":"2024-09-10T15:58:51.146267+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":2519,"log_dups_size":0,"ondisk_log_size":2519,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10317824,"num_objects":2519,"num_object_clones":0,"num_object_copies":5038,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2519,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2519,"num_write_kb":10076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"36'2565","reported_seq":2624,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-10T15:58:51.161049+0000","last_change":"2024-09-10T15:58:24.766952+0000","last_active":"2024-09-10T15:58:51.161049+0000","last_peered":"2024-09-10T15:58:51.161049+0000","last_clean":"2024-09-10T15:57:51.217636+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T15:58:51.161049+0000","last_undegraded":"2024-09-10T15:58:24.759175+0000","last_fullsized":"2024-09-10T15:58:51.161049+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":2565,"log_dups_size":0,"ondisk_log_size":2565,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10478534,"num_objects":2560,"num_object_clones":0,"num_object_copies":5120,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":1,"num_objects_dirty":2560,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2563,"num_write_kb":10236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"","objects":1},{"shards":"0,2","objects":2559}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"36'2599","reported_seq":2657,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-10T15:58:51.175901+0000","last_change":"2024-09-10T15:58:24.768439+0000","last_active":"2024-09-10T15:58:51.175901+0000","last_peered":"2024-09-10T15:58:51.175901+0000","last_clean":"2024-09-10T15:57:52.065335+0000","last_became_active":"2024-09-10T15:58:24.768198+0000","last_became_peered":"2024-09-10T15:58:24.768198+0000","last_unstale":"2024-09-10T15:58:51.175901+0000","last_undegraded":"2024-09-10T15:58:24.758149+0000","last_fullsized":"2024-09-10T15:58:51.175901+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":2599,"log_dups_size":0,"ondisk_log_size":2599,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10625990,"num_objects":2596,"num_object_clones":0,"num_object_copies":5192,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":1,"num_objects_dirty":2596,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2597,"num_write_kb":10379,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"","objects":1},{"shards":"0,2","objects":2595}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"36'2493","reported_seq":2552,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-10T15:58:51.139986+0000","last_change":"2024-09-10T15:58:24.769216+0000","last_active":"2024-09-10T15:58:51.139986+0000","last_peered":"2024-09-10T15:58:51.139986+0000","last_clean":"2024-09-10T15:57:51.467206+0000","last_became_active":"2024-09-10T15:58:24.768972+0000","last_became_peered":"2024-09-10T15:58:24.768972+0000","last_unstale":"2024-09-10T15:58:51.139986+0000","last_undegraded":"2024-09-10T15:58:24.758006+0000","last_fullsized":"2024-09-10T15:58:51.139986+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":2493,"log_dups_size":0,"ondisk_log_size":2493,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10187718,"num_objects":2489,"num_object_clones":0,"num_object_copies":4978,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":2489,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2491,"num_write_kb":9952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"","objects":2},{"shards":"0,2","objects":2487}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"36'2536","reported_seq":2598,"reported_epoch":36,"state":"active+recovering","last_fresh":"2024-09-10T15:58:51.163836+0000","last_change":"2024-09-10T15:58:25.762598+0000","last_active":"2024-09-10T15:58:51.163836+0000","last_peered":"2024-09-10T15:58:51.163836+0000","last_clean":"2024-09-10T15:57:42.760798+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T15:58:51.163836+0000","last_undegraded":"2024-09-10T15:58:51.163836+0000","last_fullsized":"2024-09-10T15:58:51.163836+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":2536,"log_dups_size":0,"ondisk_log_size":2536,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10379747,"num_objects":2535,"num_object_clones":0,"num_object_copies":5070,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2535,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2535,"num_write_kb":10137,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2535}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T15:58:25.762375+0000","last_change":"2024-09-10T15:58:24.767197+0000","last_active":"2024-09-10T15:58:25.762375+0000","last_peered":"2024-09-10T15:58:25.762375+0000","last_clean":"2024-09-10T15:58:25.762375+0000","last_became_active":"2024-09-10T15:58:24.766691+0000","last_became_peered":"2024-09-10T15:58:24.766691+0000","last_unstale":"2024-09-10T15:58:25.762375+0000","last_undegraded":"2024-09-10T15:58:25.762375+0000","last_fullsized":"2024-09-10T15:58:25.762375+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:51:00.161051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"36'2483","reported_seq":2591,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T15:58:51.148806+0000","last_change":"2024-09-10T15:58:48.061941+0000","last_active":"2024-09-10T15:58:51.148806+0000","last_peered":"2024-09-10T15:58:51.148806+0000","last_clean":"2024-09-10T15:58:51.148806+0000","last_became_active":"2024-09-10T15:58:25.769634+0000","last_became_peered":"2024-09-10T15:58:25.769634+0000","last_unstale":"2024-09-10T15:58:51.148806+0000","last_undegraded":"2024-09-10T15:58:51.148806+0000","last_fullsized":"2024-09-10T15:58:51.148806+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":2483,"log_dups_size":0,"ondisk_log_size":2483,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:24:15.030848+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10142179,"num_objects":2477,"num_object_clones":0,"num_object_copies":4954,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2477,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":302,"num_write_kb":1208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":74,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T15:58:25.768681+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T15:58:25.768681+0000","last_peered":"2024-09-10T15:58:25.768681+0000","last_clean":"2024-09-10T15:58:25.768681+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T15:58:25.768681+0000","last_undegraded":"2024-09-10T15:58:25.768681+0000","last_fullsized":"2024-09-10T15:58:25.768681+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"36'2546","reported_seq":2596,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-10T15:58:51.131215+0000","last_change":"2024-09-10T15:58:24.768933+0000","last_active":"2024-09-10T15:58:51.131215+0000","last_peered":"2024-09-10T15:58:51.131215+0000","last_clean":"2024-09-10T15:57:52.118434+0000","last_became_active":"2024-09-10T15:58:24.768693+0000","last_became_peered":"2024-09-10T15:58:24.768693+0000","last_unstale":"2024-09-10T15:58:51.131215+0000","last_undegraded":"2024-09-10T15:58:24.758580+0000","last_fullsized":"2024-09-10T15:58:51.131215+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":2546,"log_dups_size":0,"ondisk_log_size":2546,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10396614,"num_objects":2540,"num_object_clones":0,"num_object_copies":5080,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2540,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2544,"num_write_kb":10156,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2540}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":67,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-10T15:58:25.762469+0000","last_change":"2024-09-10T15:58:24.768453+0000","last_active":"2024-09-10T15:58:25.762469+0000","last_peered":"2024-09-10T15:58:25.762469+0000","last_clean":"2024-09-10T15:58:03.029025+0000","last_became_active":"2024-09-10T15:58:24.766316+0000","last_became_peered":"2024-09-10T15:58:24.766316+0000","last_unstale":"2024-09-10T15:58:25.762469+0000","last_undegraded":"2024-09-10T15:58:24.757744+0000","last_fullsized":"2024-09-10T15:58:25.762469+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0"],"object_location_counts":[{"shards":"0","objects":1}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"36'2551","reported_seq":2654,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T15:58:51.147868+0000","last_change":"2024-09-10T15:58:49.040907+0000","last_active":"2024-09-10T15:58:51.147868+0000","last_peered":"2024-09-10T15:58:51.147868+0000","last_clean":"2024-09-10T15:58:51.147868+0000","last_became_active":"2024-09-10T15:58:25.769424+0000","last_became_peered":"2024-09-10T15:58:25.769424+0000","last_unstale":"2024-09-10T15:58:51.147868+0000","last_undegraded":"2024-09-10T15:58:51.147868+0000","last_fullsized":"2024-09-10T15:58:51.147868+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":2551,"log_dups_size":0,"ondisk_log_size":2551,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:52:01.617164+0000","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10428899,"num_objects":2547,"num_object_clones":0,"num_object_copies":5094,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2547,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":231,"num_write_kb":924,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":92,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T15:58:25.770150+0000","last_change":"2024-09-10T15:58:25.768489+0000","last_active":"2024-09-10T15:58:25.770150+0000","last_peered":"2024-09-10T15:58:25.770150+0000","last_clean":"2024-09-10T15:58:25.770150+0000","last_became_active":"2024-09-10T15:58:25.768117+0000","last_became_peered":"2024-09-10T15:58:25.768117+0000","last_unstale":"2024-09-10T15:58:25.770150+0000","last_undegraded":"2024-09-10T15:58:25.770150+0000","last_fullsized":"2024-09-10T15:58:25.770150+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:56:47.213564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"36'2591","reported_seq":2694,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T15:58:51.148428+0000","last_change":"2024-09-10T15:58:50.079160+0000","last_active":"2024-09-10T15:58:51.148428+0000","last_peered":"2024-09-10T15:58:51.148428+0000","last_clean":"2024-09-10T15:58:51.148428+0000","last_became_active":"2024-09-10T15:58:25.769762+0000","last_became_peered":"2024-09-10T15:58:25.769762+0000","last_unstale":"2024-09-10T15:58:51.148428+0000","last_undegraded":"2024-09-10T15:58:51.148428+0000","last_fullsized":"2024-09-10T15:58:51.148428+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":2591,"log_dups_size":0,"ondisk_log_size":2591,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:50:48.841755+0000","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10588160,"num_objects":2585,"num_object_clones":0,"num_object_copies":5170,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2585,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":145,"num_write_kb":580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"36'2569","reported_seq":2627,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-10T15:58:51.169066+0000","last_change":"2024-09-10T15:58:24.768825+0000","last_active":"2024-09-10T15:58:51.169066+0000","last_peered":"2024-09-10T15:58:51.169066+0000","last_clean":"2024-09-10T15:57:45.763054+0000","last_became_active":"2024-09-10T15:58:24.768609+0000","last_became_peered":"2024-09-10T15:58:24.768609+0000","last_unstale":"2024-09-10T15:58:51.169066+0000","last_undegraded":"2024-09-10T15:58:24.757704+0000","last_fullsized":"2024-09-10T15:58:51.169066+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":2569,"log_dups_size":0,"ondisk_log_size":2569,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10514915,"num_objects":2568,"num_object_clones":0,"num_object_copies":5136,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2568,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2568,"num_write_kb":10269,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2568}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T15:58:25.762555+0000","last_change":"2024-09-10T15:58:24.767677+0000","last_active":"2024-09-10T15:58:25.762555+0000","last_peered":"2024-09-10T15:58:25.762555+0000","last_clean":"2024-09-10T15:58:25.762555+0000","last_became_active":"2024-09-10T15:58:24.767409+0000","last_became_peered":"2024-09-10T15:58:24.767409+0000","last_unstale":"2024-09-10T15:58:25.762555+0000","last_undegraded":"2024-09-10T15:58:25.762555+0000","last_fullsized":"2024-09-10T15:58:25.762555+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:35:35.763396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"36'2495","reported_seq":2552,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-10T15:58:51.169398+0000","last_change":"2024-09-10T15:58:24.769497+0000","last_active":"2024-09-10T15:58:51.169398+0000","last_peered":"2024-09-10T15:58:51.169398+0000","last_clean":"2024-09-10T15:57:52.013178+0000","last_became_active":"2024-09-10T15:58:24.769268+0000","last_became_peered":"2024-09-10T15:58:24.769268+0000","last_unstale":"2024-09-10T15:58:51.169398+0000","last_undegraded":"2024-09-10T15:58:24.759230+0000","last_fullsized":"2024-09-10T15:58:51.169398+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":2495,"log_dups_size":0,"ondisk_log_size":2495,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10188201,"num_objects":2490,"num_object_clones":0,"num_object_copies":4980,"num_objects_missing_on_primary":3,"num_objects_missing":3,"num_objects_degraded":6,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2490,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2492,"num_write_kb":9952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2490}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":74,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T15:58:25.769812+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T15:58:25.769812+0000","last_peered":"2024-09-10T15:58:25.769812+0000","last_clean":"2024-09-10T15:58:25.769812+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T15:58:25.769812+0000","last_undegraded":"2024-09-10T15:58:25.769812+0000","last_fullsized":"2024-09-10T15:58:25.769812+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"36'2655","reported_seq":2712,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-10T15:58:51.150001+0000","last_change":"2024-09-10T15:58:24.769323+0000","last_active":"2024-09-10T15:58:51.150001+0000","last_peered":"2024-09-10T15:58:51.150001+0000","last_clean":"2024-09-10T15:57:45.763338+0000","last_became_active":"2024-09-10T15:58:24.769074+0000","last_became_peered":"2024-09-10T15:58:24.769074+0000","last_unstale":"2024-09-10T15:58:51.150001+0000","last_undegraded":"2024-09-10T15:58:24.759853+0000","last_fullsized":"2024-09-10T15:58:51.150001+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":2655,"log_dups_size":0,"ondisk_log_size":2655,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10859462,"num_objects":2653,"num_object_clones":0,"num_object_copies":5306,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2653,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2653,"num_write_kb":10606,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2653}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T15:58:25.762992+0000","last_change":"2024-09-10T15:58:24.767867+0000","last_active":"2024-09-10T15:58:25.762992+0000","last_peered":"2024-09-10T15:58:25.762992+0000","last_clean":"2024-09-10T15:58:25.762992+0000","last_became_active":"2024-09-10T15:58:24.767615+0000","last_became_peered":"2024-09-10T15:58:24.767615+0000","last_unstale":"2024-09-10T15:58:25.762992+0000","last_undegraded":"2024-09-10T15:58:25.762992+0000","last_fullsized":"2024-09-10T15:58:25.762992+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:11:00.908668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":87,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T15:58:25.770511+0000","last_change":"2024-09-10T15:58:25.768847+0000","last_active":"2024-09-10T15:58:25.770511+0000","last_peered":"2024-09-10T15:58:25.770511+0000","last_clean":"2024-09-10T15:58:25.770511+0000","last_became_active":"2024-09-10T15:58:25.768040+0000","last_became_peered":"2024-09-10T15:58:25.768040+0000","last_unstale":"2024-09-10T15:58:25.770511+0000","last_undegraded":"2024-09-10T15:58:25.770511+0000","last_fullsized":"2024-09-10T15:58:25.770511+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:39:30.402776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"36'2526","reported_seq":2583,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-10T15:58:51.131128+0000","last_change":"2024-09-10T15:58:24.769859+0000","last_active":"2024-09-10T15:58:51.131128+0000","last_peered":"2024-09-10T15:58:51.131128+0000","last_clean":"2024-09-10T15:57:51.765160+0000","last_became_active":"2024-09-10T15:58:24.769616+0000","last_became_peered":"2024-09-10T15:58:24.769616+0000","last_unstale":"2024-09-10T15:58:51.131128+0000","last_undegraded":"2024-09-10T15:58:24.756607+0000","last_fullsized":"2024-09-10T15:58:51.131128+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":2526,"log_dups_size":0,"ondisk_log_size":2526,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10299276,"num_objects":2518,"num_object_clones":0,"num_object_copies":5036,"num_objects_missing_on_primary":3,"num_objects_missing":3,"num_objects_degraded":7,"num_objects_misplaced":0,"num_objects_unfound":1,"num_objects_dirty":2518,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2523,"num_write_kb":10063,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"","objects":1},{"shards":"0","objects":1},{"shards":"0,2","objects":2516}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":166583141,"num_objects":40690,"num_object_clones":0,"num_object_copies":81380,"num_objects_missing_on_primary":18,"num_objects_missing":18,"num_objects_degraded":35,"num_objects_misplaced":0,"num_objects_unfound":5,"num_objects_dirty":40690,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":29893,"num_write_kb":119458,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":333475840,"data_stored":333324094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":40749,"ondisk_log_size":40749,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986186,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":265560,"kb_used_data":157512,"kb_used_omap":4,"kb_used_meta":108027,"kb_avail":94106280,"statfs":{"total":96636764160,"available":96364830720,"internally_reserved":0,"allocated":161292288,"data_stored":161069631,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":110620330},"hb_peers":[0],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,3,3,2],"upper_bound":32},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182415,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":265336,"kb_used_data":161512,"kb_used_omap":4,"kb_used_meta":103803,"kb_avail":94106504,"statfs":{"total":96636764160,"available":96365060096,"internally_reserved":0,"allocated":165388288,"data_stored":165139876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":106294954},"hb_peers":[2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,5,0,8,3],"upper_bound":32},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":166711296,"data_stored":166675166,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":13524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":166649856,"data_stored":166635404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:58:53.972 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:58:54.492 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T15:58:54.493 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T15:58:54.512 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":78,"stamp":"2024-09-10T15:58:53.541530+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":166583160,"num_objects":40692,"num_object_clones":0,"num_object_copies":81384,"num_objects_missing_on_primary":18,"num_objects_missing":18,"num_objects_degraded":36,"num_objects_misplaced":0,"num_objects_unfound":5,"num_objects_dirty":40692,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":29895,"num_write_kb":119460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":40752,"ondisk_log_size":40752,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":48,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":530896,"kb_used_data":319024,"kb_used_omap":8,"kb_used_meta":211831,"kb_avail":188212784,"statfs":{"total":193273528320,"available":192729890816,"internally_reserved":0,"allocated":326680576,"data_stored":326209507,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8876,"internal_metadata":216915284},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,5,3,11,5],"upper_bound":32},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":81290897,"num_objects":19842,"num_object_clones":0,"num_object_copies":39684,"num_objects_missing_on_primary":-9,"num_objects_missing":-9,"num_objects_degraded":-18,"num_objects_misplaced":0,"num_objects_unfound":-22,"num_objects_dirty":19842,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9018,"num_write_kb":36132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002785"},"pg_stats":[{"pgid":"2.f","version":"36'2522","reported_seq":2574,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T15:58:51.168341+0000","last_change":"2024-09-10T15:58:24.769328+0000","last_active":"2024-09-10T15:58:51.168341+0000","last_peered":"2024-09-10T15:58:51.168341+0000","last_clean":"2024-09-10T15:58:51.168341+0000","last_became_active":"2024-09-10T15:58:24.769050+0000","last_became_peered":"2024-09-10T15:58:24.769050+0000","last_unstale":"2024-09-10T15:58:51.168341+0000","last_undegraded":"2024-09-10T15:58:51.168341+0000","last_fullsized":"2024-09-10T15:58:51.168341+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":2522,"log_dups_size":0,"ondisk_log_size":2522,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:08:36.167566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10330112,"num_objects":2522,"num_object_clones":0,"num_object_copies":5044,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2522,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2522,"num_write_kb":10088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"36'2590","reported_seq":2681,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T15:58:51.148185+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T15:58:51.148185+0000","last_peered":"2024-09-10T15:58:51.148185+0000","last_clean":"2024-09-10T15:58:51.148185+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T15:58:51.148185+0000","last_undegraded":"2024-09-10T15:58:51.148185+0000","last_fullsized":"2024-09-10T15:58:51.148185+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":2590,"log_dups_size":0,"ondisk_log_size":2590,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10585030,"num_objects":2586,"num_object_clones":0,"num_object_copies":5172,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2586,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":695,"num_write_kb":2780,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"36'2509","reported_seq":2612,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T15:58:51.155598+0000","last_change":"2024-09-10T15:58:45.933955+0000","last_active":"2024-09-10T15:58:51.155598+0000","last_peered":"2024-09-10T15:58:51.155598+0000","last_clean":"2024-09-10T15:58:51.155598+0000","last_became_active":"2024-09-10T15:58:25.767861+0000","last_became_peered":"2024-09-10T15:58:25.767861+0000","last_unstale":"2024-09-10T15:58:51.155598+0000","last_undegraded":"2024-09-10T15:58:51.155598+0000","last_fullsized":"2024-09-10T15:58:51.155598+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":2509,"log_dups_size":0,"ondisk_log_size":2509,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:22:24.003676+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10260480,"num_objects":2505,"num_object_clones":0,"num_object_copies":5010,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2505,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":513,"num_write_kb":2052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"36'2519","reported_seq":2580,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T15:58:51.146267+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T15:58:51.146267+0000","last_peered":"2024-09-10T15:58:51.146267+0000","last_clean":"2024-09-10T15:58:51.146267+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T15:58:51.146267+0000","last_undegraded":"2024-09-10T15:58:51.146267+0000","last_fullsized":"2024-09-10T15:58:51.146267+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":2519,"log_dups_size":0,"ondisk_log_size":2519,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10317824,"num_objects":2519,"num_object_clones":0,"num_object_copies":5038,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2519,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2519,"num_write_kb":10076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"36'2565","reported_seq":2624,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-10T15:58:51.161049+0000","last_change":"2024-09-10T15:58:24.766952+0000","last_active":"2024-09-10T15:58:51.161049+0000","last_peered":"2024-09-10T15:58:51.161049+0000","last_clean":"2024-09-10T15:57:51.217636+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T15:58:51.161049+0000","last_undegraded":"2024-09-10T15:58:24.759175+0000","last_fullsized":"2024-09-10T15:58:51.161049+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":2565,"log_dups_size":0,"ondisk_log_size":2565,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10478534,"num_objects":2560,"num_object_clones":0,"num_object_copies":5120,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":1,"num_objects_dirty":2560,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2563,"num_write_kb":10236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"","objects":1},{"shards":"0,2","objects":2559}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"36'2599","reported_seq":2657,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-10T15:58:51.175901+0000","last_change":"2024-09-10T15:58:24.768439+0000","last_active":"2024-09-10T15:58:51.175901+0000","last_peered":"2024-09-10T15:58:51.175901+0000","last_clean":"2024-09-10T15:57:52.065335+0000","last_became_active":"2024-09-10T15:58:24.768198+0000","last_became_peered":"2024-09-10T15:58:24.768198+0000","last_unstale":"2024-09-10T15:58:51.175901+0000","last_undegraded":"2024-09-10T15:58:24.758149+0000","last_fullsized":"2024-09-10T15:58:51.175901+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":2599,"log_dups_size":0,"ondisk_log_size":2599,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10625990,"num_objects":2596,"num_object_clones":0,"num_object_copies":5192,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":1,"num_objects_dirty":2596,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2597,"num_write_kb":10379,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"","objects":1},{"shards":"0,2","objects":2595}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"36'2493","reported_seq":2552,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-10T15:58:51.139986+0000","last_change":"2024-09-10T15:58:24.769216+0000","last_active":"2024-09-10T15:58:51.139986+0000","last_peered":"2024-09-10T15:58:51.139986+0000","last_clean":"2024-09-10T15:57:51.467206+0000","last_became_active":"2024-09-10T15:58:24.768972+0000","last_became_peered":"2024-09-10T15:58:24.768972+0000","last_unstale":"2024-09-10T15:58:51.139986+0000","last_undegraded":"2024-09-10T15:58:24.758006+0000","last_fullsized":"2024-09-10T15:58:51.139986+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":2493,"log_dups_size":0,"ondisk_log_size":2493,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10187718,"num_objects":2489,"num_object_clones":0,"num_object_copies":4978,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":2489,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2491,"num_write_kb":9952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"","objects":2},{"shards":"0,2","objects":2487}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"36'2536","reported_seq":2598,"reported_epoch":36,"state":"active+recovering","last_fresh":"2024-09-10T15:58:51.163836+0000","last_change":"2024-09-10T15:58:25.762598+0000","last_active":"2024-09-10T15:58:51.163836+0000","last_peered":"2024-09-10T15:58:51.163836+0000","last_clean":"2024-09-10T15:57:42.760798+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T15:58:51.163836+0000","last_undegraded":"2024-09-10T15:58:51.163836+0000","last_fullsized":"2024-09-10T15:58:51.163836+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":2536,"log_dups_size":0,"ondisk_log_size":2536,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10379747,"num_objects":2535,"num_object_clones":0,"num_object_copies":5070,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2535,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2535,"num_write_kb":10137,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2535}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T15:58:25.762375+0000","last_change":"2024-09-10T15:58:24.767197+0000","last_active":"2024-09-10T15:58:25.762375+0000","last_peered":"2024-09-10T15:58:25.762375+0000","last_clean":"2024-09-10T15:58:25.762375+0000","last_became_active":"2024-09-10T15:58:24.766691+0000","last_became_peered":"2024-09-10T15:58:24.766691+0000","last_unstale":"2024-09-10T15:58:25.762375+0000","last_undegraded":"2024-09-10T15:58:25.762375+0000","last_fullsized":"2024-09-10T15:58:25.762375+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:51:00.161051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"36'2483","reported_seq":2591,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T15:58:51.148806+0000","last_change":"2024-09-10T15:58:48.061941+0000","last_active":"2024-09-10T15:58:51.148806+0000","last_peered":"2024-09-10T15:58:51.148806+0000","last_clean":"2024-09-10T15:58:51.148806+0000","last_became_active":"2024-09-10T15:58:25.769634+0000","last_became_peered":"2024-09-10T15:58:25.769634+0000","last_unstale":"2024-09-10T15:58:51.148806+0000","last_undegraded":"2024-09-10T15:58:51.148806+0000","last_fullsized":"2024-09-10T15:58:51.148806+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":2483,"log_dups_size":0,"ondisk_log_size":2483,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:24:15.030848+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10142179,"num_objects":2477,"num_object_clones":0,"num_object_copies":4954,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2477,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":302,"num_write_kb":1208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":74,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T15:58:25.768681+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T15:58:25.768681+0000","last_peered":"2024-09-10T15:58:25.768681+0000","last_clean":"2024-09-10T15:58:25.768681+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T15:58:25.768681+0000","last_undegraded":"2024-09-10T15:58:25.768681+0000","last_fullsized":"2024-09-10T15:58:25.768681+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"36'2546","reported_seq":2596,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-10T15:58:51.131215+0000","last_change":"2024-09-10T15:58:24.768933+0000","last_active":"2024-09-10T15:58:51.131215+0000","last_peered":"2024-09-10T15:58:51.131215+0000","last_clean":"2024-09-10T15:57:52.118434+0000","last_became_active":"2024-09-10T15:58:24.768693+0000","last_became_peered":"2024-09-10T15:58:24.768693+0000","last_unstale":"2024-09-10T15:58:51.131215+0000","last_undegraded":"2024-09-10T15:58:24.758580+0000","last_fullsized":"2024-09-10T15:58:51.131215+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":2546,"log_dups_size":0,"ondisk_log_size":2546,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10396614,"num_objects":2540,"num_object_clones":0,"num_object_copies":5080,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2540,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2544,"num_write_kb":10156,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2540}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":67,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-10T15:58:25.762469+0000","last_change":"2024-09-10T15:58:24.768453+0000","last_active":"2024-09-10T15:58:25.762469+0000","last_peered":"2024-09-10T15:58:25.762469+0000","last_clean":"2024-09-10T15:58:03.029025+0000","last_became_active":"2024-09-10T15:58:24.766316+0000","last_became_peered":"2024-09-10T15:58:24.766316+0000","last_unstale":"2024-09-10T15:58:25.762469+0000","last_undegraded":"2024-09-10T15:58:24.757744+0000","last_fullsized":"2024-09-10T15:58:25.762469+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0"],"object_location_counts":[{"shards":"0","objects":1}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"36'2551","reported_seq":2654,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T15:58:51.147868+0000","last_change":"2024-09-10T15:58:49.040907+0000","last_active":"2024-09-10T15:58:51.147868+0000","last_peered":"2024-09-10T15:58:51.147868+0000","last_clean":"2024-09-10T15:58:51.147868+0000","last_became_active":"2024-09-10T15:58:25.769424+0000","last_became_peered":"2024-09-10T15:58:25.769424+0000","last_unstale":"2024-09-10T15:58:51.147868+0000","last_undegraded":"2024-09-10T15:58:51.147868+0000","last_fullsized":"2024-09-10T15:58:51.147868+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":2551,"log_dups_size":0,"ondisk_log_size":2551,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:52:01.617164+0000","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10428899,"num_objects":2547,"num_object_clones":0,"num_object_copies":5094,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2547,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":231,"num_write_kb":924,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":92,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T15:58:25.770150+0000","last_change":"2024-09-10T15:58:25.768489+0000","last_active":"2024-09-10T15:58:25.770150+0000","last_peered":"2024-09-10T15:58:25.770150+0000","last_clean":"2024-09-10T15:58:25.770150+0000","last_became_active":"2024-09-10T15:58:25.768117+0000","last_became_peered":"2024-09-10T15:58:25.768117+0000","last_unstale":"2024-09-10T15:58:25.770150+0000","last_undegraded":"2024-09-10T15:58:25.770150+0000","last_fullsized":"2024-09-10T15:58:25.770150+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:56:47.213564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"36'2591","reported_seq":2694,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T15:58:51.148428+0000","last_change":"2024-09-10T15:58:50.079160+0000","last_active":"2024-09-10T15:58:51.148428+0000","last_peered":"2024-09-10T15:58:51.148428+0000","last_clean":"2024-09-10T15:58:51.148428+0000","last_became_active":"2024-09-10T15:58:25.769762+0000","last_became_peered":"2024-09-10T15:58:25.769762+0000","last_unstale":"2024-09-10T15:58:51.148428+0000","last_undegraded":"2024-09-10T15:58:51.148428+0000","last_fullsized":"2024-09-10T15:58:51.148428+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":2591,"log_dups_size":0,"ondisk_log_size":2591,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:50:48.841755+0000","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10588160,"num_objects":2585,"num_object_clones":0,"num_object_copies":5170,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2585,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":145,"num_write_kb":580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"36'2569","reported_seq":2627,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-10T15:58:51.169066+0000","last_change":"2024-09-10T15:58:24.768825+0000","last_active":"2024-09-10T15:58:51.169066+0000","last_peered":"2024-09-10T15:58:51.169066+0000","last_clean":"2024-09-10T15:57:45.763054+0000","last_became_active":"2024-09-10T15:58:24.768609+0000","last_became_peered":"2024-09-10T15:58:24.768609+0000","last_unstale":"2024-09-10T15:58:51.169066+0000","last_undegraded":"2024-09-10T15:58:24.757704+0000","last_fullsized":"2024-09-10T15:58:51.169066+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":2569,"log_dups_size":0,"ondisk_log_size":2569,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10514915,"num_objects":2568,"num_object_clones":0,"num_object_copies":5136,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2568,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2568,"num_write_kb":10269,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2568}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T15:58:25.762555+0000","last_change":"2024-09-10T15:58:24.767677+0000","last_active":"2024-09-10T15:58:25.762555+0000","last_peered":"2024-09-10T15:58:25.762555+0000","last_clean":"2024-09-10T15:58:25.762555+0000","last_became_active":"2024-09-10T15:58:24.767409+0000","last_became_peered":"2024-09-10T15:58:24.767409+0000","last_unstale":"2024-09-10T15:58:25.762555+0000","last_undegraded":"2024-09-10T15:58:25.762555+0000","last_fullsized":"2024-09-10T15:58:25.762555+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:35:35.763396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"36'2495","reported_seq":2552,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-10T15:58:51.169398+0000","last_change":"2024-09-10T15:58:24.769497+0000","last_active":"2024-09-10T15:58:51.169398+0000","last_peered":"2024-09-10T15:58:51.169398+0000","last_clean":"2024-09-10T15:57:52.013178+0000","last_became_active":"2024-09-10T15:58:24.769268+0000","last_became_peered":"2024-09-10T15:58:24.769268+0000","last_unstale":"2024-09-10T15:58:51.169398+0000","last_undegraded":"2024-09-10T15:58:24.759230+0000","last_fullsized":"2024-09-10T15:58:51.169398+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":2495,"log_dups_size":0,"ondisk_log_size":2495,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10188201,"num_objects":2490,"num_object_clones":0,"num_object_copies":4980,"num_objects_missing_on_primary":3,"num_objects_missing":3,"num_objects_degraded":6,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2490,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2492,"num_write_kb":9952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2490}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":74,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T15:58:25.769812+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T15:58:25.769812+0000","last_peered":"2024-09-10T15:58:25.769812+0000","last_clean":"2024-09-10T15:58:25.769812+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T15:58:25.769812+0000","last_undegraded":"2024-09-10T15:58:25.769812+0000","last_fullsized":"2024-09-10T15:58:25.769812+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"36'2655","reported_seq":2712,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-10T15:58:51.150001+0000","last_change":"2024-09-10T15:58:24.769323+0000","last_active":"2024-09-10T15:58:51.150001+0000","last_peered":"2024-09-10T15:58:51.150001+0000","last_clean":"2024-09-10T15:57:45.763338+0000","last_became_active":"2024-09-10T15:58:24.769074+0000","last_became_peered":"2024-09-10T15:58:24.769074+0000","last_unstale":"2024-09-10T15:58:51.150001+0000","last_undegraded":"2024-09-10T15:58:24.759853+0000","last_fullsized":"2024-09-10T15:58:51.150001+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":2655,"log_dups_size":0,"ondisk_log_size":2655,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10859462,"num_objects":2653,"num_object_clones":0,"num_object_copies":5306,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2653,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2653,"num_write_kb":10606,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2653}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T15:58:25.762992+0000","last_change":"2024-09-10T15:58:24.767867+0000","last_active":"2024-09-10T15:58:25.762992+0000","last_peered":"2024-09-10T15:58:25.762992+0000","last_clean":"2024-09-10T15:58:25.762992+0000","last_became_active":"2024-09-10T15:58:24.767615+0000","last_became_peered":"2024-09-10T15:58:24.767615+0000","last_unstale":"2024-09-10T15:58:25.762992+0000","last_undegraded":"2024-09-10T15:58:25.762992+0000","last_fullsized":"2024-09-10T15:58:25.762992+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:11:00.908668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":87,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T15:58:25.770511+0000","last_change":"2024-09-10T15:58:25.768847+0000","last_active":"2024-09-10T15:58:25.770511+0000","last_peered":"2024-09-10T15:58:25.770511+0000","last_clean":"2024-09-10T15:58:25.770511+0000","last_became_active":"2024-09-10T15:58:25.768040+0000","last_became_peered":"2024-09-10T15:58:25.768040+0000","last_unstale":"2024-09-10T15:58:25.770511+0000","last_undegraded":"2024-09-10T15:58:25.770511+0000","last_fullsized":"2024-09-10T15:58:25.770511+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:39:30.402776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"36'2526","reported_seq":2583,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-10T15:58:51.131128+0000","last_change":"2024-09-10T15:58:24.769859+0000","last_active":"2024-09-10T15:58:51.131128+0000","last_peered":"2024-09-10T15:58:51.131128+0000","last_clean":"2024-09-10T15:57:51.765160+0000","last_became_active":"2024-09-10T15:58:24.769616+0000","last_became_peered":"2024-09-10T15:58:24.769616+0000","last_unstale":"2024-09-10T15:58:51.131128+0000","last_undegraded":"2024-09-10T15:58:24.756607+0000","last_fullsized":"2024-09-10T15:58:51.131128+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":2526,"log_dups_size":0,"ondisk_log_size":2526,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10299276,"num_objects":2518,"num_object_clones":0,"num_object_copies":5036,"num_objects_missing_on_primary":3,"num_objects_missing":3,"num_objects_degraded":7,"num_objects_misplaced":0,"num_objects_unfound":1,"num_objects_dirty":2518,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2523,"num_write_kb":10063,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"","objects":1},{"shards":"0","objects":1},{"shards":"0,2","objects":2516}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":166583141,"num_objects":40690,"num_object_clones":0,"num_object_copies":81380,"num_objects_missing_on_primary":18,"num_objects_missing":18,"num_objects_degraded":35,"num_objects_misplaced":0,"num_objects_unfound":5,"num_objects_dirty":40690,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":29893,"num_write_kb":119458,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":333475840,"data_stored":333324094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":40749,"ondisk_log_size":40749,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986186,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":265560,"kb_used_data":157512,"kb_used_omap":4,"kb_used_meta":108027,"kb_avail":94106280,"statfs":{"total":96636764160,"available":96364830720,"internally_reserved":0,"allocated":161292288,"data_stored":161069631,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":110620330},"hb_peers":[0],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,3,3,2],"upper_bound":32},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182415,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":265336,"kb_used_data":161512,"kb_used_omap":4,"kb_used_meta":103803,"kb_avail":94106504,"statfs":{"total":96636764160,"available":96365060096,"internally_reserved":0,"allocated":165388288,"data_stored":165139876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":106294954},"hb_peers":[2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,5,0,8,3],"upper_bound":32},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":166711296,"data_stored":166675166,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":13524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":166649856,"data_stored":166635404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:58:54.514 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:58:54.963 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 22 20 46214 46194 8.20107 5.65234 0.0150409 0.00952042 2024-09-10T15:58:55.040 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T15:58:55.042 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T15:58:55.061 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":78,"stamp":"2024-09-10T15:58:53.541530+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":166583160,"num_objects":40692,"num_object_clones":0,"num_object_copies":81384,"num_objects_missing_on_primary":18,"num_objects_missing":18,"num_objects_degraded":36,"num_objects_misplaced":0,"num_objects_unfound":5,"num_objects_dirty":40692,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":29895,"num_write_kb":119460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":40752,"ondisk_log_size":40752,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":48,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":530896,"kb_used_data":319024,"kb_used_omap":8,"kb_used_meta":211831,"kb_avail":188212784,"statfs":{"total":193273528320,"available":192729890816,"internally_reserved":0,"allocated":326680576,"data_stored":326209507,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8876,"internal_metadata":216915284},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,5,3,11,5],"upper_bound":32},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":81290897,"num_objects":19842,"num_object_clones":0,"num_object_copies":39684,"num_objects_missing_on_primary":-9,"num_objects_missing":-9,"num_objects_degraded":-18,"num_objects_misplaced":0,"num_objects_unfound":-22,"num_objects_dirty":19842,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9018,"num_write_kb":36132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002785"},"pg_stats":[{"pgid":"2.f","version":"36'2522","reported_seq":2574,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T15:58:51.168341+0000","last_change":"2024-09-10T15:58:24.769328+0000","last_active":"2024-09-10T15:58:51.168341+0000","last_peered":"2024-09-10T15:58:51.168341+0000","last_clean":"2024-09-10T15:58:51.168341+0000","last_became_active":"2024-09-10T15:58:24.769050+0000","last_became_peered":"2024-09-10T15:58:24.769050+0000","last_unstale":"2024-09-10T15:58:51.168341+0000","last_undegraded":"2024-09-10T15:58:51.168341+0000","last_fullsized":"2024-09-10T15:58:51.168341+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":2522,"log_dups_size":0,"ondisk_log_size":2522,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:08:36.167566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10330112,"num_objects":2522,"num_object_clones":0,"num_object_copies":5044,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2522,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2522,"num_write_kb":10088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"36'2590","reported_seq":2681,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T15:58:51.148185+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T15:58:51.148185+0000","last_peered":"2024-09-10T15:58:51.148185+0000","last_clean":"2024-09-10T15:58:51.148185+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T15:58:51.148185+0000","last_undegraded":"2024-09-10T15:58:51.148185+0000","last_fullsized":"2024-09-10T15:58:51.148185+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":2590,"log_dups_size":0,"ondisk_log_size":2590,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10585030,"num_objects":2586,"num_object_clones":0,"num_object_copies":5172,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2586,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":695,"num_write_kb":2780,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"36'2509","reported_seq":2612,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T15:58:51.155598+0000","last_change":"2024-09-10T15:58:45.933955+0000","last_active":"2024-09-10T15:58:51.155598+0000","last_peered":"2024-09-10T15:58:51.155598+0000","last_clean":"2024-09-10T15:58:51.155598+0000","last_became_active":"2024-09-10T15:58:25.767861+0000","last_became_peered":"2024-09-10T15:58:25.767861+0000","last_unstale":"2024-09-10T15:58:51.155598+0000","last_undegraded":"2024-09-10T15:58:51.155598+0000","last_fullsized":"2024-09-10T15:58:51.155598+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":2509,"log_dups_size":0,"ondisk_log_size":2509,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:22:24.003676+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10260480,"num_objects":2505,"num_object_clones":0,"num_object_copies":5010,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2505,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":513,"num_write_kb":2052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"36'2519","reported_seq":2580,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T15:58:51.146267+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T15:58:51.146267+0000","last_peered":"2024-09-10T15:58:51.146267+0000","last_clean":"2024-09-10T15:58:51.146267+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T15:58:51.146267+0000","last_undegraded":"2024-09-10T15:58:51.146267+0000","last_fullsized":"2024-09-10T15:58:51.146267+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":2519,"log_dups_size":0,"ondisk_log_size":2519,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10317824,"num_objects":2519,"num_object_clones":0,"num_object_copies":5038,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2519,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2519,"num_write_kb":10076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"36'2565","reported_seq":2624,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-10T15:58:51.161049+0000","last_change":"2024-09-10T15:58:24.766952+0000","last_active":"2024-09-10T15:58:51.161049+0000","last_peered":"2024-09-10T15:58:51.161049+0000","last_clean":"2024-09-10T15:57:51.217636+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T15:58:51.161049+0000","last_undegraded":"2024-09-10T15:58:24.759175+0000","last_fullsized":"2024-09-10T15:58:51.161049+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":2565,"log_dups_size":0,"ondisk_log_size":2565,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10478534,"num_objects":2560,"num_object_clones":0,"num_object_copies":5120,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":1,"num_objects_dirty":2560,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2563,"num_write_kb":10236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"","objects":1},{"shards":"0,2","objects":2559}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"36'2599","reported_seq":2657,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-10T15:58:51.175901+0000","last_change":"2024-09-10T15:58:24.768439+0000","last_active":"2024-09-10T15:58:51.175901+0000","last_peered":"2024-09-10T15:58:51.175901+0000","last_clean":"2024-09-10T15:57:52.065335+0000","last_became_active":"2024-09-10T15:58:24.768198+0000","last_became_peered":"2024-09-10T15:58:24.768198+0000","last_unstale":"2024-09-10T15:58:51.175901+0000","last_undegraded":"2024-09-10T15:58:24.758149+0000","last_fullsized":"2024-09-10T15:58:51.175901+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":2599,"log_dups_size":0,"ondisk_log_size":2599,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10625990,"num_objects":2596,"num_object_clones":0,"num_object_copies":5192,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":1,"num_objects_dirty":2596,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2597,"num_write_kb":10379,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"","objects":1},{"shards":"0,2","objects":2595}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"36'2493","reported_seq":2552,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-10T15:58:51.139986+0000","last_change":"2024-09-10T15:58:24.769216+0000","last_active":"2024-09-10T15:58:51.139986+0000","last_peered":"2024-09-10T15:58:51.139986+0000","last_clean":"2024-09-10T15:57:51.467206+0000","last_became_active":"2024-09-10T15:58:24.768972+0000","last_became_peered":"2024-09-10T15:58:24.768972+0000","last_unstale":"2024-09-10T15:58:51.139986+0000","last_undegraded":"2024-09-10T15:58:24.758006+0000","last_fullsized":"2024-09-10T15:58:51.139986+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":2493,"log_dups_size":0,"ondisk_log_size":2493,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10187718,"num_objects":2489,"num_object_clones":0,"num_object_copies":4978,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":2489,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2491,"num_write_kb":9952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"","objects":2},{"shards":"0,2","objects":2487}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"36'2536","reported_seq":2598,"reported_epoch":36,"state":"active+recovering","last_fresh":"2024-09-10T15:58:51.163836+0000","last_change":"2024-09-10T15:58:25.762598+0000","last_active":"2024-09-10T15:58:51.163836+0000","last_peered":"2024-09-10T15:58:51.163836+0000","last_clean":"2024-09-10T15:57:42.760798+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T15:58:51.163836+0000","last_undegraded":"2024-09-10T15:58:51.163836+0000","last_fullsized":"2024-09-10T15:58:51.163836+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":2536,"log_dups_size":0,"ondisk_log_size":2536,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10379747,"num_objects":2535,"num_object_clones":0,"num_object_copies":5070,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2535,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2535,"num_write_kb":10137,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2535}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T15:58:25.762375+0000","last_change":"2024-09-10T15:58:24.767197+0000","last_active":"2024-09-10T15:58:25.762375+0000","last_peered":"2024-09-10T15:58:25.762375+0000","last_clean":"2024-09-10T15:58:25.762375+0000","last_became_active":"2024-09-10T15:58:24.766691+0000","last_became_peered":"2024-09-10T15:58:24.766691+0000","last_unstale":"2024-09-10T15:58:25.762375+0000","last_undegraded":"2024-09-10T15:58:25.762375+0000","last_fullsized":"2024-09-10T15:58:25.762375+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:51:00.161051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"36'2483","reported_seq":2591,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T15:58:51.148806+0000","last_change":"2024-09-10T15:58:48.061941+0000","last_active":"2024-09-10T15:58:51.148806+0000","last_peered":"2024-09-10T15:58:51.148806+0000","last_clean":"2024-09-10T15:58:51.148806+0000","last_became_active":"2024-09-10T15:58:25.769634+0000","last_became_peered":"2024-09-10T15:58:25.769634+0000","last_unstale":"2024-09-10T15:58:51.148806+0000","last_undegraded":"2024-09-10T15:58:51.148806+0000","last_fullsized":"2024-09-10T15:58:51.148806+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":2483,"log_dups_size":0,"ondisk_log_size":2483,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:24:15.030848+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10142179,"num_objects":2477,"num_object_clones":0,"num_object_copies":4954,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2477,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":302,"num_write_kb":1208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":74,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T15:58:25.768681+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T15:58:25.768681+0000","last_peered":"2024-09-10T15:58:25.768681+0000","last_clean":"2024-09-10T15:58:25.768681+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T15:58:25.768681+0000","last_undegraded":"2024-09-10T15:58:25.768681+0000","last_fullsized":"2024-09-10T15:58:25.768681+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"36'2546","reported_seq":2596,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-10T15:58:51.131215+0000","last_change":"2024-09-10T15:58:24.768933+0000","last_active":"2024-09-10T15:58:51.131215+0000","last_peered":"2024-09-10T15:58:51.131215+0000","last_clean":"2024-09-10T15:57:52.118434+0000","last_became_active":"2024-09-10T15:58:24.768693+0000","last_became_peered":"2024-09-10T15:58:24.768693+0000","last_unstale":"2024-09-10T15:58:51.131215+0000","last_undegraded":"2024-09-10T15:58:24.758580+0000","last_fullsized":"2024-09-10T15:58:51.131215+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":2546,"log_dups_size":0,"ondisk_log_size":2546,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10396614,"num_objects":2540,"num_object_clones":0,"num_object_copies":5080,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2540,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2544,"num_write_kb":10156,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2540}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":67,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-10T15:58:25.762469+0000","last_change":"2024-09-10T15:58:24.768453+0000","last_active":"2024-09-10T15:58:25.762469+0000","last_peered":"2024-09-10T15:58:25.762469+0000","last_clean":"2024-09-10T15:58:03.029025+0000","last_became_active":"2024-09-10T15:58:24.766316+0000","last_became_peered":"2024-09-10T15:58:24.766316+0000","last_unstale":"2024-09-10T15:58:25.762469+0000","last_undegraded":"2024-09-10T15:58:24.757744+0000","last_fullsized":"2024-09-10T15:58:25.762469+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0"],"object_location_counts":[{"shards":"0","objects":1}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"36'2551","reported_seq":2654,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T15:58:51.147868+0000","last_change":"2024-09-10T15:58:49.040907+0000","last_active":"2024-09-10T15:58:51.147868+0000","last_peered":"2024-09-10T15:58:51.147868+0000","last_clean":"2024-09-10T15:58:51.147868+0000","last_became_active":"2024-09-10T15:58:25.769424+0000","last_became_peered":"2024-09-10T15:58:25.769424+0000","last_unstale":"2024-09-10T15:58:51.147868+0000","last_undegraded":"2024-09-10T15:58:51.147868+0000","last_fullsized":"2024-09-10T15:58:51.147868+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":2551,"log_dups_size":0,"ondisk_log_size":2551,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:52:01.617164+0000","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10428899,"num_objects":2547,"num_object_clones":0,"num_object_copies":5094,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2547,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":231,"num_write_kb":924,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":92,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T15:58:25.770150+0000","last_change":"2024-09-10T15:58:25.768489+0000","last_active":"2024-09-10T15:58:25.770150+0000","last_peered":"2024-09-10T15:58:25.770150+0000","last_clean":"2024-09-10T15:58:25.770150+0000","last_became_active":"2024-09-10T15:58:25.768117+0000","last_became_peered":"2024-09-10T15:58:25.768117+0000","last_unstale":"2024-09-10T15:58:25.770150+0000","last_undegraded":"2024-09-10T15:58:25.770150+0000","last_fullsized":"2024-09-10T15:58:25.770150+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:56:47.213564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"36'2591","reported_seq":2694,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T15:58:51.148428+0000","last_change":"2024-09-10T15:58:50.079160+0000","last_active":"2024-09-10T15:58:51.148428+0000","last_peered":"2024-09-10T15:58:51.148428+0000","last_clean":"2024-09-10T15:58:51.148428+0000","last_became_active":"2024-09-10T15:58:25.769762+0000","last_became_peered":"2024-09-10T15:58:25.769762+0000","last_unstale":"2024-09-10T15:58:51.148428+0000","last_undegraded":"2024-09-10T15:58:51.148428+0000","last_fullsized":"2024-09-10T15:58:51.148428+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":2591,"log_dups_size":0,"ondisk_log_size":2591,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:50:48.841755+0000","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10588160,"num_objects":2585,"num_object_clones":0,"num_object_copies":5170,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2585,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":145,"num_write_kb":580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"36'2569","reported_seq":2627,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-10T15:58:51.169066+0000","last_change":"2024-09-10T15:58:24.768825+0000","last_active":"2024-09-10T15:58:51.169066+0000","last_peered":"2024-09-10T15:58:51.169066+0000","last_clean":"2024-09-10T15:57:45.763054+0000","last_became_active":"2024-09-10T15:58:24.768609+0000","last_became_peered":"2024-09-10T15:58:24.768609+0000","last_unstale":"2024-09-10T15:58:51.169066+0000","last_undegraded":"2024-09-10T15:58:24.757704+0000","last_fullsized":"2024-09-10T15:58:51.169066+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":2569,"log_dups_size":0,"ondisk_log_size":2569,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10514915,"num_objects":2568,"num_object_clones":0,"num_object_copies":5136,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2568,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2568,"num_write_kb":10269,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2568}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T15:58:25.762555+0000","last_change":"2024-09-10T15:58:24.767677+0000","last_active":"2024-09-10T15:58:25.762555+0000","last_peered":"2024-09-10T15:58:25.762555+0000","last_clean":"2024-09-10T15:58:25.762555+0000","last_became_active":"2024-09-10T15:58:24.767409+0000","last_became_peered":"2024-09-10T15:58:24.767409+0000","last_unstale":"2024-09-10T15:58:25.762555+0000","last_undegraded":"2024-09-10T15:58:25.762555+0000","last_fullsized":"2024-09-10T15:58:25.762555+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:35:35.763396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"36'2495","reported_seq":2552,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-10T15:58:51.169398+0000","last_change":"2024-09-10T15:58:24.769497+0000","last_active":"2024-09-10T15:58:51.169398+0000","last_peered":"2024-09-10T15:58:51.169398+0000","last_clean":"2024-09-10T15:57:52.013178+0000","last_became_active":"2024-09-10T15:58:24.769268+0000","last_became_peered":"2024-09-10T15:58:24.769268+0000","last_unstale":"2024-09-10T15:58:51.169398+0000","last_undegraded":"2024-09-10T15:58:24.759230+0000","last_fullsized":"2024-09-10T15:58:51.169398+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":2495,"log_dups_size":0,"ondisk_log_size":2495,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10188201,"num_objects":2490,"num_object_clones":0,"num_object_copies":4980,"num_objects_missing_on_primary":3,"num_objects_missing":3,"num_objects_degraded":6,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2490,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2492,"num_write_kb":9952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2490}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":74,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T15:58:25.769812+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T15:58:25.769812+0000","last_peered":"2024-09-10T15:58:25.769812+0000","last_clean":"2024-09-10T15:58:25.769812+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T15:58:25.769812+0000","last_undegraded":"2024-09-10T15:58:25.769812+0000","last_fullsized":"2024-09-10T15:58:25.769812+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"36'2655","reported_seq":2712,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-10T15:58:51.150001+0000","last_change":"2024-09-10T15:58:24.769323+0000","last_active":"2024-09-10T15:58:51.150001+0000","last_peered":"2024-09-10T15:58:51.150001+0000","last_clean":"2024-09-10T15:57:45.763338+0000","last_became_active":"2024-09-10T15:58:24.769074+0000","last_became_peered":"2024-09-10T15:58:24.769074+0000","last_unstale":"2024-09-10T15:58:51.150001+0000","last_undegraded":"2024-09-10T15:58:24.759853+0000","last_fullsized":"2024-09-10T15:58:51.150001+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":2655,"log_dups_size":0,"ondisk_log_size":2655,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10859462,"num_objects":2653,"num_object_clones":0,"num_object_copies":5306,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2653,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2653,"num_write_kb":10606,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2653}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T15:58:25.762992+0000","last_change":"2024-09-10T15:58:24.767867+0000","last_active":"2024-09-10T15:58:25.762992+0000","last_peered":"2024-09-10T15:58:25.762992+0000","last_clean":"2024-09-10T15:58:25.762992+0000","last_became_active":"2024-09-10T15:58:24.767615+0000","last_became_peered":"2024-09-10T15:58:24.767615+0000","last_unstale":"2024-09-10T15:58:25.762992+0000","last_undegraded":"2024-09-10T15:58:25.762992+0000","last_fullsized":"2024-09-10T15:58:25.762992+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:11:00.908668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":87,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T15:58:25.770511+0000","last_change":"2024-09-10T15:58:25.768847+0000","last_active":"2024-09-10T15:58:25.770511+0000","last_peered":"2024-09-10T15:58:25.770511+0000","last_clean":"2024-09-10T15:58:25.770511+0000","last_became_active":"2024-09-10T15:58:25.768040+0000","last_became_peered":"2024-09-10T15:58:25.768040+0000","last_unstale":"2024-09-10T15:58:25.770511+0000","last_undegraded":"2024-09-10T15:58:25.770511+0000","last_fullsized":"2024-09-10T15:58:25.770511+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:39:30.402776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"36'2526","reported_seq":2583,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-10T15:58:51.131128+0000","last_change":"2024-09-10T15:58:24.769859+0000","last_active":"2024-09-10T15:58:51.131128+0000","last_peered":"2024-09-10T15:58:51.131128+0000","last_clean":"2024-09-10T15:57:51.765160+0000","last_became_active":"2024-09-10T15:58:24.769616+0000","last_became_peered":"2024-09-10T15:58:24.769616+0000","last_unstale":"2024-09-10T15:58:51.131128+0000","last_undegraded":"2024-09-10T15:58:24.756607+0000","last_fullsized":"2024-09-10T15:58:51.131128+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":2526,"log_dups_size":0,"ondisk_log_size":2526,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10299276,"num_objects":2518,"num_object_clones":0,"num_object_copies":5036,"num_objects_missing_on_primary":3,"num_objects_missing":3,"num_objects_degraded":7,"num_objects_misplaced":0,"num_objects_unfound":1,"num_objects_dirty":2518,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2523,"num_write_kb":10063,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"","objects":1},{"shards":"0","objects":1},{"shards":"0,2","objects":2516}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":166583141,"num_objects":40690,"num_object_clones":0,"num_object_copies":81380,"num_objects_missing_on_primary":18,"num_objects_missing":18,"num_objects_degraded":35,"num_objects_misplaced":0,"num_objects_unfound":5,"num_objects_dirty":40690,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":29893,"num_write_kb":119458,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":333475840,"data_stored":333324094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":40749,"ondisk_log_size":40749,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986186,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":265560,"kb_used_data":157512,"kb_used_omap":4,"kb_used_meta":108027,"kb_avail":94106280,"statfs":{"total":96636764160,"available":96364830720,"internally_reserved":0,"allocated":161292288,"data_stored":161069631,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":110620330},"hb_peers":[0],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,3,3,2],"upper_bound":32},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182415,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":265336,"kb_used_data":161512,"kb_used_omap":4,"kb_used_meta":103803,"kb_avail":94106504,"statfs":{"total":96636764160,"available":96365060096,"internally_reserved":0,"allocated":165388288,"data_stored":165139876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":106294954},"hb_peers":[2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,5,0,8,3],"upper_bound":32},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":166711296,"data_stored":166675166,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":13524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":166649856,"data_stored":166635404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:58:55.963 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 23 20 47780 47760 8.11045 6.11719 0.0147665 0.00962663 2024-09-10T15:58:56.963 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 24 20 49373 49353 8.03177 6.22266 0.0146861 0.00972088 2024-09-10T15:58:57.963 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 25 20 50818 50798 7.93627 5.64453 0.00745202 0.00983701 2024-09-10T15:58:58.064 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:58:58.600 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T15:58:58.601 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T15:58:58.620 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":80,"stamp":"2024-09-10T15:58:57.542379+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":191871864,"num_objects":46866,"num_object_clones":0,"num_object_copies":93732,"num_objects_missing_on_primary":11,"num_objects_missing":11,"num_objects_degraded":22,"num_objects_misplaced":0,"num_objects_unfound":3,"num_objects_dirty":46866,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":36069,"num_write_kb":144156,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":46926,"ondisk_log_size":46926,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":48,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":578056,"kb_used_data":366184,"kb_used_omap":8,"kb_used_meta":211831,"kb_avail":188165624,"statfs":{"total":193273528320,"available":192681598976,"internally_reserved":0,"allocated":374972416,"data_stored":374501347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8876,"internal_metadata":216915284},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[6,1,9,5,6],"upper_bound":32},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":88426129,"num_objects":21584,"num_object_clones":0,"num_object_copies":43168,"num_objects_missing_on_primary":-14,"num_objects_missing":-14,"num_objects_degraded":-28,"num_objects_misplaced":0,"num_objects_unfound":-22,"num_objects_dirty":21584,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10760,"num_write_kb":43100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002785"},"pg_stats":[{"pgid":"2.f","version":"36'2964","reported_seq":3016,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T15:58:56.036468+0000","last_change":"2024-09-10T15:58:24.769328+0000","last_active":"2024-09-10T15:58:56.036468+0000","last_peered":"2024-09-10T15:58:56.036468+0000","last_clean":"2024-09-10T15:58:56.036468+0000","last_became_active":"2024-09-10T15:58:24.769050+0000","last_became_peered":"2024-09-10T15:58:24.769050+0000","last_unstale":"2024-09-10T15:58:56.036468+0000","last_undegraded":"2024-09-10T15:58:56.036468+0000","last_fullsized":"2024-09-10T15:58:56.036468+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":2964,"log_dups_size":0,"ondisk_log_size":2964,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:08:36.167566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12140544,"num_objects":2964,"num_object_clones":0,"num_object_copies":5928,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2964,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2964,"num_write_kb":11856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"36'2887","reported_seq":2978,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T15:58:54.164943+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T15:58:54.164943+0000","last_peered":"2024-09-10T15:58:54.164943+0000","last_clean":"2024-09-10T15:58:54.164943+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T15:58:54.164943+0000","last_undegraded":"2024-09-10T15:58:54.164943+0000","last_fullsized":"2024-09-10T15:58:54.164943+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":2887,"log_dups_size":0,"ondisk_log_size":2887,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11801542,"num_objects":2883,"num_object_clones":0,"num_object_copies":5766,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2883,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":992,"num_write_kb":3968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"36'2776","reported_seq":2879,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T15:58:54.158943+0000","last_change":"2024-09-10T15:58:45.933955+0000","last_active":"2024-09-10T15:58:54.158943+0000","last_peered":"2024-09-10T15:58:54.158943+0000","last_clean":"2024-09-10T15:58:54.158943+0000","last_became_active":"2024-09-10T15:58:25.767861+0000","last_became_peered":"2024-09-10T15:58:25.767861+0000","last_unstale":"2024-09-10T15:58:54.158943+0000","last_undegraded":"2024-09-10T15:58:54.158943+0000","last_fullsized":"2024-09-10T15:58:54.158943+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":2776,"log_dups_size":0,"ondisk_log_size":2776,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:22:24.003676+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11354112,"num_objects":2772,"num_object_clones":0,"num_object_copies":5544,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2772,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":780,"num_write_kb":3120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"36'2767","reported_seq":2828,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T15:58:54.113266+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T15:58:54.113266+0000","last_peered":"2024-09-10T15:58:54.113266+0000","last_clean":"2024-09-10T15:58:54.113266+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T15:58:54.113266+0000","last_undegraded":"2024-09-10T15:58:54.113266+0000","last_fullsized":"2024-09-10T15:58:54.113266+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":2767,"log_dups_size":0,"ondisk_log_size":2767,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11333632,"num_objects":2767,"num_object_clones":0,"num_object_copies":5534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2767,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2767,"num_write_kb":11068,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"36'3056","reported_seq":3125,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T15:58:56.036963+0000","last_change":"2024-09-10T15:58:55.983360+0000","last_active":"2024-09-10T15:58:56.036963+0000","last_peered":"2024-09-10T15:58:56.036963+0000","last_clean":"2024-09-10T15:58:56.036963+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T15:58:56.036963+0000","last_undegraded":"2024-09-10T15:58:56.036963+0000","last_fullsized":"2024-09-10T15:58:56.036963+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":3056,"log_dups_size":0,"ondisk_log_size":3056,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12489670,"num_objects":3051,"num_object_clones":0,"num_object_copies":6102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3051,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3054,"num_write_kb":12200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"36'3038","reported_seq":3106,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T15:58:56.026596+0000","last_change":"2024-09-10T15:58:56.013033+0000","last_active":"2024-09-10T15:58:56.026596+0000","last_peered":"2024-09-10T15:58:56.026596+0000","last_clean":"2024-09-10T15:58:56.026596+0000","last_became_active":"2024-09-10T15:58:24.768198+0000","last_became_peered":"2024-09-10T15:58:24.768198+0000","last_unstale":"2024-09-10T15:58:56.026596+0000","last_undegraded":"2024-09-10T15:58:56.026596+0000","last_fullsized":"2024-09-10T15:58:56.026596+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":3038,"log_dups_size":0,"ondisk_log_size":3038,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12424134,"num_objects":3035,"num_object_clones":0,"num_object_copies":6070,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3035,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3036,"num_write_kb":12135,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"36'2957","reported_seq":3016,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-10T15:58:56.040026+0000","last_change":"2024-09-10T15:58:24.769216+0000","last_active":"2024-09-10T15:58:56.040026+0000","last_peered":"2024-09-10T15:58:56.040026+0000","last_clean":"2024-09-10T15:57:51.467206+0000","last_became_active":"2024-09-10T15:58:24.768972+0000","last_became_peered":"2024-09-10T15:58:24.768972+0000","last_unstale":"2024-09-10T15:58:56.040026+0000","last_undegraded":"2024-09-10T15:58:24.758006+0000","last_fullsized":"2024-09-10T15:58:56.040026+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":2957,"log_dups_size":0,"ondisk_log_size":2957,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12088262,"num_objects":2953,"num_object_clones":0,"num_object_copies":5906,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":2953,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2955,"num_write_kb":11808,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"","objects":2},{"shards":"0,2","objects":2951}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"36'3008","reported_seq":3074,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T15:58:56.049673+0000","last_change":"2024-09-10T15:58:55.928290+0000","last_active":"2024-09-10T15:58:56.049673+0000","last_peered":"2024-09-10T15:58:56.049673+0000","last_clean":"2024-09-10T15:58:56.049673+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T15:58:56.049673+0000","last_undegraded":"2024-09-10T15:58:56.049673+0000","last_fullsized":"2024-09-10T15:58:56.049673+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":3008,"log_dups_size":0,"ondisk_log_size":3008,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12313059,"num_objects":3007,"num_object_clones":0,"num_object_copies":6014,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3007,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3007,"num_write_kb":12025,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T15:58:25.762375+0000","last_change":"2024-09-10T15:58:24.767197+0000","last_active":"2024-09-10T15:58:25.762375+0000","last_peered":"2024-09-10T15:58:25.762375+0000","last_clean":"2024-09-10T15:58:25.762375+0000","last_became_active":"2024-09-10T15:58:24.766691+0000","last_became_peered":"2024-09-10T15:58:24.766691+0000","last_unstale":"2024-09-10T15:58:25.762375+0000","last_undegraded":"2024-09-10T15:58:25.762375+0000","last_fullsized":"2024-09-10T15:58:25.762375+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:51:00.161051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"36'2735","reported_seq":2842,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T15:58:54.143073+0000","last_change":"2024-09-10T15:58:48.061941+0000","last_active":"2024-09-10T15:58:54.143073+0000","last_peered":"2024-09-10T15:58:54.143073+0000","last_clean":"2024-09-10T15:58:54.143073+0000","last_became_active":"2024-09-10T15:58:25.769634+0000","last_became_peered":"2024-09-10T15:58:25.769634+0000","last_unstale":"2024-09-10T15:58:54.143073+0000","last_undegraded":"2024-09-10T15:58:54.143073+0000","last_fullsized":"2024-09-10T15:58:54.143073+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":2735,"log_dups_size":0,"ondisk_log_size":2735,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:24:15.030848+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11174371,"num_objects":2729,"num_object_clones":0,"num_object_copies":5458,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2729,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":554,"num_write_kb":2216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":74,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T15:58:25.768681+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T15:58:25.768681+0000","last_peered":"2024-09-10T15:58:25.768681+0000","last_clean":"2024-09-10T15:58:25.768681+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T15:58:25.768681+0000","last_undegraded":"2024-09-10T15:58:25.768681+0000","last_fullsized":"2024-09-10T15:58:25.768681+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"36'3015","reported_seq":3067,"reported_epoch":36,"state":"active+recovering","last_fresh":"2024-09-10T15:58:56.046774+0000","last_change":"2024-09-10T15:58:56.049583+0000","last_active":"2024-09-10T15:58:56.046774+0000","last_peered":"2024-09-10T15:58:56.046774+0000","last_clean":"2024-09-10T15:57:52.118434+0000","last_became_active":"2024-09-10T15:58:24.768693+0000","last_became_peered":"2024-09-10T15:58:24.768693+0000","last_unstale":"2024-09-10T15:58:56.046774+0000","last_undegraded":"2024-09-10T15:58:24.758580+0000","last_fullsized":"2024-09-10T15:58:56.046774+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":3015,"log_dups_size":0,"ondisk_log_size":3015,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12317638,"num_objects":3009,"num_object_clones":0,"num_object_copies":6018,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3009,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3013,"num_write_kb":12032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":3009}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":67,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-10T15:58:25.762469+0000","last_change":"2024-09-10T15:58:24.768453+0000","last_active":"2024-09-10T15:58:25.762469+0000","last_peered":"2024-09-10T15:58:25.762469+0000","last_clean":"2024-09-10T15:58:03.029025+0000","last_became_active":"2024-09-10T15:58:24.766316+0000","last_became_peered":"2024-09-10T15:58:24.766316+0000","last_unstale":"2024-09-10T15:58:25.762469+0000","last_undegraded":"2024-09-10T15:58:24.757744+0000","last_fullsized":"2024-09-10T15:58:25.762469+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0"],"object_location_counts":[{"shards":"0","objects":1}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"36'2816","reported_seq":2919,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T15:58:54.134537+0000","last_change":"2024-09-10T15:58:49.040907+0000","last_active":"2024-09-10T15:58:54.134537+0000","last_peered":"2024-09-10T15:58:54.134537+0000","last_clean":"2024-09-10T15:58:54.134537+0000","last_became_active":"2024-09-10T15:58:25.769424+0000","last_became_peered":"2024-09-10T15:58:25.769424+0000","last_unstale":"2024-09-10T15:58:54.134537+0000","last_undegraded":"2024-09-10T15:58:54.134537+0000","last_fullsized":"2024-09-10T15:58:54.134537+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":2816,"log_dups_size":0,"ondisk_log_size":2816,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:52:01.617164+0000","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11514339,"num_objects":2812,"num_object_clones":0,"num_object_copies":5624,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2812,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":496,"num_write_kb":1984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":92,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T15:58:25.770150+0000","last_change":"2024-09-10T15:58:25.768489+0000","last_active":"2024-09-10T15:58:25.770150+0000","last_peered":"2024-09-10T15:58:25.770150+0000","last_clean":"2024-09-10T15:58:25.770150+0000","last_became_active":"2024-09-10T15:58:25.768117+0000","last_became_peered":"2024-09-10T15:58:25.768117+0000","last_unstale":"2024-09-10T15:58:25.770150+0000","last_undegraded":"2024-09-10T15:58:25.770150+0000","last_fullsized":"2024-09-10T15:58:25.770150+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:56:47.213564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"36'2860","reported_seq":2963,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T15:58:54.127582+0000","last_change":"2024-09-10T15:58:50.079160+0000","last_active":"2024-09-10T15:58:54.127582+0000","last_peered":"2024-09-10T15:58:54.127582+0000","last_clean":"2024-09-10T15:58:54.127582+0000","last_became_active":"2024-09-10T15:58:25.769762+0000","last_became_peered":"2024-09-10T15:58:25.769762+0000","last_unstale":"2024-09-10T15:58:54.127582+0000","last_undegraded":"2024-09-10T15:58:54.127582+0000","last_fullsized":"2024-09-10T15:58:54.127582+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":2860,"log_dups_size":0,"ondisk_log_size":2860,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:50:48.841755+0000","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11689984,"num_objects":2854,"num_object_clones":0,"num_object_copies":5708,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2854,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":414,"num_write_kb":1656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"36'3002","reported_seq":3068,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T15:58:56.045545+0000","last_change":"2024-09-10T15:58:56.044931+0000","last_active":"2024-09-10T15:58:56.045545+0000","last_peered":"2024-09-10T15:58:56.045545+0000","last_clean":"2024-09-10T15:58:56.045545+0000","last_became_active":"2024-09-10T15:58:24.768609+0000","last_became_peered":"2024-09-10T15:58:24.768609+0000","last_unstale":"2024-09-10T15:58:56.045545+0000","last_undegraded":"2024-09-10T15:58:56.045545+0000","last_fullsized":"2024-09-10T15:58:56.045545+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":3002,"log_dups_size":0,"ondisk_log_size":3002,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12288483,"num_objects":3001,"num_object_clones":0,"num_object_copies":6002,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3001,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3001,"num_write_kb":12001,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T15:58:25.762555+0000","last_change":"2024-09-10T15:58:24.767677+0000","last_active":"2024-09-10T15:58:25.762555+0000","last_peered":"2024-09-10T15:58:25.762555+0000","last_clean":"2024-09-10T15:58:25.762555+0000","last_became_active":"2024-09-10T15:58:24.767409+0000","last_became_peered":"2024-09-10T15:58:24.767409+0000","last_unstale":"2024-09-10T15:58:25.762555+0000","last_undegraded":"2024-09-10T15:58:25.762555+0000","last_fullsized":"2024-09-10T15:58:25.762555+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:35:35.763396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"36'2940","reported_seq":2997,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-10T15:58:56.033570+0000","last_change":"2024-09-10T15:58:24.769497+0000","last_active":"2024-09-10T15:58:56.033570+0000","last_peered":"2024-09-10T15:58:56.033570+0000","last_clean":"2024-09-10T15:57:52.013178+0000","last_became_active":"2024-09-10T15:58:24.769268+0000","last_became_peered":"2024-09-10T15:58:24.769268+0000","last_unstale":"2024-09-10T15:58:56.033570+0000","last_undegraded":"2024-09-10T15:58:24.759230+0000","last_fullsized":"2024-09-10T15:58:56.033570+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":2940,"log_dups_size":0,"ondisk_log_size":2940,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12010921,"num_objects":2935,"num_object_clones":0,"num_object_copies":5870,"num_objects_missing_on_primary":3,"num_objects_missing":3,"num_objects_degraded":6,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2935,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2937,"num_write_kb":11732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2935}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":74,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T15:58:25.769812+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T15:58:25.769812+0000","last_peered":"2024-09-10T15:58:25.769812+0000","last_clean":"2024-09-10T15:58:25.769812+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T15:58:25.769812+0000","last_undegraded":"2024-09-10T15:58:25.769812+0000","last_fullsized":"2024-09-10T15:58:25.769812+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"36'3111","reported_seq":3168,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-10T15:58:56.002443+0000","last_change":"2024-09-10T15:58:24.769323+0000","last_active":"2024-09-10T15:58:56.002443+0000","last_peered":"2024-09-10T15:58:56.002443+0000","last_clean":"2024-09-10T15:57:45.763338+0000","last_became_active":"2024-09-10T15:58:24.769074+0000","last_became_peered":"2024-09-10T15:58:24.769074+0000","last_unstale":"2024-09-10T15:58:56.002443+0000","last_undegraded":"2024-09-10T15:58:24.759853+0000","last_fullsized":"2024-09-10T15:58:56.002443+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":3111,"log_dups_size":0,"ondisk_log_size":3111,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12727238,"num_objects":3109,"num_object_clones":0,"num_object_copies":6218,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3109,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3109,"num_write_kb":12430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":3109}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T15:58:25.762992+0000","last_change":"2024-09-10T15:58:24.767867+0000","last_active":"2024-09-10T15:58:25.762992+0000","last_peered":"2024-09-10T15:58:25.762992+0000","last_clean":"2024-09-10T15:58:25.762992+0000","last_became_active":"2024-09-10T15:58:24.767615+0000","last_became_peered":"2024-09-10T15:58:24.767615+0000","last_unstale":"2024-09-10T15:58:25.762992+0000","last_undegraded":"2024-09-10T15:58:25.762992+0000","last_fullsized":"2024-09-10T15:58:25.762992+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:11:00.908668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":87,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T15:58:25.770511+0000","last_change":"2024-09-10T15:58:25.768847+0000","last_active":"2024-09-10T15:58:25.770511+0000","last_peered":"2024-09-10T15:58:25.770511+0000","last_clean":"2024-09-10T15:58:25.770511+0000","last_became_active":"2024-09-10T15:58:25.768040+0000","last_became_peered":"2024-09-10T15:58:25.768040+0000","last_unstale":"2024-09-10T15:58:25.770511+0000","last_undegraded":"2024-09-10T15:58:25.770511+0000","last_fullsized":"2024-09-10T15:58:25.770511+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:39:30.402776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"36'2991","reported_seq":3048,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-10T15:58:56.025841+0000","last_change":"2024-09-10T15:58:24.769859+0000","last_active":"2024-09-10T15:58:56.025841+0000","last_peered":"2024-09-10T15:58:56.025841+0000","last_clean":"2024-09-10T15:57:51.765160+0000","last_became_active":"2024-09-10T15:58:24.769616+0000","last_became_peered":"2024-09-10T15:58:24.769616+0000","last_unstale":"2024-09-10T15:58:56.025841+0000","last_undegraded":"2024-09-10T15:58:24.756607+0000","last_fullsized":"2024-09-10T15:58:56.025841+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":2991,"log_dups_size":0,"ondisk_log_size":2991,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12203916,"num_objects":2983,"num_object_clones":0,"num_object_copies":5966,"num_objects_missing_on_primary":3,"num_objects_missing":3,"num_objects_degraded":7,"num_objects_misplaced":0,"num_objects_unfound":1,"num_objects_dirty":2983,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2988,"num_write_kb":11923,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"","objects":1},{"shards":"0","objects":1},{"shards":"0,2","objects":2981}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":191871845,"num_objects":46864,"num_object_clones":0,"num_object_copies":93728,"num_objects_missing_on_primary":11,"num_objects_missing":11,"num_objects_degraded":21,"num_objects_misplaced":0,"num_objects_unfound":3,"num_objects_dirty":46864,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":36067,"num_write_kb":144154,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":381005824,"data_stored":380854078,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":46923,"ondisk_log_size":46923,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986187,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":274880,"kb_used_data":175152,"kb_used_omap":4,"kb_used_meta":99707,"kb_avail":94096960,"statfs":{"total":96636764160,"available":96355287040,"internally_reserved":0,"allocated":179355648,"data_stored":179132991,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":102100650},"hb_peers":[0],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,3,3,2],"upper_bound":32},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182416,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303176,"kb_used_data":191032,"kb_used_omap":4,"kb_used_meta":112123,"kb_avail":94068664,"statfs":{"total":96636764160,"available":96326311936,"internally_reserved":0,"allocated":195616768,"data_stored":195368356,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":114814634},"hb_peers":[2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,1,6,2,4],"upper_bound":32},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":196296704,"data_stored":196260574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":13524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":184594432,"data_stored":184579980,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:58:58.622 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:58:58.964 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 26 20 52084 52064 7.82122 4.94531 0.0249803 0.00998099 2024-09-10T15:58:59.159 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T15:58:59.161 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T15:58:59.183 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":80,"stamp":"2024-09-10T15:58:57.542379+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":191871864,"num_objects":46866,"num_object_clones":0,"num_object_copies":93732,"num_objects_missing_on_primary":11,"num_objects_missing":11,"num_objects_degraded":22,"num_objects_misplaced":0,"num_objects_unfound":3,"num_objects_dirty":46866,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":36069,"num_write_kb":144156,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":46926,"ondisk_log_size":46926,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":48,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":578056,"kb_used_data":366184,"kb_used_omap":8,"kb_used_meta":211831,"kb_avail":188165624,"statfs":{"total":193273528320,"available":192681598976,"internally_reserved":0,"allocated":374972416,"data_stored":374501347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8876,"internal_metadata":216915284},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[6,1,9,5,6],"upper_bound":32},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":88426129,"num_objects":21584,"num_object_clones":0,"num_object_copies":43168,"num_objects_missing_on_primary":-14,"num_objects_missing":-14,"num_objects_degraded":-28,"num_objects_misplaced":0,"num_objects_unfound":-22,"num_objects_dirty":21584,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10760,"num_write_kb":43100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002785"},"pg_stats":[{"pgid":"2.f","version":"36'2964","reported_seq":3016,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T15:58:56.036468+0000","last_change":"2024-09-10T15:58:24.769328+0000","last_active":"2024-09-10T15:58:56.036468+0000","last_peered":"2024-09-10T15:58:56.036468+0000","last_clean":"2024-09-10T15:58:56.036468+0000","last_became_active":"2024-09-10T15:58:24.769050+0000","last_became_peered":"2024-09-10T15:58:24.769050+0000","last_unstale":"2024-09-10T15:58:56.036468+0000","last_undegraded":"2024-09-10T15:58:56.036468+0000","last_fullsized":"2024-09-10T15:58:56.036468+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":2964,"log_dups_size":0,"ondisk_log_size":2964,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:08:36.167566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12140544,"num_objects":2964,"num_object_clones":0,"num_object_copies":5928,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2964,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2964,"num_write_kb":11856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"36'2887","reported_seq":2978,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T15:58:54.164943+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T15:58:54.164943+0000","last_peered":"2024-09-10T15:58:54.164943+0000","last_clean":"2024-09-10T15:58:54.164943+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T15:58:54.164943+0000","last_undegraded":"2024-09-10T15:58:54.164943+0000","last_fullsized":"2024-09-10T15:58:54.164943+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":2887,"log_dups_size":0,"ondisk_log_size":2887,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11801542,"num_objects":2883,"num_object_clones":0,"num_object_copies":5766,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2883,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":992,"num_write_kb":3968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"36'2776","reported_seq":2879,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T15:58:54.158943+0000","last_change":"2024-09-10T15:58:45.933955+0000","last_active":"2024-09-10T15:58:54.158943+0000","last_peered":"2024-09-10T15:58:54.158943+0000","last_clean":"2024-09-10T15:58:54.158943+0000","last_became_active":"2024-09-10T15:58:25.767861+0000","last_became_peered":"2024-09-10T15:58:25.767861+0000","last_unstale":"2024-09-10T15:58:54.158943+0000","last_undegraded":"2024-09-10T15:58:54.158943+0000","last_fullsized":"2024-09-10T15:58:54.158943+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":2776,"log_dups_size":0,"ondisk_log_size":2776,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:22:24.003676+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11354112,"num_objects":2772,"num_object_clones":0,"num_object_copies":5544,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2772,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":780,"num_write_kb":3120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"36'2767","reported_seq":2828,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T15:58:54.113266+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T15:58:54.113266+0000","last_peered":"2024-09-10T15:58:54.113266+0000","last_clean":"2024-09-10T15:58:54.113266+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T15:58:54.113266+0000","last_undegraded":"2024-09-10T15:58:54.113266+0000","last_fullsized":"2024-09-10T15:58:54.113266+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":2767,"log_dups_size":0,"ondisk_log_size":2767,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11333632,"num_objects":2767,"num_object_clones":0,"num_object_copies":5534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2767,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2767,"num_write_kb":11068,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"36'3056","reported_seq":3125,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T15:58:56.036963+0000","last_change":"2024-09-10T15:58:55.983360+0000","last_active":"2024-09-10T15:58:56.036963+0000","last_peered":"2024-09-10T15:58:56.036963+0000","last_clean":"2024-09-10T15:58:56.036963+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T15:58:56.036963+0000","last_undegraded":"2024-09-10T15:58:56.036963+0000","last_fullsized":"2024-09-10T15:58:56.036963+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":3056,"log_dups_size":0,"ondisk_log_size":3056,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12489670,"num_objects":3051,"num_object_clones":0,"num_object_copies":6102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3051,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3054,"num_write_kb":12200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"36'3038","reported_seq":3106,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T15:58:56.026596+0000","last_change":"2024-09-10T15:58:56.013033+0000","last_active":"2024-09-10T15:58:56.026596+0000","last_peered":"2024-09-10T15:58:56.026596+0000","last_clean":"2024-09-10T15:58:56.026596+0000","last_became_active":"2024-09-10T15:58:24.768198+0000","last_became_peered":"2024-09-10T15:58:24.768198+0000","last_unstale":"2024-09-10T15:58:56.026596+0000","last_undegraded":"2024-09-10T15:58:56.026596+0000","last_fullsized":"2024-09-10T15:58:56.026596+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":3038,"log_dups_size":0,"ondisk_log_size":3038,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12424134,"num_objects":3035,"num_object_clones":0,"num_object_copies":6070,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3035,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3036,"num_write_kb":12135,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"36'2957","reported_seq":3016,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-10T15:58:56.040026+0000","last_change":"2024-09-10T15:58:24.769216+0000","last_active":"2024-09-10T15:58:56.040026+0000","last_peered":"2024-09-10T15:58:56.040026+0000","last_clean":"2024-09-10T15:57:51.467206+0000","last_became_active":"2024-09-10T15:58:24.768972+0000","last_became_peered":"2024-09-10T15:58:24.768972+0000","last_unstale":"2024-09-10T15:58:56.040026+0000","last_undegraded":"2024-09-10T15:58:24.758006+0000","last_fullsized":"2024-09-10T15:58:56.040026+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":2957,"log_dups_size":0,"ondisk_log_size":2957,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12088262,"num_objects":2953,"num_object_clones":0,"num_object_copies":5906,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":2953,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2955,"num_write_kb":11808,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"","objects":2},{"shards":"0,2","objects":2951}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"36'3008","reported_seq":3074,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T15:58:56.049673+0000","last_change":"2024-09-10T15:58:55.928290+0000","last_active":"2024-09-10T15:58:56.049673+0000","last_peered":"2024-09-10T15:58:56.049673+0000","last_clean":"2024-09-10T15:58:56.049673+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T15:58:56.049673+0000","last_undegraded":"2024-09-10T15:58:56.049673+0000","last_fullsized":"2024-09-10T15:58:56.049673+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":3008,"log_dups_size":0,"ondisk_log_size":3008,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12313059,"num_objects":3007,"num_object_clones":0,"num_object_copies":6014,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3007,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3007,"num_write_kb":12025,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T15:58:25.762375+0000","last_change":"2024-09-10T15:58:24.767197+0000","last_active":"2024-09-10T15:58:25.762375+0000","last_peered":"2024-09-10T15:58:25.762375+0000","last_clean":"2024-09-10T15:58:25.762375+0000","last_became_active":"2024-09-10T15:58:24.766691+0000","last_became_peered":"2024-09-10T15:58:24.766691+0000","last_unstale":"2024-09-10T15:58:25.762375+0000","last_undegraded":"2024-09-10T15:58:25.762375+0000","last_fullsized":"2024-09-10T15:58:25.762375+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:51:00.161051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"36'2735","reported_seq":2842,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T15:58:54.143073+0000","last_change":"2024-09-10T15:58:48.061941+0000","last_active":"2024-09-10T15:58:54.143073+0000","last_peered":"2024-09-10T15:58:54.143073+0000","last_clean":"2024-09-10T15:58:54.143073+0000","last_became_active":"2024-09-10T15:58:25.769634+0000","last_became_peered":"2024-09-10T15:58:25.769634+0000","last_unstale":"2024-09-10T15:58:54.143073+0000","last_undegraded":"2024-09-10T15:58:54.143073+0000","last_fullsized":"2024-09-10T15:58:54.143073+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":2735,"log_dups_size":0,"ondisk_log_size":2735,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:24:15.030848+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11174371,"num_objects":2729,"num_object_clones":0,"num_object_copies":5458,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2729,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":554,"num_write_kb":2216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":74,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T15:58:25.768681+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T15:58:25.768681+0000","last_peered":"2024-09-10T15:58:25.768681+0000","last_clean":"2024-09-10T15:58:25.768681+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T15:58:25.768681+0000","last_undegraded":"2024-09-10T15:58:25.768681+0000","last_fullsized":"2024-09-10T15:58:25.768681+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"36'3015","reported_seq":3067,"reported_epoch":36,"state":"active+recovering","last_fresh":"2024-09-10T15:58:56.046774+0000","last_change":"2024-09-10T15:58:56.049583+0000","last_active":"2024-09-10T15:58:56.046774+0000","last_peered":"2024-09-10T15:58:56.046774+0000","last_clean":"2024-09-10T15:57:52.118434+0000","last_became_active":"2024-09-10T15:58:24.768693+0000","last_became_peered":"2024-09-10T15:58:24.768693+0000","last_unstale":"2024-09-10T15:58:56.046774+0000","last_undegraded":"2024-09-10T15:58:24.758580+0000","last_fullsized":"2024-09-10T15:58:56.046774+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":3015,"log_dups_size":0,"ondisk_log_size":3015,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12317638,"num_objects":3009,"num_object_clones":0,"num_object_copies":6018,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3009,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3013,"num_write_kb":12032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":3009}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":67,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-10T15:58:25.762469+0000","last_change":"2024-09-10T15:58:24.768453+0000","last_active":"2024-09-10T15:58:25.762469+0000","last_peered":"2024-09-10T15:58:25.762469+0000","last_clean":"2024-09-10T15:58:03.029025+0000","last_became_active":"2024-09-10T15:58:24.766316+0000","last_became_peered":"2024-09-10T15:58:24.766316+0000","last_unstale":"2024-09-10T15:58:25.762469+0000","last_undegraded":"2024-09-10T15:58:24.757744+0000","last_fullsized":"2024-09-10T15:58:25.762469+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0"],"object_location_counts":[{"shards":"0","objects":1}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"36'2816","reported_seq":2919,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T15:58:54.134537+0000","last_change":"2024-09-10T15:58:49.040907+0000","last_active":"2024-09-10T15:58:54.134537+0000","last_peered":"2024-09-10T15:58:54.134537+0000","last_clean":"2024-09-10T15:58:54.134537+0000","last_became_active":"2024-09-10T15:58:25.769424+0000","last_became_peered":"2024-09-10T15:58:25.769424+0000","last_unstale":"2024-09-10T15:58:54.134537+0000","last_undegraded":"2024-09-10T15:58:54.134537+0000","last_fullsized":"2024-09-10T15:58:54.134537+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":2816,"log_dups_size":0,"ondisk_log_size":2816,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:52:01.617164+0000","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11514339,"num_objects":2812,"num_object_clones":0,"num_object_copies":5624,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2812,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":496,"num_write_kb":1984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":92,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T15:58:25.770150+0000","last_change":"2024-09-10T15:58:25.768489+0000","last_active":"2024-09-10T15:58:25.770150+0000","last_peered":"2024-09-10T15:58:25.770150+0000","last_clean":"2024-09-10T15:58:25.770150+0000","last_became_active":"2024-09-10T15:58:25.768117+0000","last_became_peered":"2024-09-10T15:58:25.768117+0000","last_unstale":"2024-09-10T15:58:25.770150+0000","last_undegraded":"2024-09-10T15:58:25.770150+0000","last_fullsized":"2024-09-10T15:58:25.770150+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:56:47.213564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"36'2860","reported_seq":2963,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T15:58:54.127582+0000","last_change":"2024-09-10T15:58:50.079160+0000","last_active":"2024-09-10T15:58:54.127582+0000","last_peered":"2024-09-10T15:58:54.127582+0000","last_clean":"2024-09-10T15:58:54.127582+0000","last_became_active":"2024-09-10T15:58:25.769762+0000","last_became_peered":"2024-09-10T15:58:25.769762+0000","last_unstale":"2024-09-10T15:58:54.127582+0000","last_undegraded":"2024-09-10T15:58:54.127582+0000","last_fullsized":"2024-09-10T15:58:54.127582+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":2860,"log_dups_size":0,"ondisk_log_size":2860,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:50:48.841755+0000","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11689984,"num_objects":2854,"num_object_clones":0,"num_object_copies":5708,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2854,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":414,"num_write_kb":1656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"36'3002","reported_seq":3068,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T15:58:56.045545+0000","last_change":"2024-09-10T15:58:56.044931+0000","last_active":"2024-09-10T15:58:56.045545+0000","last_peered":"2024-09-10T15:58:56.045545+0000","last_clean":"2024-09-10T15:58:56.045545+0000","last_became_active":"2024-09-10T15:58:24.768609+0000","last_became_peered":"2024-09-10T15:58:24.768609+0000","last_unstale":"2024-09-10T15:58:56.045545+0000","last_undegraded":"2024-09-10T15:58:56.045545+0000","last_fullsized":"2024-09-10T15:58:56.045545+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":3002,"log_dups_size":0,"ondisk_log_size":3002,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12288483,"num_objects":3001,"num_object_clones":0,"num_object_copies":6002,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3001,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3001,"num_write_kb":12001,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T15:58:25.762555+0000","last_change":"2024-09-10T15:58:24.767677+0000","last_active":"2024-09-10T15:58:25.762555+0000","last_peered":"2024-09-10T15:58:25.762555+0000","last_clean":"2024-09-10T15:58:25.762555+0000","last_became_active":"2024-09-10T15:58:24.767409+0000","last_became_peered":"2024-09-10T15:58:24.767409+0000","last_unstale":"2024-09-10T15:58:25.762555+0000","last_undegraded":"2024-09-10T15:58:25.762555+0000","last_fullsized":"2024-09-10T15:58:25.762555+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:35:35.763396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"36'2940","reported_seq":2997,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-10T15:58:56.033570+0000","last_change":"2024-09-10T15:58:24.769497+0000","last_active":"2024-09-10T15:58:56.033570+0000","last_peered":"2024-09-10T15:58:56.033570+0000","last_clean":"2024-09-10T15:57:52.013178+0000","last_became_active":"2024-09-10T15:58:24.769268+0000","last_became_peered":"2024-09-10T15:58:24.769268+0000","last_unstale":"2024-09-10T15:58:56.033570+0000","last_undegraded":"2024-09-10T15:58:24.759230+0000","last_fullsized":"2024-09-10T15:58:56.033570+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":2940,"log_dups_size":0,"ondisk_log_size":2940,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12010921,"num_objects":2935,"num_object_clones":0,"num_object_copies":5870,"num_objects_missing_on_primary":3,"num_objects_missing":3,"num_objects_degraded":6,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2935,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2937,"num_write_kb":11732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2935}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":74,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T15:58:25.769812+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T15:58:25.769812+0000","last_peered":"2024-09-10T15:58:25.769812+0000","last_clean":"2024-09-10T15:58:25.769812+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T15:58:25.769812+0000","last_undegraded":"2024-09-10T15:58:25.769812+0000","last_fullsized":"2024-09-10T15:58:25.769812+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"36'3111","reported_seq":3168,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-10T15:58:56.002443+0000","last_change":"2024-09-10T15:58:24.769323+0000","last_active":"2024-09-10T15:58:56.002443+0000","last_peered":"2024-09-10T15:58:56.002443+0000","last_clean":"2024-09-10T15:57:45.763338+0000","last_became_active":"2024-09-10T15:58:24.769074+0000","last_became_peered":"2024-09-10T15:58:24.769074+0000","last_unstale":"2024-09-10T15:58:56.002443+0000","last_undegraded":"2024-09-10T15:58:24.759853+0000","last_fullsized":"2024-09-10T15:58:56.002443+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":3111,"log_dups_size":0,"ondisk_log_size":3111,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12727238,"num_objects":3109,"num_object_clones":0,"num_object_copies":6218,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3109,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3109,"num_write_kb":12430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":3109}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T15:58:25.762992+0000","last_change":"2024-09-10T15:58:24.767867+0000","last_active":"2024-09-10T15:58:25.762992+0000","last_peered":"2024-09-10T15:58:25.762992+0000","last_clean":"2024-09-10T15:58:25.762992+0000","last_became_active":"2024-09-10T15:58:24.767615+0000","last_became_peered":"2024-09-10T15:58:24.767615+0000","last_unstale":"2024-09-10T15:58:25.762992+0000","last_undegraded":"2024-09-10T15:58:25.762992+0000","last_fullsized":"2024-09-10T15:58:25.762992+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:11:00.908668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":87,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T15:58:25.770511+0000","last_change":"2024-09-10T15:58:25.768847+0000","last_active":"2024-09-10T15:58:25.770511+0000","last_peered":"2024-09-10T15:58:25.770511+0000","last_clean":"2024-09-10T15:58:25.770511+0000","last_became_active":"2024-09-10T15:58:25.768040+0000","last_became_peered":"2024-09-10T15:58:25.768040+0000","last_unstale":"2024-09-10T15:58:25.770511+0000","last_undegraded":"2024-09-10T15:58:25.770511+0000","last_fullsized":"2024-09-10T15:58:25.770511+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:39:30.402776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"36'2991","reported_seq":3048,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-10T15:58:56.025841+0000","last_change":"2024-09-10T15:58:24.769859+0000","last_active":"2024-09-10T15:58:56.025841+0000","last_peered":"2024-09-10T15:58:56.025841+0000","last_clean":"2024-09-10T15:57:51.765160+0000","last_became_active":"2024-09-10T15:58:24.769616+0000","last_became_peered":"2024-09-10T15:58:24.769616+0000","last_unstale":"2024-09-10T15:58:56.025841+0000","last_undegraded":"2024-09-10T15:58:24.756607+0000","last_fullsized":"2024-09-10T15:58:56.025841+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":2991,"log_dups_size":0,"ondisk_log_size":2991,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12203916,"num_objects":2983,"num_object_clones":0,"num_object_copies":5966,"num_objects_missing_on_primary":3,"num_objects_missing":3,"num_objects_degraded":7,"num_objects_misplaced":0,"num_objects_unfound":1,"num_objects_dirty":2983,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2988,"num_write_kb":11923,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"","objects":1},{"shards":"0","objects":1},{"shards":"0,2","objects":2981}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":191871845,"num_objects":46864,"num_object_clones":0,"num_object_copies":93728,"num_objects_missing_on_primary":11,"num_objects_missing":11,"num_objects_degraded":21,"num_objects_misplaced":0,"num_objects_unfound":3,"num_objects_dirty":46864,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":36067,"num_write_kb":144154,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":381005824,"data_stored":380854078,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":46923,"ondisk_log_size":46923,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986187,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":274880,"kb_used_data":175152,"kb_used_omap":4,"kb_used_meta":99707,"kb_avail":94096960,"statfs":{"total":96636764160,"available":96355287040,"internally_reserved":0,"allocated":179355648,"data_stored":179132991,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":102100650},"hb_peers":[0],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,3,3,2],"upper_bound":32},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182416,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303176,"kb_used_data":191032,"kb_used_omap":4,"kb_used_meta":112123,"kb_avail":94068664,"statfs":{"total":96636764160,"available":96326311936,"internally_reserved":0,"allocated":195616768,"data_stored":195368356,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":114814634},"hb_peers":[2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,1,6,2,4],"upper_bound":32},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":196296704,"data_stored":196260574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":13524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":184594432,"data_stored":184579980,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:58:59.963 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 27 20 53447 53427 7.72872 5.32422 0.0121252 0.0101017 2024-09-10T15:59:00.963 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 28 20 54787 54767 7.63962 5.23438 0.0130212 0.0102199 2024-09-10T15:59:01.964 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 29 20 56197 56177 7.56609 5.50781 0.0199151 0.0103201 2024-09-10T15:59:02.185 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:59:02.716 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T15:59:02.716 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T15:59:02.735 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":82,"stamp":"2024-09-10T15:59:01.543141+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":220857324,"num_objects":53939,"num_object_clones":0,"num_object_copies":107878,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53939,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":29953,"num_write_kb":119723,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":1951,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":54003,"ondisk_log_size":54003,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":48,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":668680,"kb_used_data":424872,"kb_used_omap":8,"kb_used_meta":243767,"kb_avail":188075000,"statfs":{"total":193273528320,"available":192588800000,"internally_reserved":0,"allocated":435068928,"data_stored":434572104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8876,"internal_metadata":249617748},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,4,7,9,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":79221934,"num_objects":19336,"num_object_clones":0,"num_object_copies":38672,"num_objects_missing_on_primary":-18,"num_objects_missing":-18,"num_objects_degraded":-36,"num_objects_misplaced":0,"num_objects_unfound":-16,"num_objects_dirty":19336,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3705,"num_write_kb":14864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":1951,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002556"},"pg_stats":[{"pgid":"2.f","version":"36'3384","reported_seq":3436,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T15:59:01.068143+0000","last_change":"2024-09-10T15:58:24.769328+0000","last_active":"2024-09-10T15:59:01.068143+0000","last_peered":"2024-09-10T15:59:01.068143+0000","last_clean":"2024-09-10T15:59:01.068143+0000","last_became_active":"2024-09-10T15:58:24.769050+0000","last_became_peered":"2024-09-10T15:58:24.769050+0000","last_unstale":"2024-09-10T15:59:01.068143+0000","last_undegraded":"2024-09-10T15:59:01.068143+0000","last_fullsized":"2024-09-10T15:59:01.068143+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":3384,"log_dups_size":0,"ondisk_log_size":3384,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:08:36.167566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13860864,"num_objects":3384,"num_object_clones":0,"num_object_copies":6768,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3384,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3384,"num_write_kb":13536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"36'3327","reported_seq":3418,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T15:58:59.142041+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T15:58:59.142041+0000","last_peered":"2024-09-10T15:58:59.142041+0000","last_clean":"2024-09-10T15:58:59.142041+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T15:58:59.142041+0000","last_undegraded":"2024-09-10T15:58:59.142041+0000","last_fullsized":"2024-09-10T15:58:59.142041+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":3327,"log_dups_size":0,"ondisk_log_size":3327,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13603782,"num_objects":3323,"num_object_clones":0,"num_object_copies":6646,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3323,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1432,"num_write_kb":5728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"36'3271","reported_seq":3374,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T15:58:59.177075+0000","last_change":"2024-09-10T15:58:45.933955+0000","last_active":"2024-09-10T15:58:59.177075+0000","last_peered":"2024-09-10T15:58:59.177075+0000","last_clean":"2024-09-10T15:58:59.177075+0000","last_became_active":"2024-09-10T15:58:25.767861+0000","last_became_peered":"2024-09-10T15:58:25.767861+0000","last_unstale":"2024-09-10T15:58:59.177075+0000","last_undegraded":"2024-09-10T15:58:59.177075+0000","last_fullsized":"2024-09-10T15:58:59.177075+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":3271,"log_dups_size":0,"ondisk_log_size":3271,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:22:24.003676+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13381632,"num_objects":3267,"num_object_clones":0,"num_object_copies":6534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3267,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1275,"num_write_kb":5100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"36'3214","reported_seq":3275,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T15:58:59.172731+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T15:58:59.172731+0000","last_peered":"2024-09-10T15:58:59.172731+0000","last_clean":"2024-09-10T15:58:59.172731+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T15:58:59.172731+0000","last_undegraded":"2024-09-10T15:58:59.172731+0000","last_fullsized":"2024-09-10T15:58:59.172731+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":3214,"log_dups_size":0,"ondisk_log_size":3214,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13164544,"num_objects":3214,"num_object_clones":0,"num_object_copies":6428,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3214,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3214,"num_write_kb":12856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"36'3498","reported_seq":3607,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T15:59:01.068223+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T15:59:01.068223+0000","last_peered":"2024-09-10T15:59:01.068223+0000","last_clean":"2024-09-10T15:59:01.068223+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T15:59:01.068223+0000","last_undegraded":"2024-09-10T15:59:01.068223+0000","last_fullsized":"2024-09-10T15:59:01.068223+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":3498,"log_dups_size":0,"ondisk_log_size":3498,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14299619,"num_objects":3492,"num_object_clones":0,"num_object_copies":6984,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3492,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":209,"num_write_kb":836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"36'3523","reported_seq":3634,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T15:59:01.061067+0000","last_change":"2024-09-10T15:59:00.194355+0000","last_active":"2024-09-10T15:59:01.061067+0000","last_peered":"2024-09-10T15:59:01.061067+0000","last_clean":"2024-09-10T15:59:01.061067+0000","last_became_active":"2024-09-10T15:58:24.768198+0000","last_became_peered":"2024-09-10T15:58:24.768198+0000","last_unstale":"2024-09-10T15:59:01.061067+0000","last_undegraded":"2024-09-10T15:59:01.061067+0000","last_fullsized":"2024-09-10T15:59:01.061067+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":3523,"log_dups_size":0,"ondisk_log_size":3523,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:51:05.573369+0000","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14410211,"num_objects":3519,"num_object_clones":0,"num_object_copies":7038,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3519,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":135,"num_write_kb":540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"36'3375","reported_seq":3443,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T15:59:01.055294+0000","last_change":"2024-09-10T15:58:56.103386+0000","last_active":"2024-09-10T15:59:01.055294+0000","last_peered":"2024-09-10T15:59:01.055294+0000","last_clean":"2024-09-10T15:59:01.055294+0000","last_became_active":"2024-09-10T15:58:24.768972+0000","last_became_peered":"2024-09-10T15:58:24.768972+0000","last_unstale":"2024-09-10T15:59:01.055294+0000","last_undegraded":"2024-09-10T15:59:01.055294+0000","last_fullsized":"2024-09-10T15:59:01.055294+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":3375,"log_dups_size":0,"ondisk_log_size":3375,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13800390,"num_objects":3371,"num_object_clones":0,"num_object_copies":6742,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3371,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3373,"num_write_kb":13480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"36'3483","reported_seq":3593,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T15:59:01.052841+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T15:59:01.052841+0000","last_peered":"2024-09-10T15:59:01.052841+0000","last_clean":"2024-09-10T15:59:01.052841+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T15:59:01.052841+0000","last_undegraded":"2024-09-10T15:59:01.052841+0000","last_fullsized":"2024-09-10T15:59:01.052841+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":3483,"log_dups_size":0,"ondisk_log_size":3483,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14258176,"num_objects":3481,"num_object_clones":0,"num_object_copies":6962,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3481,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":330,"num_write_kb":1320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T15:58:25.762375+0000","last_change":"2024-09-10T15:58:24.767197+0000","last_active":"2024-09-10T15:58:25.762375+0000","last_peered":"2024-09-10T15:58:25.762375+0000","last_clean":"2024-09-10T15:58:25.762375+0000","last_became_active":"2024-09-10T15:58:24.766691+0000","last_became_peered":"2024-09-10T15:58:24.766691+0000","last_unstale":"2024-09-10T15:58:25.762375+0000","last_undegraded":"2024-09-10T15:58:25.762375+0000","last_fullsized":"2024-09-10T15:58:25.762375+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:51:00.161051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"36'3153","reported_seq":3261,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T15:58:59.161355+0000","last_change":"2024-09-10T15:58:48.061941+0000","last_active":"2024-09-10T15:58:59.161355+0000","last_peered":"2024-09-10T15:58:59.161355+0000","last_clean":"2024-09-10T15:58:59.161355+0000","last_became_active":"2024-09-10T15:58:25.769634+0000","last_became_peered":"2024-09-10T15:58:25.769634+0000","last_unstale":"2024-09-10T15:58:59.161355+0000","last_undegraded":"2024-09-10T15:58:59.161355+0000","last_fullsized":"2024-09-10T15:58:59.161355+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":3153,"log_dups_size":0,"ondisk_log_size":3153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:24:15.030848+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12886499,"num_objects":3147,"num_object_clones":0,"num_object_copies":6294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3147,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":972,"num_write_kb":3888,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":74,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T15:58:25.768681+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T15:58:25.768681+0000","last_peered":"2024-09-10T15:58:25.768681+0000","last_clean":"2024-09-10T15:58:25.768681+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T15:58:25.768681+0000","last_undegraded":"2024-09-10T15:58:25.768681+0000","last_fullsized":"2024-09-10T15:58:25.768681+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"36'3427","reported_seq":3489,"reported_epoch":36,"state":"active+clean+scrubbing","last_fresh":"2024-09-10T15:59:01.061153+0000","last_change":"2024-09-10T15:59:00.989681+0000","last_active":"2024-09-10T15:59:01.061153+0000","last_peered":"2024-09-10T15:59:01.061153+0000","last_clean":"2024-09-10T15:59:01.061153+0000","last_became_active":"2024-09-10T15:58:24.768693+0000","last_became_peered":"2024-09-10T15:58:24.768693+0000","last_unstale":"2024-09-10T15:59:01.061153+0000","last_undegraded":"2024-09-10T15:59:01.061153+0000","last_fullsized":"2024-09-10T15:59:01.061153+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":3427,"log_dups_size":0,"ondisk_log_size":3427,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"scrubbing for 0s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14005190,"num_objects":3421,"num_object_clones":0,"num_object_copies":6842,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3421,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3425,"num_write_kb":13680,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":74,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T15:58:56.345927+0000","last_change":"2024-09-10T15:58:56.345818+0000","last_active":"2024-09-10T15:58:56.345927+0000","last_peered":"2024-09-10T15:58:56.345927+0000","last_clean":"2024-09-10T15:58:56.345927+0000","last_became_active":"2024-09-10T15:58:24.766316+0000","last_became_peered":"2024-09-10T15:58:24.766316+0000","last_unstale":"2024-09-10T15:58:56.345927+0000","last_undegraded":"2024-09-10T15:58:56.345927+0000","last_fullsized":"2024-09-10T15:58:56.345927+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:21:55.386457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"36'3267","reported_seq":3370,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T15:58:59.159791+0000","last_change":"2024-09-10T15:58:49.040907+0000","last_active":"2024-09-10T15:58:59.159791+0000","last_peered":"2024-09-10T15:58:59.159791+0000","last_clean":"2024-09-10T15:58:59.159791+0000","last_became_active":"2024-09-10T15:58:25.769424+0000","last_became_peered":"2024-09-10T15:58:25.769424+0000","last_unstale":"2024-09-10T15:58:59.159791+0000","last_undegraded":"2024-09-10T15:58:59.159791+0000","last_fullsized":"2024-09-10T15:58:59.159791+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":3267,"log_dups_size":0,"ondisk_log_size":3267,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:52:01.617164+0000","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13361635,"num_objects":3263,"num_object_clones":0,"num_object_copies":6526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3263,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":947,"num_write_kb":3788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":92,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T15:58:25.770150+0000","last_change":"2024-09-10T15:58:25.768489+0000","last_active":"2024-09-10T15:58:25.770150+0000","last_peered":"2024-09-10T15:58:25.770150+0000","last_clean":"2024-09-10T15:58:25.770150+0000","last_became_active":"2024-09-10T15:58:25.768117+0000","last_became_peered":"2024-09-10T15:58:25.768117+0000","last_unstale":"2024-09-10T15:58:25.770150+0000","last_undegraded":"2024-09-10T15:58:25.770150+0000","last_fullsized":"2024-09-10T15:58:25.770150+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:56:47.213564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"36'3304","reported_seq":3407,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T15:58:59.155610+0000","last_change":"2024-09-10T15:58:50.079160+0000","last_active":"2024-09-10T15:58:59.155610+0000","last_peered":"2024-09-10T15:58:59.155610+0000","last_clean":"2024-09-10T15:58:59.155610+0000","last_became_active":"2024-09-10T15:58:25.769762+0000","last_became_peered":"2024-09-10T15:58:25.769762+0000","last_unstale":"2024-09-10T15:58:59.155610+0000","last_undegraded":"2024-09-10T15:58:59.155610+0000","last_fullsized":"2024-09-10T15:58:59.155610+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":3304,"log_dups_size":0,"ondisk_log_size":3304,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:50:48.841755+0000","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13508608,"num_objects":3298,"num_object_clones":0,"num_object_copies":6596,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3298,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":858,"num_write_kb":3432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"36'3423","reported_seq":3530,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T15:59:01.067888+0000","last_change":"2024-09-10T15:59:00.927708+0000","last_active":"2024-09-10T15:59:01.067888+0000","last_peered":"2024-09-10T15:59:01.067888+0000","last_clean":"2024-09-10T15:59:01.067888+0000","last_became_active":"2024-09-10T15:58:24.768609+0000","last_became_peered":"2024-09-10T15:58:24.768609+0000","last_unstale":"2024-09-10T15:59:01.067888+0000","last_undegraded":"2024-09-10T15:59:01.067888+0000","last_fullsized":"2024-09-10T15:59:01.067888+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":3423,"log_dups_size":0,"ondisk_log_size":3423,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:02:25.176788+0000","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14012416,"num_objects":3421,"num_object_clones":0,"num_object_copies":6842,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3421,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":54,"num_write_kb":216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T15:58:25.762555+0000","last_change":"2024-09-10T15:58:24.767677+0000","last_active":"2024-09-10T15:58:25.762555+0000","last_peered":"2024-09-10T15:58:25.762555+0000","last_clean":"2024-09-10T15:58:25.762555+0000","last_became_active":"2024-09-10T15:58:24.767409+0000","last_became_peered":"2024-09-10T15:58:24.767409+0000","last_unstale":"2024-09-10T15:58:25.762555+0000","last_undegraded":"2024-09-10T15:58:25.762555+0000","last_fullsized":"2024-09-10T15:58:25.762555+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:35:35.763396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"36'3354","reported_seq":3425,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T15:59:01.024383+0000","last_change":"2024-09-10T15:58:56.233256+0000","last_active":"2024-09-10T15:59:01.024383+0000","last_peered":"2024-09-10T15:59:01.024383+0000","last_clean":"2024-09-10T15:59:01.024383+0000","last_became_active":"2024-09-10T15:58:24.769268+0000","last_became_peered":"2024-09-10T15:58:24.769268+0000","last_unstale":"2024-09-10T15:59:01.024383+0000","last_undegraded":"2024-09-10T15:59:01.024383+0000","last_fullsized":"2024-09-10T15:59:01.024383+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":3354,"log_dups_size":0,"ondisk_log_size":3354,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13706665,"num_objects":3349,"num_object_clones":0,"num_object_copies":6698,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3349,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3351,"num_write_kb":13388,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":74,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T15:58:25.769812+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T15:58:25.769812+0000","last_peered":"2024-09-10T15:58:25.769812+0000","last_clean":"2024-09-10T15:58:25.769812+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T15:58:25.769812+0000","last_undegraded":"2024-09-10T15:58:25.769812+0000","last_fullsized":"2024-09-10T15:58:25.769812+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"36'3531","reported_seq":3599,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T15:59:01.031962+0000","last_change":"2024-09-10T15:58:56.152194+0000","last_active":"2024-09-10T15:59:01.031962+0000","last_peered":"2024-09-10T15:59:01.031962+0000","last_clean":"2024-09-10T15:59:01.031962+0000","last_became_active":"2024-09-10T15:58:24.769074+0000","last_became_peered":"2024-09-10T15:58:24.769074+0000","last_unstale":"2024-09-10T15:59:01.031962+0000","last_undegraded":"2024-09-10T15:59:01.031962+0000","last_fullsized":"2024-09-10T15:59:01.031962+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":3531,"log_dups_size":0,"ondisk_log_size":3531,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14447558,"num_objects":3529,"num_object_clones":0,"num_object_copies":7058,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3529,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3529,"num_write_kb":14110,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T15:58:25.762992+0000","last_change":"2024-09-10T15:58:24.767867+0000","last_active":"2024-09-10T15:58:25.762992+0000","last_peered":"2024-09-10T15:58:25.762992+0000","last_clean":"2024-09-10T15:58:25.762992+0000","last_became_active":"2024-09-10T15:58:24.767615+0000","last_became_peered":"2024-09-10T15:58:24.767615+0000","last_unstale":"2024-09-10T15:58:25.762992+0000","last_undegraded":"2024-09-10T15:58:25.762992+0000","last_fullsized":"2024-09-10T15:58:25.762992+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:11:00.908668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":87,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T15:58:25.770511+0000","last_change":"2024-09-10T15:58:25.768847+0000","last_active":"2024-09-10T15:58:25.770511+0000","last_peered":"2024-09-10T15:58:25.770511+0000","last_clean":"2024-09-10T15:58:25.770511+0000","last_became_active":"2024-09-10T15:58:25.768040+0000","last_became_peered":"2024-09-10T15:58:25.768040+0000","last_unstale":"2024-09-10T15:58:25.770511+0000","last_undegraded":"2024-09-10T15:58:25.770511+0000","last_fullsized":"2024-09-10T15:58:25.770511+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:39:30.402776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"36'3466","reported_seq":3538,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T15:59:01.053039+0000","last_change":"2024-09-10T15:58:56.333935+0000","last_active":"2024-09-10T15:59:01.053039+0000","last_peered":"2024-09-10T15:59:01.053039+0000","last_clean":"2024-09-10T15:59:01.053039+0000","last_became_active":"2024-09-10T15:58:24.769616+0000","last_became_peered":"2024-09-10T15:58:24.769616+0000","last_unstale":"2024-09-10T15:59:01.053039+0000","last_undegraded":"2024-09-10T15:59:01.053039+0000","last_fullsized":"2024-09-10T15:59:01.053039+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":3466,"log_dups_size":0,"ondisk_log_size":3466,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14149516,"num_objects":3458,"num_object_clones":0,"num_object_copies":6916,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3458,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3463,"num_write_kb":13823,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":220857305,"num_objects":53937,"num_object_clones":0,"num_object_copies":107874,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53937,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":29951,"num_write_kb":119721,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":1932,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":439767040,"data_stored":439593616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":54000,"ondisk_log_size":54000,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986188,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":323016,"kb_used_data":205240,"kb_used_omap":4,"kb_used_meta":117755,"kb_avail":94048824,"statfs":{"total":96636764160,"available":96305995776,"internally_reserved":0,"allocated":210165760,"data_stored":209917348,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":120581802},"hb_peers":[0],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,2,2,6,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182417,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":345664,"kb_used_data":219632,"kb_used_omap":4,"kb_used_meta":126011,"kb_avail":94026176,"statfs":{"total":96636764160,"available":96282804224,"internally_reserved":0,"allocated":224903168,"data_stored":224654756,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":129035946},"hb_peers":[2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,5,3,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":225181696,"data_stored":225145566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":13524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":214470656,"data_stored":214434526,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:59:02.736 INFO:tasks.lost_unfound.ceph_manager:recovered! 2024-09-10T15:59:02.737 INFO:tasks.util.rados:rados -p unfound_pool get new_1 - 2024-09-10T15:59:02.737 DEBUG:teuthology.orchestra.run.smithi115:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool get new_1 - 2024-09-10T15:59:02.846 INFO:teuthology.orchestra.run.smithi115.stderr:error getting unfound_pool/new_1: (2) No such file or directory 2024-09-10T15:59:02.854 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T15:59:02.854 INFO:tasks.util.rados:rados -p unfound_pool get existed_1 - 2024-09-10T15:59:02.855 DEBUG:teuthology.orchestra.run.smithi115:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool get existed_1 - 2024-09-10T15:59:02.962 INFO:teuthology.orchestra.run.smithi115.stderr:error getting unfound_pool/existed_1: (2) No such file or directory 2024-09-10T15:59:02.963 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 30 20 57365 57345 7.46596 4.5625 0.017317 0.0104595 2024-09-10T15:59:02.973 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T15:59:02.973 INFO:tasks.util.rados:rados -p unfound_pool get existing_1 - 2024-09-10T15:59:02.973 DEBUG:teuthology.orchestra.run.smithi115:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool get existing_1 - 2024-09-10T15:59:03.073 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T15:59:03.073 INFO:teuthology.orchestra.run.smithi115.stdout:# 2024-09-10T15:59:03.073 INFO:teuthology.orchestra.run.smithi115.stdout:# /etc/fstab 2024-09-10T15:59:03.073 INFO:teuthology.orchestra.run.smithi115.stdout:# Created by anaconda on Wed Aug 28 20:55:26 2024 2024-09-10T15:59:03.073 INFO:teuthology.orchestra.run.smithi115.stdout:# 2024-09-10T15:59:03.074 INFO:teuthology.orchestra.run.smithi115.stdout:# Accessible filesystems, by reference, are maintained under '/dev/disk/'. 2024-09-10T15:59:03.074 INFO:teuthology.orchestra.run.smithi115.stdout:# See man pages fstab(5), findfs(8), mount(8) and/or blkid(8) for more info. 2024-09-10T15:59:03.074 INFO:teuthology.orchestra.run.smithi115.stdout:# 2024-09-10T15:59:03.074 INFO:teuthology.orchestra.run.smithi115.stdout:# After editing this file, run 'systemctl daemon-reload' to update systemd 2024-09-10T15:59:03.074 INFO:teuthology.orchestra.run.smithi115.stdout:# units generated from this file. 2024-09-10T15:59:03.074 INFO:teuthology.orchestra.run.smithi115.stdout:# 2024-09-10T15:59:03.074 INFO:teuthology.orchestra.run.smithi115.stdout:UUID=fa8d2923-554d-41e2-ac4a-c752ff079ceb / ext4 defaults 1 1 2024-09-10T15:59:03.074 INFO:teuthology.orchestra.run.smithi115.stdout:/dev/vg_nvme/lv_5 /var/lib/ceph xfs defaults,nofail 0 0 2024-09-10T15:59:03.084 INFO:tasks.util.rados:rados -p unfound_pool get new_2 - 2024-09-10T15:59:03.084 DEBUG:teuthology.orchestra.run.smithi115:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool get new_2 - 2024-09-10T15:59:03.186 INFO:teuthology.orchestra.run.smithi115.stderr:error getting unfound_pool/new_2: (2) No such file or directory 2024-09-10T15:59:03.196 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T15:59:03.197 INFO:tasks.util.rados:rados -p unfound_pool get existed_2 - 2024-09-10T15:59:03.197 DEBUG:teuthology.orchestra.run.smithi115:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool get existed_2 - 2024-09-10T15:59:03.303 INFO:teuthology.orchestra.run.smithi115.stderr:error getting unfound_pool/existed_2: (2) No such file or directory 2024-09-10T15:59:03.310 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T15:59:03.311 INFO:tasks.util.rados:rados -p unfound_pool get existing_2 - 2024-09-10T15:59:03.311 DEBUG:teuthology.orchestra.run.smithi115:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool get existing_2 - 2024-09-10T15:59:03.421 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T15:59:03.422 INFO:teuthology.orchestra.run.smithi115.stdout:# 2024-09-10T15:59:03.422 INFO:teuthology.orchestra.run.smithi115.stdout:# /etc/fstab 2024-09-10T15:59:03.422 INFO:teuthology.orchestra.run.smithi115.stdout:# Created by anaconda on Wed Aug 28 20:55:26 2024 2024-09-10T15:59:03.422 INFO:teuthology.orchestra.run.smithi115.stdout:# 2024-09-10T15:59:03.422 INFO:teuthology.orchestra.run.smithi115.stdout:# Accessible filesystems, by reference, are maintained under '/dev/disk/'. 2024-09-10T15:59:03.422 INFO:teuthology.orchestra.run.smithi115.stdout:# See man pages fstab(5), findfs(8), mount(8) and/or blkid(8) for more info. 2024-09-10T15:59:03.422 INFO:teuthology.orchestra.run.smithi115.stdout:# 2024-09-10T15:59:03.422 INFO:teuthology.orchestra.run.smithi115.stdout:# After editing this file, run 'systemctl daemon-reload' to update systemd 2024-09-10T15:59:03.422 INFO:teuthology.orchestra.run.smithi115.stdout:# units generated from this file. 2024-09-10T15:59:03.422 INFO:teuthology.orchestra.run.smithi115.stdout:# 2024-09-10T15:59:03.422 INFO:teuthology.orchestra.run.smithi115.stdout:UUID=fa8d2923-554d-41e2-ac4a-c752ff079ceb / ext4 defaults 1 1 2024-09-10T15:59:03.423 INFO:teuthology.orchestra.run.smithi115.stdout:/dev/vg_nvme/lv_5 /var/lib/ceph xfs defaults,nofail 0 0 2024-09-10T15:59:03.427 INFO:tasks.util.rados:rados -p unfound_pool get new_3 - 2024-09-10T15:59:03.428 DEBUG:teuthology.orchestra.run.smithi115:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool get new_3 - 2024-09-10T15:59:03.518 INFO:teuthology.orchestra.run.smithi115.stderr:error getting unfound_pool/new_3: (2) No such file or directory 2024-09-10T15:59:03.527 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T15:59:03.527 INFO:tasks.util.rados:rados -p unfound_pool get existed_3 - 2024-09-10T15:59:03.528 DEBUG:teuthology.orchestra.run.smithi115:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool get existed_3 - 2024-09-10T15:59:03.619 INFO:teuthology.orchestra.run.smithi115.stderr:error getting unfound_pool/existed_3: (2) No such file or directory 2024-09-10T15:59:03.636 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T15:59:03.636 INFO:tasks.util.rados:rados -p unfound_pool get existing_3 - 2024-09-10T15:59:03.636 DEBUG:teuthology.orchestra.run.smithi115:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool get existing_3 - 2024-09-10T15:59:03.738 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T15:59:03.738 INFO:teuthology.orchestra.run.smithi115.stdout:# 2024-09-10T15:59:03.738 INFO:teuthology.orchestra.run.smithi115.stdout:# /etc/fstab 2024-09-10T15:59:03.738 INFO:teuthology.orchestra.run.smithi115.stdout:# Created by anaconda on Wed Aug 28 20:55:26 2024 2024-09-10T15:59:03.738 INFO:teuthology.orchestra.run.smithi115.stdout:# 2024-09-10T15:59:03.738 INFO:teuthology.orchestra.run.smithi115.stdout:# Accessible filesystems, by reference, are maintained under '/dev/disk/'. 2024-09-10T15:59:03.738 INFO:teuthology.orchestra.run.smithi115.stdout:# See man pages fstab(5), findfs(8), mount(8) and/or blkid(8) for more info. 2024-09-10T15:59:03.738 INFO:teuthology.orchestra.run.smithi115.stdout:# 2024-09-10T15:59:03.738 INFO:teuthology.orchestra.run.smithi115.stdout:# After editing this file, run 'systemctl daemon-reload' to update systemd 2024-09-10T15:59:03.738 INFO:teuthology.orchestra.run.smithi115.stdout:# units generated from this file. 2024-09-10T15:59:03.738 INFO:teuthology.orchestra.run.smithi115.stdout:# 2024-09-10T15:59:03.739 INFO:teuthology.orchestra.run.smithi115.stdout:UUID=fa8d2923-554d-41e2-ac4a-c752ff079ceb / ext4 defaults 1 1 2024-09-10T15:59:03.739 INFO:teuthology.orchestra.run.smithi115.stdout:/dev/vg_nvme/lv_5 /var/lib/ceph xfs defaults,nofail 0 0 2024-09-10T15:59:03.751 INFO:tasks.util.rados:rados -p unfound_pool get new_4 - 2024-09-10T15:59:03.751 DEBUG:teuthology.orchestra.run.smithi115:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool get new_4 - 2024-09-10T15:59:03.842 INFO:teuthology.orchestra.run.smithi115.stderr:error getting unfound_pool/new_4: (2) No such file or directory 2024-09-10T15:59:03.854 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T15:59:03.854 INFO:tasks.util.rados:rados -p unfound_pool get existed_4 - 2024-09-10T15:59:03.854 DEBUG:teuthology.orchestra.run.smithi115:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool get existed_4 - 2024-09-10T15:59:03.947 INFO:teuthology.orchestra.run.smithi115.stderr:error getting unfound_pool/existed_4: (2) No such file or directory 2024-09-10T15:59:03.959 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T15:59:03.959 INFO:tasks.util.rados:rados -p unfound_pool get existing_4 - 2024-09-10T15:59:03.959 DEBUG:teuthology.orchestra.run.smithi115:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool get existing_4 - 2024-09-10T15:59:03.964 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 31 20 58541 58521 7.3733 4.59375 0.00874186 0.0105896 2024-09-10T15:59:04.053 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T15:59:04.053 INFO:teuthology.orchestra.run.smithi115.stdout:# 2024-09-10T15:59:04.053 INFO:teuthology.orchestra.run.smithi115.stdout:# /etc/fstab 2024-09-10T15:59:04.053 INFO:teuthology.orchestra.run.smithi115.stdout:# Created by anaconda on Wed Aug 28 20:55:26 2024 2024-09-10T15:59:04.053 INFO:teuthology.orchestra.run.smithi115.stdout:# 2024-09-10T15:59:04.053 INFO:teuthology.orchestra.run.smithi115.stdout:# Accessible filesystems, by reference, are maintained under '/dev/disk/'. 2024-09-10T15:59:04.053 INFO:teuthology.orchestra.run.smithi115.stdout:# See man pages fstab(5), findfs(8), mount(8) and/or blkid(8) for more info. 2024-09-10T15:59:04.053 INFO:teuthology.orchestra.run.smithi115.stdout:# 2024-09-10T15:59:04.053 INFO:teuthology.orchestra.run.smithi115.stdout:# After editing this file, run 'systemctl daemon-reload' to update systemd 2024-09-10T15:59:04.053 INFO:teuthology.orchestra.run.smithi115.stdout:# units generated from this file. 2024-09-10T15:59:04.054 INFO:teuthology.orchestra.run.smithi115.stdout:# 2024-09-10T15:59:04.054 INFO:teuthology.orchestra.run.smithi115.stdout:UUID=fa8d2923-554d-41e2-ac4a-c752ff079ceb / ext4 defaults 1 1 2024-09-10T15:59:04.054 INFO:teuthology.orchestra.run.smithi115.stdout:/dev/vg_nvme/lv_5 /var/lib/ceph xfs defaults,nofail 0 0 2024-09-10T15:59:04.057 INFO:tasks.util.rados:rados -p unfound_pool get new_5 - 2024-09-10T15:59:04.058 DEBUG:teuthology.orchestra.run.smithi115:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool get new_5 - 2024-09-10T15:59:04.198 INFO:teuthology.orchestra.run.smithi115.stderr:error getting unfound_pool/new_5: (2) No such file or directory 2024-09-10T15:59:04.208 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T15:59:04.208 INFO:tasks.util.rados:rados -p unfound_pool get existed_5 - 2024-09-10T15:59:04.208 DEBUG:teuthology.orchestra.run.smithi115:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool get existed_5 - 2024-09-10T15:59:04.302 INFO:teuthology.orchestra.run.smithi115.stderr:error getting unfound_pool/existed_5: (2) No such file or directory 2024-09-10T15:59:04.312 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T15:59:04.313 INFO:tasks.util.rados:rados -p unfound_pool get existing_5 - 2024-09-10T15:59:04.313 DEBUG:teuthology.orchestra.run.smithi115:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool get existing_5 - 2024-09-10T15:59:04.407 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T15:59:04.407 INFO:teuthology.orchestra.run.smithi115.stdout:# 2024-09-10T15:59:04.407 INFO:teuthology.orchestra.run.smithi115.stdout:# /etc/fstab 2024-09-10T15:59:04.407 INFO:teuthology.orchestra.run.smithi115.stdout:# Created by anaconda on Wed Aug 28 20:55:26 2024 2024-09-10T15:59:04.407 INFO:teuthology.orchestra.run.smithi115.stdout:# 2024-09-10T15:59:04.407 INFO:teuthology.orchestra.run.smithi115.stdout:# Accessible filesystems, by reference, are maintained under '/dev/disk/'. 2024-09-10T15:59:04.407 INFO:teuthology.orchestra.run.smithi115.stdout:# See man pages fstab(5), findfs(8), mount(8) and/or blkid(8) for more info. 2024-09-10T15:59:04.408 INFO:teuthology.orchestra.run.smithi115.stdout:# 2024-09-10T15:59:04.408 INFO:teuthology.orchestra.run.smithi115.stdout:# After editing this file, run 'systemctl daemon-reload' to update systemd 2024-09-10T15:59:04.408 INFO:teuthology.orchestra.run.smithi115.stdout:# units generated from this file. 2024-09-10T15:59:04.408 INFO:teuthology.orchestra.run.smithi115.stdout:# 2024-09-10T15:59:04.408 INFO:teuthology.orchestra.run.smithi115.stdout:UUID=fa8d2923-554d-41e2-ac4a-c752ff079ceb / ext4 defaults 1 1 2024-09-10T15:59:04.408 INFO:teuthology.orchestra.run.smithi115.stdout:/dev/vg_nvme/lv_5 /var/lib/ceph xfs defaults,nofail 0 0 2024-09-10T15:59:04.413 INFO:tasks.util.rados:rados -p unfound_pool get new_6 - 2024-09-10T15:59:04.413 DEBUG:teuthology.orchestra.run.smithi115:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool get new_6 - 2024-09-10T15:59:04.512 INFO:teuthology.orchestra.run.smithi115.stderr:error getting unfound_pool/new_6: (2) No such file or directory 2024-09-10T15:59:04.520 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T15:59:04.520 INFO:tasks.util.rados:rados -p unfound_pool get existed_6 - 2024-09-10T15:59:04.520 DEBUG:teuthology.orchestra.run.smithi115:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool get existed_6 - 2024-09-10T15:59:04.616 INFO:teuthology.orchestra.run.smithi115.stderr:error getting unfound_pool/existed_6: (2) No such file or directory 2024-09-10T15:59:04.626 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T15:59:04.626 INFO:tasks.util.rados:rados -p unfound_pool get existing_6 - 2024-09-10T15:59:04.626 DEBUG:teuthology.orchestra.run.smithi115:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool get existing_6 - 2024-09-10T15:59:04.726 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T15:59:04.727 INFO:teuthology.orchestra.run.smithi115.stdout:# 2024-09-10T15:59:04.727 INFO:teuthology.orchestra.run.smithi115.stdout:# /etc/fstab 2024-09-10T15:59:04.727 INFO:teuthology.orchestra.run.smithi115.stdout:# Created by anaconda on Wed Aug 28 20:55:26 2024 2024-09-10T15:59:04.727 INFO:teuthology.orchestra.run.smithi115.stdout:# 2024-09-10T15:59:04.727 INFO:teuthology.orchestra.run.smithi115.stdout:# Accessible filesystems, by reference, are maintained under '/dev/disk/'. 2024-09-10T15:59:04.727 INFO:teuthology.orchestra.run.smithi115.stdout:# See man pages fstab(5), findfs(8), mount(8) and/or blkid(8) for more info. 2024-09-10T15:59:04.727 INFO:teuthology.orchestra.run.smithi115.stdout:# 2024-09-10T15:59:04.727 INFO:teuthology.orchestra.run.smithi115.stdout:# After editing this file, run 'systemctl daemon-reload' to update systemd 2024-09-10T15:59:04.727 INFO:teuthology.orchestra.run.smithi115.stdout:# units generated from this file. 2024-09-10T15:59:04.727 INFO:teuthology.orchestra.run.smithi115.stdout:# 2024-09-10T15:59:04.727 INFO:teuthology.orchestra.run.smithi115.stdout:UUID=fa8d2923-554d-41e2-ac4a-c752ff079ceb / ext4 defaults 1 1 2024-09-10T15:59:04.727 INFO:teuthology.orchestra.run.smithi115.stdout:/dev/vg_nvme/lv_5 /var/lib/ceph xfs defaults,nofail 0 0 2024-09-10T15:59:04.739 INFO:tasks.util.rados:rados -p unfound_pool get new_7 - 2024-09-10T15:59:04.739 DEBUG:teuthology.orchestra.run.smithi115:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool get new_7 - 2024-09-10T15:59:04.829 INFO:teuthology.orchestra.run.smithi115.stderr:error getting unfound_pool/new_7: (2) No such file or directory 2024-09-10T15:59:04.835 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T15:59:04.836 INFO:tasks.util.rados:rados -p unfound_pool get existed_7 - 2024-09-10T15:59:04.836 DEBUG:teuthology.orchestra.run.smithi115:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool get existed_7 - 2024-09-10T15:59:04.929 INFO:teuthology.orchestra.run.smithi115.stderr:error getting unfound_pool/existed_7: (2) No such file or directory 2024-09-10T15:59:04.933 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T15:59:04.933 INFO:tasks.util.rados:rados -p unfound_pool get existing_7 - 2024-09-10T15:59:04.933 DEBUG:teuthology.orchestra.run.smithi115:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool get existing_7 - 2024-09-10T15:59:04.964 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 32 20 59648 59628 7.27801 4.32422 0.0247426 0.0107292 2024-09-10T15:59:05.039 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T15:59:05.039 INFO:teuthology.orchestra.run.smithi115.stdout:# 2024-09-10T15:59:05.039 INFO:teuthology.orchestra.run.smithi115.stdout:# /etc/fstab 2024-09-10T15:59:05.039 INFO:teuthology.orchestra.run.smithi115.stdout:# Created by anaconda on Wed Aug 28 20:55:26 2024 2024-09-10T15:59:05.039 INFO:teuthology.orchestra.run.smithi115.stdout:# 2024-09-10T15:59:05.039 INFO:teuthology.orchestra.run.smithi115.stdout:# Accessible filesystems, by reference, are maintained under '/dev/disk/'. 2024-09-10T15:59:05.040 INFO:teuthology.orchestra.run.smithi115.stdout:# See man pages fstab(5), findfs(8), mount(8) and/or blkid(8) for more info. 2024-09-10T15:59:05.040 INFO:teuthology.orchestra.run.smithi115.stdout:# 2024-09-10T15:59:05.040 INFO:teuthology.orchestra.run.smithi115.stdout:# After editing this file, run 'systemctl daemon-reload' to update systemd 2024-09-10T15:59:05.040 INFO:teuthology.orchestra.run.smithi115.stdout:# units generated from this file. 2024-09-10T15:59:05.040 INFO:teuthology.orchestra.run.smithi115.stdout:# 2024-09-10T15:59:05.040 INFO:teuthology.orchestra.run.smithi115.stdout:UUID=fa8d2923-554d-41e2-ac4a-c752ff079ceb / ext4 defaults 1 1 2024-09-10T15:59:05.041 INFO:teuthology.orchestra.run.smithi115.stdout:/dev/vg_nvme/lv_5 /var/lib/ceph xfs defaults,nofail 0 0 2024-09-10T15:59:05.049 INFO:tasks.util.rados:rados -p unfound_pool get new_8 - 2024-09-10T15:59:05.049 DEBUG:teuthology.orchestra.run.smithi115:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool get new_8 - 2024-09-10T15:59:05.184 INFO:teuthology.orchestra.run.smithi115.stderr:error getting unfound_pool/new_8: (2) No such file or directory 2024-09-10T15:59:05.193 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T15:59:05.194 INFO:tasks.util.rados:rados -p unfound_pool get existed_8 - 2024-09-10T15:59:05.194 DEBUG:teuthology.orchestra.run.smithi115:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool get existed_8 - 2024-09-10T15:59:05.286 INFO:teuthology.orchestra.run.smithi115.stderr:error getting unfound_pool/existed_8: (2) No such file or directory 2024-09-10T15:59:05.300 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T15:59:05.301 INFO:tasks.util.rados:rados -p unfound_pool get existing_8 - 2024-09-10T15:59:05.301 DEBUG:teuthology.orchestra.run.smithi115:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool get existing_8 - 2024-09-10T15:59:05.401 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T15:59:05.401 INFO:teuthology.orchestra.run.smithi115.stdout:# 2024-09-10T15:59:05.401 INFO:teuthology.orchestra.run.smithi115.stdout:# /etc/fstab 2024-09-10T15:59:05.401 INFO:teuthology.orchestra.run.smithi115.stdout:# Created by anaconda on Wed Aug 28 20:55:26 2024 2024-09-10T15:59:05.401 INFO:teuthology.orchestra.run.smithi115.stdout:# 2024-09-10T15:59:05.401 INFO:teuthology.orchestra.run.smithi115.stdout:# Accessible filesystems, by reference, are maintained under '/dev/disk/'. 2024-09-10T15:59:05.403 INFO:teuthology.orchestra.run.smithi115.stdout:# See man pages fstab(5), findfs(8), mount(8) and/or blkid(8) for more info. 2024-09-10T15:59:05.404 INFO:teuthology.orchestra.run.smithi115.stdout:# 2024-09-10T15:59:05.404 INFO:teuthology.orchestra.run.smithi115.stdout:# After editing this file, run 'systemctl daemon-reload' to update systemd 2024-09-10T15:59:05.404 INFO:teuthology.orchestra.run.smithi115.stdout:# units generated from this file. 2024-09-10T15:59:05.404 INFO:teuthology.orchestra.run.smithi115.stdout:# 2024-09-10T15:59:05.404 INFO:teuthology.orchestra.run.smithi115.stdout:UUID=fa8d2923-554d-41e2-ac4a-c752ff079ceb / ext4 defaults 1 1 2024-09-10T15:59:05.404 INFO:teuthology.orchestra.run.smithi115.stdout:/dev/vg_nvme/lv_5 /var/lib/ceph xfs defaults,nofail 0 0 2024-09-10T15:59:05.412 INFO:tasks.util.rados:rados -p unfound_pool get new_9 - 2024-09-10T15:59:05.412 DEBUG:teuthology.orchestra.run.smithi115:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool get new_9 - 2024-09-10T15:59:05.499 INFO:teuthology.orchestra.run.smithi115.stderr:error getting unfound_pool/new_9: (2) No such file or directory 2024-09-10T15:59:05.505 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T15:59:05.505 INFO:tasks.util.rados:rados -p unfound_pool get existed_9 - 2024-09-10T15:59:05.505 DEBUG:teuthology.orchestra.run.smithi115:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool get existed_9 - 2024-09-10T15:59:05.597 INFO:teuthology.orchestra.run.smithi115.stderr:error getting unfound_pool/existed_9: (2) No such file or directory 2024-09-10T15:59:05.609 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T15:59:05.609 INFO:tasks.util.rados:rados -p unfound_pool get existing_9 - 2024-09-10T15:59:05.610 DEBUG:teuthology.orchestra.run.smithi115:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool get existing_9 - 2024-09-10T15:59:05.709 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T15:59:05.709 INFO:teuthology.orchestra.run.smithi115.stdout:# 2024-09-10T15:59:05.709 INFO:teuthology.orchestra.run.smithi115.stdout:# /etc/fstab 2024-09-10T15:59:05.710 INFO:teuthology.orchestra.run.smithi115.stdout:# Created by anaconda on Wed Aug 28 20:55:26 2024 2024-09-10T15:59:05.710 INFO:teuthology.orchestra.run.smithi115.stdout:# 2024-09-10T15:59:05.710 INFO:teuthology.orchestra.run.smithi115.stdout:# Accessible filesystems, by reference, are maintained under '/dev/disk/'. 2024-09-10T15:59:05.710 INFO:teuthology.orchestra.run.smithi115.stdout:# See man pages fstab(5), findfs(8), mount(8) and/or blkid(8) for more info. 2024-09-10T15:59:05.710 INFO:teuthology.orchestra.run.smithi115.stdout:# 2024-09-10T15:59:05.710 INFO:teuthology.orchestra.run.smithi115.stdout:# After editing this file, run 'systemctl daemon-reload' to update systemd 2024-09-10T15:59:05.710 INFO:teuthology.orchestra.run.smithi115.stdout:# units generated from this file. 2024-09-10T15:59:05.710 INFO:teuthology.orchestra.run.smithi115.stdout:# 2024-09-10T15:59:05.710 INFO:teuthology.orchestra.run.smithi115.stdout:UUID=fa8d2923-554d-41e2-ac4a-c752ff079ceb / ext4 defaults 1 1 2024-09-10T15:59:05.710 INFO:teuthology.orchestra.run.smithi115.stdout:/dev/vg_nvme/lv_5 /var/lib/ceph xfs defaults,nofail 0 0 2024-09-10T15:59:05.716 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 1 2024-09-10T15:59:05.964 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 33 20 60765 60745 7.18968 4.36328 0.0202735 0.0108603 2024-09-10T15:59:06.297 INFO:tasks.ceph.mon.a.smithi115.stderr:2024-09-10T15:59:06.296+0000 7fa206bdb640 -1 mon.a@0(leader).osd e36 definitely_dead 0 2024-09-10T15:59:06.964 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 34 20 62054 62034 7.1263 5.03516 0.0129545 0.0109574 2024-09-10T15:59:07.073 INFO:teuthology.orchestra.run.smithi115.stderr:marked in osd.1. 2024-09-10T15:59:07.091 INFO:tasks.ceph.osd.1:Restarting daemon 2024-09-10T15:59:07.092 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-09-10T15:59:07.095 INFO:tasks.ceph.osd.1:Started 2024-09-10T15:59:07.095 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-10T15:59:07.193 INFO:teuthology.orchestra.run.smithi115.stderr:no valid command found; 10 closest matches: 2024-09-10T15:59:07.194 INFO:teuthology.orchestra.run.smithi115.stderr:0 2024-09-10T15:59:07.194 INFO:teuthology.orchestra.run.smithi115.stderr:1 2024-09-10T15:59:07.194 INFO:teuthology.orchestra.run.smithi115.stderr:2 2024-09-10T15:59:07.194 INFO:teuthology.orchestra.run.smithi115.stderr:abort 2024-09-10T15:59:07.194 INFO:teuthology.orchestra.run.smithi115.stderr:assert 2024-09-10T15:59:07.194 INFO:teuthology.orchestra.run.smithi115.stderr:config diff 2024-09-10T15:59:07.194 INFO:teuthology.orchestra.run.smithi115.stderr:config diff get 2024-09-10T15:59:07.194 INFO:teuthology.orchestra.run.smithi115.stderr:config get 2024-09-10T15:59:07.194 INFO:teuthology.orchestra.run.smithi115.stderr:config help [] 2024-09-10T15:59:07.194 INFO:teuthology.orchestra.run.smithi115.stderr:config set ... 2024-09-10T15:59:07.194 INFO:teuthology.orchestra.run.smithi115.stderr:admin_socket: invalid command 2024-09-10T15:59:07.196 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-10T15:59:07.196 INFO:tasks.lost_unfound.ceph_manager:waiting on admin_socket for osd-1, ['dump_ops_in_flight'] 2024-09-10T15:59:07.964 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 35 20 62209 62189 6.93998 0.605469 0.0155205 0.0109678 2024-09-10T15:59:08.006 INFO:tasks.ceph.osd.1.smithi115.stderr:2024-09-10T15:59:08.005+0000 7ffa06ea5640 -1 Falling back to public interface 2024-09-10T15:59:08.964 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 36 20 62209 62189 6.7472 0 - 0.0109678 2024-09-10T15:59:09.965 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 37 20 63304 63284 6.68044 2.13867 0.0241826 0.0116898 2024-09-10T15:59:10.965 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 38 20 64622 64602 6.64011 5.14844 0.0189866 0.01176 2024-09-10T15:59:11.965 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 39 20 65858 65838 6.59364 4.82812 0.00735609 0.0118432 2024-09-10T15:59:12.197 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-10T15:59:12.365 INFO:tasks.ceph.osd.1.smithi115.stderr:2024-09-10T15:59:12.363+0000 7ffa06ea5640 -1 osd.1 26 log_to_monitors true 2024-09-10T15:59:12.373 INFO:teuthology.orchestra.run.smithi115.stderr:no valid command found; 10 closest matches: 2024-09-10T15:59:12.373 INFO:teuthology.orchestra.run.smithi115.stderr:0 2024-09-10T15:59:12.373 INFO:teuthology.orchestra.run.smithi115.stderr:1 2024-09-10T15:59:12.373 INFO:teuthology.orchestra.run.smithi115.stderr:2 2024-09-10T15:59:12.373 INFO:teuthology.orchestra.run.smithi115.stderr:abort 2024-09-10T15:59:12.374 INFO:teuthology.orchestra.run.smithi115.stderr:assert 2024-09-10T15:59:12.374 INFO:teuthology.orchestra.run.smithi115.stderr:bluefs debug_inject_read_zeros 2024-09-10T15:59:12.374 INFO:teuthology.orchestra.run.smithi115.stderr:bluefs files list 2024-09-10T15:59:12.374 INFO:teuthology.orchestra.run.smithi115.stderr:bluefs stats 2024-09-10T15:59:12.374 INFO:teuthology.orchestra.run.smithi115.stderr:bluestore allocator dump block 2024-09-10T15:59:12.374 INFO:teuthology.orchestra.run.smithi115.stderr:bluestore allocator fragmentation block 2024-09-10T15:59:12.374 INFO:teuthology.orchestra.run.smithi115.stderr:admin_socket: invalid command 2024-09-10T15:59:12.376 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-10T15:59:12.376 INFO:tasks.lost_unfound.ceph_manager:waiting on admin_socket for osd-1, ['dump_ops_in_flight'] 2024-09-10T15:59:12.965 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout:2024-09-10T15:59:12.964545+0000 min lat: 0.00225581 max lat: 2.06047 avg lat: 0.0119243 2024-09-10T15:59:12.965 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-10T15:59:12.965 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 40 20 67086 67066 6.54871 4.79688 0.0166695 0.0119243 2024-09-10T15:59:13.132 INFO:tasks.ceph.osd.1.smithi115.stderr:2024-09-10T15:59:13.129+0000 7ff9fec5b640 -1 osd.1 26 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T15:59:13.965 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 41 20 68337 68317 6.50817 4.88672 0.0160862 0.0119985 2024-09-10T15:59:14.965 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 42 20 68551 68531 6.37311 0.835938 0.00740284 0.0120113 2024-09-10T15:59:15.965 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 43 20 68551 68531 6.2249 0 - 0.0120113 2024-09-10T15:59:16.965 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 44 20 68987 68967 6.12213 0.851562 0.0428851 0.0127539 2024-09-10T15:59:17.377 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-10T15:59:17.584 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-10T15:59:17.584 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-09-10T15:59:17.584 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-10T15:59:17.584 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_repop(client.15002.0:69362 2.0 e43/42 2:0f0654ac:::benchmark_data_smithi115_43363_object69361:head v 43'4415, mlcod=23'4)", 2024-09-10T15:59:17.585 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-10T15:59:17.562289+0000", 2024-09-10T15:59:17.585 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.020959821, 2024-09-10T15:59:17.585 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.020985736000000001, 2024-09-10T15:59:17.585 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-10T15:59:17.585 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-10T15:59:17.585 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-10T15:59:17.585 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-10T15:59:17.585 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-10T15:59:17.585 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-10T15:59:17.585 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-10T15:59:17.562289+0000", 2024-09-10T15:59:17.585 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-10T15:59:17.586 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-10T15:59:17.586 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-10T15:59:17.586 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-10T15:59:17.586 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-10T15:59:17.562289+0000", 2024-09-10T15:59:17.586 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-10T15:59:17.586 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-10T15:59:17.586 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-10T15:59:17.586 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-10T15:59:17.586 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-10T15:59:17.562290+0000", 2024-09-10T15:59:17.586 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9600000000000008e-07 2024-09-10T15:59:17.586 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-10T15:59:17.586 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-10T15:59:17.586 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-10T15:59:17.587 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-10T15:59:17.562301+0000", 2024-09-10T15:59:17.587 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0753e-05 2024-09-10T15:59:17.587 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-10T15:59:17.587 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-10T15:59:17.587 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-10T15:59:17.587 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-10T15:59:17.562314+0000", 2024-09-10T15:59:17.587 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3054000000000001e-05 2024-09-10T15:59:17.587 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-10T15:59:17.587 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-10T15:59:17.587 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-10T15:59:17.587 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-10T15:59:17.562321+0000", 2024-09-10T15:59:17.587 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.3529999999999997e-06 2024-09-10T15:59:17.587 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-10T15:59:17.588 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-10T15:59:17.588 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-10T15:59:17.588 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-10T15:59:17.562392+0000", 2024-09-10T15:59:17.588 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.0703000000000004e-05 2024-09-10T15:59:17.588 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-10T15:59:17.588 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-10T15:59:17.588 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-10T15:59:17.588 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-10T15:59:17.562415+0000", 2024-09-10T15:59:17.588 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2867e-05 2024-09-10T15:59:17.588 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-10T15:59:17.588 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-10T15:59:17.588 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-10T15:59:17.588 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-10T15:59:17.589 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-10T15:59:17.589 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_repop(client.15002.0:69363 2.5 e43/42 2:a17ab5b7:::benchmark_data_smithi115_43363_object69362:head v 43'4432, mlcod=23'2)", 2024-09-10T15:59:17.589 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-10T15:59:17.566841+0000", 2024-09-10T15:59:17.589 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.016407499999999998, 2024-09-10T15:59:17.589 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.016486048999999999, 2024-09-10T15:59:17.589 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-10T15:59:17.589 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-10T15:59:17.589 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-10T15:59:17.589 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-10T15:59:17.589 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-10T15:59:17.589 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-10T15:59:17.589 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-10T15:59:17.566841+0000", 2024-09-10T15:59:17.589 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-10T15:59:17.590 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-10T15:59:17.590 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-10T15:59:17.590 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-10T15:59:17.590 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-10T15:59:17.566841+0000", 2024-09-10T15:59:17.590 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-10T15:59:17.590 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-10T15:59:17.590 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-10T15:59:17.590 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-10T15:59:17.590 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-10T15:59:17.566842+0000", 2024-09-10T15:59:17.590 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9099999999999991e-07 2024-09-10T15:59:17.590 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-10T15:59:17.590 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-10T15:59:17.590 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-10T15:59:17.590 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-10T15:59:17.566851+0000", 2024-09-10T15:59:17.591 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.7960000000000001e-06 2024-09-10T15:59:17.591 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-10T15:59:17.591 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-10T15:59:17.591 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-10T15:59:17.591 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-10T15:59:17.566861+0000", 2024-09-10T15:59:17.591 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0132999999999999e-05 2024-09-10T15:59:17.591 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-10T15:59:17.591 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-10T15:59:17.591 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-10T15:59:17.591 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-10T15:59:17.566869+0000", 2024-09-10T15:59:17.591 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2049999999999996e-06 2024-09-10T15:59:17.591 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-10T15:59:17.591 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-10T15:59:17.592 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-10T15:59:17.592 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-10T15:59:17.569593+0000", 2024-09-10T15:59:17.592 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0027248060000000002 2024-09-10T15:59:17.592 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-10T15:59:17.592 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-10T15:59:17.592 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-10T15:59:17.592 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-10T15:59:17.569616+0000", 2024-09-10T15:59:17.592 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2387000000000001e-05 2024-09-10T15:59:17.592 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-10T15:59:17.592 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-10T15:59:17.592 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-10T15:59:17.592 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-10T15:59:17.592 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-10T15:59:17.592 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_repop(client.15002.0:69365 2.0 e43/42 2:0a1d0817:::benchmark_data_smithi115_43363_object69364:head v 43'4416, mlcod=23'4)", 2024-09-10T15:59:17.593 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-10T15:59:17.569251+0000", 2024-09-10T15:59:17.593 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.013998333, 2024-09-10T15:59:17.593 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.014121458999999999, 2024-09-10T15:59:17.593 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-10T15:59:17.593 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-10T15:59:17.593 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-10T15:59:17.593 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-10T15:59:17.593 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-10T15:59:17.593 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-10T15:59:17.593 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-10T15:59:17.569251+0000", 2024-09-10T15:59:17.593 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-10T15:59:17.593 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-10T15:59:17.593 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-10T15:59:17.594 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-10T15:59:17.594 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-10T15:59:17.569251+0000", 2024-09-10T15:59:17.594 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-10T15:59:17.594 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-10T15:59:17.594 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-10T15:59:17.594 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-10T15:59:17.594 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-10T15:59:17.569252+0000", 2024-09-10T15:59:17.594 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.7199999999999997e-07 2024-09-10T15:59:17.594 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-10T15:59:17.594 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-10T15:59:17.594 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-10T15:59:17.594 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-10T15:59:17.569263+0000", 2024-09-10T15:59:17.594 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1674999999999999e-05 2024-09-10T15:59:17.595 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-10T15:59:17.595 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-10T15:59:17.595 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-10T15:59:17.595 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-10T15:59:17.569272+0000", 2024-09-10T15:59:17.595 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.9509999999999995e-06 2024-09-10T15:59:17.595 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-10T15:59:17.595 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-10T15:59:17.595 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-10T15:59:17.595 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-10T15:59:17.569280+0000", 2024-09-10T15:59:17.595 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1669999999999999e-06 2024-09-10T15:59:17.595 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-10T15:59:17.595 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-10T15:59:17.596 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-10T15:59:17.596 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-10T15:59:17.570948+0000", 2024-09-10T15:59:17.596 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0016671570000000001 2024-09-10T15:59:17.596 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-10T15:59:17.596 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-10T15:59:17.596 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-10T15:59:17.596 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-10T15:59:17.570970+0000", 2024-09-10T15:59:17.596 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2755000000000002e-05 2024-09-10T15:59:17.596 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-10T15:59:17.596 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-10T15:59:17.596 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-10T15:59:17.596 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-10T15:59:17.596 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-10T15:59:17.597 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_repop(client.15002.0:69367 2.a e43/42 2:561bde23:::benchmark_data_smithi115_43363_object69366:head v 43'4424, mlcod=23'3)", 2024-09-10T15:59:17.597 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-10T15:59:17.577205+0000", 2024-09-10T15:59:17.597 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0060441569999999997, 2024-09-10T15:59:17.597 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0062101609999999996, 2024-09-10T15:59:17.597 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-10T15:59:17.597 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-10T15:59:17.597 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-10T15:59:17.597 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-10T15:59:17.597 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-10T15:59:17.597 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-10T15:59:17.597 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-10T15:59:17.577205+0000", 2024-09-10T15:59:17.597 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-10T15:59:17.597 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-10T15:59:17.598 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-10T15:59:17.598 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-10T15:59:17.598 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-10T15:59:17.577205+0000", 2024-09-10T15:59:17.598 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-10T15:59:17.598 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-10T15:59:17.598 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-10T15:59:17.598 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-10T15:59:17.598 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-10T15:59:17.577206+0000", 2024-09-10T15:59:17.598 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.389e-06 2024-09-10T15:59:17.598 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-10T15:59:17.598 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-10T15:59:17.598 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-10T15:59:17.598 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-10T15:59:17.577224+0000", 2024-09-10T15:59:17.598 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8050000000000002e-05 2024-09-10T15:59:17.599 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-10T15:59:17.599 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-10T15:59:17.599 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-10T15:59:17.599 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-10T15:59:17.577236+0000", 2024-09-10T15:59:17.599 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1372999999999999e-05 2024-09-10T15:59:17.599 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-10T15:59:17.599 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-10T15:59:17.599 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-10T15:59:17.599 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-10T15:59:17.577244+0000", 2024-09-10T15:59:17.599 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6600000000000001e-06 2024-09-10T15:59:17.599 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-10T15:59:17.599 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-10T15:59:17.599 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-10T15:59:17.600 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-10T15:59:17.577346+0000", 2024-09-10T15:59:17.600 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000101928 2024-09-10T15:59:17.600 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-10T15:59:17.600 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-10T15:59:17.600 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-10T15:59:17.600 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-10T15:59:17.577376+0000", 2024-09-10T15:59:17.600 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9706e-05 2024-09-10T15:59:17.600 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-10T15:59:17.600 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-10T15:59:17.600 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-10T15:59:17.600 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-10T15:59:17.600 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-10T15:59:17.600 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_repop(client.15002.0:69368 2.4 e43/42 2:2fe6b129:::benchmark_data_smithi115_43363_object69367:head v 43'4386, mlcod=23'9)", 2024-09-10T15:59:17.601 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-10T15:59:17.577276+0000", 2024-09-10T15:59:17.601 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0059734150000000001, 2024-09-10T15:59:17.601 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0061842729999999997, 2024-09-10T15:59:17.601 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-10T15:59:17.601 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-10T15:59:17.601 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-10T15:59:17.601 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-10T15:59:17.601 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-10T15:59:17.601 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-10T15:59:17.601 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-10T15:59:17.577276+0000", 2024-09-10T15:59:17.601 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-10T15:59:17.601 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-10T15:59:17.601 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-10T15:59:17.601 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-10T15:59:17.602 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-10T15:59:17.577276+0000", 2024-09-10T15:59:17.602 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-10T15:59:17.602 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-10T15:59:17.602 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-10T15:59:17.602 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-10T15:59:17.602 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-10T15:59:17.577276+0000", 2024-09-10T15:59:17.602 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.2000000000000001e-07 2024-09-10T15:59:17.602 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-10T15:59:17.602 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-10T15:59:17.602 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-10T15:59:17.602 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-10T15:59:17.577286+0000", 2024-09-10T15:59:17.602 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.6870000000000006e-06 2024-09-10T15:59:17.602 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-10T15:59:17.603 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-10T15:59:17.603 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-10T15:59:17.603 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-10T15:59:17.577291+0000", 2024-09-10T15:59:17.603 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2020000000000003e-06 2024-09-10T15:59:17.603 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-10T15:59:17.603 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-10T15:59:17.603 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-10T15:59:17.603 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-10T15:59:17.577295+0000", 2024-09-10T15:59:17.603 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1189999999999999e-06 2024-09-10T15:59:17.603 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-10T15:59:17.603 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-10T15:59:17.603 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-10T15:59:17.603 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-10T15:59:17.578936+0000", 2024-09-10T15:59:17.603 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0016411049999999999 2024-09-10T15:59:17.604 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-10T15:59:17.604 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-10T15:59:17.604 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-10T15:59:17.604 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-10T15:59:17.578963+0000", 2024-09-10T15:59:17.604 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6645999999999999e-05 2024-09-10T15:59:17.604 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-10T15:59:17.604 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-10T15:59:17.604 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-10T15:59:17.604 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-10T15:59:17.604 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-09-10T15:59:17.604 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 5 2024-09-10T15:59:17.604 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-10T15:59:17.605 INFO:tasks.lost_unfound.ceph_manager:waiting for osd.1 to be up 2024-09-10T15:59:17.605 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T15:59:17.965 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 45 20 69597 69577 6.03903 2.38281 0.0703773 0.0129263 2024-09-10T15:59:18.140 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T15:59:18.140 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":43,"fsid":"7effee29-df05-4351-9741-ee203564ed81","created":"2024-09-10T15:57:14.967498+0000","modified":"2024-09-10T15:59:15.761950+0000","last_up_change":"2024-09-10T15:59:14.117575+0000","last_in_change":"2024-09-10T15:59:07.057956+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":8,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2024-09-10T15:57:30.381324+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":14,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"unfound_pool","create_time":"2024-09-10T15:57:39.758755+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":16,"pg_placement_num":16,"pg_placement_num_target":16,"pg_num_target":16,"pg_num_pending":16,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rados":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"859508e0-e15d-4011-a39f-65e7aa04a129","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":21,"up_from":25,"up_thru":42,"down_at":22,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":1567796493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6801","nonce":1567796493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6803","nonce":1567796493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":1567796493}]},"public_addr":"172.21.15.115:6800/1567796493","cluster_addr":"172.21.15.115:6801/1567796493","heartbeat_back_addr":"172.21.15.115:6803/1567796493","heartbeat_front_addr":"172.21.15.115:6802/1567796493","state":["exists","up"]},{"osd":1,"uuid":"66daed2f-a693-4a13-8d8c-024794a032e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":26,"up_from":41,"up_thru":42,"down_at":27,"lost_at":27,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":257282173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6809","nonce":257282173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6811","nonce":257282173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":257282173}]},"public_addr":"172.21.15.115:6808/257282173","cluster_addr":"172.21.15.115:6809/257282173","heartbeat_back_addr":"172.21.15.115:6811/257282173","heartbeat_front_addr":"172.21.15.115:6810/257282173","state":["exists","up"]},{"osd":2,"uuid":"6a2ab153-d99e-4a5d-b79c-32b7f8ea9ffb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":18,"up_from":31,"up_thru":42,"down_at":19,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":2928349610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6805","nonce":2928349610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6807","nonce":2928349610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":2928349610}]},"public_addr":"172.21.15.115:6804/2928349610","cluster_addr":"172.21.15.115:6805/2928349610","heartbeat_back_addr":"172.21.15.115:6807/2928349610","heartbeat_front_addr":"172.21.15.115:6806/2928349610","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T15:57:23.007077+0000","dead_epoch":22},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T15:57:23.605294+0000","dead_epoch":27},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T15:57:22.852045+0000","dead_epoch":19}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[{"pgid":"2.0","osds":[2]},{"pgid":"2.1","osds":[2]},{"pgid":"2.2","osds":[0]},{"pgid":"2.3","osds":[2]},{"pgid":"2.4","osds":[0]},{"pgid":"2.5","osds":[0]},{"pgid":"2.6","osds":[0]},{"pgid":"2.7","osds":[0]},{"pgid":"2.9","osds":[0]},{"pgid":"2.a","osds":[0]},{"pgid":"2.d","osds":[2]},{"pgid":"2.f","osds":[0]}],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T15:59:18.174 INFO:tasks.lost_unfound.ceph_manager:osd.1 is up 2024-09-10T15:59:18.174 INFO:tasks.lost_unfound.ceph_manager:waiting for clean 2024-09-10T15:59:18.174 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:59:18.741 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T15:59:18.741 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T15:59:18.771 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":97,"stamp":"2024-09-10T15:59:17.545856+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":280756977,"num_objects":68555,"num_object_clones":0,"num_object_copies":137110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34334,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":68555,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":26401,"num_write_kb":105598,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":16403,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":68628,"ondisk_log_size":68628,"up":45,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":71,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":856572,"kb_used_data":544956,"kb_used_omap":12,"kb_used_meta":311539,"kb_avail":282258948,"statfs":{"total":289910292480,"available":289033162752,"internally_reserved":0,"allocated":558034944,"data_stored":557223967,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":319015938},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,2,4,5,0,0,0,0,0,0,13],"upper_bound":4096},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19075072,"num_objects":4657,"num_object_clones":0,"num_object_copies":9314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34334,"num_objects_misplaced":-47942,"num_objects_unfound":0,"num_objects_dirty":4657,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4657,"num_write_kb":18628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.422659"},"pg_stats":[{"pgid":"2.f","version":"40'4259","reported_seq":4324,"reported_epoch":43,"state":"activating+undersized+remapped","last_fresh":"2024-09-10T15:59:15.774356+0000","last_change":"2024-09-10T15:59:15.851279+0000","last_active":"2024-09-10T15:59:14.134656+0000","last_peered":"2024-09-10T15:59:14.132456+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:09.094143+0000","last_became_peered":"2024-09-10T15:59:09.094143+0000","last_unstale":"2024-09-10T15:59:15.774356+0000","last_undegraded":"2024-09-10T15:59:15.774356+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":4259,"log_dups_size":0,"ondisk_log_size":4259,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17444864,"num_objects":4259,"num_object_clones":0,"num_object_copies":8518,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4259,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4259,"num_write_kb":17036,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0"],"object_location_counts":[{"shards":"0","objects":4259}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"41'4295","reported_seq":4398,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-10T15:59:14.148289+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T15:59:14.148289+0000","last_peered":"2024-09-10T15:59:14.148289+0000","last_clean":"2024-09-10T15:59:14.148289+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T15:59:14.148289+0000","last_undegraded":"2024-09-10T15:59:14.148289+0000","last_fullsized":"2024-09-10T15:59:14.148289+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":4295,"log_dups_size":0,"ondisk_log_size":4295,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17568710,"num_objects":4291,"num_object_clones":0,"num_object_copies":8582,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4291,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":2400,"num_write_kb":9600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"40'4271","reported_seq":4389,"reported_epoch":40,"state":"remapped+peering","last_fresh":"2024-09-10T15:59:14.092473+0000","last_change":"2024-09-10T15:59:14.133399+0000","last_active":"2024-09-10T15:59:14.133399+0000","last_peered":"2024-09-10T15:59:14.092473+0000","last_clean":"2024-09-10T15:59:14.092473+0000","last_became_active":"2024-09-10T15:59:09.119657+0000","last_became_peered":"2024-09-10T15:59:09.119657+0000","last_unstale":"2024-09-10T15:59:14.092473+0000","last_undegraded":"2024-09-10T15:59:14.092473+0000","last_fullsized":"2024-09-10T15:59:14.092473+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":4271,"log_dups_size":0,"ondisk_log_size":4271,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17477632,"num_objects":4267,"num_object_clones":0,"num_object_copies":8534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4267,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2275,"num_write_kb":9100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,1],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"41'4234","reported_seq":4305,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-10T15:59:14.143979+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T15:59:14.143979+0000","last_peered":"2024-09-10T15:59:14.143979+0000","last_clean":"2024-09-10T15:59:14.143979+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T15:59:14.143979+0000","last_undegraded":"2024-09-10T15:59:14.143979+0000","last_fullsized":"2024-09-10T15:59:14.143979+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":4234,"log_dups_size":0,"ondisk_log_size":4234,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17342464,"num_objects":4234,"num_object_clones":0,"num_object_copies":8468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4234,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4234,"num_write_kb":16936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"41'4314","reported_seq":4438,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.769953+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T15:59:15.769953+0000","last_peered":"2024-09-10T15:59:15.769953+0000","last_clean":"2024-09-10T15:59:15.769953+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T15:59:15.769953+0000","last_undegraded":"2024-09-10T15:59:15.769953+0000","last_fullsized":"2024-09-10T15:59:15.769953+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":4314,"log_dups_size":0,"ondisk_log_size":4314,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17641955,"num_objects":4308,"num_object_clones":0,"num_object_copies":8616,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4308,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1025,"num_write_kb":4100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"40'4375","reported_seq":4503,"reported_epoch":43,"state":"activating+undersized+remapped","last_fresh":"2024-09-10T15:59:15.769328+0000","last_change":"2024-09-10T15:59:15.878234+0000","last_active":"2024-09-10T15:59:14.135985+0000","last_peered":"2024-09-10T15:59:14.111551+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:09.086899+0000","last_became_peered":"2024-09-10T15:59:09.086899+0000","last_unstale":"2024-09-10T15:59:15.769328+0000","last_undegraded":"2024-09-10T15:59:15.769328+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":4375,"log_dups_size":0,"ondisk_log_size":4375,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17900003,"num_objects":4371,"num_object_clones":0,"num_object_copies":8742,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4371,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":987,"num_write_kb":3948,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"40'4241","reported_seq":4369,"reported_epoch":43,"state":"activating+undersized+remapped","last_fresh":"2024-09-10T15:59:15.769042+0000","last_change":"2024-09-10T15:59:15.877524+0000","last_active":"2024-09-10T15:59:14.142085+0000","last_peered":"2024-09-10T15:59:14.139436+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:09.106757+0000","last_became_peered":"2024-09-10T15:59:09.106757+0000","last_unstale":"2024-09-10T15:59:15.769042+0000","last_undegraded":"2024-09-10T15:59:15.769042+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":4241,"log_dups_size":0,"ondisk_log_size":4241,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17347526,"num_objects":4237,"num_object_clones":0,"num_object_copies":8474,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4237,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":740,"num_write_kb":2960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4237}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"40'4352","reported_seq":4476,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.772151+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T15:59:15.772151+0000","last_peered":"2024-09-10T15:59:15.772151+0000","last_clean":"2024-09-10T15:59:15.772151+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T15:59:15.772151+0000","last_undegraded":"2024-09-10T15:59:15.772151+0000","last_fullsized":"2024-09-10T15:59:15.772151+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":4352,"log_dups_size":0,"ondisk_log_size":4352,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17817600,"num_objects":4350,"num_object_clones":0,"num_object_copies":8700,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4350,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1199,"num_write_kb":4796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":91,"reported_epoch":39,"state":"active+clean+remapped","last_fresh":"2024-09-10T15:59:09.088860+0000","last_change":"2024-09-10T15:59:09.088735+0000","last_active":"2024-09-10T15:59:09.088860+0000","last_peered":"2024-09-10T15:59:09.088860+0000","last_clean":"2024-09-10T15:59:09.088860+0000","last_became_active":"2024-09-10T15:59:09.088373+0000","last_became_peered":"2024-09-10T15:59:09.088373+0000","last_unstale":"2024-09-10T15:59:09.088860+0000","last_undegraded":"2024-09-10T15:59:09.088860+0000","last_fullsized":"2024-09-10T15:59:09.088860+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:22:33.930977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"40'4129","reported_seq":4255,"reported_epoch":40,"state":"remapped+peering","last_fresh":"2024-09-10T15:59:14.088187+0000","last_change":"2024-09-10T15:59:14.129908+0000","last_active":"2024-09-10T15:59:14.129908+0000","last_peered":"2024-09-10T15:59:14.088187+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:09.118399+0000","last_became_peered":"2024-09-10T15:59:09.118399+0000","last_unstale":"2024-09-10T15:59:14.088187+0000","last_undegraded":"2024-09-10T15:59:14.088187+0000","last_fullsized":"2024-09-10T15:59:14.088187+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":4129,"log_dups_size":0,"ondisk_log_size":4129,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16884195,"num_objects":4123,"num_object_clones":0,"num_object_copies":8246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4123,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1948,"num_write_kb":7792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,1],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":84,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-10T15:59:14.127288+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T15:59:14.127288+0000","last_peered":"2024-09-10T15:59:14.127288+0000","last_clean":"2024-09-10T15:59:14.127288+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T15:59:14.127288+0000","last_undegraded":"2024-09-10T15:59:14.127288+0000","last_fullsized":"2024-09-10T15:59:14.127288+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"40'4263","reported_seq":4384,"reported_epoch":43,"state":"activating+undersized+remapped","last_fresh":"2024-09-10T15:59:15.771410+0000","last_change":"2024-09-10T15:59:15.860857+0000","last_active":"2024-09-10T15:59:14.145841+0000","last_peered":"2024-09-10T15:59:14.129081+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:09.087496+0000","last_became_peered":"2024-09-10T15:59:09.087496+0000","last_unstale":"2024-09-10T15:59:15.771410+0000","last_undegraded":"2024-09-10T15:59:15.771410+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":4263,"log_dups_size":0,"ondisk_log_size":4263,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17428480,"num_objects":4255,"num_object_clones":0,"num_object_copies":8510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4255,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":767,"num_write_kb":3068,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4255}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":98,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.880567+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T15:59:15.880567+0000","last_peered":"2024-09-10T15:59:15.880567+0000","last_clean":"2024-09-10T15:59:15.880567+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T15:59:15.880567+0000","last_undegraded":"2024-09-10T15:59:15.880567+0000","last_fullsized":"2024-09-10T15:59:15.880567+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"40'4308","reported_seq":4429,"reported_epoch":40,"state":"remapped+peering","last_fresh":"2024-09-10T15:59:14.116041+0000","last_change":"2024-09-10T15:59:14.128679+0000","last_active":"2024-09-10T15:59:14.128679+0000","last_peered":"2024-09-10T15:59:14.116041+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:09.119191+0000","last_became_peered":"2024-09-10T15:59:09.119191+0000","last_unstale":"2024-09-10T15:59:14.116041+0000","last_undegraded":"2024-09-10T15:59:14.116041+0000","last_fullsized":"2024-09-10T15:59:14.116041+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":4308,"log_dups_size":0,"ondisk_log_size":4308,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17625571,"num_objects":4304,"num_object_clones":0,"num_object_copies":8608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4304,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1988,"num_write_kb":7952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":106,"reported_epoch":40,"state":"remapped+peering","last_fresh":"2024-09-10T15:59:13.127243+0000","last_change":"2024-09-10T15:59:14.125807+0000","last_active":"2024-09-10T15:59:14.125807+0000","last_peered":"2024-09-10T15:59:13.127243+0000","last_clean":"2024-09-10T15:59:13.127243+0000","last_became_active":"2024-09-10T15:59:09.095899+0000","last_became_peered":"2024-09-10T15:59:09.095899+0000","last_unstale":"2024-09-10T15:59:13.127243+0000","last_undegraded":"2024-09-10T15:59:13.127243+0000","last_fullsized":"2024-09-10T15:59:13.127243+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,1],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"40'4358","reported_seq":4473,"reported_epoch":40,"state":"remapped+peering","last_fresh":"2024-09-10T15:59:14.110654+0000","last_change":"2024-09-10T15:59:14.126286+0000","last_active":"2024-09-10T15:59:14.126286+0000","last_peered":"2024-09-10T15:59:14.110654+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:09.089328+0000","last_became_peered":"2024-09-10T15:59:09.089328+0000","last_unstale":"2024-09-10T15:59:14.110654+0000","last_undegraded":"2024-09-10T15:59:14.110654+0000","last_fullsized":"2024-09-10T15:59:14.110654+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":4358,"log_dups_size":0,"ondisk_log_size":4358,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17825792,"num_objects":4352,"num_object_clones":0,"num_object_copies":8704,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4352,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1912,"num_write_kb":7648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"40'4306","reported_seq":4431,"reported_epoch":43,"state":"activating+undersized+remapped","last_fresh":"2024-09-10T15:59:15.770337+0000","last_change":"2024-09-10T15:59:15.845012+0000","last_active":"2024-09-10T15:59:14.135619+0000","last_peered":"2024-09-10T15:59:14.117001+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:09.093889+0000","last_became_peered":"2024-09-10T15:59:09.093889+0000","last_unstale":"2024-09-10T15:59:15.770337+0000","last_undegraded":"2024-09-10T15:59:15.770337+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":4306,"log_dups_size":0,"ondisk_log_size":4306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17629184,"num_objects":4304,"num_object_clones":0,"num_object_copies":8608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4304,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":937,"num_write_kb":3748,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":91,"reported_epoch":39,"state":"active+clean+remapped","last_fresh":"2024-09-10T15:59:09.089977+0000","last_change":"2024-09-10T15:59:09.089850+0000","last_active":"2024-09-10T15:59:09.089977+0000","last_peered":"2024-09-10T15:59:09.089977+0000","last_clean":"2024-09-10T15:59:09.089977+0000","last_became_active":"2024-09-10T15:59:09.089239+0000","last_became_peered":"2024-09-10T15:59:09.089239+0000","last_unstale":"2024-09-10T15:59:09.089977+0000","last_undegraded":"2024-09-10T15:59:09.089977+0000","last_fullsized":"2024-09-10T15:59:09.089977+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:19:15.957824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"40'4211","reported_seq":4344,"reported_epoch":43,"state":"activating+undersized+remapped","last_fresh":"2024-09-10T15:59:15.774437+0000","last_change":"2024-09-10T15:59:15.847127+0000","last_active":"2024-09-10T15:59:14.136333+0000","last_peered":"2024-09-10T15:59:14.115595+0000","last_clean":"2024-09-10T15:59:14.115595+0000","last_became_active":"2024-09-10T15:59:09.089584+0000","last_became_peered":"2024-09-10T15:59:09.089584+0000","last_unstale":"2024-09-10T15:59:15.774437+0000","last_undegraded":"2024-09-10T15:59:15.774437+0000","last_fullsized":"2024-09-10T15:59:15.774437+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":4211,"log_dups_size":0,"ondisk_log_size":4211,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17215488,"num_objects":4203,"num_object_clones":0,"num_object_copies":8406,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4203,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":601,"num_write_kb":2404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4203}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":84,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-10T15:59:14.132228+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T15:59:14.132228+0000","last_peered":"2024-09-10T15:59:14.132228+0000","last_clean":"2024-09-10T15:59:14.132228+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T15:59:14.132228+0000","last_undegraded":"2024-09-10T15:59:14.132228+0000","last_fullsized":"2024-09-10T15:59:14.132228+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"40'4375","reported_seq":4507,"reported_epoch":43,"state":"activating+undersized+remapped","last_fresh":"2024-09-10T15:59:15.772756+0000","last_change":"2024-09-10T15:59:15.883434+0000","last_active":"2024-09-10T15:59:14.141008+0000","last_peered":"2024-09-10T15:59:14.116363+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:09.106202+0000","last_became_peered":"2024-09-10T15:59:09.106202+0000","last_unstale":"2024-09-10T15:59:15.772756+0000","last_undegraded":"2024-09-10T15:59:15.772756+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":4375,"log_dups_size":0,"ondisk_log_size":4375,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17903616,"num_objects":4371,"num_object_clones":0,"num_object_copies":8742,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4371,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":630,"num_write_kb":2520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":91,"reported_epoch":39,"state":"active+clean+remapped","last_fresh":"2024-09-10T15:59:09.089232+0000","last_change":"2024-09-10T15:59:09.089096+0000","last_active":"2024-09-10T15:59:09.089232+0000","last_peered":"2024-09-10T15:59:09.089232+0000","last_clean":"2024-09-10T15:59:09.089232+0000","last_became_active":"2024-09-10T15:59:09.086870+0000","last_became_peered":"2024-09-10T15:59:09.086870+0000","last_unstale":"2024-09-10T15:59:09.089232+0000","last_undegraded":"2024-09-10T15:59:09.089232+0000","last_fullsized":"2024-09-10T15:59:09.089232+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:33:54.200608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":101,"reported_epoch":40,"state":"remapped+peering","last_fresh":"2024-09-10T15:59:13.129660+0000","last_change":"2024-09-10T15:59:14.129617+0000","last_active":"2024-09-10T15:59:14.129617+0000","last_peered":"2024-09-10T15:59:13.129660+0000","last_clean":"2024-09-10T15:59:13.129660+0000","last_became_active":"2024-09-10T15:59:09.087344+0000","last_became_peered":"2024-09-10T15:59:09.087344+0000","last_unstale":"2024-09-10T15:59:13.129660+0000","last_undegraded":"2024-09-10T15:59:13.129660+0000","last_fullsized":"2024-09-10T15:59:13.129660+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,1],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"40'4334","reported_seq":4469,"reported_epoch":43,"state":"activating+undersized+remapped","last_fresh":"2024-09-10T15:59:15.768513+0000","last_change":"2024-09-10T15:59:15.887823+0000","last_active":"2024-09-10T15:59:14.141174+0000","last_peered":"2024-09-10T15:59:14.129780+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:09.083692+0000","last_became_peered":"2024-09-10T15:59:09.083692+0000","last_unstale":"2024-09-10T15:59:15.768513+0000","last_undegraded":"2024-09-10T15:59:15.768513+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":4334,"log_dups_size":0,"ondisk_log_size":4334,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17703878,"num_objects":4324,"num_object_clones":0,"num_object_copies":8648,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4324,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":497,"num_write_kb":1988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":280756958,"num_objects":68553,"num_object_clones":0,"num_object_copies":137106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34334,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":68553,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":26399,"num_write_kb":105596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":16384,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":561696768,"data_stored":561523344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":68625,"ondisk_log_size":68625,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":13,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986191,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":405884,"kb_used_data":270056,"kb_used_omap":4,"kb_used_meta":135803,"kb_avail":93965956,"statfs":{"total":96636764160,"available":96221138944,"internally_reserved":0,"allocated":276537344,"data_stored":276270163,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":139062954},"hb_peers":[0],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,2,4,5],"upper_bound":32},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":0,"seq":1,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27868,"kb_used_data":332,"kb_used_omap":4,"kb_used_meta":27515,"kb_avail":94343972,"statfs":{"total":96636764160,"available":96608227328,"internally_reserved":0,"allocated":339968,"data_stored":78009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":28176046},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182420,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":422820,"kb_used_data":274568,"kb_used_omap":4,"kb_used_meta":148219,"kb_avail":93949020,"statfs":{"total":96636764160,"available":96203796480,"internally_reserved":0,"allocated":281157632,"data_stored":280875795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":151776938},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,13],"upper_bound":4096},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":280793088,"data_stored":280756958,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":13524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":280788992,"data_stored":280752862,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:59:18.772 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:59:18.965 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 46 20 70168 70148 5.95624 2.23047 0.0276304 0.0131072 2024-09-10T15:59:19.347 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T15:59:19.350 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T15:59:19.373 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":97,"stamp":"2024-09-10T15:59:17.545856+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":280756977,"num_objects":68555,"num_object_clones":0,"num_object_copies":137110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34334,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":68555,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":26401,"num_write_kb":105598,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":16403,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":68628,"ondisk_log_size":68628,"up":45,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":71,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":856572,"kb_used_data":544956,"kb_used_omap":12,"kb_used_meta":311539,"kb_avail":282258948,"statfs":{"total":289910292480,"available":289033162752,"internally_reserved":0,"allocated":558034944,"data_stored":557223967,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":319015938},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,2,4,5,0,0,0,0,0,0,13],"upper_bound":4096},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19075072,"num_objects":4657,"num_object_clones":0,"num_object_copies":9314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34334,"num_objects_misplaced":-47942,"num_objects_unfound":0,"num_objects_dirty":4657,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4657,"num_write_kb":18628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.422659"},"pg_stats":[{"pgid":"2.f","version":"40'4259","reported_seq":4324,"reported_epoch":43,"state":"activating+undersized+remapped","last_fresh":"2024-09-10T15:59:15.774356+0000","last_change":"2024-09-10T15:59:15.851279+0000","last_active":"2024-09-10T15:59:14.134656+0000","last_peered":"2024-09-10T15:59:14.132456+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:09.094143+0000","last_became_peered":"2024-09-10T15:59:09.094143+0000","last_unstale":"2024-09-10T15:59:15.774356+0000","last_undegraded":"2024-09-10T15:59:15.774356+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":4259,"log_dups_size":0,"ondisk_log_size":4259,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17444864,"num_objects":4259,"num_object_clones":0,"num_object_copies":8518,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4259,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4259,"num_write_kb":17036,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0"],"object_location_counts":[{"shards":"0","objects":4259}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"41'4295","reported_seq":4398,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-10T15:59:14.148289+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T15:59:14.148289+0000","last_peered":"2024-09-10T15:59:14.148289+0000","last_clean":"2024-09-10T15:59:14.148289+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T15:59:14.148289+0000","last_undegraded":"2024-09-10T15:59:14.148289+0000","last_fullsized":"2024-09-10T15:59:14.148289+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":4295,"log_dups_size":0,"ondisk_log_size":4295,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17568710,"num_objects":4291,"num_object_clones":0,"num_object_copies":8582,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4291,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":2400,"num_write_kb":9600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"40'4271","reported_seq":4389,"reported_epoch":40,"state":"remapped+peering","last_fresh":"2024-09-10T15:59:14.092473+0000","last_change":"2024-09-10T15:59:14.133399+0000","last_active":"2024-09-10T15:59:14.133399+0000","last_peered":"2024-09-10T15:59:14.092473+0000","last_clean":"2024-09-10T15:59:14.092473+0000","last_became_active":"2024-09-10T15:59:09.119657+0000","last_became_peered":"2024-09-10T15:59:09.119657+0000","last_unstale":"2024-09-10T15:59:14.092473+0000","last_undegraded":"2024-09-10T15:59:14.092473+0000","last_fullsized":"2024-09-10T15:59:14.092473+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":4271,"log_dups_size":0,"ondisk_log_size":4271,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17477632,"num_objects":4267,"num_object_clones":0,"num_object_copies":8534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4267,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2275,"num_write_kb":9100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,1],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"41'4234","reported_seq":4305,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-10T15:59:14.143979+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T15:59:14.143979+0000","last_peered":"2024-09-10T15:59:14.143979+0000","last_clean":"2024-09-10T15:59:14.143979+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T15:59:14.143979+0000","last_undegraded":"2024-09-10T15:59:14.143979+0000","last_fullsized":"2024-09-10T15:59:14.143979+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":4234,"log_dups_size":0,"ondisk_log_size":4234,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17342464,"num_objects":4234,"num_object_clones":0,"num_object_copies":8468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4234,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4234,"num_write_kb":16936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"41'4314","reported_seq":4438,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.769953+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T15:59:15.769953+0000","last_peered":"2024-09-10T15:59:15.769953+0000","last_clean":"2024-09-10T15:59:15.769953+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T15:59:15.769953+0000","last_undegraded":"2024-09-10T15:59:15.769953+0000","last_fullsized":"2024-09-10T15:59:15.769953+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":4314,"log_dups_size":0,"ondisk_log_size":4314,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17641955,"num_objects":4308,"num_object_clones":0,"num_object_copies":8616,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4308,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1025,"num_write_kb":4100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"40'4375","reported_seq":4503,"reported_epoch":43,"state":"activating+undersized+remapped","last_fresh":"2024-09-10T15:59:15.769328+0000","last_change":"2024-09-10T15:59:15.878234+0000","last_active":"2024-09-10T15:59:14.135985+0000","last_peered":"2024-09-10T15:59:14.111551+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:09.086899+0000","last_became_peered":"2024-09-10T15:59:09.086899+0000","last_unstale":"2024-09-10T15:59:15.769328+0000","last_undegraded":"2024-09-10T15:59:15.769328+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":4375,"log_dups_size":0,"ondisk_log_size":4375,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17900003,"num_objects":4371,"num_object_clones":0,"num_object_copies":8742,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4371,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":987,"num_write_kb":3948,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"40'4241","reported_seq":4369,"reported_epoch":43,"state":"activating+undersized+remapped","last_fresh":"2024-09-10T15:59:15.769042+0000","last_change":"2024-09-10T15:59:15.877524+0000","last_active":"2024-09-10T15:59:14.142085+0000","last_peered":"2024-09-10T15:59:14.139436+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:09.106757+0000","last_became_peered":"2024-09-10T15:59:09.106757+0000","last_unstale":"2024-09-10T15:59:15.769042+0000","last_undegraded":"2024-09-10T15:59:15.769042+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":4241,"log_dups_size":0,"ondisk_log_size":4241,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17347526,"num_objects":4237,"num_object_clones":0,"num_object_copies":8474,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4237,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":740,"num_write_kb":2960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4237}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"40'4352","reported_seq":4476,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.772151+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T15:59:15.772151+0000","last_peered":"2024-09-10T15:59:15.772151+0000","last_clean":"2024-09-10T15:59:15.772151+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T15:59:15.772151+0000","last_undegraded":"2024-09-10T15:59:15.772151+0000","last_fullsized":"2024-09-10T15:59:15.772151+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":4352,"log_dups_size":0,"ondisk_log_size":4352,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17817600,"num_objects":4350,"num_object_clones":0,"num_object_copies":8700,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4350,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1199,"num_write_kb":4796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":91,"reported_epoch":39,"state":"active+clean+remapped","last_fresh":"2024-09-10T15:59:09.088860+0000","last_change":"2024-09-10T15:59:09.088735+0000","last_active":"2024-09-10T15:59:09.088860+0000","last_peered":"2024-09-10T15:59:09.088860+0000","last_clean":"2024-09-10T15:59:09.088860+0000","last_became_active":"2024-09-10T15:59:09.088373+0000","last_became_peered":"2024-09-10T15:59:09.088373+0000","last_unstale":"2024-09-10T15:59:09.088860+0000","last_undegraded":"2024-09-10T15:59:09.088860+0000","last_fullsized":"2024-09-10T15:59:09.088860+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:22:33.930977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"40'4129","reported_seq":4255,"reported_epoch":40,"state":"remapped+peering","last_fresh":"2024-09-10T15:59:14.088187+0000","last_change":"2024-09-10T15:59:14.129908+0000","last_active":"2024-09-10T15:59:14.129908+0000","last_peered":"2024-09-10T15:59:14.088187+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:09.118399+0000","last_became_peered":"2024-09-10T15:59:09.118399+0000","last_unstale":"2024-09-10T15:59:14.088187+0000","last_undegraded":"2024-09-10T15:59:14.088187+0000","last_fullsized":"2024-09-10T15:59:14.088187+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":4129,"log_dups_size":0,"ondisk_log_size":4129,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16884195,"num_objects":4123,"num_object_clones":0,"num_object_copies":8246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4123,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1948,"num_write_kb":7792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,1],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":84,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-10T15:59:14.127288+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T15:59:14.127288+0000","last_peered":"2024-09-10T15:59:14.127288+0000","last_clean":"2024-09-10T15:59:14.127288+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T15:59:14.127288+0000","last_undegraded":"2024-09-10T15:59:14.127288+0000","last_fullsized":"2024-09-10T15:59:14.127288+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"40'4263","reported_seq":4384,"reported_epoch":43,"state":"activating+undersized+remapped","last_fresh":"2024-09-10T15:59:15.771410+0000","last_change":"2024-09-10T15:59:15.860857+0000","last_active":"2024-09-10T15:59:14.145841+0000","last_peered":"2024-09-10T15:59:14.129081+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:09.087496+0000","last_became_peered":"2024-09-10T15:59:09.087496+0000","last_unstale":"2024-09-10T15:59:15.771410+0000","last_undegraded":"2024-09-10T15:59:15.771410+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":4263,"log_dups_size":0,"ondisk_log_size":4263,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17428480,"num_objects":4255,"num_object_clones":0,"num_object_copies":8510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4255,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":767,"num_write_kb":3068,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4255}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":98,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.880567+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T15:59:15.880567+0000","last_peered":"2024-09-10T15:59:15.880567+0000","last_clean":"2024-09-10T15:59:15.880567+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T15:59:15.880567+0000","last_undegraded":"2024-09-10T15:59:15.880567+0000","last_fullsized":"2024-09-10T15:59:15.880567+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"40'4308","reported_seq":4429,"reported_epoch":40,"state":"remapped+peering","last_fresh":"2024-09-10T15:59:14.116041+0000","last_change":"2024-09-10T15:59:14.128679+0000","last_active":"2024-09-10T15:59:14.128679+0000","last_peered":"2024-09-10T15:59:14.116041+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:09.119191+0000","last_became_peered":"2024-09-10T15:59:09.119191+0000","last_unstale":"2024-09-10T15:59:14.116041+0000","last_undegraded":"2024-09-10T15:59:14.116041+0000","last_fullsized":"2024-09-10T15:59:14.116041+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":4308,"log_dups_size":0,"ondisk_log_size":4308,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17625571,"num_objects":4304,"num_object_clones":0,"num_object_copies":8608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4304,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1988,"num_write_kb":7952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":106,"reported_epoch":40,"state":"remapped+peering","last_fresh":"2024-09-10T15:59:13.127243+0000","last_change":"2024-09-10T15:59:14.125807+0000","last_active":"2024-09-10T15:59:14.125807+0000","last_peered":"2024-09-10T15:59:13.127243+0000","last_clean":"2024-09-10T15:59:13.127243+0000","last_became_active":"2024-09-10T15:59:09.095899+0000","last_became_peered":"2024-09-10T15:59:09.095899+0000","last_unstale":"2024-09-10T15:59:13.127243+0000","last_undegraded":"2024-09-10T15:59:13.127243+0000","last_fullsized":"2024-09-10T15:59:13.127243+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,1],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"40'4358","reported_seq":4473,"reported_epoch":40,"state":"remapped+peering","last_fresh":"2024-09-10T15:59:14.110654+0000","last_change":"2024-09-10T15:59:14.126286+0000","last_active":"2024-09-10T15:59:14.126286+0000","last_peered":"2024-09-10T15:59:14.110654+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:09.089328+0000","last_became_peered":"2024-09-10T15:59:09.089328+0000","last_unstale":"2024-09-10T15:59:14.110654+0000","last_undegraded":"2024-09-10T15:59:14.110654+0000","last_fullsized":"2024-09-10T15:59:14.110654+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":4358,"log_dups_size":0,"ondisk_log_size":4358,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17825792,"num_objects":4352,"num_object_clones":0,"num_object_copies":8704,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4352,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1912,"num_write_kb":7648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"40'4306","reported_seq":4431,"reported_epoch":43,"state":"activating+undersized+remapped","last_fresh":"2024-09-10T15:59:15.770337+0000","last_change":"2024-09-10T15:59:15.845012+0000","last_active":"2024-09-10T15:59:14.135619+0000","last_peered":"2024-09-10T15:59:14.117001+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:09.093889+0000","last_became_peered":"2024-09-10T15:59:09.093889+0000","last_unstale":"2024-09-10T15:59:15.770337+0000","last_undegraded":"2024-09-10T15:59:15.770337+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":4306,"log_dups_size":0,"ondisk_log_size":4306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17629184,"num_objects":4304,"num_object_clones":0,"num_object_copies":8608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4304,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":937,"num_write_kb":3748,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":91,"reported_epoch":39,"state":"active+clean+remapped","last_fresh":"2024-09-10T15:59:09.089977+0000","last_change":"2024-09-10T15:59:09.089850+0000","last_active":"2024-09-10T15:59:09.089977+0000","last_peered":"2024-09-10T15:59:09.089977+0000","last_clean":"2024-09-10T15:59:09.089977+0000","last_became_active":"2024-09-10T15:59:09.089239+0000","last_became_peered":"2024-09-10T15:59:09.089239+0000","last_unstale":"2024-09-10T15:59:09.089977+0000","last_undegraded":"2024-09-10T15:59:09.089977+0000","last_fullsized":"2024-09-10T15:59:09.089977+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:19:15.957824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"40'4211","reported_seq":4344,"reported_epoch":43,"state":"activating+undersized+remapped","last_fresh":"2024-09-10T15:59:15.774437+0000","last_change":"2024-09-10T15:59:15.847127+0000","last_active":"2024-09-10T15:59:14.136333+0000","last_peered":"2024-09-10T15:59:14.115595+0000","last_clean":"2024-09-10T15:59:14.115595+0000","last_became_active":"2024-09-10T15:59:09.089584+0000","last_became_peered":"2024-09-10T15:59:09.089584+0000","last_unstale":"2024-09-10T15:59:15.774437+0000","last_undegraded":"2024-09-10T15:59:15.774437+0000","last_fullsized":"2024-09-10T15:59:15.774437+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":4211,"log_dups_size":0,"ondisk_log_size":4211,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17215488,"num_objects":4203,"num_object_clones":0,"num_object_copies":8406,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4203,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":601,"num_write_kb":2404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4203}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":84,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-10T15:59:14.132228+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T15:59:14.132228+0000","last_peered":"2024-09-10T15:59:14.132228+0000","last_clean":"2024-09-10T15:59:14.132228+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T15:59:14.132228+0000","last_undegraded":"2024-09-10T15:59:14.132228+0000","last_fullsized":"2024-09-10T15:59:14.132228+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"40'4375","reported_seq":4507,"reported_epoch":43,"state":"activating+undersized+remapped","last_fresh":"2024-09-10T15:59:15.772756+0000","last_change":"2024-09-10T15:59:15.883434+0000","last_active":"2024-09-10T15:59:14.141008+0000","last_peered":"2024-09-10T15:59:14.116363+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:09.106202+0000","last_became_peered":"2024-09-10T15:59:09.106202+0000","last_unstale":"2024-09-10T15:59:15.772756+0000","last_undegraded":"2024-09-10T15:59:15.772756+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":4375,"log_dups_size":0,"ondisk_log_size":4375,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17903616,"num_objects":4371,"num_object_clones":0,"num_object_copies":8742,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4371,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":630,"num_write_kb":2520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":91,"reported_epoch":39,"state":"active+clean+remapped","last_fresh":"2024-09-10T15:59:09.089232+0000","last_change":"2024-09-10T15:59:09.089096+0000","last_active":"2024-09-10T15:59:09.089232+0000","last_peered":"2024-09-10T15:59:09.089232+0000","last_clean":"2024-09-10T15:59:09.089232+0000","last_became_active":"2024-09-10T15:59:09.086870+0000","last_became_peered":"2024-09-10T15:59:09.086870+0000","last_unstale":"2024-09-10T15:59:09.089232+0000","last_undegraded":"2024-09-10T15:59:09.089232+0000","last_fullsized":"2024-09-10T15:59:09.089232+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:33:54.200608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":101,"reported_epoch":40,"state":"remapped+peering","last_fresh":"2024-09-10T15:59:13.129660+0000","last_change":"2024-09-10T15:59:14.129617+0000","last_active":"2024-09-10T15:59:14.129617+0000","last_peered":"2024-09-10T15:59:13.129660+0000","last_clean":"2024-09-10T15:59:13.129660+0000","last_became_active":"2024-09-10T15:59:09.087344+0000","last_became_peered":"2024-09-10T15:59:09.087344+0000","last_unstale":"2024-09-10T15:59:13.129660+0000","last_undegraded":"2024-09-10T15:59:13.129660+0000","last_fullsized":"2024-09-10T15:59:13.129660+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,1],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"40'4334","reported_seq":4469,"reported_epoch":43,"state":"activating+undersized+remapped","last_fresh":"2024-09-10T15:59:15.768513+0000","last_change":"2024-09-10T15:59:15.887823+0000","last_active":"2024-09-10T15:59:14.141174+0000","last_peered":"2024-09-10T15:59:14.129780+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:09.083692+0000","last_became_peered":"2024-09-10T15:59:09.083692+0000","last_unstale":"2024-09-10T15:59:15.768513+0000","last_undegraded":"2024-09-10T15:59:15.768513+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":4334,"log_dups_size":0,"ondisk_log_size":4334,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17703878,"num_objects":4324,"num_object_clones":0,"num_object_copies":8648,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4324,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":497,"num_write_kb":1988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":280756958,"num_objects":68553,"num_object_clones":0,"num_object_copies":137106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34334,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":68553,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":26399,"num_write_kb":105596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":16384,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":561696768,"data_stored":561523344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":68625,"ondisk_log_size":68625,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":13,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986191,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":405884,"kb_used_data":270056,"kb_used_omap":4,"kb_used_meta":135803,"kb_avail":93965956,"statfs":{"total":96636764160,"available":96221138944,"internally_reserved":0,"allocated":276537344,"data_stored":276270163,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":139062954},"hb_peers":[0],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,2,4,5],"upper_bound":32},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":0,"seq":1,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27868,"kb_used_data":332,"kb_used_omap":4,"kb_used_meta":27515,"kb_avail":94343972,"statfs":{"total":96636764160,"available":96608227328,"internally_reserved":0,"allocated":339968,"data_stored":78009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":28176046},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182420,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":422820,"kb_used_data":274568,"kb_used_omap":4,"kb_used_meta":148219,"kb_avail":93949020,"statfs":{"total":96636764160,"available":96203796480,"internally_reserved":0,"allocated":281157632,"data_stored":280875795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":151776938},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,13],"upper_bound":4096},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":280793088,"data_stored":280756958,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":13524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":280788992,"data_stored":280752862,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:59:19.374 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:59:19.907 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T15:59:19.907 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T15:59:19.937 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":98,"stamp":"2024-09-10T15:59:19.546233+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":283382513,"num_objects":69196,"num_object_clones":0,"num_object_copies":138392,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":47117,"num_objects_misplaced":4202,"num_objects_unfound":0,"num_objects_dirty":69196,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":27042,"num_write_kb":108162,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":71,"num_bytes_recovered":282643,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":69269,"ondisk_log_size":69269,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":66,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":892136,"kb_used_data":554148,"kb_used_omap":12,"kb_used_meta":337907,"kb_avail":282223384,"statfs":{"total":289910292480,"available":288996745216,"internally_reserved":0,"allocated":567447552,"data_stored":566559149,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":346016770},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,20],"upper_bound":4096},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":21700608,"num_objects":5298,"num_object_clones":0,"num_object_copies":10596,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":47117,"num_objects_misplaced":-43740,"num_objects_unfound":0,"num_objects_dirty":5298,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5298,"num_write_kb":21192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":65,"num_bytes_recovered":266240,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001157"},"pg_stats":[{"pgid":"2.f","version":"40'4259","reported_seq":4324,"reported_epoch":43,"state":"activating+undersized+remapped","last_fresh":"2024-09-10T15:59:15.774356+0000","last_change":"2024-09-10T15:59:15.851279+0000","last_active":"2024-09-10T15:59:14.134656+0000","last_peered":"2024-09-10T15:59:14.132456+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:09.094143+0000","last_became_peered":"2024-09-10T15:59:09.094143+0000","last_unstale":"2024-09-10T15:59:15.774356+0000","last_undegraded":"2024-09-10T15:59:15.774356+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":4259,"log_dups_size":0,"ondisk_log_size":4259,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17444864,"num_objects":4259,"num_object_clones":0,"num_object_copies":8518,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4259,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4259,"num_write_kb":17036,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0"],"object_location_counts":[{"shards":"0","objects":4259}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'4398","reported_seq":4505,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:19.044943+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T15:59:19.044943+0000","last_peered":"2024-09-10T15:59:19.044943+0000","last_clean":"2024-09-10T15:59:19.044943+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T15:59:19.044943+0000","last_undegraded":"2024-09-10T15:59:19.044943+0000","last_fullsized":"2024-09-10T15:59:19.044943+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":4398,"log_dups_size":0,"ondisk_log_size":4398,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17990598,"num_objects":4394,"num_object_clones":0,"num_object_copies":8788,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4394,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":2503,"num_write_kb":10012,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'4376","reported_seq":4633,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T15:59:19.182418+0000","last_change":"2024-09-10T15:59:16.142612+0000","last_active":"2024-09-10T15:59:19.182418+0000","last_peered":"2024-09-10T15:59:19.182418+0000","last_clean":"2024-09-10T15:59:14.092473+0000","last_became_active":"2024-09-10T15:59:16.109611+0000","last_became_peered":"2024-09-10T15:59:16.109611+0000","last_unstale":"2024-09-10T15:59:19.182418+0000","last_undegraded":"2024-09-10T15:59:19.182418+0000","last_fullsized":"2024-09-10T15:59:15.770264+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":4376,"log_dups_size":0,"ondisk_log_size":4376,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17907712,"num_objects":4372,"num_object_clones":0,"num_object_copies":8744,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":4202,"num_objects_unfound":0,"num_objects_dirty":4372,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2380,"num_write_kb":9520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":270336,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4202},{"shards":"2","objects":170}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'4336","reported_seq":4411,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:19.206436+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T15:59:19.206436+0000","last_peered":"2024-09-10T15:59:19.206436+0000","last_clean":"2024-09-10T15:59:19.206436+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T15:59:19.206436+0000","last_undegraded":"2024-09-10T15:59:19.206436+0000","last_fullsized":"2024-09-10T15:59:19.206436+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":4336,"log_dups_size":0,"ondisk_log_size":4336,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17760256,"num_objects":4336,"num_object_clones":0,"num_object_copies":8672,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4336,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4336,"num_write_kb":17344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"41'4314","reported_seq":4438,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.769953+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T15:59:15.769953+0000","last_peered":"2024-09-10T15:59:15.769953+0000","last_clean":"2024-09-10T15:59:15.769953+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T15:59:15.769953+0000","last_undegraded":"2024-09-10T15:59:15.769953+0000","last_fullsized":"2024-09-10T15:59:15.769953+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":4314,"log_dups_size":0,"ondisk_log_size":4314,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17641955,"num_objects":4308,"num_object_clones":0,"num_object_copies":8616,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4308,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1025,"num_write_kb":4100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"40'4375","reported_seq":4503,"reported_epoch":43,"state":"activating+undersized+remapped","last_fresh":"2024-09-10T15:59:15.769328+0000","last_change":"2024-09-10T15:59:15.878234+0000","last_active":"2024-09-10T15:59:14.135985+0000","last_peered":"2024-09-10T15:59:14.111551+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:09.086899+0000","last_became_peered":"2024-09-10T15:59:09.086899+0000","last_unstale":"2024-09-10T15:59:15.769328+0000","last_undegraded":"2024-09-10T15:59:15.769328+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":4375,"log_dups_size":0,"ondisk_log_size":4375,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17900003,"num_objects":4371,"num_object_clones":0,"num_object_copies":8742,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4371,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":987,"num_write_kb":3948,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"40'4241","reported_seq":4369,"reported_epoch":43,"state":"activating+undersized+remapped","last_fresh":"2024-09-10T15:59:15.769042+0000","last_change":"2024-09-10T15:59:15.877524+0000","last_active":"2024-09-10T15:59:14.142085+0000","last_peered":"2024-09-10T15:59:14.139436+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:09.106757+0000","last_became_peered":"2024-09-10T15:59:09.106757+0000","last_unstale":"2024-09-10T15:59:15.769042+0000","last_undegraded":"2024-09-10T15:59:15.769042+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":4241,"log_dups_size":0,"ondisk_log_size":4241,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17347526,"num_objects":4237,"num_object_clones":0,"num_object_copies":8474,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4237,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":740,"num_write_kb":2960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4237}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"40'4352","reported_seq":4476,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.772151+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T15:59:15.772151+0000","last_peered":"2024-09-10T15:59:15.772151+0000","last_clean":"2024-09-10T15:59:15.772151+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T15:59:15.772151+0000","last_undegraded":"2024-09-10T15:59:15.772151+0000","last_fullsized":"2024-09-10T15:59:15.772151+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":4352,"log_dups_size":0,"ondisk_log_size":4352,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17817600,"num_objects":4350,"num_object_clones":0,"num_object_copies":8700,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4350,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1199,"num_write_kb":4796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.775865+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T15:59:15.775865+0000","last_peered":"2024-09-10T15:59:15.775865+0000","last_clean":"2024-09-10T15:59:15.775865+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T15:59:15.775865+0000","last_undegraded":"2024-09-10T15:59:15.775865+0000","last_fullsized":"2024-09-10T15:59:15.775865+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'4249","reported_seq":4383,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:19.136107+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T15:59:19.136107+0000","last_peered":"2024-09-10T15:59:19.136107+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T15:59:19.136107+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":4249,"log_dups_size":0,"ondisk_log_size":4249,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17375715,"num_objects":4243,"num_object_clones":0,"num_object_copies":8486,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4243,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2068,"num_write_kb":8272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4123},{"shards":"2","objects":120}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":88,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.772572+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T15:59:15.772572+0000","last_peered":"2024-09-10T15:59:15.772572+0000","last_clean":"2024-09-10T15:59:15.772572+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T15:59:15.772572+0000","last_undegraded":"2024-09-10T15:59:15.772572+0000","last_fullsized":"2024-09-10T15:59:15.772572+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"40'4263","reported_seq":4384,"reported_epoch":43,"state":"activating+undersized+remapped","last_fresh":"2024-09-10T15:59:15.771410+0000","last_change":"2024-09-10T15:59:15.860857+0000","last_active":"2024-09-10T15:59:14.145841+0000","last_peered":"2024-09-10T15:59:14.129081+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:09.087496+0000","last_became_peered":"2024-09-10T15:59:09.087496+0000","last_unstale":"2024-09-10T15:59:15.771410+0000","last_undegraded":"2024-09-10T15:59:15.771410+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":4263,"log_dups_size":0,"ondisk_log_size":4263,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17428480,"num_objects":4255,"num_object_clones":0,"num_object_copies":8510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4255,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":767,"num_write_kb":3068,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4255}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":98,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.880567+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T15:59:15.880567+0000","last_peered":"2024-09-10T15:59:15.880567+0000","last_clean":"2024-09-10T15:59:15.880567+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T15:59:15.880567+0000","last_undegraded":"2024-09-10T15:59:15.880567+0000","last_fullsized":"2024-09-10T15:59:15.880567+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'4413","reported_seq":4541,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:19.051655+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T15:59:19.051655+0000","last_peered":"2024-09-10T15:59:19.051655+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T15:59:19.051655+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":4413,"log_dups_size":0,"ondisk_log_size":4413,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18055651,"num_objects":4409,"num_object_clones":0,"num_object_copies":8818,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4409,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2093,"num_write_kb":8372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4304},{"shards":"2","objects":105}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.837509+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T15:59:15.837509+0000","last_peered":"2024-09-10T15:59:15.837509+0000","last_clean":"2024-09-10T15:59:15.837509+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T15:59:15.837509+0000","last_undegraded":"2024-09-10T15:59:15.837509+0000","last_fullsized":"2024-09-10T15:59:15.837509+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'4464","reported_seq":4587,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:19.137413+0000","last_change":"2024-09-10T15:59:16.174113+0000","last_active":"2024-09-10T15:59:19.137413+0000","last_peered":"2024-09-10T15:59:19.137413+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T15:59:19.137413+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":4464,"log_dups_size":0,"ondisk_log_size":4464,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18259968,"num_objects":4458,"num_object_clones":0,"num_object_copies":8916,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4354,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4458,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2018,"num_write_kb":8072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4352},{"shards":"2","objects":106}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"40'4306","reported_seq":4431,"reported_epoch":43,"state":"activating+undersized+remapped","last_fresh":"2024-09-10T15:59:15.770337+0000","last_change":"2024-09-10T15:59:15.845012+0000","last_active":"2024-09-10T15:59:14.135619+0000","last_peered":"2024-09-10T15:59:14.117001+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:09.093889+0000","last_became_peered":"2024-09-10T15:59:09.093889+0000","last_unstale":"2024-09-10T15:59:15.770337+0000","last_undegraded":"2024-09-10T15:59:15.770337+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":4306,"log_dups_size":0,"ondisk_log_size":4306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17629184,"num_objects":4304,"num_object_clones":0,"num_object_copies":8608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4304,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":937,"num_write_kb":3748,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.989413+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T15:59:15.989413+0000","last_peered":"2024-09-10T15:59:15.989413+0000","last_clean":"2024-09-10T15:59:15.989413+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T15:59:15.989413+0000","last_undegraded":"2024-09-10T15:59:15.989413+0000","last_fullsized":"2024-09-10T15:59:15.989413+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"40'4211","reported_seq":4344,"reported_epoch":43,"state":"activating+undersized+remapped","last_fresh":"2024-09-10T15:59:15.774437+0000","last_change":"2024-09-10T15:59:15.847127+0000","last_active":"2024-09-10T15:59:14.136333+0000","last_peered":"2024-09-10T15:59:14.115595+0000","last_clean":"2024-09-10T15:59:14.115595+0000","last_became_active":"2024-09-10T15:59:09.089584+0000","last_became_peered":"2024-09-10T15:59:09.089584+0000","last_unstale":"2024-09-10T15:59:15.774437+0000","last_undegraded":"2024-09-10T15:59:15.774437+0000","last_fullsized":"2024-09-10T15:59:15.774437+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":4211,"log_dups_size":0,"ondisk_log_size":4211,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17215488,"num_objects":4203,"num_object_clones":0,"num_object_copies":8406,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4203,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":601,"num_write_kb":2404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4203}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":88,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.773583+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T15:59:15.773583+0000","last_peered":"2024-09-10T15:59:15.773583+0000","last_clean":"2024-09-10T15:59:15.773583+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T15:59:15.773583+0000","last_undegraded":"2024-09-10T15:59:15.773583+0000","last_fullsized":"2024-09-10T15:59:15.773583+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"40'4375","reported_seq":4507,"reported_epoch":43,"state":"activating+undersized+remapped","last_fresh":"2024-09-10T15:59:15.772756+0000","last_change":"2024-09-10T15:59:15.883434+0000","last_active":"2024-09-10T15:59:14.141008+0000","last_peered":"2024-09-10T15:59:14.116363+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:09.106202+0000","last_became_peered":"2024-09-10T15:59:09.106202+0000","last_unstale":"2024-09-10T15:59:15.772756+0000","last_undegraded":"2024-09-10T15:59:15.772756+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":4375,"log_dups_size":0,"ondisk_log_size":4375,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17903616,"num_objects":4371,"num_object_clones":0,"num_object_copies":8742,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4371,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":630,"num_write_kb":2520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.916017+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T15:59:15.916017+0000","last_peered":"2024-09-10T15:59:15.916017+0000","last_clean":"2024-09-10T15:59:15.916017+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T15:59:15.916017+0000","last_undegraded":"2024-09-10T15:59:15.916017+0000","last_fullsized":"2024-09-10T15:59:15.916017+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.776033+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T15:59:15.776033+0000","last_peered":"2024-09-10T15:59:15.776033+0000","last_clean":"2024-09-10T15:59:15.776033+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T15:59:15.776033+0000","last_undegraded":"2024-09-10T15:59:15.776033+0000","last_fullsized":"2024-09-10T15:59:15.776033+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"40'4334","reported_seq":4469,"reported_epoch":43,"state":"activating+undersized+remapped","last_fresh":"2024-09-10T15:59:15.768513+0000","last_change":"2024-09-10T15:59:15.887823+0000","last_active":"2024-09-10T15:59:14.141174+0000","last_peered":"2024-09-10T15:59:14.129780+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:09.083692+0000","last_became_peered":"2024-09-10T15:59:09.083692+0000","last_unstale":"2024-09-10T15:59:15.768513+0000","last_undegraded":"2024-09-10T15:59:15.768513+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":4334,"log_dups_size":0,"ondisk_log_size":4334,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17703878,"num_objects":4324,"num_object_clones":0,"num_object_copies":8648,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4324,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":497,"num_write_kb":1988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":283382494,"num_objects":69194,"num_object_clones":0,"num_object_copies":138388,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":47117,"num_objects_misplaced":4202,"num_objects_unfound":0,"num_objects_dirty":69194,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":27040,"num_write_kb":108160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":282624,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":568967168,"data_stored":568811809,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":69266,"ondisk_log_size":69266,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986192,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":416964,"kb_used_data":276976,"kb_used_omap":4,"kb_used_meta":139963,"kb_avail":93954876,"statfs":{"total":96636764160,"available":96209793024,"internally_reserved":0,"allocated":283623424,"data_stored":283345664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":143322794},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,7],"upper_bound":4096},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659138,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52352,"kb_used_data":2604,"kb_used_omap":4,"kb_used_meta":49723,"kb_avail":94319488,"statfs":{"total":96636764160,"available":96583155712,"internally_reserved":0,"allocated":2666496,"data_stored":2337690,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":50917038},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182420,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":422820,"kb_used_data":274568,"kb_used_omap":4,"kb_used_meta":148219,"kb_avail":93949020,"statfs":{"total":96636764160,"available":96203796480,"internally_reserved":0,"allocated":281157632,"data_stored":280875795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":151776938},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,13],"upper_bound":4096},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":280793088,"data_stored":280756958,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":3829760,"data_stored":3746661,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":284344320,"data_stored":284308190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:59:19.938 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-10T15:59:19.938 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "40'4259", 'reported_seq': 4324, 'reported_epoch': 43, 'state': 'activating+undersized+remapped', 'last_fresh': '2024-09-10T15:59:15.774356+0000', 'last_change': '2024-09-10T15:59:15.851279+0000', 'last_active': '2024-09-10T15:59:14.134656+0000', 'last_peered': '2024-09-10T15:59:14.132456+0000', 'last_clean': '2024-09-10T15:59:14.132456+0000', 'last_became_active': '2024-09-10T15:59:09.094143+0000', 'last_became_peered': '2024-09-10T15:59:09.094143+0000', 'last_unstale': '2024-09-10T15:59:15.774356+0000', 'last_undegraded': '2024-09-10T15:59:15.774356+0000', 'last_fullsized': '2024-09-10T15:59:15.774356+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'objects_scrubbed': 0, 'log_size': 4259, 'log_dups_size': 0, 'ondisk_log_size': 4259, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17444864, 'num_objects': 4259, 'num_object_clones': 0, 'num_object_copies': 8518, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4259, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4259, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4259, 'num_write_kb': 17036, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0'], 'object_location_counts': [{'shards': '0', 'objects': 4259}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:59:19.938 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-10T15:59:19.938 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "43'4376", 'reported_seq': 4633, 'reported_epoch': 43, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-10T15:59:19.182418+0000', 'last_change': '2024-09-10T15:59:16.142612+0000', 'last_active': '2024-09-10T15:59:19.182418+0000', 'last_peered': '2024-09-10T15:59:19.182418+0000', 'last_clean': '2024-09-10T15:59:14.092473+0000', 'last_became_active': '2024-09-10T15:59:16.109611+0000', 'last_became_peered': '2024-09-10T15:59:16.109611+0000', 'last_unstale': '2024-09-10T15:59:19.182418+0000', 'last_undegraded': '2024-09-10T15:59:19.182418+0000', 'last_fullsized': '2024-09-10T15:59:15.770264+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2029", 'last_scrub_stamp': '2024-09-10T15:58:45.933789+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:45.933789+0000', 'objects_scrubbed': 1992, 'log_size': 4376, 'log_dups_size': 0, 'ondisk_log_size': 4376, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 668, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17907712, 'num_objects': 4372, 'num_object_clones': 0, 'num_object_copies': 8744, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 4202, 'num_objects_unfound': 0, 'num_objects_dirty': 4372, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2380, 'num_write_kb': 9520, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 66, 'num_bytes_recovered': 270336, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4202}, {'shards': '2', 'objects': 170}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:59:19.938 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-10T15:59:19.939 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "40'4375", 'reported_seq': 4503, 'reported_epoch': 43, 'state': 'activating+undersized+remapped', 'last_fresh': '2024-09-10T15:59:15.769328+0000', 'last_change': '2024-09-10T15:59:15.878234+0000', 'last_active': '2024-09-10T15:59:14.135985+0000', 'last_peered': '2024-09-10T15:59:14.111551+0000', 'last_clean': '2024-09-10T15:59:14.111551+0000', 'last_became_active': '2024-09-10T15:59:09.086899+0000', 'last_became_peered': '2024-09-10T15:59:09.086899+0000', 'last_unstale': '2024-09-10T15:59:15.769328+0000', 'last_undegraded': '2024-09-10T15:59:15.769328+0000', 'last_fullsized': '2024-09-10T15:59:15.769328+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3442", 'last_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'objects_scrubbed': 3384, 'log_size': 4375, 'log_dups_size': 0, 'ondisk_log_size': 4375, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1292, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17900003, 'num_objects': 4371, 'num_object_clones': 0, 'num_object_copies': 8742, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4372, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4371, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 987, 'num_write_kb': 3948, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4371}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:59:19.939 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-10T15:59:19.939 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "40'4241", 'reported_seq': 4369, 'reported_epoch': 43, 'state': 'activating+undersized+remapped', 'last_fresh': '2024-09-10T15:59:15.769042+0000', 'last_change': '2024-09-10T15:59:15.877524+0000', 'last_active': '2024-09-10T15:59:14.142085+0000', 'last_peered': '2024-09-10T15:59:14.139436+0000', 'last_clean': '2024-09-10T15:59:14.139436+0000', 'last_became_active': '2024-09-10T15:59:09.106757+0000', 'last_became_peered': '2024-09-10T15:59:09.106757+0000', 'last_unstale': '2024-09-10T15:59:15.769042+0000', 'last_undegraded': '2024-09-10T15:59:15.769042+0000', 'last_fullsized': '2024-09-10T15:59:15.769042+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3537", 'last_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'objects_scrubbed': 3497, 'log_size': 4241, 'log_dups_size': 0, 'ondisk_log_size': 4241, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1224, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17347526, 'num_objects': 4237, 'num_object_clones': 0, 'num_object_copies': 8474, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4237, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4237, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 740, 'num_write_kb': 2960, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4237}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:59:19.939 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-10T15:59:19.939 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "43'4249", 'reported_seq': 4383, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:59:19.136107+0000', 'last_change': '2024-09-10T15:59:16.175680+0000', 'last_active': '2024-09-10T15:59:19.136107+0000', 'last_peered': '2024-09-10T15:59:19.136107+0000', 'last_clean': '2024-09-10T15:59:14.088187+0000', 'last_became_active': '2024-09-10T15:59:16.165810+0000', 'last_became_peered': '2024-09-10T15:59:16.165810+0000', 'last_unstale': '2024-09-10T15:59:19.136107+0000', 'last_undegraded': '2024-09-10T15:59:15.822098+0000', 'last_fullsized': '2024-09-10T15:59:15.769123+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2218", 'last_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'objects_scrubbed': 2175, 'log_size': 4249, 'log_dups_size': 0, 'ondisk_log_size': 4249, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 825, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17375715, 'num_objects': 4243, 'num_object_clones': 0, 'num_object_copies': 8486, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4124, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4243, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 2068, 'num_write_kb': 8272, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4123}, {'shards': '2', 'objects': 120}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:59:19.939 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-10T15:59:19.939 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "40'4263", 'reported_seq': 4384, 'reported_epoch': 43, 'state': 'activating+undersized+remapped', 'last_fresh': '2024-09-10T15:59:15.771410+0000', 'last_change': '2024-09-10T15:59:15.860857+0000', 'last_active': '2024-09-10T15:59:14.145841+0000', 'last_peered': '2024-09-10T15:59:14.129081+0000', 'last_clean': '2024-09-10T15:59:14.129081+0000', 'last_became_active': '2024-09-10T15:59:09.087496+0000', 'last_became_peered': '2024-09-10T15:59:09.087496+0000', 'last_unstale': '2024-09-10T15:59:15.771410+0000', 'last_undegraded': '2024-09-10T15:59:15.771410+0000', 'last_fullsized': '2024-09-10T15:59:15.771410+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3557", 'last_scrub_stamp': '2024-09-10T15:59:02.493678+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:02.493678+0000', 'objects_scrubbed': 3488, 'log_size': 4263, 'log_dups_size': 0, 'ondisk_log_size': 4263, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1472, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17428480, 'num_objects': 4255, 'num_object_clones': 0, 'num_object_copies': 8510, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4257, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4255, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 767, 'num_write_kb': 3068, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4255}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:59:19.939 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-10T15:59:19.939 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "43'4413", 'reported_seq': 4541, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:59:19.051655+0000', 'last_change': '2024-09-10T15:59:16.284311+0000', 'last_active': '2024-09-10T15:59:19.051655+0000', 'last_peered': '2024-09-10T15:59:19.051655+0000', 'last_clean': '2024-09-10T15:59:14.116041+0000', 'last_became_active': '2024-09-10T15:59:16.274543+0000', 'last_became_peered': '2024-09-10T15:59:16.274543+0000', 'last_unstale': '2024-09-10T15:59:19.051655+0000', 'last_undegraded': '2024-09-10T15:59:15.812484+0000', 'last_fullsized': '2024-09-10T15:59:15.770885+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2356", 'last_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'objects_scrubbed': 2316, 'log_size': 4413, 'log_dups_size': 0, 'ondisk_log_size': 4413, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 783, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18055651, 'num_objects': 4409, 'num_object_clones': 0, 'num_object_copies': 8818, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4409, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 2093, 'num_write_kb': 8372, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4304}, {'shards': '2', 'objects': 105}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:59:19.939 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-10T15:59:19.939 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "43'4464", 'reported_seq': 4587, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:59:19.137413+0000', 'last_change': '2024-09-10T15:59:16.174113+0000', 'last_active': '2024-09-10T15:59:19.137413+0000', 'last_peered': '2024-09-10T15:59:19.137413+0000', 'last_clean': '2024-09-10T15:59:14.110654+0000', 'last_became_active': '2024-09-10T15:59:16.164645+0000', 'last_became_peered': '2024-09-10T15:59:16.164645+0000', 'last_unstale': '2024-09-10T15:59:19.137413+0000', 'last_undegraded': '2024-09-10T15:59:15.803593+0000', 'last_fullsized': '2024-09-10T15:59:15.769083+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2486", 'last_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'objects_scrubbed': 2440, 'log_size': 4464, 'log_dups_size': 0, 'ondisk_log_size': 4464, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 792, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18259968, 'num_objects': 4458, 'num_object_clones': 0, 'num_object_copies': 8916, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4354, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4458, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2018, 'num_write_kb': 8072, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4352}, {'shards': '2', 'objects': 106}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:59:19.940 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-10T15:59:19.940 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "40'4306", 'reported_seq': 4431, 'reported_epoch': 43, 'state': 'activating+undersized+remapped', 'last_fresh': '2024-09-10T15:59:15.770337+0000', 'last_change': '2024-09-10T15:59:15.845012+0000', 'last_active': '2024-09-10T15:59:14.135619+0000', 'last_peered': '2024-09-10T15:59:14.117001+0000', 'last_clean': '2024-09-10T15:59:14.117001+0000', 'last_became_active': '2024-09-10T15:59:09.093889+0000', 'last_became_peered': '2024-09-10T15:59:09.093889+0000', 'last_unstale': '2024-09-10T15:59:15.770337+0000', 'last_undegraded': '2024-09-10T15:59:15.770337+0000', 'last_fullsized': '2024-09-10T15:59:15.770337+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3404", 'last_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'objects_scrubbed': 3367, 'log_size': 4306, 'log_dups_size': 0, 'ondisk_log_size': 4306, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 983, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17629184, 'num_objects': 4304, 'num_object_clones': 0, 'num_object_copies': 8608, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4304, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 937, 'num_write_kb': 3748, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4304}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:59:19.940 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-10T15:59:19.940 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "40'4211", 'reported_seq': 4344, 'reported_epoch': 43, 'state': 'activating+undersized+remapped', 'last_fresh': '2024-09-10T15:59:15.774437+0000', 'last_change': '2024-09-10T15:59:15.847127+0000', 'last_active': '2024-09-10T15:59:14.136333+0000', 'last_peered': '2024-09-10T15:59:14.115595+0000', 'last_clean': '2024-09-10T15:59:14.115595+0000', 'last_became_active': '2024-09-10T15:59:09.089584+0000', 'last_became_peered': '2024-09-10T15:59:09.089584+0000', 'last_unstale': '2024-09-10T15:59:15.774437+0000', 'last_undegraded': '2024-09-10T15:59:15.774437+0000', 'last_fullsized': '2024-09-10T15:59:15.774437+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3653", 'last_scrub_stamp': '2024-09-10T15:59:05.298468+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:05.298468+0000', 'objects_scrubbed': 3602, 'log_size': 4211, 'log_dups_size': 0, 'ondisk_log_size': 4211, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1289, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17215488, 'num_objects': 4203, 'num_object_clones': 0, 'num_object_copies': 8406, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4206, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4203, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 601, 'num_write_kb': 2404, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4203}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:59:19.940 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-10T15:59:19.940 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "40'4375", 'reported_seq': 4507, 'reported_epoch': 43, 'state': 'activating+undersized+remapped', 'last_fresh': '2024-09-10T15:59:15.772756+0000', 'last_change': '2024-09-10T15:59:15.883434+0000', 'last_active': '2024-09-10T15:59:14.141008+0000', 'last_peered': '2024-09-10T15:59:14.116363+0000', 'last_clean': '2024-09-10T15:59:14.116363+0000', 'last_became_active': '2024-09-10T15:59:09.106202+0000', 'last_became_peered': '2024-09-10T15:59:09.106202+0000', 'last_unstale': '2024-09-10T15:59:15.772756+0000', 'last_undegraded': '2024-09-10T15:59:15.772756+0000', 'last_fullsized': '2024-09-10T15:59:15.772756+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3789", 'last_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'objects_scrubbed': 3741, 'log_size': 4375, 'log_dups_size': 0, 'ondisk_log_size': 4375, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1440, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17903616, 'num_objects': 4371, 'num_object_clones': 0, 'num_object_copies': 8742, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4373, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4371, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 630, 'num_write_kb': 2520, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4371}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:59:19.940 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-10T15:59:19.940 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "40'4334", 'reported_seq': 4469, 'reported_epoch': 43, 'state': 'activating+undersized+remapped', 'last_fresh': '2024-09-10T15:59:15.768513+0000', 'last_change': '2024-09-10T15:59:15.887823+0000', 'last_active': '2024-09-10T15:59:14.141174+0000', 'last_peered': '2024-09-10T15:59:14.129780+0000', 'last_clean': '2024-09-10T15:59:14.129780+0000', 'last_became_active': '2024-09-10T15:59:09.083692+0000', 'last_became_peered': '2024-09-10T15:59:09.083692+0000', 'last_unstale': '2024-09-10T15:59:15.768513+0000', 'last_undegraded': '2024-09-10T15:59:15.768513+0000', 'last_fullsized': '2024-09-10T15:59:15.768513+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3880", 'last_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'objects_scrubbed': 3827, 'log_size': 4334, 'log_dups_size': 0, 'ondisk_log_size': 4334, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1397, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17703878, 'num_objects': 4324, 'num_object_clones': 0, 'num_object_copies': 8648, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4325, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4324, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 497, 'num_write_kb': 1988, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:59:19.941 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T15:59:19.965 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 47 20 70725 70705 5.87579 2.17578 0.0557594 0.0132893 2024-09-10T15:59:20.648 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T15:59:20.648 INFO:teuthology.orchestra.run.smithi115.stdout:{"fsid":"7effee29-df05-4351-9741-ee203564ed81","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 47117/138392 objects degraded (34.046%), 3 pgs degraded","count":3},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":124,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":43,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983954,"num_in_osds":3,"osd_in_since":1725983947,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"activating+undersized+remapped","count":8},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":3},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":69196,"data_bytes":283382513,"bytes_used":913547264,"bytes_avail":288996745216,"bytes_total":289910292480,"inactive_pgs_ratio":0.3333333432674408,"degraded_objects":47117,"degraded_total":138392,"degraded_ratio":0.34046043123879988,"misplaced_objects":4202,"misplaced_total":138392,"misplaced_ratio":0.030363026764552867,"recovering_objects_per_sec":10,"recovering_bytes_per_sec":44364,"recovering_keys_per_sec":0,"num_objects_recovered":65,"num_bytes_recovered":266240,"num_keys_recovered":0,"write_bytes_sec":3616070,"read_op_per_sec":0,"write_op_per_sec":882},"fsmap":{"epoch":1,"btime":"2024-09-10T15:57:14:967120+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":5,"modified":"2024-09-10T15:59:19.546323+0000","services":{"osd":{"daemons":{"summary":"","1":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-10T15:59:20.671 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-10T15:59:20.671 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:59:20.965 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 48 20 71287 71267 5.79911 2.19531 0.0778836 0.0134619 2024-09-10T15:59:21.263 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T15:59:21.266 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T15:59:21.293 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":98,"stamp":"2024-09-10T15:59:19.546233+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":283382513,"num_objects":69196,"num_object_clones":0,"num_object_copies":138392,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":47117,"num_objects_misplaced":4202,"num_objects_unfound":0,"num_objects_dirty":69196,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":27042,"num_write_kb":108162,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":71,"num_bytes_recovered":282643,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":69269,"ondisk_log_size":69269,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":66,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":892136,"kb_used_data":554148,"kb_used_omap":12,"kb_used_meta":337907,"kb_avail":282223384,"statfs":{"total":289910292480,"available":288996745216,"internally_reserved":0,"allocated":567447552,"data_stored":566559149,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":346016770},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,20],"upper_bound":4096},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":21700608,"num_objects":5298,"num_object_clones":0,"num_object_copies":10596,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":47117,"num_objects_misplaced":-43740,"num_objects_unfound":0,"num_objects_dirty":5298,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5298,"num_write_kb":21192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":65,"num_bytes_recovered":266240,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001157"},"pg_stats":[{"pgid":"2.f","version":"40'4259","reported_seq":4324,"reported_epoch":43,"state":"activating+undersized+remapped","last_fresh":"2024-09-10T15:59:15.774356+0000","last_change":"2024-09-10T15:59:15.851279+0000","last_active":"2024-09-10T15:59:14.134656+0000","last_peered":"2024-09-10T15:59:14.132456+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:09.094143+0000","last_became_peered":"2024-09-10T15:59:09.094143+0000","last_unstale":"2024-09-10T15:59:15.774356+0000","last_undegraded":"2024-09-10T15:59:15.774356+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":4259,"log_dups_size":0,"ondisk_log_size":4259,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17444864,"num_objects":4259,"num_object_clones":0,"num_object_copies":8518,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4259,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4259,"num_write_kb":17036,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0"],"object_location_counts":[{"shards":"0","objects":4259}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'4398","reported_seq":4505,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:19.044943+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T15:59:19.044943+0000","last_peered":"2024-09-10T15:59:19.044943+0000","last_clean":"2024-09-10T15:59:19.044943+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T15:59:19.044943+0000","last_undegraded":"2024-09-10T15:59:19.044943+0000","last_fullsized":"2024-09-10T15:59:19.044943+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":4398,"log_dups_size":0,"ondisk_log_size":4398,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17990598,"num_objects":4394,"num_object_clones":0,"num_object_copies":8788,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4394,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":2503,"num_write_kb":10012,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'4376","reported_seq":4633,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T15:59:19.182418+0000","last_change":"2024-09-10T15:59:16.142612+0000","last_active":"2024-09-10T15:59:19.182418+0000","last_peered":"2024-09-10T15:59:19.182418+0000","last_clean":"2024-09-10T15:59:14.092473+0000","last_became_active":"2024-09-10T15:59:16.109611+0000","last_became_peered":"2024-09-10T15:59:16.109611+0000","last_unstale":"2024-09-10T15:59:19.182418+0000","last_undegraded":"2024-09-10T15:59:19.182418+0000","last_fullsized":"2024-09-10T15:59:15.770264+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":4376,"log_dups_size":0,"ondisk_log_size":4376,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17907712,"num_objects":4372,"num_object_clones":0,"num_object_copies":8744,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":4202,"num_objects_unfound":0,"num_objects_dirty":4372,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2380,"num_write_kb":9520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":270336,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4202},{"shards":"2","objects":170}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'4336","reported_seq":4411,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:19.206436+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T15:59:19.206436+0000","last_peered":"2024-09-10T15:59:19.206436+0000","last_clean":"2024-09-10T15:59:19.206436+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T15:59:19.206436+0000","last_undegraded":"2024-09-10T15:59:19.206436+0000","last_fullsized":"2024-09-10T15:59:19.206436+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":4336,"log_dups_size":0,"ondisk_log_size":4336,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17760256,"num_objects":4336,"num_object_clones":0,"num_object_copies":8672,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4336,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4336,"num_write_kb":17344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"41'4314","reported_seq":4438,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.769953+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T15:59:15.769953+0000","last_peered":"2024-09-10T15:59:15.769953+0000","last_clean":"2024-09-10T15:59:15.769953+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T15:59:15.769953+0000","last_undegraded":"2024-09-10T15:59:15.769953+0000","last_fullsized":"2024-09-10T15:59:15.769953+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":4314,"log_dups_size":0,"ondisk_log_size":4314,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17641955,"num_objects":4308,"num_object_clones":0,"num_object_copies":8616,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4308,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1025,"num_write_kb":4100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"40'4375","reported_seq":4503,"reported_epoch":43,"state":"activating+undersized+remapped","last_fresh":"2024-09-10T15:59:15.769328+0000","last_change":"2024-09-10T15:59:15.878234+0000","last_active":"2024-09-10T15:59:14.135985+0000","last_peered":"2024-09-10T15:59:14.111551+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:09.086899+0000","last_became_peered":"2024-09-10T15:59:09.086899+0000","last_unstale":"2024-09-10T15:59:15.769328+0000","last_undegraded":"2024-09-10T15:59:15.769328+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":4375,"log_dups_size":0,"ondisk_log_size":4375,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17900003,"num_objects":4371,"num_object_clones":0,"num_object_copies":8742,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4371,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":987,"num_write_kb":3948,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"40'4241","reported_seq":4369,"reported_epoch":43,"state":"activating+undersized+remapped","last_fresh":"2024-09-10T15:59:15.769042+0000","last_change":"2024-09-10T15:59:15.877524+0000","last_active":"2024-09-10T15:59:14.142085+0000","last_peered":"2024-09-10T15:59:14.139436+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:09.106757+0000","last_became_peered":"2024-09-10T15:59:09.106757+0000","last_unstale":"2024-09-10T15:59:15.769042+0000","last_undegraded":"2024-09-10T15:59:15.769042+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":4241,"log_dups_size":0,"ondisk_log_size":4241,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17347526,"num_objects":4237,"num_object_clones":0,"num_object_copies":8474,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4237,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":740,"num_write_kb":2960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4237}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"40'4352","reported_seq":4476,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.772151+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T15:59:15.772151+0000","last_peered":"2024-09-10T15:59:15.772151+0000","last_clean":"2024-09-10T15:59:15.772151+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T15:59:15.772151+0000","last_undegraded":"2024-09-10T15:59:15.772151+0000","last_fullsized":"2024-09-10T15:59:15.772151+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":4352,"log_dups_size":0,"ondisk_log_size":4352,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17817600,"num_objects":4350,"num_object_clones":0,"num_object_copies":8700,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4350,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1199,"num_write_kb":4796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.775865+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T15:59:15.775865+0000","last_peered":"2024-09-10T15:59:15.775865+0000","last_clean":"2024-09-10T15:59:15.775865+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T15:59:15.775865+0000","last_undegraded":"2024-09-10T15:59:15.775865+0000","last_fullsized":"2024-09-10T15:59:15.775865+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'4249","reported_seq":4383,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:19.136107+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T15:59:19.136107+0000","last_peered":"2024-09-10T15:59:19.136107+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T15:59:19.136107+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":4249,"log_dups_size":0,"ondisk_log_size":4249,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17375715,"num_objects":4243,"num_object_clones":0,"num_object_copies":8486,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4243,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2068,"num_write_kb":8272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4123},{"shards":"2","objects":120}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":88,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.772572+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T15:59:15.772572+0000","last_peered":"2024-09-10T15:59:15.772572+0000","last_clean":"2024-09-10T15:59:15.772572+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T15:59:15.772572+0000","last_undegraded":"2024-09-10T15:59:15.772572+0000","last_fullsized":"2024-09-10T15:59:15.772572+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"40'4263","reported_seq":4384,"reported_epoch":43,"state":"activating+undersized+remapped","last_fresh":"2024-09-10T15:59:15.771410+0000","last_change":"2024-09-10T15:59:15.860857+0000","last_active":"2024-09-10T15:59:14.145841+0000","last_peered":"2024-09-10T15:59:14.129081+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:09.087496+0000","last_became_peered":"2024-09-10T15:59:09.087496+0000","last_unstale":"2024-09-10T15:59:15.771410+0000","last_undegraded":"2024-09-10T15:59:15.771410+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":4263,"log_dups_size":0,"ondisk_log_size":4263,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17428480,"num_objects":4255,"num_object_clones":0,"num_object_copies":8510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4255,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":767,"num_write_kb":3068,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4255}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":98,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.880567+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T15:59:15.880567+0000","last_peered":"2024-09-10T15:59:15.880567+0000","last_clean":"2024-09-10T15:59:15.880567+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T15:59:15.880567+0000","last_undegraded":"2024-09-10T15:59:15.880567+0000","last_fullsized":"2024-09-10T15:59:15.880567+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'4413","reported_seq":4541,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:19.051655+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T15:59:19.051655+0000","last_peered":"2024-09-10T15:59:19.051655+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T15:59:19.051655+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":4413,"log_dups_size":0,"ondisk_log_size":4413,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18055651,"num_objects":4409,"num_object_clones":0,"num_object_copies":8818,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4409,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2093,"num_write_kb":8372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4304},{"shards":"2","objects":105}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.837509+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T15:59:15.837509+0000","last_peered":"2024-09-10T15:59:15.837509+0000","last_clean":"2024-09-10T15:59:15.837509+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T15:59:15.837509+0000","last_undegraded":"2024-09-10T15:59:15.837509+0000","last_fullsized":"2024-09-10T15:59:15.837509+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'4464","reported_seq":4587,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:19.137413+0000","last_change":"2024-09-10T15:59:16.174113+0000","last_active":"2024-09-10T15:59:19.137413+0000","last_peered":"2024-09-10T15:59:19.137413+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T15:59:19.137413+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":4464,"log_dups_size":0,"ondisk_log_size":4464,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18259968,"num_objects":4458,"num_object_clones":0,"num_object_copies":8916,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4354,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4458,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2018,"num_write_kb":8072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4352},{"shards":"2","objects":106}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"40'4306","reported_seq":4431,"reported_epoch":43,"state":"activating+undersized+remapped","last_fresh":"2024-09-10T15:59:15.770337+0000","last_change":"2024-09-10T15:59:15.845012+0000","last_active":"2024-09-10T15:59:14.135619+0000","last_peered":"2024-09-10T15:59:14.117001+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:09.093889+0000","last_became_peered":"2024-09-10T15:59:09.093889+0000","last_unstale":"2024-09-10T15:59:15.770337+0000","last_undegraded":"2024-09-10T15:59:15.770337+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":4306,"log_dups_size":0,"ondisk_log_size":4306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17629184,"num_objects":4304,"num_object_clones":0,"num_object_copies":8608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4304,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":937,"num_write_kb":3748,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.989413+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T15:59:15.989413+0000","last_peered":"2024-09-10T15:59:15.989413+0000","last_clean":"2024-09-10T15:59:15.989413+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T15:59:15.989413+0000","last_undegraded":"2024-09-10T15:59:15.989413+0000","last_fullsized":"2024-09-10T15:59:15.989413+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"40'4211","reported_seq":4344,"reported_epoch":43,"state":"activating+undersized+remapped","last_fresh":"2024-09-10T15:59:15.774437+0000","last_change":"2024-09-10T15:59:15.847127+0000","last_active":"2024-09-10T15:59:14.136333+0000","last_peered":"2024-09-10T15:59:14.115595+0000","last_clean":"2024-09-10T15:59:14.115595+0000","last_became_active":"2024-09-10T15:59:09.089584+0000","last_became_peered":"2024-09-10T15:59:09.089584+0000","last_unstale":"2024-09-10T15:59:15.774437+0000","last_undegraded":"2024-09-10T15:59:15.774437+0000","last_fullsized":"2024-09-10T15:59:15.774437+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":4211,"log_dups_size":0,"ondisk_log_size":4211,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17215488,"num_objects":4203,"num_object_clones":0,"num_object_copies":8406,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4203,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":601,"num_write_kb":2404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4203}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":88,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.773583+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T15:59:15.773583+0000","last_peered":"2024-09-10T15:59:15.773583+0000","last_clean":"2024-09-10T15:59:15.773583+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T15:59:15.773583+0000","last_undegraded":"2024-09-10T15:59:15.773583+0000","last_fullsized":"2024-09-10T15:59:15.773583+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"40'4375","reported_seq":4507,"reported_epoch":43,"state":"activating+undersized+remapped","last_fresh":"2024-09-10T15:59:15.772756+0000","last_change":"2024-09-10T15:59:15.883434+0000","last_active":"2024-09-10T15:59:14.141008+0000","last_peered":"2024-09-10T15:59:14.116363+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:09.106202+0000","last_became_peered":"2024-09-10T15:59:09.106202+0000","last_unstale":"2024-09-10T15:59:15.772756+0000","last_undegraded":"2024-09-10T15:59:15.772756+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":4375,"log_dups_size":0,"ondisk_log_size":4375,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17903616,"num_objects":4371,"num_object_clones":0,"num_object_copies":8742,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4371,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":630,"num_write_kb":2520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.916017+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T15:59:15.916017+0000","last_peered":"2024-09-10T15:59:15.916017+0000","last_clean":"2024-09-10T15:59:15.916017+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T15:59:15.916017+0000","last_undegraded":"2024-09-10T15:59:15.916017+0000","last_fullsized":"2024-09-10T15:59:15.916017+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.776033+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T15:59:15.776033+0000","last_peered":"2024-09-10T15:59:15.776033+0000","last_clean":"2024-09-10T15:59:15.776033+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T15:59:15.776033+0000","last_undegraded":"2024-09-10T15:59:15.776033+0000","last_fullsized":"2024-09-10T15:59:15.776033+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"40'4334","reported_seq":4469,"reported_epoch":43,"state":"activating+undersized+remapped","last_fresh":"2024-09-10T15:59:15.768513+0000","last_change":"2024-09-10T15:59:15.887823+0000","last_active":"2024-09-10T15:59:14.141174+0000","last_peered":"2024-09-10T15:59:14.129780+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:09.083692+0000","last_became_peered":"2024-09-10T15:59:09.083692+0000","last_unstale":"2024-09-10T15:59:15.768513+0000","last_undegraded":"2024-09-10T15:59:15.768513+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":4334,"log_dups_size":0,"ondisk_log_size":4334,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17703878,"num_objects":4324,"num_object_clones":0,"num_object_copies":8648,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4324,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":497,"num_write_kb":1988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":283382494,"num_objects":69194,"num_object_clones":0,"num_object_copies":138388,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":47117,"num_objects_misplaced":4202,"num_objects_unfound":0,"num_objects_dirty":69194,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":27040,"num_write_kb":108160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":282624,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":568967168,"data_stored":568811809,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":69266,"ondisk_log_size":69266,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986192,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":416964,"kb_used_data":276976,"kb_used_omap":4,"kb_used_meta":139963,"kb_avail":93954876,"statfs":{"total":96636764160,"available":96209793024,"internally_reserved":0,"allocated":283623424,"data_stored":283345664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":143322794},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,7],"upper_bound":4096},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659138,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52352,"kb_used_data":2604,"kb_used_omap":4,"kb_used_meta":49723,"kb_avail":94319488,"statfs":{"total":96636764160,"available":96583155712,"internally_reserved":0,"allocated":2666496,"data_stored":2337690,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":50917038},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182420,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":422820,"kb_used_data":274568,"kb_used_omap":4,"kb_used_meta":148219,"kb_avail":93949020,"statfs":{"total":96636764160,"available":96203796480,"internally_reserved":0,"allocated":281157632,"data_stored":280875795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":151776938},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,13],"upper_bound":4096},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":280793088,"data_stored":280756958,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":3829760,"data_stored":3746661,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":284344320,"data_stored":284308190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:59:21.966 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 49 20 71913 71893 5.73066 2.44531 0.033563 0.0136249 2024-09-10T15:59:22.966 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 50 20 72574 72554 5.66768 2.58203 0.0268927 0.0137739 2024-09-10T15:59:23.966 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 51 20 73215 73195 5.60564 2.50391 0.0679921 0.0139272 2024-09-10T15:59:24.296 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:59:24.843 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T15:59:24.845 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T15:59:24.872 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":100,"stamp":"2024-09-10T15:59:23.547102+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":290702065,"num_objects":70983,"num_object_clones":0,"num_object_copies":141966,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":47117,"num_objects_misplaced":4202,"num_objects_unfound":0,"num_objects_dirty":70983,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":28829,"num_write_kb":115310,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":71,"num_bytes_recovered":282643,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":71056,"ondisk_log_size":71056,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":901248,"kb_used_data":571596,"kb_used_omap":12,"kb_used_meta":329587,"kb_avail":282214272,"statfs":{"total":289910292480,"available":288987414528,"internally_reserved":0,"allocated":585314304,"data_stored":584425901,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":337497090},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,9,5,9,6,0,0,0,0,7],"upper_bound":4096},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19673088,"num_objects":4803,"num_object_clones":0,"num_object_copies":9606,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":47117,"num_objects_misplaced":-28227,"num_objects_unfound":0,"num_objects_dirty":4803,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4803,"num_write_kb":19212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":65,"num_bytes_recovered":266240,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.781862"},"pg_stats":[{"pgid":"2.f","version":"43'4448","reported_seq":4517,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:21.091317+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T15:59:21.091317+0000","last_peered":"2024-09-10T15:59:21.091317+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T15:59:21.091317+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":4448,"log_dups_size":0,"ondisk_log_size":4448,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18219008,"num_objects":4448,"num_object_clones":0,"num_object_copies":8896,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4448,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4448,"num_write_kb":17792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":189},{"shards":"0,2","objects":4258}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'4398","reported_seq":4505,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:19.044943+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T15:59:19.044943+0000","last_peered":"2024-09-10T15:59:19.044943+0000","last_clean":"2024-09-10T15:59:19.044943+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T15:59:19.044943+0000","last_undegraded":"2024-09-10T15:59:19.044943+0000","last_fullsized":"2024-09-10T15:59:19.044943+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":4398,"log_dups_size":0,"ondisk_log_size":4398,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17990598,"num_objects":4394,"num_object_clones":0,"num_object_copies":8788,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4394,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":2503,"num_write_kb":10012,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'4376","reported_seq":4633,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T15:59:19.182418+0000","last_change":"2024-09-10T15:59:16.142612+0000","last_active":"2024-09-10T15:59:19.182418+0000","last_peered":"2024-09-10T15:59:19.182418+0000","last_clean":"2024-09-10T15:59:14.092473+0000","last_became_active":"2024-09-10T15:59:16.109611+0000","last_became_peered":"2024-09-10T15:59:16.109611+0000","last_unstale":"2024-09-10T15:59:19.182418+0000","last_undegraded":"2024-09-10T15:59:19.182418+0000","last_fullsized":"2024-09-10T15:59:15.770264+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":4376,"log_dups_size":0,"ondisk_log_size":4376,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17907712,"num_objects":4372,"num_object_clones":0,"num_object_copies":8744,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":4202,"num_objects_unfound":0,"num_objects_dirty":4372,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2380,"num_write_kb":9520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":270336,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4202},{"shards":"2","objects":170}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'4336","reported_seq":4411,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:19.206436+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T15:59:19.206436+0000","last_peered":"2024-09-10T15:59:19.206436+0000","last_clean":"2024-09-10T15:59:19.206436+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T15:59:19.206436+0000","last_undegraded":"2024-09-10T15:59:19.206436+0000","last_fullsized":"2024-09-10T15:59:19.206436+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":4336,"log_dups_size":0,"ondisk_log_size":4336,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17760256,"num_objects":4336,"num_object_clones":0,"num_object_copies":8672,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4336,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4336,"num_write_kb":17344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'4469","reported_seq":4593,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:21.101100+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T15:59:21.101100+0000","last_peered":"2024-09-10T15:59:21.101100+0000","last_clean":"2024-09-10T15:59:21.101100+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T15:59:21.101100+0000","last_undegraded":"2024-09-10T15:59:21.101100+0000","last_fullsized":"2024-09-10T15:59:21.101100+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":4469,"log_dups_size":0,"ondisk_log_size":4469,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18276835,"num_objects":4463,"num_object_clones":0,"num_object_copies":8926,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4463,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1180,"num_write_kb":4720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'4552","reported_seq":4684,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:21.134759+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T15:59:21.134759+0000","last_peered":"2024-09-10T15:59:21.134759+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T15:59:21.134759+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":4552,"log_dups_size":0,"ondisk_log_size":4552,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18624995,"num_objects":4548,"num_object_clones":0,"num_object_copies":9096,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4548,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1164,"num_write_kb":4656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":177},{"shards":"0,2","objects":4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'4412","reported_seq":4542,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:21.149654+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T15:59:21.149654+0000","last_peered":"2024-09-10T15:59:21.149654+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T15:59:21.149654+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":4412,"log_dups_size":0,"ondisk_log_size":4412,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18047942,"num_objects":4408,"num_object_clones":0,"num_object_copies":8816,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4408,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":911,"num_write_kb":3644,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":171},{"shards":"0,2","objects":4237}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'4529","reported_seq":4653,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:21.149463+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T15:59:21.149463+0000","last_peered":"2024-09-10T15:59:21.149463+0000","last_clean":"2024-09-10T15:59:21.149463+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T15:59:21.149463+0000","last_undegraded":"2024-09-10T15:59:21.149463+0000","last_fullsized":"2024-09-10T15:59:21.149463+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":4529,"log_dups_size":0,"ondisk_log_size":4529,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18542592,"num_objects":4527,"num_object_clones":0,"num_object_copies":9054,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4527,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1376,"num_write_kb":5504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.775865+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T15:59:15.775865+0000","last_peered":"2024-09-10T15:59:15.775865+0000","last_clean":"2024-09-10T15:59:15.775865+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T15:59:15.775865+0000","last_undegraded":"2024-09-10T15:59:15.775865+0000","last_fullsized":"2024-09-10T15:59:15.775865+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'4249","reported_seq":4383,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:19.136107+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T15:59:19.136107+0000","last_peered":"2024-09-10T15:59:19.136107+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T15:59:19.136107+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":4249,"log_dups_size":0,"ondisk_log_size":4249,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17375715,"num_objects":4243,"num_object_clones":0,"num_object_copies":8486,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4243,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2068,"num_write_kb":8272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4123},{"shards":"2","objects":120}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":88,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.772572+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T15:59:15.772572+0000","last_peered":"2024-09-10T15:59:15.772572+0000","last_clean":"2024-09-10T15:59:15.772572+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T15:59:15.772572+0000","last_undegraded":"2024-09-10T15:59:15.772572+0000","last_fullsized":"2024-09-10T15:59:15.772572+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'4439","reported_seq":4560,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:21.156792+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T15:59:21.156792+0000","last_peered":"2024-09-10T15:59:21.156792+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T15:59:21.156792+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":4439,"log_dups_size":0,"ondisk_log_size":4439,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18149376,"num_objects":4431,"num_object_clones":0,"num_object_copies":8862,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4431,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":943,"num_write_kb":3772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":176},{"shards":"0,2","objects":4255}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":98,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.880567+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T15:59:15.880567+0000","last_peered":"2024-09-10T15:59:15.880567+0000","last_clean":"2024-09-10T15:59:15.880567+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T15:59:15.880567+0000","last_undegraded":"2024-09-10T15:59:15.880567+0000","last_fullsized":"2024-09-10T15:59:15.880567+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'4413","reported_seq":4541,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:19.051655+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T15:59:19.051655+0000","last_peered":"2024-09-10T15:59:19.051655+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T15:59:19.051655+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":4413,"log_dups_size":0,"ondisk_log_size":4413,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18055651,"num_objects":4409,"num_object_clones":0,"num_object_copies":8818,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4409,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2093,"num_write_kb":8372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4304},{"shards":"2","objects":105}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.837509+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T15:59:15.837509+0000","last_peered":"2024-09-10T15:59:15.837509+0000","last_clean":"2024-09-10T15:59:15.837509+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T15:59:15.837509+0000","last_undegraded":"2024-09-10T15:59:15.837509+0000","last_fullsized":"2024-09-10T15:59:15.837509+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'4464","reported_seq":4587,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:19.137413+0000","last_change":"2024-09-10T15:59:16.174113+0000","last_active":"2024-09-10T15:59:19.137413+0000","last_peered":"2024-09-10T15:59:19.137413+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T15:59:19.137413+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":4464,"log_dups_size":0,"ondisk_log_size":4464,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18259968,"num_objects":4458,"num_object_clones":0,"num_object_copies":8916,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4354,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4458,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2018,"num_write_kb":8072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4352},{"shards":"2","objects":106}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'4498","reported_seq":4627,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:21.127244+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T15:59:21.127244+0000","last_peered":"2024-09-10T15:59:21.127244+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T15:59:21.127244+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":4498,"log_dups_size":0,"ondisk_log_size":4498,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18415616,"num_objects":4496,"num_object_clones":0,"num_object_copies":8992,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4496,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1129,"num_write_kb":4516,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":192},{"shards":"0,2","objects":4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.989413+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T15:59:15.989413+0000","last_peered":"2024-09-10T15:59:15.989413+0000","last_clean":"2024-09-10T15:59:15.989413+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T15:59:15.989413+0000","last_undegraded":"2024-09-10T15:59:15.989413+0000","last_fullsized":"2024-09-10T15:59:15.989413+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'4398","reported_seq":4535,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:21.130937+0000","last_change":"2024-09-10T15:59:16.251610+0000","last_active":"2024-09-10T15:59:21.130937+0000","last_peered":"2024-09-10T15:59:21.130937+0000","last_clean":"2024-09-10T15:59:14.115595+0000","last_became_active":"2024-09-10T15:59:16.244089+0000","last_became_peered":"2024-09-10T15:59:16.244089+0000","last_unstale":"2024-09-10T15:59:21.130937+0000","last_undegraded":"2024-09-10T15:59:15.847127+0000","last_fullsized":"2024-09-10T15:59:15.774437+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":4398,"log_dups_size":0,"ondisk_log_size":4398,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17981440,"num_objects":4390,"num_object_clones":0,"num_object_copies":8780,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4390,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":788,"num_write_kb":3152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":187},{"shards":"0,2","objects":4203}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":88,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.773583+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T15:59:15.773583+0000","last_peered":"2024-09-10T15:59:15.773583+0000","last_clean":"2024-09-10T15:59:15.773583+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T15:59:15.773583+0000","last_undegraded":"2024-09-10T15:59:15.773583+0000","last_fullsized":"2024-09-10T15:59:15.773583+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'4558","reported_seq":4690,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:21.155161+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T15:59:21.155161+0000","last_peered":"2024-09-10T15:59:21.155161+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T15:59:21.155161+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":4558,"log_dups_size":0,"ondisk_log_size":4558,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18653184,"num_objects":4554,"num_object_clones":0,"num_object_copies":9108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4554,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":813,"num_write_kb":3252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":183},{"shards":"0,2","objects":4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.916017+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T15:59:15.916017+0000","last_peered":"2024-09-10T15:59:15.916017+0000","last_clean":"2024-09-10T15:59:15.916017+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T15:59:15.916017+0000","last_undegraded":"2024-09-10T15:59:15.916017+0000","last_fullsized":"2024-09-10T15:59:15.916017+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.776033+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T15:59:15.776033+0000","last_peered":"2024-09-10T15:59:15.776033+0000","last_clean":"2024-09-10T15:59:15.776033+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T15:59:15.776033+0000","last_undegraded":"2024-09-10T15:59:15.776033+0000","last_fullsized":"2024-09-10T15:59:15.776033+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'4514","reported_seq":4653,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:21.133435+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T15:59:21.133435+0000","last_peered":"2024-09-10T15:59:21.133435+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T15:59:21.133435+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":4514,"log_dups_size":0,"ondisk_log_size":4514,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18441158,"num_objects":4504,"num_object_clones":0,"num_object_copies":9008,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4504,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":677,"num_write_kb":2708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":181},{"shards":"0,2","objects":4323}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":290702046,"num_objects":70981,"num_object_clones":0,"num_object_copies":141962,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":47117,"num_objects_misplaced":4202,"num_objects_unfound":0,"num_objects_dirty":70981,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":28827,"num_write_kb":115308,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":282624,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":587444224,"data_stored":587288865,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":71053,"ondisk_log_size":71053,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986192,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":416964,"kb_used_data":276976,"kb_used_omap":4,"kb_used_meta":139963,"kb_avail":93954876,"statfs":{"total":96636764160,"available":96209793024,"internally_reserved":0,"allocated":283623424,"data_stored":283345664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":143322794},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,7],"upper_bound":4096},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659139,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":65952,"kb_used_data":12048,"kb_used_omap":4,"kb_used_meta":53883,"kb_avail":94305888,"statfs":{"total":96636764160,"available":96569229312,"internally_reserved":0,"allocated":12337152,"data_stored":12008346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":55176878},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,4,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182421,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":418332,"kb_used_data":282572,"kb_used_omap":4,"kb_used_meta":135739,"kb_avail":93953508,"statfs":{"total":96636764160,"available":96208392192,"internally_reserved":0,"allocated":289353728,"data_stored":289071891,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":138997418},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,5,1,6,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":289460224,"data_stored":289424094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":13639680,"data_stored":13556581,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":284344320,"data_stored":284308190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:59:24.874 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:59:24.966 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 52 20 73798 73778 5.54163 2.27734 0.0300177 0.0140874 2024-09-10T15:59:25.427 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T15:59:25.428 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T15:59:25.455 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":100,"stamp":"2024-09-10T15:59:23.547102+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":290702065,"num_objects":70983,"num_object_clones":0,"num_object_copies":141966,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":47117,"num_objects_misplaced":4202,"num_objects_unfound":0,"num_objects_dirty":70983,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":28829,"num_write_kb":115310,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":71,"num_bytes_recovered":282643,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":71056,"ondisk_log_size":71056,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":901248,"kb_used_data":571596,"kb_used_omap":12,"kb_used_meta":329587,"kb_avail":282214272,"statfs":{"total":289910292480,"available":288987414528,"internally_reserved":0,"allocated":585314304,"data_stored":584425901,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":337497090},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,9,5,9,6,0,0,0,0,7],"upper_bound":4096},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19673088,"num_objects":4803,"num_object_clones":0,"num_object_copies":9606,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":47117,"num_objects_misplaced":-28227,"num_objects_unfound":0,"num_objects_dirty":4803,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4803,"num_write_kb":19212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":65,"num_bytes_recovered":266240,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.781862"},"pg_stats":[{"pgid":"2.f","version":"43'4448","reported_seq":4517,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:21.091317+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T15:59:21.091317+0000","last_peered":"2024-09-10T15:59:21.091317+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T15:59:21.091317+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":4448,"log_dups_size":0,"ondisk_log_size":4448,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18219008,"num_objects":4448,"num_object_clones":0,"num_object_copies":8896,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4448,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4448,"num_write_kb":17792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":189},{"shards":"0,2","objects":4258}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'4398","reported_seq":4505,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:19.044943+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T15:59:19.044943+0000","last_peered":"2024-09-10T15:59:19.044943+0000","last_clean":"2024-09-10T15:59:19.044943+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T15:59:19.044943+0000","last_undegraded":"2024-09-10T15:59:19.044943+0000","last_fullsized":"2024-09-10T15:59:19.044943+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":4398,"log_dups_size":0,"ondisk_log_size":4398,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17990598,"num_objects":4394,"num_object_clones":0,"num_object_copies":8788,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4394,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":2503,"num_write_kb":10012,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'4376","reported_seq":4633,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T15:59:19.182418+0000","last_change":"2024-09-10T15:59:16.142612+0000","last_active":"2024-09-10T15:59:19.182418+0000","last_peered":"2024-09-10T15:59:19.182418+0000","last_clean":"2024-09-10T15:59:14.092473+0000","last_became_active":"2024-09-10T15:59:16.109611+0000","last_became_peered":"2024-09-10T15:59:16.109611+0000","last_unstale":"2024-09-10T15:59:19.182418+0000","last_undegraded":"2024-09-10T15:59:19.182418+0000","last_fullsized":"2024-09-10T15:59:15.770264+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":4376,"log_dups_size":0,"ondisk_log_size":4376,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17907712,"num_objects":4372,"num_object_clones":0,"num_object_copies":8744,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":4202,"num_objects_unfound":0,"num_objects_dirty":4372,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2380,"num_write_kb":9520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":270336,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4202},{"shards":"2","objects":170}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'4336","reported_seq":4411,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:19.206436+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T15:59:19.206436+0000","last_peered":"2024-09-10T15:59:19.206436+0000","last_clean":"2024-09-10T15:59:19.206436+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T15:59:19.206436+0000","last_undegraded":"2024-09-10T15:59:19.206436+0000","last_fullsized":"2024-09-10T15:59:19.206436+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":4336,"log_dups_size":0,"ondisk_log_size":4336,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17760256,"num_objects":4336,"num_object_clones":0,"num_object_copies":8672,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4336,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4336,"num_write_kb":17344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'4469","reported_seq":4593,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:21.101100+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T15:59:21.101100+0000","last_peered":"2024-09-10T15:59:21.101100+0000","last_clean":"2024-09-10T15:59:21.101100+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T15:59:21.101100+0000","last_undegraded":"2024-09-10T15:59:21.101100+0000","last_fullsized":"2024-09-10T15:59:21.101100+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":4469,"log_dups_size":0,"ondisk_log_size":4469,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18276835,"num_objects":4463,"num_object_clones":0,"num_object_copies":8926,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4463,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1180,"num_write_kb":4720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'4552","reported_seq":4684,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:21.134759+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T15:59:21.134759+0000","last_peered":"2024-09-10T15:59:21.134759+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T15:59:21.134759+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":4552,"log_dups_size":0,"ondisk_log_size":4552,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18624995,"num_objects":4548,"num_object_clones":0,"num_object_copies":9096,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4548,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1164,"num_write_kb":4656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":177},{"shards":"0,2","objects":4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'4412","reported_seq":4542,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:21.149654+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T15:59:21.149654+0000","last_peered":"2024-09-10T15:59:21.149654+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T15:59:21.149654+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":4412,"log_dups_size":0,"ondisk_log_size":4412,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18047942,"num_objects":4408,"num_object_clones":0,"num_object_copies":8816,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4408,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":911,"num_write_kb":3644,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":171},{"shards":"0,2","objects":4237}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'4529","reported_seq":4653,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:21.149463+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T15:59:21.149463+0000","last_peered":"2024-09-10T15:59:21.149463+0000","last_clean":"2024-09-10T15:59:21.149463+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T15:59:21.149463+0000","last_undegraded":"2024-09-10T15:59:21.149463+0000","last_fullsized":"2024-09-10T15:59:21.149463+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":4529,"log_dups_size":0,"ondisk_log_size":4529,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18542592,"num_objects":4527,"num_object_clones":0,"num_object_copies":9054,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4527,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1376,"num_write_kb":5504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.775865+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T15:59:15.775865+0000","last_peered":"2024-09-10T15:59:15.775865+0000","last_clean":"2024-09-10T15:59:15.775865+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T15:59:15.775865+0000","last_undegraded":"2024-09-10T15:59:15.775865+0000","last_fullsized":"2024-09-10T15:59:15.775865+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'4249","reported_seq":4383,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:19.136107+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T15:59:19.136107+0000","last_peered":"2024-09-10T15:59:19.136107+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T15:59:19.136107+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":4249,"log_dups_size":0,"ondisk_log_size":4249,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17375715,"num_objects":4243,"num_object_clones":0,"num_object_copies":8486,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4243,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2068,"num_write_kb":8272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4123},{"shards":"2","objects":120}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":88,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.772572+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T15:59:15.772572+0000","last_peered":"2024-09-10T15:59:15.772572+0000","last_clean":"2024-09-10T15:59:15.772572+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T15:59:15.772572+0000","last_undegraded":"2024-09-10T15:59:15.772572+0000","last_fullsized":"2024-09-10T15:59:15.772572+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'4439","reported_seq":4560,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:21.156792+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T15:59:21.156792+0000","last_peered":"2024-09-10T15:59:21.156792+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T15:59:21.156792+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":4439,"log_dups_size":0,"ondisk_log_size":4439,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18149376,"num_objects":4431,"num_object_clones":0,"num_object_copies":8862,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4431,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":943,"num_write_kb":3772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":176},{"shards":"0,2","objects":4255}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":98,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.880567+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T15:59:15.880567+0000","last_peered":"2024-09-10T15:59:15.880567+0000","last_clean":"2024-09-10T15:59:15.880567+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T15:59:15.880567+0000","last_undegraded":"2024-09-10T15:59:15.880567+0000","last_fullsized":"2024-09-10T15:59:15.880567+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'4413","reported_seq":4541,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:19.051655+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T15:59:19.051655+0000","last_peered":"2024-09-10T15:59:19.051655+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T15:59:19.051655+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":4413,"log_dups_size":0,"ondisk_log_size":4413,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18055651,"num_objects":4409,"num_object_clones":0,"num_object_copies":8818,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4409,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2093,"num_write_kb":8372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4304},{"shards":"2","objects":105}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.837509+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T15:59:15.837509+0000","last_peered":"2024-09-10T15:59:15.837509+0000","last_clean":"2024-09-10T15:59:15.837509+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T15:59:15.837509+0000","last_undegraded":"2024-09-10T15:59:15.837509+0000","last_fullsized":"2024-09-10T15:59:15.837509+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'4464","reported_seq":4587,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:19.137413+0000","last_change":"2024-09-10T15:59:16.174113+0000","last_active":"2024-09-10T15:59:19.137413+0000","last_peered":"2024-09-10T15:59:19.137413+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T15:59:19.137413+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":4464,"log_dups_size":0,"ondisk_log_size":4464,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18259968,"num_objects":4458,"num_object_clones":0,"num_object_copies":8916,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4354,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4458,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2018,"num_write_kb":8072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4352},{"shards":"2","objects":106}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'4498","reported_seq":4627,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:21.127244+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T15:59:21.127244+0000","last_peered":"2024-09-10T15:59:21.127244+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T15:59:21.127244+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":4498,"log_dups_size":0,"ondisk_log_size":4498,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18415616,"num_objects":4496,"num_object_clones":0,"num_object_copies":8992,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4496,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1129,"num_write_kb":4516,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":192},{"shards":"0,2","objects":4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.989413+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T15:59:15.989413+0000","last_peered":"2024-09-10T15:59:15.989413+0000","last_clean":"2024-09-10T15:59:15.989413+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T15:59:15.989413+0000","last_undegraded":"2024-09-10T15:59:15.989413+0000","last_fullsized":"2024-09-10T15:59:15.989413+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'4398","reported_seq":4535,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:21.130937+0000","last_change":"2024-09-10T15:59:16.251610+0000","last_active":"2024-09-10T15:59:21.130937+0000","last_peered":"2024-09-10T15:59:21.130937+0000","last_clean":"2024-09-10T15:59:14.115595+0000","last_became_active":"2024-09-10T15:59:16.244089+0000","last_became_peered":"2024-09-10T15:59:16.244089+0000","last_unstale":"2024-09-10T15:59:21.130937+0000","last_undegraded":"2024-09-10T15:59:15.847127+0000","last_fullsized":"2024-09-10T15:59:15.774437+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":4398,"log_dups_size":0,"ondisk_log_size":4398,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17981440,"num_objects":4390,"num_object_clones":0,"num_object_copies":8780,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4390,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":788,"num_write_kb":3152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":187},{"shards":"0,2","objects":4203}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":88,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.773583+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T15:59:15.773583+0000","last_peered":"2024-09-10T15:59:15.773583+0000","last_clean":"2024-09-10T15:59:15.773583+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T15:59:15.773583+0000","last_undegraded":"2024-09-10T15:59:15.773583+0000","last_fullsized":"2024-09-10T15:59:15.773583+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'4558","reported_seq":4690,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:21.155161+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T15:59:21.155161+0000","last_peered":"2024-09-10T15:59:21.155161+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T15:59:21.155161+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":4558,"log_dups_size":0,"ondisk_log_size":4558,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18653184,"num_objects":4554,"num_object_clones":0,"num_object_copies":9108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4554,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":813,"num_write_kb":3252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":183},{"shards":"0,2","objects":4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.916017+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T15:59:15.916017+0000","last_peered":"2024-09-10T15:59:15.916017+0000","last_clean":"2024-09-10T15:59:15.916017+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T15:59:15.916017+0000","last_undegraded":"2024-09-10T15:59:15.916017+0000","last_fullsized":"2024-09-10T15:59:15.916017+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.776033+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T15:59:15.776033+0000","last_peered":"2024-09-10T15:59:15.776033+0000","last_clean":"2024-09-10T15:59:15.776033+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T15:59:15.776033+0000","last_undegraded":"2024-09-10T15:59:15.776033+0000","last_fullsized":"2024-09-10T15:59:15.776033+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'4514","reported_seq":4653,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:21.133435+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T15:59:21.133435+0000","last_peered":"2024-09-10T15:59:21.133435+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T15:59:21.133435+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":4514,"log_dups_size":0,"ondisk_log_size":4514,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18441158,"num_objects":4504,"num_object_clones":0,"num_object_copies":9008,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4504,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":677,"num_write_kb":2708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":181},{"shards":"0,2","objects":4323}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":290702046,"num_objects":70981,"num_object_clones":0,"num_object_copies":141962,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":47117,"num_objects_misplaced":4202,"num_objects_unfound":0,"num_objects_dirty":70981,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":28827,"num_write_kb":115308,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":282624,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":587444224,"data_stored":587288865,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":71053,"ondisk_log_size":71053,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986192,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":416964,"kb_used_data":276976,"kb_used_omap":4,"kb_used_meta":139963,"kb_avail":93954876,"statfs":{"total":96636764160,"available":96209793024,"internally_reserved":0,"allocated":283623424,"data_stored":283345664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":143322794},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,7],"upper_bound":4096},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659139,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":65952,"kb_used_data":12048,"kb_used_omap":4,"kb_used_meta":53883,"kb_avail":94305888,"statfs":{"total":96636764160,"available":96569229312,"internally_reserved":0,"allocated":12337152,"data_stored":12008346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":55176878},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,4,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182421,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":418332,"kb_used_data":282572,"kb_used_omap":4,"kb_used_meta":135739,"kb_avail":93953508,"statfs":{"total":96636764160,"available":96208392192,"internally_reserved":0,"allocated":289353728,"data_stored":289071891,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":138997418},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,5,1,6,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":289460224,"data_stored":289424094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":13639680,"data_stored":13556581,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":284344320,"data_stored":284308190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:59:25.456 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-10T15:59:25.456 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "43'4448", 'reported_seq': 4517, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:59:21.091317+0000', 'last_change': '2024-09-10T15:59:16.281558+0000', 'last_active': '2024-09-10T15:59:21.091317+0000', 'last_peered': '2024-09-10T15:59:21.091317+0000', 'last_clean': '2024-09-10T15:59:14.132456+0000', 'last_became_active': '2024-09-10T15:59:16.274407+0000', 'last_became_peered': '2024-09-10T15:59:16.274407+0000', 'last_unstale': '2024-09-10T15:59:21.091317+0000', 'last_undegraded': '2024-09-10T15:59:15.851279+0000', 'last_fullsized': '2024-09-10T15:59:15.774356+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'objects_scrubbed': 0, 'log_size': 4448, 'log_dups_size': 0, 'ondisk_log_size': 4448, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18219008, 'num_objects': 4448, 'num_object_clones': 0, 'num_object_copies': 8896, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4259, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4448, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4448, 'num_write_kb': 17792, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 189}, {'shards': '0,2', 'objects': 4258}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:59:25.456 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-10T15:59:25.456 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "43'4376", 'reported_seq': 4633, 'reported_epoch': 43, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-10T15:59:19.182418+0000', 'last_change': '2024-09-10T15:59:16.142612+0000', 'last_active': '2024-09-10T15:59:19.182418+0000', 'last_peered': '2024-09-10T15:59:19.182418+0000', 'last_clean': '2024-09-10T15:59:14.092473+0000', 'last_became_active': '2024-09-10T15:59:16.109611+0000', 'last_became_peered': '2024-09-10T15:59:16.109611+0000', 'last_unstale': '2024-09-10T15:59:19.182418+0000', 'last_undegraded': '2024-09-10T15:59:19.182418+0000', 'last_fullsized': '2024-09-10T15:59:15.770264+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2029", 'last_scrub_stamp': '2024-09-10T15:58:45.933789+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:45.933789+0000', 'objects_scrubbed': 1992, 'log_size': 4376, 'log_dups_size': 0, 'ondisk_log_size': 4376, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 668, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17907712, 'num_objects': 4372, 'num_object_clones': 0, 'num_object_copies': 8744, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 4202, 'num_objects_unfound': 0, 'num_objects_dirty': 4372, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2380, 'num_write_kb': 9520, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 66, 'num_bytes_recovered': 270336, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4202}, {'shards': '2', 'objects': 170}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:59:25.457 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-10T15:59:25.457 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "43'4552", 'reported_seq': 4684, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:59:21.134759+0000', 'last_change': '2024-09-10T15:59:16.308264+0000', 'last_active': '2024-09-10T15:59:21.134759+0000', 'last_peered': '2024-09-10T15:59:21.134759+0000', 'last_clean': '2024-09-10T15:59:14.111551+0000', 'last_became_active': '2024-09-10T15:59:16.300040+0000', 'last_became_peered': '2024-09-10T15:59:16.300040+0000', 'last_unstale': '2024-09-10T15:59:21.134759+0000', 'last_undegraded': '2024-09-10T15:59:15.878234+0000', 'last_fullsized': '2024-09-10T15:59:15.769328+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3442", 'last_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'objects_scrubbed': 3384, 'log_size': 4552, 'log_dups_size': 0, 'ondisk_log_size': 4552, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1292, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18624995, 'num_objects': 4548, 'num_object_clones': 0, 'num_object_copies': 9096, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4372, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4548, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 1164, 'num_write_kb': 4656, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 177}, {'shards': '0,2', 'objects': 4371}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:59:25.457 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-10T15:59:25.457 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "43'4412", 'reported_seq': 4542, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:59:21.149654+0000', 'last_change': '2024-09-10T15:59:16.331899+0000', 'last_active': '2024-09-10T15:59:21.149654+0000', 'last_peered': '2024-09-10T15:59:21.149654+0000', 'last_clean': '2024-09-10T15:59:14.139436+0000', 'last_became_active': '2024-09-10T15:59:16.318715+0000', 'last_became_peered': '2024-09-10T15:59:16.318715+0000', 'last_unstale': '2024-09-10T15:59:21.149654+0000', 'last_undegraded': '2024-09-10T15:59:15.877524+0000', 'last_fullsized': '2024-09-10T15:59:15.769042+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3537", 'last_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'objects_scrubbed': 3497, 'log_size': 4412, 'log_dups_size': 0, 'ondisk_log_size': 4412, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1224, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18047942, 'num_objects': 4408, 'num_object_clones': 0, 'num_object_copies': 8816, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4237, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4408, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 911, 'num_write_kb': 3644, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 171}, {'shards': '0,2', 'objects': 4237}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:59:25.457 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-10T15:59:25.457 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "43'4249", 'reported_seq': 4383, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:59:19.136107+0000', 'last_change': '2024-09-10T15:59:16.175680+0000', 'last_active': '2024-09-10T15:59:19.136107+0000', 'last_peered': '2024-09-10T15:59:19.136107+0000', 'last_clean': '2024-09-10T15:59:14.088187+0000', 'last_became_active': '2024-09-10T15:59:16.165810+0000', 'last_became_peered': '2024-09-10T15:59:16.165810+0000', 'last_unstale': '2024-09-10T15:59:19.136107+0000', 'last_undegraded': '2024-09-10T15:59:15.822098+0000', 'last_fullsized': '2024-09-10T15:59:15.769123+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2218", 'last_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'objects_scrubbed': 2175, 'log_size': 4249, 'log_dups_size': 0, 'ondisk_log_size': 4249, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 825, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17375715, 'num_objects': 4243, 'num_object_clones': 0, 'num_object_copies': 8486, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4124, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4243, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 2068, 'num_write_kb': 8272, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4123}, {'shards': '2', 'objects': 120}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:59:25.457 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-10T15:59:25.458 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "43'4439", 'reported_seq': 4560, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:59:21.156792+0000', 'last_change': '2024-09-10T15:59:16.277742+0000', 'last_active': '2024-09-10T15:59:21.156792+0000', 'last_peered': '2024-09-10T15:59:21.156792+0000', 'last_clean': '2024-09-10T15:59:14.129081+0000', 'last_became_active': '2024-09-10T15:59:16.269835+0000', 'last_became_peered': '2024-09-10T15:59:16.269835+0000', 'last_unstale': '2024-09-10T15:59:21.156792+0000', 'last_undegraded': '2024-09-10T15:59:15.860857+0000', 'last_fullsized': '2024-09-10T15:59:15.771410+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3557", 'last_scrub_stamp': '2024-09-10T15:59:02.493678+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:02.493678+0000', 'objects_scrubbed': 3488, 'log_size': 4439, 'log_dups_size': 0, 'ondisk_log_size': 4439, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1472, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18149376, 'num_objects': 4431, 'num_object_clones': 0, 'num_object_copies': 8862, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4257, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4431, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 943, 'num_write_kb': 3772, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 176}, {'shards': '0,2', 'objects': 4255}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:59:25.458 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-10T15:59:25.458 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "43'4413", 'reported_seq': 4541, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:59:19.051655+0000', 'last_change': '2024-09-10T15:59:16.284311+0000', 'last_active': '2024-09-10T15:59:19.051655+0000', 'last_peered': '2024-09-10T15:59:19.051655+0000', 'last_clean': '2024-09-10T15:59:14.116041+0000', 'last_became_active': '2024-09-10T15:59:16.274543+0000', 'last_became_peered': '2024-09-10T15:59:16.274543+0000', 'last_unstale': '2024-09-10T15:59:19.051655+0000', 'last_undegraded': '2024-09-10T15:59:15.812484+0000', 'last_fullsized': '2024-09-10T15:59:15.770885+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2356", 'last_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'objects_scrubbed': 2316, 'log_size': 4413, 'log_dups_size': 0, 'ondisk_log_size': 4413, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 783, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18055651, 'num_objects': 4409, 'num_object_clones': 0, 'num_object_copies': 8818, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4409, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 2093, 'num_write_kb': 8372, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4304}, {'shards': '2', 'objects': 105}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:59:25.458 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-10T15:59:25.458 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "43'4464", 'reported_seq': 4587, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:59:19.137413+0000', 'last_change': '2024-09-10T15:59:16.174113+0000', 'last_active': '2024-09-10T15:59:19.137413+0000', 'last_peered': '2024-09-10T15:59:19.137413+0000', 'last_clean': '2024-09-10T15:59:14.110654+0000', 'last_became_active': '2024-09-10T15:59:16.164645+0000', 'last_became_peered': '2024-09-10T15:59:16.164645+0000', 'last_unstale': '2024-09-10T15:59:19.137413+0000', 'last_undegraded': '2024-09-10T15:59:15.803593+0000', 'last_fullsized': '2024-09-10T15:59:15.769083+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2486", 'last_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'objects_scrubbed': 2440, 'log_size': 4464, 'log_dups_size': 0, 'ondisk_log_size': 4464, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 792, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18259968, 'num_objects': 4458, 'num_object_clones': 0, 'num_object_copies': 8916, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4354, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4458, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2018, 'num_write_kb': 8072, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4352}, {'shards': '2', 'objects': 106}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:59:25.458 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-10T15:59:25.458 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "43'4498", 'reported_seq': 4627, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:59:21.127244+0000', 'last_change': '2024-09-10T15:59:16.315641+0000', 'last_active': '2024-09-10T15:59:21.127244+0000', 'last_peered': '2024-09-10T15:59:21.127244+0000', 'last_clean': '2024-09-10T15:59:14.117001+0000', 'last_became_active': '2024-09-10T15:59:16.302245+0000', 'last_became_peered': '2024-09-10T15:59:16.302245+0000', 'last_unstale': '2024-09-10T15:59:21.127244+0000', 'last_undegraded': '2024-09-10T15:59:15.845012+0000', 'last_fullsized': '2024-09-10T15:59:15.770337+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3404", 'last_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'objects_scrubbed': 3367, 'log_size': 4498, 'log_dups_size': 0, 'ondisk_log_size': 4498, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 983, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18415616, 'num_objects': 4496, 'num_object_clones': 0, 'num_object_copies': 8992, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4496, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1129, 'num_write_kb': 4516, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 192}, {'shards': '0,2', 'objects': 4304}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:59:25.458 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-10T15:59:25.458 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "43'4398", 'reported_seq': 4535, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:59:21.130937+0000', 'last_change': '2024-09-10T15:59:16.251610+0000', 'last_active': '2024-09-10T15:59:21.130937+0000', 'last_peered': '2024-09-10T15:59:21.130937+0000', 'last_clean': '2024-09-10T15:59:14.115595+0000', 'last_became_active': '2024-09-10T15:59:16.244089+0000', 'last_became_peered': '2024-09-10T15:59:16.244089+0000', 'last_unstale': '2024-09-10T15:59:21.130937+0000', 'last_undegraded': '2024-09-10T15:59:15.847127+0000', 'last_fullsized': '2024-09-10T15:59:15.774437+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3653", 'last_scrub_stamp': '2024-09-10T15:59:05.298468+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:05.298468+0000', 'objects_scrubbed': 3602, 'log_size': 4398, 'log_dups_size': 0, 'ondisk_log_size': 4398, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1289, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17981440, 'num_objects': 4390, 'num_object_clones': 0, 'num_object_copies': 8780, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4206, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4390, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 788, 'num_write_kb': 3152, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 187}, {'shards': '0,2', 'objects': 4203}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:59:25.459 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-10T15:59:25.459 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "43'4558", 'reported_seq': 4690, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:59:21.155161+0000', 'last_change': '2024-09-10T15:59:16.311291+0000', 'last_active': '2024-09-10T15:59:21.155161+0000', 'last_peered': '2024-09-10T15:59:21.155161+0000', 'last_clean': '2024-09-10T15:59:14.116363+0000', 'last_became_active': '2024-09-10T15:59:16.301244+0000', 'last_became_peered': '2024-09-10T15:59:16.301244+0000', 'last_unstale': '2024-09-10T15:59:21.155161+0000', 'last_undegraded': '2024-09-10T15:59:15.883434+0000', 'last_fullsized': '2024-09-10T15:59:15.772756+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3789", 'last_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'objects_scrubbed': 3741, 'log_size': 4558, 'log_dups_size': 0, 'ondisk_log_size': 4558, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1440, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18653184, 'num_objects': 4554, 'num_object_clones': 0, 'num_object_copies': 9108, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4373, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4554, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 813, 'num_write_kb': 3252, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 183}, {'shards': '0,2', 'objects': 4371}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:59:25.459 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-10T15:59:25.459 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "43'4514", 'reported_seq': 4653, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:59:21.133435+0000', 'last_change': '2024-09-10T15:59:16.313409+0000', 'last_active': '2024-09-10T15:59:21.133435+0000', 'last_peered': '2024-09-10T15:59:21.133435+0000', 'last_clean': '2024-09-10T15:59:14.129780+0000', 'last_became_active': '2024-09-10T15:59:16.300932+0000', 'last_became_peered': '2024-09-10T15:59:16.300932+0000', 'last_unstale': '2024-09-10T15:59:21.133435+0000', 'last_undegraded': '2024-09-10T15:59:15.887823+0000', 'last_fullsized': '2024-09-10T15:59:15.768513+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3880", 'last_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'objects_scrubbed': 3827, 'log_size': 4514, 'log_dups_size': 0, 'ondisk_log_size': 4514, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1397, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18441158, 'num_objects': 4504, 'num_object_clones': 0, 'num_object_copies': 9008, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4325, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4504, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 677, 'num_write_kb': 2708, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 181}, {'shards': '0,2', 'objects': 4323}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:59:25.459 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T15:59:25.966 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 53 20 74369 74349 5.47916 2.23047 0.0403902 0.014249 2024-09-10T15:59:26.140 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T15:59:26.140 INFO:teuthology.orchestra.run.smithi115.stdout:{"fsid":"7effee29-df05-4351-9741-ee203564ed81","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 47117/144306 objects degraded (32.651%), 11 pgs degraded","count":11},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":130,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":43,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983954,"num_in_osds":3,"osd_in_since":1725983947,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":72153,"data_bytes":295494385,"bytes_used":933249024,"bytes_avail":288977043456,"bytes_total":289910292480,"degraded_objects":47117,"degraded_total":144306,"degraded_ratio":0.32650756032320211,"misplaced_objects":4114,"misplaced_total":144306,"misplaced_ratio":0.02850886311033498,"recovering_objects_per_sec":15,"recovering_bytes_per_sec":62655,"recovering_keys_per_sec":0,"num_objects_recovered":153,"num_bytes_recovered":626688,"num_keys_recovered":0,"write_bytes_sec":2446014,"read_op_per_sec":0,"write_op_per_sec":597},"fsmap":{"epoch":1,"btime":"2024-09-10T15:57:14:967120+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":5,"modified":"2024-09-10T15:59:19.546323+0000","services":{"osd":{"daemons":{"summary":"","1":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-10T15:59:26.169 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-10T15:59:26.170 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:59:26.748 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T15:59:26.748 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T15:59:26.775 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":101,"stamp":"2024-09-10T15:59:25.547556+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":295494385,"num_objects":72153,"num_object_clones":0,"num_object_copies":144306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":47117,"num_objects_misplaced":4114,"num_objects_unfound":0,"num_objects_dirty":72153,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":29999,"num_write_kb":119990,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":159,"num_bytes_recovered":643091,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72226,"ondisk_log_size":72226,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":911376,"kb_used_data":577568,"kb_used_omap":12,"kb_used_meta":333747,"kb_avail":282204144,"statfs":{"total":289910292480,"available":288977043456,"internally_reserved":0,"allocated":591429632,"data_stored":590541229,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":341756930},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,9,7,11,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":24465408,"num_objects":5973,"num_object_clones":0,"num_object_copies":11946,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":47117,"num_objects_misplaced":-28315,"num_objects_unfound":0,"num_objects_dirty":5973,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5973,"num_write_kb":23892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":153,"num_bytes_recovered":626688,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.002150"},"pg_stats":[{"pgid":"2.f","version":"43'4448","reported_seq":4517,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:21.091317+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T15:59:21.091317+0000","last_peered":"2024-09-10T15:59:21.091317+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T15:59:21.091317+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":4448,"log_dups_size":0,"ondisk_log_size":4448,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18219008,"num_objects":4448,"num_object_clones":0,"num_object_copies":8896,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4448,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4448,"num_write_kb":17792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":189},{"shards":"0,2","objects":4258}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'4580","reported_seq":4687,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:24.174286+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T15:59:24.174286+0000","last_peered":"2024-09-10T15:59:24.174286+0000","last_clean":"2024-09-10T15:59:24.174286+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T15:59:24.174286+0000","last_undegraded":"2024-09-10T15:59:24.174286+0000","last_fullsized":"2024-09-10T15:59:24.174286+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":4580,"log_dups_size":0,"ondisk_log_size":4580,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18736070,"num_objects":4576,"num_object_clones":0,"num_object_copies":9152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4576,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":2685,"num_write_kb":10740,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'4586","reported_seq":5019,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T15:59:24.158595+0000","last_change":"2024-09-10T15:59:16.142612+0000","last_active":"2024-09-10T15:59:24.158595+0000","last_peered":"2024-09-10T15:59:24.158595+0000","last_clean":"2024-09-10T15:59:14.092473+0000","last_became_active":"2024-09-10T15:59:16.109611+0000","last_became_peered":"2024-09-10T15:59:16.109611+0000","last_unstale":"2024-09-10T15:59:24.158595+0000","last_undegraded":"2024-09-10T15:59:24.158595+0000","last_fullsized":"2024-09-10T15:59:15.770264+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":4586,"log_dups_size":0,"ondisk_log_size":4586,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18767872,"num_objects":4582,"num_object_clones":0,"num_object_copies":9164,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":4114,"num_objects_unfound":0,"num_objects_dirty":4582,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2590,"num_write_kb":10360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":154,"num_bytes_recovered":630784,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4114},{"shards":"2","objects":468}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'4516","reported_seq":4590,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:24.207176+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T15:59:24.207176+0000","last_peered":"2024-09-10T15:59:24.207176+0000","last_clean":"2024-09-10T15:59:24.207176+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T15:59:24.207176+0000","last_undegraded":"2024-09-10T15:59:24.207176+0000","last_fullsized":"2024-09-10T15:59:24.207176+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":4516,"log_dups_size":0,"ondisk_log_size":4516,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18497536,"num_objects":4516,"num_object_clones":0,"num_object_copies":9032,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4516,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4516,"num_write_kb":18064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'4469","reported_seq":4593,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:21.101100+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T15:59:21.101100+0000","last_peered":"2024-09-10T15:59:21.101100+0000","last_clean":"2024-09-10T15:59:21.101100+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T15:59:21.101100+0000","last_undegraded":"2024-09-10T15:59:21.101100+0000","last_fullsized":"2024-09-10T15:59:21.101100+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":4469,"log_dups_size":0,"ondisk_log_size":4469,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18276835,"num_objects":4463,"num_object_clones":0,"num_object_copies":8926,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4463,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1180,"num_write_kb":4720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'4552","reported_seq":4684,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:21.134759+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T15:59:21.134759+0000","last_peered":"2024-09-10T15:59:21.134759+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T15:59:21.134759+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":4552,"log_dups_size":0,"ondisk_log_size":4552,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18624995,"num_objects":4548,"num_object_clones":0,"num_object_copies":9096,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4548,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1164,"num_write_kb":4656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":177},{"shards":"0,2","objects":4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'4412","reported_seq":4542,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:21.149654+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T15:59:21.149654+0000","last_peered":"2024-09-10T15:59:21.149654+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T15:59:21.149654+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":4412,"log_dups_size":0,"ondisk_log_size":4412,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18047942,"num_objects":4408,"num_object_clones":0,"num_object_copies":8816,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4408,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":911,"num_write_kb":3644,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":171},{"shards":"0,2","objects":4237}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'4529","reported_seq":4653,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:21.149463+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T15:59:21.149463+0000","last_peered":"2024-09-10T15:59:21.149463+0000","last_clean":"2024-09-10T15:59:21.149463+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T15:59:21.149463+0000","last_undegraded":"2024-09-10T15:59:21.149463+0000","last_fullsized":"2024-09-10T15:59:21.149463+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":4529,"log_dups_size":0,"ondisk_log_size":4529,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18542592,"num_objects":4527,"num_object_clones":0,"num_object_copies":9054,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4527,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1376,"num_write_kb":5504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.775865+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T15:59:15.775865+0000","last_peered":"2024-09-10T15:59:15.775865+0000","last_clean":"2024-09-10T15:59:15.775865+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T15:59:15.775865+0000","last_undegraded":"2024-09-10T15:59:15.775865+0000","last_fullsized":"2024-09-10T15:59:15.775865+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'4429","reported_seq":4563,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:24.195263+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T15:59:24.195263+0000","last_peered":"2024-09-10T15:59:24.195263+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T15:59:24.195263+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":4429,"log_dups_size":0,"ondisk_log_size":4429,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18112995,"num_objects":4423,"num_object_clones":0,"num_object_copies":8846,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4423,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2248,"num_write_kb":8992,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4123},{"shards":"2","objects":300}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":88,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.772572+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T15:59:15.772572+0000","last_peered":"2024-09-10T15:59:15.772572+0000","last_clean":"2024-09-10T15:59:15.772572+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T15:59:15.772572+0000","last_undegraded":"2024-09-10T15:59:15.772572+0000","last_fullsized":"2024-09-10T15:59:15.772572+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'4439","reported_seq":4560,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:21.156792+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T15:59:21.156792+0000","last_peered":"2024-09-10T15:59:21.156792+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T15:59:21.156792+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":4439,"log_dups_size":0,"ondisk_log_size":4439,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18149376,"num_objects":4431,"num_object_clones":0,"num_object_copies":8862,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4431,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":943,"num_write_kb":3772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":176},{"shards":"0,2","objects":4255}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":98,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.880567+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T15:59:15.880567+0000","last_peered":"2024-09-10T15:59:15.880567+0000","last_clean":"2024-09-10T15:59:15.880567+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T15:59:15.880567+0000","last_undegraded":"2024-09-10T15:59:15.880567+0000","last_fullsized":"2024-09-10T15:59:15.880567+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'4603","reported_seq":4732,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:24.195761+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T15:59:24.195761+0000","last_peered":"2024-09-10T15:59:24.195761+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T15:59:24.195761+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":4603,"log_dups_size":0,"ondisk_log_size":4603,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18833891,"num_objects":4599,"num_object_clones":0,"num_object_copies":9198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4599,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2283,"num_write_kb":9132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4304},{"shards":"2","objects":295}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.837509+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T15:59:15.837509+0000","last_peered":"2024-09-10T15:59:15.837509+0000","last_clean":"2024-09-10T15:59:15.837509+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T15:59:15.837509+0000","last_undegraded":"2024-09-10T15:59:15.837509+0000","last_fullsized":"2024-09-10T15:59:15.837509+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'4692","reported_seq":4814,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:24.215791+0000","last_change":"2024-09-10T15:59:16.174113+0000","last_active":"2024-09-10T15:59:24.215791+0000","last_peered":"2024-09-10T15:59:24.215791+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T15:59:24.215791+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":4692,"log_dups_size":0,"ondisk_log_size":4692,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19193856,"num_objects":4686,"num_object_clones":0,"num_object_copies":9372,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4354,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4686,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2246,"num_write_kb":8984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4352},{"shards":"2","objects":334}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'4498","reported_seq":4627,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:21.127244+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T15:59:21.127244+0000","last_peered":"2024-09-10T15:59:21.127244+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T15:59:21.127244+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":4498,"log_dups_size":0,"ondisk_log_size":4498,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18415616,"num_objects":4496,"num_object_clones":0,"num_object_copies":8992,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4496,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1129,"num_write_kb":4516,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":192},{"shards":"0,2","objects":4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.989413+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T15:59:15.989413+0000","last_peered":"2024-09-10T15:59:15.989413+0000","last_clean":"2024-09-10T15:59:15.989413+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T15:59:15.989413+0000","last_undegraded":"2024-09-10T15:59:15.989413+0000","last_fullsized":"2024-09-10T15:59:15.989413+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'4398","reported_seq":4535,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:21.130937+0000","last_change":"2024-09-10T15:59:16.251610+0000","last_active":"2024-09-10T15:59:21.130937+0000","last_peered":"2024-09-10T15:59:21.130937+0000","last_clean":"2024-09-10T15:59:14.115595+0000","last_became_active":"2024-09-10T15:59:16.244089+0000","last_became_peered":"2024-09-10T15:59:16.244089+0000","last_unstale":"2024-09-10T15:59:21.130937+0000","last_undegraded":"2024-09-10T15:59:15.847127+0000","last_fullsized":"2024-09-10T15:59:15.774437+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":4398,"log_dups_size":0,"ondisk_log_size":4398,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17981440,"num_objects":4390,"num_object_clones":0,"num_object_copies":8780,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4390,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":788,"num_write_kb":3152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":187},{"shards":"0,2","objects":4203}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":88,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.773583+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T15:59:15.773583+0000","last_peered":"2024-09-10T15:59:15.773583+0000","last_clean":"2024-09-10T15:59:15.773583+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T15:59:15.773583+0000","last_undegraded":"2024-09-10T15:59:15.773583+0000","last_fullsized":"2024-09-10T15:59:15.773583+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'4558","reported_seq":4690,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:21.155161+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T15:59:21.155161+0000","last_peered":"2024-09-10T15:59:21.155161+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T15:59:21.155161+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":4558,"log_dups_size":0,"ondisk_log_size":4558,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18653184,"num_objects":4554,"num_object_clones":0,"num_object_copies":9108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4554,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":813,"num_write_kb":3252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":183},{"shards":"0,2","objects":4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.916017+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T15:59:15.916017+0000","last_peered":"2024-09-10T15:59:15.916017+0000","last_clean":"2024-09-10T15:59:15.916017+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T15:59:15.916017+0000","last_undegraded":"2024-09-10T15:59:15.916017+0000","last_fullsized":"2024-09-10T15:59:15.916017+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.776033+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T15:59:15.776033+0000","last_peered":"2024-09-10T15:59:15.776033+0000","last_clean":"2024-09-10T15:59:15.776033+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T15:59:15.776033+0000","last_undegraded":"2024-09-10T15:59:15.776033+0000","last_fullsized":"2024-09-10T15:59:15.776033+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'4514","reported_seq":4653,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:21.133435+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T15:59:21.133435+0000","last_peered":"2024-09-10T15:59:21.133435+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T15:59:21.133435+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":4514,"log_dups_size":0,"ondisk_log_size":4514,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18441158,"num_objects":4504,"num_object_clones":0,"num_object_copies":9008,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4504,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":677,"num_write_kb":2708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":181},{"shards":"0,2","objects":4323}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":295494366,"num_objects":72151,"num_object_clones":0,"num_object_copies":144302,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":47117,"num_objects_misplaced":4114,"num_objects_unfound":0,"num_objects_dirty":72151,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":29997,"num_write_kb":119988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":157,"num_bytes_recovered":643072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":593760256,"data_stored":593604897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72223,"ondisk_log_size":72223,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986193,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":427092,"kb_used_data":282948,"kb_used_omap":4,"kb_used_meta":144123,"kb_avail":93944748,"statfs":{"total":96636764160,"available":96199421952,"internally_reserved":0,"allocated":289738752,"data_stored":289460992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":147582634},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,0,2,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659139,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":65952,"kb_used_data":12048,"kb_used_omap":4,"kb_used_meta":53883,"kb_avail":94305888,"statfs":{"total":96636764160,"available":96569229312,"internally_reserved":0,"allocated":12337152,"data_stored":12008346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":55176878},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,4,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182421,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":418332,"kb_used_data":282572,"kb_used_omap":4,"kb_used_meta":135739,"kb_avail":93953508,"statfs":{"total":96636764160,"available":96208392192,"internally_reserved":0,"allocated":289353728,"data_stored":289071891,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":138997418},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,5,1,6,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":289460224,"data_stored":289424094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":13639680,"data_stored":13556581,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":290660352,"data_stored":290624222,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:59:26.966 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 54 20 74964 74944 5.42073 2.32422 0.0299326 0.0144042 2024-09-10T15:59:27.966 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 55 20 75612 75592 5.36819 2.53125 0.0168223 0.014545 2024-09-10T15:59:28.966 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 56 20 76257 76237 5.31732 2.51953 0.027813 0.0146847 2024-09-10T15:59:29.777 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:59:29.966 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 57 20 76893 76873 5.26761 2.48438 0.0333599 0.0148233 2024-09-10T15:59:30.348 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T15:59:30.351 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T15:59:30.371 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":103,"stamp":"2024-09-10T15:59:29.548468+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":307954417,"num_objects":75195,"num_object_clones":0,"num_object_copies":150390,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":47117,"num_objects_misplaced":4010,"num_objects_unfound":0,"num_objects_dirty":75195,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":33041,"num_write_kb":132158,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":1069075,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":75268,"ondisk_log_size":75268,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":976512,"kb_used_data":602448,"kb_used_omap":12,"kb_used_meta":374003,"kb_avail":282139008,"statfs":{"total":289910292480,"available":288910344192,"internally_reserved":0,"allocated":616906752,"data_stored":616018349,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":382979074},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,7,1,4,9,10,6,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":27197440,"num_objects":6640,"num_object_clones":0,"num_object_copies":13280,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":12783,"num_objects_misplaced":4010,"num_objects_unfound":0,"num_objects_dirty":6640,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6640,"num_write_kb":26560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":257,"num_bytes_recovered":1052672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002611"},"pg_stats":[{"pgid":"2.f","version":"43'4633","reported_seq":4702,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:26.166933+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T15:59:26.166933+0000","last_peered":"2024-09-10T15:59:26.166933+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T15:59:26.166933+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":4633,"log_dups_size":0,"ondisk_log_size":4633,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18976768,"num_objects":4633,"num_object_clones":0,"num_object_copies":9266,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4633,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4633,"num_write_kb":18532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":374},{"shards":"0,2","objects":4258}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'4756","reported_seq":4863,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:29.147552+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T15:59:29.147552+0000","last_peered":"2024-09-10T15:59:29.147552+0000","last_clean":"2024-09-10T15:59:29.147552+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T15:59:29.147552+0000","last_undegraded":"2024-09-10T15:59:29.147552+0000","last_fullsized":"2024-09-10T15:59:29.147552+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":4756,"log_dups_size":0,"ondisk_log_size":4756,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19456966,"num_objects":4752,"num_object_clones":0,"num_object_copies":9504,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4752,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":2861,"num_write_kb":11444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'4774","reported_seq":5415,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T15:59:29.233096+0000","last_change":"2024-09-10T15:59:16.142612+0000","last_active":"2024-09-10T15:59:29.233096+0000","last_peered":"2024-09-10T15:59:29.233096+0000","last_clean":"2024-09-10T15:59:14.092473+0000","last_became_active":"2024-09-10T15:59:16.109611+0000","last_became_peered":"2024-09-10T15:59:16.109611+0000","last_unstale":"2024-09-10T15:59:29.233096+0000","last_undegraded":"2024-09-10T15:59:29.233096+0000","last_fullsized":"2024-09-10T15:59:15.770264+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":4774,"log_dups_size":0,"ondisk_log_size":4774,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19537920,"num_objects":4770,"num_object_clones":0,"num_object_copies":9540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":4010,"num_objects_unfound":0,"num_objects_dirty":4770,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2778,"num_write_kb":11112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":258,"num_bytes_recovered":1056768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4010},{"shards":"2","objects":760}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'4758","reported_seq":4833,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:29.148732+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T15:59:29.148732+0000","last_peered":"2024-09-10T15:59:29.148732+0000","last_clean":"2024-09-10T15:59:29.148732+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T15:59:29.148732+0000","last_undegraded":"2024-09-10T15:59:29.148732+0000","last_fullsized":"2024-09-10T15:59:29.148732+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":4758,"log_dups_size":0,"ondisk_log_size":4758,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19488768,"num_objects":4758,"num_object_clones":0,"num_object_copies":9516,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4758,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4758,"num_write_kb":19032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'4667","reported_seq":4791,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:26.166511+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T15:59:26.166511+0000","last_peered":"2024-09-10T15:59:26.166511+0000","last_clean":"2024-09-10T15:59:26.166511+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T15:59:26.166511+0000","last_undegraded":"2024-09-10T15:59:26.166511+0000","last_fullsized":"2024-09-10T15:59:26.166511+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":4667,"log_dups_size":0,"ondisk_log_size":4667,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19087843,"num_objects":4661,"num_object_clones":0,"num_object_copies":9322,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4661,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1378,"num_write_kb":5512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'4742","reported_seq":4874,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:26.011375+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T15:59:26.011375+0000","last_peered":"2024-09-10T15:59:26.011375+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T15:59:26.011375+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":4742,"log_dups_size":0,"ondisk_log_size":4742,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19403235,"num_objects":4738,"num_object_clones":0,"num_object_copies":9476,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4738,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1354,"num_write_kb":5416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":367},{"shards":"0,2","objects":4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'4583","reported_seq":4715,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:26.138327+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T15:59:26.138327+0000","last_peered":"2024-09-10T15:59:26.138327+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T15:59:26.138327+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":4583,"log_dups_size":0,"ondisk_log_size":4583,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18748358,"num_objects":4579,"num_object_clones":0,"num_object_copies":9158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4579,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1082,"num_write_kb":4328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":342},{"shards":"0,2","objects":4237}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'4730","reported_seq":4854,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:26.105672+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T15:59:26.105672+0000","last_peered":"2024-09-10T15:59:26.105672+0000","last_clean":"2024-09-10T15:59:26.105672+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T15:59:26.105672+0000","last_undegraded":"2024-09-10T15:59:26.105672+0000","last_fullsized":"2024-09-10T15:59:26.105672+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":4730,"log_dups_size":0,"ondisk_log_size":4730,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19365888,"num_objects":4728,"num_object_clones":0,"num_object_copies":9456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4728,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1577,"num_write_kb":6308,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.775865+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T15:59:15.775865+0000","last_peered":"2024-09-10T15:59:15.775865+0000","last_clean":"2024-09-10T15:59:15.775865+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T15:59:15.775865+0000","last_undegraded":"2024-09-10T15:59:15.775865+0000","last_fullsized":"2024-09-10T15:59:15.775865+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'4586","reported_seq":4719,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:29.226213+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T15:59:29.226213+0000","last_peered":"2024-09-10T15:59:29.226213+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T15:59:29.226213+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":4586,"log_dups_size":0,"ondisk_log_size":4586,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18756067,"num_objects":4580,"num_object_clones":0,"num_object_copies":9160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4580,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2405,"num_write_kb":9620,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4123},{"shards":"2","objects":457}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":88,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.772572+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T15:59:15.772572+0000","last_peered":"2024-09-10T15:59:15.772572+0000","last_clean":"2024-09-10T15:59:15.772572+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T15:59:15.772572+0000","last_undegraded":"2024-09-10T15:59:15.772572+0000","last_fullsized":"2024-09-10T15:59:15.772572+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'4634","reported_seq":4759,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:26.171098+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T15:59:26.171098+0000","last_peered":"2024-09-10T15:59:26.171098+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T15:59:26.171098+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":4634,"log_dups_size":0,"ondisk_log_size":4634,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18948096,"num_objects":4626,"num_object_clones":0,"num_object_copies":9252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4626,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1138,"num_write_kb":4552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":371},{"shards":"0,2","objects":4255}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":98,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.880567+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T15:59:15.880567+0000","last_peered":"2024-09-10T15:59:15.880567+0000","last_clean":"2024-09-10T15:59:15.880567+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T15:59:15.880567+0000","last_undegraded":"2024-09-10T15:59:15.880567+0000","last_fullsized":"2024-09-10T15:59:15.880567+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'4771","reported_seq":4900,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:29.100149+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T15:59:29.100149+0000","last_peered":"2024-09-10T15:59:29.100149+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T15:59:29.100149+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":4771,"log_dups_size":0,"ondisk_log_size":4771,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19522019,"num_objects":4767,"num_object_clones":0,"num_object_copies":9534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4767,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2451,"num_write_kb":9804,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4304},{"shards":"2","objects":463}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.837509+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T15:59:15.837509+0000","last_peered":"2024-09-10T15:59:15.837509+0000","last_clean":"2024-09-10T15:59:15.837509+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T15:59:15.837509+0000","last_undegraded":"2024-09-10T15:59:15.837509+0000","last_fullsized":"2024-09-10T15:59:15.837509+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'4879","reported_seq":5001,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:29.158212+0000","last_change":"2024-09-10T15:59:16.174113+0000","last_active":"2024-09-10T15:59:29.158212+0000","last_peered":"2024-09-10T15:59:29.158212+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T15:59:29.158212+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":4879,"log_dups_size":0,"ondisk_log_size":4879,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19959808,"num_objects":4873,"num_object_clones":0,"num_object_copies":9746,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4354,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4873,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2433,"num_write_kb":9732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4352},{"shards":"2","objects":521}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'4699","reported_seq":4826,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:26.165187+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T15:59:26.165187+0000","last_peered":"2024-09-10T15:59:26.165187+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T15:59:26.165187+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":4699,"log_dups_size":0,"ondisk_log_size":4699,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19238912,"num_objects":4697,"num_object_clones":0,"num_object_copies":9394,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4697,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1330,"num_write_kb":5320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":393},{"shards":"0,2","objects":4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.989413+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T15:59:15.989413+0000","last_peered":"2024-09-10T15:59:15.989413+0000","last_clean":"2024-09-10T15:59:15.989413+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T15:59:15.989413+0000","last_undegraded":"2024-09-10T15:59:15.989413+0000","last_fullsized":"2024-09-10T15:59:15.989413+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'4571","reported_seq":4708,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:26.110000+0000","last_change":"2024-09-10T15:59:16.251610+0000","last_active":"2024-09-10T15:59:26.110000+0000","last_peered":"2024-09-10T15:59:26.110000+0000","last_clean":"2024-09-10T15:59:14.115595+0000","last_became_active":"2024-09-10T15:59:16.244089+0000","last_became_peered":"2024-09-10T15:59:16.244089+0000","last_unstale":"2024-09-10T15:59:26.110000+0000","last_undegraded":"2024-09-10T15:59:15.847127+0000","last_fullsized":"2024-09-10T15:59:15.774437+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":4571,"log_dups_size":0,"ondisk_log_size":4571,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18690048,"num_objects":4563,"num_object_clones":0,"num_object_copies":9126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4563,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":961,"num_write_kb":3844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":360},{"shards":"0,2","objects":4203}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":88,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.773583+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T15:59:15.773583+0000","last_peered":"2024-09-10T15:59:15.773583+0000","last_clean":"2024-09-10T15:59:15.773583+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T15:59:15.773583+0000","last_undegraded":"2024-09-10T15:59:15.773583+0000","last_fullsized":"2024-09-10T15:59:15.773583+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'4770","reported_seq":4906,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:26.180897+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T15:59:26.180897+0000","last_peered":"2024-09-10T15:59:26.180897+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T15:59:26.180897+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":4770,"log_dups_size":0,"ondisk_log_size":4770,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19521536,"num_objects":4766,"num_object_clones":0,"num_object_copies":9532,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4766,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1025,"num_write_kb":4100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":395},{"shards":"0,2","objects":4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.916017+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T15:59:15.916017+0000","last_peered":"2024-09-10T15:59:15.916017+0000","last_clean":"2024-09-10T15:59:15.916017+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T15:59:15.916017+0000","last_undegraded":"2024-09-10T15:59:15.916017+0000","last_fullsized":"2024-09-10T15:59:15.916017+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.776033+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T15:59:15.776033+0000","last_peered":"2024-09-10T15:59:15.776033+0000","last_clean":"2024-09-10T15:59:15.776033+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T15:59:15.776033+0000","last_undegraded":"2024-09-10T15:59:15.776033+0000","last_fullsized":"2024-09-10T15:59:15.776033+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'4712","reported_seq":4851,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:26.053551+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T15:59:26.053551+0000","last_peered":"2024-09-10T15:59:26.053551+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T15:59:26.053551+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":4712,"log_dups_size":0,"ondisk_log_size":4712,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19252166,"num_objects":4702,"num_object_clones":0,"num_object_copies":9404,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4702,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":875,"num_write_kb":3500,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":379},{"shards":"0,2","objects":4323}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":307954398,"num_objects":75193,"num_object_clones":0,"num_object_copies":150386,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":47117,"num_objects_misplaced":4010,"num_objects_unfound":0,"num_objects_dirty":75193,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":33039,"num_write_kb":132156,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":261,"num_bytes_recovered":1069056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":619167744,"data_stored":619012385,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":75265,"ondisk_log_size":75265,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986194,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":437500,"kb_used_data":289196,"kb_used_omap":4,"kb_used_meta":148283,"kb_avail":93934340,"statfs":{"total":96636764160,"available":96188764160,"internally_reserved":0,"allocated":296136704,"data_stored":295858944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":151842474},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,4,1,0,2,3,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659140,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":85060,"kb_used_data":21428,"kb_used_omap":4,"kb_used_meta":63611,"kb_avail":94286780,"statfs":{"total":96636764160,"available":96549662720,"internally_reserved":0,"allocated":21942272,"data_stored":21613466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":65138350},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,0,4,5],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182422,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":453952,"kb_used_data":291824,"kb_used_omap":4,"kb_used_meta":162107,"kb_avail":93917888,"statfs":{"total":96636764160,"available":96171917312,"internally_reserved":0,"allocated":298827776,"data_stored":298545939,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":165998250},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,0,4,3,2,5,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":299016192,"data_stored":298980062,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":23322624,"data_stored":23239525,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":296828928,"data_stored":296792798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:59:30.372 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:59:30.951 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T15:59:30.951 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T15:59:30.966 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 58 20 77457 77437 5.21478 2.20312 0.041729 0.0149746 2024-09-10T15:59:30.989 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":103,"stamp":"2024-09-10T15:59:29.548468+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":307954417,"num_objects":75195,"num_object_clones":0,"num_object_copies":150390,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":47117,"num_objects_misplaced":4010,"num_objects_unfound":0,"num_objects_dirty":75195,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":33041,"num_write_kb":132158,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":1069075,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":75268,"ondisk_log_size":75268,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":976512,"kb_used_data":602448,"kb_used_omap":12,"kb_used_meta":374003,"kb_avail":282139008,"statfs":{"total":289910292480,"available":288910344192,"internally_reserved":0,"allocated":616906752,"data_stored":616018349,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":382979074},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,7,1,4,9,10,6,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":27197440,"num_objects":6640,"num_object_clones":0,"num_object_copies":13280,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":12783,"num_objects_misplaced":4010,"num_objects_unfound":0,"num_objects_dirty":6640,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6640,"num_write_kb":26560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":257,"num_bytes_recovered":1052672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002611"},"pg_stats":[{"pgid":"2.f","version":"43'4633","reported_seq":4702,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:26.166933+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T15:59:26.166933+0000","last_peered":"2024-09-10T15:59:26.166933+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T15:59:26.166933+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":4633,"log_dups_size":0,"ondisk_log_size":4633,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18976768,"num_objects":4633,"num_object_clones":0,"num_object_copies":9266,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4633,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4633,"num_write_kb":18532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":374},{"shards":"0,2","objects":4258}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'4756","reported_seq":4863,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:29.147552+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T15:59:29.147552+0000","last_peered":"2024-09-10T15:59:29.147552+0000","last_clean":"2024-09-10T15:59:29.147552+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T15:59:29.147552+0000","last_undegraded":"2024-09-10T15:59:29.147552+0000","last_fullsized":"2024-09-10T15:59:29.147552+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":4756,"log_dups_size":0,"ondisk_log_size":4756,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19456966,"num_objects":4752,"num_object_clones":0,"num_object_copies":9504,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4752,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":2861,"num_write_kb":11444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'4774","reported_seq":5415,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T15:59:29.233096+0000","last_change":"2024-09-10T15:59:16.142612+0000","last_active":"2024-09-10T15:59:29.233096+0000","last_peered":"2024-09-10T15:59:29.233096+0000","last_clean":"2024-09-10T15:59:14.092473+0000","last_became_active":"2024-09-10T15:59:16.109611+0000","last_became_peered":"2024-09-10T15:59:16.109611+0000","last_unstale":"2024-09-10T15:59:29.233096+0000","last_undegraded":"2024-09-10T15:59:29.233096+0000","last_fullsized":"2024-09-10T15:59:15.770264+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":4774,"log_dups_size":0,"ondisk_log_size":4774,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19537920,"num_objects":4770,"num_object_clones":0,"num_object_copies":9540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":4010,"num_objects_unfound":0,"num_objects_dirty":4770,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2778,"num_write_kb":11112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":258,"num_bytes_recovered":1056768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4010},{"shards":"2","objects":760}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'4758","reported_seq":4833,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:29.148732+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T15:59:29.148732+0000","last_peered":"2024-09-10T15:59:29.148732+0000","last_clean":"2024-09-10T15:59:29.148732+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T15:59:29.148732+0000","last_undegraded":"2024-09-10T15:59:29.148732+0000","last_fullsized":"2024-09-10T15:59:29.148732+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":4758,"log_dups_size":0,"ondisk_log_size":4758,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19488768,"num_objects":4758,"num_object_clones":0,"num_object_copies":9516,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4758,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4758,"num_write_kb":19032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'4667","reported_seq":4791,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:26.166511+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T15:59:26.166511+0000","last_peered":"2024-09-10T15:59:26.166511+0000","last_clean":"2024-09-10T15:59:26.166511+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T15:59:26.166511+0000","last_undegraded":"2024-09-10T15:59:26.166511+0000","last_fullsized":"2024-09-10T15:59:26.166511+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":4667,"log_dups_size":0,"ondisk_log_size":4667,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19087843,"num_objects":4661,"num_object_clones":0,"num_object_copies":9322,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4661,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1378,"num_write_kb":5512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'4742","reported_seq":4874,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:26.011375+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T15:59:26.011375+0000","last_peered":"2024-09-10T15:59:26.011375+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T15:59:26.011375+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":4742,"log_dups_size":0,"ondisk_log_size":4742,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19403235,"num_objects":4738,"num_object_clones":0,"num_object_copies":9476,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4738,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1354,"num_write_kb":5416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":367},{"shards":"0,2","objects":4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'4583","reported_seq":4715,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:26.138327+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T15:59:26.138327+0000","last_peered":"2024-09-10T15:59:26.138327+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T15:59:26.138327+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":4583,"log_dups_size":0,"ondisk_log_size":4583,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18748358,"num_objects":4579,"num_object_clones":0,"num_object_copies":9158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4579,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1082,"num_write_kb":4328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":342},{"shards":"0,2","objects":4237}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'4730","reported_seq":4854,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:26.105672+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T15:59:26.105672+0000","last_peered":"2024-09-10T15:59:26.105672+0000","last_clean":"2024-09-10T15:59:26.105672+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T15:59:26.105672+0000","last_undegraded":"2024-09-10T15:59:26.105672+0000","last_fullsized":"2024-09-10T15:59:26.105672+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":4730,"log_dups_size":0,"ondisk_log_size":4730,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19365888,"num_objects":4728,"num_object_clones":0,"num_object_copies":9456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4728,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1577,"num_write_kb":6308,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.775865+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T15:59:15.775865+0000","last_peered":"2024-09-10T15:59:15.775865+0000","last_clean":"2024-09-10T15:59:15.775865+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T15:59:15.775865+0000","last_undegraded":"2024-09-10T15:59:15.775865+0000","last_fullsized":"2024-09-10T15:59:15.775865+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'4586","reported_seq":4719,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:29.226213+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T15:59:29.226213+0000","last_peered":"2024-09-10T15:59:29.226213+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T15:59:29.226213+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":4586,"log_dups_size":0,"ondisk_log_size":4586,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18756067,"num_objects":4580,"num_object_clones":0,"num_object_copies":9160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4580,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2405,"num_write_kb":9620,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4123},{"shards":"2","objects":457}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":88,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.772572+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T15:59:15.772572+0000","last_peered":"2024-09-10T15:59:15.772572+0000","last_clean":"2024-09-10T15:59:15.772572+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T15:59:15.772572+0000","last_undegraded":"2024-09-10T15:59:15.772572+0000","last_fullsized":"2024-09-10T15:59:15.772572+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'4634","reported_seq":4759,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:26.171098+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T15:59:26.171098+0000","last_peered":"2024-09-10T15:59:26.171098+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T15:59:26.171098+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":4634,"log_dups_size":0,"ondisk_log_size":4634,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18948096,"num_objects":4626,"num_object_clones":0,"num_object_copies":9252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4626,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1138,"num_write_kb":4552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":371},{"shards":"0,2","objects":4255}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":98,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.880567+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T15:59:15.880567+0000","last_peered":"2024-09-10T15:59:15.880567+0000","last_clean":"2024-09-10T15:59:15.880567+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T15:59:15.880567+0000","last_undegraded":"2024-09-10T15:59:15.880567+0000","last_fullsized":"2024-09-10T15:59:15.880567+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'4771","reported_seq":4900,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:29.100149+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T15:59:29.100149+0000","last_peered":"2024-09-10T15:59:29.100149+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T15:59:29.100149+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":4771,"log_dups_size":0,"ondisk_log_size":4771,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19522019,"num_objects":4767,"num_object_clones":0,"num_object_copies":9534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4767,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2451,"num_write_kb":9804,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4304},{"shards":"2","objects":463}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.837509+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T15:59:15.837509+0000","last_peered":"2024-09-10T15:59:15.837509+0000","last_clean":"2024-09-10T15:59:15.837509+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T15:59:15.837509+0000","last_undegraded":"2024-09-10T15:59:15.837509+0000","last_fullsized":"2024-09-10T15:59:15.837509+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'4879","reported_seq":5001,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:29.158212+0000","last_change":"2024-09-10T15:59:16.174113+0000","last_active":"2024-09-10T15:59:29.158212+0000","last_peered":"2024-09-10T15:59:29.158212+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T15:59:29.158212+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":4879,"log_dups_size":0,"ondisk_log_size":4879,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19959808,"num_objects":4873,"num_object_clones":0,"num_object_copies":9746,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4354,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4873,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2433,"num_write_kb":9732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4352},{"shards":"2","objects":521}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'4699","reported_seq":4826,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:26.165187+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T15:59:26.165187+0000","last_peered":"2024-09-10T15:59:26.165187+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T15:59:26.165187+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":4699,"log_dups_size":0,"ondisk_log_size":4699,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19238912,"num_objects":4697,"num_object_clones":0,"num_object_copies":9394,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4697,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1330,"num_write_kb":5320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":393},{"shards":"0,2","objects":4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.989413+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T15:59:15.989413+0000","last_peered":"2024-09-10T15:59:15.989413+0000","last_clean":"2024-09-10T15:59:15.989413+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T15:59:15.989413+0000","last_undegraded":"2024-09-10T15:59:15.989413+0000","last_fullsized":"2024-09-10T15:59:15.989413+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'4571","reported_seq":4708,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:26.110000+0000","last_change":"2024-09-10T15:59:16.251610+0000","last_active":"2024-09-10T15:59:26.110000+0000","last_peered":"2024-09-10T15:59:26.110000+0000","last_clean":"2024-09-10T15:59:14.115595+0000","last_became_active":"2024-09-10T15:59:16.244089+0000","last_became_peered":"2024-09-10T15:59:16.244089+0000","last_unstale":"2024-09-10T15:59:26.110000+0000","last_undegraded":"2024-09-10T15:59:15.847127+0000","last_fullsized":"2024-09-10T15:59:15.774437+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":4571,"log_dups_size":0,"ondisk_log_size":4571,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18690048,"num_objects":4563,"num_object_clones":0,"num_object_copies":9126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4563,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":961,"num_write_kb":3844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":360},{"shards":"0,2","objects":4203}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":88,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.773583+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T15:59:15.773583+0000","last_peered":"2024-09-10T15:59:15.773583+0000","last_clean":"2024-09-10T15:59:15.773583+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T15:59:15.773583+0000","last_undegraded":"2024-09-10T15:59:15.773583+0000","last_fullsized":"2024-09-10T15:59:15.773583+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'4770","reported_seq":4906,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:26.180897+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T15:59:26.180897+0000","last_peered":"2024-09-10T15:59:26.180897+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T15:59:26.180897+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":4770,"log_dups_size":0,"ondisk_log_size":4770,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19521536,"num_objects":4766,"num_object_clones":0,"num_object_copies":9532,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4766,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1025,"num_write_kb":4100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":395},{"shards":"0,2","objects":4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.916017+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T15:59:15.916017+0000","last_peered":"2024-09-10T15:59:15.916017+0000","last_clean":"2024-09-10T15:59:15.916017+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T15:59:15.916017+0000","last_undegraded":"2024-09-10T15:59:15.916017+0000","last_fullsized":"2024-09-10T15:59:15.916017+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.776033+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T15:59:15.776033+0000","last_peered":"2024-09-10T15:59:15.776033+0000","last_clean":"2024-09-10T15:59:15.776033+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T15:59:15.776033+0000","last_undegraded":"2024-09-10T15:59:15.776033+0000","last_fullsized":"2024-09-10T15:59:15.776033+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'4712","reported_seq":4851,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:26.053551+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T15:59:26.053551+0000","last_peered":"2024-09-10T15:59:26.053551+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T15:59:26.053551+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":4712,"log_dups_size":0,"ondisk_log_size":4712,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19252166,"num_objects":4702,"num_object_clones":0,"num_object_copies":9404,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4702,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":875,"num_write_kb":3500,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":379},{"shards":"0,2","objects":4323}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":307954398,"num_objects":75193,"num_object_clones":0,"num_object_copies":150386,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":47117,"num_objects_misplaced":4010,"num_objects_unfound":0,"num_objects_dirty":75193,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":33039,"num_write_kb":132156,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":261,"num_bytes_recovered":1069056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":619167744,"data_stored":619012385,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":75265,"ondisk_log_size":75265,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986194,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":437500,"kb_used_data":289196,"kb_used_omap":4,"kb_used_meta":148283,"kb_avail":93934340,"statfs":{"total":96636764160,"available":96188764160,"internally_reserved":0,"allocated":296136704,"data_stored":295858944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":151842474},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,4,1,0,2,3,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659140,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":85060,"kb_used_data":21428,"kb_used_omap":4,"kb_used_meta":63611,"kb_avail":94286780,"statfs":{"total":96636764160,"available":96549662720,"internally_reserved":0,"allocated":21942272,"data_stored":21613466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":65138350},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,0,4,5],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182422,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":453952,"kb_used_data":291824,"kb_used_omap":4,"kb_used_meta":162107,"kb_avail":93917888,"statfs":{"total":96636764160,"available":96171917312,"internally_reserved":0,"allocated":298827776,"data_stored":298545939,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":165998250},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,0,4,3,2,5,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":299016192,"data_stored":298980062,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":23322624,"data_stored":23239525,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":296828928,"data_stored":296792798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:59:30.991 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-10T15:59:30.991 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "43'4633", 'reported_seq': 4702, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:59:26.166933+0000', 'last_change': '2024-09-10T15:59:16.281558+0000', 'last_active': '2024-09-10T15:59:26.166933+0000', 'last_peered': '2024-09-10T15:59:26.166933+0000', 'last_clean': '2024-09-10T15:59:14.132456+0000', 'last_became_active': '2024-09-10T15:59:16.274407+0000', 'last_became_peered': '2024-09-10T15:59:16.274407+0000', 'last_unstale': '2024-09-10T15:59:26.166933+0000', 'last_undegraded': '2024-09-10T15:59:15.851279+0000', 'last_fullsized': '2024-09-10T15:59:15.774356+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'objects_scrubbed': 0, 'log_size': 4633, 'log_dups_size': 0, 'ondisk_log_size': 4633, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18976768, 'num_objects': 4633, 'num_object_clones': 0, 'num_object_copies': 9266, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4259, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4633, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4633, 'num_write_kb': 18532, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 374}, {'shards': '0,2', 'objects': 4258}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:59:30.991 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-10T15:59:30.992 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "43'4774", 'reported_seq': 5415, 'reported_epoch': 43, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-10T15:59:29.233096+0000', 'last_change': '2024-09-10T15:59:16.142612+0000', 'last_active': '2024-09-10T15:59:29.233096+0000', 'last_peered': '2024-09-10T15:59:29.233096+0000', 'last_clean': '2024-09-10T15:59:14.092473+0000', 'last_became_active': '2024-09-10T15:59:16.109611+0000', 'last_became_peered': '2024-09-10T15:59:16.109611+0000', 'last_unstale': '2024-09-10T15:59:29.233096+0000', 'last_undegraded': '2024-09-10T15:59:29.233096+0000', 'last_fullsized': '2024-09-10T15:59:15.770264+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2029", 'last_scrub_stamp': '2024-09-10T15:58:45.933789+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:45.933789+0000', 'objects_scrubbed': 1992, 'log_size': 4774, 'log_dups_size': 0, 'ondisk_log_size': 4774, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 668, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19537920, 'num_objects': 4770, 'num_object_clones': 0, 'num_object_copies': 9540, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 4010, 'num_objects_unfound': 0, 'num_objects_dirty': 4770, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2778, 'num_write_kb': 11112, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 258, 'num_bytes_recovered': 1056768, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4010}, {'shards': '2', 'objects': 760}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:59:30.992 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-10T15:59:30.992 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "43'4742", 'reported_seq': 4874, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:59:26.011375+0000', 'last_change': '2024-09-10T15:59:16.308264+0000', 'last_active': '2024-09-10T15:59:26.011375+0000', 'last_peered': '2024-09-10T15:59:26.011375+0000', 'last_clean': '2024-09-10T15:59:14.111551+0000', 'last_became_active': '2024-09-10T15:59:16.300040+0000', 'last_became_peered': '2024-09-10T15:59:16.300040+0000', 'last_unstale': '2024-09-10T15:59:26.011375+0000', 'last_undegraded': '2024-09-10T15:59:15.878234+0000', 'last_fullsized': '2024-09-10T15:59:15.769328+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3442", 'last_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'objects_scrubbed': 3384, 'log_size': 4742, 'log_dups_size': 0, 'ondisk_log_size': 4742, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1292, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19403235, 'num_objects': 4738, 'num_object_clones': 0, 'num_object_copies': 9476, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4372, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4738, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 1354, 'num_write_kb': 5416, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 367}, {'shards': '0,2', 'objects': 4371}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:59:30.992 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-10T15:59:30.992 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "43'4583", 'reported_seq': 4715, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:59:26.138327+0000', 'last_change': '2024-09-10T15:59:16.331899+0000', 'last_active': '2024-09-10T15:59:26.138327+0000', 'last_peered': '2024-09-10T15:59:26.138327+0000', 'last_clean': '2024-09-10T15:59:14.139436+0000', 'last_became_active': '2024-09-10T15:59:16.318715+0000', 'last_became_peered': '2024-09-10T15:59:16.318715+0000', 'last_unstale': '2024-09-10T15:59:26.138327+0000', 'last_undegraded': '2024-09-10T15:59:15.877524+0000', 'last_fullsized': '2024-09-10T15:59:15.769042+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3537", 'last_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'objects_scrubbed': 3497, 'log_size': 4583, 'log_dups_size': 0, 'ondisk_log_size': 4583, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1224, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18748358, 'num_objects': 4579, 'num_object_clones': 0, 'num_object_copies': 9158, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4237, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4579, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 1082, 'num_write_kb': 4328, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 342}, {'shards': '0,2', 'objects': 4237}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:59:30.992 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-10T15:59:30.992 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "43'4586", 'reported_seq': 4719, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:59:29.226213+0000', 'last_change': '2024-09-10T15:59:16.175680+0000', 'last_active': '2024-09-10T15:59:29.226213+0000', 'last_peered': '2024-09-10T15:59:29.226213+0000', 'last_clean': '2024-09-10T15:59:14.088187+0000', 'last_became_active': '2024-09-10T15:59:16.165810+0000', 'last_became_peered': '2024-09-10T15:59:16.165810+0000', 'last_unstale': '2024-09-10T15:59:29.226213+0000', 'last_undegraded': '2024-09-10T15:59:15.822098+0000', 'last_fullsized': '2024-09-10T15:59:15.769123+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2218", 'last_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'objects_scrubbed': 2175, 'log_size': 4586, 'log_dups_size': 0, 'ondisk_log_size': 4586, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 825, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18756067, 'num_objects': 4580, 'num_object_clones': 0, 'num_object_copies': 9160, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4124, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4580, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 2405, 'num_write_kb': 9620, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4123}, {'shards': '2', 'objects': 457}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:59:30.992 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-10T15:59:30.992 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "43'4634", 'reported_seq': 4759, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:59:26.171098+0000', 'last_change': '2024-09-10T15:59:16.277742+0000', 'last_active': '2024-09-10T15:59:26.171098+0000', 'last_peered': '2024-09-10T15:59:26.171098+0000', 'last_clean': '2024-09-10T15:59:14.129081+0000', 'last_became_active': '2024-09-10T15:59:16.269835+0000', 'last_became_peered': '2024-09-10T15:59:16.269835+0000', 'last_unstale': '2024-09-10T15:59:26.171098+0000', 'last_undegraded': '2024-09-10T15:59:15.860857+0000', 'last_fullsized': '2024-09-10T15:59:15.771410+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3557", 'last_scrub_stamp': '2024-09-10T15:59:02.493678+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:02.493678+0000', 'objects_scrubbed': 3488, 'log_size': 4634, 'log_dups_size': 0, 'ondisk_log_size': 4634, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1472, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18948096, 'num_objects': 4626, 'num_object_clones': 0, 'num_object_copies': 9252, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4257, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4626, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1138, 'num_write_kb': 4552, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 371}, {'shards': '0,2', 'objects': 4255}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:59:30.993 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-10T15:59:30.993 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "43'4771", 'reported_seq': 4900, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:59:29.100149+0000', 'last_change': '2024-09-10T15:59:16.284311+0000', 'last_active': '2024-09-10T15:59:29.100149+0000', 'last_peered': '2024-09-10T15:59:29.100149+0000', 'last_clean': '2024-09-10T15:59:14.116041+0000', 'last_became_active': '2024-09-10T15:59:16.274543+0000', 'last_became_peered': '2024-09-10T15:59:16.274543+0000', 'last_unstale': '2024-09-10T15:59:29.100149+0000', 'last_undegraded': '2024-09-10T15:59:15.812484+0000', 'last_fullsized': '2024-09-10T15:59:15.770885+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2356", 'last_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'objects_scrubbed': 2316, 'log_size': 4771, 'log_dups_size': 0, 'ondisk_log_size': 4771, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 783, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19522019, 'num_objects': 4767, 'num_object_clones': 0, 'num_object_copies': 9534, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4767, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 2451, 'num_write_kb': 9804, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4304}, {'shards': '2', 'objects': 463}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:59:30.993 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-10T15:59:30.993 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "43'4879", 'reported_seq': 5001, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:59:29.158212+0000', 'last_change': '2024-09-10T15:59:16.174113+0000', 'last_active': '2024-09-10T15:59:29.158212+0000', 'last_peered': '2024-09-10T15:59:29.158212+0000', 'last_clean': '2024-09-10T15:59:14.110654+0000', 'last_became_active': '2024-09-10T15:59:16.164645+0000', 'last_became_peered': '2024-09-10T15:59:16.164645+0000', 'last_unstale': '2024-09-10T15:59:29.158212+0000', 'last_undegraded': '2024-09-10T15:59:15.803593+0000', 'last_fullsized': '2024-09-10T15:59:15.769083+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2486", 'last_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'objects_scrubbed': 2440, 'log_size': 4879, 'log_dups_size': 0, 'ondisk_log_size': 4879, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 792, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19959808, 'num_objects': 4873, 'num_object_clones': 0, 'num_object_copies': 9746, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4354, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4873, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2433, 'num_write_kb': 9732, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4352}, {'shards': '2', 'objects': 521}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:59:30.993 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-10T15:59:30.993 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "43'4699", 'reported_seq': 4826, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:59:26.165187+0000', 'last_change': '2024-09-10T15:59:16.315641+0000', 'last_active': '2024-09-10T15:59:26.165187+0000', 'last_peered': '2024-09-10T15:59:26.165187+0000', 'last_clean': '2024-09-10T15:59:14.117001+0000', 'last_became_active': '2024-09-10T15:59:16.302245+0000', 'last_became_peered': '2024-09-10T15:59:16.302245+0000', 'last_unstale': '2024-09-10T15:59:26.165187+0000', 'last_undegraded': '2024-09-10T15:59:15.845012+0000', 'last_fullsized': '2024-09-10T15:59:15.770337+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3404", 'last_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'objects_scrubbed': 3367, 'log_size': 4699, 'log_dups_size': 0, 'ondisk_log_size': 4699, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 983, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19238912, 'num_objects': 4697, 'num_object_clones': 0, 'num_object_copies': 9394, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4697, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1330, 'num_write_kb': 5320, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 393}, {'shards': '0,2', 'objects': 4304}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:59:30.993 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-10T15:59:30.993 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "43'4571", 'reported_seq': 4708, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:59:26.110000+0000', 'last_change': '2024-09-10T15:59:16.251610+0000', 'last_active': '2024-09-10T15:59:26.110000+0000', 'last_peered': '2024-09-10T15:59:26.110000+0000', 'last_clean': '2024-09-10T15:59:14.115595+0000', 'last_became_active': '2024-09-10T15:59:16.244089+0000', 'last_became_peered': '2024-09-10T15:59:16.244089+0000', 'last_unstale': '2024-09-10T15:59:26.110000+0000', 'last_undegraded': '2024-09-10T15:59:15.847127+0000', 'last_fullsized': '2024-09-10T15:59:15.774437+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3653", 'last_scrub_stamp': '2024-09-10T15:59:05.298468+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:05.298468+0000', 'objects_scrubbed': 3602, 'log_size': 4571, 'log_dups_size': 0, 'ondisk_log_size': 4571, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1289, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18690048, 'num_objects': 4563, 'num_object_clones': 0, 'num_object_copies': 9126, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4206, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4563, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 961, 'num_write_kb': 3844, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 360}, {'shards': '0,2', 'objects': 4203}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:59:30.993 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-10T15:59:30.993 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "43'4770", 'reported_seq': 4906, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:59:26.180897+0000', 'last_change': '2024-09-10T15:59:16.311291+0000', 'last_active': '2024-09-10T15:59:26.180897+0000', 'last_peered': '2024-09-10T15:59:26.180897+0000', 'last_clean': '2024-09-10T15:59:14.116363+0000', 'last_became_active': '2024-09-10T15:59:16.301244+0000', 'last_became_peered': '2024-09-10T15:59:16.301244+0000', 'last_unstale': '2024-09-10T15:59:26.180897+0000', 'last_undegraded': '2024-09-10T15:59:15.883434+0000', 'last_fullsized': '2024-09-10T15:59:15.772756+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3789", 'last_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'objects_scrubbed': 3741, 'log_size': 4770, 'log_dups_size': 0, 'ondisk_log_size': 4770, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1440, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19521536, 'num_objects': 4766, 'num_object_clones': 0, 'num_object_copies': 9532, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4373, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4766, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1025, 'num_write_kb': 4100, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 395}, {'shards': '0,2', 'objects': 4371}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:59:30.994 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-10T15:59:30.994 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "43'4712", 'reported_seq': 4851, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:59:26.053551+0000', 'last_change': '2024-09-10T15:59:16.313409+0000', 'last_active': '2024-09-10T15:59:26.053551+0000', 'last_peered': '2024-09-10T15:59:26.053551+0000', 'last_clean': '2024-09-10T15:59:14.129780+0000', 'last_became_active': '2024-09-10T15:59:16.300932+0000', 'last_became_peered': '2024-09-10T15:59:16.300932+0000', 'last_unstale': '2024-09-10T15:59:26.053551+0000', 'last_undegraded': '2024-09-10T15:59:15.887823+0000', 'last_fullsized': '2024-09-10T15:59:15.768513+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3880", 'last_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'objects_scrubbed': 3827, 'log_size': 4712, 'log_dups_size': 0, 'ondisk_log_size': 4712, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1397, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19252166, 'num_objects': 4702, 'num_object_clones': 0, 'num_object_copies': 9404, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4325, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4702, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 875, 'num_write_kb': 3500, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 379}, {'shards': '0,2', 'objects': 4323}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:59:30.994 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T15:59:31.689 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T15:59:31.689 INFO:teuthology.orchestra.run.smithi115.stdout:{"fsid":"7effee29-df05-4351-9741-ee203564ed81","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 47117/150390 objects degraded (31.330%), 11 pgs degraded","count":11},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":135,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":43,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983954,"num_in_osds":3,"osd_in_since":1725983947,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":75195,"data_bytes":307954417,"bytes_used":999948288,"bytes_avail":288910344192,"bytes_total":289910292480,"degraded_objects":47117,"degraded_total":150390,"degraded_ratio":0.31329875656626105,"misplaced_objects":4010,"misplaced_total":150390,"misplaced_ratio":0.026664006915353412,"recovering_objects_per_sec":21,"recovering_bytes_per_sec":87703,"recovering_keys_per_sec":0,"num_objects_recovered":257,"num_bytes_recovered":1052672,"num_keys_recovered":0,"write_bytes_sec":2265960,"read_op_per_sec":0,"write_op_per_sec":553},"fsmap":{"epoch":1,"btime":"2024-09-10T15:57:14:967120+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":5,"modified":"2024-09-10T15:59:19.546323+0000","services":{"osd":{"daemons":{"summary":"","1":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-10T15:59:31.710 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-10T15:59:31.711 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:59:31.969 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 59 20 78018 77998 5.16353 2.19141 0.0466395 0.015123 2024-09-10T15:59:32.260 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T15:59:32.262 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T15:59:32.288 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":104,"stamp":"2024-09-10T15:59:31.548925+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":316097265,"num_objects":77183,"num_object_clones":0,"num_object_copies":154366,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":47117,"num_objects_misplaced":4010,"num_objects_unfound":0,"num_objects_dirty":77183,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":35029,"num_write_kb":140110,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":1069075,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":77256,"ondisk_log_size":77256,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":990308,"kb_used_data":612084,"kb_used_omap":12,"kb_used_meta":378163,"kb_avail":282125212,"statfs":{"total":289910292480,"available":288896217088,"internally_reserved":0,"allocated":626774016,"data_stored":625885613,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":387238914},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,6,1,3,9,17,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":32714752,"num_objects":7987,"num_object_clones":0,"num_object_copies":15974,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-192,"num_objects_unfound":0,"num_objects_dirty":7987,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7987,"num_write_kb":31948,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":192,"num_bytes_recovered":786432,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002692"},"pg_stats":[{"pgid":"2.f","version":"43'4831","reported_seq":4900,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:31.181856+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T15:59:31.181856+0000","last_peered":"2024-09-10T15:59:31.181856+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T15:59:31.181856+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":4831,"log_dups_size":0,"ondisk_log_size":4831,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19787776,"num_objects":4831,"num_object_clones":0,"num_object_copies":9662,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4831,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4831,"num_write_kb":19324,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":572},{"shards":"0,2","objects":4258}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'4756","reported_seq":4863,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:29.147552+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T15:59:29.147552+0000","last_peered":"2024-09-10T15:59:29.147552+0000","last_clean":"2024-09-10T15:59:29.147552+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T15:59:29.147552+0000","last_undegraded":"2024-09-10T15:59:29.147552+0000","last_fullsized":"2024-09-10T15:59:29.147552+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":4756,"log_dups_size":0,"ondisk_log_size":4756,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19456966,"num_objects":4752,"num_object_clones":0,"num_object_copies":9504,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4752,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":2861,"num_write_kb":11444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'4774","reported_seq":5415,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T15:59:29.233096+0000","last_change":"2024-09-10T15:59:16.142612+0000","last_active":"2024-09-10T15:59:29.233096+0000","last_peered":"2024-09-10T15:59:29.233096+0000","last_clean":"2024-09-10T15:59:14.092473+0000","last_became_active":"2024-09-10T15:59:16.109611+0000","last_became_peered":"2024-09-10T15:59:16.109611+0000","last_unstale":"2024-09-10T15:59:29.233096+0000","last_undegraded":"2024-09-10T15:59:29.233096+0000","last_fullsized":"2024-09-10T15:59:15.770264+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":4774,"log_dups_size":0,"ondisk_log_size":4774,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19537920,"num_objects":4770,"num_object_clones":0,"num_object_copies":9540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":4010,"num_objects_unfound":0,"num_objects_dirty":4770,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2778,"num_write_kb":11112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":258,"num_bytes_recovered":1056768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4010},{"shards":"2","objects":760}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'4758","reported_seq":4833,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:29.148732+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T15:59:29.148732+0000","last_peered":"2024-09-10T15:59:29.148732+0000","last_clean":"2024-09-10T15:59:29.148732+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T15:59:29.148732+0000","last_undegraded":"2024-09-10T15:59:29.148732+0000","last_fullsized":"2024-09-10T15:59:29.148732+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":4758,"log_dups_size":0,"ondisk_log_size":4758,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19488768,"num_objects":4758,"num_object_clones":0,"num_object_copies":9516,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4758,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4758,"num_write_kb":19032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'4868","reported_seq":4992,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:31.118196+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T15:59:31.118196+0000","last_peered":"2024-09-10T15:59:31.118196+0000","last_clean":"2024-09-10T15:59:31.118196+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T15:59:31.118196+0000","last_undegraded":"2024-09-10T15:59:31.118196+0000","last_fullsized":"2024-09-10T15:59:31.118196+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":4868,"log_dups_size":0,"ondisk_log_size":4868,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19911139,"num_objects":4862,"num_object_clones":0,"num_object_copies":9724,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4862,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1579,"num_write_kb":6316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'4942","reported_seq":5069,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:31.174037+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T15:59:31.174037+0000","last_peered":"2024-09-10T15:59:31.174037+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T15:59:31.174037+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":4942,"log_dups_size":0,"ondisk_log_size":4942,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20222435,"num_objects":4938,"num_object_clones":0,"num_object_copies":9876,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4938,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1554,"num_write_kb":6216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":567},{"shards":"0,2","objects":4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'4770","reported_seq":4902,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:31.172003+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T15:59:31.172003+0000","last_peered":"2024-09-10T15:59:31.172003+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T15:59:31.172003+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":4770,"log_dups_size":0,"ondisk_log_size":4770,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19514310,"num_objects":4766,"num_object_clones":0,"num_object_copies":9532,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4766,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1269,"num_write_kb":5076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":529},{"shards":"0,2","objects":4237}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'4947","reported_seq":5070,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:31.195762+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T15:59:31.195762+0000","last_peered":"2024-09-10T15:59:31.195762+0000","last_clean":"2024-09-10T15:59:31.195762+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T15:59:31.195762+0000","last_undegraded":"2024-09-10T15:59:31.195762+0000","last_fullsized":"2024-09-10T15:59:31.195762+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":4947,"log_dups_size":0,"ondisk_log_size":4947,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20254720,"num_objects":4945,"num_object_clones":0,"num_object_copies":9890,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4945,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1794,"num_write_kb":7176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.775865+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T15:59:15.775865+0000","last_peered":"2024-09-10T15:59:15.775865+0000","last_clean":"2024-09-10T15:59:15.775865+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T15:59:15.775865+0000","last_undegraded":"2024-09-10T15:59:15.775865+0000","last_fullsized":"2024-09-10T15:59:15.775865+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'4586","reported_seq":4719,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:29.226213+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T15:59:29.226213+0000","last_peered":"2024-09-10T15:59:29.226213+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T15:59:29.226213+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":4586,"log_dups_size":0,"ondisk_log_size":4586,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18756067,"num_objects":4580,"num_object_clones":0,"num_object_copies":9160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4580,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2405,"num_write_kb":9620,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4123},{"shards":"2","objects":457}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":88,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.772572+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T15:59:15.772572+0000","last_peered":"2024-09-10T15:59:15.772572+0000","last_clean":"2024-09-10T15:59:15.772572+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T15:59:15.772572+0000","last_undegraded":"2024-09-10T15:59:15.772572+0000","last_fullsized":"2024-09-10T15:59:15.772572+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'4809","reported_seq":4932,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:31.198869+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T15:59:31.198869+0000","last_peered":"2024-09-10T15:59:31.198869+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T15:59:31.198869+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":4809,"log_dups_size":0,"ondisk_log_size":4809,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19664896,"num_objects":4801,"num_object_clones":0,"num_object_copies":9602,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4801,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1313,"num_write_kb":5252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":546},{"shards":"0,2","objects":4255}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":98,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.880567+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T15:59:15.880567+0000","last_peered":"2024-09-10T15:59:15.880567+0000","last_clean":"2024-09-10T15:59:15.880567+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T15:59:15.880567+0000","last_undegraded":"2024-09-10T15:59:15.880567+0000","last_fullsized":"2024-09-10T15:59:15.880567+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'4771","reported_seq":4900,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:29.100149+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T15:59:29.100149+0000","last_peered":"2024-09-10T15:59:29.100149+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T15:59:29.100149+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":4771,"log_dups_size":0,"ondisk_log_size":4771,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19522019,"num_objects":4767,"num_object_clones":0,"num_object_copies":9534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4767,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2451,"num_write_kb":9804,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4304},{"shards":"2","objects":463}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.837509+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T15:59:15.837509+0000","last_peered":"2024-09-10T15:59:15.837509+0000","last_clean":"2024-09-10T15:59:15.837509+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T15:59:15.837509+0000","last_undegraded":"2024-09-10T15:59:15.837509+0000","last_fullsized":"2024-09-10T15:59:15.837509+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'4879","reported_seq":5001,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:29.158212+0000","last_change":"2024-09-10T15:59:16.174113+0000","last_active":"2024-09-10T15:59:29.158212+0000","last_peered":"2024-09-10T15:59:29.158212+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T15:59:29.158212+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":4879,"log_dups_size":0,"ondisk_log_size":4879,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19959808,"num_objects":4873,"num_object_clones":0,"num_object_copies":9746,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4354,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4873,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2433,"num_write_kb":9732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4352},{"shards":"2","objects":521}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'4890","reported_seq":5019,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:31.185151+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T15:59:31.185151+0000","last_peered":"2024-09-10T15:59:31.185151+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T15:59:31.185151+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":4890,"log_dups_size":0,"ondisk_log_size":4890,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20021248,"num_objects":4888,"num_object_clones":0,"num_object_copies":9776,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4888,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1521,"num_write_kb":6084,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":584},{"shards":"0,2","objects":4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.989413+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T15:59:15.989413+0000","last_peered":"2024-09-10T15:59:15.989413+0000","last_clean":"2024-09-10T15:59:15.989413+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T15:59:15.989413+0000","last_undegraded":"2024-09-10T15:59:15.989413+0000","last_fullsized":"2024-09-10T15:59:15.989413+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'4771","reported_seq":4908,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:31.161344+0000","last_change":"2024-09-10T15:59:16.251610+0000","last_active":"2024-09-10T15:59:31.161344+0000","last_peered":"2024-09-10T15:59:31.161344+0000","last_clean":"2024-09-10T15:59:14.115595+0000","last_became_active":"2024-09-10T15:59:16.244089+0000","last_became_peered":"2024-09-10T15:59:16.244089+0000","last_unstale":"2024-09-10T15:59:31.161344+0000","last_undegraded":"2024-09-10T15:59:15.847127+0000","last_fullsized":"2024-09-10T15:59:15.774437+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":4771,"log_dups_size":0,"ondisk_log_size":4771,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19509248,"num_objects":4763,"num_object_clones":0,"num_object_copies":9526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4763,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1161,"num_write_kb":4644,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":560},{"shards":"0,2","objects":4203}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":88,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.773583+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T15:59:15.773583+0000","last_peered":"2024-09-10T15:59:15.773583+0000","last_clean":"2024-09-10T15:59:15.773583+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T15:59:15.773583+0000","last_undegraded":"2024-09-10T15:59:15.773583+0000","last_fullsized":"2024-09-10T15:59:15.773583+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'4975","reported_seq":5111,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:31.136433+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T15:59:31.136433+0000","last_peered":"2024-09-10T15:59:31.136433+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T15:59:31.136433+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":4975,"log_dups_size":0,"ondisk_log_size":4975,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20361216,"num_objects":4971,"num_object_clones":0,"num_object_copies":9942,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4971,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1230,"num_write_kb":4920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":600},{"shards":"0,2","objects":4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.916017+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T15:59:15.916017+0000","last_peered":"2024-09-10T15:59:15.916017+0000","last_clean":"2024-09-10T15:59:15.916017+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T15:59:15.916017+0000","last_undegraded":"2024-09-10T15:59:15.916017+0000","last_fullsized":"2024-09-10T15:59:15.916017+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.776033+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T15:59:15.776033+0000","last_peered":"2024-09-10T15:59:15.776033+0000","last_clean":"2024-09-10T15:59:15.776033+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T15:59:15.776033+0000","last_undegraded":"2024-09-10T15:59:15.776033+0000","last_fullsized":"2024-09-10T15:59:15.776033+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'4926","reported_seq":5064,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:31.147049+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T15:59:31.147049+0000","last_peered":"2024-09-10T15:59:31.147049+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T15:59:31.147049+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":4926,"log_dups_size":0,"ondisk_log_size":4926,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20128710,"num_objects":4916,"num_object_clones":0,"num_object_copies":9832,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4916,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1089,"num_write_kb":4356,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":593},{"shards":"0,2","objects":4323}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":316097246,"num_objects":77181,"num_object_clones":0,"num_object_copies":154362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":47117,"num_objects_misplaced":4010,"num_objects_unfound":0,"num_objects_dirty":77181,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":35027,"num_write_kb":140108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":261,"num_bytes_recovered":1069056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":628932608,"data_stored":628777249,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":77253,"ondisk_log_size":77253,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986194,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":437500,"kb_used_data":289196,"kb_used_omap":4,"kb_used_meta":148283,"kb_avail":93934340,"statfs":{"total":96636764160,"available":96188764160,"internally_reserved":0,"allocated":296136704,"data_stored":295858944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":151842474},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,4,1,0,2,3,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659140,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":85060,"kb_used_data":21428,"kb_used_omap":4,"kb_used_meta":63611,"kb_avail":94286780,"statfs":{"total":96636764160,"available":96549662720,"internally_reserved":0,"allocated":21942272,"data_stored":21613466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":65138350},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,0,4,5],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182423,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":467748,"kb_used_data":301460,"kb_used_omap":4,"kb_used_meta":166267,"kb_avail":93904092,"statfs":{"total":96636764160,"available":96157790208,"internally_reserved":0,"allocated":308695040,"data_stored":308413203,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":170258090},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,3,9,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":308781056,"data_stored":308744926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":23322624,"data_stored":23239525,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":296828928,"data_stored":296792798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:59:32.967 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout:2024-09-10T15:59:32.966435+0000 min lat: 0.00225581 max lat: 2.21736 avg lat: 0.0152555 2024-09-10T15:59:32.967 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-10T15:59:32.967 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 60 20 78648 78628 5.11848 2.46094 0.0159494 0.0152555 2024-09-10T15:59:33.967 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 61 20 79251 79231 5.07319 2.35547 0.0309172 0.015392 2024-09-10T15:59:34.967 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 62 20 79883 79863 5.03118 2.46875 0.0261076 0.0155216 2024-09-10T15:59:35.290 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:59:35.855 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T15:59:35.856 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T15:59:35.889 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":106,"stamp":"2024-09-10T15:59:35.549676+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":320623345,"num_objects":78288,"num_object_clones":0,"num_object_copies":156576,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":47117,"num_objects_misplaced":3899,"num_objects_unfound":0,"num_objects_dirty":78288,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":36134,"num_write_kb":144530,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":374,"num_bytes_recovered":1523731,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":78361,"ondisk_log_size":78361,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1023860,"kb_used_data":627588,"kb_used_omap":12,"kb_used_meta":396211,"kb_avail":282091660,"statfs":{"total":289910292480,"available":288861859840,"internally_reserved":0,"allocated":642650112,"data_stored":641761709,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":405720066},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,3,1,6,11,9,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":29921280,"num_objects":7305,"num_object_clones":0,"num_object_copies":14610,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-303,"num_objects_unfound":0,"num_objects_dirty":7305,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7305,"num_write_kb":29220,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":303,"num_bytes_recovered":1241088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002573"},"pg_stats":[{"pgid":"2.f","version":"43'4831","reported_seq":4900,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:31.181856+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T15:59:31.181856+0000","last_peered":"2024-09-10T15:59:31.181856+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T15:59:31.181856+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":4831,"log_dups_size":0,"ondisk_log_size":4831,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19787776,"num_objects":4831,"num_object_clones":0,"num_object_copies":9662,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4831,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4831,"num_write_kb":19324,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":572},{"shards":"0,2","objects":4258}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'4959","reported_seq":5066,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:34.236100+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T15:59:34.236100+0000","last_peered":"2024-09-10T15:59:34.236100+0000","last_clean":"2024-09-10T15:59:34.236100+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T15:59:34.236100+0000","last_undegraded":"2024-09-10T15:59:34.236100+0000","last_fullsized":"2024-09-10T15:59:34.236100+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":4959,"log_dups_size":0,"ondisk_log_size":4959,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20288454,"num_objects":4955,"num_object_clones":0,"num_object_copies":9910,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4955,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3064,"num_write_kb":12256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'4946","reported_seq":5809,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T15:59:34.216934+0000","last_change":"2024-09-10T15:59:16.142612+0000","last_active":"2024-09-10T15:59:34.216934+0000","last_peered":"2024-09-10T15:59:34.216934+0000","last_clean":"2024-09-10T15:59:14.092473+0000","last_became_active":"2024-09-10T15:59:16.109611+0000","last_became_peered":"2024-09-10T15:59:16.109611+0000","last_unstale":"2024-09-10T15:59:34.216934+0000","last_undegraded":"2024-09-10T15:59:34.216934+0000","last_fullsized":"2024-09-10T15:59:15.770264+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":4946,"log_dups_size":0,"ondisk_log_size":4946,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20242432,"num_objects":4942,"num_object_clones":0,"num_object_copies":9884,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3899,"num_objects_unfound":0,"num_objects_dirty":4942,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2950,"num_write_kb":11800,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":369,"num_bytes_recovered":1511424,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3899},{"shards":"2","objects":1043}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'4947","reported_seq":5022,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:34.236228+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T15:59:34.236228+0000","last_peered":"2024-09-10T15:59:34.236228+0000","last_clean":"2024-09-10T15:59:34.236228+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T15:59:34.236228+0000","last_undegraded":"2024-09-10T15:59:34.236228+0000","last_fullsized":"2024-09-10T15:59:34.236228+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":4947,"log_dups_size":0,"ondisk_log_size":4947,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20262912,"num_objects":4947,"num_object_clones":0,"num_object_copies":9894,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4947,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4947,"num_write_kb":19788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'4868","reported_seq":4992,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:31.118196+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T15:59:31.118196+0000","last_peered":"2024-09-10T15:59:31.118196+0000","last_clean":"2024-09-10T15:59:31.118196+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T15:59:31.118196+0000","last_undegraded":"2024-09-10T15:59:31.118196+0000","last_fullsized":"2024-09-10T15:59:31.118196+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":4868,"log_dups_size":0,"ondisk_log_size":4868,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19911139,"num_objects":4862,"num_object_clones":0,"num_object_copies":9724,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4862,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1579,"num_write_kb":6316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'4942","reported_seq":5069,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:31.174037+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T15:59:31.174037+0000","last_peered":"2024-09-10T15:59:31.174037+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T15:59:31.174037+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":4942,"log_dups_size":0,"ondisk_log_size":4942,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20222435,"num_objects":4938,"num_object_clones":0,"num_object_copies":9876,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4938,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1554,"num_write_kb":6216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":567},{"shards":"0,2","objects":4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'4770","reported_seq":4902,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:31.172003+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T15:59:31.172003+0000","last_peered":"2024-09-10T15:59:31.172003+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T15:59:31.172003+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":4770,"log_dups_size":0,"ondisk_log_size":4770,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19514310,"num_objects":4766,"num_object_clones":0,"num_object_copies":9532,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4766,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1269,"num_write_kb":5076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":529},{"shards":"0,2","objects":4237}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'4947","reported_seq":5070,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:31.195762+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T15:59:31.195762+0000","last_peered":"2024-09-10T15:59:31.195762+0000","last_clean":"2024-09-10T15:59:31.195762+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T15:59:31.195762+0000","last_undegraded":"2024-09-10T15:59:31.195762+0000","last_fullsized":"2024-09-10T15:59:31.195762+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":4947,"log_dups_size":0,"ondisk_log_size":4947,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20254720,"num_objects":4945,"num_object_clones":0,"num_object_copies":9890,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4945,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1794,"num_write_kb":7176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.775865+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T15:59:15.775865+0000","last_peered":"2024-09-10T15:59:15.775865+0000","last_clean":"2024-09-10T15:59:15.775865+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T15:59:15.775865+0000","last_undegraded":"2024-09-10T15:59:15.775865+0000","last_fullsized":"2024-09-10T15:59:15.775865+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'4774","reported_seq":4906,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:34.224072+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T15:59:34.224072+0000","last_peered":"2024-09-10T15:59:34.224072+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T15:59:34.224072+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":4774,"log_dups_size":0,"ondisk_log_size":4774,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19526115,"num_objects":4768,"num_object_clones":0,"num_object_copies":9536,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4768,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2593,"num_write_kb":10372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4123},{"shards":"2","objects":645}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":88,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.772572+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T15:59:15.772572+0000","last_peered":"2024-09-10T15:59:15.772572+0000","last_clean":"2024-09-10T15:59:15.772572+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T15:59:15.772572+0000","last_undegraded":"2024-09-10T15:59:15.772572+0000","last_fullsized":"2024-09-10T15:59:15.772572+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'4809","reported_seq":4932,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:31.198869+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T15:59:31.198869+0000","last_peered":"2024-09-10T15:59:31.198869+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T15:59:31.198869+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":4809,"log_dups_size":0,"ondisk_log_size":4809,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19664896,"num_objects":4801,"num_object_clones":0,"num_object_copies":9602,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4801,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1313,"num_write_kb":5252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":546},{"shards":"0,2","objects":4255}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":98,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.880567+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T15:59:15.880567+0000","last_peered":"2024-09-10T15:59:15.880567+0000","last_clean":"2024-09-10T15:59:15.880567+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T15:59:15.880567+0000","last_undegraded":"2024-09-10T15:59:15.880567+0000","last_fullsized":"2024-09-10T15:59:15.880567+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'4949","reported_seq":5078,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:34.237353+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T15:59:34.237353+0000","last_peered":"2024-09-10T15:59:34.237353+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T15:59:34.237353+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":4949,"log_dups_size":0,"ondisk_log_size":4949,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20251107,"num_objects":4945,"num_object_clones":0,"num_object_copies":9890,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4945,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2629,"num_write_kb":10516,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4304},{"shards":"2","objects":641}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.837509+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T15:59:15.837509+0000","last_peered":"2024-09-10T15:59:15.837509+0000","last_clean":"2024-09-10T15:59:15.837509+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T15:59:15.837509+0000","last_undegraded":"2024-09-10T15:59:15.837509+0000","last_fullsized":"2024-09-10T15:59:15.837509+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'5054","reported_seq":5177,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:34.139215+0000","last_change":"2024-09-10T15:59:16.174113+0000","last_active":"2024-09-10T15:59:34.139215+0000","last_peered":"2024-09-10T15:59:34.139215+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T15:59:34.139215+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":5054,"log_dups_size":0,"ondisk_log_size":5054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20676608,"num_objects":5048,"num_object_clones":0,"num_object_copies":10096,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4354,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5048,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2608,"num_write_kb":10432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4352},{"shards":"2","objects":696}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'4890","reported_seq":5019,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:31.185151+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T15:59:31.185151+0000","last_peered":"2024-09-10T15:59:31.185151+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T15:59:31.185151+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":4890,"log_dups_size":0,"ondisk_log_size":4890,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20021248,"num_objects":4888,"num_object_clones":0,"num_object_copies":9776,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4888,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1521,"num_write_kb":6084,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":584},{"shards":"0,2","objects":4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.989413+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T15:59:15.989413+0000","last_peered":"2024-09-10T15:59:15.989413+0000","last_clean":"2024-09-10T15:59:15.989413+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T15:59:15.989413+0000","last_undegraded":"2024-09-10T15:59:15.989413+0000","last_fullsized":"2024-09-10T15:59:15.989413+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'4771","reported_seq":4908,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:31.161344+0000","last_change":"2024-09-10T15:59:16.251610+0000","last_active":"2024-09-10T15:59:31.161344+0000","last_peered":"2024-09-10T15:59:31.161344+0000","last_clean":"2024-09-10T15:59:14.115595+0000","last_became_active":"2024-09-10T15:59:16.244089+0000","last_became_peered":"2024-09-10T15:59:16.244089+0000","last_unstale":"2024-09-10T15:59:31.161344+0000","last_undegraded":"2024-09-10T15:59:15.847127+0000","last_fullsized":"2024-09-10T15:59:15.774437+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":4771,"log_dups_size":0,"ondisk_log_size":4771,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19509248,"num_objects":4763,"num_object_clones":0,"num_object_copies":9526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4763,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1161,"num_write_kb":4644,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":560},{"shards":"0,2","objects":4203}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":88,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.773583+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T15:59:15.773583+0000","last_peered":"2024-09-10T15:59:15.773583+0000","last_clean":"2024-09-10T15:59:15.773583+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T15:59:15.773583+0000","last_undegraded":"2024-09-10T15:59:15.773583+0000","last_fullsized":"2024-09-10T15:59:15.773583+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'4975","reported_seq":5111,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:31.136433+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T15:59:31.136433+0000","last_peered":"2024-09-10T15:59:31.136433+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T15:59:31.136433+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":4975,"log_dups_size":0,"ondisk_log_size":4975,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20361216,"num_objects":4971,"num_object_clones":0,"num_object_copies":9942,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4971,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1230,"num_write_kb":4920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":600},{"shards":"0,2","objects":4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.916017+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T15:59:15.916017+0000","last_peered":"2024-09-10T15:59:15.916017+0000","last_clean":"2024-09-10T15:59:15.916017+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T15:59:15.916017+0000","last_undegraded":"2024-09-10T15:59:15.916017+0000","last_fullsized":"2024-09-10T15:59:15.916017+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.776033+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T15:59:15.776033+0000","last_peered":"2024-09-10T15:59:15.776033+0000","last_clean":"2024-09-10T15:59:15.776033+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T15:59:15.776033+0000","last_undegraded":"2024-09-10T15:59:15.776033+0000","last_fullsized":"2024-09-10T15:59:15.776033+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'4926","reported_seq":5064,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:31.147049+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T15:59:31.147049+0000","last_peered":"2024-09-10T15:59:31.147049+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T15:59:31.147049+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":4926,"log_dups_size":0,"ondisk_log_size":4926,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20128710,"num_objects":4916,"num_object_clones":0,"num_object_copies":9832,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4916,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1089,"num_write_kb":4356,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":593},{"shards":"0,2","objects":4323}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":320623326,"num_objects":78286,"num_object_clones":0,"num_object_copies":156572,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":47117,"num_objects_misplaced":3899,"num_objects_unfound":0,"num_objects_dirty":78286,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":36132,"num_write_kb":144528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":372,"num_bytes_recovered":1523712,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":644706304,"data_stored":644550945,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":78358,"ondisk_log_size":78358,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986195,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":453408,"kb_used_data":295376,"kb_used_omap":4,"kb_used_meta":158011,"kb_avail":93918432,"statfs":{"total":96636764160,"available":96172474368,"internally_reserved":0,"allocated":302465024,"data_stored":302187264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":161803946},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,1,0,3],"upper_bound":32},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659141,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":102704,"kb_used_data":30752,"kb_used_omap":4,"kb_used_meta":71931,"kb_avail":94269136,"statfs":{"total":96636764160,"available":96531595264,"internally_reserved":0,"allocated":31490048,"data_stored":31161242,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":73658030},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,3,5],"upper_bound":32},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182423,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":467748,"kb_used_data":301460,"kb_used_omap":4,"kb_used_meta":166267,"kb_avail":93904092,"statfs":{"total":96636764160,"available":96157790208,"internally_reserved":0,"allocated":308695040,"data_stored":308413203,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":170258090},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,3,9,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":308781056,"data_stored":308744926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":32886784,"data_stored":32803685,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":303038464,"data_stored":303002334,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:59:35.891 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:59:35.967 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 63 20 80470 80450 4.98771 2.29297 0.0152707 0.0156534 2024-09-10T15:59:36.463 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T15:59:36.466 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T15:59:36.490 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":106,"stamp":"2024-09-10T15:59:35.549676+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":320623345,"num_objects":78288,"num_object_clones":0,"num_object_copies":156576,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":47117,"num_objects_misplaced":3899,"num_objects_unfound":0,"num_objects_dirty":78288,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":36134,"num_write_kb":144530,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":374,"num_bytes_recovered":1523731,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":78361,"ondisk_log_size":78361,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1023860,"kb_used_data":627588,"kb_used_omap":12,"kb_used_meta":396211,"kb_avail":282091660,"statfs":{"total":289910292480,"available":288861859840,"internally_reserved":0,"allocated":642650112,"data_stored":641761709,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":405720066},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,3,1,6,11,9,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":29921280,"num_objects":7305,"num_object_clones":0,"num_object_copies":14610,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-303,"num_objects_unfound":0,"num_objects_dirty":7305,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7305,"num_write_kb":29220,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":303,"num_bytes_recovered":1241088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002573"},"pg_stats":[{"pgid":"2.f","version":"43'4831","reported_seq":4900,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:31.181856+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T15:59:31.181856+0000","last_peered":"2024-09-10T15:59:31.181856+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T15:59:31.181856+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":4831,"log_dups_size":0,"ondisk_log_size":4831,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19787776,"num_objects":4831,"num_object_clones":0,"num_object_copies":9662,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4831,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4831,"num_write_kb":19324,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":572},{"shards":"0,2","objects":4258}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'4959","reported_seq":5066,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:34.236100+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T15:59:34.236100+0000","last_peered":"2024-09-10T15:59:34.236100+0000","last_clean":"2024-09-10T15:59:34.236100+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T15:59:34.236100+0000","last_undegraded":"2024-09-10T15:59:34.236100+0000","last_fullsized":"2024-09-10T15:59:34.236100+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":4959,"log_dups_size":0,"ondisk_log_size":4959,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20288454,"num_objects":4955,"num_object_clones":0,"num_object_copies":9910,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4955,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3064,"num_write_kb":12256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'4946","reported_seq":5809,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T15:59:34.216934+0000","last_change":"2024-09-10T15:59:16.142612+0000","last_active":"2024-09-10T15:59:34.216934+0000","last_peered":"2024-09-10T15:59:34.216934+0000","last_clean":"2024-09-10T15:59:14.092473+0000","last_became_active":"2024-09-10T15:59:16.109611+0000","last_became_peered":"2024-09-10T15:59:16.109611+0000","last_unstale":"2024-09-10T15:59:34.216934+0000","last_undegraded":"2024-09-10T15:59:34.216934+0000","last_fullsized":"2024-09-10T15:59:15.770264+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":4946,"log_dups_size":0,"ondisk_log_size":4946,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20242432,"num_objects":4942,"num_object_clones":0,"num_object_copies":9884,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3899,"num_objects_unfound":0,"num_objects_dirty":4942,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2950,"num_write_kb":11800,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":369,"num_bytes_recovered":1511424,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3899},{"shards":"2","objects":1043}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'4947","reported_seq":5022,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:34.236228+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T15:59:34.236228+0000","last_peered":"2024-09-10T15:59:34.236228+0000","last_clean":"2024-09-10T15:59:34.236228+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T15:59:34.236228+0000","last_undegraded":"2024-09-10T15:59:34.236228+0000","last_fullsized":"2024-09-10T15:59:34.236228+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":4947,"log_dups_size":0,"ondisk_log_size":4947,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20262912,"num_objects":4947,"num_object_clones":0,"num_object_copies":9894,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4947,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4947,"num_write_kb":19788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'4868","reported_seq":4992,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:31.118196+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T15:59:31.118196+0000","last_peered":"2024-09-10T15:59:31.118196+0000","last_clean":"2024-09-10T15:59:31.118196+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T15:59:31.118196+0000","last_undegraded":"2024-09-10T15:59:31.118196+0000","last_fullsized":"2024-09-10T15:59:31.118196+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":4868,"log_dups_size":0,"ondisk_log_size":4868,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19911139,"num_objects":4862,"num_object_clones":0,"num_object_copies":9724,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4862,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1579,"num_write_kb":6316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'4942","reported_seq":5069,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:31.174037+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T15:59:31.174037+0000","last_peered":"2024-09-10T15:59:31.174037+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T15:59:31.174037+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":4942,"log_dups_size":0,"ondisk_log_size":4942,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20222435,"num_objects":4938,"num_object_clones":0,"num_object_copies":9876,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4938,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1554,"num_write_kb":6216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":567},{"shards":"0,2","objects":4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'4770","reported_seq":4902,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:31.172003+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T15:59:31.172003+0000","last_peered":"2024-09-10T15:59:31.172003+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T15:59:31.172003+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":4770,"log_dups_size":0,"ondisk_log_size":4770,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19514310,"num_objects":4766,"num_object_clones":0,"num_object_copies":9532,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4766,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1269,"num_write_kb":5076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":529},{"shards":"0,2","objects":4237}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'4947","reported_seq":5070,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:31.195762+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T15:59:31.195762+0000","last_peered":"2024-09-10T15:59:31.195762+0000","last_clean":"2024-09-10T15:59:31.195762+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T15:59:31.195762+0000","last_undegraded":"2024-09-10T15:59:31.195762+0000","last_fullsized":"2024-09-10T15:59:31.195762+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":4947,"log_dups_size":0,"ondisk_log_size":4947,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20254720,"num_objects":4945,"num_object_clones":0,"num_object_copies":9890,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4945,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1794,"num_write_kb":7176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.775865+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T15:59:15.775865+0000","last_peered":"2024-09-10T15:59:15.775865+0000","last_clean":"2024-09-10T15:59:15.775865+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T15:59:15.775865+0000","last_undegraded":"2024-09-10T15:59:15.775865+0000","last_fullsized":"2024-09-10T15:59:15.775865+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'4774","reported_seq":4906,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:34.224072+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T15:59:34.224072+0000","last_peered":"2024-09-10T15:59:34.224072+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T15:59:34.224072+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":4774,"log_dups_size":0,"ondisk_log_size":4774,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19526115,"num_objects":4768,"num_object_clones":0,"num_object_copies":9536,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4768,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2593,"num_write_kb":10372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4123},{"shards":"2","objects":645}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":88,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.772572+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T15:59:15.772572+0000","last_peered":"2024-09-10T15:59:15.772572+0000","last_clean":"2024-09-10T15:59:15.772572+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T15:59:15.772572+0000","last_undegraded":"2024-09-10T15:59:15.772572+0000","last_fullsized":"2024-09-10T15:59:15.772572+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'4809","reported_seq":4932,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:31.198869+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T15:59:31.198869+0000","last_peered":"2024-09-10T15:59:31.198869+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T15:59:31.198869+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":4809,"log_dups_size":0,"ondisk_log_size":4809,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19664896,"num_objects":4801,"num_object_clones":0,"num_object_copies":9602,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4801,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1313,"num_write_kb":5252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":546},{"shards":"0,2","objects":4255}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":98,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.880567+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T15:59:15.880567+0000","last_peered":"2024-09-10T15:59:15.880567+0000","last_clean":"2024-09-10T15:59:15.880567+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T15:59:15.880567+0000","last_undegraded":"2024-09-10T15:59:15.880567+0000","last_fullsized":"2024-09-10T15:59:15.880567+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'4949","reported_seq":5078,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:34.237353+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T15:59:34.237353+0000","last_peered":"2024-09-10T15:59:34.237353+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T15:59:34.237353+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":4949,"log_dups_size":0,"ondisk_log_size":4949,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20251107,"num_objects":4945,"num_object_clones":0,"num_object_copies":9890,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4945,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2629,"num_write_kb":10516,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4304},{"shards":"2","objects":641}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.837509+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T15:59:15.837509+0000","last_peered":"2024-09-10T15:59:15.837509+0000","last_clean":"2024-09-10T15:59:15.837509+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T15:59:15.837509+0000","last_undegraded":"2024-09-10T15:59:15.837509+0000","last_fullsized":"2024-09-10T15:59:15.837509+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'5054","reported_seq":5177,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:34.139215+0000","last_change":"2024-09-10T15:59:16.174113+0000","last_active":"2024-09-10T15:59:34.139215+0000","last_peered":"2024-09-10T15:59:34.139215+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T15:59:34.139215+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":5054,"log_dups_size":0,"ondisk_log_size":5054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20676608,"num_objects":5048,"num_object_clones":0,"num_object_copies":10096,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4354,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5048,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2608,"num_write_kb":10432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4352},{"shards":"2","objects":696}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'4890","reported_seq":5019,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:31.185151+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T15:59:31.185151+0000","last_peered":"2024-09-10T15:59:31.185151+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T15:59:31.185151+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":4890,"log_dups_size":0,"ondisk_log_size":4890,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20021248,"num_objects":4888,"num_object_clones":0,"num_object_copies":9776,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4888,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1521,"num_write_kb":6084,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":584},{"shards":"0,2","objects":4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.989413+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T15:59:15.989413+0000","last_peered":"2024-09-10T15:59:15.989413+0000","last_clean":"2024-09-10T15:59:15.989413+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T15:59:15.989413+0000","last_undegraded":"2024-09-10T15:59:15.989413+0000","last_fullsized":"2024-09-10T15:59:15.989413+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'4771","reported_seq":4908,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:31.161344+0000","last_change":"2024-09-10T15:59:16.251610+0000","last_active":"2024-09-10T15:59:31.161344+0000","last_peered":"2024-09-10T15:59:31.161344+0000","last_clean":"2024-09-10T15:59:14.115595+0000","last_became_active":"2024-09-10T15:59:16.244089+0000","last_became_peered":"2024-09-10T15:59:16.244089+0000","last_unstale":"2024-09-10T15:59:31.161344+0000","last_undegraded":"2024-09-10T15:59:15.847127+0000","last_fullsized":"2024-09-10T15:59:15.774437+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":4771,"log_dups_size":0,"ondisk_log_size":4771,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19509248,"num_objects":4763,"num_object_clones":0,"num_object_copies":9526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4763,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1161,"num_write_kb":4644,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":560},{"shards":"0,2","objects":4203}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":88,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.773583+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T15:59:15.773583+0000","last_peered":"2024-09-10T15:59:15.773583+0000","last_clean":"2024-09-10T15:59:15.773583+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T15:59:15.773583+0000","last_undegraded":"2024-09-10T15:59:15.773583+0000","last_fullsized":"2024-09-10T15:59:15.773583+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'4975","reported_seq":5111,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:31.136433+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T15:59:31.136433+0000","last_peered":"2024-09-10T15:59:31.136433+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T15:59:31.136433+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":4975,"log_dups_size":0,"ondisk_log_size":4975,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20361216,"num_objects":4971,"num_object_clones":0,"num_object_copies":9942,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4971,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1230,"num_write_kb":4920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":600},{"shards":"0,2","objects":4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.916017+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T15:59:15.916017+0000","last_peered":"2024-09-10T15:59:15.916017+0000","last_clean":"2024-09-10T15:59:15.916017+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T15:59:15.916017+0000","last_undegraded":"2024-09-10T15:59:15.916017+0000","last_fullsized":"2024-09-10T15:59:15.916017+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.776033+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T15:59:15.776033+0000","last_peered":"2024-09-10T15:59:15.776033+0000","last_clean":"2024-09-10T15:59:15.776033+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T15:59:15.776033+0000","last_undegraded":"2024-09-10T15:59:15.776033+0000","last_fullsized":"2024-09-10T15:59:15.776033+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'4926","reported_seq":5064,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:31.147049+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T15:59:31.147049+0000","last_peered":"2024-09-10T15:59:31.147049+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T15:59:31.147049+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":4926,"log_dups_size":0,"ondisk_log_size":4926,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20128710,"num_objects":4916,"num_object_clones":0,"num_object_copies":9832,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4916,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1089,"num_write_kb":4356,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":593},{"shards":"0,2","objects":4323}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":320623326,"num_objects":78286,"num_object_clones":0,"num_object_copies":156572,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":47117,"num_objects_misplaced":3899,"num_objects_unfound":0,"num_objects_dirty":78286,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":36132,"num_write_kb":144528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":372,"num_bytes_recovered":1523712,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":644706304,"data_stored":644550945,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":78358,"ondisk_log_size":78358,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986195,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":453408,"kb_used_data":295376,"kb_used_omap":4,"kb_used_meta":158011,"kb_avail":93918432,"statfs":{"total":96636764160,"available":96172474368,"internally_reserved":0,"allocated":302465024,"data_stored":302187264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":161803946},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,1,0,3],"upper_bound":32},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659141,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":102704,"kb_used_data":30752,"kb_used_omap":4,"kb_used_meta":71931,"kb_avail":94269136,"statfs":{"total":96636764160,"available":96531595264,"internally_reserved":0,"allocated":31490048,"data_stored":31161242,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":73658030},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,3,5],"upper_bound":32},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182423,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":467748,"kb_used_data":301460,"kb_used_omap":4,"kb_used_meta":166267,"kb_avail":93904092,"statfs":{"total":96636764160,"available":96157790208,"internally_reserved":0,"allocated":308695040,"data_stored":308413203,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":170258090},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,3,9,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":308781056,"data_stored":308744926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":32886784,"data_stored":32803685,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":303038464,"data_stored":303002334,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:59:36.491 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-10T15:59:36.491 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "43'4831", 'reported_seq': 4900, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:59:31.181856+0000', 'last_change': '2024-09-10T15:59:16.281558+0000', 'last_active': '2024-09-10T15:59:31.181856+0000', 'last_peered': '2024-09-10T15:59:31.181856+0000', 'last_clean': '2024-09-10T15:59:14.132456+0000', 'last_became_active': '2024-09-10T15:59:16.274407+0000', 'last_became_peered': '2024-09-10T15:59:16.274407+0000', 'last_unstale': '2024-09-10T15:59:31.181856+0000', 'last_undegraded': '2024-09-10T15:59:15.851279+0000', 'last_fullsized': '2024-09-10T15:59:15.774356+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'objects_scrubbed': 0, 'log_size': 4831, 'log_dups_size': 0, 'ondisk_log_size': 4831, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19787776, 'num_objects': 4831, 'num_object_clones': 0, 'num_object_copies': 9662, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4259, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4831, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4831, 'num_write_kb': 19324, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 572}, {'shards': '0,2', 'objects': 4258}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:59:36.491 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-10T15:59:36.492 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "43'4946", 'reported_seq': 5809, 'reported_epoch': 43, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-10T15:59:34.216934+0000', 'last_change': '2024-09-10T15:59:16.142612+0000', 'last_active': '2024-09-10T15:59:34.216934+0000', 'last_peered': '2024-09-10T15:59:34.216934+0000', 'last_clean': '2024-09-10T15:59:14.092473+0000', 'last_became_active': '2024-09-10T15:59:16.109611+0000', 'last_became_peered': '2024-09-10T15:59:16.109611+0000', 'last_unstale': '2024-09-10T15:59:34.216934+0000', 'last_undegraded': '2024-09-10T15:59:34.216934+0000', 'last_fullsized': '2024-09-10T15:59:15.770264+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2029", 'last_scrub_stamp': '2024-09-10T15:58:45.933789+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:45.933789+0000', 'objects_scrubbed': 1992, 'log_size': 4946, 'log_dups_size': 0, 'ondisk_log_size': 4946, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 668, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20242432, 'num_objects': 4942, 'num_object_clones': 0, 'num_object_copies': 9884, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 3899, 'num_objects_unfound': 0, 'num_objects_dirty': 4942, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2950, 'num_write_kb': 11800, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 369, 'num_bytes_recovered': 1511424, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3899}, {'shards': '2', 'objects': 1043}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:59:36.492 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-10T15:59:36.492 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "43'4942", 'reported_seq': 5069, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:59:31.174037+0000', 'last_change': '2024-09-10T15:59:16.308264+0000', 'last_active': '2024-09-10T15:59:31.174037+0000', 'last_peered': '2024-09-10T15:59:31.174037+0000', 'last_clean': '2024-09-10T15:59:14.111551+0000', 'last_became_active': '2024-09-10T15:59:16.300040+0000', 'last_became_peered': '2024-09-10T15:59:16.300040+0000', 'last_unstale': '2024-09-10T15:59:31.174037+0000', 'last_undegraded': '2024-09-10T15:59:15.878234+0000', 'last_fullsized': '2024-09-10T15:59:15.769328+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3442", 'last_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'objects_scrubbed': 3384, 'log_size': 4942, 'log_dups_size': 0, 'ondisk_log_size': 4942, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1292, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20222435, 'num_objects': 4938, 'num_object_clones': 0, 'num_object_copies': 9876, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4372, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4938, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 1554, 'num_write_kb': 6216, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 567}, {'shards': '0,2', 'objects': 4371}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:59:36.492 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-10T15:59:36.492 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "43'4770", 'reported_seq': 4902, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:59:31.172003+0000', 'last_change': '2024-09-10T15:59:16.331899+0000', 'last_active': '2024-09-10T15:59:31.172003+0000', 'last_peered': '2024-09-10T15:59:31.172003+0000', 'last_clean': '2024-09-10T15:59:14.139436+0000', 'last_became_active': '2024-09-10T15:59:16.318715+0000', 'last_became_peered': '2024-09-10T15:59:16.318715+0000', 'last_unstale': '2024-09-10T15:59:31.172003+0000', 'last_undegraded': '2024-09-10T15:59:15.877524+0000', 'last_fullsized': '2024-09-10T15:59:15.769042+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3537", 'last_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'objects_scrubbed': 3497, 'log_size': 4770, 'log_dups_size': 0, 'ondisk_log_size': 4770, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1224, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19514310, 'num_objects': 4766, 'num_object_clones': 0, 'num_object_copies': 9532, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4237, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4766, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 1269, 'num_write_kb': 5076, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 529}, {'shards': '0,2', 'objects': 4237}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:59:36.492 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-10T15:59:36.492 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "43'4774", 'reported_seq': 4906, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:59:34.224072+0000', 'last_change': '2024-09-10T15:59:16.175680+0000', 'last_active': '2024-09-10T15:59:34.224072+0000', 'last_peered': '2024-09-10T15:59:34.224072+0000', 'last_clean': '2024-09-10T15:59:14.088187+0000', 'last_became_active': '2024-09-10T15:59:16.165810+0000', 'last_became_peered': '2024-09-10T15:59:16.165810+0000', 'last_unstale': '2024-09-10T15:59:34.224072+0000', 'last_undegraded': '2024-09-10T15:59:15.822098+0000', 'last_fullsized': '2024-09-10T15:59:15.769123+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2218", 'last_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'objects_scrubbed': 2175, 'log_size': 4774, 'log_dups_size': 0, 'ondisk_log_size': 4774, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 825, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19526115, 'num_objects': 4768, 'num_object_clones': 0, 'num_object_copies': 9536, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4124, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4768, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 2593, 'num_write_kb': 10372, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4123}, {'shards': '2', 'objects': 645}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:59:36.492 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-10T15:59:36.492 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "43'4809", 'reported_seq': 4932, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:59:31.198869+0000', 'last_change': '2024-09-10T15:59:16.277742+0000', 'last_active': '2024-09-10T15:59:31.198869+0000', 'last_peered': '2024-09-10T15:59:31.198869+0000', 'last_clean': '2024-09-10T15:59:14.129081+0000', 'last_became_active': '2024-09-10T15:59:16.269835+0000', 'last_became_peered': '2024-09-10T15:59:16.269835+0000', 'last_unstale': '2024-09-10T15:59:31.198869+0000', 'last_undegraded': '2024-09-10T15:59:15.860857+0000', 'last_fullsized': '2024-09-10T15:59:15.771410+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3557", 'last_scrub_stamp': '2024-09-10T15:59:02.493678+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:02.493678+0000', 'objects_scrubbed': 3488, 'log_size': 4809, 'log_dups_size': 0, 'ondisk_log_size': 4809, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1472, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19664896, 'num_objects': 4801, 'num_object_clones': 0, 'num_object_copies': 9602, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4257, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4801, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1313, 'num_write_kb': 5252, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 546}, {'shards': '0,2', 'objects': 4255}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:59:36.493 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-10T15:59:36.493 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "43'4949", 'reported_seq': 5078, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:59:34.237353+0000', 'last_change': '2024-09-10T15:59:16.284311+0000', 'last_active': '2024-09-10T15:59:34.237353+0000', 'last_peered': '2024-09-10T15:59:34.237353+0000', 'last_clean': '2024-09-10T15:59:14.116041+0000', 'last_became_active': '2024-09-10T15:59:16.274543+0000', 'last_became_peered': '2024-09-10T15:59:16.274543+0000', 'last_unstale': '2024-09-10T15:59:34.237353+0000', 'last_undegraded': '2024-09-10T15:59:15.812484+0000', 'last_fullsized': '2024-09-10T15:59:15.770885+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2356", 'last_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'objects_scrubbed': 2316, 'log_size': 4949, 'log_dups_size': 0, 'ondisk_log_size': 4949, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 783, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20251107, 'num_objects': 4945, 'num_object_clones': 0, 'num_object_copies': 9890, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4945, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 2629, 'num_write_kb': 10516, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4304}, {'shards': '2', 'objects': 641}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:59:36.493 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-10T15:59:36.493 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "43'5054", 'reported_seq': 5177, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:59:34.139215+0000', 'last_change': '2024-09-10T15:59:16.174113+0000', 'last_active': '2024-09-10T15:59:34.139215+0000', 'last_peered': '2024-09-10T15:59:34.139215+0000', 'last_clean': '2024-09-10T15:59:14.110654+0000', 'last_became_active': '2024-09-10T15:59:16.164645+0000', 'last_became_peered': '2024-09-10T15:59:16.164645+0000', 'last_unstale': '2024-09-10T15:59:34.139215+0000', 'last_undegraded': '2024-09-10T15:59:15.803593+0000', 'last_fullsized': '2024-09-10T15:59:15.769083+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2486", 'last_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'objects_scrubbed': 2440, 'log_size': 5054, 'log_dups_size': 0, 'ondisk_log_size': 5054, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 792, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20676608, 'num_objects': 5048, 'num_object_clones': 0, 'num_object_copies': 10096, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4354, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5048, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2608, 'num_write_kb': 10432, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4352}, {'shards': '2', 'objects': 696}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:59:36.493 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-10T15:59:36.493 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "43'4890", 'reported_seq': 5019, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:59:31.185151+0000', 'last_change': '2024-09-10T15:59:16.315641+0000', 'last_active': '2024-09-10T15:59:31.185151+0000', 'last_peered': '2024-09-10T15:59:31.185151+0000', 'last_clean': '2024-09-10T15:59:14.117001+0000', 'last_became_active': '2024-09-10T15:59:16.302245+0000', 'last_became_peered': '2024-09-10T15:59:16.302245+0000', 'last_unstale': '2024-09-10T15:59:31.185151+0000', 'last_undegraded': '2024-09-10T15:59:15.845012+0000', 'last_fullsized': '2024-09-10T15:59:15.770337+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3404", 'last_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'objects_scrubbed': 3367, 'log_size': 4890, 'log_dups_size': 0, 'ondisk_log_size': 4890, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 983, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20021248, 'num_objects': 4888, 'num_object_clones': 0, 'num_object_copies': 9776, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4888, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1521, 'num_write_kb': 6084, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 584}, {'shards': '0,2', 'objects': 4304}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:59:36.493 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-10T15:59:36.493 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "43'4771", 'reported_seq': 4908, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:59:31.161344+0000', 'last_change': '2024-09-10T15:59:16.251610+0000', 'last_active': '2024-09-10T15:59:31.161344+0000', 'last_peered': '2024-09-10T15:59:31.161344+0000', 'last_clean': '2024-09-10T15:59:14.115595+0000', 'last_became_active': '2024-09-10T15:59:16.244089+0000', 'last_became_peered': '2024-09-10T15:59:16.244089+0000', 'last_unstale': '2024-09-10T15:59:31.161344+0000', 'last_undegraded': '2024-09-10T15:59:15.847127+0000', 'last_fullsized': '2024-09-10T15:59:15.774437+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3653", 'last_scrub_stamp': '2024-09-10T15:59:05.298468+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:05.298468+0000', 'objects_scrubbed': 3602, 'log_size': 4771, 'log_dups_size': 0, 'ondisk_log_size': 4771, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1289, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19509248, 'num_objects': 4763, 'num_object_clones': 0, 'num_object_copies': 9526, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4206, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4763, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1161, 'num_write_kb': 4644, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 560}, {'shards': '0,2', 'objects': 4203}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:59:36.493 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-10T15:59:36.493 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "43'4975", 'reported_seq': 5111, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:59:31.136433+0000', 'last_change': '2024-09-10T15:59:16.311291+0000', 'last_active': '2024-09-10T15:59:31.136433+0000', 'last_peered': '2024-09-10T15:59:31.136433+0000', 'last_clean': '2024-09-10T15:59:14.116363+0000', 'last_became_active': '2024-09-10T15:59:16.301244+0000', 'last_became_peered': '2024-09-10T15:59:16.301244+0000', 'last_unstale': '2024-09-10T15:59:31.136433+0000', 'last_undegraded': '2024-09-10T15:59:15.883434+0000', 'last_fullsized': '2024-09-10T15:59:15.772756+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3789", 'last_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'objects_scrubbed': 3741, 'log_size': 4975, 'log_dups_size': 0, 'ondisk_log_size': 4975, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1440, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20361216, 'num_objects': 4971, 'num_object_clones': 0, 'num_object_copies': 9942, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4373, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4971, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1230, 'num_write_kb': 4920, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 600}, {'shards': '0,2', 'objects': 4371}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:59:36.494 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-10T15:59:36.494 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "43'4926", 'reported_seq': 5064, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:59:31.147049+0000', 'last_change': '2024-09-10T15:59:16.313409+0000', 'last_active': '2024-09-10T15:59:31.147049+0000', 'last_peered': '2024-09-10T15:59:31.147049+0000', 'last_clean': '2024-09-10T15:59:14.129780+0000', 'last_became_active': '2024-09-10T15:59:16.300932+0000', 'last_became_peered': '2024-09-10T15:59:16.300932+0000', 'last_unstale': '2024-09-10T15:59:31.147049+0000', 'last_undegraded': '2024-09-10T15:59:15.887823+0000', 'last_fullsized': '2024-09-10T15:59:15.768513+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3880", 'last_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'objects_scrubbed': 3827, 'log_size': 4926, 'log_dups_size': 0, 'ondisk_log_size': 4926, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1397, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20128710, 'num_objects': 4916, 'num_object_clones': 0, 'num_object_copies': 9832, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4325, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4916, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1089, 'num_write_kb': 4356, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 593}, {'shards': '0,2', 'objects': 4323}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:59:36.494 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T15:59:36.969 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 64 20 81017 80997 4.943 2.13672 0.0569517 0.0157964 2024-09-10T15:59:37.173 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T15:59:37.174 INFO:teuthology.orchestra.run.smithi115.stdout:{"fsid":"7effee29-df05-4351-9741-ee203564ed81","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 47117/156576 objects degraded (30.092%), 11 pgs degraded","count":11},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":141,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":43,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983954,"num_in_osds":3,"osd_in_since":1725983947,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":78288,"data_bytes":320623345,"bytes_used":1048432640,"bytes_avail":288861859840,"bytes_total":289910292480,"degraded_objects":47117,"degraded_total":156576,"degraded_ratio":0.30092095851216022,"misplaced_objects":3899,"misplaced_total":156576,"misplaced_ratio":0.024901645207439194,"recovering_objects_per_sec":25,"recovering_bytes_per_sec":103401,"recovering_keys_per_sec":0,"num_objects_recovered":303,"num_bytes_recovered":1241088,"num_keys_recovered":0,"write_bytes_sec":2492905,"read_op_per_sec":0,"write_op_per_sec":608},"fsmap":{"epoch":1,"btime":"2024-09-10T15:57:14:967120+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":5,"modified":"2024-09-10T15:59:19.546323+0000","services":{"osd":{"daemons":{"summary":"","1":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-10T15:59:37.197 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-10T15:59:37.197 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:59:37.751 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T15:59:37.754 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T15:59:37.775 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":107,"stamp":"2024-09-10T15:59:37.550141+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":328286961,"num_objects":80159,"num_object_clones":0,"num_object_copies":160318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":47117,"num_objects_misplaced":3899,"num_objects_unfound":0,"num_objects_dirty":80159,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":38005,"num_write_kb":152014,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":374,"num_bytes_recovered":1523731,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80232,"ondisk_log_size":80232,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1020428,"kb_used_data":636636,"kb_used_omap":12,"kb_used_meta":383731,"kb_avail":282095092,"statfs":{"total":289910292480,"available":288865374208,"internally_reserved":0,"allocated":651915264,"data_stored":651026861,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":392940546},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,3,1,5,9,2,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":32792576,"num_objects":8006,"num_object_clones":0,"num_object_copies":16012,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-215,"num_objects_unfound":0,"num_objects_dirty":8006,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8006,"num_write_kb":32024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":215,"num_bytes_recovered":880640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002584"},"pg_stats":[{"pgid":"2.f","version":"43'5014","reported_seq":5083,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:36.198291+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T15:59:36.198291+0000","last_peered":"2024-09-10T15:59:36.198291+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T15:59:36.198291+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":5014,"log_dups_size":0,"ondisk_log_size":5014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20537344,"num_objects":5014,"num_object_clones":0,"num_object_copies":10028,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5014,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5014,"num_write_kb":20056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":755},{"shards":"0,2","objects":4258}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'4959","reported_seq":5066,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:34.236100+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T15:59:34.236100+0000","last_peered":"2024-09-10T15:59:34.236100+0000","last_clean":"2024-09-10T15:59:34.236100+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T15:59:34.236100+0000","last_undegraded":"2024-09-10T15:59:34.236100+0000","last_fullsized":"2024-09-10T15:59:34.236100+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":4959,"log_dups_size":0,"ondisk_log_size":4959,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20288454,"num_objects":4955,"num_object_clones":0,"num_object_copies":9910,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4955,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3064,"num_write_kb":12256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'4946","reported_seq":5809,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T15:59:34.216934+0000","last_change":"2024-09-10T15:59:16.142612+0000","last_active":"2024-09-10T15:59:34.216934+0000","last_peered":"2024-09-10T15:59:34.216934+0000","last_clean":"2024-09-10T15:59:14.092473+0000","last_became_active":"2024-09-10T15:59:16.109611+0000","last_became_peered":"2024-09-10T15:59:16.109611+0000","last_unstale":"2024-09-10T15:59:34.216934+0000","last_undegraded":"2024-09-10T15:59:34.216934+0000","last_fullsized":"2024-09-10T15:59:15.770264+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":4946,"log_dups_size":0,"ondisk_log_size":4946,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20242432,"num_objects":4942,"num_object_clones":0,"num_object_copies":9884,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3899,"num_objects_unfound":0,"num_objects_dirty":4942,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2950,"num_write_kb":11800,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":369,"num_bytes_recovered":1511424,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3899},{"shards":"2","objects":1043}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'4947","reported_seq":5022,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:34.236228+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T15:59:34.236228+0000","last_peered":"2024-09-10T15:59:34.236228+0000","last_clean":"2024-09-10T15:59:34.236228+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T15:59:34.236228+0000","last_undegraded":"2024-09-10T15:59:34.236228+0000","last_fullsized":"2024-09-10T15:59:34.236228+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":4947,"log_dups_size":0,"ondisk_log_size":4947,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20262912,"num_objects":4947,"num_object_clones":0,"num_object_copies":9894,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4947,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4947,"num_write_kb":19788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'5051","reported_seq":5175,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:36.175032+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T15:59:36.175032+0000","last_peered":"2024-09-10T15:59:36.175032+0000","last_clean":"2024-09-10T15:59:36.175032+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T15:59:36.175032+0000","last_undegraded":"2024-09-10T15:59:36.175032+0000","last_fullsized":"2024-09-10T15:59:36.175032+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":5051,"log_dups_size":0,"ondisk_log_size":5051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20660707,"num_objects":5045,"num_object_clones":0,"num_object_copies":10090,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5045,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1762,"num_write_kb":7048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'5146","reported_seq":5277,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:36.218750+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T15:59:36.218750+0000","last_peered":"2024-09-10T15:59:36.218750+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T15:59:36.218750+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":5146,"log_dups_size":0,"ondisk_log_size":5146,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21058019,"num_objects":5142,"num_object_clones":0,"num_object_copies":10284,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5142,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1758,"num_write_kb":7032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":771},{"shards":"0,2","objects":4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'4968","reported_seq":5100,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:36.192655+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T15:59:36.192655+0000","last_peered":"2024-09-10T15:59:36.192655+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T15:59:36.192655+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":4968,"log_dups_size":0,"ondisk_log_size":4968,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20325318,"num_objects":4964,"num_object_clones":0,"num_object_copies":9928,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4964,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1467,"num_write_kb":5868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":727},{"shards":"0,2","objects":4237}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'5150","reported_seq":5272,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:36.216046+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T15:59:36.216046+0000","last_peered":"2024-09-10T15:59:36.216046+0000","last_clean":"2024-09-10T15:59:36.216046+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T15:59:36.216046+0000","last_undegraded":"2024-09-10T15:59:36.216046+0000","last_fullsized":"2024-09-10T15:59:36.216046+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":5150,"log_dups_size":0,"ondisk_log_size":5150,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21086208,"num_objects":5148,"num_object_clones":0,"num_object_copies":10296,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5148,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1997,"num_write_kb":7988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.775865+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T15:59:15.775865+0000","last_peered":"2024-09-10T15:59:15.775865+0000","last_clean":"2024-09-10T15:59:15.775865+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T15:59:15.775865+0000","last_undegraded":"2024-09-10T15:59:15.775865+0000","last_fullsized":"2024-09-10T15:59:15.775865+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'4774","reported_seq":4906,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:34.224072+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T15:59:34.224072+0000","last_peered":"2024-09-10T15:59:34.224072+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T15:59:34.224072+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":4774,"log_dups_size":0,"ondisk_log_size":4774,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19526115,"num_objects":4768,"num_object_clones":0,"num_object_copies":9536,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4768,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2593,"num_write_kb":10372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4123},{"shards":"2","objects":645}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":88,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.772572+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T15:59:15.772572+0000","last_peered":"2024-09-10T15:59:15.772572+0000","last_clean":"2024-09-10T15:59:15.772572+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T15:59:15.772572+0000","last_undegraded":"2024-09-10T15:59:15.772572+0000","last_fullsized":"2024-09-10T15:59:15.772572+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'4975","reported_seq":5100,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:36.212625+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T15:59:36.212625+0000","last_peered":"2024-09-10T15:59:36.212625+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T15:59:36.212625+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":4975,"log_dups_size":0,"ondisk_log_size":4975,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20344832,"num_objects":4967,"num_object_clones":0,"num_object_copies":9934,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4967,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1479,"num_write_kb":5916,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":712},{"shards":"0,2","objects":4255}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":98,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.880567+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T15:59:15.880567+0000","last_peered":"2024-09-10T15:59:15.880567+0000","last_clean":"2024-09-10T15:59:15.880567+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T15:59:15.880567+0000","last_undegraded":"2024-09-10T15:59:15.880567+0000","last_fullsized":"2024-09-10T15:59:15.880567+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'4949","reported_seq":5078,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:34.237353+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T15:59:34.237353+0000","last_peered":"2024-09-10T15:59:34.237353+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T15:59:34.237353+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":4949,"log_dups_size":0,"ondisk_log_size":4949,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20251107,"num_objects":4945,"num_object_clones":0,"num_object_copies":9890,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4945,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2629,"num_write_kb":10516,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4304},{"shards":"2","objects":641}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.837509+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T15:59:15.837509+0000","last_peered":"2024-09-10T15:59:15.837509+0000","last_clean":"2024-09-10T15:59:15.837509+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T15:59:15.837509+0000","last_undegraded":"2024-09-10T15:59:15.837509+0000","last_fullsized":"2024-09-10T15:59:15.837509+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'5054","reported_seq":5177,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:34.139215+0000","last_change":"2024-09-10T15:59:16.174113+0000","last_active":"2024-09-10T15:59:34.139215+0000","last_peered":"2024-09-10T15:59:34.139215+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T15:59:34.139215+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":5054,"log_dups_size":0,"ondisk_log_size":5054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20676608,"num_objects":5048,"num_object_clones":0,"num_object_copies":10096,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4354,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5048,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2608,"num_write_kb":10432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4352},{"shards":"2","objects":696}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'5064","reported_seq":5193,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:36.198842+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T15:59:36.198842+0000","last_peered":"2024-09-10T15:59:36.198842+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T15:59:36.198842+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":5064,"log_dups_size":0,"ondisk_log_size":5064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20733952,"num_objects":5062,"num_object_clones":0,"num_object_copies":10124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5062,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1695,"num_write_kb":6780,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":758},{"shards":"0,2","objects":4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.989413+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T15:59:15.989413+0000","last_peered":"2024-09-10T15:59:15.989413+0000","last_clean":"2024-09-10T15:59:15.989413+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T15:59:15.989413+0000","last_undegraded":"2024-09-10T15:59:15.989413+0000","last_fullsized":"2024-09-10T15:59:15.989413+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'4960","reported_seq":5097,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:36.160282+0000","last_change":"2024-09-10T15:59:16.251610+0000","last_active":"2024-09-10T15:59:36.160282+0000","last_peered":"2024-09-10T15:59:36.160282+0000","last_clean":"2024-09-10T15:59:14.115595+0000","last_became_active":"2024-09-10T15:59:16.244089+0000","last_became_peered":"2024-09-10T15:59:16.244089+0000","last_unstale":"2024-09-10T15:59:36.160282+0000","last_undegraded":"2024-09-10T15:59:15.847127+0000","last_fullsized":"2024-09-10T15:59:15.774437+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":4960,"log_dups_size":0,"ondisk_log_size":4960,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20283392,"num_objects":4952,"num_object_clones":0,"num_object_copies":9904,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4952,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1350,"num_write_kb":5400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":749},{"shards":"0,2","objects":4203}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":88,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.773583+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T15:59:15.773583+0000","last_peered":"2024-09-10T15:59:15.773583+0000","last_clean":"2024-09-10T15:59:15.773583+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T15:59:15.773583+0000","last_undegraded":"2024-09-10T15:59:15.773583+0000","last_fullsized":"2024-09-10T15:59:15.773583+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'5160","reported_seq":5296,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:36.130907+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T15:59:36.130907+0000","last_peered":"2024-09-10T15:59:36.130907+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T15:59:36.130907+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":5160,"log_dups_size":0,"ondisk_log_size":5160,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21118976,"num_objects":5156,"num_object_clones":0,"num_object_copies":10312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5156,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1415,"num_write_kb":5660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":785},{"shards":"0,2","objects":4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.916017+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T15:59:15.916017+0000","last_peered":"2024-09-10T15:59:15.916017+0000","last_clean":"2024-09-10T15:59:15.916017+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T15:59:15.916017+0000","last_undegraded":"2024-09-10T15:59:15.916017+0000","last_fullsized":"2024-09-10T15:59:15.916017+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.776033+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T15:59:15.776033+0000","last_peered":"2024-09-10T15:59:15.776033+0000","last_clean":"2024-09-10T15:59:15.776033+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T15:59:15.776033+0000","last_undegraded":"2024-09-10T15:59:15.776033+0000","last_fullsized":"2024-09-10T15:59:15.776033+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'5112","reported_seq":5251,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:36.149258+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T15:59:36.149258+0000","last_peered":"2024-09-10T15:59:36.149258+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T15:59:36.149258+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":5112,"log_dups_size":0,"ondisk_log_size":5112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20890566,"num_objects":5102,"num_object_clones":0,"num_object_copies":10204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5102,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1275,"num_write_kb":5100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":779},{"shards":"0,2","objects":4323}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":328286942,"num_objects":80157,"num_object_clones":0,"num_object_copies":160314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":47117,"num_objects_misplaced":3899,"num_objects_unfound":0,"num_objects_dirty":80157,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":38003,"num_write_kb":152012,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":372,"num_bytes_recovered":1523712,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":654082048,"data_stored":653926689,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80229,"ondisk_log_size":80229,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986195,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":453408,"kb_used_data":295376,"kb_used_omap":4,"kb_used_meta":158011,"kb_avail":93918432,"statfs":{"total":96636764160,"available":96172474368,"internally_reserved":0,"allocated":302465024,"data_stored":302187264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":161803946},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,1,0,3],"upper_bound":32},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659141,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":102704,"kb_used_data":30752,"kb_used_omap":4,"kb_used_meta":71931,"kb_avail":94269136,"statfs":{"total":96636764160,"available":96531595264,"internally_reserved":0,"allocated":31490048,"data_stored":31161242,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":73658030},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,3,5],"upper_bound":32},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182424,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":464316,"kb_used_data":310508,"kb_used_omap":4,"kb_used_meta":153787,"kb_avail":93907524,"statfs":{"total":96636764160,"available":96161304576,"internally_reserved":0,"allocated":317960192,"data_stored":317678355,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":157478570},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,2,1,2,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":318156800,"data_stored":318120670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":32886784,"data_stored":32803685,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":303038464,"data_stored":303002334,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:59:37.969 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 65 20 81582 81562 4.90091 2.20703 0.0227215 0.0159334 2024-09-10T15:59:38.969 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 66 20 82200 82180 4.86323 2.41406 0.00819061 0.0160567 2024-09-10T15:59:39.969 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 67 20 82812 82792 4.82632 2.39062 0.0313699 0.0161798 2024-09-10T15:59:40.778 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:59:40.969 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 68 20 83419 83399 4.79021 2.37109 0.0297221 0.0163019 2024-09-10T15:59:41.363 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T15:59:41.365 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T15:59:41.389 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":108,"stamp":"2024-09-10T15:59:39.550604+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":332845809,"num_objects":81272,"num_object_clones":0,"num_object_copies":162544,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":47117,"num_objects_misplaced":3786,"num_objects_unfound":0,"num_objects_dirty":81272,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":39118,"num_write_kb":156466,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":487,"num_bytes_recovered":1986579,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":81345,"ondisk_log_size":81345,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1043960,"kb_used_data":651848,"kb_used_omap":12,"kb_used_meta":392051,"kb_avail":282071560,"statfs":{"total":289910292480,"available":288841277440,"internally_reserved":0,"allocated":667492352,"data_stored":666603949,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":401460226},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,1,4,5,6,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":29470720,"num_objects":7195,"num_object_clones":0,"num_object_copies":14390,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-328,"num_objects_unfound":0,"num_objects_dirty":7195,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7195,"num_write_kb":28780,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":328,"num_bytes_recovered":1343488,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002633"},"pg_stats":[{"pgid":"2.f","version":"43'5014","reported_seq":5083,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:36.198291+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T15:59:36.198291+0000","last_peered":"2024-09-10T15:59:36.198291+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T15:59:36.198291+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":5014,"log_dups_size":0,"ondisk_log_size":5014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20537344,"num_objects":5014,"num_object_clones":0,"num_object_copies":10028,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5014,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5014,"num_write_kb":20056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":755},{"shards":"0,2","objects":4258}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'5153","reported_seq":5260,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:39.210025+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T15:59:39.210025+0000","last_peered":"2024-09-10T15:59:39.210025+0000","last_clean":"2024-09-10T15:59:39.210025+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T15:59:39.210025+0000","last_undegraded":"2024-09-10T15:59:39.210025+0000","last_fullsized":"2024-09-10T15:59:39.210025+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":5153,"log_dups_size":0,"ondisk_log_size":5153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21083078,"num_objects":5149,"num_object_clones":0,"num_object_copies":10298,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5149,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3258,"num_write_kb":13032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'5114","reported_seq":6202,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T15:59:39.261499+0000","last_change":"2024-09-10T15:59:16.142612+0000","last_active":"2024-09-10T15:59:39.261499+0000","last_peered":"2024-09-10T15:59:39.261499+0000","last_clean":"2024-09-10T15:59:14.092473+0000","last_became_active":"2024-09-10T15:59:16.109611+0000","last_became_peered":"2024-09-10T15:59:16.109611+0000","last_unstale":"2024-09-10T15:59:39.261499+0000","last_undegraded":"2024-09-10T15:59:39.261499+0000","last_fullsized":"2024-09-10T15:59:15.770264+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":5114,"log_dups_size":0,"ondisk_log_size":5114,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20930560,"num_objects":5110,"num_object_clones":0,"num_object_copies":10220,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3786,"num_objects_unfound":0,"num_objects_dirty":5110,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3118,"num_write_kb":12472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":482,"num_bytes_recovered":1974272,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3786},{"shards":"2","objects":1324}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'5150","reported_seq":5225,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:39.198032+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T15:59:39.198032+0000","last_peered":"2024-09-10T15:59:39.198032+0000","last_clean":"2024-09-10T15:59:39.198032+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T15:59:39.198032+0000","last_undegraded":"2024-09-10T15:59:39.198032+0000","last_fullsized":"2024-09-10T15:59:39.198032+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":5150,"log_dups_size":0,"ondisk_log_size":5150,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21094400,"num_objects":5150,"num_object_clones":0,"num_object_copies":10300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5150,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5150,"num_write_kb":20600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'5051","reported_seq":5175,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:36.175032+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T15:59:36.175032+0000","last_peered":"2024-09-10T15:59:36.175032+0000","last_clean":"2024-09-10T15:59:36.175032+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T15:59:36.175032+0000","last_undegraded":"2024-09-10T15:59:36.175032+0000","last_fullsized":"2024-09-10T15:59:36.175032+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":5051,"log_dups_size":0,"ondisk_log_size":5051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20660707,"num_objects":5045,"num_object_clones":0,"num_object_copies":10090,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5045,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1762,"num_write_kb":7048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'5146","reported_seq":5277,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:36.218750+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T15:59:36.218750+0000","last_peered":"2024-09-10T15:59:36.218750+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T15:59:36.218750+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":5146,"log_dups_size":0,"ondisk_log_size":5146,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21058019,"num_objects":5142,"num_object_clones":0,"num_object_copies":10284,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5142,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1758,"num_write_kb":7032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":771},{"shards":"0,2","objects":4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'4968","reported_seq":5100,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:36.192655+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T15:59:36.192655+0000","last_peered":"2024-09-10T15:59:36.192655+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T15:59:36.192655+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":4968,"log_dups_size":0,"ondisk_log_size":4968,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20325318,"num_objects":4964,"num_object_clones":0,"num_object_copies":9928,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4964,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1467,"num_write_kb":5868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":727},{"shards":"0,2","objects":4237}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'5150","reported_seq":5272,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:36.216046+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T15:59:36.216046+0000","last_peered":"2024-09-10T15:59:36.216046+0000","last_clean":"2024-09-10T15:59:36.216046+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T15:59:36.216046+0000","last_undegraded":"2024-09-10T15:59:36.216046+0000","last_fullsized":"2024-09-10T15:59:36.216046+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":5150,"log_dups_size":0,"ondisk_log_size":5150,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21086208,"num_objects":5148,"num_object_clones":0,"num_object_copies":10296,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5148,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1997,"num_write_kb":7988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.775865+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T15:59:15.775865+0000","last_peered":"2024-09-10T15:59:15.775865+0000","last_clean":"2024-09-10T15:59:15.775865+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T15:59:15.775865+0000","last_undegraded":"2024-09-10T15:59:15.775865+0000","last_fullsized":"2024-09-10T15:59:15.775865+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'4965","reported_seq":5098,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:39.202610+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T15:59:39.202610+0000","last_peered":"2024-09-10T15:59:39.202610+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T15:59:39.202610+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":4965,"log_dups_size":0,"ondisk_log_size":4965,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20308451,"num_objects":4959,"num_object_clones":0,"num_object_copies":9918,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4959,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2784,"num_write_kb":11136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4123},{"shards":"2","objects":836}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":88,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.772572+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T15:59:15.772572+0000","last_peered":"2024-09-10T15:59:15.772572+0000","last_clean":"2024-09-10T15:59:15.772572+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T15:59:15.772572+0000","last_undegraded":"2024-09-10T15:59:15.772572+0000","last_fullsized":"2024-09-10T15:59:15.772572+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'4975","reported_seq":5100,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:36.212625+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T15:59:36.212625+0000","last_peered":"2024-09-10T15:59:36.212625+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T15:59:36.212625+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":4975,"log_dups_size":0,"ondisk_log_size":4975,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20344832,"num_objects":4967,"num_object_clones":0,"num_object_copies":9934,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4967,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1479,"num_write_kb":5916,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":712},{"shards":"0,2","objects":4255}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":98,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.880567+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T15:59:15.880567+0000","last_peered":"2024-09-10T15:59:15.880567+0000","last_clean":"2024-09-10T15:59:15.880567+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T15:59:15.880567+0000","last_undegraded":"2024-09-10T15:59:15.880567+0000","last_fullsized":"2024-09-10T15:59:15.880567+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'5136","reported_seq":5262,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:39.210194+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T15:59:39.210194+0000","last_peered":"2024-09-10T15:59:39.210194+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T15:59:39.210194+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":5136,"log_dups_size":0,"ondisk_log_size":5136,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21017059,"num_objects":5132,"num_object_clones":0,"num_object_copies":10264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5132,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2816,"num_write_kb":11264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4304},{"shards":"2","objects":828}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.837509+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T15:59:15.837509+0000","last_peered":"2024-09-10T15:59:15.837509+0000","last_clean":"2024-09-10T15:59:15.837509+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T15:59:15.837509+0000","last_undegraded":"2024-09-10T15:59:15.837509+0000","last_fullsized":"2024-09-10T15:59:15.837509+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'5224","reported_seq":5347,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:39.209897+0000","last_change":"2024-09-10T15:59:16.174113+0000","last_active":"2024-09-10T15:59:39.209897+0000","last_peered":"2024-09-10T15:59:39.209897+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T15:59:39.209897+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":5224,"log_dups_size":0,"ondisk_log_size":5224,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21372928,"num_objects":5218,"num_object_clones":0,"num_object_copies":10436,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4354,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5218,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2778,"num_write_kb":11112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4352},{"shards":"2","objects":866}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'5064","reported_seq":5193,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:36.198842+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T15:59:36.198842+0000","last_peered":"2024-09-10T15:59:36.198842+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T15:59:36.198842+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":5064,"log_dups_size":0,"ondisk_log_size":5064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20733952,"num_objects":5062,"num_object_clones":0,"num_object_copies":10124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5062,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1695,"num_write_kb":6780,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":758},{"shards":"0,2","objects":4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.989413+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T15:59:15.989413+0000","last_peered":"2024-09-10T15:59:15.989413+0000","last_clean":"2024-09-10T15:59:15.989413+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T15:59:15.989413+0000","last_undegraded":"2024-09-10T15:59:15.989413+0000","last_fullsized":"2024-09-10T15:59:15.989413+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'4960","reported_seq":5097,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:36.160282+0000","last_change":"2024-09-10T15:59:16.251610+0000","last_active":"2024-09-10T15:59:36.160282+0000","last_peered":"2024-09-10T15:59:36.160282+0000","last_clean":"2024-09-10T15:59:14.115595+0000","last_became_active":"2024-09-10T15:59:16.244089+0000","last_became_peered":"2024-09-10T15:59:16.244089+0000","last_unstale":"2024-09-10T15:59:36.160282+0000","last_undegraded":"2024-09-10T15:59:15.847127+0000","last_fullsized":"2024-09-10T15:59:15.774437+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":4960,"log_dups_size":0,"ondisk_log_size":4960,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20283392,"num_objects":4952,"num_object_clones":0,"num_object_copies":9904,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4952,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1350,"num_write_kb":5400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":749},{"shards":"0,2","objects":4203}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":88,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.773583+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T15:59:15.773583+0000","last_peered":"2024-09-10T15:59:15.773583+0000","last_clean":"2024-09-10T15:59:15.773583+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T15:59:15.773583+0000","last_undegraded":"2024-09-10T15:59:15.773583+0000","last_fullsized":"2024-09-10T15:59:15.773583+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'5160","reported_seq":5296,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:36.130907+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T15:59:36.130907+0000","last_peered":"2024-09-10T15:59:36.130907+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T15:59:36.130907+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":5160,"log_dups_size":0,"ondisk_log_size":5160,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21118976,"num_objects":5156,"num_object_clones":0,"num_object_copies":10312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5156,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1415,"num_write_kb":5660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":785},{"shards":"0,2","objects":4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.916017+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T15:59:15.916017+0000","last_peered":"2024-09-10T15:59:15.916017+0000","last_clean":"2024-09-10T15:59:15.916017+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T15:59:15.916017+0000","last_undegraded":"2024-09-10T15:59:15.916017+0000","last_fullsized":"2024-09-10T15:59:15.916017+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.776033+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T15:59:15.776033+0000","last_peered":"2024-09-10T15:59:15.776033+0000","last_clean":"2024-09-10T15:59:15.776033+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T15:59:15.776033+0000","last_undegraded":"2024-09-10T15:59:15.776033+0000","last_fullsized":"2024-09-10T15:59:15.776033+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'5112","reported_seq":5251,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:36.149258+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T15:59:36.149258+0000","last_peered":"2024-09-10T15:59:36.149258+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T15:59:36.149258+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":5112,"log_dups_size":0,"ondisk_log_size":5112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20890566,"num_objects":5102,"num_object_clones":0,"num_object_copies":10204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5102,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1275,"num_write_kb":5100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":779},{"shards":"0,2","objects":4323}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":332845790,"num_objects":81270,"num_object_clones":0,"num_object_copies":162540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":47117,"num_objects_misplaced":3786,"num_objects_unfound":0,"num_objects_dirty":81270,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":39116,"num_write_kb":156464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":485,"num_bytes_recovered":1986560,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":669552640,"data_stored":669397281,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":81342,"ondisk_log_size":81342,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986196,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":463416,"kb_used_data":301224,"kb_used_omap":4,"kb_used_meta":162171,"kb_avail":93908424,"statfs":{"total":96636764160,"available":96162226176,"internally_reserved":0,"allocated":308453376,"data_stored":308175616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":166063786},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,2,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659142,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":116228,"kb_used_data":40116,"kb_used_omap":4,"kb_used_meta":76091,"kb_avail":94255612,"statfs":{"total":96636764160,"available":96517746688,"internally_reserved":0,"allocated":41078784,"data_stored":40749978,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":77917870},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,2,3,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182424,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":464316,"kb_used_data":310508,"kb_used_omap":4,"kb_used_meta":153787,"kb_avail":93907524,"statfs":{"total":96636764160,"available":96161304576,"internally_reserved":0,"allocated":317960192,"data_stored":317678355,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":157478570},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,2,1,2,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":318156800,"data_stored":318120670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":42348544,"data_stored":42265445,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":309047296,"data_stored":309011166,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:59:41.390 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:59:41.945 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T15:59:41.946 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T15:59:41.969 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 69 20 83967 83947 4.75181 2.14062 0.0639192 0.016433 2024-09-10T15:59:41.972 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":109,"stamp":"2024-09-10T15:59:41.551062+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":340534001,"num_objects":83149,"num_object_clones":0,"num_object_copies":166298,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":47117,"num_objects_misplaced":3786,"num_objects_unfound":0,"num_objects_dirty":83149,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":40995,"num_write_kb":163974,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":487,"num_bytes_recovered":1986579,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":83222,"ondisk_log_size":83222,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1079396,"kb_used_data":660916,"kb_used_omap":12,"kb_used_meta":418419,"kb_avail":282036124,"statfs":{"total":289910292480,"available":288804990976,"internally_reserved":0,"allocated":676777984,"data_stored":675889581,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":428461058},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,1,4,5,6,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":32579584,"num_objects":7954,"num_object_clones":0,"num_object_copies":15908,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-224,"num_objects_unfound":0,"num_objects_dirty":7954,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7954,"num_write_kb":31816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":224,"num_bytes_recovered":917504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002594"},"pg_stats":[{"pgid":"2.f","version":"43'5189","reported_seq":5258,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:41.149386+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T15:59:41.149386+0000","last_peered":"2024-09-10T15:59:41.149386+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T15:59:41.149386+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":5189,"log_dups_size":0,"ondisk_log_size":5189,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21254144,"num_objects":5189,"num_object_clones":0,"num_object_copies":10378,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5189,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5189,"num_write_kb":20756,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":930},{"shards":"0,2","objects":4258}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'5153","reported_seq":5260,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:39.210025+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T15:59:39.210025+0000","last_peered":"2024-09-10T15:59:39.210025+0000","last_clean":"2024-09-10T15:59:39.210025+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T15:59:39.210025+0000","last_undegraded":"2024-09-10T15:59:39.210025+0000","last_fullsized":"2024-09-10T15:59:39.210025+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":5153,"log_dups_size":0,"ondisk_log_size":5153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21083078,"num_objects":5149,"num_object_clones":0,"num_object_copies":10298,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5149,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3258,"num_write_kb":13032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'5114","reported_seq":6202,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T15:59:39.261499+0000","last_change":"2024-09-10T15:59:16.142612+0000","last_active":"2024-09-10T15:59:39.261499+0000","last_peered":"2024-09-10T15:59:39.261499+0000","last_clean":"2024-09-10T15:59:14.092473+0000","last_became_active":"2024-09-10T15:59:16.109611+0000","last_became_peered":"2024-09-10T15:59:16.109611+0000","last_unstale":"2024-09-10T15:59:39.261499+0000","last_undegraded":"2024-09-10T15:59:39.261499+0000","last_fullsized":"2024-09-10T15:59:15.770264+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":5114,"log_dups_size":0,"ondisk_log_size":5114,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20930560,"num_objects":5110,"num_object_clones":0,"num_object_copies":10220,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3786,"num_objects_unfound":0,"num_objects_dirty":5110,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3118,"num_write_kb":12472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":482,"num_bytes_recovered":1974272,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3786},{"shards":"2","objects":1324}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'5150","reported_seq":5225,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:39.198032+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T15:59:39.198032+0000","last_peered":"2024-09-10T15:59:39.198032+0000","last_clean":"2024-09-10T15:59:39.198032+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T15:59:39.198032+0000","last_undegraded":"2024-09-10T15:59:39.198032+0000","last_fullsized":"2024-09-10T15:59:39.198032+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":5150,"log_dups_size":0,"ondisk_log_size":5150,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21094400,"num_objects":5150,"num_object_clones":0,"num_object_copies":10300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5150,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5150,"num_write_kb":20600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'5222","reported_seq":5346,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:41.224606+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T15:59:41.224606+0000","last_peered":"2024-09-10T15:59:41.224606+0000","last_clean":"2024-09-10T15:59:41.224606+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T15:59:41.224606+0000","last_undegraded":"2024-09-10T15:59:41.224606+0000","last_fullsized":"2024-09-10T15:59:41.224606+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":5222,"log_dups_size":0,"ondisk_log_size":5222,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21361123,"num_objects":5216,"num_object_clones":0,"num_object_copies":10432,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5216,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1933,"num_write_kb":7732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'5328","reported_seq":5459,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:41.232445+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T15:59:41.232445+0000","last_peered":"2024-09-10T15:59:41.232445+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T15:59:41.232445+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":5328,"log_dups_size":0,"ondisk_log_size":5328,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21803491,"num_objects":5324,"num_object_clones":0,"num_object_copies":10648,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5324,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1940,"num_write_kb":7760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":953},{"shards":"0,2","objects":4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'5166","reported_seq":5298,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:41.226762+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T15:59:41.226762+0000","last_peered":"2024-09-10T15:59:41.226762+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T15:59:41.226762+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":5166,"log_dups_size":0,"ondisk_log_size":5166,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21136326,"num_objects":5162,"num_object_clones":0,"num_object_copies":10324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5162,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1665,"num_write_kb":6660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":925},{"shards":"0,2","objects":4237}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'5345","reported_seq":5469,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:41.239615+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T15:59:41.239615+0000","last_peered":"2024-09-10T15:59:41.239615+0000","last_clean":"2024-09-10T15:59:41.239615+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T15:59:41.239615+0000","last_undegraded":"2024-09-10T15:59:41.239615+0000","last_fullsized":"2024-09-10T15:59:41.239615+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":5345,"log_dups_size":0,"ondisk_log_size":5345,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21884928,"num_objects":5343,"num_object_clones":0,"num_object_copies":10686,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5343,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2192,"num_write_kb":8768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.775865+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T15:59:15.775865+0000","last_peered":"2024-09-10T15:59:15.775865+0000","last_clean":"2024-09-10T15:59:15.775865+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T15:59:15.775865+0000","last_undegraded":"2024-09-10T15:59:15.775865+0000","last_fullsized":"2024-09-10T15:59:15.775865+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'4965","reported_seq":5098,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:39.202610+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T15:59:39.202610+0000","last_peered":"2024-09-10T15:59:39.202610+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T15:59:39.202610+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":4965,"log_dups_size":0,"ondisk_log_size":4965,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20308451,"num_objects":4959,"num_object_clones":0,"num_object_copies":9918,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4959,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2784,"num_write_kb":11136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4123},{"shards":"2","objects":836}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":88,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.772572+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T15:59:15.772572+0000","last_peered":"2024-09-10T15:59:15.772572+0000","last_clean":"2024-09-10T15:59:15.772572+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T15:59:15.772572+0000","last_undegraded":"2024-09-10T15:59:15.772572+0000","last_fullsized":"2024-09-10T15:59:15.772572+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'5177","reported_seq":5300,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:41.254520+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T15:59:41.254520+0000","last_peered":"2024-09-10T15:59:41.254520+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T15:59:41.254520+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":5177,"log_dups_size":0,"ondisk_log_size":5177,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21172224,"num_objects":5169,"num_object_clones":0,"num_object_copies":10338,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5169,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1681,"num_write_kb":6724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":914},{"shards":"0,2","objects":4255}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":98,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.880567+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T15:59:15.880567+0000","last_peered":"2024-09-10T15:59:15.880567+0000","last_clean":"2024-09-10T15:59:15.880567+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T15:59:15.880567+0000","last_undegraded":"2024-09-10T15:59:15.880567+0000","last_fullsized":"2024-09-10T15:59:15.880567+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'5136","reported_seq":5262,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:39.210194+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T15:59:39.210194+0000","last_peered":"2024-09-10T15:59:39.210194+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T15:59:39.210194+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":5136,"log_dups_size":0,"ondisk_log_size":5136,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21017059,"num_objects":5132,"num_object_clones":0,"num_object_copies":10264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5132,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2816,"num_write_kb":11264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4304},{"shards":"2","objects":828}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.837509+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T15:59:15.837509+0000","last_peered":"2024-09-10T15:59:15.837509+0000","last_clean":"2024-09-10T15:59:15.837509+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T15:59:15.837509+0000","last_undegraded":"2024-09-10T15:59:15.837509+0000","last_fullsized":"2024-09-10T15:59:15.837509+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'5224","reported_seq":5347,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:39.209897+0000","last_change":"2024-09-10T15:59:16.174113+0000","last_active":"2024-09-10T15:59:39.209897+0000","last_peered":"2024-09-10T15:59:39.209897+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T15:59:39.209897+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":5224,"log_dups_size":0,"ondisk_log_size":5224,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21372928,"num_objects":5218,"num_object_clones":0,"num_object_copies":10436,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4354,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5218,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2778,"num_write_kb":11112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4352},{"shards":"2","objects":866}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'5241","reported_seq":5370,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:41.244744+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T15:59:41.244744+0000","last_peered":"2024-09-10T15:59:41.244744+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T15:59:41.244744+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":5241,"log_dups_size":0,"ondisk_log_size":5241,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21458944,"num_objects":5239,"num_object_clones":0,"num_object_copies":10478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5239,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1872,"num_write_kb":7488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":935},{"shards":"0,2","objects":4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.989413+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T15:59:15.989413+0000","last_peered":"2024-09-10T15:59:15.989413+0000","last_clean":"2024-09-10T15:59:15.989413+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T15:59:15.989413+0000","last_undegraded":"2024-09-10T15:59:15.989413+0000","last_fullsized":"2024-09-10T15:59:15.989413+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'5156","reported_seq":5293,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:41.245040+0000","last_change":"2024-09-10T15:59:16.251610+0000","last_active":"2024-09-10T15:59:41.245040+0000","last_peered":"2024-09-10T15:59:41.245040+0000","last_clean":"2024-09-10T15:59:14.115595+0000","last_became_active":"2024-09-10T15:59:16.244089+0000","last_became_peered":"2024-09-10T15:59:16.244089+0000","last_unstale":"2024-09-10T15:59:41.245040+0000","last_undegraded":"2024-09-10T15:59:15.847127+0000","last_fullsized":"2024-09-10T15:59:15.774437+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":5156,"log_dups_size":0,"ondisk_log_size":5156,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21086208,"num_objects":5148,"num_object_clones":0,"num_object_copies":10296,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5148,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1546,"num_write_kb":6184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":945},{"shards":"0,2","objects":4203}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":88,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.773583+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T15:59:15.773583+0000","last_peered":"2024-09-10T15:59:15.773583+0000","last_clean":"2024-09-10T15:59:15.773583+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T15:59:15.773583+0000","last_undegraded":"2024-09-10T15:59:15.773583+0000","last_fullsized":"2024-09-10T15:59:15.773583+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'5338","reported_seq":5474,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:41.161957+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T15:59:41.161957+0000","last_peered":"2024-09-10T15:59:41.161957+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T15:59:41.161957+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":5338,"log_dups_size":0,"ondisk_log_size":5338,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21848064,"num_objects":5334,"num_object_clones":0,"num_object_copies":10668,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5334,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1593,"num_write_kb":6372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":963},{"shards":"0,2","objects":4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.916017+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T15:59:15.916017+0000","last_peered":"2024-09-10T15:59:15.916017+0000","last_clean":"2024-09-10T15:59:15.916017+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T15:59:15.916017+0000","last_undegraded":"2024-09-10T15:59:15.916017+0000","last_fullsized":"2024-09-10T15:59:15.916017+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.776033+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T15:59:15.776033+0000","last_peered":"2024-09-10T15:59:15.776033+0000","last_clean":"2024-09-10T15:59:15.776033+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T15:59:15.776033+0000","last_undegraded":"2024-09-10T15:59:15.776033+0000","last_fullsized":"2024-09-10T15:59:15.776033+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'5315","reported_seq":5454,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:41.175544+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T15:59:41.175544+0000","last_peered":"2024-09-10T15:59:41.175544+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T15:59:41.175544+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":5315,"log_dups_size":0,"ondisk_log_size":5315,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21722054,"num_objects":5305,"num_object_clones":0,"num_object_copies":10610,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5305,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1478,"num_write_kb":5912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":982},{"shards":"0,2","objects":4323}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":340533982,"num_objects":83147,"num_object_clones":0,"num_object_copies":166294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":47117,"num_objects_misplaced":3786,"num_objects_unfound":0,"num_objects_dirty":83147,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":40993,"num_write_kb":163972,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":485,"num_bytes_recovered":1986560,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":678797312,"data_stored":678641953,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":83219,"ondisk_log_size":83219,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986196,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":463416,"kb_used_data":301224,"kb_used_omap":4,"kb_used_meta":162171,"kb_avail":93908424,"statfs":{"total":96636764160,"available":96162226176,"internally_reserved":0,"allocated":308453376,"data_stored":308175616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":166063786},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,2,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659142,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":116228,"kb_used_data":40116,"kb_used_omap":4,"kb_used_meta":76091,"kb_avail":94255612,"statfs":{"total":96636764160,"available":96517746688,"internally_reserved":0,"allocated":41078784,"data_stored":40749978,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":77917870},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,2,3,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182425,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":499752,"kb_used_data":319576,"kb_used_omap":4,"kb_used_meta":180155,"kb_avail":93872088,"statfs":{"total":96636764160,"available":96125018112,"internally_reserved":0,"allocated":327245824,"data_stored":326963987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":184479402},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,2,1,2,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":327401472,"data_stored":327365342,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":42348544,"data_stored":42265445,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":309047296,"data_stored":309011166,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:59:41.973 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-10T15:59:41.973 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "43'5189", 'reported_seq': 5258, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:59:41.149386+0000', 'last_change': '2024-09-10T15:59:16.281558+0000', 'last_active': '2024-09-10T15:59:41.149386+0000', 'last_peered': '2024-09-10T15:59:41.149386+0000', 'last_clean': '2024-09-10T15:59:14.132456+0000', 'last_became_active': '2024-09-10T15:59:16.274407+0000', 'last_became_peered': '2024-09-10T15:59:16.274407+0000', 'last_unstale': '2024-09-10T15:59:41.149386+0000', 'last_undegraded': '2024-09-10T15:59:15.851279+0000', 'last_fullsized': '2024-09-10T15:59:15.774356+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'objects_scrubbed': 0, 'log_size': 5189, 'log_dups_size': 0, 'ondisk_log_size': 5189, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21254144, 'num_objects': 5189, 'num_object_clones': 0, 'num_object_copies': 10378, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4259, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5189, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5189, 'num_write_kb': 20756, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 930}, {'shards': '0,2', 'objects': 4258}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:59:41.974 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-10T15:59:41.974 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "43'5114", 'reported_seq': 6202, 'reported_epoch': 43, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-10T15:59:39.261499+0000', 'last_change': '2024-09-10T15:59:16.142612+0000', 'last_active': '2024-09-10T15:59:39.261499+0000', 'last_peered': '2024-09-10T15:59:39.261499+0000', 'last_clean': '2024-09-10T15:59:14.092473+0000', 'last_became_active': '2024-09-10T15:59:16.109611+0000', 'last_became_peered': '2024-09-10T15:59:16.109611+0000', 'last_unstale': '2024-09-10T15:59:39.261499+0000', 'last_undegraded': '2024-09-10T15:59:39.261499+0000', 'last_fullsized': '2024-09-10T15:59:15.770264+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2029", 'last_scrub_stamp': '2024-09-10T15:58:45.933789+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:45.933789+0000', 'objects_scrubbed': 1992, 'log_size': 5114, 'log_dups_size': 0, 'ondisk_log_size': 5114, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 668, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20930560, 'num_objects': 5110, 'num_object_clones': 0, 'num_object_copies': 10220, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 3786, 'num_objects_unfound': 0, 'num_objects_dirty': 5110, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3118, 'num_write_kb': 12472, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 482, 'num_bytes_recovered': 1974272, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3786}, {'shards': '2', 'objects': 1324}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:59:41.974 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-10T15:59:41.974 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "43'5328", 'reported_seq': 5459, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:59:41.232445+0000', 'last_change': '2024-09-10T15:59:16.308264+0000', 'last_active': '2024-09-10T15:59:41.232445+0000', 'last_peered': '2024-09-10T15:59:41.232445+0000', 'last_clean': '2024-09-10T15:59:14.111551+0000', 'last_became_active': '2024-09-10T15:59:16.300040+0000', 'last_became_peered': '2024-09-10T15:59:16.300040+0000', 'last_unstale': '2024-09-10T15:59:41.232445+0000', 'last_undegraded': '2024-09-10T15:59:15.878234+0000', 'last_fullsized': '2024-09-10T15:59:15.769328+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3442", 'last_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'objects_scrubbed': 3384, 'log_size': 5328, 'log_dups_size': 0, 'ondisk_log_size': 5328, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1292, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21803491, 'num_objects': 5324, 'num_object_clones': 0, 'num_object_copies': 10648, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4372, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5324, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 1940, 'num_write_kb': 7760, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 953}, {'shards': '0,2', 'objects': 4371}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:59:41.974 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-10T15:59:41.974 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "43'5166", 'reported_seq': 5298, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:59:41.226762+0000', 'last_change': '2024-09-10T15:59:16.331899+0000', 'last_active': '2024-09-10T15:59:41.226762+0000', 'last_peered': '2024-09-10T15:59:41.226762+0000', 'last_clean': '2024-09-10T15:59:14.139436+0000', 'last_became_active': '2024-09-10T15:59:16.318715+0000', 'last_became_peered': '2024-09-10T15:59:16.318715+0000', 'last_unstale': '2024-09-10T15:59:41.226762+0000', 'last_undegraded': '2024-09-10T15:59:15.877524+0000', 'last_fullsized': '2024-09-10T15:59:15.769042+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3537", 'last_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'objects_scrubbed': 3497, 'log_size': 5166, 'log_dups_size': 0, 'ondisk_log_size': 5166, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1224, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21136326, 'num_objects': 5162, 'num_object_clones': 0, 'num_object_copies': 10324, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4237, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5162, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 1665, 'num_write_kb': 6660, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 925}, {'shards': '0,2', 'objects': 4237}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:59:41.974 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-10T15:59:41.974 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "43'4965", 'reported_seq': 5098, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:59:39.202610+0000', 'last_change': '2024-09-10T15:59:16.175680+0000', 'last_active': '2024-09-10T15:59:39.202610+0000', 'last_peered': '2024-09-10T15:59:39.202610+0000', 'last_clean': '2024-09-10T15:59:14.088187+0000', 'last_became_active': '2024-09-10T15:59:16.165810+0000', 'last_became_peered': '2024-09-10T15:59:16.165810+0000', 'last_unstale': '2024-09-10T15:59:39.202610+0000', 'last_undegraded': '2024-09-10T15:59:15.822098+0000', 'last_fullsized': '2024-09-10T15:59:15.769123+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2218", 'last_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'objects_scrubbed': 2175, 'log_size': 4965, 'log_dups_size': 0, 'ondisk_log_size': 4965, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 825, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20308451, 'num_objects': 4959, 'num_object_clones': 0, 'num_object_copies': 9918, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4124, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4959, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 2784, 'num_write_kb': 11136, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4123}, {'shards': '2', 'objects': 836}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:59:41.974 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-10T15:59:41.974 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "43'5177", 'reported_seq': 5300, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:59:41.254520+0000', 'last_change': '2024-09-10T15:59:16.277742+0000', 'last_active': '2024-09-10T15:59:41.254520+0000', 'last_peered': '2024-09-10T15:59:41.254520+0000', 'last_clean': '2024-09-10T15:59:14.129081+0000', 'last_became_active': '2024-09-10T15:59:16.269835+0000', 'last_became_peered': '2024-09-10T15:59:16.269835+0000', 'last_unstale': '2024-09-10T15:59:41.254520+0000', 'last_undegraded': '2024-09-10T15:59:15.860857+0000', 'last_fullsized': '2024-09-10T15:59:15.771410+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3557", 'last_scrub_stamp': '2024-09-10T15:59:02.493678+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:02.493678+0000', 'objects_scrubbed': 3488, 'log_size': 5177, 'log_dups_size': 0, 'ondisk_log_size': 5177, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1472, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21172224, 'num_objects': 5169, 'num_object_clones': 0, 'num_object_copies': 10338, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4257, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5169, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1681, 'num_write_kb': 6724, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 914}, {'shards': '0,2', 'objects': 4255}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:59:41.975 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-10T15:59:41.975 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "43'5136", 'reported_seq': 5262, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:59:39.210194+0000', 'last_change': '2024-09-10T15:59:16.284311+0000', 'last_active': '2024-09-10T15:59:39.210194+0000', 'last_peered': '2024-09-10T15:59:39.210194+0000', 'last_clean': '2024-09-10T15:59:14.116041+0000', 'last_became_active': '2024-09-10T15:59:16.274543+0000', 'last_became_peered': '2024-09-10T15:59:16.274543+0000', 'last_unstale': '2024-09-10T15:59:39.210194+0000', 'last_undegraded': '2024-09-10T15:59:15.812484+0000', 'last_fullsized': '2024-09-10T15:59:15.770885+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2356", 'last_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'objects_scrubbed': 2316, 'log_size': 5136, 'log_dups_size': 0, 'ondisk_log_size': 5136, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 783, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21017059, 'num_objects': 5132, 'num_object_clones': 0, 'num_object_copies': 10264, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5132, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 2816, 'num_write_kb': 11264, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4304}, {'shards': '2', 'objects': 828}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:59:41.975 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-10T15:59:41.975 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "43'5224", 'reported_seq': 5347, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:59:39.209897+0000', 'last_change': '2024-09-10T15:59:16.174113+0000', 'last_active': '2024-09-10T15:59:39.209897+0000', 'last_peered': '2024-09-10T15:59:39.209897+0000', 'last_clean': '2024-09-10T15:59:14.110654+0000', 'last_became_active': '2024-09-10T15:59:16.164645+0000', 'last_became_peered': '2024-09-10T15:59:16.164645+0000', 'last_unstale': '2024-09-10T15:59:39.209897+0000', 'last_undegraded': '2024-09-10T15:59:15.803593+0000', 'last_fullsized': '2024-09-10T15:59:15.769083+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2486", 'last_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'objects_scrubbed': 2440, 'log_size': 5224, 'log_dups_size': 0, 'ondisk_log_size': 5224, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 792, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21372928, 'num_objects': 5218, 'num_object_clones': 0, 'num_object_copies': 10436, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4354, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5218, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2778, 'num_write_kb': 11112, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4352}, {'shards': '2', 'objects': 866}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:59:41.975 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-10T15:59:41.975 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "43'5241", 'reported_seq': 5370, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:59:41.244744+0000', 'last_change': '2024-09-10T15:59:16.315641+0000', 'last_active': '2024-09-10T15:59:41.244744+0000', 'last_peered': '2024-09-10T15:59:41.244744+0000', 'last_clean': '2024-09-10T15:59:14.117001+0000', 'last_became_active': '2024-09-10T15:59:16.302245+0000', 'last_became_peered': '2024-09-10T15:59:16.302245+0000', 'last_unstale': '2024-09-10T15:59:41.244744+0000', 'last_undegraded': '2024-09-10T15:59:15.845012+0000', 'last_fullsized': '2024-09-10T15:59:15.770337+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3404", 'last_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'objects_scrubbed': 3367, 'log_size': 5241, 'log_dups_size': 0, 'ondisk_log_size': 5241, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 983, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21458944, 'num_objects': 5239, 'num_object_clones': 0, 'num_object_copies': 10478, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5239, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1872, 'num_write_kb': 7488, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 935}, {'shards': '0,2', 'objects': 4304}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:59:41.975 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-10T15:59:41.975 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "43'5156", 'reported_seq': 5293, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:59:41.245040+0000', 'last_change': '2024-09-10T15:59:16.251610+0000', 'last_active': '2024-09-10T15:59:41.245040+0000', 'last_peered': '2024-09-10T15:59:41.245040+0000', 'last_clean': '2024-09-10T15:59:14.115595+0000', 'last_became_active': '2024-09-10T15:59:16.244089+0000', 'last_became_peered': '2024-09-10T15:59:16.244089+0000', 'last_unstale': '2024-09-10T15:59:41.245040+0000', 'last_undegraded': '2024-09-10T15:59:15.847127+0000', 'last_fullsized': '2024-09-10T15:59:15.774437+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3653", 'last_scrub_stamp': '2024-09-10T15:59:05.298468+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:05.298468+0000', 'objects_scrubbed': 3602, 'log_size': 5156, 'log_dups_size': 0, 'ondisk_log_size': 5156, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1289, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21086208, 'num_objects': 5148, 'num_object_clones': 0, 'num_object_copies': 10296, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4206, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5148, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1546, 'num_write_kb': 6184, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 945}, {'shards': '0,2', 'objects': 4203}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:59:41.975 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-10T15:59:41.975 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "43'5338", 'reported_seq': 5474, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:59:41.161957+0000', 'last_change': '2024-09-10T15:59:16.311291+0000', 'last_active': '2024-09-10T15:59:41.161957+0000', 'last_peered': '2024-09-10T15:59:41.161957+0000', 'last_clean': '2024-09-10T15:59:14.116363+0000', 'last_became_active': '2024-09-10T15:59:16.301244+0000', 'last_became_peered': '2024-09-10T15:59:16.301244+0000', 'last_unstale': '2024-09-10T15:59:41.161957+0000', 'last_undegraded': '2024-09-10T15:59:15.883434+0000', 'last_fullsized': '2024-09-10T15:59:15.772756+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3789", 'last_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'objects_scrubbed': 3741, 'log_size': 5338, 'log_dups_size': 0, 'ondisk_log_size': 5338, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1440, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21848064, 'num_objects': 5334, 'num_object_clones': 0, 'num_object_copies': 10668, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4373, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5334, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1593, 'num_write_kb': 6372, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 963}, {'shards': '0,2', 'objects': 4371}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:59:41.975 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-10T15:59:41.976 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "43'5315", 'reported_seq': 5454, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:59:41.175544+0000', 'last_change': '2024-09-10T15:59:16.313409+0000', 'last_active': '2024-09-10T15:59:41.175544+0000', 'last_peered': '2024-09-10T15:59:41.175544+0000', 'last_clean': '2024-09-10T15:59:14.129780+0000', 'last_became_active': '2024-09-10T15:59:16.300932+0000', 'last_became_peered': '2024-09-10T15:59:16.300932+0000', 'last_unstale': '2024-09-10T15:59:41.175544+0000', 'last_undegraded': '2024-09-10T15:59:15.887823+0000', 'last_fullsized': '2024-09-10T15:59:15.768513+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3880", 'last_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'objects_scrubbed': 3827, 'log_size': 5315, 'log_dups_size': 0, 'ondisk_log_size': 5315, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1397, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21722054, 'num_objects': 5305, 'num_object_clones': 0, 'num_object_copies': 10610, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4325, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5305, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1478, 'num_write_kb': 5912, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 982}, {'shards': '0,2', 'objects': 4323}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:59:41.976 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T15:59:42.671 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T15:59:42.672 INFO:teuthology.orchestra.run.smithi115.stdout:{"fsid":"7effee29-df05-4351-9741-ee203564ed81","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 47117/166298 objects degraded (28.333%), 11 pgs degraded","count":11},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":146,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":43,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983954,"num_in_osds":3,"osd_in_since":1725983947,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":83149,"data_bytes":340534001,"bytes_used":1105301504,"bytes_avail":288804990976,"bytes_total":289910292480,"degraded_objects":47117,"degraded_total":166298,"degraded_ratio":0.28332872313557589,"misplaced_objects":3786,"misplaced_total":166298,"misplaced_ratio":0.02276635918652058,"recovering_objects_per_sec":18,"recovering_bytes_per_sec":76442,"recovering_keys_per_sec":0,"num_objects_recovered":224,"num_bytes_recovered":917504,"num_keys_recovered":0,"write_bytes_sec":2714378,"read_op_per_sec":0,"write_op_per_sec":662},"fsmap":{"epoch":1,"btime":"2024-09-10T15:57:14:967120+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":5,"modified":"2024-09-10T15:59:19.546323+0000","services":{"osd":{"daemons":{"summary":"","1":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-10T15:59:42.696 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-10T15:59:42.696 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:59:42.970 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 70 20 84517 84497 4.71462 2.14844 0.0157041 0.0165621 2024-09-10T15:59:43.246 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T15:59:43.247 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T15:59:43.269 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":109,"stamp":"2024-09-10T15:59:41.551062+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":340534001,"num_objects":83149,"num_object_clones":0,"num_object_copies":166298,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":47117,"num_objects_misplaced":3786,"num_objects_unfound":0,"num_objects_dirty":83149,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":40995,"num_write_kb":163974,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":487,"num_bytes_recovered":1986579,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":83222,"ondisk_log_size":83222,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1079396,"kb_used_data":660916,"kb_used_omap":12,"kb_used_meta":418419,"kb_avail":282036124,"statfs":{"total":289910292480,"available":288804990976,"internally_reserved":0,"allocated":676777984,"data_stored":675889581,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":428461058},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,1,4,5,6,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":32579584,"num_objects":7954,"num_object_clones":0,"num_object_copies":15908,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-224,"num_objects_unfound":0,"num_objects_dirty":7954,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7954,"num_write_kb":31816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":224,"num_bytes_recovered":917504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002594"},"pg_stats":[{"pgid":"2.f","version":"43'5189","reported_seq":5258,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:41.149386+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T15:59:41.149386+0000","last_peered":"2024-09-10T15:59:41.149386+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T15:59:41.149386+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":5189,"log_dups_size":0,"ondisk_log_size":5189,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21254144,"num_objects":5189,"num_object_clones":0,"num_object_copies":10378,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5189,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5189,"num_write_kb":20756,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":930},{"shards":"0,2","objects":4258}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'5153","reported_seq":5260,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:39.210025+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T15:59:39.210025+0000","last_peered":"2024-09-10T15:59:39.210025+0000","last_clean":"2024-09-10T15:59:39.210025+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T15:59:39.210025+0000","last_undegraded":"2024-09-10T15:59:39.210025+0000","last_fullsized":"2024-09-10T15:59:39.210025+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":5153,"log_dups_size":0,"ondisk_log_size":5153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21083078,"num_objects":5149,"num_object_clones":0,"num_object_copies":10298,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5149,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3258,"num_write_kb":13032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'5114","reported_seq":6202,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T15:59:39.261499+0000","last_change":"2024-09-10T15:59:16.142612+0000","last_active":"2024-09-10T15:59:39.261499+0000","last_peered":"2024-09-10T15:59:39.261499+0000","last_clean":"2024-09-10T15:59:14.092473+0000","last_became_active":"2024-09-10T15:59:16.109611+0000","last_became_peered":"2024-09-10T15:59:16.109611+0000","last_unstale":"2024-09-10T15:59:39.261499+0000","last_undegraded":"2024-09-10T15:59:39.261499+0000","last_fullsized":"2024-09-10T15:59:15.770264+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":5114,"log_dups_size":0,"ondisk_log_size":5114,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20930560,"num_objects":5110,"num_object_clones":0,"num_object_copies":10220,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3786,"num_objects_unfound":0,"num_objects_dirty":5110,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3118,"num_write_kb":12472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":482,"num_bytes_recovered":1974272,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3786},{"shards":"2","objects":1324}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'5150","reported_seq":5225,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:39.198032+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T15:59:39.198032+0000","last_peered":"2024-09-10T15:59:39.198032+0000","last_clean":"2024-09-10T15:59:39.198032+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T15:59:39.198032+0000","last_undegraded":"2024-09-10T15:59:39.198032+0000","last_fullsized":"2024-09-10T15:59:39.198032+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":5150,"log_dups_size":0,"ondisk_log_size":5150,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21094400,"num_objects":5150,"num_object_clones":0,"num_object_copies":10300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5150,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5150,"num_write_kb":20600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'5222","reported_seq":5346,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:41.224606+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T15:59:41.224606+0000","last_peered":"2024-09-10T15:59:41.224606+0000","last_clean":"2024-09-10T15:59:41.224606+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T15:59:41.224606+0000","last_undegraded":"2024-09-10T15:59:41.224606+0000","last_fullsized":"2024-09-10T15:59:41.224606+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":5222,"log_dups_size":0,"ondisk_log_size":5222,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21361123,"num_objects":5216,"num_object_clones":0,"num_object_copies":10432,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5216,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1933,"num_write_kb":7732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'5328","reported_seq":5459,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:41.232445+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T15:59:41.232445+0000","last_peered":"2024-09-10T15:59:41.232445+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T15:59:41.232445+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":5328,"log_dups_size":0,"ondisk_log_size":5328,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21803491,"num_objects":5324,"num_object_clones":0,"num_object_copies":10648,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5324,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1940,"num_write_kb":7760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":953},{"shards":"0,2","objects":4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'5166","reported_seq":5298,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:41.226762+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T15:59:41.226762+0000","last_peered":"2024-09-10T15:59:41.226762+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T15:59:41.226762+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":5166,"log_dups_size":0,"ondisk_log_size":5166,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21136326,"num_objects":5162,"num_object_clones":0,"num_object_copies":10324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5162,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1665,"num_write_kb":6660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":925},{"shards":"0,2","objects":4237}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'5345","reported_seq":5469,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:41.239615+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T15:59:41.239615+0000","last_peered":"2024-09-10T15:59:41.239615+0000","last_clean":"2024-09-10T15:59:41.239615+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T15:59:41.239615+0000","last_undegraded":"2024-09-10T15:59:41.239615+0000","last_fullsized":"2024-09-10T15:59:41.239615+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":5345,"log_dups_size":0,"ondisk_log_size":5345,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21884928,"num_objects":5343,"num_object_clones":0,"num_object_copies":10686,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5343,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2192,"num_write_kb":8768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.775865+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T15:59:15.775865+0000","last_peered":"2024-09-10T15:59:15.775865+0000","last_clean":"2024-09-10T15:59:15.775865+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T15:59:15.775865+0000","last_undegraded":"2024-09-10T15:59:15.775865+0000","last_fullsized":"2024-09-10T15:59:15.775865+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'4965","reported_seq":5098,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:39.202610+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T15:59:39.202610+0000","last_peered":"2024-09-10T15:59:39.202610+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T15:59:39.202610+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":4965,"log_dups_size":0,"ondisk_log_size":4965,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20308451,"num_objects":4959,"num_object_clones":0,"num_object_copies":9918,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4959,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2784,"num_write_kb":11136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4123},{"shards":"2","objects":836}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":88,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.772572+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T15:59:15.772572+0000","last_peered":"2024-09-10T15:59:15.772572+0000","last_clean":"2024-09-10T15:59:15.772572+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T15:59:15.772572+0000","last_undegraded":"2024-09-10T15:59:15.772572+0000","last_fullsized":"2024-09-10T15:59:15.772572+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'5177","reported_seq":5300,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:41.254520+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T15:59:41.254520+0000","last_peered":"2024-09-10T15:59:41.254520+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T15:59:41.254520+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":5177,"log_dups_size":0,"ondisk_log_size":5177,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21172224,"num_objects":5169,"num_object_clones":0,"num_object_copies":10338,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5169,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1681,"num_write_kb":6724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":914},{"shards":"0,2","objects":4255}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":98,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.880567+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T15:59:15.880567+0000","last_peered":"2024-09-10T15:59:15.880567+0000","last_clean":"2024-09-10T15:59:15.880567+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T15:59:15.880567+0000","last_undegraded":"2024-09-10T15:59:15.880567+0000","last_fullsized":"2024-09-10T15:59:15.880567+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'5136","reported_seq":5262,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:39.210194+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T15:59:39.210194+0000","last_peered":"2024-09-10T15:59:39.210194+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T15:59:39.210194+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":5136,"log_dups_size":0,"ondisk_log_size":5136,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21017059,"num_objects":5132,"num_object_clones":0,"num_object_copies":10264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5132,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2816,"num_write_kb":11264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4304},{"shards":"2","objects":828}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.837509+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T15:59:15.837509+0000","last_peered":"2024-09-10T15:59:15.837509+0000","last_clean":"2024-09-10T15:59:15.837509+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T15:59:15.837509+0000","last_undegraded":"2024-09-10T15:59:15.837509+0000","last_fullsized":"2024-09-10T15:59:15.837509+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'5224","reported_seq":5347,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:39.209897+0000","last_change":"2024-09-10T15:59:16.174113+0000","last_active":"2024-09-10T15:59:39.209897+0000","last_peered":"2024-09-10T15:59:39.209897+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T15:59:39.209897+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":5224,"log_dups_size":0,"ondisk_log_size":5224,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21372928,"num_objects":5218,"num_object_clones":0,"num_object_copies":10436,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4354,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5218,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2778,"num_write_kb":11112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4352},{"shards":"2","objects":866}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'5241","reported_seq":5370,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:41.244744+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T15:59:41.244744+0000","last_peered":"2024-09-10T15:59:41.244744+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T15:59:41.244744+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":5241,"log_dups_size":0,"ondisk_log_size":5241,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21458944,"num_objects":5239,"num_object_clones":0,"num_object_copies":10478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5239,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1872,"num_write_kb":7488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":935},{"shards":"0,2","objects":4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.989413+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T15:59:15.989413+0000","last_peered":"2024-09-10T15:59:15.989413+0000","last_clean":"2024-09-10T15:59:15.989413+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T15:59:15.989413+0000","last_undegraded":"2024-09-10T15:59:15.989413+0000","last_fullsized":"2024-09-10T15:59:15.989413+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'5156","reported_seq":5293,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:41.245040+0000","last_change":"2024-09-10T15:59:16.251610+0000","last_active":"2024-09-10T15:59:41.245040+0000","last_peered":"2024-09-10T15:59:41.245040+0000","last_clean":"2024-09-10T15:59:14.115595+0000","last_became_active":"2024-09-10T15:59:16.244089+0000","last_became_peered":"2024-09-10T15:59:16.244089+0000","last_unstale":"2024-09-10T15:59:41.245040+0000","last_undegraded":"2024-09-10T15:59:15.847127+0000","last_fullsized":"2024-09-10T15:59:15.774437+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":5156,"log_dups_size":0,"ondisk_log_size":5156,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21086208,"num_objects":5148,"num_object_clones":0,"num_object_copies":10296,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5148,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1546,"num_write_kb":6184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":945},{"shards":"0,2","objects":4203}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":88,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.773583+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T15:59:15.773583+0000","last_peered":"2024-09-10T15:59:15.773583+0000","last_clean":"2024-09-10T15:59:15.773583+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T15:59:15.773583+0000","last_undegraded":"2024-09-10T15:59:15.773583+0000","last_fullsized":"2024-09-10T15:59:15.773583+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'5338","reported_seq":5474,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:41.161957+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T15:59:41.161957+0000","last_peered":"2024-09-10T15:59:41.161957+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T15:59:41.161957+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":5338,"log_dups_size":0,"ondisk_log_size":5338,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21848064,"num_objects":5334,"num_object_clones":0,"num_object_copies":10668,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5334,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1593,"num_write_kb":6372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":963},{"shards":"0,2","objects":4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.916017+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T15:59:15.916017+0000","last_peered":"2024-09-10T15:59:15.916017+0000","last_clean":"2024-09-10T15:59:15.916017+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T15:59:15.916017+0000","last_undegraded":"2024-09-10T15:59:15.916017+0000","last_fullsized":"2024-09-10T15:59:15.916017+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.776033+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T15:59:15.776033+0000","last_peered":"2024-09-10T15:59:15.776033+0000","last_clean":"2024-09-10T15:59:15.776033+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T15:59:15.776033+0000","last_undegraded":"2024-09-10T15:59:15.776033+0000","last_fullsized":"2024-09-10T15:59:15.776033+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'5315","reported_seq":5454,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:41.175544+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T15:59:41.175544+0000","last_peered":"2024-09-10T15:59:41.175544+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T15:59:41.175544+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":5315,"log_dups_size":0,"ondisk_log_size":5315,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21722054,"num_objects":5305,"num_object_clones":0,"num_object_copies":10610,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5305,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1478,"num_write_kb":5912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":982},{"shards":"0,2","objects":4323}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":340533982,"num_objects":83147,"num_object_clones":0,"num_object_copies":166294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":47117,"num_objects_misplaced":3786,"num_objects_unfound":0,"num_objects_dirty":83147,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":40993,"num_write_kb":163972,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":485,"num_bytes_recovered":1986560,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":678797312,"data_stored":678641953,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":83219,"ondisk_log_size":83219,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986196,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":463416,"kb_used_data":301224,"kb_used_omap":4,"kb_used_meta":162171,"kb_avail":93908424,"statfs":{"total":96636764160,"available":96162226176,"internally_reserved":0,"allocated":308453376,"data_stored":308175616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":166063786},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,2,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659142,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":116228,"kb_used_data":40116,"kb_used_omap":4,"kb_used_meta":76091,"kb_avail":94255612,"statfs":{"total":96636764160,"available":96517746688,"internally_reserved":0,"allocated":41078784,"data_stored":40749978,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":77917870},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,2,3,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182425,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":499752,"kb_used_data":319576,"kb_used_omap":4,"kb_used_meta":180155,"kb_avail":93872088,"statfs":{"total":96636764160,"available":96125018112,"internally_reserved":0,"allocated":327245824,"data_stored":326963987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":184479402},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,2,1,2,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":327401472,"data_stored":327365342,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":42348544,"data_stored":42265445,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":309047296,"data_stored":309011166,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:59:43.970 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 71 20 85112 85092 4.68095 2.32422 0.0156872 0.0166812 2024-09-10T15:59:44.970 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 72 20 85722 85702 4.64903 2.38281 0.0301641 0.0167967 2024-09-10T15:59:45.970 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 73 20 86336 86316 4.6182 2.39844 0.0270299 0.0169088 2024-09-10T15:59:46.271 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:59:46.829 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T15:59:46.830 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T15:59:46.857 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":111,"stamp":"2024-09-10T15:59:45.551957+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":345051889,"num_objects":84252,"num_object_clones":0,"num_object_copies":168504,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":47117,"num_objects_misplaced":3678,"num_objects_unfound":0,"num_objects_dirty":84252,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":42098,"num_write_kb":168386,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":595,"num_bytes_recovered":2428947,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":84325,"ondisk_log_size":84325,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1090252,"kb_used_data":675928,"kb_used_omap":12,"kb_used_meta":414259,"kb_avail":282025268,"statfs":{"total":289910292480,"available":288793874432,"internally_reserved":0,"allocated":692150272,"data_stored":691261869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":424201218},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,2,4,5,7,11],"upper_bound":128},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":28954624,"num_objects":7069,"num_object_clones":0,"num_object_copies":14138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-332,"num_objects_unfound":0,"num_objects_dirty":7069,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7069,"num_write_kb":28276,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":332,"num_bytes_recovered":1359872,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002608"},"pg_stats":[{"pgid":"2.f","version":"43'5189","reported_seq":5258,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:41.149386+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T15:59:41.149386+0000","last_peered":"2024-09-10T15:59:41.149386+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T15:59:41.149386+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":5189,"log_dups_size":0,"ondisk_log_size":5189,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21254144,"num_objects":5189,"num_object_clones":0,"num_object_copies":10378,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5189,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5189,"num_write_kb":20756,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":930},{"shards":"0,2","objects":4258}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'5343","reported_seq":5449,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:44.302637+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T15:59:44.302637+0000","last_peered":"2024-09-10T15:59:44.302637+0000","last_clean":"2024-09-10T15:59:44.302637+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T15:59:44.302637+0000","last_undegraded":"2024-09-10T15:59:44.302637+0000","last_fullsized":"2024-09-10T15:59:44.302637+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":5343,"log_dups_size":0,"ondisk_log_size":5343,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21861318,"num_objects":5339,"num_object_clones":0,"num_object_copies":10678,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5339,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3448,"num_write_kb":13792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'5287","reported_seq":6592,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T15:59:44.277656+0000","last_change":"2024-09-10T15:59:16.142612+0000","last_active":"2024-09-10T15:59:44.277656+0000","last_peered":"2024-09-10T15:59:44.277656+0000","last_clean":"2024-09-10T15:59:14.092473+0000","last_became_active":"2024-09-10T15:59:16.109611+0000","last_became_peered":"2024-09-10T15:59:16.109611+0000","last_unstale":"2024-09-10T15:59:44.277656+0000","last_undegraded":"2024-09-10T15:59:44.277656+0000","last_fullsized":"2024-09-10T15:59:15.770264+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":5287,"log_dups_size":0,"ondisk_log_size":5287,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21639168,"num_objects":5283,"num_object_clones":0,"num_object_copies":10566,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3678,"num_objects_unfound":0,"num_objects_dirty":5283,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3291,"num_write_kb":13164,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":590,"num_bytes_recovered":2416640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3678},{"shards":"2","objects":1605}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'5348","reported_seq":5423,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:44.224547+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T15:59:44.224547+0000","last_peered":"2024-09-10T15:59:44.224547+0000","last_clean":"2024-09-10T15:59:44.224547+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T15:59:44.224547+0000","last_undegraded":"2024-09-10T15:59:44.224547+0000","last_fullsized":"2024-09-10T15:59:44.224547+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":5348,"log_dups_size":0,"ondisk_log_size":5348,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21905408,"num_objects":5348,"num_object_clones":0,"num_object_copies":10696,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5348,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5348,"num_write_kb":21392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'5222","reported_seq":5346,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:41.224606+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T15:59:41.224606+0000","last_peered":"2024-09-10T15:59:41.224606+0000","last_clean":"2024-09-10T15:59:41.224606+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T15:59:41.224606+0000","last_undegraded":"2024-09-10T15:59:41.224606+0000","last_fullsized":"2024-09-10T15:59:41.224606+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":5222,"log_dups_size":0,"ondisk_log_size":5222,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21361123,"num_objects":5216,"num_object_clones":0,"num_object_copies":10432,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5216,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1933,"num_write_kb":7732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'5328","reported_seq":5459,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:41.232445+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T15:59:41.232445+0000","last_peered":"2024-09-10T15:59:41.232445+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T15:59:41.232445+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":5328,"log_dups_size":0,"ondisk_log_size":5328,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21803491,"num_objects":5324,"num_object_clones":0,"num_object_copies":10648,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5324,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1940,"num_write_kb":7760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":953},{"shards":"0,2","objects":4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'5166","reported_seq":5298,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:41.226762+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T15:59:41.226762+0000","last_peered":"2024-09-10T15:59:41.226762+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T15:59:41.226762+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":5166,"log_dups_size":0,"ondisk_log_size":5166,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21136326,"num_objects":5162,"num_object_clones":0,"num_object_copies":10324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5162,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1665,"num_write_kb":6660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":925},{"shards":"0,2","objects":4237}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'5345","reported_seq":5469,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:41.239615+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T15:59:41.239615+0000","last_peered":"2024-09-10T15:59:41.239615+0000","last_clean":"2024-09-10T15:59:41.239615+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T15:59:41.239615+0000","last_undegraded":"2024-09-10T15:59:41.239615+0000","last_fullsized":"2024-09-10T15:59:41.239615+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":5345,"log_dups_size":0,"ondisk_log_size":5345,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21884928,"num_objects":5343,"num_object_clones":0,"num_object_copies":10686,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5343,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2192,"num_write_kb":8768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.775865+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T15:59:15.775865+0000","last_peered":"2024-09-10T15:59:15.775865+0000","last_clean":"2024-09-10T15:59:15.775865+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T15:59:15.775865+0000","last_undegraded":"2024-09-10T15:59:15.775865+0000","last_fullsized":"2024-09-10T15:59:15.775865+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'5119","reported_seq":5253,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:44.215958+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T15:59:44.215958+0000","last_peered":"2024-09-10T15:59:44.215958+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T15:59:44.215958+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":5119,"log_dups_size":0,"ondisk_log_size":5119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20939235,"num_objects":5113,"num_object_clones":0,"num_object_copies":10226,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5113,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2938,"num_write_kb":11752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4123},{"shards":"2","objects":990}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":88,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.772572+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T15:59:15.772572+0000","last_peered":"2024-09-10T15:59:15.772572+0000","last_clean":"2024-09-10T15:59:15.772572+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T15:59:15.772572+0000","last_undegraded":"2024-09-10T15:59:15.772572+0000","last_fullsized":"2024-09-10T15:59:15.772572+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'5177","reported_seq":5300,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:41.254520+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T15:59:41.254520+0000","last_peered":"2024-09-10T15:59:41.254520+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T15:59:41.254520+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":5177,"log_dups_size":0,"ondisk_log_size":5177,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21172224,"num_objects":5169,"num_object_clones":0,"num_object_copies":10338,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5169,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1681,"num_write_kb":6724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":914},{"shards":"0,2","objects":4255}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":98,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.880567+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T15:59:15.880567+0000","last_peered":"2024-09-10T15:59:15.880567+0000","last_clean":"2024-09-10T15:59:15.880567+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T15:59:15.880567+0000","last_undegraded":"2024-09-10T15:59:15.880567+0000","last_fullsized":"2024-09-10T15:59:15.880567+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'5330","reported_seq":5458,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:44.280493+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T15:59:44.280493+0000","last_peered":"2024-09-10T15:59:44.280493+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T15:59:44.280493+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":5330,"log_dups_size":0,"ondisk_log_size":5330,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21811683,"num_objects":5326,"num_object_clones":0,"num_object_copies":10652,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5326,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3010,"num_write_kb":12040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4304},{"shards":"2","objects":1022}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.837509+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T15:59:15.837509+0000","last_peered":"2024-09-10T15:59:15.837509+0000","last_clean":"2024-09-10T15:59:15.837509+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T15:59:15.837509+0000","last_undegraded":"2024-09-10T15:59:15.837509+0000","last_fullsized":"2024-09-10T15:59:15.837509+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'5418","reported_seq":5541,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:44.268264+0000","last_change":"2024-09-10T15:59:16.174113+0000","last_active":"2024-09-10T15:59:44.268264+0000","last_peered":"2024-09-10T15:59:44.268264+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T15:59:44.268264+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":5418,"log_dups_size":0,"ondisk_log_size":5418,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22167552,"num_objects":5412,"num_object_clones":0,"num_object_copies":10824,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4354,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5412,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2972,"num_write_kb":11888,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4352},{"shards":"2","objects":1060}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'5241","reported_seq":5370,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:41.244744+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T15:59:41.244744+0000","last_peered":"2024-09-10T15:59:41.244744+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T15:59:41.244744+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":5241,"log_dups_size":0,"ondisk_log_size":5241,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21458944,"num_objects":5239,"num_object_clones":0,"num_object_copies":10478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5239,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1872,"num_write_kb":7488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":935},{"shards":"0,2","objects":4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.989413+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T15:59:15.989413+0000","last_peered":"2024-09-10T15:59:15.989413+0000","last_clean":"2024-09-10T15:59:15.989413+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T15:59:15.989413+0000","last_undegraded":"2024-09-10T15:59:15.989413+0000","last_fullsized":"2024-09-10T15:59:15.989413+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'5156","reported_seq":5293,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:41.245040+0000","last_change":"2024-09-10T15:59:16.251610+0000","last_active":"2024-09-10T15:59:41.245040+0000","last_peered":"2024-09-10T15:59:41.245040+0000","last_clean":"2024-09-10T15:59:14.115595+0000","last_became_active":"2024-09-10T15:59:16.244089+0000","last_became_peered":"2024-09-10T15:59:16.244089+0000","last_unstale":"2024-09-10T15:59:41.245040+0000","last_undegraded":"2024-09-10T15:59:15.847127+0000","last_fullsized":"2024-09-10T15:59:15.774437+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":5156,"log_dups_size":0,"ondisk_log_size":5156,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21086208,"num_objects":5148,"num_object_clones":0,"num_object_copies":10296,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5148,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1546,"num_write_kb":6184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":945},{"shards":"0,2","objects":4203}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":88,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.773583+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T15:59:15.773583+0000","last_peered":"2024-09-10T15:59:15.773583+0000","last_clean":"2024-09-10T15:59:15.773583+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T15:59:15.773583+0000","last_undegraded":"2024-09-10T15:59:15.773583+0000","last_fullsized":"2024-09-10T15:59:15.773583+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'5338","reported_seq":5474,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:41.161957+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T15:59:41.161957+0000","last_peered":"2024-09-10T15:59:41.161957+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T15:59:41.161957+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":5338,"log_dups_size":0,"ondisk_log_size":5338,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21848064,"num_objects":5334,"num_object_clones":0,"num_object_copies":10668,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5334,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1593,"num_write_kb":6372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":963},{"shards":"0,2","objects":4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.916017+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T15:59:15.916017+0000","last_peered":"2024-09-10T15:59:15.916017+0000","last_clean":"2024-09-10T15:59:15.916017+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T15:59:15.916017+0000","last_undegraded":"2024-09-10T15:59:15.916017+0000","last_fullsized":"2024-09-10T15:59:15.916017+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.776033+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T15:59:15.776033+0000","last_peered":"2024-09-10T15:59:15.776033+0000","last_clean":"2024-09-10T15:59:15.776033+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T15:59:15.776033+0000","last_undegraded":"2024-09-10T15:59:15.776033+0000","last_fullsized":"2024-09-10T15:59:15.776033+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'5315","reported_seq":5454,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:41.175544+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T15:59:41.175544+0000","last_peered":"2024-09-10T15:59:41.175544+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T15:59:41.175544+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":5315,"log_dups_size":0,"ondisk_log_size":5315,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21722054,"num_objects":5305,"num_object_clones":0,"num_object_copies":10610,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5305,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1478,"num_write_kb":5912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":982},{"shards":"0,2","objects":4323}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":345051870,"num_objects":84250,"num_object_clones":0,"num_object_copies":168500,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":47117,"num_objects_misplaced":3678,"num_objects_unfound":0,"num_objects_dirty":84250,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":42096,"num_write_kb":168384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":593,"num_bytes_recovered":2428928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":694263808,"data_stored":694108449,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":84322,"ondisk_log_size":84322,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986197,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":473432,"kb_used_data":307080,"kb_used_omap":4,"kb_used_meta":166331,"kb_avail":93898408,"statfs":{"total":96636764160,"available":96151969792,"internally_reserved":0,"allocated":314449920,"data_stored":314172160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":170323626},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,0,2,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659143,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":117068,"kb_used_data":49272,"kb_used_omap":4,"kb_used_meta":67771,"kb_avail":94254772,"statfs":{"total":96636764160,"available":96516886528,"internally_reserved":0,"allocated":50454528,"data_stored":50125722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":69398190},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,1,4,3,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182425,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":499752,"kb_used_data":319576,"kb_used_omap":4,"kb_used_meta":180155,"kb_avail":93872088,"statfs":{"total":96636764160,"available":96125018112,"internally_reserved":0,"allocated":327245824,"data_stored":326963987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":184479402},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,2,1,2,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":327401472,"data_stored":327365342,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":51687424,"data_stored":51604325,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":315174912,"data_stored":315138782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:59:46.859 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:59:46.970 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 74 20 86894 86874 4.58525 2.17969 0.0278462 0.0170305 2024-09-10T15:59:47.402 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T15:59:47.403 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T15:59:47.427 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":111,"stamp":"2024-09-10T15:59:45.551957+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":345051889,"num_objects":84252,"num_object_clones":0,"num_object_copies":168504,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":47117,"num_objects_misplaced":3678,"num_objects_unfound":0,"num_objects_dirty":84252,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":42098,"num_write_kb":168386,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":595,"num_bytes_recovered":2428947,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":84325,"ondisk_log_size":84325,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1090252,"kb_used_data":675928,"kb_used_omap":12,"kb_used_meta":414259,"kb_avail":282025268,"statfs":{"total":289910292480,"available":288793874432,"internally_reserved":0,"allocated":692150272,"data_stored":691261869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":424201218},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,2,4,5,7,11],"upper_bound":128},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":28954624,"num_objects":7069,"num_object_clones":0,"num_object_copies":14138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-332,"num_objects_unfound":0,"num_objects_dirty":7069,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7069,"num_write_kb":28276,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":332,"num_bytes_recovered":1359872,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002608"},"pg_stats":[{"pgid":"2.f","version":"43'5189","reported_seq":5258,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:41.149386+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T15:59:41.149386+0000","last_peered":"2024-09-10T15:59:41.149386+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T15:59:41.149386+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":5189,"log_dups_size":0,"ondisk_log_size":5189,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21254144,"num_objects":5189,"num_object_clones":0,"num_object_copies":10378,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5189,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5189,"num_write_kb":20756,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":930},{"shards":"0,2","objects":4258}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'5343","reported_seq":5449,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:44.302637+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T15:59:44.302637+0000","last_peered":"2024-09-10T15:59:44.302637+0000","last_clean":"2024-09-10T15:59:44.302637+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T15:59:44.302637+0000","last_undegraded":"2024-09-10T15:59:44.302637+0000","last_fullsized":"2024-09-10T15:59:44.302637+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":5343,"log_dups_size":0,"ondisk_log_size":5343,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21861318,"num_objects":5339,"num_object_clones":0,"num_object_copies":10678,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5339,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3448,"num_write_kb":13792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'5287","reported_seq":6592,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T15:59:44.277656+0000","last_change":"2024-09-10T15:59:16.142612+0000","last_active":"2024-09-10T15:59:44.277656+0000","last_peered":"2024-09-10T15:59:44.277656+0000","last_clean":"2024-09-10T15:59:14.092473+0000","last_became_active":"2024-09-10T15:59:16.109611+0000","last_became_peered":"2024-09-10T15:59:16.109611+0000","last_unstale":"2024-09-10T15:59:44.277656+0000","last_undegraded":"2024-09-10T15:59:44.277656+0000","last_fullsized":"2024-09-10T15:59:15.770264+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":5287,"log_dups_size":0,"ondisk_log_size":5287,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21639168,"num_objects":5283,"num_object_clones":0,"num_object_copies":10566,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3678,"num_objects_unfound":0,"num_objects_dirty":5283,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3291,"num_write_kb":13164,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":590,"num_bytes_recovered":2416640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3678},{"shards":"2","objects":1605}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'5348","reported_seq":5423,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:44.224547+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T15:59:44.224547+0000","last_peered":"2024-09-10T15:59:44.224547+0000","last_clean":"2024-09-10T15:59:44.224547+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T15:59:44.224547+0000","last_undegraded":"2024-09-10T15:59:44.224547+0000","last_fullsized":"2024-09-10T15:59:44.224547+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":5348,"log_dups_size":0,"ondisk_log_size":5348,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21905408,"num_objects":5348,"num_object_clones":0,"num_object_copies":10696,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5348,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5348,"num_write_kb":21392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'5222","reported_seq":5346,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:41.224606+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T15:59:41.224606+0000","last_peered":"2024-09-10T15:59:41.224606+0000","last_clean":"2024-09-10T15:59:41.224606+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T15:59:41.224606+0000","last_undegraded":"2024-09-10T15:59:41.224606+0000","last_fullsized":"2024-09-10T15:59:41.224606+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":5222,"log_dups_size":0,"ondisk_log_size":5222,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21361123,"num_objects":5216,"num_object_clones":0,"num_object_copies":10432,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5216,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1933,"num_write_kb":7732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'5328","reported_seq":5459,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:41.232445+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T15:59:41.232445+0000","last_peered":"2024-09-10T15:59:41.232445+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T15:59:41.232445+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":5328,"log_dups_size":0,"ondisk_log_size":5328,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21803491,"num_objects":5324,"num_object_clones":0,"num_object_copies":10648,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5324,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1940,"num_write_kb":7760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":953},{"shards":"0,2","objects":4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'5166","reported_seq":5298,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:41.226762+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T15:59:41.226762+0000","last_peered":"2024-09-10T15:59:41.226762+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T15:59:41.226762+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":5166,"log_dups_size":0,"ondisk_log_size":5166,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21136326,"num_objects":5162,"num_object_clones":0,"num_object_copies":10324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5162,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1665,"num_write_kb":6660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":925},{"shards":"0,2","objects":4237}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'5345","reported_seq":5469,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:41.239615+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T15:59:41.239615+0000","last_peered":"2024-09-10T15:59:41.239615+0000","last_clean":"2024-09-10T15:59:41.239615+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T15:59:41.239615+0000","last_undegraded":"2024-09-10T15:59:41.239615+0000","last_fullsized":"2024-09-10T15:59:41.239615+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":5345,"log_dups_size":0,"ondisk_log_size":5345,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21884928,"num_objects":5343,"num_object_clones":0,"num_object_copies":10686,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5343,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2192,"num_write_kb":8768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.775865+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T15:59:15.775865+0000","last_peered":"2024-09-10T15:59:15.775865+0000","last_clean":"2024-09-10T15:59:15.775865+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T15:59:15.775865+0000","last_undegraded":"2024-09-10T15:59:15.775865+0000","last_fullsized":"2024-09-10T15:59:15.775865+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'5119","reported_seq":5253,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:44.215958+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T15:59:44.215958+0000","last_peered":"2024-09-10T15:59:44.215958+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T15:59:44.215958+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":5119,"log_dups_size":0,"ondisk_log_size":5119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20939235,"num_objects":5113,"num_object_clones":0,"num_object_copies":10226,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5113,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2938,"num_write_kb":11752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4123},{"shards":"2","objects":990}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":88,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.772572+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T15:59:15.772572+0000","last_peered":"2024-09-10T15:59:15.772572+0000","last_clean":"2024-09-10T15:59:15.772572+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T15:59:15.772572+0000","last_undegraded":"2024-09-10T15:59:15.772572+0000","last_fullsized":"2024-09-10T15:59:15.772572+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'5177","reported_seq":5300,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:41.254520+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T15:59:41.254520+0000","last_peered":"2024-09-10T15:59:41.254520+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T15:59:41.254520+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":5177,"log_dups_size":0,"ondisk_log_size":5177,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21172224,"num_objects":5169,"num_object_clones":0,"num_object_copies":10338,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5169,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1681,"num_write_kb":6724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":914},{"shards":"0,2","objects":4255}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":98,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.880567+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T15:59:15.880567+0000","last_peered":"2024-09-10T15:59:15.880567+0000","last_clean":"2024-09-10T15:59:15.880567+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T15:59:15.880567+0000","last_undegraded":"2024-09-10T15:59:15.880567+0000","last_fullsized":"2024-09-10T15:59:15.880567+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'5330","reported_seq":5458,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:44.280493+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T15:59:44.280493+0000","last_peered":"2024-09-10T15:59:44.280493+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T15:59:44.280493+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":5330,"log_dups_size":0,"ondisk_log_size":5330,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21811683,"num_objects":5326,"num_object_clones":0,"num_object_copies":10652,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5326,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3010,"num_write_kb":12040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4304},{"shards":"2","objects":1022}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.837509+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T15:59:15.837509+0000","last_peered":"2024-09-10T15:59:15.837509+0000","last_clean":"2024-09-10T15:59:15.837509+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T15:59:15.837509+0000","last_undegraded":"2024-09-10T15:59:15.837509+0000","last_fullsized":"2024-09-10T15:59:15.837509+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'5418","reported_seq":5541,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:44.268264+0000","last_change":"2024-09-10T15:59:16.174113+0000","last_active":"2024-09-10T15:59:44.268264+0000","last_peered":"2024-09-10T15:59:44.268264+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T15:59:44.268264+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":5418,"log_dups_size":0,"ondisk_log_size":5418,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22167552,"num_objects":5412,"num_object_clones":0,"num_object_copies":10824,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4354,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5412,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2972,"num_write_kb":11888,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4352},{"shards":"2","objects":1060}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'5241","reported_seq":5370,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:41.244744+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T15:59:41.244744+0000","last_peered":"2024-09-10T15:59:41.244744+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T15:59:41.244744+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":5241,"log_dups_size":0,"ondisk_log_size":5241,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21458944,"num_objects":5239,"num_object_clones":0,"num_object_copies":10478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5239,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1872,"num_write_kb":7488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":935},{"shards":"0,2","objects":4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.989413+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T15:59:15.989413+0000","last_peered":"2024-09-10T15:59:15.989413+0000","last_clean":"2024-09-10T15:59:15.989413+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T15:59:15.989413+0000","last_undegraded":"2024-09-10T15:59:15.989413+0000","last_fullsized":"2024-09-10T15:59:15.989413+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'5156","reported_seq":5293,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:41.245040+0000","last_change":"2024-09-10T15:59:16.251610+0000","last_active":"2024-09-10T15:59:41.245040+0000","last_peered":"2024-09-10T15:59:41.245040+0000","last_clean":"2024-09-10T15:59:14.115595+0000","last_became_active":"2024-09-10T15:59:16.244089+0000","last_became_peered":"2024-09-10T15:59:16.244089+0000","last_unstale":"2024-09-10T15:59:41.245040+0000","last_undegraded":"2024-09-10T15:59:15.847127+0000","last_fullsized":"2024-09-10T15:59:15.774437+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":5156,"log_dups_size":0,"ondisk_log_size":5156,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21086208,"num_objects":5148,"num_object_clones":0,"num_object_copies":10296,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5148,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1546,"num_write_kb":6184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":945},{"shards":"0,2","objects":4203}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":88,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.773583+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T15:59:15.773583+0000","last_peered":"2024-09-10T15:59:15.773583+0000","last_clean":"2024-09-10T15:59:15.773583+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T15:59:15.773583+0000","last_undegraded":"2024-09-10T15:59:15.773583+0000","last_fullsized":"2024-09-10T15:59:15.773583+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'5338","reported_seq":5474,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:41.161957+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T15:59:41.161957+0000","last_peered":"2024-09-10T15:59:41.161957+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T15:59:41.161957+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":5338,"log_dups_size":0,"ondisk_log_size":5338,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21848064,"num_objects":5334,"num_object_clones":0,"num_object_copies":10668,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5334,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1593,"num_write_kb":6372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":963},{"shards":"0,2","objects":4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.916017+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T15:59:15.916017+0000","last_peered":"2024-09-10T15:59:15.916017+0000","last_clean":"2024-09-10T15:59:15.916017+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T15:59:15.916017+0000","last_undegraded":"2024-09-10T15:59:15.916017+0000","last_fullsized":"2024-09-10T15:59:15.916017+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.776033+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T15:59:15.776033+0000","last_peered":"2024-09-10T15:59:15.776033+0000","last_clean":"2024-09-10T15:59:15.776033+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T15:59:15.776033+0000","last_undegraded":"2024-09-10T15:59:15.776033+0000","last_fullsized":"2024-09-10T15:59:15.776033+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'5315","reported_seq":5454,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:41.175544+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T15:59:41.175544+0000","last_peered":"2024-09-10T15:59:41.175544+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T15:59:41.175544+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":5315,"log_dups_size":0,"ondisk_log_size":5315,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21722054,"num_objects":5305,"num_object_clones":0,"num_object_copies":10610,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5305,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1478,"num_write_kb":5912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":982},{"shards":"0,2","objects":4323}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":345051870,"num_objects":84250,"num_object_clones":0,"num_object_copies":168500,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":47117,"num_objects_misplaced":3678,"num_objects_unfound":0,"num_objects_dirty":84250,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":42096,"num_write_kb":168384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":593,"num_bytes_recovered":2428928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":694263808,"data_stored":694108449,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":84322,"ondisk_log_size":84322,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986197,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":473432,"kb_used_data":307080,"kb_used_omap":4,"kb_used_meta":166331,"kb_avail":93898408,"statfs":{"total":96636764160,"available":96151969792,"internally_reserved":0,"allocated":314449920,"data_stored":314172160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":170323626},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,0,2,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659143,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":117068,"kb_used_data":49272,"kb_used_omap":4,"kb_used_meta":67771,"kb_avail":94254772,"statfs":{"total":96636764160,"available":96516886528,"internally_reserved":0,"allocated":50454528,"data_stored":50125722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":69398190},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,1,4,3,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182425,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":499752,"kb_used_data":319576,"kb_used_omap":4,"kb_used_meta":180155,"kb_avail":93872088,"statfs":{"total":96636764160,"available":96125018112,"internally_reserved":0,"allocated":327245824,"data_stored":326963987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":184479402},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,2,1,2,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":327401472,"data_stored":327365342,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":51687424,"data_stored":51604325,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":315174912,"data_stored":315138782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:59:47.429 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-10T15:59:47.429 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "43'5189", 'reported_seq': 5258, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:59:41.149386+0000', 'last_change': '2024-09-10T15:59:16.281558+0000', 'last_active': '2024-09-10T15:59:41.149386+0000', 'last_peered': '2024-09-10T15:59:41.149386+0000', 'last_clean': '2024-09-10T15:59:14.132456+0000', 'last_became_active': '2024-09-10T15:59:16.274407+0000', 'last_became_peered': '2024-09-10T15:59:16.274407+0000', 'last_unstale': '2024-09-10T15:59:41.149386+0000', 'last_undegraded': '2024-09-10T15:59:15.851279+0000', 'last_fullsized': '2024-09-10T15:59:15.774356+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'objects_scrubbed': 0, 'log_size': 5189, 'log_dups_size': 0, 'ondisk_log_size': 5189, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21254144, 'num_objects': 5189, 'num_object_clones': 0, 'num_object_copies': 10378, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4259, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5189, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5189, 'num_write_kb': 20756, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 930}, {'shards': '0,2', 'objects': 4258}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:59:47.429 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-10T15:59:47.429 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "43'5287", 'reported_seq': 6592, 'reported_epoch': 43, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-10T15:59:44.277656+0000', 'last_change': '2024-09-10T15:59:16.142612+0000', 'last_active': '2024-09-10T15:59:44.277656+0000', 'last_peered': '2024-09-10T15:59:44.277656+0000', 'last_clean': '2024-09-10T15:59:14.092473+0000', 'last_became_active': '2024-09-10T15:59:16.109611+0000', 'last_became_peered': '2024-09-10T15:59:16.109611+0000', 'last_unstale': '2024-09-10T15:59:44.277656+0000', 'last_undegraded': '2024-09-10T15:59:44.277656+0000', 'last_fullsized': '2024-09-10T15:59:15.770264+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2029", 'last_scrub_stamp': '2024-09-10T15:58:45.933789+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:45.933789+0000', 'objects_scrubbed': 1992, 'log_size': 5287, 'log_dups_size': 0, 'ondisk_log_size': 5287, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 668, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21639168, 'num_objects': 5283, 'num_object_clones': 0, 'num_object_copies': 10566, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 3678, 'num_objects_unfound': 0, 'num_objects_dirty': 5283, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3291, 'num_write_kb': 13164, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 590, 'num_bytes_recovered': 2416640, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3678}, {'shards': '2', 'objects': 1605}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:59:47.429 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-10T15:59:47.429 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "43'5328", 'reported_seq': 5459, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:59:41.232445+0000', 'last_change': '2024-09-10T15:59:16.308264+0000', 'last_active': '2024-09-10T15:59:41.232445+0000', 'last_peered': '2024-09-10T15:59:41.232445+0000', 'last_clean': '2024-09-10T15:59:14.111551+0000', 'last_became_active': '2024-09-10T15:59:16.300040+0000', 'last_became_peered': '2024-09-10T15:59:16.300040+0000', 'last_unstale': '2024-09-10T15:59:41.232445+0000', 'last_undegraded': '2024-09-10T15:59:15.878234+0000', 'last_fullsized': '2024-09-10T15:59:15.769328+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3442", 'last_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'objects_scrubbed': 3384, 'log_size': 5328, 'log_dups_size': 0, 'ondisk_log_size': 5328, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1292, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21803491, 'num_objects': 5324, 'num_object_clones': 0, 'num_object_copies': 10648, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4372, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5324, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 1940, 'num_write_kb': 7760, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 953}, {'shards': '0,2', 'objects': 4371}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:59:47.429 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-10T15:59:47.429 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "43'5166", 'reported_seq': 5298, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:59:41.226762+0000', 'last_change': '2024-09-10T15:59:16.331899+0000', 'last_active': '2024-09-10T15:59:41.226762+0000', 'last_peered': '2024-09-10T15:59:41.226762+0000', 'last_clean': '2024-09-10T15:59:14.139436+0000', 'last_became_active': '2024-09-10T15:59:16.318715+0000', 'last_became_peered': '2024-09-10T15:59:16.318715+0000', 'last_unstale': '2024-09-10T15:59:41.226762+0000', 'last_undegraded': '2024-09-10T15:59:15.877524+0000', 'last_fullsized': '2024-09-10T15:59:15.769042+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3537", 'last_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'objects_scrubbed': 3497, 'log_size': 5166, 'log_dups_size': 0, 'ondisk_log_size': 5166, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1224, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21136326, 'num_objects': 5162, 'num_object_clones': 0, 'num_object_copies': 10324, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4237, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5162, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 1665, 'num_write_kb': 6660, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 925}, {'shards': '0,2', 'objects': 4237}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:59:47.430 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-10T15:59:47.430 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "43'5119", 'reported_seq': 5253, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:59:44.215958+0000', 'last_change': '2024-09-10T15:59:16.175680+0000', 'last_active': '2024-09-10T15:59:44.215958+0000', 'last_peered': '2024-09-10T15:59:44.215958+0000', 'last_clean': '2024-09-10T15:59:14.088187+0000', 'last_became_active': '2024-09-10T15:59:16.165810+0000', 'last_became_peered': '2024-09-10T15:59:16.165810+0000', 'last_unstale': '2024-09-10T15:59:44.215958+0000', 'last_undegraded': '2024-09-10T15:59:15.822098+0000', 'last_fullsized': '2024-09-10T15:59:15.769123+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2218", 'last_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'objects_scrubbed': 2175, 'log_size': 5119, 'log_dups_size': 0, 'ondisk_log_size': 5119, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 825, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20939235, 'num_objects': 5113, 'num_object_clones': 0, 'num_object_copies': 10226, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4124, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5113, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 2938, 'num_write_kb': 11752, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4123}, {'shards': '2', 'objects': 990}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:59:47.430 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-10T15:59:47.430 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "43'5177", 'reported_seq': 5300, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:59:41.254520+0000', 'last_change': '2024-09-10T15:59:16.277742+0000', 'last_active': '2024-09-10T15:59:41.254520+0000', 'last_peered': '2024-09-10T15:59:41.254520+0000', 'last_clean': '2024-09-10T15:59:14.129081+0000', 'last_became_active': '2024-09-10T15:59:16.269835+0000', 'last_became_peered': '2024-09-10T15:59:16.269835+0000', 'last_unstale': '2024-09-10T15:59:41.254520+0000', 'last_undegraded': '2024-09-10T15:59:15.860857+0000', 'last_fullsized': '2024-09-10T15:59:15.771410+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3557", 'last_scrub_stamp': '2024-09-10T15:59:02.493678+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:02.493678+0000', 'objects_scrubbed': 3488, 'log_size': 5177, 'log_dups_size': 0, 'ondisk_log_size': 5177, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1472, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21172224, 'num_objects': 5169, 'num_object_clones': 0, 'num_object_copies': 10338, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4257, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5169, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1681, 'num_write_kb': 6724, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 914}, {'shards': '0,2', 'objects': 4255}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:59:47.430 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-10T15:59:47.430 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "43'5330", 'reported_seq': 5458, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:59:44.280493+0000', 'last_change': '2024-09-10T15:59:16.284311+0000', 'last_active': '2024-09-10T15:59:44.280493+0000', 'last_peered': '2024-09-10T15:59:44.280493+0000', 'last_clean': '2024-09-10T15:59:14.116041+0000', 'last_became_active': '2024-09-10T15:59:16.274543+0000', 'last_became_peered': '2024-09-10T15:59:16.274543+0000', 'last_unstale': '2024-09-10T15:59:44.280493+0000', 'last_undegraded': '2024-09-10T15:59:15.812484+0000', 'last_fullsized': '2024-09-10T15:59:15.770885+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2356", 'last_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'objects_scrubbed': 2316, 'log_size': 5330, 'log_dups_size': 0, 'ondisk_log_size': 5330, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 783, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21811683, 'num_objects': 5326, 'num_object_clones': 0, 'num_object_copies': 10652, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5326, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 3010, 'num_write_kb': 12040, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4304}, {'shards': '2', 'objects': 1022}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:59:47.430 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-10T15:59:47.430 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "43'5418", 'reported_seq': 5541, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:59:44.268264+0000', 'last_change': '2024-09-10T15:59:16.174113+0000', 'last_active': '2024-09-10T15:59:44.268264+0000', 'last_peered': '2024-09-10T15:59:44.268264+0000', 'last_clean': '2024-09-10T15:59:14.110654+0000', 'last_became_active': '2024-09-10T15:59:16.164645+0000', 'last_became_peered': '2024-09-10T15:59:16.164645+0000', 'last_unstale': '2024-09-10T15:59:44.268264+0000', 'last_undegraded': '2024-09-10T15:59:15.803593+0000', 'last_fullsized': '2024-09-10T15:59:15.769083+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2486", 'last_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'objects_scrubbed': 2440, 'log_size': 5418, 'log_dups_size': 0, 'ondisk_log_size': 5418, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 792, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22167552, 'num_objects': 5412, 'num_object_clones': 0, 'num_object_copies': 10824, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4354, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5412, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2972, 'num_write_kb': 11888, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4352}, {'shards': '2', 'objects': 1060}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:59:47.430 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-10T15:59:47.430 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "43'5241", 'reported_seq': 5370, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:59:41.244744+0000', 'last_change': '2024-09-10T15:59:16.315641+0000', 'last_active': '2024-09-10T15:59:41.244744+0000', 'last_peered': '2024-09-10T15:59:41.244744+0000', 'last_clean': '2024-09-10T15:59:14.117001+0000', 'last_became_active': '2024-09-10T15:59:16.302245+0000', 'last_became_peered': '2024-09-10T15:59:16.302245+0000', 'last_unstale': '2024-09-10T15:59:41.244744+0000', 'last_undegraded': '2024-09-10T15:59:15.845012+0000', 'last_fullsized': '2024-09-10T15:59:15.770337+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3404", 'last_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'objects_scrubbed': 3367, 'log_size': 5241, 'log_dups_size': 0, 'ondisk_log_size': 5241, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 983, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21458944, 'num_objects': 5239, 'num_object_clones': 0, 'num_object_copies': 10478, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5239, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1872, 'num_write_kb': 7488, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 935}, {'shards': '0,2', 'objects': 4304}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:59:47.431 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-10T15:59:47.431 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "43'5156", 'reported_seq': 5293, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:59:41.245040+0000', 'last_change': '2024-09-10T15:59:16.251610+0000', 'last_active': '2024-09-10T15:59:41.245040+0000', 'last_peered': '2024-09-10T15:59:41.245040+0000', 'last_clean': '2024-09-10T15:59:14.115595+0000', 'last_became_active': '2024-09-10T15:59:16.244089+0000', 'last_became_peered': '2024-09-10T15:59:16.244089+0000', 'last_unstale': '2024-09-10T15:59:41.245040+0000', 'last_undegraded': '2024-09-10T15:59:15.847127+0000', 'last_fullsized': '2024-09-10T15:59:15.774437+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3653", 'last_scrub_stamp': '2024-09-10T15:59:05.298468+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:05.298468+0000', 'objects_scrubbed': 3602, 'log_size': 5156, 'log_dups_size': 0, 'ondisk_log_size': 5156, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1289, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21086208, 'num_objects': 5148, 'num_object_clones': 0, 'num_object_copies': 10296, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4206, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5148, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1546, 'num_write_kb': 6184, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 945}, {'shards': '0,2', 'objects': 4203}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:59:47.431 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-10T15:59:47.431 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "43'5338", 'reported_seq': 5474, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:59:41.161957+0000', 'last_change': '2024-09-10T15:59:16.311291+0000', 'last_active': '2024-09-10T15:59:41.161957+0000', 'last_peered': '2024-09-10T15:59:41.161957+0000', 'last_clean': '2024-09-10T15:59:14.116363+0000', 'last_became_active': '2024-09-10T15:59:16.301244+0000', 'last_became_peered': '2024-09-10T15:59:16.301244+0000', 'last_unstale': '2024-09-10T15:59:41.161957+0000', 'last_undegraded': '2024-09-10T15:59:15.883434+0000', 'last_fullsized': '2024-09-10T15:59:15.772756+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3789", 'last_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'objects_scrubbed': 3741, 'log_size': 5338, 'log_dups_size': 0, 'ondisk_log_size': 5338, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1440, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21848064, 'num_objects': 5334, 'num_object_clones': 0, 'num_object_copies': 10668, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4373, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5334, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1593, 'num_write_kb': 6372, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 963}, {'shards': '0,2', 'objects': 4371}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:59:47.431 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-10T15:59:47.431 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "43'5315", 'reported_seq': 5454, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:59:41.175544+0000', 'last_change': '2024-09-10T15:59:16.313409+0000', 'last_active': '2024-09-10T15:59:41.175544+0000', 'last_peered': '2024-09-10T15:59:41.175544+0000', 'last_clean': '2024-09-10T15:59:14.129780+0000', 'last_became_active': '2024-09-10T15:59:16.300932+0000', 'last_became_peered': '2024-09-10T15:59:16.300932+0000', 'last_unstale': '2024-09-10T15:59:41.175544+0000', 'last_undegraded': '2024-09-10T15:59:15.887823+0000', 'last_fullsized': '2024-09-10T15:59:15.768513+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3880", 'last_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'objects_scrubbed': 3827, 'log_size': 5315, 'log_dups_size': 0, 'ondisk_log_size': 5315, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1397, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21722054, 'num_objects': 5305, 'num_object_clones': 0, 'num_object_copies': 10610, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4325, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5305, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1478, 'num_write_kb': 5912, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 982}, {'shards': '0,2', 'objects': 4323}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:59:47.431 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T15:59:47.971 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 75 20 87419 87399 4.55145 2.05078 0.0288674 0.0171556 2024-09-10T15:59:48.120 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T15:59:48.120 INFO:teuthology.orchestra.run.smithi115.stdout:{"fsid":"7effee29-df05-4351-9741-ee203564ed81","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 47117/172174 objects degraded (27.366%), 11 pgs degraded","count":11},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":152,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":43,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983954,"num_in_osds":3,"osd_in_since":1725983947,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":86087,"data_bytes":352568049,"bytes_used":1129656320,"bytes_avail":288780636160,"bytes_total":289910292480,"degraded_objects":47117,"degraded_total":172174,"degraded_ratio":0.2736592052226236,"misplaced_objects":3678,"misplaced_total":172174,"misplaced_ratio":0.021362110423176553,"recovering_objects_per_sec":18,"recovering_bytes_per_sec":75416,"recovering_keys_per_sec":0,"num_objects_recovered":221,"num_bytes_recovered":905216,"num_keys_recovered":0,"write_bytes_sec":2661424,"read_op_per_sec":0,"write_op_per_sec":649},"fsmap":{"epoch":1,"btime":"2024-09-10T15:57:14:967120+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":5,"modified":"2024-09-10T15:59:19.546323+0000","services":{"osd":{"daemons":{"summary":"","1":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-10T15:59:48.145 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-10T15:59:48.146 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:59:48.727 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T15:59:48.729 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T15:59:48.756 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":112,"stamp":"2024-09-10T15:59:47.552537+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":352568049,"num_objects":86087,"num_object_clones":0,"num_object_copies":172174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":47117,"num_objects_misplaced":3678,"num_objects_unfound":0,"num_objects_dirty":86087,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":43933,"num_write_kb":175726,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":595,"num_bytes_recovered":2428947,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":86160,"ondisk_log_size":86160,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1103180,"kb_used_data":684696,"kb_used_omap":12,"kb_used_meta":418419,"kb_avail":282012340,"statfs":{"total":289910292480,"available":288780636160,"internally_reserved":0,"allocated":701128704,"data_stored":700240301,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":428461058},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,2,4,7,11,9],"upper_bound":128},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":31944704,"num_objects":7799,"num_object_clones":0,"num_object_copies":15598,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-221,"num_objects_unfound":0,"num_objects_dirty":7799,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7799,"num_write_kb":31196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":221,"num_bytes_recovered":905216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002861"},"pg_stats":[{"pgid":"2.f","version":"43'5381","reported_seq":5449,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:46.231418+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T15:59:46.231418+0000","last_peered":"2024-09-10T15:59:46.231418+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T15:59:46.231418+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":5381,"log_dups_size":0,"ondisk_log_size":5381,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22040576,"num_objects":5381,"num_object_clones":0,"num_object_copies":10762,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5381,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5381,"num_write_kb":21524,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1122},{"shards":"0,2","objects":4258}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'5343","reported_seq":5449,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:44.302637+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T15:59:44.302637+0000","last_peered":"2024-09-10T15:59:44.302637+0000","last_clean":"2024-09-10T15:59:44.302637+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T15:59:44.302637+0000","last_undegraded":"2024-09-10T15:59:44.302637+0000","last_fullsized":"2024-09-10T15:59:44.302637+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":5343,"log_dups_size":0,"ondisk_log_size":5343,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21861318,"num_objects":5339,"num_object_clones":0,"num_object_copies":10678,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5339,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3448,"num_write_kb":13792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'5287","reported_seq":6592,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T15:59:44.277656+0000","last_change":"2024-09-10T15:59:16.142612+0000","last_active":"2024-09-10T15:59:44.277656+0000","last_peered":"2024-09-10T15:59:44.277656+0000","last_clean":"2024-09-10T15:59:14.092473+0000","last_became_active":"2024-09-10T15:59:16.109611+0000","last_became_peered":"2024-09-10T15:59:16.109611+0000","last_unstale":"2024-09-10T15:59:44.277656+0000","last_undegraded":"2024-09-10T15:59:44.277656+0000","last_fullsized":"2024-09-10T15:59:15.770264+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":5287,"log_dups_size":0,"ondisk_log_size":5287,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21639168,"num_objects":5283,"num_object_clones":0,"num_object_copies":10566,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3678,"num_objects_unfound":0,"num_objects_dirty":5283,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3291,"num_write_kb":13164,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":590,"num_bytes_recovered":2416640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3678},{"shards":"2","objects":1605}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'5348","reported_seq":5423,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:44.224547+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T15:59:44.224547+0000","last_peered":"2024-09-10T15:59:44.224547+0000","last_clean":"2024-09-10T15:59:44.224547+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T15:59:44.224547+0000","last_undegraded":"2024-09-10T15:59:44.224547+0000","last_fullsized":"2024-09-10T15:59:44.224547+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":5348,"log_dups_size":0,"ondisk_log_size":5348,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21905408,"num_objects":5348,"num_object_clones":0,"num_object_copies":10696,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5348,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5348,"num_write_kb":21392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'5427","reported_seq":5551,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:46.234706+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T15:59:46.234706+0000","last_peered":"2024-09-10T15:59:46.234706+0000","last_clean":"2024-09-10T15:59:46.234706+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T15:59:46.234706+0000","last_undegraded":"2024-09-10T15:59:46.234706+0000","last_fullsized":"2024-09-10T15:59:46.234706+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":5427,"log_dups_size":0,"ondisk_log_size":5427,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22200803,"num_objects":5421,"num_object_clones":0,"num_object_copies":10842,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5421,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2138,"num_write_kb":8552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'5513","reported_seq":5645,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:46.250945+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T15:59:46.250945+0000","last_peered":"2024-09-10T15:59:46.250945+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T15:59:46.250945+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":5513,"log_dups_size":0,"ondisk_log_size":5513,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22561251,"num_objects":5509,"num_object_clones":0,"num_object_copies":11018,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5509,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2125,"num_write_kb":8500,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1138},{"shards":"0,2","objects":4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'5343","reported_seq":5474,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:46.188805+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T15:59:46.188805+0000","last_peered":"2024-09-10T15:59:46.188805+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T15:59:46.188805+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":5343,"log_dups_size":0,"ondisk_log_size":5343,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21861318,"num_objects":5339,"num_object_clones":0,"num_object_copies":10678,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5339,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1842,"num_write_kb":7368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1102},{"shards":"0,2","objects":4237}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'5526","reported_seq":5650,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:46.202629+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T15:59:46.202629+0000","last_peered":"2024-09-10T15:59:46.202629+0000","last_clean":"2024-09-10T15:59:46.202629+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T15:59:46.202629+0000","last_undegraded":"2024-09-10T15:59:46.202629+0000","last_fullsized":"2024-09-10T15:59:46.202629+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":5526,"log_dups_size":0,"ondisk_log_size":5526,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22626304,"num_objects":5524,"num_object_clones":0,"num_object_copies":11048,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5524,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2373,"num_write_kb":9492,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.775865+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T15:59:15.775865+0000","last_peered":"2024-09-10T15:59:15.775865+0000","last_clean":"2024-09-10T15:59:15.775865+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T15:59:15.775865+0000","last_undegraded":"2024-09-10T15:59:15.775865+0000","last_fullsized":"2024-09-10T15:59:15.775865+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'5119","reported_seq":5253,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:44.215958+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T15:59:44.215958+0000","last_peered":"2024-09-10T15:59:44.215958+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T15:59:44.215958+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":5119,"log_dups_size":0,"ondisk_log_size":5119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20939235,"num_objects":5113,"num_object_clones":0,"num_object_copies":10226,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5113,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2938,"num_write_kb":11752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4123},{"shards":"2","objects":990}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":88,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.772572+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T15:59:15.772572+0000","last_peered":"2024-09-10T15:59:15.772572+0000","last_clean":"2024-09-10T15:59:15.772572+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T15:59:15.772572+0000","last_undegraded":"2024-09-10T15:59:15.772572+0000","last_fullsized":"2024-09-10T15:59:15.772572+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'5359","reported_seq":5484,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:46.265001+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T15:59:46.265001+0000","last_peered":"2024-09-10T15:59:46.265001+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T15:59:46.265001+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":5359,"log_dups_size":0,"ondisk_log_size":5359,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21917696,"num_objects":5351,"num_object_clones":0,"num_object_copies":10702,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5351,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1863,"num_write_kb":7452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1096},{"shards":"0,2","objects":4255}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":98,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.880567+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T15:59:15.880567+0000","last_peered":"2024-09-10T15:59:15.880567+0000","last_clean":"2024-09-10T15:59:15.880567+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T15:59:15.880567+0000","last_undegraded":"2024-09-10T15:59:15.880567+0000","last_fullsized":"2024-09-10T15:59:15.880567+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'5330","reported_seq":5458,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:44.280493+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T15:59:44.280493+0000","last_peered":"2024-09-10T15:59:44.280493+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T15:59:44.280493+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":5330,"log_dups_size":0,"ondisk_log_size":5330,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21811683,"num_objects":5326,"num_object_clones":0,"num_object_copies":10652,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5326,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3010,"num_write_kb":12040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4304},{"shards":"2","objects":1022}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.837509+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T15:59:15.837509+0000","last_peered":"2024-09-10T15:59:15.837509+0000","last_clean":"2024-09-10T15:59:15.837509+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T15:59:15.837509+0000","last_undegraded":"2024-09-10T15:59:15.837509+0000","last_fullsized":"2024-09-10T15:59:15.837509+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'5418","reported_seq":5541,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:44.268264+0000","last_change":"2024-09-10T15:59:16.174113+0000","last_active":"2024-09-10T15:59:44.268264+0000","last_peered":"2024-09-10T15:59:44.268264+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T15:59:44.268264+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":5418,"log_dups_size":0,"ondisk_log_size":5418,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22167552,"num_objects":5412,"num_object_clones":0,"num_object_copies":10824,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4354,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5412,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2972,"num_write_kb":11888,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4352},{"shards":"2","objects":1060}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'5414","reported_seq":5543,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:46.226559+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T15:59:46.226559+0000","last_peered":"2024-09-10T15:59:46.226559+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T15:59:46.226559+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":5414,"log_dups_size":0,"ondisk_log_size":5414,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22167552,"num_objects":5412,"num_object_clones":0,"num_object_copies":10824,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5412,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2045,"num_write_kb":8180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1108},{"shards":"0,2","objects":4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.989413+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T15:59:15.989413+0000","last_peered":"2024-09-10T15:59:15.989413+0000","last_clean":"2024-09-10T15:59:15.989413+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T15:59:15.989413+0000","last_undegraded":"2024-09-10T15:59:15.989413+0000","last_fullsized":"2024-09-10T15:59:15.989413+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'5324","reported_seq":5460,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:46.280567+0000","last_change":"2024-09-10T15:59:16.251610+0000","last_active":"2024-09-10T15:59:46.280567+0000","last_peered":"2024-09-10T15:59:46.280567+0000","last_clean":"2024-09-10T15:59:14.115595+0000","last_became_active":"2024-09-10T15:59:16.244089+0000","last_became_peered":"2024-09-10T15:59:16.244089+0000","last_unstale":"2024-09-10T15:59:46.280567+0000","last_undegraded":"2024-09-10T15:59:15.847127+0000","last_fullsized":"2024-09-10T15:59:15.774437+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":5324,"log_dups_size":0,"ondisk_log_size":5324,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21774336,"num_objects":5316,"num_object_clones":0,"num_object_copies":10632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5316,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1714,"num_write_kb":6856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1113},{"shards":"0,2","objects":4203}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":88,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.773583+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T15:59:15.773583+0000","last_peered":"2024-09-10T15:59:15.773583+0000","last_clean":"2024-09-10T15:59:15.773583+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T15:59:15.773583+0000","last_undegraded":"2024-09-10T15:59:15.773583+0000","last_fullsized":"2024-09-10T15:59:15.773583+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'5528","reported_seq":5664,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:46.271714+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T15:59:46.271714+0000","last_peered":"2024-09-10T15:59:46.271714+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T15:59:46.271714+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":5528,"log_dups_size":0,"ondisk_log_size":5528,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22626304,"num_objects":5524,"num_object_clones":0,"num_object_copies":11048,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5524,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1783,"num_write_kb":7132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1153},{"shards":"0,2","objects":4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.916017+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T15:59:15.916017+0000","last_peered":"2024-09-10T15:59:15.916017+0000","last_clean":"2024-09-10T15:59:15.916017+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T15:59:15.916017+0000","last_undegraded":"2024-09-10T15:59:15.916017+0000","last_fullsized":"2024-09-10T15:59:15.916017+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.776033+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T15:59:15.776033+0000","last_peered":"2024-09-10T15:59:15.776033+0000","last_clean":"2024-09-10T15:59:15.776033+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T15:59:15.776033+0000","last_undegraded":"2024-09-10T15:59:15.776033+0000","last_fullsized":"2024-09-10T15:59:15.776033+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'5497","reported_seq":5636,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:46.271945+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T15:59:46.271945+0000","last_peered":"2024-09-10T15:59:46.271945+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T15:59:46.271945+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":5497,"log_dups_size":0,"ondisk_log_size":5497,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22467526,"num_objects":5487,"num_object_clones":0,"num_object_copies":10974,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5487,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1660,"num_write_kb":6640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1164},{"shards":"0,2","objects":4323}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":352568030,"num_objects":86085,"num_object_clones":0,"num_object_copies":172170,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":47117,"num_objects_misplaced":3678,"num_objects_unfound":0,"num_objects_dirty":86085,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":43931,"num_write_kb":175724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":593,"num_bytes_recovered":2428928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":703307776,"data_stored":703152417,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":86157,"ondisk_log_size":86157,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986197,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":473432,"kb_used_data":307080,"kb_used_omap":4,"kb_used_meta":166331,"kb_avail":93898408,"statfs":{"total":96636764160,"available":96151969792,"internally_reserved":0,"allocated":314449920,"data_stored":314172160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":170323626},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,0,2,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659143,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":117068,"kb_used_data":49272,"kb_used_omap":4,"kb_used_meta":67771,"kb_avail":94254772,"statfs":{"total":96636764160,"available":96516886528,"internally_reserved":0,"allocated":50454528,"data_stored":50125722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":69398190},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,1,4,3,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182426,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":512680,"kb_used_data":328344,"kb_used_omap":4,"kb_used_meta":184315,"kb_avail":93859160,"statfs":{"total":96636764160,"available":96111779840,"internally_reserved":0,"allocated":336224256,"data_stored":335942419,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":188739242},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,2,3,6,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":336445440,"data_stored":336409310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":51687424,"data_stored":51604325,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":315174912,"data_stored":315138782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:59:48.972 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 76 20 87978 87958 4.52019 2.18359 0.0555038 0.0172749 2024-09-10T15:59:49.972 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 77 20 88576 88556 4.49182 2.33594 0.0248333 0.0173848 2024-09-10T15:59:50.972 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 78 20 89174 89154 4.46418 2.33594 0.0419426 0.017494 2024-09-10T15:59:51.758 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:59:51.972 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 79 20 89768 89748 4.43705 2.32031 0.0406234 0.0175992 2024-09-10T15:59:52.353 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T15:59:52.355 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T15:59:52.377 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":114,"stamp":"2024-09-10T15:59:51.553729+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":364200689,"num_objects":88927,"num_object_clones":0,"num_object_copies":177854,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":47117,"num_objects_misplaced":3563,"num_objects_unfound":0,"num_objects_dirty":88927,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":46773,"num_write_kb":187086,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":710,"num_bytes_recovered":2899987,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":89000,"ondisk_log_size":89000,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1111676,"kb_used_data":709836,"kb_used_omap":12,"kb_used_meta":401779,"kb_avail":282003844,"statfs":{"total":289910292480,"available":288771936256,"internally_reserved":0,"allocated":726872064,"data_stored":725983661,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":411421698},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,7,4,7,15,4,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":31354880,"num_objects":7655,"num_object_clones":0,"num_object_copies":15310,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-223,"num_objects_unfound":0,"num_objects_dirty":7655,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7655,"num_write_kb":30620,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":223,"num_bytes_recovered":913408,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003124"},"pg_stats":[{"pgid":"2.f","version":"43'5546","reported_seq":5615,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:51.278437+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T15:59:51.278437+0000","last_peered":"2024-09-10T15:59:51.278437+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T15:59:51.278437+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":5546,"log_dups_size":0,"ondisk_log_size":5546,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22716416,"num_objects":5546,"num_object_clones":0,"num_object_copies":11092,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5546,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5546,"num_write_kb":22184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1287},{"shards":"0,2","objects":4258}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'5517","reported_seq":5624,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:49.303314+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T15:59:49.303314+0000","last_peered":"2024-09-10T15:59:49.303314+0000","last_clean":"2024-09-10T15:59:49.303314+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T15:59:49.303314+0000","last_undegraded":"2024-09-10T15:59:49.303314+0000","last_fullsized":"2024-09-10T15:59:49.303314+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":5517,"log_dups_size":0,"ondisk_log_size":5517,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22574022,"num_objects":5513,"num_object_clones":0,"num_object_copies":11026,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5513,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3622,"num_write_kb":14488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'5456","reported_seq":6991,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T15:59:49.292181+0000","last_change":"2024-09-10T15:59:16.142612+0000","last_active":"2024-09-10T15:59:49.292181+0000","last_peered":"2024-09-10T15:59:49.292181+0000","last_clean":"2024-09-10T15:59:14.092473+0000","last_became_active":"2024-09-10T15:59:16.109611+0000","last_became_peered":"2024-09-10T15:59:16.109611+0000","last_unstale":"2024-09-10T15:59:49.292181+0000","last_undegraded":"2024-09-10T15:59:49.292181+0000","last_fullsized":"2024-09-10T15:59:15.770264+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":5456,"log_dups_size":0,"ondisk_log_size":5456,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22331392,"num_objects":5452,"num_object_clones":0,"num_object_copies":10904,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3563,"num_objects_unfound":0,"num_objects_dirty":5452,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3460,"num_write_kb":13840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":705,"num_bytes_recovered":2887680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3563},{"shards":"2","objects":1889}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'5510","reported_seq":5585,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:49.195846+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T15:59:49.195846+0000","last_peered":"2024-09-10T15:59:49.195846+0000","last_clean":"2024-09-10T15:59:49.195846+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T15:59:49.195846+0000","last_undegraded":"2024-09-10T15:59:49.195846+0000","last_fullsized":"2024-09-10T15:59:49.195846+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":5510,"log_dups_size":0,"ondisk_log_size":5510,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22568960,"num_objects":5510,"num_object_clones":0,"num_object_copies":11020,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5510,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5510,"num_write_kb":22040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'5612","reported_seq":5736,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:51.249929+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T15:59:51.249929+0000","last_peered":"2024-09-10T15:59:51.249929+0000","last_clean":"2024-09-10T15:59:51.249929+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T15:59:51.249929+0000","last_undegraded":"2024-09-10T15:59:51.249929+0000","last_fullsized":"2024-09-10T15:59:51.249929+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":5612,"log_dups_size":0,"ondisk_log_size":5612,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22958563,"num_objects":5606,"num_object_clones":0,"num_object_copies":11212,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5606,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2323,"num_write_kb":9292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'5685","reported_seq":5817,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:51.214373+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T15:59:51.214373+0000","last_peered":"2024-09-10T15:59:51.214373+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T15:59:51.214373+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":5685,"log_dups_size":0,"ondisk_log_size":5685,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23265763,"num_objects":5681,"num_object_clones":0,"num_object_copies":11362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5681,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2297,"num_write_kb":9188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1310},{"shards":"0,2","objects":4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'5524","reported_seq":5656,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:51.246244+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T15:59:51.246244+0000","last_peered":"2024-09-10T15:59:51.246244+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T15:59:51.246244+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":5524,"log_dups_size":0,"ondisk_log_size":5524,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22602694,"num_objects":5520,"num_object_clones":0,"num_object_copies":11040,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5520,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":2023,"num_write_kb":8092,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1283},{"shards":"0,2","objects":4237}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'5711","reported_seq":5835,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:51.290418+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T15:59:51.290418+0000","last_peered":"2024-09-10T15:59:51.290418+0000","last_clean":"2024-09-10T15:59:51.290418+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T15:59:51.290418+0000","last_undegraded":"2024-09-10T15:59:51.290418+0000","last_fullsized":"2024-09-10T15:59:51.290418+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":5711,"log_dups_size":0,"ondisk_log_size":5711,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23384064,"num_objects":5709,"num_object_clones":0,"num_object_copies":11418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5709,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2558,"num_write_kb":10232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.775865+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T15:59:15.775865+0000","last_peered":"2024-09-10T15:59:15.775865+0000","last_clean":"2024-09-10T15:59:15.775865+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T15:59:15.775865+0000","last_undegraded":"2024-09-10T15:59:15.775865+0000","last_fullsized":"2024-09-10T15:59:15.775865+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'5300","reported_seq":5434,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:49.273062+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T15:59:49.273062+0000","last_peered":"2024-09-10T15:59:49.273062+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T15:59:49.273062+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":5300,"log_dups_size":0,"ondisk_log_size":5300,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21680611,"num_objects":5294,"num_object_clones":0,"num_object_copies":10588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5294,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3119,"num_write_kb":12476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4123},{"shards":"2","objects":1171}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":88,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.772572+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T15:59:15.772572+0000","last_peered":"2024-09-10T15:59:15.772572+0000","last_clean":"2024-09-10T15:59:15.772572+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T15:59:15.772572+0000","last_undegraded":"2024-09-10T15:59:15.772572+0000","last_fullsized":"2024-09-10T15:59:15.772572+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'5532","reported_seq":5655,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:51.284232+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T15:59:51.284232+0000","last_peered":"2024-09-10T15:59:51.284232+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T15:59:51.284232+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":5532,"log_dups_size":0,"ondisk_log_size":5532,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22626304,"num_objects":5524,"num_object_clones":0,"num_object_copies":11048,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5524,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2036,"num_write_kb":8144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1269},{"shards":"0,2","objects":4255}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":98,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.880567+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T15:59:15.880567+0000","last_peered":"2024-09-10T15:59:15.880567+0000","last_clean":"2024-09-10T15:59:15.880567+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T15:59:15.880567+0000","last_undegraded":"2024-09-10T15:59:15.880567+0000","last_fullsized":"2024-09-10T15:59:15.880567+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'5508","reported_seq":5637,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:49.284201+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T15:59:49.284201+0000","last_peered":"2024-09-10T15:59:49.284201+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T15:59:49.284201+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":5508,"log_dups_size":0,"ondisk_log_size":5508,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22540771,"num_objects":5504,"num_object_clones":0,"num_object_copies":11008,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5504,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3188,"num_write_kb":12752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4304},{"shards":"2","objects":1200}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.837509+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T15:59:15.837509+0000","last_peered":"2024-09-10T15:59:15.837509+0000","last_clean":"2024-09-10T15:59:15.837509+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T15:59:15.837509+0000","last_undegraded":"2024-09-10T15:59:15.837509+0000","last_fullsized":"2024-09-10T15:59:15.837509+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'5594","reported_seq":5717,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:49.183178+0000","last_change":"2024-09-10T15:59:16.174113+0000","last_active":"2024-09-10T15:59:49.183178+0000","last_peered":"2024-09-10T15:59:49.183178+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T15:59:49.183178+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":5594,"log_dups_size":0,"ondisk_log_size":5594,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22888448,"num_objects":5588,"num_object_clones":0,"num_object_copies":11176,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4354,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5588,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3148,"num_write_kb":12592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4352},{"shards":"2","objects":1236}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'5588","reported_seq":5717,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:51.285012+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T15:59:51.285012+0000","last_peered":"2024-09-10T15:59:51.285012+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T15:59:51.285012+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":5588,"log_dups_size":0,"ondisk_log_size":5588,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22880256,"num_objects":5586,"num_object_clones":0,"num_object_copies":11172,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5586,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2219,"num_write_kb":8876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1282},{"shards":"0,2","objects":4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.989413+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T15:59:15.989413+0000","last_peered":"2024-09-10T15:59:15.989413+0000","last_clean":"2024-09-10T15:59:15.989413+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T15:59:15.989413+0000","last_undegraded":"2024-09-10T15:59:15.989413+0000","last_fullsized":"2024-09-10T15:59:15.989413+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'5504","reported_seq":5641,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:51.219551+0000","last_change":"2024-09-10T15:59:16.251610+0000","last_active":"2024-09-10T15:59:51.219551+0000","last_peered":"2024-09-10T15:59:51.219551+0000","last_clean":"2024-09-10T15:59:14.115595+0000","last_became_active":"2024-09-10T15:59:16.244089+0000","last_became_peered":"2024-09-10T15:59:16.244089+0000","last_unstale":"2024-09-10T15:59:51.219551+0000","last_undegraded":"2024-09-10T15:59:15.847127+0000","last_fullsized":"2024-09-10T15:59:15.774437+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":5504,"log_dups_size":0,"ondisk_log_size":5504,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22511616,"num_objects":5496,"num_object_clones":0,"num_object_copies":10992,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5496,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1894,"num_write_kb":7576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1293},{"shards":"0,2","objects":4203}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":88,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.773583+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T15:59:15.773583+0000","last_peered":"2024-09-10T15:59:15.773583+0000","last_clean":"2024-09-10T15:59:15.773583+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T15:59:15.773583+0000","last_undegraded":"2024-09-10T15:59:15.773583+0000","last_fullsized":"2024-09-10T15:59:15.773583+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'5718","reported_seq":5854,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:51.268741+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T15:59:51.268741+0000","last_peered":"2024-09-10T15:59:51.268741+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T15:59:51.268741+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":5718,"log_dups_size":0,"ondisk_log_size":5718,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23404544,"num_objects":5714,"num_object_clones":0,"num_object_copies":11428,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5714,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1973,"num_write_kb":7892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1343},{"shards":"0,2","objects":4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.916017+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T15:59:15.916017+0000","last_peered":"2024-09-10T15:59:15.916017+0000","last_clean":"2024-09-10T15:59:15.916017+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T15:59:15.916017+0000","last_undegraded":"2024-09-10T15:59:15.916017+0000","last_fullsized":"2024-09-10T15:59:15.916017+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.776033+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T15:59:15.776033+0000","last_peered":"2024-09-10T15:59:15.776033+0000","last_clean":"2024-09-10T15:59:15.776033+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T15:59:15.776033+0000","last_undegraded":"2024-09-10T15:59:15.776033+0000","last_fullsized":"2024-09-10T15:59:15.776033+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'5692","reported_seq":5829,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:51.279604+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T15:59:51.279604+0000","last_peered":"2024-09-10T15:59:51.279604+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T15:59:51.279604+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":5692,"log_dups_size":0,"ondisk_log_size":5692,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23266246,"num_objects":5682,"num_object_clones":0,"num_object_copies":11364,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5682,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1855,"num_write_kb":7420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1359},{"shards":"0,2","objects":4323}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":364200670,"num_objects":88925,"num_object_clones":0,"num_object_copies":177850,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":47117,"num_objects_misplaced":3563,"num_objects_unfound":0,"num_objects_dirty":88925,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":46771,"num_write_kb":187084,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":708,"num_bytes_recovered":2899968,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":727314432,"data_stored":727159073,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":88997,"ondisk_log_size":88997,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986198,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":466836,"kb_used_data":312964,"kb_used_omap":4,"kb_used_meta":153851,"kb_avail":93905004,"statfs":{"total":96636764160,"available":96158724096,"internally_reserved":0,"allocated":320475136,"data_stored":320197376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":157544106},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,2,0,4,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659144,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":136196,"kb_used_data":60084,"kb_used_omap":4,"kb_used_meta":76091,"kb_avail":94235644,"statfs":{"total":96636764160,"available":96497299456,"internally_reserved":0,"allocated":61526016,"data_stored":61197210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":77917870},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,5,0,3,6],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182427,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":508644,"kb_used_data":336788,"kb_used_omap":4,"kb_used_meta":171835,"kb_avail":93863196,"statfs":{"total":96636764160,"available":96115912704,"internally_reserved":0,"allocated":344870912,"data_stored":344589075,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":175959722},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,4,2,9],"upper_bound":64},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":345182208,"data_stored":345146078,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61198336,"data_stored":61115237,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":320933888,"data_stored":320897758,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:59:52.378 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:59:52.972 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout:2024-09-10T15:59:52.971927+0000 min lat: 0.00225581 max lat: 2.21736 avg lat: 0.0177277 2024-09-10T15:59:52.972 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-10T15:59:52.972 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 80 20 90250 90230 4.40512 1.88281 0.0620051 0.0177277 2024-09-10T15:59:52.996 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T15:59:52.999 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T15:59:53.024 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":114,"stamp":"2024-09-10T15:59:51.553729+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":364200689,"num_objects":88927,"num_object_clones":0,"num_object_copies":177854,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":47117,"num_objects_misplaced":3563,"num_objects_unfound":0,"num_objects_dirty":88927,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":46773,"num_write_kb":187086,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":710,"num_bytes_recovered":2899987,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":89000,"ondisk_log_size":89000,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1111676,"kb_used_data":709836,"kb_used_omap":12,"kb_used_meta":401779,"kb_avail":282003844,"statfs":{"total":289910292480,"available":288771936256,"internally_reserved":0,"allocated":726872064,"data_stored":725983661,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":411421698},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,7,4,7,15,4,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":31354880,"num_objects":7655,"num_object_clones":0,"num_object_copies":15310,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-223,"num_objects_unfound":0,"num_objects_dirty":7655,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7655,"num_write_kb":30620,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":223,"num_bytes_recovered":913408,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003124"},"pg_stats":[{"pgid":"2.f","version":"43'5546","reported_seq":5615,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:51.278437+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T15:59:51.278437+0000","last_peered":"2024-09-10T15:59:51.278437+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T15:59:51.278437+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":5546,"log_dups_size":0,"ondisk_log_size":5546,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22716416,"num_objects":5546,"num_object_clones":0,"num_object_copies":11092,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5546,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5546,"num_write_kb":22184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1287},{"shards":"0,2","objects":4258}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'5517","reported_seq":5624,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:49.303314+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T15:59:49.303314+0000","last_peered":"2024-09-10T15:59:49.303314+0000","last_clean":"2024-09-10T15:59:49.303314+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T15:59:49.303314+0000","last_undegraded":"2024-09-10T15:59:49.303314+0000","last_fullsized":"2024-09-10T15:59:49.303314+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":5517,"log_dups_size":0,"ondisk_log_size":5517,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22574022,"num_objects":5513,"num_object_clones":0,"num_object_copies":11026,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5513,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3622,"num_write_kb":14488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'5456","reported_seq":6991,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T15:59:49.292181+0000","last_change":"2024-09-10T15:59:16.142612+0000","last_active":"2024-09-10T15:59:49.292181+0000","last_peered":"2024-09-10T15:59:49.292181+0000","last_clean":"2024-09-10T15:59:14.092473+0000","last_became_active":"2024-09-10T15:59:16.109611+0000","last_became_peered":"2024-09-10T15:59:16.109611+0000","last_unstale":"2024-09-10T15:59:49.292181+0000","last_undegraded":"2024-09-10T15:59:49.292181+0000","last_fullsized":"2024-09-10T15:59:15.770264+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":5456,"log_dups_size":0,"ondisk_log_size":5456,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22331392,"num_objects":5452,"num_object_clones":0,"num_object_copies":10904,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3563,"num_objects_unfound":0,"num_objects_dirty":5452,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3460,"num_write_kb":13840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":705,"num_bytes_recovered":2887680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3563},{"shards":"2","objects":1889}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'5510","reported_seq":5585,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:49.195846+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T15:59:49.195846+0000","last_peered":"2024-09-10T15:59:49.195846+0000","last_clean":"2024-09-10T15:59:49.195846+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T15:59:49.195846+0000","last_undegraded":"2024-09-10T15:59:49.195846+0000","last_fullsized":"2024-09-10T15:59:49.195846+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":5510,"log_dups_size":0,"ondisk_log_size":5510,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22568960,"num_objects":5510,"num_object_clones":0,"num_object_copies":11020,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5510,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5510,"num_write_kb":22040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'5612","reported_seq":5736,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:51.249929+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T15:59:51.249929+0000","last_peered":"2024-09-10T15:59:51.249929+0000","last_clean":"2024-09-10T15:59:51.249929+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T15:59:51.249929+0000","last_undegraded":"2024-09-10T15:59:51.249929+0000","last_fullsized":"2024-09-10T15:59:51.249929+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":5612,"log_dups_size":0,"ondisk_log_size":5612,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22958563,"num_objects":5606,"num_object_clones":0,"num_object_copies":11212,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5606,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2323,"num_write_kb":9292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'5685","reported_seq":5817,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:51.214373+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T15:59:51.214373+0000","last_peered":"2024-09-10T15:59:51.214373+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T15:59:51.214373+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":5685,"log_dups_size":0,"ondisk_log_size":5685,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23265763,"num_objects":5681,"num_object_clones":0,"num_object_copies":11362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5681,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2297,"num_write_kb":9188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1310},{"shards":"0,2","objects":4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'5524","reported_seq":5656,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:51.246244+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T15:59:51.246244+0000","last_peered":"2024-09-10T15:59:51.246244+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T15:59:51.246244+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":5524,"log_dups_size":0,"ondisk_log_size":5524,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22602694,"num_objects":5520,"num_object_clones":0,"num_object_copies":11040,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5520,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":2023,"num_write_kb":8092,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1283},{"shards":"0,2","objects":4237}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'5711","reported_seq":5835,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:51.290418+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T15:59:51.290418+0000","last_peered":"2024-09-10T15:59:51.290418+0000","last_clean":"2024-09-10T15:59:51.290418+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T15:59:51.290418+0000","last_undegraded":"2024-09-10T15:59:51.290418+0000","last_fullsized":"2024-09-10T15:59:51.290418+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":5711,"log_dups_size":0,"ondisk_log_size":5711,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23384064,"num_objects":5709,"num_object_clones":0,"num_object_copies":11418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5709,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2558,"num_write_kb":10232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.775865+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T15:59:15.775865+0000","last_peered":"2024-09-10T15:59:15.775865+0000","last_clean":"2024-09-10T15:59:15.775865+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T15:59:15.775865+0000","last_undegraded":"2024-09-10T15:59:15.775865+0000","last_fullsized":"2024-09-10T15:59:15.775865+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'5300","reported_seq":5434,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:49.273062+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T15:59:49.273062+0000","last_peered":"2024-09-10T15:59:49.273062+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T15:59:49.273062+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":5300,"log_dups_size":0,"ondisk_log_size":5300,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21680611,"num_objects":5294,"num_object_clones":0,"num_object_copies":10588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5294,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3119,"num_write_kb":12476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4123},{"shards":"2","objects":1171}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":88,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.772572+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T15:59:15.772572+0000","last_peered":"2024-09-10T15:59:15.772572+0000","last_clean":"2024-09-10T15:59:15.772572+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T15:59:15.772572+0000","last_undegraded":"2024-09-10T15:59:15.772572+0000","last_fullsized":"2024-09-10T15:59:15.772572+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'5532","reported_seq":5655,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:51.284232+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T15:59:51.284232+0000","last_peered":"2024-09-10T15:59:51.284232+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T15:59:51.284232+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":5532,"log_dups_size":0,"ondisk_log_size":5532,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22626304,"num_objects":5524,"num_object_clones":0,"num_object_copies":11048,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5524,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2036,"num_write_kb":8144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1269},{"shards":"0,2","objects":4255}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":98,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.880567+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T15:59:15.880567+0000","last_peered":"2024-09-10T15:59:15.880567+0000","last_clean":"2024-09-10T15:59:15.880567+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T15:59:15.880567+0000","last_undegraded":"2024-09-10T15:59:15.880567+0000","last_fullsized":"2024-09-10T15:59:15.880567+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'5508","reported_seq":5637,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:49.284201+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T15:59:49.284201+0000","last_peered":"2024-09-10T15:59:49.284201+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T15:59:49.284201+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":5508,"log_dups_size":0,"ondisk_log_size":5508,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22540771,"num_objects":5504,"num_object_clones":0,"num_object_copies":11008,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5504,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3188,"num_write_kb":12752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4304},{"shards":"2","objects":1200}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.837509+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T15:59:15.837509+0000","last_peered":"2024-09-10T15:59:15.837509+0000","last_clean":"2024-09-10T15:59:15.837509+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T15:59:15.837509+0000","last_undegraded":"2024-09-10T15:59:15.837509+0000","last_fullsized":"2024-09-10T15:59:15.837509+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'5594","reported_seq":5717,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:49.183178+0000","last_change":"2024-09-10T15:59:16.174113+0000","last_active":"2024-09-10T15:59:49.183178+0000","last_peered":"2024-09-10T15:59:49.183178+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T15:59:49.183178+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":5594,"log_dups_size":0,"ondisk_log_size":5594,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22888448,"num_objects":5588,"num_object_clones":0,"num_object_copies":11176,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4354,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5588,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3148,"num_write_kb":12592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4352},{"shards":"2","objects":1236}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'5588","reported_seq":5717,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:51.285012+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T15:59:51.285012+0000","last_peered":"2024-09-10T15:59:51.285012+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T15:59:51.285012+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":5588,"log_dups_size":0,"ondisk_log_size":5588,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22880256,"num_objects":5586,"num_object_clones":0,"num_object_copies":11172,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5586,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2219,"num_write_kb":8876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1282},{"shards":"0,2","objects":4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.989413+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T15:59:15.989413+0000","last_peered":"2024-09-10T15:59:15.989413+0000","last_clean":"2024-09-10T15:59:15.989413+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T15:59:15.989413+0000","last_undegraded":"2024-09-10T15:59:15.989413+0000","last_fullsized":"2024-09-10T15:59:15.989413+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'5504","reported_seq":5641,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:51.219551+0000","last_change":"2024-09-10T15:59:16.251610+0000","last_active":"2024-09-10T15:59:51.219551+0000","last_peered":"2024-09-10T15:59:51.219551+0000","last_clean":"2024-09-10T15:59:14.115595+0000","last_became_active":"2024-09-10T15:59:16.244089+0000","last_became_peered":"2024-09-10T15:59:16.244089+0000","last_unstale":"2024-09-10T15:59:51.219551+0000","last_undegraded":"2024-09-10T15:59:15.847127+0000","last_fullsized":"2024-09-10T15:59:15.774437+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":5504,"log_dups_size":0,"ondisk_log_size":5504,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22511616,"num_objects":5496,"num_object_clones":0,"num_object_copies":10992,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5496,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1894,"num_write_kb":7576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1293},{"shards":"0,2","objects":4203}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":88,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.773583+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T15:59:15.773583+0000","last_peered":"2024-09-10T15:59:15.773583+0000","last_clean":"2024-09-10T15:59:15.773583+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T15:59:15.773583+0000","last_undegraded":"2024-09-10T15:59:15.773583+0000","last_fullsized":"2024-09-10T15:59:15.773583+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'5718","reported_seq":5854,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:51.268741+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T15:59:51.268741+0000","last_peered":"2024-09-10T15:59:51.268741+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T15:59:51.268741+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":5718,"log_dups_size":0,"ondisk_log_size":5718,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23404544,"num_objects":5714,"num_object_clones":0,"num_object_copies":11428,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5714,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1973,"num_write_kb":7892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1343},{"shards":"0,2","objects":4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.916017+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T15:59:15.916017+0000","last_peered":"2024-09-10T15:59:15.916017+0000","last_clean":"2024-09-10T15:59:15.916017+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T15:59:15.916017+0000","last_undegraded":"2024-09-10T15:59:15.916017+0000","last_fullsized":"2024-09-10T15:59:15.916017+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.776033+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T15:59:15.776033+0000","last_peered":"2024-09-10T15:59:15.776033+0000","last_clean":"2024-09-10T15:59:15.776033+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T15:59:15.776033+0000","last_undegraded":"2024-09-10T15:59:15.776033+0000","last_fullsized":"2024-09-10T15:59:15.776033+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'5692","reported_seq":5829,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:51.279604+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T15:59:51.279604+0000","last_peered":"2024-09-10T15:59:51.279604+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T15:59:51.279604+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":5692,"log_dups_size":0,"ondisk_log_size":5692,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23266246,"num_objects":5682,"num_object_clones":0,"num_object_copies":11364,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5682,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1855,"num_write_kb":7420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1359},{"shards":"0,2","objects":4323}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":364200670,"num_objects":88925,"num_object_clones":0,"num_object_copies":177850,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":47117,"num_objects_misplaced":3563,"num_objects_unfound":0,"num_objects_dirty":88925,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":46771,"num_write_kb":187084,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":708,"num_bytes_recovered":2899968,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":727314432,"data_stored":727159073,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":88997,"ondisk_log_size":88997,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986198,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":466836,"kb_used_data":312964,"kb_used_omap":4,"kb_used_meta":153851,"kb_avail":93905004,"statfs":{"total":96636764160,"available":96158724096,"internally_reserved":0,"allocated":320475136,"data_stored":320197376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":157544106},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,2,0,4,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659144,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":136196,"kb_used_data":60084,"kb_used_omap":4,"kb_used_meta":76091,"kb_avail":94235644,"statfs":{"total":96636764160,"available":96497299456,"internally_reserved":0,"allocated":61526016,"data_stored":61197210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":77917870},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,5,0,3,6],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182427,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":508644,"kb_used_data":336788,"kb_used_omap":4,"kb_used_meta":171835,"kb_avail":93863196,"statfs":{"total":96636764160,"available":96115912704,"internally_reserved":0,"allocated":344870912,"data_stored":344589075,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":175959722},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,4,2,9],"upper_bound":64},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":345182208,"data_stored":345146078,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61198336,"data_stored":61115237,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":320933888,"data_stored":320897758,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:59:53.025 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-10T15:59:53.026 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "43'5546", 'reported_seq': 5615, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:59:51.278437+0000', 'last_change': '2024-09-10T15:59:16.281558+0000', 'last_active': '2024-09-10T15:59:51.278437+0000', 'last_peered': '2024-09-10T15:59:51.278437+0000', 'last_clean': '2024-09-10T15:59:14.132456+0000', 'last_became_active': '2024-09-10T15:59:16.274407+0000', 'last_became_peered': '2024-09-10T15:59:16.274407+0000', 'last_unstale': '2024-09-10T15:59:51.278437+0000', 'last_undegraded': '2024-09-10T15:59:15.851279+0000', 'last_fullsized': '2024-09-10T15:59:15.774356+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'objects_scrubbed': 0, 'log_size': 5546, 'log_dups_size': 0, 'ondisk_log_size': 5546, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22716416, 'num_objects': 5546, 'num_object_clones': 0, 'num_object_copies': 11092, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4259, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5546, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5546, 'num_write_kb': 22184, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1287}, {'shards': '0,2', 'objects': 4258}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:59:53.026 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-10T15:59:53.026 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "43'5456", 'reported_seq': 6991, 'reported_epoch': 43, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-10T15:59:49.292181+0000', 'last_change': '2024-09-10T15:59:16.142612+0000', 'last_active': '2024-09-10T15:59:49.292181+0000', 'last_peered': '2024-09-10T15:59:49.292181+0000', 'last_clean': '2024-09-10T15:59:14.092473+0000', 'last_became_active': '2024-09-10T15:59:16.109611+0000', 'last_became_peered': '2024-09-10T15:59:16.109611+0000', 'last_unstale': '2024-09-10T15:59:49.292181+0000', 'last_undegraded': '2024-09-10T15:59:49.292181+0000', 'last_fullsized': '2024-09-10T15:59:15.770264+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2029", 'last_scrub_stamp': '2024-09-10T15:58:45.933789+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:45.933789+0000', 'objects_scrubbed': 1992, 'log_size': 5456, 'log_dups_size': 0, 'ondisk_log_size': 5456, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 668, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22331392, 'num_objects': 5452, 'num_object_clones': 0, 'num_object_copies': 10904, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 3563, 'num_objects_unfound': 0, 'num_objects_dirty': 5452, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3460, 'num_write_kb': 13840, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 705, 'num_bytes_recovered': 2887680, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3563}, {'shards': '2', 'objects': 1889}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:59:53.026 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-10T15:59:53.026 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "43'5685", 'reported_seq': 5817, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:59:51.214373+0000', 'last_change': '2024-09-10T15:59:16.308264+0000', 'last_active': '2024-09-10T15:59:51.214373+0000', 'last_peered': '2024-09-10T15:59:51.214373+0000', 'last_clean': '2024-09-10T15:59:14.111551+0000', 'last_became_active': '2024-09-10T15:59:16.300040+0000', 'last_became_peered': '2024-09-10T15:59:16.300040+0000', 'last_unstale': '2024-09-10T15:59:51.214373+0000', 'last_undegraded': '2024-09-10T15:59:15.878234+0000', 'last_fullsized': '2024-09-10T15:59:15.769328+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3442", 'last_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'objects_scrubbed': 3384, 'log_size': 5685, 'log_dups_size': 0, 'ondisk_log_size': 5685, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1292, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23265763, 'num_objects': 5681, 'num_object_clones': 0, 'num_object_copies': 11362, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4372, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5681, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 2297, 'num_write_kb': 9188, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1310}, {'shards': '0,2', 'objects': 4371}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:59:53.026 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-10T15:59:53.026 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "43'5524", 'reported_seq': 5656, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:59:51.246244+0000', 'last_change': '2024-09-10T15:59:16.331899+0000', 'last_active': '2024-09-10T15:59:51.246244+0000', 'last_peered': '2024-09-10T15:59:51.246244+0000', 'last_clean': '2024-09-10T15:59:14.139436+0000', 'last_became_active': '2024-09-10T15:59:16.318715+0000', 'last_became_peered': '2024-09-10T15:59:16.318715+0000', 'last_unstale': '2024-09-10T15:59:51.246244+0000', 'last_undegraded': '2024-09-10T15:59:15.877524+0000', 'last_fullsized': '2024-09-10T15:59:15.769042+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3537", 'last_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'objects_scrubbed': 3497, 'log_size': 5524, 'log_dups_size': 0, 'ondisk_log_size': 5524, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1224, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22602694, 'num_objects': 5520, 'num_object_clones': 0, 'num_object_copies': 11040, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4237, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5520, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 2023, 'num_write_kb': 8092, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1283}, {'shards': '0,2', 'objects': 4237}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:59:53.026 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-10T15:59:53.026 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "43'5300", 'reported_seq': 5434, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:59:49.273062+0000', 'last_change': '2024-09-10T15:59:16.175680+0000', 'last_active': '2024-09-10T15:59:49.273062+0000', 'last_peered': '2024-09-10T15:59:49.273062+0000', 'last_clean': '2024-09-10T15:59:14.088187+0000', 'last_became_active': '2024-09-10T15:59:16.165810+0000', 'last_became_peered': '2024-09-10T15:59:16.165810+0000', 'last_unstale': '2024-09-10T15:59:49.273062+0000', 'last_undegraded': '2024-09-10T15:59:15.822098+0000', 'last_fullsized': '2024-09-10T15:59:15.769123+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2218", 'last_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'objects_scrubbed': 2175, 'log_size': 5300, 'log_dups_size': 0, 'ondisk_log_size': 5300, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 825, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21680611, 'num_objects': 5294, 'num_object_clones': 0, 'num_object_copies': 10588, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4124, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5294, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 3119, 'num_write_kb': 12476, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4123}, {'shards': '2', 'objects': 1171}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:59:53.026 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-10T15:59:53.027 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "43'5532", 'reported_seq': 5655, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:59:51.284232+0000', 'last_change': '2024-09-10T15:59:16.277742+0000', 'last_active': '2024-09-10T15:59:51.284232+0000', 'last_peered': '2024-09-10T15:59:51.284232+0000', 'last_clean': '2024-09-10T15:59:14.129081+0000', 'last_became_active': '2024-09-10T15:59:16.269835+0000', 'last_became_peered': '2024-09-10T15:59:16.269835+0000', 'last_unstale': '2024-09-10T15:59:51.284232+0000', 'last_undegraded': '2024-09-10T15:59:15.860857+0000', 'last_fullsized': '2024-09-10T15:59:15.771410+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3557", 'last_scrub_stamp': '2024-09-10T15:59:02.493678+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:02.493678+0000', 'objects_scrubbed': 3488, 'log_size': 5532, 'log_dups_size': 0, 'ondisk_log_size': 5532, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1472, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22626304, 'num_objects': 5524, 'num_object_clones': 0, 'num_object_copies': 11048, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4257, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5524, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2036, 'num_write_kb': 8144, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1269}, {'shards': '0,2', 'objects': 4255}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:59:53.027 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-10T15:59:53.027 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "43'5508", 'reported_seq': 5637, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:59:49.284201+0000', 'last_change': '2024-09-10T15:59:16.284311+0000', 'last_active': '2024-09-10T15:59:49.284201+0000', 'last_peered': '2024-09-10T15:59:49.284201+0000', 'last_clean': '2024-09-10T15:59:14.116041+0000', 'last_became_active': '2024-09-10T15:59:16.274543+0000', 'last_became_peered': '2024-09-10T15:59:16.274543+0000', 'last_unstale': '2024-09-10T15:59:49.284201+0000', 'last_undegraded': '2024-09-10T15:59:15.812484+0000', 'last_fullsized': '2024-09-10T15:59:15.770885+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2356", 'last_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'objects_scrubbed': 2316, 'log_size': 5508, 'log_dups_size': 0, 'ondisk_log_size': 5508, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 783, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22540771, 'num_objects': 5504, 'num_object_clones': 0, 'num_object_copies': 11008, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5504, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 3188, 'num_write_kb': 12752, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4304}, {'shards': '2', 'objects': 1200}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:59:53.027 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-10T15:59:53.027 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "43'5594", 'reported_seq': 5717, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:59:49.183178+0000', 'last_change': '2024-09-10T15:59:16.174113+0000', 'last_active': '2024-09-10T15:59:49.183178+0000', 'last_peered': '2024-09-10T15:59:49.183178+0000', 'last_clean': '2024-09-10T15:59:14.110654+0000', 'last_became_active': '2024-09-10T15:59:16.164645+0000', 'last_became_peered': '2024-09-10T15:59:16.164645+0000', 'last_unstale': '2024-09-10T15:59:49.183178+0000', 'last_undegraded': '2024-09-10T15:59:15.803593+0000', 'last_fullsized': '2024-09-10T15:59:15.769083+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2486", 'last_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'objects_scrubbed': 2440, 'log_size': 5594, 'log_dups_size': 0, 'ondisk_log_size': 5594, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 792, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22888448, 'num_objects': 5588, 'num_object_clones': 0, 'num_object_copies': 11176, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4354, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5588, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3148, 'num_write_kb': 12592, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4352}, {'shards': '2', 'objects': 1236}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:59:53.027 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-10T15:59:53.027 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "43'5588", 'reported_seq': 5717, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:59:51.285012+0000', 'last_change': '2024-09-10T15:59:16.315641+0000', 'last_active': '2024-09-10T15:59:51.285012+0000', 'last_peered': '2024-09-10T15:59:51.285012+0000', 'last_clean': '2024-09-10T15:59:14.117001+0000', 'last_became_active': '2024-09-10T15:59:16.302245+0000', 'last_became_peered': '2024-09-10T15:59:16.302245+0000', 'last_unstale': '2024-09-10T15:59:51.285012+0000', 'last_undegraded': '2024-09-10T15:59:15.845012+0000', 'last_fullsized': '2024-09-10T15:59:15.770337+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3404", 'last_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'objects_scrubbed': 3367, 'log_size': 5588, 'log_dups_size': 0, 'ondisk_log_size': 5588, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 983, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22880256, 'num_objects': 5586, 'num_object_clones': 0, 'num_object_copies': 11172, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5586, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2219, 'num_write_kb': 8876, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1282}, {'shards': '0,2', 'objects': 4304}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:59:53.027 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-10T15:59:53.027 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "43'5504", 'reported_seq': 5641, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:59:51.219551+0000', 'last_change': '2024-09-10T15:59:16.251610+0000', 'last_active': '2024-09-10T15:59:51.219551+0000', 'last_peered': '2024-09-10T15:59:51.219551+0000', 'last_clean': '2024-09-10T15:59:14.115595+0000', 'last_became_active': '2024-09-10T15:59:16.244089+0000', 'last_became_peered': '2024-09-10T15:59:16.244089+0000', 'last_unstale': '2024-09-10T15:59:51.219551+0000', 'last_undegraded': '2024-09-10T15:59:15.847127+0000', 'last_fullsized': '2024-09-10T15:59:15.774437+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3653", 'last_scrub_stamp': '2024-09-10T15:59:05.298468+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:05.298468+0000', 'objects_scrubbed': 3602, 'log_size': 5504, 'log_dups_size': 0, 'ondisk_log_size': 5504, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1289, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22511616, 'num_objects': 5496, 'num_object_clones': 0, 'num_object_copies': 10992, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4206, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5496, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1894, 'num_write_kb': 7576, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1293}, {'shards': '0,2', 'objects': 4203}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:59:53.027 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-10T15:59:53.027 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "43'5718", 'reported_seq': 5854, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:59:51.268741+0000', 'last_change': '2024-09-10T15:59:16.311291+0000', 'last_active': '2024-09-10T15:59:51.268741+0000', 'last_peered': '2024-09-10T15:59:51.268741+0000', 'last_clean': '2024-09-10T15:59:14.116363+0000', 'last_became_active': '2024-09-10T15:59:16.301244+0000', 'last_became_peered': '2024-09-10T15:59:16.301244+0000', 'last_unstale': '2024-09-10T15:59:51.268741+0000', 'last_undegraded': '2024-09-10T15:59:15.883434+0000', 'last_fullsized': '2024-09-10T15:59:15.772756+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3789", 'last_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'objects_scrubbed': 3741, 'log_size': 5718, 'log_dups_size': 0, 'ondisk_log_size': 5718, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1440, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23404544, 'num_objects': 5714, 'num_object_clones': 0, 'num_object_copies': 11428, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4373, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5714, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1973, 'num_write_kb': 7892, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1343}, {'shards': '0,2', 'objects': 4371}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:59:53.028 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-10T15:59:53.028 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "43'5692", 'reported_seq': 5829, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:59:51.279604+0000', 'last_change': '2024-09-10T15:59:16.313409+0000', 'last_active': '2024-09-10T15:59:51.279604+0000', 'last_peered': '2024-09-10T15:59:51.279604+0000', 'last_clean': '2024-09-10T15:59:14.129780+0000', 'last_became_active': '2024-09-10T15:59:16.300932+0000', 'last_became_peered': '2024-09-10T15:59:16.300932+0000', 'last_unstale': '2024-09-10T15:59:51.279604+0000', 'last_undegraded': '2024-09-10T15:59:15.887823+0000', 'last_fullsized': '2024-09-10T15:59:15.768513+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3880", 'last_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'objects_scrubbed': 3827, 'log_size': 5692, 'log_dups_size': 0, 'ondisk_log_size': 5692, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1397, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23266246, 'num_objects': 5682, 'num_object_clones': 0, 'num_object_copies': 11364, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4325, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5682, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1855, 'num_write_kb': 7420, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1359}, {'shards': '0,2', 'objects': 4323}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:59:53.028 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T15:59:53.730 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T15:59:53.731 INFO:teuthology.orchestra.run.smithi115.stdout:{"fsid":"7effee29-df05-4351-9741-ee203564ed81","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 47117/177854 objects degraded (26.492%), 11 pgs degraded","count":11},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":157,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":43,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983954,"num_in_osds":3,"osd_in_since":1725983947,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":88927,"data_bytes":364200689,"bytes_used":1138356224,"bytes_avail":288771936256,"bytes_total":289910292480,"degraded_objects":47117,"degraded_total":177854,"degraded_ratio":0.26491954074690477,"misplaced_objects":3563,"misplaced_total":177854,"misplaced_ratio":0.020033285728743803,"recovering_objects_per_sec":18,"recovering_bytes_per_sec":76097,"recovering_keys_per_sec":0,"num_objects_recovered":223,"num_bytes_recovered":913408,"num_keys_recovered":0,"write_bytes_sec":2612226,"read_op_per_sec":0,"write_op_per_sec":637},"fsmap":{"epoch":1,"btime":"2024-09-10T15:57:14:967120+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":5,"modified":"2024-09-10T15:59:19.546323+0000","services":{"osd":{"daemons":{"summary":"","1":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-10T15:59:53.753 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-10T15:59:53.753 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:59:53.972 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 81 20 90772 90752 4.37591 2.03906 0.108401 0.017844 2024-09-10T15:59:54.312 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T15:59:54.315 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T15:59:54.345 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":115,"stamp":"2024-09-10T15:59:53.555029+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":364200689,"num_objects":88927,"num_object_clones":0,"num_object_copies":177854,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":47117,"num_objects_misplaced":3563,"num_objects_unfound":0,"num_objects_dirty":88927,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":46773,"num_write_kb":187086,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":710,"num_bytes_recovered":2899987,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":89000,"ondisk_log_size":89000,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1124720,"kb_used_data":717312,"kb_used_omap":12,"kb_used_meta":407347,"kb_avail":281990800,"statfs":{"total":289910292480,"available":288758579200,"internally_reserved":0,"allocated":734527488,"data_stored":733639085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":417123330},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,5,9,13,6,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":23666688,"num_objects":5778,"num_object_clones":0,"num_object_copies":11556,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-223,"num_objects_unfound":0,"num_objects_dirty":5778,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5778,"num_write_kb":23112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":223,"num_bytes_recovered":913408,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003967"},"pg_stats":[{"pgid":"2.f","version":"43'5546","reported_seq":5615,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:51.278437+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T15:59:51.278437+0000","last_peered":"2024-09-10T15:59:51.278437+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T15:59:51.278437+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":5546,"log_dups_size":0,"ondisk_log_size":5546,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22716416,"num_objects":5546,"num_object_clones":0,"num_object_copies":11092,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5546,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5546,"num_write_kb":22184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1287},{"shards":"0,2","objects":4258}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'5517","reported_seq":5624,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:49.303314+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T15:59:49.303314+0000","last_peered":"2024-09-10T15:59:49.303314+0000","last_clean":"2024-09-10T15:59:49.303314+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T15:59:49.303314+0000","last_undegraded":"2024-09-10T15:59:49.303314+0000","last_fullsized":"2024-09-10T15:59:49.303314+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":5517,"log_dups_size":0,"ondisk_log_size":5517,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22574022,"num_objects":5513,"num_object_clones":0,"num_object_copies":11026,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5513,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3622,"num_write_kb":14488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'5456","reported_seq":6991,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T15:59:49.292181+0000","last_change":"2024-09-10T15:59:16.142612+0000","last_active":"2024-09-10T15:59:49.292181+0000","last_peered":"2024-09-10T15:59:49.292181+0000","last_clean":"2024-09-10T15:59:14.092473+0000","last_became_active":"2024-09-10T15:59:16.109611+0000","last_became_peered":"2024-09-10T15:59:16.109611+0000","last_unstale":"2024-09-10T15:59:49.292181+0000","last_undegraded":"2024-09-10T15:59:49.292181+0000","last_fullsized":"2024-09-10T15:59:15.770264+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":5456,"log_dups_size":0,"ondisk_log_size":5456,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22331392,"num_objects":5452,"num_object_clones":0,"num_object_copies":10904,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3563,"num_objects_unfound":0,"num_objects_dirty":5452,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3460,"num_write_kb":13840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":705,"num_bytes_recovered":2887680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3563},{"shards":"2","objects":1889}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'5510","reported_seq":5585,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:49.195846+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T15:59:49.195846+0000","last_peered":"2024-09-10T15:59:49.195846+0000","last_clean":"2024-09-10T15:59:49.195846+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T15:59:49.195846+0000","last_undegraded":"2024-09-10T15:59:49.195846+0000","last_fullsized":"2024-09-10T15:59:49.195846+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":5510,"log_dups_size":0,"ondisk_log_size":5510,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22568960,"num_objects":5510,"num_object_clones":0,"num_object_copies":11020,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5510,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5510,"num_write_kb":22040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'5612","reported_seq":5736,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:51.249929+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T15:59:51.249929+0000","last_peered":"2024-09-10T15:59:51.249929+0000","last_clean":"2024-09-10T15:59:51.249929+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T15:59:51.249929+0000","last_undegraded":"2024-09-10T15:59:51.249929+0000","last_fullsized":"2024-09-10T15:59:51.249929+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":5612,"log_dups_size":0,"ondisk_log_size":5612,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22958563,"num_objects":5606,"num_object_clones":0,"num_object_copies":11212,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5606,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2323,"num_write_kb":9292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'5685","reported_seq":5817,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:51.214373+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T15:59:51.214373+0000","last_peered":"2024-09-10T15:59:51.214373+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T15:59:51.214373+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":5685,"log_dups_size":0,"ondisk_log_size":5685,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23265763,"num_objects":5681,"num_object_clones":0,"num_object_copies":11362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5681,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2297,"num_write_kb":9188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1310},{"shards":"0,2","objects":4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'5524","reported_seq":5656,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:51.246244+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T15:59:51.246244+0000","last_peered":"2024-09-10T15:59:51.246244+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T15:59:51.246244+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":5524,"log_dups_size":0,"ondisk_log_size":5524,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22602694,"num_objects":5520,"num_object_clones":0,"num_object_copies":11040,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5520,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":2023,"num_write_kb":8092,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1283},{"shards":"0,2","objects":4237}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'5711","reported_seq":5835,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:51.290418+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T15:59:51.290418+0000","last_peered":"2024-09-10T15:59:51.290418+0000","last_clean":"2024-09-10T15:59:51.290418+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T15:59:51.290418+0000","last_undegraded":"2024-09-10T15:59:51.290418+0000","last_fullsized":"2024-09-10T15:59:51.290418+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":5711,"log_dups_size":0,"ondisk_log_size":5711,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23384064,"num_objects":5709,"num_object_clones":0,"num_object_copies":11418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5709,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2558,"num_write_kb":10232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.775865+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T15:59:15.775865+0000","last_peered":"2024-09-10T15:59:15.775865+0000","last_clean":"2024-09-10T15:59:15.775865+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T15:59:15.775865+0000","last_undegraded":"2024-09-10T15:59:15.775865+0000","last_fullsized":"2024-09-10T15:59:15.775865+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'5300","reported_seq":5434,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:49.273062+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T15:59:49.273062+0000","last_peered":"2024-09-10T15:59:49.273062+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T15:59:49.273062+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":5300,"log_dups_size":0,"ondisk_log_size":5300,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21680611,"num_objects":5294,"num_object_clones":0,"num_object_copies":10588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5294,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3119,"num_write_kb":12476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4123},{"shards":"2","objects":1171}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":88,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.772572+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T15:59:15.772572+0000","last_peered":"2024-09-10T15:59:15.772572+0000","last_clean":"2024-09-10T15:59:15.772572+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T15:59:15.772572+0000","last_undegraded":"2024-09-10T15:59:15.772572+0000","last_fullsized":"2024-09-10T15:59:15.772572+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'5532","reported_seq":5655,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:51.284232+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T15:59:51.284232+0000","last_peered":"2024-09-10T15:59:51.284232+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T15:59:51.284232+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":5532,"log_dups_size":0,"ondisk_log_size":5532,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22626304,"num_objects":5524,"num_object_clones":0,"num_object_copies":11048,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5524,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2036,"num_write_kb":8144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1269},{"shards":"0,2","objects":4255}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":98,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.880567+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T15:59:15.880567+0000","last_peered":"2024-09-10T15:59:15.880567+0000","last_clean":"2024-09-10T15:59:15.880567+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T15:59:15.880567+0000","last_undegraded":"2024-09-10T15:59:15.880567+0000","last_fullsized":"2024-09-10T15:59:15.880567+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'5508","reported_seq":5637,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:49.284201+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T15:59:49.284201+0000","last_peered":"2024-09-10T15:59:49.284201+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T15:59:49.284201+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":5508,"log_dups_size":0,"ondisk_log_size":5508,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22540771,"num_objects":5504,"num_object_clones":0,"num_object_copies":11008,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5504,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3188,"num_write_kb":12752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4304},{"shards":"2","objects":1200}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.837509+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T15:59:15.837509+0000","last_peered":"2024-09-10T15:59:15.837509+0000","last_clean":"2024-09-10T15:59:15.837509+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T15:59:15.837509+0000","last_undegraded":"2024-09-10T15:59:15.837509+0000","last_fullsized":"2024-09-10T15:59:15.837509+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'5594","reported_seq":5717,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:49.183178+0000","last_change":"2024-09-10T15:59:16.174113+0000","last_active":"2024-09-10T15:59:49.183178+0000","last_peered":"2024-09-10T15:59:49.183178+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T15:59:49.183178+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":5594,"log_dups_size":0,"ondisk_log_size":5594,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22888448,"num_objects":5588,"num_object_clones":0,"num_object_copies":11176,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4354,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5588,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3148,"num_write_kb":12592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4352},{"shards":"2","objects":1236}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'5588","reported_seq":5717,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:51.285012+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T15:59:51.285012+0000","last_peered":"2024-09-10T15:59:51.285012+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T15:59:51.285012+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":5588,"log_dups_size":0,"ondisk_log_size":5588,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22880256,"num_objects":5586,"num_object_clones":0,"num_object_copies":11172,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5586,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2219,"num_write_kb":8876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1282},{"shards":"0,2","objects":4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.989413+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T15:59:15.989413+0000","last_peered":"2024-09-10T15:59:15.989413+0000","last_clean":"2024-09-10T15:59:15.989413+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T15:59:15.989413+0000","last_undegraded":"2024-09-10T15:59:15.989413+0000","last_fullsized":"2024-09-10T15:59:15.989413+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'5504","reported_seq":5641,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:51.219551+0000","last_change":"2024-09-10T15:59:16.251610+0000","last_active":"2024-09-10T15:59:51.219551+0000","last_peered":"2024-09-10T15:59:51.219551+0000","last_clean":"2024-09-10T15:59:14.115595+0000","last_became_active":"2024-09-10T15:59:16.244089+0000","last_became_peered":"2024-09-10T15:59:16.244089+0000","last_unstale":"2024-09-10T15:59:51.219551+0000","last_undegraded":"2024-09-10T15:59:15.847127+0000","last_fullsized":"2024-09-10T15:59:15.774437+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":5504,"log_dups_size":0,"ondisk_log_size":5504,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22511616,"num_objects":5496,"num_object_clones":0,"num_object_copies":10992,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5496,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1894,"num_write_kb":7576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1293},{"shards":"0,2","objects":4203}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":88,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.773583+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T15:59:15.773583+0000","last_peered":"2024-09-10T15:59:15.773583+0000","last_clean":"2024-09-10T15:59:15.773583+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T15:59:15.773583+0000","last_undegraded":"2024-09-10T15:59:15.773583+0000","last_fullsized":"2024-09-10T15:59:15.773583+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'5718","reported_seq":5854,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:51.268741+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T15:59:51.268741+0000","last_peered":"2024-09-10T15:59:51.268741+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T15:59:51.268741+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":5718,"log_dups_size":0,"ondisk_log_size":5718,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23404544,"num_objects":5714,"num_object_clones":0,"num_object_copies":11428,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5714,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1973,"num_write_kb":7892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1343},{"shards":"0,2","objects":4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.916017+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T15:59:15.916017+0000","last_peered":"2024-09-10T15:59:15.916017+0000","last_clean":"2024-09-10T15:59:15.916017+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T15:59:15.916017+0000","last_undegraded":"2024-09-10T15:59:15.916017+0000","last_fullsized":"2024-09-10T15:59:15.916017+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.776033+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T15:59:15.776033+0000","last_peered":"2024-09-10T15:59:15.776033+0000","last_clean":"2024-09-10T15:59:15.776033+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T15:59:15.776033+0000","last_undegraded":"2024-09-10T15:59:15.776033+0000","last_fullsized":"2024-09-10T15:59:15.776033+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'5692","reported_seq":5829,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:51.279604+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T15:59:51.279604+0000","last_peered":"2024-09-10T15:59:51.279604+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T15:59:51.279604+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":5692,"log_dups_size":0,"ondisk_log_size":5692,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23266246,"num_objects":5682,"num_object_clones":0,"num_object_copies":11364,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5682,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1855,"num_write_kb":7420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1359},{"shards":"0,2","objects":4323}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":364200670,"num_objects":88925,"num_object_clones":0,"num_object_copies":177850,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":47117,"num_objects_misplaced":3563,"num_objects_unfound":0,"num_objects_dirty":88925,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":46771,"num_write_kb":187084,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":708,"num_bytes_recovered":2899968,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":736403456,"data_stored":736248097,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":88997,"ondisk_log_size":88997,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986198,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":466836,"kb_used_data":312964,"kb_used_omap":4,"kb_used_meta":153851,"kb_avail":93905004,"statfs":{"total":96636764160,"available":96158724096,"internally_reserved":0,"allocated":320475136,"data_stored":320197376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":157544106},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,2,0,4,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659145,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":149240,"kb_used_data":67560,"kb_used_omap":4,"kb_used_meta":81659,"kb_avail":94222600,"statfs":{"total":96636764160,"available":96483942400,"internally_reserved":0,"allocated":69181440,"data_stored":68852634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":83619502},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,5,4,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182427,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":508644,"kb_used_data":336788,"kb_used_omap":4,"kb_used_meta":171835,"kb_avail":93863196,"statfs":{"total":96636764160,"available":96115912704,"internally_reserved":0,"allocated":344870912,"data_stored":344589075,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":175959722},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,4,2,9],"upper_bound":64},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":345182208,"data_stored":345146078,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":70287360,"data_stored":70204261,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":320933888,"data_stored":320897758,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:59:54.974 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 82 20 91329 91309 4.34908 2.17578 0.0853026 0.0179551 2024-09-10T15:59:55.972 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 83 20 91913 91893 4.32416 2.28125 0.0169505 0.0180594 2024-09-10T15:59:56.972 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 84 20 92518 92498 4.30082 2.36328 0.0163037 0.0181576 2024-09-10T15:59:57.347 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:59:57.921 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T15:59:57.921 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T15:59:57.953 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":117,"stamp":"2024-09-10T15:59:57.556032+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":375591665,"num_objects":91708,"num_object_clones":0,"num_object_copies":183416,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":47117,"num_objects_misplaced":3458,"num_objects_unfound":0,"num_objects_dirty":91708,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":49554,"num_write_kb":198210,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":815,"num_bytes_recovered":3330067,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":91781,"ondisk_log_size":91781,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1147192,"kb_used_data":731460,"kb_used_omap":12,"kb_used_meta":415667,"kb_avail":281968328,"statfs":{"total":289910292480,"available":288735567872,"internally_reserved":0,"allocated":749015040,"data_stored":748126637,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":425643010},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,1,4,3,11,14,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":30539776,"num_objects":7456,"num_object_clones":0,"num_object_copies":14912,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-220,"num_objects_unfound":0,"num_objects_dirty":7456,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7456,"num_write_kb":29824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":220,"num_bytes_recovered":901120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004075"},"pg_stats":[{"pgid":"2.f","version":"43'5711","reported_seq":5780,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:56.248484+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T15:59:56.248484+0000","last_peered":"2024-09-10T15:59:56.248484+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T15:59:56.248484+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":5711,"log_dups_size":0,"ondisk_log_size":5711,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23392256,"num_objects":5711,"num_object_clones":0,"num_object_copies":11422,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5711,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5711,"num_write_kb":22844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1452},{"shards":"0,2","objects":4258}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'5689","reported_seq":5796,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:54.310642+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T15:59:54.310642+0000","last_peered":"2024-09-10T15:59:54.310642+0000","last_clean":"2024-09-10T15:59:54.310642+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T15:59:54.310642+0000","last_undegraded":"2024-09-10T15:59:54.310642+0000","last_fullsized":"2024-09-10T15:59:54.310642+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":5689,"log_dups_size":0,"ondisk_log_size":5689,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23278534,"num_objects":5685,"num_object_clones":0,"num_object_copies":11370,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5685,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3794,"num_write_kb":15176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'5618","reported_seq":7362,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T15:59:54.316153+0000","last_change":"2024-09-10T15:59:16.142612+0000","last_active":"2024-09-10T15:59:54.316153+0000","last_peered":"2024-09-10T15:59:54.316153+0000","last_clean":"2024-09-10T15:59:14.092473+0000","last_became_active":"2024-09-10T15:59:16.109611+0000","last_became_peered":"2024-09-10T15:59:16.109611+0000","last_unstale":"2024-09-10T15:59:54.316153+0000","last_undegraded":"2024-09-10T15:59:54.316153+0000","last_fullsized":"2024-09-10T15:59:15.770264+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":5618,"log_dups_size":0,"ondisk_log_size":5618,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22994944,"num_objects":5614,"num_object_clones":0,"num_object_copies":11228,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3458,"num_objects_unfound":0,"num_objects_dirty":5614,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3622,"num_write_kb":14488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":810,"num_bytes_recovered":3317760,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3458},{"shards":"2","objects":2156}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'5692","reported_seq":5766,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:54.297972+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T15:59:54.297972+0000","last_peered":"2024-09-10T15:59:54.297972+0000","last_clean":"2024-09-10T15:59:54.297972+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T15:59:54.297972+0000","last_undegraded":"2024-09-10T15:59:54.297972+0000","last_fullsized":"2024-09-10T15:59:54.297972+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":5692,"log_dups_size":0,"ondisk_log_size":5692,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23314432,"num_objects":5692,"num_object_clones":0,"num_object_copies":11384,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5692,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5692,"num_write_kb":22768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'5787","reported_seq":5911,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:56.257217+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T15:59:56.257217+0000","last_peered":"2024-09-10T15:59:56.257217+0000","last_clean":"2024-09-10T15:59:56.257217+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T15:59:56.257217+0000","last_undegraded":"2024-09-10T15:59:56.257217+0000","last_fullsized":"2024-09-10T15:59:56.257217+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":5787,"log_dups_size":0,"ondisk_log_size":5787,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23675363,"num_objects":5781,"num_object_clones":0,"num_object_copies":11562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5781,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2498,"num_write_kb":9992,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'5863","reported_seq":5995,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:56.207298+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T15:59:56.207298+0000","last_peered":"2024-09-10T15:59:56.207298+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T15:59:56.207298+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":5863,"log_dups_size":0,"ondisk_log_size":5863,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23994851,"num_objects":5859,"num_object_clones":0,"num_object_copies":11718,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5859,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2475,"num_write_kb":9900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1488},{"shards":"0,2","objects":4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'5703","reported_seq":5835,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:56.312724+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T15:59:56.312724+0000","last_peered":"2024-09-10T15:59:56.312724+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T15:59:56.312724+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":5703,"log_dups_size":0,"ondisk_log_size":5703,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23335878,"num_objects":5699,"num_object_clones":0,"num_object_copies":11398,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5699,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":2202,"num_write_kb":8808,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1462},{"shards":"0,2","objects":4237}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'5876","reported_seq":6000,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:56.317738+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T15:59:56.317738+0000","last_peered":"2024-09-10T15:59:56.317738+0000","last_clean":"2024-09-10T15:59:56.317738+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T15:59:56.317738+0000","last_undegraded":"2024-09-10T15:59:56.317738+0000","last_fullsized":"2024-09-10T15:59:56.317738+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":5876,"log_dups_size":0,"ondisk_log_size":5876,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24059904,"num_objects":5874,"num_object_clones":0,"num_object_copies":11748,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5874,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2723,"num_write_kb":10892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.775865+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T15:59:15.775865+0000","last_peered":"2024-09-10T15:59:15.775865+0000","last_clean":"2024-09-10T15:59:15.775865+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T15:59:15.775865+0000","last_undegraded":"2024-09-10T15:59:15.775865+0000","last_fullsized":"2024-09-10T15:59:15.775865+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'5482","reported_seq":5615,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:54.236516+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T15:59:54.236516+0000","last_peered":"2024-09-10T15:59:54.236516+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T15:59:54.236516+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":5482,"log_dups_size":0,"ondisk_log_size":5482,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22426083,"num_objects":5476,"num_object_clones":0,"num_object_copies":10952,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5476,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3301,"num_write_kb":13204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4123},{"shards":"2","objects":1353}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":88,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.772572+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T15:59:15.772572+0000","last_peered":"2024-09-10T15:59:15.772572+0000","last_clean":"2024-09-10T15:59:15.772572+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T15:59:15.772572+0000","last_undegraded":"2024-09-10T15:59:15.772572+0000","last_fullsized":"2024-09-10T15:59:15.772572+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'5688","reported_seq":5811,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:56.282492+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T15:59:56.282492+0000","last_peered":"2024-09-10T15:59:56.282492+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T15:59:56.282492+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":5688,"log_dups_size":0,"ondisk_log_size":5688,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23265280,"num_objects":5680,"num_object_clones":0,"num_object_copies":11360,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5680,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2192,"num_write_kb":8768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1425},{"shards":"0,2","objects":4255}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":98,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.880567+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T15:59:15.880567+0000","last_peered":"2024-09-10T15:59:15.880567+0000","last_clean":"2024-09-10T15:59:15.880567+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T15:59:15.880567+0000","last_undegraded":"2024-09-10T15:59:15.880567+0000","last_fullsized":"2024-09-10T15:59:15.880567+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'5684","reported_seq":5813,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:54.266426+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T15:59:54.266426+0000","last_peered":"2024-09-10T15:59:54.266426+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T15:59:54.266426+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":5684,"log_dups_size":0,"ondisk_log_size":5684,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23261667,"num_objects":5680,"num_object_clones":0,"num_object_copies":11360,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5680,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3364,"num_write_kb":13456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4304},{"shards":"2","objects":1376}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.837509+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T15:59:15.837509+0000","last_peered":"2024-09-10T15:59:15.837509+0000","last_clean":"2024-09-10T15:59:15.837509+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T15:59:15.837509+0000","last_undegraded":"2024-09-10T15:59:15.837509+0000","last_fullsized":"2024-09-10T15:59:15.837509+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'5774","reported_seq":5897,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:54.292557+0000","last_change":"2024-09-10T15:59:16.174113+0000","last_active":"2024-09-10T15:59:54.292557+0000","last_peered":"2024-09-10T15:59:54.292557+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T15:59:54.292557+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":5774,"log_dups_size":0,"ondisk_log_size":5774,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23625728,"num_objects":5768,"num_object_clones":0,"num_object_copies":11536,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4354,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5768,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3328,"num_write_kb":13312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4352},{"shards":"2","objects":1416}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'5781","reported_seq":5910,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:56.299895+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T15:59:56.299895+0000","last_peered":"2024-09-10T15:59:56.299895+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T15:59:56.299895+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":5781,"log_dups_size":0,"ondisk_log_size":5781,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23670784,"num_objects":5779,"num_object_clones":0,"num_object_copies":11558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5779,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2412,"num_write_kb":9648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1475},{"shards":"0,2","objects":4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.989413+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T15:59:15.989413+0000","last_peered":"2024-09-10T15:59:15.989413+0000","last_clean":"2024-09-10T15:59:15.989413+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T15:59:15.989413+0000","last_undegraded":"2024-09-10T15:59:15.989413+0000","last_fullsized":"2024-09-10T15:59:15.989413+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'5680","reported_seq":5817,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:56.318137+0000","last_change":"2024-09-10T15:59:16.251610+0000","last_active":"2024-09-10T15:59:56.318137+0000","last_peered":"2024-09-10T15:59:56.318137+0000","last_clean":"2024-09-10T15:59:14.115595+0000","last_became_active":"2024-09-10T15:59:16.244089+0000","last_became_peered":"2024-09-10T15:59:16.244089+0000","last_unstale":"2024-09-10T15:59:56.318137+0000","last_undegraded":"2024-09-10T15:59:15.847127+0000","last_fullsized":"2024-09-10T15:59:15.774437+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":5680,"log_dups_size":0,"ondisk_log_size":5680,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23232512,"num_objects":5672,"num_object_clones":0,"num_object_copies":11344,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5672,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2070,"num_write_kb":8280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1469},{"shards":"0,2","objects":4203}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":88,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.773583+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T15:59:15.773583+0000","last_peered":"2024-09-10T15:59:15.773583+0000","last_clean":"2024-09-10T15:59:15.773583+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T15:59:15.773583+0000","last_undegraded":"2024-09-10T15:59:15.773583+0000","last_fullsized":"2024-09-10T15:59:15.773583+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'5891","reported_seq":6027,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:56.291352+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T15:59:56.291352+0000","last_peered":"2024-09-10T15:59:56.291352+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T15:59:56.291352+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":5891,"log_dups_size":0,"ondisk_log_size":5891,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24113152,"num_objects":5887,"num_object_clones":0,"num_object_copies":11774,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5887,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2146,"num_write_kb":8584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1516},{"shards":"0,2","objects":4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.916017+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T15:59:15.916017+0000","last_peered":"2024-09-10T15:59:15.916017+0000","last_clean":"2024-09-10T15:59:15.916017+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T15:59:15.916017+0000","last_undegraded":"2024-09-10T15:59:15.916017+0000","last_fullsized":"2024-09-10T15:59:15.916017+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.776033+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T15:59:15.776033+0000","last_peered":"2024-09-10T15:59:15.776033+0000","last_clean":"2024-09-10T15:59:15.776033+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T15:59:15.776033+0000","last_undegraded":"2024-09-10T15:59:15.776033+0000","last_fullsized":"2024-09-10T15:59:15.776033+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'5859","reported_seq":5998,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:56.306756+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T15:59:56.306756+0000","last_peered":"2024-09-10T15:59:56.306756+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T15:59:56.306756+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":5859,"log_dups_size":0,"ondisk_log_size":5859,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23950278,"num_objects":5849,"num_object_clones":0,"num_object_copies":11698,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5849,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2022,"num_write_kb":8088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1526},{"shards":"0,2","objects":4323}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":375591646,"num_objects":91706,"num_object_clones":0,"num_object_copies":183412,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":47117,"num_objects_misplaced":3458,"num_objects_unfound":0,"num_objects_dirty":91706,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":49552,"num_write_kb":198208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":813,"num_bytes_recovered":3330048,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":750665728,"data_stored":750510369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":91778,"ondisk_log_size":91778,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986199,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":476828,"kb_used_data":318796,"kb_used_omap":4,"kb_used_meta":158011,"kb_avail":93895012,"statfs":{"total":96636764160,"available":96148492288,"internally_reserved":0,"allocated":326447104,"data_stored":326169344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":161803946},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,3,0,1,5],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659145,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":149240,"kb_used_data":67560,"kb_used_omap":4,"kb_used_meta":81659,"kb_avail":94222600,"statfs":{"total":96636764160,"available":96483942400,"internally_reserved":0,"allocated":69181440,"data_stored":68852634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":83619502},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,5,4,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182428,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":521124,"kb_used_data":345104,"kb_used_omap":4,"kb_used_meta":175995,"kb_avail":93850716,"statfs":{"total":96636764160,"available":96103133184,"internally_reserved":0,"allocated":353386496,"data_stored":353104659,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":180219562},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,0,0,2,5,5,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":353710080,"data_stored":353673950,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":70287360,"data_stored":70204261,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":326668288,"data_stored":326632158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:59:57.954 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:59:57.973 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 85 20 93074 93054 4.27573 2.17188 0.0555137 0.0182634 2024-09-10T15:59:58.518 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T15:59:58.519 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T15:59:58.549 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":117,"stamp":"2024-09-10T15:59:57.556032+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":375591665,"num_objects":91708,"num_object_clones":0,"num_object_copies":183416,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":47117,"num_objects_misplaced":3458,"num_objects_unfound":0,"num_objects_dirty":91708,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":49554,"num_write_kb":198210,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":815,"num_bytes_recovered":3330067,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":91781,"ondisk_log_size":91781,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1147192,"kb_used_data":731460,"kb_used_omap":12,"kb_used_meta":415667,"kb_avail":281968328,"statfs":{"total":289910292480,"available":288735567872,"internally_reserved":0,"allocated":749015040,"data_stored":748126637,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":425643010},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,1,4,3,11,14,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":30539776,"num_objects":7456,"num_object_clones":0,"num_object_copies":14912,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-220,"num_objects_unfound":0,"num_objects_dirty":7456,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7456,"num_write_kb":29824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":220,"num_bytes_recovered":901120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004075"},"pg_stats":[{"pgid":"2.f","version":"43'5711","reported_seq":5780,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:56.248484+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T15:59:56.248484+0000","last_peered":"2024-09-10T15:59:56.248484+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T15:59:56.248484+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":5711,"log_dups_size":0,"ondisk_log_size":5711,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23392256,"num_objects":5711,"num_object_clones":0,"num_object_copies":11422,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5711,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5711,"num_write_kb":22844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1452},{"shards":"0,2","objects":4258}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'5689","reported_seq":5796,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:54.310642+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T15:59:54.310642+0000","last_peered":"2024-09-10T15:59:54.310642+0000","last_clean":"2024-09-10T15:59:54.310642+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T15:59:54.310642+0000","last_undegraded":"2024-09-10T15:59:54.310642+0000","last_fullsized":"2024-09-10T15:59:54.310642+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":5689,"log_dups_size":0,"ondisk_log_size":5689,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23278534,"num_objects":5685,"num_object_clones":0,"num_object_copies":11370,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5685,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3794,"num_write_kb":15176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'5618","reported_seq":7362,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T15:59:54.316153+0000","last_change":"2024-09-10T15:59:16.142612+0000","last_active":"2024-09-10T15:59:54.316153+0000","last_peered":"2024-09-10T15:59:54.316153+0000","last_clean":"2024-09-10T15:59:14.092473+0000","last_became_active":"2024-09-10T15:59:16.109611+0000","last_became_peered":"2024-09-10T15:59:16.109611+0000","last_unstale":"2024-09-10T15:59:54.316153+0000","last_undegraded":"2024-09-10T15:59:54.316153+0000","last_fullsized":"2024-09-10T15:59:15.770264+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":5618,"log_dups_size":0,"ondisk_log_size":5618,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22994944,"num_objects":5614,"num_object_clones":0,"num_object_copies":11228,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3458,"num_objects_unfound":0,"num_objects_dirty":5614,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3622,"num_write_kb":14488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":810,"num_bytes_recovered":3317760,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3458},{"shards":"2","objects":2156}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'5692","reported_seq":5766,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:54.297972+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T15:59:54.297972+0000","last_peered":"2024-09-10T15:59:54.297972+0000","last_clean":"2024-09-10T15:59:54.297972+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T15:59:54.297972+0000","last_undegraded":"2024-09-10T15:59:54.297972+0000","last_fullsized":"2024-09-10T15:59:54.297972+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":5692,"log_dups_size":0,"ondisk_log_size":5692,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23314432,"num_objects":5692,"num_object_clones":0,"num_object_copies":11384,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5692,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5692,"num_write_kb":22768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'5787","reported_seq":5911,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:56.257217+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T15:59:56.257217+0000","last_peered":"2024-09-10T15:59:56.257217+0000","last_clean":"2024-09-10T15:59:56.257217+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T15:59:56.257217+0000","last_undegraded":"2024-09-10T15:59:56.257217+0000","last_fullsized":"2024-09-10T15:59:56.257217+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":5787,"log_dups_size":0,"ondisk_log_size":5787,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23675363,"num_objects":5781,"num_object_clones":0,"num_object_copies":11562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5781,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2498,"num_write_kb":9992,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'5863","reported_seq":5995,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:56.207298+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T15:59:56.207298+0000","last_peered":"2024-09-10T15:59:56.207298+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T15:59:56.207298+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":5863,"log_dups_size":0,"ondisk_log_size":5863,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23994851,"num_objects":5859,"num_object_clones":0,"num_object_copies":11718,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5859,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2475,"num_write_kb":9900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1488},{"shards":"0,2","objects":4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'5703","reported_seq":5835,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:56.312724+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T15:59:56.312724+0000","last_peered":"2024-09-10T15:59:56.312724+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T15:59:56.312724+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":5703,"log_dups_size":0,"ondisk_log_size":5703,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23335878,"num_objects":5699,"num_object_clones":0,"num_object_copies":11398,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5699,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":2202,"num_write_kb":8808,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1462},{"shards":"0,2","objects":4237}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'5876","reported_seq":6000,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:56.317738+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T15:59:56.317738+0000","last_peered":"2024-09-10T15:59:56.317738+0000","last_clean":"2024-09-10T15:59:56.317738+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T15:59:56.317738+0000","last_undegraded":"2024-09-10T15:59:56.317738+0000","last_fullsized":"2024-09-10T15:59:56.317738+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":5876,"log_dups_size":0,"ondisk_log_size":5876,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24059904,"num_objects":5874,"num_object_clones":0,"num_object_copies":11748,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5874,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2723,"num_write_kb":10892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.775865+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T15:59:15.775865+0000","last_peered":"2024-09-10T15:59:15.775865+0000","last_clean":"2024-09-10T15:59:15.775865+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T15:59:15.775865+0000","last_undegraded":"2024-09-10T15:59:15.775865+0000","last_fullsized":"2024-09-10T15:59:15.775865+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'5482","reported_seq":5615,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:54.236516+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T15:59:54.236516+0000","last_peered":"2024-09-10T15:59:54.236516+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T15:59:54.236516+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":5482,"log_dups_size":0,"ondisk_log_size":5482,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22426083,"num_objects":5476,"num_object_clones":0,"num_object_copies":10952,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5476,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3301,"num_write_kb":13204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4123},{"shards":"2","objects":1353}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":88,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.772572+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T15:59:15.772572+0000","last_peered":"2024-09-10T15:59:15.772572+0000","last_clean":"2024-09-10T15:59:15.772572+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T15:59:15.772572+0000","last_undegraded":"2024-09-10T15:59:15.772572+0000","last_fullsized":"2024-09-10T15:59:15.772572+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'5688","reported_seq":5811,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:56.282492+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T15:59:56.282492+0000","last_peered":"2024-09-10T15:59:56.282492+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T15:59:56.282492+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":5688,"log_dups_size":0,"ondisk_log_size":5688,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23265280,"num_objects":5680,"num_object_clones":0,"num_object_copies":11360,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5680,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2192,"num_write_kb":8768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1425},{"shards":"0,2","objects":4255}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":98,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.880567+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T15:59:15.880567+0000","last_peered":"2024-09-10T15:59:15.880567+0000","last_clean":"2024-09-10T15:59:15.880567+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T15:59:15.880567+0000","last_undegraded":"2024-09-10T15:59:15.880567+0000","last_fullsized":"2024-09-10T15:59:15.880567+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'5684","reported_seq":5813,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:54.266426+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T15:59:54.266426+0000","last_peered":"2024-09-10T15:59:54.266426+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T15:59:54.266426+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":5684,"log_dups_size":0,"ondisk_log_size":5684,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23261667,"num_objects":5680,"num_object_clones":0,"num_object_copies":11360,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5680,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3364,"num_write_kb":13456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4304},{"shards":"2","objects":1376}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.837509+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T15:59:15.837509+0000","last_peered":"2024-09-10T15:59:15.837509+0000","last_clean":"2024-09-10T15:59:15.837509+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T15:59:15.837509+0000","last_undegraded":"2024-09-10T15:59:15.837509+0000","last_fullsized":"2024-09-10T15:59:15.837509+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'5774","reported_seq":5897,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:54.292557+0000","last_change":"2024-09-10T15:59:16.174113+0000","last_active":"2024-09-10T15:59:54.292557+0000","last_peered":"2024-09-10T15:59:54.292557+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T15:59:54.292557+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":5774,"log_dups_size":0,"ondisk_log_size":5774,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23625728,"num_objects":5768,"num_object_clones":0,"num_object_copies":11536,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4354,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5768,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3328,"num_write_kb":13312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4352},{"shards":"2","objects":1416}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'5781","reported_seq":5910,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:56.299895+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T15:59:56.299895+0000","last_peered":"2024-09-10T15:59:56.299895+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T15:59:56.299895+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":5781,"log_dups_size":0,"ondisk_log_size":5781,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23670784,"num_objects":5779,"num_object_clones":0,"num_object_copies":11558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5779,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2412,"num_write_kb":9648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1475},{"shards":"0,2","objects":4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.989413+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T15:59:15.989413+0000","last_peered":"2024-09-10T15:59:15.989413+0000","last_clean":"2024-09-10T15:59:15.989413+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T15:59:15.989413+0000","last_undegraded":"2024-09-10T15:59:15.989413+0000","last_fullsized":"2024-09-10T15:59:15.989413+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'5680","reported_seq":5817,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:56.318137+0000","last_change":"2024-09-10T15:59:16.251610+0000","last_active":"2024-09-10T15:59:56.318137+0000","last_peered":"2024-09-10T15:59:56.318137+0000","last_clean":"2024-09-10T15:59:14.115595+0000","last_became_active":"2024-09-10T15:59:16.244089+0000","last_became_peered":"2024-09-10T15:59:16.244089+0000","last_unstale":"2024-09-10T15:59:56.318137+0000","last_undegraded":"2024-09-10T15:59:15.847127+0000","last_fullsized":"2024-09-10T15:59:15.774437+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":5680,"log_dups_size":0,"ondisk_log_size":5680,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23232512,"num_objects":5672,"num_object_clones":0,"num_object_copies":11344,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5672,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2070,"num_write_kb":8280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1469},{"shards":"0,2","objects":4203}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":88,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.773583+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T15:59:15.773583+0000","last_peered":"2024-09-10T15:59:15.773583+0000","last_clean":"2024-09-10T15:59:15.773583+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T15:59:15.773583+0000","last_undegraded":"2024-09-10T15:59:15.773583+0000","last_fullsized":"2024-09-10T15:59:15.773583+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'5891","reported_seq":6027,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:56.291352+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T15:59:56.291352+0000","last_peered":"2024-09-10T15:59:56.291352+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T15:59:56.291352+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":5891,"log_dups_size":0,"ondisk_log_size":5891,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24113152,"num_objects":5887,"num_object_clones":0,"num_object_copies":11774,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5887,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2146,"num_write_kb":8584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1516},{"shards":"0,2","objects":4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.916017+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T15:59:15.916017+0000","last_peered":"2024-09-10T15:59:15.916017+0000","last_clean":"2024-09-10T15:59:15.916017+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T15:59:15.916017+0000","last_undegraded":"2024-09-10T15:59:15.916017+0000","last_fullsized":"2024-09-10T15:59:15.916017+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.776033+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T15:59:15.776033+0000","last_peered":"2024-09-10T15:59:15.776033+0000","last_clean":"2024-09-10T15:59:15.776033+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T15:59:15.776033+0000","last_undegraded":"2024-09-10T15:59:15.776033+0000","last_fullsized":"2024-09-10T15:59:15.776033+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'5859","reported_seq":5998,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:56.306756+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T15:59:56.306756+0000","last_peered":"2024-09-10T15:59:56.306756+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T15:59:56.306756+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":5859,"log_dups_size":0,"ondisk_log_size":5859,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23950278,"num_objects":5849,"num_object_clones":0,"num_object_copies":11698,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5849,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2022,"num_write_kb":8088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1526},{"shards":"0,2","objects":4323}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":375591646,"num_objects":91706,"num_object_clones":0,"num_object_copies":183412,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":47117,"num_objects_misplaced":3458,"num_objects_unfound":0,"num_objects_dirty":91706,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":49552,"num_write_kb":198208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":813,"num_bytes_recovered":3330048,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":750665728,"data_stored":750510369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":91778,"ondisk_log_size":91778,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986199,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":476828,"kb_used_data":318796,"kb_used_omap":4,"kb_used_meta":158011,"kb_avail":93895012,"statfs":{"total":96636764160,"available":96148492288,"internally_reserved":0,"allocated":326447104,"data_stored":326169344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":161803946},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,3,0,1,5],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659145,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":149240,"kb_used_data":67560,"kb_used_omap":4,"kb_used_meta":81659,"kb_avail":94222600,"statfs":{"total":96636764160,"available":96483942400,"internally_reserved":0,"allocated":69181440,"data_stored":68852634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":83619502},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,5,4,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182428,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":521124,"kb_used_data":345104,"kb_used_omap":4,"kb_used_meta":175995,"kb_avail":93850716,"statfs":{"total":96636764160,"available":96103133184,"internally_reserved":0,"allocated":353386496,"data_stored":353104659,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":180219562},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,0,0,2,5,5,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":353710080,"data_stored":353673950,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":70287360,"data_stored":70204261,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":326668288,"data_stored":326632158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:59:58.550 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-10T15:59:58.550 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "43'5711", 'reported_seq': 5780, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:59:56.248484+0000', 'last_change': '2024-09-10T15:59:16.281558+0000', 'last_active': '2024-09-10T15:59:56.248484+0000', 'last_peered': '2024-09-10T15:59:56.248484+0000', 'last_clean': '2024-09-10T15:59:14.132456+0000', 'last_became_active': '2024-09-10T15:59:16.274407+0000', 'last_became_peered': '2024-09-10T15:59:16.274407+0000', 'last_unstale': '2024-09-10T15:59:56.248484+0000', 'last_undegraded': '2024-09-10T15:59:15.851279+0000', 'last_fullsized': '2024-09-10T15:59:15.774356+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'objects_scrubbed': 0, 'log_size': 5711, 'log_dups_size': 0, 'ondisk_log_size': 5711, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23392256, 'num_objects': 5711, 'num_object_clones': 0, 'num_object_copies': 11422, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4259, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5711, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5711, 'num_write_kb': 22844, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1452}, {'shards': '0,2', 'objects': 4258}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:59:58.550 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-10T15:59:58.550 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "43'5618", 'reported_seq': 7362, 'reported_epoch': 43, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-10T15:59:54.316153+0000', 'last_change': '2024-09-10T15:59:16.142612+0000', 'last_active': '2024-09-10T15:59:54.316153+0000', 'last_peered': '2024-09-10T15:59:54.316153+0000', 'last_clean': '2024-09-10T15:59:14.092473+0000', 'last_became_active': '2024-09-10T15:59:16.109611+0000', 'last_became_peered': '2024-09-10T15:59:16.109611+0000', 'last_unstale': '2024-09-10T15:59:54.316153+0000', 'last_undegraded': '2024-09-10T15:59:54.316153+0000', 'last_fullsized': '2024-09-10T15:59:15.770264+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2029", 'last_scrub_stamp': '2024-09-10T15:58:45.933789+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:45.933789+0000', 'objects_scrubbed': 1992, 'log_size': 5618, 'log_dups_size': 0, 'ondisk_log_size': 5618, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 668, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22994944, 'num_objects': 5614, 'num_object_clones': 0, 'num_object_copies': 11228, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 3458, 'num_objects_unfound': 0, 'num_objects_dirty': 5614, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3622, 'num_write_kb': 14488, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 810, 'num_bytes_recovered': 3317760, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3458}, {'shards': '2', 'objects': 2156}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:59:58.550 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-10T15:59:58.550 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "43'5863", 'reported_seq': 5995, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:59:56.207298+0000', 'last_change': '2024-09-10T15:59:16.308264+0000', 'last_active': '2024-09-10T15:59:56.207298+0000', 'last_peered': '2024-09-10T15:59:56.207298+0000', 'last_clean': '2024-09-10T15:59:14.111551+0000', 'last_became_active': '2024-09-10T15:59:16.300040+0000', 'last_became_peered': '2024-09-10T15:59:16.300040+0000', 'last_unstale': '2024-09-10T15:59:56.207298+0000', 'last_undegraded': '2024-09-10T15:59:15.878234+0000', 'last_fullsized': '2024-09-10T15:59:15.769328+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3442", 'last_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'objects_scrubbed': 3384, 'log_size': 5863, 'log_dups_size': 0, 'ondisk_log_size': 5863, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1292, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23994851, 'num_objects': 5859, 'num_object_clones': 0, 'num_object_copies': 11718, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4372, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5859, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 2475, 'num_write_kb': 9900, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1488}, {'shards': '0,2', 'objects': 4371}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:59:58.550 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-10T15:59:58.551 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "43'5703", 'reported_seq': 5835, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:59:56.312724+0000', 'last_change': '2024-09-10T15:59:16.331899+0000', 'last_active': '2024-09-10T15:59:56.312724+0000', 'last_peered': '2024-09-10T15:59:56.312724+0000', 'last_clean': '2024-09-10T15:59:14.139436+0000', 'last_became_active': '2024-09-10T15:59:16.318715+0000', 'last_became_peered': '2024-09-10T15:59:16.318715+0000', 'last_unstale': '2024-09-10T15:59:56.312724+0000', 'last_undegraded': '2024-09-10T15:59:15.877524+0000', 'last_fullsized': '2024-09-10T15:59:15.769042+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3537", 'last_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'objects_scrubbed': 3497, 'log_size': 5703, 'log_dups_size': 0, 'ondisk_log_size': 5703, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1224, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23335878, 'num_objects': 5699, 'num_object_clones': 0, 'num_object_copies': 11398, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4237, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5699, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 2202, 'num_write_kb': 8808, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1462}, {'shards': '0,2', 'objects': 4237}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:59:58.551 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-10T15:59:58.551 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "43'5482", 'reported_seq': 5615, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:59:54.236516+0000', 'last_change': '2024-09-10T15:59:16.175680+0000', 'last_active': '2024-09-10T15:59:54.236516+0000', 'last_peered': '2024-09-10T15:59:54.236516+0000', 'last_clean': '2024-09-10T15:59:14.088187+0000', 'last_became_active': '2024-09-10T15:59:16.165810+0000', 'last_became_peered': '2024-09-10T15:59:16.165810+0000', 'last_unstale': '2024-09-10T15:59:54.236516+0000', 'last_undegraded': '2024-09-10T15:59:15.822098+0000', 'last_fullsized': '2024-09-10T15:59:15.769123+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2218", 'last_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'objects_scrubbed': 2175, 'log_size': 5482, 'log_dups_size': 0, 'ondisk_log_size': 5482, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 825, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22426083, 'num_objects': 5476, 'num_object_clones': 0, 'num_object_copies': 10952, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4124, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5476, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 3301, 'num_write_kb': 13204, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4123}, {'shards': '2', 'objects': 1353}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:59:58.551 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-10T15:59:58.551 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "43'5688", 'reported_seq': 5811, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:59:56.282492+0000', 'last_change': '2024-09-10T15:59:16.277742+0000', 'last_active': '2024-09-10T15:59:56.282492+0000', 'last_peered': '2024-09-10T15:59:56.282492+0000', 'last_clean': '2024-09-10T15:59:14.129081+0000', 'last_became_active': '2024-09-10T15:59:16.269835+0000', 'last_became_peered': '2024-09-10T15:59:16.269835+0000', 'last_unstale': '2024-09-10T15:59:56.282492+0000', 'last_undegraded': '2024-09-10T15:59:15.860857+0000', 'last_fullsized': '2024-09-10T15:59:15.771410+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3557", 'last_scrub_stamp': '2024-09-10T15:59:02.493678+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:02.493678+0000', 'objects_scrubbed': 3488, 'log_size': 5688, 'log_dups_size': 0, 'ondisk_log_size': 5688, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1472, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23265280, 'num_objects': 5680, 'num_object_clones': 0, 'num_object_copies': 11360, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4257, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5680, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2192, 'num_write_kb': 8768, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1425}, {'shards': '0,2', 'objects': 4255}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:59:58.551 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-10T15:59:58.551 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "43'5684", 'reported_seq': 5813, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:59:54.266426+0000', 'last_change': '2024-09-10T15:59:16.284311+0000', 'last_active': '2024-09-10T15:59:54.266426+0000', 'last_peered': '2024-09-10T15:59:54.266426+0000', 'last_clean': '2024-09-10T15:59:14.116041+0000', 'last_became_active': '2024-09-10T15:59:16.274543+0000', 'last_became_peered': '2024-09-10T15:59:16.274543+0000', 'last_unstale': '2024-09-10T15:59:54.266426+0000', 'last_undegraded': '2024-09-10T15:59:15.812484+0000', 'last_fullsized': '2024-09-10T15:59:15.770885+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2356", 'last_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'objects_scrubbed': 2316, 'log_size': 5684, 'log_dups_size': 0, 'ondisk_log_size': 5684, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 783, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23261667, 'num_objects': 5680, 'num_object_clones': 0, 'num_object_copies': 11360, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5680, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 3364, 'num_write_kb': 13456, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4304}, {'shards': '2', 'objects': 1376}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:59:58.551 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-10T15:59:58.551 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "43'5774", 'reported_seq': 5897, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:59:54.292557+0000', 'last_change': '2024-09-10T15:59:16.174113+0000', 'last_active': '2024-09-10T15:59:54.292557+0000', 'last_peered': '2024-09-10T15:59:54.292557+0000', 'last_clean': '2024-09-10T15:59:14.110654+0000', 'last_became_active': '2024-09-10T15:59:16.164645+0000', 'last_became_peered': '2024-09-10T15:59:16.164645+0000', 'last_unstale': '2024-09-10T15:59:54.292557+0000', 'last_undegraded': '2024-09-10T15:59:15.803593+0000', 'last_fullsized': '2024-09-10T15:59:15.769083+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2486", 'last_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'objects_scrubbed': 2440, 'log_size': 5774, 'log_dups_size': 0, 'ondisk_log_size': 5774, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 792, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23625728, 'num_objects': 5768, 'num_object_clones': 0, 'num_object_copies': 11536, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4354, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5768, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3328, 'num_write_kb': 13312, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4352}, {'shards': '2', 'objects': 1416}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:59:58.551 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-10T15:59:58.552 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "43'5781", 'reported_seq': 5910, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:59:56.299895+0000', 'last_change': '2024-09-10T15:59:16.315641+0000', 'last_active': '2024-09-10T15:59:56.299895+0000', 'last_peered': '2024-09-10T15:59:56.299895+0000', 'last_clean': '2024-09-10T15:59:14.117001+0000', 'last_became_active': '2024-09-10T15:59:16.302245+0000', 'last_became_peered': '2024-09-10T15:59:16.302245+0000', 'last_unstale': '2024-09-10T15:59:56.299895+0000', 'last_undegraded': '2024-09-10T15:59:15.845012+0000', 'last_fullsized': '2024-09-10T15:59:15.770337+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3404", 'last_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'objects_scrubbed': 3367, 'log_size': 5781, 'log_dups_size': 0, 'ondisk_log_size': 5781, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 983, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23670784, 'num_objects': 5779, 'num_object_clones': 0, 'num_object_copies': 11558, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5779, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2412, 'num_write_kb': 9648, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1475}, {'shards': '0,2', 'objects': 4304}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:59:58.552 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-10T15:59:58.552 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "43'5680", 'reported_seq': 5817, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:59:56.318137+0000', 'last_change': '2024-09-10T15:59:16.251610+0000', 'last_active': '2024-09-10T15:59:56.318137+0000', 'last_peered': '2024-09-10T15:59:56.318137+0000', 'last_clean': '2024-09-10T15:59:14.115595+0000', 'last_became_active': '2024-09-10T15:59:16.244089+0000', 'last_became_peered': '2024-09-10T15:59:16.244089+0000', 'last_unstale': '2024-09-10T15:59:56.318137+0000', 'last_undegraded': '2024-09-10T15:59:15.847127+0000', 'last_fullsized': '2024-09-10T15:59:15.774437+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3653", 'last_scrub_stamp': '2024-09-10T15:59:05.298468+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:05.298468+0000', 'objects_scrubbed': 3602, 'log_size': 5680, 'log_dups_size': 0, 'ondisk_log_size': 5680, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1289, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23232512, 'num_objects': 5672, 'num_object_clones': 0, 'num_object_copies': 11344, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4206, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5672, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2070, 'num_write_kb': 8280, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1469}, {'shards': '0,2', 'objects': 4203}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:59:58.552 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-10T15:59:58.552 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "43'5891", 'reported_seq': 6027, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:59:56.291352+0000', 'last_change': '2024-09-10T15:59:16.311291+0000', 'last_active': '2024-09-10T15:59:56.291352+0000', 'last_peered': '2024-09-10T15:59:56.291352+0000', 'last_clean': '2024-09-10T15:59:14.116363+0000', 'last_became_active': '2024-09-10T15:59:16.301244+0000', 'last_became_peered': '2024-09-10T15:59:16.301244+0000', 'last_unstale': '2024-09-10T15:59:56.291352+0000', 'last_undegraded': '2024-09-10T15:59:15.883434+0000', 'last_fullsized': '2024-09-10T15:59:15.772756+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3789", 'last_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'objects_scrubbed': 3741, 'log_size': 5891, 'log_dups_size': 0, 'ondisk_log_size': 5891, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1440, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24113152, 'num_objects': 5887, 'num_object_clones': 0, 'num_object_copies': 11774, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4373, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5887, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2146, 'num_write_kb': 8584, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1516}, {'shards': '0,2', 'objects': 4371}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:59:58.552 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-10T15:59:58.552 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "43'5859", 'reported_seq': 5998, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:59:56.306756+0000', 'last_change': '2024-09-10T15:59:16.313409+0000', 'last_active': '2024-09-10T15:59:56.306756+0000', 'last_peered': '2024-09-10T15:59:56.306756+0000', 'last_clean': '2024-09-10T15:59:14.129780+0000', 'last_became_active': '2024-09-10T15:59:16.300932+0000', 'last_became_peered': '2024-09-10T15:59:16.300932+0000', 'last_unstale': '2024-09-10T15:59:56.306756+0000', 'last_undegraded': '2024-09-10T15:59:15.887823+0000', 'last_fullsized': '2024-09-10T15:59:15.768513+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3880", 'last_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'objects_scrubbed': 3827, 'log_size': 5859, 'log_dups_size': 0, 'ondisk_log_size': 5859, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1397, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23950278, 'num_objects': 5849, 'num_object_clones': 0, 'num_object_copies': 11698, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4325, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5849, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2022, 'num_write_kb': 8088, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1526}, {'shards': '0,2', 'objects': 4323}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:59:58.552 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T15:59:58.973 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 86 20 93590 93570 4.24945 2.01562 0.0503712 0.018377 2024-09-10T15:59:59.242 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T15:59:59.242 INFO:teuthology.orchestra.run.smithi115.stdout:{"fsid":"7effee29-df05-4351-9741-ee203564ed81","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 47117/183416 objects degraded (25.689%), 11 pgs degraded","count":11},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":163,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":43,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983954,"num_in_osds":3,"osd_in_since":1725983947,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":91708,"data_bytes":375591665,"bytes_used":1174724608,"bytes_avail":288735567872,"bytes_total":289910292480,"degraded_objects":47117,"degraded_total":183416,"degraded_ratio":0.25688598595542372,"misplaced_objects":3458,"misplaced_total":183416,"misplaced_ratio":0.018853317049766651,"recovering_objects_per_sec":18,"recovering_bytes_per_sec":75067,"recovering_keys_per_sec":0,"num_objects_recovered":220,"num_bytes_recovered":901120,"num_keys_recovered":0,"write_bytes_sec":2544117,"read_op_per_sec":0,"write_op_per_sec":621},"fsmap":{"epoch":1,"btime":"2024-09-10T15:57:14:967120+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":5,"modified":"2024-09-10T15:59:19.546323+0000","services":{"osd":{"daemons":{"summary":"","1":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-10T15:59:59.272 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-10T15:59:59.273 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:59:59.813 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T15:59:59.816 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T15:59:59.837 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":118,"stamp":"2024-09-10T15:59:59.556646+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":379740913,"num_objects":92721,"num_object_clones":0,"num_object_copies":185442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":47117,"num_objects_misplaced":3346,"num_objects_unfound":0,"num_objects_dirty":92721,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":50567,"num_write_kb":202262,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":927,"num_bytes_recovered":3788819,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92794,"ondisk_log_size":92794,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1193636,"kb_used_data":747376,"kb_used_omap":12,"kb_used_meta":446195,"kb_avail":281921884,"statfs":{"total":289910292480,"available":288688009216,"internally_reserved":0,"allocated":765313024,"data_stored":764424621,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":456903682},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,0,2,5,9,9,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":27172864,"num_objects":6634,"num_object_clones":0,"num_object_copies":13268,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-332,"num_objects_unfound":0,"num_objects_dirty":6634,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6634,"num_write_kb":26536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":332,"num_bytes_recovered":1359872,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004109"},"pg_stats":[{"pgid":"2.f","version":"43'5711","reported_seq":5780,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:56.248484+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T15:59:56.248484+0000","last_peered":"2024-09-10T15:59:56.248484+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T15:59:56.248484+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":5711,"log_dups_size":0,"ondisk_log_size":5711,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23392256,"num_objects":5711,"num_object_clones":0,"num_object_copies":11422,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5711,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5711,"num_write_kb":22844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1452},{"shards":"0,2","objects":4258}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'5846","reported_seq":5953,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:59.275413+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T15:59:59.275413+0000","last_peered":"2024-09-10T15:59:59.275413+0000","last_clean":"2024-09-10T15:59:59.275413+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T15:59:59.275413+0000","last_undegraded":"2024-09-10T15:59:59.275413+0000","last_fullsized":"2024-09-10T15:59:59.275413+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":5846,"log_dups_size":0,"ondisk_log_size":5846,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23921606,"num_objects":5842,"num_object_clones":0,"num_object_copies":11684,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5842,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3951,"num_write_kb":15804,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'5786","reported_seq":7755,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T15:59:59.317170+0000","last_change":"2024-09-10T15:59:16.142612+0000","last_active":"2024-09-10T15:59:59.317170+0000","last_peered":"2024-09-10T15:59:59.317170+0000","last_clean":"2024-09-10T15:59:14.092473+0000","last_became_active":"2024-09-10T15:59:16.109611+0000","last_became_peered":"2024-09-10T15:59:16.109611+0000","last_unstale":"2024-09-10T15:59:59.317170+0000","last_undegraded":"2024-09-10T15:59:59.317170+0000","last_fullsized":"2024-09-10T15:59:15.770264+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":5786,"log_dups_size":0,"ondisk_log_size":5786,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23683072,"num_objects":5782,"num_object_clones":0,"num_object_copies":11564,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3346,"num_objects_unfound":0,"num_objects_dirty":5782,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3790,"num_write_kb":15160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":922,"num_bytes_recovered":3776512,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3346},{"shards":"2","objects":2436}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'5864","reported_seq":5939,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:59.193726+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T15:59:59.193726+0000","last_peered":"2024-09-10T15:59:59.193726+0000","last_clean":"2024-09-10T15:59:59.193726+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T15:59:59.193726+0000","last_undegraded":"2024-09-10T15:59:59.193726+0000","last_fullsized":"2024-09-10T15:59:59.193726+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":5864,"log_dups_size":0,"ondisk_log_size":5864,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24018944,"num_objects":5864,"num_object_clones":0,"num_object_copies":11728,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5864,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5864,"num_write_kb":23456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'5787","reported_seq":5911,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:56.257217+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T15:59:56.257217+0000","last_peered":"2024-09-10T15:59:56.257217+0000","last_clean":"2024-09-10T15:59:56.257217+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T15:59:56.257217+0000","last_undegraded":"2024-09-10T15:59:56.257217+0000","last_fullsized":"2024-09-10T15:59:56.257217+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":5787,"log_dups_size":0,"ondisk_log_size":5787,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23675363,"num_objects":5781,"num_object_clones":0,"num_object_copies":11562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5781,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2498,"num_write_kb":9992,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'5863","reported_seq":5995,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:56.207298+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T15:59:56.207298+0000","last_peered":"2024-09-10T15:59:56.207298+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T15:59:56.207298+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":5863,"log_dups_size":0,"ondisk_log_size":5863,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23994851,"num_objects":5859,"num_object_clones":0,"num_object_copies":11718,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5859,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2475,"num_write_kb":9900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1488},{"shards":"0,2","objects":4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'5703","reported_seq":5835,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:56.312724+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T15:59:56.312724+0000","last_peered":"2024-09-10T15:59:56.312724+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T15:59:56.312724+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":5703,"log_dups_size":0,"ondisk_log_size":5703,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23335878,"num_objects":5699,"num_object_clones":0,"num_object_copies":11398,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5699,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":2202,"num_write_kb":8808,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1462},{"shards":"0,2","objects":4237}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'5876","reported_seq":6000,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:56.317738+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T15:59:56.317738+0000","last_peered":"2024-09-10T15:59:56.317738+0000","last_clean":"2024-09-10T15:59:56.317738+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T15:59:56.317738+0000","last_undegraded":"2024-09-10T15:59:56.317738+0000","last_fullsized":"2024-09-10T15:59:56.317738+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":5876,"log_dups_size":0,"ondisk_log_size":5876,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24059904,"num_objects":5874,"num_object_clones":0,"num_object_copies":11748,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5874,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2723,"num_write_kb":10892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.775865+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T15:59:15.775865+0000","last_peered":"2024-09-10T15:59:15.775865+0000","last_clean":"2024-09-10T15:59:15.775865+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T15:59:15.775865+0000","last_undegraded":"2024-09-10T15:59:15.775865+0000","last_fullsized":"2024-09-10T15:59:15.775865+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'5639","reported_seq":5773,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:59.215006+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T15:59:59.215006+0000","last_peered":"2024-09-10T15:59:59.215006+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T15:59:59.215006+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":5639,"log_dups_size":0,"ondisk_log_size":5639,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23069155,"num_objects":5633,"num_object_clones":0,"num_object_copies":11266,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5633,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3458,"num_write_kb":13832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4123},{"shards":"2","objects":1510}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":88,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.772572+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T15:59:15.772572+0000","last_peered":"2024-09-10T15:59:15.772572+0000","last_clean":"2024-09-10T15:59:15.772572+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T15:59:15.772572+0000","last_undegraded":"2024-09-10T15:59:15.772572+0000","last_fullsized":"2024-09-10T15:59:15.772572+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'5688","reported_seq":5811,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:56.282492+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T15:59:56.282492+0000","last_peered":"2024-09-10T15:59:56.282492+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T15:59:56.282492+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":5688,"log_dups_size":0,"ondisk_log_size":5688,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23265280,"num_objects":5680,"num_object_clones":0,"num_object_copies":11360,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5680,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2192,"num_write_kb":8768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1425},{"shards":"0,2","objects":4255}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":98,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.880567+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T15:59:15.880567+0000","last_peered":"2024-09-10T15:59:15.880567+0000","last_clean":"2024-09-10T15:59:15.880567+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T15:59:15.880567+0000","last_undegraded":"2024-09-10T15:59:15.880567+0000","last_fullsized":"2024-09-10T15:59:15.880567+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'5859","reported_seq":5988,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:59.326139+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T15:59:59.326139+0000","last_peered":"2024-09-10T15:59:59.326139+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T15:59:59.326139+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":5859,"log_dups_size":0,"ondisk_log_size":5859,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23978467,"num_objects":5855,"num_object_clones":0,"num_object_copies":11710,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5855,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3539,"num_write_kb":14156,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4304},{"shards":"2","objects":1551}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.837509+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T15:59:15.837509+0000","last_peered":"2024-09-10T15:59:15.837509+0000","last_clean":"2024-09-10T15:59:15.837509+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T15:59:15.837509+0000","last_undegraded":"2024-09-10T15:59:15.837509+0000","last_fullsized":"2024-09-10T15:59:15.837509+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'5958","reported_seq":6081,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:59.290234+0000","last_change":"2024-09-10T15:59:16.174113+0000","last_active":"2024-09-10T15:59:59.290234+0000","last_peered":"2024-09-10T15:59:59.290234+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T15:59:59.290234+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":5958,"log_dups_size":0,"ondisk_log_size":5958,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24379392,"num_objects":5952,"num_object_clones":0,"num_object_copies":11904,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4354,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5952,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3512,"num_write_kb":14048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4352},{"shards":"2","objects":1600}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'5781","reported_seq":5910,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:56.299895+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T15:59:56.299895+0000","last_peered":"2024-09-10T15:59:56.299895+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T15:59:56.299895+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":5781,"log_dups_size":0,"ondisk_log_size":5781,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23670784,"num_objects":5779,"num_object_clones":0,"num_object_copies":11558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5779,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2412,"num_write_kb":9648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1475},{"shards":"0,2","objects":4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.989413+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T15:59:15.989413+0000","last_peered":"2024-09-10T15:59:15.989413+0000","last_clean":"2024-09-10T15:59:15.989413+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T15:59:15.989413+0000","last_undegraded":"2024-09-10T15:59:15.989413+0000","last_fullsized":"2024-09-10T15:59:15.989413+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'5680","reported_seq":5817,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:56.318137+0000","last_change":"2024-09-10T15:59:16.251610+0000","last_active":"2024-09-10T15:59:56.318137+0000","last_peered":"2024-09-10T15:59:56.318137+0000","last_clean":"2024-09-10T15:59:14.115595+0000","last_became_active":"2024-09-10T15:59:16.244089+0000","last_became_peered":"2024-09-10T15:59:16.244089+0000","last_unstale":"2024-09-10T15:59:56.318137+0000","last_undegraded":"2024-09-10T15:59:15.847127+0000","last_fullsized":"2024-09-10T15:59:15.774437+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":5680,"log_dups_size":0,"ondisk_log_size":5680,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23232512,"num_objects":5672,"num_object_clones":0,"num_object_copies":11344,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5672,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2070,"num_write_kb":8280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1469},{"shards":"0,2","objects":4203}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":88,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.773583+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T15:59:15.773583+0000","last_peered":"2024-09-10T15:59:15.773583+0000","last_clean":"2024-09-10T15:59:15.773583+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T15:59:15.773583+0000","last_undegraded":"2024-09-10T15:59:15.773583+0000","last_fullsized":"2024-09-10T15:59:15.773583+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'5891","reported_seq":6027,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:56.291352+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T15:59:56.291352+0000","last_peered":"2024-09-10T15:59:56.291352+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T15:59:56.291352+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":5891,"log_dups_size":0,"ondisk_log_size":5891,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24113152,"num_objects":5887,"num_object_clones":0,"num_object_copies":11774,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5887,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2146,"num_write_kb":8584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1516},{"shards":"0,2","objects":4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.916017+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T15:59:15.916017+0000","last_peered":"2024-09-10T15:59:15.916017+0000","last_clean":"2024-09-10T15:59:15.916017+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T15:59:15.916017+0000","last_undegraded":"2024-09-10T15:59:15.916017+0000","last_fullsized":"2024-09-10T15:59:15.916017+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.776033+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T15:59:15.776033+0000","last_peered":"2024-09-10T15:59:15.776033+0000","last_clean":"2024-09-10T15:59:15.776033+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T15:59:15.776033+0000","last_undegraded":"2024-09-10T15:59:15.776033+0000","last_fullsized":"2024-09-10T15:59:15.776033+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'5859","reported_seq":5998,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:56.306756+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T15:59:56.306756+0000","last_peered":"2024-09-10T15:59:56.306756+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T15:59:56.306756+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":5859,"log_dups_size":0,"ondisk_log_size":5859,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23950278,"num_objects":5849,"num_object_clones":0,"num_object_copies":11698,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5849,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2022,"num_write_kb":8088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1526},{"shards":"0,2","objects":4323}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":379740894,"num_objects":92719,"num_object_clones":0,"num_object_copies":185438,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":47117,"num_objects_misplaced":3346,"num_objects_unfound":0,"num_objects_dirty":92719,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":50565,"num_write_kb":202260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":925,"num_bytes_recovered":3788800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":765448192,"data_stored":765292833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92791,"ondisk_log_size":92791,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986200,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":504528,"kb_used_data":324288,"kb_used_omap":4,"kb_used_meta":180219,"kb_avail":93867312,"statfs":{"total":96636764160,"available":96120127488,"internally_reserved":0,"allocated":332070912,"data_stored":331793152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":184544938},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,2,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659146,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":167984,"kb_used_data":77984,"kb_used_omap":4,"kb_used_meta":89979,"kb_avail":94203856,"statfs":{"total":96636764160,"available":96464748544,"internally_reserved":0,"allocated":79855616,"data_stored":79526810,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":92139182},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,2,2,3,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182428,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":521124,"kb_used_data":345104,"kb_used_omap":4,"kb_used_meta":175995,"kb_avail":93850716,"statfs":{"total":96636764160,"available":96103133184,"internally_reserved":0,"allocated":353386496,"data_stored":353104659,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":180219562},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,0,0,2,5,5,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":353710080,"data_stored":353673950,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":79491072,"data_stored":79407973,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":332247040,"data_stored":332210910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:59:59.973 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 87 20 94110 94090 4.22395 2.03125 0.0499091 0.0184886 2024-09-10T16:00:00.974 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 88 20 94708 94688 4.20249 2.33594 0.0137142 0.0185841 2024-09-10T16:00:01.974 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 89 20 95288 95268 4.18073 2.26562 0.0149964 0.0186783 2024-09-10T16:00:02.839 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:00:02.974 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 90 20 95864 95844 4.15928 2.25 0.0844658 0.0187759 2024-09-10T16:00:03.400 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:00:03.401 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:00:03.428 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":119,"stamp":"2024-09-10T16:00:01.557403+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":387068657,"num_objects":94510,"num_object_clones":0,"num_object_copies":189020,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":47117,"num_objects_misplaced":3346,"num_objects_unfound":0,"num_objects_dirty":94510,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":52356,"num_write_kb":209418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":927,"num_bytes_recovered":3788819,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":94583,"ondisk_log_size":94583,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1210420,"kb_used_data":755844,"kb_used_omap":12,"kb_used_meta":454515,"kb_avail":281905100,"statfs":{"total":289910292480,"available":288670822400,"internally_reserved":0,"allocated":773984256,"data_stored":773095853,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":465423362},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,6,8,8,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":30240768,"num_objects":7383,"num_object_clones":0,"num_object_copies":14766,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-217,"num_objects_unfound":0,"num_objects_dirty":7383,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7383,"num_write_kb":29532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":217,"num_bytes_recovered":888832,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004281"},"pg_stats":[{"pgid":"2.f","version":"43'5887","reported_seq":5956,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:01.322815+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:00:01.322815+0000","last_peered":"2024-09-10T16:00:01.322815+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:00:01.322815+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":5887,"log_dups_size":0,"ondisk_log_size":5887,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24113152,"num_objects":5887,"num_object_clones":0,"num_object_copies":11774,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5887,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5887,"num_write_kb":23548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1628},{"shards":"0,2","objects":4258}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'5846","reported_seq":5953,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:59.275413+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T15:59:59.275413+0000","last_peered":"2024-09-10T15:59:59.275413+0000","last_clean":"2024-09-10T15:59:59.275413+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T15:59:59.275413+0000","last_undegraded":"2024-09-10T15:59:59.275413+0000","last_fullsized":"2024-09-10T15:59:59.275413+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":5846,"log_dups_size":0,"ondisk_log_size":5846,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23921606,"num_objects":5842,"num_object_clones":0,"num_object_copies":11684,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5842,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3951,"num_write_kb":15804,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'5786","reported_seq":7755,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T15:59:59.317170+0000","last_change":"2024-09-10T15:59:16.142612+0000","last_active":"2024-09-10T15:59:59.317170+0000","last_peered":"2024-09-10T15:59:59.317170+0000","last_clean":"2024-09-10T15:59:14.092473+0000","last_became_active":"2024-09-10T15:59:16.109611+0000","last_became_peered":"2024-09-10T15:59:16.109611+0000","last_unstale":"2024-09-10T15:59:59.317170+0000","last_undegraded":"2024-09-10T15:59:59.317170+0000","last_fullsized":"2024-09-10T15:59:15.770264+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":5786,"log_dups_size":0,"ondisk_log_size":5786,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23683072,"num_objects":5782,"num_object_clones":0,"num_object_copies":11564,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3346,"num_objects_unfound":0,"num_objects_dirty":5782,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3790,"num_write_kb":15160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":922,"num_bytes_recovered":3776512,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3346},{"shards":"2","objects":2436}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'5864","reported_seq":5939,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:59.193726+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T15:59:59.193726+0000","last_peered":"2024-09-10T15:59:59.193726+0000","last_clean":"2024-09-10T15:59:59.193726+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T15:59:59.193726+0000","last_undegraded":"2024-09-10T15:59:59.193726+0000","last_fullsized":"2024-09-10T15:59:59.193726+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":5864,"log_dups_size":0,"ondisk_log_size":5864,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24018944,"num_objects":5864,"num_object_clones":0,"num_object_copies":11728,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5864,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5864,"num_write_kb":23456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'5962","reported_seq":6086,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:00:01.343222+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:00:01.343222+0000","last_peered":"2024-09-10T16:00:01.343222+0000","last_clean":"2024-09-10T16:00:01.343222+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:00:01.343222+0000","last_undegraded":"2024-09-10T16:00:01.343222+0000","last_fullsized":"2024-09-10T16:00:01.343222+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":5962,"log_dups_size":0,"ondisk_log_size":5962,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24392163,"num_objects":5956,"num_object_clones":0,"num_object_copies":11912,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5956,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2673,"num_write_kb":10692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'6058","reported_seq":6187,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:01.257466+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:00:01.257466+0000","last_peered":"2024-09-10T16:00:01.257466+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:00:01.257466+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":6058,"log_dups_size":0,"ondisk_log_size":6058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24793571,"num_objects":6054,"num_object_clones":0,"num_object_copies":12108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6054,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2670,"num_write_kb":10680,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1683},{"shards":"0,2","objects":4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'5886","reported_seq":6018,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:01.317943+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:00:01.317943+0000","last_peered":"2024-09-10T16:00:01.317943+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:00:01.317943+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":5886,"log_dups_size":0,"ondisk_log_size":5886,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24085446,"num_objects":5882,"num_object_clones":0,"num_object_copies":11764,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5882,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":2385,"num_write_kb":9540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1645},{"shards":"0,2","objects":4237}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'6053","reported_seq":6177,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:00:01.275730+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:00:01.275730+0000","last_peered":"2024-09-10T16:00:01.275730+0000","last_clean":"2024-09-10T16:00:01.275730+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:00:01.275730+0000","last_undegraded":"2024-09-10T16:00:01.275730+0000","last_fullsized":"2024-09-10T16:00:01.275730+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":6053,"log_dups_size":0,"ondisk_log_size":6053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24784896,"num_objects":6051,"num_object_clones":0,"num_object_copies":12102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6051,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2900,"num_write_kb":11600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.775865+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T15:59:15.775865+0000","last_peered":"2024-09-10T15:59:15.775865+0000","last_clean":"2024-09-10T15:59:15.775865+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T15:59:15.775865+0000","last_undegraded":"2024-09-10T15:59:15.775865+0000","last_fullsized":"2024-09-10T15:59:15.775865+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'5639","reported_seq":5773,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:59.215006+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T15:59:59.215006+0000","last_peered":"2024-09-10T15:59:59.215006+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T15:59:59.215006+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":5639,"log_dups_size":0,"ondisk_log_size":5639,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23069155,"num_objects":5633,"num_object_clones":0,"num_object_copies":11266,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5633,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3458,"num_write_kb":13832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4123},{"shards":"2","objects":1510}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":88,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.772572+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T15:59:15.772572+0000","last_peered":"2024-09-10T15:59:15.772572+0000","last_clean":"2024-09-10T15:59:15.772572+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T15:59:15.772572+0000","last_undegraded":"2024-09-10T15:59:15.772572+0000","last_fullsized":"2024-09-10T15:59:15.772572+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'5866","reported_seq":5990,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:01.294307+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:00:01.294307+0000","last_peered":"2024-09-10T16:00:01.294307+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:00:01.294307+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":5866,"log_dups_size":0,"ondisk_log_size":5866,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23994368,"num_objects":5858,"num_object_clones":0,"num_object_copies":11716,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5858,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2370,"num_write_kb":9480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1603},{"shards":"0,2","objects":4255}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":98,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.880567+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T15:59:15.880567+0000","last_peered":"2024-09-10T15:59:15.880567+0000","last_clean":"2024-09-10T15:59:15.880567+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T15:59:15.880567+0000","last_undegraded":"2024-09-10T15:59:15.880567+0000","last_fullsized":"2024-09-10T15:59:15.880567+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'5859","reported_seq":5988,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:59.326139+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T15:59:59.326139+0000","last_peered":"2024-09-10T15:59:59.326139+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T15:59:59.326139+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":5859,"log_dups_size":0,"ondisk_log_size":5859,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23978467,"num_objects":5855,"num_object_clones":0,"num_object_copies":11710,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5855,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3539,"num_write_kb":14156,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4304},{"shards":"2","objects":1551}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.837509+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T15:59:15.837509+0000","last_peered":"2024-09-10T15:59:15.837509+0000","last_clean":"2024-09-10T15:59:15.837509+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T15:59:15.837509+0000","last_undegraded":"2024-09-10T15:59:15.837509+0000","last_fullsized":"2024-09-10T15:59:15.837509+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'5958","reported_seq":6081,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:59.290234+0000","last_change":"2024-09-10T15:59:16.174113+0000","last_active":"2024-09-10T15:59:59.290234+0000","last_peered":"2024-09-10T15:59:59.290234+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T15:59:59.290234+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":5958,"log_dups_size":0,"ondisk_log_size":5958,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24379392,"num_objects":5952,"num_object_clones":0,"num_object_copies":11904,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4354,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5952,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3512,"num_write_kb":14048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4352},{"shards":"2","objects":1600}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'5958","reported_seq":6086,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:01.309121+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:00:01.309121+0000","last_peered":"2024-09-10T16:00:01.309121+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:00:01.309121+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":5958,"log_dups_size":0,"ondisk_log_size":5958,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24395776,"num_objects":5956,"num_object_clones":0,"num_object_copies":11912,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5956,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2589,"num_write_kb":10356,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1652},{"shards":"0,2","objects":4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.989413+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T15:59:15.989413+0000","last_peered":"2024-09-10T15:59:15.989413+0000","last_clean":"2024-09-10T15:59:15.989413+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T15:59:15.989413+0000","last_undegraded":"2024-09-10T15:59:15.989413+0000","last_fullsized":"2024-09-10T15:59:15.989413+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'5838","reported_seq":5975,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:01.317197+0000","last_change":"2024-09-10T15:59:16.251610+0000","last_active":"2024-09-10T16:00:01.317197+0000","last_peered":"2024-09-10T16:00:01.317197+0000","last_clean":"2024-09-10T15:59:14.115595+0000","last_became_active":"2024-09-10T15:59:16.244089+0000","last_became_peered":"2024-09-10T15:59:16.244089+0000","last_unstale":"2024-09-10T16:00:01.317197+0000","last_undegraded":"2024-09-10T15:59:15.847127+0000","last_fullsized":"2024-09-10T15:59:15.774437+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":5838,"log_dups_size":0,"ondisk_log_size":5838,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23879680,"num_objects":5830,"num_object_clones":0,"num_object_copies":11660,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5830,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2228,"num_write_kb":8912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1627},{"shards":"0,2","objects":4203}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":88,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.773583+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T15:59:15.773583+0000","last_peered":"2024-09-10T15:59:15.773583+0000","last_clean":"2024-09-10T15:59:15.773583+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T15:59:15.773583+0000","last_undegraded":"2024-09-10T15:59:15.773583+0000","last_fullsized":"2024-09-10T15:59:15.773583+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'6069","reported_seq":6201,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:01.322209+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:00:01.322209+0000","last_peered":"2024-09-10T16:00:01.322209+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:00:01.322209+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":6069,"log_dups_size":0,"ondisk_log_size":6069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24842240,"num_objects":6065,"num_object_clones":0,"num_object_copies":12130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6065,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2324,"num_write_kb":9296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1694},{"shards":"0,2","objects":4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.916017+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T15:59:15.916017+0000","last_peered":"2024-09-10T15:59:15.916017+0000","last_clean":"2024-09-10T15:59:15.916017+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T15:59:15.916017+0000","last_undegraded":"2024-09-10T15:59:15.916017+0000","last_fullsized":"2024-09-10T15:59:15.916017+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.776033+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T15:59:15.776033+0000","last_peered":"2024-09-10T15:59:15.776033+0000","last_clean":"2024-09-10T15:59:15.776033+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T15:59:15.776033+0000","last_undegraded":"2024-09-10T15:59:15.776033+0000","last_fullsized":"2024-09-10T15:59:15.776033+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'6051","reported_seq":6190,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:01.336089+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:00:01.336089+0000","last_peered":"2024-09-10T16:00:01.336089+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:00:01.336089+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":6051,"log_dups_size":0,"ondisk_log_size":6051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24736710,"num_objects":6041,"num_object_clones":0,"num_object_copies":12082,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6041,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2214,"num_write_kb":8856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1718},{"shards":"0,2","objects":4323}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":387068638,"num_objects":94508,"num_object_clones":0,"num_object_copies":189016,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":47117,"num_objects_misplaced":3346,"num_objects_unfound":0,"num_objects_dirty":94508,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":52354,"num_write_kb":209416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":925,"num_bytes_recovered":3788800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":774168576,"data_stored":774013217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":94580,"ondisk_log_size":94580,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986200,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":504528,"kb_used_data":324288,"kb_used_omap":4,"kb_used_meta":180219,"kb_avail":93867312,"statfs":{"total":96636764160,"available":96120127488,"internally_reserved":0,"allocated":332070912,"data_stored":331793152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":184544938},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,2,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659146,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":167984,"kb_used_data":77984,"kb_used_omap":4,"kb_used_meta":89979,"kb_avail":94203856,"statfs":{"total":96636764160,"available":96464748544,"internally_reserved":0,"allocated":79855616,"data_stored":79526810,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":92139182},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,2,2,3,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182429,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":537908,"kb_used_data":353572,"kb_used_omap":4,"kb_used_meta":184315,"kb_avail":93833932,"statfs":{"total":96636764160,"available":96085946368,"internally_reserved":0,"allocated":362057728,"data_stored":361775891,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":188739242},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,3,4,4,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":362430464,"data_stored":362394334,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":79491072,"data_stored":79407973,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":332247040,"data_stored":332210910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:00:03.431 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:00:03.974 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 91 20 96365 96345 4.13508 1.95703 0.0171278 0.0188835 2024-09-10T16:00:03.993 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:00:03.996 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:00:04.018 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":120,"stamp":"2024-09-10T16:00:03.557981+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":387068657,"num_objects":94510,"num_object_clones":0,"num_object_copies":189020,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":47117,"num_objects_misplaced":3346,"num_objects_unfound":0,"num_objects_dirty":94510,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":52356,"num_write_kb":209418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":927,"num_bytes_recovered":3788819,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":94583,"ondisk_log_size":94583,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1228908,"kb_used_data":764604,"kb_used_omap":12,"kb_used_meta":464243,"kb_avail":281886612,"statfs":{"total":289910292480,"available":288651890688,"internally_reserved":0,"allocated":782954496,"data_stored":782066093,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":475384834},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,3,4,8,5,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":22867968,"num_objects":5583,"num_object_clones":0,"num_object_copies":11166,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-217,"num_objects_unfound":0,"num_objects_dirty":5583,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5583,"num_write_kb":22332,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":217,"num_bytes_recovered":888832,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004252"},"pg_stats":[{"pgid":"2.f","version":"43'5887","reported_seq":5956,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:01.322815+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:00:01.322815+0000","last_peered":"2024-09-10T16:00:01.322815+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:00:01.322815+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":5887,"log_dups_size":0,"ondisk_log_size":5887,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24113152,"num_objects":5887,"num_object_clones":0,"num_object_copies":11774,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5887,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5887,"num_write_kb":23548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1628},{"shards":"0,2","objects":4258}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'5846","reported_seq":5953,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:59.275413+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T15:59:59.275413+0000","last_peered":"2024-09-10T15:59:59.275413+0000","last_clean":"2024-09-10T15:59:59.275413+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T15:59:59.275413+0000","last_undegraded":"2024-09-10T15:59:59.275413+0000","last_fullsized":"2024-09-10T15:59:59.275413+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":5846,"log_dups_size":0,"ondisk_log_size":5846,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23921606,"num_objects":5842,"num_object_clones":0,"num_object_copies":11684,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5842,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3951,"num_write_kb":15804,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'5786","reported_seq":7755,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T15:59:59.317170+0000","last_change":"2024-09-10T15:59:16.142612+0000","last_active":"2024-09-10T15:59:59.317170+0000","last_peered":"2024-09-10T15:59:59.317170+0000","last_clean":"2024-09-10T15:59:14.092473+0000","last_became_active":"2024-09-10T15:59:16.109611+0000","last_became_peered":"2024-09-10T15:59:16.109611+0000","last_unstale":"2024-09-10T15:59:59.317170+0000","last_undegraded":"2024-09-10T15:59:59.317170+0000","last_fullsized":"2024-09-10T15:59:15.770264+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":5786,"log_dups_size":0,"ondisk_log_size":5786,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23683072,"num_objects":5782,"num_object_clones":0,"num_object_copies":11564,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3346,"num_objects_unfound":0,"num_objects_dirty":5782,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3790,"num_write_kb":15160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":922,"num_bytes_recovered":3776512,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3346},{"shards":"2","objects":2436}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'5864","reported_seq":5939,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:59.193726+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T15:59:59.193726+0000","last_peered":"2024-09-10T15:59:59.193726+0000","last_clean":"2024-09-10T15:59:59.193726+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T15:59:59.193726+0000","last_undegraded":"2024-09-10T15:59:59.193726+0000","last_fullsized":"2024-09-10T15:59:59.193726+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":5864,"log_dups_size":0,"ondisk_log_size":5864,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24018944,"num_objects":5864,"num_object_clones":0,"num_object_copies":11728,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5864,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5864,"num_write_kb":23456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'5962","reported_seq":6086,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:00:01.343222+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:00:01.343222+0000","last_peered":"2024-09-10T16:00:01.343222+0000","last_clean":"2024-09-10T16:00:01.343222+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:00:01.343222+0000","last_undegraded":"2024-09-10T16:00:01.343222+0000","last_fullsized":"2024-09-10T16:00:01.343222+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":5962,"log_dups_size":0,"ondisk_log_size":5962,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24392163,"num_objects":5956,"num_object_clones":0,"num_object_copies":11912,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5956,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2673,"num_write_kb":10692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'6058","reported_seq":6187,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:01.257466+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:00:01.257466+0000","last_peered":"2024-09-10T16:00:01.257466+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:00:01.257466+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":6058,"log_dups_size":0,"ondisk_log_size":6058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24793571,"num_objects":6054,"num_object_clones":0,"num_object_copies":12108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6054,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2670,"num_write_kb":10680,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1683},{"shards":"0,2","objects":4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'5886","reported_seq":6018,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:01.317943+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:00:01.317943+0000","last_peered":"2024-09-10T16:00:01.317943+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:00:01.317943+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":5886,"log_dups_size":0,"ondisk_log_size":5886,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24085446,"num_objects":5882,"num_object_clones":0,"num_object_copies":11764,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5882,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":2385,"num_write_kb":9540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1645},{"shards":"0,2","objects":4237}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'6053","reported_seq":6177,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:00:01.275730+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:00:01.275730+0000","last_peered":"2024-09-10T16:00:01.275730+0000","last_clean":"2024-09-10T16:00:01.275730+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:00:01.275730+0000","last_undegraded":"2024-09-10T16:00:01.275730+0000","last_fullsized":"2024-09-10T16:00:01.275730+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":6053,"log_dups_size":0,"ondisk_log_size":6053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24784896,"num_objects":6051,"num_object_clones":0,"num_object_copies":12102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6051,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2900,"num_write_kb":11600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.775865+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T15:59:15.775865+0000","last_peered":"2024-09-10T15:59:15.775865+0000","last_clean":"2024-09-10T15:59:15.775865+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T15:59:15.775865+0000","last_undegraded":"2024-09-10T15:59:15.775865+0000","last_fullsized":"2024-09-10T15:59:15.775865+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'5639","reported_seq":5773,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:59.215006+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T15:59:59.215006+0000","last_peered":"2024-09-10T15:59:59.215006+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T15:59:59.215006+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":5639,"log_dups_size":0,"ondisk_log_size":5639,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23069155,"num_objects":5633,"num_object_clones":0,"num_object_copies":11266,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5633,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3458,"num_write_kb":13832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4123},{"shards":"2","objects":1510}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":88,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.772572+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T15:59:15.772572+0000","last_peered":"2024-09-10T15:59:15.772572+0000","last_clean":"2024-09-10T15:59:15.772572+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T15:59:15.772572+0000","last_undegraded":"2024-09-10T15:59:15.772572+0000","last_fullsized":"2024-09-10T15:59:15.772572+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'5866","reported_seq":5990,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:01.294307+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:00:01.294307+0000","last_peered":"2024-09-10T16:00:01.294307+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:00:01.294307+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":5866,"log_dups_size":0,"ondisk_log_size":5866,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23994368,"num_objects":5858,"num_object_clones":0,"num_object_copies":11716,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5858,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2370,"num_write_kb":9480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1603},{"shards":"0,2","objects":4255}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":98,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.880567+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T15:59:15.880567+0000","last_peered":"2024-09-10T15:59:15.880567+0000","last_clean":"2024-09-10T15:59:15.880567+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T15:59:15.880567+0000","last_undegraded":"2024-09-10T15:59:15.880567+0000","last_fullsized":"2024-09-10T15:59:15.880567+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'5859","reported_seq":5988,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:59.326139+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T15:59:59.326139+0000","last_peered":"2024-09-10T15:59:59.326139+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T15:59:59.326139+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":5859,"log_dups_size":0,"ondisk_log_size":5859,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23978467,"num_objects":5855,"num_object_clones":0,"num_object_copies":11710,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5855,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3539,"num_write_kb":14156,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4304},{"shards":"2","objects":1551}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.837509+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T15:59:15.837509+0000","last_peered":"2024-09-10T15:59:15.837509+0000","last_clean":"2024-09-10T15:59:15.837509+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T15:59:15.837509+0000","last_undegraded":"2024-09-10T15:59:15.837509+0000","last_fullsized":"2024-09-10T15:59:15.837509+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'5958","reported_seq":6081,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:59.290234+0000","last_change":"2024-09-10T15:59:16.174113+0000","last_active":"2024-09-10T15:59:59.290234+0000","last_peered":"2024-09-10T15:59:59.290234+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T15:59:59.290234+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":5958,"log_dups_size":0,"ondisk_log_size":5958,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24379392,"num_objects":5952,"num_object_clones":0,"num_object_copies":11904,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4354,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5952,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3512,"num_write_kb":14048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4352},{"shards":"2","objects":1600}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'5958","reported_seq":6086,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:01.309121+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:00:01.309121+0000","last_peered":"2024-09-10T16:00:01.309121+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:00:01.309121+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":5958,"log_dups_size":0,"ondisk_log_size":5958,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24395776,"num_objects":5956,"num_object_clones":0,"num_object_copies":11912,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5956,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2589,"num_write_kb":10356,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1652},{"shards":"0,2","objects":4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.989413+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T15:59:15.989413+0000","last_peered":"2024-09-10T15:59:15.989413+0000","last_clean":"2024-09-10T15:59:15.989413+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T15:59:15.989413+0000","last_undegraded":"2024-09-10T15:59:15.989413+0000","last_fullsized":"2024-09-10T15:59:15.989413+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'5838","reported_seq":5975,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:01.317197+0000","last_change":"2024-09-10T15:59:16.251610+0000","last_active":"2024-09-10T16:00:01.317197+0000","last_peered":"2024-09-10T16:00:01.317197+0000","last_clean":"2024-09-10T15:59:14.115595+0000","last_became_active":"2024-09-10T15:59:16.244089+0000","last_became_peered":"2024-09-10T15:59:16.244089+0000","last_unstale":"2024-09-10T16:00:01.317197+0000","last_undegraded":"2024-09-10T15:59:15.847127+0000","last_fullsized":"2024-09-10T15:59:15.774437+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":5838,"log_dups_size":0,"ondisk_log_size":5838,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23879680,"num_objects":5830,"num_object_clones":0,"num_object_copies":11660,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5830,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2228,"num_write_kb":8912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1627},{"shards":"0,2","objects":4203}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":88,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.773583+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T15:59:15.773583+0000","last_peered":"2024-09-10T15:59:15.773583+0000","last_clean":"2024-09-10T15:59:15.773583+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T15:59:15.773583+0000","last_undegraded":"2024-09-10T15:59:15.773583+0000","last_fullsized":"2024-09-10T15:59:15.773583+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'6069","reported_seq":6201,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:01.322209+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:00:01.322209+0000","last_peered":"2024-09-10T16:00:01.322209+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:00:01.322209+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":6069,"log_dups_size":0,"ondisk_log_size":6069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24842240,"num_objects":6065,"num_object_clones":0,"num_object_copies":12130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6065,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2324,"num_write_kb":9296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1694},{"shards":"0,2","objects":4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.916017+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T15:59:15.916017+0000","last_peered":"2024-09-10T15:59:15.916017+0000","last_clean":"2024-09-10T15:59:15.916017+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T15:59:15.916017+0000","last_undegraded":"2024-09-10T15:59:15.916017+0000","last_fullsized":"2024-09-10T15:59:15.916017+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.776033+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T15:59:15.776033+0000","last_peered":"2024-09-10T15:59:15.776033+0000","last_clean":"2024-09-10T15:59:15.776033+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T15:59:15.776033+0000","last_undegraded":"2024-09-10T15:59:15.776033+0000","last_fullsized":"2024-09-10T15:59:15.776033+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'6051","reported_seq":6190,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:01.336089+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:00:01.336089+0000","last_peered":"2024-09-10T16:00:01.336089+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:00:01.336089+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":6051,"log_dups_size":0,"ondisk_log_size":6051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24736710,"num_objects":6041,"num_object_clones":0,"num_object_copies":12082,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6041,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2214,"num_write_kb":8856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1718},{"shards":"0,2","objects":4323}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":387068638,"num_objects":94508,"num_object_clones":0,"num_object_copies":189016,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":47117,"num_objects_misplaced":3346,"num_objects_unfound":0,"num_objects_dirty":94508,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":52354,"num_write_kb":209416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":925,"num_bytes_recovered":3788800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":783360000,"data_stored":783204641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":94580,"ondisk_log_size":94580,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986200,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":504528,"kb_used_data":324288,"kb_used_omap":4,"kb_used_meta":180219,"kb_avail":93867312,"statfs":{"total":96636764160,"available":96120127488,"internally_reserved":0,"allocated":332070912,"data_stored":331793152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":184544938},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,2,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659147,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":186472,"kb_used_data":86744,"kb_used_omap":4,"kb_used_meta":99707,"kb_avail":94185368,"statfs":{"total":96636764160,"available":96445816832,"internally_reserved":0,"allocated":88825856,"data_stored":88497050,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":102100654},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,2,0,2],"upper_bound":32},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182429,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":537908,"kb_used_data":353572,"kb_used_omap":4,"kb_used_meta":184315,"kb_avail":93833932,"statfs":{"total":96636764160,"available":96085946368,"internally_reserved":0,"allocated":362057728,"data_stored":361775891,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":188739242},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,3,4,4,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":362430464,"data_stored":362394334,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":88682496,"data_stored":88599397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":332247040,"data_stored":332210910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:00:04.019 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-10T16:00:04.019 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "43'5887", 'reported_seq': 5956, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:00:01.322815+0000', 'last_change': '2024-09-10T15:59:16.281558+0000', 'last_active': '2024-09-10T16:00:01.322815+0000', 'last_peered': '2024-09-10T16:00:01.322815+0000', 'last_clean': '2024-09-10T15:59:14.132456+0000', 'last_became_active': '2024-09-10T15:59:16.274407+0000', 'last_became_peered': '2024-09-10T15:59:16.274407+0000', 'last_unstale': '2024-09-10T16:00:01.322815+0000', 'last_undegraded': '2024-09-10T15:59:15.851279+0000', 'last_fullsized': '2024-09-10T15:59:15.774356+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'objects_scrubbed': 0, 'log_size': 5887, 'log_dups_size': 0, 'ondisk_log_size': 5887, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24113152, 'num_objects': 5887, 'num_object_clones': 0, 'num_object_copies': 11774, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4259, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5887, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5887, 'num_write_kb': 23548, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1628}, {'shards': '0,2', 'objects': 4258}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:00:04.019 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-10T16:00:04.019 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "43'5786", 'reported_seq': 7755, 'reported_epoch': 43, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-10T15:59:59.317170+0000', 'last_change': '2024-09-10T15:59:16.142612+0000', 'last_active': '2024-09-10T15:59:59.317170+0000', 'last_peered': '2024-09-10T15:59:59.317170+0000', 'last_clean': '2024-09-10T15:59:14.092473+0000', 'last_became_active': '2024-09-10T15:59:16.109611+0000', 'last_became_peered': '2024-09-10T15:59:16.109611+0000', 'last_unstale': '2024-09-10T15:59:59.317170+0000', 'last_undegraded': '2024-09-10T15:59:59.317170+0000', 'last_fullsized': '2024-09-10T15:59:15.770264+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2029", 'last_scrub_stamp': '2024-09-10T15:58:45.933789+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:45.933789+0000', 'objects_scrubbed': 1992, 'log_size': 5786, 'log_dups_size': 0, 'ondisk_log_size': 5786, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 668, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23683072, 'num_objects': 5782, 'num_object_clones': 0, 'num_object_copies': 11564, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 3346, 'num_objects_unfound': 0, 'num_objects_dirty': 5782, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3790, 'num_write_kb': 15160, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 922, 'num_bytes_recovered': 3776512, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3346}, {'shards': '2', 'objects': 2436}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:00:04.020 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-10T16:00:04.020 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "43'6058", 'reported_seq': 6187, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:00:01.257466+0000', 'last_change': '2024-09-10T15:59:16.308264+0000', 'last_active': '2024-09-10T16:00:01.257466+0000', 'last_peered': '2024-09-10T16:00:01.257466+0000', 'last_clean': '2024-09-10T15:59:14.111551+0000', 'last_became_active': '2024-09-10T15:59:16.300040+0000', 'last_became_peered': '2024-09-10T15:59:16.300040+0000', 'last_unstale': '2024-09-10T16:00:01.257466+0000', 'last_undegraded': '2024-09-10T15:59:15.878234+0000', 'last_fullsized': '2024-09-10T15:59:15.769328+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3442", 'last_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'objects_scrubbed': 3384, 'log_size': 6058, 'log_dups_size': 0, 'ondisk_log_size': 6058, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1292, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24793571, 'num_objects': 6054, 'num_object_clones': 0, 'num_object_copies': 12108, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4372, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6054, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 2670, 'num_write_kb': 10680, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1683}, {'shards': '0,2', 'objects': 4371}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:00:04.020 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-10T16:00:04.020 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "43'5886", 'reported_seq': 6018, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:00:01.317943+0000', 'last_change': '2024-09-10T15:59:16.331899+0000', 'last_active': '2024-09-10T16:00:01.317943+0000', 'last_peered': '2024-09-10T16:00:01.317943+0000', 'last_clean': '2024-09-10T15:59:14.139436+0000', 'last_became_active': '2024-09-10T15:59:16.318715+0000', 'last_became_peered': '2024-09-10T15:59:16.318715+0000', 'last_unstale': '2024-09-10T16:00:01.317943+0000', 'last_undegraded': '2024-09-10T15:59:15.877524+0000', 'last_fullsized': '2024-09-10T15:59:15.769042+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3537", 'last_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'objects_scrubbed': 3497, 'log_size': 5886, 'log_dups_size': 0, 'ondisk_log_size': 5886, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1224, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24085446, 'num_objects': 5882, 'num_object_clones': 0, 'num_object_copies': 11764, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4237, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5882, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 2385, 'num_write_kb': 9540, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1645}, {'shards': '0,2', 'objects': 4237}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:00:04.020 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-10T16:00:04.020 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "43'5639", 'reported_seq': 5773, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:59:59.215006+0000', 'last_change': '2024-09-10T15:59:16.175680+0000', 'last_active': '2024-09-10T15:59:59.215006+0000', 'last_peered': '2024-09-10T15:59:59.215006+0000', 'last_clean': '2024-09-10T15:59:14.088187+0000', 'last_became_active': '2024-09-10T15:59:16.165810+0000', 'last_became_peered': '2024-09-10T15:59:16.165810+0000', 'last_unstale': '2024-09-10T15:59:59.215006+0000', 'last_undegraded': '2024-09-10T15:59:15.822098+0000', 'last_fullsized': '2024-09-10T15:59:15.769123+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2218", 'last_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'objects_scrubbed': 2175, 'log_size': 5639, 'log_dups_size': 0, 'ondisk_log_size': 5639, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 825, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23069155, 'num_objects': 5633, 'num_object_clones': 0, 'num_object_copies': 11266, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4124, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5633, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 3458, 'num_write_kb': 13832, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4123}, {'shards': '2', 'objects': 1510}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:00:04.020 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-10T16:00:04.020 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "43'5866", 'reported_seq': 5990, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:00:01.294307+0000', 'last_change': '2024-09-10T15:59:16.277742+0000', 'last_active': '2024-09-10T16:00:01.294307+0000', 'last_peered': '2024-09-10T16:00:01.294307+0000', 'last_clean': '2024-09-10T15:59:14.129081+0000', 'last_became_active': '2024-09-10T15:59:16.269835+0000', 'last_became_peered': '2024-09-10T15:59:16.269835+0000', 'last_unstale': '2024-09-10T16:00:01.294307+0000', 'last_undegraded': '2024-09-10T15:59:15.860857+0000', 'last_fullsized': '2024-09-10T15:59:15.771410+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3557", 'last_scrub_stamp': '2024-09-10T15:59:02.493678+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:02.493678+0000', 'objects_scrubbed': 3488, 'log_size': 5866, 'log_dups_size': 0, 'ondisk_log_size': 5866, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1472, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23994368, 'num_objects': 5858, 'num_object_clones': 0, 'num_object_copies': 11716, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4257, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5858, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2370, 'num_write_kb': 9480, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1603}, {'shards': '0,2', 'objects': 4255}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:00:04.020 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-10T16:00:04.020 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "43'5859", 'reported_seq': 5988, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:59:59.326139+0000', 'last_change': '2024-09-10T15:59:16.284311+0000', 'last_active': '2024-09-10T15:59:59.326139+0000', 'last_peered': '2024-09-10T15:59:59.326139+0000', 'last_clean': '2024-09-10T15:59:14.116041+0000', 'last_became_active': '2024-09-10T15:59:16.274543+0000', 'last_became_peered': '2024-09-10T15:59:16.274543+0000', 'last_unstale': '2024-09-10T15:59:59.326139+0000', 'last_undegraded': '2024-09-10T15:59:15.812484+0000', 'last_fullsized': '2024-09-10T15:59:15.770885+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2356", 'last_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'objects_scrubbed': 2316, 'log_size': 5859, 'log_dups_size': 0, 'ondisk_log_size': 5859, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 783, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23978467, 'num_objects': 5855, 'num_object_clones': 0, 'num_object_copies': 11710, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5855, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 3539, 'num_write_kb': 14156, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4304}, {'shards': '2', 'objects': 1551}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:00:04.021 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-10T16:00:04.021 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "43'5958", 'reported_seq': 6081, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:59:59.290234+0000', 'last_change': '2024-09-10T15:59:16.174113+0000', 'last_active': '2024-09-10T15:59:59.290234+0000', 'last_peered': '2024-09-10T15:59:59.290234+0000', 'last_clean': '2024-09-10T15:59:14.110654+0000', 'last_became_active': '2024-09-10T15:59:16.164645+0000', 'last_became_peered': '2024-09-10T15:59:16.164645+0000', 'last_unstale': '2024-09-10T15:59:59.290234+0000', 'last_undegraded': '2024-09-10T15:59:15.803593+0000', 'last_fullsized': '2024-09-10T15:59:15.769083+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2486", 'last_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'objects_scrubbed': 2440, 'log_size': 5958, 'log_dups_size': 0, 'ondisk_log_size': 5958, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 792, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24379392, 'num_objects': 5952, 'num_object_clones': 0, 'num_object_copies': 11904, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4354, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5952, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3512, 'num_write_kb': 14048, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4352}, {'shards': '2', 'objects': 1600}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:00:04.021 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-10T16:00:04.021 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "43'5958", 'reported_seq': 6086, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:00:01.309121+0000', 'last_change': '2024-09-10T15:59:16.315641+0000', 'last_active': '2024-09-10T16:00:01.309121+0000', 'last_peered': '2024-09-10T16:00:01.309121+0000', 'last_clean': '2024-09-10T15:59:14.117001+0000', 'last_became_active': '2024-09-10T15:59:16.302245+0000', 'last_became_peered': '2024-09-10T15:59:16.302245+0000', 'last_unstale': '2024-09-10T16:00:01.309121+0000', 'last_undegraded': '2024-09-10T15:59:15.845012+0000', 'last_fullsized': '2024-09-10T15:59:15.770337+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3404", 'last_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'objects_scrubbed': 3367, 'log_size': 5958, 'log_dups_size': 0, 'ondisk_log_size': 5958, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 983, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24395776, 'num_objects': 5956, 'num_object_clones': 0, 'num_object_copies': 11912, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5956, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2589, 'num_write_kb': 10356, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1652}, {'shards': '0,2', 'objects': 4304}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:00:04.021 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-10T16:00:04.021 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "43'5838", 'reported_seq': 5975, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:00:01.317197+0000', 'last_change': '2024-09-10T15:59:16.251610+0000', 'last_active': '2024-09-10T16:00:01.317197+0000', 'last_peered': '2024-09-10T16:00:01.317197+0000', 'last_clean': '2024-09-10T15:59:14.115595+0000', 'last_became_active': '2024-09-10T15:59:16.244089+0000', 'last_became_peered': '2024-09-10T15:59:16.244089+0000', 'last_unstale': '2024-09-10T16:00:01.317197+0000', 'last_undegraded': '2024-09-10T15:59:15.847127+0000', 'last_fullsized': '2024-09-10T15:59:15.774437+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3653", 'last_scrub_stamp': '2024-09-10T15:59:05.298468+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:05.298468+0000', 'objects_scrubbed': 3602, 'log_size': 5838, 'log_dups_size': 0, 'ondisk_log_size': 5838, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1289, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23879680, 'num_objects': 5830, 'num_object_clones': 0, 'num_object_copies': 11660, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4206, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5830, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2228, 'num_write_kb': 8912, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1627}, {'shards': '0,2', 'objects': 4203}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:00:04.021 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-10T16:00:04.021 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "43'6069", 'reported_seq': 6201, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:00:01.322209+0000', 'last_change': '2024-09-10T15:59:16.311291+0000', 'last_active': '2024-09-10T16:00:01.322209+0000', 'last_peered': '2024-09-10T16:00:01.322209+0000', 'last_clean': '2024-09-10T15:59:14.116363+0000', 'last_became_active': '2024-09-10T15:59:16.301244+0000', 'last_became_peered': '2024-09-10T15:59:16.301244+0000', 'last_unstale': '2024-09-10T16:00:01.322209+0000', 'last_undegraded': '2024-09-10T15:59:15.883434+0000', 'last_fullsized': '2024-09-10T15:59:15.772756+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3789", 'last_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'objects_scrubbed': 3741, 'log_size': 6069, 'log_dups_size': 0, 'ondisk_log_size': 6069, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1440, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24842240, 'num_objects': 6065, 'num_object_clones': 0, 'num_object_copies': 12130, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4373, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6065, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2324, 'num_write_kb': 9296, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1694}, {'shards': '0,2', 'objects': 4371}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:00:04.021 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-10T16:00:04.021 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "43'6051", 'reported_seq': 6190, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:00:01.336089+0000', 'last_change': '2024-09-10T15:59:16.313409+0000', 'last_active': '2024-09-10T16:00:01.336089+0000', 'last_peered': '2024-09-10T16:00:01.336089+0000', 'last_clean': '2024-09-10T15:59:14.129780+0000', 'last_became_active': '2024-09-10T15:59:16.300932+0000', 'last_became_peered': '2024-09-10T15:59:16.300932+0000', 'last_unstale': '2024-09-10T16:00:01.336089+0000', 'last_undegraded': '2024-09-10T15:59:15.887823+0000', 'last_fullsized': '2024-09-10T15:59:15.768513+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3880", 'last_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'objects_scrubbed': 3827, 'log_size': 6051, 'log_dups_size': 0, 'ondisk_log_size': 6051, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1397, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24736710, 'num_objects': 6041, 'num_object_clones': 0, 'num_object_copies': 12082, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4325, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6041, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2214, 'num_write_kb': 8856, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1718}, {'shards': '0,2', 'objects': 4323}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:00:04.022 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T16:00:04.700 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:00:04.700 INFO:teuthology.orchestra.run.smithi115.stdout:{"fsid":"7effee29-df05-4351-9741-ee203564ed81","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 47117/189020 objects degraded (24.927%), 11 pgs degraded","count":11},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":168,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":43,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983954,"num_in_osds":3,"osd_in_since":1725983947,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":94510,"data_bytes":387068657,"bytes_used":1258401792,"bytes_avail":288651890688,"bytes_total":289910292480,"degraded_objects":47117,"degraded_total":189020,"degraded_ratio":0.24926991852713998,"misplaced_objects":3346,"misplaced_total":189020,"misplaced_ratio":0.017701830494127605,"recovering_objects_per_sec":18,"recovering_bytes_per_sec":74043,"recovering_keys_per_sec":0,"num_objects_recovered":217,"num_bytes_recovered":888832,"num_keys_recovered":0,"write_bytes_sec":1904988,"read_op_per_sec":0,"write_op_per_sec":465},"fsmap":{"epoch":1,"btime":"2024-09-10T15:57:14:967120+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":5,"modified":"2024-09-10T15:59:19.546323+0000","services":{"osd":{"daemons":{"summary":"","1":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-10T16:00:04.725 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-10T16:00:04.725 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:00:04.974 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 92 20 96879 96859 4.11195 2.00781 0.0327126 0.0189913 2024-09-10T16:00:05.268 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:00:05.269 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:00:05.295 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":120,"stamp":"2024-09-10T16:00:03.557981+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":387068657,"num_objects":94510,"num_object_clones":0,"num_object_copies":189020,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":47117,"num_objects_misplaced":3346,"num_objects_unfound":0,"num_objects_dirty":94510,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":52356,"num_write_kb":209418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":927,"num_bytes_recovered":3788819,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":94583,"ondisk_log_size":94583,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1228908,"kb_used_data":764604,"kb_used_omap":12,"kb_used_meta":464243,"kb_avail":281886612,"statfs":{"total":289910292480,"available":288651890688,"internally_reserved":0,"allocated":782954496,"data_stored":782066093,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":475384834},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,3,4,8,5,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":22867968,"num_objects":5583,"num_object_clones":0,"num_object_copies":11166,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-217,"num_objects_unfound":0,"num_objects_dirty":5583,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5583,"num_write_kb":22332,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":217,"num_bytes_recovered":888832,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004252"},"pg_stats":[{"pgid":"2.f","version":"43'5887","reported_seq":5956,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:01.322815+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:00:01.322815+0000","last_peered":"2024-09-10T16:00:01.322815+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:00:01.322815+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":5887,"log_dups_size":0,"ondisk_log_size":5887,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24113152,"num_objects":5887,"num_object_clones":0,"num_object_copies":11774,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5887,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5887,"num_write_kb":23548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1628},{"shards":"0,2","objects":4258}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'5846","reported_seq":5953,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:59.275413+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T15:59:59.275413+0000","last_peered":"2024-09-10T15:59:59.275413+0000","last_clean":"2024-09-10T15:59:59.275413+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T15:59:59.275413+0000","last_undegraded":"2024-09-10T15:59:59.275413+0000","last_fullsized":"2024-09-10T15:59:59.275413+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":5846,"log_dups_size":0,"ondisk_log_size":5846,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23921606,"num_objects":5842,"num_object_clones":0,"num_object_copies":11684,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5842,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3951,"num_write_kb":15804,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'5786","reported_seq":7755,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T15:59:59.317170+0000","last_change":"2024-09-10T15:59:16.142612+0000","last_active":"2024-09-10T15:59:59.317170+0000","last_peered":"2024-09-10T15:59:59.317170+0000","last_clean":"2024-09-10T15:59:14.092473+0000","last_became_active":"2024-09-10T15:59:16.109611+0000","last_became_peered":"2024-09-10T15:59:16.109611+0000","last_unstale":"2024-09-10T15:59:59.317170+0000","last_undegraded":"2024-09-10T15:59:59.317170+0000","last_fullsized":"2024-09-10T15:59:15.770264+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":5786,"log_dups_size":0,"ondisk_log_size":5786,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23683072,"num_objects":5782,"num_object_clones":0,"num_object_copies":11564,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3346,"num_objects_unfound":0,"num_objects_dirty":5782,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3790,"num_write_kb":15160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":922,"num_bytes_recovered":3776512,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3346},{"shards":"2","objects":2436}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'5864","reported_seq":5939,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:59.193726+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T15:59:59.193726+0000","last_peered":"2024-09-10T15:59:59.193726+0000","last_clean":"2024-09-10T15:59:59.193726+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T15:59:59.193726+0000","last_undegraded":"2024-09-10T15:59:59.193726+0000","last_fullsized":"2024-09-10T15:59:59.193726+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":5864,"log_dups_size":0,"ondisk_log_size":5864,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24018944,"num_objects":5864,"num_object_clones":0,"num_object_copies":11728,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5864,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5864,"num_write_kb":23456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'5962","reported_seq":6086,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:00:01.343222+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:00:01.343222+0000","last_peered":"2024-09-10T16:00:01.343222+0000","last_clean":"2024-09-10T16:00:01.343222+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:00:01.343222+0000","last_undegraded":"2024-09-10T16:00:01.343222+0000","last_fullsized":"2024-09-10T16:00:01.343222+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":5962,"log_dups_size":0,"ondisk_log_size":5962,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24392163,"num_objects":5956,"num_object_clones":0,"num_object_copies":11912,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5956,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2673,"num_write_kb":10692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'6058","reported_seq":6187,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:01.257466+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:00:01.257466+0000","last_peered":"2024-09-10T16:00:01.257466+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:00:01.257466+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":6058,"log_dups_size":0,"ondisk_log_size":6058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24793571,"num_objects":6054,"num_object_clones":0,"num_object_copies":12108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6054,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2670,"num_write_kb":10680,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1683},{"shards":"0,2","objects":4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'5886","reported_seq":6018,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:01.317943+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:00:01.317943+0000","last_peered":"2024-09-10T16:00:01.317943+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:00:01.317943+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":5886,"log_dups_size":0,"ondisk_log_size":5886,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24085446,"num_objects":5882,"num_object_clones":0,"num_object_copies":11764,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5882,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":2385,"num_write_kb":9540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1645},{"shards":"0,2","objects":4237}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'6053","reported_seq":6177,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:00:01.275730+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:00:01.275730+0000","last_peered":"2024-09-10T16:00:01.275730+0000","last_clean":"2024-09-10T16:00:01.275730+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:00:01.275730+0000","last_undegraded":"2024-09-10T16:00:01.275730+0000","last_fullsized":"2024-09-10T16:00:01.275730+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":6053,"log_dups_size":0,"ondisk_log_size":6053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24784896,"num_objects":6051,"num_object_clones":0,"num_object_copies":12102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6051,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2900,"num_write_kb":11600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.775865+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T15:59:15.775865+0000","last_peered":"2024-09-10T15:59:15.775865+0000","last_clean":"2024-09-10T15:59:15.775865+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T15:59:15.775865+0000","last_undegraded":"2024-09-10T15:59:15.775865+0000","last_fullsized":"2024-09-10T15:59:15.775865+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'5639","reported_seq":5773,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:59.215006+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T15:59:59.215006+0000","last_peered":"2024-09-10T15:59:59.215006+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T15:59:59.215006+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":5639,"log_dups_size":0,"ondisk_log_size":5639,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23069155,"num_objects":5633,"num_object_clones":0,"num_object_copies":11266,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5633,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3458,"num_write_kb":13832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4123},{"shards":"2","objects":1510}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":88,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.772572+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T15:59:15.772572+0000","last_peered":"2024-09-10T15:59:15.772572+0000","last_clean":"2024-09-10T15:59:15.772572+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T15:59:15.772572+0000","last_undegraded":"2024-09-10T15:59:15.772572+0000","last_fullsized":"2024-09-10T15:59:15.772572+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'5866","reported_seq":5990,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:01.294307+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:00:01.294307+0000","last_peered":"2024-09-10T16:00:01.294307+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:00:01.294307+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":5866,"log_dups_size":0,"ondisk_log_size":5866,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23994368,"num_objects":5858,"num_object_clones":0,"num_object_copies":11716,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5858,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2370,"num_write_kb":9480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1603},{"shards":"0,2","objects":4255}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":98,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.880567+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T15:59:15.880567+0000","last_peered":"2024-09-10T15:59:15.880567+0000","last_clean":"2024-09-10T15:59:15.880567+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T15:59:15.880567+0000","last_undegraded":"2024-09-10T15:59:15.880567+0000","last_fullsized":"2024-09-10T15:59:15.880567+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'5859","reported_seq":5988,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:59.326139+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T15:59:59.326139+0000","last_peered":"2024-09-10T15:59:59.326139+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T15:59:59.326139+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":5859,"log_dups_size":0,"ondisk_log_size":5859,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23978467,"num_objects":5855,"num_object_clones":0,"num_object_copies":11710,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5855,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3539,"num_write_kb":14156,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4304},{"shards":"2","objects":1551}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.837509+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T15:59:15.837509+0000","last_peered":"2024-09-10T15:59:15.837509+0000","last_clean":"2024-09-10T15:59:15.837509+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T15:59:15.837509+0000","last_undegraded":"2024-09-10T15:59:15.837509+0000","last_fullsized":"2024-09-10T15:59:15.837509+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'5958","reported_seq":6081,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:59.290234+0000","last_change":"2024-09-10T15:59:16.174113+0000","last_active":"2024-09-10T15:59:59.290234+0000","last_peered":"2024-09-10T15:59:59.290234+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T15:59:59.290234+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":5958,"log_dups_size":0,"ondisk_log_size":5958,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24379392,"num_objects":5952,"num_object_clones":0,"num_object_copies":11904,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4354,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5952,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3512,"num_write_kb":14048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4352},{"shards":"2","objects":1600}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'5958","reported_seq":6086,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:01.309121+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:00:01.309121+0000","last_peered":"2024-09-10T16:00:01.309121+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:00:01.309121+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":5958,"log_dups_size":0,"ondisk_log_size":5958,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24395776,"num_objects":5956,"num_object_clones":0,"num_object_copies":11912,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5956,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2589,"num_write_kb":10356,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1652},{"shards":"0,2","objects":4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.989413+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T15:59:15.989413+0000","last_peered":"2024-09-10T15:59:15.989413+0000","last_clean":"2024-09-10T15:59:15.989413+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T15:59:15.989413+0000","last_undegraded":"2024-09-10T15:59:15.989413+0000","last_fullsized":"2024-09-10T15:59:15.989413+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'5838","reported_seq":5975,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:01.317197+0000","last_change":"2024-09-10T15:59:16.251610+0000","last_active":"2024-09-10T16:00:01.317197+0000","last_peered":"2024-09-10T16:00:01.317197+0000","last_clean":"2024-09-10T15:59:14.115595+0000","last_became_active":"2024-09-10T15:59:16.244089+0000","last_became_peered":"2024-09-10T15:59:16.244089+0000","last_unstale":"2024-09-10T16:00:01.317197+0000","last_undegraded":"2024-09-10T15:59:15.847127+0000","last_fullsized":"2024-09-10T15:59:15.774437+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":5838,"log_dups_size":0,"ondisk_log_size":5838,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23879680,"num_objects":5830,"num_object_clones":0,"num_object_copies":11660,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5830,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2228,"num_write_kb":8912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1627},{"shards":"0,2","objects":4203}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":88,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.773583+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T15:59:15.773583+0000","last_peered":"2024-09-10T15:59:15.773583+0000","last_clean":"2024-09-10T15:59:15.773583+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T15:59:15.773583+0000","last_undegraded":"2024-09-10T15:59:15.773583+0000","last_fullsized":"2024-09-10T15:59:15.773583+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'6069","reported_seq":6201,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:01.322209+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:00:01.322209+0000","last_peered":"2024-09-10T16:00:01.322209+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:00:01.322209+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":6069,"log_dups_size":0,"ondisk_log_size":6069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24842240,"num_objects":6065,"num_object_clones":0,"num_object_copies":12130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6065,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2324,"num_write_kb":9296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1694},{"shards":"0,2","objects":4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.916017+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T15:59:15.916017+0000","last_peered":"2024-09-10T15:59:15.916017+0000","last_clean":"2024-09-10T15:59:15.916017+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T15:59:15.916017+0000","last_undegraded":"2024-09-10T15:59:15.916017+0000","last_fullsized":"2024-09-10T15:59:15.916017+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.776033+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T15:59:15.776033+0000","last_peered":"2024-09-10T15:59:15.776033+0000","last_clean":"2024-09-10T15:59:15.776033+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T15:59:15.776033+0000","last_undegraded":"2024-09-10T15:59:15.776033+0000","last_fullsized":"2024-09-10T15:59:15.776033+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'6051","reported_seq":6190,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:01.336089+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:00:01.336089+0000","last_peered":"2024-09-10T16:00:01.336089+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:00:01.336089+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":6051,"log_dups_size":0,"ondisk_log_size":6051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24736710,"num_objects":6041,"num_object_clones":0,"num_object_copies":12082,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6041,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2214,"num_write_kb":8856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1718},{"shards":"0,2","objects":4323}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":387068638,"num_objects":94508,"num_object_clones":0,"num_object_copies":189016,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":47117,"num_objects_misplaced":3346,"num_objects_unfound":0,"num_objects_dirty":94508,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":52354,"num_write_kb":209416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":925,"num_bytes_recovered":3788800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":783360000,"data_stored":783204641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":94580,"ondisk_log_size":94580,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986200,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":504528,"kb_used_data":324288,"kb_used_omap":4,"kb_used_meta":180219,"kb_avail":93867312,"statfs":{"total":96636764160,"available":96120127488,"internally_reserved":0,"allocated":332070912,"data_stored":331793152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":184544938},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,2,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659147,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":186472,"kb_used_data":86744,"kb_used_omap":4,"kb_used_meta":99707,"kb_avail":94185368,"statfs":{"total":96636764160,"available":96445816832,"internally_reserved":0,"allocated":88825856,"data_stored":88497050,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":102100654},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,2,0,2],"upper_bound":32},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182429,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":537908,"kb_used_data":353572,"kb_used_omap":4,"kb_used_meta":184315,"kb_avail":93833932,"statfs":{"total":96636764160,"available":96085946368,"internally_reserved":0,"allocated":362057728,"data_stored":361775891,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":188739242},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,3,4,4,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":362430464,"data_stored":362394334,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":88682496,"data_stored":88599397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":332247040,"data_stored":332210910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:00:05.974 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 93 20 97448 97428 4.09164 2.22266 0.0198532 0.0190853 2024-09-10T16:00:06.974 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 94 20 98027 98007 4.07217 2.26172 0.0454586 0.0191791 2024-09-10T16:00:07.976 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 95 20 98599 98579 4.05275 2.23438 0.0211858 0.0192698 2024-09-10T16:00:08.298 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:00:08.864 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:00:08.865 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:00:08.890 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":122,"stamp":"2024-09-10T16:00:07.559076+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":398213873,"num_objects":97231,"num_object_clones":0,"num_object_copies":194462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":47117,"num_objects_misplaced":3234,"num_objects_unfound":0,"num_objects_dirty":97231,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":55077,"num_write_kb":220302,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1039,"num_bytes_recovered":4247571,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97304,"ondisk_log_size":97304,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1234252,"kb_used_data":778268,"kb_used_omap":12,"kb_used_meta":455923,"kb_avail":281881268,"statfs":{"total":289910292480,"available":288646418432,"internally_reserved":0,"allocated":796946432,"data_stored":796058029,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":466865154},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,3,9,6,5,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":29696000,"num_objects":7250,"num_object_clones":0,"num_object_copies":14500,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-224,"num_objects_unfound":0,"num_objects_dirty":7250,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7250,"num_write_kb":29000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":224,"num_bytes_recovered":917504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003497"},"pg_stats":[{"pgid":"2.f","version":"43'6064","reported_seq":6133,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:06.271775+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:00:06.271775+0000","last_peered":"2024-09-10T16:00:06.271775+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:00:06.271775+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":6064,"log_dups_size":0,"ondisk_log_size":6064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24838144,"num_objects":6064,"num_object_clones":0,"num_object_copies":12128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6064,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6064,"num_write_kb":24256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1805},{"shards":"0,2","objects":4258}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'6010","reported_seq":6117,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:00:04.357598+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:00:04.357598+0000","last_peered":"2024-09-10T16:00:04.357598+0000","last_clean":"2024-09-10T16:00:04.357598+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:00:04.357598+0000","last_undegraded":"2024-09-10T16:00:04.357598+0000","last_fullsized":"2024-09-10T16:00:04.357598+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":6010,"log_dups_size":0,"ondisk_log_size":6010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24593350,"num_objects":6006,"num_object_clones":0,"num_object_copies":12012,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6006,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4115,"num_write_kb":16460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'5973","reported_seq":8166,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T16:00:04.358325+0000","last_change":"2024-09-10T15:59:16.142612+0000","last_active":"2024-09-10T16:00:04.358325+0000","last_peered":"2024-09-10T16:00:04.358325+0000","last_clean":"2024-09-10T15:59:14.092473+0000","last_became_active":"2024-09-10T15:59:16.109611+0000","last_became_peered":"2024-09-10T15:59:16.109611+0000","last_unstale":"2024-09-10T16:00:04.358325+0000","last_undegraded":"2024-09-10T16:00:04.358325+0000","last_fullsized":"2024-09-10T15:59:15.770264+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":5973,"log_dups_size":0,"ondisk_log_size":5973,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24449024,"num_objects":5969,"num_object_clones":0,"num_object_copies":11938,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3234,"num_objects_unfound":0,"num_objects_dirty":5969,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3977,"num_write_kb":15908,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1034,"num_bytes_recovered":4235264,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3234},{"shards":"2","objects":2735}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'6048","reported_seq":6122,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:00:04.281625+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:00:04.281625+0000","last_peered":"2024-09-10T16:00:04.281625+0000","last_clean":"2024-09-10T16:00:04.281625+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:00:04.281625+0000","last_undegraded":"2024-09-10T16:00:04.281625+0000","last_fullsized":"2024-09-10T16:00:04.281625+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":6048,"log_dups_size":0,"ondisk_log_size":6048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24772608,"num_objects":6048,"num_object_clones":0,"num_object_copies":12096,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6048,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6048,"num_write_kb":24192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'6140","reported_seq":6264,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:00:06.307940+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:00:06.307940+0000","last_peered":"2024-09-10T16:00:06.307940+0000","last_clean":"2024-09-10T16:00:06.307940+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:00:06.307940+0000","last_undegraded":"2024-09-10T16:00:06.307940+0000","last_fullsized":"2024-09-10T16:00:06.307940+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":6140,"log_dups_size":0,"ondisk_log_size":6140,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25121251,"num_objects":6134,"num_object_clones":0,"num_object_copies":12268,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6134,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2851,"num_write_kb":11404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'6227","reported_seq":6359,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:06.277641+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:00:06.277641+0000","last_peered":"2024-09-10T16:00:06.277641+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:00:06.277641+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":6227,"log_dups_size":0,"ondisk_log_size":6227,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25485795,"num_objects":6223,"num_object_clones":0,"num_object_copies":12446,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6223,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2839,"num_write_kb":11356,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1852},{"shards":"0,2","objects":4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'6052","reported_seq":6182,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:06.283207+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:00:06.283207+0000","last_peered":"2024-09-10T16:00:06.283207+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:00:06.283207+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":6052,"log_dups_size":0,"ondisk_log_size":6052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24765382,"num_objects":6048,"num_object_clones":0,"num_object_copies":12096,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6048,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":2551,"num_write_kb":10204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1811},{"shards":"0,2","objects":4237}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'6197","reported_seq":6321,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:00:06.290154+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:00:06.290154+0000","last_peered":"2024-09-10T16:00:06.290154+0000","last_clean":"2024-09-10T16:00:06.290154+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:00:06.290154+0000","last_undegraded":"2024-09-10T16:00:06.290154+0000","last_fullsized":"2024-09-10T16:00:06.290154+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":6197,"log_dups_size":0,"ondisk_log_size":6197,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25374720,"num_objects":6195,"num_object_clones":0,"num_object_copies":12390,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6195,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3044,"num_write_kb":12176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.775865+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T15:59:15.775865+0000","last_peered":"2024-09-10T15:59:15.775865+0000","last_clean":"2024-09-10T15:59:15.775865+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T15:59:15.775865+0000","last_undegraded":"2024-09-10T15:59:15.775865+0000","last_fullsized":"2024-09-10T15:59:15.775865+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'5817","reported_seq":5951,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:04.257523+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:00:04.257523+0000","last_peered":"2024-09-10T16:00:04.257523+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:00:04.257523+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":5817,"log_dups_size":0,"ondisk_log_size":5817,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23798243,"num_objects":5811,"num_object_clones":0,"num_object_copies":11622,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5811,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3636,"num_write_kb":14544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4123},{"shards":"2","objects":1688}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":88,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.772572+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T15:59:15.772572+0000","last_peered":"2024-09-10T15:59:15.772572+0000","last_clean":"2024-09-10T15:59:15.772572+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T15:59:15.772572+0000","last_undegraded":"2024-09-10T15:59:15.772572+0000","last_fullsized":"2024-09-10T15:59:15.772572+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'6035","reported_seq":6160,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:06.332640+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:00:06.332640+0000","last_peered":"2024-09-10T16:00:06.332640+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:00:06.332640+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":6035,"log_dups_size":0,"ondisk_log_size":6035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24686592,"num_objects":6027,"num_object_clones":0,"num_object_copies":12054,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6027,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2539,"num_write_kb":10156,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1772},{"shards":"0,2","objects":4255}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":99,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.884297+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T15:59:15.884297+0000","last_peered":"2024-09-10T15:59:15.884297+0000","last_clean":"2024-09-10T15:59:15.884297+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T15:59:15.884297+0000","last_undegraded":"2024-09-10T15:59:15.884297+0000","last_fullsized":"2024-09-10T15:59:15.884297+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'6022","reported_seq":6151,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:04.373809+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:00:04.373809+0000","last_peered":"2024-09-10T16:00:04.373809+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:00:04.373809+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":6022,"log_dups_size":0,"ondisk_log_size":6022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24646115,"num_objects":6018,"num_object_clones":0,"num_object_copies":12036,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6018,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3702,"num_write_kb":14808,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4304},{"shards":"2","objects":1714}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.837509+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T15:59:15.837509+0000","last_peered":"2024-09-10T15:59:15.837509+0000","last_clean":"2024-09-10T15:59:15.837509+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T15:59:15.837509+0000","last_undegraded":"2024-09-10T15:59:15.837509+0000","last_fullsized":"2024-09-10T15:59:15.837509+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'6143","reported_seq":6265,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:04.372473+0000","last_change":"2024-09-10T15:59:16.174113+0000","last_active":"2024-09-10T16:00:04.372473+0000","last_peered":"2024-09-10T16:00:04.372473+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:00:04.372473+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":6143,"log_dups_size":0,"ondisk_log_size":6143,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25137152,"num_objects":6137,"num_object_clones":0,"num_object_copies":12274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4354,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6137,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3697,"num_write_kb":14788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4352},{"shards":"2","objects":1785}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'6121","reported_seq":6250,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:06.302724+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:00:06.302724+0000","last_peered":"2024-09-10T16:00:06.302724+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:00:06.302724+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":6121,"log_dups_size":0,"ondisk_log_size":6121,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25063424,"num_objects":6119,"num_object_clones":0,"num_object_copies":12238,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6119,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2752,"num_write_kb":11008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1815},{"shards":"0,2","objects":4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.989413+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T15:59:15.989413+0000","last_peered":"2024-09-10T15:59:15.989413+0000","last_clean":"2024-09-10T15:59:15.989413+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T15:59:15.989413+0000","last_undegraded":"2024-09-10T15:59:15.989413+0000","last_fullsized":"2024-09-10T15:59:15.989413+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'5983","reported_seq":6120,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:06.344526+0000","last_change":"2024-09-10T15:59:16.251610+0000","last_active":"2024-09-10T16:00:06.344526+0000","last_peered":"2024-09-10T16:00:06.344526+0000","last_clean":"2024-09-10T15:59:14.115595+0000","last_became_active":"2024-09-10T15:59:16.244089+0000","last_became_peered":"2024-09-10T15:59:16.244089+0000","last_unstale":"2024-09-10T16:00:06.344526+0000","last_undegraded":"2024-09-10T15:59:15.847127+0000","last_fullsized":"2024-09-10T15:59:15.774437+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":5983,"log_dups_size":0,"ondisk_log_size":5983,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24473600,"num_objects":5975,"num_object_clones":0,"num_object_copies":11950,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5975,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2373,"num_write_kb":9492,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1772},{"shards":"0,2","objects":4203}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":88,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.773583+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T15:59:15.773583+0000","last_peered":"2024-09-10T15:59:15.773583+0000","last_clean":"2024-09-10T15:59:15.773583+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T15:59:15.773583+0000","last_undegraded":"2024-09-10T15:59:15.773583+0000","last_fullsized":"2024-09-10T15:59:15.773583+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'6240","reported_seq":6376,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:06.204545+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:00:06.204545+0000","last_peered":"2024-09-10T16:00:06.204545+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:00:06.204545+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":6240,"log_dups_size":0,"ondisk_log_size":6240,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25542656,"num_objects":6236,"num_object_clones":0,"num_object_copies":12472,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6236,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2495,"num_write_kb":9980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1865},{"shards":"0,2","objects":4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.916017+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T15:59:15.916017+0000","last_peered":"2024-09-10T15:59:15.916017+0000","last_clean":"2024-09-10T15:59:15.916017+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T15:59:15.916017+0000","last_undegraded":"2024-09-10T15:59:15.916017+0000","last_fullsized":"2024-09-10T15:59:15.916017+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.776033+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T15:59:15.776033+0000","last_peered":"2024-09-10T15:59:15.776033+0000","last_clean":"2024-09-10T15:59:15.776033+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T15:59:15.776033+0000","last_undegraded":"2024-09-10T15:59:15.776033+0000","last_fullsized":"2024-09-10T15:59:15.776033+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'6229","reported_seq":6368,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:06.329528+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:00:06.329528+0000","last_peered":"2024-09-10T16:00:06.329528+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:00:06.329528+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":6229,"log_dups_size":0,"ondisk_log_size":6229,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25465798,"num_objects":6219,"num_object_clones":0,"num_object_copies":12438,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6219,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2392,"num_write_kb":9568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1896},{"shards":"0,2","objects":4323}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":398213854,"num_objects":97229,"num_object_clones":0,"num_object_copies":194458,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":47117,"num_objects_misplaced":3234,"num_objects_unfound":0,"num_objects_dirty":97229,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":55075,"num_write_kb":220300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1037,"num_bytes_recovered":4247552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":797237248,"data_stored":797081889,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97301,"ondisk_log_size":97301,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986201,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":514152,"kb_used_data":329752,"kb_used_omap":4,"kb_used_meta":184379,"kb_avail":93857688,"statfs":{"total":96636764160,"available":96110272512,"internally_reserved":0,"allocated":337666048,"data_stored":337388288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":188804778},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,3,2,2,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659147,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":186472,"kb_used_data":86744,"kb_used_omap":4,"kb_used_meta":99707,"kb_avail":94185368,"statfs":{"total":96636764160,"available":96445816832,"internally_reserved":0,"allocated":88825856,"data_stored":88497050,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":102100654},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,2,0,2],"upper_bound":32},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182430,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":533628,"kb_used_data":361772,"kb_used_omap":4,"kb_used_meta":171835,"kb_avail":93838212,"statfs":{"total":96636764160,"available":96090329088,"internally_reserved":0,"allocated":370454528,"data_stored":370172691,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":175959722},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,6,2,3,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":370638848,"data_stored":370602718,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":88682496,"data_stored":88599397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":337915904,"data_stored":337879774,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:00:08.892 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:00:08.976 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 96 20 99137 99117 4.03242 2.10156 0.0341615 0.0193671 2024-09-10T16:00:09.450 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:00:09.450 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:00:09.475 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":122,"stamp":"2024-09-10T16:00:07.559076+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":398213873,"num_objects":97231,"num_object_clones":0,"num_object_copies":194462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":47117,"num_objects_misplaced":3234,"num_objects_unfound":0,"num_objects_dirty":97231,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":55077,"num_write_kb":220302,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1039,"num_bytes_recovered":4247571,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97304,"ondisk_log_size":97304,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1234252,"kb_used_data":778268,"kb_used_omap":12,"kb_used_meta":455923,"kb_avail":281881268,"statfs":{"total":289910292480,"available":288646418432,"internally_reserved":0,"allocated":796946432,"data_stored":796058029,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":466865154},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,3,9,6,5,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":29696000,"num_objects":7250,"num_object_clones":0,"num_object_copies":14500,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-224,"num_objects_unfound":0,"num_objects_dirty":7250,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7250,"num_write_kb":29000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":224,"num_bytes_recovered":917504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003497"},"pg_stats":[{"pgid":"2.f","version":"43'6064","reported_seq":6133,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:06.271775+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:00:06.271775+0000","last_peered":"2024-09-10T16:00:06.271775+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:00:06.271775+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":6064,"log_dups_size":0,"ondisk_log_size":6064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24838144,"num_objects":6064,"num_object_clones":0,"num_object_copies":12128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6064,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6064,"num_write_kb":24256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1805},{"shards":"0,2","objects":4258}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'6010","reported_seq":6117,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:00:04.357598+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:00:04.357598+0000","last_peered":"2024-09-10T16:00:04.357598+0000","last_clean":"2024-09-10T16:00:04.357598+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:00:04.357598+0000","last_undegraded":"2024-09-10T16:00:04.357598+0000","last_fullsized":"2024-09-10T16:00:04.357598+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":6010,"log_dups_size":0,"ondisk_log_size":6010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24593350,"num_objects":6006,"num_object_clones":0,"num_object_copies":12012,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6006,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4115,"num_write_kb":16460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'5973","reported_seq":8166,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T16:00:04.358325+0000","last_change":"2024-09-10T15:59:16.142612+0000","last_active":"2024-09-10T16:00:04.358325+0000","last_peered":"2024-09-10T16:00:04.358325+0000","last_clean":"2024-09-10T15:59:14.092473+0000","last_became_active":"2024-09-10T15:59:16.109611+0000","last_became_peered":"2024-09-10T15:59:16.109611+0000","last_unstale":"2024-09-10T16:00:04.358325+0000","last_undegraded":"2024-09-10T16:00:04.358325+0000","last_fullsized":"2024-09-10T15:59:15.770264+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":5973,"log_dups_size":0,"ondisk_log_size":5973,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24449024,"num_objects":5969,"num_object_clones":0,"num_object_copies":11938,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3234,"num_objects_unfound":0,"num_objects_dirty":5969,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3977,"num_write_kb":15908,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1034,"num_bytes_recovered":4235264,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3234},{"shards":"2","objects":2735}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'6048","reported_seq":6122,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:00:04.281625+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:00:04.281625+0000","last_peered":"2024-09-10T16:00:04.281625+0000","last_clean":"2024-09-10T16:00:04.281625+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:00:04.281625+0000","last_undegraded":"2024-09-10T16:00:04.281625+0000","last_fullsized":"2024-09-10T16:00:04.281625+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":6048,"log_dups_size":0,"ondisk_log_size":6048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24772608,"num_objects":6048,"num_object_clones":0,"num_object_copies":12096,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6048,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6048,"num_write_kb":24192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'6140","reported_seq":6264,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:00:06.307940+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:00:06.307940+0000","last_peered":"2024-09-10T16:00:06.307940+0000","last_clean":"2024-09-10T16:00:06.307940+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:00:06.307940+0000","last_undegraded":"2024-09-10T16:00:06.307940+0000","last_fullsized":"2024-09-10T16:00:06.307940+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":6140,"log_dups_size":0,"ondisk_log_size":6140,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25121251,"num_objects":6134,"num_object_clones":0,"num_object_copies":12268,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6134,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2851,"num_write_kb":11404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'6227","reported_seq":6359,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:06.277641+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:00:06.277641+0000","last_peered":"2024-09-10T16:00:06.277641+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:00:06.277641+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":6227,"log_dups_size":0,"ondisk_log_size":6227,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25485795,"num_objects":6223,"num_object_clones":0,"num_object_copies":12446,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6223,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2839,"num_write_kb":11356,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1852},{"shards":"0,2","objects":4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'6052","reported_seq":6182,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:06.283207+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:00:06.283207+0000","last_peered":"2024-09-10T16:00:06.283207+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:00:06.283207+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":6052,"log_dups_size":0,"ondisk_log_size":6052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24765382,"num_objects":6048,"num_object_clones":0,"num_object_copies":12096,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6048,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":2551,"num_write_kb":10204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1811},{"shards":"0,2","objects":4237}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'6197","reported_seq":6321,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:00:06.290154+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:00:06.290154+0000","last_peered":"2024-09-10T16:00:06.290154+0000","last_clean":"2024-09-10T16:00:06.290154+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:00:06.290154+0000","last_undegraded":"2024-09-10T16:00:06.290154+0000","last_fullsized":"2024-09-10T16:00:06.290154+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":6197,"log_dups_size":0,"ondisk_log_size":6197,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25374720,"num_objects":6195,"num_object_clones":0,"num_object_copies":12390,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6195,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3044,"num_write_kb":12176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.775865+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T15:59:15.775865+0000","last_peered":"2024-09-10T15:59:15.775865+0000","last_clean":"2024-09-10T15:59:15.775865+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T15:59:15.775865+0000","last_undegraded":"2024-09-10T15:59:15.775865+0000","last_fullsized":"2024-09-10T15:59:15.775865+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'5817","reported_seq":5951,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:04.257523+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:00:04.257523+0000","last_peered":"2024-09-10T16:00:04.257523+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:00:04.257523+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":5817,"log_dups_size":0,"ondisk_log_size":5817,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23798243,"num_objects":5811,"num_object_clones":0,"num_object_copies":11622,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5811,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3636,"num_write_kb":14544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4123},{"shards":"2","objects":1688}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":88,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.772572+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T15:59:15.772572+0000","last_peered":"2024-09-10T15:59:15.772572+0000","last_clean":"2024-09-10T15:59:15.772572+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T15:59:15.772572+0000","last_undegraded":"2024-09-10T15:59:15.772572+0000","last_fullsized":"2024-09-10T15:59:15.772572+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'6035","reported_seq":6160,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:06.332640+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:00:06.332640+0000","last_peered":"2024-09-10T16:00:06.332640+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:00:06.332640+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":6035,"log_dups_size":0,"ondisk_log_size":6035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24686592,"num_objects":6027,"num_object_clones":0,"num_object_copies":12054,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6027,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2539,"num_write_kb":10156,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1772},{"shards":"0,2","objects":4255}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":99,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.884297+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T15:59:15.884297+0000","last_peered":"2024-09-10T15:59:15.884297+0000","last_clean":"2024-09-10T15:59:15.884297+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T15:59:15.884297+0000","last_undegraded":"2024-09-10T15:59:15.884297+0000","last_fullsized":"2024-09-10T15:59:15.884297+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'6022","reported_seq":6151,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:04.373809+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:00:04.373809+0000","last_peered":"2024-09-10T16:00:04.373809+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:00:04.373809+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":6022,"log_dups_size":0,"ondisk_log_size":6022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24646115,"num_objects":6018,"num_object_clones":0,"num_object_copies":12036,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6018,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3702,"num_write_kb":14808,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4304},{"shards":"2","objects":1714}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.837509+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T15:59:15.837509+0000","last_peered":"2024-09-10T15:59:15.837509+0000","last_clean":"2024-09-10T15:59:15.837509+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T15:59:15.837509+0000","last_undegraded":"2024-09-10T15:59:15.837509+0000","last_fullsized":"2024-09-10T15:59:15.837509+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'6143","reported_seq":6265,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:04.372473+0000","last_change":"2024-09-10T15:59:16.174113+0000","last_active":"2024-09-10T16:00:04.372473+0000","last_peered":"2024-09-10T16:00:04.372473+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:00:04.372473+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":6143,"log_dups_size":0,"ondisk_log_size":6143,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25137152,"num_objects":6137,"num_object_clones":0,"num_object_copies":12274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4354,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6137,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3697,"num_write_kb":14788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4352},{"shards":"2","objects":1785}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'6121","reported_seq":6250,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:06.302724+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:00:06.302724+0000","last_peered":"2024-09-10T16:00:06.302724+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:00:06.302724+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":6121,"log_dups_size":0,"ondisk_log_size":6121,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25063424,"num_objects":6119,"num_object_clones":0,"num_object_copies":12238,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6119,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2752,"num_write_kb":11008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1815},{"shards":"0,2","objects":4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.989413+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T15:59:15.989413+0000","last_peered":"2024-09-10T15:59:15.989413+0000","last_clean":"2024-09-10T15:59:15.989413+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T15:59:15.989413+0000","last_undegraded":"2024-09-10T15:59:15.989413+0000","last_fullsized":"2024-09-10T15:59:15.989413+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'5983","reported_seq":6120,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:06.344526+0000","last_change":"2024-09-10T15:59:16.251610+0000","last_active":"2024-09-10T16:00:06.344526+0000","last_peered":"2024-09-10T16:00:06.344526+0000","last_clean":"2024-09-10T15:59:14.115595+0000","last_became_active":"2024-09-10T15:59:16.244089+0000","last_became_peered":"2024-09-10T15:59:16.244089+0000","last_unstale":"2024-09-10T16:00:06.344526+0000","last_undegraded":"2024-09-10T15:59:15.847127+0000","last_fullsized":"2024-09-10T15:59:15.774437+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":5983,"log_dups_size":0,"ondisk_log_size":5983,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24473600,"num_objects":5975,"num_object_clones":0,"num_object_copies":11950,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5975,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2373,"num_write_kb":9492,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1772},{"shards":"0,2","objects":4203}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":88,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.773583+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T15:59:15.773583+0000","last_peered":"2024-09-10T15:59:15.773583+0000","last_clean":"2024-09-10T15:59:15.773583+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T15:59:15.773583+0000","last_undegraded":"2024-09-10T15:59:15.773583+0000","last_fullsized":"2024-09-10T15:59:15.773583+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'6240","reported_seq":6376,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:06.204545+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:00:06.204545+0000","last_peered":"2024-09-10T16:00:06.204545+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:00:06.204545+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":6240,"log_dups_size":0,"ondisk_log_size":6240,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25542656,"num_objects":6236,"num_object_clones":0,"num_object_copies":12472,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6236,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2495,"num_write_kb":9980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1865},{"shards":"0,2","objects":4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.916017+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T15:59:15.916017+0000","last_peered":"2024-09-10T15:59:15.916017+0000","last_clean":"2024-09-10T15:59:15.916017+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T15:59:15.916017+0000","last_undegraded":"2024-09-10T15:59:15.916017+0000","last_fullsized":"2024-09-10T15:59:15.916017+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.776033+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T15:59:15.776033+0000","last_peered":"2024-09-10T15:59:15.776033+0000","last_clean":"2024-09-10T15:59:15.776033+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T15:59:15.776033+0000","last_undegraded":"2024-09-10T15:59:15.776033+0000","last_fullsized":"2024-09-10T15:59:15.776033+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'6229","reported_seq":6368,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:06.329528+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:00:06.329528+0000","last_peered":"2024-09-10T16:00:06.329528+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:00:06.329528+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":6229,"log_dups_size":0,"ondisk_log_size":6229,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25465798,"num_objects":6219,"num_object_clones":0,"num_object_copies":12438,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6219,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2392,"num_write_kb":9568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1896},{"shards":"0,2","objects":4323}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":398213854,"num_objects":97229,"num_object_clones":0,"num_object_copies":194458,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":47117,"num_objects_misplaced":3234,"num_objects_unfound":0,"num_objects_dirty":97229,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":55075,"num_write_kb":220300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1037,"num_bytes_recovered":4247552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":797237248,"data_stored":797081889,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97301,"ondisk_log_size":97301,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986201,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":514152,"kb_used_data":329752,"kb_used_omap":4,"kb_used_meta":184379,"kb_avail":93857688,"statfs":{"total":96636764160,"available":96110272512,"internally_reserved":0,"allocated":337666048,"data_stored":337388288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":188804778},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,3,2,2,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659147,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":186472,"kb_used_data":86744,"kb_used_omap":4,"kb_used_meta":99707,"kb_avail":94185368,"statfs":{"total":96636764160,"available":96445816832,"internally_reserved":0,"allocated":88825856,"data_stored":88497050,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":102100654},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,2,0,2],"upper_bound":32},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182430,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":533628,"kb_used_data":361772,"kb_used_omap":4,"kb_used_meta":171835,"kb_avail":93838212,"statfs":{"total":96636764160,"available":96090329088,"internally_reserved":0,"allocated":370454528,"data_stored":370172691,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":175959722},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,6,2,3,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":370638848,"data_stored":370602718,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":88682496,"data_stored":88599397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":337915904,"data_stored":337879774,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:00:09.476 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-10T16:00:09.476 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "43'6064", 'reported_seq': 6133, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:00:06.271775+0000', 'last_change': '2024-09-10T15:59:16.281558+0000', 'last_active': '2024-09-10T16:00:06.271775+0000', 'last_peered': '2024-09-10T16:00:06.271775+0000', 'last_clean': '2024-09-10T15:59:14.132456+0000', 'last_became_active': '2024-09-10T15:59:16.274407+0000', 'last_became_peered': '2024-09-10T15:59:16.274407+0000', 'last_unstale': '2024-09-10T16:00:06.271775+0000', 'last_undegraded': '2024-09-10T15:59:15.851279+0000', 'last_fullsized': '2024-09-10T15:59:15.774356+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'objects_scrubbed': 0, 'log_size': 6064, 'log_dups_size': 0, 'ondisk_log_size': 6064, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24838144, 'num_objects': 6064, 'num_object_clones': 0, 'num_object_copies': 12128, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4259, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6064, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6064, 'num_write_kb': 24256, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1805}, {'shards': '0,2', 'objects': 4258}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:00:09.477 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-10T16:00:09.477 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "43'5973", 'reported_seq': 8166, 'reported_epoch': 43, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-10T16:00:04.358325+0000', 'last_change': '2024-09-10T15:59:16.142612+0000', 'last_active': '2024-09-10T16:00:04.358325+0000', 'last_peered': '2024-09-10T16:00:04.358325+0000', 'last_clean': '2024-09-10T15:59:14.092473+0000', 'last_became_active': '2024-09-10T15:59:16.109611+0000', 'last_became_peered': '2024-09-10T15:59:16.109611+0000', 'last_unstale': '2024-09-10T16:00:04.358325+0000', 'last_undegraded': '2024-09-10T16:00:04.358325+0000', 'last_fullsized': '2024-09-10T15:59:15.770264+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2029", 'last_scrub_stamp': '2024-09-10T15:58:45.933789+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:45.933789+0000', 'objects_scrubbed': 1992, 'log_size': 5973, 'log_dups_size': 0, 'ondisk_log_size': 5973, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 668, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24449024, 'num_objects': 5969, 'num_object_clones': 0, 'num_object_copies': 11938, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 3234, 'num_objects_unfound': 0, 'num_objects_dirty': 5969, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3977, 'num_write_kb': 15908, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1034, 'num_bytes_recovered': 4235264, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3234}, {'shards': '2', 'objects': 2735}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:00:09.477 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-10T16:00:09.477 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "43'6227", 'reported_seq': 6359, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:00:06.277641+0000', 'last_change': '2024-09-10T15:59:16.308264+0000', 'last_active': '2024-09-10T16:00:06.277641+0000', 'last_peered': '2024-09-10T16:00:06.277641+0000', 'last_clean': '2024-09-10T15:59:14.111551+0000', 'last_became_active': '2024-09-10T15:59:16.300040+0000', 'last_became_peered': '2024-09-10T15:59:16.300040+0000', 'last_unstale': '2024-09-10T16:00:06.277641+0000', 'last_undegraded': '2024-09-10T15:59:15.878234+0000', 'last_fullsized': '2024-09-10T15:59:15.769328+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3442", 'last_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'objects_scrubbed': 3384, 'log_size': 6227, 'log_dups_size': 0, 'ondisk_log_size': 6227, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1292, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25485795, 'num_objects': 6223, 'num_object_clones': 0, 'num_object_copies': 12446, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4372, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6223, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 2839, 'num_write_kb': 11356, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1852}, {'shards': '0,2', 'objects': 4371}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:00:09.477 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-10T16:00:09.477 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "43'6052", 'reported_seq': 6182, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:00:06.283207+0000', 'last_change': '2024-09-10T15:59:16.331899+0000', 'last_active': '2024-09-10T16:00:06.283207+0000', 'last_peered': '2024-09-10T16:00:06.283207+0000', 'last_clean': '2024-09-10T15:59:14.139436+0000', 'last_became_active': '2024-09-10T15:59:16.318715+0000', 'last_became_peered': '2024-09-10T15:59:16.318715+0000', 'last_unstale': '2024-09-10T16:00:06.283207+0000', 'last_undegraded': '2024-09-10T15:59:15.877524+0000', 'last_fullsized': '2024-09-10T15:59:15.769042+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3537", 'last_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'objects_scrubbed': 3497, 'log_size': 6052, 'log_dups_size': 0, 'ondisk_log_size': 6052, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1224, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24765382, 'num_objects': 6048, 'num_object_clones': 0, 'num_object_copies': 12096, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4237, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6048, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 2551, 'num_write_kb': 10204, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1811}, {'shards': '0,2', 'objects': 4237}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:00:09.477 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-10T16:00:09.477 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "43'5817", 'reported_seq': 5951, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:00:04.257523+0000', 'last_change': '2024-09-10T15:59:16.175680+0000', 'last_active': '2024-09-10T16:00:04.257523+0000', 'last_peered': '2024-09-10T16:00:04.257523+0000', 'last_clean': '2024-09-10T15:59:14.088187+0000', 'last_became_active': '2024-09-10T15:59:16.165810+0000', 'last_became_peered': '2024-09-10T15:59:16.165810+0000', 'last_unstale': '2024-09-10T16:00:04.257523+0000', 'last_undegraded': '2024-09-10T15:59:15.822098+0000', 'last_fullsized': '2024-09-10T15:59:15.769123+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2218", 'last_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'objects_scrubbed': 2175, 'log_size': 5817, 'log_dups_size': 0, 'ondisk_log_size': 5817, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 825, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23798243, 'num_objects': 5811, 'num_object_clones': 0, 'num_object_copies': 11622, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4124, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5811, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 3636, 'num_write_kb': 14544, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4123}, {'shards': '2', 'objects': 1688}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:00:09.477 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-10T16:00:09.477 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "43'6035", 'reported_seq': 6160, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:00:06.332640+0000', 'last_change': '2024-09-10T15:59:16.277742+0000', 'last_active': '2024-09-10T16:00:06.332640+0000', 'last_peered': '2024-09-10T16:00:06.332640+0000', 'last_clean': '2024-09-10T15:59:14.129081+0000', 'last_became_active': '2024-09-10T15:59:16.269835+0000', 'last_became_peered': '2024-09-10T15:59:16.269835+0000', 'last_unstale': '2024-09-10T16:00:06.332640+0000', 'last_undegraded': '2024-09-10T15:59:15.860857+0000', 'last_fullsized': '2024-09-10T15:59:15.771410+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3557", 'last_scrub_stamp': '2024-09-10T15:59:02.493678+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:02.493678+0000', 'objects_scrubbed': 3488, 'log_size': 6035, 'log_dups_size': 0, 'ondisk_log_size': 6035, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1472, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24686592, 'num_objects': 6027, 'num_object_clones': 0, 'num_object_copies': 12054, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4257, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6027, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2539, 'num_write_kb': 10156, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1772}, {'shards': '0,2', 'objects': 4255}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:00:09.477 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-10T16:00:09.478 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "43'6022", 'reported_seq': 6151, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:00:04.373809+0000', 'last_change': '2024-09-10T15:59:16.284311+0000', 'last_active': '2024-09-10T16:00:04.373809+0000', 'last_peered': '2024-09-10T16:00:04.373809+0000', 'last_clean': '2024-09-10T15:59:14.116041+0000', 'last_became_active': '2024-09-10T15:59:16.274543+0000', 'last_became_peered': '2024-09-10T15:59:16.274543+0000', 'last_unstale': '2024-09-10T16:00:04.373809+0000', 'last_undegraded': '2024-09-10T15:59:15.812484+0000', 'last_fullsized': '2024-09-10T15:59:15.770885+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2356", 'last_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'objects_scrubbed': 2316, 'log_size': 6022, 'log_dups_size': 0, 'ondisk_log_size': 6022, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 783, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24646115, 'num_objects': 6018, 'num_object_clones': 0, 'num_object_copies': 12036, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6018, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 3702, 'num_write_kb': 14808, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4304}, {'shards': '2', 'objects': 1714}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:00:09.478 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-10T16:00:09.478 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "43'6143", 'reported_seq': 6265, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:00:04.372473+0000', 'last_change': '2024-09-10T15:59:16.174113+0000', 'last_active': '2024-09-10T16:00:04.372473+0000', 'last_peered': '2024-09-10T16:00:04.372473+0000', 'last_clean': '2024-09-10T15:59:14.110654+0000', 'last_became_active': '2024-09-10T15:59:16.164645+0000', 'last_became_peered': '2024-09-10T15:59:16.164645+0000', 'last_unstale': '2024-09-10T16:00:04.372473+0000', 'last_undegraded': '2024-09-10T15:59:15.803593+0000', 'last_fullsized': '2024-09-10T15:59:15.769083+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2486", 'last_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'objects_scrubbed': 2440, 'log_size': 6143, 'log_dups_size': 0, 'ondisk_log_size': 6143, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 792, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25137152, 'num_objects': 6137, 'num_object_clones': 0, 'num_object_copies': 12274, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4354, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6137, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3697, 'num_write_kb': 14788, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4352}, {'shards': '2', 'objects': 1785}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:00:09.478 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-10T16:00:09.478 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "43'6121", 'reported_seq': 6250, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:00:06.302724+0000', 'last_change': '2024-09-10T15:59:16.315641+0000', 'last_active': '2024-09-10T16:00:06.302724+0000', 'last_peered': '2024-09-10T16:00:06.302724+0000', 'last_clean': '2024-09-10T15:59:14.117001+0000', 'last_became_active': '2024-09-10T15:59:16.302245+0000', 'last_became_peered': '2024-09-10T15:59:16.302245+0000', 'last_unstale': '2024-09-10T16:00:06.302724+0000', 'last_undegraded': '2024-09-10T15:59:15.845012+0000', 'last_fullsized': '2024-09-10T15:59:15.770337+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3404", 'last_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'objects_scrubbed': 3367, 'log_size': 6121, 'log_dups_size': 0, 'ondisk_log_size': 6121, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 983, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25063424, 'num_objects': 6119, 'num_object_clones': 0, 'num_object_copies': 12238, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6119, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2752, 'num_write_kb': 11008, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1815}, {'shards': '0,2', 'objects': 4304}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:00:09.478 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-10T16:00:09.478 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "43'5983", 'reported_seq': 6120, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:00:06.344526+0000', 'last_change': '2024-09-10T15:59:16.251610+0000', 'last_active': '2024-09-10T16:00:06.344526+0000', 'last_peered': '2024-09-10T16:00:06.344526+0000', 'last_clean': '2024-09-10T15:59:14.115595+0000', 'last_became_active': '2024-09-10T15:59:16.244089+0000', 'last_became_peered': '2024-09-10T15:59:16.244089+0000', 'last_unstale': '2024-09-10T16:00:06.344526+0000', 'last_undegraded': '2024-09-10T15:59:15.847127+0000', 'last_fullsized': '2024-09-10T15:59:15.774437+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3653", 'last_scrub_stamp': '2024-09-10T15:59:05.298468+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:05.298468+0000', 'objects_scrubbed': 3602, 'log_size': 5983, 'log_dups_size': 0, 'ondisk_log_size': 5983, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1289, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24473600, 'num_objects': 5975, 'num_object_clones': 0, 'num_object_copies': 11950, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4206, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5975, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2373, 'num_write_kb': 9492, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1772}, {'shards': '0,2', 'objects': 4203}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:00:09.478 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-10T16:00:09.478 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "43'6240", 'reported_seq': 6376, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:00:06.204545+0000', 'last_change': '2024-09-10T15:59:16.311291+0000', 'last_active': '2024-09-10T16:00:06.204545+0000', 'last_peered': '2024-09-10T16:00:06.204545+0000', 'last_clean': '2024-09-10T15:59:14.116363+0000', 'last_became_active': '2024-09-10T15:59:16.301244+0000', 'last_became_peered': '2024-09-10T15:59:16.301244+0000', 'last_unstale': '2024-09-10T16:00:06.204545+0000', 'last_undegraded': '2024-09-10T15:59:15.883434+0000', 'last_fullsized': '2024-09-10T15:59:15.772756+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3789", 'last_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'objects_scrubbed': 3741, 'log_size': 6240, 'log_dups_size': 0, 'ondisk_log_size': 6240, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1440, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25542656, 'num_objects': 6236, 'num_object_clones': 0, 'num_object_copies': 12472, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4373, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6236, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2495, 'num_write_kb': 9980, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1865}, {'shards': '0,2', 'objects': 4371}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:00:09.478 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-10T16:00:09.479 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "43'6229", 'reported_seq': 6368, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:00:06.329528+0000', 'last_change': '2024-09-10T15:59:16.313409+0000', 'last_active': '2024-09-10T16:00:06.329528+0000', 'last_peered': '2024-09-10T16:00:06.329528+0000', 'last_clean': '2024-09-10T15:59:14.129780+0000', 'last_became_active': '2024-09-10T15:59:16.300932+0000', 'last_became_peered': '2024-09-10T15:59:16.300932+0000', 'last_unstale': '2024-09-10T16:00:06.329528+0000', 'last_undegraded': '2024-09-10T15:59:15.887823+0000', 'last_fullsized': '2024-09-10T15:59:15.768513+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3880", 'last_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'objects_scrubbed': 3827, 'log_size': 6229, 'log_dups_size': 0, 'ondisk_log_size': 6229, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1397, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25465798, 'num_objects': 6219, 'num_object_clones': 0, 'num_object_copies': 12438, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4325, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6219, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2392, 'num_write_kb': 9568, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1896}, {'shards': '0,2', 'objects': 4323}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:00:09.479 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T16:00:09.978 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 97 20 99635 99615 4.0109 1.94531 0.0423313 0.0194703 2024-09-10T16:00:10.164 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:00:10.164 INFO:teuthology.orchestra.run.smithi115.stdout:{"fsid":"7effee29-df05-4351-9741-ee203564ed81","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 47117/196570 objects degraded (23.970%), 11 pgs degraded","count":11},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":174,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":43,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983954,"num_in_osds":3,"osd_in_since":1725983947,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":98285,"data_bytes":402531057,"bytes_used":1274392576,"bytes_avail":288635899904,"bytes_total":289910292480,"degraded_objects":47117,"degraded_total":196570,"degraded_ratio":0.23969578267283922,"misplaced_objects":3125,"misplaced_total":196570,"misplaced_ratio":0.015897644604975326,"recovering_objects_per_sec":27,"recovering_bytes_per_sec":113629,"recovering_keys_per_sec":0,"num_objects_recovered":333,"num_bytes_recovered":1363968,"num_keys_recovered":0,"write_bytes_sec":2244274,"read_op_per_sec":0,"write_op_per_sec":547},"fsmap":{"epoch":1,"btime":"2024-09-10T15:57:14:967120+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":5,"modified":"2024-09-10T15:59:19.546323+0000","services":{"osd":{"daemons":{"summary":"","1":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-10T16:00:10.191 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-10T16:00:10.191 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:00:10.730 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:00:10.731 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:00:10.762 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":123,"stamp":"2024-09-10T16:00:09.559642+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":402531057,"num_objects":98285,"num_object_clones":0,"num_object_copies":196570,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":47117,"num_objects_misplaced":3125,"num_objects_unfound":0,"num_objects_dirty":98285,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":56131,"num_write_kb":224518,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1148,"num_bytes_recovered":4694035,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":98358,"ondisk_log_size":98358,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1244524,"kb_used_data":792700,"kb_used_omap":12,"kb_used_meta":451763,"kb_avail":281870996,"statfs":{"total":289910292480,"available":288635899904,"internally_reserved":0,"allocated":811724800,"data_stored":810836397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":462605314},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,11,6,7,5,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":26939392,"num_objects":6577,"num_object_clones":0,"num_object_copies":13154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-333,"num_objects_unfound":0,"num_objects_dirty":6577,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6577,"num_write_kb":26308,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":333,"num_bytes_recovered":1363968,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003609"},"pg_stats":[{"pgid":"2.f","version":"43'6064","reported_seq":6133,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:06.271775+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:00:06.271775+0000","last_peered":"2024-09-10T16:00:06.271775+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:00:06.271775+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":6064,"log_dups_size":0,"ondisk_log_size":6064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24838144,"num_objects":6064,"num_object_clones":0,"num_object_copies":12128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6064,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6064,"num_write_kb":24256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1805},{"shards":"0,2","objects":4258}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'6206","reported_seq":6313,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:00:09.301964+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:00:09.301964+0000","last_peered":"2024-09-10T16:00:09.301964+0000","last_clean":"2024-09-10T16:00:09.301964+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:00:09.301964+0000","last_undegraded":"2024-09-10T16:00:09.301964+0000","last_fullsized":"2024-09-10T16:00:09.301964+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":6206,"log_dups_size":0,"ondisk_log_size":6206,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25396166,"num_objects":6202,"num_object_clones":0,"num_object_copies":12404,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6202,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4311,"num_write_kb":17244,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'6152","reported_seq":8562,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T16:00:09.387043+0000","last_change":"2024-09-10T15:59:16.142612+0000","last_active":"2024-09-10T16:00:09.387043+0000","last_peered":"2024-09-10T16:00:09.387043+0000","last_clean":"2024-09-10T15:59:14.092473+0000","last_became_active":"2024-09-10T15:59:16.109611+0000","last_became_peered":"2024-09-10T15:59:16.109611+0000","last_unstale":"2024-09-10T16:00:09.387043+0000","last_undegraded":"2024-09-10T16:00:09.387043+0000","last_fullsized":"2024-09-10T15:59:15.770264+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":6152,"log_dups_size":0,"ondisk_log_size":6152,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25182208,"num_objects":6148,"num_object_clones":0,"num_object_copies":12296,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3125,"num_objects_unfound":0,"num_objects_dirty":6148,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4156,"num_write_kb":16624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1143,"num_bytes_recovered":4681728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3125},{"shards":"2","objects":3023}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'6203","reported_seq":6278,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:00:09.345877+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:00:09.345877+0000","last_peered":"2024-09-10T16:00:09.345877+0000","last_clean":"2024-09-10T16:00:09.345877+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:00:09.345877+0000","last_undegraded":"2024-09-10T16:00:09.345877+0000","last_fullsized":"2024-09-10T16:00:09.345877+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":6203,"log_dups_size":0,"ondisk_log_size":6203,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25407488,"num_objects":6203,"num_object_clones":0,"num_object_copies":12406,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6203,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6203,"num_write_kb":24812,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'6140","reported_seq":6264,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:00:06.307940+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:00:06.307940+0000","last_peered":"2024-09-10T16:00:06.307940+0000","last_clean":"2024-09-10T16:00:06.307940+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:00:06.307940+0000","last_undegraded":"2024-09-10T16:00:06.307940+0000","last_fullsized":"2024-09-10T16:00:06.307940+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":6140,"log_dups_size":0,"ondisk_log_size":6140,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25121251,"num_objects":6134,"num_object_clones":0,"num_object_copies":12268,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6134,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2851,"num_write_kb":11404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'6227","reported_seq":6359,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:06.277641+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:00:06.277641+0000","last_peered":"2024-09-10T16:00:06.277641+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:00:06.277641+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":6227,"log_dups_size":0,"ondisk_log_size":6227,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25485795,"num_objects":6223,"num_object_clones":0,"num_object_copies":12446,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6223,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2839,"num_write_kb":11356,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1852},{"shards":"0,2","objects":4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'6052","reported_seq":6182,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:06.283207+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:00:06.283207+0000","last_peered":"2024-09-10T16:00:06.283207+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:00:06.283207+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":6052,"log_dups_size":0,"ondisk_log_size":6052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24765382,"num_objects":6048,"num_object_clones":0,"num_object_copies":12096,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6048,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":2551,"num_write_kb":10204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1811},{"shards":"0,2","objects":4237}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'6197","reported_seq":6321,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:00:06.290154+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:00:06.290154+0000","last_peered":"2024-09-10T16:00:06.290154+0000","last_clean":"2024-09-10T16:00:06.290154+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:00:06.290154+0000","last_undegraded":"2024-09-10T16:00:06.290154+0000","last_fullsized":"2024-09-10T16:00:06.290154+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":6197,"log_dups_size":0,"ondisk_log_size":6197,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25374720,"num_objects":6195,"num_object_clones":0,"num_object_copies":12390,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6195,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3044,"num_write_kb":12176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.775865+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T15:59:15.775865+0000","last_peered":"2024-09-10T15:59:15.775865+0000","last_clean":"2024-09-10T15:59:15.775865+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T15:59:15.775865+0000","last_undegraded":"2024-09-10T15:59:15.775865+0000","last_fullsized":"2024-09-10T15:59:15.775865+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'5985","reported_seq":6119,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:09.275109+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:00:09.275109+0000","last_peered":"2024-09-10T16:00:09.275109+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:00:09.275109+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":5985,"log_dups_size":0,"ondisk_log_size":5985,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24486371,"num_objects":5979,"num_object_clones":0,"num_object_copies":11958,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5979,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3804,"num_write_kb":15216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4123},{"shards":"2","objects":1856}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":88,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.772572+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T15:59:15.772572+0000","last_peered":"2024-09-10T15:59:15.772572+0000","last_clean":"2024-09-10T15:59:15.772572+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T15:59:15.772572+0000","last_undegraded":"2024-09-10T15:59:15.772572+0000","last_fullsized":"2024-09-10T15:59:15.772572+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'6035","reported_seq":6160,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:06.332640+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:00:06.332640+0000","last_peered":"2024-09-10T16:00:06.332640+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:00:06.332640+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":6035,"log_dups_size":0,"ondisk_log_size":6035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24686592,"num_objects":6027,"num_object_clones":0,"num_object_copies":12054,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6027,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2539,"num_write_kb":10156,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1772},{"shards":"0,2","objects":4255}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":99,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.884297+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T15:59:15.884297+0000","last_peered":"2024-09-10T15:59:15.884297+0000","last_clean":"2024-09-10T15:59:15.884297+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T15:59:15.884297+0000","last_undegraded":"2024-09-10T15:59:15.884297+0000","last_fullsized":"2024-09-10T15:59:15.884297+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'6197","reported_seq":6326,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:09.302173+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:00:09.302173+0000","last_peered":"2024-09-10T16:00:09.302173+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:00:09.302173+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":6197,"log_dups_size":0,"ondisk_log_size":6197,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25362915,"num_objects":6193,"num_object_clones":0,"num_object_copies":12386,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6193,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3877,"num_write_kb":15508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4304},{"shards":"2","objects":1889}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.837509+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T15:59:15.837509+0000","last_peered":"2024-09-10T15:59:15.837509+0000","last_clean":"2024-09-10T15:59:15.837509+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T15:59:15.837509+0000","last_undegraded":"2024-09-10T15:59:15.837509+0000","last_fullsized":"2024-09-10T15:59:15.837509+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'6324","reported_seq":6447,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:09.281734+0000","last_change":"2024-09-10T15:59:16.174113+0000","last_active":"2024-09-10T16:00:09.281734+0000","last_peered":"2024-09-10T16:00:09.281734+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:00:09.281734+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":6324,"log_dups_size":0,"ondisk_log_size":6324,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25878528,"num_objects":6318,"num_object_clones":0,"num_object_copies":12636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4354,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6318,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3878,"num_write_kb":15512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4352},{"shards":"2","objects":1966}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'6121","reported_seq":6250,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:06.302724+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:00:06.302724+0000","last_peered":"2024-09-10T16:00:06.302724+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:00:06.302724+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":6121,"log_dups_size":0,"ondisk_log_size":6121,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25063424,"num_objects":6119,"num_object_clones":0,"num_object_copies":12238,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6119,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2752,"num_write_kb":11008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1815},{"shards":"0,2","objects":4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.989413+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T15:59:15.989413+0000","last_peered":"2024-09-10T15:59:15.989413+0000","last_clean":"2024-09-10T15:59:15.989413+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T15:59:15.989413+0000","last_undegraded":"2024-09-10T15:59:15.989413+0000","last_fullsized":"2024-09-10T15:59:15.989413+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'5983","reported_seq":6120,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:06.344526+0000","last_change":"2024-09-10T15:59:16.251610+0000","last_active":"2024-09-10T16:00:06.344526+0000","last_peered":"2024-09-10T16:00:06.344526+0000","last_clean":"2024-09-10T15:59:14.115595+0000","last_became_active":"2024-09-10T15:59:16.244089+0000","last_became_peered":"2024-09-10T15:59:16.244089+0000","last_unstale":"2024-09-10T16:00:06.344526+0000","last_undegraded":"2024-09-10T15:59:15.847127+0000","last_fullsized":"2024-09-10T15:59:15.774437+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":5983,"log_dups_size":0,"ondisk_log_size":5983,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24473600,"num_objects":5975,"num_object_clones":0,"num_object_copies":11950,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5975,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2373,"num_write_kb":9492,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1772},{"shards":"0,2","objects":4203}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":88,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.773583+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T15:59:15.773583+0000","last_peered":"2024-09-10T15:59:15.773583+0000","last_clean":"2024-09-10T15:59:15.773583+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T15:59:15.773583+0000","last_undegraded":"2024-09-10T15:59:15.773583+0000","last_fullsized":"2024-09-10T15:59:15.773583+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'6240","reported_seq":6376,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:06.204545+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:00:06.204545+0000","last_peered":"2024-09-10T16:00:06.204545+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:00:06.204545+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":6240,"log_dups_size":0,"ondisk_log_size":6240,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25542656,"num_objects":6236,"num_object_clones":0,"num_object_copies":12472,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6236,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2495,"num_write_kb":9980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1865},{"shards":"0,2","objects":4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.916017+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T15:59:15.916017+0000","last_peered":"2024-09-10T15:59:15.916017+0000","last_clean":"2024-09-10T15:59:15.916017+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T15:59:15.916017+0000","last_undegraded":"2024-09-10T15:59:15.916017+0000","last_fullsized":"2024-09-10T15:59:15.916017+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.776033+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T15:59:15.776033+0000","last_peered":"2024-09-10T15:59:15.776033+0000","last_clean":"2024-09-10T15:59:15.776033+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T15:59:15.776033+0000","last_undegraded":"2024-09-10T15:59:15.776033+0000","last_fullsized":"2024-09-10T15:59:15.776033+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'6229","reported_seq":6368,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:06.329528+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:00:06.329528+0000","last_peered":"2024-09-10T16:00:06.329528+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:00:06.329528+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":6229,"log_dups_size":0,"ondisk_log_size":6229,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25465798,"num_objects":6219,"num_object_clones":0,"num_object_copies":12438,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6219,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2392,"num_write_kb":9568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1896},{"shards":"0,2","objects":4323}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":402531038,"num_objects":98283,"num_object_clones":0,"num_object_copies":196566,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":47117,"num_objects_misplaced":3125,"num_objects_unfound":0,"num_objects_dirty":98283,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":56129,"num_write_kb":224516,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1146,"num_bytes_recovered":4694016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":812064768,"data_stored":811909409,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":98355,"ondisk_log_size":98355,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986202,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":507360,"kb_used_data":335440,"kb_used_omap":4,"kb_used_meta":171899,"kb_avail":93864480,"statfs":{"total":96636764160,"available":96117227520,"internally_reserved":0,"allocated":343490560,"data_stored":343212800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":176025258},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,5,3,1,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659148,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":203536,"kb_used_data":95488,"kb_used_omap":4,"kb_used_meta":108027,"kb_avail":94168304,"statfs":{"total":96636764160,"available":96428343296,"internally_reserved":0,"allocated":97779712,"data_stored":97450906,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":110620334},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,0,1,3,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182430,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":533628,"kb_used_data":361772,"kb_used_omap":4,"kb_used_meta":171835,"kb_avail":93838212,"statfs":{"total":96636764160,"available":96090329088,"internally_reserved":0,"allocated":370454528,"data_stored":370172691,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":175959722},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,6,2,3,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":370638848,"data_stored":370602718,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":97689600,"data_stored":97606501,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":343736320,"data_stored":343700190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:00:10.976 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 98 20 100148 100128 3.99042 2.00391 0.0288287 0.0195704 2024-09-10T16:00:11.977 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 99 20 100727 100707 3.97296 2.26172 0.0680991 0.0196576 2024-09-10T16:00:12.977 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout:2024-09-10T16:00:12.976417+0000 min lat: 0.00225581 max lat: 2.21736 avg lat: 0.0197422 2024-09-10T16:00:12.977 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-10T16:00:12.977 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 100 20 101301 101281 3.95565 2.24219 0.0169316 0.0197422 2024-09-10T16:00:13.764 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:00:13.983 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 101 20 101847 101827 3.93751 2.13281 0.0224445 0.0198332 2024-09-10T16:00:14.335 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:00:14.335 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:00:14.363 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":125,"stamp":"2024-09-10T16:00:13.560745+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":409535217,"num_objects":99995,"num_object_clones":0,"num_object_copies":199990,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":47117,"num_objects_misplaced":3125,"num_objects_unfound":0,"num_objects_dirty":99995,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":57841,"num_write_kb":231358,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1148,"num_bytes_recovered":4694035,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":100068,"ondisk_log_size":100068,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1256964,"kb_used_data":800980,"kb_used_omap":12,"kb_used_meta":455923,"kb_avail":281858556,"statfs":{"total":289910292480,"available":288623161344,"internally_reserved":0,"allocated":820203520,"data_stored":819315117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":466865154},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,11,6,7,5,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":22466560,"num_objects":5485,"num_object_clones":0,"num_object_copies":10970,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-221,"num_objects_unfound":0,"num_objects_dirty":5485,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5485,"num_write_kb":21940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":221,"num_bytes_recovered":905216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003342"},"pg_stats":[{"pgid":"2.f","version":"43'6237","reported_seq":6306,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:11.341997+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:00:11.341997+0000","last_peered":"2024-09-10T16:00:11.341997+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:00:11.341997+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":6237,"log_dups_size":0,"ondisk_log_size":6237,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25546752,"num_objects":6237,"num_object_clones":0,"num_object_copies":12474,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6237,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6237,"num_write_kb":24948,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1978},{"shards":"0,2","objects":4258}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'6206","reported_seq":6313,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:00:09.301964+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:00:09.301964+0000","last_peered":"2024-09-10T16:00:09.301964+0000","last_clean":"2024-09-10T16:00:09.301964+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:00:09.301964+0000","last_undegraded":"2024-09-10T16:00:09.301964+0000","last_fullsized":"2024-09-10T16:00:09.301964+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":6206,"log_dups_size":0,"ondisk_log_size":6206,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25396166,"num_objects":6202,"num_object_clones":0,"num_object_copies":12404,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6202,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4311,"num_write_kb":17244,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'6152","reported_seq":8562,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T16:00:09.387043+0000","last_change":"2024-09-10T15:59:16.142612+0000","last_active":"2024-09-10T16:00:09.387043+0000","last_peered":"2024-09-10T16:00:09.387043+0000","last_clean":"2024-09-10T15:59:14.092473+0000","last_became_active":"2024-09-10T15:59:16.109611+0000","last_became_peered":"2024-09-10T15:59:16.109611+0000","last_unstale":"2024-09-10T16:00:09.387043+0000","last_undegraded":"2024-09-10T16:00:09.387043+0000","last_fullsized":"2024-09-10T15:59:15.770264+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":6152,"log_dups_size":0,"ondisk_log_size":6152,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25182208,"num_objects":6148,"num_object_clones":0,"num_object_copies":12296,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3125,"num_objects_unfound":0,"num_objects_dirty":6148,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4156,"num_write_kb":16624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1143,"num_bytes_recovered":4681728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3125},{"shards":"2","objects":3023}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'6203","reported_seq":6278,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:00:09.345877+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:00:09.345877+0000","last_peered":"2024-09-10T16:00:09.345877+0000","last_clean":"2024-09-10T16:00:09.345877+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:00:09.345877+0000","last_undegraded":"2024-09-10T16:00:09.345877+0000","last_fullsized":"2024-09-10T16:00:09.345877+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":6203,"log_dups_size":0,"ondisk_log_size":6203,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25407488,"num_objects":6203,"num_object_clones":0,"num_object_copies":12406,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6203,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6203,"num_write_kb":24812,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'6313","reported_seq":6437,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:00:11.322198+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:00:11.322198+0000","last_peered":"2024-09-10T16:00:11.322198+0000","last_clean":"2024-09-10T16:00:11.322198+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:00:11.322198+0000","last_undegraded":"2024-09-10T16:00:11.322198+0000","last_fullsized":"2024-09-10T16:00:11.322198+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":6313,"log_dups_size":0,"ondisk_log_size":6313,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25829859,"num_objects":6307,"num_object_clones":0,"num_object_copies":12614,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6307,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3024,"num_write_kb":12096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'6397","reported_seq":6528,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:11.352016+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:00:11.352016+0000","last_peered":"2024-09-10T16:00:11.352016+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:00:11.352016+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":6397,"log_dups_size":0,"ondisk_log_size":6397,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26182115,"num_objects":6393,"num_object_clones":0,"num_object_copies":12786,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6393,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3009,"num_write_kb":12036,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2022},{"shards":"0,2","objects":4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'6239","reported_seq":6371,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:11.339874+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:00:11.339874+0000","last_peered":"2024-09-10T16:00:11.339874+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:00:11.339874+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":6239,"log_dups_size":0,"ondisk_log_size":6239,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25531334,"num_objects":6235,"num_object_clones":0,"num_object_copies":12470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6235,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":2738,"num_write_kb":10952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1998},{"shards":"0,2","objects":4237}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'6383","reported_seq":6508,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:00:11.363960+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:00:11.363960+0000","last_peered":"2024-09-10T16:00:11.363960+0000","last_clean":"2024-09-10T16:00:11.363960+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:00:11.363960+0000","last_undegraded":"2024-09-10T16:00:11.363960+0000","last_fullsized":"2024-09-10T16:00:11.363960+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":6383,"log_dups_size":0,"ondisk_log_size":6383,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26136576,"num_objects":6381,"num_object_clones":0,"num_object_copies":12762,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6381,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3230,"num_write_kb":12920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.775865+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T15:59:15.775865+0000","last_peered":"2024-09-10T15:59:15.775865+0000","last_clean":"2024-09-10T15:59:15.775865+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T15:59:15.775865+0000","last_undegraded":"2024-09-10T15:59:15.775865+0000","last_fullsized":"2024-09-10T15:59:15.775865+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'5985","reported_seq":6119,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:09.275109+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:00:09.275109+0000","last_peered":"2024-09-10T16:00:09.275109+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:00:09.275109+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":5985,"log_dups_size":0,"ondisk_log_size":5985,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24486371,"num_objects":5979,"num_object_clones":0,"num_object_copies":11958,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5979,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3804,"num_write_kb":15216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4123},{"shards":"2","objects":1856}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":88,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.772572+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T15:59:15.772572+0000","last_peered":"2024-09-10T15:59:15.772572+0000","last_clean":"2024-09-10T15:59:15.772572+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T15:59:15.772572+0000","last_undegraded":"2024-09-10T15:59:15.772572+0000","last_fullsized":"2024-09-10T15:59:15.772572+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'6200","reported_seq":6323,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:11.331419+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:00:11.331419+0000","last_peered":"2024-09-10T16:00:11.331419+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:00:11.331419+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":6200,"log_dups_size":0,"ondisk_log_size":6200,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25362432,"num_objects":6192,"num_object_clones":0,"num_object_copies":12384,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6192,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2704,"num_write_kb":10816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1937},{"shards":"0,2","objects":4255}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":99,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.884297+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T15:59:15.884297+0000","last_peered":"2024-09-10T15:59:15.884297+0000","last_clean":"2024-09-10T15:59:15.884297+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T15:59:15.884297+0000","last_undegraded":"2024-09-10T15:59:15.884297+0000","last_fullsized":"2024-09-10T15:59:15.884297+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'6197","reported_seq":6326,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:09.302173+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:00:09.302173+0000","last_peered":"2024-09-10T16:00:09.302173+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:00:09.302173+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":6197,"log_dups_size":0,"ondisk_log_size":6197,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25362915,"num_objects":6193,"num_object_clones":0,"num_object_copies":12386,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6193,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3877,"num_write_kb":15508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4304},{"shards":"2","objects":1889}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.837509+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T15:59:15.837509+0000","last_peered":"2024-09-10T15:59:15.837509+0000","last_clean":"2024-09-10T15:59:15.837509+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T15:59:15.837509+0000","last_undegraded":"2024-09-10T15:59:15.837509+0000","last_fullsized":"2024-09-10T15:59:15.837509+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'6324","reported_seq":6447,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:09.281734+0000","last_change":"2024-09-10T15:59:16.174113+0000","last_active":"2024-09-10T16:00:09.281734+0000","last_peered":"2024-09-10T16:00:09.281734+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:00:09.281734+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":6324,"log_dups_size":0,"ondisk_log_size":6324,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25878528,"num_objects":6318,"num_object_clones":0,"num_object_copies":12636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4354,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6318,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3878,"num_write_kb":15512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4352},{"shards":"2","objects":1966}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'6268","reported_seq":6397,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:11.347236+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:00:11.347236+0000","last_peered":"2024-09-10T16:00:11.347236+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:00:11.347236+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":6268,"log_dups_size":0,"ondisk_log_size":6268,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25665536,"num_objects":6266,"num_object_clones":0,"num_object_copies":12532,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6266,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2899,"num_write_kb":11596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1962},{"shards":"0,2","objects":4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.989413+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T15:59:15.989413+0000","last_peered":"2024-09-10T15:59:15.989413+0000","last_clean":"2024-09-10T15:59:15.989413+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T15:59:15.989413+0000","last_undegraded":"2024-09-10T15:59:15.989413+0000","last_fullsized":"2024-09-10T15:59:15.989413+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'6155","reported_seq":6292,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:11.281394+0000","last_change":"2024-09-10T15:59:16.251610+0000","last_active":"2024-09-10T16:00:11.281394+0000","last_peered":"2024-09-10T16:00:11.281394+0000","last_clean":"2024-09-10T15:59:14.115595+0000","last_became_active":"2024-09-10T15:59:16.244089+0000","last_became_peered":"2024-09-10T15:59:16.244089+0000","last_unstale":"2024-09-10T16:00:11.281394+0000","last_undegraded":"2024-09-10T15:59:15.847127+0000","last_fullsized":"2024-09-10T15:59:15.774437+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":6155,"log_dups_size":0,"ondisk_log_size":6155,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25178112,"num_objects":6147,"num_object_clones":0,"num_object_copies":12294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6147,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2545,"num_write_kb":10180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1944},{"shards":"0,2","objects":4203}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":88,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.773583+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T15:59:15.773583+0000","last_peered":"2024-09-10T15:59:15.773583+0000","last_clean":"2024-09-10T15:59:15.773583+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T15:59:15.773583+0000","last_undegraded":"2024-09-10T15:59:15.773583+0000","last_fullsized":"2024-09-10T15:59:15.773583+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'6406","reported_seq":6542,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:11.299093+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:00:11.299093+0000","last_peered":"2024-09-10T16:00:11.299093+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:00:11.299093+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":6406,"log_dups_size":0,"ondisk_log_size":6406,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26222592,"num_objects":6402,"num_object_clones":0,"num_object_copies":12804,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6402,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2661,"num_write_kb":10644,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2031},{"shards":"0,2","objects":4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.916017+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T15:59:15.916017+0000","last_peered":"2024-09-10T15:59:15.916017+0000","last_clean":"2024-09-10T15:59:15.916017+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T15:59:15.916017+0000","last_undegraded":"2024-09-10T15:59:15.916017+0000","last_fullsized":"2024-09-10T15:59:15.916017+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.776033+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T15:59:15.776033+0000","last_peered":"2024-09-10T15:59:15.776033+0000","last_clean":"2024-09-10T15:59:15.776033+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T15:59:15.776033+0000","last_undegraded":"2024-09-10T15:59:15.776033+0000","last_fullsized":"2024-09-10T15:59:15.776033+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'6400","reported_seq":6539,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:11.244673+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:00:11.244673+0000","last_peered":"2024-09-10T16:00:11.244673+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:00:11.244673+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":6400,"log_dups_size":0,"ondisk_log_size":6400,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26166214,"num_objects":6390,"num_object_clones":0,"num_object_copies":12780,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6390,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2563,"num_write_kb":10252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2067},{"shards":"0,2","objects":4323}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":409535198,"num_objects":99993,"num_object_clones":0,"num_object_copies":199986,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":47117,"num_objects_misplaced":3125,"num_objects_unfound":0,"num_objects_dirty":99993,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":57839,"num_write_kb":231356,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1146,"num_bytes_recovered":4694016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":820461568,"data_stored":820306209,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":100065,"ondisk_log_size":100065,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986202,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":507360,"kb_used_data":335440,"kb_used_omap":4,"kb_used_meta":171899,"kb_avail":93864480,"statfs":{"total":96636764160,"available":96117227520,"internally_reserved":0,"allocated":343490560,"data_stored":343212800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":176025258},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,5,3,1,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659148,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":203536,"kb_used_data":95488,"kb_used_omap":4,"kb_used_meta":108027,"kb_avail":94168304,"statfs":{"total":96636764160,"available":96428343296,"internally_reserved":0,"allocated":97779712,"data_stored":97450906,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":110620334},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,0,1,3,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182431,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":546068,"kb_used_data":370052,"kb_used_omap":4,"kb_used_meta":175995,"kb_avail":93825772,"statfs":{"total":96636764160,"available":96077590528,"internally_reserved":0,"allocated":378933248,"data_stored":378651411,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":180219562},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,6,2,3,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":379035648,"data_stored":378999518,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":97689600,"data_stored":97606501,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":343736320,"data_stored":343700190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:00:14.364 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:00:14.927 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:00:14.927 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:00:14.949 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":125,"stamp":"2024-09-10T16:00:13.560745+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":409535217,"num_objects":99995,"num_object_clones":0,"num_object_copies":199990,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":47117,"num_objects_misplaced":3125,"num_objects_unfound":0,"num_objects_dirty":99995,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":57841,"num_write_kb":231358,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1148,"num_bytes_recovered":4694035,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":100068,"ondisk_log_size":100068,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1256964,"kb_used_data":800980,"kb_used_omap":12,"kb_used_meta":455923,"kb_avail":281858556,"statfs":{"total":289910292480,"available":288623161344,"internally_reserved":0,"allocated":820203520,"data_stored":819315117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":466865154},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,11,6,7,5,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":22466560,"num_objects":5485,"num_object_clones":0,"num_object_copies":10970,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-221,"num_objects_unfound":0,"num_objects_dirty":5485,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5485,"num_write_kb":21940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":221,"num_bytes_recovered":905216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003342"},"pg_stats":[{"pgid":"2.f","version":"43'6237","reported_seq":6306,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:11.341997+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:00:11.341997+0000","last_peered":"2024-09-10T16:00:11.341997+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:00:11.341997+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":6237,"log_dups_size":0,"ondisk_log_size":6237,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25546752,"num_objects":6237,"num_object_clones":0,"num_object_copies":12474,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6237,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6237,"num_write_kb":24948,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1978},{"shards":"0,2","objects":4258}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'6206","reported_seq":6313,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:00:09.301964+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:00:09.301964+0000","last_peered":"2024-09-10T16:00:09.301964+0000","last_clean":"2024-09-10T16:00:09.301964+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:00:09.301964+0000","last_undegraded":"2024-09-10T16:00:09.301964+0000","last_fullsized":"2024-09-10T16:00:09.301964+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":6206,"log_dups_size":0,"ondisk_log_size":6206,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25396166,"num_objects":6202,"num_object_clones":0,"num_object_copies":12404,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6202,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4311,"num_write_kb":17244,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'6152","reported_seq":8562,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T16:00:09.387043+0000","last_change":"2024-09-10T15:59:16.142612+0000","last_active":"2024-09-10T16:00:09.387043+0000","last_peered":"2024-09-10T16:00:09.387043+0000","last_clean":"2024-09-10T15:59:14.092473+0000","last_became_active":"2024-09-10T15:59:16.109611+0000","last_became_peered":"2024-09-10T15:59:16.109611+0000","last_unstale":"2024-09-10T16:00:09.387043+0000","last_undegraded":"2024-09-10T16:00:09.387043+0000","last_fullsized":"2024-09-10T15:59:15.770264+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":6152,"log_dups_size":0,"ondisk_log_size":6152,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25182208,"num_objects":6148,"num_object_clones":0,"num_object_copies":12296,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3125,"num_objects_unfound":0,"num_objects_dirty":6148,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4156,"num_write_kb":16624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1143,"num_bytes_recovered":4681728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3125},{"shards":"2","objects":3023}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'6203","reported_seq":6278,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:00:09.345877+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:00:09.345877+0000","last_peered":"2024-09-10T16:00:09.345877+0000","last_clean":"2024-09-10T16:00:09.345877+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:00:09.345877+0000","last_undegraded":"2024-09-10T16:00:09.345877+0000","last_fullsized":"2024-09-10T16:00:09.345877+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":6203,"log_dups_size":0,"ondisk_log_size":6203,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25407488,"num_objects":6203,"num_object_clones":0,"num_object_copies":12406,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6203,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6203,"num_write_kb":24812,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'6313","reported_seq":6437,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:00:11.322198+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:00:11.322198+0000","last_peered":"2024-09-10T16:00:11.322198+0000","last_clean":"2024-09-10T16:00:11.322198+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:00:11.322198+0000","last_undegraded":"2024-09-10T16:00:11.322198+0000","last_fullsized":"2024-09-10T16:00:11.322198+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":6313,"log_dups_size":0,"ondisk_log_size":6313,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25829859,"num_objects":6307,"num_object_clones":0,"num_object_copies":12614,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6307,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3024,"num_write_kb":12096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'6397","reported_seq":6528,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:11.352016+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:00:11.352016+0000","last_peered":"2024-09-10T16:00:11.352016+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:00:11.352016+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":6397,"log_dups_size":0,"ondisk_log_size":6397,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26182115,"num_objects":6393,"num_object_clones":0,"num_object_copies":12786,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6393,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3009,"num_write_kb":12036,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2022},{"shards":"0,2","objects":4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'6239","reported_seq":6371,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:11.339874+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:00:11.339874+0000","last_peered":"2024-09-10T16:00:11.339874+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:00:11.339874+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":6239,"log_dups_size":0,"ondisk_log_size":6239,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25531334,"num_objects":6235,"num_object_clones":0,"num_object_copies":12470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6235,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":2738,"num_write_kb":10952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1998},{"shards":"0,2","objects":4237}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'6383","reported_seq":6508,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:00:11.363960+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:00:11.363960+0000","last_peered":"2024-09-10T16:00:11.363960+0000","last_clean":"2024-09-10T16:00:11.363960+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:00:11.363960+0000","last_undegraded":"2024-09-10T16:00:11.363960+0000","last_fullsized":"2024-09-10T16:00:11.363960+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":6383,"log_dups_size":0,"ondisk_log_size":6383,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26136576,"num_objects":6381,"num_object_clones":0,"num_object_copies":12762,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6381,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3230,"num_write_kb":12920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.775865+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T15:59:15.775865+0000","last_peered":"2024-09-10T15:59:15.775865+0000","last_clean":"2024-09-10T15:59:15.775865+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T15:59:15.775865+0000","last_undegraded":"2024-09-10T15:59:15.775865+0000","last_fullsized":"2024-09-10T15:59:15.775865+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'5985","reported_seq":6119,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:09.275109+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:00:09.275109+0000","last_peered":"2024-09-10T16:00:09.275109+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:00:09.275109+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":5985,"log_dups_size":0,"ondisk_log_size":5985,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24486371,"num_objects":5979,"num_object_clones":0,"num_object_copies":11958,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5979,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3804,"num_write_kb":15216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4123},{"shards":"2","objects":1856}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":88,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.772572+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T15:59:15.772572+0000","last_peered":"2024-09-10T15:59:15.772572+0000","last_clean":"2024-09-10T15:59:15.772572+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T15:59:15.772572+0000","last_undegraded":"2024-09-10T15:59:15.772572+0000","last_fullsized":"2024-09-10T15:59:15.772572+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'6200","reported_seq":6323,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:11.331419+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:00:11.331419+0000","last_peered":"2024-09-10T16:00:11.331419+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:00:11.331419+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":6200,"log_dups_size":0,"ondisk_log_size":6200,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25362432,"num_objects":6192,"num_object_clones":0,"num_object_copies":12384,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6192,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2704,"num_write_kb":10816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1937},{"shards":"0,2","objects":4255}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":99,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.884297+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T15:59:15.884297+0000","last_peered":"2024-09-10T15:59:15.884297+0000","last_clean":"2024-09-10T15:59:15.884297+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T15:59:15.884297+0000","last_undegraded":"2024-09-10T15:59:15.884297+0000","last_fullsized":"2024-09-10T15:59:15.884297+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'6197","reported_seq":6326,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:09.302173+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:00:09.302173+0000","last_peered":"2024-09-10T16:00:09.302173+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:00:09.302173+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":6197,"log_dups_size":0,"ondisk_log_size":6197,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25362915,"num_objects":6193,"num_object_clones":0,"num_object_copies":12386,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6193,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3877,"num_write_kb":15508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4304},{"shards":"2","objects":1889}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.837509+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T15:59:15.837509+0000","last_peered":"2024-09-10T15:59:15.837509+0000","last_clean":"2024-09-10T15:59:15.837509+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T15:59:15.837509+0000","last_undegraded":"2024-09-10T15:59:15.837509+0000","last_fullsized":"2024-09-10T15:59:15.837509+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'6324","reported_seq":6447,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:09.281734+0000","last_change":"2024-09-10T15:59:16.174113+0000","last_active":"2024-09-10T16:00:09.281734+0000","last_peered":"2024-09-10T16:00:09.281734+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:00:09.281734+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":6324,"log_dups_size":0,"ondisk_log_size":6324,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25878528,"num_objects":6318,"num_object_clones":0,"num_object_copies":12636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4354,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6318,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3878,"num_write_kb":15512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4352},{"shards":"2","objects":1966}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'6268","reported_seq":6397,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:11.347236+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:00:11.347236+0000","last_peered":"2024-09-10T16:00:11.347236+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:00:11.347236+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":6268,"log_dups_size":0,"ondisk_log_size":6268,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25665536,"num_objects":6266,"num_object_clones":0,"num_object_copies":12532,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6266,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2899,"num_write_kb":11596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1962},{"shards":"0,2","objects":4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.989413+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T15:59:15.989413+0000","last_peered":"2024-09-10T15:59:15.989413+0000","last_clean":"2024-09-10T15:59:15.989413+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T15:59:15.989413+0000","last_undegraded":"2024-09-10T15:59:15.989413+0000","last_fullsized":"2024-09-10T15:59:15.989413+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'6155","reported_seq":6292,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:11.281394+0000","last_change":"2024-09-10T15:59:16.251610+0000","last_active":"2024-09-10T16:00:11.281394+0000","last_peered":"2024-09-10T16:00:11.281394+0000","last_clean":"2024-09-10T15:59:14.115595+0000","last_became_active":"2024-09-10T15:59:16.244089+0000","last_became_peered":"2024-09-10T15:59:16.244089+0000","last_unstale":"2024-09-10T16:00:11.281394+0000","last_undegraded":"2024-09-10T15:59:15.847127+0000","last_fullsized":"2024-09-10T15:59:15.774437+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":6155,"log_dups_size":0,"ondisk_log_size":6155,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25178112,"num_objects":6147,"num_object_clones":0,"num_object_copies":12294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6147,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2545,"num_write_kb":10180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1944},{"shards":"0,2","objects":4203}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":88,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.773583+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T15:59:15.773583+0000","last_peered":"2024-09-10T15:59:15.773583+0000","last_clean":"2024-09-10T15:59:15.773583+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T15:59:15.773583+0000","last_undegraded":"2024-09-10T15:59:15.773583+0000","last_fullsized":"2024-09-10T15:59:15.773583+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'6406","reported_seq":6542,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:11.299093+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:00:11.299093+0000","last_peered":"2024-09-10T16:00:11.299093+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:00:11.299093+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":6406,"log_dups_size":0,"ondisk_log_size":6406,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26222592,"num_objects":6402,"num_object_clones":0,"num_object_copies":12804,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6402,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2661,"num_write_kb":10644,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2031},{"shards":"0,2","objects":4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.916017+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T15:59:15.916017+0000","last_peered":"2024-09-10T15:59:15.916017+0000","last_clean":"2024-09-10T15:59:15.916017+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T15:59:15.916017+0000","last_undegraded":"2024-09-10T15:59:15.916017+0000","last_fullsized":"2024-09-10T15:59:15.916017+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.776033+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T15:59:15.776033+0000","last_peered":"2024-09-10T15:59:15.776033+0000","last_clean":"2024-09-10T15:59:15.776033+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T15:59:15.776033+0000","last_undegraded":"2024-09-10T15:59:15.776033+0000","last_fullsized":"2024-09-10T15:59:15.776033+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'6400","reported_seq":6539,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:11.244673+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:00:11.244673+0000","last_peered":"2024-09-10T16:00:11.244673+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:00:11.244673+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":6400,"log_dups_size":0,"ondisk_log_size":6400,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26166214,"num_objects":6390,"num_object_clones":0,"num_object_copies":12780,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6390,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2563,"num_write_kb":10252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2067},{"shards":"0,2","objects":4323}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":409535198,"num_objects":99993,"num_object_clones":0,"num_object_copies":199986,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":47117,"num_objects_misplaced":3125,"num_objects_unfound":0,"num_objects_dirty":99993,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":57839,"num_write_kb":231356,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1146,"num_bytes_recovered":4694016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":820461568,"data_stored":820306209,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":100065,"ondisk_log_size":100065,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986202,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":507360,"kb_used_data":335440,"kb_used_omap":4,"kb_used_meta":171899,"kb_avail":93864480,"statfs":{"total":96636764160,"available":96117227520,"internally_reserved":0,"allocated":343490560,"data_stored":343212800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":176025258},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,5,3,1,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659148,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":203536,"kb_used_data":95488,"kb_used_omap":4,"kb_used_meta":108027,"kb_avail":94168304,"statfs":{"total":96636764160,"available":96428343296,"internally_reserved":0,"allocated":97779712,"data_stored":97450906,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":110620334},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,0,1,3,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182431,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":546068,"kb_used_data":370052,"kb_used_omap":4,"kb_used_meta":175995,"kb_avail":93825772,"statfs":{"total":96636764160,"available":96077590528,"internally_reserved":0,"allocated":378933248,"data_stored":378651411,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":180219562},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,6,2,3,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":379035648,"data_stored":378999518,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":97689600,"data_stored":97606501,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":343736320,"data_stored":343700190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:00:14.950 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-10T16:00:14.950 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "43'6237", 'reported_seq': 6306, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:00:11.341997+0000', 'last_change': '2024-09-10T15:59:16.281558+0000', 'last_active': '2024-09-10T16:00:11.341997+0000', 'last_peered': '2024-09-10T16:00:11.341997+0000', 'last_clean': '2024-09-10T15:59:14.132456+0000', 'last_became_active': '2024-09-10T15:59:16.274407+0000', 'last_became_peered': '2024-09-10T15:59:16.274407+0000', 'last_unstale': '2024-09-10T16:00:11.341997+0000', 'last_undegraded': '2024-09-10T15:59:15.851279+0000', 'last_fullsized': '2024-09-10T15:59:15.774356+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'objects_scrubbed': 0, 'log_size': 6237, 'log_dups_size': 0, 'ondisk_log_size': 6237, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25546752, 'num_objects': 6237, 'num_object_clones': 0, 'num_object_copies': 12474, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4259, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6237, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6237, 'num_write_kb': 24948, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1978}, {'shards': '0,2', 'objects': 4258}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:00:14.951 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-10T16:00:14.951 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "43'6152", 'reported_seq': 8562, 'reported_epoch': 43, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-10T16:00:09.387043+0000', 'last_change': '2024-09-10T15:59:16.142612+0000', 'last_active': '2024-09-10T16:00:09.387043+0000', 'last_peered': '2024-09-10T16:00:09.387043+0000', 'last_clean': '2024-09-10T15:59:14.092473+0000', 'last_became_active': '2024-09-10T15:59:16.109611+0000', 'last_became_peered': '2024-09-10T15:59:16.109611+0000', 'last_unstale': '2024-09-10T16:00:09.387043+0000', 'last_undegraded': '2024-09-10T16:00:09.387043+0000', 'last_fullsized': '2024-09-10T15:59:15.770264+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2029", 'last_scrub_stamp': '2024-09-10T15:58:45.933789+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:45.933789+0000', 'objects_scrubbed': 1992, 'log_size': 6152, 'log_dups_size': 0, 'ondisk_log_size': 6152, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 668, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25182208, 'num_objects': 6148, 'num_object_clones': 0, 'num_object_copies': 12296, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 3125, 'num_objects_unfound': 0, 'num_objects_dirty': 6148, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4156, 'num_write_kb': 16624, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1143, 'num_bytes_recovered': 4681728, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3125}, {'shards': '2', 'objects': 3023}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:00:14.951 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-10T16:00:14.951 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "43'6397", 'reported_seq': 6528, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:00:11.352016+0000', 'last_change': '2024-09-10T15:59:16.308264+0000', 'last_active': '2024-09-10T16:00:11.352016+0000', 'last_peered': '2024-09-10T16:00:11.352016+0000', 'last_clean': '2024-09-10T15:59:14.111551+0000', 'last_became_active': '2024-09-10T15:59:16.300040+0000', 'last_became_peered': '2024-09-10T15:59:16.300040+0000', 'last_unstale': '2024-09-10T16:00:11.352016+0000', 'last_undegraded': '2024-09-10T15:59:15.878234+0000', 'last_fullsized': '2024-09-10T15:59:15.769328+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3442", 'last_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'objects_scrubbed': 3384, 'log_size': 6397, 'log_dups_size': 0, 'ondisk_log_size': 6397, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1292, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26182115, 'num_objects': 6393, 'num_object_clones': 0, 'num_object_copies': 12786, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4372, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6393, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 3009, 'num_write_kb': 12036, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2022}, {'shards': '0,2', 'objects': 4371}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:00:14.951 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-10T16:00:14.951 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "43'6239", 'reported_seq': 6371, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:00:11.339874+0000', 'last_change': '2024-09-10T15:59:16.331899+0000', 'last_active': '2024-09-10T16:00:11.339874+0000', 'last_peered': '2024-09-10T16:00:11.339874+0000', 'last_clean': '2024-09-10T15:59:14.139436+0000', 'last_became_active': '2024-09-10T15:59:16.318715+0000', 'last_became_peered': '2024-09-10T15:59:16.318715+0000', 'last_unstale': '2024-09-10T16:00:11.339874+0000', 'last_undegraded': '2024-09-10T15:59:15.877524+0000', 'last_fullsized': '2024-09-10T15:59:15.769042+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3537", 'last_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'objects_scrubbed': 3497, 'log_size': 6239, 'log_dups_size': 0, 'ondisk_log_size': 6239, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1224, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25531334, 'num_objects': 6235, 'num_object_clones': 0, 'num_object_copies': 12470, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4237, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6235, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 2738, 'num_write_kb': 10952, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1998}, {'shards': '0,2', 'objects': 4237}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:00:14.951 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-10T16:00:14.951 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "43'5985", 'reported_seq': 6119, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:00:09.275109+0000', 'last_change': '2024-09-10T15:59:16.175680+0000', 'last_active': '2024-09-10T16:00:09.275109+0000', 'last_peered': '2024-09-10T16:00:09.275109+0000', 'last_clean': '2024-09-10T15:59:14.088187+0000', 'last_became_active': '2024-09-10T15:59:16.165810+0000', 'last_became_peered': '2024-09-10T15:59:16.165810+0000', 'last_unstale': '2024-09-10T16:00:09.275109+0000', 'last_undegraded': '2024-09-10T15:59:15.822098+0000', 'last_fullsized': '2024-09-10T15:59:15.769123+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2218", 'last_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'objects_scrubbed': 2175, 'log_size': 5985, 'log_dups_size': 0, 'ondisk_log_size': 5985, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 825, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24486371, 'num_objects': 5979, 'num_object_clones': 0, 'num_object_copies': 11958, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4124, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5979, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 3804, 'num_write_kb': 15216, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4123}, {'shards': '2', 'objects': 1856}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:00:14.951 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-10T16:00:14.951 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "43'6200", 'reported_seq': 6323, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:00:11.331419+0000', 'last_change': '2024-09-10T15:59:16.277742+0000', 'last_active': '2024-09-10T16:00:11.331419+0000', 'last_peered': '2024-09-10T16:00:11.331419+0000', 'last_clean': '2024-09-10T15:59:14.129081+0000', 'last_became_active': '2024-09-10T15:59:16.269835+0000', 'last_became_peered': '2024-09-10T15:59:16.269835+0000', 'last_unstale': '2024-09-10T16:00:11.331419+0000', 'last_undegraded': '2024-09-10T15:59:15.860857+0000', 'last_fullsized': '2024-09-10T15:59:15.771410+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3557", 'last_scrub_stamp': '2024-09-10T15:59:02.493678+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:02.493678+0000', 'objects_scrubbed': 3488, 'log_size': 6200, 'log_dups_size': 0, 'ondisk_log_size': 6200, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1472, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25362432, 'num_objects': 6192, 'num_object_clones': 0, 'num_object_copies': 12384, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4257, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6192, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2704, 'num_write_kb': 10816, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1937}, {'shards': '0,2', 'objects': 4255}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:00:14.952 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-10T16:00:14.952 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "43'6197", 'reported_seq': 6326, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:00:09.302173+0000', 'last_change': '2024-09-10T15:59:16.284311+0000', 'last_active': '2024-09-10T16:00:09.302173+0000', 'last_peered': '2024-09-10T16:00:09.302173+0000', 'last_clean': '2024-09-10T15:59:14.116041+0000', 'last_became_active': '2024-09-10T15:59:16.274543+0000', 'last_became_peered': '2024-09-10T15:59:16.274543+0000', 'last_unstale': '2024-09-10T16:00:09.302173+0000', 'last_undegraded': '2024-09-10T15:59:15.812484+0000', 'last_fullsized': '2024-09-10T15:59:15.770885+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2356", 'last_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'objects_scrubbed': 2316, 'log_size': 6197, 'log_dups_size': 0, 'ondisk_log_size': 6197, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 783, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25362915, 'num_objects': 6193, 'num_object_clones': 0, 'num_object_copies': 12386, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6193, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 3877, 'num_write_kb': 15508, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4304}, {'shards': '2', 'objects': 1889}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:00:14.952 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-10T16:00:14.952 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "43'6324", 'reported_seq': 6447, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:00:09.281734+0000', 'last_change': '2024-09-10T15:59:16.174113+0000', 'last_active': '2024-09-10T16:00:09.281734+0000', 'last_peered': '2024-09-10T16:00:09.281734+0000', 'last_clean': '2024-09-10T15:59:14.110654+0000', 'last_became_active': '2024-09-10T15:59:16.164645+0000', 'last_became_peered': '2024-09-10T15:59:16.164645+0000', 'last_unstale': '2024-09-10T16:00:09.281734+0000', 'last_undegraded': '2024-09-10T15:59:15.803593+0000', 'last_fullsized': '2024-09-10T15:59:15.769083+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2486", 'last_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'objects_scrubbed': 2440, 'log_size': 6324, 'log_dups_size': 0, 'ondisk_log_size': 6324, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 792, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25878528, 'num_objects': 6318, 'num_object_clones': 0, 'num_object_copies': 12636, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4354, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6318, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3878, 'num_write_kb': 15512, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4352}, {'shards': '2', 'objects': 1966}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:00:14.952 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-10T16:00:14.952 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "43'6268", 'reported_seq': 6397, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:00:11.347236+0000', 'last_change': '2024-09-10T15:59:16.315641+0000', 'last_active': '2024-09-10T16:00:11.347236+0000', 'last_peered': '2024-09-10T16:00:11.347236+0000', 'last_clean': '2024-09-10T15:59:14.117001+0000', 'last_became_active': '2024-09-10T15:59:16.302245+0000', 'last_became_peered': '2024-09-10T15:59:16.302245+0000', 'last_unstale': '2024-09-10T16:00:11.347236+0000', 'last_undegraded': '2024-09-10T15:59:15.845012+0000', 'last_fullsized': '2024-09-10T15:59:15.770337+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3404", 'last_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'objects_scrubbed': 3367, 'log_size': 6268, 'log_dups_size': 0, 'ondisk_log_size': 6268, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 983, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25665536, 'num_objects': 6266, 'num_object_clones': 0, 'num_object_copies': 12532, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6266, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2899, 'num_write_kb': 11596, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1962}, {'shards': '0,2', 'objects': 4304}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:00:14.952 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-10T16:00:14.952 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "43'6155", 'reported_seq': 6292, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:00:11.281394+0000', 'last_change': '2024-09-10T15:59:16.251610+0000', 'last_active': '2024-09-10T16:00:11.281394+0000', 'last_peered': '2024-09-10T16:00:11.281394+0000', 'last_clean': '2024-09-10T15:59:14.115595+0000', 'last_became_active': '2024-09-10T15:59:16.244089+0000', 'last_became_peered': '2024-09-10T15:59:16.244089+0000', 'last_unstale': '2024-09-10T16:00:11.281394+0000', 'last_undegraded': '2024-09-10T15:59:15.847127+0000', 'last_fullsized': '2024-09-10T15:59:15.774437+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3653", 'last_scrub_stamp': '2024-09-10T15:59:05.298468+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:05.298468+0000', 'objects_scrubbed': 3602, 'log_size': 6155, 'log_dups_size': 0, 'ondisk_log_size': 6155, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1289, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25178112, 'num_objects': 6147, 'num_object_clones': 0, 'num_object_copies': 12294, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4206, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6147, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2545, 'num_write_kb': 10180, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1944}, {'shards': '0,2', 'objects': 4203}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:00:14.952 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-10T16:00:14.952 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "43'6406", 'reported_seq': 6542, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:00:11.299093+0000', 'last_change': '2024-09-10T15:59:16.311291+0000', 'last_active': '2024-09-10T16:00:11.299093+0000', 'last_peered': '2024-09-10T16:00:11.299093+0000', 'last_clean': '2024-09-10T15:59:14.116363+0000', 'last_became_active': '2024-09-10T15:59:16.301244+0000', 'last_became_peered': '2024-09-10T15:59:16.301244+0000', 'last_unstale': '2024-09-10T16:00:11.299093+0000', 'last_undegraded': '2024-09-10T15:59:15.883434+0000', 'last_fullsized': '2024-09-10T15:59:15.772756+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3789", 'last_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'objects_scrubbed': 3741, 'log_size': 6406, 'log_dups_size': 0, 'ondisk_log_size': 6406, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1440, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26222592, 'num_objects': 6402, 'num_object_clones': 0, 'num_object_copies': 12804, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4373, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6402, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2661, 'num_write_kb': 10644, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2031}, {'shards': '0,2', 'objects': 4371}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:00:14.952 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-10T16:00:14.953 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "43'6400", 'reported_seq': 6539, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:00:11.244673+0000', 'last_change': '2024-09-10T15:59:16.313409+0000', 'last_active': '2024-09-10T16:00:11.244673+0000', 'last_peered': '2024-09-10T16:00:11.244673+0000', 'last_clean': '2024-09-10T15:59:14.129780+0000', 'last_became_active': '2024-09-10T15:59:16.300932+0000', 'last_became_peered': '2024-09-10T15:59:16.300932+0000', 'last_unstale': '2024-09-10T16:00:11.244673+0000', 'last_undegraded': '2024-09-10T15:59:15.887823+0000', 'last_fullsized': '2024-09-10T15:59:15.768513+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3880", 'last_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'objects_scrubbed': 3827, 'log_size': 6400, 'log_dups_size': 0, 'ondisk_log_size': 6400, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1397, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26166214, 'num_objects': 6390, 'num_object_clones': 0, 'num_object_copies': 12780, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4325, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6390, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2563, 'num_write_kb': 10252, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2067}, {'shards': '0,2', 'objects': 4323}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:00:14.953 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T16:00:14.979 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 102 20 102338 102318 3.91771 1.91797 0.014425 0.019931 2024-09-10T16:00:15.673 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:00:15.673 INFO:teuthology.orchestra.run.smithi115.stdout:{"fsid":"7effee29-df05-4351-9741-ee203564ed81","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 47117/199990 objects degraded (23.560%), 11 pgs degraded","count":11},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":179,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":43,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983954,"num_in_osds":3,"osd_in_since":1725983947,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":99995,"data_bytes":409535217,"bytes_used":1287131136,"bytes_avail":288623161344,"bytes_total":289910292480,"degraded_objects":47117,"degraded_total":199990,"degraded_ratio":0.23559677983899191,"misplaced_objects":3125,"misplaced_total":199990,"misplaced_ratio":0.015625781289064455,"recovering_objects_per_sec":18,"recovering_bytes_per_sec":75413,"recovering_keys_per_sec":0,"num_objects_recovered":221,"num_bytes_recovered":905216,"num_keys_recovered":0,"write_bytes_sec":1871692,"read_op_per_sec":0,"write_op_per_sec":456},"fsmap":{"epoch":1,"btime":"2024-09-10T15:57:14:967120+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T16:00:13.560776+0000","services":{}},"progress_events":{}} 2024-09-10T16:00:15.705 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-10T16:00:15.706 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:00:15.980 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 103 20 102831 102811 3.89837 1.92578 0.0221052 0.020033 2024-09-10T16:00:16.328 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:00:16.329 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:00:16.359 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":126,"stamp":"2024-09-10T16:00:15.561293+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":413700849,"num_objects":101012,"num_object_clones":0,"num_object_copies":202024,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":47117,"num_objects_misplaced":3009,"num_objects_unfound":0,"num_objects_dirty":101012,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":58858,"num_write_kb":235426,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1264,"num_bytes_recovered":5169171,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":101085,"ondisk_log_size":101085,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1279120,"kb_used_data":814816,"kb_used_omap":12,"kb_used_meta":464243,"kb_avail":281836400,"statfs":{"total":289910292480,"available":288600473600,"internally_reserved":0,"allocated":834371584,"data_stored":833483181,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":475384834},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,6,9,7,10,7,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":26632192,"num_objects":6502,"num_object_clones":0,"num_object_copies":13004,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-337,"num_objects_unfound":0,"num_objects_dirty":6502,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6502,"num_write_kb":26008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":337,"num_bytes_recovered":1380352,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003312"},"pg_stats":[{"pgid":"2.f","version":"43'6237","reported_seq":6306,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:11.341997+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:00:11.341997+0000","last_peered":"2024-09-10T16:00:11.341997+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:00:11.341997+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":6237,"log_dups_size":0,"ondisk_log_size":6237,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25546752,"num_objects":6237,"num_object_clones":0,"num_object_copies":12474,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6237,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6237,"num_write_kb":24948,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1978},{"shards":"0,2","objects":4258}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'6361","reported_seq":6468,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:00:14.402609+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:00:14.402609+0000","last_peered":"2024-09-10T16:00:14.402609+0000","last_clean":"2024-09-10T16:00:14.402609+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:00:14.402609+0000","last_undegraded":"2024-09-10T16:00:14.402609+0000","last_fullsized":"2024-09-10T16:00:14.402609+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":6361,"log_dups_size":0,"ondisk_log_size":6361,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26031046,"num_objects":6357,"num_object_clones":0,"num_object_copies":12714,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6357,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4466,"num_write_kb":17864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'6332","reported_seq":8974,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T16:00:14.414115+0000","last_change":"2024-09-10T15:59:16.142612+0000","last_active":"2024-09-10T16:00:14.414115+0000","last_peered":"2024-09-10T16:00:14.414115+0000","last_clean":"2024-09-10T15:59:14.092473+0000","last_became_active":"2024-09-10T15:59:16.109611+0000","last_became_peered":"2024-09-10T15:59:16.109611+0000","last_unstale":"2024-09-10T16:00:14.414115+0000","last_undegraded":"2024-09-10T16:00:14.414115+0000","last_fullsized":"2024-09-10T15:59:15.770264+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":6332,"log_dups_size":0,"ondisk_log_size":6332,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25919488,"num_objects":6328,"num_object_clones":0,"num_object_copies":12656,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3009,"num_objects_unfound":0,"num_objects_dirty":6328,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4336,"num_write_kb":17344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1259,"num_bytes_recovered":5156864,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3009},{"shards":"2","objects":3319}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'6366","reported_seq":6441,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:00:14.354728+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:00:14.354728+0000","last_peered":"2024-09-10T16:00:14.354728+0000","last_clean":"2024-09-10T16:00:14.354728+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:00:14.354728+0000","last_undegraded":"2024-09-10T16:00:14.354728+0000","last_fullsized":"2024-09-10T16:00:14.354728+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":6366,"log_dups_size":0,"ondisk_log_size":6366,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26075136,"num_objects":6366,"num_object_clones":0,"num_object_copies":12732,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6366,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6366,"num_write_kb":25464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'6313","reported_seq":6437,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:00:11.322198+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:00:11.322198+0000","last_peered":"2024-09-10T16:00:11.322198+0000","last_clean":"2024-09-10T16:00:11.322198+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:00:11.322198+0000","last_undegraded":"2024-09-10T16:00:11.322198+0000","last_fullsized":"2024-09-10T16:00:11.322198+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":6313,"log_dups_size":0,"ondisk_log_size":6313,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25829859,"num_objects":6307,"num_object_clones":0,"num_object_copies":12614,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6307,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3024,"num_write_kb":12096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'6397","reported_seq":6528,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:11.352016+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:00:11.352016+0000","last_peered":"2024-09-10T16:00:11.352016+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:00:11.352016+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":6397,"log_dups_size":0,"ondisk_log_size":6397,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26182115,"num_objects":6393,"num_object_clones":0,"num_object_copies":12786,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6393,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3009,"num_write_kb":12036,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2022},{"shards":"0,2","objects":4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'6239","reported_seq":6371,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:11.339874+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:00:11.339874+0000","last_peered":"2024-09-10T16:00:11.339874+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:00:11.339874+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":6239,"log_dups_size":0,"ondisk_log_size":6239,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25531334,"num_objects":6235,"num_object_clones":0,"num_object_copies":12470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6235,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":2738,"num_write_kb":10952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1998},{"shards":"0,2","objects":4237}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'6383","reported_seq":6508,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:00:11.363960+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:00:11.363960+0000","last_peered":"2024-09-10T16:00:11.363960+0000","last_clean":"2024-09-10T16:00:11.363960+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:00:11.363960+0000","last_undegraded":"2024-09-10T16:00:11.363960+0000","last_fullsized":"2024-09-10T16:00:11.363960+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":6383,"log_dups_size":0,"ondisk_log_size":6383,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26136576,"num_objects":6381,"num_object_clones":0,"num_object_copies":12762,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6381,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3230,"num_write_kb":12920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.775865+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T15:59:15.775865+0000","last_peered":"2024-09-10T15:59:15.775865+0000","last_clean":"2024-09-10T15:59:15.775865+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T15:59:15.775865+0000","last_undegraded":"2024-09-10T15:59:15.775865+0000","last_fullsized":"2024-09-10T15:59:15.775865+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'6142","reported_seq":6276,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:14.372488+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:00:14.372488+0000","last_peered":"2024-09-10T16:00:14.372488+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:00:14.372488+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":6142,"log_dups_size":0,"ondisk_log_size":6142,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25129443,"num_objects":6136,"num_object_clones":0,"num_object_copies":12272,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6136,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3961,"num_write_kb":15844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4123},{"shards":"2","objects":2013}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":88,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.772572+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T15:59:15.772572+0000","last_peered":"2024-09-10T15:59:15.772572+0000","last_clean":"2024-09-10T15:59:15.772572+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T15:59:15.772572+0000","last_undegraded":"2024-09-10T15:59:15.772572+0000","last_fullsized":"2024-09-10T15:59:15.772572+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'6200","reported_seq":6323,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:11.331419+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:00:11.331419+0000","last_peered":"2024-09-10T16:00:11.331419+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:00:11.331419+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":6200,"log_dups_size":0,"ondisk_log_size":6200,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25362432,"num_objects":6192,"num_object_clones":0,"num_object_copies":12384,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6192,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2704,"num_write_kb":10816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1937},{"shards":"0,2","objects":4255}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":99,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.884297+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T15:59:15.884297+0000","last_peered":"2024-09-10T15:59:15.884297+0000","last_clean":"2024-09-10T15:59:15.884297+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T15:59:15.884297+0000","last_undegraded":"2024-09-10T15:59:15.884297+0000","last_fullsized":"2024-09-10T15:59:15.884297+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'6381","reported_seq":6510,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:14.409717+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:00:14.409717+0000","last_peered":"2024-09-10T16:00:14.409717+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:00:14.409717+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":6381,"log_dups_size":0,"ondisk_log_size":6381,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26116579,"num_objects":6377,"num_object_clones":0,"num_object_copies":12754,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6377,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4061,"num_write_kb":16244,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4304},{"shards":"2","objects":2073}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.837509+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T15:59:15.837509+0000","last_peered":"2024-09-10T15:59:15.837509+0000","last_clean":"2024-09-10T15:59:15.837509+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T15:59:15.837509+0000","last_undegraded":"2024-09-10T15:59:15.837509+0000","last_fullsized":"2024-09-10T15:59:15.837509+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'6502","reported_seq":6625,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:14.383008+0000","last_change":"2024-09-10T15:59:16.174113+0000","last_active":"2024-09-10T16:00:14.383008+0000","last_peered":"2024-09-10T16:00:14.383008+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:00:14.383008+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":6502,"log_dups_size":0,"ondisk_log_size":6502,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26607616,"num_objects":6496,"num_object_clones":0,"num_object_copies":12992,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4354,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6496,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4056,"num_write_kb":16224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4352},{"shards":"2","objects":2144}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'6268","reported_seq":6397,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:11.347236+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:00:11.347236+0000","last_peered":"2024-09-10T16:00:11.347236+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:00:11.347236+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":6268,"log_dups_size":0,"ondisk_log_size":6268,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25665536,"num_objects":6266,"num_object_clones":0,"num_object_copies":12532,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6266,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2899,"num_write_kb":11596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1962},{"shards":"0,2","objects":4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.989413+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T15:59:15.989413+0000","last_peered":"2024-09-10T15:59:15.989413+0000","last_clean":"2024-09-10T15:59:15.989413+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T15:59:15.989413+0000","last_undegraded":"2024-09-10T15:59:15.989413+0000","last_fullsized":"2024-09-10T15:59:15.989413+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'6155","reported_seq":6292,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:11.281394+0000","last_change":"2024-09-10T15:59:16.251610+0000","last_active":"2024-09-10T16:00:11.281394+0000","last_peered":"2024-09-10T16:00:11.281394+0000","last_clean":"2024-09-10T15:59:14.115595+0000","last_became_active":"2024-09-10T15:59:16.244089+0000","last_became_peered":"2024-09-10T15:59:16.244089+0000","last_unstale":"2024-09-10T16:00:11.281394+0000","last_undegraded":"2024-09-10T15:59:15.847127+0000","last_fullsized":"2024-09-10T15:59:15.774437+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":6155,"log_dups_size":0,"ondisk_log_size":6155,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25178112,"num_objects":6147,"num_object_clones":0,"num_object_copies":12294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6147,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2545,"num_write_kb":10180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1944},{"shards":"0,2","objects":4203}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":88,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.773583+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T15:59:15.773583+0000","last_peered":"2024-09-10T15:59:15.773583+0000","last_clean":"2024-09-10T15:59:15.773583+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T15:59:15.773583+0000","last_undegraded":"2024-09-10T15:59:15.773583+0000","last_fullsized":"2024-09-10T15:59:15.773583+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'6406","reported_seq":6542,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:11.299093+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:00:11.299093+0000","last_peered":"2024-09-10T16:00:11.299093+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:00:11.299093+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":6406,"log_dups_size":0,"ondisk_log_size":6406,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26222592,"num_objects":6402,"num_object_clones":0,"num_object_copies":12804,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6402,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2661,"num_write_kb":10644,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2031},{"shards":"0,2","objects":4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.916017+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T15:59:15.916017+0000","last_peered":"2024-09-10T15:59:15.916017+0000","last_clean":"2024-09-10T15:59:15.916017+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T15:59:15.916017+0000","last_undegraded":"2024-09-10T15:59:15.916017+0000","last_fullsized":"2024-09-10T15:59:15.916017+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.776033+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T15:59:15.776033+0000","last_peered":"2024-09-10T15:59:15.776033+0000","last_clean":"2024-09-10T15:59:15.776033+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T15:59:15.776033+0000","last_undegraded":"2024-09-10T15:59:15.776033+0000","last_fullsized":"2024-09-10T15:59:15.776033+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'6400","reported_seq":6539,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:11.244673+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:00:11.244673+0000","last_peered":"2024-09-10T16:00:11.244673+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:00:11.244673+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":6400,"log_dups_size":0,"ondisk_log_size":6400,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26166214,"num_objects":6390,"num_object_clones":0,"num_object_copies":12780,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6390,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2563,"num_write_kb":10252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2067},{"shards":"0,2","objects":4323}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":413700830,"num_objects":101010,"num_object_clones":0,"num_object_copies":202020,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":47117,"num_objects_misplaced":3009,"num_objects_unfound":0,"num_objects_dirty":101010,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":58856,"num_write_kb":235424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1262,"num_bytes_recovered":5169152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":834883584,"data_stored":834728225,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":101082,"ondisk_log_size":101082,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986203,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":516960,"kb_used_data":340880,"kb_used_omap":4,"kb_used_meta":176059,"kb_avail":93854880,"statfs":{"total":96636764160,"available":96107397120,"internally_reserved":0,"allocated":349061120,"data_stored":348783360,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":180285098},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,1,4,3,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659149,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":216092,"kb_used_data":103884,"kb_used_omap":4,"kb_used_meta":112187,"kb_avail":94155748,"statfs":{"total":96636764160,"available":96415485952,"internally_reserved":0,"allocated":106377216,"data_stored":106048410,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":114880174},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,4,2,4,3,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182431,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":546068,"kb_used_data":370052,"kb_used_omap":4,"kb_used_meta":175995,"kb_avail":93825772,"statfs":{"total":96636764160,"available":96077590528,"internally_reserved":0,"allocated":378933248,"data_stored":378651411,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":180219562},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,6,2,3,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":379035648,"data_stored":378999518,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":106569728,"data_stored":106486629,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":349278208,"data_stored":349242078,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:00:16.979 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 104 20 103374 103354 3.88128 2.12109 0.0469861 0.0201218 2024-09-10T16:00:17.979 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 105 20 103929 103909 3.86496 2.16797 0.0181301 0.0202051 2024-09-10T16:00:18.980 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 106 20 104499 104479 3.8495 2.22656 0.0229436 0.0202878 2024-09-10T16:00:19.361 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:00:19.927 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:00:19.928 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:00:19.953 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":128,"stamp":"2024-09-10T16:00:19.563035+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":424776433,"num_objects":103716,"num_object_clones":0,"num_object_copies":207432,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":47117,"num_objects_misplaced":2894,"num_objects_unfound":0,"num_objects_dirty":103716,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":61562,"num_write_kb":246242,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1379,"num_bytes_recovered":5640211,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":103789,"ondisk_log_size":103789,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1318648,"kb_used_data":836296,"kb_used_omap":12,"kb_used_meta":482291,"kb_avail":281796872,"statfs":{"total":289910292480,"available":288559996928,"internally_reserved":0,"allocated":856367104,"data_stored":855478701,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":493865986},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,8,3,7,9,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":26562560,"num_objects":6485,"num_object_clones":0,"num_object_copies":12970,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-340,"num_objects_unfound":0,"num_objects_dirty":6485,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6485,"num_write_kb":25940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":340,"num_bytes_recovered":1392640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003958"},"pg_stats":[{"pgid":"2.f","version":"43'6400","reported_seq":6466,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:16.402739+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:00:16.402739+0000","last_peered":"2024-09-10T16:00:16.402739+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:00:16.402739+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":6400,"log_dups_size":0,"ondisk_log_size":6400,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26214400,"num_objects":6400,"num_object_clones":0,"num_object_copies":12800,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6400,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6400,"num_write_kb":25600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2141},{"shards":"0,2","objects":4258}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'6530","reported_seq":6637,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:00:19.397543+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:00:19.397543+0000","last_peered":"2024-09-10T16:00:19.397543+0000","last_clean":"2024-09-10T16:00:19.397543+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:00:19.397543+0000","last_undegraded":"2024-09-10T16:00:19.397543+0000","last_fullsized":"2024-09-10T16:00:19.397543+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":6530,"log_dups_size":0,"ondisk_log_size":6530,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26723270,"num_objects":6526,"num_object_clones":0,"num_object_copies":13052,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6526,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4635,"num_write_kb":18540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'6496","reported_seq":9369,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T16:00:19.435342+0000","last_change":"2024-09-10T15:59:16.142612+0000","last_active":"2024-09-10T16:00:19.435342+0000","last_peered":"2024-09-10T16:00:19.435342+0000","last_clean":"2024-09-10T15:59:14.092473+0000","last_became_active":"2024-09-10T15:59:16.109611+0000","last_became_peered":"2024-09-10T15:59:16.109611+0000","last_unstale":"2024-09-10T16:00:19.435342+0000","last_undegraded":"2024-09-10T16:00:19.435342+0000","last_fullsized":"2024-09-10T15:59:15.770264+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":6496,"log_dups_size":0,"ondisk_log_size":6496,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26591232,"num_objects":6492,"num_object_clones":0,"num_object_copies":12984,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2894,"num_objects_unfound":0,"num_objects_dirty":6492,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4500,"num_write_kb":18000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1374,"num_bytes_recovered":5627904,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2894},{"shards":"2","objects":3598}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'6547","reported_seq":6622,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:00:19.422985+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:00:19.422985+0000","last_peered":"2024-09-10T16:00:19.422985+0000","last_clean":"2024-09-10T16:00:19.422985+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:00:19.422985+0000","last_undegraded":"2024-09-10T16:00:19.422985+0000","last_fullsized":"2024-09-10T16:00:19.422985+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":6547,"log_dups_size":0,"ondisk_log_size":6547,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26816512,"num_objects":6547,"num_object_clones":0,"num_object_copies":13094,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6547,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6547,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'6459","reported_seq":6584,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:00:16.396266+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:00:16.396266+0000","last_peered":"2024-09-10T16:00:16.396266+0000","last_clean":"2024-09-10T16:00:16.396266+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:00:16.396266+0000","last_undegraded":"2024-09-10T16:00:16.396266+0000","last_fullsized":"2024-09-10T16:00:16.396266+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":6459,"log_dups_size":0,"ondisk_log_size":6459,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26427875,"num_objects":6453,"num_object_clones":0,"num_object_copies":12906,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6453,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3170,"num_write_kb":12680,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'6551","reported_seq":6682,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:16.395762+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:00:16.395762+0000","last_peered":"2024-09-10T16:00:16.395762+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:00:16.395762+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":6551,"log_dups_size":0,"ondisk_log_size":6551,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26812899,"num_objects":6547,"num_object_clones":0,"num_object_copies":13094,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6547,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3163,"num_write_kb":12652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2176},{"shards":"0,2","objects":4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'6447","reported_seq":6579,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:16.370580+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:00:16.370580+0000","last_peered":"2024-09-10T16:00:16.370580+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:00:16.370580+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":6447,"log_dups_size":0,"ondisk_log_size":6447,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26383302,"num_objects":6443,"num_object_clones":0,"num_object_copies":12886,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6443,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":2946,"num_write_kb":11784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2206},{"shards":"0,2","objects":4237}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'6541","reported_seq":6665,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:00:16.378507+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:00:16.378507+0000","last_peered":"2024-09-10T16:00:16.378507+0000","last_clean":"2024-09-10T16:00:16.378507+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:00:16.378507+0000","last_undegraded":"2024-09-10T16:00:16.378507+0000","last_fullsized":"2024-09-10T16:00:16.378507+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":6541,"log_dups_size":0,"ondisk_log_size":6541,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26783744,"num_objects":6539,"num_object_clones":0,"num_object_copies":13078,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6539,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3388,"num_write_kb":13552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.775865+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T15:59:15.775865+0000","last_peered":"2024-09-10T15:59:15.775865+0000","last_clean":"2024-09-10T15:59:15.775865+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T15:59:15.775865+0000","last_undegraded":"2024-09-10T15:59:15.775865+0000","last_fullsized":"2024-09-10T15:59:15.775865+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'6325","reported_seq":6459,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:19.386748+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:00:19.386748+0000","last_peered":"2024-09-10T16:00:19.386748+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:00:19.386748+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":6325,"log_dups_size":0,"ondisk_log_size":6325,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25879011,"num_objects":6319,"num_object_clones":0,"num_object_copies":12638,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6319,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4144,"num_write_kb":16576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4123},{"shards":"2","objects":2196}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":88,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.772572+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T15:59:15.772572+0000","last_peered":"2024-09-10T15:59:15.772572+0000","last_clean":"2024-09-10T15:59:15.772572+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T15:59:15.772572+0000","last_undegraded":"2024-09-10T15:59:15.772572+0000","last_fullsized":"2024-09-10T15:59:15.772572+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'6372","reported_seq":6496,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:16.393228+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:00:16.393228+0000","last_peered":"2024-09-10T16:00:16.393228+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:00:16.393228+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":6372,"log_dups_size":0,"ondisk_log_size":6372,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26066944,"num_objects":6364,"num_object_clones":0,"num_object_copies":12728,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6364,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2876,"num_write_kb":11504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2109},{"shards":"0,2","objects":4255}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":99,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.884297+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T15:59:15.884297+0000","last_peered":"2024-09-10T15:59:15.884297+0000","last_clean":"2024-09-10T15:59:15.884297+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T15:59:15.884297+0000","last_undegraded":"2024-09-10T15:59:15.884297+0000","last_fullsized":"2024-09-10T15:59:15.884297+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'6561","reported_seq":6688,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:19.430892+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:00:19.430892+0000","last_peered":"2024-09-10T16:00:19.430892+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:00:19.430892+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":6561,"log_dups_size":0,"ondisk_log_size":6561,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26853859,"num_objects":6557,"num_object_clones":0,"num_object_copies":13114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6557,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4241,"num_write_kb":16964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4304},{"shards":"2","objects":2253}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.837509+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T15:59:15.837509+0000","last_peered":"2024-09-10T15:59:15.837509+0000","last_clean":"2024-09-10T15:59:15.837509+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T15:59:15.837509+0000","last_undegraded":"2024-09-10T15:59:15.837509+0000","last_fullsized":"2024-09-10T15:59:15.837509+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'6673","reported_seq":6797,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:19.436623+0000","last_change":"2024-09-10T15:59:16.174113+0000","last_active":"2024-09-10T16:00:19.436623+0000","last_peered":"2024-09-10T16:00:19.436623+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:00:19.436623+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":6673,"log_dups_size":0,"ondisk_log_size":6673,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27308032,"num_objects":6667,"num_object_clones":0,"num_object_copies":13334,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4354,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6667,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4227,"num_write_kb":16908,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4352},{"shards":"2","objects":2315}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'6427","reported_seq":6556,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:16.341569+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:00:16.341569+0000","last_peered":"2024-09-10T16:00:16.341569+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:00:16.341569+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":6427,"log_dups_size":0,"ondisk_log_size":6427,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26316800,"num_objects":6425,"num_object_clones":0,"num_object_copies":12850,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6425,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3058,"num_write_kb":12232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2121},{"shards":"0,2","objects":4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.989413+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T15:59:15.989413+0000","last_peered":"2024-09-10T15:59:15.989413+0000","last_clean":"2024-09-10T15:59:15.989413+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T15:59:15.989413+0000","last_undegraded":"2024-09-10T15:59:15.989413+0000","last_fullsized":"2024-09-10T15:59:15.989413+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'6335","reported_seq":6472,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:16.366283+0000","last_change":"2024-09-10T15:59:16.251610+0000","last_active":"2024-09-10T16:00:16.366283+0000","last_peered":"2024-09-10T16:00:16.366283+0000","last_clean":"2024-09-10T15:59:14.115595+0000","last_became_active":"2024-09-10T15:59:16.244089+0000","last_became_peered":"2024-09-10T15:59:16.244089+0000","last_unstale":"2024-09-10T16:00:16.366283+0000","last_undegraded":"2024-09-10T15:59:15.847127+0000","last_fullsized":"2024-09-10T15:59:15.774437+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":6335,"log_dups_size":0,"ondisk_log_size":6335,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25915392,"num_objects":6327,"num_object_clones":0,"num_object_copies":12654,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6327,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2725,"num_write_kb":10900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2124},{"shards":"0,2","objects":4203}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":88,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.773583+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T15:59:15.773583+0000","last_peered":"2024-09-10T15:59:15.773583+0000","last_clean":"2024-09-10T15:59:15.773583+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T15:59:15.773583+0000","last_undegraded":"2024-09-10T15:59:15.773583+0000","last_fullsized":"2024-09-10T15:59:15.773583+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'6560","reported_seq":6696,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:16.371837+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:00:16.371837+0000","last_peered":"2024-09-10T16:00:16.371837+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:00:16.371837+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":6560,"log_dups_size":0,"ondisk_log_size":6560,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26853376,"num_objects":6556,"num_object_clones":0,"num_object_copies":13112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6556,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2815,"num_write_kb":11260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2185},{"shards":"0,2","objects":4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.916017+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T15:59:15.916017+0000","last_peered":"2024-09-10T15:59:15.916017+0000","last_clean":"2024-09-10T15:59:15.916017+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T15:59:15.916017+0000","last_undegraded":"2024-09-10T15:59:15.916017+0000","last_fullsized":"2024-09-10T15:59:15.916017+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.776033+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T15:59:15.776033+0000","last_peered":"2024-09-10T15:59:15.776033+0000","last_clean":"2024-09-10T15:59:15.776033+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T15:59:15.776033+0000","last_undegraded":"2024-09-10T15:59:15.776033+0000","last_fullsized":"2024-09-10T15:59:15.776033+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'6562","reported_seq":6701,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:16.355664+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:00:16.355664+0000","last_peered":"2024-09-10T16:00:16.355664+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:00:16.355664+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":6562,"log_dups_size":0,"ondisk_log_size":6562,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26829766,"num_objects":6552,"num_object_clones":0,"num_object_copies":13104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6552,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2725,"num_write_kb":10900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2229},{"shards":"0,2","objects":4323}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":424776414,"num_objects":103714,"num_object_clones":0,"num_object_copies":207428,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":47117,"num_objects_misplaced":2894,"num_objects_unfound":0,"num_objects_dirty":103714,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":61560,"num_write_kb":246240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1377,"num_bytes_recovered":5640192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":857387008,"data_stored":857231649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":103786,"ondisk_log_size":103786,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986204,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":526308,"kb_used_data":346068,"kb_used_omap":4,"kb_used_meta":180219,"kb_avail":93845532,"statfs":{"total":96636764160,"available":96097824768,"internally_reserved":0,"allocated":354373632,"data_stored":354095872,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":184544938},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,1,0,4,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659150,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":216216,"kb_used_data":112328,"kb_used_omap":4,"kb_used_meta":103867,"kb_avail":94155624,"statfs":{"total":96636764160,"available":96415358976,"internally_reserved":0,"allocated":115023872,"data_stored":114695066,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":106360494},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,2,3,3,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182432,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":576124,"kb_used_data":377900,"kb_used_omap":4,"kb_used_meta":198203,"kb_avail":93795716,"statfs":{"total":96636764160,"available":96046813184,"internally_reserved":0,"allocated":386969600,"data_stored":386687763,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":202960554},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,5,0,4,2,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":387100672,"data_stored":387064542,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":115306496,"data_stored":115223397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":354979840,"data_stored":354943710,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:00:19.955 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:00:19.980 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 107 20 105040 105020 3.83328 2.11328 0.0288472 0.0203718 2024-09-10T16:00:20.529 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:00:20.531 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:00:20.554 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":128,"stamp":"2024-09-10T16:00:19.563035+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":424776433,"num_objects":103716,"num_object_clones":0,"num_object_copies":207432,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":47117,"num_objects_misplaced":2894,"num_objects_unfound":0,"num_objects_dirty":103716,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":61562,"num_write_kb":246242,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1379,"num_bytes_recovered":5640211,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":103789,"ondisk_log_size":103789,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1318648,"kb_used_data":836296,"kb_used_omap":12,"kb_used_meta":482291,"kb_avail":281796872,"statfs":{"total":289910292480,"available":288559996928,"internally_reserved":0,"allocated":856367104,"data_stored":855478701,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":493865986},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,8,3,7,9,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":26562560,"num_objects":6485,"num_object_clones":0,"num_object_copies":12970,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-340,"num_objects_unfound":0,"num_objects_dirty":6485,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6485,"num_write_kb":25940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":340,"num_bytes_recovered":1392640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003958"},"pg_stats":[{"pgid":"2.f","version":"43'6400","reported_seq":6466,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:16.402739+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:00:16.402739+0000","last_peered":"2024-09-10T16:00:16.402739+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:00:16.402739+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":6400,"log_dups_size":0,"ondisk_log_size":6400,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26214400,"num_objects":6400,"num_object_clones":0,"num_object_copies":12800,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6400,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6400,"num_write_kb":25600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2141},{"shards":"0,2","objects":4258}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'6530","reported_seq":6637,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:00:19.397543+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:00:19.397543+0000","last_peered":"2024-09-10T16:00:19.397543+0000","last_clean":"2024-09-10T16:00:19.397543+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:00:19.397543+0000","last_undegraded":"2024-09-10T16:00:19.397543+0000","last_fullsized":"2024-09-10T16:00:19.397543+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":6530,"log_dups_size":0,"ondisk_log_size":6530,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26723270,"num_objects":6526,"num_object_clones":0,"num_object_copies":13052,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6526,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4635,"num_write_kb":18540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'6496","reported_seq":9369,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T16:00:19.435342+0000","last_change":"2024-09-10T15:59:16.142612+0000","last_active":"2024-09-10T16:00:19.435342+0000","last_peered":"2024-09-10T16:00:19.435342+0000","last_clean":"2024-09-10T15:59:14.092473+0000","last_became_active":"2024-09-10T15:59:16.109611+0000","last_became_peered":"2024-09-10T15:59:16.109611+0000","last_unstale":"2024-09-10T16:00:19.435342+0000","last_undegraded":"2024-09-10T16:00:19.435342+0000","last_fullsized":"2024-09-10T15:59:15.770264+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":6496,"log_dups_size":0,"ondisk_log_size":6496,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26591232,"num_objects":6492,"num_object_clones":0,"num_object_copies":12984,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2894,"num_objects_unfound":0,"num_objects_dirty":6492,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4500,"num_write_kb":18000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1374,"num_bytes_recovered":5627904,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2894},{"shards":"2","objects":3598}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'6547","reported_seq":6622,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:00:19.422985+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:00:19.422985+0000","last_peered":"2024-09-10T16:00:19.422985+0000","last_clean":"2024-09-10T16:00:19.422985+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:00:19.422985+0000","last_undegraded":"2024-09-10T16:00:19.422985+0000","last_fullsized":"2024-09-10T16:00:19.422985+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":6547,"log_dups_size":0,"ondisk_log_size":6547,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26816512,"num_objects":6547,"num_object_clones":0,"num_object_copies":13094,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6547,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6547,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'6459","reported_seq":6584,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:00:16.396266+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:00:16.396266+0000","last_peered":"2024-09-10T16:00:16.396266+0000","last_clean":"2024-09-10T16:00:16.396266+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:00:16.396266+0000","last_undegraded":"2024-09-10T16:00:16.396266+0000","last_fullsized":"2024-09-10T16:00:16.396266+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":6459,"log_dups_size":0,"ondisk_log_size":6459,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26427875,"num_objects":6453,"num_object_clones":0,"num_object_copies":12906,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6453,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3170,"num_write_kb":12680,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'6551","reported_seq":6682,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:16.395762+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:00:16.395762+0000","last_peered":"2024-09-10T16:00:16.395762+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:00:16.395762+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":6551,"log_dups_size":0,"ondisk_log_size":6551,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26812899,"num_objects":6547,"num_object_clones":0,"num_object_copies":13094,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6547,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3163,"num_write_kb":12652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2176},{"shards":"0,2","objects":4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'6447","reported_seq":6579,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:16.370580+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:00:16.370580+0000","last_peered":"2024-09-10T16:00:16.370580+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:00:16.370580+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":6447,"log_dups_size":0,"ondisk_log_size":6447,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26383302,"num_objects":6443,"num_object_clones":0,"num_object_copies":12886,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6443,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":2946,"num_write_kb":11784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2206},{"shards":"0,2","objects":4237}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'6541","reported_seq":6665,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:00:16.378507+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:00:16.378507+0000","last_peered":"2024-09-10T16:00:16.378507+0000","last_clean":"2024-09-10T16:00:16.378507+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:00:16.378507+0000","last_undegraded":"2024-09-10T16:00:16.378507+0000","last_fullsized":"2024-09-10T16:00:16.378507+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":6541,"log_dups_size":0,"ondisk_log_size":6541,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26783744,"num_objects":6539,"num_object_clones":0,"num_object_copies":13078,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6539,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3388,"num_write_kb":13552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.775865+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T15:59:15.775865+0000","last_peered":"2024-09-10T15:59:15.775865+0000","last_clean":"2024-09-10T15:59:15.775865+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T15:59:15.775865+0000","last_undegraded":"2024-09-10T15:59:15.775865+0000","last_fullsized":"2024-09-10T15:59:15.775865+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'6325","reported_seq":6459,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:19.386748+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:00:19.386748+0000","last_peered":"2024-09-10T16:00:19.386748+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:00:19.386748+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":6325,"log_dups_size":0,"ondisk_log_size":6325,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25879011,"num_objects":6319,"num_object_clones":0,"num_object_copies":12638,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6319,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4144,"num_write_kb":16576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4123},{"shards":"2","objects":2196}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":88,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.772572+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T15:59:15.772572+0000","last_peered":"2024-09-10T15:59:15.772572+0000","last_clean":"2024-09-10T15:59:15.772572+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T15:59:15.772572+0000","last_undegraded":"2024-09-10T15:59:15.772572+0000","last_fullsized":"2024-09-10T15:59:15.772572+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'6372","reported_seq":6496,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:16.393228+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:00:16.393228+0000","last_peered":"2024-09-10T16:00:16.393228+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:00:16.393228+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":6372,"log_dups_size":0,"ondisk_log_size":6372,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26066944,"num_objects":6364,"num_object_clones":0,"num_object_copies":12728,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6364,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2876,"num_write_kb":11504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2109},{"shards":"0,2","objects":4255}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":99,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.884297+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T15:59:15.884297+0000","last_peered":"2024-09-10T15:59:15.884297+0000","last_clean":"2024-09-10T15:59:15.884297+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T15:59:15.884297+0000","last_undegraded":"2024-09-10T15:59:15.884297+0000","last_fullsized":"2024-09-10T15:59:15.884297+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'6561","reported_seq":6688,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:19.430892+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:00:19.430892+0000","last_peered":"2024-09-10T16:00:19.430892+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:00:19.430892+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":6561,"log_dups_size":0,"ondisk_log_size":6561,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26853859,"num_objects":6557,"num_object_clones":0,"num_object_copies":13114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6557,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4241,"num_write_kb":16964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4304},{"shards":"2","objects":2253}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.837509+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T15:59:15.837509+0000","last_peered":"2024-09-10T15:59:15.837509+0000","last_clean":"2024-09-10T15:59:15.837509+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T15:59:15.837509+0000","last_undegraded":"2024-09-10T15:59:15.837509+0000","last_fullsized":"2024-09-10T15:59:15.837509+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'6673","reported_seq":6797,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:19.436623+0000","last_change":"2024-09-10T15:59:16.174113+0000","last_active":"2024-09-10T16:00:19.436623+0000","last_peered":"2024-09-10T16:00:19.436623+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:00:19.436623+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":6673,"log_dups_size":0,"ondisk_log_size":6673,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27308032,"num_objects":6667,"num_object_clones":0,"num_object_copies":13334,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4354,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6667,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4227,"num_write_kb":16908,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4352},{"shards":"2","objects":2315}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'6427","reported_seq":6556,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:16.341569+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:00:16.341569+0000","last_peered":"2024-09-10T16:00:16.341569+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:00:16.341569+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":6427,"log_dups_size":0,"ondisk_log_size":6427,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26316800,"num_objects":6425,"num_object_clones":0,"num_object_copies":12850,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6425,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3058,"num_write_kb":12232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2121},{"shards":"0,2","objects":4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.989413+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T15:59:15.989413+0000","last_peered":"2024-09-10T15:59:15.989413+0000","last_clean":"2024-09-10T15:59:15.989413+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T15:59:15.989413+0000","last_undegraded":"2024-09-10T15:59:15.989413+0000","last_fullsized":"2024-09-10T15:59:15.989413+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'6335","reported_seq":6472,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:16.366283+0000","last_change":"2024-09-10T15:59:16.251610+0000","last_active":"2024-09-10T16:00:16.366283+0000","last_peered":"2024-09-10T16:00:16.366283+0000","last_clean":"2024-09-10T15:59:14.115595+0000","last_became_active":"2024-09-10T15:59:16.244089+0000","last_became_peered":"2024-09-10T15:59:16.244089+0000","last_unstale":"2024-09-10T16:00:16.366283+0000","last_undegraded":"2024-09-10T15:59:15.847127+0000","last_fullsized":"2024-09-10T15:59:15.774437+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":6335,"log_dups_size":0,"ondisk_log_size":6335,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25915392,"num_objects":6327,"num_object_clones":0,"num_object_copies":12654,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6327,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2725,"num_write_kb":10900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2124},{"shards":"0,2","objects":4203}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":88,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.773583+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T15:59:15.773583+0000","last_peered":"2024-09-10T15:59:15.773583+0000","last_clean":"2024-09-10T15:59:15.773583+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T15:59:15.773583+0000","last_undegraded":"2024-09-10T15:59:15.773583+0000","last_fullsized":"2024-09-10T15:59:15.773583+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'6560","reported_seq":6696,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:16.371837+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:00:16.371837+0000","last_peered":"2024-09-10T16:00:16.371837+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:00:16.371837+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":6560,"log_dups_size":0,"ondisk_log_size":6560,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26853376,"num_objects":6556,"num_object_clones":0,"num_object_copies":13112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6556,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2815,"num_write_kb":11260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2185},{"shards":"0,2","objects":4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.916017+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T15:59:15.916017+0000","last_peered":"2024-09-10T15:59:15.916017+0000","last_clean":"2024-09-10T15:59:15.916017+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T15:59:15.916017+0000","last_undegraded":"2024-09-10T15:59:15.916017+0000","last_fullsized":"2024-09-10T15:59:15.916017+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.776033+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T15:59:15.776033+0000","last_peered":"2024-09-10T15:59:15.776033+0000","last_clean":"2024-09-10T15:59:15.776033+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T15:59:15.776033+0000","last_undegraded":"2024-09-10T15:59:15.776033+0000","last_fullsized":"2024-09-10T15:59:15.776033+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'6562","reported_seq":6701,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:16.355664+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:00:16.355664+0000","last_peered":"2024-09-10T16:00:16.355664+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:00:16.355664+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":6562,"log_dups_size":0,"ondisk_log_size":6562,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26829766,"num_objects":6552,"num_object_clones":0,"num_object_copies":13104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6552,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2725,"num_write_kb":10900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2229},{"shards":"0,2","objects":4323}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":424776414,"num_objects":103714,"num_object_clones":0,"num_object_copies":207428,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":47117,"num_objects_misplaced":2894,"num_objects_unfound":0,"num_objects_dirty":103714,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":61560,"num_write_kb":246240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1377,"num_bytes_recovered":5640192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":857387008,"data_stored":857231649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":103786,"ondisk_log_size":103786,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986204,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":526308,"kb_used_data":346068,"kb_used_omap":4,"kb_used_meta":180219,"kb_avail":93845532,"statfs":{"total":96636764160,"available":96097824768,"internally_reserved":0,"allocated":354373632,"data_stored":354095872,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":184544938},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,1,0,4,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659150,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":216216,"kb_used_data":112328,"kb_used_omap":4,"kb_used_meta":103867,"kb_avail":94155624,"statfs":{"total":96636764160,"available":96415358976,"internally_reserved":0,"allocated":115023872,"data_stored":114695066,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":106360494},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,2,3,3,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182432,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":576124,"kb_used_data":377900,"kb_used_omap":4,"kb_used_meta":198203,"kb_avail":93795716,"statfs":{"total":96636764160,"available":96046813184,"internally_reserved":0,"allocated":386969600,"data_stored":386687763,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":202960554},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,5,0,4,2,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":387100672,"data_stored":387064542,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":115306496,"data_stored":115223397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":354979840,"data_stored":354943710,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:00:20.555 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-10T16:00:20.555 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "43'6400", 'reported_seq': 6466, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:00:16.402739+0000', 'last_change': '2024-09-10T15:59:16.281558+0000', 'last_active': '2024-09-10T16:00:16.402739+0000', 'last_peered': '2024-09-10T16:00:16.402739+0000', 'last_clean': '2024-09-10T15:59:14.132456+0000', 'last_became_active': '2024-09-10T15:59:16.274407+0000', 'last_became_peered': '2024-09-10T15:59:16.274407+0000', 'last_unstale': '2024-09-10T16:00:16.402739+0000', 'last_undegraded': '2024-09-10T15:59:15.851279+0000', 'last_fullsized': '2024-09-10T15:59:15.774356+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'objects_scrubbed': 0, 'log_size': 6400, 'log_dups_size': 0, 'ondisk_log_size': 6400, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26214400, 'num_objects': 6400, 'num_object_clones': 0, 'num_object_copies': 12800, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4259, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6400, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6400, 'num_write_kb': 25600, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2141}, {'shards': '0,2', 'objects': 4258}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:00:20.556 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-10T16:00:20.556 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "43'6496", 'reported_seq': 9369, 'reported_epoch': 43, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-10T16:00:19.435342+0000', 'last_change': '2024-09-10T15:59:16.142612+0000', 'last_active': '2024-09-10T16:00:19.435342+0000', 'last_peered': '2024-09-10T16:00:19.435342+0000', 'last_clean': '2024-09-10T15:59:14.092473+0000', 'last_became_active': '2024-09-10T15:59:16.109611+0000', 'last_became_peered': '2024-09-10T15:59:16.109611+0000', 'last_unstale': '2024-09-10T16:00:19.435342+0000', 'last_undegraded': '2024-09-10T16:00:19.435342+0000', 'last_fullsized': '2024-09-10T15:59:15.770264+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2029", 'last_scrub_stamp': '2024-09-10T15:58:45.933789+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:45.933789+0000', 'objects_scrubbed': 1992, 'log_size': 6496, 'log_dups_size': 0, 'ondisk_log_size': 6496, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 668, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26591232, 'num_objects': 6492, 'num_object_clones': 0, 'num_object_copies': 12984, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 2894, 'num_objects_unfound': 0, 'num_objects_dirty': 6492, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4500, 'num_write_kb': 18000, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1374, 'num_bytes_recovered': 5627904, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2894}, {'shards': '2', 'objects': 3598}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:00:20.556 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-10T16:00:20.556 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "43'6551", 'reported_seq': 6682, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:00:16.395762+0000', 'last_change': '2024-09-10T15:59:16.308264+0000', 'last_active': '2024-09-10T16:00:16.395762+0000', 'last_peered': '2024-09-10T16:00:16.395762+0000', 'last_clean': '2024-09-10T15:59:14.111551+0000', 'last_became_active': '2024-09-10T15:59:16.300040+0000', 'last_became_peered': '2024-09-10T15:59:16.300040+0000', 'last_unstale': '2024-09-10T16:00:16.395762+0000', 'last_undegraded': '2024-09-10T15:59:15.878234+0000', 'last_fullsized': '2024-09-10T15:59:15.769328+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3442", 'last_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'objects_scrubbed': 3384, 'log_size': 6551, 'log_dups_size': 0, 'ondisk_log_size': 6551, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1292, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26812899, 'num_objects': 6547, 'num_object_clones': 0, 'num_object_copies': 13094, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4372, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6547, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 3163, 'num_write_kb': 12652, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2176}, {'shards': '0,2', 'objects': 4371}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:00:20.556 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-10T16:00:20.556 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "43'6447", 'reported_seq': 6579, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:00:16.370580+0000', 'last_change': '2024-09-10T15:59:16.331899+0000', 'last_active': '2024-09-10T16:00:16.370580+0000', 'last_peered': '2024-09-10T16:00:16.370580+0000', 'last_clean': '2024-09-10T15:59:14.139436+0000', 'last_became_active': '2024-09-10T15:59:16.318715+0000', 'last_became_peered': '2024-09-10T15:59:16.318715+0000', 'last_unstale': '2024-09-10T16:00:16.370580+0000', 'last_undegraded': '2024-09-10T15:59:15.877524+0000', 'last_fullsized': '2024-09-10T15:59:15.769042+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3537", 'last_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'objects_scrubbed': 3497, 'log_size': 6447, 'log_dups_size': 0, 'ondisk_log_size': 6447, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1224, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26383302, 'num_objects': 6443, 'num_object_clones': 0, 'num_object_copies': 12886, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4237, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6443, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 2946, 'num_write_kb': 11784, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2206}, {'shards': '0,2', 'objects': 4237}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:00:20.556 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-10T16:00:20.556 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "43'6325", 'reported_seq': 6459, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:00:19.386748+0000', 'last_change': '2024-09-10T15:59:16.175680+0000', 'last_active': '2024-09-10T16:00:19.386748+0000', 'last_peered': '2024-09-10T16:00:19.386748+0000', 'last_clean': '2024-09-10T15:59:14.088187+0000', 'last_became_active': '2024-09-10T15:59:16.165810+0000', 'last_became_peered': '2024-09-10T15:59:16.165810+0000', 'last_unstale': '2024-09-10T16:00:19.386748+0000', 'last_undegraded': '2024-09-10T15:59:15.822098+0000', 'last_fullsized': '2024-09-10T15:59:15.769123+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2218", 'last_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'objects_scrubbed': 2175, 'log_size': 6325, 'log_dups_size': 0, 'ondisk_log_size': 6325, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 825, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25879011, 'num_objects': 6319, 'num_object_clones': 0, 'num_object_copies': 12638, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4124, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6319, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 4144, 'num_write_kb': 16576, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4123}, {'shards': '2', 'objects': 2196}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:00:20.556 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-10T16:00:20.556 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "43'6372", 'reported_seq': 6496, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:00:16.393228+0000', 'last_change': '2024-09-10T15:59:16.277742+0000', 'last_active': '2024-09-10T16:00:16.393228+0000', 'last_peered': '2024-09-10T16:00:16.393228+0000', 'last_clean': '2024-09-10T15:59:14.129081+0000', 'last_became_active': '2024-09-10T15:59:16.269835+0000', 'last_became_peered': '2024-09-10T15:59:16.269835+0000', 'last_unstale': '2024-09-10T16:00:16.393228+0000', 'last_undegraded': '2024-09-10T15:59:15.860857+0000', 'last_fullsized': '2024-09-10T15:59:15.771410+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3557", 'last_scrub_stamp': '2024-09-10T15:59:02.493678+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:02.493678+0000', 'objects_scrubbed': 3488, 'log_size': 6372, 'log_dups_size': 0, 'ondisk_log_size': 6372, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1472, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26066944, 'num_objects': 6364, 'num_object_clones': 0, 'num_object_copies': 12728, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4257, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6364, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2876, 'num_write_kb': 11504, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2109}, {'shards': '0,2', 'objects': 4255}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:00:20.557 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-10T16:00:20.557 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "43'6561", 'reported_seq': 6688, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:00:19.430892+0000', 'last_change': '2024-09-10T15:59:16.284311+0000', 'last_active': '2024-09-10T16:00:19.430892+0000', 'last_peered': '2024-09-10T16:00:19.430892+0000', 'last_clean': '2024-09-10T15:59:14.116041+0000', 'last_became_active': '2024-09-10T15:59:16.274543+0000', 'last_became_peered': '2024-09-10T15:59:16.274543+0000', 'last_unstale': '2024-09-10T16:00:19.430892+0000', 'last_undegraded': '2024-09-10T15:59:15.812484+0000', 'last_fullsized': '2024-09-10T15:59:15.770885+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2356", 'last_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'objects_scrubbed': 2316, 'log_size': 6561, 'log_dups_size': 0, 'ondisk_log_size': 6561, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 783, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26853859, 'num_objects': 6557, 'num_object_clones': 0, 'num_object_copies': 13114, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6557, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 4241, 'num_write_kb': 16964, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4304}, {'shards': '2', 'objects': 2253}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:00:20.557 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-10T16:00:20.557 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "43'6673", 'reported_seq': 6797, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:00:19.436623+0000', 'last_change': '2024-09-10T15:59:16.174113+0000', 'last_active': '2024-09-10T16:00:19.436623+0000', 'last_peered': '2024-09-10T16:00:19.436623+0000', 'last_clean': '2024-09-10T15:59:14.110654+0000', 'last_became_active': '2024-09-10T15:59:16.164645+0000', 'last_became_peered': '2024-09-10T15:59:16.164645+0000', 'last_unstale': '2024-09-10T16:00:19.436623+0000', 'last_undegraded': '2024-09-10T15:59:15.803593+0000', 'last_fullsized': '2024-09-10T15:59:15.769083+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2486", 'last_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'objects_scrubbed': 2440, 'log_size': 6673, 'log_dups_size': 0, 'ondisk_log_size': 6673, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 792, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27308032, 'num_objects': 6667, 'num_object_clones': 0, 'num_object_copies': 13334, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4354, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6667, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4227, 'num_write_kb': 16908, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4352}, {'shards': '2', 'objects': 2315}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:00:20.557 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-10T16:00:20.557 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "43'6427", 'reported_seq': 6556, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:00:16.341569+0000', 'last_change': '2024-09-10T15:59:16.315641+0000', 'last_active': '2024-09-10T16:00:16.341569+0000', 'last_peered': '2024-09-10T16:00:16.341569+0000', 'last_clean': '2024-09-10T15:59:14.117001+0000', 'last_became_active': '2024-09-10T15:59:16.302245+0000', 'last_became_peered': '2024-09-10T15:59:16.302245+0000', 'last_unstale': '2024-09-10T16:00:16.341569+0000', 'last_undegraded': '2024-09-10T15:59:15.845012+0000', 'last_fullsized': '2024-09-10T15:59:15.770337+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3404", 'last_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'objects_scrubbed': 3367, 'log_size': 6427, 'log_dups_size': 0, 'ondisk_log_size': 6427, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 983, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26316800, 'num_objects': 6425, 'num_object_clones': 0, 'num_object_copies': 12850, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6425, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3058, 'num_write_kb': 12232, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2121}, {'shards': '0,2', 'objects': 4304}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:00:20.557 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-10T16:00:20.557 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "43'6335", 'reported_seq': 6472, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:00:16.366283+0000', 'last_change': '2024-09-10T15:59:16.251610+0000', 'last_active': '2024-09-10T16:00:16.366283+0000', 'last_peered': '2024-09-10T16:00:16.366283+0000', 'last_clean': '2024-09-10T15:59:14.115595+0000', 'last_became_active': '2024-09-10T15:59:16.244089+0000', 'last_became_peered': '2024-09-10T15:59:16.244089+0000', 'last_unstale': '2024-09-10T16:00:16.366283+0000', 'last_undegraded': '2024-09-10T15:59:15.847127+0000', 'last_fullsized': '2024-09-10T15:59:15.774437+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3653", 'last_scrub_stamp': '2024-09-10T15:59:05.298468+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:05.298468+0000', 'objects_scrubbed': 3602, 'log_size': 6335, 'log_dups_size': 0, 'ondisk_log_size': 6335, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1289, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25915392, 'num_objects': 6327, 'num_object_clones': 0, 'num_object_copies': 12654, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4206, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6327, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2725, 'num_write_kb': 10900, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2124}, {'shards': '0,2', 'objects': 4203}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:00:20.557 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-10T16:00:20.557 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "43'6560", 'reported_seq': 6696, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:00:16.371837+0000', 'last_change': '2024-09-10T15:59:16.311291+0000', 'last_active': '2024-09-10T16:00:16.371837+0000', 'last_peered': '2024-09-10T16:00:16.371837+0000', 'last_clean': '2024-09-10T15:59:14.116363+0000', 'last_became_active': '2024-09-10T15:59:16.301244+0000', 'last_became_peered': '2024-09-10T15:59:16.301244+0000', 'last_unstale': '2024-09-10T16:00:16.371837+0000', 'last_undegraded': '2024-09-10T15:59:15.883434+0000', 'last_fullsized': '2024-09-10T15:59:15.772756+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3789", 'last_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'objects_scrubbed': 3741, 'log_size': 6560, 'log_dups_size': 0, 'ondisk_log_size': 6560, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1440, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26853376, 'num_objects': 6556, 'num_object_clones': 0, 'num_object_copies': 13112, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4373, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6556, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2815, 'num_write_kb': 11260, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2185}, {'shards': '0,2', 'objects': 4371}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:00:20.558 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-10T16:00:20.558 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "43'6562", 'reported_seq': 6701, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:00:16.355664+0000', 'last_change': '2024-09-10T15:59:16.313409+0000', 'last_active': '2024-09-10T16:00:16.355664+0000', 'last_peered': '2024-09-10T16:00:16.355664+0000', 'last_clean': '2024-09-10T15:59:14.129780+0000', 'last_became_active': '2024-09-10T15:59:16.300932+0000', 'last_became_peered': '2024-09-10T15:59:16.300932+0000', 'last_unstale': '2024-09-10T16:00:16.355664+0000', 'last_undegraded': '2024-09-10T15:59:15.887823+0000', 'last_fullsized': '2024-09-10T15:59:15.768513+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3880", 'last_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'objects_scrubbed': 3827, 'log_size': 6562, 'log_dups_size': 0, 'ondisk_log_size': 6562, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1397, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26829766, 'num_objects': 6552, 'num_object_clones': 0, 'num_object_copies': 13104, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4325, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6552, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2725, 'num_write_kb': 10900, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2229}, {'shards': '0,2', 'objects': 4323}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:00:20.558 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T16:00:20.980 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 108 20 105539 105519 3.81583 1.94922 0.0263763 0.0204639 2024-09-10T16:00:21.295 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:00:21.295 INFO:teuthology.orchestra.run.smithi115.stdout:{"fsid":"7effee29-df05-4351-9741-ee203564ed81","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 47117/207432 objects degraded (22.714%), 11 pgs degraded, 12 pgs undersized","count":23},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":185,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":43,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983954,"num_in_osds":3,"osd_in_since":1725983947,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":103716,"data_bytes":424776433,"bytes_used":1350295552,"bytes_avail":288559996928,"bytes_total":289910292480,"degraded_objects":47117,"degraded_total":207432,"degraded_ratio":0.22714431717382083,"misplaced_objects":2894,"misplaced_total":207432,"misplaced_ratio":0.013951560029310811,"recovering_objects_per_sec":28,"recovering_bytes_per_sec":116015,"recovering_keys_per_sec":0,"num_objects_recovered":340,"num_bytes_recovered":1392640,"num_keys_recovered":0,"write_bytes_sec":2212816,"read_op_per_sec":0,"write_op_per_sec":540},"fsmap":{"epoch":1,"btime":"2024-09-10T15:57:14:967120+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T16:00:13.560776+0000","services":{}},"progress_events":{}} 2024-09-10T16:00:21.317 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-10T16:00:21.317 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:00:21.882 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:00:21.882 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:00:21.911 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":129,"stamp":"2024-09-10T16:00:21.563969+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":431665905,"num_objects":105398,"num_object_clones":0,"num_object_copies":210796,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":47117,"num_objects_misplaced":2894,"num_objects_unfound":0,"num_objects_dirty":105398,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":63244,"num_write_kb":252970,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1379,"num_bytes_recovered":5640211,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":105471,"ondisk_log_size":105471,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1353064,"kb_used_data":844344,"kb_used_omap":12,"kb_used_meta":508659,"kb_avail":281762456,"statfs":{"total":289910292480,"available":288524754944,"internally_reserved":0,"allocated":864608256,"data_stored":863719853,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":520866818},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,6,4,6,12,7,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":29134848,"num_objects":7113,"num_object_clones":0,"num_object_copies":14226,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-231,"num_objects_unfound":0,"num_objects_dirty":7113,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7113,"num_write_kb":28452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":231,"num_bytes_recovered":946176,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004326"},"pg_stats":[{"pgid":"2.f","version":"43'6552","reported_seq":6621,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:21.449589+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:00:21.449589+0000","last_peered":"2024-09-10T16:00:21.449589+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:00:21.449589+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":6552,"log_dups_size":0,"ondisk_log_size":6552,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26836992,"num_objects":6552,"num_object_clones":0,"num_object_copies":13104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6552,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6552,"num_write_kb":26208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2293},{"shards":"0,2","objects":4258}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'6530","reported_seq":6637,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:00:19.397543+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:00:19.397543+0000","last_peered":"2024-09-10T16:00:19.397543+0000","last_clean":"2024-09-10T16:00:19.397543+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:00:19.397543+0000","last_undegraded":"2024-09-10T16:00:19.397543+0000","last_fullsized":"2024-09-10T16:00:19.397543+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":6530,"log_dups_size":0,"ondisk_log_size":6530,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26723270,"num_objects":6526,"num_object_clones":0,"num_object_copies":13052,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6526,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4635,"num_write_kb":18540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'6496","reported_seq":9369,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T16:00:19.435342+0000","last_change":"2024-09-10T15:59:16.142612+0000","last_active":"2024-09-10T16:00:19.435342+0000","last_peered":"2024-09-10T16:00:19.435342+0000","last_clean":"2024-09-10T15:59:14.092473+0000","last_became_active":"2024-09-10T15:59:16.109611+0000","last_became_peered":"2024-09-10T15:59:16.109611+0000","last_unstale":"2024-09-10T16:00:19.435342+0000","last_undegraded":"2024-09-10T16:00:19.435342+0000","last_fullsized":"2024-09-10T15:59:15.770264+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":6496,"log_dups_size":0,"ondisk_log_size":6496,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26591232,"num_objects":6492,"num_object_clones":0,"num_object_copies":12984,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2894,"num_objects_unfound":0,"num_objects_dirty":6492,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4500,"num_write_kb":18000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1374,"num_bytes_recovered":5627904,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2894},{"shards":"2","objects":3598}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'6547","reported_seq":6622,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:00:19.422985+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:00:19.422985+0000","last_peered":"2024-09-10T16:00:19.422985+0000","last_clean":"2024-09-10T16:00:19.422985+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:00:19.422985+0000","last_undegraded":"2024-09-10T16:00:19.422985+0000","last_fullsized":"2024-09-10T16:00:19.422985+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":6547,"log_dups_size":0,"ondisk_log_size":6547,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26816512,"num_objects":6547,"num_object_clones":0,"num_object_copies":13094,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6547,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6547,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'6632","reported_seq":6757,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:00:21.401959+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:00:21.401959+0000","last_peered":"2024-09-10T16:00:21.401959+0000","last_clean":"2024-09-10T16:00:21.401959+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:00:21.401959+0000","last_undegraded":"2024-09-10T16:00:21.401959+0000","last_fullsized":"2024-09-10T16:00:21.401959+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":6632,"log_dups_size":0,"ondisk_log_size":6632,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27136483,"num_objects":6626,"num_object_clones":0,"num_object_copies":13252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6626,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3343,"num_write_kb":13372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'6696","reported_seq":6827,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:21.432251+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:00:21.432251+0000","last_peered":"2024-09-10T16:00:21.432251+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:00:21.432251+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":6696,"log_dups_size":0,"ondisk_log_size":6696,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27406819,"num_objects":6692,"num_object_clones":0,"num_object_copies":13384,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6692,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3308,"num_write_kb":13232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2321},{"shards":"0,2","objects":4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'6628","reported_seq":6758,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:21.432480+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:00:21.432480+0000","last_peered":"2024-09-10T16:00:21.432480+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:00:21.432480+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":6628,"log_dups_size":0,"ondisk_log_size":6628,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27124678,"num_objects":6624,"num_object_clones":0,"num_object_copies":13248,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6624,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3127,"num_write_kb":12508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2387},{"shards":"0,2","objects":4237}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'6745","reported_seq":6870,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:00:21.345566+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:00:21.345566+0000","last_peered":"2024-09-10T16:00:21.345566+0000","last_clean":"2024-09-10T16:00:21.345566+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:00:21.345566+0000","last_undegraded":"2024-09-10T16:00:21.345566+0000","last_fullsized":"2024-09-10T16:00:21.345566+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":6745,"log_dups_size":0,"ondisk_log_size":6745,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27619328,"num_objects":6743,"num_object_clones":0,"num_object_copies":13486,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6743,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3592,"num_write_kb":14368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.775865+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T15:59:15.775865+0000","last_peered":"2024-09-10T15:59:15.775865+0000","last_clean":"2024-09-10T15:59:15.775865+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T15:59:15.775865+0000","last_undegraded":"2024-09-10T15:59:15.775865+0000","last_fullsized":"2024-09-10T15:59:15.775865+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'6325","reported_seq":6459,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:19.386748+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:00:19.386748+0000","last_peered":"2024-09-10T16:00:19.386748+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:00:19.386748+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":6325,"log_dups_size":0,"ondisk_log_size":6325,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25879011,"num_objects":6319,"num_object_clones":0,"num_object_copies":12638,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6319,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4144,"num_write_kb":16576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4123},{"shards":"2","objects":2196}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":88,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.772572+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T15:59:15.772572+0000","last_peered":"2024-09-10T15:59:15.772572+0000","last_clean":"2024-09-10T15:59:15.772572+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T15:59:15.772572+0000","last_undegraded":"2024-09-10T15:59:15.772572+0000","last_fullsized":"2024-09-10T15:59:15.772572+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'6538","reported_seq":6663,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:21.385302+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:00:21.385302+0000","last_peered":"2024-09-10T16:00:21.385302+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:00:21.385302+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":6538,"log_dups_size":0,"ondisk_log_size":6538,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26746880,"num_objects":6530,"num_object_clones":0,"num_object_copies":13060,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6530,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3042,"num_write_kb":12168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2275},{"shards":"0,2","objects":4255}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":99,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.884297+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T15:59:15.884297+0000","last_peered":"2024-09-10T15:59:15.884297+0000","last_clean":"2024-09-10T15:59:15.884297+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T15:59:15.884297+0000","last_undegraded":"2024-09-10T15:59:15.884297+0000","last_fullsized":"2024-09-10T15:59:15.884297+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'6561","reported_seq":6688,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:19.430892+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:00:19.430892+0000","last_peered":"2024-09-10T16:00:19.430892+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:00:19.430892+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":6561,"log_dups_size":0,"ondisk_log_size":6561,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26853859,"num_objects":6557,"num_object_clones":0,"num_object_copies":13114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6557,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4241,"num_write_kb":16964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4304},{"shards":"2","objects":2253}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.837509+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T15:59:15.837509+0000","last_peered":"2024-09-10T15:59:15.837509+0000","last_clean":"2024-09-10T15:59:15.837509+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T15:59:15.837509+0000","last_undegraded":"2024-09-10T15:59:15.837509+0000","last_fullsized":"2024-09-10T15:59:15.837509+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'6673","reported_seq":6797,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:19.436623+0000","last_change":"2024-09-10T15:59:16.174113+0000","last_active":"2024-09-10T16:00:19.436623+0000","last_peered":"2024-09-10T16:00:19.436623+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:00:19.436623+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":6673,"log_dups_size":0,"ondisk_log_size":6673,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27308032,"num_objects":6667,"num_object_clones":0,"num_object_copies":13334,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4354,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6667,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4227,"num_write_kb":16908,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4352},{"shards":"2","objects":2315}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'6597","reported_seq":6725,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:21.420095+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:00:21.420095+0000","last_peered":"2024-09-10T16:00:21.420095+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:00:21.420095+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":6597,"log_dups_size":0,"ondisk_log_size":6597,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27013120,"num_objects":6595,"num_object_clones":0,"num_object_copies":13190,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6595,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3228,"num_write_kb":12912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2291},{"shards":"0,2","objects":4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.989413+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T15:59:15.989413+0000","last_peered":"2024-09-10T15:59:15.989413+0000","last_clean":"2024-09-10T15:59:15.989413+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T15:59:15.989413+0000","last_undegraded":"2024-09-10T15:59:15.989413+0000","last_fullsized":"2024-09-10T15:59:15.989413+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'6478","reported_seq":6614,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:21.376610+0000","last_change":"2024-09-10T15:59:16.251610+0000","last_active":"2024-09-10T16:00:21.376610+0000","last_peered":"2024-09-10T16:00:21.376610+0000","last_clean":"2024-09-10T15:59:14.115595+0000","last_became_active":"2024-09-10T15:59:16.244089+0000","last_became_peered":"2024-09-10T15:59:16.244089+0000","last_unstale":"2024-09-10T16:00:21.376610+0000","last_undegraded":"2024-09-10T15:59:15.847127+0000","last_fullsized":"2024-09-10T15:59:15.774437+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":6478,"log_dups_size":0,"ondisk_log_size":6478,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26501120,"num_objects":6470,"num_object_clones":0,"num_object_copies":12940,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6470,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2868,"num_write_kb":11472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2267},{"shards":"0,2","objects":4203}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":88,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.773583+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T15:59:15.773583+0000","last_peered":"2024-09-10T15:59:15.773583+0000","last_clean":"2024-09-10T15:59:15.773583+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T15:59:15.773583+0000","last_undegraded":"2024-09-10T15:59:15.773583+0000","last_fullsized":"2024-09-10T15:59:15.773583+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'6719","reported_seq":6854,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:21.423588+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:00:21.423588+0000","last_peered":"2024-09-10T16:00:21.423588+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:00:21.423588+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":6719,"log_dups_size":0,"ondisk_log_size":6719,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27504640,"num_objects":6715,"num_object_clones":0,"num_object_copies":13430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2974,"num_write_kb":11896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2344},{"shards":"0,2","objects":4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.916017+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T15:59:15.916017+0000","last_peered":"2024-09-10T15:59:15.916017+0000","last_clean":"2024-09-10T15:59:15.916017+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T15:59:15.916017+0000","last_undegraded":"2024-09-10T15:59:15.916017+0000","last_fullsized":"2024-09-10T15:59:15.916017+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.776033+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T15:59:15.776033+0000","last_peered":"2024-09-10T15:59:15.776033+0000","last_clean":"2024-09-10T15:59:15.776033+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T15:59:15.776033+0000","last_undegraded":"2024-09-10T15:59:15.776033+0000","last_fullsized":"2024-09-10T15:59:15.776033+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'6751","reported_seq":6890,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:21.411447+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:00:21.411447+0000","last_peered":"2024-09-10T16:00:21.411447+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:00:21.411447+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":6751,"log_dups_size":0,"ondisk_log_size":6751,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27603910,"num_objects":6741,"num_object_clones":0,"num_object_copies":13482,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6741,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2914,"num_write_kb":11656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2418},{"shards":"0,2","objects":4323}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":431665886,"num_objects":105396,"num_object_clones":0,"num_object_copies":210792,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":47117,"num_objects_misplaced":2894,"num_objects_unfound":0,"num_objects_dirty":105396,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":63242,"num_write_kb":252968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1377,"num_bytes_recovered":5640192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":865734656,"data_stored":865579297,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":105468,"ondisk_log_size":105468,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986204,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":526308,"kb_used_data":346068,"kb_used_omap":4,"kb_used_meta":180219,"kb_avail":93845532,"statfs":{"total":96636764160,"available":96097824768,"internally_reserved":0,"allocated":354373632,"data_stored":354095872,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":184544938},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,1,0,4,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659150,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":216216,"kb_used_data":112328,"kb_used_omap":4,"kb_used_meta":103867,"kb_avail":94155624,"statfs":{"total":96636764160,"available":96415358976,"internally_reserved":0,"allocated":115023872,"data_stored":114695066,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":106360494},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,2,3,3,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182433,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":610540,"kb_used_data":385948,"kb_used_omap":4,"kb_used_meta":224571,"kb_avail":93761300,"statfs":{"total":96636764160,"available":96011571200,"internally_reserved":0,"allocated":395210752,"data_stored":394928915,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":229961386},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,1,3,5,4,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":395448320,"data_stored":395412190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":115306496,"data_stored":115223397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":354979840,"data_stored":354943710,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:00:21.980 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 109 20 106033 106013 3.79853 1.92969 0.0550799 0.0205591 2024-09-10T16:00:22.980 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 110 20 106590 106570 3.78378 2.17578 0.031581 0.0206394 2024-09-10T16:00:23.980 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 111 20 107154 107134 3.76954 2.20312 0.0579075 0.0207183 2024-09-10T16:00:24.913 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:00:24.980 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 112 20 107724 107704 3.75576 2.22656 0.0342452 0.0207929 2024-09-10T16:00:25.458 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:00:25.460 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:00:25.483 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":130,"stamp":"2024-09-10T16:00:23.564593+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":431665905,"num_objects":105398,"num_object_clones":0,"num_object_copies":210796,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":47117,"num_objects_misplaced":2894,"num_objects_unfound":0,"num_objects_dirty":105398,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":63244,"num_write_kb":252970,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1379,"num_bytes_recovered":5640211,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":105471,"ondisk_log_size":105471,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1353064,"kb_used_data":844344,"kb_used_omap":12,"kb_used_meta":508659,"kb_avail":281762456,"statfs":{"total":289910292480,"available":288524754944,"internally_reserved":0,"allocated":864608256,"data_stored":863719853,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":520866818},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,6,4,6,12,7,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":22130688,"num_objects":5403,"num_object_clones":0,"num_object_copies":10806,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-231,"num_objects_unfound":0,"num_objects_dirty":5403,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5403,"num_write_kb":21612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":231,"num_bytes_recovered":946176,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004272"},"pg_stats":[{"pgid":"2.f","version":"43'6552","reported_seq":6621,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:21.449589+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:00:21.449589+0000","last_peered":"2024-09-10T16:00:21.449589+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:00:21.449589+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":6552,"log_dups_size":0,"ondisk_log_size":6552,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26836992,"num_objects":6552,"num_object_clones":0,"num_object_copies":13104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6552,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6552,"num_write_kb":26208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2293},{"shards":"0,2","objects":4258}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'6530","reported_seq":6637,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:00:19.397543+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:00:19.397543+0000","last_peered":"2024-09-10T16:00:19.397543+0000","last_clean":"2024-09-10T16:00:19.397543+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:00:19.397543+0000","last_undegraded":"2024-09-10T16:00:19.397543+0000","last_fullsized":"2024-09-10T16:00:19.397543+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":6530,"log_dups_size":0,"ondisk_log_size":6530,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26723270,"num_objects":6526,"num_object_clones":0,"num_object_copies":13052,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6526,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4635,"num_write_kb":18540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'6496","reported_seq":9369,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T16:00:19.435342+0000","last_change":"2024-09-10T15:59:16.142612+0000","last_active":"2024-09-10T16:00:19.435342+0000","last_peered":"2024-09-10T16:00:19.435342+0000","last_clean":"2024-09-10T15:59:14.092473+0000","last_became_active":"2024-09-10T15:59:16.109611+0000","last_became_peered":"2024-09-10T15:59:16.109611+0000","last_unstale":"2024-09-10T16:00:19.435342+0000","last_undegraded":"2024-09-10T16:00:19.435342+0000","last_fullsized":"2024-09-10T15:59:15.770264+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":6496,"log_dups_size":0,"ondisk_log_size":6496,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26591232,"num_objects":6492,"num_object_clones":0,"num_object_copies":12984,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2894,"num_objects_unfound":0,"num_objects_dirty":6492,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4500,"num_write_kb":18000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1374,"num_bytes_recovered":5627904,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2894},{"shards":"2","objects":3598}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'6547","reported_seq":6622,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:00:19.422985+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:00:19.422985+0000","last_peered":"2024-09-10T16:00:19.422985+0000","last_clean":"2024-09-10T16:00:19.422985+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:00:19.422985+0000","last_undegraded":"2024-09-10T16:00:19.422985+0000","last_fullsized":"2024-09-10T16:00:19.422985+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":6547,"log_dups_size":0,"ondisk_log_size":6547,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26816512,"num_objects":6547,"num_object_clones":0,"num_object_copies":13094,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6547,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6547,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'6632","reported_seq":6757,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:00:21.401959+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:00:21.401959+0000","last_peered":"2024-09-10T16:00:21.401959+0000","last_clean":"2024-09-10T16:00:21.401959+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:00:21.401959+0000","last_undegraded":"2024-09-10T16:00:21.401959+0000","last_fullsized":"2024-09-10T16:00:21.401959+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":6632,"log_dups_size":0,"ondisk_log_size":6632,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27136483,"num_objects":6626,"num_object_clones":0,"num_object_copies":13252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6626,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3343,"num_write_kb":13372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'6696","reported_seq":6827,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:21.432251+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:00:21.432251+0000","last_peered":"2024-09-10T16:00:21.432251+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:00:21.432251+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":6696,"log_dups_size":0,"ondisk_log_size":6696,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27406819,"num_objects":6692,"num_object_clones":0,"num_object_copies":13384,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6692,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3308,"num_write_kb":13232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2321},{"shards":"0,2","objects":4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'6628","reported_seq":6758,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:21.432480+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:00:21.432480+0000","last_peered":"2024-09-10T16:00:21.432480+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:00:21.432480+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":6628,"log_dups_size":0,"ondisk_log_size":6628,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27124678,"num_objects":6624,"num_object_clones":0,"num_object_copies":13248,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6624,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3127,"num_write_kb":12508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2387},{"shards":"0,2","objects":4237}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'6745","reported_seq":6870,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:00:21.345566+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:00:21.345566+0000","last_peered":"2024-09-10T16:00:21.345566+0000","last_clean":"2024-09-10T16:00:21.345566+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:00:21.345566+0000","last_undegraded":"2024-09-10T16:00:21.345566+0000","last_fullsized":"2024-09-10T16:00:21.345566+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":6745,"log_dups_size":0,"ondisk_log_size":6745,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27619328,"num_objects":6743,"num_object_clones":0,"num_object_copies":13486,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6743,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3592,"num_write_kb":14368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.775865+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T15:59:15.775865+0000","last_peered":"2024-09-10T15:59:15.775865+0000","last_clean":"2024-09-10T15:59:15.775865+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T15:59:15.775865+0000","last_undegraded":"2024-09-10T15:59:15.775865+0000","last_fullsized":"2024-09-10T15:59:15.775865+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'6325","reported_seq":6459,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:19.386748+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:00:19.386748+0000","last_peered":"2024-09-10T16:00:19.386748+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:00:19.386748+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":6325,"log_dups_size":0,"ondisk_log_size":6325,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25879011,"num_objects":6319,"num_object_clones":0,"num_object_copies":12638,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6319,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4144,"num_write_kb":16576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4123},{"shards":"2","objects":2196}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":88,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.772572+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T15:59:15.772572+0000","last_peered":"2024-09-10T15:59:15.772572+0000","last_clean":"2024-09-10T15:59:15.772572+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T15:59:15.772572+0000","last_undegraded":"2024-09-10T15:59:15.772572+0000","last_fullsized":"2024-09-10T15:59:15.772572+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'6538","reported_seq":6663,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:21.385302+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:00:21.385302+0000","last_peered":"2024-09-10T16:00:21.385302+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:00:21.385302+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":6538,"log_dups_size":0,"ondisk_log_size":6538,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26746880,"num_objects":6530,"num_object_clones":0,"num_object_copies":13060,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6530,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3042,"num_write_kb":12168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2275},{"shards":"0,2","objects":4255}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":99,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.884297+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T15:59:15.884297+0000","last_peered":"2024-09-10T15:59:15.884297+0000","last_clean":"2024-09-10T15:59:15.884297+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T15:59:15.884297+0000","last_undegraded":"2024-09-10T15:59:15.884297+0000","last_fullsized":"2024-09-10T15:59:15.884297+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'6561","reported_seq":6688,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:19.430892+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:00:19.430892+0000","last_peered":"2024-09-10T16:00:19.430892+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:00:19.430892+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":6561,"log_dups_size":0,"ondisk_log_size":6561,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26853859,"num_objects":6557,"num_object_clones":0,"num_object_copies":13114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6557,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4241,"num_write_kb":16964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4304},{"shards":"2","objects":2253}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.837509+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T15:59:15.837509+0000","last_peered":"2024-09-10T15:59:15.837509+0000","last_clean":"2024-09-10T15:59:15.837509+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T15:59:15.837509+0000","last_undegraded":"2024-09-10T15:59:15.837509+0000","last_fullsized":"2024-09-10T15:59:15.837509+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'6673","reported_seq":6797,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:19.436623+0000","last_change":"2024-09-10T15:59:16.174113+0000","last_active":"2024-09-10T16:00:19.436623+0000","last_peered":"2024-09-10T16:00:19.436623+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:00:19.436623+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":6673,"log_dups_size":0,"ondisk_log_size":6673,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27308032,"num_objects":6667,"num_object_clones":0,"num_object_copies":13334,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4354,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6667,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4227,"num_write_kb":16908,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4352},{"shards":"2","objects":2315}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'6597","reported_seq":6725,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:21.420095+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:00:21.420095+0000","last_peered":"2024-09-10T16:00:21.420095+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:00:21.420095+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":6597,"log_dups_size":0,"ondisk_log_size":6597,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27013120,"num_objects":6595,"num_object_clones":0,"num_object_copies":13190,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6595,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3228,"num_write_kb":12912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2291},{"shards":"0,2","objects":4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.989413+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T15:59:15.989413+0000","last_peered":"2024-09-10T15:59:15.989413+0000","last_clean":"2024-09-10T15:59:15.989413+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T15:59:15.989413+0000","last_undegraded":"2024-09-10T15:59:15.989413+0000","last_fullsized":"2024-09-10T15:59:15.989413+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'6478","reported_seq":6614,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:21.376610+0000","last_change":"2024-09-10T15:59:16.251610+0000","last_active":"2024-09-10T16:00:21.376610+0000","last_peered":"2024-09-10T16:00:21.376610+0000","last_clean":"2024-09-10T15:59:14.115595+0000","last_became_active":"2024-09-10T15:59:16.244089+0000","last_became_peered":"2024-09-10T15:59:16.244089+0000","last_unstale":"2024-09-10T16:00:21.376610+0000","last_undegraded":"2024-09-10T15:59:15.847127+0000","last_fullsized":"2024-09-10T15:59:15.774437+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":6478,"log_dups_size":0,"ondisk_log_size":6478,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26501120,"num_objects":6470,"num_object_clones":0,"num_object_copies":12940,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6470,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2868,"num_write_kb":11472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2267},{"shards":"0,2","objects":4203}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":88,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.773583+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T15:59:15.773583+0000","last_peered":"2024-09-10T15:59:15.773583+0000","last_clean":"2024-09-10T15:59:15.773583+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T15:59:15.773583+0000","last_undegraded":"2024-09-10T15:59:15.773583+0000","last_fullsized":"2024-09-10T15:59:15.773583+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'6719","reported_seq":6854,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:21.423588+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:00:21.423588+0000","last_peered":"2024-09-10T16:00:21.423588+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:00:21.423588+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":6719,"log_dups_size":0,"ondisk_log_size":6719,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27504640,"num_objects":6715,"num_object_clones":0,"num_object_copies":13430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2974,"num_write_kb":11896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2344},{"shards":"0,2","objects":4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.916017+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T15:59:15.916017+0000","last_peered":"2024-09-10T15:59:15.916017+0000","last_clean":"2024-09-10T15:59:15.916017+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T15:59:15.916017+0000","last_undegraded":"2024-09-10T15:59:15.916017+0000","last_fullsized":"2024-09-10T15:59:15.916017+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.776033+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T15:59:15.776033+0000","last_peered":"2024-09-10T15:59:15.776033+0000","last_clean":"2024-09-10T15:59:15.776033+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T15:59:15.776033+0000","last_undegraded":"2024-09-10T15:59:15.776033+0000","last_fullsized":"2024-09-10T15:59:15.776033+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'6751","reported_seq":6890,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:21.411447+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:00:21.411447+0000","last_peered":"2024-09-10T16:00:21.411447+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:00:21.411447+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":6751,"log_dups_size":0,"ondisk_log_size":6751,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27603910,"num_objects":6741,"num_object_clones":0,"num_object_copies":13482,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6741,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2914,"num_write_kb":11656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2418},{"shards":"0,2","objects":4323}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":431665886,"num_objects":105396,"num_object_clones":0,"num_object_copies":210792,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":47117,"num_objects_misplaced":2894,"num_objects_unfound":0,"num_objects_dirty":105396,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":63242,"num_write_kb":252968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1377,"num_bytes_recovered":5640192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":865734656,"data_stored":865579297,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":105468,"ondisk_log_size":105468,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986204,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":526308,"kb_used_data":346068,"kb_used_omap":4,"kb_used_meta":180219,"kb_avail":93845532,"statfs":{"total":96636764160,"available":96097824768,"internally_reserved":0,"allocated":354373632,"data_stored":354095872,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":184544938},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,1,0,4,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659150,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":216216,"kb_used_data":112328,"kb_used_omap":4,"kb_used_meta":103867,"kb_avail":94155624,"statfs":{"total":96636764160,"available":96415358976,"internally_reserved":0,"allocated":115023872,"data_stored":114695066,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":106360494},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,2,3,3,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182433,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":610540,"kb_used_data":385948,"kb_used_omap":4,"kb_used_meta":224571,"kb_avail":93761300,"statfs":{"total":96636764160,"available":96011571200,"internally_reserved":0,"allocated":395210752,"data_stored":394928915,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":229961386},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,1,3,5,4,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":395448320,"data_stored":395412190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":115306496,"data_stored":115223397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":354979840,"data_stored":354943710,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:00:25.485 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:00:25.980 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 113 20 108216 108196 3.73953 1.92188 0.0523963 0.0208832 2024-09-10T16:00:26.044 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:00:26.044 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:00:26.065 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":131,"stamp":"2024-09-10T16:00:25.565209+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":435716849,"num_objects":106387,"num_object_clones":0,"num_object_copies":212774,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":47117,"num_objects_misplaced":2780,"num_objects_unfound":0,"num_objects_dirty":106387,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":64233,"num_write_kb":256926,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1493,"num_bytes_recovered":6107155,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":106460,"ondisk_log_size":106460,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1379036,"kb_used_data":857836,"kb_used_omap":12,"kb_used_meta":521139,"kb_avail":281736484,"statfs":{"total":289910292480,"available":288498159616,"internally_reserved":0,"allocated":878424064,"data_stored":877535661,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":533646338},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,4,4,6,13,7,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":26181632,"num_objects":6392,"num_object_clones":0,"num_object_copies":12784,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-345,"num_objects_unfound":0,"num_objects_dirty":6392,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6392,"num_write_kb":25568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":345,"num_bytes_recovered":1413120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004463"},"pg_stats":[{"pgid":"2.f","version":"43'6552","reported_seq":6621,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:21.449589+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:00:21.449589+0000","last_peered":"2024-09-10T16:00:21.449589+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:00:21.449589+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":6552,"log_dups_size":0,"ondisk_log_size":6552,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26836992,"num_objects":6552,"num_object_clones":0,"num_object_copies":13104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6552,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6552,"num_write_kb":26208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2293},{"shards":"0,2","objects":4258}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'6701","reported_seq":6808,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:00:24.441876+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:00:24.441876+0000","last_peered":"2024-09-10T16:00:24.441876+0000","last_clean":"2024-09-10T16:00:24.441876+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:00:24.441876+0000","last_undegraded":"2024-09-10T16:00:24.441876+0000","last_fullsized":"2024-09-10T16:00:24.441876+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":6701,"log_dups_size":0,"ondisk_log_size":6701,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27423686,"num_objects":6697,"num_object_clones":0,"num_object_copies":13394,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6697,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4806,"num_write_kb":19224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'6662","reported_seq":9762,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T16:00:24.438969+0000","last_change":"2024-09-10T15:59:16.142612+0000","last_active":"2024-09-10T16:00:24.438969+0000","last_peered":"2024-09-10T16:00:24.438969+0000","last_clean":"2024-09-10T15:59:14.092473+0000","last_became_active":"2024-09-10T15:59:16.109611+0000","last_became_peered":"2024-09-10T15:59:16.109611+0000","last_unstale":"2024-09-10T16:00:24.438969+0000","last_undegraded":"2024-09-10T16:00:24.438969+0000","last_fullsized":"2024-09-10T15:59:15.770264+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":6662,"log_dups_size":0,"ondisk_log_size":6662,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27271168,"num_objects":6658,"num_object_clones":0,"num_object_copies":13316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2780,"num_objects_unfound":0,"num_objects_dirty":6658,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4666,"num_write_kb":18664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1488,"num_bytes_recovered":6094848,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2780},{"shards":"2","objects":3878}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'6716","reported_seq":6789,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:00:24.451489+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:00:24.451489+0000","last_peered":"2024-09-10T16:00:24.451489+0000","last_clean":"2024-09-10T16:00:24.451489+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:00:24.451489+0000","last_undegraded":"2024-09-10T16:00:24.451489+0000","last_fullsized":"2024-09-10T16:00:24.451489+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":6716,"log_dups_size":0,"ondisk_log_size":6716,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27508736,"num_objects":6716,"num_object_clones":0,"num_object_copies":13432,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6716,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6716,"num_write_kb":26864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'6632","reported_seq":6757,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:00:21.401959+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:00:21.401959+0000","last_peered":"2024-09-10T16:00:21.401959+0000","last_clean":"2024-09-10T16:00:21.401959+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:00:21.401959+0000","last_undegraded":"2024-09-10T16:00:21.401959+0000","last_fullsized":"2024-09-10T16:00:21.401959+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":6632,"log_dups_size":0,"ondisk_log_size":6632,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27136483,"num_objects":6626,"num_object_clones":0,"num_object_copies":13252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6626,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3343,"num_write_kb":13372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'6696","reported_seq":6827,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:21.432251+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:00:21.432251+0000","last_peered":"2024-09-10T16:00:21.432251+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:00:21.432251+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":6696,"log_dups_size":0,"ondisk_log_size":6696,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27406819,"num_objects":6692,"num_object_clones":0,"num_object_copies":13384,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6692,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3308,"num_write_kb":13232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2321},{"shards":"0,2","objects":4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'6628","reported_seq":6758,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:21.432480+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:00:21.432480+0000","last_peered":"2024-09-10T16:00:21.432480+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:00:21.432480+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":6628,"log_dups_size":0,"ondisk_log_size":6628,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27124678,"num_objects":6624,"num_object_clones":0,"num_object_copies":13248,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6624,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3127,"num_write_kb":12508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2387},{"shards":"0,2","objects":4237}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'6745","reported_seq":6870,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:00:21.345566+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:00:21.345566+0000","last_peered":"2024-09-10T16:00:21.345566+0000","last_clean":"2024-09-10T16:00:21.345566+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:00:21.345566+0000","last_undegraded":"2024-09-10T16:00:21.345566+0000","last_fullsized":"2024-09-10T16:00:21.345566+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":6745,"log_dups_size":0,"ondisk_log_size":6745,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27619328,"num_objects":6743,"num_object_clones":0,"num_object_copies":13486,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6743,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3592,"num_write_kb":14368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.775865+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T15:59:15.775865+0000","last_peered":"2024-09-10T15:59:15.775865+0000","last_clean":"2024-09-10T15:59:15.775865+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T15:59:15.775865+0000","last_undegraded":"2024-09-10T15:59:15.775865+0000","last_fullsized":"2024-09-10T15:59:15.775865+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'6470","reported_seq":6604,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:24.429116+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:00:24.429116+0000","last_peered":"2024-09-10T16:00:24.429116+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:00:24.429116+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":6470,"log_dups_size":0,"ondisk_log_size":6470,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26472931,"num_objects":6464,"num_object_clones":0,"num_object_copies":12928,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6464,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4289,"num_write_kb":17156,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4123},{"shards":"2","objects":2341}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":89,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.772619+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T15:59:15.772619+0000","last_peered":"2024-09-10T15:59:15.772619+0000","last_clean":"2024-09-10T15:59:15.772619+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T15:59:15.772619+0000","last_undegraded":"2024-09-10T15:59:15.772619+0000","last_fullsized":"2024-09-10T15:59:15.772619+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'6538","reported_seq":6663,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:21.385302+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:00:21.385302+0000","last_peered":"2024-09-10T16:00:21.385302+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:00:21.385302+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":6538,"log_dups_size":0,"ondisk_log_size":6538,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26746880,"num_objects":6530,"num_object_clones":0,"num_object_copies":13060,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6530,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3042,"num_write_kb":12168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2275},{"shards":"0,2","objects":4255}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":99,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.884297+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T15:59:15.884297+0000","last_peered":"2024-09-10T15:59:15.884297+0000","last_clean":"2024-09-10T15:59:15.884297+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T15:59:15.884297+0000","last_undegraded":"2024-09-10T15:59:15.884297+0000","last_fullsized":"2024-09-10T15:59:15.884297+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'6739","reported_seq":6869,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:24.421508+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:00:24.421508+0000","last_peered":"2024-09-10T16:00:24.421508+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:00:24.421508+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":6739,"log_dups_size":0,"ondisk_log_size":6739,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27582947,"num_objects":6735,"num_object_clones":0,"num_object_copies":13470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6735,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4419,"num_write_kb":17676,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4304},{"shards":"2","objects":2431}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.837509+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T15:59:15.837509+0000","last_peered":"2024-09-10T15:59:15.837509+0000","last_clean":"2024-09-10T15:59:15.837509+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T15:59:15.837509+0000","last_undegraded":"2024-09-10T15:59:15.837509+0000","last_fullsized":"2024-09-10T15:59:15.837509+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'6833","reported_seq":6957,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:24.406094+0000","last_change":"2024-09-10T15:59:16.174113+0000","last_active":"2024-09-10T16:00:24.406094+0000","last_peered":"2024-09-10T16:00:24.406094+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:00:24.406094+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":6833,"log_dups_size":0,"ondisk_log_size":6833,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27963392,"num_objects":6827,"num_object_clones":0,"num_object_copies":13654,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4354,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6827,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4387,"num_write_kb":17548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4352},{"shards":"2","objects":2475}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'6597","reported_seq":6725,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:21.420095+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:00:21.420095+0000","last_peered":"2024-09-10T16:00:21.420095+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:00:21.420095+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":6597,"log_dups_size":0,"ondisk_log_size":6597,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27013120,"num_objects":6595,"num_object_clones":0,"num_object_copies":13190,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6595,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3228,"num_write_kb":12912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2291},{"shards":"0,2","objects":4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.989413+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T15:59:15.989413+0000","last_peered":"2024-09-10T15:59:15.989413+0000","last_clean":"2024-09-10T15:59:15.989413+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T15:59:15.989413+0000","last_undegraded":"2024-09-10T15:59:15.989413+0000","last_fullsized":"2024-09-10T15:59:15.989413+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'6478","reported_seq":6614,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:21.376610+0000","last_change":"2024-09-10T15:59:16.251610+0000","last_active":"2024-09-10T16:00:21.376610+0000","last_peered":"2024-09-10T16:00:21.376610+0000","last_clean":"2024-09-10T15:59:14.115595+0000","last_became_active":"2024-09-10T15:59:16.244089+0000","last_became_peered":"2024-09-10T15:59:16.244089+0000","last_unstale":"2024-09-10T16:00:21.376610+0000","last_undegraded":"2024-09-10T15:59:15.847127+0000","last_fullsized":"2024-09-10T15:59:15.774437+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":6478,"log_dups_size":0,"ondisk_log_size":6478,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26501120,"num_objects":6470,"num_object_clones":0,"num_object_copies":12940,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6470,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2868,"num_write_kb":11472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2267},{"shards":"0,2","objects":4203}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":89,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.773633+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T15:59:15.773633+0000","last_peered":"2024-09-10T15:59:15.773633+0000","last_clean":"2024-09-10T15:59:15.773633+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T15:59:15.773633+0000","last_undegraded":"2024-09-10T15:59:15.773633+0000","last_fullsized":"2024-09-10T15:59:15.773633+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'6719","reported_seq":6854,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:21.423588+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:00:21.423588+0000","last_peered":"2024-09-10T16:00:21.423588+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:00:21.423588+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":6719,"log_dups_size":0,"ondisk_log_size":6719,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27504640,"num_objects":6715,"num_object_clones":0,"num_object_copies":13430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2974,"num_write_kb":11896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2344},{"shards":"0,2","objects":4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.916017+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T15:59:15.916017+0000","last_peered":"2024-09-10T15:59:15.916017+0000","last_clean":"2024-09-10T15:59:15.916017+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T15:59:15.916017+0000","last_undegraded":"2024-09-10T15:59:15.916017+0000","last_fullsized":"2024-09-10T15:59:15.916017+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.776033+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T15:59:15.776033+0000","last_peered":"2024-09-10T15:59:15.776033+0000","last_clean":"2024-09-10T15:59:15.776033+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T15:59:15.776033+0000","last_undegraded":"2024-09-10T15:59:15.776033+0000","last_fullsized":"2024-09-10T15:59:15.776033+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'6751","reported_seq":6890,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:21.411447+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:00:21.411447+0000","last_peered":"2024-09-10T16:00:21.411447+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:00:21.411447+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":6751,"log_dups_size":0,"ondisk_log_size":6751,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27603910,"num_objects":6741,"num_object_clones":0,"num_object_copies":13482,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6741,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2914,"num_write_kb":11656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2418},{"shards":"0,2","objects":4323}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":435716830,"num_objects":106385,"num_object_clones":0,"num_object_copies":212770,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":47117,"num_objects_misplaced":2780,"num_objects_unfound":0,"num_objects_dirty":106385,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":64231,"num_write_kb":256924,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1491,"num_bytes_recovered":6107136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":879828992,"data_stored":879673633,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":106457,"ondisk_log_size":106457,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986205,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":535832,"kb_used_data":351432,"kb_used_omap":4,"kb_used_meta":184379,"kb_avail":93836008,"statfs":{"total":96636764160,"available":96088072192,"internally_reserved":0,"allocated":359866368,"data_stored":359588608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":188804778},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,2,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659151,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":232664,"kb_used_data":120456,"kb_used_omap":4,"kb_used_meta":112187,"kb_avail":94139176,"statfs":{"total":96636764160,"available":96398516224,"internally_reserved":0,"allocated":123346944,"data_stored":123018138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":114880174},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,1,5,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182433,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":610540,"kb_used_data":385948,"kb_used_omap":4,"kb_used_meta":224571,"kb_avail":93761300,"statfs":{"total":96636764160,"available":96011571200,"internally_reserved":0,"allocated":395210752,"data_stored":394928915,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":229961386},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,1,3,5,4,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":395448320,"data_stored":395412190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":123944960,"data_stored":123861861,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":360435712,"data_stored":360399582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:00:26.066 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-10T16:00:26.066 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "43'6552", 'reported_seq': 6621, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:00:21.449589+0000', 'last_change': '2024-09-10T15:59:16.281558+0000', 'last_active': '2024-09-10T16:00:21.449589+0000', 'last_peered': '2024-09-10T16:00:21.449589+0000', 'last_clean': '2024-09-10T15:59:14.132456+0000', 'last_became_active': '2024-09-10T15:59:16.274407+0000', 'last_became_peered': '2024-09-10T15:59:16.274407+0000', 'last_unstale': '2024-09-10T16:00:21.449589+0000', 'last_undegraded': '2024-09-10T15:59:15.851279+0000', 'last_fullsized': '2024-09-10T15:59:15.774356+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'objects_scrubbed': 0, 'log_size': 6552, 'log_dups_size': 0, 'ondisk_log_size': 6552, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26836992, 'num_objects': 6552, 'num_object_clones': 0, 'num_object_copies': 13104, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4259, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6552, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6552, 'num_write_kb': 26208, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2293}, {'shards': '0,2', 'objects': 4258}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:00:26.066 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-10T16:00:26.067 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "43'6662", 'reported_seq': 9762, 'reported_epoch': 43, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-10T16:00:24.438969+0000', 'last_change': '2024-09-10T15:59:16.142612+0000', 'last_active': '2024-09-10T16:00:24.438969+0000', 'last_peered': '2024-09-10T16:00:24.438969+0000', 'last_clean': '2024-09-10T15:59:14.092473+0000', 'last_became_active': '2024-09-10T15:59:16.109611+0000', 'last_became_peered': '2024-09-10T15:59:16.109611+0000', 'last_unstale': '2024-09-10T16:00:24.438969+0000', 'last_undegraded': '2024-09-10T16:00:24.438969+0000', 'last_fullsized': '2024-09-10T15:59:15.770264+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2029", 'last_scrub_stamp': '2024-09-10T15:58:45.933789+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:45.933789+0000', 'objects_scrubbed': 1992, 'log_size': 6662, 'log_dups_size': 0, 'ondisk_log_size': 6662, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 668, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27271168, 'num_objects': 6658, 'num_object_clones': 0, 'num_object_copies': 13316, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 2780, 'num_objects_unfound': 0, 'num_objects_dirty': 6658, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4666, 'num_write_kb': 18664, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1488, 'num_bytes_recovered': 6094848, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2780}, {'shards': '2', 'objects': 3878}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:00:26.067 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-10T16:00:26.067 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "43'6696", 'reported_seq': 6827, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:00:21.432251+0000', 'last_change': '2024-09-10T15:59:16.308264+0000', 'last_active': '2024-09-10T16:00:21.432251+0000', 'last_peered': '2024-09-10T16:00:21.432251+0000', 'last_clean': '2024-09-10T15:59:14.111551+0000', 'last_became_active': '2024-09-10T15:59:16.300040+0000', 'last_became_peered': '2024-09-10T15:59:16.300040+0000', 'last_unstale': '2024-09-10T16:00:21.432251+0000', 'last_undegraded': '2024-09-10T15:59:15.878234+0000', 'last_fullsized': '2024-09-10T15:59:15.769328+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3442", 'last_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'objects_scrubbed': 3384, 'log_size': 6696, 'log_dups_size': 0, 'ondisk_log_size': 6696, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1292, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27406819, 'num_objects': 6692, 'num_object_clones': 0, 'num_object_copies': 13384, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4372, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6692, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 3308, 'num_write_kb': 13232, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2321}, {'shards': '0,2', 'objects': 4371}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:00:26.067 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-10T16:00:26.067 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "43'6628", 'reported_seq': 6758, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:00:21.432480+0000', 'last_change': '2024-09-10T15:59:16.331899+0000', 'last_active': '2024-09-10T16:00:21.432480+0000', 'last_peered': '2024-09-10T16:00:21.432480+0000', 'last_clean': '2024-09-10T15:59:14.139436+0000', 'last_became_active': '2024-09-10T15:59:16.318715+0000', 'last_became_peered': '2024-09-10T15:59:16.318715+0000', 'last_unstale': '2024-09-10T16:00:21.432480+0000', 'last_undegraded': '2024-09-10T15:59:15.877524+0000', 'last_fullsized': '2024-09-10T15:59:15.769042+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3537", 'last_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'objects_scrubbed': 3497, 'log_size': 6628, 'log_dups_size': 0, 'ondisk_log_size': 6628, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1224, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27124678, 'num_objects': 6624, 'num_object_clones': 0, 'num_object_copies': 13248, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4237, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6624, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 3127, 'num_write_kb': 12508, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2387}, {'shards': '0,2', 'objects': 4237}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:00:26.067 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-10T16:00:26.067 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "43'6470", 'reported_seq': 6604, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:00:24.429116+0000', 'last_change': '2024-09-10T15:59:16.175680+0000', 'last_active': '2024-09-10T16:00:24.429116+0000', 'last_peered': '2024-09-10T16:00:24.429116+0000', 'last_clean': '2024-09-10T15:59:14.088187+0000', 'last_became_active': '2024-09-10T15:59:16.165810+0000', 'last_became_peered': '2024-09-10T15:59:16.165810+0000', 'last_unstale': '2024-09-10T16:00:24.429116+0000', 'last_undegraded': '2024-09-10T15:59:15.822098+0000', 'last_fullsized': '2024-09-10T15:59:15.769123+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2218", 'last_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'objects_scrubbed': 2175, 'log_size': 6470, 'log_dups_size': 0, 'ondisk_log_size': 6470, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 825, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26472931, 'num_objects': 6464, 'num_object_clones': 0, 'num_object_copies': 12928, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4124, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6464, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 4289, 'num_write_kb': 17156, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4123}, {'shards': '2', 'objects': 2341}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:00:26.067 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-10T16:00:26.067 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "43'6538", 'reported_seq': 6663, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:00:21.385302+0000', 'last_change': '2024-09-10T15:59:16.277742+0000', 'last_active': '2024-09-10T16:00:21.385302+0000', 'last_peered': '2024-09-10T16:00:21.385302+0000', 'last_clean': '2024-09-10T15:59:14.129081+0000', 'last_became_active': '2024-09-10T15:59:16.269835+0000', 'last_became_peered': '2024-09-10T15:59:16.269835+0000', 'last_unstale': '2024-09-10T16:00:21.385302+0000', 'last_undegraded': '2024-09-10T15:59:15.860857+0000', 'last_fullsized': '2024-09-10T15:59:15.771410+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3557", 'last_scrub_stamp': '2024-09-10T15:59:02.493678+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:02.493678+0000', 'objects_scrubbed': 3488, 'log_size': 6538, 'log_dups_size': 0, 'ondisk_log_size': 6538, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1472, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26746880, 'num_objects': 6530, 'num_object_clones': 0, 'num_object_copies': 13060, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4257, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6530, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3042, 'num_write_kb': 12168, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2275}, {'shards': '0,2', 'objects': 4255}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:00:26.067 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-10T16:00:26.067 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "43'6739", 'reported_seq': 6869, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:00:24.421508+0000', 'last_change': '2024-09-10T15:59:16.284311+0000', 'last_active': '2024-09-10T16:00:24.421508+0000', 'last_peered': '2024-09-10T16:00:24.421508+0000', 'last_clean': '2024-09-10T15:59:14.116041+0000', 'last_became_active': '2024-09-10T15:59:16.274543+0000', 'last_became_peered': '2024-09-10T15:59:16.274543+0000', 'last_unstale': '2024-09-10T16:00:24.421508+0000', 'last_undegraded': '2024-09-10T15:59:15.812484+0000', 'last_fullsized': '2024-09-10T15:59:15.770885+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2356", 'last_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'objects_scrubbed': 2316, 'log_size': 6739, 'log_dups_size': 0, 'ondisk_log_size': 6739, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 783, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27582947, 'num_objects': 6735, 'num_object_clones': 0, 'num_object_copies': 13470, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6735, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 4419, 'num_write_kb': 17676, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4304}, {'shards': '2', 'objects': 2431}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:00:26.068 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-10T16:00:26.068 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "43'6833", 'reported_seq': 6957, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:00:24.406094+0000', 'last_change': '2024-09-10T15:59:16.174113+0000', 'last_active': '2024-09-10T16:00:24.406094+0000', 'last_peered': '2024-09-10T16:00:24.406094+0000', 'last_clean': '2024-09-10T15:59:14.110654+0000', 'last_became_active': '2024-09-10T15:59:16.164645+0000', 'last_became_peered': '2024-09-10T15:59:16.164645+0000', 'last_unstale': '2024-09-10T16:00:24.406094+0000', 'last_undegraded': '2024-09-10T15:59:15.803593+0000', 'last_fullsized': '2024-09-10T15:59:15.769083+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2486", 'last_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'objects_scrubbed': 2440, 'log_size': 6833, 'log_dups_size': 0, 'ondisk_log_size': 6833, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 792, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27963392, 'num_objects': 6827, 'num_object_clones': 0, 'num_object_copies': 13654, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4354, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6827, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4387, 'num_write_kb': 17548, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4352}, {'shards': '2', 'objects': 2475}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:00:26.068 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-10T16:00:26.068 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "43'6597", 'reported_seq': 6725, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:00:21.420095+0000', 'last_change': '2024-09-10T15:59:16.315641+0000', 'last_active': '2024-09-10T16:00:21.420095+0000', 'last_peered': '2024-09-10T16:00:21.420095+0000', 'last_clean': '2024-09-10T15:59:14.117001+0000', 'last_became_active': '2024-09-10T15:59:16.302245+0000', 'last_became_peered': '2024-09-10T15:59:16.302245+0000', 'last_unstale': '2024-09-10T16:00:21.420095+0000', 'last_undegraded': '2024-09-10T15:59:15.845012+0000', 'last_fullsized': '2024-09-10T15:59:15.770337+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3404", 'last_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'objects_scrubbed': 3367, 'log_size': 6597, 'log_dups_size': 0, 'ondisk_log_size': 6597, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 983, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27013120, 'num_objects': 6595, 'num_object_clones': 0, 'num_object_copies': 13190, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6595, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3228, 'num_write_kb': 12912, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2291}, {'shards': '0,2', 'objects': 4304}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:00:26.068 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-10T16:00:26.068 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "43'6478", 'reported_seq': 6614, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:00:21.376610+0000', 'last_change': '2024-09-10T15:59:16.251610+0000', 'last_active': '2024-09-10T16:00:21.376610+0000', 'last_peered': '2024-09-10T16:00:21.376610+0000', 'last_clean': '2024-09-10T15:59:14.115595+0000', 'last_became_active': '2024-09-10T15:59:16.244089+0000', 'last_became_peered': '2024-09-10T15:59:16.244089+0000', 'last_unstale': '2024-09-10T16:00:21.376610+0000', 'last_undegraded': '2024-09-10T15:59:15.847127+0000', 'last_fullsized': '2024-09-10T15:59:15.774437+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3653", 'last_scrub_stamp': '2024-09-10T15:59:05.298468+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:05.298468+0000', 'objects_scrubbed': 3602, 'log_size': 6478, 'log_dups_size': 0, 'ondisk_log_size': 6478, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1289, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26501120, 'num_objects': 6470, 'num_object_clones': 0, 'num_object_copies': 12940, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4206, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6470, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2868, 'num_write_kb': 11472, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2267}, {'shards': '0,2', 'objects': 4203}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:00:26.068 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-10T16:00:26.068 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "43'6719", 'reported_seq': 6854, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:00:21.423588+0000', 'last_change': '2024-09-10T15:59:16.311291+0000', 'last_active': '2024-09-10T16:00:21.423588+0000', 'last_peered': '2024-09-10T16:00:21.423588+0000', 'last_clean': '2024-09-10T15:59:14.116363+0000', 'last_became_active': '2024-09-10T15:59:16.301244+0000', 'last_became_peered': '2024-09-10T15:59:16.301244+0000', 'last_unstale': '2024-09-10T16:00:21.423588+0000', 'last_undegraded': '2024-09-10T15:59:15.883434+0000', 'last_fullsized': '2024-09-10T15:59:15.772756+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3789", 'last_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'objects_scrubbed': 3741, 'log_size': 6719, 'log_dups_size': 0, 'ondisk_log_size': 6719, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1440, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27504640, 'num_objects': 6715, 'num_object_clones': 0, 'num_object_copies': 13430, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4373, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6715, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2974, 'num_write_kb': 11896, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2344}, {'shards': '0,2', 'objects': 4371}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:00:26.068 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-10T16:00:26.068 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "43'6751", 'reported_seq': 6890, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:00:21.411447+0000', 'last_change': '2024-09-10T15:59:16.313409+0000', 'last_active': '2024-09-10T16:00:21.411447+0000', 'last_peered': '2024-09-10T16:00:21.411447+0000', 'last_clean': '2024-09-10T15:59:14.129780+0000', 'last_became_active': '2024-09-10T15:59:16.300932+0000', 'last_became_peered': '2024-09-10T15:59:16.300932+0000', 'last_unstale': '2024-09-10T16:00:21.411447+0000', 'last_undegraded': '2024-09-10T15:59:15.887823+0000', 'last_fullsized': '2024-09-10T15:59:15.768513+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3880", 'last_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'objects_scrubbed': 3827, 'log_size': 6751, 'log_dups_size': 0, 'ondisk_log_size': 6751, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1397, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27603910, 'num_objects': 6741, 'num_object_clones': 0, 'num_object_copies': 13482, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4325, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6741, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2914, 'num_write_kb': 11656, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2418}, {'shards': '0,2', 'objects': 4323}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:00:26.069 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T16:00:26.775 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:00:26.775 INFO:teuthology.orchestra.run.smithi115.stdout:{"fsid":"7effee29-df05-4351-9741-ee203564ed81","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 47117/212774 objects degraded (22.144%), 11 pgs degraded, 12 pgs undersized","count":23},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":190,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":43,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983954,"num_in_osds":3,"osd_in_since":1725983947,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":106387,"data_bytes":435716849,"bytes_used":1412132864,"bytes_avail":288498159616,"bytes_total":289910292480,"degraded_objects":47117,"degraded_total":212774,"degraded_ratio":0.2214415295101845,"misplaced_objects":2780,"misplaced_total":212774,"misplaced_ratio":0.013065506123868517,"recovering_objects_per_sec":28,"recovering_bytes_per_sec":117716,"recovering_keys_per_sec":0,"num_objects_recovered":345,"num_bytes_recovered":1413120,"num_keys_recovered":0,"write_bytes_sec":2180991,"read_op_per_sec":0,"write_op_per_sec":532},"fsmap":{"epoch":1,"btime":"2024-09-10T15:57:14:967120+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T16:00:13.560776+0000","services":{}},"progress_events":{}} 2024-09-10T16:00:26.797 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-10T16:00:26.797 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:00:26.980 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 114 20 108703 108683 3.72341 1.90234 0.074935 0.0209753 2024-09-10T16:00:27.355 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:00:27.358 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:00:27.384 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":131,"stamp":"2024-09-10T16:00:25.565209+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":435716849,"num_objects":106387,"num_object_clones":0,"num_object_copies":212774,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":47117,"num_objects_misplaced":2780,"num_objects_unfound":0,"num_objects_dirty":106387,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":64233,"num_write_kb":256926,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1493,"num_bytes_recovered":6107155,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":106460,"ondisk_log_size":106460,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1379036,"kb_used_data":857836,"kb_used_omap":12,"kb_used_meta":521139,"kb_avail":281736484,"statfs":{"total":289910292480,"available":288498159616,"internally_reserved":0,"allocated":878424064,"data_stored":877535661,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":533646338},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,4,4,6,13,7,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":26181632,"num_objects":6392,"num_object_clones":0,"num_object_copies":12784,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-345,"num_objects_unfound":0,"num_objects_dirty":6392,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6392,"num_write_kb":25568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":345,"num_bytes_recovered":1413120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004463"},"pg_stats":[{"pgid":"2.f","version":"43'6552","reported_seq":6621,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:21.449589+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:00:21.449589+0000","last_peered":"2024-09-10T16:00:21.449589+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:00:21.449589+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":6552,"log_dups_size":0,"ondisk_log_size":6552,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26836992,"num_objects":6552,"num_object_clones":0,"num_object_copies":13104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6552,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6552,"num_write_kb":26208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2293},{"shards":"0,2","objects":4258}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'6701","reported_seq":6808,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:00:24.441876+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:00:24.441876+0000","last_peered":"2024-09-10T16:00:24.441876+0000","last_clean":"2024-09-10T16:00:24.441876+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:00:24.441876+0000","last_undegraded":"2024-09-10T16:00:24.441876+0000","last_fullsized":"2024-09-10T16:00:24.441876+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":6701,"log_dups_size":0,"ondisk_log_size":6701,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27423686,"num_objects":6697,"num_object_clones":0,"num_object_copies":13394,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6697,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4806,"num_write_kb":19224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'6662","reported_seq":9762,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T16:00:24.438969+0000","last_change":"2024-09-10T15:59:16.142612+0000","last_active":"2024-09-10T16:00:24.438969+0000","last_peered":"2024-09-10T16:00:24.438969+0000","last_clean":"2024-09-10T15:59:14.092473+0000","last_became_active":"2024-09-10T15:59:16.109611+0000","last_became_peered":"2024-09-10T15:59:16.109611+0000","last_unstale":"2024-09-10T16:00:24.438969+0000","last_undegraded":"2024-09-10T16:00:24.438969+0000","last_fullsized":"2024-09-10T15:59:15.770264+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":6662,"log_dups_size":0,"ondisk_log_size":6662,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27271168,"num_objects":6658,"num_object_clones":0,"num_object_copies":13316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2780,"num_objects_unfound":0,"num_objects_dirty":6658,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4666,"num_write_kb":18664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1488,"num_bytes_recovered":6094848,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2780},{"shards":"2","objects":3878}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'6716","reported_seq":6789,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:00:24.451489+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:00:24.451489+0000","last_peered":"2024-09-10T16:00:24.451489+0000","last_clean":"2024-09-10T16:00:24.451489+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:00:24.451489+0000","last_undegraded":"2024-09-10T16:00:24.451489+0000","last_fullsized":"2024-09-10T16:00:24.451489+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":6716,"log_dups_size":0,"ondisk_log_size":6716,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27508736,"num_objects":6716,"num_object_clones":0,"num_object_copies":13432,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6716,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6716,"num_write_kb":26864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'6632","reported_seq":6757,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:00:21.401959+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:00:21.401959+0000","last_peered":"2024-09-10T16:00:21.401959+0000","last_clean":"2024-09-10T16:00:21.401959+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:00:21.401959+0000","last_undegraded":"2024-09-10T16:00:21.401959+0000","last_fullsized":"2024-09-10T16:00:21.401959+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":6632,"log_dups_size":0,"ondisk_log_size":6632,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27136483,"num_objects":6626,"num_object_clones":0,"num_object_copies":13252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6626,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3343,"num_write_kb":13372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'6696","reported_seq":6827,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:21.432251+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:00:21.432251+0000","last_peered":"2024-09-10T16:00:21.432251+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:00:21.432251+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":6696,"log_dups_size":0,"ondisk_log_size":6696,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27406819,"num_objects":6692,"num_object_clones":0,"num_object_copies":13384,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6692,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3308,"num_write_kb":13232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2321},{"shards":"0,2","objects":4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'6628","reported_seq":6758,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:21.432480+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:00:21.432480+0000","last_peered":"2024-09-10T16:00:21.432480+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:00:21.432480+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":6628,"log_dups_size":0,"ondisk_log_size":6628,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27124678,"num_objects":6624,"num_object_clones":0,"num_object_copies":13248,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6624,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3127,"num_write_kb":12508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2387},{"shards":"0,2","objects":4237}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'6745","reported_seq":6870,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:00:21.345566+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:00:21.345566+0000","last_peered":"2024-09-10T16:00:21.345566+0000","last_clean":"2024-09-10T16:00:21.345566+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:00:21.345566+0000","last_undegraded":"2024-09-10T16:00:21.345566+0000","last_fullsized":"2024-09-10T16:00:21.345566+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":6745,"log_dups_size":0,"ondisk_log_size":6745,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27619328,"num_objects":6743,"num_object_clones":0,"num_object_copies":13486,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6743,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3592,"num_write_kb":14368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.775865+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T15:59:15.775865+0000","last_peered":"2024-09-10T15:59:15.775865+0000","last_clean":"2024-09-10T15:59:15.775865+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T15:59:15.775865+0000","last_undegraded":"2024-09-10T15:59:15.775865+0000","last_fullsized":"2024-09-10T15:59:15.775865+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'6470","reported_seq":6604,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:24.429116+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:00:24.429116+0000","last_peered":"2024-09-10T16:00:24.429116+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:00:24.429116+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":6470,"log_dups_size":0,"ondisk_log_size":6470,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26472931,"num_objects":6464,"num_object_clones":0,"num_object_copies":12928,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6464,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4289,"num_write_kb":17156,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4123},{"shards":"2","objects":2341}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":89,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.772619+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T15:59:15.772619+0000","last_peered":"2024-09-10T15:59:15.772619+0000","last_clean":"2024-09-10T15:59:15.772619+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T15:59:15.772619+0000","last_undegraded":"2024-09-10T15:59:15.772619+0000","last_fullsized":"2024-09-10T15:59:15.772619+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'6538","reported_seq":6663,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:21.385302+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:00:21.385302+0000","last_peered":"2024-09-10T16:00:21.385302+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:00:21.385302+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":6538,"log_dups_size":0,"ondisk_log_size":6538,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26746880,"num_objects":6530,"num_object_clones":0,"num_object_copies":13060,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6530,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3042,"num_write_kb":12168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2275},{"shards":"0,2","objects":4255}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":99,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.884297+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T15:59:15.884297+0000","last_peered":"2024-09-10T15:59:15.884297+0000","last_clean":"2024-09-10T15:59:15.884297+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T15:59:15.884297+0000","last_undegraded":"2024-09-10T15:59:15.884297+0000","last_fullsized":"2024-09-10T15:59:15.884297+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'6739","reported_seq":6869,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:24.421508+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:00:24.421508+0000","last_peered":"2024-09-10T16:00:24.421508+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:00:24.421508+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":6739,"log_dups_size":0,"ondisk_log_size":6739,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27582947,"num_objects":6735,"num_object_clones":0,"num_object_copies":13470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6735,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4419,"num_write_kb":17676,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4304},{"shards":"2","objects":2431}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.837509+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T15:59:15.837509+0000","last_peered":"2024-09-10T15:59:15.837509+0000","last_clean":"2024-09-10T15:59:15.837509+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T15:59:15.837509+0000","last_undegraded":"2024-09-10T15:59:15.837509+0000","last_fullsized":"2024-09-10T15:59:15.837509+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'6833","reported_seq":6957,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:24.406094+0000","last_change":"2024-09-10T15:59:16.174113+0000","last_active":"2024-09-10T16:00:24.406094+0000","last_peered":"2024-09-10T16:00:24.406094+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:00:24.406094+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":6833,"log_dups_size":0,"ondisk_log_size":6833,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27963392,"num_objects":6827,"num_object_clones":0,"num_object_copies":13654,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4354,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6827,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4387,"num_write_kb":17548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4352},{"shards":"2","objects":2475}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'6597","reported_seq":6725,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:21.420095+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:00:21.420095+0000","last_peered":"2024-09-10T16:00:21.420095+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:00:21.420095+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":6597,"log_dups_size":0,"ondisk_log_size":6597,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27013120,"num_objects":6595,"num_object_clones":0,"num_object_copies":13190,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6595,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3228,"num_write_kb":12912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2291},{"shards":"0,2","objects":4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.989413+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T15:59:15.989413+0000","last_peered":"2024-09-10T15:59:15.989413+0000","last_clean":"2024-09-10T15:59:15.989413+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T15:59:15.989413+0000","last_undegraded":"2024-09-10T15:59:15.989413+0000","last_fullsized":"2024-09-10T15:59:15.989413+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'6478","reported_seq":6614,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:21.376610+0000","last_change":"2024-09-10T15:59:16.251610+0000","last_active":"2024-09-10T16:00:21.376610+0000","last_peered":"2024-09-10T16:00:21.376610+0000","last_clean":"2024-09-10T15:59:14.115595+0000","last_became_active":"2024-09-10T15:59:16.244089+0000","last_became_peered":"2024-09-10T15:59:16.244089+0000","last_unstale":"2024-09-10T16:00:21.376610+0000","last_undegraded":"2024-09-10T15:59:15.847127+0000","last_fullsized":"2024-09-10T15:59:15.774437+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":6478,"log_dups_size":0,"ondisk_log_size":6478,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26501120,"num_objects":6470,"num_object_clones":0,"num_object_copies":12940,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6470,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2868,"num_write_kb":11472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2267},{"shards":"0,2","objects":4203}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":89,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.773633+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T15:59:15.773633+0000","last_peered":"2024-09-10T15:59:15.773633+0000","last_clean":"2024-09-10T15:59:15.773633+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T15:59:15.773633+0000","last_undegraded":"2024-09-10T15:59:15.773633+0000","last_fullsized":"2024-09-10T15:59:15.773633+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'6719","reported_seq":6854,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:21.423588+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:00:21.423588+0000","last_peered":"2024-09-10T16:00:21.423588+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:00:21.423588+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":6719,"log_dups_size":0,"ondisk_log_size":6719,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27504640,"num_objects":6715,"num_object_clones":0,"num_object_copies":13430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2974,"num_write_kb":11896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2344},{"shards":"0,2","objects":4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.916017+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T15:59:15.916017+0000","last_peered":"2024-09-10T15:59:15.916017+0000","last_clean":"2024-09-10T15:59:15.916017+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T15:59:15.916017+0000","last_undegraded":"2024-09-10T15:59:15.916017+0000","last_fullsized":"2024-09-10T15:59:15.916017+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.776033+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T15:59:15.776033+0000","last_peered":"2024-09-10T15:59:15.776033+0000","last_clean":"2024-09-10T15:59:15.776033+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T15:59:15.776033+0000","last_undegraded":"2024-09-10T15:59:15.776033+0000","last_fullsized":"2024-09-10T15:59:15.776033+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'6751","reported_seq":6890,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:21.411447+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:00:21.411447+0000","last_peered":"2024-09-10T16:00:21.411447+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:00:21.411447+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":6751,"log_dups_size":0,"ondisk_log_size":6751,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27603910,"num_objects":6741,"num_object_clones":0,"num_object_copies":13482,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6741,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2914,"num_write_kb":11656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2418},{"shards":"0,2","objects":4323}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":435716830,"num_objects":106385,"num_object_clones":0,"num_object_copies":212770,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":47117,"num_objects_misplaced":2780,"num_objects_unfound":0,"num_objects_dirty":106385,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":64231,"num_write_kb":256924,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1491,"num_bytes_recovered":6107136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":879828992,"data_stored":879673633,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":106457,"ondisk_log_size":106457,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986205,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":535832,"kb_used_data":351432,"kb_used_omap":4,"kb_used_meta":184379,"kb_avail":93836008,"statfs":{"total":96636764160,"available":96088072192,"internally_reserved":0,"allocated":359866368,"data_stored":359588608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":188804778},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,2,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659151,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":232664,"kb_used_data":120456,"kb_used_omap":4,"kb_used_meta":112187,"kb_avail":94139176,"statfs":{"total":96636764160,"available":96398516224,"internally_reserved":0,"allocated":123346944,"data_stored":123018138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":114880174},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,1,5,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182433,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":610540,"kb_used_data":385948,"kb_used_omap":4,"kb_used_meta":224571,"kb_avail":93761300,"statfs":{"total":96636764160,"available":96011571200,"internally_reserved":0,"allocated":395210752,"data_stored":394928915,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":229961386},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,1,3,5,4,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":395448320,"data_stored":395412190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":123944960,"data_stored":123861861,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":360435712,"data_stored":360399582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:00:27.980 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 115 20 109235 109215 3.70911 2.07812 0.0406053 0.0210568 2024-09-10T16:00:28.980 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 116 20 109794 109774 3.69596 2.18359 0.0373447 0.0211317 2024-09-10T16:00:29.980 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 117 20 110346 110326 3.68279 2.15625 0.0180843 0.0212062 2024-09-10T16:00:30.386 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:00:30.947 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:00:30.949 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:00:30.972 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":133,"stamp":"2024-09-10T16:00:29.566374+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":446644977,"num_objects":109055,"num_object_clones":0,"num_object_copies":218110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":47117,"num_objects_misplaced":2672,"num_objects_unfound":0,"num_objects_dirty":109055,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":66901,"num_write_kb":267598,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1601,"num_bytes_recovered":6549523,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":109128,"ondisk_log_size":109128,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1417452,"kb_used_data":879548,"kb_used_omap":12,"kb_used_meta":537843,"kb_avail":281698068,"statfs":{"total":289910292480,"available":288458821632,"internally_reserved":0,"allocated":900657152,"data_stored":899768749,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":550751234},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,6,8,7,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":26161152,"num_objects":6387,"num_object_clones":0,"num_object_copies":12774,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-337,"num_objects_unfound":0,"num_objects_dirty":6387,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6387,"num_write_kb":25548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":337,"num_bytes_recovered":1380352,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004532"},"pg_stats":[{"pgid":"2.f","version":"43'6709","reported_seq":6778,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:26.331994+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:00:26.331994+0000","last_peered":"2024-09-10T16:00:26.331994+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:00:26.331994+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":6709,"log_dups_size":0,"ondisk_log_size":6709,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27480064,"num_objects":6709,"num_object_clones":0,"num_object_copies":13418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6709,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6709,"num_write_kb":26836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2450},{"shards":"0,2","objects":4258}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'6875","reported_seq":6982,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:00:29.370631+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:00:29.370631+0000","last_peered":"2024-09-10T16:00:29.370631+0000","last_clean":"2024-09-10T16:00:29.370631+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:00:29.370631+0000","last_undegraded":"2024-09-10T16:00:29.370631+0000","last_fullsized":"2024-09-10T16:00:29.370631+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":6875,"log_dups_size":0,"ondisk_log_size":6875,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28136390,"num_objects":6871,"num_object_clones":0,"num_object_copies":13742,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6871,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4980,"num_write_kb":19920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'6848","reported_seq":10165,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T16:00:29.424339+0000","last_change":"2024-09-10T15:59:16.142612+0000","last_active":"2024-09-10T16:00:29.424339+0000","last_peered":"2024-09-10T16:00:29.424339+0000","last_clean":"2024-09-10T15:59:14.092473+0000","last_became_active":"2024-09-10T15:59:16.109611+0000","last_became_peered":"2024-09-10T15:59:16.109611+0000","last_unstale":"2024-09-10T16:00:29.424339+0000","last_undegraded":"2024-09-10T16:00:29.424339+0000","last_fullsized":"2024-09-10T15:59:15.770264+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":6848,"log_dups_size":0,"ondisk_log_size":6848,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28033024,"num_objects":6844,"num_object_clones":0,"num_object_copies":13688,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2672,"num_objects_unfound":0,"num_objects_dirty":6844,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4852,"num_write_kb":19408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1596,"num_bytes_recovered":6537216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2672},{"shards":"2","objects":4172}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'6875","reported_seq":6950,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:00:29.226323+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:00:29.226323+0000","last_peered":"2024-09-10T16:00:29.226323+0000","last_clean":"2024-09-10T16:00:29.226323+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:00:29.226323+0000","last_undegraded":"2024-09-10T16:00:29.226323+0000","last_fullsized":"2024-09-10T16:00:29.226323+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":6875,"log_dups_size":0,"ondisk_log_size":6875,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28160000,"num_objects":6875,"num_object_clones":0,"num_object_copies":13750,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6875,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6875,"num_write_kb":27500,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'6808","reported_seq":6933,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:00:26.446808+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:00:26.446808+0000","last_peered":"2024-09-10T16:00:26.446808+0000","last_clean":"2024-09-10T16:00:26.446808+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:00:26.446808+0000","last_undegraded":"2024-09-10T16:00:26.446808+0000","last_fullsized":"2024-09-10T16:00:26.446808+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":6808,"log_dups_size":0,"ondisk_log_size":6808,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27857379,"num_objects":6802,"num_object_clones":0,"num_object_copies":13604,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6802,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3519,"num_write_kb":14076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'6871","reported_seq":7001,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:26.426591+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:00:26.426591+0000","last_peered":"2024-09-10T16:00:26.426591+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:00:26.426591+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":6871,"log_dups_size":0,"ondisk_log_size":6871,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28123619,"num_objects":6867,"num_object_clones":0,"num_object_copies":13734,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6867,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3483,"num_write_kb":13932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2496},{"shards":"0,2","objects":4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'6809","reported_seq":6942,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:26.483196+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:00:26.483196+0000","last_peered":"2024-09-10T16:00:26.483196+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:00:26.483196+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":6809,"log_dups_size":0,"ondisk_log_size":6809,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27866054,"num_objects":6805,"num_object_clones":0,"num_object_copies":13610,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6805,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3308,"num_write_kb":13232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2568},{"shards":"0,2","objects":4237}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'6911","reported_seq":7036,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:00:26.458567+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:00:26.458567+0000","last_peered":"2024-09-10T16:00:26.458567+0000","last_clean":"2024-09-10T16:00:26.458567+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:00:26.458567+0000","last_undegraded":"2024-09-10T16:00:26.458567+0000","last_fullsized":"2024-09-10T16:00:26.458567+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":6911,"log_dups_size":0,"ondisk_log_size":6911,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28299264,"num_objects":6909,"num_object_clones":0,"num_object_copies":13818,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6909,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3758,"num_write_kb":15032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.775865+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T15:59:15.775865+0000","last_peered":"2024-09-10T15:59:15.775865+0000","last_clean":"2024-09-10T15:59:15.775865+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T15:59:15.775865+0000","last_undegraded":"2024-09-10T15:59:15.775865+0000","last_fullsized":"2024-09-10T15:59:15.775865+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'6620","reported_seq":6753,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:29.445789+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:00:29.445789+0000","last_peered":"2024-09-10T16:00:29.445789+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:00:29.445789+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":6620,"log_dups_size":0,"ondisk_log_size":6620,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27087331,"num_objects":6614,"num_object_clones":0,"num_object_copies":13228,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6614,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4439,"num_write_kb":17756,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4123},{"shards":"2","objects":2491}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":89,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.772619+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T15:59:15.772619+0000","last_peered":"2024-09-10T15:59:15.772619+0000","last_clean":"2024-09-10T15:59:15.772619+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T15:59:15.772619+0000","last_undegraded":"2024-09-10T15:59:15.772619+0000","last_fullsized":"2024-09-10T15:59:15.772619+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'6725","reported_seq":6849,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:26.480070+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:00:26.480070+0000","last_peered":"2024-09-10T16:00:26.480070+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:00:26.480070+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":6725,"log_dups_size":0,"ondisk_log_size":6725,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27512832,"num_objects":6717,"num_object_clones":0,"num_object_copies":13434,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6717,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3229,"num_write_kb":12916,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2462},{"shards":"0,2","objects":4255}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":99,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.884297+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T15:59:15.884297+0000","last_peered":"2024-09-10T15:59:15.884297+0000","last_clean":"2024-09-10T15:59:15.884297+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T15:59:15.884297+0000","last_undegraded":"2024-09-10T15:59:15.884297+0000","last_fullsized":"2024-09-10T15:59:15.884297+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'6894","reported_seq":7024,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:29.397113+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:00:29.397113+0000","last_peered":"2024-09-10T16:00:29.397113+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:00:29.397113+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":6894,"log_dups_size":0,"ondisk_log_size":6894,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28217827,"num_objects":6890,"num_object_clones":0,"num_object_copies":13780,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6890,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4574,"num_write_kb":18296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4304},{"shards":"2","objects":2586}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.837509+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T15:59:15.837509+0000","last_peered":"2024-09-10T15:59:15.837509+0000","last_clean":"2024-09-10T15:59:15.837509+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T15:59:15.837509+0000","last_undegraded":"2024-09-10T15:59:15.837509+0000","last_fullsized":"2024-09-10T15:59:15.837509+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'6989","reported_seq":7113,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:29.357500+0000","last_change":"2024-09-10T15:59:16.174113+0000","last_active":"2024-09-10T16:00:29.357500+0000","last_peered":"2024-09-10T16:00:29.357500+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:00:29.357500+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":6989,"log_dups_size":0,"ondisk_log_size":6989,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28602368,"num_objects":6983,"num_object_clones":0,"num_object_copies":13966,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4354,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6983,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4543,"num_write_kb":18172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4352},{"shards":"2","objects":2631}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'6770","reported_seq":6900,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:26.513434+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:00:26.513434+0000","last_peered":"2024-09-10T16:00:26.513434+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:00:26.513434+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":6770,"log_dups_size":0,"ondisk_log_size":6770,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27721728,"num_objects":6768,"num_object_clones":0,"num_object_copies":13536,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6768,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3401,"num_write_kb":13604,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2464},{"shards":"0,2","objects":4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.989413+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T15:59:15.989413+0000","last_peered":"2024-09-10T15:59:15.989413+0000","last_clean":"2024-09-10T15:59:15.989413+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T15:59:15.989413+0000","last_undegraded":"2024-09-10T15:59:15.989413+0000","last_fullsized":"2024-09-10T15:59:15.989413+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'6626","reported_seq":6760,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:26.491430+0000","last_change":"2024-09-10T15:59:16.251610+0000","last_active":"2024-09-10T16:00:26.491430+0000","last_peered":"2024-09-10T16:00:26.491430+0000","last_clean":"2024-09-10T15:59:14.115595+0000","last_became_active":"2024-09-10T15:59:16.244089+0000","last_became_peered":"2024-09-10T15:59:16.244089+0000","last_unstale":"2024-09-10T16:00:26.491430+0000","last_undegraded":"2024-09-10T15:59:15.847127+0000","last_fullsized":"2024-09-10T15:59:15.774437+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":6626,"log_dups_size":0,"ondisk_log_size":6626,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27107328,"num_objects":6618,"num_object_clones":0,"num_object_copies":13236,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6618,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3016,"num_write_kb":12064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2415},{"shards":"0,2","objects":4203}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":89,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.773633+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T15:59:15.773633+0000","last_peered":"2024-09-10T15:59:15.773633+0000","last_clean":"2024-09-10T15:59:15.773633+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T15:59:15.773633+0000","last_undegraded":"2024-09-10T15:59:15.773633+0000","last_fullsized":"2024-09-10T15:59:15.773633+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'6906","reported_seq":7043,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:26.487321+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:00:26.487321+0000","last_peered":"2024-09-10T16:00:26.487321+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:00:26.487321+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":6906,"log_dups_size":0,"ondisk_log_size":6906,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28270592,"num_objects":6902,"num_object_clones":0,"num_object_copies":13804,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6902,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3161,"num_write_kb":12644,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2531},{"shards":"0,2","objects":4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.916017+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T15:59:15.916017+0000","last_peered":"2024-09-10T15:59:15.916017+0000","last_clean":"2024-09-10T15:59:15.916017+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T15:59:15.916017+0000","last_undegraded":"2024-09-10T15:59:15.916017+0000","last_fullsized":"2024-09-10T15:59:15.916017+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.776033+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T15:59:15.776033+0000","last_peered":"2024-09-10T15:59:15.776033+0000","last_clean":"2024-09-10T15:59:15.776033+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T15:59:15.776033+0000","last_undegraded":"2024-09-10T15:59:15.776033+0000","last_fullsized":"2024-09-10T15:59:15.776033+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'6889","reported_seq":7028,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:26.475392+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:00:26.475392+0000","last_peered":"2024-09-10T16:00:26.475392+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:00:26.475392+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":6889,"log_dups_size":0,"ondisk_log_size":6889,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28169158,"num_objects":6879,"num_object_clones":0,"num_object_copies":13758,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6879,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3052,"num_write_kb":12208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2556},{"shards":"0,2","objects":4323}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":446644958,"num_objects":109053,"num_object_clones":0,"num_object_copies":218106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":47117,"num_objects_misplaced":2672,"num_objects_unfound":0,"num_objects_dirty":109053,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":66899,"num_write_kb":267596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1599,"num_bytes_recovered":6549504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":902094848,"data_stored":901939489,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":109125,"ondisk_log_size":109125,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986206,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":528628,"kb_used_data":356708,"kb_used_omap":4,"kb_used_meta":171899,"kb_avail":93843212,"statfs":{"total":96636764160,"available":96095449088,"internally_reserved":0,"allocated":365268992,"data_stored":364991232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":176025258},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,2,1,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659152,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":228504,"kb_used_data":128776,"kb_used_omap":4,"kb_used_meta":99707,"kb_avail":94143336,"statfs":{"total":96636764160,"available":96402776064,"internally_reserved":0,"allocated":131866624,"data_stored":131537818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":102100654},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,0,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182434,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":660320,"kb_used_data":394064,"kb_used_omap":4,"kb_used_meta":266235,"kb_avail":93711520,"statfs":{"total":96636764160,"available":95960596480,"internally_reserved":0,"allocated":403521536,"data_stored":403239699,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":272625322},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,7,4,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":403763200,"data_stored":403727070,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":132472832,"data_stored":132389733,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":365858816,"data_stored":365822686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:00:30.974 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:00:30.980 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 118 20 110842 110822 3.668 1.9375 0.0764723 0.021289 2024-09-10T16:00:31.519 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:00:31.521 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:00:31.546 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":133,"stamp":"2024-09-10T16:00:29.566374+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":446644977,"num_objects":109055,"num_object_clones":0,"num_object_copies":218110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":47117,"num_objects_misplaced":2672,"num_objects_unfound":0,"num_objects_dirty":109055,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":66901,"num_write_kb":267598,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1601,"num_bytes_recovered":6549523,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":109128,"ondisk_log_size":109128,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1417452,"kb_used_data":879548,"kb_used_omap":12,"kb_used_meta":537843,"kb_avail":281698068,"statfs":{"total":289910292480,"available":288458821632,"internally_reserved":0,"allocated":900657152,"data_stored":899768749,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":550751234},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,6,8,7,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":26161152,"num_objects":6387,"num_object_clones":0,"num_object_copies":12774,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-337,"num_objects_unfound":0,"num_objects_dirty":6387,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6387,"num_write_kb":25548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":337,"num_bytes_recovered":1380352,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004532"},"pg_stats":[{"pgid":"2.f","version":"43'6709","reported_seq":6778,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:26.331994+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:00:26.331994+0000","last_peered":"2024-09-10T16:00:26.331994+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:00:26.331994+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":6709,"log_dups_size":0,"ondisk_log_size":6709,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27480064,"num_objects":6709,"num_object_clones":0,"num_object_copies":13418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6709,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6709,"num_write_kb":26836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2450},{"shards":"0,2","objects":4258}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'6875","reported_seq":6982,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:00:29.370631+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:00:29.370631+0000","last_peered":"2024-09-10T16:00:29.370631+0000","last_clean":"2024-09-10T16:00:29.370631+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:00:29.370631+0000","last_undegraded":"2024-09-10T16:00:29.370631+0000","last_fullsized":"2024-09-10T16:00:29.370631+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":6875,"log_dups_size":0,"ondisk_log_size":6875,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28136390,"num_objects":6871,"num_object_clones":0,"num_object_copies":13742,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6871,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4980,"num_write_kb":19920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'6848","reported_seq":10165,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T16:00:29.424339+0000","last_change":"2024-09-10T15:59:16.142612+0000","last_active":"2024-09-10T16:00:29.424339+0000","last_peered":"2024-09-10T16:00:29.424339+0000","last_clean":"2024-09-10T15:59:14.092473+0000","last_became_active":"2024-09-10T15:59:16.109611+0000","last_became_peered":"2024-09-10T15:59:16.109611+0000","last_unstale":"2024-09-10T16:00:29.424339+0000","last_undegraded":"2024-09-10T16:00:29.424339+0000","last_fullsized":"2024-09-10T15:59:15.770264+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":6848,"log_dups_size":0,"ondisk_log_size":6848,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28033024,"num_objects":6844,"num_object_clones":0,"num_object_copies":13688,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2672,"num_objects_unfound":0,"num_objects_dirty":6844,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4852,"num_write_kb":19408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1596,"num_bytes_recovered":6537216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2672},{"shards":"2","objects":4172}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'6875","reported_seq":6950,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:00:29.226323+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:00:29.226323+0000","last_peered":"2024-09-10T16:00:29.226323+0000","last_clean":"2024-09-10T16:00:29.226323+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:00:29.226323+0000","last_undegraded":"2024-09-10T16:00:29.226323+0000","last_fullsized":"2024-09-10T16:00:29.226323+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":6875,"log_dups_size":0,"ondisk_log_size":6875,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28160000,"num_objects":6875,"num_object_clones":0,"num_object_copies":13750,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6875,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6875,"num_write_kb":27500,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'6808","reported_seq":6933,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:00:26.446808+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:00:26.446808+0000","last_peered":"2024-09-10T16:00:26.446808+0000","last_clean":"2024-09-10T16:00:26.446808+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:00:26.446808+0000","last_undegraded":"2024-09-10T16:00:26.446808+0000","last_fullsized":"2024-09-10T16:00:26.446808+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":6808,"log_dups_size":0,"ondisk_log_size":6808,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27857379,"num_objects":6802,"num_object_clones":0,"num_object_copies":13604,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6802,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3519,"num_write_kb":14076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'6871","reported_seq":7001,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:26.426591+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:00:26.426591+0000","last_peered":"2024-09-10T16:00:26.426591+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:00:26.426591+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":6871,"log_dups_size":0,"ondisk_log_size":6871,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28123619,"num_objects":6867,"num_object_clones":0,"num_object_copies":13734,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6867,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3483,"num_write_kb":13932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2496},{"shards":"0,2","objects":4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'6809","reported_seq":6942,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:26.483196+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:00:26.483196+0000","last_peered":"2024-09-10T16:00:26.483196+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:00:26.483196+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":6809,"log_dups_size":0,"ondisk_log_size":6809,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27866054,"num_objects":6805,"num_object_clones":0,"num_object_copies":13610,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6805,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3308,"num_write_kb":13232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2568},{"shards":"0,2","objects":4237}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'6911","reported_seq":7036,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:00:26.458567+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:00:26.458567+0000","last_peered":"2024-09-10T16:00:26.458567+0000","last_clean":"2024-09-10T16:00:26.458567+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:00:26.458567+0000","last_undegraded":"2024-09-10T16:00:26.458567+0000","last_fullsized":"2024-09-10T16:00:26.458567+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":6911,"log_dups_size":0,"ondisk_log_size":6911,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28299264,"num_objects":6909,"num_object_clones":0,"num_object_copies":13818,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6909,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3758,"num_write_kb":15032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.775865+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T15:59:15.775865+0000","last_peered":"2024-09-10T15:59:15.775865+0000","last_clean":"2024-09-10T15:59:15.775865+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T15:59:15.775865+0000","last_undegraded":"2024-09-10T15:59:15.775865+0000","last_fullsized":"2024-09-10T15:59:15.775865+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'6620","reported_seq":6753,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:29.445789+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:00:29.445789+0000","last_peered":"2024-09-10T16:00:29.445789+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:00:29.445789+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":6620,"log_dups_size":0,"ondisk_log_size":6620,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27087331,"num_objects":6614,"num_object_clones":0,"num_object_copies":13228,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6614,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4439,"num_write_kb":17756,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4123},{"shards":"2","objects":2491}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":89,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.772619+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T15:59:15.772619+0000","last_peered":"2024-09-10T15:59:15.772619+0000","last_clean":"2024-09-10T15:59:15.772619+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T15:59:15.772619+0000","last_undegraded":"2024-09-10T15:59:15.772619+0000","last_fullsized":"2024-09-10T15:59:15.772619+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'6725","reported_seq":6849,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:26.480070+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:00:26.480070+0000","last_peered":"2024-09-10T16:00:26.480070+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:00:26.480070+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":6725,"log_dups_size":0,"ondisk_log_size":6725,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27512832,"num_objects":6717,"num_object_clones":0,"num_object_copies":13434,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6717,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3229,"num_write_kb":12916,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2462},{"shards":"0,2","objects":4255}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":99,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.884297+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T15:59:15.884297+0000","last_peered":"2024-09-10T15:59:15.884297+0000","last_clean":"2024-09-10T15:59:15.884297+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T15:59:15.884297+0000","last_undegraded":"2024-09-10T15:59:15.884297+0000","last_fullsized":"2024-09-10T15:59:15.884297+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'6894","reported_seq":7024,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:29.397113+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:00:29.397113+0000","last_peered":"2024-09-10T16:00:29.397113+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:00:29.397113+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":6894,"log_dups_size":0,"ondisk_log_size":6894,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28217827,"num_objects":6890,"num_object_clones":0,"num_object_copies":13780,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6890,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4574,"num_write_kb":18296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4304},{"shards":"2","objects":2586}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.837509+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T15:59:15.837509+0000","last_peered":"2024-09-10T15:59:15.837509+0000","last_clean":"2024-09-10T15:59:15.837509+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T15:59:15.837509+0000","last_undegraded":"2024-09-10T15:59:15.837509+0000","last_fullsized":"2024-09-10T15:59:15.837509+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'6989","reported_seq":7113,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:29.357500+0000","last_change":"2024-09-10T15:59:16.174113+0000","last_active":"2024-09-10T16:00:29.357500+0000","last_peered":"2024-09-10T16:00:29.357500+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:00:29.357500+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":6989,"log_dups_size":0,"ondisk_log_size":6989,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28602368,"num_objects":6983,"num_object_clones":0,"num_object_copies":13966,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4354,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6983,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4543,"num_write_kb":18172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4352},{"shards":"2","objects":2631}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'6770","reported_seq":6900,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:26.513434+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:00:26.513434+0000","last_peered":"2024-09-10T16:00:26.513434+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:00:26.513434+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":6770,"log_dups_size":0,"ondisk_log_size":6770,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27721728,"num_objects":6768,"num_object_clones":0,"num_object_copies":13536,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6768,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3401,"num_write_kb":13604,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2464},{"shards":"0,2","objects":4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.989413+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T15:59:15.989413+0000","last_peered":"2024-09-10T15:59:15.989413+0000","last_clean":"2024-09-10T15:59:15.989413+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T15:59:15.989413+0000","last_undegraded":"2024-09-10T15:59:15.989413+0000","last_fullsized":"2024-09-10T15:59:15.989413+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'6626","reported_seq":6760,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:26.491430+0000","last_change":"2024-09-10T15:59:16.251610+0000","last_active":"2024-09-10T16:00:26.491430+0000","last_peered":"2024-09-10T16:00:26.491430+0000","last_clean":"2024-09-10T15:59:14.115595+0000","last_became_active":"2024-09-10T15:59:16.244089+0000","last_became_peered":"2024-09-10T15:59:16.244089+0000","last_unstale":"2024-09-10T16:00:26.491430+0000","last_undegraded":"2024-09-10T15:59:15.847127+0000","last_fullsized":"2024-09-10T15:59:15.774437+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":6626,"log_dups_size":0,"ondisk_log_size":6626,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27107328,"num_objects":6618,"num_object_clones":0,"num_object_copies":13236,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6618,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3016,"num_write_kb":12064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2415},{"shards":"0,2","objects":4203}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":89,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.773633+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T15:59:15.773633+0000","last_peered":"2024-09-10T15:59:15.773633+0000","last_clean":"2024-09-10T15:59:15.773633+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T15:59:15.773633+0000","last_undegraded":"2024-09-10T15:59:15.773633+0000","last_fullsized":"2024-09-10T15:59:15.773633+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'6906","reported_seq":7043,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:26.487321+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:00:26.487321+0000","last_peered":"2024-09-10T16:00:26.487321+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:00:26.487321+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":6906,"log_dups_size":0,"ondisk_log_size":6906,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28270592,"num_objects":6902,"num_object_clones":0,"num_object_copies":13804,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6902,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3161,"num_write_kb":12644,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2531},{"shards":"0,2","objects":4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.916017+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T15:59:15.916017+0000","last_peered":"2024-09-10T15:59:15.916017+0000","last_clean":"2024-09-10T15:59:15.916017+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T15:59:15.916017+0000","last_undegraded":"2024-09-10T15:59:15.916017+0000","last_fullsized":"2024-09-10T15:59:15.916017+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.776033+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T15:59:15.776033+0000","last_peered":"2024-09-10T15:59:15.776033+0000","last_clean":"2024-09-10T15:59:15.776033+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T15:59:15.776033+0000","last_undegraded":"2024-09-10T15:59:15.776033+0000","last_fullsized":"2024-09-10T15:59:15.776033+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'6889","reported_seq":7028,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:26.475392+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:00:26.475392+0000","last_peered":"2024-09-10T16:00:26.475392+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:00:26.475392+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":6889,"log_dups_size":0,"ondisk_log_size":6889,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28169158,"num_objects":6879,"num_object_clones":0,"num_object_copies":13758,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6879,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3052,"num_write_kb":12208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2556},{"shards":"0,2","objects":4323}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":446644958,"num_objects":109053,"num_object_clones":0,"num_object_copies":218106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":47117,"num_objects_misplaced":2672,"num_objects_unfound":0,"num_objects_dirty":109053,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":66899,"num_write_kb":267596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1599,"num_bytes_recovered":6549504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":902094848,"data_stored":901939489,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":109125,"ondisk_log_size":109125,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986206,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":528628,"kb_used_data":356708,"kb_used_omap":4,"kb_used_meta":171899,"kb_avail":93843212,"statfs":{"total":96636764160,"available":96095449088,"internally_reserved":0,"allocated":365268992,"data_stored":364991232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":176025258},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,2,1,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659152,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":228504,"kb_used_data":128776,"kb_used_omap":4,"kb_used_meta":99707,"kb_avail":94143336,"statfs":{"total":96636764160,"available":96402776064,"internally_reserved":0,"allocated":131866624,"data_stored":131537818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":102100654},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,0,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182434,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":660320,"kb_used_data":394064,"kb_used_omap":4,"kb_used_meta":266235,"kb_avail":93711520,"statfs":{"total":96636764160,"available":95960596480,"internally_reserved":0,"allocated":403521536,"data_stored":403239699,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":272625322},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,7,4,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":403763200,"data_stored":403727070,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":132472832,"data_stored":132389733,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":365858816,"data_stored":365822686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:00:31.547 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-10T16:00:31.547 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "43'6709", 'reported_seq': 6778, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:00:26.331994+0000', 'last_change': '2024-09-10T15:59:16.281558+0000', 'last_active': '2024-09-10T16:00:26.331994+0000', 'last_peered': '2024-09-10T16:00:26.331994+0000', 'last_clean': '2024-09-10T15:59:14.132456+0000', 'last_became_active': '2024-09-10T15:59:16.274407+0000', 'last_became_peered': '2024-09-10T15:59:16.274407+0000', 'last_unstale': '2024-09-10T16:00:26.331994+0000', 'last_undegraded': '2024-09-10T15:59:15.851279+0000', 'last_fullsized': '2024-09-10T15:59:15.774356+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'objects_scrubbed': 0, 'log_size': 6709, 'log_dups_size': 0, 'ondisk_log_size': 6709, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27480064, 'num_objects': 6709, 'num_object_clones': 0, 'num_object_copies': 13418, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4259, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6709, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6709, 'num_write_kb': 26836, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2450}, {'shards': '0,2', 'objects': 4258}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:00:31.547 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-10T16:00:31.547 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "43'6848", 'reported_seq': 10165, 'reported_epoch': 43, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-10T16:00:29.424339+0000', 'last_change': '2024-09-10T15:59:16.142612+0000', 'last_active': '2024-09-10T16:00:29.424339+0000', 'last_peered': '2024-09-10T16:00:29.424339+0000', 'last_clean': '2024-09-10T15:59:14.092473+0000', 'last_became_active': '2024-09-10T15:59:16.109611+0000', 'last_became_peered': '2024-09-10T15:59:16.109611+0000', 'last_unstale': '2024-09-10T16:00:29.424339+0000', 'last_undegraded': '2024-09-10T16:00:29.424339+0000', 'last_fullsized': '2024-09-10T15:59:15.770264+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2029", 'last_scrub_stamp': '2024-09-10T15:58:45.933789+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:45.933789+0000', 'objects_scrubbed': 1992, 'log_size': 6848, 'log_dups_size': 0, 'ondisk_log_size': 6848, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 668, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28033024, 'num_objects': 6844, 'num_object_clones': 0, 'num_object_copies': 13688, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 2672, 'num_objects_unfound': 0, 'num_objects_dirty': 6844, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4852, 'num_write_kb': 19408, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1596, 'num_bytes_recovered': 6537216, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2672}, {'shards': '2', 'objects': 4172}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:00:31.547 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-10T16:00:31.548 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "43'6871", 'reported_seq': 7001, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:00:26.426591+0000', 'last_change': '2024-09-10T15:59:16.308264+0000', 'last_active': '2024-09-10T16:00:26.426591+0000', 'last_peered': '2024-09-10T16:00:26.426591+0000', 'last_clean': '2024-09-10T15:59:14.111551+0000', 'last_became_active': '2024-09-10T15:59:16.300040+0000', 'last_became_peered': '2024-09-10T15:59:16.300040+0000', 'last_unstale': '2024-09-10T16:00:26.426591+0000', 'last_undegraded': '2024-09-10T15:59:15.878234+0000', 'last_fullsized': '2024-09-10T15:59:15.769328+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3442", 'last_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'objects_scrubbed': 3384, 'log_size': 6871, 'log_dups_size': 0, 'ondisk_log_size': 6871, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1292, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28123619, 'num_objects': 6867, 'num_object_clones': 0, 'num_object_copies': 13734, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4372, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6867, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 3483, 'num_write_kb': 13932, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2496}, {'shards': '0,2', 'objects': 4371}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:00:31.548 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-10T16:00:31.548 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "43'6809", 'reported_seq': 6942, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:00:26.483196+0000', 'last_change': '2024-09-10T15:59:16.331899+0000', 'last_active': '2024-09-10T16:00:26.483196+0000', 'last_peered': '2024-09-10T16:00:26.483196+0000', 'last_clean': '2024-09-10T15:59:14.139436+0000', 'last_became_active': '2024-09-10T15:59:16.318715+0000', 'last_became_peered': '2024-09-10T15:59:16.318715+0000', 'last_unstale': '2024-09-10T16:00:26.483196+0000', 'last_undegraded': '2024-09-10T15:59:15.877524+0000', 'last_fullsized': '2024-09-10T15:59:15.769042+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3537", 'last_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'objects_scrubbed': 3497, 'log_size': 6809, 'log_dups_size': 0, 'ondisk_log_size': 6809, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1224, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27866054, 'num_objects': 6805, 'num_object_clones': 0, 'num_object_copies': 13610, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4237, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6805, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 3308, 'num_write_kb': 13232, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2568}, {'shards': '0,2', 'objects': 4237}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:00:31.548 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-10T16:00:31.548 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "43'6620", 'reported_seq': 6753, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:00:29.445789+0000', 'last_change': '2024-09-10T15:59:16.175680+0000', 'last_active': '2024-09-10T16:00:29.445789+0000', 'last_peered': '2024-09-10T16:00:29.445789+0000', 'last_clean': '2024-09-10T15:59:14.088187+0000', 'last_became_active': '2024-09-10T15:59:16.165810+0000', 'last_became_peered': '2024-09-10T15:59:16.165810+0000', 'last_unstale': '2024-09-10T16:00:29.445789+0000', 'last_undegraded': '2024-09-10T15:59:15.822098+0000', 'last_fullsized': '2024-09-10T15:59:15.769123+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2218", 'last_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'objects_scrubbed': 2175, 'log_size': 6620, 'log_dups_size': 0, 'ondisk_log_size': 6620, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 825, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27087331, 'num_objects': 6614, 'num_object_clones': 0, 'num_object_copies': 13228, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4124, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6614, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 4439, 'num_write_kb': 17756, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4123}, {'shards': '2', 'objects': 2491}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:00:31.548 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-10T16:00:31.548 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "43'6725", 'reported_seq': 6849, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:00:26.480070+0000', 'last_change': '2024-09-10T15:59:16.277742+0000', 'last_active': '2024-09-10T16:00:26.480070+0000', 'last_peered': '2024-09-10T16:00:26.480070+0000', 'last_clean': '2024-09-10T15:59:14.129081+0000', 'last_became_active': '2024-09-10T15:59:16.269835+0000', 'last_became_peered': '2024-09-10T15:59:16.269835+0000', 'last_unstale': '2024-09-10T16:00:26.480070+0000', 'last_undegraded': '2024-09-10T15:59:15.860857+0000', 'last_fullsized': '2024-09-10T15:59:15.771410+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3557", 'last_scrub_stamp': '2024-09-10T15:59:02.493678+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:02.493678+0000', 'objects_scrubbed': 3488, 'log_size': 6725, 'log_dups_size': 0, 'ondisk_log_size': 6725, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1472, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27512832, 'num_objects': 6717, 'num_object_clones': 0, 'num_object_copies': 13434, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4257, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6717, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3229, 'num_write_kb': 12916, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2462}, {'shards': '0,2', 'objects': 4255}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:00:31.548 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-10T16:00:31.548 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "43'6894", 'reported_seq': 7024, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:00:29.397113+0000', 'last_change': '2024-09-10T15:59:16.284311+0000', 'last_active': '2024-09-10T16:00:29.397113+0000', 'last_peered': '2024-09-10T16:00:29.397113+0000', 'last_clean': '2024-09-10T15:59:14.116041+0000', 'last_became_active': '2024-09-10T15:59:16.274543+0000', 'last_became_peered': '2024-09-10T15:59:16.274543+0000', 'last_unstale': '2024-09-10T16:00:29.397113+0000', 'last_undegraded': '2024-09-10T15:59:15.812484+0000', 'last_fullsized': '2024-09-10T15:59:15.770885+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2356", 'last_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'objects_scrubbed': 2316, 'log_size': 6894, 'log_dups_size': 0, 'ondisk_log_size': 6894, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 783, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28217827, 'num_objects': 6890, 'num_object_clones': 0, 'num_object_copies': 13780, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6890, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 4574, 'num_write_kb': 18296, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4304}, {'shards': '2', 'objects': 2586}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:00:31.548 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-10T16:00:31.548 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "43'6989", 'reported_seq': 7113, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:00:29.357500+0000', 'last_change': '2024-09-10T15:59:16.174113+0000', 'last_active': '2024-09-10T16:00:29.357500+0000', 'last_peered': '2024-09-10T16:00:29.357500+0000', 'last_clean': '2024-09-10T15:59:14.110654+0000', 'last_became_active': '2024-09-10T15:59:16.164645+0000', 'last_became_peered': '2024-09-10T15:59:16.164645+0000', 'last_unstale': '2024-09-10T16:00:29.357500+0000', 'last_undegraded': '2024-09-10T15:59:15.803593+0000', 'last_fullsized': '2024-09-10T15:59:15.769083+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2486", 'last_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'objects_scrubbed': 2440, 'log_size': 6989, 'log_dups_size': 0, 'ondisk_log_size': 6989, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 792, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28602368, 'num_objects': 6983, 'num_object_clones': 0, 'num_object_copies': 13966, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4354, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6983, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4543, 'num_write_kb': 18172, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4352}, {'shards': '2', 'objects': 2631}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:00:31.549 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-10T16:00:31.549 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "43'6770", 'reported_seq': 6900, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:00:26.513434+0000', 'last_change': '2024-09-10T15:59:16.315641+0000', 'last_active': '2024-09-10T16:00:26.513434+0000', 'last_peered': '2024-09-10T16:00:26.513434+0000', 'last_clean': '2024-09-10T15:59:14.117001+0000', 'last_became_active': '2024-09-10T15:59:16.302245+0000', 'last_became_peered': '2024-09-10T15:59:16.302245+0000', 'last_unstale': '2024-09-10T16:00:26.513434+0000', 'last_undegraded': '2024-09-10T15:59:15.845012+0000', 'last_fullsized': '2024-09-10T15:59:15.770337+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3404", 'last_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'objects_scrubbed': 3367, 'log_size': 6770, 'log_dups_size': 0, 'ondisk_log_size': 6770, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 983, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27721728, 'num_objects': 6768, 'num_object_clones': 0, 'num_object_copies': 13536, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6768, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3401, 'num_write_kb': 13604, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2464}, {'shards': '0,2', 'objects': 4304}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:00:31.549 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-10T16:00:31.549 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "43'6626", 'reported_seq': 6760, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:00:26.491430+0000', 'last_change': '2024-09-10T15:59:16.251610+0000', 'last_active': '2024-09-10T16:00:26.491430+0000', 'last_peered': '2024-09-10T16:00:26.491430+0000', 'last_clean': '2024-09-10T15:59:14.115595+0000', 'last_became_active': '2024-09-10T15:59:16.244089+0000', 'last_became_peered': '2024-09-10T15:59:16.244089+0000', 'last_unstale': '2024-09-10T16:00:26.491430+0000', 'last_undegraded': '2024-09-10T15:59:15.847127+0000', 'last_fullsized': '2024-09-10T15:59:15.774437+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3653", 'last_scrub_stamp': '2024-09-10T15:59:05.298468+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:05.298468+0000', 'objects_scrubbed': 3602, 'log_size': 6626, 'log_dups_size': 0, 'ondisk_log_size': 6626, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1289, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27107328, 'num_objects': 6618, 'num_object_clones': 0, 'num_object_copies': 13236, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4206, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6618, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3016, 'num_write_kb': 12064, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2415}, {'shards': '0,2', 'objects': 4203}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:00:31.549 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-10T16:00:31.549 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "43'6906", 'reported_seq': 7043, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:00:26.487321+0000', 'last_change': '2024-09-10T15:59:16.311291+0000', 'last_active': '2024-09-10T16:00:26.487321+0000', 'last_peered': '2024-09-10T16:00:26.487321+0000', 'last_clean': '2024-09-10T15:59:14.116363+0000', 'last_became_active': '2024-09-10T15:59:16.301244+0000', 'last_became_peered': '2024-09-10T15:59:16.301244+0000', 'last_unstale': '2024-09-10T16:00:26.487321+0000', 'last_undegraded': '2024-09-10T15:59:15.883434+0000', 'last_fullsized': '2024-09-10T15:59:15.772756+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3789", 'last_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'objects_scrubbed': 3741, 'log_size': 6906, 'log_dups_size': 0, 'ondisk_log_size': 6906, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1440, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28270592, 'num_objects': 6902, 'num_object_clones': 0, 'num_object_copies': 13804, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4373, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6902, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3161, 'num_write_kb': 12644, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2531}, {'shards': '0,2', 'objects': 4371}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:00:31.549 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-10T16:00:31.549 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "43'6889", 'reported_seq': 7028, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:00:26.475392+0000', 'last_change': '2024-09-10T15:59:16.313409+0000', 'last_active': '2024-09-10T16:00:26.475392+0000', 'last_peered': '2024-09-10T16:00:26.475392+0000', 'last_clean': '2024-09-10T15:59:14.129780+0000', 'last_became_active': '2024-09-10T15:59:16.300932+0000', 'last_became_peered': '2024-09-10T15:59:16.300932+0000', 'last_unstale': '2024-09-10T16:00:26.475392+0000', 'last_undegraded': '2024-09-10T15:59:15.887823+0000', 'last_fullsized': '2024-09-10T15:59:15.768513+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3880", 'last_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'objects_scrubbed': 3827, 'log_size': 6889, 'log_dups_size': 0, 'ondisk_log_size': 6889, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1397, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28169158, 'num_objects': 6879, 'num_object_clones': 0, 'num_object_copies': 13758, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4325, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6879, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3052, 'num_write_kb': 12208, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2556}, {'shards': '0,2', 'objects': 4323}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:00:31.550 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T16:00:31.981 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 119 20 111329 111309 3.65317 1.90234 0.0380643 0.0213779 2024-09-10T16:00:32.290 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:00:32.291 INFO:teuthology.orchestra.run.smithi115.stdout:{"fsid":"7effee29-df05-4351-9741-ee203564ed81","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 47117/221442 objects degraded (21.277%), 11 pgs degraded, 12 pgs undersized","count":23},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":196,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":43,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983954,"num_in_osds":3,"osd_in_since":1725983947,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":110721,"data_bytes":453468913,"bytes_used":1463881728,"bytes_avail":288446410752,"bytes_total":289910292480,"degraded_objects":47117,"degraded_total":221442,"degraded_ratio":0.21277354792677086,"misplaced_objects":2672,"misplaced_total":221442,"misplaced_ratio":0.01206636500754148,"recovering_objects_per_sec":18,"recovering_bytes_per_sec":75750,"recovering_keys_per_sec":0,"num_objects_recovered":222,"num_bytes_recovered":909312,"num_keys_recovered":0,"write_bytes_sec":2390237,"read_op_per_sec":0,"write_op_per_sec":583},"fsmap":{"epoch":1,"btime":"2024-09-10T15:57:14:967120+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T16:00:13.560776+0000","services":{}},"progress_events":{}} 2024-09-10T16:00:32.317 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-10T16:00:32.317 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:00:32.892 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:00:32.893 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:00:32.921 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":134,"stamp":"2024-09-10T16:00:31.567062+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":453468913,"num_objects":110721,"num_object_clones":0,"num_object_copies":221442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":47117,"num_objects_misplaced":2672,"num_objects_unfound":0,"num_objects_dirty":110721,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":68567,"num_write_kb":274262,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1601,"num_bytes_recovered":6549523,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":110794,"ondisk_log_size":110794,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1429572,"kb_used_data":887508,"kb_used_omap":12,"kb_used_meta":542003,"kb_avail":281685948,"statfs":{"total":289910292480,"available":288446410752,"internally_reserved":0,"allocated":908808192,"data_stored":907919789,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":555011074},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,10,1,8,5,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":28692480,"num_objects":7005,"num_object_clones":0,"num_object_copies":14010,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-222,"num_objects_unfound":0,"num_objects_dirty":7005,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7005,"num_write_kb":28020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":222,"num_bytes_recovered":909312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004027"},"pg_stats":[{"pgid":"2.f","version":"43'6893","reported_seq":6963,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:31.480619+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:00:31.480619+0000","last_peered":"2024-09-10T16:00:31.480619+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:00:31.480619+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":6893,"log_dups_size":0,"ondisk_log_size":6893,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28233728,"num_objects":6893,"num_object_clones":0,"num_object_copies":13786,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6893,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6893,"num_write_kb":27572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2634},{"shards":"0,2","objects":4258}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'6875","reported_seq":6982,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:00:29.370631+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:00:29.370631+0000","last_peered":"2024-09-10T16:00:29.370631+0000","last_clean":"2024-09-10T16:00:29.370631+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:00:29.370631+0000","last_undegraded":"2024-09-10T16:00:29.370631+0000","last_fullsized":"2024-09-10T16:00:29.370631+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":6875,"log_dups_size":0,"ondisk_log_size":6875,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28136390,"num_objects":6871,"num_object_clones":0,"num_object_copies":13742,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6871,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4980,"num_write_kb":19920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'6848","reported_seq":10165,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T16:00:29.424339+0000","last_change":"2024-09-10T15:59:16.142612+0000","last_active":"2024-09-10T16:00:29.424339+0000","last_peered":"2024-09-10T16:00:29.424339+0000","last_clean":"2024-09-10T15:59:14.092473+0000","last_became_active":"2024-09-10T15:59:16.109611+0000","last_became_peered":"2024-09-10T15:59:16.109611+0000","last_unstale":"2024-09-10T16:00:29.424339+0000","last_undegraded":"2024-09-10T16:00:29.424339+0000","last_fullsized":"2024-09-10T15:59:15.770264+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":6848,"log_dups_size":0,"ondisk_log_size":6848,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28033024,"num_objects":6844,"num_object_clones":0,"num_object_copies":13688,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2672,"num_objects_unfound":0,"num_objects_dirty":6844,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4852,"num_write_kb":19408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1596,"num_bytes_recovered":6537216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2672},{"shards":"2","objects":4172}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'6875","reported_seq":6950,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:00:29.226323+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:00:29.226323+0000","last_peered":"2024-09-10T16:00:29.226323+0000","last_clean":"2024-09-10T16:00:29.226323+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:00:29.226323+0000","last_undegraded":"2024-09-10T16:00:29.226323+0000","last_fullsized":"2024-09-10T16:00:29.226323+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":6875,"log_dups_size":0,"ondisk_log_size":6875,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28160000,"num_objects":6875,"num_object_clones":0,"num_object_copies":13750,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6875,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6875,"num_write_kb":27500,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'6976","reported_seq":7101,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:00:31.458666+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:00:31.458666+0000","last_peered":"2024-09-10T16:00:31.458666+0000","last_clean":"2024-09-10T16:00:31.458666+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:00:31.458666+0000","last_undegraded":"2024-09-10T16:00:31.458666+0000","last_fullsized":"2024-09-10T16:00:31.458666+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":6976,"log_dups_size":0,"ondisk_log_size":6976,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28545507,"num_objects":6970,"num_object_clones":0,"num_object_copies":13940,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6970,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3687,"num_write_kb":14748,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'7032","reported_seq":7165,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:31.487367+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:00:31.487367+0000","last_peered":"2024-09-10T16:00:31.487367+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:00:31.487367+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":7032,"log_dups_size":0,"ondisk_log_size":7032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28783075,"num_objects":7028,"num_object_clones":0,"num_object_copies":14056,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7028,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3644,"num_write_kb":14576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2657},{"shards":"0,2","objects":4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'6960","reported_seq":7093,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:31.479180+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:00:31.479180+0000","last_peered":"2024-09-10T16:00:31.479180+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:00:31.479180+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":6960,"log_dups_size":0,"ondisk_log_size":6960,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28484550,"num_objects":6956,"num_object_clones":0,"num_object_copies":13912,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6956,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3459,"num_write_kb":13836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2719},{"shards":"0,2","objects":4237}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'7082","reported_seq":7207,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:00:31.506798+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:00:31.506798+0000","last_peered":"2024-09-10T16:00:31.506798+0000","last_clean":"2024-09-10T16:00:31.506798+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:00:31.506798+0000","last_undegraded":"2024-09-10T16:00:31.506798+0000","last_fullsized":"2024-09-10T16:00:31.506798+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":7082,"log_dups_size":0,"ondisk_log_size":7082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28999680,"num_objects":7080,"num_object_clones":0,"num_object_copies":14160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7080,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3929,"num_write_kb":15716,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.775865+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T15:59:15.775865+0000","last_peered":"2024-09-10T15:59:15.775865+0000","last_clean":"2024-09-10T15:59:15.775865+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T15:59:15.775865+0000","last_undegraded":"2024-09-10T15:59:15.775865+0000","last_fullsized":"2024-09-10T15:59:15.775865+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'6620","reported_seq":6753,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:29.445789+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:00:29.445789+0000","last_peered":"2024-09-10T16:00:29.445789+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:00:29.445789+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":6620,"log_dups_size":0,"ondisk_log_size":6620,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27087331,"num_objects":6614,"num_object_clones":0,"num_object_copies":13228,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6614,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4439,"num_write_kb":17756,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4123},{"shards":"2","objects":2491}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":89,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.772619+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T15:59:15.772619+0000","last_peered":"2024-09-10T15:59:15.772619+0000","last_clean":"2024-09-10T15:59:15.772619+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T15:59:15.772619+0000","last_undegraded":"2024-09-10T15:59:15.772619+0000","last_fullsized":"2024-09-10T15:59:15.772619+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'6915","reported_seq":7040,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:31.453079+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:00:31.453079+0000","last_peered":"2024-09-10T16:00:31.453079+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:00:31.453079+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":6915,"log_dups_size":0,"ondisk_log_size":6915,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28291072,"num_objects":6907,"num_object_clones":0,"num_object_copies":13814,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6907,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3419,"num_write_kb":13676,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2652},{"shards":"0,2","objects":4255}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":99,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.884297+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T15:59:15.884297+0000","last_peered":"2024-09-10T15:59:15.884297+0000","last_clean":"2024-09-10T15:59:15.884297+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T15:59:15.884297+0000","last_undegraded":"2024-09-10T15:59:15.884297+0000","last_fullsized":"2024-09-10T15:59:15.884297+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'6894","reported_seq":7024,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:29.397113+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:00:29.397113+0000","last_peered":"2024-09-10T16:00:29.397113+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:00:29.397113+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":6894,"log_dups_size":0,"ondisk_log_size":6894,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28217827,"num_objects":6890,"num_object_clones":0,"num_object_copies":13780,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6890,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4574,"num_write_kb":18296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4304},{"shards":"2","objects":2586}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.837509+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T15:59:15.837509+0000","last_peered":"2024-09-10T15:59:15.837509+0000","last_clean":"2024-09-10T15:59:15.837509+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T15:59:15.837509+0000","last_undegraded":"2024-09-10T15:59:15.837509+0000","last_fullsized":"2024-09-10T15:59:15.837509+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'6989","reported_seq":7113,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:29.357500+0000","last_change":"2024-09-10T15:59:16.174113+0000","last_active":"2024-09-10T16:00:29.357500+0000","last_peered":"2024-09-10T16:00:29.357500+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:00:29.357500+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":6989,"log_dups_size":0,"ondisk_log_size":6989,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28602368,"num_objects":6983,"num_object_clones":0,"num_object_copies":13966,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4354,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6983,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4543,"num_write_kb":18172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4352},{"shards":"2","objects":2631}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'6932","reported_seq":7062,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:31.481649+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:00:31.481649+0000","last_peered":"2024-09-10T16:00:31.481649+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:00:31.481649+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":6932,"log_dups_size":0,"ondisk_log_size":6932,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28385280,"num_objects":6930,"num_object_clones":0,"num_object_copies":13860,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6930,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3563,"num_write_kb":14252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2626},{"shards":"0,2","objects":4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.989413+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T15:59:15.989413+0000","last_peered":"2024-09-10T15:59:15.989413+0000","last_clean":"2024-09-10T15:59:15.989413+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T15:59:15.989413+0000","last_undegraded":"2024-09-10T15:59:15.989413+0000","last_fullsized":"2024-09-10T15:59:15.989413+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'6786","reported_seq":6924,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:31.482968+0000","last_change":"2024-09-10T15:59:16.251610+0000","last_active":"2024-09-10T16:00:31.482968+0000","last_peered":"2024-09-10T16:00:31.482968+0000","last_clean":"2024-09-10T15:59:14.115595+0000","last_became_active":"2024-09-10T15:59:16.244089+0000","last_became_peered":"2024-09-10T15:59:16.244089+0000","last_unstale":"2024-09-10T16:00:31.482968+0000","last_undegraded":"2024-09-10T15:59:15.847127+0000","last_fullsized":"2024-09-10T15:59:15.774437+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":6786,"log_dups_size":0,"ondisk_log_size":6786,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27762688,"num_objects":6778,"num_object_clones":0,"num_object_copies":13556,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6778,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3176,"num_write_kb":12704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2575},{"shards":"0,2","objects":4203}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":89,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.773633+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T15:59:15.773633+0000","last_peered":"2024-09-10T15:59:15.773633+0000","last_clean":"2024-09-10T15:59:15.773633+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T15:59:15.773633+0000","last_undegraded":"2024-09-10T15:59:15.773633+0000","last_fullsized":"2024-09-10T15:59:15.773633+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'7062","reported_seq":7199,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:31.499745+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:00:31.499745+0000","last_peered":"2024-09-10T16:00:31.499745+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:00:31.499745+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":7062,"log_dups_size":0,"ondisk_log_size":7062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28909568,"num_objects":7058,"num_object_clones":0,"num_object_copies":14116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7058,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3317,"num_write_kb":13268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2687},{"shards":"0,2","objects":4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.916017+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T15:59:15.916017+0000","last_peered":"2024-09-10T15:59:15.916017+0000","last_clean":"2024-09-10T15:59:15.916017+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T15:59:15.916017+0000","last_undegraded":"2024-09-10T15:59:15.916017+0000","last_fullsized":"2024-09-10T15:59:15.916017+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.776033+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T15:59:15.776033+0000","last_peered":"2024-09-10T15:59:15.776033+0000","last_clean":"2024-09-10T15:59:15.776033+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T15:59:15.776033+0000","last_undegraded":"2024-09-10T15:59:15.776033+0000","last_fullsized":"2024-09-10T15:59:15.776033+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'7052","reported_seq":7186,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:31.514032+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:00:31.514032+0000","last_peered":"2024-09-10T16:00:31.514032+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:00:31.514032+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":7052,"log_dups_size":0,"ondisk_log_size":7052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28836806,"num_objects":7042,"num_object_clones":0,"num_object_copies":14084,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7042,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3215,"num_write_kb":12860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2719},{"shards":"0,2","objects":4323}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":453468894,"num_objects":110719,"num_object_clones":0,"num_object_copies":221438,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":47117,"num_objects_misplaced":2672,"num_objects_unfound":0,"num_objects_dirty":110719,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":68565,"num_write_kb":274260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1599,"num_bytes_recovered":6549504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":910237696,"data_stored":910082337,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":110791,"ondisk_log_size":110791,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986206,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":528628,"kb_used_data":356708,"kb_used_omap":4,"kb_used_meta":171899,"kb_avail":93843212,"statfs":{"total":96636764160,"available":96095449088,"internally_reserved":0,"allocated":365268992,"data_stored":364991232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":176025258},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,2,1,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659152,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":228504,"kb_used_data":128776,"kb_used_omap":4,"kb_used_meta":99707,"kb_avail":94143336,"statfs":{"total":96636764160,"available":96402776064,"internally_reserved":0,"allocated":131866624,"data_stored":131537818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":102100654},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,0,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182435,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":672440,"kb_used_data":402024,"kb_used_omap":4,"kb_used_meta":270395,"kb_avail":93699400,"statfs":{"total":96636764160,"available":95948185600,"internally_reserved":0,"allocated":411672576,"data_stored":411390739,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":276885162},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,4,0,5,5,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":411906048,"data_stored":411869918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":132472832,"data_stored":132389733,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":365858816,"data_stored":365822686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:00:32.981 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout:2024-09-10T16:00:32.980590+0000 min lat: 0.00225581 max lat: 2.21736 avg lat: 0.021464 2024-09-10T16:00:32.981 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-10T16:00:32.981 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 120 20 111811 111791 3.63841 1.88281 0.0783742 0.021464 2024-09-10T16:00:33.981 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 121 20 112347 112327 3.62565 2.09375 0.0204602 0.0215406 2024-09-10T16:00:34.981 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 122 20 112890 112870 3.61331 2.12109 0.0424246 0.0216136 2024-09-10T16:00:35.924 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:00:35.981 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 123 20 113439 113419 3.60137 2.14453 0.0601444 0.0216846 2024-09-10T16:00:36.484 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:00:36.485 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:00:36.507 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":136,"stamp":"2024-09-10T16:00:35.568317+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":457323249,"num_objects":111662,"num_object_clones":0,"num_object_copies":223324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":47117,"num_objects_misplaced":2553,"num_objects_unfound":0,"num_objects_dirty":111662,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":69508,"num_write_kb":278026,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1720,"num_bytes_recovered":7036947,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111735,"ondisk_log_size":111735,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1473588,"kb_used_data":900996,"kb_used_omap":12,"kb_used_meta":572531,"kb_avail":281641932,"statfs":{"total":289910292480,"available":288401338368,"internally_reserved":0,"allocated":922619904,"data_stored":921731501,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":586271746},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,4,3,5,6,10,7,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":25657344,"num_objects":6264,"num_object_clones":0,"num_object_copies":12528,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-341,"num_objects_unfound":0,"num_objects_dirty":6264,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6264,"num_write_kb":25056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":341,"num_bytes_recovered":1396736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003723"},"pg_stats":[{"pgid":"2.f","version":"43'6893","reported_seq":6963,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:31.480619+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:00:31.480619+0000","last_peered":"2024-09-10T16:00:31.480619+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:00:31.480619+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":6893,"log_dups_size":0,"ondisk_log_size":6893,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28233728,"num_objects":6893,"num_object_clones":0,"num_object_copies":13786,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6893,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6893,"num_write_kb":27572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2634},{"shards":"0,2","objects":4258}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'7015","reported_seq":7123,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:00:34.448346+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:00:34.448346+0000","last_peered":"2024-09-10T16:00:34.448346+0000","last_clean":"2024-09-10T16:00:34.448346+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:00:34.448346+0000","last_undegraded":"2024-09-10T16:00:34.448346+0000","last_fullsized":"2024-09-10T16:00:34.448346+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":7015,"log_dups_size":0,"ondisk_log_size":7015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28709830,"num_objects":7011,"num_object_clones":0,"num_object_copies":14022,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7011,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5120,"num_write_kb":20480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'7014","reported_seq":10569,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T16:00:34.483140+0000","last_change":"2024-09-10T15:59:16.142612+0000","last_active":"2024-09-10T16:00:34.483140+0000","last_peered":"2024-09-10T16:00:34.483140+0000","last_clean":"2024-09-10T15:59:14.092473+0000","last_became_active":"2024-09-10T15:59:16.109611+0000","last_became_peered":"2024-09-10T15:59:16.109611+0000","last_unstale":"2024-09-10T16:00:34.483140+0000","last_undegraded":"2024-09-10T16:00:34.483140+0000","last_fullsized":"2024-09-10T15:59:15.770264+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":7014,"log_dups_size":0,"ondisk_log_size":7014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28712960,"num_objects":7010,"num_object_clones":0,"num_object_copies":14020,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2553,"num_objects_unfound":0,"num_objects_dirty":7010,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5018,"num_write_kb":20072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1715,"num_bytes_recovered":7024640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2553},{"shards":"2","objects":4457}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'7029","reported_seq":7104,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:00:34.430693+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:00:34.430693+0000","last_peered":"2024-09-10T16:00:34.430693+0000","last_clean":"2024-09-10T16:00:34.430693+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:00:34.430693+0000","last_undegraded":"2024-09-10T16:00:34.430693+0000","last_fullsized":"2024-09-10T16:00:34.430693+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":7029,"log_dups_size":0,"ondisk_log_size":7029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28790784,"num_objects":7029,"num_object_clones":0,"num_object_copies":14058,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7029,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7029,"num_write_kb":28116,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'6976","reported_seq":7101,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:00:31.458666+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:00:31.458666+0000","last_peered":"2024-09-10T16:00:31.458666+0000","last_clean":"2024-09-10T16:00:31.458666+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:00:31.458666+0000","last_undegraded":"2024-09-10T16:00:31.458666+0000","last_fullsized":"2024-09-10T16:00:31.458666+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":6976,"log_dups_size":0,"ondisk_log_size":6976,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28545507,"num_objects":6970,"num_object_clones":0,"num_object_copies":13940,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6970,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3687,"num_write_kb":14748,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'7032","reported_seq":7165,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:31.487367+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:00:31.487367+0000","last_peered":"2024-09-10T16:00:31.487367+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:00:31.487367+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":7032,"log_dups_size":0,"ondisk_log_size":7032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28783075,"num_objects":7028,"num_object_clones":0,"num_object_copies":14056,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7028,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3644,"num_write_kb":14576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2657},{"shards":"0,2","objects":4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'6960","reported_seq":7093,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:31.479180+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:00:31.479180+0000","last_peered":"2024-09-10T16:00:31.479180+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:00:31.479180+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":6960,"log_dups_size":0,"ondisk_log_size":6960,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28484550,"num_objects":6956,"num_object_clones":0,"num_object_copies":13912,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6956,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3459,"num_write_kb":13836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2719},{"shards":"0,2","objects":4237}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'7082","reported_seq":7207,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:00:31.506798+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:00:31.506798+0000","last_peered":"2024-09-10T16:00:31.506798+0000","last_clean":"2024-09-10T16:00:31.506798+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:00:31.506798+0000","last_undegraded":"2024-09-10T16:00:31.506798+0000","last_fullsized":"2024-09-10T16:00:31.506798+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":7082,"log_dups_size":0,"ondisk_log_size":7082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28999680,"num_objects":7080,"num_object_clones":0,"num_object_copies":14160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7080,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3929,"num_write_kb":15716,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.775865+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T15:59:15.775865+0000","last_peered":"2024-09-10T15:59:15.775865+0000","last_clean":"2024-09-10T15:59:15.775865+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T15:59:15.775865+0000","last_undegraded":"2024-09-10T15:59:15.775865+0000","last_fullsized":"2024-09-10T15:59:15.775865+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'6767","reported_seq":6902,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:34.473041+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:00:34.473041+0000","last_peered":"2024-09-10T16:00:34.473041+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:00:34.473041+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":6767,"log_dups_size":0,"ondisk_log_size":6767,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27689443,"num_objects":6761,"num_object_clones":0,"num_object_copies":13522,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6761,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4586,"num_write_kb":18344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4123},{"shards":"2","objects":2638}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":89,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.772619+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T15:59:15.772619+0000","last_peered":"2024-09-10T15:59:15.772619+0000","last_clean":"2024-09-10T15:59:15.772619+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T15:59:15.772619+0000","last_undegraded":"2024-09-10T15:59:15.772619+0000","last_fullsized":"2024-09-10T15:59:15.772619+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'6915","reported_seq":7040,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:31.453079+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:00:31.453079+0000","last_peered":"2024-09-10T16:00:31.453079+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:00:31.453079+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":6915,"log_dups_size":0,"ondisk_log_size":6915,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28291072,"num_objects":6907,"num_object_clones":0,"num_object_copies":13814,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6907,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3419,"num_write_kb":13676,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2652},{"shards":"0,2","objects":4255}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":99,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.884297+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T15:59:15.884297+0000","last_peered":"2024-09-10T15:59:15.884297+0000","last_clean":"2024-09-10T15:59:15.884297+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T15:59:15.884297+0000","last_undegraded":"2024-09-10T15:59:15.884297+0000","last_fullsized":"2024-09-10T15:59:15.884297+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'7054","reported_seq":7180,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:34.391776+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:00:34.391776+0000","last_peered":"2024-09-10T16:00:34.391776+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:00:34.391776+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":7054,"log_dups_size":0,"ondisk_log_size":7054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28873187,"num_objects":7050,"num_object_clones":0,"num_object_copies":14100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7050,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4734,"num_write_kb":18936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4304},{"shards":"2","objects":2746}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.837509+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T15:59:15.837509+0000","last_peered":"2024-09-10T15:59:15.837509+0000","last_clean":"2024-09-10T15:59:15.837509+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T15:59:15.837509+0000","last_undegraded":"2024-09-10T15:59:15.837509+0000","last_fullsized":"2024-09-10T15:59:15.837509+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'7163","reported_seq":7284,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:34.453014+0000","last_change":"2024-09-10T15:59:16.174113+0000","last_active":"2024-09-10T16:00:34.453014+0000","last_peered":"2024-09-10T16:00:34.453014+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:00:34.453014+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":7163,"log_dups_size":0,"ondisk_log_size":7163,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29315072,"num_objects":7157,"num_object_clones":0,"num_object_copies":14314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4354,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7157,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4717,"num_write_kb":18868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4352},{"shards":"2","objects":2805}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'6932","reported_seq":7062,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:31.481649+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:00:31.481649+0000","last_peered":"2024-09-10T16:00:31.481649+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:00:31.481649+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":6932,"log_dups_size":0,"ondisk_log_size":6932,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28385280,"num_objects":6930,"num_object_clones":0,"num_object_copies":13860,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6930,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3563,"num_write_kb":14252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2626},{"shards":"0,2","objects":4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.989413+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T15:59:15.989413+0000","last_peered":"2024-09-10T15:59:15.989413+0000","last_clean":"2024-09-10T15:59:15.989413+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T15:59:15.989413+0000","last_undegraded":"2024-09-10T15:59:15.989413+0000","last_fullsized":"2024-09-10T15:59:15.989413+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'6786","reported_seq":6924,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:31.482968+0000","last_change":"2024-09-10T15:59:16.251610+0000","last_active":"2024-09-10T16:00:31.482968+0000","last_peered":"2024-09-10T16:00:31.482968+0000","last_clean":"2024-09-10T15:59:14.115595+0000","last_became_active":"2024-09-10T15:59:16.244089+0000","last_became_peered":"2024-09-10T15:59:16.244089+0000","last_unstale":"2024-09-10T16:00:31.482968+0000","last_undegraded":"2024-09-10T15:59:15.847127+0000","last_fullsized":"2024-09-10T15:59:15.774437+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":6786,"log_dups_size":0,"ondisk_log_size":6786,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27762688,"num_objects":6778,"num_object_clones":0,"num_object_copies":13556,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6778,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3176,"num_write_kb":12704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2575},{"shards":"0,2","objects":4203}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":89,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.773633+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T15:59:15.773633+0000","last_peered":"2024-09-10T15:59:15.773633+0000","last_clean":"2024-09-10T15:59:15.773633+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T15:59:15.773633+0000","last_undegraded":"2024-09-10T15:59:15.773633+0000","last_fullsized":"2024-09-10T15:59:15.773633+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'7062","reported_seq":7199,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:31.499745+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:00:31.499745+0000","last_peered":"2024-09-10T16:00:31.499745+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:00:31.499745+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":7062,"log_dups_size":0,"ondisk_log_size":7062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28909568,"num_objects":7058,"num_object_clones":0,"num_object_copies":14116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7058,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3317,"num_write_kb":13268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2687},{"shards":"0,2","objects":4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.916017+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T15:59:15.916017+0000","last_peered":"2024-09-10T15:59:15.916017+0000","last_clean":"2024-09-10T15:59:15.916017+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T15:59:15.916017+0000","last_undegraded":"2024-09-10T15:59:15.916017+0000","last_fullsized":"2024-09-10T15:59:15.916017+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.776033+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T15:59:15.776033+0000","last_peered":"2024-09-10T15:59:15.776033+0000","last_clean":"2024-09-10T15:59:15.776033+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T15:59:15.776033+0000","last_undegraded":"2024-09-10T15:59:15.776033+0000","last_fullsized":"2024-09-10T15:59:15.776033+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'7052","reported_seq":7186,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:31.514032+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:00:31.514032+0000","last_peered":"2024-09-10T16:00:31.514032+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:00:31.514032+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":7052,"log_dups_size":0,"ondisk_log_size":7052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28836806,"num_objects":7042,"num_object_clones":0,"num_object_copies":14084,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7042,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3215,"num_write_kb":12860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2719},{"shards":"0,2","objects":4323}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":457323230,"num_objects":111660,"num_object_clones":0,"num_object_copies":223320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":47117,"num_objects_misplaced":2553,"num_objects_unfound":0,"num_objects_dirty":111660,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":69506,"num_write_kb":278024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1718,"num_bytes_recovered":7036928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":923836416,"data_stored":923681057,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111732,"ondisk_log_size":111732,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986207,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":537904,"kb_used_data":361824,"kb_used_omap":4,"kb_used_meta":176059,"kb_avail":93833936,"statfs":{"total":96636764160,"available":96085950464,"internally_reserved":0,"allocated":370507776,"data_stored":370230016,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":180285098},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,3,2,1,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659153,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":263244,"kb_used_data":137148,"kb_used_omap":4,"kb_used_meta":126075,"kb_avail":94108596,"statfs":{"total":96636764160,"available":96367202304,"internally_reserved":0,"allocated":140439552,"data_stored":140110746,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":129101486},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,4,0,0,3,3,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182435,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":672440,"kb_used_data":402024,"kb_used_omap":4,"kb_used_meta":270395,"kb_avail":93699400,"statfs":{"total":96636764160,"available":95948185600,"internally_reserved":0,"allocated":411672576,"data_stored":411390739,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":276885162},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,4,0,5,5,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":411906048,"data_stored":411869918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":140939264,"data_stored":140856165,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":370991104,"data_stored":370954974,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:00:36.509 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:00:36.981 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 124 20 113918 113898 3.58742 1.87109 0.0301081 0.0217702 2024-09-10T16:00:37.068 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:00:37.069 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:00:37.099 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":136,"stamp":"2024-09-10T16:00:35.568317+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":457323249,"num_objects":111662,"num_object_clones":0,"num_object_copies":223324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":47117,"num_objects_misplaced":2553,"num_objects_unfound":0,"num_objects_dirty":111662,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":69508,"num_write_kb":278026,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1720,"num_bytes_recovered":7036947,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111735,"ondisk_log_size":111735,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1473588,"kb_used_data":900996,"kb_used_omap":12,"kb_used_meta":572531,"kb_avail":281641932,"statfs":{"total":289910292480,"available":288401338368,"internally_reserved":0,"allocated":922619904,"data_stored":921731501,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":586271746},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,4,3,5,6,10,7,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":25657344,"num_objects":6264,"num_object_clones":0,"num_object_copies":12528,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-341,"num_objects_unfound":0,"num_objects_dirty":6264,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6264,"num_write_kb":25056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":341,"num_bytes_recovered":1396736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003723"},"pg_stats":[{"pgid":"2.f","version":"43'6893","reported_seq":6963,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:31.480619+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:00:31.480619+0000","last_peered":"2024-09-10T16:00:31.480619+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:00:31.480619+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":6893,"log_dups_size":0,"ondisk_log_size":6893,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28233728,"num_objects":6893,"num_object_clones":0,"num_object_copies":13786,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6893,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6893,"num_write_kb":27572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2634},{"shards":"0,2","objects":4258}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'7015","reported_seq":7123,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:00:34.448346+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:00:34.448346+0000","last_peered":"2024-09-10T16:00:34.448346+0000","last_clean":"2024-09-10T16:00:34.448346+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:00:34.448346+0000","last_undegraded":"2024-09-10T16:00:34.448346+0000","last_fullsized":"2024-09-10T16:00:34.448346+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":7015,"log_dups_size":0,"ondisk_log_size":7015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28709830,"num_objects":7011,"num_object_clones":0,"num_object_copies":14022,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7011,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5120,"num_write_kb":20480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'7014","reported_seq":10569,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T16:00:34.483140+0000","last_change":"2024-09-10T15:59:16.142612+0000","last_active":"2024-09-10T16:00:34.483140+0000","last_peered":"2024-09-10T16:00:34.483140+0000","last_clean":"2024-09-10T15:59:14.092473+0000","last_became_active":"2024-09-10T15:59:16.109611+0000","last_became_peered":"2024-09-10T15:59:16.109611+0000","last_unstale":"2024-09-10T16:00:34.483140+0000","last_undegraded":"2024-09-10T16:00:34.483140+0000","last_fullsized":"2024-09-10T15:59:15.770264+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":7014,"log_dups_size":0,"ondisk_log_size":7014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28712960,"num_objects":7010,"num_object_clones":0,"num_object_copies":14020,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2553,"num_objects_unfound":0,"num_objects_dirty":7010,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5018,"num_write_kb":20072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1715,"num_bytes_recovered":7024640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2553},{"shards":"2","objects":4457}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'7029","reported_seq":7104,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:00:34.430693+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:00:34.430693+0000","last_peered":"2024-09-10T16:00:34.430693+0000","last_clean":"2024-09-10T16:00:34.430693+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:00:34.430693+0000","last_undegraded":"2024-09-10T16:00:34.430693+0000","last_fullsized":"2024-09-10T16:00:34.430693+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":7029,"log_dups_size":0,"ondisk_log_size":7029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28790784,"num_objects":7029,"num_object_clones":0,"num_object_copies":14058,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7029,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7029,"num_write_kb":28116,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'6976","reported_seq":7101,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:00:31.458666+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:00:31.458666+0000","last_peered":"2024-09-10T16:00:31.458666+0000","last_clean":"2024-09-10T16:00:31.458666+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:00:31.458666+0000","last_undegraded":"2024-09-10T16:00:31.458666+0000","last_fullsized":"2024-09-10T16:00:31.458666+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":6976,"log_dups_size":0,"ondisk_log_size":6976,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28545507,"num_objects":6970,"num_object_clones":0,"num_object_copies":13940,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6970,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3687,"num_write_kb":14748,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'7032","reported_seq":7165,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:31.487367+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:00:31.487367+0000","last_peered":"2024-09-10T16:00:31.487367+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:00:31.487367+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":7032,"log_dups_size":0,"ondisk_log_size":7032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28783075,"num_objects":7028,"num_object_clones":0,"num_object_copies":14056,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7028,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3644,"num_write_kb":14576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2657},{"shards":"0,2","objects":4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'6960","reported_seq":7093,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:31.479180+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:00:31.479180+0000","last_peered":"2024-09-10T16:00:31.479180+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:00:31.479180+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":6960,"log_dups_size":0,"ondisk_log_size":6960,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28484550,"num_objects":6956,"num_object_clones":0,"num_object_copies":13912,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6956,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3459,"num_write_kb":13836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2719},{"shards":"0,2","objects":4237}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'7082","reported_seq":7207,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:00:31.506798+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:00:31.506798+0000","last_peered":"2024-09-10T16:00:31.506798+0000","last_clean":"2024-09-10T16:00:31.506798+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:00:31.506798+0000","last_undegraded":"2024-09-10T16:00:31.506798+0000","last_fullsized":"2024-09-10T16:00:31.506798+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":7082,"log_dups_size":0,"ondisk_log_size":7082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28999680,"num_objects":7080,"num_object_clones":0,"num_object_copies":14160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7080,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3929,"num_write_kb":15716,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.775865+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T15:59:15.775865+0000","last_peered":"2024-09-10T15:59:15.775865+0000","last_clean":"2024-09-10T15:59:15.775865+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T15:59:15.775865+0000","last_undegraded":"2024-09-10T15:59:15.775865+0000","last_fullsized":"2024-09-10T15:59:15.775865+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'6767","reported_seq":6902,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:34.473041+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:00:34.473041+0000","last_peered":"2024-09-10T16:00:34.473041+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:00:34.473041+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":6767,"log_dups_size":0,"ondisk_log_size":6767,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27689443,"num_objects":6761,"num_object_clones":0,"num_object_copies":13522,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6761,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4586,"num_write_kb":18344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4123},{"shards":"2","objects":2638}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":89,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.772619+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T15:59:15.772619+0000","last_peered":"2024-09-10T15:59:15.772619+0000","last_clean":"2024-09-10T15:59:15.772619+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T15:59:15.772619+0000","last_undegraded":"2024-09-10T15:59:15.772619+0000","last_fullsized":"2024-09-10T15:59:15.772619+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'6915","reported_seq":7040,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:31.453079+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:00:31.453079+0000","last_peered":"2024-09-10T16:00:31.453079+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:00:31.453079+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":6915,"log_dups_size":0,"ondisk_log_size":6915,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28291072,"num_objects":6907,"num_object_clones":0,"num_object_copies":13814,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6907,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3419,"num_write_kb":13676,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2652},{"shards":"0,2","objects":4255}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":99,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.884297+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T15:59:15.884297+0000","last_peered":"2024-09-10T15:59:15.884297+0000","last_clean":"2024-09-10T15:59:15.884297+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T15:59:15.884297+0000","last_undegraded":"2024-09-10T15:59:15.884297+0000","last_fullsized":"2024-09-10T15:59:15.884297+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'7054","reported_seq":7180,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:34.391776+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:00:34.391776+0000","last_peered":"2024-09-10T16:00:34.391776+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:00:34.391776+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":7054,"log_dups_size":0,"ondisk_log_size":7054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28873187,"num_objects":7050,"num_object_clones":0,"num_object_copies":14100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7050,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4734,"num_write_kb":18936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4304},{"shards":"2","objects":2746}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.837509+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T15:59:15.837509+0000","last_peered":"2024-09-10T15:59:15.837509+0000","last_clean":"2024-09-10T15:59:15.837509+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T15:59:15.837509+0000","last_undegraded":"2024-09-10T15:59:15.837509+0000","last_fullsized":"2024-09-10T15:59:15.837509+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'7163","reported_seq":7284,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:34.453014+0000","last_change":"2024-09-10T15:59:16.174113+0000","last_active":"2024-09-10T16:00:34.453014+0000","last_peered":"2024-09-10T16:00:34.453014+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:00:34.453014+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":7163,"log_dups_size":0,"ondisk_log_size":7163,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29315072,"num_objects":7157,"num_object_clones":0,"num_object_copies":14314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4354,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7157,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4717,"num_write_kb":18868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4352},{"shards":"2","objects":2805}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'6932","reported_seq":7062,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:31.481649+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:00:31.481649+0000","last_peered":"2024-09-10T16:00:31.481649+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:00:31.481649+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":6932,"log_dups_size":0,"ondisk_log_size":6932,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28385280,"num_objects":6930,"num_object_clones":0,"num_object_copies":13860,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6930,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3563,"num_write_kb":14252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2626},{"shards":"0,2","objects":4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.989413+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T15:59:15.989413+0000","last_peered":"2024-09-10T15:59:15.989413+0000","last_clean":"2024-09-10T15:59:15.989413+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T15:59:15.989413+0000","last_undegraded":"2024-09-10T15:59:15.989413+0000","last_fullsized":"2024-09-10T15:59:15.989413+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'6786","reported_seq":6924,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:31.482968+0000","last_change":"2024-09-10T15:59:16.251610+0000","last_active":"2024-09-10T16:00:31.482968+0000","last_peered":"2024-09-10T16:00:31.482968+0000","last_clean":"2024-09-10T15:59:14.115595+0000","last_became_active":"2024-09-10T15:59:16.244089+0000","last_became_peered":"2024-09-10T15:59:16.244089+0000","last_unstale":"2024-09-10T16:00:31.482968+0000","last_undegraded":"2024-09-10T15:59:15.847127+0000","last_fullsized":"2024-09-10T15:59:15.774437+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":6786,"log_dups_size":0,"ondisk_log_size":6786,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27762688,"num_objects":6778,"num_object_clones":0,"num_object_copies":13556,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6778,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3176,"num_write_kb":12704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2575},{"shards":"0,2","objects":4203}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":89,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.773633+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T15:59:15.773633+0000","last_peered":"2024-09-10T15:59:15.773633+0000","last_clean":"2024-09-10T15:59:15.773633+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T15:59:15.773633+0000","last_undegraded":"2024-09-10T15:59:15.773633+0000","last_fullsized":"2024-09-10T15:59:15.773633+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'7062","reported_seq":7199,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:31.499745+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:00:31.499745+0000","last_peered":"2024-09-10T16:00:31.499745+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:00:31.499745+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":7062,"log_dups_size":0,"ondisk_log_size":7062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28909568,"num_objects":7058,"num_object_clones":0,"num_object_copies":14116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7058,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3317,"num_write_kb":13268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2687},{"shards":"0,2","objects":4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.916017+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T15:59:15.916017+0000","last_peered":"2024-09-10T15:59:15.916017+0000","last_clean":"2024-09-10T15:59:15.916017+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T15:59:15.916017+0000","last_undegraded":"2024-09-10T15:59:15.916017+0000","last_fullsized":"2024-09-10T15:59:15.916017+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.776033+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T15:59:15.776033+0000","last_peered":"2024-09-10T15:59:15.776033+0000","last_clean":"2024-09-10T15:59:15.776033+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T15:59:15.776033+0000","last_undegraded":"2024-09-10T15:59:15.776033+0000","last_fullsized":"2024-09-10T15:59:15.776033+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'7052","reported_seq":7186,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:31.514032+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:00:31.514032+0000","last_peered":"2024-09-10T16:00:31.514032+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:00:31.514032+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":7052,"log_dups_size":0,"ondisk_log_size":7052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28836806,"num_objects":7042,"num_object_clones":0,"num_object_copies":14084,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7042,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3215,"num_write_kb":12860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2719},{"shards":"0,2","objects":4323}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":457323230,"num_objects":111660,"num_object_clones":0,"num_object_copies":223320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":47117,"num_objects_misplaced":2553,"num_objects_unfound":0,"num_objects_dirty":111660,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":69506,"num_write_kb":278024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1718,"num_bytes_recovered":7036928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":923836416,"data_stored":923681057,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111732,"ondisk_log_size":111732,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986207,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":537904,"kb_used_data":361824,"kb_used_omap":4,"kb_used_meta":176059,"kb_avail":93833936,"statfs":{"total":96636764160,"available":96085950464,"internally_reserved":0,"allocated":370507776,"data_stored":370230016,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":180285098},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,3,2,1,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659153,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":263244,"kb_used_data":137148,"kb_used_omap":4,"kb_used_meta":126075,"kb_avail":94108596,"statfs":{"total":96636764160,"available":96367202304,"internally_reserved":0,"allocated":140439552,"data_stored":140110746,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":129101486},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,4,0,0,3,3,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182435,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":672440,"kb_used_data":402024,"kb_used_omap":4,"kb_used_meta":270395,"kb_avail":93699400,"statfs":{"total":96636764160,"available":95948185600,"internally_reserved":0,"allocated":411672576,"data_stored":411390739,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":276885162},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,4,0,5,5,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":411906048,"data_stored":411869918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":140939264,"data_stored":140856165,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":370991104,"data_stored":370954974,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:00:37.101 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-10T16:00:37.101 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "43'6893", 'reported_seq': 6963, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:00:31.480619+0000', 'last_change': '2024-09-10T15:59:16.281558+0000', 'last_active': '2024-09-10T16:00:31.480619+0000', 'last_peered': '2024-09-10T16:00:31.480619+0000', 'last_clean': '2024-09-10T15:59:14.132456+0000', 'last_became_active': '2024-09-10T15:59:16.274407+0000', 'last_became_peered': '2024-09-10T15:59:16.274407+0000', 'last_unstale': '2024-09-10T16:00:31.480619+0000', 'last_undegraded': '2024-09-10T15:59:15.851279+0000', 'last_fullsized': '2024-09-10T15:59:15.774356+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'objects_scrubbed': 0, 'log_size': 6893, 'log_dups_size': 0, 'ondisk_log_size': 6893, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28233728, 'num_objects': 6893, 'num_object_clones': 0, 'num_object_copies': 13786, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4259, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6893, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6893, 'num_write_kb': 27572, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2634}, {'shards': '0,2', 'objects': 4258}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:00:37.102 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-10T16:00:37.102 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "43'7014", 'reported_seq': 10569, 'reported_epoch': 43, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-10T16:00:34.483140+0000', 'last_change': '2024-09-10T15:59:16.142612+0000', 'last_active': '2024-09-10T16:00:34.483140+0000', 'last_peered': '2024-09-10T16:00:34.483140+0000', 'last_clean': '2024-09-10T15:59:14.092473+0000', 'last_became_active': '2024-09-10T15:59:16.109611+0000', 'last_became_peered': '2024-09-10T15:59:16.109611+0000', 'last_unstale': '2024-09-10T16:00:34.483140+0000', 'last_undegraded': '2024-09-10T16:00:34.483140+0000', 'last_fullsized': '2024-09-10T15:59:15.770264+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2029", 'last_scrub_stamp': '2024-09-10T15:58:45.933789+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:45.933789+0000', 'objects_scrubbed': 1992, 'log_size': 7014, 'log_dups_size': 0, 'ondisk_log_size': 7014, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 668, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28712960, 'num_objects': 7010, 'num_object_clones': 0, 'num_object_copies': 14020, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 2553, 'num_objects_unfound': 0, 'num_objects_dirty': 7010, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5018, 'num_write_kb': 20072, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1715, 'num_bytes_recovered': 7024640, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2553}, {'shards': '2', 'objects': 4457}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:00:37.102 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-10T16:00:37.102 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "43'7032", 'reported_seq': 7165, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:00:31.487367+0000', 'last_change': '2024-09-10T15:59:16.308264+0000', 'last_active': '2024-09-10T16:00:31.487367+0000', 'last_peered': '2024-09-10T16:00:31.487367+0000', 'last_clean': '2024-09-10T15:59:14.111551+0000', 'last_became_active': '2024-09-10T15:59:16.300040+0000', 'last_became_peered': '2024-09-10T15:59:16.300040+0000', 'last_unstale': '2024-09-10T16:00:31.487367+0000', 'last_undegraded': '2024-09-10T15:59:15.878234+0000', 'last_fullsized': '2024-09-10T15:59:15.769328+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3442", 'last_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'objects_scrubbed': 3384, 'log_size': 7032, 'log_dups_size': 0, 'ondisk_log_size': 7032, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1292, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28783075, 'num_objects': 7028, 'num_object_clones': 0, 'num_object_copies': 14056, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4372, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7028, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 3644, 'num_write_kb': 14576, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2657}, {'shards': '0,2', 'objects': 4371}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:00:37.102 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-10T16:00:37.102 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "43'6960", 'reported_seq': 7093, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:00:31.479180+0000', 'last_change': '2024-09-10T15:59:16.331899+0000', 'last_active': '2024-09-10T16:00:31.479180+0000', 'last_peered': '2024-09-10T16:00:31.479180+0000', 'last_clean': '2024-09-10T15:59:14.139436+0000', 'last_became_active': '2024-09-10T15:59:16.318715+0000', 'last_became_peered': '2024-09-10T15:59:16.318715+0000', 'last_unstale': '2024-09-10T16:00:31.479180+0000', 'last_undegraded': '2024-09-10T15:59:15.877524+0000', 'last_fullsized': '2024-09-10T15:59:15.769042+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3537", 'last_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'objects_scrubbed': 3497, 'log_size': 6960, 'log_dups_size': 0, 'ondisk_log_size': 6960, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1224, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28484550, 'num_objects': 6956, 'num_object_clones': 0, 'num_object_copies': 13912, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4237, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6956, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 3459, 'num_write_kb': 13836, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2719}, {'shards': '0,2', 'objects': 4237}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:00:37.103 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-10T16:00:37.103 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "43'6767", 'reported_seq': 6902, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:00:34.473041+0000', 'last_change': '2024-09-10T15:59:16.175680+0000', 'last_active': '2024-09-10T16:00:34.473041+0000', 'last_peered': '2024-09-10T16:00:34.473041+0000', 'last_clean': '2024-09-10T15:59:14.088187+0000', 'last_became_active': '2024-09-10T15:59:16.165810+0000', 'last_became_peered': '2024-09-10T15:59:16.165810+0000', 'last_unstale': '2024-09-10T16:00:34.473041+0000', 'last_undegraded': '2024-09-10T15:59:15.822098+0000', 'last_fullsized': '2024-09-10T15:59:15.769123+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2218", 'last_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'objects_scrubbed': 2175, 'log_size': 6767, 'log_dups_size': 0, 'ondisk_log_size': 6767, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 825, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27689443, 'num_objects': 6761, 'num_object_clones': 0, 'num_object_copies': 13522, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4124, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6761, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 4586, 'num_write_kb': 18344, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4123}, {'shards': '2', 'objects': 2638}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:00:37.103 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-10T16:00:37.103 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "43'6915", 'reported_seq': 7040, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:00:31.453079+0000', 'last_change': '2024-09-10T15:59:16.277742+0000', 'last_active': '2024-09-10T16:00:31.453079+0000', 'last_peered': '2024-09-10T16:00:31.453079+0000', 'last_clean': '2024-09-10T15:59:14.129081+0000', 'last_became_active': '2024-09-10T15:59:16.269835+0000', 'last_became_peered': '2024-09-10T15:59:16.269835+0000', 'last_unstale': '2024-09-10T16:00:31.453079+0000', 'last_undegraded': '2024-09-10T15:59:15.860857+0000', 'last_fullsized': '2024-09-10T15:59:15.771410+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3557", 'last_scrub_stamp': '2024-09-10T15:59:02.493678+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:02.493678+0000', 'objects_scrubbed': 3488, 'log_size': 6915, 'log_dups_size': 0, 'ondisk_log_size': 6915, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1472, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28291072, 'num_objects': 6907, 'num_object_clones': 0, 'num_object_copies': 13814, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4257, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6907, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3419, 'num_write_kb': 13676, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2652}, {'shards': '0,2', 'objects': 4255}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:00:37.103 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-10T16:00:37.104 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "43'7054", 'reported_seq': 7180, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:00:34.391776+0000', 'last_change': '2024-09-10T15:59:16.284311+0000', 'last_active': '2024-09-10T16:00:34.391776+0000', 'last_peered': '2024-09-10T16:00:34.391776+0000', 'last_clean': '2024-09-10T15:59:14.116041+0000', 'last_became_active': '2024-09-10T15:59:16.274543+0000', 'last_became_peered': '2024-09-10T15:59:16.274543+0000', 'last_unstale': '2024-09-10T16:00:34.391776+0000', 'last_undegraded': '2024-09-10T15:59:15.812484+0000', 'last_fullsized': '2024-09-10T15:59:15.770885+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2356", 'last_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'objects_scrubbed': 2316, 'log_size': 7054, 'log_dups_size': 0, 'ondisk_log_size': 7054, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 783, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28873187, 'num_objects': 7050, 'num_object_clones': 0, 'num_object_copies': 14100, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7050, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 4734, 'num_write_kb': 18936, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4304}, {'shards': '2', 'objects': 2746}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:00:37.104 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-10T16:00:37.104 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "43'7163", 'reported_seq': 7284, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:00:34.453014+0000', 'last_change': '2024-09-10T15:59:16.174113+0000', 'last_active': '2024-09-10T16:00:34.453014+0000', 'last_peered': '2024-09-10T16:00:34.453014+0000', 'last_clean': '2024-09-10T15:59:14.110654+0000', 'last_became_active': '2024-09-10T15:59:16.164645+0000', 'last_became_peered': '2024-09-10T15:59:16.164645+0000', 'last_unstale': '2024-09-10T16:00:34.453014+0000', 'last_undegraded': '2024-09-10T15:59:15.803593+0000', 'last_fullsized': '2024-09-10T15:59:15.769083+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2486", 'last_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'objects_scrubbed': 2440, 'log_size': 7163, 'log_dups_size': 0, 'ondisk_log_size': 7163, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 792, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29315072, 'num_objects': 7157, 'num_object_clones': 0, 'num_object_copies': 14314, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4354, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7157, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4717, 'num_write_kb': 18868, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4352}, {'shards': '2', 'objects': 2805}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:00:37.104 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-10T16:00:37.105 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "43'6932", 'reported_seq': 7062, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:00:31.481649+0000', 'last_change': '2024-09-10T15:59:16.315641+0000', 'last_active': '2024-09-10T16:00:31.481649+0000', 'last_peered': '2024-09-10T16:00:31.481649+0000', 'last_clean': '2024-09-10T15:59:14.117001+0000', 'last_became_active': '2024-09-10T15:59:16.302245+0000', 'last_became_peered': '2024-09-10T15:59:16.302245+0000', 'last_unstale': '2024-09-10T16:00:31.481649+0000', 'last_undegraded': '2024-09-10T15:59:15.845012+0000', 'last_fullsized': '2024-09-10T15:59:15.770337+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3404", 'last_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'objects_scrubbed': 3367, 'log_size': 6932, 'log_dups_size': 0, 'ondisk_log_size': 6932, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 983, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28385280, 'num_objects': 6930, 'num_object_clones': 0, 'num_object_copies': 13860, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6930, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3563, 'num_write_kb': 14252, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2626}, {'shards': '0,2', 'objects': 4304}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:00:37.105 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-10T16:00:37.105 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "43'6786", 'reported_seq': 6924, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:00:31.482968+0000', 'last_change': '2024-09-10T15:59:16.251610+0000', 'last_active': '2024-09-10T16:00:31.482968+0000', 'last_peered': '2024-09-10T16:00:31.482968+0000', 'last_clean': '2024-09-10T15:59:14.115595+0000', 'last_became_active': '2024-09-10T15:59:16.244089+0000', 'last_became_peered': '2024-09-10T15:59:16.244089+0000', 'last_unstale': '2024-09-10T16:00:31.482968+0000', 'last_undegraded': '2024-09-10T15:59:15.847127+0000', 'last_fullsized': '2024-09-10T15:59:15.774437+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3653", 'last_scrub_stamp': '2024-09-10T15:59:05.298468+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:05.298468+0000', 'objects_scrubbed': 3602, 'log_size': 6786, 'log_dups_size': 0, 'ondisk_log_size': 6786, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1289, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27762688, 'num_objects': 6778, 'num_object_clones': 0, 'num_object_copies': 13556, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4206, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6778, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3176, 'num_write_kb': 12704, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2575}, {'shards': '0,2', 'objects': 4203}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:00:37.105 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-10T16:00:37.105 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "43'7062", 'reported_seq': 7199, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:00:31.499745+0000', 'last_change': '2024-09-10T15:59:16.311291+0000', 'last_active': '2024-09-10T16:00:31.499745+0000', 'last_peered': '2024-09-10T16:00:31.499745+0000', 'last_clean': '2024-09-10T15:59:14.116363+0000', 'last_became_active': '2024-09-10T15:59:16.301244+0000', 'last_became_peered': '2024-09-10T15:59:16.301244+0000', 'last_unstale': '2024-09-10T16:00:31.499745+0000', 'last_undegraded': '2024-09-10T15:59:15.883434+0000', 'last_fullsized': '2024-09-10T15:59:15.772756+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3789", 'last_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'objects_scrubbed': 3741, 'log_size': 7062, 'log_dups_size': 0, 'ondisk_log_size': 7062, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1440, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28909568, 'num_objects': 7058, 'num_object_clones': 0, 'num_object_copies': 14116, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4373, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7058, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3317, 'num_write_kb': 13268, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2687}, {'shards': '0,2', 'objects': 4371}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:00:37.106 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-10T16:00:37.106 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "43'7052", 'reported_seq': 7186, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:00:31.514032+0000', 'last_change': '2024-09-10T15:59:16.313409+0000', 'last_active': '2024-09-10T16:00:31.514032+0000', 'last_peered': '2024-09-10T16:00:31.514032+0000', 'last_clean': '2024-09-10T15:59:14.129780+0000', 'last_became_active': '2024-09-10T15:59:16.300932+0000', 'last_became_peered': '2024-09-10T15:59:16.300932+0000', 'last_unstale': '2024-09-10T16:00:31.514032+0000', 'last_undegraded': '2024-09-10T15:59:15.887823+0000', 'last_fullsized': '2024-09-10T15:59:15.768513+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3880", 'last_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'objects_scrubbed': 3827, 'log_size': 7052, 'log_dups_size': 0, 'ondisk_log_size': 7052, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1397, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28836806, 'num_objects': 7042, 'num_object_clones': 0, 'num_object_copies': 14084, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4325, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7042, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3215, 'num_write_kb': 12860, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2719}, {'shards': '0,2', 'objects': 4323}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:00:37.107 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T16:00:37.784 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:00:37.784 INFO:teuthology.orchestra.run.smithi115.stdout:{"fsid":"7effee29-df05-4351-9741-ee203564ed81","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 47117/223324 objects degraded (21.098%), 11 pgs degraded, 12 pgs undersized","count":23},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":201,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":43,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983954,"num_in_osds":3,"osd_in_since":1725983947,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":111662,"data_bytes":457323249,"bytes_used":1508954112,"bytes_avail":288401338368,"bytes_total":289910292480,"degraded_objects":47117,"degraded_total":223324,"degraded_ratio":0.21098045888484893,"misplaced_objects":2553,"misplaced_total":223324,"misplaced_ratio":0.011431821031326683,"recovering_objects_per_sec":28,"recovering_bytes_per_sec":116358,"recovering_keys_per_sec":0,"num_objects_recovered":341,"num_bytes_recovered":1396736,"num_keys_recovered":0,"write_bytes_sec":2137448,"read_op_per_sec":0,"write_op_per_sec":521},"fsmap":{"epoch":1,"btime":"2024-09-10T15:57:14:967120+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T16:00:13.560776+0000","services":{}},"progress_events":{}} 2024-09-10T16:00:37.819 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-10T16:00:37.819 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:00:37.981 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 125 20 114394 114374 3.57359 1.85938 0.0533161 0.0218548 2024-09-10T16:00:38.392 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:00:38.393 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:00:38.414 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":137,"stamp":"2024-09-10T16:00:37.568939+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":464093937,"num_objects":113315,"num_object_clones":0,"num_object_copies":226630,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":47117,"num_objects_misplaced":2553,"num_objects_unfound":0,"num_objects_dirty":113315,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":71161,"num_write_kb":284638,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1720,"num_bytes_recovered":7036947,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":113388,"ondisk_log_size":113388,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1485584,"kb_used_data":908832,"kb_used_omap":12,"kb_used_meta":576691,"kb_avail":281629936,"statfs":{"total":289910292480,"available":288389054464,"internally_reserved":0,"allocated":930643968,"data_stored":929755565,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":590531586},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,4,2,7,8,9,5,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":28377088,"num_objects":6928,"num_object_clones":0,"num_object_copies":13856,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-227,"num_objects_unfound":0,"num_objects_dirty":6928,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6928,"num_write_kb":27712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":227,"num_bytes_recovered":929792,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003730"},"pg_stats":[{"pgid":"2.f","version":"43'7051","reported_seq":7121,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:36.412075+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:00:36.412075+0000","last_peered":"2024-09-10T16:00:36.412075+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:00:36.412075+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":7051,"log_dups_size":0,"ondisk_log_size":7051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28880896,"num_objects":7051,"num_object_clones":0,"num_object_copies":14102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7051,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7051,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2792},{"shards":"0,2","objects":4258}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'7015","reported_seq":7123,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:00:34.448346+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:00:34.448346+0000","last_peered":"2024-09-10T16:00:34.448346+0000","last_clean":"2024-09-10T16:00:34.448346+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:00:34.448346+0000","last_undegraded":"2024-09-10T16:00:34.448346+0000","last_fullsized":"2024-09-10T16:00:34.448346+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":7015,"log_dups_size":0,"ondisk_log_size":7015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28709830,"num_objects":7011,"num_object_clones":0,"num_object_copies":14022,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7011,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5120,"num_write_kb":20480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'7014","reported_seq":10569,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T16:00:34.483140+0000","last_change":"2024-09-10T15:59:16.142612+0000","last_active":"2024-09-10T16:00:34.483140+0000","last_peered":"2024-09-10T16:00:34.483140+0000","last_clean":"2024-09-10T15:59:14.092473+0000","last_became_active":"2024-09-10T15:59:16.109611+0000","last_became_peered":"2024-09-10T15:59:16.109611+0000","last_unstale":"2024-09-10T16:00:34.483140+0000","last_undegraded":"2024-09-10T16:00:34.483140+0000","last_fullsized":"2024-09-10T15:59:15.770264+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":7014,"log_dups_size":0,"ondisk_log_size":7014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28712960,"num_objects":7010,"num_object_clones":0,"num_object_copies":14020,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2553,"num_objects_unfound":0,"num_objects_dirty":7010,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5018,"num_write_kb":20072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1715,"num_bytes_recovered":7024640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2553},{"shards":"2","objects":4457}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'7029","reported_seq":7104,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:00:34.430693+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:00:34.430693+0000","last_peered":"2024-09-10T16:00:34.430693+0000","last_clean":"2024-09-10T16:00:34.430693+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:00:34.430693+0000","last_undegraded":"2024-09-10T16:00:34.430693+0000","last_fullsized":"2024-09-10T16:00:34.430693+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":7029,"log_dups_size":0,"ondisk_log_size":7029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28790784,"num_objects":7029,"num_object_clones":0,"num_object_copies":14058,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7029,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7029,"num_write_kb":28116,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'7144","reported_seq":7268,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:00:36.601257+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:00:36.601257+0000","last_peered":"2024-09-10T16:00:36.601257+0000","last_clean":"2024-09-10T16:00:36.601257+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:00:36.601257+0000","last_undegraded":"2024-09-10T16:00:36.601257+0000","last_fullsized":"2024-09-10T16:00:36.601257+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":7144,"log_dups_size":0,"ondisk_log_size":7144,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29233635,"num_objects":7138,"num_object_clones":0,"num_object_copies":14276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7138,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3855,"num_write_kb":15420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'7192","reported_seq":7325,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:36.468046+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:00:36.468046+0000","last_peered":"2024-09-10T16:00:36.468046+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:00:36.468046+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":7192,"log_dups_size":0,"ondisk_log_size":7192,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29438435,"num_objects":7188,"num_object_clones":0,"num_object_copies":14376,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7188,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3804,"num_write_kb":15216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2817},{"shards":"0,2","objects":4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'7134","reported_seq":7267,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:36.578040+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:00:36.578040+0000","last_peered":"2024-09-10T16:00:36.578040+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:00:36.578040+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":7134,"log_dups_size":0,"ondisk_log_size":7134,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29197254,"num_objects":7130,"num_object_clones":0,"num_object_copies":14260,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7130,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3633,"num_write_kb":14532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2893},{"shards":"0,2","objects":4237}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'7253","reported_seq":7378,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:00:36.597105+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:00:36.597105+0000","last_peered":"2024-09-10T16:00:36.597105+0000","last_clean":"2024-09-10T16:00:36.597105+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:00:36.597105+0000","last_undegraded":"2024-09-10T16:00:36.597105+0000","last_fullsized":"2024-09-10T16:00:36.597105+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":7253,"log_dups_size":0,"ondisk_log_size":7253,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29700096,"num_objects":7251,"num_object_clones":0,"num_object_copies":14502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7251,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4100,"num_write_kb":16400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.775865+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T15:59:15.775865+0000","last_peered":"2024-09-10T15:59:15.775865+0000","last_clean":"2024-09-10T15:59:15.775865+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T15:59:15.775865+0000","last_undegraded":"2024-09-10T15:59:15.775865+0000","last_fullsized":"2024-09-10T15:59:15.775865+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'6767","reported_seq":6902,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:34.473041+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:00:34.473041+0000","last_peered":"2024-09-10T16:00:34.473041+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:00:34.473041+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":6767,"log_dups_size":0,"ondisk_log_size":6767,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27689443,"num_objects":6761,"num_object_clones":0,"num_object_copies":13522,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6761,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4586,"num_write_kb":18344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4123},{"shards":"2","objects":2638}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":89,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.772619+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T15:59:15.772619+0000","last_peered":"2024-09-10T15:59:15.772619+0000","last_clean":"2024-09-10T15:59:15.772619+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T15:59:15.772619+0000","last_undegraded":"2024-09-10T15:59:15.772619+0000","last_fullsized":"2024-09-10T15:59:15.772619+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'7081","reported_seq":7206,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:36.470618+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:00:36.470618+0000","last_peered":"2024-09-10T16:00:36.470618+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:00:36.470618+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":7081,"log_dups_size":0,"ondisk_log_size":7081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28971008,"num_objects":7073,"num_object_clones":0,"num_object_copies":14146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7073,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3585,"num_write_kb":14340,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2818},{"shards":"0,2","objects":4255}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":99,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.884297+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T15:59:15.884297+0000","last_peered":"2024-09-10T15:59:15.884297+0000","last_clean":"2024-09-10T15:59:15.884297+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T15:59:15.884297+0000","last_undegraded":"2024-09-10T15:59:15.884297+0000","last_fullsized":"2024-09-10T15:59:15.884297+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'7054","reported_seq":7180,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:34.391776+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:00:34.391776+0000","last_peered":"2024-09-10T16:00:34.391776+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:00:34.391776+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":7054,"log_dups_size":0,"ondisk_log_size":7054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28873187,"num_objects":7050,"num_object_clones":0,"num_object_copies":14100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7050,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4734,"num_write_kb":18936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4304},{"shards":"2","objects":2746}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.837509+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T15:59:15.837509+0000","last_peered":"2024-09-10T15:59:15.837509+0000","last_clean":"2024-09-10T15:59:15.837509+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T15:59:15.837509+0000","last_undegraded":"2024-09-10T15:59:15.837509+0000","last_fullsized":"2024-09-10T15:59:15.837509+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'7163","reported_seq":7284,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:34.453014+0000","last_change":"2024-09-10T15:59:16.174113+0000","last_active":"2024-09-10T16:00:34.453014+0000","last_peered":"2024-09-10T16:00:34.453014+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:00:34.453014+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":7163,"log_dups_size":0,"ondisk_log_size":7163,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29315072,"num_objects":7157,"num_object_clones":0,"num_object_copies":14314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4354,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7157,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4717,"num_write_kb":18868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4352},{"shards":"2","objects":2805}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'7099","reported_seq":7224,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:36.573739+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:00:36.573739+0000","last_peered":"2024-09-10T16:00:36.573739+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:00:36.573739+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":7099,"log_dups_size":0,"ondisk_log_size":7099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29069312,"num_objects":7097,"num_object_clones":0,"num_object_copies":14194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7097,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3730,"num_write_kb":14920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2793},{"shards":"0,2","objects":4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.989413+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T15:59:15.989413+0000","last_peered":"2024-09-10T15:59:15.989413+0000","last_clean":"2024-09-10T15:59:15.989413+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T15:59:15.989413+0000","last_undegraded":"2024-09-10T15:59:15.989413+0000","last_fullsized":"2024-09-10T15:59:15.989413+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'6938","reported_seq":7076,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:36.537234+0000","last_change":"2024-09-10T15:59:16.251610+0000","last_active":"2024-09-10T16:00:36.537234+0000","last_peered":"2024-09-10T16:00:36.537234+0000","last_clean":"2024-09-10T15:59:14.115595+0000","last_became_active":"2024-09-10T15:59:16.244089+0000","last_became_peered":"2024-09-10T15:59:16.244089+0000","last_unstale":"2024-09-10T16:00:36.537234+0000","last_undegraded":"2024-09-10T15:59:15.847127+0000","last_fullsized":"2024-09-10T15:59:15.774437+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":6938,"log_dups_size":0,"ondisk_log_size":6938,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28385280,"num_objects":6930,"num_object_clones":0,"num_object_copies":13860,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6930,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3328,"num_write_kb":13312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2727},{"shards":"0,2","objects":4203}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":89,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.773633+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T15:59:15.773633+0000","last_peered":"2024-09-10T15:59:15.773633+0000","last_clean":"2024-09-10T15:59:15.773633+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T15:59:15.773633+0000","last_undegraded":"2024-09-10T15:59:15.773633+0000","last_fullsized":"2024-09-10T15:59:15.773633+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'7241","reported_seq":7378,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:36.544380+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:00:36.544380+0000","last_peered":"2024-09-10T16:00:36.544380+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:00:36.544380+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":7241,"log_dups_size":0,"ondisk_log_size":7241,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29642752,"num_objects":7237,"num_object_clones":0,"num_object_copies":14474,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7237,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3496,"num_write_kb":13984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2866},{"shards":"0,2","objects":4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.916017+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T15:59:15.916017+0000","last_peered":"2024-09-10T15:59:15.916017+0000","last_clean":"2024-09-10T15:59:15.916017+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T15:59:15.916017+0000","last_undegraded":"2024-09-10T15:59:15.916017+0000","last_fullsized":"2024-09-10T15:59:15.916017+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.776033+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T15:59:15.776033+0000","last_peered":"2024-09-10T15:59:15.776033+0000","last_clean":"2024-09-10T15:59:15.776033+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T15:59:15.776033+0000","last_undegraded":"2024-09-10T15:59:15.776033+0000","last_fullsized":"2024-09-10T15:59:15.776033+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'7210","reported_seq":7349,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:36.557988+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:00:36.557988+0000","last_peered":"2024-09-10T16:00:36.557988+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:00:36.557988+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":7210,"log_dups_size":0,"ondisk_log_size":7210,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29483974,"num_objects":7200,"num_object_clones":0,"num_object_copies":14400,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7200,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3373,"num_write_kb":13492,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2877},{"shards":"0,2","objects":4323}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":464093918,"num_objects":113313,"num_object_clones":0,"num_object_copies":226626,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":47117,"num_objects_misplaced":2553,"num_objects_unfound":0,"num_objects_dirty":113313,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":71159,"num_write_kb":284636,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1718,"num_bytes_recovered":7036928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":931893248,"data_stored":931737889,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":113385,"ondisk_log_size":113385,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986207,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":537904,"kb_used_data":361824,"kb_used_omap":4,"kb_used_meta":176059,"kb_avail":93833936,"statfs":{"total":96636764160,"available":96085950464,"internally_reserved":0,"allocated":370507776,"data_stored":370230016,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":180285098},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,3,2,1,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659153,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":263244,"kb_used_data":137148,"kb_used_omap":4,"kb_used_meta":126075,"kb_avail":94108596,"statfs":{"total":96636764160,"available":96367202304,"internally_reserved":0,"allocated":140439552,"data_stored":140110746,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":129101486},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,4,0,0,3,3,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182436,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":684436,"kb_used_data":409860,"kb_used_omap":4,"kb_used_meta":274555,"kb_avail":93687404,"statfs":{"total":96636764160,"available":95935901696,"internally_reserved":0,"allocated":419696640,"data_stored":419414803,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":281145002},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,6,2,4,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":419962880,"data_stored":419926750,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":140939264,"data_stored":140856165,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":370991104,"data_stored":370954974,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:00:38.981 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 126 20 114909 114889 3.5612 2.01172 0.0271731 0.0219306 2024-09-10T16:00:39.981 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 127 20 115458 115438 3.55004 2.14453 0.0799442 0.0219992 2024-09-10T16:00:40.981 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 128 20 116000 115980 3.53885 2.11719 0.0144003 0.0220683 2024-09-10T16:00:41.416 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:00:41.982 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 129 20 116496 116476 3.52643 1.9375 0.0324358 0.0221467 2024-09-10T16:00:41.987 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:00:41.987 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:00:42.013 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":139,"stamp":"2024-09-10T16:00:41.570389+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":468026097,"num_objects":114275,"num_object_clones":0,"num_object_copies":228550,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":47117,"num_objects_misplaced":2434,"num_objects_unfound":0,"num_objects_dirty":114275,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":72121,"num_write_kb":288478,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1839,"num_bytes_recovered":7524371,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":114348,"ondisk_log_size":114348,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1525256,"kb_used_data":922136,"kb_used_omap":12,"kb_used_meta":603059,"kb_avail":281590264,"statfs":{"total":289910292480,"available":288348430336,"internally_reserved":0,"allocated":944267264,"data_stored":943378861,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":617532418},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,2,7,9,5,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":21381120,"num_objects":5220,"num_object_clones":0,"num_object_copies":10440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-238,"num_objects_unfound":0,"num_objects_dirty":5220,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5220,"num_write_kb":20880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":238,"num_bytes_recovered":974848,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004015"},"pg_stats":[{"pgid":"2.f","version":"43'7051","reported_seq":7121,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:36.412075+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:00:36.412075+0000","last_peered":"2024-09-10T16:00:36.412075+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:00:36.412075+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":7051,"log_dups_size":0,"ondisk_log_size":7051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28880896,"num_objects":7051,"num_object_clones":0,"num_object_copies":14102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7051,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7051,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2792},{"shards":"0,2","objects":4258}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'7171","reported_seq":7279,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:00:39.459351+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:00:39.459351+0000","last_peered":"2024-09-10T16:00:39.459351+0000","last_clean":"2024-09-10T16:00:39.459351+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:00:39.459351+0000","last_undegraded":"2024-09-10T16:00:39.459351+0000","last_fullsized":"2024-09-10T16:00:39.459351+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":7171,"log_dups_size":0,"ondisk_log_size":7171,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29348806,"num_objects":7167,"num_object_clones":0,"num_object_copies":14334,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7167,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5276,"num_write_kb":21104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'7173","reported_seq":10965,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T16:00:39.485822+0000","last_change":"2024-09-10T15:59:16.142612+0000","last_active":"2024-09-10T16:00:39.485822+0000","last_peered":"2024-09-10T16:00:39.485822+0000","last_clean":"2024-09-10T15:59:14.092473+0000","last_became_active":"2024-09-10T15:59:16.109611+0000","last_became_peered":"2024-09-10T15:59:16.109611+0000","last_unstale":"2024-09-10T16:00:39.485822+0000","last_undegraded":"2024-09-10T16:00:39.485822+0000","last_fullsized":"2024-09-10T15:59:15.770264+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":7173,"log_dups_size":0,"ondisk_log_size":7173,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29364224,"num_objects":7169,"num_object_clones":0,"num_object_copies":14338,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2434,"num_objects_unfound":0,"num_objects_dirty":7169,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5177,"num_write_kb":20708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1834,"num_bytes_recovered":7512064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2434},{"shards":"2","objects":4735}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'7182","reported_seq":7258,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:00:39.449797+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:00:39.449797+0000","last_peered":"2024-09-10T16:00:39.449797+0000","last_clean":"2024-09-10T16:00:39.449797+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:00:39.449797+0000","last_undegraded":"2024-09-10T16:00:39.449797+0000","last_fullsized":"2024-09-10T16:00:39.449797+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":7182,"log_dups_size":0,"ondisk_log_size":7182,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29417472,"num_objects":7182,"num_object_clones":0,"num_object_copies":14364,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7182,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7182,"num_write_kb":28728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'7144","reported_seq":7268,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:00:36.601257+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:00:36.601257+0000","last_peered":"2024-09-10T16:00:36.601257+0000","last_clean":"2024-09-10T16:00:36.601257+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:00:36.601257+0000","last_undegraded":"2024-09-10T16:00:36.601257+0000","last_fullsized":"2024-09-10T16:00:36.601257+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":7144,"log_dups_size":0,"ondisk_log_size":7144,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29233635,"num_objects":7138,"num_object_clones":0,"num_object_copies":14276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7138,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3855,"num_write_kb":15420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'7192","reported_seq":7325,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:36.468046+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:00:36.468046+0000","last_peered":"2024-09-10T16:00:36.468046+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:00:36.468046+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":7192,"log_dups_size":0,"ondisk_log_size":7192,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29438435,"num_objects":7188,"num_object_clones":0,"num_object_copies":14376,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7188,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3804,"num_write_kb":15216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2817},{"shards":"0,2","objects":4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'7134","reported_seq":7267,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:36.578040+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:00:36.578040+0000","last_peered":"2024-09-10T16:00:36.578040+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:00:36.578040+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":7134,"log_dups_size":0,"ondisk_log_size":7134,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29197254,"num_objects":7130,"num_object_clones":0,"num_object_copies":14260,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7130,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3633,"num_write_kb":14532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2893},{"shards":"0,2","objects":4237}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'7253","reported_seq":7378,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:00:36.597105+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:00:36.597105+0000","last_peered":"2024-09-10T16:00:36.597105+0000","last_clean":"2024-09-10T16:00:36.597105+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:00:36.597105+0000","last_undegraded":"2024-09-10T16:00:36.597105+0000","last_fullsized":"2024-09-10T16:00:36.597105+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":7253,"log_dups_size":0,"ondisk_log_size":7253,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29700096,"num_objects":7251,"num_object_clones":0,"num_object_copies":14502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7251,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4100,"num_write_kb":16400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.775865+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T15:59:15.775865+0000","last_peered":"2024-09-10T15:59:15.775865+0000","last_clean":"2024-09-10T15:59:15.775865+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T15:59:15.775865+0000","last_undegraded":"2024-09-10T15:59:15.775865+0000","last_fullsized":"2024-09-10T15:59:15.775865+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'6908","reported_seq":7043,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:39.426820+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:00:39.426820+0000","last_peered":"2024-09-10T16:00:39.426820+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:00:39.426820+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":6908,"log_dups_size":0,"ondisk_log_size":6908,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28266979,"num_objects":6902,"num_object_clones":0,"num_object_copies":13804,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6902,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4727,"num_write_kb":18908,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4123},{"shards":"2","objects":2779}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":89,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.772619+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T15:59:15.772619+0000","last_peered":"2024-09-10T15:59:15.772619+0000","last_clean":"2024-09-10T15:59:15.772619+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T15:59:15.772619+0000","last_undegraded":"2024-09-10T15:59:15.772619+0000","last_fullsized":"2024-09-10T15:59:15.772619+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'7081","reported_seq":7206,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:36.470618+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:00:36.470618+0000","last_peered":"2024-09-10T16:00:36.470618+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:00:36.470618+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":7081,"log_dups_size":0,"ondisk_log_size":7081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28971008,"num_objects":7073,"num_object_clones":0,"num_object_copies":14146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7073,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3585,"num_write_kb":14340,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2818},{"shards":"0,2","objects":4255}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":99,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.884297+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T15:59:15.884297+0000","last_peered":"2024-09-10T15:59:15.884297+0000","last_clean":"2024-09-10T15:59:15.884297+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T15:59:15.884297+0000","last_undegraded":"2024-09-10T15:59:15.884297+0000","last_fullsized":"2024-09-10T15:59:15.884297+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'7239","reported_seq":7366,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:39.445569+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:00:39.445569+0000","last_peered":"2024-09-10T16:00:39.445569+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:00:39.445569+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":7239,"log_dups_size":0,"ondisk_log_size":7239,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29630947,"num_objects":7235,"num_object_clones":0,"num_object_copies":14470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7235,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4919,"num_write_kb":19676,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4304},{"shards":"2","objects":2931}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.837509+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T15:59:15.837509+0000","last_peered":"2024-09-10T15:59:15.837509+0000","last_clean":"2024-09-10T15:59:15.837509+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T15:59:15.837509+0000","last_undegraded":"2024-09-10T15:59:15.837509+0000","last_fullsized":"2024-09-10T15:59:15.837509+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'7329","reported_seq":7452,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:39.454784+0000","last_change":"2024-09-10T15:59:16.174113+0000","last_active":"2024-09-10T16:00:39.454784+0000","last_peered":"2024-09-10T16:00:39.454784+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:00:39.454784+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":7329,"log_dups_size":0,"ondisk_log_size":7329,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29995008,"num_objects":7323,"num_object_clones":0,"num_object_copies":14646,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4354,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7323,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4883,"num_write_kb":19532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4352},{"shards":"2","objects":2971}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'7099","reported_seq":7224,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:36.573739+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:00:36.573739+0000","last_peered":"2024-09-10T16:00:36.573739+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:00:36.573739+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":7099,"log_dups_size":0,"ondisk_log_size":7099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29069312,"num_objects":7097,"num_object_clones":0,"num_object_copies":14194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7097,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3730,"num_write_kb":14920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2793},{"shards":"0,2","objects":4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.989413+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T15:59:15.989413+0000","last_peered":"2024-09-10T15:59:15.989413+0000","last_clean":"2024-09-10T15:59:15.989413+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T15:59:15.989413+0000","last_undegraded":"2024-09-10T15:59:15.989413+0000","last_fullsized":"2024-09-10T15:59:15.989413+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'6938","reported_seq":7076,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:36.537234+0000","last_change":"2024-09-10T15:59:16.251610+0000","last_active":"2024-09-10T16:00:36.537234+0000","last_peered":"2024-09-10T16:00:36.537234+0000","last_clean":"2024-09-10T15:59:14.115595+0000","last_became_active":"2024-09-10T15:59:16.244089+0000","last_became_peered":"2024-09-10T15:59:16.244089+0000","last_unstale":"2024-09-10T16:00:36.537234+0000","last_undegraded":"2024-09-10T15:59:15.847127+0000","last_fullsized":"2024-09-10T15:59:15.774437+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":6938,"log_dups_size":0,"ondisk_log_size":6938,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28385280,"num_objects":6930,"num_object_clones":0,"num_object_copies":13860,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6930,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3328,"num_write_kb":13312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2727},{"shards":"0,2","objects":4203}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":89,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.773633+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T15:59:15.773633+0000","last_peered":"2024-09-10T15:59:15.773633+0000","last_clean":"2024-09-10T15:59:15.773633+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T15:59:15.773633+0000","last_undegraded":"2024-09-10T15:59:15.773633+0000","last_fullsized":"2024-09-10T15:59:15.773633+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'7241","reported_seq":7378,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:36.544380+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:00:36.544380+0000","last_peered":"2024-09-10T16:00:36.544380+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:00:36.544380+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":7241,"log_dups_size":0,"ondisk_log_size":7241,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29642752,"num_objects":7237,"num_object_clones":0,"num_object_copies":14474,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7237,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3496,"num_write_kb":13984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2866},{"shards":"0,2","objects":4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.916017+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T15:59:15.916017+0000","last_peered":"2024-09-10T15:59:15.916017+0000","last_clean":"2024-09-10T15:59:15.916017+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T15:59:15.916017+0000","last_undegraded":"2024-09-10T15:59:15.916017+0000","last_fullsized":"2024-09-10T15:59:15.916017+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.776033+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T15:59:15.776033+0000","last_peered":"2024-09-10T15:59:15.776033+0000","last_clean":"2024-09-10T15:59:15.776033+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T15:59:15.776033+0000","last_undegraded":"2024-09-10T15:59:15.776033+0000","last_fullsized":"2024-09-10T15:59:15.776033+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'7210","reported_seq":7349,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:36.557988+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:00:36.557988+0000","last_peered":"2024-09-10T16:00:36.557988+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:00:36.557988+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":7210,"log_dups_size":0,"ondisk_log_size":7210,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29483974,"num_objects":7200,"num_object_clones":0,"num_object_copies":14400,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7200,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3373,"num_write_kb":13492,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2877},{"shards":"0,2","objects":4323}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":468026078,"num_objects":114273,"num_object_clones":0,"num_object_copies":228546,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":47117,"num_objects_misplaced":2434,"num_objects_unfound":0,"num_objects_dirty":114273,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":72119,"num_write_kb":288476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1837,"num_bytes_recovered":7524352,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":945643520,"data_stored":945488161,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":114345,"ondisk_log_size":114345,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986208,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":547320,"kb_used_data":367080,"kb_used_omap":4,"kb_used_meta":180219,"kb_avail":93824520,"statfs":{"total":96636764160,"available":96076308480,"internally_reserved":0,"allocated":375889920,"data_stored":375612160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":184544938},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,0,3,1,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659154,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":293500,"kb_used_data":145196,"kb_used_omap":4,"kb_used_meta":148283,"kb_avail":94078340,"statfs":{"total":96636764160,"available":96336220160,"internally_reserved":0,"allocated":148680704,"data_stored":148351898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":151842478},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,1,4],"upper_bound":32},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182436,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":684436,"kb_used_data":409860,"kb_used_omap":4,"kb_used_meta":274555,"kb_avail":93687404,"statfs":{"total":96636764160,"available":95935901696,"internally_reserved":0,"allocated":419696640,"data_stored":419414803,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":281145002},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,6,2,4,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":419962880,"data_stored":419926750,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":149291008,"data_stored":149207909,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":376389632,"data_stored":376353502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:00:42.014 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:00:42.565 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:00:42.565 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:00:42.594 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":139,"stamp":"2024-09-10T16:00:41.570389+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":468026097,"num_objects":114275,"num_object_clones":0,"num_object_copies":228550,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":47117,"num_objects_misplaced":2434,"num_objects_unfound":0,"num_objects_dirty":114275,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":72121,"num_write_kb":288478,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1839,"num_bytes_recovered":7524371,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":114348,"ondisk_log_size":114348,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1525256,"kb_used_data":922136,"kb_used_omap":12,"kb_used_meta":603059,"kb_avail":281590264,"statfs":{"total":289910292480,"available":288348430336,"internally_reserved":0,"allocated":944267264,"data_stored":943378861,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":617532418},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,2,7,9,5,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":21381120,"num_objects":5220,"num_object_clones":0,"num_object_copies":10440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-238,"num_objects_unfound":0,"num_objects_dirty":5220,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5220,"num_write_kb":20880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":238,"num_bytes_recovered":974848,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004015"},"pg_stats":[{"pgid":"2.f","version":"43'7051","reported_seq":7121,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:36.412075+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:00:36.412075+0000","last_peered":"2024-09-10T16:00:36.412075+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:00:36.412075+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":7051,"log_dups_size":0,"ondisk_log_size":7051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28880896,"num_objects":7051,"num_object_clones":0,"num_object_copies":14102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7051,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7051,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2792},{"shards":"0,2","objects":4258}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'7171","reported_seq":7279,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:00:39.459351+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:00:39.459351+0000","last_peered":"2024-09-10T16:00:39.459351+0000","last_clean":"2024-09-10T16:00:39.459351+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:00:39.459351+0000","last_undegraded":"2024-09-10T16:00:39.459351+0000","last_fullsized":"2024-09-10T16:00:39.459351+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":7171,"log_dups_size":0,"ondisk_log_size":7171,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29348806,"num_objects":7167,"num_object_clones":0,"num_object_copies":14334,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7167,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5276,"num_write_kb":21104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'7173","reported_seq":10965,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T16:00:39.485822+0000","last_change":"2024-09-10T15:59:16.142612+0000","last_active":"2024-09-10T16:00:39.485822+0000","last_peered":"2024-09-10T16:00:39.485822+0000","last_clean":"2024-09-10T15:59:14.092473+0000","last_became_active":"2024-09-10T15:59:16.109611+0000","last_became_peered":"2024-09-10T15:59:16.109611+0000","last_unstale":"2024-09-10T16:00:39.485822+0000","last_undegraded":"2024-09-10T16:00:39.485822+0000","last_fullsized":"2024-09-10T15:59:15.770264+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":7173,"log_dups_size":0,"ondisk_log_size":7173,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29364224,"num_objects":7169,"num_object_clones":0,"num_object_copies":14338,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2434,"num_objects_unfound":0,"num_objects_dirty":7169,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5177,"num_write_kb":20708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1834,"num_bytes_recovered":7512064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2434},{"shards":"2","objects":4735}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'7182","reported_seq":7258,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:00:39.449797+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:00:39.449797+0000","last_peered":"2024-09-10T16:00:39.449797+0000","last_clean":"2024-09-10T16:00:39.449797+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:00:39.449797+0000","last_undegraded":"2024-09-10T16:00:39.449797+0000","last_fullsized":"2024-09-10T16:00:39.449797+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":7182,"log_dups_size":0,"ondisk_log_size":7182,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29417472,"num_objects":7182,"num_object_clones":0,"num_object_copies":14364,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7182,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7182,"num_write_kb":28728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'7144","reported_seq":7268,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:00:36.601257+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:00:36.601257+0000","last_peered":"2024-09-10T16:00:36.601257+0000","last_clean":"2024-09-10T16:00:36.601257+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:00:36.601257+0000","last_undegraded":"2024-09-10T16:00:36.601257+0000","last_fullsized":"2024-09-10T16:00:36.601257+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":7144,"log_dups_size":0,"ondisk_log_size":7144,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29233635,"num_objects":7138,"num_object_clones":0,"num_object_copies":14276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7138,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3855,"num_write_kb":15420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'7192","reported_seq":7325,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:36.468046+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:00:36.468046+0000","last_peered":"2024-09-10T16:00:36.468046+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:00:36.468046+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":7192,"log_dups_size":0,"ondisk_log_size":7192,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29438435,"num_objects":7188,"num_object_clones":0,"num_object_copies":14376,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7188,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3804,"num_write_kb":15216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2817},{"shards":"0,2","objects":4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'7134","reported_seq":7267,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:36.578040+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:00:36.578040+0000","last_peered":"2024-09-10T16:00:36.578040+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:00:36.578040+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":7134,"log_dups_size":0,"ondisk_log_size":7134,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29197254,"num_objects":7130,"num_object_clones":0,"num_object_copies":14260,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7130,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3633,"num_write_kb":14532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2893},{"shards":"0,2","objects":4237}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'7253","reported_seq":7378,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:00:36.597105+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:00:36.597105+0000","last_peered":"2024-09-10T16:00:36.597105+0000","last_clean":"2024-09-10T16:00:36.597105+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:00:36.597105+0000","last_undegraded":"2024-09-10T16:00:36.597105+0000","last_fullsized":"2024-09-10T16:00:36.597105+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":7253,"log_dups_size":0,"ondisk_log_size":7253,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29700096,"num_objects":7251,"num_object_clones":0,"num_object_copies":14502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7251,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4100,"num_write_kb":16400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.775865+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T15:59:15.775865+0000","last_peered":"2024-09-10T15:59:15.775865+0000","last_clean":"2024-09-10T15:59:15.775865+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T15:59:15.775865+0000","last_undegraded":"2024-09-10T15:59:15.775865+0000","last_fullsized":"2024-09-10T15:59:15.775865+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'6908","reported_seq":7043,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:39.426820+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:00:39.426820+0000","last_peered":"2024-09-10T16:00:39.426820+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:00:39.426820+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":6908,"log_dups_size":0,"ondisk_log_size":6908,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28266979,"num_objects":6902,"num_object_clones":0,"num_object_copies":13804,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6902,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4727,"num_write_kb":18908,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4123},{"shards":"2","objects":2779}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":89,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.772619+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T15:59:15.772619+0000","last_peered":"2024-09-10T15:59:15.772619+0000","last_clean":"2024-09-10T15:59:15.772619+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T15:59:15.772619+0000","last_undegraded":"2024-09-10T15:59:15.772619+0000","last_fullsized":"2024-09-10T15:59:15.772619+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'7081","reported_seq":7206,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:36.470618+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:00:36.470618+0000","last_peered":"2024-09-10T16:00:36.470618+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:00:36.470618+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":7081,"log_dups_size":0,"ondisk_log_size":7081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28971008,"num_objects":7073,"num_object_clones":0,"num_object_copies":14146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7073,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3585,"num_write_kb":14340,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2818},{"shards":"0,2","objects":4255}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":99,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.884297+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T15:59:15.884297+0000","last_peered":"2024-09-10T15:59:15.884297+0000","last_clean":"2024-09-10T15:59:15.884297+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T15:59:15.884297+0000","last_undegraded":"2024-09-10T15:59:15.884297+0000","last_fullsized":"2024-09-10T15:59:15.884297+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'7239","reported_seq":7366,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:39.445569+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:00:39.445569+0000","last_peered":"2024-09-10T16:00:39.445569+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:00:39.445569+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":7239,"log_dups_size":0,"ondisk_log_size":7239,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29630947,"num_objects":7235,"num_object_clones":0,"num_object_copies":14470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7235,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4919,"num_write_kb":19676,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4304},{"shards":"2","objects":2931}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.837509+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T15:59:15.837509+0000","last_peered":"2024-09-10T15:59:15.837509+0000","last_clean":"2024-09-10T15:59:15.837509+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T15:59:15.837509+0000","last_undegraded":"2024-09-10T15:59:15.837509+0000","last_fullsized":"2024-09-10T15:59:15.837509+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'7329","reported_seq":7452,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:39.454784+0000","last_change":"2024-09-10T15:59:16.174113+0000","last_active":"2024-09-10T16:00:39.454784+0000","last_peered":"2024-09-10T16:00:39.454784+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:00:39.454784+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":7329,"log_dups_size":0,"ondisk_log_size":7329,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29995008,"num_objects":7323,"num_object_clones":0,"num_object_copies":14646,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4354,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7323,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4883,"num_write_kb":19532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4352},{"shards":"2","objects":2971}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'7099","reported_seq":7224,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:36.573739+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:00:36.573739+0000","last_peered":"2024-09-10T16:00:36.573739+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:00:36.573739+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":7099,"log_dups_size":0,"ondisk_log_size":7099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29069312,"num_objects":7097,"num_object_clones":0,"num_object_copies":14194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7097,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3730,"num_write_kb":14920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2793},{"shards":"0,2","objects":4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.989413+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T15:59:15.989413+0000","last_peered":"2024-09-10T15:59:15.989413+0000","last_clean":"2024-09-10T15:59:15.989413+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T15:59:15.989413+0000","last_undegraded":"2024-09-10T15:59:15.989413+0000","last_fullsized":"2024-09-10T15:59:15.989413+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'6938","reported_seq":7076,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:36.537234+0000","last_change":"2024-09-10T15:59:16.251610+0000","last_active":"2024-09-10T16:00:36.537234+0000","last_peered":"2024-09-10T16:00:36.537234+0000","last_clean":"2024-09-10T15:59:14.115595+0000","last_became_active":"2024-09-10T15:59:16.244089+0000","last_became_peered":"2024-09-10T15:59:16.244089+0000","last_unstale":"2024-09-10T16:00:36.537234+0000","last_undegraded":"2024-09-10T15:59:15.847127+0000","last_fullsized":"2024-09-10T15:59:15.774437+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":6938,"log_dups_size":0,"ondisk_log_size":6938,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28385280,"num_objects":6930,"num_object_clones":0,"num_object_copies":13860,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6930,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3328,"num_write_kb":13312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2727},{"shards":"0,2","objects":4203}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":89,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.773633+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T15:59:15.773633+0000","last_peered":"2024-09-10T15:59:15.773633+0000","last_clean":"2024-09-10T15:59:15.773633+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T15:59:15.773633+0000","last_undegraded":"2024-09-10T15:59:15.773633+0000","last_fullsized":"2024-09-10T15:59:15.773633+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'7241","reported_seq":7378,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:36.544380+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:00:36.544380+0000","last_peered":"2024-09-10T16:00:36.544380+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:00:36.544380+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":7241,"log_dups_size":0,"ondisk_log_size":7241,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29642752,"num_objects":7237,"num_object_clones":0,"num_object_copies":14474,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7237,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3496,"num_write_kb":13984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2866},{"shards":"0,2","objects":4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.916017+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T15:59:15.916017+0000","last_peered":"2024-09-10T15:59:15.916017+0000","last_clean":"2024-09-10T15:59:15.916017+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T15:59:15.916017+0000","last_undegraded":"2024-09-10T15:59:15.916017+0000","last_fullsized":"2024-09-10T15:59:15.916017+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.776033+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T15:59:15.776033+0000","last_peered":"2024-09-10T15:59:15.776033+0000","last_clean":"2024-09-10T15:59:15.776033+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T15:59:15.776033+0000","last_undegraded":"2024-09-10T15:59:15.776033+0000","last_fullsized":"2024-09-10T15:59:15.776033+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'7210","reported_seq":7349,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:36.557988+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:00:36.557988+0000","last_peered":"2024-09-10T16:00:36.557988+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:00:36.557988+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":7210,"log_dups_size":0,"ondisk_log_size":7210,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29483974,"num_objects":7200,"num_object_clones":0,"num_object_copies":14400,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7200,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3373,"num_write_kb":13492,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2877},{"shards":"0,2","objects":4323}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":468026078,"num_objects":114273,"num_object_clones":0,"num_object_copies":228546,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":47117,"num_objects_misplaced":2434,"num_objects_unfound":0,"num_objects_dirty":114273,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":72119,"num_write_kb":288476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1837,"num_bytes_recovered":7524352,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":945643520,"data_stored":945488161,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":114345,"ondisk_log_size":114345,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986208,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":547320,"kb_used_data":367080,"kb_used_omap":4,"kb_used_meta":180219,"kb_avail":93824520,"statfs":{"total":96636764160,"available":96076308480,"internally_reserved":0,"allocated":375889920,"data_stored":375612160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":184544938},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,0,3,1,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659154,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":293500,"kb_used_data":145196,"kb_used_omap":4,"kb_used_meta":148283,"kb_avail":94078340,"statfs":{"total":96636764160,"available":96336220160,"internally_reserved":0,"allocated":148680704,"data_stored":148351898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":151842478},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,1,4],"upper_bound":32},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182436,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":684436,"kb_used_data":409860,"kb_used_omap":4,"kb_used_meta":274555,"kb_avail":93687404,"statfs":{"total":96636764160,"available":95935901696,"internally_reserved":0,"allocated":419696640,"data_stored":419414803,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":281145002},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,6,2,4,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":419962880,"data_stored":419926750,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":149291008,"data_stored":149207909,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":376389632,"data_stored":376353502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:00:42.596 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-10T16:00:42.596 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "43'7051", 'reported_seq': 7121, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:00:36.412075+0000', 'last_change': '2024-09-10T15:59:16.281558+0000', 'last_active': '2024-09-10T16:00:36.412075+0000', 'last_peered': '2024-09-10T16:00:36.412075+0000', 'last_clean': '2024-09-10T15:59:14.132456+0000', 'last_became_active': '2024-09-10T15:59:16.274407+0000', 'last_became_peered': '2024-09-10T15:59:16.274407+0000', 'last_unstale': '2024-09-10T16:00:36.412075+0000', 'last_undegraded': '2024-09-10T15:59:15.851279+0000', 'last_fullsized': '2024-09-10T15:59:15.774356+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'objects_scrubbed': 0, 'log_size': 7051, 'log_dups_size': 0, 'ondisk_log_size': 7051, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28880896, 'num_objects': 7051, 'num_object_clones': 0, 'num_object_copies': 14102, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4259, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7051, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7051, 'num_write_kb': 28204, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2792}, {'shards': '0,2', 'objects': 4258}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:00:42.596 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-10T16:00:42.596 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "43'7173", 'reported_seq': 10965, 'reported_epoch': 43, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-10T16:00:39.485822+0000', 'last_change': '2024-09-10T15:59:16.142612+0000', 'last_active': '2024-09-10T16:00:39.485822+0000', 'last_peered': '2024-09-10T16:00:39.485822+0000', 'last_clean': '2024-09-10T15:59:14.092473+0000', 'last_became_active': '2024-09-10T15:59:16.109611+0000', 'last_became_peered': '2024-09-10T15:59:16.109611+0000', 'last_unstale': '2024-09-10T16:00:39.485822+0000', 'last_undegraded': '2024-09-10T16:00:39.485822+0000', 'last_fullsized': '2024-09-10T15:59:15.770264+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2029", 'last_scrub_stamp': '2024-09-10T15:58:45.933789+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:45.933789+0000', 'objects_scrubbed': 1992, 'log_size': 7173, 'log_dups_size': 0, 'ondisk_log_size': 7173, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 668, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29364224, 'num_objects': 7169, 'num_object_clones': 0, 'num_object_copies': 14338, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 2434, 'num_objects_unfound': 0, 'num_objects_dirty': 7169, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5177, 'num_write_kb': 20708, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1834, 'num_bytes_recovered': 7512064, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2434}, {'shards': '2', 'objects': 4735}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:00:42.596 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-10T16:00:42.596 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "43'7192", 'reported_seq': 7325, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:00:36.468046+0000', 'last_change': '2024-09-10T15:59:16.308264+0000', 'last_active': '2024-09-10T16:00:36.468046+0000', 'last_peered': '2024-09-10T16:00:36.468046+0000', 'last_clean': '2024-09-10T15:59:14.111551+0000', 'last_became_active': '2024-09-10T15:59:16.300040+0000', 'last_became_peered': '2024-09-10T15:59:16.300040+0000', 'last_unstale': '2024-09-10T16:00:36.468046+0000', 'last_undegraded': '2024-09-10T15:59:15.878234+0000', 'last_fullsized': '2024-09-10T15:59:15.769328+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3442", 'last_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'objects_scrubbed': 3384, 'log_size': 7192, 'log_dups_size': 0, 'ondisk_log_size': 7192, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1292, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29438435, 'num_objects': 7188, 'num_object_clones': 0, 'num_object_copies': 14376, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4372, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7188, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 3804, 'num_write_kb': 15216, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2817}, {'shards': '0,2', 'objects': 4371}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:00:42.596 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-10T16:00:42.596 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "43'7134", 'reported_seq': 7267, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:00:36.578040+0000', 'last_change': '2024-09-10T15:59:16.331899+0000', 'last_active': '2024-09-10T16:00:36.578040+0000', 'last_peered': '2024-09-10T16:00:36.578040+0000', 'last_clean': '2024-09-10T15:59:14.139436+0000', 'last_became_active': '2024-09-10T15:59:16.318715+0000', 'last_became_peered': '2024-09-10T15:59:16.318715+0000', 'last_unstale': '2024-09-10T16:00:36.578040+0000', 'last_undegraded': '2024-09-10T15:59:15.877524+0000', 'last_fullsized': '2024-09-10T15:59:15.769042+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3537", 'last_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'objects_scrubbed': 3497, 'log_size': 7134, 'log_dups_size': 0, 'ondisk_log_size': 7134, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1224, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29197254, 'num_objects': 7130, 'num_object_clones': 0, 'num_object_copies': 14260, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4237, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7130, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 3633, 'num_write_kb': 14532, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2893}, {'shards': '0,2', 'objects': 4237}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:00:42.597 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-10T16:00:42.597 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "43'6908", 'reported_seq': 7043, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:00:39.426820+0000', 'last_change': '2024-09-10T15:59:16.175680+0000', 'last_active': '2024-09-10T16:00:39.426820+0000', 'last_peered': '2024-09-10T16:00:39.426820+0000', 'last_clean': '2024-09-10T15:59:14.088187+0000', 'last_became_active': '2024-09-10T15:59:16.165810+0000', 'last_became_peered': '2024-09-10T15:59:16.165810+0000', 'last_unstale': '2024-09-10T16:00:39.426820+0000', 'last_undegraded': '2024-09-10T15:59:15.822098+0000', 'last_fullsized': '2024-09-10T15:59:15.769123+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2218", 'last_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'objects_scrubbed': 2175, 'log_size': 6908, 'log_dups_size': 0, 'ondisk_log_size': 6908, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 825, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28266979, 'num_objects': 6902, 'num_object_clones': 0, 'num_object_copies': 13804, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4124, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6902, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 4727, 'num_write_kb': 18908, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4123}, {'shards': '2', 'objects': 2779}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:00:42.597 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-10T16:00:42.597 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "43'7081", 'reported_seq': 7206, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:00:36.470618+0000', 'last_change': '2024-09-10T15:59:16.277742+0000', 'last_active': '2024-09-10T16:00:36.470618+0000', 'last_peered': '2024-09-10T16:00:36.470618+0000', 'last_clean': '2024-09-10T15:59:14.129081+0000', 'last_became_active': '2024-09-10T15:59:16.269835+0000', 'last_became_peered': '2024-09-10T15:59:16.269835+0000', 'last_unstale': '2024-09-10T16:00:36.470618+0000', 'last_undegraded': '2024-09-10T15:59:15.860857+0000', 'last_fullsized': '2024-09-10T15:59:15.771410+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3557", 'last_scrub_stamp': '2024-09-10T15:59:02.493678+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:02.493678+0000', 'objects_scrubbed': 3488, 'log_size': 7081, 'log_dups_size': 0, 'ondisk_log_size': 7081, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1472, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28971008, 'num_objects': 7073, 'num_object_clones': 0, 'num_object_copies': 14146, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4257, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7073, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3585, 'num_write_kb': 14340, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2818}, {'shards': '0,2', 'objects': 4255}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:00:42.597 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-10T16:00:42.597 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "43'7239", 'reported_seq': 7366, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:00:39.445569+0000', 'last_change': '2024-09-10T15:59:16.284311+0000', 'last_active': '2024-09-10T16:00:39.445569+0000', 'last_peered': '2024-09-10T16:00:39.445569+0000', 'last_clean': '2024-09-10T15:59:14.116041+0000', 'last_became_active': '2024-09-10T15:59:16.274543+0000', 'last_became_peered': '2024-09-10T15:59:16.274543+0000', 'last_unstale': '2024-09-10T16:00:39.445569+0000', 'last_undegraded': '2024-09-10T15:59:15.812484+0000', 'last_fullsized': '2024-09-10T15:59:15.770885+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2356", 'last_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'objects_scrubbed': 2316, 'log_size': 7239, 'log_dups_size': 0, 'ondisk_log_size': 7239, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 783, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29630947, 'num_objects': 7235, 'num_object_clones': 0, 'num_object_copies': 14470, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7235, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 4919, 'num_write_kb': 19676, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4304}, {'shards': '2', 'objects': 2931}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:00:42.597 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-10T16:00:42.597 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "43'7329", 'reported_seq': 7452, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:00:39.454784+0000', 'last_change': '2024-09-10T15:59:16.174113+0000', 'last_active': '2024-09-10T16:00:39.454784+0000', 'last_peered': '2024-09-10T16:00:39.454784+0000', 'last_clean': '2024-09-10T15:59:14.110654+0000', 'last_became_active': '2024-09-10T15:59:16.164645+0000', 'last_became_peered': '2024-09-10T15:59:16.164645+0000', 'last_unstale': '2024-09-10T16:00:39.454784+0000', 'last_undegraded': '2024-09-10T15:59:15.803593+0000', 'last_fullsized': '2024-09-10T15:59:15.769083+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2486", 'last_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'objects_scrubbed': 2440, 'log_size': 7329, 'log_dups_size': 0, 'ondisk_log_size': 7329, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 792, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29995008, 'num_objects': 7323, 'num_object_clones': 0, 'num_object_copies': 14646, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4354, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7323, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4883, 'num_write_kb': 19532, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4352}, {'shards': '2', 'objects': 2971}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:00:42.597 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-10T16:00:42.597 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "43'7099", 'reported_seq': 7224, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:00:36.573739+0000', 'last_change': '2024-09-10T15:59:16.315641+0000', 'last_active': '2024-09-10T16:00:36.573739+0000', 'last_peered': '2024-09-10T16:00:36.573739+0000', 'last_clean': '2024-09-10T15:59:14.117001+0000', 'last_became_active': '2024-09-10T15:59:16.302245+0000', 'last_became_peered': '2024-09-10T15:59:16.302245+0000', 'last_unstale': '2024-09-10T16:00:36.573739+0000', 'last_undegraded': '2024-09-10T15:59:15.845012+0000', 'last_fullsized': '2024-09-10T15:59:15.770337+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3404", 'last_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'objects_scrubbed': 3367, 'log_size': 7099, 'log_dups_size': 0, 'ondisk_log_size': 7099, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 983, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29069312, 'num_objects': 7097, 'num_object_clones': 0, 'num_object_copies': 14194, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7097, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3730, 'num_write_kb': 14920, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2793}, {'shards': '0,2', 'objects': 4304}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:00:42.598 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-10T16:00:42.598 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "43'6938", 'reported_seq': 7076, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:00:36.537234+0000', 'last_change': '2024-09-10T15:59:16.251610+0000', 'last_active': '2024-09-10T16:00:36.537234+0000', 'last_peered': '2024-09-10T16:00:36.537234+0000', 'last_clean': '2024-09-10T15:59:14.115595+0000', 'last_became_active': '2024-09-10T15:59:16.244089+0000', 'last_became_peered': '2024-09-10T15:59:16.244089+0000', 'last_unstale': '2024-09-10T16:00:36.537234+0000', 'last_undegraded': '2024-09-10T15:59:15.847127+0000', 'last_fullsized': '2024-09-10T15:59:15.774437+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3653", 'last_scrub_stamp': '2024-09-10T15:59:05.298468+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:05.298468+0000', 'objects_scrubbed': 3602, 'log_size': 6938, 'log_dups_size': 0, 'ondisk_log_size': 6938, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1289, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28385280, 'num_objects': 6930, 'num_object_clones': 0, 'num_object_copies': 13860, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4206, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6930, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3328, 'num_write_kb': 13312, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2727}, {'shards': '0,2', 'objects': 4203}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:00:42.598 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-10T16:00:42.598 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "43'7241", 'reported_seq': 7378, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:00:36.544380+0000', 'last_change': '2024-09-10T15:59:16.311291+0000', 'last_active': '2024-09-10T16:00:36.544380+0000', 'last_peered': '2024-09-10T16:00:36.544380+0000', 'last_clean': '2024-09-10T15:59:14.116363+0000', 'last_became_active': '2024-09-10T15:59:16.301244+0000', 'last_became_peered': '2024-09-10T15:59:16.301244+0000', 'last_unstale': '2024-09-10T16:00:36.544380+0000', 'last_undegraded': '2024-09-10T15:59:15.883434+0000', 'last_fullsized': '2024-09-10T15:59:15.772756+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3789", 'last_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'objects_scrubbed': 3741, 'log_size': 7241, 'log_dups_size': 0, 'ondisk_log_size': 7241, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1440, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29642752, 'num_objects': 7237, 'num_object_clones': 0, 'num_object_copies': 14474, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4373, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7237, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3496, 'num_write_kb': 13984, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2866}, {'shards': '0,2', 'objects': 4371}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:00:42.598 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-10T16:00:42.598 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "43'7210", 'reported_seq': 7349, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:00:36.557988+0000', 'last_change': '2024-09-10T15:59:16.313409+0000', 'last_active': '2024-09-10T16:00:36.557988+0000', 'last_peered': '2024-09-10T16:00:36.557988+0000', 'last_clean': '2024-09-10T15:59:14.129780+0000', 'last_became_active': '2024-09-10T15:59:16.300932+0000', 'last_became_peered': '2024-09-10T15:59:16.300932+0000', 'last_unstale': '2024-09-10T16:00:36.557988+0000', 'last_undegraded': '2024-09-10T15:59:15.887823+0000', 'last_fullsized': '2024-09-10T15:59:15.768513+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3880", 'last_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'objects_scrubbed': 3827, 'log_size': 7210, 'log_dups_size': 0, 'ondisk_log_size': 7210, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1397, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29483974, 'num_objects': 7200, 'num_object_clones': 0, 'num_object_copies': 14400, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4325, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7200, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3373, 'num_write_kb': 13492, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2877}, {'shards': '0,2', 'objects': 4323}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:00:42.598 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T16:00:42.982 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 130 20 116959 116939 3.51322 1.80859 0.1057 0.0222276 2024-09-10T16:00:43.305 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:00:43.305 INFO:teuthology.orchestra.run.smithi115.stdout:{"fsid":"7effee29-df05-4351-9741-ee203564ed81","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 47117/228550 objects degraded (20.616%), 11 pgs degraded, 12 pgs undersized","count":23},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":207,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":43,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983954,"num_in_osds":3,"osd_in_since":1725983947,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":114275,"data_bytes":468026097,"bytes_used":1561862144,"bytes_avail":288348430336,"bytes_total":289910292480,"degraded_objects":47117,"degraded_total":228550,"degraded_ratio":0.206156202143951,"misplaced_objects":2434,"misplaced_total":228550,"misplaced_ratio":0.010649748413913805,"recovering_objects_per_sec":19,"recovering_bytes_per_sec":81210,"recovering_keys_per_sec":0,"num_objects_recovered":238,"num_bytes_recovered":974848,"num_keys_recovered":0,"write_bytes_sec":1781163,"read_op_per_sec":0,"write_op_per_sec":434},"fsmap":{"epoch":1,"btime":"2024-09-10T15:57:14:967120+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T16:00:13.560776+0000","services":{}},"progress_events":{}} 2024-09-10T16:00:43.333 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-10T16:00:43.334 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:00:43.913 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:00:43.915 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:00:43.942 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":140,"stamp":"2024-09-10T16:00:43.570980+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":474882801,"num_objects":115949,"num_object_clones":0,"num_object_copies":231898,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":47117,"num_objects_misplaced":2434,"num_objects_unfound":0,"num_objects_dirty":115949,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":73795,"num_write_kb":295174,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1839,"num_bytes_recovered":7524371,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":116022,"ondisk_log_size":116022,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1524652,"kb_used_data":929852,"kb_used_omap":12,"kb_used_meta":594739,"kb_avail":281590868,"statfs":{"total":289910292480,"available":288349048832,"internally_reserved":0,"allocated":952168448,"data_stored":951280045,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":609012738},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,2,7,9,5,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":21413888,"num_objects":5228,"num_object_clones":0,"num_object_copies":10456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-238,"num_objects_unfound":0,"num_objects_dirty":5228,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5228,"num_write_kb":20912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":238,"num_bytes_recovered":974848,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003918"},"pg_stats":[{"pgid":"2.f","version":"43'7207","reported_seq":7276,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:41.606932+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:00:41.606932+0000","last_peered":"2024-09-10T16:00:41.606932+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:00:41.606932+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":7207,"log_dups_size":0,"ondisk_log_size":7207,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29519872,"num_objects":7207,"num_object_clones":0,"num_object_copies":14414,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7207,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7207,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2948},{"shards":"0,2","objects":4258}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'7171","reported_seq":7279,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:00:39.459351+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:00:39.459351+0000","last_peered":"2024-09-10T16:00:39.459351+0000","last_clean":"2024-09-10T16:00:39.459351+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:00:39.459351+0000","last_undegraded":"2024-09-10T16:00:39.459351+0000","last_fullsized":"2024-09-10T16:00:39.459351+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":7171,"log_dups_size":0,"ondisk_log_size":7171,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29348806,"num_objects":7167,"num_object_clones":0,"num_object_copies":14334,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7167,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5276,"num_write_kb":21104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'7173","reported_seq":10965,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T16:00:39.485822+0000","last_change":"2024-09-10T15:59:16.142612+0000","last_active":"2024-09-10T16:00:39.485822+0000","last_peered":"2024-09-10T16:00:39.485822+0000","last_clean":"2024-09-10T15:59:14.092473+0000","last_became_active":"2024-09-10T15:59:16.109611+0000","last_became_peered":"2024-09-10T15:59:16.109611+0000","last_unstale":"2024-09-10T16:00:39.485822+0000","last_undegraded":"2024-09-10T16:00:39.485822+0000","last_fullsized":"2024-09-10T15:59:15.770264+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":7173,"log_dups_size":0,"ondisk_log_size":7173,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29364224,"num_objects":7169,"num_object_clones":0,"num_object_copies":14338,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2434,"num_objects_unfound":0,"num_objects_dirty":7169,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5177,"num_write_kb":20708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1834,"num_bytes_recovered":7512064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2434},{"shards":"2","objects":4735}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'7182","reported_seq":7258,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:00:39.449797+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:00:39.449797+0000","last_peered":"2024-09-10T16:00:39.449797+0000","last_clean":"2024-09-10T16:00:39.449797+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:00:39.449797+0000","last_undegraded":"2024-09-10T16:00:39.449797+0000","last_fullsized":"2024-09-10T16:00:39.449797+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":7182,"log_dups_size":0,"ondisk_log_size":7182,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29417472,"num_objects":7182,"num_object_clones":0,"num_object_copies":14364,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7182,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7182,"num_write_kb":28728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'7334","reported_seq":7459,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:00:41.594985+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:00:41.594985+0000","last_peered":"2024-09-10T16:00:41.594985+0000","last_clean":"2024-09-10T16:00:41.594985+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:00:41.594985+0000","last_undegraded":"2024-09-10T16:00:41.594985+0000","last_fullsized":"2024-09-10T16:00:41.594985+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":7334,"log_dups_size":0,"ondisk_log_size":7334,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30011875,"num_objects":7328,"num_object_clones":0,"num_object_copies":14656,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7328,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4045,"num_write_kb":16180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'7347","reported_seq":7478,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:41.631079+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:00:41.631079+0000","last_peered":"2024-09-10T16:00:41.631079+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:00:41.631079+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":7347,"log_dups_size":0,"ondisk_log_size":7347,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30073315,"num_objects":7343,"num_object_clones":0,"num_object_copies":14686,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7343,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3959,"num_write_kb":15836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2972},{"shards":"0,2","objects":4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'7302","reported_seq":7435,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:41.508146+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:00:41.508146+0000","last_peered":"2024-09-10T16:00:41.508146+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:00:41.508146+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":7302,"log_dups_size":0,"ondisk_log_size":7302,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29885382,"num_objects":7298,"num_object_clones":0,"num_object_copies":14596,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7298,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3801,"num_write_kb":15204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3061},{"shards":"0,2","objects":4237}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'7427","reported_seq":7552,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:00:41.611297+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:00:41.611297+0000","last_peered":"2024-09-10T16:00:41.611297+0000","last_clean":"2024-09-10T16:00:41.611297+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:00:41.611297+0000","last_undegraded":"2024-09-10T16:00:41.611297+0000","last_fullsized":"2024-09-10T16:00:41.611297+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":7427,"log_dups_size":0,"ondisk_log_size":7427,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30412800,"num_objects":7425,"num_object_clones":0,"num_object_copies":14850,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7425,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4274,"num_write_kb":17096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.775865+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T15:59:15.775865+0000","last_peered":"2024-09-10T15:59:15.775865+0000","last_clean":"2024-09-10T15:59:15.775865+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T15:59:15.775865+0000","last_undegraded":"2024-09-10T15:59:15.775865+0000","last_fullsized":"2024-09-10T15:59:15.775865+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'6908","reported_seq":7043,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:39.426820+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:00:39.426820+0000","last_peered":"2024-09-10T16:00:39.426820+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:00:39.426820+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":6908,"log_dups_size":0,"ondisk_log_size":6908,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28266979,"num_objects":6902,"num_object_clones":0,"num_object_copies":13804,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6902,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4727,"num_write_kb":18908,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4123},{"shards":"2","objects":2779}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":89,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.772619+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T15:59:15.772619+0000","last_peered":"2024-09-10T15:59:15.772619+0000","last_clean":"2024-09-10T15:59:15.772619+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T15:59:15.772619+0000","last_undegraded":"2024-09-10T15:59:15.772619+0000","last_fullsized":"2024-09-10T15:59:15.772619+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'7254","reported_seq":7378,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:41.606441+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:00:41.606441+0000","last_peered":"2024-09-10T16:00:41.606441+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:00:41.606441+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":7254,"log_dups_size":0,"ondisk_log_size":7254,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29679616,"num_objects":7246,"num_object_clones":0,"num_object_copies":14492,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7246,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3758,"num_write_kb":15032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2991},{"shards":"0,2","objects":4255}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":99,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.884297+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T15:59:15.884297+0000","last_peered":"2024-09-10T15:59:15.884297+0000","last_clean":"2024-09-10T15:59:15.884297+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T15:59:15.884297+0000","last_undegraded":"2024-09-10T15:59:15.884297+0000","last_fullsized":"2024-09-10T15:59:15.884297+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'7239","reported_seq":7366,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:39.445569+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:00:39.445569+0000","last_peered":"2024-09-10T16:00:39.445569+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:00:39.445569+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":7239,"log_dups_size":0,"ondisk_log_size":7239,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29630947,"num_objects":7235,"num_object_clones":0,"num_object_copies":14470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7235,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4919,"num_write_kb":19676,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4304},{"shards":"2","objects":2931}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.837509+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T15:59:15.837509+0000","last_peered":"2024-09-10T15:59:15.837509+0000","last_clean":"2024-09-10T15:59:15.837509+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T15:59:15.837509+0000","last_undegraded":"2024-09-10T15:59:15.837509+0000","last_fullsized":"2024-09-10T15:59:15.837509+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'7329","reported_seq":7452,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:39.454784+0000","last_change":"2024-09-10T15:59:16.174113+0000","last_active":"2024-09-10T16:00:39.454784+0000","last_peered":"2024-09-10T16:00:39.454784+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:00:39.454784+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":7329,"log_dups_size":0,"ondisk_log_size":7329,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29995008,"num_objects":7323,"num_object_clones":0,"num_object_copies":14646,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4354,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7323,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4883,"num_write_kb":19532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4352},{"shards":"2","objects":2971}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'7260","reported_seq":7389,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:41.624606+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:00:41.624606+0000","last_peered":"2024-09-10T16:00:41.624606+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:00:41.624606+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":7260,"log_dups_size":0,"ondisk_log_size":7260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29728768,"num_objects":7258,"num_object_clones":0,"num_object_copies":14516,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7258,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3891,"num_write_kb":15564,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2954},{"shards":"0,2","objects":4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.989413+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T15:59:15.989413+0000","last_peered":"2024-09-10T15:59:15.989413+0000","last_clean":"2024-09-10T15:59:15.989413+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T15:59:15.989413+0000","last_undegraded":"2024-09-10T15:59:15.989413+0000","last_fullsized":"2024-09-10T15:59:15.989413+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'7092","reported_seq":7230,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:41.647449+0000","last_change":"2024-09-10T15:59:16.251610+0000","last_active":"2024-09-10T16:00:41.647449+0000","last_peered":"2024-09-10T16:00:41.647449+0000","last_clean":"2024-09-10T15:59:14.115595+0000","last_became_active":"2024-09-10T15:59:16.244089+0000","last_became_peered":"2024-09-10T15:59:16.244089+0000","last_unstale":"2024-09-10T16:00:41.647449+0000","last_undegraded":"2024-09-10T15:59:15.847127+0000","last_fullsized":"2024-09-10T15:59:15.774437+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":7092,"log_dups_size":0,"ondisk_log_size":7092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29016064,"num_objects":7084,"num_object_clones":0,"num_object_copies":14168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7084,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3482,"num_write_kb":13928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2881},{"shards":"0,2","objects":4203}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":89,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.773633+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T15:59:15.773633+0000","last_peered":"2024-09-10T15:59:15.773633+0000","last_clean":"2024-09-10T15:59:15.773633+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T15:59:15.773633+0000","last_undegraded":"2024-09-10T15:59:15.773633+0000","last_fullsized":"2024-09-10T15:59:15.773633+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'7405","reported_seq":7542,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:41.625270+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:00:41.625270+0000","last_peered":"2024-09-10T16:00:41.625270+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:00:41.625270+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":7405,"log_dups_size":0,"ondisk_log_size":7405,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30314496,"num_objects":7401,"num_object_clones":0,"num_object_copies":14802,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7401,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3660,"num_write_kb":14640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3030},{"shards":"0,2","objects":4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.916017+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T15:59:15.916017+0000","last_peered":"2024-09-10T15:59:15.916017+0000","last_clean":"2024-09-10T15:59:15.916017+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T15:59:15.916017+0000","last_undegraded":"2024-09-10T15:59:15.916017+0000","last_fullsized":"2024-09-10T15:59:15.916017+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.776033+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T15:59:15.776033+0000","last_peered":"2024-09-10T15:59:15.776033+0000","last_clean":"2024-09-10T15:59:15.776033+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T15:59:15.776033+0000","last_undegraded":"2024-09-10T15:59:15.776033+0000","last_fullsized":"2024-09-10T15:59:15.776033+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'7389","reported_seq":7527,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:41.653040+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:00:41.653040+0000","last_peered":"2024-09-10T16:00:41.653040+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:00:41.653040+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":7389,"log_dups_size":0,"ondisk_log_size":7389,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30217158,"num_objects":7379,"num_object_clones":0,"num_object_copies":14758,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7379,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3552,"num_write_kb":14208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3056},{"shards":"0,2","objects":4323}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":474882782,"num_objects":115947,"num_object_clones":0,"num_object_copies":231894,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":47117,"num_objects_misplaced":2434,"num_objects_unfound":0,"num_objects_dirty":115947,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":73793,"num_write_kb":295172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1837,"num_bytes_recovered":7524352,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":953733120,"data_stored":953577761,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":116019,"ondisk_log_size":116019,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986208,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":547320,"kb_used_data":367080,"kb_used_omap":4,"kb_used_meta":180219,"kb_avail":93824520,"statfs":{"total":96636764160,"available":96076308480,"internally_reserved":0,"allocated":375889920,"data_stored":375612160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":184544938},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,0,3,1,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659154,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":293500,"kb_used_data":145196,"kb_used_omap":4,"kb_used_meta":148283,"kb_avail":94078340,"statfs":{"total":96636764160,"available":96336220160,"internally_reserved":0,"allocated":148680704,"data_stored":148351898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":151842478},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,1,4],"upper_bound":32},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182437,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":683832,"kb_used_data":417576,"kb_used_omap":4,"kb_used_meta":266235,"kb_avail":93688008,"statfs":{"total":96636764160,"available":95936520192,"internally_reserved":0,"allocated":427597824,"data_stored":427315987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":272625322},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,6,2,4,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":428052480,"data_stored":428016350,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":149291008,"data_stored":149207909,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":376389632,"data_stored":376353502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:00:43.982 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 131 20 117419 117399 3.50012 1.79688 0.0248987 0.0223138 2024-09-10T16:00:44.982 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 132 20 117952 117932 3.48937 2.08203 0.0335594 0.0223812 2024-09-10T16:00:45.982 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 133 20 118483 118463 3.47873 2.07422 0.00942113 0.0224481 2024-09-10T16:00:46.944 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:00:46.985 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 134 20 119016 118996 3.46831 2.08203 0.0262585 0.0225184 2024-09-10T16:00:47.513 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:00:47.516 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:00:47.537 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":141,"stamp":"2024-09-10T16:00:45.571759+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":478589681,"num_objects":116854,"num_object_clones":0,"num_object_copies":233708,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":47117,"num_objects_misplaced":2316,"num_objects_unfound":0,"num_objects_dirty":116854,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":74700,"num_write_kb":298794,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1958,"num_bytes_recovered":8007699,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":116927,"ondisk_log_size":116927,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1533640,"kb_used_data":943000,"kb_used_omap":12,"kb_used_meta":590579,"kb_avail":281581880,"statfs":{"total":289910292480,"available":288339845120,"internally_reserved":0,"allocated":965632000,"data_stored":964747210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":604752898},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,8,8,9,7],"upper_bound":128},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":25120768,"num_objects":6133,"num_object_clones":0,"num_object_copies":12266,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-356,"num_objects_unfound":0,"num_objects_dirty":6133,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6133,"num_write_kb":24532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":357,"num_bytes_recovered":1458176,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004057"},"pg_stats":[{"pgid":"2.f","version":"43'7207","reported_seq":7276,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:41.606932+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:00:41.606932+0000","last_peered":"2024-09-10T16:00:41.606932+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:00:41.606932+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":7207,"log_dups_size":0,"ondisk_log_size":7207,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29519872,"num_objects":7207,"num_object_clones":0,"num_object_copies":14414,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7207,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7207,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2948},{"shards":"0,2","objects":4258}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'7337","reported_seq":7445,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:00:44.452209+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:00:44.452209+0000","last_peered":"2024-09-10T16:00:44.452209+0000","last_clean":"2024-09-10T16:00:44.452209+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:00:44.452209+0000","last_undegraded":"2024-09-10T16:00:44.452209+0000","last_fullsized":"2024-09-10T16:00:44.452209+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":7337,"log_dups_size":0,"ondisk_log_size":7337,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30028742,"num_objects":7333,"num_object_clones":0,"num_object_copies":14666,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7333,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5442,"num_write_kb":21768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'7319","reported_seq":11348,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T16:00:44.496433+0000","last_change":"2024-09-10T15:59:16.142612+0000","last_active":"2024-09-10T16:00:44.496433+0000","last_peered":"2024-09-10T16:00:44.496433+0000","last_clean":"2024-09-10T15:59:14.092473+0000","last_became_active":"2024-09-10T15:59:16.109611+0000","last_became_peered":"2024-09-10T15:59:16.109611+0000","last_unstale":"2024-09-10T16:00:44.496433+0000","last_undegraded":"2024-09-10T16:00:44.496433+0000","last_fullsized":"2024-09-10T15:59:15.770264+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":7319,"log_dups_size":0,"ondisk_log_size":7319,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29962240,"num_objects":7315,"num_object_clones":0,"num_object_copies":14630,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2316,"num_objects_unfound":0,"num_objects_dirty":7315,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5323,"num_write_kb":21292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1953,"num_bytes_recovered":7995392,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2316},{"shards":"2","objects":4999}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'7316","reported_seq":7392,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:00:44.477343+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:00:44.477343+0000","last_peered":"2024-09-10T16:00:44.477343+0000","last_clean":"2024-09-10T16:00:44.477343+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:00:44.477343+0000","last_undegraded":"2024-09-10T16:00:44.477343+0000","last_fullsized":"2024-09-10T16:00:44.477343+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":7316,"log_dups_size":0,"ondisk_log_size":7316,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29966336,"num_objects":7316,"num_object_clones":0,"num_object_copies":14632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7316,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7316,"num_write_kb":29264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'7334","reported_seq":7459,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:00:41.594985+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:00:41.594985+0000","last_peered":"2024-09-10T16:00:41.594985+0000","last_clean":"2024-09-10T16:00:41.594985+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:00:41.594985+0000","last_undegraded":"2024-09-10T16:00:41.594985+0000","last_fullsized":"2024-09-10T16:00:41.594985+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":7334,"log_dups_size":0,"ondisk_log_size":7334,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30011875,"num_objects":7328,"num_object_clones":0,"num_object_copies":14656,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7328,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4045,"num_write_kb":16180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'7347","reported_seq":7478,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:41.631079+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:00:41.631079+0000","last_peered":"2024-09-10T16:00:41.631079+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:00:41.631079+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":7347,"log_dups_size":0,"ondisk_log_size":7347,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30073315,"num_objects":7343,"num_object_clones":0,"num_object_copies":14686,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7343,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3959,"num_write_kb":15836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2972},{"shards":"0,2","objects":4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'7302","reported_seq":7435,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:41.508146+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:00:41.508146+0000","last_peered":"2024-09-10T16:00:41.508146+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:00:41.508146+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":7302,"log_dups_size":0,"ondisk_log_size":7302,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29885382,"num_objects":7298,"num_object_clones":0,"num_object_copies":14596,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7298,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3801,"num_write_kb":15204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3061},{"shards":"0,2","objects":4237}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'7427","reported_seq":7552,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:00:41.611297+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:00:41.611297+0000","last_peered":"2024-09-10T16:00:41.611297+0000","last_clean":"2024-09-10T16:00:41.611297+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:00:41.611297+0000","last_undegraded":"2024-09-10T16:00:41.611297+0000","last_fullsized":"2024-09-10T16:00:41.611297+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":7427,"log_dups_size":0,"ondisk_log_size":7427,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30412800,"num_objects":7425,"num_object_clones":0,"num_object_copies":14850,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7425,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4274,"num_write_kb":17096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.775865+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T15:59:15.775865+0000","last_peered":"2024-09-10T15:59:15.775865+0000","last_clean":"2024-09-10T15:59:15.775865+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T15:59:15.775865+0000","last_undegraded":"2024-09-10T15:59:15.775865+0000","last_fullsized":"2024-09-10T15:59:15.775865+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'7056","reported_seq":7191,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:44.358227+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:00:44.358227+0000","last_peered":"2024-09-10T16:00:44.358227+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:00:44.358227+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":7056,"log_dups_size":0,"ondisk_log_size":7056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28873187,"num_objects":7050,"num_object_clones":0,"num_object_copies":14100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7050,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4875,"num_write_kb":19500,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4123},{"shards":"2","objects":2927}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":89,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.772619+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T15:59:15.772619+0000","last_peered":"2024-09-10T15:59:15.772619+0000","last_clean":"2024-09-10T15:59:15.772619+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T15:59:15.772619+0000","last_undegraded":"2024-09-10T15:59:15.772619+0000","last_fullsized":"2024-09-10T15:59:15.772619+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'7254","reported_seq":7378,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:41.606441+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:00:41.606441+0000","last_peered":"2024-09-10T16:00:41.606441+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:00:41.606441+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":7254,"log_dups_size":0,"ondisk_log_size":7254,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29679616,"num_objects":7246,"num_object_clones":0,"num_object_copies":14492,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7246,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3758,"num_write_kb":15032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2991},{"shards":"0,2","objects":4255}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":99,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.884297+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T15:59:15.884297+0000","last_peered":"2024-09-10T15:59:15.884297+0000","last_clean":"2024-09-10T15:59:15.884297+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T15:59:15.884297+0000","last_undegraded":"2024-09-10T15:59:15.884297+0000","last_fullsized":"2024-09-10T15:59:15.884297+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'7381","reported_seq":7511,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:44.432552+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:00:44.432552+0000","last_peered":"2024-09-10T16:00:44.432552+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:00:44.432552+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":7381,"log_dups_size":0,"ondisk_log_size":7381,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30212579,"num_objects":7377,"num_object_clones":0,"num_object_copies":14754,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7377,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5061,"num_write_kb":20244,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4304},{"shards":"2","objects":3073}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.837509+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T15:59:15.837509+0000","last_peered":"2024-09-10T15:59:15.837509+0000","last_clean":"2024-09-10T15:59:15.837509+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T15:59:15.837509+0000","last_undegraded":"2024-09-10T15:59:15.837509+0000","last_fullsized":"2024-09-10T15:59:15.837509+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'7498","reported_seq":7622,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:44.418700+0000","last_change":"2024-09-10T15:59:16.174113+0000","last_active":"2024-09-10T16:00:44.418700+0000","last_peered":"2024-09-10T16:00:44.418700+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:00:44.418700+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":7498,"log_dups_size":0,"ondisk_log_size":7498,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30687232,"num_objects":7492,"num_object_clones":0,"num_object_copies":14984,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4354,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7492,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5052,"num_write_kb":20208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4352},{"shards":"2","objects":3140}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'7260","reported_seq":7389,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:41.624606+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:00:41.624606+0000","last_peered":"2024-09-10T16:00:41.624606+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:00:41.624606+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":7260,"log_dups_size":0,"ondisk_log_size":7260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29728768,"num_objects":7258,"num_object_clones":0,"num_object_copies":14516,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7258,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3891,"num_write_kb":15564,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2954},{"shards":"0,2","objects":4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.989413+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T15:59:15.989413+0000","last_peered":"2024-09-10T15:59:15.989413+0000","last_clean":"2024-09-10T15:59:15.989413+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T15:59:15.989413+0000","last_undegraded":"2024-09-10T15:59:15.989413+0000","last_fullsized":"2024-09-10T15:59:15.989413+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'7092","reported_seq":7230,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:41.647449+0000","last_change":"2024-09-10T15:59:16.251610+0000","last_active":"2024-09-10T16:00:41.647449+0000","last_peered":"2024-09-10T16:00:41.647449+0000","last_clean":"2024-09-10T15:59:14.115595+0000","last_became_active":"2024-09-10T15:59:16.244089+0000","last_became_peered":"2024-09-10T15:59:16.244089+0000","last_unstale":"2024-09-10T16:00:41.647449+0000","last_undegraded":"2024-09-10T15:59:15.847127+0000","last_fullsized":"2024-09-10T15:59:15.774437+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":7092,"log_dups_size":0,"ondisk_log_size":7092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29016064,"num_objects":7084,"num_object_clones":0,"num_object_copies":14168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7084,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3482,"num_write_kb":13928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2881},{"shards":"0,2","objects":4203}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":89,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.773633+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T15:59:15.773633+0000","last_peered":"2024-09-10T15:59:15.773633+0000","last_clean":"2024-09-10T15:59:15.773633+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T15:59:15.773633+0000","last_undegraded":"2024-09-10T15:59:15.773633+0000","last_fullsized":"2024-09-10T15:59:15.773633+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'7405","reported_seq":7542,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:41.625270+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:00:41.625270+0000","last_peered":"2024-09-10T16:00:41.625270+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:00:41.625270+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":7405,"log_dups_size":0,"ondisk_log_size":7405,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30314496,"num_objects":7401,"num_object_clones":0,"num_object_copies":14802,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7401,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3660,"num_write_kb":14640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3030},{"shards":"0,2","objects":4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.916017+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T15:59:15.916017+0000","last_peered":"2024-09-10T15:59:15.916017+0000","last_clean":"2024-09-10T15:59:15.916017+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T15:59:15.916017+0000","last_undegraded":"2024-09-10T15:59:15.916017+0000","last_fullsized":"2024-09-10T15:59:15.916017+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.776033+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T15:59:15.776033+0000","last_peered":"2024-09-10T15:59:15.776033+0000","last_clean":"2024-09-10T15:59:15.776033+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T15:59:15.776033+0000","last_undegraded":"2024-09-10T15:59:15.776033+0000","last_fullsized":"2024-09-10T15:59:15.776033+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'7389","reported_seq":7527,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:41.653040+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:00:41.653040+0000","last_peered":"2024-09-10T16:00:41.653040+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:00:41.653040+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":7389,"log_dups_size":0,"ondisk_log_size":7389,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30217158,"num_objects":7379,"num_object_clones":0,"num_object_copies":14758,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7379,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3552,"num_write_kb":14208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3056},{"shards":"0,2","objects":4323}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":478589662,"num_objects":116852,"num_object_clones":0,"num_object_copies":233704,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":47117,"num_objects_misplaced":2316,"num_objects_unfound":0,"num_objects_dirty":116852,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":74698,"num_write_kb":298792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1956,"num_bytes_recovered":8007680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":967122944,"data_stored":966971198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":116924,"ondisk_log_size":116924,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986209,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":556480,"kb_used_data":372080,"kb_used_omap":4,"kb_used_meta":184379,"kb_avail":93815360,"statfs":{"total":96636764160,"available":96066928640,"internally_reserved":0,"allocated":381009920,"data_stored":380732160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":188804778},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,3,1,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659155,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":293328,"kb_used_data":153344,"kb_used_omap":4,"kb_used_meta":139963,"kb_avail":94078512,"statfs":{"total":96636764160,"available":96336396288,"internally_reserved":0,"allocated":157024256,"data_stored":156699063,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":143322798},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,2,3,4,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182437,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":683832,"kb_used_data":417576,"kb_used_omap":4,"kb_used_meta":266235,"kb_avail":93688008,"statfs":{"total":96636764160,"available":95936520192,"internally_reserved":0,"allocated":427597824,"data_stored":427315987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":272625322},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,6,2,4,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":428052480,"data_stored":428016350,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":157622272,"data_stored":157542786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":381448192,"data_stored":381412062,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:00:47.539 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:00:47.982 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 135 20 119481 119461 3.45607 1.81641 0.0260409 0.0225974 2024-09-10T16:00:48.112 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:00:48.114 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:00:48.142 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":142,"stamp":"2024-09-10T16:00:47.572598+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":485069553,"num_objects":118436,"num_object_clones":0,"num_object_copies":236872,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":47117,"num_objects_misplaced":2316,"num_objects_unfound":0,"num_objects_dirty":118436,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":76282,"num_write_kb":305122,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1958,"num_bytes_recovered":8007699,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":118509,"ondisk_log_size":118509,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1545228,"kb_used_data":950428,"kb_used_omap":12,"kb_used_meta":594739,"kb_avail":281570292,"statfs":{"total":289910292480,"available":288327979008,"internally_reserved":0,"allocated":973238272,"data_stored":972353482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":609012738},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,3,0,2,10,13,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":27746304,"num_objects":6774,"num_object_clones":0,"num_object_copies":13548,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-237,"num_objects_unfound":0,"num_objects_dirty":6774,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6774,"num_write_kb":27096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":238,"num_bytes_recovered":970752,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004280"},"pg_stats":[{"pgid":"2.f","version":"43'7378","reported_seq":7448,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:46.675714+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:00:46.675714+0000","last_peered":"2024-09-10T16:00:46.675714+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:00:46.675714+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":7378,"log_dups_size":0,"ondisk_log_size":7378,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30220288,"num_objects":7378,"num_object_clones":0,"num_object_copies":14756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7378,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7378,"num_write_kb":29512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3119},{"shards":"0,2","objects":4258}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'7337","reported_seq":7445,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:00:44.452209+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:00:44.452209+0000","last_peered":"2024-09-10T16:00:44.452209+0000","last_clean":"2024-09-10T16:00:44.452209+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:00:44.452209+0000","last_undegraded":"2024-09-10T16:00:44.452209+0000","last_fullsized":"2024-09-10T16:00:44.452209+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":7337,"log_dups_size":0,"ondisk_log_size":7337,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30028742,"num_objects":7333,"num_object_clones":0,"num_object_copies":14666,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7333,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5442,"num_write_kb":21768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'7319","reported_seq":11348,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T16:00:44.496433+0000","last_change":"2024-09-10T15:59:16.142612+0000","last_active":"2024-09-10T16:00:44.496433+0000","last_peered":"2024-09-10T16:00:44.496433+0000","last_clean":"2024-09-10T15:59:14.092473+0000","last_became_active":"2024-09-10T15:59:16.109611+0000","last_became_peered":"2024-09-10T15:59:16.109611+0000","last_unstale":"2024-09-10T16:00:44.496433+0000","last_undegraded":"2024-09-10T16:00:44.496433+0000","last_fullsized":"2024-09-10T15:59:15.770264+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":7319,"log_dups_size":0,"ondisk_log_size":7319,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29962240,"num_objects":7315,"num_object_clones":0,"num_object_copies":14630,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2316,"num_objects_unfound":0,"num_objects_dirty":7315,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5323,"num_write_kb":21292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1953,"num_bytes_recovered":7995392,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2316},{"shards":"2","objects":4999}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'7316","reported_seq":7392,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:00:44.477343+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:00:44.477343+0000","last_peered":"2024-09-10T16:00:44.477343+0000","last_clean":"2024-09-10T16:00:44.477343+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:00:44.477343+0000","last_undegraded":"2024-09-10T16:00:44.477343+0000","last_fullsized":"2024-09-10T16:00:44.477343+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":7316,"log_dups_size":0,"ondisk_log_size":7316,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29966336,"num_objects":7316,"num_object_clones":0,"num_object_copies":14632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7316,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7316,"num_write_kb":29264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'7487","reported_seq":7612,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:00:46.682514+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:00:46.682514+0000","last_peered":"2024-09-10T16:00:46.682514+0000","last_clean":"2024-09-10T16:00:46.682514+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:00:46.682514+0000","last_undegraded":"2024-09-10T16:00:46.682514+0000","last_fullsized":"2024-09-10T16:00:46.682514+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":7487,"log_dups_size":0,"ondisk_log_size":7487,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30638563,"num_objects":7481,"num_object_clones":0,"num_object_copies":14962,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7481,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4198,"num_write_kb":16792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'7506","reported_seq":7639,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:46.652109+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:00:46.652109+0000","last_peered":"2024-09-10T16:00:46.652109+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:00:46.652109+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":7506,"log_dups_size":0,"ondisk_log_size":7506,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30724579,"num_objects":7502,"num_object_clones":0,"num_object_copies":15004,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7502,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4118,"num_write_kb":16472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3131},{"shards":"0,2","objects":4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'7437","reported_seq":7569,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:46.599254+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:00:46.599254+0000","last_peered":"2024-09-10T16:00:46.599254+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:00:46.599254+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":7437,"log_dups_size":0,"ondisk_log_size":7437,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30438342,"num_objects":7433,"num_object_clones":0,"num_object_copies":14866,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7433,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3936,"num_write_kb":15744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3196},{"shards":"0,2","objects":4237}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'7573","reported_seq":7698,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:00:46.680035+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:00:46.680035+0000","last_peered":"2024-09-10T16:00:46.680035+0000","last_clean":"2024-09-10T16:00:46.680035+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:00:46.680035+0000","last_undegraded":"2024-09-10T16:00:46.680035+0000","last_fullsized":"2024-09-10T16:00:46.680035+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":7573,"log_dups_size":0,"ondisk_log_size":7573,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31010816,"num_objects":7571,"num_object_clones":0,"num_object_copies":15142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7571,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4420,"num_write_kb":17680,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.775865+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T15:59:15.775865+0000","last_peered":"2024-09-10T15:59:15.775865+0000","last_clean":"2024-09-10T15:59:15.775865+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T15:59:15.775865+0000","last_undegraded":"2024-09-10T15:59:15.775865+0000","last_fullsized":"2024-09-10T15:59:15.775865+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'7056","reported_seq":7191,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:44.358227+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:00:44.358227+0000","last_peered":"2024-09-10T16:00:44.358227+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:00:44.358227+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":7056,"log_dups_size":0,"ondisk_log_size":7056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28873187,"num_objects":7050,"num_object_clones":0,"num_object_copies":14100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7050,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4875,"num_write_kb":19500,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4123},{"shards":"2","objects":2927}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":89,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.772619+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T15:59:15.772619+0000","last_peered":"2024-09-10T15:59:15.772619+0000","last_clean":"2024-09-10T15:59:15.772619+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T15:59:15.772619+0000","last_undegraded":"2024-09-10T15:59:15.772619+0000","last_fullsized":"2024-09-10T15:59:15.772619+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'7414","reported_seq":7538,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:46.611722+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:00:46.611722+0000","last_peered":"2024-09-10T16:00:46.611722+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:00:46.611722+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":7414,"log_dups_size":0,"ondisk_log_size":7414,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30334976,"num_objects":7406,"num_object_clones":0,"num_object_copies":14812,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7406,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3918,"num_write_kb":15672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3151},{"shards":"0,2","objects":4255}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":99,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.884297+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T15:59:15.884297+0000","last_peered":"2024-09-10T15:59:15.884297+0000","last_clean":"2024-09-10T15:59:15.884297+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T15:59:15.884297+0000","last_undegraded":"2024-09-10T15:59:15.884297+0000","last_fullsized":"2024-09-10T15:59:15.884297+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'7381","reported_seq":7511,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:44.432552+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:00:44.432552+0000","last_peered":"2024-09-10T16:00:44.432552+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:00:44.432552+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":7381,"log_dups_size":0,"ondisk_log_size":7381,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30212579,"num_objects":7377,"num_object_clones":0,"num_object_copies":14754,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7377,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5061,"num_write_kb":20244,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4304},{"shards":"2","objects":3073}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.837509+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T15:59:15.837509+0000","last_peered":"2024-09-10T15:59:15.837509+0000","last_clean":"2024-09-10T15:59:15.837509+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T15:59:15.837509+0000","last_undegraded":"2024-09-10T15:59:15.837509+0000","last_fullsized":"2024-09-10T15:59:15.837509+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'7498","reported_seq":7622,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:44.418700+0000","last_change":"2024-09-10T15:59:16.174113+0000","last_active":"2024-09-10T16:00:44.418700+0000","last_peered":"2024-09-10T16:00:44.418700+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:00:44.418700+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":7498,"log_dups_size":0,"ondisk_log_size":7498,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30687232,"num_objects":7492,"num_object_clones":0,"num_object_copies":14984,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4354,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7492,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5052,"num_write_kb":20208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4352},{"shards":"2","objects":3140}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'7428","reported_seq":7558,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:46.628123+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:00:46.628123+0000","last_peered":"2024-09-10T16:00:46.628123+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:00:46.628123+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":7428,"log_dups_size":0,"ondisk_log_size":7428,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30416896,"num_objects":7426,"num_object_clones":0,"num_object_copies":14852,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7426,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4059,"num_write_kb":16236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3122},{"shards":"0,2","objects":4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.989413+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T15:59:15.989413+0000","last_peered":"2024-09-10T15:59:15.989413+0000","last_clean":"2024-09-10T15:59:15.989413+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T15:59:15.989413+0000","last_undegraded":"2024-09-10T15:59:15.989413+0000","last_fullsized":"2024-09-10T15:59:15.989413+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'7264","reported_seq":7402,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:46.652641+0000","last_change":"2024-09-10T15:59:16.251610+0000","last_active":"2024-09-10T16:00:46.652641+0000","last_peered":"2024-09-10T16:00:46.652641+0000","last_clean":"2024-09-10T15:59:14.115595+0000","last_became_active":"2024-09-10T15:59:16.244089+0000","last_became_peered":"2024-09-10T15:59:16.244089+0000","last_unstale":"2024-09-10T16:00:46.652641+0000","last_undegraded":"2024-09-10T15:59:15.847127+0000","last_fullsized":"2024-09-10T15:59:15.774437+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":7264,"log_dups_size":0,"ondisk_log_size":7264,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29720576,"num_objects":7256,"num_object_clones":0,"num_object_copies":14512,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7256,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3654,"num_write_kb":14616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3053},{"shards":"0,2","objects":4203}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":89,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.773633+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T15:59:15.773633+0000","last_peered":"2024-09-10T15:59:15.773633+0000","last_clean":"2024-09-10T15:59:15.773633+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T15:59:15.773633+0000","last_undegraded":"2024-09-10T15:59:15.773633+0000","last_fullsized":"2024-09-10T15:59:15.773633+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'7569","reported_seq":7706,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:46.677404+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:00:46.677404+0000","last_peered":"2024-09-10T16:00:46.677404+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:00:46.677404+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":7569,"log_dups_size":0,"ondisk_log_size":7569,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30986240,"num_objects":7565,"num_object_clones":0,"num_object_copies":15130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7565,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3824,"num_write_kb":15296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3194},{"shards":"0,2","objects":4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.916017+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T15:59:15.916017+0000","last_peered":"2024-09-10T15:59:15.916017+0000","last_clean":"2024-09-10T15:59:15.916017+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T15:59:15.916017+0000","last_undegraded":"2024-09-10T15:59:15.916017+0000","last_fullsized":"2024-09-10T15:59:15.916017+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.776033+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T15:59:15.776033+0000","last_peered":"2024-09-10T15:59:15.776033+0000","last_clean":"2024-09-10T15:59:15.776033+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T15:59:15.776033+0000","last_undegraded":"2024-09-10T15:59:15.776033+0000","last_fullsized":"2024-09-10T15:59:15.776033+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'7543","reported_seq":7682,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:46.549983+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:00:46.549983+0000","last_peered":"2024-09-10T16:00:46.549983+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:00:46.549983+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":7543,"log_dups_size":0,"ondisk_log_size":7543,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30847942,"num_objects":7533,"num_object_clones":0,"num_object_copies":15066,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7533,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3706,"num_write_kb":14824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3210},{"shards":"0,2","objects":4323}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":485069534,"num_objects":118434,"num_object_clones":0,"num_object_copies":236868,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":47117,"num_objects_misplaced":2316,"num_objects_unfound":0,"num_objects_dirty":118434,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":76280,"num_write_kb":305120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1956,"num_bytes_recovered":8007680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":974880768,"data_stored":974729022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":118506,"ondisk_log_size":118506,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986209,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":556480,"kb_used_data":372080,"kb_used_omap":4,"kb_used_meta":184379,"kb_avail":93815360,"statfs":{"total":96636764160,"available":96066928640,"internally_reserved":0,"allocated":381009920,"data_stored":380732160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":188804778},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,3,1,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659155,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":293328,"kb_used_data":153344,"kb_used_omap":4,"kb_used_meta":139963,"kb_avail":94078512,"statfs":{"total":96636764160,"available":96336396288,"internally_reserved":0,"allocated":157024256,"data_stored":156699063,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":143322798},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,2,3,4,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182438,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":695420,"kb_used_data":425004,"kb_used_omap":4,"kb_used_meta":270395,"kb_avail":93676420,"statfs":{"total":96636764160,"available":95924654080,"internally_reserved":0,"allocated":435204096,"data_stored":434922259,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":276885162},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3,0,0,4,8,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":435810304,"data_stored":435774174,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":157622272,"data_stored":157542786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":381448192,"data_stored":381412062,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:00:48.144 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-10T16:00:48.144 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "43'7378", 'reported_seq': 7448, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:00:46.675714+0000', 'last_change': '2024-09-10T15:59:16.281558+0000', 'last_active': '2024-09-10T16:00:46.675714+0000', 'last_peered': '2024-09-10T16:00:46.675714+0000', 'last_clean': '2024-09-10T15:59:14.132456+0000', 'last_became_active': '2024-09-10T15:59:16.274407+0000', 'last_became_peered': '2024-09-10T15:59:16.274407+0000', 'last_unstale': '2024-09-10T16:00:46.675714+0000', 'last_undegraded': '2024-09-10T15:59:15.851279+0000', 'last_fullsized': '2024-09-10T15:59:15.774356+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'objects_scrubbed': 0, 'log_size': 7378, 'log_dups_size': 0, 'ondisk_log_size': 7378, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30220288, 'num_objects': 7378, 'num_object_clones': 0, 'num_object_copies': 14756, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4259, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7378, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7378, 'num_write_kb': 29512, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3119}, {'shards': '0,2', 'objects': 4258}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:00:48.144 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-10T16:00:48.144 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "43'7319", 'reported_seq': 11348, 'reported_epoch': 43, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-10T16:00:44.496433+0000', 'last_change': '2024-09-10T15:59:16.142612+0000', 'last_active': '2024-09-10T16:00:44.496433+0000', 'last_peered': '2024-09-10T16:00:44.496433+0000', 'last_clean': '2024-09-10T15:59:14.092473+0000', 'last_became_active': '2024-09-10T15:59:16.109611+0000', 'last_became_peered': '2024-09-10T15:59:16.109611+0000', 'last_unstale': '2024-09-10T16:00:44.496433+0000', 'last_undegraded': '2024-09-10T16:00:44.496433+0000', 'last_fullsized': '2024-09-10T15:59:15.770264+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2029", 'last_scrub_stamp': '2024-09-10T15:58:45.933789+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:45.933789+0000', 'objects_scrubbed': 1992, 'log_size': 7319, 'log_dups_size': 0, 'ondisk_log_size': 7319, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 668, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29962240, 'num_objects': 7315, 'num_object_clones': 0, 'num_object_copies': 14630, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 2316, 'num_objects_unfound': 0, 'num_objects_dirty': 7315, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5323, 'num_write_kb': 21292, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1953, 'num_bytes_recovered': 7995392, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2316}, {'shards': '2', 'objects': 4999}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:00:48.144 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-10T16:00:48.144 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "43'7506", 'reported_seq': 7639, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:00:46.652109+0000', 'last_change': '2024-09-10T15:59:16.308264+0000', 'last_active': '2024-09-10T16:00:46.652109+0000', 'last_peered': '2024-09-10T16:00:46.652109+0000', 'last_clean': '2024-09-10T15:59:14.111551+0000', 'last_became_active': '2024-09-10T15:59:16.300040+0000', 'last_became_peered': '2024-09-10T15:59:16.300040+0000', 'last_unstale': '2024-09-10T16:00:46.652109+0000', 'last_undegraded': '2024-09-10T15:59:15.878234+0000', 'last_fullsized': '2024-09-10T15:59:15.769328+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3442", 'last_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'objects_scrubbed': 3384, 'log_size': 7506, 'log_dups_size': 0, 'ondisk_log_size': 7506, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1292, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30724579, 'num_objects': 7502, 'num_object_clones': 0, 'num_object_copies': 15004, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4372, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7502, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 4118, 'num_write_kb': 16472, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3131}, {'shards': '0,2', 'objects': 4371}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:00:48.145 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-10T16:00:48.145 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "43'7437", 'reported_seq': 7569, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:00:46.599254+0000', 'last_change': '2024-09-10T15:59:16.331899+0000', 'last_active': '2024-09-10T16:00:46.599254+0000', 'last_peered': '2024-09-10T16:00:46.599254+0000', 'last_clean': '2024-09-10T15:59:14.139436+0000', 'last_became_active': '2024-09-10T15:59:16.318715+0000', 'last_became_peered': '2024-09-10T15:59:16.318715+0000', 'last_unstale': '2024-09-10T16:00:46.599254+0000', 'last_undegraded': '2024-09-10T15:59:15.877524+0000', 'last_fullsized': '2024-09-10T15:59:15.769042+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3537", 'last_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'objects_scrubbed': 3497, 'log_size': 7437, 'log_dups_size': 0, 'ondisk_log_size': 7437, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1224, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30438342, 'num_objects': 7433, 'num_object_clones': 0, 'num_object_copies': 14866, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4237, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7433, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 3936, 'num_write_kb': 15744, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3196}, {'shards': '0,2', 'objects': 4237}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:00:48.145 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-10T16:00:48.145 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "43'7056", 'reported_seq': 7191, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:00:44.358227+0000', 'last_change': '2024-09-10T15:59:16.175680+0000', 'last_active': '2024-09-10T16:00:44.358227+0000', 'last_peered': '2024-09-10T16:00:44.358227+0000', 'last_clean': '2024-09-10T15:59:14.088187+0000', 'last_became_active': '2024-09-10T15:59:16.165810+0000', 'last_became_peered': '2024-09-10T15:59:16.165810+0000', 'last_unstale': '2024-09-10T16:00:44.358227+0000', 'last_undegraded': '2024-09-10T15:59:15.822098+0000', 'last_fullsized': '2024-09-10T15:59:15.769123+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2218", 'last_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'objects_scrubbed': 2175, 'log_size': 7056, 'log_dups_size': 0, 'ondisk_log_size': 7056, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 825, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28873187, 'num_objects': 7050, 'num_object_clones': 0, 'num_object_copies': 14100, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4124, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7050, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 4875, 'num_write_kb': 19500, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4123}, {'shards': '2', 'objects': 2927}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:00:48.145 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-10T16:00:48.145 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "43'7414", 'reported_seq': 7538, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:00:46.611722+0000', 'last_change': '2024-09-10T15:59:16.277742+0000', 'last_active': '2024-09-10T16:00:46.611722+0000', 'last_peered': '2024-09-10T16:00:46.611722+0000', 'last_clean': '2024-09-10T15:59:14.129081+0000', 'last_became_active': '2024-09-10T15:59:16.269835+0000', 'last_became_peered': '2024-09-10T15:59:16.269835+0000', 'last_unstale': '2024-09-10T16:00:46.611722+0000', 'last_undegraded': '2024-09-10T15:59:15.860857+0000', 'last_fullsized': '2024-09-10T15:59:15.771410+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3557", 'last_scrub_stamp': '2024-09-10T15:59:02.493678+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:02.493678+0000', 'objects_scrubbed': 3488, 'log_size': 7414, 'log_dups_size': 0, 'ondisk_log_size': 7414, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1472, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30334976, 'num_objects': 7406, 'num_object_clones': 0, 'num_object_copies': 14812, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4257, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7406, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3918, 'num_write_kb': 15672, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3151}, {'shards': '0,2', 'objects': 4255}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:00:48.145 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-10T16:00:48.145 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "43'7381", 'reported_seq': 7511, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:00:44.432552+0000', 'last_change': '2024-09-10T15:59:16.284311+0000', 'last_active': '2024-09-10T16:00:44.432552+0000', 'last_peered': '2024-09-10T16:00:44.432552+0000', 'last_clean': '2024-09-10T15:59:14.116041+0000', 'last_became_active': '2024-09-10T15:59:16.274543+0000', 'last_became_peered': '2024-09-10T15:59:16.274543+0000', 'last_unstale': '2024-09-10T16:00:44.432552+0000', 'last_undegraded': '2024-09-10T15:59:15.812484+0000', 'last_fullsized': '2024-09-10T15:59:15.770885+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2356", 'last_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'objects_scrubbed': 2316, 'log_size': 7381, 'log_dups_size': 0, 'ondisk_log_size': 7381, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 783, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30212579, 'num_objects': 7377, 'num_object_clones': 0, 'num_object_copies': 14754, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7377, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 5061, 'num_write_kb': 20244, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4304}, {'shards': '2', 'objects': 3073}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:00:48.145 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-10T16:00:48.145 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "43'7498", 'reported_seq': 7622, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:00:44.418700+0000', 'last_change': '2024-09-10T15:59:16.174113+0000', 'last_active': '2024-09-10T16:00:44.418700+0000', 'last_peered': '2024-09-10T16:00:44.418700+0000', 'last_clean': '2024-09-10T15:59:14.110654+0000', 'last_became_active': '2024-09-10T15:59:16.164645+0000', 'last_became_peered': '2024-09-10T15:59:16.164645+0000', 'last_unstale': '2024-09-10T16:00:44.418700+0000', 'last_undegraded': '2024-09-10T15:59:15.803593+0000', 'last_fullsized': '2024-09-10T15:59:15.769083+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2486", 'last_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'objects_scrubbed': 2440, 'log_size': 7498, 'log_dups_size': 0, 'ondisk_log_size': 7498, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 792, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30687232, 'num_objects': 7492, 'num_object_clones': 0, 'num_object_copies': 14984, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4354, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7492, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5052, 'num_write_kb': 20208, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4352}, {'shards': '2', 'objects': 3140}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:00:48.146 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-10T16:00:48.146 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "43'7428", 'reported_seq': 7558, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:00:46.628123+0000', 'last_change': '2024-09-10T15:59:16.315641+0000', 'last_active': '2024-09-10T16:00:46.628123+0000', 'last_peered': '2024-09-10T16:00:46.628123+0000', 'last_clean': '2024-09-10T15:59:14.117001+0000', 'last_became_active': '2024-09-10T15:59:16.302245+0000', 'last_became_peered': '2024-09-10T15:59:16.302245+0000', 'last_unstale': '2024-09-10T16:00:46.628123+0000', 'last_undegraded': '2024-09-10T15:59:15.845012+0000', 'last_fullsized': '2024-09-10T15:59:15.770337+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3404", 'last_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'objects_scrubbed': 3367, 'log_size': 7428, 'log_dups_size': 0, 'ondisk_log_size': 7428, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 983, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30416896, 'num_objects': 7426, 'num_object_clones': 0, 'num_object_copies': 14852, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7426, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4059, 'num_write_kb': 16236, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3122}, {'shards': '0,2', 'objects': 4304}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:00:48.146 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-10T16:00:48.146 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "43'7264", 'reported_seq': 7402, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:00:46.652641+0000', 'last_change': '2024-09-10T15:59:16.251610+0000', 'last_active': '2024-09-10T16:00:46.652641+0000', 'last_peered': '2024-09-10T16:00:46.652641+0000', 'last_clean': '2024-09-10T15:59:14.115595+0000', 'last_became_active': '2024-09-10T15:59:16.244089+0000', 'last_became_peered': '2024-09-10T15:59:16.244089+0000', 'last_unstale': '2024-09-10T16:00:46.652641+0000', 'last_undegraded': '2024-09-10T15:59:15.847127+0000', 'last_fullsized': '2024-09-10T15:59:15.774437+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3653", 'last_scrub_stamp': '2024-09-10T15:59:05.298468+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:05.298468+0000', 'objects_scrubbed': 3602, 'log_size': 7264, 'log_dups_size': 0, 'ondisk_log_size': 7264, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1289, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29720576, 'num_objects': 7256, 'num_object_clones': 0, 'num_object_copies': 14512, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4206, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7256, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3654, 'num_write_kb': 14616, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3053}, {'shards': '0,2', 'objects': 4203}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:00:48.146 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-10T16:00:48.146 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "43'7569", 'reported_seq': 7706, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:00:46.677404+0000', 'last_change': '2024-09-10T15:59:16.311291+0000', 'last_active': '2024-09-10T16:00:46.677404+0000', 'last_peered': '2024-09-10T16:00:46.677404+0000', 'last_clean': '2024-09-10T15:59:14.116363+0000', 'last_became_active': '2024-09-10T15:59:16.301244+0000', 'last_became_peered': '2024-09-10T15:59:16.301244+0000', 'last_unstale': '2024-09-10T16:00:46.677404+0000', 'last_undegraded': '2024-09-10T15:59:15.883434+0000', 'last_fullsized': '2024-09-10T15:59:15.772756+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3789", 'last_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'objects_scrubbed': 3741, 'log_size': 7569, 'log_dups_size': 0, 'ondisk_log_size': 7569, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1440, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30986240, 'num_objects': 7565, 'num_object_clones': 0, 'num_object_copies': 15130, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4373, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7565, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3824, 'num_write_kb': 15296, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3194}, {'shards': '0,2', 'objects': 4371}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:00:48.146 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-10T16:00:48.146 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "43'7543", 'reported_seq': 7682, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:00:46.549983+0000', 'last_change': '2024-09-10T15:59:16.313409+0000', 'last_active': '2024-09-10T16:00:46.549983+0000', 'last_peered': '2024-09-10T16:00:46.549983+0000', 'last_clean': '2024-09-10T15:59:14.129780+0000', 'last_became_active': '2024-09-10T15:59:16.300932+0000', 'last_became_peered': '2024-09-10T15:59:16.300932+0000', 'last_unstale': '2024-09-10T16:00:46.549983+0000', 'last_undegraded': '2024-09-10T15:59:15.887823+0000', 'last_fullsized': '2024-09-10T15:59:15.768513+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3880", 'last_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'objects_scrubbed': 3827, 'log_size': 7543, 'log_dups_size': 0, 'ondisk_log_size': 7543, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1397, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30847942, 'num_objects': 7533, 'num_object_clones': 0, 'num_object_copies': 15066, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4325, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7533, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3706, 'num_write_kb': 14824, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3210}, {'shards': '0,2', 'objects': 4323}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:00:48.147 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T16:00:48.837 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:00:48.837 INFO:teuthology.orchestra.run.smithi115.stdout:{"fsid":"7effee29-df05-4351-9741-ee203564ed81","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 47117/236872 objects degraded (19.891%), 11 pgs degraded, 12 pgs undersized","count":23},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":212,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":43,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983954,"num_in_osds":3,"osd_in_since":1725983947,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":118436,"data_bytes":485069553,"bytes_used":1582313472,"bytes_avail":288327979008,"bytes_total":289910292480,"degraded_objects":47117,"degraded_total":236872,"degraded_ratio":0.19891333716099838,"misplaced_objects":2316,"misplaced_total":236872,"misplaced_ratio":0.0097774325374041671,"recovering_objects_per_sec":19,"recovering_bytes_per_sec":80867,"recovering_keys_per_sec":0,"num_objects_recovered":238,"num_bytes_recovered":970752,"num_keys_recovered":0,"write_bytes_sec":2311367,"read_op_per_sec":0,"write_op_per_sec":564},"fsmap":{"epoch":1,"btime":"2024-09-10T15:57:14:967120+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T16:00:13.560776+0000","services":{}},"progress_events":{}} 2024-09-10T16:00:48.863 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-10T16:00:48.863 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:00:48.983 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 136 20 119949 119929 3.4441 1.82812 0.0396837 0.0226753 2024-09-10T16:00:49.440 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:00:49.442 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:00:49.472 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":142,"stamp":"2024-09-10T16:00:47.572598+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":485069553,"num_objects":118436,"num_object_clones":0,"num_object_copies":236872,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":47117,"num_objects_misplaced":2316,"num_objects_unfound":0,"num_objects_dirty":118436,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":76282,"num_write_kb":305122,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1958,"num_bytes_recovered":8007699,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":118509,"ondisk_log_size":118509,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1545228,"kb_used_data":950428,"kb_used_omap":12,"kb_used_meta":594739,"kb_avail":281570292,"statfs":{"total":289910292480,"available":288327979008,"internally_reserved":0,"allocated":973238272,"data_stored":972353482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":609012738},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,3,0,2,10,13,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":27746304,"num_objects":6774,"num_object_clones":0,"num_object_copies":13548,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-237,"num_objects_unfound":0,"num_objects_dirty":6774,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6774,"num_write_kb":27096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":238,"num_bytes_recovered":970752,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004280"},"pg_stats":[{"pgid":"2.f","version":"43'7378","reported_seq":7448,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:46.675714+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:00:46.675714+0000","last_peered":"2024-09-10T16:00:46.675714+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:00:46.675714+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":7378,"log_dups_size":0,"ondisk_log_size":7378,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30220288,"num_objects":7378,"num_object_clones":0,"num_object_copies":14756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7378,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7378,"num_write_kb":29512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3119},{"shards":"0,2","objects":4258}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'7337","reported_seq":7445,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:00:44.452209+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:00:44.452209+0000","last_peered":"2024-09-10T16:00:44.452209+0000","last_clean":"2024-09-10T16:00:44.452209+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:00:44.452209+0000","last_undegraded":"2024-09-10T16:00:44.452209+0000","last_fullsized":"2024-09-10T16:00:44.452209+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":7337,"log_dups_size":0,"ondisk_log_size":7337,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30028742,"num_objects":7333,"num_object_clones":0,"num_object_copies":14666,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7333,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5442,"num_write_kb":21768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'7319","reported_seq":11348,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T16:00:44.496433+0000","last_change":"2024-09-10T15:59:16.142612+0000","last_active":"2024-09-10T16:00:44.496433+0000","last_peered":"2024-09-10T16:00:44.496433+0000","last_clean":"2024-09-10T15:59:14.092473+0000","last_became_active":"2024-09-10T15:59:16.109611+0000","last_became_peered":"2024-09-10T15:59:16.109611+0000","last_unstale":"2024-09-10T16:00:44.496433+0000","last_undegraded":"2024-09-10T16:00:44.496433+0000","last_fullsized":"2024-09-10T15:59:15.770264+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":7319,"log_dups_size":0,"ondisk_log_size":7319,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29962240,"num_objects":7315,"num_object_clones":0,"num_object_copies":14630,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2316,"num_objects_unfound":0,"num_objects_dirty":7315,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5323,"num_write_kb":21292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1953,"num_bytes_recovered":7995392,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2316},{"shards":"2","objects":4999}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'7316","reported_seq":7392,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:00:44.477343+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:00:44.477343+0000","last_peered":"2024-09-10T16:00:44.477343+0000","last_clean":"2024-09-10T16:00:44.477343+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:00:44.477343+0000","last_undegraded":"2024-09-10T16:00:44.477343+0000","last_fullsized":"2024-09-10T16:00:44.477343+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":7316,"log_dups_size":0,"ondisk_log_size":7316,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29966336,"num_objects":7316,"num_object_clones":0,"num_object_copies":14632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7316,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7316,"num_write_kb":29264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'7487","reported_seq":7612,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:00:46.682514+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:00:46.682514+0000","last_peered":"2024-09-10T16:00:46.682514+0000","last_clean":"2024-09-10T16:00:46.682514+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:00:46.682514+0000","last_undegraded":"2024-09-10T16:00:46.682514+0000","last_fullsized":"2024-09-10T16:00:46.682514+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":7487,"log_dups_size":0,"ondisk_log_size":7487,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30638563,"num_objects":7481,"num_object_clones":0,"num_object_copies":14962,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7481,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4198,"num_write_kb":16792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'7506","reported_seq":7639,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:46.652109+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:00:46.652109+0000","last_peered":"2024-09-10T16:00:46.652109+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:00:46.652109+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":7506,"log_dups_size":0,"ondisk_log_size":7506,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30724579,"num_objects":7502,"num_object_clones":0,"num_object_copies":15004,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7502,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4118,"num_write_kb":16472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3131},{"shards":"0,2","objects":4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'7437","reported_seq":7569,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:46.599254+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:00:46.599254+0000","last_peered":"2024-09-10T16:00:46.599254+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:00:46.599254+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":7437,"log_dups_size":0,"ondisk_log_size":7437,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30438342,"num_objects":7433,"num_object_clones":0,"num_object_copies":14866,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7433,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3936,"num_write_kb":15744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3196},{"shards":"0,2","objects":4237}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'7573","reported_seq":7698,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:00:46.680035+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:00:46.680035+0000","last_peered":"2024-09-10T16:00:46.680035+0000","last_clean":"2024-09-10T16:00:46.680035+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:00:46.680035+0000","last_undegraded":"2024-09-10T16:00:46.680035+0000","last_fullsized":"2024-09-10T16:00:46.680035+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":7573,"log_dups_size":0,"ondisk_log_size":7573,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31010816,"num_objects":7571,"num_object_clones":0,"num_object_copies":15142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7571,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4420,"num_write_kb":17680,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.775865+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T15:59:15.775865+0000","last_peered":"2024-09-10T15:59:15.775865+0000","last_clean":"2024-09-10T15:59:15.775865+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T15:59:15.775865+0000","last_undegraded":"2024-09-10T15:59:15.775865+0000","last_fullsized":"2024-09-10T15:59:15.775865+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'7056","reported_seq":7191,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:44.358227+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:00:44.358227+0000","last_peered":"2024-09-10T16:00:44.358227+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:00:44.358227+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":7056,"log_dups_size":0,"ondisk_log_size":7056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28873187,"num_objects":7050,"num_object_clones":0,"num_object_copies":14100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7050,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4875,"num_write_kb":19500,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4123},{"shards":"2","objects":2927}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":89,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.772619+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T15:59:15.772619+0000","last_peered":"2024-09-10T15:59:15.772619+0000","last_clean":"2024-09-10T15:59:15.772619+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T15:59:15.772619+0000","last_undegraded":"2024-09-10T15:59:15.772619+0000","last_fullsized":"2024-09-10T15:59:15.772619+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'7414","reported_seq":7538,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:46.611722+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:00:46.611722+0000","last_peered":"2024-09-10T16:00:46.611722+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:00:46.611722+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":7414,"log_dups_size":0,"ondisk_log_size":7414,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30334976,"num_objects":7406,"num_object_clones":0,"num_object_copies":14812,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7406,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3918,"num_write_kb":15672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3151},{"shards":"0,2","objects":4255}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":99,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.884297+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T15:59:15.884297+0000","last_peered":"2024-09-10T15:59:15.884297+0000","last_clean":"2024-09-10T15:59:15.884297+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T15:59:15.884297+0000","last_undegraded":"2024-09-10T15:59:15.884297+0000","last_fullsized":"2024-09-10T15:59:15.884297+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'7381","reported_seq":7511,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:44.432552+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:00:44.432552+0000","last_peered":"2024-09-10T16:00:44.432552+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:00:44.432552+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":7381,"log_dups_size":0,"ondisk_log_size":7381,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30212579,"num_objects":7377,"num_object_clones":0,"num_object_copies":14754,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7377,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5061,"num_write_kb":20244,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4304},{"shards":"2","objects":3073}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.837509+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T15:59:15.837509+0000","last_peered":"2024-09-10T15:59:15.837509+0000","last_clean":"2024-09-10T15:59:15.837509+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T15:59:15.837509+0000","last_undegraded":"2024-09-10T15:59:15.837509+0000","last_fullsized":"2024-09-10T15:59:15.837509+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'7498","reported_seq":7622,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:44.418700+0000","last_change":"2024-09-10T15:59:16.174113+0000","last_active":"2024-09-10T16:00:44.418700+0000","last_peered":"2024-09-10T16:00:44.418700+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:00:44.418700+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":7498,"log_dups_size":0,"ondisk_log_size":7498,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30687232,"num_objects":7492,"num_object_clones":0,"num_object_copies":14984,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4354,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7492,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5052,"num_write_kb":20208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4352},{"shards":"2","objects":3140}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'7428","reported_seq":7558,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:46.628123+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:00:46.628123+0000","last_peered":"2024-09-10T16:00:46.628123+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:00:46.628123+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":7428,"log_dups_size":0,"ondisk_log_size":7428,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30416896,"num_objects":7426,"num_object_clones":0,"num_object_copies":14852,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7426,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4059,"num_write_kb":16236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3122},{"shards":"0,2","objects":4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.989413+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T15:59:15.989413+0000","last_peered":"2024-09-10T15:59:15.989413+0000","last_clean":"2024-09-10T15:59:15.989413+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T15:59:15.989413+0000","last_undegraded":"2024-09-10T15:59:15.989413+0000","last_fullsized":"2024-09-10T15:59:15.989413+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'7264","reported_seq":7402,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:46.652641+0000","last_change":"2024-09-10T15:59:16.251610+0000","last_active":"2024-09-10T16:00:46.652641+0000","last_peered":"2024-09-10T16:00:46.652641+0000","last_clean":"2024-09-10T15:59:14.115595+0000","last_became_active":"2024-09-10T15:59:16.244089+0000","last_became_peered":"2024-09-10T15:59:16.244089+0000","last_unstale":"2024-09-10T16:00:46.652641+0000","last_undegraded":"2024-09-10T15:59:15.847127+0000","last_fullsized":"2024-09-10T15:59:15.774437+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":7264,"log_dups_size":0,"ondisk_log_size":7264,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29720576,"num_objects":7256,"num_object_clones":0,"num_object_copies":14512,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7256,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3654,"num_write_kb":14616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3053},{"shards":"0,2","objects":4203}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":89,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.773633+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T15:59:15.773633+0000","last_peered":"2024-09-10T15:59:15.773633+0000","last_clean":"2024-09-10T15:59:15.773633+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T15:59:15.773633+0000","last_undegraded":"2024-09-10T15:59:15.773633+0000","last_fullsized":"2024-09-10T15:59:15.773633+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'7569","reported_seq":7706,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:46.677404+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:00:46.677404+0000","last_peered":"2024-09-10T16:00:46.677404+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:00:46.677404+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":7569,"log_dups_size":0,"ondisk_log_size":7569,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30986240,"num_objects":7565,"num_object_clones":0,"num_object_copies":15130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7565,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3824,"num_write_kb":15296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3194},{"shards":"0,2","objects":4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.916017+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T15:59:15.916017+0000","last_peered":"2024-09-10T15:59:15.916017+0000","last_clean":"2024-09-10T15:59:15.916017+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T15:59:15.916017+0000","last_undegraded":"2024-09-10T15:59:15.916017+0000","last_fullsized":"2024-09-10T15:59:15.916017+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.776033+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T15:59:15.776033+0000","last_peered":"2024-09-10T15:59:15.776033+0000","last_clean":"2024-09-10T15:59:15.776033+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T15:59:15.776033+0000","last_undegraded":"2024-09-10T15:59:15.776033+0000","last_fullsized":"2024-09-10T15:59:15.776033+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'7543","reported_seq":7682,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:46.549983+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:00:46.549983+0000","last_peered":"2024-09-10T16:00:46.549983+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:00:46.549983+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":7543,"log_dups_size":0,"ondisk_log_size":7543,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30847942,"num_objects":7533,"num_object_clones":0,"num_object_copies":15066,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7533,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3706,"num_write_kb":14824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3210},{"shards":"0,2","objects":4323}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":485069534,"num_objects":118434,"num_object_clones":0,"num_object_copies":236868,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":47117,"num_objects_misplaced":2316,"num_objects_unfound":0,"num_objects_dirty":118434,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":76280,"num_write_kb":305120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1956,"num_bytes_recovered":8007680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":974880768,"data_stored":974729022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":118506,"ondisk_log_size":118506,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986209,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":556480,"kb_used_data":372080,"kb_used_omap":4,"kb_used_meta":184379,"kb_avail":93815360,"statfs":{"total":96636764160,"available":96066928640,"internally_reserved":0,"allocated":381009920,"data_stored":380732160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":188804778},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,3,1,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659155,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":293328,"kb_used_data":153344,"kb_used_omap":4,"kb_used_meta":139963,"kb_avail":94078512,"statfs":{"total":96636764160,"available":96336396288,"internally_reserved":0,"allocated":157024256,"data_stored":156699063,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":143322798},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,2,3,4,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182438,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":695420,"kb_used_data":425004,"kb_used_omap":4,"kb_used_meta":270395,"kb_avail":93676420,"statfs":{"total":96636764160,"available":95924654080,"internally_reserved":0,"allocated":435204096,"data_stored":434922259,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":276885162},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3,0,0,4,8,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":435810304,"data_stored":435774174,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":157622272,"data_stored":157542786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":381448192,"data_stored":381412062,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:00:49.982 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 137 20 120450 120430 3.43325 1.95703 0.0343592 0.0227472 2024-09-10T16:00:50.982 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 138 20 120987 120967 3.42357 2.09766 0.0328906 0.0228122 2024-09-10T16:00:51.982 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 139 20 121507 121487 3.41355 2.03125 0.0332084 0.0228797 2024-09-10T16:00:52.474 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:00:52.982 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout:2024-09-10T16:00:52.982341+0000 min lat: 0.00225581 max lat: 2.21736 avg lat: 0.0229505 2024-09-10T16:00:52.983 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-10T16:00:52.983 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 140 20 121999 121979 3.4029 1.92188 0.0518063 0.0229505 2024-09-10T16:00:53.048 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:00:53.048 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:00:53.074 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":144,"stamp":"2024-09-10T16:00:51.574275+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":488706801,"num_objects":119324,"num_object_clones":0,"num_object_copies":238648,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":47117,"num_objects_misplaced":2195,"num_objects_unfound":0,"num_objects_dirty":119324,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":77170,"num_write_kb":308674,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2079,"num_bytes_recovered":8503315,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119397,"ondisk_log_size":119397,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1549888,"kb_used_data":963408,"kb_used_omap":12,"kb_used_meta":586419,"kb_avail":281565632,"statfs":{"total":289910292480,"available":288323207168,"internally_reserved":0,"allocated":986529792,"data_stored":985645002,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":600493058},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,3,0,7,10,17,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":20680704,"num_objects":5049,"num_object_clones":0,"num_object_copies":10098,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-239,"num_objects_unfound":0,"num_objects_dirty":5049,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5049,"num_write_kb":20196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":240,"num_bytes_recovered":978944,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004697"},"pg_stats":[{"pgid":"2.f","version":"43'7378","reported_seq":7448,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:46.675714+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:00:46.675714+0000","last_peered":"2024-09-10T16:00:46.675714+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:00:46.675714+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":7378,"log_dups_size":0,"ondisk_log_size":7378,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30220288,"num_objects":7378,"num_object_clones":0,"num_object_copies":14756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7378,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7378,"num_write_kb":29512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3119},{"shards":"0,2","objects":4258}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'7479","reported_seq":7587,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:00:49.421701+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:00:49.421701+0000","last_peered":"2024-09-10T16:00:49.421701+0000","last_clean":"2024-09-10T16:00:49.421701+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:00:49.421701+0000","last_undegraded":"2024-09-10T16:00:49.421701+0000","last_fullsized":"2024-09-10T16:00:49.421701+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":7479,"log_dups_size":0,"ondisk_log_size":7479,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30610374,"num_objects":7475,"num_object_clones":0,"num_object_copies":14950,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7475,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5584,"num_write_kb":22336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'7469","reported_seq":11743,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T16:00:49.467288+0000","last_change":"2024-09-10T15:59:16.142612+0000","last_active":"2024-09-10T16:00:49.467288+0000","last_peered":"2024-09-10T16:00:49.467288+0000","last_clean":"2024-09-10T15:59:14.092473+0000","last_became_active":"2024-09-10T15:59:16.109611+0000","last_became_peered":"2024-09-10T15:59:16.109611+0000","last_unstale":"2024-09-10T16:00:49.467288+0000","last_undegraded":"2024-09-10T16:00:49.467288+0000","last_fullsized":"2024-09-10T15:59:15.770264+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":7469,"log_dups_size":0,"ondisk_log_size":7469,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30576640,"num_objects":7465,"num_object_clones":0,"num_object_copies":14930,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2195,"num_objects_unfound":0,"num_objects_dirty":7465,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5473,"num_write_kb":21892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2074,"num_bytes_recovered":8491008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2195},{"shards":"2","objects":5270}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'7466","reported_seq":7542,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:00:49.456705+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:00:49.456705+0000","last_peered":"2024-09-10T16:00:49.456705+0000","last_clean":"2024-09-10T16:00:49.456705+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:00:49.456705+0000","last_undegraded":"2024-09-10T16:00:49.456705+0000","last_fullsized":"2024-09-10T16:00:49.456705+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":7466,"log_dups_size":0,"ondisk_log_size":7466,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30580736,"num_objects":7466,"num_object_clones":0,"num_object_copies":14932,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7466,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7466,"num_write_kb":29864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'7487","reported_seq":7612,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:00:46.682514+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:00:46.682514+0000","last_peered":"2024-09-10T16:00:46.682514+0000","last_clean":"2024-09-10T16:00:46.682514+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:00:46.682514+0000","last_undegraded":"2024-09-10T16:00:46.682514+0000","last_fullsized":"2024-09-10T16:00:46.682514+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":7487,"log_dups_size":0,"ondisk_log_size":7487,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30638563,"num_objects":7481,"num_object_clones":0,"num_object_copies":14962,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7481,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4198,"num_write_kb":16792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'7506","reported_seq":7639,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:46.652109+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:00:46.652109+0000","last_peered":"2024-09-10T16:00:46.652109+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:00:46.652109+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":7506,"log_dups_size":0,"ondisk_log_size":7506,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30724579,"num_objects":7502,"num_object_clones":0,"num_object_copies":15004,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7502,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4118,"num_write_kb":16472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3131},{"shards":"0,2","objects":4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'7437","reported_seq":7569,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:46.599254+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:00:46.599254+0000","last_peered":"2024-09-10T16:00:46.599254+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:00:46.599254+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":7437,"log_dups_size":0,"ondisk_log_size":7437,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30438342,"num_objects":7433,"num_object_clones":0,"num_object_copies":14866,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7433,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3936,"num_write_kb":15744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3196},{"shards":"0,2","objects":4237}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'7573","reported_seq":7698,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:00:46.680035+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:00:46.680035+0000","last_peered":"2024-09-10T16:00:46.680035+0000","last_clean":"2024-09-10T16:00:46.680035+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:00:46.680035+0000","last_undegraded":"2024-09-10T16:00:46.680035+0000","last_fullsized":"2024-09-10T16:00:46.680035+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":7573,"log_dups_size":0,"ondisk_log_size":7573,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31010816,"num_objects":7571,"num_object_clones":0,"num_object_copies":15142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7571,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4420,"num_write_kb":17680,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.775865+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T15:59:15.775865+0000","last_peered":"2024-09-10T15:59:15.775865+0000","last_clean":"2024-09-10T15:59:15.775865+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T15:59:15.775865+0000","last_undegraded":"2024-09-10T15:59:15.775865+0000","last_fullsized":"2024-09-10T15:59:15.775865+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'7196","reported_seq":7331,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:49.490490+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:00:49.490490+0000","last_peered":"2024-09-10T16:00:49.490490+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:00:49.490490+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":7196,"log_dups_size":0,"ondisk_log_size":7196,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29446627,"num_objects":7190,"num_object_clones":0,"num_object_copies":14380,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7190,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5015,"num_write_kb":20060,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4123},{"shards":"2","objects":3067}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":89,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.772619+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T15:59:15.772619+0000","last_peered":"2024-09-10T15:59:15.772619+0000","last_clean":"2024-09-10T15:59:15.772619+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T15:59:15.772619+0000","last_undegraded":"2024-09-10T15:59:15.772619+0000","last_fullsized":"2024-09-10T15:59:15.772619+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'7414","reported_seq":7538,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:46.611722+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:00:46.611722+0000","last_peered":"2024-09-10T16:00:46.611722+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:00:46.611722+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":7414,"log_dups_size":0,"ondisk_log_size":7414,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30334976,"num_objects":7406,"num_object_clones":0,"num_object_copies":14812,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7406,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3918,"num_write_kb":15672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3151},{"shards":"0,2","objects":4255}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":99,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.884297+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T15:59:15.884297+0000","last_peered":"2024-09-10T15:59:15.884297+0000","last_clean":"2024-09-10T15:59:15.884297+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T15:59:15.884297+0000","last_undegraded":"2024-09-10T15:59:15.884297+0000","last_fullsized":"2024-09-10T15:59:15.884297+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'7533","reported_seq":7663,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:49.486175+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:00:49.486175+0000","last_peered":"2024-09-10T16:00:49.486175+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:00:49.486175+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":7533,"log_dups_size":0,"ondisk_log_size":7533,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30835171,"num_objects":7529,"num_object_clones":0,"num_object_copies":15058,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7529,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5213,"num_write_kb":20852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4304},{"shards":"2","objects":3225}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.837509+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T15:59:15.837509+0000","last_peered":"2024-09-10T15:59:15.837509+0000","last_clean":"2024-09-10T15:59:15.837509+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T15:59:15.837509+0000","last_undegraded":"2024-09-10T15:59:15.837509+0000","last_fullsized":"2024-09-10T15:59:15.837509+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'7652","reported_seq":7776,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:49.500268+0000","last_change":"2024-09-10T15:59:16.174113+0000","last_active":"2024-09-10T16:00:49.500268+0000","last_peered":"2024-09-10T16:00:49.500268+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:00:49.500268+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":7652,"log_dups_size":0,"ondisk_log_size":7652,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31318016,"num_objects":7646,"num_object_clones":0,"num_object_copies":15292,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4354,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7646,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5206,"num_write_kb":20824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4352},{"shards":"2","objects":3294}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'7428","reported_seq":7558,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:46.628123+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:00:46.628123+0000","last_peered":"2024-09-10T16:00:46.628123+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:00:46.628123+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":7428,"log_dups_size":0,"ondisk_log_size":7428,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30416896,"num_objects":7426,"num_object_clones":0,"num_object_copies":14852,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7426,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4059,"num_write_kb":16236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3122},{"shards":"0,2","objects":4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.989413+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T15:59:15.989413+0000","last_peered":"2024-09-10T15:59:15.989413+0000","last_clean":"2024-09-10T15:59:15.989413+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T15:59:15.989413+0000","last_undegraded":"2024-09-10T15:59:15.989413+0000","last_fullsized":"2024-09-10T15:59:15.989413+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'7264","reported_seq":7402,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:46.652641+0000","last_change":"2024-09-10T15:59:16.251610+0000","last_active":"2024-09-10T16:00:46.652641+0000","last_peered":"2024-09-10T16:00:46.652641+0000","last_clean":"2024-09-10T15:59:14.115595+0000","last_became_active":"2024-09-10T15:59:16.244089+0000","last_became_peered":"2024-09-10T15:59:16.244089+0000","last_unstale":"2024-09-10T16:00:46.652641+0000","last_undegraded":"2024-09-10T15:59:15.847127+0000","last_fullsized":"2024-09-10T15:59:15.774437+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":7264,"log_dups_size":0,"ondisk_log_size":7264,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29720576,"num_objects":7256,"num_object_clones":0,"num_object_copies":14512,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7256,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3654,"num_write_kb":14616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3053},{"shards":"0,2","objects":4203}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":89,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.773633+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T15:59:15.773633+0000","last_peered":"2024-09-10T15:59:15.773633+0000","last_clean":"2024-09-10T15:59:15.773633+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T15:59:15.773633+0000","last_undegraded":"2024-09-10T15:59:15.773633+0000","last_fullsized":"2024-09-10T15:59:15.773633+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'7569","reported_seq":7706,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:46.677404+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:00:46.677404+0000","last_peered":"2024-09-10T16:00:46.677404+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:00:46.677404+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":7569,"log_dups_size":0,"ondisk_log_size":7569,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30986240,"num_objects":7565,"num_object_clones":0,"num_object_copies":15130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7565,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3824,"num_write_kb":15296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3194},{"shards":"0,2","objects":4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.916017+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T15:59:15.916017+0000","last_peered":"2024-09-10T15:59:15.916017+0000","last_clean":"2024-09-10T15:59:15.916017+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T15:59:15.916017+0000","last_undegraded":"2024-09-10T15:59:15.916017+0000","last_fullsized":"2024-09-10T15:59:15.916017+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.776033+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T15:59:15.776033+0000","last_peered":"2024-09-10T15:59:15.776033+0000","last_clean":"2024-09-10T15:59:15.776033+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T15:59:15.776033+0000","last_undegraded":"2024-09-10T15:59:15.776033+0000","last_fullsized":"2024-09-10T15:59:15.776033+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'7543","reported_seq":7682,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:46.549983+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:00:46.549983+0000","last_peered":"2024-09-10T16:00:46.549983+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:00:46.549983+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":7543,"log_dups_size":0,"ondisk_log_size":7543,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30847942,"num_objects":7533,"num_object_clones":0,"num_object_copies":15066,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7533,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3706,"num_write_kb":14824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3210},{"shards":"0,2","objects":4323}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":488706782,"num_objects":119322,"num_object_clones":0,"num_object_copies":238644,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":47117,"num_objects_misplaced":2195,"num_objects_unfound":0,"num_objects_dirty":119322,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":77168,"num_write_kb":308672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2077,"num_bytes_recovered":8503296,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":988217344,"data_stored":988065598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119394,"ondisk_log_size":119394,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986210,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":549016,"kb_used_data":377096,"kb_used_omap":4,"kb_used_meta":171899,"kb_avail":93822824,"statfs":{"total":96636764160,"available":96074571776,"internally_reserved":0,"allocated":386146304,"data_stored":385868544,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":176025258},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,2,3,6,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659156,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":305452,"kb_used_data":161308,"kb_used_omap":4,"kb_used_meta":144123,"kb_avail":94066388,"statfs":{"total":96636764160,"available":96323981312,"internally_reserved":0,"allocated":165179392,"data_stored":164854199,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":147582638},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,5,3,3,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182438,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":695420,"kb_used_data":425004,"kb_used_omap":4,"kb_used_meta":270395,"kb_avail":93676420,"statfs":{"total":96636764160,"available":95924654080,"internally_reserved":0,"allocated":435204096,"data_stored":434922259,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":276885162},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3,0,0,4,8,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":435810304,"data_stored":435774174,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":165986304,"data_stored":165906818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":386420736,"data_stored":386384606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:00:53.075 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:00:53.656 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:00:53.658 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:00:53.686 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":145,"stamp":"2024-09-10T16:00:53.575120+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":495334129,"num_objects":120942,"num_object_clones":0,"num_object_copies":241884,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":47117,"num_objects_misplaced":2195,"num_objects_unfound":0,"num_objects_dirty":120942,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":78788,"num_write_kb":315146,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2079,"num_bytes_recovered":8503315,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":121015,"ondisk_log_size":121015,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1561708,"kb_used_data":971068,"kb_used_omap":12,"kb_used_meta":590579,"kb_avail":281553812,"statfs":{"total":289910292480,"available":288311103488,"internally_reserved":0,"allocated":994373632,"data_stored":993488842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":604752898},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,10,9,13,12,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":27308032,"num_objects":6667,"num_object_clones":0,"num_object_copies":13334,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-239,"num_objects_unfound":0,"num_objects_dirty":6667,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6667,"num_write_kb":26668,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":240,"num_bytes_recovered":978944,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004730"},"pg_stats":[{"pgid":"2.f","version":"43'7537","reported_seq":7607,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:51.571148+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:00:51.571148+0000","last_peered":"2024-09-10T16:00:51.571148+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:00:51.571148+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":7537,"log_dups_size":0,"ondisk_log_size":7537,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30871552,"num_objects":7537,"num_object_clones":0,"num_object_copies":15074,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7537,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7537,"num_write_kb":30148,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3278},{"shards":"0,2","objects":4258}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'7479","reported_seq":7587,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:00:49.421701+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:00:49.421701+0000","last_peered":"2024-09-10T16:00:49.421701+0000","last_clean":"2024-09-10T16:00:49.421701+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:00:49.421701+0000","last_undegraded":"2024-09-10T16:00:49.421701+0000","last_fullsized":"2024-09-10T16:00:49.421701+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":7479,"log_dups_size":0,"ondisk_log_size":7479,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30610374,"num_objects":7475,"num_object_clones":0,"num_object_copies":14950,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7475,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5584,"num_write_kb":22336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'7469","reported_seq":11743,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T16:00:49.467288+0000","last_change":"2024-09-10T15:59:16.142612+0000","last_active":"2024-09-10T16:00:49.467288+0000","last_peered":"2024-09-10T16:00:49.467288+0000","last_clean":"2024-09-10T15:59:14.092473+0000","last_became_active":"2024-09-10T15:59:16.109611+0000","last_became_peered":"2024-09-10T15:59:16.109611+0000","last_unstale":"2024-09-10T16:00:49.467288+0000","last_undegraded":"2024-09-10T16:00:49.467288+0000","last_fullsized":"2024-09-10T15:59:15.770264+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":7469,"log_dups_size":0,"ondisk_log_size":7469,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30576640,"num_objects":7465,"num_object_clones":0,"num_object_copies":14930,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2195,"num_objects_unfound":0,"num_objects_dirty":7465,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5473,"num_write_kb":21892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2074,"num_bytes_recovered":8491008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2195},{"shards":"2","objects":5270}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'7466","reported_seq":7542,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:00:49.456705+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:00:49.456705+0000","last_peered":"2024-09-10T16:00:49.456705+0000","last_clean":"2024-09-10T16:00:49.456705+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:00:49.456705+0000","last_undegraded":"2024-09-10T16:00:49.456705+0000","last_fullsized":"2024-09-10T16:00:49.456705+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":7466,"log_dups_size":0,"ondisk_log_size":7466,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30580736,"num_objects":7466,"num_object_clones":0,"num_object_copies":14932,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7466,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7466,"num_write_kb":29864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'7651","reported_seq":7776,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:00:51.706417+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:00:51.706417+0000","last_peered":"2024-09-10T16:00:51.706417+0000","last_clean":"2024-09-10T16:00:51.706417+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:00:51.706417+0000","last_undegraded":"2024-09-10T16:00:51.706417+0000","last_fullsized":"2024-09-10T16:00:51.706417+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":7651,"log_dups_size":0,"ondisk_log_size":7651,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31310307,"num_objects":7645,"num_object_clones":0,"num_object_copies":15290,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7645,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4362,"num_write_kb":17448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'7681","reported_seq":7814,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:51.600905+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:00:51.600905+0000","last_peered":"2024-09-10T16:00:51.600905+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:00:51.600905+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":7681,"log_dups_size":0,"ondisk_log_size":7681,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31441379,"num_objects":7677,"num_object_clones":0,"num_object_copies":15354,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7677,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4293,"num_write_kb":17172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3306},{"shards":"0,2","objects":4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'7610","reported_seq":7743,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:51.642369+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:00:51.642369+0000","last_peered":"2024-09-10T16:00:51.642369+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:00:51.642369+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":7610,"log_dups_size":0,"ondisk_log_size":7610,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31146950,"num_objects":7606,"num_object_clones":0,"num_object_copies":15212,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7606,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4109,"num_write_kb":16436,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3369},{"shards":"0,2","objects":4237}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'7733","reported_seq":7858,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:00:51.660305+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:00:51.660305+0000","last_peered":"2024-09-10T16:00:51.660305+0000","last_clean":"2024-09-10T16:00:51.660305+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:00:51.660305+0000","last_undegraded":"2024-09-10T16:00:51.660305+0000","last_fullsized":"2024-09-10T16:00:51.660305+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":7733,"log_dups_size":0,"ondisk_log_size":7733,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31666176,"num_objects":7731,"num_object_clones":0,"num_object_copies":15462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7731,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4580,"num_write_kb":18320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.775865+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T15:59:15.775865+0000","last_peered":"2024-09-10T15:59:15.775865+0000","last_clean":"2024-09-10T15:59:15.775865+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T15:59:15.775865+0000","last_undegraded":"2024-09-10T15:59:15.775865+0000","last_fullsized":"2024-09-10T15:59:15.775865+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'7196","reported_seq":7331,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:49.490490+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:00:49.490490+0000","last_peered":"2024-09-10T16:00:49.490490+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:00:49.490490+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":7196,"log_dups_size":0,"ondisk_log_size":7196,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29446627,"num_objects":7190,"num_object_clones":0,"num_object_copies":14380,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7190,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5015,"num_write_kb":20060,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4123},{"shards":"2","objects":3067}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":89,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.772619+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T15:59:15.772619+0000","last_peered":"2024-09-10T15:59:15.772619+0000","last_clean":"2024-09-10T15:59:15.772619+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T15:59:15.772619+0000","last_undegraded":"2024-09-10T15:59:15.772619+0000","last_fullsized":"2024-09-10T15:59:15.772619+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'7568","reported_seq":7693,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:51.643775+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:00:51.643775+0000","last_peered":"2024-09-10T16:00:51.643775+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:00:51.643775+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":7568,"log_dups_size":0,"ondisk_log_size":7568,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30965760,"num_objects":7560,"num_object_clones":0,"num_object_copies":15120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7560,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4072,"num_write_kb":16288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3305},{"shards":"0,2","objects":4255}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":99,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.884297+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T15:59:15.884297+0000","last_peered":"2024-09-10T15:59:15.884297+0000","last_clean":"2024-09-10T15:59:15.884297+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T15:59:15.884297+0000","last_undegraded":"2024-09-10T15:59:15.884297+0000","last_fullsized":"2024-09-10T15:59:15.884297+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'7533","reported_seq":7663,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:49.486175+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:00:49.486175+0000","last_peered":"2024-09-10T16:00:49.486175+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:00:49.486175+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":7533,"log_dups_size":0,"ondisk_log_size":7533,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30835171,"num_objects":7529,"num_object_clones":0,"num_object_copies":15058,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7529,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5213,"num_write_kb":20852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4304},{"shards":"2","objects":3225}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.837509+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T15:59:15.837509+0000","last_peered":"2024-09-10T15:59:15.837509+0000","last_clean":"2024-09-10T15:59:15.837509+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T15:59:15.837509+0000","last_undegraded":"2024-09-10T15:59:15.837509+0000","last_fullsized":"2024-09-10T15:59:15.837509+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'7652","reported_seq":7776,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:49.500268+0000","last_change":"2024-09-10T15:59:16.174113+0000","last_active":"2024-09-10T16:00:49.500268+0000","last_peered":"2024-09-10T16:00:49.500268+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:00:49.500268+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":7652,"log_dups_size":0,"ondisk_log_size":7652,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31318016,"num_objects":7646,"num_object_clones":0,"num_object_copies":15292,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4354,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7646,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5206,"num_write_kb":20824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4352},{"shards":"2","objects":3294}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'7588","reported_seq":7717,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:51.672936+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:00:51.672936+0000","last_peered":"2024-09-10T16:00:51.672936+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:00:51.672936+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":7588,"log_dups_size":0,"ondisk_log_size":7588,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31072256,"num_objects":7586,"num_object_clones":0,"num_object_copies":15172,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7586,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4219,"num_write_kb":16876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3282},{"shards":"0,2","objects":4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.989413+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T15:59:15.989413+0000","last_peered":"2024-09-10T15:59:15.989413+0000","last_clean":"2024-09-10T15:59:15.989413+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T15:59:15.989413+0000","last_undegraded":"2024-09-10T15:59:15.989413+0000","last_fullsized":"2024-09-10T15:59:15.989413+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'7435","reported_seq":7573,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:51.688783+0000","last_change":"2024-09-10T15:59:16.251610+0000","last_active":"2024-09-10T16:00:51.688783+0000","last_peered":"2024-09-10T16:00:51.688783+0000","last_clean":"2024-09-10T15:59:14.115595+0000","last_became_active":"2024-09-10T15:59:16.244089+0000","last_became_peered":"2024-09-10T15:59:16.244089+0000","last_unstale":"2024-09-10T16:00:51.688783+0000","last_undegraded":"2024-09-10T15:59:15.847127+0000","last_fullsized":"2024-09-10T15:59:15.774437+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":7435,"log_dups_size":0,"ondisk_log_size":7435,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30420992,"num_objects":7427,"num_object_clones":0,"num_object_copies":14854,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7427,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3825,"num_write_kb":15300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3224},{"shards":"0,2","objects":4203}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":89,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.773633+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T15:59:15.773633+0000","last_peered":"2024-09-10T15:59:15.773633+0000","last_clean":"2024-09-10T15:59:15.773633+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T15:59:15.773633+0000","last_undegraded":"2024-09-10T15:59:15.773633+0000","last_fullsized":"2024-09-10T15:59:15.773633+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'7720","reported_seq":7857,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:51.683701+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:00:51.683701+0000","last_peered":"2024-09-10T16:00:51.683701+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:00:51.683701+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":7720,"log_dups_size":0,"ondisk_log_size":7720,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31604736,"num_objects":7716,"num_object_clones":0,"num_object_copies":15432,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7716,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3975,"num_write_kb":15900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3345},{"shards":"0,2","objects":4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.916017+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T15:59:15.916017+0000","last_peered":"2024-09-10T15:59:15.916017+0000","last_clean":"2024-09-10T15:59:15.916017+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T15:59:15.916017+0000","last_undegraded":"2024-09-10T15:59:15.916017+0000","last_fullsized":"2024-09-10T15:59:15.916017+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.776033+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T15:59:15.776033+0000","last_peered":"2024-09-10T15:59:15.776033+0000","last_clean":"2024-09-10T15:59:15.776033+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T15:59:15.776033+0000","last_undegraded":"2024-09-10T15:59:15.776033+0000","last_fullsized":"2024-09-10T15:59:15.776033+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'7694","reported_seq":7833,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:51.717904+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:00:51.717904+0000","last_peered":"2024-09-10T16:00:51.717904+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:00:51.717904+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":7694,"log_dups_size":0,"ondisk_log_size":7694,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31466438,"num_objects":7684,"num_object_clones":0,"num_object_copies":15368,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7684,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3857,"num_write_kb":15428,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3361},{"shards":"0,2","objects":4323}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":495334110,"num_objects":120940,"num_object_clones":0,"num_object_copies":241880,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":47117,"num_objects_misplaced":2195,"num_objects_unfound":0,"num_objects_dirty":120940,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":78786,"num_write_kb":315144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2077,"num_bytes_recovered":8503296,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":996085760,"data_stored":995934014,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":121012,"ondisk_log_size":121012,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986210,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":549016,"kb_used_data":377096,"kb_used_omap":4,"kb_used_meta":171899,"kb_avail":93822824,"statfs":{"total":96636764160,"available":96074571776,"internally_reserved":0,"allocated":386146304,"data_stored":385868544,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":176025258},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,2,3,6,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659156,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":305452,"kb_used_data":161308,"kb_used_omap":4,"kb_used_meta":144123,"kb_avail":94066388,"statfs":{"total":96636764160,"available":96323981312,"internally_reserved":0,"allocated":165179392,"data_stored":164854199,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":147582638},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,5,3,3,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182439,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":707240,"kb_used_data":432664,"kb_used_omap":4,"kb_used_meta":274555,"kb_avail":93664600,"statfs":{"total":96636764160,"available":95912550400,"internally_reserved":0,"allocated":443047936,"data_stored":442766099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":281145002},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,3,3,4,8,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":443678720,"data_stored":443642590,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":165986304,"data_stored":165906818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":386420736,"data_stored":386384606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:00:53.687 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-10T16:00:53.688 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "43'7537", 'reported_seq': 7607, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:00:51.571148+0000', 'last_change': '2024-09-10T15:59:16.281558+0000', 'last_active': '2024-09-10T16:00:51.571148+0000', 'last_peered': '2024-09-10T16:00:51.571148+0000', 'last_clean': '2024-09-10T15:59:14.132456+0000', 'last_became_active': '2024-09-10T15:59:16.274407+0000', 'last_became_peered': '2024-09-10T15:59:16.274407+0000', 'last_unstale': '2024-09-10T16:00:51.571148+0000', 'last_undegraded': '2024-09-10T15:59:15.851279+0000', 'last_fullsized': '2024-09-10T15:59:15.774356+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'objects_scrubbed': 0, 'log_size': 7537, 'log_dups_size': 0, 'ondisk_log_size': 7537, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30871552, 'num_objects': 7537, 'num_object_clones': 0, 'num_object_copies': 15074, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4259, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7537, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7537, 'num_write_kb': 30148, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3278}, {'shards': '0,2', 'objects': 4258}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:00:53.688 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-10T16:00:53.688 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "43'7469", 'reported_seq': 11743, 'reported_epoch': 43, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-10T16:00:49.467288+0000', 'last_change': '2024-09-10T15:59:16.142612+0000', 'last_active': '2024-09-10T16:00:49.467288+0000', 'last_peered': '2024-09-10T16:00:49.467288+0000', 'last_clean': '2024-09-10T15:59:14.092473+0000', 'last_became_active': '2024-09-10T15:59:16.109611+0000', 'last_became_peered': '2024-09-10T15:59:16.109611+0000', 'last_unstale': '2024-09-10T16:00:49.467288+0000', 'last_undegraded': '2024-09-10T16:00:49.467288+0000', 'last_fullsized': '2024-09-10T15:59:15.770264+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2029", 'last_scrub_stamp': '2024-09-10T15:58:45.933789+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:45.933789+0000', 'objects_scrubbed': 1992, 'log_size': 7469, 'log_dups_size': 0, 'ondisk_log_size': 7469, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 668, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30576640, 'num_objects': 7465, 'num_object_clones': 0, 'num_object_copies': 14930, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 2195, 'num_objects_unfound': 0, 'num_objects_dirty': 7465, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5473, 'num_write_kb': 21892, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2074, 'num_bytes_recovered': 8491008, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2195}, {'shards': '2', 'objects': 5270}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:00:53.688 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-10T16:00:53.688 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "43'7681", 'reported_seq': 7814, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:00:51.600905+0000', 'last_change': '2024-09-10T15:59:16.308264+0000', 'last_active': '2024-09-10T16:00:51.600905+0000', 'last_peered': '2024-09-10T16:00:51.600905+0000', 'last_clean': '2024-09-10T15:59:14.111551+0000', 'last_became_active': '2024-09-10T15:59:16.300040+0000', 'last_became_peered': '2024-09-10T15:59:16.300040+0000', 'last_unstale': '2024-09-10T16:00:51.600905+0000', 'last_undegraded': '2024-09-10T15:59:15.878234+0000', 'last_fullsized': '2024-09-10T15:59:15.769328+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3442", 'last_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'objects_scrubbed': 3384, 'log_size': 7681, 'log_dups_size': 0, 'ondisk_log_size': 7681, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1292, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31441379, 'num_objects': 7677, 'num_object_clones': 0, 'num_object_copies': 15354, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4372, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7677, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 4293, 'num_write_kb': 17172, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3306}, {'shards': '0,2', 'objects': 4371}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:00:53.688 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-10T16:00:53.688 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "43'7610", 'reported_seq': 7743, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:00:51.642369+0000', 'last_change': '2024-09-10T15:59:16.331899+0000', 'last_active': '2024-09-10T16:00:51.642369+0000', 'last_peered': '2024-09-10T16:00:51.642369+0000', 'last_clean': '2024-09-10T15:59:14.139436+0000', 'last_became_active': '2024-09-10T15:59:16.318715+0000', 'last_became_peered': '2024-09-10T15:59:16.318715+0000', 'last_unstale': '2024-09-10T16:00:51.642369+0000', 'last_undegraded': '2024-09-10T15:59:15.877524+0000', 'last_fullsized': '2024-09-10T15:59:15.769042+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3537", 'last_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'objects_scrubbed': 3497, 'log_size': 7610, 'log_dups_size': 0, 'ondisk_log_size': 7610, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1224, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31146950, 'num_objects': 7606, 'num_object_clones': 0, 'num_object_copies': 15212, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4237, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7606, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 4109, 'num_write_kb': 16436, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3369}, {'shards': '0,2', 'objects': 4237}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:00:53.688 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-10T16:00:53.688 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "43'7196", 'reported_seq': 7331, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:00:49.490490+0000', 'last_change': '2024-09-10T15:59:16.175680+0000', 'last_active': '2024-09-10T16:00:49.490490+0000', 'last_peered': '2024-09-10T16:00:49.490490+0000', 'last_clean': '2024-09-10T15:59:14.088187+0000', 'last_became_active': '2024-09-10T15:59:16.165810+0000', 'last_became_peered': '2024-09-10T15:59:16.165810+0000', 'last_unstale': '2024-09-10T16:00:49.490490+0000', 'last_undegraded': '2024-09-10T15:59:15.822098+0000', 'last_fullsized': '2024-09-10T15:59:15.769123+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2218", 'last_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'objects_scrubbed': 2175, 'log_size': 7196, 'log_dups_size': 0, 'ondisk_log_size': 7196, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 825, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29446627, 'num_objects': 7190, 'num_object_clones': 0, 'num_object_copies': 14380, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4124, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7190, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 5015, 'num_write_kb': 20060, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4123}, {'shards': '2', 'objects': 3067}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:00:53.689 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-10T16:00:53.689 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "43'7568", 'reported_seq': 7693, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:00:51.643775+0000', 'last_change': '2024-09-10T15:59:16.277742+0000', 'last_active': '2024-09-10T16:00:51.643775+0000', 'last_peered': '2024-09-10T16:00:51.643775+0000', 'last_clean': '2024-09-10T15:59:14.129081+0000', 'last_became_active': '2024-09-10T15:59:16.269835+0000', 'last_became_peered': '2024-09-10T15:59:16.269835+0000', 'last_unstale': '2024-09-10T16:00:51.643775+0000', 'last_undegraded': '2024-09-10T15:59:15.860857+0000', 'last_fullsized': '2024-09-10T15:59:15.771410+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3557", 'last_scrub_stamp': '2024-09-10T15:59:02.493678+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:02.493678+0000', 'objects_scrubbed': 3488, 'log_size': 7568, 'log_dups_size': 0, 'ondisk_log_size': 7568, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1472, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30965760, 'num_objects': 7560, 'num_object_clones': 0, 'num_object_copies': 15120, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4257, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7560, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4072, 'num_write_kb': 16288, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3305}, {'shards': '0,2', 'objects': 4255}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:00:53.689 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-10T16:00:53.689 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "43'7533", 'reported_seq': 7663, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:00:49.486175+0000', 'last_change': '2024-09-10T15:59:16.284311+0000', 'last_active': '2024-09-10T16:00:49.486175+0000', 'last_peered': '2024-09-10T16:00:49.486175+0000', 'last_clean': '2024-09-10T15:59:14.116041+0000', 'last_became_active': '2024-09-10T15:59:16.274543+0000', 'last_became_peered': '2024-09-10T15:59:16.274543+0000', 'last_unstale': '2024-09-10T16:00:49.486175+0000', 'last_undegraded': '2024-09-10T15:59:15.812484+0000', 'last_fullsized': '2024-09-10T15:59:15.770885+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2356", 'last_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'objects_scrubbed': 2316, 'log_size': 7533, 'log_dups_size': 0, 'ondisk_log_size': 7533, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 783, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30835171, 'num_objects': 7529, 'num_object_clones': 0, 'num_object_copies': 15058, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7529, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 5213, 'num_write_kb': 20852, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4304}, {'shards': '2', 'objects': 3225}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:00:53.690 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-10T16:00:53.690 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "43'7652", 'reported_seq': 7776, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:00:49.500268+0000', 'last_change': '2024-09-10T15:59:16.174113+0000', 'last_active': '2024-09-10T16:00:49.500268+0000', 'last_peered': '2024-09-10T16:00:49.500268+0000', 'last_clean': '2024-09-10T15:59:14.110654+0000', 'last_became_active': '2024-09-10T15:59:16.164645+0000', 'last_became_peered': '2024-09-10T15:59:16.164645+0000', 'last_unstale': '2024-09-10T16:00:49.500268+0000', 'last_undegraded': '2024-09-10T15:59:15.803593+0000', 'last_fullsized': '2024-09-10T15:59:15.769083+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2486", 'last_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'objects_scrubbed': 2440, 'log_size': 7652, 'log_dups_size': 0, 'ondisk_log_size': 7652, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 792, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31318016, 'num_objects': 7646, 'num_object_clones': 0, 'num_object_copies': 15292, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4354, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7646, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5206, 'num_write_kb': 20824, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4352}, {'shards': '2', 'objects': 3294}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:00:53.690 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-10T16:00:53.690 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "43'7588", 'reported_seq': 7717, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:00:51.672936+0000', 'last_change': '2024-09-10T15:59:16.315641+0000', 'last_active': '2024-09-10T16:00:51.672936+0000', 'last_peered': '2024-09-10T16:00:51.672936+0000', 'last_clean': '2024-09-10T15:59:14.117001+0000', 'last_became_active': '2024-09-10T15:59:16.302245+0000', 'last_became_peered': '2024-09-10T15:59:16.302245+0000', 'last_unstale': '2024-09-10T16:00:51.672936+0000', 'last_undegraded': '2024-09-10T15:59:15.845012+0000', 'last_fullsized': '2024-09-10T15:59:15.770337+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3404", 'last_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'objects_scrubbed': 3367, 'log_size': 7588, 'log_dups_size': 0, 'ondisk_log_size': 7588, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 983, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31072256, 'num_objects': 7586, 'num_object_clones': 0, 'num_object_copies': 15172, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7586, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4219, 'num_write_kb': 16876, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3282}, {'shards': '0,2', 'objects': 4304}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:00:53.690 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-10T16:00:53.690 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "43'7435", 'reported_seq': 7573, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:00:51.688783+0000', 'last_change': '2024-09-10T15:59:16.251610+0000', 'last_active': '2024-09-10T16:00:51.688783+0000', 'last_peered': '2024-09-10T16:00:51.688783+0000', 'last_clean': '2024-09-10T15:59:14.115595+0000', 'last_became_active': '2024-09-10T15:59:16.244089+0000', 'last_became_peered': '2024-09-10T15:59:16.244089+0000', 'last_unstale': '2024-09-10T16:00:51.688783+0000', 'last_undegraded': '2024-09-10T15:59:15.847127+0000', 'last_fullsized': '2024-09-10T15:59:15.774437+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3653", 'last_scrub_stamp': '2024-09-10T15:59:05.298468+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:05.298468+0000', 'objects_scrubbed': 3602, 'log_size': 7435, 'log_dups_size': 0, 'ondisk_log_size': 7435, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1289, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30420992, 'num_objects': 7427, 'num_object_clones': 0, 'num_object_copies': 14854, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4206, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7427, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3825, 'num_write_kb': 15300, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3224}, {'shards': '0,2', 'objects': 4203}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:00:53.691 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-10T16:00:53.691 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "43'7720", 'reported_seq': 7857, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:00:51.683701+0000', 'last_change': '2024-09-10T15:59:16.311291+0000', 'last_active': '2024-09-10T16:00:51.683701+0000', 'last_peered': '2024-09-10T16:00:51.683701+0000', 'last_clean': '2024-09-10T15:59:14.116363+0000', 'last_became_active': '2024-09-10T15:59:16.301244+0000', 'last_became_peered': '2024-09-10T15:59:16.301244+0000', 'last_unstale': '2024-09-10T16:00:51.683701+0000', 'last_undegraded': '2024-09-10T15:59:15.883434+0000', 'last_fullsized': '2024-09-10T15:59:15.772756+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3789", 'last_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'objects_scrubbed': 3741, 'log_size': 7720, 'log_dups_size': 0, 'ondisk_log_size': 7720, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1440, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31604736, 'num_objects': 7716, 'num_object_clones': 0, 'num_object_copies': 15432, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4373, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7716, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3975, 'num_write_kb': 15900, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3345}, {'shards': '0,2', 'objects': 4371}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:00:53.691 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-10T16:00:53.691 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "43'7694", 'reported_seq': 7833, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:00:51.717904+0000', 'last_change': '2024-09-10T15:59:16.313409+0000', 'last_active': '2024-09-10T16:00:51.717904+0000', 'last_peered': '2024-09-10T16:00:51.717904+0000', 'last_clean': '2024-09-10T15:59:14.129780+0000', 'last_became_active': '2024-09-10T15:59:16.300932+0000', 'last_became_peered': '2024-09-10T15:59:16.300932+0000', 'last_unstale': '2024-09-10T16:00:51.717904+0000', 'last_undegraded': '2024-09-10T15:59:15.887823+0000', 'last_fullsized': '2024-09-10T15:59:15.768513+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3880", 'last_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'objects_scrubbed': 3827, 'log_size': 7694, 'log_dups_size': 0, 'ondisk_log_size': 7694, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1397, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31466438, 'num_objects': 7684, 'num_object_clones': 0, 'num_object_copies': 15368, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4325, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7684, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3857, 'num_write_kb': 15428, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3361}, {'shards': '0,2', 'objects': 4323}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:00:53.691 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T16:00:53.983 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 141 20 122455 122435 3.39139 1.78125 0.0533021 0.0230295 2024-09-10T16:00:54.379 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:00:54.379 INFO:teuthology.orchestra.run.smithi115.stdout:{"fsid":"7effee29-df05-4351-9741-ee203564ed81","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 47117/241884 objects degraded (19.479%), 11 pgs degraded, 12 pgs undersized","count":23},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":218,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":43,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983954,"num_in_osds":3,"osd_in_since":1725983947,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":120942,"data_bytes":495334129,"bytes_used":1599188992,"bytes_avail":288311103488,"bytes_total":289910292480,"degraded_objects":47117,"degraded_total":241884,"degraded_ratio":0.19479171834433034,"misplaced_objects":2195,"misplaced_total":241884,"misplaced_ratio":0.0090745977410659658,"recovering_objects_per_sec":19,"recovering_bytes_per_sec":81546,"recovering_keys_per_sec":0,"num_objects_recovered":240,"num_bytes_recovered":978944,"num_keys_recovered":0,"write_bytes_sec":2274772,"read_op_per_sec":0,"write_op_per_sec":555},"fsmap":{"epoch":1,"btime":"2024-09-10T15:57:14:967120+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T16:00:13.560776+0000","services":{}},"progress_events":{}} 2024-09-10T16:00:54.397 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-10T16:00:54.397 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:00:54.932 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:00:54.934 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:00:54.959 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":145,"stamp":"2024-09-10T16:00:53.575120+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":495334129,"num_objects":120942,"num_object_clones":0,"num_object_copies":241884,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":47117,"num_objects_misplaced":2195,"num_objects_unfound":0,"num_objects_dirty":120942,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":78788,"num_write_kb":315146,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2079,"num_bytes_recovered":8503315,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":121015,"ondisk_log_size":121015,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1561708,"kb_used_data":971068,"kb_used_omap":12,"kb_used_meta":590579,"kb_avail":281553812,"statfs":{"total":289910292480,"available":288311103488,"internally_reserved":0,"allocated":994373632,"data_stored":993488842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":604752898},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,10,9,13,12,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":27308032,"num_objects":6667,"num_object_clones":0,"num_object_copies":13334,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-239,"num_objects_unfound":0,"num_objects_dirty":6667,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6667,"num_write_kb":26668,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":240,"num_bytes_recovered":978944,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004730"},"pg_stats":[{"pgid":"2.f","version":"43'7537","reported_seq":7607,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:51.571148+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:00:51.571148+0000","last_peered":"2024-09-10T16:00:51.571148+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:00:51.571148+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":7537,"log_dups_size":0,"ondisk_log_size":7537,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30871552,"num_objects":7537,"num_object_clones":0,"num_object_copies":15074,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7537,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7537,"num_write_kb":30148,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3278},{"shards":"0,2","objects":4258}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'7479","reported_seq":7587,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:00:49.421701+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:00:49.421701+0000","last_peered":"2024-09-10T16:00:49.421701+0000","last_clean":"2024-09-10T16:00:49.421701+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:00:49.421701+0000","last_undegraded":"2024-09-10T16:00:49.421701+0000","last_fullsized":"2024-09-10T16:00:49.421701+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":7479,"log_dups_size":0,"ondisk_log_size":7479,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30610374,"num_objects":7475,"num_object_clones":0,"num_object_copies":14950,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7475,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5584,"num_write_kb":22336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'7469","reported_seq":11743,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T16:00:49.467288+0000","last_change":"2024-09-10T15:59:16.142612+0000","last_active":"2024-09-10T16:00:49.467288+0000","last_peered":"2024-09-10T16:00:49.467288+0000","last_clean":"2024-09-10T15:59:14.092473+0000","last_became_active":"2024-09-10T15:59:16.109611+0000","last_became_peered":"2024-09-10T15:59:16.109611+0000","last_unstale":"2024-09-10T16:00:49.467288+0000","last_undegraded":"2024-09-10T16:00:49.467288+0000","last_fullsized":"2024-09-10T15:59:15.770264+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":7469,"log_dups_size":0,"ondisk_log_size":7469,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30576640,"num_objects":7465,"num_object_clones":0,"num_object_copies":14930,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2195,"num_objects_unfound":0,"num_objects_dirty":7465,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5473,"num_write_kb":21892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2074,"num_bytes_recovered":8491008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2195},{"shards":"2","objects":5270}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'7466","reported_seq":7542,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:00:49.456705+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:00:49.456705+0000","last_peered":"2024-09-10T16:00:49.456705+0000","last_clean":"2024-09-10T16:00:49.456705+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:00:49.456705+0000","last_undegraded":"2024-09-10T16:00:49.456705+0000","last_fullsized":"2024-09-10T16:00:49.456705+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":7466,"log_dups_size":0,"ondisk_log_size":7466,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30580736,"num_objects":7466,"num_object_clones":0,"num_object_copies":14932,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7466,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7466,"num_write_kb":29864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'7651","reported_seq":7776,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:00:51.706417+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:00:51.706417+0000","last_peered":"2024-09-10T16:00:51.706417+0000","last_clean":"2024-09-10T16:00:51.706417+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:00:51.706417+0000","last_undegraded":"2024-09-10T16:00:51.706417+0000","last_fullsized":"2024-09-10T16:00:51.706417+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":7651,"log_dups_size":0,"ondisk_log_size":7651,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31310307,"num_objects":7645,"num_object_clones":0,"num_object_copies":15290,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7645,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4362,"num_write_kb":17448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'7681","reported_seq":7814,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:51.600905+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:00:51.600905+0000","last_peered":"2024-09-10T16:00:51.600905+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:00:51.600905+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":7681,"log_dups_size":0,"ondisk_log_size":7681,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31441379,"num_objects":7677,"num_object_clones":0,"num_object_copies":15354,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7677,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4293,"num_write_kb":17172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3306},{"shards":"0,2","objects":4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'7610","reported_seq":7743,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:51.642369+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:00:51.642369+0000","last_peered":"2024-09-10T16:00:51.642369+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:00:51.642369+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":7610,"log_dups_size":0,"ondisk_log_size":7610,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31146950,"num_objects":7606,"num_object_clones":0,"num_object_copies":15212,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7606,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4109,"num_write_kb":16436,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3369},{"shards":"0,2","objects":4237}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'7733","reported_seq":7858,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:00:51.660305+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:00:51.660305+0000","last_peered":"2024-09-10T16:00:51.660305+0000","last_clean":"2024-09-10T16:00:51.660305+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:00:51.660305+0000","last_undegraded":"2024-09-10T16:00:51.660305+0000","last_fullsized":"2024-09-10T16:00:51.660305+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":7733,"log_dups_size":0,"ondisk_log_size":7733,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31666176,"num_objects":7731,"num_object_clones":0,"num_object_copies":15462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7731,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4580,"num_write_kb":18320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.775865+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T15:59:15.775865+0000","last_peered":"2024-09-10T15:59:15.775865+0000","last_clean":"2024-09-10T15:59:15.775865+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T15:59:15.775865+0000","last_undegraded":"2024-09-10T15:59:15.775865+0000","last_fullsized":"2024-09-10T15:59:15.775865+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'7196","reported_seq":7331,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:49.490490+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:00:49.490490+0000","last_peered":"2024-09-10T16:00:49.490490+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:00:49.490490+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":7196,"log_dups_size":0,"ondisk_log_size":7196,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29446627,"num_objects":7190,"num_object_clones":0,"num_object_copies":14380,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7190,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5015,"num_write_kb":20060,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4123},{"shards":"2","objects":3067}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":89,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.772619+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T15:59:15.772619+0000","last_peered":"2024-09-10T15:59:15.772619+0000","last_clean":"2024-09-10T15:59:15.772619+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T15:59:15.772619+0000","last_undegraded":"2024-09-10T15:59:15.772619+0000","last_fullsized":"2024-09-10T15:59:15.772619+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'7568","reported_seq":7693,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:51.643775+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:00:51.643775+0000","last_peered":"2024-09-10T16:00:51.643775+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:00:51.643775+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":7568,"log_dups_size":0,"ondisk_log_size":7568,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30965760,"num_objects":7560,"num_object_clones":0,"num_object_copies":15120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7560,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4072,"num_write_kb":16288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3305},{"shards":"0,2","objects":4255}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":99,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.884297+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T15:59:15.884297+0000","last_peered":"2024-09-10T15:59:15.884297+0000","last_clean":"2024-09-10T15:59:15.884297+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T15:59:15.884297+0000","last_undegraded":"2024-09-10T15:59:15.884297+0000","last_fullsized":"2024-09-10T15:59:15.884297+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'7533","reported_seq":7663,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:49.486175+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:00:49.486175+0000","last_peered":"2024-09-10T16:00:49.486175+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:00:49.486175+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":7533,"log_dups_size":0,"ondisk_log_size":7533,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30835171,"num_objects":7529,"num_object_clones":0,"num_object_copies":15058,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7529,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5213,"num_write_kb":20852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4304},{"shards":"2","objects":3225}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.837509+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T15:59:15.837509+0000","last_peered":"2024-09-10T15:59:15.837509+0000","last_clean":"2024-09-10T15:59:15.837509+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T15:59:15.837509+0000","last_undegraded":"2024-09-10T15:59:15.837509+0000","last_fullsized":"2024-09-10T15:59:15.837509+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'7652","reported_seq":7776,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:49.500268+0000","last_change":"2024-09-10T15:59:16.174113+0000","last_active":"2024-09-10T16:00:49.500268+0000","last_peered":"2024-09-10T16:00:49.500268+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:00:49.500268+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":7652,"log_dups_size":0,"ondisk_log_size":7652,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31318016,"num_objects":7646,"num_object_clones":0,"num_object_copies":15292,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4354,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7646,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5206,"num_write_kb":20824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4352},{"shards":"2","objects":3294}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'7588","reported_seq":7717,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:51.672936+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:00:51.672936+0000","last_peered":"2024-09-10T16:00:51.672936+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:00:51.672936+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":7588,"log_dups_size":0,"ondisk_log_size":7588,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31072256,"num_objects":7586,"num_object_clones":0,"num_object_copies":15172,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7586,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4219,"num_write_kb":16876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3282},{"shards":"0,2","objects":4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.989413+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T15:59:15.989413+0000","last_peered":"2024-09-10T15:59:15.989413+0000","last_clean":"2024-09-10T15:59:15.989413+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T15:59:15.989413+0000","last_undegraded":"2024-09-10T15:59:15.989413+0000","last_fullsized":"2024-09-10T15:59:15.989413+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'7435","reported_seq":7573,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:51.688783+0000","last_change":"2024-09-10T15:59:16.251610+0000","last_active":"2024-09-10T16:00:51.688783+0000","last_peered":"2024-09-10T16:00:51.688783+0000","last_clean":"2024-09-10T15:59:14.115595+0000","last_became_active":"2024-09-10T15:59:16.244089+0000","last_became_peered":"2024-09-10T15:59:16.244089+0000","last_unstale":"2024-09-10T16:00:51.688783+0000","last_undegraded":"2024-09-10T15:59:15.847127+0000","last_fullsized":"2024-09-10T15:59:15.774437+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":7435,"log_dups_size":0,"ondisk_log_size":7435,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30420992,"num_objects":7427,"num_object_clones":0,"num_object_copies":14854,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7427,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3825,"num_write_kb":15300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3224},{"shards":"0,2","objects":4203}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":89,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.773633+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T15:59:15.773633+0000","last_peered":"2024-09-10T15:59:15.773633+0000","last_clean":"2024-09-10T15:59:15.773633+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T15:59:15.773633+0000","last_undegraded":"2024-09-10T15:59:15.773633+0000","last_fullsized":"2024-09-10T15:59:15.773633+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'7720","reported_seq":7857,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:51.683701+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:00:51.683701+0000","last_peered":"2024-09-10T16:00:51.683701+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:00:51.683701+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":7720,"log_dups_size":0,"ondisk_log_size":7720,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31604736,"num_objects":7716,"num_object_clones":0,"num_object_copies":15432,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7716,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3975,"num_write_kb":15900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3345},{"shards":"0,2","objects":4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.916017+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T15:59:15.916017+0000","last_peered":"2024-09-10T15:59:15.916017+0000","last_clean":"2024-09-10T15:59:15.916017+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T15:59:15.916017+0000","last_undegraded":"2024-09-10T15:59:15.916017+0000","last_fullsized":"2024-09-10T15:59:15.916017+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.776033+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T15:59:15.776033+0000","last_peered":"2024-09-10T15:59:15.776033+0000","last_clean":"2024-09-10T15:59:15.776033+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T15:59:15.776033+0000","last_undegraded":"2024-09-10T15:59:15.776033+0000","last_fullsized":"2024-09-10T15:59:15.776033+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'7694","reported_seq":7833,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:51.717904+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:00:51.717904+0000","last_peered":"2024-09-10T16:00:51.717904+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:00:51.717904+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":7694,"log_dups_size":0,"ondisk_log_size":7694,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31466438,"num_objects":7684,"num_object_clones":0,"num_object_copies":15368,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7684,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3857,"num_write_kb":15428,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3361},{"shards":"0,2","objects":4323}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":495334110,"num_objects":120940,"num_object_clones":0,"num_object_copies":241880,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":47117,"num_objects_misplaced":2195,"num_objects_unfound":0,"num_objects_dirty":120940,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":78786,"num_write_kb":315144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2077,"num_bytes_recovered":8503296,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":996085760,"data_stored":995934014,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":121012,"ondisk_log_size":121012,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986210,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":549016,"kb_used_data":377096,"kb_used_omap":4,"kb_used_meta":171899,"kb_avail":93822824,"statfs":{"total":96636764160,"available":96074571776,"internally_reserved":0,"allocated":386146304,"data_stored":385868544,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":176025258},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,2,3,6,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659156,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":305452,"kb_used_data":161308,"kb_used_omap":4,"kb_used_meta":144123,"kb_avail":94066388,"statfs":{"total":96636764160,"available":96323981312,"internally_reserved":0,"allocated":165179392,"data_stored":164854199,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":147582638},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,5,3,3,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182439,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":707240,"kb_used_data":432664,"kb_used_omap":4,"kb_used_meta":274555,"kb_avail":93664600,"statfs":{"total":96636764160,"available":95912550400,"internally_reserved":0,"allocated":443047936,"data_stored":442766099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":281145002},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,3,3,4,8,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":443678720,"data_stored":443642590,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":165986304,"data_stored":165906818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":386420736,"data_stored":386384606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:00:54.983 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 142 20 122916 122896 3.38019 1.80078 0.0605527 0.0231056 2024-09-10T16:00:55.983 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 143 20 123449 123429 3.37111 2.08203 0.0283102 0.0231674 2024-09-10T16:00:56.983 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 144 20 123961 123941 3.36159 2 0.0194585 0.0232336 2024-09-10T16:00:57.961 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:00:57.984 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 145 20 124468 124448 3.35207 1.98047 0.0212517 0.0232978 2024-09-10T16:00:58.536 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:00:58.537 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:00:58.567 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":147,"stamp":"2024-09-10T16:00:57.576878+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":505443057,"num_objects":123410,"num_object_clones":0,"num_object_copies":246820,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":47117,"num_objects_misplaced":2064,"num_objects_unfound":0,"num_objects_dirty":123410,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":81256,"num_write_kb":325018,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2210,"num_bytes_recovered":9039891,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":123483,"ondisk_log_size":123483,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1583724,"kb_used_data":991676,"kb_used_omap":12,"kb_used_meta":591987,"kb_avail":281531796,"statfs":{"total":289910292480,"available":288288559104,"internally_reserved":0,"allocated":1015476224,"data_stored":1014591434,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":606194690},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,4,4,7,10,13],"upper_bound":128},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":26853376,"num_objects":6556,"num_object_clones":0,"num_object_copies":13112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-252,"num_objects_unfound":0,"num_objects_dirty":6556,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6556,"num_write_kb":26224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":252,"num_bytes_recovered":1032192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005119"},"pg_stats":[{"pgid":"2.f","version":"43'7712","reported_seq":7780,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:56.746607+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:00:56.746607+0000","last_peered":"2024-09-10T16:00:56.746607+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:00:56.746607+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":7712,"log_dups_size":0,"ondisk_log_size":7712,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31588352,"num_objects":7712,"num_object_clones":0,"num_object_copies":15424,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7712,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7712,"num_write_kb":30848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3453},{"shards":"0,2","objects":4258}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'7633","reported_seq":7740,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:00:54.518562+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:00:54.518562+0000","last_peered":"2024-09-10T16:00:54.518562+0000","last_clean":"2024-09-10T16:00:54.518562+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:00:54.518562+0000","last_undegraded":"2024-09-10T16:00:54.518562+0000","last_fullsized":"2024-09-10T16:00:54.518562+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":7633,"log_dups_size":0,"ondisk_log_size":7633,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31241158,"num_objects":7629,"num_object_clones":0,"num_object_copies":15258,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7629,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5738,"num_write_kb":22952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'7623","reported_seq":12159,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T16:00:54.519824+0000","last_change":"2024-09-10T15:59:16.142612+0000","last_active":"2024-09-10T16:00:54.519824+0000","last_peered":"2024-09-10T16:00:54.519824+0000","last_clean":"2024-09-10T15:59:14.092473+0000","last_became_active":"2024-09-10T15:59:16.109611+0000","last_became_peered":"2024-09-10T15:59:16.109611+0000","last_unstale":"2024-09-10T16:00:54.519824+0000","last_undegraded":"2024-09-10T16:00:54.519824+0000","last_fullsized":"2024-09-10T15:59:15.770264+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":7623,"log_dups_size":0,"ondisk_log_size":7623,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31207424,"num_objects":7619,"num_object_clones":0,"num_object_copies":15238,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2064,"num_objects_unfound":0,"num_objects_dirty":7619,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5627,"num_write_kb":22508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2205,"num_bytes_recovered":9027584,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2064},{"shards":"2","objects":5555}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'7609","reported_seq":7685,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:00:54.344023+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:00:54.344023+0000","last_peered":"2024-09-10T16:00:54.344023+0000","last_clean":"2024-09-10T16:00:54.344023+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:00:54.344023+0000","last_undegraded":"2024-09-10T16:00:54.344023+0000","last_fullsized":"2024-09-10T16:00:54.344023+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":7609,"log_dups_size":0,"ondisk_log_size":7609,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31166464,"num_objects":7609,"num_object_clones":0,"num_object_copies":15218,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7609,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7609,"num_write_kb":30436,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'7820","reported_seq":7945,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:00:56.709172+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:00:56.709172+0000","last_peered":"2024-09-10T16:00:56.709172+0000","last_clean":"2024-09-10T16:00:56.709172+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:00:56.709172+0000","last_undegraded":"2024-09-10T16:00:56.709172+0000","last_fullsized":"2024-09-10T16:00:56.709172+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":7820,"log_dups_size":0,"ondisk_log_size":7820,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32002531,"num_objects":7814,"num_object_clones":0,"num_object_copies":15628,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7814,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4531,"num_write_kb":18124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'7826","reported_seq":7959,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:56.692091+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:00:56.692091+0000","last_peered":"2024-09-10T16:00:56.692091+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:00:56.692091+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":7826,"log_dups_size":0,"ondisk_log_size":7826,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32035299,"num_objects":7822,"num_object_clones":0,"num_object_copies":15644,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7822,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4438,"num_write_kb":17752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3451},{"shards":"0,2","objects":4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'7771","reported_seq":7903,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:56.751621+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:00:56.751621+0000","last_peered":"2024-09-10T16:00:56.751621+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:00:56.751621+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":7771,"log_dups_size":0,"ondisk_log_size":7771,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31806406,"num_objects":7767,"num_object_clones":0,"num_object_copies":15534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7767,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4270,"num_write_kb":17080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3530},{"shards":"0,2","objects":4237}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'7897","reported_seq":8022,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:00:56.646756+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:00:56.646756+0000","last_peered":"2024-09-10T16:00:56.646756+0000","last_clean":"2024-09-10T16:00:56.646756+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:00:56.646756+0000","last_undegraded":"2024-09-10T16:00:56.646756+0000","last_fullsized":"2024-09-10T16:00:56.646756+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":7897,"log_dups_size":0,"ondisk_log_size":7897,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32337920,"num_objects":7895,"num_object_clones":0,"num_object_copies":15790,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7895,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4744,"num_write_kb":18976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.775865+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T15:59:15.775865+0000","last_peered":"2024-09-10T15:59:15.775865+0000","last_clean":"2024-09-10T15:59:15.775865+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T15:59:15.775865+0000","last_undegraded":"2024-09-10T15:59:15.775865+0000","last_fullsized":"2024-09-10T15:59:15.775865+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'7341","reported_seq":7474,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:54.537961+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:00:54.537961+0000","last_peered":"2024-09-10T16:00:54.537961+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:00:54.537961+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":7341,"log_dups_size":0,"ondisk_log_size":7341,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30040547,"num_objects":7335,"num_object_clones":0,"num_object_copies":14670,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7335,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5160,"num_write_kb":20640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4123},{"shards":"2","objects":3212}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":89,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.772619+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T15:59:15.772619+0000","last_peered":"2024-09-10T15:59:15.772619+0000","last_clean":"2024-09-10T15:59:15.772619+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T15:59:15.772619+0000","last_undegraded":"2024-09-10T15:59:15.772619+0000","last_fullsized":"2024-09-10T15:59:15.772619+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'7729","reported_seq":7854,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:56.686812+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:00:56.686812+0000","last_peered":"2024-09-10T16:00:56.686812+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:00:56.686812+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":7729,"log_dups_size":0,"ondisk_log_size":7729,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31625216,"num_objects":7721,"num_object_clones":0,"num_object_copies":15442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7721,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4233,"num_write_kb":16932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3466},{"shards":"0,2","objects":4255}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":99,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.884297+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T15:59:15.884297+0000","last_peered":"2024-09-10T15:59:15.884297+0000","last_clean":"2024-09-10T15:59:15.884297+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T15:59:15.884297+0000","last_undegraded":"2024-09-10T15:59:15.884297+0000","last_fullsized":"2024-09-10T15:59:15.884297+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'7662","reported_seq":7792,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:54.365018+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:00:54.365018+0000","last_peered":"2024-09-10T16:00:54.365018+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:00:54.365018+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":7662,"log_dups_size":0,"ondisk_log_size":7662,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31363555,"num_objects":7658,"num_object_clones":0,"num_object_copies":15316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7658,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5342,"num_write_kb":21368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4304},{"shards":"2","objects":3354}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.837509+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T15:59:15.837509+0000","last_peered":"2024-09-10T15:59:15.837509+0000","last_clean":"2024-09-10T15:59:15.837509+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T15:59:15.837509+0000","last_undegraded":"2024-09-10T15:59:15.837509+0000","last_fullsized":"2024-09-10T15:59:15.837509+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'7794","reported_seq":7917,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:54.518155+0000","last_change":"2024-09-10T15:59:16.174113+0000","last_active":"2024-09-10T16:00:54.518155+0000","last_peered":"2024-09-10T16:00:54.518155+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:00:54.518155+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":7794,"log_dups_size":0,"ondisk_log_size":7794,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31899648,"num_objects":7788,"num_object_clones":0,"num_object_copies":15576,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4354,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7788,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5348,"num_write_kb":21392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4352},{"shards":"2","objects":3436}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'7730","reported_seq":7860,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:56.681530+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:00:56.681530+0000","last_peered":"2024-09-10T16:00:56.681530+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:00:56.681530+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":7730,"log_dups_size":0,"ondisk_log_size":7730,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31653888,"num_objects":7728,"num_object_clones":0,"num_object_copies":15456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7728,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4361,"num_write_kb":17444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3424},{"shards":"0,2","objects":4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.989413+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T15:59:15.989413+0000","last_peered":"2024-09-10T15:59:15.989413+0000","last_clean":"2024-09-10T15:59:15.989413+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T15:59:15.989413+0000","last_undegraded":"2024-09-10T15:59:15.989413+0000","last_fullsized":"2024-09-10T15:59:15.989413+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'7600","reported_seq":7738,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:56.719805+0000","last_change":"2024-09-10T15:59:16.251610+0000","last_active":"2024-09-10T16:00:56.719805+0000","last_peered":"2024-09-10T16:00:56.719805+0000","last_clean":"2024-09-10T15:59:14.115595+0000","last_became_active":"2024-09-10T15:59:16.244089+0000","last_became_peered":"2024-09-10T15:59:16.244089+0000","last_unstale":"2024-09-10T16:00:56.719805+0000","last_undegraded":"2024-09-10T15:59:15.847127+0000","last_fullsized":"2024-09-10T15:59:15.774437+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":7600,"log_dups_size":0,"ondisk_log_size":7600,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31096832,"num_objects":7592,"num_object_clones":0,"num_object_copies":15184,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7592,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3990,"num_write_kb":15960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3389},{"shards":"0,2","objects":4203}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":89,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.773633+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T15:59:15.773633+0000","last_peered":"2024-09-10T15:59:15.773633+0000","last_clean":"2024-09-10T15:59:15.773633+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T15:59:15.773633+0000","last_undegraded":"2024-09-10T15:59:15.773633+0000","last_fullsized":"2024-09-10T15:59:15.773633+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'7878","reported_seq":8015,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:56.739697+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:00:56.739697+0000","last_peered":"2024-09-10T16:00:56.739697+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:00:56.739697+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":7878,"log_dups_size":0,"ondisk_log_size":7878,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32251904,"num_objects":7874,"num_object_clones":0,"num_object_copies":15748,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7874,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4133,"num_write_kb":16532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3503},{"shards":"0,2","objects":4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.916017+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T15:59:15.916017+0000","last_peered":"2024-09-10T15:59:15.916017+0000","last_clean":"2024-09-10T15:59:15.916017+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T15:59:15.916017+0000","last_undegraded":"2024-09-10T15:59:15.916017+0000","last_fullsized":"2024-09-10T15:59:15.916017+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.776033+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T15:59:15.776033+0000","last_peered":"2024-09-10T15:59:15.776033+0000","last_clean":"2024-09-10T15:59:15.776033+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T15:59:15.776033+0000","last_undegraded":"2024-09-10T15:59:15.776033+0000","last_fullsized":"2024-09-10T15:59:15.776033+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'7855","reported_seq":7994,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:56.726018+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:00:56.726018+0000","last_peered":"2024-09-10T16:00:56.726018+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:00:56.726018+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":7855,"log_dups_size":0,"ondisk_log_size":7855,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32125894,"num_objects":7845,"num_object_clones":0,"num_object_copies":15690,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7845,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4018,"num_write_kb":16072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3522},{"shards":"0,2","objects":4323}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":505443038,"num_objects":123408,"num_object_clones":0,"num_object_copies":246816,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":47117,"num_objects_misplaced":2064,"num_objects_unfound":0,"num_objects_dirty":123408,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":81254,"num_write_kb":325016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2208,"num_bytes_recovered":9039872,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1017118720,"data_stored":1016966974,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":123480,"ondisk_log_size":123480,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986211,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":576116,"kb_used_data":381988,"kb_used_omap":4,"kb_used_meta":194107,"kb_avail":93795724,"statfs":{"total":96636764160,"available":96046821376,"internally_reserved":0,"allocated":391155712,"data_stored":390877952,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":198766250},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,2,1,5,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659157,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":305176,"kb_used_data":169352,"kb_used_omap":4,"kb_used_meta":135803,"kb_avail":94066664,"statfs":{"total":96636764160,"available":96324263936,"internally_reserved":0,"allocated":173416448,"data_stored":173091255,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":139062958},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,2,3,2,7],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182440,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":702432,"kb_used_data":440336,"kb_used_omap":4,"kb_used_meta":262075,"kb_avail":93669408,"statfs":{"total":96636764160,"available":95917473792,"internally_reserved":0,"allocated":450904064,"data_stored":450622227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":268365482},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,3,3,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":451428352,"data_stored":451392222,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":174379008,"data_stored":174299522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":391311360,"data_stored":391275230,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:00:58.569 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:00:58.983 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 146 20 124918 124898 3.34115 1.75781 0.0470848 0.0233707 2024-09-10T16:00:59.157 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:00:59.158 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:00:59.185 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":147,"stamp":"2024-09-10T16:00:57.576878+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":505443057,"num_objects":123410,"num_object_clones":0,"num_object_copies":246820,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":47117,"num_objects_misplaced":2064,"num_objects_unfound":0,"num_objects_dirty":123410,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":81256,"num_write_kb":325018,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2210,"num_bytes_recovered":9039891,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":123483,"ondisk_log_size":123483,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1583724,"kb_used_data":991676,"kb_used_omap":12,"kb_used_meta":591987,"kb_avail":281531796,"statfs":{"total":289910292480,"available":288288559104,"internally_reserved":0,"allocated":1015476224,"data_stored":1014591434,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":606194690},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,4,4,7,10,13],"upper_bound":128},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":26853376,"num_objects":6556,"num_object_clones":0,"num_object_copies":13112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-252,"num_objects_unfound":0,"num_objects_dirty":6556,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6556,"num_write_kb":26224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":252,"num_bytes_recovered":1032192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005119"},"pg_stats":[{"pgid":"2.f","version":"43'7712","reported_seq":7780,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:56.746607+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:00:56.746607+0000","last_peered":"2024-09-10T16:00:56.746607+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:00:56.746607+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":7712,"log_dups_size":0,"ondisk_log_size":7712,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31588352,"num_objects":7712,"num_object_clones":0,"num_object_copies":15424,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7712,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7712,"num_write_kb":30848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3453},{"shards":"0,2","objects":4258}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'7633","reported_seq":7740,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:00:54.518562+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:00:54.518562+0000","last_peered":"2024-09-10T16:00:54.518562+0000","last_clean":"2024-09-10T16:00:54.518562+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:00:54.518562+0000","last_undegraded":"2024-09-10T16:00:54.518562+0000","last_fullsized":"2024-09-10T16:00:54.518562+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":7633,"log_dups_size":0,"ondisk_log_size":7633,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31241158,"num_objects":7629,"num_object_clones":0,"num_object_copies":15258,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7629,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5738,"num_write_kb":22952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'7623","reported_seq":12159,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T16:00:54.519824+0000","last_change":"2024-09-10T15:59:16.142612+0000","last_active":"2024-09-10T16:00:54.519824+0000","last_peered":"2024-09-10T16:00:54.519824+0000","last_clean":"2024-09-10T15:59:14.092473+0000","last_became_active":"2024-09-10T15:59:16.109611+0000","last_became_peered":"2024-09-10T15:59:16.109611+0000","last_unstale":"2024-09-10T16:00:54.519824+0000","last_undegraded":"2024-09-10T16:00:54.519824+0000","last_fullsized":"2024-09-10T15:59:15.770264+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":7623,"log_dups_size":0,"ondisk_log_size":7623,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31207424,"num_objects":7619,"num_object_clones":0,"num_object_copies":15238,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2064,"num_objects_unfound":0,"num_objects_dirty":7619,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5627,"num_write_kb":22508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2205,"num_bytes_recovered":9027584,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2064},{"shards":"2","objects":5555}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'7609","reported_seq":7685,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:00:54.344023+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:00:54.344023+0000","last_peered":"2024-09-10T16:00:54.344023+0000","last_clean":"2024-09-10T16:00:54.344023+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:00:54.344023+0000","last_undegraded":"2024-09-10T16:00:54.344023+0000","last_fullsized":"2024-09-10T16:00:54.344023+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":7609,"log_dups_size":0,"ondisk_log_size":7609,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31166464,"num_objects":7609,"num_object_clones":0,"num_object_copies":15218,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7609,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7609,"num_write_kb":30436,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'7820","reported_seq":7945,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:00:56.709172+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:00:56.709172+0000","last_peered":"2024-09-10T16:00:56.709172+0000","last_clean":"2024-09-10T16:00:56.709172+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:00:56.709172+0000","last_undegraded":"2024-09-10T16:00:56.709172+0000","last_fullsized":"2024-09-10T16:00:56.709172+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":7820,"log_dups_size":0,"ondisk_log_size":7820,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32002531,"num_objects":7814,"num_object_clones":0,"num_object_copies":15628,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7814,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4531,"num_write_kb":18124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'7826","reported_seq":7959,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:56.692091+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:00:56.692091+0000","last_peered":"2024-09-10T16:00:56.692091+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:00:56.692091+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":7826,"log_dups_size":0,"ondisk_log_size":7826,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32035299,"num_objects":7822,"num_object_clones":0,"num_object_copies":15644,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7822,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4438,"num_write_kb":17752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3451},{"shards":"0,2","objects":4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'7771","reported_seq":7903,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:56.751621+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:00:56.751621+0000","last_peered":"2024-09-10T16:00:56.751621+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:00:56.751621+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":7771,"log_dups_size":0,"ondisk_log_size":7771,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31806406,"num_objects":7767,"num_object_clones":0,"num_object_copies":15534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7767,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4270,"num_write_kb":17080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3530},{"shards":"0,2","objects":4237}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'7897","reported_seq":8022,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:00:56.646756+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:00:56.646756+0000","last_peered":"2024-09-10T16:00:56.646756+0000","last_clean":"2024-09-10T16:00:56.646756+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:00:56.646756+0000","last_undegraded":"2024-09-10T16:00:56.646756+0000","last_fullsized":"2024-09-10T16:00:56.646756+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":7897,"log_dups_size":0,"ondisk_log_size":7897,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32337920,"num_objects":7895,"num_object_clones":0,"num_object_copies":15790,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7895,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4744,"num_write_kb":18976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.775865+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T15:59:15.775865+0000","last_peered":"2024-09-10T15:59:15.775865+0000","last_clean":"2024-09-10T15:59:15.775865+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T15:59:15.775865+0000","last_undegraded":"2024-09-10T15:59:15.775865+0000","last_fullsized":"2024-09-10T15:59:15.775865+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'7341","reported_seq":7474,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:54.537961+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:00:54.537961+0000","last_peered":"2024-09-10T16:00:54.537961+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:00:54.537961+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":7341,"log_dups_size":0,"ondisk_log_size":7341,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30040547,"num_objects":7335,"num_object_clones":0,"num_object_copies":14670,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7335,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5160,"num_write_kb":20640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4123},{"shards":"2","objects":3212}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":89,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.772619+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T15:59:15.772619+0000","last_peered":"2024-09-10T15:59:15.772619+0000","last_clean":"2024-09-10T15:59:15.772619+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T15:59:15.772619+0000","last_undegraded":"2024-09-10T15:59:15.772619+0000","last_fullsized":"2024-09-10T15:59:15.772619+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'7729","reported_seq":7854,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:56.686812+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:00:56.686812+0000","last_peered":"2024-09-10T16:00:56.686812+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:00:56.686812+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":7729,"log_dups_size":0,"ondisk_log_size":7729,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31625216,"num_objects":7721,"num_object_clones":0,"num_object_copies":15442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7721,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4233,"num_write_kb":16932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3466},{"shards":"0,2","objects":4255}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":99,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.884297+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T15:59:15.884297+0000","last_peered":"2024-09-10T15:59:15.884297+0000","last_clean":"2024-09-10T15:59:15.884297+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T15:59:15.884297+0000","last_undegraded":"2024-09-10T15:59:15.884297+0000","last_fullsized":"2024-09-10T15:59:15.884297+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'7662","reported_seq":7792,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:54.365018+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:00:54.365018+0000","last_peered":"2024-09-10T16:00:54.365018+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:00:54.365018+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":7662,"log_dups_size":0,"ondisk_log_size":7662,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31363555,"num_objects":7658,"num_object_clones":0,"num_object_copies":15316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7658,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5342,"num_write_kb":21368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4304},{"shards":"2","objects":3354}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.837509+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T15:59:15.837509+0000","last_peered":"2024-09-10T15:59:15.837509+0000","last_clean":"2024-09-10T15:59:15.837509+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T15:59:15.837509+0000","last_undegraded":"2024-09-10T15:59:15.837509+0000","last_fullsized":"2024-09-10T15:59:15.837509+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'7794","reported_seq":7917,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:54.518155+0000","last_change":"2024-09-10T15:59:16.174113+0000","last_active":"2024-09-10T16:00:54.518155+0000","last_peered":"2024-09-10T16:00:54.518155+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:00:54.518155+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":7794,"log_dups_size":0,"ondisk_log_size":7794,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31899648,"num_objects":7788,"num_object_clones":0,"num_object_copies":15576,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4354,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7788,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5348,"num_write_kb":21392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4352},{"shards":"2","objects":3436}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'7730","reported_seq":7860,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:56.681530+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:00:56.681530+0000","last_peered":"2024-09-10T16:00:56.681530+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:00:56.681530+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":7730,"log_dups_size":0,"ondisk_log_size":7730,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31653888,"num_objects":7728,"num_object_clones":0,"num_object_copies":15456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7728,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4361,"num_write_kb":17444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3424},{"shards":"0,2","objects":4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.989413+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T15:59:15.989413+0000","last_peered":"2024-09-10T15:59:15.989413+0000","last_clean":"2024-09-10T15:59:15.989413+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T15:59:15.989413+0000","last_undegraded":"2024-09-10T15:59:15.989413+0000","last_fullsized":"2024-09-10T15:59:15.989413+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'7600","reported_seq":7738,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:56.719805+0000","last_change":"2024-09-10T15:59:16.251610+0000","last_active":"2024-09-10T16:00:56.719805+0000","last_peered":"2024-09-10T16:00:56.719805+0000","last_clean":"2024-09-10T15:59:14.115595+0000","last_became_active":"2024-09-10T15:59:16.244089+0000","last_became_peered":"2024-09-10T15:59:16.244089+0000","last_unstale":"2024-09-10T16:00:56.719805+0000","last_undegraded":"2024-09-10T15:59:15.847127+0000","last_fullsized":"2024-09-10T15:59:15.774437+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":7600,"log_dups_size":0,"ondisk_log_size":7600,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31096832,"num_objects":7592,"num_object_clones":0,"num_object_copies":15184,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7592,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3990,"num_write_kb":15960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3389},{"shards":"0,2","objects":4203}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":89,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.773633+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T15:59:15.773633+0000","last_peered":"2024-09-10T15:59:15.773633+0000","last_clean":"2024-09-10T15:59:15.773633+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T15:59:15.773633+0000","last_undegraded":"2024-09-10T15:59:15.773633+0000","last_fullsized":"2024-09-10T15:59:15.773633+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'7878","reported_seq":8015,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:56.739697+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:00:56.739697+0000","last_peered":"2024-09-10T16:00:56.739697+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:00:56.739697+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":7878,"log_dups_size":0,"ondisk_log_size":7878,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32251904,"num_objects":7874,"num_object_clones":0,"num_object_copies":15748,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7874,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4133,"num_write_kb":16532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3503},{"shards":"0,2","objects":4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.916017+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T15:59:15.916017+0000","last_peered":"2024-09-10T15:59:15.916017+0000","last_clean":"2024-09-10T15:59:15.916017+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T15:59:15.916017+0000","last_undegraded":"2024-09-10T15:59:15.916017+0000","last_fullsized":"2024-09-10T15:59:15.916017+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.776033+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T15:59:15.776033+0000","last_peered":"2024-09-10T15:59:15.776033+0000","last_clean":"2024-09-10T15:59:15.776033+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T15:59:15.776033+0000","last_undegraded":"2024-09-10T15:59:15.776033+0000","last_fullsized":"2024-09-10T15:59:15.776033+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'7855","reported_seq":7994,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:56.726018+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:00:56.726018+0000","last_peered":"2024-09-10T16:00:56.726018+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:00:56.726018+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":7855,"log_dups_size":0,"ondisk_log_size":7855,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32125894,"num_objects":7845,"num_object_clones":0,"num_object_copies":15690,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7845,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4018,"num_write_kb":16072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3522},{"shards":"0,2","objects":4323}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":505443038,"num_objects":123408,"num_object_clones":0,"num_object_copies":246816,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":47117,"num_objects_misplaced":2064,"num_objects_unfound":0,"num_objects_dirty":123408,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":81254,"num_write_kb":325016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2208,"num_bytes_recovered":9039872,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1017118720,"data_stored":1016966974,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":123480,"ondisk_log_size":123480,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986211,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":576116,"kb_used_data":381988,"kb_used_omap":4,"kb_used_meta":194107,"kb_avail":93795724,"statfs":{"total":96636764160,"available":96046821376,"internally_reserved":0,"allocated":391155712,"data_stored":390877952,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":198766250},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,2,1,5,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659157,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":305176,"kb_used_data":169352,"kb_used_omap":4,"kb_used_meta":135803,"kb_avail":94066664,"statfs":{"total":96636764160,"available":96324263936,"internally_reserved":0,"allocated":173416448,"data_stored":173091255,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":139062958},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,2,3,2,7],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182440,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":702432,"kb_used_data":440336,"kb_used_omap":4,"kb_used_meta":262075,"kb_avail":93669408,"statfs":{"total":96636764160,"available":95917473792,"internally_reserved":0,"allocated":450904064,"data_stored":450622227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":268365482},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,3,3,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":451428352,"data_stored":451392222,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":174379008,"data_stored":174299522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":391311360,"data_stored":391275230,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:00:59.187 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-10T16:00:59.187 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "43'7712", 'reported_seq': 7780, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:00:56.746607+0000', 'last_change': '2024-09-10T15:59:16.281558+0000', 'last_active': '2024-09-10T16:00:56.746607+0000', 'last_peered': '2024-09-10T16:00:56.746607+0000', 'last_clean': '2024-09-10T15:59:14.132456+0000', 'last_became_active': '2024-09-10T15:59:16.274407+0000', 'last_became_peered': '2024-09-10T15:59:16.274407+0000', 'last_unstale': '2024-09-10T16:00:56.746607+0000', 'last_undegraded': '2024-09-10T15:59:15.851279+0000', 'last_fullsized': '2024-09-10T15:59:15.774356+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'objects_scrubbed': 0, 'log_size': 7712, 'log_dups_size': 0, 'ondisk_log_size': 7712, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31588352, 'num_objects': 7712, 'num_object_clones': 0, 'num_object_copies': 15424, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4259, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7712, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7712, 'num_write_kb': 30848, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3453}, {'shards': '0,2', 'objects': 4258}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:00:59.187 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-10T16:00:59.187 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "43'7623", 'reported_seq': 12159, 'reported_epoch': 43, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-10T16:00:54.519824+0000', 'last_change': '2024-09-10T15:59:16.142612+0000', 'last_active': '2024-09-10T16:00:54.519824+0000', 'last_peered': '2024-09-10T16:00:54.519824+0000', 'last_clean': '2024-09-10T15:59:14.092473+0000', 'last_became_active': '2024-09-10T15:59:16.109611+0000', 'last_became_peered': '2024-09-10T15:59:16.109611+0000', 'last_unstale': '2024-09-10T16:00:54.519824+0000', 'last_undegraded': '2024-09-10T16:00:54.519824+0000', 'last_fullsized': '2024-09-10T15:59:15.770264+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2029", 'last_scrub_stamp': '2024-09-10T15:58:45.933789+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:45.933789+0000', 'objects_scrubbed': 1992, 'log_size': 7623, 'log_dups_size': 0, 'ondisk_log_size': 7623, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 668, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31207424, 'num_objects': 7619, 'num_object_clones': 0, 'num_object_copies': 15238, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 2064, 'num_objects_unfound': 0, 'num_objects_dirty': 7619, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5627, 'num_write_kb': 22508, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2205, 'num_bytes_recovered': 9027584, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2064}, {'shards': '2', 'objects': 5555}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:00:59.187 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-10T16:00:59.187 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "43'7826", 'reported_seq': 7959, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:00:56.692091+0000', 'last_change': '2024-09-10T15:59:16.308264+0000', 'last_active': '2024-09-10T16:00:56.692091+0000', 'last_peered': '2024-09-10T16:00:56.692091+0000', 'last_clean': '2024-09-10T15:59:14.111551+0000', 'last_became_active': '2024-09-10T15:59:16.300040+0000', 'last_became_peered': '2024-09-10T15:59:16.300040+0000', 'last_unstale': '2024-09-10T16:00:56.692091+0000', 'last_undegraded': '2024-09-10T15:59:15.878234+0000', 'last_fullsized': '2024-09-10T15:59:15.769328+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3442", 'last_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'objects_scrubbed': 3384, 'log_size': 7826, 'log_dups_size': 0, 'ondisk_log_size': 7826, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1292, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32035299, 'num_objects': 7822, 'num_object_clones': 0, 'num_object_copies': 15644, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4372, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7822, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 4438, 'num_write_kb': 17752, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3451}, {'shards': '0,2', 'objects': 4371}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:00:59.187 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-10T16:00:59.187 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "43'7771", 'reported_seq': 7903, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:00:56.751621+0000', 'last_change': '2024-09-10T15:59:16.331899+0000', 'last_active': '2024-09-10T16:00:56.751621+0000', 'last_peered': '2024-09-10T16:00:56.751621+0000', 'last_clean': '2024-09-10T15:59:14.139436+0000', 'last_became_active': '2024-09-10T15:59:16.318715+0000', 'last_became_peered': '2024-09-10T15:59:16.318715+0000', 'last_unstale': '2024-09-10T16:00:56.751621+0000', 'last_undegraded': '2024-09-10T15:59:15.877524+0000', 'last_fullsized': '2024-09-10T15:59:15.769042+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3537", 'last_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'objects_scrubbed': 3497, 'log_size': 7771, 'log_dups_size': 0, 'ondisk_log_size': 7771, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1224, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31806406, 'num_objects': 7767, 'num_object_clones': 0, 'num_object_copies': 15534, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4237, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7767, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 4270, 'num_write_kb': 17080, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3530}, {'shards': '0,2', 'objects': 4237}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:00:59.187 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-10T16:00:59.188 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "43'7341", 'reported_seq': 7474, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:00:54.537961+0000', 'last_change': '2024-09-10T15:59:16.175680+0000', 'last_active': '2024-09-10T16:00:54.537961+0000', 'last_peered': '2024-09-10T16:00:54.537961+0000', 'last_clean': '2024-09-10T15:59:14.088187+0000', 'last_became_active': '2024-09-10T15:59:16.165810+0000', 'last_became_peered': '2024-09-10T15:59:16.165810+0000', 'last_unstale': '2024-09-10T16:00:54.537961+0000', 'last_undegraded': '2024-09-10T15:59:15.822098+0000', 'last_fullsized': '2024-09-10T15:59:15.769123+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2218", 'last_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'objects_scrubbed': 2175, 'log_size': 7341, 'log_dups_size': 0, 'ondisk_log_size': 7341, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 825, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30040547, 'num_objects': 7335, 'num_object_clones': 0, 'num_object_copies': 14670, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4124, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7335, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 5160, 'num_write_kb': 20640, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4123}, {'shards': '2', 'objects': 3212}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:00:59.188 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-10T16:00:59.188 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "43'7729", 'reported_seq': 7854, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:00:56.686812+0000', 'last_change': '2024-09-10T15:59:16.277742+0000', 'last_active': '2024-09-10T16:00:56.686812+0000', 'last_peered': '2024-09-10T16:00:56.686812+0000', 'last_clean': '2024-09-10T15:59:14.129081+0000', 'last_became_active': '2024-09-10T15:59:16.269835+0000', 'last_became_peered': '2024-09-10T15:59:16.269835+0000', 'last_unstale': '2024-09-10T16:00:56.686812+0000', 'last_undegraded': '2024-09-10T15:59:15.860857+0000', 'last_fullsized': '2024-09-10T15:59:15.771410+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3557", 'last_scrub_stamp': '2024-09-10T15:59:02.493678+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:02.493678+0000', 'objects_scrubbed': 3488, 'log_size': 7729, 'log_dups_size': 0, 'ondisk_log_size': 7729, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1472, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31625216, 'num_objects': 7721, 'num_object_clones': 0, 'num_object_copies': 15442, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4257, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7721, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4233, 'num_write_kb': 16932, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3466}, {'shards': '0,2', 'objects': 4255}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:00:59.188 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-10T16:00:59.188 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "43'7662", 'reported_seq': 7792, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:00:54.365018+0000', 'last_change': '2024-09-10T15:59:16.284311+0000', 'last_active': '2024-09-10T16:00:54.365018+0000', 'last_peered': '2024-09-10T16:00:54.365018+0000', 'last_clean': '2024-09-10T15:59:14.116041+0000', 'last_became_active': '2024-09-10T15:59:16.274543+0000', 'last_became_peered': '2024-09-10T15:59:16.274543+0000', 'last_unstale': '2024-09-10T16:00:54.365018+0000', 'last_undegraded': '2024-09-10T15:59:15.812484+0000', 'last_fullsized': '2024-09-10T15:59:15.770885+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2356", 'last_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'objects_scrubbed': 2316, 'log_size': 7662, 'log_dups_size': 0, 'ondisk_log_size': 7662, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 783, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31363555, 'num_objects': 7658, 'num_object_clones': 0, 'num_object_copies': 15316, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7658, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 5342, 'num_write_kb': 21368, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4304}, {'shards': '2', 'objects': 3354}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:00:59.188 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-10T16:00:59.188 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "43'7794", 'reported_seq': 7917, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:00:54.518155+0000', 'last_change': '2024-09-10T15:59:16.174113+0000', 'last_active': '2024-09-10T16:00:54.518155+0000', 'last_peered': '2024-09-10T16:00:54.518155+0000', 'last_clean': '2024-09-10T15:59:14.110654+0000', 'last_became_active': '2024-09-10T15:59:16.164645+0000', 'last_became_peered': '2024-09-10T15:59:16.164645+0000', 'last_unstale': '2024-09-10T16:00:54.518155+0000', 'last_undegraded': '2024-09-10T15:59:15.803593+0000', 'last_fullsized': '2024-09-10T15:59:15.769083+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2486", 'last_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'objects_scrubbed': 2440, 'log_size': 7794, 'log_dups_size': 0, 'ondisk_log_size': 7794, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 792, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31899648, 'num_objects': 7788, 'num_object_clones': 0, 'num_object_copies': 15576, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4354, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7788, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5348, 'num_write_kb': 21392, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4352}, {'shards': '2', 'objects': 3436}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:00:59.188 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-10T16:00:59.188 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "43'7730", 'reported_seq': 7860, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:00:56.681530+0000', 'last_change': '2024-09-10T15:59:16.315641+0000', 'last_active': '2024-09-10T16:00:56.681530+0000', 'last_peered': '2024-09-10T16:00:56.681530+0000', 'last_clean': '2024-09-10T15:59:14.117001+0000', 'last_became_active': '2024-09-10T15:59:16.302245+0000', 'last_became_peered': '2024-09-10T15:59:16.302245+0000', 'last_unstale': '2024-09-10T16:00:56.681530+0000', 'last_undegraded': '2024-09-10T15:59:15.845012+0000', 'last_fullsized': '2024-09-10T15:59:15.770337+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3404", 'last_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'objects_scrubbed': 3367, 'log_size': 7730, 'log_dups_size': 0, 'ondisk_log_size': 7730, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 983, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31653888, 'num_objects': 7728, 'num_object_clones': 0, 'num_object_copies': 15456, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7728, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4361, 'num_write_kb': 17444, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3424}, {'shards': '0,2', 'objects': 4304}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:00:59.189 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-10T16:00:59.189 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "43'7600", 'reported_seq': 7738, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:00:56.719805+0000', 'last_change': '2024-09-10T15:59:16.251610+0000', 'last_active': '2024-09-10T16:00:56.719805+0000', 'last_peered': '2024-09-10T16:00:56.719805+0000', 'last_clean': '2024-09-10T15:59:14.115595+0000', 'last_became_active': '2024-09-10T15:59:16.244089+0000', 'last_became_peered': '2024-09-10T15:59:16.244089+0000', 'last_unstale': '2024-09-10T16:00:56.719805+0000', 'last_undegraded': '2024-09-10T15:59:15.847127+0000', 'last_fullsized': '2024-09-10T15:59:15.774437+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3653", 'last_scrub_stamp': '2024-09-10T15:59:05.298468+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:05.298468+0000', 'objects_scrubbed': 3602, 'log_size': 7600, 'log_dups_size': 0, 'ondisk_log_size': 7600, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1289, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31096832, 'num_objects': 7592, 'num_object_clones': 0, 'num_object_copies': 15184, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4206, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7592, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3990, 'num_write_kb': 15960, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3389}, {'shards': '0,2', 'objects': 4203}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:00:59.189 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-10T16:00:59.189 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "43'7878", 'reported_seq': 8015, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:00:56.739697+0000', 'last_change': '2024-09-10T15:59:16.311291+0000', 'last_active': '2024-09-10T16:00:56.739697+0000', 'last_peered': '2024-09-10T16:00:56.739697+0000', 'last_clean': '2024-09-10T15:59:14.116363+0000', 'last_became_active': '2024-09-10T15:59:16.301244+0000', 'last_became_peered': '2024-09-10T15:59:16.301244+0000', 'last_unstale': '2024-09-10T16:00:56.739697+0000', 'last_undegraded': '2024-09-10T15:59:15.883434+0000', 'last_fullsized': '2024-09-10T15:59:15.772756+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3789", 'last_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'objects_scrubbed': 3741, 'log_size': 7878, 'log_dups_size': 0, 'ondisk_log_size': 7878, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1440, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32251904, 'num_objects': 7874, 'num_object_clones': 0, 'num_object_copies': 15748, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4373, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7874, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4133, 'num_write_kb': 16532, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3503}, {'shards': '0,2', 'objects': 4371}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:00:59.189 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-10T16:00:59.189 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "43'7855", 'reported_seq': 7994, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:00:56.726018+0000', 'last_change': '2024-09-10T15:59:16.313409+0000', 'last_active': '2024-09-10T16:00:56.726018+0000', 'last_peered': '2024-09-10T16:00:56.726018+0000', 'last_clean': '2024-09-10T15:59:14.129780+0000', 'last_became_active': '2024-09-10T15:59:16.300932+0000', 'last_became_peered': '2024-09-10T15:59:16.300932+0000', 'last_unstale': '2024-09-10T16:00:56.726018+0000', 'last_undegraded': '2024-09-10T15:59:15.887823+0000', 'last_fullsized': '2024-09-10T15:59:15.768513+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3880", 'last_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'objects_scrubbed': 3827, 'log_size': 7855, 'log_dups_size': 0, 'ondisk_log_size': 7855, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1397, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32125894, 'num_objects': 7845, 'num_object_clones': 0, 'num_object_copies': 15690, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4325, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7845, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4018, 'num_write_kb': 16072, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3522}, {'shards': '0,2', 'objects': 4323}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:00:59.189 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T16:00:59.886 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:00:59.886 INFO:teuthology.orchestra.run.smithi115.stdout:{"fsid":"7effee29-df05-4351-9741-ee203564ed81","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 47117/246820 objects degraded (19.090%), 11 pgs degraded, 12 pgs undersized","count":23},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":223,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":43,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983954,"num_in_osds":3,"osd_in_since":1725983947,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":123410,"data_bytes":505443057,"bytes_used":1621733376,"bytes_avail":288288559104,"bytes_total":289910292480,"degraded_objects":47117,"degraded_total":246820,"degraded_ratio":0.19089619965967103,"misplaced_objects":2064,"misplaced_total":246820,"misplaced_ratio":0.0083623693379790941,"recovering_objects_per_sec":20,"recovering_bytes_per_sec":85979,"recovering_keys_per_sec":0,"num_objects_recovered":252,"num_bytes_recovered":1032192,"num_keys_recovered":0,"write_bytes_sec":2236827,"read_op_per_sec":0,"write_op_per_sec":546},"fsmap":{"epoch":1,"btime":"2024-09-10T15:57:14:967120+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T16:00:13.560776+0000","services":{}},"progress_events":{}} 2024-09-10T16:00:59.912 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-10T16:00:59.913 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:00:59.984 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 147 20 125382 125362 3.33075 1.8125 0.0355906 0.0234476 2024-09-10T16:01:00.472 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:01:00.472 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:01:00.497 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":148,"stamp":"2024-09-10T16:00:59.577714+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":509412081,"num_objects":124379,"num_object_clones":0,"num_object_copies":248758,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":47117,"num_objects_misplaced":1930,"num_objects_unfound":0,"num_objects_dirty":124379,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":82225,"num_write_kb":328894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2344,"num_bytes_recovered":9588755,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":124452,"ondisk_log_size":124452,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1641104,"kb_used_data":1004640,"kb_used_omap":12,"kb_used_meta":636403,"kb_avail":281474416,"statfs":{"total":289910292480,"available":288229801984,"internally_reserved":0,"allocated":1028751360,"data_stored":1027866570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":651676674},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,1,4,6,15,14,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":24342528,"num_objects":5943,"num_object_clones":0,"num_object_copies":11886,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-386,"num_objects_unfound":0,"num_objects_dirty":5943,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5943,"num_write_kb":23772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":386,"num_bytes_recovered":1581056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005116"},"pg_stats":[{"pgid":"2.f","version":"43'7712","reported_seq":7780,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:56.746607+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:00:56.746607+0000","last_peered":"2024-09-10T16:00:56.746607+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:00:56.746607+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":7712,"log_dups_size":0,"ondisk_log_size":7712,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31588352,"num_objects":7712,"num_object_clones":0,"num_object_copies":15424,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7712,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7712,"num_write_kb":30848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3453},{"shards":"0,2","objects":4258}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'7792","reported_seq":7900,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:00:59.444223+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:00:59.444223+0000","last_peered":"2024-09-10T16:00:59.444223+0000","last_clean":"2024-09-10T16:00:59.444223+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:00:59.444223+0000","last_undegraded":"2024-09-10T16:00:59.444223+0000","last_fullsized":"2024-09-10T16:00:59.444223+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":7792,"log_dups_size":0,"ondisk_log_size":7792,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31892422,"num_objects":7788,"num_object_clones":0,"num_object_copies":15576,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7788,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5897,"num_write_kb":23588,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'7780","reported_seq":12584,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T16:00:59.508712+0000","last_change":"2024-09-10T15:59:16.142612+0000","last_active":"2024-09-10T16:00:59.508712+0000","last_peered":"2024-09-10T16:00:59.508712+0000","last_clean":"2024-09-10T15:59:14.092473+0000","last_became_active":"2024-09-10T15:59:16.109611+0000","last_became_peered":"2024-09-10T15:59:16.109611+0000","last_unstale":"2024-09-10T16:00:59.508712+0000","last_undegraded":"2024-09-10T16:00:59.508712+0000","last_fullsized":"2024-09-10T15:59:15.770264+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":7780,"log_dups_size":0,"ondisk_log_size":7780,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31850496,"num_objects":7776,"num_object_clones":0,"num_object_copies":15552,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1930,"num_objects_unfound":0,"num_objects_dirty":7776,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5784,"num_write_kb":23136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2339,"num_bytes_recovered":9576448,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1930},{"shards":"2","objects":5846}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'7768","reported_seq":7843,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:00:59.529970+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:00:59.529970+0000","last_peered":"2024-09-10T16:00:59.529970+0000","last_clean":"2024-09-10T16:00:59.529970+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:00:59.529970+0000","last_undegraded":"2024-09-10T16:00:59.529970+0000","last_fullsized":"2024-09-10T16:00:59.529970+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":7768,"log_dups_size":0,"ondisk_log_size":7768,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31817728,"num_objects":7768,"num_object_clones":0,"num_object_copies":15536,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7768,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7768,"num_write_kb":31072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'7820","reported_seq":7945,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:00:56.709172+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:00:56.709172+0000","last_peered":"2024-09-10T16:00:56.709172+0000","last_clean":"2024-09-10T16:00:56.709172+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:00:56.709172+0000","last_undegraded":"2024-09-10T16:00:56.709172+0000","last_fullsized":"2024-09-10T16:00:56.709172+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":7820,"log_dups_size":0,"ondisk_log_size":7820,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32002531,"num_objects":7814,"num_object_clones":0,"num_object_copies":15628,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7814,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4531,"num_write_kb":18124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'7826","reported_seq":7959,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:56.692091+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:00:56.692091+0000","last_peered":"2024-09-10T16:00:56.692091+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:00:56.692091+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":7826,"log_dups_size":0,"ondisk_log_size":7826,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32035299,"num_objects":7822,"num_object_clones":0,"num_object_copies":15644,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7822,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4438,"num_write_kb":17752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3451},{"shards":"0,2","objects":4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'7771","reported_seq":7903,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:56.751621+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:00:56.751621+0000","last_peered":"2024-09-10T16:00:56.751621+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:00:56.751621+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":7771,"log_dups_size":0,"ondisk_log_size":7771,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31806406,"num_objects":7767,"num_object_clones":0,"num_object_copies":15534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7767,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4270,"num_write_kb":17080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3530},{"shards":"0,2","objects":4237}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'7897","reported_seq":8022,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:00:56.646756+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:00:56.646756+0000","last_peered":"2024-09-10T16:00:56.646756+0000","last_clean":"2024-09-10T16:00:56.646756+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:00:56.646756+0000","last_undegraded":"2024-09-10T16:00:56.646756+0000","last_fullsized":"2024-09-10T16:00:56.646756+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":7897,"log_dups_size":0,"ondisk_log_size":7897,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32337920,"num_objects":7895,"num_object_clones":0,"num_object_copies":15790,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7895,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4744,"num_write_kb":18976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.775865+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T15:59:15.775865+0000","last_peered":"2024-09-10T15:59:15.775865+0000","last_clean":"2024-09-10T15:59:15.775865+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T15:59:15.775865+0000","last_undegraded":"2024-09-10T15:59:15.775865+0000","last_fullsized":"2024-09-10T15:59:15.775865+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'7501","reported_seq":7636,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:59.542187+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:00:59.542187+0000","last_peered":"2024-09-10T16:00:59.542187+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:00:59.542187+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":7501,"log_dups_size":0,"ondisk_log_size":7501,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30695907,"num_objects":7495,"num_object_clones":0,"num_object_copies":14990,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7495,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5320,"num_write_kb":21280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4123},{"shards":"2","objects":3372}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":89,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.772619+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T15:59:15.772619+0000","last_peered":"2024-09-10T15:59:15.772619+0000","last_clean":"2024-09-10T15:59:15.772619+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T15:59:15.772619+0000","last_undegraded":"2024-09-10T15:59:15.772619+0000","last_fullsized":"2024-09-10T15:59:15.772619+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'7729","reported_seq":7854,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:56.686812+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:00:56.686812+0000","last_peered":"2024-09-10T16:00:56.686812+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:00:56.686812+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":7729,"log_dups_size":0,"ondisk_log_size":7729,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31625216,"num_objects":7721,"num_object_clones":0,"num_object_copies":15442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7721,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4233,"num_write_kb":16932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3466},{"shards":"0,2","objects":4255}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":99,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.884297+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T15:59:15.884297+0000","last_peered":"2024-09-10T15:59:15.884297+0000","last_clean":"2024-09-10T15:59:15.884297+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T15:59:15.884297+0000","last_undegraded":"2024-09-10T15:59:15.884297+0000","last_fullsized":"2024-09-10T15:59:15.884297+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'7837","reported_seq":7967,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:59.497301+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:00:59.497301+0000","last_peered":"2024-09-10T16:00:59.497301+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:00:59.497301+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":7837,"log_dups_size":0,"ondisk_log_size":7837,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32080355,"num_objects":7833,"num_object_clones":0,"num_object_copies":15666,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7833,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5517,"num_write_kb":22068,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4304},{"shards":"2","objects":3529}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.837509+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T15:59:15.837509+0000","last_peered":"2024-09-10T15:59:15.837509+0000","last_clean":"2024-09-10T15:59:15.837509+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T15:59:15.837509+0000","last_undegraded":"2024-09-10T15:59:15.837509+0000","last_fullsized":"2024-09-10T15:59:15.837509+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'7953","reported_seq":8077,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:59.346753+0000","last_change":"2024-09-10T15:59:16.174113+0000","last_active":"2024-09-10T16:00:59.346753+0000","last_peered":"2024-09-10T16:00:59.346753+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:00:59.346753+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":7953,"log_dups_size":0,"ondisk_log_size":7953,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32550912,"num_objects":7947,"num_object_clones":0,"num_object_copies":15894,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4354,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7947,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5507,"num_write_kb":22028,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4352},{"shards":"2","objects":3595}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'7730","reported_seq":7860,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:56.681530+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:00:56.681530+0000","last_peered":"2024-09-10T16:00:56.681530+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:00:56.681530+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":7730,"log_dups_size":0,"ondisk_log_size":7730,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31653888,"num_objects":7728,"num_object_clones":0,"num_object_copies":15456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7728,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4361,"num_write_kb":17444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3424},{"shards":"0,2","objects":4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.989413+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T15:59:15.989413+0000","last_peered":"2024-09-10T15:59:15.989413+0000","last_clean":"2024-09-10T15:59:15.989413+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T15:59:15.989413+0000","last_undegraded":"2024-09-10T15:59:15.989413+0000","last_fullsized":"2024-09-10T15:59:15.989413+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'7600","reported_seq":7738,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:56.719805+0000","last_change":"2024-09-10T15:59:16.251610+0000","last_active":"2024-09-10T16:00:56.719805+0000","last_peered":"2024-09-10T16:00:56.719805+0000","last_clean":"2024-09-10T15:59:14.115595+0000","last_became_active":"2024-09-10T15:59:16.244089+0000","last_became_peered":"2024-09-10T15:59:16.244089+0000","last_unstale":"2024-09-10T16:00:56.719805+0000","last_undegraded":"2024-09-10T15:59:15.847127+0000","last_fullsized":"2024-09-10T15:59:15.774437+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":7600,"log_dups_size":0,"ondisk_log_size":7600,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31096832,"num_objects":7592,"num_object_clones":0,"num_object_copies":15184,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7592,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3990,"num_write_kb":15960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3389},{"shards":"0,2","objects":4203}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":89,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.773633+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T15:59:15.773633+0000","last_peered":"2024-09-10T15:59:15.773633+0000","last_clean":"2024-09-10T15:59:15.773633+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T15:59:15.773633+0000","last_undegraded":"2024-09-10T15:59:15.773633+0000","last_fullsized":"2024-09-10T15:59:15.773633+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'7878","reported_seq":8015,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:56.739697+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:00:56.739697+0000","last_peered":"2024-09-10T16:00:56.739697+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:00:56.739697+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":7878,"log_dups_size":0,"ondisk_log_size":7878,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32251904,"num_objects":7874,"num_object_clones":0,"num_object_copies":15748,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7874,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4133,"num_write_kb":16532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3503},{"shards":"0,2","objects":4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.916017+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T15:59:15.916017+0000","last_peered":"2024-09-10T15:59:15.916017+0000","last_clean":"2024-09-10T15:59:15.916017+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T15:59:15.916017+0000","last_undegraded":"2024-09-10T15:59:15.916017+0000","last_fullsized":"2024-09-10T15:59:15.916017+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.776033+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T15:59:15.776033+0000","last_peered":"2024-09-10T15:59:15.776033+0000","last_clean":"2024-09-10T15:59:15.776033+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T15:59:15.776033+0000","last_undegraded":"2024-09-10T15:59:15.776033+0000","last_fullsized":"2024-09-10T15:59:15.776033+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'7855","reported_seq":7994,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:56.726018+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:00:56.726018+0000","last_peered":"2024-09-10T16:00:56.726018+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:00:56.726018+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":7855,"log_dups_size":0,"ondisk_log_size":7855,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32125894,"num_objects":7845,"num_object_clones":0,"num_object_copies":15690,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7845,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4018,"num_write_kb":16072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3522},{"shards":"0,2","objects":4323}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":509412062,"num_objects":124377,"num_object_clones":0,"num_object_copies":248754,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":47117,"num_objects_misplaced":1930,"num_objects_unfound":0,"num_objects_dirty":124377,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":82223,"num_write_kb":328892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2342,"num_bytes_recovered":9588736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1030545408,"data_stored":1030393662,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":124449,"ondisk_log_size":124449,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986212,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":621456,"kb_used_data":387072,"kb_used_omap":4,"kb_used_meta":234363,"kb_avail":93750384,"statfs":{"total":96636764160,"available":96000393216,"internally_reserved":0,"allocated":396361728,"data_stored":396083968,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":239988394},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,0,2,8,1,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659158,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":317216,"kb_used_data":177232,"kb_used_omap":4,"kb_used_meta":139963,"kb_avail":94054624,"statfs":{"total":96636764160,"available":96311934976,"internally_reserved":0,"allocated":181485568,"data_stored":181160375,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":143322798},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,0,4,1,4,8],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182440,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":702432,"kb_used_data":440336,"kb_used_omap":4,"kb_used_meta":262075,"kb_avail":93669408,"statfs":{"total":96636764160,"available":95917473792,"internally_reserved":0,"allocated":450904064,"data_stored":450622227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":268365482},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,3,3,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":451428352,"data_stored":451392222,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":182644736,"data_stored":182565250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":396472320,"data_stored":396436190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:01:00.983 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 148 20 125867 125847 3.32104 1.89453 0.0503867 0.0235166 2024-09-10T16:01:01.983 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 149 20 126376 126356 3.3121 1.98828 0.00837648 0.0235794 2024-09-10T16:01:02.983 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 150 20 126893 126873 3.30348 2.01953 0.0378599 0.0236423 2024-09-10T16:01:03.499 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:01:03.984 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 151 20 127368 127348 3.29388 1.85547 0.0216451 0.0237092 2024-09-10T16:01:04.043 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:01:04.044 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:01:04.067 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":150,"stamp":"2024-09-10T16:01:03.579231+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":515498737,"num_objects":125865,"num_object_clones":0,"num_object_copies":251730,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":47117,"num_objects_misplaced":1930,"num_objects_unfound":0,"num_objects_dirty":125865,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":83711,"num_write_kb":334838,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2344,"num_bytes_recovered":9588755,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":125938,"ondisk_log_size":125938,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1652360,"kb_used_data":1011736,"kb_used_omap":12,"kb_used_meta":640563,"kb_avail":281463160,"statfs":{"total":289910292480,"available":288218275840,"internally_reserved":0,"allocated":1036017664,"data_stored":1035132874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":655936514},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,2,7,7,13,14,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":26791936,"num_objects":6541,"num_object_clones":0,"num_object_copies":13082,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-265,"num_objects_unfound":0,"num_objects_dirty":6541,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6541,"num_write_kb":26164,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":265,"num_bytes_recovered":1085440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004955"},"pg_stats":[{"pgid":"2.f","version":"43'7862","reported_seq":7932,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:01.799614+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:01:01.799614+0000","last_peered":"2024-09-10T16:01:01.799614+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:01:01.799614+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":7862,"log_dups_size":0,"ondisk_log_size":7862,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32202752,"num_objects":7862,"num_object_clones":0,"num_object_copies":15724,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7862,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7862,"num_write_kb":31448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3603},{"shards":"0,2","objects":4258}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'7792","reported_seq":7900,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:00:59.444223+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:00:59.444223+0000","last_peered":"2024-09-10T16:00:59.444223+0000","last_clean":"2024-09-10T16:00:59.444223+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:00:59.444223+0000","last_undegraded":"2024-09-10T16:00:59.444223+0000","last_fullsized":"2024-09-10T16:00:59.444223+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":7792,"log_dups_size":0,"ondisk_log_size":7792,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31892422,"num_objects":7788,"num_object_clones":0,"num_object_copies":15576,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7788,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5897,"num_write_kb":23588,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'7780","reported_seq":12584,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T16:00:59.508712+0000","last_change":"2024-09-10T15:59:16.142612+0000","last_active":"2024-09-10T16:00:59.508712+0000","last_peered":"2024-09-10T16:00:59.508712+0000","last_clean":"2024-09-10T15:59:14.092473+0000","last_became_active":"2024-09-10T15:59:16.109611+0000","last_became_peered":"2024-09-10T15:59:16.109611+0000","last_unstale":"2024-09-10T16:00:59.508712+0000","last_undegraded":"2024-09-10T16:00:59.508712+0000","last_fullsized":"2024-09-10T15:59:15.770264+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":7780,"log_dups_size":0,"ondisk_log_size":7780,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31850496,"num_objects":7776,"num_object_clones":0,"num_object_copies":15552,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1930,"num_objects_unfound":0,"num_objects_dirty":7776,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5784,"num_write_kb":23136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2339,"num_bytes_recovered":9576448,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1930},{"shards":"2","objects":5846}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'7768","reported_seq":7843,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:00:59.529970+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:00:59.529970+0000","last_peered":"2024-09-10T16:00:59.529970+0000","last_clean":"2024-09-10T16:00:59.529970+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:00:59.529970+0000","last_undegraded":"2024-09-10T16:00:59.529970+0000","last_fullsized":"2024-09-10T16:00:59.529970+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":7768,"log_dups_size":0,"ondisk_log_size":7768,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31817728,"num_objects":7768,"num_object_clones":0,"num_object_copies":15536,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7768,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7768,"num_write_kb":31072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'7943","reported_seq":8068,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:01:01.582154+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:01:01.582154+0000","last_peered":"2024-09-10T16:01:01.582154+0000","last_clean":"2024-09-10T16:01:01.582154+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:01:01.582154+0000","last_undegraded":"2024-09-10T16:01:01.582154+0000","last_fullsized":"2024-09-10T16:01:01.582154+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":7943,"log_dups_size":0,"ondisk_log_size":7943,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32506339,"num_objects":7937,"num_object_clones":0,"num_object_copies":15874,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7937,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4654,"num_write_kb":18616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'7974","reported_seq":8107,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:01.705200+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:01:01.705200+0000","last_peered":"2024-09-10T16:01:01.705200+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:01:01.705200+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":7974,"log_dups_size":0,"ondisk_log_size":7974,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32641507,"num_objects":7970,"num_object_clones":0,"num_object_copies":15940,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7970,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4586,"num_write_kb":18344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3599},{"shards":"0,2","objects":4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'7947","reported_seq":8080,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:01.776948+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:01:01.776948+0000","last_peered":"2024-09-10T16:01:01.776948+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:01:01.776948+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":7947,"log_dups_size":0,"ondisk_log_size":7947,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32527302,"num_objects":7943,"num_object_clones":0,"num_object_copies":15886,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7943,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4446,"num_write_kb":17784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3706},{"shards":"0,2","objects":4237}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'8045","reported_seq":8169,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:01:01.741436+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:01:01.741436+0000","last_peered":"2024-09-10T16:01:01.741436+0000","last_clean":"2024-09-10T16:01:01.741436+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:01:01.741436+0000","last_undegraded":"2024-09-10T16:01:01.741436+0000","last_fullsized":"2024-09-10T16:01:01.741436+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":8045,"log_dups_size":0,"ondisk_log_size":8045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32944128,"num_objects":8043,"num_object_clones":0,"num_object_copies":16086,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8043,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4892,"num_write_kb":19568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.775865+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T15:59:15.775865+0000","last_peered":"2024-09-10T15:59:15.775865+0000","last_clean":"2024-09-10T15:59:15.775865+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T15:59:15.775865+0000","last_undegraded":"2024-09-10T15:59:15.775865+0000","last_fullsized":"2024-09-10T15:59:15.775865+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'7501","reported_seq":7636,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:59.542187+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:00:59.542187+0000","last_peered":"2024-09-10T16:00:59.542187+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:00:59.542187+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":7501,"log_dups_size":0,"ondisk_log_size":7501,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30695907,"num_objects":7495,"num_object_clones":0,"num_object_copies":14990,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7495,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5320,"num_write_kb":21280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4123},{"shards":"2","objects":3372}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":89,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.772619+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T15:59:15.772619+0000","last_peered":"2024-09-10T15:59:15.772619+0000","last_clean":"2024-09-10T15:59:15.772619+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T15:59:15.772619+0000","last_undegraded":"2024-09-10T15:59:15.772619+0000","last_fullsized":"2024-09-10T15:59:15.772619+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'7882","reported_seq":8007,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:01.639997+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:01:01.639997+0000","last_peered":"2024-09-10T16:01:01.639997+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:01:01.639997+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":7882,"log_dups_size":0,"ondisk_log_size":7882,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32251904,"num_objects":7874,"num_object_clones":0,"num_object_copies":15748,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7874,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4386,"num_write_kb":17544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3619},{"shards":"0,2","objects":4255}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":99,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.884297+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T15:59:15.884297+0000","last_peered":"2024-09-10T15:59:15.884297+0000","last_clean":"2024-09-10T15:59:15.884297+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T15:59:15.884297+0000","last_undegraded":"2024-09-10T15:59:15.884297+0000","last_fullsized":"2024-09-10T15:59:15.884297+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'7837","reported_seq":7967,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:59.497301+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:00:59.497301+0000","last_peered":"2024-09-10T16:00:59.497301+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:00:59.497301+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":7837,"log_dups_size":0,"ondisk_log_size":7837,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32080355,"num_objects":7833,"num_object_clones":0,"num_object_copies":15666,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7833,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5517,"num_write_kb":22068,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4304},{"shards":"2","objects":3529}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.837509+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T15:59:15.837509+0000","last_peered":"2024-09-10T15:59:15.837509+0000","last_clean":"2024-09-10T15:59:15.837509+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T15:59:15.837509+0000","last_undegraded":"2024-09-10T15:59:15.837509+0000","last_fullsized":"2024-09-10T15:59:15.837509+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'7953","reported_seq":8077,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:59.346753+0000","last_change":"2024-09-10T15:59:16.174113+0000","last_active":"2024-09-10T16:00:59.346753+0000","last_peered":"2024-09-10T16:00:59.346753+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:00:59.346753+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":7953,"log_dups_size":0,"ondisk_log_size":7953,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32550912,"num_objects":7947,"num_object_clones":0,"num_object_copies":15894,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4354,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7947,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5507,"num_write_kb":22028,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4352},{"shards":"2","objects":3595}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'7881","reported_seq":8011,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:01.792527+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:01:01.792527+0000","last_peered":"2024-09-10T16:01:01.792527+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:01:01.792527+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":7881,"log_dups_size":0,"ondisk_log_size":7881,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32272384,"num_objects":7879,"num_object_clones":0,"num_object_copies":15758,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7879,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4512,"num_write_kb":18048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3575},{"shards":"0,2","objects":4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.989413+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T15:59:15.989413+0000","last_peered":"2024-09-10T15:59:15.989413+0000","last_clean":"2024-09-10T15:59:15.989413+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T15:59:15.989413+0000","last_undegraded":"2024-09-10T15:59:15.989413+0000","last_fullsized":"2024-09-10T15:59:15.989413+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'7755","reported_seq":7893,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:01.778576+0000","last_change":"2024-09-10T15:59:16.251610+0000","last_active":"2024-09-10T16:01:01.778576+0000","last_peered":"2024-09-10T16:01:01.778576+0000","last_clean":"2024-09-10T15:59:14.115595+0000","last_became_active":"2024-09-10T15:59:16.244089+0000","last_became_peered":"2024-09-10T15:59:16.244089+0000","last_unstale":"2024-09-10T16:01:01.778576+0000","last_undegraded":"2024-09-10T15:59:15.847127+0000","last_fullsized":"2024-09-10T15:59:15.774437+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":7755,"log_dups_size":0,"ondisk_log_size":7755,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31731712,"num_objects":7747,"num_object_clones":0,"num_object_copies":15494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7747,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4145,"num_write_kb":16580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3544},{"shards":"0,2","objects":4203}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":89,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.773633+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T15:59:15.773633+0000","last_peered":"2024-09-10T15:59:15.773633+0000","last_clean":"2024-09-10T15:59:15.773633+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T15:59:15.773633+0000","last_undegraded":"2024-09-10T15:59:15.773633+0000","last_fullsized":"2024-09-10T15:59:15.773633+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'8020","reported_seq":8157,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:01.786350+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:01:01.786350+0000","last_peered":"2024-09-10T16:01:01.786350+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:01:01.786350+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":8020,"log_dups_size":0,"ondisk_log_size":8020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32833536,"num_objects":8016,"num_object_clones":0,"num_object_copies":16032,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8016,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4275,"num_write_kb":17100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3645},{"shards":"0,2","objects":4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.916017+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T15:59:15.916017+0000","last_peered":"2024-09-10T15:59:15.916017+0000","last_clean":"2024-09-10T15:59:15.916017+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T15:59:15.916017+0000","last_undegraded":"2024-09-10T15:59:15.916017+0000","last_fullsized":"2024-09-10T15:59:15.916017+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.776033+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T15:59:15.776033+0000","last_peered":"2024-09-10T15:59:15.776033+0000","last_clean":"2024-09-10T15:59:15.776033+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T15:59:15.776033+0000","last_undegraded":"2024-09-10T15:59:15.776033+0000","last_fullsized":"2024-09-10T15:59:15.776033+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'7995","reported_seq":8134,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:01.716122+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:01:01.716122+0000","last_peered":"2024-09-10T16:01:01.716122+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:01:01.716122+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":7995,"log_dups_size":0,"ondisk_log_size":7995,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32699334,"num_objects":7985,"num_object_clones":0,"num_object_copies":15970,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7985,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4158,"num_write_kb":16632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3662},{"shards":"0,2","objects":4323}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":515498718,"num_objects":125863,"num_object_clones":0,"num_object_copies":251726,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":47117,"num_objects_misplaced":1930,"num_objects_unfound":0,"num_objects_dirty":125863,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":83709,"num_write_kb":334836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2342,"num_bytes_recovered":9588736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1037930496,"data_stored":1037778750,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":125935,"ondisk_log_size":125935,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986212,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":621456,"kb_used_data":387072,"kb_used_omap":4,"kb_used_meta":234363,"kb_avail":93750384,"statfs":{"total":96636764160,"available":96000393216,"internally_reserved":0,"allocated":396361728,"data_stored":396083968,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":239988394},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,0,2,8,1,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659158,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":317216,"kb_used_data":177232,"kb_used_omap":4,"kb_used_meta":139963,"kb_avail":94054624,"statfs":{"total":96636764160,"available":96311934976,"internally_reserved":0,"allocated":181485568,"data_stored":181160375,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":143322798},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,0,4,1,4,8],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182441,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":713688,"kb_used_data":447432,"kb_used_omap":4,"kb_used_meta":266235,"kb_avail":93658152,"statfs":{"total":96636764160,"available":95905947648,"internally_reserved":0,"allocated":458170368,"data_stored":457888531,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":272625322},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,3,4,1,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":458813440,"data_stored":458777310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":182644736,"data_stored":182565250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":396472320,"data_stored":396436190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:01:04.069 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:01:04.614 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:01:04.616 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:01:04.635 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":150,"stamp":"2024-09-10T16:01:03.579231+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":515498737,"num_objects":125865,"num_object_clones":0,"num_object_copies":251730,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":47117,"num_objects_misplaced":1930,"num_objects_unfound":0,"num_objects_dirty":125865,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":83711,"num_write_kb":334838,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2344,"num_bytes_recovered":9588755,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":125938,"ondisk_log_size":125938,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1652360,"kb_used_data":1011736,"kb_used_omap":12,"kb_used_meta":640563,"kb_avail":281463160,"statfs":{"total":289910292480,"available":288218275840,"internally_reserved":0,"allocated":1036017664,"data_stored":1035132874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":655936514},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,2,7,7,13,14,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":26791936,"num_objects":6541,"num_object_clones":0,"num_object_copies":13082,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-265,"num_objects_unfound":0,"num_objects_dirty":6541,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6541,"num_write_kb":26164,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":265,"num_bytes_recovered":1085440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004955"},"pg_stats":[{"pgid":"2.f","version":"43'7862","reported_seq":7932,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:01.799614+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:01:01.799614+0000","last_peered":"2024-09-10T16:01:01.799614+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:01:01.799614+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":7862,"log_dups_size":0,"ondisk_log_size":7862,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32202752,"num_objects":7862,"num_object_clones":0,"num_object_copies":15724,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7862,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7862,"num_write_kb":31448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3603},{"shards":"0,2","objects":4258}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'7792","reported_seq":7900,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:00:59.444223+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:00:59.444223+0000","last_peered":"2024-09-10T16:00:59.444223+0000","last_clean":"2024-09-10T16:00:59.444223+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:00:59.444223+0000","last_undegraded":"2024-09-10T16:00:59.444223+0000","last_fullsized":"2024-09-10T16:00:59.444223+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":7792,"log_dups_size":0,"ondisk_log_size":7792,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31892422,"num_objects":7788,"num_object_clones":0,"num_object_copies":15576,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7788,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5897,"num_write_kb":23588,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'7780","reported_seq":12584,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T16:00:59.508712+0000","last_change":"2024-09-10T15:59:16.142612+0000","last_active":"2024-09-10T16:00:59.508712+0000","last_peered":"2024-09-10T16:00:59.508712+0000","last_clean":"2024-09-10T15:59:14.092473+0000","last_became_active":"2024-09-10T15:59:16.109611+0000","last_became_peered":"2024-09-10T15:59:16.109611+0000","last_unstale":"2024-09-10T16:00:59.508712+0000","last_undegraded":"2024-09-10T16:00:59.508712+0000","last_fullsized":"2024-09-10T15:59:15.770264+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":7780,"log_dups_size":0,"ondisk_log_size":7780,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31850496,"num_objects":7776,"num_object_clones":0,"num_object_copies":15552,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1930,"num_objects_unfound":0,"num_objects_dirty":7776,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5784,"num_write_kb":23136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2339,"num_bytes_recovered":9576448,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1930},{"shards":"2","objects":5846}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'7768","reported_seq":7843,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:00:59.529970+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:00:59.529970+0000","last_peered":"2024-09-10T16:00:59.529970+0000","last_clean":"2024-09-10T16:00:59.529970+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:00:59.529970+0000","last_undegraded":"2024-09-10T16:00:59.529970+0000","last_fullsized":"2024-09-10T16:00:59.529970+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":7768,"log_dups_size":0,"ondisk_log_size":7768,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31817728,"num_objects":7768,"num_object_clones":0,"num_object_copies":15536,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7768,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7768,"num_write_kb":31072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'7943","reported_seq":8068,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:01:01.582154+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:01:01.582154+0000","last_peered":"2024-09-10T16:01:01.582154+0000","last_clean":"2024-09-10T16:01:01.582154+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:01:01.582154+0000","last_undegraded":"2024-09-10T16:01:01.582154+0000","last_fullsized":"2024-09-10T16:01:01.582154+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":7943,"log_dups_size":0,"ondisk_log_size":7943,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32506339,"num_objects":7937,"num_object_clones":0,"num_object_copies":15874,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7937,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4654,"num_write_kb":18616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'7974","reported_seq":8107,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:01.705200+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:01:01.705200+0000","last_peered":"2024-09-10T16:01:01.705200+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:01:01.705200+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":7974,"log_dups_size":0,"ondisk_log_size":7974,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32641507,"num_objects":7970,"num_object_clones":0,"num_object_copies":15940,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7970,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4586,"num_write_kb":18344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3599},{"shards":"0,2","objects":4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'7947","reported_seq":8080,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:01.776948+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:01:01.776948+0000","last_peered":"2024-09-10T16:01:01.776948+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:01:01.776948+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":7947,"log_dups_size":0,"ondisk_log_size":7947,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32527302,"num_objects":7943,"num_object_clones":0,"num_object_copies":15886,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7943,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4446,"num_write_kb":17784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3706},{"shards":"0,2","objects":4237}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'8045","reported_seq":8169,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:01:01.741436+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:01:01.741436+0000","last_peered":"2024-09-10T16:01:01.741436+0000","last_clean":"2024-09-10T16:01:01.741436+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:01:01.741436+0000","last_undegraded":"2024-09-10T16:01:01.741436+0000","last_fullsized":"2024-09-10T16:01:01.741436+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":8045,"log_dups_size":0,"ondisk_log_size":8045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32944128,"num_objects":8043,"num_object_clones":0,"num_object_copies":16086,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8043,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4892,"num_write_kb":19568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.775865+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T15:59:15.775865+0000","last_peered":"2024-09-10T15:59:15.775865+0000","last_clean":"2024-09-10T15:59:15.775865+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T15:59:15.775865+0000","last_undegraded":"2024-09-10T15:59:15.775865+0000","last_fullsized":"2024-09-10T15:59:15.775865+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'7501","reported_seq":7636,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:59.542187+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:00:59.542187+0000","last_peered":"2024-09-10T16:00:59.542187+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:00:59.542187+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":7501,"log_dups_size":0,"ondisk_log_size":7501,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30695907,"num_objects":7495,"num_object_clones":0,"num_object_copies":14990,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7495,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5320,"num_write_kb":21280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4123},{"shards":"2","objects":3372}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":89,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.772619+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T15:59:15.772619+0000","last_peered":"2024-09-10T15:59:15.772619+0000","last_clean":"2024-09-10T15:59:15.772619+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T15:59:15.772619+0000","last_undegraded":"2024-09-10T15:59:15.772619+0000","last_fullsized":"2024-09-10T15:59:15.772619+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'7882","reported_seq":8007,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:01.639997+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:01:01.639997+0000","last_peered":"2024-09-10T16:01:01.639997+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:01:01.639997+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":7882,"log_dups_size":0,"ondisk_log_size":7882,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32251904,"num_objects":7874,"num_object_clones":0,"num_object_copies":15748,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7874,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4386,"num_write_kb":17544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3619},{"shards":"0,2","objects":4255}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":99,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.884297+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T15:59:15.884297+0000","last_peered":"2024-09-10T15:59:15.884297+0000","last_clean":"2024-09-10T15:59:15.884297+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T15:59:15.884297+0000","last_undegraded":"2024-09-10T15:59:15.884297+0000","last_fullsized":"2024-09-10T15:59:15.884297+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'7837","reported_seq":7967,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:59.497301+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:00:59.497301+0000","last_peered":"2024-09-10T16:00:59.497301+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:00:59.497301+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":7837,"log_dups_size":0,"ondisk_log_size":7837,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32080355,"num_objects":7833,"num_object_clones":0,"num_object_copies":15666,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7833,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5517,"num_write_kb":22068,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4304},{"shards":"2","objects":3529}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.837509+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T15:59:15.837509+0000","last_peered":"2024-09-10T15:59:15.837509+0000","last_clean":"2024-09-10T15:59:15.837509+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T15:59:15.837509+0000","last_undegraded":"2024-09-10T15:59:15.837509+0000","last_fullsized":"2024-09-10T15:59:15.837509+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'7953","reported_seq":8077,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:59.346753+0000","last_change":"2024-09-10T15:59:16.174113+0000","last_active":"2024-09-10T16:00:59.346753+0000","last_peered":"2024-09-10T16:00:59.346753+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:00:59.346753+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":7953,"log_dups_size":0,"ondisk_log_size":7953,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32550912,"num_objects":7947,"num_object_clones":0,"num_object_copies":15894,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4354,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7947,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5507,"num_write_kb":22028,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4352},{"shards":"2","objects":3595}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'7881","reported_seq":8011,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:01.792527+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:01:01.792527+0000","last_peered":"2024-09-10T16:01:01.792527+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:01:01.792527+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":7881,"log_dups_size":0,"ondisk_log_size":7881,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32272384,"num_objects":7879,"num_object_clones":0,"num_object_copies":15758,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7879,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4512,"num_write_kb":18048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3575},{"shards":"0,2","objects":4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.989413+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T15:59:15.989413+0000","last_peered":"2024-09-10T15:59:15.989413+0000","last_clean":"2024-09-10T15:59:15.989413+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T15:59:15.989413+0000","last_undegraded":"2024-09-10T15:59:15.989413+0000","last_fullsized":"2024-09-10T15:59:15.989413+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'7755","reported_seq":7893,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:01.778576+0000","last_change":"2024-09-10T15:59:16.251610+0000","last_active":"2024-09-10T16:01:01.778576+0000","last_peered":"2024-09-10T16:01:01.778576+0000","last_clean":"2024-09-10T15:59:14.115595+0000","last_became_active":"2024-09-10T15:59:16.244089+0000","last_became_peered":"2024-09-10T15:59:16.244089+0000","last_unstale":"2024-09-10T16:01:01.778576+0000","last_undegraded":"2024-09-10T15:59:15.847127+0000","last_fullsized":"2024-09-10T15:59:15.774437+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":7755,"log_dups_size":0,"ondisk_log_size":7755,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31731712,"num_objects":7747,"num_object_clones":0,"num_object_copies":15494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7747,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4145,"num_write_kb":16580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3544},{"shards":"0,2","objects":4203}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":89,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.773633+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T15:59:15.773633+0000","last_peered":"2024-09-10T15:59:15.773633+0000","last_clean":"2024-09-10T15:59:15.773633+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T15:59:15.773633+0000","last_undegraded":"2024-09-10T15:59:15.773633+0000","last_fullsized":"2024-09-10T15:59:15.773633+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'8020","reported_seq":8157,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:01.786350+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:01:01.786350+0000","last_peered":"2024-09-10T16:01:01.786350+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:01:01.786350+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":8020,"log_dups_size":0,"ondisk_log_size":8020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32833536,"num_objects":8016,"num_object_clones":0,"num_object_copies":16032,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8016,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4275,"num_write_kb":17100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3645},{"shards":"0,2","objects":4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.916017+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T15:59:15.916017+0000","last_peered":"2024-09-10T15:59:15.916017+0000","last_clean":"2024-09-10T15:59:15.916017+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T15:59:15.916017+0000","last_undegraded":"2024-09-10T15:59:15.916017+0000","last_fullsized":"2024-09-10T15:59:15.916017+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.776033+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T15:59:15.776033+0000","last_peered":"2024-09-10T15:59:15.776033+0000","last_clean":"2024-09-10T15:59:15.776033+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T15:59:15.776033+0000","last_undegraded":"2024-09-10T15:59:15.776033+0000","last_fullsized":"2024-09-10T15:59:15.776033+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'7995","reported_seq":8134,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:01.716122+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:01:01.716122+0000","last_peered":"2024-09-10T16:01:01.716122+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:01:01.716122+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":7995,"log_dups_size":0,"ondisk_log_size":7995,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32699334,"num_objects":7985,"num_object_clones":0,"num_object_copies":15970,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7985,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4158,"num_write_kb":16632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3662},{"shards":"0,2","objects":4323}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":515498718,"num_objects":125863,"num_object_clones":0,"num_object_copies":251726,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":47117,"num_objects_misplaced":1930,"num_objects_unfound":0,"num_objects_dirty":125863,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":83709,"num_write_kb":334836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2342,"num_bytes_recovered":9588736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1037930496,"data_stored":1037778750,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":125935,"ondisk_log_size":125935,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986212,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":621456,"kb_used_data":387072,"kb_used_omap":4,"kb_used_meta":234363,"kb_avail":93750384,"statfs":{"total":96636764160,"available":96000393216,"internally_reserved":0,"allocated":396361728,"data_stored":396083968,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":239988394},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,0,2,8,1,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659158,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":317216,"kb_used_data":177232,"kb_used_omap":4,"kb_used_meta":139963,"kb_avail":94054624,"statfs":{"total":96636764160,"available":96311934976,"internally_reserved":0,"allocated":181485568,"data_stored":181160375,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":143322798},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,0,4,1,4,8],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182441,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":713688,"kb_used_data":447432,"kb_used_omap":4,"kb_used_meta":266235,"kb_avail":93658152,"statfs":{"total":96636764160,"available":95905947648,"internally_reserved":0,"allocated":458170368,"data_stored":457888531,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":272625322},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,3,4,1,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":458813440,"data_stored":458777310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":182644736,"data_stored":182565250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":396472320,"data_stored":396436190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:01:04.637 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-10T16:01:04.637 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "43'7862", 'reported_seq': 7932, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:01:01.799614+0000', 'last_change': '2024-09-10T15:59:16.281558+0000', 'last_active': '2024-09-10T16:01:01.799614+0000', 'last_peered': '2024-09-10T16:01:01.799614+0000', 'last_clean': '2024-09-10T15:59:14.132456+0000', 'last_became_active': '2024-09-10T15:59:16.274407+0000', 'last_became_peered': '2024-09-10T15:59:16.274407+0000', 'last_unstale': '2024-09-10T16:01:01.799614+0000', 'last_undegraded': '2024-09-10T15:59:15.851279+0000', 'last_fullsized': '2024-09-10T15:59:15.774356+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'objects_scrubbed': 0, 'log_size': 7862, 'log_dups_size': 0, 'ondisk_log_size': 7862, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32202752, 'num_objects': 7862, 'num_object_clones': 0, 'num_object_copies': 15724, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4259, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7862, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7862, 'num_write_kb': 31448, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3603}, {'shards': '0,2', 'objects': 4258}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:01:04.637 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-10T16:01:04.637 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "43'7780", 'reported_seq': 12584, 'reported_epoch': 43, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-10T16:00:59.508712+0000', 'last_change': '2024-09-10T15:59:16.142612+0000', 'last_active': '2024-09-10T16:00:59.508712+0000', 'last_peered': '2024-09-10T16:00:59.508712+0000', 'last_clean': '2024-09-10T15:59:14.092473+0000', 'last_became_active': '2024-09-10T15:59:16.109611+0000', 'last_became_peered': '2024-09-10T15:59:16.109611+0000', 'last_unstale': '2024-09-10T16:00:59.508712+0000', 'last_undegraded': '2024-09-10T16:00:59.508712+0000', 'last_fullsized': '2024-09-10T15:59:15.770264+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2029", 'last_scrub_stamp': '2024-09-10T15:58:45.933789+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:45.933789+0000', 'objects_scrubbed': 1992, 'log_size': 7780, 'log_dups_size': 0, 'ondisk_log_size': 7780, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 668, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31850496, 'num_objects': 7776, 'num_object_clones': 0, 'num_object_copies': 15552, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1930, 'num_objects_unfound': 0, 'num_objects_dirty': 7776, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5784, 'num_write_kb': 23136, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2339, 'num_bytes_recovered': 9576448, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1930}, {'shards': '2', 'objects': 5846}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:01:04.637 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-10T16:01:04.637 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "43'7974", 'reported_seq': 8107, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:01:01.705200+0000', 'last_change': '2024-09-10T15:59:16.308264+0000', 'last_active': '2024-09-10T16:01:01.705200+0000', 'last_peered': '2024-09-10T16:01:01.705200+0000', 'last_clean': '2024-09-10T15:59:14.111551+0000', 'last_became_active': '2024-09-10T15:59:16.300040+0000', 'last_became_peered': '2024-09-10T15:59:16.300040+0000', 'last_unstale': '2024-09-10T16:01:01.705200+0000', 'last_undegraded': '2024-09-10T15:59:15.878234+0000', 'last_fullsized': '2024-09-10T15:59:15.769328+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3442", 'last_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'objects_scrubbed': 3384, 'log_size': 7974, 'log_dups_size': 0, 'ondisk_log_size': 7974, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1292, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32641507, 'num_objects': 7970, 'num_object_clones': 0, 'num_object_copies': 15940, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4372, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7970, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 4586, 'num_write_kb': 18344, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3599}, {'shards': '0,2', 'objects': 4371}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:01:04.637 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-10T16:01:04.637 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "43'7947", 'reported_seq': 8080, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:01:01.776948+0000', 'last_change': '2024-09-10T15:59:16.331899+0000', 'last_active': '2024-09-10T16:01:01.776948+0000', 'last_peered': '2024-09-10T16:01:01.776948+0000', 'last_clean': '2024-09-10T15:59:14.139436+0000', 'last_became_active': '2024-09-10T15:59:16.318715+0000', 'last_became_peered': '2024-09-10T15:59:16.318715+0000', 'last_unstale': '2024-09-10T16:01:01.776948+0000', 'last_undegraded': '2024-09-10T15:59:15.877524+0000', 'last_fullsized': '2024-09-10T15:59:15.769042+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3537", 'last_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'objects_scrubbed': 3497, 'log_size': 7947, 'log_dups_size': 0, 'ondisk_log_size': 7947, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1224, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32527302, 'num_objects': 7943, 'num_object_clones': 0, 'num_object_copies': 15886, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4237, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7943, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 4446, 'num_write_kb': 17784, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3706}, {'shards': '0,2', 'objects': 4237}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:01:04.637 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-10T16:01:04.637 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "43'7501", 'reported_seq': 7636, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:00:59.542187+0000', 'last_change': '2024-09-10T15:59:16.175680+0000', 'last_active': '2024-09-10T16:00:59.542187+0000', 'last_peered': '2024-09-10T16:00:59.542187+0000', 'last_clean': '2024-09-10T15:59:14.088187+0000', 'last_became_active': '2024-09-10T15:59:16.165810+0000', 'last_became_peered': '2024-09-10T15:59:16.165810+0000', 'last_unstale': '2024-09-10T16:00:59.542187+0000', 'last_undegraded': '2024-09-10T15:59:15.822098+0000', 'last_fullsized': '2024-09-10T15:59:15.769123+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2218", 'last_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'objects_scrubbed': 2175, 'log_size': 7501, 'log_dups_size': 0, 'ondisk_log_size': 7501, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 825, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30695907, 'num_objects': 7495, 'num_object_clones': 0, 'num_object_copies': 14990, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4124, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7495, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 5320, 'num_write_kb': 21280, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4123}, {'shards': '2', 'objects': 3372}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:01:04.638 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-10T16:01:04.638 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "43'7882", 'reported_seq': 8007, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:01:01.639997+0000', 'last_change': '2024-09-10T15:59:16.277742+0000', 'last_active': '2024-09-10T16:01:01.639997+0000', 'last_peered': '2024-09-10T16:01:01.639997+0000', 'last_clean': '2024-09-10T15:59:14.129081+0000', 'last_became_active': '2024-09-10T15:59:16.269835+0000', 'last_became_peered': '2024-09-10T15:59:16.269835+0000', 'last_unstale': '2024-09-10T16:01:01.639997+0000', 'last_undegraded': '2024-09-10T15:59:15.860857+0000', 'last_fullsized': '2024-09-10T15:59:15.771410+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3557", 'last_scrub_stamp': '2024-09-10T15:59:02.493678+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:02.493678+0000', 'objects_scrubbed': 3488, 'log_size': 7882, 'log_dups_size': 0, 'ondisk_log_size': 7882, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1472, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32251904, 'num_objects': 7874, 'num_object_clones': 0, 'num_object_copies': 15748, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4257, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7874, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4386, 'num_write_kb': 17544, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3619}, {'shards': '0,2', 'objects': 4255}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:01:04.638 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-10T16:01:04.638 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "43'7837", 'reported_seq': 7967, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:00:59.497301+0000', 'last_change': '2024-09-10T15:59:16.284311+0000', 'last_active': '2024-09-10T16:00:59.497301+0000', 'last_peered': '2024-09-10T16:00:59.497301+0000', 'last_clean': '2024-09-10T15:59:14.116041+0000', 'last_became_active': '2024-09-10T15:59:16.274543+0000', 'last_became_peered': '2024-09-10T15:59:16.274543+0000', 'last_unstale': '2024-09-10T16:00:59.497301+0000', 'last_undegraded': '2024-09-10T15:59:15.812484+0000', 'last_fullsized': '2024-09-10T15:59:15.770885+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2356", 'last_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'objects_scrubbed': 2316, 'log_size': 7837, 'log_dups_size': 0, 'ondisk_log_size': 7837, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 783, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32080355, 'num_objects': 7833, 'num_object_clones': 0, 'num_object_copies': 15666, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7833, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 5517, 'num_write_kb': 22068, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4304}, {'shards': '2', 'objects': 3529}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:01:04.638 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-10T16:01:04.638 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "43'7953", 'reported_seq': 8077, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:00:59.346753+0000', 'last_change': '2024-09-10T15:59:16.174113+0000', 'last_active': '2024-09-10T16:00:59.346753+0000', 'last_peered': '2024-09-10T16:00:59.346753+0000', 'last_clean': '2024-09-10T15:59:14.110654+0000', 'last_became_active': '2024-09-10T15:59:16.164645+0000', 'last_became_peered': '2024-09-10T15:59:16.164645+0000', 'last_unstale': '2024-09-10T16:00:59.346753+0000', 'last_undegraded': '2024-09-10T15:59:15.803593+0000', 'last_fullsized': '2024-09-10T15:59:15.769083+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2486", 'last_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'objects_scrubbed': 2440, 'log_size': 7953, 'log_dups_size': 0, 'ondisk_log_size': 7953, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 792, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32550912, 'num_objects': 7947, 'num_object_clones': 0, 'num_object_copies': 15894, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4354, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7947, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5507, 'num_write_kb': 22028, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4352}, {'shards': '2', 'objects': 3595}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:01:04.638 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-10T16:01:04.638 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "43'7881", 'reported_seq': 8011, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:01:01.792527+0000', 'last_change': '2024-09-10T15:59:16.315641+0000', 'last_active': '2024-09-10T16:01:01.792527+0000', 'last_peered': '2024-09-10T16:01:01.792527+0000', 'last_clean': '2024-09-10T15:59:14.117001+0000', 'last_became_active': '2024-09-10T15:59:16.302245+0000', 'last_became_peered': '2024-09-10T15:59:16.302245+0000', 'last_unstale': '2024-09-10T16:01:01.792527+0000', 'last_undegraded': '2024-09-10T15:59:15.845012+0000', 'last_fullsized': '2024-09-10T15:59:15.770337+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3404", 'last_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'objects_scrubbed': 3367, 'log_size': 7881, 'log_dups_size': 0, 'ondisk_log_size': 7881, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 983, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32272384, 'num_objects': 7879, 'num_object_clones': 0, 'num_object_copies': 15758, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7879, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4512, 'num_write_kb': 18048, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3575}, {'shards': '0,2', 'objects': 4304}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:01:04.638 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-10T16:01:04.638 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "43'7755", 'reported_seq': 7893, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:01:01.778576+0000', 'last_change': '2024-09-10T15:59:16.251610+0000', 'last_active': '2024-09-10T16:01:01.778576+0000', 'last_peered': '2024-09-10T16:01:01.778576+0000', 'last_clean': '2024-09-10T15:59:14.115595+0000', 'last_became_active': '2024-09-10T15:59:16.244089+0000', 'last_became_peered': '2024-09-10T15:59:16.244089+0000', 'last_unstale': '2024-09-10T16:01:01.778576+0000', 'last_undegraded': '2024-09-10T15:59:15.847127+0000', 'last_fullsized': '2024-09-10T15:59:15.774437+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3653", 'last_scrub_stamp': '2024-09-10T15:59:05.298468+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:05.298468+0000', 'objects_scrubbed': 3602, 'log_size': 7755, 'log_dups_size': 0, 'ondisk_log_size': 7755, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1289, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31731712, 'num_objects': 7747, 'num_object_clones': 0, 'num_object_copies': 15494, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4206, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7747, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4145, 'num_write_kb': 16580, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3544}, {'shards': '0,2', 'objects': 4203}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:01:04.639 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-10T16:01:04.639 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "43'8020", 'reported_seq': 8157, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:01:01.786350+0000', 'last_change': '2024-09-10T15:59:16.311291+0000', 'last_active': '2024-09-10T16:01:01.786350+0000', 'last_peered': '2024-09-10T16:01:01.786350+0000', 'last_clean': '2024-09-10T15:59:14.116363+0000', 'last_became_active': '2024-09-10T15:59:16.301244+0000', 'last_became_peered': '2024-09-10T15:59:16.301244+0000', 'last_unstale': '2024-09-10T16:01:01.786350+0000', 'last_undegraded': '2024-09-10T15:59:15.883434+0000', 'last_fullsized': '2024-09-10T15:59:15.772756+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3789", 'last_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'objects_scrubbed': 3741, 'log_size': 8020, 'log_dups_size': 0, 'ondisk_log_size': 8020, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1440, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32833536, 'num_objects': 8016, 'num_object_clones': 0, 'num_object_copies': 16032, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4373, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8016, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4275, 'num_write_kb': 17100, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3645}, {'shards': '0,2', 'objects': 4371}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:01:04.639 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-10T16:01:04.639 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "43'7995", 'reported_seq': 8134, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:01:01.716122+0000', 'last_change': '2024-09-10T15:59:16.313409+0000', 'last_active': '2024-09-10T16:01:01.716122+0000', 'last_peered': '2024-09-10T16:01:01.716122+0000', 'last_clean': '2024-09-10T15:59:14.129780+0000', 'last_became_active': '2024-09-10T15:59:16.300932+0000', 'last_became_peered': '2024-09-10T15:59:16.300932+0000', 'last_unstale': '2024-09-10T16:01:01.716122+0000', 'last_undegraded': '2024-09-10T15:59:15.887823+0000', 'last_fullsized': '2024-09-10T15:59:15.768513+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3880", 'last_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'objects_scrubbed': 3827, 'log_size': 7995, 'log_dups_size': 0, 'ondisk_log_size': 7995, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1397, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32699334, 'num_objects': 7985, 'num_object_clones': 0, 'num_object_copies': 15970, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4325, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7985, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4158, 'num_write_kb': 16632, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3662}, {'shards': '0,2', 'objects': 4323}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:01:04.639 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T16:01:04.984 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 152 20 127782 127762 3.28285 1.61719 0.0365457 0.0237902 2024-09-10T16:01:05.263 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:01:05.263 INFO:teuthology.orchestra.run.smithi115.stdout:{"fsid":"7effee29-df05-4351-9741-ee203564ed81","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 47117/251730 objects degraded (18.717%), 11 pgs degraded, 12 pgs undersized","count":23},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":229,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":43,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983954,"num_in_osds":3,"osd_in_since":1725983947,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":125865,"data_bytes":515498737,"bytes_used":1692016640,"bytes_avail":288218275840,"bytes_total":289910292480,"degraded_objects":47117,"degraded_total":251730,"degraded_ratio":0.18717276446986852,"misplaced_objects":1930,"misplaced_total":251730,"misplaced_ratio":0.0076669447423827119,"recovering_objects_per_sec":22,"recovering_bytes_per_sec":90415,"recovering_keys_per_sec":0,"num_objects_recovered":265,"num_bytes_recovered":1085440,"num_keys_recovered":0,"write_bytes_sec":2231739,"read_op_per_sec":0,"write_op_per_sec":544},"fsmap":{"epoch":1,"btime":"2024-09-10T15:57:14:967120+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T16:00:13.560776+0000","services":{}},"progress_events":{}} 2024-09-10T16:01:05.287 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-10T16:01:05.287 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:01:05.835 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:01:05.838 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:01:05.863 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":151,"stamp":"2024-09-10T16:01:05.579887+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":519226097,"num_objects":126775,"num_object_clones":0,"num_object_copies":253550,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":47117,"num_objects_misplaced":1797,"num_objects_unfound":0,"num_objects_dirty":126775,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":84621,"num_write_kb":338478,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2477,"num_bytes_recovered":10133523,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":126848,"ondisk_log_size":126848,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1695868,"kb_used_data":1024716,"kb_used_omap":12,"kb_used_meta":671091,"kb_avail":281419652,"statfs":{"total":289910292480,"available":288173723648,"internally_reserved":0,"allocated":1049309184,"data_stored":1048424394,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":687197186},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,4,5,8,4,11,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":23891968,"num_objects":5833,"num_object_clones":0,"num_object_copies":11666,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-398,"num_objects_unfound":0,"num_objects_dirty":5833,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5833,"num_write_kb":23332,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":398,"num_bytes_recovered":1630208,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004766"},"pg_stats":[{"pgid":"2.f","version":"43'7862","reported_seq":7932,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:01.799614+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:01:01.799614+0000","last_peered":"2024-09-10T16:01:01.799614+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:01:01.799614+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":7862,"log_dups_size":0,"ondisk_log_size":7862,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32202752,"num_objects":7862,"num_object_clones":0,"num_object_copies":15724,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7862,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7862,"num_write_kb":31448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3603},{"shards":"0,2","objects":4258}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'7949","reported_seq":8056,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:01:04.467097+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:01:04.467097+0000","last_peered":"2024-09-10T16:01:04.467097+0000","last_clean":"2024-09-10T16:01:04.467097+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:01:04.467097+0000","last_undegraded":"2024-09-10T16:01:04.467097+0000","last_fullsized":"2024-09-10T16:01:04.467097+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":7949,"log_dups_size":0,"ondisk_log_size":7949,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32535494,"num_objects":7945,"num_object_clones":0,"num_object_copies":15890,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7945,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":6054,"num_write_kb":24216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'7933","reported_seq":13003,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T16:01:04.542330+0000","last_change":"2024-09-10T15:59:16.142612+0000","last_active":"2024-09-10T16:01:04.542330+0000","last_peered":"2024-09-10T16:01:04.542330+0000","last_clean":"2024-09-10T15:59:14.092473+0000","last_became_active":"2024-09-10T15:59:16.109611+0000","last_became_peered":"2024-09-10T15:59:16.109611+0000","last_unstale":"2024-09-10T16:01:04.542330+0000","last_undegraded":"2024-09-10T16:01:04.542330+0000","last_fullsized":"2024-09-10T15:59:15.770264+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":7933,"log_dups_size":0,"ondisk_log_size":7933,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32477184,"num_objects":7929,"num_object_clones":0,"num_object_copies":15858,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1797,"num_objects_unfound":0,"num_objects_dirty":7929,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5937,"num_write_kb":23748,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2472,"num_bytes_recovered":10121216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1797},{"shards":"2","objects":6132}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'7919","reported_seq":7995,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:01:04.565095+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:01:04.565095+0000","last_peered":"2024-09-10T16:01:04.565095+0000","last_clean":"2024-09-10T16:01:04.565095+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:01:04.565095+0000","last_undegraded":"2024-09-10T16:01:04.565095+0000","last_fullsized":"2024-09-10T16:01:04.565095+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":7919,"log_dups_size":0,"ondisk_log_size":7919,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32436224,"num_objects":7919,"num_object_clones":0,"num_object_copies":15838,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7919,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7919,"num_write_kb":31676,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'7943","reported_seq":8068,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:01:01.582154+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:01:01.582154+0000","last_peered":"2024-09-10T16:01:01.582154+0000","last_clean":"2024-09-10T16:01:01.582154+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:01:01.582154+0000","last_undegraded":"2024-09-10T16:01:01.582154+0000","last_fullsized":"2024-09-10T16:01:01.582154+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":7943,"log_dups_size":0,"ondisk_log_size":7943,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32506339,"num_objects":7937,"num_object_clones":0,"num_object_copies":15874,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7937,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4654,"num_write_kb":18616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'7974","reported_seq":8107,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:01.705200+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:01:01.705200+0000","last_peered":"2024-09-10T16:01:01.705200+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:01:01.705200+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":7974,"log_dups_size":0,"ondisk_log_size":7974,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32641507,"num_objects":7970,"num_object_clones":0,"num_object_copies":15940,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7970,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4586,"num_write_kb":18344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3599},{"shards":"0,2","objects":4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'7947","reported_seq":8080,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:01.776948+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:01:01.776948+0000","last_peered":"2024-09-10T16:01:01.776948+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:01:01.776948+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":7947,"log_dups_size":0,"ondisk_log_size":7947,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32527302,"num_objects":7943,"num_object_clones":0,"num_object_copies":15886,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7943,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4446,"num_write_kb":17784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3706},{"shards":"0,2","objects":4237}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'8045","reported_seq":8169,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:01:01.741436+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:01:01.741436+0000","last_peered":"2024-09-10T16:01:01.741436+0000","last_clean":"2024-09-10T16:01:01.741436+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:01:01.741436+0000","last_undegraded":"2024-09-10T16:01:01.741436+0000","last_fullsized":"2024-09-10T16:01:01.741436+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":8045,"log_dups_size":0,"ondisk_log_size":8045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32944128,"num_objects":8043,"num_object_clones":0,"num_object_copies":16086,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8043,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4892,"num_write_kb":19568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.775865+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T15:59:15.775865+0000","last_peered":"2024-09-10T15:59:15.775865+0000","last_clean":"2024-09-10T15:59:15.775865+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T15:59:15.775865+0000","last_undegraded":"2024-09-10T15:59:15.775865+0000","last_fullsized":"2024-09-10T15:59:15.775865+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'7649","reported_seq":7784,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:04.489946+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:01:04.489946+0000","last_peered":"2024-09-10T16:01:04.489946+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:01:04.489946+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":7649,"log_dups_size":0,"ondisk_log_size":7649,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31302115,"num_objects":7643,"num_object_clones":0,"num_object_copies":15286,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7643,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5468,"num_write_kb":21872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4123},{"shards":"2","objects":3520}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":89,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.772619+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T15:59:15.772619+0000","last_peered":"2024-09-10T15:59:15.772619+0000","last_clean":"2024-09-10T15:59:15.772619+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T15:59:15.772619+0000","last_undegraded":"2024-09-10T15:59:15.772619+0000","last_fullsized":"2024-09-10T15:59:15.772619+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'7882","reported_seq":8007,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:01.639997+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:01:01.639997+0000","last_peered":"2024-09-10T16:01:01.639997+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:01:01.639997+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":7882,"log_dups_size":0,"ondisk_log_size":7882,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32251904,"num_objects":7874,"num_object_clones":0,"num_object_copies":15748,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7874,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4386,"num_write_kb":17544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3619},{"shards":"0,2","objects":4255}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":99,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.884297+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T15:59:15.884297+0000","last_peered":"2024-09-10T15:59:15.884297+0000","last_clean":"2024-09-10T15:59:15.884297+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T15:59:15.884297+0000","last_undegraded":"2024-09-10T15:59:15.884297+0000","last_fullsized":"2024-09-10T15:59:15.884297+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'7978","reported_seq":8108,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:04.436700+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:01:04.436700+0000","last_peered":"2024-09-10T16:01:04.436700+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:01:04.436700+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":7978,"log_dups_size":0,"ondisk_log_size":7978,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32657891,"num_objects":7974,"num_object_clones":0,"num_object_copies":15948,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7974,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5658,"num_write_kb":22632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4304},{"shards":"2","objects":3670}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.837509+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T15:59:15.837509+0000","last_peered":"2024-09-10T15:59:15.837509+0000","last_clean":"2024-09-10T15:59:15.837509+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T15:59:15.837509+0000","last_undegraded":"2024-09-10T15:59:15.837509+0000","last_fullsized":"2024-09-10T15:59:15.837509+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'8113","reported_seq":8237,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:04.452045+0000","last_change":"2024-09-10T15:59:16.174113+0000","last_active":"2024-09-10T16:01:04.452045+0000","last_peered":"2024-09-10T16:01:04.452045+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:01:04.452045+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":8113,"log_dups_size":0,"ondisk_log_size":8113,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33206272,"num_objects":8107,"num_object_clones":0,"num_object_copies":16214,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4354,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8107,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5667,"num_write_kb":22668,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4352},{"shards":"2","objects":3755}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'7881","reported_seq":8011,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:01.792527+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:01:01.792527+0000","last_peered":"2024-09-10T16:01:01.792527+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:01:01.792527+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":7881,"log_dups_size":0,"ondisk_log_size":7881,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32272384,"num_objects":7879,"num_object_clones":0,"num_object_copies":15758,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7879,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4512,"num_write_kb":18048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3575},{"shards":"0,2","objects":4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.989413+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T15:59:15.989413+0000","last_peered":"2024-09-10T15:59:15.989413+0000","last_clean":"2024-09-10T15:59:15.989413+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T15:59:15.989413+0000","last_undegraded":"2024-09-10T15:59:15.989413+0000","last_fullsized":"2024-09-10T15:59:15.989413+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'7755","reported_seq":7893,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:01.778576+0000","last_change":"2024-09-10T15:59:16.251610+0000","last_active":"2024-09-10T16:01:01.778576+0000","last_peered":"2024-09-10T16:01:01.778576+0000","last_clean":"2024-09-10T15:59:14.115595+0000","last_became_active":"2024-09-10T15:59:16.244089+0000","last_became_peered":"2024-09-10T15:59:16.244089+0000","last_unstale":"2024-09-10T16:01:01.778576+0000","last_undegraded":"2024-09-10T15:59:15.847127+0000","last_fullsized":"2024-09-10T15:59:15.774437+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":7755,"log_dups_size":0,"ondisk_log_size":7755,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31731712,"num_objects":7747,"num_object_clones":0,"num_object_copies":15494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7747,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4145,"num_write_kb":16580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3544},{"shards":"0,2","objects":4203}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":89,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.773633+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T15:59:15.773633+0000","last_peered":"2024-09-10T15:59:15.773633+0000","last_clean":"2024-09-10T15:59:15.773633+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T15:59:15.773633+0000","last_undegraded":"2024-09-10T15:59:15.773633+0000","last_fullsized":"2024-09-10T15:59:15.773633+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'8020","reported_seq":8157,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:01.786350+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:01:01.786350+0000","last_peered":"2024-09-10T16:01:01.786350+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:01:01.786350+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":8020,"log_dups_size":0,"ondisk_log_size":8020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32833536,"num_objects":8016,"num_object_clones":0,"num_object_copies":16032,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8016,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4275,"num_write_kb":17100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3645},{"shards":"0,2","objects":4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.916017+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T15:59:15.916017+0000","last_peered":"2024-09-10T15:59:15.916017+0000","last_clean":"2024-09-10T15:59:15.916017+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T15:59:15.916017+0000","last_undegraded":"2024-09-10T15:59:15.916017+0000","last_fullsized":"2024-09-10T15:59:15.916017+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.776033+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T15:59:15.776033+0000","last_peered":"2024-09-10T15:59:15.776033+0000","last_clean":"2024-09-10T15:59:15.776033+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T15:59:15.776033+0000","last_undegraded":"2024-09-10T15:59:15.776033+0000","last_fullsized":"2024-09-10T15:59:15.776033+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'7995","reported_seq":8134,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:01.716122+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:01:01.716122+0000","last_peered":"2024-09-10T16:01:01.716122+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:01:01.716122+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":7995,"log_dups_size":0,"ondisk_log_size":7995,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32699334,"num_objects":7985,"num_object_clones":0,"num_object_copies":15970,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7985,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4158,"num_write_kb":16632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3662},{"shards":"0,2","objects":4323}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":519226078,"num_objects":126773,"num_object_clones":0,"num_object_copies":253546,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":47117,"num_objects_misplaced":1797,"num_objects_unfound":0,"num_objects_dirty":126773,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":84619,"num_write_kb":338476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2475,"num_bytes_recovered":10133504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1050980352,"data_stored":1050828606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":126845,"ondisk_log_size":126845,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986213,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":648612,"kb_used_data":392020,"kb_used_omap":4,"kb_used_meta":256571,"kb_avail":93723228,"statfs":{"total":96636764160,"available":95972585472,"internally_reserved":0,"allocated":401428480,"data_stored":401150720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":262729386},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,1,1,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659159,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":333568,"kb_used_data":185264,"kb_used_omap":4,"kb_used_meta":148283,"kb_avail":94038272,"statfs":{"total":96636764160,"available":96295190528,"internally_reserved":0,"allocated":189710336,"data_stored":189385143,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":151842478},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,2,0,3,2,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182441,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":713688,"kb_used_data":447432,"kb_used_omap":4,"kb_used_meta":266235,"kb_avail":93658152,"statfs":{"total":96636764160,"available":95905947648,"internally_reserved":0,"allocated":458170368,"data_stored":457888531,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":272625322},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,3,4,1,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":458813440,"data_stored":458777310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":190775296,"data_stored":190695810,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":401391616,"data_stored":401355486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:01:05.984 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 153 20 128236 128216 3.27299 1.77344 0.0241542 0.0238618 2024-09-10T16:01:06.984 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 154 20 128753 128733 3.26485 2.01953 0.0567879 0.0239227 2024-09-10T16:01:07.984 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 155 20 129270 129250 3.25681 2.01953 0.0478101 0.0239808 2024-09-10T16:01:08.865 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:01:08.984 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 156 20 129772 129752 3.2485 1.96094 0.02062 0.0240416 2024-09-10T16:01:09.449 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:01:09.587 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:01:09.589 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":152,"stamp":"2024-09-10T16:01:07.580568+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":525501169,"num_objects":128307,"num_object_clones":0,"num_object_copies":256614,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":47117,"num_objects_misplaced":1797,"num_objects_unfound":0,"num_objects_dirty":128307,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":86153,"num_write_kb":344606,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2477,"num_bytes_recovered":10133523,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128380,"ondisk_log_size":128380,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1725328,"kb_used_data":1031968,"kb_used_omap":12,"kb_used_meta":693299,"kb_avail":281390192,"statfs":{"total":289910292480,"available":288143556608,"internally_reserved":0,"allocated":1056735232,"data_stored":1055850442,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":709938178},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,4,2,4,7,8,9,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":26615808,"num_objects":6498,"num_object_clones":0,"num_object_copies":12996,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-267,"num_objects_unfound":0,"num_objects_dirty":6498,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6498,"num_write_kb":25992,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":267,"num_bytes_recovered":1093632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004558"},"pg_stats":[{"pgid":"2.f","version":"43'8003","reported_seq":8073,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:06.852417+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:01:06.852417+0000","last_peered":"2024-09-10T16:01:06.852417+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:01:06.852417+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":8003,"log_dups_size":0,"ondisk_log_size":8003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32780288,"num_objects":8003,"num_object_clones":0,"num_object_copies":16006,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8003,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8003,"num_write_kb":32012,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3744},{"shards":"0,2","objects":4258}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'7949","reported_seq":8056,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:01:04.467097+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:01:04.467097+0000","last_peered":"2024-09-10T16:01:04.467097+0000","last_clean":"2024-09-10T16:01:04.467097+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:01:04.467097+0000","last_undegraded":"2024-09-10T16:01:04.467097+0000","last_fullsized":"2024-09-10T16:01:04.467097+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":7949,"log_dups_size":0,"ondisk_log_size":7949,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32535494,"num_objects":7945,"num_object_clones":0,"num_object_copies":15890,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7945,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":6054,"num_write_kb":24216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'7933","reported_seq":13003,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T16:01:04.542330+0000","last_change":"2024-09-10T15:59:16.142612+0000","last_active":"2024-09-10T16:01:04.542330+0000","last_peered":"2024-09-10T16:01:04.542330+0000","last_clean":"2024-09-10T15:59:14.092473+0000","last_became_active":"2024-09-10T15:59:16.109611+0000","last_became_peered":"2024-09-10T15:59:16.109611+0000","last_unstale":"2024-09-10T16:01:04.542330+0000","last_undegraded":"2024-09-10T16:01:04.542330+0000","last_fullsized":"2024-09-10T15:59:15.770264+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":7933,"log_dups_size":0,"ondisk_log_size":7933,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32477184,"num_objects":7929,"num_object_clones":0,"num_object_copies":15858,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1797,"num_objects_unfound":0,"num_objects_dirty":7929,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5937,"num_write_kb":23748,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2472,"num_bytes_recovered":10121216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1797},{"shards":"2","objects":6132}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'7919","reported_seq":7995,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:01:04.565095+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:01:04.565095+0000","last_peered":"2024-09-10T16:01:04.565095+0000","last_clean":"2024-09-10T16:01:04.565095+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:01:04.565095+0000","last_undegraded":"2024-09-10T16:01:04.565095+0000","last_fullsized":"2024-09-10T16:01:04.565095+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":7919,"log_dups_size":0,"ondisk_log_size":7919,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32436224,"num_objects":7919,"num_object_clones":0,"num_object_copies":15838,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7919,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7919,"num_write_kb":31676,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'8096","reported_seq":8221,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:01:06.820217+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:01:06.820217+0000","last_peered":"2024-09-10T16:01:06.820217+0000","last_clean":"2024-09-10T16:01:06.820217+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:01:06.820217+0000","last_undegraded":"2024-09-10T16:01:06.820217+0000","last_fullsized":"2024-09-10T16:01:06.820217+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":8096,"log_dups_size":0,"ondisk_log_size":8096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33133027,"num_objects":8090,"num_object_clones":0,"num_object_copies":16180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8090,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4807,"num_write_kb":19228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'8121","reported_seq":8254,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:06.759603+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:01:06.759603+0000","last_peered":"2024-09-10T16:01:06.759603+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:01:06.759603+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":8121,"log_dups_size":0,"ondisk_log_size":8121,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33243619,"num_objects":8117,"num_object_clones":0,"num_object_copies":16234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8117,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4733,"num_write_kb":18932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3746},{"shards":"0,2","objects":4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'8105","reported_seq":8238,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:06.761710+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:01:06.761710+0000","last_peered":"2024-09-10T16:01:06.761710+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:01:06.761710+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":8105,"log_dups_size":0,"ondisk_log_size":8105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33174470,"num_objects":8101,"num_object_clones":0,"num_object_copies":16202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8101,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4604,"num_write_kb":18416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3864},{"shards":"0,2","objects":4237}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'8214","reported_seq":8339,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:01:06.751291+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:01:06.751291+0000","last_peered":"2024-09-10T16:01:06.751291+0000","last_clean":"2024-09-10T16:01:06.751291+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:01:06.751291+0000","last_undegraded":"2024-09-10T16:01:06.751291+0000","last_fullsized":"2024-09-10T16:01:06.751291+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":8214,"log_dups_size":0,"ondisk_log_size":8214,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33636352,"num_objects":8212,"num_object_clones":0,"num_object_copies":16424,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8212,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5061,"num_write_kb":20244,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.775865+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T15:59:15.775865+0000","last_peered":"2024-09-10T15:59:15.775865+0000","last_clean":"2024-09-10T15:59:15.775865+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T15:59:15.775865+0000","last_undegraded":"2024-09-10T15:59:15.775865+0000","last_fullsized":"2024-09-10T15:59:15.775865+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'7649","reported_seq":7784,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:04.489946+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:01:04.489946+0000","last_peered":"2024-09-10T16:01:04.489946+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:01:04.489946+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":7649,"log_dups_size":0,"ondisk_log_size":7649,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31302115,"num_objects":7643,"num_object_clones":0,"num_object_copies":15286,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7643,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5468,"num_write_kb":21872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4123},{"shards":"2","objects":3520}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":89,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.772619+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T15:59:15.772619+0000","last_peered":"2024-09-10T15:59:15.772619+0000","last_clean":"2024-09-10T15:59:15.772619+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T15:59:15.772619+0000","last_undegraded":"2024-09-10T15:59:15.772619+0000","last_fullsized":"2024-09-10T15:59:15.772619+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'8049","reported_seq":8173,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:06.845337+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:01:06.845337+0000","last_peered":"2024-09-10T16:01:06.845337+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:01:06.845337+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":8049,"log_dups_size":0,"ondisk_log_size":8049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32935936,"num_objects":8041,"num_object_clones":0,"num_object_copies":16082,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8041,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4553,"num_write_kb":18212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3786},{"shards":"0,2","objects":4255}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":99,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.884297+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T15:59:15.884297+0000","last_peered":"2024-09-10T15:59:15.884297+0000","last_clean":"2024-09-10T15:59:15.884297+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T15:59:15.884297+0000","last_undegraded":"2024-09-10T15:59:15.884297+0000","last_fullsized":"2024-09-10T15:59:15.884297+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'7978","reported_seq":8108,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:04.436700+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:01:04.436700+0000","last_peered":"2024-09-10T16:01:04.436700+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:01:04.436700+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":7978,"log_dups_size":0,"ondisk_log_size":7978,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32657891,"num_objects":7974,"num_object_clones":0,"num_object_copies":15948,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7974,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5658,"num_write_kb":22632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4304},{"shards":"2","objects":3670}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.837509+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T15:59:15.837509+0000","last_peered":"2024-09-10T15:59:15.837509+0000","last_clean":"2024-09-10T15:59:15.837509+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T15:59:15.837509+0000","last_undegraded":"2024-09-10T15:59:15.837509+0000","last_fullsized":"2024-09-10T15:59:15.837509+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'8113","reported_seq":8237,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:04.452045+0000","last_change":"2024-09-10T15:59:16.174113+0000","last_active":"2024-09-10T16:01:04.452045+0000","last_peered":"2024-09-10T16:01:04.452045+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:01:04.452045+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":8113,"log_dups_size":0,"ondisk_log_size":8113,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33206272,"num_objects":8107,"num_object_clones":0,"num_object_copies":16214,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4354,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8107,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5667,"num_write_kb":22668,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4352},{"shards":"2","objects":3755}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'8038","reported_seq":8168,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:06.803268+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:01:06.803268+0000","last_peered":"2024-09-10T16:01:06.803268+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:01:06.803268+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":8038,"log_dups_size":0,"ondisk_log_size":8038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32915456,"num_objects":8036,"num_object_clones":0,"num_object_copies":16072,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8036,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4669,"num_write_kb":18676,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3732},{"shards":"0,2","objects":4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.989413+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T15:59:15.989413+0000","last_peered":"2024-09-10T15:59:15.989413+0000","last_clean":"2024-09-10T15:59:15.989413+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T15:59:15.989413+0000","last_undegraded":"2024-09-10T15:59:15.989413+0000","last_fullsized":"2024-09-10T15:59:15.989413+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'7921","reported_seq":8058,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:06.866028+0000","last_change":"2024-09-10T15:59:16.251610+0000","last_active":"2024-09-10T16:01:06.866028+0000","last_peered":"2024-09-10T16:01:06.866028+0000","last_clean":"2024-09-10T15:59:14.115595+0000","last_became_active":"2024-09-10T15:59:16.244089+0000","last_became_peered":"2024-09-10T15:59:16.244089+0000","last_unstale":"2024-09-10T16:01:06.866028+0000","last_undegraded":"2024-09-10T15:59:15.847127+0000","last_fullsized":"2024-09-10T15:59:15.774437+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":7921,"log_dups_size":0,"ondisk_log_size":7921,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32411648,"num_objects":7913,"num_object_clones":0,"num_object_copies":15826,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7913,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4311,"num_write_kb":17244,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3710},{"shards":"0,2","objects":4203}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":89,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.773633+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T15:59:15.773633+0000","last_peered":"2024-09-10T15:59:15.773633+0000","last_clean":"2024-09-10T15:59:15.773633+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T15:59:15.773633+0000","last_undegraded":"2024-09-10T15:59:15.773633+0000","last_fullsized":"2024-09-10T15:59:15.773633+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'8141","reported_seq":8278,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:06.844139+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:01:06.844139+0000","last_peered":"2024-09-10T16:01:06.844139+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:01:06.844139+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":8141,"log_dups_size":0,"ondisk_log_size":8141,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33329152,"num_objects":8137,"num_object_clones":0,"num_object_copies":16274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8137,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4396,"num_write_kb":17584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3766},{"shards":"0,2","objects":4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.916017+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T15:59:15.916017+0000","last_peered":"2024-09-10T15:59:15.916017+0000","last_clean":"2024-09-10T15:59:15.916017+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T15:59:15.916017+0000","last_undegraded":"2024-09-10T15:59:15.916017+0000","last_fullsized":"2024-09-10T15:59:15.916017+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.776033+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T15:59:15.776033+0000","last_peered":"2024-09-10T15:59:15.776033+0000","last_clean":"2024-09-10T15:59:15.776033+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T15:59:15.776033+0000","last_undegraded":"2024-09-10T15:59:15.776033+0000","last_fullsized":"2024-09-10T15:59:15.776033+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'8148","reported_seq":8287,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:06.811628+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:01:06.811628+0000","last_peered":"2024-09-10T16:01:06.811628+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:01:06.811628+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":8148,"log_dups_size":0,"ondisk_log_size":8148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33326022,"num_objects":8138,"num_object_clones":0,"num_object_copies":16276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8138,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4311,"num_write_kb":17244,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3815},{"shards":"0,2","objects":4323}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":525501150,"num_objects":128305,"num_object_clones":0,"num_object_copies":256610,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":47117,"num_objects_misplaced":1797,"num_objects_unfound":0,"num_objects_dirty":128305,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":86151,"num_write_kb":344604,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2475,"num_bytes_recovered":10133504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1058570240,"data_stored":1058418494,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128377,"ondisk_log_size":128377,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986213,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":648612,"kb_used_data":392020,"kb_used_omap":4,"kb_used_meta":256571,"kb_avail":93723228,"statfs":{"total":96636764160,"available":95972585472,"internally_reserved":0,"allocated":401428480,"data_stored":401150720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":262729386},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,1,1,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659159,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":333568,"kb_used_data":185264,"kb_used_omap":4,"kb_used_meta":148283,"kb_avail":94038272,"statfs":{"total":96636764160,"available":96295190528,"internally_reserved":0,"allocated":189710336,"data_stored":189385143,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":151842478},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,2,0,3,2,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182442,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":743148,"kb_used_data":454684,"kb_used_omap":4,"kb_used_meta":288443,"kb_avail":93628692,"statfs":{"total":96636764160,"available":95875780608,"internally_reserved":0,"allocated":465596416,"data_stored":465314579,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":295366314},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,0,2,3,5,3,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":466403328,"data_stored":466367198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":190775296,"data_stored":190695810,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":401391616,"data_stored":401355486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:01:09.590 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:01:09.984 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 157 20 130230 130210 3.23921 1.78906 0.0847162 0.0241109 2024-09-10T16:01:10.126 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:01:10.126 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:01:10.148 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":153,"stamp":"2024-09-10T16:01:09.581213+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":525501169,"num_objects":128307,"num_object_clones":0,"num_object_copies":256614,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":47117,"num_objects_misplaced":1797,"num_objects_unfound":0,"num_objects_dirty":128307,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":86153,"num_write_kb":344606,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2477,"num_bytes_recovered":10133523,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128380,"ondisk_log_size":128380,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1739912,"kb_used_data":1040980,"kb_used_omap":12,"kb_used_meta":698867,"kb_avail":281375608,"statfs":{"total":289910292480,"available":288128622592,"internally_reserved":0,"allocated":1065963520,"data_stored":1065078730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":715639810},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,0,4,11,9,9,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":20058112,"num_objects":4897,"num_object_clones":0,"num_object_copies":9794,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-267,"num_objects_unfound":0,"num_objects_dirty":4897,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4897,"num_write_kb":19588,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":267,"num_bytes_recovered":1093632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004334"},"pg_stats":[{"pgid":"2.f","version":"43'8003","reported_seq":8073,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:06.852417+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:01:06.852417+0000","last_peered":"2024-09-10T16:01:06.852417+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:01:06.852417+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":8003,"log_dups_size":0,"ondisk_log_size":8003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32780288,"num_objects":8003,"num_object_clones":0,"num_object_copies":16006,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8003,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8003,"num_write_kb":32012,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3744},{"shards":"0,2","objects":4258}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'7949","reported_seq":8056,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:01:04.467097+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:01:04.467097+0000","last_peered":"2024-09-10T16:01:04.467097+0000","last_clean":"2024-09-10T16:01:04.467097+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:01:04.467097+0000","last_undegraded":"2024-09-10T16:01:04.467097+0000","last_fullsized":"2024-09-10T16:01:04.467097+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":7949,"log_dups_size":0,"ondisk_log_size":7949,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32535494,"num_objects":7945,"num_object_clones":0,"num_object_copies":15890,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7945,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":6054,"num_write_kb":24216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'7933","reported_seq":13003,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T16:01:04.542330+0000","last_change":"2024-09-10T15:59:16.142612+0000","last_active":"2024-09-10T16:01:04.542330+0000","last_peered":"2024-09-10T16:01:04.542330+0000","last_clean":"2024-09-10T15:59:14.092473+0000","last_became_active":"2024-09-10T15:59:16.109611+0000","last_became_peered":"2024-09-10T15:59:16.109611+0000","last_unstale":"2024-09-10T16:01:04.542330+0000","last_undegraded":"2024-09-10T16:01:04.542330+0000","last_fullsized":"2024-09-10T15:59:15.770264+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":7933,"log_dups_size":0,"ondisk_log_size":7933,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32477184,"num_objects":7929,"num_object_clones":0,"num_object_copies":15858,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1797,"num_objects_unfound":0,"num_objects_dirty":7929,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5937,"num_write_kb":23748,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2472,"num_bytes_recovered":10121216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1797},{"shards":"2","objects":6132}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'7919","reported_seq":7995,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:01:04.565095+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:01:04.565095+0000","last_peered":"2024-09-10T16:01:04.565095+0000","last_clean":"2024-09-10T16:01:04.565095+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:01:04.565095+0000","last_undegraded":"2024-09-10T16:01:04.565095+0000","last_fullsized":"2024-09-10T16:01:04.565095+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":7919,"log_dups_size":0,"ondisk_log_size":7919,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32436224,"num_objects":7919,"num_object_clones":0,"num_object_copies":15838,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7919,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7919,"num_write_kb":31676,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'8096","reported_seq":8221,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:01:06.820217+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:01:06.820217+0000","last_peered":"2024-09-10T16:01:06.820217+0000","last_clean":"2024-09-10T16:01:06.820217+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:01:06.820217+0000","last_undegraded":"2024-09-10T16:01:06.820217+0000","last_fullsized":"2024-09-10T16:01:06.820217+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":8096,"log_dups_size":0,"ondisk_log_size":8096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33133027,"num_objects":8090,"num_object_clones":0,"num_object_copies":16180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8090,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4807,"num_write_kb":19228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'8121","reported_seq":8254,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:06.759603+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:01:06.759603+0000","last_peered":"2024-09-10T16:01:06.759603+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:01:06.759603+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":8121,"log_dups_size":0,"ondisk_log_size":8121,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33243619,"num_objects":8117,"num_object_clones":0,"num_object_copies":16234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8117,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4733,"num_write_kb":18932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3746},{"shards":"0,2","objects":4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'8105","reported_seq":8238,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:06.761710+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:01:06.761710+0000","last_peered":"2024-09-10T16:01:06.761710+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:01:06.761710+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":8105,"log_dups_size":0,"ondisk_log_size":8105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33174470,"num_objects":8101,"num_object_clones":0,"num_object_copies":16202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8101,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4604,"num_write_kb":18416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3864},{"shards":"0,2","objects":4237}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'8214","reported_seq":8339,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:01:06.751291+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:01:06.751291+0000","last_peered":"2024-09-10T16:01:06.751291+0000","last_clean":"2024-09-10T16:01:06.751291+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:01:06.751291+0000","last_undegraded":"2024-09-10T16:01:06.751291+0000","last_fullsized":"2024-09-10T16:01:06.751291+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":8214,"log_dups_size":0,"ondisk_log_size":8214,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33636352,"num_objects":8212,"num_object_clones":0,"num_object_copies":16424,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8212,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5061,"num_write_kb":20244,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.775865+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T15:59:15.775865+0000","last_peered":"2024-09-10T15:59:15.775865+0000","last_clean":"2024-09-10T15:59:15.775865+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T15:59:15.775865+0000","last_undegraded":"2024-09-10T15:59:15.775865+0000","last_fullsized":"2024-09-10T15:59:15.775865+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'7649","reported_seq":7784,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:04.489946+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:01:04.489946+0000","last_peered":"2024-09-10T16:01:04.489946+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:01:04.489946+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":7649,"log_dups_size":0,"ondisk_log_size":7649,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31302115,"num_objects":7643,"num_object_clones":0,"num_object_copies":15286,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7643,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5468,"num_write_kb":21872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4123},{"shards":"2","objects":3520}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":89,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.772619+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T15:59:15.772619+0000","last_peered":"2024-09-10T15:59:15.772619+0000","last_clean":"2024-09-10T15:59:15.772619+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T15:59:15.772619+0000","last_undegraded":"2024-09-10T15:59:15.772619+0000","last_fullsized":"2024-09-10T15:59:15.772619+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'8049","reported_seq":8173,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:06.845337+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:01:06.845337+0000","last_peered":"2024-09-10T16:01:06.845337+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:01:06.845337+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":8049,"log_dups_size":0,"ondisk_log_size":8049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32935936,"num_objects":8041,"num_object_clones":0,"num_object_copies":16082,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8041,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4553,"num_write_kb":18212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3786},{"shards":"0,2","objects":4255}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":99,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.884297+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T15:59:15.884297+0000","last_peered":"2024-09-10T15:59:15.884297+0000","last_clean":"2024-09-10T15:59:15.884297+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T15:59:15.884297+0000","last_undegraded":"2024-09-10T15:59:15.884297+0000","last_fullsized":"2024-09-10T15:59:15.884297+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'7978","reported_seq":8108,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:04.436700+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:01:04.436700+0000","last_peered":"2024-09-10T16:01:04.436700+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:01:04.436700+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":7978,"log_dups_size":0,"ondisk_log_size":7978,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32657891,"num_objects":7974,"num_object_clones":0,"num_object_copies":15948,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7974,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5658,"num_write_kb":22632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4304},{"shards":"2","objects":3670}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.837509+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T15:59:15.837509+0000","last_peered":"2024-09-10T15:59:15.837509+0000","last_clean":"2024-09-10T15:59:15.837509+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T15:59:15.837509+0000","last_undegraded":"2024-09-10T15:59:15.837509+0000","last_fullsized":"2024-09-10T15:59:15.837509+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'8113","reported_seq":8237,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:04.452045+0000","last_change":"2024-09-10T15:59:16.174113+0000","last_active":"2024-09-10T16:01:04.452045+0000","last_peered":"2024-09-10T16:01:04.452045+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:01:04.452045+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":8113,"log_dups_size":0,"ondisk_log_size":8113,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33206272,"num_objects":8107,"num_object_clones":0,"num_object_copies":16214,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4354,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8107,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5667,"num_write_kb":22668,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4352},{"shards":"2","objects":3755}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'8038","reported_seq":8168,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:06.803268+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:01:06.803268+0000","last_peered":"2024-09-10T16:01:06.803268+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:01:06.803268+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":8038,"log_dups_size":0,"ondisk_log_size":8038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32915456,"num_objects":8036,"num_object_clones":0,"num_object_copies":16072,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8036,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4669,"num_write_kb":18676,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3732},{"shards":"0,2","objects":4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.989413+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T15:59:15.989413+0000","last_peered":"2024-09-10T15:59:15.989413+0000","last_clean":"2024-09-10T15:59:15.989413+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T15:59:15.989413+0000","last_undegraded":"2024-09-10T15:59:15.989413+0000","last_fullsized":"2024-09-10T15:59:15.989413+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'7921","reported_seq":8058,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:06.866028+0000","last_change":"2024-09-10T15:59:16.251610+0000","last_active":"2024-09-10T16:01:06.866028+0000","last_peered":"2024-09-10T16:01:06.866028+0000","last_clean":"2024-09-10T15:59:14.115595+0000","last_became_active":"2024-09-10T15:59:16.244089+0000","last_became_peered":"2024-09-10T15:59:16.244089+0000","last_unstale":"2024-09-10T16:01:06.866028+0000","last_undegraded":"2024-09-10T15:59:15.847127+0000","last_fullsized":"2024-09-10T15:59:15.774437+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":7921,"log_dups_size":0,"ondisk_log_size":7921,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32411648,"num_objects":7913,"num_object_clones":0,"num_object_copies":15826,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7913,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4311,"num_write_kb":17244,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3710},{"shards":"0,2","objects":4203}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":89,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.773633+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T15:59:15.773633+0000","last_peered":"2024-09-10T15:59:15.773633+0000","last_clean":"2024-09-10T15:59:15.773633+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T15:59:15.773633+0000","last_undegraded":"2024-09-10T15:59:15.773633+0000","last_fullsized":"2024-09-10T15:59:15.773633+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'8141","reported_seq":8278,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:06.844139+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:01:06.844139+0000","last_peered":"2024-09-10T16:01:06.844139+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:01:06.844139+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":8141,"log_dups_size":0,"ondisk_log_size":8141,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33329152,"num_objects":8137,"num_object_clones":0,"num_object_copies":16274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8137,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4396,"num_write_kb":17584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3766},{"shards":"0,2","objects":4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.916017+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T15:59:15.916017+0000","last_peered":"2024-09-10T15:59:15.916017+0000","last_clean":"2024-09-10T15:59:15.916017+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T15:59:15.916017+0000","last_undegraded":"2024-09-10T15:59:15.916017+0000","last_fullsized":"2024-09-10T15:59:15.916017+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.776033+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T15:59:15.776033+0000","last_peered":"2024-09-10T15:59:15.776033+0000","last_clean":"2024-09-10T15:59:15.776033+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T15:59:15.776033+0000","last_undegraded":"2024-09-10T15:59:15.776033+0000","last_fullsized":"2024-09-10T15:59:15.776033+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'8148","reported_seq":8287,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:06.811628+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:01:06.811628+0000","last_peered":"2024-09-10T16:01:06.811628+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:01:06.811628+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":8148,"log_dups_size":0,"ondisk_log_size":8148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33326022,"num_objects":8138,"num_object_clones":0,"num_object_copies":16276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8138,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4311,"num_write_kb":17244,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3815},{"shards":"0,2","objects":4323}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":525501150,"num_objects":128305,"num_object_clones":0,"num_object_copies":256610,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":47117,"num_objects_misplaced":1797,"num_objects_unfound":0,"num_objects_dirty":128305,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":86151,"num_write_kb":344604,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2475,"num_bytes_recovered":10133504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1066409984,"data_stored":1066258238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128377,"ondisk_log_size":128377,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986213,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":648612,"kb_used_data":392020,"kb_used_omap":4,"kb_used_meta":256571,"kb_avail":93723228,"statfs":{"total":96636764160,"available":95972585472,"internally_reserved":0,"allocated":401428480,"data_stored":401150720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":262729386},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,1,1,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659160,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":348152,"kb_used_data":194276,"kb_used_omap":4,"kb_used_meta":153851,"kb_avail":94023688,"statfs":{"total":96636764160,"available":96280256512,"internally_reserved":0,"allocated":198938624,"data_stored":198613431,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":157544110},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,7,3,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182442,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":743148,"kb_used_data":454684,"kb_used_omap":4,"kb_used_meta":288443,"kb_avail":93628692,"statfs":{"total":96636764160,"available":95875780608,"internally_reserved":0,"allocated":465596416,"data_stored":465314579,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":295366314},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,0,2,3,5,3,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":466403328,"data_stored":466367198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":198615040,"data_stored":198535554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":401391616,"data_stored":401355486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:01:10.149 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-10T16:01:10.150 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "43'8003", 'reported_seq': 8073, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:01:06.852417+0000', 'last_change': '2024-09-10T15:59:16.281558+0000', 'last_active': '2024-09-10T16:01:06.852417+0000', 'last_peered': '2024-09-10T16:01:06.852417+0000', 'last_clean': '2024-09-10T15:59:14.132456+0000', 'last_became_active': '2024-09-10T15:59:16.274407+0000', 'last_became_peered': '2024-09-10T15:59:16.274407+0000', 'last_unstale': '2024-09-10T16:01:06.852417+0000', 'last_undegraded': '2024-09-10T15:59:15.851279+0000', 'last_fullsized': '2024-09-10T15:59:15.774356+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'objects_scrubbed': 0, 'log_size': 8003, 'log_dups_size': 0, 'ondisk_log_size': 8003, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32780288, 'num_objects': 8003, 'num_object_clones': 0, 'num_object_copies': 16006, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4259, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8003, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8003, 'num_write_kb': 32012, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3744}, {'shards': '0,2', 'objects': 4258}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:01:10.152 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-10T16:01:10.152 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "43'7933", 'reported_seq': 13003, 'reported_epoch': 43, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-10T16:01:04.542330+0000', 'last_change': '2024-09-10T15:59:16.142612+0000', 'last_active': '2024-09-10T16:01:04.542330+0000', 'last_peered': '2024-09-10T16:01:04.542330+0000', 'last_clean': '2024-09-10T15:59:14.092473+0000', 'last_became_active': '2024-09-10T15:59:16.109611+0000', 'last_became_peered': '2024-09-10T15:59:16.109611+0000', 'last_unstale': '2024-09-10T16:01:04.542330+0000', 'last_undegraded': '2024-09-10T16:01:04.542330+0000', 'last_fullsized': '2024-09-10T15:59:15.770264+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2029", 'last_scrub_stamp': '2024-09-10T15:58:45.933789+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:45.933789+0000', 'objects_scrubbed': 1992, 'log_size': 7933, 'log_dups_size': 0, 'ondisk_log_size': 7933, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 668, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32477184, 'num_objects': 7929, 'num_object_clones': 0, 'num_object_copies': 15858, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1797, 'num_objects_unfound': 0, 'num_objects_dirty': 7929, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5937, 'num_write_kb': 23748, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2472, 'num_bytes_recovered': 10121216, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1797}, {'shards': '2', 'objects': 6132}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:01:10.152 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-10T16:01:10.152 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "43'8121", 'reported_seq': 8254, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:01:06.759603+0000', 'last_change': '2024-09-10T15:59:16.308264+0000', 'last_active': '2024-09-10T16:01:06.759603+0000', 'last_peered': '2024-09-10T16:01:06.759603+0000', 'last_clean': '2024-09-10T15:59:14.111551+0000', 'last_became_active': '2024-09-10T15:59:16.300040+0000', 'last_became_peered': '2024-09-10T15:59:16.300040+0000', 'last_unstale': '2024-09-10T16:01:06.759603+0000', 'last_undegraded': '2024-09-10T15:59:15.878234+0000', 'last_fullsized': '2024-09-10T15:59:15.769328+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3442", 'last_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'objects_scrubbed': 3384, 'log_size': 8121, 'log_dups_size': 0, 'ondisk_log_size': 8121, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1292, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33243619, 'num_objects': 8117, 'num_object_clones': 0, 'num_object_copies': 16234, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4372, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8117, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 4733, 'num_write_kb': 18932, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3746}, {'shards': '0,2', 'objects': 4371}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:01:10.152 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-10T16:01:10.152 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "43'8105", 'reported_seq': 8238, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:01:06.761710+0000', 'last_change': '2024-09-10T15:59:16.331899+0000', 'last_active': '2024-09-10T16:01:06.761710+0000', 'last_peered': '2024-09-10T16:01:06.761710+0000', 'last_clean': '2024-09-10T15:59:14.139436+0000', 'last_became_active': '2024-09-10T15:59:16.318715+0000', 'last_became_peered': '2024-09-10T15:59:16.318715+0000', 'last_unstale': '2024-09-10T16:01:06.761710+0000', 'last_undegraded': '2024-09-10T15:59:15.877524+0000', 'last_fullsized': '2024-09-10T15:59:15.769042+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3537", 'last_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'objects_scrubbed': 3497, 'log_size': 8105, 'log_dups_size': 0, 'ondisk_log_size': 8105, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1224, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33174470, 'num_objects': 8101, 'num_object_clones': 0, 'num_object_copies': 16202, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4237, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8101, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 4604, 'num_write_kb': 18416, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3864}, {'shards': '0,2', 'objects': 4237}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:01:10.152 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-10T16:01:10.152 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "43'7649", 'reported_seq': 7784, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:01:04.489946+0000', 'last_change': '2024-09-10T15:59:16.175680+0000', 'last_active': '2024-09-10T16:01:04.489946+0000', 'last_peered': '2024-09-10T16:01:04.489946+0000', 'last_clean': '2024-09-10T15:59:14.088187+0000', 'last_became_active': '2024-09-10T15:59:16.165810+0000', 'last_became_peered': '2024-09-10T15:59:16.165810+0000', 'last_unstale': '2024-09-10T16:01:04.489946+0000', 'last_undegraded': '2024-09-10T15:59:15.822098+0000', 'last_fullsized': '2024-09-10T15:59:15.769123+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2218", 'last_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'objects_scrubbed': 2175, 'log_size': 7649, 'log_dups_size': 0, 'ondisk_log_size': 7649, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 825, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31302115, 'num_objects': 7643, 'num_object_clones': 0, 'num_object_copies': 15286, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4124, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7643, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 5468, 'num_write_kb': 21872, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4123}, {'shards': '2', 'objects': 3520}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:01:10.153 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-10T16:01:10.153 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "43'8049", 'reported_seq': 8173, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:01:06.845337+0000', 'last_change': '2024-09-10T15:59:16.277742+0000', 'last_active': '2024-09-10T16:01:06.845337+0000', 'last_peered': '2024-09-10T16:01:06.845337+0000', 'last_clean': '2024-09-10T15:59:14.129081+0000', 'last_became_active': '2024-09-10T15:59:16.269835+0000', 'last_became_peered': '2024-09-10T15:59:16.269835+0000', 'last_unstale': '2024-09-10T16:01:06.845337+0000', 'last_undegraded': '2024-09-10T15:59:15.860857+0000', 'last_fullsized': '2024-09-10T15:59:15.771410+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3557", 'last_scrub_stamp': '2024-09-10T15:59:02.493678+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:02.493678+0000', 'objects_scrubbed': 3488, 'log_size': 8049, 'log_dups_size': 0, 'ondisk_log_size': 8049, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1472, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32935936, 'num_objects': 8041, 'num_object_clones': 0, 'num_object_copies': 16082, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4257, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8041, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4553, 'num_write_kb': 18212, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3786}, {'shards': '0,2', 'objects': 4255}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:01:10.153 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-10T16:01:10.153 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "43'7978", 'reported_seq': 8108, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:01:04.436700+0000', 'last_change': '2024-09-10T15:59:16.284311+0000', 'last_active': '2024-09-10T16:01:04.436700+0000', 'last_peered': '2024-09-10T16:01:04.436700+0000', 'last_clean': '2024-09-10T15:59:14.116041+0000', 'last_became_active': '2024-09-10T15:59:16.274543+0000', 'last_became_peered': '2024-09-10T15:59:16.274543+0000', 'last_unstale': '2024-09-10T16:01:04.436700+0000', 'last_undegraded': '2024-09-10T15:59:15.812484+0000', 'last_fullsized': '2024-09-10T15:59:15.770885+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2356", 'last_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'objects_scrubbed': 2316, 'log_size': 7978, 'log_dups_size': 0, 'ondisk_log_size': 7978, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 783, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32657891, 'num_objects': 7974, 'num_object_clones': 0, 'num_object_copies': 15948, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7974, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 5658, 'num_write_kb': 22632, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4304}, {'shards': '2', 'objects': 3670}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:01:10.153 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-10T16:01:10.153 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "43'8113", 'reported_seq': 8237, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:01:04.452045+0000', 'last_change': '2024-09-10T15:59:16.174113+0000', 'last_active': '2024-09-10T16:01:04.452045+0000', 'last_peered': '2024-09-10T16:01:04.452045+0000', 'last_clean': '2024-09-10T15:59:14.110654+0000', 'last_became_active': '2024-09-10T15:59:16.164645+0000', 'last_became_peered': '2024-09-10T15:59:16.164645+0000', 'last_unstale': '2024-09-10T16:01:04.452045+0000', 'last_undegraded': '2024-09-10T15:59:15.803593+0000', 'last_fullsized': '2024-09-10T15:59:15.769083+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2486", 'last_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'objects_scrubbed': 2440, 'log_size': 8113, 'log_dups_size': 0, 'ondisk_log_size': 8113, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 792, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33206272, 'num_objects': 8107, 'num_object_clones': 0, 'num_object_copies': 16214, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4354, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8107, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5667, 'num_write_kb': 22668, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4352}, {'shards': '2', 'objects': 3755}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:01:10.153 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-10T16:01:10.153 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "43'8038", 'reported_seq': 8168, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:01:06.803268+0000', 'last_change': '2024-09-10T15:59:16.315641+0000', 'last_active': '2024-09-10T16:01:06.803268+0000', 'last_peered': '2024-09-10T16:01:06.803268+0000', 'last_clean': '2024-09-10T15:59:14.117001+0000', 'last_became_active': '2024-09-10T15:59:16.302245+0000', 'last_became_peered': '2024-09-10T15:59:16.302245+0000', 'last_unstale': '2024-09-10T16:01:06.803268+0000', 'last_undegraded': '2024-09-10T15:59:15.845012+0000', 'last_fullsized': '2024-09-10T15:59:15.770337+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3404", 'last_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'objects_scrubbed': 3367, 'log_size': 8038, 'log_dups_size': 0, 'ondisk_log_size': 8038, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 983, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32915456, 'num_objects': 8036, 'num_object_clones': 0, 'num_object_copies': 16072, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8036, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4669, 'num_write_kb': 18676, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3732}, {'shards': '0,2', 'objects': 4304}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:01:10.153 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-10T16:01:10.153 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "43'7921", 'reported_seq': 8058, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:01:06.866028+0000', 'last_change': '2024-09-10T15:59:16.251610+0000', 'last_active': '2024-09-10T16:01:06.866028+0000', 'last_peered': '2024-09-10T16:01:06.866028+0000', 'last_clean': '2024-09-10T15:59:14.115595+0000', 'last_became_active': '2024-09-10T15:59:16.244089+0000', 'last_became_peered': '2024-09-10T15:59:16.244089+0000', 'last_unstale': '2024-09-10T16:01:06.866028+0000', 'last_undegraded': '2024-09-10T15:59:15.847127+0000', 'last_fullsized': '2024-09-10T15:59:15.774437+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3653", 'last_scrub_stamp': '2024-09-10T15:59:05.298468+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:05.298468+0000', 'objects_scrubbed': 3602, 'log_size': 7921, 'log_dups_size': 0, 'ondisk_log_size': 7921, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1289, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32411648, 'num_objects': 7913, 'num_object_clones': 0, 'num_object_copies': 15826, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4206, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7913, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4311, 'num_write_kb': 17244, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3710}, {'shards': '0,2', 'objects': 4203}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:01:10.154 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-10T16:01:10.154 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "43'8141", 'reported_seq': 8278, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:01:06.844139+0000', 'last_change': '2024-09-10T15:59:16.311291+0000', 'last_active': '2024-09-10T16:01:06.844139+0000', 'last_peered': '2024-09-10T16:01:06.844139+0000', 'last_clean': '2024-09-10T15:59:14.116363+0000', 'last_became_active': '2024-09-10T15:59:16.301244+0000', 'last_became_peered': '2024-09-10T15:59:16.301244+0000', 'last_unstale': '2024-09-10T16:01:06.844139+0000', 'last_undegraded': '2024-09-10T15:59:15.883434+0000', 'last_fullsized': '2024-09-10T15:59:15.772756+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3789", 'last_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'objects_scrubbed': 3741, 'log_size': 8141, 'log_dups_size': 0, 'ondisk_log_size': 8141, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1440, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33329152, 'num_objects': 8137, 'num_object_clones': 0, 'num_object_copies': 16274, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4373, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8137, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4396, 'num_write_kb': 17584, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3766}, {'shards': '0,2', 'objects': 4371}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:01:10.154 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-10T16:01:10.154 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "43'8148", 'reported_seq': 8287, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:01:06.811628+0000', 'last_change': '2024-09-10T15:59:16.313409+0000', 'last_active': '2024-09-10T16:01:06.811628+0000', 'last_peered': '2024-09-10T16:01:06.811628+0000', 'last_clean': '2024-09-10T15:59:14.129780+0000', 'last_became_active': '2024-09-10T15:59:16.300932+0000', 'last_became_peered': '2024-09-10T15:59:16.300932+0000', 'last_unstale': '2024-09-10T16:01:06.811628+0000', 'last_undegraded': '2024-09-10T15:59:15.887823+0000', 'last_fullsized': '2024-09-10T15:59:15.768513+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3880", 'last_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'objects_scrubbed': 3827, 'log_size': 8148, 'log_dups_size': 0, 'ondisk_log_size': 8148, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1397, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33326022, 'num_objects': 8138, 'num_object_clones': 0, 'num_object_copies': 16276, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4325, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8138, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4311, 'num_write_kb': 17244, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3815}, {'shards': '0,2', 'objects': 4323}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:01:10.154 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T16:01:10.861 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:01:10.861 INFO:teuthology.orchestra.run.smithi115.stdout:{"fsid":"7effee29-df05-4351-9741-ee203564ed81","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 47117/256614 objects degraded (18.361%), 11 pgs degraded, 12 pgs undersized","count":23},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":234,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":43,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983954,"num_in_osds":3,"osd_in_since":1725983947,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":128307,"data_bytes":525501169,"bytes_used":1781669888,"bytes_avail":288128622592,"bytes_total":289910292480,"degraded_objects":47117,"degraded_total":256614,"degraded_ratio":0.18361040317363822,"misplaced_objects":1797,"misplaced_total":256614,"misplaced_ratio":0.007002735626271365,"recovering_objects_per_sec":22,"recovering_bytes_per_sec":91103,"recovering_keys_per_sec":0,"num_objects_recovered":267,"num_bytes_recovered":1093632,"num_keys_recovered":0,"write_bytes_sec":1670905,"read_op_per_sec":0,"write_op_per_sec":407},"fsmap":{"epoch":1,"btime":"2024-09-10T15:57:14:967120+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T16:00:13.560776+0000","services":{}},"progress_events":{}} 2024-09-10T16:01:10.883 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-10T16:01:10.883 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:01:10.984 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 158 20 130674 130654 3.22968 1.73438 0.0270198 0.0241816 2024-09-10T16:01:11.445 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:01:11.448 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:01:11.471 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":153,"stamp":"2024-09-10T16:01:09.581213+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":525501169,"num_objects":128307,"num_object_clones":0,"num_object_copies":256614,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":47117,"num_objects_misplaced":1797,"num_objects_unfound":0,"num_objects_dirty":128307,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":86153,"num_write_kb":344606,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2477,"num_bytes_recovered":10133523,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128380,"ondisk_log_size":128380,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1739912,"kb_used_data":1040980,"kb_used_omap":12,"kb_used_meta":698867,"kb_avail":281375608,"statfs":{"total":289910292480,"available":288128622592,"internally_reserved":0,"allocated":1065963520,"data_stored":1065078730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":715639810},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,0,4,11,9,9,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":20058112,"num_objects":4897,"num_object_clones":0,"num_object_copies":9794,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-267,"num_objects_unfound":0,"num_objects_dirty":4897,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4897,"num_write_kb":19588,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":267,"num_bytes_recovered":1093632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004334"},"pg_stats":[{"pgid":"2.f","version":"43'8003","reported_seq":8073,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:06.852417+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:01:06.852417+0000","last_peered":"2024-09-10T16:01:06.852417+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:01:06.852417+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":8003,"log_dups_size":0,"ondisk_log_size":8003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32780288,"num_objects":8003,"num_object_clones":0,"num_object_copies":16006,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8003,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8003,"num_write_kb":32012,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3744},{"shards":"0,2","objects":4258}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'7949","reported_seq":8056,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:01:04.467097+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:01:04.467097+0000","last_peered":"2024-09-10T16:01:04.467097+0000","last_clean":"2024-09-10T16:01:04.467097+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:01:04.467097+0000","last_undegraded":"2024-09-10T16:01:04.467097+0000","last_fullsized":"2024-09-10T16:01:04.467097+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":7949,"log_dups_size":0,"ondisk_log_size":7949,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32535494,"num_objects":7945,"num_object_clones":0,"num_object_copies":15890,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7945,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":6054,"num_write_kb":24216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'7933","reported_seq":13003,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T16:01:04.542330+0000","last_change":"2024-09-10T15:59:16.142612+0000","last_active":"2024-09-10T16:01:04.542330+0000","last_peered":"2024-09-10T16:01:04.542330+0000","last_clean":"2024-09-10T15:59:14.092473+0000","last_became_active":"2024-09-10T15:59:16.109611+0000","last_became_peered":"2024-09-10T15:59:16.109611+0000","last_unstale":"2024-09-10T16:01:04.542330+0000","last_undegraded":"2024-09-10T16:01:04.542330+0000","last_fullsized":"2024-09-10T15:59:15.770264+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":7933,"log_dups_size":0,"ondisk_log_size":7933,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32477184,"num_objects":7929,"num_object_clones":0,"num_object_copies":15858,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1797,"num_objects_unfound":0,"num_objects_dirty":7929,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5937,"num_write_kb":23748,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2472,"num_bytes_recovered":10121216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1797},{"shards":"2","objects":6132}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'7919","reported_seq":7995,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:01:04.565095+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:01:04.565095+0000","last_peered":"2024-09-10T16:01:04.565095+0000","last_clean":"2024-09-10T16:01:04.565095+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:01:04.565095+0000","last_undegraded":"2024-09-10T16:01:04.565095+0000","last_fullsized":"2024-09-10T16:01:04.565095+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":7919,"log_dups_size":0,"ondisk_log_size":7919,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32436224,"num_objects":7919,"num_object_clones":0,"num_object_copies":15838,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7919,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7919,"num_write_kb":31676,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'8096","reported_seq":8221,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:01:06.820217+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:01:06.820217+0000","last_peered":"2024-09-10T16:01:06.820217+0000","last_clean":"2024-09-10T16:01:06.820217+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:01:06.820217+0000","last_undegraded":"2024-09-10T16:01:06.820217+0000","last_fullsized":"2024-09-10T16:01:06.820217+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":8096,"log_dups_size":0,"ondisk_log_size":8096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33133027,"num_objects":8090,"num_object_clones":0,"num_object_copies":16180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8090,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4807,"num_write_kb":19228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'8121","reported_seq":8254,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:06.759603+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:01:06.759603+0000","last_peered":"2024-09-10T16:01:06.759603+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:01:06.759603+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":8121,"log_dups_size":0,"ondisk_log_size":8121,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33243619,"num_objects":8117,"num_object_clones":0,"num_object_copies":16234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8117,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4733,"num_write_kb":18932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3746},{"shards":"0,2","objects":4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'8105","reported_seq":8238,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:06.761710+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:01:06.761710+0000","last_peered":"2024-09-10T16:01:06.761710+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:01:06.761710+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":8105,"log_dups_size":0,"ondisk_log_size":8105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33174470,"num_objects":8101,"num_object_clones":0,"num_object_copies":16202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8101,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4604,"num_write_kb":18416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3864},{"shards":"0,2","objects":4237}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'8214","reported_seq":8339,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:01:06.751291+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:01:06.751291+0000","last_peered":"2024-09-10T16:01:06.751291+0000","last_clean":"2024-09-10T16:01:06.751291+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:01:06.751291+0000","last_undegraded":"2024-09-10T16:01:06.751291+0000","last_fullsized":"2024-09-10T16:01:06.751291+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":8214,"log_dups_size":0,"ondisk_log_size":8214,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33636352,"num_objects":8212,"num_object_clones":0,"num_object_copies":16424,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8212,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5061,"num_write_kb":20244,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.775865+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T15:59:15.775865+0000","last_peered":"2024-09-10T15:59:15.775865+0000","last_clean":"2024-09-10T15:59:15.775865+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T15:59:15.775865+0000","last_undegraded":"2024-09-10T15:59:15.775865+0000","last_fullsized":"2024-09-10T15:59:15.775865+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'7649","reported_seq":7784,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:04.489946+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:01:04.489946+0000","last_peered":"2024-09-10T16:01:04.489946+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:01:04.489946+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":7649,"log_dups_size":0,"ondisk_log_size":7649,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31302115,"num_objects":7643,"num_object_clones":0,"num_object_copies":15286,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7643,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5468,"num_write_kb":21872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4123},{"shards":"2","objects":3520}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":89,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.772619+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T15:59:15.772619+0000","last_peered":"2024-09-10T15:59:15.772619+0000","last_clean":"2024-09-10T15:59:15.772619+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T15:59:15.772619+0000","last_undegraded":"2024-09-10T15:59:15.772619+0000","last_fullsized":"2024-09-10T15:59:15.772619+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'8049","reported_seq":8173,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:06.845337+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:01:06.845337+0000","last_peered":"2024-09-10T16:01:06.845337+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:01:06.845337+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":8049,"log_dups_size":0,"ondisk_log_size":8049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32935936,"num_objects":8041,"num_object_clones":0,"num_object_copies":16082,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8041,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4553,"num_write_kb":18212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3786},{"shards":"0,2","objects":4255}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":99,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.884297+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T15:59:15.884297+0000","last_peered":"2024-09-10T15:59:15.884297+0000","last_clean":"2024-09-10T15:59:15.884297+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T15:59:15.884297+0000","last_undegraded":"2024-09-10T15:59:15.884297+0000","last_fullsized":"2024-09-10T15:59:15.884297+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'7978","reported_seq":8108,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:04.436700+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:01:04.436700+0000","last_peered":"2024-09-10T16:01:04.436700+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:01:04.436700+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":7978,"log_dups_size":0,"ondisk_log_size":7978,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32657891,"num_objects":7974,"num_object_clones":0,"num_object_copies":15948,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7974,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5658,"num_write_kb":22632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4304},{"shards":"2","objects":3670}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.837509+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T15:59:15.837509+0000","last_peered":"2024-09-10T15:59:15.837509+0000","last_clean":"2024-09-10T15:59:15.837509+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T15:59:15.837509+0000","last_undegraded":"2024-09-10T15:59:15.837509+0000","last_fullsized":"2024-09-10T15:59:15.837509+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'8113","reported_seq":8237,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:04.452045+0000","last_change":"2024-09-10T15:59:16.174113+0000","last_active":"2024-09-10T16:01:04.452045+0000","last_peered":"2024-09-10T16:01:04.452045+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:01:04.452045+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":8113,"log_dups_size":0,"ondisk_log_size":8113,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33206272,"num_objects":8107,"num_object_clones":0,"num_object_copies":16214,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4354,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8107,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5667,"num_write_kb":22668,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4352},{"shards":"2","objects":3755}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'8038","reported_seq":8168,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:06.803268+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:01:06.803268+0000","last_peered":"2024-09-10T16:01:06.803268+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:01:06.803268+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":8038,"log_dups_size":0,"ondisk_log_size":8038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32915456,"num_objects":8036,"num_object_clones":0,"num_object_copies":16072,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8036,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4669,"num_write_kb":18676,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3732},{"shards":"0,2","objects":4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.989413+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T15:59:15.989413+0000","last_peered":"2024-09-10T15:59:15.989413+0000","last_clean":"2024-09-10T15:59:15.989413+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T15:59:15.989413+0000","last_undegraded":"2024-09-10T15:59:15.989413+0000","last_fullsized":"2024-09-10T15:59:15.989413+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'7921","reported_seq":8058,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:06.866028+0000","last_change":"2024-09-10T15:59:16.251610+0000","last_active":"2024-09-10T16:01:06.866028+0000","last_peered":"2024-09-10T16:01:06.866028+0000","last_clean":"2024-09-10T15:59:14.115595+0000","last_became_active":"2024-09-10T15:59:16.244089+0000","last_became_peered":"2024-09-10T15:59:16.244089+0000","last_unstale":"2024-09-10T16:01:06.866028+0000","last_undegraded":"2024-09-10T15:59:15.847127+0000","last_fullsized":"2024-09-10T15:59:15.774437+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":7921,"log_dups_size":0,"ondisk_log_size":7921,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32411648,"num_objects":7913,"num_object_clones":0,"num_object_copies":15826,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7913,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4311,"num_write_kb":17244,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3710},{"shards":"0,2","objects":4203}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":89,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.773633+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T15:59:15.773633+0000","last_peered":"2024-09-10T15:59:15.773633+0000","last_clean":"2024-09-10T15:59:15.773633+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T15:59:15.773633+0000","last_undegraded":"2024-09-10T15:59:15.773633+0000","last_fullsized":"2024-09-10T15:59:15.773633+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'8141","reported_seq":8278,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:06.844139+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:01:06.844139+0000","last_peered":"2024-09-10T16:01:06.844139+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:01:06.844139+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":8141,"log_dups_size":0,"ondisk_log_size":8141,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33329152,"num_objects":8137,"num_object_clones":0,"num_object_copies":16274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8137,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4396,"num_write_kb":17584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3766},{"shards":"0,2","objects":4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.916017+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T15:59:15.916017+0000","last_peered":"2024-09-10T15:59:15.916017+0000","last_clean":"2024-09-10T15:59:15.916017+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T15:59:15.916017+0000","last_undegraded":"2024-09-10T15:59:15.916017+0000","last_fullsized":"2024-09-10T15:59:15.916017+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.776033+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T15:59:15.776033+0000","last_peered":"2024-09-10T15:59:15.776033+0000","last_clean":"2024-09-10T15:59:15.776033+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T15:59:15.776033+0000","last_undegraded":"2024-09-10T15:59:15.776033+0000","last_fullsized":"2024-09-10T15:59:15.776033+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'8148","reported_seq":8287,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:06.811628+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:01:06.811628+0000","last_peered":"2024-09-10T16:01:06.811628+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:01:06.811628+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":8148,"log_dups_size":0,"ondisk_log_size":8148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33326022,"num_objects":8138,"num_object_clones":0,"num_object_copies":16276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8138,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4311,"num_write_kb":17244,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3815},{"shards":"0,2","objects":4323}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":525501150,"num_objects":128305,"num_object_clones":0,"num_object_copies":256610,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":47117,"num_objects_misplaced":1797,"num_objects_unfound":0,"num_objects_dirty":128305,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":86151,"num_write_kb":344604,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2475,"num_bytes_recovered":10133504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1066409984,"data_stored":1066258238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128377,"ondisk_log_size":128377,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986213,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":648612,"kb_used_data":392020,"kb_used_omap":4,"kb_used_meta":256571,"kb_avail":93723228,"statfs":{"total":96636764160,"available":95972585472,"internally_reserved":0,"allocated":401428480,"data_stored":401150720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":262729386},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,1,1,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659160,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":348152,"kb_used_data":194276,"kb_used_omap":4,"kb_used_meta":153851,"kb_avail":94023688,"statfs":{"total":96636764160,"available":96280256512,"internally_reserved":0,"allocated":198938624,"data_stored":198613431,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":157544110},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,7,3,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182442,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":743148,"kb_used_data":454684,"kb_used_omap":4,"kb_used_meta":288443,"kb_avail":93628692,"statfs":{"total":96636764160,"available":95875780608,"internally_reserved":0,"allocated":465596416,"data_stored":465314579,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":295366314},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,0,2,3,5,3,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":466403328,"data_stored":466367198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":198615040,"data_stored":198535554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":401391616,"data_stored":401355486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:01:11.985 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 159 20 131149 131129 3.22104 1.85547 0.0433635 0.0242478 2024-09-10T16:01:12.985 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout:2024-09-10T16:01:12.984414+0000 min lat: 0.00225581 max lat: 2.21736 avg lat: 0.024306 2024-09-10T16:01:12.985 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-10T16:01:12.985 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 160 20 131645 131625 3.21302 1.9375 0.0410042 0.024306 2024-09-10T16:01:13.985 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 161 20 132167 132147 3.20572 2.03906 0.0496955 0.0243647 2024-09-10T16:01:14.473 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:01:14.985 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 162 20 132634 132614 3.1972 1.82422 0.0358275 0.0244287 2024-09-10T16:01:15.021 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:01:15.022 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:01:15.044 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":155,"stamp":"2024-09-10T16:01:13.585100+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":535372529,"num_objects":130717,"num_object_clones":0,"num_object_copies":261434,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":47117,"num_objects_misplaced":1669,"num_objects_unfound":0,"num_objects_dirty":130717,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":88563,"num_write_kb":354246,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2605,"num_bytes_recovered":10657811,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":130790,"ondisk_log_size":130790,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1747856,"kb_used_data":1053084,"kb_used_omap":12,"kb_used_meta":694707,"kb_avail":281367664,"statfs":{"total":289910292480,"available":288120487936,"internally_reserved":0,"allocated":1078358016,"data_stored":1077473226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":711379970},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,0,16,11,9],"upper_bound":128},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":25960448,"num_objects":6338,"num_object_clones":0,"num_object_copies":12676,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-261,"num_objects_unfound":0,"num_objects_dirty":6338,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6338,"num_write_kb":25352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":261,"num_bytes_recovered":1069056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006600"},"pg_stats":[{"pgid":"2.f","version":"43'8136","reported_seq":8205,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:11.847939+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:01:11.847939+0000","last_peered":"2024-09-10T16:01:11.847939+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:01:11.847939+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":8136,"log_dups_size":0,"ondisk_log_size":8136,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33325056,"num_objects":8136,"num_object_clones":0,"num_object_copies":16272,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8136,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8136,"num_write_kb":32544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3877},{"shards":"0,2","objects":4258}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'8110","reported_seq":8218,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:01:09.548520+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:01:09.548520+0000","last_peered":"2024-09-10T16:01:09.548520+0000","last_clean":"2024-09-10T16:01:09.548520+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:01:09.548520+0000","last_undegraded":"2024-09-10T16:01:09.548520+0000","last_fullsized":"2024-09-10T16:01:09.548520+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":8110,"log_dups_size":0,"ondisk_log_size":8110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33194950,"num_objects":8106,"num_object_clones":0,"num_object_copies":16212,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8106,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":6215,"num_write_kb":24860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'8070","reported_seq":13395,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T16:01:09.541925+0000","last_change":"2024-09-10T15:59:16.142612+0000","last_active":"2024-09-10T16:01:09.541925+0000","last_peered":"2024-09-10T16:01:09.541925+0000","last_clean":"2024-09-10T15:59:14.092473+0000","last_became_active":"2024-09-10T15:59:16.109611+0000","last_became_peered":"2024-09-10T15:59:16.109611+0000","last_unstale":"2024-09-10T16:01:09.541925+0000","last_undegraded":"2024-09-10T16:01:09.541925+0000","last_fullsized":"2024-09-10T15:59:15.770264+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":8070,"log_dups_size":0,"ondisk_log_size":8070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33038336,"num_objects":8066,"num_object_clones":0,"num_object_copies":16132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1669,"num_objects_unfound":0,"num_objects_dirty":8066,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6074,"num_write_kb":24296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2600,"num_bytes_recovered":10645504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1669},{"shards":"2","objects":6397}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'8081","reported_seq":8157,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:01:09.562190+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:01:09.562190+0000","last_peered":"2024-09-10T16:01:09.562190+0000","last_clean":"2024-09-10T16:01:09.562190+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:01:09.562190+0000","last_undegraded":"2024-09-10T16:01:09.562190+0000","last_fullsized":"2024-09-10T16:01:09.562190+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":8081,"log_dups_size":0,"ondisk_log_size":8081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33099776,"num_objects":8081,"num_object_clones":0,"num_object_copies":16162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8081,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8081,"num_write_kb":32324,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'8246","reported_seq":8371,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:01:11.855854+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:01:11.855854+0000","last_peered":"2024-09-10T16:01:11.855854+0000","last_clean":"2024-09-10T16:01:11.855854+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:01:11.855854+0000","last_undegraded":"2024-09-10T16:01:11.855854+0000","last_fullsized":"2024-09-10T16:01:11.855854+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":8246,"log_dups_size":0,"ondisk_log_size":8246,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33747427,"num_objects":8240,"num_object_clones":0,"num_object_copies":16480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8240,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4957,"num_write_kb":19828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'8272","reported_seq":8405,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:11.906093+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:01:11.906093+0000","last_peered":"2024-09-10T16:01:11.906093+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:01:11.906093+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":8272,"log_dups_size":0,"ondisk_log_size":8272,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33862115,"num_objects":8268,"num_object_clones":0,"num_object_copies":16536,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8268,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4884,"num_write_kb":19536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3897},{"shards":"0,2","objects":4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'8263","reported_seq":8396,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:11.834540+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:01:11.834540+0000","last_peered":"2024-09-10T16:01:11.834540+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:01:11.834540+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":8263,"log_dups_size":0,"ondisk_log_size":8263,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33821638,"num_objects":8259,"num_object_clones":0,"num_object_copies":16518,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8259,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4762,"num_write_kb":19048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4022},{"shards":"0,2","objects":4237}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'8365","reported_seq":8490,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:01:11.869976+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:01:11.869976+0000","last_peered":"2024-09-10T16:01:11.869976+0000","last_clean":"2024-09-10T16:01:11.869976+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:01:11.869976+0000","last_undegraded":"2024-09-10T16:01:11.869976+0000","last_fullsized":"2024-09-10T16:01:11.869976+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":8365,"log_dups_size":0,"ondisk_log_size":8365,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34254848,"num_objects":8363,"num_object_clones":0,"num_object_copies":16726,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8363,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5212,"num_write_kb":20848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.775865+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T15:59:15.775865+0000","last_peered":"2024-09-10T15:59:15.775865+0000","last_clean":"2024-09-10T15:59:15.775865+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T15:59:15.775865+0000","last_undegraded":"2024-09-10T15:59:15.775865+0000","last_fullsized":"2024-09-10T15:59:15.775865+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'7807","reported_seq":7942,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:09.501102+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:01:09.501102+0000","last_peered":"2024-09-10T16:01:09.501102+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:01:09.501102+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":7807,"log_dups_size":0,"ondisk_log_size":7807,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31949283,"num_objects":7801,"num_object_clones":0,"num_object_copies":15602,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7801,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5626,"num_write_kb":22504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4123},{"shards":"2","objects":3678}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":89,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.772619+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T15:59:15.772619+0000","last_peered":"2024-09-10T15:59:15.772619+0000","last_clean":"2024-09-10T15:59:15.772619+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T15:59:15.772619+0000","last_undegraded":"2024-09-10T15:59:15.772619+0000","last_fullsized":"2024-09-10T15:59:15.772619+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'8192","reported_seq":8318,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:11.860693+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:01:11.860693+0000","last_peered":"2024-09-10T16:01:11.860693+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:01:11.860693+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":8192,"log_dups_size":0,"ondisk_log_size":8192,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33521664,"num_objects":8184,"num_object_clones":0,"num_object_copies":16368,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8184,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4696,"num_write_kb":18784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3929},{"shards":"0,2","objects":4255}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":99,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.884297+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T15:59:15.884297+0000","last_peered":"2024-09-10T15:59:15.884297+0000","last_clean":"2024-09-10T15:59:15.884297+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T15:59:15.884297+0000","last_undegraded":"2024-09-10T15:59:15.884297+0000","last_fullsized":"2024-09-10T15:59:15.884297+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'8122","reported_seq":8252,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:09.574149+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:01:09.574149+0000","last_peered":"2024-09-10T16:01:09.574149+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:01:09.574149+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":8122,"log_dups_size":0,"ondisk_log_size":8122,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33247715,"num_objects":8118,"num_object_clones":0,"num_object_copies":16236,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8118,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5802,"num_write_kb":23208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4304},{"shards":"2","objects":3814}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.837509+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T15:59:15.837509+0000","last_peered":"2024-09-10T15:59:15.837509+0000","last_clean":"2024-09-10T15:59:15.837509+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T15:59:15.837509+0000","last_undegraded":"2024-09-10T15:59:15.837509+0000","last_fullsized":"2024-09-10T15:59:15.837509+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'8249","reported_seq":8373,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:09.578787+0000","last_change":"2024-09-10T15:59:16.174113+0000","last_active":"2024-09-10T16:01:09.578787+0000","last_peered":"2024-09-10T16:01:09.578787+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:01:09.578787+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":8249,"log_dups_size":0,"ondisk_log_size":8249,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33763328,"num_objects":8243,"num_object_clones":0,"num_object_copies":16486,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4354,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8243,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5803,"num_write_kb":23212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4352},{"shards":"2","objects":3891}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'8191","reported_seq":8321,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:11.859364+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:01:11.859364+0000","last_peered":"2024-09-10T16:01:11.859364+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:01:11.859364+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":8191,"log_dups_size":0,"ondisk_log_size":8191,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33542144,"num_objects":8189,"num_object_clones":0,"num_object_copies":16378,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8189,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4822,"num_write_kb":19288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3885},{"shards":"0,2","objects":4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.989413+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T15:59:15.989413+0000","last_peered":"2024-09-10T15:59:15.989413+0000","last_clean":"2024-09-10T15:59:15.989413+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T15:59:15.989413+0000","last_undegraded":"2024-09-10T15:59:15.989413+0000","last_fullsized":"2024-09-10T15:59:15.989413+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'8073","reported_seq":8211,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:11.837481+0000","last_change":"2024-09-10T15:59:16.251610+0000","last_active":"2024-09-10T16:01:11.837481+0000","last_peered":"2024-09-10T16:01:11.837481+0000","last_clean":"2024-09-10T15:59:14.115595+0000","last_became_active":"2024-09-10T15:59:16.244089+0000","last_became_peered":"2024-09-10T15:59:16.244089+0000","last_unstale":"2024-09-10T16:01:11.837481+0000","last_undegraded":"2024-09-10T15:59:15.847127+0000","last_fullsized":"2024-09-10T15:59:15.774437+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":8073,"log_dups_size":0,"ondisk_log_size":8073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33034240,"num_objects":8065,"num_object_clones":0,"num_object_copies":16130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8065,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4463,"num_write_kb":17852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3862},{"shards":"0,2","objects":4203}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":89,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.773633+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T15:59:15.773633+0000","last_peered":"2024-09-10T15:59:15.773633+0000","last_clean":"2024-09-10T15:59:15.773633+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T15:59:15.773633+0000","last_undegraded":"2024-09-10T15:59:15.773633+0000","last_fullsized":"2024-09-10T15:59:15.773633+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'8307","reported_seq":8440,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:11.885933+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:01:11.885933+0000","last_peered":"2024-09-10T16:01:11.885933+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:01:11.885933+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":8307,"log_dups_size":0,"ondisk_log_size":8307,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34009088,"num_objects":8303,"num_object_clones":0,"num_object_copies":16606,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8303,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4562,"num_write_kb":18248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3932},{"shards":"0,2","objects":4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.916017+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T15:59:15.916017+0000","last_peered":"2024-09-10T15:59:15.916017+0000","last_clean":"2024-09-10T15:59:15.916017+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T15:59:15.916017+0000","last_undegraded":"2024-09-10T15:59:15.916017+0000","last_fullsized":"2024-09-10T15:59:15.916017+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.776033+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T15:59:15.776033+0000","last_peered":"2024-09-10T15:59:15.776033+0000","last_clean":"2024-09-10T15:59:15.776033+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T15:59:15.776033+0000","last_undegraded":"2024-09-10T15:59:15.776033+0000","last_fullsized":"2024-09-10T15:59:15.776033+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'8303","reported_seq":8443,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:11.920787+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:01:11.920787+0000","last_peered":"2024-09-10T16:01:11.920787+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:01:11.920787+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":8303,"log_dups_size":0,"ondisk_log_size":8303,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33960902,"num_objects":8293,"num_object_clones":0,"num_object_copies":16586,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8293,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4466,"num_write_kb":17864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3970},{"shards":"0,2","objects":4323}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":535372510,"num_objects":130715,"num_object_clones":0,"num_object_copies":261430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":47117,"num_objects_misplaced":1669,"num_objects_unfound":0,"num_objects_dirty":130715,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":88561,"num_write_kb":354244,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2603,"num_bytes_recovered":10657792,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1078812672,"data_stored":1078660926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":130787,"ondisk_log_size":130787,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986214,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":641012,"kb_used_data":396900,"kb_used_omap":4,"kb_used_meta":244091,"kb_avail":93730828,"statfs":{"total":96636764160,"available":95980367872,"internally_reserved":0,"allocated":406425600,"data_stored":406147840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":249949866},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,8,3,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659160,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":348152,"kb_used_data":194276,"kb_used_omap":4,"kb_used_meta":153851,"kb_avail":94023688,"statfs":{"total":96636764160,"available":96280256512,"internally_reserved":0,"allocated":198938624,"data_stored":198613431,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":157544110},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,7,3,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182443,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":758692,"kb_used_data":461908,"kb_used_omap":4,"kb_used_meta":296763,"kb_avail":93613148,"statfs":{"total":96636764160,"available":95859863552,"internally_reserved":0,"allocated":472993792,"data_stored":472711955,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":303885994},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,0,1,5,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":473784320,"data_stored":473748190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":198615040,"data_stored":198535554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":406413312,"data_stored":406377182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:01:15.046 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:01:15.591 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:01:15.594 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:01:15.616 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":156,"stamp":"2024-09-10T16:01:15.585867+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":538993393,"num_objects":131601,"num_object_clones":0,"num_object_copies":263202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":47117,"num_objects_misplaced":1539,"num_objects_unfound":0,"num_objects_dirty":131601,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":89447,"num_write_kb":357782,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2735,"num_bytes_recovered":11190291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":131674,"ondisk_log_size":131674,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1768696,"kb_used_data":1065608,"kb_used_omap":12,"kb_used_meta":703027,"kb_avail":281346824,"statfs":{"total":289910292480,"available":288099147776,"internally_reserved":0,"allocated":1091182592,"data_stored":1090297802,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":719899650},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,1,2,4,8,12,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":23494656,"num_objects":5736,"num_object_clones":0,"num_object_copies":11472,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-391,"num_objects_unfound":0,"num_objects_dirty":5736,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5736,"num_write_kb":22944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":391,"num_bytes_recovered":1601536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006635"},"pg_stats":[{"pgid":"2.f","version":"43'8136","reported_seq":8205,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:11.847939+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:01:11.847939+0000","last_peered":"2024-09-10T16:01:11.847939+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:01:11.847939+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":8136,"log_dups_size":0,"ondisk_log_size":8136,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33325056,"num_objects":8136,"num_object_clones":0,"num_object_copies":16272,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8136,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8136,"num_write_kb":32544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3877},{"shards":"0,2","objects":4258}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'8268","reported_seq":8376,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:01:14.462495+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:01:14.462495+0000","last_peered":"2024-09-10T16:01:14.462495+0000","last_clean":"2024-09-10T16:01:14.462495+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:01:14.462495+0000","last_undegraded":"2024-09-10T16:01:14.462495+0000","last_fullsized":"2024-09-10T16:01:14.462495+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":8268,"log_dups_size":0,"ondisk_log_size":8268,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33842118,"num_objects":8264,"num_object_clones":0,"num_object_copies":16528,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8264,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":6373,"num_write_kb":25492,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'8220","reported_seq":13806,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T16:01:14.538881+0000","last_change":"2024-09-10T15:59:16.142612+0000","last_active":"2024-09-10T16:01:14.538881+0000","last_peered":"2024-09-10T16:01:14.538881+0000","last_clean":"2024-09-10T15:59:14.092473+0000","last_became_active":"2024-09-10T15:59:16.109611+0000","last_became_peered":"2024-09-10T15:59:16.109611+0000","last_unstale":"2024-09-10T16:01:14.538881+0000","last_undegraded":"2024-09-10T16:01:14.538881+0000","last_fullsized":"2024-09-10T15:59:15.770264+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":8220,"log_dups_size":0,"ondisk_log_size":8220,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33652736,"num_objects":8216,"num_object_clones":0,"num_object_copies":16432,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1539,"num_objects_unfound":0,"num_objects_dirty":8216,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6224,"num_write_kb":24896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2730,"num_bytes_recovered":11177984,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1539},{"shards":"2","objects":6677}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'8221","reported_seq":8297,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:01:14.492072+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:01:14.492072+0000","last_peered":"2024-09-10T16:01:14.492072+0000","last_clean":"2024-09-10T16:01:14.492072+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:01:14.492072+0000","last_undegraded":"2024-09-10T16:01:14.492072+0000","last_fullsized":"2024-09-10T16:01:14.492072+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":8221,"log_dups_size":0,"ondisk_log_size":8221,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33673216,"num_objects":8221,"num_object_clones":0,"num_object_copies":16442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8221,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8221,"num_write_kb":32884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'8246","reported_seq":8371,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:01:11.855854+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:01:11.855854+0000","last_peered":"2024-09-10T16:01:11.855854+0000","last_clean":"2024-09-10T16:01:11.855854+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:01:11.855854+0000","last_undegraded":"2024-09-10T16:01:11.855854+0000","last_fullsized":"2024-09-10T16:01:11.855854+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":8246,"log_dups_size":0,"ondisk_log_size":8246,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33747427,"num_objects":8240,"num_object_clones":0,"num_object_copies":16480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8240,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4957,"num_write_kb":19828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'8272","reported_seq":8405,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:11.906093+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:01:11.906093+0000","last_peered":"2024-09-10T16:01:11.906093+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:01:11.906093+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":8272,"log_dups_size":0,"ondisk_log_size":8272,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33862115,"num_objects":8268,"num_object_clones":0,"num_object_copies":16536,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8268,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4884,"num_write_kb":19536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3897},{"shards":"0,2","objects":4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'8263","reported_seq":8396,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:11.834540+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:01:11.834540+0000","last_peered":"2024-09-10T16:01:11.834540+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:01:11.834540+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":8263,"log_dups_size":0,"ondisk_log_size":8263,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33821638,"num_objects":8259,"num_object_clones":0,"num_object_copies":16518,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8259,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4762,"num_write_kb":19048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4022},{"shards":"0,2","objects":4237}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'8365","reported_seq":8490,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:01:11.869976+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:01:11.869976+0000","last_peered":"2024-09-10T16:01:11.869976+0000","last_clean":"2024-09-10T16:01:11.869976+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:01:11.869976+0000","last_undegraded":"2024-09-10T16:01:11.869976+0000","last_fullsized":"2024-09-10T16:01:11.869976+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":8365,"log_dups_size":0,"ondisk_log_size":8365,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34254848,"num_objects":8363,"num_object_clones":0,"num_object_copies":16726,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8363,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5212,"num_write_kb":20848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.777326+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T15:59:15.777326+0000","last_peered":"2024-09-10T15:59:15.777326+0000","last_clean":"2024-09-10T15:59:15.777326+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T15:59:15.777326+0000","last_undegraded":"2024-09-10T15:59:15.777326+0000","last_fullsized":"2024-09-10T15:59:15.777326+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'7969","reported_seq":8104,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:14.502406+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:01:14.502406+0000","last_peered":"2024-09-10T16:01:14.502406+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:01:14.502406+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":7969,"log_dups_size":0,"ondisk_log_size":7969,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32612835,"num_objects":7963,"num_object_clones":0,"num_object_copies":15926,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7963,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5788,"num_write_kb":23152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4123},{"shards":"2","objects":3840}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":89,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.772619+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T15:59:15.772619+0000","last_peered":"2024-09-10T15:59:15.772619+0000","last_clean":"2024-09-10T15:59:15.772619+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T15:59:15.772619+0000","last_undegraded":"2024-09-10T15:59:15.772619+0000","last_fullsized":"2024-09-10T15:59:15.772619+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'8192","reported_seq":8318,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:11.860693+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:01:11.860693+0000","last_peered":"2024-09-10T16:01:11.860693+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:01:11.860693+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":8192,"log_dups_size":0,"ondisk_log_size":8192,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33521664,"num_objects":8184,"num_object_clones":0,"num_object_copies":16368,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8184,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4696,"num_write_kb":18784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3929},{"shards":"0,2","objects":4255}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":99,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.884297+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T15:59:15.884297+0000","last_peered":"2024-09-10T15:59:15.884297+0000","last_clean":"2024-09-10T15:59:15.884297+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T15:59:15.884297+0000","last_undegraded":"2024-09-10T15:59:15.884297+0000","last_fullsized":"2024-09-10T15:59:15.884297+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'8247","reported_seq":8377,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:14.576432+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:01:14.576432+0000","last_peered":"2024-09-10T16:01:14.576432+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:01:14.576432+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":8247,"log_dups_size":0,"ondisk_log_size":8247,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33759715,"num_objects":8243,"num_object_clones":0,"num_object_copies":16486,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8243,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5927,"num_write_kb":23708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4304},{"shards":"2","objects":3939}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.837509+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T15:59:15.837509+0000","last_peered":"2024-09-10T15:59:15.837509+0000","last_clean":"2024-09-10T15:59:15.837509+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T15:59:15.837509+0000","last_undegraded":"2024-09-10T15:59:15.837509+0000","last_fullsized":"2024-09-10T15:59:15.837509+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'8398","reported_seq":8520,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:14.598066+0000","last_change":"2024-09-10T15:59:16.174113+0000","last_active":"2024-09-10T16:01:14.598066+0000","last_peered":"2024-09-10T16:01:14.598066+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:01:14.598066+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":8398,"log_dups_size":0,"ondisk_log_size":8398,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34373632,"num_objects":8392,"num_object_clones":0,"num_object_copies":16784,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4354,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8392,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5952,"num_write_kb":23808,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4352},{"shards":"2","objects":4040}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'8191","reported_seq":8321,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:11.859364+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:01:11.859364+0000","last_peered":"2024-09-10T16:01:11.859364+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:01:11.859364+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":8191,"log_dups_size":0,"ondisk_log_size":8191,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33542144,"num_objects":8189,"num_object_clones":0,"num_object_copies":16378,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8189,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4822,"num_write_kb":19288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3885},{"shards":"0,2","objects":4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.989413+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T15:59:15.989413+0000","last_peered":"2024-09-10T15:59:15.989413+0000","last_clean":"2024-09-10T15:59:15.989413+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T15:59:15.989413+0000","last_undegraded":"2024-09-10T15:59:15.989413+0000","last_fullsized":"2024-09-10T15:59:15.989413+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'8073","reported_seq":8211,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:11.837481+0000","last_change":"2024-09-10T15:59:16.251610+0000","last_active":"2024-09-10T16:01:11.837481+0000","last_peered":"2024-09-10T16:01:11.837481+0000","last_clean":"2024-09-10T15:59:14.115595+0000","last_became_active":"2024-09-10T15:59:16.244089+0000","last_became_peered":"2024-09-10T15:59:16.244089+0000","last_unstale":"2024-09-10T16:01:11.837481+0000","last_undegraded":"2024-09-10T15:59:15.847127+0000","last_fullsized":"2024-09-10T15:59:15.774437+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":8073,"log_dups_size":0,"ondisk_log_size":8073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33034240,"num_objects":8065,"num_object_clones":0,"num_object_copies":16130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8065,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4463,"num_write_kb":17852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3862},{"shards":"0,2","objects":4203}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":89,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.773633+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T15:59:15.773633+0000","last_peered":"2024-09-10T15:59:15.773633+0000","last_clean":"2024-09-10T15:59:15.773633+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T15:59:15.773633+0000","last_undegraded":"2024-09-10T15:59:15.773633+0000","last_fullsized":"2024-09-10T15:59:15.773633+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'8307","reported_seq":8440,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:11.885933+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:01:11.885933+0000","last_peered":"2024-09-10T16:01:11.885933+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:01:11.885933+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":8307,"log_dups_size":0,"ondisk_log_size":8307,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34009088,"num_objects":8303,"num_object_clones":0,"num_object_copies":16606,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8303,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4562,"num_write_kb":18248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3932},{"shards":"0,2","objects":4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.916017+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T15:59:15.916017+0000","last_peered":"2024-09-10T15:59:15.916017+0000","last_clean":"2024-09-10T15:59:15.916017+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T15:59:15.916017+0000","last_undegraded":"2024-09-10T15:59:15.916017+0000","last_fullsized":"2024-09-10T15:59:15.916017+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.776033+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T15:59:15.776033+0000","last_peered":"2024-09-10T15:59:15.776033+0000","last_clean":"2024-09-10T15:59:15.776033+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T15:59:15.776033+0000","last_undegraded":"2024-09-10T15:59:15.776033+0000","last_fullsized":"2024-09-10T15:59:15.776033+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'8303","reported_seq":8443,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:11.920787+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:01:11.920787+0000","last_peered":"2024-09-10T16:01:11.920787+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:01:11.920787+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":8303,"log_dups_size":0,"ondisk_log_size":8303,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33960902,"num_objects":8293,"num_object_clones":0,"num_object_copies":16586,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8293,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4466,"num_write_kb":17864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3970},{"shards":"0,2","objects":4323}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":538993374,"num_objects":131599,"num_object_clones":0,"num_object_copies":263198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":47117,"num_objects_misplaced":1539,"num_objects_unfound":0,"num_objects_dirty":131599,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":89445,"num_write_kb":357780,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2733,"num_bytes_recovered":11190272,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1091518464,"data_stored":1091366718,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":131671,"ondisk_log_size":131671,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986215,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":649936,"kb_used_data":401664,"kb_used_omap":4,"kb_used_meta":248251,"kb_avail":93721904,"statfs":{"total":96636764160,"available":95971229696,"internally_reserved":0,"allocated":411303936,"data_stored":411026176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":254209706},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,1,0,2,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659161,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":360068,"kb_used_data":202036,"kb_used_omap":4,"kb_used_meta":158011,"kb_avail":94011772,"statfs":{"total":96636764160,"available":96268054528,"internally_reserved":0,"allocated":206884864,"data_stored":206559671,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":161803950},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,3,1,7,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182443,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":758692,"kb_used_data":461908,"kb_used_omap":4,"kb_used_meta":296763,"kb_avail":93613148,"statfs":{"total":96636764160,"available":95859863552,"internally_reserved":0,"allocated":472993792,"data_stored":472711955,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":303885994},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,0,1,5,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":473784320,"data_stored":473748190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":206524416,"data_stored":206444930,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":411209728,"data_stored":411173598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:01:15.618 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-10T16:01:15.618 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "43'8136", 'reported_seq': 8205, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:01:11.847939+0000', 'last_change': '2024-09-10T15:59:16.281558+0000', 'last_active': '2024-09-10T16:01:11.847939+0000', 'last_peered': '2024-09-10T16:01:11.847939+0000', 'last_clean': '2024-09-10T15:59:14.132456+0000', 'last_became_active': '2024-09-10T15:59:16.274407+0000', 'last_became_peered': '2024-09-10T15:59:16.274407+0000', 'last_unstale': '2024-09-10T16:01:11.847939+0000', 'last_undegraded': '2024-09-10T15:59:15.851279+0000', 'last_fullsized': '2024-09-10T15:59:15.774356+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'objects_scrubbed': 0, 'log_size': 8136, 'log_dups_size': 0, 'ondisk_log_size': 8136, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33325056, 'num_objects': 8136, 'num_object_clones': 0, 'num_object_copies': 16272, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4259, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8136, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8136, 'num_write_kb': 32544, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3877}, {'shards': '0,2', 'objects': 4258}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:01:15.619 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-10T16:01:15.619 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "43'8220", 'reported_seq': 13806, 'reported_epoch': 43, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-10T16:01:14.538881+0000', 'last_change': '2024-09-10T15:59:16.142612+0000', 'last_active': '2024-09-10T16:01:14.538881+0000', 'last_peered': '2024-09-10T16:01:14.538881+0000', 'last_clean': '2024-09-10T15:59:14.092473+0000', 'last_became_active': '2024-09-10T15:59:16.109611+0000', 'last_became_peered': '2024-09-10T15:59:16.109611+0000', 'last_unstale': '2024-09-10T16:01:14.538881+0000', 'last_undegraded': '2024-09-10T16:01:14.538881+0000', 'last_fullsized': '2024-09-10T15:59:15.770264+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2029", 'last_scrub_stamp': '2024-09-10T15:58:45.933789+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:45.933789+0000', 'objects_scrubbed': 1992, 'log_size': 8220, 'log_dups_size': 0, 'ondisk_log_size': 8220, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 668, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33652736, 'num_objects': 8216, 'num_object_clones': 0, 'num_object_copies': 16432, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1539, 'num_objects_unfound': 0, 'num_objects_dirty': 8216, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6224, 'num_write_kb': 24896, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2730, 'num_bytes_recovered': 11177984, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1539}, {'shards': '2', 'objects': 6677}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:01:15.619 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-10T16:01:15.619 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "43'8272", 'reported_seq': 8405, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:01:11.906093+0000', 'last_change': '2024-09-10T15:59:16.308264+0000', 'last_active': '2024-09-10T16:01:11.906093+0000', 'last_peered': '2024-09-10T16:01:11.906093+0000', 'last_clean': '2024-09-10T15:59:14.111551+0000', 'last_became_active': '2024-09-10T15:59:16.300040+0000', 'last_became_peered': '2024-09-10T15:59:16.300040+0000', 'last_unstale': '2024-09-10T16:01:11.906093+0000', 'last_undegraded': '2024-09-10T15:59:15.878234+0000', 'last_fullsized': '2024-09-10T15:59:15.769328+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3442", 'last_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'objects_scrubbed': 3384, 'log_size': 8272, 'log_dups_size': 0, 'ondisk_log_size': 8272, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1292, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33862115, 'num_objects': 8268, 'num_object_clones': 0, 'num_object_copies': 16536, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4372, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8268, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 4884, 'num_write_kb': 19536, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3897}, {'shards': '0,2', 'objects': 4371}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:01:15.619 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-10T16:01:15.620 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "43'8263", 'reported_seq': 8396, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:01:11.834540+0000', 'last_change': '2024-09-10T15:59:16.331899+0000', 'last_active': '2024-09-10T16:01:11.834540+0000', 'last_peered': '2024-09-10T16:01:11.834540+0000', 'last_clean': '2024-09-10T15:59:14.139436+0000', 'last_became_active': '2024-09-10T15:59:16.318715+0000', 'last_became_peered': '2024-09-10T15:59:16.318715+0000', 'last_unstale': '2024-09-10T16:01:11.834540+0000', 'last_undegraded': '2024-09-10T15:59:15.877524+0000', 'last_fullsized': '2024-09-10T15:59:15.769042+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3537", 'last_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'objects_scrubbed': 3497, 'log_size': 8263, 'log_dups_size': 0, 'ondisk_log_size': 8263, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1224, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33821638, 'num_objects': 8259, 'num_object_clones': 0, 'num_object_copies': 16518, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4237, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8259, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 4762, 'num_write_kb': 19048, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4022}, {'shards': '0,2', 'objects': 4237}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:01:15.620 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-10T16:01:15.620 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "43'7969", 'reported_seq': 8104, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:01:14.502406+0000', 'last_change': '2024-09-10T15:59:16.175680+0000', 'last_active': '2024-09-10T16:01:14.502406+0000', 'last_peered': '2024-09-10T16:01:14.502406+0000', 'last_clean': '2024-09-10T15:59:14.088187+0000', 'last_became_active': '2024-09-10T15:59:16.165810+0000', 'last_became_peered': '2024-09-10T15:59:16.165810+0000', 'last_unstale': '2024-09-10T16:01:14.502406+0000', 'last_undegraded': '2024-09-10T15:59:15.822098+0000', 'last_fullsized': '2024-09-10T15:59:15.769123+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2218", 'last_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'objects_scrubbed': 2175, 'log_size': 7969, 'log_dups_size': 0, 'ondisk_log_size': 7969, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 825, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32612835, 'num_objects': 7963, 'num_object_clones': 0, 'num_object_copies': 15926, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4124, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7963, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 5788, 'num_write_kb': 23152, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4123}, {'shards': '2', 'objects': 3840}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:01:15.620 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-10T16:01:15.620 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "43'8192", 'reported_seq': 8318, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:01:11.860693+0000', 'last_change': '2024-09-10T15:59:16.277742+0000', 'last_active': '2024-09-10T16:01:11.860693+0000', 'last_peered': '2024-09-10T16:01:11.860693+0000', 'last_clean': '2024-09-10T15:59:14.129081+0000', 'last_became_active': '2024-09-10T15:59:16.269835+0000', 'last_became_peered': '2024-09-10T15:59:16.269835+0000', 'last_unstale': '2024-09-10T16:01:11.860693+0000', 'last_undegraded': '2024-09-10T15:59:15.860857+0000', 'last_fullsized': '2024-09-10T15:59:15.771410+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3557", 'last_scrub_stamp': '2024-09-10T15:59:02.493678+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:02.493678+0000', 'objects_scrubbed': 3488, 'log_size': 8192, 'log_dups_size': 0, 'ondisk_log_size': 8192, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1472, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33521664, 'num_objects': 8184, 'num_object_clones': 0, 'num_object_copies': 16368, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4257, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8184, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4696, 'num_write_kb': 18784, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3929}, {'shards': '0,2', 'objects': 4255}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:01:15.621 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-10T16:01:15.621 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "43'8247", 'reported_seq': 8377, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:01:14.576432+0000', 'last_change': '2024-09-10T15:59:16.284311+0000', 'last_active': '2024-09-10T16:01:14.576432+0000', 'last_peered': '2024-09-10T16:01:14.576432+0000', 'last_clean': '2024-09-10T15:59:14.116041+0000', 'last_became_active': '2024-09-10T15:59:16.274543+0000', 'last_became_peered': '2024-09-10T15:59:16.274543+0000', 'last_unstale': '2024-09-10T16:01:14.576432+0000', 'last_undegraded': '2024-09-10T15:59:15.812484+0000', 'last_fullsized': '2024-09-10T15:59:15.770885+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2356", 'last_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'objects_scrubbed': 2316, 'log_size': 8247, 'log_dups_size': 0, 'ondisk_log_size': 8247, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 783, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33759715, 'num_objects': 8243, 'num_object_clones': 0, 'num_object_copies': 16486, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8243, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 5927, 'num_write_kb': 23708, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4304}, {'shards': '2', 'objects': 3939}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:01:15.621 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-10T16:01:15.621 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "43'8398", 'reported_seq': 8520, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:01:14.598066+0000', 'last_change': '2024-09-10T15:59:16.174113+0000', 'last_active': '2024-09-10T16:01:14.598066+0000', 'last_peered': '2024-09-10T16:01:14.598066+0000', 'last_clean': '2024-09-10T15:59:14.110654+0000', 'last_became_active': '2024-09-10T15:59:16.164645+0000', 'last_became_peered': '2024-09-10T15:59:16.164645+0000', 'last_unstale': '2024-09-10T16:01:14.598066+0000', 'last_undegraded': '2024-09-10T15:59:15.803593+0000', 'last_fullsized': '2024-09-10T15:59:15.769083+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2486", 'last_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'objects_scrubbed': 2440, 'log_size': 8398, 'log_dups_size': 0, 'ondisk_log_size': 8398, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 792, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34373632, 'num_objects': 8392, 'num_object_clones': 0, 'num_object_copies': 16784, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4354, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8392, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5952, 'num_write_kb': 23808, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4352}, {'shards': '2', 'objects': 4040}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:01:15.622 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-10T16:01:15.622 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "43'8191", 'reported_seq': 8321, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:01:11.859364+0000', 'last_change': '2024-09-10T15:59:16.315641+0000', 'last_active': '2024-09-10T16:01:11.859364+0000', 'last_peered': '2024-09-10T16:01:11.859364+0000', 'last_clean': '2024-09-10T15:59:14.117001+0000', 'last_became_active': '2024-09-10T15:59:16.302245+0000', 'last_became_peered': '2024-09-10T15:59:16.302245+0000', 'last_unstale': '2024-09-10T16:01:11.859364+0000', 'last_undegraded': '2024-09-10T15:59:15.845012+0000', 'last_fullsized': '2024-09-10T15:59:15.770337+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3404", 'last_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'objects_scrubbed': 3367, 'log_size': 8191, 'log_dups_size': 0, 'ondisk_log_size': 8191, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 983, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33542144, 'num_objects': 8189, 'num_object_clones': 0, 'num_object_copies': 16378, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8189, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4822, 'num_write_kb': 19288, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3885}, {'shards': '0,2', 'objects': 4304}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:01:15.622 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-10T16:01:15.622 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "43'8073", 'reported_seq': 8211, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:01:11.837481+0000', 'last_change': '2024-09-10T15:59:16.251610+0000', 'last_active': '2024-09-10T16:01:11.837481+0000', 'last_peered': '2024-09-10T16:01:11.837481+0000', 'last_clean': '2024-09-10T15:59:14.115595+0000', 'last_became_active': '2024-09-10T15:59:16.244089+0000', 'last_became_peered': '2024-09-10T15:59:16.244089+0000', 'last_unstale': '2024-09-10T16:01:11.837481+0000', 'last_undegraded': '2024-09-10T15:59:15.847127+0000', 'last_fullsized': '2024-09-10T15:59:15.774437+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3653", 'last_scrub_stamp': '2024-09-10T15:59:05.298468+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:05.298468+0000', 'objects_scrubbed': 3602, 'log_size': 8073, 'log_dups_size': 0, 'ondisk_log_size': 8073, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1289, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33034240, 'num_objects': 8065, 'num_object_clones': 0, 'num_object_copies': 16130, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4206, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8065, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4463, 'num_write_kb': 17852, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3862}, {'shards': '0,2', 'objects': 4203}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:01:15.623 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-10T16:01:15.623 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "43'8307", 'reported_seq': 8440, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:01:11.885933+0000', 'last_change': '2024-09-10T15:59:16.311291+0000', 'last_active': '2024-09-10T16:01:11.885933+0000', 'last_peered': '2024-09-10T16:01:11.885933+0000', 'last_clean': '2024-09-10T15:59:14.116363+0000', 'last_became_active': '2024-09-10T15:59:16.301244+0000', 'last_became_peered': '2024-09-10T15:59:16.301244+0000', 'last_unstale': '2024-09-10T16:01:11.885933+0000', 'last_undegraded': '2024-09-10T15:59:15.883434+0000', 'last_fullsized': '2024-09-10T15:59:15.772756+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3789", 'last_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'objects_scrubbed': 3741, 'log_size': 8307, 'log_dups_size': 0, 'ondisk_log_size': 8307, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1440, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34009088, 'num_objects': 8303, 'num_object_clones': 0, 'num_object_copies': 16606, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4373, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8303, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4562, 'num_write_kb': 18248, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3932}, {'shards': '0,2', 'objects': 4371}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:01:15.623 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-10T16:01:15.623 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "43'8303", 'reported_seq': 8443, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:01:11.920787+0000', 'last_change': '2024-09-10T15:59:16.313409+0000', 'last_active': '2024-09-10T16:01:11.920787+0000', 'last_peered': '2024-09-10T16:01:11.920787+0000', 'last_clean': '2024-09-10T15:59:14.129780+0000', 'last_became_active': '2024-09-10T15:59:16.300932+0000', 'last_became_peered': '2024-09-10T15:59:16.300932+0000', 'last_unstale': '2024-09-10T16:01:11.920787+0000', 'last_undegraded': '2024-09-10T15:59:15.887823+0000', 'last_fullsized': '2024-09-10T15:59:15.768513+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3880", 'last_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'objects_scrubbed': 3827, 'log_size': 8303, 'log_dups_size': 0, 'ondisk_log_size': 8303, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1397, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33960902, 'num_objects': 8293, 'num_object_clones': 0, 'num_object_copies': 16586, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4325, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8293, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4466, 'num_write_kb': 17864, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3970}, {'shards': '0,2', 'objects': 4323}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:01:15.624 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T16:01:15.985 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 163 20 133086 133066 3.18841 1.76562 0.0198115 0.0244954 2024-09-10T16:01:16.354 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:01:16.354 INFO:teuthology.orchestra.run.smithi115.stdout:{"fsid":"7effee29-df05-4351-9741-ee203564ed81","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 47117/263202 objects degraded (17.901%), 11 pgs degraded, 12 pgs undersized","count":23},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":240,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":43,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983954,"num_in_osds":3,"osd_in_since":1725983947,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":131601,"data_bytes":538993393,"bytes_used":1811144704,"bytes_avail":288099147776,"bytes_total":289910292480,"degraded_objects":47117,"degraded_total":263202,"degraded_ratio":0.17901459715351706,"misplaced_objects":1539,"misplaced_total":263202,"misplaced_ratio":0.0058472200059270065,"recovering_objects_per_sec":32,"recovering_bytes_per_sec":133387,"recovering_keys_per_sec":0,"num_objects_recovered":391,"num_bytes_recovered":1601536,"num_keys_recovered":0,"write_bytes_sec":1956805,"read_op_per_sec":0,"write_op_per_sec":477},"fsmap":{"epoch":1,"btime":"2024-09-10T15:57:14:967120+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T16:00:13.560776+0000","services":{}},"progress_events":{}} 2024-09-10T16:01:16.378 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-10T16:01:16.378 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:01:16.926 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:01:16.929 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:01:16.953 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":156,"stamp":"2024-09-10T16:01:15.585867+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":538993393,"num_objects":131601,"num_object_clones":0,"num_object_copies":263202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":47117,"num_objects_misplaced":1539,"num_objects_unfound":0,"num_objects_dirty":131601,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":89447,"num_write_kb":357782,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2735,"num_bytes_recovered":11190291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":131674,"ondisk_log_size":131674,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1768696,"kb_used_data":1065608,"kb_used_omap":12,"kb_used_meta":703027,"kb_avail":281346824,"statfs":{"total":289910292480,"available":288099147776,"internally_reserved":0,"allocated":1091182592,"data_stored":1090297802,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":719899650},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,1,2,4,8,12,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":23494656,"num_objects":5736,"num_object_clones":0,"num_object_copies":11472,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-391,"num_objects_unfound":0,"num_objects_dirty":5736,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5736,"num_write_kb":22944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":391,"num_bytes_recovered":1601536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006635"},"pg_stats":[{"pgid":"2.f","version":"43'8136","reported_seq":8205,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:11.847939+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:01:11.847939+0000","last_peered":"2024-09-10T16:01:11.847939+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:01:11.847939+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":8136,"log_dups_size":0,"ondisk_log_size":8136,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33325056,"num_objects":8136,"num_object_clones":0,"num_object_copies":16272,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8136,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8136,"num_write_kb":32544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3877},{"shards":"0,2","objects":4258}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'8268","reported_seq":8376,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:01:14.462495+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:01:14.462495+0000","last_peered":"2024-09-10T16:01:14.462495+0000","last_clean":"2024-09-10T16:01:14.462495+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:01:14.462495+0000","last_undegraded":"2024-09-10T16:01:14.462495+0000","last_fullsized":"2024-09-10T16:01:14.462495+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":8268,"log_dups_size":0,"ondisk_log_size":8268,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33842118,"num_objects":8264,"num_object_clones":0,"num_object_copies":16528,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8264,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":6373,"num_write_kb":25492,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'8220","reported_seq":13806,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T16:01:14.538881+0000","last_change":"2024-09-10T15:59:16.142612+0000","last_active":"2024-09-10T16:01:14.538881+0000","last_peered":"2024-09-10T16:01:14.538881+0000","last_clean":"2024-09-10T15:59:14.092473+0000","last_became_active":"2024-09-10T15:59:16.109611+0000","last_became_peered":"2024-09-10T15:59:16.109611+0000","last_unstale":"2024-09-10T16:01:14.538881+0000","last_undegraded":"2024-09-10T16:01:14.538881+0000","last_fullsized":"2024-09-10T15:59:15.770264+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":8220,"log_dups_size":0,"ondisk_log_size":8220,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33652736,"num_objects":8216,"num_object_clones":0,"num_object_copies":16432,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1539,"num_objects_unfound":0,"num_objects_dirty":8216,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6224,"num_write_kb":24896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2730,"num_bytes_recovered":11177984,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1539},{"shards":"2","objects":6677}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'8221","reported_seq":8297,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:01:14.492072+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:01:14.492072+0000","last_peered":"2024-09-10T16:01:14.492072+0000","last_clean":"2024-09-10T16:01:14.492072+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:01:14.492072+0000","last_undegraded":"2024-09-10T16:01:14.492072+0000","last_fullsized":"2024-09-10T16:01:14.492072+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":8221,"log_dups_size":0,"ondisk_log_size":8221,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33673216,"num_objects":8221,"num_object_clones":0,"num_object_copies":16442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8221,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8221,"num_write_kb":32884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'8246","reported_seq":8371,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:01:11.855854+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:01:11.855854+0000","last_peered":"2024-09-10T16:01:11.855854+0000","last_clean":"2024-09-10T16:01:11.855854+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:01:11.855854+0000","last_undegraded":"2024-09-10T16:01:11.855854+0000","last_fullsized":"2024-09-10T16:01:11.855854+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":8246,"log_dups_size":0,"ondisk_log_size":8246,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33747427,"num_objects":8240,"num_object_clones":0,"num_object_copies":16480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8240,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4957,"num_write_kb":19828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'8272","reported_seq":8405,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:11.906093+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:01:11.906093+0000","last_peered":"2024-09-10T16:01:11.906093+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:01:11.906093+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":8272,"log_dups_size":0,"ondisk_log_size":8272,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33862115,"num_objects":8268,"num_object_clones":0,"num_object_copies":16536,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8268,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4884,"num_write_kb":19536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3897},{"shards":"0,2","objects":4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'8263","reported_seq":8396,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:11.834540+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:01:11.834540+0000","last_peered":"2024-09-10T16:01:11.834540+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:01:11.834540+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":8263,"log_dups_size":0,"ondisk_log_size":8263,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33821638,"num_objects":8259,"num_object_clones":0,"num_object_copies":16518,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8259,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4762,"num_write_kb":19048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4022},{"shards":"0,2","objects":4237}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'8365","reported_seq":8490,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:01:11.869976+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:01:11.869976+0000","last_peered":"2024-09-10T16:01:11.869976+0000","last_clean":"2024-09-10T16:01:11.869976+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:01:11.869976+0000","last_undegraded":"2024-09-10T16:01:11.869976+0000","last_fullsized":"2024-09-10T16:01:11.869976+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":8365,"log_dups_size":0,"ondisk_log_size":8365,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34254848,"num_objects":8363,"num_object_clones":0,"num_object_copies":16726,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8363,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5212,"num_write_kb":20848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.777326+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T15:59:15.777326+0000","last_peered":"2024-09-10T15:59:15.777326+0000","last_clean":"2024-09-10T15:59:15.777326+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T15:59:15.777326+0000","last_undegraded":"2024-09-10T15:59:15.777326+0000","last_fullsized":"2024-09-10T15:59:15.777326+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'7969","reported_seq":8104,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:14.502406+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:01:14.502406+0000","last_peered":"2024-09-10T16:01:14.502406+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:01:14.502406+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":7969,"log_dups_size":0,"ondisk_log_size":7969,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32612835,"num_objects":7963,"num_object_clones":0,"num_object_copies":15926,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7963,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5788,"num_write_kb":23152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4123},{"shards":"2","objects":3840}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":89,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.772619+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T15:59:15.772619+0000","last_peered":"2024-09-10T15:59:15.772619+0000","last_clean":"2024-09-10T15:59:15.772619+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T15:59:15.772619+0000","last_undegraded":"2024-09-10T15:59:15.772619+0000","last_fullsized":"2024-09-10T15:59:15.772619+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'8192","reported_seq":8318,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:11.860693+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:01:11.860693+0000","last_peered":"2024-09-10T16:01:11.860693+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:01:11.860693+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":8192,"log_dups_size":0,"ondisk_log_size":8192,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33521664,"num_objects":8184,"num_object_clones":0,"num_object_copies":16368,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8184,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4696,"num_write_kb":18784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3929},{"shards":"0,2","objects":4255}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":99,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.884297+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T15:59:15.884297+0000","last_peered":"2024-09-10T15:59:15.884297+0000","last_clean":"2024-09-10T15:59:15.884297+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T15:59:15.884297+0000","last_undegraded":"2024-09-10T15:59:15.884297+0000","last_fullsized":"2024-09-10T15:59:15.884297+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'8247","reported_seq":8377,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:14.576432+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:01:14.576432+0000","last_peered":"2024-09-10T16:01:14.576432+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:01:14.576432+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":8247,"log_dups_size":0,"ondisk_log_size":8247,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33759715,"num_objects":8243,"num_object_clones":0,"num_object_copies":16486,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8243,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5927,"num_write_kb":23708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4304},{"shards":"2","objects":3939}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.837509+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T15:59:15.837509+0000","last_peered":"2024-09-10T15:59:15.837509+0000","last_clean":"2024-09-10T15:59:15.837509+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T15:59:15.837509+0000","last_undegraded":"2024-09-10T15:59:15.837509+0000","last_fullsized":"2024-09-10T15:59:15.837509+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'8398","reported_seq":8520,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:14.598066+0000","last_change":"2024-09-10T15:59:16.174113+0000","last_active":"2024-09-10T16:01:14.598066+0000","last_peered":"2024-09-10T16:01:14.598066+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:01:14.598066+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":8398,"log_dups_size":0,"ondisk_log_size":8398,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34373632,"num_objects":8392,"num_object_clones":0,"num_object_copies":16784,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4354,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8392,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5952,"num_write_kb":23808,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4352},{"shards":"2","objects":4040}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'8191","reported_seq":8321,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:11.859364+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:01:11.859364+0000","last_peered":"2024-09-10T16:01:11.859364+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:01:11.859364+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":8191,"log_dups_size":0,"ondisk_log_size":8191,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33542144,"num_objects":8189,"num_object_clones":0,"num_object_copies":16378,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8189,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4822,"num_write_kb":19288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3885},{"shards":"0,2","objects":4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.989413+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T15:59:15.989413+0000","last_peered":"2024-09-10T15:59:15.989413+0000","last_clean":"2024-09-10T15:59:15.989413+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T15:59:15.989413+0000","last_undegraded":"2024-09-10T15:59:15.989413+0000","last_fullsized":"2024-09-10T15:59:15.989413+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'8073","reported_seq":8211,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:11.837481+0000","last_change":"2024-09-10T15:59:16.251610+0000","last_active":"2024-09-10T16:01:11.837481+0000","last_peered":"2024-09-10T16:01:11.837481+0000","last_clean":"2024-09-10T15:59:14.115595+0000","last_became_active":"2024-09-10T15:59:16.244089+0000","last_became_peered":"2024-09-10T15:59:16.244089+0000","last_unstale":"2024-09-10T16:01:11.837481+0000","last_undegraded":"2024-09-10T15:59:15.847127+0000","last_fullsized":"2024-09-10T15:59:15.774437+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":8073,"log_dups_size":0,"ondisk_log_size":8073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33034240,"num_objects":8065,"num_object_clones":0,"num_object_copies":16130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8065,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4463,"num_write_kb":17852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3862},{"shards":"0,2","objects":4203}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":89,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.773633+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T15:59:15.773633+0000","last_peered":"2024-09-10T15:59:15.773633+0000","last_clean":"2024-09-10T15:59:15.773633+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T15:59:15.773633+0000","last_undegraded":"2024-09-10T15:59:15.773633+0000","last_fullsized":"2024-09-10T15:59:15.773633+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'8307","reported_seq":8440,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:11.885933+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:01:11.885933+0000","last_peered":"2024-09-10T16:01:11.885933+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:01:11.885933+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":8307,"log_dups_size":0,"ondisk_log_size":8307,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34009088,"num_objects":8303,"num_object_clones":0,"num_object_copies":16606,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8303,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4562,"num_write_kb":18248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3932},{"shards":"0,2","objects":4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.916017+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T15:59:15.916017+0000","last_peered":"2024-09-10T15:59:15.916017+0000","last_clean":"2024-09-10T15:59:15.916017+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T15:59:15.916017+0000","last_undegraded":"2024-09-10T15:59:15.916017+0000","last_fullsized":"2024-09-10T15:59:15.916017+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.776033+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T15:59:15.776033+0000","last_peered":"2024-09-10T15:59:15.776033+0000","last_clean":"2024-09-10T15:59:15.776033+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T15:59:15.776033+0000","last_undegraded":"2024-09-10T15:59:15.776033+0000","last_fullsized":"2024-09-10T15:59:15.776033+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'8303","reported_seq":8443,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:11.920787+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:01:11.920787+0000","last_peered":"2024-09-10T16:01:11.920787+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:01:11.920787+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":8303,"log_dups_size":0,"ondisk_log_size":8303,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33960902,"num_objects":8293,"num_object_clones":0,"num_object_copies":16586,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8293,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4466,"num_write_kb":17864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3970},{"shards":"0,2","objects":4323}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":538993374,"num_objects":131599,"num_object_clones":0,"num_object_copies":263198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":47117,"num_objects_misplaced":1539,"num_objects_unfound":0,"num_objects_dirty":131599,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":89445,"num_write_kb":357780,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2733,"num_bytes_recovered":11190272,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1091518464,"data_stored":1091366718,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":131671,"ondisk_log_size":131671,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986215,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":649936,"kb_used_data":401664,"kb_used_omap":4,"kb_used_meta":248251,"kb_avail":93721904,"statfs":{"total":96636764160,"available":95971229696,"internally_reserved":0,"allocated":411303936,"data_stored":411026176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":254209706},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,1,0,2,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659161,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":360068,"kb_used_data":202036,"kb_used_omap":4,"kb_used_meta":158011,"kb_avail":94011772,"statfs":{"total":96636764160,"available":96268054528,"internally_reserved":0,"allocated":206884864,"data_stored":206559671,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":161803950},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,3,1,7,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182443,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":758692,"kb_used_data":461908,"kb_used_omap":4,"kb_used_meta":296763,"kb_avail":93613148,"statfs":{"total":96636764160,"available":95859863552,"internally_reserved":0,"allocated":472993792,"data_stored":472711955,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":303885994},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,0,1,5,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":473784320,"data_stored":473748190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":206524416,"data_stored":206444930,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":411209728,"data_stored":411173598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:01:16.985 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 164 20 133514 133494 3.17916 1.67188 0.0224358 0.0245668 2024-09-10T16:01:17.985 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 165 20 134009 133989 3.17162 1.93359 0.0158532 0.0246248 2024-09-10T16:01:18.985 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 166 20 134510 134490 3.1643 1.95703 0.0150144 0.0246816 2024-09-10T16:01:19.956 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:01:19.985 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 167 19 135015 134996 3.15719 1.97656 0.034593 0.0247387 2024-09-10T16:01:20.522 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:01:20.523 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:01:20.546 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":158,"stamp":"2024-09-10T16:01:19.587405+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":545137393,"num_objects":133101,"num_object_clones":0,"num_object_copies":266202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":47117,"num_objects_misplaced":1539,"num_objects_unfound":0,"num_objects_dirty":133101,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":90947,"num_write_kb":363782,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2735,"num_bytes_recovered":11190291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":133174,"ondisk_log_size":133174,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1797376,"kb_used_data":1080396,"kb_used_omap":12,"kb_used_meta":716915,"kb_avail":281318144,"statfs":{"total":289910292480,"available":288069779456,"internally_reserved":0,"allocated":1106325504,"data_stored":1105440714,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":734120962},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,2,4,7,8,8,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19636224,"num_objects":4794,"num_object_clones":0,"num_object_copies":9588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-258,"num_objects_unfound":0,"num_objects_dirty":4794,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4794,"num_write_kb":19176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":258,"num_bytes_recovered":1056768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006836"},"pg_stats":[{"pgid":"2.f","version":"43'8271","reported_seq":8340,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:16.899830+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:01:16.899830+0000","last_peered":"2024-09-10T16:01:16.899830+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:01:16.899830+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":8271,"log_dups_size":0,"ondisk_log_size":8271,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33878016,"num_objects":8271,"num_object_clones":0,"num_object_copies":16542,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8271,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8271,"num_write_kb":33084,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4012},{"shards":"0,2","objects":4258}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'8268","reported_seq":8376,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:01:14.462495+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:01:14.462495+0000","last_peered":"2024-09-10T16:01:14.462495+0000","last_clean":"2024-09-10T16:01:14.462495+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:01:14.462495+0000","last_undegraded":"2024-09-10T16:01:14.462495+0000","last_fullsized":"2024-09-10T16:01:14.462495+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":8268,"log_dups_size":0,"ondisk_log_size":8268,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33842118,"num_objects":8264,"num_object_clones":0,"num_object_copies":16528,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8264,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":6373,"num_write_kb":25492,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'8220","reported_seq":13806,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T16:01:14.538881+0000","last_change":"2024-09-10T15:59:16.142612+0000","last_active":"2024-09-10T16:01:14.538881+0000","last_peered":"2024-09-10T16:01:14.538881+0000","last_clean":"2024-09-10T15:59:14.092473+0000","last_became_active":"2024-09-10T15:59:16.109611+0000","last_became_peered":"2024-09-10T15:59:16.109611+0000","last_unstale":"2024-09-10T16:01:14.538881+0000","last_undegraded":"2024-09-10T16:01:14.538881+0000","last_fullsized":"2024-09-10T15:59:15.770264+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":8220,"log_dups_size":0,"ondisk_log_size":8220,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33652736,"num_objects":8216,"num_object_clones":0,"num_object_copies":16432,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1539,"num_objects_unfound":0,"num_objects_dirty":8216,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6224,"num_write_kb":24896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2730,"num_bytes_recovered":11177984,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1539},{"shards":"2","objects":6677}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'8221","reported_seq":8297,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:01:14.492072+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:01:14.492072+0000","last_peered":"2024-09-10T16:01:14.492072+0000","last_clean":"2024-09-10T16:01:14.492072+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:01:14.492072+0000","last_undegraded":"2024-09-10T16:01:14.492072+0000","last_fullsized":"2024-09-10T16:01:14.492072+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":8221,"log_dups_size":0,"ondisk_log_size":8221,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33673216,"num_objects":8221,"num_object_clones":0,"num_object_copies":16442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8221,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8221,"num_write_kb":32884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'8384","reported_seq":8509,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:01:16.867283+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:01:16.867283+0000","last_peered":"2024-09-10T16:01:16.867283+0000","last_clean":"2024-09-10T16:01:16.867283+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:01:16.867283+0000","last_undegraded":"2024-09-10T16:01:16.867283+0000","last_fullsized":"2024-09-10T16:01:16.867283+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":8384,"log_dups_size":0,"ondisk_log_size":8384,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34312675,"num_objects":8378,"num_object_clones":0,"num_object_copies":16756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8378,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5095,"num_write_kb":20380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'8423","reported_seq":8554,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:16.922035+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:01:16.922035+0000","last_peered":"2024-09-10T16:01:16.922035+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:01:16.922035+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":8423,"log_dups_size":0,"ondisk_log_size":8423,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34480611,"num_objects":8419,"num_object_clones":0,"num_object_copies":16838,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8419,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5035,"num_write_kb":20140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4048},{"shards":"0,2","objects":4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'8415","reported_seq":8548,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:16.842574+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:01:16.842574+0000","last_peered":"2024-09-10T16:01:16.842574+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:01:16.842574+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":8415,"log_dups_size":0,"ondisk_log_size":8415,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34444230,"num_objects":8411,"num_object_clones":0,"num_object_copies":16822,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8411,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4914,"num_write_kb":19656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4174},{"shards":"0,2","objects":4237}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'8509","reported_seq":8634,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:01:16.922155+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:01:16.922155+0000","last_peered":"2024-09-10T16:01:16.922155+0000","last_clean":"2024-09-10T16:01:16.922155+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:01:16.922155+0000","last_undegraded":"2024-09-10T16:01:16.922155+0000","last_fullsized":"2024-09-10T16:01:16.922155+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":8509,"log_dups_size":0,"ondisk_log_size":8509,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34844672,"num_objects":8507,"num_object_clones":0,"num_object_copies":17014,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8507,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5356,"num_write_kb":21424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.777326+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T15:59:15.777326+0000","last_peered":"2024-09-10T15:59:15.777326+0000","last_clean":"2024-09-10T15:59:15.777326+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T15:59:15.777326+0000","last_undegraded":"2024-09-10T15:59:15.777326+0000","last_fullsized":"2024-09-10T15:59:15.777326+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'7969","reported_seq":8104,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:14.502406+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:01:14.502406+0000","last_peered":"2024-09-10T16:01:14.502406+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:01:14.502406+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":7969,"log_dups_size":0,"ondisk_log_size":7969,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32612835,"num_objects":7963,"num_object_clones":0,"num_object_copies":15926,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7963,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5788,"num_write_kb":23152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4123},{"shards":"2","objects":3840}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":89,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.772619+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T15:59:15.772619+0000","last_peered":"2024-09-10T15:59:15.772619+0000","last_clean":"2024-09-10T15:59:15.772619+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T15:59:15.772619+0000","last_undegraded":"2024-09-10T15:59:15.772619+0000","last_fullsized":"2024-09-10T15:59:15.772619+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'8347","reported_seq":8473,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:16.953117+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:01:16.953117+0000","last_peered":"2024-09-10T16:01:16.953117+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:01:16.953117+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":8347,"log_dups_size":0,"ondisk_log_size":8347,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34156544,"num_objects":8339,"num_object_clones":0,"num_object_copies":16678,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8339,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4851,"num_write_kb":19404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4084},{"shards":"0,2","objects":4255}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":99,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.884297+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T15:59:15.884297+0000","last_peered":"2024-09-10T15:59:15.884297+0000","last_clean":"2024-09-10T15:59:15.884297+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T15:59:15.884297+0000","last_undegraded":"2024-09-10T15:59:15.884297+0000","last_fullsized":"2024-09-10T15:59:15.884297+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'8247","reported_seq":8377,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:14.576432+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:01:14.576432+0000","last_peered":"2024-09-10T16:01:14.576432+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:01:14.576432+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":8247,"log_dups_size":0,"ondisk_log_size":8247,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33759715,"num_objects":8243,"num_object_clones":0,"num_object_copies":16486,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8243,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5927,"num_write_kb":23708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4304},{"shards":"2","objects":3939}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.848670+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T15:59:15.848670+0000","last_peered":"2024-09-10T15:59:15.848670+0000","last_clean":"2024-09-10T15:59:15.848670+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T15:59:15.848670+0000","last_undegraded":"2024-09-10T15:59:15.848670+0000","last_fullsized":"2024-09-10T15:59:15.848670+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'8398","reported_seq":8520,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:14.598066+0000","last_change":"2024-09-10T15:59:16.174113+0000","last_active":"2024-09-10T16:01:14.598066+0000","last_peered":"2024-09-10T16:01:14.598066+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:01:14.598066+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":8398,"log_dups_size":0,"ondisk_log_size":8398,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34373632,"num_objects":8392,"num_object_clones":0,"num_object_copies":16784,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4354,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8392,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5952,"num_write_kb":23808,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4352},{"shards":"2","objects":4040}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'8335","reported_seq":8465,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:16.951094+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:01:16.951094+0000","last_peered":"2024-09-10T16:01:16.951094+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:01:16.951094+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":8335,"log_dups_size":0,"ondisk_log_size":8335,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34131968,"num_objects":8333,"num_object_clones":0,"num_object_copies":16666,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8333,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4966,"num_write_kb":19864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4029},{"shards":"0,2","objects":4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.992406+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T15:59:15.992406+0000","last_peered":"2024-09-10T15:59:15.992406+0000","last_clean":"2024-09-10T15:59:15.992406+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T15:59:15.992406+0000","last_undegraded":"2024-09-10T15:59:15.992406+0000","last_fullsized":"2024-09-10T15:59:15.992406+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'8203","reported_seq":8341,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:16.942618+0000","last_change":"2024-09-10T15:59:16.251610+0000","last_active":"2024-09-10T16:01:16.942618+0000","last_peered":"2024-09-10T16:01:16.942618+0000","last_clean":"2024-09-10T15:59:14.115595+0000","last_became_active":"2024-09-10T15:59:16.244089+0000","last_became_peered":"2024-09-10T15:59:16.244089+0000","last_unstale":"2024-09-10T16:01:16.942618+0000","last_undegraded":"2024-09-10T15:59:15.847127+0000","last_fullsized":"2024-09-10T15:59:15.774437+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":8203,"log_dups_size":0,"ondisk_log_size":8203,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33566720,"num_objects":8195,"num_object_clones":0,"num_object_copies":16390,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8195,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4593,"num_write_kb":18372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3992},{"shards":"0,2","objects":4203}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":89,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.773633+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T15:59:15.773633+0000","last_peered":"2024-09-10T15:59:15.773633+0000","last_clean":"2024-09-10T15:59:15.773633+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T15:59:15.773633+0000","last_undegraded":"2024-09-10T15:59:15.773633+0000","last_fullsized":"2024-09-10T15:59:15.773633+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'8466","reported_seq":8601,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:16.937870+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:01:16.937870+0000","last_peered":"2024-09-10T16:01:16.937870+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:01:16.937870+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":8466,"log_dups_size":0,"ondisk_log_size":8466,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34660352,"num_objects":8462,"num_object_clones":0,"num_object_copies":16924,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8462,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4721,"num_write_kb":18884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4091},{"shards":"0,2","objects":4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.917634+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T15:59:15.917634+0000","last_peered":"2024-09-10T15:59:15.917634+0000","last_clean":"2024-09-10T15:59:15.917634+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T15:59:15.917634+0000","last_undegraded":"2024-09-10T15:59:15.917634+0000","last_fullsized":"2024-09-10T15:59:15.917634+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.776033+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T15:59:15.776033+0000","last_peered":"2024-09-10T15:59:15.776033+0000","last_clean":"2024-09-10T15:59:15.776033+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T15:59:15.776033+0000","last_undegraded":"2024-09-10T15:59:15.776033+0000","last_fullsized":"2024-09-10T15:59:15.776033+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'8495","reported_seq":8635,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:16.906623+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:01:16.906623+0000","last_peered":"2024-09-10T16:01:16.906623+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:01:16.906623+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":8495,"log_dups_size":0,"ondisk_log_size":8495,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34747334,"num_objects":8485,"num_object_clones":0,"num_object_copies":16970,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8485,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4658,"num_write_kb":18632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4162},{"shards":"0,2","objects":4323}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":545137374,"num_objects":133099,"num_object_clones":0,"num_object_copies":266198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":47117,"num_objects_misplaced":1539,"num_objects_unfound":0,"num_objects_dirty":133099,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":90945,"num_write_kb":363780,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2733,"num_bytes_recovered":11190272,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1106739200,"data_stored":1106587454,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":133171,"ondisk_log_size":133171,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986215,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":649936,"kb_used_data":401664,"kb_used_omap":4,"kb_used_meta":248251,"kb_avail":93721904,"statfs":{"total":96636764160,"available":95971229696,"internally_reserved":0,"allocated":411303936,"data_stored":411026176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":254209706},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,1,0,2,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659162,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":394044,"kb_used_data":209640,"kb_used_omap":4,"kb_used_meta":184379,"kb_avail":93977796,"statfs":{"total":96636764160,"available":96233263104,"internally_reserved":0,"allocated":214671360,"data_stored":214346167,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":188804782},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,2,1,1,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182444,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":753396,"kb_used_data":469092,"kb_used_omap":4,"kb_used_meta":284283,"kb_avail":93618444,"statfs":{"total":96636764160,"available":95865286656,"internally_reserved":0,"allocated":480350208,"data_stored":480068371,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":291106474},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,0,1,6,5,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":481165312,"data_stored":481129182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":214364160,"data_stored":214284674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":411209728,"data_stored":411173598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:01:20.547 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:01:20.985 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 168 20 135435 135415 3.14813 1.63672 0.0618186 0.0248091 2024-09-10T16:01:21.160 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:01:21.162 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:01:21.180 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":158,"stamp":"2024-09-10T16:01:19.587405+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":545137393,"num_objects":133101,"num_object_clones":0,"num_object_copies":266202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":47117,"num_objects_misplaced":1539,"num_objects_unfound":0,"num_objects_dirty":133101,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":90947,"num_write_kb":363782,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2735,"num_bytes_recovered":11190291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":133174,"ondisk_log_size":133174,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1797376,"kb_used_data":1080396,"kb_used_omap":12,"kb_used_meta":716915,"kb_avail":281318144,"statfs":{"total":289910292480,"available":288069779456,"internally_reserved":0,"allocated":1106325504,"data_stored":1105440714,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":734120962},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,2,4,7,8,8,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19636224,"num_objects":4794,"num_object_clones":0,"num_object_copies":9588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-258,"num_objects_unfound":0,"num_objects_dirty":4794,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4794,"num_write_kb":19176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":258,"num_bytes_recovered":1056768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006836"},"pg_stats":[{"pgid":"2.f","version":"43'8271","reported_seq":8340,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:16.899830+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:01:16.899830+0000","last_peered":"2024-09-10T16:01:16.899830+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:01:16.899830+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":8271,"log_dups_size":0,"ondisk_log_size":8271,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33878016,"num_objects":8271,"num_object_clones":0,"num_object_copies":16542,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8271,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8271,"num_write_kb":33084,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4012},{"shards":"0,2","objects":4258}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'8268","reported_seq":8376,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:01:14.462495+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:01:14.462495+0000","last_peered":"2024-09-10T16:01:14.462495+0000","last_clean":"2024-09-10T16:01:14.462495+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:01:14.462495+0000","last_undegraded":"2024-09-10T16:01:14.462495+0000","last_fullsized":"2024-09-10T16:01:14.462495+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":8268,"log_dups_size":0,"ondisk_log_size":8268,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33842118,"num_objects":8264,"num_object_clones":0,"num_object_copies":16528,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8264,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":6373,"num_write_kb":25492,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'8220","reported_seq":13806,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T16:01:14.538881+0000","last_change":"2024-09-10T15:59:16.142612+0000","last_active":"2024-09-10T16:01:14.538881+0000","last_peered":"2024-09-10T16:01:14.538881+0000","last_clean":"2024-09-10T15:59:14.092473+0000","last_became_active":"2024-09-10T15:59:16.109611+0000","last_became_peered":"2024-09-10T15:59:16.109611+0000","last_unstale":"2024-09-10T16:01:14.538881+0000","last_undegraded":"2024-09-10T16:01:14.538881+0000","last_fullsized":"2024-09-10T15:59:15.770264+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":8220,"log_dups_size":0,"ondisk_log_size":8220,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33652736,"num_objects":8216,"num_object_clones":0,"num_object_copies":16432,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1539,"num_objects_unfound":0,"num_objects_dirty":8216,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6224,"num_write_kb":24896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2730,"num_bytes_recovered":11177984,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1539},{"shards":"2","objects":6677}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'8221","reported_seq":8297,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:01:14.492072+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:01:14.492072+0000","last_peered":"2024-09-10T16:01:14.492072+0000","last_clean":"2024-09-10T16:01:14.492072+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:01:14.492072+0000","last_undegraded":"2024-09-10T16:01:14.492072+0000","last_fullsized":"2024-09-10T16:01:14.492072+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":8221,"log_dups_size":0,"ondisk_log_size":8221,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33673216,"num_objects":8221,"num_object_clones":0,"num_object_copies":16442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8221,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8221,"num_write_kb":32884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'8384","reported_seq":8509,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:01:16.867283+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:01:16.867283+0000","last_peered":"2024-09-10T16:01:16.867283+0000","last_clean":"2024-09-10T16:01:16.867283+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:01:16.867283+0000","last_undegraded":"2024-09-10T16:01:16.867283+0000","last_fullsized":"2024-09-10T16:01:16.867283+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":8384,"log_dups_size":0,"ondisk_log_size":8384,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34312675,"num_objects":8378,"num_object_clones":0,"num_object_copies":16756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8378,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5095,"num_write_kb":20380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'8423","reported_seq":8554,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:16.922035+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:01:16.922035+0000","last_peered":"2024-09-10T16:01:16.922035+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:01:16.922035+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":8423,"log_dups_size":0,"ondisk_log_size":8423,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34480611,"num_objects":8419,"num_object_clones":0,"num_object_copies":16838,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8419,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5035,"num_write_kb":20140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4048},{"shards":"0,2","objects":4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'8415","reported_seq":8548,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:16.842574+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:01:16.842574+0000","last_peered":"2024-09-10T16:01:16.842574+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:01:16.842574+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":8415,"log_dups_size":0,"ondisk_log_size":8415,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34444230,"num_objects":8411,"num_object_clones":0,"num_object_copies":16822,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8411,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4914,"num_write_kb":19656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4174},{"shards":"0,2","objects":4237}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'8509","reported_seq":8634,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:01:16.922155+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:01:16.922155+0000","last_peered":"2024-09-10T16:01:16.922155+0000","last_clean":"2024-09-10T16:01:16.922155+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:01:16.922155+0000","last_undegraded":"2024-09-10T16:01:16.922155+0000","last_fullsized":"2024-09-10T16:01:16.922155+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":8509,"log_dups_size":0,"ondisk_log_size":8509,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34844672,"num_objects":8507,"num_object_clones":0,"num_object_copies":17014,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8507,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5356,"num_write_kb":21424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.777326+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T15:59:15.777326+0000","last_peered":"2024-09-10T15:59:15.777326+0000","last_clean":"2024-09-10T15:59:15.777326+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T15:59:15.777326+0000","last_undegraded":"2024-09-10T15:59:15.777326+0000","last_fullsized":"2024-09-10T15:59:15.777326+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'7969","reported_seq":8104,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:14.502406+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:01:14.502406+0000","last_peered":"2024-09-10T16:01:14.502406+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:01:14.502406+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":7969,"log_dups_size":0,"ondisk_log_size":7969,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32612835,"num_objects":7963,"num_object_clones":0,"num_object_copies":15926,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7963,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5788,"num_write_kb":23152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4123},{"shards":"2","objects":3840}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":89,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.772619+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T15:59:15.772619+0000","last_peered":"2024-09-10T15:59:15.772619+0000","last_clean":"2024-09-10T15:59:15.772619+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T15:59:15.772619+0000","last_undegraded":"2024-09-10T15:59:15.772619+0000","last_fullsized":"2024-09-10T15:59:15.772619+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'8347","reported_seq":8473,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:16.953117+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:01:16.953117+0000","last_peered":"2024-09-10T16:01:16.953117+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:01:16.953117+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":8347,"log_dups_size":0,"ondisk_log_size":8347,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34156544,"num_objects":8339,"num_object_clones":0,"num_object_copies":16678,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8339,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4851,"num_write_kb":19404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4084},{"shards":"0,2","objects":4255}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":99,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.884297+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T15:59:15.884297+0000","last_peered":"2024-09-10T15:59:15.884297+0000","last_clean":"2024-09-10T15:59:15.884297+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T15:59:15.884297+0000","last_undegraded":"2024-09-10T15:59:15.884297+0000","last_fullsized":"2024-09-10T15:59:15.884297+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'8247","reported_seq":8377,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:14.576432+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:01:14.576432+0000","last_peered":"2024-09-10T16:01:14.576432+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:01:14.576432+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":8247,"log_dups_size":0,"ondisk_log_size":8247,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33759715,"num_objects":8243,"num_object_clones":0,"num_object_copies":16486,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8243,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5927,"num_write_kb":23708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4304},{"shards":"2","objects":3939}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.848670+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T15:59:15.848670+0000","last_peered":"2024-09-10T15:59:15.848670+0000","last_clean":"2024-09-10T15:59:15.848670+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T15:59:15.848670+0000","last_undegraded":"2024-09-10T15:59:15.848670+0000","last_fullsized":"2024-09-10T15:59:15.848670+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'8398","reported_seq":8520,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:14.598066+0000","last_change":"2024-09-10T15:59:16.174113+0000","last_active":"2024-09-10T16:01:14.598066+0000","last_peered":"2024-09-10T16:01:14.598066+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:01:14.598066+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":8398,"log_dups_size":0,"ondisk_log_size":8398,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34373632,"num_objects":8392,"num_object_clones":0,"num_object_copies":16784,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4354,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8392,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5952,"num_write_kb":23808,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4352},{"shards":"2","objects":4040}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'8335","reported_seq":8465,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:16.951094+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:01:16.951094+0000","last_peered":"2024-09-10T16:01:16.951094+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:01:16.951094+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":8335,"log_dups_size":0,"ondisk_log_size":8335,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34131968,"num_objects":8333,"num_object_clones":0,"num_object_copies":16666,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8333,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4966,"num_write_kb":19864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4029},{"shards":"0,2","objects":4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.992406+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T15:59:15.992406+0000","last_peered":"2024-09-10T15:59:15.992406+0000","last_clean":"2024-09-10T15:59:15.992406+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T15:59:15.992406+0000","last_undegraded":"2024-09-10T15:59:15.992406+0000","last_fullsized":"2024-09-10T15:59:15.992406+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'8203","reported_seq":8341,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:16.942618+0000","last_change":"2024-09-10T15:59:16.251610+0000","last_active":"2024-09-10T16:01:16.942618+0000","last_peered":"2024-09-10T16:01:16.942618+0000","last_clean":"2024-09-10T15:59:14.115595+0000","last_became_active":"2024-09-10T15:59:16.244089+0000","last_became_peered":"2024-09-10T15:59:16.244089+0000","last_unstale":"2024-09-10T16:01:16.942618+0000","last_undegraded":"2024-09-10T15:59:15.847127+0000","last_fullsized":"2024-09-10T15:59:15.774437+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":8203,"log_dups_size":0,"ondisk_log_size":8203,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33566720,"num_objects":8195,"num_object_clones":0,"num_object_copies":16390,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8195,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4593,"num_write_kb":18372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3992},{"shards":"0,2","objects":4203}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":89,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.773633+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T15:59:15.773633+0000","last_peered":"2024-09-10T15:59:15.773633+0000","last_clean":"2024-09-10T15:59:15.773633+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T15:59:15.773633+0000","last_undegraded":"2024-09-10T15:59:15.773633+0000","last_fullsized":"2024-09-10T15:59:15.773633+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'8466","reported_seq":8601,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:16.937870+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:01:16.937870+0000","last_peered":"2024-09-10T16:01:16.937870+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:01:16.937870+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":8466,"log_dups_size":0,"ondisk_log_size":8466,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34660352,"num_objects":8462,"num_object_clones":0,"num_object_copies":16924,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8462,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4721,"num_write_kb":18884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4091},{"shards":"0,2","objects":4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.917634+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T15:59:15.917634+0000","last_peered":"2024-09-10T15:59:15.917634+0000","last_clean":"2024-09-10T15:59:15.917634+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T15:59:15.917634+0000","last_undegraded":"2024-09-10T15:59:15.917634+0000","last_fullsized":"2024-09-10T15:59:15.917634+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.776033+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T15:59:15.776033+0000","last_peered":"2024-09-10T15:59:15.776033+0000","last_clean":"2024-09-10T15:59:15.776033+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T15:59:15.776033+0000","last_undegraded":"2024-09-10T15:59:15.776033+0000","last_fullsized":"2024-09-10T15:59:15.776033+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'8495","reported_seq":8635,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:16.906623+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:01:16.906623+0000","last_peered":"2024-09-10T16:01:16.906623+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:01:16.906623+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":8495,"log_dups_size":0,"ondisk_log_size":8495,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34747334,"num_objects":8485,"num_object_clones":0,"num_object_copies":16970,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8485,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4658,"num_write_kb":18632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4162},{"shards":"0,2","objects":4323}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":545137374,"num_objects":133099,"num_object_clones":0,"num_object_copies":266198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":47117,"num_objects_misplaced":1539,"num_objects_unfound":0,"num_objects_dirty":133099,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":90945,"num_write_kb":363780,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2733,"num_bytes_recovered":11190272,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1106739200,"data_stored":1106587454,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":133171,"ondisk_log_size":133171,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986215,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":649936,"kb_used_data":401664,"kb_used_omap":4,"kb_used_meta":248251,"kb_avail":93721904,"statfs":{"total":96636764160,"available":95971229696,"internally_reserved":0,"allocated":411303936,"data_stored":411026176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":254209706},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,1,0,2,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659162,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":394044,"kb_used_data":209640,"kb_used_omap":4,"kb_used_meta":184379,"kb_avail":93977796,"statfs":{"total":96636764160,"available":96233263104,"internally_reserved":0,"allocated":214671360,"data_stored":214346167,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":188804782},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,2,1,1,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182444,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":753396,"kb_used_data":469092,"kb_used_omap":4,"kb_used_meta":284283,"kb_avail":93618444,"statfs":{"total":96636764160,"available":95865286656,"internally_reserved":0,"allocated":480350208,"data_stored":480068371,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":291106474},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,0,1,6,5,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":481165312,"data_stored":481129182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":214364160,"data_stored":214284674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":411209728,"data_stored":411173598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:01:21.181 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-10T16:01:21.181 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "43'8271", 'reported_seq': 8340, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:01:16.899830+0000', 'last_change': '2024-09-10T15:59:16.281558+0000', 'last_active': '2024-09-10T16:01:16.899830+0000', 'last_peered': '2024-09-10T16:01:16.899830+0000', 'last_clean': '2024-09-10T15:59:14.132456+0000', 'last_became_active': '2024-09-10T15:59:16.274407+0000', 'last_became_peered': '2024-09-10T15:59:16.274407+0000', 'last_unstale': '2024-09-10T16:01:16.899830+0000', 'last_undegraded': '2024-09-10T15:59:15.851279+0000', 'last_fullsized': '2024-09-10T15:59:15.774356+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'objects_scrubbed': 0, 'log_size': 8271, 'log_dups_size': 0, 'ondisk_log_size': 8271, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33878016, 'num_objects': 8271, 'num_object_clones': 0, 'num_object_copies': 16542, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4259, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8271, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8271, 'num_write_kb': 33084, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4012}, {'shards': '0,2', 'objects': 4258}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:01:21.181 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-10T16:01:21.181 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "43'8220", 'reported_seq': 13806, 'reported_epoch': 43, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-10T16:01:14.538881+0000', 'last_change': '2024-09-10T15:59:16.142612+0000', 'last_active': '2024-09-10T16:01:14.538881+0000', 'last_peered': '2024-09-10T16:01:14.538881+0000', 'last_clean': '2024-09-10T15:59:14.092473+0000', 'last_became_active': '2024-09-10T15:59:16.109611+0000', 'last_became_peered': '2024-09-10T15:59:16.109611+0000', 'last_unstale': '2024-09-10T16:01:14.538881+0000', 'last_undegraded': '2024-09-10T16:01:14.538881+0000', 'last_fullsized': '2024-09-10T15:59:15.770264+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2029", 'last_scrub_stamp': '2024-09-10T15:58:45.933789+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:45.933789+0000', 'objects_scrubbed': 1992, 'log_size': 8220, 'log_dups_size': 0, 'ondisk_log_size': 8220, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 668, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33652736, 'num_objects': 8216, 'num_object_clones': 0, 'num_object_copies': 16432, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1539, 'num_objects_unfound': 0, 'num_objects_dirty': 8216, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6224, 'num_write_kb': 24896, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2730, 'num_bytes_recovered': 11177984, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1539}, {'shards': '2', 'objects': 6677}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:01:21.181 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-10T16:01:21.182 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "43'8423", 'reported_seq': 8554, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:01:16.922035+0000', 'last_change': '2024-09-10T15:59:16.308264+0000', 'last_active': '2024-09-10T16:01:16.922035+0000', 'last_peered': '2024-09-10T16:01:16.922035+0000', 'last_clean': '2024-09-10T15:59:14.111551+0000', 'last_became_active': '2024-09-10T15:59:16.300040+0000', 'last_became_peered': '2024-09-10T15:59:16.300040+0000', 'last_unstale': '2024-09-10T16:01:16.922035+0000', 'last_undegraded': '2024-09-10T15:59:15.878234+0000', 'last_fullsized': '2024-09-10T15:59:15.769328+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3442", 'last_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'objects_scrubbed': 3384, 'log_size': 8423, 'log_dups_size': 0, 'ondisk_log_size': 8423, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1292, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34480611, 'num_objects': 8419, 'num_object_clones': 0, 'num_object_copies': 16838, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4372, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8419, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 5035, 'num_write_kb': 20140, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4048}, {'shards': '0,2', 'objects': 4371}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:01:21.182 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-10T16:01:21.182 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "43'8415", 'reported_seq': 8548, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:01:16.842574+0000', 'last_change': '2024-09-10T15:59:16.331899+0000', 'last_active': '2024-09-10T16:01:16.842574+0000', 'last_peered': '2024-09-10T16:01:16.842574+0000', 'last_clean': '2024-09-10T15:59:14.139436+0000', 'last_became_active': '2024-09-10T15:59:16.318715+0000', 'last_became_peered': '2024-09-10T15:59:16.318715+0000', 'last_unstale': '2024-09-10T16:01:16.842574+0000', 'last_undegraded': '2024-09-10T15:59:15.877524+0000', 'last_fullsized': '2024-09-10T15:59:15.769042+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3537", 'last_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'objects_scrubbed': 3497, 'log_size': 8415, 'log_dups_size': 0, 'ondisk_log_size': 8415, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1224, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34444230, 'num_objects': 8411, 'num_object_clones': 0, 'num_object_copies': 16822, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4237, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8411, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 4914, 'num_write_kb': 19656, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4174}, {'shards': '0,2', 'objects': 4237}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:01:21.182 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-10T16:01:21.182 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "43'7969", 'reported_seq': 8104, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:01:14.502406+0000', 'last_change': '2024-09-10T15:59:16.175680+0000', 'last_active': '2024-09-10T16:01:14.502406+0000', 'last_peered': '2024-09-10T16:01:14.502406+0000', 'last_clean': '2024-09-10T15:59:14.088187+0000', 'last_became_active': '2024-09-10T15:59:16.165810+0000', 'last_became_peered': '2024-09-10T15:59:16.165810+0000', 'last_unstale': '2024-09-10T16:01:14.502406+0000', 'last_undegraded': '2024-09-10T15:59:15.822098+0000', 'last_fullsized': '2024-09-10T15:59:15.769123+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2218", 'last_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'objects_scrubbed': 2175, 'log_size': 7969, 'log_dups_size': 0, 'ondisk_log_size': 7969, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 825, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32612835, 'num_objects': 7963, 'num_object_clones': 0, 'num_object_copies': 15926, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4124, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7963, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 5788, 'num_write_kb': 23152, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4123}, {'shards': '2', 'objects': 3840}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:01:21.182 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-10T16:01:21.182 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "43'8347", 'reported_seq': 8473, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:01:16.953117+0000', 'last_change': '2024-09-10T15:59:16.277742+0000', 'last_active': '2024-09-10T16:01:16.953117+0000', 'last_peered': '2024-09-10T16:01:16.953117+0000', 'last_clean': '2024-09-10T15:59:14.129081+0000', 'last_became_active': '2024-09-10T15:59:16.269835+0000', 'last_became_peered': '2024-09-10T15:59:16.269835+0000', 'last_unstale': '2024-09-10T16:01:16.953117+0000', 'last_undegraded': '2024-09-10T15:59:15.860857+0000', 'last_fullsized': '2024-09-10T15:59:15.771410+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3557", 'last_scrub_stamp': '2024-09-10T15:59:02.493678+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:02.493678+0000', 'objects_scrubbed': 3488, 'log_size': 8347, 'log_dups_size': 0, 'ondisk_log_size': 8347, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1472, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34156544, 'num_objects': 8339, 'num_object_clones': 0, 'num_object_copies': 16678, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4257, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8339, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4851, 'num_write_kb': 19404, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4084}, {'shards': '0,2', 'objects': 4255}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:01:21.182 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-10T16:01:21.182 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "43'8247", 'reported_seq': 8377, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:01:14.576432+0000', 'last_change': '2024-09-10T15:59:16.284311+0000', 'last_active': '2024-09-10T16:01:14.576432+0000', 'last_peered': '2024-09-10T16:01:14.576432+0000', 'last_clean': '2024-09-10T15:59:14.116041+0000', 'last_became_active': '2024-09-10T15:59:16.274543+0000', 'last_became_peered': '2024-09-10T15:59:16.274543+0000', 'last_unstale': '2024-09-10T16:01:14.576432+0000', 'last_undegraded': '2024-09-10T15:59:15.812484+0000', 'last_fullsized': '2024-09-10T15:59:15.770885+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2356", 'last_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'objects_scrubbed': 2316, 'log_size': 8247, 'log_dups_size': 0, 'ondisk_log_size': 8247, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 783, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33759715, 'num_objects': 8243, 'num_object_clones': 0, 'num_object_copies': 16486, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8243, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 5927, 'num_write_kb': 23708, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4304}, {'shards': '2', 'objects': 3939}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:01:21.182 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-10T16:01:21.183 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "43'8398", 'reported_seq': 8520, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:01:14.598066+0000', 'last_change': '2024-09-10T15:59:16.174113+0000', 'last_active': '2024-09-10T16:01:14.598066+0000', 'last_peered': '2024-09-10T16:01:14.598066+0000', 'last_clean': '2024-09-10T15:59:14.110654+0000', 'last_became_active': '2024-09-10T15:59:16.164645+0000', 'last_became_peered': '2024-09-10T15:59:16.164645+0000', 'last_unstale': '2024-09-10T16:01:14.598066+0000', 'last_undegraded': '2024-09-10T15:59:15.803593+0000', 'last_fullsized': '2024-09-10T15:59:15.769083+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2486", 'last_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'objects_scrubbed': 2440, 'log_size': 8398, 'log_dups_size': 0, 'ondisk_log_size': 8398, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 792, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34373632, 'num_objects': 8392, 'num_object_clones': 0, 'num_object_copies': 16784, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4354, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8392, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5952, 'num_write_kb': 23808, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4352}, {'shards': '2', 'objects': 4040}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:01:21.183 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-10T16:01:21.183 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "43'8335", 'reported_seq': 8465, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:01:16.951094+0000', 'last_change': '2024-09-10T15:59:16.315641+0000', 'last_active': '2024-09-10T16:01:16.951094+0000', 'last_peered': '2024-09-10T16:01:16.951094+0000', 'last_clean': '2024-09-10T15:59:14.117001+0000', 'last_became_active': '2024-09-10T15:59:16.302245+0000', 'last_became_peered': '2024-09-10T15:59:16.302245+0000', 'last_unstale': '2024-09-10T16:01:16.951094+0000', 'last_undegraded': '2024-09-10T15:59:15.845012+0000', 'last_fullsized': '2024-09-10T15:59:15.770337+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3404", 'last_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'objects_scrubbed': 3367, 'log_size': 8335, 'log_dups_size': 0, 'ondisk_log_size': 8335, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 983, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34131968, 'num_objects': 8333, 'num_object_clones': 0, 'num_object_copies': 16666, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8333, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4966, 'num_write_kb': 19864, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4029}, {'shards': '0,2', 'objects': 4304}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:01:21.183 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-10T16:01:21.183 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "43'8203", 'reported_seq': 8341, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:01:16.942618+0000', 'last_change': '2024-09-10T15:59:16.251610+0000', 'last_active': '2024-09-10T16:01:16.942618+0000', 'last_peered': '2024-09-10T16:01:16.942618+0000', 'last_clean': '2024-09-10T15:59:14.115595+0000', 'last_became_active': '2024-09-10T15:59:16.244089+0000', 'last_became_peered': '2024-09-10T15:59:16.244089+0000', 'last_unstale': '2024-09-10T16:01:16.942618+0000', 'last_undegraded': '2024-09-10T15:59:15.847127+0000', 'last_fullsized': '2024-09-10T15:59:15.774437+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3653", 'last_scrub_stamp': '2024-09-10T15:59:05.298468+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:05.298468+0000', 'objects_scrubbed': 3602, 'log_size': 8203, 'log_dups_size': 0, 'ondisk_log_size': 8203, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1289, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33566720, 'num_objects': 8195, 'num_object_clones': 0, 'num_object_copies': 16390, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4206, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8195, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4593, 'num_write_kb': 18372, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3992}, {'shards': '0,2', 'objects': 4203}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:01:21.183 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-10T16:01:21.183 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "43'8466", 'reported_seq': 8601, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:01:16.937870+0000', 'last_change': '2024-09-10T15:59:16.311291+0000', 'last_active': '2024-09-10T16:01:16.937870+0000', 'last_peered': '2024-09-10T16:01:16.937870+0000', 'last_clean': '2024-09-10T15:59:14.116363+0000', 'last_became_active': '2024-09-10T15:59:16.301244+0000', 'last_became_peered': '2024-09-10T15:59:16.301244+0000', 'last_unstale': '2024-09-10T16:01:16.937870+0000', 'last_undegraded': '2024-09-10T15:59:15.883434+0000', 'last_fullsized': '2024-09-10T15:59:15.772756+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3789", 'last_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'objects_scrubbed': 3741, 'log_size': 8466, 'log_dups_size': 0, 'ondisk_log_size': 8466, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1440, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34660352, 'num_objects': 8462, 'num_object_clones': 0, 'num_object_copies': 16924, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4373, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8462, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4721, 'num_write_kb': 18884, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4091}, {'shards': '0,2', 'objects': 4371}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:01:21.183 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-10T16:01:21.183 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "43'8495", 'reported_seq': 8635, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:01:16.906623+0000', 'last_change': '2024-09-10T15:59:16.313409+0000', 'last_active': '2024-09-10T16:01:16.906623+0000', 'last_peered': '2024-09-10T16:01:16.906623+0000', 'last_clean': '2024-09-10T15:59:14.129780+0000', 'last_became_active': '2024-09-10T15:59:16.300932+0000', 'last_became_peered': '2024-09-10T15:59:16.300932+0000', 'last_unstale': '2024-09-10T16:01:16.906623+0000', 'last_undegraded': '2024-09-10T15:59:15.887823+0000', 'last_fullsized': '2024-09-10T15:59:15.768513+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3880", 'last_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'objects_scrubbed': 3827, 'log_size': 8495, 'log_dups_size': 0, 'ondisk_log_size': 8495, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1397, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34747334, 'num_objects': 8485, 'num_object_clones': 0, 'num_object_copies': 16970, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4325, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8485, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4658, 'num_write_kb': 18632, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4162}, {'shards': '0,2', 'objects': 4323}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:01:21.184 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T16:01:21.862 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:01:21.862 INFO:teuthology.orchestra.run.smithi115.stdout:{"fsid":"7effee29-df05-4351-9741-ee203564ed81","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 47117/268092 objects degraded (17.575%), 11 pgs degraded, 12 pgs undersized","count":23},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":245,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":43,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983954,"num_in_osds":3,"osd_in_since":1725983947,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":134046,"data_bytes":549008113,"bytes_used":1868185600,"bytes_avail":288042106880,"bytes_total":289910292480,"degraded_objects":47117,"degraded_total":268092,"degraded_ratio":0.17574936961938442,"misplaced_objects":1409,"misplaced_total":268092,"misplaced_ratio":0.0052556585052892287,"recovering_objects_per_sec":32,"recovering_bytes_per_sec":132361,"recovering_keys_per_sec":0,"num_objects_recovered":388,"num_bytes_recovered":1589248,"num_keys_recovered":0,"write_bytes_sec":1957792,"read_op_per_sec":0,"write_op_per_sec":477},"fsmap":{"epoch":1,"btime":"2024-09-10T15:57:14:967120+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T16:00:13.560776+0000","services":{}},"progress_events":{}} 2024-09-10T16:01:21.891 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-10T16:01:21.891 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:01:21.985 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 169 20 135871 135851 3.13958 1.70312 0.0178234 0.0248767 2024-09-10T16:01:22.457 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:01:22.457 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:01:22.481 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":159,"stamp":"2024-09-10T16:01:21.588073+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":549008113,"num_objects":134046,"num_object_clones":0,"num_object_copies":268092,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":47117,"num_objects_misplaced":1409,"num_objects_unfound":0,"num_objects_dirty":134046,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":91892,"num_write_kb":367562,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2865,"num_bytes_recovered":11722771,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":134119,"ondisk_log_size":134119,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1824400,"kb_used_data":1085212,"kb_used_omap":12,"kb_used_meta":739123,"kb_avail":281291120,"statfs":{"total":289910292480,"available":288042106880,"internally_reserved":0,"allocated":1111257088,"data_stored":1110372298,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":756861954},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,2,4,7,8,8,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":23506944,"num_objects":5739,"num_object_clones":0,"num_object_copies":11478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-388,"num_objects_unfound":0,"num_objects_dirty":5739,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5739,"num_write_kb":22956,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":388,"num_bytes_recovered":1589248,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006860"},"pg_stats":[{"pgid":"2.f","version":"43'8271","reported_seq":8340,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:16.899830+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:01:16.899830+0000","last_peered":"2024-09-10T16:01:16.899830+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:01:16.899830+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":8271,"log_dups_size":0,"ondisk_log_size":8271,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33878016,"num_objects":8271,"num_object_clones":0,"num_object_copies":16542,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8271,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8271,"num_write_kb":33084,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4012},{"shards":"0,2","objects":4258}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'8429","reported_seq":8537,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:01:19.641108+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:01:19.641108+0000","last_peered":"2024-09-10T16:01:19.641108+0000","last_clean":"2024-09-10T16:01:19.641108+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:01:19.641108+0000","last_undegraded":"2024-09-10T16:01:19.641108+0000","last_fullsized":"2024-09-10T16:01:19.641108+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":8429,"log_dups_size":0,"ondisk_log_size":8429,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34501574,"num_objects":8425,"num_object_clones":0,"num_object_copies":16850,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8425,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":6534,"num_write_kb":26136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'8370","reported_seq":14216,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T16:01:19.599981+0000","last_change":"2024-09-10T15:59:16.142612+0000","last_active":"2024-09-10T16:01:19.599981+0000","last_peered":"2024-09-10T16:01:19.599981+0000","last_clean":"2024-09-10T15:59:14.092473+0000","last_became_active":"2024-09-10T15:59:16.109611+0000","last_became_peered":"2024-09-10T15:59:16.109611+0000","last_unstale":"2024-09-10T16:01:19.599981+0000","last_undegraded":"2024-09-10T16:01:19.599981+0000","last_fullsized":"2024-09-10T15:59:15.770264+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":8370,"log_dups_size":0,"ondisk_log_size":8370,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34267136,"num_objects":8366,"num_object_clones":0,"num_object_copies":16732,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1409,"num_objects_unfound":0,"num_objects_dirty":8366,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6374,"num_write_kb":25496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2860,"num_bytes_recovered":11710464,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1409},{"shards":"2","objects":6957}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'8375","reported_seq":8451,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:01:19.488672+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:01:19.488672+0000","last_peered":"2024-09-10T16:01:19.488672+0000","last_clean":"2024-09-10T16:01:19.488672+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:01:19.488672+0000","last_undegraded":"2024-09-10T16:01:19.488672+0000","last_fullsized":"2024-09-10T16:01:19.488672+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":8375,"log_dups_size":0,"ondisk_log_size":8375,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34304000,"num_objects":8375,"num_object_clones":0,"num_object_copies":16750,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8375,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8375,"num_write_kb":33500,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'8384","reported_seq":8509,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:01:16.867283+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:01:16.867283+0000","last_peered":"2024-09-10T16:01:16.867283+0000","last_clean":"2024-09-10T16:01:16.867283+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:01:16.867283+0000","last_undegraded":"2024-09-10T16:01:16.867283+0000","last_fullsized":"2024-09-10T16:01:16.867283+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":8384,"log_dups_size":0,"ondisk_log_size":8384,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34312675,"num_objects":8378,"num_object_clones":0,"num_object_copies":16756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8378,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5095,"num_write_kb":20380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'8423","reported_seq":8554,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:16.922035+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:01:16.922035+0000","last_peered":"2024-09-10T16:01:16.922035+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:01:16.922035+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":8423,"log_dups_size":0,"ondisk_log_size":8423,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34480611,"num_objects":8419,"num_object_clones":0,"num_object_copies":16838,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8419,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5035,"num_write_kb":20140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4048},{"shards":"0,2","objects":4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'8415","reported_seq":8548,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:16.842574+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:01:16.842574+0000","last_peered":"2024-09-10T16:01:16.842574+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:01:16.842574+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":8415,"log_dups_size":0,"ondisk_log_size":8415,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34444230,"num_objects":8411,"num_object_clones":0,"num_object_copies":16822,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8411,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4914,"num_write_kb":19656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4174},{"shards":"0,2","objects":4237}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'8509","reported_seq":8634,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:01:16.922155+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:01:16.922155+0000","last_peered":"2024-09-10T16:01:16.922155+0000","last_clean":"2024-09-10T16:01:16.922155+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:01:16.922155+0000","last_undegraded":"2024-09-10T16:01:16.922155+0000","last_fullsized":"2024-09-10T16:01:16.922155+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":8509,"log_dups_size":0,"ondisk_log_size":8509,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34844672,"num_objects":8507,"num_object_clones":0,"num_object_copies":17014,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8507,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5356,"num_write_kb":21424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.777326+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T15:59:15.777326+0000","last_peered":"2024-09-10T15:59:15.777326+0000","last_clean":"2024-09-10T15:59:15.777326+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T15:59:15.777326+0000","last_undegraded":"2024-09-10T15:59:15.777326+0000","last_fullsized":"2024-09-10T15:59:15.777326+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'8123","reported_seq":8257,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:19.606338+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:01:19.606338+0000","last_peered":"2024-09-10T16:01:19.606338+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:01:19.606338+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":8123,"log_dups_size":0,"ondisk_log_size":8123,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33243619,"num_objects":8117,"num_object_clones":0,"num_object_copies":16234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8117,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5942,"num_write_kb":23768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4123},{"shards":"2","objects":3994}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":89,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.772619+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T15:59:15.772619+0000","last_peered":"2024-09-10T15:59:15.772619+0000","last_clean":"2024-09-10T15:59:15.772619+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T15:59:15.772619+0000","last_undegraded":"2024-09-10T15:59:15.772619+0000","last_fullsized":"2024-09-10T15:59:15.772619+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'8347","reported_seq":8473,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:16.953117+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:01:16.953117+0000","last_peered":"2024-09-10T16:01:16.953117+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:01:16.953117+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":8347,"log_dups_size":0,"ondisk_log_size":8347,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34156544,"num_objects":8339,"num_object_clones":0,"num_object_copies":16678,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8339,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4851,"num_write_kb":19404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4084},{"shards":"0,2","objects":4255}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":99,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.884297+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T15:59:15.884297+0000","last_peered":"2024-09-10T15:59:15.884297+0000","last_clean":"2024-09-10T15:59:15.884297+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T15:59:15.884297+0000","last_undegraded":"2024-09-10T15:59:15.884297+0000","last_fullsized":"2024-09-10T15:59:15.884297+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'8398","reported_seq":8528,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:19.543969+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:01:19.543969+0000","last_peered":"2024-09-10T16:01:19.543969+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:01:19.543969+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":8398,"log_dups_size":0,"ondisk_log_size":8398,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34378211,"num_objects":8394,"num_object_clones":0,"num_object_copies":16788,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8394,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6078,"num_write_kb":24312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4304},{"shards":"2","objects":4090}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.848670+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T15:59:15.848670+0000","last_peered":"2024-09-10T15:59:15.848670+0000","last_clean":"2024-09-10T15:59:15.848670+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T15:59:15.848670+0000","last_undegraded":"2024-09-10T15:59:15.848670+0000","last_fullsized":"2024-09-10T15:59:15.848670+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'8573","reported_seq":8697,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:19.635155+0000","last_change":"2024-09-10T15:59:16.174113+0000","last_active":"2024-09-10T16:01:19.635155+0000","last_peered":"2024-09-10T16:01:19.635155+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:01:19.635155+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":8573,"log_dups_size":0,"ondisk_log_size":8573,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35090432,"num_objects":8567,"num_object_clones":0,"num_object_copies":17134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4354,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8567,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6127,"num_write_kb":24508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4352},{"shards":"2","objects":4215}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'8335","reported_seq":8465,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:16.951094+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:01:16.951094+0000","last_peered":"2024-09-10T16:01:16.951094+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:01:16.951094+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":8335,"log_dups_size":0,"ondisk_log_size":8335,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34131968,"num_objects":8333,"num_object_clones":0,"num_object_copies":16666,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8333,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4966,"num_write_kb":19864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4029},{"shards":"0,2","objects":4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.992406+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T15:59:15.992406+0000","last_peered":"2024-09-10T15:59:15.992406+0000","last_clean":"2024-09-10T15:59:15.992406+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T15:59:15.992406+0000","last_undegraded":"2024-09-10T15:59:15.992406+0000","last_fullsized":"2024-09-10T15:59:15.992406+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'8203","reported_seq":8341,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:16.942618+0000","last_change":"2024-09-10T15:59:16.251610+0000","last_active":"2024-09-10T16:01:16.942618+0000","last_peered":"2024-09-10T16:01:16.942618+0000","last_clean":"2024-09-10T15:59:14.115595+0000","last_became_active":"2024-09-10T15:59:16.244089+0000","last_became_peered":"2024-09-10T15:59:16.244089+0000","last_unstale":"2024-09-10T16:01:16.942618+0000","last_undegraded":"2024-09-10T15:59:15.847127+0000","last_fullsized":"2024-09-10T15:59:15.774437+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":8203,"log_dups_size":0,"ondisk_log_size":8203,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33566720,"num_objects":8195,"num_object_clones":0,"num_object_copies":16390,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8195,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4593,"num_write_kb":18372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3992},{"shards":"0,2","objects":4203}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":89,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.773633+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T15:59:15.773633+0000","last_peered":"2024-09-10T15:59:15.773633+0000","last_clean":"2024-09-10T15:59:15.773633+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T15:59:15.773633+0000","last_undegraded":"2024-09-10T15:59:15.773633+0000","last_fullsized":"2024-09-10T15:59:15.773633+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'8466","reported_seq":8601,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:16.937870+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:01:16.937870+0000","last_peered":"2024-09-10T16:01:16.937870+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:01:16.937870+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":8466,"log_dups_size":0,"ondisk_log_size":8466,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34660352,"num_objects":8462,"num_object_clones":0,"num_object_copies":16924,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8462,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4721,"num_write_kb":18884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4091},{"shards":"0,2","objects":4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.917634+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T15:59:15.917634+0000","last_peered":"2024-09-10T15:59:15.917634+0000","last_clean":"2024-09-10T15:59:15.917634+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T15:59:15.917634+0000","last_undegraded":"2024-09-10T15:59:15.917634+0000","last_fullsized":"2024-09-10T15:59:15.917634+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.776033+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T15:59:15.776033+0000","last_peered":"2024-09-10T15:59:15.776033+0000","last_clean":"2024-09-10T15:59:15.776033+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T15:59:15.776033+0000","last_undegraded":"2024-09-10T15:59:15.776033+0000","last_fullsized":"2024-09-10T15:59:15.776033+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'8495","reported_seq":8635,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:16.906623+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:01:16.906623+0000","last_peered":"2024-09-10T16:01:16.906623+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:01:16.906623+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":8495,"log_dups_size":0,"ondisk_log_size":8495,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34747334,"num_objects":8485,"num_object_clones":0,"num_object_copies":16970,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8485,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4658,"num_write_kb":18632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4162},{"shards":"0,2","objects":4323}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":549008094,"num_objects":134044,"num_object_clones":0,"num_object_copies":268088,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":47117,"num_objects_misplaced":1409,"num_objects_unfound":0,"num_objects_dirty":134044,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":91890,"num_write_kb":367560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2863,"num_bytes_recovered":11722752,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1111752704,"data_stored":1111600958,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":134116,"ondisk_log_size":134116,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986216,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":676960,"kb_used_data":406480,"kb_used_omap":4,"kb_used_meta":270459,"kb_avail":93694880,"statfs":{"total":96636764160,"available":95943557120,"internally_reserved":0,"allocated":416235520,"data_stored":415957760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":276950698},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,1,0,2,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659162,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":394044,"kb_used_data":209640,"kb_used_omap":4,"kb_used_meta":184379,"kb_avail":93977796,"statfs":{"total":96636764160,"available":96233263104,"internally_reserved":0,"allocated":214671360,"data_stored":214346167,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":188804782},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,2,1,1,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182444,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":753396,"kb_used_data":469092,"kb_used_omap":4,"kb_used_meta":284283,"kb_avail":93618444,"statfs":{"total":96636764160,"available":95865286656,"internally_reserved":0,"allocated":480350208,"data_stored":480068371,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":291106474},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,0,1,6,5,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":481165312,"data_stored":481129182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":214364160,"data_stored":214284674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":416223232,"data_stored":416187102,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:01:22.986 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 170 20 136332 136312 3.13171 1.80078 0.0376357 0.0249388 2024-09-10T16:01:23.986 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 171 20 136835 136815 3.12488 1.96484 0.0861343 0.024994 2024-09-10T16:01:24.986 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 172 20 137332 137312 3.118 1.94141 0.0247577 0.0250478 2024-09-10T16:01:25.483 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:01:25.986 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 173 20 137807 137787 3.1107 1.85547 0.0227015 0.0251062 2024-09-10T16:01:26.028 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:01:26.028 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:01:26.061 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":161,"stamp":"2024-09-10T16:01:25.589587+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":558449393,"num_objects":136351,"num_object_clones":0,"num_object_copies":272702,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":47117,"num_objects_misplaced":1287,"num_objects_unfound":0,"num_objects_dirty":136351,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":94197,"num_write_kb":376782,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2987,"num_bytes_recovered":12222483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":136424,"ondisk_log_size":136424,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1857348,"kb_used_data":1104276,"kb_used_omap":12,"kb_used_meta":753011,"kb_avail":281258172,"statfs":{"total":289910292480,"available":288008368128,"internally_reserved":0,"allocated":1130778624,"data_stored":1129893834,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":771083266},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,1,3,12,15,4,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":23076864,"num_objects":5634,"num_object_clones":0,"num_object_copies":11268,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-382,"num_objects_unfound":0,"num_objects_dirty":5634,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5634,"num_write_kb":22536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":382,"num_bytes_recovered":1564672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004487"},"pg_stats":[{"pgid":"2.f","version":"43'8410","reported_seq":8479,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:21.926675+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:01:21.926675+0000","last_peered":"2024-09-10T16:01:21.926675+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:01:21.926675+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":8410,"log_dups_size":0,"ondisk_log_size":8410,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34447360,"num_objects":8410,"num_object_clones":0,"num_object_copies":16820,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8410,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8410,"num_write_kb":33640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4151},{"shards":"0,2","objects":4258}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'8559","reported_seq":8667,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:01:24.489805+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:01:24.489805+0000","last_peered":"2024-09-10T16:01:24.489805+0000","last_clean":"2024-09-10T16:01:24.489805+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:01:24.489805+0000","last_undegraded":"2024-09-10T16:01:24.489805+0000","last_fullsized":"2024-09-10T16:01:24.489805+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":8559,"log_dups_size":0,"ondisk_log_size":8559,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35034054,"num_objects":8555,"num_object_clones":0,"num_object_copies":17110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8555,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":6664,"num_write_kb":26656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'8532","reported_seq":14622,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T16:01:24.605856+0000","last_change":"2024-09-10T15:59:16.142612+0000","last_active":"2024-09-10T16:01:24.605856+0000","last_peered":"2024-09-10T16:01:24.605856+0000","last_clean":"2024-09-10T15:59:14.092473+0000","last_became_active":"2024-09-10T15:59:16.109611+0000","last_became_peered":"2024-09-10T15:59:16.109611+0000","last_unstale":"2024-09-10T16:01:24.605856+0000","last_undegraded":"2024-09-10T16:01:24.605856+0000","last_fullsized":"2024-09-10T15:59:15.770264+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":8532,"log_dups_size":0,"ondisk_log_size":8532,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34930688,"num_objects":8528,"num_object_clones":0,"num_object_copies":17056,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1287,"num_objects_unfound":0,"num_objects_dirty":8528,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6536,"num_write_kb":26144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2982,"num_bytes_recovered":12210176,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1287},{"shards":"2","objects":7241}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'8514","reported_seq":8590,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:01:24.561991+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:01:24.561991+0000","last_peered":"2024-09-10T16:01:24.561991+0000","last_clean":"2024-09-10T16:01:24.561991+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:01:24.561991+0000","last_undegraded":"2024-09-10T16:01:24.561991+0000","last_fullsized":"2024-09-10T16:01:24.561991+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":8514,"log_dups_size":0,"ondisk_log_size":8514,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34873344,"num_objects":8514,"num_object_clones":0,"num_object_copies":17028,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8514,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8514,"num_write_kb":34056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'8524","reported_seq":8649,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:01:21.906171+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:01:21.906171+0000","last_peered":"2024-09-10T16:01:21.906171+0000","last_clean":"2024-09-10T16:01:21.906171+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:01:21.906171+0000","last_undegraded":"2024-09-10T16:01:21.906171+0000","last_fullsized":"2024-09-10T16:01:21.906171+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":8524,"log_dups_size":0,"ondisk_log_size":8524,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34886115,"num_objects":8518,"num_object_clones":0,"num_object_copies":17036,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8518,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5235,"num_write_kb":20940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'8546","reported_seq":8679,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:21.824954+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:01:21.824954+0000","last_peered":"2024-09-10T16:01:21.824954+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:01:21.824954+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":8546,"log_dups_size":0,"ondisk_log_size":8546,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34984419,"num_objects":8542,"num_object_clones":0,"num_object_copies":17084,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8542,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5158,"num_write_kb":20632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4171},{"shards":"0,2","objects":4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'8556","reported_seq":8689,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:21.962169+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:01:21.962169+0000","last_peered":"2024-09-10T16:01:21.962169+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:01:21.962169+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":8556,"log_dups_size":0,"ondisk_log_size":8556,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35021766,"num_objects":8552,"num_object_clones":0,"num_object_copies":17104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8552,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5055,"num_write_kb":20220,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4315},{"shards":"0,2","objects":4237}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'8665","reported_seq":8790,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:01:21.983095+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:01:21.983095+0000","last_peered":"2024-09-10T16:01:21.983095+0000","last_clean":"2024-09-10T16:01:21.983095+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:01:21.983095+0000","last_undegraded":"2024-09-10T16:01:21.983095+0000","last_fullsized":"2024-09-10T16:01:21.983095+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":8665,"log_dups_size":0,"ondisk_log_size":8665,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35483648,"num_objects":8663,"num_object_clones":0,"num_object_copies":17326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8663,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5512,"num_write_kb":22048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.777326+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T15:59:15.777326+0000","last_peered":"2024-09-10T15:59:15.777326+0000","last_clean":"2024-09-10T15:59:15.777326+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T15:59:15.777326+0000","last_undegraded":"2024-09-10T15:59:15.777326+0000","last_fullsized":"2024-09-10T15:59:15.777326+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'8266","reported_seq":8401,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:24.595151+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:01:24.595151+0000","last_peered":"2024-09-10T16:01:24.595151+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:01:24.595151+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":8266,"log_dups_size":0,"ondisk_log_size":8266,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33829347,"num_objects":8260,"num_object_clones":0,"num_object_copies":16520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8260,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6085,"num_write_kb":24340,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4123},{"shards":"2","objects":4137}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":89,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.772619+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T15:59:15.772619+0000","last_peered":"2024-09-10T15:59:15.772619+0000","last_clean":"2024-09-10T15:59:15.772619+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T15:59:15.772619+0000","last_undegraded":"2024-09-10T15:59:15.772619+0000","last_fullsized":"2024-09-10T15:59:15.772619+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'8490","reported_seq":8615,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:21.900063+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:01:21.900063+0000","last_peered":"2024-09-10T16:01:21.900063+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:01:21.900063+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":8490,"log_dups_size":0,"ondisk_log_size":8490,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34742272,"num_objects":8482,"num_object_clones":0,"num_object_copies":16964,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8482,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4994,"num_write_kb":19976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4227},{"shards":"0,2","objects":4255}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":99,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.884297+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T15:59:15.884297+0000","last_peered":"2024-09-10T15:59:15.884297+0000","last_clean":"2024-09-10T15:59:15.884297+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T15:59:15.884297+0000","last_undegraded":"2024-09-10T15:59:15.884297+0000","last_fullsized":"2024-09-10T15:59:15.884297+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'8540","reported_seq":8670,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:24.634220+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:01:24.634220+0000","last_peered":"2024-09-10T16:01:24.634220+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:01:24.634220+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":8540,"log_dups_size":0,"ondisk_log_size":8540,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34959843,"num_objects":8536,"num_object_clones":0,"num_object_copies":17072,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8536,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6220,"num_write_kb":24880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4304},{"shards":"2","objects":4232}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.848670+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T15:59:15.848670+0000","last_peered":"2024-09-10T15:59:15.848670+0000","last_clean":"2024-09-10T15:59:15.848670+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T15:59:15.848670+0000","last_undegraded":"2024-09-10T15:59:15.848670+0000","last_fullsized":"2024-09-10T15:59:15.848670+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'8722","reported_seq":8845,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:24.613111+0000","last_change":"2024-09-10T15:59:16.174113+0000","last_active":"2024-09-10T16:01:24.613111+0000","last_peered":"2024-09-10T16:01:24.613111+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:01:24.613111+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":8722,"log_dups_size":0,"ondisk_log_size":8722,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35700736,"num_objects":8716,"num_object_clones":0,"num_object_copies":17432,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4354,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8716,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6276,"num_write_kb":25104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4352},{"shards":"2","objects":4364}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'8479","reported_seq":8609,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:21.934178+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:01:21.934178+0000","last_peered":"2024-09-10T16:01:21.934178+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:01:21.934178+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":8479,"log_dups_size":0,"ondisk_log_size":8479,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34721792,"num_objects":8477,"num_object_clones":0,"num_object_copies":16954,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8477,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5110,"num_write_kb":20440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4173},{"shards":"0,2","objects":4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.992406+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T15:59:15.992406+0000","last_peered":"2024-09-10T15:59:15.992406+0000","last_clean":"2024-09-10T15:59:15.992406+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T15:59:15.992406+0000","last_undegraded":"2024-09-10T15:59:15.992406+0000","last_fullsized":"2024-09-10T15:59:15.992406+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'8359","reported_seq":8497,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:21.974161+0000","last_change":"2024-09-10T15:59:16.251610+0000","last_active":"2024-09-10T16:01:21.974161+0000","last_peered":"2024-09-10T16:01:21.974161+0000","last_clean":"2024-09-10T15:59:14.115595+0000","last_became_active":"2024-09-10T15:59:16.244089+0000","last_became_peered":"2024-09-10T15:59:16.244089+0000","last_unstale":"2024-09-10T16:01:21.974161+0000","last_undegraded":"2024-09-10T15:59:15.847127+0000","last_fullsized":"2024-09-10T15:59:15.774437+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":8359,"log_dups_size":0,"ondisk_log_size":8359,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34205696,"num_objects":8351,"num_object_clones":0,"num_object_copies":16702,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8351,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4749,"num_write_kb":18996,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4148},{"shards":"0,2","objects":4203}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":89,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.773633+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T15:59:15.773633+0000","last_peered":"2024-09-10T15:59:15.773633+0000","last_clean":"2024-09-10T15:59:15.773633+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T15:59:15.773633+0000","last_undegraded":"2024-09-10T15:59:15.773633+0000","last_fullsized":"2024-09-10T15:59:15.773633+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'8625","reported_seq":8762,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:21.980523+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:01:21.980523+0000","last_peered":"2024-09-10T16:01:21.980523+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:01:21.980523+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":8625,"log_dups_size":0,"ondisk_log_size":8625,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35311616,"num_objects":8621,"num_object_clones":0,"num_object_copies":17242,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8621,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4880,"num_write_kb":19520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4250},{"shards":"0,2","objects":4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.917634+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T15:59:15.917634+0000","last_peered":"2024-09-10T15:59:15.917634+0000","last_clean":"2024-09-10T15:59:15.917634+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T15:59:15.917634+0000","last_undegraded":"2024-09-10T15:59:15.917634+0000","last_fullsized":"2024-09-10T15:59:15.917634+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":70,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.778232+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T15:59:15.778232+0000","last_peered":"2024-09-10T15:59:15.778232+0000","last_clean":"2024-09-10T15:59:15.778232+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T15:59:15.778232+0000","last_undegraded":"2024-09-10T15:59:15.778232+0000","last_fullsized":"2024-09-10T15:59:15.778232+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'8634","reported_seq":8773,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:21.937698+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:01:21.937698+0000","last_peered":"2024-09-10T16:01:21.937698+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:01:21.937698+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":8634,"log_dups_size":0,"ondisk_log_size":8634,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35316678,"num_objects":8624,"num_object_clones":0,"num_object_copies":17248,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8624,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4797,"num_write_kb":19188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4301},{"shards":"0,2","objects":4323}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":558449374,"num_objects":136349,"num_object_clones":0,"num_object_copies":272698,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":47117,"num_objects_misplaced":1287,"num_objects_unfound":0,"num_objects_dirty":136349,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":94195,"num_write_kb":376780,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2985,"num_bytes_recovered":12222464,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1131360256,"data_stored":1131208510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":136421,"ondisk_log_size":136421,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986217,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":685828,"kb_used_data":411188,"kb_used_omap":4,"kb_used_meta":274619,"kb_avail":93686012,"statfs":{"total":96636764160,"available":95934476288,"internally_reserved":0,"allocated":421056512,"data_stored":420778752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":281210538},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,1,0,3,4,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659163,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":389100,"kb_used_data":217180,"kb_used_omap":4,"kb_used_meta":171899,"kb_avail":93982740,"statfs":{"total":96636764160,"available":96238325760,"internally_reserved":0,"allocated":222392320,"data_stored":222067127,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":176025262},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,6,6,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182445,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":782420,"kb_used_data":475908,"kb_used_omap":4,"kb_used_meta":306491,"kb_avail":93589420,"statfs":{"total":96636764160,"available":95835566080,"internally_reserved":0,"allocated":487329792,"data_stored":487047955,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":313847466},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,3,5,1,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":488316928,"data_stored":488280798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":222035968,"data_stored":221956482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":421007360,"data_stored":420971230,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:01:26.063 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:01:26.622 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:01:26.623 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:01:26.646 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":161,"stamp":"2024-09-10T16:01:25.589587+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":558449393,"num_objects":136351,"num_object_clones":0,"num_object_copies":272702,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":47117,"num_objects_misplaced":1287,"num_objects_unfound":0,"num_objects_dirty":136351,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":94197,"num_write_kb":376782,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2987,"num_bytes_recovered":12222483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":136424,"ondisk_log_size":136424,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1857348,"kb_used_data":1104276,"kb_used_omap":12,"kb_used_meta":753011,"kb_avail":281258172,"statfs":{"total":289910292480,"available":288008368128,"internally_reserved":0,"allocated":1130778624,"data_stored":1129893834,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":771083266},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,1,3,12,15,4,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":23076864,"num_objects":5634,"num_object_clones":0,"num_object_copies":11268,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-382,"num_objects_unfound":0,"num_objects_dirty":5634,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5634,"num_write_kb":22536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":382,"num_bytes_recovered":1564672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004487"},"pg_stats":[{"pgid":"2.f","version":"43'8410","reported_seq":8479,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:21.926675+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:01:21.926675+0000","last_peered":"2024-09-10T16:01:21.926675+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:01:21.926675+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":8410,"log_dups_size":0,"ondisk_log_size":8410,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34447360,"num_objects":8410,"num_object_clones":0,"num_object_copies":16820,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8410,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8410,"num_write_kb":33640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4151},{"shards":"0,2","objects":4258}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'8559","reported_seq":8667,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:01:24.489805+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:01:24.489805+0000","last_peered":"2024-09-10T16:01:24.489805+0000","last_clean":"2024-09-10T16:01:24.489805+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:01:24.489805+0000","last_undegraded":"2024-09-10T16:01:24.489805+0000","last_fullsized":"2024-09-10T16:01:24.489805+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":8559,"log_dups_size":0,"ondisk_log_size":8559,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35034054,"num_objects":8555,"num_object_clones":0,"num_object_copies":17110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8555,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":6664,"num_write_kb":26656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'8532","reported_seq":14622,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T16:01:24.605856+0000","last_change":"2024-09-10T15:59:16.142612+0000","last_active":"2024-09-10T16:01:24.605856+0000","last_peered":"2024-09-10T16:01:24.605856+0000","last_clean":"2024-09-10T15:59:14.092473+0000","last_became_active":"2024-09-10T15:59:16.109611+0000","last_became_peered":"2024-09-10T15:59:16.109611+0000","last_unstale":"2024-09-10T16:01:24.605856+0000","last_undegraded":"2024-09-10T16:01:24.605856+0000","last_fullsized":"2024-09-10T15:59:15.770264+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":8532,"log_dups_size":0,"ondisk_log_size":8532,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34930688,"num_objects":8528,"num_object_clones":0,"num_object_copies":17056,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1287,"num_objects_unfound":0,"num_objects_dirty":8528,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6536,"num_write_kb":26144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2982,"num_bytes_recovered":12210176,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1287},{"shards":"2","objects":7241}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'8514","reported_seq":8590,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:01:24.561991+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:01:24.561991+0000","last_peered":"2024-09-10T16:01:24.561991+0000","last_clean":"2024-09-10T16:01:24.561991+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:01:24.561991+0000","last_undegraded":"2024-09-10T16:01:24.561991+0000","last_fullsized":"2024-09-10T16:01:24.561991+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":8514,"log_dups_size":0,"ondisk_log_size":8514,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34873344,"num_objects":8514,"num_object_clones":0,"num_object_copies":17028,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8514,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8514,"num_write_kb":34056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'8524","reported_seq":8649,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:01:21.906171+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:01:21.906171+0000","last_peered":"2024-09-10T16:01:21.906171+0000","last_clean":"2024-09-10T16:01:21.906171+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:01:21.906171+0000","last_undegraded":"2024-09-10T16:01:21.906171+0000","last_fullsized":"2024-09-10T16:01:21.906171+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":8524,"log_dups_size":0,"ondisk_log_size":8524,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34886115,"num_objects":8518,"num_object_clones":0,"num_object_copies":17036,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8518,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5235,"num_write_kb":20940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'8546","reported_seq":8679,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:21.824954+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:01:21.824954+0000","last_peered":"2024-09-10T16:01:21.824954+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:01:21.824954+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":8546,"log_dups_size":0,"ondisk_log_size":8546,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34984419,"num_objects":8542,"num_object_clones":0,"num_object_copies":17084,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8542,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5158,"num_write_kb":20632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4171},{"shards":"0,2","objects":4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'8556","reported_seq":8689,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:21.962169+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:01:21.962169+0000","last_peered":"2024-09-10T16:01:21.962169+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:01:21.962169+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":8556,"log_dups_size":0,"ondisk_log_size":8556,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35021766,"num_objects":8552,"num_object_clones":0,"num_object_copies":17104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8552,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5055,"num_write_kb":20220,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4315},{"shards":"0,2","objects":4237}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'8665","reported_seq":8790,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:01:21.983095+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:01:21.983095+0000","last_peered":"2024-09-10T16:01:21.983095+0000","last_clean":"2024-09-10T16:01:21.983095+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:01:21.983095+0000","last_undegraded":"2024-09-10T16:01:21.983095+0000","last_fullsized":"2024-09-10T16:01:21.983095+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":8665,"log_dups_size":0,"ondisk_log_size":8665,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35483648,"num_objects":8663,"num_object_clones":0,"num_object_copies":17326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8663,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5512,"num_write_kb":22048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.777326+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T15:59:15.777326+0000","last_peered":"2024-09-10T15:59:15.777326+0000","last_clean":"2024-09-10T15:59:15.777326+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T15:59:15.777326+0000","last_undegraded":"2024-09-10T15:59:15.777326+0000","last_fullsized":"2024-09-10T15:59:15.777326+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'8266","reported_seq":8401,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:24.595151+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:01:24.595151+0000","last_peered":"2024-09-10T16:01:24.595151+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:01:24.595151+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":8266,"log_dups_size":0,"ondisk_log_size":8266,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33829347,"num_objects":8260,"num_object_clones":0,"num_object_copies":16520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8260,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6085,"num_write_kb":24340,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4123},{"shards":"2","objects":4137}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":89,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.772619+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T15:59:15.772619+0000","last_peered":"2024-09-10T15:59:15.772619+0000","last_clean":"2024-09-10T15:59:15.772619+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T15:59:15.772619+0000","last_undegraded":"2024-09-10T15:59:15.772619+0000","last_fullsized":"2024-09-10T15:59:15.772619+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'8490","reported_seq":8615,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:21.900063+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:01:21.900063+0000","last_peered":"2024-09-10T16:01:21.900063+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:01:21.900063+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":8490,"log_dups_size":0,"ondisk_log_size":8490,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34742272,"num_objects":8482,"num_object_clones":0,"num_object_copies":16964,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8482,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4994,"num_write_kb":19976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4227},{"shards":"0,2","objects":4255}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":99,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.884297+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T15:59:15.884297+0000","last_peered":"2024-09-10T15:59:15.884297+0000","last_clean":"2024-09-10T15:59:15.884297+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T15:59:15.884297+0000","last_undegraded":"2024-09-10T15:59:15.884297+0000","last_fullsized":"2024-09-10T15:59:15.884297+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'8540","reported_seq":8670,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:24.634220+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:01:24.634220+0000","last_peered":"2024-09-10T16:01:24.634220+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:01:24.634220+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":8540,"log_dups_size":0,"ondisk_log_size":8540,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34959843,"num_objects":8536,"num_object_clones":0,"num_object_copies":17072,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8536,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6220,"num_write_kb":24880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4304},{"shards":"2","objects":4232}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.848670+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T15:59:15.848670+0000","last_peered":"2024-09-10T15:59:15.848670+0000","last_clean":"2024-09-10T15:59:15.848670+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T15:59:15.848670+0000","last_undegraded":"2024-09-10T15:59:15.848670+0000","last_fullsized":"2024-09-10T15:59:15.848670+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'8722","reported_seq":8845,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:24.613111+0000","last_change":"2024-09-10T15:59:16.174113+0000","last_active":"2024-09-10T16:01:24.613111+0000","last_peered":"2024-09-10T16:01:24.613111+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:01:24.613111+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":8722,"log_dups_size":0,"ondisk_log_size":8722,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35700736,"num_objects":8716,"num_object_clones":0,"num_object_copies":17432,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4354,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8716,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6276,"num_write_kb":25104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4352},{"shards":"2","objects":4364}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'8479","reported_seq":8609,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:21.934178+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:01:21.934178+0000","last_peered":"2024-09-10T16:01:21.934178+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:01:21.934178+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":8479,"log_dups_size":0,"ondisk_log_size":8479,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34721792,"num_objects":8477,"num_object_clones":0,"num_object_copies":16954,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8477,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5110,"num_write_kb":20440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4173},{"shards":"0,2","objects":4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.992406+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T15:59:15.992406+0000","last_peered":"2024-09-10T15:59:15.992406+0000","last_clean":"2024-09-10T15:59:15.992406+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T15:59:15.992406+0000","last_undegraded":"2024-09-10T15:59:15.992406+0000","last_fullsized":"2024-09-10T15:59:15.992406+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'8359","reported_seq":8497,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:21.974161+0000","last_change":"2024-09-10T15:59:16.251610+0000","last_active":"2024-09-10T16:01:21.974161+0000","last_peered":"2024-09-10T16:01:21.974161+0000","last_clean":"2024-09-10T15:59:14.115595+0000","last_became_active":"2024-09-10T15:59:16.244089+0000","last_became_peered":"2024-09-10T15:59:16.244089+0000","last_unstale":"2024-09-10T16:01:21.974161+0000","last_undegraded":"2024-09-10T15:59:15.847127+0000","last_fullsized":"2024-09-10T15:59:15.774437+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":8359,"log_dups_size":0,"ondisk_log_size":8359,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34205696,"num_objects":8351,"num_object_clones":0,"num_object_copies":16702,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8351,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4749,"num_write_kb":18996,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4148},{"shards":"0,2","objects":4203}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":89,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.773633+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T15:59:15.773633+0000","last_peered":"2024-09-10T15:59:15.773633+0000","last_clean":"2024-09-10T15:59:15.773633+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T15:59:15.773633+0000","last_undegraded":"2024-09-10T15:59:15.773633+0000","last_fullsized":"2024-09-10T15:59:15.773633+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'8625","reported_seq":8762,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:21.980523+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:01:21.980523+0000","last_peered":"2024-09-10T16:01:21.980523+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:01:21.980523+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":8625,"log_dups_size":0,"ondisk_log_size":8625,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35311616,"num_objects":8621,"num_object_clones":0,"num_object_copies":17242,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8621,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4880,"num_write_kb":19520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4250},{"shards":"0,2","objects":4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.917634+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T15:59:15.917634+0000","last_peered":"2024-09-10T15:59:15.917634+0000","last_clean":"2024-09-10T15:59:15.917634+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T15:59:15.917634+0000","last_undegraded":"2024-09-10T15:59:15.917634+0000","last_fullsized":"2024-09-10T15:59:15.917634+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":70,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.778232+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T15:59:15.778232+0000","last_peered":"2024-09-10T15:59:15.778232+0000","last_clean":"2024-09-10T15:59:15.778232+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T15:59:15.778232+0000","last_undegraded":"2024-09-10T15:59:15.778232+0000","last_fullsized":"2024-09-10T15:59:15.778232+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'8634","reported_seq":8773,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:21.937698+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:01:21.937698+0000","last_peered":"2024-09-10T16:01:21.937698+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:01:21.937698+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":8634,"log_dups_size":0,"ondisk_log_size":8634,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35316678,"num_objects":8624,"num_object_clones":0,"num_object_copies":17248,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8624,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4797,"num_write_kb":19188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4301},{"shards":"0,2","objects":4323}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":558449374,"num_objects":136349,"num_object_clones":0,"num_object_copies":272698,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":47117,"num_objects_misplaced":1287,"num_objects_unfound":0,"num_objects_dirty":136349,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":94195,"num_write_kb":376780,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2985,"num_bytes_recovered":12222464,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1131360256,"data_stored":1131208510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":136421,"ondisk_log_size":136421,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986217,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":685828,"kb_used_data":411188,"kb_used_omap":4,"kb_used_meta":274619,"kb_avail":93686012,"statfs":{"total":96636764160,"available":95934476288,"internally_reserved":0,"allocated":421056512,"data_stored":420778752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":281210538},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,1,0,3,4,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659163,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":389100,"kb_used_data":217180,"kb_used_omap":4,"kb_used_meta":171899,"kb_avail":93982740,"statfs":{"total":96636764160,"available":96238325760,"internally_reserved":0,"allocated":222392320,"data_stored":222067127,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":176025262},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,6,6,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182445,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":782420,"kb_used_data":475908,"kb_used_omap":4,"kb_used_meta":306491,"kb_avail":93589420,"statfs":{"total":96636764160,"available":95835566080,"internally_reserved":0,"allocated":487329792,"data_stored":487047955,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":313847466},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,3,5,1,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":488316928,"data_stored":488280798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":222035968,"data_stored":221956482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":421007360,"data_stored":420971230,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:01:26.647 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-10T16:01:26.647 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "43'8410", 'reported_seq': 8479, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:01:21.926675+0000', 'last_change': '2024-09-10T15:59:16.281558+0000', 'last_active': '2024-09-10T16:01:21.926675+0000', 'last_peered': '2024-09-10T16:01:21.926675+0000', 'last_clean': '2024-09-10T15:59:14.132456+0000', 'last_became_active': '2024-09-10T15:59:16.274407+0000', 'last_became_peered': '2024-09-10T15:59:16.274407+0000', 'last_unstale': '2024-09-10T16:01:21.926675+0000', 'last_undegraded': '2024-09-10T15:59:15.851279+0000', 'last_fullsized': '2024-09-10T15:59:15.774356+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'objects_scrubbed': 0, 'log_size': 8410, 'log_dups_size': 0, 'ondisk_log_size': 8410, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34447360, 'num_objects': 8410, 'num_object_clones': 0, 'num_object_copies': 16820, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4259, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8410, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8410, 'num_write_kb': 33640, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4151}, {'shards': '0,2', 'objects': 4258}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:01:26.647 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-10T16:01:26.647 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "43'8532", 'reported_seq': 14622, 'reported_epoch': 43, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-10T16:01:24.605856+0000', 'last_change': '2024-09-10T15:59:16.142612+0000', 'last_active': '2024-09-10T16:01:24.605856+0000', 'last_peered': '2024-09-10T16:01:24.605856+0000', 'last_clean': '2024-09-10T15:59:14.092473+0000', 'last_became_active': '2024-09-10T15:59:16.109611+0000', 'last_became_peered': '2024-09-10T15:59:16.109611+0000', 'last_unstale': '2024-09-10T16:01:24.605856+0000', 'last_undegraded': '2024-09-10T16:01:24.605856+0000', 'last_fullsized': '2024-09-10T15:59:15.770264+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2029", 'last_scrub_stamp': '2024-09-10T15:58:45.933789+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:45.933789+0000', 'objects_scrubbed': 1992, 'log_size': 8532, 'log_dups_size': 0, 'ondisk_log_size': 8532, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 668, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34930688, 'num_objects': 8528, 'num_object_clones': 0, 'num_object_copies': 17056, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1287, 'num_objects_unfound': 0, 'num_objects_dirty': 8528, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6536, 'num_write_kb': 26144, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2982, 'num_bytes_recovered': 12210176, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1287}, {'shards': '2', 'objects': 7241}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:01:26.648 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-10T16:01:26.648 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "43'8546", 'reported_seq': 8679, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:01:21.824954+0000', 'last_change': '2024-09-10T15:59:16.308264+0000', 'last_active': '2024-09-10T16:01:21.824954+0000', 'last_peered': '2024-09-10T16:01:21.824954+0000', 'last_clean': '2024-09-10T15:59:14.111551+0000', 'last_became_active': '2024-09-10T15:59:16.300040+0000', 'last_became_peered': '2024-09-10T15:59:16.300040+0000', 'last_unstale': '2024-09-10T16:01:21.824954+0000', 'last_undegraded': '2024-09-10T15:59:15.878234+0000', 'last_fullsized': '2024-09-10T15:59:15.769328+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3442", 'last_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'objects_scrubbed': 3384, 'log_size': 8546, 'log_dups_size': 0, 'ondisk_log_size': 8546, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1292, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34984419, 'num_objects': 8542, 'num_object_clones': 0, 'num_object_copies': 17084, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4372, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8542, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 5158, 'num_write_kb': 20632, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4171}, {'shards': '0,2', 'objects': 4371}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:01:26.648 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-10T16:01:26.648 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "43'8556", 'reported_seq': 8689, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:01:21.962169+0000', 'last_change': '2024-09-10T15:59:16.331899+0000', 'last_active': '2024-09-10T16:01:21.962169+0000', 'last_peered': '2024-09-10T16:01:21.962169+0000', 'last_clean': '2024-09-10T15:59:14.139436+0000', 'last_became_active': '2024-09-10T15:59:16.318715+0000', 'last_became_peered': '2024-09-10T15:59:16.318715+0000', 'last_unstale': '2024-09-10T16:01:21.962169+0000', 'last_undegraded': '2024-09-10T15:59:15.877524+0000', 'last_fullsized': '2024-09-10T15:59:15.769042+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3537", 'last_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'objects_scrubbed': 3497, 'log_size': 8556, 'log_dups_size': 0, 'ondisk_log_size': 8556, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1224, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35021766, 'num_objects': 8552, 'num_object_clones': 0, 'num_object_copies': 17104, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4237, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8552, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 5055, 'num_write_kb': 20220, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4315}, {'shards': '0,2', 'objects': 4237}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:01:26.648 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-10T16:01:26.648 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "43'8266", 'reported_seq': 8401, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:01:24.595151+0000', 'last_change': '2024-09-10T15:59:16.175680+0000', 'last_active': '2024-09-10T16:01:24.595151+0000', 'last_peered': '2024-09-10T16:01:24.595151+0000', 'last_clean': '2024-09-10T15:59:14.088187+0000', 'last_became_active': '2024-09-10T15:59:16.165810+0000', 'last_became_peered': '2024-09-10T15:59:16.165810+0000', 'last_unstale': '2024-09-10T16:01:24.595151+0000', 'last_undegraded': '2024-09-10T15:59:15.822098+0000', 'last_fullsized': '2024-09-10T15:59:15.769123+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2218", 'last_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'objects_scrubbed': 2175, 'log_size': 8266, 'log_dups_size': 0, 'ondisk_log_size': 8266, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 825, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33829347, 'num_objects': 8260, 'num_object_clones': 0, 'num_object_copies': 16520, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4124, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8260, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 6085, 'num_write_kb': 24340, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4123}, {'shards': '2', 'objects': 4137}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:01:26.648 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-10T16:01:26.648 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "43'8490", 'reported_seq': 8615, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:01:21.900063+0000', 'last_change': '2024-09-10T15:59:16.277742+0000', 'last_active': '2024-09-10T16:01:21.900063+0000', 'last_peered': '2024-09-10T16:01:21.900063+0000', 'last_clean': '2024-09-10T15:59:14.129081+0000', 'last_became_active': '2024-09-10T15:59:16.269835+0000', 'last_became_peered': '2024-09-10T15:59:16.269835+0000', 'last_unstale': '2024-09-10T16:01:21.900063+0000', 'last_undegraded': '2024-09-10T15:59:15.860857+0000', 'last_fullsized': '2024-09-10T15:59:15.771410+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3557", 'last_scrub_stamp': '2024-09-10T15:59:02.493678+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:02.493678+0000', 'objects_scrubbed': 3488, 'log_size': 8490, 'log_dups_size': 0, 'ondisk_log_size': 8490, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1472, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34742272, 'num_objects': 8482, 'num_object_clones': 0, 'num_object_copies': 16964, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4257, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8482, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4994, 'num_write_kb': 19976, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4227}, {'shards': '0,2', 'objects': 4255}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:01:26.648 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-10T16:01:26.648 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "43'8540", 'reported_seq': 8670, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:01:24.634220+0000', 'last_change': '2024-09-10T15:59:16.284311+0000', 'last_active': '2024-09-10T16:01:24.634220+0000', 'last_peered': '2024-09-10T16:01:24.634220+0000', 'last_clean': '2024-09-10T15:59:14.116041+0000', 'last_became_active': '2024-09-10T15:59:16.274543+0000', 'last_became_peered': '2024-09-10T15:59:16.274543+0000', 'last_unstale': '2024-09-10T16:01:24.634220+0000', 'last_undegraded': '2024-09-10T15:59:15.812484+0000', 'last_fullsized': '2024-09-10T15:59:15.770885+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2356", 'last_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'objects_scrubbed': 2316, 'log_size': 8540, 'log_dups_size': 0, 'ondisk_log_size': 8540, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 783, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34959843, 'num_objects': 8536, 'num_object_clones': 0, 'num_object_copies': 17072, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8536, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 6220, 'num_write_kb': 24880, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4304}, {'shards': '2', 'objects': 4232}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:01:26.648 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-10T16:01:26.649 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "43'8722", 'reported_seq': 8845, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:01:24.613111+0000', 'last_change': '2024-09-10T15:59:16.174113+0000', 'last_active': '2024-09-10T16:01:24.613111+0000', 'last_peered': '2024-09-10T16:01:24.613111+0000', 'last_clean': '2024-09-10T15:59:14.110654+0000', 'last_became_active': '2024-09-10T15:59:16.164645+0000', 'last_became_peered': '2024-09-10T15:59:16.164645+0000', 'last_unstale': '2024-09-10T16:01:24.613111+0000', 'last_undegraded': '2024-09-10T15:59:15.803593+0000', 'last_fullsized': '2024-09-10T15:59:15.769083+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2486", 'last_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'objects_scrubbed': 2440, 'log_size': 8722, 'log_dups_size': 0, 'ondisk_log_size': 8722, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 792, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35700736, 'num_objects': 8716, 'num_object_clones': 0, 'num_object_copies': 17432, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4354, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8716, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6276, 'num_write_kb': 25104, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4352}, {'shards': '2', 'objects': 4364}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:01:26.649 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-10T16:01:26.649 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "43'8479", 'reported_seq': 8609, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:01:21.934178+0000', 'last_change': '2024-09-10T15:59:16.315641+0000', 'last_active': '2024-09-10T16:01:21.934178+0000', 'last_peered': '2024-09-10T16:01:21.934178+0000', 'last_clean': '2024-09-10T15:59:14.117001+0000', 'last_became_active': '2024-09-10T15:59:16.302245+0000', 'last_became_peered': '2024-09-10T15:59:16.302245+0000', 'last_unstale': '2024-09-10T16:01:21.934178+0000', 'last_undegraded': '2024-09-10T15:59:15.845012+0000', 'last_fullsized': '2024-09-10T15:59:15.770337+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3404", 'last_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'objects_scrubbed': 3367, 'log_size': 8479, 'log_dups_size': 0, 'ondisk_log_size': 8479, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 983, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34721792, 'num_objects': 8477, 'num_object_clones': 0, 'num_object_copies': 16954, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8477, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5110, 'num_write_kb': 20440, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4173}, {'shards': '0,2', 'objects': 4304}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:01:26.649 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-10T16:01:26.649 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "43'8359", 'reported_seq': 8497, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:01:21.974161+0000', 'last_change': '2024-09-10T15:59:16.251610+0000', 'last_active': '2024-09-10T16:01:21.974161+0000', 'last_peered': '2024-09-10T16:01:21.974161+0000', 'last_clean': '2024-09-10T15:59:14.115595+0000', 'last_became_active': '2024-09-10T15:59:16.244089+0000', 'last_became_peered': '2024-09-10T15:59:16.244089+0000', 'last_unstale': '2024-09-10T16:01:21.974161+0000', 'last_undegraded': '2024-09-10T15:59:15.847127+0000', 'last_fullsized': '2024-09-10T15:59:15.774437+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3653", 'last_scrub_stamp': '2024-09-10T15:59:05.298468+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:05.298468+0000', 'objects_scrubbed': 3602, 'log_size': 8359, 'log_dups_size': 0, 'ondisk_log_size': 8359, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1289, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34205696, 'num_objects': 8351, 'num_object_clones': 0, 'num_object_copies': 16702, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4206, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8351, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4749, 'num_write_kb': 18996, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4148}, {'shards': '0,2', 'objects': 4203}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:01:26.649 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-10T16:01:26.649 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "43'8625", 'reported_seq': 8762, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:01:21.980523+0000', 'last_change': '2024-09-10T15:59:16.311291+0000', 'last_active': '2024-09-10T16:01:21.980523+0000', 'last_peered': '2024-09-10T16:01:21.980523+0000', 'last_clean': '2024-09-10T15:59:14.116363+0000', 'last_became_active': '2024-09-10T15:59:16.301244+0000', 'last_became_peered': '2024-09-10T15:59:16.301244+0000', 'last_unstale': '2024-09-10T16:01:21.980523+0000', 'last_undegraded': '2024-09-10T15:59:15.883434+0000', 'last_fullsized': '2024-09-10T15:59:15.772756+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3789", 'last_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'objects_scrubbed': 3741, 'log_size': 8625, 'log_dups_size': 0, 'ondisk_log_size': 8625, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1440, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35311616, 'num_objects': 8621, 'num_object_clones': 0, 'num_object_copies': 17242, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4373, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8621, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4880, 'num_write_kb': 19520, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4250}, {'shards': '0,2', 'objects': 4371}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:01:26.649 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-10T16:01:26.649 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "43'8634", 'reported_seq': 8773, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:01:21.937698+0000', 'last_change': '2024-09-10T15:59:16.313409+0000', 'last_active': '2024-09-10T16:01:21.937698+0000', 'last_peered': '2024-09-10T16:01:21.937698+0000', 'last_clean': '2024-09-10T15:59:14.129780+0000', 'last_became_active': '2024-09-10T15:59:16.300932+0000', 'last_became_peered': '2024-09-10T15:59:16.300932+0000', 'last_unstale': '2024-09-10T16:01:21.937698+0000', 'last_undegraded': '2024-09-10T15:59:15.887823+0000', 'last_fullsized': '2024-09-10T15:59:15.768513+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3880", 'last_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'objects_scrubbed': 3827, 'log_size': 8634, 'log_dups_size': 0, 'ondisk_log_size': 8634, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1397, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35316678, 'num_objects': 8624, 'num_object_clones': 0, 'num_object_copies': 17248, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4325, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8624, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4797, 'num_write_kb': 19188, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4301}, {'shards': '0,2', 'objects': 4323}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:01:26.650 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T16:01:26.986 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 174 20 138234 138214 3.10241 1.66797 0.0303259 0.025172 2024-09-10T16:01:27.179 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:01:27.180 INFO:teuthology.orchestra.run.smithi115.stdout:{"fsid":"7effee29-df05-4351-9741-ee203564ed81","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 47117/272702 objects degraded (17.278%), 11 pgs degraded, 12 pgs undersized","count":23},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":251,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":43,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983954,"num_in_osds":3,"osd_in_since":1725983947,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":136351,"data_bytes":558449393,"bytes_used":1901924352,"bytes_avail":288008368128,"bytes_total":289910292480,"degraded_objects":47117,"degraded_total":272702,"degraded_ratio":0.17277834412655571,"misplaced_objects":1287,"misplaced_total":272702,"misplaced_ratio":0.0047194373345263328,"recovering_objects_per_sec":31,"recovering_bytes_per_sec":130340,"recovering_keys_per_sec":0,"num_objects_recovered":382,"num_bytes_recovered":1564672,"num_keys_recovered":0,"write_bytes_sec":1922353,"read_op_per_sec":0,"write_op_per_sec":469},"fsmap":{"epoch":1,"btime":"2024-09-10T15:57:14:967120+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T16:00:13.560776+0000","services":{}},"progress_events":{}} 2024-09-10T16:01:27.192 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-10T16:01:27.192 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:01:27.735 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:01:27.737 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:01:27.761 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":162,"stamp":"2024-09-10T16:01:27.590210+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":564540145,"num_objects":137838,"num_object_clones":0,"num_object_copies":275676,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":47117,"num_objects_misplaced":1287,"num_objects_unfound":0,"num_objects_dirty":137838,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":95684,"num_write_kb":382730,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2987,"num_bytes_recovered":12222483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137911,"ondisk_log_size":137911,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1869804,"kb_used_data":1112572,"kb_used_omap":12,"kb_used_meta":757171,"kb_avail":281245716,"statfs":{"total":289910292480,"available":287995613184,"internally_reserved":0,"allocated":1139273728,"data_stored":1138388938,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":775343106},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,2,1,3,9,13,8,6],"upper_bound":256},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":25546752,"num_objects":6237,"num_object_clones":0,"num_object_copies":12474,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-252,"num_objects_unfound":0,"num_objects_dirty":6237,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6237,"num_write_kb":24948,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":252,"num_bytes_recovered":1032192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004343"},"pg_stats":[{"pgid":"2.f","version":"43'8563","reported_seq":8632,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:26.924969+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:01:26.924969+0000","last_peered":"2024-09-10T16:01:26.924969+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:01:26.924969+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":8563,"log_dups_size":0,"ondisk_log_size":8563,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35074048,"num_objects":8563,"num_object_clones":0,"num_object_copies":17126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8563,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8563,"num_write_kb":34252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4304},{"shards":"0,2","objects":4258}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'8559","reported_seq":8667,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:01:24.489805+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:01:24.489805+0000","last_peered":"2024-09-10T16:01:24.489805+0000","last_clean":"2024-09-10T16:01:24.489805+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:01:24.489805+0000","last_undegraded":"2024-09-10T16:01:24.489805+0000","last_fullsized":"2024-09-10T16:01:24.489805+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":8559,"log_dups_size":0,"ondisk_log_size":8559,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35034054,"num_objects":8555,"num_object_clones":0,"num_object_copies":17110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8555,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":6664,"num_write_kb":26656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'8532","reported_seq":14622,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T16:01:24.605856+0000","last_change":"2024-09-10T15:59:16.142612+0000","last_active":"2024-09-10T16:01:24.605856+0000","last_peered":"2024-09-10T16:01:24.605856+0000","last_clean":"2024-09-10T15:59:14.092473+0000","last_became_active":"2024-09-10T15:59:16.109611+0000","last_became_peered":"2024-09-10T15:59:16.109611+0000","last_unstale":"2024-09-10T16:01:24.605856+0000","last_undegraded":"2024-09-10T16:01:24.605856+0000","last_fullsized":"2024-09-10T15:59:15.770264+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":8532,"log_dups_size":0,"ondisk_log_size":8532,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34930688,"num_objects":8528,"num_object_clones":0,"num_object_copies":17056,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1287,"num_objects_unfound":0,"num_objects_dirty":8528,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6536,"num_write_kb":26144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2982,"num_bytes_recovered":12210176,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1287},{"shards":"2","objects":7241}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'8514","reported_seq":8590,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:01:24.561991+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:01:24.561991+0000","last_peered":"2024-09-10T16:01:24.561991+0000","last_clean":"2024-09-10T16:01:24.561991+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:01:24.561991+0000","last_undegraded":"2024-09-10T16:01:24.561991+0000","last_fullsized":"2024-09-10T16:01:24.561991+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":8514,"log_dups_size":0,"ondisk_log_size":8514,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34873344,"num_objects":8514,"num_object_clones":0,"num_object_copies":17028,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8514,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8514,"num_write_kb":34056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'8678","reported_seq":8803,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:01:26.931563+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:01:26.931563+0000","last_peered":"2024-09-10T16:01:26.931563+0000","last_clean":"2024-09-10T16:01:26.931563+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:01:26.931563+0000","last_undegraded":"2024-09-10T16:01:26.931563+0000","last_fullsized":"2024-09-10T16:01:26.931563+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":8678,"log_dups_size":0,"ondisk_log_size":8678,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35516899,"num_objects":8672,"num_object_clones":0,"num_object_copies":17344,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8672,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5389,"num_write_kb":21556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'8696","reported_seq":8826,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:26.820222+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:01:26.820222+0000","last_peered":"2024-09-10T16:01:26.820222+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:01:26.820222+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":8696,"log_dups_size":0,"ondisk_log_size":8696,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35598819,"num_objects":8692,"num_object_clones":0,"num_object_copies":17384,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8692,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5308,"num_write_kb":21232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4321},{"shards":"0,2","objects":4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'8707","reported_seq":8839,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:26.943145+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:01:26.943145+0000","last_peered":"2024-09-10T16:01:26.943145+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:01:26.943145+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":8707,"log_dups_size":0,"ondisk_log_size":8707,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35640262,"num_objects":8703,"num_object_clones":0,"num_object_copies":17406,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8703,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5206,"num_write_kb":20824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4466},{"shards":"0,2","objects":4237}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'8834","reported_seq":8959,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:01:26.943383+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:01:26.943383+0000","last_peered":"2024-09-10T16:01:26.943383+0000","last_clean":"2024-09-10T16:01:26.943383+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:01:26.943383+0000","last_undegraded":"2024-09-10T16:01:26.943383+0000","last_fullsized":"2024-09-10T16:01:26.943383+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":8834,"log_dups_size":0,"ondisk_log_size":8834,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36175872,"num_objects":8832,"num_object_clones":0,"num_object_copies":17664,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8832,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5681,"num_write_kb":22724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.777326+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T15:59:15.777326+0000","last_peered":"2024-09-10T15:59:15.777326+0000","last_clean":"2024-09-10T15:59:15.777326+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T15:59:15.777326+0000","last_undegraded":"2024-09-10T15:59:15.777326+0000","last_fullsized":"2024-09-10T15:59:15.777326+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'8266","reported_seq":8401,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:24.595151+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:01:24.595151+0000","last_peered":"2024-09-10T16:01:24.595151+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:01:24.595151+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":8266,"log_dups_size":0,"ondisk_log_size":8266,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33829347,"num_objects":8260,"num_object_clones":0,"num_object_copies":16520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8260,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6085,"num_write_kb":24340,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4123},{"shards":"2","objects":4137}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":89,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.772619+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T15:59:15.772619+0000","last_peered":"2024-09-10T15:59:15.772619+0000","last_clean":"2024-09-10T15:59:15.772619+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T15:59:15.772619+0000","last_undegraded":"2024-09-10T15:59:15.772619+0000","last_fullsized":"2024-09-10T15:59:15.772619+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'8638","reported_seq":8761,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:26.946997+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:01:26.946997+0000","last_peered":"2024-09-10T16:01:26.946997+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:01:26.946997+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":8638,"log_dups_size":0,"ondisk_log_size":8638,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35348480,"num_objects":8630,"num_object_clones":0,"num_object_copies":17260,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8630,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5142,"num_write_kb":20568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4375},{"shards":"0,2","objects":4255}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":99,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.884297+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T15:59:15.884297+0000","last_peered":"2024-09-10T15:59:15.884297+0000","last_clean":"2024-09-10T15:59:15.884297+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T15:59:15.884297+0000","last_undegraded":"2024-09-10T15:59:15.884297+0000","last_fullsized":"2024-09-10T15:59:15.884297+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'8540","reported_seq":8670,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:24.634220+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:01:24.634220+0000","last_peered":"2024-09-10T16:01:24.634220+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:01:24.634220+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":8540,"log_dups_size":0,"ondisk_log_size":8540,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34959843,"num_objects":8536,"num_object_clones":0,"num_object_copies":17072,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8536,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6220,"num_write_kb":24880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4304},{"shards":"2","objects":4232}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.848670+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T15:59:15.848670+0000","last_peered":"2024-09-10T15:59:15.848670+0000","last_clean":"2024-09-10T15:59:15.848670+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T15:59:15.848670+0000","last_undegraded":"2024-09-10T15:59:15.848670+0000","last_fullsized":"2024-09-10T15:59:15.848670+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'8722","reported_seq":8845,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:24.613111+0000","last_change":"2024-09-10T15:59:16.174113+0000","last_active":"2024-09-10T16:01:24.613111+0000","last_peered":"2024-09-10T16:01:24.613111+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:01:24.613111+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":8722,"log_dups_size":0,"ondisk_log_size":8722,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35700736,"num_objects":8716,"num_object_clones":0,"num_object_copies":17432,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4354,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8716,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6276,"num_write_kb":25104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4352},{"shards":"2","objects":4364}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'8619","reported_seq":8749,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:26.943530+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:01:26.943530+0000","last_peered":"2024-09-10T16:01:26.943530+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:01:26.943530+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":8619,"log_dups_size":0,"ondisk_log_size":8619,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35295232,"num_objects":8617,"num_object_clones":0,"num_object_copies":17234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8617,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5250,"num_write_kb":21000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4313},{"shards":"0,2","objects":4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.992406+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T15:59:15.992406+0000","last_peered":"2024-09-10T15:59:15.992406+0000","last_clean":"2024-09-10T15:59:15.992406+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T15:59:15.992406+0000","last_undegraded":"2024-09-10T15:59:15.992406+0000","last_fullsized":"2024-09-10T15:59:15.992406+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'8496","reported_seq":8633,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:26.908524+0000","last_change":"2024-09-10T15:59:16.251610+0000","last_active":"2024-09-10T16:01:26.908524+0000","last_peered":"2024-09-10T16:01:26.908524+0000","last_clean":"2024-09-10T15:59:14.115595+0000","last_became_active":"2024-09-10T15:59:16.244089+0000","last_became_peered":"2024-09-10T15:59:16.244089+0000","last_unstale":"2024-09-10T16:01:26.908524+0000","last_undegraded":"2024-09-10T15:59:15.847127+0000","last_fullsized":"2024-09-10T15:59:15.774437+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":8496,"log_dups_size":0,"ondisk_log_size":8496,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34766848,"num_objects":8488,"num_object_clones":0,"num_object_copies":16976,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8488,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4886,"num_write_kb":19544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4285},{"shards":"0,2","objects":4203}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":89,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.773633+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T15:59:15.773633+0000","last_peered":"2024-09-10T15:59:15.773633+0000","last_clean":"2024-09-10T15:59:15.773633+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T15:59:15.773633+0000","last_undegraded":"2024-09-10T15:59:15.773633+0000","last_fullsized":"2024-09-10T15:59:15.773633+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'8772","reported_seq":8909,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:26.878276+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:01:26.878276+0000","last_peered":"2024-09-10T16:01:26.878276+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:01:26.878276+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":8772,"log_dups_size":0,"ondisk_log_size":8772,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35913728,"num_objects":8768,"num_object_clones":0,"num_object_copies":17536,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8768,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5027,"num_write_kb":20108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4397},{"shards":"0,2","objects":4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.917634+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T15:59:15.917634+0000","last_peered":"2024-09-10T15:59:15.917634+0000","last_clean":"2024-09-10T15:59:15.917634+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T15:59:15.917634+0000","last_undegraded":"2024-09-10T15:59:15.917634+0000","last_fullsized":"2024-09-10T15:59:15.917634+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":70,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.778232+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T15:59:15.778232+0000","last_peered":"2024-09-10T15:59:15.778232+0000","last_clean":"2024-09-10T15:59:15.778232+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T15:59:15.778232+0000","last_undegraded":"2024-09-10T15:59:15.778232+0000","last_fullsized":"2024-09-10T15:59:15.778232+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'8772","reported_seq":8912,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:26.947316+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:01:26.947316+0000","last_peered":"2024-09-10T16:01:26.947316+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:01:26.947316+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":8772,"log_dups_size":0,"ondisk_log_size":8772,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35881926,"num_objects":8762,"num_object_clones":0,"num_object_copies":17524,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8762,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4935,"num_write_kb":19740,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4439},{"shards":"0,2","objects":4323}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":564540126,"num_objects":137836,"num_object_clones":0,"num_object_copies":275672,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":47117,"num_objects_misplaced":1287,"num_objects_unfound":0,"num_objects_dirty":137836,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":95682,"num_write_kb":382728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2985,"num_bytes_recovered":12222464,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1138515968,"data_stored":1138364222,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137908,"ondisk_log_size":137908,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986217,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":685828,"kb_used_data":411188,"kb_used_omap":4,"kb_used_meta":274619,"kb_avail":93686012,"statfs":{"total":96636764160,"available":95934476288,"internally_reserved":0,"allocated":421056512,"data_stored":420778752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":281210538},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,1,0,3,4,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659163,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":389100,"kb_used_data":217180,"kb_used_omap":4,"kb_used_meta":171899,"kb_avail":93982740,"statfs":{"total":96636764160,"available":96238325760,"internally_reserved":0,"allocated":222392320,"data_stored":222067127,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":176025262},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,6,6,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182446,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":794876,"kb_used_data":484204,"kb_used_omap":4,"kb_used_meta":310651,"kb_avail":93576964,"statfs":{"total":96636764160,"available":95822811136,"internally_reserved":0,"allocated":495824896,"data_stored":495543059,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":318107306},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,0,2,0,3,5,6],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":495472640,"data_stored":495436510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":222035968,"data_stored":221956482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":421007360,"data_stored":420971230,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:01:27.986 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 175 20 138584 138564 3.0925 1.36719 0.0194578 0.0252545 2024-09-10T16:01:28.986 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 176 20 139075 139055 3.08582 1.91797 0.0187455 0.0253104 2024-09-10T16:01:29.986 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 177 20 139566 139546 3.07922 1.91797 0.0343455 0.0253639 2024-09-10T16:01:30.762 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:01:30.986 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 178 20 140047 140027 3.07248 1.87891 0.0832786 0.0254198 2024-09-10T16:01:31.317 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:01:31.317 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:01:31.341 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":163,"stamp":"2024-09-10T16:01:29.590987+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":564540145,"num_objects":137838,"num_object_clones":0,"num_object_copies":275676,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":47117,"num_objects_misplaced":1287,"num_objects_unfound":0,"num_objects_dirty":137838,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":95684,"num_write_kb":382730,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2987,"num_bytes_recovered":12222483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137911,"ondisk_log_size":137911,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1881416,"kb_used_data":1120020,"kb_used_omap":12,"kb_used_meta":761331,"kb_avail":281234104,"statfs":{"total":289910292480,"available":287983722496,"internally_reserved":0,"allocated":1146900480,"data_stored":1146015690,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":779602946},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,2,2,2,6,11,12,6],"upper_bound":256},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19402752,"num_objects":4737,"num_object_clones":0,"num_object_copies":9474,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-252,"num_objects_unfound":0,"num_objects_dirty":4737,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4737,"num_write_kb":18948,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":252,"num_bytes_recovered":1032192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004342"},"pg_stats":[{"pgid":"2.f","version":"43'8563","reported_seq":8632,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:26.924969+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:01:26.924969+0000","last_peered":"2024-09-10T16:01:26.924969+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:01:26.924969+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":8563,"log_dups_size":0,"ondisk_log_size":8563,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35074048,"num_objects":8563,"num_object_clones":0,"num_object_copies":17126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8563,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8563,"num_write_kb":34252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4304},{"shards":"0,2","objects":4258}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'8559","reported_seq":8667,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:01:24.489805+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:01:24.489805+0000","last_peered":"2024-09-10T16:01:24.489805+0000","last_clean":"2024-09-10T16:01:24.489805+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:01:24.489805+0000","last_undegraded":"2024-09-10T16:01:24.489805+0000","last_fullsized":"2024-09-10T16:01:24.489805+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":8559,"log_dups_size":0,"ondisk_log_size":8559,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35034054,"num_objects":8555,"num_object_clones":0,"num_object_copies":17110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8555,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":6664,"num_write_kb":26656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'8532","reported_seq":14622,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T16:01:24.605856+0000","last_change":"2024-09-10T15:59:16.142612+0000","last_active":"2024-09-10T16:01:24.605856+0000","last_peered":"2024-09-10T16:01:24.605856+0000","last_clean":"2024-09-10T15:59:14.092473+0000","last_became_active":"2024-09-10T15:59:16.109611+0000","last_became_peered":"2024-09-10T15:59:16.109611+0000","last_unstale":"2024-09-10T16:01:24.605856+0000","last_undegraded":"2024-09-10T16:01:24.605856+0000","last_fullsized":"2024-09-10T15:59:15.770264+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":8532,"log_dups_size":0,"ondisk_log_size":8532,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34930688,"num_objects":8528,"num_object_clones":0,"num_object_copies":17056,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1287,"num_objects_unfound":0,"num_objects_dirty":8528,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6536,"num_write_kb":26144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2982,"num_bytes_recovered":12210176,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1287},{"shards":"2","objects":7241}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'8514","reported_seq":8590,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:01:24.561991+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:01:24.561991+0000","last_peered":"2024-09-10T16:01:24.561991+0000","last_clean":"2024-09-10T16:01:24.561991+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:01:24.561991+0000","last_undegraded":"2024-09-10T16:01:24.561991+0000","last_fullsized":"2024-09-10T16:01:24.561991+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":8514,"log_dups_size":0,"ondisk_log_size":8514,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34873344,"num_objects":8514,"num_object_clones":0,"num_object_copies":17028,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8514,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8514,"num_write_kb":34056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'8678","reported_seq":8803,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:01:26.931563+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:01:26.931563+0000","last_peered":"2024-09-10T16:01:26.931563+0000","last_clean":"2024-09-10T16:01:26.931563+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:01:26.931563+0000","last_undegraded":"2024-09-10T16:01:26.931563+0000","last_fullsized":"2024-09-10T16:01:26.931563+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":8678,"log_dups_size":0,"ondisk_log_size":8678,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35516899,"num_objects":8672,"num_object_clones":0,"num_object_copies":17344,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8672,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5389,"num_write_kb":21556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'8696","reported_seq":8826,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:26.820222+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:01:26.820222+0000","last_peered":"2024-09-10T16:01:26.820222+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:01:26.820222+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":8696,"log_dups_size":0,"ondisk_log_size":8696,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35598819,"num_objects":8692,"num_object_clones":0,"num_object_copies":17384,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8692,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5308,"num_write_kb":21232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4321},{"shards":"0,2","objects":4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'8707","reported_seq":8839,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:26.943145+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:01:26.943145+0000","last_peered":"2024-09-10T16:01:26.943145+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:01:26.943145+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":8707,"log_dups_size":0,"ondisk_log_size":8707,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35640262,"num_objects":8703,"num_object_clones":0,"num_object_copies":17406,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8703,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5206,"num_write_kb":20824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4466},{"shards":"0,2","objects":4237}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'8834","reported_seq":8959,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:01:26.943383+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:01:26.943383+0000","last_peered":"2024-09-10T16:01:26.943383+0000","last_clean":"2024-09-10T16:01:26.943383+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:01:26.943383+0000","last_undegraded":"2024-09-10T16:01:26.943383+0000","last_fullsized":"2024-09-10T16:01:26.943383+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":8834,"log_dups_size":0,"ondisk_log_size":8834,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36175872,"num_objects":8832,"num_object_clones":0,"num_object_copies":17664,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8832,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5681,"num_write_kb":22724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.777326+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T15:59:15.777326+0000","last_peered":"2024-09-10T15:59:15.777326+0000","last_clean":"2024-09-10T15:59:15.777326+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T15:59:15.777326+0000","last_undegraded":"2024-09-10T15:59:15.777326+0000","last_fullsized":"2024-09-10T15:59:15.777326+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'8266","reported_seq":8401,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:24.595151+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:01:24.595151+0000","last_peered":"2024-09-10T16:01:24.595151+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:01:24.595151+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":8266,"log_dups_size":0,"ondisk_log_size":8266,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33829347,"num_objects":8260,"num_object_clones":0,"num_object_copies":16520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8260,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6085,"num_write_kb":24340,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4123},{"shards":"2","objects":4137}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":89,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.772619+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T15:59:15.772619+0000","last_peered":"2024-09-10T15:59:15.772619+0000","last_clean":"2024-09-10T15:59:15.772619+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T15:59:15.772619+0000","last_undegraded":"2024-09-10T15:59:15.772619+0000","last_fullsized":"2024-09-10T15:59:15.772619+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'8638","reported_seq":8761,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:26.946997+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:01:26.946997+0000","last_peered":"2024-09-10T16:01:26.946997+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:01:26.946997+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":8638,"log_dups_size":0,"ondisk_log_size":8638,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35348480,"num_objects":8630,"num_object_clones":0,"num_object_copies":17260,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8630,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5142,"num_write_kb":20568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4375},{"shards":"0,2","objects":4255}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":99,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.884297+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T15:59:15.884297+0000","last_peered":"2024-09-10T15:59:15.884297+0000","last_clean":"2024-09-10T15:59:15.884297+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T15:59:15.884297+0000","last_undegraded":"2024-09-10T15:59:15.884297+0000","last_fullsized":"2024-09-10T15:59:15.884297+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'8540","reported_seq":8670,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:24.634220+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:01:24.634220+0000","last_peered":"2024-09-10T16:01:24.634220+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:01:24.634220+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":8540,"log_dups_size":0,"ondisk_log_size":8540,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34959843,"num_objects":8536,"num_object_clones":0,"num_object_copies":17072,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8536,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6220,"num_write_kb":24880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4304},{"shards":"2","objects":4232}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.848670+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T15:59:15.848670+0000","last_peered":"2024-09-10T15:59:15.848670+0000","last_clean":"2024-09-10T15:59:15.848670+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T15:59:15.848670+0000","last_undegraded":"2024-09-10T15:59:15.848670+0000","last_fullsized":"2024-09-10T15:59:15.848670+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'8722","reported_seq":8845,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:24.613111+0000","last_change":"2024-09-10T15:59:16.174113+0000","last_active":"2024-09-10T16:01:24.613111+0000","last_peered":"2024-09-10T16:01:24.613111+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:01:24.613111+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":8722,"log_dups_size":0,"ondisk_log_size":8722,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35700736,"num_objects":8716,"num_object_clones":0,"num_object_copies":17432,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4354,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8716,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6276,"num_write_kb":25104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4352},{"shards":"2","objects":4364}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'8619","reported_seq":8749,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:26.943530+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:01:26.943530+0000","last_peered":"2024-09-10T16:01:26.943530+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:01:26.943530+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":8619,"log_dups_size":0,"ondisk_log_size":8619,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35295232,"num_objects":8617,"num_object_clones":0,"num_object_copies":17234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8617,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5250,"num_write_kb":21000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4313},{"shards":"0,2","objects":4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.992406+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T15:59:15.992406+0000","last_peered":"2024-09-10T15:59:15.992406+0000","last_clean":"2024-09-10T15:59:15.992406+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T15:59:15.992406+0000","last_undegraded":"2024-09-10T15:59:15.992406+0000","last_fullsized":"2024-09-10T15:59:15.992406+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'8496","reported_seq":8633,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:26.908524+0000","last_change":"2024-09-10T15:59:16.251610+0000","last_active":"2024-09-10T16:01:26.908524+0000","last_peered":"2024-09-10T16:01:26.908524+0000","last_clean":"2024-09-10T15:59:14.115595+0000","last_became_active":"2024-09-10T15:59:16.244089+0000","last_became_peered":"2024-09-10T15:59:16.244089+0000","last_unstale":"2024-09-10T16:01:26.908524+0000","last_undegraded":"2024-09-10T15:59:15.847127+0000","last_fullsized":"2024-09-10T15:59:15.774437+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":8496,"log_dups_size":0,"ondisk_log_size":8496,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34766848,"num_objects":8488,"num_object_clones":0,"num_object_copies":16976,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8488,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4886,"num_write_kb":19544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4285},{"shards":"0,2","objects":4203}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":89,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.773633+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T15:59:15.773633+0000","last_peered":"2024-09-10T15:59:15.773633+0000","last_clean":"2024-09-10T15:59:15.773633+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T15:59:15.773633+0000","last_undegraded":"2024-09-10T15:59:15.773633+0000","last_fullsized":"2024-09-10T15:59:15.773633+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'8772","reported_seq":8909,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:26.878276+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:01:26.878276+0000","last_peered":"2024-09-10T16:01:26.878276+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:01:26.878276+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":8772,"log_dups_size":0,"ondisk_log_size":8772,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35913728,"num_objects":8768,"num_object_clones":0,"num_object_copies":17536,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8768,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5027,"num_write_kb":20108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4397},{"shards":"0,2","objects":4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.917634+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T15:59:15.917634+0000","last_peered":"2024-09-10T15:59:15.917634+0000","last_clean":"2024-09-10T15:59:15.917634+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T15:59:15.917634+0000","last_undegraded":"2024-09-10T15:59:15.917634+0000","last_fullsized":"2024-09-10T15:59:15.917634+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":70,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.778232+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T15:59:15.778232+0000","last_peered":"2024-09-10T15:59:15.778232+0000","last_clean":"2024-09-10T15:59:15.778232+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T15:59:15.778232+0000","last_undegraded":"2024-09-10T15:59:15.778232+0000","last_fullsized":"2024-09-10T15:59:15.778232+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'8772","reported_seq":8912,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:26.947316+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:01:26.947316+0000","last_peered":"2024-09-10T16:01:26.947316+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:01:26.947316+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":8772,"log_dups_size":0,"ondisk_log_size":8772,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35881926,"num_objects":8762,"num_object_clones":0,"num_object_copies":17524,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8762,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4935,"num_write_kb":19740,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4439},{"shards":"0,2","objects":4323}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":564540126,"num_objects":137836,"num_object_clones":0,"num_object_copies":275672,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":47117,"num_objects_misplaced":1287,"num_objects_unfound":0,"num_objects_dirty":137836,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":95682,"num_write_kb":382728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2985,"num_bytes_recovered":12222464,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1146146816,"data_stored":1145995070,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137908,"ondisk_log_size":137908,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986217,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":685828,"kb_used_data":411188,"kb_used_omap":4,"kb_used_meta":274619,"kb_avail":93686012,"statfs":{"total":96636764160,"available":95934476288,"internally_reserved":0,"allocated":421056512,"data_stored":420778752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":281210538},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,1,0,3,4,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659164,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":400712,"kb_used_data":224628,"kb_used_omap":4,"kb_used_meta":176059,"kb_avail":93971128,"statfs":{"total":96636764160,"available":96226435072,"internally_reserved":0,"allocated":230019072,"data_stored":229693879,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":180285102},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,3,4,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182446,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":794876,"kb_used_data":484204,"kb_used_omap":4,"kb_used_meta":310651,"kb_avail":93576964,"statfs":{"total":96636764160,"available":95822811136,"internally_reserved":0,"allocated":495824896,"data_stored":495543059,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":318107306},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,0,2,0,3,5,6],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":495472640,"data_stored":495436510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":229666816,"data_stored":229587330,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":421007360,"data_stored":420971230,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:01:31.342 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:01:31.909 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:01:31.910 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:01:31.938 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":164,"stamp":"2024-09-10T16:01:31.591641+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":567997169,"num_objects":138682,"num_object_clones":0,"num_object_copies":277364,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":47117,"num_objects_misplaced":1157,"num_objects_unfound":0,"num_objects_dirty":138682,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":96528,"num_write_kb":386106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3117,"num_bytes_recovered":12754963,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":138755,"ondisk_log_size":138755,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1873436,"kb_used_data":1124520,"kb_used_omap":12,"kb_used_meta":748851,"kb_avail":281242084,"statfs":{"total":289910292480,"available":287991894016,"internally_reserved":0,"allocated":1151508480,"data_stored":1150623690,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":766823426},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,1,1,2,4,10,11,6],"upper_bound":256},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":22859776,"num_objects":5581,"num_object_clones":0,"num_object_copies":11162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-382,"num_objects_unfound":0,"num_objects_dirty":5581,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5581,"num_write_kb":22324,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":382,"num_bytes_recovered":1564672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004235"},"pg_stats":[{"pgid":"2.f","version":"43'8563","reported_seq":8632,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:26.924969+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:01:26.924969+0000","last_peered":"2024-09-10T16:01:26.924969+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:01:26.924969+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":8563,"log_dups_size":0,"ondisk_log_size":8563,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35074048,"num_objects":8563,"num_object_clones":0,"num_object_copies":17126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8563,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8563,"num_write_kb":34252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4304},{"shards":"0,2","objects":4258}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'8695","reported_seq":8803,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:01:29.532722+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:01:29.532722+0000","last_peered":"2024-09-10T16:01:29.532722+0000","last_clean":"2024-09-10T16:01:29.532722+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:01:29.532722+0000","last_undegraded":"2024-09-10T16:01:29.532722+0000","last_fullsized":"2024-09-10T16:01:29.532722+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":8695,"log_dups_size":0,"ondisk_log_size":8695,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35591110,"num_objects":8691,"num_object_clones":0,"num_object_copies":17382,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8691,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":6800,"num_write_kb":27200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'8687","reported_seq":15037,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T16:01:29.653747+0000","last_change":"2024-09-10T15:59:16.142612+0000","last_active":"2024-09-10T16:01:29.653747+0000","last_peered":"2024-09-10T16:01:29.653747+0000","last_clean":"2024-09-10T15:59:14.092473+0000","last_became_active":"2024-09-10T15:59:16.109611+0000","last_became_peered":"2024-09-10T15:59:16.109611+0000","last_unstale":"2024-09-10T16:01:29.653747+0000","last_undegraded":"2024-09-10T16:01:29.653747+0000","last_fullsized":"2024-09-10T15:59:15.770264+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":8687,"log_dups_size":0,"ondisk_log_size":8687,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35565568,"num_objects":8683,"num_object_clones":0,"num_object_copies":17366,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1157,"num_objects_unfound":0,"num_objects_dirty":8683,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6691,"num_write_kb":26764,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3112,"num_bytes_recovered":12742656,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1157},{"shards":"2","objects":7526}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'8628","reported_seq":8701,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:01:29.604505+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:01:29.604505+0000","last_peered":"2024-09-10T16:01:29.604505+0000","last_clean":"2024-09-10T16:01:29.604505+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:01:29.604505+0000","last_undegraded":"2024-09-10T16:01:29.604505+0000","last_fullsized":"2024-09-10T16:01:29.604505+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":8628,"log_dups_size":0,"ondisk_log_size":8628,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35340288,"num_objects":8628,"num_object_clones":0,"num_object_copies":17256,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8628,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8628,"num_write_kb":34512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'8678","reported_seq":8803,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:01:26.931563+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:01:26.931563+0000","last_peered":"2024-09-10T16:01:26.931563+0000","last_clean":"2024-09-10T16:01:26.931563+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:01:26.931563+0000","last_undegraded":"2024-09-10T16:01:26.931563+0000","last_fullsized":"2024-09-10T16:01:26.931563+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":8678,"log_dups_size":0,"ondisk_log_size":8678,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35516899,"num_objects":8672,"num_object_clones":0,"num_object_copies":17344,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8672,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5389,"num_write_kb":21556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'8696","reported_seq":8826,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:26.820222+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:01:26.820222+0000","last_peered":"2024-09-10T16:01:26.820222+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:01:26.820222+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":8696,"log_dups_size":0,"ondisk_log_size":8696,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35598819,"num_objects":8692,"num_object_clones":0,"num_object_copies":17384,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8692,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5308,"num_write_kb":21232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4321},{"shards":"0,2","objects":4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'8707","reported_seq":8839,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:26.943145+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:01:26.943145+0000","last_peered":"2024-09-10T16:01:26.943145+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:01:26.943145+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":8707,"log_dups_size":0,"ondisk_log_size":8707,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35640262,"num_objects":8703,"num_object_clones":0,"num_object_copies":17406,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8703,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5206,"num_write_kb":20824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4466},{"shards":"0,2","objects":4237}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'8834","reported_seq":8959,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:01:26.943383+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:01:26.943383+0000","last_peered":"2024-09-10T16:01:26.943383+0000","last_clean":"2024-09-10T16:01:26.943383+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:01:26.943383+0000","last_undegraded":"2024-09-10T16:01:26.943383+0000","last_fullsized":"2024-09-10T16:01:26.943383+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":8834,"log_dups_size":0,"ondisk_log_size":8834,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36175872,"num_objects":8832,"num_object_clones":0,"num_object_copies":17664,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8832,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5681,"num_write_kb":22724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.777326+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T15:59:15.777326+0000","last_peered":"2024-09-10T15:59:15.777326+0000","last_clean":"2024-09-10T15:59:15.777326+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T15:59:15.777326+0000","last_undegraded":"2024-09-10T15:59:15.777326+0000","last_fullsized":"2024-09-10T15:59:15.777326+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'8419","reported_seq":8554,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:29.652409+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:01:29.652409+0000","last_peered":"2024-09-10T16:01:29.652409+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:01:29.652409+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":8419,"log_dups_size":0,"ondisk_log_size":8419,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34456035,"num_objects":8413,"num_object_clones":0,"num_object_copies":16826,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8413,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6238,"num_write_kb":24952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4123},{"shards":"2","objects":4290}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":89,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.772619+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T15:59:15.772619+0000","last_peered":"2024-09-10T15:59:15.772619+0000","last_clean":"2024-09-10T15:59:15.772619+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T15:59:15.772619+0000","last_undegraded":"2024-09-10T15:59:15.772619+0000","last_fullsized":"2024-09-10T15:59:15.772619+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'8638","reported_seq":8761,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:26.946997+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:01:26.946997+0000","last_peered":"2024-09-10T16:01:26.946997+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:01:26.946997+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":8638,"log_dups_size":0,"ondisk_log_size":8638,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35348480,"num_objects":8630,"num_object_clones":0,"num_object_copies":17260,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8630,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5142,"num_write_kb":20568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4375},{"shards":"0,2","objects":4255}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":99,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.884297+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T15:59:15.884297+0000","last_peered":"2024-09-10T15:59:15.884297+0000","last_clean":"2024-09-10T15:59:15.884297+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T15:59:15.884297+0000","last_undegraded":"2024-09-10T15:59:15.884297+0000","last_fullsized":"2024-09-10T15:59:15.884297+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'8690","reported_seq":8820,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:29.614446+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:01:29.614446+0000","last_peered":"2024-09-10T16:01:29.614446+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:01:29.614446+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":8690,"log_dups_size":0,"ondisk_log_size":8690,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35574243,"num_objects":8686,"num_object_clones":0,"num_object_copies":17372,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8686,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6370,"num_write_kb":25480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4304},{"shards":"2","objects":4382}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.848670+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T15:59:15.848670+0000","last_peered":"2024-09-10T15:59:15.848670+0000","last_clean":"2024-09-10T15:59:15.848670+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T15:59:15.848670+0000","last_undegraded":"2024-09-10T15:59:15.848670+0000","last_fullsized":"2024-09-10T15:59:15.848670+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'8858","reported_seq":8982,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:29.559446+0000","last_change":"2024-09-10T15:59:16.174113+0000","last_active":"2024-09-10T16:01:29.559446+0000","last_peered":"2024-09-10T16:01:29.559446+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:01:29.559446+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":8858,"log_dups_size":0,"ondisk_log_size":8858,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36257792,"num_objects":8852,"num_object_clones":0,"num_object_copies":17704,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4354,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8852,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6412,"num_write_kb":25648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4352},{"shards":"2","objects":4500}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'8619","reported_seq":8749,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:26.943530+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:01:26.943530+0000","last_peered":"2024-09-10T16:01:26.943530+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:01:26.943530+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":8619,"log_dups_size":0,"ondisk_log_size":8619,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35295232,"num_objects":8617,"num_object_clones":0,"num_object_copies":17234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8617,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5250,"num_write_kb":21000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4313},{"shards":"0,2","objects":4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.992406+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T15:59:15.992406+0000","last_peered":"2024-09-10T15:59:15.992406+0000","last_clean":"2024-09-10T15:59:15.992406+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T15:59:15.992406+0000","last_undegraded":"2024-09-10T15:59:15.992406+0000","last_fullsized":"2024-09-10T15:59:15.992406+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'8496","reported_seq":8633,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:26.908524+0000","last_change":"2024-09-10T15:59:16.251610+0000","last_active":"2024-09-10T16:01:26.908524+0000","last_peered":"2024-09-10T16:01:26.908524+0000","last_clean":"2024-09-10T15:59:14.115595+0000","last_became_active":"2024-09-10T15:59:16.244089+0000","last_became_peered":"2024-09-10T15:59:16.244089+0000","last_unstale":"2024-09-10T16:01:26.908524+0000","last_undegraded":"2024-09-10T15:59:15.847127+0000","last_fullsized":"2024-09-10T15:59:15.774437+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":8496,"log_dups_size":0,"ondisk_log_size":8496,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34766848,"num_objects":8488,"num_object_clones":0,"num_object_copies":16976,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8488,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4886,"num_write_kb":19544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4285},{"shards":"0,2","objects":4203}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":89,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.773633+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T15:59:15.773633+0000","last_peered":"2024-09-10T15:59:15.773633+0000","last_clean":"2024-09-10T15:59:15.773633+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T15:59:15.773633+0000","last_undegraded":"2024-09-10T15:59:15.773633+0000","last_fullsized":"2024-09-10T15:59:15.773633+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'8772","reported_seq":8909,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:26.878276+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:01:26.878276+0000","last_peered":"2024-09-10T16:01:26.878276+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:01:26.878276+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":8772,"log_dups_size":0,"ondisk_log_size":8772,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35913728,"num_objects":8768,"num_object_clones":0,"num_object_copies":17536,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8768,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5027,"num_write_kb":20108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4397},{"shards":"0,2","objects":4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.917634+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T15:59:15.917634+0000","last_peered":"2024-09-10T15:59:15.917634+0000","last_clean":"2024-09-10T15:59:15.917634+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T15:59:15.917634+0000","last_undegraded":"2024-09-10T15:59:15.917634+0000","last_fullsized":"2024-09-10T15:59:15.917634+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":70,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.778232+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T15:59:15.778232+0000","last_peered":"2024-09-10T15:59:15.778232+0000","last_clean":"2024-09-10T15:59:15.778232+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T15:59:15.778232+0000","last_undegraded":"2024-09-10T15:59:15.778232+0000","last_fullsized":"2024-09-10T15:59:15.778232+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'8772","reported_seq":8912,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:26.947316+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:01:26.947316+0000","last_peered":"2024-09-10T16:01:26.947316+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:01:26.947316+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":8772,"log_dups_size":0,"ondisk_log_size":8772,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35881926,"num_objects":8762,"num_object_clones":0,"num_object_copies":17524,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8762,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4935,"num_write_kb":19740,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4439},{"shards":"0,2","objects":4323}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":567997150,"num_objects":138680,"num_object_clones":0,"num_object_copies":277360,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":47117,"num_objects_misplaced":1157,"num_objects_unfound":0,"num_objects_dirty":138680,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":96526,"num_write_kb":386104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3115,"num_bytes_recovered":12754944,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1150787584,"data_stored":1150635838,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":138752,"ondisk_log_size":138752,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986218,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":677848,"kb_used_data":415688,"kb_used_omap":4,"kb_used_meta":262139,"kb_avail":93693992,"statfs":{"total":96636764160,"available":95942647808,"internally_reserved":0,"allocated":425664512,"data_stored":425386752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":268431018},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,0,1,3,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659164,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":400712,"kb_used_data":224628,"kb_used_omap":4,"kb_used_meta":176059,"kb_avail":93971128,"statfs":{"total":96636764160,"available":96226435072,"internally_reserved":0,"allocated":230019072,"data_stored":229693879,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":180285102},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,3,4,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182446,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":794876,"kb_used_data":484204,"kb_used_omap":4,"kb_used_meta":310651,"kb_avail":93576964,"statfs":{"total":96636764160,"available":95822811136,"internally_reserved":0,"allocated":495824896,"data_stored":495543059,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":318107306},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,0,2,0,3,5,6],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":495472640,"data_stored":495436510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":229666816,"data_stored":229587330,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":425648128,"data_stored":425611998,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:01:31.940 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-10T16:01:31.940 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "43'8563", 'reported_seq': 8632, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:01:26.924969+0000', 'last_change': '2024-09-10T15:59:16.281558+0000', 'last_active': '2024-09-10T16:01:26.924969+0000', 'last_peered': '2024-09-10T16:01:26.924969+0000', 'last_clean': '2024-09-10T15:59:14.132456+0000', 'last_became_active': '2024-09-10T15:59:16.274407+0000', 'last_became_peered': '2024-09-10T15:59:16.274407+0000', 'last_unstale': '2024-09-10T16:01:26.924969+0000', 'last_undegraded': '2024-09-10T15:59:15.851279+0000', 'last_fullsized': '2024-09-10T15:59:15.774356+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'objects_scrubbed': 0, 'log_size': 8563, 'log_dups_size': 0, 'ondisk_log_size': 8563, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35074048, 'num_objects': 8563, 'num_object_clones': 0, 'num_object_copies': 17126, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4259, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8563, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8563, 'num_write_kb': 34252, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4304}, {'shards': '0,2', 'objects': 4258}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:01:31.940 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-10T16:01:31.940 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "43'8687", 'reported_seq': 15037, 'reported_epoch': 43, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-10T16:01:29.653747+0000', 'last_change': '2024-09-10T15:59:16.142612+0000', 'last_active': '2024-09-10T16:01:29.653747+0000', 'last_peered': '2024-09-10T16:01:29.653747+0000', 'last_clean': '2024-09-10T15:59:14.092473+0000', 'last_became_active': '2024-09-10T15:59:16.109611+0000', 'last_became_peered': '2024-09-10T15:59:16.109611+0000', 'last_unstale': '2024-09-10T16:01:29.653747+0000', 'last_undegraded': '2024-09-10T16:01:29.653747+0000', 'last_fullsized': '2024-09-10T15:59:15.770264+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2029", 'last_scrub_stamp': '2024-09-10T15:58:45.933789+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:45.933789+0000', 'objects_scrubbed': 1992, 'log_size': 8687, 'log_dups_size': 0, 'ondisk_log_size': 8687, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 668, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35565568, 'num_objects': 8683, 'num_object_clones': 0, 'num_object_copies': 17366, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1157, 'num_objects_unfound': 0, 'num_objects_dirty': 8683, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6691, 'num_write_kb': 26764, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3112, 'num_bytes_recovered': 12742656, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1157}, {'shards': '2', 'objects': 7526}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:01:31.940 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-10T16:01:31.940 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "43'8696", 'reported_seq': 8826, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:01:26.820222+0000', 'last_change': '2024-09-10T15:59:16.308264+0000', 'last_active': '2024-09-10T16:01:26.820222+0000', 'last_peered': '2024-09-10T16:01:26.820222+0000', 'last_clean': '2024-09-10T15:59:14.111551+0000', 'last_became_active': '2024-09-10T15:59:16.300040+0000', 'last_became_peered': '2024-09-10T15:59:16.300040+0000', 'last_unstale': '2024-09-10T16:01:26.820222+0000', 'last_undegraded': '2024-09-10T15:59:15.878234+0000', 'last_fullsized': '2024-09-10T15:59:15.769328+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3442", 'last_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'objects_scrubbed': 3384, 'log_size': 8696, 'log_dups_size': 0, 'ondisk_log_size': 8696, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1292, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35598819, 'num_objects': 8692, 'num_object_clones': 0, 'num_object_copies': 17384, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4372, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8692, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 5308, 'num_write_kb': 21232, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4321}, {'shards': '0,2', 'objects': 4371}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:01:31.940 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-10T16:01:31.940 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "43'8707", 'reported_seq': 8839, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:01:26.943145+0000', 'last_change': '2024-09-10T15:59:16.331899+0000', 'last_active': '2024-09-10T16:01:26.943145+0000', 'last_peered': '2024-09-10T16:01:26.943145+0000', 'last_clean': '2024-09-10T15:59:14.139436+0000', 'last_became_active': '2024-09-10T15:59:16.318715+0000', 'last_became_peered': '2024-09-10T15:59:16.318715+0000', 'last_unstale': '2024-09-10T16:01:26.943145+0000', 'last_undegraded': '2024-09-10T15:59:15.877524+0000', 'last_fullsized': '2024-09-10T15:59:15.769042+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3537", 'last_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'objects_scrubbed': 3497, 'log_size': 8707, 'log_dups_size': 0, 'ondisk_log_size': 8707, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1224, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35640262, 'num_objects': 8703, 'num_object_clones': 0, 'num_object_copies': 17406, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4237, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8703, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 5206, 'num_write_kb': 20824, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4466}, {'shards': '0,2', 'objects': 4237}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:01:31.940 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-10T16:01:31.940 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "43'8419", 'reported_seq': 8554, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:01:29.652409+0000', 'last_change': '2024-09-10T15:59:16.175680+0000', 'last_active': '2024-09-10T16:01:29.652409+0000', 'last_peered': '2024-09-10T16:01:29.652409+0000', 'last_clean': '2024-09-10T15:59:14.088187+0000', 'last_became_active': '2024-09-10T15:59:16.165810+0000', 'last_became_peered': '2024-09-10T15:59:16.165810+0000', 'last_unstale': '2024-09-10T16:01:29.652409+0000', 'last_undegraded': '2024-09-10T15:59:15.822098+0000', 'last_fullsized': '2024-09-10T15:59:15.769123+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2218", 'last_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'objects_scrubbed': 2175, 'log_size': 8419, 'log_dups_size': 0, 'ondisk_log_size': 8419, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 825, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34456035, 'num_objects': 8413, 'num_object_clones': 0, 'num_object_copies': 16826, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4124, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8413, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 6238, 'num_write_kb': 24952, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4123}, {'shards': '2', 'objects': 4290}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:01:31.941 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-10T16:01:31.941 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "43'8638", 'reported_seq': 8761, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:01:26.946997+0000', 'last_change': '2024-09-10T15:59:16.277742+0000', 'last_active': '2024-09-10T16:01:26.946997+0000', 'last_peered': '2024-09-10T16:01:26.946997+0000', 'last_clean': '2024-09-10T15:59:14.129081+0000', 'last_became_active': '2024-09-10T15:59:16.269835+0000', 'last_became_peered': '2024-09-10T15:59:16.269835+0000', 'last_unstale': '2024-09-10T16:01:26.946997+0000', 'last_undegraded': '2024-09-10T15:59:15.860857+0000', 'last_fullsized': '2024-09-10T15:59:15.771410+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3557", 'last_scrub_stamp': '2024-09-10T15:59:02.493678+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:02.493678+0000', 'objects_scrubbed': 3488, 'log_size': 8638, 'log_dups_size': 0, 'ondisk_log_size': 8638, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1472, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35348480, 'num_objects': 8630, 'num_object_clones': 0, 'num_object_copies': 17260, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4257, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8630, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5142, 'num_write_kb': 20568, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4375}, {'shards': '0,2', 'objects': 4255}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:01:31.941 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-10T16:01:31.941 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "43'8690", 'reported_seq': 8820, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:01:29.614446+0000', 'last_change': '2024-09-10T15:59:16.284311+0000', 'last_active': '2024-09-10T16:01:29.614446+0000', 'last_peered': '2024-09-10T16:01:29.614446+0000', 'last_clean': '2024-09-10T15:59:14.116041+0000', 'last_became_active': '2024-09-10T15:59:16.274543+0000', 'last_became_peered': '2024-09-10T15:59:16.274543+0000', 'last_unstale': '2024-09-10T16:01:29.614446+0000', 'last_undegraded': '2024-09-10T15:59:15.812484+0000', 'last_fullsized': '2024-09-10T15:59:15.770885+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2356", 'last_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'objects_scrubbed': 2316, 'log_size': 8690, 'log_dups_size': 0, 'ondisk_log_size': 8690, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 783, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35574243, 'num_objects': 8686, 'num_object_clones': 0, 'num_object_copies': 17372, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8686, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 6370, 'num_write_kb': 25480, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4304}, {'shards': '2', 'objects': 4382}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:01:31.941 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-10T16:01:31.941 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "43'8858", 'reported_seq': 8982, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:01:29.559446+0000', 'last_change': '2024-09-10T15:59:16.174113+0000', 'last_active': '2024-09-10T16:01:29.559446+0000', 'last_peered': '2024-09-10T16:01:29.559446+0000', 'last_clean': '2024-09-10T15:59:14.110654+0000', 'last_became_active': '2024-09-10T15:59:16.164645+0000', 'last_became_peered': '2024-09-10T15:59:16.164645+0000', 'last_unstale': '2024-09-10T16:01:29.559446+0000', 'last_undegraded': '2024-09-10T15:59:15.803593+0000', 'last_fullsized': '2024-09-10T15:59:15.769083+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2486", 'last_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'objects_scrubbed': 2440, 'log_size': 8858, 'log_dups_size': 0, 'ondisk_log_size': 8858, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 792, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36257792, 'num_objects': 8852, 'num_object_clones': 0, 'num_object_copies': 17704, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4354, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8852, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6412, 'num_write_kb': 25648, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4352}, {'shards': '2', 'objects': 4500}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:01:31.941 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-10T16:01:31.941 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "43'8619", 'reported_seq': 8749, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:01:26.943530+0000', 'last_change': '2024-09-10T15:59:16.315641+0000', 'last_active': '2024-09-10T16:01:26.943530+0000', 'last_peered': '2024-09-10T16:01:26.943530+0000', 'last_clean': '2024-09-10T15:59:14.117001+0000', 'last_became_active': '2024-09-10T15:59:16.302245+0000', 'last_became_peered': '2024-09-10T15:59:16.302245+0000', 'last_unstale': '2024-09-10T16:01:26.943530+0000', 'last_undegraded': '2024-09-10T15:59:15.845012+0000', 'last_fullsized': '2024-09-10T15:59:15.770337+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3404", 'last_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'objects_scrubbed': 3367, 'log_size': 8619, 'log_dups_size': 0, 'ondisk_log_size': 8619, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 983, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35295232, 'num_objects': 8617, 'num_object_clones': 0, 'num_object_copies': 17234, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8617, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5250, 'num_write_kb': 21000, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4313}, {'shards': '0,2', 'objects': 4304}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:01:31.941 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-10T16:01:31.941 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "43'8496", 'reported_seq': 8633, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:01:26.908524+0000', 'last_change': '2024-09-10T15:59:16.251610+0000', 'last_active': '2024-09-10T16:01:26.908524+0000', 'last_peered': '2024-09-10T16:01:26.908524+0000', 'last_clean': '2024-09-10T15:59:14.115595+0000', 'last_became_active': '2024-09-10T15:59:16.244089+0000', 'last_became_peered': '2024-09-10T15:59:16.244089+0000', 'last_unstale': '2024-09-10T16:01:26.908524+0000', 'last_undegraded': '2024-09-10T15:59:15.847127+0000', 'last_fullsized': '2024-09-10T15:59:15.774437+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3653", 'last_scrub_stamp': '2024-09-10T15:59:05.298468+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:05.298468+0000', 'objects_scrubbed': 3602, 'log_size': 8496, 'log_dups_size': 0, 'ondisk_log_size': 8496, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1289, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34766848, 'num_objects': 8488, 'num_object_clones': 0, 'num_object_copies': 16976, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4206, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8488, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4886, 'num_write_kb': 19544, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4285}, {'shards': '0,2', 'objects': 4203}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:01:31.942 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-10T16:01:31.942 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "43'8772", 'reported_seq': 8909, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:01:26.878276+0000', 'last_change': '2024-09-10T15:59:16.311291+0000', 'last_active': '2024-09-10T16:01:26.878276+0000', 'last_peered': '2024-09-10T16:01:26.878276+0000', 'last_clean': '2024-09-10T15:59:14.116363+0000', 'last_became_active': '2024-09-10T15:59:16.301244+0000', 'last_became_peered': '2024-09-10T15:59:16.301244+0000', 'last_unstale': '2024-09-10T16:01:26.878276+0000', 'last_undegraded': '2024-09-10T15:59:15.883434+0000', 'last_fullsized': '2024-09-10T15:59:15.772756+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3789", 'last_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'objects_scrubbed': 3741, 'log_size': 8772, 'log_dups_size': 0, 'ondisk_log_size': 8772, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1440, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35913728, 'num_objects': 8768, 'num_object_clones': 0, 'num_object_copies': 17536, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4373, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8768, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5027, 'num_write_kb': 20108, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4397}, {'shards': '0,2', 'objects': 4371}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:01:31.942 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-10T16:01:31.942 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "43'8772", 'reported_seq': 8912, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:01:26.947316+0000', 'last_change': '2024-09-10T15:59:16.313409+0000', 'last_active': '2024-09-10T16:01:26.947316+0000', 'last_peered': '2024-09-10T16:01:26.947316+0000', 'last_clean': '2024-09-10T15:59:14.129780+0000', 'last_became_active': '2024-09-10T15:59:16.300932+0000', 'last_became_peered': '2024-09-10T15:59:16.300932+0000', 'last_unstale': '2024-09-10T16:01:26.947316+0000', 'last_undegraded': '2024-09-10T15:59:15.887823+0000', 'last_fullsized': '2024-09-10T15:59:15.768513+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3880", 'last_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'objects_scrubbed': 3827, 'log_size': 8772, 'log_dups_size': 0, 'ondisk_log_size': 8772, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1397, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35881926, 'num_objects': 8762, 'num_object_clones': 0, 'num_object_copies': 17524, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4325, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8762, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4935, 'num_write_kb': 19740, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4439}, {'shards': '0,2', 'objects': 4323}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:01:31.942 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T16:01:31.986 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 179 20 140473 140453 3.06461 1.66406 0.0193293 0.025482 2024-09-10T16:01:32.612 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:01:32.613 INFO:teuthology.orchestra.run.smithi115.stdout:{"fsid":"7effee29-df05-4351-9741-ee203564ed81","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 47117/277364 objects degraded (16.987%), 11 pgs degraded, 12 pgs undersized","count":23},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":256,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":43,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983954,"num_in_osds":3,"osd_in_since":1725983947,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":138682,"data_bytes":567997169,"bytes_used":1918398464,"bytes_avail":287991894016,"bytes_total":289910292480,"degraded_objects":47117,"degraded_total":277364,"degraded_ratio":0.16987424467486767,"misplaced_objects":1157,"misplaced_total":277364,"misplaced_ratio":0.0041714137379039815,"recovering_objects_per_sec":31,"recovering_bytes_per_sec":130343,"recovering_keys_per_sec":0,"num_objects_recovered":382,"num_bytes_recovered":1564672,"num_keys_recovered":0,"write_bytes_sec":1904309,"read_op_per_sec":0,"write_op_per_sec":464},"fsmap":{"epoch":1,"btime":"2024-09-10T15:57:14:967120+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T16:00:13.560776+0000","services":{}},"progress_events":{}} 2024-09-10T16:01:32.642 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-10T16:01:32.642 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:01:32.987 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout:2024-09-10T16:01:32.986134+0000 min lat: 0.00225581 max lat: 2.21736 avg lat: 0.0255483 2024-09-10T16:01:32.987 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-10T16:01:32.987 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 180 20 140911 140891 3.05709 1.71094 0.0461151 0.0255483 2024-09-10T16:01:33.232 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:01:33.232 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:01:33.253 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":164,"stamp":"2024-09-10T16:01:31.591641+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":567997169,"num_objects":138682,"num_object_clones":0,"num_object_copies":277364,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":47117,"num_objects_misplaced":1157,"num_objects_unfound":0,"num_objects_dirty":138682,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":96528,"num_write_kb":386106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3117,"num_bytes_recovered":12754963,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":138755,"ondisk_log_size":138755,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1873436,"kb_used_data":1124520,"kb_used_omap":12,"kb_used_meta":748851,"kb_avail":281242084,"statfs":{"total":289910292480,"available":287991894016,"internally_reserved":0,"allocated":1151508480,"data_stored":1150623690,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":766823426},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,1,1,2,4,10,11,6],"upper_bound":256},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":22859776,"num_objects":5581,"num_object_clones":0,"num_object_copies":11162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-382,"num_objects_unfound":0,"num_objects_dirty":5581,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5581,"num_write_kb":22324,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":382,"num_bytes_recovered":1564672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004235"},"pg_stats":[{"pgid":"2.f","version":"43'8563","reported_seq":8632,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:26.924969+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:01:26.924969+0000","last_peered":"2024-09-10T16:01:26.924969+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:01:26.924969+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":8563,"log_dups_size":0,"ondisk_log_size":8563,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35074048,"num_objects":8563,"num_object_clones":0,"num_object_copies":17126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8563,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8563,"num_write_kb":34252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4304},{"shards":"0,2","objects":4258}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'8695","reported_seq":8803,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:01:29.532722+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:01:29.532722+0000","last_peered":"2024-09-10T16:01:29.532722+0000","last_clean":"2024-09-10T16:01:29.532722+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:01:29.532722+0000","last_undegraded":"2024-09-10T16:01:29.532722+0000","last_fullsized":"2024-09-10T16:01:29.532722+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":8695,"log_dups_size":0,"ondisk_log_size":8695,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35591110,"num_objects":8691,"num_object_clones":0,"num_object_copies":17382,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8691,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":6800,"num_write_kb":27200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'8687","reported_seq":15037,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T16:01:29.653747+0000","last_change":"2024-09-10T15:59:16.142612+0000","last_active":"2024-09-10T16:01:29.653747+0000","last_peered":"2024-09-10T16:01:29.653747+0000","last_clean":"2024-09-10T15:59:14.092473+0000","last_became_active":"2024-09-10T15:59:16.109611+0000","last_became_peered":"2024-09-10T15:59:16.109611+0000","last_unstale":"2024-09-10T16:01:29.653747+0000","last_undegraded":"2024-09-10T16:01:29.653747+0000","last_fullsized":"2024-09-10T15:59:15.770264+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":8687,"log_dups_size":0,"ondisk_log_size":8687,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35565568,"num_objects":8683,"num_object_clones":0,"num_object_copies":17366,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1157,"num_objects_unfound":0,"num_objects_dirty":8683,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6691,"num_write_kb":26764,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3112,"num_bytes_recovered":12742656,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1157},{"shards":"2","objects":7526}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'8628","reported_seq":8701,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:01:29.604505+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:01:29.604505+0000","last_peered":"2024-09-10T16:01:29.604505+0000","last_clean":"2024-09-10T16:01:29.604505+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:01:29.604505+0000","last_undegraded":"2024-09-10T16:01:29.604505+0000","last_fullsized":"2024-09-10T16:01:29.604505+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":8628,"log_dups_size":0,"ondisk_log_size":8628,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35340288,"num_objects":8628,"num_object_clones":0,"num_object_copies":17256,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8628,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8628,"num_write_kb":34512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'8678","reported_seq":8803,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:01:26.931563+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:01:26.931563+0000","last_peered":"2024-09-10T16:01:26.931563+0000","last_clean":"2024-09-10T16:01:26.931563+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:01:26.931563+0000","last_undegraded":"2024-09-10T16:01:26.931563+0000","last_fullsized":"2024-09-10T16:01:26.931563+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":8678,"log_dups_size":0,"ondisk_log_size":8678,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35516899,"num_objects":8672,"num_object_clones":0,"num_object_copies":17344,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8672,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5389,"num_write_kb":21556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'8696","reported_seq":8826,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:26.820222+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:01:26.820222+0000","last_peered":"2024-09-10T16:01:26.820222+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:01:26.820222+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":8696,"log_dups_size":0,"ondisk_log_size":8696,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35598819,"num_objects":8692,"num_object_clones":0,"num_object_copies":17384,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8692,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5308,"num_write_kb":21232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4321},{"shards":"0,2","objects":4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'8707","reported_seq":8839,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:26.943145+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:01:26.943145+0000","last_peered":"2024-09-10T16:01:26.943145+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:01:26.943145+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":8707,"log_dups_size":0,"ondisk_log_size":8707,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35640262,"num_objects":8703,"num_object_clones":0,"num_object_copies":17406,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8703,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5206,"num_write_kb":20824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4466},{"shards":"0,2","objects":4237}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'8834","reported_seq":8959,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:01:26.943383+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:01:26.943383+0000","last_peered":"2024-09-10T16:01:26.943383+0000","last_clean":"2024-09-10T16:01:26.943383+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:01:26.943383+0000","last_undegraded":"2024-09-10T16:01:26.943383+0000","last_fullsized":"2024-09-10T16:01:26.943383+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":8834,"log_dups_size":0,"ondisk_log_size":8834,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36175872,"num_objects":8832,"num_object_clones":0,"num_object_copies":17664,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8832,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5681,"num_write_kb":22724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.777326+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T15:59:15.777326+0000","last_peered":"2024-09-10T15:59:15.777326+0000","last_clean":"2024-09-10T15:59:15.777326+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T15:59:15.777326+0000","last_undegraded":"2024-09-10T15:59:15.777326+0000","last_fullsized":"2024-09-10T15:59:15.777326+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'8419","reported_seq":8554,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:29.652409+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:01:29.652409+0000","last_peered":"2024-09-10T16:01:29.652409+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:01:29.652409+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":8419,"log_dups_size":0,"ondisk_log_size":8419,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34456035,"num_objects":8413,"num_object_clones":0,"num_object_copies":16826,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8413,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6238,"num_write_kb":24952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4123},{"shards":"2","objects":4290}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":89,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.772619+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T15:59:15.772619+0000","last_peered":"2024-09-10T15:59:15.772619+0000","last_clean":"2024-09-10T15:59:15.772619+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T15:59:15.772619+0000","last_undegraded":"2024-09-10T15:59:15.772619+0000","last_fullsized":"2024-09-10T15:59:15.772619+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'8638","reported_seq":8761,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:26.946997+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:01:26.946997+0000","last_peered":"2024-09-10T16:01:26.946997+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:01:26.946997+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":8638,"log_dups_size":0,"ondisk_log_size":8638,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35348480,"num_objects":8630,"num_object_clones":0,"num_object_copies":17260,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8630,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5142,"num_write_kb":20568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4375},{"shards":"0,2","objects":4255}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":99,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.884297+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T15:59:15.884297+0000","last_peered":"2024-09-10T15:59:15.884297+0000","last_clean":"2024-09-10T15:59:15.884297+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T15:59:15.884297+0000","last_undegraded":"2024-09-10T15:59:15.884297+0000","last_fullsized":"2024-09-10T15:59:15.884297+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'8690","reported_seq":8820,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:29.614446+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:01:29.614446+0000","last_peered":"2024-09-10T16:01:29.614446+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:01:29.614446+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":8690,"log_dups_size":0,"ondisk_log_size":8690,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35574243,"num_objects":8686,"num_object_clones":0,"num_object_copies":17372,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8686,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6370,"num_write_kb":25480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4304},{"shards":"2","objects":4382}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.848670+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T15:59:15.848670+0000","last_peered":"2024-09-10T15:59:15.848670+0000","last_clean":"2024-09-10T15:59:15.848670+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T15:59:15.848670+0000","last_undegraded":"2024-09-10T15:59:15.848670+0000","last_fullsized":"2024-09-10T15:59:15.848670+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'8858","reported_seq":8982,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:29.559446+0000","last_change":"2024-09-10T15:59:16.174113+0000","last_active":"2024-09-10T16:01:29.559446+0000","last_peered":"2024-09-10T16:01:29.559446+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:01:29.559446+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":8858,"log_dups_size":0,"ondisk_log_size":8858,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36257792,"num_objects":8852,"num_object_clones":0,"num_object_copies":17704,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4354,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8852,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6412,"num_write_kb":25648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4352},{"shards":"2","objects":4500}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'8619","reported_seq":8749,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:26.943530+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:01:26.943530+0000","last_peered":"2024-09-10T16:01:26.943530+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:01:26.943530+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":8619,"log_dups_size":0,"ondisk_log_size":8619,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35295232,"num_objects":8617,"num_object_clones":0,"num_object_copies":17234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8617,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5250,"num_write_kb":21000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4313},{"shards":"0,2","objects":4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.992406+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T15:59:15.992406+0000","last_peered":"2024-09-10T15:59:15.992406+0000","last_clean":"2024-09-10T15:59:15.992406+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T15:59:15.992406+0000","last_undegraded":"2024-09-10T15:59:15.992406+0000","last_fullsized":"2024-09-10T15:59:15.992406+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'8496","reported_seq":8633,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:26.908524+0000","last_change":"2024-09-10T15:59:16.251610+0000","last_active":"2024-09-10T16:01:26.908524+0000","last_peered":"2024-09-10T16:01:26.908524+0000","last_clean":"2024-09-10T15:59:14.115595+0000","last_became_active":"2024-09-10T15:59:16.244089+0000","last_became_peered":"2024-09-10T15:59:16.244089+0000","last_unstale":"2024-09-10T16:01:26.908524+0000","last_undegraded":"2024-09-10T15:59:15.847127+0000","last_fullsized":"2024-09-10T15:59:15.774437+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":8496,"log_dups_size":0,"ondisk_log_size":8496,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34766848,"num_objects":8488,"num_object_clones":0,"num_object_copies":16976,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8488,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4886,"num_write_kb":19544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4285},{"shards":"0,2","objects":4203}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":89,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.773633+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T15:59:15.773633+0000","last_peered":"2024-09-10T15:59:15.773633+0000","last_clean":"2024-09-10T15:59:15.773633+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T15:59:15.773633+0000","last_undegraded":"2024-09-10T15:59:15.773633+0000","last_fullsized":"2024-09-10T15:59:15.773633+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'8772","reported_seq":8909,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:26.878276+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:01:26.878276+0000","last_peered":"2024-09-10T16:01:26.878276+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:01:26.878276+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":8772,"log_dups_size":0,"ondisk_log_size":8772,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35913728,"num_objects":8768,"num_object_clones":0,"num_object_copies":17536,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8768,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5027,"num_write_kb":20108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4397},{"shards":"0,2","objects":4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.917634+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T15:59:15.917634+0000","last_peered":"2024-09-10T15:59:15.917634+0000","last_clean":"2024-09-10T15:59:15.917634+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T15:59:15.917634+0000","last_undegraded":"2024-09-10T15:59:15.917634+0000","last_fullsized":"2024-09-10T15:59:15.917634+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":70,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.778232+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T15:59:15.778232+0000","last_peered":"2024-09-10T15:59:15.778232+0000","last_clean":"2024-09-10T15:59:15.778232+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T15:59:15.778232+0000","last_undegraded":"2024-09-10T15:59:15.778232+0000","last_fullsized":"2024-09-10T15:59:15.778232+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'8772","reported_seq":8912,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:26.947316+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:01:26.947316+0000","last_peered":"2024-09-10T16:01:26.947316+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:01:26.947316+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":8772,"log_dups_size":0,"ondisk_log_size":8772,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35881926,"num_objects":8762,"num_object_clones":0,"num_object_copies":17524,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8762,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4935,"num_write_kb":19740,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4439},{"shards":"0,2","objects":4323}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":567997150,"num_objects":138680,"num_object_clones":0,"num_object_copies":277360,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":47117,"num_objects_misplaced":1157,"num_objects_unfound":0,"num_objects_dirty":138680,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":96526,"num_write_kb":386104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3115,"num_bytes_recovered":12754944,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1150787584,"data_stored":1150635838,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":138752,"ondisk_log_size":138752,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986218,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":677848,"kb_used_data":415688,"kb_used_omap":4,"kb_used_meta":262139,"kb_avail":93693992,"statfs":{"total":96636764160,"available":95942647808,"internally_reserved":0,"allocated":425664512,"data_stored":425386752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":268431018},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,0,1,3,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659164,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":400712,"kb_used_data":224628,"kb_used_omap":4,"kb_used_meta":176059,"kb_avail":93971128,"statfs":{"total":96636764160,"available":96226435072,"internally_reserved":0,"allocated":230019072,"data_stored":229693879,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":180285102},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,3,4,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182446,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":794876,"kb_used_data":484204,"kb_used_omap":4,"kb_used_meta":310651,"kb_avail":93576964,"statfs":{"total":96636764160,"available":95822811136,"internally_reserved":0,"allocated":495824896,"data_stored":495543059,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":318107306},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,0,2,0,3,5,6],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":495472640,"data_stored":495436510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":229666816,"data_stored":229587330,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":425648128,"data_stored":425611998,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:01:33.986 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 181 20 141372 141352 3.05015 1.80078 0.0147769 0.025605 2024-09-10T16:01:34.987 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 182 20 141863 141843 3.04393 1.91797 0.0275277 0.0256588 2024-09-10T16:01:35.989 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 183 20 142344 142324 3.03756 1.87891 0.0300826 0.0257103 2024-09-10T16:01:36.256 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:01:36.789 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:01:36.791 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:01:36.817 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":166,"stamp":"2024-09-10T16:01:35.593338+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":577340145,"num_objects":140963,"num_object_clones":0,"num_object_copies":281926,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":47117,"num_objects_misplaced":1025,"num_objects_unfound":0,"num_objects_dirty":140963,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":98809,"num_write_kb":395230,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3249,"num_bytes_recovered":13295635,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":141036,"ondisk_log_size":141036,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1892164,"kb_used_data":1143252,"kb_used_omap":12,"kb_used_meta":748851,"kb_avail":281223356,"statfs":{"total":289910292480,"available":287972716544,"internally_reserved":0,"allocated":1170690048,"data_stored":1169805258,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":766823426},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,3,3,10,8,6,10],"upper_bound":256},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":22433792,"num_objects":5477,"num_object_clones":0,"num_object_copies":10954,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-384,"num_objects_unfound":0,"num_objects_dirty":5477,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5477,"num_write_kb":21908,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":384,"num_bytes_recovered":1572864,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004521"},"pg_stats":[{"pgid":"2.f","version":"43'8721","reported_seq":8791,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:32.046645+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:01:32.046645+0000","last_peered":"2024-09-10T16:01:32.046645+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:01:32.046645+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":8721,"log_dups_size":0,"ondisk_log_size":8721,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35721216,"num_objects":8721,"num_object_clones":0,"num_object_copies":17442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8721,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8721,"num_write_kb":34884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4462},{"shards":"0,2","objects":4258}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'8834","reported_seq":8942,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:01:34.549956+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:01:34.549956+0000","last_peered":"2024-09-10T16:01:34.549956+0000","last_clean":"2024-09-10T16:01:34.549956+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:01:34.549956+0000","last_undegraded":"2024-09-10T16:01:34.549956+0000","last_fullsized":"2024-09-10T16:01:34.549956+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":8834,"log_dups_size":0,"ondisk_log_size":8834,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36160454,"num_objects":8830,"num_object_clones":0,"num_object_copies":17660,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8830,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":6939,"num_write_kb":27756,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'8836","reported_seq":15450,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T16:01:34.678957+0000","last_change":"2024-09-10T15:59:16.142612+0000","last_active":"2024-09-10T16:01:34.678957+0000","last_peered":"2024-09-10T16:01:34.678957+0000","last_clean":"2024-09-10T15:59:14.092473+0000","last_became_active":"2024-09-10T15:59:16.109611+0000","last_became_peered":"2024-09-10T15:59:16.109611+0000","last_unstale":"2024-09-10T16:01:34.678957+0000","last_undegraded":"2024-09-10T16:01:34.678957+0000","last_fullsized":"2024-09-10T15:59:15.770264+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":8836,"log_dups_size":0,"ondisk_log_size":8836,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36175872,"num_objects":8832,"num_object_clones":0,"num_object_copies":17664,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1025,"num_objects_unfound":0,"num_objects_dirty":8832,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6840,"num_write_kb":27360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3244,"num_bytes_recovered":13283328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1025},{"shards":"2","objects":7807}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'8775","reported_seq":8851,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:01:34.625143+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:01:34.625143+0000","last_peered":"2024-09-10T16:01:34.625143+0000","last_clean":"2024-09-10T16:01:34.625143+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:01:34.625143+0000","last_undegraded":"2024-09-10T16:01:34.625143+0000","last_fullsized":"2024-09-10T16:01:34.625143+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":8775,"log_dups_size":0,"ondisk_log_size":8775,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35942400,"num_objects":8775,"num_object_clones":0,"num_object_copies":17550,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8775,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8775,"num_write_kb":35100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'8806","reported_seq":8931,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:01:31.859278+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:01:31.859278+0000","last_peered":"2024-09-10T16:01:31.859278+0000","last_clean":"2024-09-10T16:01:31.859278+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:01:31.859278+0000","last_undegraded":"2024-09-10T16:01:31.859278+0000","last_fullsized":"2024-09-10T16:01:31.859278+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":8806,"log_dups_size":0,"ondisk_log_size":8806,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36041187,"num_objects":8800,"num_object_clones":0,"num_object_copies":17600,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8800,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5517,"num_write_kb":22068,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'8847","reported_seq":8980,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:32.041979+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:01:32.041979+0000","last_peered":"2024-09-10T16:01:32.041979+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:01:32.041979+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":8847,"log_dups_size":0,"ondisk_log_size":8847,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36217315,"num_objects":8843,"num_object_clones":0,"num_object_copies":17686,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8843,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5459,"num_write_kb":21836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4472},{"shards":"0,2","objects":4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'8858","reported_seq":8989,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:31.995971+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:01:31.995971+0000","last_peered":"2024-09-10T16:01:31.995971+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:01:31.995971+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":8858,"log_dups_size":0,"ondisk_log_size":8858,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36258758,"num_objects":8854,"num_object_clones":0,"num_object_copies":17708,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8854,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5357,"num_write_kb":21428,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4617},{"shards":"0,2","objects":4237}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'8965","reported_seq":9090,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:01:32.041381+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:01:32.041381+0000","last_peered":"2024-09-10T16:01:32.041381+0000","last_clean":"2024-09-10T16:01:32.041381+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:01:32.041381+0000","last_undegraded":"2024-09-10T16:01:32.041381+0000","last_fullsized":"2024-09-10T16:01:32.041381+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":8965,"log_dups_size":0,"ondisk_log_size":8965,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36712448,"num_objects":8963,"num_object_clones":0,"num_object_copies":17926,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8963,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5812,"num_write_kb":23248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.777326+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T15:59:15.777326+0000","last_peered":"2024-09-10T15:59:15.777326+0000","last_clean":"2024-09-10T15:59:15.777326+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T15:59:15.777326+0000","last_undegraded":"2024-09-10T15:59:15.777326+0000","last_fullsized":"2024-09-10T15:59:15.777326+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'8574","reported_seq":8709,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:34.649753+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:01:34.649753+0000","last_peered":"2024-09-10T16:01:34.649753+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:01:34.649753+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":8574,"log_dups_size":0,"ondisk_log_size":8574,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35090915,"num_objects":8568,"num_object_clones":0,"num_object_copies":17136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8568,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6393,"num_write_kb":25572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4123},{"shards":"2","objects":4445}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":89,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.772619+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T15:59:15.772619+0000","last_peered":"2024-09-10T15:59:15.772619+0000","last_clean":"2024-09-10T15:59:15.772619+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T15:59:15.772619+0000","last_undegraded":"2024-09-10T15:59:15.772619+0000","last_fullsized":"2024-09-10T15:59:15.772619+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'8787","reported_seq":8912,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:31.959691+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:01:31.959691+0000","last_peered":"2024-09-10T16:01:31.959691+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:01:31.959691+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":8787,"log_dups_size":0,"ondisk_log_size":8787,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35958784,"num_objects":8779,"num_object_clones":0,"num_object_copies":17558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8779,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5291,"num_write_kb":21164,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4524},{"shards":"0,2","objects":4255}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":99,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.884297+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T15:59:15.884297+0000","last_peered":"2024-09-10T15:59:15.884297+0000","last_clean":"2024-09-10T15:59:15.884297+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T15:59:15.884297+0000","last_undegraded":"2024-09-10T15:59:15.884297+0000","last_fullsized":"2024-09-10T15:59:15.884297+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'8834","reported_seq":8962,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:34.599265+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:01:34.599265+0000","last_peered":"2024-09-10T16:01:34.599265+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:01:34.599265+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":8834,"log_dups_size":0,"ondisk_log_size":8834,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36164067,"num_objects":8830,"num_object_clones":0,"num_object_copies":17660,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8830,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6514,"num_write_kb":26056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4304},{"shards":"2","objects":4526}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.848670+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T15:59:15.848670+0000","last_peered":"2024-09-10T15:59:15.848670+0000","last_clean":"2024-09-10T15:59:15.848670+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T15:59:15.848670+0000","last_undegraded":"2024-09-10T15:59:15.848670+0000","last_fullsized":"2024-09-10T15:59:15.848670+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'8996","reported_seq":9120,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:34.658926+0000","last_change":"2024-09-10T15:59:16.174113+0000","last_active":"2024-09-10T16:01:34.658926+0000","last_peered":"2024-09-10T16:01:34.658926+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:01:34.658926+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":8996,"log_dups_size":0,"ondisk_log_size":8996,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36823040,"num_objects":8990,"num_object_clones":0,"num_object_copies":17980,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4354,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8990,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6550,"num_write_kb":26200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4352},{"shards":"2","objects":4638}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'8754","reported_seq":8884,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:32.040825+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:01:32.040825+0000","last_peered":"2024-09-10T16:01:32.040825+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:01:32.040825+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":8754,"log_dups_size":0,"ondisk_log_size":8754,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35848192,"num_objects":8752,"num_object_clones":0,"num_object_copies":17504,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8752,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5385,"num_write_kb":21540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4448},{"shards":"0,2","objects":4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.992406+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T15:59:15.992406+0000","last_peered":"2024-09-10T15:59:15.992406+0000","last_clean":"2024-09-10T15:59:15.992406+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T15:59:15.992406+0000","last_undegraded":"2024-09-10T15:59:15.992406+0000","last_fullsized":"2024-09-10T15:59:15.992406+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'8634","reported_seq":8772,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:31.993640+0000","last_change":"2024-09-10T15:59:16.251610+0000","last_active":"2024-09-10T16:01:31.993640+0000","last_peered":"2024-09-10T16:01:31.993640+0000","last_clean":"2024-09-10T15:59:14.115595+0000","last_became_active":"2024-09-10T15:59:16.244089+0000","last_became_peered":"2024-09-10T15:59:16.244089+0000","last_unstale":"2024-09-10T16:01:31.993640+0000","last_undegraded":"2024-09-10T15:59:15.847127+0000","last_fullsized":"2024-09-10T15:59:15.774437+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":8634,"log_dups_size":0,"ondisk_log_size":8634,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35332096,"num_objects":8626,"num_object_clones":0,"num_object_copies":17252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8626,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5024,"num_write_kb":20096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4423},{"shards":"0,2","objects":4203}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":89,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.773633+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T15:59:15.773633+0000","last_peered":"2024-09-10T15:59:15.773633+0000","last_clean":"2024-09-10T15:59:15.773633+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T15:59:15.773633+0000","last_undegraded":"2024-09-10T15:59:15.773633+0000","last_fullsized":"2024-09-10T15:59:15.773633+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'8910","reported_seq":9047,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:32.022991+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:01:32.022991+0000","last_peered":"2024-09-10T16:01:32.022991+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:01:32.022991+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":8910,"log_dups_size":0,"ondisk_log_size":8910,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36478976,"num_objects":8906,"num_object_clones":0,"num_object_copies":17812,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8906,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5165,"num_write_kb":20660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4535},{"shards":"0,2","objects":4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.917634+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T15:59:15.917634+0000","last_peered":"2024-09-10T15:59:15.917634+0000","last_clean":"2024-09-10T15:59:15.917634+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T15:59:15.917634+0000","last_undegraded":"2024-09-10T15:59:15.917634+0000","last_fullsized":"2024-09-10T15:59:15.917634+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":70,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.778232+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T15:59:15.778232+0000","last_peered":"2024-09-10T15:59:15.778232+0000","last_clean":"2024-09-10T15:59:15.778232+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T15:59:15.778232+0000","last_undegraded":"2024-09-10T15:59:15.778232+0000","last_fullsized":"2024-09-10T15:59:15.778232+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'8902","reported_seq":9042,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:31.982809+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:01:31.982809+0000","last_peered":"2024-09-10T16:01:31.982809+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:01:31.982809+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":8902,"log_dups_size":0,"ondisk_log_size":8902,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36414406,"num_objects":8892,"num_object_clones":0,"num_object_copies":17784,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8892,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5065,"num_write_kb":20260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4569},{"shards":"0,2","objects":4323}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":577340126,"num_objects":140961,"num_object_clones":0,"num_object_copies":281922,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":47117,"num_objects_misplaced":1025,"num_objects_unfound":0,"num_objects_dirty":140961,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":98807,"num_write_kb":395228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3247,"num_bytes_recovered":13295616,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1170124800,"data_stored":1169973054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":141033,"ondisk_log_size":141033,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986219,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":686540,"kb_used_data":420220,"kb_used_omap":4,"kb_used_meta":266299,"kb_avail":93685300,"statfs":{"total":96636764160,"available":95933747200,"internally_reserved":0,"allocated":430305280,"data_stored":430027520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":272690858},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,0,3,1,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659165,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":412348,"kb_used_data":232108,"kb_used_omap":4,"kb_used_meta":180219,"kb_avail":93959492,"statfs":{"total":96636764160,"available":96214519808,"internally_reserved":0,"allocated":237678592,"data_stored":237353399,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":184544942},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,3,1,3,1,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182447,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":793276,"kb_used_data":490924,"kb_used_omap":4,"kb_used_meta":302331,"kb_avail":93578564,"statfs":{"total":96636764160,"available":95824449536,"internally_reserved":0,"allocated":502706176,"data_stored":502424339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":309587626},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,0,0,6,4,3,6],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":502390784,"data_stored":502354654,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":237395968,"data_stored":237316482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":430338048,"data_stored":430301918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:01:36.818 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:01:36.987 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 184 20 142788 142768 3.03048 1.73438 0.0124592 0.0257718 2024-09-10T16:01:37.383 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:01:37.383 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:01:37.406 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":166,"stamp":"2024-09-10T16:01:35.593338+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":577340145,"num_objects":140963,"num_object_clones":0,"num_object_copies":281926,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":47117,"num_objects_misplaced":1025,"num_objects_unfound":0,"num_objects_dirty":140963,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":98809,"num_write_kb":395230,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3249,"num_bytes_recovered":13295635,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":141036,"ondisk_log_size":141036,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1892164,"kb_used_data":1143252,"kb_used_omap":12,"kb_used_meta":748851,"kb_avail":281223356,"statfs":{"total":289910292480,"available":287972716544,"internally_reserved":0,"allocated":1170690048,"data_stored":1169805258,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":766823426},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,3,3,10,8,6,10],"upper_bound":256},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":22433792,"num_objects":5477,"num_object_clones":0,"num_object_copies":10954,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-384,"num_objects_unfound":0,"num_objects_dirty":5477,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5477,"num_write_kb":21908,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":384,"num_bytes_recovered":1572864,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004521"},"pg_stats":[{"pgid":"2.f","version":"43'8721","reported_seq":8791,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:32.046645+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:01:32.046645+0000","last_peered":"2024-09-10T16:01:32.046645+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:01:32.046645+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":8721,"log_dups_size":0,"ondisk_log_size":8721,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35721216,"num_objects":8721,"num_object_clones":0,"num_object_copies":17442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8721,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8721,"num_write_kb":34884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4462},{"shards":"0,2","objects":4258}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'8834","reported_seq":8942,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:01:34.549956+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:01:34.549956+0000","last_peered":"2024-09-10T16:01:34.549956+0000","last_clean":"2024-09-10T16:01:34.549956+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:01:34.549956+0000","last_undegraded":"2024-09-10T16:01:34.549956+0000","last_fullsized":"2024-09-10T16:01:34.549956+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":8834,"log_dups_size":0,"ondisk_log_size":8834,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36160454,"num_objects":8830,"num_object_clones":0,"num_object_copies":17660,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8830,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":6939,"num_write_kb":27756,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'8836","reported_seq":15450,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T16:01:34.678957+0000","last_change":"2024-09-10T15:59:16.142612+0000","last_active":"2024-09-10T16:01:34.678957+0000","last_peered":"2024-09-10T16:01:34.678957+0000","last_clean":"2024-09-10T15:59:14.092473+0000","last_became_active":"2024-09-10T15:59:16.109611+0000","last_became_peered":"2024-09-10T15:59:16.109611+0000","last_unstale":"2024-09-10T16:01:34.678957+0000","last_undegraded":"2024-09-10T16:01:34.678957+0000","last_fullsized":"2024-09-10T15:59:15.770264+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":8836,"log_dups_size":0,"ondisk_log_size":8836,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36175872,"num_objects":8832,"num_object_clones":0,"num_object_copies":17664,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1025,"num_objects_unfound":0,"num_objects_dirty":8832,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6840,"num_write_kb":27360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3244,"num_bytes_recovered":13283328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1025},{"shards":"2","objects":7807}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'8775","reported_seq":8851,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:01:34.625143+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:01:34.625143+0000","last_peered":"2024-09-10T16:01:34.625143+0000","last_clean":"2024-09-10T16:01:34.625143+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:01:34.625143+0000","last_undegraded":"2024-09-10T16:01:34.625143+0000","last_fullsized":"2024-09-10T16:01:34.625143+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":8775,"log_dups_size":0,"ondisk_log_size":8775,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35942400,"num_objects":8775,"num_object_clones":0,"num_object_copies":17550,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8775,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8775,"num_write_kb":35100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'8806","reported_seq":8931,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:01:31.859278+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:01:31.859278+0000","last_peered":"2024-09-10T16:01:31.859278+0000","last_clean":"2024-09-10T16:01:31.859278+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:01:31.859278+0000","last_undegraded":"2024-09-10T16:01:31.859278+0000","last_fullsized":"2024-09-10T16:01:31.859278+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":8806,"log_dups_size":0,"ondisk_log_size":8806,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36041187,"num_objects":8800,"num_object_clones":0,"num_object_copies":17600,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8800,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5517,"num_write_kb":22068,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'8847","reported_seq":8980,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:32.041979+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:01:32.041979+0000","last_peered":"2024-09-10T16:01:32.041979+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:01:32.041979+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":8847,"log_dups_size":0,"ondisk_log_size":8847,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36217315,"num_objects":8843,"num_object_clones":0,"num_object_copies":17686,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8843,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5459,"num_write_kb":21836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4472},{"shards":"0,2","objects":4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'8858","reported_seq":8989,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:31.995971+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:01:31.995971+0000","last_peered":"2024-09-10T16:01:31.995971+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:01:31.995971+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":8858,"log_dups_size":0,"ondisk_log_size":8858,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36258758,"num_objects":8854,"num_object_clones":0,"num_object_copies":17708,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8854,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5357,"num_write_kb":21428,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4617},{"shards":"0,2","objects":4237}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'8965","reported_seq":9090,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:01:32.041381+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:01:32.041381+0000","last_peered":"2024-09-10T16:01:32.041381+0000","last_clean":"2024-09-10T16:01:32.041381+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:01:32.041381+0000","last_undegraded":"2024-09-10T16:01:32.041381+0000","last_fullsized":"2024-09-10T16:01:32.041381+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":8965,"log_dups_size":0,"ondisk_log_size":8965,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36712448,"num_objects":8963,"num_object_clones":0,"num_object_copies":17926,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8963,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5812,"num_write_kb":23248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.777326+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T15:59:15.777326+0000","last_peered":"2024-09-10T15:59:15.777326+0000","last_clean":"2024-09-10T15:59:15.777326+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T15:59:15.777326+0000","last_undegraded":"2024-09-10T15:59:15.777326+0000","last_fullsized":"2024-09-10T15:59:15.777326+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'8574","reported_seq":8709,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:34.649753+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:01:34.649753+0000","last_peered":"2024-09-10T16:01:34.649753+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:01:34.649753+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":8574,"log_dups_size":0,"ondisk_log_size":8574,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35090915,"num_objects":8568,"num_object_clones":0,"num_object_copies":17136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8568,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6393,"num_write_kb":25572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4123},{"shards":"2","objects":4445}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":89,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.772619+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T15:59:15.772619+0000","last_peered":"2024-09-10T15:59:15.772619+0000","last_clean":"2024-09-10T15:59:15.772619+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T15:59:15.772619+0000","last_undegraded":"2024-09-10T15:59:15.772619+0000","last_fullsized":"2024-09-10T15:59:15.772619+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'8787","reported_seq":8912,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:31.959691+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:01:31.959691+0000","last_peered":"2024-09-10T16:01:31.959691+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:01:31.959691+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":8787,"log_dups_size":0,"ondisk_log_size":8787,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35958784,"num_objects":8779,"num_object_clones":0,"num_object_copies":17558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8779,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5291,"num_write_kb":21164,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4524},{"shards":"0,2","objects":4255}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":99,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.884297+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T15:59:15.884297+0000","last_peered":"2024-09-10T15:59:15.884297+0000","last_clean":"2024-09-10T15:59:15.884297+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T15:59:15.884297+0000","last_undegraded":"2024-09-10T15:59:15.884297+0000","last_fullsized":"2024-09-10T15:59:15.884297+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'8834","reported_seq":8962,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:34.599265+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:01:34.599265+0000","last_peered":"2024-09-10T16:01:34.599265+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:01:34.599265+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":8834,"log_dups_size":0,"ondisk_log_size":8834,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36164067,"num_objects":8830,"num_object_clones":0,"num_object_copies":17660,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8830,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6514,"num_write_kb":26056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4304},{"shards":"2","objects":4526}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.848670+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T15:59:15.848670+0000","last_peered":"2024-09-10T15:59:15.848670+0000","last_clean":"2024-09-10T15:59:15.848670+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T15:59:15.848670+0000","last_undegraded":"2024-09-10T15:59:15.848670+0000","last_fullsized":"2024-09-10T15:59:15.848670+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'8996","reported_seq":9120,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:34.658926+0000","last_change":"2024-09-10T15:59:16.174113+0000","last_active":"2024-09-10T16:01:34.658926+0000","last_peered":"2024-09-10T16:01:34.658926+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:01:34.658926+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":8996,"log_dups_size":0,"ondisk_log_size":8996,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36823040,"num_objects":8990,"num_object_clones":0,"num_object_copies":17980,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4354,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8990,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6550,"num_write_kb":26200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4352},{"shards":"2","objects":4638}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'8754","reported_seq":8884,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:32.040825+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:01:32.040825+0000","last_peered":"2024-09-10T16:01:32.040825+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:01:32.040825+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":8754,"log_dups_size":0,"ondisk_log_size":8754,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35848192,"num_objects":8752,"num_object_clones":0,"num_object_copies":17504,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8752,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5385,"num_write_kb":21540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4448},{"shards":"0,2","objects":4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.992406+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T15:59:15.992406+0000","last_peered":"2024-09-10T15:59:15.992406+0000","last_clean":"2024-09-10T15:59:15.992406+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T15:59:15.992406+0000","last_undegraded":"2024-09-10T15:59:15.992406+0000","last_fullsized":"2024-09-10T15:59:15.992406+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'8634","reported_seq":8772,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:31.993640+0000","last_change":"2024-09-10T15:59:16.251610+0000","last_active":"2024-09-10T16:01:31.993640+0000","last_peered":"2024-09-10T16:01:31.993640+0000","last_clean":"2024-09-10T15:59:14.115595+0000","last_became_active":"2024-09-10T15:59:16.244089+0000","last_became_peered":"2024-09-10T15:59:16.244089+0000","last_unstale":"2024-09-10T16:01:31.993640+0000","last_undegraded":"2024-09-10T15:59:15.847127+0000","last_fullsized":"2024-09-10T15:59:15.774437+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":8634,"log_dups_size":0,"ondisk_log_size":8634,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35332096,"num_objects":8626,"num_object_clones":0,"num_object_copies":17252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8626,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5024,"num_write_kb":20096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4423},{"shards":"0,2","objects":4203}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":89,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.773633+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T15:59:15.773633+0000","last_peered":"2024-09-10T15:59:15.773633+0000","last_clean":"2024-09-10T15:59:15.773633+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T15:59:15.773633+0000","last_undegraded":"2024-09-10T15:59:15.773633+0000","last_fullsized":"2024-09-10T15:59:15.773633+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'8910","reported_seq":9047,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:32.022991+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:01:32.022991+0000","last_peered":"2024-09-10T16:01:32.022991+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:01:32.022991+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":8910,"log_dups_size":0,"ondisk_log_size":8910,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36478976,"num_objects":8906,"num_object_clones":0,"num_object_copies":17812,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8906,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5165,"num_write_kb":20660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4535},{"shards":"0,2","objects":4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.917634+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T15:59:15.917634+0000","last_peered":"2024-09-10T15:59:15.917634+0000","last_clean":"2024-09-10T15:59:15.917634+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T15:59:15.917634+0000","last_undegraded":"2024-09-10T15:59:15.917634+0000","last_fullsized":"2024-09-10T15:59:15.917634+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":70,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.778232+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T15:59:15.778232+0000","last_peered":"2024-09-10T15:59:15.778232+0000","last_clean":"2024-09-10T15:59:15.778232+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T15:59:15.778232+0000","last_undegraded":"2024-09-10T15:59:15.778232+0000","last_fullsized":"2024-09-10T15:59:15.778232+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'8902","reported_seq":9042,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:31.982809+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:01:31.982809+0000","last_peered":"2024-09-10T16:01:31.982809+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:01:31.982809+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":8902,"log_dups_size":0,"ondisk_log_size":8902,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36414406,"num_objects":8892,"num_object_clones":0,"num_object_copies":17784,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8892,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5065,"num_write_kb":20260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4569},{"shards":"0,2","objects":4323}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":577340126,"num_objects":140961,"num_object_clones":0,"num_object_copies":281922,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":47117,"num_objects_misplaced":1025,"num_objects_unfound":0,"num_objects_dirty":140961,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":98807,"num_write_kb":395228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3247,"num_bytes_recovered":13295616,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1170124800,"data_stored":1169973054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":141033,"ondisk_log_size":141033,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986219,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":686540,"kb_used_data":420220,"kb_used_omap":4,"kb_used_meta":266299,"kb_avail":93685300,"statfs":{"total":96636764160,"available":95933747200,"internally_reserved":0,"allocated":430305280,"data_stored":430027520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":272690858},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,0,3,1,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659165,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":412348,"kb_used_data":232108,"kb_used_omap":4,"kb_used_meta":180219,"kb_avail":93959492,"statfs":{"total":96636764160,"available":96214519808,"internally_reserved":0,"allocated":237678592,"data_stored":237353399,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":184544942},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,3,1,3,1,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182447,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":793276,"kb_used_data":490924,"kb_used_omap":4,"kb_used_meta":302331,"kb_avail":93578564,"statfs":{"total":96636764160,"available":95824449536,"internally_reserved":0,"allocated":502706176,"data_stored":502424339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":309587626},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,0,0,6,4,3,6],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":502390784,"data_stored":502354654,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":237395968,"data_stored":237316482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":430338048,"data_stored":430301918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:01:37.407 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-10T16:01:37.407 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "43'8721", 'reported_seq': 8791, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:01:32.046645+0000', 'last_change': '2024-09-10T15:59:16.281558+0000', 'last_active': '2024-09-10T16:01:32.046645+0000', 'last_peered': '2024-09-10T16:01:32.046645+0000', 'last_clean': '2024-09-10T15:59:14.132456+0000', 'last_became_active': '2024-09-10T15:59:16.274407+0000', 'last_became_peered': '2024-09-10T15:59:16.274407+0000', 'last_unstale': '2024-09-10T16:01:32.046645+0000', 'last_undegraded': '2024-09-10T15:59:15.851279+0000', 'last_fullsized': '2024-09-10T15:59:15.774356+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'objects_scrubbed': 0, 'log_size': 8721, 'log_dups_size': 0, 'ondisk_log_size': 8721, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35721216, 'num_objects': 8721, 'num_object_clones': 0, 'num_object_copies': 17442, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4259, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8721, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8721, 'num_write_kb': 34884, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4462}, {'shards': '0,2', 'objects': 4258}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:01:37.408 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-10T16:01:37.408 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "43'8836", 'reported_seq': 15450, 'reported_epoch': 43, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-10T16:01:34.678957+0000', 'last_change': '2024-09-10T15:59:16.142612+0000', 'last_active': '2024-09-10T16:01:34.678957+0000', 'last_peered': '2024-09-10T16:01:34.678957+0000', 'last_clean': '2024-09-10T15:59:14.092473+0000', 'last_became_active': '2024-09-10T15:59:16.109611+0000', 'last_became_peered': '2024-09-10T15:59:16.109611+0000', 'last_unstale': '2024-09-10T16:01:34.678957+0000', 'last_undegraded': '2024-09-10T16:01:34.678957+0000', 'last_fullsized': '2024-09-10T15:59:15.770264+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2029", 'last_scrub_stamp': '2024-09-10T15:58:45.933789+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:45.933789+0000', 'objects_scrubbed': 1992, 'log_size': 8836, 'log_dups_size': 0, 'ondisk_log_size': 8836, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 668, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36175872, 'num_objects': 8832, 'num_object_clones': 0, 'num_object_copies': 17664, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1025, 'num_objects_unfound': 0, 'num_objects_dirty': 8832, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6840, 'num_write_kb': 27360, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3244, 'num_bytes_recovered': 13283328, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1025}, {'shards': '2', 'objects': 7807}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:01:37.408 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-10T16:01:37.408 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "43'8847", 'reported_seq': 8980, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:01:32.041979+0000', 'last_change': '2024-09-10T15:59:16.308264+0000', 'last_active': '2024-09-10T16:01:32.041979+0000', 'last_peered': '2024-09-10T16:01:32.041979+0000', 'last_clean': '2024-09-10T15:59:14.111551+0000', 'last_became_active': '2024-09-10T15:59:16.300040+0000', 'last_became_peered': '2024-09-10T15:59:16.300040+0000', 'last_unstale': '2024-09-10T16:01:32.041979+0000', 'last_undegraded': '2024-09-10T15:59:15.878234+0000', 'last_fullsized': '2024-09-10T15:59:15.769328+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3442", 'last_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'objects_scrubbed': 3384, 'log_size': 8847, 'log_dups_size': 0, 'ondisk_log_size': 8847, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1292, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36217315, 'num_objects': 8843, 'num_object_clones': 0, 'num_object_copies': 17686, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4372, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8843, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 5459, 'num_write_kb': 21836, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4472}, {'shards': '0,2', 'objects': 4371}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:01:37.408 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-10T16:01:37.408 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "43'8858", 'reported_seq': 8989, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:01:31.995971+0000', 'last_change': '2024-09-10T15:59:16.331899+0000', 'last_active': '2024-09-10T16:01:31.995971+0000', 'last_peered': '2024-09-10T16:01:31.995971+0000', 'last_clean': '2024-09-10T15:59:14.139436+0000', 'last_became_active': '2024-09-10T15:59:16.318715+0000', 'last_became_peered': '2024-09-10T15:59:16.318715+0000', 'last_unstale': '2024-09-10T16:01:31.995971+0000', 'last_undegraded': '2024-09-10T15:59:15.877524+0000', 'last_fullsized': '2024-09-10T15:59:15.769042+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3537", 'last_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'objects_scrubbed': 3497, 'log_size': 8858, 'log_dups_size': 0, 'ondisk_log_size': 8858, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1224, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36258758, 'num_objects': 8854, 'num_object_clones': 0, 'num_object_copies': 17708, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4237, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8854, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 5357, 'num_write_kb': 21428, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4617}, {'shards': '0,2', 'objects': 4237}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:01:37.408 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-10T16:01:37.408 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "43'8574", 'reported_seq': 8709, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:01:34.649753+0000', 'last_change': '2024-09-10T15:59:16.175680+0000', 'last_active': '2024-09-10T16:01:34.649753+0000', 'last_peered': '2024-09-10T16:01:34.649753+0000', 'last_clean': '2024-09-10T15:59:14.088187+0000', 'last_became_active': '2024-09-10T15:59:16.165810+0000', 'last_became_peered': '2024-09-10T15:59:16.165810+0000', 'last_unstale': '2024-09-10T16:01:34.649753+0000', 'last_undegraded': '2024-09-10T15:59:15.822098+0000', 'last_fullsized': '2024-09-10T15:59:15.769123+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2218", 'last_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'objects_scrubbed': 2175, 'log_size': 8574, 'log_dups_size': 0, 'ondisk_log_size': 8574, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 825, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35090915, 'num_objects': 8568, 'num_object_clones': 0, 'num_object_copies': 17136, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4124, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8568, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 6393, 'num_write_kb': 25572, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4123}, {'shards': '2', 'objects': 4445}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:01:37.408 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-10T16:01:37.408 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "43'8787", 'reported_seq': 8912, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:01:31.959691+0000', 'last_change': '2024-09-10T15:59:16.277742+0000', 'last_active': '2024-09-10T16:01:31.959691+0000', 'last_peered': '2024-09-10T16:01:31.959691+0000', 'last_clean': '2024-09-10T15:59:14.129081+0000', 'last_became_active': '2024-09-10T15:59:16.269835+0000', 'last_became_peered': '2024-09-10T15:59:16.269835+0000', 'last_unstale': '2024-09-10T16:01:31.959691+0000', 'last_undegraded': '2024-09-10T15:59:15.860857+0000', 'last_fullsized': '2024-09-10T15:59:15.771410+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3557", 'last_scrub_stamp': '2024-09-10T15:59:02.493678+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:02.493678+0000', 'objects_scrubbed': 3488, 'log_size': 8787, 'log_dups_size': 0, 'ondisk_log_size': 8787, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1472, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35958784, 'num_objects': 8779, 'num_object_clones': 0, 'num_object_copies': 17558, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4257, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8779, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5291, 'num_write_kb': 21164, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4524}, {'shards': '0,2', 'objects': 4255}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:01:37.409 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-10T16:01:37.409 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "43'8834", 'reported_seq': 8962, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:01:34.599265+0000', 'last_change': '2024-09-10T15:59:16.284311+0000', 'last_active': '2024-09-10T16:01:34.599265+0000', 'last_peered': '2024-09-10T16:01:34.599265+0000', 'last_clean': '2024-09-10T15:59:14.116041+0000', 'last_became_active': '2024-09-10T15:59:16.274543+0000', 'last_became_peered': '2024-09-10T15:59:16.274543+0000', 'last_unstale': '2024-09-10T16:01:34.599265+0000', 'last_undegraded': '2024-09-10T15:59:15.812484+0000', 'last_fullsized': '2024-09-10T15:59:15.770885+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2356", 'last_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'objects_scrubbed': 2316, 'log_size': 8834, 'log_dups_size': 0, 'ondisk_log_size': 8834, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 783, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36164067, 'num_objects': 8830, 'num_object_clones': 0, 'num_object_copies': 17660, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8830, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 6514, 'num_write_kb': 26056, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4304}, {'shards': '2', 'objects': 4526}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:01:37.409 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-10T16:01:37.409 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "43'8996", 'reported_seq': 9120, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:01:34.658926+0000', 'last_change': '2024-09-10T15:59:16.174113+0000', 'last_active': '2024-09-10T16:01:34.658926+0000', 'last_peered': '2024-09-10T16:01:34.658926+0000', 'last_clean': '2024-09-10T15:59:14.110654+0000', 'last_became_active': '2024-09-10T15:59:16.164645+0000', 'last_became_peered': '2024-09-10T15:59:16.164645+0000', 'last_unstale': '2024-09-10T16:01:34.658926+0000', 'last_undegraded': '2024-09-10T15:59:15.803593+0000', 'last_fullsized': '2024-09-10T15:59:15.769083+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2486", 'last_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'objects_scrubbed': 2440, 'log_size': 8996, 'log_dups_size': 0, 'ondisk_log_size': 8996, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 792, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36823040, 'num_objects': 8990, 'num_object_clones': 0, 'num_object_copies': 17980, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4354, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8990, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6550, 'num_write_kb': 26200, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4352}, {'shards': '2', 'objects': 4638}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:01:37.409 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-10T16:01:37.409 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "43'8754", 'reported_seq': 8884, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:01:32.040825+0000', 'last_change': '2024-09-10T15:59:16.315641+0000', 'last_active': '2024-09-10T16:01:32.040825+0000', 'last_peered': '2024-09-10T16:01:32.040825+0000', 'last_clean': '2024-09-10T15:59:14.117001+0000', 'last_became_active': '2024-09-10T15:59:16.302245+0000', 'last_became_peered': '2024-09-10T15:59:16.302245+0000', 'last_unstale': '2024-09-10T16:01:32.040825+0000', 'last_undegraded': '2024-09-10T15:59:15.845012+0000', 'last_fullsized': '2024-09-10T15:59:15.770337+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3404", 'last_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'objects_scrubbed': 3367, 'log_size': 8754, 'log_dups_size': 0, 'ondisk_log_size': 8754, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 983, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35848192, 'num_objects': 8752, 'num_object_clones': 0, 'num_object_copies': 17504, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8752, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5385, 'num_write_kb': 21540, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4448}, {'shards': '0,2', 'objects': 4304}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:01:37.409 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-10T16:01:37.409 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "43'8634", 'reported_seq': 8772, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:01:31.993640+0000', 'last_change': '2024-09-10T15:59:16.251610+0000', 'last_active': '2024-09-10T16:01:31.993640+0000', 'last_peered': '2024-09-10T16:01:31.993640+0000', 'last_clean': '2024-09-10T15:59:14.115595+0000', 'last_became_active': '2024-09-10T15:59:16.244089+0000', 'last_became_peered': '2024-09-10T15:59:16.244089+0000', 'last_unstale': '2024-09-10T16:01:31.993640+0000', 'last_undegraded': '2024-09-10T15:59:15.847127+0000', 'last_fullsized': '2024-09-10T15:59:15.774437+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3653", 'last_scrub_stamp': '2024-09-10T15:59:05.298468+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:05.298468+0000', 'objects_scrubbed': 3602, 'log_size': 8634, 'log_dups_size': 0, 'ondisk_log_size': 8634, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1289, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35332096, 'num_objects': 8626, 'num_object_clones': 0, 'num_object_copies': 17252, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4206, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8626, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5024, 'num_write_kb': 20096, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4423}, {'shards': '0,2', 'objects': 4203}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:01:37.409 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-10T16:01:37.409 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "43'8910", 'reported_seq': 9047, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:01:32.022991+0000', 'last_change': '2024-09-10T15:59:16.311291+0000', 'last_active': '2024-09-10T16:01:32.022991+0000', 'last_peered': '2024-09-10T16:01:32.022991+0000', 'last_clean': '2024-09-10T15:59:14.116363+0000', 'last_became_active': '2024-09-10T15:59:16.301244+0000', 'last_became_peered': '2024-09-10T15:59:16.301244+0000', 'last_unstale': '2024-09-10T16:01:32.022991+0000', 'last_undegraded': '2024-09-10T15:59:15.883434+0000', 'last_fullsized': '2024-09-10T15:59:15.772756+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3789", 'last_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'objects_scrubbed': 3741, 'log_size': 8910, 'log_dups_size': 0, 'ondisk_log_size': 8910, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1440, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36478976, 'num_objects': 8906, 'num_object_clones': 0, 'num_object_copies': 17812, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4373, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8906, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5165, 'num_write_kb': 20660, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4535}, {'shards': '0,2', 'objects': 4371}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:01:37.410 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-10T16:01:37.410 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "43'8902", 'reported_seq': 9042, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:01:31.982809+0000', 'last_change': '2024-09-10T15:59:16.313409+0000', 'last_active': '2024-09-10T16:01:31.982809+0000', 'last_peered': '2024-09-10T16:01:31.982809+0000', 'last_clean': '2024-09-10T15:59:14.129780+0000', 'last_became_active': '2024-09-10T15:59:16.300932+0000', 'last_became_peered': '2024-09-10T15:59:16.300932+0000', 'last_unstale': '2024-09-10T16:01:31.982809+0000', 'last_undegraded': '2024-09-10T15:59:15.887823+0000', 'last_fullsized': '2024-09-10T15:59:15.768513+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3880", 'last_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'objects_scrubbed': 3827, 'log_size': 8902, 'log_dups_size': 0, 'ondisk_log_size': 8902, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1397, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36414406, 'num_objects': 8892, 'num_object_clones': 0, 'num_object_copies': 17784, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4325, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8892, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5065, 'num_write_kb': 20260, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4569}, {'shards': '0,2', 'objects': 4323}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:01:37.410 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T16:01:37.987 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 185 20 143216 143196 3.02313 1.67188 0.0928107 0.0258344 2024-09-10T16:01:38.100 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:01:38.101 INFO:teuthology.orchestra.run.smithi115.stdout:{"fsid":"7effee29-df05-4351-9741-ee203564ed81","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 47117/284846 objects degraded (16.541%), 11 pgs degraded, 12 pgs undersized","count":23},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":262,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":43,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983954,"num_in_osds":3,"osd_in_since":1725983947,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":142423,"data_bytes":583320305,"bytes_used":1948893184,"bytes_avail":287961399296,"bytes_total":289910292480,"degraded_objects":47117,"degraded_total":284846,"degraded_ratio":0.16541218763823259,"misplaced_objects":1025,"misplaced_total":284846,"misplaced_ratio":0.0035984356459279753,"recovering_objects_per_sec":21,"recovering_bytes_per_sec":89394,"recovering_keys_per_sec":0,"num_objects_recovered":262,"num_bytes_recovered":1073152,"num_keys_recovered":0,"write_bytes_sec":2071773,"read_op_per_sec":0,"write_op_per_sec":505},"fsmap":{"epoch":1,"btime":"2024-09-10T15:57:14:967120+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T16:00:13.560776+0000","services":{}},"progress_events":{}} 2024-09-10T16:01:38.124 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-10T16:01:38.124 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:01:38.715 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:01:38.717 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:01:38.739 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":167,"stamp":"2024-09-10T16:01:37.594238+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":583320305,"num_objects":142423,"num_object_clones":0,"num_object_copies":284846,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":47117,"num_objects_misplaced":1025,"num_objects_unfound":0,"num_objects_dirty":142423,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":100269,"num_write_kb":401070,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3249,"num_bytes_recovered":13295635,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":142496,"ondisk_log_size":142496,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1903216,"kb_used_data":1150144,"kb_used_omap":12,"kb_used_meta":753011,"kb_avail":281212304,"statfs":{"total":289910292480,"available":287961399296,"internally_reserved":0,"allocated":1177747456,"data_stored":1176862666,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":771083266},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,9,3,8,6,9,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":24870912,"num_objects":6072,"num_object_clones":0,"num_object_copies":12144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-262,"num_objects_unfound":0,"num_objects_dirty":6072,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6072,"num_write_kb":24288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":262,"num_bytes_recovered":1073152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004650"},"pg_stats":[{"pgid":"2.f","version":"43'8851","reported_seq":8918,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:37.064389+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:01:37.064389+0000","last_peered":"2024-09-10T16:01:37.064389+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:01:37.064389+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":8851,"log_dups_size":0,"ondisk_log_size":8851,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36253696,"num_objects":8851,"num_object_clones":0,"num_object_copies":17702,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8851,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8851,"num_write_kb":35404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4592},{"shards":"0,2","objects":4258}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'8834","reported_seq":8942,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:01:34.549956+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:01:34.549956+0000","last_peered":"2024-09-10T16:01:34.549956+0000","last_clean":"2024-09-10T16:01:34.549956+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:01:34.549956+0000","last_undegraded":"2024-09-10T16:01:34.549956+0000","last_fullsized":"2024-09-10T16:01:34.549956+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":8834,"log_dups_size":0,"ondisk_log_size":8834,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36160454,"num_objects":8830,"num_object_clones":0,"num_object_copies":17660,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8830,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":6939,"num_write_kb":27756,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'8836","reported_seq":15450,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T16:01:34.678957+0000","last_change":"2024-09-10T15:59:16.142612+0000","last_active":"2024-09-10T16:01:34.678957+0000","last_peered":"2024-09-10T16:01:34.678957+0000","last_clean":"2024-09-10T15:59:14.092473+0000","last_became_active":"2024-09-10T15:59:16.109611+0000","last_became_peered":"2024-09-10T15:59:16.109611+0000","last_unstale":"2024-09-10T16:01:34.678957+0000","last_undegraded":"2024-09-10T16:01:34.678957+0000","last_fullsized":"2024-09-10T15:59:15.770264+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":8836,"log_dups_size":0,"ondisk_log_size":8836,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36175872,"num_objects":8832,"num_object_clones":0,"num_object_copies":17664,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1025,"num_objects_unfound":0,"num_objects_dirty":8832,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6840,"num_write_kb":27360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3244,"num_bytes_recovered":13283328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1025},{"shards":"2","objects":7807}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'8775","reported_seq":8851,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:01:34.625143+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:01:34.625143+0000","last_peered":"2024-09-10T16:01:34.625143+0000","last_clean":"2024-09-10T16:01:34.625143+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:01:34.625143+0000","last_undegraded":"2024-09-10T16:01:34.625143+0000","last_fullsized":"2024-09-10T16:01:34.625143+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":8775,"log_dups_size":0,"ondisk_log_size":8775,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35942400,"num_objects":8775,"num_object_clones":0,"num_object_copies":17550,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8775,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8775,"num_write_kb":35100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'8947","reported_seq":9072,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:01:37.052049+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:01:37.052049+0000","last_peered":"2024-09-10T16:01:37.052049+0000","last_clean":"2024-09-10T16:01:37.052049+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:01:37.052049+0000","last_undegraded":"2024-09-10T16:01:37.052049+0000","last_fullsized":"2024-09-10T16:01:37.052049+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":8947,"log_dups_size":0,"ondisk_log_size":8947,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36618723,"num_objects":8941,"num_object_clones":0,"num_object_copies":17882,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8941,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5658,"num_write_kb":22632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'8976","reported_seq":9109,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:37.010273+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:01:37.010273+0000","last_peered":"2024-09-10T16:01:37.010273+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:01:37.010273+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":8976,"log_dups_size":0,"ondisk_log_size":8976,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36745699,"num_objects":8972,"num_object_clones":0,"num_object_copies":17944,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8972,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5588,"num_write_kb":22352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4601},{"shards":"0,2","objects":4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'8999","reported_seq":9131,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:37.012689+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:01:37.012689+0000","last_peered":"2024-09-10T16:01:37.012689+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:01:37.012689+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":8999,"log_dups_size":0,"ondisk_log_size":8999,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36836294,"num_objects":8995,"num_object_clones":0,"num_object_copies":17990,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8995,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5498,"num_write_kb":21992,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4758},{"shards":"0,2","objects":4237}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'9111","reported_seq":9236,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:01:37.075014+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:01:37.075014+0000","last_peered":"2024-09-10T16:01:37.075014+0000","last_clean":"2024-09-10T16:01:37.075014+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:01:37.075014+0000","last_undegraded":"2024-09-10T16:01:37.075014+0000","last_fullsized":"2024-09-10T16:01:37.075014+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":9111,"log_dups_size":0,"ondisk_log_size":9111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37310464,"num_objects":9109,"num_object_clones":0,"num_object_copies":18218,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9109,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5958,"num_write_kb":23832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.777326+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T15:59:15.777326+0000","last_peered":"2024-09-10T15:59:15.777326+0000","last_clean":"2024-09-10T15:59:15.777326+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T15:59:15.777326+0000","last_undegraded":"2024-09-10T15:59:15.777326+0000","last_fullsized":"2024-09-10T15:59:15.777326+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'8574","reported_seq":8709,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:34.649753+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:01:34.649753+0000","last_peered":"2024-09-10T16:01:34.649753+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:01:34.649753+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":8574,"log_dups_size":0,"ondisk_log_size":8574,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35090915,"num_objects":8568,"num_object_clones":0,"num_object_copies":17136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8568,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6393,"num_write_kb":25572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4123},{"shards":"2","objects":4445}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":89,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.772619+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T15:59:15.772619+0000","last_peered":"2024-09-10T15:59:15.772619+0000","last_clean":"2024-09-10T15:59:15.772619+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T15:59:15.772619+0000","last_undegraded":"2024-09-10T15:59:15.772619+0000","last_fullsized":"2024-09-10T15:59:15.772619+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'8936","reported_seq":9062,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:37.026290+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:01:37.026290+0000","last_peered":"2024-09-10T16:01:37.026290+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:01:37.026290+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":8936,"log_dups_size":0,"ondisk_log_size":8936,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36569088,"num_objects":8928,"num_object_clones":0,"num_object_copies":17856,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8928,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5440,"num_write_kb":21760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4673},{"shards":"0,2","objects":4255}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":99,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.884297+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T15:59:15.884297+0000","last_peered":"2024-09-10T15:59:15.884297+0000","last_clean":"2024-09-10T15:59:15.884297+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T15:59:15.884297+0000","last_undegraded":"2024-09-10T15:59:15.884297+0000","last_fullsized":"2024-09-10T15:59:15.884297+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'8834","reported_seq":8962,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:34.599265+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:01:34.599265+0000","last_peered":"2024-09-10T16:01:34.599265+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:01:34.599265+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":8834,"log_dups_size":0,"ondisk_log_size":8834,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36164067,"num_objects":8830,"num_object_clones":0,"num_object_copies":17660,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8830,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6514,"num_write_kb":26056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4304},{"shards":"2","objects":4526}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.848670+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T15:59:15.848670+0000","last_peered":"2024-09-10T15:59:15.848670+0000","last_clean":"2024-09-10T15:59:15.848670+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T15:59:15.848670+0000","last_undegraded":"2024-09-10T15:59:15.848670+0000","last_fullsized":"2024-09-10T15:59:15.848670+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'8996","reported_seq":9120,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:34.658926+0000","last_change":"2024-09-10T15:59:16.174113+0000","last_active":"2024-09-10T16:01:34.658926+0000","last_peered":"2024-09-10T16:01:34.658926+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:01:34.658926+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":8996,"log_dups_size":0,"ondisk_log_size":8996,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36823040,"num_objects":8990,"num_object_clones":0,"num_object_copies":17980,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4354,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8990,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6550,"num_write_kb":26200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4352},{"shards":"2","objects":4638}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'8910","reported_seq":9040,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:36.927199+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:01:36.927199+0000","last_peered":"2024-09-10T16:01:36.927199+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:01:36.927199+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":8910,"log_dups_size":0,"ondisk_log_size":8910,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36487168,"num_objects":8908,"num_object_clones":0,"num_object_copies":17816,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8908,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5541,"num_write_kb":22164,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4604},{"shards":"0,2","objects":4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.992406+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T15:59:15.992406+0000","last_peered":"2024-09-10T15:59:15.992406+0000","last_clean":"2024-09-10T15:59:15.992406+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T15:59:15.992406+0000","last_undegraded":"2024-09-10T15:59:15.992406+0000","last_fullsized":"2024-09-10T15:59:15.992406+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'8787","reported_seq":8925,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:37.017762+0000","last_change":"2024-09-10T15:59:16.251610+0000","last_active":"2024-09-10T16:01:37.017762+0000","last_peered":"2024-09-10T16:01:37.017762+0000","last_clean":"2024-09-10T15:59:14.115595+0000","last_became_active":"2024-09-10T15:59:16.244089+0000","last_became_peered":"2024-09-10T15:59:16.244089+0000","last_unstale":"2024-09-10T16:01:37.017762+0000","last_undegraded":"2024-09-10T15:59:15.847127+0000","last_fullsized":"2024-09-10T15:59:15.774437+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":8787,"log_dups_size":0,"ondisk_log_size":8787,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35958784,"num_objects":8779,"num_object_clones":0,"num_object_copies":17558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8779,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5177,"num_write_kb":20708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4576},{"shards":"0,2","objects":4203}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":89,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.773633+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T15:59:15.773633+0000","last_peered":"2024-09-10T15:59:15.773633+0000","last_clean":"2024-09-10T15:59:15.773633+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T15:59:15.773633+0000","last_undegraded":"2024-09-10T15:59:15.773633+0000","last_fullsized":"2024-09-10T15:59:15.773633+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'9071","reported_seq":9208,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:37.038648+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:01:37.038648+0000","last_peered":"2024-09-10T16:01:37.038648+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:01:37.038648+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":9071,"log_dups_size":0,"ondisk_log_size":9071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37138432,"num_objects":9067,"num_object_clones":0,"num_object_copies":18134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9067,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5326,"num_write_kb":21304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4696},{"shards":"0,2","objects":4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.917634+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T15:59:15.917634+0000","last_peered":"2024-09-10T15:59:15.917634+0000","last_clean":"2024-09-10T15:59:15.917634+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T15:59:15.917634+0000","last_undegraded":"2024-09-10T15:59:15.917634+0000","last_fullsized":"2024-09-10T15:59:15.917634+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":70,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.778232+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T15:59:15.778232+0000","last_peered":"2024-09-10T15:59:15.778232+0000","last_clean":"2024-09-10T15:59:15.778232+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T15:59:15.778232+0000","last_undegraded":"2024-09-10T15:59:15.778232+0000","last_fullsized":"2024-09-10T15:59:15.778232+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'9056","reported_seq":9196,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:36.796105+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:01:36.796105+0000","last_peered":"2024-09-10T16:01:36.796105+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:01:36.796105+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":9056,"log_dups_size":0,"ondisk_log_size":9056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37045190,"num_objects":9046,"num_object_clones":0,"num_object_copies":18092,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9046,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5219,"num_write_kb":20876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4723},{"shards":"0,2","objects":4323}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":583320286,"num_objects":142421,"num_object_clones":0,"num_object_copies":284842,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":47117,"num_objects_misplaced":1025,"num_objects_unfound":0,"num_objects_dirty":142421,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":100267,"num_write_kb":401068,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3247,"num_bytes_recovered":13295616,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1177214976,"data_stored":1177063230,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":142493,"ondisk_log_size":142493,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986219,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":686540,"kb_used_data":420220,"kb_used_omap":4,"kb_used_meta":266299,"kb_avail":93685300,"statfs":{"total":96636764160,"available":95933747200,"internally_reserved":0,"allocated":430305280,"data_stored":430027520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":272690858},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,0,3,1,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659165,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":412348,"kb_used_data":232108,"kb_used_omap":4,"kb_used_meta":180219,"kb_avail":93959492,"statfs":{"total":96636764160,"available":96214519808,"internally_reserved":0,"allocated":237678592,"data_stored":237353399,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":184544942},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,3,1,3,1,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182448,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":804328,"kb_used_data":497816,"kb_used_omap":4,"kb_used_meta":306491,"kb_avail":93567512,"statfs":{"total":96636764160,"available":95813132288,"internally_reserved":0,"allocated":509763584,"data_stored":509481747,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":313847466},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,6,0,4,2,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":509480960,"data_stored":509444830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":237395968,"data_stored":237316482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":430338048,"data_stored":430301918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:01:38.987 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 186 20 143633 143613 3.01564 1.62891 0.030544 0.0259002 2024-09-10T16:01:39.987 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 187 20 144107 144087 3.00941 1.85156 0.122443 0.0259502 2024-09-10T16:01:40.987 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 188 20 144596 144576 3.00357 1.91016 0.0255859 0.0260034 2024-09-10T16:01:41.742 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:01:41.987 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 189 20 145075 145055 2.99757 1.87109 0.0332085 0.0260542 2024-09-10T16:01:42.312 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:01:42.313 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:01:42.332 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":169,"stamp":"2024-09-10T16:01:41.595705+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":586822385,"num_objects":143278,"num_object_clones":0,"num_object_copies":286556,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":47117,"num_objects_misplaced":893,"num_objects_unfound":0,"num_objects_dirty":143278,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":101124,"num_write_kb":404490,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3381,"num_bytes_recovered":13836307,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":143351,"ondisk_log_size":143351,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1922008,"kb_used_data":1160616,"kb_used_omap":12,"kb_used_meta":761331,"kb_avail":281193512,"statfs":{"total":289910292480,"available":287942156288,"internally_reserved":0,"allocated":1188470784,"data_stored":1187585994,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":779602946},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,11,3,9,6,8],"upper_bound":128},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":22282240,"num_objects":5440,"num_object_clones":0,"num_object_copies":10880,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-394,"num_objects_unfound":0,"num_objects_dirty":5440,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5440,"num_write_kb":21760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":394,"num_bytes_recovered":1613824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004717"},"pg_stats":[{"pgid":"2.f","version":"43'8851","reported_seq":8918,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:37.064389+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:01:37.064389+0000","last_peered":"2024-09-10T16:01:37.064389+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:01:37.064389+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":8851,"log_dups_size":0,"ondisk_log_size":8851,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36253696,"num_objects":8851,"num_object_clones":0,"num_object_copies":17702,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8851,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8851,"num_write_kb":35404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4592},{"shards":"0,2","objects":4258}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'8959","reported_seq":9067,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:01:39.661121+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:01:39.661121+0000","last_peered":"2024-09-10T16:01:39.661121+0000","last_clean":"2024-09-10T16:01:39.661121+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:01:39.661121+0000","last_undegraded":"2024-09-10T16:01:39.661121+0000","last_fullsized":"2024-09-10T16:01:39.661121+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":8959,"log_dups_size":0,"ondisk_log_size":8959,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36672454,"num_objects":8955,"num_object_clones":0,"num_object_copies":17910,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8955,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":7064,"num_write_kb":28256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'8967","reported_seq":15845,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T16:01:39.720675+0000","last_change":"2024-09-10T15:59:16.142612+0000","last_active":"2024-09-10T16:01:39.720675+0000","last_peered":"2024-09-10T16:01:39.720675+0000","last_clean":"2024-09-10T15:59:14.092473+0000","last_became_active":"2024-09-10T15:59:16.109611+0000","last_became_peered":"2024-09-10T15:59:16.109611+0000","last_unstale":"2024-09-10T16:01:39.720675+0000","last_undegraded":"2024-09-10T16:01:39.720675+0000","last_fullsized":"2024-09-10T15:59:15.770264+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":8967,"log_dups_size":0,"ondisk_log_size":8967,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36712448,"num_objects":8963,"num_object_clones":0,"num_object_copies":17926,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":893,"num_objects_unfound":0,"num_objects_dirty":8963,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6971,"num_write_kb":27884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3376,"num_bytes_recovered":13824000,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":893},{"shards":"2","objects":8070}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'8936","reported_seq":9010,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:01:39.694321+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:01:39.694321+0000","last_peered":"2024-09-10T16:01:39.694321+0000","last_clean":"2024-09-10T16:01:39.694321+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:01:39.694321+0000","last_undegraded":"2024-09-10T16:01:39.694321+0000","last_fullsized":"2024-09-10T16:01:39.694321+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":8936,"log_dups_size":0,"ondisk_log_size":8936,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36601856,"num_objects":8936,"num_object_clones":0,"num_object_copies":17872,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8936,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8936,"num_write_kb":35744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'8947","reported_seq":9072,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:01:37.052049+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:01:37.052049+0000","last_peered":"2024-09-10T16:01:37.052049+0000","last_clean":"2024-09-10T16:01:37.052049+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:01:37.052049+0000","last_undegraded":"2024-09-10T16:01:37.052049+0000","last_fullsized":"2024-09-10T16:01:37.052049+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":8947,"log_dups_size":0,"ondisk_log_size":8947,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36618723,"num_objects":8941,"num_object_clones":0,"num_object_copies":17882,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8941,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5658,"num_write_kb":22632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'8976","reported_seq":9109,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:37.010273+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:01:37.010273+0000","last_peered":"2024-09-10T16:01:37.010273+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:01:37.010273+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":8976,"log_dups_size":0,"ondisk_log_size":8976,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36745699,"num_objects":8972,"num_object_clones":0,"num_object_copies":17944,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8972,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5588,"num_write_kb":22352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4601},{"shards":"0,2","objects":4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'8999","reported_seq":9131,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:37.012689+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:01:37.012689+0000","last_peered":"2024-09-10T16:01:37.012689+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:01:37.012689+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":8999,"log_dups_size":0,"ondisk_log_size":8999,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36836294,"num_objects":8995,"num_object_clones":0,"num_object_copies":17990,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8995,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5498,"num_write_kb":21992,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4758},{"shards":"0,2","objects":4237}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'9111","reported_seq":9236,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:01:37.075014+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:01:37.075014+0000","last_peered":"2024-09-10T16:01:37.075014+0000","last_clean":"2024-09-10T16:01:37.075014+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:01:37.075014+0000","last_undegraded":"2024-09-10T16:01:37.075014+0000","last_fullsized":"2024-09-10T16:01:37.075014+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":9111,"log_dups_size":0,"ondisk_log_size":9111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37310464,"num_objects":9109,"num_object_clones":0,"num_object_copies":18218,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9109,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5958,"num_write_kb":23832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.777326+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T15:59:15.777326+0000","last_peered":"2024-09-10T15:59:15.777326+0000","last_clean":"2024-09-10T15:59:15.777326+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T15:59:15.777326+0000","last_undegraded":"2024-09-10T15:59:15.777326+0000","last_fullsized":"2024-09-10T15:59:15.777326+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'8740","reported_seq":8875,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:39.668569+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:01:39.668569+0000","last_peered":"2024-09-10T16:01:39.668569+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:01:39.668569+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":8740,"log_dups_size":0,"ondisk_log_size":8740,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35770851,"num_objects":8734,"num_object_clones":0,"num_object_copies":17468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8734,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6559,"num_write_kb":26236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4123},{"shards":"2","objects":4611}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":89,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.772619+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T15:59:15.772619+0000","last_peered":"2024-09-10T15:59:15.772619+0000","last_clean":"2024-09-10T15:59:15.772619+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T15:59:15.772619+0000","last_undegraded":"2024-09-10T15:59:15.772619+0000","last_fullsized":"2024-09-10T15:59:15.772619+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'8936","reported_seq":9062,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:37.026290+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:01:37.026290+0000","last_peered":"2024-09-10T16:01:37.026290+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:01:37.026290+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":8936,"log_dups_size":0,"ondisk_log_size":8936,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36569088,"num_objects":8928,"num_object_clones":0,"num_object_copies":17856,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8928,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5440,"num_write_kb":21760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4673},{"shards":"0,2","objects":4255}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":99,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.884297+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T15:59:15.884297+0000","last_peered":"2024-09-10T15:59:15.884297+0000","last_clean":"2024-09-10T15:59:15.884297+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T15:59:15.884297+0000","last_undegraded":"2024-09-10T15:59:15.884297+0000","last_fullsized":"2024-09-10T15:59:15.884297+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'8970","reported_seq":9100,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:39.681125+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:01:39.681125+0000","last_peered":"2024-09-10T16:01:39.681125+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:01:39.681125+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":8970,"log_dups_size":0,"ondisk_log_size":8970,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36721123,"num_objects":8966,"num_object_clones":0,"num_object_copies":17932,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8966,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6650,"num_write_kb":26600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4304},{"shards":"2","objects":4662}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.848670+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T15:59:15.848670+0000","last_peered":"2024-09-10T15:59:15.848670+0000","last_clean":"2024-09-10T15:59:15.848670+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T15:59:15.848670+0000","last_undegraded":"2024-09-10T15:59:15.848670+0000","last_fullsized":"2024-09-10T15:59:15.848670+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'9132","reported_seq":9256,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:39.681438+0000","last_change":"2024-09-10T15:59:16.174113+0000","last_active":"2024-09-10T16:01:39.681438+0000","last_peered":"2024-09-10T16:01:39.681438+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:01:39.681438+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":9132,"log_dups_size":0,"ondisk_log_size":9132,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37380096,"num_objects":9126,"num_object_clones":0,"num_object_copies":18252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4354,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9126,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6686,"num_write_kb":26744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4352},{"shards":"2","objects":4774}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'8910","reported_seq":9040,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:36.927199+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:01:36.927199+0000","last_peered":"2024-09-10T16:01:36.927199+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:01:36.927199+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":8910,"log_dups_size":0,"ondisk_log_size":8910,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36487168,"num_objects":8908,"num_object_clones":0,"num_object_copies":17816,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8908,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5541,"num_write_kb":22164,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4604},{"shards":"0,2","objects":4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.992406+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T15:59:15.992406+0000","last_peered":"2024-09-10T15:59:15.992406+0000","last_clean":"2024-09-10T15:59:15.992406+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T15:59:15.992406+0000","last_undegraded":"2024-09-10T15:59:15.992406+0000","last_fullsized":"2024-09-10T15:59:15.992406+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'8787","reported_seq":8925,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:37.017762+0000","last_change":"2024-09-10T15:59:16.251610+0000","last_active":"2024-09-10T16:01:37.017762+0000","last_peered":"2024-09-10T16:01:37.017762+0000","last_clean":"2024-09-10T15:59:14.115595+0000","last_became_active":"2024-09-10T15:59:16.244089+0000","last_became_peered":"2024-09-10T15:59:16.244089+0000","last_unstale":"2024-09-10T16:01:37.017762+0000","last_undegraded":"2024-09-10T15:59:15.847127+0000","last_fullsized":"2024-09-10T15:59:15.774437+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":8787,"log_dups_size":0,"ondisk_log_size":8787,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35958784,"num_objects":8779,"num_object_clones":0,"num_object_copies":17558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8779,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5177,"num_write_kb":20708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4576},{"shards":"0,2","objects":4203}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":89,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.773633+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T15:59:15.773633+0000","last_peered":"2024-09-10T15:59:15.773633+0000","last_clean":"2024-09-10T15:59:15.773633+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T15:59:15.773633+0000","last_undegraded":"2024-09-10T15:59:15.773633+0000","last_fullsized":"2024-09-10T15:59:15.773633+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'9071","reported_seq":9208,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:37.038648+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:01:37.038648+0000","last_peered":"2024-09-10T16:01:37.038648+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:01:37.038648+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":9071,"log_dups_size":0,"ondisk_log_size":9071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37138432,"num_objects":9067,"num_object_clones":0,"num_object_copies":18134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9067,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5326,"num_write_kb":21304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4696},{"shards":"0,2","objects":4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.917634+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T15:59:15.917634+0000","last_peered":"2024-09-10T15:59:15.917634+0000","last_clean":"2024-09-10T15:59:15.917634+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T15:59:15.917634+0000","last_undegraded":"2024-09-10T15:59:15.917634+0000","last_fullsized":"2024-09-10T15:59:15.917634+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":70,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.778232+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T15:59:15.778232+0000","last_peered":"2024-09-10T15:59:15.778232+0000","last_clean":"2024-09-10T15:59:15.778232+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T15:59:15.778232+0000","last_undegraded":"2024-09-10T15:59:15.778232+0000","last_fullsized":"2024-09-10T15:59:15.778232+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'9056","reported_seq":9196,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:36.796105+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:01:36.796105+0000","last_peered":"2024-09-10T16:01:36.796105+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:01:36.796105+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":9056,"log_dups_size":0,"ondisk_log_size":9056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37045190,"num_objects":9046,"num_object_clones":0,"num_object_copies":18092,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9046,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5219,"num_write_kb":20876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4723},{"shards":"0,2","objects":4323}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":586822366,"num_objects":143276,"num_object_clones":0,"num_object_copies":286552,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":47117,"num_objects_misplaced":893,"num_objects_unfound":0,"num_objects_dirty":143276,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":101122,"num_write_kb":404488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3379,"num_bytes_recovered":13836288,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1189470208,"data_stored":1189318462,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":143348,"ondisk_log_size":143348,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986220,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":695192,"kb_used_data":424712,"kb_used_omap":4,"kb_used_meta":270459,"kb_avail":93676648,"statfs":{"total":96636764160,"available":95924887552,"internally_reserved":0,"allocated":434905088,"data_stored":434627328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":276950698},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,2,3,3,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659166,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":422488,"kb_used_data":238088,"kb_used_omap":4,"kb_used_meta":184379,"kb_avail":93949352,"statfs":{"total":96636764160,"available":96204136448,"internally_reserved":0,"allocated":243802112,"data_stored":243476919,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":188804782},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,2,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182448,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":804328,"kb_used_data":497816,"kb_used_omap":4,"kb_used_meta":306491,"kb_avail":93567512,"statfs":{"total":96636764160,"available":95813132288,"internally_reserved":0,"allocated":509763584,"data_stored":509481747,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":313847466},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,6,0,4,2,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":509480960,"data_stored":509444830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":245002240,"data_stored":244922754,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":434987008,"data_stored":434950878,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:01:42.333 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:01:42.862 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:01:42.862 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:01:42.885 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":169,"stamp":"2024-09-10T16:01:41.595705+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":586822385,"num_objects":143278,"num_object_clones":0,"num_object_copies":286556,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":47117,"num_objects_misplaced":893,"num_objects_unfound":0,"num_objects_dirty":143278,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":101124,"num_write_kb":404490,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3381,"num_bytes_recovered":13836307,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":143351,"ondisk_log_size":143351,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1922008,"kb_used_data":1160616,"kb_used_omap":12,"kb_used_meta":761331,"kb_avail":281193512,"statfs":{"total":289910292480,"available":287942156288,"internally_reserved":0,"allocated":1188470784,"data_stored":1187585994,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":779602946},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,11,3,9,6,8],"upper_bound":128},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":22282240,"num_objects":5440,"num_object_clones":0,"num_object_copies":10880,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-394,"num_objects_unfound":0,"num_objects_dirty":5440,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5440,"num_write_kb":21760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":394,"num_bytes_recovered":1613824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004717"},"pg_stats":[{"pgid":"2.f","version":"43'8851","reported_seq":8918,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:37.064389+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:01:37.064389+0000","last_peered":"2024-09-10T16:01:37.064389+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:01:37.064389+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":8851,"log_dups_size":0,"ondisk_log_size":8851,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36253696,"num_objects":8851,"num_object_clones":0,"num_object_copies":17702,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8851,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8851,"num_write_kb":35404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4592},{"shards":"0,2","objects":4258}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'8959","reported_seq":9067,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:01:39.661121+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:01:39.661121+0000","last_peered":"2024-09-10T16:01:39.661121+0000","last_clean":"2024-09-10T16:01:39.661121+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:01:39.661121+0000","last_undegraded":"2024-09-10T16:01:39.661121+0000","last_fullsized":"2024-09-10T16:01:39.661121+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":8959,"log_dups_size":0,"ondisk_log_size":8959,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36672454,"num_objects":8955,"num_object_clones":0,"num_object_copies":17910,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8955,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":7064,"num_write_kb":28256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'8967","reported_seq":15845,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T16:01:39.720675+0000","last_change":"2024-09-10T15:59:16.142612+0000","last_active":"2024-09-10T16:01:39.720675+0000","last_peered":"2024-09-10T16:01:39.720675+0000","last_clean":"2024-09-10T15:59:14.092473+0000","last_became_active":"2024-09-10T15:59:16.109611+0000","last_became_peered":"2024-09-10T15:59:16.109611+0000","last_unstale":"2024-09-10T16:01:39.720675+0000","last_undegraded":"2024-09-10T16:01:39.720675+0000","last_fullsized":"2024-09-10T15:59:15.770264+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":8967,"log_dups_size":0,"ondisk_log_size":8967,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36712448,"num_objects":8963,"num_object_clones":0,"num_object_copies":17926,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":893,"num_objects_unfound":0,"num_objects_dirty":8963,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6971,"num_write_kb":27884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3376,"num_bytes_recovered":13824000,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":893},{"shards":"2","objects":8070}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'8936","reported_seq":9010,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:01:39.694321+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:01:39.694321+0000","last_peered":"2024-09-10T16:01:39.694321+0000","last_clean":"2024-09-10T16:01:39.694321+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:01:39.694321+0000","last_undegraded":"2024-09-10T16:01:39.694321+0000","last_fullsized":"2024-09-10T16:01:39.694321+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":8936,"log_dups_size":0,"ondisk_log_size":8936,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36601856,"num_objects":8936,"num_object_clones":0,"num_object_copies":17872,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8936,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8936,"num_write_kb":35744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'8947","reported_seq":9072,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:01:37.052049+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:01:37.052049+0000","last_peered":"2024-09-10T16:01:37.052049+0000","last_clean":"2024-09-10T16:01:37.052049+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:01:37.052049+0000","last_undegraded":"2024-09-10T16:01:37.052049+0000","last_fullsized":"2024-09-10T16:01:37.052049+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":8947,"log_dups_size":0,"ondisk_log_size":8947,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36618723,"num_objects":8941,"num_object_clones":0,"num_object_copies":17882,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8941,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5658,"num_write_kb":22632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'8976","reported_seq":9109,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:37.010273+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:01:37.010273+0000","last_peered":"2024-09-10T16:01:37.010273+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:01:37.010273+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":8976,"log_dups_size":0,"ondisk_log_size":8976,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36745699,"num_objects":8972,"num_object_clones":0,"num_object_copies":17944,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8972,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5588,"num_write_kb":22352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4601},{"shards":"0,2","objects":4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'8999","reported_seq":9131,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:37.012689+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:01:37.012689+0000","last_peered":"2024-09-10T16:01:37.012689+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:01:37.012689+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":8999,"log_dups_size":0,"ondisk_log_size":8999,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36836294,"num_objects":8995,"num_object_clones":0,"num_object_copies":17990,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8995,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5498,"num_write_kb":21992,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4758},{"shards":"0,2","objects":4237}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'9111","reported_seq":9236,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:01:37.075014+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:01:37.075014+0000","last_peered":"2024-09-10T16:01:37.075014+0000","last_clean":"2024-09-10T16:01:37.075014+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:01:37.075014+0000","last_undegraded":"2024-09-10T16:01:37.075014+0000","last_fullsized":"2024-09-10T16:01:37.075014+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":9111,"log_dups_size":0,"ondisk_log_size":9111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37310464,"num_objects":9109,"num_object_clones":0,"num_object_copies":18218,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9109,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5958,"num_write_kb":23832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.777326+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T15:59:15.777326+0000","last_peered":"2024-09-10T15:59:15.777326+0000","last_clean":"2024-09-10T15:59:15.777326+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T15:59:15.777326+0000","last_undegraded":"2024-09-10T15:59:15.777326+0000","last_fullsized":"2024-09-10T15:59:15.777326+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'8740","reported_seq":8875,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:39.668569+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:01:39.668569+0000","last_peered":"2024-09-10T16:01:39.668569+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:01:39.668569+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":8740,"log_dups_size":0,"ondisk_log_size":8740,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35770851,"num_objects":8734,"num_object_clones":0,"num_object_copies":17468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8734,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6559,"num_write_kb":26236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4123},{"shards":"2","objects":4611}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":89,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.772619+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T15:59:15.772619+0000","last_peered":"2024-09-10T15:59:15.772619+0000","last_clean":"2024-09-10T15:59:15.772619+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T15:59:15.772619+0000","last_undegraded":"2024-09-10T15:59:15.772619+0000","last_fullsized":"2024-09-10T15:59:15.772619+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'8936","reported_seq":9062,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:37.026290+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:01:37.026290+0000","last_peered":"2024-09-10T16:01:37.026290+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:01:37.026290+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":8936,"log_dups_size":0,"ondisk_log_size":8936,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36569088,"num_objects":8928,"num_object_clones":0,"num_object_copies":17856,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8928,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5440,"num_write_kb":21760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4673},{"shards":"0,2","objects":4255}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":99,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.884297+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T15:59:15.884297+0000","last_peered":"2024-09-10T15:59:15.884297+0000","last_clean":"2024-09-10T15:59:15.884297+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T15:59:15.884297+0000","last_undegraded":"2024-09-10T15:59:15.884297+0000","last_fullsized":"2024-09-10T15:59:15.884297+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'8970","reported_seq":9100,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:39.681125+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:01:39.681125+0000","last_peered":"2024-09-10T16:01:39.681125+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:01:39.681125+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":8970,"log_dups_size":0,"ondisk_log_size":8970,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36721123,"num_objects":8966,"num_object_clones":0,"num_object_copies":17932,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8966,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6650,"num_write_kb":26600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4304},{"shards":"2","objects":4662}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.848670+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T15:59:15.848670+0000","last_peered":"2024-09-10T15:59:15.848670+0000","last_clean":"2024-09-10T15:59:15.848670+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T15:59:15.848670+0000","last_undegraded":"2024-09-10T15:59:15.848670+0000","last_fullsized":"2024-09-10T15:59:15.848670+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'9132","reported_seq":9256,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:39.681438+0000","last_change":"2024-09-10T15:59:16.174113+0000","last_active":"2024-09-10T16:01:39.681438+0000","last_peered":"2024-09-10T16:01:39.681438+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:01:39.681438+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":9132,"log_dups_size":0,"ondisk_log_size":9132,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37380096,"num_objects":9126,"num_object_clones":0,"num_object_copies":18252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4354,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9126,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6686,"num_write_kb":26744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4352},{"shards":"2","objects":4774}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'8910","reported_seq":9040,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:36.927199+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:01:36.927199+0000","last_peered":"2024-09-10T16:01:36.927199+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:01:36.927199+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":8910,"log_dups_size":0,"ondisk_log_size":8910,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36487168,"num_objects":8908,"num_object_clones":0,"num_object_copies":17816,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8908,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5541,"num_write_kb":22164,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4604},{"shards":"0,2","objects":4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.992406+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T15:59:15.992406+0000","last_peered":"2024-09-10T15:59:15.992406+0000","last_clean":"2024-09-10T15:59:15.992406+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T15:59:15.992406+0000","last_undegraded":"2024-09-10T15:59:15.992406+0000","last_fullsized":"2024-09-10T15:59:15.992406+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'8787","reported_seq":8925,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:37.017762+0000","last_change":"2024-09-10T15:59:16.251610+0000","last_active":"2024-09-10T16:01:37.017762+0000","last_peered":"2024-09-10T16:01:37.017762+0000","last_clean":"2024-09-10T15:59:14.115595+0000","last_became_active":"2024-09-10T15:59:16.244089+0000","last_became_peered":"2024-09-10T15:59:16.244089+0000","last_unstale":"2024-09-10T16:01:37.017762+0000","last_undegraded":"2024-09-10T15:59:15.847127+0000","last_fullsized":"2024-09-10T15:59:15.774437+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":8787,"log_dups_size":0,"ondisk_log_size":8787,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35958784,"num_objects":8779,"num_object_clones":0,"num_object_copies":17558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8779,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5177,"num_write_kb":20708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4576},{"shards":"0,2","objects":4203}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":89,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.773633+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T15:59:15.773633+0000","last_peered":"2024-09-10T15:59:15.773633+0000","last_clean":"2024-09-10T15:59:15.773633+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T15:59:15.773633+0000","last_undegraded":"2024-09-10T15:59:15.773633+0000","last_fullsized":"2024-09-10T15:59:15.773633+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'9071","reported_seq":9208,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:37.038648+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:01:37.038648+0000","last_peered":"2024-09-10T16:01:37.038648+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:01:37.038648+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":9071,"log_dups_size":0,"ondisk_log_size":9071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37138432,"num_objects":9067,"num_object_clones":0,"num_object_copies":18134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9067,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5326,"num_write_kb":21304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4696},{"shards":"0,2","objects":4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.917634+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T15:59:15.917634+0000","last_peered":"2024-09-10T15:59:15.917634+0000","last_clean":"2024-09-10T15:59:15.917634+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T15:59:15.917634+0000","last_undegraded":"2024-09-10T15:59:15.917634+0000","last_fullsized":"2024-09-10T15:59:15.917634+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":70,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.778232+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T15:59:15.778232+0000","last_peered":"2024-09-10T15:59:15.778232+0000","last_clean":"2024-09-10T15:59:15.778232+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T15:59:15.778232+0000","last_undegraded":"2024-09-10T15:59:15.778232+0000","last_fullsized":"2024-09-10T15:59:15.778232+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'9056","reported_seq":9196,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:36.796105+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:01:36.796105+0000","last_peered":"2024-09-10T16:01:36.796105+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:01:36.796105+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":9056,"log_dups_size":0,"ondisk_log_size":9056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37045190,"num_objects":9046,"num_object_clones":0,"num_object_copies":18092,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9046,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5219,"num_write_kb":20876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4723},{"shards":"0,2","objects":4323}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":586822366,"num_objects":143276,"num_object_clones":0,"num_object_copies":286552,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":47117,"num_objects_misplaced":893,"num_objects_unfound":0,"num_objects_dirty":143276,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":101122,"num_write_kb":404488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3379,"num_bytes_recovered":13836288,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1189470208,"data_stored":1189318462,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":143348,"ondisk_log_size":143348,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986220,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":695192,"kb_used_data":424712,"kb_used_omap":4,"kb_used_meta":270459,"kb_avail":93676648,"statfs":{"total":96636764160,"available":95924887552,"internally_reserved":0,"allocated":434905088,"data_stored":434627328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":276950698},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,2,3,3,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659166,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":422488,"kb_used_data":238088,"kb_used_omap":4,"kb_used_meta":184379,"kb_avail":93949352,"statfs":{"total":96636764160,"available":96204136448,"internally_reserved":0,"allocated":243802112,"data_stored":243476919,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":188804782},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,2,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182448,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":804328,"kb_used_data":497816,"kb_used_omap":4,"kb_used_meta":306491,"kb_avail":93567512,"statfs":{"total":96636764160,"available":95813132288,"internally_reserved":0,"allocated":509763584,"data_stored":509481747,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":313847466},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,6,0,4,2,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":509480960,"data_stored":509444830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":245002240,"data_stored":244922754,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":434987008,"data_stored":434950878,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:01:42.886 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-10T16:01:42.887 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "43'8851", 'reported_seq': 8918, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:01:37.064389+0000', 'last_change': '2024-09-10T15:59:16.281558+0000', 'last_active': '2024-09-10T16:01:37.064389+0000', 'last_peered': '2024-09-10T16:01:37.064389+0000', 'last_clean': '2024-09-10T15:59:14.132456+0000', 'last_became_active': '2024-09-10T15:59:16.274407+0000', 'last_became_peered': '2024-09-10T15:59:16.274407+0000', 'last_unstale': '2024-09-10T16:01:37.064389+0000', 'last_undegraded': '2024-09-10T15:59:15.851279+0000', 'last_fullsized': '2024-09-10T15:59:15.774356+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'objects_scrubbed': 0, 'log_size': 8851, 'log_dups_size': 0, 'ondisk_log_size': 8851, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36253696, 'num_objects': 8851, 'num_object_clones': 0, 'num_object_copies': 17702, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4259, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8851, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8851, 'num_write_kb': 35404, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4592}, {'shards': '0,2', 'objects': 4258}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:01:42.887 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-10T16:01:42.887 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "43'8967", 'reported_seq': 15845, 'reported_epoch': 43, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-10T16:01:39.720675+0000', 'last_change': '2024-09-10T15:59:16.142612+0000', 'last_active': '2024-09-10T16:01:39.720675+0000', 'last_peered': '2024-09-10T16:01:39.720675+0000', 'last_clean': '2024-09-10T15:59:14.092473+0000', 'last_became_active': '2024-09-10T15:59:16.109611+0000', 'last_became_peered': '2024-09-10T15:59:16.109611+0000', 'last_unstale': '2024-09-10T16:01:39.720675+0000', 'last_undegraded': '2024-09-10T16:01:39.720675+0000', 'last_fullsized': '2024-09-10T15:59:15.770264+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2029", 'last_scrub_stamp': '2024-09-10T15:58:45.933789+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:45.933789+0000', 'objects_scrubbed': 1992, 'log_size': 8967, 'log_dups_size': 0, 'ondisk_log_size': 8967, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 668, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36712448, 'num_objects': 8963, 'num_object_clones': 0, 'num_object_copies': 17926, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 893, 'num_objects_unfound': 0, 'num_objects_dirty': 8963, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6971, 'num_write_kb': 27884, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3376, 'num_bytes_recovered': 13824000, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 893}, {'shards': '2', 'objects': 8070}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:01:42.887 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-10T16:01:42.887 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "43'8976", 'reported_seq': 9109, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:01:37.010273+0000', 'last_change': '2024-09-10T15:59:16.308264+0000', 'last_active': '2024-09-10T16:01:37.010273+0000', 'last_peered': '2024-09-10T16:01:37.010273+0000', 'last_clean': '2024-09-10T15:59:14.111551+0000', 'last_became_active': '2024-09-10T15:59:16.300040+0000', 'last_became_peered': '2024-09-10T15:59:16.300040+0000', 'last_unstale': '2024-09-10T16:01:37.010273+0000', 'last_undegraded': '2024-09-10T15:59:15.878234+0000', 'last_fullsized': '2024-09-10T15:59:15.769328+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3442", 'last_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'objects_scrubbed': 3384, 'log_size': 8976, 'log_dups_size': 0, 'ondisk_log_size': 8976, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1292, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36745699, 'num_objects': 8972, 'num_object_clones': 0, 'num_object_copies': 17944, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4372, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8972, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 5588, 'num_write_kb': 22352, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4601}, {'shards': '0,2', 'objects': 4371}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:01:42.887 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-10T16:01:42.887 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "43'8999", 'reported_seq': 9131, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:01:37.012689+0000', 'last_change': '2024-09-10T15:59:16.331899+0000', 'last_active': '2024-09-10T16:01:37.012689+0000', 'last_peered': '2024-09-10T16:01:37.012689+0000', 'last_clean': '2024-09-10T15:59:14.139436+0000', 'last_became_active': '2024-09-10T15:59:16.318715+0000', 'last_became_peered': '2024-09-10T15:59:16.318715+0000', 'last_unstale': '2024-09-10T16:01:37.012689+0000', 'last_undegraded': '2024-09-10T15:59:15.877524+0000', 'last_fullsized': '2024-09-10T15:59:15.769042+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3537", 'last_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'objects_scrubbed': 3497, 'log_size': 8999, 'log_dups_size': 0, 'ondisk_log_size': 8999, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1224, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36836294, 'num_objects': 8995, 'num_object_clones': 0, 'num_object_copies': 17990, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4237, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8995, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 5498, 'num_write_kb': 21992, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4758}, {'shards': '0,2', 'objects': 4237}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:01:42.887 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-10T16:01:42.887 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "43'8740", 'reported_seq': 8875, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:01:39.668569+0000', 'last_change': '2024-09-10T15:59:16.175680+0000', 'last_active': '2024-09-10T16:01:39.668569+0000', 'last_peered': '2024-09-10T16:01:39.668569+0000', 'last_clean': '2024-09-10T15:59:14.088187+0000', 'last_became_active': '2024-09-10T15:59:16.165810+0000', 'last_became_peered': '2024-09-10T15:59:16.165810+0000', 'last_unstale': '2024-09-10T16:01:39.668569+0000', 'last_undegraded': '2024-09-10T15:59:15.822098+0000', 'last_fullsized': '2024-09-10T15:59:15.769123+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2218", 'last_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'objects_scrubbed': 2175, 'log_size': 8740, 'log_dups_size': 0, 'ondisk_log_size': 8740, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 825, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35770851, 'num_objects': 8734, 'num_object_clones': 0, 'num_object_copies': 17468, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4124, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8734, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 6559, 'num_write_kb': 26236, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4123}, {'shards': '2', 'objects': 4611}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:01:42.887 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-10T16:01:42.888 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "43'8936", 'reported_seq': 9062, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:01:37.026290+0000', 'last_change': '2024-09-10T15:59:16.277742+0000', 'last_active': '2024-09-10T16:01:37.026290+0000', 'last_peered': '2024-09-10T16:01:37.026290+0000', 'last_clean': '2024-09-10T15:59:14.129081+0000', 'last_became_active': '2024-09-10T15:59:16.269835+0000', 'last_became_peered': '2024-09-10T15:59:16.269835+0000', 'last_unstale': '2024-09-10T16:01:37.026290+0000', 'last_undegraded': '2024-09-10T15:59:15.860857+0000', 'last_fullsized': '2024-09-10T15:59:15.771410+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3557", 'last_scrub_stamp': '2024-09-10T15:59:02.493678+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:02.493678+0000', 'objects_scrubbed': 3488, 'log_size': 8936, 'log_dups_size': 0, 'ondisk_log_size': 8936, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1472, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36569088, 'num_objects': 8928, 'num_object_clones': 0, 'num_object_copies': 17856, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4257, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8928, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5440, 'num_write_kb': 21760, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4673}, {'shards': '0,2', 'objects': 4255}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:01:42.888 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-10T16:01:42.888 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "43'8970", 'reported_seq': 9100, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:01:39.681125+0000', 'last_change': '2024-09-10T15:59:16.284311+0000', 'last_active': '2024-09-10T16:01:39.681125+0000', 'last_peered': '2024-09-10T16:01:39.681125+0000', 'last_clean': '2024-09-10T15:59:14.116041+0000', 'last_became_active': '2024-09-10T15:59:16.274543+0000', 'last_became_peered': '2024-09-10T15:59:16.274543+0000', 'last_unstale': '2024-09-10T16:01:39.681125+0000', 'last_undegraded': '2024-09-10T15:59:15.812484+0000', 'last_fullsized': '2024-09-10T15:59:15.770885+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2356", 'last_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'objects_scrubbed': 2316, 'log_size': 8970, 'log_dups_size': 0, 'ondisk_log_size': 8970, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 783, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36721123, 'num_objects': 8966, 'num_object_clones': 0, 'num_object_copies': 17932, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8966, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 6650, 'num_write_kb': 26600, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4304}, {'shards': '2', 'objects': 4662}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:01:42.888 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-10T16:01:42.888 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "43'9132", 'reported_seq': 9256, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:01:39.681438+0000', 'last_change': '2024-09-10T15:59:16.174113+0000', 'last_active': '2024-09-10T16:01:39.681438+0000', 'last_peered': '2024-09-10T16:01:39.681438+0000', 'last_clean': '2024-09-10T15:59:14.110654+0000', 'last_became_active': '2024-09-10T15:59:16.164645+0000', 'last_became_peered': '2024-09-10T15:59:16.164645+0000', 'last_unstale': '2024-09-10T16:01:39.681438+0000', 'last_undegraded': '2024-09-10T15:59:15.803593+0000', 'last_fullsized': '2024-09-10T15:59:15.769083+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2486", 'last_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'objects_scrubbed': 2440, 'log_size': 9132, 'log_dups_size': 0, 'ondisk_log_size': 9132, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 792, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37380096, 'num_objects': 9126, 'num_object_clones': 0, 'num_object_copies': 18252, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4354, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9126, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6686, 'num_write_kb': 26744, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4352}, {'shards': '2', 'objects': 4774}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:01:42.888 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-10T16:01:42.888 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "43'8910", 'reported_seq': 9040, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:01:36.927199+0000', 'last_change': '2024-09-10T15:59:16.315641+0000', 'last_active': '2024-09-10T16:01:36.927199+0000', 'last_peered': '2024-09-10T16:01:36.927199+0000', 'last_clean': '2024-09-10T15:59:14.117001+0000', 'last_became_active': '2024-09-10T15:59:16.302245+0000', 'last_became_peered': '2024-09-10T15:59:16.302245+0000', 'last_unstale': '2024-09-10T16:01:36.927199+0000', 'last_undegraded': '2024-09-10T15:59:15.845012+0000', 'last_fullsized': '2024-09-10T15:59:15.770337+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3404", 'last_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'objects_scrubbed': 3367, 'log_size': 8910, 'log_dups_size': 0, 'ondisk_log_size': 8910, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 983, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36487168, 'num_objects': 8908, 'num_object_clones': 0, 'num_object_copies': 17816, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8908, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5541, 'num_write_kb': 22164, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4604}, {'shards': '0,2', 'objects': 4304}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:01:42.888 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-10T16:01:42.888 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "43'8787", 'reported_seq': 8925, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:01:37.017762+0000', 'last_change': '2024-09-10T15:59:16.251610+0000', 'last_active': '2024-09-10T16:01:37.017762+0000', 'last_peered': '2024-09-10T16:01:37.017762+0000', 'last_clean': '2024-09-10T15:59:14.115595+0000', 'last_became_active': '2024-09-10T15:59:16.244089+0000', 'last_became_peered': '2024-09-10T15:59:16.244089+0000', 'last_unstale': '2024-09-10T16:01:37.017762+0000', 'last_undegraded': '2024-09-10T15:59:15.847127+0000', 'last_fullsized': '2024-09-10T15:59:15.774437+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3653", 'last_scrub_stamp': '2024-09-10T15:59:05.298468+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:05.298468+0000', 'objects_scrubbed': 3602, 'log_size': 8787, 'log_dups_size': 0, 'ondisk_log_size': 8787, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1289, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35958784, 'num_objects': 8779, 'num_object_clones': 0, 'num_object_copies': 17558, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4206, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8779, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5177, 'num_write_kb': 20708, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4576}, {'shards': '0,2', 'objects': 4203}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:01:42.888 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-10T16:01:42.888 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "43'9071", 'reported_seq': 9208, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:01:37.038648+0000', 'last_change': '2024-09-10T15:59:16.311291+0000', 'last_active': '2024-09-10T16:01:37.038648+0000', 'last_peered': '2024-09-10T16:01:37.038648+0000', 'last_clean': '2024-09-10T15:59:14.116363+0000', 'last_became_active': '2024-09-10T15:59:16.301244+0000', 'last_became_peered': '2024-09-10T15:59:16.301244+0000', 'last_unstale': '2024-09-10T16:01:37.038648+0000', 'last_undegraded': '2024-09-10T15:59:15.883434+0000', 'last_fullsized': '2024-09-10T15:59:15.772756+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3789", 'last_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'objects_scrubbed': 3741, 'log_size': 9071, 'log_dups_size': 0, 'ondisk_log_size': 9071, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1440, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37138432, 'num_objects': 9067, 'num_object_clones': 0, 'num_object_copies': 18134, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4373, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9067, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5326, 'num_write_kb': 21304, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4696}, {'shards': '0,2', 'objects': 4371}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:01:42.889 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-10T16:01:42.889 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "43'9056", 'reported_seq': 9196, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:01:36.796105+0000', 'last_change': '2024-09-10T15:59:16.313409+0000', 'last_active': '2024-09-10T16:01:36.796105+0000', 'last_peered': '2024-09-10T16:01:36.796105+0000', 'last_clean': '2024-09-10T15:59:14.129780+0000', 'last_became_active': '2024-09-10T15:59:16.300932+0000', 'last_became_peered': '2024-09-10T15:59:16.300932+0000', 'last_unstale': '2024-09-10T16:01:36.796105+0000', 'last_undegraded': '2024-09-10T15:59:15.887823+0000', 'last_fullsized': '2024-09-10T15:59:15.768513+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3880", 'last_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'objects_scrubbed': 3827, 'log_size': 9056, 'log_dups_size': 0, 'ondisk_log_size': 9056, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1397, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37045190, 'num_objects': 9046, 'num_object_clones': 0, 'num_object_copies': 18092, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4325, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9046, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5219, 'num_write_kb': 20876, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4723}, {'shards': '0,2', 'objects': 4323}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:01:42.892 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T16:01:42.987 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 190 20 145496 145476 2.99045 1.64453 0.043894 0.026117 2024-09-10T16:01:43.612 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:01:43.613 INFO:teuthology.orchestra.run.smithi115.stdout:{"fsid":"7effee29-df05-4351-9741-ee203564ed81","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 47117/286556 objects degraded (16.443%), 11 pgs degraded, 12 pgs undersized","count":23},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":267,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":43,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983954,"num_in_osds":3,"osd_in_since":1725983947,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":143278,"data_bytes":586822385,"bytes_used":1968136192,"bytes_avail":287942156288,"bytes_total":289910292480,"degraded_objects":47117,"degraded_total":286556,"degraded_ratio":0.16442510364466278,"misplaced_objects":893,"misplaced_total":286556,"misplaced_ratio":0.0031163193232736359,"recovering_objects_per_sec":32,"recovering_bytes_per_sec":134432,"recovering_keys_per_sec":0,"num_objects_recovered":394,"num_bytes_recovered":1613824,"num_keys_recovered":0,"write_bytes_sec":1856123,"read_op_per_sec":0,"write_op_per_sec":453},"fsmap":{"epoch":1,"btime":"2024-09-10T15:57:14:967120+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T16:00:13.560776+0000","services":{}},"progress_events":{}} 2024-09-10T16:01:43.643 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-10T16:01:43.643 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:01:43.987 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 191 20 145926 145906 2.98359 1.67969 0.0481066 0.0261778 2024-09-10T16:01:44.232 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:01:44.233 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:01:44.257 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":170,"stamp":"2024-09-10T16:01:43.596357+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592745201,"num_objects":144724,"num_object_clones":0,"num_object_copies":289448,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":47117,"num_objects_misplaced":893,"num_objects_unfound":0,"num_objects_dirty":144724,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":102570,"num_write_kb":410274,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3381,"num_bytes_recovered":13836307,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":144797,"ondisk_log_size":144797,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1933148,"kb_used_data":1167596,"kb_used_omap":12,"kb_used_meta":765491,"kb_avail":281182372,"statfs":{"total":289910292480,"available":287930748928,"internally_reserved":0,"allocated":1195618304,"data_stored":1194733514,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":783862786},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,5,3,7,12,4,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":24748032,"num_objects":6042,"num_object_clones":0,"num_object_copies":12084,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-264,"num_objects_unfound":0,"num_objects_dirty":6042,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6042,"num_write_kb":24168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":264,"num_bytes_recovered":1081344,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004715"},"pg_stats":[{"pgid":"2.f","version":"43'8995","reported_seq":9065,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:42.055278+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:01:42.055278+0000","last_peered":"2024-09-10T16:01:42.055278+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:01:42.055278+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":8995,"log_dups_size":0,"ondisk_log_size":8995,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36843520,"num_objects":8995,"num_object_clones":0,"num_object_copies":17990,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8995,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8995,"num_write_kb":35980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4736},{"shards":"0,2","objects":4258}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'8959","reported_seq":9067,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:01:39.661121+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:01:39.661121+0000","last_peered":"2024-09-10T16:01:39.661121+0000","last_clean":"2024-09-10T16:01:39.661121+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:01:39.661121+0000","last_undegraded":"2024-09-10T16:01:39.661121+0000","last_fullsized":"2024-09-10T16:01:39.661121+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":8959,"log_dups_size":0,"ondisk_log_size":8959,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36672454,"num_objects":8955,"num_object_clones":0,"num_object_copies":17910,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8955,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":7064,"num_write_kb":28256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'8967","reported_seq":15845,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T16:01:39.720675+0000","last_change":"2024-09-10T15:59:16.142612+0000","last_active":"2024-09-10T16:01:39.720675+0000","last_peered":"2024-09-10T16:01:39.720675+0000","last_clean":"2024-09-10T15:59:14.092473+0000","last_became_active":"2024-09-10T15:59:16.109611+0000","last_became_peered":"2024-09-10T15:59:16.109611+0000","last_unstale":"2024-09-10T16:01:39.720675+0000","last_undegraded":"2024-09-10T16:01:39.720675+0000","last_fullsized":"2024-09-10T15:59:15.770264+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":8967,"log_dups_size":0,"ondisk_log_size":8967,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36712448,"num_objects":8963,"num_object_clones":0,"num_object_copies":17926,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":893,"num_objects_unfound":0,"num_objects_dirty":8963,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6971,"num_write_kb":27884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3376,"num_bytes_recovered":13824000,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":893},{"shards":"2","objects":8070}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'8936","reported_seq":9010,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:01:39.694321+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:01:39.694321+0000","last_peered":"2024-09-10T16:01:39.694321+0000","last_clean":"2024-09-10T16:01:39.694321+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:01:39.694321+0000","last_undegraded":"2024-09-10T16:01:39.694321+0000","last_fullsized":"2024-09-10T16:01:39.694321+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":8936,"log_dups_size":0,"ondisk_log_size":8936,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36601856,"num_objects":8936,"num_object_clones":0,"num_object_copies":17872,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8936,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8936,"num_write_kb":35744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'9078","reported_seq":9203,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:01:42.023674+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:01:42.023674+0000","last_peered":"2024-09-10T16:01:42.023674+0000","last_clean":"2024-09-10T16:01:42.023674+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:01:42.023674+0000","last_undegraded":"2024-09-10T16:01:42.023674+0000","last_fullsized":"2024-09-10T16:01:42.023674+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":9078,"log_dups_size":0,"ondisk_log_size":9078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37155299,"num_objects":9072,"num_object_clones":0,"num_object_copies":18144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9072,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5789,"num_write_kb":23156,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'9137","reported_seq":9268,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:42.114743+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:01:42.114743+0000","last_peered":"2024-09-10T16:01:42.114743+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:01:42.114743+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":9137,"log_dups_size":0,"ondisk_log_size":9137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37405155,"num_objects":9133,"num_object_clones":0,"num_object_copies":18266,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9133,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5749,"num_write_kb":22996,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4762},{"shards":"0,2","objects":4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'9156","reported_seq":9289,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:42.099561+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:01:42.099561+0000","last_peered":"2024-09-10T16:01:42.099561+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:01:42.099561+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":9156,"log_dups_size":0,"ondisk_log_size":9156,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37479366,"num_objects":9152,"num_object_clones":0,"num_object_copies":18304,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9152,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5655,"num_write_kb":22620,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4915},{"shards":"0,2","objects":4237}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'9268","reported_seq":9393,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:01:41.999998+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:01:41.999998+0000","last_peered":"2024-09-10T16:01:41.999998+0000","last_clean":"2024-09-10T16:01:41.999998+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:01:41.999998+0000","last_undegraded":"2024-09-10T16:01:41.999998+0000","last_fullsized":"2024-09-10T16:01:41.999998+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":9268,"log_dups_size":0,"ondisk_log_size":9268,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37953536,"num_objects":9266,"num_object_clones":0,"num_object_copies":18532,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9266,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6115,"num_write_kb":24460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.777326+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T15:59:15.777326+0000","last_peered":"2024-09-10T15:59:15.777326+0000","last_clean":"2024-09-10T15:59:15.777326+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T15:59:15.777326+0000","last_undegraded":"2024-09-10T15:59:15.777326+0000","last_fullsized":"2024-09-10T15:59:15.777326+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'8740","reported_seq":8875,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:39.668569+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:01:39.668569+0000","last_peered":"2024-09-10T16:01:39.668569+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:01:39.668569+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":8740,"log_dups_size":0,"ondisk_log_size":8740,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35770851,"num_objects":8734,"num_object_clones":0,"num_object_copies":17468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8734,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6559,"num_write_kb":26236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4123},{"shards":"2","objects":4611}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":89,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.772619+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T15:59:15.772619+0000","last_peered":"2024-09-10T15:59:15.772619+0000","last_clean":"2024-09-10T15:59:15.772619+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T15:59:15.772619+0000","last_undegraded":"2024-09-10T15:59:15.772619+0000","last_fullsized":"2024-09-10T15:59:15.772619+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'9084","reported_seq":9210,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:42.113211+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:01:42.113211+0000","last_peered":"2024-09-10T16:01:42.113211+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:01:42.113211+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":9084,"log_dups_size":0,"ondisk_log_size":9084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37175296,"num_objects":9076,"num_object_clones":0,"num_object_copies":18152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9076,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5588,"num_write_kb":22352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4821},{"shards":"0,2","objects":4255}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":99,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.884297+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T15:59:15.884297+0000","last_peered":"2024-09-10T15:59:15.884297+0000","last_clean":"2024-09-10T15:59:15.884297+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T15:59:15.884297+0000","last_undegraded":"2024-09-10T15:59:15.884297+0000","last_fullsized":"2024-09-10T15:59:15.884297+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'8970","reported_seq":9100,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:39.681125+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:01:39.681125+0000","last_peered":"2024-09-10T16:01:39.681125+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:01:39.681125+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":8970,"log_dups_size":0,"ondisk_log_size":8970,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36721123,"num_objects":8966,"num_object_clones":0,"num_object_copies":17932,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8966,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6650,"num_write_kb":26600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4304},{"shards":"2","objects":4662}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.848670+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T15:59:15.848670+0000","last_peered":"2024-09-10T15:59:15.848670+0000","last_clean":"2024-09-10T15:59:15.848670+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T15:59:15.848670+0000","last_undegraded":"2024-09-10T15:59:15.848670+0000","last_fullsized":"2024-09-10T15:59:15.848670+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'9132","reported_seq":9256,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:39.681438+0000","last_change":"2024-09-10T15:59:16.174113+0000","last_active":"2024-09-10T16:01:39.681438+0000","last_peered":"2024-09-10T16:01:39.681438+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:01:39.681438+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":9132,"log_dups_size":0,"ondisk_log_size":9132,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37380096,"num_objects":9126,"num_object_clones":0,"num_object_copies":18252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4354,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9126,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6686,"num_write_kb":26744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4352},{"shards":"2","objects":4774}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'9035","reported_seq":9165,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:41.991942+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:01:41.991942+0000","last_peered":"2024-09-10T16:01:41.991942+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:01:41.991942+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":9035,"log_dups_size":0,"ondisk_log_size":9035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36999168,"num_objects":9033,"num_object_clones":0,"num_object_copies":18066,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9033,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5666,"num_write_kb":22664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4729},{"shards":"0,2","objects":4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.992406+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T15:59:15.992406+0000","last_peered":"2024-09-10T15:59:15.992406+0000","last_clean":"2024-09-10T15:59:15.992406+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T15:59:15.992406+0000","last_undegraded":"2024-09-10T15:59:15.992406+0000","last_fullsized":"2024-09-10T15:59:15.992406+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'8917","reported_seq":9055,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:42.017941+0000","last_change":"2024-09-10T15:59:16.251610+0000","last_active":"2024-09-10T16:01:42.017941+0000","last_peered":"2024-09-10T16:01:42.017941+0000","last_clean":"2024-09-10T15:59:14.115595+0000","last_became_active":"2024-09-10T15:59:16.244089+0000","last_became_peered":"2024-09-10T15:59:16.244089+0000","last_unstale":"2024-09-10T16:01:42.017941+0000","last_undegraded":"2024-09-10T15:59:15.847127+0000","last_fullsized":"2024-09-10T15:59:15.774437+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":8917,"log_dups_size":0,"ondisk_log_size":8917,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36491264,"num_objects":8909,"num_object_clones":0,"num_object_copies":17818,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8909,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5307,"num_write_kb":21228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4706},{"shards":"0,2","objects":4203}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":89,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.773633+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T15:59:15.773633+0000","last_peered":"2024-09-10T15:59:15.773633+0000","last_clean":"2024-09-10T15:59:15.773633+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T15:59:15.773633+0000","last_undegraded":"2024-09-10T15:59:15.773633+0000","last_fullsized":"2024-09-10T15:59:15.773633+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'9221","reported_seq":9358,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:42.063414+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:01:42.063414+0000","last_peered":"2024-09-10T16:01:42.063414+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:01:42.063414+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":9221,"log_dups_size":0,"ondisk_log_size":9221,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37752832,"num_objects":9217,"num_object_clones":0,"num_object_copies":18434,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9217,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5476,"num_write_kb":21904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4846},{"shards":"0,2","objects":4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.917634+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T15:59:15.917634+0000","last_peered":"2024-09-10T15:59:15.917634+0000","last_clean":"2024-09-10T15:59:15.917634+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T15:59:15.917634+0000","last_undegraded":"2024-09-10T15:59:15.917634+0000","last_fullsized":"2024-09-10T15:59:15.917634+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":70,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.778232+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T15:59:15.778232+0000","last_peered":"2024-09-10T15:59:15.778232+0000","last_clean":"2024-09-10T15:59:15.778232+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T15:59:15.778232+0000","last_undegraded":"2024-09-10T15:59:15.778232+0000","last_fullsized":"2024-09-10T15:59:15.778232+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'9199","reported_seq":9338,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:42.099824+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:01:42.099824+0000","last_peered":"2024-09-10T16:01:42.099824+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:01:42.099824+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":9199,"log_dups_size":0,"ondisk_log_size":9199,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37630918,"num_objects":9189,"num_object_clones":0,"num_object_copies":18378,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9189,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5362,"num_write_kb":21448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4866},{"shards":"0,2","objects":4323}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":592745182,"num_objects":144722,"num_object_clones":0,"num_object_copies":289444,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":47117,"num_objects_misplaced":893,"num_objects_unfound":0,"num_objects_dirty":144722,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":102568,"num_write_kb":410272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3379,"num_bytes_recovered":13836288,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1196658688,"data_stored":1196506942,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":144794,"ondisk_log_size":144794,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986220,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":695192,"kb_used_data":424712,"kb_used_omap":4,"kb_used_meta":270459,"kb_avail":93676648,"statfs":{"total":96636764160,"available":95924887552,"internally_reserved":0,"allocated":434905088,"data_stored":434627328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":276950698},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,2,3,3,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659166,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":422488,"kb_used_data":238088,"kb_used_omap":4,"kb_used_meta":184379,"kb_avail":93949352,"statfs":{"total":96636764160,"available":96204136448,"internally_reserved":0,"allocated":243802112,"data_stored":243476919,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":188804782},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,2,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182449,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":815468,"kb_used_data":504796,"kb_used_omap":4,"kb_used_meta":310651,"kb_avail":93556372,"statfs":{"total":96636764160,"available":95801724928,"internally_reserved":0,"allocated":516911104,"data_stored":516629267,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":318107306},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,0,2,8,2,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":516669440,"data_stored":516633310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":245002240,"data_stored":244922754,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":434987008,"data_stored":434950878,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:01:44.987 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 192 20 146385 146365 2.97739 1.79297 0.0383101 0.0262317 2024-09-10T16:01:45.989 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 193 20 146864 146844 2.97164 1.87109 0.0227162 0.0262833 2024-09-10T16:01:46.989 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 194 20 147344 147324 2.96598 1.875 0.042477 0.0263331 2024-09-10T16:01:47.259 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:01:47.822 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:01:47.823 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:01:47.859 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":172,"stamp":"2024-09-10T16:01:47.597714+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":602022641,"num_objects":146989,"num_object_clones":0,"num_object_copies":293978,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":47117,"num_objects_misplaced":743,"num_objects_unfound":0,"num_objects_dirty":146989,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":104835,"num_write_kb":419334,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3531,"num_bytes_recovered":14450707,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":147062,"ondisk_log_size":147062,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1936808,"kb_used_data":1187896,"kb_used_omap":12,"kb_used_meta":748851,"kb_avail":281178712,"statfs":{"total":289910292480,"available":287927001088,"internally_reserved":0,"allocated":1216405504,"data_stored":1215520714,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":766823426},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,0,1,2,6,10,8],"upper_bound":128},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":24682496,"num_objects":6026,"num_object_clones":0,"num_object_copies":12052,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-282,"num_objects_unfound":0,"num_objects_dirty":6026,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6026,"num_write_kb":24104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":282,"num_bytes_recovered":1155072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004376"},"pg_stats":[{"pgid":"2.f","version":"43'9138","reported_seq":9208,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:47.025180+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:01:47.025180+0000","last_peered":"2024-09-10T16:01:47.025180+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:01:47.025180+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":9138,"log_dups_size":0,"ondisk_log_size":9138,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37429248,"num_objects":9138,"num_object_clones":0,"num_object_copies":18276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9138,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9138,"num_write_kb":36552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4879},{"shards":"0,2","objects":4258}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'9104","reported_seq":9212,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:01:44.686988+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:01:44.686988+0000","last_peered":"2024-09-10T16:01:44.686988+0000","last_clean":"2024-09-10T16:01:44.686988+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:01:44.686988+0000","last_undegraded":"2024-09-10T16:01:44.686988+0000","last_fullsized":"2024-09-10T16:01:44.686988+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":9104,"log_dups_size":0,"ondisk_log_size":9104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37266374,"num_objects":9100,"num_object_clones":0,"num_object_copies":18200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9100,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":7209,"num_write_kb":28836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'9111","reported_seq":16287,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T16:01:44.691450+0000","last_change":"2024-09-10T15:59:16.142612+0000","last_active":"2024-09-10T16:01:44.691450+0000","last_peered":"2024-09-10T16:01:44.691450+0000","last_clean":"2024-09-10T15:59:14.092473+0000","last_became_active":"2024-09-10T15:59:16.109611+0000","last_became_peered":"2024-09-10T15:59:16.109611+0000","last_unstale":"2024-09-10T16:01:44.691450+0000","last_undegraded":"2024-09-10T16:01:44.691450+0000","last_fullsized":"2024-09-10T15:59:15.770264+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":9111,"log_dups_size":0,"ondisk_log_size":9111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37302272,"num_objects":9107,"num_object_clones":0,"num_object_copies":18214,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":743,"num_objects_unfound":0,"num_objects_dirty":9107,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7115,"num_write_kb":28460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3526,"num_bytes_recovered":14438400,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":743},{"shards":"2","objects":8364}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'9077","reported_seq":9153,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:01:44.685233+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:01:44.685233+0000","last_peered":"2024-09-10T16:01:44.685233+0000","last_clean":"2024-09-10T16:01:44.685233+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:01:44.685233+0000","last_undegraded":"2024-09-10T16:01:44.685233+0000","last_fullsized":"2024-09-10T16:01:44.685233+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":9077,"log_dups_size":0,"ondisk_log_size":9077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37179392,"num_objects":9077,"num_object_clones":0,"num_object_copies":18154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9077,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9077,"num_write_kb":36308,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'9238","reported_seq":9362,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:01:47.168716+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:01:47.168716+0000","last_peered":"2024-09-10T16:01:47.168716+0000","last_clean":"2024-09-10T16:01:47.168716+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:01:47.168716+0000","last_undegraded":"2024-09-10T16:01:47.168716+0000","last_fullsized":"2024-09-10T16:01:47.168716+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":9238,"log_dups_size":0,"ondisk_log_size":9238,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37810659,"num_objects":9232,"num_object_clones":0,"num_object_copies":18464,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9232,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5949,"num_write_kb":23796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'9300","reported_seq":9432,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:47.149818+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:01:47.149818+0000","last_peered":"2024-09-10T16:01:47.149818+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:01:47.149818+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":9300,"log_dups_size":0,"ondisk_log_size":9300,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38072803,"num_objects":9296,"num_object_clones":0,"num_object_copies":18592,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9296,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5912,"num_write_kb":23648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4925},{"shards":"0,2","objects":4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'9296","reported_seq":9429,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:47.106112+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:01:47.106112+0000","last_peered":"2024-09-10T16:01:47.106112+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:01:47.106112+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":9296,"log_dups_size":0,"ondisk_log_size":9296,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38052806,"num_objects":9292,"num_object_clones":0,"num_object_copies":18584,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9292,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5795,"num_write_kb":23180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5055},{"shards":"0,2","objects":4237}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'9405","reported_seq":9529,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:01:47.179371+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:01:47.179371+0000","last_peered":"2024-09-10T16:01:47.179371+0000","last_clean":"2024-09-10T16:01:47.179371+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:01:47.179371+0000","last_undegraded":"2024-09-10T16:01:47.179371+0000","last_fullsized":"2024-09-10T16:01:47.179371+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":9405,"log_dups_size":0,"ondisk_log_size":9405,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38514688,"num_objects":9403,"num_object_clones":0,"num_object_copies":18806,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9403,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6252,"num_write_kb":25008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.777326+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T15:59:15.777326+0000","last_peered":"2024-09-10T15:59:15.777326+0000","last_clean":"2024-09-10T15:59:15.777326+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T15:59:15.777326+0000","last_undegraded":"2024-09-10T15:59:15.777326+0000","last_fullsized":"2024-09-10T15:59:15.777326+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'8877","reported_seq":9012,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:44.683078+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:01:44.683078+0000","last_peered":"2024-09-10T16:01:44.683078+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:01:44.683078+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":8877,"log_dups_size":0,"ondisk_log_size":8877,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36332003,"num_objects":8871,"num_object_clones":0,"num_object_copies":17742,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8871,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6696,"num_write_kb":26784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4123},{"shards":"2","objects":4748}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":89,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.772619+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T15:59:15.772619+0000","last_peered":"2024-09-10T15:59:15.772619+0000","last_clean":"2024-09-10T15:59:15.772619+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T15:59:15.772619+0000","last_undegraded":"2024-09-10T15:59:15.772619+0000","last_fullsized":"2024-09-10T15:59:15.772619+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'9228","reported_seq":9354,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:47.077365+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:01:47.077365+0000","last_peered":"2024-09-10T16:01:47.077365+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:01:47.077365+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":9228,"log_dups_size":0,"ondisk_log_size":9228,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37765120,"num_objects":9220,"num_object_clones":0,"num_object_copies":18440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9220,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5732,"num_write_kb":22928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4965},{"shards":"0,2","objects":4255}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":99,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.884297+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T15:59:15.884297+0000","last_peered":"2024-09-10T15:59:15.884297+0000","last_clean":"2024-09-10T15:59:15.884297+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T15:59:15.884297+0000","last_undegraded":"2024-09-10T15:59:15.884297+0000","last_fullsized":"2024-09-10T15:59:15.884297+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'9101","reported_seq":9231,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:44.687401+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:01:44.687401+0000","last_peered":"2024-09-10T16:01:44.687401+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:01:44.687401+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":9101,"log_dups_size":0,"ondisk_log_size":9101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37257699,"num_objects":9097,"num_object_clones":0,"num_object_copies":18194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9097,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6781,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4304},{"shards":"2","objects":4793}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.848670+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T15:59:15.848670+0000","last_peered":"2024-09-10T15:59:15.848670+0000","last_clean":"2024-09-10T15:59:15.848670+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T15:59:15.848670+0000","last_undegraded":"2024-09-10T15:59:15.848670+0000","last_fullsized":"2024-09-10T15:59:15.848670+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'9260","reported_seq":9384,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:44.686246+0000","last_change":"2024-09-10T15:59:16.174113+0000","last_active":"2024-09-10T16:01:44.686246+0000","last_peered":"2024-09-10T16:01:44.686246+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:01:44.686246+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":9260,"log_dups_size":0,"ondisk_log_size":9260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37904384,"num_objects":9254,"num_object_clones":0,"num_object_copies":18508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4354,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9254,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6814,"num_write_kb":27256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4352},{"shards":"2","objects":4902}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'9172","reported_seq":9302,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:46.968369+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:01:46.968369+0000","last_peered":"2024-09-10T16:01:46.968369+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:01:46.968369+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":9172,"log_dups_size":0,"ondisk_log_size":9172,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37560320,"num_objects":9170,"num_object_clones":0,"num_object_copies":18340,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9170,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5803,"num_write_kb":23212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4866},{"shards":"0,2","objects":4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.992406+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T15:59:15.992406+0000","last_peered":"2024-09-10T15:59:15.992406+0000","last_clean":"2024-09-10T15:59:15.992406+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T15:59:15.992406+0000","last_undegraded":"2024-09-10T15:59:15.992406+0000","last_fullsized":"2024-09-10T15:59:15.992406+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'9049","reported_seq":9187,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:47.064912+0000","last_change":"2024-09-10T15:59:16.251610+0000","last_active":"2024-09-10T16:01:47.064912+0000","last_peered":"2024-09-10T16:01:47.064912+0000","last_clean":"2024-09-10T15:59:14.115595+0000","last_became_active":"2024-09-10T15:59:16.244089+0000","last_became_peered":"2024-09-10T15:59:16.244089+0000","last_unstale":"2024-09-10T16:01:47.064912+0000","last_undegraded":"2024-09-10T15:59:15.847127+0000","last_fullsized":"2024-09-10T15:59:15.774437+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":9049,"log_dups_size":0,"ondisk_log_size":9049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37031936,"num_objects":9041,"num_object_clones":0,"num_object_copies":18082,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9041,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5439,"num_write_kb":21756,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4838},{"shards":"0,2","objects":4203}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":89,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.773633+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T15:59:15.773633+0000","last_peered":"2024-09-10T15:59:15.773633+0000","last_clean":"2024-09-10T15:59:15.773633+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T15:59:15.773633+0000","last_undegraded":"2024-09-10T15:59:15.773633+0000","last_fullsized":"2024-09-10T15:59:15.773633+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'9354","reported_seq":9491,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:47.161387+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:01:47.161387+0000","last_peered":"2024-09-10T16:01:47.161387+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:01:47.161387+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":9354,"log_dups_size":0,"ondisk_log_size":9354,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38297600,"num_objects":9350,"num_object_clones":0,"num_object_copies":18700,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9350,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5609,"num_write_kb":22436,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4979},{"shards":"0,2","objects":4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.917634+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T15:59:15.917634+0000","last_peered":"2024-09-10T15:59:15.917634+0000","last_clean":"2024-09-10T15:59:15.917634+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T15:59:15.917634+0000","last_undegraded":"2024-09-10T15:59:15.917634+0000","last_fullsized":"2024-09-10T15:59:15.917634+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":70,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.778232+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T15:59:15.778232+0000","last_peered":"2024-09-10T15:59:15.778232+0000","last_clean":"2024-09-10T15:59:15.778232+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T15:59:15.778232+0000","last_undegraded":"2024-09-10T15:59:15.778232+0000","last_fullsized":"2024-09-10T15:59:15.778232+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'9349","reported_seq":9488,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:47.145544+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:01:47.145544+0000","last_peered":"2024-09-10T16:01:47.145544+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:01:47.145544+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":9349,"log_dups_size":0,"ondisk_log_size":9349,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38245318,"num_objects":9339,"num_object_clones":0,"num_object_copies":18678,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9339,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5512,"num_write_kb":22048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5016},{"shards":"0,2","objects":4323}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":602022622,"num_objects":146987,"num_object_clones":0,"num_object_copies":293974,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":47117,"num_objects_misplaced":743,"num_objects_unfound":0,"num_objects_dirty":146987,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":104833,"num_write_kb":419332,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3529,"num_bytes_recovered":14450688,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1215934464,"data_stored":1215782718,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":147059,"ondisk_log_size":147059,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986221,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":703788,"kb_used_data":429148,"kb_used_omap":4,"kb_used_meta":274619,"kb_avail":93668052,"statfs":{"total":96636764160,"available":95916085248,"internally_reserved":0,"allocated":439447552,"data_stored":439169792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":281210538},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,0,1,0,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659167,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":423048,"kb_used_data":246968,"kb_used_omap":4,"kb_used_meta":176059,"kb_avail":93948792,"statfs":{"total":96636764160,"available":96203563008,"internally_reserved":0,"allocated":252895232,"data_stored":252570039,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":180285102},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,0,1,3,5,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182450,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":809972,"kb_used_data":511780,"kb_used_omap":4,"kb_used_meta":298171,"kb_avail":93561868,"statfs":{"total":96636764160,"available":95807352832,"internally_reserved":0,"allocated":524062720,"data_stored":523780883,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":305327786},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,2,5,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":523739136,"data_stored":523703006,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":252583936,"data_stored":252504450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":439611392,"data_stored":439575262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:01:47.861 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:01:47.989 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 195 20 147780 147760 2.95951 1.70312 0.0237788 0.0263903 2024-09-10T16:01:48.410 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:01:48.413 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:01:48.437 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":172,"stamp":"2024-09-10T16:01:47.597714+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":602022641,"num_objects":146989,"num_object_clones":0,"num_object_copies":293978,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":47117,"num_objects_misplaced":743,"num_objects_unfound":0,"num_objects_dirty":146989,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":104835,"num_write_kb":419334,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3531,"num_bytes_recovered":14450707,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":147062,"ondisk_log_size":147062,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1936808,"kb_used_data":1187896,"kb_used_omap":12,"kb_used_meta":748851,"kb_avail":281178712,"statfs":{"total":289910292480,"available":287927001088,"internally_reserved":0,"allocated":1216405504,"data_stored":1215520714,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":766823426},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,0,1,2,6,10,8],"upper_bound":128},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":24682496,"num_objects":6026,"num_object_clones":0,"num_object_copies":12052,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-282,"num_objects_unfound":0,"num_objects_dirty":6026,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6026,"num_write_kb":24104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":282,"num_bytes_recovered":1155072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004376"},"pg_stats":[{"pgid":"2.f","version":"43'9138","reported_seq":9208,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:47.025180+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:01:47.025180+0000","last_peered":"2024-09-10T16:01:47.025180+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:01:47.025180+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":9138,"log_dups_size":0,"ondisk_log_size":9138,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37429248,"num_objects":9138,"num_object_clones":0,"num_object_copies":18276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9138,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9138,"num_write_kb":36552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4879},{"shards":"0,2","objects":4258}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'9104","reported_seq":9212,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:01:44.686988+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:01:44.686988+0000","last_peered":"2024-09-10T16:01:44.686988+0000","last_clean":"2024-09-10T16:01:44.686988+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:01:44.686988+0000","last_undegraded":"2024-09-10T16:01:44.686988+0000","last_fullsized":"2024-09-10T16:01:44.686988+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":9104,"log_dups_size":0,"ondisk_log_size":9104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37266374,"num_objects":9100,"num_object_clones":0,"num_object_copies":18200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9100,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":7209,"num_write_kb":28836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'9111","reported_seq":16287,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T16:01:44.691450+0000","last_change":"2024-09-10T15:59:16.142612+0000","last_active":"2024-09-10T16:01:44.691450+0000","last_peered":"2024-09-10T16:01:44.691450+0000","last_clean":"2024-09-10T15:59:14.092473+0000","last_became_active":"2024-09-10T15:59:16.109611+0000","last_became_peered":"2024-09-10T15:59:16.109611+0000","last_unstale":"2024-09-10T16:01:44.691450+0000","last_undegraded":"2024-09-10T16:01:44.691450+0000","last_fullsized":"2024-09-10T15:59:15.770264+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":9111,"log_dups_size":0,"ondisk_log_size":9111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37302272,"num_objects":9107,"num_object_clones":0,"num_object_copies":18214,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":743,"num_objects_unfound":0,"num_objects_dirty":9107,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7115,"num_write_kb":28460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3526,"num_bytes_recovered":14438400,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":743},{"shards":"2","objects":8364}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'9077","reported_seq":9153,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:01:44.685233+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:01:44.685233+0000","last_peered":"2024-09-10T16:01:44.685233+0000","last_clean":"2024-09-10T16:01:44.685233+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:01:44.685233+0000","last_undegraded":"2024-09-10T16:01:44.685233+0000","last_fullsized":"2024-09-10T16:01:44.685233+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":9077,"log_dups_size":0,"ondisk_log_size":9077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37179392,"num_objects":9077,"num_object_clones":0,"num_object_copies":18154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9077,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9077,"num_write_kb":36308,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'9238","reported_seq":9362,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:01:47.168716+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:01:47.168716+0000","last_peered":"2024-09-10T16:01:47.168716+0000","last_clean":"2024-09-10T16:01:47.168716+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:01:47.168716+0000","last_undegraded":"2024-09-10T16:01:47.168716+0000","last_fullsized":"2024-09-10T16:01:47.168716+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":9238,"log_dups_size":0,"ondisk_log_size":9238,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37810659,"num_objects":9232,"num_object_clones":0,"num_object_copies":18464,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9232,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5949,"num_write_kb":23796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'9300","reported_seq":9432,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:47.149818+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:01:47.149818+0000","last_peered":"2024-09-10T16:01:47.149818+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:01:47.149818+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":9300,"log_dups_size":0,"ondisk_log_size":9300,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38072803,"num_objects":9296,"num_object_clones":0,"num_object_copies":18592,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9296,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5912,"num_write_kb":23648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4925},{"shards":"0,2","objects":4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'9296","reported_seq":9429,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:47.106112+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:01:47.106112+0000","last_peered":"2024-09-10T16:01:47.106112+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:01:47.106112+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":9296,"log_dups_size":0,"ondisk_log_size":9296,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38052806,"num_objects":9292,"num_object_clones":0,"num_object_copies":18584,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9292,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5795,"num_write_kb":23180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5055},{"shards":"0,2","objects":4237}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'9405","reported_seq":9529,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:01:47.179371+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:01:47.179371+0000","last_peered":"2024-09-10T16:01:47.179371+0000","last_clean":"2024-09-10T16:01:47.179371+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:01:47.179371+0000","last_undegraded":"2024-09-10T16:01:47.179371+0000","last_fullsized":"2024-09-10T16:01:47.179371+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":9405,"log_dups_size":0,"ondisk_log_size":9405,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38514688,"num_objects":9403,"num_object_clones":0,"num_object_copies":18806,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9403,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6252,"num_write_kb":25008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.777326+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T15:59:15.777326+0000","last_peered":"2024-09-10T15:59:15.777326+0000","last_clean":"2024-09-10T15:59:15.777326+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T15:59:15.777326+0000","last_undegraded":"2024-09-10T15:59:15.777326+0000","last_fullsized":"2024-09-10T15:59:15.777326+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'8877","reported_seq":9012,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:44.683078+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:01:44.683078+0000","last_peered":"2024-09-10T16:01:44.683078+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:01:44.683078+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":8877,"log_dups_size":0,"ondisk_log_size":8877,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36332003,"num_objects":8871,"num_object_clones":0,"num_object_copies":17742,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8871,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6696,"num_write_kb":26784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4123},{"shards":"2","objects":4748}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":89,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.772619+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T15:59:15.772619+0000","last_peered":"2024-09-10T15:59:15.772619+0000","last_clean":"2024-09-10T15:59:15.772619+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T15:59:15.772619+0000","last_undegraded":"2024-09-10T15:59:15.772619+0000","last_fullsized":"2024-09-10T15:59:15.772619+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'9228","reported_seq":9354,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:47.077365+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:01:47.077365+0000","last_peered":"2024-09-10T16:01:47.077365+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:01:47.077365+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":9228,"log_dups_size":0,"ondisk_log_size":9228,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37765120,"num_objects":9220,"num_object_clones":0,"num_object_copies":18440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9220,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5732,"num_write_kb":22928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4965},{"shards":"0,2","objects":4255}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":99,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.884297+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T15:59:15.884297+0000","last_peered":"2024-09-10T15:59:15.884297+0000","last_clean":"2024-09-10T15:59:15.884297+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T15:59:15.884297+0000","last_undegraded":"2024-09-10T15:59:15.884297+0000","last_fullsized":"2024-09-10T15:59:15.884297+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'9101","reported_seq":9231,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:44.687401+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:01:44.687401+0000","last_peered":"2024-09-10T16:01:44.687401+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:01:44.687401+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":9101,"log_dups_size":0,"ondisk_log_size":9101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37257699,"num_objects":9097,"num_object_clones":0,"num_object_copies":18194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9097,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6781,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4304},{"shards":"2","objects":4793}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.848670+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T15:59:15.848670+0000","last_peered":"2024-09-10T15:59:15.848670+0000","last_clean":"2024-09-10T15:59:15.848670+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T15:59:15.848670+0000","last_undegraded":"2024-09-10T15:59:15.848670+0000","last_fullsized":"2024-09-10T15:59:15.848670+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'9260","reported_seq":9384,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:44.686246+0000","last_change":"2024-09-10T15:59:16.174113+0000","last_active":"2024-09-10T16:01:44.686246+0000","last_peered":"2024-09-10T16:01:44.686246+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:01:44.686246+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":9260,"log_dups_size":0,"ondisk_log_size":9260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37904384,"num_objects":9254,"num_object_clones":0,"num_object_copies":18508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4354,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9254,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6814,"num_write_kb":27256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4352},{"shards":"2","objects":4902}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'9172","reported_seq":9302,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:46.968369+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:01:46.968369+0000","last_peered":"2024-09-10T16:01:46.968369+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:01:46.968369+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":9172,"log_dups_size":0,"ondisk_log_size":9172,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37560320,"num_objects":9170,"num_object_clones":0,"num_object_copies":18340,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9170,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5803,"num_write_kb":23212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4866},{"shards":"0,2","objects":4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.992406+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T15:59:15.992406+0000","last_peered":"2024-09-10T15:59:15.992406+0000","last_clean":"2024-09-10T15:59:15.992406+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T15:59:15.992406+0000","last_undegraded":"2024-09-10T15:59:15.992406+0000","last_fullsized":"2024-09-10T15:59:15.992406+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'9049","reported_seq":9187,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:47.064912+0000","last_change":"2024-09-10T15:59:16.251610+0000","last_active":"2024-09-10T16:01:47.064912+0000","last_peered":"2024-09-10T16:01:47.064912+0000","last_clean":"2024-09-10T15:59:14.115595+0000","last_became_active":"2024-09-10T15:59:16.244089+0000","last_became_peered":"2024-09-10T15:59:16.244089+0000","last_unstale":"2024-09-10T16:01:47.064912+0000","last_undegraded":"2024-09-10T15:59:15.847127+0000","last_fullsized":"2024-09-10T15:59:15.774437+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":9049,"log_dups_size":0,"ondisk_log_size":9049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37031936,"num_objects":9041,"num_object_clones":0,"num_object_copies":18082,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9041,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5439,"num_write_kb":21756,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4838},{"shards":"0,2","objects":4203}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":89,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.773633+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T15:59:15.773633+0000","last_peered":"2024-09-10T15:59:15.773633+0000","last_clean":"2024-09-10T15:59:15.773633+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T15:59:15.773633+0000","last_undegraded":"2024-09-10T15:59:15.773633+0000","last_fullsized":"2024-09-10T15:59:15.773633+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'9354","reported_seq":9491,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:47.161387+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:01:47.161387+0000","last_peered":"2024-09-10T16:01:47.161387+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:01:47.161387+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":9354,"log_dups_size":0,"ondisk_log_size":9354,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38297600,"num_objects":9350,"num_object_clones":0,"num_object_copies":18700,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9350,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5609,"num_write_kb":22436,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4979},{"shards":"0,2","objects":4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.917634+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T15:59:15.917634+0000","last_peered":"2024-09-10T15:59:15.917634+0000","last_clean":"2024-09-10T15:59:15.917634+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T15:59:15.917634+0000","last_undegraded":"2024-09-10T15:59:15.917634+0000","last_fullsized":"2024-09-10T15:59:15.917634+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":70,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.778232+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T15:59:15.778232+0000","last_peered":"2024-09-10T15:59:15.778232+0000","last_clean":"2024-09-10T15:59:15.778232+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T15:59:15.778232+0000","last_undegraded":"2024-09-10T15:59:15.778232+0000","last_fullsized":"2024-09-10T15:59:15.778232+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'9349","reported_seq":9488,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:47.145544+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:01:47.145544+0000","last_peered":"2024-09-10T16:01:47.145544+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:01:47.145544+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":9349,"log_dups_size":0,"ondisk_log_size":9349,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38245318,"num_objects":9339,"num_object_clones":0,"num_object_copies":18678,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9339,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5512,"num_write_kb":22048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5016},{"shards":"0,2","objects":4323}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":602022622,"num_objects":146987,"num_object_clones":0,"num_object_copies":293974,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":47117,"num_objects_misplaced":743,"num_objects_unfound":0,"num_objects_dirty":146987,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":104833,"num_write_kb":419332,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3529,"num_bytes_recovered":14450688,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1215934464,"data_stored":1215782718,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":147059,"ondisk_log_size":147059,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986221,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":703788,"kb_used_data":429148,"kb_used_omap":4,"kb_used_meta":274619,"kb_avail":93668052,"statfs":{"total":96636764160,"available":95916085248,"internally_reserved":0,"allocated":439447552,"data_stored":439169792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":281210538},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,0,1,0,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659167,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":423048,"kb_used_data":246968,"kb_used_omap":4,"kb_used_meta":176059,"kb_avail":93948792,"statfs":{"total":96636764160,"available":96203563008,"internally_reserved":0,"allocated":252895232,"data_stored":252570039,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":180285102},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,0,1,3,5,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182450,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":809972,"kb_used_data":511780,"kb_used_omap":4,"kb_used_meta":298171,"kb_avail":93561868,"statfs":{"total":96636764160,"available":95807352832,"internally_reserved":0,"allocated":524062720,"data_stored":523780883,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":305327786},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,2,5,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":523739136,"data_stored":523703006,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":252583936,"data_stored":252504450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":439611392,"data_stored":439575262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:01:48.438 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-10T16:01:48.439 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "43'9138", 'reported_seq': 9208, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:01:47.025180+0000', 'last_change': '2024-09-10T15:59:16.281558+0000', 'last_active': '2024-09-10T16:01:47.025180+0000', 'last_peered': '2024-09-10T16:01:47.025180+0000', 'last_clean': '2024-09-10T15:59:14.132456+0000', 'last_became_active': '2024-09-10T15:59:16.274407+0000', 'last_became_peered': '2024-09-10T15:59:16.274407+0000', 'last_unstale': '2024-09-10T16:01:47.025180+0000', 'last_undegraded': '2024-09-10T15:59:15.851279+0000', 'last_fullsized': '2024-09-10T15:59:15.774356+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'objects_scrubbed': 0, 'log_size': 9138, 'log_dups_size': 0, 'ondisk_log_size': 9138, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37429248, 'num_objects': 9138, 'num_object_clones': 0, 'num_object_copies': 18276, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4259, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9138, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9138, 'num_write_kb': 36552, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4879}, {'shards': '0,2', 'objects': 4258}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:01:48.439 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-10T16:01:48.439 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "43'9111", 'reported_seq': 16287, 'reported_epoch': 43, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-10T16:01:44.691450+0000', 'last_change': '2024-09-10T15:59:16.142612+0000', 'last_active': '2024-09-10T16:01:44.691450+0000', 'last_peered': '2024-09-10T16:01:44.691450+0000', 'last_clean': '2024-09-10T15:59:14.092473+0000', 'last_became_active': '2024-09-10T15:59:16.109611+0000', 'last_became_peered': '2024-09-10T15:59:16.109611+0000', 'last_unstale': '2024-09-10T16:01:44.691450+0000', 'last_undegraded': '2024-09-10T16:01:44.691450+0000', 'last_fullsized': '2024-09-10T15:59:15.770264+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2029", 'last_scrub_stamp': '2024-09-10T15:58:45.933789+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:45.933789+0000', 'objects_scrubbed': 1992, 'log_size': 9111, 'log_dups_size': 0, 'ondisk_log_size': 9111, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 668, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37302272, 'num_objects': 9107, 'num_object_clones': 0, 'num_object_copies': 18214, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 743, 'num_objects_unfound': 0, 'num_objects_dirty': 9107, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7115, 'num_write_kb': 28460, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3526, 'num_bytes_recovered': 14438400, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 743}, {'shards': '2', 'objects': 8364}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:01:48.439 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-10T16:01:48.439 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "43'9300", 'reported_seq': 9432, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:01:47.149818+0000', 'last_change': '2024-09-10T15:59:16.308264+0000', 'last_active': '2024-09-10T16:01:47.149818+0000', 'last_peered': '2024-09-10T16:01:47.149818+0000', 'last_clean': '2024-09-10T15:59:14.111551+0000', 'last_became_active': '2024-09-10T15:59:16.300040+0000', 'last_became_peered': '2024-09-10T15:59:16.300040+0000', 'last_unstale': '2024-09-10T16:01:47.149818+0000', 'last_undegraded': '2024-09-10T15:59:15.878234+0000', 'last_fullsized': '2024-09-10T15:59:15.769328+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3442", 'last_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'objects_scrubbed': 3384, 'log_size': 9300, 'log_dups_size': 0, 'ondisk_log_size': 9300, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1292, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38072803, 'num_objects': 9296, 'num_object_clones': 0, 'num_object_copies': 18592, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4372, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9296, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 5912, 'num_write_kb': 23648, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4925}, {'shards': '0,2', 'objects': 4371}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:01:48.439 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-10T16:01:48.439 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "43'9296", 'reported_seq': 9429, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:01:47.106112+0000', 'last_change': '2024-09-10T15:59:16.331899+0000', 'last_active': '2024-09-10T16:01:47.106112+0000', 'last_peered': '2024-09-10T16:01:47.106112+0000', 'last_clean': '2024-09-10T15:59:14.139436+0000', 'last_became_active': '2024-09-10T15:59:16.318715+0000', 'last_became_peered': '2024-09-10T15:59:16.318715+0000', 'last_unstale': '2024-09-10T16:01:47.106112+0000', 'last_undegraded': '2024-09-10T15:59:15.877524+0000', 'last_fullsized': '2024-09-10T15:59:15.769042+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3537", 'last_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'objects_scrubbed': 3497, 'log_size': 9296, 'log_dups_size': 0, 'ondisk_log_size': 9296, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1224, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38052806, 'num_objects': 9292, 'num_object_clones': 0, 'num_object_copies': 18584, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4237, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9292, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 5795, 'num_write_kb': 23180, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5055}, {'shards': '0,2', 'objects': 4237}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:01:48.439 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-10T16:01:48.439 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "43'8877", 'reported_seq': 9012, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:01:44.683078+0000', 'last_change': '2024-09-10T15:59:16.175680+0000', 'last_active': '2024-09-10T16:01:44.683078+0000', 'last_peered': '2024-09-10T16:01:44.683078+0000', 'last_clean': '2024-09-10T15:59:14.088187+0000', 'last_became_active': '2024-09-10T15:59:16.165810+0000', 'last_became_peered': '2024-09-10T15:59:16.165810+0000', 'last_unstale': '2024-09-10T16:01:44.683078+0000', 'last_undegraded': '2024-09-10T15:59:15.822098+0000', 'last_fullsized': '2024-09-10T15:59:15.769123+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2218", 'last_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'objects_scrubbed': 2175, 'log_size': 8877, 'log_dups_size': 0, 'ondisk_log_size': 8877, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 825, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36332003, 'num_objects': 8871, 'num_object_clones': 0, 'num_object_copies': 17742, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4124, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8871, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 6696, 'num_write_kb': 26784, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4123}, {'shards': '2', 'objects': 4748}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:01:48.440 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-10T16:01:48.440 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "43'9228", 'reported_seq': 9354, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:01:47.077365+0000', 'last_change': '2024-09-10T15:59:16.277742+0000', 'last_active': '2024-09-10T16:01:47.077365+0000', 'last_peered': '2024-09-10T16:01:47.077365+0000', 'last_clean': '2024-09-10T15:59:14.129081+0000', 'last_became_active': '2024-09-10T15:59:16.269835+0000', 'last_became_peered': '2024-09-10T15:59:16.269835+0000', 'last_unstale': '2024-09-10T16:01:47.077365+0000', 'last_undegraded': '2024-09-10T15:59:15.860857+0000', 'last_fullsized': '2024-09-10T15:59:15.771410+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3557", 'last_scrub_stamp': '2024-09-10T15:59:02.493678+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:02.493678+0000', 'objects_scrubbed': 3488, 'log_size': 9228, 'log_dups_size': 0, 'ondisk_log_size': 9228, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1472, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37765120, 'num_objects': 9220, 'num_object_clones': 0, 'num_object_copies': 18440, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4257, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9220, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5732, 'num_write_kb': 22928, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4965}, {'shards': '0,2', 'objects': 4255}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:01:48.440 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-10T16:01:48.440 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "43'9101", 'reported_seq': 9231, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:01:44.687401+0000', 'last_change': '2024-09-10T15:59:16.284311+0000', 'last_active': '2024-09-10T16:01:44.687401+0000', 'last_peered': '2024-09-10T16:01:44.687401+0000', 'last_clean': '2024-09-10T15:59:14.116041+0000', 'last_became_active': '2024-09-10T15:59:16.274543+0000', 'last_became_peered': '2024-09-10T15:59:16.274543+0000', 'last_unstale': '2024-09-10T16:01:44.687401+0000', 'last_undegraded': '2024-09-10T15:59:15.812484+0000', 'last_fullsized': '2024-09-10T15:59:15.770885+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2356", 'last_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'objects_scrubbed': 2316, 'log_size': 9101, 'log_dups_size': 0, 'ondisk_log_size': 9101, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 783, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37257699, 'num_objects': 9097, 'num_object_clones': 0, 'num_object_copies': 18194, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9097, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 6781, 'num_write_kb': 27124, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4304}, {'shards': '2', 'objects': 4793}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:01:48.440 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-10T16:01:48.440 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "43'9260", 'reported_seq': 9384, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:01:44.686246+0000', 'last_change': '2024-09-10T15:59:16.174113+0000', 'last_active': '2024-09-10T16:01:44.686246+0000', 'last_peered': '2024-09-10T16:01:44.686246+0000', 'last_clean': '2024-09-10T15:59:14.110654+0000', 'last_became_active': '2024-09-10T15:59:16.164645+0000', 'last_became_peered': '2024-09-10T15:59:16.164645+0000', 'last_unstale': '2024-09-10T16:01:44.686246+0000', 'last_undegraded': '2024-09-10T15:59:15.803593+0000', 'last_fullsized': '2024-09-10T15:59:15.769083+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2486", 'last_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'objects_scrubbed': 2440, 'log_size': 9260, 'log_dups_size': 0, 'ondisk_log_size': 9260, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 792, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37904384, 'num_objects': 9254, 'num_object_clones': 0, 'num_object_copies': 18508, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4354, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9254, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6814, 'num_write_kb': 27256, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4352}, {'shards': '2', 'objects': 4902}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:01:48.440 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-10T16:01:48.440 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "43'9172", 'reported_seq': 9302, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:01:46.968369+0000', 'last_change': '2024-09-10T15:59:16.315641+0000', 'last_active': '2024-09-10T16:01:46.968369+0000', 'last_peered': '2024-09-10T16:01:46.968369+0000', 'last_clean': '2024-09-10T15:59:14.117001+0000', 'last_became_active': '2024-09-10T15:59:16.302245+0000', 'last_became_peered': '2024-09-10T15:59:16.302245+0000', 'last_unstale': '2024-09-10T16:01:46.968369+0000', 'last_undegraded': '2024-09-10T15:59:15.845012+0000', 'last_fullsized': '2024-09-10T15:59:15.770337+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3404", 'last_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'objects_scrubbed': 3367, 'log_size': 9172, 'log_dups_size': 0, 'ondisk_log_size': 9172, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 983, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37560320, 'num_objects': 9170, 'num_object_clones': 0, 'num_object_copies': 18340, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9170, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5803, 'num_write_kb': 23212, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4866}, {'shards': '0,2', 'objects': 4304}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:01:48.440 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-10T16:01:48.440 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "43'9049", 'reported_seq': 9187, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:01:47.064912+0000', 'last_change': '2024-09-10T15:59:16.251610+0000', 'last_active': '2024-09-10T16:01:47.064912+0000', 'last_peered': '2024-09-10T16:01:47.064912+0000', 'last_clean': '2024-09-10T15:59:14.115595+0000', 'last_became_active': '2024-09-10T15:59:16.244089+0000', 'last_became_peered': '2024-09-10T15:59:16.244089+0000', 'last_unstale': '2024-09-10T16:01:47.064912+0000', 'last_undegraded': '2024-09-10T15:59:15.847127+0000', 'last_fullsized': '2024-09-10T15:59:15.774437+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3653", 'last_scrub_stamp': '2024-09-10T15:59:05.298468+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:05.298468+0000', 'objects_scrubbed': 3602, 'log_size': 9049, 'log_dups_size': 0, 'ondisk_log_size': 9049, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1289, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37031936, 'num_objects': 9041, 'num_object_clones': 0, 'num_object_copies': 18082, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4206, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9041, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5439, 'num_write_kb': 21756, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4838}, {'shards': '0,2', 'objects': 4203}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:01:48.441 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-10T16:01:48.441 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "43'9354", 'reported_seq': 9491, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:01:47.161387+0000', 'last_change': '2024-09-10T15:59:16.311291+0000', 'last_active': '2024-09-10T16:01:47.161387+0000', 'last_peered': '2024-09-10T16:01:47.161387+0000', 'last_clean': '2024-09-10T15:59:14.116363+0000', 'last_became_active': '2024-09-10T15:59:16.301244+0000', 'last_became_peered': '2024-09-10T15:59:16.301244+0000', 'last_unstale': '2024-09-10T16:01:47.161387+0000', 'last_undegraded': '2024-09-10T15:59:15.883434+0000', 'last_fullsized': '2024-09-10T15:59:15.772756+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3789", 'last_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'objects_scrubbed': 3741, 'log_size': 9354, 'log_dups_size': 0, 'ondisk_log_size': 9354, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1440, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38297600, 'num_objects': 9350, 'num_object_clones': 0, 'num_object_copies': 18700, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4373, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9350, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5609, 'num_write_kb': 22436, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4979}, {'shards': '0,2', 'objects': 4371}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:01:48.441 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-10T16:01:48.441 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "43'9349", 'reported_seq': 9488, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:01:47.145544+0000', 'last_change': '2024-09-10T15:59:16.313409+0000', 'last_active': '2024-09-10T16:01:47.145544+0000', 'last_peered': '2024-09-10T16:01:47.145544+0000', 'last_clean': '2024-09-10T15:59:14.129780+0000', 'last_became_active': '2024-09-10T15:59:16.300932+0000', 'last_became_peered': '2024-09-10T15:59:16.300932+0000', 'last_unstale': '2024-09-10T16:01:47.145544+0000', 'last_undegraded': '2024-09-10T15:59:15.887823+0000', 'last_fullsized': '2024-09-10T15:59:15.768513+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3880", 'last_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'objects_scrubbed': 3827, 'log_size': 9349, 'log_dups_size': 0, 'ondisk_log_size': 9349, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1397, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38245318, 'num_objects': 9339, 'num_object_clones': 0, 'num_object_copies': 18678, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4325, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9339, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5512, 'num_write_kb': 22048, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5016}, {'shards': '0,2', 'objects': 4323}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:01:48.441 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T16:01:48.990 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 196 20 148204 148184 2.95284 1.65625 0.0477641 0.0264502 2024-09-10T16:01:49.141 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:01:49.141 INFO:teuthology.orchestra.run.smithi115.stdout:{"fsid":"7effee29-df05-4351-9741-ee203564ed81","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 47117/293978 objects degraded (16.027%), 11 pgs degraded, 12 pgs undersized","count":23},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":273,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":43,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983954,"num_in_osds":3,"osd_in_since":1725983947,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":146989,"data_bytes":602022641,"bytes_used":1983291392,"bytes_avail":287927001088,"bytes_total":289910292480,"degraded_objects":47117,"degraded_total":293978,"degraded_ratio":0.16027389804679262,"misplaced_objects":743,"misplaced_total":293978,"misplaced_ratio":0.0025274000095245218,"recovering_objects_per_sec":23,"recovering_bytes_per_sec":96220,"recovering_keys_per_sec":0,"num_objects_recovered":282,"num_bytes_recovered":1155072,"num_keys_recovered":0,"write_bytes_sec":2056124,"read_op_per_sec":0,"write_op_per_sec":501},"fsmap":{"epoch":1,"btime":"2024-09-10T15:57:14:967120+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T16:00:13.560776+0000","services":{}},"progress_events":{}} 2024-09-10T16:01:49.169 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-10T16:01:49.169 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:01:49.734 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:01:49.734 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:01:49.759 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":173,"stamp":"2024-09-10T16:01:49.598526+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":602022641,"num_objects":146989,"num_object_clones":0,"num_object_copies":293978,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":47117,"num_objects_misplaced":743,"num_objects_unfound":0,"num_objects_dirty":146989,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":104835,"num_write_kb":419334,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3531,"num_bytes_recovered":14450707,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":147062,"ondisk_log_size":147062,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1948440,"kb_used_data":1195368,"kb_used_omap":12,"kb_used_meta":753011,"kb_avail":281167080,"statfs":{"total":289910292480,"available":287915089920,"internally_reserved":0,"allocated":1224056832,"data_stored":1223172042,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":771083266},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,2,6,7,11],"upper_bound":128},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":18702336,"num_objects":4566,"num_object_clones":0,"num_object_copies":9132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-282,"num_objects_unfound":0,"num_objects_dirty":4566,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4566,"num_write_kb":18264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":282,"num_bytes_recovered":1155072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004288"},"pg_stats":[{"pgid":"2.f","version":"43'9138","reported_seq":9208,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:47.025180+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:01:47.025180+0000","last_peered":"2024-09-10T16:01:47.025180+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:01:47.025180+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":9138,"log_dups_size":0,"ondisk_log_size":9138,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37429248,"num_objects":9138,"num_object_clones":0,"num_object_copies":18276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9138,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9138,"num_write_kb":36552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4879},{"shards":"0,2","objects":4258}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'9104","reported_seq":9212,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:01:44.686988+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:01:44.686988+0000","last_peered":"2024-09-10T16:01:44.686988+0000","last_clean":"2024-09-10T16:01:44.686988+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:01:44.686988+0000","last_undegraded":"2024-09-10T16:01:44.686988+0000","last_fullsized":"2024-09-10T16:01:44.686988+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":9104,"log_dups_size":0,"ondisk_log_size":9104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37266374,"num_objects":9100,"num_object_clones":0,"num_object_copies":18200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9100,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":7209,"num_write_kb":28836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'9111","reported_seq":16287,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T16:01:44.691450+0000","last_change":"2024-09-10T15:59:16.142612+0000","last_active":"2024-09-10T16:01:44.691450+0000","last_peered":"2024-09-10T16:01:44.691450+0000","last_clean":"2024-09-10T15:59:14.092473+0000","last_became_active":"2024-09-10T15:59:16.109611+0000","last_became_peered":"2024-09-10T15:59:16.109611+0000","last_unstale":"2024-09-10T16:01:44.691450+0000","last_undegraded":"2024-09-10T16:01:44.691450+0000","last_fullsized":"2024-09-10T15:59:15.770264+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":9111,"log_dups_size":0,"ondisk_log_size":9111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37302272,"num_objects":9107,"num_object_clones":0,"num_object_copies":18214,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":743,"num_objects_unfound":0,"num_objects_dirty":9107,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7115,"num_write_kb":28460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3526,"num_bytes_recovered":14438400,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":743},{"shards":"2","objects":8364}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'9077","reported_seq":9153,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:01:44.685233+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:01:44.685233+0000","last_peered":"2024-09-10T16:01:44.685233+0000","last_clean":"2024-09-10T16:01:44.685233+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:01:44.685233+0000","last_undegraded":"2024-09-10T16:01:44.685233+0000","last_fullsized":"2024-09-10T16:01:44.685233+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":9077,"log_dups_size":0,"ondisk_log_size":9077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37179392,"num_objects":9077,"num_object_clones":0,"num_object_copies":18154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9077,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9077,"num_write_kb":36308,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'9238","reported_seq":9362,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:01:47.168716+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:01:47.168716+0000","last_peered":"2024-09-10T16:01:47.168716+0000","last_clean":"2024-09-10T16:01:47.168716+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:01:47.168716+0000","last_undegraded":"2024-09-10T16:01:47.168716+0000","last_fullsized":"2024-09-10T16:01:47.168716+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":9238,"log_dups_size":0,"ondisk_log_size":9238,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37810659,"num_objects":9232,"num_object_clones":0,"num_object_copies":18464,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9232,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5949,"num_write_kb":23796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'9300","reported_seq":9432,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:47.149818+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:01:47.149818+0000","last_peered":"2024-09-10T16:01:47.149818+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:01:47.149818+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":9300,"log_dups_size":0,"ondisk_log_size":9300,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38072803,"num_objects":9296,"num_object_clones":0,"num_object_copies":18592,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9296,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5912,"num_write_kb":23648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4925},{"shards":"0,2","objects":4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'9296","reported_seq":9429,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:47.106112+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:01:47.106112+0000","last_peered":"2024-09-10T16:01:47.106112+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:01:47.106112+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":9296,"log_dups_size":0,"ondisk_log_size":9296,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38052806,"num_objects":9292,"num_object_clones":0,"num_object_copies":18584,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9292,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5795,"num_write_kb":23180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5055},{"shards":"0,2","objects":4237}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'9405","reported_seq":9529,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:01:47.179371+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:01:47.179371+0000","last_peered":"2024-09-10T16:01:47.179371+0000","last_clean":"2024-09-10T16:01:47.179371+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:01:47.179371+0000","last_undegraded":"2024-09-10T16:01:47.179371+0000","last_fullsized":"2024-09-10T16:01:47.179371+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":9405,"log_dups_size":0,"ondisk_log_size":9405,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38514688,"num_objects":9403,"num_object_clones":0,"num_object_copies":18806,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9403,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6252,"num_write_kb":25008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.777326+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T15:59:15.777326+0000","last_peered":"2024-09-10T15:59:15.777326+0000","last_clean":"2024-09-10T15:59:15.777326+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T15:59:15.777326+0000","last_undegraded":"2024-09-10T15:59:15.777326+0000","last_fullsized":"2024-09-10T15:59:15.777326+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'8877","reported_seq":9012,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:44.683078+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:01:44.683078+0000","last_peered":"2024-09-10T16:01:44.683078+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:01:44.683078+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":8877,"log_dups_size":0,"ondisk_log_size":8877,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36332003,"num_objects":8871,"num_object_clones":0,"num_object_copies":17742,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8871,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6696,"num_write_kb":26784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4123},{"shards":"2","objects":4748}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":89,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.772619+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T15:59:15.772619+0000","last_peered":"2024-09-10T15:59:15.772619+0000","last_clean":"2024-09-10T15:59:15.772619+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T15:59:15.772619+0000","last_undegraded":"2024-09-10T15:59:15.772619+0000","last_fullsized":"2024-09-10T15:59:15.772619+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'9228","reported_seq":9354,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:47.077365+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:01:47.077365+0000","last_peered":"2024-09-10T16:01:47.077365+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:01:47.077365+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":9228,"log_dups_size":0,"ondisk_log_size":9228,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37765120,"num_objects":9220,"num_object_clones":0,"num_object_copies":18440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9220,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5732,"num_write_kb":22928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4965},{"shards":"0,2","objects":4255}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":99,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.884297+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T15:59:15.884297+0000","last_peered":"2024-09-10T15:59:15.884297+0000","last_clean":"2024-09-10T15:59:15.884297+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T15:59:15.884297+0000","last_undegraded":"2024-09-10T15:59:15.884297+0000","last_fullsized":"2024-09-10T15:59:15.884297+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'9101","reported_seq":9231,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:44.687401+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:01:44.687401+0000","last_peered":"2024-09-10T16:01:44.687401+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:01:44.687401+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":9101,"log_dups_size":0,"ondisk_log_size":9101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37257699,"num_objects":9097,"num_object_clones":0,"num_object_copies":18194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9097,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6781,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4304},{"shards":"2","objects":4793}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.848670+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T15:59:15.848670+0000","last_peered":"2024-09-10T15:59:15.848670+0000","last_clean":"2024-09-10T15:59:15.848670+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T15:59:15.848670+0000","last_undegraded":"2024-09-10T15:59:15.848670+0000","last_fullsized":"2024-09-10T15:59:15.848670+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'9260","reported_seq":9384,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:44.686246+0000","last_change":"2024-09-10T15:59:16.174113+0000","last_active":"2024-09-10T16:01:44.686246+0000","last_peered":"2024-09-10T16:01:44.686246+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:01:44.686246+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":9260,"log_dups_size":0,"ondisk_log_size":9260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37904384,"num_objects":9254,"num_object_clones":0,"num_object_copies":18508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4354,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9254,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6814,"num_write_kb":27256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4352},{"shards":"2","objects":4902}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'9172","reported_seq":9302,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:46.968369+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:01:46.968369+0000","last_peered":"2024-09-10T16:01:46.968369+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:01:46.968369+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":9172,"log_dups_size":0,"ondisk_log_size":9172,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37560320,"num_objects":9170,"num_object_clones":0,"num_object_copies":18340,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9170,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5803,"num_write_kb":23212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4866},{"shards":"0,2","objects":4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.992406+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T15:59:15.992406+0000","last_peered":"2024-09-10T15:59:15.992406+0000","last_clean":"2024-09-10T15:59:15.992406+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T15:59:15.992406+0000","last_undegraded":"2024-09-10T15:59:15.992406+0000","last_fullsized":"2024-09-10T15:59:15.992406+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'9049","reported_seq":9187,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:47.064912+0000","last_change":"2024-09-10T15:59:16.251610+0000","last_active":"2024-09-10T16:01:47.064912+0000","last_peered":"2024-09-10T16:01:47.064912+0000","last_clean":"2024-09-10T15:59:14.115595+0000","last_became_active":"2024-09-10T15:59:16.244089+0000","last_became_peered":"2024-09-10T15:59:16.244089+0000","last_unstale":"2024-09-10T16:01:47.064912+0000","last_undegraded":"2024-09-10T15:59:15.847127+0000","last_fullsized":"2024-09-10T15:59:15.774437+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":9049,"log_dups_size":0,"ondisk_log_size":9049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37031936,"num_objects":9041,"num_object_clones":0,"num_object_copies":18082,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9041,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5439,"num_write_kb":21756,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4838},{"shards":"0,2","objects":4203}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":89,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.773633+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T15:59:15.773633+0000","last_peered":"2024-09-10T15:59:15.773633+0000","last_clean":"2024-09-10T15:59:15.773633+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T15:59:15.773633+0000","last_undegraded":"2024-09-10T15:59:15.773633+0000","last_fullsized":"2024-09-10T15:59:15.773633+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'9354","reported_seq":9491,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:47.161387+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:01:47.161387+0000","last_peered":"2024-09-10T16:01:47.161387+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:01:47.161387+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":9354,"log_dups_size":0,"ondisk_log_size":9354,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38297600,"num_objects":9350,"num_object_clones":0,"num_object_copies":18700,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9350,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5609,"num_write_kb":22436,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4979},{"shards":"0,2","objects":4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.917634+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T15:59:15.917634+0000","last_peered":"2024-09-10T15:59:15.917634+0000","last_clean":"2024-09-10T15:59:15.917634+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T15:59:15.917634+0000","last_undegraded":"2024-09-10T15:59:15.917634+0000","last_fullsized":"2024-09-10T15:59:15.917634+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":70,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.778232+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T15:59:15.778232+0000","last_peered":"2024-09-10T15:59:15.778232+0000","last_clean":"2024-09-10T15:59:15.778232+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T15:59:15.778232+0000","last_undegraded":"2024-09-10T15:59:15.778232+0000","last_fullsized":"2024-09-10T15:59:15.778232+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'9349","reported_seq":9488,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:47.145544+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:01:47.145544+0000","last_peered":"2024-09-10T16:01:47.145544+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:01:47.145544+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":9349,"log_dups_size":0,"ondisk_log_size":9349,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38245318,"num_objects":9339,"num_object_clones":0,"num_object_copies":18678,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9339,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5512,"num_write_kb":22048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5016},{"shards":"0,2","objects":4323}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":602022622,"num_objects":146987,"num_object_clones":0,"num_object_copies":293974,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":47117,"num_objects_misplaced":743,"num_objects_unfound":0,"num_objects_dirty":146987,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":104833,"num_write_kb":419332,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3529,"num_bytes_recovered":14450688,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1223680000,"data_stored":1223528254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":147059,"ondisk_log_size":147059,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986221,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":703788,"kb_used_data":429148,"kb_used_omap":4,"kb_used_meta":274619,"kb_avail":93668052,"statfs":{"total":96636764160,"available":95916085248,"internally_reserved":0,"allocated":439447552,"data_stored":439169792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":281210538},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,0,1,0,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659168,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":434680,"kb_used_data":254440,"kb_used_omap":4,"kb_used_meta":180219,"kb_avail":93937160,"statfs":{"total":96636764160,"available":96191651840,"internally_reserved":0,"allocated":260546560,"data_stored":260221367,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":184544942},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,3,2,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182450,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":809972,"kb_used_data":511780,"kb_used_omap":4,"kb_used_meta":298171,"kb_avail":93561868,"statfs":{"total":96636764160,"available":95807352832,"internally_reserved":0,"allocated":524062720,"data_stored":523780883,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":305327786},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,2,5,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":523739136,"data_stored":523703006,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":260329472,"data_stored":260249986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":439611392,"data_stored":439575262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:01:49.990 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 197 20 148638 148618 2.94646 1.69531 0.079917 0.0265082 2024-09-10T16:01:50.990 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 198 20 149119 149099 2.94107 1.87891 0.0259821 0.0265558 2024-09-10T16:01:51.990 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 199 20 149586 149566 2.93545 1.82422 0.0258961 0.0266061 2024-09-10T16:01:52.761 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:01:52.990 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout:2024-09-10T16:01:52.989956+0000 min lat: 0.00225581 max lat: 2.21736 avg lat: 0.0266588 2024-09-10T16:01:52.990 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-10T16:01:52.990 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 200 20 150044 150024 2.92972 1.78906 0.0571852 0.0266588 2024-09-10T16:01:53.315 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:01:53.316 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:01:53.343 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":174,"stamp":"2024-09-10T16:01:51.599348+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":605504241,"num_objects":147839,"num_object_clones":0,"num_object_copies":295678,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":47117,"num_objects_misplaced":574,"num_objects_unfound":0,"num_objects_dirty":147839,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":105685,"num_write_kb":422734,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3700,"num_bytes_recovered":15142931,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":147912,"ondisk_log_size":147912,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1957120,"kb_used_data":1199888,"kb_used_omap":12,"kb_used_meta":757171,"kb_avail":281158400,"statfs":{"total":289910292480,"available":287906201600,"internally_reserved":0,"allocated":1228685312,"data_stored":1227800522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":775343106},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,2,6,7,11],"upper_bound":128},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":22183936,"num_objects":5416,"num_object_clones":0,"num_object_copies":10832,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-451,"num_objects_unfound":0,"num_objects_dirty":5416,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5416,"num_write_kb":21664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":451,"num_bytes_recovered":1847296,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004255"},"pg_stats":[{"pgid":"2.f","version":"43'9138","reported_seq":9208,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:47.025180+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:01:47.025180+0000","last_peered":"2024-09-10T16:01:47.025180+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:01:47.025180+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":9138,"log_dups_size":0,"ondisk_log_size":9138,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37429248,"num_objects":9138,"num_object_clones":0,"num_object_copies":18276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9138,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9138,"num_write_kb":36552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4879},{"shards":"0,2","objects":4258}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'9240","reported_seq":9348,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:01:49.699286+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:01:49.699286+0000","last_peered":"2024-09-10T16:01:49.699286+0000","last_clean":"2024-09-10T16:01:49.699286+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:01:49.699286+0000","last_undegraded":"2024-09-10T16:01:49.699286+0000","last_fullsized":"2024-09-10T16:01:49.699286+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":9240,"log_dups_size":0,"ondisk_log_size":9240,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37823430,"num_objects":9236,"num_object_clones":0,"num_object_copies":18472,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9236,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":7345,"num_write_kb":29380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'9235","reported_seq":16748,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T16:01:49.748161+0000","last_change":"2024-09-10T15:59:16.142612+0000","last_active":"2024-09-10T16:01:49.748161+0000","last_peered":"2024-09-10T16:01:49.748161+0000","last_clean":"2024-09-10T15:59:14.092473+0000","last_became_active":"2024-09-10T15:59:16.109611+0000","last_became_peered":"2024-09-10T15:59:16.109611+0000","last_unstale":"2024-09-10T16:01:49.748161+0000","last_undegraded":"2024-09-10T16:01:49.748161+0000","last_fullsized":"2024-09-10T15:59:15.770264+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":9235,"log_dups_size":0,"ondisk_log_size":9235,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37810176,"num_objects":9231,"num_object_clones":0,"num_object_copies":18462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":574,"num_objects_unfound":0,"num_objects_dirty":9231,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7239,"num_write_kb":28956,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3695,"num_bytes_recovered":15130624,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":574},{"shards":"2","objects":8657}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'9225","reported_seq":9301,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:01:49.742124+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:01:49.742124+0000","last_peered":"2024-09-10T16:01:49.742124+0000","last_clean":"2024-09-10T16:01:49.742124+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:01:49.742124+0000","last_undegraded":"2024-09-10T16:01:49.742124+0000","last_fullsized":"2024-09-10T16:01:49.742124+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":9225,"log_dups_size":0,"ondisk_log_size":9225,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37785600,"num_objects":9225,"num_object_clones":0,"num_object_copies":18450,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9225,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9225,"num_write_kb":36900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'9238","reported_seq":9362,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:01:47.168716+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:01:47.168716+0000","last_peered":"2024-09-10T16:01:47.168716+0000","last_clean":"2024-09-10T16:01:47.168716+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:01:47.168716+0000","last_undegraded":"2024-09-10T16:01:47.168716+0000","last_fullsized":"2024-09-10T16:01:47.168716+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":9238,"log_dups_size":0,"ondisk_log_size":9238,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37810659,"num_objects":9232,"num_object_clones":0,"num_object_copies":18464,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9232,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5949,"num_write_kb":23796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'9300","reported_seq":9432,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:47.149818+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:01:47.149818+0000","last_peered":"2024-09-10T16:01:47.149818+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:01:47.149818+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":9300,"log_dups_size":0,"ondisk_log_size":9300,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38072803,"num_objects":9296,"num_object_clones":0,"num_object_copies":18592,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9296,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5912,"num_write_kb":23648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4925},{"shards":"0,2","objects":4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'9296","reported_seq":9429,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:47.106112+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:01:47.106112+0000","last_peered":"2024-09-10T16:01:47.106112+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:01:47.106112+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":9296,"log_dups_size":0,"ondisk_log_size":9296,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38052806,"num_objects":9292,"num_object_clones":0,"num_object_copies":18584,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9292,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5795,"num_write_kb":23180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5055},{"shards":"0,2","objects":4237}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'9405","reported_seq":9529,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:01:47.179371+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:01:47.179371+0000","last_peered":"2024-09-10T16:01:47.179371+0000","last_clean":"2024-09-10T16:01:47.179371+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:01:47.179371+0000","last_undegraded":"2024-09-10T16:01:47.179371+0000","last_fullsized":"2024-09-10T16:01:47.179371+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":9405,"log_dups_size":0,"ondisk_log_size":9405,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38514688,"num_objects":9403,"num_object_clones":0,"num_object_copies":18806,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9403,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6252,"num_write_kb":25008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.777326+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T15:59:15.777326+0000","last_peered":"2024-09-10T15:59:15.777326+0000","last_clean":"2024-09-10T15:59:15.777326+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T15:59:15.777326+0000","last_undegraded":"2024-09-10T15:59:15.777326+0000","last_fullsized":"2024-09-10T15:59:15.777326+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'9026","reported_seq":9161,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:49.668391+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:01:49.668391+0000","last_peered":"2024-09-10T16:01:49.668391+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:01:49.668391+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":9026,"log_dups_size":0,"ondisk_log_size":9026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36942307,"num_objects":9020,"num_object_clones":0,"num_object_copies":18040,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9020,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6845,"num_write_kb":27380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4123},{"shards":"2","objects":4897}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":89,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.772619+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T15:59:15.772619+0000","last_peered":"2024-09-10T15:59:15.772619+0000","last_clean":"2024-09-10T15:59:15.772619+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T15:59:15.772619+0000","last_undegraded":"2024-09-10T15:59:15.772619+0000","last_fullsized":"2024-09-10T15:59:15.772619+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'9228","reported_seq":9354,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:47.077365+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:01:47.077365+0000","last_peered":"2024-09-10T16:01:47.077365+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:01:47.077365+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":9228,"log_dups_size":0,"ondisk_log_size":9228,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37765120,"num_objects":9220,"num_object_clones":0,"num_object_copies":18440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9220,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5732,"num_write_kb":22928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4965},{"shards":"0,2","objects":4255}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":99,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.884297+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T15:59:15.884297+0000","last_peered":"2024-09-10T15:59:15.884297+0000","last_clean":"2024-09-10T15:59:15.884297+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T15:59:15.884297+0000","last_undegraded":"2024-09-10T15:59:15.884297+0000","last_fullsized":"2024-09-10T15:59:15.884297+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'9252","reported_seq":9382,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:49.723773+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:01:49.723773+0000","last_peered":"2024-09-10T16:01:49.723773+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:01:49.723773+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":9252,"log_dups_size":0,"ondisk_log_size":9252,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37876195,"num_objects":9248,"num_object_clones":0,"num_object_copies":18496,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9248,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6932,"num_write_kb":27728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4304},{"shards":"2","objects":4944}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.848670+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T15:59:15.848670+0000","last_peered":"2024-09-10T15:59:15.848670+0000","last_clean":"2024-09-10T15:59:15.848670+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T15:59:15.848670+0000","last_undegraded":"2024-09-10T15:59:15.848670+0000","last_fullsized":"2024-09-10T15:59:15.848670+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'9402","reported_seq":9526,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:49.619835+0000","last_change":"2024-09-10T15:59:16.174113+0000","last_active":"2024-09-10T16:01:49.619835+0000","last_peered":"2024-09-10T16:01:49.619835+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:01:49.619835+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":9402,"log_dups_size":0,"ondisk_log_size":9402,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38486016,"num_objects":9396,"num_object_clones":0,"num_object_copies":18792,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4354,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9396,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6956,"num_write_kb":27824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4352},{"shards":"2","objects":5044}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'9172","reported_seq":9302,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:46.968369+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:01:46.968369+0000","last_peered":"2024-09-10T16:01:46.968369+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:01:46.968369+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":9172,"log_dups_size":0,"ondisk_log_size":9172,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37560320,"num_objects":9170,"num_object_clones":0,"num_object_copies":18340,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9170,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5803,"num_write_kb":23212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4866},{"shards":"0,2","objects":4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.992406+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T15:59:15.992406+0000","last_peered":"2024-09-10T15:59:15.992406+0000","last_clean":"2024-09-10T15:59:15.992406+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T15:59:15.992406+0000","last_undegraded":"2024-09-10T15:59:15.992406+0000","last_fullsized":"2024-09-10T15:59:15.992406+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'9049","reported_seq":9187,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:47.064912+0000","last_change":"2024-09-10T15:59:16.251610+0000","last_active":"2024-09-10T16:01:47.064912+0000","last_peered":"2024-09-10T16:01:47.064912+0000","last_clean":"2024-09-10T15:59:14.115595+0000","last_became_active":"2024-09-10T15:59:16.244089+0000","last_became_peered":"2024-09-10T15:59:16.244089+0000","last_unstale":"2024-09-10T16:01:47.064912+0000","last_undegraded":"2024-09-10T15:59:15.847127+0000","last_fullsized":"2024-09-10T15:59:15.774437+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":9049,"log_dups_size":0,"ondisk_log_size":9049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37031936,"num_objects":9041,"num_object_clones":0,"num_object_copies":18082,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9041,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5439,"num_write_kb":21756,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4838},{"shards":"0,2","objects":4203}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":89,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.773633+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T15:59:15.773633+0000","last_peered":"2024-09-10T15:59:15.773633+0000","last_clean":"2024-09-10T15:59:15.773633+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T15:59:15.773633+0000","last_undegraded":"2024-09-10T15:59:15.773633+0000","last_fullsized":"2024-09-10T15:59:15.773633+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'9354","reported_seq":9491,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:47.161387+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:01:47.161387+0000","last_peered":"2024-09-10T16:01:47.161387+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:01:47.161387+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":9354,"log_dups_size":0,"ondisk_log_size":9354,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38297600,"num_objects":9350,"num_object_clones":0,"num_object_copies":18700,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9350,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5609,"num_write_kb":22436,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4979},{"shards":"0,2","objects":4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.917634+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T15:59:15.917634+0000","last_peered":"2024-09-10T15:59:15.917634+0000","last_clean":"2024-09-10T15:59:15.917634+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T15:59:15.917634+0000","last_undegraded":"2024-09-10T15:59:15.917634+0000","last_fullsized":"2024-09-10T15:59:15.917634+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":70,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.778232+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T15:59:15.778232+0000","last_peered":"2024-09-10T15:59:15.778232+0000","last_clean":"2024-09-10T15:59:15.778232+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T15:59:15.778232+0000","last_undegraded":"2024-09-10T15:59:15.778232+0000","last_fullsized":"2024-09-10T15:59:15.778232+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'9349","reported_seq":9488,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:47.145544+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:01:47.145544+0000","last_peered":"2024-09-10T16:01:47.145544+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:01:47.145544+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":9349,"log_dups_size":0,"ondisk_log_size":9349,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38245318,"num_objects":9339,"num_object_clones":0,"num_object_copies":18678,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9339,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5512,"num_write_kb":22048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5016},{"shards":"0,2","objects":4323}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":605504222,"num_objects":147837,"num_object_clones":0,"num_object_copies":295674,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":47117,"num_objects_misplaced":574,"num_objects_unfound":0,"num_objects_dirty":147837,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":105683,"num_write_kb":422732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3698,"num_bytes_recovered":15142912,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1228374016,"data_stored":1228222270,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":147909,"ondisk_log_size":147909,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986222,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":712468,"kb_used_data":433668,"kb_used_omap":4,"kb_used_meta":278779,"kb_avail":93659372,"statfs":{"total":96636764160,"available":95907196928,"internally_reserved":0,"allocated":444076032,"data_stored":443798272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":285470378},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,0,1,0,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659168,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":434680,"kb_used_data":254440,"kb_used_omap":4,"kb_used_meta":180219,"kb_avail":93937160,"statfs":{"total":96636764160,"available":96191651840,"internally_reserved":0,"allocated":260546560,"data_stored":260221367,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":184544942},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,3,2,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182450,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":809972,"kb_used_data":511780,"kb_used_omap":4,"kb_used_meta":298171,"kb_avail":93561868,"statfs":{"total":96636764160,"available":95807352832,"internally_reserved":0,"allocated":524062720,"data_stored":523780883,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":305327786},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,2,5,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":523739136,"data_stored":523703006,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":260329472,"data_stored":260249986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":444305408,"data_stored":444269278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:01:53.344 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:01:53.844 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:01:53.845 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:01:53.878 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":175,"stamp":"2024-09-10T16:01:53.599974+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":611304177,"num_objects":149255,"num_object_clones":0,"num_object_copies":298510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":47117,"num_objects_misplaced":574,"num_objects_unfound":0,"num_objects_dirty":149255,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":107101,"num_write_kb":428398,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3700,"num_bytes_recovered":15142931,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":149328,"ondisk_log_size":149328,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1966776,"kb_used_data":1205384,"kb_used_omap":12,"kb_used_meta":761331,"kb_avail":281148744,"statfs":{"total":289910292480,"available":287896313856,"internally_reserved":0,"allocated":1234313216,"data_stored":1233428426,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":779602946},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,0,3,8,5,8,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":24481792,"num_objects":5977,"num_object_clones":0,"num_object_copies":11954,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-319,"num_objects_unfound":0,"num_objects_dirty":5977,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5977,"num_write_kb":23908,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":319,"num_bytes_recovered":1306624,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004269"},"pg_stats":[{"pgid":"2.f","version":"43'9280","reported_seq":9350,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:52.136350+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:01:52.136350+0000","last_peered":"2024-09-10T16:01:52.136350+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:01:52.136350+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":9280,"log_dups_size":0,"ondisk_log_size":9280,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38010880,"num_objects":9280,"num_object_clones":0,"num_object_copies":18560,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9280,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9280,"num_write_kb":37120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5021},{"shards":"0,2","objects":4258}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'9240","reported_seq":9348,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:01:49.699286+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:01:49.699286+0000","last_peered":"2024-09-10T16:01:49.699286+0000","last_clean":"2024-09-10T16:01:49.699286+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:01:49.699286+0000","last_undegraded":"2024-09-10T16:01:49.699286+0000","last_fullsized":"2024-09-10T16:01:49.699286+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":9240,"log_dups_size":0,"ondisk_log_size":9240,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37823430,"num_objects":9236,"num_object_clones":0,"num_object_copies":18472,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9236,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":7345,"num_write_kb":29380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'9235","reported_seq":16748,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T16:01:49.748161+0000","last_change":"2024-09-10T15:59:16.142612+0000","last_active":"2024-09-10T16:01:49.748161+0000","last_peered":"2024-09-10T16:01:49.748161+0000","last_clean":"2024-09-10T15:59:14.092473+0000","last_became_active":"2024-09-10T15:59:16.109611+0000","last_became_peered":"2024-09-10T15:59:16.109611+0000","last_unstale":"2024-09-10T16:01:49.748161+0000","last_undegraded":"2024-09-10T16:01:49.748161+0000","last_fullsized":"2024-09-10T15:59:15.770264+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":9235,"log_dups_size":0,"ondisk_log_size":9235,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37810176,"num_objects":9231,"num_object_clones":0,"num_object_copies":18462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":574,"num_objects_unfound":0,"num_objects_dirty":9231,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7239,"num_write_kb":28956,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3695,"num_bytes_recovered":15130624,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":574},{"shards":"2","objects":8657}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'9225","reported_seq":9301,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:01:49.742124+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:01:49.742124+0000","last_peered":"2024-09-10T16:01:49.742124+0000","last_clean":"2024-09-10T16:01:49.742124+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:01:49.742124+0000","last_undegraded":"2024-09-10T16:01:49.742124+0000","last_fullsized":"2024-09-10T16:01:49.742124+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":9225,"log_dups_size":0,"ondisk_log_size":9225,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37785600,"num_objects":9225,"num_object_clones":0,"num_object_copies":18450,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9225,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9225,"num_write_kb":36900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'9366","reported_seq":9491,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:01:52.247277+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:01:52.247277+0000","last_peered":"2024-09-10T16:01:52.247277+0000","last_clean":"2024-09-10T16:01:52.247277+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:01:52.247277+0000","last_undegraded":"2024-09-10T16:01:52.247277+0000","last_fullsized":"2024-09-10T16:01:52.247277+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":9366,"log_dups_size":0,"ondisk_log_size":9366,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38334947,"num_objects":9360,"num_object_clones":0,"num_object_copies":18720,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9360,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6077,"num_write_kb":24308,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'9437","reported_seq":9570,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:52.157258+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:01:52.157258+0000","last_peered":"2024-09-10T16:01:52.157258+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:01:52.157258+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":9437,"log_dups_size":0,"ondisk_log_size":9437,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38633955,"num_objects":9433,"num_object_clones":0,"num_object_copies":18866,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9433,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6049,"num_write_kb":24196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5062},{"shards":"0,2","objects":4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'9442","reported_seq":9575,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:52.187622+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:01:52.187622+0000","last_peered":"2024-09-10T16:01:52.187622+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:01:52.187622+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":9442,"log_dups_size":0,"ondisk_log_size":9442,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38650822,"num_objects":9438,"num_object_clones":0,"num_object_copies":18876,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9438,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5941,"num_write_kb":23764,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5201},{"shards":"0,2","objects":4237}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'9571","reported_seq":9695,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:01:52.229693+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:01:52.229693+0000","last_peered":"2024-09-10T16:01:52.229693+0000","last_clean":"2024-09-10T16:01:52.229693+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:01:52.229693+0000","last_undegraded":"2024-09-10T16:01:52.229693+0000","last_fullsized":"2024-09-10T16:01:52.229693+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":9571,"log_dups_size":0,"ondisk_log_size":9571,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39194624,"num_objects":9569,"num_object_clones":0,"num_object_copies":19138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9569,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6418,"num_write_kb":25672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.777326+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T15:59:15.777326+0000","last_peered":"2024-09-10T15:59:15.777326+0000","last_clean":"2024-09-10T15:59:15.777326+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T15:59:15.777326+0000","last_undegraded":"2024-09-10T15:59:15.777326+0000","last_fullsized":"2024-09-10T15:59:15.777326+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'9026","reported_seq":9161,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:49.668391+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:01:49.668391+0000","last_peered":"2024-09-10T16:01:49.668391+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:01:49.668391+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":9026,"log_dups_size":0,"ondisk_log_size":9026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36942307,"num_objects":9020,"num_object_clones":0,"num_object_copies":18040,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9020,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6845,"num_write_kb":27380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4123},{"shards":"2","objects":4897}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":89,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.772619+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T15:59:15.772619+0000","last_peered":"2024-09-10T15:59:15.772619+0000","last_clean":"2024-09-10T15:59:15.772619+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T15:59:15.772619+0000","last_undegraded":"2024-09-10T15:59:15.772619+0000","last_fullsized":"2024-09-10T15:59:15.772619+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'9366","reported_seq":9492,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:52.164805+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:01:52.164805+0000","last_peered":"2024-09-10T16:01:52.164805+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:01:52.164805+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":9366,"log_dups_size":0,"ondisk_log_size":9366,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38330368,"num_objects":9358,"num_object_clones":0,"num_object_copies":18716,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9358,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5870,"num_write_kb":23480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5103},{"shards":"0,2","objects":4255}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":99,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.884297+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T15:59:15.884297+0000","last_peered":"2024-09-10T15:59:15.884297+0000","last_clean":"2024-09-10T15:59:15.884297+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T15:59:15.884297+0000","last_undegraded":"2024-09-10T15:59:15.884297+0000","last_fullsized":"2024-09-10T15:59:15.884297+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'9252","reported_seq":9382,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:49.723773+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:01:49.723773+0000","last_peered":"2024-09-10T16:01:49.723773+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:01:49.723773+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":9252,"log_dups_size":0,"ondisk_log_size":9252,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37876195,"num_objects":9248,"num_object_clones":0,"num_object_copies":18496,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9248,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6932,"num_write_kb":27728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4304},{"shards":"2","objects":4944}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.848670+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T15:59:15.848670+0000","last_peered":"2024-09-10T15:59:15.848670+0000","last_clean":"2024-09-10T15:59:15.848670+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T15:59:15.848670+0000","last_undegraded":"2024-09-10T15:59:15.848670+0000","last_fullsized":"2024-09-10T15:59:15.848670+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'9402","reported_seq":9526,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:49.619835+0000","last_change":"2024-09-10T15:59:16.174113+0000","last_active":"2024-09-10T16:01:49.619835+0000","last_peered":"2024-09-10T16:01:49.619835+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:01:49.619835+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":9402,"log_dups_size":0,"ondisk_log_size":9402,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38486016,"num_objects":9396,"num_object_clones":0,"num_object_copies":18792,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4354,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9396,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6956,"num_write_kb":27824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4352},{"shards":"2","objects":5044}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'9322","reported_seq":9451,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:52.229195+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:01:52.229195+0000","last_peered":"2024-09-10T16:01:52.229195+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:01:52.229195+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":9322,"log_dups_size":0,"ondisk_log_size":9322,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38174720,"num_objects":9320,"num_object_clones":0,"num_object_copies":18640,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9320,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5953,"num_write_kb":23812,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5016},{"shards":"0,2","objects":4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.992406+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T15:59:15.992406+0000","last_peered":"2024-09-10T15:59:15.992406+0000","last_clean":"2024-09-10T15:59:15.992406+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T15:59:15.992406+0000","last_undegraded":"2024-09-10T15:59:15.992406+0000","last_fullsized":"2024-09-10T15:59:15.992406+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'9178","reported_seq":9313,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:52.187332+0000","last_change":"2024-09-10T15:59:16.251610+0000","last_active":"2024-09-10T16:01:52.187332+0000","last_peered":"2024-09-10T16:01:52.187332+0000","last_clean":"2024-09-10T15:59:14.115595+0000","last_became_active":"2024-09-10T15:59:16.244089+0000","last_became_peered":"2024-09-10T15:59:16.244089+0000","last_unstale":"2024-09-10T16:01:52.187332+0000","last_undegraded":"2024-09-10T15:59:15.847127+0000","last_fullsized":"2024-09-10T15:59:15.774437+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":9178,"log_dups_size":0,"ondisk_log_size":9178,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37560320,"num_objects":9170,"num_object_clones":0,"num_object_copies":18340,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9170,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5568,"num_write_kb":22272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4967},{"shards":"0,2","objects":4203}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":89,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.773633+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T15:59:15.773633+0000","last_peered":"2024-09-10T15:59:15.773633+0000","last_clean":"2024-09-10T15:59:15.773633+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T15:59:15.773633+0000","last_undegraded":"2024-09-10T15:59:15.773633+0000","last_fullsized":"2024-09-10T15:59:15.773633+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'9489","reported_seq":9624,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:52.130767+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:01:52.130767+0000","last_peered":"2024-09-10T16:01:52.130767+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:01:52.130767+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":9489,"log_dups_size":0,"ondisk_log_size":9489,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38850560,"num_objects":9485,"num_object_clones":0,"num_object_copies":18970,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9485,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5744,"num_write_kb":22976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5114},{"shards":"0,2","objects":4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.917634+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T15:59:15.917634+0000","last_peered":"2024-09-10T15:59:15.917634+0000","last_clean":"2024-09-10T15:59:15.917634+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T15:59:15.917634+0000","last_undegraded":"2024-09-10T15:59:15.917634+0000","last_fullsized":"2024-09-10T15:59:15.917634+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":70,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.778232+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T15:59:15.778232+0000","last_peered":"2024-09-10T15:59:15.778232+0000","last_clean":"2024-09-10T15:59:15.778232+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T15:59:15.778232+0000","last_undegraded":"2024-09-10T15:59:15.778232+0000","last_fullsized":"2024-09-10T15:59:15.778232+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'9494","reported_seq":9634,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:52.196955+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:01:52.196955+0000","last_peered":"2024-09-10T16:01:52.196955+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:01:52.196955+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":9494,"log_dups_size":0,"ondisk_log_size":9494,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38839238,"num_objects":9484,"num_object_clones":0,"num_object_copies":18968,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9484,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5657,"num_write_kb":22628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5161},{"shards":"0,2","objects":4323}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":611304158,"num_objects":149253,"num_object_clones":0,"num_object_copies":298506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":47117,"num_objects_misplaced":574,"num_objects_unfound":0,"num_objects_dirty":149253,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":107099,"num_write_kb":428396,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3698,"num_bytes_recovered":15142912,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1235308544,"data_stored":1235156798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":149325,"ondisk_log_size":149325,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986222,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":712468,"kb_used_data":433668,"kb_used_omap":4,"kb_used_meta":278779,"kb_avail":93659372,"statfs":{"total":96636764160,"available":95907196928,"internally_reserved":0,"allocated":444076032,"data_stored":443798272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":285470378},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,0,1,0,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659168,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":434680,"kb_used_data":254440,"kb_used_omap":4,"kb_used_meta":180219,"kb_avail":93937160,"statfs":{"total":96636764160,"available":96191651840,"internally_reserved":0,"allocated":260546560,"data_stored":260221367,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":184544942},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,3,2,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182451,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":819628,"kb_used_data":517276,"kb_used_omap":4,"kb_used_meta":302331,"kb_avail":93552212,"statfs":{"total":96636764160,"available":95797465088,"internally_reserved":0,"allocated":529690624,"data_stored":529408787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":309587626},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,2,4,3,2,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":530673664,"data_stored":530637534,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":260329472,"data_stored":260249986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":444305408,"data_stored":444269278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:01:53.879 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-10T16:01:53.879 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "43'9280", 'reported_seq': 9350, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:01:52.136350+0000', 'last_change': '2024-09-10T15:59:16.281558+0000', 'last_active': '2024-09-10T16:01:52.136350+0000', 'last_peered': '2024-09-10T16:01:52.136350+0000', 'last_clean': '2024-09-10T15:59:14.132456+0000', 'last_became_active': '2024-09-10T15:59:16.274407+0000', 'last_became_peered': '2024-09-10T15:59:16.274407+0000', 'last_unstale': '2024-09-10T16:01:52.136350+0000', 'last_undegraded': '2024-09-10T15:59:15.851279+0000', 'last_fullsized': '2024-09-10T15:59:15.774356+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'objects_scrubbed': 0, 'log_size': 9280, 'log_dups_size': 0, 'ondisk_log_size': 9280, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38010880, 'num_objects': 9280, 'num_object_clones': 0, 'num_object_copies': 18560, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4259, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9280, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9280, 'num_write_kb': 37120, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5021}, {'shards': '0,2', 'objects': 4258}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:01:53.879 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-10T16:01:53.879 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "43'9235", 'reported_seq': 16748, 'reported_epoch': 43, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-10T16:01:49.748161+0000', 'last_change': '2024-09-10T15:59:16.142612+0000', 'last_active': '2024-09-10T16:01:49.748161+0000', 'last_peered': '2024-09-10T16:01:49.748161+0000', 'last_clean': '2024-09-10T15:59:14.092473+0000', 'last_became_active': '2024-09-10T15:59:16.109611+0000', 'last_became_peered': '2024-09-10T15:59:16.109611+0000', 'last_unstale': '2024-09-10T16:01:49.748161+0000', 'last_undegraded': '2024-09-10T16:01:49.748161+0000', 'last_fullsized': '2024-09-10T15:59:15.770264+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2029", 'last_scrub_stamp': '2024-09-10T15:58:45.933789+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:45.933789+0000', 'objects_scrubbed': 1992, 'log_size': 9235, 'log_dups_size': 0, 'ondisk_log_size': 9235, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 668, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37810176, 'num_objects': 9231, 'num_object_clones': 0, 'num_object_copies': 18462, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 574, 'num_objects_unfound': 0, 'num_objects_dirty': 9231, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7239, 'num_write_kb': 28956, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3695, 'num_bytes_recovered': 15130624, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 574}, {'shards': '2', 'objects': 8657}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:01:53.879 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-10T16:01:53.879 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "43'9437", 'reported_seq': 9570, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:01:52.157258+0000', 'last_change': '2024-09-10T15:59:16.308264+0000', 'last_active': '2024-09-10T16:01:52.157258+0000', 'last_peered': '2024-09-10T16:01:52.157258+0000', 'last_clean': '2024-09-10T15:59:14.111551+0000', 'last_became_active': '2024-09-10T15:59:16.300040+0000', 'last_became_peered': '2024-09-10T15:59:16.300040+0000', 'last_unstale': '2024-09-10T16:01:52.157258+0000', 'last_undegraded': '2024-09-10T15:59:15.878234+0000', 'last_fullsized': '2024-09-10T15:59:15.769328+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3442", 'last_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'objects_scrubbed': 3384, 'log_size': 9437, 'log_dups_size': 0, 'ondisk_log_size': 9437, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1292, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38633955, 'num_objects': 9433, 'num_object_clones': 0, 'num_object_copies': 18866, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4372, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9433, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 6049, 'num_write_kb': 24196, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5062}, {'shards': '0,2', 'objects': 4371}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:01:53.880 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-10T16:01:53.880 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "43'9442", 'reported_seq': 9575, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:01:52.187622+0000', 'last_change': '2024-09-10T15:59:16.331899+0000', 'last_active': '2024-09-10T16:01:52.187622+0000', 'last_peered': '2024-09-10T16:01:52.187622+0000', 'last_clean': '2024-09-10T15:59:14.139436+0000', 'last_became_active': '2024-09-10T15:59:16.318715+0000', 'last_became_peered': '2024-09-10T15:59:16.318715+0000', 'last_unstale': '2024-09-10T16:01:52.187622+0000', 'last_undegraded': '2024-09-10T15:59:15.877524+0000', 'last_fullsized': '2024-09-10T15:59:15.769042+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3537", 'last_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'objects_scrubbed': 3497, 'log_size': 9442, 'log_dups_size': 0, 'ondisk_log_size': 9442, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1224, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38650822, 'num_objects': 9438, 'num_object_clones': 0, 'num_object_copies': 18876, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4237, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9438, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 5941, 'num_write_kb': 23764, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5201}, {'shards': '0,2', 'objects': 4237}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:01:53.880 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-10T16:01:53.880 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "43'9026", 'reported_seq': 9161, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:01:49.668391+0000', 'last_change': '2024-09-10T15:59:16.175680+0000', 'last_active': '2024-09-10T16:01:49.668391+0000', 'last_peered': '2024-09-10T16:01:49.668391+0000', 'last_clean': '2024-09-10T15:59:14.088187+0000', 'last_became_active': '2024-09-10T15:59:16.165810+0000', 'last_became_peered': '2024-09-10T15:59:16.165810+0000', 'last_unstale': '2024-09-10T16:01:49.668391+0000', 'last_undegraded': '2024-09-10T15:59:15.822098+0000', 'last_fullsized': '2024-09-10T15:59:15.769123+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2218", 'last_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'objects_scrubbed': 2175, 'log_size': 9026, 'log_dups_size': 0, 'ondisk_log_size': 9026, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 825, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36942307, 'num_objects': 9020, 'num_object_clones': 0, 'num_object_copies': 18040, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4124, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9020, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 6845, 'num_write_kb': 27380, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4123}, {'shards': '2', 'objects': 4897}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:01:53.880 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-10T16:01:53.880 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "43'9366", 'reported_seq': 9492, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:01:52.164805+0000', 'last_change': '2024-09-10T15:59:16.277742+0000', 'last_active': '2024-09-10T16:01:52.164805+0000', 'last_peered': '2024-09-10T16:01:52.164805+0000', 'last_clean': '2024-09-10T15:59:14.129081+0000', 'last_became_active': '2024-09-10T15:59:16.269835+0000', 'last_became_peered': '2024-09-10T15:59:16.269835+0000', 'last_unstale': '2024-09-10T16:01:52.164805+0000', 'last_undegraded': '2024-09-10T15:59:15.860857+0000', 'last_fullsized': '2024-09-10T15:59:15.771410+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3557", 'last_scrub_stamp': '2024-09-10T15:59:02.493678+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:02.493678+0000', 'objects_scrubbed': 3488, 'log_size': 9366, 'log_dups_size': 0, 'ondisk_log_size': 9366, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1472, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38330368, 'num_objects': 9358, 'num_object_clones': 0, 'num_object_copies': 18716, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4257, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9358, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5870, 'num_write_kb': 23480, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5103}, {'shards': '0,2', 'objects': 4255}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:01:53.880 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-10T16:01:53.880 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "43'9252", 'reported_seq': 9382, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:01:49.723773+0000', 'last_change': '2024-09-10T15:59:16.284311+0000', 'last_active': '2024-09-10T16:01:49.723773+0000', 'last_peered': '2024-09-10T16:01:49.723773+0000', 'last_clean': '2024-09-10T15:59:14.116041+0000', 'last_became_active': '2024-09-10T15:59:16.274543+0000', 'last_became_peered': '2024-09-10T15:59:16.274543+0000', 'last_unstale': '2024-09-10T16:01:49.723773+0000', 'last_undegraded': '2024-09-10T15:59:15.812484+0000', 'last_fullsized': '2024-09-10T15:59:15.770885+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2356", 'last_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'objects_scrubbed': 2316, 'log_size': 9252, 'log_dups_size': 0, 'ondisk_log_size': 9252, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 783, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37876195, 'num_objects': 9248, 'num_object_clones': 0, 'num_object_copies': 18496, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9248, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 6932, 'num_write_kb': 27728, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4304}, {'shards': '2', 'objects': 4944}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:01:53.880 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-10T16:01:53.880 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "43'9402", 'reported_seq': 9526, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:01:49.619835+0000', 'last_change': '2024-09-10T15:59:16.174113+0000', 'last_active': '2024-09-10T16:01:49.619835+0000', 'last_peered': '2024-09-10T16:01:49.619835+0000', 'last_clean': '2024-09-10T15:59:14.110654+0000', 'last_became_active': '2024-09-10T15:59:16.164645+0000', 'last_became_peered': '2024-09-10T15:59:16.164645+0000', 'last_unstale': '2024-09-10T16:01:49.619835+0000', 'last_undegraded': '2024-09-10T15:59:15.803593+0000', 'last_fullsized': '2024-09-10T15:59:15.769083+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2486", 'last_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'objects_scrubbed': 2440, 'log_size': 9402, 'log_dups_size': 0, 'ondisk_log_size': 9402, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 792, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38486016, 'num_objects': 9396, 'num_object_clones': 0, 'num_object_copies': 18792, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4354, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9396, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6956, 'num_write_kb': 27824, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4352}, {'shards': '2', 'objects': 5044}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:01:53.881 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-10T16:01:53.881 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "43'9322", 'reported_seq': 9451, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:01:52.229195+0000', 'last_change': '2024-09-10T15:59:16.315641+0000', 'last_active': '2024-09-10T16:01:52.229195+0000', 'last_peered': '2024-09-10T16:01:52.229195+0000', 'last_clean': '2024-09-10T15:59:14.117001+0000', 'last_became_active': '2024-09-10T15:59:16.302245+0000', 'last_became_peered': '2024-09-10T15:59:16.302245+0000', 'last_unstale': '2024-09-10T16:01:52.229195+0000', 'last_undegraded': '2024-09-10T15:59:15.845012+0000', 'last_fullsized': '2024-09-10T15:59:15.770337+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3404", 'last_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'objects_scrubbed': 3367, 'log_size': 9322, 'log_dups_size': 0, 'ondisk_log_size': 9322, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 983, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38174720, 'num_objects': 9320, 'num_object_clones': 0, 'num_object_copies': 18640, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9320, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5953, 'num_write_kb': 23812, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5016}, {'shards': '0,2', 'objects': 4304}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:01:53.881 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-10T16:01:53.881 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "43'9178", 'reported_seq': 9313, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:01:52.187332+0000', 'last_change': '2024-09-10T15:59:16.251610+0000', 'last_active': '2024-09-10T16:01:52.187332+0000', 'last_peered': '2024-09-10T16:01:52.187332+0000', 'last_clean': '2024-09-10T15:59:14.115595+0000', 'last_became_active': '2024-09-10T15:59:16.244089+0000', 'last_became_peered': '2024-09-10T15:59:16.244089+0000', 'last_unstale': '2024-09-10T16:01:52.187332+0000', 'last_undegraded': '2024-09-10T15:59:15.847127+0000', 'last_fullsized': '2024-09-10T15:59:15.774437+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3653", 'last_scrub_stamp': '2024-09-10T15:59:05.298468+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:05.298468+0000', 'objects_scrubbed': 3602, 'log_size': 9178, 'log_dups_size': 0, 'ondisk_log_size': 9178, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1289, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37560320, 'num_objects': 9170, 'num_object_clones': 0, 'num_object_copies': 18340, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4206, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9170, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5568, 'num_write_kb': 22272, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4967}, {'shards': '0,2', 'objects': 4203}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:01:53.881 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-10T16:01:53.881 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "43'9489", 'reported_seq': 9624, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:01:52.130767+0000', 'last_change': '2024-09-10T15:59:16.311291+0000', 'last_active': '2024-09-10T16:01:52.130767+0000', 'last_peered': '2024-09-10T16:01:52.130767+0000', 'last_clean': '2024-09-10T15:59:14.116363+0000', 'last_became_active': '2024-09-10T15:59:16.301244+0000', 'last_became_peered': '2024-09-10T15:59:16.301244+0000', 'last_unstale': '2024-09-10T16:01:52.130767+0000', 'last_undegraded': '2024-09-10T15:59:15.883434+0000', 'last_fullsized': '2024-09-10T15:59:15.772756+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3789", 'last_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'objects_scrubbed': 3741, 'log_size': 9489, 'log_dups_size': 0, 'ondisk_log_size': 9489, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1440, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38850560, 'num_objects': 9485, 'num_object_clones': 0, 'num_object_copies': 18970, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4373, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9485, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5744, 'num_write_kb': 22976, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5114}, {'shards': '0,2', 'objects': 4371}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:01:53.881 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-10T16:01:53.881 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "43'9494", 'reported_seq': 9634, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:01:52.196955+0000', 'last_change': '2024-09-10T15:59:16.313409+0000', 'last_active': '2024-09-10T16:01:52.196955+0000', 'last_peered': '2024-09-10T16:01:52.196955+0000', 'last_clean': '2024-09-10T15:59:14.129780+0000', 'last_became_active': '2024-09-10T15:59:16.300932+0000', 'last_became_peered': '2024-09-10T15:59:16.300932+0000', 'last_unstale': '2024-09-10T16:01:52.196955+0000', 'last_undegraded': '2024-09-10T15:59:15.887823+0000', 'last_fullsized': '2024-09-10T15:59:15.768513+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3880", 'last_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'objects_scrubbed': 3827, 'log_size': 9494, 'log_dups_size': 0, 'ondisk_log_size': 9494, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1397, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38839238, 'num_objects': 9484, 'num_object_clones': 0, 'num_object_copies': 18968, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4325, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9484, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5657, 'num_write_kb': 22628, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5161}, {'shards': '0,2', 'objects': 4323}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:01:53.881 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T16:01:53.990 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 201 20 150437 150417 2.92278 1.53516 0.0369547 0.0267211 2024-09-10T16:01:54.579 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:01:54.579 INFO:teuthology.orchestra.run.smithi115.stdout:{"fsid":"7effee29-df05-4351-9741-ee203564ed81","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 47117/298510 objects degraded (15.784%), 11 pgs degraded, 12 pgs undersized","count":23},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":278,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":43,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983954,"num_in_osds":3,"osd_in_since":1725983947,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":149255,"data_bytes":611304177,"bytes_used":2013978624,"bytes_avail":287896313856,"bytes_total":289910292480,"degraded_objects":47117,"degraded_total":298510,"degraded_ratio":0.15784060835482899,"misplaced_objects":574,"misplaced_total":298510,"misplaced_ratio":0.0019228836554889283,"recovering_objects_per_sec":26,"recovering_bytes_per_sec":108846,"recovering_keys_per_sec":0,"num_objects_recovered":319,"num_bytes_recovered":1306624,"num_keys_recovered":0,"write_bytes_sec":2039423,"read_op_per_sec":0,"write_op_per_sec":497},"fsmap":{"epoch":1,"btime":"2024-09-10T15:57:14:967120+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T16:00:13.560776+0000","services":{}},"progress_events":{}} 2024-09-10T16:01:54.601 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-10T16:01:54.601 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:01:54.990 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 202 20 150845 150825 2.9162 1.59375 0.0816251 0.0267813 2024-09-10T16:01:55.132 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:01:55.134 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:01:55.151 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":175,"stamp":"2024-09-10T16:01:53.599974+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":611304177,"num_objects":149255,"num_object_clones":0,"num_object_copies":298510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":47117,"num_objects_misplaced":574,"num_objects_unfound":0,"num_objects_dirty":149255,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":107101,"num_write_kb":428398,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3700,"num_bytes_recovered":15142931,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":149328,"ondisk_log_size":149328,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1966776,"kb_used_data":1205384,"kb_used_omap":12,"kb_used_meta":761331,"kb_avail":281148744,"statfs":{"total":289910292480,"available":287896313856,"internally_reserved":0,"allocated":1234313216,"data_stored":1233428426,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":779602946},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,0,3,8,5,8,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":24481792,"num_objects":5977,"num_object_clones":0,"num_object_copies":11954,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-319,"num_objects_unfound":0,"num_objects_dirty":5977,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5977,"num_write_kb":23908,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":319,"num_bytes_recovered":1306624,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004269"},"pg_stats":[{"pgid":"2.f","version":"43'9280","reported_seq":9350,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:52.136350+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:01:52.136350+0000","last_peered":"2024-09-10T16:01:52.136350+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:01:52.136350+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":9280,"log_dups_size":0,"ondisk_log_size":9280,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38010880,"num_objects":9280,"num_object_clones":0,"num_object_copies":18560,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9280,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9280,"num_write_kb":37120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5021},{"shards":"0,2","objects":4258}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'9240","reported_seq":9348,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:01:49.699286+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:01:49.699286+0000","last_peered":"2024-09-10T16:01:49.699286+0000","last_clean":"2024-09-10T16:01:49.699286+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:01:49.699286+0000","last_undegraded":"2024-09-10T16:01:49.699286+0000","last_fullsized":"2024-09-10T16:01:49.699286+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":9240,"log_dups_size":0,"ondisk_log_size":9240,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37823430,"num_objects":9236,"num_object_clones":0,"num_object_copies":18472,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9236,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":7345,"num_write_kb":29380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'9235","reported_seq":16748,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T16:01:49.748161+0000","last_change":"2024-09-10T15:59:16.142612+0000","last_active":"2024-09-10T16:01:49.748161+0000","last_peered":"2024-09-10T16:01:49.748161+0000","last_clean":"2024-09-10T15:59:14.092473+0000","last_became_active":"2024-09-10T15:59:16.109611+0000","last_became_peered":"2024-09-10T15:59:16.109611+0000","last_unstale":"2024-09-10T16:01:49.748161+0000","last_undegraded":"2024-09-10T16:01:49.748161+0000","last_fullsized":"2024-09-10T15:59:15.770264+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":9235,"log_dups_size":0,"ondisk_log_size":9235,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37810176,"num_objects":9231,"num_object_clones":0,"num_object_copies":18462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":574,"num_objects_unfound":0,"num_objects_dirty":9231,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7239,"num_write_kb":28956,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3695,"num_bytes_recovered":15130624,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":574},{"shards":"2","objects":8657}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'9225","reported_seq":9301,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:01:49.742124+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:01:49.742124+0000","last_peered":"2024-09-10T16:01:49.742124+0000","last_clean":"2024-09-10T16:01:49.742124+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:01:49.742124+0000","last_undegraded":"2024-09-10T16:01:49.742124+0000","last_fullsized":"2024-09-10T16:01:49.742124+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":9225,"log_dups_size":0,"ondisk_log_size":9225,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37785600,"num_objects":9225,"num_object_clones":0,"num_object_copies":18450,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9225,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9225,"num_write_kb":36900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'9366","reported_seq":9491,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:01:52.247277+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:01:52.247277+0000","last_peered":"2024-09-10T16:01:52.247277+0000","last_clean":"2024-09-10T16:01:52.247277+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:01:52.247277+0000","last_undegraded":"2024-09-10T16:01:52.247277+0000","last_fullsized":"2024-09-10T16:01:52.247277+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":9366,"log_dups_size":0,"ondisk_log_size":9366,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38334947,"num_objects":9360,"num_object_clones":0,"num_object_copies":18720,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9360,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6077,"num_write_kb":24308,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'9437","reported_seq":9570,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:52.157258+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:01:52.157258+0000","last_peered":"2024-09-10T16:01:52.157258+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:01:52.157258+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":9437,"log_dups_size":0,"ondisk_log_size":9437,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38633955,"num_objects":9433,"num_object_clones":0,"num_object_copies":18866,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9433,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6049,"num_write_kb":24196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5062},{"shards":"0,2","objects":4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'9442","reported_seq":9575,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:52.187622+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:01:52.187622+0000","last_peered":"2024-09-10T16:01:52.187622+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:01:52.187622+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":9442,"log_dups_size":0,"ondisk_log_size":9442,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38650822,"num_objects":9438,"num_object_clones":0,"num_object_copies":18876,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9438,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5941,"num_write_kb":23764,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5201},{"shards":"0,2","objects":4237}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'9571","reported_seq":9695,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:01:52.229693+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:01:52.229693+0000","last_peered":"2024-09-10T16:01:52.229693+0000","last_clean":"2024-09-10T16:01:52.229693+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:01:52.229693+0000","last_undegraded":"2024-09-10T16:01:52.229693+0000","last_fullsized":"2024-09-10T16:01:52.229693+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":9571,"log_dups_size":0,"ondisk_log_size":9571,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39194624,"num_objects":9569,"num_object_clones":0,"num_object_copies":19138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9569,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6418,"num_write_kb":25672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.777326+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T15:59:15.777326+0000","last_peered":"2024-09-10T15:59:15.777326+0000","last_clean":"2024-09-10T15:59:15.777326+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T15:59:15.777326+0000","last_undegraded":"2024-09-10T15:59:15.777326+0000","last_fullsized":"2024-09-10T15:59:15.777326+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'9026","reported_seq":9161,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:49.668391+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:01:49.668391+0000","last_peered":"2024-09-10T16:01:49.668391+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:01:49.668391+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":9026,"log_dups_size":0,"ondisk_log_size":9026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36942307,"num_objects":9020,"num_object_clones":0,"num_object_copies":18040,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9020,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6845,"num_write_kb":27380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4123},{"shards":"2","objects":4897}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":89,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.772619+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T15:59:15.772619+0000","last_peered":"2024-09-10T15:59:15.772619+0000","last_clean":"2024-09-10T15:59:15.772619+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T15:59:15.772619+0000","last_undegraded":"2024-09-10T15:59:15.772619+0000","last_fullsized":"2024-09-10T15:59:15.772619+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'9366","reported_seq":9492,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:52.164805+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:01:52.164805+0000","last_peered":"2024-09-10T16:01:52.164805+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:01:52.164805+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":9366,"log_dups_size":0,"ondisk_log_size":9366,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38330368,"num_objects":9358,"num_object_clones":0,"num_object_copies":18716,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9358,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5870,"num_write_kb":23480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5103},{"shards":"0,2","objects":4255}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":99,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.884297+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T15:59:15.884297+0000","last_peered":"2024-09-10T15:59:15.884297+0000","last_clean":"2024-09-10T15:59:15.884297+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T15:59:15.884297+0000","last_undegraded":"2024-09-10T15:59:15.884297+0000","last_fullsized":"2024-09-10T15:59:15.884297+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'9252","reported_seq":9382,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:49.723773+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:01:49.723773+0000","last_peered":"2024-09-10T16:01:49.723773+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:01:49.723773+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":9252,"log_dups_size":0,"ondisk_log_size":9252,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37876195,"num_objects":9248,"num_object_clones":0,"num_object_copies":18496,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9248,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6932,"num_write_kb":27728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4304},{"shards":"2","objects":4944}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.848670+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T15:59:15.848670+0000","last_peered":"2024-09-10T15:59:15.848670+0000","last_clean":"2024-09-10T15:59:15.848670+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T15:59:15.848670+0000","last_undegraded":"2024-09-10T15:59:15.848670+0000","last_fullsized":"2024-09-10T15:59:15.848670+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'9402","reported_seq":9526,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:49.619835+0000","last_change":"2024-09-10T15:59:16.174113+0000","last_active":"2024-09-10T16:01:49.619835+0000","last_peered":"2024-09-10T16:01:49.619835+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:01:49.619835+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":9402,"log_dups_size":0,"ondisk_log_size":9402,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38486016,"num_objects":9396,"num_object_clones":0,"num_object_copies":18792,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4354,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9396,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6956,"num_write_kb":27824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4352},{"shards":"2","objects":5044}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'9322","reported_seq":9451,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:52.229195+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:01:52.229195+0000","last_peered":"2024-09-10T16:01:52.229195+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:01:52.229195+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":9322,"log_dups_size":0,"ondisk_log_size":9322,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38174720,"num_objects":9320,"num_object_clones":0,"num_object_copies":18640,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9320,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5953,"num_write_kb":23812,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5016},{"shards":"0,2","objects":4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.992406+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T15:59:15.992406+0000","last_peered":"2024-09-10T15:59:15.992406+0000","last_clean":"2024-09-10T15:59:15.992406+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T15:59:15.992406+0000","last_undegraded":"2024-09-10T15:59:15.992406+0000","last_fullsized":"2024-09-10T15:59:15.992406+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'9178","reported_seq":9313,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:52.187332+0000","last_change":"2024-09-10T15:59:16.251610+0000","last_active":"2024-09-10T16:01:52.187332+0000","last_peered":"2024-09-10T16:01:52.187332+0000","last_clean":"2024-09-10T15:59:14.115595+0000","last_became_active":"2024-09-10T15:59:16.244089+0000","last_became_peered":"2024-09-10T15:59:16.244089+0000","last_unstale":"2024-09-10T16:01:52.187332+0000","last_undegraded":"2024-09-10T15:59:15.847127+0000","last_fullsized":"2024-09-10T15:59:15.774437+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":9178,"log_dups_size":0,"ondisk_log_size":9178,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37560320,"num_objects":9170,"num_object_clones":0,"num_object_copies":18340,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9170,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5568,"num_write_kb":22272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4967},{"shards":"0,2","objects":4203}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":89,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.773633+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T15:59:15.773633+0000","last_peered":"2024-09-10T15:59:15.773633+0000","last_clean":"2024-09-10T15:59:15.773633+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T15:59:15.773633+0000","last_undegraded":"2024-09-10T15:59:15.773633+0000","last_fullsized":"2024-09-10T15:59:15.773633+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'9489","reported_seq":9624,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:52.130767+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:01:52.130767+0000","last_peered":"2024-09-10T16:01:52.130767+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:01:52.130767+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":9489,"log_dups_size":0,"ondisk_log_size":9489,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38850560,"num_objects":9485,"num_object_clones":0,"num_object_copies":18970,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9485,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5744,"num_write_kb":22976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5114},{"shards":"0,2","objects":4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.917634+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T15:59:15.917634+0000","last_peered":"2024-09-10T15:59:15.917634+0000","last_clean":"2024-09-10T15:59:15.917634+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T15:59:15.917634+0000","last_undegraded":"2024-09-10T15:59:15.917634+0000","last_fullsized":"2024-09-10T15:59:15.917634+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":70,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.778232+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T15:59:15.778232+0000","last_peered":"2024-09-10T15:59:15.778232+0000","last_clean":"2024-09-10T15:59:15.778232+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T15:59:15.778232+0000","last_undegraded":"2024-09-10T15:59:15.778232+0000","last_fullsized":"2024-09-10T15:59:15.778232+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'9494","reported_seq":9634,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:52.196955+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:01:52.196955+0000","last_peered":"2024-09-10T16:01:52.196955+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:01:52.196955+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":9494,"log_dups_size":0,"ondisk_log_size":9494,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38839238,"num_objects":9484,"num_object_clones":0,"num_object_copies":18968,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9484,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5657,"num_write_kb":22628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5161},{"shards":"0,2","objects":4323}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":611304158,"num_objects":149253,"num_object_clones":0,"num_object_copies":298506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":47117,"num_objects_misplaced":574,"num_objects_unfound":0,"num_objects_dirty":149253,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":107099,"num_write_kb":428396,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3698,"num_bytes_recovered":15142912,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1235308544,"data_stored":1235156798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":149325,"ondisk_log_size":149325,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986222,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":712468,"kb_used_data":433668,"kb_used_omap":4,"kb_used_meta":278779,"kb_avail":93659372,"statfs":{"total":96636764160,"available":95907196928,"internally_reserved":0,"allocated":444076032,"data_stored":443798272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":285470378},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,0,1,0,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659168,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":434680,"kb_used_data":254440,"kb_used_omap":4,"kb_used_meta":180219,"kb_avail":93937160,"statfs":{"total":96636764160,"available":96191651840,"internally_reserved":0,"allocated":260546560,"data_stored":260221367,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":184544942},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,3,2,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182451,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":819628,"kb_used_data":517276,"kb_used_omap":4,"kb_used_meta":302331,"kb_avail":93552212,"statfs":{"total":96636764160,"available":95797465088,"internally_reserved":0,"allocated":529690624,"data_stored":529408787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":309587626},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,2,4,3,2,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":530673664,"data_stored":530637534,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":260329472,"data_stored":260249986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":444305408,"data_stored":444269278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:01:55.990 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 203 20 151310 151290 2.91079 1.81641 0.0517768 0.0268321 2024-09-10T16:01:56.990 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 204 20 151783 151763 2.90557 1.84766 0.0561028 0.02688 2024-09-10T16:01:57.991 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 205 20 152261 152241 2.90051 1.86719 0.0103269 0.0269282 2024-09-10T16:01:58.154 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:01:58.747 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:01:58.748 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:01:58.773 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":177,"stamp":"2024-09-10T16:01:57.601406+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":620344049,"num_objects":151462,"num_object_clones":0,"num_object_copies":302924,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":47117,"num_objects_misplaced":416,"num_objects_unfound":0,"num_objects_dirty":151462,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":109308,"num_write_kb":437226,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3858,"num_bytes_recovered":15790099,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":151535,"ondisk_log_size":151535,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2004596,"kb_used_data":1225156,"kb_used_omap":12,"kb_used_meta":779379,"kb_avail":281110924,"statfs":{"total":289910292480,"available":287857586176,"internally_reserved":0,"allocated":1254559744,"data_stored":1253674954,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":798084098},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,5,11,6,14,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":24215552,"num_objects":5912,"num_object_clones":0,"num_object_copies":11824,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-327,"num_objects_unfound":0,"num_objects_dirty":5912,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5912,"num_write_kb":23648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":327,"num_bytes_recovered":1339392,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004365"},"pg_stats":[{"pgid":"2.f","version":"43'9410","reported_seq":9479,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:57.184035+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:01:57.184035+0000","last_peered":"2024-09-10T16:01:57.184035+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:01:57.184035+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":9410,"log_dups_size":0,"ondisk_log_size":9410,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38543360,"num_objects":9410,"num_object_clones":0,"num_object_copies":18820,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9410,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9410,"num_write_kb":37640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5151},{"shards":"0,2","objects":4258}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'9392","reported_seq":9500,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:01:54.656681+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:01:54.656681+0000","last_peered":"2024-09-10T16:01:54.656681+0000","last_clean":"2024-09-10T16:01:54.656681+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:01:54.656681+0000","last_undegraded":"2024-09-10T16:01:54.656681+0000","last_fullsized":"2024-09-10T16:01:54.656681+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":9392,"log_dups_size":0,"ondisk_log_size":9392,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38446022,"num_objects":9388,"num_object_clones":0,"num_object_copies":18776,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9388,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":7497,"num_write_kb":29988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'9384","reported_seq":17213,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T16:01:54.743145+0000","last_change":"2024-09-10T15:59:16.142612+0000","last_active":"2024-09-10T16:01:54.743145+0000","last_peered":"2024-09-10T16:01:54.743145+0000","last_clean":"2024-09-10T15:59:14.092473+0000","last_became_active":"2024-09-10T15:59:16.109611+0000","last_became_peered":"2024-09-10T15:59:16.109611+0000","last_unstale":"2024-09-10T16:01:54.743145+0000","last_undegraded":"2024-09-10T16:01:54.743145+0000","last_fullsized":"2024-09-10T15:59:15.770264+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":9384,"log_dups_size":0,"ondisk_log_size":9384,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38420480,"num_objects":9380,"num_object_clones":0,"num_object_copies":18760,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":416,"num_objects_unfound":0,"num_objects_dirty":9380,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7388,"num_write_kb":29552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3853,"num_bytes_recovered":15777792,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":416},{"shards":"2","objects":8964}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'9342","reported_seq":9418,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:01:54.732978+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:01:54.732978+0000","last_peered":"2024-09-10T16:01:54.732978+0000","last_clean":"2024-09-10T16:01:54.732978+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:01:54.732978+0000","last_undegraded":"2024-09-10T16:01:54.732978+0000","last_fullsized":"2024-09-10T16:01:54.732978+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":9342,"log_dups_size":0,"ondisk_log_size":9342,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38264832,"num_objects":9342,"num_object_clones":0,"num_object_copies":18684,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9342,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9342,"num_write_kb":37368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'9490","reported_seq":9615,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:01:57.224906+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:01:57.224906+0000","last_peered":"2024-09-10T16:01:57.224906+0000","last_clean":"2024-09-10T16:01:57.224906+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:01:57.224906+0000","last_undegraded":"2024-09-10T16:01:57.224906+0000","last_fullsized":"2024-09-10T16:01:57.224906+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":9490,"log_dups_size":0,"ondisk_log_size":9490,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38842851,"num_objects":9484,"num_object_clones":0,"num_object_copies":18968,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9484,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6201,"num_write_kb":24804,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'9582","reported_seq":9715,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:57.177833+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:01:57.177833+0000","last_peered":"2024-09-10T16:01:57.177833+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:01:57.177833+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":9582,"log_dups_size":0,"ondisk_log_size":9582,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39227875,"num_objects":9578,"num_object_clones":0,"num_object_copies":19156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9578,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6194,"num_write_kb":24776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5207},{"shards":"0,2","objects":4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'9584","reported_seq":9714,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:57.167906+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:01:57.167906+0000","last_peered":"2024-09-10T16:01:57.167906+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:01:57.167906+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":9584,"log_dups_size":0,"ondisk_log_size":9584,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39232454,"num_objects":9580,"num_object_clones":0,"num_object_copies":19160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9580,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":6083,"num_write_kb":24332,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5343},{"shards":"0,2","objects":4237}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'9702","reported_seq":9827,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:01:57.199968+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:01:57.199968+0000","last_peered":"2024-09-10T16:01:57.199968+0000","last_clean":"2024-09-10T16:01:57.199968+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:01:57.199968+0000","last_undegraded":"2024-09-10T16:01:57.199968+0000","last_fullsized":"2024-09-10T16:01:57.199968+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":9702,"log_dups_size":0,"ondisk_log_size":9702,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39731200,"num_objects":9700,"num_object_clones":0,"num_object_copies":19400,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9700,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6549,"num_write_kb":26196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.777326+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T15:59:15.777326+0000","last_peered":"2024-09-10T15:59:15.777326+0000","last_clean":"2024-09-10T15:59:15.777326+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T15:59:15.777326+0000","last_undegraded":"2024-09-10T15:59:15.777326+0000","last_fullsized":"2024-09-10T15:59:15.777326+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'9183","reported_seq":9318,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:54.636565+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:01:54.636565+0000","last_peered":"2024-09-10T16:01:54.636565+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:01:54.636565+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":9183,"log_dups_size":0,"ondisk_log_size":9183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37585379,"num_objects":9177,"num_object_clones":0,"num_object_copies":18354,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9177,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7002,"num_write_kb":28008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4123},{"shards":"2","objects":5054}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":89,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.772619+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T15:59:15.772619+0000","last_peered":"2024-09-10T15:59:15.772619+0000","last_clean":"2024-09-10T15:59:15.772619+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T15:59:15.772619+0000","last_undegraded":"2024-09-10T15:59:15.772619+0000","last_fullsized":"2024-09-10T15:59:15.772619+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'9508","reported_seq":9634,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:57.173342+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:01:57.173342+0000","last_peered":"2024-09-10T16:01:57.173342+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:01:57.173342+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":9508,"log_dups_size":0,"ondisk_log_size":9508,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38912000,"num_objects":9500,"num_object_clones":0,"num_object_copies":19000,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9500,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6012,"num_write_kb":24048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5245},{"shards":"0,2","objects":4255}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":99,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.884297+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T15:59:15.884297+0000","last_peered":"2024-09-10T15:59:15.884297+0000","last_clean":"2024-09-10T15:59:15.884297+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T15:59:15.884297+0000","last_undegraded":"2024-09-10T15:59:15.884297+0000","last_fullsized":"2024-09-10T15:59:15.884297+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'9374","reported_seq":9504,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:54.695710+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:01:54.695710+0000","last_peered":"2024-09-10T16:01:54.695710+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:01:54.695710+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":9374,"log_dups_size":0,"ondisk_log_size":9374,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38375907,"num_objects":9370,"num_object_clones":0,"num_object_copies":18740,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9370,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7054,"num_write_kb":28216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4304},{"shards":"2","objects":5066}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.848670+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T15:59:15.848670+0000","last_peered":"2024-09-10T15:59:15.848670+0000","last_clean":"2024-09-10T15:59:15.848670+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T15:59:15.848670+0000","last_undegraded":"2024-09-10T15:59:15.848670+0000","last_fullsized":"2024-09-10T15:59:15.848670+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'9568","reported_seq":9692,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:54.752515+0000","last_change":"2024-09-10T15:59:16.174113+0000","last_active":"2024-09-10T16:01:54.752515+0000","last_peered":"2024-09-10T16:01:54.752515+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:01:54.752515+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":9568,"log_dups_size":0,"ondisk_log_size":9568,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39165952,"num_objects":9562,"num_object_clones":0,"num_object_copies":19124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4354,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9562,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7122,"num_write_kb":28488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4352},{"shards":"2","objects":5210}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'9458","reported_seq":9588,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:57.121417+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:01:57.121417+0000","last_peered":"2024-09-10T16:01:57.121417+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:01:57.121417+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":9458,"log_dups_size":0,"ondisk_log_size":9458,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38731776,"num_objects":9456,"num_object_clones":0,"num_object_copies":18912,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9456,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6089,"num_write_kb":24356,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5152},{"shards":"0,2","objects":4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.992406+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T15:59:15.992406+0000","last_peered":"2024-09-10T15:59:15.992406+0000","last_clean":"2024-09-10T15:59:15.992406+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T15:59:15.992406+0000","last_undegraded":"2024-09-10T15:59:15.992406+0000","last_fullsized":"2024-09-10T15:59:15.992406+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'9315","reported_seq":9453,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:57.229676+0000","last_change":"2024-09-10T15:59:16.251610+0000","last_active":"2024-09-10T16:01:57.229676+0000","last_peered":"2024-09-10T16:01:57.229676+0000","last_clean":"2024-09-10T15:59:14.115595+0000","last_became_active":"2024-09-10T15:59:16.244089+0000","last_became_peered":"2024-09-10T15:59:16.244089+0000","last_unstale":"2024-09-10T16:01:57.229676+0000","last_undegraded":"2024-09-10T15:59:15.847127+0000","last_fullsized":"2024-09-10T15:59:15.774437+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":9315,"log_dups_size":0,"ondisk_log_size":9315,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38121472,"num_objects":9307,"num_object_clones":0,"num_object_copies":18614,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9307,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5705,"num_write_kb":22820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5104},{"shards":"0,2","objects":4203}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":89,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.773633+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T15:59:15.773633+0000","last_peered":"2024-09-10T15:59:15.773633+0000","last_clean":"2024-09-10T15:59:15.773633+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T15:59:15.773633+0000","last_undegraded":"2024-09-10T15:59:15.773633+0000","last_fullsized":"2024-09-10T15:59:15.773633+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'9624","reported_seq":9761,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:57.142374+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:01:57.142374+0000","last_peered":"2024-09-10T16:01:57.142374+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:01:57.142374+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":9624,"log_dups_size":0,"ondisk_log_size":9624,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39403520,"num_objects":9620,"num_object_clones":0,"num_object_copies":19240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9620,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5879,"num_write_kb":23516,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5249},{"shards":"0,2","objects":4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.917634+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T15:59:15.917634+0000","last_peered":"2024-09-10T15:59:15.917634+0000","last_clean":"2024-09-10T15:59:15.917634+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T15:59:15.917634+0000","last_undegraded":"2024-09-10T15:59:15.917634+0000","last_fullsized":"2024-09-10T15:59:15.917634+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":70,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.778232+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T15:59:15.778232+0000","last_peered":"2024-09-10T15:59:15.778232+0000","last_clean":"2024-09-10T15:59:15.778232+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T15:59:15.778232+0000","last_undegraded":"2024-09-10T15:59:15.778232+0000","last_fullsized":"2024-09-10T15:59:15.778232+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'9616","reported_seq":9756,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:57.223087+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:01:57.223087+0000","last_peered":"2024-09-10T16:01:57.223087+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:01:57.223087+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":9616,"log_dups_size":0,"ondisk_log_size":9616,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39338950,"num_objects":9606,"num_object_clones":0,"num_object_copies":19212,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9606,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5779,"num_write_kb":23116,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5283},{"shards":"0,2","objects":4323}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":620344030,"num_objects":151460,"num_object_clones":0,"num_object_copies":302920,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":47117,"num_objects_misplaced":416,"num_objects_unfound":0,"num_objects_dirty":151460,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":109306,"num_write_kb":437224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3856,"num_bytes_recovered":15790080,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1254240256,"data_stored":1254088510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":151532,"ondisk_log_size":151532,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986223,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":704552,"kb_used_data":438232,"kb_used_omap":4,"kb_used_meta":266299,"kb_avail":93667288,"statfs":{"total":96636764160,"available":95915302912,"internally_reserved":0,"allocated":448749568,"data_stored":448471808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":272690858},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,4,1,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659169,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":468464,"kb_used_data":261856,"kb_used_omap":4,"kb_used_meta":206587,"kb_avail":93903376,"statfs":{"total":96636764160,"available":96157057024,"internally_reserved":0,"allocated":268140544,"data_stored":267815351,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":211545774},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,4,4,5,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182452,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":831580,"kb_used_data":525068,"kb_used_omap":4,"kb_used_meta":306491,"kb_avail":93540260,"statfs":{"total":96636764160,"available":95785226240,"internally_reserved":0,"allocated":537669632,"data_stored":537387795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":313847466},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,1,3,1,6,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":537321472,"data_stored":537285342,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":268054528,"data_stored":267975042,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":448864256,"data_stored":448828126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:01:58.775 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:01:58.993 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 206 20 152682 152662 2.89437 1.64453 0.0508852 0.0269839 2024-09-10T16:01:59.336 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:01:59.337 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:01:59.360 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":177,"stamp":"2024-09-10T16:01:57.601406+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":620344049,"num_objects":151462,"num_object_clones":0,"num_object_copies":302924,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":47117,"num_objects_misplaced":416,"num_objects_unfound":0,"num_objects_dirty":151462,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":109308,"num_write_kb":437226,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3858,"num_bytes_recovered":15790099,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":151535,"ondisk_log_size":151535,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2004596,"kb_used_data":1225156,"kb_used_omap":12,"kb_used_meta":779379,"kb_avail":281110924,"statfs":{"total":289910292480,"available":287857586176,"internally_reserved":0,"allocated":1254559744,"data_stored":1253674954,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":798084098},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,5,11,6,14,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":24215552,"num_objects":5912,"num_object_clones":0,"num_object_copies":11824,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-327,"num_objects_unfound":0,"num_objects_dirty":5912,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5912,"num_write_kb":23648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":327,"num_bytes_recovered":1339392,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004365"},"pg_stats":[{"pgid":"2.f","version":"43'9410","reported_seq":9479,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:57.184035+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:01:57.184035+0000","last_peered":"2024-09-10T16:01:57.184035+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:01:57.184035+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":9410,"log_dups_size":0,"ondisk_log_size":9410,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38543360,"num_objects":9410,"num_object_clones":0,"num_object_copies":18820,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9410,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9410,"num_write_kb":37640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5151},{"shards":"0,2","objects":4258}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'9392","reported_seq":9500,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:01:54.656681+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:01:54.656681+0000","last_peered":"2024-09-10T16:01:54.656681+0000","last_clean":"2024-09-10T16:01:54.656681+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:01:54.656681+0000","last_undegraded":"2024-09-10T16:01:54.656681+0000","last_fullsized":"2024-09-10T16:01:54.656681+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":9392,"log_dups_size":0,"ondisk_log_size":9392,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38446022,"num_objects":9388,"num_object_clones":0,"num_object_copies":18776,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9388,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":7497,"num_write_kb":29988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'9384","reported_seq":17213,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T16:01:54.743145+0000","last_change":"2024-09-10T15:59:16.142612+0000","last_active":"2024-09-10T16:01:54.743145+0000","last_peered":"2024-09-10T16:01:54.743145+0000","last_clean":"2024-09-10T15:59:14.092473+0000","last_became_active":"2024-09-10T15:59:16.109611+0000","last_became_peered":"2024-09-10T15:59:16.109611+0000","last_unstale":"2024-09-10T16:01:54.743145+0000","last_undegraded":"2024-09-10T16:01:54.743145+0000","last_fullsized":"2024-09-10T15:59:15.770264+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":9384,"log_dups_size":0,"ondisk_log_size":9384,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38420480,"num_objects":9380,"num_object_clones":0,"num_object_copies":18760,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":416,"num_objects_unfound":0,"num_objects_dirty":9380,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7388,"num_write_kb":29552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3853,"num_bytes_recovered":15777792,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":416},{"shards":"2","objects":8964}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'9342","reported_seq":9418,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:01:54.732978+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:01:54.732978+0000","last_peered":"2024-09-10T16:01:54.732978+0000","last_clean":"2024-09-10T16:01:54.732978+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:01:54.732978+0000","last_undegraded":"2024-09-10T16:01:54.732978+0000","last_fullsized":"2024-09-10T16:01:54.732978+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":9342,"log_dups_size":0,"ondisk_log_size":9342,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38264832,"num_objects":9342,"num_object_clones":0,"num_object_copies":18684,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9342,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9342,"num_write_kb":37368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'9490","reported_seq":9615,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:01:57.224906+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:01:57.224906+0000","last_peered":"2024-09-10T16:01:57.224906+0000","last_clean":"2024-09-10T16:01:57.224906+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:01:57.224906+0000","last_undegraded":"2024-09-10T16:01:57.224906+0000","last_fullsized":"2024-09-10T16:01:57.224906+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":9490,"log_dups_size":0,"ondisk_log_size":9490,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38842851,"num_objects":9484,"num_object_clones":0,"num_object_copies":18968,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9484,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6201,"num_write_kb":24804,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'9582","reported_seq":9715,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:57.177833+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:01:57.177833+0000","last_peered":"2024-09-10T16:01:57.177833+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:01:57.177833+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":9582,"log_dups_size":0,"ondisk_log_size":9582,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39227875,"num_objects":9578,"num_object_clones":0,"num_object_copies":19156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9578,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6194,"num_write_kb":24776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5207},{"shards":"0,2","objects":4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'9584","reported_seq":9714,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:57.167906+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:01:57.167906+0000","last_peered":"2024-09-10T16:01:57.167906+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:01:57.167906+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":9584,"log_dups_size":0,"ondisk_log_size":9584,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39232454,"num_objects":9580,"num_object_clones":0,"num_object_copies":19160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9580,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":6083,"num_write_kb":24332,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5343},{"shards":"0,2","objects":4237}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'9702","reported_seq":9827,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:01:57.199968+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:01:57.199968+0000","last_peered":"2024-09-10T16:01:57.199968+0000","last_clean":"2024-09-10T16:01:57.199968+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:01:57.199968+0000","last_undegraded":"2024-09-10T16:01:57.199968+0000","last_fullsized":"2024-09-10T16:01:57.199968+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":9702,"log_dups_size":0,"ondisk_log_size":9702,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39731200,"num_objects":9700,"num_object_clones":0,"num_object_copies":19400,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9700,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6549,"num_write_kb":26196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.777326+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T15:59:15.777326+0000","last_peered":"2024-09-10T15:59:15.777326+0000","last_clean":"2024-09-10T15:59:15.777326+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T15:59:15.777326+0000","last_undegraded":"2024-09-10T15:59:15.777326+0000","last_fullsized":"2024-09-10T15:59:15.777326+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'9183","reported_seq":9318,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:54.636565+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:01:54.636565+0000","last_peered":"2024-09-10T16:01:54.636565+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:01:54.636565+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":9183,"log_dups_size":0,"ondisk_log_size":9183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37585379,"num_objects":9177,"num_object_clones":0,"num_object_copies":18354,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9177,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7002,"num_write_kb":28008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4123},{"shards":"2","objects":5054}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":89,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.772619+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T15:59:15.772619+0000","last_peered":"2024-09-10T15:59:15.772619+0000","last_clean":"2024-09-10T15:59:15.772619+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T15:59:15.772619+0000","last_undegraded":"2024-09-10T15:59:15.772619+0000","last_fullsized":"2024-09-10T15:59:15.772619+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'9508","reported_seq":9634,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:57.173342+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:01:57.173342+0000","last_peered":"2024-09-10T16:01:57.173342+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:01:57.173342+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":9508,"log_dups_size":0,"ondisk_log_size":9508,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38912000,"num_objects":9500,"num_object_clones":0,"num_object_copies":19000,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9500,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6012,"num_write_kb":24048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5245},{"shards":"0,2","objects":4255}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":99,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.884297+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T15:59:15.884297+0000","last_peered":"2024-09-10T15:59:15.884297+0000","last_clean":"2024-09-10T15:59:15.884297+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T15:59:15.884297+0000","last_undegraded":"2024-09-10T15:59:15.884297+0000","last_fullsized":"2024-09-10T15:59:15.884297+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'9374","reported_seq":9504,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:54.695710+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:01:54.695710+0000","last_peered":"2024-09-10T16:01:54.695710+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:01:54.695710+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":9374,"log_dups_size":0,"ondisk_log_size":9374,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38375907,"num_objects":9370,"num_object_clones":0,"num_object_copies":18740,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9370,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7054,"num_write_kb":28216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4304},{"shards":"2","objects":5066}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.848670+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T15:59:15.848670+0000","last_peered":"2024-09-10T15:59:15.848670+0000","last_clean":"2024-09-10T15:59:15.848670+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T15:59:15.848670+0000","last_undegraded":"2024-09-10T15:59:15.848670+0000","last_fullsized":"2024-09-10T15:59:15.848670+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'9568","reported_seq":9692,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:54.752515+0000","last_change":"2024-09-10T15:59:16.174113+0000","last_active":"2024-09-10T16:01:54.752515+0000","last_peered":"2024-09-10T16:01:54.752515+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:01:54.752515+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":9568,"log_dups_size":0,"ondisk_log_size":9568,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39165952,"num_objects":9562,"num_object_clones":0,"num_object_copies":19124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4354,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9562,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7122,"num_write_kb":28488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4352},{"shards":"2","objects":5210}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'9458","reported_seq":9588,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:57.121417+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:01:57.121417+0000","last_peered":"2024-09-10T16:01:57.121417+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:01:57.121417+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":9458,"log_dups_size":0,"ondisk_log_size":9458,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38731776,"num_objects":9456,"num_object_clones":0,"num_object_copies":18912,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9456,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6089,"num_write_kb":24356,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5152},{"shards":"0,2","objects":4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.992406+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T15:59:15.992406+0000","last_peered":"2024-09-10T15:59:15.992406+0000","last_clean":"2024-09-10T15:59:15.992406+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T15:59:15.992406+0000","last_undegraded":"2024-09-10T15:59:15.992406+0000","last_fullsized":"2024-09-10T15:59:15.992406+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'9315","reported_seq":9453,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:57.229676+0000","last_change":"2024-09-10T15:59:16.251610+0000","last_active":"2024-09-10T16:01:57.229676+0000","last_peered":"2024-09-10T16:01:57.229676+0000","last_clean":"2024-09-10T15:59:14.115595+0000","last_became_active":"2024-09-10T15:59:16.244089+0000","last_became_peered":"2024-09-10T15:59:16.244089+0000","last_unstale":"2024-09-10T16:01:57.229676+0000","last_undegraded":"2024-09-10T15:59:15.847127+0000","last_fullsized":"2024-09-10T15:59:15.774437+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":9315,"log_dups_size":0,"ondisk_log_size":9315,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38121472,"num_objects":9307,"num_object_clones":0,"num_object_copies":18614,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9307,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5705,"num_write_kb":22820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5104},{"shards":"0,2","objects":4203}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":89,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.773633+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T15:59:15.773633+0000","last_peered":"2024-09-10T15:59:15.773633+0000","last_clean":"2024-09-10T15:59:15.773633+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T15:59:15.773633+0000","last_undegraded":"2024-09-10T15:59:15.773633+0000","last_fullsized":"2024-09-10T15:59:15.773633+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'9624","reported_seq":9761,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:57.142374+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:01:57.142374+0000","last_peered":"2024-09-10T16:01:57.142374+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:01:57.142374+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":9624,"log_dups_size":0,"ondisk_log_size":9624,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39403520,"num_objects":9620,"num_object_clones":0,"num_object_copies":19240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9620,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5879,"num_write_kb":23516,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5249},{"shards":"0,2","objects":4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.917634+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T15:59:15.917634+0000","last_peered":"2024-09-10T15:59:15.917634+0000","last_clean":"2024-09-10T15:59:15.917634+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T15:59:15.917634+0000","last_undegraded":"2024-09-10T15:59:15.917634+0000","last_fullsized":"2024-09-10T15:59:15.917634+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":70,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.778232+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T15:59:15.778232+0000","last_peered":"2024-09-10T15:59:15.778232+0000","last_clean":"2024-09-10T15:59:15.778232+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T15:59:15.778232+0000","last_undegraded":"2024-09-10T15:59:15.778232+0000","last_fullsized":"2024-09-10T15:59:15.778232+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'9616","reported_seq":9756,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:57.223087+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:01:57.223087+0000","last_peered":"2024-09-10T16:01:57.223087+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:01:57.223087+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":9616,"log_dups_size":0,"ondisk_log_size":9616,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39338950,"num_objects":9606,"num_object_clones":0,"num_object_copies":19212,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9606,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5779,"num_write_kb":23116,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5283},{"shards":"0,2","objects":4323}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":620344030,"num_objects":151460,"num_object_clones":0,"num_object_copies":302920,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":47117,"num_objects_misplaced":416,"num_objects_unfound":0,"num_objects_dirty":151460,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":109306,"num_write_kb":437224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3856,"num_bytes_recovered":15790080,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1254240256,"data_stored":1254088510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":151532,"ondisk_log_size":151532,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986223,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":704552,"kb_used_data":438232,"kb_used_omap":4,"kb_used_meta":266299,"kb_avail":93667288,"statfs":{"total":96636764160,"available":95915302912,"internally_reserved":0,"allocated":448749568,"data_stored":448471808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":272690858},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,4,1,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659169,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":468464,"kb_used_data":261856,"kb_used_omap":4,"kb_used_meta":206587,"kb_avail":93903376,"statfs":{"total":96636764160,"available":96157057024,"internally_reserved":0,"allocated":268140544,"data_stored":267815351,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":211545774},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,4,4,5,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182452,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":831580,"kb_used_data":525068,"kb_used_omap":4,"kb_used_meta":306491,"kb_avail":93540260,"statfs":{"total":96636764160,"available":95785226240,"internally_reserved":0,"allocated":537669632,"data_stored":537387795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":313847466},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,1,3,1,6,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":537321472,"data_stored":537285342,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":268054528,"data_stored":267975042,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":448864256,"data_stored":448828126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:01:59.362 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-10T16:01:59.362 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "43'9410", 'reported_seq': 9479, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:01:57.184035+0000', 'last_change': '2024-09-10T15:59:16.281558+0000', 'last_active': '2024-09-10T16:01:57.184035+0000', 'last_peered': '2024-09-10T16:01:57.184035+0000', 'last_clean': '2024-09-10T15:59:14.132456+0000', 'last_became_active': '2024-09-10T15:59:16.274407+0000', 'last_became_peered': '2024-09-10T15:59:16.274407+0000', 'last_unstale': '2024-09-10T16:01:57.184035+0000', 'last_undegraded': '2024-09-10T15:59:15.851279+0000', 'last_fullsized': '2024-09-10T15:59:15.774356+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'objects_scrubbed': 0, 'log_size': 9410, 'log_dups_size': 0, 'ondisk_log_size': 9410, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38543360, 'num_objects': 9410, 'num_object_clones': 0, 'num_object_copies': 18820, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4259, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9410, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9410, 'num_write_kb': 37640, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5151}, {'shards': '0,2', 'objects': 4258}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:01:59.362 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-10T16:01:59.362 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "43'9384", 'reported_seq': 17213, 'reported_epoch': 43, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-10T16:01:54.743145+0000', 'last_change': '2024-09-10T15:59:16.142612+0000', 'last_active': '2024-09-10T16:01:54.743145+0000', 'last_peered': '2024-09-10T16:01:54.743145+0000', 'last_clean': '2024-09-10T15:59:14.092473+0000', 'last_became_active': '2024-09-10T15:59:16.109611+0000', 'last_became_peered': '2024-09-10T15:59:16.109611+0000', 'last_unstale': '2024-09-10T16:01:54.743145+0000', 'last_undegraded': '2024-09-10T16:01:54.743145+0000', 'last_fullsized': '2024-09-10T15:59:15.770264+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2029", 'last_scrub_stamp': '2024-09-10T15:58:45.933789+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:45.933789+0000', 'objects_scrubbed': 1992, 'log_size': 9384, 'log_dups_size': 0, 'ondisk_log_size': 9384, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 668, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38420480, 'num_objects': 9380, 'num_object_clones': 0, 'num_object_copies': 18760, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 416, 'num_objects_unfound': 0, 'num_objects_dirty': 9380, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7388, 'num_write_kb': 29552, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3853, 'num_bytes_recovered': 15777792, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 416}, {'shards': '2', 'objects': 8964}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:01:59.362 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-10T16:01:59.362 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "43'9582", 'reported_seq': 9715, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:01:57.177833+0000', 'last_change': '2024-09-10T15:59:16.308264+0000', 'last_active': '2024-09-10T16:01:57.177833+0000', 'last_peered': '2024-09-10T16:01:57.177833+0000', 'last_clean': '2024-09-10T15:59:14.111551+0000', 'last_became_active': '2024-09-10T15:59:16.300040+0000', 'last_became_peered': '2024-09-10T15:59:16.300040+0000', 'last_unstale': '2024-09-10T16:01:57.177833+0000', 'last_undegraded': '2024-09-10T15:59:15.878234+0000', 'last_fullsized': '2024-09-10T15:59:15.769328+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3442", 'last_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'objects_scrubbed': 3384, 'log_size': 9582, 'log_dups_size': 0, 'ondisk_log_size': 9582, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1292, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39227875, 'num_objects': 9578, 'num_object_clones': 0, 'num_object_copies': 19156, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4372, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9578, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 6194, 'num_write_kb': 24776, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5207}, {'shards': '0,2', 'objects': 4371}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:01:59.363 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-10T16:01:59.363 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "43'9584", 'reported_seq': 9714, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:01:57.167906+0000', 'last_change': '2024-09-10T15:59:16.331899+0000', 'last_active': '2024-09-10T16:01:57.167906+0000', 'last_peered': '2024-09-10T16:01:57.167906+0000', 'last_clean': '2024-09-10T15:59:14.139436+0000', 'last_became_active': '2024-09-10T15:59:16.318715+0000', 'last_became_peered': '2024-09-10T15:59:16.318715+0000', 'last_unstale': '2024-09-10T16:01:57.167906+0000', 'last_undegraded': '2024-09-10T15:59:15.877524+0000', 'last_fullsized': '2024-09-10T15:59:15.769042+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3537", 'last_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'objects_scrubbed': 3497, 'log_size': 9584, 'log_dups_size': 0, 'ondisk_log_size': 9584, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1224, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39232454, 'num_objects': 9580, 'num_object_clones': 0, 'num_object_copies': 19160, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4237, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9580, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 6083, 'num_write_kb': 24332, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5343}, {'shards': '0,2', 'objects': 4237}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:01:59.363 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-10T16:01:59.363 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "43'9183", 'reported_seq': 9318, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:01:54.636565+0000', 'last_change': '2024-09-10T15:59:16.175680+0000', 'last_active': '2024-09-10T16:01:54.636565+0000', 'last_peered': '2024-09-10T16:01:54.636565+0000', 'last_clean': '2024-09-10T15:59:14.088187+0000', 'last_became_active': '2024-09-10T15:59:16.165810+0000', 'last_became_peered': '2024-09-10T15:59:16.165810+0000', 'last_unstale': '2024-09-10T16:01:54.636565+0000', 'last_undegraded': '2024-09-10T15:59:15.822098+0000', 'last_fullsized': '2024-09-10T15:59:15.769123+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2218", 'last_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'objects_scrubbed': 2175, 'log_size': 9183, 'log_dups_size': 0, 'ondisk_log_size': 9183, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 825, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37585379, 'num_objects': 9177, 'num_object_clones': 0, 'num_object_copies': 18354, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4124, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9177, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 7002, 'num_write_kb': 28008, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4123}, {'shards': '2', 'objects': 5054}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:01:59.363 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-10T16:01:59.363 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "43'9508", 'reported_seq': 9634, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:01:57.173342+0000', 'last_change': '2024-09-10T15:59:16.277742+0000', 'last_active': '2024-09-10T16:01:57.173342+0000', 'last_peered': '2024-09-10T16:01:57.173342+0000', 'last_clean': '2024-09-10T15:59:14.129081+0000', 'last_became_active': '2024-09-10T15:59:16.269835+0000', 'last_became_peered': '2024-09-10T15:59:16.269835+0000', 'last_unstale': '2024-09-10T16:01:57.173342+0000', 'last_undegraded': '2024-09-10T15:59:15.860857+0000', 'last_fullsized': '2024-09-10T15:59:15.771410+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3557", 'last_scrub_stamp': '2024-09-10T15:59:02.493678+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:02.493678+0000', 'objects_scrubbed': 3488, 'log_size': 9508, 'log_dups_size': 0, 'ondisk_log_size': 9508, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1472, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38912000, 'num_objects': 9500, 'num_object_clones': 0, 'num_object_copies': 19000, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4257, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9500, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6012, 'num_write_kb': 24048, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5245}, {'shards': '0,2', 'objects': 4255}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:01:59.363 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-10T16:01:59.363 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "43'9374", 'reported_seq': 9504, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:01:54.695710+0000', 'last_change': '2024-09-10T15:59:16.284311+0000', 'last_active': '2024-09-10T16:01:54.695710+0000', 'last_peered': '2024-09-10T16:01:54.695710+0000', 'last_clean': '2024-09-10T15:59:14.116041+0000', 'last_became_active': '2024-09-10T15:59:16.274543+0000', 'last_became_peered': '2024-09-10T15:59:16.274543+0000', 'last_unstale': '2024-09-10T16:01:54.695710+0000', 'last_undegraded': '2024-09-10T15:59:15.812484+0000', 'last_fullsized': '2024-09-10T15:59:15.770885+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2356", 'last_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'objects_scrubbed': 2316, 'log_size': 9374, 'log_dups_size': 0, 'ondisk_log_size': 9374, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 783, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38375907, 'num_objects': 9370, 'num_object_clones': 0, 'num_object_copies': 18740, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9370, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 7054, 'num_write_kb': 28216, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4304}, {'shards': '2', 'objects': 5066}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:01:59.363 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-10T16:01:59.363 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "43'9568", 'reported_seq': 9692, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:01:54.752515+0000', 'last_change': '2024-09-10T15:59:16.174113+0000', 'last_active': '2024-09-10T16:01:54.752515+0000', 'last_peered': '2024-09-10T16:01:54.752515+0000', 'last_clean': '2024-09-10T15:59:14.110654+0000', 'last_became_active': '2024-09-10T15:59:16.164645+0000', 'last_became_peered': '2024-09-10T15:59:16.164645+0000', 'last_unstale': '2024-09-10T16:01:54.752515+0000', 'last_undegraded': '2024-09-10T15:59:15.803593+0000', 'last_fullsized': '2024-09-10T15:59:15.769083+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2486", 'last_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'objects_scrubbed': 2440, 'log_size': 9568, 'log_dups_size': 0, 'ondisk_log_size': 9568, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 792, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39165952, 'num_objects': 9562, 'num_object_clones': 0, 'num_object_copies': 19124, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4354, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9562, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7122, 'num_write_kb': 28488, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4352}, {'shards': '2', 'objects': 5210}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:01:59.364 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-10T16:01:59.364 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "43'9458", 'reported_seq': 9588, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:01:57.121417+0000', 'last_change': '2024-09-10T15:59:16.315641+0000', 'last_active': '2024-09-10T16:01:57.121417+0000', 'last_peered': '2024-09-10T16:01:57.121417+0000', 'last_clean': '2024-09-10T15:59:14.117001+0000', 'last_became_active': '2024-09-10T15:59:16.302245+0000', 'last_became_peered': '2024-09-10T15:59:16.302245+0000', 'last_unstale': '2024-09-10T16:01:57.121417+0000', 'last_undegraded': '2024-09-10T15:59:15.845012+0000', 'last_fullsized': '2024-09-10T15:59:15.770337+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3404", 'last_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'objects_scrubbed': 3367, 'log_size': 9458, 'log_dups_size': 0, 'ondisk_log_size': 9458, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 983, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38731776, 'num_objects': 9456, 'num_object_clones': 0, 'num_object_copies': 18912, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9456, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6089, 'num_write_kb': 24356, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5152}, {'shards': '0,2', 'objects': 4304}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:01:59.364 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-10T16:01:59.364 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "43'9315", 'reported_seq': 9453, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:01:57.229676+0000', 'last_change': '2024-09-10T15:59:16.251610+0000', 'last_active': '2024-09-10T16:01:57.229676+0000', 'last_peered': '2024-09-10T16:01:57.229676+0000', 'last_clean': '2024-09-10T15:59:14.115595+0000', 'last_became_active': '2024-09-10T15:59:16.244089+0000', 'last_became_peered': '2024-09-10T15:59:16.244089+0000', 'last_unstale': '2024-09-10T16:01:57.229676+0000', 'last_undegraded': '2024-09-10T15:59:15.847127+0000', 'last_fullsized': '2024-09-10T15:59:15.774437+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3653", 'last_scrub_stamp': '2024-09-10T15:59:05.298468+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:05.298468+0000', 'objects_scrubbed': 3602, 'log_size': 9315, 'log_dups_size': 0, 'ondisk_log_size': 9315, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1289, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38121472, 'num_objects': 9307, 'num_object_clones': 0, 'num_object_copies': 18614, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4206, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9307, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5705, 'num_write_kb': 22820, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5104}, {'shards': '0,2', 'objects': 4203}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:01:59.364 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-10T16:01:59.364 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "43'9624", 'reported_seq': 9761, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:01:57.142374+0000', 'last_change': '2024-09-10T15:59:16.311291+0000', 'last_active': '2024-09-10T16:01:57.142374+0000', 'last_peered': '2024-09-10T16:01:57.142374+0000', 'last_clean': '2024-09-10T15:59:14.116363+0000', 'last_became_active': '2024-09-10T15:59:16.301244+0000', 'last_became_peered': '2024-09-10T15:59:16.301244+0000', 'last_unstale': '2024-09-10T16:01:57.142374+0000', 'last_undegraded': '2024-09-10T15:59:15.883434+0000', 'last_fullsized': '2024-09-10T15:59:15.772756+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3789", 'last_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'objects_scrubbed': 3741, 'log_size': 9624, 'log_dups_size': 0, 'ondisk_log_size': 9624, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1440, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39403520, 'num_objects': 9620, 'num_object_clones': 0, 'num_object_copies': 19240, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4373, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9620, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5879, 'num_write_kb': 23516, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5249}, {'shards': '0,2', 'objects': 4371}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:01:59.364 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-10T16:01:59.364 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "43'9616", 'reported_seq': 9756, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:01:57.223087+0000', 'last_change': '2024-09-10T15:59:16.313409+0000', 'last_active': '2024-09-10T16:01:57.223087+0000', 'last_peered': '2024-09-10T16:01:57.223087+0000', 'last_clean': '2024-09-10T15:59:14.129780+0000', 'last_became_active': '2024-09-10T15:59:16.300932+0000', 'last_became_peered': '2024-09-10T15:59:16.300932+0000', 'last_unstale': '2024-09-10T16:01:57.223087+0000', 'last_undegraded': '2024-09-10T15:59:15.887823+0000', 'last_fullsized': '2024-09-10T15:59:15.768513+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3880", 'last_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'objects_scrubbed': 3827, 'log_size': 9616, 'log_dups_size': 0, 'ondisk_log_size': 9616, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1397, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39338950, 'num_objects': 9606, 'num_object_clones': 0, 'num_object_copies': 19212, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4325, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9606, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5779, 'num_write_kb': 23116, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5283}, {'shards': '0,2', 'objects': 4323}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:01:59.365 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T16:01:59.994 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 207 20 153090 153070 2.88809 1.59375 0.0843206 0.0270418 2024-09-10T16:02:00.067 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:02:00.067 INFO:teuthology.orchestra.run.smithi115.stdout:{"fsid":"7effee29-df05-4351-9741-ee203564ed81","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 47117/302924 objects degraded (15.554%), 11 pgs degraded, 12 pgs undersized","count":23},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":284,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":43,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983954,"num_in_osds":3,"osd_in_since":1725983947,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":151462,"data_bytes":620344049,"bytes_used":2052706304,"bytes_avail":287857586176,"bytes_total":289910292480,"degraded_objects":47117,"degraded_total":302924,"degraded_ratio":0.15554066366481362,"misplaced_objects":416,"misplaced_total":302924,"misplaced_ratio":0.0013732817472369308,"recovering_objects_per_sec":27,"recovering_bytes_per_sec":111576,"recovering_keys_per_sec":0,"num_objects_recovered":327,"num_bytes_recovered":1339392,"num_keys_recovered":0,"write_bytes_sec":1526241,"read_op_per_sec":0,"write_op_per_sec":372},"fsmap":{"epoch":1,"btime":"2024-09-10T15:57:14:967120+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T16:00:13.560776+0000","services":{}},"progress_events":{}} 2024-09-10T16:02:00.094 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-10T16:02:00.095 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:02:00.640 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:02:00.641 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:02:00.667 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":178,"stamp":"2024-09-10T16:01:59.601982+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":620344049,"num_objects":151462,"num_object_clones":0,"num_object_copies":302924,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":47117,"num_objects_misplaced":416,"num_objects_unfound":0,"num_objects_dirty":151462,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":109308,"num_write_kb":437226,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3858,"num_bytes_recovered":15790099,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":151535,"ondisk_log_size":151535,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2004596,"kb_used_data":1225156,"kb_used_omap":12,"kb_used_meta":779379,"kb_avail":281110924,"statfs":{"total":289910292480,"available":287857586176,"internally_reserved":0,"allocated":1254559744,"data_stored":1253674954,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":798084098},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,5,11,6,14,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":18321408,"num_objects":4473,"num_object_clones":0,"num_object_copies":8946,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-327,"num_objects_unfound":0,"num_objects_dirty":4473,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4473,"num_write_kb":17892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":327,"num_bytes_recovered":1339392,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004268"},"pg_stats":[{"pgid":"2.f","version":"43'9410","reported_seq":9479,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:57.184035+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:01:57.184035+0000","last_peered":"2024-09-10T16:01:57.184035+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:01:57.184035+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":9410,"log_dups_size":0,"ondisk_log_size":9410,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38543360,"num_objects":9410,"num_object_clones":0,"num_object_copies":18820,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9410,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9410,"num_write_kb":37640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5151},{"shards":"0,2","objects":4258}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'9392","reported_seq":9500,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:01:54.656681+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:01:54.656681+0000","last_peered":"2024-09-10T16:01:54.656681+0000","last_clean":"2024-09-10T16:01:54.656681+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:01:54.656681+0000","last_undegraded":"2024-09-10T16:01:54.656681+0000","last_fullsized":"2024-09-10T16:01:54.656681+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":9392,"log_dups_size":0,"ondisk_log_size":9392,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38446022,"num_objects":9388,"num_object_clones":0,"num_object_copies":18776,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9388,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":7497,"num_write_kb":29988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'9384","reported_seq":17213,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T16:01:54.743145+0000","last_change":"2024-09-10T15:59:16.142612+0000","last_active":"2024-09-10T16:01:54.743145+0000","last_peered":"2024-09-10T16:01:54.743145+0000","last_clean":"2024-09-10T15:59:14.092473+0000","last_became_active":"2024-09-10T15:59:16.109611+0000","last_became_peered":"2024-09-10T15:59:16.109611+0000","last_unstale":"2024-09-10T16:01:54.743145+0000","last_undegraded":"2024-09-10T16:01:54.743145+0000","last_fullsized":"2024-09-10T15:59:15.770264+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":9384,"log_dups_size":0,"ondisk_log_size":9384,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38420480,"num_objects":9380,"num_object_clones":0,"num_object_copies":18760,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":416,"num_objects_unfound":0,"num_objects_dirty":9380,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7388,"num_write_kb":29552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3853,"num_bytes_recovered":15777792,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":416},{"shards":"2","objects":8964}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'9342","reported_seq":9418,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:01:54.732978+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:01:54.732978+0000","last_peered":"2024-09-10T16:01:54.732978+0000","last_clean":"2024-09-10T16:01:54.732978+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:01:54.732978+0000","last_undegraded":"2024-09-10T16:01:54.732978+0000","last_fullsized":"2024-09-10T16:01:54.732978+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":9342,"log_dups_size":0,"ondisk_log_size":9342,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38264832,"num_objects":9342,"num_object_clones":0,"num_object_copies":18684,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9342,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9342,"num_write_kb":37368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'9490","reported_seq":9615,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:01:57.224906+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:01:57.224906+0000","last_peered":"2024-09-10T16:01:57.224906+0000","last_clean":"2024-09-10T16:01:57.224906+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:01:57.224906+0000","last_undegraded":"2024-09-10T16:01:57.224906+0000","last_fullsized":"2024-09-10T16:01:57.224906+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":9490,"log_dups_size":0,"ondisk_log_size":9490,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38842851,"num_objects":9484,"num_object_clones":0,"num_object_copies":18968,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9484,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6201,"num_write_kb":24804,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'9582","reported_seq":9715,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:57.177833+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:01:57.177833+0000","last_peered":"2024-09-10T16:01:57.177833+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:01:57.177833+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":9582,"log_dups_size":0,"ondisk_log_size":9582,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39227875,"num_objects":9578,"num_object_clones":0,"num_object_copies":19156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9578,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6194,"num_write_kb":24776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5207},{"shards":"0,2","objects":4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'9584","reported_seq":9714,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:57.167906+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:01:57.167906+0000","last_peered":"2024-09-10T16:01:57.167906+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:01:57.167906+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":9584,"log_dups_size":0,"ondisk_log_size":9584,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39232454,"num_objects":9580,"num_object_clones":0,"num_object_copies":19160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9580,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":6083,"num_write_kb":24332,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5343},{"shards":"0,2","objects":4237}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'9702","reported_seq":9827,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:01:57.199968+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:01:57.199968+0000","last_peered":"2024-09-10T16:01:57.199968+0000","last_clean":"2024-09-10T16:01:57.199968+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:01:57.199968+0000","last_undegraded":"2024-09-10T16:01:57.199968+0000","last_fullsized":"2024-09-10T16:01:57.199968+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":9702,"log_dups_size":0,"ondisk_log_size":9702,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39731200,"num_objects":9700,"num_object_clones":0,"num_object_copies":19400,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9700,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6549,"num_write_kb":26196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.777326+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T15:59:15.777326+0000","last_peered":"2024-09-10T15:59:15.777326+0000","last_clean":"2024-09-10T15:59:15.777326+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T15:59:15.777326+0000","last_undegraded":"2024-09-10T15:59:15.777326+0000","last_fullsized":"2024-09-10T15:59:15.777326+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'9183","reported_seq":9318,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:54.636565+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:01:54.636565+0000","last_peered":"2024-09-10T16:01:54.636565+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:01:54.636565+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":9183,"log_dups_size":0,"ondisk_log_size":9183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37585379,"num_objects":9177,"num_object_clones":0,"num_object_copies":18354,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9177,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7002,"num_write_kb":28008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4123},{"shards":"2","objects":5054}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":89,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.772619+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T15:59:15.772619+0000","last_peered":"2024-09-10T15:59:15.772619+0000","last_clean":"2024-09-10T15:59:15.772619+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T15:59:15.772619+0000","last_undegraded":"2024-09-10T15:59:15.772619+0000","last_fullsized":"2024-09-10T15:59:15.772619+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'9508","reported_seq":9634,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:57.173342+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:01:57.173342+0000","last_peered":"2024-09-10T16:01:57.173342+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:01:57.173342+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":9508,"log_dups_size":0,"ondisk_log_size":9508,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38912000,"num_objects":9500,"num_object_clones":0,"num_object_copies":19000,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9500,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6012,"num_write_kb":24048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5245},{"shards":"0,2","objects":4255}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":99,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.884297+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T15:59:15.884297+0000","last_peered":"2024-09-10T15:59:15.884297+0000","last_clean":"2024-09-10T15:59:15.884297+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T15:59:15.884297+0000","last_undegraded":"2024-09-10T15:59:15.884297+0000","last_fullsized":"2024-09-10T15:59:15.884297+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'9374","reported_seq":9504,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:54.695710+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:01:54.695710+0000","last_peered":"2024-09-10T16:01:54.695710+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:01:54.695710+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":9374,"log_dups_size":0,"ondisk_log_size":9374,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38375907,"num_objects":9370,"num_object_clones":0,"num_object_copies":18740,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9370,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7054,"num_write_kb":28216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4304},{"shards":"2","objects":5066}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.848670+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T15:59:15.848670+0000","last_peered":"2024-09-10T15:59:15.848670+0000","last_clean":"2024-09-10T15:59:15.848670+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T15:59:15.848670+0000","last_undegraded":"2024-09-10T15:59:15.848670+0000","last_fullsized":"2024-09-10T15:59:15.848670+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'9568","reported_seq":9692,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:54.752515+0000","last_change":"2024-09-10T15:59:16.174113+0000","last_active":"2024-09-10T16:01:54.752515+0000","last_peered":"2024-09-10T16:01:54.752515+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:01:54.752515+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":9568,"log_dups_size":0,"ondisk_log_size":9568,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39165952,"num_objects":9562,"num_object_clones":0,"num_object_copies":19124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4354,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9562,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7122,"num_write_kb":28488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4352},{"shards":"2","objects":5210}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'9458","reported_seq":9588,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:57.121417+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:01:57.121417+0000","last_peered":"2024-09-10T16:01:57.121417+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:01:57.121417+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":9458,"log_dups_size":0,"ondisk_log_size":9458,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38731776,"num_objects":9456,"num_object_clones":0,"num_object_copies":18912,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9456,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6089,"num_write_kb":24356,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5152},{"shards":"0,2","objects":4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.992406+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T15:59:15.992406+0000","last_peered":"2024-09-10T15:59:15.992406+0000","last_clean":"2024-09-10T15:59:15.992406+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T15:59:15.992406+0000","last_undegraded":"2024-09-10T15:59:15.992406+0000","last_fullsized":"2024-09-10T15:59:15.992406+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'9315","reported_seq":9453,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:57.229676+0000","last_change":"2024-09-10T15:59:16.251610+0000","last_active":"2024-09-10T16:01:57.229676+0000","last_peered":"2024-09-10T16:01:57.229676+0000","last_clean":"2024-09-10T15:59:14.115595+0000","last_became_active":"2024-09-10T15:59:16.244089+0000","last_became_peered":"2024-09-10T15:59:16.244089+0000","last_unstale":"2024-09-10T16:01:57.229676+0000","last_undegraded":"2024-09-10T15:59:15.847127+0000","last_fullsized":"2024-09-10T15:59:15.774437+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":9315,"log_dups_size":0,"ondisk_log_size":9315,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38121472,"num_objects":9307,"num_object_clones":0,"num_object_copies":18614,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9307,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5705,"num_write_kb":22820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5104},{"shards":"0,2","objects":4203}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":89,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.773633+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T15:59:15.773633+0000","last_peered":"2024-09-10T15:59:15.773633+0000","last_clean":"2024-09-10T15:59:15.773633+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T15:59:15.773633+0000","last_undegraded":"2024-09-10T15:59:15.773633+0000","last_fullsized":"2024-09-10T15:59:15.773633+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'9624","reported_seq":9761,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:57.142374+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:01:57.142374+0000","last_peered":"2024-09-10T16:01:57.142374+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:01:57.142374+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":9624,"log_dups_size":0,"ondisk_log_size":9624,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39403520,"num_objects":9620,"num_object_clones":0,"num_object_copies":19240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9620,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5879,"num_write_kb":23516,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5249},{"shards":"0,2","objects":4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.917634+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T15:59:15.917634+0000","last_peered":"2024-09-10T15:59:15.917634+0000","last_clean":"2024-09-10T15:59:15.917634+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T15:59:15.917634+0000","last_undegraded":"2024-09-10T15:59:15.917634+0000","last_fullsized":"2024-09-10T15:59:15.917634+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":70,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.778232+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T15:59:15.778232+0000","last_peered":"2024-09-10T15:59:15.778232+0000","last_clean":"2024-09-10T15:59:15.778232+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T15:59:15.778232+0000","last_undegraded":"2024-09-10T15:59:15.778232+0000","last_fullsized":"2024-09-10T15:59:15.778232+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'9616","reported_seq":9756,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:57.223087+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:01:57.223087+0000","last_peered":"2024-09-10T16:01:57.223087+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:01:57.223087+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":9616,"log_dups_size":0,"ondisk_log_size":9616,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39338950,"num_objects":9606,"num_object_clones":0,"num_object_copies":19212,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9606,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5779,"num_write_kb":23116,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5283},{"shards":"0,2","objects":4323}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":620344030,"num_objects":151460,"num_object_clones":0,"num_object_copies":302920,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":47117,"num_objects_misplaced":416,"num_objects_unfound":0,"num_objects_dirty":151460,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":109306,"num_write_kb":437224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3856,"num_bytes_recovered":15790080,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1254240256,"data_stored":1254088510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":151532,"ondisk_log_size":151532,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986223,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":704552,"kb_used_data":438232,"kb_used_omap":4,"kb_used_meta":266299,"kb_avail":93667288,"statfs":{"total":96636764160,"available":95915302912,"internally_reserved":0,"allocated":448749568,"data_stored":448471808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":272690858},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,4,1,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659169,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":468464,"kb_used_data":261856,"kb_used_omap":4,"kb_used_meta":206587,"kb_avail":93903376,"statfs":{"total":96636764160,"available":96157057024,"internally_reserved":0,"allocated":268140544,"data_stored":267815351,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":211545774},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,4,4,5,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182452,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":831580,"kb_used_data":525068,"kb_used_omap":4,"kb_used_meta":306491,"kb_avail":93540260,"statfs":{"total":96636764160,"available":95785226240,"internally_reserved":0,"allocated":537669632,"data_stored":537387795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":313847466},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,1,3,1,6,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":537321472,"data_stored":537285342,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":268054528,"data_stored":267975042,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":448864256,"data_stored":448828126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:02:00.994 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 208 19 153521 153502 2.88231 1.6875 0.0434603 0.0270979 2024-09-10T16:02:01.994 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 209 20 153988 153968 2.87723 1.82031 0.0377512 0.0271442 2024-09-10T16:02:02.994 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 210 20 154466 154446 2.87242 1.86719 0.0240076 0.0271916 2024-09-10T16:02:03.669 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:02:03.994 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 211 20 154931 154911 2.86742 1.81641 0.020061 0.0272398 2024-09-10T16:02:04.280 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:02:04.281 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:02:04.306 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":180,"stamp":"2024-09-10T16:02:03.603523+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":629641969,"num_objects":153732,"num_object_clones":0,"num_object_copies":307464,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":47117,"num_objects_misplaced":234,"num_objects_unfound":0,"num_objects_dirty":153732,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":111578,"num_write_kb":446306,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4040,"num_bytes_recovered":16535571,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":153805,"ondisk_log_size":153805,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2037164,"kb_used_data":1243836,"kb_used_omap":12,"kb_used_meta":793267,"kb_avail":281078356,"statfs":{"total":289910292480,"available":287824236544,"internally_reserved":0,"allocated":1273688064,"data_stored":1272803274,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":812305410},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,4,5,7,2,7,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":24137728,"num_objects":5893,"num_object_clones":0,"num_object_copies":11786,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-340,"num_objects_unfound":0,"num_objects_dirty":5893,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5893,"num_write_kb":23572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":340,"num_bytes_recovered":1392640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004175"},"pg_stats":[{"pgid":"2.f","version":"43'9538","reported_seq":9606,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:02.263581+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:02:02.263581+0000","last_peered":"2024-09-10T16:02:02.263581+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:02:02.263581+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":9538,"log_dups_size":0,"ondisk_log_size":9538,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39067648,"num_objects":9538,"num_object_clones":0,"num_object_copies":19076,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9538,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9538,"num_write_kb":38152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5279},{"shards":"0,2","objects":4258}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'9537","reported_seq":9645,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:01:59.743509+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:01:59.743509+0000","last_peered":"2024-09-10T16:01:59.743509+0000","last_clean":"2024-09-10T16:01:59.743509+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:01:59.743509+0000","last_undegraded":"2024-09-10T16:01:59.743509+0000","last_fullsized":"2024-09-10T16:01:59.743509+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":9537,"log_dups_size":0,"ondisk_log_size":9537,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39039942,"num_objects":9533,"num_object_clones":0,"num_object_copies":19066,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9533,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":7642,"num_write_kb":30568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'9518","reported_seq":17714,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T16:01:59.763825+0000","last_change":"2024-09-10T15:59:16.142612+0000","last_active":"2024-09-10T16:01:59.763825+0000","last_peered":"2024-09-10T16:01:59.763825+0000","last_clean":"2024-09-10T15:59:14.092473+0000","last_became_active":"2024-09-10T15:59:16.109611+0000","last_became_peered":"2024-09-10T15:59:16.109611+0000","last_unstale":"2024-09-10T16:01:59.763825+0000","last_undegraded":"2024-09-10T16:01:59.763825+0000","last_fullsized":"2024-09-10T15:59:15.770264+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":9518,"log_dups_size":0,"ondisk_log_size":9518,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38969344,"num_objects":9514,"num_object_clones":0,"num_object_copies":19028,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":234,"num_objects_unfound":0,"num_objects_dirty":9514,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7522,"num_write_kb":30088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4035,"num_bytes_recovered":16523264,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":234},{"shards":"2","objects":9280}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'9477","reported_seq":9553,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:01:59.736604+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:01:59.736604+0000","last_peered":"2024-09-10T16:01:59.736604+0000","last_clean":"2024-09-10T16:01:59.736604+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:01:59.736604+0000","last_undegraded":"2024-09-10T16:01:59.736604+0000","last_fullsized":"2024-09-10T16:01:59.736604+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":9477,"log_dups_size":0,"ondisk_log_size":9477,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38817792,"num_objects":9477,"num_object_clones":0,"num_object_copies":18954,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9477,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9477,"num_write_kb":37908,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'9635","reported_seq":9760,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:02:02.251441+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:02:02.251441+0000","last_peered":"2024-09-10T16:02:02.251441+0000","last_clean":"2024-09-10T16:02:02.251441+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:02:02.251441+0000","last_undegraded":"2024-09-10T16:02:02.251441+0000","last_fullsized":"2024-09-10T16:02:02.251441+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":9635,"log_dups_size":0,"ondisk_log_size":9635,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39436771,"num_objects":9629,"num_object_clones":0,"num_object_copies":19258,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9629,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6346,"num_write_kb":25384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'9740","reported_seq":9873,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:02.316562+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:02:02.316562+0000","last_peered":"2024-09-10T16:02:02.316562+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:02:02.316562+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":9740,"log_dups_size":0,"ondisk_log_size":9740,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39875043,"num_objects":9736,"num_object_clones":0,"num_object_copies":19472,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9736,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6352,"num_write_kb":25408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5365},{"shards":"0,2","objects":4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'9730","reported_seq":9863,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:02.190694+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:02:02.190694+0000","last_peered":"2024-09-10T16:02:02.190694+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:02:02.190694+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":9730,"log_dups_size":0,"ondisk_log_size":9730,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39830470,"num_objects":9726,"num_object_clones":0,"num_object_copies":19452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9726,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":6229,"num_write_kb":24916,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5489},{"shards":"0,2","objects":4237}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'9838","reported_seq":9963,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:02:02.143892+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:02:02.143892+0000","last_peered":"2024-09-10T16:02:02.143892+0000","last_clean":"2024-09-10T16:02:02.143892+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:02:02.143892+0000","last_undegraded":"2024-09-10T16:02:02.143892+0000","last_fullsized":"2024-09-10T16:02:02.143892+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":9838,"log_dups_size":0,"ondisk_log_size":9838,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40288256,"num_objects":9836,"num_object_clones":0,"num_object_copies":19672,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9836,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6685,"num_write_kb":26740,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.777326+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T15:59:15.777326+0000","last_peered":"2024-09-10T15:59:15.777326+0000","last_clean":"2024-09-10T15:59:15.777326+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T15:59:15.777326+0000","last_undegraded":"2024-09-10T15:59:15.777326+0000","last_fullsized":"2024-09-10T15:59:15.777326+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'9338","reported_seq":9470,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:59.733354+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:01:59.733354+0000","last_peered":"2024-09-10T16:01:59.733354+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:01:59.733354+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":9338,"log_dups_size":0,"ondisk_log_size":9338,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38220259,"num_objects":9332,"num_object_clones":0,"num_object_copies":18664,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9332,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7157,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4123},{"shards":"2","objects":5209}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":89,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.772619+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T15:59:15.772619+0000","last_peered":"2024-09-10T15:59:15.772619+0000","last_clean":"2024-09-10T15:59:15.772619+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T15:59:15.772619+0000","last_undegraded":"2024-09-10T15:59:15.772619+0000","last_fullsized":"2024-09-10T15:59:15.772619+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'9658","reported_seq":9784,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:02.257337+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:02:02.257337+0000","last_peered":"2024-09-10T16:02:02.257337+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:02:02.257337+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":9658,"log_dups_size":0,"ondisk_log_size":9658,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39526400,"num_objects":9650,"num_object_clones":0,"num_object_copies":19300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9650,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6162,"num_write_kb":24648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5395},{"shards":"0,2","objects":4255}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":99,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.884297+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T15:59:15.884297+0000","last_peered":"2024-09-10T15:59:15.884297+0000","last_clean":"2024-09-10T15:59:15.884297+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T15:59:15.884297+0000","last_undegraded":"2024-09-10T15:59:15.884297+0000","last_fullsized":"2024-09-10T15:59:15.884297+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'9506","reported_seq":9636,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:59.749553+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:01:59.749553+0000","last_peered":"2024-09-10T16:01:59.749553+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:01:59.749553+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":9506,"log_dups_size":0,"ondisk_log_size":9506,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38916579,"num_objects":9502,"num_object_clones":0,"num_object_copies":19004,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9502,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7186,"num_write_kb":28744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4304},{"shards":"2","objects":5198}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.848670+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T15:59:15.848670+0000","last_peered":"2024-09-10T15:59:15.848670+0000","last_clean":"2024-09-10T15:59:15.848670+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T15:59:15.848670+0000","last_undegraded":"2024-09-10T15:59:15.848670+0000","last_fullsized":"2024-09-10T15:59:15.848670+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'9714","reported_seq":9838,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:59.649308+0000","last_change":"2024-09-10T15:59:16.174113+0000","last_active":"2024-09-10T16:01:59.649308+0000","last_peered":"2024-09-10T16:01:59.649308+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:01:59.649308+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":9714,"log_dups_size":0,"ondisk_log_size":9714,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39763968,"num_objects":9708,"num_object_clones":0,"num_object_copies":19416,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4354,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9708,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7268,"num_write_kb":29072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4352},{"shards":"2","objects":5356}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'9595","reported_seq":9725,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:02.253142+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:02:02.253142+0000","last_peered":"2024-09-10T16:02:02.253142+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:02:02.253142+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":9595,"log_dups_size":0,"ondisk_log_size":9595,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39292928,"num_objects":9593,"num_object_clones":0,"num_object_copies":19186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9593,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6226,"num_write_kb":24904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5289},{"shards":"0,2","objects":4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.992406+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T15:59:15.992406+0000","last_peered":"2024-09-10T15:59:15.992406+0000","last_clean":"2024-09-10T15:59:15.992406+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T15:59:15.992406+0000","last_undegraded":"2024-09-10T15:59:15.992406+0000","last_fullsized":"2024-09-10T15:59:15.992406+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'9466","reported_seq":9604,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:02.275964+0000","last_change":"2024-09-10T15:59:16.251610+0000","last_active":"2024-09-10T16:02:02.275964+0000","last_peered":"2024-09-10T16:02:02.275964+0000","last_clean":"2024-09-10T15:59:14.115595+0000","last_became_active":"2024-09-10T15:59:16.244089+0000","last_became_peered":"2024-09-10T15:59:16.244089+0000","last_unstale":"2024-09-10T16:02:02.275964+0000","last_undegraded":"2024-09-10T15:59:15.847127+0000","last_fullsized":"2024-09-10T15:59:15.774437+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":9466,"log_dups_size":0,"ondisk_log_size":9466,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38739968,"num_objects":9458,"num_object_clones":0,"num_object_copies":18916,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9458,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5856,"num_write_kb":23424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5255},{"shards":"0,2","objects":4203}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":89,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.773633+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T15:59:15.773633+0000","last_peered":"2024-09-10T15:59:15.773633+0000","last_clean":"2024-09-10T15:59:15.773633+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T15:59:15.773633+0000","last_undegraded":"2024-09-10T15:59:15.773633+0000","last_fullsized":"2024-09-10T15:59:15.773633+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'9751","reported_seq":9888,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:02.231259+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:02:02.231259+0000","last_peered":"2024-09-10T16:02:02.231259+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:02:02.231259+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":9751,"log_dups_size":0,"ondisk_log_size":9751,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39923712,"num_objects":9747,"num_object_clones":0,"num_object_copies":19494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9747,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6006,"num_write_kb":24024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5376},{"shards":"0,2","objects":4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.917634+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T15:59:15.917634+0000","last_peered":"2024-09-10T15:59:15.917634+0000","last_clean":"2024-09-10T15:59:15.917634+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T15:59:15.917634+0000","last_undegraded":"2024-09-10T15:59:15.917634+0000","last_fullsized":"2024-09-10T15:59:15.917634+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":70,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.778232+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T15:59:15.778232+0000","last_peered":"2024-09-10T15:59:15.778232+0000","last_clean":"2024-09-10T15:59:15.778232+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T15:59:15.778232+0000","last_undegraded":"2024-09-10T15:59:15.778232+0000","last_fullsized":"2024-09-10T15:59:15.778232+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'9761","reported_seq":9900,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:02.238272+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:02:02.238272+0000","last_peered":"2024-09-10T16:02:02.238272+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:02:02.238272+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":9761,"log_dups_size":0,"ondisk_log_size":9761,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39932870,"num_objects":9751,"num_object_clones":0,"num_object_copies":19502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9751,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5924,"num_write_kb":23696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5428},{"shards":"0,2","objects":4323}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":629641950,"num_objects":153730,"num_object_clones":0,"num_object_copies":307460,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":47117,"num_objects_misplaced":234,"num_objects_unfound":0,"num_objects_dirty":153730,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":111576,"num_write_kb":446304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4038,"num_bytes_recovered":16535552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1273524224,"data_stored":1273372478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":153802,"ondisk_log_size":153802,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986224,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":713224,"kb_used_data":442744,"kb_used_omap":4,"kb_used_meta":270459,"kb_avail":93658616,"statfs":{"total":96636764160,"available":95906422784,"internally_reserved":0,"allocated":453369856,"data_stored":453092096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":276950698},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659170,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":463580,"kb_used_data":269452,"kb_used_omap":4,"kb_used_meta":194107,"kb_avail":93908260,"statfs":{"total":96636764160,"available":96162058240,"internally_reserved":0,"allocated":275918848,"data_stored":275593655,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":198766254},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,4,3,1,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182453,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":860360,"kb_used_data":531640,"kb_used_omap":4,"kb_used_meta":328699,"kb_avail":93511480,"statfs":{"total":96636764160,"available":95755755520,"internally_reserved":0,"allocated":544399360,"data_stored":544117523,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":336588458},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,1,3,1,6,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":544215040,"data_stored":544178910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":275816448,"data_stored":275736962,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":453492736,"data_stored":453456606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:02:04.308 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:02:04.854 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:02:04.854 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:02:04.887 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":180,"stamp":"2024-09-10T16:02:03.603523+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":629641969,"num_objects":153732,"num_object_clones":0,"num_object_copies":307464,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":47117,"num_objects_misplaced":234,"num_objects_unfound":0,"num_objects_dirty":153732,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":111578,"num_write_kb":446306,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4040,"num_bytes_recovered":16535571,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":153805,"ondisk_log_size":153805,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2037164,"kb_used_data":1243836,"kb_used_omap":12,"kb_used_meta":793267,"kb_avail":281078356,"statfs":{"total":289910292480,"available":287824236544,"internally_reserved":0,"allocated":1273688064,"data_stored":1272803274,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":812305410},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,4,5,7,2,7,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":24137728,"num_objects":5893,"num_object_clones":0,"num_object_copies":11786,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-340,"num_objects_unfound":0,"num_objects_dirty":5893,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5893,"num_write_kb":23572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":340,"num_bytes_recovered":1392640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004175"},"pg_stats":[{"pgid":"2.f","version":"43'9538","reported_seq":9606,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:02.263581+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:02:02.263581+0000","last_peered":"2024-09-10T16:02:02.263581+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:02:02.263581+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":9538,"log_dups_size":0,"ondisk_log_size":9538,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39067648,"num_objects":9538,"num_object_clones":0,"num_object_copies":19076,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9538,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9538,"num_write_kb":38152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5279},{"shards":"0,2","objects":4258}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'9537","reported_seq":9645,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:01:59.743509+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:01:59.743509+0000","last_peered":"2024-09-10T16:01:59.743509+0000","last_clean":"2024-09-10T16:01:59.743509+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:01:59.743509+0000","last_undegraded":"2024-09-10T16:01:59.743509+0000","last_fullsized":"2024-09-10T16:01:59.743509+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":9537,"log_dups_size":0,"ondisk_log_size":9537,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39039942,"num_objects":9533,"num_object_clones":0,"num_object_copies":19066,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9533,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":7642,"num_write_kb":30568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'9518","reported_seq":17714,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T16:01:59.763825+0000","last_change":"2024-09-10T15:59:16.142612+0000","last_active":"2024-09-10T16:01:59.763825+0000","last_peered":"2024-09-10T16:01:59.763825+0000","last_clean":"2024-09-10T15:59:14.092473+0000","last_became_active":"2024-09-10T15:59:16.109611+0000","last_became_peered":"2024-09-10T15:59:16.109611+0000","last_unstale":"2024-09-10T16:01:59.763825+0000","last_undegraded":"2024-09-10T16:01:59.763825+0000","last_fullsized":"2024-09-10T15:59:15.770264+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":9518,"log_dups_size":0,"ondisk_log_size":9518,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38969344,"num_objects":9514,"num_object_clones":0,"num_object_copies":19028,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":234,"num_objects_unfound":0,"num_objects_dirty":9514,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7522,"num_write_kb":30088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4035,"num_bytes_recovered":16523264,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":234},{"shards":"2","objects":9280}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'9477","reported_seq":9553,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:01:59.736604+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:01:59.736604+0000","last_peered":"2024-09-10T16:01:59.736604+0000","last_clean":"2024-09-10T16:01:59.736604+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:01:59.736604+0000","last_undegraded":"2024-09-10T16:01:59.736604+0000","last_fullsized":"2024-09-10T16:01:59.736604+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":9477,"log_dups_size":0,"ondisk_log_size":9477,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38817792,"num_objects":9477,"num_object_clones":0,"num_object_copies":18954,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9477,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9477,"num_write_kb":37908,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'9635","reported_seq":9760,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:02:02.251441+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:02:02.251441+0000","last_peered":"2024-09-10T16:02:02.251441+0000","last_clean":"2024-09-10T16:02:02.251441+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:02:02.251441+0000","last_undegraded":"2024-09-10T16:02:02.251441+0000","last_fullsized":"2024-09-10T16:02:02.251441+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":9635,"log_dups_size":0,"ondisk_log_size":9635,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39436771,"num_objects":9629,"num_object_clones":0,"num_object_copies":19258,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9629,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6346,"num_write_kb":25384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'9740","reported_seq":9873,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:02.316562+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:02:02.316562+0000","last_peered":"2024-09-10T16:02:02.316562+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:02:02.316562+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":9740,"log_dups_size":0,"ondisk_log_size":9740,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39875043,"num_objects":9736,"num_object_clones":0,"num_object_copies":19472,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9736,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6352,"num_write_kb":25408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5365},{"shards":"0,2","objects":4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'9730","reported_seq":9863,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:02.190694+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:02:02.190694+0000","last_peered":"2024-09-10T16:02:02.190694+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:02:02.190694+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":9730,"log_dups_size":0,"ondisk_log_size":9730,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39830470,"num_objects":9726,"num_object_clones":0,"num_object_copies":19452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9726,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":6229,"num_write_kb":24916,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5489},{"shards":"0,2","objects":4237}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'9838","reported_seq":9963,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:02:02.143892+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:02:02.143892+0000","last_peered":"2024-09-10T16:02:02.143892+0000","last_clean":"2024-09-10T16:02:02.143892+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:02:02.143892+0000","last_undegraded":"2024-09-10T16:02:02.143892+0000","last_fullsized":"2024-09-10T16:02:02.143892+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":9838,"log_dups_size":0,"ondisk_log_size":9838,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40288256,"num_objects":9836,"num_object_clones":0,"num_object_copies":19672,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9836,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6685,"num_write_kb":26740,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.777326+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T15:59:15.777326+0000","last_peered":"2024-09-10T15:59:15.777326+0000","last_clean":"2024-09-10T15:59:15.777326+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T15:59:15.777326+0000","last_undegraded":"2024-09-10T15:59:15.777326+0000","last_fullsized":"2024-09-10T15:59:15.777326+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'9338","reported_seq":9470,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:59.733354+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:01:59.733354+0000","last_peered":"2024-09-10T16:01:59.733354+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:01:59.733354+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":9338,"log_dups_size":0,"ondisk_log_size":9338,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38220259,"num_objects":9332,"num_object_clones":0,"num_object_copies":18664,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9332,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7157,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4123},{"shards":"2","objects":5209}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":89,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.772619+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T15:59:15.772619+0000","last_peered":"2024-09-10T15:59:15.772619+0000","last_clean":"2024-09-10T15:59:15.772619+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T15:59:15.772619+0000","last_undegraded":"2024-09-10T15:59:15.772619+0000","last_fullsized":"2024-09-10T15:59:15.772619+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'9658","reported_seq":9784,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:02.257337+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:02:02.257337+0000","last_peered":"2024-09-10T16:02:02.257337+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:02:02.257337+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":9658,"log_dups_size":0,"ondisk_log_size":9658,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39526400,"num_objects":9650,"num_object_clones":0,"num_object_copies":19300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9650,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6162,"num_write_kb":24648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5395},{"shards":"0,2","objects":4255}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":99,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.884297+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T15:59:15.884297+0000","last_peered":"2024-09-10T15:59:15.884297+0000","last_clean":"2024-09-10T15:59:15.884297+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T15:59:15.884297+0000","last_undegraded":"2024-09-10T15:59:15.884297+0000","last_fullsized":"2024-09-10T15:59:15.884297+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'9506","reported_seq":9636,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:59.749553+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:01:59.749553+0000","last_peered":"2024-09-10T16:01:59.749553+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:01:59.749553+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":9506,"log_dups_size":0,"ondisk_log_size":9506,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38916579,"num_objects":9502,"num_object_clones":0,"num_object_copies":19004,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9502,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7186,"num_write_kb":28744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4304},{"shards":"2","objects":5198}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.848670+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T15:59:15.848670+0000","last_peered":"2024-09-10T15:59:15.848670+0000","last_clean":"2024-09-10T15:59:15.848670+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T15:59:15.848670+0000","last_undegraded":"2024-09-10T15:59:15.848670+0000","last_fullsized":"2024-09-10T15:59:15.848670+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'9714","reported_seq":9838,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:59.649308+0000","last_change":"2024-09-10T15:59:16.174113+0000","last_active":"2024-09-10T16:01:59.649308+0000","last_peered":"2024-09-10T16:01:59.649308+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:01:59.649308+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":9714,"log_dups_size":0,"ondisk_log_size":9714,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39763968,"num_objects":9708,"num_object_clones":0,"num_object_copies":19416,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4354,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9708,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7268,"num_write_kb":29072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4352},{"shards":"2","objects":5356}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'9595","reported_seq":9725,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:02.253142+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:02:02.253142+0000","last_peered":"2024-09-10T16:02:02.253142+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:02:02.253142+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":9595,"log_dups_size":0,"ondisk_log_size":9595,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39292928,"num_objects":9593,"num_object_clones":0,"num_object_copies":19186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9593,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6226,"num_write_kb":24904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5289},{"shards":"0,2","objects":4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.992406+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T15:59:15.992406+0000","last_peered":"2024-09-10T15:59:15.992406+0000","last_clean":"2024-09-10T15:59:15.992406+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T15:59:15.992406+0000","last_undegraded":"2024-09-10T15:59:15.992406+0000","last_fullsized":"2024-09-10T15:59:15.992406+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'9466","reported_seq":9604,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:02.275964+0000","last_change":"2024-09-10T15:59:16.251610+0000","last_active":"2024-09-10T16:02:02.275964+0000","last_peered":"2024-09-10T16:02:02.275964+0000","last_clean":"2024-09-10T15:59:14.115595+0000","last_became_active":"2024-09-10T15:59:16.244089+0000","last_became_peered":"2024-09-10T15:59:16.244089+0000","last_unstale":"2024-09-10T16:02:02.275964+0000","last_undegraded":"2024-09-10T15:59:15.847127+0000","last_fullsized":"2024-09-10T15:59:15.774437+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":9466,"log_dups_size":0,"ondisk_log_size":9466,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38739968,"num_objects":9458,"num_object_clones":0,"num_object_copies":18916,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9458,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5856,"num_write_kb":23424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5255},{"shards":"0,2","objects":4203}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":89,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.773633+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T15:59:15.773633+0000","last_peered":"2024-09-10T15:59:15.773633+0000","last_clean":"2024-09-10T15:59:15.773633+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T15:59:15.773633+0000","last_undegraded":"2024-09-10T15:59:15.773633+0000","last_fullsized":"2024-09-10T15:59:15.773633+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'9751","reported_seq":9888,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:02.231259+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:02:02.231259+0000","last_peered":"2024-09-10T16:02:02.231259+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:02:02.231259+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":9751,"log_dups_size":0,"ondisk_log_size":9751,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39923712,"num_objects":9747,"num_object_clones":0,"num_object_copies":19494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9747,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6006,"num_write_kb":24024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5376},{"shards":"0,2","objects":4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.917634+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T15:59:15.917634+0000","last_peered":"2024-09-10T15:59:15.917634+0000","last_clean":"2024-09-10T15:59:15.917634+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T15:59:15.917634+0000","last_undegraded":"2024-09-10T15:59:15.917634+0000","last_fullsized":"2024-09-10T15:59:15.917634+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":70,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.778232+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T15:59:15.778232+0000","last_peered":"2024-09-10T15:59:15.778232+0000","last_clean":"2024-09-10T15:59:15.778232+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T15:59:15.778232+0000","last_undegraded":"2024-09-10T15:59:15.778232+0000","last_fullsized":"2024-09-10T15:59:15.778232+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'9761","reported_seq":9900,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:02.238272+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:02:02.238272+0000","last_peered":"2024-09-10T16:02:02.238272+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:02:02.238272+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":9761,"log_dups_size":0,"ondisk_log_size":9761,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39932870,"num_objects":9751,"num_object_clones":0,"num_object_copies":19502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9751,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5924,"num_write_kb":23696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5428},{"shards":"0,2","objects":4323}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":629641950,"num_objects":153730,"num_object_clones":0,"num_object_copies":307460,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":47117,"num_objects_misplaced":234,"num_objects_unfound":0,"num_objects_dirty":153730,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":111576,"num_write_kb":446304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4038,"num_bytes_recovered":16535552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1273524224,"data_stored":1273372478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":153802,"ondisk_log_size":153802,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986224,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":713224,"kb_used_data":442744,"kb_used_omap":4,"kb_used_meta":270459,"kb_avail":93658616,"statfs":{"total":96636764160,"available":95906422784,"internally_reserved":0,"allocated":453369856,"data_stored":453092096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":276950698},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659170,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":463580,"kb_used_data":269452,"kb_used_omap":4,"kb_used_meta":194107,"kb_avail":93908260,"statfs":{"total":96636764160,"available":96162058240,"internally_reserved":0,"allocated":275918848,"data_stored":275593655,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":198766254},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,4,3,1,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182453,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":860360,"kb_used_data":531640,"kb_used_omap":4,"kb_used_meta":328699,"kb_avail":93511480,"statfs":{"total":96636764160,"available":95755755520,"internally_reserved":0,"allocated":544399360,"data_stored":544117523,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":336588458},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,1,3,1,6,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":544215040,"data_stored":544178910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":275816448,"data_stored":275736962,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":453492736,"data_stored":453456606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:02:04.888 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-10T16:02:04.888 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "43'9538", 'reported_seq': 9606, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:02:02.263581+0000', 'last_change': '2024-09-10T15:59:16.281558+0000', 'last_active': '2024-09-10T16:02:02.263581+0000', 'last_peered': '2024-09-10T16:02:02.263581+0000', 'last_clean': '2024-09-10T15:59:14.132456+0000', 'last_became_active': '2024-09-10T15:59:16.274407+0000', 'last_became_peered': '2024-09-10T15:59:16.274407+0000', 'last_unstale': '2024-09-10T16:02:02.263581+0000', 'last_undegraded': '2024-09-10T15:59:15.851279+0000', 'last_fullsized': '2024-09-10T15:59:15.774356+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'objects_scrubbed': 0, 'log_size': 9538, 'log_dups_size': 0, 'ondisk_log_size': 9538, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39067648, 'num_objects': 9538, 'num_object_clones': 0, 'num_object_copies': 19076, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4259, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9538, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9538, 'num_write_kb': 38152, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5279}, {'shards': '0,2', 'objects': 4258}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:02:04.888 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-10T16:02:04.889 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "43'9518", 'reported_seq': 17714, 'reported_epoch': 43, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-10T16:01:59.763825+0000', 'last_change': '2024-09-10T15:59:16.142612+0000', 'last_active': '2024-09-10T16:01:59.763825+0000', 'last_peered': '2024-09-10T16:01:59.763825+0000', 'last_clean': '2024-09-10T15:59:14.092473+0000', 'last_became_active': '2024-09-10T15:59:16.109611+0000', 'last_became_peered': '2024-09-10T15:59:16.109611+0000', 'last_unstale': '2024-09-10T16:01:59.763825+0000', 'last_undegraded': '2024-09-10T16:01:59.763825+0000', 'last_fullsized': '2024-09-10T15:59:15.770264+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2029", 'last_scrub_stamp': '2024-09-10T15:58:45.933789+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:45.933789+0000', 'objects_scrubbed': 1992, 'log_size': 9518, 'log_dups_size': 0, 'ondisk_log_size': 9518, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 668, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38969344, 'num_objects': 9514, 'num_object_clones': 0, 'num_object_copies': 19028, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 234, 'num_objects_unfound': 0, 'num_objects_dirty': 9514, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7522, 'num_write_kb': 30088, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 4035, 'num_bytes_recovered': 16523264, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 234}, {'shards': '2', 'objects': 9280}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:02:04.889 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-10T16:02:04.889 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "43'9740", 'reported_seq': 9873, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:02:02.316562+0000', 'last_change': '2024-09-10T15:59:16.308264+0000', 'last_active': '2024-09-10T16:02:02.316562+0000', 'last_peered': '2024-09-10T16:02:02.316562+0000', 'last_clean': '2024-09-10T15:59:14.111551+0000', 'last_became_active': '2024-09-10T15:59:16.300040+0000', 'last_became_peered': '2024-09-10T15:59:16.300040+0000', 'last_unstale': '2024-09-10T16:02:02.316562+0000', 'last_undegraded': '2024-09-10T15:59:15.878234+0000', 'last_fullsized': '2024-09-10T15:59:15.769328+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3442", 'last_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'objects_scrubbed': 3384, 'log_size': 9740, 'log_dups_size': 0, 'ondisk_log_size': 9740, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1292, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39875043, 'num_objects': 9736, 'num_object_clones': 0, 'num_object_copies': 19472, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4372, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9736, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 6352, 'num_write_kb': 25408, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5365}, {'shards': '0,2', 'objects': 4371}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:02:04.889 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-10T16:02:04.889 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "43'9730", 'reported_seq': 9863, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:02:02.190694+0000', 'last_change': '2024-09-10T15:59:16.331899+0000', 'last_active': '2024-09-10T16:02:02.190694+0000', 'last_peered': '2024-09-10T16:02:02.190694+0000', 'last_clean': '2024-09-10T15:59:14.139436+0000', 'last_became_active': '2024-09-10T15:59:16.318715+0000', 'last_became_peered': '2024-09-10T15:59:16.318715+0000', 'last_unstale': '2024-09-10T16:02:02.190694+0000', 'last_undegraded': '2024-09-10T15:59:15.877524+0000', 'last_fullsized': '2024-09-10T15:59:15.769042+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3537", 'last_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'objects_scrubbed': 3497, 'log_size': 9730, 'log_dups_size': 0, 'ondisk_log_size': 9730, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1224, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39830470, 'num_objects': 9726, 'num_object_clones': 0, 'num_object_copies': 19452, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4237, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9726, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 6229, 'num_write_kb': 24916, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5489}, {'shards': '0,2', 'objects': 4237}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:02:04.889 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-10T16:02:04.889 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "43'9338", 'reported_seq': 9470, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:01:59.733354+0000', 'last_change': '2024-09-10T15:59:16.175680+0000', 'last_active': '2024-09-10T16:01:59.733354+0000', 'last_peered': '2024-09-10T16:01:59.733354+0000', 'last_clean': '2024-09-10T15:59:14.088187+0000', 'last_became_active': '2024-09-10T15:59:16.165810+0000', 'last_became_peered': '2024-09-10T15:59:16.165810+0000', 'last_unstale': '2024-09-10T16:01:59.733354+0000', 'last_undegraded': '2024-09-10T15:59:15.822098+0000', 'last_fullsized': '2024-09-10T15:59:15.769123+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2218", 'last_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'objects_scrubbed': 2175, 'log_size': 9338, 'log_dups_size': 0, 'ondisk_log_size': 9338, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 825, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38220259, 'num_objects': 9332, 'num_object_clones': 0, 'num_object_copies': 18664, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4124, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9332, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 7157, 'num_write_kb': 28628, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4123}, {'shards': '2', 'objects': 5209}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:02:04.889 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-10T16:02:04.889 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "43'9658", 'reported_seq': 9784, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:02:02.257337+0000', 'last_change': '2024-09-10T15:59:16.277742+0000', 'last_active': '2024-09-10T16:02:02.257337+0000', 'last_peered': '2024-09-10T16:02:02.257337+0000', 'last_clean': '2024-09-10T15:59:14.129081+0000', 'last_became_active': '2024-09-10T15:59:16.269835+0000', 'last_became_peered': '2024-09-10T15:59:16.269835+0000', 'last_unstale': '2024-09-10T16:02:02.257337+0000', 'last_undegraded': '2024-09-10T15:59:15.860857+0000', 'last_fullsized': '2024-09-10T15:59:15.771410+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3557", 'last_scrub_stamp': '2024-09-10T15:59:02.493678+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:02.493678+0000', 'objects_scrubbed': 3488, 'log_size': 9658, 'log_dups_size': 0, 'ondisk_log_size': 9658, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1472, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39526400, 'num_objects': 9650, 'num_object_clones': 0, 'num_object_copies': 19300, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4257, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9650, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6162, 'num_write_kb': 24648, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5395}, {'shards': '0,2', 'objects': 4255}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:02:04.890 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-10T16:02:04.890 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "43'9506", 'reported_seq': 9636, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:01:59.749553+0000', 'last_change': '2024-09-10T15:59:16.284311+0000', 'last_active': '2024-09-10T16:01:59.749553+0000', 'last_peered': '2024-09-10T16:01:59.749553+0000', 'last_clean': '2024-09-10T15:59:14.116041+0000', 'last_became_active': '2024-09-10T15:59:16.274543+0000', 'last_became_peered': '2024-09-10T15:59:16.274543+0000', 'last_unstale': '2024-09-10T16:01:59.749553+0000', 'last_undegraded': '2024-09-10T15:59:15.812484+0000', 'last_fullsized': '2024-09-10T15:59:15.770885+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2356", 'last_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'objects_scrubbed': 2316, 'log_size': 9506, 'log_dups_size': 0, 'ondisk_log_size': 9506, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 783, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38916579, 'num_objects': 9502, 'num_object_clones': 0, 'num_object_copies': 19004, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9502, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 7186, 'num_write_kb': 28744, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4304}, {'shards': '2', 'objects': 5198}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:02:04.890 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-10T16:02:04.890 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "43'9714", 'reported_seq': 9838, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:01:59.649308+0000', 'last_change': '2024-09-10T15:59:16.174113+0000', 'last_active': '2024-09-10T16:01:59.649308+0000', 'last_peered': '2024-09-10T16:01:59.649308+0000', 'last_clean': '2024-09-10T15:59:14.110654+0000', 'last_became_active': '2024-09-10T15:59:16.164645+0000', 'last_became_peered': '2024-09-10T15:59:16.164645+0000', 'last_unstale': '2024-09-10T16:01:59.649308+0000', 'last_undegraded': '2024-09-10T15:59:15.803593+0000', 'last_fullsized': '2024-09-10T15:59:15.769083+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2486", 'last_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'objects_scrubbed': 2440, 'log_size': 9714, 'log_dups_size': 0, 'ondisk_log_size': 9714, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 792, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39763968, 'num_objects': 9708, 'num_object_clones': 0, 'num_object_copies': 19416, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4354, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9708, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7268, 'num_write_kb': 29072, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4352}, {'shards': '2', 'objects': 5356}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:02:04.890 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-10T16:02:04.890 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "43'9595", 'reported_seq': 9725, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:02:02.253142+0000', 'last_change': '2024-09-10T15:59:16.315641+0000', 'last_active': '2024-09-10T16:02:02.253142+0000', 'last_peered': '2024-09-10T16:02:02.253142+0000', 'last_clean': '2024-09-10T15:59:14.117001+0000', 'last_became_active': '2024-09-10T15:59:16.302245+0000', 'last_became_peered': '2024-09-10T15:59:16.302245+0000', 'last_unstale': '2024-09-10T16:02:02.253142+0000', 'last_undegraded': '2024-09-10T15:59:15.845012+0000', 'last_fullsized': '2024-09-10T15:59:15.770337+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3404", 'last_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'objects_scrubbed': 3367, 'log_size': 9595, 'log_dups_size': 0, 'ondisk_log_size': 9595, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 983, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39292928, 'num_objects': 9593, 'num_object_clones': 0, 'num_object_copies': 19186, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9593, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6226, 'num_write_kb': 24904, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5289}, {'shards': '0,2', 'objects': 4304}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:02:04.890 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-10T16:02:04.890 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "43'9466", 'reported_seq': 9604, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:02:02.275964+0000', 'last_change': '2024-09-10T15:59:16.251610+0000', 'last_active': '2024-09-10T16:02:02.275964+0000', 'last_peered': '2024-09-10T16:02:02.275964+0000', 'last_clean': '2024-09-10T15:59:14.115595+0000', 'last_became_active': '2024-09-10T15:59:16.244089+0000', 'last_became_peered': '2024-09-10T15:59:16.244089+0000', 'last_unstale': '2024-09-10T16:02:02.275964+0000', 'last_undegraded': '2024-09-10T15:59:15.847127+0000', 'last_fullsized': '2024-09-10T15:59:15.774437+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3653", 'last_scrub_stamp': '2024-09-10T15:59:05.298468+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:05.298468+0000', 'objects_scrubbed': 3602, 'log_size': 9466, 'log_dups_size': 0, 'ondisk_log_size': 9466, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1289, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38739968, 'num_objects': 9458, 'num_object_clones': 0, 'num_object_copies': 18916, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4206, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9458, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5856, 'num_write_kb': 23424, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5255}, {'shards': '0,2', 'objects': 4203}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:02:04.890 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-10T16:02:04.890 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "43'9751", 'reported_seq': 9888, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:02:02.231259+0000', 'last_change': '2024-09-10T15:59:16.311291+0000', 'last_active': '2024-09-10T16:02:02.231259+0000', 'last_peered': '2024-09-10T16:02:02.231259+0000', 'last_clean': '2024-09-10T15:59:14.116363+0000', 'last_became_active': '2024-09-10T15:59:16.301244+0000', 'last_became_peered': '2024-09-10T15:59:16.301244+0000', 'last_unstale': '2024-09-10T16:02:02.231259+0000', 'last_undegraded': '2024-09-10T15:59:15.883434+0000', 'last_fullsized': '2024-09-10T15:59:15.772756+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3789", 'last_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'objects_scrubbed': 3741, 'log_size': 9751, 'log_dups_size': 0, 'ondisk_log_size': 9751, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1440, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39923712, 'num_objects': 9747, 'num_object_clones': 0, 'num_object_copies': 19494, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4373, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9747, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6006, 'num_write_kb': 24024, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5376}, {'shards': '0,2', 'objects': 4371}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:02:04.891 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-10T16:02:04.891 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "43'9761", 'reported_seq': 9900, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:02:02.238272+0000', 'last_change': '2024-09-10T15:59:16.313409+0000', 'last_active': '2024-09-10T16:02:02.238272+0000', 'last_peered': '2024-09-10T16:02:02.238272+0000', 'last_clean': '2024-09-10T15:59:14.129780+0000', 'last_became_active': '2024-09-10T15:59:16.300932+0000', 'last_became_peered': '2024-09-10T15:59:16.300932+0000', 'last_unstale': '2024-09-10T16:02:02.238272+0000', 'last_undegraded': '2024-09-10T15:59:15.887823+0000', 'last_fullsized': '2024-09-10T15:59:15.768513+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3880", 'last_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'objects_scrubbed': 3827, 'log_size': 9761, 'log_dups_size': 0, 'ondisk_log_size': 9761, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1397, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39932870, 'num_objects': 9751, 'num_object_clones': 0, 'num_object_copies': 19502, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4325, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9751, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5924, 'num_write_kb': 23696, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5428}, {'shards': '0,2', 'objects': 4323}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:02:04.891 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T16:02:04.994 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 212 20 155341 155321 2.86145 1.60156 0.0611802 0.0272954 2024-09-10T16:02:05.571 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:02:05.571 INFO:teuthology.orchestra.run.smithi115.stdout:{"fsid":"7effee29-df05-4351-9741-ee203564ed81","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 47117/307464 objects degraded (15.324%), 11 pgs degraded, 12 pgs undersized","count":23},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":289,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":43,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983954,"num_in_osds":3,"osd_in_since":1725983947,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":153732,"data_bytes":629641969,"bytes_used":2086055936,"bytes_avail":287824236544,"bytes_total":289910292480,"degraded_objects":47117,"degraded_total":307464,"degraded_ratio":0.15324395701610596,"misplaced_objects":234,"misplaced_total":307464,"misplaced_ratio":0.00076106471001483099,"recovering_objects_per_sec":28,"recovering_bytes_per_sec":116012,"recovering_keys_per_sec":0,"num_objects_recovered":340,"num_bytes_recovered":1392640,"num_keys_recovered":0,"write_bytes_sec":2010777,"read_op_per_sec":0,"write_op_per_sec":490},"fsmap":{"epoch":1,"btime":"2024-09-10T15:57:14:967120+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T16:00:13.560776+0000","services":{}},"progress_events":{}} 2024-09-10T16:02:05.597 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-10T16:02:05.597 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:02:05.994 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 213 20 155756 155736 2.85562 1.62109 0.0444758 0.027352 2024-09-10T16:02:06.157 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:02:06.157 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:02:06.185 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":181,"stamp":"2024-09-10T16:02:05.604983+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":633000689,"num_objects":154552,"num_object_clones":0,"num_object_copies":309104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":47117,"num_objects_misplaced":70,"num_objects_unfound":0,"num_objects_dirty":154552,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":112398,"num_write_kb":449586,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4204,"num_bytes_recovered":17207315,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":154625,"ondisk_log_size":154625,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2071036,"kb_used_data":1255500,"kb_used_omap":12,"kb_used_meta":815475,"kb_avail":281044484,"statfs":{"total":289910292480,"available":287789551616,"internally_reserved":0,"allocated":1285632000,"data_stored":1284747210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":835046402},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,4,2,5,8,13,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":21696512,"num_objects":5297,"num_object_clones":0,"num_object_copies":10594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-504,"num_objects_unfound":0,"num_objects_dirty":5297,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5297,"num_write_kb":21188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":504,"num_bytes_recovered":2064384,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005008"},"pg_stats":[{"pgid":"2.f","version":"43'9538","reported_seq":9606,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:02.263581+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:02:02.263581+0000","last_peered":"2024-09-10T16:02:02.263581+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:02:02.263581+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":9538,"log_dups_size":0,"ondisk_log_size":9538,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39067648,"num_objects":9538,"num_object_clones":0,"num_object_copies":19076,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9538,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9538,"num_write_kb":38152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5279},{"shards":"0,2","objects":4258}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'9686","reported_seq":9791,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:02:04.793468+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:02:04.793468+0000","last_peered":"2024-09-10T16:02:04.793468+0000","last_clean":"2024-09-10T16:02:04.793468+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:02:04.793468+0000","last_undegraded":"2024-09-10T16:02:04.793468+0000","last_fullsized":"2024-09-10T16:02:04.793468+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":9686,"log_dups_size":0,"ondisk_log_size":9686,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39650246,"num_objects":9682,"num_object_clones":0,"num_object_copies":19364,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9682,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":7791,"num_write_kb":31164,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'9668","reported_seq":18192,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T16:02:04.781174+0000","last_change":"2024-09-10T15:59:16.142612+0000","last_active":"2024-09-10T16:02:04.781174+0000","last_peered":"2024-09-10T16:02:04.781174+0000","last_clean":"2024-09-10T15:59:14.092473+0000","last_became_active":"2024-09-10T15:59:16.109611+0000","last_became_peered":"2024-09-10T15:59:16.109611+0000","last_unstale":"2024-09-10T16:02:04.781174+0000","last_undegraded":"2024-09-10T16:02:04.781174+0000","last_fullsized":"2024-09-10T15:59:15.770264+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":9668,"log_dups_size":0,"ondisk_log_size":9668,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39583744,"num_objects":9664,"num_object_clones":0,"num_object_copies":19328,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":70,"num_objects_unfound":0,"num_objects_dirty":9664,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7672,"num_write_kb":30688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4199,"num_bytes_recovered":17195008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":70},{"shards":"2","objects":9594}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'9615","reported_seq":9691,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:02:04.779576+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:02:04.779576+0000","last_peered":"2024-09-10T16:02:04.779576+0000","last_clean":"2024-09-10T16:02:04.779576+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:02:04.779576+0000","last_undegraded":"2024-09-10T16:02:04.779576+0000","last_fullsized":"2024-09-10T16:02:04.779576+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":9615,"log_dups_size":0,"ondisk_log_size":9615,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39383040,"num_objects":9615,"num_object_clones":0,"num_object_copies":19230,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9615,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9615,"num_write_kb":38460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'9635","reported_seq":9760,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:02:02.251441+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:02:02.251441+0000","last_peered":"2024-09-10T16:02:02.251441+0000","last_clean":"2024-09-10T16:02:02.251441+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:02:02.251441+0000","last_undegraded":"2024-09-10T16:02:02.251441+0000","last_fullsized":"2024-09-10T16:02:02.251441+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":9635,"log_dups_size":0,"ondisk_log_size":9635,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39436771,"num_objects":9629,"num_object_clones":0,"num_object_copies":19258,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9629,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6346,"num_write_kb":25384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'9740","reported_seq":9873,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:02.316562+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:02:02.316562+0000","last_peered":"2024-09-10T16:02:02.316562+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:02:02.316562+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":9740,"log_dups_size":0,"ondisk_log_size":9740,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39875043,"num_objects":9736,"num_object_clones":0,"num_object_copies":19472,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9736,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6352,"num_write_kb":25408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5365},{"shards":"0,2","objects":4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'9730","reported_seq":9863,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:02.190694+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:02:02.190694+0000","last_peered":"2024-09-10T16:02:02.190694+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:02:02.190694+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":9730,"log_dups_size":0,"ondisk_log_size":9730,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39830470,"num_objects":9726,"num_object_clones":0,"num_object_copies":19452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9726,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":6229,"num_write_kb":24916,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5489},{"shards":"0,2","objects":4237}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'9838","reported_seq":9963,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:02:02.143892+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:02:02.143892+0000","last_peered":"2024-09-10T16:02:02.143892+0000","last_clean":"2024-09-10T16:02:02.143892+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:02:02.143892+0000","last_undegraded":"2024-09-10T16:02:02.143892+0000","last_fullsized":"2024-09-10T16:02:02.143892+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":9838,"log_dups_size":0,"ondisk_log_size":9838,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40288256,"num_objects":9836,"num_object_clones":0,"num_object_copies":19672,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9836,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6685,"num_write_kb":26740,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.777326+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T15:59:15.777326+0000","last_peered":"2024-09-10T15:59:15.777326+0000","last_clean":"2024-09-10T15:59:15.777326+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T15:59:15.777326+0000","last_undegraded":"2024-09-10T15:59:15.777326+0000","last_fullsized":"2024-09-10T15:59:15.777326+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'9472","reported_seq":9607,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:04.759011+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:02:04.759011+0000","last_peered":"2024-09-10T16:02:04.759011+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:02:04.759011+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":9472,"log_dups_size":0,"ondisk_log_size":9472,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38769123,"num_objects":9466,"num_object_clones":0,"num_object_copies":18932,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9466,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7291,"num_write_kb":29164,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4123},{"shards":"2","objects":5343}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":89,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.772619+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T15:59:15.772619+0000","last_peered":"2024-09-10T15:59:15.772619+0000","last_clean":"2024-09-10T15:59:15.772619+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T15:59:15.772619+0000","last_undegraded":"2024-09-10T15:59:15.772619+0000","last_fullsized":"2024-09-10T15:59:15.772619+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'9658","reported_seq":9784,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:02.257337+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:02:02.257337+0000","last_peered":"2024-09-10T16:02:02.257337+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:02:02.257337+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":9658,"log_dups_size":0,"ondisk_log_size":9658,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39526400,"num_objects":9650,"num_object_clones":0,"num_object_copies":19300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9650,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6162,"num_write_kb":24648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5395},{"shards":"0,2","objects":4255}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":99,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.884297+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T15:59:15.884297+0000","last_peered":"2024-09-10T15:59:15.884297+0000","last_clean":"2024-09-10T15:59:15.884297+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T15:59:15.884297+0000","last_undegraded":"2024-09-10T15:59:15.884297+0000","last_fullsized":"2024-09-10T15:59:15.884297+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'9630","reported_seq":9759,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:04.774978+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:02:04.774978+0000","last_peered":"2024-09-10T16:02:04.774978+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:02:04.774978+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":9630,"log_dups_size":0,"ondisk_log_size":9630,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39424483,"num_objects":9626,"num_object_clones":0,"num_object_copies":19252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9626,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7310,"num_write_kb":29240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4304},{"shards":"2","objects":5322}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.848670+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T15:59:15.848670+0000","last_peered":"2024-09-10T15:59:15.848670+0000","last_clean":"2024-09-10T15:59:15.848670+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T15:59:15.848670+0000","last_undegraded":"2024-09-10T15:59:15.848670+0000","last_fullsized":"2024-09-10T15:59:15.848670+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'9839","reported_seq":9963,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:04.618366+0000","last_change":"2024-09-10T15:59:16.174113+0000","last_active":"2024-09-10T16:02:04.618366+0000","last_peered":"2024-09-10T16:02:04.618366+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:02:04.618366+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":9839,"log_dups_size":0,"ondisk_log_size":9839,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40275968,"num_objects":9833,"num_object_clones":0,"num_object_copies":19666,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4354,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9833,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7393,"num_write_kb":29572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4352},{"shards":"2","objects":5481}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'9595","reported_seq":9725,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:02.253142+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:02:02.253142+0000","last_peered":"2024-09-10T16:02:02.253142+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:02:02.253142+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":9595,"log_dups_size":0,"ondisk_log_size":9595,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39292928,"num_objects":9593,"num_object_clones":0,"num_object_copies":19186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9593,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6226,"num_write_kb":24904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5289},{"shards":"0,2","objects":4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.992406+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T15:59:15.992406+0000","last_peered":"2024-09-10T15:59:15.992406+0000","last_clean":"2024-09-10T15:59:15.992406+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T15:59:15.992406+0000","last_undegraded":"2024-09-10T15:59:15.992406+0000","last_fullsized":"2024-09-10T15:59:15.992406+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'9466","reported_seq":9604,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:02.275964+0000","last_change":"2024-09-10T15:59:16.251610+0000","last_active":"2024-09-10T16:02:02.275964+0000","last_peered":"2024-09-10T16:02:02.275964+0000","last_clean":"2024-09-10T15:59:14.115595+0000","last_became_active":"2024-09-10T15:59:16.244089+0000","last_became_peered":"2024-09-10T15:59:16.244089+0000","last_unstale":"2024-09-10T16:02:02.275964+0000","last_undegraded":"2024-09-10T15:59:15.847127+0000","last_fullsized":"2024-09-10T15:59:15.774437+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":9466,"log_dups_size":0,"ondisk_log_size":9466,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38739968,"num_objects":9458,"num_object_clones":0,"num_object_copies":18916,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9458,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5856,"num_write_kb":23424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5255},{"shards":"0,2","objects":4203}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":89,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.773633+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T15:59:15.773633+0000","last_peered":"2024-09-10T15:59:15.773633+0000","last_clean":"2024-09-10T15:59:15.773633+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T15:59:15.773633+0000","last_undegraded":"2024-09-10T15:59:15.773633+0000","last_fullsized":"2024-09-10T15:59:15.773633+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'9751","reported_seq":9888,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:02.231259+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:02:02.231259+0000","last_peered":"2024-09-10T16:02:02.231259+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:02:02.231259+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":9751,"log_dups_size":0,"ondisk_log_size":9751,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39923712,"num_objects":9747,"num_object_clones":0,"num_object_copies":19494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9747,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6006,"num_write_kb":24024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5376},{"shards":"0,2","objects":4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.917634+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T15:59:15.917634+0000","last_peered":"2024-09-10T15:59:15.917634+0000","last_clean":"2024-09-10T15:59:15.917634+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T15:59:15.917634+0000","last_undegraded":"2024-09-10T15:59:15.917634+0000","last_fullsized":"2024-09-10T15:59:15.917634+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":70,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.778232+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T15:59:15.778232+0000","last_peered":"2024-09-10T15:59:15.778232+0000","last_clean":"2024-09-10T15:59:15.778232+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T15:59:15.778232+0000","last_undegraded":"2024-09-10T15:59:15.778232+0000","last_fullsized":"2024-09-10T15:59:15.778232+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'9761","reported_seq":9900,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:02.238272+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:02:02.238272+0000","last_peered":"2024-09-10T16:02:02.238272+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:02:02.238272+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":9761,"log_dups_size":0,"ondisk_log_size":9761,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39932870,"num_objects":9751,"num_object_clones":0,"num_object_copies":19502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9751,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5924,"num_write_kb":23696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5428},{"shards":"0,2","objects":4323}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":633000670,"num_objects":154550,"num_object_clones":0,"num_object_copies":309100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":47117,"num_objects_misplaced":70,"num_objects_unfound":0,"num_objects_dirty":154550,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":112396,"num_write_kb":449584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4202,"num_bytes_recovered":17207296,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1285681152,"data_stored":1285529406,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":154622,"ondisk_log_size":154622,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986225,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":717576,"kb_used_data":447096,"kb_used_omap":4,"kb_used_meta":270459,"kb_avail":93654264,"statfs":{"total":96636764160,"available":95901966336,"internally_reserved":0,"allocated":457826304,"data_stored":457548544,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":276950698},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,1,3,1,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659171,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":493100,"kb_used_data":276764,"kb_used_omap":4,"kb_used_meta":216315,"kb_avail":93878740,"statfs":{"total":96636764160,"available":96131829760,"internally_reserved":0,"allocated":283406336,"data_stored":283081143,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":221507246},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,1,4,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182453,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":860360,"kb_used_data":531640,"kb_used_omap":4,"kb_used_meta":328699,"kb_avail":93511480,"statfs":{"total":96636764160,"available":95755755520,"internally_reserved":0,"allocated":544399360,"data_stored":544117523,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":336588458},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,1,3,1,6,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":544215040,"data_stored":544178910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":283467776,"data_stored":283388290,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":457998336,"data_stored":457962206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:02:06.994 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 214 20 156199 156179 2.85037 1.73047 0.0385909 0.0274 2024-09-10T16:02:07.994 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 215 20 156553 156533 2.84354 1.38281 0.00750991 0.0274072 2024-09-10T16:02:08.994 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 216 20 156943 156923 2.83743 1.52344 0.0456167 0.0275274 2024-09-10T16:02:09.187 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:02:09.728 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:02:09.729 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:02:09.764 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":185,"stamp":"2024-09-10T16:02:09.606419+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":638698225,"num_objects":155943,"num_object_clones":0,"num_object_copies":311886,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42911,"num_objects_misplaced":4256,"num_objects_unfound":0,"num_objects_dirty":155943,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":113789,"num_write_kb":455150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4221,"num_bytes_recovered":17276947,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156016,"ondisk_log_size":156016,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2065196,"kb_used_data":1262136,"kb_used_omap":12,"kb_used_meta":802995,"kb_avail":281050324,"statfs":{"total":289910292480,"available":287795531776,"internally_reserved":0,"allocated":1292427264,"data_stored":1291537284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":822266882},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,2,3,11,12,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":14884864,"num_objects":3634,"num_object_clones":0,"num_object_copies":7268,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-4206,"num_objects_misplaced":4022,"num_objects_unfound":0,"num_objects_dirty":3634,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3634,"num_write_kb":14536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":181,"num_bytes_recovered":741376,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.003779"},"pg_stats":[{"pgid":"2.f","version":"44'9667","reported_seq":9739,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:07.230350+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:02:07.230350+0000","last_peered":"2024-09-10T16:02:07.230350+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:02:07.230350+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":9667,"log_dups_size":0,"ondisk_log_size":9667,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39596032,"num_objects":9667,"num_object_clones":0,"num_object_copies":19334,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9667,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9667,"num_write_kb":38668,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5408},{"shards":"0,2","objects":4258}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'9686","reported_seq":9791,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:02:04.793468+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:02:04.793468+0000","last_peered":"2024-09-10T16:02:04.793468+0000","last_clean":"2024-09-10T16:02:04.793468+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:02:04.793468+0000","last_undegraded":"2024-09-10T16:02:04.793468+0000","last_fullsized":"2024-09-10T16:02:04.793468+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":9686,"log_dups_size":0,"ondisk_log_size":9686,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39650246,"num_objects":9682,"num_object_clones":0,"num_object_copies":19364,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9682,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":7791,"num_write_kb":31164,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'9668","reported_seq":18192,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T16:02:04.781174+0000","last_change":"2024-09-10T15:59:16.142612+0000","last_active":"2024-09-10T16:02:04.781174+0000","last_peered":"2024-09-10T16:02:04.781174+0000","last_clean":"2024-09-10T15:59:14.092473+0000","last_became_active":"2024-09-10T15:59:16.109611+0000","last_became_peered":"2024-09-10T15:59:16.109611+0000","last_unstale":"2024-09-10T16:02:04.781174+0000","last_undegraded":"2024-09-10T16:02:04.781174+0000","last_fullsized":"2024-09-10T15:59:15.770264+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":9668,"log_dups_size":0,"ondisk_log_size":9668,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39583744,"num_objects":9664,"num_object_clones":0,"num_object_copies":19328,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":70,"num_objects_unfound":0,"num_objects_dirty":9664,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7672,"num_write_kb":30688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4199,"num_bytes_recovered":17195008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":70},{"shards":"2","objects":9594}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'9615","reported_seq":9691,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:02:04.779576+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:02:04.779576+0000","last_peered":"2024-09-10T16:02:04.779576+0000","last_clean":"2024-09-10T16:02:04.779576+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:02:04.779576+0000","last_undegraded":"2024-09-10T16:02:04.779576+0000","last_fullsized":"2024-09-10T16:02:04.779576+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":9615,"log_dups_size":0,"ondisk_log_size":9615,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39383040,"num_objects":9615,"num_object_clones":0,"num_object_copies":19230,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9615,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9615,"num_write_kb":38460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"44'9774","reported_seq":9901,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T16:02:07.279445+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:02:07.279445+0000","last_peered":"2024-09-10T16:02:07.279445+0000","last_clean":"2024-09-10T16:02:07.279445+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:02:07.279445+0000","last_undegraded":"2024-09-10T16:02:07.279445+0000","last_fullsized":"2024-09-10T16:02:07.279445+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":9774,"log_dups_size":0,"ondisk_log_size":9774,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40006115,"num_objects":9768,"num_object_clones":0,"num_object_copies":19536,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9768,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6485,"num_write_kb":25940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"44'9882","reported_seq":10017,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:07.262073+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:02:07.262073+0000","last_peered":"2024-09-10T16:02:07.262073+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:02:07.262073+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":9882,"log_dups_size":0,"ondisk_log_size":9882,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40456675,"num_objects":9878,"num_object_clones":0,"num_object_copies":19756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9878,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6494,"num_write_kb":25976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5507},{"shards":"0,2","objects":4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"44'9870","reported_seq":10005,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:07.341329+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:02:07.341329+0000","last_peered":"2024-09-10T16:02:07.341329+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:02:07.341329+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":9870,"log_dups_size":0,"ondisk_log_size":9870,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40403910,"num_objects":9866,"num_object_clones":0,"num_object_copies":19732,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9866,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":6369,"num_write_kb":25476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5629},{"shards":"0,2","objects":4237}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"44'9987","reported_seq":10114,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T16:02:07.357260+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:02:07.357260+0000","last_peered":"2024-09-10T16:02:07.357260+0000","last_clean":"2024-09-10T16:02:07.357260+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:02:07.357260+0000","last_undegraded":"2024-09-10T16:02:07.357260+0000","last_fullsized":"2024-09-10T16:02:07.357260+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":9987,"log_dups_size":0,"ondisk_log_size":9987,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40898560,"num_objects":9985,"num_object_clones":0,"num_object_copies":19970,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9985,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6834,"num_write_kb":27336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.777326+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T15:59:15.777326+0000","last_peered":"2024-09-10T15:59:15.777326+0000","last_clean":"2024-09-10T15:59:15.777326+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T15:59:15.777326+0000","last_undegraded":"2024-09-10T15:59:15.777326+0000","last_fullsized":"2024-09-10T15:59:15.777326+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'9472","reported_seq":9607,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:04.759011+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:02:04.759011+0000","last_peered":"2024-09-10T16:02:04.759011+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:02:04.759011+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":9472,"log_dups_size":0,"ondisk_log_size":9472,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38769123,"num_objects":9466,"num_object_clones":0,"num_object_copies":18932,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9466,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7291,"num_write_kb":29164,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4123},{"shards":"2","objects":5343}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":89,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.772619+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T15:59:15.772619+0000","last_peered":"2024-09-10T15:59:15.772619+0000","last_clean":"2024-09-10T15:59:15.772619+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T15:59:15.772619+0000","last_undegraded":"2024-09-10T15:59:15.772619+0000","last_fullsized":"2024-09-10T15:59:15.772619+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"44'9796","reported_seq":9924,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:07.297599+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:02:07.297599+0000","last_peered":"2024-09-10T16:02:07.297599+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:02:07.297599+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":9796,"log_dups_size":0,"ondisk_log_size":9796,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40091648,"num_objects":9788,"num_object_clones":0,"num_object_copies":19576,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9788,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6300,"num_write_kb":25200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5533},{"shards":"0,2","objects":4255}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":101,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T16:02:07.122545+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:02:07.122545+0000","last_peered":"2024-09-10T16:02:07.122545+0000","last_clean":"2024-09-10T16:02:07.122545+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:02:07.122545+0000","last_undegraded":"2024-09-10T16:02:07.122545+0000","last_fullsized":"2024-09-10T16:02:07.122545+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'9630","reported_seq":9759,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:04.774978+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:02:04.774978+0000","last_peered":"2024-09-10T16:02:04.774978+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:02:04.774978+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":9630,"log_dups_size":0,"ondisk_log_size":9630,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39424483,"num_objects":9626,"num_object_clones":0,"num_object_copies":19252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9626,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7310,"num_write_kb":29240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4304},{"shards":"2","objects":5322}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.848670+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T15:59:15.848670+0000","last_peered":"2024-09-10T15:59:15.848670+0000","last_clean":"2024-09-10T15:59:15.848670+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T15:59:15.848670+0000","last_undegraded":"2024-09-10T15:59:15.848670+0000","last_fullsized":"2024-09-10T15:59:15.848670+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'9839","reported_seq":9963,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:04.618366+0000","last_change":"2024-09-10T15:59:16.174113+0000","last_active":"2024-09-10T16:02:04.618366+0000","last_peered":"2024-09-10T16:02:04.618366+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:02:04.618366+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":9839,"log_dups_size":0,"ondisk_log_size":9839,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40275968,"num_objects":9833,"num_object_clones":0,"num_object_copies":19666,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4354,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9833,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7393,"num_write_kb":29572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4352},{"shards":"2","objects":5481}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"44'9732","reported_seq":9864,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:07.290955+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:02:07.290955+0000","last_peered":"2024-09-10T16:02:07.290955+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:02:07.290955+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":9732,"log_dups_size":0,"ondisk_log_size":9732,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39854080,"num_objects":9730,"num_object_clones":0,"num_object_copies":19460,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9730,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6363,"num_write_kb":25452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5426},{"shards":"0,2","objects":4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.992406+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T15:59:15.992406+0000","last_peered":"2024-09-10T15:59:15.992406+0000","last_clean":"2024-09-10T15:59:15.992406+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T15:59:15.992406+0000","last_undegraded":"2024-09-10T15:59:15.992406+0000","last_fullsized":"2024-09-10T15:59:15.992406+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"44'9620","reported_seq":9795,"reported_epoch":44,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T16:02:07.330921+0000","last_change":"2024-09-10T16:02:06.480033+0000","last_active":"2024-09-10T16:02:07.330921+0000","last_peered":"2024-09-10T16:02:07.330921+0000","last_clean":"2024-09-10T15:59:14.115595+0000","last_became_active":"2024-09-10T15:59:16.244089+0000","last_became_peered":"2024-09-10T15:59:16.244089+0000","last_unstale":"2024-09-10T16:02:07.330921+0000","last_undegraded":"2024-09-10T16:02:07.330921+0000","last_fullsized":"2024-09-10T15:59:15.774437+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":9620,"log_dups_size":0,"ondisk_log_size":9620,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39370752,"num_objects":9612,"num_object_clones":0,"num_object_copies":19224,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":4186,"num_objects_unfound":0,"num_objects_dirty":9612,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6010,"num_write_kb":24040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":69632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5426},{"shards":"0,2","objects":4186}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":89,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.773633+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T15:59:15.773633+0000","last_peered":"2024-09-10T15:59:15.773633+0000","last_clean":"2024-09-10T15:59:15.773633+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T15:59:15.773633+0000","last_undegraded":"2024-09-10T15:59:15.773633+0000","last_fullsized":"2024-09-10T15:59:15.773633+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"44'9869","reported_seq":10008,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:07.286258+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:02:07.286258+0000","last_peered":"2024-09-10T16:02:07.286258+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:02:07.286258+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":9869,"log_dups_size":0,"ondisk_log_size":9869,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40407040,"num_objects":9865,"num_object_clones":0,"num_object_copies":19730,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9865,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6124,"num_write_kb":24496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5494},{"shards":"0,2","objects":4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.917634+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T15:59:15.917634+0000","last_peered":"2024-09-10T15:59:15.917634+0000","last_clean":"2024-09-10T15:59:15.917634+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T15:59:15.917634+0000","last_undegraded":"2024-09-10T15:59:15.917634+0000","last_fullsized":"2024-09-10T15:59:15.917634+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":70,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.778232+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T15:59:15.778232+0000","last_peered":"2024-09-10T15:59:15.778232+0000","last_clean":"2024-09-10T15:59:15.778232+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T15:59:15.778232+0000","last_undegraded":"2024-09-10T15:59:15.778232+0000","last_fullsized":"2024-09-10T15:59:15.778232+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"44'9906","reported_seq":10048,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:07.333397+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:02:07.333397+0000","last_peered":"2024-09-10T16:02:07.333397+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:02:07.333397+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":9906,"log_dups_size":0,"ondisk_log_size":9906,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40526790,"num_objects":9896,"num_object_clones":0,"num_object_copies":19792,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9896,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6069,"num_write_kb":24276,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5573},{"shards":"0,2","objects":4323}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":638698206,"num_objects":155941,"num_object_clones":0,"num_object_copies":311882,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42911,"num_objects_misplaced":4256,"num_objects_unfound":0,"num_objects_dirty":155941,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":113787,"num_write_kb":455148,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4219,"num_bytes_recovered":17276928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1292578816,"data_stored":1292427070,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156013,"ondisk_log_size":156013,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986225,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":717576,"kb_used_data":447096,"kb_used_omap":4,"kb_used_meta":270459,"kb_avail":93654264,"statfs":{"total":96636764160,"available":95901966336,"internally_reserved":0,"allocated":457826304,"data_stored":457548544,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":276950698},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,1,3,1,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659171,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":493100,"kb_used_data":276764,"kb_used_omap":4,"kb_used_meta":216315,"kb_avail":93878740,"statfs":{"total":96636764160,"available":96131829760,"internally_reserved":0,"allocated":283406336,"data_stored":283081143,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":221507246},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,1,4,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182454,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":854520,"kb_used_data":538276,"kb_used_omap":4,"kb_used_meta":316219,"kb_avail":93517320,"statfs":{"total":96636764160,"available":95761735680,"internally_reserved":0,"allocated":551194624,"data_stored":550907597,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":323808938},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,1,4,5,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":551112704,"data_stored":551076574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":283467776,"data_stored":283388290,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":457998336,"data_stored":457962206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:02:09.766 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:02:09.994 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 217 20 157363 157343 2.83191 1.64062 0.113993 0.0275804 2024-09-10T16:02:10.336 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:02:10.339 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:02:10.361 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":185,"stamp":"2024-09-10T16:02:09.606419+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":638698225,"num_objects":155943,"num_object_clones":0,"num_object_copies":311886,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42911,"num_objects_misplaced":4256,"num_objects_unfound":0,"num_objects_dirty":155943,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":113789,"num_write_kb":455150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4221,"num_bytes_recovered":17276947,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156016,"ondisk_log_size":156016,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2065196,"kb_used_data":1262136,"kb_used_omap":12,"kb_used_meta":802995,"kb_avail":281050324,"statfs":{"total":289910292480,"available":287795531776,"internally_reserved":0,"allocated":1292427264,"data_stored":1291537284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":822266882},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,2,3,11,12,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":14884864,"num_objects":3634,"num_object_clones":0,"num_object_copies":7268,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-4206,"num_objects_misplaced":4022,"num_objects_unfound":0,"num_objects_dirty":3634,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3634,"num_write_kb":14536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":181,"num_bytes_recovered":741376,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.003779"},"pg_stats":[{"pgid":"2.f","version":"44'9667","reported_seq":9739,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:07.230350+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:02:07.230350+0000","last_peered":"2024-09-10T16:02:07.230350+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:02:07.230350+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":9667,"log_dups_size":0,"ondisk_log_size":9667,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39596032,"num_objects":9667,"num_object_clones":0,"num_object_copies":19334,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9667,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9667,"num_write_kb":38668,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5408},{"shards":"0,2","objects":4258}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'9686","reported_seq":9791,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:02:04.793468+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:02:04.793468+0000","last_peered":"2024-09-10T16:02:04.793468+0000","last_clean":"2024-09-10T16:02:04.793468+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:02:04.793468+0000","last_undegraded":"2024-09-10T16:02:04.793468+0000","last_fullsized":"2024-09-10T16:02:04.793468+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":9686,"log_dups_size":0,"ondisk_log_size":9686,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39650246,"num_objects":9682,"num_object_clones":0,"num_object_copies":19364,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9682,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":7791,"num_write_kb":31164,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'9668","reported_seq":18192,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T16:02:04.781174+0000","last_change":"2024-09-10T15:59:16.142612+0000","last_active":"2024-09-10T16:02:04.781174+0000","last_peered":"2024-09-10T16:02:04.781174+0000","last_clean":"2024-09-10T15:59:14.092473+0000","last_became_active":"2024-09-10T15:59:16.109611+0000","last_became_peered":"2024-09-10T15:59:16.109611+0000","last_unstale":"2024-09-10T16:02:04.781174+0000","last_undegraded":"2024-09-10T16:02:04.781174+0000","last_fullsized":"2024-09-10T15:59:15.770264+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":9668,"log_dups_size":0,"ondisk_log_size":9668,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39583744,"num_objects":9664,"num_object_clones":0,"num_object_copies":19328,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":70,"num_objects_unfound":0,"num_objects_dirty":9664,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7672,"num_write_kb":30688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4199,"num_bytes_recovered":17195008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":70},{"shards":"2","objects":9594}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'9615","reported_seq":9691,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T16:02:04.779576+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:02:04.779576+0000","last_peered":"2024-09-10T16:02:04.779576+0000","last_clean":"2024-09-10T16:02:04.779576+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:02:04.779576+0000","last_undegraded":"2024-09-10T16:02:04.779576+0000","last_fullsized":"2024-09-10T16:02:04.779576+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":9615,"log_dups_size":0,"ondisk_log_size":9615,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39383040,"num_objects":9615,"num_object_clones":0,"num_object_copies":19230,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9615,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9615,"num_write_kb":38460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"44'9774","reported_seq":9901,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T16:02:07.279445+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:02:07.279445+0000","last_peered":"2024-09-10T16:02:07.279445+0000","last_clean":"2024-09-10T16:02:07.279445+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:02:07.279445+0000","last_undegraded":"2024-09-10T16:02:07.279445+0000","last_fullsized":"2024-09-10T16:02:07.279445+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":9774,"log_dups_size":0,"ondisk_log_size":9774,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40006115,"num_objects":9768,"num_object_clones":0,"num_object_copies":19536,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9768,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6485,"num_write_kb":25940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"44'9882","reported_seq":10017,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:07.262073+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:02:07.262073+0000","last_peered":"2024-09-10T16:02:07.262073+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:02:07.262073+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":9882,"log_dups_size":0,"ondisk_log_size":9882,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40456675,"num_objects":9878,"num_object_clones":0,"num_object_copies":19756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9878,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6494,"num_write_kb":25976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5507},{"shards":"0,2","objects":4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"44'9870","reported_seq":10005,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:07.341329+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:02:07.341329+0000","last_peered":"2024-09-10T16:02:07.341329+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:02:07.341329+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":9870,"log_dups_size":0,"ondisk_log_size":9870,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40403910,"num_objects":9866,"num_object_clones":0,"num_object_copies":19732,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9866,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":6369,"num_write_kb":25476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5629},{"shards":"0,2","objects":4237}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"44'9987","reported_seq":10114,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T16:02:07.357260+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:02:07.357260+0000","last_peered":"2024-09-10T16:02:07.357260+0000","last_clean":"2024-09-10T16:02:07.357260+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:02:07.357260+0000","last_undegraded":"2024-09-10T16:02:07.357260+0000","last_fullsized":"2024-09-10T16:02:07.357260+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":9987,"log_dups_size":0,"ondisk_log_size":9987,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40898560,"num_objects":9985,"num_object_clones":0,"num_object_copies":19970,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9985,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6834,"num_write_kb":27336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.777326+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T15:59:15.777326+0000","last_peered":"2024-09-10T15:59:15.777326+0000","last_clean":"2024-09-10T15:59:15.777326+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T15:59:15.777326+0000","last_undegraded":"2024-09-10T15:59:15.777326+0000","last_fullsized":"2024-09-10T15:59:15.777326+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'9472","reported_seq":9607,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:04.759011+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:02:04.759011+0000","last_peered":"2024-09-10T16:02:04.759011+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:02:04.759011+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":9472,"log_dups_size":0,"ondisk_log_size":9472,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38769123,"num_objects":9466,"num_object_clones":0,"num_object_copies":18932,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9466,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7291,"num_write_kb":29164,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4123},{"shards":"2","objects":5343}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":89,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.772619+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T15:59:15.772619+0000","last_peered":"2024-09-10T15:59:15.772619+0000","last_clean":"2024-09-10T15:59:15.772619+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T15:59:15.772619+0000","last_undegraded":"2024-09-10T15:59:15.772619+0000","last_fullsized":"2024-09-10T15:59:15.772619+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"44'9796","reported_seq":9924,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:07.297599+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:02:07.297599+0000","last_peered":"2024-09-10T16:02:07.297599+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:02:07.297599+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":9796,"log_dups_size":0,"ondisk_log_size":9796,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40091648,"num_objects":9788,"num_object_clones":0,"num_object_copies":19576,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9788,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6300,"num_write_kb":25200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5533},{"shards":"0,2","objects":4255}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":101,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T16:02:07.122545+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:02:07.122545+0000","last_peered":"2024-09-10T16:02:07.122545+0000","last_clean":"2024-09-10T16:02:07.122545+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:02:07.122545+0000","last_undegraded":"2024-09-10T16:02:07.122545+0000","last_fullsized":"2024-09-10T16:02:07.122545+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'9630","reported_seq":9759,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:04.774978+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:02:04.774978+0000","last_peered":"2024-09-10T16:02:04.774978+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:02:04.774978+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":9630,"log_dups_size":0,"ondisk_log_size":9630,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39424483,"num_objects":9626,"num_object_clones":0,"num_object_copies":19252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9626,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7310,"num_write_kb":29240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4304},{"shards":"2","objects":5322}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.848670+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T15:59:15.848670+0000","last_peered":"2024-09-10T15:59:15.848670+0000","last_clean":"2024-09-10T15:59:15.848670+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T15:59:15.848670+0000","last_undegraded":"2024-09-10T15:59:15.848670+0000","last_fullsized":"2024-09-10T15:59:15.848670+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'9839","reported_seq":9963,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:04.618366+0000","last_change":"2024-09-10T15:59:16.174113+0000","last_active":"2024-09-10T16:02:04.618366+0000","last_peered":"2024-09-10T16:02:04.618366+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:02:04.618366+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":9839,"log_dups_size":0,"ondisk_log_size":9839,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40275968,"num_objects":9833,"num_object_clones":0,"num_object_copies":19666,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4354,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9833,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7393,"num_write_kb":29572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4352},{"shards":"2","objects":5481}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"44'9732","reported_seq":9864,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:07.290955+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:02:07.290955+0000","last_peered":"2024-09-10T16:02:07.290955+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:02:07.290955+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":9732,"log_dups_size":0,"ondisk_log_size":9732,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39854080,"num_objects":9730,"num_object_clones":0,"num_object_copies":19460,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9730,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6363,"num_write_kb":25452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5426},{"shards":"0,2","objects":4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.992406+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T15:59:15.992406+0000","last_peered":"2024-09-10T15:59:15.992406+0000","last_clean":"2024-09-10T15:59:15.992406+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T15:59:15.992406+0000","last_undegraded":"2024-09-10T15:59:15.992406+0000","last_fullsized":"2024-09-10T15:59:15.992406+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"44'9620","reported_seq":9795,"reported_epoch":44,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T16:02:07.330921+0000","last_change":"2024-09-10T16:02:06.480033+0000","last_active":"2024-09-10T16:02:07.330921+0000","last_peered":"2024-09-10T16:02:07.330921+0000","last_clean":"2024-09-10T15:59:14.115595+0000","last_became_active":"2024-09-10T15:59:16.244089+0000","last_became_peered":"2024-09-10T15:59:16.244089+0000","last_unstale":"2024-09-10T16:02:07.330921+0000","last_undegraded":"2024-09-10T16:02:07.330921+0000","last_fullsized":"2024-09-10T15:59:15.774437+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":9620,"log_dups_size":0,"ondisk_log_size":9620,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39370752,"num_objects":9612,"num_object_clones":0,"num_object_copies":19224,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":4186,"num_objects_unfound":0,"num_objects_dirty":9612,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6010,"num_write_kb":24040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":69632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5426},{"shards":"0,2","objects":4186}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":89,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.773633+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T15:59:15.773633+0000","last_peered":"2024-09-10T15:59:15.773633+0000","last_clean":"2024-09-10T15:59:15.773633+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T15:59:15.773633+0000","last_undegraded":"2024-09-10T15:59:15.773633+0000","last_fullsized":"2024-09-10T15:59:15.773633+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"44'9869","reported_seq":10008,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:07.286258+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:02:07.286258+0000","last_peered":"2024-09-10T16:02:07.286258+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:02:07.286258+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":9869,"log_dups_size":0,"ondisk_log_size":9869,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40407040,"num_objects":9865,"num_object_clones":0,"num_object_copies":19730,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9865,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6124,"num_write_kb":24496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5494},{"shards":"0,2","objects":4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.917634+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T15:59:15.917634+0000","last_peered":"2024-09-10T15:59:15.917634+0000","last_clean":"2024-09-10T15:59:15.917634+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T15:59:15.917634+0000","last_undegraded":"2024-09-10T15:59:15.917634+0000","last_fullsized":"2024-09-10T15:59:15.917634+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":70,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.778232+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T15:59:15.778232+0000","last_peered":"2024-09-10T15:59:15.778232+0000","last_clean":"2024-09-10T15:59:15.778232+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T15:59:15.778232+0000","last_undegraded":"2024-09-10T15:59:15.778232+0000","last_fullsized":"2024-09-10T15:59:15.778232+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"44'9906","reported_seq":10048,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:07.333397+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:02:07.333397+0000","last_peered":"2024-09-10T16:02:07.333397+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:02:07.333397+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":9906,"log_dups_size":0,"ondisk_log_size":9906,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40526790,"num_objects":9896,"num_object_clones":0,"num_object_copies":19792,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9896,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6069,"num_write_kb":24276,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5573},{"shards":"0,2","objects":4323}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":638698206,"num_objects":155941,"num_object_clones":0,"num_object_copies":311882,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42911,"num_objects_misplaced":4256,"num_objects_unfound":0,"num_objects_dirty":155941,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":113787,"num_write_kb":455148,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4219,"num_bytes_recovered":17276928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1292578816,"data_stored":1292427070,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156013,"ondisk_log_size":156013,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986225,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":717576,"kb_used_data":447096,"kb_used_omap":4,"kb_used_meta":270459,"kb_avail":93654264,"statfs":{"total":96636764160,"available":95901966336,"internally_reserved":0,"allocated":457826304,"data_stored":457548544,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":276950698},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,1,3,1,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659171,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":493100,"kb_used_data":276764,"kb_used_omap":4,"kb_used_meta":216315,"kb_avail":93878740,"statfs":{"total":96636764160,"available":96131829760,"internally_reserved":0,"allocated":283406336,"data_stored":283081143,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":221507246},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,1,4,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182454,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":854520,"kb_used_data":538276,"kb_used_omap":4,"kb_used_meta":316219,"kb_avail":93517320,"statfs":{"total":96636764160,"available":95761735680,"internally_reserved":0,"allocated":551194624,"data_stored":550907597,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":323808938},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,1,4,5,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":551112704,"data_stored":551076574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":283467776,"data_stored":283388290,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":457998336,"data_stored":457962206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:02:10.362 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-10T16:02:10.362 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "44'9667", 'reported_seq': 9739, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:02:07.230350+0000', 'last_change': '2024-09-10T15:59:16.281558+0000', 'last_active': '2024-09-10T16:02:07.230350+0000', 'last_peered': '2024-09-10T16:02:07.230350+0000', 'last_clean': '2024-09-10T15:59:14.132456+0000', 'last_became_active': '2024-09-10T15:59:16.274407+0000', 'last_became_peered': '2024-09-10T15:59:16.274407+0000', 'last_unstale': '2024-09-10T16:02:07.230350+0000', 'last_undegraded': '2024-09-10T15:59:15.851279+0000', 'last_fullsized': '2024-09-10T15:59:15.774356+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'objects_scrubbed': 0, 'log_size': 9667, 'log_dups_size': 0, 'ondisk_log_size': 9667, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39596032, 'num_objects': 9667, 'num_object_clones': 0, 'num_object_copies': 19334, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4259, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9667, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9667, 'num_write_kb': 38668, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5408}, {'shards': '0,2', 'objects': 4258}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:02:10.363 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-10T16:02:10.363 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "43'9668", 'reported_seq': 18192, 'reported_epoch': 43, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-10T16:02:04.781174+0000', 'last_change': '2024-09-10T15:59:16.142612+0000', 'last_active': '2024-09-10T16:02:04.781174+0000', 'last_peered': '2024-09-10T16:02:04.781174+0000', 'last_clean': '2024-09-10T15:59:14.092473+0000', 'last_became_active': '2024-09-10T15:59:16.109611+0000', 'last_became_peered': '2024-09-10T15:59:16.109611+0000', 'last_unstale': '2024-09-10T16:02:04.781174+0000', 'last_undegraded': '2024-09-10T16:02:04.781174+0000', 'last_fullsized': '2024-09-10T15:59:15.770264+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2029", 'last_scrub_stamp': '2024-09-10T15:58:45.933789+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:45.933789+0000', 'objects_scrubbed': 1992, 'log_size': 9668, 'log_dups_size': 0, 'ondisk_log_size': 9668, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 668, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39583744, 'num_objects': 9664, 'num_object_clones': 0, 'num_object_copies': 19328, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 70, 'num_objects_unfound': 0, 'num_objects_dirty': 9664, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7672, 'num_write_kb': 30688, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 4199, 'num_bytes_recovered': 17195008, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 70}, {'shards': '2', 'objects': 9594}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:02:10.363 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-10T16:02:10.363 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "44'9882", 'reported_seq': 10017, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:02:07.262073+0000', 'last_change': '2024-09-10T15:59:16.308264+0000', 'last_active': '2024-09-10T16:02:07.262073+0000', 'last_peered': '2024-09-10T16:02:07.262073+0000', 'last_clean': '2024-09-10T15:59:14.111551+0000', 'last_became_active': '2024-09-10T15:59:16.300040+0000', 'last_became_peered': '2024-09-10T15:59:16.300040+0000', 'last_unstale': '2024-09-10T16:02:07.262073+0000', 'last_undegraded': '2024-09-10T15:59:15.878234+0000', 'last_fullsized': '2024-09-10T15:59:15.769328+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3442", 'last_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'objects_scrubbed': 3384, 'log_size': 9882, 'log_dups_size': 0, 'ondisk_log_size': 9882, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1292, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40456675, 'num_objects': 9878, 'num_object_clones': 0, 'num_object_copies': 19756, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4372, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9878, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 6494, 'num_write_kb': 25976, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5507}, {'shards': '0,2', 'objects': 4371}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:02:10.363 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-10T16:02:10.363 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "44'9870", 'reported_seq': 10005, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:02:07.341329+0000', 'last_change': '2024-09-10T15:59:16.331899+0000', 'last_active': '2024-09-10T16:02:07.341329+0000', 'last_peered': '2024-09-10T16:02:07.341329+0000', 'last_clean': '2024-09-10T15:59:14.139436+0000', 'last_became_active': '2024-09-10T15:59:16.318715+0000', 'last_became_peered': '2024-09-10T15:59:16.318715+0000', 'last_unstale': '2024-09-10T16:02:07.341329+0000', 'last_undegraded': '2024-09-10T15:59:15.877524+0000', 'last_fullsized': '2024-09-10T15:59:15.769042+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3537", 'last_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'objects_scrubbed': 3497, 'log_size': 9870, 'log_dups_size': 0, 'ondisk_log_size': 9870, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1224, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40403910, 'num_objects': 9866, 'num_object_clones': 0, 'num_object_copies': 19732, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4237, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9866, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 6369, 'num_write_kb': 25476, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5629}, {'shards': '0,2', 'objects': 4237}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:02:10.363 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-10T16:02:10.363 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "43'9472", 'reported_seq': 9607, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:02:04.759011+0000', 'last_change': '2024-09-10T15:59:16.175680+0000', 'last_active': '2024-09-10T16:02:04.759011+0000', 'last_peered': '2024-09-10T16:02:04.759011+0000', 'last_clean': '2024-09-10T15:59:14.088187+0000', 'last_became_active': '2024-09-10T15:59:16.165810+0000', 'last_became_peered': '2024-09-10T15:59:16.165810+0000', 'last_unstale': '2024-09-10T16:02:04.759011+0000', 'last_undegraded': '2024-09-10T15:59:15.822098+0000', 'last_fullsized': '2024-09-10T15:59:15.769123+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2218", 'last_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'objects_scrubbed': 2175, 'log_size': 9472, 'log_dups_size': 0, 'ondisk_log_size': 9472, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 825, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38769123, 'num_objects': 9466, 'num_object_clones': 0, 'num_object_copies': 18932, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4124, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9466, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 7291, 'num_write_kb': 29164, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4123}, {'shards': '2', 'objects': 5343}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:02:10.363 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-10T16:02:10.363 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "44'9796", 'reported_seq': 9924, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:02:07.297599+0000', 'last_change': '2024-09-10T15:59:16.277742+0000', 'last_active': '2024-09-10T16:02:07.297599+0000', 'last_peered': '2024-09-10T16:02:07.297599+0000', 'last_clean': '2024-09-10T15:59:14.129081+0000', 'last_became_active': '2024-09-10T15:59:16.269835+0000', 'last_became_peered': '2024-09-10T15:59:16.269835+0000', 'last_unstale': '2024-09-10T16:02:07.297599+0000', 'last_undegraded': '2024-09-10T15:59:15.860857+0000', 'last_fullsized': '2024-09-10T15:59:15.771410+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3557", 'last_scrub_stamp': '2024-09-10T15:59:02.493678+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:02.493678+0000', 'objects_scrubbed': 3488, 'log_size': 9796, 'log_dups_size': 0, 'ondisk_log_size': 9796, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1472, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40091648, 'num_objects': 9788, 'num_object_clones': 0, 'num_object_copies': 19576, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4257, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9788, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6300, 'num_write_kb': 25200, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5533}, {'shards': '0,2', 'objects': 4255}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:02:10.363 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-10T16:02:10.364 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "43'9630", 'reported_seq': 9759, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:02:04.774978+0000', 'last_change': '2024-09-10T15:59:16.284311+0000', 'last_active': '2024-09-10T16:02:04.774978+0000', 'last_peered': '2024-09-10T16:02:04.774978+0000', 'last_clean': '2024-09-10T15:59:14.116041+0000', 'last_became_active': '2024-09-10T15:59:16.274543+0000', 'last_became_peered': '2024-09-10T15:59:16.274543+0000', 'last_unstale': '2024-09-10T16:02:04.774978+0000', 'last_undegraded': '2024-09-10T15:59:15.812484+0000', 'last_fullsized': '2024-09-10T15:59:15.770885+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2356", 'last_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'objects_scrubbed': 2316, 'log_size': 9630, 'log_dups_size': 0, 'ondisk_log_size': 9630, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 783, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39424483, 'num_objects': 9626, 'num_object_clones': 0, 'num_object_copies': 19252, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9626, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 7310, 'num_write_kb': 29240, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4304}, {'shards': '2', 'objects': 5322}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:02:10.364 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-10T16:02:10.364 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "43'9839", 'reported_seq': 9963, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:02:04.618366+0000', 'last_change': '2024-09-10T15:59:16.174113+0000', 'last_active': '2024-09-10T16:02:04.618366+0000', 'last_peered': '2024-09-10T16:02:04.618366+0000', 'last_clean': '2024-09-10T15:59:14.110654+0000', 'last_became_active': '2024-09-10T15:59:16.164645+0000', 'last_became_peered': '2024-09-10T15:59:16.164645+0000', 'last_unstale': '2024-09-10T16:02:04.618366+0000', 'last_undegraded': '2024-09-10T15:59:15.803593+0000', 'last_fullsized': '2024-09-10T15:59:15.769083+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2486", 'last_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'objects_scrubbed': 2440, 'log_size': 9839, 'log_dups_size': 0, 'ondisk_log_size': 9839, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 792, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40275968, 'num_objects': 9833, 'num_object_clones': 0, 'num_object_copies': 19666, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4354, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9833, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7393, 'num_write_kb': 29572, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4352}, {'shards': '2', 'objects': 5481}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:02:10.364 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-10T16:02:10.364 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "44'9732", 'reported_seq': 9864, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:02:07.290955+0000', 'last_change': '2024-09-10T15:59:16.315641+0000', 'last_active': '2024-09-10T16:02:07.290955+0000', 'last_peered': '2024-09-10T16:02:07.290955+0000', 'last_clean': '2024-09-10T15:59:14.117001+0000', 'last_became_active': '2024-09-10T15:59:16.302245+0000', 'last_became_peered': '2024-09-10T15:59:16.302245+0000', 'last_unstale': '2024-09-10T16:02:07.290955+0000', 'last_undegraded': '2024-09-10T15:59:15.845012+0000', 'last_fullsized': '2024-09-10T15:59:15.770337+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3404", 'last_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'objects_scrubbed': 3367, 'log_size': 9732, 'log_dups_size': 0, 'ondisk_log_size': 9732, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 983, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39854080, 'num_objects': 9730, 'num_object_clones': 0, 'num_object_copies': 19460, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9730, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6363, 'num_write_kb': 25452, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5426}, {'shards': '0,2', 'objects': 4304}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:02:10.364 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-10T16:02:10.364 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "44'9620", 'reported_seq': 9795, 'reported_epoch': 44, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-10T16:02:07.330921+0000', 'last_change': '2024-09-10T16:02:06.480033+0000', 'last_active': '2024-09-10T16:02:07.330921+0000', 'last_peered': '2024-09-10T16:02:07.330921+0000', 'last_clean': '2024-09-10T15:59:14.115595+0000', 'last_became_active': '2024-09-10T15:59:16.244089+0000', 'last_became_peered': '2024-09-10T15:59:16.244089+0000', 'last_unstale': '2024-09-10T16:02:07.330921+0000', 'last_undegraded': '2024-09-10T16:02:07.330921+0000', 'last_fullsized': '2024-09-10T15:59:15.774437+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3653", 'last_scrub_stamp': '2024-09-10T15:59:05.298468+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:05.298468+0000', 'objects_scrubbed': 3602, 'log_size': 9620, 'log_dups_size': 0, 'ondisk_log_size': 9620, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1289, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39370752, 'num_objects': 9612, 'num_object_clones': 0, 'num_object_copies': 19224, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 4186, 'num_objects_unfound': 0, 'num_objects_dirty': 9612, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6010, 'num_write_kb': 24040, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 17, 'num_bytes_recovered': 69632, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5426}, {'shards': '0,2', 'objects': 4186}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:02:10.364 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-10T16:02:10.364 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "44'9869", 'reported_seq': 10008, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:02:07.286258+0000', 'last_change': '2024-09-10T15:59:16.311291+0000', 'last_active': '2024-09-10T16:02:07.286258+0000', 'last_peered': '2024-09-10T16:02:07.286258+0000', 'last_clean': '2024-09-10T15:59:14.116363+0000', 'last_became_active': '2024-09-10T15:59:16.301244+0000', 'last_became_peered': '2024-09-10T15:59:16.301244+0000', 'last_unstale': '2024-09-10T16:02:07.286258+0000', 'last_undegraded': '2024-09-10T15:59:15.883434+0000', 'last_fullsized': '2024-09-10T15:59:15.772756+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3789", 'last_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'objects_scrubbed': 3741, 'log_size': 9869, 'log_dups_size': 0, 'ondisk_log_size': 9869, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1440, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40407040, 'num_objects': 9865, 'num_object_clones': 0, 'num_object_copies': 19730, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4373, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9865, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6124, 'num_write_kb': 24496, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5494}, {'shards': '0,2', 'objects': 4371}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:02:10.364 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-10T16:02:10.364 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "44'9906", 'reported_seq': 10048, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:02:07.333397+0000', 'last_change': '2024-09-10T15:59:16.313409+0000', 'last_active': '2024-09-10T16:02:07.333397+0000', 'last_peered': '2024-09-10T16:02:07.333397+0000', 'last_clean': '2024-09-10T15:59:14.129780+0000', 'last_became_active': '2024-09-10T15:59:16.300932+0000', 'last_became_peered': '2024-09-10T15:59:16.300932+0000', 'last_unstale': '2024-09-10T16:02:07.333397+0000', 'last_undegraded': '2024-09-10T15:59:15.887823+0000', 'last_fullsized': '2024-09-10T15:59:15.768513+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3880", 'last_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'objects_scrubbed': 3827, 'log_size': 9906, 'log_dups_size': 0, 'ondisk_log_size': 9906, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1397, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40526790, 'num_objects': 9896, 'num_object_clones': 0, 'num_object_copies': 19792, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4325, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9896, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6069, 'num_write_kb': 24276, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5573}, {'shards': '0,2', 'objects': 4323}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:02:10.365 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T16:02:10.994 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 218 20 157763 157743 2.82609 1.5625 0.0409036 0.0276368 2024-09-10T16:02:11.054 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:02:11.054 INFO:teuthology.orchestra.run.smithi115.stdout:{"fsid":"7effee29-df05-4351-9741-ee203564ed81","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 42911/311886 objects degraded (13.759%), 10 pgs degraded, 12 pgs undersized","count":22},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":295,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983954,"num_in_osds":3,"osd_in_since":1725983947,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+remapped","count":2}],"num_pgs":24,"num_pools":2,"num_objects":155943,"data_bytes":638698225,"bytes_used":2114760704,"bytes_avail":287795531776,"bytes_total":289910292480,"degraded_objects":42911,"degraded_total":311886,"degraded_ratio":0.13758552804550381,"misplaced_objects":4256,"misplaced_total":311886,"misplaced_ratio":0.013646011683756242,"recovering_objects_per_sec":22,"recovering_bytes_per_sec":92628,"recovering_keys_per_sec":0,"num_objects_recovered":181,"num_bytes_recovered":741376,"num_keys_recovered":0,"write_bytes_sec":1859729,"read_op_per_sec":0,"write_op_per_sec":454},"fsmap":{"epoch":1,"btime":"2024-09-10T15:57:14:967120+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T16:00:13.560776+0000","services":{}},"progress_events":{}} 2024-09-10T16:02:11.081 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-10T16:02:11.081 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:02:11.627 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:02:11.630 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:02:11.650 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":186,"stamp":"2024-09-10T16:02:11.606977+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":641778417,"num_objects":156695,"num_object_clones":0,"num_object_copies":313390,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42911,"num_objects_misplaced":4186,"num_objects_unfound":0,"num_objects_dirty":156695,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":114541,"num_write_kb":458158,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4291,"num_bytes_recovered":17563667,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156768,"ondisk_log_size":156768,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2084160,"kb_used_data":1272772,"kb_used_omap":12,"kb_used_meta":811315,"kb_avail":281031360,"statfs":{"total":289910292480,"available":287776112640,"internally_reserved":0,"allocated":1303318528,"data_stored":1302407746,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":830786562},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,2,7,9,8,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":12136448,"num_objects":2963,"num_object_clones":0,"num_object_copies":5926,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-4206,"num_objects_misplaced":3952,"num_objects_unfound":0,"num_objects_dirty":2963,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2963,"num_write_kb":11852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":251,"num_bytes_recovered":1028096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.003453"},"pg_stats":[{"pgid":"2.f","version":"44'9667","reported_seq":9739,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:07.230350+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:02:07.230350+0000","last_peered":"2024-09-10T16:02:07.230350+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:02:07.230350+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":9667,"log_dups_size":0,"ondisk_log_size":9667,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39596032,"num_objects":9667,"num_object_clones":0,"num_object_copies":19334,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9667,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9667,"num_write_kb":38668,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5408},{"shards":"0,2","objects":4258}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'9798","reported_seq":9910,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:09.790254+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:02:09.790254+0000","last_peered":"2024-09-10T16:02:09.790254+0000","last_clean":"2024-09-10T16:02:09.790254+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:02:09.790254+0000","last_undegraded":"2024-09-10T16:02:09.790254+0000","last_fullsized":"2024-09-10T16:02:09.790254+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":9798,"log_dups_size":0,"ondisk_log_size":9798,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40108998,"num_objects":9794,"num_object_clones":0,"num_object_copies":19588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9794,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":7903,"num_write_kb":31612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'9794","reported_seq":18470,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:09.865158+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:02:09.865158+0000","last_peered":"2024-09-10T16:02:09.865158+0000","last_clean":"2024-09-10T16:02:09.865158+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:02:09.865158+0000","last_undegraded":"2024-09-10T16:02:09.865158+0000","last_fullsized":"2024-09-10T16:02:09.865158+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":9794,"log_dups_size":0,"ondisk_log_size":9794,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40099840,"num_objects":9790,"num_object_clones":0,"num_object_copies":19580,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9790,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7798,"num_write_kb":31192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'9741","reported_seq":9821,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:09.765180+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:02:09.765180+0000","last_peered":"2024-09-10T16:02:09.765180+0000","last_clean":"2024-09-10T16:02:09.765180+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:02:09.765180+0000","last_undegraded":"2024-09-10T16:02:09.765180+0000","last_fullsized":"2024-09-10T16:02:09.765180+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":9741,"log_dups_size":0,"ondisk_log_size":9741,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39899136,"num_objects":9741,"num_object_clones":0,"num_object_copies":19482,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9741,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9741,"num_write_kb":38964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"44'9774","reported_seq":9901,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T16:02:07.279445+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:02:07.279445+0000","last_peered":"2024-09-10T16:02:07.279445+0000","last_clean":"2024-09-10T16:02:07.279445+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:02:07.279445+0000","last_undegraded":"2024-09-10T16:02:07.279445+0000","last_fullsized":"2024-09-10T16:02:07.279445+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":9774,"log_dups_size":0,"ondisk_log_size":9774,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40006115,"num_objects":9768,"num_object_clones":0,"num_object_copies":19536,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9768,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6485,"num_write_kb":25940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"44'9882","reported_seq":10017,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:07.262073+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:02:07.262073+0000","last_peered":"2024-09-10T16:02:07.262073+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:02:07.262073+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":9882,"log_dups_size":0,"ondisk_log_size":9882,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40456675,"num_objects":9878,"num_object_clones":0,"num_object_copies":19756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9878,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6494,"num_write_kb":25976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5507},{"shards":"0,2","objects":4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"44'9870","reported_seq":10005,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:07.341329+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:02:07.341329+0000","last_peered":"2024-09-10T16:02:07.341329+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:02:07.341329+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":9870,"log_dups_size":0,"ondisk_log_size":9870,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40403910,"num_objects":9866,"num_object_clones":0,"num_object_copies":19732,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9866,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":6369,"num_write_kb":25476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5629},{"shards":"0,2","objects":4237}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"44'9987","reported_seq":10114,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T16:02:07.357260+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:02:07.357260+0000","last_peered":"2024-09-10T16:02:07.357260+0000","last_clean":"2024-09-10T16:02:07.357260+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:02:07.357260+0000","last_undegraded":"2024-09-10T16:02:07.357260+0000","last_fullsized":"2024-09-10T16:02:07.357260+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":9987,"log_dups_size":0,"ondisk_log_size":9987,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40898560,"num_objects":9985,"num_object_clones":0,"num_object_copies":19970,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9985,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6834,"num_write_kb":27336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104219+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:02:08.104219+0000","last_peered":"2024-09-10T16:02:08.104219+0000","last_clean":"2024-09-10T16:02:08.104219+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:02:08.104219+0000","last_undegraded":"2024-09-10T16:02:08.104219+0000","last_fullsized":"2024-09-10T16:02:08.104219+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'9585","reported_seq":9723,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:09.781560+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:02:09.781560+0000","last_peered":"2024-09-10T16:02:09.781560+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:02:09.781560+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":9585,"log_dups_size":0,"ondisk_log_size":9585,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39231971,"num_objects":9579,"num_object_clones":0,"num_object_copies":19158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9579,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7404,"num_write_kb":29616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4123},{"shards":"2","objects":5456}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":93,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.102747+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:02:08.102747+0000","last_peered":"2024-09-10T16:02:08.102747+0000","last_clean":"2024-09-10T16:02:08.102747+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:02:08.102747+0000","last_undegraded":"2024-09-10T16:02:08.102747+0000","last_fullsized":"2024-09-10T16:02:08.102747+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"44'9796","reported_seq":9924,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:07.297599+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:02:07.297599+0000","last_peered":"2024-09-10T16:02:07.297599+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:02:07.297599+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":9796,"log_dups_size":0,"ondisk_log_size":9796,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40091648,"num_objects":9788,"num_object_clones":0,"num_object_copies":19576,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9788,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6300,"num_write_kb":25200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5533},{"shards":"0,2","objects":4255}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":101,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T16:02:07.122545+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:02:07.122545+0000","last_peered":"2024-09-10T16:02:07.122545+0000","last_clean":"2024-09-10T16:02:07.122545+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:02:07.122545+0000","last_undegraded":"2024-09-10T16:02:07.122545+0000","last_fullsized":"2024-09-10T16:02:07.122545+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'9765","reported_seq":9899,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:09.696056+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:02:09.696056+0000","last_peered":"2024-09-10T16:02:09.696056+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:02:09.696056+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":9765,"log_dups_size":0,"ondisk_log_size":9765,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39977443,"num_objects":9761,"num_object_clones":0,"num_object_copies":19522,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9761,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7445,"num_write_kb":29780,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4304},{"shards":"2","objects":5457}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104572+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:02:08.104572+0000","last_peered":"2024-09-10T16:02:08.104572+0000","last_clean":"2024-09-10T16:02:08.104572+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:02:08.104572+0000","last_undegraded":"2024-09-10T16:02:08.104572+0000","last_fullsized":"2024-09-10T16:02:08.104572+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'9979","reported_seq":10105,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:09.788039+0000","last_change":"2024-09-10T15:59:16.174113+0000","last_active":"2024-09-10T16:02:09.788039+0000","last_peered":"2024-09-10T16:02:09.788039+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:02:09.788039+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":9979,"log_dups_size":0,"ondisk_log_size":9979,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40849408,"num_objects":9973,"num_object_clones":0,"num_object_copies":19946,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4354,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9973,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7533,"num_write_kb":30132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4352},{"shards":"2","objects":5621}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"44'9732","reported_seq":9864,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:07.290955+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:02:07.290955+0000","last_peered":"2024-09-10T16:02:07.290955+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:02:07.290955+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":9732,"log_dups_size":0,"ondisk_log_size":9732,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39854080,"num_objects":9730,"num_object_clones":0,"num_object_copies":19460,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9730,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6363,"num_write_kb":25452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5426},{"shards":"0,2","objects":4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103276+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:02:08.103276+0000","last_peered":"2024-09-10T16:02:08.103276+0000","last_clean":"2024-09-10T16:02:08.103276+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:02:08.103276+0000","last_undegraded":"2024-09-10T16:02:08.103276+0000","last_fullsized":"2024-09-10T16:02:08.103276+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"44'9620","reported_seq":9795,"reported_epoch":44,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T16:02:07.330921+0000","last_change":"2024-09-10T16:02:06.480033+0000","last_active":"2024-09-10T16:02:07.330921+0000","last_peered":"2024-09-10T16:02:07.330921+0000","last_clean":"2024-09-10T15:59:14.115595+0000","last_became_active":"2024-09-10T15:59:16.244089+0000","last_became_peered":"2024-09-10T15:59:16.244089+0000","last_unstale":"2024-09-10T16:02:07.330921+0000","last_undegraded":"2024-09-10T16:02:07.330921+0000","last_fullsized":"2024-09-10T15:59:15.774437+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":9620,"log_dups_size":0,"ondisk_log_size":9620,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39370752,"num_objects":9612,"num_object_clones":0,"num_object_copies":19224,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":4186,"num_objects_unfound":0,"num_objects_dirty":9612,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6010,"num_write_kb":24040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":69632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5426},{"shards":"0,2","objects":4186}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":93,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103338+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:02:08.103338+0000","last_peered":"2024-09-10T16:02:08.103338+0000","last_clean":"2024-09-10T16:02:08.103338+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:02:08.103338+0000","last_undegraded":"2024-09-10T16:02:08.103338+0000","last_fullsized":"2024-09-10T16:02:08.103338+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"44'9869","reported_seq":10008,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:07.286258+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:02:07.286258+0000","last_peered":"2024-09-10T16:02:07.286258+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:02:07.286258+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":9869,"log_dups_size":0,"ondisk_log_size":9869,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40407040,"num_objects":9865,"num_object_clones":0,"num_object_copies":19730,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9865,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6124,"num_write_kb":24496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5494},{"shards":"0,2","objects":4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104431+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:02:08.104431+0000","last_peered":"2024-09-10T16:02:08.104431+0000","last_clean":"2024-09-10T16:02:08.104431+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:02:08.104431+0000","last_undegraded":"2024-09-10T16:02:08.104431+0000","last_fullsized":"2024-09-10T16:02:08.104431+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104410+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:02:08.104410+0000","last_peered":"2024-09-10T16:02:08.104410+0000","last_clean":"2024-09-10T16:02:08.104410+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:02:08.104410+0000","last_undegraded":"2024-09-10T16:02:08.104410+0000","last_fullsized":"2024-09-10T16:02:08.104410+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"44'9906","reported_seq":10048,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:07.333397+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:02:07.333397+0000","last_peered":"2024-09-10T16:02:07.333397+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:02:07.333397+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":9906,"log_dups_size":0,"ondisk_log_size":9906,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40526790,"num_objects":9896,"num_object_clones":0,"num_object_copies":19792,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9896,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6069,"num_write_kb":24276,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5573},{"shards":"0,2","objects":4323}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":641778398,"num_objects":156693,"num_object_clones":0,"num_object_copies":313386,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42911,"num_objects_misplaced":4186,"num_objects_unfound":0,"num_objects_dirty":156693,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":114539,"num_write_kb":458156,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4289,"num_bytes_recovered":17563648,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1303691264,"data_stored":1303539518,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156765,"ondisk_log_size":156765,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986226,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":725720,"kb_used_data":451076,"kb_used_omap":4,"kb_used_meta":274619,"kb_avail":93646120,"statfs":{"total":96636764160,"available":95893626880,"internally_reserved":0,"allocated":461901824,"data_stored":461613663,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":281210538},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,1,3],"upper_bound":32},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659172,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":503920,"kb_used_data":283420,"kb_used_omap":4,"kb_used_meta":220475,"kb_avail":93867920,"statfs":{"total":96636764160,"available":96120750080,"internally_reserved":0,"allocated":290222080,"data_stored":289886486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":225767086},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,3,5,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182454,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":854520,"kb_used_data":538276,"kb_used_omap":4,"kb_used_meta":316219,"kb_avail":93517320,"statfs":{"total":96636764160,"available":95761735680,"internally_reserved":0,"allocated":551194624,"data_stored":550907597,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":323808938},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,1,4,5,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":551112704,"data_stored":551076574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":290414592,"data_stored":290335106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":462163968,"data_stored":462127838,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:02:11.995 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 219 20 158183 158163 2.82068 1.64062 0.0365718 0.0276898 2024-09-10T16:02:12.995 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout:2024-09-10T16:02:12.994500+0000 min lat: 0.00225581 max lat: 2.21736 avg lat: 0.0277373 2024-09-10T16:02:12.995 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-10T16:02:12.995 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 220 20 158629 158609 2.81577 1.74219 0.0259869 0.0277373 2024-09-10T16:02:13.995 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 221 20 159088 159068 2.81114 1.79297 0.02242 0.0277833 2024-09-10T16:02:14.653 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:02:14.995 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 222 20 159521 159501 2.8061 1.69141 0.0632242 0.0278316 2024-09-10T16:02:15.227 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:02:15.227 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:02:15.252 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":187,"stamp":"2024-09-10T16:02:13.607470+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":647021297,"num_objects":157975,"num_object_clones":0,"num_object_copies":315950,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42911,"num_objects_misplaced":4076,"num_objects_unfound":0,"num_objects_dirty":157975,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":115821,"num_write_kb":463278,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4401,"num_bytes_recovered":18014227,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":157948,"ondisk_log_size":157948,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2094240,"kb_used_data":1278696,"kb_used_omap":12,"kb_used_meta":815475,"kb_avail":281021280,"statfs":{"total":289910292480,"available":287765790720,"internally_reserved":0,"allocated":1309384704,"data_stored":1308468711,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":835046402},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,1,9,7,5,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":14020608,"num_objects":3423,"num_object_clones":0,"num_object_copies":6846,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-4206,"num_objects_misplaced":4006,"num_objects_unfound":0,"num_objects_dirty":3423,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3423,"num_write_kb":13692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":197,"num_bytes_recovered":806912,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002486"},"pg_stats":[{"pgid":"2.f","version":"45'9807","reported_seq":9881,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:12.353891+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:02:12.353891+0000","last_peered":"2024-09-10T16:02:12.353891+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:02:12.353891+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":9807,"log_dups_size":0,"ondisk_log_size":9807,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40169472,"num_objects":9807,"num_object_clones":0,"num_object_copies":19614,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9807,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9807,"num_write_kb":39228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5548},{"shards":"0,2","objects":4258}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'9798","reported_seq":9910,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:09.790254+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:02:09.790254+0000","last_peered":"2024-09-10T16:02:09.790254+0000","last_clean":"2024-09-10T16:02:09.790254+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:02:09.790254+0000","last_undegraded":"2024-09-10T16:02:09.790254+0000","last_fullsized":"2024-09-10T16:02:09.790254+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":9798,"log_dups_size":0,"ondisk_log_size":9798,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40108998,"num_objects":9794,"num_object_clones":0,"num_object_copies":19588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9794,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":7903,"num_write_kb":31612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'9794","reported_seq":18470,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:09.865158+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:02:09.865158+0000","last_peered":"2024-09-10T16:02:09.865158+0000","last_clean":"2024-09-10T16:02:09.865158+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:02:09.865158+0000","last_undegraded":"2024-09-10T16:02:09.865158+0000","last_fullsized":"2024-09-10T16:02:09.865158+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":9794,"log_dups_size":0,"ondisk_log_size":9794,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40099840,"num_objects":9790,"num_object_clones":0,"num_object_copies":19580,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9790,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7798,"num_write_kb":31192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'9741","reported_seq":9821,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:09.765180+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:02:09.765180+0000","last_peered":"2024-09-10T16:02:09.765180+0000","last_clean":"2024-09-10T16:02:09.765180+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:02:09.765180+0000","last_undegraded":"2024-09-10T16:02:09.765180+0000","last_fullsized":"2024-09-10T16:02:09.765180+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":9741,"log_dups_size":0,"ondisk_log_size":9741,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39899136,"num_objects":9741,"num_object_clones":0,"num_object_copies":19482,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9741,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9741,"num_write_kb":38964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'9905","reported_seq":10034,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:12.351164+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:02:12.351164+0000","last_peered":"2024-09-10T16:02:12.351164+0000","last_clean":"2024-09-10T16:02:12.351164+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:02:12.351164+0000","last_undegraded":"2024-09-10T16:02:12.351164+0000","last_fullsized":"2024-09-10T16:02:12.351164+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":9905,"log_dups_size":0,"ondisk_log_size":9905,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40542691,"num_objects":9899,"num_object_clones":0,"num_object_copies":19798,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9899,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6616,"num_write_kb":26464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'9995","reported_seq":10132,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:12.360325+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:02:12.360325+0000","last_peered":"2024-09-10T16:02:12.360325+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:02:12.360325+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":9995,"log_dups_size":0,"ondisk_log_size":9995,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40919523,"num_objects":9991,"num_object_clones":0,"num_object_copies":19982,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9991,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6607,"num_write_kb":26428,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5620},{"shards":"0,2","objects":4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'10000","reported_seq":10137,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:12.309492+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:02:12.309492+0000","last_peered":"2024-09-10T16:02:12.309492+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:02:12.309492+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10000,"log_dups_size":0,"ondisk_log_size":10000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40936390,"num_objects":9996,"num_object_clones":0,"num_object_copies":19992,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9996,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":6499,"num_write_kb":25996,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5759},{"shards":"0,2","objects":4237}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'10117","reported_seq":10246,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:12.388619+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:02:12.388619+0000","last_peered":"2024-09-10T16:02:12.388619+0000","last_clean":"2024-09-10T16:02:12.388619+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:02:12.388619+0000","last_undegraded":"2024-09-10T16:02:12.388619+0000","last_fullsized":"2024-09-10T16:02:12.388619+0000","mapping_epoch":34,"log_start":"36'100","ondisk_log_start":"36'100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41431040,"num_objects":10115,"num_object_clones":0,"num_object_copies":20230,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10115,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6964,"num_write_kb":27856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104219+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:02:08.104219+0000","last_peered":"2024-09-10T16:02:08.104219+0000","last_clean":"2024-09-10T16:02:08.104219+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:02:08.104219+0000","last_undegraded":"2024-09-10T16:02:08.104219+0000","last_fullsized":"2024-09-10T16:02:08.104219+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'9585","reported_seq":9723,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:09.781560+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:02:09.781560+0000","last_peered":"2024-09-10T16:02:09.781560+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:02:09.781560+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":9585,"log_dups_size":0,"ondisk_log_size":9585,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39231971,"num_objects":9579,"num_object_clones":0,"num_object_copies":19158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9579,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7404,"num_write_kb":29616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4123},{"shards":"2","objects":5456}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":93,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.102747+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:02:08.102747+0000","last_peered":"2024-09-10T16:02:08.102747+0000","last_clean":"2024-09-10T16:02:08.102747+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:02:08.102747+0000","last_undegraded":"2024-09-10T16:02:08.102747+0000","last_fullsized":"2024-09-10T16:02:08.102747+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'9915","reported_seq":10045,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:12.350332+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:02:12.350332+0000","last_peered":"2024-09-10T16:02:12.350332+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:02:12.350332+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":9915,"log_dups_size":0,"ondisk_log_size":9915,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40579072,"num_objects":9907,"num_object_clones":0,"num_object_copies":19814,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9907,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6419,"num_write_kb":25676,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5652},{"shards":"0,2","objects":4255}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":104,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.105055+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:02:08.105055+0000","last_peered":"2024-09-10T16:02:08.105055+0000","last_clean":"2024-09-10T16:02:08.105055+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:02:08.105055+0000","last_undegraded":"2024-09-10T16:02:08.105055+0000","last_fullsized":"2024-09-10T16:02:08.105055+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'9765","reported_seq":9899,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:09.696056+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:02:09.696056+0000","last_peered":"2024-09-10T16:02:09.696056+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:02:09.696056+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":9765,"log_dups_size":0,"ondisk_log_size":9765,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39977443,"num_objects":9761,"num_object_clones":0,"num_object_copies":19522,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9761,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7445,"num_write_kb":29780,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4304},{"shards":"2","objects":5457}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104572+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:02:08.104572+0000","last_peered":"2024-09-10T16:02:08.104572+0000","last_clean":"2024-09-10T16:02:08.104572+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:02:08.104572+0000","last_undegraded":"2024-09-10T16:02:08.104572+0000","last_fullsized":"2024-09-10T16:02:08.104572+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'9979","reported_seq":10105,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:09.788039+0000","last_change":"2024-09-10T15:59:16.174113+0000","last_active":"2024-09-10T16:02:09.788039+0000","last_peered":"2024-09-10T16:02:09.788039+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:02:09.788039+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":9979,"log_dups_size":0,"ondisk_log_size":9979,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40849408,"num_objects":9973,"num_object_clones":0,"num_object_copies":19946,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4354,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9973,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7533,"num_write_kb":30132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4352},{"shards":"2","objects":5621}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'9878","reported_seq":10009,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:12.389096+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:02:12.389096+0000","last_peered":"2024-09-10T16:02:12.389096+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:02:12.389096+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":9878,"log_dups_size":0,"ondisk_log_size":9878,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40452096,"num_objects":9876,"num_object_clones":0,"num_object_copies":19752,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9876,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6509,"num_write_kb":26036,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5572},{"shards":"0,2","objects":4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103276+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:02:08.103276+0000","last_peered":"2024-09-10T16:02:08.103276+0000","last_clean":"2024-09-10T16:02:08.103276+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:02:08.103276+0000","last_undegraded":"2024-09-10T16:02:08.103276+0000","last_fullsized":"2024-09-10T16:02:08.103276+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'9756","reported_seq":10153,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T16:02:12.372285+0000","last_change":"2024-09-10T16:02:06.480033+0000","last_active":"2024-09-10T16:02:12.372285+0000","last_peered":"2024-09-10T16:02:12.372285+0000","last_clean":"2024-09-10T15:59:14.115595+0000","last_became_active":"2024-09-10T15:59:16.244089+0000","last_became_peered":"2024-09-10T15:59:16.244089+0000","last_unstale":"2024-09-10T16:02:12.372285+0000","last_undegraded":"2024-09-10T16:02:12.372285+0000","last_fullsized":"2024-09-10T15:59:15.774437+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":9756,"log_dups_size":0,"ondisk_log_size":9756,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39927808,"num_objects":9748,"num_object_clones":0,"num_object_copies":19496,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":4076,"num_objects_unfound":0,"num_objects_dirty":9748,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6146,"num_write_kb":24584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":127,"num_bytes_recovered":520192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5672},{"shards":"0,2","objects":4076}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":93,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103338+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:02:08.103338+0000","last_peered":"2024-09-10T16:02:08.103338+0000","last_clean":"2024-09-10T16:02:08.103338+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:02:08.103338+0000","last_undegraded":"2024-09-10T16:02:08.103338+0000","last_fullsized":"2024-09-10T16:02:08.103338+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'9981","reported_seq":10120,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:12.392823+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:02:12.392823+0000","last_peered":"2024-09-10T16:02:12.392823+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:02:12.392823+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":9981,"log_dups_size":0,"ondisk_log_size":9981,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40865792,"num_objects":9977,"num_object_clones":0,"num_object_copies":19954,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9977,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6236,"num_write_kb":24944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5606},{"shards":"0,2","objects":4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104431+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:02:08.104431+0000","last_peered":"2024-09-10T16:02:08.104431+0000","last_clean":"2024-09-10T16:02:08.104431+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:02:08.104431+0000","last_undegraded":"2024-09-10T16:02:08.104431+0000","last_fullsized":"2024-09-10T16:02:08.104431+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104410+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:02:08.104410+0000","last_peered":"2024-09-10T16:02:08.104410+0000","last_clean":"2024-09-10T16:02:08.104410+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:02:08.104410+0000","last_undegraded":"2024-09-10T16:02:08.104410+0000","last_fullsized":"2024-09-10T16:02:08.104410+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'10029","reported_seq":10173,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:12.381871+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:02:12.381871+0000","last_peered":"2024-09-10T16:02:12.381871+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:02:12.381871+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41030598,"num_objects":10019,"num_object_clones":0,"num_object_copies":20038,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10019,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6192,"num_write_kb":24768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5696},{"shards":"0,2","objects":4323}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":647021278,"num_objects":157973,"num_object_clones":0,"num_object_copies":315946,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42911,"num_objects_misplaced":4076,"num_objects_unfound":0,"num_objects_dirty":157973,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":115819,"num_write_kb":463276,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4399,"num_bytes_recovered":18014208,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1309880320,"data_stored":1309728574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":157945,"ondisk_log_size":157945,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986226,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":725720,"kb_used_data":451076,"kb_used_omap":4,"kb_used_meta":274619,"kb_avail":93646120,"statfs":{"total":96636764160,"available":95893626880,"internally_reserved":0,"allocated":461901824,"data_stored":461613663,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":281210538},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,1,3],"upper_bound":32},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659172,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":503920,"kb_used_data":283420,"kb_used_omap":4,"kb_used_meta":220475,"kb_avail":93867920,"statfs":{"total":96636764160,"available":96120750080,"internally_reserved":0,"allocated":290222080,"data_stored":289886486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":225767086},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,3,5,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182455,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":864600,"kb_used_data":544200,"kb_used_omap":4,"kb_used_meta":320379,"kb_avail":93507240,"statfs":{"total":96636764160,"available":95751413760,"internally_reserved":0,"allocated":557260800,"data_stored":556968562,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":328068778},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,0,3,2,2,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":557301760,"data_stored":557265630,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":290414592,"data_stored":290335106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":462163968,"data_stored":462127838,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:02:15.253 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:02:15.806 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:02:15.808 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:02:15.837 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":188,"stamp":"2024-09-10T16:02:15.608302+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":650179313,"num_objects":158746,"num_object_clones":0,"num_object_copies":317492,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42911,"num_objects_misplaced":4076,"num_objects_unfound":0,"num_objects_dirty":158746,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":116592,"num_write_kb":466362,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4401,"num_bytes_recovered":18014227,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":158619,"ondisk_log_size":158619,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2084228,"kb_used_data":1289492,"kb_used_omap":12,"kb_used_meta":794675,"kb_avail":281031292,"statfs":{"total":289910292480,"available":287776043008,"internally_reserved":0,"allocated":1320439808,"data_stored":1319523815,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":813747202},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,1,0,9,8,8,8],"upper_bound":256},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":17178624,"num_objects":4194,"num_object_clones":0,"num_object_copies":8388,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-4206,"num_objects_misplaced":4006,"num_objects_unfound":0,"num_objects_dirty":4194,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4194,"num_write_kb":16776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":197,"num_bytes_recovered":806912,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.525938"},"pg_stats":[{"pgid":"2.f","version":"45'9807","reported_seq":9881,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:12.353891+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:02:12.353891+0000","last_peered":"2024-09-10T16:02:12.353891+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:02:12.353891+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":9807,"log_dups_size":0,"ondisk_log_size":9807,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40169472,"num_objects":9807,"num_object_clones":0,"num_object_copies":19614,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9807,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9807,"num_write_kb":39228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5548},{"shards":"0,2","objects":4258}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'9925","reported_seq":10037,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:14.758419+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:02:14.758419+0000","last_peered":"2024-09-10T16:02:14.758419+0000","last_clean":"2024-09-10T16:02:14.758419+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:02:14.758419+0000","last_undegraded":"2024-09-10T16:02:14.758419+0000","last_fullsized":"2024-09-10T16:02:14.758419+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":9925,"log_dups_size":0,"ondisk_log_size":9925,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40629190,"num_objects":9921,"num_object_clones":0,"num_object_copies":19842,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9921,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":8030,"num_write_kb":32120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'9931","reported_seq":18607,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:14.885899+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:02:14.885899+0000","last_peered":"2024-09-10T16:02:14.885899+0000","last_clean":"2024-09-10T16:02:14.885899+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:02:14.885899+0000","last_undegraded":"2024-09-10T16:02:14.885899+0000","last_fullsized":"2024-09-10T16:02:14.885899+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":9931,"log_dups_size":0,"ondisk_log_size":9931,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40660992,"num_objects":9927,"num_object_clones":0,"num_object_copies":19854,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9927,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7935,"num_write_kb":31740,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'9858","reported_seq":9938,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:14.621803+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:02:14.621803+0000","last_peered":"2024-09-10T16:02:14.621803+0000","last_clean":"2024-09-10T16:02:14.621803+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:02:14.621803+0000","last_undegraded":"2024-09-10T16:02:14.621803+0000","last_fullsized":"2024-09-10T16:02:14.621803+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":9858,"log_dups_size":0,"ondisk_log_size":9858,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40378368,"num_objects":9858,"num_object_clones":0,"num_object_copies":19716,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9858,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9858,"num_write_kb":39432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'9905","reported_seq":10034,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:12.351164+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:02:12.351164+0000","last_peered":"2024-09-10T16:02:12.351164+0000","last_clean":"2024-09-10T16:02:12.351164+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:02:12.351164+0000","last_undegraded":"2024-09-10T16:02:12.351164+0000","last_fullsized":"2024-09-10T16:02:12.351164+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":9905,"log_dups_size":0,"ondisk_log_size":9905,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40542691,"num_objects":9899,"num_object_clones":0,"num_object_copies":19798,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9899,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6616,"num_write_kb":26464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'9995","reported_seq":10132,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:12.360325+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:02:12.360325+0000","last_peered":"2024-09-10T16:02:12.360325+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:02:12.360325+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":9995,"log_dups_size":0,"ondisk_log_size":9995,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40919523,"num_objects":9991,"num_object_clones":0,"num_object_copies":19982,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9991,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6607,"num_write_kb":26428,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5620},{"shards":"0,2","objects":4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'10000","reported_seq":10137,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:12.309492+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:02:12.309492+0000","last_peered":"2024-09-10T16:02:12.309492+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:02:12.309492+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10000,"log_dups_size":0,"ondisk_log_size":10000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40936390,"num_objects":9996,"num_object_clones":0,"num_object_copies":19992,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9996,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":6499,"num_write_kb":25996,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5759},{"shards":"0,2","objects":4237}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'10117","reported_seq":10246,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:12.388619+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:02:12.388619+0000","last_peered":"2024-09-10T16:02:12.388619+0000","last_clean":"2024-09-10T16:02:12.388619+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:02:12.388619+0000","last_undegraded":"2024-09-10T16:02:12.388619+0000","last_fullsized":"2024-09-10T16:02:12.388619+0000","mapping_epoch":34,"log_start":"36'100","ondisk_log_start":"36'100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41431040,"num_objects":10115,"num_object_clones":0,"num_object_copies":20230,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10115,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6964,"num_write_kb":27856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104219+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:02:08.104219+0000","last_peered":"2024-09-10T16:02:08.104219+0000","last_clean":"2024-09-10T16:02:08.104219+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:02:08.104219+0000","last_undegraded":"2024-09-10T16:02:08.104219+0000","last_fullsized":"2024-09-10T16:02:08.104219+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'9715","reported_seq":9854,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:14.723055+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:02:14.723055+0000","last_peered":"2024-09-10T16:02:14.723055+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:02:14.723055+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":9715,"log_dups_size":0,"ondisk_log_size":9715,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39764451,"num_objects":9709,"num_object_clones":0,"num_object_copies":19418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9709,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7534,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4123},{"shards":"2","objects":5586}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":93,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.102747+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:02:08.102747+0000","last_peered":"2024-09-10T16:02:08.102747+0000","last_clean":"2024-09-10T16:02:08.102747+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:02:08.102747+0000","last_undegraded":"2024-09-10T16:02:08.102747+0000","last_fullsized":"2024-09-10T16:02:08.102747+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'9915","reported_seq":10045,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:12.350332+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:02:12.350332+0000","last_peered":"2024-09-10T16:02:12.350332+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:02:12.350332+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":9915,"log_dups_size":0,"ondisk_log_size":9915,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40579072,"num_objects":9907,"num_object_clones":0,"num_object_copies":19814,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9907,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6419,"num_write_kb":25676,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5652},{"shards":"0,2","objects":4255}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":104,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.105055+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:02:08.105055+0000","last_peered":"2024-09-10T16:02:08.105055+0000","last_clean":"2024-09-10T16:02:08.105055+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:02:08.105055+0000","last_undegraded":"2024-09-10T16:02:08.105055+0000","last_fullsized":"2024-09-10T16:02:08.105055+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'9897","reported_seq":10031,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:14.743493+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:02:14.743493+0000","last_peered":"2024-09-10T16:02:14.743493+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:02:14.743493+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":9897,"log_dups_size":0,"ondisk_log_size":9897,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40518115,"num_objects":9893,"num_object_clones":0,"num_object_copies":19786,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9893,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7577,"num_write_kb":30308,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4304},{"shards":"2","objects":5589}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104572+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:02:08.104572+0000","last_peered":"2024-09-10T16:02:08.104572+0000","last_clean":"2024-09-10T16:02:08.104572+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:02:08.104572+0000","last_undegraded":"2024-09-10T16:02:08.104572+0000","last_fullsized":"2024-09-10T16:02:08.104572+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'10107","reported_seq":10235,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:14.645353+0000","last_change":"2024-09-10T15:59:16.174113+0000","last_active":"2024-09-10T16:02:14.645353+0000","last_peered":"2024-09-10T16:02:14.645353+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:02:14.645353+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"36'100","ondisk_log_start":"36'100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41373696,"num_objects":10101,"num_object_clones":0,"num_object_copies":20202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4354,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10101,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7661,"num_write_kb":30644,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4352},{"shards":"2","objects":5749}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'9878","reported_seq":10009,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:12.389096+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:02:12.389096+0000","last_peered":"2024-09-10T16:02:12.389096+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:02:12.389096+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":9878,"log_dups_size":0,"ondisk_log_size":9878,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40452096,"num_objects":9876,"num_object_clones":0,"num_object_copies":19752,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9876,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6509,"num_write_kb":26036,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5572},{"shards":"0,2","objects":4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103276+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:02:08.103276+0000","last_peered":"2024-09-10T16:02:08.103276+0000","last_clean":"2024-09-10T16:02:08.103276+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:02:08.103276+0000","last_undegraded":"2024-09-10T16:02:08.103276+0000","last_fullsized":"2024-09-10T16:02:08.103276+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'9756","reported_seq":10153,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T16:02:12.372285+0000","last_change":"2024-09-10T16:02:06.480033+0000","last_active":"2024-09-10T16:02:12.372285+0000","last_peered":"2024-09-10T16:02:12.372285+0000","last_clean":"2024-09-10T15:59:14.115595+0000","last_became_active":"2024-09-10T15:59:16.244089+0000","last_became_peered":"2024-09-10T15:59:16.244089+0000","last_unstale":"2024-09-10T16:02:12.372285+0000","last_undegraded":"2024-09-10T16:02:12.372285+0000","last_fullsized":"2024-09-10T15:59:15.774437+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":9756,"log_dups_size":0,"ondisk_log_size":9756,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39927808,"num_objects":9748,"num_object_clones":0,"num_object_copies":19496,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":4076,"num_objects_unfound":0,"num_objects_dirty":9748,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6146,"num_write_kb":24584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":127,"num_bytes_recovered":520192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5672},{"shards":"0,2","objects":4076}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":93,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103338+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:02:08.103338+0000","last_peered":"2024-09-10T16:02:08.103338+0000","last_clean":"2024-09-10T16:02:08.103338+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:02:08.103338+0000","last_undegraded":"2024-09-10T16:02:08.103338+0000","last_fullsized":"2024-09-10T16:02:08.103338+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'9981","reported_seq":10120,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:12.392823+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:02:12.392823+0000","last_peered":"2024-09-10T16:02:12.392823+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:02:12.392823+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":9981,"log_dups_size":0,"ondisk_log_size":9981,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40865792,"num_objects":9977,"num_object_clones":0,"num_object_copies":19954,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9977,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6236,"num_write_kb":24944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5606},{"shards":"0,2","objects":4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104431+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:02:08.104431+0000","last_peered":"2024-09-10T16:02:08.104431+0000","last_clean":"2024-09-10T16:02:08.104431+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:02:08.104431+0000","last_undegraded":"2024-09-10T16:02:08.104431+0000","last_fullsized":"2024-09-10T16:02:08.104431+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104410+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:02:08.104410+0000","last_peered":"2024-09-10T16:02:08.104410+0000","last_clean":"2024-09-10T16:02:08.104410+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:02:08.104410+0000","last_undegraded":"2024-09-10T16:02:08.104410+0000","last_fullsized":"2024-09-10T16:02:08.104410+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'10029","reported_seq":10173,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:12.381871+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:02:12.381871+0000","last_peered":"2024-09-10T16:02:12.381871+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:02:12.381871+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41030598,"num_objects":10019,"num_object_clones":0,"num_object_copies":20038,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10019,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6192,"num_write_kb":24768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5696},{"shards":"0,2","objects":4323}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":650179294,"num_objects":158744,"num_object_clones":0,"num_object_copies":317488,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42911,"num_objects_misplaced":4076,"num_objects_unfound":0,"num_objects_dirty":158744,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":116590,"num_write_kb":466360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4399,"num_bytes_recovered":18014208,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1321340928,"data_stored":1321189182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":158616,"ondisk_log_size":158616,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986227,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":717400,"kb_used_data":455240,"kb_used_omap":4,"kb_used_meta":262139,"kb_avail":93654440,"statfs":{"total":96636764160,"available":95902146560,"internally_reserved":0,"allocated":466165760,"data_stored":465877599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":268431018},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,0,4,1,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659173,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":502228,"kb_used_data":290052,"kb_used_omap":4,"kb_used_meta":212155,"kb_avail":93869612,"statfs":{"total":96636764160,"available":96122482688,"internally_reserved":0,"allocated":297013248,"data_stored":296677654,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":217247406},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,0,2,5,5,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182455,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":864600,"kb_used_data":544200,"kb_used_omap":4,"kb_used_meta":320379,"kb_avail":93507240,"statfs":{"total":96636764160,"available":95751413760,"internally_reserved":0,"allocated":557260800,"data_stored":556968562,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":328068778},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,0,3,2,2,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":557301760,"data_stored":557265630,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":297566208,"data_stored":297486722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":466472960,"data_stored":466436830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:02:15.839 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-10T16:02:15.839 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "45'9807", 'reported_seq': 9881, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:02:12.353891+0000', 'last_change': '2024-09-10T15:59:16.281558+0000', 'last_active': '2024-09-10T16:02:12.353891+0000', 'last_peered': '2024-09-10T16:02:12.353891+0000', 'last_clean': '2024-09-10T15:59:14.132456+0000', 'last_became_active': '2024-09-10T15:59:16.274407+0000', 'last_became_peered': '2024-09-10T15:59:16.274407+0000', 'last_unstale': '2024-09-10T16:02:12.353891+0000', 'last_undegraded': '2024-09-10T15:59:15.851279+0000', 'last_fullsized': '2024-09-10T15:59:15.774356+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'objects_scrubbed': 0, 'log_size': 9807, 'log_dups_size': 0, 'ondisk_log_size': 9807, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40169472, 'num_objects': 9807, 'num_object_clones': 0, 'num_object_copies': 19614, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4259, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9807, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9807, 'num_write_kb': 39228, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5548}, {'shards': '0,2', 'objects': 4258}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:02:15.839 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-10T16:02:15.839 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "45'9995", 'reported_seq': 10132, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:02:12.360325+0000', 'last_change': '2024-09-10T15:59:16.308264+0000', 'last_active': '2024-09-10T16:02:12.360325+0000', 'last_peered': '2024-09-10T16:02:12.360325+0000', 'last_clean': '2024-09-10T15:59:14.111551+0000', 'last_became_active': '2024-09-10T15:59:16.300040+0000', 'last_became_peered': '2024-09-10T15:59:16.300040+0000', 'last_unstale': '2024-09-10T16:02:12.360325+0000', 'last_undegraded': '2024-09-10T15:59:15.878234+0000', 'last_fullsized': '2024-09-10T15:59:15.769328+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3442", 'last_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'objects_scrubbed': 3384, 'log_size': 9995, 'log_dups_size': 0, 'ondisk_log_size': 9995, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1292, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40919523, 'num_objects': 9991, 'num_object_clones': 0, 'num_object_copies': 19982, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4372, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9991, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 6607, 'num_write_kb': 26428, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5620}, {'shards': '0,2', 'objects': 4371}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:02:15.839 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-10T16:02:15.839 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "45'10000", 'reported_seq': 10137, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:02:12.309492+0000', 'last_change': '2024-09-10T15:59:16.331899+0000', 'last_active': '2024-09-10T16:02:12.309492+0000', 'last_peered': '2024-09-10T16:02:12.309492+0000', 'last_clean': '2024-09-10T15:59:14.139436+0000', 'last_became_active': '2024-09-10T15:59:16.318715+0000', 'last_became_peered': '2024-09-10T15:59:16.318715+0000', 'last_unstale': '2024-09-10T16:02:12.309492+0000', 'last_undegraded': '2024-09-10T15:59:15.877524+0000', 'last_fullsized': '2024-09-10T15:59:15.769042+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3537", 'last_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'objects_scrubbed': 3497, 'log_size': 10000, 'log_dups_size': 0, 'ondisk_log_size': 10000, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1224, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40936390, 'num_objects': 9996, 'num_object_clones': 0, 'num_object_copies': 19992, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4237, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9996, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 6499, 'num_write_kb': 25996, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5759}, {'shards': '0,2', 'objects': 4237}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:02:15.839 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-10T16:02:15.839 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "45'9715", 'reported_seq': 9854, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:02:14.723055+0000', 'last_change': '2024-09-10T15:59:16.175680+0000', 'last_active': '2024-09-10T16:02:14.723055+0000', 'last_peered': '2024-09-10T16:02:14.723055+0000', 'last_clean': '2024-09-10T15:59:14.088187+0000', 'last_became_active': '2024-09-10T15:59:16.165810+0000', 'last_became_peered': '2024-09-10T15:59:16.165810+0000', 'last_unstale': '2024-09-10T16:02:14.723055+0000', 'last_undegraded': '2024-09-10T15:59:15.822098+0000', 'last_fullsized': '2024-09-10T15:59:15.769123+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2218", 'last_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'objects_scrubbed': 2175, 'log_size': 9715, 'log_dups_size': 0, 'ondisk_log_size': 9715, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 825, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39764451, 'num_objects': 9709, 'num_object_clones': 0, 'num_object_copies': 19418, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4124, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9709, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 7534, 'num_write_kb': 30136, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4123}, {'shards': '2', 'objects': 5586}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:02:15.840 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-10T16:02:15.840 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "45'9915", 'reported_seq': 10045, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:02:12.350332+0000', 'last_change': '2024-09-10T15:59:16.277742+0000', 'last_active': '2024-09-10T16:02:12.350332+0000', 'last_peered': '2024-09-10T16:02:12.350332+0000', 'last_clean': '2024-09-10T15:59:14.129081+0000', 'last_became_active': '2024-09-10T15:59:16.269835+0000', 'last_became_peered': '2024-09-10T15:59:16.269835+0000', 'last_unstale': '2024-09-10T16:02:12.350332+0000', 'last_undegraded': '2024-09-10T15:59:15.860857+0000', 'last_fullsized': '2024-09-10T15:59:15.771410+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3557", 'last_scrub_stamp': '2024-09-10T15:59:02.493678+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:02.493678+0000', 'objects_scrubbed': 3488, 'log_size': 9915, 'log_dups_size': 0, 'ondisk_log_size': 9915, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1472, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40579072, 'num_objects': 9907, 'num_object_clones': 0, 'num_object_copies': 19814, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4257, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9907, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6419, 'num_write_kb': 25676, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5652}, {'shards': '0,2', 'objects': 4255}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:02:15.840 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-10T16:02:15.840 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "45'9897", 'reported_seq': 10031, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:02:14.743493+0000', 'last_change': '2024-09-10T15:59:16.284311+0000', 'last_active': '2024-09-10T16:02:14.743493+0000', 'last_peered': '2024-09-10T16:02:14.743493+0000', 'last_clean': '2024-09-10T15:59:14.116041+0000', 'last_became_active': '2024-09-10T15:59:16.274543+0000', 'last_became_peered': '2024-09-10T15:59:16.274543+0000', 'last_unstale': '2024-09-10T16:02:14.743493+0000', 'last_undegraded': '2024-09-10T15:59:15.812484+0000', 'last_fullsized': '2024-09-10T15:59:15.770885+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2356", 'last_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'objects_scrubbed': 2316, 'log_size': 9897, 'log_dups_size': 0, 'ondisk_log_size': 9897, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 783, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40518115, 'num_objects': 9893, 'num_object_clones': 0, 'num_object_copies': 19786, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9893, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 7577, 'num_write_kb': 30308, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4304}, {'shards': '2', 'objects': 5589}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:02:15.840 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-10T16:02:15.840 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "45'10107", 'reported_seq': 10235, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:02:14.645353+0000', 'last_change': '2024-09-10T15:59:16.174113+0000', 'last_active': '2024-09-10T16:02:14.645353+0000', 'last_peered': '2024-09-10T16:02:14.645353+0000', 'last_clean': '2024-09-10T15:59:14.110654+0000', 'last_became_active': '2024-09-10T15:59:16.164645+0000', 'last_became_peered': '2024-09-10T15:59:16.164645+0000', 'last_unstale': '2024-09-10T16:02:14.645353+0000', 'last_undegraded': '2024-09-10T15:59:15.803593+0000', 'last_fullsized': '2024-09-10T15:59:15.769083+0000', 'mapping_epoch': 42, 'log_start': "36'100", 'ondisk_log_start': "36'100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2486", 'last_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'objects_scrubbed': 2440, 'log_size': 10007, 'log_dups_size': 0, 'ondisk_log_size': 10007, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 792, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 41373696, 'num_objects': 10101, 'num_object_clones': 0, 'num_object_copies': 20202, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4354, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10101, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7661, 'num_write_kb': 30644, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4352}, {'shards': '2', 'objects': 5749}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:02:15.840 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-10T16:02:15.840 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "45'9878", 'reported_seq': 10009, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:02:12.389096+0000', 'last_change': '2024-09-10T15:59:16.315641+0000', 'last_active': '2024-09-10T16:02:12.389096+0000', 'last_peered': '2024-09-10T16:02:12.389096+0000', 'last_clean': '2024-09-10T15:59:14.117001+0000', 'last_became_active': '2024-09-10T15:59:16.302245+0000', 'last_became_peered': '2024-09-10T15:59:16.302245+0000', 'last_unstale': '2024-09-10T16:02:12.389096+0000', 'last_undegraded': '2024-09-10T15:59:15.845012+0000', 'last_fullsized': '2024-09-10T15:59:15.770337+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3404", 'last_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'objects_scrubbed': 3367, 'log_size': 9878, 'log_dups_size': 0, 'ondisk_log_size': 9878, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 983, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40452096, 'num_objects': 9876, 'num_object_clones': 0, 'num_object_copies': 19752, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9876, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6509, 'num_write_kb': 26036, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5572}, {'shards': '0,2', 'objects': 4304}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:02:15.840 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-10T16:02:15.840 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "45'9756", 'reported_seq': 10153, 'reported_epoch': 45, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-10T16:02:12.372285+0000', 'last_change': '2024-09-10T16:02:06.480033+0000', 'last_active': '2024-09-10T16:02:12.372285+0000', 'last_peered': '2024-09-10T16:02:12.372285+0000', 'last_clean': '2024-09-10T15:59:14.115595+0000', 'last_became_active': '2024-09-10T15:59:16.244089+0000', 'last_became_peered': '2024-09-10T15:59:16.244089+0000', 'last_unstale': '2024-09-10T16:02:12.372285+0000', 'last_undegraded': '2024-09-10T16:02:12.372285+0000', 'last_fullsized': '2024-09-10T15:59:15.774437+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3653", 'last_scrub_stamp': '2024-09-10T15:59:05.298468+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:05.298468+0000', 'objects_scrubbed': 3602, 'log_size': 9756, 'log_dups_size': 0, 'ondisk_log_size': 9756, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1289, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39927808, 'num_objects': 9748, 'num_object_clones': 0, 'num_object_copies': 19496, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 4076, 'num_objects_unfound': 0, 'num_objects_dirty': 9748, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6146, 'num_write_kb': 24584, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 127, 'num_bytes_recovered': 520192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5672}, {'shards': '0,2', 'objects': 4076}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:02:15.840 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-10T16:02:15.841 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "45'9981", 'reported_seq': 10120, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:02:12.392823+0000', 'last_change': '2024-09-10T15:59:16.311291+0000', 'last_active': '2024-09-10T16:02:12.392823+0000', 'last_peered': '2024-09-10T16:02:12.392823+0000', 'last_clean': '2024-09-10T15:59:14.116363+0000', 'last_became_active': '2024-09-10T15:59:16.301244+0000', 'last_became_peered': '2024-09-10T15:59:16.301244+0000', 'last_unstale': '2024-09-10T16:02:12.392823+0000', 'last_undegraded': '2024-09-10T15:59:15.883434+0000', 'last_fullsized': '2024-09-10T15:59:15.772756+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3789", 'last_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'objects_scrubbed': 3741, 'log_size': 9981, 'log_dups_size': 0, 'ondisk_log_size': 9981, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1440, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40865792, 'num_objects': 9977, 'num_object_clones': 0, 'num_object_copies': 19954, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4373, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9977, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6236, 'num_write_kb': 24944, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5606}, {'shards': '0,2', 'objects': 4371}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:02:15.841 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-10T16:02:15.841 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "45'10029", 'reported_seq': 10173, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:02:12.381871+0000', 'last_change': '2024-09-10T15:59:16.313409+0000', 'last_active': '2024-09-10T16:02:12.381871+0000', 'last_peered': '2024-09-10T16:02:12.381871+0000', 'last_clean': '2024-09-10T15:59:14.129780+0000', 'last_became_active': '2024-09-10T15:59:16.300932+0000', 'last_became_peered': '2024-09-10T15:59:16.300932+0000', 'last_unstale': '2024-09-10T16:02:12.381871+0000', 'last_undegraded': '2024-09-10T15:59:15.887823+0000', 'last_fullsized': '2024-09-10T15:59:15.768513+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3880", 'last_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'objects_scrubbed': 3827, 'log_size': 10029, 'log_dups_size': 0, 'ondisk_log_size': 10029, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1397, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 41030598, 'num_objects': 10019, 'num_object_clones': 0, 'num_object_copies': 20038, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4325, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10019, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6192, 'num_write_kb': 24768, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5696}, {'shards': '0,2', 'objects': 4323}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:02:15.844 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T16:02:15.995 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 223 20 159922 159902 2.80054 1.56641 0.106982 0.0278874 2024-09-10T16:02:16.521 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:02:16.522 INFO:teuthology.orchestra.run.smithi115.stdout:{"fsid":"7effee29-df05-4351-9741-ee203564ed81","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 42911/317492 objects degraded (13.516%), 10 pgs degraded, 11 pgs undersized","count":21},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":300,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983954,"num_in_osds":3,"osd_in_since":1725983947,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":158746,"data_bytes":650179313,"bytes_used":2134249472,"bytes_avail":287776043008,"bytes_total":289910292480,"degraded_objects":42911,"degraded_total":317492,"degraded_ratio":0.13515616141509076,"misplaced_objects":4076,"misplaced_total":317492,"misplaced_ratio":0.012838118755748175,"recovering_objects_per_sec":23,"recovering_bytes_per_sec":94642,"recovering_keys_per_sec":0,"num_objects_recovered":197,"num_bytes_recovered":806912,"num_keys_recovered":0,"write_bytes_sec":2014866,"read_op_per_sec":0,"write_op_per_sec":491},"fsmap":{"epoch":1,"btime":"2024-09-10T15:57:14:967120+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T16:00:13.560776+0000","services":{}},"progress_events":{}} 2024-09-10T16:02:16.546 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-10T16:02:16.546 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:02:16.995 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 224 20 160319 160299 2.79496 1.55078 0.0450173 0.0279447 2024-09-10T16:02:17.113 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:02:17.114 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:02:17.145 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":188,"stamp":"2024-09-10T16:02:15.608302+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":650179313,"num_objects":158746,"num_object_clones":0,"num_object_copies":317492,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42911,"num_objects_misplaced":4076,"num_objects_unfound":0,"num_objects_dirty":158746,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":116592,"num_write_kb":466362,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4401,"num_bytes_recovered":18014227,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":158619,"ondisk_log_size":158619,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2084228,"kb_used_data":1289492,"kb_used_omap":12,"kb_used_meta":794675,"kb_avail":281031292,"statfs":{"total":289910292480,"available":287776043008,"internally_reserved":0,"allocated":1320439808,"data_stored":1319523815,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":813747202},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,1,0,9,8,8,8],"upper_bound":256},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":17178624,"num_objects":4194,"num_object_clones":0,"num_object_copies":8388,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-4206,"num_objects_misplaced":4006,"num_objects_unfound":0,"num_objects_dirty":4194,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4194,"num_write_kb":16776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":197,"num_bytes_recovered":806912,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.525938"},"pg_stats":[{"pgid":"2.f","version":"45'9807","reported_seq":9881,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:12.353891+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:02:12.353891+0000","last_peered":"2024-09-10T16:02:12.353891+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:02:12.353891+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":9807,"log_dups_size":0,"ondisk_log_size":9807,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40169472,"num_objects":9807,"num_object_clones":0,"num_object_copies":19614,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9807,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9807,"num_write_kb":39228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5548},{"shards":"0,2","objects":4258}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'9925","reported_seq":10037,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:14.758419+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:02:14.758419+0000","last_peered":"2024-09-10T16:02:14.758419+0000","last_clean":"2024-09-10T16:02:14.758419+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:02:14.758419+0000","last_undegraded":"2024-09-10T16:02:14.758419+0000","last_fullsized":"2024-09-10T16:02:14.758419+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":9925,"log_dups_size":0,"ondisk_log_size":9925,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40629190,"num_objects":9921,"num_object_clones":0,"num_object_copies":19842,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9921,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":8030,"num_write_kb":32120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'9931","reported_seq":18607,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:14.885899+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:02:14.885899+0000","last_peered":"2024-09-10T16:02:14.885899+0000","last_clean":"2024-09-10T16:02:14.885899+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:02:14.885899+0000","last_undegraded":"2024-09-10T16:02:14.885899+0000","last_fullsized":"2024-09-10T16:02:14.885899+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":9931,"log_dups_size":0,"ondisk_log_size":9931,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40660992,"num_objects":9927,"num_object_clones":0,"num_object_copies":19854,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9927,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7935,"num_write_kb":31740,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'9858","reported_seq":9938,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:14.621803+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:02:14.621803+0000","last_peered":"2024-09-10T16:02:14.621803+0000","last_clean":"2024-09-10T16:02:14.621803+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:02:14.621803+0000","last_undegraded":"2024-09-10T16:02:14.621803+0000","last_fullsized":"2024-09-10T16:02:14.621803+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":9858,"log_dups_size":0,"ondisk_log_size":9858,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40378368,"num_objects":9858,"num_object_clones":0,"num_object_copies":19716,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9858,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9858,"num_write_kb":39432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'9905","reported_seq":10034,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:12.351164+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:02:12.351164+0000","last_peered":"2024-09-10T16:02:12.351164+0000","last_clean":"2024-09-10T16:02:12.351164+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:02:12.351164+0000","last_undegraded":"2024-09-10T16:02:12.351164+0000","last_fullsized":"2024-09-10T16:02:12.351164+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":9905,"log_dups_size":0,"ondisk_log_size":9905,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40542691,"num_objects":9899,"num_object_clones":0,"num_object_copies":19798,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9899,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6616,"num_write_kb":26464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'9995","reported_seq":10132,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:12.360325+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:02:12.360325+0000","last_peered":"2024-09-10T16:02:12.360325+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:02:12.360325+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":9995,"log_dups_size":0,"ondisk_log_size":9995,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40919523,"num_objects":9991,"num_object_clones":0,"num_object_copies":19982,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9991,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6607,"num_write_kb":26428,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5620},{"shards":"0,2","objects":4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'10000","reported_seq":10137,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:12.309492+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:02:12.309492+0000","last_peered":"2024-09-10T16:02:12.309492+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:02:12.309492+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10000,"log_dups_size":0,"ondisk_log_size":10000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40936390,"num_objects":9996,"num_object_clones":0,"num_object_copies":19992,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9996,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":6499,"num_write_kb":25996,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5759},{"shards":"0,2","objects":4237}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'10117","reported_seq":10246,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:12.388619+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:02:12.388619+0000","last_peered":"2024-09-10T16:02:12.388619+0000","last_clean":"2024-09-10T16:02:12.388619+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:02:12.388619+0000","last_undegraded":"2024-09-10T16:02:12.388619+0000","last_fullsized":"2024-09-10T16:02:12.388619+0000","mapping_epoch":34,"log_start":"36'100","ondisk_log_start":"36'100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41431040,"num_objects":10115,"num_object_clones":0,"num_object_copies":20230,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10115,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6964,"num_write_kb":27856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104219+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:02:08.104219+0000","last_peered":"2024-09-10T16:02:08.104219+0000","last_clean":"2024-09-10T16:02:08.104219+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:02:08.104219+0000","last_undegraded":"2024-09-10T16:02:08.104219+0000","last_fullsized":"2024-09-10T16:02:08.104219+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'9715","reported_seq":9854,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:14.723055+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:02:14.723055+0000","last_peered":"2024-09-10T16:02:14.723055+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:02:14.723055+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":9715,"log_dups_size":0,"ondisk_log_size":9715,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39764451,"num_objects":9709,"num_object_clones":0,"num_object_copies":19418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9709,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7534,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4123},{"shards":"2","objects":5586}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":93,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.102747+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:02:08.102747+0000","last_peered":"2024-09-10T16:02:08.102747+0000","last_clean":"2024-09-10T16:02:08.102747+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:02:08.102747+0000","last_undegraded":"2024-09-10T16:02:08.102747+0000","last_fullsized":"2024-09-10T16:02:08.102747+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'9915","reported_seq":10045,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:12.350332+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:02:12.350332+0000","last_peered":"2024-09-10T16:02:12.350332+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:02:12.350332+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":9915,"log_dups_size":0,"ondisk_log_size":9915,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40579072,"num_objects":9907,"num_object_clones":0,"num_object_copies":19814,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9907,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6419,"num_write_kb":25676,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5652},{"shards":"0,2","objects":4255}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":104,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.105055+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:02:08.105055+0000","last_peered":"2024-09-10T16:02:08.105055+0000","last_clean":"2024-09-10T16:02:08.105055+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:02:08.105055+0000","last_undegraded":"2024-09-10T16:02:08.105055+0000","last_fullsized":"2024-09-10T16:02:08.105055+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'9897","reported_seq":10031,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:14.743493+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:02:14.743493+0000","last_peered":"2024-09-10T16:02:14.743493+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:02:14.743493+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":9897,"log_dups_size":0,"ondisk_log_size":9897,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40518115,"num_objects":9893,"num_object_clones":0,"num_object_copies":19786,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9893,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7577,"num_write_kb":30308,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4304},{"shards":"2","objects":5589}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104572+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:02:08.104572+0000","last_peered":"2024-09-10T16:02:08.104572+0000","last_clean":"2024-09-10T16:02:08.104572+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:02:08.104572+0000","last_undegraded":"2024-09-10T16:02:08.104572+0000","last_fullsized":"2024-09-10T16:02:08.104572+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'10107","reported_seq":10235,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:14.645353+0000","last_change":"2024-09-10T15:59:16.174113+0000","last_active":"2024-09-10T16:02:14.645353+0000","last_peered":"2024-09-10T16:02:14.645353+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:02:14.645353+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"36'100","ondisk_log_start":"36'100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41373696,"num_objects":10101,"num_object_clones":0,"num_object_copies":20202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4354,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10101,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7661,"num_write_kb":30644,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4352},{"shards":"2","objects":5749}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'9878","reported_seq":10009,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:12.389096+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:02:12.389096+0000","last_peered":"2024-09-10T16:02:12.389096+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:02:12.389096+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":9878,"log_dups_size":0,"ondisk_log_size":9878,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40452096,"num_objects":9876,"num_object_clones":0,"num_object_copies":19752,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9876,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6509,"num_write_kb":26036,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5572},{"shards":"0,2","objects":4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103276+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:02:08.103276+0000","last_peered":"2024-09-10T16:02:08.103276+0000","last_clean":"2024-09-10T16:02:08.103276+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:02:08.103276+0000","last_undegraded":"2024-09-10T16:02:08.103276+0000","last_fullsized":"2024-09-10T16:02:08.103276+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'9756","reported_seq":10153,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T16:02:12.372285+0000","last_change":"2024-09-10T16:02:06.480033+0000","last_active":"2024-09-10T16:02:12.372285+0000","last_peered":"2024-09-10T16:02:12.372285+0000","last_clean":"2024-09-10T15:59:14.115595+0000","last_became_active":"2024-09-10T15:59:16.244089+0000","last_became_peered":"2024-09-10T15:59:16.244089+0000","last_unstale":"2024-09-10T16:02:12.372285+0000","last_undegraded":"2024-09-10T16:02:12.372285+0000","last_fullsized":"2024-09-10T15:59:15.774437+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":9756,"log_dups_size":0,"ondisk_log_size":9756,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39927808,"num_objects":9748,"num_object_clones":0,"num_object_copies":19496,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":4076,"num_objects_unfound":0,"num_objects_dirty":9748,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6146,"num_write_kb":24584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":127,"num_bytes_recovered":520192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5672},{"shards":"0,2","objects":4076}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":93,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103338+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:02:08.103338+0000","last_peered":"2024-09-10T16:02:08.103338+0000","last_clean":"2024-09-10T16:02:08.103338+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:02:08.103338+0000","last_undegraded":"2024-09-10T16:02:08.103338+0000","last_fullsized":"2024-09-10T16:02:08.103338+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'9981","reported_seq":10120,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:12.392823+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:02:12.392823+0000","last_peered":"2024-09-10T16:02:12.392823+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:02:12.392823+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":9981,"log_dups_size":0,"ondisk_log_size":9981,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40865792,"num_objects":9977,"num_object_clones":0,"num_object_copies":19954,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9977,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6236,"num_write_kb":24944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5606},{"shards":"0,2","objects":4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104431+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:02:08.104431+0000","last_peered":"2024-09-10T16:02:08.104431+0000","last_clean":"2024-09-10T16:02:08.104431+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:02:08.104431+0000","last_undegraded":"2024-09-10T16:02:08.104431+0000","last_fullsized":"2024-09-10T16:02:08.104431+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104410+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:02:08.104410+0000","last_peered":"2024-09-10T16:02:08.104410+0000","last_clean":"2024-09-10T16:02:08.104410+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:02:08.104410+0000","last_undegraded":"2024-09-10T16:02:08.104410+0000","last_fullsized":"2024-09-10T16:02:08.104410+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'10029","reported_seq":10173,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:12.381871+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:02:12.381871+0000","last_peered":"2024-09-10T16:02:12.381871+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:02:12.381871+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41030598,"num_objects":10019,"num_object_clones":0,"num_object_copies":20038,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10019,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6192,"num_write_kb":24768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5696},{"shards":"0,2","objects":4323}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":650179294,"num_objects":158744,"num_object_clones":0,"num_object_copies":317488,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42911,"num_objects_misplaced":4076,"num_objects_unfound":0,"num_objects_dirty":158744,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":116590,"num_write_kb":466360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4399,"num_bytes_recovered":18014208,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1321340928,"data_stored":1321189182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":158616,"ondisk_log_size":158616,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986227,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":717400,"kb_used_data":455240,"kb_used_omap":4,"kb_used_meta":262139,"kb_avail":93654440,"statfs":{"total":96636764160,"available":95902146560,"internally_reserved":0,"allocated":466165760,"data_stored":465877599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":268431018},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,0,4,1,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659173,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":502228,"kb_used_data":290052,"kb_used_omap":4,"kb_used_meta":212155,"kb_avail":93869612,"statfs":{"total":96636764160,"available":96122482688,"internally_reserved":0,"allocated":297013248,"data_stored":296677654,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":217247406},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,0,2,5,5,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182455,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":864600,"kb_used_data":544200,"kb_used_omap":4,"kb_used_meta":320379,"kb_avail":93507240,"statfs":{"total":96636764160,"available":95751413760,"internally_reserved":0,"allocated":557260800,"data_stored":556968562,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":328068778},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,0,3,2,2,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":557301760,"data_stored":557265630,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":297566208,"data_stored":297486722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":466472960,"data_stored":466436830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:02:17.995 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 225 20 160775 160755 2.79046 1.78125 0.0266168 0.0279899 2024-09-10T16:02:18.995 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 226 20 161230 161210 2.78597 1.77734 0.0562881 0.028036 2024-09-10T16:02:19.995 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 227 20 161676 161656 2.78137 1.74219 0.0945346 0.028081 2024-09-10T16:02:20.147 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:02:20.704 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:02:20.707 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:02:20.724 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":190,"stamp":"2024-09-10T16:02:19.609901+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":655815409,"num_objects":160122,"num_object_clones":0,"num_object_copies":320244,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42911,"num_objects_misplaced":4002,"num_objects_unfound":0,"num_objects_dirty":160122,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":117968,"num_write_kb":471866,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4475,"num_bytes_recovered":18317331,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159495,"ondisk_log_size":159495,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2112776,"kb_used_data":1295832,"kb_used_omap":12,"kb_used_meta":816883,"kb_avail":281002744,"statfs":{"total":289910292480,"available":287746809856,"internally_reserved":0,"allocated":1326931968,"data_stored":1326015975,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":836488194},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,3,5,0,6,8,12,10],"upper_bound":256},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":17117184,"num_objects":4179,"num_object_clones":0,"num_object_copies":8358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-254,"num_objects_unfound":0,"num_objects_dirty":4179,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4179,"num_write_kb":16716,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":254,"num_bytes_recovered":1040384,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"11.508710"},"pg_stats":[{"pgid":"2.f","version":"45'9933","reported_seq":10007,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:17.432223+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:02:17.432223+0000","last_peered":"2024-09-10T16:02:17.432223+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:02:17.432223+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":9933,"log_dups_size":0,"ondisk_log_size":9933,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40685568,"num_objects":9933,"num_object_clones":0,"num_object_copies":19866,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9933,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9933,"num_write_kb":39732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5674},{"shards":"0,2","objects":4258}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'9925","reported_seq":10037,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:14.758419+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:02:14.758419+0000","last_peered":"2024-09-10T16:02:14.758419+0000","last_clean":"2024-09-10T16:02:14.758419+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:02:14.758419+0000","last_undegraded":"2024-09-10T16:02:14.758419+0000","last_fullsized":"2024-09-10T16:02:14.758419+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":9925,"log_dups_size":0,"ondisk_log_size":9925,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40629190,"num_objects":9921,"num_object_clones":0,"num_object_copies":19842,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9921,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":8030,"num_write_kb":32120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'9931","reported_seq":18607,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:14.885899+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:02:14.885899+0000","last_peered":"2024-09-10T16:02:14.885899+0000","last_clean":"2024-09-10T16:02:14.885899+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:02:14.885899+0000","last_undegraded":"2024-09-10T16:02:14.885899+0000","last_fullsized":"2024-09-10T16:02:14.885899+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":9931,"log_dups_size":0,"ondisk_log_size":9931,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40660992,"num_objects":9927,"num_object_clones":0,"num_object_copies":19854,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9927,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7935,"num_write_kb":31740,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'9858","reported_seq":9938,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:14.621803+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:02:14.621803+0000","last_peered":"2024-09-10T16:02:14.621803+0000","last_clean":"2024-09-10T16:02:14.621803+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:02:14.621803+0000","last_undegraded":"2024-09-10T16:02:14.621803+0000","last_fullsized":"2024-09-10T16:02:14.621803+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":9858,"log_dups_size":0,"ondisk_log_size":9858,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40378368,"num_objects":9858,"num_object_clones":0,"num_object_copies":19716,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9858,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9858,"num_write_kb":39432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'10022","reported_seq":10151,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:17.448028+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:02:17.448028+0000","last_peered":"2024-09-10T16:02:17.448028+0000","last_clean":"2024-09-10T16:02:17.448028+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:02:17.448028+0000","last_undegraded":"2024-09-10T16:02:17.448028+0000","last_fullsized":"2024-09-10T16:02:17.448028+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41021923,"num_objects":10016,"num_object_clones":0,"num_object_copies":20032,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10016,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6733,"num_write_kb":26932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'10122","reported_seq":10257,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:17.479713+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:02:17.479713+0000","last_peered":"2024-09-10T16:02:17.479713+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:02:17.479713+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"36'100","ondisk_log_start":"36'100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41439715,"num_objects":10118,"num_object_clones":0,"num_object_copies":20236,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10118,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6734,"num_write_kb":26936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5747},{"shards":"0,2","objects":4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'10135","reported_seq":10272,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:17.394551+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:02:17.394551+0000","last_peered":"2024-09-10T16:02:17.394551+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:02:17.394551+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"36'100","ondisk_log_start":"36'100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41489350,"num_objects":10131,"num_object_clones":0,"num_object_copies":20262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10131,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":6634,"num_write_kb":26536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5894},{"shards":"0,2","objects":4237}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'10266","reported_seq":10395,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:17.454080+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:02:17.454080+0000","last_peered":"2024-09-10T16:02:17.454080+0000","last_clean":"2024-09-10T16:02:17.454080+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:02:17.454080+0000","last_undegraded":"2024-09-10T16:02:17.454080+0000","last_fullsized":"2024-09-10T16:02:17.454080+0000","mapping_epoch":34,"log_start":"36'200","ondisk_log_start":"36'200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42041344,"num_objects":10264,"num_object_clones":0,"num_object_copies":20528,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10264,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7113,"num_write_kb":28452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104219+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:02:08.104219+0000","last_peered":"2024-09-10T16:02:08.104219+0000","last_clean":"2024-09-10T16:02:08.104219+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:02:08.104219+0000","last_undegraded":"2024-09-10T16:02:08.104219+0000","last_fullsized":"2024-09-10T16:02:08.104219+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'9715","reported_seq":9854,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:14.723055+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:02:14.723055+0000","last_peered":"2024-09-10T16:02:14.723055+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:02:14.723055+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":9715,"log_dups_size":0,"ondisk_log_size":9715,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39764451,"num_objects":9709,"num_object_clones":0,"num_object_copies":19418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9709,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7534,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4123},{"shards":"2","objects":5586}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":93,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.102747+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:02:08.102747+0000","last_peered":"2024-09-10T16:02:08.102747+0000","last_clean":"2024-09-10T16:02:08.102747+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:02:08.102747+0000","last_undegraded":"2024-09-10T16:02:08.102747+0000","last_fullsized":"2024-09-10T16:02:08.102747+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'10052","reported_seq":10181,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:17.481648+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:02:17.481648+0000","last_peered":"2024-09-10T16:02:17.481648+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:02:17.481648+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41140224,"num_objects":10044,"num_object_clones":0,"num_object_copies":20088,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10044,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6556,"num_write_kb":26224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5789},{"shards":"0,2","objects":4255}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":104,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.105055+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:02:08.105055+0000","last_peered":"2024-09-10T16:02:08.105055+0000","last_clean":"2024-09-10T16:02:08.105055+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:02:08.105055+0000","last_undegraded":"2024-09-10T16:02:08.105055+0000","last_fullsized":"2024-09-10T16:02:08.105055+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'9897","reported_seq":10031,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:14.743493+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:02:14.743493+0000","last_peered":"2024-09-10T16:02:14.743493+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:02:14.743493+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":9897,"log_dups_size":0,"ondisk_log_size":9897,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40518115,"num_objects":9893,"num_object_clones":0,"num_object_copies":19786,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9893,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7577,"num_write_kb":30308,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4304},{"shards":"2","objects":5589}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104572+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:02:08.104572+0000","last_peered":"2024-09-10T16:02:08.104572+0000","last_clean":"2024-09-10T16:02:08.104572+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:02:08.104572+0000","last_undegraded":"2024-09-10T16:02:08.104572+0000","last_fullsized":"2024-09-10T16:02:08.104572+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'10107","reported_seq":10235,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:14.645353+0000","last_change":"2024-09-10T15:59:16.174113+0000","last_active":"2024-09-10T16:02:14.645353+0000","last_peered":"2024-09-10T16:02:14.645353+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:02:14.645353+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"36'100","ondisk_log_start":"36'100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41373696,"num_objects":10101,"num_object_clones":0,"num_object_copies":20202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4354,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10101,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7661,"num_write_kb":30644,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4352},{"shards":"2","objects":5749}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'10004","reported_seq":10136,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:17.423158+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:02:17.423158+0000","last_peered":"2024-09-10T16:02:17.423158+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:02:17.423158+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40968192,"num_objects":10002,"num_object_clones":0,"num_object_copies":20004,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10002,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6635,"num_write_kb":26540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5698},{"shards":"0,2","objects":4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103276+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:02:08.103276+0000","last_peered":"2024-09-10T16:02:08.103276+0000","last_clean":"2024-09-10T16:02:08.103276+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:02:08.103276+0000","last_undegraded":"2024-09-10T16:02:08.103276+0000","last_fullsized":"2024-09-10T16:02:08.103276+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'9926","reported_seq":10470,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T16:02:17.492796+0000","last_change":"2024-09-10T16:02:06.480033+0000","last_active":"2024-09-10T16:02:17.492796+0000","last_peered":"2024-09-10T16:02:17.492796+0000","last_clean":"2024-09-10T15:59:14.115595+0000","last_became_active":"2024-09-10T15:59:16.244089+0000","last_became_peered":"2024-09-10T15:59:16.244089+0000","last_unstale":"2024-09-10T16:02:17.492796+0000","last_undegraded":"2024-09-10T16:02:17.492796+0000","last_fullsized":"2024-09-10T15:59:15.774437+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":9926,"log_dups_size":0,"ondisk_log_size":9926,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40624128,"num_objects":9918,"num_object_clones":0,"num_object_copies":19836,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":4002,"num_objects_unfound":0,"num_objects_dirty":9918,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6316,"num_write_kb":25264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":201,"num_bytes_recovered":823296,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5916},{"shards":"0,2","objects":4002}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":93,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103338+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:02:08.103338+0000","last_peered":"2024-09-10T16:02:08.103338+0000","last_clean":"2024-09-10T16:02:08.103338+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:02:08.103338+0000","last_undegraded":"2024-09-10T16:02:08.103338+0000","last_fullsized":"2024-09-10T16:02:08.103338+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'10111","reported_seq":10252,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:17.442847+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:02:17.442847+0000","last_peered":"2024-09-10T16:02:17.442847+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:02:17.442847+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"36'100","ondisk_log_start":"36'100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41398272,"num_objects":10107,"num_object_clones":0,"num_object_copies":20214,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10107,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6366,"num_write_kb":25464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5736},{"shards":"0,2","objects":4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104431+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:02:08.104431+0000","last_peered":"2024-09-10T16:02:08.104431+0000","last_clean":"2024-09-10T16:02:08.104431+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:02:08.104431+0000","last_undegraded":"2024-09-10T16:02:08.104431+0000","last_fullsized":"2024-09-10T16:02:08.104431+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104410+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:02:08.104410+0000","last_peered":"2024-09-10T16:02:08.104410+0000","last_clean":"2024-09-10T16:02:08.104410+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:02:08.104410+0000","last_undegraded":"2024-09-10T16:02:08.104410+0000","last_fullsized":"2024-09-10T16:02:08.104410+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'10188","reported_seq":10332,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:17.431955+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:02:17.431955+0000","last_peered":"2024-09-10T16:02:17.431955+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:02:17.431955+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"36'100","ondisk_log_start":"36'100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41681862,"num_objects":10178,"num_object_clones":0,"num_object_copies":20356,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10178,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6351,"num_write_kb":25404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5855},{"shards":"0,2","objects":4323}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":655815390,"num_objects":160120,"num_object_clones":0,"num_object_copies":320240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42911,"num_objects_misplaced":4002,"num_objects_unfound":0,"num_objects_dirty":160120,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":117966,"num_write_kb":471864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4473,"num_bytes_recovered":18317312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1328017408,"data_stored":1327865662,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159492,"ondisk_log_size":159492,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986227,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":717400,"kb_used_data":455240,"kb_used_omap":4,"kb_used_meta":262139,"kb_avail":93654440,"statfs":{"total":96636764160,"available":95902146560,"internally_reserved":0,"allocated":466165760,"data_stored":465877599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":268431018},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,0,4,1,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659173,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":502228,"kb_used_data":290052,"kb_used_omap":4,"kb_used_meta":212155,"kb_avail":93869612,"statfs":{"total":96636764160,"available":96122482688,"internally_reserved":0,"allocated":297013248,"data_stored":296677654,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":217247406},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,0,2,5,5,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182456,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":893148,"kb_used_data":550540,"kb_used_omap":4,"kb_used_meta":342587,"kb_avail":93478692,"statfs":{"total":96636764160,"available":95722180608,"internally_reserved":0,"allocated":563752960,"data_stored":563460722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":350809770},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,3,4,0,0,2,6,6],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":563978240,"data_stored":563942110,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":297566208,"data_stored":297486722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":466472960,"data_stored":466436830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:02:20.726 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:02:20.995 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 228 20 162077 162057 2.77604 1.56641 0.0369725 0.0281343 2024-09-10T16:02:21.277 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:02:21.278 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:02:21.300 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":190,"stamp":"2024-09-10T16:02:19.609901+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":655815409,"num_objects":160122,"num_object_clones":0,"num_object_copies":320244,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42911,"num_objects_misplaced":4002,"num_objects_unfound":0,"num_objects_dirty":160122,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":117968,"num_write_kb":471866,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4475,"num_bytes_recovered":18317331,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159495,"ondisk_log_size":159495,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2112776,"kb_used_data":1295832,"kb_used_omap":12,"kb_used_meta":816883,"kb_avail":281002744,"statfs":{"total":289910292480,"available":287746809856,"internally_reserved":0,"allocated":1326931968,"data_stored":1326015975,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":836488194},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,3,5,0,6,8,12,10],"upper_bound":256},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":17117184,"num_objects":4179,"num_object_clones":0,"num_object_copies":8358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-254,"num_objects_unfound":0,"num_objects_dirty":4179,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4179,"num_write_kb":16716,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":254,"num_bytes_recovered":1040384,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"11.508710"},"pg_stats":[{"pgid":"2.f","version":"45'9933","reported_seq":10007,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:17.432223+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:02:17.432223+0000","last_peered":"2024-09-10T16:02:17.432223+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:02:17.432223+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":9933,"log_dups_size":0,"ondisk_log_size":9933,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40685568,"num_objects":9933,"num_object_clones":0,"num_object_copies":19866,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9933,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9933,"num_write_kb":39732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5674},{"shards":"0,2","objects":4258}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'9925","reported_seq":10037,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:14.758419+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:02:14.758419+0000","last_peered":"2024-09-10T16:02:14.758419+0000","last_clean":"2024-09-10T16:02:14.758419+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:02:14.758419+0000","last_undegraded":"2024-09-10T16:02:14.758419+0000","last_fullsized":"2024-09-10T16:02:14.758419+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":9925,"log_dups_size":0,"ondisk_log_size":9925,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40629190,"num_objects":9921,"num_object_clones":0,"num_object_copies":19842,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9921,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":8030,"num_write_kb":32120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'9931","reported_seq":18607,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:14.885899+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:02:14.885899+0000","last_peered":"2024-09-10T16:02:14.885899+0000","last_clean":"2024-09-10T16:02:14.885899+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:02:14.885899+0000","last_undegraded":"2024-09-10T16:02:14.885899+0000","last_fullsized":"2024-09-10T16:02:14.885899+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":9931,"log_dups_size":0,"ondisk_log_size":9931,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40660992,"num_objects":9927,"num_object_clones":0,"num_object_copies":19854,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9927,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7935,"num_write_kb":31740,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'9858","reported_seq":9938,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:14.621803+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:02:14.621803+0000","last_peered":"2024-09-10T16:02:14.621803+0000","last_clean":"2024-09-10T16:02:14.621803+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:02:14.621803+0000","last_undegraded":"2024-09-10T16:02:14.621803+0000","last_fullsized":"2024-09-10T16:02:14.621803+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":9858,"log_dups_size":0,"ondisk_log_size":9858,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40378368,"num_objects":9858,"num_object_clones":0,"num_object_copies":19716,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9858,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9858,"num_write_kb":39432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'10022","reported_seq":10151,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:17.448028+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:02:17.448028+0000","last_peered":"2024-09-10T16:02:17.448028+0000","last_clean":"2024-09-10T16:02:17.448028+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:02:17.448028+0000","last_undegraded":"2024-09-10T16:02:17.448028+0000","last_fullsized":"2024-09-10T16:02:17.448028+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41021923,"num_objects":10016,"num_object_clones":0,"num_object_copies":20032,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10016,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6733,"num_write_kb":26932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'10122","reported_seq":10257,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:17.479713+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:02:17.479713+0000","last_peered":"2024-09-10T16:02:17.479713+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:02:17.479713+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"36'100","ondisk_log_start":"36'100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41439715,"num_objects":10118,"num_object_clones":0,"num_object_copies":20236,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10118,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6734,"num_write_kb":26936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5747},{"shards":"0,2","objects":4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'10135","reported_seq":10272,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:17.394551+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:02:17.394551+0000","last_peered":"2024-09-10T16:02:17.394551+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:02:17.394551+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"36'100","ondisk_log_start":"36'100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41489350,"num_objects":10131,"num_object_clones":0,"num_object_copies":20262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10131,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":6634,"num_write_kb":26536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5894},{"shards":"0,2","objects":4237}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'10266","reported_seq":10395,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:17.454080+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:02:17.454080+0000","last_peered":"2024-09-10T16:02:17.454080+0000","last_clean":"2024-09-10T16:02:17.454080+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:02:17.454080+0000","last_undegraded":"2024-09-10T16:02:17.454080+0000","last_fullsized":"2024-09-10T16:02:17.454080+0000","mapping_epoch":34,"log_start":"36'200","ondisk_log_start":"36'200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42041344,"num_objects":10264,"num_object_clones":0,"num_object_copies":20528,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10264,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7113,"num_write_kb":28452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104219+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:02:08.104219+0000","last_peered":"2024-09-10T16:02:08.104219+0000","last_clean":"2024-09-10T16:02:08.104219+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:02:08.104219+0000","last_undegraded":"2024-09-10T16:02:08.104219+0000","last_fullsized":"2024-09-10T16:02:08.104219+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'9715","reported_seq":9854,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:14.723055+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:02:14.723055+0000","last_peered":"2024-09-10T16:02:14.723055+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:02:14.723055+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":9715,"log_dups_size":0,"ondisk_log_size":9715,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39764451,"num_objects":9709,"num_object_clones":0,"num_object_copies":19418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9709,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7534,"num_write_kb":30136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4123},{"shards":"2","objects":5586}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":93,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.102747+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:02:08.102747+0000","last_peered":"2024-09-10T16:02:08.102747+0000","last_clean":"2024-09-10T16:02:08.102747+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:02:08.102747+0000","last_undegraded":"2024-09-10T16:02:08.102747+0000","last_fullsized":"2024-09-10T16:02:08.102747+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'10052","reported_seq":10181,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:17.481648+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:02:17.481648+0000","last_peered":"2024-09-10T16:02:17.481648+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:02:17.481648+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41140224,"num_objects":10044,"num_object_clones":0,"num_object_copies":20088,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10044,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6556,"num_write_kb":26224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5789},{"shards":"0,2","objects":4255}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":104,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.105055+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:02:08.105055+0000","last_peered":"2024-09-10T16:02:08.105055+0000","last_clean":"2024-09-10T16:02:08.105055+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:02:08.105055+0000","last_undegraded":"2024-09-10T16:02:08.105055+0000","last_fullsized":"2024-09-10T16:02:08.105055+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'9897","reported_seq":10031,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:14.743493+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:02:14.743493+0000","last_peered":"2024-09-10T16:02:14.743493+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:02:14.743493+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":9897,"log_dups_size":0,"ondisk_log_size":9897,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40518115,"num_objects":9893,"num_object_clones":0,"num_object_copies":19786,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9893,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7577,"num_write_kb":30308,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4304},{"shards":"2","objects":5589}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104572+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:02:08.104572+0000","last_peered":"2024-09-10T16:02:08.104572+0000","last_clean":"2024-09-10T16:02:08.104572+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:02:08.104572+0000","last_undegraded":"2024-09-10T16:02:08.104572+0000","last_fullsized":"2024-09-10T16:02:08.104572+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'10107","reported_seq":10235,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:14.645353+0000","last_change":"2024-09-10T15:59:16.174113+0000","last_active":"2024-09-10T16:02:14.645353+0000","last_peered":"2024-09-10T16:02:14.645353+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:02:14.645353+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"36'100","ondisk_log_start":"36'100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41373696,"num_objects":10101,"num_object_clones":0,"num_object_copies":20202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4354,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10101,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7661,"num_write_kb":30644,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4352},{"shards":"2","objects":5749}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'10004","reported_seq":10136,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:17.423158+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:02:17.423158+0000","last_peered":"2024-09-10T16:02:17.423158+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:02:17.423158+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40968192,"num_objects":10002,"num_object_clones":0,"num_object_copies":20004,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10002,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6635,"num_write_kb":26540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5698},{"shards":"0,2","objects":4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103276+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:02:08.103276+0000","last_peered":"2024-09-10T16:02:08.103276+0000","last_clean":"2024-09-10T16:02:08.103276+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:02:08.103276+0000","last_undegraded":"2024-09-10T16:02:08.103276+0000","last_fullsized":"2024-09-10T16:02:08.103276+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'9926","reported_seq":10470,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T16:02:17.492796+0000","last_change":"2024-09-10T16:02:06.480033+0000","last_active":"2024-09-10T16:02:17.492796+0000","last_peered":"2024-09-10T16:02:17.492796+0000","last_clean":"2024-09-10T15:59:14.115595+0000","last_became_active":"2024-09-10T15:59:16.244089+0000","last_became_peered":"2024-09-10T15:59:16.244089+0000","last_unstale":"2024-09-10T16:02:17.492796+0000","last_undegraded":"2024-09-10T16:02:17.492796+0000","last_fullsized":"2024-09-10T15:59:15.774437+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":9926,"log_dups_size":0,"ondisk_log_size":9926,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40624128,"num_objects":9918,"num_object_clones":0,"num_object_copies":19836,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":4002,"num_objects_unfound":0,"num_objects_dirty":9918,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6316,"num_write_kb":25264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":201,"num_bytes_recovered":823296,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5916},{"shards":"0,2","objects":4002}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":93,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103338+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:02:08.103338+0000","last_peered":"2024-09-10T16:02:08.103338+0000","last_clean":"2024-09-10T16:02:08.103338+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:02:08.103338+0000","last_undegraded":"2024-09-10T16:02:08.103338+0000","last_fullsized":"2024-09-10T16:02:08.103338+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'10111","reported_seq":10252,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:17.442847+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:02:17.442847+0000","last_peered":"2024-09-10T16:02:17.442847+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:02:17.442847+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"36'100","ondisk_log_start":"36'100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41398272,"num_objects":10107,"num_object_clones":0,"num_object_copies":20214,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10107,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6366,"num_write_kb":25464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5736},{"shards":"0,2","objects":4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104431+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:02:08.104431+0000","last_peered":"2024-09-10T16:02:08.104431+0000","last_clean":"2024-09-10T16:02:08.104431+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:02:08.104431+0000","last_undegraded":"2024-09-10T16:02:08.104431+0000","last_fullsized":"2024-09-10T16:02:08.104431+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104410+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:02:08.104410+0000","last_peered":"2024-09-10T16:02:08.104410+0000","last_clean":"2024-09-10T16:02:08.104410+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:02:08.104410+0000","last_undegraded":"2024-09-10T16:02:08.104410+0000","last_fullsized":"2024-09-10T16:02:08.104410+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'10188","reported_seq":10332,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:17.431955+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:02:17.431955+0000","last_peered":"2024-09-10T16:02:17.431955+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:02:17.431955+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"36'100","ondisk_log_start":"36'100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41681862,"num_objects":10178,"num_object_clones":0,"num_object_copies":20356,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10178,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6351,"num_write_kb":25404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5855},{"shards":"0,2","objects":4323}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":655815390,"num_objects":160120,"num_object_clones":0,"num_object_copies":320240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42911,"num_objects_misplaced":4002,"num_objects_unfound":0,"num_objects_dirty":160120,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":117966,"num_write_kb":471864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4473,"num_bytes_recovered":18317312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1328017408,"data_stored":1327865662,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159492,"ondisk_log_size":159492,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986227,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":717400,"kb_used_data":455240,"kb_used_omap":4,"kb_used_meta":262139,"kb_avail":93654440,"statfs":{"total":96636764160,"available":95902146560,"internally_reserved":0,"allocated":466165760,"data_stored":465877599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":268431018},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,0,4,1,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659173,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":502228,"kb_used_data":290052,"kb_used_omap":4,"kb_used_meta":212155,"kb_avail":93869612,"statfs":{"total":96636764160,"available":96122482688,"internally_reserved":0,"allocated":297013248,"data_stored":296677654,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":217247406},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,0,2,5,5,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182456,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":893148,"kb_used_data":550540,"kb_used_omap":4,"kb_used_meta":342587,"kb_avail":93478692,"statfs":{"total":96636764160,"available":95722180608,"internally_reserved":0,"allocated":563752960,"data_stored":563460722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":350809770},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,3,4,0,0,2,6,6],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":563978240,"data_stored":563942110,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":297566208,"data_stored":297486722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":466472960,"data_stored":466436830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:02:21.302 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-10T16:02:21.302 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "45'9933", 'reported_seq': 10007, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:02:17.432223+0000', 'last_change': '2024-09-10T15:59:16.281558+0000', 'last_active': '2024-09-10T16:02:17.432223+0000', 'last_peered': '2024-09-10T16:02:17.432223+0000', 'last_clean': '2024-09-10T15:59:14.132456+0000', 'last_became_active': '2024-09-10T15:59:16.274407+0000', 'last_became_peered': '2024-09-10T15:59:16.274407+0000', 'last_unstale': '2024-09-10T16:02:17.432223+0000', 'last_undegraded': '2024-09-10T15:59:15.851279+0000', 'last_fullsized': '2024-09-10T15:59:15.774356+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'objects_scrubbed': 0, 'log_size': 9933, 'log_dups_size': 0, 'ondisk_log_size': 9933, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40685568, 'num_objects': 9933, 'num_object_clones': 0, 'num_object_copies': 19866, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4259, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9933, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9933, 'num_write_kb': 39732, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5674}, {'shards': '0,2', 'objects': 4258}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:02:21.302 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-10T16:02:21.302 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "45'10122", 'reported_seq': 10257, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:02:17.479713+0000', 'last_change': '2024-09-10T15:59:16.308264+0000', 'last_active': '2024-09-10T16:02:17.479713+0000', 'last_peered': '2024-09-10T16:02:17.479713+0000', 'last_clean': '2024-09-10T15:59:14.111551+0000', 'last_became_active': '2024-09-10T15:59:16.300040+0000', 'last_became_peered': '2024-09-10T15:59:16.300040+0000', 'last_unstale': '2024-09-10T16:02:17.479713+0000', 'last_undegraded': '2024-09-10T15:59:15.878234+0000', 'last_fullsized': '2024-09-10T15:59:15.769328+0000', 'mapping_epoch': 42, 'log_start': "36'100", 'ondisk_log_start': "36'100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3442", 'last_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'objects_scrubbed': 3384, 'log_size': 10022, 'log_dups_size': 0, 'ondisk_log_size': 10022, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1292, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 41439715, 'num_objects': 10118, 'num_object_clones': 0, 'num_object_copies': 20236, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4372, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10118, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 6734, 'num_write_kb': 26936, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5747}, {'shards': '0,2', 'objects': 4371}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:02:21.302 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-10T16:02:21.302 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "45'10135", 'reported_seq': 10272, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:02:17.394551+0000', 'last_change': '2024-09-10T15:59:16.331899+0000', 'last_active': '2024-09-10T16:02:17.394551+0000', 'last_peered': '2024-09-10T16:02:17.394551+0000', 'last_clean': '2024-09-10T15:59:14.139436+0000', 'last_became_active': '2024-09-10T15:59:16.318715+0000', 'last_became_peered': '2024-09-10T15:59:16.318715+0000', 'last_unstale': '2024-09-10T16:02:17.394551+0000', 'last_undegraded': '2024-09-10T15:59:15.877524+0000', 'last_fullsized': '2024-09-10T15:59:15.769042+0000', 'mapping_epoch': 42, 'log_start': "36'100", 'ondisk_log_start': "36'100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3537", 'last_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'objects_scrubbed': 3497, 'log_size': 10035, 'log_dups_size': 0, 'ondisk_log_size': 10035, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1224, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 41489350, 'num_objects': 10131, 'num_object_clones': 0, 'num_object_copies': 20262, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4237, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10131, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 6634, 'num_write_kb': 26536, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5894}, {'shards': '0,2', 'objects': 4237}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:02:21.302 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-10T16:02:21.302 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "45'9715", 'reported_seq': 9854, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:02:14.723055+0000', 'last_change': '2024-09-10T15:59:16.175680+0000', 'last_active': '2024-09-10T16:02:14.723055+0000', 'last_peered': '2024-09-10T16:02:14.723055+0000', 'last_clean': '2024-09-10T15:59:14.088187+0000', 'last_became_active': '2024-09-10T15:59:16.165810+0000', 'last_became_peered': '2024-09-10T15:59:16.165810+0000', 'last_unstale': '2024-09-10T16:02:14.723055+0000', 'last_undegraded': '2024-09-10T15:59:15.822098+0000', 'last_fullsized': '2024-09-10T15:59:15.769123+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2218", 'last_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'objects_scrubbed': 2175, 'log_size': 9715, 'log_dups_size': 0, 'ondisk_log_size': 9715, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 825, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39764451, 'num_objects': 9709, 'num_object_clones': 0, 'num_object_copies': 19418, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4124, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9709, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 7534, 'num_write_kb': 30136, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4123}, {'shards': '2', 'objects': 5586}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:02:21.303 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-10T16:02:21.303 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "45'10052", 'reported_seq': 10181, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:02:17.481648+0000', 'last_change': '2024-09-10T15:59:16.277742+0000', 'last_active': '2024-09-10T16:02:17.481648+0000', 'last_peered': '2024-09-10T16:02:17.481648+0000', 'last_clean': '2024-09-10T15:59:14.129081+0000', 'last_became_active': '2024-09-10T15:59:16.269835+0000', 'last_became_peered': '2024-09-10T15:59:16.269835+0000', 'last_unstale': '2024-09-10T16:02:17.481648+0000', 'last_undegraded': '2024-09-10T15:59:15.860857+0000', 'last_fullsized': '2024-09-10T15:59:15.771410+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3557", 'last_scrub_stamp': '2024-09-10T15:59:02.493678+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:02.493678+0000', 'objects_scrubbed': 3488, 'log_size': 10052, 'log_dups_size': 0, 'ondisk_log_size': 10052, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1472, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 41140224, 'num_objects': 10044, 'num_object_clones': 0, 'num_object_copies': 20088, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4257, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10044, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6556, 'num_write_kb': 26224, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5789}, {'shards': '0,2', 'objects': 4255}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:02:21.303 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-10T16:02:21.303 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "45'9897", 'reported_seq': 10031, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:02:14.743493+0000', 'last_change': '2024-09-10T15:59:16.284311+0000', 'last_active': '2024-09-10T16:02:14.743493+0000', 'last_peered': '2024-09-10T16:02:14.743493+0000', 'last_clean': '2024-09-10T15:59:14.116041+0000', 'last_became_active': '2024-09-10T15:59:16.274543+0000', 'last_became_peered': '2024-09-10T15:59:16.274543+0000', 'last_unstale': '2024-09-10T16:02:14.743493+0000', 'last_undegraded': '2024-09-10T15:59:15.812484+0000', 'last_fullsized': '2024-09-10T15:59:15.770885+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2356", 'last_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'objects_scrubbed': 2316, 'log_size': 9897, 'log_dups_size': 0, 'ondisk_log_size': 9897, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 783, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40518115, 'num_objects': 9893, 'num_object_clones': 0, 'num_object_copies': 19786, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9893, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 7577, 'num_write_kb': 30308, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4304}, {'shards': '2', 'objects': 5589}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:02:21.303 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-10T16:02:21.303 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "45'10107", 'reported_seq': 10235, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:02:14.645353+0000', 'last_change': '2024-09-10T15:59:16.174113+0000', 'last_active': '2024-09-10T16:02:14.645353+0000', 'last_peered': '2024-09-10T16:02:14.645353+0000', 'last_clean': '2024-09-10T15:59:14.110654+0000', 'last_became_active': '2024-09-10T15:59:16.164645+0000', 'last_became_peered': '2024-09-10T15:59:16.164645+0000', 'last_unstale': '2024-09-10T16:02:14.645353+0000', 'last_undegraded': '2024-09-10T15:59:15.803593+0000', 'last_fullsized': '2024-09-10T15:59:15.769083+0000', 'mapping_epoch': 42, 'log_start': "36'100", 'ondisk_log_start': "36'100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2486", 'last_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'objects_scrubbed': 2440, 'log_size': 10007, 'log_dups_size': 0, 'ondisk_log_size': 10007, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 792, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 41373696, 'num_objects': 10101, 'num_object_clones': 0, 'num_object_copies': 20202, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4354, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10101, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7661, 'num_write_kb': 30644, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4352}, {'shards': '2', 'objects': 5749}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:02:21.303 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-10T16:02:21.303 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "45'10004", 'reported_seq': 10136, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:02:17.423158+0000', 'last_change': '2024-09-10T15:59:16.315641+0000', 'last_active': '2024-09-10T16:02:17.423158+0000', 'last_peered': '2024-09-10T16:02:17.423158+0000', 'last_clean': '2024-09-10T15:59:14.117001+0000', 'last_became_active': '2024-09-10T15:59:16.302245+0000', 'last_became_peered': '2024-09-10T15:59:16.302245+0000', 'last_unstale': '2024-09-10T16:02:17.423158+0000', 'last_undegraded': '2024-09-10T15:59:15.845012+0000', 'last_fullsized': '2024-09-10T15:59:15.770337+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3404", 'last_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'objects_scrubbed': 3367, 'log_size': 10004, 'log_dups_size': 0, 'ondisk_log_size': 10004, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 983, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40968192, 'num_objects': 10002, 'num_object_clones': 0, 'num_object_copies': 20004, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10002, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6635, 'num_write_kb': 26540, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5698}, {'shards': '0,2', 'objects': 4304}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:02:21.303 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-10T16:02:21.303 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "45'9926", 'reported_seq': 10470, 'reported_epoch': 45, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-10T16:02:17.492796+0000', 'last_change': '2024-09-10T16:02:06.480033+0000', 'last_active': '2024-09-10T16:02:17.492796+0000', 'last_peered': '2024-09-10T16:02:17.492796+0000', 'last_clean': '2024-09-10T15:59:14.115595+0000', 'last_became_active': '2024-09-10T15:59:16.244089+0000', 'last_became_peered': '2024-09-10T15:59:16.244089+0000', 'last_unstale': '2024-09-10T16:02:17.492796+0000', 'last_undegraded': '2024-09-10T16:02:17.492796+0000', 'last_fullsized': '2024-09-10T15:59:15.774437+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3653", 'last_scrub_stamp': '2024-09-10T15:59:05.298468+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:05.298468+0000', 'objects_scrubbed': 3602, 'log_size': 9926, 'log_dups_size': 0, 'ondisk_log_size': 9926, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1289, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40624128, 'num_objects': 9918, 'num_object_clones': 0, 'num_object_copies': 19836, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 4002, 'num_objects_unfound': 0, 'num_objects_dirty': 9918, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6316, 'num_write_kb': 25264, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 201, 'num_bytes_recovered': 823296, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5916}, {'shards': '0,2', 'objects': 4002}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:02:21.303 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-10T16:02:21.304 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "45'10111", 'reported_seq': 10252, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:02:17.442847+0000', 'last_change': '2024-09-10T15:59:16.311291+0000', 'last_active': '2024-09-10T16:02:17.442847+0000', 'last_peered': '2024-09-10T16:02:17.442847+0000', 'last_clean': '2024-09-10T15:59:14.116363+0000', 'last_became_active': '2024-09-10T15:59:16.301244+0000', 'last_became_peered': '2024-09-10T15:59:16.301244+0000', 'last_unstale': '2024-09-10T16:02:17.442847+0000', 'last_undegraded': '2024-09-10T15:59:15.883434+0000', 'last_fullsized': '2024-09-10T15:59:15.772756+0000', 'mapping_epoch': 42, 'log_start': "36'100", 'ondisk_log_start': "36'100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3789", 'last_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'objects_scrubbed': 3741, 'log_size': 10011, 'log_dups_size': 0, 'ondisk_log_size': 10011, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1440, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 41398272, 'num_objects': 10107, 'num_object_clones': 0, 'num_object_copies': 20214, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4373, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10107, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6366, 'num_write_kb': 25464, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5736}, {'shards': '0,2', 'objects': 4371}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:02:21.304 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-10T16:02:21.304 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "45'10188", 'reported_seq': 10332, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:02:17.431955+0000', 'last_change': '2024-09-10T15:59:16.313409+0000', 'last_active': '2024-09-10T16:02:17.431955+0000', 'last_peered': '2024-09-10T16:02:17.431955+0000', 'last_clean': '2024-09-10T15:59:14.129780+0000', 'last_became_active': '2024-09-10T15:59:16.300932+0000', 'last_became_peered': '2024-09-10T15:59:16.300932+0000', 'last_unstale': '2024-09-10T16:02:17.431955+0000', 'last_undegraded': '2024-09-10T15:59:15.887823+0000', 'last_fullsized': '2024-09-10T15:59:15.768513+0000', 'mapping_epoch': 42, 'log_start': "36'100", 'ondisk_log_start': "36'100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3880", 'last_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'objects_scrubbed': 3827, 'log_size': 10088, 'log_dups_size': 0, 'ondisk_log_size': 10088, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1397, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 41681862, 'num_objects': 10178, 'num_object_clones': 0, 'num_object_copies': 20356, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4325, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10178, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6351, 'num_write_kb': 25404, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5855}, {'shards': '0,2', 'objects': 4323}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:02:21.304 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T16:02:21.995 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 229 20 162473 162453 2.77068 1.54688 0.0540045 0.0281891 2024-09-10T16:02:22.005 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:02:22.005 INFO:teuthology.orchestra.run.smithi115.stdout:{"fsid":"7effee29-df05-4351-9741-ee203564ed81","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 42911/320244 objects degraded (13.399%), 10 pgs degraded, 11 pgs undersized","count":21},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":306,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983954,"num_in_osds":3,"osd_in_since":1725983947,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":160122,"data_bytes":655815409,"bytes_used":2163482624,"bytes_avail":287746809856,"bytes_total":289910292480,"degraded_objects":42911,"degraded_total":320244,"degraded_ratio":0.1339947040381709,"misplaced_objects":4002,"misplaced_total":320244,"misplaced_ratio":0.01249672125004684,"recovering_objects_per_sec":22,"recovering_bytes_per_sec":90399,"recovering_keys_per_sec":0,"num_objects_recovered":254,"num_bytes_recovered":1040384,"num_keys_recovered":0,"write_bytes_sec":1487324,"read_op_per_sec":0,"write_op_per_sec":363},"fsmap":{"epoch":1,"btime":"2024-09-10T15:57:14:967120+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T16:00:13.560776+0000","services":{}},"progress_events":{}} 2024-09-10T16:02:22.033 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-10T16:02:22.033 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:02:22.592 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:02:22.592 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:02:22.614 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":191,"stamp":"2024-09-10T16:02:21.610549+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":659174129,"num_objects":160942,"num_object_clones":0,"num_object_copies":321884,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42911,"num_objects_misplaced":4002,"num_objects_unfound":0,"num_objects_dirty":160942,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":118788,"num_write_kb":475146,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4475,"num_bytes_recovered":18317331,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160215,"ondisk_log_size":160215,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2132116,"kb_used_data":1306848,"kb_used_omap":12,"kb_used_meta":825203,"kb_avail":280983404,"statfs":{"total":289910292480,"available":287727005696,"internally_reserved":0,"allocated":1338212352,"data_stored":1337296359,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":845007874},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,4,5,4,3,6,7,6],"upper_bound":256},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":20475904,"num_objects":4999,"num_object_clones":0,"num_object_copies":9998,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-254,"num_objects_unfound":0,"num_objects_dirty":4999,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4999,"num_write_kb":19996,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":254,"num_bytes_recovered":1040384,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004129"},"pg_stats":[{"pgid":"2.f","version":"45'9933","reported_seq":10007,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:17.432223+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:02:17.432223+0000","last_peered":"2024-09-10T16:02:17.432223+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:02:17.432223+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":9933,"log_dups_size":0,"ondisk_log_size":9933,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40685568,"num_objects":9933,"num_object_clones":0,"num_object_copies":19866,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9933,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9933,"num_write_kb":39732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5674},{"shards":"0,2","objects":4258}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'10064","reported_seq":10176,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:19.781273+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:02:19.781273+0000","last_peered":"2024-09-10T16:02:19.781273+0000","last_clean":"2024-09-10T16:02:19.781273+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:02:19.781273+0000","last_undegraded":"2024-09-10T16:02:19.781273+0000","last_fullsized":"2024-09-10T16:02:19.781273+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41198534,"num_objects":10060,"num_object_clones":0,"num_object_copies":20120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10060,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":8169,"num_write_kb":32676,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'10058","reported_seq":18735,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:19.965676+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:02:19.965676+0000","last_peered":"2024-09-10T16:02:19.965676+0000","last_clean":"2024-09-10T16:02:19.965676+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:02:19.965676+0000","last_undegraded":"2024-09-10T16:02:19.965676+0000","last_fullsized":"2024-09-10T16:02:19.965676+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41181184,"num_objects":10054,"num_object_clones":0,"num_object_copies":20108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10054,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8062,"num_write_kb":32248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'9992","reported_seq":10072,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:19.781417+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:02:19.781417+0000","last_peered":"2024-09-10T16:02:19.781417+0000","last_clean":"2024-09-10T16:02:19.781417+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:02:19.781417+0000","last_undegraded":"2024-09-10T16:02:19.781417+0000","last_fullsized":"2024-09-10T16:02:19.781417+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":9992,"log_dups_size":0,"ondisk_log_size":9992,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40927232,"num_objects":9992,"num_object_clones":0,"num_object_copies":19984,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9992,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9992,"num_write_kb":39968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'10022","reported_seq":10151,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:17.448028+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:02:17.448028+0000","last_peered":"2024-09-10T16:02:17.448028+0000","last_clean":"2024-09-10T16:02:17.448028+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:02:17.448028+0000","last_undegraded":"2024-09-10T16:02:17.448028+0000","last_fullsized":"2024-09-10T16:02:17.448028+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41021923,"num_objects":10016,"num_object_clones":0,"num_object_copies":20032,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10016,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6733,"num_write_kb":26932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'10122","reported_seq":10257,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:17.479713+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:02:17.479713+0000","last_peered":"2024-09-10T16:02:17.479713+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:02:17.479713+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"36'100","ondisk_log_start":"36'100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41439715,"num_objects":10118,"num_object_clones":0,"num_object_copies":20236,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10118,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6734,"num_write_kb":26936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5747},{"shards":"0,2","objects":4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'10135","reported_seq":10272,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:17.394551+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:02:17.394551+0000","last_peered":"2024-09-10T16:02:17.394551+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:02:17.394551+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"36'100","ondisk_log_start":"36'100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41489350,"num_objects":10131,"num_object_clones":0,"num_object_copies":20262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10131,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":6634,"num_write_kb":26536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5894},{"shards":"0,2","objects":4237}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'10266","reported_seq":10395,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:17.454080+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:02:17.454080+0000","last_peered":"2024-09-10T16:02:17.454080+0000","last_clean":"2024-09-10T16:02:17.454080+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:02:17.454080+0000","last_undegraded":"2024-09-10T16:02:17.454080+0000","last_fullsized":"2024-09-10T16:02:17.454080+0000","mapping_epoch":34,"log_start":"36'200","ondisk_log_start":"36'200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42041344,"num_objects":10264,"num_object_clones":0,"num_object_copies":20528,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10264,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7113,"num_write_kb":28452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104219+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:02:08.104219+0000","last_peered":"2024-09-10T16:02:08.104219+0000","last_clean":"2024-09-10T16:02:08.104219+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:02:08.104219+0000","last_undegraded":"2024-09-10T16:02:08.104219+0000","last_fullsized":"2024-09-10T16:02:08.104219+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'9863","reported_seq":10002,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:19.780661+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:02:19.780661+0000","last_peered":"2024-09-10T16:02:19.780661+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:02:19.780661+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":9863,"log_dups_size":0,"ondisk_log_size":9863,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40370659,"num_objects":9857,"num_object_clones":0,"num_object_copies":19714,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9857,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7682,"num_write_kb":30728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4123},{"shards":"2","objects":5734}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":93,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.102747+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:02:08.102747+0000","last_peered":"2024-09-10T16:02:08.102747+0000","last_clean":"2024-09-10T16:02:08.102747+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:02:08.102747+0000","last_undegraded":"2024-09-10T16:02:08.102747+0000","last_fullsized":"2024-09-10T16:02:08.102747+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'10052","reported_seq":10181,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:17.481648+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:02:17.481648+0000","last_peered":"2024-09-10T16:02:17.481648+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:02:17.481648+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41140224,"num_objects":10044,"num_object_clones":0,"num_object_copies":20088,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10044,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6556,"num_write_kb":26224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5789},{"shards":"0,2","objects":4255}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":104,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.105055+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:02:08.105055+0000","last_peered":"2024-09-10T16:02:08.105055+0000","last_clean":"2024-09-10T16:02:08.105055+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:02:08.105055+0000","last_undegraded":"2024-09-10T16:02:08.105055+0000","last_fullsized":"2024-09-10T16:02:08.105055+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'10040","reported_seq":10172,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:19.814529+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:02:19.814529+0000","last_peered":"2024-09-10T16:02:19.814529+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:02:19.814529+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41103843,"num_objects":10036,"num_object_clones":0,"num_object_copies":20072,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10036,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7720,"num_write_kb":30880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4304},{"shards":"2","objects":5732}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104572+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:02:08.104572+0000","last_peered":"2024-09-10T16:02:08.104572+0000","last_clean":"2024-09-10T16:02:08.104572+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:02:08.104572+0000","last_undegraded":"2024-09-10T16:02:08.104572+0000","last_fullsized":"2024-09-10T16:02:08.104572+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'10236","reported_seq":10364,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:19.819995+0000","last_change":"2024-09-10T15:59:16.174113+0000","last_active":"2024-09-10T16:02:19.819995+0000","last_peered":"2024-09-10T16:02:19.819995+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:02:19.819995+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"36'200","ondisk_log_start":"36'200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41902080,"num_objects":10230,"num_object_clones":0,"num_object_copies":20460,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4354,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10230,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7790,"num_write_kb":31160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4352},{"shards":"2","objects":5878}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'10004","reported_seq":10136,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:17.423158+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:02:17.423158+0000","last_peered":"2024-09-10T16:02:17.423158+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:02:17.423158+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40968192,"num_objects":10002,"num_object_clones":0,"num_object_copies":20004,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10002,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6635,"num_write_kb":26540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5698},{"shards":"0,2","objects":4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103276+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:02:08.103276+0000","last_peered":"2024-09-10T16:02:08.103276+0000","last_clean":"2024-09-10T16:02:08.103276+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:02:08.103276+0000","last_undegraded":"2024-09-10T16:02:08.103276+0000","last_fullsized":"2024-09-10T16:02:08.103276+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'9926","reported_seq":10470,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T16:02:17.492796+0000","last_change":"2024-09-10T16:02:06.480033+0000","last_active":"2024-09-10T16:02:17.492796+0000","last_peered":"2024-09-10T16:02:17.492796+0000","last_clean":"2024-09-10T15:59:14.115595+0000","last_became_active":"2024-09-10T15:59:16.244089+0000","last_became_peered":"2024-09-10T15:59:16.244089+0000","last_unstale":"2024-09-10T16:02:17.492796+0000","last_undegraded":"2024-09-10T16:02:17.492796+0000","last_fullsized":"2024-09-10T15:59:15.774437+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":9926,"log_dups_size":0,"ondisk_log_size":9926,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40624128,"num_objects":9918,"num_object_clones":0,"num_object_copies":19836,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":4002,"num_objects_unfound":0,"num_objects_dirty":9918,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6316,"num_write_kb":25264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":201,"num_bytes_recovered":823296,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5916},{"shards":"0,2","objects":4002}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":93,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103338+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:02:08.103338+0000","last_peered":"2024-09-10T16:02:08.103338+0000","last_clean":"2024-09-10T16:02:08.103338+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:02:08.103338+0000","last_undegraded":"2024-09-10T16:02:08.103338+0000","last_fullsized":"2024-09-10T16:02:08.103338+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'10111","reported_seq":10252,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:17.442847+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:02:17.442847+0000","last_peered":"2024-09-10T16:02:17.442847+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:02:17.442847+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"36'100","ondisk_log_start":"36'100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41398272,"num_objects":10107,"num_object_clones":0,"num_object_copies":20214,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10107,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6366,"num_write_kb":25464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5736},{"shards":"0,2","objects":4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104431+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:02:08.104431+0000","last_peered":"2024-09-10T16:02:08.104431+0000","last_clean":"2024-09-10T16:02:08.104431+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:02:08.104431+0000","last_undegraded":"2024-09-10T16:02:08.104431+0000","last_fullsized":"2024-09-10T16:02:08.104431+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104410+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:02:08.104410+0000","last_peered":"2024-09-10T16:02:08.104410+0000","last_clean":"2024-09-10T16:02:08.104410+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:02:08.104410+0000","last_undegraded":"2024-09-10T16:02:08.104410+0000","last_fullsized":"2024-09-10T16:02:08.104410+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'10188","reported_seq":10332,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:17.431955+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:02:17.431955+0000","last_peered":"2024-09-10T16:02:17.431955+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:02:17.431955+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"36'100","ondisk_log_start":"36'100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41681862,"num_objects":10178,"num_object_clones":0,"num_object_copies":20356,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10178,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6351,"num_write_kb":25404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5855},{"shards":"0,2","objects":4323}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":659174110,"num_objects":160940,"num_object_clones":0,"num_object_copies":321880,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42911,"num_objects_misplaced":4002,"num_objects_unfound":0,"num_objects_dirty":160940,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":118786,"num_write_kb":475144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4473,"num_bytes_recovered":18317312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1339502592,"data_stored":1339350846,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160212,"ondisk_log_size":160212,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986228,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":725816,"kb_used_data":459492,"kb_used_omap":4,"kb_used_meta":266299,"kb_avail":93646024,"statfs":{"total":96636764160,"available":95893528576,"internally_reserved":0,"allocated":470519808,"data_stored":470231647,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":272690858},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,1,2,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659174,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":513152,"kb_used_data":296816,"kb_used_omap":4,"kb_used_meta":216315,"kb_avail":93858688,"statfs":{"total":96636764160,"available":96111296512,"internally_reserved":0,"allocated":303939584,"data_stored":303603990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":221507246},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,0,4,2,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182456,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":893148,"kb_used_data":550540,"kb_used_omap":4,"kb_used_meta":342587,"kb_avail":93478692,"statfs":{"total":96636764160,"available":95722180608,"internally_reserved":0,"allocated":563752960,"data_stored":563460722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":350809770},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,3,4,0,0,2,6,6],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":563978240,"data_stored":563942110,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":304611328,"data_stored":304531842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":470913024,"data_stored":470876894,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:02:22.997 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 230 20 162898 162878 2.76583 1.66016 0.0587467 0.02824 2024-09-10T16:02:23.997 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 231 20 163347 163327 2.76145 1.75391 0.0279054 0.0282839 2024-09-10T16:02:24.997 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 232 20 163785 163765 2.75692 1.71094 0.0675136 0.02833 2024-09-10T16:02:25.616 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:02:25.997 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 233 20 164218 164198 2.75235 1.69141 0.0485536 0.0283768 2024-09-10T16:02:26.181 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:02:26.181 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:02:26.204 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":193,"stamp":"2024-09-10T16:02:25.612533+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":667964145,"num_objects":163088,"num_object_clones":0,"num_object_copies":326176,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42911,"num_objects_misplaced":3916,"num_objects_unfound":0,"num_objects_dirty":163088,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":120934,"num_write_kb":483730,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4561,"num_bytes_recovered":18669587,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160961,"ondisk_log_size":160961,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2180052,"kb_used_data":1324260,"kb_used_omap":12,"kb_used_meta":855731,"kb_avail":280935468,"statfs":{"total":289910292480,"available":287677919232,"internally_reserved":0,"allocated":1356042240,"data_stored":1355126247,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":876268546},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,7,12,6,11,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":20942848,"num_objects":5113,"num_object_clones":0,"num_object_copies":10226,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-160,"num_objects_unfound":0,"num_objects_dirty":5113,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5113,"num_write_kb":20452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":160,"num_bytes_recovered":655360,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005062"},"pg_stats":[{"pgid":"2.f","version":"45'10062","reported_seq":10136,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:22.535255+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:02:22.535255+0000","last_peered":"2024-09-10T16:02:22.535255+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:02:22.535255+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41213952,"num_objects":10062,"num_object_clones":0,"num_object_copies":20124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10062,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10062,"num_write_kb":40248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5803},{"shards":"0,2","objects":4258}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'10197","reported_seq":10309,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:24.773296+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:02:24.773296+0000","last_peered":"2024-09-10T16:02:24.773296+0000","last_clean":"2024-09-10T16:02:24.773296+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:02:24.773296+0000","last_undegraded":"2024-09-10T16:02:24.773296+0000","last_fullsized":"2024-09-10T16:02:24.773296+0000","mapping_epoch":35,"log_start":"36'100","ondisk_log_start":"36'100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41743302,"num_objects":10193,"num_object_clones":0,"num_object_copies":20386,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10193,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":8302,"num_write_kb":33208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'10193","reported_seq":18870,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:25.038304+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:02:25.038304+0000","last_peered":"2024-09-10T16:02:25.038304+0000","last_clean":"2024-09-10T16:02:25.038304+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:02:25.038304+0000","last_undegraded":"2024-09-10T16:02:25.038304+0000","last_fullsized":"2024-09-10T16:02:25.038304+0000","mapping_epoch":44,"log_start":"36'100","ondisk_log_start":"36'100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41734144,"num_objects":10189,"num_object_clones":0,"num_object_copies":20378,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10189,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8197,"num_write_kb":32788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'10121","reported_seq":10201,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:24.820679+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:02:24.820679+0000","last_peered":"2024-09-10T16:02:24.820679+0000","last_clean":"2024-09-10T16:02:24.820679+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:02:24.820679+0000","last_undegraded":"2024-09-10T16:02:24.820679+0000","last_fullsized":"2024-09-10T16:02:24.820679+0000","mapping_epoch":35,"log_start":"36'100","ondisk_log_start":"36'100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41455616,"num_objects":10121,"num_object_clones":0,"num_object_copies":20242,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10121,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10121,"num_write_kb":40484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'10161","reported_seq":10290,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:22.523201+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:02:22.523201+0000","last_peered":"2024-09-10T16:02:22.523201+0000","last_clean":"2024-09-10T16:02:22.523201+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:02:22.523201+0000","last_undegraded":"2024-09-10T16:02:22.523201+0000","last_fullsized":"2024-09-10T16:02:22.523201+0000","mapping_epoch":34,"log_start":"36'100","ondisk_log_start":"36'100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41591267,"num_objects":10155,"num_object_clones":0,"num_object_copies":20310,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10155,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6872,"num_write_kb":27488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'10253","reported_seq":10390,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:22.240514+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:02:22.240514+0000","last_peered":"2024-09-10T16:02:22.240514+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:02:22.240514+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"36'200","ondisk_log_start":"36'200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41976291,"num_objects":10249,"num_object_clones":0,"num_object_copies":20498,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10249,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6865,"num_write_kb":27460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5878},{"shards":"0,2","objects":4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'10283","reported_seq":10420,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:22.430316+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:02:22.430316+0000","last_peered":"2024-09-10T16:02:22.430316+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:02:22.430316+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"36'200","ondisk_log_start":"36'200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42095558,"num_objects":10279,"num_object_clones":0,"num_object_copies":20558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10279,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":6782,"num_write_kb":27128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6042},{"shards":"0,2","objects":4237}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'10399","reported_seq":10529,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:22.475139+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:02:22.475139+0000","last_peered":"2024-09-10T16:02:22.475139+0000","last_clean":"2024-09-10T16:02:22.475139+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:02:22.475139+0000","last_undegraded":"2024-09-10T16:02:22.475139+0000","last_fullsized":"2024-09-10T16:02:22.475139+0000","mapping_epoch":34,"log_start":"36'300","ondisk_log_start":"36'300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42586112,"num_objects":10397,"num_object_clones":0,"num_object_copies":20794,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10397,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7246,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104219+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:02:08.104219+0000","last_peered":"2024-09-10T16:02:08.104219+0000","last_clean":"2024-09-10T16:02:08.104219+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:02:08.104219+0000","last_undegraded":"2024-09-10T16:02:08.104219+0000","last_fullsized":"2024-09-10T16:02:08.104219+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'9998","reported_seq":10137,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:24.800862+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:02:24.800862+0000","last_peered":"2024-09-10T16:02:24.800862+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:02:24.800862+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":9998,"log_dups_size":0,"ondisk_log_size":9998,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40923619,"num_objects":9992,"num_object_clones":0,"num_object_copies":19984,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9992,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7817,"num_write_kb":31268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4123},{"shards":"2","objects":5869}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":93,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.102747+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:02:08.102747+0000","last_peered":"2024-09-10T16:02:08.102747+0000","last_clean":"2024-09-10T16:02:08.102747+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:02:08.102747+0000","last_undegraded":"2024-09-10T16:02:08.102747+0000","last_fullsized":"2024-09-10T16:02:08.102747+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'10164","reported_seq":10293,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:22.491380+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:02:22.491380+0000","last_peered":"2024-09-10T16:02:22.491380+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:02:22.491380+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"36'100","ondisk_log_start":"36'100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41598976,"num_objects":10156,"num_object_clones":0,"num_object_copies":20312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10156,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6668,"num_write_kb":26672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5901},{"shards":"0,2","objects":4255}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":104,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.105055+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:02:08.105055+0000","last_peered":"2024-09-10T16:02:08.105055+0000","last_clean":"2024-09-10T16:02:08.105055+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:02:08.105055+0000","last_undegraded":"2024-09-10T16:02:08.105055+0000","last_fullsized":"2024-09-10T16:02:08.105055+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'10169","reported_seq":10303,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:24.699174+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:02:24.699174+0000","last_peered":"2024-09-10T16:02:24.699174+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:02:24.699174+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"36'100","ondisk_log_start":"36'100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41632227,"num_objects":10165,"num_object_clones":0,"num_object_copies":20330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10165,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7849,"num_write_kb":31396,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4304},{"shards":"2","objects":5861}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104572+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:02:08.104572+0000","last_peered":"2024-09-10T16:02:08.104572+0000","last_clean":"2024-09-10T16:02:08.104572+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:02:08.104572+0000","last_undegraded":"2024-09-10T16:02:08.104572+0000","last_fullsized":"2024-09-10T16:02:08.104572+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'10385","reported_seq":10514,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:24.796185+0000","last_change":"2024-09-10T15:59:16.174113+0000","last_active":"2024-09-10T16:02:24.796185+0000","last_peered":"2024-09-10T16:02:24.796185+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:02:24.796185+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"36'300","ondisk_log_start":"36'300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42512384,"num_objects":10379,"num_object_clones":0,"num_object_copies":20758,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4354,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10379,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7939,"num_write_kb":31756,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4352},{"shards":"2","objects":6027}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'10147","reported_seq":10281,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:22.411555+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:02:22.411555+0000","last_peered":"2024-09-10T16:02:22.411555+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:02:22.411555+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"36'100","ondisk_log_start":"36'100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41553920,"num_objects":10145,"num_object_clones":0,"num_object_copies":20290,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10145,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6778,"num_write_kb":27112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5841},{"shards":"0,2","objects":4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103276+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:02:08.103276+0000","last_peered":"2024-09-10T16:02:08.103276+0000","last_clean":"2024-09-10T16:02:08.103276+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:02:08.103276+0000","last_undegraded":"2024-09-10T16:02:08.103276+0000","last_fullsized":"2024-09-10T16:02:08.103276+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'10053","reported_seq":10770,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T16:02:22.464862+0000","last_change":"2024-09-10T16:02:06.480033+0000","last_active":"2024-09-10T16:02:22.464862+0000","last_peered":"2024-09-10T16:02:22.464862+0000","last_clean":"2024-09-10T15:59:14.115595+0000","last_became_active":"2024-09-10T15:59:16.244089+0000","last_became_peered":"2024-09-10T15:59:16.244089+0000","last_unstale":"2024-09-10T16:02:22.464862+0000","last_undegraded":"2024-09-10T16:02:22.464862+0000","last_fullsized":"2024-09-10T15:59:15.774437+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41144320,"num_objects":10045,"num_object_clones":0,"num_object_copies":20090,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3916,"num_objects_unfound":0,"num_objects_dirty":10045,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6443,"num_write_kb":25772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":287,"num_bytes_recovered":1175552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6129},{"shards":"0,2","objects":3916}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":93,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103338+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:02:08.103338+0000","last_peered":"2024-09-10T16:02:08.103338+0000","last_clean":"2024-09-10T16:02:08.103338+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:02:08.103338+0000","last_undegraded":"2024-09-10T16:02:08.103338+0000","last_fullsized":"2024-09-10T16:02:08.103338+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'10254","reported_seq":10394,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:22.486421+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:02:22.486421+0000","last_peered":"2024-09-10T16:02:22.486421+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:02:22.486421+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"36'200","ondisk_log_start":"36'200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41984000,"num_objects":10250,"num_object_clones":0,"num_object_copies":20500,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10250,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6509,"num_write_kb":26036,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5879},{"shards":"0,2","objects":4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104431+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:02:08.104431+0000","last_peered":"2024-09-10T16:02:08.104431+0000","last_clean":"2024-09-10T16:02:08.104431+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:02:08.104431+0000","last_undegraded":"2024-09-10T16:02:08.104431+0000","last_fullsized":"2024-09-10T16:02:08.104431+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104410+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:02:08.104410+0000","last_peered":"2024-09-10T16:02:08.104410+0000","last_clean":"2024-09-10T16:02:08.104410+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:02:08.104410+0000","last_undegraded":"2024-09-10T16:02:08.104410+0000","last_fullsized":"2024-09-10T16:02:08.104410+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'10319","reported_seq":10463,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:22.431742+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:02:22.431742+0000","last_peered":"2024-09-10T16:02:22.431742+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:02:22.431742+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"36'300","ondisk_log_start":"36'300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42218438,"num_objects":10309,"num_object_clones":0,"num_object_copies":20618,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10309,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6482,"num_write_kb":25928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5986},{"shards":"0,2","objects":4323}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":667964126,"num_objects":163086,"num_object_clones":0,"num_object_copies":326172,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42911,"num_objects_misplaced":3916,"num_objects_unfound":0,"num_objects_dirty":163086,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":120932,"num_write_kb":483728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4559,"num_bytes_recovered":18669568,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1357385728,"data_stored":1357233982,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160958,"ondisk_log_size":160958,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986229,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":752316,"kb_used_data":463788,"kb_used_omap":4,"kb_used_meta":288507,"kb_avail":93619524,"statfs":{"total":96636764160,"available":95866392576,"internally_reserved":0,"allocated":474918912,"data_stored":474630751,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":295431850},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,2,5,2,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659175,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":524000,"kb_used_data":303504,"kb_used_omap":4,"kb_used_meta":220475,"kb_avail":93847840,"statfs":{"total":96636764160,"available":96100188160,"internally_reserved":0,"allocated":310788096,"data_stored":310452502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":225767086},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,4,1,1,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182457,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":903736,"kb_used_data":556968,"kb_used_omap":4,"kb_used_meta":346747,"kb_avail":93468104,"statfs":{"total":96636764160,"available":95711338496,"internally_reserved":0,"allocated":570335232,"data_stored":570042994,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":355069610},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,6,3,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":570523648,"data_stored":570487518,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":311590912,"data_stored":311511426,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":475271168,"data_stored":475235038,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:02:26.206 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:02:26.765 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:02:26.771 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:02:26.788 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":193,"stamp":"2024-09-10T16:02:25.612533+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":667964145,"num_objects":163088,"num_object_clones":0,"num_object_copies":326176,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42911,"num_objects_misplaced":3916,"num_objects_unfound":0,"num_objects_dirty":163088,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":120934,"num_write_kb":483730,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4561,"num_bytes_recovered":18669587,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160961,"ondisk_log_size":160961,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2180052,"kb_used_data":1324260,"kb_used_omap":12,"kb_used_meta":855731,"kb_avail":280935468,"statfs":{"total":289910292480,"available":287677919232,"internally_reserved":0,"allocated":1356042240,"data_stored":1355126247,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":876268546},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,7,12,6,11,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":20942848,"num_objects":5113,"num_object_clones":0,"num_object_copies":10226,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-160,"num_objects_unfound":0,"num_objects_dirty":5113,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5113,"num_write_kb":20452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":160,"num_bytes_recovered":655360,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005062"},"pg_stats":[{"pgid":"2.f","version":"45'10062","reported_seq":10136,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:22.535255+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:02:22.535255+0000","last_peered":"2024-09-10T16:02:22.535255+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:02:22.535255+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41213952,"num_objects":10062,"num_object_clones":0,"num_object_copies":20124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10062,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10062,"num_write_kb":40248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5803},{"shards":"0,2","objects":4258}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'10197","reported_seq":10309,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:24.773296+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:02:24.773296+0000","last_peered":"2024-09-10T16:02:24.773296+0000","last_clean":"2024-09-10T16:02:24.773296+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:02:24.773296+0000","last_undegraded":"2024-09-10T16:02:24.773296+0000","last_fullsized":"2024-09-10T16:02:24.773296+0000","mapping_epoch":35,"log_start":"36'100","ondisk_log_start":"36'100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41743302,"num_objects":10193,"num_object_clones":0,"num_object_copies":20386,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10193,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":8302,"num_write_kb":33208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'10193","reported_seq":18870,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:25.038304+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:02:25.038304+0000","last_peered":"2024-09-10T16:02:25.038304+0000","last_clean":"2024-09-10T16:02:25.038304+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:02:25.038304+0000","last_undegraded":"2024-09-10T16:02:25.038304+0000","last_fullsized":"2024-09-10T16:02:25.038304+0000","mapping_epoch":44,"log_start":"36'100","ondisk_log_start":"36'100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41734144,"num_objects":10189,"num_object_clones":0,"num_object_copies":20378,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10189,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8197,"num_write_kb":32788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'10121","reported_seq":10201,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:24.820679+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:02:24.820679+0000","last_peered":"2024-09-10T16:02:24.820679+0000","last_clean":"2024-09-10T16:02:24.820679+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:02:24.820679+0000","last_undegraded":"2024-09-10T16:02:24.820679+0000","last_fullsized":"2024-09-10T16:02:24.820679+0000","mapping_epoch":35,"log_start":"36'100","ondisk_log_start":"36'100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41455616,"num_objects":10121,"num_object_clones":0,"num_object_copies":20242,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10121,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10121,"num_write_kb":40484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'10161","reported_seq":10290,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:22.523201+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:02:22.523201+0000","last_peered":"2024-09-10T16:02:22.523201+0000","last_clean":"2024-09-10T16:02:22.523201+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:02:22.523201+0000","last_undegraded":"2024-09-10T16:02:22.523201+0000","last_fullsized":"2024-09-10T16:02:22.523201+0000","mapping_epoch":34,"log_start":"36'100","ondisk_log_start":"36'100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41591267,"num_objects":10155,"num_object_clones":0,"num_object_copies":20310,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10155,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6872,"num_write_kb":27488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'10253","reported_seq":10390,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:22.240514+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:02:22.240514+0000","last_peered":"2024-09-10T16:02:22.240514+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:02:22.240514+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"36'200","ondisk_log_start":"36'200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41976291,"num_objects":10249,"num_object_clones":0,"num_object_copies":20498,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10249,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6865,"num_write_kb":27460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5878},{"shards":"0,2","objects":4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'10283","reported_seq":10420,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:22.430316+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:02:22.430316+0000","last_peered":"2024-09-10T16:02:22.430316+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:02:22.430316+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"36'200","ondisk_log_start":"36'200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42095558,"num_objects":10279,"num_object_clones":0,"num_object_copies":20558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10279,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":6782,"num_write_kb":27128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6042},{"shards":"0,2","objects":4237}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'10399","reported_seq":10529,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:22.475139+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:02:22.475139+0000","last_peered":"2024-09-10T16:02:22.475139+0000","last_clean":"2024-09-10T16:02:22.475139+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:02:22.475139+0000","last_undegraded":"2024-09-10T16:02:22.475139+0000","last_fullsized":"2024-09-10T16:02:22.475139+0000","mapping_epoch":34,"log_start":"36'300","ondisk_log_start":"36'300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42586112,"num_objects":10397,"num_object_clones":0,"num_object_copies":20794,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10397,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7246,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104219+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:02:08.104219+0000","last_peered":"2024-09-10T16:02:08.104219+0000","last_clean":"2024-09-10T16:02:08.104219+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:02:08.104219+0000","last_undegraded":"2024-09-10T16:02:08.104219+0000","last_fullsized":"2024-09-10T16:02:08.104219+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'9998","reported_seq":10137,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:24.800862+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:02:24.800862+0000","last_peered":"2024-09-10T16:02:24.800862+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:02:24.800862+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":9998,"log_dups_size":0,"ondisk_log_size":9998,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40923619,"num_objects":9992,"num_object_clones":0,"num_object_copies":19984,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9992,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7817,"num_write_kb":31268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4123},{"shards":"2","objects":5869}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":93,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.102747+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:02:08.102747+0000","last_peered":"2024-09-10T16:02:08.102747+0000","last_clean":"2024-09-10T16:02:08.102747+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:02:08.102747+0000","last_undegraded":"2024-09-10T16:02:08.102747+0000","last_fullsized":"2024-09-10T16:02:08.102747+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'10164","reported_seq":10293,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:22.491380+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:02:22.491380+0000","last_peered":"2024-09-10T16:02:22.491380+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:02:22.491380+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"36'100","ondisk_log_start":"36'100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41598976,"num_objects":10156,"num_object_clones":0,"num_object_copies":20312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10156,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6668,"num_write_kb":26672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5901},{"shards":"0,2","objects":4255}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":104,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.105055+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:02:08.105055+0000","last_peered":"2024-09-10T16:02:08.105055+0000","last_clean":"2024-09-10T16:02:08.105055+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:02:08.105055+0000","last_undegraded":"2024-09-10T16:02:08.105055+0000","last_fullsized":"2024-09-10T16:02:08.105055+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'10169","reported_seq":10303,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:24.699174+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:02:24.699174+0000","last_peered":"2024-09-10T16:02:24.699174+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:02:24.699174+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"36'100","ondisk_log_start":"36'100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41632227,"num_objects":10165,"num_object_clones":0,"num_object_copies":20330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10165,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7849,"num_write_kb":31396,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4304},{"shards":"2","objects":5861}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104572+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:02:08.104572+0000","last_peered":"2024-09-10T16:02:08.104572+0000","last_clean":"2024-09-10T16:02:08.104572+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:02:08.104572+0000","last_undegraded":"2024-09-10T16:02:08.104572+0000","last_fullsized":"2024-09-10T16:02:08.104572+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'10385","reported_seq":10514,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:24.796185+0000","last_change":"2024-09-10T15:59:16.174113+0000","last_active":"2024-09-10T16:02:24.796185+0000","last_peered":"2024-09-10T16:02:24.796185+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:02:24.796185+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"36'300","ondisk_log_start":"36'300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42512384,"num_objects":10379,"num_object_clones":0,"num_object_copies":20758,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4354,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10379,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7939,"num_write_kb":31756,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4352},{"shards":"2","objects":6027}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'10147","reported_seq":10281,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:22.411555+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:02:22.411555+0000","last_peered":"2024-09-10T16:02:22.411555+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:02:22.411555+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"36'100","ondisk_log_start":"36'100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41553920,"num_objects":10145,"num_object_clones":0,"num_object_copies":20290,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10145,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6778,"num_write_kb":27112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5841},{"shards":"0,2","objects":4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103276+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:02:08.103276+0000","last_peered":"2024-09-10T16:02:08.103276+0000","last_clean":"2024-09-10T16:02:08.103276+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:02:08.103276+0000","last_undegraded":"2024-09-10T16:02:08.103276+0000","last_fullsized":"2024-09-10T16:02:08.103276+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'10053","reported_seq":10770,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T16:02:22.464862+0000","last_change":"2024-09-10T16:02:06.480033+0000","last_active":"2024-09-10T16:02:22.464862+0000","last_peered":"2024-09-10T16:02:22.464862+0000","last_clean":"2024-09-10T15:59:14.115595+0000","last_became_active":"2024-09-10T15:59:16.244089+0000","last_became_peered":"2024-09-10T15:59:16.244089+0000","last_unstale":"2024-09-10T16:02:22.464862+0000","last_undegraded":"2024-09-10T16:02:22.464862+0000","last_fullsized":"2024-09-10T15:59:15.774437+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41144320,"num_objects":10045,"num_object_clones":0,"num_object_copies":20090,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3916,"num_objects_unfound":0,"num_objects_dirty":10045,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6443,"num_write_kb":25772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":287,"num_bytes_recovered":1175552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6129},{"shards":"0,2","objects":3916}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":93,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103338+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:02:08.103338+0000","last_peered":"2024-09-10T16:02:08.103338+0000","last_clean":"2024-09-10T16:02:08.103338+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:02:08.103338+0000","last_undegraded":"2024-09-10T16:02:08.103338+0000","last_fullsized":"2024-09-10T16:02:08.103338+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'10254","reported_seq":10394,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:22.486421+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:02:22.486421+0000","last_peered":"2024-09-10T16:02:22.486421+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:02:22.486421+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"36'200","ondisk_log_start":"36'200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41984000,"num_objects":10250,"num_object_clones":0,"num_object_copies":20500,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10250,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6509,"num_write_kb":26036,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5879},{"shards":"0,2","objects":4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104431+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:02:08.104431+0000","last_peered":"2024-09-10T16:02:08.104431+0000","last_clean":"2024-09-10T16:02:08.104431+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:02:08.104431+0000","last_undegraded":"2024-09-10T16:02:08.104431+0000","last_fullsized":"2024-09-10T16:02:08.104431+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104410+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:02:08.104410+0000","last_peered":"2024-09-10T16:02:08.104410+0000","last_clean":"2024-09-10T16:02:08.104410+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:02:08.104410+0000","last_undegraded":"2024-09-10T16:02:08.104410+0000","last_fullsized":"2024-09-10T16:02:08.104410+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'10319","reported_seq":10463,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:22.431742+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:02:22.431742+0000","last_peered":"2024-09-10T16:02:22.431742+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:02:22.431742+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"36'300","ondisk_log_start":"36'300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42218438,"num_objects":10309,"num_object_clones":0,"num_object_copies":20618,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10309,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6482,"num_write_kb":25928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5986},{"shards":"0,2","objects":4323}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":667964126,"num_objects":163086,"num_object_clones":0,"num_object_copies":326172,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42911,"num_objects_misplaced":3916,"num_objects_unfound":0,"num_objects_dirty":163086,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":120932,"num_write_kb":483728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4559,"num_bytes_recovered":18669568,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1357385728,"data_stored":1357233982,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160958,"ondisk_log_size":160958,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986229,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":752316,"kb_used_data":463788,"kb_used_omap":4,"kb_used_meta":288507,"kb_avail":93619524,"statfs":{"total":96636764160,"available":95866392576,"internally_reserved":0,"allocated":474918912,"data_stored":474630751,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":295431850},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,2,5,2,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659175,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":524000,"kb_used_data":303504,"kb_used_omap":4,"kb_used_meta":220475,"kb_avail":93847840,"statfs":{"total":96636764160,"available":96100188160,"internally_reserved":0,"allocated":310788096,"data_stored":310452502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":225767086},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,4,1,1,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182457,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":903736,"kb_used_data":556968,"kb_used_omap":4,"kb_used_meta":346747,"kb_avail":93468104,"statfs":{"total":96636764160,"available":95711338496,"internally_reserved":0,"allocated":570335232,"data_stored":570042994,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":355069610},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,6,3,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":570523648,"data_stored":570487518,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":311590912,"data_stored":311511426,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":475271168,"data_stored":475235038,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:02:26.790 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-10T16:02:26.790 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "45'10062", 'reported_seq': 10136, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:02:22.535255+0000', 'last_change': '2024-09-10T15:59:16.281558+0000', 'last_active': '2024-09-10T16:02:22.535255+0000', 'last_peered': '2024-09-10T16:02:22.535255+0000', 'last_clean': '2024-09-10T15:59:14.132456+0000', 'last_became_active': '2024-09-10T15:59:16.274407+0000', 'last_became_peered': '2024-09-10T15:59:16.274407+0000', 'last_unstale': '2024-09-10T16:02:22.535255+0000', 'last_undegraded': '2024-09-10T15:59:15.851279+0000', 'last_fullsized': '2024-09-10T15:59:15.774356+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'objects_scrubbed': 0, 'log_size': 10062, 'log_dups_size': 0, 'ondisk_log_size': 10062, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 41213952, 'num_objects': 10062, 'num_object_clones': 0, 'num_object_copies': 20124, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4259, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10062, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10062, 'num_write_kb': 40248, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5803}, {'shards': '0,2', 'objects': 4258}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:02:26.791 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-10T16:02:26.791 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "45'10253", 'reported_seq': 10390, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:02:22.240514+0000', 'last_change': '2024-09-10T15:59:16.308264+0000', 'last_active': '2024-09-10T16:02:22.240514+0000', 'last_peered': '2024-09-10T16:02:22.240514+0000', 'last_clean': '2024-09-10T15:59:14.111551+0000', 'last_became_active': '2024-09-10T15:59:16.300040+0000', 'last_became_peered': '2024-09-10T15:59:16.300040+0000', 'last_unstale': '2024-09-10T16:02:22.240514+0000', 'last_undegraded': '2024-09-10T15:59:15.878234+0000', 'last_fullsized': '2024-09-10T15:59:15.769328+0000', 'mapping_epoch': 42, 'log_start': "36'200", 'ondisk_log_start': "36'200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3442", 'last_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'objects_scrubbed': 3384, 'log_size': 10053, 'log_dups_size': 0, 'ondisk_log_size': 10053, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1292, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 41976291, 'num_objects': 10249, 'num_object_clones': 0, 'num_object_copies': 20498, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4372, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10249, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 6865, 'num_write_kb': 27460, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5878}, {'shards': '0,2', 'objects': 4371}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:02:26.791 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-10T16:02:26.791 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "45'10283", 'reported_seq': 10420, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:02:22.430316+0000', 'last_change': '2024-09-10T15:59:16.331899+0000', 'last_active': '2024-09-10T16:02:22.430316+0000', 'last_peered': '2024-09-10T16:02:22.430316+0000', 'last_clean': '2024-09-10T15:59:14.139436+0000', 'last_became_active': '2024-09-10T15:59:16.318715+0000', 'last_became_peered': '2024-09-10T15:59:16.318715+0000', 'last_unstale': '2024-09-10T16:02:22.430316+0000', 'last_undegraded': '2024-09-10T15:59:15.877524+0000', 'last_fullsized': '2024-09-10T15:59:15.769042+0000', 'mapping_epoch': 42, 'log_start': "36'200", 'ondisk_log_start': "36'200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3537", 'last_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'objects_scrubbed': 3497, 'log_size': 10083, 'log_dups_size': 0, 'ondisk_log_size': 10083, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1224, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 42095558, 'num_objects': 10279, 'num_object_clones': 0, 'num_object_copies': 20558, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4237, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10279, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 6782, 'num_write_kb': 27128, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6042}, {'shards': '0,2', 'objects': 4237}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:02:26.791 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-10T16:02:26.792 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "45'9998", 'reported_seq': 10137, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:02:24.800862+0000', 'last_change': '2024-09-10T15:59:16.175680+0000', 'last_active': '2024-09-10T16:02:24.800862+0000', 'last_peered': '2024-09-10T16:02:24.800862+0000', 'last_clean': '2024-09-10T15:59:14.088187+0000', 'last_became_active': '2024-09-10T15:59:16.165810+0000', 'last_became_peered': '2024-09-10T15:59:16.165810+0000', 'last_unstale': '2024-09-10T16:02:24.800862+0000', 'last_undegraded': '2024-09-10T15:59:15.822098+0000', 'last_fullsized': '2024-09-10T15:59:15.769123+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2218", 'last_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'objects_scrubbed': 2175, 'log_size': 9998, 'log_dups_size': 0, 'ondisk_log_size': 9998, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 825, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40923619, 'num_objects': 9992, 'num_object_clones': 0, 'num_object_copies': 19984, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4124, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9992, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 7817, 'num_write_kb': 31268, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4123}, {'shards': '2', 'objects': 5869}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:02:26.792 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-10T16:02:26.792 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "45'10164", 'reported_seq': 10293, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:02:22.491380+0000', 'last_change': '2024-09-10T15:59:16.277742+0000', 'last_active': '2024-09-10T16:02:22.491380+0000', 'last_peered': '2024-09-10T16:02:22.491380+0000', 'last_clean': '2024-09-10T15:59:14.129081+0000', 'last_became_active': '2024-09-10T15:59:16.269835+0000', 'last_became_peered': '2024-09-10T15:59:16.269835+0000', 'last_unstale': '2024-09-10T16:02:22.491380+0000', 'last_undegraded': '2024-09-10T15:59:15.860857+0000', 'last_fullsized': '2024-09-10T15:59:15.771410+0000', 'mapping_epoch': 42, 'log_start': "36'100", 'ondisk_log_start': "36'100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3557", 'last_scrub_stamp': '2024-09-10T15:59:02.493678+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:02.493678+0000', 'objects_scrubbed': 3488, 'log_size': 10064, 'log_dups_size': 0, 'ondisk_log_size': 10064, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1472, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 41598976, 'num_objects': 10156, 'num_object_clones': 0, 'num_object_copies': 20312, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4257, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10156, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6668, 'num_write_kb': 26672, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5901}, {'shards': '0,2', 'objects': 4255}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:02:26.792 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-10T16:02:26.792 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "45'10169", 'reported_seq': 10303, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:02:24.699174+0000', 'last_change': '2024-09-10T15:59:16.284311+0000', 'last_active': '2024-09-10T16:02:24.699174+0000', 'last_peered': '2024-09-10T16:02:24.699174+0000', 'last_clean': '2024-09-10T15:59:14.116041+0000', 'last_became_active': '2024-09-10T15:59:16.274543+0000', 'last_became_peered': '2024-09-10T15:59:16.274543+0000', 'last_unstale': '2024-09-10T16:02:24.699174+0000', 'last_undegraded': '2024-09-10T15:59:15.812484+0000', 'last_fullsized': '2024-09-10T15:59:15.770885+0000', 'mapping_epoch': 42, 'log_start': "36'100", 'ondisk_log_start': "36'100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2356", 'last_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'objects_scrubbed': 2316, 'log_size': 10069, 'log_dups_size': 0, 'ondisk_log_size': 10069, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 783, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 41632227, 'num_objects': 10165, 'num_object_clones': 0, 'num_object_copies': 20330, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10165, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 7849, 'num_write_kb': 31396, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4304}, {'shards': '2', 'objects': 5861}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:02:26.793 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-10T16:02:26.793 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "45'10385", 'reported_seq': 10514, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:02:24.796185+0000', 'last_change': '2024-09-10T15:59:16.174113+0000', 'last_active': '2024-09-10T16:02:24.796185+0000', 'last_peered': '2024-09-10T16:02:24.796185+0000', 'last_clean': '2024-09-10T15:59:14.110654+0000', 'last_became_active': '2024-09-10T15:59:16.164645+0000', 'last_became_peered': '2024-09-10T15:59:16.164645+0000', 'last_unstale': '2024-09-10T16:02:24.796185+0000', 'last_undegraded': '2024-09-10T15:59:15.803593+0000', 'last_fullsized': '2024-09-10T15:59:15.769083+0000', 'mapping_epoch': 42, 'log_start': "36'300", 'ondisk_log_start': "36'300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2486", 'last_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'objects_scrubbed': 2440, 'log_size': 10085, 'log_dups_size': 0, 'ondisk_log_size': 10085, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 792, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 42512384, 'num_objects': 10379, 'num_object_clones': 0, 'num_object_copies': 20758, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4354, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10379, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7939, 'num_write_kb': 31756, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4352}, {'shards': '2', 'objects': 6027}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:02:26.793 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-10T16:02:26.793 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "45'10147", 'reported_seq': 10281, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:02:22.411555+0000', 'last_change': '2024-09-10T15:59:16.315641+0000', 'last_active': '2024-09-10T16:02:22.411555+0000', 'last_peered': '2024-09-10T16:02:22.411555+0000', 'last_clean': '2024-09-10T15:59:14.117001+0000', 'last_became_active': '2024-09-10T15:59:16.302245+0000', 'last_became_peered': '2024-09-10T15:59:16.302245+0000', 'last_unstale': '2024-09-10T16:02:22.411555+0000', 'last_undegraded': '2024-09-10T15:59:15.845012+0000', 'last_fullsized': '2024-09-10T15:59:15.770337+0000', 'mapping_epoch': 42, 'log_start': "36'100", 'ondisk_log_start': "36'100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3404", 'last_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'objects_scrubbed': 3367, 'log_size': 10047, 'log_dups_size': 0, 'ondisk_log_size': 10047, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 983, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 41553920, 'num_objects': 10145, 'num_object_clones': 0, 'num_object_copies': 20290, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10145, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6778, 'num_write_kb': 27112, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5841}, {'shards': '0,2', 'objects': 4304}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:02:26.793 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-10T16:02:26.794 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "45'10053", 'reported_seq': 10770, 'reported_epoch': 45, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-10T16:02:22.464862+0000', 'last_change': '2024-09-10T16:02:06.480033+0000', 'last_active': '2024-09-10T16:02:22.464862+0000', 'last_peered': '2024-09-10T16:02:22.464862+0000', 'last_clean': '2024-09-10T15:59:14.115595+0000', 'last_became_active': '2024-09-10T15:59:16.244089+0000', 'last_became_peered': '2024-09-10T15:59:16.244089+0000', 'last_unstale': '2024-09-10T16:02:22.464862+0000', 'last_undegraded': '2024-09-10T16:02:22.464862+0000', 'last_fullsized': '2024-09-10T15:59:15.774437+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3653", 'last_scrub_stamp': '2024-09-10T15:59:05.298468+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:05.298468+0000', 'objects_scrubbed': 3602, 'log_size': 10053, 'log_dups_size': 0, 'ondisk_log_size': 10053, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1289, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 41144320, 'num_objects': 10045, 'num_object_clones': 0, 'num_object_copies': 20090, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 3916, 'num_objects_unfound': 0, 'num_objects_dirty': 10045, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6443, 'num_write_kb': 25772, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 287, 'num_bytes_recovered': 1175552, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6129}, {'shards': '0,2', 'objects': 3916}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:02:26.794 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-10T16:02:26.794 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "45'10254", 'reported_seq': 10394, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:02:22.486421+0000', 'last_change': '2024-09-10T15:59:16.311291+0000', 'last_active': '2024-09-10T16:02:22.486421+0000', 'last_peered': '2024-09-10T16:02:22.486421+0000', 'last_clean': '2024-09-10T15:59:14.116363+0000', 'last_became_active': '2024-09-10T15:59:16.301244+0000', 'last_became_peered': '2024-09-10T15:59:16.301244+0000', 'last_unstale': '2024-09-10T16:02:22.486421+0000', 'last_undegraded': '2024-09-10T15:59:15.883434+0000', 'last_fullsized': '2024-09-10T15:59:15.772756+0000', 'mapping_epoch': 42, 'log_start': "36'200", 'ondisk_log_start': "36'200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3789", 'last_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'objects_scrubbed': 3741, 'log_size': 10054, 'log_dups_size': 0, 'ondisk_log_size': 10054, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1440, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 41984000, 'num_objects': 10250, 'num_object_clones': 0, 'num_object_copies': 20500, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4373, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10250, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6509, 'num_write_kb': 26036, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5879}, {'shards': '0,2', 'objects': 4371}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:02:26.794 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-10T16:02:26.794 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "45'10319", 'reported_seq': 10463, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:02:22.431742+0000', 'last_change': '2024-09-10T15:59:16.313409+0000', 'last_active': '2024-09-10T16:02:22.431742+0000', 'last_peered': '2024-09-10T16:02:22.431742+0000', 'last_clean': '2024-09-10T15:59:14.129780+0000', 'last_became_active': '2024-09-10T15:59:16.300932+0000', 'last_became_peered': '2024-09-10T15:59:16.300932+0000', 'last_unstale': '2024-09-10T16:02:22.431742+0000', 'last_undegraded': '2024-09-10T15:59:15.887823+0000', 'last_fullsized': '2024-09-10T15:59:15.768513+0000', 'mapping_epoch': 42, 'log_start': "36'300", 'ondisk_log_start': "36'300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3880", 'last_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'objects_scrubbed': 3827, 'log_size': 10019, 'log_dups_size': 0, 'ondisk_log_size': 10019, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1397, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 42218438, 'num_objects': 10309, 'num_object_clones': 0, 'num_object_copies': 20618, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4325, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10309, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6482, 'num_write_kb': 25928, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5986}, {'shards': '0,2', 'objects': 4323}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:02:26.795 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T16:02:26.998 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 234 20 164615 164595 2.74721 1.55078 0.0537696 0.0284308 2024-09-10T16:02:27.453 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:02:27.453 INFO:teuthology.orchestra.run.smithi115.stdout:{"fsid":"7effee29-df05-4351-9741-ee203564ed81","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 42911/326176 objects degraded (13.156%), 10 pgs degraded, 11 pgs undersized","count":21},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":311,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983954,"num_in_osds":3,"osd_in_since":1725983947,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":163088,"data_bytes":667964145,"bytes_used":2232373248,"bytes_avail":287677919232,"bytes_total":289910292480,"degraded_objects":42911,"degraded_total":326176,"degraded_ratio":0.13155780928087904,"misplaced_objects":3916,"misplaced_total":326176,"misplaced_ratio":0.012005788286078683,"recovering_objects_per_sec":13,"recovering_bytes_per_sec":54590,"recovering_keys_per_sec":0,"num_objects_recovered":160,"num_bytes_recovered":655360,"num_keys_recovered":0,"write_bytes_sec":1744501,"read_op_per_sec":0,"write_op_per_sec":425},"fsmap":{"epoch":1,"btime":"2024-09-10T15:57:14:967120+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T16:00:13.560776+0000","services":{}},"progress_events":{}} 2024-09-10T16:02:27.478 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-10T16:02:27.479 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:02:27.998 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 235 20 164982 164962 2.74162 1.43359 0.0434831 0.0284884 2024-09-10T16:02:28.001 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:02:28.003 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:02:28.023 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":194,"stamp":"2024-09-10T16:02:27.613363+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":667964145,"num_objects":163088,"num_object_clones":0,"num_object_copies":326176,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42911,"num_objects_misplaced":3916,"num_objects_unfound":0,"num_objects_dirty":163088,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":120934,"num_write_kb":483730,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4561,"num_bytes_recovered":18669587,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160961,"ondisk_log_size":160961,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2180052,"kb_used_data":1324260,"kb_used_omap":12,"kb_used_meta":855731,"kb_avail":280935468,"statfs":{"total":289910292480,"available":287677919232,"internally_reserved":0,"allocated":1356042240,"data_stored":1355126247,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":876268546},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,7,12,6,11,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":17784832,"num_objects":4342,"num_object_clones":0,"num_object_copies":8684,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-160,"num_objects_unfound":0,"num_objects_dirty":4342,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4342,"num_write_kb":17368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":160,"num_bytes_recovered":655360,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005060"},"pg_stats":[{"pgid":"2.f","version":"45'10062","reported_seq":10136,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:22.535255+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:02:22.535255+0000","last_peered":"2024-09-10T16:02:22.535255+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:02:22.535255+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41213952,"num_objects":10062,"num_object_clones":0,"num_object_copies":20124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10062,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10062,"num_write_kb":40248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5803},{"shards":"0,2","objects":4258}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'10197","reported_seq":10309,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:24.773296+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:02:24.773296+0000","last_peered":"2024-09-10T16:02:24.773296+0000","last_clean":"2024-09-10T16:02:24.773296+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:02:24.773296+0000","last_undegraded":"2024-09-10T16:02:24.773296+0000","last_fullsized":"2024-09-10T16:02:24.773296+0000","mapping_epoch":35,"log_start":"36'100","ondisk_log_start":"36'100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41743302,"num_objects":10193,"num_object_clones":0,"num_object_copies":20386,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10193,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":8302,"num_write_kb":33208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'10193","reported_seq":18870,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:25.038304+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:02:25.038304+0000","last_peered":"2024-09-10T16:02:25.038304+0000","last_clean":"2024-09-10T16:02:25.038304+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:02:25.038304+0000","last_undegraded":"2024-09-10T16:02:25.038304+0000","last_fullsized":"2024-09-10T16:02:25.038304+0000","mapping_epoch":44,"log_start":"36'100","ondisk_log_start":"36'100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41734144,"num_objects":10189,"num_object_clones":0,"num_object_copies":20378,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10189,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8197,"num_write_kb":32788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'10121","reported_seq":10201,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:24.820679+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:02:24.820679+0000","last_peered":"2024-09-10T16:02:24.820679+0000","last_clean":"2024-09-10T16:02:24.820679+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:02:24.820679+0000","last_undegraded":"2024-09-10T16:02:24.820679+0000","last_fullsized":"2024-09-10T16:02:24.820679+0000","mapping_epoch":35,"log_start":"36'100","ondisk_log_start":"36'100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41455616,"num_objects":10121,"num_object_clones":0,"num_object_copies":20242,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10121,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10121,"num_write_kb":40484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'10161","reported_seq":10290,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:22.523201+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:02:22.523201+0000","last_peered":"2024-09-10T16:02:22.523201+0000","last_clean":"2024-09-10T16:02:22.523201+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:02:22.523201+0000","last_undegraded":"2024-09-10T16:02:22.523201+0000","last_fullsized":"2024-09-10T16:02:22.523201+0000","mapping_epoch":34,"log_start":"36'100","ondisk_log_start":"36'100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41591267,"num_objects":10155,"num_object_clones":0,"num_object_copies":20310,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10155,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6872,"num_write_kb":27488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'10253","reported_seq":10390,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:22.240514+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:02:22.240514+0000","last_peered":"2024-09-10T16:02:22.240514+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:02:22.240514+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"36'200","ondisk_log_start":"36'200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41976291,"num_objects":10249,"num_object_clones":0,"num_object_copies":20498,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10249,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6865,"num_write_kb":27460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5878},{"shards":"0,2","objects":4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'10283","reported_seq":10420,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:22.430316+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:02:22.430316+0000","last_peered":"2024-09-10T16:02:22.430316+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:02:22.430316+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"36'200","ondisk_log_start":"36'200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42095558,"num_objects":10279,"num_object_clones":0,"num_object_copies":20558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10279,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":6782,"num_write_kb":27128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6042},{"shards":"0,2","objects":4237}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'10399","reported_seq":10529,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:22.475139+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:02:22.475139+0000","last_peered":"2024-09-10T16:02:22.475139+0000","last_clean":"2024-09-10T16:02:22.475139+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:02:22.475139+0000","last_undegraded":"2024-09-10T16:02:22.475139+0000","last_fullsized":"2024-09-10T16:02:22.475139+0000","mapping_epoch":34,"log_start":"36'300","ondisk_log_start":"36'300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42586112,"num_objects":10397,"num_object_clones":0,"num_object_copies":20794,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10397,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7246,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104219+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:02:08.104219+0000","last_peered":"2024-09-10T16:02:08.104219+0000","last_clean":"2024-09-10T16:02:08.104219+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:02:08.104219+0000","last_undegraded":"2024-09-10T16:02:08.104219+0000","last_fullsized":"2024-09-10T16:02:08.104219+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'9998","reported_seq":10137,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:24.800862+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:02:24.800862+0000","last_peered":"2024-09-10T16:02:24.800862+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:02:24.800862+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":9998,"log_dups_size":0,"ondisk_log_size":9998,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40923619,"num_objects":9992,"num_object_clones":0,"num_object_copies":19984,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9992,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7817,"num_write_kb":31268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4123},{"shards":"2","objects":5869}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":93,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.102747+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:02:08.102747+0000","last_peered":"2024-09-10T16:02:08.102747+0000","last_clean":"2024-09-10T16:02:08.102747+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:02:08.102747+0000","last_undegraded":"2024-09-10T16:02:08.102747+0000","last_fullsized":"2024-09-10T16:02:08.102747+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'10164","reported_seq":10293,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:22.491380+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:02:22.491380+0000","last_peered":"2024-09-10T16:02:22.491380+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:02:22.491380+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"36'100","ondisk_log_start":"36'100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41598976,"num_objects":10156,"num_object_clones":0,"num_object_copies":20312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10156,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6668,"num_write_kb":26672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5901},{"shards":"0,2","objects":4255}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":104,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.105055+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:02:08.105055+0000","last_peered":"2024-09-10T16:02:08.105055+0000","last_clean":"2024-09-10T16:02:08.105055+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:02:08.105055+0000","last_undegraded":"2024-09-10T16:02:08.105055+0000","last_fullsized":"2024-09-10T16:02:08.105055+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'10169","reported_seq":10303,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:24.699174+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:02:24.699174+0000","last_peered":"2024-09-10T16:02:24.699174+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:02:24.699174+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"36'100","ondisk_log_start":"36'100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41632227,"num_objects":10165,"num_object_clones":0,"num_object_copies":20330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10165,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7849,"num_write_kb":31396,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4304},{"shards":"2","objects":5861}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104572+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:02:08.104572+0000","last_peered":"2024-09-10T16:02:08.104572+0000","last_clean":"2024-09-10T16:02:08.104572+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:02:08.104572+0000","last_undegraded":"2024-09-10T16:02:08.104572+0000","last_fullsized":"2024-09-10T16:02:08.104572+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'10385","reported_seq":10514,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:24.796185+0000","last_change":"2024-09-10T15:59:16.174113+0000","last_active":"2024-09-10T16:02:24.796185+0000","last_peered":"2024-09-10T16:02:24.796185+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:02:24.796185+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"36'300","ondisk_log_start":"36'300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42512384,"num_objects":10379,"num_object_clones":0,"num_object_copies":20758,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4354,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10379,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7939,"num_write_kb":31756,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4352},{"shards":"2","objects":6027}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'10147","reported_seq":10281,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:22.411555+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:02:22.411555+0000","last_peered":"2024-09-10T16:02:22.411555+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:02:22.411555+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"36'100","ondisk_log_start":"36'100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41553920,"num_objects":10145,"num_object_clones":0,"num_object_copies":20290,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10145,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6778,"num_write_kb":27112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5841},{"shards":"0,2","objects":4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103276+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:02:08.103276+0000","last_peered":"2024-09-10T16:02:08.103276+0000","last_clean":"2024-09-10T16:02:08.103276+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:02:08.103276+0000","last_undegraded":"2024-09-10T16:02:08.103276+0000","last_fullsized":"2024-09-10T16:02:08.103276+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'10053","reported_seq":10770,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T16:02:22.464862+0000","last_change":"2024-09-10T16:02:06.480033+0000","last_active":"2024-09-10T16:02:22.464862+0000","last_peered":"2024-09-10T16:02:22.464862+0000","last_clean":"2024-09-10T15:59:14.115595+0000","last_became_active":"2024-09-10T15:59:16.244089+0000","last_became_peered":"2024-09-10T15:59:16.244089+0000","last_unstale":"2024-09-10T16:02:22.464862+0000","last_undegraded":"2024-09-10T16:02:22.464862+0000","last_fullsized":"2024-09-10T15:59:15.774437+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41144320,"num_objects":10045,"num_object_clones":0,"num_object_copies":20090,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3916,"num_objects_unfound":0,"num_objects_dirty":10045,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6443,"num_write_kb":25772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":287,"num_bytes_recovered":1175552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6129},{"shards":"0,2","objects":3916}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":93,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103338+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:02:08.103338+0000","last_peered":"2024-09-10T16:02:08.103338+0000","last_clean":"2024-09-10T16:02:08.103338+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:02:08.103338+0000","last_undegraded":"2024-09-10T16:02:08.103338+0000","last_fullsized":"2024-09-10T16:02:08.103338+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'10254","reported_seq":10394,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:22.486421+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:02:22.486421+0000","last_peered":"2024-09-10T16:02:22.486421+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:02:22.486421+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"36'200","ondisk_log_start":"36'200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41984000,"num_objects":10250,"num_object_clones":0,"num_object_copies":20500,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10250,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6509,"num_write_kb":26036,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5879},{"shards":"0,2","objects":4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104431+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:02:08.104431+0000","last_peered":"2024-09-10T16:02:08.104431+0000","last_clean":"2024-09-10T16:02:08.104431+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:02:08.104431+0000","last_undegraded":"2024-09-10T16:02:08.104431+0000","last_fullsized":"2024-09-10T16:02:08.104431+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104410+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:02:08.104410+0000","last_peered":"2024-09-10T16:02:08.104410+0000","last_clean":"2024-09-10T16:02:08.104410+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:02:08.104410+0000","last_undegraded":"2024-09-10T16:02:08.104410+0000","last_fullsized":"2024-09-10T16:02:08.104410+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'10319","reported_seq":10463,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:22.431742+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:02:22.431742+0000","last_peered":"2024-09-10T16:02:22.431742+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:02:22.431742+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"36'300","ondisk_log_start":"36'300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42218438,"num_objects":10309,"num_object_clones":0,"num_object_copies":20618,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10309,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6482,"num_write_kb":25928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5986},{"shards":"0,2","objects":4323}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":667964126,"num_objects":163086,"num_object_clones":0,"num_object_copies":326172,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42911,"num_objects_misplaced":3916,"num_objects_unfound":0,"num_objects_dirty":163086,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":120932,"num_write_kb":483728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4559,"num_bytes_recovered":18669568,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1357385728,"data_stored":1357233982,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160958,"ondisk_log_size":160958,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986229,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":752316,"kb_used_data":463788,"kb_used_omap":4,"kb_used_meta":288507,"kb_avail":93619524,"statfs":{"total":96636764160,"available":95866392576,"internally_reserved":0,"allocated":474918912,"data_stored":474630751,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":295431850},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,2,5,2,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659175,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":524000,"kb_used_data":303504,"kb_used_omap":4,"kb_used_meta":220475,"kb_avail":93847840,"statfs":{"total":96636764160,"available":96100188160,"internally_reserved":0,"allocated":310788096,"data_stored":310452502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":225767086},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,4,1,1,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182457,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":903736,"kb_used_data":556968,"kb_used_omap":4,"kb_used_meta":346747,"kb_avail":93468104,"statfs":{"total":96636764160,"available":95711338496,"internally_reserved":0,"allocated":570335232,"data_stored":570042994,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":355069610},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,6,3,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":570523648,"data_stored":570487518,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":311590912,"data_stored":311511426,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":475271168,"data_stored":475235038,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:02:28.998 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 236 20 165428 165408 2.73739 1.74219 0.0168136 0.0285324 2024-09-10T16:02:29.998 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 237 20 165884 165864 2.73335 1.78125 0.0128948 0.0285744 2024-09-10T16:02:30.998 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 238 20 166334 166314 2.72925 1.75781 0.0240011 0.0286175 2024-09-10T16:02:31.026 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:02:31.585 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:02:31.586 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:02:31.611 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":195,"stamp":"2024-09-10T16:02:29.614091+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":673444593,"num_objects":164426,"num_object_clones":0,"num_object_copies":328852,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42911,"num_objects_misplaced":3829,"num_objects_unfound":0,"num_objects_dirty":164426,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":122272,"num_write_kb":489082,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4648,"num_bytes_recovered":19025939,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160799,"ondisk_log_size":160799,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2173832,"kb_used_data":1330520,"kb_used_omap":12,"kb_used_meta":843251,"kb_avail":280941688,"statfs":{"total":289910292480,"available":287684288512,"internally_reserved":0,"allocated":1362452480,"data_stored":1361536487,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":863489026},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,3,8,7,4,15,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":17629184,"num_objects":4304,"num_object_clones":0,"num_object_copies":8608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-173,"num_objects_unfound":0,"num_objects_dirty":4304,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4304,"num_write_kb":17216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":173,"num_bytes_recovered":708608,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004851"},"pg_stats":[{"pgid":"2.f","version":"45'10202","reported_seq":10276,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:27.606446+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:02:27.606446+0000","last_peered":"2024-09-10T16:02:27.606446+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:02:27.606446+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"36'200","ondisk_log_start":"36'200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41787392,"num_objects":10202,"num_object_clones":0,"num_object_copies":20404,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10202,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10202,"num_write_kb":40808,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5943},{"shards":"0,2","objects":4258}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'10197","reported_seq":10309,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:24.773296+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:02:24.773296+0000","last_peered":"2024-09-10T16:02:24.773296+0000","last_clean":"2024-09-10T16:02:24.773296+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:02:24.773296+0000","last_undegraded":"2024-09-10T16:02:24.773296+0000","last_fullsized":"2024-09-10T16:02:24.773296+0000","mapping_epoch":35,"log_start":"36'100","ondisk_log_start":"36'100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41743302,"num_objects":10193,"num_object_clones":0,"num_object_copies":20386,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10193,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":8302,"num_write_kb":33208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'10193","reported_seq":18870,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:25.038304+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:02:25.038304+0000","last_peered":"2024-09-10T16:02:25.038304+0000","last_clean":"2024-09-10T16:02:25.038304+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:02:25.038304+0000","last_undegraded":"2024-09-10T16:02:25.038304+0000","last_fullsized":"2024-09-10T16:02:25.038304+0000","mapping_epoch":44,"log_start":"36'100","ondisk_log_start":"36'100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41734144,"num_objects":10189,"num_object_clones":0,"num_object_copies":20378,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10189,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8197,"num_write_kb":32788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'10121","reported_seq":10201,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:24.820679+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:02:24.820679+0000","last_peered":"2024-09-10T16:02:24.820679+0000","last_clean":"2024-09-10T16:02:24.820679+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:02:24.820679+0000","last_undegraded":"2024-09-10T16:02:24.820679+0000","last_fullsized":"2024-09-10T16:02:24.820679+0000","mapping_epoch":35,"log_start":"36'100","ondisk_log_start":"36'100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41455616,"num_objects":10121,"num_object_clones":0,"num_object_copies":20242,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10121,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10121,"num_write_kb":40484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'10302","reported_seq":10432,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:27.613372+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:02:27.613372+0000","last_peered":"2024-09-10T16:02:27.613372+0000","last_clean":"2024-09-10T16:02:27.613372+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:02:27.613372+0000","last_undegraded":"2024-09-10T16:02:27.613372+0000","last_fullsized":"2024-09-10T16:02:27.613372+0000","mapping_epoch":34,"log_start":"36'300","ondisk_log_start":"36'300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42168803,"num_objects":10296,"num_object_clones":0,"num_object_copies":20592,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10296,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7013,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'10384","reported_seq":10521,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:27.642161+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:02:27.642161+0000","last_peered":"2024-09-10T16:02:27.642161+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:02:27.642161+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"36'300","ondisk_log_start":"36'300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42512867,"num_objects":10380,"num_object_clones":0,"num_object_copies":20760,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10380,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6996,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6009},{"shards":"0,2","objects":4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'10416","reported_seq":10553,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:27.475138+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:02:27.475138+0000","last_peered":"2024-09-10T16:02:27.475138+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:02:27.475138+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"36'400","ondisk_log_start":"36'400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42640326,"num_objects":10412,"num_object_clones":0,"num_object_copies":20824,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10412,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":6915,"num_write_kb":27660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6175},{"shards":"0,2","objects":4237}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'10517","reported_seq":10647,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:27.596557+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:02:27.596557+0000","last_peered":"2024-09-10T16:02:27.596557+0000","last_clean":"2024-09-10T16:02:27.596557+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:02:27.596557+0000","last_undegraded":"2024-09-10T16:02:27.596557+0000","last_fullsized":"2024-09-10T16:02:27.596557+0000","mapping_epoch":34,"log_start":"36'500","ondisk_log_start":"36'500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":43069440,"num_objects":10515,"num_object_clones":0,"num_object_copies":21030,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10515,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7364,"num_write_kb":29456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104219+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:02:08.104219+0000","last_peered":"2024-09-10T16:02:08.104219+0000","last_clean":"2024-09-10T16:02:08.104219+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:02:08.104219+0000","last_undegraded":"2024-09-10T16:02:08.104219+0000","last_fullsized":"2024-09-10T16:02:08.104219+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'9998","reported_seq":10137,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:24.800862+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:02:24.800862+0000","last_peered":"2024-09-10T16:02:24.800862+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:02:24.800862+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":9998,"log_dups_size":0,"ondisk_log_size":9998,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40923619,"num_objects":9992,"num_object_clones":0,"num_object_copies":19984,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9992,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7817,"num_write_kb":31268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4123},{"shards":"2","objects":5869}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":93,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.102747+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:02:08.102747+0000","last_peered":"2024-09-10T16:02:08.102747+0000","last_clean":"2024-09-10T16:02:08.102747+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:02:08.102747+0000","last_undegraded":"2024-09-10T16:02:08.102747+0000","last_fullsized":"2024-09-10T16:02:08.102747+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'10307","reported_seq":10436,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:27.651535+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:02:27.651535+0000","last_peered":"2024-09-10T16:02:27.651535+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:02:27.651535+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"36'300","ondisk_log_start":"36'300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42184704,"num_objects":10299,"num_object_clones":0,"num_object_copies":20598,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10299,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6811,"num_write_kb":27244,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6044},{"shards":"0,2","objects":4255}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":104,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.105055+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:02:08.105055+0000","last_peered":"2024-09-10T16:02:08.105055+0000","last_clean":"2024-09-10T16:02:08.105055+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:02:08.105055+0000","last_undegraded":"2024-09-10T16:02:08.105055+0000","last_fullsized":"2024-09-10T16:02:08.105055+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'10169","reported_seq":10303,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:24.699174+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:02:24.699174+0000","last_peered":"2024-09-10T16:02:24.699174+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:02:24.699174+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"36'100","ondisk_log_start":"36'100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41632227,"num_objects":10165,"num_object_clones":0,"num_object_copies":20330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10165,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7849,"num_write_kb":31396,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4304},{"shards":"2","objects":5861}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104572+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:02:08.104572+0000","last_peered":"2024-09-10T16:02:08.104572+0000","last_clean":"2024-09-10T16:02:08.104572+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:02:08.104572+0000","last_undegraded":"2024-09-10T16:02:08.104572+0000","last_fullsized":"2024-09-10T16:02:08.104572+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'10385","reported_seq":10514,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:24.796185+0000","last_change":"2024-09-10T15:59:16.174113+0000","last_active":"2024-09-10T16:02:24.796185+0000","last_peered":"2024-09-10T16:02:24.796185+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:02:24.796185+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"36'300","ondisk_log_start":"36'300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42512384,"num_objects":10379,"num_object_clones":0,"num_object_copies":20758,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4354,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10379,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7939,"num_write_kb":31756,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4352},{"shards":"2","objects":6027}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'10276","reported_seq":10407,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:27.618104+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:02:27.618104+0000","last_peered":"2024-09-10T16:02:27.618104+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:02:27.618104+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"36'200","ondisk_log_start":"36'200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42082304,"num_objects":10274,"num_object_clones":0,"num_object_copies":20548,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10274,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6907,"num_write_kb":27628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5970},{"shards":"0,2","objects":4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103276+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:02:08.103276+0000","last_peered":"2024-09-10T16:02:08.103276+0000","last_clean":"2024-09-10T16:02:08.103276+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:02:08.103276+0000","last_undegraded":"2024-09-10T16:02:08.103276+0000","last_fullsized":"2024-09-10T16:02:08.103276+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'10172","reported_seq":11062,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T16:02:27.646302+0000","last_change":"2024-09-10T16:02:06.480033+0000","last_active":"2024-09-10T16:02:27.646302+0000","last_peered":"2024-09-10T16:02:27.646302+0000","last_clean":"2024-09-10T15:59:14.115595+0000","last_became_active":"2024-09-10T15:59:16.244089+0000","last_became_peered":"2024-09-10T15:59:16.244089+0000","last_unstale":"2024-09-10T16:02:27.646302+0000","last_undegraded":"2024-09-10T16:02:27.646302+0000","last_fullsized":"2024-09-10T15:59:15.774437+0000","mapping_epoch":42,"log_start":"36'100","ondisk_log_start":"36'100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41631744,"num_objects":10164,"num_object_clones":0,"num_object_copies":20328,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3829,"num_objects_unfound":0,"num_objects_dirty":10164,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6562,"num_write_kb":26248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":374,"num_bytes_recovered":1531904,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6335},{"shards":"0,2","objects":3829}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":93,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103338+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:02:08.103338+0000","last_peered":"2024-09-10T16:02:08.103338+0000","last_clean":"2024-09-10T16:02:08.103338+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:02:08.103338+0000","last_undegraded":"2024-09-10T16:02:08.103338+0000","last_fullsized":"2024-09-10T16:02:08.103338+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'10392","reported_seq":10532,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:27.665041+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:02:27.665041+0000","last_peered":"2024-09-10T16:02:27.665041+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:02:27.665041+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"36'300","ondisk_log_start":"36'300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42549248,"num_objects":10388,"num_object_clones":0,"num_object_copies":20776,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10388,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6647,"num_write_kb":26588,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6017},{"shards":"0,2","objects":4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104431+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:02:08.104431+0000","last_peered":"2024-09-10T16:02:08.104431+0000","last_clean":"2024-09-10T16:02:08.104431+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:02:08.104431+0000","last_undegraded":"2024-09-10T16:02:08.104431+0000","last_fullsized":"2024-09-10T16:02:08.104431+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104410+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:02:08.104410+0000","last_peered":"2024-09-10T16:02:08.104410+0000","last_clean":"2024-09-10T16:02:08.104410+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:02:08.104410+0000","last_undegraded":"2024-09-10T16:02:08.104410+0000","last_fullsized":"2024-09-10T16:02:08.104410+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'10465","reported_seq":10609,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:27.598239+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:02:27.598239+0000","last_peered":"2024-09-10T16:02:27.598239+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:02:27.598239+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"36'400","ondisk_log_start":"36'400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42816454,"num_objects":10455,"num_object_clones":0,"num_object_copies":20910,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10455,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6628,"num_write_kb":26512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6132},{"shards":"0,2","objects":4323}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":673444574,"num_objects":164424,"num_object_clones":0,"num_object_copies":328848,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42911,"num_objects_misplaced":3829,"num_objects_unfound":0,"num_objects_dirty":164424,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":122270,"num_write_kb":489080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4646,"num_bytes_recovered":19025920,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1363931136,"data_stored":1363779390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160796,"ondisk_log_size":160796,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986229,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":752316,"kb_used_data":463788,"kb_used_omap":4,"kb_used_meta":288507,"kb_avail":93619524,"statfs":{"total":96636764160,"available":95866392576,"internally_reserved":0,"allocated":474918912,"data_stored":474630751,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":295431850},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,2,5,2,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659175,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":524000,"kb_used_data":303504,"kb_used_omap":4,"kb_used_meta":220475,"kb_avail":93847840,"statfs":{"total":96636764160,"available":96100188160,"internally_reserved":0,"allocated":310788096,"data_stored":310452502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":225767086},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,4,1,1,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182458,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":897516,"kb_used_data":563228,"kb_used_omap":4,"kb_used_meta":334267,"kb_avail":93474324,"statfs":{"total":96636764160,"available":95717707776,"internally_reserved":0,"allocated":576745472,"data_stored":576453234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":342290090},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,1,1,7,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":577069056,"data_stored":577032926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":311590912,"data_stored":311511426,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":475271168,"data_stored":475235038,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:02:31.612 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:02:31.998 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout: 239 20 166737 166717 2.72442 1.57422 0.0732633 0.0286688 2024-09-10T16:02:32.174 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:02:32.180 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:02:32.205 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":196,"stamp":"2024-09-10T16:02:31.614799+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":676807409,"num_objects":165247,"num_object_clones":0,"num_object_copies":330494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42911,"num_objects_misplaced":3829,"num_objects_unfound":0,"num_objects_dirty":165247,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":123093,"num_write_kb":492366,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4648,"num_bytes_recovered":19025939,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160620,"ondisk_log_size":160620,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2176404,"kb_used_data":1341412,"kb_used_omap":12,"kb_used_meta":834931,"kb_avail":280939116,"statfs":{"total":289910292480,"available":287681654784,"internally_reserved":0,"allocated":1373605888,"data_stored":1372689895,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":854969346},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,5,9,9,10,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":20992000,"num_objects":5125,"num_object_clones":0,"num_object_copies":10250,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-173,"num_objects_unfound":0,"num_objects_dirty":5125,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5125,"num_write_kb":20500,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":173,"num_bytes_recovered":708608,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004898"},"pg_stats":[{"pgid":"2.f","version":"45'10202","reported_seq":10276,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:27.606446+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:02:27.606446+0000","last_peered":"2024-09-10T16:02:27.606446+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:02:27.606446+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"36'200","ondisk_log_start":"36'200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41787392,"num_objects":10202,"num_object_clones":0,"num_object_copies":20404,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10202,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10202,"num_write_kb":40808,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5943},{"shards":"0,2","objects":4258}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'10338","reported_seq":10450,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:29.832059+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:02:29.832059+0000","last_peered":"2024-09-10T16:02:29.832059+0000","last_clean":"2024-09-10T16:02:29.832059+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:02:29.832059+0000","last_undegraded":"2024-09-10T16:02:29.832059+0000","last_fullsized":"2024-09-10T16:02:29.832059+0000","mapping_epoch":35,"log_start":"36'300","ondisk_log_start":"36'300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42320838,"num_objects":10334,"num_object_clones":0,"num_object_copies":20668,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10334,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":8443,"num_write_kb":33772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'10331","reported_seq":19008,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:30.210898+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:02:30.210898+0000","last_peered":"2024-09-10T16:02:30.210898+0000","last_clean":"2024-09-10T16:02:30.210898+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:02:30.210898+0000","last_undegraded":"2024-09-10T16:02:30.210898+0000","last_fullsized":"2024-09-10T16:02:30.210898+0000","mapping_epoch":44,"log_start":"36'300","ondisk_log_start":"36'300","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42299392,"num_objects":10327,"num_object_clones":0,"num_object_copies":20654,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10327,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8335,"num_write_kb":33340,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'10247","reported_seq":10327,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:29.806063+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:02:29.806063+0000","last_peered":"2024-09-10T16:02:29.806063+0000","last_clean":"2024-09-10T16:02:29.806063+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:02:29.806063+0000","last_undegraded":"2024-09-10T16:02:29.806063+0000","last_fullsized":"2024-09-10T16:02:29.806063+0000","mapping_epoch":35,"log_start":"36'200","ondisk_log_start":"36'200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41971712,"num_objects":10247,"num_object_clones":0,"num_object_copies":20494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10247,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10247,"num_write_kb":40988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'10302","reported_seq":10432,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:27.613372+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:02:27.613372+0000","last_peered":"2024-09-10T16:02:27.613372+0000","last_clean":"2024-09-10T16:02:27.613372+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:02:27.613372+0000","last_undegraded":"2024-09-10T16:02:27.613372+0000","last_fullsized":"2024-09-10T16:02:27.613372+0000","mapping_epoch":34,"log_start":"36'300","ondisk_log_start":"36'300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42168803,"num_objects":10296,"num_object_clones":0,"num_object_copies":20592,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10296,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7013,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'10384","reported_seq":10521,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:27.642161+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:02:27.642161+0000","last_peered":"2024-09-10T16:02:27.642161+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:02:27.642161+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"36'300","ondisk_log_start":"36'300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42512867,"num_objects":10380,"num_object_clones":0,"num_object_copies":20760,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10380,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6996,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6009},{"shards":"0,2","objects":4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'10416","reported_seq":10553,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:27.475138+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:02:27.475138+0000","last_peered":"2024-09-10T16:02:27.475138+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:02:27.475138+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"36'400","ondisk_log_start":"36'400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42640326,"num_objects":10412,"num_object_clones":0,"num_object_copies":20824,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10412,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":6915,"num_write_kb":27660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6175},{"shards":"0,2","objects":4237}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'10517","reported_seq":10647,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:27.596557+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:02:27.596557+0000","last_peered":"2024-09-10T16:02:27.596557+0000","last_clean":"2024-09-10T16:02:27.596557+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:02:27.596557+0000","last_undegraded":"2024-09-10T16:02:27.596557+0000","last_fullsized":"2024-09-10T16:02:27.596557+0000","mapping_epoch":34,"log_start":"36'500","ondisk_log_start":"36'500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":43069440,"num_objects":10515,"num_object_clones":0,"num_object_copies":21030,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10515,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7364,"num_write_kb":29456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104219+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:02:08.104219+0000","last_peered":"2024-09-10T16:02:08.104219+0000","last_clean":"2024-09-10T16:02:08.104219+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:02:08.104219+0000","last_undegraded":"2024-09-10T16:02:08.104219+0000","last_fullsized":"2024-09-10T16:02:08.104219+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'10155","reported_seq":10294,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:29.770957+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:02:29.770957+0000","last_peered":"2024-09-10T16:02:29.770957+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:02:29.770957+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"36'100","ondisk_log_start":"36'100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41566691,"num_objects":10149,"num_object_clones":0,"num_object_copies":20298,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10149,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7974,"num_write_kb":31896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4123},{"shards":"2","objects":6026}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103774+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:02:08.103774+0000","last_peered":"2024-09-10T16:02:08.103774+0000","last_clean":"2024-09-10T16:02:08.103774+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:02:08.103774+0000","last_undegraded":"2024-09-10T16:02:08.103774+0000","last_fullsized":"2024-09-10T16:02:08.103774+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'10307","reported_seq":10436,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:27.651535+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:02:27.651535+0000","last_peered":"2024-09-10T16:02:27.651535+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:02:27.651535+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"36'300","ondisk_log_start":"36'300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42184704,"num_objects":10299,"num_object_clones":0,"num_object_copies":20598,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10299,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6811,"num_write_kb":27244,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6044},{"shards":"0,2","objects":4255}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":104,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.105055+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:02:08.105055+0000","last_peered":"2024-09-10T16:02:08.105055+0000","last_clean":"2024-09-10T16:02:08.105055+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:02:08.105055+0000","last_undegraded":"2024-09-10T16:02:08.105055+0000","last_fullsized":"2024-09-10T16:02:08.105055+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'10305","reported_seq":10440,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:29.830952+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:02:29.830952+0000","last_peered":"2024-09-10T16:02:29.830952+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:02:29.830952+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"36'300","ondisk_log_start":"36'300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42189283,"num_objects":10301,"num_object_clones":0,"num_object_copies":20602,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10301,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7985,"num_write_kb":31940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4304},{"shards":"2","objects":5997}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104572+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:02:08.104572+0000","last_peered":"2024-09-10T16:02:08.104572+0000","last_clean":"2024-09-10T16:02:08.104572+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:02:08.104572+0000","last_undegraded":"2024-09-10T16:02:08.104572+0000","last_fullsized":"2024-09-10T16:02:08.104572+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'10508","reported_seq":10637,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:29.607073+0000","last_change":"2024-09-10T15:59:16.174113+0000","last_active":"2024-09-10T16:02:29.607073+0000","last_peered":"2024-09-10T16:02:29.607073+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:02:29.607073+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"36'500","ondisk_log_start":"36'500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":43016192,"num_objects":10502,"num_object_clones":0,"num_object_copies":21004,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4354,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10502,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8062,"num_write_kb":32248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4352},{"shards":"2","objects":6150}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'10276","reported_seq":10407,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:27.618104+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:02:27.618104+0000","last_peered":"2024-09-10T16:02:27.618104+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:02:27.618104+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"36'200","ondisk_log_start":"36'200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42082304,"num_objects":10274,"num_object_clones":0,"num_object_copies":20548,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10274,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6907,"num_write_kb":27628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5970},{"shards":"0,2","objects":4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103276+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:02:08.103276+0000","last_peered":"2024-09-10T16:02:08.103276+0000","last_clean":"2024-09-10T16:02:08.103276+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:02:08.103276+0000","last_undegraded":"2024-09-10T16:02:08.103276+0000","last_fullsized":"2024-09-10T16:02:08.103276+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'10172","reported_seq":11062,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T16:02:27.646302+0000","last_change":"2024-09-10T16:02:06.480033+0000","last_active":"2024-09-10T16:02:27.646302+0000","last_peered":"2024-09-10T16:02:27.646302+0000","last_clean":"2024-09-10T15:59:14.115595+0000","last_became_active":"2024-09-10T15:59:16.244089+0000","last_became_peered":"2024-09-10T15:59:16.244089+0000","last_unstale":"2024-09-10T16:02:27.646302+0000","last_undegraded":"2024-09-10T16:02:27.646302+0000","last_fullsized":"2024-09-10T15:59:15.774437+0000","mapping_epoch":42,"log_start":"36'100","ondisk_log_start":"36'100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41631744,"num_objects":10164,"num_object_clones":0,"num_object_copies":20328,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3829,"num_objects_unfound":0,"num_objects_dirty":10164,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6562,"num_write_kb":26248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":374,"num_bytes_recovered":1531904,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6335},{"shards":"0,2","objects":3829}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103831+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:02:08.103831+0000","last_peered":"2024-09-10T16:02:08.103831+0000","last_clean":"2024-09-10T16:02:08.103831+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:02:08.103831+0000","last_undegraded":"2024-09-10T16:02:08.103831+0000","last_fullsized":"2024-09-10T16:02:08.103831+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'10392","reported_seq":10532,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:27.665041+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:02:27.665041+0000","last_peered":"2024-09-10T16:02:27.665041+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:02:27.665041+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"36'300","ondisk_log_start":"36'300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42549248,"num_objects":10388,"num_object_clones":0,"num_object_copies":20776,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10388,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6647,"num_write_kb":26588,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6017},{"shards":"0,2","objects":4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104431+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:02:08.104431+0000","last_peered":"2024-09-10T16:02:08.104431+0000","last_clean":"2024-09-10T16:02:08.104431+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:02:08.104431+0000","last_undegraded":"2024-09-10T16:02:08.104431+0000","last_fullsized":"2024-09-10T16:02:08.104431+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104410+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:02:08.104410+0000","last_peered":"2024-09-10T16:02:08.104410+0000","last_clean":"2024-09-10T16:02:08.104410+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:02:08.104410+0000","last_undegraded":"2024-09-10T16:02:08.104410+0000","last_fullsized":"2024-09-10T16:02:08.104410+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'10465","reported_seq":10609,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:27.598239+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:02:27.598239+0000","last_peered":"2024-09-10T16:02:27.598239+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:02:27.598239+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"36'400","ondisk_log_start":"36'400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42816454,"num_objects":10455,"num_object_clones":0,"num_object_copies":20910,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10455,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6628,"num_write_kb":26512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6132},{"shards":"0,2","objects":4323}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":676807390,"num_objects":165245,"num_object_clones":0,"num_object_copies":330490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42911,"num_objects_misplaced":3829,"num_objects_unfound":0,"num_objects_dirty":165245,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":123091,"num_write_kb":492364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4646,"num_bytes_recovered":19025920,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1375338496,"data_stored":1375186750,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160617,"ondisk_log_size":160617,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986230,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":760612,"kb_used_data":467924,"kb_used_omap":4,"kb_used_meta":292667,"kb_avail":93611228,"statfs":{"total":96636764160,"available":95857897472,"internally_reserved":0,"allocated":479154176,"data_stored":478866015,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":299691690},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,3,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659176,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518276,"kb_used_data":310260,"kb_used_omap":4,"kb_used_meta":207995,"kb_avail":93853564,"statfs":{"total":96636764160,"available":96106049536,"internally_reserved":0,"allocated":317706240,"data_stored":317370646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":212987566},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,5,7,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182458,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":897516,"kb_used_data":563228,"kb_used_omap":4,"kb_used_meta":334267,"kb_avail":93474324,"statfs":{"total":96636764160,"available":95717707776,"internally_reserved":0,"allocated":576745472,"data_stored":576453234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":342290090},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,1,1,7,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":577069056,"data_stored":577032926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":318599168,"data_stored":318519682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":479670272,"data_stored":479634142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:02:32.207 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-10T16:02:32.207 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "45'10202", 'reported_seq': 10276, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:02:27.606446+0000', 'last_change': '2024-09-10T15:59:16.281558+0000', 'last_active': '2024-09-10T16:02:27.606446+0000', 'last_peered': '2024-09-10T16:02:27.606446+0000', 'last_clean': '2024-09-10T15:59:14.132456+0000', 'last_became_active': '2024-09-10T15:59:16.274407+0000', 'last_became_peered': '2024-09-10T15:59:16.274407+0000', 'last_unstale': '2024-09-10T16:02:27.606446+0000', 'last_undegraded': '2024-09-10T15:59:15.851279+0000', 'last_fullsized': '2024-09-10T15:59:15.774356+0000', 'mapping_epoch': 42, 'log_start': "36'200", 'ondisk_log_start': "36'200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'objects_scrubbed': 0, 'log_size': 10002, 'log_dups_size': 0, 'ondisk_log_size': 10002, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 41787392, 'num_objects': 10202, 'num_object_clones': 0, 'num_object_copies': 20404, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4259, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10202, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10202, 'num_write_kb': 40808, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5943}, {'shards': '0,2', 'objects': 4258}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:02:32.207 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-10T16:02:32.207 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "45'10384", 'reported_seq': 10521, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:02:27.642161+0000', 'last_change': '2024-09-10T15:59:16.308264+0000', 'last_active': '2024-09-10T16:02:27.642161+0000', 'last_peered': '2024-09-10T16:02:27.642161+0000', 'last_clean': '2024-09-10T15:59:14.111551+0000', 'last_became_active': '2024-09-10T15:59:16.300040+0000', 'last_became_peered': '2024-09-10T15:59:16.300040+0000', 'last_unstale': '2024-09-10T16:02:27.642161+0000', 'last_undegraded': '2024-09-10T15:59:15.878234+0000', 'last_fullsized': '2024-09-10T15:59:15.769328+0000', 'mapping_epoch': 42, 'log_start': "36'300", 'ondisk_log_start': "36'300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3442", 'last_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'objects_scrubbed': 3384, 'log_size': 10084, 'log_dups_size': 0, 'ondisk_log_size': 10084, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1292, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 42512867, 'num_objects': 10380, 'num_object_clones': 0, 'num_object_copies': 20760, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4372, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10380, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 6996, 'num_write_kb': 27984, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6009}, {'shards': '0,2', 'objects': 4371}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:02:32.207 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-10T16:02:32.207 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "45'10416", 'reported_seq': 10553, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:02:27.475138+0000', 'last_change': '2024-09-10T15:59:16.331899+0000', 'last_active': '2024-09-10T16:02:27.475138+0000', 'last_peered': '2024-09-10T16:02:27.475138+0000', 'last_clean': '2024-09-10T15:59:14.139436+0000', 'last_became_active': '2024-09-10T15:59:16.318715+0000', 'last_became_peered': '2024-09-10T15:59:16.318715+0000', 'last_unstale': '2024-09-10T16:02:27.475138+0000', 'last_undegraded': '2024-09-10T15:59:15.877524+0000', 'last_fullsized': '2024-09-10T15:59:15.769042+0000', 'mapping_epoch': 42, 'log_start': "36'400", 'ondisk_log_start': "36'400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3537", 'last_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'objects_scrubbed': 3497, 'log_size': 10016, 'log_dups_size': 0, 'ondisk_log_size': 10016, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1224, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 42640326, 'num_objects': 10412, 'num_object_clones': 0, 'num_object_copies': 20824, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4237, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10412, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 6915, 'num_write_kb': 27660, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6175}, {'shards': '0,2', 'objects': 4237}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:02:32.207 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-10T16:02:32.207 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "45'10155", 'reported_seq': 10294, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:02:29.770957+0000', 'last_change': '2024-09-10T15:59:16.175680+0000', 'last_active': '2024-09-10T16:02:29.770957+0000', 'last_peered': '2024-09-10T16:02:29.770957+0000', 'last_clean': '2024-09-10T15:59:14.088187+0000', 'last_became_active': '2024-09-10T15:59:16.165810+0000', 'last_became_peered': '2024-09-10T15:59:16.165810+0000', 'last_unstale': '2024-09-10T16:02:29.770957+0000', 'last_undegraded': '2024-09-10T15:59:15.822098+0000', 'last_fullsized': '2024-09-10T15:59:15.769123+0000', 'mapping_epoch': 42, 'log_start': "36'100", 'ondisk_log_start': "36'100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2218", 'last_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'objects_scrubbed': 2175, 'log_size': 10055, 'log_dups_size': 0, 'ondisk_log_size': 10055, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 825, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 41566691, 'num_objects': 10149, 'num_object_clones': 0, 'num_object_copies': 20298, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4124, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10149, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 7974, 'num_write_kb': 31896, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4123}, {'shards': '2', 'objects': 6026}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:02:32.207 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-10T16:02:32.207 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "45'10307", 'reported_seq': 10436, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:02:27.651535+0000', 'last_change': '2024-09-10T15:59:16.277742+0000', 'last_active': '2024-09-10T16:02:27.651535+0000', 'last_peered': '2024-09-10T16:02:27.651535+0000', 'last_clean': '2024-09-10T15:59:14.129081+0000', 'last_became_active': '2024-09-10T15:59:16.269835+0000', 'last_became_peered': '2024-09-10T15:59:16.269835+0000', 'last_unstale': '2024-09-10T16:02:27.651535+0000', 'last_undegraded': '2024-09-10T15:59:15.860857+0000', 'last_fullsized': '2024-09-10T15:59:15.771410+0000', 'mapping_epoch': 42, 'log_start': "36'300", 'ondisk_log_start': "36'300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3557", 'last_scrub_stamp': '2024-09-10T15:59:02.493678+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:02.493678+0000', 'objects_scrubbed': 3488, 'log_size': 10007, 'log_dups_size': 0, 'ondisk_log_size': 10007, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1472, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 42184704, 'num_objects': 10299, 'num_object_clones': 0, 'num_object_copies': 20598, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4257, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10299, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6811, 'num_write_kb': 27244, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6044}, {'shards': '0,2', 'objects': 4255}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:02:32.208 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-10T16:02:32.208 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "45'10305", 'reported_seq': 10440, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:02:29.830952+0000', 'last_change': '2024-09-10T15:59:16.284311+0000', 'last_active': '2024-09-10T16:02:29.830952+0000', 'last_peered': '2024-09-10T16:02:29.830952+0000', 'last_clean': '2024-09-10T15:59:14.116041+0000', 'last_became_active': '2024-09-10T15:59:16.274543+0000', 'last_became_peered': '2024-09-10T15:59:16.274543+0000', 'last_unstale': '2024-09-10T16:02:29.830952+0000', 'last_undegraded': '2024-09-10T15:59:15.812484+0000', 'last_fullsized': '2024-09-10T15:59:15.770885+0000', 'mapping_epoch': 42, 'log_start': "36'300", 'ondisk_log_start': "36'300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2356", 'last_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'objects_scrubbed': 2316, 'log_size': 10005, 'log_dups_size': 0, 'ondisk_log_size': 10005, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 783, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 42189283, 'num_objects': 10301, 'num_object_clones': 0, 'num_object_copies': 20602, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10301, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 7985, 'num_write_kb': 31940, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4304}, {'shards': '2', 'objects': 5997}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:02:32.208 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-10T16:02:32.208 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "45'10508", 'reported_seq': 10637, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:02:29.607073+0000', 'last_change': '2024-09-10T15:59:16.174113+0000', 'last_active': '2024-09-10T16:02:29.607073+0000', 'last_peered': '2024-09-10T16:02:29.607073+0000', 'last_clean': '2024-09-10T15:59:14.110654+0000', 'last_became_active': '2024-09-10T15:59:16.164645+0000', 'last_became_peered': '2024-09-10T15:59:16.164645+0000', 'last_unstale': '2024-09-10T16:02:29.607073+0000', 'last_undegraded': '2024-09-10T15:59:15.803593+0000', 'last_fullsized': '2024-09-10T15:59:15.769083+0000', 'mapping_epoch': 42, 'log_start': "36'500", 'ondisk_log_start': "36'500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2486", 'last_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'objects_scrubbed': 2440, 'log_size': 10008, 'log_dups_size': 0, 'ondisk_log_size': 10008, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 792, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 43016192, 'num_objects': 10502, 'num_object_clones': 0, 'num_object_copies': 21004, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4354, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10502, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8062, 'num_write_kb': 32248, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4352}, {'shards': '2', 'objects': 6150}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:02:32.208 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-10T16:02:32.208 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "45'10276", 'reported_seq': 10407, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:02:27.618104+0000', 'last_change': '2024-09-10T15:59:16.315641+0000', 'last_active': '2024-09-10T16:02:27.618104+0000', 'last_peered': '2024-09-10T16:02:27.618104+0000', 'last_clean': '2024-09-10T15:59:14.117001+0000', 'last_became_active': '2024-09-10T15:59:16.302245+0000', 'last_became_peered': '2024-09-10T15:59:16.302245+0000', 'last_unstale': '2024-09-10T16:02:27.618104+0000', 'last_undegraded': '2024-09-10T15:59:15.845012+0000', 'last_fullsized': '2024-09-10T15:59:15.770337+0000', 'mapping_epoch': 42, 'log_start': "36'200", 'ondisk_log_start': "36'200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3404", 'last_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'objects_scrubbed': 3367, 'log_size': 10076, 'log_dups_size': 0, 'ondisk_log_size': 10076, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 983, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 42082304, 'num_objects': 10274, 'num_object_clones': 0, 'num_object_copies': 20548, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10274, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6907, 'num_write_kb': 27628, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5970}, {'shards': '0,2', 'objects': 4304}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:02:32.208 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-10T16:02:32.208 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "45'10172", 'reported_seq': 11062, 'reported_epoch': 45, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-10T16:02:27.646302+0000', 'last_change': '2024-09-10T16:02:06.480033+0000', 'last_active': '2024-09-10T16:02:27.646302+0000', 'last_peered': '2024-09-10T16:02:27.646302+0000', 'last_clean': '2024-09-10T15:59:14.115595+0000', 'last_became_active': '2024-09-10T15:59:16.244089+0000', 'last_became_peered': '2024-09-10T15:59:16.244089+0000', 'last_unstale': '2024-09-10T16:02:27.646302+0000', 'last_undegraded': '2024-09-10T16:02:27.646302+0000', 'last_fullsized': '2024-09-10T15:59:15.774437+0000', 'mapping_epoch': 42, 'log_start': "36'100", 'ondisk_log_start': "36'100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3653", 'last_scrub_stamp': '2024-09-10T15:59:05.298468+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:05.298468+0000', 'objects_scrubbed': 3602, 'log_size': 10072, 'log_dups_size': 0, 'ondisk_log_size': 10072, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1289, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 41631744, 'num_objects': 10164, 'num_object_clones': 0, 'num_object_copies': 20328, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 3829, 'num_objects_unfound': 0, 'num_objects_dirty': 10164, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6562, 'num_write_kb': 26248, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 374, 'num_bytes_recovered': 1531904, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6335}, {'shards': '0,2', 'objects': 3829}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:02:32.208 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-10T16:02:32.209 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "45'10392", 'reported_seq': 10532, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:02:27.665041+0000', 'last_change': '2024-09-10T15:59:16.311291+0000', 'last_active': '2024-09-10T16:02:27.665041+0000', 'last_peered': '2024-09-10T16:02:27.665041+0000', 'last_clean': '2024-09-10T15:59:14.116363+0000', 'last_became_active': '2024-09-10T15:59:16.301244+0000', 'last_became_peered': '2024-09-10T15:59:16.301244+0000', 'last_unstale': '2024-09-10T16:02:27.665041+0000', 'last_undegraded': '2024-09-10T15:59:15.883434+0000', 'last_fullsized': '2024-09-10T15:59:15.772756+0000', 'mapping_epoch': 42, 'log_start': "36'300", 'ondisk_log_start': "36'300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3789", 'last_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'objects_scrubbed': 3741, 'log_size': 10092, 'log_dups_size': 0, 'ondisk_log_size': 10092, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1440, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 42549248, 'num_objects': 10388, 'num_object_clones': 0, 'num_object_copies': 20776, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4373, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10388, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6647, 'num_write_kb': 26588, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6017}, {'shards': '0,2', 'objects': 4371}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:02:32.209 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-10T16:02:32.209 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "45'10465", 'reported_seq': 10609, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:02:27.598239+0000', 'last_change': '2024-09-10T15:59:16.313409+0000', 'last_active': '2024-09-10T16:02:27.598239+0000', 'last_peered': '2024-09-10T16:02:27.598239+0000', 'last_clean': '2024-09-10T15:59:14.129780+0000', 'last_became_active': '2024-09-10T15:59:16.300932+0000', 'last_became_peered': '2024-09-10T15:59:16.300932+0000', 'last_unstale': '2024-09-10T16:02:27.598239+0000', 'last_undegraded': '2024-09-10T15:59:15.887823+0000', 'last_fullsized': '2024-09-10T15:59:15.768513+0000', 'mapping_epoch': 42, 'log_start': "36'400", 'ondisk_log_start': "36'400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3880", 'last_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'objects_scrubbed': 3827, 'log_size': 10065, 'log_dups_size': 0, 'ondisk_log_size': 10065, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1397, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 42816454, 'num_objects': 10455, 'num_object_clones': 0, 'num_object_copies': 20910, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4325, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10455, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6628, 'num_write_kb': 26512, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6132}, {'shards': '0,2', 'objects': 4323}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:02:32.209 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T16:02:32.917 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:02:32.917 INFO:teuthology.orchestra.run.smithi115.stdout:{"fsid":"7effee29-df05-4351-9741-ee203564ed81","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 42911/330494 objects degraded (12.984%), 10 pgs degraded, 11 pgs undersized","count":21},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":316,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983954,"num_in_osds":3,"osd_in_since":1725983947,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":165247,"data_bytes":676807409,"bytes_used":2228637696,"bytes_avail":287681654784,"bytes_total":289910292480,"degraded_objects":42911,"degraded_total":330494,"degraded_ratio":0.12983896833225414,"misplaced_objects":3829,"misplaced_total":330494,"misplaced_ratio":0.01158568688085109,"recovering_objects_per_sec":14,"recovering_bytes_per_sec":59026,"recovering_keys_per_sec":0,"num_objects_recovered":173,"num_bytes_recovered":708608,"num_keys_recovered":0,"write_bytes_sec":1748619,"read_op_per_sec":0,"write_op_per_sec":426},"fsmap":{"epoch":1,"btime":"2024-09-10T15:57:14:967120+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T16:00:13.560776+0000","services":{}},"progress_events":{}} 2024-09-10T16:02:32.939 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-10T16:02:32.939 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:02:32.998 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout:Total time run: 240.022 2024-09-10T16:02:32.998 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout:Total writes made: 167119 2024-09-10T16:02:32.998 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout:Write size: 4096 2024-09-10T16:02:32.998 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout:Object size: 4096 2024-09-10T16:02:32.998 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout:Bandwidth (MB/sec): 2.71978 2024-09-10T16:02:32.998 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout:Stddev Bandwidth: 2.0815 2024-09-10T16:02:32.999 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout:Max bandwidth (MB/sec): 11.6836 2024-09-10T16:02:32.999 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout:Min bandwidth (MB/sec): 0 2024-09-10T16:02:32.999 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout:Average IOPS: 696 2024-09-10T16:02:32.999 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout:Stddev IOPS: 532.866 2024-09-10T16:02:32.999 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout:Max IOPS: 2991 2024-09-10T16:02:32.999 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout:Min IOPS: 0 2024-09-10T16:02:32.999 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout:Average Latency(s): 0.0287198 2024-09-10T16:02:32.999 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout:Stddev Latency(s): 0.0397627 2024-09-10T16:02:32.999 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout:Max latency(s): 2.21736 2024-09-10T16:02:32.999 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout:Min latency(s): 0.00225581 2024-09-10T16:02:33.006 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout:Cleaning up (deleting benchmark objects) 2024-09-10T16:02:33.485 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:02:33.486 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:02:33.521 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":196,"stamp":"2024-09-10T16:02:31.614799+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":676807409,"num_objects":165247,"num_object_clones":0,"num_object_copies":330494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42911,"num_objects_misplaced":3829,"num_objects_unfound":0,"num_objects_dirty":165247,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":123093,"num_write_kb":492366,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4648,"num_bytes_recovered":19025939,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160620,"ondisk_log_size":160620,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2176404,"kb_used_data":1341412,"kb_used_omap":12,"kb_used_meta":834931,"kb_avail":280939116,"statfs":{"total":289910292480,"available":287681654784,"internally_reserved":0,"allocated":1373605888,"data_stored":1372689895,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":854969346},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,5,9,9,10,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":20992000,"num_objects":5125,"num_object_clones":0,"num_object_copies":10250,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-173,"num_objects_unfound":0,"num_objects_dirty":5125,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5125,"num_write_kb":20500,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":173,"num_bytes_recovered":708608,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004898"},"pg_stats":[{"pgid":"2.f","version":"45'10202","reported_seq":10276,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:27.606446+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:02:27.606446+0000","last_peered":"2024-09-10T16:02:27.606446+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:02:27.606446+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"36'200","ondisk_log_start":"36'200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41787392,"num_objects":10202,"num_object_clones":0,"num_object_copies":20404,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10202,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10202,"num_write_kb":40808,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5943},{"shards":"0,2","objects":4258}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'10338","reported_seq":10450,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:29.832059+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:02:29.832059+0000","last_peered":"2024-09-10T16:02:29.832059+0000","last_clean":"2024-09-10T16:02:29.832059+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:02:29.832059+0000","last_undegraded":"2024-09-10T16:02:29.832059+0000","last_fullsized":"2024-09-10T16:02:29.832059+0000","mapping_epoch":35,"log_start":"36'300","ondisk_log_start":"36'300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42320838,"num_objects":10334,"num_object_clones":0,"num_object_copies":20668,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10334,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":8443,"num_write_kb":33772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'10331","reported_seq":19008,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:30.210898+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:02:30.210898+0000","last_peered":"2024-09-10T16:02:30.210898+0000","last_clean":"2024-09-10T16:02:30.210898+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:02:30.210898+0000","last_undegraded":"2024-09-10T16:02:30.210898+0000","last_fullsized":"2024-09-10T16:02:30.210898+0000","mapping_epoch":44,"log_start":"36'300","ondisk_log_start":"36'300","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42299392,"num_objects":10327,"num_object_clones":0,"num_object_copies":20654,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10327,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8335,"num_write_kb":33340,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'10247","reported_seq":10327,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:29.806063+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:02:29.806063+0000","last_peered":"2024-09-10T16:02:29.806063+0000","last_clean":"2024-09-10T16:02:29.806063+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:02:29.806063+0000","last_undegraded":"2024-09-10T16:02:29.806063+0000","last_fullsized":"2024-09-10T16:02:29.806063+0000","mapping_epoch":35,"log_start":"36'200","ondisk_log_start":"36'200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41971712,"num_objects":10247,"num_object_clones":0,"num_object_copies":20494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10247,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10247,"num_write_kb":40988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'10302","reported_seq":10432,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:27.613372+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:02:27.613372+0000","last_peered":"2024-09-10T16:02:27.613372+0000","last_clean":"2024-09-10T16:02:27.613372+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:02:27.613372+0000","last_undegraded":"2024-09-10T16:02:27.613372+0000","last_fullsized":"2024-09-10T16:02:27.613372+0000","mapping_epoch":34,"log_start":"36'300","ondisk_log_start":"36'300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42168803,"num_objects":10296,"num_object_clones":0,"num_object_copies":20592,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10296,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7013,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'10384","reported_seq":10521,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:27.642161+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:02:27.642161+0000","last_peered":"2024-09-10T16:02:27.642161+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:02:27.642161+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"36'300","ondisk_log_start":"36'300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42512867,"num_objects":10380,"num_object_clones":0,"num_object_copies":20760,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10380,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6996,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6009},{"shards":"0,2","objects":4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'10416","reported_seq":10553,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:27.475138+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:02:27.475138+0000","last_peered":"2024-09-10T16:02:27.475138+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:02:27.475138+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"36'400","ondisk_log_start":"36'400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42640326,"num_objects":10412,"num_object_clones":0,"num_object_copies":20824,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10412,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":6915,"num_write_kb":27660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6175},{"shards":"0,2","objects":4237}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'10517","reported_seq":10647,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:27.596557+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:02:27.596557+0000","last_peered":"2024-09-10T16:02:27.596557+0000","last_clean":"2024-09-10T16:02:27.596557+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:02:27.596557+0000","last_undegraded":"2024-09-10T16:02:27.596557+0000","last_fullsized":"2024-09-10T16:02:27.596557+0000","mapping_epoch":34,"log_start":"36'500","ondisk_log_start":"36'500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":43069440,"num_objects":10515,"num_object_clones":0,"num_object_copies":21030,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10515,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7364,"num_write_kb":29456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104219+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:02:08.104219+0000","last_peered":"2024-09-10T16:02:08.104219+0000","last_clean":"2024-09-10T16:02:08.104219+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:02:08.104219+0000","last_undegraded":"2024-09-10T16:02:08.104219+0000","last_fullsized":"2024-09-10T16:02:08.104219+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'10155","reported_seq":10294,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:29.770957+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:02:29.770957+0000","last_peered":"2024-09-10T16:02:29.770957+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:02:29.770957+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"36'100","ondisk_log_start":"36'100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41566691,"num_objects":10149,"num_object_clones":0,"num_object_copies":20298,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10149,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7974,"num_write_kb":31896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4123},{"shards":"2","objects":6026}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103774+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:02:08.103774+0000","last_peered":"2024-09-10T16:02:08.103774+0000","last_clean":"2024-09-10T16:02:08.103774+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:02:08.103774+0000","last_undegraded":"2024-09-10T16:02:08.103774+0000","last_fullsized":"2024-09-10T16:02:08.103774+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'10307","reported_seq":10436,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:27.651535+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:02:27.651535+0000","last_peered":"2024-09-10T16:02:27.651535+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:02:27.651535+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"36'300","ondisk_log_start":"36'300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42184704,"num_objects":10299,"num_object_clones":0,"num_object_copies":20598,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10299,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6811,"num_write_kb":27244,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6044},{"shards":"0,2","objects":4255}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":104,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.105055+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:02:08.105055+0000","last_peered":"2024-09-10T16:02:08.105055+0000","last_clean":"2024-09-10T16:02:08.105055+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:02:08.105055+0000","last_undegraded":"2024-09-10T16:02:08.105055+0000","last_fullsized":"2024-09-10T16:02:08.105055+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'10305","reported_seq":10440,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:29.830952+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:02:29.830952+0000","last_peered":"2024-09-10T16:02:29.830952+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:02:29.830952+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"36'300","ondisk_log_start":"36'300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42189283,"num_objects":10301,"num_object_clones":0,"num_object_copies":20602,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10301,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7985,"num_write_kb":31940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4304},{"shards":"2","objects":5997}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104572+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:02:08.104572+0000","last_peered":"2024-09-10T16:02:08.104572+0000","last_clean":"2024-09-10T16:02:08.104572+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:02:08.104572+0000","last_undegraded":"2024-09-10T16:02:08.104572+0000","last_fullsized":"2024-09-10T16:02:08.104572+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'10508","reported_seq":10637,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:29.607073+0000","last_change":"2024-09-10T15:59:16.174113+0000","last_active":"2024-09-10T16:02:29.607073+0000","last_peered":"2024-09-10T16:02:29.607073+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:02:29.607073+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"36'500","ondisk_log_start":"36'500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":43016192,"num_objects":10502,"num_object_clones":0,"num_object_copies":21004,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4354,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10502,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8062,"num_write_kb":32248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4352},{"shards":"2","objects":6150}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'10276","reported_seq":10407,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:27.618104+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:02:27.618104+0000","last_peered":"2024-09-10T16:02:27.618104+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:02:27.618104+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"36'200","ondisk_log_start":"36'200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42082304,"num_objects":10274,"num_object_clones":0,"num_object_copies":20548,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10274,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6907,"num_write_kb":27628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5970},{"shards":"0,2","objects":4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103276+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:02:08.103276+0000","last_peered":"2024-09-10T16:02:08.103276+0000","last_clean":"2024-09-10T16:02:08.103276+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:02:08.103276+0000","last_undegraded":"2024-09-10T16:02:08.103276+0000","last_fullsized":"2024-09-10T16:02:08.103276+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'10172","reported_seq":11062,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T16:02:27.646302+0000","last_change":"2024-09-10T16:02:06.480033+0000","last_active":"2024-09-10T16:02:27.646302+0000","last_peered":"2024-09-10T16:02:27.646302+0000","last_clean":"2024-09-10T15:59:14.115595+0000","last_became_active":"2024-09-10T15:59:16.244089+0000","last_became_peered":"2024-09-10T15:59:16.244089+0000","last_unstale":"2024-09-10T16:02:27.646302+0000","last_undegraded":"2024-09-10T16:02:27.646302+0000","last_fullsized":"2024-09-10T15:59:15.774437+0000","mapping_epoch":42,"log_start":"36'100","ondisk_log_start":"36'100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41631744,"num_objects":10164,"num_object_clones":0,"num_object_copies":20328,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3829,"num_objects_unfound":0,"num_objects_dirty":10164,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6562,"num_write_kb":26248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":374,"num_bytes_recovered":1531904,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6335},{"shards":"0,2","objects":3829}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103831+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:02:08.103831+0000","last_peered":"2024-09-10T16:02:08.103831+0000","last_clean":"2024-09-10T16:02:08.103831+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:02:08.103831+0000","last_undegraded":"2024-09-10T16:02:08.103831+0000","last_fullsized":"2024-09-10T16:02:08.103831+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'10392","reported_seq":10532,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:27.665041+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:02:27.665041+0000","last_peered":"2024-09-10T16:02:27.665041+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:02:27.665041+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"36'300","ondisk_log_start":"36'300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42549248,"num_objects":10388,"num_object_clones":0,"num_object_copies":20776,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10388,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6647,"num_write_kb":26588,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6017},{"shards":"0,2","objects":4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104431+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:02:08.104431+0000","last_peered":"2024-09-10T16:02:08.104431+0000","last_clean":"2024-09-10T16:02:08.104431+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:02:08.104431+0000","last_undegraded":"2024-09-10T16:02:08.104431+0000","last_fullsized":"2024-09-10T16:02:08.104431+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104410+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:02:08.104410+0000","last_peered":"2024-09-10T16:02:08.104410+0000","last_clean":"2024-09-10T16:02:08.104410+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:02:08.104410+0000","last_undegraded":"2024-09-10T16:02:08.104410+0000","last_fullsized":"2024-09-10T16:02:08.104410+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'10465","reported_seq":10609,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:27.598239+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:02:27.598239+0000","last_peered":"2024-09-10T16:02:27.598239+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:02:27.598239+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"36'400","ondisk_log_start":"36'400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42816454,"num_objects":10455,"num_object_clones":0,"num_object_copies":20910,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10455,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6628,"num_write_kb":26512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6132},{"shards":"0,2","objects":4323}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":676807390,"num_objects":165245,"num_object_clones":0,"num_object_copies":330490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42911,"num_objects_misplaced":3829,"num_objects_unfound":0,"num_objects_dirty":165245,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":123091,"num_write_kb":492364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4646,"num_bytes_recovered":19025920,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1375338496,"data_stored":1375186750,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160617,"ondisk_log_size":160617,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986230,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":760612,"kb_used_data":467924,"kb_used_omap":4,"kb_used_meta":292667,"kb_avail":93611228,"statfs":{"total":96636764160,"available":95857897472,"internally_reserved":0,"allocated":479154176,"data_stored":478866015,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":299691690},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,3,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659176,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518276,"kb_used_data":310260,"kb_used_omap":4,"kb_used_meta":207995,"kb_avail":93853564,"statfs":{"total":96636764160,"available":96106049536,"internally_reserved":0,"allocated":317706240,"data_stored":317370646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":212987566},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,5,7,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182458,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":897516,"kb_used_data":563228,"kb_used_omap":4,"kb_used_meta":334267,"kb_avail":93474324,"statfs":{"total":96636764160,"available":95717707776,"internally_reserved":0,"allocated":576745472,"data_stored":576453234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":342290090},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,1,1,7,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":577069056,"data_stored":577032926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":318599168,"data_stored":318519682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":479670272,"data_stored":479634142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:02:36.524 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:02:37.083 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:02:37.084 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:02:37.113 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":198,"stamp":"2024-09-10T16:02:35.616311+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":682873609,"num_objects":166729,"num_object_clones":0,"num_object_copies":333458,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42911,"num_objects_misplaced":3740,"num_objects_unfound":0,"num_objects_dirty":166729,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":125239,"num_write_kb":499619,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4737,"num_bytes_recovered":19390483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160866,"ondisk_log_size":160866,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2202576,"kb_used_data":1355104,"kb_used_omap":12,"kb_used_meta":847411,"kb_avail":280912944,"statfs":{"total":289910292480,"available":287654854656,"internally_reserved":0,"allocated":1387626496,"data_stored":1386702359,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":867748866},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,0,3,7,13,9],"upper_bound":128},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":18227224,"num_objects":4451,"num_object_clones":0,"num_object_copies":8902,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-176,"num_objects_unfound":0,"num_objects_dirty":4451,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5115,"num_write_kb":19129,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":176,"num_bytes_recovered":720896,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004905"},"pg_stats":[{"pgid":"2.f","version":"45'10332","reported_seq":10405,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:32.646678+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:02:32.646678+0000","last_peered":"2024-09-10T16:02:32.646678+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:02:32.646678+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"36'300","ondisk_log_start":"36'300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42319872,"num_objects":10332,"num_object_clones":0,"num_object_copies":20664,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10332,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10332,"num_write_kb":41328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6073},{"shards":"0,2","objects":4258}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'10470","reported_seq":10582,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:34.826490+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:02:34.826490+0000","last_peered":"2024-09-10T16:02:34.826490+0000","last_clean":"2024-09-10T16:02:34.826490+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:02:34.826490+0000","last_undegraded":"2024-09-10T16:02:34.826490+0000","last_fullsized":"2024-09-10T16:02:34.826490+0000","mapping_epoch":35,"log_start":"36'400","ondisk_log_start":"36'400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42492870,"num_objects":10376,"num_object_clones":0,"num_object_copies":20752,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10376,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":8575,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'10478","reported_seq":19156,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:35.108299+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:02:35.108299+0000","last_peered":"2024-09-10T16:02:35.108299+0000","last_clean":"2024-09-10T16:02:35.108299+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:02:35.108299+0000","last_undegraded":"2024-09-10T16:02:35.108299+0000","last_fullsized":"2024-09-10T16:02:35.108299+0000","mapping_epoch":44,"log_start":"36'400","ondisk_log_start":"36'400","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42266648,"num_objects":10320,"num_object_clones":0,"num_object_copies":20640,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10320,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8482,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'10377","reported_seq":10457,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:34.827547+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:02:34.827547+0000","last_peered":"2024-09-10T16:02:34.827547+0000","last_clean":"2024-09-10T16:02:34.827547+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:02:34.827547+0000","last_undegraded":"2024-09-10T16:02:34.827547+0000","last_fullsized":"2024-09-10T16:02:34.827547+0000","mapping_epoch":35,"log_start":"36'300","ondisk_log_start":"36'300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42119168,"num_objects":10283,"num_object_clones":0,"num_object_copies":20566,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10283,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10377,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'10440","reported_seq":10570,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:32.668694+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:02:32.668694+0000","last_peered":"2024-09-10T16:02:32.668694+0000","last_clean":"2024-09-10T16:02:32.668694+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:02:32.668694+0000","last_undegraded":"2024-09-10T16:02:32.668694+0000","last_fullsized":"2024-09-10T16:02:32.668694+0000","mapping_epoch":34,"log_start":"36'400","ondisk_log_start":"36'400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42734051,"num_objects":10434,"num_object_clones":0,"num_object_copies":20868,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10434,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7151,"num_write_kb":28604,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'10528","reported_seq":10665,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:32.690815+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:02:32.690815+0000","last_peered":"2024-09-10T16:02:32.690815+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:02:32.690815+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"36'500","ondisk_log_start":"36'500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":43102691,"num_objects":10524,"num_object_clones":0,"num_object_copies":21048,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10524,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7140,"num_write_kb":28560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6153},{"shards":"0,2","objects":4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'10540","reported_seq":10677,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:32.566072+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:02:32.566072+0000","last_peered":"2024-09-10T16:02:32.566072+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:02:32.566072+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"36'500","ondisk_log_start":"36'500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":43148230,"num_objects":10536,"num_object_clones":0,"num_object_copies":21072,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10536,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":7039,"num_write_kb":28156,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6299},{"shards":"0,2","objects":4237}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'10653","reported_seq":10783,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:32.635272+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:02:32.635272+0000","last_peered":"2024-09-10T16:02:32.635272+0000","last_clean":"2024-09-10T16:02:32.635272+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:02:32.635272+0000","last_undegraded":"2024-09-10T16:02:32.635272+0000","last_fullsized":"2024-09-10T16:02:32.635272+0000","mapping_epoch":34,"log_start":"36'600","ondisk_log_start":"36'600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":43626496,"num_objects":10651,"num_object_clones":0,"num_object_copies":21302,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10651,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7500,"num_write_kb":30000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104219+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:02:08.104219+0000","last_peered":"2024-09-10T16:02:08.104219+0000","last_clean":"2024-09-10T16:02:08.104219+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:02:08.104219+0000","last_undegraded":"2024-09-10T16:02:08.104219+0000","last_fullsized":"2024-09-10T16:02:08.104219+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'10291","reported_seq":10430,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:34.859452+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:02:34.859452+0000","last_peered":"2024-09-10T16:02:34.859452+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:02:34.859452+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"36'200","ondisk_log_start":"36'200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41640419,"num_objects":10167,"num_object_clones":0,"num_object_copies":20334,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10167,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8110,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4064},{"shards":"2","objects":6044}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103774+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:02:08.103774+0000","last_peered":"2024-09-10T16:02:08.103774+0000","last_clean":"2024-09-10T16:02:08.103774+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:02:08.103774+0000","last_undegraded":"2024-09-10T16:02:08.103774+0000","last_fullsized":"2024-09-10T16:02:08.103774+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'10448","reported_seq":10578,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:32.684612+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:02:32.684612+0000","last_peered":"2024-09-10T16:02:32.684612+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:02:32.684612+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"36'400","ondisk_log_start":"36'400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42762240,"num_objects":10440,"num_object_clones":0,"num_object_copies":20880,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10440,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6952,"num_write_kb":27808,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6185},{"shards":"0,2","objects":4255}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":104,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.105055+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:02:08.105055+0000","last_peered":"2024-09-10T16:02:08.105055+0000","last_clean":"2024-09-10T16:02:08.105055+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:02:08.105055+0000","last_undegraded":"2024-09-10T16:02:08.105055+0000","last_fullsized":"2024-09-10T16:02:08.105055+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'10444","reported_seq":10576,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:34.793037+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:02:34.793037+0000","last_peered":"2024-09-10T16:02:34.793037+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:02:34.793037+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"36'400","ondisk_log_start":"36'400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42340835,"num_objects":10338,"num_object_clones":0,"num_object_copies":20676,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10338,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8124,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4253},{"shards":"2","objects":6034}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104572+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:02:08.104572+0000","last_peered":"2024-09-10T16:02:08.104572+0000","last_clean":"2024-09-10T16:02:08.104572+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:02:08.104572+0000","last_undegraded":"2024-09-10T16:02:08.104572+0000","last_fullsized":"2024-09-10T16:02:08.104572+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'10641","reported_seq":10770,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:34.848292+0000","last_change":"2024-09-10T15:59:16.174113+0000","last_active":"2024-09-10T16:02:34.848292+0000","last_peered":"2024-09-10T16:02:34.848292+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:02:34.848292+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"36'600","ondisk_log_start":"36'600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":43126784,"num_objects":10529,"num_object_clones":0,"num_object_copies":21058,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4354,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10529,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8195,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4299},{"shards":"2","objects":6177}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'10414","reported_seq":10548,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:32.592322+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:02:32.592322+0000","last_peered":"2024-09-10T16:02:32.592322+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:02:32.592322+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"36'400","ondisk_log_start":"36'400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42647552,"num_objects":10412,"num_object_clones":0,"num_object_copies":20824,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10412,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7045,"num_write_kb":28180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6108},{"shards":"0,2","objects":4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103276+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:02:08.103276+0000","last_peered":"2024-09-10T16:02:08.103276+0000","last_clean":"2024-09-10T16:02:08.103276+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:02:08.103276+0000","last_undegraded":"2024-09-10T16:02:08.103276+0000","last_fullsized":"2024-09-10T16:02:08.103276+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'10296","reported_seq":11366,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T16:02:32.667526+0000","last_change":"2024-09-10T16:02:06.480033+0000","last_active":"2024-09-10T16:02:32.667526+0000","last_peered":"2024-09-10T16:02:32.667526+0000","last_clean":"2024-09-10T15:59:14.115595+0000","last_became_active":"2024-09-10T15:59:16.244089+0000","last_became_peered":"2024-09-10T15:59:16.244089+0000","last_unstale":"2024-09-10T16:02:32.667526+0000","last_undegraded":"2024-09-10T16:02:32.667526+0000","last_fullsized":"2024-09-10T15:59:15.774437+0000","mapping_epoch":42,"log_start":"36'200","ondisk_log_start":"36'200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42139648,"num_objects":10288,"num_object_clones":0,"num_object_copies":20576,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3740,"num_objects_unfound":0,"num_objects_dirty":10288,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6686,"num_write_kb":26744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":463,"num_bytes_recovered":1896448,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6548},{"shards":"0,2","objects":3740}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103831+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:02:08.103831+0000","last_peered":"2024-09-10T16:02:08.103831+0000","last_clean":"2024-09-10T16:02:08.103831+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:02:08.103831+0000","last_undegraded":"2024-09-10T16:02:08.103831+0000","last_fullsized":"2024-09-10T16:02:08.103831+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'10513","reported_seq":10655,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:32.557510+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:02:32.557510+0000","last_peered":"2024-09-10T16:02:32.557510+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:02:32.557510+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"36'500","ondisk_log_start":"36'500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":43044864,"num_objects":10509,"num_object_clones":0,"num_object_copies":21018,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10509,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6768,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6138},{"shards":"0,2","objects":4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104431+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:02:08.104431+0000","last_peered":"2024-09-10T16:02:08.104431+0000","last_clean":"2024-09-10T16:02:08.104431+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:02:08.104431+0000","last_undegraded":"2024-09-10T16:02:08.104431+0000","last_fullsized":"2024-09-10T16:02:08.104431+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104410+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:02:08.104410+0000","last_peered":"2024-09-10T16:02:08.104410+0000","last_clean":"2024-09-10T16:02:08.104410+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:02:08.104410+0000","last_undegraded":"2024-09-10T16:02:08.104410+0000","last_fullsized":"2024-09-10T16:02:08.104410+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'10598","reported_seq":10742,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:32.589495+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:02:32.589495+0000","last_peered":"2024-09-10T16:02:32.589495+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:02:32.589495+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"36'500","ondisk_log_start":"36'500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":43361222,"num_objects":10588,"num_object_clones":0,"num_object_copies":21176,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10588,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6761,"num_write_kb":27044,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6265},{"shards":"0,2","objects":4323}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":682873590,"num_objects":166727,"num_object_clones":0,"num_object_copies":333454,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42911,"num_objects_misplaced":3740,"num_objects_unfound":0,"num_objects_dirty":166727,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":125237,"num_write_kb":499617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4735,"num_bytes_recovered":19390464,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1386192896,"data_stored":1386033006,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160863,"ondisk_log_size":160863,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986231,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":763252,"kb_used_data":470564,"kb_used_omap":4,"kb_used_meta":292667,"kb_avail":93608588,"statfs":{"total":96636764160,"available":95855194112,"internally_reserved":0,"allocated":481857536,"data_stored":481565303,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":299691690},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,1,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659177,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":526972,"kb_used_data":314796,"kb_used_omap":4,"kb_used_meta":212155,"kb_avail":93844868,"statfs":{"total":96636764160,"available":96097144832,"internally_reserved":0,"allocated":322351104,"data_stored":322011438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":217247406},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,1,3,5,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182459,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":912352,"kb_used_data":569744,"kb_used_omap":4,"kb_used_meta":342587,"kb_avail":93459488,"statfs":{"total":96636764160,"available":95702515712,"internally_reserved":0,"allocated":583417856,"data_stored":583125618,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":350809770},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,3,5,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":583692288,"data_stored":583656158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":321814528,"data_stored":321730970,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":480686080,"data_stored":480645878,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:02:37.114 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:02:37.670 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:02:37.672 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:02:37.694 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":199,"stamp":"2024-09-10T16:02:37.617189+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":682873609,"num_objects":166729,"num_object_clones":0,"num_object_copies":333458,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42911,"num_objects_misplaced":3740,"num_objects_unfound":0,"num_objects_dirty":166729,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":125239,"num_write_kb":499619,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4737,"num_bytes_recovered":19390483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160866,"ondisk_log_size":160866,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2202576,"kb_used_data":1355104,"kb_used_omap":12,"kb_used_meta":847411,"kb_avail":280912944,"statfs":{"total":289910292480,"available":287654854656,"internally_reserved":0,"allocated":1387626496,"data_stored":1386702359,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":867748866},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,0,3,7,13,9],"upper_bound":128},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":14909464,"num_objects":3641,"num_object_clones":0,"num_object_copies":7282,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-176,"num_objects_unfound":0,"num_objects_dirty":3641,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4305,"num_write_kb":15889,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":176,"num_bytes_recovered":720896,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004656"},"pg_stats":[{"pgid":"2.f","version":"45'10332","reported_seq":10405,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:32.646678+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:02:32.646678+0000","last_peered":"2024-09-10T16:02:32.646678+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:02:32.646678+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"36'300","ondisk_log_start":"36'300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42319872,"num_objects":10332,"num_object_clones":0,"num_object_copies":20664,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10332,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10332,"num_write_kb":41328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6073},{"shards":"0,2","objects":4258}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'10470","reported_seq":10582,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:34.826490+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:02:34.826490+0000","last_peered":"2024-09-10T16:02:34.826490+0000","last_clean":"2024-09-10T16:02:34.826490+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:02:34.826490+0000","last_undegraded":"2024-09-10T16:02:34.826490+0000","last_fullsized":"2024-09-10T16:02:34.826490+0000","mapping_epoch":35,"log_start":"36'400","ondisk_log_start":"36'400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42492870,"num_objects":10376,"num_object_clones":0,"num_object_copies":20752,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10376,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":8575,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'10478","reported_seq":19156,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:35.108299+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:02:35.108299+0000","last_peered":"2024-09-10T16:02:35.108299+0000","last_clean":"2024-09-10T16:02:35.108299+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:02:35.108299+0000","last_undegraded":"2024-09-10T16:02:35.108299+0000","last_fullsized":"2024-09-10T16:02:35.108299+0000","mapping_epoch":44,"log_start":"36'400","ondisk_log_start":"36'400","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42266648,"num_objects":10320,"num_object_clones":0,"num_object_copies":20640,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10320,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8482,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'10377","reported_seq":10457,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:34.827547+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:02:34.827547+0000","last_peered":"2024-09-10T16:02:34.827547+0000","last_clean":"2024-09-10T16:02:34.827547+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:02:34.827547+0000","last_undegraded":"2024-09-10T16:02:34.827547+0000","last_fullsized":"2024-09-10T16:02:34.827547+0000","mapping_epoch":35,"log_start":"36'300","ondisk_log_start":"36'300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42119168,"num_objects":10283,"num_object_clones":0,"num_object_copies":20566,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10283,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10377,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'10440","reported_seq":10570,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:32.668694+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:02:32.668694+0000","last_peered":"2024-09-10T16:02:32.668694+0000","last_clean":"2024-09-10T16:02:32.668694+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:02:32.668694+0000","last_undegraded":"2024-09-10T16:02:32.668694+0000","last_fullsized":"2024-09-10T16:02:32.668694+0000","mapping_epoch":34,"log_start":"36'400","ondisk_log_start":"36'400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42734051,"num_objects":10434,"num_object_clones":0,"num_object_copies":20868,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10434,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7151,"num_write_kb":28604,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'10528","reported_seq":10665,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:32.690815+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:02:32.690815+0000","last_peered":"2024-09-10T16:02:32.690815+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:02:32.690815+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"36'500","ondisk_log_start":"36'500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":43102691,"num_objects":10524,"num_object_clones":0,"num_object_copies":21048,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10524,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7140,"num_write_kb":28560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6153},{"shards":"0,2","objects":4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'10540","reported_seq":10677,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:32.566072+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:02:32.566072+0000","last_peered":"2024-09-10T16:02:32.566072+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:02:32.566072+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"36'500","ondisk_log_start":"36'500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":43148230,"num_objects":10536,"num_object_clones":0,"num_object_copies":21072,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10536,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":7039,"num_write_kb":28156,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6299},{"shards":"0,2","objects":4237}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'10653","reported_seq":10783,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:32.635272+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:02:32.635272+0000","last_peered":"2024-09-10T16:02:32.635272+0000","last_clean":"2024-09-10T16:02:32.635272+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:02:32.635272+0000","last_undegraded":"2024-09-10T16:02:32.635272+0000","last_fullsized":"2024-09-10T16:02:32.635272+0000","mapping_epoch":34,"log_start":"36'600","ondisk_log_start":"36'600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":43626496,"num_objects":10651,"num_object_clones":0,"num_object_copies":21302,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10651,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7500,"num_write_kb":30000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104219+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:02:08.104219+0000","last_peered":"2024-09-10T16:02:08.104219+0000","last_clean":"2024-09-10T16:02:08.104219+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:02:08.104219+0000","last_undegraded":"2024-09-10T16:02:08.104219+0000","last_fullsized":"2024-09-10T16:02:08.104219+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'10291","reported_seq":10430,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:34.859452+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:02:34.859452+0000","last_peered":"2024-09-10T16:02:34.859452+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:02:34.859452+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"36'200","ondisk_log_start":"36'200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41640419,"num_objects":10167,"num_object_clones":0,"num_object_copies":20334,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10167,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8110,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4064},{"shards":"2","objects":6044}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103774+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:02:08.103774+0000","last_peered":"2024-09-10T16:02:08.103774+0000","last_clean":"2024-09-10T16:02:08.103774+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:02:08.103774+0000","last_undegraded":"2024-09-10T16:02:08.103774+0000","last_fullsized":"2024-09-10T16:02:08.103774+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'10448","reported_seq":10578,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:32.684612+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:02:32.684612+0000","last_peered":"2024-09-10T16:02:32.684612+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:02:32.684612+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"36'400","ondisk_log_start":"36'400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42762240,"num_objects":10440,"num_object_clones":0,"num_object_copies":20880,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10440,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6952,"num_write_kb":27808,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6185},{"shards":"0,2","objects":4255}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":104,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.105055+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:02:08.105055+0000","last_peered":"2024-09-10T16:02:08.105055+0000","last_clean":"2024-09-10T16:02:08.105055+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:02:08.105055+0000","last_undegraded":"2024-09-10T16:02:08.105055+0000","last_fullsized":"2024-09-10T16:02:08.105055+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'10444","reported_seq":10576,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:34.793037+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:02:34.793037+0000","last_peered":"2024-09-10T16:02:34.793037+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:02:34.793037+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"36'400","ondisk_log_start":"36'400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42340835,"num_objects":10338,"num_object_clones":0,"num_object_copies":20676,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10338,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8124,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4253},{"shards":"2","objects":6034}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104572+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:02:08.104572+0000","last_peered":"2024-09-10T16:02:08.104572+0000","last_clean":"2024-09-10T16:02:08.104572+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:02:08.104572+0000","last_undegraded":"2024-09-10T16:02:08.104572+0000","last_fullsized":"2024-09-10T16:02:08.104572+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'10641","reported_seq":10770,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:34.848292+0000","last_change":"2024-09-10T15:59:16.174113+0000","last_active":"2024-09-10T16:02:34.848292+0000","last_peered":"2024-09-10T16:02:34.848292+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:02:34.848292+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"36'600","ondisk_log_start":"36'600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":43126784,"num_objects":10529,"num_object_clones":0,"num_object_copies":21058,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4354,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10529,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8195,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4299},{"shards":"2","objects":6177}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'10414","reported_seq":10548,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:32.592322+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:02:32.592322+0000","last_peered":"2024-09-10T16:02:32.592322+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:02:32.592322+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"36'400","ondisk_log_start":"36'400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42647552,"num_objects":10412,"num_object_clones":0,"num_object_copies":20824,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10412,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7045,"num_write_kb":28180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6108},{"shards":"0,2","objects":4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103276+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:02:08.103276+0000","last_peered":"2024-09-10T16:02:08.103276+0000","last_clean":"2024-09-10T16:02:08.103276+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:02:08.103276+0000","last_undegraded":"2024-09-10T16:02:08.103276+0000","last_fullsized":"2024-09-10T16:02:08.103276+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'10296","reported_seq":11366,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T16:02:32.667526+0000","last_change":"2024-09-10T16:02:06.480033+0000","last_active":"2024-09-10T16:02:32.667526+0000","last_peered":"2024-09-10T16:02:32.667526+0000","last_clean":"2024-09-10T15:59:14.115595+0000","last_became_active":"2024-09-10T15:59:16.244089+0000","last_became_peered":"2024-09-10T15:59:16.244089+0000","last_unstale":"2024-09-10T16:02:32.667526+0000","last_undegraded":"2024-09-10T16:02:32.667526+0000","last_fullsized":"2024-09-10T15:59:15.774437+0000","mapping_epoch":42,"log_start":"36'200","ondisk_log_start":"36'200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42139648,"num_objects":10288,"num_object_clones":0,"num_object_copies":20576,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3740,"num_objects_unfound":0,"num_objects_dirty":10288,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6686,"num_write_kb":26744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":463,"num_bytes_recovered":1896448,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6548},{"shards":"0,2","objects":3740}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103831+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:02:08.103831+0000","last_peered":"2024-09-10T16:02:08.103831+0000","last_clean":"2024-09-10T16:02:08.103831+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:02:08.103831+0000","last_undegraded":"2024-09-10T16:02:08.103831+0000","last_fullsized":"2024-09-10T16:02:08.103831+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'10513","reported_seq":10655,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:32.557510+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:02:32.557510+0000","last_peered":"2024-09-10T16:02:32.557510+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:02:32.557510+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"36'500","ondisk_log_start":"36'500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":43044864,"num_objects":10509,"num_object_clones":0,"num_object_copies":21018,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10509,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6768,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6138},{"shards":"0,2","objects":4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104431+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:02:08.104431+0000","last_peered":"2024-09-10T16:02:08.104431+0000","last_clean":"2024-09-10T16:02:08.104431+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:02:08.104431+0000","last_undegraded":"2024-09-10T16:02:08.104431+0000","last_fullsized":"2024-09-10T16:02:08.104431+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104410+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:02:08.104410+0000","last_peered":"2024-09-10T16:02:08.104410+0000","last_clean":"2024-09-10T16:02:08.104410+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:02:08.104410+0000","last_undegraded":"2024-09-10T16:02:08.104410+0000","last_fullsized":"2024-09-10T16:02:08.104410+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'10598","reported_seq":10742,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:32.589495+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:02:32.589495+0000","last_peered":"2024-09-10T16:02:32.589495+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:02:32.589495+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"36'500","ondisk_log_start":"36'500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":43361222,"num_objects":10588,"num_object_clones":0,"num_object_copies":21176,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10588,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6761,"num_write_kb":27044,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6265},{"shards":"0,2","objects":4323}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":682873590,"num_objects":166727,"num_object_clones":0,"num_object_copies":333454,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42911,"num_objects_misplaced":3740,"num_objects_unfound":0,"num_objects_dirty":166727,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":125237,"num_write_kb":499617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4735,"num_bytes_recovered":19390464,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1386192896,"data_stored":1386033006,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160863,"ondisk_log_size":160863,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986231,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":763252,"kb_used_data":470564,"kb_used_omap":4,"kb_used_meta":292667,"kb_avail":93608588,"statfs":{"total":96636764160,"available":95855194112,"internally_reserved":0,"allocated":481857536,"data_stored":481565303,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":299691690},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,1,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659177,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":526972,"kb_used_data":314796,"kb_used_omap":4,"kb_used_meta":212155,"kb_avail":93844868,"statfs":{"total":96636764160,"available":96097144832,"internally_reserved":0,"allocated":322351104,"data_stored":322011438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":217247406},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,1,3,5,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182459,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":912352,"kb_used_data":569744,"kb_used_omap":4,"kb_used_meta":342587,"kb_avail":93459488,"statfs":{"total":96636764160,"available":95702515712,"internally_reserved":0,"allocated":583417856,"data_stored":583125618,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":350809770},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,3,5,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":583692288,"data_stored":583656158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":321814528,"data_stored":321730970,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":480686080,"data_stored":480645878,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:02:37.696 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-10T16:02:37.696 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "45'10332", 'reported_seq': 10405, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:02:32.646678+0000', 'last_change': '2024-09-10T15:59:16.281558+0000', 'last_active': '2024-09-10T16:02:32.646678+0000', 'last_peered': '2024-09-10T16:02:32.646678+0000', 'last_clean': '2024-09-10T15:59:14.132456+0000', 'last_became_active': '2024-09-10T15:59:16.274407+0000', 'last_became_peered': '2024-09-10T15:59:16.274407+0000', 'last_unstale': '2024-09-10T16:02:32.646678+0000', 'last_undegraded': '2024-09-10T15:59:15.851279+0000', 'last_fullsized': '2024-09-10T15:59:15.774356+0000', 'mapping_epoch': 42, 'log_start': "36'300", 'ondisk_log_start': "36'300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'objects_scrubbed': 0, 'log_size': 10032, 'log_dups_size': 0, 'ondisk_log_size': 10032, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 42319872, 'num_objects': 10332, 'num_object_clones': 0, 'num_object_copies': 20664, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4259, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10332, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10332, 'num_write_kb': 41328, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6073}, {'shards': '0,2', 'objects': 4258}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:02:37.696 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-10T16:02:37.696 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "45'10528", 'reported_seq': 10665, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:02:32.690815+0000', 'last_change': '2024-09-10T15:59:16.308264+0000', 'last_active': '2024-09-10T16:02:32.690815+0000', 'last_peered': '2024-09-10T16:02:32.690815+0000', 'last_clean': '2024-09-10T15:59:14.111551+0000', 'last_became_active': '2024-09-10T15:59:16.300040+0000', 'last_became_peered': '2024-09-10T15:59:16.300040+0000', 'last_unstale': '2024-09-10T16:02:32.690815+0000', 'last_undegraded': '2024-09-10T15:59:15.878234+0000', 'last_fullsized': '2024-09-10T15:59:15.769328+0000', 'mapping_epoch': 42, 'log_start': "36'500", 'ondisk_log_start': "36'500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3442", 'last_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'objects_scrubbed': 3384, 'log_size': 10028, 'log_dups_size': 0, 'ondisk_log_size': 10028, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1292, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 43102691, 'num_objects': 10524, 'num_object_clones': 0, 'num_object_copies': 21048, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4372, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10524, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 7140, 'num_write_kb': 28560, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6153}, {'shards': '0,2', 'objects': 4371}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:02:37.696 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-10T16:02:37.696 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "45'10540", 'reported_seq': 10677, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:02:32.566072+0000', 'last_change': '2024-09-10T15:59:16.331899+0000', 'last_active': '2024-09-10T16:02:32.566072+0000', 'last_peered': '2024-09-10T16:02:32.566072+0000', 'last_clean': '2024-09-10T15:59:14.139436+0000', 'last_became_active': '2024-09-10T15:59:16.318715+0000', 'last_became_peered': '2024-09-10T15:59:16.318715+0000', 'last_unstale': '2024-09-10T16:02:32.566072+0000', 'last_undegraded': '2024-09-10T15:59:15.877524+0000', 'last_fullsized': '2024-09-10T15:59:15.769042+0000', 'mapping_epoch': 42, 'log_start': "36'500", 'ondisk_log_start': "36'500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3537", 'last_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'objects_scrubbed': 3497, 'log_size': 10040, 'log_dups_size': 0, 'ondisk_log_size': 10040, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1224, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 43148230, 'num_objects': 10536, 'num_object_clones': 0, 'num_object_copies': 21072, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4237, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10536, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 7039, 'num_write_kb': 28156, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6299}, {'shards': '0,2', 'objects': 4237}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:02:37.696 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-10T16:02:37.696 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "45'10291", 'reported_seq': 10430, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:02:34.859452+0000', 'last_change': '2024-09-10T15:59:16.175680+0000', 'last_active': '2024-09-10T16:02:34.859452+0000', 'last_peered': '2024-09-10T16:02:34.859452+0000', 'last_clean': '2024-09-10T15:59:14.088187+0000', 'last_became_active': '2024-09-10T15:59:16.165810+0000', 'last_became_peered': '2024-09-10T15:59:16.165810+0000', 'last_unstale': '2024-09-10T16:02:34.859452+0000', 'last_undegraded': '2024-09-10T15:59:15.822098+0000', 'last_fullsized': '2024-09-10T15:59:15.769123+0000', 'mapping_epoch': 42, 'log_start': "36'200", 'ondisk_log_start': "36'200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2218", 'last_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'objects_scrubbed': 2175, 'log_size': 10091, 'log_dups_size': 0, 'ondisk_log_size': 10091, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 825, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 41640419, 'num_objects': 10167, 'num_object_clones': 0, 'num_object_copies': 20334, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4124, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10167, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 8110, 'num_write_kb': 32204, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4064}, {'shards': '2', 'objects': 6044}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:02:37.696 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-10T16:02:37.696 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "45'10448", 'reported_seq': 10578, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:02:32.684612+0000', 'last_change': '2024-09-10T15:59:16.277742+0000', 'last_active': '2024-09-10T16:02:32.684612+0000', 'last_peered': '2024-09-10T16:02:32.684612+0000', 'last_clean': '2024-09-10T15:59:14.129081+0000', 'last_became_active': '2024-09-10T15:59:16.269835+0000', 'last_became_peered': '2024-09-10T15:59:16.269835+0000', 'last_unstale': '2024-09-10T16:02:32.684612+0000', 'last_undegraded': '2024-09-10T15:59:15.860857+0000', 'last_fullsized': '2024-09-10T15:59:15.771410+0000', 'mapping_epoch': 42, 'log_start': "36'400", 'ondisk_log_start': "36'400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3557", 'last_scrub_stamp': '2024-09-10T15:59:02.493678+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:02.493678+0000', 'objects_scrubbed': 3488, 'log_size': 10048, 'log_dups_size': 0, 'ondisk_log_size': 10048, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1472, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 42762240, 'num_objects': 10440, 'num_object_clones': 0, 'num_object_copies': 20880, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4257, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10440, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6952, 'num_write_kb': 27808, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6185}, {'shards': '0,2', 'objects': 4255}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:02:37.697 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-10T16:02:37.697 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "45'10444", 'reported_seq': 10576, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:02:34.793037+0000', 'last_change': '2024-09-10T15:59:16.284311+0000', 'last_active': '2024-09-10T16:02:34.793037+0000', 'last_peered': '2024-09-10T16:02:34.793037+0000', 'last_clean': '2024-09-10T15:59:14.116041+0000', 'last_became_active': '2024-09-10T15:59:16.274543+0000', 'last_became_peered': '2024-09-10T15:59:16.274543+0000', 'last_unstale': '2024-09-10T16:02:34.793037+0000', 'last_undegraded': '2024-09-10T15:59:15.812484+0000', 'last_fullsized': '2024-09-10T15:59:15.770885+0000', 'mapping_epoch': 42, 'log_start': "36'400", 'ondisk_log_start': "36'400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2356", 'last_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'objects_scrubbed': 2316, 'log_size': 10044, 'log_dups_size': 0, 'ondisk_log_size': 10044, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 783, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 42340835, 'num_objects': 10338, 'num_object_clones': 0, 'num_object_copies': 20676, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10338, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 8124, 'num_write_kb': 32292, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4253}, {'shards': '2', 'objects': 6034}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:02:37.697 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-10T16:02:37.697 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "45'10641", 'reported_seq': 10770, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:02:34.848292+0000', 'last_change': '2024-09-10T15:59:16.174113+0000', 'last_active': '2024-09-10T16:02:34.848292+0000', 'last_peered': '2024-09-10T16:02:34.848292+0000', 'last_clean': '2024-09-10T15:59:14.110654+0000', 'last_became_active': '2024-09-10T15:59:16.164645+0000', 'last_became_peered': '2024-09-10T15:59:16.164645+0000', 'last_unstale': '2024-09-10T16:02:34.848292+0000', 'last_undegraded': '2024-09-10T15:59:15.803593+0000', 'last_fullsized': '2024-09-10T15:59:15.769083+0000', 'mapping_epoch': 42, 'log_start': "36'600", 'ondisk_log_start': "36'600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2486", 'last_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'objects_scrubbed': 2440, 'log_size': 10041, 'log_dups_size': 0, 'ondisk_log_size': 10041, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 792, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 43126784, 'num_objects': 10529, 'num_object_clones': 0, 'num_object_copies': 21058, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4354, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10529, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8195, 'num_write_kb': 32568, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4299}, {'shards': '2', 'objects': 6177}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:02:37.697 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-10T16:02:37.697 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "45'10414", 'reported_seq': 10548, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:02:32.592322+0000', 'last_change': '2024-09-10T15:59:16.315641+0000', 'last_active': '2024-09-10T16:02:32.592322+0000', 'last_peered': '2024-09-10T16:02:32.592322+0000', 'last_clean': '2024-09-10T15:59:14.117001+0000', 'last_became_active': '2024-09-10T15:59:16.302245+0000', 'last_became_peered': '2024-09-10T15:59:16.302245+0000', 'last_unstale': '2024-09-10T16:02:32.592322+0000', 'last_undegraded': '2024-09-10T15:59:15.845012+0000', 'last_fullsized': '2024-09-10T15:59:15.770337+0000', 'mapping_epoch': 42, 'log_start': "36'400", 'ondisk_log_start': "36'400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3404", 'last_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'objects_scrubbed': 3367, 'log_size': 10014, 'log_dups_size': 0, 'ondisk_log_size': 10014, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 983, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 42647552, 'num_objects': 10412, 'num_object_clones': 0, 'num_object_copies': 20824, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10412, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7045, 'num_write_kb': 28180, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6108}, {'shards': '0,2', 'objects': 4304}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:02:37.697 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-10T16:02:37.697 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "45'10296", 'reported_seq': 11366, 'reported_epoch': 45, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-10T16:02:32.667526+0000', 'last_change': '2024-09-10T16:02:06.480033+0000', 'last_active': '2024-09-10T16:02:32.667526+0000', 'last_peered': '2024-09-10T16:02:32.667526+0000', 'last_clean': '2024-09-10T15:59:14.115595+0000', 'last_became_active': '2024-09-10T15:59:16.244089+0000', 'last_became_peered': '2024-09-10T15:59:16.244089+0000', 'last_unstale': '2024-09-10T16:02:32.667526+0000', 'last_undegraded': '2024-09-10T16:02:32.667526+0000', 'last_fullsized': '2024-09-10T15:59:15.774437+0000', 'mapping_epoch': 42, 'log_start': "36'200", 'ondisk_log_start': "36'200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3653", 'last_scrub_stamp': '2024-09-10T15:59:05.298468+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:05.298468+0000', 'objects_scrubbed': 3602, 'log_size': 10096, 'log_dups_size': 0, 'ondisk_log_size': 10096, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1289, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 42139648, 'num_objects': 10288, 'num_object_clones': 0, 'num_object_copies': 20576, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 3740, 'num_objects_unfound': 0, 'num_objects_dirty': 10288, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6686, 'num_write_kb': 26744, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 463, 'num_bytes_recovered': 1896448, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6548}, {'shards': '0,2', 'objects': 3740}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:02:37.697 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-10T16:02:37.697 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "45'10513", 'reported_seq': 10655, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:02:32.557510+0000', 'last_change': '2024-09-10T15:59:16.311291+0000', 'last_active': '2024-09-10T16:02:32.557510+0000', 'last_peered': '2024-09-10T16:02:32.557510+0000', 'last_clean': '2024-09-10T15:59:14.116363+0000', 'last_became_active': '2024-09-10T15:59:16.301244+0000', 'last_became_peered': '2024-09-10T15:59:16.301244+0000', 'last_unstale': '2024-09-10T16:02:32.557510+0000', 'last_undegraded': '2024-09-10T15:59:15.883434+0000', 'last_fullsized': '2024-09-10T15:59:15.772756+0000', 'mapping_epoch': 42, 'log_start': "36'500", 'ondisk_log_start': "36'500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3789", 'last_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'objects_scrubbed': 3741, 'log_size': 10013, 'log_dups_size': 0, 'ondisk_log_size': 10013, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1440, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 43044864, 'num_objects': 10509, 'num_object_clones': 0, 'num_object_copies': 21018, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4373, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10509, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6768, 'num_write_kb': 27072, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6138}, {'shards': '0,2', 'objects': 4371}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:02:37.697 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-10T16:02:37.698 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "45'10598", 'reported_seq': 10742, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:02:32.589495+0000', 'last_change': '2024-09-10T15:59:16.313409+0000', 'last_active': '2024-09-10T16:02:32.589495+0000', 'last_peered': '2024-09-10T16:02:32.589495+0000', 'last_clean': '2024-09-10T15:59:14.129780+0000', 'last_became_active': '2024-09-10T15:59:16.300932+0000', 'last_became_peered': '2024-09-10T15:59:16.300932+0000', 'last_unstale': '2024-09-10T16:02:32.589495+0000', 'last_undegraded': '2024-09-10T15:59:15.887823+0000', 'last_fullsized': '2024-09-10T15:59:15.768513+0000', 'mapping_epoch': 42, 'log_start': "36'500", 'ondisk_log_start': "36'500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3880", 'last_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'objects_scrubbed': 3827, 'log_size': 10098, 'log_dups_size': 0, 'ondisk_log_size': 10098, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1397, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 43361222, 'num_objects': 10588, 'num_object_clones': 0, 'num_object_copies': 21176, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4325, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10588, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6761, 'num_write_kb': 27044, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6265}, {'shards': '0,2', 'objects': 4323}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:02:37.698 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T16:02:38.404 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:02:38.405 INFO:teuthology.orchestra.run.smithi115.stdout:{"fsid":"7effee29-df05-4351-9741-ee203564ed81","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 42911/333458 objects degraded (12.868%), 10 pgs degraded, 11 pgs undersized","count":21},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":322,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983954,"num_in_osds":3,"osd_in_since":1725983947,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":166729,"data_bytes":682873609,"bytes_used":2255437824,"bytes_avail":287654854656,"bytes_total":289910292480,"degraded_objects":42911,"degraded_total":333458,"degraded_ratio":0.12868487185792513,"misplaced_objects":3740,"misplaced_total":333458,"misplaced_ratio":0.011215805288821981,"recovering_objects_per_sec":14,"recovering_bytes_per_sec":60051,"recovering_keys_per_sec":0,"num_objects_recovered":176,"num_bytes_recovered":720896,"num_keys_recovered":0,"read_bytes_sec":85,"write_bytes_sec":1355335,"read_op_per_sec":0,"write_op_per_sec":358},"fsmap":{"epoch":1,"btime":"2024-09-10T15:57:14:967120+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T16:00:13.560776+0000","services":{}},"progress_events":{}} 2024-09-10T16:02:38.436 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-10T16:02:38.436 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:02:39.020 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:02:39.020 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:02:39.046 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":199,"stamp":"2024-09-10T16:02:37.617189+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":682873609,"num_objects":166729,"num_object_clones":0,"num_object_copies":333458,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42911,"num_objects_misplaced":3740,"num_objects_unfound":0,"num_objects_dirty":166729,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":125239,"num_write_kb":499619,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4737,"num_bytes_recovered":19390483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160866,"ondisk_log_size":160866,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2202576,"kb_used_data":1355104,"kb_used_omap":12,"kb_used_meta":847411,"kb_avail":280912944,"statfs":{"total":289910292480,"available":287654854656,"internally_reserved":0,"allocated":1387626496,"data_stored":1386702359,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":867748866},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,0,3,7,13,9],"upper_bound":128},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":14909464,"num_objects":3641,"num_object_clones":0,"num_object_copies":7282,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-176,"num_objects_unfound":0,"num_objects_dirty":3641,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4305,"num_write_kb":15889,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":176,"num_bytes_recovered":720896,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004656"},"pg_stats":[{"pgid":"2.f","version":"45'10332","reported_seq":10405,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:32.646678+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:02:32.646678+0000","last_peered":"2024-09-10T16:02:32.646678+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:02:32.646678+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"36'300","ondisk_log_start":"36'300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42319872,"num_objects":10332,"num_object_clones":0,"num_object_copies":20664,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10332,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10332,"num_write_kb":41328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6073},{"shards":"0,2","objects":4258}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'10470","reported_seq":10582,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:34.826490+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:02:34.826490+0000","last_peered":"2024-09-10T16:02:34.826490+0000","last_clean":"2024-09-10T16:02:34.826490+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:02:34.826490+0000","last_undegraded":"2024-09-10T16:02:34.826490+0000","last_fullsized":"2024-09-10T16:02:34.826490+0000","mapping_epoch":35,"log_start":"36'400","ondisk_log_start":"36'400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42492870,"num_objects":10376,"num_object_clones":0,"num_object_copies":20752,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10376,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":8575,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'10478","reported_seq":19156,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:35.108299+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:02:35.108299+0000","last_peered":"2024-09-10T16:02:35.108299+0000","last_clean":"2024-09-10T16:02:35.108299+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:02:35.108299+0000","last_undegraded":"2024-09-10T16:02:35.108299+0000","last_fullsized":"2024-09-10T16:02:35.108299+0000","mapping_epoch":44,"log_start":"36'400","ondisk_log_start":"36'400","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42266648,"num_objects":10320,"num_object_clones":0,"num_object_copies":20640,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10320,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8482,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'10377","reported_seq":10457,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:34.827547+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:02:34.827547+0000","last_peered":"2024-09-10T16:02:34.827547+0000","last_clean":"2024-09-10T16:02:34.827547+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:02:34.827547+0000","last_undegraded":"2024-09-10T16:02:34.827547+0000","last_fullsized":"2024-09-10T16:02:34.827547+0000","mapping_epoch":35,"log_start":"36'300","ondisk_log_start":"36'300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42119168,"num_objects":10283,"num_object_clones":0,"num_object_copies":20566,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10283,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10377,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'10440","reported_seq":10570,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:32.668694+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:02:32.668694+0000","last_peered":"2024-09-10T16:02:32.668694+0000","last_clean":"2024-09-10T16:02:32.668694+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:02:32.668694+0000","last_undegraded":"2024-09-10T16:02:32.668694+0000","last_fullsized":"2024-09-10T16:02:32.668694+0000","mapping_epoch":34,"log_start":"36'400","ondisk_log_start":"36'400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42734051,"num_objects":10434,"num_object_clones":0,"num_object_copies":20868,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10434,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7151,"num_write_kb":28604,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'10528","reported_seq":10665,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:32.690815+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:02:32.690815+0000","last_peered":"2024-09-10T16:02:32.690815+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:02:32.690815+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"36'500","ondisk_log_start":"36'500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":43102691,"num_objects":10524,"num_object_clones":0,"num_object_copies":21048,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10524,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7140,"num_write_kb":28560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6153},{"shards":"0,2","objects":4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'10540","reported_seq":10677,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:32.566072+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:02:32.566072+0000","last_peered":"2024-09-10T16:02:32.566072+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:02:32.566072+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"36'500","ondisk_log_start":"36'500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":43148230,"num_objects":10536,"num_object_clones":0,"num_object_copies":21072,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10536,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":7039,"num_write_kb":28156,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6299},{"shards":"0,2","objects":4237}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'10653","reported_seq":10783,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:32.635272+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:02:32.635272+0000","last_peered":"2024-09-10T16:02:32.635272+0000","last_clean":"2024-09-10T16:02:32.635272+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:02:32.635272+0000","last_undegraded":"2024-09-10T16:02:32.635272+0000","last_fullsized":"2024-09-10T16:02:32.635272+0000","mapping_epoch":34,"log_start":"36'600","ondisk_log_start":"36'600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":43626496,"num_objects":10651,"num_object_clones":0,"num_object_copies":21302,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10651,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7500,"num_write_kb":30000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104219+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:02:08.104219+0000","last_peered":"2024-09-10T16:02:08.104219+0000","last_clean":"2024-09-10T16:02:08.104219+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:02:08.104219+0000","last_undegraded":"2024-09-10T16:02:08.104219+0000","last_fullsized":"2024-09-10T16:02:08.104219+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'10291","reported_seq":10430,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:34.859452+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:02:34.859452+0000","last_peered":"2024-09-10T16:02:34.859452+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:02:34.859452+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"36'200","ondisk_log_start":"36'200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41640419,"num_objects":10167,"num_object_clones":0,"num_object_copies":20334,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10167,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8110,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4064},{"shards":"2","objects":6044}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103774+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:02:08.103774+0000","last_peered":"2024-09-10T16:02:08.103774+0000","last_clean":"2024-09-10T16:02:08.103774+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:02:08.103774+0000","last_undegraded":"2024-09-10T16:02:08.103774+0000","last_fullsized":"2024-09-10T16:02:08.103774+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'10448","reported_seq":10578,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:32.684612+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:02:32.684612+0000","last_peered":"2024-09-10T16:02:32.684612+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:02:32.684612+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"36'400","ondisk_log_start":"36'400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42762240,"num_objects":10440,"num_object_clones":0,"num_object_copies":20880,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10440,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6952,"num_write_kb":27808,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6185},{"shards":"0,2","objects":4255}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":104,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.105055+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:02:08.105055+0000","last_peered":"2024-09-10T16:02:08.105055+0000","last_clean":"2024-09-10T16:02:08.105055+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:02:08.105055+0000","last_undegraded":"2024-09-10T16:02:08.105055+0000","last_fullsized":"2024-09-10T16:02:08.105055+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'10444","reported_seq":10576,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:34.793037+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:02:34.793037+0000","last_peered":"2024-09-10T16:02:34.793037+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:02:34.793037+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"36'400","ondisk_log_start":"36'400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42340835,"num_objects":10338,"num_object_clones":0,"num_object_copies":20676,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10338,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8124,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4253},{"shards":"2","objects":6034}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104572+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:02:08.104572+0000","last_peered":"2024-09-10T16:02:08.104572+0000","last_clean":"2024-09-10T16:02:08.104572+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:02:08.104572+0000","last_undegraded":"2024-09-10T16:02:08.104572+0000","last_fullsized":"2024-09-10T16:02:08.104572+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'10641","reported_seq":10770,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:34.848292+0000","last_change":"2024-09-10T15:59:16.174113+0000","last_active":"2024-09-10T16:02:34.848292+0000","last_peered":"2024-09-10T16:02:34.848292+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:02:34.848292+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"36'600","ondisk_log_start":"36'600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":43126784,"num_objects":10529,"num_object_clones":0,"num_object_copies":21058,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4354,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10529,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8195,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4299},{"shards":"2","objects":6177}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'10414","reported_seq":10548,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:32.592322+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:02:32.592322+0000","last_peered":"2024-09-10T16:02:32.592322+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:02:32.592322+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"36'400","ondisk_log_start":"36'400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42647552,"num_objects":10412,"num_object_clones":0,"num_object_copies":20824,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10412,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7045,"num_write_kb":28180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6108},{"shards":"0,2","objects":4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103276+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:02:08.103276+0000","last_peered":"2024-09-10T16:02:08.103276+0000","last_clean":"2024-09-10T16:02:08.103276+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:02:08.103276+0000","last_undegraded":"2024-09-10T16:02:08.103276+0000","last_fullsized":"2024-09-10T16:02:08.103276+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'10296","reported_seq":11366,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T16:02:32.667526+0000","last_change":"2024-09-10T16:02:06.480033+0000","last_active":"2024-09-10T16:02:32.667526+0000","last_peered":"2024-09-10T16:02:32.667526+0000","last_clean":"2024-09-10T15:59:14.115595+0000","last_became_active":"2024-09-10T15:59:16.244089+0000","last_became_peered":"2024-09-10T15:59:16.244089+0000","last_unstale":"2024-09-10T16:02:32.667526+0000","last_undegraded":"2024-09-10T16:02:32.667526+0000","last_fullsized":"2024-09-10T15:59:15.774437+0000","mapping_epoch":42,"log_start":"36'200","ondisk_log_start":"36'200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42139648,"num_objects":10288,"num_object_clones":0,"num_object_copies":20576,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3740,"num_objects_unfound":0,"num_objects_dirty":10288,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6686,"num_write_kb":26744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":463,"num_bytes_recovered":1896448,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6548},{"shards":"0,2","objects":3740}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103831+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:02:08.103831+0000","last_peered":"2024-09-10T16:02:08.103831+0000","last_clean":"2024-09-10T16:02:08.103831+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:02:08.103831+0000","last_undegraded":"2024-09-10T16:02:08.103831+0000","last_fullsized":"2024-09-10T16:02:08.103831+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'10513","reported_seq":10655,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:32.557510+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:02:32.557510+0000","last_peered":"2024-09-10T16:02:32.557510+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:02:32.557510+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"36'500","ondisk_log_start":"36'500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":43044864,"num_objects":10509,"num_object_clones":0,"num_object_copies":21018,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10509,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6768,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6138},{"shards":"0,2","objects":4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104431+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:02:08.104431+0000","last_peered":"2024-09-10T16:02:08.104431+0000","last_clean":"2024-09-10T16:02:08.104431+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:02:08.104431+0000","last_undegraded":"2024-09-10T16:02:08.104431+0000","last_fullsized":"2024-09-10T16:02:08.104431+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104410+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:02:08.104410+0000","last_peered":"2024-09-10T16:02:08.104410+0000","last_clean":"2024-09-10T16:02:08.104410+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:02:08.104410+0000","last_undegraded":"2024-09-10T16:02:08.104410+0000","last_fullsized":"2024-09-10T16:02:08.104410+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'10598","reported_seq":10742,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:32.589495+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:02:32.589495+0000","last_peered":"2024-09-10T16:02:32.589495+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:02:32.589495+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"36'500","ondisk_log_start":"36'500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":43361222,"num_objects":10588,"num_object_clones":0,"num_object_copies":21176,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10588,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6761,"num_write_kb":27044,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6265},{"shards":"0,2","objects":4323}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":682873590,"num_objects":166727,"num_object_clones":0,"num_object_copies":333454,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42911,"num_objects_misplaced":3740,"num_objects_unfound":0,"num_objects_dirty":166727,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":125237,"num_write_kb":499617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4735,"num_bytes_recovered":19390464,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1386192896,"data_stored":1386033006,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160863,"ondisk_log_size":160863,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986231,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":763252,"kb_used_data":470564,"kb_used_omap":4,"kb_used_meta":292667,"kb_avail":93608588,"statfs":{"total":96636764160,"available":95855194112,"internally_reserved":0,"allocated":481857536,"data_stored":481565303,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":299691690},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,1,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659177,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":526972,"kb_used_data":314796,"kb_used_omap":4,"kb_used_meta":212155,"kb_avail":93844868,"statfs":{"total":96636764160,"available":96097144832,"internally_reserved":0,"allocated":322351104,"data_stored":322011438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":217247406},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,1,3,5,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182459,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":912352,"kb_used_data":569744,"kb_used_omap":4,"kb_used_meta":342587,"kb_avail":93459488,"statfs":{"total":96636764160,"available":95702515712,"internally_reserved":0,"allocated":583417856,"data_stored":583125618,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":350809770},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,3,5,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":583692288,"data_stored":583656158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":321814528,"data_stored":321730970,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":480686080,"data_stored":480645878,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:02:42.049 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:02:42.631 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:02:42.633 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:02:42.659 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":201,"stamp":"2024-09-10T16:02:41.618793+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":674497289,"num_objects":164684,"num_object_clones":0,"num_object_copies":329368,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42911,"num_objects_misplaced":3657,"num_objects_unfound":0,"num_objects_dirty":164684,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":127426,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4820,"num_bytes_recovered":19730451,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160853,"ondisk_log_size":160853,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2206820,"kb_used_data":1345444,"kb_used_omap":12,"kb_used_meta":861299,"kb_avail":280908700,"statfs":{"total":289910292480,"available":287650508800,"internally_reserved":0,"allocated":1377734656,"data_stored":1376810519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":881970178},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,3,4,6,7,17],"upper_bound":128},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1052696,"num_objects":258,"num_object_clones":0,"num_object_copies":516,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-172,"num_objects_unfound":0,"num_objects_dirty":258,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5154,"num_write_kb":10821,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":172,"num_bytes_recovered":704512,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004702"},"pg_stats":[{"pgid":"2.f","version":"45'10455","reported_seq":10530,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:37.744103+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:02:37.744103+0000","last_peered":"2024-09-10T16:02:37.744103+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:02:37.744103+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"36'400","ondisk_log_start":"36'400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41848832,"num_objects":10217,"num_object_clones":0,"num_object_copies":20434,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10217,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10455,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5958},{"shards":"0,2","objects":4139}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'10611","reported_seq":10723,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:39.880395+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:02:39.880395+0000","last_peered":"2024-09-10T16:02:39.880395+0000","last_clean":"2024-09-10T16:02:39.880395+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:02:39.880395+0000","last_undegraded":"2024-09-10T16:02:39.880395+0000","last_fullsized":"2024-09-10T16:02:39.880395+0000","mapping_epoch":35,"log_start":"36'600","ondisk_log_start":"36'600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41915334,"num_objects":10235,"num_object_clones":0,"num_object_copies":20470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10235,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":8716,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'10615","reported_seq":19293,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:40.348601+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:02:40.348601+0000","last_peered":"2024-09-10T16:02:40.348601+0000","last_clean":"2024-09-10T16:02:40.348601+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:02:40.348601+0000","last_undegraded":"2024-09-10T16:02:40.348601+0000","last_fullsized":"2024-09-10T16:02:40.348601+0000","mapping_epoch":44,"log_start":"36'600","ondisk_log_start":"36'600","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41705496,"num_objects":10183,"num_object_clones":0,"num_object_copies":20366,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10183,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8619,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'10515","reported_seq":10595,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:39.674237+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:02:39.674237+0000","last_peered":"2024-09-10T16:02:39.674237+0000","last_clean":"2024-09-10T16:02:39.674237+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:02:39.674237+0000","last_undegraded":"2024-09-10T16:02:39.674237+0000","last_fullsized":"2024-09-10T16:02:39.674237+0000","mapping_epoch":35,"log_start":"36'500","ondisk_log_start":"36'500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41553920,"num_objects":10145,"num_object_clones":0,"num_object_copies":20290,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10145,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10515,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'10568","reported_seq":10698,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:37.667290+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:02:37.667290+0000","last_peered":"2024-09-10T16:02:37.667290+0000","last_clean":"2024-09-10T16:02:37.667290+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:02:37.667290+0000","last_undegraded":"2024-09-10T16:02:37.667290+0000","last_fullsized":"2024-09-10T16:02:37.667290+0000","mapping_epoch":34,"log_start":"36'500","ondisk_log_start":"36'500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42283491,"num_objects":10324,"num_object_clones":0,"num_object_copies":20648,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10324,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7279,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'10660","reported_seq":10796,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:37.718383+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:02:37.718383+0000","last_peered":"2024-09-10T16:02:37.718383+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:02:37.718383+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"36'600","ondisk_log_start":"36'600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42602979,"num_objects":10402,"num_object_clones":0,"num_object_copies":20804,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10402,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7272,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6031},{"shards":"0,2","objects":4244}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'10683","reported_seq":10821,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:37.657682+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:02:37.657682+0000","last_peered":"2024-09-10T16:02:37.657682+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:02:37.657682+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"36'600","ondisk_log_start":"36'600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42595270,"num_objects":10401,"num_object_clones":0,"num_object_copies":20802,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10401,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":7182,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6164},{"shards":"0,2","objects":4098}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'10788","reported_seq":10918,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:37.683553+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:02:37.683553+0000","last_peered":"2024-09-10T16:02:37.683553+0000","last_clean":"2024-09-10T16:02:37.683553+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:02:37.683553+0000","last_undegraded":"2024-09-10T16:02:37.683553+0000","last_fullsized":"2024-09-10T16:02:37.683553+0000","mapping_epoch":34,"log_start":"36'700","ondisk_log_start":"36'700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":43114496,"num_objects":10526,"num_object_clones":0,"num_object_copies":21052,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10526,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7635,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104219+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:02:08.104219+0000","last_peered":"2024-09-10T16:02:08.104219+0000","last_clean":"2024-09-10T16:02:08.104219+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:02:08.104219+0000","last_undegraded":"2024-09-10T16:02:08.104219+0000","last_fullsized":"2024-09-10T16:02:08.104219+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'10416","reported_seq":10555,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:39.903180+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:02:39.903180+0000","last_peered":"2024-09-10T16:02:39.903180+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:02:39.903180+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"36'400","ondisk_log_start":"36'400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41128419,"num_objects":10042,"num_object_clones":0,"num_object_copies":20084,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10042,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8235,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3939},{"shards":"2","objects":5919}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103774+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:02:08.103774+0000","last_peered":"2024-09-10T16:02:08.103774+0000","last_clean":"2024-09-10T16:02:08.103774+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:02:08.103774+0000","last_undegraded":"2024-09-10T16:02:08.103774+0000","last_fullsized":"2024-09-10T16:02:08.103774+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'10577","reported_seq":10705,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:37.590175+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:02:37.590175+0000","last_peered":"2024-09-10T16:02:37.590175+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:02:37.590175+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"36'500","ondisk_log_start":"36'500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42323968,"num_objects":10333,"num_object_clones":0,"num_object_copies":20666,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10333,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7081,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6078},{"shards":"0,2","objects":4137}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":104,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.105055+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:02:08.105055+0000","last_peered":"2024-09-10T16:02:08.105055+0000","last_clean":"2024-09-10T16:02:08.105055+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:02:08.105055+0000","last_undegraded":"2024-09-10T16:02:08.105055+0000","last_fullsized":"2024-09-10T16:02:08.105055+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'10576","reported_seq":10711,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:39.870991+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:02:39.870991+0000","last_peered":"2024-09-10T16:02:39.870991+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:02:39.870991+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"36'500","ondisk_log_start":"36'500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41800163,"num_objects":10206,"num_object_clones":0,"num_object_copies":20412,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10206,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8256,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4121},{"shards":"2","objects":5902}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104572+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:02:08.104572+0000","last_peered":"2024-09-10T16:02:08.104572+0000","last_clean":"2024-09-10T16:02:08.104572+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:02:08.104572+0000","last_undegraded":"2024-09-10T16:02:08.104572+0000","last_fullsized":"2024-09-10T16:02:08.104572+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'10785","reported_seq":10914,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:39.870095+0000","last_change":"2024-09-10T15:59:16.174113+0000","last_active":"2024-09-10T16:02:39.870095+0000","last_peered":"2024-09-10T16:02:39.870095+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:02:39.870095+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"36'700","ondisk_log_start":"36'700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42536960,"num_objects":10385,"num_object_clones":0,"num_object_copies":20770,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4354,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10385,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8339,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4155},{"shards":"2","objects":6033}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'10580","reported_seq":10714,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:37.696622+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:02:37.696622+0000","last_peered":"2024-09-10T16:02:37.696622+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:02:37.696622+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"36'500","ondisk_log_start":"36'500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42016768,"num_objects":10258,"num_object_clones":0,"num_object_copies":20516,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10258,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7211,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5954},{"shards":"0,2","objects":4144}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103276+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:02:08.103276+0000","last_peered":"2024-09-10T16:02:08.103276+0000","last_clean":"2024-09-10T16:02:08.103276+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:02:08.103276+0000","last_undegraded":"2024-09-10T16:02:08.103276+0000","last_fullsized":"2024-09-10T16:02:08.103276+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'10426","reported_seq":11662,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T16:02:37.728029+0000","last_change":"2024-09-10T16:02:06.480033+0000","last_active":"2024-09-10T16:02:37.728029+0000","last_peered":"2024-09-10T16:02:37.728029+0000","last_clean":"2024-09-10T15:59:14.115595+0000","last_became_active":"2024-09-10T15:59:16.244089+0000","last_became_peered":"2024-09-10T15:59:16.244089+0000","last_unstale":"2024-09-10T16:02:37.728029+0000","last_undegraded":"2024-09-10T16:02:37.728029+0000","last_fullsized":"2024-09-10T15:59:15.774437+0000","mapping_epoch":42,"log_start":"36'400","ondisk_log_start":"36'400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41664512,"num_objects":10172,"num_object_clones":0,"num_object_copies":20344,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3657,"num_objects_unfound":0,"num_objects_dirty":10172,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6816,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":546,"num_bytes_recovered":2236416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6515},{"shards":"0,2","objects":3657}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103831+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:02:08.103831+0000","last_peered":"2024-09-10T16:02:08.103831+0000","last_clean":"2024-09-10T16:02:08.103831+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:02:08.103831+0000","last_undegraded":"2024-09-10T16:02:08.103831+0000","last_fullsized":"2024-09-10T16:02:08.103831+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'10672","reported_seq":10814,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:37.720424+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:02:37.720424+0000","last_peered":"2024-09-10T16:02:37.720424+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:02:37.720424+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"36'600","ondisk_log_start":"36'600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42500096,"num_objects":10376,"num_object_clones":0,"num_object_copies":20752,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10376,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6927,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6005},{"shards":"0,2","objects":4225}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104431+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:02:08.104431+0000","last_peered":"2024-09-10T16:02:08.104431+0000","last_clean":"2024-09-10T16:02:08.104431+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:02:08.104431+0000","last_undegraded":"2024-09-10T16:02:08.104431+0000","last_fullsized":"2024-09-10T16:02:08.104431+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104410+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:02:08.104410+0000","last_peered":"2024-09-10T16:02:08.104410+0000","last_clean":"2024-09-10T16:02:08.104410+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:02:08.104410+0000","last_undegraded":"2024-09-10T16:02:08.104410+0000","last_fullsized":"2024-09-10T16:02:08.104410+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'10723","reported_seq":10867,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:37.723686+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:02:37.723686+0000","last_peered":"2024-09-10T16:02:37.723686+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:02:37.723686+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"36'700","ondisk_log_start":"36'700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42906566,"num_objects":10477,"num_object_clones":0,"num_object_copies":20954,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10477,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6886,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6154},{"shards":"0,2","objects":4205}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":674497270,"num_objects":164682,"num_object_clones":0,"num_object_copies":329364,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42911,"num_objects_misplaced":3657,"num_objects_unfound":0,"num_objects_dirty":164682,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":127424,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4818,"num_bytes_recovered":19730432,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1374748672,"data_stored":1374588782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160850,"ondisk_log_size":160850,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986232,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":746408,"kb_used_data":466200,"kb_used_omap":4,"kb_used_meta":280187,"kb_avail":93625432,"statfs":{"total":96636764160,"available":95872442368,"internally_reserved":0,"allocated":477388800,"data_stored":477096567,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":286912170},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,2,1,0,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659178,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":548400,"kb_used_data":314016,"kb_used_omap":4,"kb_used_meta":234363,"kb_avail":93823440,"statfs":{"total":96636764160,"available":96075202560,"internally_reserved":0,"allocated":321552384,"data_stored":321212718,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":239988398},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,4,1,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182460,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":912012,"kb_used_data":565228,"kb_used_omap":4,"kb_used_meta":346747,"kb_avail":93459828,"statfs":{"total":96636764160,"available":95702863872,"internally_reserved":0,"allocated":578793472,"data_stored":578501234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":355069610},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,1,6,8],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":577568768,"data_stored":577532638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":321003520,"data_stored":320919962,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":476176384,"data_stored":476136182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:02:42.661 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:02:43.221 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:02:43.223 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:02:43.243 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":201,"stamp":"2024-09-10T16:02:41.618793+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":674497289,"num_objects":164684,"num_object_clones":0,"num_object_copies":329368,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42911,"num_objects_misplaced":3657,"num_objects_unfound":0,"num_objects_dirty":164684,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":127426,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4820,"num_bytes_recovered":19730451,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160853,"ondisk_log_size":160853,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2206820,"kb_used_data":1345444,"kb_used_omap":12,"kb_used_meta":861299,"kb_avail":280908700,"statfs":{"total":289910292480,"available":287650508800,"internally_reserved":0,"allocated":1377734656,"data_stored":1376810519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":881970178},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,3,4,6,7,17],"upper_bound":128},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1052696,"num_objects":258,"num_object_clones":0,"num_object_copies":516,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-172,"num_objects_unfound":0,"num_objects_dirty":258,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5154,"num_write_kb":10821,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":172,"num_bytes_recovered":704512,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004702"},"pg_stats":[{"pgid":"2.f","version":"45'10455","reported_seq":10530,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:37.744103+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:02:37.744103+0000","last_peered":"2024-09-10T16:02:37.744103+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:02:37.744103+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"36'400","ondisk_log_start":"36'400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41848832,"num_objects":10217,"num_object_clones":0,"num_object_copies":20434,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10217,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10455,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5958},{"shards":"0,2","objects":4139}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'10611","reported_seq":10723,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:39.880395+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:02:39.880395+0000","last_peered":"2024-09-10T16:02:39.880395+0000","last_clean":"2024-09-10T16:02:39.880395+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:02:39.880395+0000","last_undegraded":"2024-09-10T16:02:39.880395+0000","last_fullsized":"2024-09-10T16:02:39.880395+0000","mapping_epoch":35,"log_start":"36'600","ondisk_log_start":"36'600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41915334,"num_objects":10235,"num_object_clones":0,"num_object_copies":20470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10235,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":8716,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'10615","reported_seq":19293,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:40.348601+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:02:40.348601+0000","last_peered":"2024-09-10T16:02:40.348601+0000","last_clean":"2024-09-10T16:02:40.348601+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:02:40.348601+0000","last_undegraded":"2024-09-10T16:02:40.348601+0000","last_fullsized":"2024-09-10T16:02:40.348601+0000","mapping_epoch":44,"log_start":"36'600","ondisk_log_start":"36'600","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41705496,"num_objects":10183,"num_object_clones":0,"num_object_copies":20366,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10183,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8619,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'10515","reported_seq":10595,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:39.674237+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:02:39.674237+0000","last_peered":"2024-09-10T16:02:39.674237+0000","last_clean":"2024-09-10T16:02:39.674237+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:02:39.674237+0000","last_undegraded":"2024-09-10T16:02:39.674237+0000","last_fullsized":"2024-09-10T16:02:39.674237+0000","mapping_epoch":35,"log_start":"36'500","ondisk_log_start":"36'500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41553920,"num_objects":10145,"num_object_clones":0,"num_object_copies":20290,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10145,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10515,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'10568","reported_seq":10698,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:37.667290+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:02:37.667290+0000","last_peered":"2024-09-10T16:02:37.667290+0000","last_clean":"2024-09-10T16:02:37.667290+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:02:37.667290+0000","last_undegraded":"2024-09-10T16:02:37.667290+0000","last_fullsized":"2024-09-10T16:02:37.667290+0000","mapping_epoch":34,"log_start":"36'500","ondisk_log_start":"36'500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42283491,"num_objects":10324,"num_object_clones":0,"num_object_copies":20648,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10324,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7279,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'10660","reported_seq":10796,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:37.718383+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:02:37.718383+0000","last_peered":"2024-09-10T16:02:37.718383+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:02:37.718383+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"36'600","ondisk_log_start":"36'600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42602979,"num_objects":10402,"num_object_clones":0,"num_object_copies":20804,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10402,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7272,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6031},{"shards":"0,2","objects":4244}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'10683","reported_seq":10821,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:37.657682+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:02:37.657682+0000","last_peered":"2024-09-10T16:02:37.657682+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:02:37.657682+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"36'600","ondisk_log_start":"36'600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42595270,"num_objects":10401,"num_object_clones":0,"num_object_copies":20802,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10401,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":7182,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6164},{"shards":"0,2","objects":4098}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'10788","reported_seq":10918,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:37.683553+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:02:37.683553+0000","last_peered":"2024-09-10T16:02:37.683553+0000","last_clean":"2024-09-10T16:02:37.683553+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:02:37.683553+0000","last_undegraded":"2024-09-10T16:02:37.683553+0000","last_fullsized":"2024-09-10T16:02:37.683553+0000","mapping_epoch":34,"log_start":"36'700","ondisk_log_start":"36'700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":43114496,"num_objects":10526,"num_object_clones":0,"num_object_copies":21052,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10526,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7635,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104219+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:02:08.104219+0000","last_peered":"2024-09-10T16:02:08.104219+0000","last_clean":"2024-09-10T16:02:08.104219+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:02:08.104219+0000","last_undegraded":"2024-09-10T16:02:08.104219+0000","last_fullsized":"2024-09-10T16:02:08.104219+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'10416","reported_seq":10555,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:39.903180+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:02:39.903180+0000","last_peered":"2024-09-10T16:02:39.903180+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:02:39.903180+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"36'400","ondisk_log_start":"36'400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41128419,"num_objects":10042,"num_object_clones":0,"num_object_copies":20084,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10042,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8235,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3939},{"shards":"2","objects":5919}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103774+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:02:08.103774+0000","last_peered":"2024-09-10T16:02:08.103774+0000","last_clean":"2024-09-10T16:02:08.103774+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:02:08.103774+0000","last_undegraded":"2024-09-10T16:02:08.103774+0000","last_fullsized":"2024-09-10T16:02:08.103774+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'10577","reported_seq":10705,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:37.590175+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:02:37.590175+0000","last_peered":"2024-09-10T16:02:37.590175+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:02:37.590175+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"36'500","ondisk_log_start":"36'500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42323968,"num_objects":10333,"num_object_clones":0,"num_object_copies":20666,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10333,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7081,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6078},{"shards":"0,2","objects":4137}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":104,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.105055+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:02:08.105055+0000","last_peered":"2024-09-10T16:02:08.105055+0000","last_clean":"2024-09-10T16:02:08.105055+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:02:08.105055+0000","last_undegraded":"2024-09-10T16:02:08.105055+0000","last_fullsized":"2024-09-10T16:02:08.105055+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'10576","reported_seq":10711,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:39.870991+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:02:39.870991+0000","last_peered":"2024-09-10T16:02:39.870991+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:02:39.870991+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"36'500","ondisk_log_start":"36'500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41800163,"num_objects":10206,"num_object_clones":0,"num_object_copies":20412,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10206,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8256,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4121},{"shards":"2","objects":5902}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104572+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:02:08.104572+0000","last_peered":"2024-09-10T16:02:08.104572+0000","last_clean":"2024-09-10T16:02:08.104572+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:02:08.104572+0000","last_undegraded":"2024-09-10T16:02:08.104572+0000","last_fullsized":"2024-09-10T16:02:08.104572+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'10785","reported_seq":10914,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:39.870095+0000","last_change":"2024-09-10T15:59:16.174113+0000","last_active":"2024-09-10T16:02:39.870095+0000","last_peered":"2024-09-10T16:02:39.870095+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:02:39.870095+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"36'700","ondisk_log_start":"36'700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42536960,"num_objects":10385,"num_object_clones":0,"num_object_copies":20770,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4354,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10385,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8339,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4155},{"shards":"2","objects":6033}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'10580","reported_seq":10714,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:37.696622+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:02:37.696622+0000","last_peered":"2024-09-10T16:02:37.696622+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:02:37.696622+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"36'500","ondisk_log_start":"36'500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42016768,"num_objects":10258,"num_object_clones":0,"num_object_copies":20516,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10258,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7211,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5954},{"shards":"0,2","objects":4144}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103276+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:02:08.103276+0000","last_peered":"2024-09-10T16:02:08.103276+0000","last_clean":"2024-09-10T16:02:08.103276+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:02:08.103276+0000","last_undegraded":"2024-09-10T16:02:08.103276+0000","last_fullsized":"2024-09-10T16:02:08.103276+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'10426","reported_seq":11662,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T16:02:37.728029+0000","last_change":"2024-09-10T16:02:06.480033+0000","last_active":"2024-09-10T16:02:37.728029+0000","last_peered":"2024-09-10T16:02:37.728029+0000","last_clean":"2024-09-10T15:59:14.115595+0000","last_became_active":"2024-09-10T15:59:16.244089+0000","last_became_peered":"2024-09-10T15:59:16.244089+0000","last_unstale":"2024-09-10T16:02:37.728029+0000","last_undegraded":"2024-09-10T16:02:37.728029+0000","last_fullsized":"2024-09-10T15:59:15.774437+0000","mapping_epoch":42,"log_start":"36'400","ondisk_log_start":"36'400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41664512,"num_objects":10172,"num_object_clones":0,"num_object_copies":20344,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3657,"num_objects_unfound":0,"num_objects_dirty":10172,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6816,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":546,"num_bytes_recovered":2236416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6515},{"shards":"0,2","objects":3657}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103831+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:02:08.103831+0000","last_peered":"2024-09-10T16:02:08.103831+0000","last_clean":"2024-09-10T16:02:08.103831+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:02:08.103831+0000","last_undegraded":"2024-09-10T16:02:08.103831+0000","last_fullsized":"2024-09-10T16:02:08.103831+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'10672","reported_seq":10814,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:37.720424+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:02:37.720424+0000","last_peered":"2024-09-10T16:02:37.720424+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:02:37.720424+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"36'600","ondisk_log_start":"36'600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42500096,"num_objects":10376,"num_object_clones":0,"num_object_copies":20752,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10376,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6927,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6005},{"shards":"0,2","objects":4225}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104431+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:02:08.104431+0000","last_peered":"2024-09-10T16:02:08.104431+0000","last_clean":"2024-09-10T16:02:08.104431+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:02:08.104431+0000","last_undegraded":"2024-09-10T16:02:08.104431+0000","last_fullsized":"2024-09-10T16:02:08.104431+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104410+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:02:08.104410+0000","last_peered":"2024-09-10T16:02:08.104410+0000","last_clean":"2024-09-10T16:02:08.104410+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:02:08.104410+0000","last_undegraded":"2024-09-10T16:02:08.104410+0000","last_fullsized":"2024-09-10T16:02:08.104410+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'10723","reported_seq":10867,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:37.723686+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:02:37.723686+0000","last_peered":"2024-09-10T16:02:37.723686+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:02:37.723686+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"36'700","ondisk_log_start":"36'700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42906566,"num_objects":10477,"num_object_clones":0,"num_object_copies":20954,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10477,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6886,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6154},{"shards":"0,2","objects":4205}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":674497270,"num_objects":164682,"num_object_clones":0,"num_object_copies":329364,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42911,"num_objects_misplaced":3657,"num_objects_unfound":0,"num_objects_dirty":164682,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":127424,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4818,"num_bytes_recovered":19730432,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1374748672,"data_stored":1374588782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160850,"ondisk_log_size":160850,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986232,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":746408,"kb_used_data":466200,"kb_used_omap":4,"kb_used_meta":280187,"kb_avail":93625432,"statfs":{"total":96636764160,"available":95872442368,"internally_reserved":0,"allocated":477388800,"data_stored":477096567,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":286912170},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,2,1,0,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659178,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":548400,"kb_used_data":314016,"kb_used_omap":4,"kb_used_meta":234363,"kb_avail":93823440,"statfs":{"total":96636764160,"available":96075202560,"internally_reserved":0,"allocated":321552384,"data_stored":321212718,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":239988398},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,4,1,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182460,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":912012,"kb_used_data":565228,"kb_used_omap":4,"kb_used_meta":346747,"kb_avail":93459828,"statfs":{"total":96636764160,"available":95702863872,"internally_reserved":0,"allocated":578793472,"data_stored":578501234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":355069610},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,1,6,8],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":577568768,"data_stored":577532638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":321003520,"data_stored":320919962,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":476176384,"data_stored":476136182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:02:43.244 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-10T16:02:43.244 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "45'10455", 'reported_seq': 10530, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:02:37.744103+0000', 'last_change': '2024-09-10T15:59:16.281558+0000', 'last_active': '2024-09-10T16:02:37.744103+0000', 'last_peered': '2024-09-10T16:02:37.744103+0000', 'last_clean': '2024-09-10T15:59:14.132456+0000', 'last_became_active': '2024-09-10T15:59:16.274407+0000', 'last_became_peered': '2024-09-10T15:59:16.274407+0000', 'last_unstale': '2024-09-10T16:02:37.744103+0000', 'last_undegraded': '2024-09-10T15:59:15.851279+0000', 'last_fullsized': '2024-09-10T15:59:15.774356+0000', 'mapping_epoch': 42, 'log_start': "36'400", 'ondisk_log_start': "36'400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'objects_scrubbed': 0, 'log_size': 10055, 'log_dups_size': 0, 'ondisk_log_size': 10055, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 41848832, 'num_objects': 10217, 'num_object_clones': 0, 'num_object_copies': 20434, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4259, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10217, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10455, 'num_write_kb': 41344, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5958}, {'shards': '0,2', 'objects': 4139}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:02:43.244 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-10T16:02:43.244 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "45'10660", 'reported_seq': 10796, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:02:37.718383+0000', 'last_change': '2024-09-10T15:59:16.308264+0000', 'last_active': '2024-09-10T16:02:37.718383+0000', 'last_peered': '2024-09-10T16:02:37.718383+0000', 'last_clean': '2024-09-10T15:59:14.111551+0000', 'last_became_active': '2024-09-10T15:59:16.300040+0000', 'last_became_peered': '2024-09-10T15:59:16.300040+0000', 'last_unstale': '2024-09-10T16:02:37.718383+0000', 'last_undegraded': '2024-09-10T15:59:15.878234+0000', 'last_fullsized': '2024-09-10T15:59:15.769328+0000', 'mapping_epoch': 42, 'log_start': "36'600", 'ondisk_log_start': "36'600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3442", 'last_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'objects_scrubbed': 3384, 'log_size': 10060, 'log_dups_size': 0, 'ondisk_log_size': 10060, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1292, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 42602979, 'num_objects': 10402, 'num_object_clones': 0, 'num_object_copies': 20804, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4372, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10402, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 7272, 'num_write_kb': 28580, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6031}, {'shards': '0,2', 'objects': 4244}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:02:43.244 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-10T16:02:43.244 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "45'10683", 'reported_seq': 10821, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:02:37.657682+0000', 'last_change': '2024-09-10T15:59:16.331899+0000', 'last_active': '2024-09-10T16:02:37.657682+0000', 'last_peered': '2024-09-10T16:02:37.657682+0000', 'last_clean': '2024-09-10T15:59:14.139436+0000', 'last_became_active': '2024-09-10T15:59:16.318715+0000', 'last_became_peered': '2024-09-10T15:59:16.318715+0000', 'last_unstale': '2024-09-10T16:02:37.657682+0000', 'last_undegraded': '2024-09-10T15:59:15.877524+0000', 'last_fullsized': '2024-09-10T15:59:15.769042+0000', 'mapping_epoch': 42, 'log_start': "36'600", 'ondisk_log_start': "36'600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3537", 'last_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'objects_scrubbed': 3497, 'log_size': 10083, 'log_dups_size': 0, 'ondisk_log_size': 10083, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1224, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 42595270, 'num_objects': 10401, 'num_object_clones': 0, 'num_object_copies': 20802, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4237, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10401, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 7182, 'num_write_kb': 28172, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6164}, {'shards': '0,2', 'objects': 4098}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:02:43.245 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-10T16:02:43.245 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "45'10416", 'reported_seq': 10555, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:02:39.903180+0000', 'last_change': '2024-09-10T15:59:16.175680+0000', 'last_active': '2024-09-10T16:02:39.903180+0000', 'last_peered': '2024-09-10T16:02:39.903180+0000', 'last_clean': '2024-09-10T15:59:14.088187+0000', 'last_became_active': '2024-09-10T15:59:16.165810+0000', 'last_became_peered': '2024-09-10T15:59:16.165810+0000', 'last_unstale': '2024-09-10T16:02:39.903180+0000', 'last_undegraded': '2024-09-10T15:59:15.822098+0000', 'last_fullsized': '2024-09-10T15:59:15.769123+0000', 'mapping_epoch': 42, 'log_start': "36'400", 'ondisk_log_start': "36'400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2218", 'last_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'objects_scrubbed': 2175, 'log_size': 10016, 'log_dups_size': 0, 'ondisk_log_size': 10016, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 825, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 41128419, 'num_objects': 10042, 'num_object_clones': 0, 'num_object_copies': 20084, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4124, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10042, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 8235, 'num_write_kb': 32204, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3939}, {'shards': '2', 'objects': 5919}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:02:43.245 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-10T16:02:43.245 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "45'10577", 'reported_seq': 10705, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:02:37.590175+0000', 'last_change': '2024-09-10T15:59:16.277742+0000', 'last_active': '2024-09-10T16:02:37.590175+0000', 'last_peered': '2024-09-10T16:02:37.590175+0000', 'last_clean': '2024-09-10T15:59:14.129081+0000', 'last_became_active': '2024-09-10T15:59:16.269835+0000', 'last_became_peered': '2024-09-10T15:59:16.269835+0000', 'last_unstale': '2024-09-10T16:02:37.590175+0000', 'last_undegraded': '2024-09-10T15:59:15.860857+0000', 'last_fullsized': '2024-09-10T15:59:15.771410+0000', 'mapping_epoch': 42, 'log_start': "36'500", 'ondisk_log_start': "36'500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3557", 'last_scrub_stamp': '2024-09-10T15:59:02.493678+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:02.493678+0000', 'objects_scrubbed': 3488, 'log_size': 10077, 'log_dups_size': 0, 'ondisk_log_size': 10077, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1472, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 42323968, 'num_objects': 10333, 'num_object_clones': 0, 'num_object_copies': 20666, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4257, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10333, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7081, 'num_write_kb': 27852, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6078}, {'shards': '0,2', 'objects': 4137}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:02:43.245 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-10T16:02:43.245 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "45'10576", 'reported_seq': 10711, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:02:39.870991+0000', 'last_change': '2024-09-10T15:59:16.284311+0000', 'last_active': '2024-09-10T16:02:39.870991+0000', 'last_peered': '2024-09-10T16:02:39.870991+0000', 'last_clean': '2024-09-10T15:59:14.116041+0000', 'last_became_active': '2024-09-10T15:59:16.274543+0000', 'last_became_peered': '2024-09-10T15:59:16.274543+0000', 'last_unstale': '2024-09-10T16:02:39.870991+0000', 'last_undegraded': '2024-09-10T15:59:15.812484+0000', 'last_fullsized': '2024-09-10T15:59:15.770885+0000', 'mapping_epoch': 42, 'log_start': "36'500", 'ondisk_log_start': "36'500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2356", 'last_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'objects_scrubbed': 2316, 'log_size': 10076, 'log_dups_size': 0, 'ondisk_log_size': 10076, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 783, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 41800163, 'num_objects': 10206, 'num_object_clones': 0, 'num_object_copies': 20412, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10206, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 8256, 'num_write_kb': 32292, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4121}, {'shards': '2', 'objects': 5902}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:02:43.245 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-10T16:02:43.245 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "45'10785", 'reported_seq': 10914, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:02:39.870095+0000', 'last_change': '2024-09-10T15:59:16.174113+0000', 'last_active': '2024-09-10T16:02:39.870095+0000', 'last_peered': '2024-09-10T16:02:39.870095+0000', 'last_clean': '2024-09-10T15:59:14.110654+0000', 'last_became_active': '2024-09-10T15:59:16.164645+0000', 'last_became_peered': '2024-09-10T15:59:16.164645+0000', 'last_unstale': '2024-09-10T16:02:39.870095+0000', 'last_undegraded': '2024-09-10T15:59:15.803593+0000', 'last_fullsized': '2024-09-10T15:59:15.769083+0000', 'mapping_epoch': 42, 'log_start': "36'700", 'ondisk_log_start': "36'700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2486", 'last_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'objects_scrubbed': 2440, 'log_size': 10085, 'log_dups_size': 0, 'ondisk_log_size': 10085, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 792, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 42536960, 'num_objects': 10385, 'num_object_clones': 0, 'num_object_copies': 20770, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4354, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10385, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8339, 'num_write_kb': 32568, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4155}, {'shards': '2', 'objects': 6033}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:02:43.245 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-10T16:02:43.246 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "45'10580", 'reported_seq': 10714, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:02:37.696622+0000', 'last_change': '2024-09-10T15:59:16.315641+0000', 'last_active': '2024-09-10T16:02:37.696622+0000', 'last_peered': '2024-09-10T16:02:37.696622+0000', 'last_clean': '2024-09-10T15:59:14.117001+0000', 'last_became_active': '2024-09-10T15:59:16.302245+0000', 'last_became_peered': '2024-09-10T15:59:16.302245+0000', 'last_unstale': '2024-09-10T16:02:37.696622+0000', 'last_undegraded': '2024-09-10T15:59:15.845012+0000', 'last_fullsized': '2024-09-10T15:59:15.770337+0000', 'mapping_epoch': 42, 'log_start': "36'500", 'ondisk_log_start': "36'500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3404", 'last_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'objects_scrubbed': 3367, 'log_size': 10080, 'log_dups_size': 0, 'ondisk_log_size': 10080, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 983, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 42016768, 'num_objects': 10258, 'num_object_clones': 0, 'num_object_copies': 20516, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10258, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7211, 'num_write_kb': 28204, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5954}, {'shards': '0,2', 'objects': 4144}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:02:43.246 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-10T16:02:43.246 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "45'10426", 'reported_seq': 11662, 'reported_epoch': 45, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-10T16:02:37.728029+0000', 'last_change': '2024-09-10T16:02:06.480033+0000', 'last_active': '2024-09-10T16:02:37.728029+0000', 'last_peered': '2024-09-10T16:02:37.728029+0000', 'last_clean': '2024-09-10T15:59:14.115595+0000', 'last_became_active': '2024-09-10T15:59:16.244089+0000', 'last_became_peered': '2024-09-10T15:59:16.244089+0000', 'last_unstale': '2024-09-10T16:02:37.728029+0000', 'last_undegraded': '2024-09-10T16:02:37.728029+0000', 'last_fullsized': '2024-09-10T15:59:15.774437+0000', 'mapping_epoch': 42, 'log_start': "36'400", 'ondisk_log_start': "36'400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3653", 'last_scrub_stamp': '2024-09-10T15:59:05.298468+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:05.298468+0000', 'objects_scrubbed': 3602, 'log_size': 10026, 'log_dups_size': 0, 'ondisk_log_size': 10026, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1289, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 41664512, 'num_objects': 10172, 'num_object_clones': 0, 'num_object_copies': 20344, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 3657, 'num_objects_unfound': 0, 'num_objects_dirty': 10172, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6816, 'num_write_kb': 26772, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 546, 'num_bytes_recovered': 2236416, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6515}, {'shards': '0,2', 'objects': 3657}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:02:43.246 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-10T16:02:43.246 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "45'10672", 'reported_seq': 10814, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:02:37.720424+0000', 'last_change': '2024-09-10T15:59:16.311291+0000', 'last_active': '2024-09-10T16:02:37.720424+0000', 'last_peered': '2024-09-10T16:02:37.720424+0000', 'last_clean': '2024-09-10T15:59:14.116363+0000', 'last_became_active': '2024-09-10T15:59:16.301244+0000', 'last_became_peered': '2024-09-10T15:59:16.301244+0000', 'last_unstale': '2024-09-10T16:02:37.720424+0000', 'last_undegraded': '2024-09-10T15:59:15.883434+0000', 'last_fullsized': '2024-09-10T15:59:15.772756+0000', 'mapping_epoch': 42, 'log_start': "36'600", 'ondisk_log_start': "36'600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3789", 'last_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'objects_scrubbed': 3741, 'log_size': 10072, 'log_dups_size': 0, 'ondisk_log_size': 10072, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1440, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 42500096, 'num_objects': 10376, 'num_object_clones': 0, 'num_object_copies': 20752, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4373, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10376, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6927, 'num_write_kb': 27124, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6005}, {'shards': '0,2', 'objects': 4225}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:02:43.246 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-10T16:02:43.246 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "45'10723", 'reported_seq': 10867, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:02:37.723686+0000', 'last_change': '2024-09-10T15:59:16.313409+0000', 'last_active': '2024-09-10T16:02:37.723686+0000', 'last_peered': '2024-09-10T16:02:37.723686+0000', 'last_clean': '2024-09-10T15:59:14.129780+0000', 'last_became_active': '2024-09-10T15:59:16.300932+0000', 'last_became_peered': '2024-09-10T15:59:16.300932+0000', 'last_unstale': '2024-09-10T16:02:37.723686+0000', 'last_undegraded': '2024-09-10T15:59:15.887823+0000', 'last_fullsized': '2024-09-10T15:59:15.768513+0000', 'mapping_epoch': 42, 'log_start': "36'700", 'ondisk_log_start': "36'700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3880", 'last_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'objects_scrubbed': 3827, 'log_size': 10023, 'log_dups_size': 0, 'ondisk_log_size': 10023, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1397, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 42906566, 'num_objects': 10477, 'num_object_clones': 0, 'num_object_copies': 20954, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4325, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10477, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6886, 'num_write_kb': 27072, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6154}, {'shards': '0,2', 'objects': 4205}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:02:43.247 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T16:02:43.943 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:02:43.943 INFO:teuthology.orchestra.run.smithi115.stdout:{"fsid":"7effee29-df05-4351-9741-ee203564ed81","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 42911/329368 objects degraded (13.028%), 10 pgs degraded, 11 pgs undersized","count":21},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":327,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983954,"num_in_osds":3,"osd_in_since":1725983947,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":164684,"data_bytes":674497289,"bytes_used":2259783680,"bytes_avail":287650508800,"bytes_total":289910292480,"degraded_objects":42911,"degraded_total":329368,"degraded_ratio":0.13028284472079862,"misplaced_objects":3657,"misplaced_total":329368,"misplaced_ratio":0.011103082266643997,"recovering_objects_per_sec":14,"recovering_bytes_per_sec":58686,"recovering_keys_per_sec":0,"num_objects_recovered":172,"num_bytes_recovered":704512,"num_keys_recovered":0,"read_bytes_sec":85,"write_bytes_sec":923030,"read_op_per_sec":0,"write_op_per_sec":429},"fsmap":{"epoch":1,"btime":"2024-09-10T15:57:14:967120+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T16:00:13.560776+0000","services":{}},"progress_events":{}} 2024-09-10T16:02:43.963 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-10T16:02:43.963 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:02:44.526 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:02:44.526 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:02:44.548 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":202,"stamp":"2024-09-10T16:02:43.619440+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":668766985,"num_objects":163285,"num_object_clones":0,"num_object_copies":326570,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42911,"num_objects_misplaced":3576,"num_objects_unfound":0,"num_objects_dirty":163285,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":128825,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4901,"num_bytes_recovered":20062227,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160552,"ondisk_log_size":160552,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2187632,"kb_used_data":1338748,"kb_used_omap":12,"kb_used_meta":848819,"kb_avail":280927888,"statfs":{"total":289910292480,"available":287670157312,"internally_reserved":0,"allocated":1370877952,"data_stored":1369953815,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":869190658},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,1,3,10,9,12],"upper_bound":128},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-8040424,"num_objects":-1962,"num_object_clones":0,"num_object_copies":-3924,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-253,"num_objects_unfound":0,"num_objects_dirty":-1962,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5732,"num_write_kb":7537,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":253,"num_bytes_recovered":1036288,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004641"},"pg_stats":[{"pgid":"2.f","version":"45'10610","reported_seq":10684,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:42.836730+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:02:42.836730+0000","last_peered":"2024-09-10T16:02:42.836730+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:02:42.836730+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"36'600","ondisk_log_start":"36'600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41213952,"num_objects":10062,"num_object_clones":0,"num_object_copies":20124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10062,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10610,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5803},{"shards":"0,2","objects":3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'10611","reported_seq":10723,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:39.880395+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:02:39.880395+0000","last_peered":"2024-09-10T16:02:39.880395+0000","last_clean":"2024-09-10T16:02:39.880395+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:02:39.880395+0000","last_undegraded":"2024-09-10T16:02:39.880395+0000","last_fullsized":"2024-09-10T16:02:39.880395+0000","mapping_epoch":35,"log_start":"36'600","ondisk_log_start":"36'600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41915334,"num_objects":10235,"num_object_clones":0,"num_object_copies":20470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10235,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":8716,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'10615","reported_seq":19293,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:40.348601+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:02:40.348601+0000","last_peered":"2024-09-10T16:02:40.348601+0000","last_clean":"2024-09-10T16:02:40.348601+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:02:40.348601+0000","last_undegraded":"2024-09-10T16:02:40.348601+0000","last_fullsized":"2024-09-10T16:02:40.348601+0000","mapping_epoch":44,"log_start":"36'600","ondisk_log_start":"36'600","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41705496,"num_objects":10183,"num_object_clones":0,"num_object_copies":20366,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10183,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8619,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'10515","reported_seq":10595,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:39.674237+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:02:39.674237+0000","last_peered":"2024-09-10T16:02:39.674237+0000","last_clean":"2024-09-10T16:02:39.674237+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:02:39.674237+0000","last_undegraded":"2024-09-10T16:02:39.674237+0000","last_fullsized":"2024-09-10T16:02:39.674237+0000","mapping_epoch":35,"log_start":"36'500","ondisk_log_start":"36'500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41553920,"num_objects":10145,"num_object_clones":0,"num_object_copies":20290,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10145,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10515,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'10710","reported_seq":10840,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:42.768767+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:02:42.768767+0000","last_peered":"2024-09-10T16:02:42.768767+0000","last_clean":"2024-09-10T16:02:42.768767+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:02:42.768767+0000","last_undegraded":"2024-09-10T16:02:42.768767+0000","last_fullsized":"2024-09-10T16:02:42.768767+0000","mapping_epoch":34,"log_start":"36'700","ondisk_log_start":"36'700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41701859,"num_objects":10182,"num_object_clones":0,"num_object_copies":20364,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10182,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7421,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'10793","reported_seq":10931,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:42.701438+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:02:42.701438+0000","last_peered":"2024-09-10T16:02:42.701438+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:02:42.701438+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"36'700","ondisk_log_start":"36'700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42058211,"num_objects":10269,"num_object_clones":0,"num_object_copies":20538,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10269,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7405,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5898},{"shards":"0,2","objects":4111}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'10815","reported_seq":10953,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:42.815777+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:02:42.815777+0000","last_peered":"2024-09-10T16:02:42.815777+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:02:42.815777+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"36'800","ondisk_log_start":"36'800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42054598,"num_objects":10269,"num_object_clones":0,"num_object_copies":20538,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10269,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":7314,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6032},{"shards":"0,2","objects":3966}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'10914","reported_seq":11044,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:42.738604+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:02:42.738604+0000","last_peered":"2024-09-10T16:02:42.738604+0000","last_clean":"2024-09-10T16:02:42.738604+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:02:42.738604+0000","last_undegraded":"2024-09-10T16:02:42.738604+0000","last_fullsized":"2024-09-10T16:02:42.738604+0000","mapping_epoch":34,"log_start":"36'900","ondisk_log_start":"36'900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42598400,"num_objects":10400,"num_object_clones":0,"num_object_copies":20800,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10400,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7761,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104219+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:02:08.104219+0000","last_peered":"2024-09-10T16:02:08.104219+0000","last_clean":"2024-09-10T16:02:08.104219+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:02:08.104219+0000","last_undegraded":"2024-09-10T16:02:08.104219+0000","last_fullsized":"2024-09-10T16:02:08.104219+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'10416","reported_seq":10555,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:39.903180+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:02:39.903180+0000","last_peered":"2024-09-10T16:02:39.903180+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:02:39.903180+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"36'400","ondisk_log_start":"36'400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41128419,"num_objects":10042,"num_object_clones":0,"num_object_copies":20084,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10042,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8235,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3939},{"shards":"2","objects":5919}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103774+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:02:08.103774+0000","last_peered":"2024-09-10T16:02:08.103774+0000","last_clean":"2024-09-10T16:02:08.103774+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:02:08.103774+0000","last_undegraded":"2024-09-10T16:02:08.103774+0000","last_fullsized":"2024-09-10T16:02:08.103774+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'10707","reported_seq":10837,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:42.695807+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:02:42.695807+0000","last_peered":"2024-09-10T16:02:42.695807+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:02:42.695807+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"36'700","ondisk_log_start":"36'700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41791488,"num_objects":10203,"num_object_clones":0,"num_object_copies":20406,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10203,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7211,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5948},{"shards":"0,2","objects":4007}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":104,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.105055+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:02:08.105055+0000","last_peered":"2024-09-10T16:02:08.105055+0000","last_clean":"2024-09-10T16:02:08.105055+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:02:08.105055+0000","last_undegraded":"2024-09-10T16:02:08.105055+0000","last_fullsized":"2024-09-10T16:02:08.105055+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'10576","reported_seq":10711,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:39.870991+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:02:39.870991+0000","last_peered":"2024-09-10T16:02:39.870991+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:02:39.870991+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"36'500","ondisk_log_start":"36'500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41800163,"num_objects":10206,"num_object_clones":0,"num_object_copies":20412,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10206,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8256,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4121},{"shards":"2","objects":5902}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104572+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:02:08.104572+0000","last_peered":"2024-09-10T16:02:08.104572+0000","last_clean":"2024-09-10T16:02:08.104572+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:02:08.104572+0000","last_undegraded":"2024-09-10T16:02:08.104572+0000","last_fullsized":"2024-09-10T16:02:08.104572+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'10785","reported_seq":10914,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:39.870095+0000","last_change":"2024-09-10T15:59:16.174113+0000","last_active":"2024-09-10T16:02:39.870095+0000","last_peered":"2024-09-10T16:02:39.870095+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:02:39.870095+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"36'700","ondisk_log_start":"36'700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42536960,"num_objects":10385,"num_object_clones":0,"num_object_copies":20770,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4354,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10385,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8339,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4155},{"shards":"2","objects":6033}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'10730","reported_seq":10864,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:42.833666+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:02:42.833666+0000","last_peered":"2024-09-10T16:02:42.833666+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:02:42.833666+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"36'700","ondisk_log_start":"36'700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41402368,"num_objects":10108,"num_object_clones":0,"num_object_copies":20216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10108,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7361,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5804},{"shards":"0,2","objects":3994}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103276+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:02:08.103276+0000","last_peered":"2024-09-10T16:02:08.103276+0000","last_clean":"2024-09-10T16:02:08.103276+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:02:08.103276+0000","last_undegraded":"2024-09-10T16:02:08.103276+0000","last_fullsized":"2024-09-10T16:02:08.103276+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'10574","reported_seq":11972,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T16:02:42.806083+0000","last_change":"2024-09-10T16:02:06.480033+0000","last_active":"2024-09-10T16:02:42.806083+0000","last_peered":"2024-09-10T16:02:42.806083+0000","last_clean":"2024-09-10T15:59:14.115595+0000","last_became_active":"2024-09-10T15:59:16.244089+0000","last_became_peered":"2024-09-10T15:59:16.244089+0000","last_unstale":"2024-09-10T16:02:42.806083+0000","last_undegraded":"2024-09-10T16:02:42.806083+0000","last_fullsized":"2024-09-10T15:59:15.774437+0000","mapping_epoch":42,"log_start":"36'500","ondisk_log_start":"36'500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41058304,"num_objects":10024,"num_object_clones":0,"num_object_copies":20048,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3576,"num_objects_unfound":0,"num_objects_dirty":10024,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6964,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":627,"num_bytes_recovered":2568192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6448},{"shards":"0,2","objects":3576}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103831+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:02:08.103831+0000","last_peered":"2024-09-10T16:02:08.103831+0000","last_clean":"2024-09-10T16:02:08.103831+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:02:08.103831+0000","last_undegraded":"2024-09-10T16:02:08.103831+0000","last_fullsized":"2024-09-10T16:02:08.103831+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'10803","reported_seq":10945,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:42.791664+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:02:42.791664+0000","last_peered":"2024-09-10T16:02:42.791664+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:02:42.791664+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"36'800","ondisk_log_start":"36'800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41963520,"num_objects":10245,"num_object_clones":0,"num_object_copies":20490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10245,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7058,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5874},{"shards":"0,2","objects":4094}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104431+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:02:08.104431+0000","last_peered":"2024-09-10T16:02:08.104431+0000","last_clean":"2024-09-10T16:02:08.104431+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:02:08.104431+0000","last_undegraded":"2024-09-10T16:02:08.104431+0000","last_fullsized":"2024-09-10T16:02:08.104431+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104410+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:02:08.104410+0000","last_peered":"2024-09-10T16:02:08.104410+0000","last_clean":"2024-09-10T16:02:08.104410+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:02:08.104410+0000","last_undegraded":"2024-09-10T16:02:08.104410+0000","last_fullsized":"2024-09-10T16:02:08.104410+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'10875","reported_seq":11019,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:42.769658+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:02:42.769658+0000","last_peered":"2024-09-10T16:02:42.769658+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:02:42.769658+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"36'800","ondisk_log_start":"36'800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42283974,"num_objects":10325,"num_object_clones":0,"num_object_copies":20650,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10325,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7038,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6002},{"shards":"0,2","objects":4053}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":668766966,"num_objects":163283,"num_object_clones":0,"num_object_copies":326566,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42911,"num_objects_misplaced":3576,"num_objects_unfound":0,"num_objects_dirty":163283,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":128823,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4899,"num_bytes_recovered":20062208,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1367941120,"data_stored":1367781230,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160549,"ondisk_log_size":160549,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986232,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":746408,"kb_used_data":466200,"kb_used_omap":4,"kb_used_meta":280187,"kb_avail":93625432,"statfs":{"total":96636764160,"available":95872442368,"internally_reserved":0,"allocated":477388800,"data_stored":477096567,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":286912170},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,2,1,0,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659178,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":548400,"kb_used_data":314016,"kb_used_omap":4,"kb_used_meta":234363,"kb_avail":93823440,"statfs":{"total":96636764160,"available":96075202560,"internally_reserved":0,"allocated":321552384,"data_stored":321212718,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":239988398},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,4,1,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182461,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":892824,"kb_used_data":558532,"kb_used_omap":4,"kb_used_meta":334267,"kb_avail":93479016,"statfs":{"total":96636764160,"available":95722512384,"internally_reserved":0,"allocated":571936768,"data_stored":571644530,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":342290090},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,0,5,8,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":570761216,"data_stored":570725086,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":321003520,"data_stored":320919962,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":476176384,"data_stored":476136182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:02:47.550 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:02:48.125 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:02:48.128 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:02:48.159 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":204,"stamp":"2024-09-10T16:02:47.620785+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":665314057,"num_objects":162442,"num_object_clones":0,"num_object_copies":324884,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42911,"num_objects_misplaced":3576,"num_objects_unfound":0,"num_objects_dirty":162442,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":129668,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4901,"num_bytes_recovered":20062227,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160595,"ondisk_log_size":160595,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2196192,"kb_used_data":1333416,"kb_used_omap":12,"kb_used_meta":862707,"kb_avail":280919328,"statfs":{"total":289910292480,"available":287661391872,"internally_reserved":0,"allocated":1365417984,"data_stored":1364493847,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":883411970},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,2,4,6,15,7,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-17559552,"num_objects":-4287,"num_object_clones":0,"num_object_copies":-8574,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-164,"num_objects_unfound":0,"num_objects_dirty":-4287,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4429,"num_write_kb":284,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":164,"num_bytes_recovered":671744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004474"},"pg_stats":[{"pgid":"2.f","version":"45'10610","reported_seq":10684,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:42.836730+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:02:42.836730+0000","last_peered":"2024-09-10T16:02:42.836730+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:02:42.836730+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"36'600","ondisk_log_start":"36'600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41213952,"num_objects":10062,"num_object_clones":0,"num_object_copies":20124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10062,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10610,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5803},{"shards":"0,2","objects":3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'10756","reported_seq":10866,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:44.894859+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:02:44.894859+0000","last_peered":"2024-09-10T16:02:44.894859+0000","last_clean":"2024-09-10T16:02:44.894859+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:02:44.894859+0000","last_undegraded":"2024-09-10T16:02:44.894859+0000","last_fullsized":"2024-09-10T16:02:44.894859+0000","mapping_epoch":35,"log_start":"36'700","ondisk_log_start":"36'700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41321414,"num_objects":10090,"num_object_clones":0,"num_object_copies":20180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10090,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":8861,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'10761","reported_seq":19439,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:45.360224+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:02:45.360224+0000","last_peered":"2024-09-10T16:02:45.360224+0000","last_clean":"2024-09-10T16:02:45.360224+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:02:45.360224+0000","last_undegraded":"2024-09-10T16:02:45.360224+0000","last_fullsized":"2024-09-10T16:02:45.360224+0000","mapping_epoch":44,"log_start":"36'700","ondisk_log_start":"36'700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41107480,"num_objects":10037,"num_object_clones":0,"num_object_copies":20074,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10037,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8765,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'10640","reported_seq":10720,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:44.834467+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:02:44.834467+0000","last_peered":"2024-09-10T16:02:44.834467+0000","last_clean":"2024-09-10T16:02:44.834467+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:02:44.834467+0000","last_undegraded":"2024-09-10T16:02:44.834467+0000","last_fullsized":"2024-09-10T16:02:44.834467+0000","mapping_epoch":35,"log_start":"36'600","ondisk_log_start":"36'600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41041920,"num_objects":10020,"num_object_clones":0,"num_object_copies":20040,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10020,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10640,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'10710","reported_seq":10840,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:42.768767+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:02:42.768767+0000","last_peered":"2024-09-10T16:02:42.768767+0000","last_clean":"2024-09-10T16:02:42.768767+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:02:42.768767+0000","last_undegraded":"2024-09-10T16:02:42.768767+0000","last_fullsized":"2024-09-10T16:02:42.768767+0000","mapping_epoch":34,"log_start":"36'700","ondisk_log_start":"36'700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41701859,"num_objects":10182,"num_object_clones":0,"num_object_copies":20364,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10182,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7421,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'10793","reported_seq":10931,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:42.701438+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:02:42.701438+0000","last_peered":"2024-09-10T16:02:42.701438+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:02:42.701438+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"36'700","ondisk_log_start":"36'700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42058211,"num_objects":10269,"num_object_clones":0,"num_object_copies":20538,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10269,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7405,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5898},{"shards":"0,2","objects":4111}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'10815","reported_seq":10953,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:42.815777+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:02:42.815777+0000","last_peered":"2024-09-10T16:02:42.815777+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:02:42.815777+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"36'800","ondisk_log_start":"36'800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42054598,"num_objects":10269,"num_object_clones":0,"num_object_copies":20538,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10269,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":7314,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6032},{"shards":"0,2","objects":3966}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'10914","reported_seq":11044,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:42.738604+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:02:42.738604+0000","last_peered":"2024-09-10T16:02:42.738604+0000","last_clean":"2024-09-10T16:02:42.738604+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:02:42.738604+0000","last_undegraded":"2024-09-10T16:02:42.738604+0000","last_fullsized":"2024-09-10T16:02:42.738604+0000","mapping_epoch":34,"log_start":"36'900","ondisk_log_start":"36'900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42598400,"num_objects":10400,"num_object_clones":0,"num_object_copies":20800,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10400,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7761,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104219+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:02:08.104219+0000","last_peered":"2024-09-10T16:02:08.104219+0000","last_clean":"2024-09-10T16:02:08.104219+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:02:08.104219+0000","last_undegraded":"2024-09-10T16:02:08.104219+0000","last_fullsized":"2024-09-10T16:02:08.104219+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'10555","reported_seq":10695,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:44.727937+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:02:44.727937+0000","last_peered":"2024-09-10T16:02:44.727937+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:02:44.727937+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"36'500","ondisk_log_start":"36'500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40559075,"num_objects":9903,"num_object_clones":0,"num_object_copies":19806,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9903,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8374,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3800},{"shards":"2","objects":5780}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103774+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:02:08.103774+0000","last_peered":"2024-09-10T16:02:08.103774+0000","last_clean":"2024-09-10T16:02:08.103774+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:02:08.103774+0000","last_undegraded":"2024-09-10T16:02:08.103774+0000","last_fullsized":"2024-09-10T16:02:08.103774+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'10707","reported_seq":10837,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:42.695807+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:02:42.695807+0000","last_peered":"2024-09-10T16:02:42.695807+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:02:42.695807+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"36'700","ondisk_log_start":"36'700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41791488,"num_objects":10203,"num_object_clones":0,"num_object_copies":20406,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10203,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7211,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5948},{"shards":"0,2","objects":4007}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":104,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.105055+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:02:08.105055+0000","last_peered":"2024-09-10T16:02:08.105055+0000","last_clean":"2024-09-10T16:02:08.105055+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:02:08.105055+0000","last_undegraded":"2024-09-10T16:02:08.105055+0000","last_fullsized":"2024-09-10T16:02:08.105055+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'10711","reported_seq":10846,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:44.765563+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:02:44.765563+0000","last_peered":"2024-09-10T16:02:44.765563+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:02:44.765563+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"36'700","ondisk_log_start":"36'700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41247203,"num_objects":10071,"num_object_clones":0,"num_object_copies":20142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10071,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8391,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3986},{"shards":"2","objects":5767}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104572+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:02:08.104572+0000","last_peered":"2024-09-10T16:02:08.104572+0000","last_clean":"2024-09-10T16:02:08.104572+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:02:08.104572+0000","last_undegraded":"2024-09-10T16:02:08.104572+0000","last_fullsized":"2024-09-10T16:02:08.104572+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'10938","reported_seq":11067,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:44.889173+0000","last_change":"2024-09-10T15:59:16.174113+0000","last_active":"2024-09-10T16:02:44.889173+0000","last_peered":"2024-09-10T16:02:44.889173+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:02:44.889173+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"36'900","ondisk_log_start":"36'900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41910272,"num_objects":10232,"num_object_clones":0,"num_object_copies":20464,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4354,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10232,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8492,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4002},{"shards":"2","objects":5880}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'10730","reported_seq":10864,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:42.833666+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:02:42.833666+0000","last_peered":"2024-09-10T16:02:42.833666+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:02:42.833666+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"36'700","ondisk_log_start":"36'700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41402368,"num_objects":10108,"num_object_clones":0,"num_object_copies":20216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10108,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7361,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5804},{"shards":"0,2","objects":3994}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103276+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:02:08.103276+0000","last_peered":"2024-09-10T16:02:08.103276+0000","last_clean":"2024-09-10T16:02:08.103276+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:02:08.103276+0000","last_undegraded":"2024-09-10T16:02:08.103276+0000","last_fullsized":"2024-09-10T16:02:08.103276+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'10574","reported_seq":11972,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T16:02:42.806083+0000","last_change":"2024-09-10T16:02:06.480033+0000","last_active":"2024-09-10T16:02:42.806083+0000","last_peered":"2024-09-10T16:02:42.806083+0000","last_clean":"2024-09-10T15:59:14.115595+0000","last_became_active":"2024-09-10T15:59:16.244089+0000","last_became_peered":"2024-09-10T15:59:16.244089+0000","last_unstale":"2024-09-10T16:02:42.806083+0000","last_undegraded":"2024-09-10T16:02:42.806083+0000","last_fullsized":"2024-09-10T15:59:15.774437+0000","mapping_epoch":42,"log_start":"36'500","ondisk_log_start":"36'500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41058304,"num_objects":10024,"num_object_clones":0,"num_object_copies":20048,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3576,"num_objects_unfound":0,"num_objects_dirty":10024,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6964,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":627,"num_bytes_recovered":2568192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6448},{"shards":"0,2","objects":3576}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103831+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:02:08.103831+0000","last_peered":"2024-09-10T16:02:08.103831+0000","last_clean":"2024-09-10T16:02:08.103831+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:02:08.103831+0000","last_undegraded":"2024-09-10T16:02:08.103831+0000","last_fullsized":"2024-09-10T16:02:08.103831+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'10803","reported_seq":10945,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:42.791664+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:02:42.791664+0000","last_peered":"2024-09-10T16:02:42.791664+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:02:42.791664+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"36'800","ondisk_log_start":"36'800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41963520,"num_objects":10245,"num_object_clones":0,"num_object_copies":20490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10245,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7058,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5874},{"shards":"0,2","objects":4094}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104431+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:02:08.104431+0000","last_peered":"2024-09-10T16:02:08.104431+0000","last_clean":"2024-09-10T16:02:08.104431+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:02:08.104431+0000","last_undegraded":"2024-09-10T16:02:08.104431+0000","last_fullsized":"2024-09-10T16:02:08.104431+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104410+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:02:08.104410+0000","last_peered":"2024-09-10T16:02:08.104410+0000","last_clean":"2024-09-10T16:02:08.104410+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:02:08.104410+0000","last_undegraded":"2024-09-10T16:02:08.104410+0000","last_fullsized":"2024-09-10T16:02:08.104410+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'10875","reported_seq":11019,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:42.769658+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:02:42.769658+0000","last_peered":"2024-09-10T16:02:42.769658+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:02:42.769658+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"36'800","ondisk_log_start":"36'800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42283974,"num_objects":10325,"num_object_clones":0,"num_object_copies":20650,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10325,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7038,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6002},{"shards":"0,2","objects":4053}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":665314038,"num_objects":162440,"num_object_clones":0,"num_object_copies":324880,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42911,"num_objects_misplaced":3576,"num_objects_unfound":0,"num_objects_dirty":162440,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":129666,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4899,"num_bytes_recovered":20062208,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1362690048,"data_stored":1362530158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160592,"ondisk_log_size":160592,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986233,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":746204,"kb_used_data":461832,"kb_used_omap":4,"kb_used_meta":284347,"kb_avail":93625636,"statfs":{"total":96636764160,"available":95872651264,"internally_reserved":0,"allocated":472915968,"data_stored":472623735,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":291172010},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,3,0,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659179,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":557164,"kb_used_data":313052,"kb_used_omap":4,"kb_used_meta":244091,"kb_avail":93814676,"statfs":{"total":96636764160,"available":96066228224,"internally_reserved":0,"allocated":320565248,"data_stored":320225582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":249949870},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,1,4,4,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182461,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":892824,"kb_used_data":558532,"kb_used_omap":4,"kb_used_meta":334267,"kb_avail":93479016,"statfs":{"total":96636764160,"available":95722512384,"internally_reserved":0,"allocated":571936768,"data_stored":571644530,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":342290090},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,0,5,8,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":570761216,"data_stored":570725086,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":320184320,"data_stored":320100762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":471744512,"data_stored":471704310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:02:48.161 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:02:48.730 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:02:48.733 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:02:48.763 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":204,"stamp":"2024-09-10T16:02:47.620785+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":665314057,"num_objects":162442,"num_object_clones":0,"num_object_copies":324884,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42911,"num_objects_misplaced":3576,"num_objects_unfound":0,"num_objects_dirty":162442,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":129668,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4901,"num_bytes_recovered":20062227,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160595,"ondisk_log_size":160595,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2196192,"kb_used_data":1333416,"kb_used_omap":12,"kb_used_meta":862707,"kb_avail":280919328,"statfs":{"total":289910292480,"available":287661391872,"internally_reserved":0,"allocated":1365417984,"data_stored":1364493847,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":883411970},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,2,4,6,15,7,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-17559552,"num_objects":-4287,"num_object_clones":0,"num_object_copies":-8574,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-164,"num_objects_unfound":0,"num_objects_dirty":-4287,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4429,"num_write_kb":284,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":164,"num_bytes_recovered":671744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004474"},"pg_stats":[{"pgid":"2.f","version":"45'10610","reported_seq":10684,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:42.836730+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:02:42.836730+0000","last_peered":"2024-09-10T16:02:42.836730+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:02:42.836730+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"36'600","ondisk_log_start":"36'600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41213952,"num_objects":10062,"num_object_clones":0,"num_object_copies":20124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10062,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10610,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5803},{"shards":"0,2","objects":3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'10756","reported_seq":10866,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:44.894859+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:02:44.894859+0000","last_peered":"2024-09-10T16:02:44.894859+0000","last_clean":"2024-09-10T16:02:44.894859+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:02:44.894859+0000","last_undegraded":"2024-09-10T16:02:44.894859+0000","last_fullsized":"2024-09-10T16:02:44.894859+0000","mapping_epoch":35,"log_start":"36'700","ondisk_log_start":"36'700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41321414,"num_objects":10090,"num_object_clones":0,"num_object_copies":20180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10090,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":8861,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'10761","reported_seq":19439,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:45.360224+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:02:45.360224+0000","last_peered":"2024-09-10T16:02:45.360224+0000","last_clean":"2024-09-10T16:02:45.360224+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:02:45.360224+0000","last_undegraded":"2024-09-10T16:02:45.360224+0000","last_fullsized":"2024-09-10T16:02:45.360224+0000","mapping_epoch":44,"log_start":"36'700","ondisk_log_start":"36'700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41107480,"num_objects":10037,"num_object_clones":0,"num_object_copies":20074,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10037,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8765,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'10640","reported_seq":10720,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:44.834467+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:02:44.834467+0000","last_peered":"2024-09-10T16:02:44.834467+0000","last_clean":"2024-09-10T16:02:44.834467+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:02:44.834467+0000","last_undegraded":"2024-09-10T16:02:44.834467+0000","last_fullsized":"2024-09-10T16:02:44.834467+0000","mapping_epoch":35,"log_start":"36'600","ondisk_log_start":"36'600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41041920,"num_objects":10020,"num_object_clones":0,"num_object_copies":20040,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10020,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10640,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'10710","reported_seq":10840,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:42.768767+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:02:42.768767+0000","last_peered":"2024-09-10T16:02:42.768767+0000","last_clean":"2024-09-10T16:02:42.768767+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:02:42.768767+0000","last_undegraded":"2024-09-10T16:02:42.768767+0000","last_fullsized":"2024-09-10T16:02:42.768767+0000","mapping_epoch":34,"log_start":"36'700","ondisk_log_start":"36'700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41701859,"num_objects":10182,"num_object_clones":0,"num_object_copies":20364,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10182,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7421,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'10793","reported_seq":10931,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:42.701438+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:02:42.701438+0000","last_peered":"2024-09-10T16:02:42.701438+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:02:42.701438+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"36'700","ondisk_log_start":"36'700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42058211,"num_objects":10269,"num_object_clones":0,"num_object_copies":20538,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10269,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7405,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5898},{"shards":"0,2","objects":4111}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'10815","reported_seq":10953,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:42.815777+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:02:42.815777+0000","last_peered":"2024-09-10T16:02:42.815777+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:02:42.815777+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"36'800","ondisk_log_start":"36'800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42054598,"num_objects":10269,"num_object_clones":0,"num_object_copies":20538,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10269,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":7314,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6032},{"shards":"0,2","objects":3966}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'10914","reported_seq":11044,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:42.738604+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:02:42.738604+0000","last_peered":"2024-09-10T16:02:42.738604+0000","last_clean":"2024-09-10T16:02:42.738604+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:02:42.738604+0000","last_undegraded":"2024-09-10T16:02:42.738604+0000","last_fullsized":"2024-09-10T16:02:42.738604+0000","mapping_epoch":34,"log_start":"36'900","ondisk_log_start":"36'900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42598400,"num_objects":10400,"num_object_clones":0,"num_object_copies":20800,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10400,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7761,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104219+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:02:08.104219+0000","last_peered":"2024-09-10T16:02:08.104219+0000","last_clean":"2024-09-10T16:02:08.104219+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:02:08.104219+0000","last_undegraded":"2024-09-10T16:02:08.104219+0000","last_fullsized":"2024-09-10T16:02:08.104219+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'10555","reported_seq":10695,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:44.727937+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:02:44.727937+0000","last_peered":"2024-09-10T16:02:44.727937+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:02:44.727937+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"36'500","ondisk_log_start":"36'500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40559075,"num_objects":9903,"num_object_clones":0,"num_object_copies":19806,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9903,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8374,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3800},{"shards":"2","objects":5780}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103774+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:02:08.103774+0000","last_peered":"2024-09-10T16:02:08.103774+0000","last_clean":"2024-09-10T16:02:08.103774+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:02:08.103774+0000","last_undegraded":"2024-09-10T16:02:08.103774+0000","last_fullsized":"2024-09-10T16:02:08.103774+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'10707","reported_seq":10837,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:42.695807+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:02:42.695807+0000","last_peered":"2024-09-10T16:02:42.695807+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:02:42.695807+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"36'700","ondisk_log_start":"36'700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41791488,"num_objects":10203,"num_object_clones":0,"num_object_copies":20406,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10203,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7211,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5948},{"shards":"0,2","objects":4007}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":104,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.105055+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:02:08.105055+0000","last_peered":"2024-09-10T16:02:08.105055+0000","last_clean":"2024-09-10T16:02:08.105055+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:02:08.105055+0000","last_undegraded":"2024-09-10T16:02:08.105055+0000","last_fullsized":"2024-09-10T16:02:08.105055+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'10711","reported_seq":10846,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:44.765563+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:02:44.765563+0000","last_peered":"2024-09-10T16:02:44.765563+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:02:44.765563+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"36'700","ondisk_log_start":"36'700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41247203,"num_objects":10071,"num_object_clones":0,"num_object_copies":20142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10071,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8391,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3986},{"shards":"2","objects":5767}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104572+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:02:08.104572+0000","last_peered":"2024-09-10T16:02:08.104572+0000","last_clean":"2024-09-10T16:02:08.104572+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:02:08.104572+0000","last_undegraded":"2024-09-10T16:02:08.104572+0000","last_fullsized":"2024-09-10T16:02:08.104572+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'10938","reported_seq":11067,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:44.889173+0000","last_change":"2024-09-10T15:59:16.174113+0000","last_active":"2024-09-10T16:02:44.889173+0000","last_peered":"2024-09-10T16:02:44.889173+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:02:44.889173+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"36'900","ondisk_log_start":"36'900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41910272,"num_objects":10232,"num_object_clones":0,"num_object_copies":20464,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4354,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10232,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8492,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4002},{"shards":"2","objects":5880}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'10730","reported_seq":10864,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:42.833666+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:02:42.833666+0000","last_peered":"2024-09-10T16:02:42.833666+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:02:42.833666+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"36'700","ondisk_log_start":"36'700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41402368,"num_objects":10108,"num_object_clones":0,"num_object_copies":20216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10108,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7361,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5804},{"shards":"0,2","objects":3994}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103276+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:02:08.103276+0000","last_peered":"2024-09-10T16:02:08.103276+0000","last_clean":"2024-09-10T16:02:08.103276+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:02:08.103276+0000","last_undegraded":"2024-09-10T16:02:08.103276+0000","last_fullsized":"2024-09-10T16:02:08.103276+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'10574","reported_seq":11972,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T16:02:42.806083+0000","last_change":"2024-09-10T16:02:06.480033+0000","last_active":"2024-09-10T16:02:42.806083+0000","last_peered":"2024-09-10T16:02:42.806083+0000","last_clean":"2024-09-10T15:59:14.115595+0000","last_became_active":"2024-09-10T15:59:16.244089+0000","last_became_peered":"2024-09-10T15:59:16.244089+0000","last_unstale":"2024-09-10T16:02:42.806083+0000","last_undegraded":"2024-09-10T16:02:42.806083+0000","last_fullsized":"2024-09-10T15:59:15.774437+0000","mapping_epoch":42,"log_start":"36'500","ondisk_log_start":"36'500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41058304,"num_objects":10024,"num_object_clones":0,"num_object_copies":20048,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3576,"num_objects_unfound":0,"num_objects_dirty":10024,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6964,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":627,"num_bytes_recovered":2568192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6448},{"shards":"0,2","objects":3576}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103831+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:02:08.103831+0000","last_peered":"2024-09-10T16:02:08.103831+0000","last_clean":"2024-09-10T16:02:08.103831+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:02:08.103831+0000","last_undegraded":"2024-09-10T16:02:08.103831+0000","last_fullsized":"2024-09-10T16:02:08.103831+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'10803","reported_seq":10945,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:42.791664+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:02:42.791664+0000","last_peered":"2024-09-10T16:02:42.791664+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:02:42.791664+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"36'800","ondisk_log_start":"36'800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41963520,"num_objects":10245,"num_object_clones":0,"num_object_copies":20490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10245,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7058,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5874},{"shards":"0,2","objects":4094}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104431+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:02:08.104431+0000","last_peered":"2024-09-10T16:02:08.104431+0000","last_clean":"2024-09-10T16:02:08.104431+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:02:08.104431+0000","last_undegraded":"2024-09-10T16:02:08.104431+0000","last_fullsized":"2024-09-10T16:02:08.104431+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104410+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:02:08.104410+0000","last_peered":"2024-09-10T16:02:08.104410+0000","last_clean":"2024-09-10T16:02:08.104410+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:02:08.104410+0000","last_undegraded":"2024-09-10T16:02:08.104410+0000","last_fullsized":"2024-09-10T16:02:08.104410+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'10875","reported_seq":11019,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:42.769658+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:02:42.769658+0000","last_peered":"2024-09-10T16:02:42.769658+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:02:42.769658+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"36'800","ondisk_log_start":"36'800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42283974,"num_objects":10325,"num_object_clones":0,"num_object_copies":20650,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10325,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7038,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6002},{"shards":"0,2","objects":4053}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":665314038,"num_objects":162440,"num_object_clones":0,"num_object_copies":324880,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42911,"num_objects_misplaced":3576,"num_objects_unfound":0,"num_objects_dirty":162440,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":129666,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4899,"num_bytes_recovered":20062208,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1362690048,"data_stored":1362530158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160592,"ondisk_log_size":160592,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986233,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":746204,"kb_used_data":461832,"kb_used_omap":4,"kb_used_meta":284347,"kb_avail":93625636,"statfs":{"total":96636764160,"available":95872651264,"internally_reserved":0,"allocated":472915968,"data_stored":472623735,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":291172010},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,3,0,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659179,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":557164,"kb_used_data":313052,"kb_used_omap":4,"kb_used_meta":244091,"kb_avail":93814676,"statfs":{"total":96636764160,"available":96066228224,"internally_reserved":0,"allocated":320565248,"data_stored":320225582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":249949870},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,1,4,4,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182461,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":892824,"kb_used_data":558532,"kb_used_omap":4,"kb_used_meta":334267,"kb_avail":93479016,"statfs":{"total":96636764160,"available":95722512384,"internally_reserved":0,"allocated":571936768,"data_stored":571644530,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":342290090},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,0,5,8,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":570761216,"data_stored":570725086,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":320184320,"data_stored":320100762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":471744512,"data_stored":471704310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:02:48.765 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-10T16:02:48.765 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "45'10610", 'reported_seq': 10684, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:02:42.836730+0000', 'last_change': '2024-09-10T15:59:16.281558+0000', 'last_active': '2024-09-10T16:02:42.836730+0000', 'last_peered': '2024-09-10T16:02:42.836730+0000', 'last_clean': '2024-09-10T15:59:14.132456+0000', 'last_became_active': '2024-09-10T15:59:16.274407+0000', 'last_became_peered': '2024-09-10T15:59:16.274407+0000', 'last_unstale': '2024-09-10T16:02:42.836730+0000', 'last_undegraded': '2024-09-10T15:59:15.851279+0000', 'last_fullsized': '2024-09-10T15:59:15.774356+0000', 'mapping_epoch': 42, 'log_start': "36'600", 'ondisk_log_start': "36'600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'objects_scrubbed': 0, 'log_size': 10010, 'log_dups_size': 0, 'ondisk_log_size': 10010, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 41213952, 'num_objects': 10062, 'num_object_clones': 0, 'num_object_copies': 20124, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4259, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10062, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10610, 'num_write_kb': 41344, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5803}, {'shards': '0,2', 'objects': 3984}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:02:48.765 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-10T16:02:48.765 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "45'10793", 'reported_seq': 10931, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:02:42.701438+0000', 'last_change': '2024-09-10T15:59:16.308264+0000', 'last_active': '2024-09-10T16:02:42.701438+0000', 'last_peered': '2024-09-10T16:02:42.701438+0000', 'last_clean': '2024-09-10T15:59:14.111551+0000', 'last_became_active': '2024-09-10T15:59:16.300040+0000', 'last_became_peered': '2024-09-10T15:59:16.300040+0000', 'last_unstale': '2024-09-10T16:02:42.701438+0000', 'last_undegraded': '2024-09-10T15:59:15.878234+0000', 'last_fullsized': '2024-09-10T15:59:15.769328+0000', 'mapping_epoch': 42, 'log_start': "36'700", 'ondisk_log_start': "36'700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3442", 'last_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'objects_scrubbed': 3384, 'log_size': 10093, 'log_dups_size': 0, 'ondisk_log_size': 10093, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1292, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 42058211, 'num_objects': 10269, 'num_object_clones': 0, 'num_object_copies': 20538, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4372, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10269, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 7405, 'num_write_kb': 28580, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5898}, {'shards': '0,2', 'objects': 4111}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:02:48.765 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-10T16:02:48.765 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "45'10815", 'reported_seq': 10953, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:02:42.815777+0000', 'last_change': '2024-09-10T15:59:16.331899+0000', 'last_active': '2024-09-10T16:02:42.815777+0000', 'last_peered': '2024-09-10T16:02:42.815777+0000', 'last_clean': '2024-09-10T15:59:14.139436+0000', 'last_became_active': '2024-09-10T15:59:16.318715+0000', 'last_became_peered': '2024-09-10T15:59:16.318715+0000', 'last_unstale': '2024-09-10T16:02:42.815777+0000', 'last_undegraded': '2024-09-10T15:59:15.877524+0000', 'last_fullsized': '2024-09-10T15:59:15.769042+0000', 'mapping_epoch': 42, 'log_start': "36'800", 'ondisk_log_start': "36'800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3537", 'last_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'objects_scrubbed': 3497, 'log_size': 10015, 'log_dups_size': 0, 'ondisk_log_size': 10015, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1224, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 42054598, 'num_objects': 10269, 'num_object_clones': 0, 'num_object_copies': 20538, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4237, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10269, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 7314, 'num_write_kb': 28172, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6032}, {'shards': '0,2', 'objects': 3966}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:02:48.765 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-10T16:02:48.765 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "45'10555", 'reported_seq': 10695, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:02:44.727937+0000', 'last_change': '2024-09-10T15:59:16.175680+0000', 'last_active': '2024-09-10T16:02:44.727937+0000', 'last_peered': '2024-09-10T16:02:44.727937+0000', 'last_clean': '2024-09-10T15:59:14.088187+0000', 'last_became_active': '2024-09-10T15:59:16.165810+0000', 'last_became_peered': '2024-09-10T15:59:16.165810+0000', 'last_unstale': '2024-09-10T16:02:44.727937+0000', 'last_undegraded': '2024-09-10T15:59:15.822098+0000', 'last_fullsized': '2024-09-10T15:59:15.769123+0000', 'mapping_epoch': 42, 'log_start': "36'500", 'ondisk_log_start': "36'500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2218", 'last_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'objects_scrubbed': 2175, 'log_size': 10055, 'log_dups_size': 0, 'ondisk_log_size': 10055, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 825, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40559075, 'num_objects': 9903, 'num_object_clones': 0, 'num_object_copies': 19806, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4124, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9903, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 8374, 'num_write_kb': 32204, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3800}, {'shards': '2', 'objects': 5780}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:02:48.765 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-10T16:02:48.766 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "45'10707", 'reported_seq': 10837, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:02:42.695807+0000', 'last_change': '2024-09-10T15:59:16.277742+0000', 'last_active': '2024-09-10T16:02:42.695807+0000', 'last_peered': '2024-09-10T16:02:42.695807+0000', 'last_clean': '2024-09-10T15:59:14.129081+0000', 'last_became_active': '2024-09-10T15:59:16.269835+0000', 'last_became_peered': '2024-09-10T15:59:16.269835+0000', 'last_unstale': '2024-09-10T16:02:42.695807+0000', 'last_undegraded': '2024-09-10T15:59:15.860857+0000', 'last_fullsized': '2024-09-10T15:59:15.771410+0000', 'mapping_epoch': 42, 'log_start': "36'700", 'ondisk_log_start': "36'700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3557", 'last_scrub_stamp': '2024-09-10T15:59:02.493678+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:02.493678+0000', 'objects_scrubbed': 3488, 'log_size': 10007, 'log_dups_size': 0, 'ondisk_log_size': 10007, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1472, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 41791488, 'num_objects': 10203, 'num_object_clones': 0, 'num_object_copies': 20406, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4257, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10203, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7211, 'num_write_kb': 27852, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5948}, {'shards': '0,2', 'objects': 4007}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:02:48.766 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-10T16:02:48.766 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "45'10711", 'reported_seq': 10846, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:02:44.765563+0000', 'last_change': '2024-09-10T15:59:16.284311+0000', 'last_active': '2024-09-10T16:02:44.765563+0000', 'last_peered': '2024-09-10T16:02:44.765563+0000', 'last_clean': '2024-09-10T15:59:14.116041+0000', 'last_became_active': '2024-09-10T15:59:16.274543+0000', 'last_became_peered': '2024-09-10T15:59:16.274543+0000', 'last_unstale': '2024-09-10T16:02:44.765563+0000', 'last_undegraded': '2024-09-10T15:59:15.812484+0000', 'last_fullsized': '2024-09-10T15:59:15.770885+0000', 'mapping_epoch': 42, 'log_start': "36'700", 'ondisk_log_start': "36'700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2356", 'last_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'objects_scrubbed': 2316, 'log_size': 10011, 'log_dups_size': 0, 'ondisk_log_size': 10011, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 783, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 41247203, 'num_objects': 10071, 'num_object_clones': 0, 'num_object_copies': 20142, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10071, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 8391, 'num_write_kb': 32292, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3986}, {'shards': '2', 'objects': 5767}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:02:48.766 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-10T16:02:48.766 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "45'10938", 'reported_seq': 11067, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:02:44.889173+0000', 'last_change': '2024-09-10T15:59:16.174113+0000', 'last_active': '2024-09-10T16:02:44.889173+0000', 'last_peered': '2024-09-10T16:02:44.889173+0000', 'last_clean': '2024-09-10T15:59:14.110654+0000', 'last_became_active': '2024-09-10T15:59:16.164645+0000', 'last_became_peered': '2024-09-10T15:59:16.164645+0000', 'last_unstale': '2024-09-10T16:02:44.889173+0000', 'last_undegraded': '2024-09-10T15:59:15.803593+0000', 'last_fullsized': '2024-09-10T15:59:15.769083+0000', 'mapping_epoch': 42, 'log_start': "36'900", 'ondisk_log_start': "36'900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2486", 'last_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'objects_scrubbed': 2440, 'log_size': 10038, 'log_dups_size': 0, 'ondisk_log_size': 10038, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 792, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 41910272, 'num_objects': 10232, 'num_object_clones': 0, 'num_object_copies': 20464, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4354, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10232, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8492, 'num_write_kb': 32568, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4002}, {'shards': '2', 'objects': 5880}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:02:48.766 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-10T16:02:48.766 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "45'10730", 'reported_seq': 10864, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:02:42.833666+0000', 'last_change': '2024-09-10T15:59:16.315641+0000', 'last_active': '2024-09-10T16:02:42.833666+0000', 'last_peered': '2024-09-10T16:02:42.833666+0000', 'last_clean': '2024-09-10T15:59:14.117001+0000', 'last_became_active': '2024-09-10T15:59:16.302245+0000', 'last_became_peered': '2024-09-10T15:59:16.302245+0000', 'last_unstale': '2024-09-10T16:02:42.833666+0000', 'last_undegraded': '2024-09-10T15:59:15.845012+0000', 'last_fullsized': '2024-09-10T15:59:15.770337+0000', 'mapping_epoch': 42, 'log_start': "36'700", 'ondisk_log_start': "36'700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3404", 'last_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'objects_scrubbed': 3367, 'log_size': 10030, 'log_dups_size': 0, 'ondisk_log_size': 10030, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 983, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 41402368, 'num_objects': 10108, 'num_object_clones': 0, 'num_object_copies': 20216, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10108, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7361, 'num_write_kb': 28204, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5804}, {'shards': '0,2', 'objects': 3994}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:02:48.766 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-10T16:02:48.766 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "45'10574", 'reported_seq': 11972, 'reported_epoch': 45, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-10T16:02:42.806083+0000', 'last_change': '2024-09-10T16:02:06.480033+0000', 'last_active': '2024-09-10T16:02:42.806083+0000', 'last_peered': '2024-09-10T16:02:42.806083+0000', 'last_clean': '2024-09-10T15:59:14.115595+0000', 'last_became_active': '2024-09-10T15:59:16.244089+0000', 'last_became_peered': '2024-09-10T15:59:16.244089+0000', 'last_unstale': '2024-09-10T16:02:42.806083+0000', 'last_undegraded': '2024-09-10T16:02:42.806083+0000', 'last_fullsized': '2024-09-10T15:59:15.774437+0000', 'mapping_epoch': 42, 'log_start': "36'500", 'ondisk_log_start': "36'500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3653", 'last_scrub_stamp': '2024-09-10T15:59:05.298468+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:05.298468+0000', 'objects_scrubbed': 3602, 'log_size': 10074, 'log_dups_size': 0, 'ondisk_log_size': 10074, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1289, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 41058304, 'num_objects': 10024, 'num_object_clones': 0, 'num_object_copies': 20048, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 3576, 'num_objects_unfound': 0, 'num_objects_dirty': 10024, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6964, 'num_write_kb': 26772, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 627, 'num_bytes_recovered': 2568192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6448}, {'shards': '0,2', 'objects': 3576}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:02:48.766 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-10T16:02:48.766 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "45'10803", 'reported_seq': 10945, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:02:42.791664+0000', 'last_change': '2024-09-10T15:59:16.311291+0000', 'last_active': '2024-09-10T16:02:42.791664+0000', 'last_peered': '2024-09-10T16:02:42.791664+0000', 'last_clean': '2024-09-10T15:59:14.116363+0000', 'last_became_active': '2024-09-10T15:59:16.301244+0000', 'last_became_peered': '2024-09-10T15:59:16.301244+0000', 'last_unstale': '2024-09-10T16:02:42.791664+0000', 'last_undegraded': '2024-09-10T15:59:15.883434+0000', 'last_fullsized': '2024-09-10T15:59:15.772756+0000', 'mapping_epoch': 42, 'log_start': "36'800", 'ondisk_log_start': "36'800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3789", 'last_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'objects_scrubbed': 3741, 'log_size': 10003, 'log_dups_size': 0, 'ondisk_log_size': 10003, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1440, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 41963520, 'num_objects': 10245, 'num_object_clones': 0, 'num_object_copies': 20490, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4373, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10245, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7058, 'num_write_kb': 27124, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5874}, {'shards': '0,2', 'objects': 4094}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:02:48.767 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-10T16:02:48.767 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "45'10875", 'reported_seq': 11019, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:02:42.769658+0000', 'last_change': '2024-09-10T15:59:16.313409+0000', 'last_active': '2024-09-10T16:02:42.769658+0000', 'last_peered': '2024-09-10T16:02:42.769658+0000', 'last_clean': '2024-09-10T15:59:14.129780+0000', 'last_became_active': '2024-09-10T15:59:16.300932+0000', 'last_became_peered': '2024-09-10T15:59:16.300932+0000', 'last_unstale': '2024-09-10T16:02:42.769658+0000', 'last_undegraded': '2024-09-10T15:59:15.887823+0000', 'last_fullsized': '2024-09-10T15:59:15.768513+0000', 'mapping_epoch': 42, 'log_start': "36'800", 'ondisk_log_start': "36'800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3880", 'last_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'objects_scrubbed': 3827, 'log_size': 10075, 'log_dups_size': 0, 'ondisk_log_size': 10075, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1397, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 42283974, 'num_objects': 10325, 'num_object_clones': 0, 'num_object_copies': 20650, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4325, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10325, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7038, 'num_write_kb': 27072, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6002}, {'shards': '0,2', 'objects': 4053}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:02:48.767 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T16:02:49.455 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:02:49.456 INFO:teuthology.orchestra.run.smithi115.stdout:{"fsid":"7effee29-df05-4351-9741-ee203564ed81","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 42911/324884 objects degraded (13.208%), 10 pgs degraded, 11 pgs undersized","count":21},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":333,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983954,"num_in_osds":3,"osd_in_since":1725983947,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":162442,"data_bytes":665314057,"bytes_used":2248900608,"bytes_avail":287661391872,"bytes_total":289910292480,"degraded_objects":42911,"degraded_total":324884,"degraded_ratio":0.1320809889068098,"misplaced_objects":3576,"misplaced_total":324884,"misplaced_ratio":0.011007005577375308,"recovering_objects_per_sec":13,"recovering_bytes_per_sec":55957,"recovering_keys_per_sec":0,"num_objects_recovered":164,"num_bytes_recovered":671744,"num_keys_recovered":0,"write_bytes_sec":24225,"read_op_per_sec":0,"write_op_per_sec":368},"fsmap":{"epoch":1,"btime":"2024-09-10T15:57:14:967120+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T16:00:13.560776+0000","services":{}},"progress_events":{}} 2024-09-10T16:02:49.482 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-10T16:02:49.482 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:02:50.045 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:02:50.046 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:02:50.074 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":205,"stamp":"2024-09-10T16:02:49.621536+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":659686153,"num_objects":161068,"num_object_clones":0,"num_object_copies":322136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42911,"num_objects_misplaced":3491,"num_objects_unfound":0,"num_objects_dirty":161068,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":131042,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4986,"num_bytes_recovered":20410387,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160669,"ondisk_log_size":160669,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2193988,"kb_used_data":1327056,"kb_used_omap":12,"kb_used_meta":866867,"kb_avail":280921532,"statfs":{"total":289910292480,"available":287663648768,"internally_reserved":0,"allocated":1358905344,"data_stored":1357981207,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":887671810},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,2,7,3,16,9,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-23187456,"num_objects":-5661,"num_object_clones":0,"num_object_copies":-11322,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-249,"num_objects_unfound":0,"num_objects_dirty":-5661,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5803,"num_write_kb":284,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":249,"num_bytes_recovered":1019904,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004346"},"pg_stats":[{"pgid":"2.f","version":"45'10746","reported_seq":10821,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:47.868045+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:02:47.868045+0000","last_peered":"2024-09-10T16:02:47.868045+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:02:47.868045+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"36'700","ondisk_log_start":"36'700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40656896,"num_objects":9926,"num_object_clones":0,"num_object_copies":19852,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9926,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10746,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5667},{"shards":"0,2","objects":3848}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'10756","reported_seq":10866,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:44.894859+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:02:44.894859+0000","last_peered":"2024-09-10T16:02:44.894859+0000","last_clean":"2024-09-10T16:02:44.894859+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:02:44.894859+0000","last_undegraded":"2024-09-10T16:02:44.894859+0000","last_fullsized":"2024-09-10T16:02:44.894859+0000","mapping_epoch":35,"log_start":"36'700","ondisk_log_start":"36'700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41321414,"num_objects":10090,"num_object_clones":0,"num_object_copies":20180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10090,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":8861,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'10761","reported_seq":19439,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:45.360224+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:02:45.360224+0000","last_peered":"2024-09-10T16:02:45.360224+0000","last_clean":"2024-09-10T16:02:45.360224+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:02:45.360224+0000","last_undegraded":"2024-09-10T16:02:45.360224+0000","last_fullsized":"2024-09-10T16:02:45.360224+0000","mapping_epoch":44,"log_start":"36'700","ondisk_log_start":"36'700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41107480,"num_objects":10037,"num_object_clones":0,"num_object_copies":20074,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10037,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8765,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'10640","reported_seq":10720,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:44.834467+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:02:44.834467+0000","last_peered":"2024-09-10T16:02:44.834467+0000","last_clean":"2024-09-10T16:02:44.834467+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:02:44.834467+0000","last_undegraded":"2024-09-10T16:02:44.834467+0000","last_fullsized":"2024-09-10T16:02:44.834467+0000","mapping_epoch":35,"log_start":"36'600","ondisk_log_start":"36'600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41041920,"num_objects":10020,"num_object_clones":0,"num_object_copies":20040,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10020,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10640,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'10842","reported_seq":10972,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:47.819340+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:02:47.819340+0000","last_peered":"2024-09-10T16:02:47.819340+0000","last_clean":"2024-09-10T16:02:47.819340+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:02:47.819340+0000","last_undegraded":"2024-09-10T16:02:47.819340+0000","last_fullsized":"2024-09-10T16:02:47.819340+0000","mapping_epoch":34,"log_start":"36'800","ondisk_log_start":"36'800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41161187,"num_objects":10050,"num_object_clones":0,"num_object_copies":20100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10050,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7553,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'10924","reported_seq":11062,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:47.845232+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:02:47.845232+0000","last_peered":"2024-09-10T16:02:47.845232+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:02:47.845232+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"36'900","ondisk_log_start":"36'900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41521635,"num_objects":10138,"num_object_clones":0,"num_object_copies":20276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10138,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7536,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5767},{"shards":"0,2","objects":3980}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'10967","reported_seq":11105,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:47.860590+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:02:47.860590+0000","last_peered":"2024-09-10T16:02:47.860590+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:02:47.860590+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"36'900","ondisk_log_start":"36'900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41432006,"num_objects":10117,"num_object_clones":0,"num_object_copies":20234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10117,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":7466,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5880},{"shards":"0,2","objects":3814}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'11033","reported_seq":11161,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:47.860367+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:02:47.860367+0000","last_peered":"2024-09-10T16:02:47.860367+0000","last_clean":"2024-09-10T16:02:47.860367+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:02:47.860367+0000","last_undegraded":"2024-09-10T16:02:47.860367+0000","last_fullsized":"2024-09-10T16:02:47.860367+0000","mapping_epoch":34,"log_start":"36'1000","ondisk_log_start":"36'1000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42110976,"num_objects":10281,"num_object_clones":0,"num_object_copies":20562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10281,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7880,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104219+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:02:08.104219+0000","last_peered":"2024-09-10T16:02:08.104219+0000","last_clean":"2024-09-10T16:02:08.104219+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:02:08.104219+0000","last_undegraded":"2024-09-10T16:02:08.104219+0000","last_fullsized":"2024-09-10T16:02:08.104219+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'10555","reported_seq":10695,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:44.727937+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:02:44.727937+0000","last_peered":"2024-09-10T16:02:44.727937+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:02:44.727937+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"36'500","ondisk_log_start":"36'500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40559075,"num_objects":9903,"num_object_clones":0,"num_object_copies":19806,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9903,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8374,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3800},{"shards":"2","objects":5780}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103774+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:02:08.103774+0000","last_peered":"2024-09-10T16:02:08.103774+0000","last_clean":"2024-09-10T16:02:08.103774+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:02:08.103774+0000","last_undegraded":"2024-09-10T16:02:08.103774+0000","last_fullsized":"2024-09-10T16:02:08.103774+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'10860","reported_seq":10990,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:47.783480+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:02:47.783480+0000","last_peered":"2024-09-10T16:02:47.783480+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:02:47.783480+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"36'800","ondisk_log_start":"36'800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41164800,"num_objects":10050,"num_object_clones":0,"num_object_copies":20100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10050,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7364,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5795},{"shards":"0,2","objects":3854}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":104,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.105055+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:02:08.105055+0000","last_peered":"2024-09-10T16:02:08.105055+0000","last_clean":"2024-09-10T16:02:08.105055+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:02:08.105055+0000","last_undegraded":"2024-09-10T16:02:08.105055+0000","last_fullsized":"2024-09-10T16:02:08.105055+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'10711","reported_seq":10846,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:44.765563+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:02:44.765563+0000","last_peered":"2024-09-10T16:02:44.765563+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:02:44.765563+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"36'700","ondisk_log_start":"36'700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41247203,"num_objects":10071,"num_object_clones":0,"num_object_copies":20142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10071,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8391,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3986},{"shards":"2","objects":5767}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104572+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:02:08.104572+0000","last_peered":"2024-09-10T16:02:08.104572+0000","last_clean":"2024-09-10T16:02:08.104572+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:02:08.104572+0000","last_undegraded":"2024-09-10T16:02:08.104572+0000","last_fullsized":"2024-09-10T16:02:08.104572+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'10938","reported_seq":11067,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:44.889173+0000","last_change":"2024-09-10T15:59:16.174113+0000","last_active":"2024-09-10T16:02:44.889173+0000","last_peered":"2024-09-10T16:02:44.889173+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:02:44.889173+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"36'900","ondisk_log_start":"36'900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41910272,"num_objects":10232,"num_object_clones":0,"num_object_copies":20464,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4354,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10232,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8492,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4002},{"shards":"2","objects":5880}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'10886","reported_seq":11020,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:47.911405+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:02:47.911405+0000","last_peered":"2024-09-10T16:02:47.911405+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:02:47.911405+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"36'800","ondisk_log_start":"36'800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40763392,"num_objects":9952,"num_object_clones":0,"num_object_copies":19904,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9952,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7517,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5648},{"shards":"0,2","objects":3838}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103276+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:02:08.103276+0000","last_peered":"2024-09-10T16:02:08.103276+0000","last_clean":"2024-09-10T16:02:08.103276+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:02:08.103276+0000","last_undegraded":"2024-09-10T16:02:08.103276+0000","last_fullsized":"2024-09-10T16:02:08.103276+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'10703","reported_seq":12270,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T16:02:47.900233+0000","last_change":"2024-09-10T16:02:06.480033+0000","last_active":"2024-09-10T16:02:47.900233+0000","last_peered":"2024-09-10T16:02:47.900233+0000","last_clean":"2024-09-10T15:59:14.115595+0000","last_became_active":"2024-09-10T15:59:16.244089+0000","last_became_peered":"2024-09-10T15:59:16.244089+0000","last_unstale":"2024-09-10T16:02:47.900233+0000","last_undegraded":"2024-09-10T16:02:47.900233+0000","last_fullsized":"2024-09-10T15:59:15.774437+0000","mapping_epoch":42,"log_start":"36'700","ondisk_log_start":"36'700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40529920,"num_objects":9895,"num_object_clones":0,"num_object_copies":19790,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3491,"num_objects_unfound":0,"num_objects_dirty":9895,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7093,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":712,"num_bytes_recovered":2916352,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6404},{"shards":"0,2","objects":3491}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103831+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:02:08.103831+0000","last_peered":"2024-09-10T16:02:08.103831+0000","last_clean":"2024-09-10T16:02:08.103831+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:02:08.103831+0000","last_undegraded":"2024-09-10T16:02:08.103831+0000","last_fullsized":"2024-09-10T16:02:08.103831+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'10935","reported_seq":11077,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:47.860631+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:02:47.860631+0000","last_peered":"2024-09-10T16:02:47.860631+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:02:47.860631+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"36'900","ondisk_log_start":"36'900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41422848,"num_objects":10113,"num_object_clones":0,"num_object_copies":20226,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10113,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7190,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5742},{"shards":"0,2","objects":3962}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104431+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:02:08.104431+0000","last_peered":"2024-09-10T16:02:08.104431+0000","last_clean":"2024-09-10T16:02:08.104431+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:02:08.104431+0000","last_undegraded":"2024-09-10T16:02:08.104431+0000","last_fullsized":"2024-09-10T16:02:08.104431+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104410+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:02:08.104410+0000","last_peered":"2024-09-10T16:02:08.104410+0000","last_clean":"2024-09-10T16:02:08.104410+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:02:08.104410+0000","last_undegraded":"2024-09-10T16:02:08.104410+0000","last_fullsized":"2024-09-10T16:02:08.104410+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'11009","reported_seq":11146,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:47.903715+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:02:47.903715+0000","last_peered":"2024-09-10T16:02:47.903715+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:02:47.903715+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"36'1000","ondisk_log_start":"36'1000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41735110,"num_objects":10191,"num_object_clones":0,"num_object_copies":20382,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10191,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7172,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5868},{"shards":"0,2","objects":3919}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":659686134,"num_objects":161066,"num_object_clones":0,"num_object_copies":322132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42911,"num_objects_misplaced":3491,"num_objects_unfound":0,"num_objects_dirty":161066,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":131040,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4984,"num_bytes_recovered":20410368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1355862016,"data_stored":1355702126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160666,"ondisk_log_size":160666,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986233,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":746204,"kb_used_data":461832,"kb_used_omap":4,"kb_used_meta":284347,"kb_avail":93625636,"statfs":{"total":96636764160,"available":95872651264,"internally_reserved":0,"allocated":472915968,"data_stored":472623735,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":291172010},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,3,0,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659179,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":557164,"kb_used_data":313052,"kb_used_omap":4,"kb_used_meta":244091,"kb_avail":93814676,"statfs":{"total":96636764160,"available":96066228224,"internally_reserved":0,"allocated":320565248,"data_stored":320225582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":249949870},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,1,4,4,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182462,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":890620,"kb_used_data":552172,"kb_used_omap":4,"kb_used_meta":338427,"kb_avail":93481220,"statfs":{"total":96636764160,"available":95724769280,"internally_reserved":0,"allocated":565424128,"data_stored":565131890,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":346549930},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,0,3,2,9,5,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":563933184,"data_stored":563897054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":320184320,"data_stored":320100762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":471744512,"data_stored":471704310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:02:53.077 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:02:53.639 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:02:53.639 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:02:53.667 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":207,"stamp":"2024-09-10T16:02:53.622822+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":650515209,"num_objects":158829,"num_object_clones":0,"num_object_copies":317658,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42911,"num_objects_misplaced":3407,"num_objects_unfound":0,"num_objects_dirty":158829,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":133281,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5070,"num_bytes_recovered":20754451,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161008,"ondisk_log_size":161008,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2194840,"kb_used_data":1315428,"kb_used_omap":12,"kb_used_meta":879347,"kb_avail":280920680,"statfs":{"total":289910292480,"available":287662776320,"internally_reserved":0,"allocated":1346998272,"data_stored":1346074135,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":900451330},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,0,6,5,8,7,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-23982080,"num_objects":-5855,"num_object_clones":0,"num_object_copies":-11710,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-250,"num_objects_unfound":0,"num_objects_dirty":-5855,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5855,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":250,"num_bytes_recovered":1024000,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004028"},"pg_stats":[{"pgid":"2.f","version":"45'10862","reported_seq":10937,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:52.853923+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:02:52.853923+0000","last_peered":"2024-09-10T16:02:52.853923+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:02:52.853923+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"36'800","ondisk_log_start":"36'800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40181760,"num_objects":9810,"num_object_clones":0,"num_object_copies":19620,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9810,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10862,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5551},{"shards":"0,2","objects":3732}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'10887","reported_seq":10999,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:49.892487+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:02:49.892487+0000","last_peered":"2024-09-10T16:02:49.892487+0000","last_clean":"2024-09-10T16:02:49.892487+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:02:49.892487+0000","last_undegraded":"2024-09-10T16:02:49.892487+0000","last_fullsized":"2024-09-10T16:02:49.892487+0000","mapping_epoch":35,"log_start":"36'800","ondisk_log_start":"36'800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40784838,"num_objects":9959,"num_object_clones":0,"num_object_copies":19918,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9959,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":8992,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'10897","reported_seq":19575,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:50.452379+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:02:50.452379+0000","last_peered":"2024-09-10T16:02:50.452379+0000","last_clean":"2024-09-10T16:02:50.452379+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:02:50.452379+0000","last_undegraded":"2024-09-10T16:02:50.452379+0000","last_fullsized":"2024-09-10T16:02:50.452379+0000","mapping_epoch":44,"log_start":"36'800","ondisk_log_start":"36'800","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40550424,"num_objects":9901,"num_object_clones":0,"num_object_copies":19802,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9901,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8901,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'10792","reported_seq":10871,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:49.852119+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:02:49.852119+0000","last_peered":"2024-09-10T16:02:49.852119+0000","last_clean":"2024-09-10T16:02:49.852119+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:02:49.852119+0000","last_undegraded":"2024-09-10T16:02:49.852119+0000","last_fullsized":"2024-09-10T16:02:49.852119+0000","mapping_epoch":35,"log_start":"36'700","ondisk_log_start":"36'700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40419328,"num_objects":9868,"num_object_clones":0,"num_object_copies":19736,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9868,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10792,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'10981","reported_seq":11108,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:52.980392+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:02:52.980392+0000","last_peered":"2024-09-10T16:02:52.980392+0000","last_clean":"2024-09-10T16:02:52.980392+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:02:52.980392+0000","last_undegraded":"2024-09-10T16:02:52.980392+0000","last_fullsized":"2024-09-10T16:02:52.980392+0000","mapping_epoch":34,"log_start":"36'900","ondisk_log_start":"36'900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40591843,"num_objects":9911,"num_object_clones":0,"num_object_copies":19822,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9911,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7692,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'11070","reported_seq":11208,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:52.980382+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:02:52.980382+0000","last_peered":"2024-09-10T16:02:52.980382+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:02:52.980382+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"36'1000","ondisk_log_start":"36'1000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40923619,"num_objects":9992,"num_object_clones":0,"num_object_copies":19984,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9992,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7682,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5621},{"shards":"0,2","objects":3834}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'11108","reported_seq":11246,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:52.896766+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:02:52.896766+0000","last_peered":"2024-09-10T16:02:52.896766+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:02:52.896766+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"36'1100","ondisk_log_start":"36'1100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40854470,"num_objects":9976,"num_object_clones":0,"num_object_copies":19952,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9976,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":7607,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5739},{"shards":"0,2","objects":3673}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'11176","reported_seq":11306,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:52.953381+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:02:52.953381+0000","last_peered":"2024-09-10T16:02:52.953381+0000","last_clean":"2024-09-10T16:02:52.953381+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:02:52.953381+0000","last_undegraded":"2024-09-10T16:02:52.953381+0000","last_fullsized":"2024-09-10T16:02:52.953381+0000","mapping_epoch":34,"log_start":"36'1100","ondisk_log_start":"36'1100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41525248,"num_objects":10138,"num_object_clones":0,"num_object_copies":20276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10138,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8023,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104219+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:02:08.104219+0000","last_peered":"2024-09-10T16:02:08.104219+0000","last_clean":"2024-09-10T16:02:08.104219+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:02:08.104219+0000","last_undegraded":"2024-09-10T16:02:08.104219+0000","last_fullsized":"2024-09-10T16:02:08.104219+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'10704","reported_seq":10844,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:49.906783+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:02:49.906783+0000","last_peered":"2024-09-10T16:02:49.906783+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:02:49.906783+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"36'700","ondisk_log_start":"36'700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39948771,"num_objects":9754,"num_object_clones":0,"num_object_copies":19508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9754,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8523,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3651},{"shards":"2","objects":5631}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103774+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:02:08.103774+0000","last_peered":"2024-09-10T16:02:08.103774+0000","last_clean":"2024-09-10T16:02:08.103774+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:02:08.103774+0000","last_undegraded":"2024-09-10T16:02:08.103774+0000","last_fullsized":"2024-09-10T16:02:08.103774+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'10990","reported_seq":11119,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:52.990592+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:02:52.990592+0000","last_peered":"2024-09-10T16:02:52.990592+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:02:52.990592+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"36'900","ondisk_log_start":"36'900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40632320,"num_objects":9920,"num_object_clones":0,"num_object_copies":19840,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9920,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7494,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5665},{"shards":"0,2","objects":3724}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":104,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.105055+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:02:08.105055+0000","last_peered":"2024-09-10T16:02:08.105055+0000","last_clean":"2024-09-10T16:02:08.105055+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:02:08.105055+0000","last_undegraded":"2024-09-10T16:02:08.105055+0000","last_fullsized":"2024-09-10T16:02:08.105055+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'10833","reported_seq":10968,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:49.799157+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:02:49.799157+0000","last_peered":"2024-09-10T16:02:49.799157+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:02:49.799157+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"36'800","ondisk_log_start":"36'800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40747491,"num_objects":9949,"num_object_clones":0,"num_object_copies":19898,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9949,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8513,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3864},{"shards":"2","objects":5645}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104572+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:02:08.104572+0000","last_peered":"2024-09-10T16:02:08.104572+0000","last_clean":"2024-09-10T16:02:08.104572+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:02:08.104572+0000","last_undegraded":"2024-09-10T16:02:08.104572+0000","last_fullsized":"2024-09-10T16:02:08.104572+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'11093","reported_seq":11222,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:49.844012+0000","last_change":"2024-09-10T15:59:16.174113+0000","last_active":"2024-09-10T16:02:49.844012+0000","last_peered":"2024-09-10T16:02:49.844012+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:02:49.844012+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"36'1000","ondisk_log_start":"36'1000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41275392,"num_objects":10077,"num_object_clones":0,"num_object_copies":20154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4354,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10077,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8647,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3847},{"shards":"2","objects":5725}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'11024","reported_seq":11159,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:52.987533+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:02:52.987533+0000","last_peered":"2024-09-10T16:02:52.987533+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:02:52.987533+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"36'1000","ondisk_log_start":"36'1000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40198144,"num_objects":9814,"num_object_clones":0,"num_object_copies":19628,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9814,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7655,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5510},{"shards":"0,2","objects":3700}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103276+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:02:08.103276+0000","last_peered":"2024-09-10T16:02:08.103276+0000","last_clean":"2024-09-10T16:02:08.103276+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:02:08.103276+0000","last_undegraded":"2024-09-10T16:02:08.103276+0000","last_fullsized":"2024-09-10T16:02:08.103276+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'10837","reported_seq":12573,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T16:02:53.017053+0000","last_change":"2024-09-10T16:02:06.480033+0000","last_active":"2024-09-10T16:02:53.017053+0000","last_peered":"2024-09-10T16:02:53.017053+0000","last_clean":"2024-09-10T15:59:14.115595+0000","last_became_active":"2024-09-10T15:59:16.244089+0000","last_became_peered":"2024-09-10T15:59:16.244089+0000","last_unstale":"2024-09-10T16:02:53.017053+0000","last_undegraded":"2024-09-10T16:02:53.017053+0000","last_fullsized":"2024-09-10T15:59:15.774437+0000","mapping_epoch":42,"log_start":"36'800","ondisk_log_start":"36'800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39981056,"num_objects":9761,"num_object_clones":0,"num_object_copies":19522,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3407,"num_objects_unfound":0,"num_objects_dirty":9761,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7227,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":796,"num_bytes_recovered":3260416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6354},{"shards":"0,2","objects":3407}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103831+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:02:08.103831+0000","last_peered":"2024-09-10T16:02:08.103831+0000","last_clean":"2024-09-10T16:02:08.103831+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:02:08.103831+0000","last_undegraded":"2024-09-10T16:02:08.103831+0000","last_fullsized":"2024-09-10T16:02:08.103831+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'11088","reported_seq":11230,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:52.976920+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:02:52.976920+0000","last_peered":"2024-09-10T16:02:52.976920+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:02:52.976920+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"36'1000","ondisk_log_start":"36'1000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40796160,"num_objects":9960,"num_object_clones":0,"num_object_copies":19920,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9960,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7343,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5589},{"shards":"0,2","objects":3809}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104431+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:02:08.104431+0000","last_peered":"2024-09-10T16:02:08.104431+0000","last_clean":"2024-09-10T16:02:08.104431+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:02:08.104431+0000","last_undegraded":"2024-09-10T16:02:08.104431+0000","last_fullsized":"2024-09-10T16:02:08.104431+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104410+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:02:08.104410+0000","last_peered":"2024-09-10T16:02:08.104410+0000","last_clean":"2024-09-10T16:02:08.104410+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:02:08.104410+0000","last_undegraded":"2024-09-10T16:02:08.104410+0000","last_fullsized":"2024-09-10T16:02:08.104410+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'11163","reported_seq":11307,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:52.890061+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:02:52.890061+0000","last_peered":"2024-09-10T16:02:52.890061+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:02:52.890061+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"36'1100","ondisk_log_start":"36'1100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41104326,"num_objects":10037,"num_object_clones":0,"num_object_copies":20074,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10037,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7326,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5714},{"shards":"0,2","objects":3765}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":650515190,"num_objects":158827,"num_object_clones":0,"num_object_copies":317654,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42911,"num_objects_misplaced":3407,"num_objects_unfound":0,"num_objects_dirty":158827,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":133279,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5068,"num_bytes_recovered":20754432,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1343672320,"data_stored":1343512430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161005,"ondisk_log_size":161005,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986234,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":745880,"kb_used_data":457352,"kb_used_omap":4,"kb_used_meta":288507,"kb_avail":93625960,"statfs":{"total":96636764160,"available":95872983040,"internally_reserved":0,"allocated":468328448,"data_stored":468036215,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":295431850},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,2,2,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659180,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":560728,"kb_used_data":312456,"kb_used_omap":4,"kb_used_meta":248251,"kb_avail":93811112,"statfs":{"total":96636764160,"available":96062578688,"internally_reserved":0,"allocated":319954944,"data_stored":319615278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":254209710},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,3,3,2,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182463,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":888232,"kb_used_data":545620,"kb_used_omap":4,"kb_used_meta":342587,"kb_avail":93483608,"statfs":{"total":96636764160,"available":95727214592,"internally_reserved":0,"allocated":558714880,"data_stored":558422642,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":350809770},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,0,3,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":556961792,"data_stored":556925662,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":319459328,"data_stored":319375770,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":467251200,"data_stored":467210998,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:02:53.669 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:02:54.210 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:02:54.212 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:02:54.231 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":207,"stamp":"2024-09-10T16:02:53.622822+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":650515209,"num_objects":158829,"num_object_clones":0,"num_object_copies":317658,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42911,"num_objects_misplaced":3407,"num_objects_unfound":0,"num_objects_dirty":158829,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":133281,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5070,"num_bytes_recovered":20754451,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161008,"ondisk_log_size":161008,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2194840,"kb_used_data":1315428,"kb_used_omap":12,"kb_used_meta":879347,"kb_avail":280920680,"statfs":{"total":289910292480,"available":287662776320,"internally_reserved":0,"allocated":1346998272,"data_stored":1346074135,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":900451330},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,0,6,5,8,7,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-23982080,"num_objects":-5855,"num_object_clones":0,"num_object_copies":-11710,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-250,"num_objects_unfound":0,"num_objects_dirty":-5855,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5855,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":250,"num_bytes_recovered":1024000,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004028"},"pg_stats":[{"pgid":"2.f","version":"45'10862","reported_seq":10937,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:52.853923+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:02:52.853923+0000","last_peered":"2024-09-10T16:02:52.853923+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:02:52.853923+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"36'800","ondisk_log_start":"36'800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40181760,"num_objects":9810,"num_object_clones":0,"num_object_copies":19620,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9810,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10862,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5551},{"shards":"0,2","objects":3732}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'10887","reported_seq":10999,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:49.892487+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:02:49.892487+0000","last_peered":"2024-09-10T16:02:49.892487+0000","last_clean":"2024-09-10T16:02:49.892487+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:02:49.892487+0000","last_undegraded":"2024-09-10T16:02:49.892487+0000","last_fullsized":"2024-09-10T16:02:49.892487+0000","mapping_epoch":35,"log_start":"36'800","ondisk_log_start":"36'800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40784838,"num_objects":9959,"num_object_clones":0,"num_object_copies":19918,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9959,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":8992,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'10897","reported_seq":19575,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:50.452379+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:02:50.452379+0000","last_peered":"2024-09-10T16:02:50.452379+0000","last_clean":"2024-09-10T16:02:50.452379+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:02:50.452379+0000","last_undegraded":"2024-09-10T16:02:50.452379+0000","last_fullsized":"2024-09-10T16:02:50.452379+0000","mapping_epoch":44,"log_start":"36'800","ondisk_log_start":"36'800","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40550424,"num_objects":9901,"num_object_clones":0,"num_object_copies":19802,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9901,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8901,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'10792","reported_seq":10871,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:49.852119+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:02:49.852119+0000","last_peered":"2024-09-10T16:02:49.852119+0000","last_clean":"2024-09-10T16:02:49.852119+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:02:49.852119+0000","last_undegraded":"2024-09-10T16:02:49.852119+0000","last_fullsized":"2024-09-10T16:02:49.852119+0000","mapping_epoch":35,"log_start":"36'700","ondisk_log_start":"36'700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40419328,"num_objects":9868,"num_object_clones":0,"num_object_copies":19736,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9868,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10792,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'10981","reported_seq":11108,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:52.980392+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:02:52.980392+0000","last_peered":"2024-09-10T16:02:52.980392+0000","last_clean":"2024-09-10T16:02:52.980392+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:02:52.980392+0000","last_undegraded":"2024-09-10T16:02:52.980392+0000","last_fullsized":"2024-09-10T16:02:52.980392+0000","mapping_epoch":34,"log_start":"36'900","ondisk_log_start":"36'900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40591843,"num_objects":9911,"num_object_clones":0,"num_object_copies":19822,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9911,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7692,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'11070","reported_seq":11208,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:52.980382+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:02:52.980382+0000","last_peered":"2024-09-10T16:02:52.980382+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:02:52.980382+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"36'1000","ondisk_log_start":"36'1000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40923619,"num_objects":9992,"num_object_clones":0,"num_object_copies":19984,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9992,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7682,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5621},{"shards":"0,2","objects":3834}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'11108","reported_seq":11246,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:52.896766+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:02:52.896766+0000","last_peered":"2024-09-10T16:02:52.896766+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:02:52.896766+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"36'1100","ondisk_log_start":"36'1100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40854470,"num_objects":9976,"num_object_clones":0,"num_object_copies":19952,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9976,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":7607,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5739},{"shards":"0,2","objects":3673}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'11176","reported_seq":11306,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:52.953381+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:02:52.953381+0000","last_peered":"2024-09-10T16:02:52.953381+0000","last_clean":"2024-09-10T16:02:52.953381+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:02:52.953381+0000","last_undegraded":"2024-09-10T16:02:52.953381+0000","last_fullsized":"2024-09-10T16:02:52.953381+0000","mapping_epoch":34,"log_start":"36'1100","ondisk_log_start":"36'1100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41525248,"num_objects":10138,"num_object_clones":0,"num_object_copies":20276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10138,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8023,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104219+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:02:08.104219+0000","last_peered":"2024-09-10T16:02:08.104219+0000","last_clean":"2024-09-10T16:02:08.104219+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:02:08.104219+0000","last_undegraded":"2024-09-10T16:02:08.104219+0000","last_fullsized":"2024-09-10T16:02:08.104219+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'10704","reported_seq":10844,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:49.906783+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:02:49.906783+0000","last_peered":"2024-09-10T16:02:49.906783+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:02:49.906783+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"36'700","ondisk_log_start":"36'700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39948771,"num_objects":9754,"num_object_clones":0,"num_object_copies":19508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9754,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8523,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3651},{"shards":"2","objects":5631}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103774+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:02:08.103774+0000","last_peered":"2024-09-10T16:02:08.103774+0000","last_clean":"2024-09-10T16:02:08.103774+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:02:08.103774+0000","last_undegraded":"2024-09-10T16:02:08.103774+0000","last_fullsized":"2024-09-10T16:02:08.103774+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'10990","reported_seq":11119,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:52.990592+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:02:52.990592+0000","last_peered":"2024-09-10T16:02:52.990592+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:02:52.990592+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"36'900","ondisk_log_start":"36'900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40632320,"num_objects":9920,"num_object_clones":0,"num_object_copies":19840,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9920,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7494,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5665},{"shards":"0,2","objects":3724}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":104,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.105055+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:02:08.105055+0000","last_peered":"2024-09-10T16:02:08.105055+0000","last_clean":"2024-09-10T16:02:08.105055+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:02:08.105055+0000","last_undegraded":"2024-09-10T16:02:08.105055+0000","last_fullsized":"2024-09-10T16:02:08.105055+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'10833","reported_seq":10968,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:49.799157+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:02:49.799157+0000","last_peered":"2024-09-10T16:02:49.799157+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:02:49.799157+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"36'800","ondisk_log_start":"36'800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40747491,"num_objects":9949,"num_object_clones":0,"num_object_copies":19898,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9949,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8513,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3864},{"shards":"2","objects":5645}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104572+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:02:08.104572+0000","last_peered":"2024-09-10T16:02:08.104572+0000","last_clean":"2024-09-10T16:02:08.104572+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:02:08.104572+0000","last_undegraded":"2024-09-10T16:02:08.104572+0000","last_fullsized":"2024-09-10T16:02:08.104572+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'11093","reported_seq":11222,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:49.844012+0000","last_change":"2024-09-10T15:59:16.174113+0000","last_active":"2024-09-10T16:02:49.844012+0000","last_peered":"2024-09-10T16:02:49.844012+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:02:49.844012+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"36'1000","ondisk_log_start":"36'1000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41275392,"num_objects":10077,"num_object_clones":0,"num_object_copies":20154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4354,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10077,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8647,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3847},{"shards":"2","objects":5725}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'11024","reported_seq":11159,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:52.987533+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:02:52.987533+0000","last_peered":"2024-09-10T16:02:52.987533+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:02:52.987533+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"36'1000","ondisk_log_start":"36'1000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40198144,"num_objects":9814,"num_object_clones":0,"num_object_copies":19628,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9814,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7655,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5510},{"shards":"0,2","objects":3700}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103276+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:02:08.103276+0000","last_peered":"2024-09-10T16:02:08.103276+0000","last_clean":"2024-09-10T16:02:08.103276+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:02:08.103276+0000","last_undegraded":"2024-09-10T16:02:08.103276+0000","last_fullsized":"2024-09-10T16:02:08.103276+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'10837","reported_seq":12573,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T16:02:53.017053+0000","last_change":"2024-09-10T16:02:06.480033+0000","last_active":"2024-09-10T16:02:53.017053+0000","last_peered":"2024-09-10T16:02:53.017053+0000","last_clean":"2024-09-10T15:59:14.115595+0000","last_became_active":"2024-09-10T15:59:16.244089+0000","last_became_peered":"2024-09-10T15:59:16.244089+0000","last_unstale":"2024-09-10T16:02:53.017053+0000","last_undegraded":"2024-09-10T16:02:53.017053+0000","last_fullsized":"2024-09-10T15:59:15.774437+0000","mapping_epoch":42,"log_start":"36'800","ondisk_log_start":"36'800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39981056,"num_objects":9761,"num_object_clones":0,"num_object_copies":19522,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3407,"num_objects_unfound":0,"num_objects_dirty":9761,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7227,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":796,"num_bytes_recovered":3260416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6354},{"shards":"0,2","objects":3407}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103831+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:02:08.103831+0000","last_peered":"2024-09-10T16:02:08.103831+0000","last_clean":"2024-09-10T16:02:08.103831+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:02:08.103831+0000","last_undegraded":"2024-09-10T16:02:08.103831+0000","last_fullsized":"2024-09-10T16:02:08.103831+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'11088","reported_seq":11230,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:52.976920+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:02:52.976920+0000","last_peered":"2024-09-10T16:02:52.976920+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:02:52.976920+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"36'1000","ondisk_log_start":"36'1000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40796160,"num_objects":9960,"num_object_clones":0,"num_object_copies":19920,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9960,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7343,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5589},{"shards":"0,2","objects":3809}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104431+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:02:08.104431+0000","last_peered":"2024-09-10T16:02:08.104431+0000","last_clean":"2024-09-10T16:02:08.104431+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:02:08.104431+0000","last_undegraded":"2024-09-10T16:02:08.104431+0000","last_fullsized":"2024-09-10T16:02:08.104431+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104410+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:02:08.104410+0000","last_peered":"2024-09-10T16:02:08.104410+0000","last_clean":"2024-09-10T16:02:08.104410+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:02:08.104410+0000","last_undegraded":"2024-09-10T16:02:08.104410+0000","last_fullsized":"2024-09-10T16:02:08.104410+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'11163","reported_seq":11307,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:52.890061+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:02:52.890061+0000","last_peered":"2024-09-10T16:02:52.890061+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:02:52.890061+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"36'1100","ondisk_log_start":"36'1100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41104326,"num_objects":10037,"num_object_clones":0,"num_object_copies":20074,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10037,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7326,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5714},{"shards":"0,2","objects":3765}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":650515190,"num_objects":158827,"num_object_clones":0,"num_object_copies":317654,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42911,"num_objects_misplaced":3407,"num_objects_unfound":0,"num_objects_dirty":158827,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":133279,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5068,"num_bytes_recovered":20754432,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1343672320,"data_stored":1343512430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161005,"ondisk_log_size":161005,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986234,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":745880,"kb_used_data":457352,"kb_used_omap":4,"kb_used_meta":288507,"kb_avail":93625960,"statfs":{"total":96636764160,"available":95872983040,"internally_reserved":0,"allocated":468328448,"data_stored":468036215,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":295431850},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,2,2,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659180,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":560728,"kb_used_data":312456,"kb_used_omap":4,"kb_used_meta":248251,"kb_avail":93811112,"statfs":{"total":96636764160,"available":96062578688,"internally_reserved":0,"allocated":319954944,"data_stored":319615278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":254209710},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,3,3,2,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182463,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":888232,"kb_used_data":545620,"kb_used_omap":4,"kb_used_meta":342587,"kb_avail":93483608,"statfs":{"total":96636764160,"available":95727214592,"internally_reserved":0,"allocated":558714880,"data_stored":558422642,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":350809770},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,0,3,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":556961792,"data_stored":556925662,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":319459328,"data_stored":319375770,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":467251200,"data_stored":467210998,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:02:54.232 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-10T16:02:54.232 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "45'10862", 'reported_seq': 10937, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:02:52.853923+0000', 'last_change': '2024-09-10T15:59:16.281558+0000', 'last_active': '2024-09-10T16:02:52.853923+0000', 'last_peered': '2024-09-10T16:02:52.853923+0000', 'last_clean': '2024-09-10T15:59:14.132456+0000', 'last_became_active': '2024-09-10T15:59:16.274407+0000', 'last_became_peered': '2024-09-10T15:59:16.274407+0000', 'last_unstale': '2024-09-10T16:02:52.853923+0000', 'last_undegraded': '2024-09-10T15:59:15.851279+0000', 'last_fullsized': '2024-09-10T15:59:15.774356+0000', 'mapping_epoch': 42, 'log_start': "36'800", 'ondisk_log_start': "36'800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'objects_scrubbed': 0, 'log_size': 10062, 'log_dups_size': 0, 'ondisk_log_size': 10062, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40181760, 'num_objects': 9810, 'num_object_clones': 0, 'num_object_copies': 19620, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4259, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9810, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10862, 'num_write_kb': 41344, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5551}, {'shards': '0,2', 'objects': 3732}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:02:54.233 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-10T16:02:54.233 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "45'11070", 'reported_seq': 11208, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:02:52.980382+0000', 'last_change': '2024-09-10T15:59:16.308264+0000', 'last_active': '2024-09-10T16:02:52.980382+0000', 'last_peered': '2024-09-10T16:02:52.980382+0000', 'last_clean': '2024-09-10T15:59:14.111551+0000', 'last_became_active': '2024-09-10T15:59:16.300040+0000', 'last_became_peered': '2024-09-10T15:59:16.300040+0000', 'last_unstale': '2024-09-10T16:02:52.980382+0000', 'last_undegraded': '2024-09-10T15:59:15.878234+0000', 'last_fullsized': '2024-09-10T15:59:15.769328+0000', 'mapping_epoch': 42, 'log_start': "36'1000", 'ondisk_log_start': "36'1000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3442", 'last_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'objects_scrubbed': 3384, 'log_size': 10070, 'log_dups_size': 0, 'ondisk_log_size': 10070, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1292, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40923619, 'num_objects': 9992, 'num_object_clones': 0, 'num_object_copies': 19984, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4372, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9992, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 7682, 'num_write_kb': 28580, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5621}, {'shards': '0,2', 'objects': 3834}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:02:54.233 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-10T16:02:54.233 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "45'11108", 'reported_seq': 11246, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:02:52.896766+0000', 'last_change': '2024-09-10T15:59:16.331899+0000', 'last_active': '2024-09-10T16:02:52.896766+0000', 'last_peered': '2024-09-10T16:02:52.896766+0000', 'last_clean': '2024-09-10T15:59:14.139436+0000', 'last_became_active': '2024-09-10T15:59:16.318715+0000', 'last_became_peered': '2024-09-10T15:59:16.318715+0000', 'last_unstale': '2024-09-10T16:02:52.896766+0000', 'last_undegraded': '2024-09-10T15:59:15.877524+0000', 'last_fullsized': '2024-09-10T15:59:15.769042+0000', 'mapping_epoch': 42, 'log_start': "36'1100", 'ondisk_log_start': "36'1100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3537", 'last_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'objects_scrubbed': 3497, 'log_size': 10008, 'log_dups_size': 0, 'ondisk_log_size': 10008, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1224, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40854470, 'num_objects': 9976, 'num_object_clones': 0, 'num_object_copies': 19952, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4237, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9976, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 7607, 'num_write_kb': 28172, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5739}, {'shards': '0,2', 'objects': 3673}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:02:54.233 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-10T16:02:54.233 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "45'10704", 'reported_seq': 10844, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:02:49.906783+0000', 'last_change': '2024-09-10T15:59:16.175680+0000', 'last_active': '2024-09-10T16:02:49.906783+0000', 'last_peered': '2024-09-10T16:02:49.906783+0000', 'last_clean': '2024-09-10T15:59:14.088187+0000', 'last_became_active': '2024-09-10T15:59:16.165810+0000', 'last_became_peered': '2024-09-10T15:59:16.165810+0000', 'last_unstale': '2024-09-10T16:02:49.906783+0000', 'last_undegraded': '2024-09-10T15:59:15.822098+0000', 'last_fullsized': '2024-09-10T15:59:15.769123+0000', 'mapping_epoch': 42, 'log_start': "36'700", 'ondisk_log_start': "36'700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2218", 'last_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'objects_scrubbed': 2175, 'log_size': 10004, 'log_dups_size': 0, 'ondisk_log_size': 10004, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 825, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39948771, 'num_objects': 9754, 'num_object_clones': 0, 'num_object_copies': 19508, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4124, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9754, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 8523, 'num_write_kb': 32204, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3651}, {'shards': '2', 'objects': 5631}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:02:54.233 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-10T16:02:54.233 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "45'10990", 'reported_seq': 11119, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:02:52.990592+0000', 'last_change': '2024-09-10T15:59:16.277742+0000', 'last_active': '2024-09-10T16:02:52.990592+0000', 'last_peered': '2024-09-10T16:02:52.990592+0000', 'last_clean': '2024-09-10T15:59:14.129081+0000', 'last_became_active': '2024-09-10T15:59:16.269835+0000', 'last_became_peered': '2024-09-10T15:59:16.269835+0000', 'last_unstale': '2024-09-10T16:02:52.990592+0000', 'last_undegraded': '2024-09-10T15:59:15.860857+0000', 'last_fullsized': '2024-09-10T15:59:15.771410+0000', 'mapping_epoch': 42, 'log_start': "36'900", 'ondisk_log_start': "36'900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3557", 'last_scrub_stamp': '2024-09-10T15:59:02.493678+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:02.493678+0000', 'objects_scrubbed': 3488, 'log_size': 10090, 'log_dups_size': 0, 'ondisk_log_size': 10090, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1472, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40632320, 'num_objects': 9920, 'num_object_clones': 0, 'num_object_copies': 19840, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4257, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9920, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7494, 'num_write_kb': 27852, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5665}, {'shards': '0,2', 'objects': 3724}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:02:54.233 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-10T16:02:54.234 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "45'10833", 'reported_seq': 10968, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:02:49.799157+0000', 'last_change': '2024-09-10T15:59:16.284311+0000', 'last_active': '2024-09-10T16:02:49.799157+0000', 'last_peered': '2024-09-10T16:02:49.799157+0000', 'last_clean': '2024-09-10T15:59:14.116041+0000', 'last_became_active': '2024-09-10T15:59:16.274543+0000', 'last_became_peered': '2024-09-10T15:59:16.274543+0000', 'last_unstale': '2024-09-10T16:02:49.799157+0000', 'last_undegraded': '2024-09-10T15:59:15.812484+0000', 'last_fullsized': '2024-09-10T15:59:15.770885+0000', 'mapping_epoch': 42, 'log_start': "36'800", 'ondisk_log_start': "36'800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2356", 'last_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'objects_scrubbed': 2316, 'log_size': 10033, 'log_dups_size': 0, 'ondisk_log_size': 10033, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 783, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40747491, 'num_objects': 9949, 'num_object_clones': 0, 'num_object_copies': 19898, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9949, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 8513, 'num_write_kb': 32292, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3864}, {'shards': '2', 'objects': 5645}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:02:54.234 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-10T16:02:54.234 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "45'11093", 'reported_seq': 11222, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:02:49.844012+0000', 'last_change': '2024-09-10T15:59:16.174113+0000', 'last_active': '2024-09-10T16:02:49.844012+0000', 'last_peered': '2024-09-10T16:02:49.844012+0000', 'last_clean': '2024-09-10T15:59:14.110654+0000', 'last_became_active': '2024-09-10T15:59:16.164645+0000', 'last_became_peered': '2024-09-10T15:59:16.164645+0000', 'last_unstale': '2024-09-10T16:02:49.844012+0000', 'last_undegraded': '2024-09-10T15:59:15.803593+0000', 'last_fullsized': '2024-09-10T15:59:15.769083+0000', 'mapping_epoch': 42, 'log_start': "36'1000", 'ondisk_log_start': "36'1000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2486", 'last_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'objects_scrubbed': 2440, 'log_size': 10093, 'log_dups_size': 0, 'ondisk_log_size': 10093, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 792, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 41275392, 'num_objects': 10077, 'num_object_clones': 0, 'num_object_copies': 20154, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4354, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10077, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8647, 'num_write_kb': 32568, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3847}, {'shards': '2', 'objects': 5725}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:02:54.234 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-10T16:02:54.234 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "45'11024", 'reported_seq': 11159, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:02:52.987533+0000', 'last_change': '2024-09-10T15:59:16.315641+0000', 'last_active': '2024-09-10T16:02:52.987533+0000', 'last_peered': '2024-09-10T16:02:52.987533+0000', 'last_clean': '2024-09-10T15:59:14.117001+0000', 'last_became_active': '2024-09-10T15:59:16.302245+0000', 'last_became_peered': '2024-09-10T15:59:16.302245+0000', 'last_unstale': '2024-09-10T16:02:52.987533+0000', 'last_undegraded': '2024-09-10T15:59:15.845012+0000', 'last_fullsized': '2024-09-10T15:59:15.770337+0000', 'mapping_epoch': 42, 'log_start': "36'1000", 'ondisk_log_start': "36'1000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3404", 'last_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'objects_scrubbed': 3367, 'log_size': 10024, 'log_dups_size': 0, 'ondisk_log_size': 10024, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 983, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40198144, 'num_objects': 9814, 'num_object_clones': 0, 'num_object_copies': 19628, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9814, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7655, 'num_write_kb': 28204, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5510}, {'shards': '0,2', 'objects': 3700}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:02:54.234 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-10T16:02:54.234 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "45'10837", 'reported_seq': 12573, 'reported_epoch': 45, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-10T16:02:53.017053+0000', 'last_change': '2024-09-10T16:02:06.480033+0000', 'last_active': '2024-09-10T16:02:53.017053+0000', 'last_peered': '2024-09-10T16:02:53.017053+0000', 'last_clean': '2024-09-10T15:59:14.115595+0000', 'last_became_active': '2024-09-10T15:59:16.244089+0000', 'last_became_peered': '2024-09-10T15:59:16.244089+0000', 'last_unstale': '2024-09-10T16:02:53.017053+0000', 'last_undegraded': '2024-09-10T16:02:53.017053+0000', 'last_fullsized': '2024-09-10T15:59:15.774437+0000', 'mapping_epoch': 42, 'log_start': "36'800", 'ondisk_log_start': "36'800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3653", 'last_scrub_stamp': '2024-09-10T15:59:05.298468+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:05.298468+0000', 'objects_scrubbed': 3602, 'log_size': 10037, 'log_dups_size': 0, 'ondisk_log_size': 10037, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1289, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39981056, 'num_objects': 9761, 'num_object_clones': 0, 'num_object_copies': 19522, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 3407, 'num_objects_unfound': 0, 'num_objects_dirty': 9761, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7227, 'num_write_kb': 26772, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 796, 'num_bytes_recovered': 3260416, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6354}, {'shards': '0,2', 'objects': 3407}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:02:54.234 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-10T16:02:54.234 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "45'11088", 'reported_seq': 11230, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:02:52.976920+0000', 'last_change': '2024-09-10T15:59:16.311291+0000', 'last_active': '2024-09-10T16:02:52.976920+0000', 'last_peered': '2024-09-10T16:02:52.976920+0000', 'last_clean': '2024-09-10T15:59:14.116363+0000', 'last_became_active': '2024-09-10T15:59:16.301244+0000', 'last_became_peered': '2024-09-10T15:59:16.301244+0000', 'last_unstale': '2024-09-10T16:02:52.976920+0000', 'last_undegraded': '2024-09-10T15:59:15.883434+0000', 'last_fullsized': '2024-09-10T15:59:15.772756+0000', 'mapping_epoch': 42, 'log_start': "36'1000", 'ondisk_log_start': "36'1000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3789", 'last_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'objects_scrubbed': 3741, 'log_size': 10088, 'log_dups_size': 0, 'ondisk_log_size': 10088, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1440, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40796160, 'num_objects': 9960, 'num_object_clones': 0, 'num_object_copies': 19920, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4373, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9960, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7343, 'num_write_kb': 27124, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5589}, {'shards': '0,2', 'objects': 3809}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:02:54.234 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-10T16:02:54.234 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "45'11163", 'reported_seq': 11307, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:02:52.890061+0000', 'last_change': '2024-09-10T15:59:16.313409+0000', 'last_active': '2024-09-10T16:02:52.890061+0000', 'last_peered': '2024-09-10T16:02:52.890061+0000', 'last_clean': '2024-09-10T15:59:14.129780+0000', 'last_became_active': '2024-09-10T15:59:16.300932+0000', 'last_became_peered': '2024-09-10T15:59:16.300932+0000', 'last_unstale': '2024-09-10T16:02:52.890061+0000', 'last_undegraded': '2024-09-10T15:59:15.887823+0000', 'last_fullsized': '2024-09-10T15:59:15.768513+0000', 'mapping_epoch': 42, 'log_start': "36'1100", 'ondisk_log_start': "36'1100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3880", 'last_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'objects_scrubbed': 3827, 'log_size': 10063, 'log_dups_size': 0, 'ondisk_log_size': 10063, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1397, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 41104326, 'num_objects': 10037, 'num_object_clones': 0, 'num_object_copies': 20074, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4325, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10037, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7326, 'num_write_kb': 27072, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5714}, {'shards': '0,2', 'objects': 3765}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:02:54.235 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T16:02:54.911 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:02:54.911 INFO:teuthology.orchestra.run.smithi115.stdout:{"fsid":"7effee29-df05-4351-9741-ee203564ed81","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 42911/317658 objects degraded (13.509%), 10 pgs degraded, 11 pgs undersized","count":21},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":338,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983954,"num_in_osds":3,"osd_in_since":1725983947,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":158829,"data_bytes":650515209,"bytes_used":2247516160,"bytes_avail":287662776320,"bytes_total":289910292480,"degraded_objects":42911,"degraded_total":317658,"degraded_ratio":0.13508553223907474,"misplaced_objects":3407,"misplaced_total":317658,"misplaced_ratio":0.010725371311284461,"recovering_objects_per_sec":20,"recovering_bytes_per_sec":85304,"recovering_keys_per_sec":0,"num_objects_recovered":250,"num_bytes_recovered":1024000,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":487},"fsmap":{"epoch":1,"btime":"2024-09-10T15:57:14:967120+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T16:00:13.560776+0000","services":{}},"progress_events":{}} 2024-09-10T16:02:54.947 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-10T16:02:54.947 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:02:55.544 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:02:55.545 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:02:55.573 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":207,"stamp":"2024-09-10T16:02:53.622822+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":650515209,"num_objects":158829,"num_object_clones":0,"num_object_copies":317658,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42911,"num_objects_misplaced":3407,"num_objects_unfound":0,"num_objects_dirty":158829,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":133281,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5070,"num_bytes_recovered":20754451,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161008,"ondisk_log_size":161008,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2194840,"kb_used_data":1315428,"kb_used_omap":12,"kb_used_meta":879347,"kb_avail":280920680,"statfs":{"total":289910292480,"available":287662776320,"internally_reserved":0,"allocated":1346998272,"data_stored":1346074135,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":900451330},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,0,6,5,8,7,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-23982080,"num_objects":-5855,"num_object_clones":0,"num_object_copies":-11710,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-250,"num_objects_unfound":0,"num_objects_dirty":-5855,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5855,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":250,"num_bytes_recovered":1024000,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004028"},"pg_stats":[{"pgid":"2.f","version":"45'10862","reported_seq":10937,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:52.853923+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:02:52.853923+0000","last_peered":"2024-09-10T16:02:52.853923+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:02:52.853923+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"36'800","ondisk_log_start":"36'800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40181760,"num_objects":9810,"num_object_clones":0,"num_object_copies":19620,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9810,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10862,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5551},{"shards":"0,2","objects":3732}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'10887","reported_seq":10999,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:49.892487+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:02:49.892487+0000","last_peered":"2024-09-10T16:02:49.892487+0000","last_clean":"2024-09-10T16:02:49.892487+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:02:49.892487+0000","last_undegraded":"2024-09-10T16:02:49.892487+0000","last_fullsized":"2024-09-10T16:02:49.892487+0000","mapping_epoch":35,"log_start":"36'800","ondisk_log_start":"36'800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40784838,"num_objects":9959,"num_object_clones":0,"num_object_copies":19918,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9959,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":8992,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'10897","reported_seq":19575,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:50.452379+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:02:50.452379+0000","last_peered":"2024-09-10T16:02:50.452379+0000","last_clean":"2024-09-10T16:02:50.452379+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:02:50.452379+0000","last_undegraded":"2024-09-10T16:02:50.452379+0000","last_fullsized":"2024-09-10T16:02:50.452379+0000","mapping_epoch":44,"log_start":"36'800","ondisk_log_start":"36'800","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40550424,"num_objects":9901,"num_object_clones":0,"num_object_copies":19802,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9901,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8901,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'10792","reported_seq":10871,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:49.852119+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:02:49.852119+0000","last_peered":"2024-09-10T16:02:49.852119+0000","last_clean":"2024-09-10T16:02:49.852119+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:02:49.852119+0000","last_undegraded":"2024-09-10T16:02:49.852119+0000","last_fullsized":"2024-09-10T16:02:49.852119+0000","mapping_epoch":35,"log_start":"36'700","ondisk_log_start":"36'700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40419328,"num_objects":9868,"num_object_clones":0,"num_object_copies":19736,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9868,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10792,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'10981","reported_seq":11108,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:52.980392+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:02:52.980392+0000","last_peered":"2024-09-10T16:02:52.980392+0000","last_clean":"2024-09-10T16:02:52.980392+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:02:52.980392+0000","last_undegraded":"2024-09-10T16:02:52.980392+0000","last_fullsized":"2024-09-10T16:02:52.980392+0000","mapping_epoch":34,"log_start":"36'900","ondisk_log_start":"36'900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40591843,"num_objects":9911,"num_object_clones":0,"num_object_copies":19822,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9911,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7692,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'11070","reported_seq":11208,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:52.980382+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:02:52.980382+0000","last_peered":"2024-09-10T16:02:52.980382+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:02:52.980382+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"36'1000","ondisk_log_start":"36'1000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40923619,"num_objects":9992,"num_object_clones":0,"num_object_copies":19984,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9992,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7682,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5621},{"shards":"0,2","objects":3834}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'11108","reported_seq":11246,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:52.896766+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:02:52.896766+0000","last_peered":"2024-09-10T16:02:52.896766+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:02:52.896766+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"36'1100","ondisk_log_start":"36'1100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40854470,"num_objects":9976,"num_object_clones":0,"num_object_copies":19952,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9976,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":7607,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5739},{"shards":"0,2","objects":3673}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'11176","reported_seq":11306,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:52.953381+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:02:52.953381+0000","last_peered":"2024-09-10T16:02:52.953381+0000","last_clean":"2024-09-10T16:02:52.953381+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:02:52.953381+0000","last_undegraded":"2024-09-10T16:02:52.953381+0000","last_fullsized":"2024-09-10T16:02:52.953381+0000","mapping_epoch":34,"log_start":"36'1100","ondisk_log_start":"36'1100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41525248,"num_objects":10138,"num_object_clones":0,"num_object_copies":20276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10138,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8023,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104219+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:02:08.104219+0000","last_peered":"2024-09-10T16:02:08.104219+0000","last_clean":"2024-09-10T16:02:08.104219+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:02:08.104219+0000","last_undegraded":"2024-09-10T16:02:08.104219+0000","last_fullsized":"2024-09-10T16:02:08.104219+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'10704","reported_seq":10844,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:49.906783+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:02:49.906783+0000","last_peered":"2024-09-10T16:02:49.906783+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:02:49.906783+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"36'700","ondisk_log_start":"36'700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39948771,"num_objects":9754,"num_object_clones":0,"num_object_copies":19508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9754,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8523,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3651},{"shards":"2","objects":5631}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103774+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:02:08.103774+0000","last_peered":"2024-09-10T16:02:08.103774+0000","last_clean":"2024-09-10T16:02:08.103774+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:02:08.103774+0000","last_undegraded":"2024-09-10T16:02:08.103774+0000","last_fullsized":"2024-09-10T16:02:08.103774+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'10990","reported_seq":11119,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:52.990592+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:02:52.990592+0000","last_peered":"2024-09-10T16:02:52.990592+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:02:52.990592+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"36'900","ondisk_log_start":"36'900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40632320,"num_objects":9920,"num_object_clones":0,"num_object_copies":19840,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9920,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7494,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5665},{"shards":"0,2","objects":3724}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":104,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.105055+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:02:08.105055+0000","last_peered":"2024-09-10T16:02:08.105055+0000","last_clean":"2024-09-10T16:02:08.105055+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:02:08.105055+0000","last_undegraded":"2024-09-10T16:02:08.105055+0000","last_fullsized":"2024-09-10T16:02:08.105055+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'10833","reported_seq":10968,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:49.799157+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:02:49.799157+0000","last_peered":"2024-09-10T16:02:49.799157+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:02:49.799157+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"36'800","ondisk_log_start":"36'800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40747491,"num_objects":9949,"num_object_clones":0,"num_object_copies":19898,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9949,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8513,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3864},{"shards":"2","objects":5645}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104572+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:02:08.104572+0000","last_peered":"2024-09-10T16:02:08.104572+0000","last_clean":"2024-09-10T16:02:08.104572+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:02:08.104572+0000","last_undegraded":"2024-09-10T16:02:08.104572+0000","last_fullsized":"2024-09-10T16:02:08.104572+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'11093","reported_seq":11222,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:49.844012+0000","last_change":"2024-09-10T15:59:16.174113+0000","last_active":"2024-09-10T16:02:49.844012+0000","last_peered":"2024-09-10T16:02:49.844012+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:02:49.844012+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"36'1000","ondisk_log_start":"36'1000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41275392,"num_objects":10077,"num_object_clones":0,"num_object_copies":20154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4354,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10077,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8647,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3847},{"shards":"2","objects":5725}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'11024","reported_seq":11159,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:52.987533+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:02:52.987533+0000","last_peered":"2024-09-10T16:02:52.987533+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:02:52.987533+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"36'1000","ondisk_log_start":"36'1000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40198144,"num_objects":9814,"num_object_clones":0,"num_object_copies":19628,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9814,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7655,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5510},{"shards":"0,2","objects":3700}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103276+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:02:08.103276+0000","last_peered":"2024-09-10T16:02:08.103276+0000","last_clean":"2024-09-10T16:02:08.103276+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:02:08.103276+0000","last_undegraded":"2024-09-10T16:02:08.103276+0000","last_fullsized":"2024-09-10T16:02:08.103276+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'10837","reported_seq":12573,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T16:02:53.017053+0000","last_change":"2024-09-10T16:02:06.480033+0000","last_active":"2024-09-10T16:02:53.017053+0000","last_peered":"2024-09-10T16:02:53.017053+0000","last_clean":"2024-09-10T15:59:14.115595+0000","last_became_active":"2024-09-10T15:59:16.244089+0000","last_became_peered":"2024-09-10T15:59:16.244089+0000","last_unstale":"2024-09-10T16:02:53.017053+0000","last_undegraded":"2024-09-10T16:02:53.017053+0000","last_fullsized":"2024-09-10T15:59:15.774437+0000","mapping_epoch":42,"log_start":"36'800","ondisk_log_start":"36'800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39981056,"num_objects":9761,"num_object_clones":0,"num_object_copies":19522,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3407,"num_objects_unfound":0,"num_objects_dirty":9761,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7227,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":796,"num_bytes_recovered":3260416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6354},{"shards":"0,2","objects":3407}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103831+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:02:08.103831+0000","last_peered":"2024-09-10T16:02:08.103831+0000","last_clean":"2024-09-10T16:02:08.103831+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:02:08.103831+0000","last_undegraded":"2024-09-10T16:02:08.103831+0000","last_fullsized":"2024-09-10T16:02:08.103831+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'11088","reported_seq":11230,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:52.976920+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:02:52.976920+0000","last_peered":"2024-09-10T16:02:52.976920+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:02:52.976920+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"36'1000","ondisk_log_start":"36'1000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40796160,"num_objects":9960,"num_object_clones":0,"num_object_copies":19920,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9960,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7343,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5589},{"shards":"0,2","objects":3809}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104431+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:02:08.104431+0000","last_peered":"2024-09-10T16:02:08.104431+0000","last_clean":"2024-09-10T16:02:08.104431+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:02:08.104431+0000","last_undegraded":"2024-09-10T16:02:08.104431+0000","last_fullsized":"2024-09-10T16:02:08.104431+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104410+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:02:08.104410+0000","last_peered":"2024-09-10T16:02:08.104410+0000","last_clean":"2024-09-10T16:02:08.104410+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:02:08.104410+0000","last_undegraded":"2024-09-10T16:02:08.104410+0000","last_fullsized":"2024-09-10T16:02:08.104410+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'11163","reported_seq":11307,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:52.890061+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:02:52.890061+0000","last_peered":"2024-09-10T16:02:52.890061+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:02:52.890061+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"36'1100","ondisk_log_start":"36'1100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41104326,"num_objects":10037,"num_object_clones":0,"num_object_copies":20074,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10037,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7326,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5714},{"shards":"0,2","objects":3765}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":650515190,"num_objects":158827,"num_object_clones":0,"num_object_copies":317654,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42911,"num_objects_misplaced":3407,"num_objects_unfound":0,"num_objects_dirty":158827,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":133279,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5068,"num_bytes_recovered":20754432,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1343672320,"data_stored":1343512430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161005,"ondisk_log_size":161005,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986234,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":745880,"kb_used_data":457352,"kb_used_omap":4,"kb_used_meta":288507,"kb_avail":93625960,"statfs":{"total":96636764160,"available":95872983040,"internally_reserved":0,"allocated":468328448,"data_stored":468036215,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":295431850},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,2,2,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659180,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":560728,"kb_used_data":312456,"kb_used_omap":4,"kb_used_meta":248251,"kb_avail":93811112,"statfs":{"total":96636764160,"available":96062578688,"internally_reserved":0,"allocated":319954944,"data_stored":319615278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":254209710},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,3,3,2,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182463,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":888232,"kb_used_data":545620,"kb_used_omap":4,"kb_used_meta":342587,"kb_avail":93483608,"statfs":{"total":96636764160,"available":95727214592,"internally_reserved":0,"allocated":558714880,"data_stored":558422642,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":350809770},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,0,3,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":556961792,"data_stored":556925662,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":319459328,"data_stored":319375770,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":467251200,"data_stored":467210998,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:02:58.575 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:02:59.146 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:02:59.148 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:02:59.167 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":209,"stamp":"2024-09-10T16:02:57.627041+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":647127817,"num_objects":158002,"num_object_clones":0,"num_object_copies":316004,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42911,"num_objects_misplaced":3407,"num_objects_unfound":0,"num_objects_dirty":158002,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":134108,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5070,"num_bytes_recovered":20754451,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160935,"ondisk_log_size":160935,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2219940,"kb_used_data":1310000,"kb_used_omap":12,"kb_used_meta":909875,"kb_avail":280895580,"statfs":{"total":289910292480,"available":287637073920,"internally_reserved":0,"allocated":1341440000,"data_stored":1340515863,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":931712002},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,3,1,11,8],"upper_bound":128},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18186240,"num_objects":-4440,"num_object_clones":0,"num_object_copies":-8880,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-169,"num_objects_unfound":0,"num_objects_dirty":-4440,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4440,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":169,"num_bytes_recovered":692224,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.007014"},"pg_stats":[{"pgid":"2.f","version":"45'10862","reported_seq":10937,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:52.853923+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:02:52.853923+0000","last_peered":"2024-09-10T16:02:52.853923+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:02:52.853923+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"36'800","ondisk_log_start":"36'800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40181760,"num_objects":9810,"num_object_clones":0,"num_object_copies":19620,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9810,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10862,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5551},{"shards":"0,2","objects":3732}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'11032","reported_seq":11145,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:54.903096+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:02:54.903096+0000","last_peered":"2024-09-10T16:02:54.903096+0000","last_clean":"2024-09-10T16:02:54.903096+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:02:54.903096+0000","last_undegraded":"2024-09-10T16:02:54.903096+0000","last_fullsized":"2024-09-10T16:02:54.903096+0000","mapping_epoch":35,"log_start":"36'1000","ondisk_log_start":"36'1000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40190918,"num_objects":9814,"num_object_clones":0,"num_object_copies":19628,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9814,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":9137,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'11041","reported_seq":19719,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:55.442511+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:02:55.442511+0000","last_peered":"2024-09-10T16:02:55.442511+0000","last_clean":"2024-09-10T16:02:55.442511+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:02:55.442511+0000","last_undegraded":"2024-09-10T16:02:55.442511+0000","last_fullsized":"2024-09-10T16:02:55.442511+0000","mapping_epoch":44,"log_start":"36'1000","ondisk_log_start":"36'1000","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39960600,"num_objects":9757,"num_object_clones":0,"num_object_copies":19514,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9757,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9045,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'10944","reported_seq":11024,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:54.882966+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:02:54.882966+0000","last_peered":"2024-09-10T16:02:54.882966+0000","last_clean":"2024-09-10T16:02:54.882966+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:02:54.882966+0000","last_undegraded":"2024-09-10T16:02:54.882966+0000","last_fullsized":"2024-09-10T16:02:54.882966+0000","mapping_epoch":35,"log_start":"36'900","ondisk_log_start":"36'900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39796736,"num_objects":9716,"num_object_clones":0,"num_object_copies":19432,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9716,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10944,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'10981","reported_seq":11108,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:52.980392+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:02:52.980392+0000","last_peered":"2024-09-10T16:02:52.980392+0000","last_clean":"2024-09-10T16:02:52.980392+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:02:52.980392+0000","last_undegraded":"2024-09-10T16:02:52.980392+0000","last_fullsized":"2024-09-10T16:02:52.980392+0000","mapping_epoch":34,"log_start":"36'900","ondisk_log_start":"36'900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40591843,"num_objects":9911,"num_object_clones":0,"num_object_copies":19822,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9911,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7692,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'11070","reported_seq":11208,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:52.980382+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:02:52.980382+0000","last_peered":"2024-09-10T16:02:52.980382+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:02:52.980382+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"36'1000","ondisk_log_start":"36'1000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40923619,"num_objects":9992,"num_object_clones":0,"num_object_copies":19984,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9992,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7682,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5621},{"shards":"0,2","objects":3834}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'11108","reported_seq":11246,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:52.896766+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:02:52.896766+0000","last_peered":"2024-09-10T16:02:52.896766+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:02:52.896766+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"36'1100","ondisk_log_start":"36'1100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40854470,"num_objects":9976,"num_object_clones":0,"num_object_copies":19952,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9976,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":7607,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5739},{"shards":"0,2","objects":3673}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'11176","reported_seq":11306,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:52.953381+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:02:52.953381+0000","last_peered":"2024-09-10T16:02:52.953381+0000","last_clean":"2024-09-10T16:02:52.953381+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:02:52.953381+0000","last_undegraded":"2024-09-10T16:02:52.953381+0000","last_fullsized":"2024-09-10T16:02:52.953381+0000","mapping_epoch":34,"log_start":"36'1100","ondisk_log_start":"36'1100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41525248,"num_objects":10138,"num_object_clones":0,"num_object_copies":20276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10138,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8023,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104219+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:02:08.104219+0000","last_peered":"2024-09-10T16:02:08.104219+0000","last_clean":"2024-09-10T16:02:08.104219+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:02:08.104219+0000","last_undegraded":"2024-09-10T16:02:08.104219+0000","last_fullsized":"2024-09-10T16:02:08.104219+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'10793","reported_seq":10933,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:54.722205+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:02:54.722205+0000","last_peered":"2024-09-10T16:02:54.722205+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:02:54.722205+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"36'700","ondisk_log_start":"36'700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39584227,"num_objects":9665,"num_object_clones":0,"num_object_copies":19330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9665,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8612,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3562},{"shards":"2","objects":5542}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103774+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:02:08.103774+0000","last_peered":"2024-09-10T16:02:08.103774+0000","last_clean":"2024-09-10T16:02:08.103774+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:02:08.103774+0000","last_undegraded":"2024-09-10T16:02:08.103774+0000","last_fullsized":"2024-09-10T16:02:08.103774+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'10990","reported_seq":11119,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:52.990592+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:02:52.990592+0000","last_peered":"2024-09-10T16:02:52.990592+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:02:52.990592+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"36'900","ondisk_log_start":"36'900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40632320,"num_objects":9920,"num_object_clones":0,"num_object_copies":19840,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9920,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7494,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5665},{"shards":"0,2","objects":3724}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":104,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.105055+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:02:08.105055+0000","last_peered":"2024-09-10T16:02:08.105055+0000","last_clean":"2024-09-10T16:02:08.105055+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:02:08.105055+0000","last_undegraded":"2024-09-10T16:02:08.105055+0000","last_fullsized":"2024-09-10T16:02:08.105055+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'10985","reported_seq":11120,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:54.861223+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:02:54.861223+0000","last_peered":"2024-09-10T16:02:54.861223+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:02:54.861223+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"36'900","ondisk_log_start":"36'900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40124899,"num_objects":9797,"num_object_clones":0,"num_object_copies":19594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9797,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8665,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3712},{"shards":"2","objects":5493}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104572+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:02:08.104572+0000","last_peered":"2024-09-10T16:02:08.104572+0000","last_clean":"2024-09-10T16:02:08.104572+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:02:08.104572+0000","last_undegraded":"2024-09-10T16:02:08.104572+0000","last_fullsized":"2024-09-10T16:02:08.104572+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'11238","reported_seq":11367,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:54.870406+0000","last_change":"2024-09-10T15:59:16.174113+0000","last_active":"2024-09-10T16:02:54.870406+0000","last_peered":"2024-09-10T16:02:54.870406+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:02:54.870406+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"36'1200","ondisk_log_start":"36'1200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40681472,"num_objects":9932,"num_object_clones":0,"num_object_copies":19864,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4354,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9932,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8792,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3702},{"shards":"2","objects":5580}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'11024","reported_seq":11159,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:52.987533+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:02:52.987533+0000","last_peered":"2024-09-10T16:02:52.987533+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:02:52.987533+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"36'1000","ondisk_log_start":"36'1000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40198144,"num_objects":9814,"num_object_clones":0,"num_object_copies":19628,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9814,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7655,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5510},{"shards":"0,2","objects":3700}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103276+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:02:08.103276+0000","last_peered":"2024-09-10T16:02:08.103276+0000","last_clean":"2024-09-10T16:02:08.103276+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:02:08.103276+0000","last_undegraded":"2024-09-10T16:02:08.103276+0000","last_fullsized":"2024-09-10T16:02:08.103276+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'10837","reported_seq":12573,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T16:02:53.017053+0000","last_change":"2024-09-10T16:02:06.480033+0000","last_active":"2024-09-10T16:02:53.017053+0000","last_peered":"2024-09-10T16:02:53.017053+0000","last_clean":"2024-09-10T15:59:14.115595+0000","last_became_active":"2024-09-10T15:59:16.244089+0000","last_became_peered":"2024-09-10T15:59:16.244089+0000","last_unstale":"2024-09-10T16:02:53.017053+0000","last_undegraded":"2024-09-10T16:02:53.017053+0000","last_fullsized":"2024-09-10T15:59:15.774437+0000","mapping_epoch":42,"log_start":"36'800","ondisk_log_start":"36'800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39981056,"num_objects":9761,"num_object_clones":0,"num_object_copies":19522,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3407,"num_objects_unfound":0,"num_objects_dirty":9761,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7227,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":796,"num_bytes_recovered":3260416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6354},{"shards":"0,2","objects":3407}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103831+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:02:08.103831+0000","last_peered":"2024-09-10T16:02:08.103831+0000","last_clean":"2024-09-10T16:02:08.103831+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:02:08.103831+0000","last_undegraded":"2024-09-10T16:02:08.103831+0000","last_fullsized":"2024-09-10T16:02:08.103831+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'11088","reported_seq":11230,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:52.976920+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:02:52.976920+0000","last_peered":"2024-09-10T16:02:52.976920+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:02:52.976920+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"36'1000","ondisk_log_start":"36'1000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40796160,"num_objects":9960,"num_object_clones":0,"num_object_copies":19920,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9960,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7343,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5589},{"shards":"0,2","objects":3809}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104431+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:02:08.104431+0000","last_peered":"2024-09-10T16:02:08.104431+0000","last_clean":"2024-09-10T16:02:08.104431+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:02:08.104431+0000","last_undegraded":"2024-09-10T16:02:08.104431+0000","last_fullsized":"2024-09-10T16:02:08.104431+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104410+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:02:08.104410+0000","last_peered":"2024-09-10T16:02:08.104410+0000","last_clean":"2024-09-10T16:02:08.104410+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:02:08.104410+0000","last_undegraded":"2024-09-10T16:02:08.104410+0000","last_fullsized":"2024-09-10T16:02:08.104410+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'11163","reported_seq":11307,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:52.890061+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:02:52.890061+0000","last_peered":"2024-09-10T16:02:52.890061+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:02:52.890061+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"36'1100","ondisk_log_start":"36'1100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41104326,"num_objects":10037,"num_object_clones":0,"num_object_copies":20074,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10037,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7326,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5714},{"shards":"0,2","objects":3765}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":647127798,"num_objects":158000,"num_object_clones":0,"num_object_copies":316000,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42911,"num_objects_misplaced":3407,"num_objects_unfound":0,"num_objects_dirty":158000,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":134106,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5068,"num_bytes_recovered":20754432,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1338261504,"data_stored":1338101614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160932,"ondisk_log_size":160932,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986235,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":763560,"kb_used_data":452824,"kb_used_omap":4,"kb_used_meta":310715,"kb_avail":93608280,"statfs":{"total":96636764160,"available":95854878720,"internally_reserved":0,"allocated":463691776,"data_stored":463399543,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":318172842},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,0,0,1,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659181,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":568148,"kb_used_data":311556,"kb_used_omap":4,"kb_used_meta":256571,"kb_avail":93803692,"statfs":{"total":96636764160,"available":96054980608,"internally_reserved":0,"allocated":319033344,"data_stored":318693678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":262729390},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,1,7,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182463,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":888232,"kb_used_data":545620,"kb_used_omap":4,"kb_used_meta":342587,"kb_avail":93483608,"statfs":{"total":96636764160,"available":95727214592,"internally_reserved":0,"allocated":558714880,"data_stored":558422642,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":350809770},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,0,3,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":556961792,"data_stored":556925662,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":318640128,"data_stored":318556570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":462659584,"data_stored":462619382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:02:59.168 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:02:59.744 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:02:59.747 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:02:59.772 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":210,"stamp":"2024-09-10T16:02:59.627831+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":641405705,"num_objects":156605,"num_object_clones":0,"num_object_copies":313210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42911,"num_objects_misplaced":3321,"num_objects_unfound":0,"num_objects_dirty":156605,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":135505,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5156,"num_bytes_recovered":21106707,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160732,"ondisk_log_size":160732,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2204928,"kb_used_data":1303304,"kb_used_omap":12,"kb_used_meta":901555,"kb_avail":280910592,"statfs":{"total":289910292480,"available":287652446208,"internally_reserved":0,"allocated":1334583296,"data_stored":1333659159,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":923192322},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,3,1,3,12,11,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-23908352,"num_objects":-5837,"num_object_clones":0,"num_object_copies":-11674,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-255,"num_objects_unfound":0,"num_objects_dirty":-5837,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5837,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":255,"num_bytes_recovered":1044480,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.007046"},"pg_stats":[{"pgid":"2.f","version":"45'11020","reported_seq":11095,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:58.020819+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:02:58.020819+0000","last_peered":"2024-09-10T16:02:58.020819+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:02:58.020819+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"36'1000","ondisk_log_start":"36'1000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39534592,"num_objects":9652,"num_object_clones":0,"num_object_copies":19304,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9652,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11020,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5393},{"shards":"0,2","objects":3574}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'11032","reported_seq":11145,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:54.903096+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:02:54.903096+0000","last_peered":"2024-09-10T16:02:54.903096+0000","last_clean":"2024-09-10T16:02:54.903096+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:02:54.903096+0000","last_undegraded":"2024-09-10T16:02:54.903096+0000","last_fullsized":"2024-09-10T16:02:54.903096+0000","mapping_epoch":35,"log_start":"36'1000","ondisk_log_start":"36'1000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40190918,"num_objects":9814,"num_object_clones":0,"num_object_copies":19628,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9814,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":9137,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'11041","reported_seq":19719,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:55.442511+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:02:55.442511+0000","last_peered":"2024-09-10T16:02:55.442511+0000","last_clean":"2024-09-10T16:02:55.442511+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:02:55.442511+0000","last_undegraded":"2024-09-10T16:02:55.442511+0000","last_fullsized":"2024-09-10T16:02:55.442511+0000","mapping_epoch":44,"log_start":"36'1000","ondisk_log_start":"36'1000","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39960600,"num_objects":9757,"num_object_clones":0,"num_object_copies":19514,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9757,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9045,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'10944","reported_seq":11024,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:54.882966+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:02:54.882966+0000","last_peered":"2024-09-10T16:02:54.882966+0000","last_clean":"2024-09-10T16:02:54.882966+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:02:54.882966+0000","last_undegraded":"2024-09-10T16:02:54.882966+0000","last_fullsized":"2024-09-10T16:02:54.882966+0000","mapping_epoch":35,"log_start":"36'900","ondisk_log_start":"36'900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39796736,"num_objects":9716,"num_object_clones":0,"num_object_copies":19432,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9716,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10944,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'11111","reported_seq":11241,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:57.958960+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:02:57.958960+0000","last_peered":"2024-09-10T16:02:57.958960+0000","last_clean":"2024-09-10T16:02:57.958960+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:02:57.958960+0000","last_undegraded":"2024-09-10T16:02:57.958960+0000","last_fullsized":"2024-09-10T16:02:57.958960+0000","mapping_epoch":34,"log_start":"36'1100","ondisk_log_start":"36'1100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40059363,"num_objects":9781,"num_object_clones":0,"num_object_copies":19562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9781,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7822,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'11200","reported_seq":11338,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:57.913698+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:02:57.913698+0000","last_peered":"2024-09-10T16:02:57.913698+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:02:57.913698+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"36'1100","ondisk_log_start":"36'1100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40391139,"num_objects":9862,"num_object_clones":0,"num_object_copies":19724,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9862,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7812,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5491},{"shards":"0,2","objects":3704}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'11236","reported_seq":11374,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:58.023834+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:02:58.023834+0000","last_peered":"2024-09-10T16:02:58.023834+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:02:58.023834+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"36'1200","ondisk_log_start":"36'1200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40330182,"num_objects":9848,"num_object_clones":0,"num_object_copies":19696,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9848,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":7735,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5611},{"shards":"0,2","objects":3545}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'11328","reported_seq":11458,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:57.981716+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:02:57.981716+0000","last_peered":"2024-09-10T16:02:57.981716+0000","last_clean":"2024-09-10T16:02:57.981716+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:02:57.981716+0000","last_undegraded":"2024-09-10T16:02:57.981716+0000","last_fullsized":"2024-09-10T16:02:57.981716+0000","mapping_epoch":34,"log_start":"36'1300","ondisk_log_start":"36'1300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40902656,"num_objects":9986,"num_object_clones":0,"num_object_copies":19972,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9986,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8175,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104219+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:02:08.104219+0000","last_peered":"2024-09-10T16:02:08.104219+0000","last_clean":"2024-09-10T16:02:08.104219+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:02:08.104219+0000","last_undegraded":"2024-09-10T16:02:08.104219+0000","last_fullsized":"2024-09-10T16:02:08.104219+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'10793","reported_seq":10933,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:54.722205+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:02:54.722205+0000","last_peered":"2024-09-10T16:02:54.722205+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:02:54.722205+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"36'700","ondisk_log_start":"36'700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39584227,"num_objects":9665,"num_object_clones":0,"num_object_copies":19330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9665,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8612,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3562},{"shards":"2","objects":5542}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103774+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:02:08.103774+0000","last_peered":"2024-09-10T16:02:08.103774+0000","last_clean":"2024-09-10T16:02:08.103774+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:02:08.103774+0000","last_undegraded":"2024-09-10T16:02:08.103774+0000","last_fullsized":"2024-09-10T16:02:08.103774+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'11139","reported_seq":11269,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:58.033504+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:02:58.033504+0000","last_peered":"2024-09-10T16:02:58.033504+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:02:58.033504+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"36'1100","ondisk_log_start":"36'1100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40022016,"num_objects":9771,"num_object_clones":0,"num_object_copies":19542,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9771,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7643,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5516},{"shards":"0,2","objects":3575}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":104,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.105055+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:02:08.105055+0000","last_peered":"2024-09-10T16:02:08.105055+0000","last_clean":"2024-09-10T16:02:08.105055+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:02:08.105055+0000","last_undegraded":"2024-09-10T16:02:08.105055+0000","last_fullsized":"2024-09-10T16:02:08.105055+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'10985","reported_seq":11120,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:54.861223+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:02:54.861223+0000","last_peered":"2024-09-10T16:02:54.861223+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:02:54.861223+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"36'900","ondisk_log_start":"36'900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40124899,"num_objects":9797,"num_object_clones":0,"num_object_copies":19594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9797,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8665,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3712},{"shards":"2","objects":5493}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104572+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:02:08.104572+0000","last_peered":"2024-09-10T16:02:08.104572+0000","last_clean":"2024-09-10T16:02:08.104572+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:02:08.104572+0000","last_undegraded":"2024-09-10T16:02:08.104572+0000","last_fullsized":"2024-09-10T16:02:08.104572+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'11238","reported_seq":11367,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:54.870406+0000","last_change":"2024-09-10T15:59:16.174113+0000","last_active":"2024-09-10T16:02:54.870406+0000","last_peered":"2024-09-10T16:02:54.870406+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:02:54.870406+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"36'1200","ondisk_log_start":"36'1200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40681472,"num_objects":9932,"num_object_clones":0,"num_object_copies":19864,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4354,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9932,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8792,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3702},{"shards":"2","objects":5580}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'11155","reported_seq":11289,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:58.015897+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:02:58.015897+0000","last_peered":"2024-09-10T16:02:58.015897+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:02:58.015897+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"36'1100","ondisk_log_start":"36'1100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39661568,"num_objects":9683,"num_object_clones":0,"num_object_copies":19366,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9683,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7786,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5379},{"shards":"0,2","objects":3569}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103276+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:02:08.103276+0000","last_peered":"2024-09-10T16:02:08.103276+0000","last_clean":"2024-09-10T16:02:08.103276+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:02:08.103276+0000","last_undegraded":"2024-09-10T16:02:08.103276+0000","last_fullsized":"2024-09-10T16:02:08.103276+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'10961","reported_seq":12868,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T16:02:58.079770+0000","last_change":"2024-09-10T16:02:06.480033+0000","last_active":"2024-09-10T16:02:58.079770+0000","last_peered":"2024-09-10T16:02:58.079770+0000","last_clean":"2024-09-10T15:59:14.115595+0000","last_became_active":"2024-09-10T15:59:16.244089+0000","last_became_peered":"2024-09-10T15:59:16.244089+0000","last_unstale":"2024-09-10T16:02:58.079770+0000","last_undegraded":"2024-09-10T16:02:58.079770+0000","last_fullsized":"2024-09-10T15:59:15.774437+0000","mapping_epoch":42,"log_start":"36'900","ondisk_log_start":"36'900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39473152,"num_objects":9637,"num_object_clones":0,"num_object_copies":19274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3321,"num_objects_unfound":0,"num_objects_dirty":9637,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7351,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":882,"num_bytes_recovered":3612672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6316},{"shards":"0,2","objects":3321}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103831+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:02:08.103831+0000","last_peered":"2024-09-10T16:02:08.103831+0000","last_clean":"2024-09-10T16:02:08.103831+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:02:08.103831+0000","last_undegraded":"2024-09-10T16:02:08.103831+0000","last_fullsized":"2024-09-10T16:02:08.103831+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'11233","reported_seq":11374,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:58.067267+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:02:58.067267+0000","last_peered":"2024-09-10T16:02:58.067267+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:02:58.067267+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"36'1200","ondisk_log_start":"36'1200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40202240,"num_objects":9815,"num_object_clones":0,"num_object_copies":19630,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9815,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7488,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5444},{"shards":"0,2","objects":3664}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104431+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:02:08.104431+0000","last_peered":"2024-09-10T16:02:08.104431+0000","last_clean":"2024-09-10T16:02:08.104431+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:02:08.104431+0000","last_undegraded":"2024-09-10T16:02:08.104431+0000","last_fullsized":"2024-09-10T16:02:08.104431+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104410+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:02:08.104410+0000","last_peered":"2024-09-10T16:02:08.104410+0000","last_clean":"2024-09-10T16:02:08.104410+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:02:08.104410+0000","last_undegraded":"2024-09-10T16:02:08.104410+0000","last_fullsized":"2024-09-10T16:02:08.104410+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'11313","reported_seq":11455,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:58.044090+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:02:58.044090+0000","last_peered":"2024-09-10T16:02:58.044090+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:02:58.044090+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"36'1300","ondisk_log_start":"36'1300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40489926,"num_objects":9887,"num_object_clones":0,"num_object_copies":19774,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9887,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7476,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5564},{"shards":"0,2","objects":3615}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":641405686,"num_objects":156603,"num_object_clones":0,"num_object_copies":313206,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42911,"num_objects_misplaced":3321,"num_objects_unfound":0,"num_objects_dirty":156603,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":135503,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5154,"num_bytes_recovered":21106688,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1331462144,"data_stored":1331302254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160729,"ondisk_log_size":160729,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986235,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":763560,"kb_used_data":452824,"kb_used_omap":4,"kb_used_meta":310715,"kb_avail":93608280,"statfs":{"total":96636764160,"available":95854878720,"internally_reserved":0,"allocated":463691776,"data_stored":463399543,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":318172842},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,0,0,1,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659181,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":568148,"kb_used_data":311556,"kb_used_omap":4,"kb_used_meta":256571,"kb_avail":93803692,"statfs":{"total":96636764160,"available":96054980608,"internally_reserved":0,"allocated":319033344,"data_stored":318693678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":262729390},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,1,7,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182464,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":873220,"kb_used_data":538924,"kb_used_omap":4,"kb_used_meta":334267,"kb_avail":93498620,"statfs":{"total":96636764160,"available":95742586880,"internally_reserved":0,"allocated":551858176,"data_stored":551565938,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":342290090},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,1,1,2,4,8,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":550162432,"data_stored":550126302,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":318640128,"data_stored":318556570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":462659584,"data_stored":462619382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:02:59.773 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-10T16:02:59.774 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "45'11020", 'reported_seq': 11095, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:02:58.020819+0000', 'last_change': '2024-09-10T15:59:16.281558+0000', 'last_active': '2024-09-10T16:02:58.020819+0000', 'last_peered': '2024-09-10T16:02:58.020819+0000', 'last_clean': '2024-09-10T15:59:14.132456+0000', 'last_became_active': '2024-09-10T15:59:16.274407+0000', 'last_became_peered': '2024-09-10T15:59:16.274407+0000', 'last_unstale': '2024-09-10T16:02:58.020819+0000', 'last_undegraded': '2024-09-10T15:59:15.851279+0000', 'last_fullsized': '2024-09-10T15:59:15.774356+0000', 'mapping_epoch': 42, 'log_start': "36'1000", 'ondisk_log_start': "36'1000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'objects_scrubbed': 0, 'log_size': 10020, 'log_dups_size': 0, 'ondisk_log_size': 10020, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39534592, 'num_objects': 9652, 'num_object_clones': 0, 'num_object_copies': 19304, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4259, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9652, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11020, 'num_write_kb': 41344, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5393}, {'shards': '0,2', 'objects': 3574}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:02:59.774 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-10T16:02:59.774 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "45'11200", 'reported_seq': 11338, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:02:57.913698+0000', 'last_change': '2024-09-10T15:59:16.308264+0000', 'last_active': '2024-09-10T16:02:57.913698+0000', 'last_peered': '2024-09-10T16:02:57.913698+0000', 'last_clean': '2024-09-10T15:59:14.111551+0000', 'last_became_active': '2024-09-10T15:59:16.300040+0000', 'last_became_peered': '2024-09-10T15:59:16.300040+0000', 'last_unstale': '2024-09-10T16:02:57.913698+0000', 'last_undegraded': '2024-09-10T15:59:15.878234+0000', 'last_fullsized': '2024-09-10T15:59:15.769328+0000', 'mapping_epoch': 42, 'log_start': "36'1100", 'ondisk_log_start': "36'1100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3442", 'last_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'objects_scrubbed': 3384, 'log_size': 10100, 'log_dups_size': 0, 'ondisk_log_size': 10100, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1292, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40391139, 'num_objects': 9862, 'num_object_clones': 0, 'num_object_copies': 19724, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4372, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9862, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 7812, 'num_write_kb': 28580, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5491}, {'shards': '0,2', 'objects': 3704}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:02:59.774 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-10T16:02:59.774 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "45'11236", 'reported_seq': 11374, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:02:58.023834+0000', 'last_change': '2024-09-10T15:59:16.331899+0000', 'last_active': '2024-09-10T16:02:58.023834+0000', 'last_peered': '2024-09-10T16:02:58.023834+0000', 'last_clean': '2024-09-10T15:59:14.139436+0000', 'last_became_active': '2024-09-10T15:59:16.318715+0000', 'last_became_peered': '2024-09-10T15:59:16.318715+0000', 'last_unstale': '2024-09-10T16:02:58.023834+0000', 'last_undegraded': '2024-09-10T15:59:15.877524+0000', 'last_fullsized': '2024-09-10T15:59:15.769042+0000', 'mapping_epoch': 42, 'log_start': "36'1200", 'ondisk_log_start': "36'1200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3537", 'last_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'objects_scrubbed': 3497, 'log_size': 10036, 'log_dups_size': 0, 'ondisk_log_size': 10036, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1224, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40330182, 'num_objects': 9848, 'num_object_clones': 0, 'num_object_copies': 19696, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4237, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9848, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 7735, 'num_write_kb': 28172, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5611}, {'shards': '0,2', 'objects': 3545}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:02:59.774 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-10T16:02:59.774 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "45'10793", 'reported_seq': 10933, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:02:54.722205+0000', 'last_change': '2024-09-10T15:59:16.175680+0000', 'last_active': '2024-09-10T16:02:54.722205+0000', 'last_peered': '2024-09-10T16:02:54.722205+0000', 'last_clean': '2024-09-10T15:59:14.088187+0000', 'last_became_active': '2024-09-10T15:59:16.165810+0000', 'last_became_peered': '2024-09-10T15:59:16.165810+0000', 'last_unstale': '2024-09-10T16:02:54.722205+0000', 'last_undegraded': '2024-09-10T15:59:15.822098+0000', 'last_fullsized': '2024-09-10T15:59:15.769123+0000', 'mapping_epoch': 42, 'log_start': "36'700", 'ondisk_log_start': "36'700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2218", 'last_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'objects_scrubbed': 2175, 'log_size': 10093, 'log_dups_size': 0, 'ondisk_log_size': 10093, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 825, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39584227, 'num_objects': 9665, 'num_object_clones': 0, 'num_object_copies': 19330, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4124, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9665, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 8612, 'num_write_kb': 32204, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3562}, {'shards': '2', 'objects': 5542}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:02:59.774 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-10T16:02:59.774 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "45'11139", 'reported_seq': 11269, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:02:58.033504+0000', 'last_change': '2024-09-10T15:59:16.277742+0000', 'last_active': '2024-09-10T16:02:58.033504+0000', 'last_peered': '2024-09-10T16:02:58.033504+0000', 'last_clean': '2024-09-10T15:59:14.129081+0000', 'last_became_active': '2024-09-10T15:59:16.269835+0000', 'last_became_peered': '2024-09-10T15:59:16.269835+0000', 'last_unstale': '2024-09-10T16:02:58.033504+0000', 'last_undegraded': '2024-09-10T15:59:15.860857+0000', 'last_fullsized': '2024-09-10T15:59:15.771410+0000', 'mapping_epoch': 42, 'log_start': "36'1100", 'ondisk_log_start': "36'1100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3557", 'last_scrub_stamp': '2024-09-10T15:59:02.493678+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:02.493678+0000', 'objects_scrubbed': 3488, 'log_size': 10039, 'log_dups_size': 0, 'ondisk_log_size': 10039, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1472, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40022016, 'num_objects': 9771, 'num_object_clones': 0, 'num_object_copies': 19542, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4257, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9771, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7643, 'num_write_kb': 27852, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5516}, {'shards': '0,2', 'objects': 3575}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:02:59.774 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-10T16:02:59.775 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "45'10985", 'reported_seq': 11120, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:02:54.861223+0000', 'last_change': '2024-09-10T15:59:16.284311+0000', 'last_active': '2024-09-10T16:02:54.861223+0000', 'last_peered': '2024-09-10T16:02:54.861223+0000', 'last_clean': '2024-09-10T15:59:14.116041+0000', 'last_became_active': '2024-09-10T15:59:16.274543+0000', 'last_became_peered': '2024-09-10T15:59:16.274543+0000', 'last_unstale': '2024-09-10T16:02:54.861223+0000', 'last_undegraded': '2024-09-10T15:59:15.812484+0000', 'last_fullsized': '2024-09-10T15:59:15.770885+0000', 'mapping_epoch': 42, 'log_start': "36'900", 'ondisk_log_start': "36'900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2356", 'last_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'objects_scrubbed': 2316, 'log_size': 10085, 'log_dups_size': 0, 'ondisk_log_size': 10085, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 783, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40124899, 'num_objects': 9797, 'num_object_clones': 0, 'num_object_copies': 19594, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9797, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 8665, 'num_write_kb': 32292, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3712}, {'shards': '2', 'objects': 5493}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:02:59.775 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-10T16:02:59.775 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "45'11238", 'reported_seq': 11367, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:02:54.870406+0000', 'last_change': '2024-09-10T15:59:16.174113+0000', 'last_active': '2024-09-10T16:02:54.870406+0000', 'last_peered': '2024-09-10T16:02:54.870406+0000', 'last_clean': '2024-09-10T15:59:14.110654+0000', 'last_became_active': '2024-09-10T15:59:16.164645+0000', 'last_became_peered': '2024-09-10T15:59:16.164645+0000', 'last_unstale': '2024-09-10T16:02:54.870406+0000', 'last_undegraded': '2024-09-10T15:59:15.803593+0000', 'last_fullsized': '2024-09-10T15:59:15.769083+0000', 'mapping_epoch': 42, 'log_start': "36'1200", 'ondisk_log_start': "36'1200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2486", 'last_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'objects_scrubbed': 2440, 'log_size': 10038, 'log_dups_size': 0, 'ondisk_log_size': 10038, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 792, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40681472, 'num_objects': 9932, 'num_object_clones': 0, 'num_object_copies': 19864, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4354, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9932, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8792, 'num_write_kb': 32568, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3702}, {'shards': '2', 'objects': 5580}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:02:59.775 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-10T16:02:59.775 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "45'11155", 'reported_seq': 11289, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:02:58.015897+0000', 'last_change': '2024-09-10T15:59:16.315641+0000', 'last_active': '2024-09-10T16:02:58.015897+0000', 'last_peered': '2024-09-10T16:02:58.015897+0000', 'last_clean': '2024-09-10T15:59:14.117001+0000', 'last_became_active': '2024-09-10T15:59:16.302245+0000', 'last_became_peered': '2024-09-10T15:59:16.302245+0000', 'last_unstale': '2024-09-10T16:02:58.015897+0000', 'last_undegraded': '2024-09-10T15:59:15.845012+0000', 'last_fullsized': '2024-09-10T15:59:15.770337+0000', 'mapping_epoch': 42, 'log_start': "36'1100", 'ondisk_log_start': "36'1100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3404", 'last_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'objects_scrubbed': 3367, 'log_size': 10055, 'log_dups_size': 0, 'ondisk_log_size': 10055, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 983, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39661568, 'num_objects': 9683, 'num_object_clones': 0, 'num_object_copies': 19366, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9683, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7786, 'num_write_kb': 28204, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5379}, {'shards': '0,2', 'objects': 3569}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:02:59.775 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-10T16:02:59.775 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "45'10961", 'reported_seq': 12868, 'reported_epoch': 45, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-10T16:02:58.079770+0000', 'last_change': '2024-09-10T16:02:06.480033+0000', 'last_active': '2024-09-10T16:02:58.079770+0000', 'last_peered': '2024-09-10T16:02:58.079770+0000', 'last_clean': '2024-09-10T15:59:14.115595+0000', 'last_became_active': '2024-09-10T15:59:16.244089+0000', 'last_became_peered': '2024-09-10T15:59:16.244089+0000', 'last_unstale': '2024-09-10T16:02:58.079770+0000', 'last_undegraded': '2024-09-10T16:02:58.079770+0000', 'last_fullsized': '2024-09-10T15:59:15.774437+0000', 'mapping_epoch': 42, 'log_start': "36'900", 'ondisk_log_start': "36'900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3653", 'last_scrub_stamp': '2024-09-10T15:59:05.298468+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:05.298468+0000', 'objects_scrubbed': 3602, 'log_size': 10061, 'log_dups_size': 0, 'ondisk_log_size': 10061, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1289, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39473152, 'num_objects': 9637, 'num_object_clones': 0, 'num_object_copies': 19274, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 3321, 'num_objects_unfound': 0, 'num_objects_dirty': 9637, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7351, 'num_write_kb': 26772, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 882, 'num_bytes_recovered': 3612672, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6316}, {'shards': '0,2', 'objects': 3321}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:02:59.775 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-10T16:02:59.775 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "45'11233", 'reported_seq': 11374, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:02:58.067267+0000', 'last_change': '2024-09-10T15:59:16.311291+0000', 'last_active': '2024-09-10T16:02:58.067267+0000', 'last_peered': '2024-09-10T16:02:58.067267+0000', 'last_clean': '2024-09-10T15:59:14.116363+0000', 'last_became_active': '2024-09-10T15:59:16.301244+0000', 'last_became_peered': '2024-09-10T15:59:16.301244+0000', 'last_unstale': '2024-09-10T16:02:58.067267+0000', 'last_undegraded': '2024-09-10T15:59:15.883434+0000', 'last_fullsized': '2024-09-10T15:59:15.772756+0000', 'mapping_epoch': 42, 'log_start': "36'1200", 'ondisk_log_start': "36'1200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3789", 'last_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'objects_scrubbed': 3741, 'log_size': 10033, 'log_dups_size': 0, 'ondisk_log_size': 10033, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1440, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40202240, 'num_objects': 9815, 'num_object_clones': 0, 'num_object_copies': 19630, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4373, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9815, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7488, 'num_write_kb': 27124, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5444}, {'shards': '0,2', 'objects': 3664}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:02:59.775 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-10T16:02:59.775 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "45'11313", 'reported_seq': 11455, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:02:58.044090+0000', 'last_change': '2024-09-10T15:59:16.313409+0000', 'last_active': '2024-09-10T16:02:58.044090+0000', 'last_peered': '2024-09-10T16:02:58.044090+0000', 'last_clean': '2024-09-10T15:59:14.129780+0000', 'last_became_active': '2024-09-10T15:59:16.300932+0000', 'last_became_peered': '2024-09-10T15:59:16.300932+0000', 'last_unstale': '2024-09-10T16:02:58.044090+0000', 'last_undegraded': '2024-09-10T15:59:15.887823+0000', 'last_fullsized': '2024-09-10T15:59:15.768513+0000', 'mapping_epoch': 42, 'log_start': "36'1300", 'ondisk_log_start': "36'1300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3880", 'last_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'objects_scrubbed': 3827, 'log_size': 10013, 'log_dups_size': 0, 'ondisk_log_size': 10013, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1397, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40489926, 'num_objects': 9887, 'num_object_clones': 0, 'num_object_copies': 19774, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4325, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9887, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7476, 'num_write_kb': 27072, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5564}, {'shards': '0,2', 'objects': 3615}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:02:59.776 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T16:03:00.467 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:03:00.467 INFO:teuthology.orchestra.run.smithi115.stdout:{"fsid":"7effee29-df05-4351-9741-ee203564ed81","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 42911/313210 objects degraded (13.700%), 10 pgs degraded, 11 pgs undersized","count":21},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":344,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983954,"num_in_osds":3,"osd_in_since":1725983947,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":156605,"data_bytes":641405705,"bytes_used":2257846272,"bytes_avail":287652446208,"bytes_total":289910292480,"degraded_objects":42911,"degraded_total":313210,"degraded_ratio":0.1370039270776795,"misplaced_objects":3321,"misplaced_total":313210,"misplaced_ratio":0.0106031097346828,"recovering_objects_per_sec":21,"recovering_bytes_per_sec":86988,"recovering_keys_per_sec":0,"num_objects_recovered":255,"num_bytes_recovered":1044480,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":486},"fsmap":{"epoch":1,"btime":"2024-09-10T15:57:14:967120+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T16:00:13.560776+0000","services":{}},"progress_events":{}} 2024-09-10T16:03:00.489 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-10T16:03:00.489 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:03:01.088 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:03:01.090 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:03:01.116 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":210,"stamp":"2024-09-10T16:02:59.627831+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":641405705,"num_objects":156605,"num_object_clones":0,"num_object_copies":313210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42911,"num_objects_misplaced":3321,"num_objects_unfound":0,"num_objects_dirty":156605,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":135505,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5156,"num_bytes_recovered":21106707,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160732,"ondisk_log_size":160732,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2204928,"kb_used_data":1303304,"kb_used_omap":12,"kb_used_meta":901555,"kb_avail":280910592,"statfs":{"total":289910292480,"available":287652446208,"internally_reserved":0,"allocated":1334583296,"data_stored":1333659159,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":923192322},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,3,1,3,12,11,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-23908352,"num_objects":-5837,"num_object_clones":0,"num_object_copies":-11674,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-255,"num_objects_unfound":0,"num_objects_dirty":-5837,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5837,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":255,"num_bytes_recovered":1044480,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.007046"},"pg_stats":[{"pgid":"2.f","version":"45'11020","reported_seq":11095,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:58.020819+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:02:58.020819+0000","last_peered":"2024-09-10T16:02:58.020819+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:02:58.020819+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"36'1000","ondisk_log_start":"36'1000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39534592,"num_objects":9652,"num_object_clones":0,"num_object_copies":19304,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9652,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11020,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5393},{"shards":"0,2","objects":3574}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'11032","reported_seq":11145,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:54.903096+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:02:54.903096+0000","last_peered":"2024-09-10T16:02:54.903096+0000","last_clean":"2024-09-10T16:02:54.903096+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:02:54.903096+0000","last_undegraded":"2024-09-10T16:02:54.903096+0000","last_fullsized":"2024-09-10T16:02:54.903096+0000","mapping_epoch":35,"log_start":"36'1000","ondisk_log_start":"36'1000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40190918,"num_objects":9814,"num_object_clones":0,"num_object_copies":19628,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9814,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":9137,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'11041","reported_seq":19719,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:55.442511+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:02:55.442511+0000","last_peered":"2024-09-10T16:02:55.442511+0000","last_clean":"2024-09-10T16:02:55.442511+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:02:55.442511+0000","last_undegraded":"2024-09-10T16:02:55.442511+0000","last_fullsized":"2024-09-10T16:02:55.442511+0000","mapping_epoch":44,"log_start":"36'1000","ondisk_log_start":"36'1000","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39960600,"num_objects":9757,"num_object_clones":0,"num_object_copies":19514,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9757,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9045,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'10944","reported_seq":11024,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:54.882966+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:02:54.882966+0000","last_peered":"2024-09-10T16:02:54.882966+0000","last_clean":"2024-09-10T16:02:54.882966+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:02:54.882966+0000","last_undegraded":"2024-09-10T16:02:54.882966+0000","last_fullsized":"2024-09-10T16:02:54.882966+0000","mapping_epoch":35,"log_start":"36'900","ondisk_log_start":"36'900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39796736,"num_objects":9716,"num_object_clones":0,"num_object_copies":19432,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9716,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10944,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'11111","reported_seq":11241,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:57.958960+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:02:57.958960+0000","last_peered":"2024-09-10T16:02:57.958960+0000","last_clean":"2024-09-10T16:02:57.958960+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:02:57.958960+0000","last_undegraded":"2024-09-10T16:02:57.958960+0000","last_fullsized":"2024-09-10T16:02:57.958960+0000","mapping_epoch":34,"log_start":"36'1100","ondisk_log_start":"36'1100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40059363,"num_objects":9781,"num_object_clones":0,"num_object_copies":19562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9781,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7822,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'11200","reported_seq":11338,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:57.913698+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:02:57.913698+0000","last_peered":"2024-09-10T16:02:57.913698+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:02:57.913698+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"36'1100","ondisk_log_start":"36'1100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40391139,"num_objects":9862,"num_object_clones":0,"num_object_copies":19724,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9862,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7812,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5491},{"shards":"0,2","objects":3704}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'11236","reported_seq":11374,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:58.023834+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:02:58.023834+0000","last_peered":"2024-09-10T16:02:58.023834+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:02:58.023834+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"36'1200","ondisk_log_start":"36'1200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40330182,"num_objects":9848,"num_object_clones":0,"num_object_copies":19696,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9848,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":7735,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5611},{"shards":"0,2","objects":3545}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'11328","reported_seq":11458,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:57.981716+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:02:57.981716+0000","last_peered":"2024-09-10T16:02:57.981716+0000","last_clean":"2024-09-10T16:02:57.981716+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:02:57.981716+0000","last_undegraded":"2024-09-10T16:02:57.981716+0000","last_fullsized":"2024-09-10T16:02:57.981716+0000","mapping_epoch":34,"log_start":"36'1300","ondisk_log_start":"36'1300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40902656,"num_objects":9986,"num_object_clones":0,"num_object_copies":19972,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9986,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8175,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104219+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:02:08.104219+0000","last_peered":"2024-09-10T16:02:08.104219+0000","last_clean":"2024-09-10T16:02:08.104219+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:02:08.104219+0000","last_undegraded":"2024-09-10T16:02:08.104219+0000","last_fullsized":"2024-09-10T16:02:08.104219+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'10793","reported_seq":10933,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:54.722205+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:02:54.722205+0000","last_peered":"2024-09-10T16:02:54.722205+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:02:54.722205+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"36'700","ondisk_log_start":"36'700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39584227,"num_objects":9665,"num_object_clones":0,"num_object_copies":19330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9665,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8612,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3562},{"shards":"2","objects":5542}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103774+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:02:08.103774+0000","last_peered":"2024-09-10T16:02:08.103774+0000","last_clean":"2024-09-10T16:02:08.103774+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:02:08.103774+0000","last_undegraded":"2024-09-10T16:02:08.103774+0000","last_fullsized":"2024-09-10T16:02:08.103774+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'11139","reported_seq":11269,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:58.033504+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:02:58.033504+0000","last_peered":"2024-09-10T16:02:58.033504+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:02:58.033504+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"36'1100","ondisk_log_start":"36'1100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40022016,"num_objects":9771,"num_object_clones":0,"num_object_copies":19542,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9771,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7643,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5516},{"shards":"0,2","objects":3575}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":104,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.105055+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:02:08.105055+0000","last_peered":"2024-09-10T16:02:08.105055+0000","last_clean":"2024-09-10T16:02:08.105055+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:02:08.105055+0000","last_undegraded":"2024-09-10T16:02:08.105055+0000","last_fullsized":"2024-09-10T16:02:08.105055+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'10985","reported_seq":11120,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:54.861223+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:02:54.861223+0000","last_peered":"2024-09-10T16:02:54.861223+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:02:54.861223+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"36'900","ondisk_log_start":"36'900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40124899,"num_objects":9797,"num_object_clones":0,"num_object_copies":19594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9797,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8665,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3712},{"shards":"2","objects":5493}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104572+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:02:08.104572+0000","last_peered":"2024-09-10T16:02:08.104572+0000","last_clean":"2024-09-10T16:02:08.104572+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:02:08.104572+0000","last_undegraded":"2024-09-10T16:02:08.104572+0000","last_fullsized":"2024-09-10T16:02:08.104572+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'11238","reported_seq":11367,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:54.870406+0000","last_change":"2024-09-10T15:59:16.174113+0000","last_active":"2024-09-10T16:02:54.870406+0000","last_peered":"2024-09-10T16:02:54.870406+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:02:54.870406+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"36'1200","ondisk_log_start":"36'1200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40681472,"num_objects":9932,"num_object_clones":0,"num_object_copies":19864,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4354,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9932,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8792,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3702},{"shards":"2","objects":5580}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'11155","reported_seq":11289,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:58.015897+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:02:58.015897+0000","last_peered":"2024-09-10T16:02:58.015897+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:02:58.015897+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"36'1100","ondisk_log_start":"36'1100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39661568,"num_objects":9683,"num_object_clones":0,"num_object_copies":19366,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9683,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7786,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5379},{"shards":"0,2","objects":3569}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103276+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:02:08.103276+0000","last_peered":"2024-09-10T16:02:08.103276+0000","last_clean":"2024-09-10T16:02:08.103276+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:02:08.103276+0000","last_undegraded":"2024-09-10T16:02:08.103276+0000","last_fullsized":"2024-09-10T16:02:08.103276+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'10961","reported_seq":12868,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T16:02:58.079770+0000","last_change":"2024-09-10T16:02:06.480033+0000","last_active":"2024-09-10T16:02:58.079770+0000","last_peered":"2024-09-10T16:02:58.079770+0000","last_clean":"2024-09-10T15:59:14.115595+0000","last_became_active":"2024-09-10T15:59:16.244089+0000","last_became_peered":"2024-09-10T15:59:16.244089+0000","last_unstale":"2024-09-10T16:02:58.079770+0000","last_undegraded":"2024-09-10T16:02:58.079770+0000","last_fullsized":"2024-09-10T15:59:15.774437+0000","mapping_epoch":42,"log_start":"36'900","ondisk_log_start":"36'900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39473152,"num_objects":9637,"num_object_clones":0,"num_object_copies":19274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3321,"num_objects_unfound":0,"num_objects_dirty":9637,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7351,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":882,"num_bytes_recovered":3612672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6316},{"shards":"0,2","objects":3321}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103831+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:02:08.103831+0000","last_peered":"2024-09-10T16:02:08.103831+0000","last_clean":"2024-09-10T16:02:08.103831+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:02:08.103831+0000","last_undegraded":"2024-09-10T16:02:08.103831+0000","last_fullsized":"2024-09-10T16:02:08.103831+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'11233","reported_seq":11374,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:58.067267+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:02:58.067267+0000","last_peered":"2024-09-10T16:02:58.067267+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:02:58.067267+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"36'1200","ondisk_log_start":"36'1200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40202240,"num_objects":9815,"num_object_clones":0,"num_object_copies":19630,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9815,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7488,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5444},{"shards":"0,2","objects":3664}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104431+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:02:08.104431+0000","last_peered":"2024-09-10T16:02:08.104431+0000","last_clean":"2024-09-10T16:02:08.104431+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:02:08.104431+0000","last_undegraded":"2024-09-10T16:02:08.104431+0000","last_fullsized":"2024-09-10T16:02:08.104431+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104410+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:02:08.104410+0000","last_peered":"2024-09-10T16:02:08.104410+0000","last_clean":"2024-09-10T16:02:08.104410+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:02:08.104410+0000","last_undegraded":"2024-09-10T16:02:08.104410+0000","last_fullsized":"2024-09-10T16:02:08.104410+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'11313","reported_seq":11455,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:58.044090+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:02:58.044090+0000","last_peered":"2024-09-10T16:02:58.044090+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:02:58.044090+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"36'1300","ondisk_log_start":"36'1300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40489926,"num_objects":9887,"num_object_clones":0,"num_object_copies":19774,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9887,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7476,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5564},{"shards":"0,2","objects":3615}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":641405686,"num_objects":156603,"num_object_clones":0,"num_object_copies":313206,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42911,"num_objects_misplaced":3321,"num_objects_unfound":0,"num_objects_dirty":156603,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":135503,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5154,"num_bytes_recovered":21106688,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1331462144,"data_stored":1331302254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160729,"ondisk_log_size":160729,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986235,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":763560,"kb_used_data":452824,"kb_used_omap":4,"kb_used_meta":310715,"kb_avail":93608280,"statfs":{"total":96636764160,"available":95854878720,"internally_reserved":0,"allocated":463691776,"data_stored":463399543,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":318172842},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,0,0,1,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659181,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":568148,"kb_used_data":311556,"kb_used_omap":4,"kb_used_meta":256571,"kb_avail":93803692,"statfs":{"total":96636764160,"available":96054980608,"internally_reserved":0,"allocated":319033344,"data_stored":318693678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":262729390},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,1,7,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182464,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":873220,"kb_used_data":538924,"kb_used_omap":4,"kb_used_meta":334267,"kb_avail":93498620,"statfs":{"total":96636764160,"available":95742586880,"internally_reserved":0,"allocated":551858176,"data_stored":551565938,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":342290090},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,1,1,2,4,8,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":550162432,"data_stored":550126302,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":318640128,"data_stored":318556570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":462659584,"data_stored":462619382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:03:04.119 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:03:04.693 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:03:04.694 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:03:04.714 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":212,"stamp":"2024-09-10T16:03:03.629581+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":632460041,"num_objects":154421,"num_object_clones":0,"num_object_copies":308842,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42911,"num_objects_misplaced":3229,"num_objects_unfound":0,"num_objects_dirty":154421,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":137689,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5248,"num_bytes_recovered":21483539,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161016,"ondisk_log_size":161016,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2172584,"kb_used_data":1291768,"kb_used_omap":12,"kb_used_meta":880755,"kb_avail":280942936,"statfs":{"total":289910292480,"available":287685566464,"internally_reserved":0,"allocated":1322770432,"data_stored":1321846295,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":901893122},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,7,6,5,11,9,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-23764992,"num_objects":-5802,"num_object_clones":0,"num_object_copies":-11604,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-262,"num_objects_unfound":0,"num_objects_dirty":-5802,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5802,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":262,"num_bytes_recovered":1073152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.007351"},"pg_stats":[{"pgid":"2.f","version":"45'11152","reported_seq":11224,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:03.071497+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:03:03.071497+0000","last_peered":"2024-09-10T16:03:03.071497+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:03:03.071497+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"36'1100","ondisk_log_start":"36'1100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38993920,"num_objects":9520,"num_object_clones":0,"num_object_copies":19040,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9520,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11152,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5261},{"shards":"0,2","objects":3442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'11152","reported_seq":11265,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:59.728775+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:02:59.728775+0000","last_peered":"2024-09-10T16:02:59.728775+0000","last_clean":"2024-09-10T16:02:59.728775+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:02:59.728775+0000","last_undegraded":"2024-09-10T16:02:59.728775+0000","last_fullsized":"2024-09-10T16:02:59.728775+0000","mapping_epoch":35,"log_start":"36'1100","ondisk_log_start":"36'1100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39699398,"num_objects":9694,"num_object_clones":0,"num_object_copies":19388,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9694,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":9257,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'11174","reported_seq":19850,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:03:00.612599+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:03:00.612599+0000","last_peered":"2024-09-10T16:03:00.612599+0000","last_clean":"2024-09-10T16:03:00.612599+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:03:00.612599+0000","last_undegraded":"2024-09-10T16:03:00.612599+0000","last_fullsized":"2024-09-10T16:03:00.612599+0000","mapping_epoch":44,"log_start":"36'1100","ondisk_log_start":"36'1100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39415832,"num_objects":9624,"num_object_clones":0,"num_object_copies":19248,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9624,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9178,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'11095","reported_seq":11175,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:59.880996+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:02:59.880996+0000","last_peered":"2024-09-10T16:02:59.880996+0000","last_clean":"2024-09-10T16:02:59.880996+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:02:59.880996+0000","last_undegraded":"2024-09-10T16:02:59.880996+0000","last_fullsized":"2024-09-10T16:02:59.880996+0000","mapping_epoch":35,"log_start":"36'1000","ondisk_log_start":"36'1000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39178240,"num_objects":9565,"num_object_clones":0,"num_object_copies":19130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9565,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11095,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'11227","reported_seq":11357,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:03:03.005276+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:03:03.005276+0000","last_peered":"2024-09-10T16:03:03.005276+0000","last_clean":"2024-09-10T16:03:03.005276+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:03:03.005276+0000","last_undegraded":"2024-09-10T16:03:03.005276+0000","last_fullsized":"2024-09-10T16:03:03.005276+0000","mapping_epoch":34,"log_start":"36'1200","ondisk_log_start":"36'1200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39584227,"num_objects":9665,"num_object_clones":0,"num_object_copies":19330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9665,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7938,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'11350","reported_seq":11488,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:03.064213+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:03:03.064213+0000","last_peered":"2024-09-10T16:03:03.064213+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:03:03.064213+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"36'1300","ondisk_log_start":"36'1300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39776739,"num_objects":9712,"num_object_clones":0,"num_object_copies":19424,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9712,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7962,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5341},{"shards":"0,2","objects":3554}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'11372","reported_seq":11510,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:02.858129+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:03:02.858129+0000","last_peered":"2024-09-10T16:03:02.858129+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:03:02.858129+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"36'1300","ondisk_log_start":"36'1300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39773126,"num_objects":9712,"num_object_clones":0,"num_object_copies":19424,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9712,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":7871,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5475},{"shards":"0,2","objects":3409}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'11475","reported_seq":11605,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:03:03.098794+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:03:03.098794+0000","last_peered":"2024-09-10T16:03:03.098794+0000","last_clean":"2024-09-10T16:03:03.098794+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:03:03.098794+0000","last_undegraded":"2024-09-10T16:03:03.098794+0000","last_fullsized":"2024-09-10T16:03:03.098794+0000","mapping_epoch":34,"log_start":"36'1400","ondisk_log_start":"36'1400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40300544,"num_objects":9839,"num_object_clones":0,"num_object_copies":19678,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9839,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8322,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104219+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:02:08.104219+0000","last_peered":"2024-09-10T16:02:08.104219+0000","last_clean":"2024-09-10T16:02:08.104219+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:02:08.104219+0000","last_undegraded":"2024-09-10T16:02:08.104219+0000","last_fullsized":"2024-09-10T16:02:08.104219+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'10926","reported_seq":11066,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:59.876799+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:02:59.876799+0000","last_peered":"2024-09-10T16:02:59.876799+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:02:59.876799+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"36'900","ondisk_log_start":"36'900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39039459,"num_objects":9532,"num_object_clones":0,"num_object_copies":19064,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9532,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8745,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3429},{"shards":"2","objects":5409}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103774+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:02:08.103774+0000","last_peered":"2024-09-10T16:02:08.103774+0000","last_clean":"2024-09-10T16:02:08.103774+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:02:08.103774+0000","last_undegraded":"2024-09-10T16:02:08.103774+0000","last_fullsized":"2024-09-10T16:02:08.103774+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'11272","reported_seq":11400,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:03.122071+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:03:03.122071+0000","last_peered":"2024-09-10T16:03:03.122071+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:03:03.122071+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"36'1200","ondisk_log_start":"36'1200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39477248,"num_objects":9638,"num_object_clones":0,"num_object_copies":19276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9638,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7776,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5383},{"shards":"0,2","objects":3442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":104,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.105055+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:02:08.105055+0000","last_peered":"2024-09-10T16:02:08.105055+0000","last_clean":"2024-09-10T16:02:08.105055+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:02:08.105055+0000","last_undegraded":"2024-09-10T16:02:08.105055+0000","last_fullsized":"2024-09-10T16:02:08.105055+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'11118","reported_seq":11253,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:59.891342+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:02:59.891342+0000","last_peered":"2024-09-10T16:02:59.891342+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:02:59.891342+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"36'1100","ondisk_log_start":"36'1100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39580131,"num_objects":9664,"num_object_clones":0,"num_object_copies":19328,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9664,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8798,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3579},{"shards":"2","objects":5360}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104572+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:02:08.104572+0000","last_peered":"2024-09-10T16:02:08.104572+0000","last_clean":"2024-09-10T16:02:08.104572+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:02:08.104572+0000","last_undegraded":"2024-09-10T16:02:08.104572+0000","last_fullsized":"2024-09-10T16:02:08.104572+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'11376","reported_seq":11505,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:59.748678+0000","last_change":"2024-09-10T15:59:16.174113+0000","last_active":"2024-09-10T16:02:59.748678+0000","last_peered":"2024-09-10T16:02:59.748678+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:02:59.748678+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"36'1300","ondisk_log_start":"36'1300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40116224,"num_objects":9794,"num_object_clones":0,"num_object_copies":19588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4354,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9794,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8930,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3564},{"shards":"2","objects":5442}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'11290","reported_seq":11425,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:03.064863+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:03:03.064863+0000","last_peered":"2024-09-10T16:03:03.064863+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:03:03.064863+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"36'1200","ondisk_log_start":"36'1200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39108608,"num_objects":9548,"num_object_clones":0,"num_object_copies":19096,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9548,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7921,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5244},{"shards":"0,2","objects":3434}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103276+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:02:08.103276+0000","last_peered":"2024-09-10T16:02:08.103276+0000","last_clean":"2024-09-10T16:02:08.103276+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:02:08.103276+0000","last_undegraded":"2024-09-10T16:02:08.103276+0000","last_fullsized":"2024-09-10T16:02:08.103276+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'11096","reported_seq":13188,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T16:03:03.122010+0000","last_change":"2024-09-10T16:02:06.480033+0000","last_active":"2024-09-10T16:03:03.122010+0000","last_peered":"2024-09-10T16:03:03.122010+0000","last_clean":"2024-09-10T15:59:14.115595+0000","last_became_active":"2024-09-10T15:59:16.244089+0000","last_became_peered":"2024-09-10T15:59:16.244089+0000","last_unstale":"2024-09-10T16:03:03.122010+0000","last_undegraded":"2024-09-10T16:03:03.122010+0000","last_fullsized":"2024-09-10T15:59:15.774437+0000","mapping_epoch":42,"log_start":"36'1000","ondisk_log_start":"36'1000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38920192,"num_objects":9502,"num_object_clones":0,"num_object_copies":19004,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3229,"num_objects_unfound":0,"num_objects_dirty":9502,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7486,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":974,"num_bytes_recovered":3989504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6273},{"shards":"0,2","objects":3229}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103831+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:02:08.103831+0000","last_peered":"2024-09-10T16:02:08.103831+0000","last_clean":"2024-09-10T16:02:08.103831+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:02:08.103831+0000","last_undegraded":"2024-09-10T16:02:08.103831+0000","last_fullsized":"2024-09-10T16:02:08.103831+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'11385","reported_seq":11527,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:03.083287+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:03:03.083287+0000","last_peered":"2024-09-10T16:03:03.083287+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:03:03.083287+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"36'1300","ondisk_log_start":"36'1300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39579648,"num_objects":9663,"num_object_clones":0,"num_object_copies":19326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9663,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7640,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5292},{"shards":"0,2","objects":3512}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104431+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:02:08.104431+0000","last_peered":"2024-09-10T16:02:08.104431+0000","last_clean":"2024-09-10T16:02:08.104431+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:02:08.104431+0000","last_undegraded":"2024-09-10T16:02:08.104431+0000","last_fullsized":"2024-09-10T16:02:08.104431+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104410+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:02:08.104410+0000","last_peered":"2024-09-10T16:02:08.104410+0000","last_clean":"2024-09-10T16:02:08.104410+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:02:08.104410+0000","last_undegraded":"2024-09-10T16:02:08.104410+0000","last_fullsized":"2024-09-10T16:02:08.104410+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'11453","reported_seq":11597,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:03.085560+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:03:03.085560+0000","last_peered":"2024-09-10T16:03:03.085560+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:03:03.085560+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"36'1400","ondisk_log_start":"36'1400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39916486,"num_objects":9747,"num_object_clones":0,"num_object_copies":19494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9747,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7616,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5424},{"shards":"0,2","objects":3475}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":632460022,"num_objects":154419,"num_object_clones":0,"num_object_copies":308838,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42911,"num_objects_misplaced":3229,"num_objects_unfound":0,"num_objects_dirty":154419,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":137687,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5246,"num_bytes_recovered":21483520,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1319632896,"data_stored":1319473006,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161013,"ondisk_log_size":161013,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986236,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":746724,"kb_used_data":448468,"kb_used_omap":4,"kb_used_meta":298235,"kb_avail":93625116,"statfs":{"total":96636764160,"available":95872118784,"internally_reserved":0,"allocated":459231232,"data_stored":458938999,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":305393322},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,1,2,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659182,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":555072,"kb_used_data":310960,"kb_used_omap":4,"kb_used_meta":244091,"kb_avail":93816768,"statfs":{"total":96636764160,"available":96068370432,"internally_reserved":0,"allocated":318423040,"data_stored":318083374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":249949870},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,4,0,4,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182465,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":870788,"kb_used_data":532340,"kb_used_omap":4,"kb_used_meta":338427,"kb_avail":93501052,"statfs":{"total":96636764160,"available":95745077248,"internally_reserved":0,"allocated":545116160,"data_stored":544823922,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":346549930},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,4,2,4,5,4,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":543379456,"data_stored":543343326,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":318013440,"data_stored":317929882,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":458240000,"data_stored":458199798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:03:04.715 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:03:05.292 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:03:05.294 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:03:05.315 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":212,"stamp":"2024-09-10T16:03:03.629581+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":632460041,"num_objects":154421,"num_object_clones":0,"num_object_copies":308842,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42911,"num_objects_misplaced":3229,"num_objects_unfound":0,"num_objects_dirty":154421,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":137689,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5248,"num_bytes_recovered":21483539,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161016,"ondisk_log_size":161016,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2172584,"kb_used_data":1291768,"kb_used_omap":12,"kb_used_meta":880755,"kb_avail":280942936,"statfs":{"total":289910292480,"available":287685566464,"internally_reserved":0,"allocated":1322770432,"data_stored":1321846295,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":901893122},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,7,6,5,11,9,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-23764992,"num_objects":-5802,"num_object_clones":0,"num_object_copies":-11604,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-262,"num_objects_unfound":0,"num_objects_dirty":-5802,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5802,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":262,"num_bytes_recovered":1073152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.007351"},"pg_stats":[{"pgid":"2.f","version":"45'11152","reported_seq":11224,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:03.071497+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:03:03.071497+0000","last_peered":"2024-09-10T16:03:03.071497+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:03:03.071497+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"36'1100","ondisk_log_start":"36'1100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38993920,"num_objects":9520,"num_object_clones":0,"num_object_copies":19040,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9520,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11152,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5261},{"shards":"0,2","objects":3442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'11152","reported_seq":11265,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:59.728775+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:02:59.728775+0000","last_peered":"2024-09-10T16:02:59.728775+0000","last_clean":"2024-09-10T16:02:59.728775+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:02:59.728775+0000","last_undegraded":"2024-09-10T16:02:59.728775+0000","last_fullsized":"2024-09-10T16:02:59.728775+0000","mapping_epoch":35,"log_start":"36'1100","ondisk_log_start":"36'1100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39699398,"num_objects":9694,"num_object_clones":0,"num_object_copies":19388,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9694,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":9257,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'11174","reported_seq":19850,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:03:00.612599+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:03:00.612599+0000","last_peered":"2024-09-10T16:03:00.612599+0000","last_clean":"2024-09-10T16:03:00.612599+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:03:00.612599+0000","last_undegraded":"2024-09-10T16:03:00.612599+0000","last_fullsized":"2024-09-10T16:03:00.612599+0000","mapping_epoch":44,"log_start":"36'1100","ondisk_log_start":"36'1100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39415832,"num_objects":9624,"num_object_clones":0,"num_object_copies":19248,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9624,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9178,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'11095","reported_seq":11175,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:59.880996+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:02:59.880996+0000","last_peered":"2024-09-10T16:02:59.880996+0000","last_clean":"2024-09-10T16:02:59.880996+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:02:59.880996+0000","last_undegraded":"2024-09-10T16:02:59.880996+0000","last_fullsized":"2024-09-10T16:02:59.880996+0000","mapping_epoch":35,"log_start":"36'1000","ondisk_log_start":"36'1000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39178240,"num_objects":9565,"num_object_clones":0,"num_object_copies":19130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9565,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11095,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'11227","reported_seq":11357,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:03:03.005276+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:03:03.005276+0000","last_peered":"2024-09-10T16:03:03.005276+0000","last_clean":"2024-09-10T16:03:03.005276+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:03:03.005276+0000","last_undegraded":"2024-09-10T16:03:03.005276+0000","last_fullsized":"2024-09-10T16:03:03.005276+0000","mapping_epoch":34,"log_start":"36'1200","ondisk_log_start":"36'1200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39584227,"num_objects":9665,"num_object_clones":0,"num_object_copies":19330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9665,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7938,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'11350","reported_seq":11488,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:03.064213+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:03:03.064213+0000","last_peered":"2024-09-10T16:03:03.064213+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:03:03.064213+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"36'1300","ondisk_log_start":"36'1300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39776739,"num_objects":9712,"num_object_clones":0,"num_object_copies":19424,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9712,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7962,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5341},{"shards":"0,2","objects":3554}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'11372","reported_seq":11510,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:02.858129+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:03:02.858129+0000","last_peered":"2024-09-10T16:03:02.858129+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:03:02.858129+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"36'1300","ondisk_log_start":"36'1300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39773126,"num_objects":9712,"num_object_clones":0,"num_object_copies":19424,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9712,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":7871,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5475},{"shards":"0,2","objects":3409}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'11475","reported_seq":11605,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:03:03.098794+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:03:03.098794+0000","last_peered":"2024-09-10T16:03:03.098794+0000","last_clean":"2024-09-10T16:03:03.098794+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:03:03.098794+0000","last_undegraded":"2024-09-10T16:03:03.098794+0000","last_fullsized":"2024-09-10T16:03:03.098794+0000","mapping_epoch":34,"log_start":"36'1400","ondisk_log_start":"36'1400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40300544,"num_objects":9839,"num_object_clones":0,"num_object_copies":19678,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9839,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8322,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104219+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:02:08.104219+0000","last_peered":"2024-09-10T16:02:08.104219+0000","last_clean":"2024-09-10T16:02:08.104219+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:02:08.104219+0000","last_undegraded":"2024-09-10T16:02:08.104219+0000","last_fullsized":"2024-09-10T16:02:08.104219+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'10926","reported_seq":11066,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:59.876799+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:02:59.876799+0000","last_peered":"2024-09-10T16:02:59.876799+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:02:59.876799+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"36'900","ondisk_log_start":"36'900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39039459,"num_objects":9532,"num_object_clones":0,"num_object_copies":19064,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9532,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8745,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3429},{"shards":"2","objects":5409}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103774+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:02:08.103774+0000","last_peered":"2024-09-10T16:02:08.103774+0000","last_clean":"2024-09-10T16:02:08.103774+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:02:08.103774+0000","last_undegraded":"2024-09-10T16:02:08.103774+0000","last_fullsized":"2024-09-10T16:02:08.103774+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'11272","reported_seq":11400,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:03.122071+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:03:03.122071+0000","last_peered":"2024-09-10T16:03:03.122071+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:03:03.122071+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"36'1200","ondisk_log_start":"36'1200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39477248,"num_objects":9638,"num_object_clones":0,"num_object_copies":19276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9638,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7776,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5383},{"shards":"0,2","objects":3442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":104,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.105055+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:02:08.105055+0000","last_peered":"2024-09-10T16:02:08.105055+0000","last_clean":"2024-09-10T16:02:08.105055+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:02:08.105055+0000","last_undegraded":"2024-09-10T16:02:08.105055+0000","last_fullsized":"2024-09-10T16:02:08.105055+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'11118","reported_seq":11253,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:59.891342+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:02:59.891342+0000","last_peered":"2024-09-10T16:02:59.891342+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:02:59.891342+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"36'1100","ondisk_log_start":"36'1100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39580131,"num_objects":9664,"num_object_clones":0,"num_object_copies":19328,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9664,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8798,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3579},{"shards":"2","objects":5360}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104572+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:02:08.104572+0000","last_peered":"2024-09-10T16:02:08.104572+0000","last_clean":"2024-09-10T16:02:08.104572+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:02:08.104572+0000","last_undegraded":"2024-09-10T16:02:08.104572+0000","last_fullsized":"2024-09-10T16:02:08.104572+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'11376","reported_seq":11505,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:59.748678+0000","last_change":"2024-09-10T15:59:16.174113+0000","last_active":"2024-09-10T16:02:59.748678+0000","last_peered":"2024-09-10T16:02:59.748678+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:02:59.748678+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"36'1300","ondisk_log_start":"36'1300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40116224,"num_objects":9794,"num_object_clones":0,"num_object_copies":19588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4354,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9794,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8930,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3564},{"shards":"2","objects":5442}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'11290","reported_seq":11425,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:03.064863+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:03:03.064863+0000","last_peered":"2024-09-10T16:03:03.064863+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:03:03.064863+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"36'1200","ondisk_log_start":"36'1200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39108608,"num_objects":9548,"num_object_clones":0,"num_object_copies":19096,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9548,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7921,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5244},{"shards":"0,2","objects":3434}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103276+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:02:08.103276+0000","last_peered":"2024-09-10T16:02:08.103276+0000","last_clean":"2024-09-10T16:02:08.103276+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:02:08.103276+0000","last_undegraded":"2024-09-10T16:02:08.103276+0000","last_fullsized":"2024-09-10T16:02:08.103276+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'11096","reported_seq":13188,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T16:03:03.122010+0000","last_change":"2024-09-10T16:02:06.480033+0000","last_active":"2024-09-10T16:03:03.122010+0000","last_peered":"2024-09-10T16:03:03.122010+0000","last_clean":"2024-09-10T15:59:14.115595+0000","last_became_active":"2024-09-10T15:59:16.244089+0000","last_became_peered":"2024-09-10T15:59:16.244089+0000","last_unstale":"2024-09-10T16:03:03.122010+0000","last_undegraded":"2024-09-10T16:03:03.122010+0000","last_fullsized":"2024-09-10T15:59:15.774437+0000","mapping_epoch":42,"log_start":"36'1000","ondisk_log_start":"36'1000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38920192,"num_objects":9502,"num_object_clones":0,"num_object_copies":19004,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3229,"num_objects_unfound":0,"num_objects_dirty":9502,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7486,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":974,"num_bytes_recovered":3989504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6273},{"shards":"0,2","objects":3229}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103831+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:02:08.103831+0000","last_peered":"2024-09-10T16:02:08.103831+0000","last_clean":"2024-09-10T16:02:08.103831+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:02:08.103831+0000","last_undegraded":"2024-09-10T16:02:08.103831+0000","last_fullsized":"2024-09-10T16:02:08.103831+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'11385","reported_seq":11527,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:03.083287+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:03:03.083287+0000","last_peered":"2024-09-10T16:03:03.083287+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:03:03.083287+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"36'1300","ondisk_log_start":"36'1300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39579648,"num_objects":9663,"num_object_clones":0,"num_object_copies":19326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9663,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7640,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5292},{"shards":"0,2","objects":3512}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104431+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:02:08.104431+0000","last_peered":"2024-09-10T16:02:08.104431+0000","last_clean":"2024-09-10T16:02:08.104431+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:02:08.104431+0000","last_undegraded":"2024-09-10T16:02:08.104431+0000","last_fullsized":"2024-09-10T16:02:08.104431+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104410+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:02:08.104410+0000","last_peered":"2024-09-10T16:02:08.104410+0000","last_clean":"2024-09-10T16:02:08.104410+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:02:08.104410+0000","last_undegraded":"2024-09-10T16:02:08.104410+0000","last_fullsized":"2024-09-10T16:02:08.104410+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'11453","reported_seq":11597,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:03.085560+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:03:03.085560+0000","last_peered":"2024-09-10T16:03:03.085560+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:03:03.085560+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"36'1400","ondisk_log_start":"36'1400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39916486,"num_objects":9747,"num_object_clones":0,"num_object_copies":19494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9747,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7616,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5424},{"shards":"0,2","objects":3475}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":632460022,"num_objects":154419,"num_object_clones":0,"num_object_copies":308838,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42911,"num_objects_misplaced":3229,"num_objects_unfound":0,"num_objects_dirty":154419,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":137687,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5246,"num_bytes_recovered":21483520,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1319632896,"data_stored":1319473006,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161013,"ondisk_log_size":161013,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986236,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":746724,"kb_used_data":448468,"kb_used_omap":4,"kb_used_meta":298235,"kb_avail":93625116,"statfs":{"total":96636764160,"available":95872118784,"internally_reserved":0,"allocated":459231232,"data_stored":458938999,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":305393322},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,1,2,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659182,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":555072,"kb_used_data":310960,"kb_used_omap":4,"kb_used_meta":244091,"kb_avail":93816768,"statfs":{"total":96636764160,"available":96068370432,"internally_reserved":0,"allocated":318423040,"data_stored":318083374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":249949870},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,4,0,4,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182465,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":870788,"kb_used_data":532340,"kb_used_omap":4,"kb_used_meta":338427,"kb_avail":93501052,"statfs":{"total":96636764160,"available":95745077248,"internally_reserved":0,"allocated":545116160,"data_stored":544823922,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":346549930},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,4,2,4,5,4,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":543379456,"data_stored":543343326,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":318013440,"data_stored":317929882,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":458240000,"data_stored":458199798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:03:05.317 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-10T16:03:05.317 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "45'11152", 'reported_seq': 11224, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:03:03.071497+0000', 'last_change': '2024-09-10T15:59:16.281558+0000', 'last_active': '2024-09-10T16:03:03.071497+0000', 'last_peered': '2024-09-10T16:03:03.071497+0000', 'last_clean': '2024-09-10T15:59:14.132456+0000', 'last_became_active': '2024-09-10T15:59:16.274407+0000', 'last_became_peered': '2024-09-10T15:59:16.274407+0000', 'last_unstale': '2024-09-10T16:03:03.071497+0000', 'last_undegraded': '2024-09-10T15:59:15.851279+0000', 'last_fullsized': '2024-09-10T15:59:15.774356+0000', 'mapping_epoch': 42, 'log_start': "36'1100", 'ondisk_log_start': "36'1100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'objects_scrubbed': 0, 'log_size': 10052, 'log_dups_size': 0, 'ondisk_log_size': 10052, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38993920, 'num_objects': 9520, 'num_object_clones': 0, 'num_object_copies': 19040, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4259, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9520, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11152, 'num_write_kb': 41344, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5261}, {'shards': '0,2', 'objects': 3442}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:03:05.317 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-10T16:03:05.317 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "45'11350", 'reported_seq': 11488, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:03:03.064213+0000', 'last_change': '2024-09-10T15:59:16.308264+0000', 'last_active': '2024-09-10T16:03:03.064213+0000', 'last_peered': '2024-09-10T16:03:03.064213+0000', 'last_clean': '2024-09-10T15:59:14.111551+0000', 'last_became_active': '2024-09-10T15:59:16.300040+0000', 'last_became_peered': '2024-09-10T15:59:16.300040+0000', 'last_unstale': '2024-09-10T16:03:03.064213+0000', 'last_undegraded': '2024-09-10T15:59:15.878234+0000', 'last_fullsized': '2024-09-10T15:59:15.769328+0000', 'mapping_epoch': 42, 'log_start': "36'1300", 'ondisk_log_start': "36'1300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3442", 'last_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'objects_scrubbed': 3384, 'log_size': 10050, 'log_dups_size': 0, 'ondisk_log_size': 10050, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1292, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39776739, 'num_objects': 9712, 'num_object_clones': 0, 'num_object_copies': 19424, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4372, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9712, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 7962, 'num_write_kb': 28580, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5341}, {'shards': '0,2', 'objects': 3554}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:03:05.317 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-10T16:03:05.317 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "45'11372", 'reported_seq': 11510, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:03:02.858129+0000', 'last_change': '2024-09-10T15:59:16.331899+0000', 'last_active': '2024-09-10T16:03:02.858129+0000', 'last_peered': '2024-09-10T16:03:02.858129+0000', 'last_clean': '2024-09-10T15:59:14.139436+0000', 'last_became_active': '2024-09-10T15:59:16.318715+0000', 'last_became_peered': '2024-09-10T15:59:16.318715+0000', 'last_unstale': '2024-09-10T16:03:02.858129+0000', 'last_undegraded': '2024-09-10T15:59:15.877524+0000', 'last_fullsized': '2024-09-10T15:59:15.769042+0000', 'mapping_epoch': 42, 'log_start': "36'1300", 'ondisk_log_start': "36'1300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3537", 'last_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'objects_scrubbed': 3497, 'log_size': 10072, 'log_dups_size': 0, 'ondisk_log_size': 10072, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1224, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39773126, 'num_objects': 9712, 'num_object_clones': 0, 'num_object_copies': 19424, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4237, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9712, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 7871, 'num_write_kb': 28172, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5475}, {'shards': '0,2', 'objects': 3409}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:03:05.317 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-10T16:03:05.317 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "45'10926", 'reported_seq': 11066, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:02:59.876799+0000', 'last_change': '2024-09-10T15:59:16.175680+0000', 'last_active': '2024-09-10T16:02:59.876799+0000', 'last_peered': '2024-09-10T16:02:59.876799+0000', 'last_clean': '2024-09-10T15:59:14.088187+0000', 'last_became_active': '2024-09-10T15:59:16.165810+0000', 'last_became_peered': '2024-09-10T15:59:16.165810+0000', 'last_unstale': '2024-09-10T16:02:59.876799+0000', 'last_undegraded': '2024-09-10T15:59:15.822098+0000', 'last_fullsized': '2024-09-10T15:59:15.769123+0000', 'mapping_epoch': 42, 'log_start': "36'900", 'ondisk_log_start': "36'900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2218", 'last_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'objects_scrubbed': 2175, 'log_size': 10026, 'log_dups_size': 0, 'ondisk_log_size': 10026, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 825, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39039459, 'num_objects': 9532, 'num_object_clones': 0, 'num_object_copies': 19064, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4124, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9532, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 8745, 'num_write_kb': 32204, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3429}, {'shards': '2', 'objects': 5409}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:03:05.318 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-10T16:03:05.318 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "45'11272", 'reported_seq': 11400, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:03:03.122071+0000', 'last_change': '2024-09-10T15:59:16.277742+0000', 'last_active': '2024-09-10T16:03:03.122071+0000', 'last_peered': '2024-09-10T16:03:03.122071+0000', 'last_clean': '2024-09-10T15:59:14.129081+0000', 'last_became_active': '2024-09-10T15:59:16.269835+0000', 'last_became_peered': '2024-09-10T15:59:16.269835+0000', 'last_unstale': '2024-09-10T16:03:03.122071+0000', 'last_undegraded': '2024-09-10T15:59:15.860857+0000', 'last_fullsized': '2024-09-10T15:59:15.771410+0000', 'mapping_epoch': 42, 'log_start': "36'1200", 'ondisk_log_start': "36'1200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3557", 'last_scrub_stamp': '2024-09-10T15:59:02.493678+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:02.493678+0000', 'objects_scrubbed': 3488, 'log_size': 10072, 'log_dups_size': 0, 'ondisk_log_size': 10072, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1472, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39477248, 'num_objects': 9638, 'num_object_clones': 0, 'num_object_copies': 19276, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4257, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9638, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7776, 'num_write_kb': 27852, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5383}, {'shards': '0,2', 'objects': 3442}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:03:05.318 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-10T16:03:05.318 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "45'11118", 'reported_seq': 11253, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:02:59.891342+0000', 'last_change': '2024-09-10T15:59:16.284311+0000', 'last_active': '2024-09-10T16:02:59.891342+0000', 'last_peered': '2024-09-10T16:02:59.891342+0000', 'last_clean': '2024-09-10T15:59:14.116041+0000', 'last_became_active': '2024-09-10T15:59:16.274543+0000', 'last_became_peered': '2024-09-10T15:59:16.274543+0000', 'last_unstale': '2024-09-10T16:02:59.891342+0000', 'last_undegraded': '2024-09-10T15:59:15.812484+0000', 'last_fullsized': '2024-09-10T15:59:15.770885+0000', 'mapping_epoch': 42, 'log_start': "36'1100", 'ondisk_log_start': "36'1100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2356", 'last_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'objects_scrubbed': 2316, 'log_size': 10018, 'log_dups_size': 0, 'ondisk_log_size': 10018, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 783, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39580131, 'num_objects': 9664, 'num_object_clones': 0, 'num_object_copies': 19328, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9664, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 8798, 'num_write_kb': 32292, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3579}, {'shards': '2', 'objects': 5360}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:03:05.318 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-10T16:03:05.318 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "45'11376", 'reported_seq': 11505, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:02:59.748678+0000', 'last_change': '2024-09-10T15:59:16.174113+0000', 'last_active': '2024-09-10T16:02:59.748678+0000', 'last_peered': '2024-09-10T16:02:59.748678+0000', 'last_clean': '2024-09-10T15:59:14.110654+0000', 'last_became_active': '2024-09-10T15:59:16.164645+0000', 'last_became_peered': '2024-09-10T15:59:16.164645+0000', 'last_unstale': '2024-09-10T16:02:59.748678+0000', 'last_undegraded': '2024-09-10T15:59:15.803593+0000', 'last_fullsized': '2024-09-10T15:59:15.769083+0000', 'mapping_epoch': 42, 'log_start': "36'1300", 'ondisk_log_start': "36'1300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2486", 'last_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'objects_scrubbed': 2440, 'log_size': 10076, 'log_dups_size': 0, 'ondisk_log_size': 10076, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 792, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40116224, 'num_objects': 9794, 'num_object_clones': 0, 'num_object_copies': 19588, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4354, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9794, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8930, 'num_write_kb': 32568, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3564}, {'shards': '2', 'objects': 5442}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:03:05.318 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-10T16:03:05.318 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "45'11290", 'reported_seq': 11425, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:03:03.064863+0000', 'last_change': '2024-09-10T15:59:16.315641+0000', 'last_active': '2024-09-10T16:03:03.064863+0000', 'last_peered': '2024-09-10T16:03:03.064863+0000', 'last_clean': '2024-09-10T15:59:14.117001+0000', 'last_became_active': '2024-09-10T15:59:16.302245+0000', 'last_became_peered': '2024-09-10T15:59:16.302245+0000', 'last_unstale': '2024-09-10T16:03:03.064863+0000', 'last_undegraded': '2024-09-10T15:59:15.845012+0000', 'last_fullsized': '2024-09-10T15:59:15.770337+0000', 'mapping_epoch': 42, 'log_start': "36'1200", 'ondisk_log_start': "36'1200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3404", 'last_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'objects_scrubbed': 3367, 'log_size': 10090, 'log_dups_size': 0, 'ondisk_log_size': 10090, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 983, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39108608, 'num_objects': 9548, 'num_object_clones': 0, 'num_object_copies': 19096, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9548, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7921, 'num_write_kb': 28204, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5244}, {'shards': '0,2', 'objects': 3434}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:03:05.318 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-10T16:03:05.318 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "45'11096", 'reported_seq': 13188, 'reported_epoch': 45, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-10T16:03:03.122010+0000', 'last_change': '2024-09-10T16:02:06.480033+0000', 'last_active': '2024-09-10T16:03:03.122010+0000', 'last_peered': '2024-09-10T16:03:03.122010+0000', 'last_clean': '2024-09-10T15:59:14.115595+0000', 'last_became_active': '2024-09-10T15:59:16.244089+0000', 'last_became_peered': '2024-09-10T15:59:16.244089+0000', 'last_unstale': '2024-09-10T16:03:03.122010+0000', 'last_undegraded': '2024-09-10T16:03:03.122010+0000', 'last_fullsized': '2024-09-10T15:59:15.774437+0000', 'mapping_epoch': 42, 'log_start': "36'1000", 'ondisk_log_start': "36'1000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3653", 'last_scrub_stamp': '2024-09-10T15:59:05.298468+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:05.298468+0000', 'objects_scrubbed': 3602, 'log_size': 10096, 'log_dups_size': 0, 'ondisk_log_size': 10096, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1289, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38920192, 'num_objects': 9502, 'num_object_clones': 0, 'num_object_copies': 19004, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 3229, 'num_objects_unfound': 0, 'num_objects_dirty': 9502, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7486, 'num_write_kb': 26772, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 974, 'num_bytes_recovered': 3989504, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6273}, {'shards': '0,2', 'objects': 3229}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:03:05.319 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-10T16:03:05.319 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "45'11385", 'reported_seq': 11527, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:03:03.083287+0000', 'last_change': '2024-09-10T15:59:16.311291+0000', 'last_active': '2024-09-10T16:03:03.083287+0000', 'last_peered': '2024-09-10T16:03:03.083287+0000', 'last_clean': '2024-09-10T15:59:14.116363+0000', 'last_became_active': '2024-09-10T15:59:16.301244+0000', 'last_became_peered': '2024-09-10T15:59:16.301244+0000', 'last_unstale': '2024-09-10T16:03:03.083287+0000', 'last_undegraded': '2024-09-10T15:59:15.883434+0000', 'last_fullsized': '2024-09-10T15:59:15.772756+0000', 'mapping_epoch': 42, 'log_start': "36'1300", 'ondisk_log_start': "36'1300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3789", 'last_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'objects_scrubbed': 3741, 'log_size': 10085, 'log_dups_size': 0, 'ondisk_log_size': 10085, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1440, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39579648, 'num_objects': 9663, 'num_object_clones': 0, 'num_object_copies': 19326, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4373, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9663, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7640, 'num_write_kb': 27124, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5292}, {'shards': '0,2', 'objects': 3512}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:03:05.319 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-10T16:03:05.319 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "45'11453", 'reported_seq': 11597, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:03:03.085560+0000', 'last_change': '2024-09-10T15:59:16.313409+0000', 'last_active': '2024-09-10T16:03:03.085560+0000', 'last_peered': '2024-09-10T16:03:03.085560+0000', 'last_clean': '2024-09-10T15:59:14.129780+0000', 'last_became_active': '2024-09-10T15:59:16.300932+0000', 'last_became_peered': '2024-09-10T15:59:16.300932+0000', 'last_unstale': '2024-09-10T16:03:03.085560+0000', 'last_undegraded': '2024-09-10T15:59:15.887823+0000', 'last_fullsized': '2024-09-10T15:59:15.768513+0000', 'mapping_epoch': 42, 'log_start': "36'1400", 'ondisk_log_start': "36'1400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3880", 'last_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'objects_scrubbed': 3827, 'log_size': 10053, 'log_dups_size': 0, 'ondisk_log_size': 10053, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1397, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39916486, 'num_objects': 9747, 'num_object_clones': 0, 'num_object_copies': 19494, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4325, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9747, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7616, 'num_write_kb': 27072, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5424}, {'shards': '0,2', 'objects': 3475}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:03:05.319 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T16:03:06.026 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:03:06.026 INFO:teuthology.orchestra.run.smithi115.stdout:{"fsid":"7effee29-df05-4351-9741-ee203564ed81","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 42911/307446 objects degraded (13.957%), 10 pgs degraded, 11 pgs undersized","count":21},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":350,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983954,"num_in_osds":3,"osd_in_since":1725983947,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":153723,"data_bytes":629601033,"bytes_used":2224685056,"bytes_avail":287685607424,"bytes_total":289910292480,"degraded_objects":42911,"degraded_total":307446,"degraded_ratio":0.13957247776845363,"misplaced_objects":3229,"misplaced_total":307446,"misplaced_ratio":0.010502657377230473,"recovering_objects_per_sec":14,"recovering_bytes_per_sec":60718,"recovering_keys_per_sec":0,"num_objects_recovered":178,"num_bytes_recovered":729088,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":425},"fsmap":{"epoch":1,"btime":"2024-09-10T15:57:14:967120+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T16:00:13.560776+0000","services":{}},"progress_events":{}} 2024-09-10T16:03:06.051 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-10T16:03:06.052 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:03:06.617 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:03:06.619 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:03:06.637 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":213,"stamp":"2024-09-10T16:03:05.630480+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":629601033,"num_objects":153723,"num_object_clones":0,"num_object_copies":307446,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42911,"num_objects_misplaced":3229,"num_objects_unfound":0,"num_objects_dirty":153723,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":138387,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5248,"num_bytes_recovered":21483539,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161114,"ondisk_log_size":161114,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2172544,"kb_used_data":1287568,"kb_used_omap":12,"kb_used_meta":884915,"kb_avail":280942976,"statfs":{"total":289910292480,"available":287685607424,"internally_reserved":0,"allocated":1318469632,"data_stored":1317545495,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":906152962},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,6,7,4,12,8,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-20914176,"num_objects":-5106,"num_object_clones":0,"num_object_copies":-10212,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-178,"num_objects_unfound":0,"num_objects_dirty":-5106,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5106,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":178,"num_bytes_recovered":729088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.007657"},"pg_stats":[{"pgid":"2.f","version":"45'11152","reported_seq":11224,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:03.071497+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:03:03.071497+0000","last_peered":"2024-09-10T16:03:03.071497+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:03:03.071497+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"36'1100","ondisk_log_start":"36'1100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38993920,"num_objects":9520,"num_object_clones":0,"num_object_copies":19040,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9520,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11152,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5261},{"shards":"0,2","objects":3442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'11296","reported_seq":11409,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:03:04.771717+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:03:04.771717+0000","last_peered":"2024-09-10T16:03:04.771717+0000","last_clean":"2024-09-10T16:03:04.771717+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:03:04.771717+0000","last_undegraded":"2024-09-10T16:03:04.771717+0000","last_fullsized":"2024-09-10T16:03:04.771717+0000","mapping_epoch":35,"log_start":"36'1200","ondisk_log_start":"36'1200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39109574,"num_objects":9550,"num_object_clones":0,"num_object_copies":19100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9550,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":9401,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'11174","reported_seq":19850,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:03:00.612599+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:03:00.612599+0000","last_peered":"2024-09-10T16:03:00.612599+0000","last_clean":"2024-09-10T16:03:00.612599+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:03:00.612599+0000","last_undegraded":"2024-09-10T16:03:00.612599+0000","last_fullsized":"2024-09-10T16:03:00.612599+0000","mapping_epoch":44,"log_start":"36'1100","ondisk_log_start":"36'1100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39415832,"num_objects":9624,"num_object_clones":0,"num_object_copies":19248,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9624,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9178,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'11226","reported_seq":11307,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:03:04.849150+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:03:04.849150+0000","last_peered":"2024-09-10T16:03:04.849150+0000","last_clean":"2024-09-10T16:03:04.849150+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:03:04.849150+0000","last_undegraded":"2024-09-10T16:03:04.849150+0000","last_fullsized":"2024-09-10T16:03:04.849150+0000","mapping_epoch":35,"log_start":"36'1200","ondisk_log_start":"36'1200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38641664,"num_objects":9434,"num_object_clones":0,"num_object_copies":18868,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9434,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11226,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'11227","reported_seq":11357,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:03:03.005276+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:03:03.005276+0000","last_peered":"2024-09-10T16:03:03.005276+0000","last_clean":"2024-09-10T16:03:03.005276+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:03:03.005276+0000","last_undegraded":"2024-09-10T16:03:03.005276+0000","last_fullsized":"2024-09-10T16:03:03.005276+0000","mapping_epoch":34,"log_start":"36'1200","ondisk_log_start":"36'1200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39584227,"num_objects":9665,"num_object_clones":0,"num_object_copies":19330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9665,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7938,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'11350","reported_seq":11488,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:03.064213+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:03:03.064213+0000","last_peered":"2024-09-10T16:03:03.064213+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:03:03.064213+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"36'1300","ondisk_log_start":"36'1300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39776739,"num_objects":9712,"num_object_clones":0,"num_object_copies":19424,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9712,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7962,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5341},{"shards":"0,2","objects":3554}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'11372","reported_seq":11510,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:02.858129+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:03:02.858129+0000","last_peered":"2024-09-10T16:03:02.858129+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:03:02.858129+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"36'1300","ondisk_log_start":"36'1300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39773126,"num_objects":9712,"num_object_clones":0,"num_object_copies":19424,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9712,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":7871,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5475},{"shards":"0,2","objects":3409}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'11475","reported_seq":11605,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:03:03.098794+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:03:03.098794+0000","last_peered":"2024-09-10T16:03:03.098794+0000","last_clean":"2024-09-10T16:03:03.098794+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:03:03.098794+0000","last_undegraded":"2024-09-10T16:03:03.098794+0000","last_fullsized":"2024-09-10T16:03:03.098794+0000","mapping_epoch":34,"log_start":"36'1400","ondisk_log_start":"36'1400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40300544,"num_objects":9839,"num_object_clones":0,"num_object_copies":19678,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9839,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8322,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104219+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:02:08.104219+0000","last_peered":"2024-09-10T16:02:08.104219+0000","last_clean":"2024-09-10T16:02:08.104219+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:02:08.104219+0000","last_undegraded":"2024-09-10T16:02:08.104219+0000","last_fullsized":"2024-09-10T16:02:08.104219+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'11083","reported_seq":11223,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:04.939019+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:03:04.939019+0000","last_peered":"2024-09-10T16:03:04.939019+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:03:04.939019+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"36'1000","ondisk_log_start":"36'1000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38396387,"num_objects":9375,"num_object_clones":0,"num_object_copies":18750,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9375,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8902,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3272},{"shards":"2","objects":5252}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103774+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:02:08.103774+0000","last_peered":"2024-09-10T16:02:08.103774+0000","last_clean":"2024-09-10T16:02:08.103774+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:02:08.103774+0000","last_undegraded":"2024-09-10T16:02:08.103774+0000","last_fullsized":"2024-09-10T16:02:08.103774+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'11272","reported_seq":11400,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:03.122071+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:03:03.122071+0000","last_peered":"2024-09-10T16:03:03.122071+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:03:03.122071+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"36'1200","ondisk_log_start":"36'1200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39477248,"num_objects":9638,"num_object_clones":0,"num_object_copies":19276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9638,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7776,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5383},{"shards":"0,2","objects":3442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":104,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.105055+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:02:08.105055+0000","last_peered":"2024-09-10T16:02:08.105055+0000","last_clean":"2024-09-10T16:02:08.105055+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:02:08.105055+0000","last_undegraded":"2024-09-10T16:02:08.105055+0000","last_fullsized":"2024-09-10T16:02:08.105055+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'11269","reported_seq":11404,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:04.881089+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:03:04.881089+0000","last_peered":"2024-09-10T16:03:04.881089+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:03:04.881089+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"36'1200","ondisk_log_start":"36'1200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38961635,"num_objects":9513,"num_object_clones":0,"num_object_copies":19026,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9513,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8949,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3428},{"shards":"2","objects":5209}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104572+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:02:08.104572+0000","last_peered":"2024-09-10T16:02:08.104572+0000","last_clean":"2024-09-10T16:02:08.104572+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:02:08.104572+0000","last_undegraded":"2024-09-10T16:02:08.104572+0000","last_fullsized":"2024-09-10T16:02:08.104572+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'11491","reported_seq":11620,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:04.927961+0000","last_change":"2024-09-10T15:59:16.174113+0000","last_active":"2024-09-10T16:03:04.927961+0000","last_peered":"2024-09-10T16:03:04.927961+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:03:04.927961+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"36'1400","ondisk_log_start":"36'1400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39645184,"num_objects":9679,"num_object_clones":0,"num_object_copies":19358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4354,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9679,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9045,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3449},{"shards":"2","objects":5327}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'11290","reported_seq":11425,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:03.064863+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:03:03.064863+0000","last_peered":"2024-09-10T16:03:03.064863+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:03:03.064863+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"36'1200","ondisk_log_start":"36'1200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39108608,"num_objects":9548,"num_object_clones":0,"num_object_copies":19096,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9548,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7921,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5244},{"shards":"0,2","objects":3434}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103276+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:02:08.103276+0000","last_peered":"2024-09-10T16:02:08.103276+0000","last_clean":"2024-09-10T16:02:08.103276+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:02:08.103276+0000","last_undegraded":"2024-09-10T16:02:08.103276+0000","last_fullsized":"2024-09-10T16:02:08.103276+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'11096","reported_seq":13188,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T16:03:03.122010+0000","last_change":"2024-09-10T16:02:06.480033+0000","last_active":"2024-09-10T16:03:03.122010+0000","last_peered":"2024-09-10T16:03:03.122010+0000","last_clean":"2024-09-10T15:59:14.115595+0000","last_became_active":"2024-09-10T15:59:16.244089+0000","last_became_peered":"2024-09-10T15:59:16.244089+0000","last_unstale":"2024-09-10T16:03:03.122010+0000","last_undegraded":"2024-09-10T16:03:03.122010+0000","last_fullsized":"2024-09-10T15:59:15.774437+0000","mapping_epoch":42,"log_start":"36'1000","ondisk_log_start":"36'1000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38920192,"num_objects":9502,"num_object_clones":0,"num_object_copies":19004,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3229,"num_objects_unfound":0,"num_objects_dirty":9502,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7486,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":974,"num_bytes_recovered":3989504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6273},{"shards":"0,2","objects":3229}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103831+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:02:08.103831+0000","last_peered":"2024-09-10T16:02:08.103831+0000","last_clean":"2024-09-10T16:02:08.103831+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:02:08.103831+0000","last_undegraded":"2024-09-10T16:02:08.103831+0000","last_fullsized":"2024-09-10T16:02:08.103831+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'11385","reported_seq":11527,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:03.083287+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:03:03.083287+0000","last_peered":"2024-09-10T16:03:03.083287+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:03:03.083287+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"36'1300","ondisk_log_start":"36'1300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39579648,"num_objects":9663,"num_object_clones":0,"num_object_copies":19326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9663,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7640,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5292},{"shards":"0,2","objects":3512}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104431+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:02:08.104431+0000","last_peered":"2024-09-10T16:02:08.104431+0000","last_clean":"2024-09-10T16:02:08.104431+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:02:08.104431+0000","last_undegraded":"2024-09-10T16:02:08.104431+0000","last_fullsized":"2024-09-10T16:02:08.104431+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104410+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:02:08.104410+0000","last_peered":"2024-09-10T16:02:08.104410+0000","last_clean":"2024-09-10T16:02:08.104410+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:02:08.104410+0000","last_undegraded":"2024-09-10T16:02:08.104410+0000","last_fullsized":"2024-09-10T16:02:08.104410+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'11453","reported_seq":11597,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:03.085560+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:03:03.085560+0000","last_peered":"2024-09-10T16:03:03.085560+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:03:03.085560+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"36'1400","ondisk_log_start":"36'1400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39916486,"num_objects":9747,"num_object_clones":0,"num_object_copies":19494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9747,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7616,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5424},{"shards":"0,2","objects":3475}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":629601014,"num_objects":153721,"num_object_clones":0,"num_object_copies":307442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42911,"num_objects_misplaced":3229,"num_objects_unfound":0,"num_objects_dirty":153721,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":138385,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5246,"num_bytes_recovered":21483520,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1315176448,"data_stored":1315016558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161111,"ondisk_log_size":161111,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986237,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":746684,"kb_used_data":444268,"kb_used_omap":4,"kb_used_meta":302395,"kb_avail":93625156,"statfs":{"total":96636764160,"available":95872159744,"internally_reserved":0,"allocated":454930432,"data_stored":454638199,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":309653162},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,0,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659182,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":555072,"kb_used_data":310960,"kb_used_omap":4,"kb_used_meta":244091,"kb_avail":93816768,"statfs":{"total":96636764160,"available":96068370432,"internally_reserved":0,"allocated":318423040,"data_stored":318083374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":249949870},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,4,0,4,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182465,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":870788,"kb_used_data":532340,"kb_used_omap":4,"kb_used_meta":338427,"kb_avail":93501052,"statfs":{"total":96636764160,"available":95745077248,"internally_reserved":0,"allocated":545116160,"data_stored":544823922,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":346549930},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,4,2,4,5,4,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":543379456,"data_stored":543343326,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":318013440,"data_stored":317929882,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":453783552,"data_stored":453743350,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:03:09.639 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:03:10.210 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:03:10.210 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:03:10.231 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":215,"stamp":"2024-09-10T16:03:09.632061+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":623477513,"num_objects":152228,"num_object_clones":0,"num_object_copies":304456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42911,"num_objects_misplaced":3134,"num_objects_unfound":0,"num_objects_dirty":152228,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":139882,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5343,"num_bytes_recovered":21872659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160709,"ondisk_log_size":160709,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2173572,"kb_used_data":1280272,"kb_used_omap":12,"kb_used_meta":893235,"kb_avail":280941948,"statfs":{"total":289910292480,"available":287684554752,"internally_reserved":0,"allocated":1310998528,"data_stored":1310074391,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":914672642},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,1,4,2,17,4,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-23650304,"num_objects":-5774,"num_object_clones":0,"num_object_copies":-11548,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-273,"num_objects_unfound":0,"num_objects_dirty":-5774,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5774,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":273,"num_bytes_recovered":1118208,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005020"},"pg_stats":[{"pgid":"2.f","version":"45'11313","reported_seq":11383,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:08.105171+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:03:08.105171+0000","last_peered":"2024-09-10T16:03:08.105171+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:03:08.105171+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"36'1300","ondisk_log_start":"36'1300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38334464,"num_objects":9359,"num_object_clones":0,"num_object_copies":18718,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9359,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11313,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5100},{"shards":"0,2","objects":3281}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'11296","reported_seq":11409,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:03:04.771717+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:03:04.771717+0000","last_peered":"2024-09-10T16:03:04.771717+0000","last_clean":"2024-09-10T16:03:04.771717+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:03:04.771717+0000","last_undegraded":"2024-09-10T16:03:04.771717+0000","last_fullsized":"2024-09-10T16:03:04.771717+0000","mapping_epoch":35,"log_start":"36'1200","ondisk_log_start":"36'1200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39109574,"num_objects":9550,"num_object_clones":0,"num_object_copies":19100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9550,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":9401,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'11317","reported_seq":19995,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:03:05.711020+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:03:05.711020+0000","last_peered":"2024-09-10T16:03:05.711020+0000","last_clean":"2024-09-10T16:03:05.711020+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:03:05.711020+0000","last_undegraded":"2024-09-10T16:03:05.711020+0000","last_fullsized":"2024-09-10T16:03:05.711020+0000","mapping_epoch":44,"log_start":"36'1300","ondisk_log_start":"36'1300","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38830104,"num_objects":9481,"num_object_clones":0,"num_object_copies":18962,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9481,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9321,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'11226","reported_seq":11307,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:03:04.849150+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:03:04.849150+0000","last_peered":"2024-09-10T16:03:04.849150+0000","last_clean":"2024-09-10T16:03:04.849150+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:03:04.849150+0000","last_undegraded":"2024-09-10T16:03:04.849150+0000","last_fullsized":"2024-09-10T16:03:04.849150+0000","mapping_epoch":35,"log_start":"36'1200","ondisk_log_start":"36'1200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38641664,"num_objects":9434,"num_object_clones":0,"num_object_copies":18868,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9434,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11226,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'11374","reported_seq":11504,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:03:08.129607+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:03:08.129607+0000","last_peered":"2024-09-10T16:03:08.129607+0000","last_clean":"2024-09-10T16:03:08.129607+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:03:08.129607+0000","last_undegraded":"2024-09-10T16:03:08.129607+0000","last_fullsized":"2024-09-10T16:03:08.129607+0000","mapping_epoch":34,"log_start":"36'1300","ondisk_log_start":"36'1300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38982115,"num_objects":9518,"num_object_clones":0,"num_object_copies":19036,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9518,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8085,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'11516","reported_seq":11654,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:08.150150+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:03:08.150150+0000","last_peered":"2024-09-10T16:03:08.150150+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:03:08.150150+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"36'1500","ondisk_log_start":"36'1500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39096803,"num_objects":9546,"num_object_clones":0,"num_object_copies":19092,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9546,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8128,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5175},{"shards":"0,2","objects":3388}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'11493","reported_seq":11631,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:08.106888+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:03:08.106888+0000","last_peered":"2024-09-10T16:03:08.106888+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:03:08.106888+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"36'1400","ondisk_log_start":"36'1400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39277510,"num_objects":9591,"num_object_clones":0,"num_object_copies":19182,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9591,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":7992,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5354},{"shards":"0,2","objects":3288}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'11610","reported_seq":11740,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:03:08.146355+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:03:08.146355+0000","last_peered":"2024-09-10T16:03:08.146355+0000","last_clean":"2024-09-10T16:03:08.146355+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:03:08.146355+0000","last_undegraded":"2024-09-10T16:03:08.146355+0000","last_fullsized":"2024-09-10T16:03:08.146355+0000","mapping_epoch":34,"log_start":"36'1600","ondisk_log_start":"36'1600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39747584,"num_objects":9704,"num_object_clones":0,"num_object_copies":19408,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9704,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8457,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104219+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:02:08.104219+0000","last_peered":"2024-09-10T16:02:08.104219+0000","last_clean":"2024-09-10T16:02:08.104219+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:02:08.104219+0000","last_undegraded":"2024-09-10T16:02:08.104219+0000","last_fullsized":"2024-09-10T16:02:08.104219+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'11083","reported_seq":11223,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:04.939019+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:03:04.939019+0000","last_peered":"2024-09-10T16:03:04.939019+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:03:04.939019+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"36'1000","ondisk_log_start":"36'1000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38396387,"num_objects":9375,"num_object_clones":0,"num_object_copies":18750,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9375,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8902,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3272},{"shards":"2","objects":5252}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103774+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:02:08.103774+0000","last_peered":"2024-09-10T16:02:08.103774+0000","last_clean":"2024-09-10T16:02:08.103774+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:02:08.103774+0000","last_undegraded":"2024-09-10T16:02:08.103774+0000","last_fullsized":"2024-09-10T16:02:08.103774+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'11404","reported_seq":11533,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:08.146516+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:03:08.146516+0000","last_peered":"2024-09-10T16:03:08.146516+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:03:08.146516+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"36'1400","ondisk_log_start":"36'1400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38936576,"num_objects":9506,"num_object_clones":0,"num_object_copies":19012,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9506,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7908,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5251},{"shards":"0,2","objects":3310}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":104,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.105055+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:02:08.105055+0000","last_peered":"2024-09-10T16:02:08.105055+0000","last_clean":"2024-09-10T16:02:08.105055+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:02:08.105055+0000","last_undegraded":"2024-09-10T16:02:08.105055+0000","last_fullsized":"2024-09-10T16:02:08.105055+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'11269","reported_seq":11404,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:04.881089+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:03:04.881089+0000","last_peered":"2024-09-10T16:03:04.881089+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:03:04.881089+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"36'1200","ondisk_log_start":"36'1200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38961635,"num_objects":9513,"num_object_clones":0,"num_object_copies":19026,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9513,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8949,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3428},{"shards":"2","objects":5209}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104572+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:02:08.104572+0000","last_peered":"2024-09-10T16:02:08.104572+0000","last_clean":"2024-09-10T16:02:08.104572+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:02:08.104572+0000","last_undegraded":"2024-09-10T16:02:08.104572+0000","last_fullsized":"2024-09-10T16:02:08.104572+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'11491","reported_seq":11620,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:04.927961+0000","last_change":"2024-09-10T15:59:16.174113+0000","last_active":"2024-09-10T16:03:04.927961+0000","last_peered":"2024-09-10T16:03:04.927961+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:03:04.927961+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"36'1400","ondisk_log_start":"36'1400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39645184,"num_objects":9679,"num_object_clones":0,"num_object_copies":19358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4354,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9679,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9045,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3449},{"shards":"2","objects":5327}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'11410","reported_seq":11545,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:08.106968+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:03:08.106968+0000","last_peered":"2024-09-10T16:03:08.106968+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:03:08.106968+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"36'1400","ondisk_log_start":"36'1400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38617088,"num_objects":9428,"num_object_clones":0,"num_object_copies":18856,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9428,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8041,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5124},{"shards":"0,2","objects":3314}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103276+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:02:08.103276+0000","last_peered":"2024-09-10T16:02:08.103276+0000","last_clean":"2024-09-10T16:02:08.103276+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:02:08.103276+0000","last_undegraded":"2024-09-10T16:02:08.103276+0000","last_fullsized":"2024-09-10T16:02:08.103276+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'11219","reported_seq":13501,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T16:03:08.170267+0000","last_change":"2024-09-10T16:02:06.480033+0000","last_active":"2024-09-10T16:03:08.170267+0000","last_peered":"2024-09-10T16:03:08.170267+0000","last_clean":"2024-09-10T15:59:14.115595+0000","last_became_active":"2024-09-10T15:59:16.244089+0000","last_became_peered":"2024-09-10T15:59:16.244089+0000","last_unstale":"2024-09-10T16:03:08.170267+0000","last_undegraded":"2024-09-10T16:03:08.170267+0000","last_fullsized":"2024-09-10T15:59:15.774437+0000","mapping_epoch":42,"log_start":"36'1200","ondisk_log_start":"36'1200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38416384,"num_objects":9379,"num_object_clones":0,"num_object_copies":18758,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3134,"num_objects_unfound":0,"num_objects_dirty":9379,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7609,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1069,"num_bytes_recovered":4378624,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6245},{"shards":"0,2","objects":3134}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103831+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:02:08.103831+0000","last_peered":"2024-09-10T16:02:08.103831+0000","last_clean":"2024-09-10T16:02:08.103831+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:02:08.103831+0000","last_undegraded":"2024-09-10T16:02:08.103831+0000","last_fullsized":"2024-09-10T16:02:08.103831+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'11510","reported_seq":11652,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:08.128645+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:03:08.128645+0000","last_peered":"2024-09-10T16:03:08.128645+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:03:08.128645+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"36'1500","ondisk_log_start":"36'1500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39067648,"num_objects":9538,"num_object_clones":0,"num_object_copies":19076,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9538,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7765,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5167},{"shards":"0,2","objects":3387}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104431+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:02:08.104431+0000","last_peered":"2024-09-10T16:02:08.104431+0000","last_clean":"2024-09-10T16:02:08.104431+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:02:08.104431+0000","last_undegraded":"2024-09-10T16:02:08.104431+0000","last_fullsized":"2024-09-10T16:02:08.104431+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104410+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:02:08.104410+0000","last_peered":"2024-09-10T16:02:08.104410+0000","last_clean":"2024-09-10T16:02:08.104410+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:02:08.104410+0000","last_undegraded":"2024-09-10T16:02:08.104410+0000","last_fullsized":"2024-09-10T16:02:08.104410+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'11575","reported_seq":11717,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:08.092640+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:03:08.092640+0000","last_peered":"2024-09-10T16:03:08.092640+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:03:08.092640+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"36'1500","ondisk_log_start":"36'1500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39416774,"num_objects":9625,"num_object_clones":0,"num_object_copies":19250,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9625,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7738,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5302},{"shards":"0,2","objects":3353}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":623477494,"num_objects":152226,"num_object_clones":0,"num_object_copies":304452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42911,"num_objects_misplaced":3134,"num_objects_unfound":0,"num_objects_dirty":152226,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":139880,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5341,"num_bytes_recovered":21872640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1307668480,"data_stored":1307508590,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160706,"ondisk_log_size":160706,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986237,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":746684,"kb_used_data":444268,"kb_used_omap":4,"kb_used_meta":302395,"kb_avail":93625156,"statfs":{"total":96636764160,"available":95872159744,"internally_reserved":0,"allocated":454930432,"data_stored":454638199,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":309653162},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,0,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659183,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":558468,"kb_used_data":310192,"kb_used_omap":4,"kb_used_meta":248251,"kb_avail":93813372,"statfs":{"total":96636764160,"available":96064892928,"internally_reserved":0,"allocated":317636608,"data_stored":317296942,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":254209710},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,2,2,4,1,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182466,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":868420,"kb_used_data":525812,"kb_used_omap":4,"kb_used_meta":342587,"kb_avail":93503420,"statfs":{"total":96636764160,"available":95747502080,"internally_reserved":0,"allocated":538431488,"data_stored":538139250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":350809770},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,0,10,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":536674304,"data_stored":536638174,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":317210624,"data_stored":317127066,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":453783552,"data_stored":453743350,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:03:10.232 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:03:10.777 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:03:10.777 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:03:10.804 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":215,"stamp":"2024-09-10T16:03:09.632061+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":623477513,"num_objects":152228,"num_object_clones":0,"num_object_copies":304456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42911,"num_objects_misplaced":3134,"num_objects_unfound":0,"num_objects_dirty":152228,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":139882,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5343,"num_bytes_recovered":21872659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160709,"ondisk_log_size":160709,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2173572,"kb_used_data":1280272,"kb_used_omap":12,"kb_used_meta":893235,"kb_avail":280941948,"statfs":{"total":289910292480,"available":287684554752,"internally_reserved":0,"allocated":1310998528,"data_stored":1310074391,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":914672642},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,1,4,2,17,4,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-23650304,"num_objects":-5774,"num_object_clones":0,"num_object_copies":-11548,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-273,"num_objects_unfound":0,"num_objects_dirty":-5774,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5774,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":273,"num_bytes_recovered":1118208,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005020"},"pg_stats":[{"pgid":"2.f","version":"45'11313","reported_seq":11383,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:08.105171+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:03:08.105171+0000","last_peered":"2024-09-10T16:03:08.105171+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:03:08.105171+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"36'1300","ondisk_log_start":"36'1300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38334464,"num_objects":9359,"num_object_clones":0,"num_object_copies":18718,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9359,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11313,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5100},{"shards":"0,2","objects":3281}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'11296","reported_seq":11409,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:03:04.771717+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:03:04.771717+0000","last_peered":"2024-09-10T16:03:04.771717+0000","last_clean":"2024-09-10T16:03:04.771717+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:03:04.771717+0000","last_undegraded":"2024-09-10T16:03:04.771717+0000","last_fullsized":"2024-09-10T16:03:04.771717+0000","mapping_epoch":35,"log_start":"36'1200","ondisk_log_start":"36'1200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39109574,"num_objects":9550,"num_object_clones":0,"num_object_copies":19100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9550,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":9401,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'11317","reported_seq":19995,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:03:05.711020+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:03:05.711020+0000","last_peered":"2024-09-10T16:03:05.711020+0000","last_clean":"2024-09-10T16:03:05.711020+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:03:05.711020+0000","last_undegraded":"2024-09-10T16:03:05.711020+0000","last_fullsized":"2024-09-10T16:03:05.711020+0000","mapping_epoch":44,"log_start":"36'1300","ondisk_log_start":"36'1300","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38830104,"num_objects":9481,"num_object_clones":0,"num_object_copies":18962,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9481,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9321,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'11226","reported_seq":11307,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:03:04.849150+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:03:04.849150+0000","last_peered":"2024-09-10T16:03:04.849150+0000","last_clean":"2024-09-10T16:03:04.849150+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:03:04.849150+0000","last_undegraded":"2024-09-10T16:03:04.849150+0000","last_fullsized":"2024-09-10T16:03:04.849150+0000","mapping_epoch":35,"log_start":"36'1200","ondisk_log_start":"36'1200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38641664,"num_objects":9434,"num_object_clones":0,"num_object_copies":18868,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9434,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11226,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'11374","reported_seq":11504,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:03:08.129607+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:03:08.129607+0000","last_peered":"2024-09-10T16:03:08.129607+0000","last_clean":"2024-09-10T16:03:08.129607+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:03:08.129607+0000","last_undegraded":"2024-09-10T16:03:08.129607+0000","last_fullsized":"2024-09-10T16:03:08.129607+0000","mapping_epoch":34,"log_start":"36'1300","ondisk_log_start":"36'1300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38982115,"num_objects":9518,"num_object_clones":0,"num_object_copies":19036,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9518,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8085,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'11516","reported_seq":11654,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:08.150150+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:03:08.150150+0000","last_peered":"2024-09-10T16:03:08.150150+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:03:08.150150+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"36'1500","ondisk_log_start":"36'1500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39096803,"num_objects":9546,"num_object_clones":0,"num_object_copies":19092,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9546,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8128,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5175},{"shards":"0,2","objects":3388}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'11493","reported_seq":11631,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:08.106888+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:03:08.106888+0000","last_peered":"2024-09-10T16:03:08.106888+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:03:08.106888+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"36'1400","ondisk_log_start":"36'1400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39277510,"num_objects":9591,"num_object_clones":0,"num_object_copies":19182,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9591,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":7992,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5354},{"shards":"0,2","objects":3288}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'11610","reported_seq":11740,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:03:08.146355+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:03:08.146355+0000","last_peered":"2024-09-10T16:03:08.146355+0000","last_clean":"2024-09-10T16:03:08.146355+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:03:08.146355+0000","last_undegraded":"2024-09-10T16:03:08.146355+0000","last_fullsized":"2024-09-10T16:03:08.146355+0000","mapping_epoch":34,"log_start":"36'1600","ondisk_log_start":"36'1600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39747584,"num_objects":9704,"num_object_clones":0,"num_object_copies":19408,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9704,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8457,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104219+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:02:08.104219+0000","last_peered":"2024-09-10T16:02:08.104219+0000","last_clean":"2024-09-10T16:02:08.104219+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:02:08.104219+0000","last_undegraded":"2024-09-10T16:02:08.104219+0000","last_fullsized":"2024-09-10T16:02:08.104219+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'11083","reported_seq":11223,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:04.939019+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:03:04.939019+0000","last_peered":"2024-09-10T16:03:04.939019+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:03:04.939019+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"36'1000","ondisk_log_start":"36'1000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38396387,"num_objects":9375,"num_object_clones":0,"num_object_copies":18750,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9375,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8902,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3272},{"shards":"2","objects":5252}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103774+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:02:08.103774+0000","last_peered":"2024-09-10T16:02:08.103774+0000","last_clean":"2024-09-10T16:02:08.103774+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:02:08.103774+0000","last_undegraded":"2024-09-10T16:02:08.103774+0000","last_fullsized":"2024-09-10T16:02:08.103774+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'11404","reported_seq":11533,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:08.146516+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:03:08.146516+0000","last_peered":"2024-09-10T16:03:08.146516+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:03:08.146516+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"36'1400","ondisk_log_start":"36'1400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38936576,"num_objects":9506,"num_object_clones":0,"num_object_copies":19012,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9506,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7908,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5251},{"shards":"0,2","objects":3310}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":104,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.105055+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:02:08.105055+0000","last_peered":"2024-09-10T16:02:08.105055+0000","last_clean":"2024-09-10T16:02:08.105055+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:02:08.105055+0000","last_undegraded":"2024-09-10T16:02:08.105055+0000","last_fullsized":"2024-09-10T16:02:08.105055+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'11269","reported_seq":11404,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:04.881089+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:03:04.881089+0000","last_peered":"2024-09-10T16:03:04.881089+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:03:04.881089+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"36'1200","ondisk_log_start":"36'1200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38961635,"num_objects":9513,"num_object_clones":0,"num_object_copies":19026,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9513,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8949,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3428},{"shards":"2","objects":5209}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104572+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:02:08.104572+0000","last_peered":"2024-09-10T16:02:08.104572+0000","last_clean":"2024-09-10T16:02:08.104572+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:02:08.104572+0000","last_undegraded":"2024-09-10T16:02:08.104572+0000","last_fullsized":"2024-09-10T16:02:08.104572+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'11491","reported_seq":11620,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:04.927961+0000","last_change":"2024-09-10T15:59:16.174113+0000","last_active":"2024-09-10T16:03:04.927961+0000","last_peered":"2024-09-10T16:03:04.927961+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:03:04.927961+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"36'1400","ondisk_log_start":"36'1400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39645184,"num_objects":9679,"num_object_clones":0,"num_object_copies":19358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4354,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9679,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9045,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3449},{"shards":"2","objects":5327}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'11410","reported_seq":11545,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:08.106968+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:03:08.106968+0000","last_peered":"2024-09-10T16:03:08.106968+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:03:08.106968+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"36'1400","ondisk_log_start":"36'1400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38617088,"num_objects":9428,"num_object_clones":0,"num_object_copies":18856,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9428,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8041,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5124},{"shards":"0,2","objects":3314}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103276+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:02:08.103276+0000","last_peered":"2024-09-10T16:02:08.103276+0000","last_clean":"2024-09-10T16:02:08.103276+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:02:08.103276+0000","last_undegraded":"2024-09-10T16:02:08.103276+0000","last_fullsized":"2024-09-10T16:02:08.103276+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'11219","reported_seq":13501,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T16:03:08.170267+0000","last_change":"2024-09-10T16:02:06.480033+0000","last_active":"2024-09-10T16:03:08.170267+0000","last_peered":"2024-09-10T16:03:08.170267+0000","last_clean":"2024-09-10T15:59:14.115595+0000","last_became_active":"2024-09-10T15:59:16.244089+0000","last_became_peered":"2024-09-10T15:59:16.244089+0000","last_unstale":"2024-09-10T16:03:08.170267+0000","last_undegraded":"2024-09-10T16:03:08.170267+0000","last_fullsized":"2024-09-10T15:59:15.774437+0000","mapping_epoch":42,"log_start":"36'1200","ondisk_log_start":"36'1200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38416384,"num_objects":9379,"num_object_clones":0,"num_object_copies":18758,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3134,"num_objects_unfound":0,"num_objects_dirty":9379,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7609,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1069,"num_bytes_recovered":4378624,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6245},{"shards":"0,2","objects":3134}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103831+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:02:08.103831+0000","last_peered":"2024-09-10T16:02:08.103831+0000","last_clean":"2024-09-10T16:02:08.103831+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:02:08.103831+0000","last_undegraded":"2024-09-10T16:02:08.103831+0000","last_fullsized":"2024-09-10T16:02:08.103831+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'11510","reported_seq":11652,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:08.128645+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:03:08.128645+0000","last_peered":"2024-09-10T16:03:08.128645+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:03:08.128645+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"36'1500","ondisk_log_start":"36'1500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39067648,"num_objects":9538,"num_object_clones":0,"num_object_copies":19076,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9538,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7765,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5167},{"shards":"0,2","objects":3387}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104431+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:02:08.104431+0000","last_peered":"2024-09-10T16:02:08.104431+0000","last_clean":"2024-09-10T16:02:08.104431+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:02:08.104431+0000","last_undegraded":"2024-09-10T16:02:08.104431+0000","last_fullsized":"2024-09-10T16:02:08.104431+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104410+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:02:08.104410+0000","last_peered":"2024-09-10T16:02:08.104410+0000","last_clean":"2024-09-10T16:02:08.104410+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:02:08.104410+0000","last_undegraded":"2024-09-10T16:02:08.104410+0000","last_fullsized":"2024-09-10T16:02:08.104410+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'11575","reported_seq":11717,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:08.092640+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:03:08.092640+0000","last_peered":"2024-09-10T16:03:08.092640+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:03:08.092640+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"36'1500","ondisk_log_start":"36'1500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39416774,"num_objects":9625,"num_object_clones":0,"num_object_copies":19250,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9625,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7738,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5302},{"shards":"0,2","objects":3353}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":623477494,"num_objects":152226,"num_object_clones":0,"num_object_copies":304452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42911,"num_objects_misplaced":3134,"num_objects_unfound":0,"num_objects_dirty":152226,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":139880,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5341,"num_bytes_recovered":21872640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1307668480,"data_stored":1307508590,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160706,"ondisk_log_size":160706,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986237,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":746684,"kb_used_data":444268,"kb_used_omap":4,"kb_used_meta":302395,"kb_avail":93625156,"statfs":{"total":96636764160,"available":95872159744,"internally_reserved":0,"allocated":454930432,"data_stored":454638199,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":309653162},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,0,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659183,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":558468,"kb_used_data":310192,"kb_used_omap":4,"kb_used_meta":248251,"kb_avail":93813372,"statfs":{"total":96636764160,"available":96064892928,"internally_reserved":0,"allocated":317636608,"data_stored":317296942,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":254209710},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,2,2,4,1,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182466,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":868420,"kb_used_data":525812,"kb_used_omap":4,"kb_used_meta":342587,"kb_avail":93503420,"statfs":{"total":96636764160,"available":95747502080,"internally_reserved":0,"allocated":538431488,"data_stored":538139250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":350809770},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,0,10,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":536674304,"data_stored":536638174,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":317210624,"data_stored":317127066,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":453783552,"data_stored":453743350,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:03:10.806 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-10T16:03:10.806 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "45'11313", 'reported_seq': 11383, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:03:08.105171+0000', 'last_change': '2024-09-10T15:59:16.281558+0000', 'last_active': '2024-09-10T16:03:08.105171+0000', 'last_peered': '2024-09-10T16:03:08.105171+0000', 'last_clean': '2024-09-10T15:59:14.132456+0000', 'last_became_active': '2024-09-10T15:59:16.274407+0000', 'last_became_peered': '2024-09-10T15:59:16.274407+0000', 'last_unstale': '2024-09-10T16:03:08.105171+0000', 'last_undegraded': '2024-09-10T15:59:15.851279+0000', 'last_fullsized': '2024-09-10T15:59:15.774356+0000', 'mapping_epoch': 42, 'log_start': "36'1300", 'ondisk_log_start': "36'1300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'objects_scrubbed': 0, 'log_size': 10013, 'log_dups_size': 0, 'ondisk_log_size': 10013, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38334464, 'num_objects': 9359, 'num_object_clones': 0, 'num_object_copies': 18718, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4259, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9359, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11313, 'num_write_kb': 41344, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5100}, {'shards': '0,2', 'objects': 3281}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:03:10.806 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-10T16:03:10.806 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "45'11516", 'reported_seq': 11654, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:03:08.150150+0000', 'last_change': '2024-09-10T15:59:16.308264+0000', 'last_active': '2024-09-10T16:03:08.150150+0000', 'last_peered': '2024-09-10T16:03:08.150150+0000', 'last_clean': '2024-09-10T15:59:14.111551+0000', 'last_became_active': '2024-09-10T15:59:16.300040+0000', 'last_became_peered': '2024-09-10T15:59:16.300040+0000', 'last_unstale': '2024-09-10T16:03:08.150150+0000', 'last_undegraded': '2024-09-10T15:59:15.878234+0000', 'last_fullsized': '2024-09-10T15:59:15.769328+0000', 'mapping_epoch': 42, 'log_start': "36'1500", 'ondisk_log_start': "36'1500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3442", 'last_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'objects_scrubbed': 3384, 'log_size': 10016, 'log_dups_size': 0, 'ondisk_log_size': 10016, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1292, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39096803, 'num_objects': 9546, 'num_object_clones': 0, 'num_object_copies': 19092, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4372, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9546, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 8128, 'num_write_kb': 28580, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5175}, {'shards': '0,2', 'objects': 3388}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:03:10.806 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-10T16:03:10.806 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "45'11493", 'reported_seq': 11631, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:03:08.106888+0000', 'last_change': '2024-09-10T15:59:16.331899+0000', 'last_active': '2024-09-10T16:03:08.106888+0000', 'last_peered': '2024-09-10T16:03:08.106888+0000', 'last_clean': '2024-09-10T15:59:14.139436+0000', 'last_became_active': '2024-09-10T15:59:16.318715+0000', 'last_became_peered': '2024-09-10T15:59:16.318715+0000', 'last_unstale': '2024-09-10T16:03:08.106888+0000', 'last_undegraded': '2024-09-10T15:59:15.877524+0000', 'last_fullsized': '2024-09-10T15:59:15.769042+0000', 'mapping_epoch': 42, 'log_start': "36'1400", 'ondisk_log_start': "36'1400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3537", 'last_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'objects_scrubbed': 3497, 'log_size': 10093, 'log_dups_size': 0, 'ondisk_log_size': 10093, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1224, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39277510, 'num_objects': 9591, 'num_object_clones': 0, 'num_object_copies': 19182, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4237, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9591, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 7992, 'num_write_kb': 28172, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5354}, {'shards': '0,2', 'objects': 3288}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:03:10.806 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-10T16:03:10.807 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "45'11083", 'reported_seq': 11223, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:03:04.939019+0000', 'last_change': '2024-09-10T15:59:16.175680+0000', 'last_active': '2024-09-10T16:03:04.939019+0000', 'last_peered': '2024-09-10T16:03:04.939019+0000', 'last_clean': '2024-09-10T15:59:14.088187+0000', 'last_became_active': '2024-09-10T15:59:16.165810+0000', 'last_became_peered': '2024-09-10T15:59:16.165810+0000', 'last_unstale': '2024-09-10T16:03:04.939019+0000', 'last_undegraded': '2024-09-10T15:59:15.822098+0000', 'last_fullsized': '2024-09-10T15:59:15.769123+0000', 'mapping_epoch': 42, 'log_start': "36'1000", 'ondisk_log_start': "36'1000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2218", 'last_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'objects_scrubbed': 2175, 'log_size': 10083, 'log_dups_size': 0, 'ondisk_log_size': 10083, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 825, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38396387, 'num_objects': 9375, 'num_object_clones': 0, 'num_object_copies': 18750, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4124, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9375, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 8902, 'num_write_kb': 32204, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3272}, {'shards': '2', 'objects': 5252}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:03:10.807 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-10T16:03:10.807 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "45'11404", 'reported_seq': 11533, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:03:08.146516+0000', 'last_change': '2024-09-10T15:59:16.277742+0000', 'last_active': '2024-09-10T16:03:08.146516+0000', 'last_peered': '2024-09-10T16:03:08.146516+0000', 'last_clean': '2024-09-10T15:59:14.129081+0000', 'last_became_active': '2024-09-10T15:59:16.269835+0000', 'last_became_peered': '2024-09-10T15:59:16.269835+0000', 'last_unstale': '2024-09-10T16:03:08.146516+0000', 'last_undegraded': '2024-09-10T15:59:15.860857+0000', 'last_fullsized': '2024-09-10T15:59:15.771410+0000', 'mapping_epoch': 42, 'log_start': "36'1400", 'ondisk_log_start': "36'1400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3557", 'last_scrub_stamp': '2024-09-10T15:59:02.493678+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:02.493678+0000', 'objects_scrubbed': 3488, 'log_size': 10004, 'log_dups_size': 0, 'ondisk_log_size': 10004, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1472, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38936576, 'num_objects': 9506, 'num_object_clones': 0, 'num_object_copies': 19012, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4257, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9506, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7908, 'num_write_kb': 27852, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5251}, {'shards': '0,2', 'objects': 3310}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:03:10.807 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-10T16:03:10.807 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "45'11269", 'reported_seq': 11404, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:03:04.881089+0000', 'last_change': '2024-09-10T15:59:16.284311+0000', 'last_active': '2024-09-10T16:03:04.881089+0000', 'last_peered': '2024-09-10T16:03:04.881089+0000', 'last_clean': '2024-09-10T15:59:14.116041+0000', 'last_became_active': '2024-09-10T15:59:16.274543+0000', 'last_became_peered': '2024-09-10T15:59:16.274543+0000', 'last_unstale': '2024-09-10T16:03:04.881089+0000', 'last_undegraded': '2024-09-10T15:59:15.812484+0000', 'last_fullsized': '2024-09-10T15:59:15.770885+0000', 'mapping_epoch': 42, 'log_start': "36'1200", 'ondisk_log_start': "36'1200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2356", 'last_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'objects_scrubbed': 2316, 'log_size': 10069, 'log_dups_size': 0, 'ondisk_log_size': 10069, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 783, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38961635, 'num_objects': 9513, 'num_object_clones': 0, 'num_object_copies': 19026, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9513, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 8949, 'num_write_kb': 32292, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3428}, {'shards': '2', 'objects': 5209}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:03:10.807 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-10T16:03:10.807 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "45'11491", 'reported_seq': 11620, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:03:04.927961+0000', 'last_change': '2024-09-10T15:59:16.174113+0000', 'last_active': '2024-09-10T16:03:04.927961+0000', 'last_peered': '2024-09-10T16:03:04.927961+0000', 'last_clean': '2024-09-10T15:59:14.110654+0000', 'last_became_active': '2024-09-10T15:59:16.164645+0000', 'last_became_peered': '2024-09-10T15:59:16.164645+0000', 'last_unstale': '2024-09-10T16:03:04.927961+0000', 'last_undegraded': '2024-09-10T15:59:15.803593+0000', 'last_fullsized': '2024-09-10T15:59:15.769083+0000', 'mapping_epoch': 42, 'log_start': "36'1400", 'ondisk_log_start': "36'1400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2486", 'last_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'objects_scrubbed': 2440, 'log_size': 10091, 'log_dups_size': 0, 'ondisk_log_size': 10091, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 792, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39645184, 'num_objects': 9679, 'num_object_clones': 0, 'num_object_copies': 19358, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4354, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9679, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9045, 'num_write_kb': 32568, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3449}, {'shards': '2', 'objects': 5327}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:03:10.807 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-10T16:03:10.807 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "45'11410", 'reported_seq': 11545, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:03:08.106968+0000', 'last_change': '2024-09-10T15:59:16.315641+0000', 'last_active': '2024-09-10T16:03:08.106968+0000', 'last_peered': '2024-09-10T16:03:08.106968+0000', 'last_clean': '2024-09-10T15:59:14.117001+0000', 'last_became_active': '2024-09-10T15:59:16.302245+0000', 'last_became_peered': '2024-09-10T15:59:16.302245+0000', 'last_unstale': '2024-09-10T16:03:08.106968+0000', 'last_undegraded': '2024-09-10T15:59:15.845012+0000', 'last_fullsized': '2024-09-10T15:59:15.770337+0000', 'mapping_epoch': 42, 'log_start': "36'1400", 'ondisk_log_start': "36'1400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3404", 'last_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'objects_scrubbed': 3367, 'log_size': 10010, 'log_dups_size': 0, 'ondisk_log_size': 10010, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 983, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38617088, 'num_objects': 9428, 'num_object_clones': 0, 'num_object_copies': 18856, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9428, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8041, 'num_write_kb': 28204, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5124}, {'shards': '0,2', 'objects': 3314}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:03:10.807 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-10T16:03:10.807 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "45'11219", 'reported_seq': 13501, 'reported_epoch': 45, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-10T16:03:08.170267+0000', 'last_change': '2024-09-10T16:02:06.480033+0000', 'last_active': '2024-09-10T16:03:08.170267+0000', 'last_peered': '2024-09-10T16:03:08.170267+0000', 'last_clean': '2024-09-10T15:59:14.115595+0000', 'last_became_active': '2024-09-10T15:59:16.244089+0000', 'last_became_peered': '2024-09-10T15:59:16.244089+0000', 'last_unstale': '2024-09-10T16:03:08.170267+0000', 'last_undegraded': '2024-09-10T16:03:08.170267+0000', 'last_fullsized': '2024-09-10T15:59:15.774437+0000', 'mapping_epoch': 42, 'log_start': "36'1200", 'ondisk_log_start': "36'1200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3653", 'last_scrub_stamp': '2024-09-10T15:59:05.298468+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:05.298468+0000', 'objects_scrubbed': 3602, 'log_size': 10019, 'log_dups_size': 0, 'ondisk_log_size': 10019, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1289, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38416384, 'num_objects': 9379, 'num_object_clones': 0, 'num_object_copies': 18758, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 3134, 'num_objects_unfound': 0, 'num_objects_dirty': 9379, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7609, 'num_write_kb': 26772, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1069, 'num_bytes_recovered': 4378624, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6245}, {'shards': '0,2', 'objects': 3134}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:03:10.808 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-10T16:03:10.808 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "45'11510", 'reported_seq': 11652, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:03:08.128645+0000', 'last_change': '2024-09-10T15:59:16.311291+0000', 'last_active': '2024-09-10T16:03:08.128645+0000', 'last_peered': '2024-09-10T16:03:08.128645+0000', 'last_clean': '2024-09-10T15:59:14.116363+0000', 'last_became_active': '2024-09-10T15:59:16.301244+0000', 'last_became_peered': '2024-09-10T15:59:16.301244+0000', 'last_unstale': '2024-09-10T16:03:08.128645+0000', 'last_undegraded': '2024-09-10T15:59:15.883434+0000', 'last_fullsized': '2024-09-10T15:59:15.772756+0000', 'mapping_epoch': 42, 'log_start': "36'1500", 'ondisk_log_start': "36'1500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3789", 'last_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'objects_scrubbed': 3741, 'log_size': 10010, 'log_dups_size': 0, 'ondisk_log_size': 10010, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1440, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39067648, 'num_objects': 9538, 'num_object_clones': 0, 'num_object_copies': 19076, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4373, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9538, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7765, 'num_write_kb': 27124, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5167}, {'shards': '0,2', 'objects': 3387}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:03:10.808 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-10T16:03:10.808 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "45'11575", 'reported_seq': 11717, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:03:08.092640+0000', 'last_change': '2024-09-10T15:59:16.313409+0000', 'last_active': '2024-09-10T16:03:08.092640+0000', 'last_peered': '2024-09-10T16:03:08.092640+0000', 'last_clean': '2024-09-10T15:59:14.129780+0000', 'last_became_active': '2024-09-10T15:59:16.300932+0000', 'last_became_peered': '2024-09-10T15:59:16.300932+0000', 'last_unstale': '2024-09-10T16:03:08.092640+0000', 'last_undegraded': '2024-09-10T15:59:15.887823+0000', 'last_fullsized': '2024-09-10T15:59:15.768513+0000', 'mapping_epoch': 42, 'log_start': "36'1500", 'ondisk_log_start': "36'1500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3880", 'last_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'objects_scrubbed': 3827, 'log_size': 10075, 'log_dups_size': 0, 'ondisk_log_size': 10075, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1397, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39416774, 'num_objects': 9625, 'num_object_clones': 0, 'num_object_copies': 19250, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4325, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9625, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7738, 'num_write_kb': 27072, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5302}, {'shards': '0,2', 'objects': 3353}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:03:10.808 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T16:03:11.495 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:03:11.495 INFO:teuthology.orchestra.run.smithi115.stdout:{"fsid":"7effee29-df05-4351-9741-ee203564ed81","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 42911/304456 objects degraded (14.094%), 10 pgs degraded, 11 pgs undersized","count":21},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":355,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983954,"num_in_osds":3,"osd_in_since":1725983947,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":152228,"data_bytes":623477513,"bytes_used":2225737728,"bytes_avail":287684554752,"bytes_total":289910292480,"degraded_objects":42911,"degraded_total":304456,"degraded_ratio":0.14094319047744172,"misplaced_objects":3134,"misplaced_total":304456,"misplaced_ratio":0.010293769871508526,"recovering_objects_per_sec":22,"recovering_bytes_per_sec":93145,"recovering_keys_per_sec":0,"num_objects_recovered":273,"num_bytes_recovered":1118208,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":480},"fsmap":{"epoch":1,"btime":"2024-09-10T15:57:14:967120+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T16:00:13.560776+0000","services":{}},"progress_events":{}} 2024-09-10T16:03:11.520 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-10T16:03:11.521 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:03:12.135 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:03:12.135 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:03:12.158 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":216,"stamp":"2024-09-10T16:03:11.632688+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":620004105,"num_objects":151380,"num_object_clones":0,"num_object_copies":302760,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42911,"num_objects_misplaced":3134,"num_objects_unfound":0,"num_objects_dirty":151380,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":140730,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5343,"num_bytes_recovered":21872659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160557,"ondisk_log_size":160557,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2180868,"kb_used_data":1275092,"kb_used_omap":12,"kb_used_meta":905715,"kb_avail":280934652,"statfs":{"total":289910292480,"available":287677083648,"internally_reserved":0,"allocated":1305694208,"data_stored":1304770071,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":927452162},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,2,7,2,18,7],"upper_bound":128},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-21401600,"num_objects":-5225,"num_object_clones":0,"num_object_copies":-10450,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-187,"num_objects_unfound":0,"num_objects_dirty":-5225,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5225,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":187,"num_bytes_recovered":765952,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004857"},"pg_stats":[{"pgid":"2.f","version":"45'11313","reported_seq":11383,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:08.105171+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:03:08.105171+0000","last_peered":"2024-09-10T16:03:08.105171+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:03:08.105171+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"36'1300","ondisk_log_start":"36'1300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38334464,"num_objects":9359,"num_object_clones":0,"num_object_copies":18718,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9359,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11313,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5100},{"shards":"0,2","objects":3281}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'11432","reported_seq":11541,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:03:09.861667+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:03:09.861667+0000","last_peered":"2024-09-10T16:03:09.861667+0000","last_clean":"2024-09-10T16:03:09.861667+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:03:09.861667+0000","last_undegraded":"2024-09-10T16:03:09.861667+0000","last_fullsized":"2024-09-10T16:03:09.861667+0000","mapping_epoch":35,"log_start":"36'1400","ondisk_log_start":"36'1400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38552518,"num_objects":9414,"num_object_clones":0,"num_object_copies":18828,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9414,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":9537,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'11447","reported_seq":20125,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:03:10.765079+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:03:10.765079+0000","last_peered":"2024-09-10T16:03:10.765079+0000","last_clean":"2024-09-10T16:03:10.765079+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:03:10.765079+0000","last_undegraded":"2024-09-10T16:03:10.765079+0000","last_fullsized":"2024-09-10T16:03:10.765079+0000","mapping_epoch":44,"log_start":"36'1400","ondisk_log_start":"36'1400","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38297624,"num_objects":9351,"num_object_clones":0,"num_object_copies":18702,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9351,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9451,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'11383","reported_seq":11463,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:03:09.960153+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:03:09.960153+0000","last_peered":"2024-09-10T16:03:09.960153+0000","last_clean":"2024-09-10T16:03:09.960153+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:03:09.960153+0000","last_undegraded":"2024-09-10T16:03:09.960153+0000","last_fullsized":"2024-09-10T16:03:09.960153+0000","mapping_epoch":35,"log_start":"36'1300","ondisk_log_start":"36'1300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37998592,"num_objects":9277,"num_object_clones":0,"num_object_copies":18554,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9277,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11383,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'11374","reported_seq":11504,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:03:08.129607+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:03:08.129607+0000","last_peered":"2024-09-10T16:03:08.129607+0000","last_clean":"2024-09-10T16:03:08.129607+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:03:08.129607+0000","last_undegraded":"2024-09-10T16:03:08.129607+0000","last_fullsized":"2024-09-10T16:03:08.129607+0000","mapping_epoch":34,"log_start":"36'1300","ondisk_log_start":"36'1300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38982115,"num_objects":9518,"num_object_clones":0,"num_object_copies":19036,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9518,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8085,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'11516","reported_seq":11654,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:08.150150+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:03:08.150150+0000","last_peered":"2024-09-10T16:03:08.150150+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:03:08.150150+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"36'1500","ondisk_log_start":"36'1500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39096803,"num_objects":9546,"num_object_clones":0,"num_object_copies":19092,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9546,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8128,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5175},{"shards":"0,2","objects":3388}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'11493","reported_seq":11631,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:08.106888+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:03:08.106888+0000","last_peered":"2024-09-10T16:03:08.106888+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:03:08.106888+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"36'1400","ondisk_log_start":"36'1400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39277510,"num_objects":9591,"num_object_clones":0,"num_object_copies":19182,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9591,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":7992,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5354},{"shards":"0,2","objects":3288}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'11610","reported_seq":11740,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:03:08.146355+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:03:08.146355+0000","last_peered":"2024-09-10T16:03:08.146355+0000","last_clean":"2024-09-10T16:03:08.146355+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:03:08.146355+0000","last_undegraded":"2024-09-10T16:03:08.146355+0000","last_fullsized":"2024-09-10T16:03:08.146355+0000","mapping_epoch":34,"log_start":"36'1600","ondisk_log_start":"36'1600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39747584,"num_objects":9704,"num_object_clones":0,"num_object_copies":19408,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9704,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8457,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104219+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:02:08.104219+0000","last_peered":"2024-09-10T16:02:08.104219+0000","last_clean":"2024-09-10T16:02:08.104219+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:02:08.104219+0000","last_undegraded":"2024-09-10T16:02:08.104219+0000","last_fullsized":"2024-09-10T16:02:08.104219+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'11233","reported_seq":11373,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:09.925779+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:03:09.925779+0000","last_peered":"2024-09-10T16:03:09.925779+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:03:09.925779+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"36'1200","ondisk_log_start":"36'1200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37781987,"num_objects":9225,"num_object_clones":0,"num_object_copies":18450,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9225,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9052,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3122},{"shards":"2","objects":5102}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103774+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:02:08.103774+0000","last_peered":"2024-09-10T16:02:08.103774+0000","last_clean":"2024-09-10T16:02:08.103774+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:02:08.103774+0000","last_undegraded":"2024-09-10T16:02:08.103774+0000","last_fullsized":"2024-09-10T16:02:08.103774+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'11404","reported_seq":11533,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:08.146516+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:03:08.146516+0000","last_peered":"2024-09-10T16:03:08.146516+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:03:08.146516+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"36'1400","ondisk_log_start":"36'1400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38936576,"num_objects":9506,"num_object_clones":0,"num_object_copies":19012,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9506,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7908,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5251},{"shards":"0,2","objects":3310}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":104,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.105055+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:02:08.105055+0000","last_peered":"2024-09-10T16:02:08.105055+0000","last_clean":"2024-09-10T16:02:08.105055+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:02:08.105055+0000","last_undegraded":"2024-09-10T16:02:08.105055+0000","last_fullsized":"2024-09-10T16:02:08.105055+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'11408","reported_seq":11543,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:09.911294+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:03:09.911294+0000","last_peered":"2024-09-10T16:03:09.911294+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:03:09.911294+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"36'1400","ondisk_log_start":"36'1400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38392291,"num_objects":9374,"num_object_clones":0,"num_object_copies":18748,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9374,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9088,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3289},{"shards":"2","objects":5070}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104572+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:02:08.104572+0000","last_peered":"2024-09-10T16:02:08.104572+0000","last_clean":"2024-09-10T16:02:08.104572+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:02:08.104572+0000","last_undegraded":"2024-09-10T16:02:08.104572+0000","last_fullsized":"2024-09-10T16:02:08.104572+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'11627","reported_seq":11756,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:09.919949+0000","last_change":"2024-09-10T15:59:16.174113+0000","last_active":"2024-09-10T16:03:09.919949+0000","last_peered":"2024-09-10T16:03:09.919949+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:03:09.919949+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"36'1600","ondisk_log_start":"36'1600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39088128,"num_objects":9543,"num_object_clones":0,"num_object_copies":19086,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4354,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9543,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9181,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3313},{"shards":"2","objects":5191}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'11410","reported_seq":11545,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:08.106968+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:03:08.106968+0000","last_peered":"2024-09-10T16:03:08.106968+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:03:08.106968+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"36'1400","ondisk_log_start":"36'1400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38617088,"num_objects":9428,"num_object_clones":0,"num_object_copies":18856,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9428,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8041,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5124},{"shards":"0,2","objects":3314}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103276+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:02:08.103276+0000","last_peered":"2024-09-10T16:02:08.103276+0000","last_clean":"2024-09-10T16:02:08.103276+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:02:08.103276+0000","last_undegraded":"2024-09-10T16:02:08.103276+0000","last_fullsized":"2024-09-10T16:02:08.103276+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'11219","reported_seq":13501,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T16:03:08.170267+0000","last_change":"2024-09-10T16:02:06.480033+0000","last_active":"2024-09-10T16:03:08.170267+0000","last_peered":"2024-09-10T16:03:08.170267+0000","last_clean":"2024-09-10T15:59:14.115595+0000","last_became_active":"2024-09-10T15:59:16.244089+0000","last_became_peered":"2024-09-10T15:59:16.244089+0000","last_unstale":"2024-09-10T16:03:08.170267+0000","last_undegraded":"2024-09-10T16:03:08.170267+0000","last_fullsized":"2024-09-10T15:59:15.774437+0000","mapping_epoch":42,"log_start":"36'1200","ondisk_log_start":"36'1200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38416384,"num_objects":9379,"num_object_clones":0,"num_object_copies":18758,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3134,"num_objects_unfound":0,"num_objects_dirty":9379,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7609,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1069,"num_bytes_recovered":4378624,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6245},{"shards":"0,2","objects":3134}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103831+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:02:08.103831+0000","last_peered":"2024-09-10T16:02:08.103831+0000","last_clean":"2024-09-10T16:02:08.103831+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:02:08.103831+0000","last_undegraded":"2024-09-10T16:02:08.103831+0000","last_fullsized":"2024-09-10T16:02:08.103831+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'11510","reported_seq":11652,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:08.128645+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:03:08.128645+0000","last_peered":"2024-09-10T16:03:08.128645+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:03:08.128645+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"36'1500","ondisk_log_start":"36'1500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39067648,"num_objects":9538,"num_object_clones":0,"num_object_copies":19076,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9538,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7765,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5167},{"shards":"0,2","objects":3387}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104431+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:02:08.104431+0000","last_peered":"2024-09-10T16:02:08.104431+0000","last_clean":"2024-09-10T16:02:08.104431+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:02:08.104431+0000","last_undegraded":"2024-09-10T16:02:08.104431+0000","last_fullsized":"2024-09-10T16:02:08.104431+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104410+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:02:08.104410+0000","last_peered":"2024-09-10T16:02:08.104410+0000","last_clean":"2024-09-10T16:02:08.104410+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:02:08.104410+0000","last_undegraded":"2024-09-10T16:02:08.104410+0000","last_fullsized":"2024-09-10T16:02:08.104410+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'11575","reported_seq":11717,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:08.092640+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:03:08.092640+0000","last_peered":"2024-09-10T16:03:08.092640+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:03:08.092640+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"36'1500","ondisk_log_start":"36'1500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39416774,"num_objects":9625,"num_object_clones":0,"num_object_copies":19250,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9625,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7738,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5302},{"shards":"0,2","objects":3353}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":620004086,"num_objects":151378,"num_object_clones":0,"num_object_copies":302756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42911,"num_objects_misplaced":3134,"num_objects_unfound":0,"num_objects_dirty":151378,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":140728,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5341,"num_bytes_recovered":21872640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1302335488,"data_stored":1302175598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160554,"ondisk_log_size":160554,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986238,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":746308,"kb_used_data":439732,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":93625532,"statfs":{"total":96636764160,"available":95872544768,"internally_reserved":0,"allocated":450285568,"data_stored":449993335,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":313913002},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,0,3,1,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659184,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":566140,"kb_used_data":309548,"kb_used_omap":4,"kb_used_meta":256571,"kb_avail":93805700,"statfs":{"total":96636764160,"available":96057036800,"internally_reserved":0,"allocated":316977152,"data_stored":316637486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":262729390},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,3,1,6,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182466,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":868420,"kb_used_data":525812,"kb_used_omap":4,"kb_used_meta":342587,"kb_avail":93503420,"statfs":{"total":96636764160,"available":95747502080,"internally_reserved":0,"allocated":538431488,"data_stored":538139250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":350809770},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,0,10,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":536674304,"data_stored":536638174,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":316555264,"data_stored":316471706,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":449105920,"data_stored":449065718,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:03:15.160 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:03:15.724 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:03:15.727 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:03:15.755 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":218,"stamp":"2024-09-10T16:03:15.634410+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":611635977,"num_objects":149337,"num_object_clones":0,"num_object_copies":298674,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42911,"num_objects_misplaced":3039,"num_objects_unfound":0,"num_objects_dirty":149337,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":142773,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5438,"num_bytes_recovered":22261779,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160700,"ondisk_log_size":160700,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2213564,"kb_used_data":1263372,"kb_used_omap":12,"kb_used_meta":950131,"kb_avail":280901956,"statfs":{"total":289910292480,"available":287643602944,"internally_reserved":0,"allocated":1293692928,"data_stored":1292768791,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":972934146},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3,4,5,4,11,9,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-20824064,"num_objects":-5084,"num_object_clones":0,"num_object_copies":-10168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-190,"num_objects_unfound":0,"num_objects_dirty":-5084,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5084,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":190,"num_bytes_recovered":778240,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004829"},"pg_stats":[{"pgid":"2.f","version":"45'11422","reported_seq":11497,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:13.109496+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:03:13.109496+0000","last_peered":"2024-09-10T16:03:13.109496+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:03:13.109496+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"36'1400","ondisk_log_start":"36'1400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37888000,"num_objects":9250,"num_object_clones":0,"num_object_copies":18500,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9250,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11422,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4991},{"shards":"0,2","objects":3172}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'11570","reported_seq":11683,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:03:14.944818+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:03:14.944818+0000","last_peered":"2024-09-10T16:03:14.944818+0000","last_clean":"2024-09-10T16:03:14.944818+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:03:14.944818+0000","last_undegraded":"2024-09-10T16:03:14.944818+0000","last_fullsized":"2024-09-10T16:03:14.944818+0000","mapping_epoch":35,"log_start":"36'1500","ondisk_log_start":"36'1500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37987270,"num_objects":9276,"num_object_clones":0,"num_object_copies":18552,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9276,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":9675,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'11447","reported_seq":20125,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:03:10.765079+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:03:10.765079+0000","last_peered":"2024-09-10T16:03:10.765079+0000","last_clean":"2024-09-10T16:03:10.765079+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:03:10.765079+0000","last_undegraded":"2024-09-10T16:03:10.765079+0000","last_fullsized":"2024-09-10T16:03:10.765079+0000","mapping_epoch":44,"log_start":"36'1400","ondisk_log_start":"36'1400","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38297624,"num_objects":9351,"num_object_clones":0,"num_object_copies":18702,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9351,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9451,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'11516","reported_seq":11597,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:03:14.821737+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:03:14.821737+0000","last_peered":"2024-09-10T16:03:14.821737+0000","last_clean":"2024-09-10T16:03:14.821737+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:03:14.821737+0000","last_undegraded":"2024-09-10T16:03:14.821737+0000","last_fullsized":"2024-09-10T16:03:14.821737+0000","mapping_epoch":35,"log_start":"36'1500","ondisk_log_start":"36'1500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37453824,"num_objects":9144,"num_object_clones":0,"num_object_copies":18288,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9144,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11516,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'11519","reported_seq":11649,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:03:13.152219+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:03:13.152219+0000","last_peered":"2024-09-10T16:03:13.152219+0000","last_clean":"2024-09-10T16:03:13.152219+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:03:13.152219+0000","last_undegraded":"2024-09-10T16:03:13.152219+0000","last_fullsized":"2024-09-10T16:03:13.152219+0000","mapping_epoch":34,"log_start":"36'1500","ondisk_log_start":"36'1500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38388195,"num_objects":9373,"num_object_clones":0,"num_object_copies":18746,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9373,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8230,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'11653","reported_seq":11791,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:13.069962+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:03:13.069962+0000","last_peered":"2024-09-10T16:03:13.069962+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:03:13.069962+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"36'1600","ondisk_log_start":"36'1600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38535651,"num_objects":9409,"num_object_clones":0,"num_object_copies":18818,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9409,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8265,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5038},{"shards":"0,2","objects":3251}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'11628","reported_seq":11766,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:13.162850+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:03:13.162850+0000","last_peered":"2024-09-10T16:03:13.162850+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:03:13.162850+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"36'1600","ondisk_log_start":"36'1600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38724550,"num_objects":9456,"num_object_clones":0,"num_object_copies":18912,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9456,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":8127,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5219},{"shards":"0,2","objects":3153}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'11755","reported_seq":11885,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:03:13.170028+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:03:13.170028+0000","last_peered":"2024-09-10T16:03:13.170028+0000","last_clean":"2024-09-10T16:03:13.170028+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:03:13.170028+0000","last_undegraded":"2024-09-10T16:03:13.170028+0000","last_fullsized":"2024-09-10T16:03:13.170028+0000","mapping_epoch":34,"log_start":"36'1700","ondisk_log_start":"36'1700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39153664,"num_objects":9559,"num_object_clones":0,"num_object_copies":19118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9559,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8602,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104219+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:02:08.104219+0000","last_peered":"2024-09-10T16:02:08.104219+0000","last_clean":"2024-09-10T16:02:08.104219+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:02:08.104219+0000","last_undegraded":"2024-09-10T16:02:08.104219+0000","last_fullsized":"2024-09-10T16:02:08.104219+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'11362","reported_seq":11502,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:14.943929+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:03:14.943929+0000","last_peered":"2024-09-10T16:03:14.943929+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:03:14.943929+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"36'1300","ondisk_log_start":"36'1300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37253603,"num_objects":9096,"num_object_clones":0,"num_object_copies":18192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9096,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9181,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2993},{"shards":"2","objects":4973}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103774+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:02:08.103774+0000","last_peered":"2024-09-10T16:02:08.103774+0000","last_clean":"2024-09-10T16:02:08.103774+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:02:08.103774+0000","last_undegraded":"2024-09-10T16:02:08.103774+0000","last_fullsized":"2024-09-10T16:02:08.103774+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'11546","reported_seq":11672,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:13.204114+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:03:13.204114+0000","last_peered":"2024-09-10T16:03:13.204114+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:03:13.204114+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"36'1500","ondisk_log_start":"36'1500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38354944,"num_objects":9364,"num_object_clones":0,"num_object_copies":18728,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9364,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8050,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5109},{"shards":"0,2","objects":3168}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":104,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.105055+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:02:08.105055+0000","last_peered":"2024-09-10T16:02:08.105055+0000","last_clean":"2024-09-10T16:02:08.105055+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:02:08.105055+0000","last_undegraded":"2024-09-10T16:02:08.105055+0000","last_fullsized":"2024-09-10T16:02:08.105055+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'11548","reported_seq":11683,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:14.959232+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:03:14.959232+0000","last_peered":"2024-09-10T16:03:14.959232+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:03:14.959232+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"36'1500","ondisk_log_start":"36'1500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37818851,"num_objects":9234,"num_object_clones":0,"num_object_copies":18468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9234,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9228,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3149},{"shards":"2","objects":4930}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104572+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:02:08.104572+0000","last_peered":"2024-09-10T16:02:08.104572+0000","last_clean":"2024-09-10T16:02:08.104572+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:02:08.104572+0000","last_undegraded":"2024-09-10T16:02:08.104572+0000","last_fullsized":"2024-09-10T16:02:08.104572+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'11756","reported_seq":11885,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:14.896557+0000","last_change":"2024-09-10T15:59:16.174113+0000","last_active":"2024-09-10T16:03:14.896557+0000","last_peered":"2024-09-10T16:03:14.896557+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:03:14.896557+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"36'1700","ondisk_log_start":"36'1700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38559744,"num_objects":9414,"num_object_clones":0,"num_object_copies":18828,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4354,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9414,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9310,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3184},{"shards":"2","objects":5062}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'11544","reported_seq":11679,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:13.107152+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:03:13.107152+0000","last_peered":"2024-09-10T16:03:13.107152+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:03:13.107152+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"36'1500","ondisk_log_start":"36'1500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38068224,"num_objects":9294,"num_object_clones":0,"num_object_copies":18588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9294,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8175,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4990},{"shards":"0,2","objects":3180}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103276+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:02:08.103276+0000","last_peered":"2024-09-10T16:02:08.103276+0000","last_clean":"2024-09-10T16:02:08.103276+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:02:08.103276+0000","last_undegraded":"2024-09-10T16:02:08.103276+0000","last_fullsized":"2024-09-10T16:02:08.103276+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'11335","reported_seq":13807,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T16:03:13.172267+0000","last_change":"2024-09-10T16:02:06.480033+0000","last_active":"2024-09-10T16:03:13.172267+0000","last_peered":"2024-09-10T16:03:13.172267+0000","last_clean":"2024-09-10T15:59:14.115595+0000","last_became_active":"2024-09-10T15:59:16.244089+0000","last_became_peered":"2024-09-10T15:59:16.244089+0000","last_unstale":"2024-09-10T16:03:13.172267+0000","last_undegraded":"2024-09-10T16:03:13.172267+0000","last_fullsized":"2024-09-10T15:59:15.774437+0000","mapping_epoch":42,"log_start":"36'1300","ondisk_log_start":"36'1300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37941248,"num_objects":9263,"num_object_clones":0,"num_object_copies":18526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3039,"num_objects_unfound":0,"num_objects_dirty":9263,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7725,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1164,"num_bytes_recovered":4767744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6224},{"shards":"0,2","objects":3039}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103831+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:02:08.103831+0000","last_peered":"2024-09-10T16:02:08.103831+0000","last_clean":"2024-09-10T16:02:08.103831+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:02:08.103831+0000","last_undegraded":"2024-09-10T16:02:08.103831+0000","last_fullsized":"2024-09-10T16:02:08.103831+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'11669","reported_seq":11811,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:13.191300+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:03:13.191300+0000","last_peered":"2024-09-10T16:03:13.191300+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:03:13.191300+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"36'1600","ondisk_log_start":"36'1600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38416384,"num_objects":9379,"num_object_clones":0,"num_object_copies":18758,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9379,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7924,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5008},{"shards":"0,2","objects":3228}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104431+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:02:08.104431+0000","last_peered":"2024-09-10T16:02:08.104431+0000","last_clean":"2024-09-10T16:02:08.104431+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:02:08.104431+0000","last_undegraded":"2024-09-10T16:02:08.104431+0000","last_fullsized":"2024-09-10T16:02:08.104431+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104410+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:02:08.104410+0000","last_peered":"2024-09-10T16:02:08.104410+0000","last_clean":"2024-09-10T16:02:08.104410+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:02:08.104410+0000","last_undegraded":"2024-09-10T16:02:08.104410+0000","last_fullsized":"2024-09-10T16:02:08.104410+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'11727","reported_seq":11871,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:13.136013+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:03:13.136013+0000","last_peered":"2024-09-10T16:03:13.136013+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:03:13.136013+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"36'1700","ondisk_log_start":"36'1700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38794182,"num_objects":9473,"num_object_clones":0,"num_object_copies":18946,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9473,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7890,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5150},{"shards":"0,2","objects":3201}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":611635958,"num_objects":149335,"num_object_clones":0,"num_object_copies":298670,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42911,"num_objects_misplaced":3039,"num_objects_unfound":0,"num_objects_dirty":149335,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":142771,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5436,"num_bytes_recovered":22261760,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1291157504,"data_stored":1290997614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160697,"ondisk_log_size":160697,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986239,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":745376,"kb_used_data":434640,"kb_used_omap":4,"kb_used_meta":310715,"kb_avail":93626464,"statfs":{"total":96636764160,"available":95873499136,"internally_reserved":0,"allocated":445071360,"data_stored":444779127,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":318172842},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,1,4],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659184,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":566140,"kb_used_data":309548,"kb_used_omap":4,"kb_used_meta":256571,"kb_avail":93805700,"statfs":{"total":96636764160,"available":96057036800,"internally_reserved":0,"allocated":316977152,"data_stored":316637486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":262729390},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,3,1,6,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182467,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":902048,"kb_used_data":519184,"kb_used_omap":4,"kb_used_meta":382843,"kb_avail":93469792,"statfs":{"total":96636764160,"available":95713067008,"internally_reserved":0,"allocated":531644416,"data_stored":531352178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":392031914},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3,1,1,2,1,5,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":529924096,"data_stored":529887966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":316555264,"data_stored":316471706,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":444678144,"data_stored":444637942,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:03:15.756 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:03:16.302 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:03:16.305 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:03:16.323 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":218,"stamp":"2024-09-10T16:03:15.634410+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":611635977,"num_objects":149337,"num_object_clones":0,"num_object_copies":298674,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42911,"num_objects_misplaced":3039,"num_objects_unfound":0,"num_objects_dirty":149337,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":142773,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5438,"num_bytes_recovered":22261779,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160700,"ondisk_log_size":160700,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2213564,"kb_used_data":1263372,"kb_used_omap":12,"kb_used_meta":950131,"kb_avail":280901956,"statfs":{"total":289910292480,"available":287643602944,"internally_reserved":0,"allocated":1293692928,"data_stored":1292768791,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":972934146},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3,4,5,4,11,9,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-20824064,"num_objects":-5084,"num_object_clones":0,"num_object_copies":-10168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-190,"num_objects_unfound":0,"num_objects_dirty":-5084,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5084,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":190,"num_bytes_recovered":778240,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004829"},"pg_stats":[{"pgid":"2.f","version":"45'11422","reported_seq":11497,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:13.109496+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:03:13.109496+0000","last_peered":"2024-09-10T16:03:13.109496+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:03:13.109496+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"36'1400","ondisk_log_start":"36'1400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37888000,"num_objects":9250,"num_object_clones":0,"num_object_copies":18500,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9250,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11422,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4991},{"shards":"0,2","objects":3172}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'11570","reported_seq":11683,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:03:14.944818+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:03:14.944818+0000","last_peered":"2024-09-10T16:03:14.944818+0000","last_clean":"2024-09-10T16:03:14.944818+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:03:14.944818+0000","last_undegraded":"2024-09-10T16:03:14.944818+0000","last_fullsized":"2024-09-10T16:03:14.944818+0000","mapping_epoch":35,"log_start":"36'1500","ondisk_log_start":"36'1500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37987270,"num_objects":9276,"num_object_clones":0,"num_object_copies":18552,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9276,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":9675,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'11447","reported_seq":20125,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:03:10.765079+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:03:10.765079+0000","last_peered":"2024-09-10T16:03:10.765079+0000","last_clean":"2024-09-10T16:03:10.765079+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:03:10.765079+0000","last_undegraded":"2024-09-10T16:03:10.765079+0000","last_fullsized":"2024-09-10T16:03:10.765079+0000","mapping_epoch":44,"log_start":"36'1400","ondisk_log_start":"36'1400","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38297624,"num_objects":9351,"num_object_clones":0,"num_object_copies":18702,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9351,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9451,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'11516","reported_seq":11597,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:03:14.821737+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:03:14.821737+0000","last_peered":"2024-09-10T16:03:14.821737+0000","last_clean":"2024-09-10T16:03:14.821737+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:03:14.821737+0000","last_undegraded":"2024-09-10T16:03:14.821737+0000","last_fullsized":"2024-09-10T16:03:14.821737+0000","mapping_epoch":35,"log_start":"36'1500","ondisk_log_start":"36'1500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37453824,"num_objects":9144,"num_object_clones":0,"num_object_copies":18288,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9144,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11516,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'11519","reported_seq":11649,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:03:13.152219+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:03:13.152219+0000","last_peered":"2024-09-10T16:03:13.152219+0000","last_clean":"2024-09-10T16:03:13.152219+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:03:13.152219+0000","last_undegraded":"2024-09-10T16:03:13.152219+0000","last_fullsized":"2024-09-10T16:03:13.152219+0000","mapping_epoch":34,"log_start":"36'1500","ondisk_log_start":"36'1500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38388195,"num_objects":9373,"num_object_clones":0,"num_object_copies":18746,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9373,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8230,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'11653","reported_seq":11791,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:13.069962+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:03:13.069962+0000","last_peered":"2024-09-10T16:03:13.069962+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:03:13.069962+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"36'1600","ondisk_log_start":"36'1600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38535651,"num_objects":9409,"num_object_clones":0,"num_object_copies":18818,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9409,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8265,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5038},{"shards":"0,2","objects":3251}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'11628","reported_seq":11766,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:13.162850+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:03:13.162850+0000","last_peered":"2024-09-10T16:03:13.162850+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:03:13.162850+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"36'1600","ondisk_log_start":"36'1600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38724550,"num_objects":9456,"num_object_clones":0,"num_object_copies":18912,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9456,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":8127,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5219},{"shards":"0,2","objects":3153}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'11755","reported_seq":11885,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:03:13.170028+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:03:13.170028+0000","last_peered":"2024-09-10T16:03:13.170028+0000","last_clean":"2024-09-10T16:03:13.170028+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:03:13.170028+0000","last_undegraded":"2024-09-10T16:03:13.170028+0000","last_fullsized":"2024-09-10T16:03:13.170028+0000","mapping_epoch":34,"log_start":"36'1700","ondisk_log_start":"36'1700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39153664,"num_objects":9559,"num_object_clones":0,"num_object_copies":19118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9559,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8602,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104219+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:02:08.104219+0000","last_peered":"2024-09-10T16:02:08.104219+0000","last_clean":"2024-09-10T16:02:08.104219+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:02:08.104219+0000","last_undegraded":"2024-09-10T16:02:08.104219+0000","last_fullsized":"2024-09-10T16:02:08.104219+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'11362","reported_seq":11502,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:14.943929+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:03:14.943929+0000","last_peered":"2024-09-10T16:03:14.943929+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:03:14.943929+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"36'1300","ondisk_log_start":"36'1300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37253603,"num_objects":9096,"num_object_clones":0,"num_object_copies":18192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9096,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9181,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2993},{"shards":"2","objects":4973}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103774+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:02:08.103774+0000","last_peered":"2024-09-10T16:02:08.103774+0000","last_clean":"2024-09-10T16:02:08.103774+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:02:08.103774+0000","last_undegraded":"2024-09-10T16:02:08.103774+0000","last_fullsized":"2024-09-10T16:02:08.103774+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'11546","reported_seq":11672,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:13.204114+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:03:13.204114+0000","last_peered":"2024-09-10T16:03:13.204114+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:03:13.204114+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"36'1500","ondisk_log_start":"36'1500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38354944,"num_objects":9364,"num_object_clones":0,"num_object_copies":18728,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9364,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8050,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5109},{"shards":"0,2","objects":3168}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":104,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.105055+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:02:08.105055+0000","last_peered":"2024-09-10T16:02:08.105055+0000","last_clean":"2024-09-10T16:02:08.105055+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:02:08.105055+0000","last_undegraded":"2024-09-10T16:02:08.105055+0000","last_fullsized":"2024-09-10T16:02:08.105055+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'11548","reported_seq":11683,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:14.959232+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:03:14.959232+0000","last_peered":"2024-09-10T16:03:14.959232+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:03:14.959232+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"36'1500","ondisk_log_start":"36'1500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37818851,"num_objects":9234,"num_object_clones":0,"num_object_copies":18468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9234,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9228,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3149},{"shards":"2","objects":4930}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104572+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:02:08.104572+0000","last_peered":"2024-09-10T16:02:08.104572+0000","last_clean":"2024-09-10T16:02:08.104572+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:02:08.104572+0000","last_undegraded":"2024-09-10T16:02:08.104572+0000","last_fullsized":"2024-09-10T16:02:08.104572+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'11756","reported_seq":11885,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:14.896557+0000","last_change":"2024-09-10T15:59:16.174113+0000","last_active":"2024-09-10T16:03:14.896557+0000","last_peered":"2024-09-10T16:03:14.896557+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:03:14.896557+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"36'1700","ondisk_log_start":"36'1700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38559744,"num_objects":9414,"num_object_clones":0,"num_object_copies":18828,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4354,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9414,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9310,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3184},{"shards":"2","objects":5062}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'11544","reported_seq":11679,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:13.107152+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:03:13.107152+0000","last_peered":"2024-09-10T16:03:13.107152+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:03:13.107152+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"36'1500","ondisk_log_start":"36'1500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38068224,"num_objects":9294,"num_object_clones":0,"num_object_copies":18588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9294,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8175,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4990},{"shards":"0,2","objects":3180}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103276+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:02:08.103276+0000","last_peered":"2024-09-10T16:02:08.103276+0000","last_clean":"2024-09-10T16:02:08.103276+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:02:08.103276+0000","last_undegraded":"2024-09-10T16:02:08.103276+0000","last_fullsized":"2024-09-10T16:02:08.103276+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'11335","reported_seq":13807,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T16:03:13.172267+0000","last_change":"2024-09-10T16:02:06.480033+0000","last_active":"2024-09-10T16:03:13.172267+0000","last_peered":"2024-09-10T16:03:13.172267+0000","last_clean":"2024-09-10T15:59:14.115595+0000","last_became_active":"2024-09-10T15:59:16.244089+0000","last_became_peered":"2024-09-10T15:59:16.244089+0000","last_unstale":"2024-09-10T16:03:13.172267+0000","last_undegraded":"2024-09-10T16:03:13.172267+0000","last_fullsized":"2024-09-10T15:59:15.774437+0000","mapping_epoch":42,"log_start":"36'1300","ondisk_log_start":"36'1300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37941248,"num_objects":9263,"num_object_clones":0,"num_object_copies":18526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3039,"num_objects_unfound":0,"num_objects_dirty":9263,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7725,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1164,"num_bytes_recovered":4767744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6224},{"shards":"0,2","objects":3039}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103831+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:02:08.103831+0000","last_peered":"2024-09-10T16:02:08.103831+0000","last_clean":"2024-09-10T16:02:08.103831+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:02:08.103831+0000","last_undegraded":"2024-09-10T16:02:08.103831+0000","last_fullsized":"2024-09-10T16:02:08.103831+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'11669","reported_seq":11811,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:13.191300+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:03:13.191300+0000","last_peered":"2024-09-10T16:03:13.191300+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:03:13.191300+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"36'1600","ondisk_log_start":"36'1600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38416384,"num_objects":9379,"num_object_clones":0,"num_object_copies":18758,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9379,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7924,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5008},{"shards":"0,2","objects":3228}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104431+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:02:08.104431+0000","last_peered":"2024-09-10T16:02:08.104431+0000","last_clean":"2024-09-10T16:02:08.104431+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:02:08.104431+0000","last_undegraded":"2024-09-10T16:02:08.104431+0000","last_fullsized":"2024-09-10T16:02:08.104431+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104410+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:02:08.104410+0000","last_peered":"2024-09-10T16:02:08.104410+0000","last_clean":"2024-09-10T16:02:08.104410+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:02:08.104410+0000","last_undegraded":"2024-09-10T16:02:08.104410+0000","last_fullsized":"2024-09-10T16:02:08.104410+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'11727","reported_seq":11871,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:13.136013+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:03:13.136013+0000","last_peered":"2024-09-10T16:03:13.136013+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:03:13.136013+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"36'1700","ondisk_log_start":"36'1700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38794182,"num_objects":9473,"num_object_clones":0,"num_object_copies":18946,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9473,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7890,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5150},{"shards":"0,2","objects":3201}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":611635958,"num_objects":149335,"num_object_clones":0,"num_object_copies":298670,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42911,"num_objects_misplaced":3039,"num_objects_unfound":0,"num_objects_dirty":149335,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":142771,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5436,"num_bytes_recovered":22261760,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1291157504,"data_stored":1290997614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160697,"ondisk_log_size":160697,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986239,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":745376,"kb_used_data":434640,"kb_used_omap":4,"kb_used_meta":310715,"kb_avail":93626464,"statfs":{"total":96636764160,"available":95873499136,"internally_reserved":0,"allocated":445071360,"data_stored":444779127,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":318172842},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,1,4],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659184,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":566140,"kb_used_data":309548,"kb_used_omap":4,"kb_used_meta":256571,"kb_avail":93805700,"statfs":{"total":96636764160,"available":96057036800,"internally_reserved":0,"allocated":316977152,"data_stored":316637486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":262729390},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,3,1,6,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182467,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":902048,"kb_used_data":519184,"kb_used_omap":4,"kb_used_meta":382843,"kb_avail":93469792,"statfs":{"total":96636764160,"available":95713067008,"internally_reserved":0,"allocated":531644416,"data_stored":531352178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":392031914},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3,1,1,2,1,5,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":529924096,"data_stored":529887966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":316555264,"data_stored":316471706,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":444678144,"data_stored":444637942,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:03:16.325 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-10T16:03:16.325 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "45'11422", 'reported_seq': 11497, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:03:13.109496+0000', 'last_change': '2024-09-10T15:59:16.281558+0000', 'last_active': '2024-09-10T16:03:13.109496+0000', 'last_peered': '2024-09-10T16:03:13.109496+0000', 'last_clean': '2024-09-10T15:59:14.132456+0000', 'last_became_active': '2024-09-10T15:59:16.274407+0000', 'last_became_peered': '2024-09-10T15:59:16.274407+0000', 'last_unstale': '2024-09-10T16:03:13.109496+0000', 'last_undegraded': '2024-09-10T15:59:15.851279+0000', 'last_fullsized': '2024-09-10T15:59:15.774356+0000', 'mapping_epoch': 42, 'log_start': "36'1400", 'ondisk_log_start': "36'1400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'objects_scrubbed': 0, 'log_size': 10022, 'log_dups_size': 0, 'ondisk_log_size': 10022, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37888000, 'num_objects': 9250, 'num_object_clones': 0, 'num_object_copies': 18500, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4259, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9250, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11422, 'num_write_kb': 41344, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4991}, {'shards': '0,2', 'objects': 3172}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:03:16.325 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-10T16:03:16.325 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "45'11653", 'reported_seq': 11791, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:03:13.069962+0000', 'last_change': '2024-09-10T15:59:16.308264+0000', 'last_active': '2024-09-10T16:03:13.069962+0000', 'last_peered': '2024-09-10T16:03:13.069962+0000', 'last_clean': '2024-09-10T15:59:14.111551+0000', 'last_became_active': '2024-09-10T15:59:16.300040+0000', 'last_became_peered': '2024-09-10T15:59:16.300040+0000', 'last_unstale': '2024-09-10T16:03:13.069962+0000', 'last_undegraded': '2024-09-10T15:59:15.878234+0000', 'last_fullsized': '2024-09-10T15:59:15.769328+0000', 'mapping_epoch': 42, 'log_start': "36'1600", 'ondisk_log_start': "36'1600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3442", 'last_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'objects_scrubbed': 3384, 'log_size': 10053, 'log_dups_size': 0, 'ondisk_log_size': 10053, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1292, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38535651, 'num_objects': 9409, 'num_object_clones': 0, 'num_object_copies': 18818, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4372, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9409, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 8265, 'num_write_kb': 28580, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5038}, {'shards': '0,2', 'objects': 3251}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:03:16.325 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-10T16:03:16.325 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "45'11628", 'reported_seq': 11766, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:03:13.162850+0000', 'last_change': '2024-09-10T15:59:16.331899+0000', 'last_active': '2024-09-10T16:03:13.162850+0000', 'last_peered': '2024-09-10T16:03:13.162850+0000', 'last_clean': '2024-09-10T15:59:14.139436+0000', 'last_became_active': '2024-09-10T15:59:16.318715+0000', 'last_became_peered': '2024-09-10T15:59:16.318715+0000', 'last_unstale': '2024-09-10T16:03:13.162850+0000', 'last_undegraded': '2024-09-10T15:59:15.877524+0000', 'last_fullsized': '2024-09-10T15:59:15.769042+0000', 'mapping_epoch': 42, 'log_start': "36'1600", 'ondisk_log_start': "36'1600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3537", 'last_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'objects_scrubbed': 3497, 'log_size': 10028, 'log_dups_size': 0, 'ondisk_log_size': 10028, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1224, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38724550, 'num_objects': 9456, 'num_object_clones': 0, 'num_object_copies': 18912, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4237, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9456, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 8127, 'num_write_kb': 28172, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5219}, {'shards': '0,2', 'objects': 3153}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:03:16.326 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-10T16:03:16.326 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "45'11362", 'reported_seq': 11502, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:03:14.943929+0000', 'last_change': '2024-09-10T15:59:16.175680+0000', 'last_active': '2024-09-10T16:03:14.943929+0000', 'last_peered': '2024-09-10T16:03:14.943929+0000', 'last_clean': '2024-09-10T15:59:14.088187+0000', 'last_became_active': '2024-09-10T15:59:16.165810+0000', 'last_became_peered': '2024-09-10T15:59:16.165810+0000', 'last_unstale': '2024-09-10T16:03:14.943929+0000', 'last_undegraded': '2024-09-10T15:59:15.822098+0000', 'last_fullsized': '2024-09-10T15:59:15.769123+0000', 'mapping_epoch': 42, 'log_start': "36'1300", 'ondisk_log_start': "36'1300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2218", 'last_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'objects_scrubbed': 2175, 'log_size': 10062, 'log_dups_size': 0, 'ondisk_log_size': 10062, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 825, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37253603, 'num_objects': 9096, 'num_object_clones': 0, 'num_object_copies': 18192, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4124, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9096, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 9181, 'num_write_kb': 32204, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2993}, {'shards': '2', 'objects': 4973}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:03:16.326 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-10T16:03:16.326 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "45'11546", 'reported_seq': 11672, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:03:13.204114+0000', 'last_change': '2024-09-10T15:59:16.277742+0000', 'last_active': '2024-09-10T16:03:13.204114+0000', 'last_peered': '2024-09-10T16:03:13.204114+0000', 'last_clean': '2024-09-10T15:59:14.129081+0000', 'last_became_active': '2024-09-10T15:59:16.269835+0000', 'last_became_peered': '2024-09-10T15:59:16.269835+0000', 'last_unstale': '2024-09-10T16:03:13.204114+0000', 'last_undegraded': '2024-09-10T15:59:15.860857+0000', 'last_fullsized': '2024-09-10T15:59:15.771410+0000', 'mapping_epoch': 42, 'log_start': "36'1500", 'ondisk_log_start': "36'1500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3557", 'last_scrub_stamp': '2024-09-10T15:59:02.493678+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:02.493678+0000', 'objects_scrubbed': 3488, 'log_size': 10046, 'log_dups_size': 0, 'ondisk_log_size': 10046, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1472, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38354944, 'num_objects': 9364, 'num_object_clones': 0, 'num_object_copies': 18728, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4257, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9364, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8050, 'num_write_kb': 27852, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5109}, {'shards': '0,2', 'objects': 3168}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:03:16.326 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-10T16:03:16.326 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "45'11548", 'reported_seq': 11683, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:03:14.959232+0000', 'last_change': '2024-09-10T15:59:16.284311+0000', 'last_active': '2024-09-10T16:03:14.959232+0000', 'last_peered': '2024-09-10T16:03:14.959232+0000', 'last_clean': '2024-09-10T15:59:14.116041+0000', 'last_became_active': '2024-09-10T15:59:16.274543+0000', 'last_became_peered': '2024-09-10T15:59:16.274543+0000', 'last_unstale': '2024-09-10T16:03:14.959232+0000', 'last_undegraded': '2024-09-10T15:59:15.812484+0000', 'last_fullsized': '2024-09-10T15:59:15.770885+0000', 'mapping_epoch': 42, 'log_start': "36'1500", 'ondisk_log_start': "36'1500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2356", 'last_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'objects_scrubbed': 2316, 'log_size': 10048, 'log_dups_size': 0, 'ondisk_log_size': 10048, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 783, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37818851, 'num_objects': 9234, 'num_object_clones': 0, 'num_object_copies': 18468, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9234, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 9228, 'num_write_kb': 32292, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3149}, {'shards': '2', 'objects': 4930}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:03:16.326 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-10T16:03:16.326 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "45'11756", 'reported_seq': 11885, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:03:14.896557+0000', 'last_change': '2024-09-10T15:59:16.174113+0000', 'last_active': '2024-09-10T16:03:14.896557+0000', 'last_peered': '2024-09-10T16:03:14.896557+0000', 'last_clean': '2024-09-10T15:59:14.110654+0000', 'last_became_active': '2024-09-10T15:59:16.164645+0000', 'last_became_peered': '2024-09-10T15:59:16.164645+0000', 'last_unstale': '2024-09-10T16:03:14.896557+0000', 'last_undegraded': '2024-09-10T15:59:15.803593+0000', 'last_fullsized': '2024-09-10T15:59:15.769083+0000', 'mapping_epoch': 42, 'log_start': "36'1700", 'ondisk_log_start': "36'1700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2486", 'last_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'objects_scrubbed': 2440, 'log_size': 10056, 'log_dups_size': 0, 'ondisk_log_size': 10056, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 792, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38559744, 'num_objects': 9414, 'num_object_clones': 0, 'num_object_copies': 18828, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4354, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9414, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9310, 'num_write_kb': 32568, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3184}, {'shards': '2', 'objects': 5062}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:03:16.326 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-10T16:03:16.326 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "45'11544", 'reported_seq': 11679, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:03:13.107152+0000', 'last_change': '2024-09-10T15:59:16.315641+0000', 'last_active': '2024-09-10T16:03:13.107152+0000', 'last_peered': '2024-09-10T16:03:13.107152+0000', 'last_clean': '2024-09-10T15:59:14.117001+0000', 'last_became_active': '2024-09-10T15:59:16.302245+0000', 'last_became_peered': '2024-09-10T15:59:16.302245+0000', 'last_unstale': '2024-09-10T16:03:13.107152+0000', 'last_undegraded': '2024-09-10T15:59:15.845012+0000', 'last_fullsized': '2024-09-10T15:59:15.770337+0000', 'mapping_epoch': 42, 'log_start': "36'1500", 'ondisk_log_start': "36'1500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3404", 'last_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'objects_scrubbed': 3367, 'log_size': 10044, 'log_dups_size': 0, 'ondisk_log_size': 10044, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 983, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38068224, 'num_objects': 9294, 'num_object_clones': 0, 'num_object_copies': 18588, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9294, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8175, 'num_write_kb': 28204, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4990}, {'shards': '0,2', 'objects': 3180}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:03:16.327 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-10T16:03:16.327 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "45'11335", 'reported_seq': 13807, 'reported_epoch': 45, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-10T16:03:13.172267+0000', 'last_change': '2024-09-10T16:02:06.480033+0000', 'last_active': '2024-09-10T16:03:13.172267+0000', 'last_peered': '2024-09-10T16:03:13.172267+0000', 'last_clean': '2024-09-10T15:59:14.115595+0000', 'last_became_active': '2024-09-10T15:59:16.244089+0000', 'last_became_peered': '2024-09-10T15:59:16.244089+0000', 'last_unstale': '2024-09-10T16:03:13.172267+0000', 'last_undegraded': '2024-09-10T16:03:13.172267+0000', 'last_fullsized': '2024-09-10T15:59:15.774437+0000', 'mapping_epoch': 42, 'log_start': "36'1300", 'ondisk_log_start': "36'1300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3653", 'last_scrub_stamp': '2024-09-10T15:59:05.298468+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:05.298468+0000', 'objects_scrubbed': 3602, 'log_size': 10035, 'log_dups_size': 0, 'ondisk_log_size': 10035, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1289, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37941248, 'num_objects': 9263, 'num_object_clones': 0, 'num_object_copies': 18526, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 3039, 'num_objects_unfound': 0, 'num_objects_dirty': 9263, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7725, 'num_write_kb': 26772, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1164, 'num_bytes_recovered': 4767744, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6224}, {'shards': '0,2', 'objects': 3039}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:03:16.327 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-10T16:03:16.327 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "45'11669", 'reported_seq': 11811, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:03:13.191300+0000', 'last_change': '2024-09-10T15:59:16.311291+0000', 'last_active': '2024-09-10T16:03:13.191300+0000', 'last_peered': '2024-09-10T16:03:13.191300+0000', 'last_clean': '2024-09-10T15:59:14.116363+0000', 'last_became_active': '2024-09-10T15:59:16.301244+0000', 'last_became_peered': '2024-09-10T15:59:16.301244+0000', 'last_unstale': '2024-09-10T16:03:13.191300+0000', 'last_undegraded': '2024-09-10T15:59:15.883434+0000', 'last_fullsized': '2024-09-10T15:59:15.772756+0000', 'mapping_epoch': 42, 'log_start': "36'1600", 'ondisk_log_start': "36'1600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3789", 'last_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'objects_scrubbed': 3741, 'log_size': 10069, 'log_dups_size': 0, 'ondisk_log_size': 10069, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1440, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38416384, 'num_objects': 9379, 'num_object_clones': 0, 'num_object_copies': 18758, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4373, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9379, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7924, 'num_write_kb': 27124, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5008}, {'shards': '0,2', 'objects': 3228}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:03:16.327 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-10T16:03:16.327 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "45'11727", 'reported_seq': 11871, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:03:13.136013+0000', 'last_change': '2024-09-10T15:59:16.313409+0000', 'last_active': '2024-09-10T16:03:13.136013+0000', 'last_peered': '2024-09-10T16:03:13.136013+0000', 'last_clean': '2024-09-10T15:59:14.129780+0000', 'last_became_active': '2024-09-10T15:59:16.300932+0000', 'last_became_peered': '2024-09-10T15:59:16.300932+0000', 'last_unstale': '2024-09-10T16:03:13.136013+0000', 'last_undegraded': '2024-09-10T15:59:15.887823+0000', 'last_fullsized': '2024-09-10T15:59:15.768513+0000', 'mapping_epoch': 42, 'log_start': "36'1700", 'ondisk_log_start': "36'1700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3880", 'last_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'objects_scrubbed': 3827, 'log_size': 10027, 'log_dups_size': 0, 'ondisk_log_size': 10027, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1397, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38794182, 'num_objects': 9473, 'num_object_clones': 0, 'num_object_copies': 18946, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4325, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9473, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7890, 'num_write_kb': 27072, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5150}, {'shards': '0,2', 'objects': 3201}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:03:16.327 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T16:03:17.001 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:03:17.001 INFO:teuthology.orchestra.run.smithi115.stdout:{"fsid":"7effee29-df05-4351-9741-ee203564ed81","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 42911/298674 objects degraded (14.367%), 10 pgs degraded, 11 pgs undersized","count":21},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":361,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983954,"num_in_osds":3,"osd_in_since":1725983947,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":149337,"data_bytes":611635977,"bytes_used":2266689536,"bytes_avail":287643602944,"bytes_total":289910292480,"degraded_objects":42911,"degraded_total":298674,"degraded_ratio":0.1436716955610465,"misplaced_objects":3039,"misplaced_total":298674,"misplaced_ratio":0.010174973382349987,"recovering_objects_per_sec":15,"recovering_bytes_per_sec":64827,"recovering_keys_per_sec":0,"num_objects_recovered":190,"num_bytes_recovered":778240,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":423},"fsmap":{"epoch":1,"btime":"2024-09-10T15:57:14:967120+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T16:00:13.560776+0000","services":{}},"progress_events":{}} 2024-09-10T16:03:17.023 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-10T16:03:17.023 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:03:17.590 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:03:17.591 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:03:17.615 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":218,"stamp":"2024-09-10T16:03:15.634410+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":611635977,"num_objects":149337,"num_object_clones":0,"num_object_copies":298674,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42911,"num_objects_misplaced":3039,"num_objects_unfound":0,"num_objects_dirty":149337,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":142773,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5438,"num_bytes_recovered":22261779,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160700,"ondisk_log_size":160700,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2213564,"kb_used_data":1263372,"kb_used_omap":12,"kb_used_meta":950131,"kb_avail":280901956,"statfs":{"total":289910292480,"available":287643602944,"internally_reserved":0,"allocated":1293692928,"data_stored":1292768791,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":972934146},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3,4,5,4,11,9,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-20824064,"num_objects":-5084,"num_object_clones":0,"num_object_copies":-10168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-190,"num_objects_unfound":0,"num_objects_dirty":-5084,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5084,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":190,"num_bytes_recovered":778240,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004829"},"pg_stats":[{"pgid":"2.f","version":"45'11422","reported_seq":11497,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:13.109496+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:03:13.109496+0000","last_peered":"2024-09-10T16:03:13.109496+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:03:13.109496+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"36'1400","ondisk_log_start":"36'1400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37888000,"num_objects":9250,"num_object_clones":0,"num_object_copies":18500,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9250,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11422,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4991},{"shards":"0,2","objects":3172}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'11570","reported_seq":11683,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:03:14.944818+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:03:14.944818+0000","last_peered":"2024-09-10T16:03:14.944818+0000","last_clean":"2024-09-10T16:03:14.944818+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:03:14.944818+0000","last_undegraded":"2024-09-10T16:03:14.944818+0000","last_fullsized":"2024-09-10T16:03:14.944818+0000","mapping_epoch":35,"log_start":"36'1500","ondisk_log_start":"36'1500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37987270,"num_objects":9276,"num_object_clones":0,"num_object_copies":18552,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9276,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":9675,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'11447","reported_seq":20125,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:03:10.765079+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:03:10.765079+0000","last_peered":"2024-09-10T16:03:10.765079+0000","last_clean":"2024-09-10T16:03:10.765079+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:03:10.765079+0000","last_undegraded":"2024-09-10T16:03:10.765079+0000","last_fullsized":"2024-09-10T16:03:10.765079+0000","mapping_epoch":44,"log_start":"36'1400","ondisk_log_start":"36'1400","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38297624,"num_objects":9351,"num_object_clones":0,"num_object_copies":18702,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9351,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9451,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'11516","reported_seq":11597,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:03:14.821737+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:03:14.821737+0000","last_peered":"2024-09-10T16:03:14.821737+0000","last_clean":"2024-09-10T16:03:14.821737+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:03:14.821737+0000","last_undegraded":"2024-09-10T16:03:14.821737+0000","last_fullsized":"2024-09-10T16:03:14.821737+0000","mapping_epoch":35,"log_start":"36'1500","ondisk_log_start":"36'1500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37453824,"num_objects":9144,"num_object_clones":0,"num_object_copies":18288,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9144,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11516,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'11519","reported_seq":11649,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:03:13.152219+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:03:13.152219+0000","last_peered":"2024-09-10T16:03:13.152219+0000","last_clean":"2024-09-10T16:03:13.152219+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:03:13.152219+0000","last_undegraded":"2024-09-10T16:03:13.152219+0000","last_fullsized":"2024-09-10T16:03:13.152219+0000","mapping_epoch":34,"log_start":"36'1500","ondisk_log_start":"36'1500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38388195,"num_objects":9373,"num_object_clones":0,"num_object_copies":18746,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9373,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8230,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'11653","reported_seq":11791,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:13.069962+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:03:13.069962+0000","last_peered":"2024-09-10T16:03:13.069962+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:03:13.069962+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"36'1600","ondisk_log_start":"36'1600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38535651,"num_objects":9409,"num_object_clones":0,"num_object_copies":18818,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9409,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8265,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5038},{"shards":"0,2","objects":3251}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'11628","reported_seq":11766,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:13.162850+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:03:13.162850+0000","last_peered":"2024-09-10T16:03:13.162850+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:03:13.162850+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"36'1600","ondisk_log_start":"36'1600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38724550,"num_objects":9456,"num_object_clones":0,"num_object_copies":18912,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9456,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":8127,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5219},{"shards":"0,2","objects":3153}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'11755","reported_seq":11885,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:03:13.170028+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:03:13.170028+0000","last_peered":"2024-09-10T16:03:13.170028+0000","last_clean":"2024-09-10T16:03:13.170028+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:03:13.170028+0000","last_undegraded":"2024-09-10T16:03:13.170028+0000","last_fullsized":"2024-09-10T16:03:13.170028+0000","mapping_epoch":34,"log_start":"36'1700","ondisk_log_start":"36'1700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39153664,"num_objects":9559,"num_object_clones":0,"num_object_copies":19118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9559,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8602,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104219+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:02:08.104219+0000","last_peered":"2024-09-10T16:02:08.104219+0000","last_clean":"2024-09-10T16:02:08.104219+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:02:08.104219+0000","last_undegraded":"2024-09-10T16:02:08.104219+0000","last_fullsized":"2024-09-10T16:02:08.104219+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'11362","reported_seq":11502,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:14.943929+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:03:14.943929+0000","last_peered":"2024-09-10T16:03:14.943929+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:03:14.943929+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"36'1300","ondisk_log_start":"36'1300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37253603,"num_objects":9096,"num_object_clones":0,"num_object_copies":18192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9096,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9181,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2993},{"shards":"2","objects":4973}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103774+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:02:08.103774+0000","last_peered":"2024-09-10T16:02:08.103774+0000","last_clean":"2024-09-10T16:02:08.103774+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:02:08.103774+0000","last_undegraded":"2024-09-10T16:02:08.103774+0000","last_fullsized":"2024-09-10T16:02:08.103774+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'11546","reported_seq":11672,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:13.204114+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:03:13.204114+0000","last_peered":"2024-09-10T16:03:13.204114+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:03:13.204114+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"36'1500","ondisk_log_start":"36'1500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38354944,"num_objects":9364,"num_object_clones":0,"num_object_copies":18728,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9364,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8050,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5109},{"shards":"0,2","objects":3168}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":104,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.105055+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:02:08.105055+0000","last_peered":"2024-09-10T16:02:08.105055+0000","last_clean":"2024-09-10T16:02:08.105055+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:02:08.105055+0000","last_undegraded":"2024-09-10T16:02:08.105055+0000","last_fullsized":"2024-09-10T16:02:08.105055+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'11548","reported_seq":11683,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:14.959232+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:03:14.959232+0000","last_peered":"2024-09-10T16:03:14.959232+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:03:14.959232+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"36'1500","ondisk_log_start":"36'1500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37818851,"num_objects":9234,"num_object_clones":0,"num_object_copies":18468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9234,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9228,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3149},{"shards":"2","objects":4930}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104572+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:02:08.104572+0000","last_peered":"2024-09-10T16:02:08.104572+0000","last_clean":"2024-09-10T16:02:08.104572+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:02:08.104572+0000","last_undegraded":"2024-09-10T16:02:08.104572+0000","last_fullsized":"2024-09-10T16:02:08.104572+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'11756","reported_seq":11885,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:14.896557+0000","last_change":"2024-09-10T15:59:16.174113+0000","last_active":"2024-09-10T16:03:14.896557+0000","last_peered":"2024-09-10T16:03:14.896557+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:03:14.896557+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"36'1700","ondisk_log_start":"36'1700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38559744,"num_objects":9414,"num_object_clones":0,"num_object_copies":18828,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4354,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9414,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9310,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3184},{"shards":"2","objects":5062}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'11544","reported_seq":11679,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:13.107152+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:03:13.107152+0000","last_peered":"2024-09-10T16:03:13.107152+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:03:13.107152+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"36'1500","ondisk_log_start":"36'1500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38068224,"num_objects":9294,"num_object_clones":0,"num_object_copies":18588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9294,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8175,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4990},{"shards":"0,2","objects":3180}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103276+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:02:08.103276+0000","last_peered":"2024-09-10T16:02:08.103276+0000","last_clean":"2024-09-10T16:02:08.103276+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:02:08.103276+0000","last_undegraded":"2024-09-10T16:02:08.103276+0000","last_fullsized":"2024-09-10T16:02:08.103276+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'11335","reported_seq":13807,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T16:03:13.172267+0000","last_change":"2024-09-10T16:02:06.480033+0000","last_active":"2024-09-10T16:03:13.172267+0000","last_peered":"2024-09-10T16:03:13.172267+0000","last_clean":"2024-09-10T15:59:14.115595+0000","last_became_active":"2024-09-10T15:59:16.244089+0000","last_became_peered":"2024-09-10T15:59:16.244089+0000","last_unstale":"2024-09-10T16:03:13.172267+0000","last_undegraded":"2024-09-10T16:03:13.172267+0000","last_fullsized":"2024-09-10T15:59:15.774437+0000","mapping_epoch":42,"log_start":"36'1300","ondisk_log_start":"36'1300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37941248,"num_objects":9263,"num_object_clones":0,"num_object_copies":18526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3039,"num_objects_unfound":0,"num_objects_dirty":9263,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7725,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1164,"num_bytes_recovered":4767744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6224},{"shards":"0,2","objects":3039}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103831+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:02:08.103831+0000","last_peered":"2024-09-10T16:02:08.103831+0000","last_clean":"2024-09-10T16:02:08.103831+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:02:08.103831+0000","last_undegraded":"2024-09-10T16:02:08.103831+0000","last_fullsized":"2024-09-10T16:02:08.103831+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'11669","reported_seq":11811,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:13.191300+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:03:13.191300+0000","last_peered":"2024-09-10T16:03:13.191300+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:03:13.191300+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"36'1600","ondisk_log_start":"36'1600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38416384,"num_objects":9379,"num_object_clones":0,"num_object_copies":18758,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9379,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7924,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5008},{"shards":"0,2","objects":3228}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104431+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:02:08.104431+0000","last_peered":"2024-09-10T16:02:08.104431+0000","last_clean":"2024-09-10T16:02:08.104431+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:02:08.104431+0000","last_undegraded":"2024-09-10T16:02:08.104431+0000","last_fullsized":"2024-09-10T16:02:08.104431+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104410+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:02:08.104410+0000","last_peered":"2024-09-10T16:02:08.104410+0000","last_clean":"2024-09-10T16:02:08.104410+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:02:08.104410+0000","last_undegraded":"2024-09-10T16:02:08.104410+0000","last_fullsized":"2024-09-10T16:02:08.104410+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'11727","reported_seq":11871,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:13.136013+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:03:13.136013+0000","last_peered":"2024-09-10T16:03:13.136013+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:03:13.136013+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"36'1700","ondisk_log_start":"36'1700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38794182,"num_objects":9473,"num_object_clones":0,"num_object_copies":18946,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9473,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7890,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5150},{"shards":"0,2","objects":3201}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":611635958,"num_objects":149335,"num_object_clones":0,"num_object_copies":298670,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42911,"num_objects_misplaced":3039,"num_objects_unfound":0,"num_objects_dirty":149335,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":142771,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5436,"num_bytes_recovered":22261760,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1291157504,"data_stored":1290997614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160697,"ondisk_log_size":160697,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986239,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":745376,"kb_used_data":434640,"kb_used_omap":4,"kb_used_meta":310715,"kb_avail":93626464,"statfs":{"total":96636764160,"available":95873499136,"internally_reserved":0,"allocated":445071360,"data_stored":444779127,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":318172842},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,1,4],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659184,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":566140,"kb_used_data":309548,"kb_used_omap":4,"kb_used_meta":256571,"kb_avail":93805700,"statfs":{"total":96636764160,"available":96057036800,"internally_reserved":0,"allocated":316977152,"data_stored":316637486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":262729390},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,3,1,6,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182467,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":902048,"kb_used_data":519184,"kb_used_omap":4,"kb_used_meta":382843,"kb_avail":93469792,"statfs":{"total":96636764160,"available":95713067008,"internally_reserved":0,"allocated":531644416,"data_stored":531352178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":392031914},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3,1,1,2,1,5,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":529924096,"data_stored":529887966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":316555264,"data_stored":316471706,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":444678144,"data_stored":444637942,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:03:20.618 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:03:21.169 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:03:21.172 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:03:21.195 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":220,"stamp":"2024-09-10T16:03:19.636047+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":605496073,"num_objects":147838,"num_object_clones":0,"num_object_copies":295676,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42911,"num_objects_misplaced":2946,"num_objects_unfound":0,"num_objects_dirty":147838,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":144272,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5531,"num_bytes_recovered":22642707,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160999,"ondisk_log_size":160999,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2185576,"kb_used_data":1256172,"kb_used_omap":12,"kb_used_meta":929331,"kb_avail":280929944,"statfs":{"total":289910292480,"available":287672262656,"internally_reserved":0,"allocated":1286320128,"data_stored":1285395991,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":951634946},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,6,2,7,11,5,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-23519232,"num_objects":-5742,"num_object_clones":0,"num_object_copies":-11484,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-283,"num_objects_unfound":0,"num_objects_dirty":-5742,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5742,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":283,"num_bytes_recovered":1159168,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004670"},"pg_stats":[{"pgid":"2.f","version":"45'11561","reported_seq":11633,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:18.169030+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:03:18.169030+0000","last_peered":"2024-09-10T16:03:18.169030+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:03:18.169030+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"36'1500","ondisk_log_start":"36'1500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37318656,"num_objects":9111,"num_object_clones":0,"num_object_copies":18222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9111,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11561,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4852},{"shards":"0,2","objects":3033}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'11570","reported_seq":11683,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:03:14.944818+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:03:14.944818+0000","last_peered":"2024-09-10T16:03:14.944818+0000","last_clean":"2024-09-10T16:03:14.944818+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:03:14.944818+0000","last_undegraded":"2024-09-10T16:03:14.944818+0000","last_fullsized":"2024-09-10T16:03:14.944818+0000","mapping_epoch":35,"log_start":"36'1500","ondisk_log_start":"36'1500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37987270,"num_objects":9276,"num_object_clones":0,"num_object_copies":18552,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9276,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":9675,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'11576","reported_seq":20254,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:03:15.777642+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:03:15.777642+0000","last_peered":"2024-09-10T16:03:15.777642+0000","last_clean":"2024-09-10T16:03:15.777642+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:03:15.777642+0000","last_undegraded":"2024-09-10T16:03:15.777642+0000","last_fullsized":"2024-09-10T16:03:15.777642+0000","mapping_epoch":44,"log_start":"36'1500","ondisk_log_start":"36'1500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37769240,"num_objects":9222,"num_object_clones":0,"num_object_copies":18444,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9222,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9580,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'11516","reported_seq":11597,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:03:14.821737+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:03:14.821737+0000","last_peered":"2024-09-10T16:03:14.821737+0000","last_clean":"2024-09-10T16:03:14.821737+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:03:14.821737+0000","last_undegraded":"2024-09-10T16:03:14.821737+0000","last_fullsized":"2024-09-10T16:03:14.821737+0000","mapping_epoch":35,"log_start":"36'1500","ondisk_log_start":"36'1500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37453824,"num_objects":9144,"num_object_clones":0,"num_object_copies":18288,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9144,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11516,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'11668","reported_seq":11798,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:03:18.173525+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:03:18.173525+0000","last_peered":"2024-09-10T16:03:18.173525+0000","last_clean":"2024-09-10T16:03:18.173525+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:03:18.173525+0000","last_undegraded":"2024-09-10T16:03:18.173525+0000","last_fullsized":"2024-09-10T16:03:18.173525+0000","mapping_epoch":34,"log_start":"36'1600","ondisk_log_start":"36'1600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37777891,"num_objects":9224,"num_object_clones":0,"num_object_copies":18448,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9224,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8379,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'11799","reported_seq":11937,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:18.070161+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:03:18.070161+0000","last_peered":"2024-09-10T16:03:18.070161+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:03:18.070161+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"36'1700","ondisk_log_start":"36'1700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37937635,"num_objects":9263,"num_object_clones":0,"num_object_copies":18526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9263,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8411,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4892},{"shards":"0,2","objects":3105}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'11763","reported_seq":11901,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:18.173911+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:03:18.173911+0000","last_peered":"2024-09-10T16:03:18.173911+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:03:18.173911+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"36'1700","ondisk_log_start":"36'1700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38171590,"num_objects":9321,"num_object_clones":0,"num_object_copies":18642,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9321,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":8262,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5084},{"shards":"0,2","objects":3018}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'11884","reported_seq":12014,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:03:18.188632+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:03:18.188632+0000","last_peered":"2024-09-10T16:03:18.188632+0000","last_clean":"2024-09-10T16:03:18.188632+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:03:18.188632+0000","last_undegraded":"2024-09-10T16:03:18.188632+0000","last_fullsized":"2024-09-10T16:03:18.188632+0000","mapping_epoch":34,"log_start":"36'1800","ondisk_log_start":"36'1800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38625280,"num_objects":9430,"num_object_clones":0,"num_object_copies":18860,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9430,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8731,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104261+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:02:08.104261+0000","last_peered":"2024-09-10T16:02:08.104261+0000","last_clean":"2024-09-10T16:02:08.104261+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:02:08.104261+0000","last_undegraded":"2024-09-10T16:02:08.104261+0000","last_fullsized":"2024-09-10T16:02:08.104261+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'11362","reported_seq":11502,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:14.943929+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:03:14.943929+0000","last_peered":"2024-09-10T16:03:14.943929+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:03:14.943929+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"36'1300","ondisk_log_start":"36'1300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37253603,"num_objects":9096,"num_object_clones":0,"num_object_copies":18192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9096,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9181,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2993},{"shards":"2","objects":4973}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103774+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:02:08.103774+0000","last_peered":"2024-09-10T16:02:08.103774+0000","last_clean":"2024-09-10T16:02:08.103774+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:02:08.103774+0000","last_undegraded":"2024-09-10T16:02:08.103774+0000","last_fullsized":"2024-09-10T16:02:08.103774+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'11685","reported_seq":11816,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:18.146114+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:03:18.146114+0000","last_peered":"2024-09-10T16:03:18.146114+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:03:18.146114+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"36'1600","ondisk_log_start":"36'1600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37785600,"num_objects":9225,"num_object_clones":0,"num_object_copies":18450,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9225,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8189,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4970},{"shards":"0,2","objects":3029}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":104,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.105055+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:02:08.105055+0000","last_peered":"2024-09-10T16:02:08.105055+0000","last_clean":"2024-09-10T16:02:08.105055+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:02:08.105055+0000","last_undegraded":"2024-09-10T16:02:08.105055+0000","last_fullsized":"2024-09-10T16:02:08.105055+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'11548","reported_seq":11683,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:14.959232+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:03:14.959232+0000","last_peered":"2024-09-10T16:03:14.959232+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:03:14.959232+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"36'1500","ondisk_log_start":"36'1500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37818851,"num_objects":9234,"num_object_clones":0,"num_object_copies":18468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9234,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9228,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3149},{"shards":"2","objects":4930}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104572+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:02:08.104572+0000","last_peered":"2024-09-10T16:02:08.104572+0000","last_clean":"2024-09-10T16:02:08.104572+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:02:08.104572+0000","last_undegraded":"2024-09-10T16:02:08.104572+0000","last_fullsized":"2024-09-10T16:02:08.104572+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'11756","reported_seq":11885,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:14.896557+0000","last_change":"2024-09-10T15:59:16.174113+0000","last_active":"2024-09-10T16:03:14.896557+0000","last_peered":"2024-09-10T16:03:14.896557+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:03:14.896557+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"36'1700","ondisk_log_start":"36'1700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38559744,"num_objects":9414,"num_object_clones":0,"num_object_copies":18828,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4354,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9414,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9310,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3184},{"shards":"2","objects":5062}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'11653","reported_seq":11788,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:18.040604+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:03:18.040604+0000","last_peered":"2024-09-10T16:03:18.040604+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:03:18.040604+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"36'1600","ondisk_log_start":"36'1600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37621760,"num_objects":9185,"num_object_clones":0,"num_object_copies":18370,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9185,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8284,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4881},{"shards":"0,2","objects":3071}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103276+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:02:08.103276+0000","last_peered":"2024-09-10T16:02:08.103276+0000","last_clean":"2024-09-10T16:02:08.103276+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:02:08.103276+0000","last_undegraded":"2024-09-10T16:02:08.103276+0000","last_fullsized":"2024-09-10T16:02:08.103276+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'11459","reported_seq":14117,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T16:03:18.237454+0000","last_change":"2024-09-10T16:02:06.480033+0000","last_active":"2024-09-10T16:03:18.237454+0000","last_peered":"2024-09-10T16:03:18.237454+0000","last_clean":"2024-09-10T15:59:14.115595+0000","last_became_active":"2024-09-10T15:59:16.244089+0000","last_became_peered":"2024-09-10T15:59:16.244089+0000","last_unstale":"2024-09-10T16:03:18.237454+0000","last_undegraded":"2024-09-10T16:03:18.237454+0000","last_fullsized":"2024-09-10T15:59:15.774437+0000","mapping_epoch":42,"log_start":"36'1400","ondisk_log_start":"36'1400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37433344,"num_objects":9139,"num_object_clones":0,"num_object_copies":18278,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2946,"num_objects_unfound":0,"num_objects_dirty":9139,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7849,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1257,"num_bytes_recovered":5148672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6193},{"shards":"0,2","objects":2946}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103831+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:02:08.103831+0000","last_peered":"2024-09-10T16:02:08.103831+0000","last_clean":"2024-09-10T16:02:08.103831+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:02:08.103831+0000","last_undegraded":"2024-09-10T16:02:08.103831+0000","last_fullsized":"2024-09-10T16:02:08.103831+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'11833","reported_seq":11975,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:18.183860+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:03:18.183860+0000","last_peered":"2024-09-10T16:03:18.183860+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:03:18.183860+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"36'1800","ondisk_log_start":"36'1800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37744640,"num_objects":9215,"num_object_clones":0,"num_object_copies":18430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9215,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8088,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4844},{"shards":"0,2","objects":3064}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104431+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:02:08.104431+0000","last_peered":"2024-09-10T16:02:08.104431+0000","last_clean":"2024-09-10T16:02:08.104431+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:02:08.104431+0000","last_undegraded":"2024-09-10T16:02:08.104431+0000","last_fullsized":"2024-09-10T16:02:08.104431+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104410+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:02:08.104410+0000","last_peered":"2024-09-10T16:02:08.104410+0000","last_clean":"2024-09-10T16:02:08.104410+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:02:08.104410+0000","last_undegraded":"2024-09-10T16:02:08.104410+0000","last_fullsized":"2024-09-10T16:02:08.104410+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'11863","reported_seq":12008,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:18.202459+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:03:18.202459+0000","last_peered":"2024-09-10T16:03:18.202459+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:03:18.202459+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"36'1800","ondisk_log_start":"36'1800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38237126,"num_objects":9337,"num_object_clones":0,"num_object_copies":18674,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9337,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8026,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5014},{"shards":"0,2","objects":3065}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":605496054,"num_objects":147836,"num_object_clones":0,"num_object_copies":295672,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42911,"num_objects_misplaced":2946,"num_objects_unfound":0,"num_objects_dirty":147836,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":144270,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5529,"num_bytes_recovered":22642688,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1283874816,"data_stored":1283714926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160996,"ondisk_log_size":160996,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986239,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":745376,"kb_used_data":434640,"kb_used_omap":4,"kb_used_meta":310715,"kb_avail":93626464,"statfs":{"total":96636764160,"available":95873499136,"internally_reserved":0,"allocated":445071360,"data_stored":444779127,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":318172842},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,1,4],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659185,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":553084,"kb_used_data":308972,"kb_used_omap":4,"kb_used_meta":244091,"kb_avail":93818756,"statfs":{"total":96636764160,"available":96070406144,"internally_reserved":0,"allocated":316387328,"data_stored":316047662,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":249949870},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,2,5,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182468,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":887116,"kb_used_data":512560,"kb_used_omap":4,"kb_used_meta":374523,"kb_avail":93484724,"statfs":{"total":96636764160,"available":95728357376,"internally_reserved":0,"allocated":524861440,"data_stored":524569202,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":383512234},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,1,4,2,1,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":523218944,"data_stored":523182814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":315977728,"data_stored":315894170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":444678144,"data_stored":444637942,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:03:21.196 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:03:21.754 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:03:21.757 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:03:21.788 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":221,"stamp":"2024-09-10T16:03:21.636734+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":602043145,"num_objects":146995,"num_object_clones":0,"num_object_copies":293990,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42911,"num_objects_misplaced":2946,"num_objects_unfound":0,"num_objects_dirty":146995,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":145115,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5531,"num_bytes_recovered":22642707,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160942,"ondisk_log_size":160942,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2172156,"kb_used_data":1251072,"kb_used_omap":12,"kb_used_meta":921011,"kb_avail":280943364,"statfs":{"total":289910292480,"available":287686004736,"internally_reserved":0,"allocated":1281097728,"data_stored":1280173591,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":943115266},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,4,4,13,10,5,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-21434368,"num_objects":-5233,"num_object_clones":0,"num_object_copies":-10466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-188,"num_objects_unfound":0,"num_objects_dirty":-5233,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5233,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":188,"num_bytes_recovered":770048,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004672"},"pg_stats":[{"pgid":"2.f","version":"45'11561","reported_seq":11633,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:18.169030+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:03:18.169030+0000","last_peered":"2024-09-10T16:03:18.169030+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:03:18.169030+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"36'1500","ondisk_log_start":"36'1500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37318656,"num_objects":9111,"num_object_clones":0,"num_object_copies":18222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9111,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11561,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4852},{"shards":"0,2","objects":3033}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'11721","reported_seq":11833,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:03:19.967977+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:03:19.967977+0000","last_peered":"2024-09-10T16:03:19.967977+0000","last_clean":"2024-09-10T16:03:19.967977+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:03:19.967977+0000","last_undegraded":"2024-09-10T16:03:19.967977+0000","last_fullsized":"2024-09-10T16:03:19.967977+0000","mapping_epoch":35,"log_start":"36'1700","ondisk_log_start":"36'1700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37368774,"num_objects":9125,"num_object_clones":0,"num_object_copies":18250,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9125,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":9826,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'11708","reported_seq":20386,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:03:20.765772+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:03:20.765772+0000","last_peered":"2024-09-10T16:03:20.765772+0000","last_clean":"2024-09-10T16:03:20.765772+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:03:20.765772+0000","last_undegraded":"2024-09-10T16:03:20.765772+0000","last_fullsized":"2024-09-10T16:03:20.765772+0000","mapping_epoch":44,"log_start":"36'1700","ondisk_log_start":"36'1700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37228568,"num_objects":9090,"num_object_clones":0,"num_object_copies":18180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9090,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9712,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'11646","reported_seq":11727,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:03:19.876688+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:03:19.876688+0000","last_peered":"2024-09-10T16:03:19.876688+0000","last_clean":"2024-09-10T16:03:19.876688+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:03:19.876688+0000","last_undegraded":"2024-09-10T16:03:19.876688+0000","last_fullsized":"2024-09-10T16:03:19.876688+0000","mapping_epoch":35,"log_start":"36'1600","ondisk_log_start":"36'1600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36921344,"num_objects":9014,"num_object_clones":0,"num_object_copies":18028,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9014,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11646,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'11668","reported_seq":11798,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:03:18.173525+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:03:18.173525+0000","last_peered":"2024-09-10T16:03:18.173525+0000","last_clean":"2024-09-10T16:03:18.173525+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:03:18.173525+0000","last_undegraded":"2024-09-10T16:03:18.173525+0000","last_fullsized":"2024-09-10T16:03:18.173525+0000","mapping_epoch":34,"log_start":"36'1600","ondisk_log_start":"36'1600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37777891,"num_objects":9224,"num_object_clones":0,"num_object_copies":18448,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9224,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8379,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'11799","reported_seq":11937,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:18.070161+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:03:18.070161+0000","last_peered":"2024-09-10T16:03:18.070161+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:03:18.070161+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"36'1700","ondisk_log_start":"36'1700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37937635,"num_objects":9263,"num_object_clones":0,"num_object_copies":18526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9263,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8411,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4892},{"shards":"0,2","objects":3105}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'11763","reported_seq":11901,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:18.173911+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:03:18.173911+0000","last_peered":"2024-09-10T16:03:18.173911+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:03:18.173911+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"36'1700","ondisk_log_start":"36'1700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38171590,"num_objects":9321,"num_object_clones":0,"num_object_copies":18642,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9321,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":8262,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5084},{"shards":"0,2","objects":3018}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'11884","reported_seq":12014,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:03:18.188632+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:03:18.188632+0000","last_peered":"2024-09-10T16:03:18.188632+0000","last_clean":"2024-09-10T16:03:18.188632+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:03:18.188632+0000","last_undegraded":"2024-09-10T16:03:18.188632+0000","last_fullsized":"2024-09-10T16:03:18.188632+0000","mapping_epoch":34,"log_start":"36'1800","ondisk_log_start":"36'1800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38625280,"num_objects":9430,"num_object_clones":0,"num_object_copies":18860,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9430,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8731,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104261+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:02:08.104261+0000","last_peered":"2024-09-10T16:02:08.104261+0000","last_clean":"2024-09-10T16:02:08.104261+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:02:08.104261+0000","last_undegraded":"2024-09-10T16:02:08.104261+0000","last_fullsized":"2024-09-10T16:02:08.104261+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'11518","reported_seq":11658,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:19.937438+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:03:19.937438+0000","last_peered":"2024-09-10T16:03:19.937438+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:03:19.937438+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"36'1500","ondisk_log_start":"36'1500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36614627,"num_objects":8940,"num_object_clones":0,"num_object_copies":17880,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8940,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9337,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2837},{"shards":"2","objects":4817}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103774+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:02:08.103774+0000","last_peered":"2024-09-10T16:02:08.103774+0000","last_clean":"2024-09-10T16:02:08.103774+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:02:08.103774+0000","last_undegraded":"2024-09-10T16:02:08.103774+0000","last_fullsized":"2024-09-10T16:02:08.103774+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'11685","reported_seq":11816,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:18.146114+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:03:18.146114+0000","last_peered":"2024-09-10T16:03:18.146114+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:03:18.146114+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"36'1600","ondisk_log_start":"36'1600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37785600,"num_objects":9225,"num_object_clones":0,"num_object_copies":18450,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9225,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8189,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4970},{"shards":"0,2","objects":3029}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":104,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.105055+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:02:08.105055+0000","last_peered":"2024-09-10T16:02:08.105055+0000","last_clean":"2024-09-10T16:02:08.105055+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:02:08.105055+0000","last_undegraded":"2024-09-10T16:02:08.105055+0000","last_fullsized":"2024-09-10T16:02:08.105055+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'11679","reported_seq":11812,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:20.000669+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:03:20.000669+0000","last_peered":"2024-09-10T16:03:20.000669+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:03:20.000669+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"36'1600","ondisk_log_start":"36'1600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37282275,"num_objects":9103,"num_object_clones":0,"num_object_copies":18206,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9103,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9359,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3018},{"shards":"2","objects":4799}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104572+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:02:08.104572+0000","last_peered":"2024-09-10T16:02:08.104572+0000","last_clean":"2024-09-10T16:02:08.104572+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:02:08.104572+0000","last_undegraded":"2024-09-10T16:02:08.104572+0000","last_fullsized":"2024-09-10T16:02:08.104572+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'11899","reported_seq":12028,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:19.971959+0000","last_change":"2024-09-10T15:59:16.174113+0000","last_active":"2024-09-10T16:03:19.971959+0000","last_peered":"2024-09-10T16:03:19.971959+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:03:19.971959+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"36'1800","ondisk_log_start":"36'1800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37974016,"num_objects":9271,"num_object_clones":0,"num_object_copies":18542,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4354,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9271,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9453,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3041},{"shards":"2","objects":4919}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'11653","reported_seq":11788,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:18.040604+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:03:18.040604+0000","last_peered":"2024-09-10T16:03:18.040604+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:03:18.040604+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"36'1600","ondisk_log_start":"36'1600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37621760,"num_objects":9185,"num_object_clones":0,"num_object_copies":18370,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9185,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8284,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4881},{"shards":"0,2","objects":3071}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103276+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:02:08.103276+0000","last_peered":"2024-09-10T16:02:08.103276+0000","last_clean":"2024-09-10T16:02:08.103276+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:02:08.103276+0000","last_undegraded":"2024-09-10T16:02:08.103276+0000","last_fullsized":"2024-09-10T16:02:08.103276+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'11459","reported_seq":14117,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T16:03:18.237454+0000","last_change":"2024-09-10T16:02:06.480033+0000","last_active":"2024-09-10T16:03:18.237454+0000","last_peered":"2024-09-10T16:03:18.237454+0000","last_clean":"2024-09-10T15:59:14.115595+0000","last_became_active":"2024-09-10T15:59:16.244089+0000","last_became_peered":"2024-09-10T15:59:16.244089+0000","last_unstale":"2024-09-10T16:03:18.237454+0000","last_undegraded":"2024-09-10T16:03:18.237454+0000","last_fullsized":"2024-09-10T15:59:15.774437+0000","mapping_epoch":42,"log_start":"36'1400","ondisk_log_start":"36'1400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37433344,"num_objects":9139,"num_object_clones":0,"num_object_copies":18278,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2946,"num_objects_unfound":0,"num_objects_dirty":9139,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7849,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1257,"num_bytes_recovered":5148672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6193},{"shards":"0,2","objects":2946}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103831+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:02:08.103831+0000","last_peered":"2024-09-10T16:02:08.103831+0000","last_clean":"2024-09-10T16:02:08.103831+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:02:08.103831+0000","last_undegraded":"2024-09-10T16:02:08.103831+0000","last_fullsized":"2024-09-10T16:02:08.103831+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'11833","reported_seq":11975,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:18.183860+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:03:18.183860+0000","last_peered":"2024-09-10T16:03:18.183860+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:03:18.183860+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"36'1800","ondisk_log_start":"36'1800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37744640,"num_objects":9215,"num_object_clones":0,"num_object_copies":18430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9215,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8088,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4844},{"shards":"0,2","objects":3064}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104431+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:02:08.104431+0000","last_peered":"2024-09-10T16:02:08.104431+0000","last_clean":"2024-09-10T16:02:08.104431+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:02:08.104431+0000","last_undegraded":"2024-09-10T16:02:08.104431+0000","last_fullsized":"2024-09-10T16:02:08.104431+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104410+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:02:08.104410+0000","last_peered":"2024-09-10T16:02:08.104410+0000","last_clean":"2024-09-10T16:02:08.104410+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:02:08.104410+0000","last_undegraded":"2024-09-10T16:02:08.104410+0000","last_fullsized":"2024-09-10T16:02:08.104410+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'11863","reported_seq":12008,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:18.202459+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:03:18.202459+0000","last_peered":"2024-09-10T16:03:18.202459+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:03:18.202459+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"36'1800","ondisk_log_start":"36'1800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38237126,"num_objects":9337,"num_object_clones":0,"num_object_copies":18674,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9337,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8026,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5014},{"shards":"0,2","objects":3065}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":602043126,"num_objects":146993,"num_object_clones":0,"num_object_copies":293986,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42911,"num_objects_misplaced":2946,"num_objects_unfound":0,"num_objects_dirty":146993,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":145113,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5529,"num_bytes_recovered":22642688,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1278529536,"data_stored":1278369646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160939,"ondisk_log_size":160939,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986240,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":728436,"kb_used_data":430180,"kb_used_omap":4,"kb_used_meta":298235,"kb_avail":93643404,"statfs":{"total":96636764160,"available":95890845696,"internally_reserved":0,"allocated":440504320,"data_stored":440212087,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":305393322},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,4,5,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659186,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":556604,"kb_used_data":308332,"kb_used_omap":4,"kb_used_meta":248251,"kb_avail":93815236,"statfs":{"total":96636764160,"available":96066801664,"internally_reserved":0,"allocated":315731968,"data_stored":315392302,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":254209710},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,5,3,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182468,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":887116,"kb_used_data":512560,"kb_used_omap":4,"kb_used_meta":374523,"kb_avail":93484724,"statfs":{"total":96636764160,"available":95728357376,"internally_reserved":0,"allocated":524861440,"data_stored":524569202,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":383512234},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,1,4,2,1,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":523218944,"data_stored":523182814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":315289600,"data_stored":315206042,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":440020992,"data_stored":439980790,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:03:21.792 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-10T16:03:21.792 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "45'11561", 'reported_seq': 11633, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:03:18.169030+0000', 'last_change': '2024-09-10T15:59:16.281558+0000', 'last_active': '2024-09-10T16:03:18.169030+0000', 'last_peered': '2024-09-10T16:03:18.169030+0000', 'last_clean': '2024-09-10T15:59:14.132456+0000', 'last_became_active': '2024-09-10T15:59:16.274407+0000', 'last_became_peered': '2024-09-10T15:59:16.274407+0000', 'last_unstale': '2024-09-10T16:03:18.169030+0000', 'last_undegraded': '2024-09-10T15:59:15.851279+0000', 'last_fullsized': '2024-09-10T15:59:15.774356+0000', 'mapping_epoch': 42, 'log_start': "36'1500", 'ondisk_log_start': "36'1500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'objects_scrubbed': 0, 'log_size': 10061, 'log_dups_size': 0, 'ondisk_log_size': 10061, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37318656, 'num_objects': 9111, 'num_object_clones': 0, 'num_object_copies': 18222, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4259, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9111, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11561, 'num_write_kb': 41344, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4852}, {'shards': '0,2', 'objects': 3033}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:03:21.793 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-10T16:03:21.793 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "45'11799", 'reported_seq': 11937, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:03:18.070161+0000', 'last_change': '2024-09-10T15:59:16.308264+0000', 'last_active': '2024-09-10T16:03:18.070161+0000', 'last_peered': '2024-09-10T16:03:18.070161+0000', 'last_clean': '2024-09-10T15:59:14.111551+0000', 'last_became_active': '2024-09-10T15:59:16.300040+0000', 'last_became_peered': '2024-09-10T15:59:16.300040+0000', 'last_unstale': '2024-09-10T16:03:18.070161+0000', 'last_undegraded': '2024-09-10T15:59:15.878234+0000', 'last_fullsized': '2024-09-10T15:59:15.769328+0000', 'mapping_epoch': 42, 'log_start': "36'1700", 'ondisk_log_start': "36'1700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3442", 'last_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'objects_scrubbed': 3384, 'log_size': 10099, 'log_dups_size': 0, 'ondisk_log_size': 10099, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1292, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37937635, 'num_objects': 9263, 'num_object_clones': 0, 'num_object_copies': 18526, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4372, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9263, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 8411, 'num_write_kb': 28580, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4892}, {'shards': '0,2', 'objects': 3105}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:03:21.793 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-10T16:03:21.793 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "45'11763", 'reported_seq': 11901, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:03:18.173911+0000', 'last_change': '2024-09-10T15:59:16.331899+0000', 'last_active': '2024-09-10T16:03:18.173911+0000', 'last_peered': '2024-09-10T16:03:18.173911+0000', 'last_clean': '2024-09-10T15:59:14.139436+0000', 'last_became_active': '2024-09-10T15:59:16.318715+0000', 'last_became_peered': '2024-09-10T15:59:16.318715+0000', 'last_unstale': '2024-09-10T16:03:18.173911+0000', 'last_undegraded': '2024-09-10T15:59:15.877524+0000', 'last_fullsized': '2024-09-10T15:59:15.769042+0000', 'mapping_epoch': 42, 'log_start': "36'1700", 'ondisk_log_start': "36'1700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3537", 'last_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'objects_scrubbed': 3497, 'log_size': 10063, 'log_dups_size': 0, 'ondisk_log_size': 10063, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1224, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38171590, 'num_objects': 9321, 'num_object_clones': 0, 'num_object_copies': 18642, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4237, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9321, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 8262, 'num_write_kb': 28172, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5084}, {'shards': '0,2', 'objects': 3018}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:03:21.793 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-10T16:03:21.793 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "45'11518", 'reported_seq': 11658, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:03:19.937438+0000', 'last_change': '2024-09-10T15:59:16.175680+0000', 'last_active': '2024-09-10T16:03:19.937438+0000', 'last_peered': '2024-09-10T16:03:19.937438+0000', 'last_clean': '2024-09-10T15:59:14.088187+0000', 'last_became_active': '2024-09-10T15:59:16.165810+0000', 'last_became_peered': '2024-09-10T15:59:16.165810+0000', 'last_unstale': '2024-09-10T16:03:19.937438+0000', 'last_undegraded': '2024-09-10T15:59:15.822098+0000', 'last_fullsized': '2024-09-10T15:59:15.769123+0000', 'mapping_epoch': 42, 'log_start': "36'1500", 'ondisk_log_start': "36'1500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2218", 'last_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'objects_scrubbed': 2175, 'log_size': 10018, 'log_dups_size': 0, 'ondisk_log_size': 10018, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 825, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36614627, 'num_objects': 8940, 'num_object_clones': 0, 'num_object_copies': 17880, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4124, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8940, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 9337, 'num_write_kb': 32204, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2837}, {'shards': '2', 'objects': 4817}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:03:21.793 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-10T16:03:21.793 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "45'11685", 'reported_seq': 11816, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:03:18.146114+0000', 'last_change': '2024-09-10T15:59:16.277742+0000', 'last_active': '2024-09-10T16:03:18.146114+0000', 'last_peered': '2024-09-10T16:03:18.146114+0000', 'last_clean': '2024-09-10T15:59:14.129081+0000', 'last_became_active': '2024-09-10T15:59:16.269835+0000', 'last_became_peered': '2024-09-10T15:59:16.269835+0000', 'last_unstale': '2024-09-10T16:03:18.146114+0000', 'last_undegraded': '2024-09-10T15:59:15.860857+0000', 'last_fullsized': '2024-09-10T15:59:15.771410+0000', 'mapping_epoch': 42, 'log_start': "36'1600", 'ondisk_log_start': "36'1600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3557", 'last_scrub_stamp': '2024-09-10T15:59:02.493678+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:02.493678+0000', 'objects_scrubbed': 3488, 'log_size': 10085, 'log_dups_size': 0, 'ondisk_log_size': 10085, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1472, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37785600, 'num_objects': 9225, 'num_object_clones': 0, 'num_object_copies': 18450, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4257, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9225, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8189, 'num_write_kb': 27852, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4970}, {'shards': '0,2', 'objects': 3029}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:03:21.793 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-10T16:03:21.794 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "45'11679", 'reported_seq': 11812, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:03:20.000669+0000', 'last_change': '2024-09-10T15:59:16.284311+0000', 'last_active': '2024-09-10T16:03:20.000669+0000', 'last_peered': '2024-09-10T16:03:20.000669+0000', 'last_clean': '2024-09-10T15:59:14.116041+0000', 'last_became_active': '2024-09-10T15:59:16.274543+0000', 'last_became_peered': '2024-09-10T15:59:16.274543+0000', 'last_unstale': '2024-09-10T16:03:20.000669+0000', 'last_undegraded': '2024-09-10T15:59:15.812484+0000', 'last_fullsized': '2024-09-10T15:59:15.770885+0000', 'mapping_epoch': 42, 'log_start': "36'1600", 'ondisk_log_start': "36'1600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2356", 'last_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'objects_scrubbed': 2316, 'log_size': 10079, 'log_dups_size': 0, 'ondisk_log_size': 10079, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 783, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37282275, 'num_objects': 9103, 'num_object_clones': 0, 'num_object_copies': 18206, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9103, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 9359, 'num_write_kb': 32292, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3018}, {'shards': '2', 'objects': 4799}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:03:21.794 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-10T16:03:21.794 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "45'11899", 'reported_seq': 12028, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:03:19.971959+0000', 'last_change': '2024-09-10T15:59:16.174113+0000', 'last_active': '2024-09-10T16:03:19.971959+0000', 'last_peered': '2024-09-10T16:03:19.971959+0000', 'last_clean': '2024-09-10T15:59:14.110654+0000', 'last_became_active': '2024-09-10T15:59:16.164645+0000', 'last_became_peered': '2024-09-10T15:59:16.164645+0000', 'last_unstale': '2024-09-10T16:03:19.971959+0000', 'last_undegraded': '2024-09-10T15:59:15.803593+0000', 'last_fullsized': '2024-09-10T15:59:15.769083+0000', 'mapping_epoch': 42, 'log_start': "36'1800", 'ondisk_log_start': "36'1800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2486", 'last_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'objects_scrubbed': 2440, 'log_size': 10099, 'log_dups_size': 0, 'ondisk_log_size': 10099, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 792, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37974016, 'num_objects': 9271, 'num_object_clones': 0, 'num_object_copies': 18542, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4354, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9271, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9453, 'num_write_kb': 32568, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3041}, {'shards': '2', 'objects': 4919}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:03:21.794 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-10T16:03:21.794 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "45'11653", 'reported_seq': 11788, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:03:18.040604+0000', 'last_change': '2024-09-10T15:59:16.315641+0000', 'last_active': '2024-09-10T16:03:18.040604+0000', 'last_peered': '2024-09-10T16:03:18.040604+0000', 'last_clean': '2024-09-10T15:59:14.117001+0000', 'last_became_active': '2024-09-10T15:59:16.302245+0000', 'last_became_peered': '2024-09-10T15:59:16.302245+0000', 'last_unstale': '2024-09-10T16:03:18.040604+0000', 'last_undegraded': '2024-09-10T15:59:15.845012+0000', 'last_fullsized': '2024-09-10T15:59:15.770337+0000', 'mapping_epoch': 42, 'log_start': "36'1600", 'ondisk_log_start': "36'1600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3404", 'last_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'objects_scrubbed': 3367, 'log_size': 10053, 'log_dups_size': 0, 'ondisk_log_size': 10053, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 983, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37621760, 'num_objects': 9185, 'num_object_clones': 0, 'num_object_copies': 18370, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9185, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8284, 'num_write_kb': 28204, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4881}, {'shards': '0,2', 'objects': 3071}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:03:21.794 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-10T16:03:21.794 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "45'11459", 'reported_seq': 14117, 'reported_epoch': 45, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-10T16:03:18.237454+0000', 'last_change': '2024-09-10T16:02:06.480033+0000', 'last_active': '2024-09-10T16:03:18.237454+0000', 'last_peered': '2024-09-10T16:03:18.237454+0000', 'last_clean': '2024-09-10T15:59:14.115595+0000', 'last_became_active': '2024-09-10T15:59:16.244089+0000', 'last_became_peered': '2024-09-10T15:59:16.244089+0000', 'last_unstale': '2024-09-10T16:03:18.237454+0000', 'last_undegraded': '2024-09-10T16:03:18.237454+0000', 'last_fullsized': '2024-09-10T15:59:15.774437+0000', 'mapping_epoch': 42, 'log_start': "36'1400", 'ondisk_log_start': "36'1400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3653", 'last_scrub_stamp': '2024-09-10T15:59:05.298468+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:05.298468+0000', 'objects_scrubbed': 3602, 'log_size': 10059, 'log_dups_size': 0, 'ondisk_log_size': 10059, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1289, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37433344, 'num_objects': 9139, 'num_object_clones': 0, 'num_object_copies': 18278, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 2946, 'num_objects_unfound': 0, 'num_objects_dirty': 9139, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7849, 'num_write_kb': 26772, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1257, 'num_bytes_recovered': 5148672, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6193}, {'shards': '0,2', 'objects': 2946}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:03:21.794 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-10T16:03:21.794 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "45'11833", 'reported_seq': 11975, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:03:18.183860+0000', 'last_change': '2024-09-10T15:59:16.311291+0000', 'last_active': '2024-09-10T16:03:18.183860+0000', 'last_peered': '2024-09-10T16:03:18.183860+0000', 'last_clean': '2024-09-10T15:59:14.116363+0000', 'last_became_active': '2024-09-10T15:59:16.301244+0000', 'last_became_peered': '2024-09-10T15:59:16.301244+0000', 'last_unstale': '2024-09-10T16:03:18.183860+0000', 'last_undegraded': '2024-09-10T15:59:15.883434+0000', 'last_fullsized': '2024-09-10T15:59:15.772756+0000', 'mapping_epoch': 42, 'log_start': "36'1800", 'ondisk_log_start': "36'1800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3789", 'last_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'objects_scrubbed': 3741, 'log_size': 10033, 'log_dups_size': 0, 'ondisk_log_size': 10033, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1440, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37744640, 'num_objects': 9215, 'num_object_clones': 0, 'num_object_copies': 18430, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4373, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9215, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8088, 'num_write_kb': 27124, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4844}, {'shards': '0,2', 'objects': 3064}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:03:21.794 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-10T16:03:21.794 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "45'11863", 'reported_seq': 12008, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:03:18.202459+0000', 'last_change': '2024-09-10T15:59:16.313409+0000', 'last_active': '2024-09-10T16:03:18.202459+0000', 'last_peered': '2024-09-10T16:03:18.202459+0000', 'last_clean': '2024-09-10T15:59:14.129780+0000', 'last_became_active': '2024-09-10T15:59:16.300932+0000', 'last_became_peered': '2024-09-10T15:59:16.300932+0000', 'last_unstale': '2024-09-10T16:03:18.202459+0000', 'last_undegraded': '2024-09-10T15:59:15.887823+0000', 'last_fullsized': '2024-09-10T15:59:15.768513+0000', 'mapping_epoch': 42, 'log_start': "36'1800", 'ondisk_log_start': "36'1800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3880", 'last_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'objects_scrubbed': 3827, 'log_size': 10063, 'log_dups_size': 0, 'ondisk_log_size': 10063, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1397, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38237126, 'num_objects': 9337, 'num_object_clones': 0, 'num_object_copies': 18674, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4325, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9337, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8026, 'num_write_kb': 27072, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5014}, {'shards': '0,2', 'objects': 3065}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:03:21.795 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T16:03:22.496 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:03:22.496 INFO:teuthology.orchestra.run.smithi115.stdout:{"fsid":"7effee29-df05-4351-9741-ee203564ed81","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 42911/293990 objects degraded (14.596%), 10 pgs degraded, 11 pgs undersized","count":21},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":366,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983954,"num_in_osds":3,"osd_in_since":1725983947,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":146995,"data_bytes":602043145,"bytes_used":2224287744,"bytes_avail":287686004736,"bytes_total":289910292480,"degraded_objects":42911,"degraded_total":293990,"degraded_ratio":0.14596074696418246,"misplaced_objects":2946,"misplaced_total":293990,"misplaced_ratio":0.010020749005068198,"recovering_objects_per_sec":15,"recovering_bytes_per_sec":64145,"recovering_keys_per_sec":0,"num_objects_recovered":188,"num_bytes_recovered":770048,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":435},"fsmap":{"epoch":1,"btime":"2024-09-10T15:57:14:967120+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T16:00:13.560776+0000","services":{}},"progress_events":{}} 2024-09-10T16:03:22.518 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-10T16:03:22.518 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:03:23.082 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:03:23.085 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:03:23.106 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":221,"stamp":"2024-09-10T16:03:21.636734+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":602043145,"num_objects":146995,"num_object_clones":0,"num_object_copies":293990,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42911,"num_objects_misplaced":2946,"num_objects_unfound":0,"num_objects_dirty":146995,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":145115,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5531,"num_bytes_recovered":22642707,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160942,"ondisk_log_size":160942,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2172156,"kb_used_data":1251072,"kb_used_omap":12,"kb_used_meta":921011,"kb_avail":280943364,"statfs":{"total":289910292480,"available":287686004736,"internally_reserved":0,"allocated":1281097728,"data_stored":1280173591,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":943115266},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,4,4,13,10,5,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-21434368,"num_objects":-5233,"num_object_clones":0,"num_object_copies":-10466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-188,"num_objects_unfound":0,"num_objects_dirty":-5233,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5233,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":188,"num_bytes_recovered":770048,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004672"},"pg_stats":[{"pgid":"2.f","version":"45'11561","reported_seq":11633,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:18.169030+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:03:18.169030+0000","last_peered":"2024-09-10T16:03:18.169030+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:03:18.169030+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"36'1500","ondisk_log_start":"36'1500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37318656,"num_objects":9111,"num_object_clones":0,"num_object_copies":18222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9111,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11561,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4852},{"shards":"0,2","objects":3033}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'11721","reported_seq":11833,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:03:19.967977+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:03:19.967977+0000","last_peered":"2024-09-10T16:03:19.967977+0000","last_clean":"2024-09-10T16:03:19.967977+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:03:19.967977+0000","last_undegraded":"2024-09-10T16:03:19.967977+0000","last_fullsized":"2024-09-10T16:03:19.967977+0000","mapping_epoch":35,"log_start":"36'1700","ondisk_log_start":"36'1700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37368774,"num_objects":9125,"num_object_clones":0,"num_object_copies":18250,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9125,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":9826,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'11708","reported_seq":20386,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:03:20.765772+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:03:20.765772+0000","last_peered":"2024-09-10T16:03:20.765772+0000","last_clean":"2024-09-10T16:03:20.765772+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:03:20.765772+0000","last_undegraded":"2024-09-10T16:03:20.765772+0000","last_fullsized":"2024-09-10T16:03:20.765772+0000","mapping_epoch":44,"log_start":"36'1700","ondisk_log_start":"36'1700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37228568,"num_objects":9090,"num_object_clones":0,"num_object_copies":18180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9090,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9712,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'11646","reported_seq":11727,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:03:19.876688+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:03:19.876688+0000","last_peered":"2024-09-10T16:03:19.876688+0000","last_clean":"2024-09-10T16:03:19.876688+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:03:19.876688+0000","last_undegraded":"2024-09-10T16:03:19.876688+0000","last_fullsized":"2024-09-10T16:03:19.876688+0000","mapping_epoch":35,"log_start":"36'1600","ondisk_log_start":"36'1600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36921344,"num_objects":9014,"num_object_clones":0,"num_object_copies":18028,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9014,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11646,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'11668","reported_seq":11798,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:03:18.173525+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:03:18.173525+0000","last_peered":"2024-09-10T16:03:18.173525+0000","last_clean":"2024-09-10T16:03:18.173525+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:03:18.173525+0000","last_undegraded":"2024-09-10T16:03:18.173525+0000","last_fullsized":"2024-09-10T16:03:18.173525+0000","mapping_epoch":34,"log_start":"36'1600","ondisk_log_start":"36'1600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37777891,"num_objects":9224,"num_object_clones":0,"num_object_copies":18448,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9224,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8379,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'11799","reported_seq":11937,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:18.070161+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:03:18.070161+0000","last_peered":"2024-09-10T16:03:18.070161+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:03:18.070161+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"36'1700","ondisk_log_start":"36'1700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37937635,"num_objects":9263,"num_object_clones":0,"num_object_copies":18526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9263,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8411,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4892},{"shards":"0,2","objects":3105}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'11763","reported_seq":11901,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:18.173911+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:03:18.173911+0000","last_peered":"2024-09-10T16:03:18.173911+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:03:18.173911+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"36'1700","ondisk_log_start":"36'1700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38171590,"num_objects":9321,"num_object_clones":0,"num_object_copies":18642,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9321,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":8262,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5084},{"shards":"0,2","objects":3018}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'11884","reported_seq":12014,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:03:18.188632+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:03:18.188632+0000","last_peered":"2024-09-10T16:03:18.188632+0000","last_clean":"2024-09-10T16:03:18.188632+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:03:18.188632+0000","last_undegraded":"2024-09-10T16:03:18.188632+0000","last_fullsized":"2024-09-10T16:03:18.188632+0000","mapping_epoch":34,"log_start":"36'1800","ondisk_log_start":"36'1800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38625280,"num_objects":9430,"num_object_clones":0,"num_object_copies":18860,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9430,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8731,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104261+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:02:08.104261+0000","last_peered":"2024-09-10T16:02:08.104261+0000","last_clean":"2024-09-10T16:02:08.104261+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:02:08.104261+0000","last_undegraded":"2024-09-10T16:02:08.104261+0000","last_fullsized":"2024-09-10T16:02:08.104261+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'11518","reported_seq":11658,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:19.937438+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:03:19.937438+0000","last_peered":"2024-09-10T16:03:19.937438+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:03:19.937438+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"36'1500","ondisk_log_start":"36'1500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36614627,"num_objects":8940,"num_object_clones":0,"num_object_copies":17880,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8940,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9337,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2837},{"shards":"2","objects":4817}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103774+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:02:08.103774+0000","last_peered":"2024-09-10T16:02:08.103774+0000","last_clean":"2024-09-10T16:02:08.103774+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:02:08.103774+0000","last_undegraded":"2024-09-10T16:02:08.103774+0000","last_fullsized":"2024-09-10T16:02:08.103774+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'11685","reported_seq":11816,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:18.146114+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:03:18.146114+0000","last_peered":"2024-09-10T16:03:18.146114+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:03:18.146114+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"36'1600","ondisk_log_start":"36'1600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37785600,"num_objects":9225,"num_object_clones":0,"num_object_copies":18450,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9225,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8189,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4970},{"shards":"0,2","objects":3029}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":104,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.105055+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:02:08.105055+0000","last_peered":"2024-09-10T16:02:08.105055+0000","last_clean":"2024-09-10T16:02:08.105055+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:02:08.105055+0000","last_undegraded":"2024-09-10T16:02:08.105055+0000","last_fullsized":"2024-09-10T16:02:08.105055+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'11679","reported_seq":11812,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:20.000669+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:03:20.000669+0000","last_peered":"2024-09-10T16:03:20.000669+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:03:20.000669+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"36'1600","ondisk_log_start":"36'1600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37282275,"num_objects":9103,"num_object_clones":0,"num_object_copies":18206,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9103,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9359,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3018},{"shards":"2","objects":4799}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104572+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:02:08.104572+0000","last_peered":"2024-09-10T16:02:08.104572+0000","last_clean":"2024-09-10T16:02:08.104572+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:02:08.104572+0000","last_undegraded":"2024-09-10T16:02:08.104572+0000","last_fullsized":"2024-09-10T16:02:08.104572+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'11899","reported_seq":12028,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:19.971959+0000","last_change":"2024-09-10T15:59:16.174113+0000","last_active":"2024-09-10T16:03:19.971959+0000","last_peered":"2024-09-10T16:03:19.971959+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:03:19.971959+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"36'1800","ondisk_log_start":"36'1800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37974016,"num_objects":9271,"num_object_clones":0,"num_object_copies":18542,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4354,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9271,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9453,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3041},{"shards":"2","objects":4919}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'11653","reported_seq":11788,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:18.040604+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:03:18.040604+0000","last_peered":"2024-09-10T16:03:18.040604+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:03:18.040604+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"36'1600","ondisk_log_start":"36'1600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37621760,"num_objects":9185,"num_object_clones":0,"num_object_copies":18370,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9185,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8284,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4881},{"shards":"0,2","objects":3071}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103276+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:02:08.103276+0000","last_peered":"2024-09-10T16:02:08.103276+0000","last_clean":"2024-09-10T16:02:08.103276+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:02:08.103276+0000","last_undegraded":"2024-09-10T16:02:08.103276+0000","last_fullsized":"2024-09-10T16:02:08.103276+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'11459","reported_seq":14117,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T16:03:18.237454+0000","last_change":"2024-09-10T16:02:06.480033+0000","last_active":"2024-09-10T16:03:18.237454+0000","last_peered":"2024-09-10T16:03:18.237454+0000","last_clean":"2024-09-10T15:59:14.115595+0000","last_became_active":"2024-09-10T15:59:16.244089+0000","last_became_peered":"2024-09-10T15:59:16.244089+0000","last_unstale":"2024-09-10T16:03:18.237454+0000","last_undegraded":"2024-09-10T16:03:18.237454+0000","last_fullsized":"2024-09-10T15:59:15.774437+0000","mapping_epoch":42,"log_start":"36'1400","ondisk_log_start":"36'1400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37433344,"num_objects":9139,"num_object_clones":0,"num_object_copies":18278,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2946,"num_objects_unfound":0,"num_objects_dirty":9139,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7849,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1257,"num_bytes_recovered":5148672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6193},{"shards":"0,2","objects":2946}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103831+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:02:08.103831+0000","last_peered":"2024-09-10T16:02:08.103831+0000","last_clean":"2024-09-10T16:02:08.103831+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:02:08.103831+0000","last_undegraded":"2024-09-10T16:02:08.103831+0000","last_fullsized":"2024-09-10T16:02:08.103831+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'11833","reported_seq":11975,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:18.183860+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:03:18.183860+0000","last_peered":"2024-09-10T16:03:18.183860+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:03:18.183860+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"36'1800","ondisk_log_start":"36'1800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37744640,"num_objects":9215,"num_object_clones":0,"num_object_copies":18430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9215,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8088,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4844},{"shards":"0,2","objects":3064}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104431+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:02:08.104431+0000","last_peered":"2024-09-10T16:02:08.104431+0000","last_clean":"2024-09-10T16:02:08.104431+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:02:08.104431+0000","last_undegraded":"2024-09-10T16:02:08.104431+0000","last_fullsized":"2024-09-10T16:02:08.104431+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104410+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:02:08.104410+0000","last_peered":"2024-09-10T16:02:08.104410+0000","last_clean":"2024-09-10T16:02:08.104410+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:02:08.104410+0000","last_undegraded":"2024-09-10T16:02:08.104410+0000","last_fullsized":"2024-09-10T16:02:08.104410+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'11863","reported_seq":12008,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:18.202459+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:03:18.202459+0000","last_peered":"2024-09-10T16:03:18.202459+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:03:18.202459+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"36'1800","ondisk_log_start":"36'1800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38237126,"num_objects":9337,"num_object_clones":0,"num_object_copies":18674,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9337,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8026,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5014},{"shards":"0,2","objects":3065}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":602043126,"num_objects":146993,"num_object_clones":0,"num_object_copies":293986,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42911,"num_objects_misplaced":2946,"num_objects_unfound":0,"num_objects_dirty":146993,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":145113,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5529,"num_bytes_recovered":22642688,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1278529536,"data_stored":1278369646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160939,"ondisk_log_size":160939,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986240,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":728436,"kb_used_data":430180,"kb_used_omap":4,"kb_used_meta":298235,"kb_avail":93643404,"statfs":{"total":96636764160,"available":95890845696,"internally_reserved":0,"allocated":440504320,"data_stored":440212087,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":305393322},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,4,5,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659186,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":556604,"kb_used_data":308332,"kb_used_omap":4,"kb_used_meta":248251,"kb_avail":93815236,"statfs":{"total":96636764160,"available":96066801664,"internally_reserved":0,"allocated":315731968,"data_stored":315392302,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":254209710},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,5,3,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182468,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":887116,"kb_used_data":512560,"kb_used_omap":4,"kb_used_meta":374523,"kb_avail":93484724,"statfs":{"total":96636764160,"available":95728357376,"internally_reserved":0,"allocated":524861440,"data_stored":524569202,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":383512234},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,1,4,2,1,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":523218944,"data_stored":523182814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":315289600,"data_stored":315206042,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":440020992,"data_stored":439980790,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:03:26.112 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:03:26.716 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:03:26.717 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:03:26.744 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":223,"stamp":"2024-09-10T16:03:25.638133+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":593605385,"num_objects":144935,"num_object_clones":0,"num_object_copies":289870,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42911,"num_objects_misplaced":2859,"num_objects_unfound":0,"num_objects_dirty":144935,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":147175,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5618,"num_bytes_recovered":22999059,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161002,"ondisk_log_size":161002,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2169448,"kb_used_data":1240056,"kb_used_omap":12,"kb_used_meta":929331,"kb_avail":280946072,"statfs":{"total":289910292480,"available":287688777728,"internally_reserved":0,"allocated":1269817344,"data_stored":1268893207,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":951634946},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,1,3,2,8,7,11,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-20770816,"num_objects":-5071,"num_object_clones":0,"num_object_copies":-10142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-180,"num_objects_unfound":0,"num_objects_dirty":-5071,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5071,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":180,"num_bytes_recovered":737280,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004657"},"pg_stats":[{"pgid":"2.f","version":"45'11696","reported_seq":11771,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:23.193808+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:03:23.193808+0000","last_peered":"2024-09-10T16:03:23.193808+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:03:23.193808+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"36'1600","ondisk_log_start":"36'1600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36765696,"num_objects":8976,"num_object_clones":0,"num_object_copies":17952,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8976,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11696,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4717},{"shards":"0,2","objects":2898}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'11859","reported_seq":11972,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.977729+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:03:24.977729+0000","last_peered":"2024-09-10T16:03:24.977729+0000","last_clean":"2024-09-10T16:03:24.977729+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:03:24.977729+0000","last_undegraded":"2024-09-10T16:03:24.977729+0000","last_fullsized":"2024-09-10T16:03:24.977729+0000","mapping_epoch":35,"log_start":"36'1800","ondisk_log_start":"36'1800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36803526,"num_objects":8987,"num_object_clones":0,"num_object_copies":17974,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8987,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":9964,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'11708","reported_seq":20386,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:03:20.765772+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:03:20.765772+0000","last_peered":"2024-09-10T16:03:20.765772+0000","last_clean":"2024-09-10T16:03:20.765772+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:03:20.765772+0000","last_undegraded":"2024-09-10T16:03:20.765772+0000","last_fullsized":"2024-09-10T16:03:20.765772+0000","mapping_epoch":44,"log_start":"36'1700","ondisk_log_start":"36'1700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37228568,"num_objects":9090,"num_object_clones":0,"num_object_copies":18180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9090,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9712,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'11793","reported_seq":11874,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.981634+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:03:24.981634+0000","last_peered":"2024-09-10T16:03:24.981634+0000","last_clean":"2024-09-10T16:03:24.981634+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:03:24.981634+0000","last_undegraded":"2024-09-10T16:03:24.981634+0000","last_fullsized":"2024-09-10T16:03:24.981634+0000","mapping_epoch":35,"log_start":"36'1700","ondisk_log_start":"36'1700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36319232,"num_objects":8867,"num_object_clones":0,"num_object_copies":17734,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8867,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11793,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'11790","reported_seq":11920,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:03:23.224421+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:03:23.224421+0000","last_peered":"2024-09-10T16:03:23.224421+0000","last_clean":"2024-09-10T16:03:23.224421+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:03:23.224421+0000","last_undegraded":"2024-09-10T16:03:23.224421+0000","last_fullsized":"2024-09-10T16:03:23.224421+0000","mapping_epoch":34,"log_start":"36'1700","ondisk_log_start":"36'1700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37278179,"num_objects":9102,"num_object_clones":0,"num_object_copies":18204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9102,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8501,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'11932","reported_seq":12070,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:23.244816+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:03:23.244816+0000","last_peered":"2024-09-10T16:03:23.244816+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:03:23.244816+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"36'1900","ondisk_log_start":"36'1900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37392867,"num_objects":9130,"num_object_clones":0,"num_object_copies":18260,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9130,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8544,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4759},{"shards":"0,2","objects":2972}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'11892","reported_seq":12029,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:23.233186+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:03:23.233186+0000","last_peered":"2024-09-10T16:03:23.233186+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:03:23.233186+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"36'1800","ondisk_log_start":"36'1800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37643206,"num_objects":9192,"num_object_clones":0,"num_object_copies":18384,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9192,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":8391,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4955},{"shards":"0,2","objects":2889}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'12052","reported_seq":12182,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:03:23.160958+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:03:23.160958+0000","last_peered":"2024-09-10T16:03:23.160958+0000","last_clean":"2024-09-10T16:03:23.160958+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:03:23.160958+0000","last_undegraded":"2024-09-10T16:03:23.160958+0000","last_fullsized":"2024-09-10T16:03:23.160958+0000","mapping_epoch":34,"log_start":"36'2000","ondisk_log_start":"36'2000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37937152,"num_objects":9262,"num_object_clones":0,"num_object_copies":18524,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9262,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8899,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104261+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:02:08.104261+0000","last_peered":"2024-09-10T16:02:08.104261+0000","last_clean":"2024-09-10T16:02:08.104261+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:02:08.104261+0000","last_undegraded":"2024-09-10T16:02:08.104261+0000","last_fullsized":"2024-09-10T16:02:08.104261+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'11635","reported_seq":11775,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:24.943005+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:03:24.943005+0000","last_peered":"2024-09-10T16:03:24.943005+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:03:24.943005+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"36'1600","ondisk_log_start":"36'1600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36135395,"num_objects":8823,"num_object_clones":0,"num_object_copies":17646,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8823,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9454,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2720},{"shards":"2","objects":4700}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103774+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:02:08.103774+0000","last_peered":"2024-09-10T16:02:08.103774+0000","last_clean":"2024-09-10T16:02:08.103774+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:02:08.103774+0000","last_undegraded":"2024-09-10T16:02:08.103774+0000","last_fullsized":"2024-09-10T16:02:08.103774+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'11834","reported_seq":11965,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:23.199031+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:03:23.199031+0000","last_peered":"2024-09-10T16:03:23.199031+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:03:23.199031+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"36'1800","ondisk_log_start":"36'1800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37175296,"num_objects":9076,"num_object_clones":0,"num_object_copies":18152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9076,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8338,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4821},{"shards":"0,2","objects":2880}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":104,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.105055+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:02:08.105055+0000","last_peered":"2024-09-10T16:02:08.105055+0000","last_clean":"2024-09-10T16:02:08.105055+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:02:08.105055+0000","last_undegraded":"2024-09-10T16:02:08.105055+0000","last_fullsized":"2024-09-10T16:02:08.105055+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'11828","reported_seq":11963,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:24.884810+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:03:24.884810+0000","last_peered":"2024-09-10T16:03:24.884810+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:03:24.884810+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"36'1800","ondisk_log_start":"36'1800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36671971,"num_objects":8954,"num_object_clones":0,"num_object_copies":17908,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8954,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9508,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2869},{"shards":"2","objects":4650}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104572+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:02:08.104572+0000","last_peered":"2024-09-10T16:02:08.104572+0000","last_clean":"2024-09-10T16:02:08.104572+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:02:08.104572+0000","last_undegraded":"2024-09-10T16:02:08.104572+0000","last_fullsized":"2024-09-10T16:02:08.104572+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'12037","reported_seq":12166,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:24.969571+0000","last_change":"2024-09-10T15:59:16.174113+0000","last_active":"2024-09-10T16:03:24.969571+0000","last_peered":"2024-09-10T16:03:24.969571+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:03:24.969571+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"36'2000","ondisk_log_start":"36'2000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37408768,"num_objects":9133,"num_object_clones":0,"num_object_copies":18266,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4354,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9133,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9591,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2903},{"shards":"2","objects":4781}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'11793","reported_seq":11928,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:23.196635+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:03:23.196635+0000","last_peered":"2024-09-10T16:03:23.196635+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:03:23.196635+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"36'1700","ondisk_log_start":"36'1700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37048320,"num_objects":9045,"num_object_clones":0,"num_object_copies":18090,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9045,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8424,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4741},{"shards":"0,2","objects":2931}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103276+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:02:08.103276+0000","last_peered":"2024-09-10T16:02:08.103276+0000","last_clean":"2024-09-10T16:02:08.103276+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:02:08.103276+0000","last_undegraded":"2024-09-10T16:02:08.103276+0000","last_fullsized":"2024-09-10T16:02:08.103276+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'11596","reported_seq":14427,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T16:03:23.270695+0000","last_change":"2024-09-10T16:02:06.480033+0000","last_active":"2024-09-10T16:03:23.270695+0000","last_peered":"2024-09-10T16:03:23.270695+0000","last_clean":"2024-09-10T15:59:14.115595+0000","last_became_active":"2024-09-10T15:59:16.244089+0000","last_became_peered":"2024-09-10T15:59:16.244089+0000","last_unstale":"2024-09-10T16:03:23.270695+0000","last_undegraded":"2024-09-10T16:03:23.270695+0000","last_fullsized":"2024-09-10T15:59:15.774437+0000","mapping_epoch":42,"log_start":"36'1500","ondisk_log_start":"36'1500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36872192,"num_objects":9002,"num_object_clones":0,"num_object_copies":18004,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2859,"num_objects_unfound":0,"num_objects_dirty":9002,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7986,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1344,"num_bytes_recovered":5505024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6143},{"shards":"0,2","objects":2859}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103831+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:02:08.103831+0000","last_peered":"2024-09-10T16:02:08.103831+0000","last_clean":"2024-09-10T16:02:08.103831+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:02:08.103831+0000","last_undegraded":"2024-09-10T16:02:08.103831+0000","last_fullsized":"2024-09-10T16:02:08.103831+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'11963","reported_seq":12105,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:23.154729+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:03:23.154729+0000","last_peered":"2024-09-10T16:03:23.154729+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:03:23.154729+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"36'1900","ondisk_log_start":"36'1900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37212160,"num_objects":9085,"num_object_clones":0,"num_object_copies":18170,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9085,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8218,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4714},{"shards":"0,2","objects":2934}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104431+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:02:08.104431+0000","last_peered":"2024-09-10T16:02:08.104431+0000","last_clean":"2024-09-10T16:02:08.104431+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:02:08.104431+0000","last_undegraded":"2024-09-10T16:02:08.104431+0000","last_fullsized":"2024-09-10T16:02:08.104431+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104410+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:02:08.104410+0000","last_peered":"2024-09-10T16:02:08.104410+0000","last_clean":"2024-09-10T16:02:08.104410+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:02:08.104410+0000","last_undegraded":"2024-09-10T16:02:08.104410+0000","last_fullsized":"2024-09-10T16:02:08.104410+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'11991","reported_seq":12136,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:23.259550+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:03:23.259550+0000","last_peered":"2024-09-10T16:03:23.259550+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:03:23.259550+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"36'1900","ondisk_log_start":"36'1900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37712838,"num_objects":9209,"num_object_clones":0,"num_object_copies":18418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9209,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8154,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4886},{"shards":"0,2","objects":2937}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":593605366,"num_objects":144933,"num_object_clones":0,"num_object_copies":289866,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42911,"num_objects_misplaced":2859,"num_objects_unfound":0,"num_objects_dirty":144933,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":147173,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5616,"num_bytes_recovered":22999040,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1267290112,"data_stored":1267130222,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160999,"ondisk_log_size":160999,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986241,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":728200,"kb_used_data":425784,"kb_used_omap":4,"kb_used_meta":302395,"kb_avail":93643640,"statfs":{"total":96636764160,"available":95891087360,"internally_reserved":0,"allocated":436002816,"data_stored":435710583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":309653162},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,1,0,0,4,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659186,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":556604,"kb_used_data":308332,"kb_used_omap":4,"kb_used_meta":248251,"kb_avail":93815236,"statfs":{"total":96636764160,"available":96066801664,"internally_reserved":0,"allocated":315731968,"data_stored":315392302,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":254209710},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,5,3,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182469,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":884644,"kb_used_data":505940,"kb_used_omap":4,"kb_used_meta":378683,"kb_avail":93487196,"statfs":{"total":96636764160,"available":95730888704,"internally_reserved":0,"allocated":518082560,"data_stored":517790322,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":387772074},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,3,0,3,4,5,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":516423680,"data_stored":516387550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":315289600,"data_stored":315206042,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":435576832,"data_stored":435536630,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:03:26.746 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:03:27.341 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:03:27.343 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:03:27.370 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":223,"stamp":"2024-09-10T16:03:25.638133+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":593605385,"num_objects":144935,"num_object_clones":0,"num_object_copies":289870,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42911,"num_objects_misplaced":2859,"num_objects_unfound":0,"num_objects_dirty":144935,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":147175,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5618,"num_bytes_recovered":22999059,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161002,"ondisk_log_size":161002,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2169448,"kb_used_data":1240056,"kb_used_omap":12,"kb_used_meta":929331,"kb_avail":280946072,"statfs":{"total":289910292480,"available":287688777728,"internally_reserved":0,"allocated":1269817344,"data_stored":1268893207,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":951634946},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,1,3,2,8,7,11,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-20770816,"num_objects":-5071,"num_object_clones":0,"num_object_copies":-10142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-180,"num_objects_unfound":0,"num_objects_dirty":-5071,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5071,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":180,"num_bytes_recovered":737280,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004657"},"pg_stats":[{"pgid":"2.f","version":"45'11696","reported_seq":11771,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:23.193808+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:03:23.193808+0000","last_peered":"2024-09-10T16:03:23.193808+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:03:23.193808+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"36'1600","ondisk_log_start":"36'1600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36765696,"num_objects":8976,"num_object_clones":0,"num_object_copies":17952,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8976,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11696,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4717},{"shards":"0,2","objects":2898}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'11859","reported_seq":11972,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.977729+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:03:24.977729+0000","last_peered":"2024-09-10T16:03:24.977729+0000","last_clean":"2024-09-10T16:03:24.977729+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:03:24.977729+0000","last_undegraded":"2024-09-10T16:03:24.977729+0000","last_fullsized":"2024-09-10T16:03:24.977729+0000","mapping_epoch":35,"log_start":"36'1800","ondisk_log_start":"36'1800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36803526,"num_objects":8987,"num_object_clones":0,"num_object_copies":17974,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8987,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":9964,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'11708","reported_seq":20386,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:03:20.765772+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:03:20.765772+0000","last_peered":"2024-09-10T16:03:20.765772+0000","last_clean":"2024-09-10T16:03:20.765772+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:03:20.765772+0000","last_undegraded":"2024-09-10T16:03:20.765772+0000","last_fullsized":"2024-09-10T16:03:20.765772+0000","mapping_epoch":44,"log_start":"36'1700","ondisk_log_start":"36'1700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37228568,"num_objects":9090,"num_object_clones":0,"num_object_copies":18180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9090,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9712,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'11793","reported_seq":11874,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.981634+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:03:24.981634+0000","last_peered":"2024-09-10T16:03:24.981634+0000","last_clean":"2024-09-10T16:03:24.981634+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:03:24.981634+0000","last_undegraded":"2024-09-10T16:03:24.981634+0000","last_fullsized":"2024-09-10T16:03:24.981634+0000","mapping_epoch":35,"log_start":"36'1700","ondisk_log_start":"36'1700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36319232,"num_objects":8867,"num_object_clones":0,"num_object_copies":17734,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8867,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11793,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'11790","reported_seq":11920,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:03:23.224421+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:03:23.224421+0000","last_peered":"2024-09-10T16:03:23.224421+0000","last_clean":"2024-09-10T16:03:23.224421+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:03:23.224421+0000","last_undegraded":"2024-09-10T16:03:23.224421+0000","last_fullsized":"2024-09-10T16:03:23.224421+0000","mapping_epoch":34,"log_start":"36'1700","ondisk_log_start":"36'1700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37278179,"num_objects":9102,"num_object_clones":0,"num_object_copies":18204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9102,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8501,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'11932","reported_seq":12070,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:23.244816+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:03:23.244816+0000","last_peered":"2024-09-10T16:03:23.244816+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:03:23.244816+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"36'1900","ondisk_log_start":"36'1900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37392867,"num_objects":9130,"num_object_clones":0,"num_object_copies":18260,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9130,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8544,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4759},{"shards":"0,2","objects":2972}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'11892","reported_seq":12029,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:23.233186+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:03:23.233186+0000","last_peered":"2024-09-10T16:03:23.233186+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:03:23.233186+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"36'1800","ondisk_log_start":"36'1800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37643206,"num_objects":9192,"num_object_clones":0,"num_object_copies":18384,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9192,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":8391,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4955},{"shards":"0,2","objects":2889}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'12052","reported_seq":12182,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:03:23.160958+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:03:23.160958+0000","last_peered":"2024-09-10T16:03:23.160958+0000","last_clean":"2024-09-10T16:03:23.160958+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:03:23.160958+0000","last_undegraded":"2024-09-10T16:03:23.160958+0000","last_fullsized":"2024-09-10T16:03:23.160958+0000","mapping_epoch":34,"log_start":"36'2000","ondisk_log_start":"36'2000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37937152,"num_objects":9262,"num_object_clones":0,"num_object_copies":18524,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9262,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8899,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104261+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:02:08.104261+0000","last_peered":"2024-09-10T16:02:08.104261+0000","last_clean":"2024-09-10T16:02:08.104261+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:02:08.104261+0000","last_undegraded":"2024-09-10T16:02:08.104261+0000","last_fullsized":"2024-09-10T16:02:08.104261+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'11635","reported_seq":11775,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:24.943005+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:03:24.943005+0000","last_peered":"2024-09-10T16:03:24.943005+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:03:24.943005+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"36'1600","ondisk_log_start":"36'1600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36135395,"num_objects":8823,"num_object_clones":0,"num_object_copies":17646,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8823,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9454,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2720},{"shards":"2","objects":4700}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103774+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:02:08.103774+0000","last_peered":"2024-09-10T16:02:08.103774+0000","last_clean":"2024-09-10T16:02:08.103774+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:02:08.103774+0000","last_undegraded":"2024-09-10T16:02:08.103774+0000","last_fullsized":"2024-09-10T16:02:08.103774+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'11834","reported_seq":11965,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:23.199031+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:03:23.199031+0000","last_peered":"2024-09-10T16:03:23.199031+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:03:23.199031+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"36'1800","ondisk_log_start":"36'1800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37175296,"num_objects":9076,"num_object_clones":0,"num_object_copies":18152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9076,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8338,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4821},{"shards":"0,2","objects":2880}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":104,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.105055+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:02:08.105055+0000","last_peered":"2024-09-10T16:02:08.105055+0000","last_clean":"2024-09-10T16:02:08.105055+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:02:08.105055+0000","last_undegraded":"2024-09-10T16:02:08.105055+0000","last_fullsized":"2024-09-10T16:02:08.105055+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'11828","reported_seq":11963,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:24.884810+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:03:24.884810+0000","last_peered":"2024-09-10T16:03:24.884810+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:03:24.884810+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"36'1800","ondisk_log_start":"36'1800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36671971,"num_objects":8954,"num_object_clones":0,"num_object_copies":17908,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8954,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9508,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2869},{"shards":"2","objects":4650}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104572+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:02:08.104572+0000","last_peered":"2024-09-10T16:02:08.104572+0000","last_clean":"2024-09-10T16:02:08.104572+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:02:08.104572+0000","last_undegraded":"2024-09-10T16:02:08.104572+0000","last_fullsized":"2024-09-10T16:02:08.104572+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'12037","reported_seq":12166,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:24.969571+0000","last_change":"2024-09-10T15:59:16.174113+0000","last_active":"2024-09-10T16:03:24.969571+0000","last_peered":"2024-09-10T16:03:24.969571+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:03:24.969571+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"36'2000","ondisk_log_start":"36'2000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37408768,"num_objects":9133,"num_object_clones":0,"num_object_copies":18266,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4354,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9133,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9591,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2903},{"shards":"2","objects":4781}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'11793","reported_seq":11928,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:23.196635+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:03:23.196635+0000","last_peered":"2024-09-10T16:03:23.196635+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:03:23.196635+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"36'1700","ondisk_log_start":"36'1700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37048320,"num_objects":9045,"num_object_clones":0,"num_object_copies":18090,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9045,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8424,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4741},{"shards":"0,2","objects":2931}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103276+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:02:08.103276+0000","last_peered":"2024-09-10T16:02:08.103276+0000","last_clean":"2024-09-10T16:02:08.103276+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:02:08.103276+0000","last_undegraded":"2024-09-10T16:02:08.103276+0000","last_fullsized":"2024-09-10T16:02:08.103276+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'11596","reported_seq":14427,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T16:03:23.270695+0000","last_change":"2024-09-10T16:02:06.480033+0000","last_active":"2024-09-10T16:03:23.270695+0000","last_peered":"2024-09-10T16:03:23.270695+0000","last_clean":"2024-09-10T15:59:14.115595+0000","last_became_active":"2024-09-10T15:59:16.244089+0000","last_became_peered":"2024-09-10T15:59:16.244089+0000","last_unstale":"2024-09-10T16:03:23.270695+0000","last_undegraded":"2024-09-10T16:03:23.270695+0000","last_fullsized":"2024-09-10T15:59:15.774437+0000","mapping_epoch":42,"log_start":"36'1500","ondisk_log_start":"36'1500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36872192,"num_objects":9002,"num_object_clones":0,"num_object_copies":18004,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2859,"num_objects_unfound":0,"num_objects_dirty":9002,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7986,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1344,"num_bytes_recovered":5505024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6143},{"shards":"0,2","objects":2859}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103831+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:02:08.103831+0000","last_peered":"2024-09-10T16:02:08.103831+0000","last_clean":"2024-09-10T16:02:08.103831+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:02:08.103831+0000","last_undegraded":"2024-09-10T16:02:08.103831+0000","last_fullsized":"2024-09-10T16:02:08.103831+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'11963","reported_seq":12105,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:23.154729+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:03:23.154729+0000","last_peered":"2024-09-10T16:03:23.154729+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:03:23.154729+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"36'1900","ondisk_log_start":"36'1900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37212160,"num_objects":9085,"num_object_clones":0,"num_object_copies":18170,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9085,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8218,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4714},{"shards":"0,2","objects":2934}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104431+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:02:08.104431+0000","last_peered":"2024-09-10T16:02:08.104431+0000","last_clean":"2024-09-10T16:02:08.104431+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:02:08.104431+0000","last_undegraded":"2024-09-10T16:02:08.104431+0000","last_fullsized":"2024-09-10T16:02:08.104431+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104410+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:02:08.104410+0000","last_peered":"2024-09-10T16:02:08.104410+0000","last_clean":"2024-09-10T16:02:08.104410+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:02:08.104410+0000","last_undegraded":"2024-09-10T16:02:08.104410+0000","last_fullsized":"2024-09-10T16:02:08.104410+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'11991","reported_seq":12136,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:23.259550+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:03:23.259550+0000","last_peered":"2024-09-10T16:03:23.259550+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:03:23.259550+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"36'1900","ondisk_log_start":"36'1900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37712838,"num_objects":9209,"num_object_clones":0,"num_object_copies":18418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9209,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8154,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4886},{"shards":"0,2","objects":2937}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":593605366,"num_objects":144933,"num_object_clones":0,"num_object_copies":289866,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42911,"num_objects_misplaced":2859,"num_objects_unfound":0,"num_objects_dirty":144933,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":147173,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5616,"num_bytes_recovered":22999040,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1267290112,"data_stored":1267130222,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160999,"ondisk_log_size":160999,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986241,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":728200,"kb_used_data":425784,"kb_used_omap":4,"kb_used_meta":302395,"kb_avail":93643640,"statfs":{"total":96636764160,"available":95891087360,"internally_reserved":0,"allocated":436002816,"data_stored":435710583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":309653162},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,1,0,0,4,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659186,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":556604,"kb_used_data":308332,"kb_used_omap":4,"kb_used_meta":248251,"kb_avail":93815236,"statfs":{"total":96636764160,"available":96066801664,"internally_reserved":0,"allocated":315731968,"data_stored":315392302,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":254209710},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,5,3,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182469,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":884644,"kb_used_data":505940,"kb_used_omap":4,"kb_used_meta":378683,"kb_avail":93487196,"statfs":{"total":96636764160,"available":95730888704,"internally_reserved":0,"allocated":518082560,"data_stored":517790322,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":387772074},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,3,0,3,4,5,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":516423680,"data_stored":516387550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":315289600,"data_stored":315206042,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":435576832,"data_stored":435536630,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:03:27.372 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-10T16:03:27.372 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "45'11696", 'reported_seq': 11771, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:03:23.193808+0000', 'last_change': '2024-09-10T15:59:16.281558+0000', 'last_active': '2024-09-10T16:03:23.193808+0000', 'last_peered': '2024-09-10T16:03:23.193808+0000', 'last_clean': '2024-09-10T15:59:14.132456+0000', 'last_became_active': '2024-09-10T15:59:16.274407+0000', 'last_became_peered': '2024-09-10T15:59:16.274407+0000', 'last_unstale': '2024-09-10T16:03:23.193808+0000', 'last_undegraded': '2024-09-10T15:59:15.851279+0000', 'last_fullsized': '2024-09-10T15:59:15.774356+0000', 'mapping_epoch': 42, 'log_start': "36'1600", 'ondisk_log_start': "36'1600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'objects_scrubbed': 0, 'log_size': 10096, 'log_dups_size': 0, 'ondisk_log_size': 10096, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36765696, 'num_objects': 8976, 'num_object_clones': 0, 'num_object_copies': 17952, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4259, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8976, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11696, 'num_write_kb': 41344, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4717}, {'shards': '0,2', 'objects': 2898}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:03:27.372 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-10T16:03:27.372 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "45'11932", 'reported_seq': 12070, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:03:23.244816+0000', 'last_change': '2024-09-10T15:59:16.308264+0000', 'last_active': '2024-09-10T16:03:23.244816+0000', 'last_peered': '2024-09-10T16:03:23.244816+0000', 'last_clean': '2024-09-10T15:59:14.111551+0000', 'last_became_active': '2024-09-10T15:59:16.300040+0000', 'last_became_peered': '2024-09-10T15:59:16.300040+0000', 'last_unstale': '2024-09-10T16:03:23.244816+0000', 'last_undegraded': '2024-09-10T15:59:15.878234+0000', 'last_fullsized': '2024-09-10T15:59:15.769328+0000', 'mapping_epoch': 42, 'log_start': "36'1900", 'ondisk_log_start': "36'1900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3442", 'last_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'objects_scrubbed': 3384, 'log_size': 10032, 'log_dups_size': 0, 'ondisk_log_size': 10032, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1292, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37392867, 'num_objects': 9130, 'num_object_clones': 0, 'num_object_copies': 18260, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4372, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9130, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 8544, 'num_write_kb': 28580, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4759}, {'shards': '0,2', 'objects': 2972}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:03:27.372 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-10T16:03:27.372 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "45'11892", 'reported_seq': 12029, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:03:23.233186+0000', 'last_change': '2024-09-10T15:59:16.331899+0000', 'last_active': '2024-09-10T16:03:23.233186+0000', 'last_peered': '2024-09-10T16:03:23.233186+0000', 'last_clean': '2024-09-10T15:59:14.139436+0000', 'last_became_active': '2024-09-10T15:59:16.318715+0000', 'last_became_peered': '2024-09-10T15:59:16.318715+0000', 'last_unstale': '2024-09-10T16:03:23.233186+0000', 'last_undegraded': '2024-09-10T15:59:15.877524+0000', 'last_fullsized': '2024-09-10T15:59:15.769042+0000', 'mapping_epoch': 42, 'log_start': "36'1800", 'ondisk_log_start': "36'1800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3537", 'last_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'objects_scrubbed': 3497, 'log_size': 10092, 'log_dups_size': 0, 'ondisk_log_size': 10092, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1224, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37643206, 'num_objects': 9192, 'num_object_clones': 0, 'num_object_copies': 18384, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4237, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9192, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 8391, 'num_write_kb': 28172, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4955}, {'shards': '0,2', 'objects': 2889}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:03:27.372 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-10T16:03:27.372 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "45'11635", 'reported_seq': 11775, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:03:24.943005+0000', 'last_change': '2024-09-10T15:59:16.175680+0000', 'last_active': '2024-09-10T16:03:24.943005+0000', 'last_peered': '2024-09-10T16:03:24.943005+0000', 'last_clean': '2024-09-10T15:59:14.088187+0000', 'last_became_active': '2024-09-10T15:59:16.165810+0000', 'last_became_peered': '2024-09-10T15:59:16.165810+0000', 'last_unstale': '2024-09-10T16:03:24.943005+0000', 'last_undegraded': '2024-09-10T15:59:15.822098+0000', 'last_fullsized': '2024-09-10T15:59:15.769123+0000', 'mapping_epoch': 42, 'log_start': "36'1600", 'ondisk_log_start': "36'1600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2218", 'last_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'objects_scrubbed': 2175, 'log_size': 10035, 'log_dups_size': 0, 'ondisk_log_size': 10035, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 825, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36135395, 'num_objects': 8823, 'num_object_clones': 0, 'num_object_copies': 17646, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4124, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8823, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 9454, 'num_write_kb': 32204, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2720}, {'shards': '2', 'objects': 4700}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:03:27.372 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-10T16:03:27.373 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "45'11834", 'reported_seq': 11965, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:03:23.199031+0000', 'last_change': '2024-09-10T15:59:16.277742+0000', 'last_active': '2024-09-10T16:03:23.199031+0000', 'last_peered': '2024-09-10T16:03:23.199031+0000', 'last_clean': '2024-09-10T15:59:14.129081+0000', 'last_became_active': '2024-09-10T15:59:16.269835+0000', 'last_became_peered': '2024-09-10T15:59:16.269835+0000', 'last_unstale': '2024-09-10T16:03:23.199031+0000', 'last_undegraded': '2024-09-10T15:59:15.860857+0000', 'last_fullsized': '2024-09-10T15:59:15.771410+0000', 'mapping_epoch': 42, 'log_start': "36'1800", 'ondisk_log_start': "36'1800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3557", 'last_scrub_stamp': '2024-09-10T15:59:02.493678+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:02.493678+0000', 'objects_scrubbed': 3488, 'log_size': 10034, 'log_dups_size': 0, 'ondisk_log_size': 10034, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1472, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37175296, 'num_objects': 9076, 'num_object_clones': 0, 'num_object_copies': 18152, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4257, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9076, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8338, 'num_write_kb': 27852, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4821}, {'shards': '0,2', 'objects': 2880}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:03:27.373 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-10T16:03:27.373 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "45'11828", 'reported_seq': 11963, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:03:24.884810+0000', 'last_change': '2024-09-10T15:59:16.284311+0000', 'last_active': '2024-09-10T16:03:24.884810+0000', 'last_peered': '2024-09-10T16:03:24.884810+0000', 'last_clean': '2024-09-10T15:59:14.116041+0000', 'last_became_active': '2024-09-10T15:59:16.274543+0000', 'last_became_peered': '2024-09-10T15:59:16.274543+0000', 'last_unstale': '2024-09-10T16:03:24.884810+0000', 'last_undegraded': '2024-09-10T15:59:15.812484+0000', 'last_fullsized': '2024-09-10T15:59:15.770885+0000', 'mapping_epoch': 42, 'log_start': "36'1800", 'ondisk_log_start': "36'1800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2356", 'last_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'objects_scrubbed': 2316, 'log_size': 10028, 'log_dups_size': 0, 'ondisk_log_size': 10028, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 783, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36671971, 'num_objects': 8954, 'num_object_clones': 0, 'num_object_copies': 17908, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8954, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 9508, 'num_write_kb': 32292, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2869}, {'shards': '2', 'objects': 4650}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:03:27.373 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-10T16:03:27.373 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "45'12037", 'reported_seq': 12166, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:03:24.969571+0000', 'last_change': '2024-09-10T15:59:16.174113+0000', 'last_active': '2024-09-10T16:03:24.969571+0000', 'last_peered': '2024-09-10T16:03:24.969571+0000', 'last_clean': '2024-09-10T15:59:14.110654+0000', 'last_became_active': '2024-09-10T15:59:16.164645+0000', 'last_became_peered': '2024-09-10T15:59:16.164645+0000', 'last_unstale': '2024-09-10T16:03:24.969571+0000', 'last_undegraded': '2024-09-10T15:59:15.803593+0000', 'last_fullsized': '2024-09-10T15:59:15.769083+0000', 'mapping_epoch': 42, 'log_start': "36'2000", 'ondisk_log_start': "36'2000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2486", 'last_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'objects_scrubbed': 2440, 'log_size': 10037, 'log_dups_size': 0, 'ondisk_log_size': 10037, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 792, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37408768, 'num_objects': 9133, 'num_object_clones': 0, 'num_object_copies': 18266, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4354, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9133, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9591, 'num_write_kb': 32568, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2903}, {'shards': '2', 'objects': 4781}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:03:27.373 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-10T16:03:27.373 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "45'11793", 'reported_seq': 11928, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:03:23.196635+0000', 'last_change': '2024-09-10T15:59:16.315641+0000', 'last_active': '2024-09-10T16:03:23.196635+0000', 'last_peered': '2024-09-10T16:03:23.196635+0000', 'last_clean': '2024-09-10T15:59:14.117001+0000', 'last_became_active': '2024-09-10T15:59:16.302245+0000', 'last_became_peered': '2024-09-10T15:59:16.302245+0000', 'last_unstale': '2024-09-10T16:03:23.196635+0000', 'last_undegraded': '2024-09-10T15:59:15.845012+0000', 'last_fullsized': '2024-09-10T15:59:15.770337+0000', 'mapping_epoch': 42, 'log_start': "36'1700", 'ondisk_log_start': "36'1700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3404", 'last_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'objects_scrubbed': 3367, 'log_size': 10093, 'log_dups_size': 0, 'ondisk_log_size': 10093, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 983, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37048320, 'num_objects': 9045, 'num_object_clones': 0, 'num_object_copies': 18090, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9045, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8424, 'num_write_kb': 28204, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4741}, {'shards': '0,2', 'objects': 2931}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:03:27.373 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-10T16:03:27.373 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "45'11596", 'reported_seq': 14427, 'reported_epoch': 45, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-10T16:03:23.270695+0000', 'last_change': '2024-09-10T16:02:06.480033+0000', 'last_active': '2024-09-10T16:03:23.270695+0000', 'last_peered': '2024-09-10T16:03:23.270695+0000', 'last_clean': '2024-09-10T15:59:14.115595+0000', 'last_became_active': '2024-09-10T15:59:16.244089+0000', 'last_became_peered': '2024-09-10T15:59:16.244089+0000', 'last_unstale': '2024-09-10T16:03:23.270695+0000', 'last_undegraded': '2024-09-10T16:03:23.270695+0000', 'last_fullsized': '2024-09-10T15:59:15.774437+0000', 'mapping_epoch': 42, 'log_start': "36'1500", 'ondisk_log_start': "36'1500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3653", 'last_scrub_stamp': '2024-09-10T15:59:05.298468+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:05.298468+0000', 'objects_scrubbed': 3602, 'log_size': 10096, 'log_dups_size': 0, 'ondisk_log_size': 10096, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1289, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36872192, 'num_objects': 9002, 'num_object_clones': 0, 'num_object_copies': 18004, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 2859, 'num_objects_unfound': 0, 'num_objects_dirty': 9002, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7986, 'num_write_kb': 26772, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1344, 'num_bytes_recovered': 5505024, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6143}, {'shards': '0,2', 'objects': 2859}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:03:27.374 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-10T16:03:27.374 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "45'11963", 'reported_seq': 12105, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:03:23.154729+0000', 'last_change': '2024-09-10T15:59:16.311291+0000', 'last_active': '2024-09-10T16:03:23.154729+0000', 'last_peered': '2024-09-10T16:03:23.154729+0000', 'last_clean': '2024-09-10T15:59:14.116363+0000', 'last_became_active': '2024-09-10T15:59:16.301244+0000', 'last_became_peered': '2024-09-10T15:59:16.301244+0000', 'last_unstale': '2024-09-10T16:03:23.154729+0000', 'last_undegraded': '2024-09-10T15:59:15.883434+0000', 'last_fullsized': '2024-09-10T15:59:15.772756+0000', 'mapping_epoch': 42, 'log_start': "36'1900", 'ondisk_log_start': "36'1900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3789", 'last_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'objects_scrubbed': 3741, 'log_size': 10063, 'log_dups_size': 0, 'ondisk_log_size': 10063, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1440, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37212160, 'num_objects': 9085, 'num_object_clones': 0, 'num_object_copies': 18170, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4373, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9085, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8218, 'num_write_kb': 27124, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4714}, {'shards': '0,2', 'objects': 2934}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:03:27.374 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-10T16:03:27.374 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "45'11991", 'reported_seq': 12136, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:03:23.259550+0000', 'last_change': '2024-09-10T15:59:16.313409+0000', 'last_active': '2024-09-10T16:03:23.259550+0000', 'last_peered': '2024-09-10T16:03:23.259550+0000', 'last_clean': '2024-09-10T15:59:14.129780+0000', 'last_became_active': '2024-09-10T15:59:16.300932+0000', 'last_became_peered': '2024-09-10T15:59:16.300932+0000', 'last_unstale': '2024-09-10T16:03:23.259550+0000', 'last_undegraded': '2024-09-10T15:59:15.887823+0000', 'last_fullsized': '2024-09-10T15:59:15.768513+0000', 'mapping_epoch': 42, 'log_start': "36'1900", 'ondisk_log_start': "36'1900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3880", 'last_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'objects_scrubbed': 3827, 'log_size': 10091, 'log_dups_size': 0, 'ondisk_log_size': 10091, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1397, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37712838, 'num_objects': 9209, 'num_object_clones': 0, 'num_object_copies': 18418, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4325, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9209, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8154, 'num_write_kb': 27072, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4886}, {'shards': '0,2', 'objects': 2937}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:03:27.374 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T16:03:28.082 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:03:28.082 INFO:teuthology.orchestra.run.smithi115.stdout:{"fsid":"7effee29-df05-4351-9741-ee203564ed81","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 42911/289608 objects degraded (14.817%), 10 pgs degraded, 11 pgs undersized","count":21},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":372,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983954,"num_in_osds":3,"osd_in_since":1725983947,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":144804,"data_bytes":593068809,"bytes_used":2243588096,"bytes_avail":287666704384,"bytes_total":289910292480,"degraded_objects":42911,"degraded_total":289608,"degraded_ratio":0.14816924946824672,"misplaced_objects":2859,"misplaced_total":289608,"misplaced_ratio":0.009871964862849093,"recovering_objects_per_sec":14,"recovering_bytes_per_sec":61416,"recovering_keys_per_sec":0,"num_objects_recovered":180,"num_bytes_recovered":737280,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":377},"fsmap":{"epoch":1,"btime":"2024-09-10T15:57:14:967120+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T16:00:13.560776+0000","services":{}},"progress_events":{}} 2024-09-10T16:03:28.105 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-10T16:03:28.105 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:03:28.701 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:03:28.702 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:03:28.728 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":224,"stamp":"2024-09-10T16:03:27.639000+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":593068809,"num_objects":144804,"num_object_clones":0,"num_object_copies":289608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42911,"num_objects_misplaced":2859,"num_objects_unfound":0,"num_objects_dirty":144804,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":147306,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5618,"num_bytes_recovered":22999059,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161033,"ondisk_log_size":161033,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2191004,"kb_used_data":1239400,"kb_used_omap":12,"kb_used_meta":951539,"kb_avail":280924516,"statfs":{"total":289910292480,"available":287666704384,"internally_reserved":0,"allocated":1269145600,"data_stored":1268221463,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":974375938},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,3,2,6,8,14,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18567168,"num_objects":-4533,"num_object_clones":0,"num_object_copies":-9066,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-180,"num_objects_unfound":0,"num_objects_dirty":-4533,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4533,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":180,"num_bytes_recovered":737280,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004589"},"pg_stats":[{"pgid":"2.f","version":"45'11696","reported_seq":11771,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:23.193808+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:03:23.193808+0000","last_peered":"2024-09-10T16:03:23.193808+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:03:23.193808+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"36'1600","ondisk_log_start":"36'1600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36765696,"num_objects":8976,"num_object_clones":0,"num_object_copies":17952,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8976,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11696,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4717},{"shards":"0,2","objects":2898}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'11859","reported_seq":11972,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.977729+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:03:24.977729+0000","last_peered":"2024-09-10T16:03:24.977729+0000","last_clean":"2024-09-10T16:03:24.977729+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:03:24.977729+0000","last_undegraded":"2024-09-10T16:03:24.977729+0000","last_fullsized":"2024-09-10T16:03:24.977729+0000","mapping_epoch":35,"log_start":"36'1800","ondisk_log_start":"36'1800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36803526,"num_objects":8987,"num_object_clones":0,"num_object_copies":17974,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8987,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":9964,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'11839","reported_seq":20517,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:03:25.951890+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:03:25.951890+0000","last_peered":"2024-09-10T16:03:25.951890+0000","last_clean":"2024-09-10T16:03:25.951890+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:03:25.951890+0000","last_undegraded":"2024-09-10T16:03:25.951890+0000","last_fullsized":"2024-09-10T16:03:25.951890+0000","mapping_epoch":44,"log_start":"36'1800","ondisk_log_start":"36'1800","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36691992,"num_objects":8959,"num_object_clones":0,"num_object_copies":17918,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8959,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9843,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'11793","reported_seq":11874,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.981634+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:03:24.981634+0000","last_peered":"2024-09-10T16:03:24.981634+0000","last_clean":"2024-09-10T16:03:24.981634+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:03:24.981634+0000","last_undegraded":"2024-09-10T16:03:24.981634+0000","last_fullsized":"2024-09-10T16:03:24.981634+0000","mapping_epoch":35,"log_start":"36'1700","ondisk_log_start":"36'1700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36319232,"num_objects":8867,"num_object_clones":0,"num_object_copies":17734,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8867,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11793,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'11790","reported_seq":11920,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:03:23.224421+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:03:23.224421+0000","last_peered":"2024-09-10T16:03:23.224421+0000","last_clean":"2024-09-10T16:03:23.224421+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:03:23.224421+0000","last_undegraded":"2024-09-10T16:03:23.224421+0000","last_fullsized":"2024-09-10T16:03:23.224421+0000","mapping_epoch":34,"log_start":"36'1700","ondisk_log_start":"36'1700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37278179,"num_objects":9102,"num_object_clones":0,"num_object_copies":18204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9102,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8501,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'11932","reported_seq":12070,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:23.244816+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:03:23.244816+0000","last_peered":"2024-09-10T16:03:23.244816+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:03:23.244816+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"36'1900","ondisk_log_start":"36'1900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37392867,"num_objects":9130,"num_object_clones":0,"num_object_copies":18260,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9130,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8544,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4759},{"shards":"0,2","objects":2972}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'11892","reported_seq":12029,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:23.233186+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:03:23.233186+0000","last_peered":"2024-09-10T16:03:23.233186+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:03:23.233186+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"36'1800","ondisk_log_start":"36'1800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37643206,"num_objects":9192,"num_object_clones":0,"num_object_copies":18384,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9192,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":8391,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4955},{"shards":"0,2","objects":2889}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'12052","reported_seq":12182,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:03:23.160958+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:03:23.160958+0000","last_peered":"2024-09-10T16:03:23.160958+0000","last_clean":"2024-09-10T16:03:23.160958+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:03:23.160958+0000","last_undegraded":"2024-09-10T16:03:23.160958+0000","last_fullsized":"2024-09-10T16:03:23.160958+0000","mapping_epoch":34,"log_start":"36'2000","ondisk_log_start":"36'2000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37937152,"num_objects":9262,"num_object_clones":0,"num_object_copies":18524,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9262,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8899,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104261+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:02:08.104261+0000","last_peered":"2024-09-10T16:02:08.104261+0000","last_clean":"2024-09-10T16:02:08.104261+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:02:08.104261+0000","last_undegraded":"2024-09-10T16:02:08.104261+0000","last_fullsized":"2024-09-10T16:02:08.104261+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'11635","reported_seq":11775,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:24.943005+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:03:24.943005+0000","last_peered":"2024-09-10T16:03:24.943005+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:03:24.943005+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"36'1600","ondisk_log_start":"36'1600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36135395,"num_objects":8823,"num_object_clones":0,"num_object_copies":17646,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8823,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9454,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2720},{"shards":"2","objects":4700}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103774+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:02:08.103774+0000","last_peered":"2024-09-10T16:02:08.103774+0000","last_clean":"2024-09-10T16:02:08.103774+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:02:08.103774+0000","last_undegraded":"2024-09-10T16:02:08.103774+0000","last_fullsized":"2024-09-10T16:02:08.103774+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'11834","reported_seq":11965,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:23.199031+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:03:23.199031+0000","last_peered":"2024-09-10T16:03:23.199031+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:03:23.199031+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"36'1800","ondisk_log_start":"36'1800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37175296,"num_objects":9076,"num_object_clones":0,"num_object_copies":18152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9076,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8338,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4821},{"shards":"0,2","objects":2880}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":104,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.105055+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:02:08.105055+0000","last_peered":"2024-09-10T16:02:08.105055+0000","last_clean":"2024-09-10T16:02:08.105055+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:02:08.105055+0000","last_undegraded":"2024-09-10T16:02:08.105055+0000","last_fullsized":"2024-09-10T16:02:08.105055+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'11828","reported_seq":11963,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:24.884810+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:03:24.884810+0000","last_peered":"2024-09-10T16:03:24.884810+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:03:24.884810+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"36'1800","ondisk_log_start":"36'1800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36671971,"num_objects":8954,"num_object_clones":0,"num_object_copies":17908,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8954,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9508,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2869},{"shards":"2","objects":4650}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104617+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:02:08.104617+0000","last_peered":"2024-09-10T16:02:08.104617+0000","last_clean":"2024-09-10T16:02:08.104617+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:02:08.104617+0000","last_undegraded":"2024-09-10T16:02:08.104617+0000","last_fullsized":"2024-09-10T16:02:08.104617+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'12037","reported_seq":12166,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:24.969571+0000","last_change":"2024-09-10T15:59:16.174113+0000","last_active":"2024-09-10T16:03:24.969571+0000","last_peered":"2024-09-10T16:03:24.969571+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:03:24.969571+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"36'2000","ondisk_log_start":"36'2000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37408768,"num_objects":9133,"num_object_clones":0,"num_object_copies":18266,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4354,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9133,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9591,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2903},{"shards":"2","objects":4781}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'11793","reported_seq":11928,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:23.196635+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:03:23.196635+0000","last_peered":"2024-09-10T16:03:23.196635+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:03:23.196635+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"36'1700","ondisk_log_start":"36'1700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37048320,"num_objects":9045,"num_object_clones":0,"num_object_copies":18090,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9045,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8424,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4741},{"shards":"0,2","objects":2931}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103316+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:02:08.103316+0000","last_peered":"2024-09-10T16:02:08.103316+0000","last_clean":"2024-09-10T16:02:08.103316+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:02:08.103316+0000","last_undegraded":"2024-09-10T16:02:08.103316+0000","last_fullsized":"2024-09-10T16:02:08.103316+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'11596","reported_seq":14427,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T16:03:23.270695+0000","last_change":"2024-09-10T16:02:06.480033+0000","last_active":"2024-09-10T16:03:23.270695+0000","last_peered":"2024-09-10T16:03:23.270695+0000","last_clean":"2024-09-10T15:59:14.115595+0000","last_became_active":"2024-09-10T15:59:16.244089+0000","last_became_peered":"2024-09-10T15:59:16.244089+0000","last_unstale":"2024-09-10T16:03:23.270695+0000","last_undegraded":"2024-09-10T16:03:23.270695+0000","last_fullsized":"2024-09-10T15:59:15.774437+0000","mapping_epoch":42,"log_start":"36'1500","ondisk_log_start":"36'1500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36872192,"num_objects":9002,"num_object_clones":0,"num_object_copies":18004,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2859,"num_objects_unfound":0,"num_objects_dirty":9002,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7986,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1344,"num_bytes_recovered":5505024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6143},{"shards":"0,2","objects":2859}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103831+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:02:08.103831+0000","last_peered":"2024-09-10T16:02:08.103831+0000","last_clean":"2024-09-10T16:02:08.103831+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:02:08.103831+0000","last_undegraded":"2024-09-10T16:02:08.103831+0000","last_fullsized":"2024-09-10T16:02:08.103831+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'11963","reported_seq":12105,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:23.154729+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:03:23.154729+0000","last_peered":"2024-09-10T16:03:23.154729+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:03:23.154729+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"36'1900","ondisk_log_start":"36'1900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37212160,"num_objects":9085,"num_object_clones":0,"num_object_copies":18170,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9085,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8218,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4714},{"shards":"0,2","objects":2934}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104463+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:02:08.104463+0000","last_peered":"2024-09-10T16:02:08.104463+0000","last_clean":"2024-09-10T16:02:08.104463+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:02:08.104463+0000","last_undegraded":"2024-09-10T16:02:08.104463+0000","last_fullsized":"2024-09-10T16:02:08.104463+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104410+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:02:08.104410+0000","last_peered":"2024-09-10T16:02:08.104410+0000","last_clean":"2024-09-10T16:02:08.104410+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:02:08.104410+0000","last_undegraded":"2024-09-10T16:02:08.104410+0000","last_fullsized":"2024-09-10T16:02:08.104410+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'11991","reported_seq":12136,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:23.259550+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:03:23.259550+0000","last_peered":"2024-09-10T16:03:23.259550+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:03:23.259550+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"36'1900","ondisk_log_start":"36'1900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37712838,"num_objects":9209,"num_object_clones":0,"num_object_copies":18418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9209,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8154,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4886},{"shards":"0,2","objects":2937}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":593068790,"num_objects":144802,"num_object_clones":0,"num_object_copies":289604,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42911,"num_objects_misplaced":2859,"num_objects_unfound":0,"num_objects_dirty":144802,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":147304,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5616,"num_bytes_recovered":22999040,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1266540544,"data_stored":1266380654,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161030,"ondisk_log_size":161030,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986241,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":728200,"kb_used_data":425784,"kb_used_omap":4,"kb_used_meta":302395,"kb_avail":93643640,"statfs":{"total":96636764160,"available":95891087360,"internally_reserved":0,"allocated":436002816,"data_stored":435710583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":309653162},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,1,0,0,4,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659187,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":578160,"kb_used_data":307676,"kb_used_omap":4,"kb_used_meta":270459,"kb_avail":93793680,"statfs":{"total":96636764160,"available":96044728320,"internally_reserved":0,"allocated":315060224,"data_stored":314720558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":276950702},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,3,4,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182469,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":884644,"kb_used_data":505940,"kb_used_omap":4,"kb_used_meta":378683,"kb_avail":93487196,"statfs":{"total":96636764160,"available":95730888704,"internally_reserved":0,"allocated":518082560,"data_stored":517790322,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":387772074},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,3,0,3,4,5,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":516423680,"data_stored":516387550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":314540032,"data_stored":314456474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":435576832,"data_stored":435536630,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:03:31.731 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:03:32.277 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:03:32.279 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:03:32.299 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":226,"stamp":"2024-09-10T16:03:31.640491+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":583861001,"num_objects":142556,"num_object_clones":0,"num_object_copies":285112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42911,"num_objects_misplaced":2764,"num_objects_unfound":0,"num_objects_dirty":142556,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":149554,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5713,"num_bytes_recovered":23388179,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160881,"ondisk_log_size":160881,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2177868,"kb_used_data":1226236,"kb_used_omap":12,"kb_used_meta":951539,"kb_avail":280937652,"statfs":{"total":289910292480,"available":287680155648,"internally_reserved":0,"allocated":1255665664,"data_stored":1254741527,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":974375938},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,2,3,7,11,12,5,1],"upper_bound":512},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-21635072,"num_objects":-5282,"num_object_clones":0,"num_object_copies":-10564,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-182,"num_objects_unfound":0,"num_objects_dirty":-5282,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5282,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":182,"num_bytes_recovered":745472,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004444"},"pg_stats":[{"pgid":"2.f","version":"45'11826","reported_seq":11901,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:27.998345+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:03:27.998345+0000","last_peered":"2024-09-10T16:03:27.998345+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:03:27.998345+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"36'1800","ondisk_log_start":"36'1800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36233216,"num_objects":8846,"num_object_clones":0,"num_object_copies":17692,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8846,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11826,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4587},{"shards":"0,2","objects":2768}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'12003","reported_seq":12116,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:03:29.958302+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:03:29.958302+0000","last_peered":"2024-09-10T16:03:29.958302+0000","last_clean":"2024-09-10T16:03:29.958302+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:03:29.958302+0000","last_undegraded":"2024-09-10T16:03:29.958302+0000","last_fullsized":"2024-09-10T16:03:29.958302+0000","mapping_epoch":35,"log_start":"36'2000","ondisk_log_start":"36'2000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36213702,"num_objects":8843,"num_object_clones":0,"num_object_copies":17686,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8843,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":10108,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'11982","reported_seq":20660,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:03:30.999530+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:03:30.999530+0000","last_peered":"2024-09-10T16:03:30.999530+0000","last_clean":"2024-09-10T16:03:30.999530+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:03:30.999530+0000","last_undegraded":"2024-09-10T16:03:30.999530+0000","last_fullsized":"2024-09-10T16:03:30.999530+0000","mapping_epoch":44,"log_start":"36'1900","ondisk_log_start":"36'1900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36106264,"num_objects":8816,"num_object_clones":0,"num_object_copies":17632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8816,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9986,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'11922","reported_seq":12003,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:03:30.014383+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:03:30.014383+0000","last_peered":"2024-09-10T16:03:30.014383+0000","last_clean":"2024-09-10T16:03:30.014383+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:03:30.014383+0000","last_undegraded":"2024-09-10T16:03:30.014383+0000","last_fullsized":"2024-09-10T16:03:30.014383+0000","mapping_epoch":35,"log_start":"36'1900","ondisk_log_start":"36'1900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35790848,"num_objects":8738,"num_object_clones":0,"num_object_copies":17476,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8738,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11922,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'11930","reported_seq":12060,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:03:28.316078+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:03:28.316078+0000","last_peered":"2024-09-10T16:03:28.316078+0000","last_clean":"2024-09-10T16:03:28.316078+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:03:28.316078+0000","last_undegraded":"2024-09-10T16:03:28.316078+0000","last_fullsized":"2024-09-10T16:03:28.316078+0000","mapping_epoch":34,"log_start":"36'1900","ondisk_log_start":"36'1900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36704739,"num_objects":8962,"num_object_clones":0,"num_object_copies":17924,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8962,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8641,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'12098","reported_seq":12236,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:28.316301+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:03:28.316301+0000","last_peered":"2024-09-10T16:03:28.316301+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:03:28.316301+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"36'2000","ondisk_log_start":"36'2000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36712931,"num_objects":8964,"num_object_clones":0,"num_object_copies":17928,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8964,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8710,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4593},{"shards":"0,2","objects":2806}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'12050","reported_seq":12188,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:28.276564+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:03:28.276564+0000","last_peered":"2024-09-10T16:03:28.276564+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:03:28.276564+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"36'2000","ondisk_log_start":"36'2000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36996038,"num_objects":9034,"num_object_clones":0,"num_object_copies":18068,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9034,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":8549,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4797},{"shards":"0,2","objects":2731}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'12200","reported_seq":12330,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:03:28.235262+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:03:28.235262+0000","last_peered":"2024-09-10T16:03:28.235262+0000","last_clean":"2024-09-10T16:03:28.235262+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:03:28.235262+0000","last_undegraded":"2024-09-10T16:03:28.235262+0000","last_fullsized":"2024-09-10T16:03:28.235262+0000","mapping_epoch":34,"log_start":"36'2100","ondisk_log_start":"36'2100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37330944,"num_objects":9114,"num_object_clones":0,"num_object_copies":18228,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9114,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9047,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104261+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:02:08.104261+0000","last_peered":"2024-09-10T16:02:08.104261+0000","last_clean":"2024-09-10T16:02:08.104261+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:02:08.104261+0000","last_undegraded":"2024-09-10T16:02:08.104261+0000","last_fullsized":"2024-09-10T16:02:08.104261+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'11791","reported_seq":11926,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:29.994848+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:03:29.994848+0000","last_peered":"2024-09-10T16:03:29.994848+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:03:29.994848+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"36'1700","ondisk_log_start":"36'1700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35496419,"num_objects":8667,"num_object_clones":0,"num_object_copies":17334,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8667,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9610,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2564},{"shards":"2","objects":4544}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103774+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:02:08.103774+0000","last_peered":"2024-09-10T16:02:08.103774+0000","last_clean":"2024-09-10T16:02:08.103774+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:02:08.103774+0000","last_undegraded":"2024-09-10T16:02:08.103774+0000","last_fullsized":"2024-09-10T16:02:08.103774+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'11955","reported_seq":12086,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:28.153076+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:03:28.153076+0000","last_peered":"2024-09-10T16:03:28.153076+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:03:28.153076+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"36'1900","ondisk_log_start":"36'1900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36679680,"num_objects":8955,"num_object_clones":0,"num_object_copies":17910,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8955,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8459,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4700},{"shards":"0,2","objects":2759}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":104,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.105055+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:02:08.105055+0000","last_peered":"2024-09-10T16:02:08.105055+0000","last_clean":"2024-09-10T16:02:08.105055+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:02:08.105055+0000","last_undegraded":"2024-09-10T16:02:08.105055+0000","last_fullsized":"2024-09-10T16:02:08.105055+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'11961","reported_seq":12096,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:29.992167+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:03:29.992167+0000","last_peered":"2024-09-10T16:03:29.992167+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:03:29.992167+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"36'1900","ondisk_log_start":"36'1900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36127203,"num_objects":8821,"num_object_clones":0,"num_object_copies":17642,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8821,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9641,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2736},{"shards":"2","objects":4517}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104617+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:02:08.104617+0000","last_peered":"2024-09-10T16:02:08.104617+0000","last_clean":"2024-09-10T16:02:08.104617+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:02:08.104617+0000","last_undegraded":"2024-09-10T16:02:08.104617+0000","last_fullsized":"2024-09-10T16:02:08.104617+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'12166","reported_seq":12295,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:29.970613+0000","last_change":"2024-09-10T15:59:16.174113+0000","last_active":"2024-09-10T16:03:29.970613+0000","last_peered":"2024-09-10T16:03:29.970613+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:03:29.970613+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"36'2100","ondisk_log_start":"36'2100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36880384,"num_objects":9004,"num_object_clones":0,"num_object_copies":18008,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4354,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9004,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9720,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2774},{"shards":"2","objects":4652}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'11941","reported_seq":12076,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:28.231535+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:03:28.231535+0000","last_peered":"2024-09-10T16:03:28.231535+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:03:28.231535+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"36'1900","ondisk_log_start":"36'1900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36442112,"num_objects":8897,"num_object_clones":0,"num_object_copies":17794,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8897,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8572,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4593},{"shards":"0,2","objects":2783}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103316+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:02:08.103316+0000","last_peered":"2024-09-10T16:02:08.103316+0000","last_clean":"2024-09-10T16:02:08.103316+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:02:08.103316+0000","last_undegraded":"2024-09-10T16:02:08.103316+0000","last_fullsized":"2024-09-10T16:02:08.103316+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'11732","reported_seq":14754,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T16:03:28.334508+0000","last_change":"2024-09-10T16:02:06.480033+0000","last_active":"2024-09-10T16:03:28.334508+0000","last_peered":"2024-09-10T16:03:28.334508+0000","last_clean":"2024-09-10T15:59:14.115595+0000","last_became_active":"2024-09-10T15:59:16.244089+0000","last_became_peered":"2024-09-10T15:59:16.244089+0000","last_unstale":"2024-09-10T16:03:28.334508+0000","last_undegraded":"2024-09-10T16:03:28.334508+0000","last_fullsized":"2024-09-10T15:59:15.774437+0000","mapping_epoch":42,"log_start":"36'1700","ondisk_log_start":"36'1700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36315136,"num_objects":8866,"num_object_clones":0,"num_object_copies":17732,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2764,"num_objects_unfound":0,"num_objects_dirty":8866,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8122,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1439,"num_bytes_recovered":5894144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6102},{"shards":"0,2","objects":2764}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103831+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:02:08.103831+0000","last_peered":"2024-09-10T16:02:08.103831+0000","last_clean":"2024-09-10T16:02:08.103831+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:02:08.103831+0000","last_undegraded":"2024-09-10T16:02:08.103831+0000","last_fullsized":"2024-09-10T16:02:08.103831+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'12098","reported_seq":12237,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:28.238144+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:03:28.238144+0000","last_peered":"2024-09-10T16:03:28.238144+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:03:28.238144+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"36'2000","ondisk_log_start":"36'2000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36659200,"num_objects":8950,"num_object_clones":0,"num_object_copies":17900,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8950,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8353,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4579},{"shards":"0,2","objects":2799}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104463+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:02:08.104463+0000","last_peered":"2024-09-10T16:02:08.104463+0000","last_clean":"2024-09-10T16:02:08.104463+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:02:08.104463+0000","last_undegraded":"2024-09-10T16:02:08.104463+0000","last_fullsized":"2024-09-10T16:02:08.104463+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104410+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:02:08.104410+0000","last_peered":"2024-09-10T16:02:08.104410+0000","last_clean":"2024-09-10T16:02:08.104410+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:02:08.104410+0000","last_undegraded":"2024-09-10T16:02:08.104410+0000","last_fullsized":"2024-09-10T16:02:08.104410+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'12123","reported_seq":12268,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:28.276663+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:03:28.276663+0000","last_peered":"2024-09-10T16:03:28.276663+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:03:28.276663+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"36'2100","ondisk_log_start":"36'2100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37172166,"num_objects":9077,"num_object_clones":0,"num_object_copies":18154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9077,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8286,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4754},{"shards":"0,2","objects":2805}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":583860982,"num_objects":142554,"num_object_clones":0,"num_object_copies":285108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42911,"num_objects_misplaced":2764,"num_objects_unfound":0,"num_objects_dirty":142554,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":149552,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5711,"num_bytes_recovered":23388160,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1254301696,"data_stored":1254141806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160878,"ondisk_log_size":160878,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986242,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":727968,"kb_used_data":421380,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":93643872,"statfs":{"total":96636764160,"available":95891324928,"internally_reserved":0,"allocated":431493120,"data_stored":431200887,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":313913002},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,3,0,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659188,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":569048,"kb_used_data":306888,"kb_used_omap":4,"kb_used_meta":262139,"kb_avail":93802792,"statfs":{"total":96636764160,"available":96054059008,"internally_reserved":0,"allocated":314253312,"data_stored":313913646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":268431022},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,0,3,5,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182470,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":880852,"kb_used_data":497968,"kb_used_omap":4,"kb_used_meta":382843,"kb_avail":93490988,"statfs":{"total":96636764160,"available":95734771712,"internally_reserved":0,"allocated":509919232,"data_stored":509626994,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":392031914},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,2,1,6,4,4,1],"upper_bound":512},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":509435904,"data_stored":509399774,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":313843712,"data_stored":313760154,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":431022080,"data_stored":430981878,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:03:32.300 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:03:32.846 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:03:32.848 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:03:32.882 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":226,"stamp":"2024-09-10T16:03:31.640491+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":583861001,"num_objects":142556,"num_object_clones":0,"num_object_copies":285112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42911,"num_objects_misplaced":2764,"num_objects_unfound":0,"num_objects_dirty":142556,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":149554,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5713,"num_bytes_recovered":23388179,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160881,"ondisk_log_size":160881,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2177868,"kb_used_data":1226236,"kb_used_omap":12,"kb_used_meta":951539,"kb_avail":280937652,"statfs":{"total":289910292480,"available":287680155648,"internally_reserved":0,"allocated":1255665664,"data_stored":1254741527,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":974375938},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,2,3,7,11,12,5,1],"upper_bound":512},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-21635072,"num_objects":-5282,"num_object_clones":0,"num_object_copies":-10564,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-182,"num_objects_unfound":0,"num_objects_dirty":-5282,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5282,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":182,"num_bytes_recovered":745472,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004444"},"pg_stats":[{"pgid":"2.f","version":"45'11826","reported_seq":11901,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:27.998345+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:03:27.998345+0000","last_peered":"2024-09-10T16:03:27.998345+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:03:27.998345+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"36'1800","ondisk_log_start":"36'1800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36233216,"num_objects":8846,"num_object_clones":0,"num_object_copies":17692,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8846,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11826,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4587},{"shards":"0,2","objects":2768}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'12003","reported_seq":12116,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:03:29.958302+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:03:29.958302+0000","last_peered":"2024-09-10T16:03:29.958302+0000","last_clean":"2024-09-10T16:03:29.958302+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:03:29.958302+0000","last_undegraded":"2024-09-10T16:03:29.958302+0000","last_fullsized":"2024-09-10T16:03:29.958302+0000","mapping_epoch":35,"log_start":"36'2000","ondisk_log_start":"36'2000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36213702,"num_objects":8843,"num_object_clones":0,"num_object_copies":17686,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8843,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":10108,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'11982","reported_seq":20660,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:03:30.999530+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:03:30.999530+0000","last_peered":"2024-09-10T16:03:30.999530+0000","last_clean":"2024-09-10T16:03:30.999530+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:03:30.999530+0000","last_undegraded":"2024-09-10T16:03:30.999530+0000","last_fullsized":"2024-09-10T16:03:30.999530+0000","mapping_epoch":44,"log_start":"36'1900","ondisk_log_start":"36'1900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36106264,"num_objects":8816,"num_object_clones":0,"num_object_copies":17632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8816,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9986,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'11922","reported_seq":12003,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:03:30.014383+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:03:30.014383+0000","last_peered":"2024-09-10T16:03:30.014383+0000","last_clean":"2024-09-10T16:03:30.014383+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:03:30.014383+0000","last_undegraded":"2024-09-10T16:03:30.014383+0000","last_fullsized":"2024-09-10T16:03:30.014383+0000","mapping_epoch":35,"log_start":"36'1900","ondisk_log_start":"36'1900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35790848,"num_objects":8738,"num_object_clones":0,"num_object_copies":17476,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8738,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11922,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'11930","reported_seq":12060,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:03:28.316078+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:03:28.316078+0000","last_peered":"2024-09-10T16:03:28.316078+0000","last_clean":"2024-09-10T16:03:28.316078+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:03:28.316078+0000","last_undegraded":"2024-09-10T16:03:28.316078+0000","last_fullsized":"2024-09-10T16:03:28.316078+0000","mapping_epoch":34,"log_start":"36'1900","ondisk_log_start":"36'1900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36704739,"num_objects":8962,"num_object_clones":0,"num_object_copies":17924,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8962,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8641,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'12098","reported_seq":12236,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:28.316301+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:03:28.316301+0000","last_peered":"2024-09-10T16:03:28.316301+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:03:28.316301+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"36'2000","ondisk_log_start":"36'2000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36712931,"num_objects":8964,"num_object_clones":0,"num_object_copies":17928,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8964,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8710,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4593},{"shards":"0,2","objects":2806}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'12050","reported_seq":12188,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:28.276564+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:03:28.276564+0000","last_peered":"2024-09-10T16:03:28.276564+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:03:28.276564+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"36'2000","ondisk_log_start":"36'2000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36996038,"num_objects":9034,"num_object_clones":0,"num_object_copies":18068,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9034,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":8549,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4797},{"shards":"0,2","objects":2731}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'12200","reported_seq":12330,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:03:28.235262+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:03:28.235262+0000","last_peered":"2024-09-10T16:03:28.235262+0000","last_clean":"2024-09-10T16:03:28.235262+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:03:28.235262+0000","last_undegraded":"2024-09-10T16:03:28.235262+0000","last_fullsized":"2024-09-10T16:03:28.235262+0000","mapping_epoch":34,"log_start":"36'2100","ondisk_log_start":"36'2100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37330944,"num_objects":9114,"num_object_clones":0,"num_object_copies":18228,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9114,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9047,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104261+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:02:08.104261+0000","last_peered":"2024-09-10T16:02:08.104261+0000","last_clean":"2024-09-10T16:02:08.104261+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:02:08.104261+0000","last_undegraded":"2024-09-10T16:02:08.104261+0000","last_fullsized":"2024-09-10T16:02:08.104261+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'11791","reported_seq":11926,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:29.994848+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:03:29.994848+0000","last_peered":"2024-09-10T16:03:29.994848+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:03:29.994848+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"36'1700","ondisk_log_start":"36'1700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35496419,"num_objects":8667,"num_object_clones":0,"num_object_copies":17334,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8667,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9610,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2564},{"shards":"2","objects":4544}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103774+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:02:08.103774+0000","last_peered":"2024-09-10T16:02:08.103774+0000","last_clean":"2024-09-10T16:02:08.103774+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:02:08.103774+0000","last_undegraded":"2024-09-10T16:02:08.103774+0000","last_fullsized":"2024-09-10T16:02:08.103774+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'11955","reported_seq":12086,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:28.153076+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:03:28.153076+0000","last_peered":"2024-09-10T16:03:28.153076+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:03:28.153076+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"36'1900","ondisk_log_start":"36'1900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36679680,"num_objects":8955,"num_object_clones":0,"num_object_copies":17910,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8955,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8459,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4700},{"shards":"0,2","objects":2759}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":104,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.105055+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:02:08.105055+0000","last_peered":"2024-09-10T16:02:08.105055+0000","last_clean":"2024-09-10T16:02:08.105055+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:02:08.105055+0000","last_undegraded":"2024-09-10T16:02:08.105055+0000","last_fullsized":"2024-09-10T16:02:08.105055+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'11961","reported_seq":12096,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:29.992167+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:03:29.992167+0000","last_peered":"2024-09-10T16:03:29.992167+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:03:29.992167+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"36'1900","ondisk_log_start":"36'1900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36127203,"num_objects":8821,"num_object_clones":0,"num_object_copies":17642,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8821,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9641,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2736},{"shards":"2","objects":4517}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104617+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:02:08.104617+0000","last_peered":"2024-09-10T16:02:08.104617+0000","last_clean":"2024-09-10T16:02:08.104617+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:02:08.104617+0000","last_undegraded":"2024-09-10T16:02:08.104617+0000","last_fullsized":"2024-09-10T16:02:08.104617+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'12166","reported_seq":12295,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:29.970613+0000","last_change":"2024-09-10T15:59:16.174113+0000","last_active":"2024-09-10T16:03:29.970613+0000","last_peered":"2024-09-10T16:03:29.970613+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:03:29.970613+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"36'2100","ondisk_log_start":"36'2100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36880384,"num_objects":9004,"num_object_clones":0,"num_object_copies":18008,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4354,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9004,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9720,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2774},{"shards":"2","objects":4652}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'11941","reported_seq":12076,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:28.231535+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:03:28.231535+0000","last_peered":"2024-09-10T16:03:28.231535+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:03:28.231535+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"36'1900","ondisk_log_start":"36'1900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36442112,"num_objects":8897,"num_object_clones":0,"num_object_copies":17794,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8897,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8572,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4593},{"shards":"0,2","objects":2783}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103316+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:02:08.103316+0000","last_peered":"2024-09-10T16:02:08.103316+0000","last_clean":"2024-09-10T16:02:08.103316+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:02:08.103316+0000","last_undegraded":"2024-09-10T16:02:08.103316+0000","last_fullsized":"2024-09-10T16:02:08.103316+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'11732","reported_seq":14754,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T16:03:28.334508+0000","last_change":"2024-09-10T16:02:06.480033+0000","last_active":"2024-09-10T16:03:28.334508+0000","last_peered":"2024-09-10T16:03:28.334508+0000","last_clean":"2024-09-10T15:59:14.115595+0000","last_became_active":"2024-09-10T15:59:16.244089+0000","last_became_peered":"2024-09-10T15:59:16.244089+0000","last_unstale":"2024-09-10T16:03:28.334508+0000","last_undegraded":"2024-09-10T16:03:28.334508+0000","last_fullsized":"2024-09-10T15:59:15.774437+0000","mapping_epoch":42,"log_start":"36'1700","ondisk_log_start":"36'1700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36315136,"num_objects":8866,"num_object_clones":0,"num_object_copies":17732,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2764,"num_objects_unfound":0,"num_objects_dirty":8866,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8122,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1439,"num_bytes_recovered":5894144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6102},{"shards":"0,2","objects":2764}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103831+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:02:08.103831+0000","last_peered":"2024-09-10T16:02:08.103831+0000","last_clean":"2024-09-10T16:02:08.103831+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:02:08.103831+0000","last_undegraded":"2024-09-10T16:02:08.103831+0000","last_fullsized":"2024-09-10T16:02:08.103831+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'12098","reported_seq":12237,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:28.238144+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:03:28.238144+0000","last_peered":"2024-09-10T16:03:28.238144+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:03:28.238144+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"36'2000","ondisk_log_start":"36'2000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36659200,"num_objects":8950,"num_object_clones":0,"num_object_copies":17900,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8950,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8353,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4579},{"shards":"0,2","objects":2799}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104463+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:02:08.104463+0000","last_peered":"2024-09-10T16:02:08.104463+0000","last_clean":"2024-09-10T16:02:08.104463+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:02:08.104463+0000","last_undegraded":"2024-09-10T16:02:08.104463+0000","last_fullsized":"2024-09-10T16:02:08.104463+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104410+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:02:08.104410+0000","last_peered":"2024-09-10T16:02:08.104410+0000","last_clean":"2024-09-10T16:02:08.104410+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:02:08.104410+0000","last_undegraded":"2024-09-10T16:02:08.104410+0000","last_fullsized":"2024-09-10T16:02:08.104410+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'12123","reported_seq":12268,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:28.276663+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:03:28.276663+0000","last_peered":"2024-09-10T16:03:28.276663+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:03:28.276663+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"36'2100","ondisk_log_start":"36'2100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37172166,"num_objects":9077,"num_object_clones":0,"num_object_copies":18154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9077,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8286,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4754},{"shards":"0,2","objects":2805}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":583860982,"num_objects":142554,"num_object_clones":0,"num_object_copies":285108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42911,"num_objects_misplaced":2764,"num_objects_unfound":0,"num_objects_dirty":142554,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":149552,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5711,"num_bytes_recovered":23388160,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1254301696,"data_stored":1254141806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160878,"ondisk_log_size":160878,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986242,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":727968,"kb_used_data":421380,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":93643872,"statfs":{"total":96636764160,"available":95891324928,"internally_reserved":0,"allocated":431493120,"data_stored":431200887,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":313913002},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,3,0,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659188,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":569048,"kb_used_data":306888,"kb_used_omap":4,"kb_used_meta":262139,"kb_avail":93802792,"statfs":{"total":96636764160,"available":96054059008,"internally_reserved":0,"allocated":314253312,"data_stored":313913646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":268431022},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,0,3,5,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182470,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":880852,"kb_used_data":497968,"kb_used_omap":4,"kb_used_meta":382843,"kb_avail":93490988,"statfs":{"total":96636764160,"available":95734771712,"internally_reserved":0,"allocated":509919232,"data_stored":509626994,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":392031914},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,2,1,6,4,4,1],"upper_bound":512},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":509435904,"data_stored":509399774,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":313843712,"data_stored":313760154,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":431022080,"data_stored":430981878,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:03:32.883 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-10T16:03:32.883 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "45'11826", 'reported_seq': 11901, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:03:27.998345+0000', 'last_change': '2024-09-10T15:59:16.281558+0000', 'last_active': '2024-09-10T16:03:27.998345+0000', 'last_peered': '2024-09-10T16:03:27.998345+0000', 'last_clean': '2024-09-10T15:59:14.132456+0000', 'last_became_active': '2024-09-10T15:59:16.274407+0000', 'last_became_peered': '2024-09-10T15:59:16.274407+0000', 'last_unstale': '2024-09-10T16:03:27.998345+0000', 'last_undegraded': '2024-09-10T15:59:15.851279+0000', 'last_fullsized': '2024-09-10T15:59:15.774356+0000', 'mapping_epoch': 42, 'log_start': "36'1800", 'ondisk_log_start': "36'1800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'objects_scrubbed': 0, 'log_size': 10026, 'log_dups_size': 0, 'ondisk_log_size': 10026, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36233216, 'num_objects': 8846, 'num_object_clones': 0, 'num_object_copies': 17692, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4259, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8846, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11826, 'num_write_kb': 41344, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4587}, {'shards': '0,2', 'objects': 2768}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:03:32.884 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-10T16:03:32.884 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "45'12098", 'reported_seq': 12236, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:03:28.316301+0000', 'last_change': '2024-09-10T15:59:16.308264+0000', 'last_active': '2024-09-10T16:03:28.316301+0000', 'last_peered': '2024-09-10T16:03:28.316301+0000', 'last_clean': '2024-09-10T15:59:14.111551+0000', 'last_became_active': '2024-09-10T15:59:16.300040+0000', 'last_became_peered': '2024-09-10T15:59:16.300040+0000', 'last_unstale': '2024-09-10T16:03:28.316301+0000', 'last_undegraded': '2024-09-10T15:59:15.878234+0000', 'last_fullsized': '2024-09-10T15:59:15.769328+0000', 'mapping_epoch': 42, 'log_start': "36'2000", 'ondisk_log_start': "36'2000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3442", 'last_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'objects_scrubbed': 3384, 'log_size': 10098, 'log_dups_size': 0, 'ondisk_log_size': 10098, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1292, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36712931, 'num_objects': 8964, 'num_object_clones': 0, 'num_object_copies': 17928, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4372, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8964, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 8710, 'num_write_kb': 28580, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4593}, {'shards': '0,2', 'objects': 2806}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:03:32.884 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-10T16:03:32.884 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "45'12050", 'reported_seq': 12188, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:03:28.276564+0000', 'last_change': '2024-09-10T15:59:16.331899+0000', 'last_active': '2024-09-10T16:03:28.276564+0000', 'last_peered': '2024-09-10T16:03:28.276564+0000', 'last_clean': '2024-09-10T15:59:14.139436+0000', 'last_became_active': '2024-09-10T15:59:16.318715+0000', 'last_became_peered': '2024-09-10T15:59:16.318715+0000', 'last_unstale': '2024-09-10T16:03:28.276564+0000', 'last_undegraded': '2024-09-10T15:59:15.877524+0000', 'last_fullsized': '2024-09-10T15:59:15.769042+0000', 'mapping_epoch': 42, 'log_start': "36'2000", 'ondisk_log_start': "36'2000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3537", 'last_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'objects_scrubbed': 3497, 'log_size': 10050, 'log_dups_size': 0, 'ondisk_log_size': 10050, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1224, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36996038, 'num_objects': 9034, 'num_object_clones': 0, 'num_object_copies': 18068, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4237, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9034, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 8549, 'num_write_kb': 28172, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4797}, {'shards': '0,2', 'objects': 2731}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:03:32.884 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-10T16:03:32.884 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "45'11791", 'reported_seq': 11926, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:03:29.994848+0000', 'last_change': '2024-09-10T15:59:16.175680+0000', 'last_active': '2024-09-10T16:03:29.994848+0000', 'last_peered': '2024-09-10T16:03:29.994848+0000', 'last_clean': '2024-09-10T15:59:14.088187+0000', 'last_became_active': '2024-09-10T15:59:16.165810+0000', 'last_became_peered': '2024-09-10T15:59:16.165810+0000', 'last_unstale': '2024-09-10T16:03:29.994848+0000', 'last_undegraded': '2024-09-10T15:59:15.822098+0000', 'last_fullsized': '2024-09-10T15:59:15.769123+0000', 'mapping_epoch': 42, 'log_start': "36'1700", 'ondisk_log_start': "36'1700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2218", 'last_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'objects_scrubbed': 2175, 'log_size': 10091, 'log_dups_size': 0, 'ondisk_log_size': 10091, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 825, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35496419, 'num_objects': 8667, 'num_object_clones': 0, 'num_object_copies': 17334, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4124, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8667, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 9610, 'num_write_kb': 32204, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2564}, {'shards': '2', 'objects': 4544}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:03:32.884 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-10T16:03:32.884 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "45'11955", 'reported_seq': 12086, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:03:28.153076+0000', 'last_change': '2024-09-10T15:59:16.277742+0000', 'last_active': '2024-09-10T16:03:28.153076+0000', 'last_peered': '2024-09-10T16:03:28.153076+0000', 'last_clean': '2024-09-10T15:59:14.129081+0000', 'last_became_active': '2024-09-10T15:59:16.269835+0000', 'last_became_peered': '2024-09-10T15:59:16.269835+0000', 'last_unstale': '2024-09-10T16:03:28.153076+0000', 'last_undegraded': '2024-09-10T15:59:15.860857+0000', 'last_fullsized': '2024-09-10T15:59:15.771410+0000', 'mapping_epoch': 42, 'log_start': "36'1900", 'ondisk_log_start': "36'1900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3557", 'last_scrub_stamp': '2024-09-10T15:59:02.493678+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:02.493678+0000', 'objects_scrubbed': 3488, 'log_size': 10055, 'log_dups_size': 0, 'ondisk_log_size': 10055, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1472, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36679680, 'num_objects': 8955, 'num_object_clones': 0, 'num_object_copies': 17910, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4257, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8955, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8459, 'num_write_kb': 27852, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4700}, {'shards': '0,2', 'objects': 2759}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:03:32.884 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-10T16:03:32.884 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "45'11961", 'reported_seq': 12096, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:03:29.992167+0000', 'last_change': '2024-09-10T15:59:16.284311+0000', 'last_active': '2024-09-10T16:03:29.992167+0000', 'last_peered': '2024-09-10T16:03:29.992167+0000', 'last_clean': '2024-09-10T15:59:14.116041+0000', 'last_became_active': '2024-09-10T15:59:16.274543+0000', 'last_became_peered': '2024-09-10T15:59:16.274543+0000', 'last_unstale': '2024-09-10T16:03:29.992167+0000', 'last_undegraded': '2024-09-10T15:59:15.812484+0000', 'last_fullsized': '2024-09-10T15:59:15.770885+0000', 'mapping_epoch': 42, 'log_start': "36'1900", 'ondisk_log_start': "36'1900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2356", 'last_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'objects_scrubbed': 2316, 'log_size': 10061, 'log_dups_size': 0, 'ondisk_log_size': 10061, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 783, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36127203, 'num_objects': 8821, 'num_object_clones': 0, 'num_object_copies': 17642, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8821, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 9641, 'num_write_kb': 32292, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2736}, {'shards': '2', 'objects': 4517}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:03:32.885 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-10T16:03:32.885 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "45'12166", 'reported_seq': 12295, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:03:29.970613+0000', 'last_change': '2024-09-10T15:59:16.174113+0000', 'last_active': '2024-09-10T16:03:29.970613+0000', 'last_peered': '2024-09-10T16:03:29.970613+0000', 'last_clean': '2024-09-10T15:59:14.110654+0000', 'last_became_active': '2024-09-10T15:59:16.164645+0000', 'last_became_peered': '2024-09-10T15:59:16.164645+0000', 'last_unstale': '2024-09-10T16:03:29.970613+0000', 'last_undegraded': '2024-09-10T15:59:15.803593+0000', 'last_fullsized': '2024-09-10T15:59:15.769083+0000', 'mapping_epoch': 42, 'log_start': "36'2100", 'ondisk_log_start': "36'2100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2486", 'last_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'objects_scrubbed': 2440, 'log_size': 10066, 'log_dups_size': 0, 'ondisk_log_size': 10066, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 792, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36880384, 'num_objects': 9004, 'num_object_clones': 0, 'num_object_copies': 18008, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4354, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9004, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9720, 'num_write_kb': 32568, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2774}, {'shards': '2', 'objects': 4652}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:03:32.885 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-10T16:03:32.885 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "45'11941", 'reported_seq': 12076, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:03:28.231535+0000', 'last_change': '2024-09-10T15:59:16.315641+0000', 'last_active': '2024-09-10T16:03:28.231535+0000', 'last_peered': '2024-09-10T16:03:28.231535+0000', 'last_clean': '2024-09-10T15:59:14.117001+0000', 'last_became_active': '2024-09-10T15:59:16.302245+0000', 'last_became_peered': '2024-09-10T15:59:16.302245+0000', 'last_unstale': '2024-09-10T16:03:28.231535+0000', 'last_undegraded': '2024-09-10T15:59:15.845012+0000', 'last_fullsized': '2024-09-10T15:59:15.770337+0000', 'mapping_epoch': 42, 'log_start': "36'1900", 'ondisk_log_start': "36'1900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3404", 'last_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'objects_scrubbed': 3367, 'log_size': 10041, 'log_dups_size': 0, 'ondisk_log_size': 10041, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 983, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36442112, 'num_objects': 8897, 'num_object_clones': 0, 'num_object_copies': 17794, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8897, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8572, 'num_write_kb': 28204, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4593}, {'shards': '0,2', 'objects': 2783}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:03:32.885 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-10T16:03:32.885 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "45'11732", 'reported_seq': 14754, 'reported_epoch': 45, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-10T16:03:28.334508+0000', 'last_change': '2024-09-10T16:02:06.480033+0000', 'last_active': '2024-09-10T16:03:28.334508+0000', 'last_peered': '2024-09-10T16:03:28.334508+0000', 'last_clean': '2024-09-10T15:59:14.115595+0000', 'last_became_active': '2024-09-10T15:59:16.244089+0000', 'last_became_peered': '2024-09-10T15:59:16.244089+0000', 'last_unstale': '2024-09-10T16:03:28.334508+0000', 'last_undegraded': '2024-09-10T16:03:28.334508+0000', 'last_fullsized': '2024-09-10T15:59:15.774437+0000', 'mapping_epoch': 42, 'log_start': "36'1700", 'ondisk_log_start': "36'1700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3653", 'last_scrub_stamp': '2024-09-10T15:59:05.298468+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:05.298468+0000', 'objects_scrubbed': 3602, 'log_size': 10032, 'log_dups_size': 0, 'ondisk_log_size': 10032, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1289, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36315136, 'num_objects': 8866, 'num_object_clones': 0, 'num_object_copies': 17732, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 2764, 'num_objects_unfound': 0, 'num_objects_dirty': 8866, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8122, 'num_write_kb': 26772, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1439, 'num_bytes_recovered': 5894144, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6102}, {'shards': '0,2', 'objects': 2764}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:03:32.885 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-10T16:03:32.885 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "45'12098", 'reported_seq': 12237, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:03:28.238144+0000', 'last_change': '2024-09-10T15:59:16.311291+0000', 'last_active': '2024-09-10T16:03:28.238144+0000', 'last_peered': '2024-09-10T16:03:28.238144+0000', 'last_clean': '2024-09-10T15:59:14.116363+0000', 'last_became_active': '2024-09-10T15:59:16.301244+0000', 'last_became_peered': '2024-09-10T15:59:16.301244+0000', 'last_unstale': '2024-09-10T16:03:28.238144+0000', 'last_undegraded': '2024-09-10T15:59:15.883434+0000', 'last_fullsized': '2024-09-10T15:59:15.772756+0000', 'mapping_epoch': 42, 'log_start': "36'2000", 'ondisk_log_start': "36'2000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3789", 'last_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'objects_scrubbed': 3741, 'log_size': 10098, 'log_dups_size': 0, 'ondisk_log_size': 10098, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1440, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36659200, 'num_objects': 8950, 'num_object_clones': 0, 'num_object_copies': 17900, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4373, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8950, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8353, 'num_write_kb': 27124, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4579}, {'shards': '0,2', 'objects': 2799}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:03:32.885 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-10T16:03:32.885 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "45'12123", 'reported_seq': 12268, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:03:28.276663+0000', 'last_change': '2024-09-10T15:59:16.313409+0000', 'last_active': '2024-09-10T16:03:28.276663+0000', 'last_peered': '2024-09-10T16:03:28.276663+0000', 'last_clean': '2024-09-10T15:59:14.129780+0000', 'last_became_active': '2024-09-10T15:59:16.300932+0000', 'last_became_peered': '2024-09-10T15:59:16.300932+0000', 'last_unstale': '2024-09-10T16:03:28.276663+0000', 'last_undegraded': '2024-09-10T15:59:15.887823+0000', 'last_fullsized': '2024-09-10T15:59:15.768513+0000', 'mapping_epoch': 42, 'log_start': "36'2100", 'ondisk_log_start': "36'2100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3880", 'last_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'objects_scrubbed': 3827, 'log_size': 10023, 'log_dups_size': 0, 'ondisk_log_size': 10023, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1397, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37172166, 'num_objects': 9077, 'num_object_clones': 0, 'num_object_copies': 18154, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4325, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9077, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8286, 'num_write_kb': 27072, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4754}, {'shards': '0,2', 'objects': 2805}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:03:32.886 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T16:03:33.561 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:03:33.562 INFO:teuthology.orchestra.run.smithi115.stdout:{"fsid":"7effee29-df05-4351-9741-ee203564ed81","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 42911/285112 objects degraded (15.051%), 10 pgs degraded, 11 pgs undersized","count":21},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":377,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983954,"num_in_osds":3,"osd_in_since":1725983947,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":142556,"data_bytes":583861001,"bytes_used":2230136832,"bytes_avail":287680155648,"bytes_total":289910292480,"degraded_objects":42911,"degraded_total":285112,"degraded_ratio":0.15050576615505484,"misplaced_objects":2764,"misplaced_total":285112,"misplaced_ratio":0.0096944358708156795,"recovering_objects_per_sec":15,"recovering_bytes_per_sec":62099,"recovering_keys_per_sec":0,"num_objects_recovered":182,"num_bytes_recovered":745472,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":440},"fsmap":{"epoch":1,"btime":"2024-09-10T15:57:14:967120+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T16:00:13.560776+0000","services":{}},"progress_events":{}} 2024-09-10T16:03:33.585 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-10T16:03:33.586 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:03:34.130 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:03:34.130 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:03:34.152 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":227,"stamp":"2024-09-10T16:03:33.641148+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":578159369,"num_objects":141164,"num_object_clones":0,"num_object_copies":282328,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42936,"num_objects_misplaced":2640,"num_objects_unfound":0,"num_objects_dirty":141164,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":150946,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5812,"num_bytes_recovered":23793683,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160973,"ondisk_log_size":160973,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2163012,"kb_used_data":1219712,"kb_used_omap":12,"kb_used_meta":943219,"kb_avail":280952508,"statfs":{"total":289910292480,"available":287695368192,"internally_reserved":0,"allocated":1248985088,"data_stored":1248060951,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":965856258},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,3,9,8,13,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-23883776,"num_objects":-5831,"num_object_clones":0,"num_object_copies":-11662,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":25,"num_objects_misplaced":-306,"num_objects_unfound":0,"num_objects_dirty":-5831,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5831,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":281,"num_bytes_recovered":1150976,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004414"},"pg_stats":[{"pgid":"2.f","version":"45'11951","reported_seq":12026,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:33.282541+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:03:33.282541+0000","last_peered":"2024-09-10T16:03:33.282541+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:03:33.282541+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"36'1900","ondisk_log_start":"36'1900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35721216,"num_objects":8721,"num_object_clones":0,"num_object_copies":17442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8721,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11951,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4462},{"shards":"0,2","objects":2643}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'12003","reported_seq":12116,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:03:29.958302+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:03:29.958302+0000","last_peered":"2024-09-10T16:03:29.958302+0000","last_clean":"2024-09-10T16:03:29.958302+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:03:29.958302+0000","last_undegraded":"2024-09-10T16:03:29.958302+0000","last_fullsized":"2024-09-10T16:03:29.958302+0000","mapping_epoch":35,"log_start":"36'2000","ondisk_log_start":"36'2000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36213702,"num_objects":8843,"num_object_clones":0,"num_object_copies":17686,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8843,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":10108,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'11982","reported_seq":20660,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:03:30.999530+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:03:30.999530+0000","last_peered":"2024-09-10T16:03:30.999530+0000","last_clean":"2024-09-10T16:03:30.999530+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:03:30.999530+0000","last_undegraded":"2024-09-10T16:03:30.999530+0000","last_fullsized":"2024-09-10T16:03:30.999530+0000","mapping_epoch":44,"log_start":"36'1900","ondisk_log_start":"36'1900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36106264,"num_objects":8816,"num_object_clones":0,"num_object_copies":17632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8816,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9986,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'11922","reported_seq":12003,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:03:30.014383+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:03:30.014383+0000","last_peered":"2024-09-10T16:03:30.014383+0000","last_clean":"2024-09-10T16:03:30.014383+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:03:30.014383+0000","last_undegraded":"2024-09-10T16:03:30.014383+0000","last_fullsized":"2024-09-10T16:03:30.014383+0000","mapping_epoch":35,"log_start":"36'1900","ondisk_log_start":"36'1900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35790848,"num_objects":8738,"num_object_clones":0,"num_object_copies":17476,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8738,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11922,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'12087","reported_seq":12217,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:03:33.391384+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:03:33.391384+0000","last_peered":"2024-09-10T16:03:33.391384+0000","last_clean":"2024-09-10T16:03:33.391384+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:03:33.391384+0000","last_undegraded":"2024-09-10T16:03:33.391384+0000","last_fullsized":"2024-09-10T16:03:33.391384+0000","mapping_epoch":34,"log_start":"36'2000","ondisk_log_start":"36'2000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36061667,"num_objects":8805,"num_object_clones":0,"num_object_copies":17610,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8805,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8798,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'12225","reported_seq":12363,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:33.359065+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:03:33.359065+0000","last_peered":"2024-09-10T16:03:33.359065+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:03:33.359065+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"36'2200","ondisk_log_start":"36'2200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36192739,"num_objects":8837,"num_object_clones":0,"num_object_copies":17674,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8837,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8837,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4466},{"shards":"0,2","objects":2679}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'12195","reported_seq":12333,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:33.382997+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:03:33.382997+0000","last_peered":"2024-09-10T16:03:33.382997+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:03:33.382997+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"36'2100","ondisk_log_start":"36'2100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36402118,"num_objects":8889,"num_object_clones":0,"num_object_copies":17778,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8889,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":8694,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4652},{"shards":"0,2","objects":2586}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'12324","reported_seq":12454,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:03:33.345736+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:03:33.345736+0000","last_peered":"2024-09-10T16:03:33.345736+0000","last_clean":"2024-09-10T16:03:33.345736+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:03:33.345736+0000","last_undegraded":"2024-09-10T16:03:33.345736+0000","last_fullsized":"2024-09-10T16:03:33.345736+0000","mapping_epoch":34,"log_start":"36'2300","ondisk_log_start":"36'2300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36823040,"num_objects":8990,"num_object_clones":0,"num_object_copies":17980,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8990,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9171,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104261+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:02:08.104261+0000","last_peered":"2024-09-10T16:02:08.104261+0000","last_clean":"2024-09-10T16:02:08.104261+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:02:08.104261+0000","last_undegraded":"2024-09-10T16:02:08.104261+0000","last_fullsized":"2024-09-10T16:02:08.104261+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'11791","reported_seq":11926,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:29.994848+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:03:29.994848+0000","last_peered":"2024-09-10T16:03:29.994848+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:03:29.994848+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"36'1700","ondisk_log_start":"36'1700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35496419,"num_objects":8667,"num_object_clones":0,"num_object_copies":17334,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8667,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9610,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2564},{"shards":"2","objects":4544}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103774+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:02:08.103774+0000","last_peered":"2024-09-10T16:02:08.103774+0000","last_clean":"2024-09-10T16:02:08.103774+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:02:08.103774+0000","last_undegraded":"2024-09-10T16:02:08.103774+0000","last_fullsized":"2024-09-10T16:02:08.103774+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'12100","reported_seq":12231,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:33.241741+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:03:33.241741+0000","last_peered":"2024-09-10T16:03:33.241741+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:03:33.241741+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"36'2000","ondisk_log_start":"36'2000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36085760,"num_objects":8810,"num_object_clones":0,"num_object_copies":17620,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8810,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8604,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4555},{"shards":"0,2","objects":2614}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":104,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.105055+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:02:08.105055+0000","last_peered":"2024-09-10T16:02:08.105055+0000","last_clean":"2024-09-10T16:02:08.105055+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:02:08.105055+0000","last_undegraded":"2024-09-10T16:02:08.105055+0000","last_fullsized":"2024-09-10T16:02:08.105055+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'11961","reported_seq":12096,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:29.992167+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:03:29.992167+0000","last_peered":"2024-09-10T16:03:29.992167+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:03:29.992167+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"36'1900","ondisk_log_start":"36'1900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36127203,"num_objects":8821,"num_object_clones":0,"num_object_copies":17642,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8821,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9641,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2736},{"shards":"2","objects":4517}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104617+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:02:08.104617+0000","last_peered":"2024-09-10T16:02:08.104617+0000","last_clean":"2024-09-10T16:02:08.104617+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:02:08.104617+0000","last_undegraded":"2024-09-10T16:02:08.104617+0000","last_fullsized":"2024-09-10T16:02:08.104617+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'12166","reported_seq":12295,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:29.970613+0000","last_change":"2024-09-10T15:59:16.174113+0000","last_active":"2024-09-10T16:03:29.970613+0000","last_peered":"2024-09-10T16:03:29.970613+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:03:29.970613+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"36'2100","ondisk_log_start":"36'2100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36880384,"num_objects":9004,"num_object_clones":0,"num_object_copies":18008,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4354,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9004,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9720,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2774},{"shards":"2","objects":4652}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'12086","reported_seq":12221,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:33.155068+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:03:33.155068+0000","last_peered":"2024-09-10T16:03:33.155068+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:03:33.155068+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"36'2000","ondisk_log_start":"36'2000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35848192,"num_objects":8752,"num_object_clones":0,"num_object_copies":17504,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8752,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8717,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4448},{"shards":"0,2","objects":2638}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103316+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:02:08.103316+0000","last_peered":"2024-09-10T16:02:08.103316+0000","last_clean":"2024-09-10T16:02:08.103316+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:02:08.103316+0000","last_undegraded":"2024-09-10T16:02:08.103316+0000","last_fullsized":"2024-09-10T16:02:08.103316+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'11866","reported_seq":15086,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:33.412965+0000","last_change":"2024-09-10T16:03:29.605682+0000","last_active":"2024-09-10T16:03:33.412965+0000","last_peered":"2024-09-10T16:03:33.412965+0000","last_clean":"2024-09-10T15:59:14.115595+0000","last_became_active":"2024-09-10T15:59:16.244089+0000","last_became_peered":"2024-09-10T15:59:16.244089+0000","last_unstale":"2024-09-10T16:03:33.412965+0000","last_undegraded":"2024-09-10T16:03:29.603815+0000","last_fullsized":"2024-09-10T15:59:15.774437+0000","mapping_epoch":42,"log_start":"36'1800","ondisk_log_start":"36'1800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35766272,"num_objects":8732,"num_object_clones":0,"num_object_copies":17464,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":25,"num_objects_misplaced":2640,"num_objects_unfound":0,"num_objects_dirty":8732,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8256,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1538,"num_bytes_recovered":6299648,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6067},{"shards":"0,2","objects":2640}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103831+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:02:08.103831+0000","last_peered":"2024-09-10T16:02:08.103831+0000","last_clean":"2024-09-10T16:02:08.103831+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:02:08.103831+0000","last_undegraded":"2024-09-10T16:02:08.103831+0000","last_fullsized":"2024-09-10T16:02:08.103831+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'12242","reported_seq":12384,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:33.396430+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:03:33.396430+0000","last_peered":"2024-09-10T16:03:33.396430+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:03:33.396430+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"36'2200","ondisk_log_start":"36'2200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36069376,"num_objects":8806,"num_object_clones":0,"num_object_copies":17612,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8806,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8497,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4435},{"shards":"0,2","objects":2655}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104463+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:02:08.104463+0000","last_peered":"2024-09-10T16:02:08.104463+0000","last_clean":"2024-09-10T16:02:08.104463+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:02:08.104463+0000","last_undegraded":"2024-09-10T16:02:08.104463+0000","last_fullsized":"2024-09-10T16:02:08.104463+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104410+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:02:08.104410+0000","last_peered":"2024-09-10T16:02:08.104410+0000","last_clean":"2024-09-10T16:02:08.104410+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:02:08.104410+0000","last_undegraded":"2024-09-10T16:02:08.104410+0000","last_fullsized":"2024-09-10T16:02:08.104410+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'12269","reported_seq":12413,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:33.335844+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:03:33.335844+0000","last_peered":"2024-09-10T16:03:33.335844+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:03:33.335844+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"36'2200","ondisk_log_start":"36'2200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36574150,"num_objects":8931,"num_object_clones":0,"num_object_copies":17862,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8931,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8432,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4608},{"shards":"0,2","objects":2659}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":578159350,"num_objects":141162,"num_object_clones":0,"num_object_copies":282324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42936,"num_objects_misplaced":2640,"num_objects_unfound":0,"num_objects_dirty":141162,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":150944,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5810,"num_bytes_recovered":23793664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1247502336,"data_stored":1247342446,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160970,"ondisk_log_size":160970,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986242,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":727968,"kb_used_data":421380,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":93643872,"statfs":{"total":96636764160,"available":95891324928,"internally_reserved":0,"allocated":431493120,"data_stored":431200887,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":313913002},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,3,0,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659188,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":569048,"kb_used_data":306888,"kb_used_omap":4,"kb_used_meta":262139,"kb_avail":93802792,"statfs":{"total":96636764160,"available":96054059008,"internally_reserved":0,"allocated":314253312,"data_stored":313913646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":268431022},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,0,3,5,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182471,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":865996,"kb_used_data":491444,"kb_used_omap":4,"kb_used_meta":374523,"kb_avail":93505844,"statfs":{"total":96636764160,"available":95749984256,"internally_reserved":0,"allocated":503238656,"data_stored":502946418,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":383512234},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,3,3,5,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":502636544,"data_stored":502600414,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":313843712,"data_stored":313760154,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":431022080,"data_stored":430981878,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:03:37.154 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:03:37.716 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:03:37.718 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:03:37.742 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":229,"stamp":"2024-09-10T16:03:37.642304+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":574878473,"num_objects":140363,"num_object_clones":0,"num_object_copies":280726,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42936,"num_objects_misplaced":2640,"num_objects_unfound":0,"num_objects_dirty":140363,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":151747,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5812,"num_bytes_recovered":23793683,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160874,"ondisk_log_size":160874,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2166508,"kb_used_data":1214900,"kb_used_omap":12,"kb_used_meta":951539,"kb_avail":280949012,"statfs":{"total":289910292480,"available":287691788288,"internally_reserved":0,"allocated":1244057600,"data_stored":1243133463,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":974375938},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,3,8,4,10,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18726912,"num_objects":-4572,"num_object_clones":0,"num_object_copies":-9144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":25,"num_objects_misplaced":-219,"num_objects_unfound":0,"num_objects_dirty":-4572,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4572,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":194,"num_bytes_recovered":794624,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004171"},"pg_stats":[{"pgid":"2.f","version":"45'11951","reported_seq":12026,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:33.282541+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:03:33.282541+0000","last_peered":"2024-09-10T16:03:33.282541+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:03:33.282541+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"36'1900","ondisk_log_start":"36'1900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35721216,"num_objects":8721,"num_object_clones":0,"num_object_copies":17442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8721,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11951,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4462},{"shards":"0,2","objects":2643}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'12133","reported_seq":12245,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:03:34.998768+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:03:34.998768+0000","last_peered":"2024-09-10T16:03:34.998768+0000","last_clean":"2024-09-10T16:03:34.998768+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:03:34.998768+0000","last_undegraded":"2024-09-10T16:03:34.998768+0000","last_fullsized":"2024-09-10T16:03:34.998768+0000","mapping_epoch":35,"log_start":"36'2100","ondisk_log_start":"36'2100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35681222,"num_objects":8713,"num_object_clones":0,"num_object_copies":17426,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8713,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":10238,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'12112","reported_seq":20790,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:03:36.097545+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:03:36.097545+0000","last_peered":"2024-09-10T16:03:36.097545+0000","last_clean":"2024-09-10T16:03:36.097545+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:03:36.097545+0000","last_undegraded":"2024-09-10T16:03:36.097545+0000","last_fullsized":"2024-09-10T16:03:36.097545+0000","mapping_epoch":44,"log_start":"36'2100","ondisk_log_start":"36'2100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35573784,"num_objects":8686,"num_object_clones":0,"num_object_copies":17372,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8686,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10116,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'12056","reported_seq":12137,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:03:34.865598+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:03:34.865598+0000","last_peered":"2024-09-10T16:03:34.865598+0000","last_clean":"2024-09-10T16:03:34.865598+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:03:34.865598+0000","last_undegraded":"2024-09-10T16:03:34.865598+0000","last_fullsized":"2024-09-10T16:03:34.865598+0000","mapping_epoch":35,"log_start":"36'2000","ondisk_log_start":"36'2000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35241984,"num_objects":8604,"num_object_clones":0,"num_object_copies":17208,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8604,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12056,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'12087","reported_seq":12217,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:03:33.391384+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:03:33.391384+0000","last_peered":"2024-09-10T16:03:33.391384+0000","last_clean":"2024-09-10T16:03:33.391384+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:03:33.391384+0000","last_undegraded":"2024-09-10T16:03:33.391384+0000","last_fullsized":"2024-09-10T16:03:33.391384+0000","mapping_epoch":34,"log_start":"36'2000","ondisk_log_start":"36'2000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36061667,"num_objects":8805,"num_object_clones":0,"num_object_copies":17610,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8805,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8798,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'12225","reported_seq":12363,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:33.359065+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:03:33.359065+0000","last_peered":"2024-09-10T16:03:33.359065+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:03:33.359065+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"36'2200","ondisk_log_start":"36'2200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36192739,"num_objects":8837,"num_object_clones":0,"num_object_copies":17674,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8837,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8837,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4466},{"shards":"0,2","objects":2679}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'12195","reported_seq":12333,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:33.382997+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:03:33.382997+0000","last_peered":"2024-09-10T16:03:33.382997+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:03:33.382997+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"36'2100","ondisk_log_start":"36'2100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36402118,"num_objects":8889,"num_object_clones":0,"num_object_copies":17778,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8889,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":8694,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4652},{"shards":"0,2","objects":2586}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'12324","reported_seq":12454,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:03:33.345736+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:03:33.345736+0000","last_peered":"2024-09-10T16:03:33.345736+0000","last_clean":"2024-09-10T16:03:33.345736+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:03:33.345736+0000","last_undegraded":"2024-09-10T16:03:33.345736+0000","last_fullsized":"2024-09-10T16:03:33.345736+0000","mapping_epoch":34,"log_start":"36'2300","ondisk_log_start":"36'2300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36823040,"num_objects":8990,"num_object_clones":0,"num_object_copies":17980,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8990,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9171,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104261+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:02:08.104261+0000","last_peered":"2024-09-10T16:02:08.104261+0000","last_clean":"2024-09-10T16:02:08.104261+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:02:08.104261+0000","last_undegraded":"2024-09-10T16:02:08.104261+0000","last_fullsized":"2024-09-10T16:02:08.104261+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'11909","reported_seq":12049,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:34.993098+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:03:34.993098+0000","last_peered":"2024-09-10T16:03:34.993098+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:03:34.993098+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"36'1900","ondisk_log_start":"36'1900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35013091,"num_objects":8549,"num_object_clones":0,"num_object_copies":17098,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8549,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9728,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2446},{"shards":"2","objects":4426}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103774+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:02:08.103774+0000","last_peered":"2024-09-10T16:02:08.103774+0000","last_clean":"2024-09-10T16:02:08.103774+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:02:08.103774+0000","last_undegraded":"2024-09-10T16:02:08.103774+0000","last_fullsized":"2024-09-10T16:02:08.103774+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'12100","reported_seq":12231,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:33.241741+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:03:33.241741+0000","last_peered":"2024-09-10T16:03:33.241741+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:03:33.241741+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"36'2000","ondisk_log_start":"36'2000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36085760,"num_objects":8810,"num_object_clones":0,"num_object_copies":17620,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8810,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8604,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4555},{"shards":"0,2","objects":2614}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":104,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.105055+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:02:08.105055+0000","last_peered":"2024-09-10T16:02:08.105055+0000","last_clean":"2024-09-10T16:02:08.105055+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:02:08.105055+0000","last_undegraded":"2024-09-10T16:02:08.105055+0000","last_fullsized":"2024-09-10T16:02:08.105055+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'12124","reported_seq":12259,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:34.856158+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:03:34.856158+0000","last_peered":"2024-09-10T16:03:34.856158+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:03:34.856158+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"36'2100","ondisk_log_start":"36'2100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35459555,"num_objects":8658,"num_object_clones":0,"num_object_copies":17316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8658,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9804,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2573},{"shards":"2","objects":4354}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104617+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:02:08.104617+0000","last_peered":"2024-09-10T16:02:08.104617+0000","last_clean":"2024-09-10T16:02:08.104617+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:02:08.104617+0000","last_undegraded":"2024-09-10T16:02:08.104617+0000","last_fullsized":"2024-09-10T16:02:08.104617+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'12292","reported_seq":12421,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:34.990959+0000","last_change":"2024-09-10T15:59:16.174113+0000","last_active":"2024-09-10T16:03:34.990959+0000","last_peered":"2024-09-10T16:03:34.990959+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:03:34.990959+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"36'2200","ondisk_log_start":"36'2200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36364288,"num_objects":8878,"num_object_clones":0,"num_object_copies":17756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4354,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8878,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9846,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2648},{"shards":"2","objects":4526}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'12086","reported_seq":12221,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:33.155068+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:03:33.155068+0000","last_peered":"2024-09-10T16:03:33.155068+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:03:33.155068+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"36'2000","ondisk_log_start":"36'2000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35848192,"num_objects":8752,"num_object_clones":0,"num_object_copies":17504,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8752,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8717,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4448},{"shards":"0,2","objects":2638}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103316+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:02:08.103316+0000","last_peered":"2024-09-10T16:02:08.103316+0000","last_clean":"2024-09-10T16:02:08.103316+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:02:08.103316+0000","last_undegraded":"2024-09-10T16:02:08.103316+0000","last_fullsized":"2024-09-10T16:02:08.103316+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'11866","reported_seq":15086,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:33.412965+0000","last_change":"2024-09-10T16:03:29.605682+0000","last_active":"2024-09-10T16:03:33.412965+0000","last_peered":"2024-09-10T16:03:33.412965+0000","last_clean":"2024-09-10T15:59:14.115595+0000","last_became_active":"2024-09-10T15:59:16.244089+0000","last_became_peered":"2024-09-10T15:59:16.244089+0000","last_unstale":"2024-09-10T16:03:33.412965+0000","last_undegraded":"2024-09-10T16:03:29.603815+0000","last_fullsized":"2024-09-10T15:59:15.774437+0000","mapping_epoch":42,"log_start":"36'1800","ondisk_log_start":"36'1800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35766272,"num_objects":8732,"num_object_clones":0,"num_object_copies":17464,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":25,"num_objects_misplaced":2640,"num_objects_unfound":0,"num_objects_dirty":8732,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8256,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1538,"num_bytes_recovered":6299648,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6067},{"shards":"0,2","objects":2640}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103831+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:02:08.103831+0000","last_peered":"2024-09-10T16:02:08.103831+0000","last_clean":"2024-09-10T16:02:08.103831+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:02:08.103831+0000","last_undegraded":"2024-09-10T16:02:08.103831+0000","last_fullsized":"2024-09-10T16:02:08.103831+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'12242","reported_seq":12384,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:33.396430+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:03:33.396430+0000","last_peered":"2024-09-10T16:03:33.396430+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:03:33.396430+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"36'2200","ondisk_log_start":"36'2200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36069376,"num_objects":8806,"num_object_clones":0,"num_object_copies":17612,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8806,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8497,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4435},{"shards":"0,2","objects":2655}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104463+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:02:08.104463+0000","last_peered":"2024-09-10T16:02:08.104463+0000","last_clean":"2024-09-10T16:02:08.104463+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:02:08.104463+0000","last_undegraded":"2024-09-10T16:02:08.104463+0000","last_fullsized":"2024-09-10T16:02:08.104463+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104448+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:02:08.104448+0000","last_peered":"2024-09-10T16:02:08.104448+0000","last_clean":"2024-09-10T16:02:08.104448+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:02:08.104448+0000","last_undegraded":"2024-09-10T16:02:08.104448+0000","last_fullsized":"2024-09-10T16:02:08.104448+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'12269","reported_seq":12413,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:33.335844+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:03:33.335844+0000","last_peered":"2024-09-10T16:03:33.335844+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:03:33.335844+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"36'2200","ondisk_log_start":"36'2200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36574150,"num_objects":8931,"num_object_clones":0,"num_object_copies":17862,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8931,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8432,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4608},{"shards":"0,2","objects":2659}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":574878454,"num_objects":140361,"num_object_clones":0,"num_object_copies":280722,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42936,"num_objects_misplaced":2640,"num_objects_unfound":0,"num_objects_dirty":140361,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":151745,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5810,"num_bytes_recovered":23793664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1242558464,"data_stored":1242398574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160871,"ondisk_log_size":160871,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986243,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":727784,"kb_used_data":417048,"kb_used_omap":4,"kb_used_meta":310715,"kb_avail":93644056,"statfs":{"total":96636764160,"available":95891513344,"internally_reserved":0,"allocated":427057152,"data_stored":426764919,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":318172842},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659189,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":572728,"kb_used_data":306408,"kb_used_omap":4,"kb_used_meta":266299,"kb_avail":93799112,"statfs":{"total":96636764160,"available":96050290688,"internally_reserved":0,"allocated":313761792,"data_stored":313422126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":272690862},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,4,1,5,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182471,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":865996,"kb_used_data":491444,"kb_used_omap":4,"kb_used_meta":374523,"kb_avail":93505844,"statfs":{"total":96636764160,"available":95749984256,"internally_reserved":0,"allocated":503238656,"data_stored":502946418,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":383512234},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,3,3,5,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":502636544,"data_stored":502600414,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":313298944,"data_stored":313215386,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":426622976,"data_stored":426582774,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:03:37.743 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:03:38.297 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:03:38.299 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:03:38.319 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":229,"stamp":"2024-09-10T16:03:37.642304+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":574878473,"num_objects":140363,"num_object_clones":0,"num_object_copies":280726,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42936,"num_objects_misplaced":2640,"num_objects_unfound":0,"num_objects_dirty":140363,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":151747,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5812,"num_bytes_recovered":23793683,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160874,"ondisk_log_size":160874,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2166508,"kb_used_data":1214900,"kb_used_omap":12,"kb_used_meta":951539,"kb_avail":280949012,"statfs":{"total":289910292480,"available":287691788288,"internally_reserved":0,"allocated":1244057600,"data_stored":1243133463,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":974375938},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,3,8,4,10,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18726912,"num_objects":-4572,"num_object_clones":0,"num_object_copies":-9144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":25,"num_objects_misplaced":-219,"num_objects_unfound":0,"num_objects_dirty":-4572,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4572,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":194,"num_bytes_recovered":794624,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004171"},"pg_stats":[{"pgid":"2.f","version":"45'11951","reported_seq":12026,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:33.282541+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:03:33.282541+0000","last_peered":"2024-09-10T16:03:33.282541+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:03:33.282541+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"36'1900","ondisk_log_start":"36'1900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35721216,"num_objects":8721,"num_object_clones":0,"num_object_copies":17442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8721,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11951,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4462},{"shards":"0,2","objects":2643}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'12133","reported_seq":12245,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:03:34.998768+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:03:34.998768+0000","last_peered":"2024-09-10T16:03:34.998768+0000","last_clean":"2024-09-10T16:03:34.998768+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:03:34.998768+0000","last_undegraded":"2024-09-10T16:03:34.998768+0000","last_fullsized":"2024-09-10T16:03:34.998768+0000","mapping_epoch":35,"log_start":"36'2100","ondisk_log_start":"36'2100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35681222,"num_objects":8713,"num_object_clones":0,"num_object_copies":17426,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8713,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":10238,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'12112","reported_seq":20790,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:03:36.097545+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:03:36.097545+0000","last_peered":"2024-09-10T16:03:36.097545+0000","last_clean":"2024-09-10T16:03:36.097545+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:03:36.097545+0000","last_undegraded":"2024-09-10T16:03:36.097545+0000","last_fullsized":"2024-09-10T16:03:36.097545+0000","mapping_epoch":44,"log_start":"36'2100","ondisk_log_start":"36'2100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35573784,"num_objects":8686,"num_object_clones":0,"num_object_copies":17372,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8686,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10116,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'12056","reported_seq":12137,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:03:34.865598+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:03:34.865598+0000","last_peered":"2024-09-10T16:03:34.865598+0000","last_clean":"2024-09-10T16:03:34.865598+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:03:34.865598+0000","last_undegraded":"2024-09-10T16:03:34.865598+0000","last_fullsized":"2024-09-10T16:03:34.865598+0000","mapping_epoch":35,"log_start":"36'2000","ondisk_log_start":"36'2000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35241984,"num_objects":8604,"num_object_clones":0,"num_object_copies":17208,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8604,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12056,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'12087","reported_seq":12217,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:03:33.391384+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:03:33.391384+0000","last_peered":"2024-09-10T16:03:33.391384+0000","last_clean":"2024-09-10T16:03:33.391384+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:03:33.391384+0000","last_undegraded":"2024-09-10T16:03:33.391384+0000","last_fullsized":"2024-09-10T16:03:33.391384+0000","mapping_epoch":34,"log_start":"36'2000","ondisk_log_start":"36'2000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36061667,"num_objects":8805,"num_object_clones":0,"num_object_copies":17610,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8805,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8798,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'12225","reported_seq":12363,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:33.359065+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:03:33.359065+0000","last_peered":"2024-09-10T16:03:33.359065+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:03:33.359065+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"36'2200","ondisk_log_start":"36'2200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36192739,"num_objects":8837,"num_object_clones":0,"num_object_copies":17674,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8837,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8837,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4466},{"shards":"0,2","objects":2679}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'12195","reported_seq":12333,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:33.382997+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:03:33.382997+0000","last_peered":"2024-09-10T16:03:33.382997+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:03:33.382997+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"36'2100","ondisk_log_start":"36'2100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36402118,"num_objects":8889,"num_object_clones":0,"num_object_copies":17778,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8889,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":8694,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4652},{"shards":"0,2","objects":2586}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'12324","reported_seq":12454,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:03:33.345736+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:03:33.345736+0000","last_peered":"2024-09-10T16:03:33.345736+0000","last_clean":"2024-09-10T16:03:33.345736+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:03:33.345736+0000","last_undegraded":"2024-09-10T16:03:33.345736+0000","last_fullsized":"2024-09-10T16:03:33.345736+0000","mapping_epoch":34,"log_start":"36'2300","ondisk_log_start":"36'2300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36823040,"num_objects":8990,"num_object_clones":0,"num_object_copies":17980,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8990,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9171,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104261+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:02:08.104261+0000","last_peered":"2024-09-10T16:02:08.104261+0000","last_clean":"2024-09-10T16:02:08.104261+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:02:08.104261+0000","last_undegraded":"2024-09-10T16:02:08.104261+0000","last_fullsized":"2024-09-10T16:02:08.104261+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'11909","reported_seq":12049,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:34.993098+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:03:34.993098+0000","last_peered":"2024-09-10T16:03:34.993098+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:03:34.993098+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"36'1900","ondisk_log_start":"36'1900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35013091,"num_objects":8549,"num_object_clones":0,"num_object_copies":17098,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8549,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9728,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2446},{"shards":"2","objects":4426}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103774+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:02:08.103774+0000","last_peered":"2024-09-10T16:02:08.103774+0000","last_clean":"2024-09-10T16:02:08.103774+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:02:08.103774+0000","last_undegraded":"2024-09-10T16:02:08.103774+0000","last_fullsized":"2024-09-10T16:02:08.103774+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'12100","reported_seq":12231,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:33.241741+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:03:33.241741+0000","last_peered":"2024-09-10T16:03:33.241741+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:03:33.241741+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"36'2000","ondisk_log_start":"36'2000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36085760,"num_objects":8810,"num_object_clones":0,"num_object_copies":17620,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8810,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8604,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4555},{"shards":"0,2","objects":2614}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":104,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.105055+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:02:08.105055+0000","last_peered":"2024-09-10T16:02:08.105055+0000","last_clean":"2024-09-10T16:02:08.105055+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:02:08.105055+0000","last_undegraded":"2024-09-10T16:02:08.105055+0000","last_fullsized":"2024-09-10T16:02:08.105055+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'12124","reported_seq":12259,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:34.856158+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:03:34.856158+0000","last_peered":"2024-09-10T16:03:34.856158+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:03:34.856158+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"36'2100","ondisk_log_start":"36'2100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35459555,"num_objects":8658,"num_object_clones":0,"num_object_copies":17316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8658,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9804,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2573},{"shards":"2","objects":4354}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104617+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:02:08.104617+0000","last_peered":"2024-09-10T16:02:08.104617+0000","last_clean":"2024-09-10T16:02:08.104617+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:02:08.104617+0000","last_undegraded":"2024-09-10T16:02:08.104617+0000","last_fullsized":"2024-09-10T16:02:08.104617+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'12292","reported_seq":12421,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:34.990959+0000","last_change":"2024-09-10T15:59:16.174113+0000","last_active":"2024-09-10T16:03:34.990959+0000","last_peered":"2024-09-10T16:03:34.990959+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:03:34.990959+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"36'2200","ondisk_log_start":"36'2200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36364288,"num_objects":8878,"num_object_clones":0,"num_object_copies":17756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4354,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8878,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9846,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2648},{"shards":"2","objects":4526}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'12086","reported_seq":12221,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:33.155068+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:03:33.155068+0000","last_peered":"2024-09-10T16:03:33.155068+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:03:33.155068+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"36'2000","ondisk_log_start":"36'2000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35848192,"num_objects":8752,"num_object_clones":0,"num_object_copies":17504,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8752,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8717,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4448},{"shards":"0,2","objects":2638}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103316+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:02:08.103316+0000","last_peered":"2024-09-10T16:02:08.103316+0000","last_clean":"2024-09-10T16:02:08.103316+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:02:08.103316+0000","last_undegraded":"2024-09-10T16:02:08.103316+0000","last_fullsized":"2024-09-10T16:02:08.103316+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'11866","reported_seq":15086,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:33.412965+0000","last_change":"2024-09-10T16:03:29.605682+0000","last_active":"2024-09-10T16:03:33.412965+0000","last_peered":"2024-09-10T16:03:33.412965+0000","last_clean":"2024-09-10T15:59:14.115595+0000","last_became_active":"2024-09-10T15:59:16.244089+0000","last_became_peered":"2024-09-10T15:59:16.244089+0000","last_unstale":"2024-09-10T16:03:33.412965+0000","last_undegraded":"2024-09-10T16:03:29.603815+0000","last_fullsized":"2024-09-10T15:59:15.774437+0000","mapping_epoch":42,"log_start":"36'1800","ondisk_log_start":"36'1800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35766272,"num_objects":8732,"num_object_clones":0,"num_object_copies":17464,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":25,"num_objects_misplaced":2640,"num_objects_unfound":0,"num_objects_dirty":8732,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8256,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1538,"num_bytes_recovered":6299648,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6067},{"shards":"0,2","objects":2640}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103831+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:02:08.103831+0000","last_peered":"2024-09-10T16:02:08.103831+0000","last_clean":"2024-09-10T16:02:08.103831+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:02:08.103831+0000","last_undegraded":"2024-09-10T16:02:08.103831+0000","last_fullsized":"2024-09-10T16:02:08.103831+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'12242","reported_seq":12384,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:33.396430+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:03:33.396430+0000","last_peered":"2024-09-10T16:03:33.396430+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:03:33.396430+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"36'2200","ondisk_log_start":"36'2200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36069376,"num_objects":8806,"num_object_clones":0,"num_object_copies":17612,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8806,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8497,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4435},{"shards":"0,2","objects":2655}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104463+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:02:08.104463+0000","last_peered":"2024-09-10T16:02:08.104463+0000","last_clean":"2024-09-10T16:02:08.104463+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:02:08.104463+0000","last_undegraded":"2024-09-10T16:02:08.104463+0000","last_fullsized":"2024-09-10T16:02:08.104463+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104448+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:02:08.104448+0000","last_peered":"2024-09-10T16:02:08.104448+0000","last_clean":"2024-09-10T16:02:08.104448+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:02:08.104448+0000","last_undegraded":"2024-09-10T16:02:08.104448+0000","last_fullsized":"2024-09-10T16:02:08.104448+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'12269","reported_seq":12413,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:33.335844+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:03:33.335844+0000","last_peered":"2024-09-10T16:03:33.335844+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:03:33.335844+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"36'2200","ondisk_log_start":"36'2200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36574150,"num_objects":8931,"num_object_clones":0,"num_object_copies":17862,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8931,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8432,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4608},{"shards":"0,2","objects":2659}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":574878454,"num_objects":140361,"num_object_clones":0,"num_object_copies":280722,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42936,"num_objects_misplaced":2640,"num_objects_unfound":0,"num_objects_dirty":140361,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":151745,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5810,"num_bytes_recovered":23793664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1242558464,"data_stored":1242398574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160871,"ondisk_log_size":160871,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986243,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":727784,"kb_used_data":417048,"kb_used_omap":4,"kb_used_meta":310715,"kb_avail":93644056,"statfs":{"total":96636764160,"available":95891513344,"internally_reserved":0,"allocated":427057152,"data_stored":426764919,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":318172842},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659189,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":572728,"kb_used_data":306408,"kb_used_omap":4,"kb_used_meta":266299,"kb_avail":93799112,"statfs":{"total":96636764160,"available":96050290688,"internally_reserved":0,"allocated":313761792,"data_stored":313422126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":272690862},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,4,1,5,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182471,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":865996,"kb_used_data":491444,"kb_used_omap":4,"kb_used_meta":374523,"kb_avail":93505844,"statfs":{"total":96636764160,"available":95749984256,"internally_reserved":0,"allocated":503238656,"data_stored":502946418,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":383512234},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,3,3,5,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":502636544,"data_stored":502600414,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":313298944,"data_stored":313215386,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":426622976,"data_stored":426582774,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:03:38.321 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-10T16:03:38.321 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "45'11951", 'reported_seq': 12026, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:03:33.282541+0000', 'last_change': '2024-09-10T15:59:16.281558+0000', 'last_active': '2024-09-10T16:03:33.282541+0000', 'last_peered': '2024-09-10T16:03:33.282541+0000', 'last_clean': '2024-09-10T15:59:14.132456+0000', 'last_became_active': '2024-09-10T15:59:16.274407+0000', 'last_became_peered': '2024-09-10T15:59:16.274407+0000', 'last_unstale': '2024-09-10T16:03:33.282541+0000', 'last_undegraded': '2024-09-10T15:59:15.851279+0000', 'last_fullsized': '2024-09-10T15:59:15.774356+0000', 'mapping_epoch': 42, 'log_start': "36'1900", 'ondisk_log_start': "36'1900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'objects_scrubbed': 0, 'log_size': 10051, 'log_dups_size': 0, 'ondisk_log_size': 10051, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35721216, 'num_objects': 8721, 'num_object_clones': 0, 'num_object_copies': 17442, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4259, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8721, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11951, 'num_write_kb': 41344, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4462}, {'shards': '0,2', 'objects': 2643}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:03:38.321 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-10T16:03:38.321 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "45'12225", 'reported_seq': 12363, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:03:33.359065+0000', 'last_change': '2024-09-10T15:59:16.308264+0000', 'last_active': '2024-09-10T16:03:33.359065+0000', 'last_peered': '2024-09-10T16:03:33.359065+0000', 'last_clean': '2024-09-10T15:59:14.111551+0000', 'last_became_active': '2024-09-10T15:59:16.300040+0000', 'last_became_peered': '2024-09-10T15:59:16.300040+0000', 'last_unstale': '2024-09-10T16:03:33.359065+0000', 'last_undegraded': '2024-09-10T15:59:15.878234+0000', 'last_fullsized': '2024-09-10T15:59:15.769328+0000', 'mapping_epoch': 42, 'log_start': "36'2200", 'ondisk_log_start': "36'2200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3442", 'last_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'objects_scrubbed': 3384, 'log_size': 10025, 'log_dups_size': 0, 'ondisk_log_size': 10025, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1292, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36192739, 'num_objects': 8837, 'num_object_clones': 0, 'num_object_copies': 17674, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4372, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8837, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 8837, 'num_write_kb': 28580, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4466}, {'shards': '0,2', 'objects': 2679}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:03:38.321 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-10T16:03:38.321 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "45'12195", 'reported_seq': 12333, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:03:33.382997+0000', 'last_change': '2024-09-10T15:59:16.331899+0000', 'last_active': '2024-09-10T16:03:33.382997+0000', 'last_peered': '2024-09-10T16:03:33.382997+0000', 'last_clean': '2024-09-10T15:59:14.139436+0000', 'last_became_active': '2024-09-10T15:59:16.318715+0000', 'last_became_peered': '2024-09-10T15:59:16.318715+0000', 'last_unstale': '2024-09-10T16:03:33.382997+0000', 'last_undegraded': '2024-09-10T15:59:15.877524+0000', 'last_fullsized': '2024-09-10T15:59:15.769042+0000', 'mapping_epoch': 42, 'log_start': "36'2100", 'ondisk_log_start': "36'2100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3537", 'last_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'objects_scrubbed': 3497, 'log_size': 10095, 'log_dups_size': 0, 'ondisk_log_size': 10095, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1224, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36402118, 'num_objects': 8889, 'num_object_clones': 0, 'num_object_copies': 17778, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4237, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8889, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 8694, 'num_write_kb': 28172, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4652}, {'shards': '0,2', 'objects': 2586}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:03:38.321 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-10T16:03:38.322 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "45'11909", 'reported_seq': 12049, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:03:34.993098+0000', 'last_change': '2024-09-10T15:59:16.175680+0000', 'last_active': '2024-09-10T16:03:34.993098+0000', 'last_peered': '2024-09-10T16:03:34.993098+0000', 'last_clean': '2024-09-10T15:59:14.088187+0000', 'last_became_active': '2024-09-10T15:59:16.165810+0000', 'last_became_peered': '2024-09-10T15:59:16.165810+0000', 'last_unstale': '2024-09-10T16:03:34.993098+0000', 'last_undegraded': '2024-09-10T15:59:15.822098+0000', 'last_fullsized': '2024-09-10T15:59:15.769123+0000', 'mapping_epoch': 42, 'log_start': "36'1900", 'ondisk_log_start': "36'1900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2218", 'last_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'objects_scrubbed': 2175, 'log_size': 10009, 'log_dups_size': 0, 'ondisk_log_size': 10009, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 825, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35013091, 'num_objects': 8549, 'num_object_clones': 0, 'num_object_copies': 17098, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4124, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8549, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 9728, 'num_write_kb': 32204, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2446}, {'shards': '2', 'objects': 4426}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:03:38.322 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-10T16:03:38.322 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "45'12100", 'reported_seq': 12231, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:03:33.241741+0000', 'last_change': '2024-09-10T15:59:16.277742+0000', 'last_active': '2024-09-10T16:03:33.241741+0000', 'last_peered': '2024-09-10T16:03:33.241741+0000', 'last_clean': '2024-09-10T15:59:14.129081+0000', 'last_became_active': '2024-09-10T15:59:16.269835+0000', 'last_became_peered': '2024-09-10T15:59:16.269835+0000', 'last_unstale': '2024-09-10T16:03:33.241741+0000', 'last_undegraded': '2024-09-10T15:59:15.860857+0000', 'last_fullsized': '2024-09-10T15:59:15.771410+0000', 'mapping_epoch': 42, 'log_start': "36'2000", 'ondisk_log_start': "36'2000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3557", 'last_scrub_stamp': '2024-09-10T15:59:02.493678+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:02.493678+0000', 'objects_scrubbed': 3488, 'log_size': 10100, 'log_dups_size': 0, 'ondisk_log_size': 10100, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1472, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36085760, 'num_objects': 8810, 'num_object_clones': 0, 'num_object_copies': 17620, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4257, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8810, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8604, 'num_write_kb': 27852, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4555}, {'shards': '0,2', 'objects': 2614}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:03:38.322 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-10T16:03:38.322 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "45'12124", 'reported_seq': 12259, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:03:34.856158+0000', 'last_change': '2024-09-10T15:59:16.284311+0000', 'last_active': '2024-09-10T16:03:34.856158+0000', 'last_peered': '2024-09-10T16:03:34.856158+0000', 'last_clean': '2024-09-10T15:59:14.116041+0000', 'last_became_active': '2024-09-10T15:59:16.274543+0000', 'last_became_peered': '2024-09-10T15:59:16.274543+0000', 'last_unstale': '2024-09-10T16:03:34.856158+0000', 'last_undegraded': '2024-09-10T15:59:15.812484+0000', 'last_fullsized': '2024-09-10T15:59:15.770885+0000', 'mapping_epoch': 42, 'log_start': "36'2100", 'ondisk_log_start': "36'2100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2356", 'last_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'objects_scrubbed': 2316, 'log_size': 10024, 'log_dups_size': 0, 'ondisk_log_size': 10024, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 783, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35459555, 'num_objects': 8658, 'num_object_clones': 0, 'num_object_copies': 17316, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8658, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 9804, 'num_write_kb': 32292, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2573}, {'shards': '2', 'objects': 4354}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:03:38.322 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-10T16:03:38.322 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "45'12292", 'reported_seq': 12421, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:03:34.990959+0000', 'last_change': '2024-09-10T15:59:16.174113+0000', 'last_active': '2024-09-10T16:03:34.990959+0000', 'last_peered': '2024-09-10T16:03:34.990959+0000', 'last_clean': '2024-09-10T15:59:14.110654+0000', 'last_became_active': '2024-09-10T15:59:16.164645+0000', 'last_became_peered': '2024-09-10T15:59:16.164645+0000', 'last_unstale': '2024-09-10T16:03:34.990959+0000', 'last_undegraded': '2024-09-10T15:59:15.803593+0000', 'last_fullsized': '2024-09-10T15:59:15.769083+0000', 'mapping_epoch': 42, 'log_start': "36'2200", 'ondisk_log_start': "36'2200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2486", 'last_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'objects_scrubbed': 2440, 'log_size': 10092, 'log_dups_size': 0, 'ondisk_log_size': 10092, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 792, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36364288, 'num_objects': 8878, 'num_object_clones': 0, 'num_object_copies': 17756, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4354, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8878, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9846, 'num_write_kb': 32568, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2648}, {'shards': '2', 'objects': 4526}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:03:38.322 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-10T16:03:38.322 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "45'12086", 'reported_seq': 12221, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:03:33.155068+0000', 'last_change': '2024-09-10T15:59:16.315641+0000', 'last_active': '2024-09-10T16:03:33.155068+0000', 'last_peered': '2024-09-10T16:03:33.155068+0000', 'last_clean': '2024-09-10T15:59:14.117001+0000', 'last_became_active': '2024-09-10T15:59:16.302245+0000', 'last_became_peered': '2024-09-10T15:59:16.302245+0000', 'last_unstale': '2024-09-10T16:03:33.155068+0000', 'last_undegraded': '2024-09-10T15:59:15.845012+0000', 'last_fullsized': '2024-09-10T15:59:15.770337+0000', 'mapping_epoch': 42, 'log_start': "36'2000", 'ondisk_log_start': "36'2000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3404", 'last_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'objects_scrubbed': 3367, 'log_size': 10086, 'log_dups_size': 0, 'ondisk_log_size': 10086, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 983, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35848192, 'num_objects': 8752, 'num_object_clones': 0, 'num_object_copies': 17504, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8752, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8717, 'num_write_kb': 28204, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4448}, {'shards': '0,2', 'objects': 2638}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:03:38.322 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-10T16:03:38.322 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "45'11866", 'reported_seq': 15086, 'reported_epoch': 45, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:03:33.412965+0000', 'last_change': '2024-09-10T16:03:29.605682+0000', 'last_active': '2024-09-10T16:03:33.412965+0000', 'last_peered': '2024-09-10T16:03:33.412965+0000', 'last_clean': '2024-09-10T15:59:14.115595+0000', 'last_became_active': '2024-09-10T15:59:16.244089+0000', 'last_became_peered': '2024-09-10T15:59:16.244089+0000', 'last_unstale': '2024-09-10T16:03:33.412965+0000', 'last_undegraded': '2024-09-10T16:03:29.603815+0000', 'last_fullsized': '2024-09-10T15:59:15.774437+0000', 'mapping_epoch': 42, 'log_start': "36'1800", 'ondisk_log_start': "36'1800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3653", 'last_scrub_stamp': '2024-09-10T15:59:05.298468+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:05.298468+0000', 'objects_scrubbed': 3602, 'log_size': 10066, 'log_dups_size': 0, 'ondisk_log_size': 10066, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1289, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35766272, 'num_objects': 8732, 'num_object_clones': 0, 'num_object_copies': 17464, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 25, 'num_objects_misplaced': 2640, 'num_objects_unfound': 0, 'num_objects_dirty': 8732, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8256, 'num_write_kb': 26772, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1538, 'num_bytes_recovered': 6299648, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6067}, {'shards': '0,2', 'objects': 2640}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:03:38.323 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-10T16:03:38.323 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "45'12242", 'reported_seq': 12384, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:03:33.396430+0000', 'last_change': '2024-09-10T15:59:16.311291+0000', 'last_active': '2024-09-10T16:03:33.396430+0000', 'last_peered': '2024-09-10T16:03:33.396430+0000', 'last_clean': '2024-09-10T15:59:14.116363+0000', 'last_became_active': '2024-09-10T15:59:16.301244+0000', 'last_became_peered': '2024-09-10T15:59:16.301244+0000', 'last_unstale': '2024-09-10T16:03:33.396430+0000', 'last_undegraded': '2024-09-10T15:59:15.883434+0000', 'last_fullsized': '2024-09-10T15:59:15.772756+0000', 'mapping_epoch': 42, 'log_start': "36'2200", 'ondisk_log_start': "36'2200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3789", 'last_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'objects_scrubbed': 3741, 'log_size': 10042, 'log_dups_size': 0, 'ondisk_log_size': 10042, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1440, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36069376, 'num_objects': 8806, 'num_object_clones': 0, 'num_object_copies': 17612, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4373, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8806, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8497, 'num_write_kb': 27124, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4435}, {'shards': '0,2', 'objects': 2655}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:03:38.323 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-10T16:03:38.323 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "45'12269", 'reported_seq': 12413, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:03:33.335844+0000', 'last_change': '2024-09-10T15:59:16.313409+0000', 'last_active': '2024-09-10T16:03:33.335844+0000', 'last_peered': '2024-09-10T16:03:33.335844+0000', 'last_clean': '2024-09-10T15:59:14.129780+0000', 'last_became_active': '2024-09-10T15:59:16.300932+0000', 'last_became_peered': '2024-09-10T15:59:16.300932+0000', 'last_unstale': '2024-09-10T16:03:33.335844+0000', 'last_undegraded': '2024-09-10T15:59:15.887823+0000', 'last_fullsized': '2024-09-10T15:59:15.768513+0000', 'mapping_epoch': 42, 'log_start': "36'2200", 'ondisk_log_start': "36'2200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3880", 'last_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'objects_scrubbed': 3827, 'log_size': 10069, 'log_dups_size': 0, 'ondisk_log_size': 10069, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1397, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36574150, 'num_objects': 8931, 'num_object_clones': 0, 'num_object_copies': 17862, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4325, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8931, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8432, 'num_write_kb': 27072, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4608}, {'shards': '0,2', 'objects': 2659}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:03:38.323 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T16:03:39.002 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:03:39.003 INFO:teuthology.orchestra.run.smithi115.stdout:{"fsid":"7effee29-df05-4351-9741-ee203564ed81","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 42936/280726 objects degraded (15.295%), 11 pgs degraded, 11 pgs undersized","count":22},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":383,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983954,"num_in_osds":3,"osd_in_since":1725983947,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":140363,"data_bytes":574878473,"bytes_used":2218504192,"bytes_avail":287691788288,"bytes_total":289910292480,"degraded_objects":42936,"degraded_total":280726,"degraded_ratio":0.15294628926426479,"misplaced_objects":2640,"misplaced_total":280726,"misplaced_ratio":0.0094041877132862647,"recovering_objects_per_sec":16,"recovering_bytes_per_sec":66195,"recovering_keys_per_sec":0,"num_objects_recovered":194,"num_bytes_recovered":794624,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":380},"fsmap":{"epoch":1,"btime":"2024-09-10T15:57:14:967120+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T16:00:13.560776+0000","services":{}},"progress_events":{}} 2024-09-10T16:03:39.025 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-10T16:03:39.026 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:03:39.579 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:03:39.580 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:03:39.600 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":229,"stamp":"2024-09-10T16:03:37.642304+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":574878473,"num_objects":140363,"num_object_clones":0,"num_object_copies":280726,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42936,"num_objects_misplaced":2640,"num_objects_unfound":0,"num_objects_dirty":140363,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":151747,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5812,"num_bytes_recovered":23793683,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160874,"ondisk_log_size":160874,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2166508,"kb_used_data":1214900,"kb_used_omap":12,"kb_used_meta":951539,"kb_avail":280949012,"statfs":{"total":289910292480,"available":287691788288,"internally_reserved":0,"allocated":1244057600,"data_stored":1243133463,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":974375938},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,3,8,4,10,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18726912,"num_objects":-4572,"num_object_clones":0,"num_object_copies":-9144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":25,"num_objects_misplaced":-219,"num_objects_unfound":0,"num_objects_dirty":-4572,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4572,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":194,"num_bytes_recovered":794624,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004171"},"pg_stats":[{"pgid":"2.f","version":"45'11951","reported_seq":12026,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:33.282541+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:03:33.282541+0000","last_peered":"2024-09-10T16:03:33.282541+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:03:33.282541+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"36'1900","ondisk_log_start":"36'1900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35721216,"num_objects":8721,"num_object_clones":0,"num_object_copies":17442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8721,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11951,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4462},{"shards":"0,2","objects":2643}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'12133","reported_seq":12245,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:03:34.998768+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:03:34.998768+0000","last_peered":"2024-09-10T16:03:34.998768+0000","last_clean":"2024-09-10T16:03:34.998768+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:03:34.998768+0000","last_undegraded":"2024-09-10T16:03:34.998768+0000","last_fullsized":"2024-09-10T16:03:34.998768+0000","mapping_epoch":35,"log_start":"36'2100","ondisk_log_start":"36'2100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35681222,"num_objects":8713,"num_object_clones":0,"num_object_copies":17426,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8713,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":10238,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'12112","reported_seq":20790,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:03:36.097545+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:03:36.097545+0000","last_peered":"2024-09-10T16:03:36.097545+0000","last_clean":"2024-09-10T16:03:36.097545+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:03:36.097545+0000","last_undegraded":"2024-09-10T16:03:36.097545+0000","last_fullsized":"2024-09-10T16:03:36.097545+0000","mapping_epoch":44,"log_start":"36'2100","ondisk_log_start":"36'2100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35573784,"num_objects":8686,"num_object_clones":0,"num_object_copies":17372,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8686,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10116,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'12056","reported_seq":12137,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:03:34.865598+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:03:34.865598+0000","last_peered":"2024-09-10T16:03:34.865598+0000","last_clean":"2024-09-10T16:03:34.865598+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:03:34.865598+0000","last_undegraded":"2024-09-10T16:03:34.865598+0000","last_fullsized":"2024-09-10T16:03:34.865598+0000","mapping_epoch":35,"log_start":"36'2000","ondisk_log_start":"36'2000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35241984,"num_objects":8604,"num_object_clones":0,"num_object_copies":17208,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8604,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12056,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'12087","reported_seq":12217,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:03:33.391384+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:03:33.391384+0000","last_peered":"2024-09-10T16:03:33.391384+0000","last_clean":"2024-09-10T16:03:33.391384+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:03:33.391384+0000","last_undegraded":"2024-09-10T16:03:33.391384+0000","last_fullsized":"2024-09-10T16:03:33.391384+0000","mapping_epoch":34,"log_start":"36'2000","ondisk_log_start":"36'2000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36061667,"num_objects":8805,"num_object_clones":0,"num_object_copies":17610,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8805,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8798,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'12225","reported_seq":12363,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:33.359065+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:03:33.359065+0000","last_peered":"2024-09-10T16:03:33.359065+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:03:33.359065+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"36'2200","ondisk_log_start":"36'2200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36192739,"num_objects":8837,"num_object_clones":0,"num_object_copies":17674,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8837,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8837,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4466},{"shards":"0,2","objects":2679}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'12195","reported_seq":12333,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:33.382997+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:03:33.382997+0000","last_peered":"2024-09-10T16:03:33.382997+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:03:33.382997+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"36'2100","ondisk_log_start":"36'2100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36402118,"num_objects":8889,"num_object_clones":0,"num_object_copies":17778,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8889,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":8694,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4652},{"shards":"0,2","objects":2586}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'12324","reported_seq":12454,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:03:33.345736+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:03:33.345736+0000","last_peered":"2024-09-10T16:03:33.345736+0000","last_clean":"2024-09-10T16:03:33.345736+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:03:33.345736+0000","last_undegraded":"2024-09-10T16:03:33.345736+0000","last_fullsized":"2024-09-10T16:03:33.345736+0000","mapping_epoch":34,"log_start":"36'2300","ondisk_log_start":"36'2300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36823040,"num_objects":8990,"num_object_clones":0,"num_object_copies":17980,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8990,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9171,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104261+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:02:08.104261+0000","last_peered":"2024-09-10T16:02:08.104261+0000","last_clean":"2024-09-10T16:02:08.104261+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:02:08.104261+0000","last_undegraded":"2024-09-10T16:02:08.104261+0000","last_fullsized":"2024-09-10T16:02:08.104261+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'11909","reported_seq":12049,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:34.993098+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:03:34.993098+0000","last_peered":"2024-09-10T16:03:34.993098+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:03:34.993098+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"36'1900","ondisk_log_start":"36'1900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35013091,"num_objects":8549,"num_object_clones":0,"num_object_copies":17098,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8549,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9728,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2446},{"shards":"2","objects":4426}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103774+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:02:08.103774+0000","last_peered":"2024-09-10T16:02:08.103774+0000","last_clean":"2024-09-10T16:02:08.103774+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:02:08.103774+0000","last_undegraded":"2024-09-10T16:02:08.103774+0000","last_fullsized":"2024-09-10T16:02:08.103774+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'12100","reported_seq":12231,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:33.241741+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:03:33.241741+0000","last_peered":"2024-09-10T16:03:33.241741+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:03:33.241741+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"36'2000","ondisk_log_start":"36'2000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36085760,"num_objects":8810,"num_object_clones":0,"num_object_copies":17620,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8810,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8604,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4555},{"shards":"0,2","objects":2614}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":104,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.105055+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:02:08.105055+0000","last_peered":"2024-09-10T16:02:08.105055+0000","last_clean":"2024-09-10T16:02:08.105055+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:02:08.105055+0000","last_undegraded":"2024-09-10T16:02:08.105055+0000","last_fullsized":"2024-09-10T16:02:08.105055+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'12124","reported_seq":12259,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:34.856158+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:03:34.856158+0000","last_peered":"2024-09-10T16:03:34.856158+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:03:34.856158+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"36'2100","ondisk_log_start":"36'2100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35459555,"num_objects":8658,"num_object_clones":0,"num_object_copies":17316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8658,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9804,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2573},{"shards":"2","objects":4354}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104617+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:02:08.104617+0000","last_peered":"2024-09-10T16:02:08.104617+0000","last_clean":"2024-09-10T16:02:08.104617+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:02:08.104617+0000","last_undegraded":"2024-09-10T16:02:08.104617+0000","last_fullsized":"2024-09-10T16:02:08.104617+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'12292","reported_seq":12421,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:34.990959+0000","last_change":"2024-09-10T15:59:16.174113+0000","last_active":"2024-09-10T16:03:34.990959+0000","last_peered":"2024-09-10T16:03:34.990959+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:03:34.990959+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"36'2200","ondisk_log_start":"36'2200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36364288,"num_objects":8878,"num_object_clones":0,"num_object_copies":17756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4354,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8878,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9846,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2648},{"shards":"2","objects":4526}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'12086","reported_seq":12221,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:33.155068+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:03:33.155068+0000","last_peered":"2024-09-10T16:03:33.155068+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:03:33.155068+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"36'2000","ondisk_log_start":"36'2000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35848192,"num_objects":8752,"num_object_clones":0,"num_object_copies":17504,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8752,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8717,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4448},{"shards":"0,2","objects":2638}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103316+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:02:08.103316+0000","last_peered":"2024-09-10T16:02:08.103316+0000","last_clean":"2024-09-10T16:02:08.103316+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:02:08.103316+0000","last_undegraded":"2024-09-10T16:02:08.103316+0000","last_fullsized":"2024-09-10T16:02:08.103316+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'11866","reported_seq":15086,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:33.412965+0000","last_change":"2024-09-10T16:03:29.605682+0000","last_active":"2024-09-10T16:03:33.412965+0000","last_peered":"2024-09-10T16:03:33.412965+0000","last_clean":"2024-09-10T15:59:14.115595+0000","last_became_active":"2024-09-10T15:59:16.244089+0000","last_became_peered":"2024-09-10T15:59:16.244089+0000","last_unstale":"2024-09-10T16:03:33.412965+0000","last_undegraded":"2024-09-10T16:03:29.603815+0000","last_fullsized":"2024-09-10T15:59:15.774437+0000","mapping_epoch":42,"log_start":"36'1800","ondisk_log_start":"36'1800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35766272,"num_objects":8732,"num_object_clones":0,"num_object_copies":17464,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":25,"num_objects_misplaced":2640,"num_objects_unfound":0,"num_objects_dirty":8732,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8256,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1538,"num_bytes_recovered":6299648,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6067},{"shards":"0,2","objects":2640}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103831+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:02:08.103831+0000","last_peered":"2024-09-10T16:02:08.103831+0000","last_clean":"2024-09-10T16:02:08.103831+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:02:08.103831+0000","last_undegraded":"2024-09-10T16:02:08.103831+0000","last_fullsized":"2024-09-10T16:02:08.103831+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'12242","reported_seq":12384,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:33.396430+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:03:33.396430+0000","last_peered":"2024-09-10T16:03:33.396430+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:03:33.396430+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"36'2200","ondisk_log_start":"36'2200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36069376,"num_objects":8806,"num_object_clones":0,"num_object_copies":17612,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8806,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8497,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4435},{"shards":"0,2","objects":2655}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104463+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:02:08.104463+0000","last_peered":"2024-09-10T16:02:08.104463+0000","last_clean":"2024-09-10T16:02:08.104463+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:02:08.104463+0000","last_undegraded":"2024-09-10T16:02:08.104463+0000","last_fullsized":"2024-09-10T16:02:08.104463+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104448+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:02:08.104448+0000","last_peered":"2024-09-10T16:02:08.104448+0000","last_clean":"2024-09-10T16:02:08.104448+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:02:08.104448+0000","last_undegraded":"2024-09-10T16:02:08.104448+0000","last_fullsized":"2024-09-10T16:02:08.104448+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'12269","reported_seq":12413,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:33.335844+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:03:33.335844+0000","last_peered":"2024-09-10T16:03:33.335844+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:03:33.335844+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"36'2200","ondisk_log_start":"36'2200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36574150,"num_objects":8931,"num_object_clones":0,"num_object_copies":17862,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8931,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8432,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4608},{"shards":"0,2","objects":2659}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":574878454,"num_objects":140361,"num_object_clones":0,"num_object_copies":280722,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42936,"num_objects_misplaced":2640,"num_objects_unfound":0,"num_objects_dirty":140361,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":151745,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5810,"num_bytes_recovered":23793664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1242558464,"data_stored":1242398574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160871,"ondisk_log_size":160871,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986243,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":727784,"kb_used_data":417048,"kb_used_omap":4,"kb_used_meta":310715,"kb_avail":93644056,"statfs":{"total":96636764160,"available":95891513344,"internally_reserved":0,"allocated":427057152,"data_stored":426764919,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":318172842},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659189,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":572728,"kb_used_data":306408,"kb_used_omap":4,"kb_used_meta":266299,"kb_avail":93799112,"statfs":{"total":96636764160,"available":96050290688,"internally_reserved":0,"allocated":313761792,"data_stored":313422126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":272690862},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,4,1,5,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182471,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":865996,"kb_used_data":491444,"kb_used_omap":4,"kb_used_meta":374523,"kb_avail":93505844,"statfs":{"total":96636764160,"available":95749984256,"internally_reserved":0,"allocated":503238656,"data_stored":502946418,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":383512234},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,3,3,5,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":502636544,"data_stored":502600414,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":313298944,"data_stored":313215386,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":426622976,"data_stored":426582774,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:03:42.603 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:03:43.159 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:03:43.160 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:03:43.179 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":231,"stamp":"2024-09-10T16:03:41.643279+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":565732105,"num_objects":138130,"num_object_clones":0,"num_object_copies":276260,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43006,"num_objects_misplaced":2477,"num_objects_unfound":0,"num_objects_dirty":138130,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":153980,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5905,"num_bytes_recovered":24174611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160907,"ondisk_log_size":160907,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2150936,"kb_used_data":1203496,"kb_used_omap":12,"kb_used_meta":947379,"kb_avail":280964584,"statfs":{"total":289910292480,"available":287707734016,"internally_reserved":0,"allocated":1232379904,"data_stored":1231455767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":970116098},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,3,4,5,7,1,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-21544960,"num_objects":-5260,"num_object_clones":0,"num_object_copies":-10520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":95,"num_objects_misplaced":-287,"num_objects_unfound":0,"num_objects_dirty":-5260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5260,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":192,"num_bytes_recovered":786432,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003418"},"pg_stats":[{"pgid":"2.f","version":"45'12100","reported_seq":12175,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:38.299106+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:03:38.299106+0000","last_peered":"2024-09-10T16:03:38.299106+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:03:38.299106+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"36'2000","ondisk_log_start":"36'2000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35110912,"num_objects":8572,"num_object_clones":0,"num_object_copies":17144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8572,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12100,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4313},{"shards":"0,2","objects":2494}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'12268","reported_seq":12381,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:03:40.003762+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:03:40.003762+0000","last_peered":"2024-09-10T16:03:40.003762+0000","last_clean":"2024-09-10T16:03:40.003762+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:03:40.003762+0000","last_undegraded":"2024-09-10T16:03:40.003762+0000","last_fullsized":"2024-09-10T16:03:40.003762+0000","mapping_epoch":35,"log_start":"36'2200","ondisk_log_start":"36'2200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35128262,"num_objects":8578,"num_object_clones":0,"num_object_copies":17156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8578,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":10373,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'12264","reported_seq":20941,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:03:41.197004+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:03:41.197004+0000","last_peered":"2024-09-10T16:03:41.197004+0000","last_clean":"2024-09-10T16:03:41.197004+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:03:41.197004+0000","last_undegraded":"2024-09-10T16:03:41.197004+0000","last_fullsized":"2024-09-10T16:03:41.197004+0000","mapping_epoch":44,"log_start":"36'2200","ondisk_log_start":"36'2200","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34951192,"num_objects":8534,"num_object_clones":0,"num_object_copies":17068,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8534,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10268,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'12182","reported_seq":12263,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:03:39.989840+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:03:39.989840+0000","last_peered":"2024-09-10T16:03:39.989840+0000","last_clean":"2024-09-10T16:03:39.989840+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:03:39.989840+0000","last_undegraded":"2024-09-10T16:03:39.989840+0000","last_fullsized":"2024-09-10T16:03:39.989840+0000","mapping_epoch":35,"log_start":"36'2100","ondisk_log_start":"36'2100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34725888,"num_objects":8478,"num_object_clones":0,"num_object_copies":16956,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8478,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12182,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'12199","reported_seq":12329,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:03:38.298147+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:03:38.298147+0000","last_peered":"2024-09-10T16:03:38.298147+0000","last_clean":"2024-09-10T16:03:38.298147+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:03:38.298147+0000","last_undegraded":"2024-09-10T16:03:38.298147+0000","last_fullsized":"2024-09-10T16:03:38.298147+0000","mapping_epoch":34,"log_start":"36'2100","ondisk_log_start":"36'2100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35602915,"num_objects":8693,"num_object_clones":0,"num_object_copies":17386,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8693,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8910,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'12371","reported_seq":12509,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:38.402334+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:03:38.402334+0000","last_peered":"2024-09-10T16:03:38.402334+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:03:38.402334+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"36'2300","ondisk_log_start":"36'2300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35594723,"num_objects":8691,"num_object_clones":0,"num_object_copies":17382,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8691,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8983,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4320},{"shards":"0,2","objects":2533}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'12320","reported_seq":12458,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:38.378669+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:03:38.378669+0000","last_peered":"2024-09-10T16:03:38.378669+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:03:38.378669+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"36'2300","ondisk_log_start":"36'2300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35890118,"num_objects":8764,"num_object_clones":0,"num_object_copies":17528,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8764,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":8819,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4527},{"shards":"0,2","objects":2461}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'12472","reported_seq":12602,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:03:38.383514+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:03:38.383514+0000","last_peered":"2024-09-10T16:03:38.383514+0000","last_clean":"2024-09-10T16:03:38.383514+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:03:38.383514+0000","last_undegraded":"2024-09-10T16:03:38.383514+0000","last_fullsized":"2024-09-10T16:03:38.383514+0000","mapping_epoch":34,"log_start":"36'2400","ondisk_log_start":"36'2400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36216832,"num_objects":8842,"num_object_clones":0,"num_object_copies":17684,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8842,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9319,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104261+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:02:08.104261+0000","last_peered":"2024-09-10T16:02:08.104261+0000","last_clean":"2024-09-10T16:02:08.104261+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:02:08.104261+0000","last_undegraded":"2024-09-10T16:02:08.104261+0000","last_fullsized":"2024-09-10T16:02:08.104261+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'12045","reported_seq":12185,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:39.988151+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:03:39.988151+0000","last_peered":"2024-09-10T16:03:39.988151+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:03:39.988151+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"36'2000","ondisk_log_start":"36'2000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34456035,"num_objects":8413,"num_object_clones":0,"num_object_copies":16826,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8413,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9864,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2310},{"shards":"2","objects":4290}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103774+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:02:08.103774+0000","last_peered":"2024-09-10T16:02:08.103774+0000","last_clean":"2024-09-10T16:02:08.103774+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:02:08.103774+0000","last_undegraded":"2024-09-10T16:02:08.103774+0000","last_fullsized":"2024-09-10T16:02:08.103774+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'12240","reported_seq":12371,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:38.462415+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:03:38.462415+0000","last_peered":"2024-09-10T16:03:38.462415+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:03:38.462415+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"36'2200","ondisk_log_start":"36'2200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35512320,"num_objects":8670,"num_object_clones":0,"num_object_copies":17340,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8670,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8744,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4415},{"shards":"0,2","objects":2474}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":104,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.105055+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:02:08.105055+0000","last_peered":"2024-09-10T16:02:08.105055+0000","last_clean":"2024-09-10T16:02:08.105055+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:02:08.105055+0000","last_undegraded":"2024-09-10T16:02:08.105055+0000","last_fullsized":"2024-09-10T16:02:08.105055+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'12266","reported_seq":12401,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:39.939921+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:03:39.939921+0000","last_peered":"2024-09-10T16:03:39.939921+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:03:39.939921+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"36'2200","ondisk_log_start":"36'2200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34877923,"num_objects":8516,"num_object_clones":0,"num_object_copies":17032,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8516,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9946,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2431},{"shards":"2","objects":4212}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104617+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:02:08.104617+0000","last_peered":"2024-09-10T16:02:08.104617+0000","last_clean":"2024-09-10T16:02:08.104617+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:02:08.104617+0000","last_undegraded":"2024-09-10T16:02:08.104617+0000","last_fullsized":"2024-09-10T16:02:08.104617+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'12433","reported_seq":12561,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:40.032701+0000","last_change":"2024-09-10T15:59:16.174113+0000","last_active":"2024-09-10T16:03:40.032701+0000","last_peered":"2024-09-10T16:03:40.032701+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:03:40.032701+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"36'2400","ondisk_log_start":"36'2400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35786752,"num_objects":8737,"num_object_clones":0,"num_object_copies":17474,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4354,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8737,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9987,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2507},{"shards":"2","objects":4385}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'12215","reported_seq":12350,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:38.410635+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:03:38.410635+0000","last_peered":"2024-09-10T16:03:38.410635+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:03:38.410635+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"36'2200","ondisk_log_start":"36'2200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35319808,"num_objects":8623,"num_object_clones":0,"num_object_copies":17246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8623,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8846,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4319},{"shards":"0,2","objects":2509}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103316+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:02:08.103316+0000","last_peered":"2024-09-10T16:02:08.103316+0000","last_clean":"2024-09-10T16:02:08.103316+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:02:08.103316+0000","last_undegraded":"2024-09-10T16:02:08.103316+0000","last_fullsized":"2024-09-10T16:02:08.103316+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'12025","reported_seq":15431,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:38.479610+0000","last_change":"2024-09-10T16:03:29.605682+0000","last_active":"2024-09-10T16:03:38.479610+0000","last_peered":"2024-09-10T16:03:38.479610+0000","last_clean":"2024-09-10T15:59:14.115595+0000","last_became_active":"2024-09-10T15:59:16.244089+0000","last_became_peered":"2024-09-10T15:59:16.244089+0000","last_unstale":"2024-09-10T16:03:38.479610+0000","last_undegraded":"2024-09-10T16:03:29.603815+0000","last_fullsized":"2024-09-10T15:59:15.774437+0000","mapping_epoch":42,"log_start":"36'2000","ondisk_log_start":"36'2000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35115008,"num_objects":8573,"num_object_clones":0,"num_object_copies":17146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":95,"num_objects_misplaced":2477,"num_objects_unfound":0,"num_objects_dirty":8573,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8415,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1631,"num_bytes_recovered":6680576,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6001},{"shards":"0,2","objects":2477}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103831+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:02:08.103831+0000","last_peered":"2024-09-10T16:02:08.103831+0000","last_clean":"2024-09-10T16:02:08.103831+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:02:08.103831+0000","last_undegraded":"2024-09-10T16:02:08.103831+0000","last_fullsized":"2024-09-10T16:02:08.103831+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'12386","reported_seq":12524,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:38.450801+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:03:38.450801+0000","last_peered":"2024-09-10T16:03:38.450801+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:03:38.450801+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"36'2300","ondisk_log_start":"36'2300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35479552,"num_objects":8662,"num_object_clones":0,"num_object_copies":17324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8662,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8641,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4291},{"shards":"0,2","objects":2511}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104463+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:02:08.104463+0000","last_peered":"2024-09-10T16:02:08.104463+0000","last_clean":"2024-09-10T16:02:08.104463+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:02:08.104463+0000","last_undegraded":"2024-09-10T16:02:08.104463+0000","last_fullsized":"2024-09-10T16:02:08.104463+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104448+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:02:08.104448+0000","last_peered":"2024-09-10T16:02:08.104448+0000","last_clean":"2024-09-10T16:02:08.104448+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:02:08.104448+0000","last_undegraded":"2024-09-10T16:02:08.104448+0000","last_fullsized":"2024-09-10T16:02:08.104448+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'12418","reported_seq":12563,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:38.416413+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:03:38.416413+0000","last_peered":"2024-09-10T16:03:38.416413+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:03:38.416413+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"36'2400","ondisk_log_start":"36'2400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35963846,"num_objects":8782,"num_object_clones":0,"num_object_copies":17564,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8782,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8581,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4459},{"shards":"0,2","objects":2510}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":565732086,"num_objects":138128,"num_object_clones":0,"num_object_copies":276256,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43006,"num_objects_misplaced":2477,"num_objects_unfound":0,"num_objects_dirty":138128,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":153978,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5903,"num_bytes_recovered":24174592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1230700544,"data_stored":1230540654,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160904,"ondisk_log_size":160904,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986244,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":711084,"kb_used_data":412828,"kb_used_omap":4,"kb_used_meta":298235,"kb_avail":93660756,"statfs":{"total":96636764160,"available":95908614144,"internally_reserved":0,"allocated":422735872,"data_stored":422443639,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":305393322},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2],"upper_bound":16},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659190,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":576324,"kb_used_data":305844,"kb_used_omap":4,"kb_used_meta":270459,"kb_avail":93795516,"statfs":{"total":96636764160,"available":96046608384,"internally_reserved":0,"allocated":313184256,"data_stored":312844590,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":276950702},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,2,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182472,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":863528,"kb_used_data":484824,"kb_used_omap":4,"kb_used_meta":378683,"kb_avail":93508312,"statfs":{"total":96636764160,"available":95752511488,"internally_reserved":0,"allocated":496459776,"data_stored":496167538,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":387772074},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,1,3,6,1,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":495800320,"data_stored":495764190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":312672256,"data_stored":312588698,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":422227968,"data_stored":422187766,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:03:43.181 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:03:43.745 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:03:43.745 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:03:43.772 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":232,"stamp":"2024-09-10T16:03:43.643771+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":560005897,"num_objects":136732,"num_object_clones":0,"num_object_copies":273464,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43050,"num_objects_misplaced":2331,"num_objects_unfound":0,"num_objects_dirty":136732,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":155378,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6007,"num_bytes_recovered":24592403,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160805,"ondisk_log_size":160805,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2148572,"kb_used_data":1196968,"kb_used_omap":12,"kb_used_meta":951539,"kb_avail":280966948,"statfs":{"total":289910292480,"available":287710154752,"internally_reserved":0,"allocated":1225695232,"data_stored":1224771095,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":974375938},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,4,5,5,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-23855104,"num_objects":-5824,"num_object_clones":0,"num_object_copies":-11648,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":139,"num_objects_misplaced":-433,"num_objects_unfound":0,"num_objects_dirty":-5824,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5824,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":294,"num_bytes_recovered":1204224,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003279"},"pg_stats":[{"pgid":"2.f","version":"45'12235","reported_seq":12310,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:43.473199+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:03:43.473199+0000","last_peered":"2024-09-10T16:03:43.473199+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:03:43.473199+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"36'2200","ondisk_log_start":"36'2200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34557952,"num_objects":8437,"num_object_clones":0,"num_object_copies":16874,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8437,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12235,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4178},{"shards":"0,2","objects":2359}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'12268","reported_seq":12381,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:03:40.003762+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:03:40.003762+0000","last_peered":"2024-09-10T16:03:40.003762+0000","last_clean":"2024-09-10T16:03:40.003762+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:03:40.003762+0000","last_undegraded":"2024-09-10T16:03:40.003762+0000","last_fullsized":"2024-09-10T16:03:40.003762+0000","mapping_epoch":35,"log_start":"36'2200","ondisk_log_start":"36'2200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35128262,"num_objects":8578,"num_object_clones":0,"num_object_copies":17156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8578,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":10373,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'12264","reported_seq":20941,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:03:41.197004+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:03:41.197004+0000","last_peered":"2024-09-10T16:03:41.197004+0000","last_clean":"2024-09-10T16:03:41.197004+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:03:41.197004+0000","last_undegraded":"2024-09-10T16:03:41.197004+0000","last_fullsized":"2024-09-10T16:03:41.197004+0000","mapping_epoch":44,"log_start":"36'2200","ondisk_log_start":"36'2200","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34951192,"num_objects":8534,"num_object_clones":0,"num_object_copies":17068,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8534,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10268,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'12182","reported_seq":12263,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:03:39.989840+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:03:39.989840+0000","last_peered":"2024-09-10T16:03:39.989840+0000","last_clean":"2024-09-10T16:03:39.989840+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:03:39.989840+0000","last_undegraded":"2024-09-10T16:03:39.989840+0000","last_fullsized":"2024-09-10T16:03:39.989840+0000","mapping_epoch":35,"log_start":"36'2100","ondisk_log_start":"36'2100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34725888,"num_objects":8478,"num_object_clones":0,"num_object_copies":16956,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8478,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12182,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'12345","reported_seq":12475,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:03:43.516141+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:03:43.516141+0000","last_peered":"2024-09-10T16:03:43.516141+0000","last_clean":"2024-09-10T16:03:43.516141+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:03:43.516141+0000","last_undegraded":"2024-09-10T16:03:43.516141+0000","last_fullsized":"2024-09-10T16:03:43.516141+0000","mapping_epoch":34,"log_start":"36'2300","ondisk_log_start":"36'2300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35004899,"num_objects":8547,"num_object_clones":0,"num_object_copies":17094,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8547,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9056,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'12525","reported_seq":12663,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:43.484969+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:03:43.484969+0000","last_peered":"2024-09-10T16:03:43.484969+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:03:43.484969+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"36'2500","ondisk_log_start":"36'2500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34963939,"num_objects":8537,"num_object_clones":0,"num_object_copies":17074,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8537,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9137,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4166},{"shards":"0,2","objects":2379}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'12467","reported_seq":12605,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:43.260154+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:03:43.260154+0000","last_peered":"2024-09-10T16:03:43.260154+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:03:43.260154+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"36'2400","ondisk_log_start":"36'2400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35288006,"num_objects":8617,"num_object_clones":0,"num_object_copies":17234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8617,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":8966,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4380},{"shards":"0,2","objects":2314}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'12605","reported_seq":12735,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:03:43.423562+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:03:43.423562+0000","last_peered":"2024-09-10T16:03:43.423562+0000","last_clean":"2024-09-10T16:03:43.423562+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:03:43.423562+0000","last_undegraded":"2024-09-10T16:03:43.423562+0000","last_fullsized":"2024-09-10T16:03:43.423562+0000","mapping_epoch":34,"log_start":"36'2600","ondisk_log_start":"36'2600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35672064,"num_objects":8709,"num_object_clones":0,"num_object_copies":17418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8709,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9452,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104261+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:02:08.104261+0000","last_peered":"2024-09-10T16:02:08.104261+0000","last_clean":"2024-09-10T16:02:08.104261+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:02:08.104261+0000","last_undegraded":"2024-09-10T16:02:08.104261+0000","last_fullsized":"2024-09-10T16:02:08.104261+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'12045","reported_seq":12185,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:39.988151+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:03:39.988151+0000","last_peered":"2024-09-10T16:03:39.988151+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:03:39.988151+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"36'2000","ondisk_log_start":"36'2000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34456035,"num_objects":8413,"num_object_clones":0,"num_object_copies":16826,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8413,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9864,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2310},{"shards":"2","objects":4290}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103774+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:02:08.103774+0000","last_peered":"2024-09-10T16:02:08.103774+0000","last_clean":"2024-09-10T16:02:08.103774+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:02:08.103774+0000","last_undegraded":"2024-09-10T16:02:08.103774+0000","last_fullsized":"2024-09-10T16:02:08.103774+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'12376","reported_seq":12507,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:43.485072+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:03:43.485072+0000","last_peered":"2024-09-10T16:03:43.485072+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:03:43.485072+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"36'2300","ondisk_log_start":"36'2300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34955264,"num_objects":8534,"num_object_clones":0,"num_object_copies":17068,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8534,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8880,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4279},{"shards":"0,2","objects":2338}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":104,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.105055+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:02:08.105055+0000","last_peered":"2024-09-10T16:02:08.105055+0000","last_clean":"2024-09-10T16:02:08.105055+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:02:08.105055+0000","last_undegraded":"2024-09-10T16:02:08.105055+0000","last_fullsized":"2024-09-10T16:02:08.105055+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'12266","reported_seq":12401,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:39.939921+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:03:39.939921+0000","last_peered":"2024-09-10T16:03:39.939921+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:03:39.939921+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"36'2200","ondisk_log_start":"36'2200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34877923,"num_objects":8516,"num_object_clones":0,"num_object_copies":17032,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8516,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9946,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2431},{"shards":"2","objects":4212}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104617+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:02:08.104617+0000","last_peered":"2024-09-10T16:02:08.104617+0000","last_clean":"2024-09-10T16:02:08.104617+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:02:08.104617+0000","last_undegraded":"2024-09-10T16:02:08.104617+0000","last_fullsized":"2024-09-10T16:02:08.104617+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'12433","reported_seq":12561,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:40.032701+0000","last_change":"2024-09-10T15:59:16.174113+0000","last_active":"2024-09-10T16:03:40.032701+0000","last_peered":"2024-09-10T16:03:40.032701+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:03:40.032701+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"36'2400","ondisk_log_start":"36'2400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35786752,"num_objects":8737,"num_object_clones":0,"num_object_copies":17474,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4354,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8737,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9987,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2507},{"shards":"2","objects":4385}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'12362","reported_seq":12497,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:43.376763+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:03:43.376763+0000","last_peered":"2024-09-10T16:03:43.376763+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:03:43.376763+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"36'2300","ondisk_log_start":"36'2300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34717696,"num_objects":8476,"num_object_clones":0,"num_object_copies":16952,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8476,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8993,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4172},{"shards":"0,2","objects":2362}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103316+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:02:08.103316+0000","last_peered":"2024-09-10T16:02:08.103316+0000","last_clean":"2024-09-10T16:02:08.103316+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:02:08.103316+0000","last_undegraded":"2024-09-10T16:02:08.103316+0000","last_fullsized":"2024-09-10T16:02:08.103316+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'12171","reported_seq":15781,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:43.511681+0000","last_change":"2024-09-10T16:03:29.605682+0000","last_active":"2024-09-10T16:03:43.511681+0000","last_peered":"2024-09-10T16:03:43.511681+0000","last_clean":"2024-09-10T15:59:14.115595+0000","last_became_active":"2024-09-10T15:59:16.244089+0000","last_became_peered":"2024-09-10T15:59:16.244089+0000","last_unstale":"2024-09-10T16:03:43.511681+0000","last_undegraded":"2024-09-10T16:03:29.603815+0000","last_fullsized":"2024-09-10T15:59:15.774437+0000","mapping_epoch":42,"log_start":"36'2100","ondisk_log_start":"36'2100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34516992,"num_objects":8427,"num_object_clones":0,"num_object_copies":16854,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":139,"num_objects_misplaced":2331,"num_objects_unfound":0,"num_objects_dirty":8427,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8561,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1733,"num_bytes_recovered":7098368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5957},{"shards":"0,2","objects":2331}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103831+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:02:08.103831+0000","last_peered":"2024-09-10T16:02:08.103831+0000","last_clean":"2024-09-10T16:02:08.103831+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:02:08.103831+0000","last_undegraded":"2024-09-10T16:02:08.103831+0000","last_fullsized":"2024-09-10T16:02:08.103831+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'12538","reported_seq":12679,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:43.431039+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:03:43.431039+0000","last_peered":"2024-09-10T16:03:43.431039+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:03:43.431039+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"36'2500","ondisk_log_start":"36'2500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34856960,"num_objects":8510,"num_object_clones":0,"num_object_copies":17020,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8510,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8793,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4139},{"shards":"0,2","objects":2359}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104463+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:02:08.104463+0000","last_peered":"2024-09-10T16:02:08.104463+0000","last_clean":"2024-09-10T16:02:08.104463+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:02:08.104463+0000","last_undegraded":"2024-09-10T16:02:08.104463+0000","last_fullsized":"2024-09-10T16:02:08.104463+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104448+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:02:08.104448+0000","last_peered":"2024-09-10T16:02:08.104448+0000","last_clean":"2024-09-10T16:02:08.104448+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:02:08.104448+0000","last_undegraded":"2024-09-10T16:02:08.104448+0000","last_fullsized":"2024-09-10T16:02:08.104448+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'12520","reported_seq":12665,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:43.416344+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:03:43.416344+0000","last_peered":"2024-09-10T16:03:43.416344+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:03:43.416344+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"36'2500","ondisk_log_start":"36'2500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35546054,"num_objects":8680,"num_object_clones":0,"num_object_copies":17360,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8680,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8683,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4357},{"shards":"0,2","objects":2408}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":560005878,"num_objects":136730,"num_object_clones":0,"num_object_copies":273460,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43050,"num_objects_misplaced":2331,"num_objects_unfound":0,"num_objects_dirty":136730,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":155376,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6005,"num_bytes_recovered":24592384,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1223860224,"data_stored":1223700334,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160802,"ondisk_log_size":160802,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986244,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":711084,"kb_used_data":412828,"kb_used_omap":4,"kb_used_meta":298235,"kb_avail":93660756,"statfs":{"total":96636764160,"available":95908614144,"internally_reserved":0,"allocated":422735872,"data_stored":422443639,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":305393322},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2],"upper_bound":16},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659190,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":576324,"kb_used_data":305844,"kb_used_omap":4,"kb_used_meta":270459,"kb_avail":93795516,"statfs":{"total":96636764160,"available":96046608384,"internally_reserved":0,"allocated":313184256,"data_stored":312844590,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":276950702},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,2,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182473,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":861164,"kb_used_data":478296,"kb_used_omap":4,"kb_used_meta":382843,"kb_avail":93510676,"statfs":{"total":96636764160,"available":95754932224,"internally_reserved":0,"allocated":489775104,"data_stored":489482866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":392031914},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,3,4,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":488960000,"data_stored":488923870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":312672256,"data_stored":312588698,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":422227968,"data_stored":422187766,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:03:43.773 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-10T16:03:43.773 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "45'12235", 'reported_seq': 12310, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:03:43.473199+0000', 'last_change': '2024-09-10T15:59:16.281558+0000', 'last_active': '2024-09-10T16:03:43.473199+0000', 'last_peered': '2024-09-10T16:03:43.473199+0000', 'last_clean': '2024-09-10T15:59:14.132456+0000', 'last_became_active': '2024-09-10T15:59:16.274407+0000', 'last_became_peered': '2024-09-10T15:59:16.274407+0000', 'last_unstale': '2024-09-10T16:03:43.473199+0000', 'last_undegraded': '2024-09-10T15:59:15.851279+0000', 'last_fullsized': '2024-09-10T15:59:15.774356+0000', 'mapping_epoch': 42, 'log_start': "36'2200", 'ondisk_log_start': "36'2200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'objects_scrubbed': 0, 'log_size': 10035, 'log_dups_size': 0, 'ondisk_log_size': 10035, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34557952, 'num_objects': 8437, 'num_object_clones': 0, 'num_object_copies': 16874, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4259, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8437, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12235, 'num_write_kb': 41344, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4178}, {'shards': '0,2', 'objects': 2359}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:03:43.773 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-10T16:03:43.773 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "45'12525", 'reported_seq': 12663, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:03:43.484969+0000', 'last_change': '2024-09-10T15:59:16.308264+0000', 'last_active': '2024-09-10T16:03:43.484969+0000', 'last_peered': '2024-09-10T16:03:43.484969+0000', 'last_clean': '2024-09-10T15:59:14.111551+0000', 'last_became_active': '2024-09-10T15:59:16.300040+0000', 'last_became_peered': '2024-09-10T15:59:16.300040+0000', 'last_unstale': '2024-09-10T16:03:43.484969+0000', 'last_undegraded': '2024-09-10T15:59:15.878234+0000', 'last_fullsized': '2024-09-10T15:59:15.769328+0000', 'mapping_epoch': 42, 'log_start': "36'2500", 'ondisk_log_start': "36'2500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3442", 'last_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'objects_scrubbed': 3384, 'log_size': 10025, 'log_dups_size': 0, 'ondisk_log_size': 10025, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1292, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34963939, 'num_objects': 8537, 'num_object_clones': 0, 'num_object_copies': 17074, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4372, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8537, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 9137, 'num_write_kb': 28580, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4166}, {'shards': '0,2', 'objects': 2379}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:03:43.774 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-10T16:03:43.774 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "45'12467", 'reported_seq': 12605, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:03:43.260154+0000', 'last_change': '2024-09-10T15:59:16.331899+0000', 'last_active': '2024-09-10T16:03:43.260154+0000', 'last_peered': '2024-09-10T16:03:43.260154+0000', 'last_clean': '2024-09-10T15:59:14.139436+0000', 'last_became_active': '2024-09-10T15:59:16.318715+0000', 'last_became_peered': '2024-09-10T15:59:16.318715+0000', 'last_unstale': '2024-09-10T16:03:43.260154+0000', 'last_undegraded': '2024-09-10T15:59:15.877524+0000', 'last_fullsized': '2024-09-10T15:59:15.769042+0000', 'mapping_epoch': 42, 'log_start': "36'2400", 'ondisk_log_start': "36'2400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3537", 'last_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'objects_scrubbed': 3497, 'log_size': 10067, 'log_dups_size': 0, 'ondisk_log_size': 10067, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1224, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35288006, 'num_objects': 8617, 'num_object_clones': 0, 'num_object_copies': 17234, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4237, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8617, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 8966, 'num_write_kb': 28172, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4380}, {'shards': '0,2', 'objects': 2314}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:03:43.774 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-10T16:03:43.774 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "45'12045", 'reported_seq': 12185, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:03:39.988151+0000', 'last_change': '2024-09-10T15:59:16.175680+0000', 'last_active': '2024-09-10T16:03:39.988151+0000', 'last_peered': '2024-09-10T16:03:39.988151+0000', 'last_clean': '2024-09-10T15:59:14.088187+0000', 'last_became_active': '2024-09-10T15:59:16.165810+0000', 'last_became_peered': '2024-09-10T15:59:16.165810+0000', 'last_unstale': '2024-09-10T16:03:39.988151+0000', 'last_undegraded': '2024-09-10T15:59:15.822098+0000', 'last_fullsized': '2024-09-10T15:59:15.769123+0000', 'mapping_epoch': 42, 'log_start': "36'2000", 'ondisk_log_start': "36'2000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2218", 'last_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'objects_scrubbed': 2175, 'log_size': 10045, 'log_dups_size': 0, 'ondisk_log_size': 10045, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 825, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34456035, 'num_objects': 8413, 'num_object_clones': 0, 'num_object_copies': 16826, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4124, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8413, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 9864, 'num_write_kb': 32204, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2310}, {'shards': '2', 'objects': 4290}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:03:43.774 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-10T16:03:43.774 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "45'12376", 'reported_seq': 12507, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:03:43.485072+0000', 'last_change': '2024-09-10T15:59:16.277742+0000', 'last_active': '2024-09-10T16:03:43.485072+0000', 'last_peered': '2024-09-10T16:03:43.485072+0000', 'last_clean': '2024-09-10T15:59:14.129081+0000', 'last_became_active': '2024-09-10T15:59:16.269835+0000', 'last_became_peered': '2024-09-10T15:59:16.269835+0000', 'last_unstale': '2024-09-10T16:03:43.485072+0000', 'last_undegraded': '2024-09-10T15:59:15.860857+0000', 'last_fullsized': '2024-09-10T15:59:15.771410+0000', 'mapping_epoch': 42, 'log_start': "36'2300", 'ondisk_log_start': "36'2300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3557", 'last_scrub_stamp': '2024-09-10T15:59:02.493678+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:02.493678+0000', 'objects_scrubbed': 3488, 'log_size': 10076, 'log_dups_size': 0, 'ondisk_log_size': 10076, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1472, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34955264, 'num_objects': 8534, 'num_object_clones': 0, 'num_object_copies': 17068, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4257, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8534, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8880, 'num_write_kb': 27852, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4279}, {'shards': '0,2', 'objects': 2338}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:03:43.774 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-10T16:03:43.774 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "45'12266", 'reported_seq': 12401, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:03:39.939921+0000', 'last_change': '2024-09-10T15:59:16.284311+0000', 'last_active': '2024-09-10T16:03:39.939921+0000', 'last_peered': '2024-09-10T16:03:39.939921+0000', 'last_clean': '2024-09-10T15:59:14.116041+0000', 'last_became_active': '2024-09-10T15:59:16.274543+0000', 'last_became_peered': '2024-09-10T15:59:16.274543+0000', 'last_unstale': '2024-09-10T16:03:39.939921+0000', 'last_undegraded': '2024-09-10T15:59:15.812484+0000', 'last_fullsized': '2024-09-10T15:59:15.770885+0000', 'mapping_epoch': 42, 'log_start': "36'2200", 'ondisk_log_start': "36'2200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2356", 'last_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'objects_scrubbed': 2316, 'log_size': 10066, 'log_dups_size': 0, 'ondisk_log_size': 10066, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 783, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34877923, 'num_objects': 8516, 'num_object_clones': 0, 'num_object_copies': 17032, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8516, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 9946, 'num_write_kb': 32292, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2431}, {'shards': '2', 'objects': 4212}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:03:43.774 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-10T16:03:43.774 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "45'12433", 'reported_seq': 12561, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:03:40.032701+0000', 'last_change': '2024-09-10T15:59:16.174113+0000', 'last_active': '2024-09-10T16:03:40.032701+0000', 'last_peered': '2024-09-10T16:03:40.032701+0000', 'last_clean': '2024-09-10T15:59:14.110654+0000', 'last_became_active': '2024-09-10T15:59:16.164645+0000', 'last_became_peered': '2024-09-10T15:59:16.164645+0000', 'last_unstale': '2024-09-10T16:03:40.032701+0000', 'last_undegraded': '2024-09-10T15:59:15.803593+0000', 'last_fullsized': '2024-09-10T15:59:15.769083+0000', 'mapping_epoch': 42, 'log_start': "36'2400", 'ondisk_log_start': "36'2400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2486", 'last_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'objects_scrubbed': 2440, 'log_size': 10033, 'log_dups_size': 0, 'ondisk_log_size': 10033, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 792, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35786752, 'num_objects': 8737, 'num_object_clones': 0, 'num_object_copies': 17474, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4354, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8737, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9987, 'num_write_kb': 32568, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2507}, {'shards': '2', 'objects': 4385}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:03:43.774 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-10T16:03:43.775 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "45'12362", 'reported_seq': 12497, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:03:43.376763+0000', 'last_change': '2024-09-10T15:59:16.315641+0000', 'last_active': '2024-09-10T16:03:43.376763+0000', 'last_peered': '2024-09-10T16:03:43.376763+0000', 'last_clean': '2024-09-10T15:59:14.117001+0000', 'last_became_active': '2024-09-10T15:59:16.302245+0000', 'last_became_peered': '2024-09-10T15:59:16.302245+0000', 'last_unstale': '2024-09-10T16:03:43.376763+0000', 'last_undegraded': '2024-09-10T15:59:15.845012+0000', 'last_fullsized': '2024-09-10T15:59:15.770337+0000', 'mapping_epoch': 42, 'log_start': "36'2300", 'ondisk_log_start': "36'2300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3404", 'last_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'objects_scrubbed': 3367, 'log_size': 10062, 'log_dups_size': 0, 'ondisk_log_size': 10062, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 983, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34717696, 'num_objects': 8476, 'num_object_clones': 0, 'num_object_copies': 16952, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8476, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8993, 'num_write_kb': 28204, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4172}, {'shards': '0,2', 'objects': 2362}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:03:43.775 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-10T16:03:43.775 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "45'12171", 'reported_seq': 15781, 'reported_epoch': 45, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:03:43.511681+0000', 'last_change': '2024-09-10T16:03:29.605682+0000', 'last_active': '2024-09-10T16:03:43.511681+0000', 'last_peered': '2024-09-10T16:03:43.511681+0000', 'last_clean': '2024-09-10T15:59:14.115595+0000', 'last_became_active': '2024-09-10T15:59:16.244089+0000', 'last_became_peered': '2024-09-10T15:59:16.244089+0000', 'last_unstale': '2024-09-10T16:03:43.511681+0000', 'last_undegraded': '2024-09-10T16:03:29.603815+0000', 'last_fullsized': '2024-09-10T15:59:15.774437+0000', 'mapping_epoch': 42, 'log_start': "36'2100", 'ondisk_log_start': "36'2100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3653", 'last_scrub_stamp': '2024-09-10T15:59:05.298468+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:05.298468+0000', 'objects_scrubbed': 3602, 'log_size': 10071, 'log_dups_size': 0, 'ondisk_log_size': 10071, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1289, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34516992, 'num_objects': 8427, 'num_object_clones': 0, 'num_object_copies': 16854, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 139, 'num_objects_misplaced': 2331, 'num_objects_unfound': 0, 'num_objects_dirty': 8427, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8561, 'num_write_kb': 26772, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1733, 'num_bytes_recovered': 7098368, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5957}, {'shards': '0,2', 'objects': 2331}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:03:43.775 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-10T16:03:43.775 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "45'12538", 'reported_seq': 12679, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:03:43.431039+0000', 'last_change': '2024-09-10T15:59:16.311291+0000', 'last_active': '2024-09-10T16:03:43.431039+0000', 'last_peered': '2024-09-10T16:03:43.431039+0000', 'last_clean': '2024-09-10T15:59:14.116363+0000', 'last_became_active': '2024-09-10T15:59:16.301244+0000', 'last_became_peered': '2024-09-10T15:59:16.301244+0000', 'last_unstale': '2024-09-10T16:03:43.431039+0000', 'last_undegraded': '2024-09-10T15:59:15.883434+0000', 'last_fullsized': '2024-09-10T15:59:15.772756+0000', 'mapping_epoch': 42, 'log_start': "36'2500", 'ondisk_log_start': "36'2500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3789", 'last_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'objects_scrubbed': 3741, 'log_size': 10038, 'log_dups_size': 0, 'ondisk_log_size': 10038, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1440, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34856960, 'num_objects': 8510, 'num_object_clones': 0, 'num_object_copies': 17020, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4373, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8510, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8793, 'num_write_kb': 27124, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4139}, {'shards': '0,2', 'objects': 2359}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:03:43.775 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-10T16:03:43.775 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "45'12520", 'reported_seq': 12665, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:03:43.416344+0000', 'last_change': '2024-09-10T15:59:16.313409+0000', 'last_active': '2024-09-10T16:03:43.416344+0000', 'last_peered': '2024-09-10T16:03:43.416344+0000', 'last_clean': '2024-09-10T15:59:14.129780+0000', 'last_became_active': '2024-09-10T15:59:16.300932+0000', 'last_became_peered': '2024-09-10T15:59:16.300932+0000', 'last_unstale': '2024-09-10T16:03:43.416344+0000', 'last_undegraded': '2024-09-10T15:59:15.887823+0000', 'last_fullsized': '2024-09-10T15:59:15.768513+0000', 'mapping_epoch': 42, 'log_start': "36'2500", 'ondisk_log_start': "36'2500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3880", 'last_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'objects_scrubbed': 3827, 'log_size': 10020, 'log_dups_size': 0, 'ondisk_log_size': 10020, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1397, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35546054, 'num_objects': 8680, 'num_object_clones': 0, 'num_object_copies': 17360, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4325, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8680, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8683, 'num_write_kb': 27072, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4357}, {'shards': '0,2', 'objects': 2408}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:03:43.775 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T16:03:44.381 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:03:44.382 INFO:teuthology.orchestra.run.smithi115.stdout:{"fsid":"7effee29-df05-4351-9741-ee203564ed81","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 43050/273464 objects degraded (15.742%), 11 pgs degraded, 11 pgs undersized","count":22},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":388,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983954,"num_in_osds":3,"osd_in_since":1725983947,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":136732,"data_bytes":560005897,"bytes_used":2200137728,"bytes_avail":287710154752,"bytes_total":289910292480,"degraded_objects":43050,"degraded_total":273464,"degraded_ratio":0.15742474329344996,"misplaced_objects":2331,"misplaced_total":273464,"misplaced_ratio":0.0085239739051575347,"recovering_objects_per_sec":24,"recovering_bytes_per_sec":100324,"recovering_keys_per_sec":0,"num_objects_recovered":294,"num_bytes_recovered":1204224,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":485},"fsmap":{"epoch":1,"btime":"2024-09-10T15:57:14:967120+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T16:00:13.560776+0000","services":{}},"progress_events":{}} 2024-09-10T16:03:44.405 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-10T16:03:44.405 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:03:44.996 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:03:44.997 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:03:45.023 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":232,"stamp":"2024-09-10T16:03:43.643771+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":560005897,"num_objects":136732,"num_object_clones":0,"num_object_copies":273464,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43050,"num_objects_misplaced":2331,"num_objects_unfound":0,"num_objects_dirty":136732,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":155378,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6007,"num_bytes_recovered":24592403,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160805,"ondisk_log_size":160805,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2148572,"kb_used_data":1196968,"kb_used_omap":12,"kb_used_meta":951539,"kb_avail":280966948,"statfs":{"total":289910292480,"available":287710154752,"internally_reserved":0,"allocated":1225695232,"data_stored":1224771095,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":974375938},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,4,5,5,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-23855104,"num_objects":-5824,"num_object_clones":0,"num_object_copies":-11648,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":139,"num_objects_misplaced":-433,"num_objects_unfound":0,"num_objects_dirty":-5824,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5824,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":294,"num_bytes_recovered":1204224,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003279"},"pg_stats":[{"pgid":"2.f","version":"45'12235","reported_seq":12310,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:43.473199+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:03:43.473199+0000","last_peered":"2024-09-10T16:03:43.473199+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:03:43.473199+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"36'2200","ondisk_log_start":"36'2200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34557952,"num_objects":8437,"num_object_clones":0,"num_object_copies":16874,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8437,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12235,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4178},{"shards":"0,2","objects":2359}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'12268","reported_seq":12381,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:03:40.003762+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:03:40.003762+0000","last_peered":"2024-09-10T16:03:40.003762+0000","last_clean":"2024-09-10T16:03:40.003762+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:03:40.003762+0000","last_undegraded":"2024-09-10T16:03:40.003762+0000","last_fullsized":"2024-09-10T16:03:40.003762+0000","mapping_epoch":35,"log_start":"36'2200","ondisk_log_start":"36'2200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35128262,"num_objects":8578,"num_object_clones":0,"num_object_copies":17156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8578,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":10373,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'12264","reported_seq":20941,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:03:41.197004+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:03:41.197004+0000","last_peered":"2024-09-10T16:03:41.197004+0000","last_clean":"2024-09-10T16:03:41.197004+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:03:41.197004+0000","last_undegraded":"2024-09-10T16:03:41.197004+0000","last_fullsized":"2024-09-10T16:03:41.197004+0000","mapping_epoch":44,"log_start":"36'2200","ondisk_log_start":"36'2200","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34951192,"num_objects":8534,"num_object_clones":0,"num_object_copies":17068,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8534,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10268,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'12182","reported_seq":12263,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:03:39.989840+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:03:39.989840+0000","last_peered":"2024-09-10T16:03:39.989840+0000","last_clean":"2024-09-10T16:03:39.989840+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:03:39.989840+0000","last_undegraded":"2024-09-10T16:03:39.989840+0000","last_fullsized":"2024-09-10T16:03:39.989840+0000","mapping_epoch":35,"log_start":"36'2100","ondisk_log_start":"36'2100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34725888,"num_objects":8478,"num_object_clones":0,"num_object_copies":16956,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8478,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12182,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'12345","reported_seq":12475,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:03:43.516141+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:03:43.516141+0000","last_peered":"2024-09-10T16:03:43.516141+0000","last_clean":"2024-09-10T16:03:43.516141+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:03:43.516141+0000","last_undegraded":"2024-09-10T16:03:43.516141+0000","last_fullsized":"2024-09-10T16:03:43.516141+0000","mapping_epoch":34,"log_start":"36'2300","ondisk_log_start":"36'2300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35004899,"num_objects":8547,"num_object_clones":0,"num_object_copies":17094,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8547,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9056,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'12525","reported_seq":12663,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:43.484969+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:03:43.484969+0000","last_peered":"2024-09-10T16:03:43.484969+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:03:43.484969+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"36'2500","ondisk_log_start":"36'2500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34963939,"num_objects":8537,"num_object_clones":0,"num_object_copies":17074,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8537,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9137,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4166},{"shards":"0,2","objects":2379}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'12467","reported_seq":12605,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:43.260154+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:03:43.260154+0000","last_peered":"2024-09-10T16:03:43.260154+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:03:43.260154+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"36'2400","ondisk_log_start":"36'2400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35288006,"num_objects":8617,"num_object_clones":0,"num_object_copies":17234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8617,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":8966,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4380},{"shards":"0,2","objects":2314}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'12605","reported_seq":12735,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:03:43.423562+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:03:43.423562+0000","last_peered":"2024-09-10T16:03:43.423562+0000","last_clean":"2024-09-10T16:03:43.423562+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:03:43.423562+0000","last_undegraded":"2024-09-10T16:03:43.423562+0000","last_fullsized":"2024-09-10T16:03:43.423562+0000","mapping_epoch":34,"log_start":"36'2600","ondisk_log_start":"36'2600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35672064,"num_objects":8709,"num_object_clones":0,"num_object_copies":17418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8709,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9452,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104261+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:02:08.104261+0000","last_peered":"2024-09-10T16:02:08.104261+0000","last_clean":"2024-09-10T16:02:08.104261+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:02:08.104261+0000","last_undegraded":"2024-09-10T16:02:08.104261+0000","last_fullsized":"2024-09-10T16:02:08.104261+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'12045","reported_seq":12185,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:39.988151+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:03:39.988151+0000","last_peered":"2024-09-10T16:03:39.988151+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:03:39.988151+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"36'2000","ondisk_log_start":"36'2000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34456035,"num_objects":8413,"num_object_clones":0,"num_object_copies":16826,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8413,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9864,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2310},{"shards":"2","objects":4290}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103774+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:02:08.103774+0000","last_peered":"2024-09-10T16:02:08.103774+0000","last_clean":"2024-09-10T16:02:08.103774+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:02:08.103774+0000","last_undegraded":"2024-09-10T16:02:08.103774+0000","last_fullsized":"2024-09-10T16:02:08.103774+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'12376","reported_seq":12507,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:43.485072+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:03:43.485072+0000","last_peered":"2024-09-10T16:03:43.485072+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:03:43.485072+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"36'2300","ondisk_log_start":"36'2300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34955264,"num_objects":8534,"num_object_clones":0,"num_object_copies":17068,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8534,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8880,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4279},{"shards":"0,2","objects":2338}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":104,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.105055+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:02:08.105055+0000","last_peered":"2024-09-10T16:02:08.105055+0000","last_clean":"2024-09-10T16:02:08.105055+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:02:08.105055+0000","last_undegraded":"2024-09-10T16:02:08.105055+0000","last_fullsized":"2024-09-10T16:02:08.105055+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'12266","reported_seq":12401,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:39.939921+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:03:39.939921+0000","last_peered":"2024-09-10T16:03:39.939921+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:03:39.939921+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"36'2200","ondisk_log_start":"36'2200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34877923,"num_objects":8516,"num_object_clones":0,"num_object_copies":17032,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8516,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9946,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2431},{"shards":"2","objects":4212}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104617+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:02:08.104617+0000","last_peered":"2024-09-10T16:02:08.104617+0000","last_clean":"2024-09-10T16:02:08.104617+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:02:08.104617+0000","last_undegraded":"2024-09-10T16:02:08.104617+0000","last_fullsized":"2024-09-10T16:02:08.104617+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'12433","reported_seq":12561,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:40.032701+0000","last_change":"2024-09-10T15:59:16.174113+0000","last_active":"2024-09-10T16:03:40.032701+0000","last_peered":"2024-09-10T16:03:40.032701+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:03:40.032701+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"36'2400","ondisk_log_start":"36'2400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35786752,"num_objects":8737,"num_object_clones":0,"num_object_copies":17474,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4354,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8737,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9987,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2507},{"shards":"2","objects":4385}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'12362","reported_seq":12497,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:43.376763+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:03:43.376763+0000","last_peered":"2024-09-10T16:03:43.376763+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:03:43.376763+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"36'2300","ondisk_log_start":"36'2300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34717696,"num_objects":8476,"num_object_clones":0,"num_object_copies":16952,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8476,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8993,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4172},{"shards":"0,2","objects":2362}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103316+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:02:08.103316+0000","last_peered":"2024-09-10T16:02:08.103316+0000","last_clean":"2024-09-10T16:02:08.103316+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:02:08.103316+0000","last_undegraded":"2024-09-10T16:02:08.103316+0000","last_fullsized":"2024-09-10T16:02:08.103316+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'12171","reported_seq":15781,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:43.511681+0000","last_change":"2024-09-10T16:03:29.605682+0000","last_active":"2024-09-10T16:03:43.511681+0000","last_peered":"2024-09-10T16:03:43.511681+0000","last_clean":"2024-09-10T15:59:14.115595+0000","last_became_active":"2024-09-10T15:59:16.244089+0000","last_became_peered":"2024-09-10T15:59:16.244089+0000","last_unstale":"2024-09-10T16:03:43.511681+0000","last_undegraded":"2024-09-10T16:03:29.603815+0000","last_fullsized":"2024-09-10T15:59:15.774437+0000","mapping_epoch":42,"log_start":"36'2100","ondisk_log_start":"36'2100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34516992,"num_objects":8427,"num_object_clones":0,"num_object_copies":16854,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":139,"num_objects_misplaced":2331,"num_objects_unfound":0,"num_objects_dirty":8427,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8561,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1733,"num_bytes_recovered":7098368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5957},{"shards":"0,2","objects":2331}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103831+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:02:08.103831+0000","last_peered":"2024-09-10T16:02:08.103831+0000","last_clean":"2024-09-10T16:02:08.103831+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:02:08.103831+0000","last_undegraded":"2024-09-10T16:02:08.103831+0000","last_fullsized":"2024-09-10T16:02:08.103831+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'12538","reported_seq":12679,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:43.431039+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:03:43.431039+0000","last_peered":"2024-09-10T16:03:43.431039+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:03:43.431039+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"36'2500","ondisk_log_start":"36'2500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34856960,"num_objects":8510,"num_object_clones":0,"num_object_copies":17020,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8510,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8793,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4139},{"shards":"0,2","objects":2359}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104463+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:02:08.104463+0000","last_peered":"2024-09-10T16:02:08.104463+0000","last_clean":"2024-09-10T16:02:08.104463+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:02:08.104463+0000","last_undegraded":"2024-09-10T16:02:08.104463+0000","last_fullsized":"2024-09-10T16:02:08.104463+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104448+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:02:08.104448+0000","last_peered":"2024-09-10T16:02:08.104448+0000","last_clean":"2024-09-10T16:02:08.104448+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:02:08.104448+0000","last_undegraded":"2024-09-10T16:02:08.104448+0000","last_fullsized":"2024-09-10T16:02:08.104448+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'12520","reported_seq":12665,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:43.416344+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:03:43.416344+0000","last_peered":"2024-09-10T16:03:43.416344+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:03:43.416344+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"36'2500","ondisk_log_start":"36'2500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35546054,"num_objects":8680,"num_object_clones":0,"num_object_copies":17360,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8680,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8683,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4357},{"shards":"0,2","objects":2408}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":560005878,"num_objects":136730,"num_object_clones":0,"num_object_copies":273460,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43050,"num_objects_misplaced":2331,"num_objects_unfound":0,"num_objects_dirty":136730,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":155376,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6005,"num_bytes_recovered":24592384,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1223860224,"data_stored":1223700334,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160802,"ondisk_log_size":160802,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986244,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":711084,"kb_used_data":412828,"kb_used_omap":4,"kb_used_meta":298235,"kb_avail":93660756,"statfs":{"total":96636764160,"available":95908614144,"internally_reserved":0,"allocated":422735872,"data_stored":422443639,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":305393322},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2],"upper_bound":16},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659190,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":576324,"kb_used_data":305844,"kb_used_omap":4,"kb_used_meta":270459,"kb_avail":93795516,"statfs":{"total":96636764160,"available":96046608384,"internally_reserved":0,"allocated":313184256,"data_stored":312844590,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":276950702},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,2,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182473,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":861164,"kb_used_data":478296,"kb_used_omap":4,"kb_used_meta":382843,"kb_avail":93510676,"statfs":{"total":96636764160,"available":95754932224,"internally_reserved":0,"allocated":489775104,"data_stored":489482866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":392031914},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,3,4,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":488960000,"data_stored":488923870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":312672256,"data_stored":312588698,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":422227968,"data_stored":422187766,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:03:48.025 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:03:48.558 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:03:48.561 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:03:48.588 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":234,"stamp":"2024-09-10T16:03:47.645231+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":556782345,"num_objects":135945,"num_object_clones":0,"num_object_copies":271890,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43050,"num_objects_misplaced":2331,"num_objects_unfound":0,"num_objects_dirty":135945,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":156165,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6007,"num_bytes_recovered":24592403,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160892,"ondisk_log_size":160892,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2139416,"kb_used_data":1191968,"kb_used_omap":12,"kb_used_meta":947379,"kb_avail":280976104,"statfs":{"total":289910292480,"available":287719530496,"internally_reserved":0,"allocated":1220575232,"data_stored":1219651095,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":970116098},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,5,5,6,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18628608,"num_objects":-4548,"num_object_clones":0,"num_object_copies":-9096,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":114,"num_objects_misplaced":-309,"num_objects_unfound":0,"num_objects_dirty":-4548,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4548,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":195,"num_bytes_recovered":798720,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003416"},"pg_stats":[{"pgid":"2.f","version":"45'12235","reported_seq":12310,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:43.473199+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:03:43.473199+0000","last_peered":"2024-09-10T16:03:43.473199+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:03:43.473199+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"36'2200","ondisk_log_start":"36'2200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34557952,"num_objects":8437,"num_object_clones":0,"num_object_copies":16874,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8437,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12235,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4178},{"shards":"0,2","objects":2359}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'12392","reported_seq":12505,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:03:44.936172+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:03:44.936172+0000","last_peered":"2024-09-10T16:03:44.936172+0000","last_clean":"2024-09-10T16:03:44.936172+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:03:44.936172+0000","last_undegraded":"2024-09-10T16:03:44.936172+0000","last_fullsized":"2024-09-10T16:03:44.936172+0000","mapping_epoch":35,"log_start":"36'2300","ondisk_log_start":"36'2300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34620358,"num_objects":8454,"num_object_clones":0,"num_object_copies":16908,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8454,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":10497,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'12398","reported_seq":21076,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.143495+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:03:46.143495+0000","last_peered":"2024-09-10T16:03:46.143495+0000","last_clean":"2024-09-10T16:03:46.143495+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:03:46.143495+0000","last_undegraded":"2024-09-10T16:03:46.143495+0000","last_fullsized":"2024-09-10T16:03:46.143495+0000","mapping_epoch":44,"log_start":"36'2300","ondisk_log_start":"36'2300","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34402328,"num_objects":8400,"num_object_clones":0,"num_object_copies":16800,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8400,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10402,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'12329","reported_seq":12410,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:03:45.027842+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:03:45.027842+0000","last_peered":"2024-09-10T16:03:45.027842+0000","last_clean":"2024-09-10T16:03:45.027842+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:03:45.027842+0000","last_undegraded":"2024-09-10T16:03:45.027842+0000","last_fullsized":"2024-09-10T16:03:45.027842+0000","mapping_epoch":35,"log_start":"36'2300","ondisk_log_start":"36'2300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34123776,"num_objects":8331,"num_object_clones":0,"num_object_copies":16662,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8331,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12329,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'12345","reported_seq":12475,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:03:43.516141+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:03:43.516141+0000","last_peered":"2024-09-10T16:03:43.516141+0000","last_clean":"2024-09-10T16:03:43.516141+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:03:43.516141+0000","last_undegraded":"2024-09-10T16:03:43.516141+0000","last_fullsized":"2024-09-10T16:03:43.516141+0000","mapping_epoch":34,"log_start":"36'2300","ondisk_log_start":"36'2300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35004899,"num_objects":8547,"num_object_clones":0,"num_object_copies":17094,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8547,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9056,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'12525","reported_seq":12663,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:43.484969+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:03:43.484969+0000","last_peered":"2024-09-10T16:03:43.484969+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:03:43.484969+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"36'2500","ondisk_log_start":"36'2500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34963939,"num_objects":8537,"num_object_clones":0,"num_object_copies":17074,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8537,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9137,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4166},{"shards":"0,2","objects":2379}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'12467","reported_seq":12605,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:43.260154+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:03:43.260154+0000","last_peered":"2024-09-10T16:03:43.260154+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:03:43.260154+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"36'2400","ondisk_log_start":"36'2400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35288006,"num_objects":8617,"num_object_clones":0,"num_object_copies":17234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8617,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":8966,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4380},{"shards":"0,2","objects":2314}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'12605","reported_seq":12735,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:03:43.423562+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:03:43.423562+0000","last_peered":"2024-09-10T16:03:43.423562+0000","last_clean":"2024-09-10T16:03:43.423562+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:03:43.423562+0000","last_undegraded":"2024-09-10T16:03:43.423562+0000","last_fullsized":"2024-09-10T16:03:43.423562+0000","mapping_epoch":34,"log_start":"36'2600","ondisk_log_start":"36'2600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35672064,"num_objects":8709,"num_object_clones":0,"num_object_copies":17418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8709,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9452,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104261+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:02:08.104261+0000","last_peered":"2024-09-10T16:02:08.104261+0000","last_clean":"2024-09-10T16:02:08.104261+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:02:08.104261+0000","last_undegraded":"2024-09-10T16:02:08.104261+0000","last_fullsized":"2024-09-10T16:02:08.104261+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'12168","reported_seq":12308,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:44.814437+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:03:44.814437+0000","last_peered":"2024-09-10T16:03:44.814437+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:03:44.814437+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"36'2100","ondisk_log_start":"36'2100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33952227,"num_objects":8290,"num_object_clones":0,"num_object_copies":16580,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8290,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9987,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2187},{"shards":"2","objects":4167}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103774+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:02:08.103774+0000","last_peered":"2024-09-10T16:02:08.103774+0000","last_clean":"2024-09-10T16:02:08.103774+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:02:08.103774+0000","last_undegraded":"2024-09-10T16:02:08.103774+0000","last_fullsized":"2024-09-10T16:02:08.103774+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'12376","reported_seq":12507,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:43.485072+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:03:43.485072+0000","last_peered":"2024-09-10T16:03:43.485072+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:03:43.485072+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"36'2300","ondisk_log_start":"36'2300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34955264,"num_objects":8534,"num_object_clones":0,"num_object_copies":17068,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8534,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8880,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4279},{"shards":"0,2","objects":2338}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":104,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.105055+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:02:08.105055+0000","last_peered":"2024-09-10T16:02:08.105055+0000","last_clean":"2024-09-10T16:02:08.105055+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:02:08.105055+0000","last_undegraded":"2024-09-10T16:02:08.105055+0000","last_fullsized":"2024-09-10T16:02:08.105055+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'12400","reported_seq":12535,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:44.950962+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:03:44.950962+0000","last_peered":"2024-09-10T16:03:44.950962+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:03:44.950962+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"36'2300","ondisk_log_start":"36'2300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34329059,"num_objects":8382,"num_object_clones":0,"num_object_copies":16764,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8382,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10080,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2297},{"shards":"2","objects":4078}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104617+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:02:08.104617+0000","last_peered":"2024-09-10T16:02:08.104617+0000","last_clean":"2024-09-10T16:02:08.104617+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:02:08.104617+0000","last_undegraded":"2024-09-10T16:02:08.104617+0000","last_fullsized":"2024-09-10T16:02:08.104617+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'12558","reported_seq":12687,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:44.980090+0000","last_change":"2024-09-10T15:59:16.174113+0000","last_active":"2024-09-10T16:03:44.980090+0000","last_peered":"2024-09-10T16:03:44.980090+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:03:44.980090+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"36'2500","ondisk_log_start":"36'2500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35274752,"num_objects":8612,"num_object_clones":0,"num_object_copies":17224,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4354,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8612,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10112,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2382},{"shards":"2","objects":4260}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'12362","reported_seq":12497,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:43.376763+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:03:43.376763+0000","last_peered":"2024-09-10T16:03:43.376763+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:03:43.376763+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"36'2300","ondisk_log_start":"36'2300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34717696,"num_objects":8476,"num_object_clones":0,"num_object_copies":16952,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8476,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8993,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4172},{"shards":"0,2","objects":2362}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103316+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:02:08.103316+0000","last_peered":"2024-09-10T16:02:08.103316+0000","last_clean":"2024-09-10T16:02:08.103316+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:02:08.103316+0000","last_undegraded":"2024-09-10T16:02:08.103316+0000","last_fullsized":"2024-09-10T16:02:08.103316+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'12171","reported_seq":15781,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:43.511681+0000","last_change":"2024-09-10T16:03:29.605682+0000","last_active":"2024-09-10T16:03:43.511681+0000","last_peered":"2024-09-10T16:03:43.511681+0000","last_clean":"2024-09-10T15:59:14.115595+0000","last_became_active":"2024-09-10T15:59:16.244089+0000","last_became_peered":"2024-09-10T15:59:16.244089+0000","last_unstale":"2024-09-10T16:03:43.511681+0000","last_undegraded":"2024-09-10T16:03:29.603815+0000","last_fullsized":"2024-09-10T15:59:15.774437+0000","mapping_epoch":42,"log_start":"36'2100","ondisk_log_start":"36'2100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34516992,"num_objects":8427,"num_object_clones":0,"num_object_copies":16854,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":139,"num_objects_misplaced":2331,"num_objects_unfound":0,"num_objects_dirty":8427,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8561,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1733,"num_bytes_recovered":7098368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5957},{"shards":"0,2","objects":2331}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103831+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:02:08.103831+0000","last_peered":"2024-09-10T16:02:08.103831+0000","last_clean":"2024-09-10T16:02:08.103831+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:02:08.103831+0000","last_undegraded":"2024-09-10T16:02:08.103831+0000","last_fullsized":"2024-09-10T16:02:08.103831+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'12538","reported_seq":12679,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:43.431039+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:03:43.431039+0000","last_peered":"2024-09-10T16:03:43.431039+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:03:43.431039+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"36'2500","ondisk_log_start":"36'2500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34856960,"num_objects":8510,"num_object_clones":0,"num_object_copies":17020,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8510,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8793,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4139},{"shards":"0,2","objects":2359}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104463+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:02:08.104463+0000","last_peered":"2024-09-10T16:02:08.104463+0000","last_clean":"2024-09-10T16:02:08.104463+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:02:08.104463+0000","last_undegraded":"2024-09-10T16:02:08.104463+0000","last_fullsized":"2024-09-10T16:02:08.104463+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104448+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:02:08.104448+0000","last_peered":"2024-09-10T16:02:08.104448+0000","last_clean":"2024-09-10T16:02:08.104448+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:02:08.104448+0000","last_undegraded":"2024-09-10T16:02:08.104448+0000","last_fullsized":"2024-09-10T16:02:08.104448+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'12520","reported_seq":12665,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:43.416344+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:03:43.416344+0000","last_peered":"2024-09-10T16:03:43.416344+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:03:43.416344+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"36'2500","ondisk_log_start":"36'2500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35546054,"num_objects":8680,"num_object_clones":0,"num_object_copies":17360,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8680,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8683,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4357},{"shards":"0,2","objects":2408}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":556782326,"num_objects":135943,"num_object_clones":0,"num_object_copies":271886,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43050,"num_objects_misplaced":2331,"num_objects_unfound":0,"num_objects_dirty":135943,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":156163,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6005,"num_bytes_recovered":24592384,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1218871296,"data_stored":1218711406,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160889,"ondisk_log_size":160889,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986245,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":710852,"kb_used_data":408432,"kb_used_omap":4,"kb_used_meta":302395,"kb_avail":93660988,"statfs":{"total":96636764160,"available":95908851712,"internally_reserved":0,"allocated":418234368,"data_stored":417942135,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":309653162},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,1,1,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659191,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":567400,"kb_used_data":305240,"kb_used_omap":4,"kb_used_meta":262139,"kb_avail":93804440,"statfs":{"total":96636764160,"available":96055746560,"internally_reserved":0,"allocated":312565760,"data_stored":312226094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":268431022},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,1,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182473,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":861164,"kb_used_data":478296,"kb_used_omap":4,"kb_used_meta":382843,"kb_avail":93510676,"statfs":{"total":96636764160,"available":95754932224,"internally_reserved":0,"allocated":489775104,"data_stored":489482866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":392031914},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,3,4,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":488960000,"data_stored":488923870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":312127488,"data_stored":312043930,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":417783808,"data_stored":417743606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:03:48.589 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:03:49.185 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:03:49.185 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:03:49.210 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":234,"stamp":"2024-09-10T16:03:47.645231+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":556782345,"num_objects":135945,"num_object_clones":0,"num_object_copies":271890,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43050,"num_objects_misplaced":2331,"num_objects_unfound":0,"num_objects_dirty":135945,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":156165,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6007,"num_bytes_recovered":24592403,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160892,"ondisk_log_size":160892,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2139416,"kb_used_data":1191968,"kb_used_omap":12,"kb_used_meta":947379,"kb_avail":280976104,"statfs":{"total":289910292480,"available":287719530496,"internally_reserved":0,"allocated":1220575232,"data_stored":1219651095,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":970116098},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,5,5,6,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18628608,"num_objects":-4548,"num_object_clones":0,"num_object_copies":-9096,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":114,"num_objects_misplaced":-309,"num_objects_unfound":0,"num_objects_dirty":-4548,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4548,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":195,"num_bytes_recovered":798720,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003416"},"pg_stats":[{"pgid":"2.f","version":"45'12235","reported_seq":12310,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:43.473199+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:03:43.473199+0000","last_peered":"2024-09-10T16:03:43.473199+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:03:43.473199+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"36'2200","ondisk_log_start":"36'2200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34557952,"num_objects":8437,"num_object_clones":0,"num_object_copies":16874,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8437,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12235,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4178},{"shards":"0,2","objects":2359}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'12392","reported_seq":12505,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:03:44.936172+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:03:44.936172+0000","last_peered":"2024-09-10T16:03:44.936172+0000","last_clean":"2024-09-10T16:03:44.936172+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:03:44.936172+0000","last_undegraded":"2024-09-10T16:03:44.936172+0000","last_fullsized":"2024-09-10T16:03:44.936172+0000","mapping_epoch":35,"log_start":"36'2300","ondisk_log_start":"36'2300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34620358,"num_objects":8454,"num_object_clones":0,"num_object_copies":16908,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8454,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":10497,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'12398","reported_seq":21076,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.143495+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:03:46.143495+0000","last_peered":"2024-09-10T16:03:46.143495+0000","last_clean":"2024-09-10T16:03:46.143495+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:03:46.143495+0000","last_undegraded":"2024-09-10T16:03:46.143495+0000","last_fullsized":"2024-09-10T16:03:46.143495+0000","mapping_epoch":44,"log_start":"36'2300","ondisk_log_start":"36'2300","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34402328,"num_objects":8400,"num_object_clones":0,"num_object_copies":16800,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8400,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10402,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'12329","reported_seq":12410,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:03:45.027842+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:03:45.027842+0000","last_peered":"2024-09-10T16:03:45.027842+0000","last_clean":"2024-09-10T16:03:45.027842+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:03:45.027842+0000","last_undegraded":"2024-09-10T16:03:45.027842+0000","last_fullsized":"2024-09-10T16:03:45.027842+0000","mapping_epoch":35,"log_start":"36'2300","ondisk_log_start":"36'2300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34123776,"num_objects":8331,"num_object_clones":0,"num_object_copies":16662,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8331,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12329,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'12345","reported_seq":12475,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:03:43.516141+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:03:43.516141+0000","last_peered":"2024-09-10T16:03:43.516141+0000","last_clean":"2024-09-10T16:03:43.516141+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:03:43.516141+0000","last_undegraded":"2024-09-10T16:03:43.516141+0000","last_fullsized":"2024-09-10T16:03:43.516141+0000","mapping_epoch":34,"log_start":"36'2300","ondisk_log_start":"36'2300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35004899,"num_objects":8547,"num_object_clones":0,"num_object_copies":17094,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8547,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9056,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'12525","reported_seq":12663,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:43.484969+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:03:43.484969+0000","last_peered":"2024-09-10T16:03:43.484969+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:03:43.484969+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"36'2500","ondisk_log_start":"36'2500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34963939,"num_objects":8537,"num_object_clones":0,"num_object_copies":17074,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8537,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9137,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4166},{"shards":"0,2","objects":2379}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'12467","reported_seq":12605,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:43.260154+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:03:43.260154+0000","last_peered":"2024-09-10T16:03:43.260154+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:03:43.260154+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"36'2400","ondisk_log_start":"36'2400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35288006,"num_objects":8617,"num_object_clones":0,"num_object_copies":17234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8617,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":8966,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4380},{"shards":"0,2","objects":2314}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'12605","reported_seq":12735,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:03:43.423562+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:03:43.423562+0000","last_peered":"2024-09-10T16:03:43.423562+0000","last_clean":"2024-09-10T16:03:43.423562+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:03:43.423562+0000","last_undegraded":"2024-09-10T16:03:43.423562+0000","last_fullsized":"2024-09-10T16:03:43.423562+0000","mapping_epoch":34,"log_start":"36'2600","ondisk_log_start":"36'2600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35672064,"num_objects":8709,"num_object_clones":0,"num_object_copies":17418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8709,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9452,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104261+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:02:08.104261+0000","last_peered":"2024-09-10T16:02:08.104261+0000","last_clean":"2024-09-10T16:02:08.104261+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:02:08.104261+0000","last_undegraded":"2024-09-10T16:02:08.104261+0000","last_fullsized":"2024-09-10T16:02:08.104261+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'12168","reported_seq":12308,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:44.814437+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:03:44.814437+0000","last_peered":"2024-09-10T16:03:44.814437+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:03:44.814437+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"36'2100","ondisk_log_start":"36'2100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33952227,"num_objects":8290,"num_object_clones":0,"num_object_copies":16580,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8290,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9987,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2187},{"shards":"2","objects":4167}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103774+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:02:08.103774+0000","last_peered":"2024-09-10T16:02:08.103774+0000","last_clean":"2024-09-10T16:02:08.103774+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:02:08.103774+0000","last_undegraded":"2024-09-10T16:02:08.103774+0000","last_fullsized":"2024-09-10T16:02:08.103774+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'12376","reported_seq":12507,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:43.485072+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:03:43.485072+0000","last_peered":"2024-09-10T16:03:43.485072+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:03:43.485072+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"36'2300","ondisk_log_start":"36'2300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34955264,"num_objects":8534,"num_object_clones":0,"num_object_copies":17068,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8534,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8880,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4279},{"shards":"0,2","objects":2338}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":104,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.105055+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:02:08.105055+0000","last_peered":"2024-09-10T16:02:08.105055+0000","last_clean":"2024-09-10T16:02:08.105055+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:02:08.105055+0000","last_undegraded":"2024-09-10T16:02:08.105055+0000","last_fullsized":"2024-09-10T16:02:08.105055+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'12400","reported_seq":12535,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:44.950962+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:03:44.950962+0000","last_peered":"2024-09-10T16:03:44.950962+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:03:44.950962+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"36'2300","ondisk_log_start":"36'2300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34329059,"num_objects":8382,"num_object_clones":0,"num_object_copies":16764,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8382,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10080,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2297},{"shards":"2","objects":4078}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104617+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:02:08.104617+0000","last_peered":"2024-09-10T16:02:08.104617+0000","last_clean":"2024-09-10T16:02:08.104617+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:02:08.104617+0000","last_undegraded":"2024-09-10T16:02:08.104617+0000","last_fullsized":"2024-09-10T16:02:08.104617+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'12558","reported_seq":12687,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:44.980090+0000","last_change":"2024-09-10T15:59:16.174113+0000","last_active":"2024-09-10T16:03:44.980090+0000","last_peered":"2024-09-10T16:03:44.980090+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:03:44.980090+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"36'2500","ondisk_log_start":"36'2500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35274752,"num_objects":8612,"num_object_clones":0,"num_object_copies":17224,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4354,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8612,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10112,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2382},{"shards":"2","objects":4260}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'12362","reported_seq":12497,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:43.376763+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:03:43.376763+0000","last_peered":"2024-09-10T16:03:43.376763+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:03:43.376763+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"36'2300","ondisk_log_start":"36'2300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34717696,"num_objects":8476,"num_object_clones":0,"num_object_copies":16952,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8476,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8993,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4172},{"shards":"0,2","objects":2362}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103316+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:02:08.103316+0000","last_peered":"2024-09-10T16:02:08.103316+0000","last_clean":"2024-09-10T16:02:08.103316+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:02:08.103316+0000","last_undegraded":"2024-09-10T16:02:08.103316+0000","last_fullsized":"2024-09-10T16:02:08.103316+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'12171","reported_seq":15781,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:43.511681+0000","last_change":"2024-09-10T16:03:29.605682+0000","last_active":"2024-09-10T16:03:43.511681+0000","last_peered":"2024-09-10T16:03:43.511681+0000","last_clean":"2024-09-10T15:59:14.115595+0000","last_became_active":"2024-09-10T15:59:16.244089+0000","last_became_peered":"2024-09-10T15:59:16.244089+0000","last_unstale":"2024-09-10T16:03:43.511681+0000","last_undegraded":"2024-09-10T16:03:29.603815+0000","last_fullsized":"2024-09-10T15:59:15.774437+0000","mapping_epoch":42,"log_start":"36'2100","ondisk_log_start":"36'2100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34516992,"num_objects":8427,"num_object_clones":0,"num_object_copies":16854,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":139,"num_objects_misplaced":2331,"num_objects_unfound":0,"num_objects_dirty":8427,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8561,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1733,"num_bytes_recovered":7098368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5957},{"shards":"0,2","objects":2331}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103831+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:02:08.103831+0000","last_peered":"2024-09-10T16:02:08.103831+0000","last_clean":"2024-09-10T16:02:08.103831+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:02:08.103831+0000","last_undegraded":"2024-09-10T16:02:08.103831+0000","last_fullsized":"2024-09-10T16:02:08.103831+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'12538","reported_seq":12679,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:43.431039+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:03:43.431039+0000","last_peered":"2024-09-10T16:03:43.431039+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:03:43.431039+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"36'2500","ondisk_log_start":"36'2500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34856960,"num_objects":8510,"num_object_clones":0,"num_object_copies":17020,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8510,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8793,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4139},{"shards":"0,2","objects":2359}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104463+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:02:08.104463+0000","last_peered":"2024-09-10T16:02:08.104463+0000","last_clean":"2024-09-10T16:02:08.104463+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:02:08.104463+0000","last_undegraded":"2024-09-10T16:02:08.104463+0000","last_fullsized":"2024-09-10T16:02:08.104463+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104448+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:02:08.104448+0000","last_peered":"2024-09-10T16:02:08.104448+0000","last_clean":"2024-09-10T16:02:08.104448+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:02:08.104448+0000","last_undegraded":"2024-09-10T16:02:08.104448+0000","last_fullsized":"2024-09-10T16:02:08.104448+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'12520","reported_seq":12665,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:43.416344+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:03:43.416344+0000","last_peered":"2024-09-10T16:03:43.416344+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:03:43.416344+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"36'2500","ondisk_log_start":"36'2500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35546054,"num_objects":8680,"num_object_clones":0,"num_object_copies":17360,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8680,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8683,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4357},{"shards":"0,2","objects":2408}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":556782326,"num_objects":135943,"num_object_clones":0,"num_object_copies":271886,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43050,"num_objects_misplaced":2331,"num_objects_unfound":0,"num_objects_dirty":135943,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":156163,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6005,"num_bytes_recovered":24592384,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1218871296,"data_stored":1218711406,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160889,"ondisk_log_size":160889,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986245,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":710852,"kb_used_data":408432,"kb_used_omap":4,"kb_used_meta":302395,"kb_avail":93660988,"statfs":{"total":96636764160,"available":95908851712,"internally_reserved":0,"allocated":418234368,"data_stored":417942135,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":309653162},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,1,1,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659191,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":567400,"kb_used_data":305240,"kb_used_omap":4,"kb_used_meta":262139,"kb_avail":93804440,"statfs":{"total":96636764160,"available":96055746560,"internally_reserved":0,"allocated":312565760,"data_stored":312226094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":268431022},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,1,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182473,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":861164,"kb_used_data":478296,"kb_used_omap":4,"kb_used_meta":382843,"kb_avail":93510676,"statfs":{"total":96636764160,"available":95754932224,"internally_reserved":0,"allocated":489775104,"data_stored":489482866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":392031914},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,3,4,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":488960000,"data_stored":488923870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":312127488,"data_stored":312043930,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":417783808,"data_stored":417743606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:03:49.212 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-10T16:03:49.212 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "45'12235", 'reported_seq': 12310, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:03:43.473199+0000', 'last_change': '2024-09-10T15:59:16.281558+0000', 'last_active': '2024-09-10T16:03:43.473199+0000', 'last_peered': '2024-09-10T16:03:43.473199+0000', 'last_clean': '2024-09-10T15:59:14.132456+0000', 'last_became_active': '2024-09-10T15:59:16.274407+0000', 'last_became_peered': '2024-09-10T15:59:16.274407+0000', 'last_unstale': '2024-09-10T16:03:43.473199+0000', 'last_undegraded': '2024-09-10T15:59:15.851279+0000', 'last_fullsized': '2024-09-10T15:59:15.774356+0000', 'mapping_epoch': 42, 'log_start': "36'2200", 'ondisk_log_start': "36'2200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'objects_scrubbed': 0, 'log_size': 10035, 'log_dups_size': 0, 'ondisk_log_size': 10035, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34557952, 'num_objects': 8437, 'num_object_clones': 0, 'num_object_copies': 16874, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4259, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8437, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12235, 'num_write_kb': 41344, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4178}, {'shards': '0,2', 'objects': 2359}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:03:49.213 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-10T16:03:49.213 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "45'12525", 'reported_seq': 12663, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:03:43.484969+0000', 'last_change': '2024-09-10T15:59:16.308264+0000', 'last_active': '2024-09-10T16:03:43.484969+0000', 'last_peered': '2024-09-10T16:03:43.484969+0000', 'last_clean': '2024-09-10T15:59:14.111551+0000', 'last_became_active': '2024-09-10T15:59:16.300040+0000', 'last_became_peered': '2024-09-10T15:59:16.300040+0000', 'last_unstale': '2024-09-10T16:03:43.484969+0000', 'last_undegraded': '2024-09-10T15:59:15.878234+0000', 'last_fullsized': '2024-09-10T15:59:15.769328+0000', 'mapping_epoch': 42, 'log_start': "36'2500", 'ondisk_log_start': "36'2500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3442", 'last_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'objects_scrubbed': 3384, 'log_size': 10025, 'log_dups_size': 0, 'ondisk_log_size': 10025, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1292, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34963939, 'num_objects': 8537, 'num_object_clones': 0, 'num_object_copies': 17074, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4372, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8537, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 9137, 'num_write_kb': 28580, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4166}, {'shards': '0,2', 'objects': 2379}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:03:49.213 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-10T16:03:49.213 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "45'12467", 'reported_seq': 12605, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:03:43.260154+0000', 'last_change': '2024-09-10T15:59:16.331899+0000', 'last_active': '2024-09-10T16:03:43.260154+0000', 'last_peered': '2024-09-10T16:03:43.260154+0000', 'last_clean': '2024-09-10T15:59:14.139436+0000', 'last_became_active': '2024-09-10T15:59:16.318715+0000', 'last_became_peered': '2024-09-10T15:59:16.318715+0000', 'last_unstale': '2024-09-10T16:03:43.260154+0000', 'last_undegraded': '2024-09-10T15:59:15.877524+0000', 'last_fullsized': '2024-09-10T15:59:15.769042+0000', 'mapping_epoch': 42, 'log_start': "36'2400", 'ondisk_log_start': "36'2400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3537", 'last_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'objects_scrubbed': 3497, 'log_size': 10067, 'log_dups_size': 0, 'ondisk_log_size': 10067, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1224, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35288006, 'num_objects': 8617, 'num_object_clones': 0, 'num_object_copies': 17234, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4237, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8617, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 8966, 'num_write_kb': 28172, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4380}, {'shards': '0,2', 'objects': 2314}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:03:49.213 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-10T16:03:49.213 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "45'12168", 'reported_seq': 12308, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:03:44.814437+0000', 'last_change': '2024-09-10T15:59:16.175680+0000', 'last_active': '2024-09-10T16:03:44.814437+0000', 'last_peered': '2024-09-10T16:03:44.814437+0000', 'last_clean': '2024-09-10T15:59:14.088187+0000', 'last_became_active': '2024-09-10T15:59:16.165810+0000', 'last_became_peered': '2024-09-10T15:59:16.165810+0000', 'last_unstale': '2024-09-10T16:03:44.814437+0000', 'last_undegraded': '2024-09-10T15:59:15.822098+0000', 'last_fullsized': '2024-09-10T15:59:15.769123+0000', 'mapping_epoch': 42, 'log_start': "36'2100", 'ondisk_log_start': "36'2100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2218", 'last_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'objects_scrubbed': 2175, 'log_size': 10068, 'log_dups_size': 0, 'ondisk_log_size': 10068, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 825, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33952227, 'num_objects': 8290, 'num_object_clones': 0, 'num_object_copies': 16580, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4124, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8290, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 9987, 'num_write_kb': 32204, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2187}, {'shards': '2', 'objects': 4167}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:03:49.214 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-10T16:03:49.214 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "45'12376", 'reported_seq': 12507, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:03:43.485072+0000', 'last_change': '2024-09-10T15:59:16.277742+0000', 'last_active': '2024-09-10T16:03:43.485072+0000', 'last_peered': '2024-09-10T16:03:43.485072+0000', 'last_clean': '2024-09-10T15:59:14.129081+0000', 'last_became_active': '2024-09-10T15:59:16.269835+0000', 'last_became_peered': '2024-09-10T15:59:16.269835+0000', 'last_unstale': '2024-09-10T16:03:43.485072+0000', 'last_undegraded': '2024-09-10T15:59:15.860857+0000', 'last_fullsized': '2024-09-10T15:59:15.771410+0000', 'mapping_epoch': 42, 'log_start': "36'2300", 'ondisk_log_start': "36'2300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3557", 'last_scrub_stamp': '2024-09-10T15:59:02.493678+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:02.493678+0000', 'objects_scrubbed': 3488, 'log_size': 10076, 'log_dups_size': 0, 'ondisk_log_size': 10076, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1472, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34955264, 'num_objects': 8534, 'num_object_clones': 0, 'num_object_copies': 17068, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4257, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8534, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8880, 'num_write_kb': 27852, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4279}, {'shards': '0,2', 'objects': 2338}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:03:49.214 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-10T16:03:49.214 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "45'12400", 'reported_seq': 12535, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:03:44.950962+0000', 'last_change': '2024-09-10T15:59:16.284311+0000', 'last_active': '2024-09-10T16:03:44.950962+0000', 'last_peered': '2024-09-10T16:03:44.950962+0000', 'last_clean': '2024-09-10T15:59:14.116041+0000', 'last_became_active': '2024-09-10T15:59:16.274543+0000', 'last_became_peered': '2024-09-10T15:59:16.274543+0000', 'last_unstale': '2024-09-10T16:03:44.950962+0000', 'last_undegraded': '2024-09-10T15:59:15.812484+0000', 'last_fullsized': '2024-09-10T15:59:15.770885+0000', 'mapping_epoch': 42, 'log_start': "36'2300", 'ondisk_log_start': "36'2300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2356", 'last_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'objects_scrubbed': 2316, 'log_size': 10100, 'log_dups_size': 0, 'ondisk_log_size': 10100, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 783, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34329059, 'num_objects': 8382, 'num_object_clones': 0, 'num_object_copies': 16764, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8382, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 10080, 'num_write_kb': 32292, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2297}, {'shards': '2', 'objects': 4078}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:03:49.214 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-10T16:03:49.215 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "45'12558", 'reported_seq': 12687, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:03:44.980090+0000', 'last_change': '2024-09-10T15:59:16.174113+0000', 'last_active': '2024-09-10T16:03:44.980090+0000', 'last_peered': '2024-09-10T16:03:44.980090+0000', 'last_clean': '2024-09-10T15:59:14.110654+0000', 'last_became_active': '2024-09-10T15:59:16.164645+0000', 'last_became_peered': '2024-09-10T15:59:16.164645+0000', 'last_unstale': '2024-09-10T16:03:44.980090+0000', 'last_undegraded': '2024-09-10T15:59:15.803593+0000', 'last_fullsized': '2024-09-10T15:59:15.769083+0000', 'mapping_epoch': 42, 'log_start': "36'2500", 'ondisk_log_start': "36'2500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2486", 'last_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'objects_scrubbed': 2440, 'log_size': 10058, 'log_dups_size': 0, 'ondisk_log_size': 10058, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 792, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35274752, 'num_objects': 8612, 'num_object_clones': 0, 'num_object_copies': 17224, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4354, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8612, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10112, 'num_write_kb': 32568, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2382}, {'shards': '2', 'objects': 4260}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:03:49.215 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-10T16:03:49.215 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "45'12362", 'reported_seq': 12497, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:03:43.376763+0000', 'last_change': '2024-09-10T15:59:16.315641+0000', 'last_active': '2024-09-10T16:03:43.376763+0000', 'last_peered': '2024-09-10T16:03:43.376763+0000', 'last_clean': '2024-09-10T15:59:14.117001+0000', 'last_became_active': '2024-09-10T15:59:16.302245+0000', 'last_became_peered': '2024-09-10T15:59:16.302245+0000', 'last_unstale': '2024-09-10T16:03:43.376763+0000', 'last_undegraded': '2024-09-10T15:59:15.845012+0000', 'last_fullsized': '2024-09-10T15:59:15.770337+0000', 'mapping_epoch': 42, 'log_start': "36'2300", 'ondisk_log_start': "36'2300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3404", 'last_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'objects_scrubbed': 3367, 'log_size': 10062, 'log_dups_size': 0, 'ondisk_log_size': 10062, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 983, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34717696, 'num_objects': 8476, 'num_object_clones': 0, 'num_object_copies': 16952, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8476, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8993, 'num_write_kb': 28204, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4172}, {'shards': '0,2', 'objects': 2362}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:03:49.215 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-10T16:03:49.215 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "45'12171", 'reported_seq': 15781, 'reported_epoch': 45, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:03:43.511681+0000', 'last_change': '2024-09-10T16:03:29.605682+0000', 'last_active': '2024-09-10T16:03:43.511681+0000', 'last_peered': '2024-09-10T16:03:43.511681+0000', 'last_clean': '2024-09-10T15:59:14.115595+0000', 'last_became_active': '2024-09-10T15:59:16.244089+0000', 'last_became_peered': '2024-09-10T15:59:16.244089+0000', 'last_unstale': '2024-09-10T16:03:43.511681+0000', 'last_undegraded': '2024-09-10T16:03:29.603815+0000', 'last_fullsized': '2024-09-10T15:59:15.774437+0000', 'mapping_epoch': 42, 'log_start': "36'2100", 'ondisk_log_start': "36'2100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3653", 'last_scrub_stamp': '2024-09-10T15:59:05.298468+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:05.298468+0000', 'objects_scrubbed': 3602, 'log_size': 10071, 'log_dups_size': 0, 'ondisk_log_size': 10071, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1289, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34516992, 'num_objects': 8427, 'num_object_clones': 0, 'num_object_copies': 16854, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 139, 'num_objects_misplaced': 2331, 'num_objects_unfound': 0, 'num_objects_dirty': 8427, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8561, 'num_write_kb': 26772, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1733, 'num_bytes_recovered': 7098368, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5957}, {'shards': '0,2', 'objects': 2331}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:03:49.215 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-10T16:03:49.216 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "45'12538", 'reported_seq': 12679, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:03:43.431039+0000', 'last_change': '2024-09-10T15:59:16.311291+0000', 'last_active': '2024-09-10T16:03:43.431039+0000', 'last_peered': '2024-09-10T16:03:43.431039+0000', 'last_clean': '2024-09-10T15:59:14.116363+0000', 'last_became_active': '2024-09-10T15:59:16.301244+0000', 'last_became_peered': '2024-09-10T15:59:16.301244+0000', 'last_unstale': '2024-09-10T16:03:43.431039+0000', 'last_undegraded': '2024-09-10T15:59:15.883434+0000', 'last_fullsized': '2024-09-10T15:59:15.772756+0000', 'mapping_epoch': 42, 'log_start': "36'2500", 'ondisk_log_start': "36'2500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3789", 'last_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'objects_scrubbed': 3741, 'log_size': 10038, 'log_dups_size': 0, 'ondisk_log_size': 10038, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1440, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34856960, 'num_objects': 8510, 'num_object_clones': 0, 'num_object_copies': 17020, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4373, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8510, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8793, 'num_write_kb': 27124, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4139}, {'shards': '0,2', 'objects': 2359}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:03:49.216 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-10T16:03:49.216 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "45'12520", 'reported_seq': 12665, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:03:43.416344+0000', 'last_change': '2024-09-10T15:59:16.313409+0000', 'last_active': '2024-09-10T16:03:43.416344+0000', 'last_peered': '2024-09-10T16:03:43.416344+0000', 'last_clean': '2024-09-10T15:59:14.129780+0000', 'last_became_active': '2024-09-10T15:59:16.300932+0000', 'last_became_peered': '2024-09-10T15:59:16.300932+0000', 'last_unstale': '2024-09-10T16:03:43.416344+0000', 'last_undegraded': '2024-09-10T15:59:15.887823+0000', 'last_fullsized': '2024-09-10T15:59:15.768513+0000', 'mapping_epoch': 42, 'log_start': "36'2500", 'ondisk_log_start': "36'2500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3880", 'last_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'objects_scrubbed': 3827, 'log_size': 10020, 'log_dups_size': 0, 'ondisk_log_size': 10020, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1397, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35546054, 'num_objects': 8680, 'num_object_clones': 0, 'num_object_copies': 17360, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4325, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8680, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8683, 'num_write_kb': 27072, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4357}, {'shards': '0,2', 'objects': 2408}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:03:49.216 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T16:03:49.926 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:03:49.926 INFO:teuthology.orchestra.run.smithi115.stdout:{"fsid":"7effee29-df05-4351-9741-ee203564ed81","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 43050/271890 objects degraded (15.834%), 11 pgs degraded, 11 pgs undersized","count":22},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":393,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983954,"num_in_osds":3,"osd_in_since":1725983947,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":135945,"data_bytes":556782345,"bytes_used":2190761984,"bytes_avail":287719530496,"bytes_total":289910292480,"degraded_objects":43050,"degraded_total":271890,"degraded_ratio":0.15833609180183161,"misplaced_objects":2331,"misplaced_total":271890,"misplaced_ratio":0.0085733200926845412,"recovering_objects_per_sec":16,"recovering_bytes_per_sec":66541,"recovering_keys_per_sec":0,"num_objects_recovered":195,"num_bytes_recovered":798720,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":378},"fsmap":{"epoch":1,"btime":"2024-09-10T15:57:14:967120+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T16:00:13.560776+0000","services":{}},"progress_events":{}} 2024-09-10T16:03:49.956 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-10T16:03:49.956 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:03:50.519 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:03:50.520 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:03:50.540 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":235,"stamp":"2024-09-10T16:03:49.646190+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":551093001,"num_objects":134556,"num_object_clones":0,"num_object_copies":269112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43086,"num_objects_misplaced":2194,"num_objects_unfound":0,"num_objects_dirty":134556,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":157554,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6108,"num_bytes_recovered":25006099,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160981,"ondisk_log_size":160981,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2124552,"kb_used_data":1185428,"kb_used_omap":12,"kb_used_meta":939059,"kb_avail":280990968,"statfs":{"total":289910292480,"available":287734751232,"internally_reserved":0,"allocated":1213878272,"data_stored":1212954135,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":961596418},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,0,0,5,5,5,7,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-23785472,"num_objects":-5807,"num_object_clones":0,"num_object_copies":-11614,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":150,"num_objects_misplaced":-446,"num_objects_unfound":0,"num_objects_dirty":-5807,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5807,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":296,"num_bytes_recovered":1212416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003885"},"pg_stats":[{"pgid":"2.f","version":"45'12369","reported_seq":12443,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:48.534496+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:03:48.534496+0000","last_peered":"2024-09-10T16:03:48.534496+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:03:48.534496+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"36'2300","ondisk_log_start":"36'2300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34009088,"num_objects":8303,"num_object_clones":0,"num_object_copies":16606,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8303,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12369,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4044},{"shards":"0,2","objects":2225}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'12392","reported_seq":12505,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:03:44.936172+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:03:44.936172+0000","last_peered":"2024-09-10T16:03:44.936172+0000","last_clean":"2024-09-10T16:03:44.936172+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:03:44.936172+0000","last_undegraded":"2024-09-10T16:03:44.936172+0000","last_fullsized":"2024-09-10T16:03:44.936172+0000","mapping_epoch":35,"log_start":"36'2300","ondisk_log_start":"36'2300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34620358,"num_objects":8454,"num_object_clones":0,"num_object_copies":16908,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8454,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":10497,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'12398","reported_seq":21076,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.143495+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:03:46.143495+0000","last_peered":"2024-09-10T16:03:46.143495+0000","last_clean":"2024-09-10T16:03:46.143495+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:03:46.143495+0000","last_undegraded":"2024-09-10T16:03:46.143495+0000","last_fullsized":"2024-09-10T16:03:46.143495+0000","mapping_epoch":44,"log_start":"36'2300","ondisk_log_start":"36'2300","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34402328,"num_objects":8400,"num_object_clones":0,"num_object_copies":16800,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8400,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10402,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'12329","reported_seq":12410,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:03:45.027842+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:03:45.027842+0000","last_peered":"2024-09-10T16:03:45.027842+0000","last_clean":"2024-09-10T16:03:45.027842+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:03:45.027842+0000","last_undegraded":"2024-09-10T16:03:45.027842+0000","last_fullsized":"2024-09-10T16:03:45.027842+0000","mapping_epoch":35,"log_start":"36'2300","ondisk_log_start":"36'2300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34123776,"num_objects":8331,"num_object_clones":0,"num_object_copies":16662,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8331,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12329,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'12490","reported_seq":12620,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:03:48.417536+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:03:48.417536+0000","last_peered":"2024-09-10T16:03:48.417536+0000","last_clean":"2024-09-10T16:03:48.417536+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:03:48.417536+0000","last_undegraded":"2024-09-10T16:03:48.417536+0000","last_fullsized":"2024-09-10T16:03:48.417536+0000","mapping_epoch":34,"log_start":"36'2400","ondisk_log_start":"36'2400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34410979,"num_objects":8402,"num_object_clones":0,"num_object_copies":16804,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8402,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9201,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'12660","reported_seq":12796,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:48.514398+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:03:48.514398+0000","last_peered":"2024-09-10T16:03:48.514398+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:03:48.514398+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"36'2600","ondisk_log_start":"36'2600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34410979,"num_objects":8402,"num_object_clones":0,"num_object_copies":16804,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8402,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9272,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4031},{"shards":"0,2","objects":2244}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'12609","reported_seq":12745,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:48.508534+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:03:48.508534+0000","last_peered":"2024-09-10T16:03:48.508534+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:03:48.508534+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"36'2600","ondisk_log_start":"36'2600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34706374,"num_objects":8475,"num_object_clones":0,"num_object_copies":16950,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8475,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":9108,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4238},{"shards":"0,2","objects":2172}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'12730","reported_seq":12860,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:03:48.490107+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:03:48.490107+0000","last_peered":"2024-09-10T16:03:48.490107+0000","last_clean":"2024-09-10T16:03:48.490107+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:03:48.490107+0000","last_undegraded":"2024-09-10T16:03:48.490107+0000","last_fullsized":"2024-09-10T16:03:48.490107+0000","mapping_epoch":34,"log_start":"36'2700","ondisk_log_start":"36'2700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35160064,"num_objects":8584,"num_object_clones":0,"num_object_copies":17168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8584,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9577,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104261+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:02:08.104261+0000","last_peered":"2024-09-10T16:02:08.104261+0000","last_clean":"2024-09-10T16:02:08.104261+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:02:08.104261+0000","last_undegraded":"2024-09-10T16:02:08.104261+0000","last_fullsized":"2024-09-10T16:02:08.104261+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'12168","reported_seq":12308,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:44.814437+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:03:44.814437+0000","last_peered":"2024-09-10T16:03:44.814437+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:03:44.814437+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"36'2100","ondisk_log_start":"36'2100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33952227,"num_objects":8290,"num_object_clones":0,"num_object_copies":16580,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8290,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9987,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2187},{"shards":"2","objects":4167}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103774+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:02:08.103774+0000","last_peered":"2024-09-10T16:02:08.103774+0000","last_clean":"2024-09-10T16:02:08.103774+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:02:08.103774+0000","last_undegraded":"2024-09-10T16:02:08.103774+0000","last_fullsized":"2024-09-10T16:02:08.103774+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'12509","reported_seq":12640,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:48.449027+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:03:48.449027+0000","last_peered":"2024-09-10T16:03:48.449027+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:03:48.449027+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"36'2500","ondisk_log_start":"36'2500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34410496,"num_objects":8401,"num_object_clones":0,"num_object_copies":16802,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8401,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9013,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4146},{"shards":"0,2","objects":2205}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":104,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.105055+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:02:08.105055+0000","last_peered":"2024-09-10T16:02:08.105055+0000","last_clean":"2024-09-10T16:02:08.105055+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:02:08.105055+0000","last_undegraded":"2024-09-10T16:02:08.105055+0000","last_fullsized":"2024-09-10T16:02:08.105055+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'12400","reported_seq":12535,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:44.950962+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:03:44.950962+0000","last_peered":"2024-09-10T16:03:44.950962+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:03:44.950962+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"36'2300","ondisk_log_start":"36'2300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34329059,"num_objects":8382,"num_object_clones":0,"num_object_copies":16764,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8382,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10080,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2297},{"shards":"2","objects":4078}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104617+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:02:08.104617+0000","last_peered":"2024-09-10T16:02:08.104617+0000","last_clean":"2024-09-10T16:02:08.104617+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:02:08.104617+0000","last_undegraded":"2024-09-10T16:02:08.104617+0000","last_fullsized":"2024-09-10T16:02:08.104617+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'12558","reported_seq":12687,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:44.980090+0000","last_change":"2024-09-10T15:59:16.174113+0000","last_active":"2024-09-10T16:03:44.980090+0000","last_peered":"2024-09-10T16:03:44.980090+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:03:44.980090+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"36'2500","ondisk_log_start":"36'2500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35274752,"num_objects":8612,"num_object_clones":0,"num_object_copies":17224,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4354,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8612,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10112,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2382},{"shards":"2","objects":4260}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'12497","reported_seq":12632,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:48.487702+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:03:48.487702+0000","last_peered":"2024-09-10T16:03:48.487702+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:03:48.487702+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"36'2400","ondisk_log_start":"36'2400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34164736,"num_objects":8341,"num_object_clones":0,"num_object_copies":16682,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8341,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9128,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4037},{"shards":"0,2","objects":2227}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103316+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:02:08.103316+0000","last_peered":"2024-09-10T16:02:08.103316+0000","last_clean":"2024-09-10T16:02:08.103316+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:02:08.103316+0000","last_undegraded":"2024-09-10T16:02:08.103316+0000","last_fullsized":"2024-09-10T16:02:08.103316+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'12312","reported_seq":16124,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:48.498111+0000","last_change":"2024-09-10T16:03:29.605682+0000","last_active":"2024-09-10T16:03:48.498111+0000","last_peered":"2024-09-10T16:03:48.498111+0000","last_clean":"2024-09-10T15:59:14.115595+0000","last_became_active":"2024-09-10T15:59:16.244089+0000","last_became_peered":"2024-09-10T15:59:16.244089+0000","last_unstale":"2024-09-10T16:03:48.498111+0000","last_undegraded":"2024-09-10T16:03:29.603815+0000","last_fullsized":"2024-09-10T15:59:15.774437+0000","mapping_epoch":42,"log_start":"36'2300","ondisk_log_start":"36'2300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33939456,"num_objects":8286,"num_object_clones":0,"num_object_copies":16572,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":175,"num_objects_misplaced":2194,"num_objects_unfound":0,"num_objects_dirty":8286,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8702,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1834,"num_bytes_recovered":7512064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5917},{"shards":"0,2","objects":2194}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103831+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:02:08.103831+0000","last_peered":"2024-09-10T16:02:08.103831+0000","last_clean":"2024-09-10T16:02:08.103831+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:02:08.103831+0000","last_undegraded":"2024-09-10T16:02:08.103831+0000","last_fullsized":"2024-09-10T16:02:08.103831+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'12679","reported_seq":12821,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:48.475907+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:03:48.475907+0000","last_peered":"2024-09-10T16:03:48.475907+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:03:48.475907+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"36'2600","ondisk_log_start":"36'2600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34279424,"num_objects":8369,"num_object_clones":0,"num_object_copies":16738,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8369,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8934,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3998},{"shards":"0,2","objects":2218}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104463+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:02:08.104463+0000","last_peered":"2024-09-10T16:02:08.104463+0000","last_clean":"2024-09-10T16:02:08.104463+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:02:08.104463+0000","last_undegraded":"2024-09-10T16:02:08.104463+0000","last_fullsized":"2024-09-10T16:02:08.104463+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104448+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:02:08.104448+0000","last_peered":"2024-09-10T16:02:08.104448+0000","last_clean":"2024-09-10T16:02:08.104448+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:02:08.104448+0000","last_undegraded":"2024-09-10T16:02:08.104448+0000","last_fullsized":"2024-09-10T16:02:08.104448+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'12678","reported_seq":12823,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:48.506036+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:03:48.506036+0000","last_peered":"2024-09-10T16:03:48.506036+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:03:48.506036+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"36'2600","ondisk_log_start":"36'2600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34898886,"num_objects":8522,"num_object_clones":0,"num_object_copies":17044,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8522,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8841,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4199},{"shards":"0,2","objects":2250}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":551092982,"num_objects":134554,"num_object_clones":0,"num_object_copies":269108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43086,"num_objects_misplaced":2194,"num_objects_unfound":0,"num_objects_dirty":134554,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":157552,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6106,"num_bytes_recovered":25006080,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1211985920,"data_stored":1211826030,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160978,"ondisk_log_size":160978,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986245,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":710852,"kb_used_data":408432,"kb_used_omap":4,"kb_used_meta":302395,"kb_avail":93660988,"statfs":{"total":96636764160,"available":95908851712,"internally_reserved":0,"allocated":418234368,"data_stored":417942135,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":309653162},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,1,1,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659191,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":567400,"kb_used_data":305240,"kb_used_omap":4,"kb_used_meta":262139,"kb_avail":93804440,"statfs":{"total":96636764160,"available":96055746560,"internally_reserved":0,"allocated":312565760,"data_stored":312226094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":268431022},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,1,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182474,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":846300,"kb_used_data":471756,"kb_used_omap":4,"kb_used_meta":374523,"kb_avail":93525540,"statfs":{"total":96636764160,"available":95770152960,"internally_reserved":0,"allocated":483078144,"data_stored":482785906,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":383512234},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,0,0,1,3,3,6,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":482074624,"data_stored":482038494,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":312127488,"data_stored":312043930,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":417783808,"data_stored":417743606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:03:53.543 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:03:54.115 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:03:54.118 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:03:54.140 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":237,"stamp":"2024-09-10T16:03:53.647155+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":541930249,"num_objects":132319,"num_object_clones":0,"num_object_copies":264638,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43122,"num_objects_misplaced":2057,"num_objects_unfound":0,"num_objects_dirty":132319,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":159791,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6209,"num_bytes_recovered":25419795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160818,"ondisk_log_size":160818,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2125716,"kb_used_data":1174112,"kb_used_omap":12,"kb_used_meta":951539,"kb_avail":280989804,"statfs":{"total":289910292480,"available":287733559296,"internally_reserved":0,"allocated":1202290688,"data_stored":1201366551,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":974375938},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,6,5,11,8,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-23801856,"num_objects":-5811,"num_object_clones":0,"num_object_copies":-11622,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":116,"num_objects_misplaced":-420,"num_objects_unfound":0,"num_objects_dirty":-5811,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5811,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":304,"num_bytes_recovered":1245184,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003876"},"pg_stats":[{"pgid":"2.f","version":"45'12513","reported_seq":12588,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:53.468920+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:03:53.468920+0000","last_peered":"2024-09-10T16:03:53.468920+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:03:53.468920+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"36'2500","ondisk_log_start":"36'2500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33419264,"num_objects":8159,"num_object_clones":0,"num_object_copies":16318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8159,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12513,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3900},{"shards":"0,2","objects":2081}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'12549","reported_seq":12662,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:03:50.048510+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:03:50.048510+0000","last_peered":"2024-09-10T16:03:50.048510+0000","last_clean":"2024-09-10T16:03:50.048510+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:03:50.048510+0000","last_undegraded":"2024-09-10T16:03:50.048510+0000","last_fullsized":"2024-09-10T16:03:50.048510+0000","mapping_epoch":35,"log_start":"36'2500","ondisk_log_start":"36'2500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33977286,"num_objects":8297,"num_object_clones":0,"num_object_copies":16594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8297,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":10654,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'12544","reported_seq":21222,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:03:51.217066+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:03:51.217066+0000","last_peered":"2024-09-10T16:03:51.217066+0000","last_clean":"2024-09-10T16:03:51.217066+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:03:51.217066+0000","last_undegraded":"2024-09-10T16:03:51.217066+0000","last_fullsized":"2024-09-10T16:03:51.217066+0000","mapping_epoch":44,"log_start":"36'2500","ondisk_log_start":"36'2500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33804312,"num_objects":8254,"num_object_clones":0,"num_object_copies":16508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8254,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10548,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'12465","reported_seq":12542,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:03:49.966956+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:03:49.966956+0000","last_peered":"2024-09-10T16:03:49.966956+0000","last_clean":"2024-09-10T16:03:49.966956+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:03:49.966956+0000","last_undegraded":"2024-09-10T16:03:49.966956+0000","last_fullsized":"2024-09-10T16:03:49.966956+0000","mapping_epoch":35,"log_start":"36'2400","ondisk_log_start":"36'2400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33566720,"num_objects":8195,"num_object_clones":0,"num_object_copies":16390,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8195,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12465,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'12652","reported_seq":12781,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:03:53.552877+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:03:53.552877+0000","last_peered":"2024-09-10T16:03:53.552877+0000","last_clean":"2024-09-10T16:03:53.552877+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:03:53.552877+0000","last_undegraded":"2024-09-10T16:03:53.552877+0000","last_fullsized":"2024-09-10T16:03:53.552877+0000","mapping_epoch":34,"log_start":"36'2600","ondisk_log_start":"36'2600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33747427,"num_objects":8240,"num_object_clones":0,"num_object_copies":16480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8240,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9363,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'12789","reported_seq":12927,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:53.461652+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:03:53.461652+0000","last_peered":"2024-09-10T16:03:53.461652+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:03:53.461652+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"36'2700","ondisk_log_start":"36'2700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33882595,"num_objects":8273,"num_object_clones":0,"num_object_copies":16546,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8273,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9401,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3902},{"shards":"0,2","objects":2115}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'12733","reported_seq":12868,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:53.496022+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:03:53.496022+0000","last_peered":"2024-09-10T16:03:53.496022+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:03:53.496022+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"36'2700","ondisk_log_start":"36'2700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34198470,"num_objects":8351,"num_object_clones":0,"num_object_copies":16702,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8351,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":9232,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4114},{"shards":"0,2","objects":2048}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'12854","reported_seq":12981,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:03:53.547210+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:03:53.547210+0000","last_peered":"2024-09-10T16:03:53.547210+0000","last_clean":"2024-09-10T16:03:53.547210+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:03:53.547210+0000","last_undegraded":"2024-09-10T16:03:53.547210+0000","last_fullsized":"2024-09-10T16:03:53.547210+0000","mapping_epoch":34,"log_start":"36'2800","ondisk_log_start":"36'2800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34652160,"num_objects":8460,"num_object_clones":0,"num_object_copies":16920,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8460,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9701,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104261+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:02:08.104261+0000","last_peered":"2024-09-10T16:02:08.104261+0000","last_clean":"2024-09-10T16:02:08.104261+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:02:08.104261+0000","last_undegraded":"2024-09-10T16:02:08.104261+0000","last_fullsized":"2024-09-10T16:02:08.104261+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'12294","reported_seq":12434,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:49.905468+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:03:49.905468+0000","last_peered":"2024-09-10T16:03:49.905468+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:03:49.905468+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"36'2200","ondisk_log_start":"36'2200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33436131,"num_objects":8164,"num_object_clones":0,"num_object_copies":16328,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8164,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10113,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2061},{"shards":"2","objects":4041}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103774+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:02:08.103774+0000","last_peered":"2024-09-10T16:02:08.103774+0000","last_clean":"2024-09-10T16:02:08.103774+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:02:08.103774+0000","last_undegraded":"2024-09-10T16:02:08.103774+0000","last_fullsized":"2024-09-10T16:02:08.103774+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'12661","reported_seq":12792,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:53.462645+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:03:53.462645+0000","last_peered":"2024-09-10T16:03:53.462645+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:03:53.462645+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"36'2600","ondisk_log_start":"36'2600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33787904,"num_objects":8249,"num_object_clones":0,"num_object_copies":16498,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8249,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9165,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3994},{"shards":"0,2","objects":2053}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":104,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.105055+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:02:08.105055+0000","last_peered":"2024-09-10T16:02:08.105055+0000","last_clean":"2024-09-10T16:02:08.105055+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:02:08.105055+0000","last_undegraded":"2024-09-10T16:02:08.105055+0000","last_fullsized":"2024-09-10T16:02:08.105055+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'12522","reported_seq":12657,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:49.968852+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:03:49.968852+0000","last_peered":"2024-09-10T16:03:49.968852+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:03:49.968852+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"36'2500","ondisk_log_start":"36'2500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33829347,"num_objects":8260,"num_object_clones":0,"num_object_copies":16520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8260,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10202,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2175},{"shards":"2","objects":3956}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104617+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:02:08.104617+0000","last_peered":"2024-09-10T16:02:08.104617+0000","last_clean":"2024-09-10T16:02:08.104617+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:02:08.104617+0000","last_undegraded":"2024-09-10T16:02:08.104617+0000","last_fullsized":"2024-09-10T16:02:08.104617+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'12695","reported_seq":12824,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:49.986919+0000","last_change":"2024-09-10T15:59:16.174113+0000","last_active":"2024-09-10T16:03:49.986919+0000","last_peered":"2024-09-10T16:03:49.986919+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:03:49.986919+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"36'2600","ondisk_log_start":"36'2600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34713600,"num_objects":8475,"num_object_clones":0,"num_object_copies":16950,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4354,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8475,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10249,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2245},{"shards":"2","objects":4123}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'12650","reported_seq":12785,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:53.415937+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:03:53.415937+0000","last_peered":"2024-09-10T16:03:53.415937+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:03:53.415937+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"36'2600","ondisk_log_start":"36'2600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33538048,"num_objects":8188,"num_object_clones":0,"num_object_copies":16376,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8188,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9281,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3884},{"shards":"0,2","objects":2074}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103316+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:02:08.103316+0000","last_peered":"2024-09-10T16:02:08.103316+0000","last_clean":"2024-09-10T16:02:08.103316+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:02:08.103316+0000","last_undegraded":"2024-09-10T16:02:08.103316+0000","last_fullsized":"2024-09-10T16:02:08.103316+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'12448","reported_seq":16462,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:53.547659+0000","last_change":"2024-09-10T16:03:29.605682+0000","last_active":"2024-09-10T16:03:53.547659+0000","last_peered":"2024-09-10T16:03:53.547659+0000","last_clean":"2024-09-10T15:59:14.115595+0000","last_became_active":"2024-09-10T15:59:16.244089+0000","last_became_peered":"2024-09-10T15:59:16.244089+0000","last_unstale":"2024-09-10T16:03:53.547659+0000","last_undegraded":"2024-09-10T16:03:29.603815+0000","last_fullsized":"2024-09-10T15:59:15.774437+0000","mapping_epoch":42,"log_start":"36'2400","ondisk_log_start":"36'2400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33382400,"num_objects":8150,"num_object_clones":0,"num_object_copies":16300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":211,"num_objects_misplaced":2057,"num_objects_unfound":0,"num_objects_dirty":8150,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8838,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1935,"num_bytes_recovered":7925760,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5882},{"shards":"0,2","objects":2057}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103831+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:02:08.103831+0000","last_peered":"2024-09-10T16:02:08.103831+0000","last_clean":"2024-09-10T16:02:08.103831+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:02:08.103831+0000","last_undegraded":"2024-09-10T16:02:08.103831+0000","last_fullsized":"2024-09-10T16:02:08.103831+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'12834","reported_seq":12976,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:53.540963+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:03:53.540963+0000","last_peered":"2024-09-10T16:03:53.540963+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:03:53.540963+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"36'2800","ondisk_log_start":"36'2800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33644544,"num_objects":8214,"num_object_clones":0,"num_object_copies":16428,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8214,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9089,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3843},{"shards":"0,2","objects":2063}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104463+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:02:08.104463+0000","last_peered":"2024-09-10T16:02:08.104463+0000","last_clean":"2024-09-10T16:02:08.104463+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:02:08.104463+0000","last_undegraded":"2024-09-10T16:02:08.104463+0000","last_fullsized":"2024-09-10T16:02:08.104463+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104448+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:02:08.104448+0000","last_peered":"2024-09-10T16:02:08.104448+0000","last_clean":"2024-09-10T16:02:08.104448+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:02:08.104448+0000","last_undegraded":"2024-09-10T16:02:08.104448+0000","last_fullsized":"2024-09-10T16:02:08.104448+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'12812","reported_seq":12956,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:53.494365+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:03:53.494365+0000","last_peered":"2024-09-10T16:03:53.494365+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:03:53.494365+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"36'2800","ondisk_log_start":"36'2800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34350022,"num_objects":8388,"num_object_clones":0,"num_object_copies":16776,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8388,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8975,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4065},{"shards":"0,2","objects":2116}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":541930230,"num_objects":132317,"num_object_clones":0,"num_object_copies":264634,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43122,"num_objects_misplaced":2057,"num_objects_unfound":0,"num_objects_dirty":132317,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":159789,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6207,"num_bytes_recovered":25419776,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1200152576,"data_stored":1199992686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160815,"ondisk_log_size":160815,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986246,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":710700,"kb_used_data":404120,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":93661140,"statfs":{"total":96636764160,"available":95909007360,"internally_reserved":0,"allocated":413818880,"data_stored":413526647,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":313913002},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,0,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659192,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":571044,"kb_used_data":304724,"kb_used_omap":4,"kb_used_meta":266299,"kb_avail":93800796,"statfs":{"total":96636764160,"available":96052015104,"internally_reserved":0,"allocated":312037376,"data_stored":311697710,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":272690862},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,3,3,4,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182475,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":843972,"kb_used_data":465268,"kb_used_omap":4,"kb_used_meta":378683,"kb_avail":93527868,"statfs":{"total":96636764160,"available":95772536832,"internally_reserved":0,"allocated":476434432,"data_stored":476142194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":387772074},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,2,6,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":475230208,"data_stored":475194078,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":311525376,"data_stored":311441818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":413396992,"data_stored":413356790,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:03:54.142 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:03:54.687 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:03:54.688 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:03:54.710 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":237,"stamp":"2024-09-10T16:03:53.647155+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":541930249,"num_objects":132319,"num_object_clones":0,"num_object_copies":264638,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43122,"num_objects_misplaced":2057,"num_objects_unfound":0,"num_objects_dirty":132319,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":159791,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6209,"num_bytes_recovered":25419795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160818,"ondisk_log_size":160818,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2125716,"kb_used_data":1174112,"kb_used_omap":12,"kb_used_meta":951539,"kb_avail":280989804,"statfs":{"total":289910292480,"available":287733559296,"internally_reserved":0,"allocated":1202290688,"data_stored":1201366551,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":974375938},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,6,5,11,8,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-23801856,"num_objects":-5811,"num_object_clones":0,"num_object_copies":-11622,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":116,"num_objects_misplaced":-420,"num_objects_unfound":0,"num_objects_dirty":-5811,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5811,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":304,"num_bytes_recovered":1245184,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003876"},"pg_stats":[{"pgid":"2.f","version":"45'12513","reported_seq":12588,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:53.468920+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:03:53.468920+0000","last_peered":"2024-09-10T16:03:53.468920+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:03:53.468920+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"36'2500","ondisk_log_start":"36'2500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33419264,"num_objects":8159,"num_object_clones":0,"num_object_copies":16318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8159,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12513,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3900},{"shards":"0,2","objects":2081}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'12549","reported_seq":12662,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:03:50.048510+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:03:50.048510+0000","last_peered":"2024-09-10T16:03:50.048510+0000","last_clean":"2024-09-10T16:03:50.048510+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:03:50.048510+0000","last_undegraded":"2024-09-10T16:03:50.048510+0000","last_fullsized":"2024-09-10T16:03:50.048510+0000","mapping_epoch":35,"log_start":"36'2500","ondisk_log_start":"36'2500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33977286,"num_objects":8297,"num_object_clones":0,"num_object_copies":16594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8297,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":10654,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'12544","reported_seq":21222,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:03:51.217066+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:03:51.217066+0000","last_peered":"2024-09-10T16:03:51.217066+0000","last_clean":"2024-09-10T16:03:51.217066+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:03:51.217066+0000","last_undegraded":"2024-09-10T16:03:51.217066+0000","last_fullsized":"2024-09-10T16:03:51.217066+0000","mapping_epoch":44,"log_start":"36'2500","ondisk_log_start":"36'2500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33804312,"num_objects":8254,"num_object_clones":0,"num_object_copies":16508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8254,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10548,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'12465","reported_seq":12542,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:03:49.966956+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:03:49.966956+0000","last_peered":"2024-09-10T16:03:49.966956+0000","last_clean":"2024-09-10T16:03:49.966956+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:03:49.966956+0000","last_undegraded":"2024-09-10T16:03:49.966956+0000","last_fullsized":"2024-09-10T16:03:49.966956+0000","mapping_epoch":35,"log_start":"36'2400","ondisk_log_start":"36'2400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33566720,"num_objects":8195,"num_object_clones":0,"num_object_copies":16390,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8195,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12465,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'12652","reported_seq":12781,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:03:53.552877+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:03:53.552877+0000","last_peered":"2024-09-10T16:03:53.552877+0000","last_clean":"2024-09-10T16:03:53.552877+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:03:53.552877+0000","last_undegraded":"2024-09-10T16:03:53.552877+0000","last_fullsized":"2024-09-10T16:03:53.552877+0000","mapping_epoch":34,"log_start":"36'2600","ondisk_log_start":"36'2600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33747427,"num_objects":8240,"num_object_clones":0,"num_object_copies":16480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8240,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9363,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'12789","reported_seq":12927,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:53.461652+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:03:53.461652+0000","last_peered":"2024-09-10T16:03:53.461652+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:03:53.461652+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"36'2700","ondisk_log_start":"36'2700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33882595,"num_objects":8273,"num_object_clones":0,"num_object_copies":16546,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8273,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9401,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3902},{"shards":"0,2","objects":2115}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'12733","reported_seq":12868,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:53.496022+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:03:53.496022+0000","last_peered":"2024-09-10T16:03:53.496022+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:03:53.496022+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"36'2700","ondisk_log_start":"36'2700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34198470,"num_objects":8351,"num_object_clones":0,"num_object_copies":16702,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8351,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":9232,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4114},{"shards":"0,2","objects":2048}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'12854","reported_seq":12981,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:03:53.547210+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:03:53.547210+0000","last_peered":"2024-09-10T16:03:53.547210+0000","last_clean":"2024-09-10T16:03:53.547210+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:03:53.547210+0000","last_undegraded":"2024-09-10T16:03:53.547210+0000","last_fullsized":"2024-09-10T16:03:53.547210+0000","mapping_epoch":34,"log_start":"36'2800","ondisk_log_start":"36'2800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34652160,"num_objects":8460,"num_object_clones":0,"num_object_copies":16920,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8460,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9701,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104261+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:02:08.104261+0000","last_peered":"2024-09-10T16:02:08.104261+0000","last_clean":"2024-09-10T16:02:08.104261+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:02:08.104261+0000","last_undegraded":"2024-09-10T16:02:08.104261+0000","last_fullsized":"2024-09-10T16:02:08.104261+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'12294","reported_seq":12434,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:49.905468+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:03:49.905468+0000","last_peered":"2024-09-10T16:03:49.905468+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:03:49.905468+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"36'2200","ondisk_log_start":"36'2200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33436131,"num_objects":8164,"num_object_clones":0,"num_object_copies":16328,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8164,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10113,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2061},{"shards":"2","objects":4041}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103774+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:02:08.103774+0000","last_peered":"2024-09-10T16:02:08.103774+0000","last_clean":"2024-09-10T16:02:08.103774+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:02:08.103774+0000","last_undegraded":"2024-09-10T16:02:08.103774+0000","last_fullsized":"2024-09-10T16:02:08.103774+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'12661","reported_seq":12792,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:53.462645+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:03:53.462645+0000","last_peered":"2024-09-10T16:03:53.462645+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:03:53.462645+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"36'2600","ondisk_log_start":"36'2600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33787904,"num_objects":8249,"num_object_clones":0,"num_object_copies":16498,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8249,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9165,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3994},{"shards":"0,2","objects":2053}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":104,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.105055+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:02:08.105055+0000","last_peered":"2024-09-10T16:02:08.105055+0000","last_clean":"2024-09-10T16:02:08.105055+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:02:08.105055+0000","last_undegraded":"2024-09-10T16:02:08.105055+0000","last_fullsized":"2024-09-10T16:02:08.105055+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'12522","reported_seq":12657,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:49.968852+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:03:49.968852+0000","last_peered":"2024-09-10T16:03:49.968852+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:03:49.968852+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"36'2500","ondisk_log_start":"36'2500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33829347,"num_objects":8260,"num_object_clones":0,"num_object_copies":16520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8260,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10202,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2175},{"shards":"2","objects":3956}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104617+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:02:08.104617+0000","last_peered":"2024-09-10T16:02:08.104617+0000","last_clean":"2024-09-10T16:02:08.104617+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:02:08.104617+0000","last_undegraded":"2024-09-10T16:02:08.104617+0000","last_fullsized":"2024-09-10T16:02:08.104617+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'12695","reported_seq":12824,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:49.986919+0000","last_change":"2024-09-10T15:59:16.174113+0000","last_active":"2024-09-10T16:03:49.986919+0000","last_peered":"2024-09-10T16:03:49.986919+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:03:49.986919+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"36'2600","ondisk_log_start":"36'2600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34713600,"num_objects":8475,"num_object_clones":0,"num_object_copies":16950,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4354,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8475,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10249,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2245},{"shards":"2","objects":4123}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'12650","reported_seq":12785,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:53.415937+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:03:53.415937+0000","last_peered":"2024-09-10T16:03:53.415937+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:03:53.415937+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"36'2600","ondisk_log_start":"36'2600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33538048,"num_objects":8188,"num_object_clones":0,"num_object_copies":16376,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8188,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9281,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3884},{"shards":"0,2","objects":2074}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103316+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:02:08.103316+0000","last_peered":"2024-09-10T16:02:08.103316+0000","last_clean":"2024-09-10T16:02:08.103316+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:02:08.103316+0000","last_undegraded":"2024-09-10T16:02:08.103316+0000","last_fullsized":"2024-09-10T16:02:08.103316+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'12448","reported_seq":16462,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:53.547659+0000","last_change":"2024-09-10T16:03:29.605682+0000","last_active":"2024-09-10T16:03:53.547659+0000","last_peered":"2024-09-10T16:03:53.547659+0000","last_clean":"2024-09-10T15:59:14.115595+0000","last_became_active":"2024-09-10T15:59:16.244089+0000","last_became_peered":"2024-09-10T15:59:16.244089+0000","last_unstale":"2024-09-10T16:03:53.547659+0000","last_undegraded":"2024-09-10T16:03:29.603815+0000","last_fullsized":"2024-09-10T15:59:15.774437+0000","mapping_epoch":42,"log_start":"36'2400","ondisk_log_start":"36'2400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33382400,"num_objects":8150,"num_object_clones":0,"num_object_copies":16300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":211,"num_objects_misplaced":2057,"num_objects_unfound":0,"num_objects_dirty":8150,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8838,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1935,"num_bytes_recovered":7925760,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5882},{"shards":"0,2","objects":2057}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103831+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:02:08.103831+0000","last_peered":"2024-09-10T16:02:08.103831+0000","last_clean":"2024-09-10T16:02:08.103831+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:02:08.103831+0000","last_undegraded":"2024-09-10T16:02:08.103831+0000","last_fullsized":"2024-09-10T16:02:08.103831+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'12834","reported_seq":12976,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:53.540963+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:03:53.540963+0000","last_peered":"2024-09-10T16:03:53.540963+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:03:53.540963+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"36'2800","ondisk_log_start":"36'2800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33644544,"num_objects":8214,"num_object_clones":0,"num_object_copies":16428,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8214,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9089,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3843},{"shards":"0,2","objects":2063}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104463+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:02:08.104463+0000","last_peered":"2024-09-10T16:02:08.104463+0000","last_clean":"2024-09-10T16:02:08.104463+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:02:08.104463+0000","last_undegraded":"2024-09-10T16:02:08.104463+0000","last_fullsized":"2024-09-10T16:02:08.104463+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104448+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:02:08.104448+0000","last_peered":"2024-09-10T16:02:08.104448+0000","last_clean":"2024-09-10T16:02:08.104448+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:02:08.104448+0000","last_undegraded":"2024-09-10T16:02:08.104448+0000","last_fullsized":"2024-09-10T16:02:08.104448+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'12812","reported_seq":12956,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:53.494365+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:03:53.494365+0000","last_peered":"2024-09-10T16:03:53.494365+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:03:53.494365+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"36'2800","ondisk_log_start":"36'2800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34350022,"num_objects":8388,"num_object_clones":0,"num_object_copies":16776,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8388,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8975,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4065},{"shards":"0,2","objects":2116}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":541930230,"num_objects":132317,"num_object_clones":0,"num_object_copies":264634,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43122,"num_objects_misplaced":2057,"num_objects_unfound":0,"num_objects_dirty":132317,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":159789,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6207,"num_bytes_recovered":25419776,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1200152576,"data_stored":1199992686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160815,"ondisk_log_size":160815,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986246,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":710700,"kb_used_data":404120,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":93661140,"statfs":{"total":96636764160,"available":95909007360,"internally_reserved":0,"allocated":413818880,"data_stored":413526647,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":313913002},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,0,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659192,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":571044,"kb_used_data":304724,"kb_used_omap":4,"kb_used_meta":266299,"kb_avail":93800796,"statfs":{"total":96636764160,"available":96052015104,"internally_reserved":0,"allocated":312037376,"data_stored":311697710,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":272690862},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,3,3,4,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182475,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":843972,"kb_used_data":465268,"kb_used_omap":4,"kb_used_meta":378683,"kb_avail":93527868,"statfs":{"total":96636764160,"available":95772536832,"internally_reserved":0,"allocated":476434432,"data_stored":476142194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":387772074},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,2,6,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":475230208,"data_stored":475194078,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":311525376,"data_stored":311441818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":413396992,"data_stored":413356790,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:03:54.711 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-10T16:03:54.711 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "45'12513", 'reported_seq': 12588, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:03:53.468920+0000', 'last_change': '2024-09-10T15:59:16.281558+0000', 'last_active': '2024-09-10T16:03:53.468920+0000', 'last_peered': '2024-09-10T16:03:53.468920+0000', 'last_clean': '2024-09-10T15:59:14.132456+0000', 'last_became_active': '2024-09-10T15:59:16.274407+0000', 'last_became_peered': '2024-09-10T15:59:16.274407+0000', 'last_unstale': '2024-09-10T16:03:53.468920+0000', 'last_undegraded': '2024-09-10T15:59:15.851279+0000', 'last_fullsized': '2024-09-10T15:59:15.774356+0000', 'mapping_epoch': 42, 'log_start': "36'2500", 'ondisk_log_start': "36'2500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'objects_scrubbed': 0, 'log_size': 10013, 'log_dups_size': 0, 'ondisk_log_size': 10013, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33419264, 'num_objects': 8159, 'num_object_clones': 0, 'num_object_copies': 16318, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4259, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8159, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12513, 'num_write_kb': 41344, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3900}, {'shards': '0,2', 'objects': 2081}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:03:54.711 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-10T16:03:54.711 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "45'12789", 'reported_seq': 12927, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:03:53.461652+0000', 'last_change': '2024-09-10T15:59:16.308264+0000', 'last_active': '2024-09-10T16:03:53.461652+0000', 'last_peered': '2024-09-10T16:03:53.461652+0000', 'last_clean': '2024-09-10T15:59:14.111551+0000', 'last_became_active': '2024-09-10T15:59:16.300040+0000', 'last_became_peered': '2024-09-10T15:59:16.300040+0000', 'last_unstale': '2024-09-10T16:03:53.461652+0000', 'last_undegraded': '2024-09-10T15:59:15.878234+0000', 'last_fullsized': '2024-09-10T15:59:15.769328+0000', 'mapping_epoch': 42, 'log_start': "36'2700", 'ondisk_log_start': "36'2700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3442", 'last_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'objects_scrubbed': 3384, 'log_size': 10089, 'log_dups_size': 0, 'ondisk_log_size': 10089, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1292, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33882595, 'num_objects': 8273, 'num_object_clones': 0, 'num_object_copies': 16546, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4372, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8273, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 9401, 'num_write_kb': 28580, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3902}, {'shards': '0,2', 'objects': 2115}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:03:54.712 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-10T16:03:54.712 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "45'12733", 'reported_seq': 12868, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:03:53.496022+0000', 'last_change': '2024-09-10T15:59:16.331899+0000', 'last_active': '2024-09-10T16:03:53.496022+0000', 'last_peered': '2024-09-10T16:03:53.496022+0000', 'last_clean': '2024-09-10T15:59:14.139436+0000', 'last_became_active': '2024-09-10T15:59:16.318715+0000', 'last_became_peered': '2024-09-10T15:59:16.318715+0000', 'last_unstale': '2024-09-10T16:03:53.496022+0000', 'last_undegraded': '2024-09-10T15:59:15.877524+0000', 'last_fullsized': '2024-09-10T15:59:15.769042+0000', 'mapping_epoch': 42, 'log_start': "36'2700", 'ondisk_log_start': "36'2700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3537", 'last_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'objects_scrubbed': 3497, 'log_size': 10033, 'log_dups_size': 0, 'ondisk_log_size': 10033, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1224, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34198470, 'num_objects': 8351, 'num_object_clones': 0, 'num_object_copies': 16702, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4237, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8351, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 9232, 'num_write_kb': 28172, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4114}, {'shards': '0,2', 'objects': 2048}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:03:54.712 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-10T16:03:54.712 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "45'12294", 'reported_seq': 12434, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:03:49.905468+0000', 'last_change': '2024-09-10T15:59:16.175680+0000', 'last_active': '2024-09-10T16:03:49.905468+0000', 'last_peered': '2024-09-10T16:03:49.905468+0000', 'last_clean': '2024-09-10T15:59:14.088187+0000', 'last_became_active': '2024-09-10T15:59:16.165810+0000', 'last_became_peered': '2024-09-10T15:59:16.165810+0000', 'last_unstale': '2024-09-10T16:03:49.905468+0000', 'last_undegraded': '2024-09-10T15:59:15.822098+0000', 'last_fullsized': '2024-09-10T15:59:15.769123+0000', 'mapping_epoch': 42, 'log_start': "36'2200", 'ondisk_log_start': "36'2200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2218", 'last_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'objects_scrubbed': 2175, 'log_size': 10094, 'log_dups_size': 0, 'ondisk_log_size': 10094, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 825, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33436131, 'num_objects': 8164, 'num_object_clones': 0, 'num_object_copies': 16328, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4124, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8164, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 10113, 'num_write_kb': 32204, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2061}, {'shards': '2', 'objects': 4041}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:03:54.712 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-10T16:03:54.712 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "45'12661", 'reported_seq': 12792, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:03:53.462645+0000', 'last_change': '2024-09-10T15:59:16.277742+0000', 'last_active': '2024-09-10T16:03:53.462645+0000', 'last_peered': '2024-09-10T16:03:53.462645+0000', 'last_clean': '2024-09-10T15:59:14.129081+0000', 'last_became_active': '2024-09-10T15:59:16.269835+0000', 'last_became_peered': '2024-09-10T15:59:16.269835+0000', 'last_unstale': '2024-09-10T16:03:53.462645+0000', 'last_undegraded': '2024-09-10T15:59:15.860857+0000', 'last_fullsized': '2024-09-10T15:59:15.771410+0000', 'mapping_epoch': 42, 'log_start': "36'2600", 'ondisk_log_start': "36'2600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3557", 'last_scrub_stamp': '2024-09-10T15:59:02.493678+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:02.493678+0000', 'objects_scrubbed': 3488, 'log_size': 10061, 'log_dups_size': 0, 'ondisk_log_size': 10061, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1472, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33787904, 'num_objects': 8249, 'num_object_clones': 0, 'num_object_copies': 16498, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4257, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8249, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9165, 'num_write_kb': 27852, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3994}, {'shards': '0,2', 'objects': 2053}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:03:54.712 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-10T16:03:54.712 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "45'12522", 'reported_seq': 12657, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:03:49.968852+0000', 'last_change': '2024-09-10T15:59:16.284311+0000', 'last_active': '2024-09-10T16:03:49.968852+0000', 'last_peered': '2024-09-10T16:03:49.968852+0000', 'last_clean': '2024-09-10T15:59:14.116041+0000', 'last_became_active': '2024-09-10T15:59:16.274543+0000', 'last_became_peered': '2024-09-10T15:59:16.274543+0000', 'last_unstale': '2024-09-10T16:03:49.968852+0000', 'last_undegraded': '2024-09-10T15:59:15.812484+0000', 'last_fullsized': '2024-09-10T15:59:15.770885+0000', 'mapping_epoch': 42, 'log_start': "36'2500", 'ondisk_log_start': "36'2500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2356", 'last_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'objects_scrubbed': 2316, 'log_size': 10022, 'log_dups_size': 0, 'ondisk_log_size': 10022, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 783, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33829347, 'num_objects': 8260, 'num_object_clones': 0, 'num_object_copies': 16520, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8260, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 10202, 'num_write_kb': 32292, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2175}, {'shards': '2', 'objects': 3956}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:03:54.712 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-10T16:03:54.712 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "45'12695", 'reported_seq': 12824, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:03:49.986919+0000', 'last_change': '2024-09-10T15:59:16.174113+0000', 'last_active': '2024-09-10T16:03:49.986919+0000', 'last_peered': '2024-09-10T16:03:49.986919+0000', 'last_clean': '2024-09-10T15:59:14.110654+0000', 'last_became_active': '2024-09-10T15:59:16.164645+0000', 'last_became_peered': '2024-09-10T15:59:16.164645+0000', 'last_unstale': '2024-09-10T16:03:49.986919+0000', 'last_undegraded': '2024-09-10T15:59:15.803593+0000', 'last_fullsized': '2024-09-10T15:59:15.769083+0000', 'mapping_epoch': 42, 'log_start': "36'2600", 'ondisk_log_start': "36'2600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2486", 'last_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'objects_scrubbed': 2440, 'log_size': 10095, 'log_dups_size': 0, 'ondisk_log_size': 10095, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 792, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34713600, 'num_objects': 8475, 'num_object_clones': 0, 'num_object_copies': 16950, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4354, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8475, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10249, 'num_write_kb': 32568, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2245}, {'shards': '2', 'objects': 4123}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:03:54.713 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-10T16:03:54.713 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "45'12650", 'reported_seq': 12785, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:03:53.415937+0000', 'last_change': '2024-09-10T15:59:16.315641+0000', 'last_active': '2024-09-10T16:03:53.415937+0000', 'last_peered': '2024-09-10T16:03:53.415937+0000', 'last_clean': '2024-09-10T15:59:14.117001+0000', 'last_became_active': '2024-09-10T15:59:16.302245+0000', 'last_became_peered': '2024-09-10T15:59:16.302245+0000', 'last_unstale': '2024-09-10T16:03:53.415937+0000', 'last_undegraded': '2024-09-10T15:59:15.845012+0000', 'last_fullsized': '2024-09-10T15:59:15.770337+0000', 'mapping_epoch': 42, 'log_start': "36'2600", 'ondisk_log_start': "36'2600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3404", 'last_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'objects_scrubbed': 3367, 'log_size': 10050, 'log_dups_size': 0, 'ondisk_log_size': 10050, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 983, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33538048, 'num_objects': 8188, 'num_object_clones': 0, 'num_object_copies': 16376, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8188, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9281, 'num_write_kb': 28204, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3884}, {'shards': '0,2', 'objects': 2074}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:03:54.713 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-10T16:03:54.713 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "45'12448", 'reported_seq': 16462, 'reported_epoch': 45, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:03:53.547659+0000', 'last_change': '2024-09-10T16:03:29.605682+0000', 'last_active': '2024-09-10T16:03:53.547659+0000', 'last_peered': '2024-09-10T16:03:53.547659+0000', 'last_clean': '2024-09-10T15:59:14.115595+0000', 'last_became_active': '2024-09-10T15:59:16.244089+0000', 'last_became_peered': '2024-09-10T15:59:16.244089+0000', 'last_unstale': '2024-09-10T16:03:53.547659+0000', 'last_undegraded': '2024-09-10T16:03:29.603815+0000', 'last_fullsized': '2024-09-10T15:59:15.774437+0000', 'mapping_epoch': 42, 'log_start': "36'2400", 'ondisk_log_start': "36'2400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3653", 'last_scrub_stamp': '2024-09-10T15:59:05.298468+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:05.298468+0000', 'objects_scrubbed': 3602, 'log_size': 10048, 'log_dups_size': 0, 'ondisk_log_size': 10048, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1289, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33382400, 'num_objects': 8150, 'num_object_clones': 0, 'num_object_copies': 16300, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 211, 'num_objects_misplaced': 2057, 'num_objects_unfound': 0, 'num_objects_dirty': 8150, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8838, 'num_write_kb': 26772, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1935, 'num_bytes_recovered': 7925760, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5882}, {'shards': '0,2', 'objects': 2057}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:03:54.713 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-10T16:03:54.713 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "45'12834", 'reported_seq': 12976, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:03:53.540963+0000', 'last_change': '2024-09-10T15:59:16.311291+0000', 'last_active': '2024-09-10T16:03:53.540963+0000', 'last_peered': '2024-09-10T16:03:53.540963+0000', 'last_clean': '2024-09-10T15:59:14.116363+0000', 'last_became_active': '2024-09-10T15:59:16.301244+0000', 'last_became_peered': '2024-09-10T15:59:16.301244+0000', 'last_unstale': '2024-09-10T16:03:53.540963+0000', 'last_undegraded': '2024-09-10T15:59:15.883434+0000', 'last_fullsized': '2024-09-10T15:59:15.772756+0000', 'mapping_epoch': 42, 'log_start': "36'2800", 'ondisk_log_start': "36'2800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3789", 'last_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'objects_scrubbed': 3741, 'log_size': 10034, 'log_dups_size': 0, 'ondisk_log_size': 10034, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1440, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33644544, 'num_objects': 8214, 'num_object_clones': 0, 'num_object_copies': 16428, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4373, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8214, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9089, 'num_write_kb': 27124, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3843}, {'shards': '0,2', 'objects': 2063}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:03:54.713 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-10T16:03:54.713 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "45'12812", 'reported_seq': 12956, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:03:53.494365+0000', 'last_change': '2024-09-10T15:59:16.313409+0000', 'last_active': '2024-09-10T16:03:53.494365+0000', 'last_peered': '2024-09-10T16:03:53.494365+0000', 'last_clean': '2024-09-10T15:59:14.129780+0000', 'last_became_active': '2024-09-10T15:59:16.300932+0000', 'last_became_peered': '2024-09-10T15:59:16.300932+0000', 'last_unstale': '2024-09-10T16:03:53.494365+0000', 'last_undegraded': '2024-09-10T15:59:15.887823+0000', 'last_fullsized': '2024-09-10T15:59:15.768513+0000', 'mapping_epoch': 42, 'log_start': "36'2800", 'ondisk_log_start': "36'2800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3880", 'last_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'objects_scrubbed': 3827, 'log_size': 10012, 'log_dups_size': 0, 'ondisk_log_size': 10012, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1397, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34350022, 'num_objects': 8388, 'num_object_clones': 0, 'num_object_copies': 16776, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4325, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8388, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8975, 'num_write_kb': 27072, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4065}, {'shards': '0,2', 'objects': 2116}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:03:54.714 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T16:03:55.387 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:03:55.387 INFO:teuthology.orchestra.run.smithi115.stdout:{"fsid":"7effee29-df05-4351-9741-ee203564ed81","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 43122/264638 objects degraded (16.295%), 11 pgs degraded, 11 pgs undersized","count":22},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":399,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983954,"num_in_osds":3,"osd_in_since":1725983947,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":132319,"data_bytes":541930249,"bytes_used":2176733184,"bytes_avail":287733559296,"bytes_total":289910292480,"degraded_objects":43122,"degraded_total":264638,"degraded_ratio":0.16294712021705121,"misplaced_objects":2057,"misplaced_total":264638,"misplaced_ratio":0.0077728822013467458,"recovering_objects_per_sec":25,"recovering_bytes_per_sec":103731,"recovering_keys_per_sec":0,"num_objects_recovered":304,"num_bytes_recovered":1245184,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":484},"fsmap":{"epoch":1,"btime":"2024-09-10T15:57:14:967120+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T16:00:13.560776+0000","services":{}},"progress_events":{}} 2024-09-10T16:03:55.412 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-10T16:03:55.413 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:03:55.971 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:03:55.972 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:03:55.993 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":238,"stamp":"2024-09-10T16:03:55.647637+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":539165449,"num_objects":131644,"num_object_clones":0,"num_object_copies":263288,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43122,"num_objects_misplaced":2057,"num_objects_unfound":0,"num_objects_dirty":131644,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":160466,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6209,"num_bytes_recovered":25419795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160793,"ondisk_log_size":160793,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2126264,"kb_used_data":1170504,"kb_used_omap":12,"kb_used_meta":955699,"kb_avail":280989256,"statfs":{"total":289910292480,"available":287732998144,"internally_reserved":0,"allocated":1198596096,"data_stored":1197671959,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":978635778},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,2,7,10,8,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-20840448,"num_objects":-5088,"num_object_clones":0,"num_object_copies":-10176,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":72,"num_objects_misplaced":-274,"num_objects_unfound":0,"num_objects_dirty":-5088,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5088,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":202,"num_bytes_recovered":827392,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003866"},"pg_stats":[{"pgid":"2.f","version":"45'12513","reported_seq":12588,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:53.468920+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:03:53.468920+0000","last_peered":"2024-09-10T16:03:53.468920+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:03:53.468920+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"36'2500","ondisk_log_start":"36'2500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33419264,"num_objects":8159,"num_object_clones":0,"num_object_copies":16318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8159,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12513,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3900},{"shards":"0,2","objects":2081}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'12677","reported_seq":12790,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:03:55.024290+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:03:55.024290+0000","last_peered":"2024-09-10T16:03:55.024290+0000","last_clean":"2024-09-10T16:03:55.024290+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:03:55.024290+0000","last_undegraded":"2024-09-10T16:03:55.024290+0000","last_fullsized":"2024-09-10T16:03:55.024290+0000","mapping_epoch":35,"log_start":"36'2600","ondisk_log_start":"36'2600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33452998,"num_objects":8169,"num_object_clones":0,"num_object_copies":16338,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8169,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":10782,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'12544","reported_seq":21222,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:03:51.217066+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:03:51.217066+0000","last_peered":"2024-09-10T16:03:51.217066+0000","last_clean":"2024-09-10T16:03:51.217066+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:03:51.217066+0000","last_undegraded":"2024-09-10T16:03:51.217066+0000","last_fullsized":"2024-09-10T16:03:51.217066+0000","mapping_epoch":44,"log_start":"36'2500","ondisk_log_start":"36'2500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33804312,"num_objects":8254,"num_object_clones":0,"num_object_copies":16508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8254,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10548,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'12580","reported_seq":12661,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:03:54.696608+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:03:54.696608+0000","last_peered":"2024-09-10T16:03:54.696608+0000","last_clean":"2024-09-10T16:03:54.696608+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:03:54.696608+0000","last_undegraded":"2024-09-10T16:03:54.696608+0000","last_fullsized":"2024-09-10T16:03:54.696608+0000","mapping_epoch":35,"log_start":"36'2500","ondisk_log_start":"36'2500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33095680,"num_objects":8080,"num_object_clones":0,"num_object_copies":16160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8080,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12580,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'12652","reported_seq":12781,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:03:53.552877+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:03:53.552877+0000","last_peered":"2024-09-10T16:03:53.552877+0000","last_clean":"2024-09-10T16:03:53.552877+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:03:53.552877+0000","last_undegraded":"2024-09-10T16:03:53.552877+0000","last_fullsized":"2024-09-10T16:03:53.552877+0000","mapping_epoch":34,"log_start":"36'2600","ondisk_log_start":"36'2600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33747427,"num_objects":8240,"num_object_clones":0,"num_object_copies":16480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8240,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9363,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'12789","reported_seq":12927,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:53.461652+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:03:53.461652+0000","last_peered":"2024-09-10T16:03:53.461652+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:03:53.461652+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"36'2700","ondisk_log_start":"36'2700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33882595,"num_objects":8273,"num_object_clones":0,"num_object_copies":16546,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8273,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9401,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3902},{"shards":"0,2","objects":2115}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'12733","reported_seq":12868,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:53.496022+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:03:53.496022+0000","last_peered":"2024-09-10T16:03:53.496022+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:03:53.496022+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"36'2700","ondisk_log_start":"36'2700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34198470,"num_objects":8351,"num_object_clones":0,"num_object_copies":16702,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8351,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":9232,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4114},{"shards":"0,2","objects":2048}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'12854","reported_seq":12981,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:03:53.547210+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:03:53.547210+0000","last_peered":"2024-09-10T16:03:53.547210+0000","last_clean":"2024-09-10T16:03:53.547210+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:03:53.547210+0000","last_undegraded":"2024-09-10T16:03:53.547210+0000","last_fullsized":"2024-09-10T16:03:53.547210+0000","mapping_epoch":34,"log_start":"36'2800","ondisk_log_start":"36'2800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34652160,"num_objects":8460,"num_object_clones":0,"num_object_copies":16920,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8460,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9701,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104261+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:02:08.104261+0000","last_peered":"2024-09-10T16:02:08.104261+0000","last_clean":"2024-09-10T16:02:08.104261+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:02:08.104261+0000","last_undegraded":"2024-09-10T16:02:08.104261+0000","last_fullsized":"2024-09-10T16:02:08.104261+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'12440","reported_seq":12580,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:54.984841+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:03:54.984841+0000","last_peered":"2024-09-10T16:03:54.984841+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:03:54.984841+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"36'2400","ondisk_log_start":"36'2400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32838115,"num_objects":8018,"num_object_clones":0,"num_object_copies":16036,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8018,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10259,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1915},{"shards":"2","objects":3895}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103774+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:02:08.103774+0000","last_peered":"2024-09-10T16:02:08.103774+0000","last_clean":"2024-09-10T16:02:08.103774+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:02:08.103774+0000","last_undegraded":"2024-09-10T16:02:08.103774+0000","last_fullsized":"2024-09-10T16:02:08.103774+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'12661","reported_seq":12792,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:53.462645+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:03:53.462645+0000","last_peered":"2024-09-10T16:03:53.462645+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:03:53.462645+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"36'2600","ondisk_log_start":"36'2600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33787904,"num_objects":8249,"num_object_clones":0,"num_object_copies":16498,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8249,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9165,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3994},{"shards":"0,2","objects":2053}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":104,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.105055+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:02:08.105055+0000","last_peered":"2024-09-10T16:02:08.105055+0000","last_clean":"2024-09-10T16:02:08.105055+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:02:08.105055+0000","last_undegraded":"2024-09-10T16:02:08.105055+0000","last_fullsized":"2024-09-10T16:02:08.105055+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'12650","reported_seq":12785,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:54.959030+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:03:54.959030+0000","last_peered":"2024-09-10T16:03:54.959030+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:03:54.959030+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"36'2600","ondisk_log_start":"36'2600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33305059,"num_objects":8132,"num_object_clones":0,"num_object_copies":16264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8132,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10330,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2047},{"shards":"2","objects":3828}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104617+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:02:08.104617+0000","last_peered":"2024-09-10T16:02:08.104617+0000","last_clean":"2024-09-10T16:02:08.104617+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:02:08.104617+0000","last_undegraded":"2024-09-10T16:02:08.104617+0000","last_fullsized":"2024-09-10T16:02:08.104617+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'12853","reported_seq":12982,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:55.066150+0000","last_change":"2024-09-10T15:59:16.174113+0000","last_active":"2024-09-10T16:03:55.066150+0000","last_peered":"2024-09-10T16:03:55.066150+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:03:55.066150+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"36'2800","ondisk_log_start":"36'2800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34066432,"num_objects":8317,"num_object_clones":0,"num_object_copies":16634,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4354,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8317,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10407,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2087},{"shards":"2","objects":3965}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'12650","reported_seq":12785,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:53.415937+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:03:53.415937+0000","last_peered":"2024-09-10T16:03:53.415937+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:03:53.415937+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"36'2600","ondisk_log_start":"36'2600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33538048,"num_objects":8188,"num_object_clones":0,"num_object_copies":16376,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8188,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9281,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3884},{"shards":"0,2","objects":2074}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103316+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:02:08.103316+0000","last_peered":"2024-09-10T16:02:08.103316+0000","last_clean":"2024-09-10T16:02:08.103316+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:02:08.103316+0000","last_undegraded":"2024-09-10T16:02:08.103316+0000","last_fullsized":"2024-09-10T16:02:08.103316+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'12448","reported_seq":16462,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:53.547659+0000","last_change":"2024-09-10T16:03:29.605682+0000","last_active":"2024-09-10T16:03:53.547659+0000","last_peered":"2024-09-10T16:03:53.547659+0000","last_clean":"2024-09-10T15:59:14.115595+0000","last_became_active":"2024-09-10T15:59:16.244089+0000","last_became_peered":"2024-09-10T15:59:16.244089+0000","last_unstale":"2024-09-10T16:03:53.547659+0000","last_undegraded":"2024-09-10T16:03:29.603815+0000","last_fullsized":"2024-09-10T15:59:15.774437+0000","mapping_epoch":42,"log_start":"36'2400","ondisk_log_start":"36'2400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33382400,"num_objects":8150,"num_object_clones":0,"num_object_copies":16300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":211,"num_objects_misplaced":2057,"num_objects_unfound":0,"num_objects_dirty":8150,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8838,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1935,"num_bytes_recovered":7925760,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5882},{"shards":"0,2","objects":2057}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103831+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:02:08.103831+0000","last_peered":"2024-09-10T16:02:08.103831+0000","last_clean":"2024-09-10T16:02:08.103831+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:02:08.103831+0000","last_undegraded":"2024-09-10T16:02:08.103831+0000","last_fullsized":"2024-09-10T16:02:08.103831+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'12834","reported_seq":12976,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:53.540963+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:03:53.540963+0000","last_peered":"2024-09-10T16:03:53.540963+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:03:53.540963+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"36'2800","ondisk_log_start":"36'2800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33644544,"num_objects":8214,"num_object_clones":0,"num_object_copies":16428,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8214,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9089,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3843},{"shards":"0,2","objects":2063}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104463+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:02:08.104463+0000","last_peered":"2024-09-10T16:02:08.104463+0000","last_clean":"2024-09-10T16:02:08.104463+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:02:08.104463+0000","last_undegraded":"2024-09-10T16:02:08.104463+0000","last_fullsized":"2024-09-10T16:02:08.104463+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104448+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:02:08.104448+0000","last_peered":"2024-09-10T16:02:08.104448+0000","last_clean":"2024-09-10T16:02:08.104448+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:02:08.104448+0000","last_undegraded":"2024-09-10T16:02:08.104448+0000","last_fullsized":"2024-09-10T16:02:08.104448+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'12812","reported_seq":12956,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:53.494365+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:03:53.494365+0000","last_peered":"2024-09-10T16:03:53.494365+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:03:53.494365+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"36'2800","ondisk_log_start":"36'2800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34350022,"num_objects":8388,"num_object_clones":0,"num_object_copies":16776,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8388,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8975,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4065},{"shards":"0,2","objects":2116}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":539165430,"num_objects":131642,"num_object_clones":0,"num_object_copies":263284,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43122,"num_objects_misplaced":2057,"num_objects_unfound":0,"num_objects_dirty":131642,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":160464,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6207,"num_bytes_recovered":25419776,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1195646976,"data_stored":1195487086,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160790,"ondisk_log_size":160790,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986247,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":711248,"kb_used_data":400512,"kb_used_omap":4,"kb_used_meta":310715,"kb_avail":93660592,"statfs":{"total":96636764160,"available":95908446208,"internally_reserved":0,"allocated":410124288,"data_stored":409832055,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":318172842},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,0,2,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659192,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":571044,"kb_used_data":304724,"kb_used_omap":4,"kb_used_meta":266299,"kb_avail":93800796,"statfs":{"total":96636764160,"available":96052015104,"internally_reserved":0,"allocated":312037376,"data_stored":311697710,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":272690862},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,3,3,4,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182475,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":843972,"kb_used_data":465268,"kb_used_omap":4,"kb_used_meta":378683,"kb_avail":93527868,"statfs":{"total":96636764160,"available":95772536832,"internally_reserved":0,"allocated":476434432,"data_stored":476142194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":387772074},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,2,6,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":475230208,"data_stored":475194078,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":311525376,"data_stored":311441818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":408891392,"data_stored":408851190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:03:58.996 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:03:59.551 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:03:59.553 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:03:59.576 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":239,"stamp":"2024-09-10T16:03:57.648110+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":538682121,"num_objects":131526,"num_object_clones":0,"num_object_copies":263052,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43122,"num_objects_misplaced":2057,"num_objects_unfound":0,"num_objects_dirty":131526,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":160584,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6209,"num_bytes_recovered":25419795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160811,"ondisk_log_size":160811,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2129900,"kb_used_data":1169980,"kb_used_omap":12,"kb_used_meta":959859,"kb_avail":280985620,"statfs":{"total":289910292480,"available":287729274880,"internally_reserved":0,"allocated":1198059520,"data_stored":1197135383,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":982895618},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,2,9,8,5,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18649088,"num_objects":-4553,"num_object_clones":0,"num_object_copies":-9106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":72,"num_objects_misplaced":-274,"num_objects_unfound":0,"num_objects_dirty":-4553,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4553,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":202,"num_bytes_recovered":827392,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003651"},"pg_stats":[{"pgid":"2.f","version":"45'12513","reported_seq":12588,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:53.468920+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:03:53.468920+0000","last_peered":"2024-09-10T16:03:53.468920+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:03:53.468920+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"36'2500","ondisk_log_start":"36'2500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33419264,"num_objects":8159,"num_object_clones":0,"num_object_copies":16318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8159,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12513,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3900},{"shards":"0,2","objects":2081}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'12677","reported_seq":12790,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:03:55.024290+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:03:55.024290+0000","last_peered":"2024-09-10T16:03:55.024290+0000","last_clean":"2024-09-10T16:03:55.024290+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:03:55.024290+0000","last_undegraded":"2024-09-10T16:03:55.024290+0000","last_fullsized":"2024-09-10T16:03:55.024290+0000","mapping_epoch":35,"log_start":"36'2600","ondisk_log_start":"36'2600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33452998,"num_objects":8169,"num_object_clones":0,"num_object_copies":16338,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8169,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":10782,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'12662","reported_seq":21340,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:03:56.256951+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:03:56.256951+0000","last_peered":"2024-09-10T16:03:56.256951+0000","last_clean":"2024-09-10T16:03:56.256951+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:03:56.256951+0000","last_undegraded":"2024-09-10T16:03:56.256951+0000","last_fullsized":"2024-09-10T16:03:56.256951+0000","mapping_epoch":44,"log_start":"36'2600","ondisk_log_start":"36'2600","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33320984,"num_objects":8136,"num_object_clones":0,"num_object_copies":16272,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8136,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10666,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'12580","reported_seq":12661,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:03:54.696608+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:03:54.696608+0000","last_peered":"2024-09-10T16:03:54.696608+0000","last_clean":"2024-09-10T16:03:54.696608+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:03:54.696608+0000","last_undegraded":"2024-09-10T16:03:54.696608+0000","last_fullsized":"2024-09-10T16:03:54.696608+0000","mapping_epoch":35,"log_start":"36'2500","ondisk_log_start":"36'2500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33095680,"num_objects":8080,"num_object_clones":0,"num_object_copies":16160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8080,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12580,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'12652","reported_seq":12781,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:03:53.552877+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:03:53.552877+0000","last_peered":"2024-09-10T16:03:53.552877+0000","last_clean":"2024-09-10T16:03:53.552877+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:03:53.552877+0000","last_undegraded":"2024-09-10T16:03:53.552877+0000","last_fullsized":"2024-09-10T16:03:53.552877+0000","mapping_epoch":34,"log_start":"36'2600","ondisk_log_start":"36'2600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33747427,"num_objects":8240,"num_object_clones":0,"num_object_copies":16480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8240,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9363,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'12789","reported_seq":12927,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:53.461652+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:03:53.461652+0000","last_peered":"2024-09-10T16:03:53.461652+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:03:53.461652+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"36'2700","ondisk_log_start":"36'2700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33882595,"num_objects":8273,"num_object_clones":0,"num_object_copies":16546,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8273,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9401,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3902},{"shards":"0,2","objects":2115}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'12733","reported_seq":12868,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:53.496022+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:03:53.496022+0000","last_peered":"2024-09-10T16:03:53.496022+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:03:53.496022+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"36'2700","ondisk_log_start":"36'2700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34198470,"num_objects":8351,"num_object_clones":0,"num_object_copies":16702,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8351,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":9232,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4114},{"shards":"0,2","objects":2048}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'12854","reported_seq":12981,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:03:53.547210+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:03:53.547210+0000","last_peered":"2024-09-10T16:03:53.547210+0000","last_clean":"2024-09-10T16:03:53.547210+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:03:53.547210+0000","last_undegraded":"2024-09-10T16:03:53.547210+0000","last_fullsized":"2024-09-10T16:03:53.547210+0000","mapping_epoch":34,"log_start":"36'2800","ondisk_log_start":"36'2800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34652160,"num_objects":8460,"num_object_clones":0,"num_object_copies":16920,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8460,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9701,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104261+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:02:08.104261+0000","last_peered":"2024-09-10T16:02:08.104261+0000","last_clean":"2024-09-10T16:02:08.104261+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:02:08.104261+0000","last_undegraded":"2024-09-10T16:02:08.104261+0000","last_fullsized":"2024-09-10T16:02:08.104261+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'12440","reported_seq":12580,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:54.984841+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:03:54.984841+0000","last_peered":"2024-09-10T16:03:54.984841+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:03:54.984841+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"36'2400","ondisk_log_start":"36'2400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32838115,"num_objects":8018,"num_object_clones":0,"num_object_copies":16036,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8018,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10259,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1915},{"shards":"2","objects":3895}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103774+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:02:08.103774+0000","last_peered":"2024-09-10T16:02:08.103774+0000","last_clean":"2024-09-10T16:02:08.103774+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:02:08.103774+0000","last_undegraded":"2024-09-10T16:02:08.103774+0000","last_fullsized":"2024-09-10T16:02:08.103774+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'12661","reported_seq":12792,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:53.462645+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:03:53.462645+0000","last_peered":"2024-09-10T16:03:53.462645+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:03:53.462645+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"36'2600","ondisk_log_start":"36'2600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33787904,"num_objects":8249,"num_object_clones":0,"num_object_copies":16498,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8249,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9165,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3994},{"shards":"0,2","objects":2053}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":104,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.105055+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:02:08.105055+0000","last_peered":"2024-09-10T16:02:08.105055+0000","last_clean":"2024-09-10T16:02:08.105055+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:02:08.105055+0000","last_undegraded":"2024-09-10T16:02:08.105055+0000","last_fullsized":"2024-09-10T16:02:08.105055+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'12650","reported_seq":12785,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:54.959030+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:03:54.959030+0000","last_peered":"2024-09-10T16:03:54.959030+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:03:54.959030+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"36'2600","ondisk_log_start":"36'2600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33305059,"num_objects":8132,"num_object_clones":0,"num_object_copies":16264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8132,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10330,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2047},{"shards":"2","objects":3828}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104617+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:02:08.104617+0000","last_peered":"2024-09-10T16:02:08.104617+0000","last_clean":"2024-09-10T16:02:08.104617+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:02:08.104617+0000","last_undegraded":"2024-09-10T16:02:08.104617+0000","last_fullsized":"2024-09-10T16:02:08.104617+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'12853","reported_seq":12982,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:55.066150+0000","last_change":"2024-09-10T15:59:16.174113+0000","last_active":"2024-09-10T16:03:55.066150+0000","last_peered":"2024-09-10T16:03:55.066150+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:03:55.066150+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"36'2800","ondisk_log_start":"36'2800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34066432,"num_objects":8317,"num_object_clones":0,"num_object_copies":16634,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4354,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8317,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10407,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2087},{"shards":"2","objects":3965}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'12650","reported_seq":12785,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:53.415937+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:03:53.415937+0000","last_peered":"2024-09-10T16:03:53.415937+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:03:53.415937+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"36'2600","ondisk_log_start":"36'2600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33538048,"num_objects":8188,"num_object_clones":0,"num_object_copies":16376,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8188,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9281,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3884},{"shards":"0,2","objects":2074}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103316+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:02:08.103316+0000","last_peered":"2024-09-10T16:02:08.103316+0000","last_clean":"2024-09-10T16:02:08.103316+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:02:08.103316+0000","last_undegraded":"2024-09-10T16:02:08.103316+0000","last_fullsized":"2024-09-10T16:02:08.103316+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'12448","reported_seq":16462,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:53.547659+0000","last_change":"2024-09-10T16:03:29.605682+0000","last_active":"2024-09-10T16:03:53.547659+0000","last_peered":"2024-09-10T16:03:53.547659+0000","last_clean":"2024-09-10T15:59:14.115595+0000","last_became_active":"2024-09-10T15:59:16.244089+0000","last_became_peered":"2024-09-10T15:59:16.244089+0000","last_unstale":"2024-09-10T16:03:53.547659+0000","last_undegraded":"2024-09-10T16:03:29.603815+0000","last_fullsized":"2024-09-10T15:59:15.774437+0000","mapping_epoch":42,"log_start":"36'2400","ondisk_log_start":"36'2400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33382400,"num_objects":8150,"num_object_clones":0,"num_object_copies":16300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":211,"num_objects_misplaced":2057,"num_objects_unfound":0,"num_objects_dirty":8150,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8838,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1935,"num_bytes_recovered":7925760,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5882},{"shards":"0,2","objects":2057}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103831+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:02:08.103831+0000","last_peered":"2024-09-10T16:02:08.103831+0000","last_clean":"2024-09-10T16:02:08.103831+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:02:08.103831+0000","last_undegraded":"2024-09-10T16:02:08.103831+0000","last_fullsized":"2024-09-10T16:02:08.103831+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'12834","reported_seq":12976,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:53.540963+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:03:53.540963+0000","last_peered":"2024-09-10T16:03:53.540963+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:03:53.540963+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"36'2800","ondisk_log_start":"36'2800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33644544,"num_objects":8214,"num_object_clones":0,"num_object_copies":16428,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8214,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9089,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3843},{"shards":"0,2","objects":2063}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104463+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:02:08.104463+0000","last_peered":"2024-09-10T16:02:08.104463+0000","last_clean":"2024-09-10T16:02:08.104463+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:02:08.104463+0000","last_undegraded":"2024-09-10T16:02:08.104463+0000","last_fullsized":"2024-09-10T16:02:08.104463+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104448+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:02:08.104448+0000","last_peered":"2024-09-10T16:02:08.104448+0000","last_clean":"2024-09-10T16:02:08.104448+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:02:08.104448+0000","last_undegraded":"2024-09-10T16:02:08.104448+0000","last_fullsized":"2024-09-10T16:02:08.104448+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'12812","reported_seq":12956,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:53.494365+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:03:53.494365+0000","last_peered":"2024-09-10T16:03:53.494365+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:03:53.494365+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"36'2800","ondisk_log_start":"36'2800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34350022,"num_objects":8388,"num_object_clones":0,"num_object_copies":16776,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8388,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8975,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4065},{"shards":"0,2","objects":2116}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":538682102,"num_objects":131524,"num_object_clones":0,"num_object_copies":263048,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43122,"num_objects_misplaced":2057,"num_objects_unfound":0,"num_objects_dirty":131524,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":160582,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6207,"num_bytes_recovered":25419776,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1195184128,"data_stored":1195024238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160808,"ondisk_log_size":160808,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986247,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":711248,"kb_used_data":400512,"kb_used_omap":4,"kb_used_meta":310715,"kb_avail":93660592,"statfs":{"total":96636764160,"available":95908446208,"internally_reserved":0,"allocated":410124288,"data_stored":409832055,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":318172842},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,0,2,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659193,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":574680,"kb_used_data":304200,"kb_used_omap":4,"kb_used_meta":270459,"kb_avail":93797160,"statfs":{"total":96636764160,"available":96048291840,"internally_reserved":0,"allocated":311500800,"data_stored":311161134,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":276950702},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,5,1,1,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182475,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":843972,"kb_used_data":465268,"kb_used_omap":4,"kb_used_meta":378683,"kb_avail":93527868,"statfs":{"total":96636764160,"available":95772536832,"internally_reserved":0,"allocated":476434432,"data_stored":476142194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":387772074},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,2,6,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":475230208,"data_stored":475194078,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":311062528,"data_stored":310978970,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":408891392,"data_stored":408851190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:03:59.578 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:04:00.166 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:04:00.167 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:04:00.203 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":240,"stamp":"2024-09-10T16:03:59.648595+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":533000969,"num_objects":130139,"num_object_clones":0,"num_object_copies":260278,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43158,"num_objects_misplaced":1916,"num_objects_unfound":0,"num_objects_dirty":130139,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":161971,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6314,"num_bytes_recovered":25849875,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160998,"ondisk_log_size":160998,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2127540,"kb_used_data":1163460,"kb_used_omap":12,"kb_used_meta":964019,"kb_avail":280987980,"statfs":{"total":289910292480,"available":287731691520,"internally_reserved":0,"allocated":1191383040,"data_stored":1190458903,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":987155458},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,6,7,9,7,3,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-23781376,"num_objects":-5806,"num_object_clones":0,"num_object_copies":-11612,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":108,"num_objects_misplaced":-415,"num_objects_unfound":0,"num_objects_dirty":-5806,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5806,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":307,"num_bytes_recovered":1257472,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003364"},"pg_stats":[{"pgid":"2.f","version":"45'12656","reported_seq":12729,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:58.627713+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:03:58.627713+0000","last_peered":"2024-09-10T16:03:58.627713+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:03:58.627713+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"36'2600","ondisk_log_start":"36'2600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32833536,"num_objects":8016,"num_object_clones":0,"num_object_copies":16032,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8016,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12656,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3757},{"shards":"0,2","objects":1938}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'12677","reported_seq":12790,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:03:55.024290+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:03:55.024290+0000","last_peered":"2024-09-10T16:03:55.024290+0000","last_clean":"2024-09-10T16:03:55.024290+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:03:55.024290+0000","last_undegraded":"2024-09-10T16:03:55.024290+0000","last_fullsized":"2024-09-10T16:03:55.024290+0000","mapping_epoch":35,"log_start":"36'2600","ondisk_log_start":"36'2600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33452998,"num_objects":8169,"num_object_clones":0,"num_object_copies":16338,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8169,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":10782,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'12662","reported_seq":21340,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:03:56.256951+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:03:56.256951+0000","last_peered":"2024-09-10T16:03:56.256951+0000","last_clean":"2024-09-10T16:03:56.256951+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:03:56.256951+0000","last_undegraded":"2024-09-10T16:03:56.256951+0000","last_fullsized":"2024-09-10T16:03:56.256951+0000","mapping_epoch":44,"log_start":"36'2600","ondisk_log_start":"36'2600","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33320984,"num_objects":8136,"num_object_clones":0,"num_object_copies":16272,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8136,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10666,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'12580","reported_seq":12661,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:03:54.696608+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:03:54.696608+0000","last_peered":"2024-09-10T16:03:54.696608+0000","last_clean":"2024-09-10T16:03:54.696608+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:03:54.696608+0000","last_undegraded":"2024-09-10T16:03:54.696608+0000","last_fullsized":"2024-09-10T16:03:54.696608+0000","mapping_epoch":35,"log_start":"36'2500","ondisk_log_start":"36'2500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33095680,"num_objects":8080,"num_object_clones":0,"num_object_copies":16160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8080,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12580,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'12799","reported_seq":12929,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:03:58.588190+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:03:58.588190+0000","last_peered":"2024-09-10T16:03:58.588190+0000","last_clean":"2024-09-10T16:03:58.588190+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:03:58.588190+0000","last_undegraded":"2024-09-10T16:03:58.588190+0000","last_fullsized":"2024-09-10T16:03:58.588190+0000","mapping_epoch":34,"log_start":"36'2700","ondisk_log_start":"36'2700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33145315,"num_objects":8093,"num_object_clones":0,"num_object_copies":16186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8093,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9510,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'12929","reported_seq":13067,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:58.605909+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:03:58.605909+0000","last_peered":"2024-09-10T16:03:58.605909+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:03:58.605909+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"36'2900","ondisk_log_start":"36'2900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33309155,"num_objects":8133,"num_object_clones":0,"num_object_copies":16266,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8133,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9541,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3762},{"shards":"0,2","objects":1975}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'12871","reported_seq":13009,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:58.557625+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:03:58.557625+0000","last_peered":"2024-09-10T16:03:58.557625+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:03:58.557625+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"36'2800","ondisk_log_start":"36'2800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33633222,"num_objects":8213,"num_object_clones":0,"num_object_copies":16426,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8213,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":9370,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3976},{"shards":"0,2","objects":1910}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'12981","reported_seq":13111,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:03:58.524557+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:03:58.524557+0000","last_peered":"2024-09-10T16:03:58.524557+0000","last_clean":"2024-09-10T16:03:58.524557+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:03:58.524557+0000","last_undegraded":"2024-09-10T16:03:58.524557+0000","last_fullsized":"2024-09-10T16:03:58.524557+0000","mapping_epoch":34,"log_start":"36'2900","ondisk_log_start":"36'2900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34131968,"num_objects":8333,"num_object_clones":0,"num_object_copies":16666,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8333,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9828,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104261+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:02:08.104261+0000","last_peered":"2024-09-10T16:02:08.104261+0000","last_clean":"2024-09-10T16:02:08.104261+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:02:08.104261+0000","last_undegraded":"2024-09-10T16:02:08.104261+0000","last_fullsized":"2024-09-10T16:02:08.104261+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'12440","reported_seq":12580,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:54.984841+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:03:54.984841+0000","last_peered":"2024-09-10T16:03:54.984841+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:03:54.984841+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"36'2400","ondisk_log_start":"36'2400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32838115,"num_objects":8018,"num_object_clones":0,"num_object_copies":16036,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8018,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10259,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1915},{"shards":"2","objects":3895}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103774+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:02:08.103774+0000","last_peered":"2024-09-10T16:02:08.103774+0000","last_clean":"2024-09-10T16:02:08.103774+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:02:08.103774+0000","last_undegraded":"2024-09-10T16:02:08.103774+0000","last_fullsized":"2024-09-10T16:02:08.103774+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'12804","reported_seq":12935,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:58.610062+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:03:58.610062+0000","last_peered":"2024-09-10T16:03:58.610062+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:03:58.610062+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"36'2800","ondisk_log_start":"36'2800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33202176,"num_objects":8106,"num_object_clones":0,"num_object_copies":16212,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8106,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9308,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3851},{"shards":"0,2","objects":1910}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":104,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.105055+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:02:08.105055+0000","last_peered":"2024-09-10T16:02:08.105055+0000","last_clean":"2024-09-10T16:02:08.105055+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:02:08.105055+0000","last_undegraded":"2024-09-10T16:02:08.105055+0000","last_fullsized":"2024-09-10T16:02:08.105055+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'12650","reported_seq":12785,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:54.959030+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:03:54.959030+0000","last_peered":"2024-09-10T16:03:54.959030+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:03:54.959030+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"36'2600","ondisk_log_start":"36'2600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33305059,"num_objects":8132,"num_object_clones":0,"num_object_copies":16264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8132,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10330,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2047},{"shards":"2","objects":3828}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104617+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:02:08.104617+0000","last_peered":"2024-09-10T16:02:08.104617+0000","last_clean":"2024-09-10T16:02:08.104617+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:02:08.104617+0000","last_undegraded":"2024-09-10T16:02:08.104617+0000","last_fullsized":"2024-09-10T16:02:08.104617+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'12853","reported_seq":12982,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:55.066150+0000","last_change":"2024-09-10T15:59:16.174113+0000","last_active":"2024-09-10T16:03:55.066150+0000","last_peered":"2024-09-10T16:03:55.066150+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:03:55.066150+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"36'2800","ondisk_log_start":"36'2800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34066432,"num_objects":8317,"num_object_clones":0,"num_object_copies":16634,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4354,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8317,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10407,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2087},{"shards":"2","objects":3965}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'12792","reported_seq":12927,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:58.568666+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:03:58.568666+0000","last_peered":"2024-09-10T16:03:58.568666+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:03:58.568666+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"36'2700","ondisk_log_start":"36'2700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32956416,"num_objects":8046,"num_object_clones":0,"num_object_copies":16092,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8046,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9423,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3742},{"shards":"0,2","objects":1932}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103316+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:02:08.103316+0000","last_peered":"2024-09-10T16:02:08.103316+0000","last_clean":"2024-09-10T16:02:08.103316+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:02:08.103316+0000","last_undegraded":"2024-09-10T16:02:08.103316+0000","last_fullsized":"2024-09-10T16:02:08.103316+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'12589","reported_seq":16813,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:58.627568+0000","last_change":"2024-09-10T16:03:29.605682+0000","last_active":"2024-09-10T16:03:58.627568+0000","last_peered":"2024-09-10T16:03:58.627568+0000","last_clean":"2024-09-10T15:59:14.115595+0000","last_became_active":"2024-09-10T15:59:16.244089+0000","last_became_peered":"2024-09-10T15:59:16.244089+0000","last_unstale":"2024-09-10T16:03:58.627568+0000","last_undegraded":"2024-09-10T16:03:29.603815+0000","last_fullsized":"2024-09-10T15:59:15.774437+0000","mapping_epoch":42,"log_start":"36'2500","ondisk_log_start":"36'2500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32804864,"num_objects":8009,"num_object_clones":0,"num_object_copies":16018,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":247,"num_objects_misplaced":1916,"num_objects_unfound":0,"num_objects_dirty":8009,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8979,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2040,"num_bytes_recovered":8355840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5846},{"shards":"0,2","objects":1916}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103831+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:02:08.103831+0000","last_peered":"2024-09-10T16:02:08.103831+0000","last_clean":"2024-09-10T16:02:08.103831+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:02:08.103831+0000","last_undegraded":"2024-09-10T16:02:08.103831+0000","last_fullsized":"2024-09-10T16:02:08.103831+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'12977","reported_seq":13118,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:58.620849+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:03:58.620849+0000","last_peered":"2024-09-10T16:03:58.620849+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:03:58.620849+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"36'2900","ondisk_log_start":"36'2900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33058816,"num_objects":8071,"num_object_clones":0,"num_object_copies":16142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8071,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9232,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3700},{"shards":"0,2","objects":1920}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104463+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:02:08.104463+0000","last_peered":"2024-09-10T16:02:08.104463+0000","last_clean":"2024-09-10T16:02:08.104463+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:02:08.104463+0000","last_undegraded":"2024-09-10T16:02:08.104463+0000","last_fullsized":"2024-09-10T16:02:08.104463+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104448+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:02:08.104448+0000","last_peered":"2024-09-10T16:02:08.104448+0000","last_clean":"2024-09-10T16:02:08.104448+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:02:08.104448+0000","last_undegraded":"2024-09-10T16:02:08.104448+0000","last_fullsized":"2024-09-10T16:02:08.104448+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'12935","reported_seq":13080,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:58.637312+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:03:58.637312+0000","last_peered":"2024-09-10T16:03:58.637312+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:03:58.637312+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"36'2900","ondisk_log_start":"36'2900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33846214,"num_objects":8265,"num_object_clones":0,"num_object_copies":16530,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8265,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9098,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3942},{"shards":"0,2","objects":1993}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":533000950,"num_objects":130137,"num_object_clones":0,"num_object_copies":260274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43158,"num_objects_misplaced":1916,"num_objects_unfound":0,"num_objects_dirty":130137,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":161969,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6312,"num_bytes_recovered":25849856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1188331520,"data_stored":1188171630,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160995,"ondisk_log_size":160995,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986247,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":711248,"kb_used_data":400512,"kb_used_omap":4,"kb_used_meta":310715,"kb_avail":93660592,"statfs":{"total":96636764160,"available":95908446208,"internally_reserved":0,"allocated":410124288,"data_stored":409832055,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":318172842},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,0,2,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659193,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":574680,"kb_used_data":304200,"kb_used_omap":4,"kb_used_meta":270459,"kb_avail":93797160,"statfs":{"total":96636764160,"available":96048291840,"internally_reserved":0,"allocated":311500800,"data_stored":311161134,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":276950702},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,5,1,1,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182476,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":841612,"kb_used_data":458748,"kb_used_omap":4,"kb_used_meta":382843,"kb_avail":93530228,"statfs":{"total":96636764160,"available":95774953472,"internally_reserved":0,"allocated":469757952,"data_stored":469465714,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":392031914},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,4,5,2,5,2,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":468377600,"data_stored":468341470,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":311062528,"data_stored":310978970,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":408891392,"data_stored":408851190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:04:00.204 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-10T16:04:00.204 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "45'12656", 'reported_seq': 12729, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:03:58.627713+0000', 'last_change': '2024-09-10T15:59:16.281558+0000', 'last_active': '2024-09-10T16:03:58.627713+0000', 'last_peered': '2024-09-10T16:03:58.627713+0000', 'last_clean': '2024-09-10T15:59:14.132456+0000', 'last_became_active': '2024-09-10T15:59:16.274407+0000', 'last_became_peered': '2024-09-10T15:59:16.274407+0000', 'last_unstale': '2024-09-10T16:03:58.627713+0000', 'last_undegraded': '2024-09-10T15:59:15.851279+0000', 'last_fullsized': '2024-09-10T15:59:15.774356+0000', 'mapping_epoch': 42, 'log_start': "36'2600", 'ondisk_log_start': "36'2600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'objects_scrubbed': 0, 'log_size': 10056, 'log_dups_size': 0, 'ondisk_log_size': 10056, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32833536, 'num_objects': 8016, 'num_object_clones': 0, 'num_object_copies': 16032, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4259, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8016, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12656, 'num_write_kb': 41344, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3757}, {'shards': '0,2', 'objects': 1938}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:04:00.204 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-10T16:04:00.204 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "45'12929", 'reported_seq': 13067, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:03:58.605909+0000', 'last_change': '2024-09-10T15:59:16.308264+0000', 'last_active': '2024-09-10T16:03:58.605909+0000', 'last_peered': '2024-09-10T16:03:58.605909+0000', 'last_clean': '2024-09-10T15:59:14.111551+0000', 'last_became_active': '2024-09-10T15:59:16.300040+0000', 'last_became_peered': '2024-09-10T15:59:16.300040+0000', 'last_unstale': '2024-09-10T16:03:58.605909+0000', 'last_undegraded': '2024-09-10T15:59:15.878234+0000', 'last_fullsized': '2024-09-10T15:59:15.769328+0000', 'mapping_epoch': 42, 'log_start': "36'2900", 'ondisk_log_start': "36'2900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3442", 'last_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'objects_scrubbed': 3384, 'log_size': 10029, 'log_dups_size': 0, 'ondisk_log_size': 10029, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1292, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33309155, 'num_objects': 8133, 'num_object_clones': 0, 'num_object_copies': 16266, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4372, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8133, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 9541, 'num_write_kb': 28580, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3762}, {'shards': '0,2', 'objects': 1975}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:04:00.205 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-10T16:04:00.205 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "45'12871", 'reported_seq': 13009, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:03:58.557625+0000', 'last_change': '2024-09-10T15:59:16.331899+0000', 'last_active': '2024-09-10T16:03:58.557625+0000', 'last_peered': '2024-09-10T16:03:58.557625+0000', 'last_clean': '2024-09-10T15:59:14.139436+0000', 'last_became_active': '2024-09-10T15:59:16.318715+0000', 'last_became_peered': '2024-09-10T15:59:16.318715+0000', 'last_unstale': '2024-09-10T16:03:58.557625+0000', 'last_undegraded': '2024-09-10T15:59:15.877524+0000', 'last_fullsized': '2024-09-10T15:59:15.769042+0000', 'mapping_epoch': 42, 'log_start': "36'2800", 'ondisk_log_start': "36'2800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3537", 'last_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'objects_scrubbed': 3497, 'log_size': 10071, 'log_dups_size': 0, 'ondisk_log_size': 10071, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1224, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33633222, 'num_objects': 8213, 'num_object_clones': 0, 'num_object_copies': 16426, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4237, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8213, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 9370, 'num_write_kb': 28172, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3976}, {'shards': '0,2', 'objects': 1910}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:04:00.205 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-10T16:04:00.205 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "45'12440", 'reported_seq': 12580, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:03:54.984841+0000', 'last_change': '2024-09-10T15:59:16.175680+0000', 'last_active': '2024-09-10T16:03:54.984841+0000', 'last_peered': '2024-09-10T16:03:54.984841+0000', 'last_clean': '2024-09-10T15:59:14.088187+0000', 'last_became_active': '2024-09-10T15:59:16.165810+0000', 'last_became_peered': '2024-09-10T15:59:16.165810+0000', 'last_unstale': '2024-09-10T16:03:54.984841+0000', 'last_undegraded': '2024-09-10T15:59:15.822098+0000', 'last_fullsized': '2024-09-10T15:59:15.769123+0000', 'mapping_epoch': 42, 'log_start': "36'2400", 'ondisk_log_start': "36'2400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2218", 'last_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'objects_scrubbed': 2175, 'log_size': 10040, 'log_dups_size': 0, 'ondisk_log_size': 10040, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 825, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32838115, 'num_objects': 8018, 'num_object_clones': 0, 'num_object_copies': 16036, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4124, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8018, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 10259, 'num_write_kb': 32204, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1915}, {'shards': '2', 'objects': 3895}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:04:00.205 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-10T16:04:00.205 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "45'12804", 'reported_seq': 12935, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:03:58.610062+0000', 'last_change': '2024-09-10T15:59:16.277742+0000', 'last_active': '2024-09-10T16:03:58.610062+0000', 'last_peered': '2024-09-10T16:03:58.610062+0000', 'last_clean': '2024-09-10T15:59:14.129081+0000', 'last_became_active': '2024-09-10T15:59:16.269835+0000', 'last_became_peered': '2024-09-10T15:59:16.269835+0000', 'last_unstale': '2024-09-10T16:03:58.610062+0000', 'last_undegraded': '2024-09-10T15:59:15.860857+0000', 'last_fullsized': '2024-09-10T15:59:15.771410+0000', 'mapping_epoch': 42, 'log_start': "36'2800", 'ondisk_log_start': "36'2800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3557", 'last_scrub_stamp': '2024-09-10T15:59:02.493678+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:02.493678+0000', 'objects_scrubbed': 3488, 'log_size': 10004, 'log_dups_size': 0, 'ondisk_log_size': 10004, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1472, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33202176, 'num_objects': 8106, 'num_object_clones': 0, 'num_object_copies': 16212, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4257, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8106, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9308, 'num_write_kb': 27852, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3851}, {'shards': '0,2', 'objects': 1910}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:04:00.205 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-10T16:04:00.205 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "45'12650", 'reported_seq': 12785, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:03:54.959030+0000', 'last_change': '2024-09-10T15:59:16.284311+0000', 'last_active': '2024-09-10T16:03:54.959030+0000', 'last_peered': '2024-09-10T16:03:54.959030+0000', 'last_clean': '2024-09-10T15:59:14.116041+0000', 'last_became_active': '2024-09-10T15:59:16.274543+0000', 'last_became_peered': '2024-09-10T15:59:16.274543+0000', 'last_unstale': '2024-09-10T16:03:54.959030+0000', 'last_undegraded': '2024-09-10T15:59:15.812484+0000', 'last_fullsized': '2024-09-10T15:59:15.770885+0000', 'mapping_epoch': 42, 'log_start': "36'2600", 'ondisk_log_start': "36'2600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2356", 'last_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'objects_scrubbed': 2316, 'log_size': 10050, 'log_dups_size': 0, 'ondisk_log_size': 10050, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 783, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33305059, 'num_objects': 8132, 'num_object_clones': 0, 'num_object_copies': 16264, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8132, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 10330, 'num_write_kb': 32292, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2047}, {'shards': '2', 'objects': 3828}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:04:00.205 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-10T16:04:00.205 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "45'12853", 'reported_seq': 12982, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:03:55.066150+0000', 'last_change': '2024-09-10T15:59:16.174113+0000', 'last_active': '2024-09-10T16:03:55.066150+0000', 'last_peered': '2024-09-10T16:03:55.066150+0000', 'last_clean': '2024-09-10T15:59:14.110654+0000', 'last_became_active': '2024-09-10T15:59:16.164645+0000', 'last_became_peered': '2024-09-10T15:59:16.164645+0000', 'last_unstale': '2024-09-10T16:03:55.066150+0000', 'last_undegraded': '2024-09-10T15:59:15.803593+0000', 'last_fullsized': '2024-09-10T15:59:15.769083+0000', 'mapping_epoch': 42, 'log_start': "36'2800", 'ondisk_log_start': "36'2800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2486", 'last_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'objects_scrubbed': 2440, 'log_size': 10053, 'log_dups_size': 0, 'ondisk_log_size': 10053, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 792, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34066432, 'num_objects': 8317, 'num_object_clones': 0, 'num_object_copies': 16634, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4354, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8317, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10407, 'num_write_kb': 32568, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2087}, {'shards': '2', 'objects': 3965}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:04:00.206 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-10T16:04:00.206 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "45'12792", 'reported_seq': 12927, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:03:58.568666+0000', 'last_change': '2024-09-10T15:59:16.315641+0000', 'last_active': '2024-09-10T16:03:58.568666+0000', 'last_peered': '2024-09-10T16:03:58.568666+0000', 'last_clean': '2024-09-10T15:59:14.117001+0000', 'last_became_active': '2024-09-10T15:59:16.302245+0000', 'last_became_peered': '2024-09-10T15:59:16.302245+0000', 'last_unstale': '2024-09-10T16:03:58.568666+0000', 'last_undegraded': '2024-09-10T15:59:15.845012+0000', 'last_fullsized': '2024-09-10T15:59:15.770337+0000', 'mapping_epoch': 42, 'log_start': "36'2700", 'ondisk_log_start': "36'2700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3404", 'last_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'objects_scrubbed': 3367, 'log_size': 10092, 'log_dups_size': 0, 'ondisk_log_size': 10092, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 983, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32956416, 'num_objects': 8046, 'num_object_clones': 0, 'num_object_copies': 16092, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8046, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9423, 'num_write_kb': 28204, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3742}, {'shards': '0,2', 'objects': 1932}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:04:00.206 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-10T16:04:00.206 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "45'12589", 'reported_seq': 16813, 'reported_epoch': 45, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:03:58.627568+0000', 'last_change': '2024-09-10T16:03:29.605682+0000', 'last_active': '2024-09-10T16:03:58.627568+0000', 'last_peered': '2024-09-10T16:03:58.627568+0000', 'last_clean': '2024-09-10T15:59:14.115595+0000', 'last_became_active': '2024-09-10T15:59:16.244089+0000', 'last_became_peered': '2024-09-10T15:59:16.244089+0000', 'last_unstale': '2024-09-10T16:03:58.627568+0000', 'last_undegraded': '2024-09-10T16:03:29.603815+0000', 'last_fullsized': '2024-09-10T15:59:15.774437+0000', 'mapping_epoch': 42, 'log_start': "36'2500", 'ondisk_log_start': "36'2500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3653", 'last_scrub_stamp': '2024-09-10T15:59:05.298468+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:05.298468+0000', 'objects_scrubbed': 3602, 'log_size': 10089, 'log_dups_size': 0, 'ondisk_log_size': 10089, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1289, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32804864, 'num_objects': 8009, 'num_object_clones': 0, 'num_object_copies': 16018, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 247, 'num_objects_misplaced': 1916, 'num_objects_unfound': 0, 'num_objects_dirty': 8009, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8979, 'num_write_kb': 26772, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2040, 'num_bytes_recovered': 8355840, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5846}, {'shards': '0,2', 'objects': 1916}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:04:00.206 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-10T16:04:00.206 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "45'12977", 'reported_seq': 13118, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:03:58.620849+0000', 'last_change': '2024-09-10T15:59:16.311291+0000', 'last_active': '2024-09-10T16:03:58.620849+0000', 'last_peered': '2024-09-10T16:03:58.620849+0000', 'last_clean': '2024-09-10T15:59:14.116363+0000', 'last_became_active': '2024-09-10T15:59:16.301244+0000', 'last_became_peered': '2024-09-10T15:59:16.301244+0000', 'last_unstale': '2024-09-10T16:03:58.620849+0000', 'last_undegraded': '2024-09-10T15:59:15.883434+0000', 'last_fullsized': '2024-09-10T15:59:15.772756+0000', 'mapping_epoch': 42, 'log_start': "36'2900", 'ondisk_log_start': "36'2900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3789", 'last_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'objects_scrubbed': 3741, 'log_size': 10077, 'log_dups_size': 0, 'ondisk_log_size': 10077, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1440, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33058816, 'num_objects': 8071, 'num_object_clones': 0, 'num_object_copies': 16142, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4373, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8071, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9232, 'num_write_kb': 27124, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3700}, {'shards': '0,2', 'objects': 1920}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:04:00.206 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-10T16:04:00.206 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "45'12935", 'reported_seq': 13080, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:03:58.637312+0000', 'last_change': '2024-09-10T15:59:16.313409+0000', 'last_active': '2024-09-10T16:03:58.637312+0000', 'last_peered': '2024-09-10T16:03:58.637312+0000', 'last_clean': '2024-09-10T15:59:14.129780+0000', 'last_became_active': '2024-09-10T15:59:16.300932+0000', 'last_became_peered': '2024-09-10T15:59:16.300932+0000', 'last_unstale': '2024-09-10T16:03:58.637312+0000', 'last_undegraded': '2024-09-10T15:59:15.887823+0000', 'last_fullsized': '2024-09-10T15:59:15.768513+0000', 'mapping_epoch': 42, 'log_start': "36'2900", 'ondisk_log_start': "36'2900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3880", 'last_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'objects_scrubbed': 3827, 'log_size': 10035, 'log_dups_size': 0, 'ondisk_log_size': 10035, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1397, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33846214, 'num_objects': 8265, 'num_object_clones': 0, 'num_object_copies': 16530, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4325, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8265, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9098, 'num_write_kb': 27072, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3942}, {'shards': '0,2', 'objects': 1993}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:04:00.207 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T16:04:00.892 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:04:00.892 INFO:teuthology.orchestra.run.smithi115.stdout:{"fsid":"7effee29-df05-4351-9741-ee203564ed81","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 43158/260278 objects degraded (16.582%), 11 pgs degraded, 11 pgs undersized","count":22},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":404,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983954,"num_in_osds":3,"osd_in_since":1725983947,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":130139,"data_bytes":533000969,"bytes_used":2178600960,"bytes_avail":287731691520,"bytes_total":289910292480,"degraded_objects":43158,"degraded_total":260278,"degraded_ratio":0.16581501317821715,"misplaced_objects":1916,"misplaced_total":260278,"misplaced_ratio":0.0073613597768539792,"recovering_objects_per_sec":25,"recovering_bytes_per_sec":104759,"recovering_keys_per_sec":0,"num_objects_recovered":307,"num_bytes_recovered":1257472,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":483},"fsmap":{"epoch":1,"btime":"2024-09-10T15:57:14:967120+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T16:00:13.560776+0000","services":{}},"progress_events":{}} 2024-09-10T16:04:00.912 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-10T16:04:00.912 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:04:01.499 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:04:01.500 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:04:01.525 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":240,"stamp":"2024-09-10T16:03:59.648595+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":533000969,"num_objects":130139,"num_object_clones":0,"num_object_copies":260278,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43158,"num_objects_misplaced":1916,"num_objects_unfound":0,"num_objects_dirty":130139,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":161971,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6314,"num_bytes_recovered":25849875,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160998,"ondisk_log_size":160998,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2127540,"kb_used_data":1163460,"kb_used_omap":12,"kb_used_meta":964019,"kb_avail":280987980,"statfs":{"total":289910292480,"available":287731691520,"internally_reserved":0,"allocated":1191383040,"data_stored":1190458903,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":987155458},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,6,7,9,7,3,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-23781376,"num_objects":-5806,"num_object_clones":0,"num_object_copies":-11612,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":108,"num_objects_misplaced":-415,"num_objects_unfound":0,"num_objects_dirty":-5806,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5806,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":307,"num_bytes_recovered":1257472,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003364"},"pg_stats":[{"pgid":"2.f","version":"45'12656","reported_seq":12729,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:58.627713+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:03:58.627713+0000","last_peered":"2024-09-10T16:03:58.627713+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:03:58.627713+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"36'2600","ondisk_log_start":"36'2600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32833536,"num_objects":8016,"num_object_clones":0,"num_object_copies":16032,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8016,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12656,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3757},{"shards":"0,2","objects":1938}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'12677","reported_seq":12790,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:03:55.024290+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:03:55.024290+0000","last_peered":"2024-09-10T16:03:55.024290+0000","last_clean":"2024-09-10T16:03:55.024290+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:03:55.024290+0000","last_undegraded":"2024-09-10T16:03:55.024290+0000","last_fullsized":"2024-09-10T16:03:55.024290+0000","mapping_epoch":35,"log_start":"36'2600","ondisk_log_start":"36'2600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33452998,"num_objects":8169,"num_object_clones":0,"num_object_copies":16338,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8169,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":10782,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'12662","reported_seq":21340,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:03:56.256951+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:03:56.256951+0000","last_peered":"2024-09-10T16:03:56.256951+0000","last_clean":"2024-09-10T16:03:56.256951+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:03:56.256951+0000","last_undegraded":"2024-09-10T16:03:56.256951+0000","last_fullsized":"2024-09-10T16:03:56.256951+0000","mapping_epoch":44,"log_start":"36'2600","ondisk_log_start":"36'2600","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33320984,"num_objects":8136,"num_object_clones":0,"num_object_copies":16272,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8136,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10666,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'12580","reported_seq":12661,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:03:54.696608+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:03:54.696608+0000","last_peered":"2024-09-10T16:03:54.696608+0000","last_clean":"2024-09-10T16:03:54.696608+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:03:54.696608+0000","last_undegraded":"2024-09-10T16:03:54.696608+0000","last_fullsized":"2024-09-10T16:03:54.696608+0000","mapping_epoch":35,"log_start":"36'2500","ondisk_log_start":"36'2500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33095680,"num_objects":8080,"num_object_clones":0,"num_object_copies":16160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8080,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12580,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'12799","reported_seq":12929,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:03:58.588190+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:03:58.588190+0000","last_peered":"2024-09-10T16:03:58.588190+0000","last_clean":"2024-09-10T16:03:58.588190+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:03:58.588190+0000","last_undegraded":"2024-09-10T16:03:58.588190+0000","last_fullsized":"2024-09-10T16:03:58.588190+0000","mapping_epoch":34,"log_start":"36'2700","ondisk_log_start":"36'2700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33145315,"num_objects":8093,"num_object_clones":0,"num_object_copies":16186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8093,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9510,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'12929","reported_seq":13067,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:58.605909+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:03:58.605909+0000","last_peered":"2024-09-10T16:03:58.605909+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:03:58.605909+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"36'2900","ondisk_log_start":"36'2900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33309155,"num_objects":8133,"num_object_clones":0,"num_object_copies":16266,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8133,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9541,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3762},{"shards":"0,2","objects":1975}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'12871","reported_seq":13009,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:58.557625+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:03:58.557625+0000","last_peered":"2024-09-10T16:03:58.557625+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:03:58.557625+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"36'2800","ondisk_log_start":"36'2800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33633222,"num_objects":8213,"num_object_clones":0,"num_object_copies":16426,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8213,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":9370,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3976},{"shards":"0,2","objects":1910}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'12981","reported_seq":13111,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:03:58.524557+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:03:58.524557+0000","last_peered":"2024-09-10T16:03:58.524557+0000","last_clean":"2024-09-10T16:03:58.524557+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:03:58.524557+0000","last_undegraded":"2024-09-10T16:03:58.524557+0000","last_fullsized":"2024-09-10T16:03:58.524557+0000","mapping_epoch":34,"log_start":"36'2900","ondisk_log_start":"36'2900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34131968,"num_objects":8333,"num_object_clones":0,"num_object_copies":16666,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8333,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9828,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104261+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:02:08.104261+0000","last_peered":"2024-09-10T16:02:08.104261+0000","last_clean":"2024-09-10T16:02:08.104261+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:02:08.104261+0000","last_undegraded":"2024-09-10T16:02:08.104261+0000","last_fullsized":"2024-09-10T16:02:08.104261+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'12440","reported_seq":12580,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:54.984841+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:03:54.984841+0000","last_peered":"2024-09-10T16:03:54.984841+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:03:54.984841+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"36'2400","ondisk_log_start":"36'2400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32838115,"num_objects":8018,"num_object_clones":0,"num_object_copies":16036,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8018,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10259,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1915},{"shards":"2","objects":3895}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103774+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:02:08.103774+0000","last_peered":"2024-09-10T16:02:08.103774+0000","last_clean":"2024-09-10T16:02:08.103774+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:02:08.103774+0000","last_undegraded":"2024-09-10T16:02:08.103774+0000","last_fullsized":"2024-09-10T16:02:08.103774+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'12804","reported_seq":12935,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:58.610062+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:03:58.610062+0000","last_peered":"2024-09-10T16:03:58.610062+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:03:58.610062+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"36'2800","ondisk_log_start":"36'2800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33202176,"num_objects":8106,"num_object_clones":0,"num_object_copies":16212,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8106,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9308,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3851},{"shards":"0,2","objects":1910}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":104,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.105055+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:02:08.105055+0000","last_peered":"2024-09-10T16:02:08.105055+0000","last_clean":"2024-09-10T16:02:08.105055+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:02:08.105055+0000","last_undegraded":"2024-09-10T16:02:08.105055+0000","last_fullsized":"2024-09-10T16:02:08.105055+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'12650","reported_seq":12785,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:54.959030+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:03:54.959030+0000","last_peered":"2024-09-10T16:03:54.959030+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:03:54.959030+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"36'2600","ondisk_log_start":"36'2600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33305059,"num_objects":8132,"num_object_clones":0,"num_object_copies":16264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8132,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10330,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2047},{"shards":"2","objects":3828}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104617+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:02:08.104617+0000","last_peered":"2024-09-10T16:02:08.104617+0000","last_clean":"2024-09-10T16:02:08.104617+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:02:08.104617+0000","last_undegraded":"2024-09-10T16:02:08.104617+0000","last_fullsized":"2024-09-10T16:02:08.104617+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'12853","reported_seq":12982,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:55.066150+0000","last_change":"2024-09-10T15:59:16.174113+0000","last_active":"2024-09-10T16:03:55.066150+0000","last_peered":"2024-09-10T16:03:55.066150+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:03:55.066150+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"36'2800","ondisk_log_start":"36'2800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34066432,"num_objects":8317,"num_object_clones":0,"num_object_copies":16634,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4354,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8317,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10407,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2087},{"shards":"2","objects":3965}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'12792","reported_seq":12927,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:58.568666+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:03:58.568666+0000","last_peered":"2024-09-10T16:03:58.568666+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:03:58.568666+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"36'2700","ondisk_log_start":"36'2700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32956416,"num_objects":8046,"num_object_clones":0,"num_object_copies":16092,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8046,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9423,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3742},{"shards":"0,2","objects":1932}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103316+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:02:08.103316+0000","last_peered":"2024-09-10T16:02:08.103316+0000","last_clean":"2024-09-10T16:02:08.103316+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:02:08.103316+0000","last_undegraded":"2024-09-10T16:02:08.103316+0000","last_fullsized":"2024-09-10T16:02:08.103316+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'12589","reported_seq":16813,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:58.627568+0000","last_change":"2024-09-10T16:03:29.605682+0000","last_active":"2024-09-10T16:03:58.627568+0000","last_peered":"2024-09-10T16:03:58.627568+0000","last_clean":"2024-09-10T15:59:14.115595+0000","last_became_active":"2024-09-10T15:59:16.244089+0000","last_became_peered":"2024-09-10T15:59:16.244089+0000","last_unstale":"2024-09-10T16:03:58.627568+0000","last_undegraded":"2024-09-10T16:03:29.603815+0000","last_fullsized":"2024-09-10T15:59:15.774437+0000","mapping_epoch":42,"log_start":"36'2500","ondisk_log_start":"36'2500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32804864,"num_objects":8009,"num_object_clones":0,"num_object_copies":16018,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":247,"num_objects_misplaced":1916,"num_objects_unfound":0,"num_objects_dirty":8009,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8979,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2040,"num_bytes_recovered":8355840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5846},{"shards":"0,2","objects":1916}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103831+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:02:08.103831+0000","last_peered":"2024-09-10T16:02:08.103831+0000","last_clean":"2024-09-10T16:02:08.103831+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:02:08.103831+0000","last_undegraded":"2024-09-10T16:02:08.103831+0000","last_fullsized":"2024-09-10T16:02:08.103831+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'12977","reported_seq":13118,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:58.620849+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:03:58.620849+0000","last_peered":"2024-09-10T16:03:58.620849+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:03:58.620849+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"36'2900","ondisk_log_start":"36'2900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33058816,"num_objects":8071,"num_object_clones":0,"num_object_copies":16142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8071,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9232,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3700},{"shards":"0,2","objects":1920}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104463+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:02:08.104463+0000","last_peered":"2024-09-10T16:02:08.104463+0000","last_clean":"2024-09-10T16:02:08.104463+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:02:08.104463+0000","last_undegraded":"2024-09-10T16:02:08.104463+0000","last_fullsized":"2024-09-10T16:02:08.104463+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104448+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:02:08.104448+0000","last_peered":"2024-09-10T16:02:08.104448+0000","last_clean":"2024-09-10T16:02:08.104448+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:02:08.104448+0000","last_undegraded":"2024-09-10T16:02:08.104448+0000","last_fullsized":"2024-09-10T16:02:08.104448+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'12935","reported_seq":13080,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:58.637312+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:03:58.637312+0000","last_peered":"2024-09-10T16:03:58.637312+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:03:58.637312+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"36'2900","ondisk_log_start":"36'2900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33846214,"num_objects":8265,"num_object_clones":0,"num_object_copies":16530,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8265,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9098,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3942},{"shards":"0,2","objects":1993}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":533000950,"num_objects":130137,"num_object_clones":0,"num_object_copies":260274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43158,"num_objects_misplaced":1916,"num_objects_unfound":0,"num_objects_dirty":130137,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":161969,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6312,"num_bytes_recovered":25849856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1188331520,"data_stored":1188171630,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160995,"ondisk_log_size":160995,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986247,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":711248,"kb_used_data":400512,"kb_used_omap":4,"kb_used_meta":310715,"kb_avail":93660592,"statfs":{"total":96636764160,"available":95908446208,"internally_reserved":0,"allocated":410124288,"data_stored":409832055,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":318172842},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,0,2,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659193,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":574680,"kb_used_data":304200,"kb_used_omap":4,"kb_used_meta":270459,"kb_avail":93797160,"statfs":{"total":96636764160,"available":96048291840,"internally_reserved":0,"allocated":311500800,"data_stored":311161134,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":276950702},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,5,1,1,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182476,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":841612,"kb_used_data":458748,"kb_used_omap":4,"kb_used_meta":382843,"kb_avail":93530228,"statfs":{"total":96636764160,"available":95774953472,"internally_reserved":0,"allocated":469757952,"data_stored":469465714,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":392031914},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,4,5,2,5,2,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":468377600,"data_stored":468341470,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":311062528,"data_stored":310978970,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":408891392,"data_stored":408851190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:04:04.526 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:04:05.094 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:04:05.095 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:04:05.121 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":242,"stamp":"2024-09-10T16:04:03.649718+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":529502985,"num_objects":129285,"num_object_clones":0,"num_object_copies":258570,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43158,"num_objects_misplaced":1916,"num_objects_unfound":0,"num_objects_dirty":129285,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":162825,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6314,"num_bytes_recovered":25849875,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160952,"ondisk_log_size":160952,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2135604,"kb_used_data":1157624,"kb_used_omap":12,"kb_used_meta":977907,"kb_avail":280979916,"statfs":{"total":289910292480,"available":287723433984,"internally_reserved":0,"allocated":1185406976,"data_stored":1184493678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1001376770},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,4,6,7,16,6,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18214912,"num_objects":-4447,"num_object_clones":0,"num_object_copies":-8894,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":72,"num_objects_misplaced":-278,"num_objects_unfound":0,"num_objects_dirty":-4447,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4447,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":206,"num_bytes_recovered":843776,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003045"},"pg_stats":[{"pgid":"2.f","version":"45'12656","reported_seq":12729,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:58.627713+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:03:58.627713+0000","last_peered":"2024-09-10T16:03:58.627713+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:03:58.627713+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"36'2600","ondisk_log_start":"36'2600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32833536,"num_objects":8016,"num_object_clones":0,"num_object_copies":16032,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8016,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12656,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3757},{"shards":"0,2","objects":1938}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'12839","reported_seq":12952,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:04:00.046103+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:04:00.046103+0000","last_peered":"2024-09-10T16:04:00.046103+0000","last_clean":"2024-09-10T16:04:00.046103+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:04:00.046103+0000","last_undegraded":"2024-09-10T16:04:00.046103+0000","last_fullsized":"2024-09-10T16:04:00.046103+0000","mapping_epoch":35,"log_start":"36'2800","ondisk_log_start":"36'2800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32789446,"num_objects":8007,"num_object_clones":0,"num_object_copies":16014,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8007,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":10944,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'12784","reported_seq":21462,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:04:01.460220+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:04:01.460220+0000","last_peered":"2024-09-10T16:04:01.460220+0000","last_clean":"2024-09-10T16:04:01.460220+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:04:01.460220+0000","last_undegraded":"2024-09-10T16:04:01.460220+0000","last_fullsized":"2024-09-10T16:04:01.460220+0000","mapping_epoch":44,"log_start":"36'2700","ondisk_log_start":"36'2700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32821272,"num_objects":8014,"num_object_clones":0,"num_object_copies":16028,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8014,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10788,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'12722","reported_seq":12803,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:04:00.090431+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:04:00.090431+0000","last_peered":"2024-09-10T16:04:00.090431+0000","last_clean":"2024-09-10T16:04:00.090431+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:04:00.090431+0000","last_undegraded":"2024-09-10T16:04:00.090431+0000","last_fullsized":"2024-09-10T16:04:00.090431+0000","mapping_epoch":35,"log_start":"36'2700","ondisk_log_start":"36'2700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32514048,"num_objects":7938,"num_object_clones":0,"num_object_copies":15876,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7938,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12722,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'12799","reported_seq":12929,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:03:58.588190+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:03:58.588190+0000","last_peered":"2024-09-10T16:03:58.588190+0000","last_clean":"2024-09-10T16:03:58.588190+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:03:58.588190+0000","last_undegraded":"2024-09-10T16:03:58.588190+0000","last_fullsized":"2024-09-10T16:03:58.588190+0000","mapping_epoch":34,"log_start":"36'2700","ondisk_log_start":"36'2700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33145315,"num_objects":8093,"num_object_clones":0,"num_object_copies":16186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8093,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9510,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'12929","reported_seq":13067,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:58.605909+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:03:58.605909+0000","last_peered":"2024-09-10T16:03:58.605909+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:03:58.605909+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"36'2900","ondisk_log_start":"36'2900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33309155,"num_objects":8133,"num_object_clones":0,"num_object_copies":16266,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8133,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9541,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3762},{"shards":"0,2","objects":1975}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'12871","reported_seq":13009,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:58.557625+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:03:58.557625+0000","last_peered":"2024-09-10T16:03:58.557625+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:03:58.557625+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"36'2800","ondisk_log_start":"36'2800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33633222,"num_objects":8213,"num_object_clones":0,"num_object_copies":16426,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8213,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":9370,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3976},{"shards":"0,2","objects":1910}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'12981","reported_seq":13111,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:03:58.524557+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:03:58.524557+0000","last_peered":"2024-09-10T16:03:58.524557+0000","last_clean":"2024-09-10T16:03:58.524557+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:03:58.524557+0000","last_undegraded":"2024-09-10T16:03:58.524557+0000","last_fullsized":"2024-09-10T16:03:58.524557+0000","mapping_epoch":34,"log_start":"36'2900","ondisk_log_start":"36'2900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34131968,"num_objects":8333,"num_object_clones":0,"num_object_copies":16666,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8333,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9828,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104261+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:02:08.104261+0000","last_peered":"2024-09-10T16:02:08.104261+0000","last_clean":"2024-09-10T16:02:08.104261+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:02:08.104261+0000","last_undegraded":"2024-09-10T16:02:08.104261+0000","last_fullsized":"2024-09-10T16:02:08.104261+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'12570","reported_seq":12709,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:00.072395+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:04:00.072395+0000","last_peered":"2024-09-10T16:04:00.072395+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:04:00.072395+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"36'2500","ondisk_log_start":"36'2500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32305635,"num_objects":7888,"num_object_clones":0,"num_object_copies":15776,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7888,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10389,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1785},{"shards":"2","objects":3765}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103774+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:02:08.103774+0000","last_peered":"2024-09-10T16:02:08.103774+0000","last_clean":"2024-09-10T16:02:08.103774+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:02:08.103774+0000","last_undegraded":"2024-09-10T16:02:08.103774+0000","last_fullsized":"2024-09-10T16:02:08.103774+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'12804","reported_seq":12935,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:58.610062+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:03:58.610062+0000","last_peered":"2024-09-10T16:03:58.610062+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:03:58.610062+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"36'2800","ondisk_log_start":"36'2800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33202176,"num_objects":8106,"num_object_clones":0,"num_object_copies":16212,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8106,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9308,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3851},{"shards":"0,2","objects":1910}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":104,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.105055+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:02:08.105055+0000","last_peered":"2024-09-10T16:02:08.105055+0000","last_clean":"2024-09-10T16:02:08.105055+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:02:08.105055+0000","last_undegraded":"2024-09-10T16:02:08.105055+0000","last_fullsized":"2024-09-10T16:02:08.105055+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'12785","reported_seq":12920,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:59.976154+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:03:59.976154+0000","last_peered":"2024-09-10T16:03:59.976154+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:03:59.976154+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"36'2700","ondisk_log_start":"36'2700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32752099,"num_objects":7997,"num_object_clones":0,"num_object_copies":15994,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7997,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10465,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1912},{"shards":"2","objects":3693}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104617+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:02:08.104617+0000","last_peered":"2024-09-10T16:02:08.104617+0000","last_clean":"2024-09-10T16:02:08.104617+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:02:08.104617+0000","last_undegraded":"2024-09-10T16:02:08.104617+0000","last_fullsized":"2024-09-10T16:02:08.104617+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'13016","reported_seq":13144,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:00.013325+0000","last_change":"2024-09-10T15:59:16.174113+0000","last_active":"2024-09-10T16:04:00.013325+0000","last_peered":"2024-09-10T16:04:00.013325+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:04:00.013325+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"36'3000","ondisk_log_start":"36'3000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33398784,"num_objects":8154,"num_object_clones":0,"num_object_copies":16308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4354,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8154,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10570,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1924},{"shards":"2","objects":3802}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'12792","reported_seq":12927,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:58.568666+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:03:58.568666+0000","last_peered":"2024-09-10T16:03:58.568666+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:03:58.568666+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"36'2700","ondisk_log_start":"36'2700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32956416,"num_objects":8046,"num_object_clones":0,"num_object_copies":16092,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8046,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9423,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3742},{"shards":"0,2","objects":1932}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103316+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:02:08.103316+0000","last_peered":"2024-09-10T16:02:08.103316+0000","last_clean":"2024-09-10T16:02:08.103316+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:02:08.103316+0000","last_undegraded":"2024-09-10T16:02:08.103316+0000","last_fullsized":"2024-09-10T16:02:08.103316+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'12589","reported_seq":16813,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:58.627568+0000","last_change":"2024-09-10T16:03:29.605682+0000","last_active":"2024-09-10T16:03:58.627568+0000","last_peered":"2024-09-10T16:03:58.627568+0000","last_clean":"2024-09-10T15:59:14.115595+0000","last_became_active":"2024-09-10T15:59:16.244089+0000","last_became_peered":"2024-09-10T15:59:16.244089+0000","last_unstale":"2024-09-10T16:03:58.627568+0000","last_undegraded":"2024-09-10T16:03:29.603815+0000","last_fullsized":"2024-09-10T15:59:15.774437+0000","mapping_epoch":42,"log_start":"36'2500","ondisk_log_start":"36'2500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32804864,"num_objects":8009,"num_object_clones":0,"num_object_copies":16018,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":247,"num_objects_misplaced":1916,"num_objects_unfound":0,"num_objects_dirty":8009,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8979,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2040,"num_bytes_recovered":8355840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5846},{"shards":"0,2","objects":1916}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103831+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:02:08.103831+0000","last_peered":"2024-09-10T16:02:08.103831+0000","last_clean":"2024-09-10T16:02:08.103831+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:02:08.103831+0000","last_undegraded":"2024-09-10T16:02:08.103831+0000","last_fullsized":"2024-09-10T16:02:08.103831+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'12977","reported_seq":13118,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:58.620849+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:03:58.620849+0000","last_peered":"2024-09-10T16:03:58.620849+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:03:58.620849+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"36'2900","ondisk_log_start":"36'2900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33058816,"num_objects":8071,"num_object_clones":0,"num_object_copies":16142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8071,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9232,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3700},{"shards":"0,2","objects":1920}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104463+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:02:08.104463+0000","last_peered":"2024-09-10T16:02:08.104463+0000","last_clean":"2024-09-10T16:02:08.104463+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:02:08.104463+0000","last_undegraded":"2024-09-10T16:02:08.104463+0000","last_fullsized":"2024-09-10T16:02:08.104463+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104448+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:02:08.104448+0000","last_peered":"2024-09-10T16:02:08.104448+0000","last_clean":"2024-09-10T16:02:08.104448+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:02:08.104448+0000","last_undegraded":"2024-09-10T16:02:08.104448+0000","last_fullsized":"2024-09-10T16:02:08.104448+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'12935","reported_seq":13080,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:58.637312+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:03:58.637312+0000","last_peered":"2024-09-10T16:03:58.637312+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:03:58.637312+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"36'2900","ondisk_log_start":"36'2900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33846214,"num_objects":8265,"num_object_clones":0,"num_object_copies":16530,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8265,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9098,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3942},{"shards":"0,2","objects":1993}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":529502966,"num_objects":129283,"num_object_clones":0,"num_object_copies":258566,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43158,"num_objects_misplaced":1916,"num_objects_unfound":0,"num_objects_dirty":129283,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":162823,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6312,"num_bytes_recovered":25849856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1183117312,"data_stored":1182968261,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160949,"ondisk_log_size":160949,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986248,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":693408,"kb_used_data":395152,"kb_used_omap":4,"kb_used_meta":298235,"kb_avail":93678432,"statfs":{"total":96636764160,"available":95926714368,"internally_reserved":0,"allocated":404635648,"data_stored":404343415,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":305393322},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,3,4,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659194,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":600584,"kb_used_data":303724,"kb_used_omap":4,"kb_used_meta":296827,"kb_avail":93771256,"statfs":{"total":96636764160,"available":96021766144,"internally_reserved":0,"allocated":311013376,"data_stored":310684549,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":303951534},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,2,7,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182476,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":841612,"kb_used_data":458748,"kb_used_omap":4,"kb_used_meta":382843,"kb_avail":93530228,"statfs":{"total":96636764160,"available":95774953472,"internally_reserved":0,"allocated":469757952,"data_stored":469465714,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":392031914},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,4,5,2,5,2,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":468377600,"data_stored":468341470,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":310554624,"data_stored":310481905,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":404185088,"data_stored":404144886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:04:05.122 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:04:05.684 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:04:05.684 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:04:05.706 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":243,"stamp":"2024-09-10T16:04:05.650287+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":520950537,"num_objects":127197,"num_object_clones":0,"num_object_copies":254394,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43204,"num_objects_misplaced":1773,"num_objects_unfound":0,"num_objects_dirty":127197,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":164913,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6414,"num_bytes_recovered":26247187,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160940,"ondisk_log_size":160940,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2116372,"kb_used_data":1146704,"kb_used_omap":12,"kb_used_meta":969587,"kb_avail":280999148,"statfs":{"total":289910292480,"available":287743127552,"internally_reserved":0,"allocated":1174224896,"data_stored":1173311598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":992857090},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,4,5,6,13,7,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-20979712,"num_objects":-5122,"num_object_clones":0,"num_object_copies":-10244,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":82,"num_objects_misplaced":-284,"num_objects_unfound":0,"num_objects_dirty":-5122,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5122,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":205,"num_bytes_recovered":827392,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003131"},"pg_stats":[{"pgid":"2.f","version":"45'12793","reported_seq":12868,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:03.673465+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:04:03.673465+0000","last_peered":"2024-09-10T16:04:03.673465+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:04:03.673465+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"36'2700","ondisk_log_start":"36'2700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32272384,"num_objects":7879,"num_object_clones":0,"num_object_copies":15758,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7879,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12793,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3620},{"shards":"0,2","objects":1801}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'12995","reported_seq":13108,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:04:05.026786+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:04:05.026786+0000","last_peered":"2024-09-10T16:04:05.026786+0000","last_clean":"2024-09-10T16:04:05.026786+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:04:05.026786+0000","last_undegraded":"2024-09-10T16:04:05.026786+0000","last_fullsized":"2024-09-10T16:04:05.026786+0000","mapping_epoch":35,"log_start":"36'2900","ondisk_log_start":"36'2900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32150470,"num_objects":7851,"num_object_clones":0,"num_object_copies":15702,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7851,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":11100,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'12784","reported_seq":21462,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:04:01.460220+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:04:01.460220+0000","last_peered":"2024-09-10T16:04:01.460220+0000","last_clean":"2024-09-10T16:04:01.460220+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:04:01.460220+0000","last_undegraded":"2024-09-10T16:04:01.460220+0000","last_fullsized":"2024-09-10T16:04:01.460220+0000","mapping_epoch":44,"log_start":"36'2700","ondisk_log_start":"36'2700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32821272,"num_objects":8014,"num_object_clones":0,"num_object_copies":16028,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8014,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10788,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'12834","reported_seq":12915,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:04:05.101748+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:04:05.101748+0000","last_peered":"2024-09-10T16:04:05.101748+0000","last_clean":"2024-09-10T16:04:05.101748+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:04:05.101748+0000","last_undegraded":"2024-09-10T16:04:05.101748+0000","last_fullsized":"2024-09-10T16:04:05.101748+0000","mapping_epoch":35,"log_start":"36'2800","ondisk_log_start":"36'2800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32055296,"num_objects":7826,"num_object_clones":0,"num_object_copies":15652,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7826,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12834,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'12953","reported_seq":13083,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:04:03.591392+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:04:03.591392+0000","last_peered":"2024-09-10T16:04:03.591392+0000","last_clean":"2024-09-10T16:04:03.591392+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:04:03.591392+0000","last_undegraded":"2024-09-10T16:04:03.591392+0000","last_fullsized":"2024-09-10T16:04:03.591392+0000","mapping_epoch":34,"log_start":"36'2900","ondisk_log_start":"36'2900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32514531,"num_objects":7939,"num_object_clones":0,"num_object_copies":15878,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7939,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9664,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'13070","reported_seq":13208,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:03.684624+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:04:03.684624+0000","last_peered":"2024-09-10T16:04:03.684624+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:04:03.684624+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"36'3000","ondisk_log_start":"36'3000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32731619,"num_objects":7992,"num_object_clones":0,"num_object_copies":15984,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7992,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9682,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3621},{"shards":"0,2","objects":1834}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'12978","reported_seq":13116,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:03.689202+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:04:03.689202+0000","last_peered":"2024-09-10T16:04:03.689202+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:04:03.689202+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"36'2900","ondisk_log_start":"36'2900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33194950,"num_objects":8106,"num_object_clones":0,"num_object_copies":16212,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8106,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":9477,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3869},{"shards":"0,2","objects":1803}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'13147","reported_seq":13277,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:04:03.669551+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:04:03.669551+0000","last_peered":"2024-09-10T16:04:03.669551+0000","last_clean":"2024-09-10T16:04:03.669551+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:04:03.669551+0000","last_undegraded":"2024-09-10T16:04:03.669551+0000","last_fullsized":"2024-09-10T16:04:03.669551+0000","mapping_epoch":34,"log_start":"36'3100","ondisk_log_start":"36'3100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33452032,"num_objects":8167,"num_object_clones":0,"num_object_copies":16334,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8167,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9994,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104261+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:02:08.104261+0000","last_peered":"2024-09-10T16:02:08.104261+0000","last_clean":"2024-09-10T16:02:08.104261+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:02:08.104261+0000","last_undegraded":"2024-09-10T16:02:08.104261+0000","last_fullsized":"2024-09-10T16:02:08.104261+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'12694","reported_seq":12834,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:05.061347+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:04:05.061347+0000","last_peered":"2024-09-10T16:04:05.061347+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:04:05.061347+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"36'2600","ondisk_log_start":"36'2600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31797731,"num_objects":7764,"num_object_clones":0,"num_object_copies":15528,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7764,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10513,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1661},{"shards":"2","objects":3641}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103774+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:02:08.103774+0000","last_peered":"2024-09-10T16:02:08.103774+0000","last_clean":"2024-09-10T16:02:08.103774+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:02:08.103774+0000","last_undegraded":"2024-09-10T16:02:08.103774+0000","last_fullsized":"2024-09-10T16:02:08.103774+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'12946","reported_seq":13077,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:03.712737+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:04:03.712737+0000","last_peered":"2024-09-10T16:04:03.712737+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:04:03.712737+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"36'2900","ondisk_log_start":"36'2900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32620544,"num_objects":7964,"num_object_clones":0,"num_object_copies":15928,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7964,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9450,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3709},{"shards":"0,2","objects":1768}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":104,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.105055+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:02:08.105055+0000","last_peered":"2024-09-10T16:02:08.105055+0000","last_clean":"2024-09-10T16:02:08.105055+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:02:08.105055+0000","last_undegraded":"2024-09-10T16:02:08.105055+0000","last_fullsized":"2024-09-10T16:02:08.105055+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'12926","reported_seq":13061,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:05.038518+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:04:05.038518+0000","last_peered":"2024-09-10T16:04:05.038518+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:04:05.038518+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"36'2900","ondisk_log_start":"36'2900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32174563,"num_objects":7856,"num_object_clones":0,"num_object_copies":15712,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7856,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10606,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1771},{"shards":"2","objects":3552}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104617+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:02:08.104617+0000","last_peered":"2024-09-10T16:02:08.104617+0000","last_clean":"2024-09-10T16:02:08.104617+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:02:08.104617+0000","last_undegraded":"2024-09-10T16:02:08.104617+0000","last_fullsized":"2024-09-10T16:02:08.104617+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'13157","reported_seq":13286,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:05.065910+0000","last_change":"2024-09-10T15:59:16.174113+0000","last_active":"2024-09-10T16:04:05.065910+0000","last_peered":"2024-09-10T16:04:05.065910+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:04:05.065910+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"36'3100","ondisk_log_start":"36'3100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32821248,"num_objects":8013,"num_object_clones":0,"num_object_copies":16026,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4354,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8013,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10711,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1783},{"shards":"2","objects":3661}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'12934","reported_seq":13069,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:03.601752+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:04:03.601752+0000","last_peered":"2024-09-10T16:04:03.601752+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:04:03.601752+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"36'2900","ondisk_log_start":"36'2900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32374784,"num_objects":7904,"num_object_clones":0,"num_object_copies":15808,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7904,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9565,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3600},{"shards":"0,2","objects":1790}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103316+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:02:08.103316+0000","last_peered":"2024-09-10T16:02:08.103316+0000","last_clean":"2024-09-10T16:02:08.103316+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:02:08.103316+0000","last_undegraded":"2024-09-10T16:02:08.103316+0000","last_fullsized":"2024-09-10T16:02:08.103316+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'12733","reported_seq":17157,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:03.728937+0000","last_change":"2024-09-10T16:03:29.605682+0000","last_active":"2024-09-10T16:04:03.728937+0000","last_peered":"2024-09-10T16:04:03.728937+0000","last_clean":"2024-09-10T15:59:14.115595+0000","last_became_active":"2024-09-10T15:59:16.244089+0000","last_became_peered":"2024-09-10T15:59:16.244089+0000","last_unstale":"2024-09-10T16:04:03.728937+0000","last_undegraded":"2024-09-10T16:03:29.603815+0000","last_fullsized":"2024-09-10T15:59:15.774437+0000","mapping_epoch":42,"log_start":"36'2700","ondisk_log_start":"36'2700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32215040,"num_objects":7865,"num_object_clones":0,"num_object_copies":15730,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":293,"num_objects_misplaced":1773,"num_objects_unfound":0,"num_objects_dirty":7865,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9123,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2140,"num_bytes_recovered":8753152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5799},{"shards":"0,2","objects":1773}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103831+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:02:08.103831+0000","last_peered":"2024-09-10T16:02:08.103831+0000","last_clean":"2024-09-10T16:02:08.103831+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:02:08.103831+0000","last_undegraded":"2024-09-10T16:02:08.103831+0000","last_fullsized":"2024-09-10T16:02:08.103831+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'13123","reported_seq":13265,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:03.653715+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:04:03.653715+0000","last_peered":"2024-09-10T16:04:03.653715+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:04:03.653715+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"36'3100","ondisk_log_start":"36'3100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32460800,"num_objects":7925,"num_object_clones":0,"num_object_copies":15850,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7925,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9378,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3554},{"shards":"0,2","objects":1774}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104463+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:02:08.104463+0000","last_peered":"2024-09-10T16:02:08.104463+0000","last_clean":"2024-09-10T16:02:08.104463+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:02:08.104463+0000","last_undegraded":"2024-09-10T16:02:08.104463+0000","last_fullsized":"2024-09-10T16:02:08.104463+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104448+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:02:08.104448+0000","last_peered":"2024-09-10T16:02:08.104448+0000","last_clean":"2024-09-10T16:02:08.104448+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:02:08.104448+0000","last_undegraded":"2024-09-10T16:02:08.104448+0000","last_fullsized":"2024-09-10T16:02:08.104448+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'13070","reported_seq":13215,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:03.705594+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:04:03.705594+0000","last_peered":"2024-09-10T16:04:03.705594+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:04:03.705594+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"36'3000","ondisk_log_start":"36'3000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33293254,"num_objects":8130,"num_object_clones":0,"num_object_copies":16260,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8130,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9233,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3807},{"shards":"0,2","objects":1858}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":520950518,"num_objects":127195,"num_object_clones":0,"num_object_copies":254390,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43204,"num_objects_misplaced":1773,"num_objects_unfound":0,"num_objects_dirty":127195,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":164911,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6412,"num_bytes_recovered":26247168,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1171718144,"data_stored":1171569093,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160937,"ondisk_log_size":160937,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986249,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":693200,"kb_used_data":390780,"kb_used_omap":4,"kb_used_meta":302395,"kb_avail":93678640,"statfs":{"total":96636764160,"available":95926927360,"internally_reserved":0,"allocated":400158720,"data_stored":399866487,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":309653162},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,0,2,1,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659194,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":600584,"kb_used_data":303724,"kb_used_omap":4,"kb_used_meta":296827,"kb_avail":93771256,"statfs":{"total":96636764160,"available":96021766144,"internally_reserved":0,"allocated":311013376,"data_stored":310684549,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":303951534},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,2,7,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182477,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":822588,"kb_used_data":452200,"kb_used_omap":4,"kb_used_meta":370363,"kb_avail":93549252,"statfs":{"total":96636764160,"available":95794434048,"internally_reserved":0,"allocated":463052800,"data_stored":462760562,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":379252394},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,4,5,2,5,2,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":461492224,"data_stored":461456094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":310554624,"data_stored":310481905,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":399671296,"data_stored":399631094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:04:05.708 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-10T16:04:05.708 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "45'12793", 'reported_seq': 12868, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:04:03.673465+0000', 'last_change': '2024-09-10T15:59:16.281558+0000', 'last_active': '2024-09-10T16:04:03.673465+0000', 'last_peered': '2024-09-10T16:04:03.673465+0000', 'last_clean': '2024-09-10T15:59:14.132456+0000', 'last_became_active': '2024-09-10T15:59:16.274407+0000', 'last_became_peered': '2024-09-10T15:59:16.274407+0000', 'last_unstale': '2024-09-10T16:04:03.673465+0000', 'last_undegraded': '2024-09-10T15:59:15.851279+0000', 'last_fullsized': '2024-09-10T15:59:15.774356+0000', 'mapping_epoch': 42, 'log_start': "36'2700", 'ondisk_log_start': "36'2700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'objects_scrubbed': 0, 'log_size': 10093, 'log_dups_size': 0, 'ondisk_log_size': 10093, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32272384, 'num_objects': 7879, 'num_object_clones': 0, 'num_object_copies': 15758, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4259, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7879, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12793, 'num_write_kb': 41344, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3620}, {'shards': '0,2', 'objects': 1801}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:04:05.708 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-10T16:04:05.708 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "45'13070", 'reported_seq': 13208, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:04:03.684624+0000', 'last_change': '2024-09-10T15:59:16.308264+0000', 'last_active': '2024-09-10T16:04:03.684624+0000', 'last_peered': '2024-09-10T16:04:03.684624+0000', 'last_clean': '2024-09-10T15:59:14.111551+0000', 'last_became_active': '2024-09-10T15:59:16.300040+0000', 'last_became_peered': '2024-09-10T15:59:16.300040+0000', 'last_unstale': '2024-09-10T16:04:03.684624+0000', 'last_undegraded': '2024-09-10T15:59:15.878234+0000', 'last_fullsized': '2024-09-10T15:59:15.769328+0000', 'mapping_epoch': 42, 'log_start': "36'3000", 'ondisk_log_start': "36'3000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3442", 'last_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'objects_scrubbed': 3384, 'log_size': 10070, 'log_dups_size': 0, 'ondisk_log_size': 10070, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1292, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32731619, 'num_objects': 7992, 'num_object_clones': 0, 'num_object_copies': 15984, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4372, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7992, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 9682, 'num_write_kb': 28580, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3621}, {'shards': '0,2', 'objects': 1834}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:04:05.708 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-10T16:04:05.708 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "45'12978", 'reported_seq': 13116, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:04:03.689202+0000', 'last_change': '2024-09-10T15:59:16.331899+0000', 'last_active': '2024-09-10T16:04:03.689202+0000', 'last_peered': '2024-09-10T16:04:03.689202+0000', 'last_clean': '2024-09-10T15:59:14.139436+0000', 'last_became_active': '2024-09-10T15:59:16.318715+0000', 'last_became_peered': '2024-09-10T15:59:16.318715+0000', 'last_unstale': '2024-09-10T16:04:03.689202+0000', 'last_undegraded': '2024-09-10T15:59:15.877524+0000', 'last_fullsized': '2024-09-10T15:59:15.769042+0000', 'mapping_epoch': 42, 'log_start': "36'2900", 'ondisk_log_start': "36'2900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3537", 'last_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'objects_scrubbed': 3497, 'log_size': 10078, 'log_dups_size': 0, 'ondisk_log_size': 10078, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1224, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33194950, 'num_objects': 8106, 'num_object_clones': 0, 'num_object_copies': 16212, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4237, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8106, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 9477, 'num_write_kb': 28172, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3869}, {'shards': '0,2', 'objects': 1803}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:04:05.708 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-10T16:04:05.708 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "45'12694", 'reported_seq': 12834, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:04:05.061347+0000', 'last_change': '2024-09-10T15:59:16.175680+0000', 'last_active': '2024-09-10T16:04:05.061347+0000', 'last_peered': '2024-09-10T16:04:05.061347+0000', 'last_clean': '2024-09-10T15:59:14.088187+0000', 'last_became_active': '2024-09-10T15:59:16.165810+0000', 'last_became_peered': '2024-09-10T15:59:16.165810+0000', 'last_unstale': '2024-09-10T16:04:05.061347+0000', 'last_undegraded': '2024-09-10T15:59:15.822098+0000', 'last_fullsized': '2024-09-10T15:59:15.769123+0000', 'mapping_epoch': 42, 'log_start': "36'2600", 'ondisk_log_start': "36'2600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2218", 'last_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'objects_scrubbed': 2175, 'log_size': 10094, 'log_dups_size': 0, 'ondisk_log_size': 10094, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 825, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31797731, 'num_objects': 7764, 'num_object_clones': 0, 'num_object_copies': 15528, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4124, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7764, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 10513, 'num_write_kb': 32204, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1661}, {'shards': '2', 'objects': 3641}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:04:05.709 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-10T16:04:05.709 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "45'12946", 'reported_seq': 13077, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:04:03.712737+0000', 'last_change': '2024-09-10T15:59:16.277742+0000', 'last_active': '2024-09-10T16:04:03.712737+0000', 'last_peered': '2024-09-10T16:04:03.712737+0000', 'last_clean': '2024-09-10T15:59:14.129081+0000', 'last_became_active': '2024-09-10T15:59:16.269835+0000', 'last_became_peered': '2024-09-10T15:59:16.269835+0000', 'last_unstale': '2024-09-10T16:04:03.712737+0000', 'last_undegraded': '2024-09-10T15:59:15.860857+0000', 'last_fullsized': '2024-09-10T15:59:15.771410+0000', 'mapping_epoch': 42, 'log_start': "36'2900", 'ondisk_log_start': "36'2900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3557", 'last_scrub_stamp': '2024-09-10T15:59:02.493678+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:02.493678+0000', 'objects_scrubbed': 3488, 'log_size': 10046, 'log_dups_size': 0, 'ondisk_log_size': 10046, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1472, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32620544, 'num_objects': 7964, 'num_object_clones': 0, 'num_object_copies': 15928, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4257, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7964, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9450, 'num_write_kb': 27852, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3709}, {'shards': '0,2', 'objects': 1768}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:04:05.709 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-10T16:04:05.709 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "45'12926", 'reported_seq': 13061, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:04:05.038518+0000', 'last_change': '2024-09-10T15:59:16.284311+0000', 'last_active': '2024-09-10T16:04:05.038518+0000', 'last_peered': '2024-09-10T16:04:05.038518+0000', 'last_clean': '2024-09-10T15:59:14.116041+0000', 'last_became_active': '2024-09-10T15:59:16.274543+0000', 'last_became_peered': '2024-09-10T15:59:16.274543+0000', 'last_unstale': '2024-09-10T16:04:05.038518+0000', 'last_undegraded': '2024-09-10T15:59:15.812484+0000', 'last_fullsized': '2024-09-10T15:59:15.770885+0000', 'mapping_epoch': 42, 'log_start': "36'2900", 'ondisk_log_start': "36'2900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2356", 'last_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'objects_scrubbed': 2316, 'log_size': 10026, 'log_dups_size': 0, 'ondisk_log_size': 10026, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 783, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32174563, 'num_objects': 7856, 'num_object_clones': 0, 'num_object_copies': 15712, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7856, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 10606, 'num_write_kb': 32292, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1771}, {'shards': '2', 'objects': 3552}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:04:05.709 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-10T16:04:05.709 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "45'13157", 'reported_seq': 13286, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:04:05.065910+0000', 'last_change': '2024-09-10T15:59:16.174113+0000', 'last_active': '2024-09-10T16:04:05.065910+0000', 'last_peered': '2024-09-10T16:04:05.065910+0000', 'last_clean': '2024-09-10T15:59:14.110654+0000', 'last_became_active': '2024-09-10T15:59:16.164645+0000', 'last_became_peered': '2024-09-10T15:59:16.164645+0000', 'last_unstale': '2024-09-10T16:04:05.065910+0000', 'last_undegraded': '2024-09-10T15:59:15.803593+0000', 'last_fullsized': '2024-09-10T15:59:15.769083+0000', 'mapping_epoch': 42, 'log_start': "36'3100", 'ondisk_log_start': "36'3100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2486", 'last_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'objects_scrubbed': 2440, 'log_size': 10057, 'log_dups_size': 0, 'ondisk_log_size': 10057, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 792, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32821248, 'num_objects': 8013, 'num_object_clones': 0, 'num_object_copies': 16026, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4354, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8013, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10711, 'num_write_kb': 32568, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1783}, {'shards': '2', 'objects': 3661}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:04:05.709 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-10T16:04:05.709 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "45'12934", 'reported_seq': 13069, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:04:03.601752+0000', 'last_change': '2024-09-10T15:59:16.315641+0000', 'last_active': '2024-09-10T16:04:03.601752+0000', 'last_peered': '2024-09-10T16:04:03.601752+0000', 'last_clean': '2024-09-10T15:59:14.117001+0000', 'last_became_active': '2024-09-10T15:59:16.302245+0000', 'last_became_peered': '2024-09-10T15:59:16.302245+0000', 'last_unstale': '2024-09-10T16:04:03.601752+0000', 'last_undegraded': '2024-09-10T15:59:15.845012+0000', 'last_fullsized': '2024-09-10T15:59:15.770337+0000', 'mapping_epoch': 42, 'log_start': "36'2900", 'ondisk_log_start': "36'2900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3404", 'last_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'objects_scrubbed': 3367, 'log_size': 10034, 'log_dups_size': 0, 'ondisk_log_size': 10034, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 983, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32374784, 'num_objects': 7904, 'num_object_clones': 0, 'num_object_copies': 15808, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7904, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9565, 'num_write_kb': 28204, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3600}, {'shards': '0,2', 'objects': 1790}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:04:05.709 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-10T16:04:05.709 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "45'12733", 'reported_seq': 17157, 'reported_epoch': 45, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:04:03.728937+0000', 'last_change': '2024-09-10T16:03:29.605682+0000', 'last_active': '2024-09-10T16:04:03.728937+0000', 'last_peered': '2024-09-10T16:04:03.728937+0000', 'last_clean': '2024-09-10T15:59:14.115595+0000', 'last_became_active': '2024-09-10T15:59:16.244089+0000', 'last_became_peered': '2024-09-10T15:59:16.244089+0000', 'last_unstale': '2024-09-10T16:04:03.728937+0000', 'last_undegraded': '2024-09-10T16:03:29.603815+0000', 'last_fullsized': '2024-09-10T15:59:15.774437+0000', 'mapping_epoch': 42, 'log_start': "36'2700", 'ondisk_log_start': "36'2700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3653", 'last_scrub_stamp': '2024-09-10T15:59:05.298468+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:05.298468+0000', 'objects_scrubbed': 3602, 'log_size': 10033, 'log_dups_size': 0, 'ondisk_log_size': 10033, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1289, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32215040, 'num_objects': 7865, 'num_object_clones': 0, 'num_object_copies': 15730, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 293, 'num_objects_misplaced': 1773, 'num_objects_unfound': 0, 'num_objects_dirty': 7865, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9123, 'num_write_kb': 26772, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2140, 'num_bytes_recovered': 8753152, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5799}, {'shards': '0,2', 'objects': 1773}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:04:05.709 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-10T16:04:05.710 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "45'13123", 'reported_seq': 13265, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:04:03.653715+0000', 'last_change': '2024-09-10T15:59:16.311291+0000', 'last_active': '2024-09-10T16:04:03.653715+0000', 'last_peered': '2024-09-10T16:04:03.653715+0000', 'last_clean': '2024-09-10T15:59:14.116363+0000', 'last_became_active': '2024-09-10T15:59:16.301244+0000', 'last_became_peered': '2024-09-10T15:59:16.301244+0000', 'last_unstale': '2024-09-10T16:04:03.653715+0000', 'last_undegraded': '2024-09-10T15:59:15.883434+0000', 'last_fullsized': '2024-09-10T15:59:15.772756+0000', 'mapping_epoch': 42, 'log_start': "36'3100", 'ondisk_log_start': "36'3100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3789", 'last_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'objects_scrubbed': 3741, 'log_size': 10023, 'log_dups_size': 0, 'ondisk_log_size': 10023, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1440, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32460800, 'num_objects': 7925, 'num_object_clones': 0, 'num_object_copies': 15850, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4373, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7925, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9378, 'num_write_kb': 27124, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3554}, {'shards': '0,2', 'objects': 1774}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:04:05.710 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-10T16:04:05.710 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "45'13070", 'reported_seq': 13215, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:04:03.705594+0000', 'last_change': '2024-09-10T15:59:16.313409+0000', 'last_active': '2024-09-10T16:04:03.705594+0000', 'last_peered': '2024-09-10T16:04:03.705594+0000', 'last_clean': '2024-09-10T15:59:14.129780+0000', 'last_became_active': '2024-09-10T15:59:16.300932+0000', 'last_became_peered': '2024-09-10T15:59:16.300932+0000', 'last_unstale': '2024-09-10T16:04:03.705594+0000', 'last_undegraded': '2024-09-10T15:59:15.887823+0000', 'last_fullsized': '2024-09-10T15:59:15.768513+0000', 'mapping_epoch': 42, 'log_start': "36'3000", 'ondisk_log_start': "36'3000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3880", 'last_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'objects_scrubbed': 3827, 'log_size': 10070, 'log_dups_size': 0, 'ondisk_log_size': 10070, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1397, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33293254, 'num_objects': 8130, 'num_object_clones': 0, 'num_object_copies': 16260, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4325, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8130, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9233, 'num_write_kb': 27072, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3807}, {'shards': '0,2', 'objects': 1858}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:04:05.710 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T16:04:06.385 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:04:06.386 INFO:teuthology.orchestra.run.smithi115.stdout:{"fsid":"7effee29-df05-4351-9741-ee203564ed81","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 43204/254394 objects degraded (16.983%), 11 pgs degraded, 11 pgs undersized","count":22},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":410,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983954,"num_in_osds":3,"osd_in_since":1725983947,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":127197,"data_bytes":520950537,"bytes_used":2167164928,"bytes_avail":287743127552,"bytes_total":289910292480,"degraded_objects":43204,"degraded_total":254394,"degraded_ratio":0.16983104947443728,"misplaced_objects":1773,"misplaced_total":254394,"misplaced_ratio":0.006969503997735796,"recovering_objects_per_sec":17,"recovering_bytes_per_sec":68931,"recovering_keys_per_sec":0,"num_objects_recovered":205,"num_bytes_recovered":827392,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":426},"fsmap":{"epoch":1,"btime":"2024-09-10T15:57:14:967120+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T16:00:13.560776+0000","services":{}},"progress_events":{}} 2024-09-10T16:04:06.407 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-10T16:04:06.408 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:04:06.970 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:04:06.972 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:04:06.992 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":243,"stamp":"2024-09-10T16:04:05.650287+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":520950537,"num_objects":127197,"num_object_clones":0,"num_object_copies":254394,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43204,"num_objects_misplaced":1773,"num_objects_unfound":0,"num_objects_dirty":127197,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":164913,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6414,"num_bytes_recovered":26247187,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160940,"ondisk_log_size":160940,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2116372,"kb_used_data":1146704,"kb_used_omap":12,"kb_used_meta":969587,"kb_avail":280999148,"statfs":{"total":289910292480,"available":287743127552,"internally_reserved":0,"allocated":1174224896,"data_stored":1173311598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":992857090},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,4,5,6,13,7,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-20979712,"num_objects":-5122,"num_object_clones":0,"num_object_copies":-10244,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":82,"num_objects_misplaced":-284,"num_objects_unfound":0,"num_objects_dirty":-5122,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5122,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":205,"num_bytes_recovered":827392,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003131"},"pg_stats":[{"pgid":"2.f","version":"45'12793","reported_seq":12868,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:03.673465+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:04:03.673465+0000","last_peered":"2024-09-10T16:04:03.673465+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:04:03.673465+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"36'2700","ondisk_log_start":"36'2700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32272384,"num_objects":7879,"num_object_clones":0,"num_object_copies":15758,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7879,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12793,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3620},{"shards":"0,2","objects":1801}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'12995","reported_seq":13108,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:04:05.026786+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:04:05.026786+0000","last_peered":"2024-09-10T16:04:05.026786+0000","last_clean":"2024-09-10T16:04:05.026786+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:04:05.026786+0000","last_undegraded":"2024-09-10T16:04:05.026786+0000","last_fullsized":"2024-09-10T16:04:05.026786+0000","mapping_epoch":35,"log_start":"36'2900","ondisk_log_start":"36'2900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32150470,"num_objects":7851,"num_object_clones":0,"num_object_copies":15702,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7851,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":11100,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'12784","reported_seq":21462,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:04:01.460220+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:04:01.460220+0000","last_peered":"2024-09-10T16:04:01.460220+0000","last_clean":"2024-09-10T16:04:01.460220+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:04:01.460220+0000","last_undegraded":"2024-09-10T16:04:01.460220+0000","last_fullsized":"2024-09-10T16:04:01.460220+0000","mapping_epoch":44,"log_start":"36'2700","ondisk_log_start":"36'2700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32821272,"num_objects":8014,"num_object_clones":0,"num_object_copies":16028,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8014,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10788,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'12834","reported_seq":12915,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:04:05.101748+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:04:05.101748+0000","last_peered":"2024-09-10T16:04:05.101748+0000","last_clean":"2024-09-10T16:04:05.101748+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:04:05.101748+0000","last_undegraded":"2024-09-10T16:04:05.101748+0000","last_fullsized":"2024-09-10T16:04:05.101748+0000","mapping_epoch":35,"log_start":"36'2800","ondisk_log_start":"36'2800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32055296,"num_objects":7826,"num_object_clones":0,"num_object_copies":15652,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7826,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12834,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'12953","reported_seq":13083,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:04:03.591392+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:04:03.591392+0000","last_peered":"2024-09-10T16:04:03.591392+0000","last_clean":"2024-09-10T16:04:03.591392+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:04:03.591392+0000","last_undegraded":"2024-09-10T16:04:03.591392+0000","last_fullsized":"2024-09-10T16:04:03.591392+0000","mapping_epoch":34,"log_start":"36'2900","ondisk_log_start":"36'2900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32514531,"num_objects":7939,"num_object_clones":0,"num_object_copies":15878,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7939,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9664,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'13070","reported_seq":13208,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:03.684624+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:04:03.684624+0000","last_peered":"2024-09-10T16:04:03.684624+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:04:03.684624+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"36'3000","ondisk_log_start":"36'3000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32731619,"num_objects":7992,"num_object_clones":0,"num_object_copies":15984,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7992,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9682,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3621},{"shards":"0,2","objects":1834}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'12978","reported_seq":13116,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:03.689202+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:04:03.689202+0000","last_peered":"2024-09-10T16:04:03.689202+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:04:03.689202+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"36'2900","ondisk_log_start":"36'2900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33194950,"num_objects":8106,"num_object_clones":0,"num_object_copies":16212,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8106,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":9477,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3869},{"shards":"0,2","objects":1803}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'13147","reported_seq":13277,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:04:03.669551+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:04:03.669551+0000","last_peered":"2024-09-10T16:04:03.669551+0000","last_clean":"2024-09-10T16:04:03.669551+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:04:03.669551+0000","last_undegraded":"2024-09-10T16:04:03.669551+0000","last_fullsized":"2024-09-10T16:04:03.669551+0000","mapping_epoch":34,"log_start":"36'3100","ondisk_log_start":"36'3100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33452032,"num_objects":8167,"num_object_clones":0,"num_object_copies":16334,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8167,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9994,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104261+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:02:08.104261+0000","last_peered":"2024-09-10T16:02:08.104261+0000","last_clean":"2024-09-10T16:02:08.104261+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:02:08.104261+0000","last_undegraded":"2024-09-10T16:02:08.104261+0000","last_fullsized":"2024-09-10T16:02:08.104261+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'12694","reported_seq":12834,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:05.061347+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:04:05.061347+0000","last_peered":"2024-09-10T16:04:05.061347+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:04:05.061347+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"36'2600","ondisk_log_start":"36'2600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31797731,"num_objects":7764,"num_object_clones":0,"num_object_copies":15528,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7764,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10513,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1661},{"shards":"2","objects":3641}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103774+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:02:08.103774+0000","last_peered":"2024-09-10T16:02:08.103774+0000","last_clean":"2024-09-10T16:02:08.103774+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:02:08.103774+0000","last_undegraded":"2024-09-10T16:02:08.103774+0000","last_fullsized":"2024-09-10T16:02:08.103774+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'12946","reported_seq":13077,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:03.712737+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:04:03.712737+0000","last_peered":"2024-09-10T16:04:03.712737+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:04:03.712737+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"36'2900","ondisk_log_start":"36'2900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32620544,"num_objects":7964,"num_object_clones":0,"num_object_copies":15928,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7964,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9450,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3709},{"shards":"0,2","objects":1768}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":104,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.105055+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:02:08.105055+0000","last_peered":"2024-09-10T16:02:08.105055+0000","last_clean":"2024-09-10T16:02:08.105055+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:02:08.105055+0000","last_undegraded":"2024-09-10T16:02:08.105055+0000","last_fullsized":"2024-09-10T16:02:08.105055+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'12926","reported_seq":13061,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:05.038518+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:04:05.038518+0000","last_peered":"2024-09-10T16:04:05.038518+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:04:05.038518+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"36'2900","ondisk_log_start":"36'2900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32174563,"num_objects":7856,"num_object_clones":0,"num_object_copies":15712,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7856,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10606,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1771},{"shards":"2","objects":3552}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104617+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:02:08.104617+0000","last_peered":"2024-09-10T16:02:08.104617+0000","last_clean":"2024-09-10T16:02:08.104617+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:02:08.104617+0000","last_undegraded":"2024-09-10T16:02:08.104617+0000","last_fullsized":"2024-09-10T16:02:08.104617+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'13157","reported_seq":13286,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:05.065910+0000","last_change":"2024-09-10T15:59:16.174113+0000","last_active":"2024-09-10T16:04:05.065910+0000","last_peered":"2024-09-10T16:04:05.065910+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:04:05.065910+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"36'3100","ondisk_log_start":"36'3100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32821248,"num_objects":8013,"num_object_clones":0,"num_object_copies":16026,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4354,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8013,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10711,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1783},{"shards":"2","objects":3661}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'12934","reported_seq":13069,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:03.601752+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:04:03.601752+0000","last_peered":"2024-09-10T16:04:03.601752+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:04:03.601752+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"36'2900","ondisk_log_start":"36'2900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32374784,"num_objects":7904,"num_object_clones":0,"num_object_copies":15808,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7904,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9565,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3600},{"shards":"0,2","objects":1790}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103316+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:02:08.103316+0000","last_peered":"2024-09-10T16:02:08.103316+0000","last_clean":"2024-09-10T16:02:08.103316+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:02:08.103316+0000","last_undegraded":"2024-09-10T16:02:08.103316+0000","last_fullsized":"2024-09-10T16:02:08.103316+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'12733","reported_seq":17157,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:03.728937+0000","last_change":"2024-09-10T16:03:29.605682+0000","last_active":"2024-09-10T16:04:03.728937+0000","last_peered":"2024-09-10T16:04:03.728937+0000","last_clean":"2024-09-10T15:59:14.115595+0000","last_became_active":"2024-09-10T15:59:16.244089+0000","last_became_peered":"2024-09-10T15:59:16.244089+0000","last_unstale":"2024-09-10T16:04:03.728937+0000","last_undegraded":"2024-09-10T16:03:29.603815+0000","last_fullsized":"2024-09-10T15:59:15.774437+0000","mapping_epoch":42,"log_start":"36'2700","ondisk_log_start":"36'2700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32215040,"num_objects":7865,"num_object_clones":0,"num_object_copies":15730,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":293,"num_objects_misplaced":1773,"num_objects_unfound":0,"num_objects_dirty":7865,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9123,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2140,"num_bytes_recovered":8753152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5799},{"shards":"0,2","objects":1773}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103831+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:02:08.103831+0000","last_peered":"2024-09-10T16:02:08.103831+0000","last_clean":"2024-09-10T16:02:08.103831+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:02:08.103831+0000","last_undegraded":"2024-09-10T16:02:08.103831+0000","last_fullsized":"2024-09-10T16:02:08.103831+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'13123","reported_seq":13265,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:03.653715+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:04:03.653715+0000","last_peered":"2024-09-10T16:04:03.653715+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:04:03.653715+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"36'3100","ondisk_log_start":"36'3100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32460800,"num_objects":7925,"num_object_clones":0,"num_object_copies":15850,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7925,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9378,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3554},{"shards":"0,2","objects":1774}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104463+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:02:08.104463+0000","last_peered":"2024-09-10T16:02:08.104463+0000","last_clean":"2024-09-10T16:02:08.104463+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:02:08.104463+0000","last_undegraded":"2024-09-10T16:02:08.104463+0000","last_fullsized":"2024-09-10T16:02:08.104463+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104448+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:02:08.104448+0000","last_peered":"2024-09-10T16:02:08.104448+0000","last_clean":"2024-09-10T16:02:08.104448+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:02:08.104448+0000","last_undegraded":"2024-09-10T16:02:08.104448+0000","last_fullsized":"2024-09-10T16:02:08.104448+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'13070","reported_seq":13215,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:03.705594+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:04:03.705594+0000","last_peered":"2024-09-10T16:04:03.705594+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:04:03.705594+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"36'3000","ondisk_log_start":"36'3000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33293254,"num_objects":8130,"num_object_clones":0,"num_object_copies":16260,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8130,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9233,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3807},{"shards":"0,2","objects":1858}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":520950518,"num_objects":127195,"num_object_clones":0,"num_object_copies":254390,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43204,"num_objects_misplaced":1773,"num_objects_unfound":0,"num_objects_dirty":127195,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":164911,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6412,"num_bytes_recovered":26247168,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1171718144,"data_stored":1171569093,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160937,"ondisk_log_size":160937,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986249,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":693200,"kb_used_data":390780,"kb_used_omap":4,"kb_used_meta":302395,"kb_avail":93678640,"statfs":{"total":96636764160,"available":95926927360,"internally_reserved":0,"allocated":400158720,"data_stored":399866487,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":309653162},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,0,2,1,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659194,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":600584,"kb_used_data":303724,"kb_used_omap":4,"kb_used_meta":296827,"kb_avail":93771256,"statfs":{"total":96636764160,"available":96021766144,"internally_reserved":0,"allocated":311013376,"data_stored":310684549,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":303951534},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,2,7,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182477,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":822588,"kb_used_data":452200,"kb_used_omap":4,"kb_used_meta":370363,"kb_avail":93549252,"statfs":{"total":96636764160,"available":95794434048,"internally_reserved":0,"allocated":463052800,"data_stored":462760562,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":379252394},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,4,5,2,5,2,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":461492224,"data_stored":461456094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":310554624,"data_stored":310481905,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":399671296,"data_stored":399631094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:04:09.994 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:04:10.574 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:04:10.575 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:04:10.605 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":245,"stamp":"2024-09-10T16:04:09.651497+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":514544393,"num_objects":125633,"num_object_clones":0,"num_object_copies":251266,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43248,"num_objects_misplaced":1621,"num_objects_unfound":0,"num_objects_dirty":125633,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":166477,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6522,"num_bytes_recovered":26689555,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160904,"ondisk_log_size":160904,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2122868,"kb_used_data":1139324,"kb_used_omap":12,"kb_used_meta":983475,"kb_avail":280992652,"statfs":{"total":289910292480,"available":287736475648,"internally_reserved":0,"allocated":1166667776,"data_stored":1165754478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1007078402},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,2,3,9,5,8,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-24137728,"num_objects":-5893,"num_object_clones":0,"num_object_copies":-11786,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":126,"num_objects_misplaced":-436,"num_objects_unfound":0,"num_objects_dirty":-5893,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5893,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":313,"num_bytes_recovered":1269760,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003387"},"pg_stats":[{"pgid":"2.f","version":"45'12940","reported_seq":13015,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:08.664406+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:04:08.664406+0000","last_peered":"2024-09-10T16:04:08.664406+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:04:08.664406+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"36'2900","ondisk_log_start":"36'2900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31670272,"num_objects":7732,"num_object_clones":0,"num_object_copies":15464,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7732,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12940,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3473},{"shards":"0,2","objects":1654}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'12995","reported_seq":13108,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:04:05.026786+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:04:05.026786+0000","last_peered":"2024-09-10T16:04:05.026786+0000","last_clean":"2024-09-10T16:04:05.026786+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:04:05.026786+0000","last_undegraded":"2024-09-10T16:04:05.026786+0000","last_fullsized":"2024-09-10T16:04:05.026786+0000","mapping_epoch":35,"log_start":"36'2900","ondisk_log_start":"36'2900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32150470,"num_objects":7851,"num_object_clones":0,"num_object_copies":15702,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7851,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":11100,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'12926","reported_seq":21604,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:04:06.503903+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:04:06.503903+0000","last_peered":"2024-09-10T16:04:06.503903+0000","last_clean":"2024-09-10T16:04:06.503903+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:04:06.503903+0000","last_undegraded":"2024-09-10T16:04:06.503903+0000","last_fullsized":"2024-09-10T16:04:06.503903+0000","mapping_epoch":44,"log_start":"36'2900","ondisk_log_start":"36'2900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32239640,"num_objects":7872,"num_object_clones":0,"num_object_copies":15744,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7872,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10930,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'12834","reported_seq":12915,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:04:05.101748+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:04:05.101748+0000","last_peered":"2024-09-10T16:04:05.101748+0000","last_clean":"2024-09-10T16:04:05.101748+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:04:05.101748+0000","last_undegraded":"2024-09-10T16:04:05.101748+0000","last_fullsized":"2024-09-10T16:04:05.101748+0000","mapping_epoch":35,"log_start":"36'2800","ondisk_log_start":"36'2800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32055296,"num_objects":7826,"num_object_clones":0,"num_object_copies":15652,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7826,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12834,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'13093","reported_seq":13223,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.759868+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:04:08.759868+0000","last_peered":"2024-09-10T16:04:08.759868+0000","last_clean":"2024-09-10T16:04:08.759868+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:04:08.759868+0000","last_undegraded":"2024-09-10T16:04:08.759868+0000","last_fullsized":"2024-09-10T16:04:08.759868+0000","mapping_epoch":34,"log_start":"36'3000","ondisk_log_start":"36'3000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31941091,"num_objects":7799,"num_object_clones":0,"num_object_copies":15598,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7799,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9804,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'13220","reported_seq":13356,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:08.784043+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:04:08.784043+0000","last_peered":"2024-09-10T16:04:08.784043+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:04:08.784043+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"36'3200","ondisk_log_start":"36'3200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32117219,"num_objects":7842,"num_object_clones":0,"num_object_copies":15684,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7842,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9832,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3471},{"shards":"0,2","objects":1684}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'13126","reported_seq":13264,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:08.644135+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:04:08.644135+0000","last_peered":"2024-09-10T16:04:08.644135+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:04:08.644135+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"36'3100","ondisk_log_start":"36'3100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32588742,"num_objects":7958,"num_object_clones":0,"num_object_copies":15916,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7958,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":9625,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3721},{"shards":"0,2","objects":1655}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'13282","reported_seq":13412,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.691064+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:04:08.691064+0000","last_peered":"2024-09-10T16:04:08.691064+0000","last_clean":"2024-09-10T16:04:08.691064+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:04:08.691064+0000","last_undegraded":"2024-09-10T16:04:08.691064+0000","last_fullsized":"2024-09-10T16:04:08.691064+0000","mapping_epoch":34,"log_start":"36'3200","ondisk_log_start":"36'3200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32899072,"num_objects":8032,"num_object_clones":0,"num_object_copies":16064,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8032,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10129,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104261+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:02:08.104261+0000","last_peered":"2024-09-10T16:02:08.104261+0000","last_clean":"2024-09-10T16:02:08.104261+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:02:08.104261+0000","last_undegraded":"2024-09-10T16:02:08.104261+0000","last_fullsized":"2024-09-10T16:02:08.104261+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'12694","reported_seq":12834,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:05.061347+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:04:05.061347+0000","last_peered":"2024-09-10T16:04:05.061347+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:04:05.061347+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"36'2600","ondisk_log_start":"36'2600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31797731,"num_objects":7764,"num_object_clones":0,"num_object_copies":15528,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7764,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10513,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1661},{"shards":"2","objects":3641}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103774+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:02:08.103774+0000","last_peered":"2024-09-10T16:02:08.103774+0000","last_clean":"2024-09-10T16:02:08.103774+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:02:08.103774+0000","last_undegraded":"2024-09-10T16:02:08.103774+0000","last_fullsized":"2024-09-10T16:02:08.103774+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'13076","reported_seq":13207,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:08.566822+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:04:08.566822+0000","last_peered":"2024-09-10T16:04:08.566822+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:04:08.566822+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"36'3000","ondisk_log_start":"36'3000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32088064,"num_objects":7834,"num_object_clones":0,"num_object_copies":15668,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7834,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9580,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3579},{"shards":"0,2","objects":1638}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":104,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.105055+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:02:08.105055+0000","last_peered":"2024-09-10T16:02:08.105055+0000","last_clean":"2024-09-10T16:02:08.105055+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:02:08.105055+0000","last_undegraded":"2024-09-10T16:02:08.105055+0000","last_fullsized":"2024-09-10T16:02:08.105055+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'12926","reported_seq":13061,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:05.038518+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:04:05.038518+0000","last_peered":"2024-09-10T16:04:05.038518+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:04:05.038518+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"36'2900","ondisk_log_start":"36'2900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32174563,"num_objects":7856,"num_object_clones":0,"num_object_copies":15712,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7856,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10606,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1771},{"shards":"2","objects":3552}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104617+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:02:08.104617+0000","last_peered":"2024-09-10T16:02:08.104617+0000","last_clean":"2024-09-10T16:02:08.104617+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:02:08.104617+0000","last_undegraded":"2024-09-10T16:02:08.104617+0000","last_fullsized":"2024-09-10T16:02:08.104617+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'13157","reported_seq":13286,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:05.065910+0000","last_change":"2024-09-10T15:59:16.174113+0000","last_active":"2024-09-10T16:04:05.065910+0000","last_peered":"2024-09-10T16:04:05.065910+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:04:05.065910+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"36'3100","ondisk_log_start":"36'3100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32821248,"num_objects":8013,"num_object_clones":0,"num_object_copies":16026,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4354,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8013,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10711,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1783},{"shards":"2","objects":3661}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'13074","reported_seq":13208,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:08.733043+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:04:08.733043+0000","last_peered":"2024-09-10T16:04:08.733043+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:04:08.733043+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"36'3000","ondisk_log_start":"36'3000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31801344,"num_objects":7764,"num_object_clones":0,"num_object_copies":15528,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7764,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9705,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3460},{"shards":"0,2","objects":1650}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103316+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:02:08.103316+0000","last_peered":"2024-09-10T16:02:08.103316+0000","last_clean":"2024-09-10T16:02:08.103316+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:02:08.103316+0000","last_undegraded":"2024-09-10T16:02:08.103316+0000","last_fullsized":"2024-09-10T16:02:08.103316+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'12885","reported_seq":17523,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:08.745908+0000","last_change":"2024-09-10T16:03:29.605682+0000","last_active":"2024-09-10T16:04:08.745908+0000","last_peered":"2024-09-10T16:04:08.745908+0000","last_clean":"2024-09-10T15:59:14.115595+0000","last_became_active":"2024-09-10T15:59:16.244089+0000","last_became_peered":"2024-09-10T15:59:16.244089+0000","last_unstale":"2024-09-10T16:04:08.745908+0000","last_undegraded":"2024-09-10T16:03:29.603815+0000","last_fullsized":"2024-09-10T15:59:15.774437+0000","mapping_epoch":42,"log_start":"36'2800","ondisk_log_start":"36'2800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31592448,"num_objects":7713,"num_object_clones":0,"num_object_copies":15426,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":337,"num_objects_misplaced":1621,"num_objects_unfound":0,"num_objects_dirty":7713,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9275,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2248,"num_bytes_recovered":9195520,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5755},{"shards":"0,2","objects":1621}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103831+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:02:08.103831+0000","last_peered":"2024-09-10T16:02:08.103831+0000","last_clean":"2024-09-10T16:02:08.103831+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:02:08.103831+0000","last_undegraded":"2024-09-10T16:02:08.103831+0000","last_fullsized":"2024-09-10T16:02:08.103831+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'13264","reported_seq":13406,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:08.664901+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:04:08.664901+0000","last_peered":"2024-09-10T16:04:08.664901+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:04:08.664901+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"36'3200","ondisk_log_start":"36'3200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31883264,"num_objects":7784,"num_object_clones":0,"num_object_copies":15568,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7784,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9519,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3413},{"shards":"0,2","objects":1633}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104463+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:02:08.104463+0000","last_peered":"2024-09-10T16:02:08.104463+0000","last_clean":"2024-09-10T16:02:08.104463+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:02:08.104463+0000","last_undegraded":"2024-09-10T16:02:08.104463+0000","last_fullsized":"2024-09-10T16:02:08.104463+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104448+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:02:08.104448+0000","last_peered":"2024-09-10T16:02:08.104448+0000","last_clean":"2024-09-10T16:02:08.104448+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:02:08.104448+0000","last_undegraded":"2024-09-10T16:02:08.104448+0000","last_fullsized":"2024-09-10T16:02:08.104448+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'13209","reported_seq":13351,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:08.747541+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:04:08.747541+0000","last_peered":"2024-09-10T16:04:08.747541+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:04:08.747541+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"36'3200","ondisk_log_start":"36'3200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32723910,"num_objects":7991,"num_object_clones":0,"num_object_copies":15982,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7991,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9372,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3668},{"shards":"0,2","objects":1719}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":514544374,"num_objects":125631,"num_object_clones":0,"num_object_copies":251262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43248,"num_objects_misplaced":1621,"num_objects_unfound":0,"num_objects_dirty":125631,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":166475,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6520,"num_bytes_recovered":26689536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1164140544,"data_stored":1163991493,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160901,"ondisk_log_size":160901,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986249,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":693200,"kb_used_data":390780,"kb_used_omap":4,"kb_used_meta":302395,"kb_avail":93678640,"statfs":{"total":96636764160,"available":95926927360,"internally_reserved":0,"allocated":400158720,"data_stored":399866487,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":309653162},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,0,2,1,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659195,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":587532,"kb_used_data":303164,"kb_used_omap":4,"kb_used_meta":284347,"kb_avail":93784308,"statfs":{"total":96636764160,"available":96035131392,"internally_reserved":0,"allocated":310439936,"data_stored":310111109,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":291172014},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,0,4,2,2,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182478,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":842136,"kb_used_data":445380,"kb_used_omap":4,"kb_used_meta":396731,"kb_avail":93529704,"statfs":{"total":96636764160,"available":95774416896,"internally_reserved":0,"allocated":456069120,"data_stored":455776882,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":406253226},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,3,3,2,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":454512640,"data_stored":454476510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":309956608,"data_stored":309883889,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":399671296,"data_stored":399631094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:04:10.606 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:04:11.167 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:04:11.167 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:04:11.201 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":245,"stamp":"2024-09-10T16:04:09.651497+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":514544393,"num_objects":125633,"num_object_clones":0,"num_object_copies":251266,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43248,"num_objects_misplaced":1621,"num_objects_unfound":0,"num_objects_dirty":125633,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":166477,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6522,"num_bytes_recovered":26689555,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160904,"ondisk_log_size":160904,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2122868,"kb_used_data":1139324,"kb_used_omap":12,"kb_used_meta":983475,"kb_avail":280992652,"statfs":{"total":289910292480,"available":287736475648,"internally_reserved":0,"allocated":1166667776,"data_stored":1165754478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1007078402},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,2,3,9,5,8,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-24137728,"num_objects":-5893,"num_object_clones":0,"num_object_copies":-11786,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":126,"num_objects_misplaced":-436,"num_objects_unfound":0,"num_objects_dirty":-5893,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5893,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":313,"num_bytes_recovered":1269760,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003387"},"pg_stats":[{"pgid":"2.f","version":"45'12940","reported_seq":13015,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:08.664406+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:04:08.664406+0000","last_peered":"2024-09-10T16:04:08.664406+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:04:08.664406+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"36'2900","ondisk_log_start":"36'2900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31670272,"num_objects":7732,"num_object_clones":0,"num_object_copies":15464,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7732,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12940,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3473},{"shards":"0,2","objects":1654}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'12995","reported_seq":13108,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:04:05.026786+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:04:05.026786+0000","last_peered":"2024-09-10T16:04:05.026786+0000","last_clean":"2024-09-10T16:04:05.026786+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:04:05.026786+0000","last_undegraded":"2024-09-10T16:04:05.026786+0000","last_fullsized":"2024-09-10T16:04:05.026786+0000","mapping_epoch":35,"log_start":"36'2900","ondisk_log_start":"36'2900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32150470,"num_objects":7851,"num_object_clones":0,"num_object_copies":15702,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7851,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":11100,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'12926","reported_seq":21604,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:04:06.503903+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:04:06.503903+0000","last_peered":"2024-09-10T16:04:06.503903+0000","last_clean":"2024-09-10T16:04:06.503903+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:04:06.503903+0000","last_undegraded":"2024-09-10T16:04:06.503903+0000","last_fullsized":"2024-09-10T16:04:06.503903+0000","mapping_epoch":44,"log_start":"36'2900","ondisk_log_start":"36'2900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32239640,"num_objects":7872,"num_object_clones":0,"num_object_copies":15744,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7872,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10930,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'12834","reported_seq":12915,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:04:05.101748+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:04:05.101748+0000","last_peered":"2024-09-10T16:04:05.101748+0000","last_clean":"2024-09-10T16:04:05.101748+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:04:05.101748+0000","last_undegraded":"2024-09-10T16:04:05.101748+0000","last_fullsized":"2024-09-10T16:04:05.101748+0000","mapping_epoch":35,"log_start":"36'2800","ondisk_log_start":"36'2800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32055296,"num_objects":7826,"num_object_clones":0,"num_object_copies":15652,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7826,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12834,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'13093","reported_seq":13223,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.759868+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:04:08.759868+0000","last_peered":"2024-09-10T16:04:08.759868+0000","last_clean":"2024-09-10T16:04:08.759868+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:04:08.759868+0000","last_undegraded":"2024-09-10T16:04:08.759868+0000","last_fullsized":"2024-09-10T16:04:08.759868+0000","mapping_epoch":34,"log_start":"36'3000","ondisk_log_start":"36'3000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31941091,"num_objects":7799,"num_object_clones":0,"num_object_copies":15598,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7799,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9804,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'13220","reported_seq":13356,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:08.784043+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:04:08.784043+0000","last_peered":"2024-09-10T16:04:08.784043+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:04:08.784043+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"36'3200","ondisk_log_start":"36'3200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32117219,"num_objects":7842,"num_object_clones":0,"num_object_copies":15684,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7842,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9832,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3471},{"shards":"0,2","objects":1684}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'13126","reported_seq":13264,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:08.644135+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:04:08.644135+0000","last_peered":"2024-09-10T16:04:08.644135+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:04:08.644135+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"36'3100","ondisk_log_start":"36'3100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32588742,"num_objects":7958,"num_object_clones":0,"num_object_copies":15916,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7958,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":9625,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3721},{"shards":"0,2","objects":1655}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'13282","reported_seq":13412,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.691064+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:04:08.691064+0000","last_peered":"2024-09-10T16:04:08.691064+0000","last_clean":"2024-09-10T16:04:08.691064+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:04:08.691064+0000","last_undegraded":"2024-09-10T16:04:08.691064+0000","last_fullsized":"2024-09-10T16:04:08.691064+0000","mapping_epoch":34,"log_start":"36'3200","ondisk_log_start":"36'3200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32899072,"num_objects":8032,"num_object_clones":0,"num_object_copies":16064,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8032,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10129,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104261+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:02:08.104261+0000","last_peered":"2024-09-10T16:02:08.104261+0000","last_clean":"2024-09-10T16:02:08.104261+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:02:08.104261+0000","last_undegraded":"2024-09-10T16:02:08.104261+0000","last_fullsized":"2024-09-10T16:02:08.104261+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'12694","reported_seq":12834,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:05.061347+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:04:05.061347+0000","last_peered":"2024-09-10T16:04:05.061347+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:04:05.061347+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"36'2600","ondisk_log_start":"36'2600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31797731,"num_objects":7764,"num_object_clones":0,"num_object_copies":15528,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7764,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10513,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1661},{"shards":"2","objects":3641}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103774+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:02:08.103774+0000","last_peered":"2024-09-10T16:02:08.103774+0000","last_clean":"2024-09-10T16:02:08.103774+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:02:08.103774+0000","last_undegraded":"2024-09-10T16:02:08.103774+0000","last_fullsized":"2024-09-10T16:02:08.103774+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'13076","reported_seq":13207,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:08.566822+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:04:08.566822+0000","last_peered":"2024-09-10T16:04:08.566822+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:04:08.566822+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"36'3000","ondisk_log_start":"36'3000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32088064,"num_objects":7834,"num_object_clones":0,"num_object_copies":15668,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7834,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9580,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3579},{"shards":"0,2","objects":1638}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":104,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.105055+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:02:08.105055+0000","last_peered":"2024-09-10T16:02:08.105055+0000","last_clean":"2024-09-10T16:02:08.105055+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:02:08.105055+0000","last_undegraded":"2024-09-10T16:02:08.105055+0000","last_fullsized":"2024-09-10T16:02:08.105055+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'12926","reported_seq":13061,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:05.038518+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:04:05.038518+0000","last_peered":"2024-09-10T16:04:05.038518+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:04:05.038518+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"36'2900","ondisk_log_start":"36'2900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32174563,"num_objects":7856,"num_object_clones":0,"num_object_copies":15712,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7856,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10606,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1771},{"shards":"2","objects":3552}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104617+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:02:08.104617+0000","last_peered":"2024-09-10T16:02:08.104617+0000","last_clean":"2024-09-10T16:02:08.104617+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:02:08.104617+0000","last_undegraded":"2024-09-10T16:02:08.104617+0000","last_fullsized":"2024-09-10T16:02:08.104617+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'13157","reported_seq":13286,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:05.065910+0000","last_change":"2024-09-10T15:59:16.174113+0000","last_active":"2024-09-10T16:04:05.065910+0000","last_peered":"2024-09-10T16:04:05.065910+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:04:05.065910+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"36'3100","ondisk_log_start":"36'3100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32821248,"num_objects":8013,"num_object_clones":0,"num_object_copies":16026,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4354,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8013,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10711,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1783},{"shards":"2","objects":3661}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'13074","reported_seq":13208,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:08.733043+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:04:08.733043+0000","last_peered":"2024-09-10T16:04:08.733043+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:04:08.733043+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"36'3000","ondisk_log_start":"36'3000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31801344,"num_objects":7764,"num_object_clones":0,"num_object_copies":15528,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7764,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9705,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3460},{"shards":"0,2","objects":1650}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103316+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:02:08.103316+0000","last_peered":"2024-09-10T16:02:08.103316+0000","last_clean":"2024-09-10T16:02:08.103316+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:02:08.103316+0000","last_undegraded":"2024-09-10T16:02:08.103316+0000","last_fullsized":"2024-09-10T16:02:08.103316+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'12885","reported_seq":17523,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:08.745908+0000","last_change":"2024-09-10T16:03:29.605682+0000","last_active":"2024-09-10T16:04:08.745908+0000","last_peered":"2024-09-10T16:04:08.745908+0000","last_clean":"2024-09-10T15:59:14.115595+0000","last_became_active":"2024-09-10T15:59:16.244089+0000","last_became_peered":"2024-09-10T15:59:16.244089+0000","last_unstale":"2024-09-10T16:04:08.745908+0000","last_undegraded":"2024-09-10T16:03:29.603815+0000","last_fullsized":"2024-09-10T15:59:15.774437+0000","mapping_epoch":42,"log_start":"36'2800","ondisk_log_start":"36'2800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31592448,"num_objects":7713,"num_object_clones":0,"num_object_copies":15426,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":337,"num_objects_misplaced":1621,"num_objects_unfound":0,"num_objects_dirty":7713,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9275,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2248,"num_bytes_recovered":9195520,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5755},{"shards":"0,2","objects":1621}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103831+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:02:08.103831+0000","last_peered":"2024-09-10T16:02:08.103831+0000","last_clean":"2024-09-10T16:02:08.103831+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:02:08.103831+0000","last_undegraded":"2024-09-10T16:02:08.103831+0000","last_fullsized":"2024-09-10T16:02:08.103831+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'13264","reported_seq":13406,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:08.664901+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:04:08.664901+0000","last_peered":"2024-09-10T16:04:08.664901+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:04:08.664901+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"36'3200","ondisk_log_start":"36'3200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31883264,"num_objects":7784,"num_object_clones":0,"num_object_copies":15568,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7784,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9519,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3413},{"shards":"0,2","objects":1633}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104463+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:02:08.104463+0000","last_peered":"2024-09-10T16:02:08.104463+0000","last_clean":"2024-09-10T16:02:08.104463+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:02:08.104463+0000","last_undegraded":"2024-09-10T16:02:08.104463+0000","last_fullsized":"2024-09-10T16:02:08.104463+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104448+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:02:08.104448+0000","last_peered":"2024-09-10T16:02:08.104448+0000","last_clean":"2024-09-10T16:02:08.104448+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:02:08.104448+0000","last_undegraded":"2024-09-10T16:02:08.104448+0000","last_fullsized":"2024-09-10T16:02:08.104448+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'13209","reported_seq":13351,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:08.747541+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:04:08.747541+0000","last_peered":"2024-09-10T16:04:08.747541+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:04:08.747541+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"36'3200","ondisk_log_start":"36'3200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32723910,"num_objects":7991,"num_object_clones":0,"num_object_copies":15982,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7991,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9372,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3668},{"shards":"0,2","objects":1719}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":514544374,"num_objects":125631,"num_object_clones":0,"num_object_copies":251262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43248,"num_objects_misplaced":1621,"num_objects_unfound":0,"num_objects_dirty":125631,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":166475,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6520,"num_bytes_recovered":26689536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1164140544,"data_stored":1163991493,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160901,"ondisk_log_size":160901,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986249,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":693200,"kb_used_data":390780,"kb_used_omap":4,"kb_used_meta":302395,"kb_avail":93678640,"statfs":{"total":96636764160,"available":95926927360,"internally_reserved":0,"allocated":400158720,"data_stored":399866487,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":309653162},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,0,2,1,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659195,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":587532,"kb_used_data":303164,"kb_used_omap":4,"kb_used_meta":284347,"kb_avail":93784308,"statfs":{"total":96636764160,"available":96035131392,"internally_reserved":0,"allocated":310439936,"data_stored":310111109,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":291172014},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,0,4,2,2,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182478,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":842136,"kb_used_data":445380,"kb_used_omap":4,"kb_used_meta":396731,"kb_avail":93529704,"statfs":{"total":96636764160,"available":95774416896,"internally_reserved":0,"allocated":456069120,"data_stored":455776882,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":406253226},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,3,3,2,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":454512640,"data_stored":454476510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":309956608,"data_stored":309883889,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":399671296,"data_stored":399631094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:04:11.202 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-10T16:04:11.202 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "45'12940", 'reported_seq': 13015, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:04:08.664406+0000', 'last_change': '2024-09-10T15:59:16.281558+0000', 'last_active': '2024-09-10T16:04:08.664406+0000', 'last_peered': '2024-09-10T16:04:08.664406+0000', 'last_clean': '2024-09-10T15:59:14.132456+0000', 'last_became_active': '2024-09-10T15:59:16.274407+0000', 'last_became_peered': '2024-09-10T15:59:16.274407+0000', 'last_unstale': '2024-09-10T16:04:08.664406+0000', 'last_undegraded': '2024-09-10T15:59:15.851279+0000', 'last_fullsized': '2024-09-10T15:59:15.774356+0000', 'mapping_epoch': 42, 'log_start': "36'2900", 'ondisk_log_start': "36'2900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'objects_scrubbed': 0, 'log_size': 10040, 'log_dups_size': 0, 'ondisk_log_size': 10040, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31670272, 'num_objects': 7732, 'num_object_clones': 0, 'num_object_copies': 15464, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4259, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7732, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12940, 'num_write_kb': 41344, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3473}, {'shards': '0,2', 'objects': 1654}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:04:11.202 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-10T16:04:11.202 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "45'13220", 'reported_seq': 13356, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:04:08.784043+0000', 'last_change': '2024-09-10T15:59:16.308264+0000', 'last_active': '2024-09-10T16:04:08.784043+0000', 'last_peered': '2024-09-10T16:04:08.784043+0000', 'last_clean': '2024-09-10T15:59:14.111551+0000', 'last_became_active': '2024-09-10T15:59:16.300040+0000', 'last_became_peered': '2024-09-10T15:59:16.300040+0000', 'last_unstale': '2024-09-10T16:04:08.784043+0000', 'last_undegraded': '2024-09-10T15:59:15.878234+0000', 'last_fullsized': '2024-09-10T15:59:15.769328+0000', 'mapping_epoch': 42, 'log_start': "36'3200", 'ondisk_log_start': "36'3200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3442", 'last_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'objects_scrubbed': 3384, 'log_size': 10020, 'log_dups_size': 0, 'ondisk_log_size': 10020, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1292, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32117219, 'num_objects': 7842, 'num_object_clones': 0, 'num_object_copies': 15684, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4372, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7842, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 9832, 'num_write_kb': 28580, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3471}, {'shards': '0,2', 'objects': 1684}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:04:11.202 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-10T16:04:11.202 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "45'13126", 'reported_seq': 13264, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:04:08.644135+0000', 'last_change': '2024-09-10T15:59:16.331899+0000', 'last_active': '2024-09-10T16:04:08.644135+0000', 'last_peered': '2024-09-10T16:04:08.644135+0000', 'last_clean': '2024-09-10T15:59:14.139436+0000', 'last_became_active': '2024-09-10T15:59:16.318715+0000', 'last_became_peered': '2024-09-10T15:59:16.318715+0000', 'last_unstale': '2024-09-10T16:04:08.644135+0000', 'last_undegraded': '2024-09-10T15:59:15.877524+0000', 'last_fullsized': '2024-09-10T15:59:15.769042+0000', 'mapping_epoch': 42, 'log_start': "36'3100", 'ondisk_log_start': "36'3100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3537", 'last_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'objects_scrubbed': 3497, 'log_size': 10026, 'log_dups_size': 0, 'ondisk_log_size': 10026, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1224, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32588742, 'num_objects': 7958, 'num_object_clones': 0, 'num_object_copies': 15916, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4237, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7958, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 9625, 'num_write_kb': 28172, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3721}, {'shards': '0,2', 'objects': 1655}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:04:11.203 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-10T16:04:11.203 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "45'12694", 'reported_seq': 12834, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:04:05.061347+0000', 'last_change': '2024-09-10T15:59:16.175680+0000', 'last_active': '2024-09-10T16:04:05.061347+0000', 'last_peered': '2024-09-10T16:04:05.061347+0000', 'last_clean': '2024-09-10T15:59:14.088187+0000', 'last_became_active': '2024-09-10T15:59:16.165810+0000', 'last_became_peered': '2024-09-10T15:59:16.165810+0000', 'last_unstale': '2024-09-10T16:04:05.061347+0000', 'last_undegraded': '2024-09-10T15:59:15.822098+0000', 'last_fullsized': '2024-09-10T15:59:15.769123+0000', 'mapping_epoch': 42, 'log_start': "36'2600", 'ondisk_log_start': "36'2600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2218", 'last_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'objects_scrubbed': 2175, 'log_size': 10094, 'log_dups_size': 0, 'ondisk_log_size': 10094, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 825, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31797731, 'num_objects': 7764, 'num_object_clones': 0, 'num_object_copies': 15528, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4124, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7764, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 10513, 'num_write_kb': 32204, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1661}, {'shards': '2', 'objects': 3641}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:04:11.203 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-10T16:04:11.203 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "45'13076", 'reported_seq': 13207, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:04:08.566822+0000', 'last_change': '2024-09-10T15:59:16.277742+0000', 'last_active': '2024-09-10T16:04:08.566822+0000', 'last_peered': '2024-09-10T16:04:08.566822+0000', 'last_clean': '2024-09-10T15:59:14.129081+0000', 'last_became_active': '2024-09-10T15:59:16.269835+0000', 'last_became_peered': '2024-09-10T15:59:16.269835+0000', 'last_unstale': '2024-09-10T16:04:08.566822+0000', 'last_undegraded': '2024-09-10T15:59:15.860857+0000', 'last_fullsized': '2024-09-10T15:59:15.771410+0000', 'mapping_epoch': 42, 'log_start': "36'3000", 'ondisk_log_start': "36'3000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3557", 'last_scrub_stamp': '2024-09-10T15:59:02.493678+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:02.493678+0000', 'objects_scrubbed': 3488, 'log_size': 10076, 'log_dups_size': 0, 'ondisk_log_size': 10076, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1472, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32088064, 'num_objects': 7834, 'num_object_clones': 0, 'num_object_copies': 15668, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4257, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7834, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9580, 'num_write_kb': 27852, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3579}, {'shards': '0,2', 'objects': 1638}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:04:11.203 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-10T16:04:11.203 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "45'12926", 'reported_seq': 13061, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:04:05.038518+0000', 'last_change': '2024-09-10T15:59:16.284311+0000', 'last_active': '2024-09-10T16:04:05.038518+0000', 'last_peered': '2024-09-10T16:04:05.038518+0000', 'last_clean': '2024-09-10T15:59:14.116041+0000', 'last_became_active': '2024-09-10T15:59:16.274543+0000', 'last_became_peered': '2024-09-10T15:59:16.274543+0000', 'last_unstale': '2024-09-10T16:04:05.038518+0000', 'last_undegraded': '2024-09-10T15:59:15.812484+0000', 'last_fullsized': '2024-09-10T15:59:15.770885+0000', 'mapping_epoch': 42, 'log_start': "36'2900", 'ondisk_log_start': "36'2900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2356", 'last_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'objects_scrubbed': 2316, 'log_size': 10026, 'log_dups_size': 0, 'ondisk_log_size': 10026, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 783, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32174563, 'num_objects': 7856, 'num_object_clones': 0, 'num_object_copies': 15712, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7856, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 10606, 'num_write_kb': 32292, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1771}, {'shards': '2', 'objects': 3552}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:04:11.203 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-10T16:04:11.203 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "45'13157", 'reported_seq': 13286, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:04:05.065910+0000', 'last_change': '2024-09-10T15:59:16.174113+0000', 'last_active': '2024-09-10T16:04:05.065910+0000', 'last_peered': '2024-09-10T16:04:05.065910+0000', 'last_clean': '2024-09-10T15:59:14.110654+0000', 'last_became_active': '2024-09-10T15:59:16.164645+0000', 'last_became_peered': '2024-09-10T15:59:16.164645+0000', 'last_unstale': '2024-09-10T16:04:05.065910+0000', 'last_undegraded': '2024-09-10T15:59:15.803593+0000', 'last_fullsized': '2024-09-10T15:59:15.769083+0000', 'mapping_epoch': 42, 'log_start': "36'3100", 'ondisk_log_start': "36'3100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2486", 'last_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'objects_scrubbed': 2440, 'log_size': 10057, 'log_dups_size': 0, 'ondisk_log_size': 10057, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 792, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32821248, 'num_objects': 8013, 'num_object_clones': 0, 'num_object_copies': 16026, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4354, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8013, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10711, 'num_write_kb': 32568, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1783}, {'shards': '2', 'objects': 3661}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:04:11.203 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-10T16:04:11.203 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "45'13074", 'reported_seq': 13208, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:04:08.733043+0000', 'last_change': '2024-09-10T15:59:16.315641+0000', 'last_active': '2024-09-10T16:04:08.733043+0000', 'last_peered': '2024-09-10T16:04:08.733043+0000', 'last_clean': '2024-09-10T15:59:14.117001+0000', 'last_became_active': '2024-09-10T15:59:16.302245+0000', 'last_became_peered': '2024-09-10T15:59:16.302245+0000', 'last_unstale': '2024-09-10T16:04:08.733043+0000', 'last_undegraded': '2024-09-10T15:59:15.845012+0000', 'last_fullsized': '2024-09-10T15:59:15.770337+0000', 'mapping_epoch': 42, 'log_start': "36'3000", 'ondisk_log_start': "36'3000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3404", 'last_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'objects_scrubbed': 3367, 'log_size': 10074, 'log_dups_size': 0, 'ondisk_log_size': 10074, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 983, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31801344, 'num_objects': 7764, 'num_object_clones': 0, 'num_object_copies': 15528, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7764, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9705, 'num_write_kb': 28204, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3460}, {'shards': '0,2', 'objects': 1650}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:04:11.203 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-10T16:04:11.204 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "45'12885", 'reported_seq': 17523, 'reported_epoch': 45, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:04:08.745908+0000', 'last_change': '2024-09-10T16:03:29.605682+0000', 'last_active': '2024-09-10T16:04:08.745908+0000', 'last_peered': '2024-09-10T16:04:08.745908+0000', 'last_clean': '2024-09-10T15:59:14.115595+0000', 'last_became_active': '2024-09-10T15:59:16.244089+0000', 'last_became_peered': '2024-09-10T15:59:16.244089+0000', 'last_unstale': '2024-09-10T16:04:08.745908+0000', 'last_undegraded': '2024-09-10T16:03:29.603815+0000', 'last_fullsized': '2024-09-10T15:59:15.774437+0000', 'mapping_epoch': 42, 'log_start': "36'2800", 'ondisk_log_start': "36'2800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3653", 'last_scrub_stamp': '2024-09-10T15:59:05.298468+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:05.298468+0000', 'objects_scrubbed': 3602, 'log_size': 10085, 'log_dups_size': 0, 'ondisk_log_size': 10085, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1289, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31592448, 'num_objects': 7713, 'num_object_clones': 0, 'num_object_copies': 15426, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 337, 'num_objects_misplaced': 1621, 'num_objects_unfound': 0, 'num_objects_dirty': 7713, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9275, 'num_write_kb': 26772, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2248, 'num_bytes_recovered': 9195520, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5755}, {'shards': '0,2', 'objects': 1621}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:04:11.204 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-10T16:04:11.204 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "45'13264", 'reported_seq': 13406, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:04:08.664901+0000', 'last_change': '2024-09-10T15:59:16.311291+0000', 'last_active': '2024-09-10T16:04:08.664901+0000', 'last_peered': '2024-09-10T16:04:08.664901+0000', 'last_clean': '2024-09-10T15:59:14.116363+0000', 'last_became_active': '2024-09-10T15:59:16.301244+0000', 'last_became_peered': '2024-09-10T15:59:16.301244+0000', 'last_unstale': '2024-09-10T16:04:08.664901+0000', 'last_undegraded': '2024-09-10T15:59:15.883434+0000', 'last_fullsized': '2024-09-10T15:59:15.772756+0000', 'mapping_epoch': 42, 'log_start': "36'3200", 'ondisk_log_start': "36'3200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3789", 'last_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'objects_scrubbed': 3741, 'log_size': 10064, 'log_dups_size': 0, 'ondisk_log_size': 10064, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1440, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31883264, 'num_objects': 7784, 'num_object_clones': 0, 'num_object_copies': 15568, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4373, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7784, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9519, 'num_write_kb': 27124, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3413}, {'shards': '0,2', 'objects': 1633}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:04:11.204 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-10T16:04:11.204 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "45'13209", 'reported_seq': 13351, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:04:08.747541+0000', 'last_change': '2024-09-10T15:59:16.313409+0000', 'last_active': '2024-09-10T16:04:08.747541+0000', 'last_peered': '2024-09-10T16:04:08.747541+0000', 'last_clean': '2024-09-10T15:59:14.129780+0000', 'last_became_active': '2024-09-10T15:59:16.300932+0000', 'last_became_peered': '2024-09-10T15:59:16.300932+0000', 'last_unstale': '2024-09-10T16:04:08.747541+0000', 'last_undegraded': '2024-09-10T15:59:15.887823+0000', 'last_fullsized': '2024-09-10T15:59:15.768513+0000', 'mapping_epoch': 42, 'log_start': "36'3200", 'ondisk_log_start': "36'3200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3880", 'last_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'objects_scrubbed': 3827, 'log_size': 10009, 'log_dups_size': 0, 'ondisk_log_size': 10009, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1397, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32723910, 'num_objects': 7991, 'num_object_clones': 0, 'num_object_copies': 15982, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4325, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7991, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9372, 'num_write_kb': 27072, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3668}, {'shards': '0,2', 'objects': 1719}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:04:11.204 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T16:04:11.940 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:04:11.940 INFO:teuthology.orchestra.run.smithi115.stdout:{"fsid":"7effee29-df05-4351-9741-ee203564ed81","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 43248/251266 objects degraded (17.212%), 11 pgs degraded, 11 pgs undersized","count":22},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":415,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983954,"num_in_osds":3,"osd_in_since":1725983947,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":125633,"data_bytes":514544393,"bytes_used":2173816832,"bytes_avail":287736475648,"bytes_total":289910292480,"degraded_objects":43248,"degraded_total":251266,"degraded_ratio":0.17212038238360938,"misplaced_objects":1621,"misplaced_total":251266,"misplaced_ratio":0.0064513304625377103,"recovering_objects_per_sec":26,"recovering_bytes_per_sec":105783,"recovering_keys_per_sec":0,"num_objects_recovered":313,"num_bytes_recovered":1269760,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":490},"fsmap":{"epoch":1,"btime":"2024-09-10T15:57:14:967120+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T16:00:13.560776+0000","services":{}},"progress_events":{}} 2024-09-10T16:04:11.959 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-10T16:04:11.959 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:04:12.517 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:04:12.518 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:04:12.542 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":246,"stamp":"2024-09-10T16:04:11.652269+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":511165193,"num_objects":124808,"num_object_clones":0,"num_object_copies":249616,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43248,"num_objects_misplaced":1621,"num_objects_unfound":0,"num_objects_dirty":124808,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":167302,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6522,"num_bytes_recovered":26689555,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160829,"ondisk_log_size":160829,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2125992,"kb_used_data":1134132,"kb_used_omap":12,"kb_used_meta":991795,"kb_avail":280989528,"statfs":{"total":289910292480,"available":287733276672,"internally_reserved":0,"allocated":1161351168,"data_stored":1160437870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1015598082},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,7,6,4,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-21835776,"num_objects":-5331,"num_object_clones":0,"num_object_copies":-10662,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":90,"num_objects_misplaced":-295,"num_objects_unfound":0,"num_objects_dirty":-5331,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5331,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":208,"num_bytes_recovered":839680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003673"},"pg_stats":[{"pgid":"2.f","version":"45'12940","reported_seq":13015,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:08.664406+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:04:08.664406+0000","last_peered":"2024-09-10T16:04:08.664406+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:04:08.664406+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"36'2900","ondisk_log_start":"36'2900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31670272,"num_objects":7732,"num_object_clones":0,"num_object_copies":15464,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7732,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12940,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3473},{"shards":"0,2","objects":1654}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'13155","reported_seq":13267,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:04:10.116003+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:04:10.116003+0000","last_peered":"2024-09-10T16:04:10.116003+0000","last_clean":"2024-09-10T16:04:10.116003+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:04:10.116003+0000","last_undegraded":"2024-09-10T16:04:10.116003+0000","last_fullsized":"2024-09-10T16:04:10.116003+0000","mapping_epoch":35,"log_start":"36'3100","ondisk_log_start":"36'3100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31495110,"num_objects":7691,"num_object_clones":0,"num_object_copies":15382,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7691,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":11260,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'13060","reported_seq":21738,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:04:11.486681+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:04:11.486681+0000","last_peered":"2024-09-10T16:04:11.486681+0000","last_clean":"2024-09-10T16:04:11.486681+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:04:11.486681+0000","last_undegraded":"2024-09-10T16:04:11.486681+0000","last_fullsized":"2024-09-10T16:04:11.486681+0000","mapping_epoch":44,"log_start":"36'3000","ondisk_log_start":"36'3000","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31690776,"num_objects":7738,"num_object_clones":0,"num_object_copies":15476,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7738,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11064,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'12967","reported_seq":13048,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:04:10.090175+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:04:10.090175+0000","last_peered":"2024-09-10T16:04:10.090175+0000","last_clean":"2024-09-10T16:04:10.090175+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:04:10.090175+0000","last_undegraded":"2024-09-10T16:04:10.090175+0000","last_fullsized":"2024-09-10T16:04:10.090175+0000","mapping_epoch":35,"log_start":"36'2900","ondisk_log_start":"36'2900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31510528,"num_objects":7693,"num_object_clones":0,"num_object_copies":15386,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7693,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12967,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'13093","reported_seq":13223,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.759868+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:04:08.759868+0000","last_peered":"2024-09-10T16:04:08.759868+0000","last_clean":"2024-09-10T16:04:08.759868+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:04:08.759868+0000","last_undegraded":"2024-09-10T16:04:08.759868+0000","last_fullsized":"2024-09-10T16:04:08.759868+0000","mapping_epoch":34,"log_start":"36'3000","ondisk_log_start":"36'3000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31941091,"num_objects":7799,"num_object_clones":0,"num_object_copies":15598,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7799,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9804,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'13220","reported_seq":13356,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:08.784043+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:04:08.784043+0000","last_peered":"2024-09-10T16:04:08.784043+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:04:08.784043+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"36'3200","ondisk_log_start":"36'3200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32117219,"num_objects":7842,"num_object_clones":0,"num_object_copies":15684,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7842,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9832,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3471},{"shards":"0,2","objects":1684}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'13126","reported_seq":13264,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:08.644135+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:04:08.644135+0000","last_peered":"2024-09-10T16:04:08.644135+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:04:08.644135+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"36'3100","ondisk_log_start":"36'3100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32588742,"num_objects":7958,"num_object_clones":0,"num_object_copies":15916,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7958,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":9625,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3721},{"shards":"0,2","objects":1655}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'13282","reported_seq":13412,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.691064+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:04:08.691064+0000","last_peered":"2024-09-10T16:04:08.691064+0000","last_clean":"2024-09-10T16:04:08.691064+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:04:08.691064+0000","last_undegraded":"2024-09-10T16:04:08.691064+0000","last_fullsized":"2024-09-10T16:04:08.691064+0000","mapping_epoch":34,"log_start":"36'3200","ondisk_log_start":"36'3200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32899072,"num_objects":8032,"num_object_clones":0,"num_object_copies":16064,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8032,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10129,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104261+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:02:08.104261+0000","last_peered":"2024-09-10T16:02:08.104261+0000","last_clean":"2024-09-10T16:02:08.104261+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:02:08.104261+0000","last_undegraded":"2024-09-10T16:02:08.104261+0000","last_fullsized":"2024-09-10T16:02:08.104261+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'12814","reported_seq":12954,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:10.063902+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:04:10.063902+0000","last_peered":"2024-09-10T16:04:10.063902+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:04:10.063902+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"36'2800","ondisk_log_start":"36'2800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31306211,"num_objects":7644,"num_object_clones":0,"num_object_copies":15288,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7644,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10633,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1541},{"shards":"2","objects":3521}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103774+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:02:08.103774+0000","last_peered":"2024-09-10T16:02:08.103774+0000","last_clean":"2024-09-10T16:02:08.103774+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:02:08.103774+0000","last_undegraded":"2024-09-10T16:02:08.103774+0000","last_fullsized":"2024-09-10T16:02:08.103774+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'13076","reported_seq":13207,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:08.566822+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:04:08.566822+0000","last_peered":"2024-09-10T16:04:08.566822+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:04:08.566822+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"36'3000","ondisk_log_start":"36'3000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32088064,"num_objects":7834,"num_object_clones":0,"num_object_copies":15668,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7834,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9580,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3579},{"shards":"0,2","objects":1638}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":104,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.105055+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:02:08.105055+0000","last_peered":"2024-09-10T16:02:08.105055+0000","last_clean":"2024-09-10T16:02:08.105055+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:02:08.105055+0000","last_undegraded":"2024-09-10T16:02:08.105055+0000","last_fullsized":"2024-09-10T16:02:08.105055+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'13057","reported_seq":13192,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:10.003578+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:04:10.003578+0000","last_peered":"2024-09-10T16:04:10.003578+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:04:10.003578+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"36'3000","ondisk_log_start":"36'3000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31637987,"num_objects":7725,"num_object_clones":0,"num_object_copies":15450,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7725,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10737,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1640},{"shards":"2","objects":3421}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104617+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:02:08.104617+0000","last_peered":"2024-09-10T16:02:08.104617+0000","last_clean":"2024-09-10T16:02:08.104617+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:02:08.104617+0000","last_undegraded":"2024-09-10T16:02:08.104617+0000","last_fullsized":"2024-09-10T16:02:08.104617+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'13304","reported_seq":13433,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:10.016694+0000","last_change":"2024-09-10T15:59:16.174113+0000","last_active":"2024-09-10T16:04:10.016694+0000","last_peered":"2024-09-10T16:04:10.016694+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:04:10.016694+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"36'3300","ondisk_log_start":"36'3300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32219136,"num_objects":7866,"num_object_clones":0,"num_object_copies":15732,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4354,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7866,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10858,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1636},{"shards":"2","objects":3514}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'13074","reported_seq":13208,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:08.733043+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:04:08.733043+0000","last_peered":"2024-09-10T16:04:08.733043+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:04:08.733043+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"36'3000","ondisk_log_start":"36'3000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31801344,"num_objects":7764,"num_object_clones":0,"num_object_copies":15528,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7764,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9705,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3460},{"shards":"0,2","objects":1650}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103316+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:02:08.103316+0000","last_peered":"2024-09-10T16:02:08.103316+0000","last_clean":"2024-09-10T16:02:08.103316+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:02:08.103316+0000","last_undegraded":"2024-09-10T16:02:08.103316+0000","last_fullsized":"2024-09-10T16:02:08.103316+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'12885","reported_seq":17523,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:08.745908+0000","last_change":"2024-09-10T16:03:29.605682+0000","last_active":"2024-09-10T16:04:08.745908+0000","last_peered":"2024-09-10T16:04:08.745908+0000","last_clean":"2024-09-10T15:59:14.115595+0000","last_became_active":"2024-09-10T15:59:16.244089+0000","last_became_peered":"2024-09-10T15:59:16.244089+0000","last_unstale":"2024-09-10T16:04:08.745908+0000","last_undegraded":"2024-09-10T16:03:29.603815+0000","last_fullsized":"2024-09-10T15:59:15.774437+0000","mapping_epoch":42,"log_start":"36'2800","ondisk_log_start":"36'2800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31592448,"num_objects":7713,"num_object_clones":0,"num_object_copies":15426,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":337,"num_objects_misplaced":1621,"num_objects_unfound":0,"num_objects_dirty":7713,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9275,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2248,"num_bytes_recovered":9195520,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5755},{"shards":"0,2","objects":1621}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103831+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:02:08.103831+0000","last_peered":"2024-09-10T16:02:08.103831+0000","last_clean":"2024-09-10T16:02:08.103831+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:02:08.103831+0000","last_undegraded":"2024-09-10T16:02:08.103831+0000","last_fullsized":"2024-09-10T16:02:08.103831+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'13264","reported_seq":13406,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:08.664901+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:04:08.664901+0000","last_peered":"2024-09-10T16:04:08.664901+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:04:08.664901+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"36'3200","ondisk_log_start":"36'3200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31883264,"num_objects":7784,"num_object_clones":0,"num_object_copies":15568,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7784,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9519,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3413},{"shards":"0,2","objects":1633}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104463+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:02:08.104463+0000","last_peered":"2024-09-10T16:02:08.104463+0000","last_clean":"2024-09-10T16:02:08.104463+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:02:08.104463+0000","last_undegraded":"2024-09-10T16:02:08.104463+0000","last_fullsized":"2024-09-10T16:02:08.104463+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104448+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:02:08.104448+0000","last_peered":"2024-09-10T16:02:08.104448+0000","last_clean":"2024-09-10T16:02:08.104448+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:02:08.104448+0000","last_undegraded":"2024-09-10T16:02:08.104448+0000","last_fullsized":"2024-09-10T16:02:08.104448+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'13209","reported_seq":13351,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:08.747541+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:04:08.747541+0000","last_peered":"2024-09-10T16:04:08.747541+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:04:08.747541+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"36'3200","ondisk_log_start":"36'3200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32723910,"num_objects":7991,"num_object_clones":0,"num_object_copies":15982,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7991,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9372,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3668},{"shards":"0,2","objects":1719}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":511165174,"num_objects":124806,"num_object_clones":0,"num_object_copies":249612,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43248,"num_objects_misplaced":1621,"num_objects_unfound":0,"num_objects_dirty":124806,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":167300,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6520,"num_bytes_recovered":26689536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1158938624,"data_stored":1158789573,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160826,"ondisk_log_size":160826,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986250,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":692752,"kb_used_data":386176,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":93679088,"statfs":{"total":96636764160,"available":95927386112,"internally_reserved":0,"allocated":395444224,"data_stored":395151991,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":313913002},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659196,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":591104,"kb_used_data":302576,"kb_used_omap":4,"kb_used_meta":288507,"kb_avail":93780736,"statfs":{"total":96636764160,"available":96031473664,"internally_reserved":0,"allocated":309837824,"data_stored":309508997,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":295431854},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,2,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182478,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":842136,"kb_used_data":445380,"kb_used_omap":4,"kb_used_meta":396731,"kb_avail":93529704,"statfs":{"total":96636764160,"available":95774416896,"internally_reserved":0,"allocated":456069120,"data_stored":455776882,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":406253226},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,3,3,2,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":454512640,"data_stored":454476510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":309395456,"data_stored":309322737,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":395030528,"data_stored":394990326,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:04:15.545 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:04:16.128 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:04:16.129 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:04:16.150 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":248,"stamp":"2024-09-10T16:04:15.653785+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502801161,"num_objects":122766,"num_object_clones":0,"num_object_copies":245532,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43291,"num_objects_misplaced":1465,"num_objects_unfound":0,"num_objects_dirty":122766,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":169344,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6635,"num_bytes_recovered":27152403,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160971,"ondisk_log_size":160971,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2119716,"kb_used_data":1123700,"kb_used_omap":12,"kb_used_meta":995955,"kb_avail":280995804,"statfs":{"total":289910292480,"available":287739703296,"internally_reserved":0,"allocated":1150668800,"data_stored":1149755502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1019857922},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,5,4,5,3,3,8],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-26701824,"num_objects":-6519,"num_object_clones":0,"num_object_copies":-13038,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":133,"num_objects_misplaced":-451,"num_objects_unfound":0,"num_objects_dirty":-6519,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6519,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":321,"num_bytes_recovered":1302528,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004066"},"pg_stats":[{"pgid":"2.f","version":"45'13070","reported_seq":13145,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:13.828013+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:04:13.828013+0000","last_peered":"2024-09-10T16:04:13.828013+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:04:13.828013+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"36'3000","ondisk_log_start":"36'3000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31137792,"num_objects":7602,"num_object_clones":0,"num_object_copies":15204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7602,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13070,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3343},{"shards":"0,2","objects":1524}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'13295","reported_seq":13408,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:04:15.133833+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:04:15.133833+0000","last_peered":"2024-09-10T16:04:15.133833+0000","last_clean":"2024-09-10T16:04:15.133833+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:04:15.133833+0000","last_undegraded":"2024-09-10T16:04:15.133833+0000","last_fullsized":"2024-09-10T16:04:15.133833+0000","mapping_epoch":35,"log_start":"36'3200","ondisk_log_start":"36'3200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30921670,"num_objects":7551,"num_object_clones":0,"num_object_copies":15102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7551,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":11400,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'13060","reported_seq":21738,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:04:11.486681+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:04:11.486681+0000","last_peered":"2024-09-10T16:04:11.486681+0000","last_clean":"2024-09-10T16:04:11.486681+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:04:11.486681+0000","last_undegraded":"2024-09-10T16:04:11.486681+0000","last_fullsized":"2024-09-10T16:04:11.486681+0000","mapping_epoch":44,"log_start":"36'3000","ondisk_log_start":"36'3000","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31690776,"num_objects":7738,"num_object_clones":0,"num_object_copies":15476,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7738,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11064,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'13090","reported_seq":13171,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:04:15.020217+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:04:15.020217+0000","last_peered":"2024-09-10T16:04:15.020217+0000","last_clean":"2024-09-10T16:04:15.020217+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:04:15.020217+0000","last_undegraded":"2024-09-10T16:04:15.020217+0000","last_fullsized":"2024-09-10T16:04:15.020217+0000","mapping_epoch":35,"log_start":"36'3000","ondisk_log_start":"36'3000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31006720,"num_objects":7570,"num_object_clones":0,"num_object_copies":15140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7570,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13090,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'13228","reported_seq":13358,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:04:13.709729+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:04:13.709729+0000","last_peered":"2024-09-10T16:04:13.709729+0000","last_clean":"2024-09-10T16:04:13.709729+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:04:13.709729+0000","last_undegraded":"2024-09-10T16:04:13.709729+0000","last_fullsized":"2024-09-10T16:04:13.709729+0000","mapping_epoch":34,"log_start":"36'3200","ondisk_log_start":"36'3200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31388131,"num_objects":7664,"num_object_clones":0,"num_object_copies":15328,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7664,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9939,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'13347","reported_seq":13485,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:13.752635+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:04:13.752635+0000","last_peered":"2024-09-10T16:04:13.752635+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:04:13.752635+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"36'3300","ondisk_log_start":"36'3300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31597027,"num_objects":7715,"num_object_clones":0,"num_object_copies":15430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7715,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9959,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3344},{"shards":"0,2","objects":1557}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'13259","reported_seq":13396,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:13.757147+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:04:13.757147+0000","last_peered":"2024-09-10T16:04:13.757147+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:04:13.757147+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"36'3200","ondisk_log_start":"36'3200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32043974,"num_objects":7825,"num_object_clones":0,"num_object_copies":15650,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7825,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":9758,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3588},{"shards":"0,2","objects":1522}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'13437","reported_seq":13567,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:04:13.758911+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:04:13.758911+0000","last_peered":"2024-09-10T16:04:13.758911+0000","last_clean":"2024-09-10T16:04:13.758911+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:04:13.758911+0000","last_undegraded":"2024-09-10T16:04:13.758911+0000","last_fullsized":"2024-09-10T16:04:13.758911+0000","mapping_epoch":34,"log_start":"36'3400","ondisk_log_start":"36'3400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32264192,"num_objects":7877,"num_object_clones":0,"num_object_copies":15754,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7877,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10284,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104261+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:02:08.104261+0000","last_peered":"2024-09-10T16:02:08.104261+0000","last_clean":"2024-09-10T16:02:08.104261+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:02:08.104261+0000","last_undegraded":"2024-09-10T16:02:08.104261+0000","last_fullsized":"2024-09-10T16:02:08.104261+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'12950","reported_seq":13090,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:15.085238+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:04:15.085238+0000","last_peered":"2024-09-10T16:04:15.085238+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:04:15.085238+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"36'2900","ondisk_log_start":"36'2900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30749155,"num_objects":7508,"num_object_clones":0,"num_object_copies":15016,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7508,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10769,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1405},{"shards":"2","objects":3385}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103774+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:02:08.103774+0000","last_peered":"2024-09-10T16:02:08.103774+0000","last_clean":"2024-09-10T16:02:08.103774+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:02:08.103774+0000","last_undegraded":"2024-09-10T16:02:08.103774+0000","last_fullsized":"2024-09-10T16:02:08.103774+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'13224","reported_seq":13355,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:13.706761+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:04:13.706761+0000","last_peered":"2024-09-10T16:04:13.706761+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:04:13.706761+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"36'3200","ondisk_log_start":"36'3200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31481856,"num_objects":7686,"num_object_clones":0,"num_object_copies":15372,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7686,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9728,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3431},{"shards":"0,2","objects":1490}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":104,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.105055+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:02:08.105055+0000","last_peered":"2024-09-10T16:02:08.105055+0000","last_clean":"2024-09-10T16:02:08.105055+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:02:08.105055+0000","last_undegraded":"2024-09-10T16:02:08.105055+0000","last_fullsized":"2024-09-10T16:02:08.105055+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'13197","reported_seq":13332,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:15.121699+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:04:15.121699+0000","last_peered":"2024-09-10T16:04:15.121699+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:04:15.121699+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"36'3100","ondisk_log_start":"36'3100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31064547,"num_objects":7585,"num_object_clones":0,"num_object_copies":15170,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7585,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10877,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1500},{"shards":"2","objects":3281}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104617+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:02:08.104617+0000","last_peered":"2024-09-10T16:02:08.104617+0000","last_clean":"2024-09-10T16:02:08.104617+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:02:08.104617+0000","last_undegraded":"2024-09-10T16:02:08.104617+0000","last_fullsized":"2024-09-10T16:02:08.104617+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'13436","reported_seq":13565,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:15.071881+0000","last_change":"2024-09-10T15:59:16.174113+0000","last_active":"2024-09-10T16:04:15.071881+0000","last_peered":"2024-09-10T16:04:15.071881+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:04:15.071881+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"36'3400","ondisk_log_start":"36'3400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31678464,"num_objects":7734,"num_object_clones":0,"num_object_copies":15468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4354,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7734,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10990,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1504},{"shards":"2","objects":3382}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'13198","reported_seq":13333,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:13.847671+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:04:13.847671+0000","last_peered":"2024-09-10T16:04:13.847671+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:04:13.847671+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"36'3100","ondisk_log_start":"36'3100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31293440,"num_objects":7640,"num_object_clones":0,"num_object_copies":15280,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7640,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9829,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3336},{"shards":"0,2","objects":1526}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103316+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:02:08.103316+0000","last_peered":"2024-09-10T16:02:08.103316+0000","last_clean":"2024-09-10T16:02:08.103316+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:02:08.103316+0000","last_undegraded":"2024-09-10T16:02:08.103316+0000","last_fullsized":"2024-09-10T16:02:08.103316+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'13028","reported_seq":17894,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:13.823516+0000","last_change":"2024-09-10T16:03:29.605682+0000","last_active":"2024-09-10T16:04:13.823516+0000","last_peered":"2024-09-10T16:04:13.823516+0000","last_clean":"2024-09-10T15:59:14.115595+0000","last_became_active":"2024-09-10T15:59:16.244089+0000","last_became_peered":"2024-09-10T15:59:16.244089+0000","last_unstale":"2024-09-10T16:04:13.823516+0000","last_undegraded":"2024-09-10T16:03:29.603815+0000","last_fullsized":"2024-09-10T15:59:15.774437+0000","mapping_epoch":42,"log_start":"36'3000","ondisk_log_start":"36'3000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31006720,"num_objects":7570,"num_object_clones":0,"num_object_copies":15140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":380,"num_objects_misplaced":1465,"num_objects_unfound":0,"num_objects_dirty":7570,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9418,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2361,"num_bytes_recovered":9658368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5725},{"shards":"0,2","objects":1465}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103831+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:02:08.103831+0000","last_peered":"2024-09-10T16:02:08.103831+0000","last_clean":"2024-09-10T16:02:08.103831+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:02:08.103831+0000","last_undegraded":"2024-09-10T16:02:08.103831+0000","last_fullsized":"2024-09-10T16:02:08.103831+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'13394","reported_seq":13536,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:13.769537+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:04:13.769537+0000","last_peered":"2024-09-10T16:04:13.769537+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:04:13.769537+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"36'3300","ondisk_log_start":"36'3300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31350784,"num_objects":7654,"num_object_clones":0,"num_object_copies":15308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7654,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9649,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3283},{"shards":"0,2","objects":1503}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104463+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:02:08.104463+0000","last_peered":"2024-09-10T16:02:08.104463+0000","last_clean":"2024-09-10T16:02:08.104463+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:02:08.104463+0000","last_undegraded":"2024-09-10T16:02:08.104463+0000","last_fullsized":"2024-09-10T16:02:08.104463+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104448+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:02:08.104448+0000","last_peered":"2024-09-10T16:02:08.104448+0000","last_clean":"2024-09-10T16:02:08.104448+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:02:08.104448+0000","last_undegraded":"2024-09-10T16:02:08.104448+0000","last_fullsized":"2024-09-10T16:02:08.104448+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'13355","reported_seq":13498,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:13.758559+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:04:13.758559+0000","last_peered":"2024-09-10T16:04:13.758559+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:04:13.758559+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"36'3300","ondisk_log_start":"36'3300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32125894,"num_objects":7845,"num_object_clones":0,"num_object_copies":15690,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7845,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9518,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3522},{"shards":"0,2","objects":1573}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":502801142,"num_objects":122764,"num_object_clones":0,"num_object_copies":245528,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43291,"num_objects_misplaced":1465,"num_objects_unfound":0,"num_objects_dirty":122764,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":169342,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6633,"num_bytes_recovered":27152384,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1147633664,"data_stored":1147484613,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160968,"ondisk_log_size":160968,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986251,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":689208,"kb_used_data":382632,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":93682632,"statfs":{"total":96636764160,"available":95931015168,"internally_reserved":0,"allocated":391815168,"data_stored":391522935,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":313913002},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,3,1],"upper_bound":16},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659196,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":591104,"kb_used_data":302576,"kb_used_omap":4,"kb_used_meta":288507,"kb_avail":93780736,"statfs":{"total":96636764160,"available":96031473664,"internally_reserved":0,"allocated":309837824,"data_stored":309508997,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":295431854},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,2,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182479,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":839404,"kb_used_data":438492,"kb_used_omap":4,"kb_used_meta":400891,"kb_avail":93532436,"statfs":{"total":96636764160,"available":95777214464,"internally_reserved":0,"allocated":449015808,"data_stored":448723570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":410513066},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,0,3,1,3,8],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":447692800,"data_stored":447656670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":309395456,"data_stored":309322737,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":390545408,"data_stored":390505206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:04:16.152 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:04:16.742 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:04:16.743 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:04:16.766 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":248,"stamp":"2024-09-10T16:04:15.653785+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502801161,"num_objects":122766,"num_object_clones":0,"num_object_copies":245532,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43291,"num_objects_misplaced":1465,"num_objects_unfound":0,"num_objects_dirty":122766,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":169344,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6635,"num_bytes_recovered":27152403,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160971,"ondisk_log_size":160971,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2119716,"kb_used_data":1123700,"kb_used_omap":12,"kb_used_meta":995955,"kb_avail":280995804,"statfs":{"total":289910292480,"available":287739703296,"internally_reserved":0,"allocated":1150668800,"data_stored":1149755502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1019857922},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,5,4,5,3,3,8],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-26701824,"num_objects":-6519,"num_object_clones":0,"num_object_copies":-13038,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":133,"num_objects_misplaced":-451,"num_objects_unfound":0,"num_objects_dirty":-6519,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6519,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":321,"num_bytes_recovered":1302528,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004066"},"pg_stats":[{"pgid":"2.f","version":"45'13070","reported_seq":13145,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:13.828013+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:04:13.828013+0000","last_peered":"2024-09-10T16:04:13.828013+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:04:13.828013+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"36'3000","ondisk_log_start":"36'3000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31137792,"num_objects":7602,"num_object_clones":0,"num_object_copies":15204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7602,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13070,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3343},{"shards":"0,2","objects":1524}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'13295","reported_seq":13408,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:04:15.133833+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:04:15.133833+0000","last_peered":"2024-09-10T16:04:15.133833+0000","last_clean":"2024-09-10T16:04:15.133833+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:04:15.133833+0000","last_undegraded":"2024-09-10T16:04:15.133833+0000","last_fullsized":"2024-09-10T16:04:15.133833+0000","mapping_epoch":35,"log_start":"36'3200","ondisk_log_start":"36'3200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30921670,"num_objects":7551,"num_object_clones":0,"num_object_copies":15102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7551,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":11400,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'13060","reported_seq":21738,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:04:11.486681+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:04:11.486681+0000","last_peered":"2024-09-10T16:04:11.486681+0000","last_clean":"2024-09-10T16:04:11.486681+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:04:11.486681+0000","last_undegraded":"2024-09-10T16:04:11.486681+0000","last_fullsized":"2024-09-10T16:04:11.486681+0000","mapping_epoch":44,"log_start":"36'3000","ondisk_log_start":"36'3000","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31690776,"num_objects":7738,"num_object_clones":0,"num_object_copies":15476,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7738,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11064,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'13090","reported_seq":13171,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:04:15.020217+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:04:15.020217+0000","last_peered":"2024-09-10T16:04:15.020217+0000","last_clean":"2024-09-10T16:04:15.020217+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:04:15.020217+0000","last_undegraded":"2024-09-10T16:04:15.020217+0000","last_fullsized":"2024-09-10T16:04:15.020217+0000","mapping_epoch":35,"log_start":"36'3000","ondisk_log_start":"36'3000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31006720,"num_objects":7570,"num_object_clones":0,"num_object_copies":15140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7570,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13090,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'13228","reported_seq":13358,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:04:13.709729+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:04:13.709729+0000","last_peered":"2024-09-10T16:04:13.709729+0000","last_clean":"2024-09-10T16:04:13.709729+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:04:13.709729+0000","last_undegraded":"2024-09-10T16:04:13.709729+0000","last_fullsized":"2024-09-10T16:04:13.709729+0000","mapping_epoch":34,"log_start":"36'3200","ondisk_log_start":"36'3200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31388131,"num_objects":7664,"num_object_clones":0,"num_object_copies":15328,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7664,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9939,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'13347","reported_seq":13485,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:13.752635+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:04:13.752635+0000","last_peered":"2024-09-10T16:04:13.752635+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:04:13.752635+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"36'3300","ondisk_log_start":"36'3300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31597027,"num_objects":7715,"num_object_clones":0,"num_object_copies":15430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7715,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9959,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3344},{"shards":"0,2","objects":1557}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'13259","reported_seq":13396,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:13.757147+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:04:13.757147+0000","last_peered":"2024-09-10T16:04:13.757147+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:04:13.757147+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"36'3200","ondisk_log_start":"36'3200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32043974,"num_objects":7825,"num_object_clones":0,"num_object_copies":15650,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7825,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":9758,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3588},{"shards":"0,2","objects":1522}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'13437","reported_seq":13567,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:04:13.758911+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:04:13.758911+0000","last_peered":"2024-09-10T16:04:13.758911+0000","last_clean":"2024-09-10T16:04:13.758911+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:04:13.758911+0000","last_undegraded":"2024-09-10T16:04:13.758911+0000","last_fullsized":"2024-09-10T16:04:13.758911+0000","mapping_epoch":34,"log_start":"36'3400","ondisk_log_start":"36'3400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32264192,"num_objects":7877,"num_object_clones":0,"num_object_copies":15754,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7877,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10284,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104261+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:02:08.104261+0000","last_peered":"2024-09-10T16:02:08.104261+0000","last_clean":"2024-09-10T16:02:08.104261+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:02:08.104261+0000","last_undegraded":"2024-09-10T16:02:08.104261+0000","last_fullsized":"2024-09-10T16:02:08.104261+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'12950","reported_seq":13090,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:15.085238+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:04:15.085238+0000","last_peered":"2024-09-10T16:04:15.085238+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:04:15.085238+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"36'2900","ondisk_log_start":"36'2900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30749155,"num_objects":7508,"num_object_clones":0,"num_object_copies":15016,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7508,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10769,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1405},{"shards":"2","objects":3385}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103774+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:02:08.103774+0000","last_peered":"2024-09-10T16:02:08.103774+0000","last_clean":"2024-09-10T16:02:08.103774+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:02:08.103774+0000","last_undegraded":"2024-09-10T16:02:08.103774+0000","last_fullsized":"2024-09-10T16:02:08.103774+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'13224","reported_seq":13355,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:13.706761+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:04:13.706761+0000","last_peered":"2024-09-10T16:04:13.706761+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:04:13.706761+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"36'3200","ondisk_log_start":"36'3200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31481856,"num_objects":7686,"num_object_clones":0,"num_object_copies":15372,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7686,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9728,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3431},{"shards":"0,2","objects":1490}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":104,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.105055+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:02:08.105055+0000","last_peered":"2024-09-10T16:02:08.105055+0000","last_clean":"2024-09-10T16:02:08.105055+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:02:08.105055+0000","last_undegraded":"2024-09-10T16:02:08.105055+0000","last_fullsized":"2024-09-10T16:02:08.105055+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'13197","reported_seq":13332,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:15.121699+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:04:15.121699+0000","last_peered":"2024-09-10T16:04:15.121699+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:04:15.121699+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"36'3100","ondisk_log_start":"36'3100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31064547,"num_objects":7585,"num_object_clones":0,"num_object_copies":15170,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7585,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10877,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1500},{"shards":"2","objects":3281}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104617+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:02:08.104617+0000","last_peered":"2024-09-10T16:02:08.104617+0000","last_clean":"2024-09-10T16:02:08.104617+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:02:08.104617+0000","last_undegraded":"2024-09-10T16:02:08.104617+0000","last_fullsized":"2024-09-10T16:02:08.104617+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'13436","reported_seq":13565,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:15.071881+0000","last_change":"2024-09-10T15:59:16.174113+0000","last_active":"2024-09-10T16:04:15.071881+0000","last_peered":"2024-09-10T16:04:15.071881+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:04:15.071881+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"36'3400","ondisk_log_start":"36'3400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31678464,"num_objects":7734,"num_object_clones":0,"num_object_copies":15468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4354,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7734,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10990,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1504},{"shards":"2","objects":3382}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'13198","reported_seq":13333,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:13.847671+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:04:13.847671+0000","last_peered":"2024-09-10T16:04:13.847671+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:04:13.847671+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"36'3100","ondisk_log_start":"36'3100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31293440,"num_objects":7640,"num_object_clones":0,"num_object_copies":15280,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7640,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9829,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3336},{"shards":"0,2","objects":1526}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103316+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:02:08.103316+0000","last_peered":"2024-09-10T16:02:08.103316+0000","last_clean":"2024-09-10T16:02:08.103316+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:02:08.103316+0000","last_undegraded":"2024-09-10T16:02:08.103316+0000","last_fullsized":"2024-09-10T16:02:08.103316+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'13028","reported_seq":17894,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:13.823516+0000","last_change":"2024-09-10T16:03:29.605682+0000","last_active":"2024-09-10T16:04:13.823516+0000","last_peered":"2024-09-10T16:04:13.823516+0000","last_clean":"2024-09-10T15:59:14.115595+0000","last_became_active":"2024-09-10T15:59:16.244089+0000","last_became_peered":"2024-09-10T15:59:16.244089+0000","last_unstale":"2024-09-10T16:04:13.823516+0000","last_undegraded":"2024-09-10T16:03:29.603815+0000","last_fullsized":"2024-09-10T15:59:15.774437+0000","mapping_epoch":42,"log_start":"36'3000","ondisk_log_start":"36'3000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31006720,"num_objects":7570,"num_object_clones":0,"num_object_copies":15140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":380,"num_objects_misplaced":1465,"num_objects_unfound":0,"num_objects_dirty":7570,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9418,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2361,"num_bytes_recovered":9658368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5725},{"shards":"0,2","objects":1465}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103831+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:02:08.103831+0000","last_peered":"2024-09-10T16:02:08.103831+0000","last_clean":"2024-09-10T16:02:08.103831+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:02:08.103831+0000","last_undegraded":"2024-09-10T16:02:08.103831+0000","last_fullsized":"2024-09-10T16:02:08.103831+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'13394","reported_seq":13536,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:13.769537+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:04:13.769537+0000","last_peered":"2024-09-10T16:04:13.769537+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:04:13.769537+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"36'3300","ondisk_log_start":"36'3300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31350784,"num_objects":7654,"num_object_clones":0,"num_object_copies":15308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7654,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9649,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3283},{"shards":"0,2","objects":1503}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104463+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:02:08.104463+0000","last_peered":"2024-09-10T16:02:08.104463+0000","last_clean":"2024-09-10T16:02:08.104463+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:02:08.104463+0000","last_undegraded":"2024-09-10T16:02:08.104463+0000","last_fullsized":"2024-09-10T16:02:08.104463+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104448+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:02:08.104448+0000","last_peered":"2024-09-10T16:02:08.104448+0000","last_clean":"2024-09-10T16:02:08.104448+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:02:08.104448+0000","last_undegraded":"2024-09-10T16:02:08.104448+0000","last_fullsized":"2024-09-10T16:02:08.104448+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'13355","reported_seq":13498,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:13.758559+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:04:13.758559+0000","last_peered":"2024-09-10T16:04:13.758559+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:04:13.758559+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"36'3300","ondisk_log_start":"36'3300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32125894,"num_objects":7845,"num_object_clones":0,"num_object_copies":15690,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7845,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9518,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3522},{"shards":"0,2","objects":1573}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":502801142,"num_objects":122764,"num_object_clones":0,"num_object_copies":245528,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43291,"num_objects_misplaced":1465,"num_objects_unfound":0,"num_objects_dirty":122764,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":169342,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6633,"num_bytes_recovered":27152384,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1147633664,"data_stored":1147484613,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160968,"ondisk_log_size":160968,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986251,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":689208,"kb_used_data":382632,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":93682632,"statfs":{"total":96636764160,"available":95931015168,"internally_reserved":0,"allocated":391815168,"data_stored":391522935,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":313913002},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,3,1],"upper_bound":16},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659196,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":591104,"kb_used_data":302576,"kb_used_omap":4,"kb_used_meta":288507,"kb_avail":93780736,"statfs":{"total":96636764160,"available":96031473664,"internally_reserved":0,"allocated":309837824,"data_stored":309508997,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":295431854},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,2,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182479,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":839404,"kb_used_data":438492,"kb_used_omap":4,"kb_used_meta":400891,"kb_avail":93532436,"statfs":{"total":96636764160,"available":95777214464,"internally_reserved":0,"allocated":449015808,"data_stored":448723570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":410513066},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,0,3,1,3,8],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":447692800,"data_stored":447656670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":309395456,"data_stored":309322737,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":390545408,"data_stored":390505206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:04:16.767 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-10T16:04:16.767 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "45'13070", 'reported_seq': 13145, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:04:13.828013+0000', 'last_change': '2024-09-10T15:59:16.281558+0000', 'last_active': '2024-09-10T16:04:13.828013+0000', 'last_peered': '2024-09-10T16:04:13.828013+0000', 'last_clean': '2024-09-10T15:59:14.132456+0000', 'last_became_active': '2024-09-10T15:59:16.274407+0000', 'last_became_peered': '2024-09-10T15:59:16.274407+0000', 'last_unstale': '2024-09-10T16:04:13.828013+0000', 'last_undegraded': '2024-09-10T15:59:15.851279+0000', 'last_fullsized': '2024-09-10T15:59:15.774356+0000', 'mapping_epoch': 42, 'log_start': "36'3000", 'ondisk_log_start': "36'3000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'objects_scrubbed': 0, 'log_size': 10070, 'log_dups_size': 0, 'ondisk_log_size': 10070, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31137792, 'num_objects': 7602, 'num_object_clones': 0, 'num_object_copies': 15204, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4259, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7602, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13070, 'num_write_kb': 41344, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3343}, {'shards': '0,2', 'objects': 1524}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:04:16.767 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-10T16:04:16.767 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "45'13347", 'reported_seq': 13485, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:04:13.752635+0000', 'last_change': '2024-09-10T15:59:16.308264+0000', 'last_active': '2024-09-10T16:04:13.752635+0000', 'last_peered': '2024-09-10T16:04:13.752635+0000', 'last_clean': '2024-09-10T15:59:14.111551+0000', 'last_became_active': '2024-09-10T15:59:16.300040+0000', 'last_became_peered': '2024-09-10T15:59:16.300040+0000', 'last_unstale': '2024-09-10T16:04:13.752635+0000', 'last_undegraded': '2024-09-10T15:59:15.878234+0000', 'last_fullsized': '2024-09-10T15:59:15.769328+0000', 'mapping_epoch': 42, 'log_start': "36'3300", 'ondisk_log_start': "36'3300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3442", 'last_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'objects_scrubbed': 3384, 'log_size': 10047, 'log_dups_size': 0, 'ondisk_log_size': 10047, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1292, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31597027, 'num_objects': 7715, 'num_object_clones': 0, 'num_object_copies': 15430, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4372, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7715, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 9959, 'num_write_kb': 28580, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3344}, {'shards': '0,2', 'objects': 1557}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:04:16.767 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-10T16:04:16.767 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "45'13259", 'reported_seq': 13396, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:04:13.757147+0000', 'last_change': '2024-09-10T15:59:16.331899+0000', 'last_active': '2024-09-10T16:04:13.757147+0000', 'last_peered': '2024-09-10T16:04:13.757147+0000', 'last_clean': '2024-09-10T15:59:14.139436+0000', 'last_became_active': '2024-09-10T15:59:16.318715+0000', 'last_became_peered': '2024-09-10T15:59:16.318715+0000', 'last_unstale': '2024-09-10T16:04:13.757147+0000', 'last_undegraded': '2024-09-10T15:59:15.877524+0000', 'last_fullsized': '2024-09-10T15:59:15.769042+0000', 'mapping_epoch': 42, 'log_start': "36'3200", 'ondisk_log_start': "36'3200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3537", 'last_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'objects_scrubbed': 3497, 'log_size': 10059, 'log_dups_size': 0, 'ondisk_log_size': 10059, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1224, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32043974, 'num_objects': 7825, 'num_object_clones': 0, 'num_object_copies': 15650, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4237, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7825, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 9758, 'num_write_kb': 28172, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3588}, {'shards': '0,2', 'objects': 1522}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:04:16.767 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-10T16:04:16.768 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "45'12950", 'reported_seq': 13090, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:04:15.085238+0000', 'last_change': '2024-09-10T15:59:16.175680+0000', 'last_active': '2024-09-10T16:04:15.085238+0000', 'last_peered': '2024-09-10T16:04:15.085238+0000', 'last_clean': '2024-09-10T15:59:14.088187+0000', 'last_became_active': '2024-09-10T15:59:16.165810+0000', 'last_became_peered': '2024-09-10T15:59:16.165810+0000', 'last_unstale': '2024-09-10T16:04:15.085238+0000', 'last_undegraded': '2024-09-10T15:59:15.822098+0000', 'last_fullsized': '2024-09-10T15:59:15.769123+0000', 'mapping_epoch': 42, 'log_start': "36'2900", 'ondisk_log_start': "36'2900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2218", 'last_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'objects_scrubbed': 2175, 'log_size': 10050, 'log_dups_size': 0, 'ondisk_log_size': 10050, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 825, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30749155, 'num_objects': 7508, 'num_object_clones': 0, 'num_object_copies': 15016, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4124, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7508, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 10769, 'num_write_kb': 32204, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1405}, {'shards': '2', 'objects': 3385}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:04:16.768 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-10T16:04:16.768 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "45'13224", 'reported_seq': 13355, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:04:13.706761+0000', 'last_change': '2024-09-10T15:59:16.277742+0000', 'last_active': '2024-09-10T16:04:13.706761+0000', 'last_peered': '2024-09-10T16:04:13.706761+0000', 'last_clean': '2024-09-10T15:59:14.129081+0000', 'last_became_active': '2024-09-10T15:59:16.269835+0000', 'last_became_peered': '2024-09-10T15:59:16.269835+0000', 'last_unstale': '2024-09-10T16:04:13.706761+0000', 'last_undegraded': '2024-09-10T15:59:15.860857+0000', 'last_fullsized': '2024-09-10T15:59:15.771410+0000', 'mapping_epoch': 42, 'log_start': "36'3200", 'ondisk_log_start': "36'3200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3557", 'last_scrub_stamp': '2024-09-10T15:59:02.493678+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:02.493678+0000', 'objects_scrubbed': 3488, 'log_size': 10024, 'log_dups_size': 0, 'ondisk_log_size': 10024, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1472, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31481856, 'num_objects': 7686, 'num_object_clones': 0, 'num_object_copies': 15372, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4257, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7686, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9728, 'num_write_kb': 27852, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3431}, {'shards': '0,2', 'objects': 1490}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:04:16.768 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-10T16:04:16.768 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "45'13197", 'reported_seq': 13332, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:04:15.121699+0000', 'last_change': '2024-09-10T15:59:16.284311+0000', 'last_active': '2024-09-10T16:04:15.121699+0000', 'last_peered': '2024-09-10T16:04:15.121699+0000', 'last_clean': '2024-09-10T15:59:14.116041+0000', 'last_became_active': '2024-09-10T15:59:16.274543+0000', 'last_became_peered': '2024-09-10T15:59:16.274543+0000', 'last_unstale': '2024-09-10T16:04:15.121699+0000', 'last_undegraded': '2024-09-10T15:59:15.812484+0000', 'last_fullsized': '2024-09-10T15:59:15.770885+0000', 'mapping_epoch': 42, 'log_start': "36'3100", 'ondisk_log_start': "36'3100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2356", 'last_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'objects_scrubbed': 2316, 'log_size': 10097, 'log_dups_size': 0, 'ondisk_log_size': 10097, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 783, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31064547, 'num_objects': 7585, 'num_object_clones': 0, 'num_object_copies': 15170, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7585, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 10877, 'num_write_kb': 32292, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1500}, {'shards': '2', 'objects': 3281}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:04:16.768 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-10T16:04:16.768 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "45'13436", 'reported_seq': 13565, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:04:15.071881+0000', 'last_change': '2024-09-10T15:59:16.174113+0000', 'last_active': '2024-09-10T16:04:15.071881+0000', 'last_peered': '2024-09-10T16:04:15.071881+0000', 'last_clean': '2024-09-10T15:59:14.110654+0000', 'last_became_active': '2024-09-10T15:59:16.164645+0000', 'last_became_peered': '2024-09-10T15:59:16.164645+0000', 'last_unstale': '2024-09-10T16:04:15.071881+0000', 'last_undegraded': '2024-09-10T15:59:15.803593+0000', 'last_fullsized': '2024-09-10T15:59:15.769083+0000', 'mapping_epoch': 42, 'log_start': "36'3400", 'ondisk_log_start': "36'3400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2486", 'last_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'objects_scrubbed': 2440, 'log_size': 10036, 'log_dups_size': 0, 'ondisk_log_size': 10036, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 792, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31678464, 'num_objects': 7734, 'num_object_clones': 0, 'num_object_copies': 15468, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4354, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7734, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10990, 'num_write_kb': 32568, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1504}, {'shards': '2', 'objects': 3382}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:04:16.768 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-10T16:04:16.768 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "45'13198", 'reported_seq': 13333, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:04:13.847671+0000', 'last_change': '2024-09-10T15:59:16.315641+0000', 'last_active': '2024-09-10T16:04:13.847671+0000', 'last_peered': '2024-09-10T16:04:13.847671+0000', 'last_clean': '2024-09-10T15:59:14.117001+0000', 'last_became_active': '2024-09-10T15:59:16.302245+0000', 'last_became_peered': '2024-09-10T15:59:16.302245+0000', 'last_unstale': '2024-09-10T16:04:13.847671+0000', 'last_undegraded': '2024-09-10T15:59:15.845012+0000', 'last_fullsized': '2024-09-10T15:59:15.770337+0000', 'mapping_epoch': 42, 'log_start': "36'3100", 'ondisk_log_start': "36'3100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3404", 'last_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'objects_scrubbed': 3367, 'log_size': 10098, 'log_dups_size': 0, 'ondisk_log_size': 10098, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 983, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31293440, 'num_objects': 7640, 'num_object_clones': 0, 'num_object_copies': 15280, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7640, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9829, 'num_write_kb': 28204, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3336}, {'shards': '0,2', 'objects': 1526}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:04:16.768 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-10T16:04:16.768 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "45'13028", 'reported_seq': 17894, 'reported_epoch': 45, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:04:13.823516+0000', 'last_change': '2024-09-10T16:03:29.605682+0000', 'last_active': '2024-09-10T16:04:13.823516+0000', 'last_peered': '2024-09-10T16:04:13.823516+0000', 'last_clean': '2024-09-10T15:59:14.115595+0000', 'last_became_active': '2024-09-10T15:59:16.244089+0000', 'last_became_peered': '2024-09-10T15:59:16.244089+0000', 'last_unstale': '2024-09-10T16:04:13.823516+0000', 'last_undegraded': '2024-09-10T16:03:29.603815+0000', 'last_fullsized': '2024-09-10T15:59:15.774437+0000', 'mapping_epoch': 42, 'log_start': "36'3000", 'ondisk_log_start': "36'3000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3653", 'last_scrub_stamp': '2024-09-10T15:59:05.298468+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:05.298468+0000', 'objects_scrubbed': 3602, 'log_size': 10028, 'log_dups_size': 0, 'ondisk_log_size': 10028, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1289, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31006720, 'num_objects': 7570, 'num_object_clones': 0, 'num_object_copies': 15140, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 380, 'num_objects_misplaced': 1465, 'num_objects_unfound': 0, 'num_objects_dirty': 7570, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9418, 'num_write_kb': 26772, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2361, 'num_bytes_recovered': 9658368, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5725}, {'shards': '0,2', 'objects': 1465}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:04:16.769 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-10T16:04:16.769 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "45'13394", 'reported_seq': 13536, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:04:13.769537+0000', 'last_change': '2024-09-10T15:59:16.311291+0000', 'last_active': '2024-09-10T16:04:13.769537+0000', 'last_peered': '2024-09-10T16:04:13.769537+0000', 'last_clean': '2024-09-10T15:59:14.116363+0000', 'last_became_active': '2024-09-10T15:59:16.301244+0000', 'last_became_peered': '2024-09-10T15:59:16.301244+0000', 'last_unstale': '2024-09-10T16:04:13.769537+0000', 'last_undegraded': '2024-09-10T15:59:15.883434+0000', 'last_fullsized': '2024-09-10T15:59:15.772756+0000', 'mapping_epoch': 42, 'log_start': "36'3300", 'ondisk_log_start': "36'3300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3789", 'last_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'objects_scrubbed': 3741, 'log_size': 10094, 'log_dups_size': 0, 'ondisk_log_size': 10094, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1440, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31350784, 'num_objects': 7654, 'num_object_clones': 0, 'num_object_copies': 15308, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4373, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7654, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9649, 'num_write_kb': 27124, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3283}, {'shards': '0,2', 'objects': 1503}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:04:16.769 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-10T16:04:16.769 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "45'13355", 'reported_seq': 13498, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:04:13.758559+0000', 'last_change': '2024-09-10T15:59:16.313409+0000', 'last_active': '2024-09-10T16:04:13.758559+0000', 'last_peered': '2024-09-10T16:04:13.758559+0000', 'last_clean': '2024-09-10T15:59:14.129780+0000', 'last_became_active': '2024-09-10T15:59:16.300932+0000', 'last_became_peered': '2024-09-10T15:59:16.300932+0000', 'last_unstale': '2024-09-10T16:04:13.758559+0000', 'last_undegraded': '2024-09-10T15:59:15.887823+0000', 'last_fullsized': '2024-09-10T15:59:15.768513+0000', 'mapping_epoch': 42, 'log_start': "36'3300", 'ondisk_log_start': "36'3300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3880", 'last_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'objects_scrubbed': 3827, 'log_size': 10055, 'log_dups_size': 0, 'ondisk_log_size': 10055, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1397, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32125894, 'num_objects': 7845, 'num_object_clones': 0, 'num_object_copies': 15690, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4325, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7845, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9518, 'num_write_kb': 27072, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3522}, {'shards': '0,2', 'objects': 1573}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:04:16.769 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T16:04:17.434 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:04:17.434 INFO:teuthology.orchestra.run.smithi115.stdout:{"fsid":"7effee29-df05-4351-9741-ee203564ed81","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 43291/245532 objects degraded (17.632%), 11 pgs degraded, 11 pgs undersized","count":22},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":421,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983954,"num_in_osds":3,"osd_in_since":1725983947,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":122766,"data_bytes":502801161,"bytes_used":2170589184,"bytes_avail":287739703296,"bytes_total":289910292480,"degraded_objects":43291,"degraded_total":245532,"degraded_ratio":0.17631510353029337,"misplaced_objects":1465,"misplaced_total":245532,"misplaced_ratio":0.0059666357134711568,"recovering_objects_per_sec":26,"recovering_bytes_per_sec":108507,"recovering_keys_per_sec":0,"num_objects_recovered":321,"num_bytes_recovered":1302528,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":543},"fsmap":{"epoch":1,"btime":"2024-09-10T15:57:14:967120+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T16:00:13.560776+0000","services":{}},"progress_events":{}} 2024-09-10T16:04:17.453 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-10T16:04:17.453 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:04:18.032 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:04:18.033 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:04:18.056 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":249,"stamp":"2024-09-10T16:04:17.654583+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502227721,"num_objects":122626,"num_object_clones":0,"num_object_copies":245252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43291,"num_objects_misplaced":1465,"num_objects_unfound":0,"num_objects_dirty":122626,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":169484,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6635,"num_bytes_recovered":27152403,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161011,"ondisk_log_size":161011,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2127508,"kb_used_data":1123172,"kb_used_omap":12,"kb_used_meta":1004275,"kb_avail":280988012,"statfs":{"total":289910292480,"available":287731724288,"internally_reserved":0,"allocated":1150128128,"data_stored":1149214830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1028377602},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,2,5,3,5,3,9,8],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18722816,"num_objects":-4571,"num_object_clones":0,"num_object_copies":-9142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":87,"num_objects_misplaced":-308,"num_objects_unfound":0,"num_objects_dirty":-4571,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4571,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":221,"num_bytes_recovered":905216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004296"},"pg_stats":[{"pgid":"2.f","version":"45'13070","reported_seq":13145,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:13.828013+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:04:13.828013+0000","last_peered":"2024-09-10T16:04:13.828013+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:04:13.828013+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"36'3000","ondisk_log_start":"36'3000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31137792,"num_objects":7602,"num_object_clones":0,"num_object_copies":15204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7602,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13070,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3343},{"shards":"0,2","objects":1524}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'13295","reported_seq":13408,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:04:15.133833+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:04:15.133833+0000","last_peered":"2024-09-10T16:04:15.133833+0000","last_clean":"2024-09-10T16:04:15.133833+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:04:15.133833+0000","last_undegraded":"2024-09-10T16:04:15.133833+0000","last_fullsized":"2024-09-10T16:04:15.133833+0000","mapping_epoch":35,"log_start":"36'3200","ondisk_log_start":"36'3200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30921670,"num_objects":7551,"num_object_clones":0,"num_object_copies":15102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7551,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":11400,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'13200","reported_seq":21878,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:04:16.479697+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:04:16.479697+0000","last_peered":"2024-09-10T16:04:16.479697+0000","last_clean":"2024-09-10T16:04:16.479697+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:04:16.479697+0000","last_undegraded":"2024-09-10T16:04:16.479697+0000","last_fullsized":"2024-09-10T16:04:16.479697+0000","mapping_epoch":44,"log_start":"36'3100","ondisk_log_start":"36'3100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31117336,"num_objects":7598,"num_object_clones":0,"num_object_copies":15196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7598,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11204,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'13090","reported_seq":13171,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:04:15.020217+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:04:15.020217+0000","last_peered":"2024-09-10T16:04:15.020217+0000","last_clean":"2024-09-10T16:04:15.020217+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:04:15.020217+0000","last_undegraded":"2024-09-10T16:04:15.020217+0000","last_fullsized":"2024-09-10T16:04:15.020217+0000","mapping_epoch":35,"log_start":"36'3000","ondisk_log_start":"36'3000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31006720,"num_objects":7570,"num_object_clones":0,"num_object_copies":15140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7570,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13090,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'13228","reported_seq":13358,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:04:13.709729+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:04:13.709729+0000","last_peered":"2024-09-10T16:04:13.709729+0000","last_clean":"2024-09-10T16:04:13.709729+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:04:13.709729+0000","last_undegraded":"2024-09-10T16:04:13.709729+0000","last_fullsized":"2024-09-10T16:04:13.709729+0000","mapping_epoch":34,"log_start":"36'3200","ondisk_log_start":"36'3200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31388131,"num_objects":7664,"num_object_clones":0,"num_object_copies":15328,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7664,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9939,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'13347","reported_seq":13485,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:13.752635+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:04:13.752635+0000","last_peered":"2024-09-10T16:04:13.752635+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:04:13.752635+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"36'3300","ondisk_log_start":"36'3300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31597027,"num_objects":7715,"num_object_clones":0,"num_object_copies":15430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7715,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9959,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3344},{"shards":"0,2","objects":1557}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'13259","reported_seq":13396,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:13.757147+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:04:13.757147+0000","last_peered":"2024-09-10T16:04:13.757147+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:04:13.757147+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"36'3200","ondisk_log_start":"36'3200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32043974,"num_objects":7825,"num_object_clones":0,"num_object_copies":15650,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7825,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":9758,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3588},{"shards":"0,2","objects":1522}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'13437","reported_seq":13567,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:04:13.758911+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:04:13.758911+0000","last_peered":"2024-09-10T16:04:13.758911+0000","last_clean":"2024-09-10T16:04:13.758911+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:04:13.758911+0000","last_undegraded":"2024-09-10T16:04:13.758911+0000","last_fullsized":"2024-09-10T16:04:13.758911+0000","mapping_epoch":34,"log_start":"36'3400","ondisk_log_start":"36'3400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32264192,"num_objects":7877,"num_object_clones":0,"num_object_copies":15754,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7877,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10284,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104261+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:02:08.104261+0000","last_peered":"2024-09-10T16:02:08.104261+0000","last_clean":"2024-09-10T16:02:08.104261+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:02:08.104261+0000","last_undegraded":"2024-09-10T16:02:08.104261+0000","last_fullsized":"2024-09-10T16:02:08.104261+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'12950","reported_seq":13090,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:15.085238+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:04:15.085238+0000","last_peered":"2024-09-10T16:04:15.085238+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:04:15.085238+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"36'2900","ondisk_log_start":"36'2900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30749155,"num_objects":7508,"num_object_clones":0,"num_object_copies":15016,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7508,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10769,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1405},{"shards":"2","objects":3385}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103774+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:02:08.103774+0000","last_peered":"2024-09-10T16:02:08.103774+0000","last_clean":"2024-09-10T16:02:08.103774+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:02:08.103774+0000","last_undegraded":"2024-09-10T16:02:08.103774+0000","last_fullsized":"2024-09-10T16:02:08.103774+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'13224","reported_seq":13355,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:13.706761+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:04:13.706761+0000","last_peered":"2024-09-10T16:04:13.706761+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:04:13.706761+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"36'3200","ondisk_log_start":"36'3200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31481856,"num_objects":7686,"num_object_clones":0,"num_object_copies":15372,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7686,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9728,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3431},{"shards":"0,2","objects":1490}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":104,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.105055+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:02:08.105055+0000","last_peered":"2024-09-10T16:02:08.105055+0000","last_clean":"2024-09-10T16:02:08.105055+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:02:08.105055+0000","last_undegraded":"2024-09-10T16:02:08.105055+0000","last_fullsized":"2024-09-10T16:02:08.105055+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'13197","reported_seq":13332,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:15.121699+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:04:15.121699+0000","last_peered":"2024-09-10T16:04:15.121699+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:04:15.121699+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"36'3100","ondisk_log_start":"36'3100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31064547,"num_objects":7585,"num_object_clones":0,"num_object_copies":15170,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7585,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10877,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1500},{"shards":"2","objects":3281}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104617+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:02:08.104617+0000","last_peered":"2024-09-10T16:02:08.104617+0000","last_clean":"2024-09-10T16:02:08.104617+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:02:08.104617+0000","last_undegraded":"2024-09-10T16:02:08.104617+0000","last_fullsized":"2024-09-10T16:02:08.104617+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'13436","reported_seq":13565,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:15.071881+0000","last_change":"2024-09-10T15:59:16.174113+0000","last_active":"2024-09-10T16:04:15.071881+0000","last_peered":"2024-09-10T16:04:15.071881+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:04:15.071881+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"36'3400","ondisk_log_start":"36'3400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31678464,"num_objects":7734,"num_object_clones":0,"num_object_copies":15468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4354,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7734,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10990,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1504},{"shards":"2","objects":3382}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'13198","reported_seq":13333,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:13.847671+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:04:13.847671+0000","last_peered":"2024-09-10T16:04:13.847671+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:04:13.847671+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"36'3100","ondisk_log_start":"36'3100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31293440,"num_objects":7640,"num_object_clones":0,"num_object_copies":15280,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7640,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9829,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3336},{"shards":"0,2","objects":1526}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103316+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:02:08.103316+0000","last_peered":"2024-09-10T16:02:08.103316+0000","last_clean":"2024-09-10T16:02:08.103316+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:02:08.103316+0000","last_undegraded":"2024-09-10T16:02:08.103316+0000","last_fullsized":"2024-09-10T16:02:08.103316+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'13028","reported_seq":17894,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:13.823516+0000","last_change":"2024-09-10T16:03:29.605682+0000","last_active":"2024-09-10T16:04:13.823516+0000","last_peered":"2024-09-10T16:04:13.823516+0000","last_clean":"2024-09-10T15:59:14.115595+0000","last_became_active":"2024-09-10T15:59:16.244089+0000","last_became_peered":"2024-09-10T15:59:16.244089+0000","last_unstale":"2024-09-10T16:04:13.823516+0000","last_undegraded":"2024-09-10T16:03:29.603815+0000","last_fullsized":"2024-09-10T15:59:15.774437+0000","mapping_epoch":42,"log_start":"36'3000","ondisk_log_start":"36'3000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31006720,"num_objects":7570,"num_object_clones":0,"num_object_copies":15140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":380,"num_objects_misplaced":1465,"num_objects_unfound":0,"num_objects_dirty":7570,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9418,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2361,"num_bytes_recovered":9658368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5725},{"shards":"0,2","objects":1465}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103831+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:02:08.103831+0000","last_peered":"2024-09-10T16:02:08.103831+0000","last_clean":"2024-09-10T16:02:08.103831+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:02:08.103831+0000","last_undegraded":"2024-09-10T16:02:08.103831+0000","last_fullsized":"2024-09-10T16:02:08.103831+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'13394","reported_seq":13536,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:13.769537+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:04:13.769537+0000","last_peered":"2024-09-10T16:04:13.769537+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:04:13.769537+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"36'3300","ondisk_log_start":"36'3300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31350784,"num_objects":7654,"num_object_clones":0,"num_object_copies":15308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7654,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9649,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3283},{"shards":"0,2","objects":1503}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104463+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:02:08.104463+0000","last_peered":"2024-09-10T16:02:08.104463+0000","last_clean":"2024-09-10T16:02:08.104463+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:02:08.104463+0000","last_undegraded":"2024-09-10T16:02:08.104463+0000","last_fullsized":"2024-09-10T16:02:08.104463+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104448+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:02:08.104448+0000","last_peered":"2024-09-10T16:02:08.104448+0000","last_clean":"2024-09-10T16:02:08.104448+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:02:08.104448+0000","last_undegraded":"2024-09-10T16:02:08.104448+0000","last_fullsized":"2024-09-10T16:02:08.104448+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'13355","reported_seq":13498,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:13.758559+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:04:13.758559+0000","last_peered":"2024-09-10T16:04:13.758559+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:04:13.758559+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"36'3300","ondisk_log_start":"36'3300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32125894,"num_objects":7845,"num_object_clones":0,"num_object_copies":15690,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7845,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9518,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3522},{"shards":"0,2","objects":1573}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":502227702,"num_objects":122624,"num_object_clones":0,"num_object_copies":245248,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43291,"num_objects_misplaced":1465,"num_objects_unfound":0,"num_objects_dirty":122624,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":169482,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6633,"num_bytes_recovered":27152384,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1147101184,"data_stored":1146952133,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161008,"ondisk_log_size":161008,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986251,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":689208,"kb_used_data":382632,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":93682632,"statfs":{"total":96636764160,"available":95931015168,"internally_reserved":0,"allocated":391815168,"data_stored":391522935,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":313913002},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,3,1],"upper_bound":16},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659197,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":598896,"kb_used_data":302048,"kb_used_omap":4,"kb_used_meta":296827,"kb_avail":93772944,"statfs":{"total":96636764160,"available":96023494656,"internally_reserved":0,"allocated":309297152,"data_stored":308968325,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":303951534},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,2,2,2,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182479,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":839404,"kb_used_data":438492,"kb_used_omap":4,"kb_used_meta":400891,"kb_avail":93532436,"statfs":{"total":96636764160,"available":95777214464,"internally_reserved":0,"allocated":449015808,"data_stored":448723570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":410513066},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,0,3,1,3,8],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":447692800,"data_stored":447656670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":308862976,"data_stored":308790257,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":390545408,"data_stored":390505206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:04:21.059 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:04:21.613 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:04:21.616 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:04:21.636 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":250,"stamp":"2024-09-10T16:04:19.655196+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":496276233,"num_objects":121173,"num_object_clones":0,"num_object_copies":242346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43305,"num_objects_misplaced":1330,"num_objects_unfound":0,"num_objects_dirty":121173,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":170937,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6756,"num_bytes_recovered":27648019,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161064,"ondisk_log_size":161064,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2108188,"kb_used_data":1116332,"kb_used_omap":12,"kb_used_meta":991795,"kb_avail":281007332,"statfs":{"total":289910292480,"available":287751507968,"internally_reserved":0,"allocated":1143123968,"data_stored":1142210670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1015598082},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,3,8,5,6,9,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-24092672,"num_objects":-5882,"num_object_clones":0,"num_object_copies":-11764,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":101,"num_objects_misplaced":-443,"num_objects_unfound":0,"num_objects_dirty":-5882,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5882,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":1400832,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004415"},"pg_stats":[{"pgid":"2.f","version":"45'13215","reported_seq":13290,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:18.869200+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:04:18.869200+0000","last_peered":"2024-09-10T16:04:18.869200+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:04:18.869200+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"36'3200","ondisk_log_start":"36'3200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30543872,"num_objects":7457,"num_object_clones":0,"num_object_copies":14914,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7457,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13215,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3198},{"shards":"0,2","objects":1379}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'13295","reported_seq":13408,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:04:15.133833+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:04:15.133833+0000","last_peered":"2024-09-10T16:04:15.133833+0000","last_clean":"2024-09-10T16:04:15.133833+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:04:15.133833+0000","last_undegraded":"2024-09-10T16:04:15.133833+0000","last_fullsized":"2024-09-10T16:04:15.133833+0000","mapping_epoch":35,"log_start":"36'3200","ondisk_log_start":"36'3200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30921670,"num_objects":7551,"num_object_clones":0,"num_object_copies":15102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7551,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":11400,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'13200","reported_seq":21878,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:04:16.479697+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:04:16.479697+0000","last_peered":"2024-09-10T16:04:16.479697+0000","last_clean":"2024-09-10T16:04:16.479697+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:04:16.479697+0000","last_undegraded":"2024-09-10T16:04:16.479697+0000","last_fullsized":"2024-09-10T16:04:16.479697+0000","mapping_epoch":44,"log_start":"36'3100","ondisk_log_start":"36'3100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31117336,"num_objects":7598,"num_object_clones":0,"num_object_copies":15196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7598,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11204,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'13090","reported_seq":13171,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:04:15.020217+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:04:15.020217+0000","last_peered":"2024-09-10T16:04:15.020217+0000","last_clean":"2024-09-10T16:04:15.020217+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:04:15.020217+0000","last_undegraded":"2024-09-10T16:04:15.020217+0000","last_fullsized":"2024-09-10T16:04:15.020217+0000","mapping_epoch":35,"log_start":"36'3000","ondisk_log_start":"36'3000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31006720,"num_objects":7570,"num_object_clones":0,"num_object_copies":15140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7570,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13090,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'13394","reported_seq":13524,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:04:18.812862+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:04:18.812862+0000","last_peered":"2024-09-10T16:04:18.812862+0000","last_clean":"2024-09-10T16:04:18.812862+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:04:18.812862+0000","last_undegraded":"2024-09-10T16:04:18.812862+0000","last_fullsized":"2024-09-10T16:04:18.812862+0000","mapping_epoch":34,"log_start":"36'3300","ondisk_log_start":"36'3300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30708195,"num_objects":7498,"num_object_clones":0,"num_object_copies":14996,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7498,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10105,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'13486","reported_seq":13624,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:18.885699+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:04:18.885699+0000","last_peered":"2024-09-10T16:04:18.885699+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:04:18.885699+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"36'3400","ondisk_log_start":"36'3400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31027683,"num_objects":7576,"num_object_clones":0,"num_object_copies":15152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7576,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10098,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3205},{"shards":"0,2","objects":1418}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'13402","reported_seq":13540,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:18.853258+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:04:18.853258+0000","last_peered":"2024-09-10T16:04:18.853258+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:04:18.853258+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"36'3400","ondisk_log_start":"36'3400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31458246,"num_objects":7682,"num_object_clones":0,"num_object_copies":15364,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7682,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":9901,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3445},{"shards":"0,2","objects":1379}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'13585","reported_seq":13715,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:04:18.835431+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:04:18.835431+0000","last_peered":"2024-09-10T16:04:18.835431+0000","last_clean":"2024-09-10T16:04:18.835431+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:04:18.835431+0000","last_undegraded":"2024-09-10T16:04:18.835431+0000","last_fullsized":"2024-09-10T16:04:18.835431+0000","mapping_epoch":34,"log_start":"36'3500","ondisk_log_start":"36'3500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31657984,"num_objects":7729,"num_object_clones":0,"num_object_copies":15458,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7729,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10432,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104261+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:02:08.104261+0000","last_peered":"2024-09-10T16:02:08.104261+0000","last_clean":"2024-09-10T16:02:08.104261+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:02:08.104261+0000","last_undegraded":"2024-09-10T16:02:08.104261+0000","last_fullsized":"2024-09-10T16:02:08.104261+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'12950","reported_seq":13090,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:15.085238+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:04:15.085238+0000","last_peered":"2024-09-10T16:04:15.085238+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:04:15.085238+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"36'2900","ondisk_log_start":"36'2900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30749155,"num_objects":7508,"num_object_clones":0,"num_object_copies":15016,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7508,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10769,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1405},{"shards":"2","objects":3385}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103774+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:02:08.103774+0000","last_peered":"2024-09-10T16:02:08.103774+0000","last_clean":"2024-09-10T16:02:08.103774+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:02:08.103774+0000","last_undegraded":"2024-09-10T16:02:08.103774+0000","last_fullsized":"2024-09-10T16:02:08.103774+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'13381","reported_seq":13512,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:18.884198+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:04:18.884198+0000","last_peered":"2024-09-10T16:04:18.884198+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:04:18.884198+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"36'3300","ondisk_log_start":"36'3300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30838784,"num_objects":7529,"num_object_clones":0,"num_object_copies":15058,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7529,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9885,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3274},{"shards":"0,2","objects":1333}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":105,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:12.405385+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:02:12.405385+0000","last_peered":"2024-09-10T16:02:12.405385+0000","last_clean":"2024-09-10T16:02:12.405385+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:02:12.405385+0000","last_undegraded":"2024-09-10T16:02:12.405385+0000","last_fullsized":"2024-09-10T16:02:12.405385+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'13197","reported_seq":13332,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:15.121699+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:04:15.121699+0000","last_peered":"2024-09-10T16:04:15.121699+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:04:15.121699+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"36'3100","ondisk_log_start":"36'3100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31064547,"num_objects":7585,"num_object_clones":0,"num_object_copies":15170,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7585,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10877,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1500},{"shards":"2","objects":3281}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104617+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:02:08.104617+0000","last_peered":"2024-09-10T16:02:08.104617+0000","last_clean":"2024-09-10T16:02:08.104617+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:02:08.104617+0000","last_undegraded":"2024-09-10T16:02:08.104617+0000","last_fullsized":"2024-09-10T16:02:08.104617+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'13436","reported_seq":13565,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:15.071881+0000","last_change":"2024-09-10T15:59:16.174113+0000","last_active":"2024-09-10T16:04:15.071881+0000","last_peered":"2024-09-10T16:04:15.071881+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:04:15.071881+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"36'3400","ondisk_log_start":"36'3400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31678464,"num_objects":7734,"num_object_clones":0,"num_object_copies":15468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4354,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7734,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10990,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1504},{"shards":"2","objects":3382}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'13340","reported_seq":13474,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:18.855380+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:04:18.855380+0000","last_peered":"2024-09-10T16:04:18.855380+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:04:18.855380+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"36'3300","ondisk_log_start":"36'3300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30711808,"num_objects":7498,"num_object_clones":0,"num_object_copies":14996,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7498,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9971,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3194},{"shards":"0,2","objects":1384}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103316+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:02:08.103316+0000","last_peered":"2024-09-10T16:02:08.103316+0000","last_clean":"2024-09-10T16:02:08.103316+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:02:08.103316+0000","last_undegraded":"2024-09-10T16:02:08.103316+0000","last_fullsized":"2024-09-10T16:02:08.103316+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'13162","reported_seq":18268,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:18.889422+0000","last_change":"2024-09-10T16:03:29.605682+0000","last_active":"2024-09-10T16:04:18.889422+0000","last_peered":"2024-09-10T16:04:18.889422+0000","last_clean":"2024-09-10T15:59:14.115595+0000","last_became_active":"2024-09-10T15:59:16.244089+0000","last_became_peered":"2024-09-10T15:59:16.244089+0000","last_unstale":"2024-09-10T16:04:18.889422+0000","last_undegraded":"2024-09-10T16:03:29.603815+0000","last_fullsized":"2024-09-10T15:59:15.774437+0000","mapping_epoch":42,"log_start":"36'3100","ondisk_log_start":"36'3100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30457856,"num_objects":7436,"num_object_clones":0,"num_object_copies":14872,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":394,"num_objects_misplaced":1330,"num_objects_unfound":0,"num_objects_dirty":7436,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9552,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2482,"num_bytes_recovered":10153984,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5712},{"shards":"0,2","objects":1330}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103831+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:02:08.103831+0000","last_peered":"2024-09-10T16:02:08.103831+0000","last_clean":"2024-09-10T16:02:08.103831+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:02:08.103831+0000","last_undegraded":"2024-09-10T16:02:08.103831+0000","last_fullsized":"2024-09-10T16:02:08.103831+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'13533","reported_seq":13675,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:18.777183+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:04:18.777183+0000","last_peered":"2024-09-10T16:04:18.777183+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:04:18.777183+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"36'3500","ondisk_log_start":"36'3500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30781440,"num_objects":7515,"num_object_clones":0,"num_object_copies":15030,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7515,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9788,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3144},{"shards":"0,2","objects":1364}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104463+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:02:08.104463+0000","last_peered":"2024-09-10T16:02:08.104463+0000","last_clean":"2024-09-10T16:02:08.104463+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:02:08.104463+0000","last_undegraded":"2024-09-10T16:02:08.104463+0000","last_fullsized":"2024-09-10T16:02:08.104463+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104448+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:02:08.104448+0000","last_peered":"2024-09-10T16:02:08.104448+0000","last_clean":"2024-09-10T16:02:08.104448+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:02:08.104448+0000","last_undegraded":"2024-09-10T16:02:08.104448+0000","last_fullsized":"2024-09-10T16:02:08.104448+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'13495","reported_seq":13640,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:18.701203+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:04:18.701203+0000","last_peered":"2024-09-10T16:04:18.701203+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:04:18.701203+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"36'3400","ondisk_log_start":"36'3400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31552454,"num_objects":7705,"num_object_clones":0,"num_object_copies":15410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7705,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9658,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3382},{"shards":"0,2","objects":1433}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":496276214,"num_objects":121171,"num_object_clones":0,"num_object_copies":242342,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43305,"num_objects_misplaced":1330,"num_objects_unfound":0,"num_objects_dirty":121171,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":170935,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6754,"num_bytes_recovered":27648000,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1139990528,"data_stored":1139841477,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161061,"ondisk_log_size":161061,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986251,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":689208,"kb_used_data":382632,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":93682632,"statfs":{"total":96636764160,"available":95931015168,"internally_reserved":0,"allocated":391815168,"data_stored":391522935,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":313913002},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,3,1],"upper_bound":16},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659197,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":598896,"kb_used_data":302048,"kb_used_omap":4,"kb_used_meta":296827,"kb_avail":93772944,"statfs":{"total":96636764160,"available":96023494656,"internally_reserved":0,"allocated":309297152,"data_stored":308968325,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":303951534},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,2,2,2,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182480,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":820084,"kb_used_data":431652,"kb_used_omap":4,"kb_used_meta":388411,"kb_avail":93551756,"statfs":{"total":96636764160,"available":95796998144,"internally_reserved":0,"allocated":442011648,"data_stored":441719410,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":397733546},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,5,3,4,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":440582144,"data_stored":440546014,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":308862976,"data_stored":308790257,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":390545408,"data_stored":390505206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:04:21.638 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:04:22.192 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:04:22.193 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:04:22.217 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":251,"stamp":"2024-09-10T16:04:21.655851+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":493482761,"num_objects":120491,"num_object_clones":0,"num_object_copies":240982,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43305,"num_objects_misplaced":1330,"num_objects_unfound":0,"num_objects_dirty":120491,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":171619,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6756,"num_bytes_recovered":27648019,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160946,"ondisk_log_size":160946,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2107820,"kb_used_data":1111804,"kb_used_omap":12,"kb_used_meta":995955,"kb_avail":281007700,"statfs":{"total":289910292480,"available":287751884800,"internally_reserved":0,"allocated":1138487296,"data_stored":1137573998,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1019857922},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,7,8,9,10,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-21061632,"num_objects":-5142,"num_object_clones":0,"num_object_copies":-10284,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":57,"num_objects_misplaced":-291,"num_objects_unfound":0,"num_objects_dirty":-5142,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5142,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":234,"num_bytes_recovered":958464,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004353"},"pg_stats":[{"pgid":"2.f","version":"45'13215","reported_seq":13290,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:18.869200+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:04:18.869200+0000","last_peered":"2024-09-10T16:04:18.869200+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:04:18.869200+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"36'3200","ondisk_log_start":"36'3200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30543872,"num_objects":7457,"num_object_clones":0,"num_object_copies":14914,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7457,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13215,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3198},{"shards":"0,2","objects":1379}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'13436","reported_seq":13549,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:04:20.123136+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:04:20.123136+0000","last_peered":"2024-09-10T16:04:20.123136+0000","last_clean":"2024-09-10T16:04:20.123136+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:04:20.123136+0000","last_undegraded":"2024-09-10T16:04:20.123136+0000","last_fullsized":"2024-09-10T16:04:20.123136+0000","mapping_epoch":35,"log_start":"36'3400","ondisk_log_start":"36'3400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30344134,"num_objects":7410,"num_object_clones":0,"num_object_copies":14820,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7410,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":11541,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'13200","reported_seq":21878,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:04:16.479697+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:04:16.479697+0000","last_peered":"2024-09-10T16:04:16.479697+0000","last_clean":"2024-09-10T16:04:16.479697+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:04:16.479697+0000","last_undegraded":"2024-09-10T16:04:16.479697+0000","last_fullsized":"2024-09-10T16:04:16.479697+0000","mapping_epoch":44,"log_start":"36'3100","ondisk_log_start":"36'3100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31117336,"num_objects":7598,"num_object_clones":0,"num_object_copies":15196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7598,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11204,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'13232","reported_seq":13313,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:04:20.149779+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:04:20.149779+0000","last_peered":"2024-09-10T16:04:20.149779+0000","last_clean":"2024-09-10T16:04:20.149779+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:04:20.149779+0000","last_undegraded":"2024-09-10T16:04:20.149779+0000","last_fullsized":"2024-09-10T16:04:20.149779+0000","mapping_epoch":35,"log_start":"36'3200","ondisk_log_start":"36'3200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30425088,"num_objects":7428,"num_object_clones":0,"num_object_copies":14856,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7428,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13232,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'13394","reported_seq":13524,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:04:18.812862+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:04:18.812862+0000","last_peered":"2024-09-10T16:04:18.812862+0000","last_clean":"2024-09-10T16:04:18.812862+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:04:18.812862+0000","last_undegraded":"2024-09-10T16:04:18.812862+0000","last_fullsized":"2024-09-10T16:04:18.812862+0000","mapping_epoch":34,"log_start":"36'3300","ondisk_log_start":"36'3300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30708195,"num_objects":7498,"num_object_clones":0,"num_object_copies":14996,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7498,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10105,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'13486","reported_seq":13624,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:18.885699+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:04:18.885699+0000","last_peered":"2024-09-10T16:04:18.885699+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:04:18.885699+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"36'3400","ondisk_log_start":"36'3400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31027683,"num_objects":7576,"num_object_clones":0,"num_object_copies":15152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7576,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10098,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3205},{"shards":"0,2","objects":1418}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'13402","reported_seq":13540,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:18.853258+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:04:18.853258+0000","last_peered":"2024-09-10T16:04:18.853258+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:04:18.853258+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"36'3400","ondisk_log_start":"36'3400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31458246,"num_objects":7682,"num_object_clones":0,"num_object_copies":15364,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7682,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":9901,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3445},{"shards":"0,2","objects":1379}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'13585","reported_seq":13715,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:04:18.835431+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:04:18.835431+0000","last_peered":"2024-09-10T16:04:18.835431+0000","last_clean":"2024-09-10T16:04:18.835431+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:04:18.835431+0000","last_undegraded":"2024-09-10T16:04:18.835431+0000","last_fullsized":"2024-09-10T16:04:18.835431+0000","mapping_epoch":34,"log_start":"36'3500","ondisk_log_start":"36'3500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31657984,"num_objects":7729,"num_object_clones":0,"num_object_copies":15458,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7729,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10432,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104261+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:02:08.104261+0000","last_peered":"2024-09-10T16:02:08.104261+0000","last_clean":"2024-09-10T16:02:08.104261+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:02:08.104261+0000","last_undegraded":"2024-09-10T16:02:08.104261+0000","last_fullsized":"2024-09-10T16:02:08.104261+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'13065","reported_seq":13205,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:19.971498+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:04:19.971498+0000","last_peered":"2024-09-10T16:04:19.971498+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:04:19.971498+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"36'3000","ondisk_log_start":"36'3000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30278115,"num_objects":7393,"num_object_clones":0,"num_object_copies":14786,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7393,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10884,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1290},{"shards":"2","objects":3270}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103774+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:02:08.103774+0000","last_peered":"2024-09-10T16:02:08.103774+0000","last_clean":"2024-09-10T16:02:08.103774+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:02:08.103774+0000","last_undegraded":"2024-09-10T16:02:08.103774+0000","last_fullsized":"2024-09-10T16:02:08.103774+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'13381","reported_seq":13512,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:18.884198+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:04:18.884198+0000","last_peered":"2024-09-10T16:04:18.884198+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:04:18.884198+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"36'3300","ondisk_log_start":"36'3300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30838784,"num_objects":7529,"num_object_clones":0,"num_object_copies":15058,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7529,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9885,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3274},{"shards":"0,2","objects":1333}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":105,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:12.405385+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:02:12.405385+0000","last_peered":"2024-09-10T16:02:12.405385+0000","last_clean":"2024-09-10T16:02:12.405385+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:02:12.405385+0000","last_undegraded":"2024-09-10T16:02:12.405385+0000","last_fullsized":"2024-09-10T16:02:12.405385+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'13340","reported_seq":13474,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:20.127052+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:04:20.127052+0000","last_peered":"2024-09-10T16:04:20.127052+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:04:20.127052+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"36'3300","ondisk_log_start":"36'3300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30478819,"num_objects":7442,"num_object_clones":0,"num_object_copies":14884,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7442,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11020,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1357},{"shards":"2","objects":3138}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104617+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:02:08.104617+0000","last_peered":"2024-09-10T16:02:08.104617+0000","last_clean":"2024-09-10T16:02:08.104617+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:02:08.104617+0000","last_undegraded":"2024-09-10T16:02:08.104617+0000","last_fullsized":"2024-09-10T16:02:08.104617+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'13577","reported_seq":13705,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:20.148493+0000","last_change":"2024-09-10T15:59:16.174113+0000","last_active":"2024-09-10T16:04:20.148493+0000","last_peered":"2024-09-10T16:04:20.148493+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:04:20.148493+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"36'3500","ondisk_log_start":"36'3500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31100928,"num_objects":7593,"num_object_clones":0,"num_object_copies":15186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4354,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7593,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11131,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1363},{"shards":"2","objects":3241}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'13340","reported_seq":13474,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:18.855380+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:04:18.855380+0000","last_peered":"2024-09-10T16:04:18.855380+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:04:18.855380+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"36'3300","ondisk_log_start":"36'3300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30711808,"num_objects":7498,"num_object_clones":0,"num_object_copies":14996,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7498,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9971,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3194},{"shards":"0,2","objects":1384}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103316+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:02:08.103316+0000","last_peered":"2024-09-10T16:02:08.103316+0000","last_clean":"2024-09-10T16:02:08.103316+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:02:08.103316+0000","last_undegraded":"2024-09-10T16:02:08.103316+0000","last_fullsized":"2024-09-10T16:02:08.103316+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'13162","reported_seq":18268,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:18.889422+0000","last_change":"2024-09-10T16:03:29.605682+0000","last_active":"2024-09-10T16:04:18.889422+0000","last_peered":"2024-09-10T16:04:18.889422+0000","last_clean":"2024-09-10T15:59:14.115595+0000","last_became_active":"2024-09-10T15:59:16.244089+0000","last_became_peered":"2024-09-10T15:59:16.244089+0000","last_unstale":"2024-09-10T16:04:18.889422+0000","last_undegraded":"2024-09-10T16:03:29.603815+0000","last_fullsized":"2024-09-10T15:59:15.774437+0000","mapping_epoch":42,"log_start":"36'3100","ondisk_log_start":"36'3100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30457856,"num_objects":7436,"num_object_clones":0,"num_object_copies":14872,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":394,"num_objects_misplaced":1330,"num_objects_unfound":0,"num_objects_dirty":7436,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9552,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2482,"num_bytes_recovered":10153984,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5712},{"shards":"0,2","objects":1330}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103831+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:02:08.103831+0000","last_peered":"2024-09-10T16:02:08.103831+0000","last_clean":"2024-09-10T16:02:08.103831+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:02:08.103831+0000","last_undegraded":"2024-09-10T16:02:08.103831+0000","last_fullsized":"2024-09-10T16:02:08.103831+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'13533","reported_seq":13675,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:18.777183+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:04:18.777183+0000","last_peered":"2024-09-10T16:04:18.777183+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:04:18.777183+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"36'3500","ondisk_log_start":"36'3500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30781440,"num_objects":7515,"num_object_clones":0,"num_object_copies":15030,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7515,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9788,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3144},{"shards":"0,2","objects":1364}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104463+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:02:08.104463+0000","last_peered":"2024-09-10T16:02:08.104463+0000","last_clean":"2024-09-10T16:02:08.104463+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:02:08.104463+0000","last_undegraded":"2024-09-10T16:02:08.104463+0000","last_fullsized":"2024-09-10T16:02:08.104463+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104448+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:02:08.104448+0000","last_peered":"2024-09-10T16:02:08.104448+0000","last_clean":"2024-09-10T16:02:08.104448+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:02:08.104448+0000","last_undegraded":"2024-09-10T16:02:08.104448+0000","last_fullsized":"2024-09-10T16:02:08.104448+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'13495","reported_seq":13640,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:18.701203+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:04:18.701203+0000","last_peered":"2024-09-10T16:04:18.701203+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:04:18.701203+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"36'3400","ondisk_log_start":"36'3400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31552454,"num_objects":7705,"num_object_clones":0,"num_object_copies":15410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7705,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9658,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3382},{"shards":"0,2","objects":1433}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":493482742,"num_objects":120489,"num_object_clones":0,"num_object_copies":240978,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43305,"num_objects_misplaced":1330,"num_objects_unfound":0,"num_objects_dirty":120489,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":171617,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6754,"num_bytes_recovered":27648000,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1135394816,"data_stored":1135245765,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160943,"ondisk_log_size":160943,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986252,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":688840,"kb_used_data":378104,"kb_used_omap":4,"kb_used_meta":310715,"kb_avail":93683000,"statfs":{"total":96636764160,"available":95931392000,"internally_reserved":0,"allocated":387178496,"data_stored":386886263,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":318172842},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,3,3,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659197,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":598896,"kb_used_data":302048,"kb_used_omap":4,"kb_used_meta":296827,"kb_avail":93772944,"statfs":{"total":96636764160,"available":96023494656,"internally_reserved":0,"allocated":309297152,"data_stored":308968325,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":303951534},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,2,2,2,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182480,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":820084,"kb_used_data":431652,"kb_used_omap":4,"kb_used_meta":388411,"kb_avail":93551756,"statfs":{"total":96636764160,"available":95796998144,"internally_reserved":0,"allocated":442011648,"data_stored":441719410,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":397733546},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,5,3,4,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":440582144,"data_stored":440546014,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":308862976,"data_stored":308790257,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":385949696,"data_stored":385909494,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:04:22.219 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-10T16:04:22.219 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "45'13215", 'reported_seq': 13290, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:04:18.869200+0000', 'last_change': '2024-09-10T15:59:16.281558+0000', 'last_active': '2024-09-10T16:04:18.869200+0000', 'last_peered': '2024-09-10T16:04:18.869200+0000', 'last_clean': '2024-09-10T15:59:14.132456+0000', 'last_became_active': '2024-09-10T15:59:16.274407+0000', 'last_became_peered': '2024-09-10T15:59:16.274407+0000', 'last_unstale': '2024-09-10T16:04:18.869200+0000', 'last_undegraded': '2024-09-10T15:59:15.851279+0000', 'last_fullsized': '2024-09-10T15:59:15.774356+0000', 'mapping_epoch': 42, 'log_start': "36'3200", 'ondisk_log_start': "36'3200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'objects_scrubbed': 0, 'log_size': 10015, 'log_dups_size': 0, 'ondisk_log_size': 10015, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30543872, 'num_objects': 7457, 'num_object_clones': 0, 'num_object_copies': 14914, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4259, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7457, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13215, 'num_write_kb': 41344, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3198}, {'shards': '0,2', 'objects': 1379}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:04:22.219 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-10T16:04:22.219 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "45'13486", 'reported_seq': 13624, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:04:18.885699+0000', 'last_change': '2024-09-10T15:59:16.308264+0000', 'last_active': '2024-09-10T16:04:18.885699+0000', 'last_peered': '2024-09-10T16:04:18.885699+0000', 'last_clean': '2024-09-10T15:59:14.111551+0000', 'last_became_active': '2024-09-10T15:59:16.300040+0000', 'last_became_peered': '2024-09-10T15:59:16.300040+0000', 'last_unstale': '2024-09-10T16:04:18.885699+0000', 'last_undegraded': '2024-09-10T15:59:15.878234+0000', 'last_fullsized': '2024-09-10T15:59:15.769328+0000', 'mapping_epoch': 42, 'log_start': "36'3400", 'ondisk_log_start': "36'3400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3442", 'last_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'objects_scrubbed': 3384, 'log_size': 10086, 'log_dups_size': 0, 'ondisk_log_size': 10086, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1292, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31027683, 'num_objects': 7576, 'num_object_clones': 0, 'num_object_copies': 15152, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4372, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7576, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 10098, 'num_write_kb': 28580, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3205}, {'shards': '0,2', 'objects': 1418}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:04:22.219 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-10T16:04:22.220 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "45'13402", 'reported_seq': 13540, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:04:18.853258+0000', 'last_change': '2024-09-10T15:59:16.331899+0000', 'last_active': '2024-09-10T16:04:18.853258+0000', 'last_peered': '2024-09-10T16:04:18.853258+0000', 'last_clean': '2024-09-10T15:59:14.139436+0000', 'last_became_active': '2024-09-10T15:59:16.318715+0000', 'last_became_peered': '2024-09-10T15:59:16.318715+0000', 'last_unstale': '2024-09-10T16:04:18.853258+0000', 'last_undegraded': '2024-09-10T15:59:15.877524+0000', 'last_fullsized': '2024-09-10T15:59:15.769042+0000', 'mapping_epoch': 42, 'log_start': "36'3400", 'ondisk_log_start': "36'3400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3537", 'last_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'objects_scrubbed': 3497, 'log_size': 10002, 'log_dups_size': 0, 'ondisk_log_size': 10002, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1224, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31458246, 'num_objects': 7682, 'num_object_clones': 0, 'num_object_copies': 15364, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4237, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7682, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 9901, 'num_write_kb': 28172, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3445}, {'shards': '0,2', 'objects': 1379}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:04:22.220 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-10T16:04:22.220 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "45'13065", 'reported_seq': 13205, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:04:19.971498+0000', 'last_change': '2024-09-10T15:59:16.175680+0000', 'last_active': '2024-09-10T16:04:19.971498+0000', 'last_peered': '2024-09-10T16:04:19.971498+0000', 'last_clean': '2024-09-10T15:59:14.088187+0000', 'last_became_active': '2024-09-10T15:59:16.165810+0000', 'last_became_peered': '2024-09-10T15:59:16.165810+0000', 'last_unstale': '2024-09-10T16:04:19.971498+0000', 'last_undegraded': '2024-09-10T15:59:15.822098+0000', 'last_fullsized': '2024-09-10T15:59:15.769123+0000', 'mapping_epoch': 42, 'log_start': "36'3000", 'ondisk_log_start': "36'3000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2218", 'last_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'objects_scrubbed': 2175, 'log_size': 10065, 'log_dups_size': 0, 'ondisk_log_size': 10065, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 825, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30278115, 'num_objects': 7393, 'num_object_clones': 0, 'num_object_copies': 14786, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4124, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7393, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 10884, 'num_write_kb': 32204, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1290}, {'shards': '2', 'objects': 3270}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:04:22.220 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-10T16:04:22.220 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "45'13381", 'reported_seq': 13512, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:04:18.884198+0000', 'last_change': '2024-09-10T15:59:16.277742+0000', 'last_active': '2024-09-10T16:04:18.884198+0000', 'last_peered': '2024-09-10T16:04:18.884198+0000', 'last_clean': '2024-09-10T15:59:14.129081+0000', 'last_became_active': '2024-09-10T15:59:16.269835+0000', 'last_became_peered': '2024-09-10T15:59:16.269835+0000', 'last_unstale': '2024-09-10T16:04:18.884198+0000', 'last_undegraded': '2024-09-10T15:59:15.860857+0000', 'last_fullsized': '2024-09-10T15:59:15.771410+0000', 'mapping_epoch': 42, 'log_start': "36'3300", 'ondisk_log_start': "36'3300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3557", 'last_scrub_stamp': '2024-09-10T15:59:02.493678+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:02.493678+0000', 'objects_scrubbed': 3488, 'log_size': 10081, 'log_dups_size': 0, 'ondisk_log_size': 10081, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1472, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30838784, 'num_objects': 7529, 'num_object_clones': 0, 'num_object_copies': 15058, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4257, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7529, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9885, 'num_write_kb': 27852, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3274}, {'shards': '0,2', 'objects': 1333}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:04:22.220 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-10T16:04:22.220 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "45'13340", 'reported_seq': 13474, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:04:20.127052+0000', 'last_change': '2024-09-10T15:59:16.284311+0000', 'last_active': '2024-09-10T16:04:20.127052+0000', 'last_peered': '2024-09-10T16:04:20.127052+0000', 'last_clean': '2024-09-10T15:59:14.116041+0000', 'last_became_active': '2024-09-10T15:59:16.274543+0000', 'last_became_peered': '2024-09-10T15:59:16.274543+0000', 'last_unstale': '2024-09-10T16:04:20.127052+0000', 'last_undegraded': '2024-09-10T15:59:15.812484+0000', 'last_fullsized': '2024-09-10T15:59:15.770885+0000', 'mapping_epoch': 42, 'log_start': "36'3300", 'ondisk_log_start': "36'3300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2356", 'last_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'objects_scrubbed': 2316, 'log_size': 10040, 'log_dups_size': 0, 'ondisk_log_size': 10040, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 783, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30478819, 'num_objects': 7442, 'num_object_clones': 0, 'num_object_copies': 14884, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7442, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 11020, 'num_write_kb': 32292, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1357}, {'shards': '2', 'objects': 3138}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:04:22.220 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-10T16:04:22.220 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "45'13577", 'reported_seq': 13705, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:04:20.148493+0000', 'last_change': '2024-09-10T15:59:16.174113+0000', 'last_active': '2024-09-10T16:04:20.148493+0000', 'last_peered': '2024-09-10T16:04:20.148493+0000', 'last_clean': '2024-09-10T15:59:14.110654+0000', 'last_became_active': '2024-09-10T15:59:16.164645+0000', 'last_became_peered': '2024-09-10T15:59:16.164645+0000', 'last_unstale': '2024-09-10T16:04:20.148493+0000', 'last_undegraded': '2024-09-10T15:59:15.803593+0000', 'last_fullsized': '2024-09-10T15:59:15.769083+0000', 'mapping_epoch': 42, 'log_start': "36'3500", 'ondisk_log_start': "36'3500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2486", 'last_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'objects_scrubbed': 2440, 'log_size': 10077, 'log_dups_size': 0, 'ondisk_log_size': 10077, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 792, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31100928, 'num_objects': 7593, 'num_object_clones': 0, 'num_object_copies': 15186, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4354, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7593, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11131, 'num_write_kb': 32568, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1363}, {'shards': '2', 'objects': 3241}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:04:22.220 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-10T16:04:22.221 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "45'13340", 'reported_seq': 13474, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:04:18.855380+0000', 'last_change': '2024-09-10T15:59:16.315641+0000', 'last_active': '2024-09-10T16:04:18.855380+0000', 'last_peered': '2024-09-10T16:04:18.855380+0000', 'last_clean': '2024-09-10T15:59:14.117001+0000', 'last_became_active': '2024-09-10T15:59:16.302245+0000', 'last_became_peered': '2024-09-10T15:59:16.302245+0000', 'last_unstale': '2024-09-10T16:04:18.855380+0000', 'last_undegraded': '2024-09-10T15:59:15.845012+0000', 'last_fullsized': '2024-09-10T15:59:15.770337+0000', 'mapping_epoch': 42, 'log_start': "36'3300", 'ondisk_log_start': "36'3300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3404", 'last_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'objects_scrubbed': 3367, 'log_size': 10040, 'log_dups_size': 0, 'ondisk_log_size': 10040, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 983, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30711808, 'num_objects': 7498, 'num_object_clones': 0, 'num_object_copies': 14996, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7498, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9971, 'num_write_kb': 28204, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3194}, {'shards': '0,2', 'objects': 1384}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:04:22.221 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-10T16:04:22.221 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "45'13162", 'reported_seq': 18268, 'reported_epoch': 45, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:04:18.889422+0000', 'last_change': '2024-09-10T16:03:29.605682+0000', 'last_active': '2024-09-10T16:04:18.889422+0000', 'last_peered': '2024-09-10T16:04:18.889422+0000', 'last_clean': '2024-09-10T15:59:14.115595+0000', 'last_became_active': '2024-09-10T15:59:16.244089+0000', 'last_became_peered': '2024-09-10T15:59:16.244089+0000', 'last_unstale': '2024-09-10T16:04:18.889422+0000', 'last_undegraded': '2024-09-10T16:03:29.603815+0000', 'last_fullsized': '2024-09-10T15:59:15.774437+0000', 'mapping_epoch': 42, 'log_start': "36'3100", 'ondisk_log_start': "36'3100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3653", 'last_scrub_stamp': '2024-09-10T15:59:05.298468+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:05.298468+0000', 'objects_scrubbed': 3602, 'log_size': 10062, 'log_dups_size': 0, 'ondisk_log_size': 10062, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1289, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30457856, 'num_objects': 7436, 'num_object_clones': 0, 'num_object_copies': 14872, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 394, 'num_objects_misplaced': 1330, 'num_objects_unfound': 0, 'num_objects_dirty': 7436, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9552, 'num_write_kb': 26772, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2482, 'num_bytes_recovered': 10153984, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5712}, {'shards': '0,2', 'objects': 1330}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:04:22.221 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-10T16:04:22.221 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "45'13533", 'reported_seq': 13675, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:04:18.777183+0000', 'last_change': '2024-09-10T15:59:16.311291+0000', 'last_active': '2024-09-10T16:04:18.777183+0000', 'last_peered': '2024-09-10T16:04:18.777183+0000', 'last_clean': '2024-09-10T15:59:14.116363+0000', 'last_became_active': '2024-09-10T15:59:16.301244+0000', 'last_became_peered': '2024-09-10T15:59:16.301244+0000', 'last_unstale': '2024-09-10T16:04:18.777183+0000', 'last_undegraded': '2024-09-10T15:59:15.883434+0000', 'last_fullsized': '2024-09-10T15:59:15.772756+0000', 'mapping_epoch': 42, 'log_start': "36'3500", 'ondisk_log_start': "36'3500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3789", 'last_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'objects_scrubbed': 3741, 'log_size': 10033, 'log_dups_size': 0, 'ondisk_log_size': 10033, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1440, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30781440, 'num_objects': 7515, 'num_object_clones': 0, 'num_object_copies': 15030, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4373, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7515, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9788, 'num_write_kb': 27124, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3144}, {'shards': '0,2', 'objects': 1364}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:04:22.221 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-10T16:04:22.221 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "45'13495", 'reported_seq': 13640, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:04:18.701203+0000', 'last_change': '2024-09-10T15:59:16.313409+0000', 'last_active': '2024-09-10T16:04:18.701203+0000', 'last_peered': '2024-09-10T16:04:18.701203+0000', 'last_clean': '2024-09-10T15:59:14.129780+0000', 'last_became_active': '2024-09-10T15:59:16.300932+0000', 'last_became_peered': '2024-09-10T15:59:16.300932+0000', 'last_unstale': '2024-09-10T16:04:18.701203+0000', 'last_undegraded': '2024-09-10T15:59:15.887823+0000', 'last_fullsized': '2024-09-10T15:59:15.768513+0000', 'mapping_epoch': 42, 'log_start': "36'3400", 'ondisk_log_start': "36'3400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3880", 'last_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'objects_scrubbed': 3827, 'log_size': 10095, 'log_dups_size': 0, 'ondisk_log_size': 10095, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1397, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31552454, 'num_objects': 7705, 'num_object_clones': 0, 'num_object_copies': 15410, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4325, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7705, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9658, 'num_write_kb': 27072, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3382}, {'shards': '0,2', 'objects': 1433}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:04:22.221 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T16:04:22.956 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:04:22.957 INFO:teuthology.orchestra.run.smithi115.stdout:{"fsid":"7effee29-df05-4351-9741-ee203564ed81","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 43305/240982 objects degraded (17.970%), 11 pgs degraded, 11 pgs undersized","count":22},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":426,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983954,"num_in_osds":3,"osd_in_since":1725983947,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":120491,"data_bytes":493482761,"bytes_used":2158407680,"bytes_avail":287751884800,"bytes_total":289910292480,"degraded_objects":43305,"degraded_total":240982,"degraded_ratio":0.17970221842295275,"misplaced_objects":1330,"misplaced_total":240982,"misplaced_ratio":0.0055190844129437058,"recovering_objects_per_sec":19,"recovering_bytes_per_sec":79843,"recovering_keys_per_sec":0,"num_objects_recovered":234,"num_bytes_recovered":958464,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":428},"fsmap":{"epoch":1,"btime":"2024-09-10T15:57:14:967120+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T16:00:13.560776+0000","services":{}},"progress_events":{}} 2024-09-10T16:04:22.981 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-10T16:04:22.982 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:04:23.517 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:04:23.520 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:04:23.543 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":251,"stamp":"2024-09-10T16:04:21.655851+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":493482761,"num_objects":120491,"num_object_clones":0,"num_object_copies":240982,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43305,"num_objects_misplaced":1330,"num_objects_unfound":0,"num_objects_dirty":120491,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":171619,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6756,"num_bytes_recovered":27648019,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160946,"ondisk_log_size":160946,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2107820,"kb_used_data":1111804,"kb_used_omap":12,"kb_used_meta":995955,"kb_avail":281007700,"statfs":{"total":289910292480,"available":287751884800,"internally_reserved":0,"allocated":1138487296,"data_stored":1137573998,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1019857922},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,7,8,9,10,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-21061632,"num_objects":-5142,"num_object_clones":0,"num_object_copies":-10284,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":57,"num_objects_misplaced":-291,"num_objects_unfound":0,"num_objects_dirty":-5142,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5142,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":234,"num_bytes_recovered":958464,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004353"},"pg_stats":[{"pgid":"2.f","version":"45'13215","reported_seq":13290,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:18.869200+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:04:18.869200+0000","last_peered":"2024-09-10T16:04:18.869200+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:04:18.869200+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"36'3200","ondisk_log_start":"36'3200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30543872,"num_objects":7457,"num_object_clones":0,"num_object_copies":14914,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7457,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13215,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3198},{"shards":"0,2","objects":1379}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'13436","reported_seq":13549,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:04:20.123136+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:04:20.123136+0000","last_peered":"2024-09-10T16:04:20.123136+0000","last_clean":"2024-09-10T16:04:20.123136+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:04:20.123136+0000","last_undegraded":"2024-09-10T16:04:20.123136+0000","last_fullsized":"2024-09-10T16:04:20.123136+0000","mapping_epoch":35,"log_start":"36'3400","ondisk_log_start":"36'3400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30344134,"num_objects":7410,"num_object_clones":0,"num_object_copies":14820,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7410,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":11541,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'13200","reported_seq":21878,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:04:16.479697+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:04:16.479697+0000","last_peered":"2024-09-10T16:04:16.479697+0000","last_clean":"2024-09-10T16:04:16.479697+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:04:16.479697+0000","last_undegraded":"2024-09-10T16:04:16.479697+0000","last_fullsized":"2024-09-10T16:04:16.479697+0000","mapping_epoch":44,"log_start":"36'3100","ondisk_log_start":"36'3100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31117336,"num_objects":7598,"num_object_clones":0,"num_object_copies":15196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7598,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11204,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'13232","reported_seq":13313,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:04:20.149779+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:04:20.149779+0000","last_peered":"2024-09-10T16:04:20.149779+0000","last_clean":"2024-09-10T16:04:20.149779+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:04:20.149779+0000","last_undegraded":"2024-09-10T16:04:20.149779+0000","last_fullsized":"2024-09-10T16:04:20.149779+0000","mapping_epoch":35,"log_start":"36'3200","ondisk_log_start":"36'3200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30425088,"num_objects":7428,"num_object_clones":0,"num_object_copies":14856,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7428,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13232,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'13394","reported_seq":13524,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:04:18.812862+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:04:18.812862+0000","last_peered":"2024-09-10T16:04:18.812862+0000","last_clean":"2024-09-10T16:04:18.812862+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:04:18.812862+0000","last_undegraded":"2024-09-10T16:04:18.812862+0000","last_fullsized":"2024-09-10T16:04:18.812862+0000","mapping_epoch":34,"log_start":"36'3300","ondisk_log_start":"36'3300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30708195,"num_objects":7498,"num_object_clones":0,"num_object_copies":14996,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7498,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10105,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'13486","reported_seq":13624,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:18.885699+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:04:18.885699+0000","last_peered":"2024-09-10T16:04:18.885699+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:04:18.885699+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"36'3400","ondisk_log_start":"36'3400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31027683,"num_objects":7576,"num_object_clones":0,"num_object_copies":15152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7576,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10098,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3205},{"shards":"0,2","objects":1418}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'13402","reported_seq":13540,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:18.853258+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:04:18.853258+0000","last_peered":"2024-09-10T16:04:18.853258+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:04:18.853258+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"36'3400","ondisk_log_start":"36'3400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31458246,"num_objects":7682,"num_object_clones":0,"num_object_copies":15364,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7682,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":9901,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3445},{"shards":"0,2","objects":1379}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'13585","reported_seq":13715,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:04:18.835431+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:04:18.835431+0000","last_peered":"2024-09-10T16:04:18.835431+0000","last_clean":"2024-09-10T16:04:18.835431+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:04:18.835431+0000","last_undegraded":"2024-09-10T16:04:18.835431+0000","last_fullsized":"2024-09-10T16:04:18.835431+0000","mapping_epoch":34,"log_start":"36'3500","ondisk_log_start":"36'3500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31657984,"num_objects":7729,"num_object_clones":0,"num_object_copies":15458,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7729,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10432,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104261+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:02:08.104261+0000","last_peered":"2024-09-10T16:02:08.104261+0000","last_clean":"2024-09-10T16:02:08.104261+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:02:08.104261+0000","last_undegraded":"2024-09-10T16:02:08.104261+0000","last_fullsized":"2024-09-10T16:02:08.104261+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'13065","reported_seq":13205,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:19.971498+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:04:19.971498+0000","last_peered":"2024-09-10T16:04:19.971498+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:04:19.971498+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"36'3000","ondisk_log_start":"36'3000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30278115,"num_objects":7393,"num_object_clones":0,"num_object_copies":14786,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7393,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10884,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1290},{"shards":"2","objects":3270}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103774+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:02:08.103774+0000","last_peered":"2024-09-10T16:02:08.103774+0000","last_clean":"2024-09-10T16:02:08.103774+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:02:08.103774+0000","last_undegraded":"2024-09-10T16:02:08.103774+0000","last_fullsized":"2024-09-10T16:02:08.103774+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'13381","reported_seq":13512,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:18.884198+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:04:18.884198+0000","last_peered":"2024-09-10T16:04:18.884198+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:04:18.884198+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"36'3300","ondisk_log_start":"36'3300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30838784,"num_objects":7529,"num_object_clones":0,"num_object_copies":15058,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7529,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9885,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3274},{"shards":"0,2","objects":1333}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":105,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:12.405385+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:02:12.405385+0000","last_peered":"2024-09-10T16:02:12.405385+0000","last_clean":"2024-09-10T16:02:12.405385+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:02:12.405385+0000","last_undegraded":"2024-09-10T16:02:12.405385+0000","last_fullsized":"2024-09-10T16:02:12.405385+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'13340","reported_seq":13474,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:20.127052+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:04:20.127052+0000","last_peered":"2024-09-10T16:04:20.127052+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:04:20.127052+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"36'3300","ondisk_log_start":"36'3300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30478819,"num_objects":7442,"num_object_clones":0,"num_object_copies":14884,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7442,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11020,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1357},{"shards":"2","objects":3138}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104617+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:02:08.104617+0000","last_peered":"2024-09-10T16:02:08.104617+0000","last_clean":"2024-09-10T16:02:08.104617+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:02:08.104617+0000","last_undegraded":"2024-09-10T16:02:08.104617+0000","last_fullsized":"2024-09-10T16:02:08.104617+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'13577","reported_seq":13705,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:20.148493+0000","last_change":"2024-09-10T15:59:16.174113+0000","last_active":"2024-09-10T16:04:20.148493+0000","last_peered":"2024-09-10T16:04:20.148493+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:04:20.148493+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"36'3500","ondisk_log_start":"36'3500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31100928,"num_objects":7593,"num_object_clones":0,"num_object_copies":15186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4354,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7593,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11131,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1363},{"shards":"2","objects":3241}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'13340","reported_seq":13474,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:18.855380+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:04:18.855380+0000","last_peered":"2024-09-10T16:04:18.855380+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:04:18.855380+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"36'3300","ondisk_log_start":"36'3300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30711808,"num_objects":7498,"num_object_clones":0,"num_object_copies":14996,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7498,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9971,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3194},{"shards":"0,2","objects":1384}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103316+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:02:08.103316+0000","last_peered":"2024-09-10T16:02:08.103316+0000","last_clean":"2024-09-10T16:02:08.103316+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:02:08.103316+0000","last_undegraded":"2024-09-10T16:02:08.103316+0000","last_fullsized":"2024-09-10T16:02:08.103316+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'13162","reported_seq":18268,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:18.889422+0000","last_change":"2024-09-10T16:03:29.605682+0000","last_active":"2024-09-10T16:04:18.889422+0000","last_peered":"2024-09-10T16:04:18.889422+0000","last_clean":"2024-09-10T15:59:14.115595+0000","last_became_active":"2024-09-10T15:59:16.244089+0000","last_became_peered":"2024-09-10T15:59:16.244089+0000","last_unstale":"2024-09-10T16:04:18.889422+0000","last_undegraded":"2024-09-10T16:03:29.603815+0000","last_fullsized":"2024-09-10T15:59:15.774437+0000","mapping_epoch":42,"log_start":"36'3100","ondisk_log_start":"36'3100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30457856,"num_objects":7436,"num_object_clones":0,"num_object_copies":14872,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":394,"num_objects_misplaced":1330,"num_objects_unfound":0,"num_objects_dirty":7436,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9552,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2482,"num_bytes_recovered":10153984,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5712},{"shards":"0,2","objects":1330}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103831+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:02:08.103831+0000","last_peered":"2024-09-10T16:02:08.103831+0000","last_clean":"2024-09-10T16:02:08.103831+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:02:08.103831+0000","last_undegraded":"2024-09-10T16:02:08.103831+0000","last_fullsized":"2024-09-10T16:02:08.103831+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'13533","reported_seq":13675,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:18.777183+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:04:18.777183+0000","last_peered":"2024-09-10T16:04:18.777183+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:04:18.777183+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"36'3500","ondisk_log_start":"36'3500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30781440,"num_objects":7515,"num_object_clones":0,"num_object_copies":15030,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7515,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9788,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3144},{"shards":"0,2","objects":1364}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104463+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:02:08.104463+0000","last_peered":"2024-09-10T16:02:08.104463+0000","last_clean":"2024-09-10T16:02:08.104463+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:02:08.104463+0000","last_undegraded":"2024-09-10T16:02:08.104463+0000","last_fullsized":"2024-09-10T16:02:08.104463+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104448+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:02:08.104448+0000","last_peered":"2024-09-10T16:02:08.104448+0000","last_clean":"2024-09-10T16:02:08.104448+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:02:08.104448+0000","last_undegraded":"2024-09-10T16:02:08.104448+0000","last_fullsized":"2024-09-10T16:02:08.104448+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'13495","reported_seq":13640,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:18.701203+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:04:18.701203+0000","last_peered":"2024-09-10T16:04:18.701203+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:04:18.701203+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"36'3400","ondisk_log_start":"36'3400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31552454,"num_objects":7705,"num_object_clones":0,"num_object_copies":15410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7705,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9658,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3382},{"shards":"0,2","objects":1433}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":493482742,"num_objects":120489,"num_object_clones":0,"num_object_copies":240978,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43305,"num_objects_misplaced":1330,"num_objects_unfound":0,"num_objects_dirty":120489,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":171617,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6754,"num_bytes_recovered":27648000,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1135394816,"data_stored":1135245765,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160943,"ondisk_log_size":160943,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986252,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":688840,"kb_used_data":378104,"kb_used_omap":4,"kb_used_meta":310715,"kb_avail":93683000,"statfs":{"total":96636764160,"available":95931392000,"internally_reserved":0,"allocated":387178496,"data_stored":386886263,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":318172842},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,3,3,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659197,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":598896,"kb_used_data":302048,"kb_used_omap":4,"kb_used_meta":296827,"kb_avail":93772944,"statfs":{"total":96636764160,"available":96023494656,"internally_reserved":0,"allocated":309297152,"data_stored":308968325,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":303951534},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,2,2,2,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182480,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":820084,"kb_used_data":431652,"kb_used_omap":4,"kb_used_meta":388411,"kb_avail":93551756,"statfs":{"total":96636764160,"available":95796998144,"internally_reserved":0,"allocated":442011648,"data_stored":441719410,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":397733546},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,5,3,4,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":440582144,"data_stored":440546014,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":308862976,"data_stored":308790257,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":385949696,"data_stored":385909494,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:04:26.545 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:04:27.100 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:04:27.101 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:04:27.133 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":253,"stamp":"2024-09-10T16:04:25.657387+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":484451081,"num_objects":118286,"num_object_clones":0,"num_object_copies":236572,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43305,"num_objects_misplaced":1203,"num_objects_unfound":0,"num_objects_dirty":118286,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":173824,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6883,"num_bytes_recovered":28168211,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160951,"ondisk_log_size":160951,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2075124,"kb_used_data":1099904,"kb_used_omap":12,"kb_used_meta":975155,"kb_avail":281040396,"statfs":{"total":289910292480,"available":287785365504,"internally_reserved":0,"allocated":1126301696,"data_stored":1125388398,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":998558722},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,2,2,3,4,13,12,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-26714112,"num_objects":-6522,"num_object_clones":0,"num_object_copies":-13044,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":57,"num_objects_misplaced":-418,"num_objects_unfound":0,"num_objects_dirty":-6522,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6522,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":361,"num_bytes_recovered":1478656,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004339"},"pg_stats":[{"pgid":"2.f","version":"45'13346","reported_seq":13421,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:23.903896+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:04:23.903896+0000","last_peered":"2024-09-10T16:04:23.903896+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:04:23.903896+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"36'3300","ondisk_log_start":"36'3300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30007296,"num_objects":7326,"num_object_clones":0,"num_object_copies":14652,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7326,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13346,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3067},{"shards":"0,2","objects":1248}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'13580","reported_seq":13693,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:04:25.184577+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:04:25.184577+0000","last_peered":"2024-09-10T16:04:25.184577+0000","last_clean":"2024-09-10T16:04:25.184577+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:04:25.184577+0000","last_undegraded":"2024-09-10T16:04:25.184577+0000","last_fullsized":"2024-09-10T16:04:25.184577+0000","mapping_epoch":35,"log_start":"36'3500","ondisk_log_start":"36'3500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29754310,"num_objects":7266,"num_object_clones":0,"num_object_copies":14532,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7266,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":11685,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'13362","reported_seq":22040,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:04:21.629240+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:04:21.629240+0000","last_peered":"2024-09-10T16:04:21.629240+0000","last_clean":"2024-09-10T16:04:21.629240+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:04:21.629240+0000","last_undegraded":"2024-09-10T16:04:21.629240+0000","last_fullsized":"2024-09-10T16:04:21.629240+0000","mapping_epoch":44,"log_start":"36'3300","ondisk_log_start":"36'3300","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30453784,"num_objects":7436,"num_object_clones":0,"num_object_copies":14872,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7436,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11366,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'13370","reported_seq":13451,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:04:25.184806+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:04:25.184806+0000","last_peered":"2024-09-10T16:04:25.184806+0000","last_clean":"2024-09-10T16:04:25.184806+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:04:25.184806+0000","last_undegraded":"2024-09-10T16:04:25.184806+0000","last_fullsized":"2024-09-10T16:04:25.184806+0000","mapping_epoch":35,"log_start":"36'3300","ondisk_log_start":"36'3300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29859840,"num_objects":7290,"num_object_clones":0,"num_object_copies":14580,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7290,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13370,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'13553","reported_seq":13682,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:04:23.958828+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:04:23.958828+0000","last_peered":"2024-09-10T16:04:23.958828+0000","last_clean":"2024-09-10T16:04:23.958828+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:04:23.958828+0000","last_undegraded":"2024-09-10T16:04:23.958828+0000","last_fullsized":"2024-09-10T16:04:23.958828+0000","mapping_epoch":34,"log_start":"36'3500","ondisk_log_start":"36'3500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30056931,"num_objects":7339,"num_object_clones":0,"num_object_copies":14678,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7339,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10264,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'13628","reported_seq":13765,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:23.941323+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:04:23.941323+0000","last_peered":"2024-09-10T16:04:23.941323+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:04:23.941323+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"36'3600","ondisk_log_start":"36'3600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30446051,"num_objects":7434,"num_object_clones":0,"num_object_copies":14868,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7434,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10240,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3063},{"shards":"0,2","objects":1276}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'13549","reported_seq":13686,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:23.913936+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:04:23.913936+0000","last_peered":"2024-09-10T16:04:23.913936+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:04:23.913936+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"36'3500","ondisk_log_start":"36'3500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30856134,"num_objects":7535,"num_object_clones":0,"num_object_copies":15070,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7535,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":10048,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3298},{"shards":"0,2","objects":1232}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'13719","reported_seq":13849,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:04:23.876162+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:04:23.876162+0000","last_peered":"2024-09-10T16:04:23.876162+0000","last_clean":"2024-09-10T16:04:23.876162+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:04:23.876162+0000","last_undegraded":"2024-09-10T16:04:23.876162+0000","last_fullsized":"2024-09-10T16:04:23.876162+0000","mapping_epoch":34,"log_start":"36'3700","ondisk_log_start":"36'3700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31109120,"num_objects":7595,"num_object_clones":0,"num_object_copies":15190,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7595,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10566,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104261+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:02:08.104261+0000","last_peered":"2024-09-10T16:02:08.104261+0000","last_clean":"2024-09-10T16:02:08.104261+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:02:08.104261+0000","last_undegraded":"2024-09-10T16:02:08.104261+0000","last_fullsized":"2024-09-10T16:02:08.104261+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'13199","reported_seq":13339,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:25.153041+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:04:25.153041+0000","last_peered":"2024-09-10T16:04:25.153041+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:04:25.153041+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"36'3100","ondisk_log_start":"36'3100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29729251,"num_objects":7259,"num_object_clones":0,"num_object_copies":14518,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7259,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11018,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1156},{"shards":"2","objects":3136}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103774+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:02:08.103774+0000","last_peered":"2024-09-10T16:02:08.103774+0000","last_clean":"2024-09-10T16:02:08.103774+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:02:08.103774+0000","last_undegraded":"2024-09-10T16:02:08.103774+0000","last_fullsized":"2024-09-10T16:02:08.103774+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'13522","reported_seq":13653,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:23.940596+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:04:23.940596+0000","last_peered":"2024-09-10T16:04:23.940596+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:04:23.940596+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"36'3500","ondisk_log_start":"36'3500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30261248,"num_objects":7388,"num_object_clones":0,"num_object_copies":14776,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7388,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10026,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3133},{"shards":"0,2","objects":1192}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":105,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:12.405385+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:02:12.405385+0000","last_peered":"2024-09-10T16:02:12.405385+0000","last_clean":"2024-09-10T16:02:12.405385+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:02:12.405385+0000","last_undegraded":"2024-09-10T16:02:12.405385+0000","last_fullsized":"2024-09-10T16:02:12.405385+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'13470","reported_seq":13605,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:25.048401+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:04:25.048401+0000","last_peered":"2024-09-10T16:04:25.048401+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:04:25.048401+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"36'3400","ondisk_log_start":"36'3400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29946339,"num_objects":7312,"num_object_clones":0,"num_object_copies":14624,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7312,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11150,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1227},{"shards":"2","objects":3008}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104617+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:02:08.104617+0000","last_peered":"2024-09-10T16:02:08.104617+0000","last_clean":"2024-09-10T16:02:08.104617+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:02:08.104617+0000","last_undegraded":"2024-09-10T16:02:08.104617+0000","last_fullsized":"2024-09-10T16:02:08.104617+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'13696","reported_seq":13826,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:25.055054+0000","last_change":"2024-09-10T15:59:16.174113+0000","last_active":"2024-09-10T16:04:25.055054+0000","last_peered":"2024-09-10T16:04:25.055054+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:04:25.055054+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"36'3600","ondisk_log_start":"36'3600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30613504,"num_objects":7474,"num_object_clones":0,"num_object_copies":14948,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4354,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7474,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11250,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1244},{"shards":"2","objects":3122}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'13467","reported_seq":13602,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:23.952004+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:04:23.952004+0000","last_peered":"2024-09-10T16:04:23.952004+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:04:23.952004+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"36'3400","ondisk_log_start":"36'3400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30191616,"num_objects":7371,"num_object_clones":0,"num_object_copies":14742,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7371,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10098,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3067},{"shards":"0,2","objects":1257}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103316+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:02:08.103316+0000","last_peered":"2024-09-10T16:02:08.103316+0000","last_clean":"2024-09-10T16:02:08.103316+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:02:08.103316+0000","last_undegraded":"2024-09-10T16:02:08.103316+0000","last_fullsized":"2024-09-10T16:02:08.103316+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'13281","reported_seq":18643,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:23.956259+0000","last_change":"2024-09-10T16:03:29.605682+0000","last_active":"2024-09-10T16:04:23.956259+0000","last_peered":"2024-09-10T16:04:23.956259+0000","last_clean":"2024-09-10T15:59:14.115595+0000","last_became_active":"2024-09-10T15:59:16.244089+0000","last_became_peered":"2024-09-10T15:59:16.244089+0000","last_unstale":"2024-09-10T16:04:23.956259+0000","last_undegraded":"2024-09-10T16:03:29.603815+0000","last_fullsized":"2024-09-10T15:59:15.774437+0000","mapping_epoch":42,"log_start":"36'3200","ondisk_log_start":"36'3200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29970432,"num_objects":7317,"num_object_clones":0,"num_object_copies":14634,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":394,"num_objects_misplaced":1203,"num_objects_unfound":0,"num_objects_dirty":7317,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9671,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2609,"num_bytes_recovered":10674176,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5720},{"shards":"0,2","objects":1203}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103831+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:02:08.103831+0000","last_peered":"2024-09-10T16:02:08.103831+0000","last_clean":"2024-09-10T16:02:08.103831+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:02:08.103831+0000","last_undegraded":"2024-09-10T16:02:08.103831+0000","last_fullsized":"2024-09-10T16:02:08.103831+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'13672","reported_seq":13814,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:23.921001+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:04:23.921001+0000","last_peered":"2024-09-10T16:04:23.921001+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:04:23.921001+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"36'3600","ondisk_log_start":"36'3600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30212096,"num_objects":7376,"num_object_clones":0,"num_object_copies":14752,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7376,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9927,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3005},{"shards":"0,2","objects":1225}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104463+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:02:08.104463+0000","last_peered":"2024-09-10T16:02:08.104463+0000","last_clean":"2024-09-10T16:02:08.104463+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:02:08.104463+0000","last_undegraded":"2024-09-10T16:02:08.104463+0000","last_fullsized":"2024-09-10T16:02:08.104463+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104448+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:02:08.104448+0000","last_peered":"2024-09-10T16:02:08.104448+0000","last_clean":"2024-09-10T16:02:08.104448+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:02:08.104448+0000","last_undegraded":"2024-09-10T16:02:08.104448+0000","last_fullsized":"2024-09-10T16:02:08.104448+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'13634","reported_seq":13779,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:23.841758+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:04:23.841758+0000","last_peered":"2024-09-10T16:04:23.841758+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:04:23.841758+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"36'3600","ondisk_log_start":"36'3600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30983110,"num_objects":7566,"num_object_clones":0,"num_object_copies":15132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7566,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9797,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3243},{"shards":"0,2","objects":1294}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":484451062,"num_objects":118284,"num_object_clones":0,"num_object_copies":236568,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43305,"num_objects_misplaced":1203,"num_objects_unfound":0,"num_objects_dirty":118284,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":173822,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6881,"num_bytes_recovered":28168192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1123409920,"data_stored":1123260869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160948,"ondisk_log_size":160948,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986253,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":671932,"kb_used_data":373672,"kb_used_omap":4,"kb_used_meta":298235,"kb_avail":93699908,"statfs":{"total":96636764160,"available":95948705792,"internally_reserved":0,"allocated":382640128,"data_stored":382347895,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":305393322},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,1,1,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659198,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":585756,"kb_used_data":301388,"kb_used_omap":4,"kb_used_meta":284347,"kb_avail":93786084,"statfs":{"total":96636764160,"available":96036950016,"internally_reserved":0,"allocated":308621312,"data_stored":308292485,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":291172014},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,0,1,2,8,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182481,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":817436,"kb_used_data":424844,"kb_used_omap":4,"kb_used_meta":392571,"kb_avail":93554404,"statfs":{"total":96636764160,"available":95799709696,"internally_reserved":0,"allocated":435040256,"data_stored":434748018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":401993386},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,1,1,1,4,3,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":433811456,"data_stored":433775326,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":308244480,"data_stored":308171761,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":381353984,"data_stored":381313782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:04:27.134 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:04:27.687 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:04:27.691 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:04:27.717 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":254,"stamp":"2024-09-10T16:04:27.658165+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":483857161,"num_objects":118141,"num_object_clones":0,"num_object_copies":236282,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43305,"num_objects_misplaced":1203,"num_objects_unfound":0,"num_objects_dirty":118141,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":173969,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6883,"num_bytes_recovered":28168211,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160896,"ondisk_log_size":160896,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2082948,"kb_used_data":1099408,"kb_used_omap":12,"kb_used_meta":983475,"kb_avail":281032572,"statfs":{"total":289910292480,"available":287777353728,"internally_reserved":0,"allocated":1125793792,"data_stored":1124880494,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1007078402},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,2,6,5,5,8,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18944000,"num_objects":-4625,"num_object_clones":0,"num_object_copies":-9250,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":14,"num_objects_misplaced":-262,"num_objects_unfound":0,"num_objects_dirty":-4625,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4625,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":248,"num_bytes_recovered":1015808,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004380"},"pg_stats":[{"pgid":"2.f","version":"45'13346","reported_seq":13421,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:23.903896+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:04:23.903896+0000","last_peered":"2024-09-10T16:04:23.903896+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:04:23.903896+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"36'3300","ondisk_log_start":"36'3300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30007296,"num_objects":7326,"num_object_clones":0,"num_object_copies":14652,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7326,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13346,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3067},{"shards":"0,2","objects":1248}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'13580","reported_seq":13693,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:04:25.184577+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:04:25.184577+0000","last_peered":"2024-09-10T16:04:25.184577+0000","last_clean":"2024-09-10T16:04:25.184577+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:04:25.184577+0000","last_undegraded":"2024-09-10T16:04:25.184577+0000","last_fullsized":"2024-09-10T16:04:25.184577+0000","mapping_epoch":35,"log_start":"36'3500","ondisk_log_start":"36'3500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29754310,"num_objects":7266,"num_object_clones":0,"num_object_copies":14532,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7266,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":11685,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'13507","reported_seq":22186,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:04:26.630400+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:04:26.630400+0000","last_peered":"2024-09-10T16:04:26.630400+0000","last_clean":"2024-09-10T16:04:26.630400+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:04:26.630400+0000","last_undegraded":"2024-09-10T16:04:26.630400+0000","last_fullsized":"2024-09-10T16:04:26.630400+0000","mapping_epoch":44,"log_start":"36'3500","ondisk_log_start":"36'3500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29859864,"num_objects":7291,"num_object_clones":0,"num_object_copies":14582,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7291,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11511,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'13370","reported_seq":13451,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:04:25.184806+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:04:25.184806+0000","last_peered":"2024-09-10T16:04:25.184806+0000","last_clean":"2024-09-10T16:04:25.184806+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:04:25.184806+0000","last_undegraded":"2024-09-10T16:04:25.184806+0000","last_fullsized":"2024-09-10T16:04:25.184806+0000","mapping_epoch":35,"log_start":"36'3300","ondisk_log_start":"36'3300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29859840,"num_objects":7290,"num_object_clones":0,"num_object_copies":14580,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7290,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13370,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'13553","reported_seq":13682,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:04:23.958828+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:04:23.958828+0000","last_peered":"2024-09-10T16:04:23.958828+0000","last_clean":"2024-09-10T16:04:23.958828+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:04:23.958828+0000","last_undegraded":"2024-09-10T16:04:23.958828+0000","last_fullsized":"2024-09-10T16:04:23.958828+0000","mapping_epoch":34,"log_start":"36'3500","ondisk_log_start":"36'3500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30056931,"num_objects":7339,"num_object_clones":0,"num_object_copies":14678,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7339,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10264,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'13628","reported_seq":13765,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:23.941323+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:04:23.941323+0000","last_peered":"2024-09-10T16:04:23.941323+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:04:23.941323+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"36'3600","ondisk_log_start":"36'3600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30446051,"num_objects":7434,"num_object_clones":0,"num_object_copies":14868,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7434,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10240,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3063},{"shards":"0,2","objects":1276}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'13549","reported_seq":13686,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:23.913936+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:04:23.913936+0000","last_peered":"2024-09-10T16:04:23.913936+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:04:23.913936+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"36'3500","ondisk_log_start":"36'3500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30856134,"num_objects":7535,"num_object_clones":0,"num_object_copies":15070,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7535,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":10048,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3298},{"shards":"0,2","objects":1232}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'13719","reported_seq":13849,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:04:23.876162+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:04:23.876162+0000","last_peered":"2024-09-10T16:04:23.876162+0000","last_clean":"2024-09-10T16:04:23.876162+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:04:23.876162+0000","last_undegraded":"2024-09-10T16:04:23.876162+0000","last_fullsized":"2024-09-10T16:04:23.876162+0000","mapping_epoch":34,"log_start":"36'3700","ondisk_log_start":"36'3700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31109120,"num_objects":7595,"num_object_clones":0,"num_object_copies":15190,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7595,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10566,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104261+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:02:08.104261+0000","last_peered":"2024-09-10T16:02:08.104261+0000","last_clean":"2024-09-10T16:02:08.104261+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:02:08.104261+0000","last_undegraded":"2024-09-10T16:02:08.104261+0000","last_fullsized":"2024-09-10T16:02:08.104261+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'13199","reported_seq":13339,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:25.153041+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:04:25.153041+0000","last_peered":"2024-09-10T16:04:25.153041+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:04:25.153041+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"36'3100","ondisk_log_start":"36'3100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29729251,"num_objects":7259,"num_object_clones":0,"num_object_copies":14518,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7259,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11018,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1156},{"shards":"2","objects":3136}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103774+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:02:08.103774+0000","last_peered":"2024-09-10T16:02:08.103774+0000","last_clean":"2024-09-10T16:02:08.103774+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:02:08.103774+0000","last_undegraded":"2024-09-10T16:02:08.103774+0000","last_fullsized":"2024-09-10T16:02:08.103774+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'13522","reported_seq":13653,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:23.940596+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:04:23.940596+0000","last_peered":"2024-09-10T16:04:23.940596+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:04:23.940596+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"36'3500","ondisk_log_start":"36'3500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30261248,"num_objects":7388,"num_object_clones":0,"num_object_copies":14776,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7388,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10026,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3133},{"shards":"0,2","objects":1192}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":105,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:12.405385+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:02:12.405385+0000","last_peered":"2024-09-10T16:02:12.405385+0000","last_clean":"2024-09-10T16:02:12.405385+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:02:12.405385+0000","last_undegraded":"2024-09-10T16:02:12.405385+0000","last_fullsized":"2024-09-10T16:02:12.405385+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'13470","reported_seq":13605,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:25.048401+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:04:25.048401+0000","last_peered":"2024-09-10T16:04:25.048401+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:04:25.048401+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"36'3400","ondisk_log_start":"36'3400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29946339,"num_objects":7312,"num_object_clones":0,"num_object_copies":14624,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7312,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11150,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1227},{"shards":"2","objects":3008}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104617+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:02:08.104617+0000","last_peered":"2024-09-10T16:02:08.104617+0000","last_clean":"2024-09-10T16:02:08.104617+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:02:08.104617+0000","last_undegraded":"2024-09-10T16:02:08.104617+0000","last_fullsized":"2024-09-10T16:02:08.104617+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'13696","reported_seq":13826,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:25.055054+0000","last_change":"2024-09-10T15:59:16.174113+0000","last_active":"2024-09-10T16:04:25.055054+0000","last_peered":"2024-09-10T16:04:25.055054+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:04:25.055054+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"36'3600","ondisk_log_start":"36'3600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30613504,"num_objects":7474,"num_object_clones":0,"num_object_copies":14948,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4354,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7474,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11250,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1244},{"shards":"2","objects":3122}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'13467","reported_seq":13602,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:23.952004+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:04:23.952004+0000","last_peered":"2024-09-10T16:04:23.952004+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:04:23.952004+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"36'3400","ondisk_log_start":"36'3400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30191616,"num_objects":7371,"num_object_clones":0,"num_object_copies":14742,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7371,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10098,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3067},{"shards":"0,2","objects":1257}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103316+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:02:08.103316+0000","last_peered":"2024-09-10T16:02:08.103316+0000","last_clean":"2024-09-10T16:02:08.103316+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:02:08.103316+0000","last_undegraded":"2024-09-10T16:02:08.103316+0000","last_fullsized":"2024-09-10T16:02:08.103316+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'13281","reported_seq":18643,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:23.956259+0000","last_change":"2024-09-10T16:03:29.605682+0000","last_active":"2024-09-10T16:04:23.956259+0000","last_peered":"2024-09-10T16:04:23.956259+0000","last_clean":"2024-09-10T15:59:14.115595+0000","last_became_active":"2024-09-10T15:59:16.244089+0000","last_became_peered":"2024-09-10T15:59:16.244089+0000","last_unstale":"2024-09-10T16:04:23.956259+0000","last_undegraded":"2024-09-10T16:03:29.603815+0000","last_fullsized":"2024-09-10T15:59:15.774437+0000","mapping_epoch":42,"log_start":"36'3200","ondisk_log_start":"36'3200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29970432,"num_objects":7317,"num_object_clones":0,"num_object_copies":14634,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":394,"num_objects_misplaced":1203,"num_objects_unfound":0,"num_objects_dirty":7317,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9671,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2609,"num_bytes_recovered":10674176,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5720},{"shards":"0,2","objects":1203}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103831+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:02:08.103831+0000","last_peered":"2024-09-10T16:02:08.103831+0000","last_clean":"2024-09-10T16:02:08.103831+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:02:08.103831+0000","last_undegraded":"2024-09-10T16:02:08.103831+0000","last_fullsized":"2024-09-10T16:02:08.103831+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'13672","reported_seq":13814,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:23.921001+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:04:23.921001+0000","last_peered":"2024-09-10T16:04:23.921001+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:04:23.921001+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"36'3600","ondisk_log_start":"36'3600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30212096,"num_objects":7376,"num_object_clones":0,"num_object_copies":14752,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7376,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9927,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3005},{"shards":"0,2","objects":1225}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104463+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:02:08.104463+0000","last_peered":"2024-09-10T16:02:08.104463+0000","last_clean":"2024-09-10T16:02:08.104463+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:02:08.104463+0000","last_undegraded":"2024-09-10T16:02:08.104463+0000","last_fullsized":"2024-09-10T16:02:08.104463+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104448+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:02:08.104448+0000","last_peered":"2024-09-10T16:02:08.104448+0000","last_clean":"2024-09-10T16:02:08.104448+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:02:08.104448+0000","last_undegraded":"2024-09-10T16:02:08.104448+0000","last_fullsized":"2024-09-10T16:02:08.104448+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'13634","reported_seq":13779,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:23.841758+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:04:23.841758+0000","last_peered":"2024-09-10T16:04:23.841758+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:04:23.841758+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"36'3600","ondisk_log_start":"36'3600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30983110,"num_objects":7566,"num_object_clones":0,"num_object_copies":15132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7566,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9797,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3243},{"shards":"0,2","objects":1294}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483857142,"num_objects":118139,"num_object_clones":0,"num_object_copies":236278,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43305,"num_objects_misplaced":1203,"num_objects_unfound":0,"num_objects_dirty":118139,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":173967,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6881,"num_bytes_recovered":28168192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1122897920,"data_stored":1122748869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160893,"ondisk_log_size":160893,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986253,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":671932,"kb_used_data":373672,"kb_used_omap":4,"kb_used_meta":298235,"kb_avail":93699908,"statfs":{"total":96636764160,"available":95948705792,"internally_reserved":0,"allocated":382640128,"data_stored":382347895,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":305393322},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,1,1,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659199,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":593580,"kb_used_data":300892,"kb_used_omap":4,"kb_used_meta":292667,"kb_avail":93778260,"statfs":{"total":96636764160,"available":96028938240,"internally_reserved":0,"allocated":308113408,"data_stored":307784581,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":299691694},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,3],"upper_bound":32},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182481,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":817436,"kb_used_data":424844,"kb_used_omap":4,"kb_used_meta":392571,"kb_avail":93554404,"statfs":{"total":96636764160,"available":95799709696,"internally_reserved":0,"allocated":435040256,"data_stored":434748018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":401993386},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,1,1,1,4,3,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":433811456,"data_stored":433775326,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":307732480,"data_stored":307659761,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":381353984,"data_stored":381313782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:04:27.718 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-10T16:04:27.718 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "45'13346", 'reported_seq': 13421, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:04:23.903896+0000', 'last_change': '2024-09-10T15:59:16.281558+0000', 'last_active': '2024-09-10T16:04:23.903896+0000', 'last_peered': '2024-09-10T16:04:23.903896+0000', 'last_clean': '2024-09-10T15:59:14.132456+0000', 'last_became_active': '2024-09-10T15:59:16.274407+0000', 'last_became_peered': '2024-09-10T15:59:16.274407+0000', 'last_unstale': '2024-09-10T16:04:23.903896+0000', 'last_undegraded': '2024-09-10T15:59:15.851279+0000', 'last_fullsized': '2024-09-10T15:59:15.774356+0000', 'mapping_epoch': 42, 'log_start': "36'3300", 'ondisk_log_start': "36'3300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'objects_scrubbed': 0, 'log_size': 10046, 'log_dups_size': 0, 'ondisk_log_size': 10046, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30007296, 'num_objects': 7326, 'num_object_clones': 0, 'num_object_copies': 14652, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4259, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7326, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13346, 'num_write_kb': 41344, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3067}, {'shards': '0,2', 'objects': 1248}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:04:27.718 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-10T16:04:27.719 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "45'13628", 'reported_seq': 13765, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:04:23.941323+0000', 'last_change': '2024-09-10T15:59:16.308264+0000', 'last_active': '2024-09-10T16:04:23.941323+0000', 'last_peered': '2024-09-10T16:04:23.941323+0000', 'last_clean': '2024-09-10T15:59:14.111551+0000', 'last_became_active': '2024-09-10T15:59:16.300040+0000', 'last_became_peered': '2024-09-10T15:59:16.300040+0000', 'last_unstale': '2024-09-10T16:04:23.941323+0000', 'last_undegraded': '2024-09-10T15:59:15.878234+0000', 'last_fullsized': '2024-09-10T15:59:15.769328+0000', 'mapping_epoch': 42, 'log_start': "36'3600", 'ondisk_log_start': "36'3600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3442", 'last_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'objects_scrubbed': 3384, 'log_size': 10028, 'log_dups_size': 0, 'ondisk_log_size': 10028, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1292, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30446051, 'num_objects': 7434, 'num_object_clones': 0, 'num_object_copies': 14868, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4372, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7434, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 10240, 'num_write_kb': 28580, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3063}, {'shards': '0,2', 'objects': 1276}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:04:27.719 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-10T16:04:27.719 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "45'13549", 'reported_seq': 13686, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:04:23.913936+0000', 'last_change': '2024-09-10T15:59:16.331899+0000', 'last_active': '2024-09-10T16:04:23.913936+0000', 'last_peered': '2024-09-10T16:04:23.913936+0000', 'last_clean': '2024-09-10T15:59:14.139436+0000', 'last_became_active': '2024-09-10T15:59:16.318715+0000', 'last_became_peered': '2024-09-10T15:59:16.318715+0000', 'last_unstale': '2024-09-10T16:04:23.913936+0000', 'last_undegraded': '2024-09-10T15:59:15.877524+0000', 'last_fullsized': '2024-09-10T15:59:15.769042+0000', 'mapping_epoch': 42, 'log_start': "36'3500", 'ondisk_log_start': "36'3500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3537", 'last_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'objects_scrubbed': 3497, 'log_size': 10049, 'log_dups_size': 0, 'ondisk_log_size': 10049, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1224, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30856134, 'num_objects': 7535, 'num_object_clones': 0, 'num_object_copies': 15070, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4237, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7535, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 10048, 'num_write_kb': 28172, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3298}, {'shards': '0,2', 'objects': 1232}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:04:27.719 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-10T16:04:27.719 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "45'13199", 'reported_seq': 13339, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:04:25.153041+0000', 'last_change': '2024-09-10T15:59:16.175680+0000', 'last_active': '2024-09-10T16:04:25.153041+0000', 'last_peered': '2024-09-10T16:04:25.153041+0000', 'last_clean': '2024-09-10T15:59:14.088187+0000', 'last_became_active': '2024-09-10T15:59:16.165810+0000', 'last_became_peered': '2024-09-10T15:59:16.165810+0000', 'last_unstale': '2024-09-10T16:04:25.153041+0000', 'last_undegraded': '2024-09-10T15:59:15.822098+0000', 'last_fullsized': '2024-09-10T15:59:15.769123+0000', 'mapping_epoch': 42, 'log_start': "36'3100", 'ondisk_log_start': "36'3100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2218", 'last_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'objects_scrubbed': 2175, 'log_size': 10099, 'log_dups_size': 0, 'ondisk_log_size': 10099, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 825, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29729251, 'num_objects': 7259, 'num_object_clones': 0, 'num_object_copies': 14518, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4124, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7259, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 11018, 'num_write_kb': 32204, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1156}, {'shards': '2', 'objects': 3136}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:04:27.720 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-10T16:04:27.720 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "45'13522", 'reported_seq': 13653, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:04:23.940596+0000', 'last_change': '2024-09-10T15:59:16.277742+0000', 'last_active': '2024-09-10T16:04:23.940596+0000', 'last_peered': '2024-09-10T16:04:23.940596+0000', 'last_clean': '2024-09-10T15:59:14.129081+0000', 'last_became_active': '2024-09-10T15:59:16.269835+0000', 'last_became_peered': '2024-09-10T15:59:16.269835+0000', 'last_unstale': '2024-09-10T16:04:23.940596+0000', 'last_undegraded': '2024-09-10T15:59:15.860857+0000', 'last_fullsized': '2024-09-10T15:59:15.771410+0000', 'mapping_epoch': 42, 'log_start': "36'3500", 'ondisk_log_start': "36'3500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3557", 'last_scrub_stamp': '2024-09-10T15:59:02.493678+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:02.493678+0000', 'objects_scrubbed': 3488, 'log_size': 10022, 'log_dups_size': 0, 'ondisk_log_size': 10022, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1472, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30261248, 'num_objects': 7388, 'num_object_clones': 0, 'num_object_copies': 14776, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4257, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7388, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10026, 'num_write_kb': 27852, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3133}, {'shards': '0,2', 'objects': 1192}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:04:27.720 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-10T16:04:27.720 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "45'13470", 'reported_seq': 13605, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:04:25.048401+0000', 'last_change': '2024-09-10T15:59:16.284311+0000', 'last_active': '2024-09-10T16:04:25.048401+0000', 'last_peered': '2024-09-10T16:04:25.048401+0000', 'last_clean': '2024-09-10T15:59:14.116041+0000', 'last_became_active': '2024-09-10T15:59:16.274543+0000', 'last_became_peered': '2024-09-10T15:59:16.274543+0000', 'last_unstale': '2024-09-10T16:04:25.048401+0000', 'last_undegraded': '2024-09-10T15:59:15.812484+0000', 'last_fullsized': '2024-09-10T15:59:15.770885+0000', 'mapping_epoch': 42, 'log_start': "36'3400", 'ondisk_log_start': "36'3400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2356", 'last_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'objects_scrubbed': 2316, 'log_size': 10070, 'log_dups_size': 0, 'ondisk_log_size': 10070, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 783, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29946339, 'num_objects': 7312, 'num_object_clones': 0, 'num_object_copies': 14624, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7312, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 11150, 'num_write_kb': 32292, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1227}, {'shards': '2', 'objects': 3008}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:04:27.720 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-10T16:04:27.720 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "45'13696", 'reported_seq': 13826, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:04:25.055054+0000', 'last_change': '2024-09-10T15:59:16.174113+0000', 'last_active': '2024-09-10T16:04:25.055054+0000', 'last_peered': '2024-09-10T16:04:25.055054+0000', 'last_clean': '2024-09-10T15:59:14.110654+0000', 'last_became_active': '2024-09-10T15:59:16.164645+0000', 'last_became_peered': '2024-09-10T15:59:16.164645+0000', 'last_unstale': '2024-09-10T16:04:25.055054+0000', 'last_undegraded': '2024-09-10T15:59:15.803593+0000', 'last_fullsized': '2024-09-10T15:59:15.769083+0000', 'mapping_epoch': 42, 'log_start': "36'3600", 'ondisk_log_start': "36'3600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2486", 'last_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'objects_scrubbed': 2440, 'log_size': 10096, 'log_dups_size': 0, 'ondisk_log_size': 10096, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 792, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30613504, 'num_objects': 7474, 'num_object_clones': 0, 'num_object_copies': 14948, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4354, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7474, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11250, 'num_write_kb': 32568, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1244}, {'shards': '2', 'objects': 3122}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:04:27.720 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-10T16:04:27.720 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "45'13467", 'reported_seq': 13602, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:04:23.952004+0000', 'last_change': '2024-09-10T15:59:16.315641+0000', 'last_active': '2024-09-10T16:04:23.952004+0000', 'last_peered': '2024-09-10T16:04:23.952004+0000', 'last_clean': '2024-09-10T15:59:14.117001+0000', 'last_became_active': '2024-09-10T15:59:16.302245+0000', 'last_became_peered': '2024-09-10T15:59:16.302245+0000', 'last_unstale': '2024-09-10T16:04:23.952004+0000', 'last_undegraded': '2024-09-10T15:59:15.845012+0000', 'last_fullsized': '2024-09-10T15:59:15.770337+0000', 'mapping_epoch': 42, 'log_start': "36'3400", 'ondisk_log_start': "36'3400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3404", 'last_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'objects_scrubbed': 3367, 'log_size': 10067, 'log_dups_size': 0, 'ondisk_log_size': 10067, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 983, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30191616, 'num_objects': 7371, 'num_object_clones': 0, 'num_object_copies': 14742, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7371, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10098, 'num_write_kb': 28204, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3067}, {'shards': '0,2', 'objects': 1257}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:04:27.720 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-10T16:04:27.720 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "45'13281", 'reported_seq': 18643, 'reported_epoch': 45, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:04:23.956259+0000', 'last_change': '2024-09-10T16:03:29.605682+0000', 'last_active': '2024-09-10T16:04:23.956259+0000', 'last_peered': '2024-09-10T16:04:23.956259+0000', 'last_clean': '2024-09-10T15:59:14.115595+0000', 'last_became_active': '2024-09-10T15:59:16.244089+0000', 'last_became_peered': '2024-09-10T15:59:16.244089+0000', 'last_unstale': '2024-09-10T16:04:23.956259+0000', 'last_undegraded': '2024-09-10T16:03:29.603815+0000', 'last_fullsized': '2024-09-10T15:59:15.774437+0000', 'mapping_epoch': 42, 'log_start': "36'3200", 'ondisk_log_start': "36'3200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3653", 'last_scrub_stamp': '2024-09-10T15:59:05.298468+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:05.298468+0000', 'objects_scrubbed': 3602, 'log_size': 10081, 'log_dups_size': 0, 'ondisk_log_size': 10081, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1289, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29970432, 'num_objects': 7317, 'num_object_clones': 0, 'num_object_copies': 14634, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 394, 'num_objects_misplaced': 1203, 'num_objects_unfound': 0, 'num_objects_dirty': 7317, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9671, 'num_write_kb': 26772, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2609, 'num_bytes_recovered': 10674176, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5720}, {'shards': '0,2', 'objects': 1203}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:04:27.721 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-10T16:04:27.721 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "45'13672", 'reported_seq': 13814, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:04:23.921001+0000', 'last_change': '2024-09-10T15:59:16.311291+0000', 'last_active': '2024-09-10T16:04:23.921001+0000', 'last_peered': '2024-09-10T16:04:23.921001+0000', 'last_clean': '2024-09-10T15:59:14.116363+0000', 'last_became_active': '2024-09-10T15:59:16.301244+0000', 'last_became_peered': '2024-09-10T15:59:16.301244+0000', 'last_unstale': '2024-09-10T16:04:23.921001+0000', 'last_undegraded': '2024-09-10T15:59:15.883434+0000', 'last_fullsized': '2024-09-10T15:59:15.772756+0000', 'mapping_epoch': 42, 'log_start': "36'3600", 'ondisk_log_start': "36'3600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3789", 'last_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'objects_scrubbed': 3741, 'log_size': 10072, 'log_dups_size': 0, 'ondisk_log_size': 10072, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1440, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30212096, 'num_objects': 7376, 'num_object_clones': 0, 'num_object_copies': 14752, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4373, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7376, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9927, 'num_write_kb': 27124, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3005}, {'shards': '0,2', 'objects': 1225}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:04:27.721 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-10T16:04:27.721 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "45'13634", 'reported_seq': 13779, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:04:23.841758+0000', 'last_change': '2024-09-10T15:59:16.313409+0000', 'last_active': '2024-09-10T16:04:23.841758+0000', 'last_peered': '2024-09-10T16:04:23.841758+0000', 'last_clean': '2024-09-10T15:59:14.129780+0000', 'last_became_active': '2024-09-10T15:59:16.300932+0000', 'last_became_peered': '2024-09-10T15:59:16.300932+0000', 'last_unstale': '2024-09-10T16:04:23.841758+0000', 'last_undegraded': '2024-09-10T15:59:15.887823+0000', 'last_fullsized': '2024-09-10T15:59:15.768513+0000', 'mapping_epoch': 42, 'log_start': "36'3600", 'ondisk_log_start': "36'3600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3880", 'last_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'objects_scrubbed': 3827, 'log_size': 10034, 'log_dups_size': 0, 'ondisk_log_size': 10034, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1397, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30983110, 'num_objects': 7566, 'num_object_clones': 0, 'num_object_copies': 15132, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4325, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7566, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9797, 'num_write_kb': 27072, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3243}, {'shards': '0,2', 'objects': 1294}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:04:27.721 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T16:04:28.426 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:04:28.426 INFO:teuthology.orchestra.run.smithi115.stdout:{"fsid":"7effee29-df05-4351-9741-ee203564ed81","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 43305/236282 objects degraded (18.328%), 11 pgs degraded, 11 pgs undersized","count":22},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":432,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983954,"num_in_osds":3,"osd_in_since":1725983947,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":118141,"data_bytes":483857161,"bytes_used":2132938752,"bytes_avail":287777353728,"bytes_total":289910292480,"degraded_objects":43305,"degraded_total":236282,"degraded_ratio":0.1832767625125909,"misplaced_objects":1203,"misplaced_total":236282,"misplaced_ratio":0.0050913738668201562,"recovering_objects_per_sec":20,"recovering_bytes_per_sec":84619,"recovering_keys_per_sec":0,"num_objects_recovered":248,"num_bytes_recovered":1015808,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":385},"fsmap":{"epoch":1,"btime":"2024-09-10T15:57:14:967120+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T16:00:13.560776+0000","services":{}},"progress_events":{}} 2024-09-10T16:04:28.453 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-10T16:04:28.453 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:04:29.019 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:04:29.020 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:04:29.044 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":254,"stamp":"2024-09-10T16:04:27.658165+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":483857161,"num_objects":118141,"num_object_clones":0,"num_object_copies":236282,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43305,"num_objects_misplaced":1203,"num_objects_unfound":0,"num_objects_dirty":118141,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":173969,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6883,"num_bytes_recovered":28168211,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160896,"ondisk_log_size":160896,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2082948,"kb_used_data":1099408,"kb_used_omap":12,"kb_used_meta":983475,"kb_avail":281032572,"statfs":{"total":289910292480,"available":287777353728,"internally_reserved":0,"allocated":1125793792,"data_stored":1124880494,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1007078402},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,2,6,5,5,8,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18944000,"num_objects":-4625,"num_object_clones":0,"num_object_copies":-9250,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":14,"num_objects_misplaced":-262,"num_objects_unfound":0,"num_objects_dirty":-4625,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4625,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":248,"num_bytes_recovered":1015808,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004380"},"pg_stats":[{"pgid":"2.f","version":"45'13346","reported_seq":13421,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:23.903896+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:04:23.903896+0000","last_peered":"2024-09-10T16:04:23.903896+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:04:23.903896+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"36'3300","ondisk_log_start":"36'3300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30007296,"num_objects":7326,"num_object_clones":0,"num_object_copies":14652,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7326,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13346,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3067},{"shards":"0,2","objects":1248}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'13580","reported_seq":13693,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:04:25.184577+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:04:25.184577+0000","last_peered":"2024-09-10T16:04:25.184577+0000","last_clean":"2024-09-10T16:04:25.184577+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:04:25.184577+0000","last_undegraded":"2024-09-10T16:04:25.184577+0000","last_fullsized":"2024-09-10T16:04:25.184577+0000","mapping_epoch":35,"log_start":"36'3500","ondisk_log_start":"36'3500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29754310,"num_objects":7266,"num_object_clones":0,"num_object_copies":14532,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7266,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":11685,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'13507","reported_seq":22186,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:04:26.630400+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:04:26.630400+0000","last_peered":"2024-09-10T16:04:26.630400+0000","last_clean":"2024-09-10T16:04:26.630400+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:04:26.630400+0000","last_undegraded":"2024-09-10T16:04:26.630400+0000","last_fullsized":"2024-09-10T16:04:26.630400+0000","mapping_epoch":44,"log_start":"36'3500","ondisk_log_start":"36'3500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29859864,"num_objects":7291,"num_object_clones":0,"num_object_copies":14582,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7291,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11511,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'13370","reported_seq":13451,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:04:25.184806+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:04:25.184806+0000","last_peered":"2024-09-10T16:04:25.184806+0000","last_clean":"2024-09-10T16:04:25.184806+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:04:25.184806+0000","last_undegraded":"2024-09-10T16:04:25.184806+0000","last_fullsized":"2024-09-10T16:04:25.184806+0000","mapping_epoch":35,"log_start":"36'3300","ondisk_log_start":"36'3300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29859840,"num_objects":7290,"num_object_clones":0,"num_object_copies":14580,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7290,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13370,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'13553","reported_seq":13682,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:04:23.958828+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:04:23.958828+0000","last_peered":"2024-09-10T16:04:23.958828+0000","last_clean":"2024-09-10T16:04:23.958828+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:04:23.958828+0000","last_undegraded":"2024-09-10T16:04:23.958828+0000","last_fullsized":"2024-09-10T16:04:23.958828+0000","mapping_epoch":34,"log_start":"36'3500","ondisk_log_start":"36'3500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30056931,"num_objects":7339,"num_object_clones":0,"num_object_copies":14678,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7339,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10264,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'13628","reported_seq":13765,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:23.941323+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:04:23.941323+0000","last_peered":"2024-09-10T16:04:23.941323+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:04:23.941323+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"36'3600","ondisk_log_start":"36'3600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30446051,"num_objects":7434,"num_object_clones":0,"num_object_copies":14868,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7434,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10240,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3063},{"shards":"0,2","objects":1276}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'13549","reported_seq":13686,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:23.913936+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:04:23.913936+0000","last_peered":"2024-09-10T16:04:23.913936+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:04:23.913936+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"36'3500","ondisk_log_start":"36'3500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30856134,"num_objects":7535,"num_object_clones":0,"num_object_copies":15070,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7535,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":10048,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3298},{"shards":"0,2","objects":1232}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'13719","reported_seq":13849,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:04:23.876162+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:04:23.876162+0000","last_peered":"2024-09-10T16:04:23.876162+0000","last_clean":"2024-09-10T16:04:23.876162+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:04:23.876162+0000","last_undegraded":"2024-09-10T16:04:23.876162+0000","last_fullsized":"2024-09-10T16:04:23.876162+0000","mapping_epoch":34,"log_start":"36'3700","ondisk_log_start":"36'3700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31109120,"num_objects":7595,"num_object_clones":0,"num_object_copies":15190,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7595,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10566,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104261+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:02:08.104261+0000","last_peered":"2024-09-10T16:02:08.104261+0000","last_clean":"2024-09-10T16:02:08.104261+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:02:08.104261+0000","last_undegraded":"2024-09-10T16:02:08.104261+0000","last_fullsized":"2024-09-10T16:02:08.104261+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'13199","reported_seq":13339,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:25.153041+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:04:25.153041+0000","last_peered":"2024-09-10T16:04:25.153041+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:04:25.153041+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"36'3100","ondisk_log_start":"36'3100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29729251,"num_objects":7259,"num_object_clones":0,"num_object_copies":14518,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7259,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11018,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1156},{"shards":"2","objects":3136}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103774+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:02:08.103774+0000","last_peered":"2024-09-10T16:02:08.103774+0000","last_clean":"2024-09-10T16:02:08.103774+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:02:08.103774+0000","last_undegraded":"2024-09-10T16:02:08.103774+0000","last_fullsized":"2024-09-10T16:02:08.103774+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'13522","reported_seq":13653,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:23.940596+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:04:23.940596+0000","last_peered":"2024-09-10T16:04:23.940596+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:04:23.940596+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"36'3500","ondisk_log_start":"36'3500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30261248,"num_objects":7388,"num_object_clones":0,"num_object_copies":14776,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7388,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10026,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3133},{"shards":"0,2","objects":1192}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":105,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:12.405385+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:02:12.405385+0000","last_peered":"2024-09-10T16:02:12.405385+0000","last_clean":"2024-09-10T16:02:12.405385+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:02:12.405385+0000","last_undegraded":"2024-09-10T16:02:12.405385+0000","last_fullsized":"2024-09-10T16:02:12.405385+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'13470","reported_seq":13605,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:25.048401+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:04:25.048401+0000","last_peered":"2024-09-10T16:04:25.048401+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:04:25.048401+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"36'3400","ondisk_log_start":"36'3400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29946339,"num_objects":7312,"num_object_clones":0,"num_object_copies":14624,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7312,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11150,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1227},{"shards":"2","objects":3008}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104617+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:02:08.104617+0000","last_peered":"2024-09-10T16:02:08.104617+0000","last_clean":"2024-09-10T16:02:08.104617+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:02:08.104617+0000","last_undegraded":"2024-09-10T16:02:08.104617+0000","last_fullsized":"2024-09-10T16:02:08.104617+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'13696","reported_seq":13826,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:25.055054+0000","last_change":"2024-09-10T15:59:16.174113+0000","last_active":"2024-09-10T16:04:25.055054+0000","last_peered":"2024-09-10T16:04:25.055054+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:04:25.055054+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"36'3600","ondisk_log_start":"36'3600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30613504,"num_objects":7474,"num_object_clones":0,"num_object_copies":14948,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4354,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7474,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11250,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1244},{"shards":"2","objects":3122}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'13467","reported_seq":13602,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:23.952004+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:04:23.952004+0000","last_peered":"2024-09-10T16:04:23.952004+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:04:23.952004+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"36'3400","ondisk_log_start":"36'3400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30191616,"num_objects":7371,"num_object_clones":0,"num_object_copies":14742,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7371,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10098,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3067},{"shards":"0,2","objects":1257}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103316+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:02:08.103316+0000","last_peered":"2024-09-10T16:02:08.103316+0000","last_clean":"2024-09-10T16:02:08.103316+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:02:08.103316+0000","last_undegraded":"2024-09-10T16:02:08.103316+0000","last_fullsized":"2024-09-10T16:02:08.103316+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'13281","reported_seq":18643,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:23.956259+0000","last_change":"2024-09-10T16:03:29.605682+0000","last_active":"2024-09-10T16:04:23.956259+0000","last_peered":"2024-09-10T16:04:23.956259+0000","last_clean":"2024-09-10T15:59:14.115595+0000","last_became_active":"2024-09-10T15:59:16.244089+0000","last_became_peered":"2024-09-10T15:59:16.244089+0000","last_unstale":"2024-09-10T16:04:23.956259+0000","last_undegraded":"2024-09-10T16:03:29.603815+0000","last_fullsized":"2024-09-10T15:59:15.774437+0000","mapping_epoch":42,"log_start":"36'3200","ondisk_log_start":"36'3200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29970432,"num_objects":7317,"num_object_clones":0,"num_object_copies":14634,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":394,"num_objects_misplaced":1203,"num_objects_unfound":0,"num_objects_dirty":7317,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9671,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2609,"num_bytes_recovered":10674176,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5720},{"shards":"0,2","objects":1203}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103831+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:02:08.103831+0000","last_peered":"2024-09-10T16:02:08.103831+0000","last_clean":"2024-09-10T16:02:08.103831+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:02:08.103831+0000","last_undegraded":"2024-09-10T16:02:08.103831+0000","last_fullsized":"2024-09-10T16:02:08.103831+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'13672","reported_seq":13814,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:23.921001+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:04:23.921001+0000","last_peered":"2024-09-10T16:04:23.921001+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:04:23.921001+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"36'3600","ondisk_log_start":"36'3600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30212096,"num_objects":7376,"num_object_clones":0,"num_object_copies":14752,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7376,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9927,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3005},{"shards":"0,2","objects":1225}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104463+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:02:08.104463+0000","last_peered":"2024-09-10T16:02:08.104463+0000","last_clean":"2024-09-10T16:02:08.104463+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:02:08.104463+0000","last_undegraded":"2024-09-10T16:02:08.104463+0000","last_fullsized":"2024-09-10T16:02:08.104463+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104448+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:02:08.104448+0000","last_peered":"2024-09-10T16:02:08.104448+0000","last_clean":"2024-09-10T16:02:08.104448+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:02:08.104448+0000","last_undegraded":"2024-09-10T16:02:08.104448+0000","last_fullsized":"2024-09-10T16:02:08.104448+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'13634","reported_seq":13779,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:23.841758+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:04:23.841758+0000","last_peered":"2024-09-10T16:04:23.841758+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:04:23.841758+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"36'3600","ondisk_log_start":"36'3600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30983110,"num_objects":7566,"num_object_clones":0,"num_object_copies":15132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7566,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9797,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3243},{"shards":"0,2","objects":1294}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483857142,"num_objects":118139,"num_object_clones":0,"num_object_copies":236278,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43305,"num_objects_misplaced":1203,"num_objects_unfound":0,"num_objects_dirty":118139,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":173967,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6881,"num_bytes_recovered":28168192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1122897920,"data_stored":1122748869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160893,"ondisk_log_size":160893,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986253,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":671932,"kb_used_data":373672,"kb_used_omap":4,"kb_used_meta":298235,"kb_avail":93699908,"statfs":{"total":96636764160,"available":95948705792,"internally_reserved":0,"allocated":382640128,"data_stored":382347895,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":305393322},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,1,1,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659199,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":593580,"kb_used_data":300892,"kb_used_omap":4,"kb_used_meta":292667,"kb_avail":93778260,"statfs":{"total":96636764160,"available":96028938240,"internally_reserved":0,"allocated":308113408,"data_stored":307784581,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":299691694},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,3],"upper_bound":32},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182481,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":817436,"kb_used_data":424844,"kb_used_omap":4,"kb_used_meta":392571,"kb_avail":93554404,"statfs":{"total":96636764160,"available":95799709696,"internally_reserved":0,"allocated":435040256,"data_stored":434748018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":401993386},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,1,1,1,4,3,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":433811456,"data_stored":433775326,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":307732480,"data_stored":307659761,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":381353984,"data_stored":381313782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:04:32.050 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:04:32.617 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:04:32.618 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:04:32.640 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":256,"stamp":"2024-09-10T16:04:31.659482+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":475460361,"num_objects":116091,"num_object_clones":0,"num_object_copies":232182,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43305,"num_objects_misplaced":1073,"num_objects_unfound":0,"num_objects_dirty":116091,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":176019,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7013,"num_bytes_recovered":28700691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160846,"ondisk_log_size":160846,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2101608,"kb_used_data":1087540,"kb_used_omap":12,"kb_used_meta":1014003,"kb_avail":281013912,"statfs":{"total":289910292480,"available":287758245888,"internally_reserved":0,"allocated":1113640960,"data_stored":1112727662,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1038339074},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,4,10,7,7,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-20815872,"num_objects":-5082,"num_object_clones":0,"num_object_copies":-10164,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-257,"num_objects_unfound":0,"num_objects_dirty":-5082,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5082,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":257,"num_bytes_recovered":1052672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004286"},"pg_stats":[{"pgid":"2.f","version":"45'13498","reported_seq":13573,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:28.890013+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:04:28.890013+0000","last_peered":"2024-09-10T16:04:28.890013+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:04:28.890013+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"36'3400","ondisk_log_start":"36'3400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29384704,"num_objects":7174,"num_object_clones":0,"num_object_copies":14348,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7174,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13498,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2915},{"shards":"0,2","objects":1096}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'13703","reported_seq":13816,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:04:30.114991+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:04:30.114991+0000","last_peered":"2024-09-10T16:04:30.114991+0000","last_clean":"2024-09-10T16:04:30.114991+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:04:30.114991+0000","last_undegraded":"2024-09-10T16:04:30.114991+0000","last_fullsized":"2024-09-10T16:04:30.114991+0000","mapping_epoch":35,"log_start":"36'3700","ondisk_log_start":"36'3700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29250502,"num_objects":7143,"num_object_clones":0,"num_object_copies":14286,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7143,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":11808,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'13507","reported_seq":22186,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:04:26.630400+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:04:26.630400+0000","last_peered":"2024-09-10T16:04:26.630400+0000","last_clean":"2024-09-10T16:04:26.630400+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:04:26.630400+0000","last_undegraded":"2024-09-10T16:04:26.630400+0000","last_fullsized":"2024-09-10T16:04:26.630400+0000","mapping_epoch":44,"log_start":"36'3500","ondisk_log_start":"36'3500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29859864,"num_objects":7291,"num_object_clones":0,"num_object_copies":14582,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7291,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11511,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'13495","reported_seq":13576,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:04:30.145004+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:04:30.145004+0000","last_peered":"2024-09-10T16:04:30.145004+0000","last_clean":"2024-09-10T16:04:30.145004+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:04:30.145004+0000","last_undegraded":"2024-09-10T16:04:30.145004+0000","last_fullsized":"2024-09-10T16:04:30.145004+0000","mapping_epoch":35,"log_start":"36'3400","ondisk_log_start":"36'3400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29347840,"num_objects":7165,"num_object_clones":0,"num_object_copies":14330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7165,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13495,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'13693","reported_seq":13823,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.913327+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:04:28.913327+0000","last_peered":"2024-09-10T16:04:28.913327+0000","last_clean":"2024-09-10T16:04:28.913327+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:04:28.913327+0000","last_undegraded":"2024-09-10T16:04:28.913327+0000","last_fullsized":"2024-09-10T16:04:28.913327+0000","mapping_epoch":34,"log_start":"36'3600","ondisk_log_start":"36'3600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29483491,"num_objects":7199,"num_object_clones":0,"num_object_copies":14398,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7199,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10404,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'13790","reported_seq":13928,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:28.930978+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:04:28.930978+0000","last_peered":"2024-09-10T16:04:28.930978+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:04:28.930978+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"36'3700","ondisk_log_start":"36'3700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29782499,"num_objects":7272,"num_object_clones":0,"num_object_copies":14544,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7272,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10402,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2901},{"shards":"0,2","objects":1114}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'13686","reported_seq":13824,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:28.845772+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:04:28.845772+0000","last_peered":"2024-09-10T16:04:28.845772+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:04:28.845772+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"36'3600","ondisk_log_start":"36'3600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30294982,"num_objects":7398,"num_object_clones":0,"num_object_copies":14796,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7398,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":10185,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3161},{"shards":"0,2","objects":1095}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'13864","reported_seq":13994,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.994318+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:04:28.994318+0000","last_peered":"2024-09-10T16:04:28.994318+0000","last_clean":"2024-09-10T16:04:28.994318+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:04:28.994318+0000","last_undegraded":"2024-09-10T16:04:28.994318+0000","last_fullsized":"2024-09-10T16:04:28.994318+0000","mapping_epoch":34,"log_start":"36'3800","ondisk_log_start":"36'3800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30515200,"num_objects":7450,"num_object_clones":0,"num_object_copies":14900,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7450,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10711,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104261+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:02:08.104261+0000","last_peered":"2024-09-10T16:02:08.104261+0000","last_clean":"2024-09-10T16:02:08.104261+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:02:08.104261+0000","last_undegraded":"2024-09-10T16:02:08.104261+0000","last_fullsized":"2024-09-10T16:02:08.104261+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'13335","reported_seq":13471,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:30.179353+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:04:30.179353+0000","last_peered":"2024-09-10T16:04:30.179353+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:04:30.179353+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"36'3300","ondisk_log_start":"36'3300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29172195,"num_objects":7123,"num_object_clones":0,"num_object_copies":14246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7123,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11154,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1020},{"shards":"2","objects":3000}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103774+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:02:08.103774+0000","last_peered":"2024-09-10T16:02:08.103774+0000","last_clean":"2024-09-10T16:02:08.103774+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:02:08.103774+0000","last_undegraded":"2024-09-10T16:02:08.103774+0000","last_fullsized":"2024-09-10T16:02:08.103774+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'13636","reported_seq":13767,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:28.977466+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:04:28.977466+0000","last_peered":"2024-09-10T16:04:28.977466+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:04:28.977466+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"36'3600","ondisk_log_start":"36'3600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29794304,"num_objects":7274,"num_object_clones":0,"num_object_copies":14548,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7274,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10140,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3019},{"shards":"0,2","objects":1078}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":105,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:12.405385+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:02:12.405385+0000","last_peered":"2024-09-10T16:02:12.405385+0000","last_clean":"2024-09-10T16:02:12.405385+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:02:12.405385+0000","last_undegraded":"2024-09-10T16:02:12.405385+0000","last_fullsized":"2024-09-10T16:02:12.405385+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'13598","reported_seq":13730,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:30.122694+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:04:30.122694+0000","last_peered":"2024-09-10T16:04:30.122694+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:04:30.122694+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"36'3500","ondisk_log_start":"36'3500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29422051,"num_objects":7184,"num_object_clones":0,"num_object_copies":14368,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7184,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11278,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1099},{"shards":"2","objects":2880}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104617+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:02:08.104617+0000","last_peered":"2024-09-10T16:02:08.104617+0000","last_clean":"2024-09-10T16:02:08.104617+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:02:08.104617+0000","last_undegraded":"2024-09-10T16:02:08.104617+0000","last_fullsized":"2024-09-10T16:02:08.104617+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'13823","reported_seq":13951,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:30.180474+0000","last_change":"2024-09-10T15:59:16.174113+0000","last_active":"2024-09-10T16:04:30.180474+0000","last_peered":"2024-09-10T16:04:30.180474+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:04:30.180474+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"36'3800","ondisk_log_start":"36'3800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30093312,"num_objects":7347,"num_object_clones":0,"num_object_copies":14694,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4354,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7347,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11377,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1117},{"shards":"2","objects":2995}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'13605","reported_seq":13740,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:28.888826+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:04:28.888826+0000","last_peered":"2024-09-10T16:04:28.888826+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:04:28.888826+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"36'3600","ondisk_log_start":"36'3600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29626368,"num_objects":7233,"num_object_clones":0,"num_object_copies":14466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7233,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10236,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2929},{"shards":"0,2","objects":1119}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103316+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:02:08.103316+0000","last_peered":"2024-09-10T16:02:08.103316+0000","last_clean":"2024-09-10T16:02:08.103316+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:02:08.103316+0000","last_undegraded":"2024-09-10T16:02:08.103316+0000","last_fullsized":"2024-09-10T16:02:08.103316+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'13404","reported_seq":19026,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:28.996819+0000","last_change":"2024-09-10T16:03:29.605682+0000","last_active":"2024-09-10T16:04:28.996819+0000","last_peered":"2024-09-10T16:04:28.996819+0000","last_clean":"2024-09-10T15:59:14.115595+0000","last_became_active":"2024-09-10T15:59:16.244089+0000","last_became_peered":"2024-09-10T15:59:16.244089+0000","last_unstale":"2024-09-10T16:04:28.996819+0000","last_undegraded":"2024-09-10T16:03:29.603815+0000","last_fullsized":"2024-09-10T15:59:15.774437+0000","mapping_epoch":42,"log_start":"36'3400","ondisk_log_start":"36'3400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29466624,"num_objects":7194,"num_object_clones":0,"num_object_copies":14388,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":394,"num_objects_misplaced":1073,"num_objects_unfound":0,"num_objects_dirty":7194,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9794,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2739,"num_bytes_recovered":11206656,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5727},{"shards":"0,2","objects":1073}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103831+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:02:08.103831+0000","last_peered":"2024-09-10T16:02:08.103831+0000","last_clean":"2024-09-10T16:02:08.103831+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:02:08.103831+0000","last_undegraded":"2024-09-10T16:02:08.103831+0000","last_fullsized":"2024-09-10T16:02:08.103831+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'13812","reported_seq":13954,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:28.863045+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:04:28.863045+0000","last_peered":"2024-09-10T16:04:28.863045+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:04:28.863045+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"36'3800","ondisk_log_start":"36'3800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29638656,"num_objects":7236,"num_object_clones":0,"num_object_copies":14472,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7236,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10067,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2865},{"shards":"0,2","objects":1085}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104463+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:02:08.104463+0000","last_peered":"2024-09-10T16:02:08.104463+0000","last_clean":"2024-09-10T16:02:08.104463+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:02:08.104463+0000","last_undegraded":"2024-09-10T16:02:08.104463+0000","last_fullsized":"2024-09-10T16:02:08.104463+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104448+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:02:08.104448+0000","last_peered":"2024-09-10T16:02:08.104448+0000","last_clean":"2024-09-10T16:02:08.104448+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:02:08.104448+0000","last_undegraded":"2024-09-10T16:02:08.104448+0000","last_fullsized":"2024-09-10T16:02:08.104448+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'13794","reported_seq":13938,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:28.956477+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:04:28.956477+0000","last_peered":"2024-09-10T16:04:28.956477+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:04:28.956477+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"36'3700","ondisk_log_start":"36'3700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30327750,"num_objects":7406,"num_object_clones":0,"num_object_copies":14812,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7406,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9957,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3083},{"shards":"0,2","objects":1134}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":475460342,"num_objects":116089,"num_object_clones":0,"num_object_copies":232178,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43305,"num_objects_misplaced":1073,"num_objects_unfound":0,"num_objects_dirty":116089,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":176017,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7011,"num_bytes_recovered":28700672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1111695360,"data_stored":1111546309,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160843,"ondisk_log_size":160843,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986254,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":689040,"kb_used_data":368572,"kb_used_omap":4,"kb_used_meta":320443,"kb_avail":93682800,"statfs":{"total":96636764160,"available":95931187200,"internally_reserved":0,"allocated":377417728,"data_stored":377125495,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":328134314},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,0,3,3,0,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659199,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":593580,"kb_used_data":300892,"kb_used_omap":4,"kb_used_meta":292667,"kb_avail":93778260,"statfs":{"total":96636764160,"available":96028938240,"internally_reserved":0,"allocated":308113408,"data_stored":307784581,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":299691694},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,3],"upper_bound":32},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182482,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":818988,"kb_used_data":418076,"kb_used_omap":4,"kb_used_meta":400891,"kb_avail":93552852,"statfs":{"total":96636764160,"available":95798120448,"internally_reserved":0,"allocated":428109824,"data_stored":427817586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":410513066},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,4,4,7,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":426946560,"data_stored":426910430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":307732480,"data_stored":307659761,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":377016320,"data_stored":376976118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:04:32.641 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:04:33.221 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:04:33.223 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:04:33.245 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":256,"stamp":"2024-09-10T16:04:31.659482+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":475460361,"num_objects":116091,"num_object_clones":0,"num_object_copies":232182,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43305,"num_objects_misplaced":1073,"num_objects_unfound":0,"num_objects_dirty":116091,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":176019,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7013,"num_bytes_recovered":28700691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160846,"ondisk_log_size":160846,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2101608,"kb_used_data":1087540,"kb_used_omap":12,"kb_used_meta":1014003,"kb_avail":281013912,"statfs":{"total":289910292480,"available":287758245888,"internally_reserved":0,"allocated":1113640960,"data_stored":1112727662,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1038339074},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,4,10,7,7,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-20815872,"num_objects":-5082,"num_object_clones":0,"num_object_copies":-10164,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-257,"num_objects_unfound":0,"num_objects_dirty":-5082,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5082,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":257,"num_bytes_recovered":1052672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004286"},"pg_stats":[{"pgid":"2.f","version":"45'13498","reported_seq":13573,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:28.890013+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:04:28.890013+0000","last_peered":"2024-09-10T16:04:28.890013+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:04:28.890013+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"36'3400","ondisk_log_start":"36'3400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29384704,"num_objects":7174,"num_object_clones":0,"num_object_copies":14348,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7174,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13498,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2915},{"shards":"0,2","objects":1096}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'13703","reported_seq":13816,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:04:30.114991+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:04:30.114991+0000","last_peered":"2024-09-10T16:04:30.114991+0000","last_clean":"2024-09-10T16:04:30.114991+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:04:30.114991+0000","last_undegraded":"2024-09-10T16:04:30.114991+0000","last_fullsized":"2024-09-10T16:04:30.114991+0000","mapping_epoch":35,"log_start":"36'3700","ondisk_log_start":"36'3700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29250502,"num_objects":7143,"num_object_clones":0,"num_object_copies":14286,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7143,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":11808,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'13507","reported_seq":22186,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:04:26.630400+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:04:26.630400+0000","last_peered":"2024-09-10T16:04:26.630400+0000","last_clean":"2024-09-10T16:04:26.630400+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:04:26.630400+0000","last_undegraded":"2024-09-10T16:04:26.630400+0000","last_fullsized":"2024-09-10T16:04:26.630400+0000","mapping_epoch":44,"log_start":"36'3500","ondisk_log_start":"36'3500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29859864,"num_objects":7291,"num_object_clones":0,"num_object_copies":14582,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7291,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11511,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'13495","reported_seq":13576,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:04:30.145004+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:04:30.145004+0000","last_peered":"2024-09-10T16:04:30.145004+0000","last_clean":"2024-09-10T16:04:30.145004+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:04:30.145004+0000","last_undegraded":"2024-09-10T16:04:30.145004+0000","last_fullsized":"2024-09-10T16:04:30.145004+0000","mapping_epoch":35,"log_start":"36'3400","ondisk_log_start":"36'3400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29347840,"num_objects":7165,"num_object_clones":0,"num_object_copies":14330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7165,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13495,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'13693","reported_seq":13823,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.913327+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:04:28.913327+0000","last_peered":"2024-09-10T16:04:28.913327+0000","last_clean":"2024-09-10T16:04:28.913327+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:04:28.913327+0000","last_undegraded":"2024-09-10T16:04:28.913327+0000","last_fullsized":"2024-09-10T16:04:28.913327+0000","mapping_epoch":34,"log_start":"36'3600","ondisk_log_start":"36'3600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29483491,"num_objects":7199,"num_object_clones":0,"num_object_copies":14398,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7199,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10404,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'13790","reported_seq":13928,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:28.930978+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:04:28.930978+0000","last_peered":"2024-09-10T16:04:28.930978+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:04:28.930978+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"36'3700","ondisk_log_start":"36'3700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29782499,"num_objects":7272,"num_object_clones":0,"num_object_copies":14544,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7272,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10402,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2901},{"shards":"0,2","objects":1114}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'13686","reported_seq":13824,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:28.845772+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:04:28.845772+0000","last_peered":"2024-09-10T16:04:28.845772+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:04:28.845772+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"36'3600","ondisk_log_start":"36'3600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30294982,"num_objects":7398,"num_object_clones":0,"num_object_copies":14796,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7398,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":10185,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3161},{"shards":"0,2","objects":1095}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'13864","reported_seq":13994,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.994318+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:04:28.994318+0000","last_peered":"2024-09-10T16:04:28.994318+0000","last_clean":"2024-09-10T16:04:28.994318+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:04:28.994318+0000","last_undegraded":"2024-09-10T16:04:28.994318+0000","last_fullsized":"2024-09-10T16:04:28.994318+0000","mapping_epoch":34,"log_start":"36'3800","ondisk_log_start":"36'3800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30515200,"num_objects":7450,"num_object_clones":0,"num_object_copies":14900,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7450,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10711,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104261+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:02:08.104261+0000","last_peered":"2024-09-10T16:02:08.104261+0000","last_clean":"2024-09-10T16:02:08.104261+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:02:08.104261+0000","last_undegraded":"2024-09-10T16:02:08.104261+0000","last_fullsized":"2024-09-10T16:02:08.104261+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'13335","reported_seq":13471,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:30.179353+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:04:30.179353+0000","last_peered":"2024-09-10T16:04:30.179353+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:04:30.179353+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"36'3300","ondisk_log_start":"36'3300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29172195,"num_objects":7123,"num_object_clones":0,"num_object_copies":14246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7123,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11154,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1020},{"shards":"2","objects":3000}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103774+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:02:08.103774+0000","last_peered":"2024-09-10T16:02:08.103774+0000","last_clean":"2024-09-10T16:02:08.103774+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:02:08.103774+0000","last_undegraded":"2024-09-10T16:02:08.103774+0000","last_fullsized":"2024-09-10T16:02:08.103774+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'13636","reported_seq":13767,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:28.977466+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:04:28.977466+0000","last_peered":"2024-09-10T16:04:28.977466+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:04:28.977466+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"36'3600","ondisk_log_start":"36'3600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29794304,"num_objects":7274,"num_object_clones":0,"num_object_copies":14548,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7274,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10140,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3019},{"shards":"0,2","objects":1078}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":105,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:12.405385+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:02:12.405385+0000","last_peered":"2024-09-10T16:02:12.405385+0000","last_clean":"2024-09-10T16:02:12.405385+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:02:12.405385+0000","last_undegraded":"2024-09-10T16:02:12.405385+0000","last_fullsized":"2024-09-10T16:02:12.405385+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'13598","reported_seq":13730,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:30.122694+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:04:30.122694+0000","last_peered":"2024-09-10T16:04:30.122694+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:04:30.122694+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"36'3500","ondisk_log_start":"36'3500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29422051,"num_objects":7184,"num_object_clones":0,"num_object_copies":14368,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7184,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11278,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1099},{"shards":"2","objects":2880}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104617+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:02:08.104617+0000","last_peered":"2024-09-10T16:02:08.104617+0000","last_clean":"2024-09-10T16:02:08.104617+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:02:08.104617+0000","last_undegraded":"2024-09-10T16:02:08.104617+0000","last_fullsized":"2024-09-10T16:02:08.104617+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'13823","reported_seq":13951,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:30.180474+0000","last_change":"2024-09-10T15:59:16.174113+0000","last_active":"2024-09-10T16:04:30.180474+0000","last_peered":"2024-09-10T16:04:30.180474+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:04:30.180474+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"36'3800","ondisk_log_start":"36'3800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30093312,"num_objects":7347,"num_object_clones":0,"num_object_copies":14694,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4354,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7347,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11377,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1117},{"shards":"2","objects":2995}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'13605","reported_seq":13740,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:28.888826+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:04:28.888826+0000","last_peered":"2024-09-10T16:04:28.888826+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:04:28.888826+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"36'3600","ondisk_log_start":"36'3600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29626368,"num_objects":7233,"num_object_clones":0,"num_object_copies":14466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7233,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10236,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2929},{"shards":"0,2","objects":1119}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103316+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:02:08.103316+0000","last_peered":"2024-09-10T16:02:08.103316+0000","last_clean":"2024-09-10T16:02:08.103316+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:02:08.103316+0000","last_undegraded":"2024-09-10T16:02:08.103316+0000","last_fullsized":"2024-09-10T16:02:08.103316+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'13404","reported_seq":19026,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:28.996819+0000","last_change":"2024-09-10T16:03:29.605682+0000","last_active":"2024-09-10T16:04:28.996819+0000","last_peered":"2024-09-10T16:04:28.996819+0000","last_clean":"2024-09-10T15:59:14.115595+0000","last_became_active":"2024-09-10T15:59:16.244089+0000","last_became_peered":"2024-09-10T15:59:16.244089+0000","last_unstale":"2024-09-10T16:04:28.996819+0000","last_undegraded":"2024-09-10T16:03:29.603815+0000","last_fullsized":"2024-09-10T15:59:15.774437+0000","mapping_epoch":42,"log_start":"36'3400","ondisk_log_start":"36'3400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29466624,"num_objects":7194,"num_object_clones":0,"num_object_copies":14388,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":394,"num_objects_misplaced":1073,"num_objects_unfound":0,"num_objects_dirty":7194,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9794,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2739,"num_bytes_recovered":11206656,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5727},{"shards":"0,2","objects":1073}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103831+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:02:08.103831+0000","last_peered":"2024-09-10T16:02:08.103831+0000","last_clean":"2024-09-10T16:02:08.103831+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:02:08.103831+0000","last_undegraded":"2024-09-10T16:02:08.103831+0000","last_fullsized":"2024-09-10T16:02:08.103831+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'13812","reported_seq":13954,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:28.863045+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:04:28.863045+0000","last_peered":"2024-09-10T16:04:28.863045+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:04:28.863045+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"36'3800","ondisk_log_start":"36'3800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29638656,"num_objects":7236,"num_object_clones":0,"num_object_copies":14472,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7236,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10067,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2865},{"shards":"0,2","objects":1085}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104463+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:02:08.104463+0000","last_peered":"2024-09-10T16:02:08.104463+0000","last_clean":"2024-09-10T16:02:08.104463+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:02:08.104463+0000","last_undegraded":"2024-09-10T16:02:08.104463+0000","last_fullsized":"2024-09-10T16:02:08.104463+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104448+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:02:08.104448+0000","last_peered":"2024-09-10T16:02:08.104448+0000","last_clean":"2024-09-10T16:02:08.104448+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:02:08.104448+0000","last_undegraded":"2024-09-10T16:02:08.104448+0000","last_fullsized":"2024-09-10T16:02:08.104448+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'13794","reported_seq":13938,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:28.956477+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:04:28.956477+0000","last_peered":"2024-09-10T16:04:28.956477+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:04:28.956477+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"36'3700","ondisk_log_start":"36'3700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30327750,"num_objects":7406,"num_object_clones":0,"num_object_copies":14812,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7406,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9957,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3083},{"shards":"0,2","objects":1134}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":475460342,"num_objects":116089,"num_object_clones":0,"num_object_copies":232178,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43305,"num_objects_misplaced":1073,"num_objects_unfound":0,"num_objects_dirty":116089,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":176017,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7011,"num_bytes_recovered":28700672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1111695360,"data_stored":1111546309,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160843,"ondisk_log_size":160843,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986254,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":689040,"kb_used_data":368572,"kb_used_omap":4,"kb_used_meta":320443,"kb_avail":93682800,"statfs":{"total":96636764160,"available":95931187200,"internally_reserved":0,"allocated":377417728,"data_stored":377125495,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":328134314},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,0,3,3,0,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659199,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":593580,"kb_used_data":300892,"kb_used_omap":4,"kb_used_meta":292667,"kb_avail":93778260,"statfs":{"total":96636764160,"available":96028938240,"internally_reserved":0,"allocated":308113408,"data_stored":307784581,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":299691694},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,3],"upper_bound":32},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182482,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":818988,"kb_used_data":418076,"kb_used_omap":4,"kb_used_meta":400891,"kb_avail":93552852,"statfs":{"total":96636764160,"available":95798120448,"internally_reserved":0,"allocated":428109824,"data_stored":427817586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":410513066},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,4,4,7,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":426946560,"data_stored":426910430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":307732480,"data_stored":307659761,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":377016320,"data_stored":376976118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:04:33.247 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-10T16:04:33.247 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "45'13498", 'reported_seq': 13573, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:04:28.890013+0000', 'last_change': '2024-09-10T15:59:16.281558+0000', 'last_active': '2024-09-10T16:04:28.890013+0000', 'last_peered': '2024-09-10T16:04:28.890013+0000', 'last_clean': '2024-09-10T15:59:14.132456+0000', 'last_became_active': '2024-09-10T15:59:16.274407+0000', 'last_became_peered': '2024-09-10T15:59:16.274407+0000', 'last_unstale': '2024-09-10T16:04:28.890013+0000', 'last_undegraded': '2024-09-10T15:59:15.851279+0000', 'last_fullsized': '2024-09-10T15:59:15.774356+0000', 'mapping_epoch': 42, 'log_start': "36'3400", 'ondisk_log_start': "36'3400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'objects_scrubbed': 0, 'log_size': 10098, 'log_dups_size': 0, 'ondisk_log_size': 10098, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29384704, 'num_objects': 7174, 'num_object_clones': 0, 'num_object_copies': 14348, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4259, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7174, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13498, 'num_write_kb': 41344, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2915}, {'shards': '0,2', 'objects': 1096}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:04:33.247 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-10T16:04:33.247 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "45'13790", 'reported_seq': 13928, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:04:28.930978+0000', 'last_change': '2024-09-10T15:59:16.308264+0000', 'last_active': '2024-09-10T16:04:28.930978+0000', 'last_peered': '2024-09-10T16:04:28.930978+0000', 'last_clean': '2024-09-10T15:59:14.111551+0000', 'last_became_active': '2024-09-10T15:59:16.300040+0000', 'last_became_peered': '2024-09-10T15:59:16.300040+0000', 'last_unstale': '2024-09-10T16:04:28.930978+0000', 'last_undegraded': '2024-09-10T15:59:15.878234+0000', 'last_fullsized': '2024-09-10T15:59:15.769328+0000', 'mapping_epoch': 42, 'log_start': "36'3700", 'ondisk_log_start': "36'3700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3442", 'last_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'objects_scrubbed': 3384, 'log_size': 10090, 'log_dups_size': 0, 'ondisk_log_size': 10090, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1292, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29782499, 'num_objects': 7272, 'num_object_clones': 0, 'num_object_copies': 14544, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4372, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7272, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 10402, 'num_write_kb': 28580, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2901}, {'shards': '0,2', 'objects': 1114}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:04:33.247 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-10T16:04:33.247 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "45'13686", 'reported_seq': 13824, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:04:28.845772+0000', 'last_change': '2024-09-10T15:59:16.331899+0000', 'last_active': '2024-09-10T16:04:28.845772+0000', 'last_peered': '2024-09-10T16:04:28.845772+0000', 'last_clean': '2024-09-10T15:59:14.139436+0000', 'last_became_active': '2024-09-10T15:59:16.318715+0000', 'last_became_peered': '2024-09-10T15:59:16.318715+0000', 'last_unstale': '2024-09-10T16:04:28.845772+0000', 'last_undegraded': '2024-09-10T15:59:15.877524+0000', 'last_fullsized': '2024-09-10T15:59:15.769042+0000', 'mapping_epoch': 42, 'log_start': "36'3600", 'ondisk_log_start': "36'3600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3537", 'last_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'objects_scrubbed': 3497, 'log_size': 10086, 'log_dups_size': 0, 'ondisk_log_size': 10086, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1224, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30294982, 'num_objects': 7398, 'num_object_clones': 0, 'num_object_copies': 14796, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4237, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7398, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 10185, 'num_write_kb': 28172, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3161}, {'shards': '0,2', 'objects': 1095}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:04:33.247 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-10T16:04:33.247 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "45'13335", 'reported_seq': 13471, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:04:30.179353+0000', 'last_change': '2024-09-10T15:59:16.175680+0000', 'last_active': '2024-09-10T16:04:30.179353+0000', 'last_peered': '2024-09-10T16:04:30.179353+0000', 'last_clean': '2024-09-10T15:59:14.088187+0000', 'last_became_active': '2024-09-10T15:59:16.165810+0000', 'last_became_peered': '2024-09-10T15:59:16.165810+0000', 'last_unstale': '2024-09-10T16:04:30.179353+0000', 'last_undegraded': '2024-09-10T15:59:15.822098+0000', 'last_fullsized': '2024-09-10T15:59:15.769123+0000', 'mapping_epoch': 42, 'log_start': "36'3300", 'ondisk_log_start': "36'3300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2218", 'last_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'objects_scrubbed': 2175, 'log_size': 10035, 'log_dups_size': 0, 'ondisk_log_size': 10035, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 825, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29172195, 'num_objects': 7123, 'num_object_clones': 0, 'num_object_copies': 14246, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4124, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7123, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 11154, 'num_write_kb': 32204, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1020}, {'shards': '2', 'objects': 3000}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:04:33.247 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-10T16:04:33.247 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "45'13636", 'reported_seq': 13767, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:04:28.977466+0000', 'last_change': '2024-09-10T15:59:16.277742+0000', 'last_active': '2024-09-10T16:04:28.977466+0000', 'last_peered': '2024-09-10T16:04:28.977466+0000', 'last_clean': '2024-09-10T15:59:14.129081+0000', 'last_became_active': '2024-09-10T15:59:16.269835+0000', 'last_became_peered': '2024-09-10T15:59:16.269835+0000', 'last_unstale': '2024-09-10T16:04:28.977466+0000', 'last_undegraded': '2024-09-10T15:59:15.860857+0000', 'last_fullsized': '2024-09-10T15:59:15.771410+0000', 'mapping_epoch': 42, 'log_start': "36'3600", 'ondisk_log_start': "36'3600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3557", 'last_scrub_stamp': '2024-09-10T15:59:02.493678+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:02.493678+0000', 'objects_scrubbed': 3488, 'log_size': 10036, 'log_dups_size': 0, 'ondisk_log_size': 10036, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1472, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29794304, 'num_objects': 7274, 'num_object_clones': 0, 'num_object_copies': 14548, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4257, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7274, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10140, 'num_write_kb': 27852, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3019}, {'shards': '0,2', 'objects': 1078}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:04:33.248 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-10T16:04:33.248 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "45'13598", 'reported_seq': 13730, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:04:30.122694+0000', 'last_change': '2024-09-10T15:59:16.284311+0000', 'last_active': '2024-09-10T16:04:30.122694+0000', 'last_peered': '2024-09-10T16:04:30.122694+0000', 'last_clean': '2024-09-10T15:59:14.116041+0000', 'last_became_active': '2024-09-10T15:59:16.274543+0000', 'last_became_peered': '2024-09-10T15:59:16.274543+0000', 'last_unstale': '2024-09-10T16:04:30.122694+0000', 'last_undegraded': '2024-09-10T15:59:15.812484+0000', 'last_fullsized': '2024-09-10T15:59:15.770885+0000', 'mapping_epoch': 42, 'log_start': "36'3500", 'ondisk_log_start': "36'3500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2356", 'last_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'objects_scrubbed': 2316, 'log_size': 10098, 'log_dups_size': 0, 'ondisk_log_size': 10098, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 783, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29422051, 'num_objects': 7184, 'num_object_clones': 0, 'num_object_copies': 14368, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7184, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 11278, 'num_write_kb': 32292, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1099}, {'shards': '2', 'objects': 2880}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:04:33.248 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-10T16:04:33.248 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "45'13823", 'reported_seq': 13951, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:04:30.180474+0000', 'last_change': '2024-09-10T15:59:16.174113+0000', 'last_active': '2024-09-10T16:04:30.180474+0000', 'last_peered': '2024-09-10T16:04:30.180474+0000', 'last_clean': '2024-09-10T15:59:14.110654+0000', 'last_became_active': '2024-09-10T15:59:16.164645+0000', 'last_became_peered': '2024-09-10T15:59:16.164645+0000', 'last_unstale': '2024-09-10T16:04:30.180474+0000', 'last_undegraded': '2024-09-10T15:59:15.803593+0000', 'last_fullsized': '2024-09-10T15:59:15.769083+0000', 'mapping_epoch': 42, 'log_start': "36'3800", 'ondisk_log_start': "36'3800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2486", 'last_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'objects_scrubbed': 2440, 'log_size': 10023, 'log_dups_size': 0, 'ondisk_log_size': 10023, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 792, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30093312, 'num_objects': 7347, 'num_object_clones': 0, 'num_object_copies': 14694, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4354, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7347, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11377, 'num_write_kb': 32568, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1117}, {'shards': '2', 'objects': 2995}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:04:33.248 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-10T16:04:33.248 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "45'13605", 'reported_seq': 13740, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:04:28.888826+0000', 'last_change': '2024-09-10T15:59:16.315641+0000', 'last_active': '2024-09-10T16:04:28.888826+0000', 'last_peered': '2024-09-10T16:04:28.888826+0000', 'last_clean': '2024-09-10T15:59:14.117001+0000', 'last_became_active': '2024-09-10T15:59:16.302245+0000', 'last_became_peered': '2024-09-10T15:59:16.302245+0000', 'last_unstale': '2024-09-10T16:04:28.888826+0000', 'last_undegraded': '2024-09-10T15:59:15.845012+0000', 'last_fullsized': '2024-09-10T15:59:15.770337+0000', 'mapping_epoch': 42, 'log_start': "36'3600", 'ondisk_log_start': "36'3600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3404", 'last_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'objects_scrubbed': 3367, 'log_size': 10005, 'log_dups_size': 0, 'ondisk_log_size': 10005, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 983, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29626368, 'num_objects': 7233, 'num_object_clones': 0, 'num_object_copies': 14466, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7233, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10236, 'num_write_kb': 28204, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2929}, {'shards': '0,2', 'objects': 1119}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:04:33.248 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-10T16:04:33.248 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "45'13404", 'reported_seq': 19026, 'reported_epoch': 45, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:04:28.996819+0000', 'last_change': '2024-09-10T16:03:29.605682+0000', 'last_active': '2024-09-10T16:04:28.996819+0000', 'last_peered': '2024-09-10T16:04:28.996819+0000', 'last_clean': '2024-09-10T15:59:14.115595+0000', 'last_became_active': '2024-09-10T15:59:16.244089+0000', 'last_became_peered': '2024-09-10T15:59:16.244089+0000', 'last_unstale': '2024-09-10T16:04:28.996819+0000', 'last_undegraded': '2024-09-10T16:03:29.603815+0000', 'last_fullsized': '2024-09-10T15:59:15.774437+0000', 'mapping_epoch': 42, 'log_start': "36'3400", 'ondisk_log_start': "36'3400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3653", 'last_scrub_stamp': '2024-09-10T15:59:05.298468+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:05.298468+0000', 'objects_scrubbed': 3602, 'log_size': 10004, 'log_dups_size': 0, 'ondisk_log_size': 10004, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1289, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29466624, 'num_objects': 7194, 'num_object_clones': 0, 'num_object_copies': 14388, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 394, 'num_objects_misplaced': 1073, 'num_objects_unfound': 0, 'num_objects_dirty': 7194, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9794, 'num_write_kb': 26772, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2739, 'num_bytes_recovered': 11206656, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5727}, {'shards': '0,2', 'objects': 1073}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:04:33.248 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-10T16:04:33.248 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "45'13812", 'reported_seq': 13954, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:04:28.863045+0000', 'last_change': '2024-09-10T15:59:16.311291+0000', 'last_active': '2024-09-10T16:04:28.863045+0000', 'last_peered': '2024-09-10T16:04:28.863045+0000', 'last_clean': '2024-09-10T15:59:14.116363+0000', 'last_became_active': '2024-09-10T15:59:16.301244+0000', 'last_became_peered': '2024-09-10T15:59:16.301244+0000', 'last_unstale': '2024-09-10T16:04:28.863045+0000', 'last_undegraded': '2024-09-10T15:59:15.883434+0000', 'last_fullsized': '2024-09-10T15:59:15.772756+0000', 'mapping_epoch': 42, 'log_start': "36'3800", 'ondisk_log_start': "36'3800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3789", 'last_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'objects_scrubbed': 3741, 'log_size': 10012, 'log_dups_size': 0, 'ondisk_log_size': 10012, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1440, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29638656, 'num_objects': 7236, 'num_object_clones': 0, 'num_object_copies': 14472, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4373, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7236, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10067, 'num_write_kb': 27124, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2865}, {'shards': '0,2', 'objects': 1085}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:04:33.248 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-10T16:04:33.249 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "45'13794", 'reported_seq': 13938, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:04:28.956477+0000', 'last_change': '2024-09-10T15:59:16.313409+0000', 'last_active': '2024-09-10T16:04:28.956477+0000', 'last_peered': '2024-09-10T16:04:28.956477+0000', 'last_clean': '2024-09-10T15:59:14.129780+0000', 'last_became_active': '2024-09-10T15:59:16.300932+0000', 'last_became_peered': '2024-09-10T15:59:16.300932+0000', 'last_unstale': '2024-09-10T16:04:28.956477+0000', 'last_undegraded': '2024-09-10T15:59:15.887823+0000', 'last_fullsized': '2024-09-10T15:59:15.768513+0000', 'mapping_epoch': 42, 'log_start': "36'3700", 'ondisk_log_start': "36'3700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3880", 'last_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'objects_scrubbed': 3827, 'log_size': 10094, 'log_dups_size': 0, 'ondisk_log_size': 10094, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1397, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30327750, 'num_objects': 7406, 'num_object_clones': 0, 'num_object_copies': 14812, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4325, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7406, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9957, 'num_write_kb': 27072, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3083}, {'shards': '0,2', 'objects': 1134}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:04:33.249 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T16:04:33.977 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:04:33.977 INFO:teuthology.orchestra.run.smithi115.stdout:{"fsid":"7effee29-df05-4351-9741-ee203564ed81","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 43305/232182 objects degraded (18.651%), 11 pgs degraded, 11 pgs undersized","count":22},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":437,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983954,"num_in_osds":3,"osd_in_since":1725983947,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":116091,"data_bytes":475460361,"bytes_used":2152046592,"bytes_avail":287758245888,"bytes_total":289910292480,"degraded_objects":43305,"degraded_total":232182,"degraded_ratio":0.18651316639532781,"misplaced_objects":1073,"misplaced_total":232182,"misplaced_ratio":0.0046213746112963106,"recovering_objects_per_sec":21,"recovering_bytes_per_sec":87691,"recovering_keys_per_sec":0,"num_objects_recovered":257,"num_bytes_recovered":1052672,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":423},"fsmap":{"epoch":1,"btime":"2024-09-10T15:57:14:967120+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T16:00:13.560776+0000","services":{}},"progress_events":{}} 2024-09-10T16:04:33.997 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-10T16:04:33.997 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:04:34.552 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:04:34.553 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:04:34.572 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":257,"stamp":"2024-09-10T16:04:33.660224+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":474813193,"num_objects":115933,"num_object_clones":0,"num_object_copies":231866,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43305,"num_objects_misplaced":1073,"num_objects_unfound":0,"num_objects_dirty":115933,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":176177,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7013,"num_bytes_recovered":28700691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160904,"ondisk_log_size":160904,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2105196,"kb_used_data":1086964,"kb_used_omap":12,"kb_used_meta":1018163,"kb_avail":281010324,"statfs":{"total":289910292480,"available":287754571776,"internally_reserved":0,"allocated":1113051136,"data_stored":1112137838,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1042598914},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,0,2,0,7,10,14,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18669568,"num_objects":-4558,"num_object_clones":0,"num_object_copies":-9116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-257,"num_objects_unfound":0,"num_objects_dirty":-4558,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4558,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":257,"num_bytes_recovered":1052672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004373"},"pg_stats":[{"pgid":"2.f","version":"45'13498","reported_seq":13573,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:28.890013+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:04:28.890013+0000","last_peered":"2024-09-10T16:04:28.890013+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:04:28.890013+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"36'3400","ondisk_log_start":"36'3400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29384704,"num_objects":7174,"num_object_clones":0,"num_object_copies":14348,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7174,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13498,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2915},{"shards":"0,2","objects":1096}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'13703","reported_seq":13816,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:04:30.114991+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:04:30.114991+0000","last_peered":"2024-09-10T16:04:30.114991+0000","last_clean":"2024-09-10T16:04:30.114991+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:04:30.114991+0000","last_undegraded":"2024-09-10T16:04:30.114991+0000","last_fullsized":"2024-09-10T16:04:30.114991+0000","mapping_epoch":35,"log_start":"36'3700","ondisk_log_start":"36'3700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29250502,"num_objects":7143,"num_object_clones":0,"num_object_copies":14286,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7143,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":11808,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'13665","reported_seq":22342,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:04:31.785561+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:04:31.785561+0000","last_peered":"2024-09-10T16:04:31.785561+0000","last_clean":"2024-09-10T16:04:31.785561+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:04:31.785561+0000","last_undegraded":"2024-09-10T16:04:31.785561+0000","last_fullsized":"2024-09-10T16:04:31.785561+0000","mapping_epoch":44,"log_start":"36'3600","ondisk_log_start":"36'3600","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29212696,"num_objects":7133,"num_object_clones":0,"num_object_copies":14266,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7133,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11669,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'13495","reported_seq":13576,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:04:30.145004+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:04:30.145004+0000","last_peered":"2024-09-10T16:04:30.145004+0000","last_clean":"2024-09-10T16:04:30.145004+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:04:30.145004+0000","last_undegraded":"2024-09-10T16:04:30.145004+0000","last_fullsized":"2024-09-10T16:04:30.145004+0000","mapping_epoch":35,"log_start":"36'3400","ondisk_log_start":"36'3400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29347840,"num_objects":7165,"num_object_clones":0,"num_object_copies":14330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7165,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13495,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'13693","reported_seq":13823,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.913327+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:04:28.913327+0000","last_peered":"2024-09-10T16:04:28.913327+0000","last_clean":"2024-09-10T16:04:28.913327+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:04:28.913327+0000","last_undegraded":"2024-09-10T16:04:28.913327+0000","last_fullsized":"2024-09-10T16:04:28.913327+0000","mapping_epoch":34,"log_start":"36'3600","ondisk_log_start":"36'3600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29483491,"num_objects":7199,"num_object_clones":0,"num_object_copies":14398,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7199,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10404,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'13790","reported_seq":13928,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:28.930978+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:04:28.930978+0000","last_peered":"2024-09-10T16:04:28.930978+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:04:28.930978+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"36'3700","ondisk_log_start":"36'3700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29782499,"num_objects":7272,"num_object_clones":0,"num_object_copies":14544,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7272,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10402,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2901},{"shards":"0,2","objects":1114}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'13686","reported_seq":13824,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:28.845772+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:04:28.845772+0000","last_peered":"2024-09-10T16:04:28.845772+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:04:28.845772+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"36'3600","ondisk_log_start":"36'3600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30294982,"num_objects":7398,"num_object_clones":0,"num_object_copies":14796,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7398,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":10185,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3161},{"shards":"0,2","objects":1095}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'13864","reported_seq":13994,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.994318+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:04:28.994318+0000","last_peered":"2024-09-10T16:04:28.994318+0000","last_clean":"2024-09-10T16:04:28.994318+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:04:28.994318+0000","last_undegraded":"2024-09-10T16:04:28.994318+0000","last_fullsized":"2024-09-10T16:04:28.994318+0000","mapping_epoch":34,"log_start":"36'3800","ondisk_log_start":"36'3800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30515200,"num_objects":7450,"num_object_clones":0,"num_object_copies":14900,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7450,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10711,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104261+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:02:08.104261+0000","last_peered":"2024-09-10T16:02:08.104261+0000","last_clean":"2024-09-10T16:02:08.104261+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:02:08.104261+0000","last_undegraded":"2024-09-10T16:02:08.104261+0000","last_fullsized":"2024-09-10T16:02:08.104261+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'13335","reported_seq":13471,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:30.179353+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:04:30.179353+0000","last_peered":"2024-09-10T16:04:30.179353+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:04:30.179353+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"36'3300","ondisk_log_start":"36'3300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29172195,"num_objects":7123,"num_object_clones":0,"num_object_copies":14246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7123,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11154,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1020},{"shards":"2","objects":3000}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103774+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:02:08.103774+0000","last_peered":"2024-09-10T16:02:08.103774+0000","last_clean":"2024-09-10T16:02:08.103774+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:02:08.103774+0000","last_undegraded":"2024-09-10T16:02:08.103774+0000","last_fullsized":"2024-09-10T16:02:08.103774+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'13636","reported_seq":13767,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:28.977466+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:04:28.977466+0000","last_peered":"2024-09-10T16:04:28.977466+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:04:28.977466+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"36'3600","ondisk_log_start":"36'3600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29794304,"num_objects":7274,"num_object_clones":0,"num_object_copies":14548,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7274,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10140,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3019},{"shards":"0,2","objects":1078}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":105,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:12.405385+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:02:12.405385+0000","last_peered":"2024-09-10T16:02:12.405385+0000","last_clean":"2024-09-10T16:02:12.405385+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:02:12.405385+0000","last_undegraded":"2024-09-10T16:02:12.405385+0000","last_fullsized":"2024-09-10T16:02:12.405385+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'13598","reported_seq":13730,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:30.122694+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:04:30.122694+0000","last_peered":"2024-09-10T16:04:30.122694+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:04:30.122694+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"36'3500","ondisk_log_start":"36'3500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29422051,"num_objects":7184,"num_object_clones":0,"num_object_copies":14368,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7184,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11278,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1099},{"shards":"2","objects":2880}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104617+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:02:08.104617+0000","last_peered":"2024-09-10T16:02:08.104617+0000","last_clean":"2024-09-10T16:02:08.104617+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:02:08.104617+0000","last_undegraded":"2024-09-10T16:02:08.104617+0000","last_fullsized":"2024-09-10T16:02:08.104617+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'13823","reported_seq":13951,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:30.180474+0000","last_change":"2024-09-10T15:59:16.174113+0000","last_active":"2024-09-10T16:04:30.180474+0000","last_peered":"2024-09-10T16:04:30.180474+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:04:30.180474+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"36'3800","ondisk_log_start":"36'3800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30093312,"num_objects":7347,"num_object_clones":0,"num_object_copies":14694,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4354,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7347,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11377,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1117},{"shards":"2","objects":2995}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'13605","reported_seq":13740,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:28.888826+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:04:28.888826+0000","last_peered":"2024-09-10T16:04:28.888826+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:04:28.888826+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"36'3600","ondisk_log_start":"36'3600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29626368,"num_objects":7233,"num_object_clones":0,"num_object_copies":14466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7233,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10236,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2929},{"shards":"0,2","objects":1119}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103316+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:02:08.103316+0000","last_peered":"2024-09-10T16:02:08.103316+0000","last_clean":"2024-09-10T16:02:08.103316+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:02:08.103316+0000","last_undegraded":"2024-09-10T16:02:08.103316+0000","last_fullsized":"2024-09-10T16:02:08.103316+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'13404","reported_seq":19026,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:28.996819+0000","last_change":"2024-09-10T16:03:29.605682+0000","last_active":"2024-09-10T16:04:28.996819+0000","last_peered":"2024-09-10T16:04:28.996819+0000","last_clean":"2024-09-10T15:59:14.115595+0000","last_became_active":"2024-09-10T15:59:16.244089+0000","last_became_peered":"2024-09-10T15:59:16.244089+0000","last_unstale":"2024-09-10T16:04:28.996819+0000","last_undegraded":"2024-09-10T16:03:29.603815+0000","last_fullsized":"2024-09-10T15:59:15.774437+0000","mapping_epoch":42,"log_start":"36'3400","ondisk_log_start":"36'3400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29466624,"num_objects":7194,"num_object_clones":0,"num_object_copies":14388,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":394,"num_objects_misplaced":1073,"num_objects_unfound":0,"num_objects_dirty":7194,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9794,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2739,"num_bytes_recovered":11206656,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5727},{"shards":"0,2","objects":1073}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103831+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:02:08.103831+0000","last_peered":"2024-09-10T16:02:08.103831+0000","last_clean":"2024-09-10T16:02:08.103831+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:02:08.103831+0000","last_undegraded":"2024-09-10T16:02:08.103831+0000","last_fullsized":"2024-09-10T16:02:08.103831+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'13812","reported_seq":13954,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:28.863045+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:04:28.863045+0000","last_peered":"2024-09-10T16:04:28.863045+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:04:28.863045+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"36'3800","ondisk_log_start":"36'3800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29638656,"num_objects":7236,"num_object_clones":0,"num_object_copies":14472,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7236,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10067,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2865},{"shards":"0,2","objects":1085}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104463+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:02:08.104463+0000","last_peered":"2024-09-10T16:02:08.104463+0000","last_clean":"2024-09-10T16:02:08.104463+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:02:08.104463+0000","last_undegraded":"2024-09-10T16:02:08.104463+0000","last_fullsized":"2024-09-10T16:02:08.104463+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104448+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:02:08.104448+0000","last_peered":"2024-09-10T16:02:08.104448+0000","last_clean":"2024-09-10T16:02:08.104448+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:02:08.104448+0000","last_undegraded":"2024-09-10T16:02:08.104448+0000","last_fullsized":"2024-09-10T16:02:08.104448+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'13794","reported_seq":13938,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:28.956477+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:04:28.956477+0000","last_peered":"2024-09-10T16:04:28.956477+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:04:28.956477+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"36'3700","ondisk_log_start":"36'3700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30327750,"num_objects":7406,"num_object_clones":0,"num_object_copies":14812,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7406,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9957,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3083},{"shards":"0,2","objects":1134}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":474813174,"num_objects":115931,"num_object_clones":0,"num_object_copies":231862,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43305,"num_objects_misplaced":1073,"num_objects_unfound":0,"num_objects_dirty":115931,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":176175,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7011,"num_bytes_recovered":28700672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1110929408,"data_stored":1110780357,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160901,"ondisk_log_size":160901,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986254,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":689040,"kb_used_data":368572,"kb_used_omap":4,"kb_used_meta":320443,"kb_avail":93682800,"statfs":{"total":96636764160,"available":95931187200,"internally_reserved":0,"allocated":377417728,"data_stored":377125495,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":328134314},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,0,3,3,0,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659200,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":597168,"kb_used_data":300316,"kb_used_omap":4,"kb_used_meta":296827,"kb_avail":93774672,"statfs":{"total":96636764160,"available":96025264128,"internally_reserved":0,"allocated":307523584,"data_stored":307194757,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":303951534},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,1,0,0,3,7],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182482,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":818988,"kb_used_data":418076,"kb_used_omap":4,"kb_used_meta":400891,"kb_avail":93552852,"statfs":{"total":96636764160,"available":95798120448,"internally_reserved":0,"allocated":428109824,"data_stored":427817586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":410513066},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,4,4,7,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":426946560,"data_stored":426910430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":306966528,"data_stored":306893809,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":377016320,"data_stored":376976118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:04:37.575 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:04:38.163 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:04:38.163 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:04:38.189 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":259,"stamp":"2024-09-10T16:04:37.661282+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":465580809,"num_objects":113679,"num_object_clones":0,"num_object_copies":227358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43305,"num_objects_misplaced":944,"num_objects_unfound":0,"num_objects_dirty":113679,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":178431,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7142,"num_bytes_recovered":29229075,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160658,"ondisk_log_size":160658,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2090408,"kb_used_data":1074932,"kb_used_omap":12,"kb_used_meta":1015411,"kb_avail":281025112,"statfs":{"total":289910292480,"available":287769714688,"internally_reserved":0,"allocated":1100730368,"data_stored":1099817070,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1039780866},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,5,6,9,10,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18870272,"num_objects":-4607,"num_object_clones":0,"num_object_copies":-9214,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-259,"num_objects_unfound":0,"num_objects_dirty":-4607,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4607,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":259,"num_bytes_recovered":1060864,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003895"},"pg_stats":[{"pgid":"2.f","version":"45'13630","reported_seq":13705,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:34.009664+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:04:34.009664+0000","last_peered":"2024-09-10T16:04:34.009664+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:04:34.009664+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"36'3600","ondisk_log_start":"36'3600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28844032,"num_objects":7042,"num_object_clones":0,"num_object_copies":14084,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7042,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13630,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2783},{"shards":"0,2","objects":964}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'13839","reported_seq":13952,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:04:35.164029+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:04:35.164029+0000","last_peered":"2024-09-10T16:04:35.164029+0000","last_clean":"2024-09-10T16:04:35.164029+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:04:35.164029+0000","last_undegraded":"2024-09-10T16:04:35.164029+0000","last_fullsized":"2024-09-10T16:04:35.164029+0000","mapping_epoch":35,"log_start":"36'3800","ondisk_log_start":"36'3800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28693446,"num_objects":7007,"num_object_clones":0,"num_object_copies":14014,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7007,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":11944,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'13810","reported_seq":22489,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:04:36.847834+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:04:36.847834+0000","last_peered":"2024-09-10T16:04:36.847834+0000","last_clean":"2024-09-10T16:04:36.847834+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:04:36.847834+0000","last_undegraded":"2024-09-10T16:04:36.847834+0000","last_fullsized":"2024-09-10T16:04:36.847834+0000","mapping_epoch":44,"log_start":"36'3800","ondisk_log_start":"36'3800","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28618776,"num_objects":6988,"num_object_clones":0,"num_object_copies":13976,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6988,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11814,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'13638","reported_seq":13719,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:04:35.200471+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:04:35.200471+0000","last_peered":"2024-09-10T16:04:35.200471+0000","last_clean":"2024-09-10T16:04:35.200471+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:04:35.200471+0000","last_undegraded":"2024-09-10T16:04:35.200471+0000","last_fullsized":"2024-09-10T16:04:35.200471+0000","mapping_epoch":35,"log_start":"36'3600","ondisk_log_start":"36'3600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28762112,"num_objects":7022,"num_object_clones":0,"num_object_copies":14044,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7022,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13638,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'13829","reported_seq":13959,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:04:33.921126+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:04:33.921126+0000","last_peered":"2024-09-10T16:04:33.921126+0000","last_clean":"2024-09-10T16:04:33.921126+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:04:33.921126+0000","last_undegraded":"2024-09-10T16:04:33.921126+0000","last_fullsized":"2024-09-10T16:04:33.921126+0000","mapping_epoch":34,"log_start":"36'3800","ondisk_log_start":"36'3800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28926435,"num_objects":7063,"num_object_clones":0,"num_object_copies":14126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7063,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10540,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'13934","reported_seq":14072,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:33.914285+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:04:33.914285+0000","last_peered":"2024-09-10T16:04:33.914285+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:04:33.914285+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"36'3900","ondisk_log_start":"36'3900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29192675,"num_objects":7128,"num_object_clones":0,"num_object_copies":14256,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7128,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10546,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2757},{"shards":"0,2","objects":970}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'13812","reported_seq":13950,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:33.838307+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:04:33.838307+0000","last_peered":"2024-09-10T16:04:33.838307+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:04:33.838307+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"36'3800","ondisk_log_start":"36'3800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29778886,"num_objects":7272,"num_object_clones":0,"num_object_copies":14544,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7272,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":10311,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3035},{"shards":"0,2","objects":969}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'14031","reported_seq":14161,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:04:33.979534+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:04:33.979534+0000","last_peered":"2024-09-10T16:04:33.979534+0000","last_clean":"2024-09-10T16:04:33.979534+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:04:33.979534+0000","last_undegraded":"2024-09-10T16:04:33.979534+0000","last_fullsized":"2024-09-10T16:04:33.979534+0000","mapping_epoch":34,"log_start":"39'4000","ondisk_log_start":"39'4000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29831168,"num_objects":7283,"num_object_clones":0,"num_object_copies":14566,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7283,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10878,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104261+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:02:08.104261+0000","last_peered":"2024-09-10T16:02:08.104261+0000","last_clean":"2024-09-10T16:02:08.104261+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:02:08.104261+0000","last_undegraded":"2024-09-10T16:02:08.104261+0000","last_fullsized":"2024-09-10T16:02:08.104261+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'13459","reported_seq":13599,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:35.146978+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:04:35.146978+0000","last_peered":"2024-09-10T16:04:35.146978+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:04:35.146978+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"36'3400","ondisk_log_start":"36'3400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28664291,"num_objects":6999,"num_object_clones":0,"num_object_copies":13998,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6999,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11278,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":896},{"shards":"2","objects":2876}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":95,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:29.860108+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:02:29.860108+0000","last_peered":"2024-09-10T16:02:29.860108+0000","last_clean":"2024-09-10T16:02:29.860108+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:02:29.860108+0000","last_undegraded":"2024-09-10T16:02:29.860108+0000","last_fullsized":"2024-09-10T16:02:29.860108+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'13780","reported_seq":13911,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:33.884008+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:04:33.884008+0000","last_peered":"2024-09-10T16:04:33.884008+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:04:33.884008+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"36'3700","ondisk_log_start":"36'3700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29204480,"num_objects":7130,"num_object_clones":0,"num_object_copies":14260,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7130,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10284,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2875},{"shards":"0,2","objects":934}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":105,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:12.405385+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:02:12.405385+0000","last_peered":"2024-09-10T16:02:12.405385+0000","last_clean":"2024-09-10T16:02:12.405385+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:02:12.405385+0000","last_undegraded":"2024-09-10T16:02:12.405385+0000","last_fullsized":"2024-09-10T16:02:12.405385+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'13744","reported_seq":13877,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:35.115778+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:04:35.115778+0000","last_peered":"2024-09-10T16:04:35.115778+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:04:35.115778+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"36'3700","ondisk_log_start":"36'3700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28824035,"num_objects":7038,"num_object_clones":0,"num_object_copies":14076,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7038,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11424,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":953},{"shards":"2","objects":2734}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104617+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:02:08.104617+0000","last_peered":"2024-09-10T16:02:08.104617+0000","last_clean":"2024-09-10T16:02:08.104617+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:02:08.104617+0000","last_undegraded":"2024-09-10T16:02:08.104617+0000","last_fullsized":"2024-09-10T16:02:08.104617+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'13991","reported_seq":14121,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:35.115089+0000","last_change":"2024-09-10T15:59:16.174113+0000","last_active":"2024-09-10T16:04:35.115089+0000","last_peered":"2024-09-10T16:04:35.115089+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:04:35.115089+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"36'3900","ondisk_log_start":"36'3900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29405184,"num_objects":7179,"num_object_clones":0,"num_object_copies":14358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4354,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7179,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11545,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":949},{"shards":"2","objects":2827}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'13734","reported_seq":13869,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:33.829950+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:04:33.829950+0000","last_peered":"2024-09-10T16:04:33.829950+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:04:33.829950+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"36'3700","ondisk_log_start":"36'3700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29097984,"num_objects":7104,"num_object_clones":0,"num_object_copies":14208,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7104,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10365,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2800},{"shards":"0,2","objects":990}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103316+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:02:08.103316+0000","last_peered":"2024-09-10T16:02:08.103316+0000","last_clean":"2024-09-10T16:02:08.103316+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:02:08.103316+0000","last_undegraded":"2024-09-10T16:02:08.103316+0000","last_fullsized":"2024-09-10T16:02:08.103316+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'13536","reported_seq":19416,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:34.024398+0000","last_change":"2024-09-10T16:03:29.605682+0000","last_active":"2024-09-10T16:04:34.024398+0000","last_peered":"2024-09-10T16:04:34.024398+0000","last_clean":"2024-09-10T15:59:14.115595+0000","last_became_active":"2024-09-10T15:59:16.244089+0000","last_became_peered":"2024-09-10T15:59:16.244089+0000","last_unstale":"2024-09-10T16:04:34.024398+0000","last_undegraded":"2024-09-10T16:03:29.603815+0000","last_fullsized":"2024-09-10T15:59:15.774437+0000","mapping_epoch":42,"log_start":"36'3500","ondisk_log_start":"36'3500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28925952,"num_objects":7062,"num_object_clones":0,"num_object_copies":14124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":394,"num_objects_misplaced":944,"num_objects_unfound":0,"num_objects_dirty":7062,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9926,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2868,"num_bytes_recovered":11735040,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5724},{"shards":"0,2","objects":944}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":95,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:29.877831+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:02:29.877831+0000","last_peered":"2024-09-10T16:02:29.877831+0000","last_clean":"2024-09-10T16:02:29.877831+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:02:29.877831+0000","last_undegraded":"2024-09-10T16:02:29.877831+0000","last_fullsized":"2024-09-10T16:02:29.877831+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'13945","reported_seq":14087,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:34.028596+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:04:34.028596+0000","last_peered":"2024-09-10T16:04:34.028596+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:04:34.028596+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"36'3900","ondisk_log_start":"36'3900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29093888,"num_objects":7103,"num_object_clones":0,"num_object_copies":14206,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7103,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10200,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2732},{"shards":"0,2","objects":952}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104463+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:02:08.104463+0000","last_peered":"2024-09-10T16:02:08.104463+0000","last_clean":"2024-09-10T16:02:08.104463+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:02:08.104463+0000","last_undegraded":"2024-09-10T16:02:08.104463+0000","last_fullsized":"2024-09-10T16:02:08.104463+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104448+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:02:08.104448+0000","last_peered":"2024-09-10T16:02:08.104448+0000","last_clean":"2024-09-10T16:02:08.104448+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:02:08.104448+0000","last_undegraded":"2024-09-10T16:02:08.104448+0000","last_fullsized":"2024-09-10T16:02:08.104448+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'13943","reported_seq":14088,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:33.942877+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:04:33.942877+0000","last_peered":"2024-09-10T16:04:33.942877+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:04:33.942877+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"36'3900","ondisk_log_start":"36'3900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29717446,"num_objects":7257,"num_object_clones":0,"num_object_copies":14514,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7257,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10106,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2934},{"shards":"0,2","objects":985}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":465580790,"num_objects":113677,"num_object_clones":0,"num_object_copies":227354,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43305,"num_objects_misplaced":944,"num_objects_unfound":0,"num_objects_dirty":113677,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":178429,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7140,"num_bytes_recovered":29229056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1098752000,"data_stored":1098602949,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160655,"ondisk_log_size":160655,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986255,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":688588,"kb_used_data":363964,"kb_used_omap":4,"kb_used_meta":324603,"kb_avail":93683252,"statfs":{"total":96636764160,"available":95931650048,"internally_reserved":0,"allocated":372699136,"data_stored":372406903,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":332394154},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,2,1,1,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659201,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":601968,"kb_used_data":299552,"kb_used_omap":4,"kb_used_meta":302395,"kb_avail":93769872,"statfs":{"total":96636764160,"available":96020348928,"internally_reserved":0,"allocated":306741248,"data_stored":306412421,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":309653166},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,3,5,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182483,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":799852,"kb_used_data":411416,"kb_used_omap":4,"kb_used_meta":388411,"kb_avail":93571988,"statfs":{"total":96636764160,"available":95817715712,"internally_reserved":0,"allocated":421289984,"data_stored":420997746,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":397733546},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,3,1,3,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":420179968,"data_stored":420143838,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":306360320,"data_stored":306287601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":372211712,"data_stored":372171510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:04:38.191 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:04:38.754 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:04:38.755 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:04:38.775 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":259,"stamp":"2024-09-10T16:04:37.661282+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":465580809,"num_objects":113679,"num_object_clones":0,"num_object_copies":227358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43305,"num_objects_misplaced":944,"num_objects_unfound":0,"num_objects_dirty":113679,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":178431,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7142,"num_bytes_recovered":29229075,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160658,"ondisk_log_size":160658,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2090408,"kb_used_data":1074932,"kb_used_omap":12,"kb_used_meta":1015411,"kb_avail":281025112,"statfs":{"total":289910292480,"available":287769714688,"internally_reserved":0,"allocated":1100730368,"data_stored":1099817070,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1039780866},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,5,6,9,10,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18870272,"num_objects":-4607,"num_object_clones":0,"num_object_copies":-9214,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-259,"num_objects_unfound":0,"num_objects_dirty":-4607,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4607,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":259,"num_bytes_recovered":1060864,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003895"},"pg_stats":[{"pgid":"2.f","version":"45'13630","reported_seq":13705,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:34.009664+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:04:34.009664+0000","last_peered":"2024-09-10T16:04:34.009664+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:04:34.009664+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"36'3600","ondisk_log_start":"36'3600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28844032,"num_objects":7042,"num_object_clones":0,"num_object_copies":14084,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7042,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13630,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2783},{"shards":"0,2","objects":964}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'13839","reported_seq":13952,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:04:35.164029+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:04:35.164029+0000","last_peered":"2024-09-10T16:04:35.164029+0000","last_clean":"2024-09-10T16:04:35.164029+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:04:35.164029+0000","last_undegraded":"2024-09-10T16:04:35.164029+0000","last_fullsized":"2024-09-10T16:04:35.164029+0000","mapping_epoch":35,"log_start":"36'3800","ondisk_log_start":"36'3800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28693446,"num_objects":7007,"num_object_clones":0,"num_object_copies":14014,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7007,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":11944,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'13810","reported_seq":22489,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:04:36.847834+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:04:36.847834+0000","last_peered":"2024-09-10T16:04:36.847834+0000","last_clean":"2024-09-10T16:04:36.847834+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:04:36.847834+0000","last_undegraded":"2024-09-10T16:04:36.847834+0000","last_fullsized":"2024-09-10T16:04:36.847834+0000","mapping_epoch":44,"log_start":"36'3800","ondisk_log_start":"36'3800","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28618776,"num_objects":6988,"num_object_clones":0,"num_object_copies":13976,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6988,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11814,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'13638","reported_seq":13719,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:04:35.200471+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:04:35.200471+0000","last_peered":"2024-09-10T16:04:35.200471+0000","last_clean":"2024-09-10T16:04:35.200471+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:04:35.200471+0000","last_undegraded":"2024-09-10T16:04:35.200471+0000","last_fullsized":"2024-09-10T16:04:35.200471+0000","mapping_epoch":35,"log_start":"36'3600","ondisk_log_start":"36'3600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28762112,"num_objects":7022,"num_object_clones":0,"num_object_copies":14044,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7022,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13638,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'13829","reported_seq":13959,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:04:33.921126+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:04:33.921126+0000","last_peered":"2024-09-10T16:04:33.921126+0000","last_clean":"2024-09-10T16:04:33.921126+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:04:33.921126+0000","last_undegraded":"2024-09-10T16:04:33.921126+0000","last_fullsized":"2024-09-10T16:04:33.921126+0000","mapping_epoch":34,"log_start":"36'3800","ondisk_log_start":"36'3800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28926435,"num_objects":7063,"num_object_clones":0,"num_object_copies":14126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7063,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10540,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'13934","reported_seq":14072,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:33.914285+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:04:33.914285+0000","last_peered":"2024-09-10T16:04:33.914285+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:04:33.914285+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"36'3900","ondisk_log_start":"36'3900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29192675,"num_objects":7128,"num_object_clones":0,"num_object_copies":14256,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7128,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10546,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2757},{"shards":"0,2","objects":970}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'13812","reported_seq":13950,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:33.838307+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:04:33.838307+0000","last_peered":"2024-09-10T16:04:33.838307+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:04:33.838307+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"36'3800","ondisk_log_start":"36'3800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29778886,"num_objects":7272,"num_object_clones":0,"num_object_copies":14544,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7272,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":10311,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3035},{"shards":"0,2","objects":969}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'14031","reported_seq":14161,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:04:33.979534+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:04:33.979534+0000","last_peered":"2024-09-10T16:04:33.979534+0000","last_clean":"2024-09-10T16:04:33.979534+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:04:33.979534+0000","last_undegraded":"2024-09-10T16:04:33.979534+0000","last_fullsized":"2024-09-10T16:04:33.979534+0000","mapping_epoch":34,"log_start":"39'4000","ondisk_log_start":"39'4000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29831168,"num_objects":7283,"num_object_clones":0,"num_object_copies":14566,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7283,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10878,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104261+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:02:08.104261+0000","last_peered":"2024-09-10T16:02:08.104261+0000","last_clean":"2024-09-10T16:02:08.104261+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:02:08.104261+0000","last_undegraded":"2024-09-10T16:02:08.104261+0000","last_fullsized":"2024-09-10T16:02:08.104261+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'13459","reported_seq":13599,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:35.146978+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:04:35.146978+0000","last_peered":"2024-09-10T16:04:35.146978+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:04:35.146978+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"36'3400","ondisk_log_start":"36'3400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28664291,"num_objects":6999,"num_object_clones":0,"num_object_copies":13998,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6999,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11278,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":896},{"shards":"2","objects":2876}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":95,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:29.860108+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:02:29.860108+0000","last_peered":"2024-09-10T16:02:29.860108+0000","last_clean":"2024-09-10T16:02:29.860108+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:02:29.860108+0000","last_undegraded":"2024-09-10T16:02:29.860108+0000","last_fullsized":"2024-09-10T16:02:29.860108+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'13780","reported_seq":13911,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:33.884008+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:04:33.884008+0000","last_peered":"2024-09-10T16:04:33.884008+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:04:33.884008+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"36'3700","ondisk_log_start":"36'3700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29204480,"num_objects":7130,"num_object_clones":0,"num_object_copies":14260,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7130,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10284,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2875},{"shards":"0,2","objects":934}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":105,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:12.405385+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:02:12.405385+0000","last_peered":"2024-09-10T16:02:12.405385+0000","last_clean":"2024-09-10T16:02:12.405385+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:02:12.405385+0000","last_undegraded":"2024-09-10T16:02:12.405385+0000","last_fullsized":"2024-09-10T16:02:12.405385+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'13744","reported_seq":13877,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:35.115778+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:04:35.115778+0000","last_peered":"2024-09-10T16:04:35.115778+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:04:35.115778+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"36'3700","ondisk_log_start":"36'3700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28824035,"num_objects":7038,"num_object_clones":0,"num_object_copies":14076,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7038,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11424,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":953},{"shards":"2","objects":2734}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104617+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:02:08.104617+0000","last_peered":"2024-09-10T16:02:08.104617+0000","last_clean":"2024-09-10T16:02:08.104617+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:02:08.104617+0000","last_undegraded":"2024-09-10T16:02:08.104617+0000","last_fullsized":"2024-09-10T16:02:08.104617+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'13991","reported_seq":14121,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:35.115089+0000","last_change":"2024-09-10T15:59:16.174113+0000","last_active":"2024-09-10T16:04:35.115089+0000","last_peered":"2024-09-10T16:04:35.115089+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:04:35.115089+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"36'3900","ondisk_log_start":"36'3900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29405184,"num_objects":7179,"num_object_clones":0,"num_object_copies":14358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4354,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7179,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11545,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":949},{"shards":"2","objects":2827}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'13734","reported_seq":13869,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:33.829950+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:04:33.829950+0000","last_peered":"2024-09-10T16:04:33.829950+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:04:33.829950+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"36'3700","ondisk_log_start":"36'3700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29097984,"num_objects":7104,"num_object_clones":0,"num_object_copies":14208,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7104,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10365,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2800},{"shards":"0,2","objects":990}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103316+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:02:08.103316+0000","last_peered":"2024-09-10T16:02:08.103316+0000","last_clean":"2024-09-10T16:02:08.103316+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:02:08.103316+0000","last_undegraded":"2024-09-10T16:02:08.103316+0000","last_fullsized":"2024-09-10T16:02:08.103316+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'13536","reported_seq":19416,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:34.024398+0000","last_change":"2024-09-10T16:03:29.605682+0000","last_active":"2024-09-10T16:04:34.024398+0000","last_peered":"2024-09-10T16:04:34.024398+0000","last_clean":"2024-09-10T15:59:14.115595+0000","last_became_active":"2024-09-10T15:59:16.244089+0000","last_became_peered":"2024-09-10T15:59:16.244089+0000","last_unstale":"2024-09-10T16:04:34.024398+0000","last_undegraded":"2024-09-10T16:03:29.603815+0000","last_fullsized":"2024-09-10T15:59:15.774437+0000","mapping_epoch":42,"log_start":"36'3500","ondisk_log_start":"36'3500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28925952,"num_objects":7062,"num_object_clones":0,"num_object_copies":14124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":394,"num_objects_misplaced":944,"num_objects_unfound":0,"num_objects_dirty":7062,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9926,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2868,"num_bytes_recovered":11735040,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5724},{"shards":"0,2","objects":944}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":95,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:29.877831+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:02:29.877831+0000","last_peered":"2024-09-10T16:02:29.877831+0000","last_clean":"2024-09-10T16:02:29.877831+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:02:29.877831+0000","last_undegraded":"2024-09-10T16:02:29.877831+0000","last_fullsized":"2024-09-10T16:02:29.877831+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'13945","reported_seq":14087,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:34.028596+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:04:34.028596+0000","last_peered":"2024-09-10T16:04:34.028596+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:04:34.028596+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"36'3900","ondisk_log_start":"36'3900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29093888,"num_objects":7103,"num_object_clones":0,"num_object_copies":14206,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7103,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10200,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2732},{"shards":"0,2","objects":952}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104463+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:02:08.104463+0000","last_peered":"2024-09-10T16:02:08.104463+0000","last_clean":"2024-09-10T16:02:08.104463+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:02:08.104463+0000","last_undegraded":"2024-09-10T16:02:08.104463+0000","last_fullsized":"2024-09-10T16:02:08.104463+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104448+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:02:08.104448+0000","last_peered":"2024-09-10T16:02:08.104448+0000","last_clean":"2024-09-10T16:02:08.104448+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:02:08.104448+0000","last_undegraded":"2024-09-10T16:02:08.104448+0000","last_fullsized":"2024-09-10T16:02:08.104448+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'13943","reported_seq":14088,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:33.942877+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:04:33.942877+0000","last_peered":"2024-09-10T16:04:33.942877+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:04:33.942877+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"36'3900","ondisk_log_start":"36'3900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29717446,"num_objects":7257,"num_object_clones":0,"num_object_copies":14514,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7257,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10106,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2934},{"shards":"0,2","objects":985}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":465580790,"num_objects":113677,"num_object_clones":0,"num_object_copies":227354,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43305,"num_objects_misplaced":944,"num_objects_unfound":0,"num_objects_dirty":113677,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":178429,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7140,"num_bytes_recovered":29229056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1098752000,"data_stored":1098602949,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160655,"ondisk_log_size":160655,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986255,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":688588,"kb_used_data":363964,"kb_used_omap":4,"kb_used_meta":324603,"kb_avail":93683252,"statfs":{"total":96636764160,"available":95931650048,"internally_reserved":0,"allocated":372699136,"data_stored":372406903,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":332394154},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,2,1,1,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659201,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":601968,"kb_used_data":299552,"kb_used_omap":4,"kb_used_meta":302395,"kb_avail":93769872,"statfs":{"total":96636764160,"available":96020348928,"internally_reserved":0,"allocated":306741248,"data_stored":306412421,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":309653166},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,3,5,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182483,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":799852,"kb_used_data":411416,"kb_used_omap":4,"kb_used_meta":388411,"kb_avail":93571988,"statfs":{"total":96636764160,"available":95817715712,"internally_reserved":0,"allocated":421289984,"data_stored":420997746,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":397733546},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,3,1,3,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":420179968,"data_stored":420143838,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":306360320,"data_stored":306287601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":372211712,"data_stored":372171510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:04:38.777 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-10T16:04:38.777 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "45'13630", 'reported_seq': 13705, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:04:34.009664+0000', 'last_change': '2024-09-10T15:59:16.281558+0000', 'last_active': '2024-09-10T16:04:34.009664+0000', 'last_peered': '2024-09-10T16:04:34.009664+0000', 'last_clean': '2024-09-10T15:59:14.132456+0000', 'last_became_active': '2024-09-10T15:59:16.274407+0000', 'last_became_peered': '2024-09-10T15:59:16.274407+0000', 'last_unstale': '2024-09-10T16:04:34.009664+0000', 'last_undegraded': '2024-09-10T15:59:15.851279+0000', 'last_fullsized': '2024-09-10T15:59:15.774356+0000', 'mapping_epoch': 42, 'log_start': "36'3600", 'ondisk_log_start': "36'3600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'objects_scrubbed': 0, 'log_size': 10030, 'log_dups_size': 0, 'ondisk_log_size': 10030, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28844032, 'num_objects': 7042, 'num_object_clones': 0, 'num_object_copies': 14084, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4259, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7042, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13630, 'num_write_kb': 41344, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2783}, {'shards': '0,2', 'objects': 964}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:04:38.777 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-10T16:04:38.777 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "45'13934", 'reported_seq': 14072, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:04:33.914285+0000', 'last_change': '2024-09-10T15:59:16.308264+0000', 'last_active': '2024-09-10T16:04:33.914285+0000', 'last_peered': '2024-09-10T16:04:33.914285+0000', 'last_clean': '2024-09-10T15:59:14.111551+0000', 'last_became_active': '2024-09-10T15:59:16.300040+0000', 'last_became_peered': '2024-09-10T15:59:16.300040+0000', 'last_unstale': '2024-09-10T16:04:33.914285+0000', 'last_undegraded': '2024-09-10T15:59:15.878234+0000', 'last_fullsized': '2024-09-10T15:59:15.769328+0000', 'mapping_epoch': 42, 'log_start': "36'3900", 'ondisk_log_start': "36'3900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3442", 'last_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'objects_scrubbed': 3384, 'log_size': 10034, 'log_dups_size': 0, 'ondisk_log_size': 10034, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1292, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29192675, 'num_objects': 7128, 'num_object_clones': 0, 'num_object_copies': 14256, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4372, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7128, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 10546, 'num_write_kb': 28580, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2757}, {'shards': '0,2', 'objects': 970}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:04:38.778 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-10T16:04:38.778 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "45'13812", 'reported_seq': 13950, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:04:33.838307+0000', 'last_change': '2024-09-10T15:59:16.331899+0000', 'last_active': '2024-09-10T16:04:33.838307+0000', 'last_peered': '2024-09-10T16:04:33.838307+0000', 'last_clean': '2024-09-10T15:59:14.139436+0000', 'last_became_active': '2024-09-10T15:59:16.318715+0000', 'last_became_peered': '2024-09-10T15:59:16.318715+0000', 'last_unstale': '2024-09-10T16:04:33.838307+0000', 'last_undegraded': '2024-09-10T15:59:15.877524+0000', 'last_fullsized': '2024-09-10T15:59:15.769042+0000', 'mapping_epoch': 42, 'log_start': "36'3800", 'ondisk_log_start': "36'3800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3537", 'last_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'objects_scrubbed': 3497, 'log_size': 10012, 'log_dups_size': 0, 'ondisk_log_size': 10012, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1224, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29778886, 'num_objects': 7272, 'num_object_clones': 0, 'num_object_copies': 14544, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4237, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7272, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 10311, 'num_write_kb': 28172, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3035}, {'shards': '0,2', 'objects': 969}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:04:38.778 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-10T16:04:38.778 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "45'13459", 'reported_seq': 13599, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:04:35.146978+0000', 'last_change': '2024-09-10T15:59:16.175680+0000', 'last_active': '2024-09-10T16:04:35.146978+0000', 'last_peered': '2024-09-10T16:04:35.146978+0000', 'last_clean': '2024-09-10T15:59:14.088187+0000', 'last_became_active': '2024-09-10T15:59:16.165810+0000', 'last_became_peered': '2024-09-10T15:59:16.165810+0000', 'last_unstale': '2024-09-10T16:04:35.146978+0000', 'last_undegraded': '2024-09-10T15:59:15.822098+0000', 'last_fullsized': '2024-09-10T15:59:15.769123+0000', 'mapping_epoch': 42, 'log_start': "36'3400", 'ondisk_log_start': "36'3400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2218", 'last_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'objects_scrubbed': 2175, 'log_size': 10059, 'log_dups_size': 0, 'ondisk_log_size': 10059, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 825, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28664291, 'num_objects': 6999, 'num_object_clones': 0, 'num_object_copies': 13998, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4124, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6999, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 11278, 'num_write_kb': 32204, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 896}, {'shards': '2', 'objects': 2876}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:04:38.778 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-10T16:04:38.778 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "45'13780", 'reported_seq': 13911, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:04:33.884008+0000', 'last_change': '2024-09-10T15:59:16.277742+0000', 'last_active': '2024-09-10T16:04:33.884008+0000', 'last_peered': '2024-09-10T16:04:33.884008+0000', 'last_clean': '2024-09-10T15:59:14.129081+0000', 'last_became_active': '2024-09-10T15:59:16.269835+0000', 'last_became_peered': '2024-09-10T15:59:16.269835+0000', 'last_unstale': '2024-09-10T16:04:33.884008+0000', 'last_undegraded': '2024-09-10T15:59:15.860857+0000', 'last_fullsized': '2024-09-10T15:59:15.771410+0000', 'mapping_epoch': 42, 'log_start': "36'3700", 'ondisk_log_start': "36'3700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3557", 'last_scrub_stamp': '2024-09-10T15:59:02.493678+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:02.493678+0000', 'objects_scrubbed': 3488, 'log_size': 10080, 'log_dups_size': 0, 'ondisk_log_size': 10080, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1472, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29204480, 'num_objects': 7130, 'num_object_clones': 0, 'num_object_copies': 14260, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4257, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7130, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10284, 'num_write_kb': 27852, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2875}, {'shards': '0,2', 'objects': 934}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:04:38.778 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-10T16:04:38.778 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "45'13744", 'reported_seq': 13877, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:04:35.115778+0000', 'last_change': '2024-09-10T15:59:16.284311+0000', 'last_active': '2024-09-10T16:04:35.115778+0000', 'last_peered': '2024-09-10T16:04:35.115778+0000', 'last_clean': '2024-09-10T15:59:14.116041+0000', 'last_became_active': '2024-09-10T15:59:16.274543+0000', 'last_became_peered': '2024-09-10T15:59:16.274543+0000', 'last_unstale': '2024-09-10T16:04:35.115778+0000', 'last_undegraded': '2024-09-10T15:59:15.812484+0000', 'last_fullsized': '2024-09-10T15:59:15.770885+0000', 'mapping_epoch': 42, 'log_start': "36'3700", 'ondisk_log_start': "36'3700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2356", 'last_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'objects_scrubbed': 2316, 'log_size': 10044, 'log_dups_size': 0, 'ondisk_log_size': 10044, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 783, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28824035, 'num_objects': 7038, 'num_object_clones': 0, 'num_object_copies': 14076, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7038, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 11424, 'num_write_kb': 32292, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 953}, {'shards': '2', 'objects': 2734}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:04:38.778 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-10T16:04:38.778 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "45'13991", 'reported_seq': 14121, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:04:35.115089+0000', 'last_change': '2024-09-10T15:59:16.174113+0000', 'last_active': '2024-09-10T16:04:35.115089+0000', 'last_peered': '2024-09-10T16:04:35.115089+0000', 'last_clean': '2024-09-10T15:59:14.110654+0000', 'last_became_active': '2024-09-10T15:59:16.164645+0000', 'last_became_peered': '2024-09-10T15:59:16.164645+0000', 'last_unstale': '2024-09-10T16:04:35.115089+0000', 'last_undegraded': '2024-09-10T15:59:15.803593+0000', 'last_fullsized': '2024-09-10T15:59:15.769083+0000', 'mapping_epoch': 42, 'log_start': "36'3900", 'ondisk_log_start': "36'3900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2486", 'last_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'objects_scrubbed': 2440, 'log_size': 10091, 'log_dups_size': 0, 'ondisk_log_size': 10091, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 792, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29405184, 'num_objects': 7179, 'num_object_clones': 0, 'num_object_copies': 14358, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4354, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7179, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11545, 'num_write_kb': 32568, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 949}, {'shards': '2', 'objects': 2827}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:04:38.778 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-10T16:04:38.778 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "45'13734", 'reported_seq': 13869, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:04:33.829950+0000', 'last_change': '2024-09-10T15:59:16.315641+0000', 'last_active': '2024-09-10T16:04:33.829950+0000', 'last_peered': '2024-09-10T16:04:33.829950+0000', 'last_clean': '2024-09-10T15:59:14.117001+0000', 'last_became_active': '2024-09-10T15:59:16.302245+0000', 'last_became_peered': '2024-09-10T15:59:16.302245+0000', 'last_unstale': '2024-09-10T16:04:33.829950+0000', 'last_undegraded': '2024-09-10T15:59:15.845012+0000', 'last_fullsized': '2024-09-10T15:59:15.770337+0000', 'mapping_epoch': 42, 'log_start': "36'3700", 'ondisk_log_start': "36'3700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3404", 'last_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'objects_scrubbed': 3367, 'log_size': 10034, 'log_dups_size': 0, 'ondisk_log_size': 10034, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 983, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29097984, 'num_objects': 7104, 'num_object_clones': 0, 'num_object_copies': 14208, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7104, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10365, 'num_write_kb': 28204, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2800}, {'shards': '0,2', 'objects': 990}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:04:38.779 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-10T16:04:38.779 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "45'13536", 'reported_seq': 19416, 'reported_epoch': 45, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:04:34.024398+0000', 'last_change': '2024-09-10T16:03:29.605682+0000', 'last_active': '2024-09-10T16:04:34.024398+0000', 'last_peered': '2024-09-10T16:04:34.024398+0000', 'last_clean': '2024-09-10T15:59:14.115595+0000', 'last_became_active': '2024-09-10T15:59:16.244089+0000', 'last_became_peered': '2024-09-10T15:59:16.244089+0000', 'last_unstale': '2024-09-10T16:04:34.024398+0000', 'last_undegraded': '2024-09-10T16:03:29.603815+0000', 'last_fullsized': '2024-09-10T15:59:15.774437+0000', 'mapping_epoch': 42, 'log_start': "36'3500", 'ondisk_log_start': "36'3500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3653", 'last_scrub_stamp': '2024-09-10T15:59:05.298468+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:05.298468+0000', 'objects_scrubbed': 3602, 'log_size': 10036, 'log_dups_size': 0, 'ondisk_log_size': 10036, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1289, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28925952, 'num_objects': 7062, 'num_object_clones': 0, 'num_object_copies': 14124, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 394, 'num_objects_misplaced': 944, 'num_objects_unfound': 0, 'num_objects_dirty': 7062, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9926, 'num_write_kb': 26772, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2868, 'num_bytes_recovered': 11735040, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5724}, {'shards': '0,2', 'objects': 944}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:04:38.779 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-10T16:04:38.779 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "45'13945", 'reported_seq': 14087, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:04:34.028596+0000', 'last_change': '2024-09-10T15:59:16.311291+0000', 'last_active': '2024-09-10T16:04:34.028596+0000', 'last_peered': '2024-09-10T16:04:34.028596+0000', 'last_clean': '2024-09-10T15:59:14.116363+0000', 'last_became_active': '2024-09-10T15:59:16.301244+0000', 'last_became_peered': '2024-09-10T15:59:16.301244+0000', 'last_unstale': '2024-09-10T16:04:34.028596+0000', 'last_undegraded': '2024-09-10T15:59:15.883434+0000', 'last_fullsized': '2024-09-10T15:59:15.772756+0000', 'mapping_epoch': 42, 'log_start': "36'3900", 'ondisk_log_start': "36'3900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3789", 'last_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'objects_scrubbed': 3741, 'log_size': 10045, 'log_dups_size': 0, 'ondisk_log_size': 10045, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1440, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29093888, 'num_objects': 7103, 'num_object_clones': 0, 'num_object_copies': 14206, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4373, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7103, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10200, 'num_write_kb': 27124, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2732}, {'shards': '0,2', 'objects': 952}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:04:38.779 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-10T16:04:38.779 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "45'13943", 'reported_seq': 14088, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:04:33.942877+0000', 'last_change': '2024-09-10T15:59:16.313409+0000', 'last_active': '2024-09-10T16:04:33.942877+0000', 'last_peered': '2024-09-10T16:04:33.942877+0000', 'last_clean': '2024-09-10T15:59:14.129780+0000', 'last_became_active': '2024-09-10T15:59:16.300932+0000', 'last_became_peered': '2024-09-10T15:59:16.300932+0000', 'last_unstale': '2024-09-10T16:04:33.942877+0000', 'last_undegraded': '2024-09-10T15:59:15.887823+0000', 'last_fullsized': '2024-09-10T15:59:15.768513+0000', 'mapping_epoch': 42, 'log_start': "36'3900", 'ondisk_log_start': "36'3900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3880", 'last_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'objects_scrubbed': 3827, 'log_size': 10043, 'log_dups_size': 0, 'ondisk_log_size': 10043, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1397, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29717446, 'num_objects': 7257, 'num_object_clones': 0, 'num_object_copies': 14514, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4325, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7257, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10106, 'num_write_kb': 27072, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2934}, {'shards': '0,2', 'objects': 985}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:04:38.779 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T16:04:39.471 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:04:39.471 INFO:teuthology.orchestra.run.smithi115.stdout:{"fsid":"7effee29-df05-4351-9741-ee203564ed81","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 43305/227358 objects degraded (19.047%), 11 pgs degraded, 11 pgs undersized","count":22},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":443,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983954,"num_in_osds":3,"osd_in_since":1725983947,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":113679,"data_bytes":465580809,"bytes_used":2140577792,"bytes_avail":287769714688,"bytes_total":289910292480,"degraded_objects":43305,"degraded_total":227358,"degraded_ratio":0.19047053545509723,"misplaced_objects":944,"misplaced_total":227358,"misplaced_ratio":0.0041520421537839004,"recovering_objects_per_sec":21,"recovering_bytes_per_sec":88376,"recovering_keys_per_sec":0,"num_objects_recovered":259,"num_bytes_recovered":1060864,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":383},"fsmap":{"epoch":1,"btime":"2024-09-10T15:57:14:967120+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T16:00:13.560776+0000","services":{}},"progress_events":{}} 2024-09-10T16:04:39.489 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-10T16:04:39.490 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:04:40.042 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:04:40.043 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:04:40.073 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":260,"stamp":"2024-09-10T16:04:39.661777+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":459797257,"num_objects":112267,"num_object_clones":0,"num_object_copies":224534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43305,"num_objects_misplaced":819,"num_objects_unfound":0,"num_objects_dirty":112267,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":179843,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7267,"num_bytes_recovered":29741075,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160970,"ondisk_log_size":160970,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2087784,"kb_used_data":1068148,"kb_used_omap":12,"kb_used_meta":1019571,"kb_avail":281027736,"statfs":{"total":289910292480,"available":287772401664,"internally_reserved":0,"allocated":1093783552,"data_stored":1092870254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1044040706},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,9,14,9,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-24059904,"num_objects":-5874,"num_object_clones":0,"num_object_copies":-11748,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-384,"num_objects_unfound":0,"num_objects_dirty":-5874,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5874,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":384,"num_bytes_recovered":1572864,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003612"},"pg_stats":[{"pgid":"2.f","version":"45'13752","reported_seq":13827,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:39.006361+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:04:39.006361+0000","last_peered":"2024-09-10T16:04:39.006361+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:04:39.006361+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"36'3700","ondisk_log_start":"36'3700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28344320,"num_objects":6920,"num_object_clones":0,"num_object_copies":13840,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6920,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13752,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2661},{"shards":"0,2","objects":842}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'13839","reported_seq":13952,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:04:35.164029+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:04:35.164029+0000","last_peered":"2024-09-10T16:04:35.164029+0000","last_clean":"2024-09-10T16:04:35.164029+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:04:35.164029+0000","last_undegraded":"2024-09-10T16:04:35.164029+0000","last_fullsized":"2024-09-10T16:04:35.164029+0000","mapping_epoch":35,"log_start":"36'3800","ondisk_log_start":"36'3800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28693446,"num_objects":7007,"num_object_clones":0,"num_object_copies":14014,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7007,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":11944,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'13810","reported_seq":22489,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:04:36.847834+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:04:36.847834+0000","last_peered":"2024-09-10T16:04:36.847834+0000","last_clean":"2024-09-10T16:04:36.847834+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:04:36.847834+0000","last_undegraded":"2024-09-10T16:04:36.847834+0000","last_fullsized":"2024-09-10T16:04:36.847834+0000","mapping_epoch":44,"log_start":"36'3800","ondisk_log_start":"36'3800","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28618776,"num_objects":6988,"num_object_clones":0,"num_object_copies":13976,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6988,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11814,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'13638","reported_seq":13719,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:04:35.200471+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:04:35.200471+0000","last_peered":"2024-09-10T16:04:35.200471+0000","last_clean":"2024-09-10T16:04:35.200471+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:04:35.200471+0000","last_undegraded":"2024-09-10T16:04:35.200471+0000","last_fullsized":"2024-09-10T16:04:35.200471+0000","mapping_epoch":35,"log_start":"36'3600","ondisk_log_start":"36'3600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28762112,"num_objects":7022,"num_object_clones":0,"num_object_copies":14044,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7022,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13638,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'13967","reported_seq":14098,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:04:39.056108+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:04:39.056108+0000","last_peered":"2024-09-10T16:04:39.056108+0000","last_clean":"2024-09-10T16:04:39.056108+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:04:39.056108+0000","last_undegraded":"2024-09-10T16:04:39.056108+0000","last_fullsized":"2024-09-10T16:04:39.056108+0000","mapping_epoch":34,"log_start":"36'3900","ondisk_log_start":"36'3900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28361187,"num_objects":6925,"num_object_clones":0,"num_object_copies":13850,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6925,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10678,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'14086","reported_seq":14224,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:39.054395+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:04:39.054395+0000","last_peered":"2024-09-10T16:04:39.054395+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:04:39.054395+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"39'4000","ondisk_log_start":"39'4000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28570083,"num_objects":6976,"num_object_clones":0,"num_object_copies":13952,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6976,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10698,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2605},{"shards":"0,2","objects":818}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'13952","reported_seq":14090,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:38.954533+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:04:38.954533+0000","last_peered":"2024-09-10T16:04:38.954533+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:04:38.954533+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"39'3900","ondisk_log_start":"39'3900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29205446,"num_objects":7132,"num_object_clones":0,"num_object_copies":14264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7132,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":10451,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2895},{"shards":"0,2","objects":829}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'14188","reported_seq":14319,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:04:39.005460+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:04:39.005460+0000","last_peered":"2024-09-10T16:04:39.005460+0000","last_clean":"2024-09-10T16:04:39.005460+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:04:39.005460+0000","last_undegraded":"2024-09-10T16:04:39.005460+0000","last_fullsized":"2024-09-10T16:04:39.005460+0000","mapping_epoch":34,"log_start":"39'4100","ondisk_log_start":"39'4100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29188096,"num_objects":7126,"num_object_clones":0,"num_object_copies":14252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7126,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11035,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104261+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:02:08.104261+0000","last_peered":"2024-09-10T16:02:08.104261+0000","last_clean":"2024-09-10T16:02:08.104261+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:02:08.104261+0000","last_undegraded":"2024-09-10T16:02:08.104261+0000","last_fullsized":"2024-09-10T16:02:08.104261+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'13459","reported_seq":13599,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:35.146978+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:04:35.146978+0000","last_peered":"2024-09-10T16:04:35.146978+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:04:35.146978+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"36'3400","ondisk_log_start":"36'3400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28664291,"num_objects":6999,"num_object_clones":0,"num_object_copies":13998,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6999,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11278,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":896},{"shards":"2","objects":2876}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":95,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:29.860108+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:02:29.860108+0000","last_peered":"2024-09-10T16:02:29.860108+0000","last_clean":"2024-09-10T16:02:29.860108+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:02:29.860108+0000","last_undegraded":"2024-09-10T16:02:29.860108+0000","last_fullsized":"2024-09-10T16:02:29.860108+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'13916","reported_seq":14047,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:39.031135+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:04:39.031135+0000","last_peered":"2024-09-10T16:04:39.031135+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:04:39.031135+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"39'3900","ondisk_log_start":"39'3900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28647424,"num_objects":6994,"num_object_clones":0,"num_object_copies":13988,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6994,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10420,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2739},{"shards":"0,2","objects":798}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":105,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:12.405385+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:02:12.405385+0000","last_peered":"2024-09-10T16:02:12.405385+0000","last_clean":"2024-09-10T16:02:12.405385+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:02:12.405385+0000","last_undegraded":"2024-09-10T16:02:12.405385+0000","last_fullsized":"2024-09-10T16:02:12.405385+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'13744","reported_seq":13877,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:35.115778+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:04:35.115778+0000","last_peered":"2024-09-10T16:04:35.115778+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:04:35.115778+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"36'3700","ondisk_log_start":"36'3700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28824035,"num_objects":7038,"num_object_clones":0,"num_object_copies":14076,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7038,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11424,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":953},{"shards":"2","objects":2734}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104617+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:02:08.104617+0000","last_peered":"2024-09-10T16:02:08.104617+0000","last_clean":"2024-09-10T16:02:08.104617+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:02:08.104617+0000","last_undegraded":"2024-09-10T16:02:08.104617+0000","last_fullsized":"2024-09-10T16:02:08.104617+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'13991","reported_seq":14121,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:35.115089+0000","last_change":"2024-09-10T15:59:16.174113+0000","last_active":"2024-09-10T16:04:35.115089+0000","last_peered":"2024-09-10T16:04:35.115089+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:04:35.115089+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"36'3900","ondisk_log_start":"36'3900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29405184,"num_objects":7179,"num_object_clones":0,"num_object_copies":14358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4354,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7179,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11545,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":949},{"shards":"2","objects":2827}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'13865","reported_seq":13994,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:39.007530+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:04:39.007530+0000","last_peered":"2024-09-10T16:04:39.007530+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:04:39.007530+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"36'3800","ondisk_log_start":"36'3800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28561408,"num_objects":6973,"num_object_clones":0,"num_object_copies":13946,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6973,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10496,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2669},{"shards":"0,2","objects":859}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103316+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:02:08.103316+0000","last_peered":"2024-09-10T16:02:08.103316+0000","last_clean":"2024-09-10T16:02:08.103316+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:02:08.103316+0000","last_undegraded":"2024-09-10T16:02:08.103316+0000","last_fullsized":"2024-09-10T16:02:08.103316+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'13678","reported_seq":19808,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:39.077551+0000","last_change":"2024-09-10T16:03:29.605682+0000","last_active":"2024-09-10T16:04:39.077551+0000","last_peered":"2024-09-10T16:04:39.077551+0000","last_clean":"2024-09-10T15:59:14.115595+0000","last_became_active":"2024-09-10T15:59:16.244089+0000","last_became_peered":"2024-09-10T15:59:16.244089+0000","last_unstale":"2024-09-10T16:04:39.077551+0000","last_undegraded":"2024-09-10T16:03:29.603815+0000","last_fullsized":"2024-09-10T15:59:15.774437+0000","mapping_epoch":42,"log_start":"36'3600","ondisk_log_start":"36'3600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28344320,"num_objects":6920,"num_object_clones":0,"num_object_copies":13840,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":394,"num_objects_misplaced":819,"num_objects_unfound":0,"num_objects_dirty":6920,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10068,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2993,"num_bytes_recovered":12247040,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5707},{"shards":"0,2","objects":819}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":95,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:29.877831+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:02:29.877831+0000","last_peered":"2024-09-10T16:02:29.877831+0000","last_clean":"2024-09-10T16:02:29.877831+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:02:29.877831+0000","last_undegraded":"2024-09-10T16:02:29.877831+0000","last_fullsized":"2024-09-10T16:02:29.877831+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'14085","reported_seq":14227,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:39.065176+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:04:39.065176+0000","last_peered":"2024-09-10T16:04:39.065176+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:04:39.065176+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"39'4000","ondisk_log_start":"39'4000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28520448,"num_objects":6963,"num_object_clones":0,"num_object_copies":13926,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6963,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10340,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2592},{"shards":"0,2","objects":812}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104463+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:02:08.104463+0000","last_peered":"2024-09-10T16:02:08.104463+0000","last_clean":"2024-09-10T16:02:08.104463+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:02:08.104463+0000","last_undegraded":"2024-09-10T16:02:08.104463+0000","last_fullsized":"2024-09-10T16:02:08.104463+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104448+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:02:08.104448+0000","last_peered":"2024-09-10T16:02:08.104448+0000","last_clean":"2024-09-10T16:02:08.104448+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:02:08.104448+0000","last_undegraded":"2024-09-10T16:02:08.104448+0000","last_fullsized":"2024-09-10T16:02:08.104448+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'14097","reported_seq":14242,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:38.928733+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:04:38.928733+0000","last_peered":"2024-09-10T16:04:38.928733+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:04:38.928733+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"39'4000","ondisk_log_start":"39'4000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29086662,"num_objects":7103,"num_object_clones":0,"num_object_copies":14206,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7103,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10260,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2780},{"shards":"0,2","objects":831}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":459797238,"num_objects":112265,"num_object_clones":0,"num_object_copies":224530,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43305,"num_objects_misplaced":819,"num_objects_unfound":0,"num_objects_dirty":112265,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":179841,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7265,"num_bytes_recovered":29741056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1091801088,"data_stored":1091652037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160967,"ondisk_log_size":160967,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986255,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":688588,"kb_used_data":363964,"kb_used_omap":4,"kb_used_meta":324603,"kb_avail":93683252,"statfs":{"total":96636764160,"available":95931650048,"internally_reserved":0,"allocated":372699136,"data_stored":372406903,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":332394154},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,2,1,1,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659201,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":601968,"kb_used_data":299552,"kb_used_omap":4,"kb_used_meta":302395,"kb_avail":93769872,"statfs":{"total":96636764160,"available":96020348928,"internally_reserved":0,"allocated":306741248,"data_stored":306412421,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":309653166},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,3,5,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182484,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":797228,"kb_used_data":404632,"kb_used_omap":4,"kb_used_meta":392571,"kb_avail":93574612,"statfs":{"total":96636764160,"available":95820402688,"internally_reserved":0,"allocated":414343168,"data_stored":414050930,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":401993386},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,4,8,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":413229056,"data_stored":413192926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":306360320,"data_stored":306287601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":372211712,"data_stored":372171510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:04:43.076 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:04:43.630 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:04:43.631 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:04:43.659 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":261,"stamp":"2024-09-10T16:04:41.664638+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":456872713,"num_objects":111553,"num_object_clones":0,"num_object_copies":223106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43305,"num_objects_misplaced":819,"num_objects_unfound":0,"num_objects_dirty":111553,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":180557,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7267,"num_bytes_recovered":29741075,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161084,"ondisk_log_size":161084,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2087512,"kb_used_data":1063716,"kb_used_omap":12,"kb_used_meta":1023731,"kb_avail":281028008,"statfs":{"total":289910292480,"available":287772680192,"internally_reserved":0,"allocated":1089245184,"data_stored":1088331886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1048300546},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,0,10,20,8],"upper_bound":128},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-21204992,"num_objects":-5177,"num_object_clones":0,"num_object_copies":-10354,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-254,"num_objects_unfound":0,"num_objects_dirty":-5177,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5177,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":254,"num_bytes_recovered":1040384,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005802"},"pg_stats":[{"pgid":"2.f","version":"45'13752","reported_seq":13827,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:39.006361+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:04:39.006361+0000","last_peered":"2024-09-10T16:04:39.006361+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:04:39.006361+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"36'3700","ondisk_log_start":"36'3700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28344320,"num_objects":6920,"num_object_clones":0,"num_object_copies":13840,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6920,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13752,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2661},{"shards":"0,2","objects":842}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'13975","reported_seq":14086,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:04:40.220552+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:04:40.220552+0000","last_peered":"2024-09-10T16:04:40.220552+0000","last_clean":"2024-09-10T16:04:40.220552+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:04:40.220552+0000","last_undegraded":"2024-09-10T16:04:40.220552+0000","last_fullsized":"2024-09-10T16:04:40.220552+0000","mapping_epoch":35,"log_start":"36'3900","ondisk_log_start":"36'3900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28136390,"num_objects":6871,"num_object_clones":0,"num_object_copies":13742,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6871,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":12080,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'13810","reported_seq":22489,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:04:36.847834+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:04:36.847834+0000","last_peered":"2024-09-10T16:04:36.847834+0000","last_clean":"2024-09-10T16:04:36.847834+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:04:36.847834+0000","last_undegraded":"2024-09-10T16:04:36.847834+0000","last_fullsized":"2024-09-10T16:04:36.847834+0000","mapping_epoch":44,"log_start":"36'3800","ondisk_log_start":"36'3800","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28618776,"num_objects":6988,"num_object_clones":0,"num_object_copies":13976,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6988,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11814,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'13784","reported_seq":13865,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:04:40.245522+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:04:40.245522+0000","last_peered":"2024-09-10T16:04:40.245522+0000","last_clean":"2024-09-10T16:04:40.245522+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:04:40.245522+0000","last_undegraded":"2024-09-10T16:04:40.245522+0000","last_fullsized":"2024-09-10T16:04:40.245522+0000","mapping_epoch":35,"log_start":"36'3700","ondisk_log_start":"36'3700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28164096,"num_objects":6876,"num_object_clones":0,"num_object_copies":13752,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6876,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13784,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'13967","reported_seq":14098,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:04:39.056108+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:04:39.056108+0000","last_peered":"2024-09-10T16:04:39.056108+0000","last_clean":"2024-09-10T16:04:39.056108+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:04:39.056108+0000","last_undegraded":"2024-09-10T16:04:39.056108+0000","last_fullsized":"2024-09-10T16:04:39.056108+0000","mapping_epoch":34,"log_start":"36'3900","ondisk_log_start":"36'3900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28361187,"num_objects":6925,"num_object_clones":0,"num_object_copies":13850,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6925,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10678,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'14086","reported_seq":14224,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:39.054395+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:04:39.054395+0000","last_peered":"2024-09-10T16:04:39.054395+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:04:39.054395+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"39'4000","ondisk_log_start":"39'4000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28570083,"num_objects":6976,"num_object_clones":0,"num_object_copies":13952,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6976,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10698,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2605},{"shards":"0,2","objects":818}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'13952","reported_seq":14090,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:38.954533+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:04:38.954533+0000","last_peered":"2024-09-10T16:04:38.954533+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:04:38.954533+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"39'3900","ondisk_log_start":"39'3900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29205446,"num_objects":7132,"num_object_clones":0,"num_object_copies":14264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7132,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":10451,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2895},{"shards":"0,2","objects":829}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'14188","reported_seq":14319,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:04:39.005460+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:04:39.005460+0000","last_peered":"2024-09-10T16:04:39.005460+0000","last_clean":"2024-09-10T16:04:39.005460+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:04:39.005460+0000","last_undegraded":"2024-09-10T16:04:39.005460+0000","last_fullsized":"2024-09-10T16:04:39.005460+0000","mapping_epoch":34,"log_start":"39'4100","ondisk_log_start":"39'4100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29188096,"num_objects":7126,"num_object_clones":0,"num_object_copies":14252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7126,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11035,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104261+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:02:08.104261+0000","last_peered":"2024-09-10T16:02:08.104261+0000","last_clean":"2024-09-10T16:02:08.104261+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:02:08.104261+0000","last_undegraded":"2024-09-10T16:02:08.104261+0000","last_fullsized":"2024-09-10T16:02:08.104261+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'13596","reported_seq":13736,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:40.209830+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:04:40.209830+0000","last_peered":"2024-09-10T16:04:40.209830+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:04:40.209830+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"36'3500","ondisk_log_start":"36'3500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28103139,"num_objects":6862,"num_object_clones":0,"num_object_copies":13724,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6862,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11415,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":759},{"shards":"2","objects":2739}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":95,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:29.860108+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:02:29.860108+0000","last_peered":"2024-09-10T16:02:29.860108+0000","last_clean":"2024-09-10T16:02:29.860108+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:02:29.860108+0000","last_undegraded":"2024-09-10T16:02:29.860108+0000","last_fullsized":"2024-09-10T16:02:29.860108+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'13916","reported_seq":14047,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:39.031135+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:04:39.031135+0000","last_peered":"2024-09-10T16:04:39.031135+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:04:39.031135+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"39'3900","ondisk_log_start":"39'3900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28647424,"num_objects":6994,"num_object_clones":0,"num_object_copies":13988,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6994,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10420,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2739},{"shards":"0,2","objects":798}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":105,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:12.405385+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:02:12.405385+0000","last_peered":"2024-09-10T16:02:12.405385+0000","last_clean":"2024-09-10T16:02:12.405385+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:02:12.405385+0000","last_undegraded":"2024-09-10T16:02:12.405385+0000","last_fullsized":"2024-09-10T16:02:12.405385+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'13873","reported_seq":14009,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:40.145102+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:04:40.145102+0000","last_peered":"2024-09-10T16:04:40.145102+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:04:40.145102+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"36'3800","ondisk_log_start":"36'3800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28295651,"num_objects":6909,"num_object_clones":0,"num_object_copies":13818,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6909,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11553,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":824},{"shards":"2","objects":2605}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104617+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:02:08.104617+0000","last_peered":"2024-09-10T16:02:08.104617+0000","last_clean":"2024-09-10T16:02:08.104617+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:02:08.104617+0000","last_undegraded":"2024-09-10T16:02:08.104617+0000","last_fullsized":"2024-09-10T16:02:08.104617+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'14157","reported_seq":14287,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:40.194552+0000","last_change":"2024-09-10T15:59:16.174113+0000","last_active":"2024-09-10T16:04:40.194552+0000","last_peered":"2024-09-10T16:04:40.194552+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:04:40.194552+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"39'4100","ondisk_log_start":"39'4100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28725248,"num_objects":7013,"num_object_clones":0,"num_object_copies":14026,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4354,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7013,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11711,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":783},{"shards":"2","objects":2661}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'13865","reported_seq":13994,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:39.007530+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:04:39.007530+0000","last_peered":"2024-09-10T16:04:39.007530+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:04:39.007530+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"36'3800","ondisk_log_start":"36'3800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28561408,"num_objects":6973,"num_object_clones":0,"num_object_copies":13946,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6973,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10496,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2669},{"shards":"0,2","objects":859}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103316+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:02:08.103316+0000","last_peered":"2024-09-10T16:02:08.103316+0000","last_clean":"2024-09-10T16:02:08.103316+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:02:08.103316+0000","last_undegraded":"2024-09-10T16:02:08.103316+0000","last_fullsized":"2024-09-10T16:02:08.103316+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'13678","reported_seq":19808,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:39.077551+0000","last_change":"2024-09-10T16:03:29.605682+0000","last_active":"2024-09-10T16:04:39.077551+0000","last_peered":"2024-09-10T16:04:39.077551+0000","last_clean":"2024-09-10T15:59:14.115595+0000","last_became_active":"2024-09-10T15:59:16.244089+0000","last_became_peered":"2024-09-10T15:59:16.244089+0000","last_unstale":"2024-09-10T16:04:39.077551+0000","last_undegraded":"2024-09-10T16:03:29.603815+0000","last_fullsized":"2024-09-10T15:59:15.774437+0000","mapping_epoch":42,"log_start":"36'3600","ondisk_log_start":"36'3600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28344320,"num_objects":6920,"num_object_clones":0,"num_object_copies":13840,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":394,"num_objects_misplaced":819,"num_objects_unfound":0,"num_objects_dirty":6920,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10068,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2993,"num_bytes_recovered":12247040,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5707},{"shards":"0,2","objects":819}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":95,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:29.877831+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:02:29.877831+0000","last_peered":"2024-09-10T16:02:29.877831+0000","last_clean":"2024-09-10T16:02:29.877831+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:02:29.877831+0000","last_undegraded":"2024-09-10T16:02:29.877831+0000","last_fullsized":"2024-09-10T16:02:29.877831+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'14085","reported_seq":14227,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:39.065176+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:04:39.065176+0000","last_peered":"2024-09-10T16:04:39.065176+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:04:39.065176+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"39'4000","ondisk_log_start":"39'4000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28520448,"num_objects":6963,"num_object_clones":0,"num_object_copies":13926,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6963,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10340,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2592},{"shards":"0,2","objects":812}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104463+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:02:08.104463+0000","last_peered":"2024-09-10T16:02:08.104463+0000","last_clean":"2024-09-10T16:02:08.104463+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:02:08.104463+0000","last_undegraded":"2024-09-10T16:02:08.104463+0000","last_fullsized":"2024-09-10T16:02:08.104463+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104448+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:02:08.104448+0000","last_peered":"2024-09-10T16:02:08.104448+0000","last_clean":"2024-09-10T16:02:08.104448+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:02:08.104448+0000","last_undegraded":"2024-09-10T16:02:08.104448+0000","last_fullsized":"2024-09-10T16:02:08.104448+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'14097","reported_seq":14242,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:38.928733+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:04:38.928733+0000","last_peered":"2024-09-10T16:04:38.928733+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:04:38.928733+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"39'4000","ondisk_log_start":"39'4000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29086662,"num_objects":7103,"num_object_clones":0,"num_object_copies":14206,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7103,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10260,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2780},{"shards":"0,2","objects":831}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":456872694,"num_objects":111551,"num_object_clones":0,"num_object_copies":223102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43305,"num_objects_misplaced":819,"num_objects_unfound":0,"num_objects_dirty":111551,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":180555,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7265,"num_bytes_recovered":29741056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1087275008,"data_stored":1087125957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161081,"ondisk_log_size":161081,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986256,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":688316,"kb_used_data":359532,"kb_used_omap":4,"kb_used_meta":328763,"kb_avail":93683524,"statfs":{"total":96636764160,"available":95931928576,"internally_reserved":0,"allocated":368160768,"data_stored":367868535,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":336653994},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,3,7],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659201,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":601968,"kb_used_data":299552,"kb_used_omap":4,"kb_used_meta":302395,"kb_avail":93769872,"statfs":{"total":96636764160,"available":96020348928,"internally_reserved":0,"allocated":306741248,"data_stored":306412421,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":309653166},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,3,5,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182484,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":797228,"kb_used_data":404632,"kb_used_omap":4,"kb_used_meta":392571,"kb_avail":93574612,"statfs":{"total":96636764160,"available":95820402688,"internally_reserved":0,"allocated":414343168,"data_stored":414050930,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":401993386},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,4,8,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":413229056,"data_stored":413192926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":306360320,"data_stored":306287601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":367685632,"data_stored":367645430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:04:43.661 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:04:44.223 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:04:44.223 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:04:44.247 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":262,"stamp":"2024-09-10T16:04:43.665283+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":456360713,"num_objects":111428,"num_object_clones":0,"num_object_copies":222856,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43305,"num_objects_misplaced":819,"num_objects_unfound":0,"num_objects_dirty":111428,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":180682,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7267,"num_bytes_recovered":29741075,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161109,"ondisk_log_size":161109,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2095360,"kb_used_data":1063244,"kb_used_omap":12,"kb_used_meta":1032051,"kb_avail":281020160,"statfs":{"total":289910292480,"available":287764643840,"internally_reserved":0,"allocated":1088761856,"data_stored":1087848558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1056820226},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,5,9,20,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19099648,"num_objects":-4663,"num_object_clones":0,"num_object_copies":-9326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-254,"num_objects_unfound":0,"num_objects_dirty":-4663,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4663,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":254,"num_bytes_recovered":1040384,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005800"},"pg_stats":[{"pgid":"2.f","version":"45'13752","reported_seq":13827,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:39.006361+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:04:39.006361+0000","last_peered":"2024-09-10T16:04:39.006361+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:04:39.006361+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"36'3700","ondisk_log_start":"36'3700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28344320,"num_objects":6920,"num_object_clones":0,"num_object_copies":13840,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6920,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13752,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2661},{"shards":"0,2","objects":842}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'13975","reported_seq":14086,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:04:40.220552+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:04:40.220552+0000","last_peered":"2024-09-10T16:04:40.220552+0000","last_clean":"2024-09-10T16:04:40.220552+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:04:40.220552+0000","last_undegraded":"2024-09-10T16:04:40.220552+0000","last_fullsized":"2024-09-10T16:04:40.220552+0000","mapping_epoch":35,"log_start":"36'3900","ondisk_log_start":"36'3900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28136390,"num_objects":6871,"num_object_clones":0,"num_object_copies":13742,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6871,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":12080,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'13935","reported_seq":22614,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:04:41.751850+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:04:41.751850+0000","last_peered":"2024-09-10T16:04:41.751850+0000","last_clean":"2024-09-10T16:04:41.751850+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:04:41.751850+0000","last_undegraded":"2024-09-10T16:04:41.751850+0000","last_fullsized":"2024-09-10T16:04:41.751850+0000","mapping_epoch":44,"log_start":"39'3900","ondisk_log_start":"39'3900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28106776,"num_objects":6863,"num_object_clones":0,"num_object_copies":13726,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6863,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11939,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'13784","reported_seq":13865,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:04:40.245522+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:04:40.245522+0000","last_peered":"2024-09-10T16:04:40.245522+0000","last_clean":"2024-09-10T16:04:40.245522+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:04:40.245522+0000","last_undegraded":"2024-09-10T16:04:40.245522+0000","last_fullsized":"2024-09-10T16:04:40.245522+0000","mapping_epoch":35,"log_start":"36'3700","ondisk_log_start":"36'3700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28164096,"num_objects":6876,"num_object_clones":0,"num_object_copies":13752,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6876,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13784,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'13967","reported_seq":14098,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:04:39.056108+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:04:39.056108+0000","last_peered":"2024-09-10T16:04:39.056108+0000","last_clean":"2024-09-10T16:04:39.056108+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:04:39.056108+0000","last_undegraded":"2024-09-10T16:04:39.056108+0000","last_fullsized":"2024-09-10T16:04:39.056108+0000","mapping_epoch":34,"log_start":"36'3900","ondisk_log_start":"36'3900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28361187,"num_objects":6925,"num_object_clones":0,"num_object_copies":13850,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6925,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10678,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'14086","reported_seq":14224,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:39.054395+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:04:39.054395+0000","last_peered":"2024-09-10T16:04:39.054395+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:04:39.054395+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"39'4000","ondisk_log_start":"39'4000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28570083,"num_objects":6976,"num_object_clones":0,"num_object_copies":13952,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6976,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10698,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2605},{"shards":"0,2","objects":818}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'13952","reported_seq":14090,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:38.954533+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:04:38.954533+0000","last_peered":"2024-09-10T16:04:38.954533+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:04:38.954533+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"39'3900","ondisk_log_start":"39'3900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29205446,"num_objects":7132,"num_object_clones":0,"num_object_copies":14264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7132,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":10451,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2895},{"shards":"0,2","objects":829}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'14188","reported_seq":14319,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:04:39.005460+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:04:39.005460+0000","last_peered":"2024-09-10T16:04:39.005460+0000","last_clean":"2024-09-10T16:04:39.005460+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:04:39.005460+0000","last_undegraded":"2024-09-10T16:04:39.005460+0000","last_fullsized":"2024-09-10T16:04:39.005460+0000","mapping_epoch":34,"log_start":"39'4100","ondisk_log_start":"39'4100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29188096,"num_objects":7126,"num_object_clones":0,"num_object_copies":14252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7126,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11035,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104261+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:02:08.104261+0000","last_peered":"2024-09-10T16:02:08.104261+0000","last_clean":"2024-09-10T16:02:08.104261+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:02:08.104261+0000","last_undegraded":"2024-09-10T16:02:08.104261+0000","last_fullsized":"2024-09-10T16:02:08.104261+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'13596","reported_seq":13736,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:40.209830+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:04:40.209830+0000","last_peered":"2024-09-10T16:04:40.209830+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:04:40.209830+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"36'3500","ondisk_log_start":"36'3500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28103139,"num_objects":6862,"num_object_clones":0,"num_object_copies":13724,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6862,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11415,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":759},{"shards":"2","objects":2739}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":95,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:29.860108+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:02:29.860108+0000","last_peered":"2024-09-10T16:02:29.860108+0000","last_clean":"2024-09-10T16:02:29.860108+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:02:29.860108+0000","last_undegraded":"2024-09-10T16:02:29.860108+0000","last_fullsized":"2024-09-10T16:02:29.860108+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'13916","reported_seq":14047,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:39.031135+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:04:39.031135+0000","last_peered":"2024-09-10T16:04:39.031135+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:04:39.031135+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"39'3900","ondisk_log_start":"39'3900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28647424,"num_objects":6994,"num_object_clones":0,"num_object_copies":13988,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6994,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10420,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2739},{"shards":"0,2","objects":798}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":105,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:12.405385+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:02:12.405385+0000","last_peered":"2024-09-10T16:02:12.405385+0000","last_clean":"2024-09-10T16:02:12.405385+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:02:12.405385+0000","last_undegraded":"2024-09-10T16:02:12.405385+0000","last_fullsized":"2024-09-10T16:02:12.405385+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'13873","reported_seq":14009,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:40.145102+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:04:40.145102+0000","last_peered":"2024-09-10T16:04:40.145102+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:04:40.145102+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"36'3800","ondisk_log_start":"36'3800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28295651,"num_objects":6909,"num_object_clones":0,"num_object_copies":13818,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6909,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11553,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":824},{"shards":"2","objects":2605}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104617+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:02:08.104617+0000","last_peered":"2024-09-10T16:02:08.104617+0000","last_clean":"2024-09-10T16:02:08.104617+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:02:08.104617+0000","last_undegraded":"2024-09-10T16:02:08.104617+0000","last_fullsized":"2024-09-10T16:02:08.104617+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'14157","reported_seq":14287,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:40.194552+0000","last_change":"2024-09-10T15:59:16.174113+0000","last_active":"2024-09-10T16:04:40.194552+0000","last_peered":"2024-09-10T16:04:40.194552+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:04:40.194552+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"39'4100","ondisk_log_start":"39'4100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28725248,"num_objects":7013,"num_object_clones":0,"num_object_copies":14026,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4354,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7013,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11711,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":783},{"shards":"2","objects":2661}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'13865","reported_seq":13994,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:39.007530+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:04:39.007530+0000","last_peered":"2024-09-10T16:04:39.007530+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:04:39.007530+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"36'3800","ondisk_log_start":"36'3800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28561408,"num_objects":6973,"num_object_clones":0,"num_object_copies":13946,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6973,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10496,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2669},{"shards":"0,2","objects":859}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103316+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:02:08.103316+0000","last_peered":"2024-09-10T16:02:08.103316+0000","last_clean":"2024-09-10T16:02:08.103316+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:02:08.103316+0000","last_undegraded":"2024-09-10T16:02:08.103316+0000","last_fullsized":"2024-09-10T16:02:08.103316+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'13678","reported_seq":19808,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:39.077551+0000","last_change":"2024-09-10T16:03:29.605682+0000","last_active":"2024-09-10T16:04:39.077551+0000","last_peered":"2024-09-10T16:04:39.077551+0000","last_clean":"2024-09-10T15:59:14.115595+0000","last_became_active":"2024-09-10T15:59:16.244089+0000","last_became_peered":"2024-09-10T15:59:16.244089+0000","last_unstale":"2024-09-10T16:04:39.077551+0000","last_undegraded":"2024-09-10T16:03:29.603815+0000","last_fullsized":"2024-09-10T15:59:15.774437+0000","mapping_epoch":42,"log_start":"36'3600","ondisk_log_start":"36'3600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28344320,"num_objects":6920,"num_object_clones":0,"num_object_copies":13840,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":394,"num_objects_misplaced":819,"num_objects_unfound":0,"num_objects_dirty":6920,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10068,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2993,"num_bytes_recovered":12247040,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5707},{"shards":"0,2","objects":819}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":95,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:29.877831+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:02:29.877831+0000","last_peered":"2024-09-10T16:02:29.877831+0000","last_clean":"2024-09-10T16:02:29.877831+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:02:29.877831+0000","last_undegraded":"2024-09-10T16:02:29.877831+0000","last_fullsized":"2024-09-10T16:02:29.877831+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'14085","reported_seq":14227,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:39.065176+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:04:39.065176+0000","last_peered":"2024-09-10T16:04:39.065176+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:04:39.065176+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"39'4000","ondisk_log_start":"39'4000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28520448,"num_objects":6963,"num_object_clones":0,"num_object_copies":13926,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6963,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10340,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2592},{"shards":"0,2","objects":812}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104463+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:02:08.104463+0000","last_peered":"2024-09-10T16:02:08.104463+0000","last_clean":"2024-09-10T16:02:08.104463+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:02:08.104463+0000","last_undegraded":"2024-09-10T16:02:08.104463+0000","last_fullsized":"2024-09-10T16:02:08.104463+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104448+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:02:08.104448+0000","last_peered":"2024-09-10T16:02:08.104448+0000","last_clean":"2024-09-10T16:02:08.104448+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:02:08.104448+0000","last_undegraded":"2024-09-10T16:02:08.104448+0000","last_fullsized":"2024-09-10T16:02:08.104448+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'14097","reported_seq":14242,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:38.928733+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:04:38.928733+0000","last_peered":"2024-09-10T16:04:38.928733+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:04:38.928733+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"39'4000","ondisk_log_start":"39'4000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29086662,"num_objects":7103,"num_object_clones":0,"num_object_copies":14206,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7103,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10260,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2780},{"shards":"0,2","objects":831}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":456360694,"num_objects":111426,"num_object_clones":0,"num_object_copies":222852,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43305,"num_objects_misplaced":819,"num_objects_unfound":0,"num_objects_dirty":111426,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":180680,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7265,"num_bytes_recovered":29741056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1086791680,"data_stored":1086642629,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161106,"ondisk_log_size":161106,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986256,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":688316,"kb_used_data":359532,"kb_used_omap":4,"kb_used_meta":328763,"kb_avail":93683524,"statfs":{"total":96636764160,"available":95931928576,"internally_reserved":0,"allocated":368160768,"data_stored":367868535,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":336653994},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,3,7],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659202,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":609816,"kb_used_data":299080,"kb_used_omap":4,"kb_used_meta":310715,"kb_avail":93762024,"statfs":{"total":96636764160,"available":96012312576,"internally_reserved":0,"allocated":306257920,"data_stored":305929093,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":318172846},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,5,2,5,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182484,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":797228,"kb_used_data":404632,"kb_used_omap":4,"kb_used_meta":392571,"kb_avail":93574612,"statfs":{"total":96636764160,"available":95820402688,"internally_reserved":0,"allocated":414343168,"data_stored":414050930,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":401993386},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,4,8,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":413229056,"data_stored":413192926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":305876992,"data_stored":305804273,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":367685632,"data_stored":367645430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:04:44.249 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-10T16:04:44.249 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "45'13752", 'reported_seq': 13827, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:04:39.006361+0000', 'last_change': '2024-09-10T15:59:16.281558+0000', 'last_active': '2024-09-10T16:04:39.006361+0000', 'last_peered': '2024-09-10T16:04:39.006361+0000', 'last_clean': '2024-09-10T15:59:14.132456+0000', 'last_became_active': '2024-09-10T15:59:16.274407+0000', 'last_became_peered': '2024-09-10T15:59:16.274407+0000', 'last_unstale': '2024-09-10T16:04:39.006361+0000', 'last_undegraded': '2024-09-10T15:59:15.851279+0000', 'last_fullsized': '2024-09-10T15:59:15.774356+0000', 'mapping_epoch': 42, 'log_start': "36'3700", 'ondisk_log_start': "36'3700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'objects_scrubbed': 0, 'log_size': 10052, 'log_dups_size': 0, 'ondisk_log_size': 10052, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28344320, 'num_objects': 6920, 'num_object_clones': 0, 'num_object_copies': 13840, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4259, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6920, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13752, 'num_write_kb': 41344, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2661}, {'shards': '0,2', 'objects': 842}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:04:44.249 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-10T16:04:44.249 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "45'14086", 'reported_seq': 14224, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:04:39.054395+0000', 'last_change': '2024-09-10T15:59:16.308264+0000', 'last_active': '2024-09-10T16:04:39.054395+0000', 'last_peered': '2024-09-10T16:04:39.054395+0000', 'last_clean': '2024-09-10T15:59:14.111551+0000', 'last_became_active': '2024-09-10T15:59:16.300040+0000', 'last_became_peered': '2024-09-10T15:59:16.300040+0000', 'last_unstale': '2024-09-10T16:04:39.054395+0000', 'last_undegraded': '2024-09-10T15:59:15.878234+0000', 'last_fullsized': '2024-09-10T15:59:15.769328+0000', 'mapping_epoch': 42, 'log_start': "39'4000", 'ondisk_log_start': "39'4000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3442", 'last_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'objects_scrubbed': 3384, 'log_size': 10086, 'log_dups_size': 0, 'ondisk_log_size': 10086, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1292, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28570083, 'num_objects': 6976, 'num_object_clones': 0, 'num_object_copies': 13952, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4372, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6976, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 10698, 'num_write_kb': 28580, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2605}, {'shards': '0,2', 'objects': 818}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:04:44.249 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-10T16:04:44.249 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "45'13952", 'reported_seq': 14090, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:04:38.954533+0000', 'last_change': '2024-09-10T15:59:16.331899+0000', 'last_active': '2024-09-10T16:04:38.954533+0000', 'last_peered': '2024-09-10T16:04:38.954533+0000', 'last_clean': '2024-09-10T15:59:14.139436+0000', 'last_became_active': '2024-09-10T15:59:16.318715+0000', 'last_became_peered': '2024-09-10T15:59:16.318715+0000', 'last_unstale': '2024-09-10T16:04:38.954533+0000', 'last_undegraded': '2024-09-10T15:59:15.877524+0000', 'last_fullsized': '2024-09-10T15:59:15.769042+0000', 'mapping_epoch': 42, 'log_start': "39'3900", 'ondisk_log_start': "39'3900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3537", 'last_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'objects_scrubbed': 3497, 'log_size': 10052, 'log_dups_size': 0, 'ondisk_log_size': 10052, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1224, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29205446, 'num_objects': 7132, 'num_object_clones': 0, 'num_object_copies': 14264, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4237, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7132, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 10451, 'num_write_kb': 28172, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2895}, {'shards': '0,2', 'objects': 829}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:04:44.249 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-10T16:04:44.249 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "45'13596", 'reported_seq': 13736, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:04:40.209830+0000', 'last_change': '2024-09-10T15:59:16.175680+0000', 'last_active': '2024-09-10T16:04:40.209830+0000', 'last_peered': '2024-09-10T16:04:40.209830+0000', 'last_clean': '2024-09-10T15:59:14.088187+0000', 'last_became_active': '2024-09-10T15:59:16.165810+0000', 'last_became_peered': '2024-09-10T15:59:16.165810+0000', 'last_unstale': '2024-09-10T16:04:40.209830+0000', 'last_undegraded': '2024-09-10T15:59:15.822098+0000', 'last_fullsized': '2024-09-10T15:59:15.769123+0000', 'mapping_epoch': 42, 'log_start': "36'3500", 'ondisk_log_start': "36'3500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2218", 'last_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'objects_scrubbed': 2175, 'log_size': 10096, 'log_dups_size': 0, 'ondisk_log_size': 10096, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 825, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28103139, 'num_objects': 6862, 'num_object_clones': 0, 'num_object_copies': 13724, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4124, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6862, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 11415, 'num_write_kb': 32204, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 759}, {'shards': '2', 'objects': 2739}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:04:44.250 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-10T16:04:44.250 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "45'13916", 'reported_seq': 14047, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:04:39.031135+0000', 'last_change': '2024-09-10T15:59:16.277742+0000', 'last_active': '2024-09-10T16:04:39.031135+0000', 'last_peered': '2024-09-10T16:04:39.031135+0000', 'last_clean': '2024-09-10T15:59:14.129081+0000', 'last_became_active': '2024-09-10T15:59:16.269835+0000', 'last_became_peered': '2024-09-10T15:59:16.269835+0000', 'last_unstale': '2024-09-10T16:04:39.031135+0000', 'last_undegraded': '2024-09-10T15:59:15.860857+0000', 'last_fullsized': '2024-09-10T15:59:15.771410+0000', 'mapping_epoch': 42, 'log_start': "39'3900", 'ondisk_log_start': "39'3900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3557", 'last_scrub_stamp': '2024-09-10T15:59:02.493678+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:02.493678+0000', 'objects_scrubbed': 3488, 'log_size': 10016, 'log_dups_size': 0, 'ondisk_log_size': 10016, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1472, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28647424, 'num_objects': 6994, 'num_object_clones': 0, 'num_object_copies': 13988, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4257, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6994, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10420, 'num_write_kb': 27852, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2739}, {'shards': '0,2', 'objects': 798}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:04:44.250 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-10T16:04:44.250 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "45'13873", 'reported_seq': 14009, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:04:40.145102+0000', 'last_change': '2024-09-10T15:59:16.284311+0000', 'last_active': '2024-09-10T16:04:40.145102+0000', 'last_peered': '2024-09-10T16:04:40.145102+0000', 'last_clean': '2024-09-10T15:59:14.116041+0000', 'last_became_active': '2024-09-10T15:59:16.274543+0000', 'last_became_peered': '2024-09-10T15:59:16.274543+0000', 'last_unstale': '2024-09-10T16:04:40.145102+0000', 'last_undegraded': '2024-09-10T15:59:15.812484+0000', 'last_fullsized': '2024-09-10T15:59:15.770885+0000', 'mapping_epoch': 42, 'log_start': "36'3800", 'ondisk_log_start': "36'3800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2356", 'last_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'objects_scrubbed': 2316, 'log_size': 10073, 'log_dups_size': 0, 'ondisk_log_size': 10073, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 783, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28295651, 'num_objects': 6909, 'num_object_clones': 0, 'num_object_copies': 13818, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6909, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 11553, 'num_write_kb': 32292, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 824}, {'shards': '2', 'objects': 2605}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:04:44.250 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-10T16:04:44.250 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "45'14157", 'reported_seq': 14287, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:04:40.194552+0000', 'last_change': '2024-09-10T15:59:16.174113+0000', 'last_active': '2024-09-10T16:04:40.194552+0000', 'last_peered': '2024-09-10T16:04:40.194552+0000', 'last_clean': '2024-09-10T15:59:14.110654+0000', 'last_became_active': '2024-09-10T15:59:16.164645+0000', 'last_became_peered': '2024-09-10T15:59:16.164645+0000', 'last_unstale': '2024-09-10T16:04:40.194552+0000', 'last_undegraded': '2024-09-10T15:59:15.803593+0000', 'last_fullsized': '2024-09-10T15:59:15.769083+0000', 'mapping_epoch': 42, 'log_start': "39'4100", 'ondisk_log_start': "39'4100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2486", 'last_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'objects_scrubbed': 2440, 'log_size': 10057, 'log_dups_size': 0, 'ondisk_log_size': 10057, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 792, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28725248, 'num_objects': 7013, 'num_object_clones': 0, 'num_object_copies': 14026, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4354, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7013, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11711, 'num_write_kb': 32568, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 783}, {'shards': '2', 'objects': 2661}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:04:44.250 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-10T16:04:44.250 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "45'13865", 'reported_seq': 13994, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:04:39.007530+0000', 'last_change': '2024-09-10T15:59:16.315641+0000', 'last_active': '2024-09-10T16:04:39.007530+0000', 'last_peered': '2024-09-10T16:04:39.007530+0000', 'last_clean': '2024-09-10T15:59:14.117001+0000', 'last_became_active': '2024-09-10T15:59:16.302245+0000', 'last_became_peered': '2024-09-10T15:59:16.302245+0000', 'last_unstale': '2024-09-10T16:04:39.007530+0000', 'last_undegraded': '2024-09-10T15:59:15.845012+0000', 'last_fullsized': '2024-09-10T15:59:15.770337+0000', 'mapping_epoch': 42, 'log_start': "36'3800", 'ondisk_log_start': "36'3800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3404", 'last_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'objects_scrubbed': 3367, 'log_size': 10065, 'log_dups_size': 0, 'ondisk_log_size': 10065, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 983, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28561408, 'num_objects': 6973, 'num_object_clones': 0, 'num_object_copies': 13946, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6973, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10496, 'num_write_kb': 28204, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2669}, {'shards': '0,2', 'objects': 859}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:04:44.250 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-10T16:04:44.250 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "45'13678", 'reported_seq': 19808, 'reported_epoch': 45, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:04:39.077551+0000', 'last_change': '2024-09-10T16:03:29.605682+0000', 'last_active': '2024-09-10T16:04:39.077551+0000', 'last_peered': '2024-09-10T16:04:39.077551+0000', 'last_clean': '2024-09-10T15:59:14.115595+0000', 'last_became_active': '2024-09-10T15:59:16.244089+0000', 'last_became_peered': '2024-09-10T15:59:16.244089+0000', 'last_unstale': '2024-09-10T16:04:39.077551+0000', 'last_undegraded': '2024-09-10T16:03:29.603815+0000', 'last_fullsized': '2024-09-10T15:59:15.774437+0000', 'mapping_epoch': 42, 'log_start': "36'3600", 'ondisk_log_start': "36'3600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3653", 'last_scrub_stamp': '2024-09-10T15:59:05.298468+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:05.298468+0000', 'objects_scrubbed': 3602, 'log_size': 10078, 'log_dups_size': 0, 'ondisk_log_size': 10078, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1289, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28344320, 'num_objects': 6920, 'num_object_clones': 0, 'num_object_copies': 13840, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 394, 'num_objects_misplaced': 819, 'num_objects_unfound': 0, 'num_objects_dirty': 6920, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10068, 'num_write_kb': 26772, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2993, 'num_bytes_recovered': 12247040, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5707}, {'shards': '0,2', 'objects': 819}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:04:44.251 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-10T16:04:44.251 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "45'14085", 'reported_seq': 14227, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:04:39.065176+0000', 'last_change': '2024-09-10T15:59:16.311291+0000', 'last_active': '2024-09-10T16:04:39.065176+0000', 'last_peered': '2024-09-10T16:04:39.065176+0000', 'last_clean': '2024-09-10T15:59:14.116363+0000', 'last_became_active': '2024-09-10T15:59:16.301244+0000', 'last_became_peered': '2024-09-10T15:59:16.301244+0000', 'last_unstale': '2024-09-10T16:04:39.065176+0000', 'last_undegraded': '2024-09-10T15:59:15.883434+0000', 'last_fullsized': '2024-09-10T15:59:15.772756+0000', 'mapping_epoch': 42, 'log_start': "39'4000", 'ondisk_log_start': "39'4000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3789", 'last_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'objects_scrubbed': 3741, 'log_size': 10085, 'log_dups_size': 0, 'ondisk_log_size': 10085, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1440, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28520448, 'num_objects': 6963, 'num_object_clones': 0, 'num_object_copies': 13926, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4373, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6963, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10340, 'num_write_kb': 27124, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2592}, {'shards': '0,2', 'objects': 812}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:04:44.251 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-10T16:04:44.251 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "45'14097", 'reported_seq': 14242, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:04:38.928733+0000', 'last_change': '2024-09-10T15:59:16.313409+0000', 'last_active': '2024-09-10T16:04:38.928733+0000', 'last_peered': '2024-09-10T16:04:38.928733+0000', 'last_clean': '2024-09-10T15:59:14.129780+0000', 'last_became_active': '2024-09-10T15:59:16.300932+0000', 'last_became_peered': '2024-09-10T15:59:16.300932+0000', 'last_unstale': '2024-09-10T16:04:38.928733+0000', 'last_undegraded': '2024-09-10T15:59:15.887823+0000', 'last_fullsized': '2024-09-10T15:59:15.768513+0000', 'mapping_epoch': 42, 'log_start': "39'4000", 'ondisk_log_start': "39'4000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3880", 'last_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'objects_scrubbed': 3827, 'log_size': 10097, 'log_dups_size': 0, 'ondisk_log_size': 10097, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1397, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29086662, 'num_objects': 7103, 'num_object_clones': 0, 'num_object_copies': 14206, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4325, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7103, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10260, 'num_write_kb': 27072, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2780}, {'shards': '0,2', 'objects': 831}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:04:44.251 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T16:04:44.933 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:04:44.933 INFO:teuthology.orchestra.run.smithi115.stdout:{"fsid":"7effee29-df05-4351-9741-ee203564ed81","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 43305/222856 objects degraded (19.432%), 11 pgs degraded, 11 pgs undersized","count":22},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":448,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983954,"num_in_osds":3,"osd_in_since":1725983947,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":111428,"data_bytes":456360713,"bytes_used":2145648640,"bytes_avail":287764643840,"bytes_total":289910292480,"degraded_objects":43305,"degraded_total":222856,"degraded_ratio":0.19431830419643176,"misplaced_objects":819,"misplaced_total":222856,"misplaced_ratio":0.0036750188462504937,"recovering_objects_per_sec":21,"recovering_bytes_per_sec":86656,"recovering_keys_per_sec":0,"num_objects_recovered":254,"num_bytes_recovered":1040384,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":388},"fsmap":{"epoch":1,"btime":"2024-09-10T15:57:14:967120+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T16:00:13.560776+0000","services":{}},"progress_events":{}} 2024-09-10T16:04:44.952 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-10T16:04:44.953 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:04:45.517 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:04:45.518 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:04:45.539 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":262,"stamp":"2024-09-10T16:04:43.665283+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":456360713,"num_objects":111428,"num_object_clones":0,"num_object_copies":222856,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43305,"num_objects_misplaced":819,"num_objects_unfound":0,"num_objects_dirty":111428,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":180682,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7267,"num_bytes_recovered":29741075,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161109,"ondisk_log_size":161109,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2095360,"kb_used_data":1063244,"kb_used_omap":12,"kb_used_meta":1032051,"kb_avail":281020160,"statfs":{"total":289910292480,"available":287764643840,"internally_reserved":0,"allocated":1088761856,"data_stored":1087848558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1056820226},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,5,9,20,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19099648,"num_objects":-4663,"num_object_clones":0,"num_object_copies":-9326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-254,"num_objects_unfound":0,"num_objects_dirty":-4663,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4663,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":254,"num_bytes_recovered":1040384,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005800"},"pg_stats":[{"pgid":"2.f","version":"45'13752","reported_seq":13827,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:39.006361+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:04:39.006361+0000","last_peered":"2024-09-10T16:04:39.006361+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:04:39.006361+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"36'3700","ondisk_log_start":"36'3700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28344320,"num_objects":6920,"num_object_clones":0,"num_object_copies":13840,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6920,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13752,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2661},{"shards":"0,2","objects":842}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'13975","reported_seq":14086,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:04:40.220552+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:04:40.220552+0000","last_peered":"2024-09-10T16:04:40.220552+0000","last_clean":"2024-09-10T16:04:40.220552+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:04:40.220552+0000","last_undegraded":"2024-09-10T16:04:40.220552+0000","last_fullsized":"2024-09-10T16:04:40.220552+0000","mapping_epoch":35,"log_start":"36'3900","ondisk_log_start":"36'3900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28136390,"num_objects":6871,"num_object_clones":0,"num_object_copies":13742,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6871,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":12080,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'13935","reported_seq":22614,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:04:41.751850+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:04:41.751850+0000","last_peered":"2024-09-10T16:04:41.751850+0000","last_clean":"2024-09-10T16:04:41.751850+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:04:41.751850+0000","last_undegraded":"2024-09-10T16:04:41.751850+0000","last_fullsized":"2024-09-10T16:04:41.751850+0000","mapping_epoch":44,"log_start":"39'3900","ondisk_log_start":"39'3900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28106776,"num_objects":6863,"num_object_clones":0,"num_object_copies":13726,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6863,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11939,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'13784","reported_seq":13865,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:04:40.245522+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:04:40.245522+0000","last_peered":"2024-09-10T16:04:40.245522+0000","last_clean":"2024-09-10T16:04:40.245522+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:04:40.245522+0000","last_undegraded":"2024-09-10T16:04:40.245522+0000","last_fullsized":"2024-09-10T16:04:40.245522+0000","mapping_epoch":35,"log_start":"36'3700","ondisk_log_start":"36'3700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28164096,"num_objects":6876,"num_object_clones":0,"num_object_copies":13752,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6876,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13784,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'13967","reported_seq":14098,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:04:39.056108+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:04:39.056108+0000","last_peered":"2024-09-10T16:04:39.056108+0000","last_clean":"2024-09-10T16:04:39.056108+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:04:39.056108+0000","last_undegraded":"2024-09-10T16:04:39.056108+0000","last_fullsized":"2024-09-10T16:04:39.056108+0000","mapping_epoch":34,"log_start":"36'3900","ondisk_log_start":"36'3900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28361187,"num_objects":6925,"num_object_clones":0,"num_object_copies":13850,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6925,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10678,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'14086","reported_seq":14224,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:39.054395+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:04:39.054395+0000","last_peered":"2024-09-10T16:04:39.054395+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:04:39.054395+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"39'4000","ondisk_log_start":"39'4000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28570083,"num_objects":6976,"num_object_clones":0,"num_object_copies":13952,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6976,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10698,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2605},{"shards":"0,2","objects":818}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'13952","reported_seq":14090,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:38.954533+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:04:38.954533+0000","last_peered":"2024-09-10T16:04:38.954533+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:04:38.954533+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"39'3900","ondisk_log_start":"39'3900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29205446,"num_objects":7132,"num_object_clones":0,"num_object_copies":14264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7132,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":10451,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2895},{"shards":"0,2","objects":829}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'14188","reported_seq":14319,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:04:39.005460+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:04:39.005460+0000","last_peered":"2024-09-10T16:04:39.005460+0000","last_clean":"2024-09-10T16:04:39.005460+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:04:39.005460+0000","last_undegraded":"2024-09-10T16:04:39.005460+0000","last_fullsized":"2024-09-10T16:04:39.005460+0000","mapping_epoch":34,"log_start":"39'4100","ondisk_log_start":"39'4100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29188096,"num_objects":7126,"num_object_clones":0,"num_object_copies":14252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7126,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11035,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104261+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:02:08.104261+0000","last_peered":"2024-09-10T16:02:08.104261+0000","last_clean":"2024-09-10T16:02:08.104261+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:02:08.104261+0000","last_undegraded":"2024-09-10T16:02:08.104261+0000","last_fullsized":"2024-09-10T16:02:08.104261+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'13596","reported_seq":13736,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:40.209830+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:04:40.209830+0000","last_peered":"2024-09-10T16:04:40.209830+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:04:40.209830+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"36'3500","ondisk_log_start":"36'3500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28103139,"num_objects":6862,"num_object_clones":0,"num_object_copies":13724,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6862,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11415,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":759},{"shards":"2","objects":2739}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":95,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:29.860108+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:02:29.860108+0000","last_peered":"2024-09-10T16:02:29.860108+0000","last_clean":"2024-09-10T16:02:29.860108+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:02:29.860108+0000","last_undegraded":"2024-09-10T16:02:29.860108+0000","last_fullsized":"2024-09-10T16:02:29.860108+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'13916","reported_seq":14047,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:39.031135+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:04:39.031135+0000","last_peered":"2024-09-10T16:04:39.031135+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:04:39.031135+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"39'3900","ondisk_log_start":"39'3900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28647424,"num_objects":6994,"num_object_clones":0,"num_object_copies":13988,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6994,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10420,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2739},{"shards":"0,2","objects":798}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":105,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:12.405385+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:02:12.405385+0000","last_peered":"2024-09-10T16:02:12.405385+0000","last_clean":"2024-09-10T16:02:12.405385+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:02:12.405385+0000","last_undegraded":"2024-09-10T16:02:12.405385+0000","last_fullsized":"2024-09-10T16:02:12.405385+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'13873","reported_seq":14009,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:40.145102+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:04:40.145102+0000","last_peered":"2024-09-10T16:04:40.145102+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:04:40.145102+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"36'3800","ondisk_log_start":"36'3800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28295651,"num_objects":6909,"num_object_clones":0,"num_object_copies":13818,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6909,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11553,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":824},{"shards":"2","objects":2605}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104617+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:02:08.104617+0000","last_peered":"2024-09-10T16:02:08.104617+0000","last_clean":"2024-09-10T16:02:08.104617+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:02:08.104617+0000","last_undegraded":"2024-09-10T16:02:08.104617+0000","last_fullsized":"2024-09-10T16:02:08.104617+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'14157","reported_seq":14287,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:40.194552+0000","last_change":"2024-09-10T15:59:16.174113+0000","last_active":"2024-09-10T16:04:40.194552+0000","last_peered":"2024-09-10T16:04:40.194552+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:04:40.194552+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"39'4100","ondisk_log_start":"39'4100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28725248,"num_objects":7013,"num_object_clones":0,"num_object_copies":14026,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4354,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7013,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11711,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":783},{"shards":"2","objects":2661}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'13865","reported_seq":13994,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:39.007530+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:04:39.007530+0000","last_peered":"2024-09-10T16:04:39.007530+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:04:39.007530+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"36'3800","ondisk_log_start":"36'3800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28561408,"num_objects":6973,"num_object_clones":0,"num_object_copies":13946,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6973,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10496,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2669},{"shards":"0,2","objects":859}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103316+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:02:08.103316+0000","last_peered":"2024-09-10T16:02:08.103316+0000","last_clean":"2024-09-10T16:02:08.103316+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:02:08.103316+0000","last_undegraded":"2024-09-10T16:02:08.103316+0000","last_fullsized":"2024-09-10T16:02:08.103316+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'13678","reported_seq":19808,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:39.077551+0000","last_change":"2024-09-10T16:03:29.605682+0000","last_active":"2024-09-10T16:04:39.077551+0000","last_peered":"2024-09-10T16:04:39.077551+0000","last_clean":"2024-09-10T15:59:14.115595+0000","last_became_active":"2024-09-10T15:59:16.244089+0000","last_became_peered":"2024-09-10T15:59:16.244089+0000","last_unstale":"2024-09-10T16:04:39.077551+0000","last_undegraded":"2024-09-10T16:03:29.603815+0000","last_fullsized":"2024-09-10T15:59:15.774437+0000","mapping_epoch":42,"log_start":"36'3600","ondisk_log_start":"36'3600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28344320,"num_objects":6920,"num_object_clones":0,"num_object_copies":13840,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":394,"num_objects_misplaced":819,"num_objects_unfound":0,"num_objects_dirty":6920,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10068,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2993,"num_bytes_recovered":12247040,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5707},{"shards":"0,2","objects":819}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":95,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:29.877831+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:02:29.877831+0000","last_peered":"2024-09-10T16:02:29.877831+0000","last_clean":"2024-09-10T16:02:29.877831+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:02:29.877831+0000","last_undegraded":"2024-09-10T16:02:29.877831+0000","last_fullsized":"2024-09-10T16:02:29.877831+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'14085","reported_seq":14227,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:39.065176+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:04:39.065176+0000","last_peered":"2024-09-10T16:04:39.065176+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:04:39.065176+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"39'4000","ondisk_log_start":"39'4000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28520448,"num_objects":6963,"num_object_clones":0,"num_object_copies":13926,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6963,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10340,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2592},{"shards":"0,2","objects":812}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104463+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:02:08.104463+0000","last_peered":"2024-09-10T16:02:08.104463+0000","last_clean":"2024-09-10T16:02:08.104463+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:02:08.104463+0000","last_undegraded":"2024-09-10T16:02:08.104463+0000","last_fullsized":"2024-09-10T16:02:08.104463+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104448+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:02:08.104448+0000","last_peered":"2024-09-10T16:02:08.104448+0000","last_clean":"2024-09-10T16:02:08.104448+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:02:08.104448+0000","last_undegraded":"2024-09-10T16:02:08.104448+0000","last_fullsized":"2024-09-10T16:02:08.104448+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'14097","reported_seq":14242,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:38.928733+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:04:38.928733+0000","last_peered":"2024-09-10T16:04:38.928733+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:04:38.928733+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"39'4000","ondisk_log_start":"39'4000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29086662,"num_objects":7103,"num_object_clones":0,"num_object_copies":14206,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7103,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10260,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2780},{"shards":"0,2","objects":831}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":456360694,"num_objects":111426,"num_object_clones":0,"num_object_copies":222852,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43305,"num_objects_misplaced":819,"num_objects_unfound":0,"num_objects_dirty":111426,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":180680,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7265,"num_bytes_recovered":29741056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1086791680,"data_stored":1086642629,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161106,"ondisk_log_size":161106,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986256,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":688316,"kb_used_data":359532,"kb_used_omap":4,"kb_used_meta":328763,"kb_avail":93683524,"statfs":{"total":96636764160,"available":95931928576,"internally_reserved":0,"allocated":368160768,"data_stored":367868535,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":336653994},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,3,7],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659202,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":609816,"kb_used_data":299080,"kb_used_omap":4,"kb_used_meta":310715,"kb_avail":93762024,"statfs":{"total":96636764160,"available":96012312576,"internally_reserved":0,"allocated":306257920,"data_stored":305929093,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":318172846},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,5,2,5,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182484,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":797228,"kb_used_data":404632,"kb_used_omap":4,"kb_used_meta":392571,"kb_avail":93574612,"statfs":{"total":96636764160,"available":95820402688,"internally_reserved":0,"allocated":414343168,"data_stored":414050930,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":401993386},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,4,8,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":413229056,"data_stored":413192926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":305876992,"data_stored":305804273,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":367685632,"data_stored":367645430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:04:48.541 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:04:49.085 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:04:49.088 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:04:49.109 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":264,"stamp":"2024-09-10T16:04:47.666522+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":447038217,"num_objects":109152,"num_object_clones":0,"num_object_copies":218304,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43305,"num_objects_misplaced":680,"num_objects_unfound":0,"num_objects_dirty":109152,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":182958,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7406,"num_bytes_recovered":30310419,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160685,"ondisk_log_size":160685,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2066976,"kb_used_data":1051496,"kb_used_omap":12,"kb_used_meta":1015411,"kb_avail":281048544,"statfs":{"total":289910292480,"available":287793709056,"internally_reserved":0,"allocated":1076731904,"data_stored":1075818606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1039780866},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,8,2,12,7,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19136512,"num_objects":-4672,"num_object_clones":0,"num_object_copies":-9344,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-264,"num_objects_unfound":0,"num_objects_dirty":-4672,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4672,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":264,"num_bytes_recovered":1081344,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005720"},"pg_stats":[{"pgid":"2.f","version":"45'13903","reported_seq":13978,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:43.959704+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:04:43.959704+0000","last_peered":"2024-09-10T16:04:43.959704+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:04:43.959704+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"39'3900","ondisk_log_start":"39'3900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27725824,"num_objects":6769,"num_object_clones":0,"num_object_copies":13538,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6769,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13903,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2510},{"shards":"0,2","objects":691}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'14137","reported_seq":14250,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:04:45.179347+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:04:45.179347+0000","last_peered":"2024-09-10T16:04:45.179347+0000","last_clean":"2024-09-10T16:04:45.179347+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:04:45.179347+0000","last_undegraded":"2024-09-10T16:04:45.179347+0000","last_fullsized":"2024-09-10T16:04:45.179347+0000","mapping_epoch":35,"log_start":"39'4100","ondisk_log_start":"39'4100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27472838,"num_objects":6709,"num_object_clones":0,"num_object_copies":13418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6709,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":12242,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'14076","reported_seq":22755,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:04:46.943170+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:04:46.943170+0000","last_peered":"2024-09-10T16:04:46.943170+0000","last_clean":"2024-09-10T16:04:46.943170+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:04:46.943170+0000","last_undegraded":"2024-09-10T16:04:46.943170+0000","last_fullsized":"2024-09-10T16:04:46.943170+0000","mapping_epoch":44,"log_start":"39'4000","ondisk_log_start":"39'4000","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27529240,"num_objects":6722,"num_object_clones":0,"num_object_copies":13444,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6722,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12080,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'13929","reported_seq":14010,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:04:45.237939+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:04:45.237939+0000","last_peered":"2024-09-10T16:04:45.237939+0000","last_clean":"2024-09-10T16:04:45.237939+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:04:45.237939+0000","last_undegraded":"2024-09-10T16:04:45.237939+0000","last_fullsized":"2024-09-10T16:04:45.237939+0000","mapping_epoch":35,"log_start":"39'3900","ondisk_log_start":"39'3900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27570176,"num_objects":6731,"num_object_clones":0,"num_object_copies":13462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6731,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13929,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'14090","reported_seq":14221,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:04:44.105921+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:04:44.105921+0000","last_peered":"2024-09-10T16:04:44.105921+0000","last_clean":"2024-09-10T16:04:44.105921+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:04:44.105921+0000","last_undegraded":"2024-09-10T16:04:44.105921+0000","last_fullsized":"2024-09-10T16:04:44.105921+0000","mapping_epoch":34,"log_start":"39'4000","ondisk_log_start":"39'4000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27857379,"num_objects":6802,"num_object_clones":0,"num_object_copies":13604,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6802,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10801,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'14234","reported_seq":14372,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:44.073455+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:04:44.073455+0000","last_peered":"2024-09-10T16:04:44.073455+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:04:44.073455+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"39'4200","ondisk_log_start":"39'4200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27963875,"num_objects":6828,"num_object_clones":0,"num_object_copies":13656,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6828,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10846,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2457},{"shards":"0,2","objects":670}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'14080","reported_seq":14218,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:43.974941+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:04:43.974941+0000","last_peered":"2024-09-10T16:04:43.974941+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:04:43.974941+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"39'4000","ondisk_log_start":"39'4000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28681158,"num_objects":7004,"num_object_clones":0,"num_object_copies":14008,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7004,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":10579,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2767},{"shards":"0,2","objects":701}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'14336","reported_seq":14467,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:04:43.992592+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:04:43.992592+0000","last_peered":"2024-09-10T16:04:43.992592+0000","last_clean":"2024-09-10T16:04:43.992592+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:04:43.992592+0000","last_undegraded":"2024-09-10T16:04:43.992592+0000","last_fullsized":"2024-09-10T16:04:43.992592+0000","mapping_epoch":34,"log_start":"40'4300","ondisk_log_start":"40'4300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28581888,"num_objects":6978,"num_object_clones":0,"num_object_copies":13956,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6978,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11183,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104261+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:02:08.104261+0000","last_peered":"2024-09-10T16:02:08.104261+0000","last_clean":"2024-09-10T16:02:08.104261+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:02:08.104261+0000","last_undegraded":"2024-09-10T16:02:08.104261+0000","last_fullsized":"2024-09-10T16:02:08.104261+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'13742","reported_seq":13883,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:45.212208+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:04:45.212208+0000","last_peered":"2024-09-10T16:04:45.212208+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:04:45.212208+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"36'3700","ondisk_log_start":"36'3700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27505123,"num_objects":6716,"num_object_clones":0,"num_object_copies":13432,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6716,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11561,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":613},{"shards":"2","objects":2593}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":95,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:29.860108+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:02:29.860108+0000","last_peered":"2024-09-10T16:02:29.860108+0000","last_clean":"2024-09-10T16:02:29.860108+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:02:29.860108+0000","last_undegraded":"2024-09-10T16:02:29.860108+0000","last_fullsized":"2024-09-10T16:02:29.860108+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'14058","reported_seq":14189,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:43.994125+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:04:43.994125+0000","last_peered":"2024-09-10T16:04:43.994125+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:04:43.994125+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"39'4000","ondisk_log_start":"39'4000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28065792,"num_objects":6852,"num_object_clones":0,"num_object_copies":13704,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6852,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10562,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2597},{"shards":"0,2","objects":656}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":105,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:12.405385+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:02:12.405385+0000","last_peered":"2024-09-10T16:02:12.405385+0000","last_clean":"2024-09-10T16:02:12.405385+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:02:12.405385+0000","last_undegraded":"2024-09-10T16:02:12.405385+0000","last_fullsized":"2024-09-10T16:02:12.405385+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'14009","reported_seq":14142,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:45.277092+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:04:45.277092+0000","last_peered":"2024-09-10T16:04:45.277092+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:04:45.277092+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"39'4000","ondisk_log_start":"39'4000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27738595,"num_objects":6773,"num_object_clones":0,"num_object_copies":13546,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6773,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11689,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":688},{"shards":"2","objects":2469}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104617+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:02:08.104617+0000","last_peered":"2024-09-10T16:02:08.104617+0000","last_clean":"2024-09-10T16:02:08.104617+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:02:08.104617+0000","last_undegraded":"2024-09-10T16:02:08.104617+0000","last_fullsized":"2024-09-10T16:02:08.104617+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'14302","reported_seq":14432,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:45.233869+0000","last_change":"2024-09-10T15:59:16.174113+0000","last_active":"2024-09-10T16:04:45.233869+0000","last_peered":"2024-09-10T16:04:45.233869+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:04:45.233869+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"40'4300","ondisk_log_start":"40'4300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28131328,"num_objects":6868,"num_object_clones":0,"num_object_copies":13736,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4354,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6868,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11856,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":638},{"shards":"2","objects":2516}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'13993","reported_seq":14128,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:44.020206+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:04:44.020206+0000","last_peered":"2024-09-10T16:04:44.020206+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:04:44.020206+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"39'3900","ondisk_log_start":"39'3900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28037120,"num_objects":6845,"num_object_clones":0,"num_object_copies":13690,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6845,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10624,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2541},{"shards":"0,2","objects":731}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103316+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:02:08.103316+0000","last_peered":"2024-09-10T16:02:08.103316+0000","last_clean":"2024-09-10T16:02:08.103316+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:02:08.103316+0000","last_undegraded":"2024-09-10T16:02:08.103316+0000","last_fullsized":"2024-09-10T16:02:08.103316+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'13815","reported_seq":20224,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:44.105429+0000","last_change":"2024-09-10T16:03:29.605682+0000","last_active":"2024-09-10T16:04:44.105429+0000","last_peered":"2024-09-10T16:04:44.105429+0000","last_clean":"2024-09-10T15:59:14.115595+0000","last_became_active":"2024-09-10T15:59:16.244089+0000","last_became_peered":"2024-09-10T15:59:16.244089+0000","last_unstale":"2024-09-10T16:04:44.105429+0000","last_undegraded":"2024-09-10T16:03:29.603815+0000","last_fullsized":"2024-09-10T15:59:15.774437+0000","mapping_epoch":42,"log_start":"39'3800","ondisk_log_start":"39'3800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27783168,"num_objects":6783,"num_object_clones":0,"num_object_copies":13566,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":394,"num_objects_misplaced":680,"num_objects_unfound":0,"num_objects_dirty":6783,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10205,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3132,"num_bytes_recovered":12816384,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5709},{"shards":"0,2","objects":680}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":95,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:29.877831+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:02:29.877831+0000","last_peered":"2024-09-10T16:02:29.877831+0000","last_clean":"2024-09-10T16:02:29.877831+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:02:29.877831+0000","last_undegraded":"2024-09-10T16:02:29.877831+0000","last_fullsized":"2024-09-10T16:02:29.877831+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'14231","reported_seq":14374,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:44.006464+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:04:44.006464+0000","last_peered":"2024-09-10T16:04:44.006464+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:04:44.006464+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"39'4200","ondisk_log_start":"39'4200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27922432,"num_objects":6817,"num_object_clones":0,"num_object_copies":13634,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6817,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10486,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2446},{"shards":"0,2","objects":666}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104463+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:02:08.104463+0000","last_peered":"2024-09-10T16:02:08.104463+0000","last_clean":"2024-09-10T16:02:08.104463+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:02:08.104463+0000","last_undegraded":"2024-09-10T16:02:08.104463+0000","last_fullsized":"2024-09-10T16:02:08.104463+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104448+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:02:08.104448+0000","last_peered":"2024-09-10T16:02:08.104448+0000","last_clean":"2024-09-10T16:02:08.104448+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:02:08.104448+0000","last_undegraded":"2024-09-10T16:02:08.104448+0000","last_fullsized":"2024-09-10T16:02:08.104448+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'14247","reported_seq":14392,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:44.061280+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:04:44.061280+0000","last_peered":"2024-09-10T16:04:44.061280+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:04:44.061280+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"39'4200","ondisk_log_start":"39'4200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28472262,"num_objects":6953,"num_object_clones":0,"num_object_copies":13906,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6953,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10410,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2630},{"shards":"0,2","objects":681}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":447038198,"num_objects":109150,"num_object_clones":0,"num_object_copies":218300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43305,"num_objects_misplaced":680,"num_objects_unfound":0,"num_objects_dirty":109150,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":182956,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7404,"num_bytes_recovered":30310400,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1074528256,"data_stored":1074379205,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160682,"ondisk_log_size":160682,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986257,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":671260,"kb_used_data":354948,"kb_used_omap":4,"kb_used_meta":316283,"kb_avail":93700580,"statfs":{"total":96636764160,"available":95949393920,"internally_reserved":0,"allocated":363466752,"data_stored":363174519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":323874474},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,0,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659203,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":596848,"kb_used_data":298592,"kb_used_omap":4,"kb_used_meta":298235,"kb_avail":93774992,"statfs":{"total":96636764160,"available":96025591808,"internally_reserved":0,"allocated":305758208,"data_stored":305429381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":305393326},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,5,0,6,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182485,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":798868,"kb_used_data":397956,"kb_used_omap":4,"kb_used_meta":400891,"kb_avail":93572972,"statfs":{"total":96636764160,"available":95818723328,"internally_reserved":0,"allocated":407506944,"data_stored":407214706,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":410513066},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,2,5,6,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":406212608,"data_stored":406176478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":305336320,"data_stored":305263601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":362979328,"data_stored":362939126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:04:49.110 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:04:49.676 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:04:49.677 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:04:49.703 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":265,"stamp":"2024-09-10T16:04:49.670012+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":441197321,"num_objects":107726,"num_object_clones":0,"num_object_copies":215452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43305,"num_objects_misplaced":534,"num_objects_unfound":0,"num_objects_dirty":107726,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":184384,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7552,"num_bytes_recovered":30908435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160711,"ondisk_log_size":160711,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2047776,"kb_used_data":1044776,"kb_used_omap":12,"kb_used_meta":1002931,"kb_avail":281067744,"statfs":{"total":289910292480,"available":287813369856,"internally_reserved":0,"allocated":1069850624,"data_stored":1068937326,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1027001346},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,1,6,3,15,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-24383488,"num_objects":-5953,"num_object_clones":0,"num_object_copies":-11906,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-410,"num_objects_unfound":0,"num_objects_dirty":-5953,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5953,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":410,"num_bytes_recovered":1679360,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.008729"},"pg_stats":[{"pgid":"2.f","version":"45'14048","reported_seq":14123,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:49.137535+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:04:49.137535+0000","last_peered":"2024-09-10T16:04:49.137535+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:04:49.137535+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"39'4000","ondisk_log_start":"39'4000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27131904,"num_objects":6624,"num_object_clones":0,"num_object_copies":13248,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6624,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14048,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2365},{"shards":"0,2","objects":546}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'14137","reported_seq":14250,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:04:45.179347+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:04:45.179347+0000","last_peered":"2024-09-10T16:04:45.179347+0000","last_clean":"2024-09-10T16:04:45.179347+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:04:45.179347+0000","last_undegraded":"2024-09-10T16:04:45.179347+0000","last_fullsized":"2024-09-10T16:04:45.179347+0000","mapping_epoch":35,"log_start":"39'4100","ondisk_log_start":"39'4100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27472838,"num_objects":6709,"num_object_clones":0,"num_object_copies":13418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6709,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":12242,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'14076","reported_seq":22755,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:04:46.943170+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:04:46.943170+0000","last_peered":"2024-09-10T16:04:46.943170+0000","last_clean":"2024-09-10T16:04:46.943170+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:04:46.943170+0000","last_undegraded":"2024-09-10T16:04:46.943170+0000","last_fullsized":"2024-09-10T16:04:46.943170+0000","mapping_epoch":44,"log_start":"39'4000","ondisk_log_start":"39'4000","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27529240,"num_objects":6722,"num_object_clones":0,"num_object_copies":13444,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6722,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12080,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'13929","reported_seq":14010,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:04:45.237939+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:04:45.237939+0000","last_peered":"2024-09-10T16:04:45.237939+0000","last_clean":"2024-09-10T16:04:45.237939+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:04:45.237939+0000","last_undegraded":"2024-09-10T16:04:45.237939+0000","last_fullsized":"2024-09-10T16:04:45.237939+0000","mapping_epoch":35,"log_start":"39'3900","ondisk_log_start":"39'3900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27570176,"num_objects":6731,"num_object_clones":0,"num_object_copies":13462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6731,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13929,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'14230","reported_seq":14361,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:04:49.101349+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:04:49.101349+0000","last_peered":"2024-09-10T16:04:49.101349+0000","last_clean":"2024-09-10T16:04:49.101349+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:04:49.101349+0000","last_undegraded":"2024-09-10T16:04:49.101349+0000","last_fullsized":"2024-09-10T16:04:49.101349+0000","mapping_epoch":34,"log_start":"39'4200","ondisk_log_start":"39'4200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27283939,"num_objects":6662,"num_object_clones":0,"num_object_copies":13324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6662,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10941,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'14394","reported_seq":14532,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:49.109144+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:04:49.109144+0000","last_peered":"2024-09-10T16:04:49.109144+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:04:49.109144+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"39'4300","ondisk_log_start":"39'4300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27308515,"num_objects":6668,"num_object_clones":0,"num_object_copies":13336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6668,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11006,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2297},{"shards":"0,2","objects":510}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'14231","reported_seq":14370,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:49.113269+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:04:49.113269+0000","last_peered":"2024-09-10T16:04:49.113269+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:04:49.113269+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"40'4200","ondisk_log_start":"40'4200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28062662,"num_objects":6853,"num_object_clones":0,"num_object_copies":13706,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6853,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":10730,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2616},{"shards":"0,2","objects":550}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'14468","reported_seq":14598,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:04:49.148650+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:04:49.148650+0000","last_peered":"2024-09-10T16:04:49.148650+0000","last_clean":"2024-09-10T16:04:49.148650+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:04:49.148650+0000","last_undegraded":"2024-09-10T16:04:49.148650+0000","last_fullsized":"2024-09-10T16:04:49.148650+0000","mapping_epoch":34,"log_start":"43'4400","ondisk_log_start":"43'4400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28041216,"num_objects":6846,"num_object_clones":0,"num_object_copies":13692,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6846,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11315,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104261+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:02:08.104261+0000","last_peered":"2024-09-10T16:02:08.104261+0000","last_clean":"2024-09-10T16:02:08.104261+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:02:08.104261+0000","last_undegraded":"2024-09-10T16:02:08.104261+0000","last_fullsized":"2024-09-10T16:02:08.104261+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'13742","reported_seq":13883,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:45.212208+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:04:45.212208+0000","last_peered":"2024-09-10T16:04:45.212208+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:04:45.212208+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"36'3700","ondisk_log_start":"36'3700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27505123,"num_objects":6716,"num_object_clones":0,"num_object_copies":13432,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6716,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11561,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":613},{"shards":"2","objects":2593}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":95,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:29.860108+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:02:29.860108+0000","last_peered":"2024-09-10T16:02:29.860108+0000","last_clean":"2024-09-10T16:02:29.860108+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:02:29.860108+0000","last_undegraded":"2024-09-10T16:02:29.860108+0000","last_fullsized":"2024-09-10T16:02:29.860108+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'14207","reported_seq":14335,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:49.105116+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:04:49.105116+0000","last_peered":"2024-09-10T16:04:49.105116+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:04:49.105116+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"40'4200","ondisk_log_start":"40'4200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27455488,"num_objects":6703,"num_object_clones":0,"num_object_copies":13406,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6703,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10711,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2448},{"shards":"0,2","objects":507}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":105,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:12.405385+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:02:12.405385+0000","last_peered":"2024-09-10T16:02:12.405385+0000","last_clean":"2024-09-10T16:02:12.405385+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:02:12.405385+0000","last_undegraded":"2024-09-10T16:02:12.405385+0000","last_fullsized":"2024-09-10T16:02:12.405385+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'14009","reported_seq":14142,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:45.277092+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:04:45.277092+0000","last_peered":"2024-09-10T16:04:45.277092+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:04:45.277092+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"39'4000","ondisk_log_start":"39'4000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27738595,"num_objects":6773,"num_object_clones":0,"num_object_copies":13546,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6773,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11689,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":688},{"shards":"2","objects":2469}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104617+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:02:08.104617+0000","last_peered":"2024-09-10T16:02:08.104617+0000","last_clean":"2024-09-10T16:02:08.104617+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:02:08.104617+0000","last_undegraded":"2024-09-10T16:02:08.104617+0000","last_fullsized":"2024-09-10T16:02:08.104617+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'14302","reported_seq":14432,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:45.233869+0000","last_change":"2024-09-10T15:59:16.174113+0000","last_active":"2024-09-10T16:04:45.233869+0000","last_peered":"2024-09-10T16:04:45.233869+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:04:45.233869+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"40'4300","ondisk_log_start":"40'4300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28131328,"num_objects":6868,"num_object_clones":0,"num_object_copies":13736,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4354,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6868,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11856,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":638},{"shards":"2","objects":2516}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'14132","reported_seq":14267,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:49.145520+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:04:49.145520+0000","last_peered":"2024-09-10T16:04:49.145520+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:04:49.145520+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"39'4100","ondisk_log_start":"39'4100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27467776,"num_objects":6706,"num_object_clones":0,"num_object_copies":13412,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6706,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10763,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2402},{"shards":"0,2","objects":592}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103316+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:02:08.103316+0000","last_peered":"2024-09-10T16:02:08.103316+0000","last_clean":"2024-09-10T16:02:08.103316+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:02:08.103316+0000","last_undegraded":"2024-09-10T16:02:08.103316+0000","last_fullsized":"2024-09-10T16:02:08.103316+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'13948","reported_seq":20649,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:49.147199+0000","last_change":"2024-09-10T16:03:29.605682+0000","last_active":"2024-09-10T16:04:49.147199+0000","last_peered":"2024-09-10T16:04:49.147199+0000","last_clean":"2024-09-10T15:59:14.115595+0000","last_became_active":"2024-09-10T15:59:16.244089+0000","last_became_peered":"2024-09-10T15:59:16.244089+0000","last_unstale":"2024-09-10T16:04:49.147199+0000","last_undegraded":"2024-09-10T16:03:29.603815+0000","last_fullsized":"2024-09-10T15:59:15.774437+0000","mapping_epoch":42,"log_start":"39'3900","ondisk_log_start":"39'3900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27238400,"num_objects":6650,"num_object_clones":0,"num_object_copies":13300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":394,"num_objects_misplaced":534,"num_objects_unfound":0,"num_objects_dirty":6650,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10338,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3278,"num_bytes_recovered":13414400,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5722},{"shards":"0,2","objects":534}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":95,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:29.877831+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:02:29.877831+0000","last_peered":"2024-09-10T16:02:29.877831+0000","last_clean":"2024-09-10T16:02:29.877831+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:02:29.877831+0000","last_undegraded":"2024-09-10T16:02:29.877831+0000","last_fullsized":"2024-09-10T16:02:29.877831+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'14357","reported_seq":14500,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:49.000088+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:04:49.000088+0000","last_peered":"2024-09-10T16:04:49.000088+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:04:49.000088+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"40'4300","ondisk_log_start":"40'4300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27406336,"num_objects":6691,"num_object_clones":0,"num_object_copies":13382,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6691,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10612,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2320},{"shards":"0,2","objects":540}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104463+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:02:08.104463+0000","last_peered":"2024-09-10T16:02:08.104463+0000","last_clean":"2024-09-10T16:02:08.104463+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:02:08.104463+0000","last_undegraded":"2024-09-10T16:02:08.104463+0000","last_fullsized":"2024-09-10T16:02:08.104463+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104448+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:02:08.104448+0000","last_peered":"2024-09-10T16:02:08.104448+0000","last_clean":"2024-09-10T16:02:08.104448+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:02:08.104448+0000","last_undegraded":"2024-09-10T16:02:08.104448+0000","last_fullsized":"2024-09-10T16:02:08.104448+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'14398","reported_seq":14543,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:49.072188+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:04:49.072188+0000","last_peered":"2024-09-10T16:04:49.072188+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:04:49.072188+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"40'4300","ondisk_log_start":"40'4300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27853766,"num_objects":6802,"num_object_clones":0,"num_object_copies":13604,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6802,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10561,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2479},{"shards":"0,2","objects":530}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":441197302,"num_objects":107724,"num_object_clones":0,"num_object_copies":215448,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43305,"num_objects_misplaced":534,"num_objects_unfound":0,"num_objects_dirty":107724,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":184382,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7550,"num_bytes_recovered":30908416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1067651072,"data_stored":1067502021,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160708,"ondisk_log_size":160708,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986257,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":671260,"kb_used_data":354948,"kb_used_omap":4,"kb_used_meta":316283,"kb_avail":93700580,"statfs":{"total":96636764160,"available":95949393920,"internally_reserved":0,"allocated":363466752,"data_stored":363174519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":323874474},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,0,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659203,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":596848,"kb_used_data":298592,"kb_used_omap":4,"kb_used_meta":298235,"kb_avail":93774992,"statfs":{"total":96636764160,"available":96025591808,"internally_reserved":0,"allocated":305758208,"data_stored":305429381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":305393326},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,5,0,6,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182486,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":779668,"kb_used_data":391236,"kb_used_omap":4,"kb_used_meta":388411,"kb_avail":93592172,"statfs":{"total":96636764160,"available":95838384128,"internally_reserved":0,"allocated":400625664,"data_stored":400333426,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":397733546},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,0,3,8,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":399335424,"data_stored":399299294,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":305336320,"data_stored":305263601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":362979328,"data_stored":362939126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:04:49.704 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-10T16:04:49.705 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "45'14048", 'reported_seq': 14123, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:04:49.137535+0000', 'last_change': '2024-09-10T15:59:16.281558+0000', 'last_active': '2024-09-10T16:04:49.137535+0000', 'last_peered': '2024-09-10T16:04:49.137535+0000', 'last_clean': '2024-09-10T15:59:14.132456+0000', 'last_became_active': '2024-09-10T15:59:16.274407+0000', 'last_became_peered': '2024-09-10T15:59:16.274407+0000', 'last_unstale': '2024-09-10T16:04:49.137535+0000', 'last_undegraded': '2024-09-10T15:59:15.851279+0000', 'last_fullsized': '2024-09-10T15:59:15.774356+0000', 'mapping_epoch': 42, 'log_start': "39'4000", 'ondisk_log_start': "39'4000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'objects_scrubbed': 0, 'log_size': 10048, 'log_dups_size': 0, 'ondisk_log_size': 10048, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27131904, 'num_objects': 6624, 'num_object_clones': 0, 'num_object_copies': 13248, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4259, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6624, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14048, 'num_write_kb': 41344, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2365}, {'shards': '0,2', 'objects': 546}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:04:49.705 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-10T16:04:49.705 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "45'14394", 'reported_seq': 14532, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:04:49.109144+0000', 'last_change': '2024-09-10T15:59:16.308264+0000', 'last_active': '2024-09-10T16:04:49.109144+0000', 'last_peered': '2024-09-10T16:04:49.109144+0000', 'last_clean': '2024-09-10T15:59:14.111551+0000', 'last_became_active': '2024-09-10T15:59:16.300040+0000', 'last_became_peered': '2024-09-10T15:59:16.300040+0000', 'last_unstale': '2024-09-10T16:04:49.109144+0000', 'last_undegraded': '2024-09-10T15:59:15.878234+0000', 'last_fullsized': '2024-09-10T15:59:15.769328+0000', 'mapping_epoch': 42, 'log_start': "39'4300", 'ondisk_log_start': "39'4300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3442", 'last_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'objects_scrubbed': 3384, 'log_size': 10094, 'log_dups_size': 0, 'ondisk_log_size': 10094, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1292, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27308515, 'num_objects': 6668, 'num_object_clones': 0, 'num_object_copies': 13336, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4372, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6668, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 11006, 'num_write_kb': 28580, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2297}, {'shards': '0,2', 'objects': 510}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:04:49.705 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-10T16:04:49.705 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "45'14231", 'reported_seq': 14370, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:04:49.113269+0000', 'last_change': '2024-09-10T15:59:16.331899+0000', 'last_active': '2024-09-10T16:04:49.113269+0000', 'last_peered': '2024-09-10T16:04:49.113269+0000', 'last_clean': '2024-09-10T15:59:14.139436+0000', 'last_became_active': '2024-09-10T15:59:16.318715+0000', 'last_became_peered': '2024-09-10T15:59:16.318715+0000', 'last_unstale': '2024-09-10T16:04:49.113269+0000', 'last_undegraded': '2024-09-10T15:59:15.877524+0000', 'last_fullsized': '2024-09-10T15:59:15.769042+0000', 'mapping_epoch': 42, 'log_start': "40'4200", 'ondisk_log_start': "40'4200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3537", 'last_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'objects_scrubbed': 3497, 'log_size': 10031, 'log_dups_size': 0, 'ondisk_log_size': 10031, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1224, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28062662, 'num_objects': 6853, 'num_object_clones': 0, 'num_object_copies': 13706, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4237, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6853, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 10730, 'num_write_kb': 28172, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2616}, {'shards': '0,2', 'objects': 550}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:04:49.705 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-10T16:04:49.705 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "45'13742", 'reported_seq': 13883, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:04:45.212208+0000', 'last_change': '2024-09-10T15:59:16.175680+0000', 'last_active': '2024-09-10T16:04:45.212208+0000', 'last_peered': '2024-09-10T16:04:45.212208+0000', 'last_clean': '2024-09-10T15:59:14.088187+0000', 'last_became_active': '2024-09-10T15:59:16.165810+0000', 'last_became_peered': '2024-09-10T15:59:16.165810+0000', 'last_unstale': '2024-09-10T16:04:45.212208+0000', 'last_undegraded': '2024-09-10T15:59:15.822098+0000', 'last_fullsized': '2024-09-10T15:59:15.769123+0000', 'mapping_epoch': 42, 'log_start': "36'3700", 'ondisk_log_start': "36'3700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2218", 'last_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'objects_scrubbed': 2175, 'log_size': 10042, 'log_dups_size': 0, 'ondisk_log_size': 10042, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 825, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27505123, 'num_objects': 6716, 'num_object_clones': 0, 'num_object_copies': 13432, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4124, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6716, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 11561, 'num_write_kb': 32204, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 613}, {'shards': '2', 'objects': 2593}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:04:49.705 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-10T16:04:49.705 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "45'14207", 'reported_seq': 14335, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:04:49.105116+0000', 'last_change': '2024-09-10T15:59:16.277742+0000', 'last_active': '2024-09-10T16:04:49.105116+0000', 'last_peered': '2024-09-10T16:04:49.105116+0000', 'last_clean': '2024-09-10T15:59:14.129081+0000', 'last_became_active': '2024-09-10T15:59:16.269835+0000', 'last_became_peered': '2024-09-10T15:59:16.269835+0000', 'last_unstale': '2024-09-10T16:04:49.105116+0000', 'last_undegraded': '2024-09-10T15:59:15.860857+0000', 'last_fullsized': '2024-09-10T15:59:15.771410+0000', 'mapping_epoch': 42, 'log_start': "40'4200", 'ondisk_log_start': "40'4200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3557", 'last_scrub_stamp': '2024-09-10T15:59:02.493678+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:02.493678+0000', 'objects_scrubbed': 3488, 'log_size': 10007, 'log_dups_size': 0, 'ondisk_log_size': 10007, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1472, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27455488, 'num_objects': 6703, 'num_object_clones': 0, 'num_object_copies': 13406, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4257, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6703, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10711, 'num_write_kb': 27852, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2448}, {'shards': '0,2', 'objects': 507}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:04:49.705 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-10T16:04:49.706 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "45'14009", 'reported_seq': 14142, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:04:45.277092+0000', 'last_change': '2024-09-10T15:59:16.284311+0000', 'last_active': '2024-09-10T16:04:45.277092+0000', 'last_peered': '2024-09-10T16:04:45.277092+0000', 'last_clean': '2024-09-10T15:59:14.116041+0000', 'last_became_active': '2024-09-10T15:59:16.274543+0000', 'last_became_peered': '2024-09-10T15:59:16.274543+0000', 'last_unstale': '2024-09-10T16:04:45.277092+0000', 'last_undegraded': '2024-09-10T15:59:15.812484+0000', 'last_fullsized': '2024-09-10T15:59:15.770885+0000', 'mapping_epoch': 42, 'log_start': "39'4000", 'ondisk_log_start': "39'4000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2356", 'last_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'objects_scrubbed': 2316, 'log_size': 10009, 'log_dups_size': 0, 'ondisk_log_size': 10009, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 783, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27738595, 'num_objects': 6773, 'num_object_clones': 0, 'num_object_copies': 13546, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6773, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 11689, 'num_write_kb': 32292, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 688}, {'shards': '2', 'objects': 2469}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:04:49.706 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-10T16:04:49.706 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "45'14302", 'reported_seq': 14432, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:04:45.233869+0000', 'last_change': '2024-09-10T15:59:16.174113+0000', 'last_active': '2024-09-10T16:04:45.233869+0000', 'last_peered': '2024-09-10T16:04:45.233869+0000', 'last_clean': '2024-09-10T15:59:14.110654+0000', 'last_became_active': '2024-09-10T15:59:16.164645+0000', 'last_became_peered': '2024-09-10T15:59:16.164645+0000', 'last_unstale': '2024-09-10T16:04:45.233869+0000', 'last_undegraded': '2024-09-10T15:59:15.803593+0000', 'last_fullsized': '2024-09-10T15:59:15.769083+0000', 'mapping_epoch': 42, 'log_start': "40'4300", 'ondisk_log_start': "40'4300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2486", 'last_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'objects_scrubbed': 2440, 'log_size': 10002, 'log_dups_size': 0, 'ondisk_log_size': 10002, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 792, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28131328, 'num_objects': 6868, 'num_object_clones': 0, 'num_object_copies': 13736, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4354, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6868, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11856, 'num_write_kb': 32568, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 638}, {'shards': '2', 'objects': 2516}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:04:49.706 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-10T16:04:49.706 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "45'14132", 'reported_seq': 14267, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:04:49.145520+0000', 'last_change': '2024-09-10T15:59:16.315641+0000', 'last_active': '2024-09-10T16:04:49.145520+0000', 'last_peered': '2024-09-10T16:04:49.145520+0000', 'last_clean': '2024-09-10T15:59:14.117001+0000', 'last_became_active': '2024-09-10T15:59:16.302245+0000', 'last_became_peered': '2024-09-10T15:59:16.302245+0000', 'last_unstale': '2024-09-10T16:04:49.145520+0000', 'last_undegraded': '2024-09-10T15:59:15.845012+0000', 'last_fullsized': '2024-09-10T15:59:15.770337+0000', 'mapping_epoch': 42, 'log_start': "39'4100", 'ondisk_log_start': "39'4100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3404", 'last_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'objects_scrubbed': 3367, 'log_size': 10032, 'log_dups_size': 0, 'ondisk_log_size': 10032, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 983, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27467776, 'num_objects': 6706, 'num_object_clones': 0, 'num_object_copies': 13412, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6706, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10763, 'num_write_kb': 28204, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2402}, {'shards': '0,2', 'objects': 592}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:04:49.706 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-10T16:04:49.706 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "45'13948", 'reported_seq': 20649, 'reported_epoch': 45, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:04:49.147199+0000', 'last_change': '2024-09-10T16:03:29.605682+0000', 'last_active': '2024-09-10T16:04:49.147199+0000', 'last_peered': '2024-09-10T16:04:49.147199+0000', 'last_clean': '2024-09-10T15:59:14.115595+0000', 'last_became_active': '2024-09-10T15:59:16.244089+0000', 'last_became_peered': '2024-09-10T15:59:16.244089+0000', 'last_unstale': '2024-09-10T16:04:49.147199+0000', 'last_undegraded': '2024-09-10T16:03:29.603815+0000', 'last_fullsized': '2024-09-10T15:59:15.774437+0000', 'mapping_epoch': 42, 'log_start': "39'3900", 'ondisk_log_start': "39'3900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3653", 'last_scrub_stamp': '2024-09-10T15:59:05.298468+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:05.298468+0000', 'objects_scrubbed': 3602, 'log_size': 10048, 'log_dups_size': 0, 'ondisk_log_size': 10048, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1289, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27238400, 'num_objects': 6650, 'num_object_clones': 0, 'num_object_copies': 13300, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 394, 'num_objects_misplaced': 534, 'num_objects_unfound': 0, 'num_objects_dirty': 6650, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10338, 'num_write_kb': 26772, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3278, 'num_bytes_recovered': 13414400, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5722}, {'shards': '0,2', 'objects': 534}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:04:49.706 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-10T16:04:49.706 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "45'14357", 'reported_seq': 14500, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:04:49.000088+0000', 'last_change': '2024-09-10T15:59:16.311291+0000', 'last_active': '2024-09-10T16:04:49.000088+0000', 'last_peered': '2024-09-10T16:04:49.000088+0000', 'last_clean': '2024-09-10T15:59:14.116363+0000', 'last_became_active': '2024-09-10T15:59:16.301244+0000', 'last_became_peered': '2024-09-10T15:59:16.301244+0000', 'last_unstale': '2024-09-10T16:04:49.000088+0000', 'last_undegraded': '2024-09-10T15:59:15.883434+0000', 'last_fullsized': '2024-09-10T15:59:15.772756+0000', 'mapping_epoch': 42, 'log_start': "40'4300", 'ondisk_log_start': "40'4300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3789", 'last_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'objects_scrubbed': 3741, 'log_size': 10057, 'log_dups_size': 0, 'ondisk_log_size': 10057, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1440, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27406336, 'num_objects': 6691, 'num_object_clones': 0, 'num_object_copies': 13382, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4373, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6691, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10612, 'num_write_kb': 27124, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2320}, {'shards': '0,2', 'objects': 540}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:04:49.706 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-10T16:04:49.707 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "45'14398", 'reported_seq': 14543, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:04:49.072188+0000', 'last_change': '2024-09-10T15:59:16.313409+0000', 'last_active': '2024-09-10T16:04:49.072188+0000', 'last_peered': '2024-09-10T16:04:49.072188+0000', 'last_clean': '2024-09-10T15:59:14.129780+0000', 'last_became_active': '2024-09-10T15:59:16.300932+0000', 'last_became_peered': '2024-09-10T15:59:16.300932+0000', 'last_unstale': '2024-09-10T16:04:49.072188+0000', 'last_undegraded': '2024-09-10T15:59:15.887823+0000', 'last_fullsized': '2024-09-10T15:59:15.768513+0000', 'mapping_epoch': 42, 'log_start': "40'4300", 'ondisk_log_start': "40'4300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3880", 'last_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'objects_scrubbed': 3827, 'log_size': 10098, 'log_dups_size': 0, 'ondisk_log_size': 10098, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1397, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27853766, 'num_objects': 6802, 'num_object_clones': 0, 'num_object_copies': 13604, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4325, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6802, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10561, 'num_write_kb': 27072, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2479}, {'shards': '0,2', 'objects': 530}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:04:49.707 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T16:04:50.374 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:04:50.374 INFO:teuthology.orchestra.run.smithi115.stdout:{"fsid":"7effee29-df05-4351-9741-ee203564ed81","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 43305/218304 objects degraded (19.837%), 11 pgs degraded, 11 pgs undersized","count":22},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":454,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983954,"num_in_osds":3,"osd_in_since":1725983947,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":109152,"data_bytes":447038217,"bytes_used":2116583424,"bytes_avail":287793709056,"bytes_total":289910292480,"degraded_objects":43305,"degraded_total":218304,"degraded_ratio":0.19837016270888302,"misplaced_objects":680,"misplaced_total":218304,"misplaced_ratio":0.0031149223101729693,"recovering_objects_per_sec":21,"recovering_bytes_per_sec":90069,"recovering_keys_per_sec":0,"num_objects_recovered":264,"num_bytes_recovered":1081344,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":389},"fsmap":{"epoch":1,"btime":"2024-09-10T15:57:14:967120+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T16:00:13.560776+0000","services":{}},"progress_events":{}} 2024-09-10T16:04:50.403 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-10T16:04:50.404 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:04:50.963 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:04:50.965 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:04:50.993 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":265,"stamp":"2024-09-10T16:04:49.670012+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":441197321,"num_objects":107726,"num_object_clones":0,"num_object_copies":215452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43305,"num_objects_misplaced":534,"num_objects_unfound":0,"num_objects_dirty":107726,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":184384,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7552,"num_bytes_recovered":30908435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160711,"ondisk_log_size":160711,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2047776,"kb_used_data":1044776,"kb_used_omap":12,"kb_used_meta":1002931,"kb_avail":281067744,"statfs":{"total":289910292480,"available":287813369856,"internally_reserved":0,"allocated":1069850624,"data_stored":1068937326,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1027001346},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,1,6,3,15,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-24383488,"num_objects":-5953,"num_object_clones":0,"num_object_copies":-11906,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-410,"num_objects_unfound":0,"num_objects_dirty":-5953,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5953,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":410,"num_bytes_recovered":1679360,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.008729"},"pg_stats":[{"pgid":"2.f","version":"45'14048","reported_seq":14123,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:49.137535+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:04:49.137535+0000","last_peered":"2024-09-10T16:04:49.137535+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:04:49.137535+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"39'4000","ondisk_log_start":"39'4000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27131904,"num_objects":6624,"num_object_clones":0,"num_object_copies":13248,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6624,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14048,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2365},{"shards":"0,2","objects":546}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'14137","reported_seq":14250,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:04:45.179347+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:04:45.179347+0000","last_peered":"2024-09-10T16:04:45.179347+0000","last_clean":"2024-09-10T16:04:45.179347+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:04:45.179347+0000","last_undegraded":"2024-09-10T16:04:45.179347+0000","last_fullsized":"2024-09-10T16:04:45.179347+0000","mapping_epoch":35,"log_start":"39'4100","ondisk_log_start":"39'4100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27472838,"num_objects":6709,"num_object_clones":0,"num_object_copies":13418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6709,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":12242,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'14076","reported_seq":22755,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:04:46.943170+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:04:46.943170+0000","last_peered":"2024-09-10T16:04:46.943170+0000","last_clean":"2024-09-10T16:04:46.943170+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:04:46.943170+0000","last_undegraded":"2024-09-10T16:04:46.943170+0000","last_fullsized":"2024-09-10T16:04:46.943170+0000","mapping_epoch":44,"log_start":"39'4000","ondisk_log_start":"39'4000","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27529240,"num_objects":6722,"num_object_clones":0,"num_object_copies":13444,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6722,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12080,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'13929","reported_seq":14010,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:04:45.237939+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:04:45.237939+0000","last_peered":"2024-09-10T16:04:45.237939+0000","last_clean":"2024-09-10T16:04:45.237939+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:04:45.237939+0000","last_undegraded":"2024-09-10T16:04:45.237939+0000","last_fullsized":"2024-09-10T16:04:45.237939+0000","mapping_epoch":35,"log_start":"39'3900","ondisk_log_start":"39'3900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27570176,"num_objects":6731,"num_object_clones":0,"num_object_copies":13462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6731,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13929,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'14230","reported_seq":14361,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:04:49.101349+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:04:49.101349+0000","last_peered":"2024-09-10T16:04:49.101349+0000","last_clean":"2024-09-10T16:04:49.101349+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:04:49.101349+0000","last_undegraded":"2024-09-10T16:04:49.101349+0000","last_fullsized":"2024-09-10T16:04:49.101349+0000","mapping_epoch":34,"log_start":"39'4200","ondisk_log_start":"39'4200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27283939,"num_objects":6662,"num_object_clones":0,"num_object_copies":13324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6662,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10941,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'14394","reported_seq":14532,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:49.109144+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:04:49.109144+0000","last_peered":"2024-09-10T16:04:49.109144+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:04:49.109144+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"39'4300","ondisk_log_start":"39'4300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27308515,"num_objects":6668,"num_object_clones":0,"num_object_copies":13336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6668,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11006,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2297},{"shards":"0,2","objects":510}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'14231","reported_seq":14370,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:49.113269+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:04:49.113269+0000","last_peered":"2024-09-10T16:04:49.113269+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:04:49.113269+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"40'4200","ondisk_log_start":"40'4200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28062662,"num_objects":6853,"num_object_clones":0,"num_object_copies":13706,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6853,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":10730,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2616},{"shards":"0,2","objects":550}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'14468","reported_seq":14598,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:04:49.148650+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:04:49.148650+0000","last_peered":"2024-09-10T16:04:49.148650+0000","last_clean":"2024-09-10T16:04:49.148650+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:04:49.148650+0000","last_undegraded":"2024-09-10T16:04:49.148650+0000","last_fullsized":"2024-09-10T16:04:49.148650+0000","mapping_epoch":34,"log_start":"43'4400","ondisk_log_start":"43'4400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28041216,"num_objects":6846,"num_object_clones":0,"num_object_copies":13692,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6846,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11315,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104261+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:02:08.104261+0000","last_peered":"2024-09-10T16:02:08.104261+0000","last_clean":"2024-09-10T16:02:08.104261+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:02:08.104261+0000","last_undegraded":"2024-09-10T16:02:08.104261+0000","last_fullsized":"2024-09-10T16:02:08.104261+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'13742","reported_seq":13883,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:45.212208+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:04:45.212208+0000","last_peered":"2024-09-10T16:04:45.212208+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:04:45.212208+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"36'3700","ondisk_log_start":"36'3700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27505123,"num_objects":6716,"num_object_clones":0,"num_object_copies":13432,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6716,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11561,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":613},{"shards":"2","objects":2593}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":95,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:29.860108+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:02:29.860108+0000","last_peered":"2024-09-10T16:02:29.860108+0000","last_clean":"2024-09-10T16:02:29.860108+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:02:29.860108+0000","last_undegraded":"2024-09-10T16:02:29.860108+0000","last_fullsized":"2024-09-10T16:02:29.860108+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'14207","reported_seq":14335,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:49.105116+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:04:49.105116+0000","last_peered":"2024-09-10T16:04:49.105116+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:04:49.105116+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"40'4200","ondisk_log_start":"40'4200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27455488,"num_objects":6703,"num_object_clones":0,"num_object_copies":13406,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6703,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10711,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2448},{"shards":"0,2","objects":507}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":105,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:12.405385+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:02:12.405385+0000","last_peered":"2024-09-10T16:02:12.405385+0000","last_clean":"2024-09-10T16:02:12.405385+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:02:12.405385+0000","last_undegraded":"2024-09-10T16:02:12.405385+0000","last_fullsized":"2024-09-10T16:02:12.405385+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'14009","reported_seq":14142,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:45.277092+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:04:45.277092+0000","last_peered":"2024-09-10T16:04:45.277092+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:04:45.277092+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"39'4000","ondisk_log_start":"39'4000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27738595,"num_objects":6773,"num_object_clones":0,"num_object_copies":13546,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6773,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11689,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":688},{"shards":"2","objects":2469}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104617+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:02:08.104617+0000","last_peered":"2024-09-10T16:02:08.104617+0000","last_clean":"2024-09-10T16:02:08.104617+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:02:08.104617+0000","last_undegraded":"2024-09-10T16:02:08.104617+0000","last_fullsized":"2024-09-10T16:02:08.104617+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'14302","reported_seq":14432,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:45.233869+0000","last_change":"2024-09-10T15:59:16.174113+0000","last_active":"2024-09-10T16:04:45.233869+0000","last_peered":"2024-09-10T16:04:45.233869+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:04:45.233869+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"40'4300","ondisk_log_start":"40'4300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28131328,"num_objects":6868,"num_object_clones":0,"num_object_copies":13736,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4354,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6868,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11856,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":638},{"shards":"2","objects":2516}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'14132","reported_seq":14267,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:49.145520+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:04:49.145520+0000","last_peered":"2024-09-10T16:04:49.145520+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:04:49.145520+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"39'4100","ondisk_log_start":"39'4100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27467776,"num_objects":6706,"num_object_clones":0,"num_object_copies":13412,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6706,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10763,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2402},{"shards":"0,2","objects":592}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103316+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:02:08.103316+0000","last_peered":"2024-09-10T16:02:08.103316+0000","last_clean":"2024-09-10T16:02:08.103316+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:02:08.103316+0000","last_undegraded":"2024-09-10T16:02:08.103316+0000","last_fullsized":"2024-09-10T16:02:08.103316+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'13948","reported_seq":20649,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:49.147199+0000","last_change":"2024-09-10T16:03:29.605682+0000","last_active":"2024-09-10T16:04:49.147199+0000","last_peered":"2024-09-10T16:04:49.147199+0000","last_clean":"2024-09-10T15:59:14.115595+0000","last_became_active":"2024-09-10T15:59:16.244089+0000","last_became_peered":"2024-09-10T15:59:16.244089+0000","last_unstale":"2024-09-10T16:04:49.147199+0000","last_undegraded":"2024-09-10T16:03:29.603815+0000","last_fullsized":"2024-09-10T15:59:15.774437+0000","mapping_epoch":42,"log_start":"39'3900","ondisk_log_start":"39'3900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27238400,"num_objects":6650,"num_object_clones":0,"num_object_copies":13300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":394,"num_objects_misplaced":534,"num_objects_unfound":0,"num_objects_dirty":6650,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10338,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3278,"num_bytes_recovered":13414400,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5722},{"shards":"0,2","objects":534}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":95,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:29.877831+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:02:29.877831+0000","last_peered":"2024-09-10T16:02:29.877831+0000","last_clean":"2024-09-10T16:02:29.877831+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:02:29.877831+0000","last_undegraded":"2024-09-10T16:02:29.877831+0000","last_fullsized":"2024-09-10T16:02:29.877831+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'14357","reported_seq":14500,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:49.000088+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:04:49.000088+0000","last_peered":"2024-09-10T16:04:49.000088+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:04:49.000088+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"40'4300","ondisk_log_start":"40'4300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27406336,"num_objects":6691,"num_object_clones":0,"num_object_copies":13382,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6691,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10612,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2320},{"shards":"0,2","objects":540}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104463+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:02:08.104463+0000","last_peered":"2024-09-10T16:02:08.104463+0000","last_clean":"2024-09-10T16:02:08.104463+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:02:08.104463+0000","last_undegraded":"2024-09-10T16:02:08.104463+0000","last_fullsized":"2024-09-10T16:02:08.104463+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104448+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:02:08.104448+0000","last_peered":"2024-09-10T16:02:08.104448+0000","last_clean":"2024-09-10T16:02:08.104448+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:02:08.104448+0000","last_undegraded":"2024-09-10T16:02:08.104448+0000","last_fullsized":"2024-09-10T16:02:08.104448+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'14398","reported_seq":14543,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:49.072188+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:04:49.072188+0000","last_peered":"2024-09-10T16:04:49.072188+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:04:49.072188+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"40'4300","ondisk_log_start":"40'4300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27853766,"num_objects":6802,"num_object_clones":0,"num_object_copies":13604,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6802,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10561,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2479},{"shards":"0,2","objects":530}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":441197302,"num_objects":107724,"num_object_clones":0,"num_object_copies":215448,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43305,"num_objects_misplaced":534,"num_objects_unfound":0,"num_objects_dirty":107724,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":184382,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7550,"num_bytes_recovered":30908416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1067651072,"data_stored":1067502021,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160708,"ondisk_log_size":160708,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986257,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":671260,"kb_used_data":354948,"kb_used_omap":4,"kb_used_meta":316283,"kb_avail":93700580,"statfs":{"total":96636764160,"available":95949393920,"internally_reserved":0,"allocated":363466752,"data_stored":363174519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":323874474},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,0,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659203,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":596848,"kb_used_data":298592,"kb_used_omap":4,"kb_used_meta":298235,"kb_avail":93774992,"statfs":{"total":96636764160,"available":96025591808,"internally_reserved":0,"allocated":305758208,"data_stored":305429381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":305393326},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,5,0,6,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182486,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":779668,"kb_used_data":391236,"kb_used_omap":4,"kb_used_meta":388411,"kb_avail":93592172,"statfs":{"total":96636764160,"available":95838384128,"internally_reserved":0,"allocated":400625664,"data_stored":400333426,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":397733546},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,0,3,8,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":399335424,"data_stored":399299294,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":305336320,"data_stored":305263601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":362979328,"data_stored":362939126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:04:53.995 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:04:54.562 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:04:54.562 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:04:54.595 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":267,"stamp":"2024-09-10T16:04:53.671420+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":437867273,"num_objects":106913,"num_object_clones":0,"num_object_copies":213826,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43305,"num_objects_misplaced":534,"num_objects_unfound":0,"num_objects_dirty":106913,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":185197,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7552,"num_bytes_recovered":30908435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160824,"ondisk_log_size":160824,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2051180,"kb_used_data":1039864,"kb_used_omap":12,"kb_used_meta":1011251,"kb_avail":281064340,"statfs":{"total":289910292480,"available":287809884160,"internally_reserved":0,"allocated":1064820736,"data_stored":1063907438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1035521026},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,3,3,7,17,7,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19005440,"num_objects":-4640,"num_object_clones":0,"num_object_copies":-9280,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-285,"num_objects_unfound":0,"num_objects_dirty":-4640,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4640,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":285,"num_bytes_recovered":1167360,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006782"},"pg_stats":[{"pgid":"2.f","version":"45'14048","reported_seq":14123,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:49.137535+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:04:49.137535+0000","last_peered":"2024-09-10T16:04:49.137535+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:04:49.137535+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"39'4000","ondisk_log_start":"39'4000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27131904,"num_objects":6624,"num_object_clones":0,"num_object_copies":13248,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6624,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14048,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2365},{"shards":"0,2","objects":546}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'14274","reported_seq":14387,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:04:50.175000+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:04:50.175000+0000","last_peered":"2024-09-10T16:04:50.175000+0000","last_clean":"2024-09-10T16:04:50.175000+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:04:50.175000+0000","last_undegraded":"2024-09-10T16:04:50.175000+0000","last_fullsized":"2024-09-10T16:04:50.175000+0000","mapping_epoch":35,"log_start":"39'4200","ondisk_log_start":"39'4200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26911686,"num_objects":6572,"num_object_clones":0,"num_object_copies":13144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6572,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":12379,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'14210","reported_seq":22889,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:04:51.946567+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:04:51.946567+0000","last_peered":"2024-09-10T16:04:51.946567+0000","last_clean":"2024-09-10T16:04:51.946567+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:04:51.946567+0000","last_undegraded":"2024-09-10T16:04:51.946567+0000","last_fullsized":"2024-09-10T16:04:51.946567+0000","mapping_epoch":44,"log_start":"39'4200","ondisk_log_start":"39'4200","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26980376,"num_objects":6588,"num_object_clones":0,"num_object_copies":13176,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6588,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12214,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'14039","reported_seq":14120,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:04:50.043240+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:04:50.043240+0000","last_peered":"2024-09-10T16:04:50.043240+0000","last_clean":"2024-09-10T16:04:50.043240+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:04:50.043240+0000","last_undegraded":"2024-09-10T16:04:50.043240+0000","last_fullsized":"2024-09-10T16:04:50.043240+0000","mapping_epoch":35,"log_start":"39'4000","ondisk_log_start":"39'4000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27119616,"num_objects":6621,"num_object_clones":0,"num_object_copies":13242,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6621,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14039,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'14230","reported_seq":14361,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:04:49.101349+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:04:49.101349+0000","last_peered":"2024-09-10T16:04:49.101349+0000","last_clean":"2024-09-10T16:04:49.101349+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:04:49.101349+0000","last_undegraded":"2024-09-10T16:04:49.101349+0000","last_fullsized":"2024-09-10T16:04:49.101349+0000","mapping_epoch":34,"log_start":"39'4200","ondisk_log_start":"39'4200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27283939,"num_objects":6662,"num_object_clones":0,"num_object_copies":13324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6662,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10941,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'14394","reported_seq":14532,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:49.109144+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:04:49.109144+0000","last_peered":"2024-09-10T16:04:49.109144+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:04:49.109144+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"39'4300","ondisk_log_start":"39'4300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27308515,"num_objects":6668,"num_object_clones":0,"num_object_copies":13336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6668,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11006,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2297},{"shards":"0,2","objects":510}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'14231","reported_seq":14370,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:49.113269+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:04:49.113269+0000","last_peered":"2024-09-10T16:04:49.113269+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:04:49.113269+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"40'4200","ondisk_log_start":"40'4200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28062662,"num_objects":6853,"num_object_clones":0,"num_object_copies":13706,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6853,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":10730,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2616},{"shards":"0,2","objects":550}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'14468","reported_seq":14598,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:04:49.148650+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:04:49.148650+0000","last_peered":"2024-09-10T16:04:49.148650+0000","last_clean":"2024-09-10T16:04:49.148650+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:04:49.148650+0000","last_undegraded":"2024-09-10T16:04:49.148650+0000","last_fullsized":"2024-09-10T16:04:49.148650+0000","mapping_epoch":34,"log_start":"43'4400","ondisk_log_start":"43'4400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28041216,"num_objects":6846,"num_object_clones":0,"num_object_copies":13692,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6846,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11315,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104261+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:02:08.104261+0000","last_peered":"2024-09-10T16:02:08.104261+0000","last_clean":"2024-09-10T16:02:08.104261+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:02:08.104261+0000","last_undegraded":"2024-09-10T16:02:08.104261+0000","last_fullsized":"2024-09-10T16:02:08.104261+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'13890","reported_seq":14031,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:50.258931+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:04:50.258931+0000","last_peered":"2024-09-10T16:04:50.258931+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:04:50.258931+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"39'3800","ondisk_log_start":"39'3800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26898915,"num_objects":6568,"num_object_clones":0,"num_object_copies":13136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6568,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11709,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":465},{"shards":"2","objects":2445}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":95,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:29.860108+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:02:29.860108+0000","last_peered":"2024-09-10T16:02:29.860108+0000","last_clean":"2024-09-10T16:02:29.860108+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:02:29.860108+0000","last_undegraded":"2024-09-10T16:02:29.860108+0000","last_fullsized":"2024-09-10T16:02:29.860108+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'14207","reported_seq":14335,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:49.105116+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:04:49.105116+0000","last_peered":"2024-09-10T16:04:49.105116+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:04:49.105116+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"40'4200","ondisk_log_start":"40'4200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27455488,"num_objects":6703,"num_object_clones":0,"num_object_copies":13406,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6703,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10711,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2448},{"shards":"0,2","objects":507}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":105,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:12.405385+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:02:12.405385+0000","last_peered":"2024-09-10T16:02:12.405385+0000","last_clean":"2024-09-10T16:02:12.405385+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:02:12.405385+0000","last_undegraded":"2024-09-10T16:02:12.405385+0000","last_fullsized":"2024-09-10T16:02:12.405385+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'14164","reported_seq":14300,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:50.215064+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:04:50.215064+0000","last_peered":"2024-09-10T16:04:50.215064+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:04:50.215064+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"39'4100","ondisk_log_start":"39'4100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27103715,"num_objects":6618,"num_object_clones":0,"num_object_copies":13236,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6618,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11844,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":533},{"shards":"2","objects":2314}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104617+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:02:08.104617+0000","last_peered":"2024-09-10T16:02:08.104617+0000","last_clean":"2024-09-10T16:02:08.104617+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:02:08.104617+0000","last_undegraded":"2024-09-10T16:02:08.104617+0000","last_fullsized":"2024-09-10T16:02:08.104617+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'14431","reported_seq":14561,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:50.201411+0000","last_change":"2024-09-10T15:59:16.174113+0000","last_active":"2024-09-10T16:04:50.201411+0000","last_peered":"2024-09-10T16:04:50.201411+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:04:50.201411+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"43'4400","ondisk_log_start":"43'4400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27602944,"num_objects":6739,"num_object_clones":0,"num_object_copies":13478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4354,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6739,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11985,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":509},{"shards":"2","objects":2387}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'14132","reported_seq":14267,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:49.145520+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:04:49.145520+0000","last_peered":"2024-09-10T16:04:49.145520+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:04:49.145520+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"39'4100","ondisk_log_start":"39'4100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27467776,"num_objects":6706,"num_object_clones":0,"num_object_copies":13412,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6706,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10763,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2402},{"shards":"0,2","objects":592}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103316+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:02:08.103316+0000","last_peered":"2024-09-10T16:02:08.103316+0000","last_clean":"2024-09-10T16:02:08.103316+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:02:08.103316+0000","last_undegraded":"2024-09-10T16:02:08.103316+0000","last_fullsized":"2024-09-10T16:02:08.103316+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'13948","reported_seq":20649,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:49.147199+0000","last_change":"2024-09-10T16:03:29.605682+0000","last_active":"2024-09-10T16:04:49.147199+0000","last_peered":"2024-09-10T16:04:49.147199+0000","last_clean":"2024-09-10T15:59:14.115595+0000","last_became_active":"2024-09-10T15:59:16.244089+0000","last_became_peered":"2024-09-10T15:59:16.244089+0000","last_unstale":"2024-09-10T16:04:49.147199+0000","last_undegraded":"2024-09-10T16:03:29.603815+0000","last_fullsized":"2024-09-10T15:59:15.774437+0000","mapping_epoch":42,"log_start":"39'3900","ondisk_log_start":"39'3900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27238400,"num_objects":6650,"num_object_clones":0,"num_object_copies":13300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":394,"num_objects_misplaced":534,"num_objects_unfound":0,"num_objects_dirty":6650,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10338,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3278,"num_bytes_recovered":13414400,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5722},{"shards":"0,2","objects":534}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":95,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:29.877831+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:02:29.877831+0000","last_peered":"2024-09-10T16:02:29.877831+0000","last_clean":"2024-09-10T16:02:29.877831+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:02:29.877831+0000","last_undegraded":"2024-09-10T16:02:29.877831+0000","last_fullsized":"2024-09-10T16:02:29.877831+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'14357","reported_seq":14500,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:49.000088+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:04:49.000088+0000","last_peered":"2024-09-10T16:04:49.000088+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:04:49.000088+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"40'4300","ondisk_log_start":"40'4300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27406336,"num_objects":6691,"num_object_clones":0,"num_object_copies":13382,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6691,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10612,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2320},{"shards":"0,2","objects":540}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104463+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:02:08.104463+0000","last_peered":"2024-09-10T16:02:08.104463+0000","last_clean":"2024-09-10T16:02:08.104463+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:02:08.104463+0000","last_undegraded":"2024-09-10T16:02:08.104463+0000","last_fullsized":"2024-09-10T16:02:08.104463+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104448+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:02:08.104448+0000","last_peered":"2024-09-10T16:02:08.104448+0000","last_clean":"2024-09-10T16:02:08.104448+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:02:08.104448+0000","last_undegraded":"2024-09-10T16:02:08.104448+0000","last_fullsized":"2024-09-10T16:02:08.104448+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'14398","reported_seq":14543,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:49.072188+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:04:49.072188+0000","last_peered":"2024-09-10T16:04:49.072188+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:04:49.072188+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"40'4300","ondisk_log_start":"40'4300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27853766,"num_objects":6802,"num_object_clones":0,"num_object_copies":13604,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6802,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10561,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2479},{"shards":"0,2","objects":530}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":437867254,"num_objects":106911,"num_object_clones":0,"num_object_copies":213822,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43305,"num_objects_misplaced":534,"num_objects_unfound":0,"num_objects_dirty":106911,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":185195,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7550,"num_bytes_recovered":30908416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1062629376,"data_stored":1062480325,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160821,"ondisk_log_size":160821,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986258,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":671020,"kb_used_data":350556,"kb_used_omap":4,"kb_used_meta":320443,"kb_avail":93700820,"statfs":{"total":96636764160,"available":95949639680,"internally_reserved":0,"allocated":358969344,"data_stored":358677111,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":328134314},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,2,4,1,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659204,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":600492,"kb_used_data":298072,"kb_used_omap":4,"kb_used_meta":302395,"kb_avail":93771348,"statfs":{"total":96636764160,"available":96021860352,"internally_reserved":0,"allocated":305225728,"data_stored":304896901,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":309653166},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,3,2,5,2,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182486,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":779668,"kb_used_data":391236,"kb_used_omap":4,"kb_used_meta":388411,"kb_avail":93592172,"statfs":{"total":96636764160,"available":95838384128,"internally_reserved":0,"allocated":400625664,"data_stored":400333426,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":397733546},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,0,3,8,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":399335424,"data_stored":399299294,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":304816128,"data_stored":304743409,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":358477824,"data_stored":358437622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:04:54.597 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:04:55.156 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:04:55.158 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:04:55.178 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":267,"stamp":"2024-09-10T16:04:53.671420+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":437867273,"num_objects":106913,"num_object_clones":0,"num_object_copies":213826,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43305,"num_objects_misplaced":534,"num_objects_unfound":0,"num_objects_dirty":106913,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":185197,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7552,"num_bytes_recovered":30908435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160824,"ondisk_log_size":160824,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2051180,"kb_used_data":1039864,"kb_used_omap":12,"kb_used_meta":1011251,"kb_avail":281064340,"statfs":{"total":289910292480,"available":287809884160,"internally_reserved":0,"allocated":1064820736,"data_stored":1063907438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1035521026},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,3,3,7,17,7,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19005440,"num_objects":-4640,"num_object_clones":0,"num_object_copies":-9280,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-285,"num_objects_unfound":0,"num_objects_dirty":-4640,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4640,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":285,"num_bytes_recovered":1167360,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006782"},"pg_stats":[{"pgid":"2.f","version":"45'14048","reported_seq":14123,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:49.137535+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:04:49.137535+0000","last_peered":"2024-09-10T16:04:49.137535+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:04:49.137535+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"39'4000","ondisk_log_start":"39'4000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27131904,"num_objects":6624,"num_object_clones":0,"num_object_copies":13248,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6624,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14048,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2365},{"shards":"0,2","objects":546}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'14274","reported_seq":14387,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:04:50.175000+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:04:50.175000+0000","last_peered":"2024-09-10T16:04:50.175000+0000","last_clean":"2024-09-10T16:04:50.175000+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:04:50.175000+0000","last_undegraded":"2024-09-10T16:04:50.175000+0000","last_fullsized":"2024-09-10T16:04:50.175000+0000","mapping_epoch":35,"log_start":"39'4200","ondisk_log_start":"39'4200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26911686,"num_objects":6572,"num_object_clones":0,"num_object_copies":13144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6572,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":12379,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'14210","reported_seq":22889,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:04:51.946567+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:04:51.946567+0000","last_peered":"2024-09-10T16:04:51.946567+0000","last_clean":"2024-09-10T16:04:51.946567+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:04:51.946567+0000","last_undegraded":"2024-09-10T16:04:51.946567+0000","last_fullsized":"2024-09-10T16:04:51.946567+0000","mapping_epoch":44,"log_start":"39'4200","ondisk_log_start":"39'4200","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26980376,"num_objects":6588,"num_object_clones":0,"num_object_copies":13176,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6588,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12214,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'14039","reported_seq":14120,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:04:50.043240+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:04:50.043240+0000","last_peered":"2024-09-10T16:04:50.043240+0000","last_clean":"2024-09-10T16:04:50.043240+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:04:50.043240+0000","last_undegraded":"2024-09-10T16:04:50.043240+0000","last_fullsized":"2024-09-10T16:04:50.043240+0000","mapping_epoch":35,"log_start":"39'4000","ondisk_log_start":"39'4000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27119616,"num_objects":6621,"num_object_clones":0,"num_object_copies":13242,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6621,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14039,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'14230","reported_seq":14361,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:04:49.101349+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:04:49.101349+0000","last_peered":"2024-09-10T16:04:49.101349+0000","last_clean":"2024-09-10T16:04:49.101349+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:04:49.101349+0000","last_undegraded":"2024-09-10T16:04:49.101349+0000","last_fullsized":"2024-09-10T16:04:49.101349+0000","mapping_epoch":34,"log_start":"39'4200","ondisk_log_start":"39'4200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27283939,"num_objects":6662,"num_object_clones":0,"num_object_copies":13324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6662,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10941,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'14394","reported_seq":14532,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:49.109144+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:04:49.109144+0000","last_peered":"2024-09-10T16:04:49.109144+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:04:49.109144+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"39'4300","ondisk_log_start":"39'4300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27308515,"num_objects":6668,"num_object_clones":0,"num_object_copies":13336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6668,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11006,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2297},{"shards":"0,2","objects":510}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'14231","reported_seq":14370,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:49.113269+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:04:49.113269+0000","last_peered":"2024-09-10T16:04:49.113269+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:04:49.113269+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"40'4200","ondisk_log_start":"40'4200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28062662,"num_objects":6853,"num_object_clones":0,"num_object_copies":13706,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6853,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":10730,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2616},{"shards":"0,2","objects":550}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'14468","reported_seq":14598,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:04:49.148650+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:04:49.148650+0000","last_peered":"2024-09-10T16:04:49.148650+0000","last_clean":"2024-09-10T16:04:49.148650+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:04:49.148650+0000","last_undegraded":"2024-09-10T16:04:49.148650+0000","last_fullsized":"2024-09-10T16:04:49.148650+0000","mapping_epoch":34,"log_start":"43'4400","ondisk_log_start":"43'4400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28041216,"num_objects":6846,"num_object_clones":0,"num_object_copies":13692,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6846,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11315,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104261+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:02:08.104261+0000","last_peered":"2024-09-10T16:02:08.104261+0000","last_clean":"2024-09-10T16:02:08.104261+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:02:08.104261+0000","last_undegraded":"2024-09-10T16:02:08.104261+0000","last_fullsized":"2024-09-10T16:02:08.104261+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'13890","reported_seq":14031,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:50.258931+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:04:50.258931+0000","last_peered":"2024-09-10T16:04:50.258931+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:04:50.258931+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"39'3800","ondisk_log_start":"39'3800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26898915,"num_objects":6568,"num_object_clones":0,"num_object_copies":13136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6568,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11709,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":465},{"shards":"2","objects":2445}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":95,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:29.860108+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:02:29.860108+0000","last_peered":"2024-09-10T16:02:29.860108+0000","last_clean":"2024-09-10T16:02:29.860108+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:02:29.860108+0000","last_undegraded":"2024-09-10T16:02:29.860108+0000","last_fullsized":"2024-09-10T16:02:29.860108+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'14207","reported_seq":14335,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:49.105116+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:04:49.105116+0000","last_peered":"2024-09-10T16:04:49.105116+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:04:49.105116+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"40'4200","ondisk_log_start":"40'4200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27455488,"num_objects":6703,"num_object_clones":0,"num_object_copies":13406,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6703,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10711,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2448},{"shards":"0,2","objects":507}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":105,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:12.405385+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:02:12.405385+0000","last_peered":"2024-09-10T16:02:12.405385+0000","last_clean":"2024-09-10T16:02:12.405385+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:02:12.405385+0000","last_undegraded":"2024-09-10T16:02:12.405385+0000","last_fullsized":"2024-09-10T16:02:12.405385+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'14164","reported_seq":14300,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:50.215064+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:04:50.215064+0000","last_peered":"2024-09-10T16:04:50.215064+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:04:50.215064+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"39'4100","ondisk_log_start":"39'4100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27103715,"num_objects":6618,"num_object_clones":0,"num_object_copies":13236,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6618,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11844,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":533},{"shards":"2","objects":2314}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104617+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:02:08.104617+0000","last_peered":"2024-09-10T16:02:08.104617+0000","last_clean":"2024-09-10T16:02:08.104617+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:02:08.104617+0000","last_undegraded":"2024-09-10T16:02:08.104617+0000","last_fullsized":"2024-09-10T16:02:08.104617+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'14431","reported_seq":14561,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:50.201411+0000","last_change":"2024-09-10T15:59:16.174113+0000","last_active":"2024-09-10T16:04:50.201411+0000","last_peered":"2024-09-10T16:04:50.201411+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:04:50.201411+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"43'4400","ondisk_log_start":"43'4400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27602944,"num_objects":6739,"num_object_clones":0,"num_object_copies":13478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4354,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6739,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11985,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":509},{"shards":"2","objects":2387}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'14132","reported_seq":14267,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:49.145520+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:04:49.145520+0000","last_peered":"2024-09-10T16:04:49.145520+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:04:49.145520+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"39'4100","ondisk_log_start":"39'4100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27467776,"num_objects":6706,"num_object_clones":0,"num_object_copies":13412,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6706,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10763,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2402},{"shards":"0,2","objects":592}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103316+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:02:08.103316+0000","last_peered":"2024-09-10T16:02:08.103316+0000","last_clean":"2024-09-10T16:02:08.103316+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:02:08.103316+0000","last_undegraded":"2024-09-10T16:02:08.103316+0000","last_fullsized":"2024-09-10T16:02:08.103316+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'13948","reported_seq":20649,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:49.147199+0000","last_change":"2024-09-10T16:03:29.605682+0000","last_active":"2024-09-10T16:04:49.147199+0000","last_peered":"2024-09-10T16:04:49.147199+0000","last_clean":"2024-09-10T15:59:14.115595+0000","last_became_active":"2024-09-10T15:59:16.244089+0000","last_became_peered":"2024-09-10T15:59:16.244089+0000","last_unstale":"2024-09-10T16:04:49.147199+0000","last_undegraded":"2024-09-10T16:03:29.603815+0000","last_fullsized":"2024-09-10T15:59:15.774437+0000","mapping_epoch":42,"log_start":"39'3900","ondisk_log_start":"39'3900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27238400,"num_objects":6650,"num_object_clones":0,"num_object_copies":13300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":394,"num_objects_misplaced":534,"num_objects_unfound":0,"num_objects_dirty":6650,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10338,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3278,"num_bytes_recovered":13414400,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5722},{"shards":"0,2","objects":534}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":95,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:29.877831+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:02:29.877831+0000","last_peered":"2024-09-10T16:02:29.877831+0000","last_clean":"2024-09-10T16:02:29.877831+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:02:29.877831+0000","last_undegraded":"2024-09-10T16:02:29.877831+0000","last_fullsized":"2024-09-10T16:02:29.877831+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'14357","reported_seq":14500,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:49.000088+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:04:49.000088+0000","last_peered":"2024-09-10T16:04:49.000088+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:04:49.000088+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"40'4300","ondisk_log_start":"40'4300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27406336,"num_objects":6691,"num_object_clones":0,"num_object_copies":13382,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6691,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10612,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2320},{"shards":"0,2","objects":540}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104463+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:02:08.104463+0000","last_peered":"2024-09-10T16:02:08.104463+0000","last_clean":"2024-09-10T16:02:08.104463+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:02:08.104463+0000","last_undegraded":"2024-09-10T16:02:08.104463+0000","last_fullsized":"2024-09-10T16:02:08.104463+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104448+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:02:08.104448+0000","last_peered":"2024-09-10T16:02:08.104448+0000","last_clean":"2024-09-10T16:02:08.104448+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:02:08.104448+0000","last_undegraded":"2024-09-10T16:02:08.104448+0000","last_fullsized":"2024-09-10T16:02:08.104448+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'14398","reported_seq":14543,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:49.072188+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:04:49.072188+0000","last_peered":"2024-09-10T16:04:49.072188+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:04:49.072188+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"40'4300","ondisk_log_start":"40'4300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27853766,"num_objects":6802,"num_object_clones":0,"num_object_copies":13604,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6802,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10561,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2479},{"shards":"0,2","objects":530}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":437867254,"num_objects":106911,"num_object_clones":0,"num_object_copies":213822,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43305,"num_objects_misplaced":534,"num_objects_unfound":0,"num_objects_dirty":106911,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":185195,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7550,"num_bytes_recovered":30908416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1062629376,"data_stored":1062480325,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160821,"ondisk_log_size":160821,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986258,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":671020,"kb_used_data":350556,"kb_used_omap":4,"kb_used_meta":320443,"kb_avail":93700820,"statfs":{"total":96636764160,"available":95949639680,"internally_reserved":0,"allocated":358969344,"data_stored":358677111,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":328134314},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,2,4,1,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659204,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":600492,"kb_used_data":298072,"kb_used_omap":4,"kb_used_meta":302395,"kb_avail":93771348,"statfs":{"total":96636764160,"available":96021860352,"internally_reserved":0,"allocated":305225728,"data_stored":304896901,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":309653166},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,3,2,5,2,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182486,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":779668,"kb_used_data":391236,"kb_used_omap":4,"kb_used_meta":388411,"kb_avail":93592172,"statfs":{"total":96636764160,"available":95838384128,"internally_reserved":0,"allocated":400625664,"data_stored":400333426,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":397733546},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,0,3,8,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":399335424,"data_stored":399299294,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":304816128,"data_stored":304743409,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":358477824,"data_stored":358437622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:04:55.180 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-10T16:04:55.180 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "45'14048", 'reported_seq': 14123, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:04:49.137535+0000', 'last_change': '2024-09-10T15:59:16.281558+0000', 'last_active': '2024-09-10T16:04:49.137535+0000', 'last_peered': '2024-09-10T16:04:49.137535+0000', 'last_clean': '2024-09-10T15:59:14.132456+0000', 'last_became_active': '2024-09-10T15:59:16.274407+0000', 'last_became_peered': '2024-09-10T15:59:16.274407+0000', 'last_unstale': '2024-09-10T16:04:49.137535+0000', 'last_undegraded': '2024-09-10T15:59:15.851279+0000', 'last_fullsized': '2024-09-10T15:59:15.774356+0000', 'mapping_epoch': 42, 'log_start': "39'4000", 'ondisk_log_start': "39'4000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'objects_scrubbed': 0, 'log_size': 10048, 'log_dups_size': 0, 'ondisk_log_size': 10048, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27131904, 'num_objects': 6624, 'num_object_clones': 0, 'num_object_copies': 13248, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4259, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6624, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14048, 'num_write_kb': 41344, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2365}, {'shards': '0,2', 'objects': 546}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:04:55.180 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-10T16:04:55.180 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "45'14394", 'reported_seq': 14532, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:04:49.109144+0000', 'last_change': '2024-09-10T15:59:16.308264+0000', 'last_active': '2024-09-10T16:04:49.109144+0000', 'last_peered': '2024-09-10T16:04:49.109144+0000', 'last_clean': '2024-09-10T15:59:14.111551+0000', 'last_became_active': '2024-09-10T15:59:16.300040+0000', 'last_became_peered': '2024-09-10T15:59:16.300040+0000', 'last_unstale': '2024-09-10T16:04:49.109144+0000', 'last_undegraded': '2024-09-10T15:59:15.878234+0000', 'last_fullsized': '2024-09-10T15:59:15.769328+0000', 'mapping_epoch': 42, 'log_start': "39'4300", 'ondisk_log_start': "39'4300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3442", 'last_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'objects_scrubbed': 3384, 'log_size': 10094, 'log_dups_size': 0, 'ondisk_log_size': 10094, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1292, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27308515, 'num_objects': 6668, 'num_object_clones': 0, 'num_object_copies': 13336, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4372, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6668, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 11006, 'num_write_kb': 28580, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2297}, {'shards': '0,2', 'objects': 510}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:04:55.180 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-10T16:04:55.180 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "45'14231", 'reported_seq': 14370, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:04:49.113269+0000', 'last_change': '2024-09-10T15:59:16.331899+0000', 'last_active': '2024-09-10T16:04:49.113269+0000', 'last_peered': '2024-09-10T16:04:49.113269+0000', 'last_clean': '2024-09-10T15:59:14.139436+0000', 'last_became_active': '2024-09-10T15:59:16.318715+0000', 'last_became_peered': '2024-09-10T15:59:16.318715+0000', 'last_unstale': '2024-09-10T16:04:49.113269+0000', 'last_undegraded': '2024-09-10T15:59:15.877524+0000', 'last_fullsized': '2024-09-10T15:59:15.769042+0000', 'mapping_epoch': 42, 'log_start': "40'4200", 'ondisk_log_start': "40'4200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3537", 'last_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'objects_scrubbed': 3497, 'log_size': 10031, 'log_dups_size': 0, 'ondisk_log_size': 10031, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1224, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28062662, 'num_objects': 6853, 'num_object_clones': 0, 'num_object_copies': 13706, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4237, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6853, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 10730, 'num_write_kb': 28172, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2616}, {'shards': '0,2', 'objects': 550}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:04:55.180 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-10T16:04:55.180 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "45'13890", 'reported_seq': 14031, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:04:50.258931+0000', 'last_change': '2024-09-10T15:59:16.175680+0000', 'last_active': '2024-09-10T16:04:50.258931+0000', 'last_peered': '2024-09-10T16:04:50.258931+0000', 'last_clean': '2024-09-10T15:59:14.088187+0000', 'last_became_active': '2024-09-10T15:59:16.165810+0000', 'last_became_peered': '2024-09-10T15:59:16.165810+0000', 'last_unstale': '2024-09-10T16:04:50.258931+0000', 'last_undegraded': '2024-09-10T15:59:15.822098+0000', 'last_fullsized': '2024-09-10T15:59:15.769123+0000', 'mapping_epoch': 42, 'log_start': "39'3800", 'ondisk_log_start': "39'3800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2218", 'last_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'objects_scrubbed': 2175, 'log_size': 10090, 'log_dups_size': 0, 'ondisk_log_size': 10090, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 825, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26898915, 'num_objects': 6568, 'num_object_clones': 0, 'num_object_copies': 13136, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4124, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6568, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 11709, 'num_write_kb': 32204, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 465}, {'shards': '2', 'objects': 2445}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:04:55.180 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-10T16:04:55.181 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "45'14207", 'reported_seq': 14335, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:04:49.105116+0000', 'last_change': '2024-09-10T15:59:16.277742+0000', 'last_active': '2024-09-10T16:04:49.105116+0000', 'last_peered': '2024-09-10T16:04:49.105116+0000', 'last_clean': '2024-09-10T15:59:14.129081+0000', 'last_became_active': '2024-09-10T15:59:16.269835+0000', 'last_became_peered': '2024-09-10T15:59:16.269835+0000', 'last_unstale': '2024-09-10T16:04:49.105116+0000', 'last_undegraded': '2024-09-10T15:59:15.860857+0000', 'last_fullsized': '2024-09-10T15:59:15.771410+0000', 'mapping_epoch': 42, 'log_start': "40'4200", 'ondisk_log_start': "40'4200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3557", 'last_scrub_stamp': '2024-09-10T15:59:02.493678+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:02.493678+0000', 'objects_scrubbed': 3488, 'log_size': 10007, 'log_dups_size': 0, 'ondisk_log_size': 10007, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1472, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27455488, 'num_objects': 6703, 'num_object_clones': 0, 'num_object_copies': 13406, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4257, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6703, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10711, 'num_write_kb': 27852, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2448}, {'shards': '0,2', 'objects': 507}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:04:55.181 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-10T16:04:55.181 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "45'14164", 'reported_seq': 14300, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:04:50.215064+0000', 'last_change': '2024-09-10T15:59:16.284311+0000', 'last_active': '2024-09-10T16:04:50.215064+0000', 'last_peered': '2024-09-10T16:04:50.215064+0000', 'last_clean': '2024-09-10T15:59:14.116041+0000', 'last_became_active': '2024-09-10T15:59:16.274543+0000', 'last_became_peered': '2024-09-10T15:59:16.274543+0000', 'last_unstale': '2024-09-10T16:04:50.215064+0000', 'last_undegraded': '2024-09-10T15:59:15.812484+0000', 'last_fullsized': '2024-09-10T15:59:15.770885+0000', 'mapping_epoch': 42, 'log_start': "39'4100", 'ondisk_log_start': "39'4100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2356", 'last_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'objects_scrubbed': 2316, 'log_size': 10064, 'log_dups_size': 0, 'ondisk_log_size': 10064, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 783, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27103715, 'num_objects': 6618, 'num_object_clones': 0, 'num_object_copies': 13236, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6618, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 11844, 'num_write_kb': 32292, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 533}, {'shards': '2', 'objects': 2314}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:04:55.181 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-10T16:04:55.181 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "45'14431", 'reported_seq': 14561, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:04:50.201411+0000', 'last_change': '2024-09-10T15:59:16.174113+0000', 'last_active': '2024-09-10T16:04:50.201411+0000', 'last_peered': '2024-09-10T16:04:50.201411+0000', 'last_clean': '2024-09-10T15:59:14.110654+0000', 'last_became_active': '2024-09-10T15:59:16.164645+0000', 'last_became_peered': '2024-09-10T15:59:16.164645+0000', 'last_unstale': '2024-09-10T16:04:50.201411+0000', 'last_undegraded': '2024-09-10T15:59:15.803593+0000', 'last_fullsized': '2024-09-10T15:59:15.769083+0000', 'mapping_epoch': 42, 'log_start': "43'4400", 'ondisk_log_start': "43'4400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2486", 'last_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'objects_scrubbed': 2440, 'log_size': 10031, 'log_dups_size': 0, 'ondisk_log_size': 10031, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 792, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27602944, 'num_objects': 6739, 'num_object_clones': 0, 'num_object_copies': 13478, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4354, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6739, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11985, 'num_write_kb': 32568, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 509}, {'shards': '2', 'objects': 2387}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:04:55.181 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-10T16:04:55.181 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "45'14132", 'reported_seq': 14267, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:04:49.145520+0000', 'last_change': '2024-09-10T15:59:16.315641+0000', 'last_active': '2024-09-10T16:04:49.145520+0000', 'last_peered': '2024-09-10T16:04:49.145520+0000', 'last_clean': '2024-09-10T15:59:14.117001+0000', 'last_became_active': '2024-09-10T15:59:16.302245+0000', 'last_became_peered': '2024-09-10T15:59:16.302245+0000', 'last_unstale': '2024-09-10T16:04:49.145520+0000', 'last_undegraded': '2024-09-10T15:59:15.845012+0000', 'last_fullsized': '2024-09-10T15:59:15.770337+0000', 'mapping_epoch': 42, 'log_start': "39'4100", 'ondisk_log_start': "39'4100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3404", 'last_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'objects_scrubbed': 3367, 'log_size': 10032, 'log_dups_size': 0, 'ondisk_log_size': 10032, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 983, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27467776, 'num_objects': 6706, 'num_object_clones': 0, 'num_object_copies': 13412, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6706, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10763, 'num_write_kb': 28204, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2402}, {'shards': '0,2', 'objects': 592}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:04:55.181 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-10T16:04:55.181 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "45'13948", 'reported_seq': 20649, 'reported_epoch': 45, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:04:49.147199+0000', 'last_change': '2024-09-10T16:03:29.605682+0000', 'last_active': '2024-09-10T16:04:49.147199+0000', 'last_peered': '2024-09-10T16:04:49.147199+0000', 'last_clean': '2024-09-10T15:59:14.115595+0000', 'last_became_active': '2024-09-10T15:59:16.244089+0000', 'last_became_peered': '2024-09-10T15:59:16.244089+0000', 'last_unstale': '2024-09-10T16:04:49.147199+0000', 'last_undegraded': '2024-09-10T16:03:29.603815+0000', 'last_fullsized': '2024-09-10T15:59:15.774437+0000', 'mapping_epoch': 42, 'log_start': "39'3900", 'ondisk_log_start': "39'3900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3653", 'last_scrub_stamp': '2024-09-10T15:59:05.298468+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:05.298468+0000', 'objects_scrubbed': 3602, 'log_size': 10048, 'log_dups_size': 0, 'ondisk_log_size': 10048, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1289, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27238400, 'num_objects': 6650, 'num_object_clones': 0, 'num_object_copies': 13300, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 394, 'num_objects_misplaced': 534, 'num_objects_unfound': 0, 'num_objects_dirty': 6650, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10338, 'num_write_kb': 26772, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3278, 'num_bytes_recovered': 13414400, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5722}, {'shards': '0,2', 'objects': 534}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:04:55.182 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-10T16:04:55.182 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "45'14357", 'reported_seq': 14500, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:04:49.000088+0000', 'last_change': '2024-09-10T15:59:16.311291+0000', 'last_active': '2024-09-10T16:04:49.000088+0000', 'last_peered': '2024-09-10T16:04:49.000088+0000', 'last_clean': '2024-09-10T15:59:14.116363+0000', 'last_became_active': '2024-09-10T15:59:16.301244+0000', 'last_became_peered': '2024-09-10T15:59:16.301244+0000', 'last_unstale': '2024-09-10T16:04:49.000088+0000', 'last_undegraded': '2024-09-10T15:59:15.883434+0000', 'last_fullsized': '2024-09-10T15:59:15.772756+0000', 'mapping_epoch': 42, 'log_start': "40'4300", 'ondisk_log_start': "40'4300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3789", 'last_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'objects_scrubbed': 3741, 'log_size': 10057, 'log_dups_size': 0, 'ondisk_log_size': 10057, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1440, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27406336, 'num_objects': 6691, 'num_object_clones': 0, 'num_object_copies': 13382, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4373, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6691, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10612, 'num_write_kb': 27124, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2320}, {'shards': '0,2', 'objects': 540}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:04:55.182 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-10T16:04:55.182 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "45'14398", 'reported_seq': 14543, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:04:49.072188+0000', 'last_change': '2024-09-10T15:59:16.313409+0000', 'last_active': '2024-09-10T16:04:49.072188+0000', 'last_peered': '2024-09-10T16:04:49.072188+0000', 'last_clean': '2024-09-10T15:59:14.129780+0000', 'last_became_active': '2024-09-10T15:59:16.300932+0000', 'last_became_peered': '2024-09-10T15:59:16.300932+0000', 'last_unstale': '2024-09-10T16:04:49.072188+0000', 'last_undegraded': '2024-09-10T15:59:15.887823+0000', 'last_fullsized': '2024-09-10T15:59:15.768513+0000', 'mapping_epoch': 42, 'log_start': "40'4300", 'ondisk_log_start': "40'4300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3880", 'last_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'objects_scrubbed': 3827, 'log_size': 10098, 'log_dups_size': 0, 'ondisk_log_size': 10098, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1397, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27853766, 'num_objects': 6802, 'num_object_clones': 0, 'num_object_copies': 13604, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4325, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6802, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10561, 'num_write_kb': 27072, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2479}, {'shards': '0,2', 'objects': 530}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:04:55.182 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T16:04:55.869 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:04:55.869 INFO:teuthology.orchestra.run.smithi115.stdout:{"fsid":"7effee29-df05-4351-9741-ee203564ed81","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 43305/213826 objects degraded (20.252%), 11 pgs degraded, 11 pgs undersized","count":22},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":459,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983954,"num_in_osds":3,"osd_in_since":1725983947,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":106913,"data_bytes":437867273,"bytes_used":2100408320,"bytes_avail":287809884160,"bytes_total":289910292480,"degraded_objects":43305,"degraded_total":213826,"degraded_ratio":0.20252448252317304,"misplaced_objects":534,"misplaced_total":213826,"misplaced_ratio":0.0024973576646432146,"recovering_objects_per_sec":23,"recovering_bytes_per_sec":97225,"recovering_keys_per_sec":0,"num_objects_recovered":285,"num_bytes_recovered":1167360,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":386},"fsmap":{"epoch":1,"btime":"2024-09-10T15:57:14:967120+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T16:00:13.560776+0000","services":{}},"progress_events":{}} 2024-09-10T16:04:55.899 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-10T16:04:55.899 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:04:56.472 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:04:56.473 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:04:56.496 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":268,"stamp":"2024-09-10T16:04:55.672153+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":429208329,"num_objects":104799,"num_object_clones":0,"num_object_copies":209598,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43305,"num_objects_misplaced":376,"num_objects_unfound":0,"num_objects_dirty":104799,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":187311,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7710,"num_bytes_recovered":31555603,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160838,"ondisk_log_size":160838,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2048340,"kb_used_data":1028700,"kb_used_omap":12,"kb_used_meta":1019571,"kb_avail":281067180,"statfs":{"total":289910292480,"available":287812792320,"internally_reserved":0,"allocated":1053388800,"data_stored":1052475502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1044040706},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,2,5,4,12,4,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-27152384,"num_objects":-6629,"num_object_clones":0,"num_object_copies":-13258,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-443,"num_objects_unfound":0,"num_objects_dirty":-6629,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6629,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":443,"num_bytes_recovered":1814528,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006869"},"pg_stats":[{"pgid":"2.f","version":"45'14211","reported_seq":14285,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:54.159203+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:04:54.159203+0000","last_peered":"2024-09-10T16:04:54.159203+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:04:54.159203+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"40'4200","ondisk_log_start":"40'4200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26464256,"num_objects":6461,"num_object_clones":0,"num_object_copies":12922,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6461,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14211,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2202},{"shards":"0,2","objects":383}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'14395","reported_seq":14506,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:04:55.218204+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:04:55.218204+0000","last_peered":"2024-09-10T16:04:55.218204+0000","last_clean":"2024-09-10T16:04:55.218204+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:04:55.218204+0000","last_undegraded":"2024-09-10T16:04:55.218204+0000","last_fullsized":"2024-09-10T16:04:55.218204+0000","mapping_epoch":35,"log_start":"43'4300","ondisk_log_start":"43'4300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26416070,"num_objects":6451,"num_object_clones":0,"num_object_copies":12902,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6451,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":12500,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'14210","reported_seq":22889,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:04:51.946567+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:04:51.946567+0000","last_peered":"2024-09-10T16:04:51.946567+0000","last_clean":"2024-09-10T16:04:51.946567+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:04:51.946567+0000","last_undegraded":"2024-09-10T16:04:51.946567+0000","last_fullsized":"2024-09-10T16:04:51.946567+0000","mapping_epoch":44,"log_start":"39'4200","ondisk_log_start":"39'4200","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26980376,"num_objects":6588,"num_object_clones":0,"num_object_copies":13176,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6588,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12214,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'14176","reported_seq":14257,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:04:55.274785+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:04:55.274785+0000","last_peered":"2024-09-10T16:04:55.274785+0000","last_clean":"2024-09-10T16:04:55.274785+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:04:55.274785+0000","last_undegraded":"2024-09-10T16:04:55.274785+0000","last_fullsized":"2024-09-10T16:04:55.274785+0000","mapping_epoch":35,"log_start":"39'4100","ondisk_log_start":"39'4100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26558464,"num_objects":6484,"num_object_clones":0,"num_object_copies":12968,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6484,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14176,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'14363","reported_seq":14494,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:04:54.141423+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:04:54.141423+0000","last_peered":"2024-09-10T16:04:54.141423+0000","last_clean":"2024-09-10T16:04:54.141423+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:04:54.141423+0000","last_undegraded":"2024-09-10T16:04:54.141423+0000","last_fullsized":"2024-09-10T16:04:54.141423+0000","mapping_epoch":34,"log_start":"40'4300","ondisk_log_start":"40'4300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26739171,"num_objects":6529,"num_object_clones":0,"num_object_copies":13058,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6529,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11074,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'14529","reported_seq":14668,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:54.111808+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:04:54.111808+0000","last_peered":"2024-09-10T16:04:54.111808+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:04:54.111808+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"43'4500","ondisk_log_start":"43'4500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26755555,"num_objects":6533,"num_object_clones":0,"num_object_copies":13066,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6533,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11141,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2162},{"shards":"0,2","objects":375}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'14358","reported_seq":14497,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:54.130707+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:04:54.130707+0000","last_peered":"2024-09-10T16:04:54.130707+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:04:54.130707+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"43'4300","ondisk_log_start":"43'4300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27542470,"num_objects":6726,"num_object_clones":0,"num_object_copies":13452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6726,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":10857,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2489},{"shards":"0,2","objects":423}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'14611","reported_seq":14742,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:04:54.169632+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:04:54.169632+0000","last_peered":"2024-09-10T16:04:54.169632+0000","last_clean":"2024-09-10T16:04:54.169632+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:04:54.169632+0000","last_undegraded":"2024-09-10T16:04:54.169632+0000","last_fullsized":"2024-09-10T16:04:54.169632+0000","mapping_epoch":34,"log_start":"43'4600","ondisk_log_start":"43'4600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27455488,"num_objects":6703,"num_object_clones":0,"num_object_copies":13406,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6703,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11458,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104261+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:02:08.104261+0000","last_peered":"2024-09-10T16:02:08.104261+0000","last_clean":"2024-09-10T16:02:08.104261+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:02:08.104261+0000","last_undegraded":"2024-09-10T16:02:08.104261+0000","last_fullsized":"2024-09-10T16:02:08.104261+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'14023","reported_seq":14164,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:55.260864+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:04:55.260864+0000","last_peered":"2024-09-10T16:04:55.260864+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:04:55.260864+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"39'4000","ondisk_log_start":"39'4000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26354147,"num_objects":6435,"num_object_clones":0,"num_object_copies":12870,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6435,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11842,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":332},{"shards":"2","objects":2312}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":95,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:29.860108+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:02:29.860108+0000","last_peered":"2024-09-10T16:02:29.860108+0000","last_clean":"2024-09-10T16:02:29.860108+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:02:29.860108+0000","last_undegraded":"2024-09-10T16:02:29.860108+0000","last_fullsized":"2024-09-10T16:02:29.860108+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'14352","reported_seq":14482,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:54.169131+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:04:54.169131+0000","last_peered":"2024-09-10T16:04:54.169131+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:04:54.169131+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"43'4300","ondisk_log_start":"43'4300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26861568,"num_objects":6558,"num_object_clones":0,"num_object_copies":13116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6558,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10856,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2303},{"shards":"0,2","objects":362}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":105,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:12.405385+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:02:12.405385+0000","last_peered":"2024-09-10T16:02:12.405385+0000","last_clean":"2024-09-10T16:02:12.405385+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:02:12.405385+0000","last_undegraded":"2024-09-10T16:02:12.405385+0000","last_fullsized":"2024-09-10T16:02:12.405385+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'14326","reported_seq":14461,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:55.218129+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:04:55.218129+0000","last_peered":"2024-09-10T16:04:55.218129+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:04:55.218129+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"40'4300","ondisk_log_start":"40'4300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26440163,"num_objects":6456,"num_object_clones":0,"num_object_copies":12912,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6456,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12006,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":371},{"shards":"2","objects":2152}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104617+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:02:08.104617+0000","last_peered":"2024-09-10T16:02:08.104617+0000","last_clean":"2024-09-10T16:02:08.104617+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:02:08.104617+0000","last_undegraded":"2024-09-10T16:02:08.104617+0000","last_fullsized":"2024-09-10T16:02:08.104617+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'14572","reported_seq":14701,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:55.253754+0000","last_change":"2024-09-10T15:59:16.174113+0000","last_active":"2024-09-10T16:04:55.253754+0000","last_peered":"2024-09-10T16:04:55.253754+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:04:55.253754+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"43'4500","ondisk_log_start":"43'4500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27025408,"num_objects":6598,"num_object_clones":0,"num_object_copies":13196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4354,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6598,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12126,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":368},{"shards":"2","objects":2246}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'14297","reported_seq":14429,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:54.117611+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:04:54.117611+0000","last_peered":"2024-09-10T16:04:54.117611+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:04:54.117611+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"39'4200","ondisk_log_start":"39'4200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26791936,"num_objects":6541,"num_object_clones":0,"num_object_copies":13082,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6541,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10928,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2237},{"shards":"0,2","objects":427}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103316+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:02:08.103316+0000","last_peered":"2024-09-10T16:02:08.103316+0000","last_clean":"2024-09-10T16:02:08.103316+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:02:08.103316+0000","last_undegraded":"2024-09-10T16:02:08.103316+0000","last_fullsized":"2024-09-10T16:02:08.103316+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'14077","reported_seq":21094,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:54.195978+0000","last_change":"2024-09-10T16:03:29.605682+0000","last_active":"2024-09-10T16:04:54.195978+0000","last_peered":"2024-09-10T16:04:54.195978+0000","last_clean":"2024-09-10T15:59:14.115595+0000","last_became_active":"2024-09-10T15:59:16.244089+0000","last_became_peered":"2024-09-10T15:59:16.244089+0000","last_unstale":"2024-09-10T16:04:54.195978+0000","last_undegraded":"2024-09-10T16:03:29.603815+0000","last_fullsized":"2024-09-10T15:59:15.774437+0000","mapping_epoch":42,"log_start":"39'4000","ondisk_log_start":"39'4000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26710016,"num_objects":6521,"num_object_clones":0,"num_object_copies":13042,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":394,"num_objects_misplaced":376,"num_objects_unfound":0,"num_objects_dirty":6521,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10467,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3436,"num_bytes_recovered":14061568,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5751},{"shards":"0,2","objects":376}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":95,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:29.877831+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:02:29.877831+0000","last_peered":"2024-09-10T16:02:29.877831+0000","last_clean":"2024-09-10T16:02:29.877831+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:02:29.877831+0000","last_undegraded":"2024-09-10T16:02:29.877831+0000","last_fullsized":"2024-09-10T16:02:29.877831+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'14487","reported_seq":14630,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:54.010043+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:04:54.010043+0000","last_peered":"2024-09-10T16:04:54.010043+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:04:54.010043+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"43'4400","ondisk_log_start":"43'4400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26873856,"num_objects":6561,"num_object_clones":0,"num_object_copies":13122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6561,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10742,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2190},{"shards":"0,2","objects":410}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104463+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:02:08.104463+0000","last_peered":"2024-09-10T16:02:08.104463+0000","last_clean":"2024-09-10T16:02:08.104463+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:02:08.104463+0000","last_undegraded":"2024-09-10T16:02:08.104463+0000","last_fullsized":"2024-09-10T16:02:08.104463+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104448+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:02:08.104448+0000","last_peered":"2024-09-10T16:02:08.104448+0000","last_clean":"2024-09-10T16:02:08.104448+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:02:08.104448+0000","last_undegraded":"2024-09-10T16:02:08.104448+0000","last_fullsized":"2024-09-10T16:02:08.104448+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'14548","reported_seq":14692,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:54.196243+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:04:54.196243+0000","last_peered":"2024-09-10T16:04:54.196243+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:04:54.196243+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"43'4500","ondisk_log_start":"43'4500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27239366,"num_objects":6652,"num_object_clones":0,"num_object_copies":13304,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6652,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10711,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2329},{"shards":"0,2","objects":380}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":429208310,"num_objects":104797,"num_object_clones":0,"num_object_copies":209594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43305,"num_objects_misplaced":376,"num_objects_unfound":0,"num_objects_dirty":104797,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":187309,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7708,"num_bytes_recovered":31555584,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1051136000,"data_stored":1050986949,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160835,"ondisk_log_size":160835,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986259,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":670676,"kb_used_data":346048,"kb_used_omap":4,"kb_used_meta":324603,"kb_avail":93701164,"statfs":{"total":96636764160,"available":95949991936,"internally_reserved":0,"allocated":354353152,"data_stored":354060919,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":332394154},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,2,1,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659204,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":600492,"kb_used_data":298072,"kb_used_omap":4,"kb_used_meta":302395,"kb_avail":93771348,"statfs":{"total":96636764160,"available":96021860352,"internally_reserved":0,"allocated":305225728,"data_stored":304896901,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":309653166},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,3,2,5,2,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182487,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":777172,"kb_used_data":384580,"kb_used_omap":4,"kb_used_meta":392571,"kb_avail":93594668,"statfs":{"total":96636764160,"available":95840940032,"internally_reserved":0,"allocated":393809920,"data_stored":393517682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":401993386},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,0,1,5,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":392441856,"data_stored":392405726,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":304816128,"data_stored":304743409,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":353878016,"data_stored":353837814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:04:59.499 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:05:00.089 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:05:00.089 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:05:00.126 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":270,"stamp":"2024-09-10T16:04:59.673498+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":422658825,"num_objects":103200,"num_object_clones":0,"num_object_copies":206400,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43305,"num_objects_misplaced":224,"num_objects_unfound":0,"num_objects_dirty":103200,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":188910,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7862,"num_bytes_recovered":32178195,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160837,"ondisk_log_size":160837,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2053536,"kb_used_data":1021416,"kb_used_omap":12,"kb_used_meta":1032051,"kb_avail":281061984,"statfs":{"total":289910292480,"available":287807471616,"internally_reserved":0,"allocated":1045929984,"data_stored":1045016686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1056820226},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,5,3,12,5,6],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-24379392,"num_objects":-5952,"num_object_clones":0,"num_object_copies":-11904,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-456,"num_objects_unfound":0,"num_objects_dirty":-5952,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5952,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":456,"num_bytes_recovered":1867776,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006976"},"pg_stats":[{"pgid":"2.f","version":"45'14361","reported_seq":14437,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:59.214355+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:04:59.214355+0000","last_peered":"2024-09-10T16:04:59.214355+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:04:59.214355+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"43'4300","ondisk_log_start":"43'4300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25849856,"num_objects":6311,"num_object_clones":0,"num_object_copies":12622,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6311,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14361,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2052},{"shards":"0,2","objects":233}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'14395","reported_seq":14506,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:04:55.218204+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:04:55.218204+0000","last_peered":"2024-09-10T16:04:55.218204+0000","last_clean":"2024-09-10T16:04:55.218204+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:04:55.218204+0000","last_undegraded":"2024-09-10T16:04:55.218204+0000","last_fullsized":"2024-09-10T16:04:55.218204+0000","mapping_epoch":35,"log_start":"43'4300","ondisk_log_start":"43'4300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26416070,"num_objects":6451,"num_object_clones":0,"num_object_copies":12902,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6451,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":12500,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'14363","reported_seq":23042,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:04:57.029541+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:04:57.029541+0000","last_peered":"2024-09-10T16:04:57.029541+0000","last_clean":"2024-09-10T16:04:57.029541+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:04:57.029541+0000","last_undegraded":"2024-09-10T16:04:57.029541+0000","last_fullsized":"2024-09-10T16:04:57.029541+0000","mapping_epoch":44,"log_start":"43'4300","ondisk_log_start":"43'4300","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26353688,"num_objects":6435,"num_object_clones":0,"num_object_copies":12870,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6435,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12367,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'14176","reported_seq":14257,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:04:55.274785+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:04:55.274785+0000","last_peered":"2024-09-10T16:04:55.274785+0000","last_clean":"2024-09-10T16:04:55.274785+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:04:55.274785+0000","last_undegraded":"2024-09-10T16:04:55.274785+0000","last_fullsized":"2024-09-10T16:04:55.274785+0000","mapping_epoch":35,"log_start":"39'4100","ondisk_log_start":"39'4100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26558464,"num_objects":6484,"num_object_clones":0,"num_object_copies":12968,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6484,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14176,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'14502","reported_seq":14633,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:04:59.137522+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:04:59.137522+0000","last_peered":"2024-09-10T16:04:59.137522+0000","last_clean":"2024-09-10T16:04:59.137522+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:04:59.137522+0000","last_undegraded":"2024-09-10T16:04:59.137522+0000","last_fullsized":"2024-09-10T16:04:59.137522+0000","mapping_epoch":34,"log_start":"43'4500","ondisk_log_start":"43'4500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26169827,"num_objects":6390,"num_object_clones":0,"num_object_copies":12780,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6390,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11213,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'14660","reported_seq":14799,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:59.139642+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:04:59.139642+0000","last_peered":"2024-09-10T16:04:59.139642+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:04:59.139642+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"43'4600","ondisk_log_start":"43'4600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26218979,"num_objects":6402,"num_object_clones":0,"num_object_copies":12804,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6402,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11272,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2031},{"shards":"0,2","objects":244}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'14502","reported_seq":14637,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:59.191642+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:04:59.191642+0000","last_peered":"2024-09-10T16:04:59.191642+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:04:59.191642+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"43'4500","ondisk_log_start":"43'4500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26952646,"num_objects":6582,"num_object_clones":0,"num_object_copies":13164,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6582,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":11001,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2345},{"shards":"0,2","objects":279}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'14753","reported_seq":14884,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:04:59.190285+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:04:59.190285+0000","last_peered":"2024-09-10T16:04:59.190285+0000","last_clean":"2024-09-10T16:04:59.190285+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:04:59.190285+0000","last_undegraded":"2024-09-10T16:04:59.190285+0000","last_fullsized":"2024-09-10T16:04:59.190285+0000","mapping_epoch":34,"log_start":"43'4700","ondisk_log_start":"43'4700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26873856,"num_objects":6561,"num_object_clones":0,"num_object_copies":13122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6561,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11600,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104261+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:02:08.104261+0000","last_peered":"2024-09-10T16:02:08.104261+0000","last_clean":"2024-09-10T16:02:08.104261+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:02:08.104261+0000","last_undegraded":"2024-09-10T16:02:08.104261+0000","last_fullsized":"2024-09-10T16:02:08.104261+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'14023","reported_seq":14164,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:55.260864+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:04:55.260864+0000","last_peered":"2024-09-10T16:04:55.260864+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:04:55.260864+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"39'4000","ondisk_log_start":"39'4000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26354147,"num_objects":6435,"num_object_clones":0,"num_object_copies":12870,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6435,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11842,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":332},{"shards":"2","objects":2312}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":95,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:29.860108+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:02:29.860108+0000","last_peered":"2024-09-10T16:02:29.860108+0000","last_clean":"2024-09-10T16:02:29.860108+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:02:29.860108+0000","last_undegraded":"2024-09-10T16:02:29.860108+0000","last_fullsized":"2024-09-10T16:02:29.860108+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'14485","reported_seq":14616,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:59.108393+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:04:59.108393+0000","last_peered":"2024-09-10T16:04:59.108393+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:04:59.108393+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"43'4400","ondisk_log_start":"43'4400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26316800,"num_objects":6425,"num_object_clones":0,"num_object_copies":12850,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6425,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10989,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2170},{"shards":"0,2","objects":229}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":105,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:12.405385+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:02:12.405385+0000","last_peered":"2024-09-10T16:02:12.405385+0000","last_clean":"2024-09-10T16:02:12.405385+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:02:12.405385+0000","last_undegraded":"2024-09-10T16:02:12.405385+0000","last_fullsized":"2024-09-10T16:02:12.405385+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'14326","reported_seq":14461,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:55.218129+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:04:55.218129+0000","last_peered":"2024-09-10T16:04:55.218129+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:04:55.218129+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"40'4300","ondisk_log_start":"40'4300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26440163,"num_objects":6456,"num_object_clones":0,"num_object_copies":12912,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6456,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12006,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":371},{"shards":"2","objects":2152}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104617+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:02:08.104617+0000","last_peered":"2024-09-10T16:02:08.104617+0000","last_clean":"2024-09-10T16:02:08.104617+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:02:08.104617+0000","last_undegraded":"2024-09-10T16:02:08.104617+0000","last_fullsized":"2024-09-10T16:02:08.104617+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'14572","reported_seq":14701,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:55.253754+0000","last_change":"2024-09-10T15:59:16.174113+0000","last_active":"2024-09-10T16:04:55.253754+0000","last_peered":"2024-09-10T16:04:55.253754+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:04:55.253754+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"43'4500","ondisk_log_start":"43'4500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27025408,"num_objects":6598,"num_object_clones":0,"num_object_copies":13196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4354,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6598,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12126,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":368},{"shards":"2","objects":2246}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'14469","reported_seq":14603,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:59.266846+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:04:59.266846+0000","last_peered":"2024-09-10T16:04:59.266846+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:04:59.266846+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"43'4400","ondisk_log_start":"43'4400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26087424,"num_objects":6369,"num_object_clones":0,"num_object_copies":12738,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6369,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11100,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2065},{"shards":"0,2","objects":255}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103316+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:02:08.103316+0000","last_peered":"2024-09-10T16:02:08.103316+0000","last_clean":"2024-09-10T16:02:08.103316+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:02:08.103316+0000","last_undegraded":"2024-09-10T16:02:08.103316+0000","last_fullsized":"2024-09-10T16:02:08.103316+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'14236","reported_seq":21557,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:59.250188+0000","last_change":"2024-09-10T16:03:29.605682+0000","last_active":"2024-09-10T16:04:59.250188+0000","last_peered":"2024-09-10T16:04:59.250188+0000","last_clean":"2024-09-10T15:59:14.115595+0000","last_became_active":"2024-09-10T15:59:16.244089+0000","last_became_peered":"2024-09-10T15:59:16.244089+0000","last_unstale":"2024-09-10T16:04:59.250188+0000","last_undegraded":"2024-09-10T16:03:29.603815+0000","last_fullsized":"2024-09-10T15:59:15.774437+0000","mapping_epoch":42,"log_start":"40'4200","ondisk_log_start":"40'4200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26058752,"num_objects":6362,"num_object_clones":0,"num_object_copies":12724,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":394,"num_objects_misplaced":224,"num_objects_unfound":0,"num_objects_dirty":6362,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10626,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3588,"num_bytes_recovered":14684160,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5744},{"shards":"0,2","objects":224}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":95,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:29.877831+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:02:29.877831+0000","last_peered":"2024-09-10T16:02:29.877831+0000","last_clean":"2024-09-10T16:02:29.877831+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:02:29.877831+0000","last_undegraded":"2024-09-10T16:02:29.877831+0000","last_fullsized":"2024-09-10T16:02:29.877831+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'14640","reported_seq":14781,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:59.203782+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:04:59.203782+0000","last_peered":"2024-09-10T16:04:59.203782+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:04:59.203782+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"43'4600","ondisk_log_start":"43'4600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26247168,"num_objects":6408,"num_object_clones":0,"num_object_copies":12816,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6408,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10895,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2037},{"shards":"0,2","objects":257}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104463+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:02:08.104463+0000","last_peered":"2024-09-10T16:02:08.104463+0000","last_clean":"2024-09-10T16:02:08.104463+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:02:08.104463+0000","last_undegraded":"2024-09-10T16:02:08.104463+0000","last_fullsized":"2024-09-10T16:02:08.104463+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104448+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:02:08.104448+0000","last_peered":"2024-09-10T16:02:08.104448+0000","last_clean":"2024-09-10T16:02:08.104448+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:02:08.104448+0000","last_undegraded":"2024-09-10T16:02:08.104448+0000","last_fullsized":"2024-09-10T16:02:08.104448+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'14671","reported_seq":14816,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:59.215735+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:04:59.215735+0000","last_peered":"2024-09-10T16:04:59.215735+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:04:59.215735+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"43'4600","ondisk_log_start":"43'4600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26735558,"num_objects":6529,"num_object_clones":0,"num_object_copies":13058,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6529,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10834,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2206},{"shards":"0,2","objects":257}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":422658806,"num_objects":103198,"num_object_clones":0,"num_object_copies":206396,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43305,"num_objects_misplaced":224,"num_objects_unfound":0,"num_objects_dirty":103198,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":188908,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7860,"num_bytes_recovered":32178176,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1043402752,"data_stored":1043253701,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160834,"ondisk_log_size":160834,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986259,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":670676,"kb_used_data":346048,"kb_used_omap":4,"kb_used_meta":324603,"kb_avail":93701164,"statfs":{"total":96636764160,"available":95949991936,"internally_reserved":0,"allocated":354353152,"data_stored":354060919,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":332394154},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,2,1,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659205,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":608228,"kb_used_data":297492,"kb_used_omap":4,"kb_used_meta":310715,"kb_avail":93763612,"statfs":{"total":96636764160,"available":96013938688,"internally_reserved":0,"allocated":304631808,"data_stored":304302981,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":318172846},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,1,5],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182488,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":774632,"kb_used_data":377876,"kb_used_omap":4,"kb_used_meta":396731,"kb_avail":93597208,"statfs":{"total":96636764160,"available":95843540992,"internally_reserved":0,"allocated":386945024,"data_stored":386652786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":406253226},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,1,5,5,6],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":385343488,"data_stored":385307358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":304181248,"data_stored":304108529,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":353878016,"data_stored":353837814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:05:00.127 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:05:00.687 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:05:00.688 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:05:00.721 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":270,"stamp":"2024-09-10T16:04:59.673498+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":422658825,"num_objects":103200,"num_object_clones":0,"num_object_copies":206400,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43305,"num_objects_misplaced":224,"num_objects_unfound":0,"num_objects_dirty":103200,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":188910,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7862,"num_bytes_recovered":32178195,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160837,"ondisk_log_size":160837,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2053536,"kb_used_data":1021416,"kb_used_omap":12,"kb_used_meta":1032051,"kb_avail":281061984,"statfs":{"total":289910292480,"available":287807471616,"internally_reserved":0,"allocated":1045929984,"data_stored":1045016686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1056820226},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,5,3,12,5,6],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-24379392,"num_objects":-5952,"num_object_clones":0,"num_object_copies":-11904,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-456,"num_objects_unfound":0,"num_objects_dirty":-5952,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5952,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":456,"num_bytes_recovered":1867776,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006976"},"pg_stats":[{"pgid":"2.f","version":"45'14361","reported_seq":14437,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:59.214355+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:04:59.214355+0000","last_peered":"2024-09-10T16:04:59.214355+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:04:59.214355+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"43'4300","ondisk_log_start":"43'4300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25849856,"num_objects":6311,"num_object_clones":0,"num_object_copies":12622,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6311,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14361,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2052},{"shards":"0,2","objects":233}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'14395","reported_seq":14506,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:04:55.218204+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:04:55.218204+0000","last_peered":"2024-09-10T16:04:55.218204+0000","last_clean":"2024-09-10T16:04:55.218204+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:04:55.218204+0000","last_undegraded":"2024-09-10T16:04:55.218204+0000","last_fullsized":"2024-09-10T16:04:55.218204+0000","mapping_epoch":35,"log_start":"43'4300","ondisk_log_start":"43'4300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26416070,"num_objects":6451,"num_object_clones":0,"num_object_copies":12902,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6451,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":12500,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'14363","reported_seq":23042,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:04:57.029541+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:04:57.029541+0000","last_peered":"2024-09-10T16:04:57.029541+0000","last_clean":"2024-09-10T16:04:57.029541+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:04:57.029541+0000","last_undegraded":"2024-09-10T16:04:57.029541+0000","last_fullsized":"2024-09-10T16:04:57.029541+0000","mapping_epoch":44,"log_start":"43'4300","ondisk_log_start":"43'4300","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26353688,"num_objects":6435,"num_object_clones":0,"num_object_copies":12870,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6435,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12367,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'14176","reported_seq":14257,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:04:55.274785+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:04:55.274785+0000","last_peered":"2024-09-10T16:04:55.274785+0000","last_clean":"2024-09-10T16:04:55.274785+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:04:55.274785+0000","last_undegraded":"2024-09-10T16:04:55.274785+0000","last_fullsized":"2024-09-10T16:04:55.274785+0000","mapping_epoch":35,"log_start":"39'4100","ondisk_log_start":"39'4100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26558464,"num_objects":6484,"num_object_clones":0,"num_object_copies":12968,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6484,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14176,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'14502","reported_seq":14633,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:04:59.137522+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:04:59.137522+0000","last_peered":"2024-09-10T16:04:59.137522+0000","last_clean":"2024-09-10T16:04:59.137522+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:04:59.137522+0000","last_undegraded":"2024-09-10T16:04:59.137522+0000","last_fullsized":"2024-09-10T16:04:59.137522+0000","mapping_epoch":34,"log_start":"43'4500","ondisk_log_start":"43'4500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26169827,"num_objects":6390,"num_object_clones":0,"num_object_copies":12780,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6390,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11213,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'14660","reported_seq":14799,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:59.139642+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:04:59.139642+0000","last_peered":"2024-09-10T16:04:59.139642+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:04:59.139642+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"43'4600","ondisk_log_start":"43'4600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26218979,"num_objects":6402,"num_object_clones":0,"num_object_copies":12804,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6402,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11272,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2031},{"shards":"0,2","objects":244}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'14502","reported_seq":14637,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:59.191642+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:04:59.191642+0000","last_peered":"2024-09-10T16:04:59.191642+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:04:59.191642+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"43'4500","ondisk_log_start":"43'4500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26952646,"num_objects":6582,"num_object_clones":0,"num_object_copies":13164,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6582,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":11001,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2345},{"shards":"0,2","objects":279}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'14753","reported_seq":14884,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:04:59.190285+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:04:59.190285+0000","last_peered":"2024-09-10T16:04:59.190285+0000","last_clean":"2024-09-10T16:04:59.190285+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:04:59.190285+0000","last_undegraded":"2024-09-10T16:04:59.190285+0000","last_fullsized":"2024-09-10T16:04:59.190285+0000","mapping_epoch":34,"log_start":"43'4700","ondisk_log_start":"43'4700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26873856,"num_objects":6561,"num_object_clones":0,"num_object_copies":13122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6561,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11600,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104261+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:02:08.104261+0000","last_peered":"2024-09-10T16:02:08.104261+0000","last_clean":"2024-09-10T16:02:08.104261+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:02:08.104261+0000","last_undegraded":"2024-09-10T16:02:08.104261+0000","last_fullsized":"2024-09-10T16:02:08.104261+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'14023","reported_seq":14164,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:55.260864+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:04:55.260864+0000","last_peered":"2024-09-10T16:04:55.260864+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:04:55.260864+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"39'4000","ondisk_log_start":"39'4000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26354147,"num_objects":6435,"num_object_clones":0,"num_object_copies":12870,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6435,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11842,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":332},{"shards":"2","objects":2312}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":95,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:29.860108+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:02:29.860108+0000","last_peered":"2024-09-10T16:02:29.860108+0000","last_clean":"2024-09-10T16:02:29.860108+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:02:29.860108+0000","last_undegraded":"2024-09-10T16:02:29.860108+0000","last_fullsized":"2024-09-10T16:02:29.860108+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'14485","reported_seq":14616,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:59.108393+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:04:59.108393+0000","last_peered":"2024-09-10T16:04:59.108393+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:04:59.108393+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"43'4400","ondisk_log_start":"43'4400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26316800,"num_objects":6425,"num_object_clones":0,"num_object_copies":12850,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6425,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10989,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2170},{"shards":"0,2","objects":229}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":105,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:12.405385+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:02:12.405385+0000","last_peered":"2024-09-10T16:02:12.405385+0000","last_clean":"2024-09-10T16:02:12.405385+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:02:12.405385+0000","last_undegraded":"2024-09-10T16:02:12.405385+0000","last_fullsized":"2024-09-10T16:02:12.405385+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'14326","reported_seq":14461,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:55.218129+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:04:55.218129+0000","last_peered":"2024-09-10T16:04:55.218129+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:04:55.218129+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"40'4300","ondisk_log_start":"40'4300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26440163,"num_objects":6456,"num_object_clones":0,"num_object_copies":12912,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6456,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12006,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":371},{"shards":"2","objects":2152}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104617+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:02:08.104617+0000","last_peered":"2024-09-10T16:02:08.104617+0000","last_clean":"2024-09-10T16:02:08.104617+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:02:08.104617+0000","last_undegraded":"2024-09-10T16:02:08.104617+0000","last_fullsized":"2024-09-10T16:02:08.104617+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'14572","reported_seq":14701,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:55.253754+0000","last_change":"2024-09-10T15:59:16.174113+0000","last_active":"2024-09-10T16:04:55.253754+0000","last_peered":"2024-09-10T16:04:55.253754+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:04:55.253754+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"43'4500","ondisk_log_start":"43'4500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27025408,"num_objects":6598,"num_object_clones":0,"num_object_copies":13196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4354,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6598,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12126,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":368},{"shards":"2","objects":2246}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'14469","reported_seq":14603,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:59.266846+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:04:59.266846+0000","last_peered":"2024-09-10T16:04:59.266846+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:04:59.266846+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"43'4400","ondisk_log_start":"43'4400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26087424,"num_objects":6369,"num_object_clones":0,"num_object_copies":12738,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6369,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11100,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2065},{"shards":"0,2","objects":255}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103316+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:02:08.103316+0000","last_peered":"2024-09-10T16:02:08.103316+0000","last_clean":"2024-09-10T16:02:08.103316+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:02:08.103316+0000","last_undegraded":"2024-09-10T16:02:08.103316+0000","last_fullsized":"2024-09-10T16:02:08.103316+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'14236","reported_seq":21557,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:59.250188+0000","last_change":"2024-09-10T16:03:29.605682+0000","last_active":"2024-09-10T16:04:59.250188+0000","last_peered":"2024-09-10T16:04:59.250188+0000","last_clean":"2024-09-10T15:59:14.115595+0000","last_became_active":"2024-09-10T15:59:16.244089+0000","last_became_peered":"2024-09-10T15:59:16.244089+0000","last_unstale":"2024-09-10T16:04:59.250188+0000","last_undegraded":"2024-09-10T16:03:29.603815+0000","last_fullsized":"2024-09-10T15:59:15.774437+0000","mapping_epoch":42,"log_start":"40'4200","ondisk_log_start":"40'4200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26058752,"num_objects":6362,"num_object_clones":0,"num_object_copies":12724,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":394,"num_objects_misplaced":224,"num_objects_unfound":0,"num_objects_dirty":6362,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10626,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3588,"num_bytes_recovered":14684160,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5744},{"shards":"0,2","objects":224}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":95,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:29.877831+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:02:29.877831+0000","last_peered":"2024-09-10T16:02:29.877831+0000","last_clean":"2024-09-10T16:02:29.877831+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:02:29.877831+0000","last_undegraded":"2024-09-10T16:02:29.877831+0000","last_fullsized":"2024-09-10T16:02:29.877831+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'14640","reported_seq":14781,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:59.203782+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:04:59.203782+0000","last_peered":"2024-09-10T16:04:59.203782+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:04:59.203782+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"43'4600","ondisk_log_start":"43'4600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26247168,"num_objects":6408,"num_object_clones":0,"num_object_copies":12816,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6408,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10895,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2037},{"shards":"0,2","objects":257}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104463+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:02:08.104463+0000","last_peered":"2024-09-10T16:02:08.104463+0000","last_clean":"2024-09-10T16:02:08.104463+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:02:08.104463+0000","last_undegraded":"2024-09-10T16:02:08.104463+0000","last_fullsized":"2024-09-10T16:02:08.104463+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104448+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:02:08.104448+0000","last_peered":"2024-09-10T16:02:08.104448+0000","last_clean":"2024-09-10T16:02:08.104448+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:02:08.104448+0000","last_undegraded":"2024-09-10T16:02:08.104448+0000","last_fullsized":"2024-09-10T16:02:08.104448+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'14671","reported_seq":14816,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:59.215735+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:04:59.215735+0000","last_peered":"2024-09-10T16:04:59.215735+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:04:59.215735+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"43'4600","ondisk_log_start":"43'4600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26735558,"num_objects":6529,"num_object_clones":0,"num_object_copies":13058,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6529,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10834,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2206},{"shards":"0,2","objects":257}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":422658806,"num_objects":103198,"num_object_clones":0,"num_object_copies":206396,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43305,"num_objects_misplaced":224,"num_objects_unfound":0,"num_objects_dirty":103198,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":188908,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7860,"num_bytes_recovered":32178176,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1043402752,"data_stored":1043253701,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160834,"ondisk_log_size":160834,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986259,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":670676,"kb_used_data":346048,"kb_used_omap":4,"kb_used_meta":324603,"kb_avail":93701164,"statfs":{"total":96636764160,"available":95949991936,"internally_reserved":0,"allocated":354353152,"data_stored":354060919,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":332394154},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,2,1,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659205,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":608228,"kb_used_data":297492,"kb_used_omap":4,"kb_used_meta":310715,"kb_avail":93763612,"statfs":{"total":96636764160,"available":96013938688,"internally_reserved":0,"allocated":304631808,"data_stored":304302981,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":318172846},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,1,5],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182488,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":774632,"kb_used_data":377876,"kb_used_omap":4,"kb_used_meta":396731,"kb_avail":93597208,"statfs":{"total":96636764160,"available":95843540992,"internally_reserved":0,"allocated":386945024,"data_stored":386652786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":406253226},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,1,5,5,6],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":385343488,"data_stored":385307358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":304181248,"data_stored":304108529,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":353878016,"data_stored":353837814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:05:00.722 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-10T16:05:00.722 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "45'14361", 'reported_seq': 14437, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:04:59.214355+0000', 'last_change': '2024-09-10T15:59:16.281558+0000', 'last_active': '2024-09-10T16:04:59.214355+0000', 'last_peered': '2024-09-10T16:04:59.214355+0000', 'last_clean': '2024-09-10T15:59:14.132456+0000', 'last_became_active': '2024-09-10T15:59:16.274407+0000', 'last_became_peered': '2024-09-10T15:59:16.274407+0000', 'last_unstale': '2024-09-10T16:04:59.214355+0000', 'last_undegraded': '2024-09-10T15:59:15.851279+0000', 'last_fullsized': '2024-09-10T15:59:15.774356+0000', 'mapping_epoch': 42, 'log_start': "43'4300", 'ondisk_log_start': "43'4300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'objects_scrubbed': 0, 'log_size': 10061, 'log_dups_size': 0, 'ondisk_log_size': 10061, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25849856, 'num_objects': 6311, 'num_object_clones': 0, 'num_object_copies': 12622, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4259, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6311, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14361, 'num_write_kb': 41344, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2052}, {'shards': '0,2', 'objects': 233}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:05:00.722 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-10T16:05:00.722 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "45'14660", 'reported_seq': 14799, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:04:59.139642+0000', 'last_change': '2024-09-10T15:59:16.308264+0000', 'last_active': '2024-09-10T16:04:59.139642+0000', 'last_peered': '2024-09-10T16:04:59.139642+0000', 'last_clean': '2024-09-10T15:59:14.111551+0000', 'last_became_active': '2024-09-10T15:59:16.300040+0000', 'last_became_peered': '2024-09-10T15:59:16.300040+0000', 'last_unstale': '2024-09-10T16:04:59.139642+0000', 'last_undegraded': '2024-09-10T15:59:15.878234+0000', 'last_fullsized': '2024-09-10T15:59:15.769328+0000', 'mapping_epoch': 42, 'log_start': "43'4600", 'ondisk_log_start': "43'4600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3442", 'last_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'objects_scrubbed': 3384, 'log_size': 10060, 'log_dups_size': 0, 'ondisk_log_size': 10060, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1292, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26218979, 'num_objects': 6402, 'num_object_clones': 0, 'num_object_copies': 12804, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4372, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6402, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 11272, 'num_write_kb': 28580, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2031}, {'shards': '0,2', 'objects': 244}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:05:00.722 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-10T16:05:00.722 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "45'14502", 'reported_seq': 14637, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:04:59.191642+0000', 'last_change': '2024-09-10T15:59:16.331899+0000', 'last_active': '2024-09-10T16:04:59.191642+0000', 'last_peered': '2024-09-10T16:04:59.191642+0000', 'last_clean': '2024-09-10T15:59:14.139436+0000', 'last_became_active': '2024-09-10T15:59:16.318715+0000', 'last_became_peered': '2024-09-10T15:59:16.318715+0000', 'last_unstale': '2024-09-10T16:04:59.191642+0000', 'last_undegraded': '2024-09-10T15:59:15.877524+0000', 'last_fullsized': '2024-09-10T15:59:15.769042+0000', 'mapping_epoch': 42, 'log_start': "43'4500", 'ondisk_log_start': "43'4500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3537", 'last_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'objects_scrubbed': 3497, 'log_size': 10002, 'log_dups_size': 0, 'ondisk_log_size': 10002, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1224, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26952646, 'num_objects': 6582, 'num_object_clones': 0, 'num_object_copies': 13164, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4237, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6582, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 11001, 'num_write_kb': 28172, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2345}, {'shards': '0,2', 'objects': 279}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:05:00.723 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-10T16:05:00.723 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "45'14023", 'reported_seq': 14164, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:04:55.260864+0000', 'last_change': '2024-09-10T15:59:16.175680+0000', 'last_active': '2024-09-10T16:04:55.260864+0000', 'last_peered': '2024-09-10T16:04:55.260864+0000', 'last_clean': '2024-09-10T15:59:14.088187+0000', 'last_became_active': '2024-09-10T15:59:16.165810+0000', 'last_became_peered': '2024-09-10T15:59:16.165810+0000', 'last_unstale': '2024-09-10T16:04:55.260864+0000', 'last_undegraded': '2024-09-10T15:59:15.822098+0000', 'last_fullsized': '2024-09-10T15:59:15.769123+0000', 'mapping_epoch': 42, 'log_start': "39'4000", 'ondisk_log_start': "39'4000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2218", 'last_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'objects_scrubbed': 2175, 'log_size': 10023, 'log_dups_size': 0, 'ondisk_log_size': 10023, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 825, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26354147, 'num_objects': 6435, 'num_object_clones': 0, 'num_object_copies': 12870, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4124, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6435, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 11842, 'num_write_kb': 32204, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 332}, {'shards': '2', 'objects': 2312}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:05:00.723 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-10T16:05:00.723 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "45'14485", 'reported_seq': 14616, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:04:59.108393+0000', 'last_change': '2024-09-10T15:59:16.277742+0000', 'last_active': '2024-09-10T16:04:59.108393+0000', 'last_peered': '2024-09-10T16:04:59.108393+0000', 'last_clean': '2024-09-10T15:59:14.129081+0000', 'last_became_active': '2024-09-10T15:59:16.269835+0000', 'last_became_peered': '2024-09-10T15:59:16.269835+0000', 'last_unstale': '2024-09-10T16:04:59.108393+0000', 'last_undegraded': '2024-09-10T15:59:15.860857+0000', 'last_fullsized': '2024-09-10T15:59:15.771410+0000', 'mapping_epoch': 42, 'log_start': "43'4400", 'ondisk_log_start': "43'4400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3557", 'last_scrub_stamp': '2024-09-10T15:59:02.493678+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:02.493678+0000', 'objects_scrubbed': 3488, 'log_size': 10085, 'log_dups_size': 0, 'ondisk_log_size': 10085, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1472, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26316800, 'num_objects': 6425, 'num_object_clones': 0, 'num_object_copies': 12850, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4257, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6425, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10989, 'num_write_kb': 27852, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2170}, {'shards': '0,2', 'objects': 229}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:05:00.723 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-10T16:05:00.723 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "45'14326", 'reported_seq': 14461, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:04:55.218129+0000', 'last_change': '2024-09-10T15:59:16.284311+0000', 'last_active': '2024-09-10T16:04:55.218129+0000', 'last_peered': '2024-09-10T16:04:55.218129+0000', 'last_clean': '2024-09-10T15:59:14.116041+0000', 'last_became_active': '2024-09-10T15:59:16.274543+0000', 'last_became_peered': '2024-09-10T15:59:16.274543+0000', 'last_unstale': '2024-09-10T16:04:55.218129+0000', 'last_undegraded': '2024-09-10T15:59:15.812484+0000', 'last_fullsized': '2024-09-10T15:59:15.770885+0000', 'mapping_epoch': 42, 'log_start': "40'4300", 'ondisk_log_start': "40'4300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2356", 'last_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'objects_scrubbed': 2316, 'log_size': 10026, 'log_dups_size': 0, 'ondisk_log_size': 10026, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 783, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26440163, 'num_objects': 6456, 'num_object_clones': 0, 'num_object_copies': 12912, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6456, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 12006, 'num_write_kb': 32292, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 371}, {'shards': '2', 'objects': 2152}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:05:00.723 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-10T16:05:00.723 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "45'14572", 'reported_seq': 14701, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:04:55.253754+0000', 'last_change': '2024-09-10T15:59:16.174113+0000', 'last_active': '2024-09-10T16:04:55.253754+0000', 'last_peered': '2024-09-10T16:04:55.253754+0000', 'last_clean': '2024-09-10T15:59:14.110654+0000', 'last_became_active': '2024-09-10T15:59:16.164645+0000', 'last_became_peered': '2024-09-10T15:59:16.164645+0000', 'last_unstale': '2024-09-10T16:04:55.253754+0000', 'last_undegraded': '2024-09-10T15:59:15.803593+0000', 'last_fullsized': '2024-09-10T15:59:15.769083+0000', 'mapping_epoch': 42, 'log_start': "43'4500", 'ondisk_log_start': "43'4500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2486", 'last_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'objects_scrubbed': 2440, 'log_size': 10072, 'log_dups_size': 0, 'ondisk_log_size': 10072, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 792, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27025408, 'num_objects': 6598, 'num_object_clones': 0, 'num_object_copies': 13196, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4354, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6598, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12126, 'num_write_kb': 32568, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 368}, {'shards': '2', 'objects': 2246}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:05:00.723 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-10T16:05:00.723 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "45'14469", 'reported_seq': 14603, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:04:59.266846+0000', 'last_change': '2024-09-10T15:59:16.315641+0000', 'last_active': '2024-09-10T16:04:59.266846+0000', 'last_peered': '2024-09-10T16:04:59.266846+0000', 'last_clean': '2024-09-10T15:59:14.117001+0000', 'last_became_active': '2024-09-10T15:59:16.302245+0000', 'last_became_peered': '2024-09-10T15:59:16.302245+0000', 'last_unstale': '2024-09-10T16:04:59.266846+0000', 'last_undegraded': '2024-09-10T15:59:15.845012+0000', 'last_fullsized': '2024-09-10T15:59:15.770337+0000', 'mapping_epoch': 42, 'log_start': "43'4400", 'ondisk_log_start': "43'4400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3404", 'last_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'objects_scrubbed': 3367, 'log_size': 10069, 'log_dups_size': 0, 'ondisk_log_size': 10069, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 983, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26087424, 'num_objects': 6369, 'num_object_clones': 0, 'num_object_copies': 12738, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6369, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11100, 'num_write_kb': 28204, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2065}, {'shards': '0,2', 'objects': 255}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:05:00.723 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-10T16:05:00.724 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "45'14236", 'reported_seq': 21557, 'reported_epoch': 45, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:04:59.250188+0000', 'last_change': '2024-09-10T16:03:29.605682+0000', 'last_active': '2024-09-10T16:04:59.250188+0000', 'last_peered': '2024-09-10T16:04:59.250188+0000', 'last_clean': '2024-09-10T15:59:14.115595+0000', 'last_became_active': '2024-09-10T15:59:16.244089+0000', 'last_became_peered': '2024-09-10T15:59:16.244089+0000', 'last_unstale': '2024-09-10T16:04:59.250188+0000', 'last_undegraded': '2024-09-10T16:03:29.603815+0000', 'last_fullsized': '2024-09-10T15:59:15.774437+0000', 'mapping_epoch': 42, 'log_start': "40'4200", 'ondisk_log_start': "40'4200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3653", 'last_scrub_stamp': '2024-09-10T15:59:05.298468+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:05.298468+0000', 'objects_scrubbed': 3602, 'log_size': 10036, 'log_dups_size': 0, 'ondisk_log_size': 10036, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1289, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26058752, 'num_objects': 6362, 'num_object_clones': 0, 'num_object_copies': 12724, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 394, 'num_objects_misplaced': 224, 'num_objects_unfound': 0, 'num_objects_dirty': 6362, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10626, 'num_write_kb': 26772, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3588, 'num_bytes_recovered': 14684160, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5744}, {'shards': '0,2', 'objects': 224}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:05:00.724 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-10T16:05:00.724 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "45'14640", 'reported_seq': 14781, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:04:59.203782+0000', 'last_change': '2024-09-10T15:59:16.311291+0000', 'last_active': '2024-09-10T16:04:59.203782+0000', 'last_peered': '2024-09-10T16:04:59.203782+0000', 'last_clean': '2024-09-10T15:59:14.116363+0000', 'last_became_active': '2024-09-10T15:59:16.301244+0000', 'last_became_peered': '2024-09-10T15:59:16.301244+0000', 'last_unstale': '2024-09-10T16:04:59.203782+0000', 'last_undegraded': '2024-09-10T15:59:15.883434+0000', 'last_fullsized': '2024-09-10T15:59:15.772756+0000', 'mapping_epoch': 42, 'log_start': "43'4600", 'ondisk_log_start': "43'4600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3789", 'last_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'objects_scrubbed': 3741, 'log_size': 10040, 'log_dups_size': 0, 'ondisk_log_size': 10040, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1440, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26247168, 'num_objects': 6408, 'num_object_clones': 0, 'num_object_copies': 12816, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4373, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6408, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10895, 'num_write_kb': 27124, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2037}, {'shards': '0,2', 'objects': 257}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:05:00.724 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-10T16:05:00.724 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "45'14671", 'reported_seq': 14816, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:04:59.215735+0000', 'last_change': '2024-09-10T15:59:16.313409+0000', 'last_active': '2024-09-10T16:04:59.215735+0000', 'last_peered': '2024-09-10T16:04:59.215735+0000', 'last_clean': '2024-09-10T15:59:14.129780+0000', 'last_became_active': '2024-09-10T15:59:16.300932+0000', 'last_became_peered': '2024-09-10T15:59:16.300932+0000', 'last_unstale': '2024-09-10T16:04:59.215735+0000', 'last_undegraded': '2024-09-10T15:59:15.887823+0000', 'last_fullsized': '2024-09-10T15:59:15.768513+0000', 'mapping_epoch': 42, 'log_start': "43'4600", 'ondisk_log_start': "43'4600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3880", 'last_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'objects_scrubbed': 3827, 'log_size': 10071, 'log_dups_size': 0, 'ondisk_log_size': 10071, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1397, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26735558, 'num_objects': 6529, 'num_object_clones': 0, 'num_object_copies': 13058, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4325, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6529, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10834, 'num_write_kb': 27072, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2206}, {'shards': '0,2', 'objects': 257}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:05:00.724 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T16:05:01.361 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:05:01.361 INFO:teuthology.orchestra.run.smithi115.stdout:{"fsid":"7effee29-df05-4351-9741-ee203564ed81","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 43305/206400 objects degraded (20.981%), 11 pgs degraded, 11 pgs undersized","count":22},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":465,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983954,"num_in_osds":3,"osd_in_since":1725983947,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":103200,"data_bytes":422658825,"bytes_used":2102820864,"bytes_avail":287807471616,"bytes_total":289910292480,"degraded_objects":43305,"degraded_total":206400,"degraded_ratio":0.20981104651162791,"misplaced_objects":224,"misplaced_total":206400,"misplaced_ratio":0.0010852713178294573,"recovering_objects_per_sec":37,"recovering_bytes_per_sec":155557,"recovering_keys_per_sec":0,"num_objects_recovered":456,"num_bytes_recovered":1867776,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":495},"fsmap":{"epoch":1,"btime":"2024-09-10T15:57:14:967120+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T16:00:13.560776+0000","services":{}},"progress_events":{}} 2024-09-10T16:05:01.384 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-10T16:05:01.385 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:05:01.929 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:05:01.930 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:05:01.953 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":271,"stamp":"2024-09-10T16:05:01.674297+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":419812105,"num_objects":102505,"num_object_clones":0,"num_object_copies":205010,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43305,"num_objects_misplaced":224,"num_objects_unfound":0,"num_objects_dirty":102505,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":189605,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7862,"num_bytes_recovered":32178195,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160732,"ondisk_log_size":160732,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2053240,"kb_used_data":1016964,"kb_used_omap":12,"kb_used_meta":1036211,"kb_avail":281062280,"statfs":{"total":289910292480,"available":287807774720,"internally_reserved":0,"allocated":1041371136,"data_stored":1040457838,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1061080066},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,3,4,10,5,6],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-21385216,"num_objects":-5221,"num_object_clones":0,"num_object_copies":-10442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-310,"num_objects_unfound":0,"num_objects_dirty":-5221,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5221,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":310,"num_bytes_recovered":1269760,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004285"},"pg_stats":[{"pgid":"2.f","version":"45'14361","reported_seq":14437,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:59.214355+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:04:59.214355+0000","last_peered":"2024-09-10T16:04:59.214355+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:04:59.214355+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"43'4300","ondisk_log_start":"43'4300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25849856,"num_objects":6311,"num_object_clones":0,"num_object_copies":12622,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6311,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14361,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2052},{"shards":"0,2","objects":233}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'14513","reported_seq":14626,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:05:00.235212+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:05:00.235212+0000","last_peered":"2024-09-10T16:05:00.235212+0000","last_clean":"2024-09-10T16:05:00.235212+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:05:00.235212+0000","last_undegraded":"2024-09-10T16:05:00.235212+0000","last_fullsized":"2024-09-10T16:05:00.235212+0000","mapping_epoch":35,"log_start":"43'4500","ondisk_log_start":"43'4500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25932742,"num_objects":6333,"num_object_clones":0,"num_object_copies":12666,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6333,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":12618,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'14363","reported_seq":23042,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:04:57.029541+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:04:57.029541+0000","last_peered":"2024-09-10T16:04:57.029541+0000","last_clean":"2024-09-10T16:04:57.029541+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:04:57.029541+0000","last_undegraded":"2024-09-10T16:04:57.029541+0000","last_fullsized":"2024-09-10T16:04:57.029541+0000","mapping_epoch":44,"log_start":"43'4300","ondisk_log_start":"43'4300","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26353688,"num_objects":6435,"num_object_clones":0,"num_object_copies":12870,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6435,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12367,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'14337","reported_seq":14418,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:05:00.034217+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:05:00.034217+0000","last_peered":"2024-09-10T16:05:00.034217+0000","last_clean":"2024-09-10T16:05:00.034217+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:05:00.034217+0000","last_undegraded":"2024-09-10T16:05:00.034217+0000","last_fullsized":"2024-09-10T16:05:00.034217+0000","mapping_epoch":35,"log_start":"43'4300","ondisk_log_start":"43'4300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25899008,"num_objects":6323,"num_object_clones":0,"num_object_copies":12646,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6323,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14337,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'14502","reported_seq":14633,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:04:59.137522+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:04:59.137522+0000","last_peered":"2024-09-10T16:04:59.137522+0000","last_clean":"2024-09-10T16:04:59.137522+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:04:59.137522+0000","last_undegraded":"2024-09-10T16:04:59.137522+0000","last_fullsized":"2024-09-10T16:04:59.137522+0000","mapping_epoch":34,"log_start":"43'4500","ondisk_log_start":"43'4500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26169827,"num_objects":6390,"num_object_clones":0,"num_object_copies":12780,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6390,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11213,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'14660","reported_seq":14799,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:59.139642+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:04:59.139642+0000","last_peered":"2024-09-10T16:04:59.139642+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:04:59.139642+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"43'4600","ondisk_log_start":"43'4600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26218979,"num_objects":6402,"num_object_clones":0,"num_object_copies":12804,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6402,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11272,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2031},{"shards":"0,2","objects":244}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'14502","reported_seq":14637,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:59.191642+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:04:59.191642+0000","last_peered":"2024-09-10T16:04:59.191642+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:04:59.191642+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"43'4500","ondisk_log_start":"43'4500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26952646,"num_objects":6582,"num_object_clones":0,"num_object_copies":13164,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6582,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":11001,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2345},{"shards":"0,2","objects":279}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'14753","reported_seq":14884,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:04:59.190285+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:04:59.190285+0000","last_peered":"2024-09-10T16:04:59.190285+0000","last_clean":"2024-09-10T16:04:59.190285+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:04:59.190285+0000","last_undegraded":"2024-09-10T16:04:59.190285+0000","last_fullsized":"2024-09-10T16:04:59.190285+0000","mapping_epoch":34,"log_start":"43'4700","ondisk_log_start":"43'4700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26873856,"num_objects":6561,"num_object_clones":0,"num_object_copies":13122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6561,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11600,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104261+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:02:08.104261+0000","last_peered":"2024-09-10T16:02:08.104261+0000","last_clean":"2024-09-10T16:02:08.104261+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:02:08.104261+0000","last_undegraded":"2024-09-10T16:02:08.104261+0000","last_fullsized":"2024-09-10T16:02:08.104261+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'14150","reported_seq":14291,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:00.279797+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:05:00.279797+0000","last_peered":"2024-09-10T16:05:00.279797+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:05:00.279797+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"40'4100","ondisk_log_start":"40'4100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25833955,"num_objects":6308,"num_object_clones":0,"num_object_copies":12616,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6308,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11969,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":205},{"shards":"2","objects":2185}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":95,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:29.860108+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:02:29.860108+0000","last_peered":"2024-09-10T16:02:29.860108+0000","last_clean":"2024-09-10T16:02:29.860108+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:02:29.860108+0000","last_undegraded":"2024-09-10T16:02:29.860108+0000","last_fullsized":"2024-09-10T16:02:29.860108+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'14485","reported_seq":14616,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:59.108393+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:04:59.108393+0000","last_peered":"2024-09-10T16:04:59.108393+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:04:59.108393+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"43'4400","ondisk_log_start":"43'4400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26316800,"num_objects":6425,"num_object_clones":0,"num_object_copies":12850,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6425,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10989,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2170},{"shards":"0,2","objects":229}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":105,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:12.405385+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:02:12.405385+0000","last_peered":"2024-09-10T16:02:12.405385+0000","last_clean":"2024-09-10T16:02:12.405385+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:02:12.405385+0000","last_undegraded":"2024-09-10T16:02:12.405385+0000","last_fullsized":"2024-09-10T16:02:12.405385+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'14473","reported_seq":14609,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:00.286720+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:05:00.286720+0000","last_peered":"2024-09-10T16:05:00.286720+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:05:00.286720+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"43'4400","ondisk_log_start":"43'4400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25838051,"num_objects":6309,"num_object_clones":0,"num_object_copies":12618,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6309,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12153,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":224},{"shards":"2","objects":2005}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104617+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:02:08.104617+0000","last_peered":"2024-09-10T16:02:08.104617+0000","last_clean":"2024-09-10T16:02:08.104617+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:02:08.104617+0000","last_undegraded":"2024-09-10T16:02:08.104617+0000","last_fullsized":"2024-09-10T16:02:08.104617+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'14714","reported_seq":14844,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:00.303965+0000","last_change":"2024-09-10T15:59:16.174113+0000","last_active":"2024-09-10T16:05:00.303965+0000","last_peered":"2024-09-10T16:05:00.303965+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:05:00.303965+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"43'4700","ondisk_log_start":"43'4700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26443776,"num_objects":6456,"num_object_clones":0,"num_object_copies":12912,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4354,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6456,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12268,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":226},{"shards":"2","objects":2104}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'14469","reported_seq":14603,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:59.266846+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:04:59.266846+0000","last_peered":"2024-09-10T16:04:59.266846+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:04:59.266846+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"43'4400","ondisk_log_start":"43'4400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26087424,"num_objects":6369,"num_object_clones":0,"num_object_copies":12738,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6369,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11100,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2065},{"shards":"0,2","objects":255}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103316+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:02:08.103316+0000","last_peered":"2024-09-10T16:02:08.103316+0000","last_clean":"2024-09-10T16:02:08.103316+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:02:08.103316+0000","last_undegraded":"2024-09-10T16:02:08.103316+0000","last_fullsized":"2024-09-10T16:02:08.103316+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'14236","reported_seq":21557,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:59.250188+0000","last_change":"2024-09-10T16:03:29.605682+0000","last_active":"2024-09-10T16:04:59.250188+0000","last_peered":"2024-09-10T16:04:59.250188+0000","last_clean":"2024-09-10T15:59:14.115595+0000","last_became_active":"2024-09-10T15:59:16.244089+0000","last_became_peered":"2024-09-10T15:59:16.244089+0000","last_unstale":"2024-09-10T16:04:59.250188+0000","last_undegraded":"2024-09-10T16:03:29.603815+0000","last_fullsized":"2024-09-10T15:59:15.774437+0000","mapping_epoch":42,"log_start":"40'4200","ondisk_log_start":"40'4200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26058752,"num_objects":6362,"num_object_clones":0,"num_object_copies":12724,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":394,"num_objects_misplaced":224,"num_objects_unfound":0,"num_objects_dirty":6362,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10626,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3588,"num_bytes_recovered":14684160,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5744},{"shards":"0,2","objects":224}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":95,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:29.877831+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:02:29.877831+0000","last_peered":"2024-09-10T16:02:29.877831+0000","last_clean":"2024-09-10T16:02:29.877831+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:02:29.877831+0000","last_undegraded":"2024-09-10T16:02:29.877831+0000","last_fullsized":"2024-09-10T16:02:29.877831+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'14640","reported_seq":14781,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:59.203782+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:04:59.203782+0000","last_peered":"2024-09-10T16:04:59.203782+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:04:59.203782+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"43'4600","ondisk_log_start":"43'4600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26247168,"num_objects":6408,"num_object_clones":0,"num_object_copies":12816,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6408,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10895,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2037},{"shards":"0,2","objects":257}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104463+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:02:08.104463+0000","last_peered":"2024-09-10T16:02:08.104463+0000","last_clean":"2024-09-10T16:02:08.104463+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:02:08.104463+0000","last_undegraded":"2024-09-10T16:02:08.104463+0000","last_fullsized":"2024-09-10T16:02:08.104463+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104448+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:02:08.104448+0000","last_peered":"2024-09-10T16:02:08.104448+0000","last_clean":"2024-09-10T16:02:08.104448+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:02:08.104448+0000","last_undegraded":"2024-09-10T16:02:08.104448+0000","last_fullsized":"2024-09-10T16:02:08.104448+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'14671","reported_seq":14816,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:59.215735+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:04:59.215735+0000","last_peered":"2024-09-10T16:04:59.215735+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:04:59.215735+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"43'4600","ondisk_log_start":"43'4600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26735558,"num_objects":6529,"num_object_clones":0,"num_object_copies":13058,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6529,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10834,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2206},{"shards":"0,2","objects":257}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":419812086,"num_objects":102503,"num_object_clones":0,"num_object_copies":205006,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43305,"num_objects_misplaced":224,"num_objects_unfound":0,"num_objects_dirty":102503,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":189603,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7860,"num_bytes_recovered":32178176,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1038757888,"data_stored":1038608837,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160729,"ondisk_log_size":160729,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986260,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":670380,"kb_used_data":341596,"kb_used_omap":4,"kb_used_meta":328763,"kb_avail":93701460,"statfs":{"total":96636764160,"available":95950295040,"internally_reserved":0,"allocated":349794304,"data_stored":349502071,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":336653994},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,2],"upper_bound":32},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659205,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":608228,"kb_used_data":297492,"kb_used_omap":4,"kb_used_meta":310715,"kb_avail":93763612,"statfs":{"total":96636764160,"available":96013938688,"internally_reserved":0,"allocated":304631808,"data_stored":304302981,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":318172846},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,1,5],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182488,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":774632,"kb_used_data":377876,"kb_used_omap":4,"kb_used_meta":396731,"kb_avail":93597208,"statfs":{"total":96636764160,"available":95843540992,"internally_reserved":0,"allocated":386945024,"data_stored":386652786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":406253226},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,1,5,5,6],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":385343488,"data_stored":385307358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":304181248,"data_stored":304108529,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":349233152,"data_stored":349192950,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:05:04.955 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:05:05.494 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:05:05.494 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:05:05.525 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":272,"stamp":"2024-09-10T16:05:03.674862+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":419160841,"num_objects":102346,"num_object_clones":0,"num_object_copies":204692,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43305,"num_objects_misplaced":224,"num_objects_unfound":0,"num_objects_dirty":102346,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":189764,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7862,"num_bytes_recovered":32178195,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160691,"ondisk_log_size":160691,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2040136,"kb_used_data":1016336,"kb_used_omap":12,"kb_used_meta":1023731,"kb_avail":281075384,"statfs":{"total":289910292480,"available":287821193216,"internally_reserved":0,"allocated":1040728064,"data_stored":1039814766,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1048300546},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,4,2,4,9,6,7],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19255296,"num_objects":-4701,"num_object_clones":0,"num_object_copies":-9402,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-310,"num_objects_unfound":0,"num_objects_dirty":-4701,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4701,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":310,"num_bytes_recovered":1269760,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004205"},"pg_stats":[{"pgid":"2.f","version":"45'14361","reported_seq":14437,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:59.214355+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:04:59.214355+0000","last_peered":"2024-09-10T16:04:59.214355+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:04:59.214355+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"43'4300","ondisk_log_start":"43'4300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25849856,"num_objects":6311,"num_object_clones":0,"num_object_copies":12622,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6311,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14361,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2052},{"shards":"0,2","objects":233}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'14513","reported_seq":14626,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:05:00.235212+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:05:00.235212+0000","last_peered":"2024-09-10T16:05:00.235212+0000","last_clean":"2024-09-10T16:05:00.235212+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:05:00.235212+0000","last_undegraded":"2024-09-10T16:05:00.235212+0000","last_fullsized":"2024-09-10T16:05:00.235212+0000","mapping_epoch":35,"log_start":"43'4500","ondisk_log_start":"43'4500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25932742,"num_objects":6333,"num_object_clones":0,"num_object_copies":12666,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6333,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":12618,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'14522","reported_seq":23201,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:05:02.030442+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:05:02.030442+0000","last_peered":"2024-09-10T16:05:02.030442+0000","last_clean":"2024-09-10T16:05:02.030442+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:05:02.030442+0000","last_undegraded":"2024-09-10T16:05:02.030442+0000","last_fullsized":"2024-09-10T16:05:02.030442+0000","mapping_epoch":44,"log_start":"43'4500","ondisk_log_start":"43'4500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25702424,"num_objects":6276,"num_object_clones":0,"num_object_copies":12552,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6276,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12526,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'14337","reported_seq":14418,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:05:00.034217+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:05:00.034217+0000","last_peered":"2024-09-10T16:05:00.034217+0000","last_clean":"2024-09-10T16:05:00.034217+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:05:00.034217+0000","last_undegraded":"2024-09-10T16:05:00.034217+0000","last_fullsized":"2024-09-10T16:05:00.034217+0000","mapping_epoch":35,"log_start":"43'4300","ondisk_log_start":"43'4300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25899008,"num_objects":6323,"num_object_clones":0,"num_object_copies":12646,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6323,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14337,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'14502","reported_seq":14633,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:04:59.137522+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:04:59.137522+0000","last_peered":"2024-09-10T16:04:59.137522+0000","last_clean":"2024-09-10T16:04:59.137522+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:04:59.137522+0000","last_undegraded":"2024-09-10T16:04:59.137522+0000","last_fullsized":"2024-09-10T16:04:59.137522+0000","mapping_epoch":34,"log_start":"43'4500","ondisk_log_start":"43'4500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26169827,"num_objects":6390,"num_object_clones":0,"num_object_copies":12780,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6390,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11213,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'14660","reported_seq":14799,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:59.139642+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:04:59.139642+0000","last_peered":"2024-09-10T16:04:59.139642+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:04:59.139642+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"43'4600","ondisk_log_start":"43'4600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26218979,"num_objects":6402,"num_object_clones":0,"num_object_copies":12804,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6402,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11272,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2031},{"shards":"0,2","objects":244}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'14502","reported_seq":14637,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:59.191642+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:04:59.191642+0000","last_peered":"2024-09-10T16:04:59.191642+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:04:59.191642+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"43'4500","ondisk_log_start":"43'4500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26952646,"num_objects":6582,"num_object_clones":0,"num_object_copies":13164,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6582,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":11001,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2345},{"shards":"0,2","objects":279}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'14753","reported_seq":14884,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:04:59.190285+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:04:59.190285+0000","last_peered":"2024-09-10T16:04:59.190285+0000","last_clean":"2024-09-10T16:04:59.190285+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:04:59.190285+0000","last_undegraded":"2024-09-10T16:04:59.190285+0000","last_fullsized":"2024-09-10T16:04:59.190285+0000","mapping_epoch":34,"log_start":"43'4700","ondisk_log_start":"43'4700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26873856,"num_objects":6561,"num_object_clones":0,"num_object_copies":13122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6561,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11600,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104261+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:02:08.104261+0000","last_peered":"2024-09-10T16:02:08.104261+0000","last_clean":"2024-09-10T16:02:08.104261+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:02:08.104261+0000","last_undegraded":"2024-09-10T16:02:08.104261+0000","last_fullsized":"2024-09-10T16:02:08.104261+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'14150","reported_seq":14291,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:00.279797+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:05:00.279797+0000","last_peered":"2024-09-10T16:05:00.279797+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:05:00.279797+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"40'4100","ondisk_log_start":"40'4100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25833955,"num_objects":6308,"num_object_clones":0,"num_object_copies":12616,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6308,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11969,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":205},{"shards":"2","objects":2185}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":95,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:29.860108+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:02:29.860108+0000","last_peered":"2024-09-10T16:02:29.860108+0000","last_clean":"2024-09-10T16:02:29.860108+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:02:29.860108+0000","last_undegraded":"2024-09-10T16:02:29.860108+0000","last_fullsized":"2024-09-10T16:02:29.860108+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'14485","reported_seq":14616,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:59.108393+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:04:59.108393+0000","last_peered":"2024-09-10T16:04:59.108393+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:04:59.108393+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"43'4400","ondisk_log_start":"43'4400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26316800,"num_objects":6425,"num_object_clones":0,"num_object_copies":12850,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6425,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10989,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2170},{"shards":"0,2","objects":229}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":105,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:12.405385+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:02:12.405385+0000","last_peered":"2024-09-10T16:02:12.405385+0000","last_clean":"2024-09-10T16:02:12.405385+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:02:12.405385+0000","last_undegraded":"2024-09-10T16:02:12.405385+0000","last_fullsized":"2024-09-10T16:02:12.405385+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'14473","reported_seq":14609,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:00.286720+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:05:00.286720+0000","last_peered":"2024-09-10T16:05:00.286720+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:05:00.286720+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"43'4400","ondisk_log_start":"43'4400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25838051,"num_objects":6309,"num_object_clones":0,"num_object_copies":12618,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6309,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12153,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":224},{"shards":"2","objects":2005}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104617+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:02:08.104617+0000","last_peered":"2024-09-10T16:02:08.104617+0000","last_clean":"2024-09-10T16:02:08.104617+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:02:08.104617+0000","last_undegraded":"2024-09-10T16:02:08.104617+0000","last_fullsized":"2024-09-10T16:02:08.104617+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'14714","reported_seq":14844,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:00.303965+0000","last_change":"2024-09-10T15:59:16.174113+0000","last_active":"2024-09-10T16:05:00.303965+0000","last_peered":"2024-09-10T16:05:00.303965+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:05:00.303965+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"43'4700","ondisk_log_start":"43'4700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26443776,"num_objects":6456,"num_object_clones":0,"num_object_copies":12912,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4354,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6456,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12268,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":226},{"shards":"2","objects":2104}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'14469","reported_seq":14603,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:59.266846+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:04:59.266846+0000","last_peered":"2024-09-10T16:04:59.266846+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:04:59.266846+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"43'4400","ondisk_log_start":"43'4400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26087424,"num_objects":6369,"num_object_clones":0,"num_object_copies":12738,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6369,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11100,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2065},{"shards":"0,2","objects":255}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103316+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:02:08.103316+0000","last_peered":"2024-09-10T16:02:08.103316+0000","last_clean":"2024-09-10T16:02:08.103316+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:02:08.103316+0000","last_undegraded":"2024-09-10T16:02:08.103316+0000","last_fullsized":"2024-09-10T16:02:08.103316+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'14236","reported_seq":21557,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:59.250188+0000","last_change":"2024-09-10T16:03:29.605682+0000","last_active":"2024-09-10T16:04:59.250188+0000","last_peered":"2024-09-10T16:04:59.250188+0000","last_clean":"2024-09-10T15:59:14.115595+0000","last_became_active":"2024-09-10T15:59:16.244089+0000","last_became_peered":"2024-09-10T15:59:16.244089+0000","last_unstale":"2024-09-10T16:04:59.250188+0000","last_undegraded":"2024-09-10T16:03:29.603815+0000","last_fullsized":"2024-09-10T15:59:15.774437+0000","mapping_epoch":42,"log_start":"40'4200","ondisk_log_start":"40'4200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26058752,"num_objects":6362,"num_object_clones":0,"num_object_copies":12724,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":394,"num_objects_misplaced":224,"num_objects_unfound":0,"num_objects_dirty":6362,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10626,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3588,"num_bytes_recovered":14684160,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5744},{"shards":"0,2","objects":224}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":95,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:29.877831+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:02:29.877831+0000","last_peered":"2024-09-10T16:02:29.877831+0000","last_clean":"2024-09-10T16:02:29.877831+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:02:29.877831+0000","last_undegraded":"2024-09-10T16:02:29.877831+0000","last_fullsized":"2024-09-10T16:02:29.877831+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'14640","reported_seq":14781,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:59.203782+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:04:59.203782+0000","last_peered":"2024-09-10T16:04:59.203782+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:04:59.203782+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"43'4600","ondisk_log_start":"43'4600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26247168,"num_objects":6408,"num_object_clones":0,"num_object_copies":12816,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6408,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10895,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2037},{"shards":"0,2","objects":257}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104463+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:02:08.104463+0000","last_peered":"2024-09-10T16:02:08.104463+0000","last_clean":"2024-09-10T16:02:08.104463+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:02:08.104463+0000","last_undegraded":"2024-09-10T16:02:08.104463+0000","last_fullsized":"2024-09-10T16:02:08.104463+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104448+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:02:08.104448+0000","last_peered":"2024-09-10T16:02:08.104448+0000","last_clean":"2024-09-10T16:02:08.104448+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:02:08.104448+0000","last_undegraded":"2024-09-10T16:02:08.104448+0000","last_fullsized":"2024-09-10T16:02:08.104448+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'14671","reported_seq":14816,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:59.215735+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:04:59.215735+0000","last_peered":"2024-09-10T16:04:59.215735+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:04:59.215735+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"43'4600","ondisk_log_start":"43'4600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26735558,"num_objects":6529,"num_object_clones":0,"num_object_copies":13058,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6529,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10834,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2206},{"shards":"0,2","objects":257}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":419160822,"num_objects":102344,"num_object_clones":0,"num_object_copies":204688,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43305,"num_objects_misplaced":224,"num_objects_unfound":0,"num_objects_dirty":102344,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":189762,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7860,"num_bytes_recovered":32178176,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1038139392,"data_stored":1037990341,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160688,"ondisk_log_size":160688,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986260,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":670380,"kb_used_data":341596,"kb_used_omap":4,"kb_used_meta":328763,"kb_avail":93701460,"statfs":{"total":96636764160,"available":95950295040,"internally_reserved":0,"allocated":349794304,"data_stored":349502071,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":336653994},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,2],"upper_bound":32},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659206,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":595124,"kb_used_data":296864,"kb_used_omap":4,"kb_used_meta":298235,"kb_avail":93776716,"statfs":{"total":96636764160,"available":96027357184,"internally_reserved":0,"allocated":303988736,"data_stored":303659909,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":305393326},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,1,1,4,1,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182488,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":774632,"kb_used_data":377876,"kb_used_omap":4,"kb_used_meta":396731,"kb_avail":93597208,"statfs":{"total":96636764160,"available":95843540992,"internally_reserved":0,"allocated":386945024,"data_stored":386652786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":406253226},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,1,5,5,6],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":385343488,"data_stored":385307358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":303562752,"data_stored":303490033,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":349233152,"data_stored":349192950,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:05:05.527 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:05:06.103 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:05:06.104 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:05:06.129 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":273,"stamp":"2024-09-10T16:05:05.678164+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":410452745,"num_objects":100220,"num_object_clones":0,"num_object_copies":200440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43305,"num_objects_misplaced":66,"num_objects_unfound":0,"num_objects_dirty":100220,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":191890,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8020,"num_bytes_recovered":32825363,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160917,"ondisk_log_size":160917,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2042960,"kb_used_data":1005272,"kb_used_omap":12,"kb_used_meta":1037619,"kb_avail":281072560,"statfs":{"total":289910292480,"available":287818301440,"internally_reserved":0,"allocated":1029398528,"data_stored":1028485230,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1062521858},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,3,1,3,6,13,7],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-27414528,"num_objects":-6693,"num_object_clones":0,"num_object_copies":-13386,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-468,"num_objects_unfound":0,"num_objects_dirty":-6693,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6693,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":468,"num_bytes_recovered":1916928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006743"},"pg_stats":[{"pgid":"2.f","version":"45'14491","reported_seq":14567,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:04.283999+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:05:04.283999+0000","last_peered":"2024-09-10T16:05:04.283999+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:05:04.283999+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"43'4400","ondisk_log_start":"43'4400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25317376,"num_objects":6181,"num_object_clones":0,"num_object_copies":12362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6181,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14491,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1922},{"shards":"0,2","objects":103}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'14639","reported_seq":14752,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:05:05.297237+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:05:05.297237+0000","last_peered":"2024-09-10T16:05:05.297237+0000","last_clean":"2024-09-10T16:05:05.297237+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:05:05.297237+0000","last_undegraded":"2024-09-10T16:05:05.297237+0000","last_fullsized":"2024-09-10T16:05:05.297237+0000","mapping_epoch":35,"log_start":"43'4600","ondisk_log_start":"43'4600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25416646,"num_objects":6207,"num_object_clones":0,"num_object_copies":12414,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6207,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":12744,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'14522","reported_seq":23201,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:05:02.030442+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:05:02.030442+0000","last_peered":"2024-09-10T16:05:02.030442+0000","last_clean":"2024-09-10T16:05:02.030442+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:05:02.030442+0000","last_undegraded":"2024-09-10T16:05:02.030442+0000","last_fullsized":"2024-09-10T16:05:02.030442+0000","mapping_epoch":44,"log_start":"43'4500","ondisk_log_start":"43'4500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25702424,"num_objects":6276,"num_object_clones":0,"num_object_copies":12552,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6276,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12526,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'14488","reported_seq":14570,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:05:05.305474+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:05:05.305474+0000","last_peered":"2024-09-10T16:05:05.305474+0000","last_clean":"2024-09-10T16:05:05.305474+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:05:05.305474+0000","last_undegraded":"2024-09-10T16:05:05.305474+0000","last_fullsized":"2024-09-10T16:05:05.305474+0000","mapping_epoch":35,"log_start":"43'4400","ondisk_log_start":"43'4400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25280512,"num_objects":6172,"num_object_clones":0,"num_object_copies":12344,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6172,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14488,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'14650","reported_seq":14781,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:05:04.241757+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:05:04.241757+0000","last_peered":"2024-09-10T16:05:04.241757+0000","last_clean":"2024-09-10T16:05:04.241757+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:05:04.241757+0000","last_undegraded":"2024-09-10T16:05:04.241757+0000","last_fullsized":"2024-09-10T16:05:04.241757+0000","mapping_epoch":34,"log_start":"43'4600","ondisk_log_start":"43'4600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25563619,"num_objects":6242,"num_object_clones":0,"num_object_copies":12484,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6242,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11361,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'14798","reported_seq":14937,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:04.250631+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:05:04.250631+0000","last_peered":"2024-09-10T16:05:04.250631+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:05:04.250631+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"43'4700","ondisk_log_start":"43'4700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25653731,"num_objects":6264,"num_object_clones":0,"num_object_copies":12528,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6264,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11410,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1893},{"shards":"0,2","objects":106}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'14651","reported_seq":14790,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:04.282453+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:05:04.282453+0000","last_peered":"2024-09-10T16:05:04.282453+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:05:04.282453+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"43'4600","ondisk_log_start":"43'4600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26342342,"num_objects":6433,"num_object_clones":0,"num_object_copies":12866,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6433,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":11150,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2196},{"shards":"0,2","objects":130}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'14893","reported_seq":15024,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:05:04.200340+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:05:04.200340+0000","last_peered":"2024-09-10T16:05:04.200340+0000","last_clean":"2024-09-10T16:05:04.200340+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:05:04.200340+0000","last_undegraded":"2024-09-10T16:05:04.200340+0000","last_fullsized":"2024-09-10T16:05:04.200340+0000","mapping_epoch":34,"log_start":"43'4800","ondisk_log_start":"43'4800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26300416,"num_objects":6421,"num_object_clones":0,"num_object_copies":12842,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6421,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11740,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104261+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:02:08.104261+0000","last_peered":"2024-09-10T16:02:08.104261+0000","last_clean":"2024-09-10T16:02:08.104261+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:02:08.104261+0000","last_undegraded":"2024-09-10T16:02:08.104261+0000","last_fullsized":"2024-09-10T16:02:08.104261+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'14276","reported_seq":14412,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:05.133357+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:05:05.133357+0000","last_peered":"2024-09-10T16:05:05.133357+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:05:05.133357+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"43'4200","ondisk_log_start":"43'4200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25317859,"num_objects":6182,"num_object_clones":0,"num_object_copies":12364,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6182,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12095,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":79},{"shards":"2","objects":2059}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":95,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:29.860108+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:02:29.860108+0000","last_peered":"2024-09-10T16:02:29.860108+0000","last_clean":"2024-09-10T16:02:29.860108+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:02:29.860108+0000","last_undegraded":"2024-09-10T16:02:29.860108+0000","last_fullsized":"2024-09-10T16:02:29.860108+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'14611","reported_seq":14741,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:04.259164+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:05:04.259164+0000","last_peered":"2024-09-10T16:05:04.259164+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:05:04.259164+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"43'4600","ondisk_log_start":"43'4600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25800704,"num_objects":6299,"num_object_clones":0,"num_object_copies":12598,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6299,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11115,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2044},{"shards":"0,2","objects":103}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":105,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:12.405385+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:02:12.405385+0000","last_peered":"2024-09-10T16:02:12.405385+0000","last_clean":"2024-09-10T16:02:12.405385+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:02:12.405385+0000","last_undegraded":"2024-09-10T16:02:12.405385+0000","last_fullsized":"2024-09-10T16:02:12.405385+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'14612","reported_seq":14748,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:05.278101+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:05:05.278101+0000","last_peered":"2024-09-10T16:05:05.278101+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:05:05.278101+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"43'4600","ondisk_log_start":"43'4600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25268707,"num_objects":6170,"num_object_clones":0,"num_object_copies":12340,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6170,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12292,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":85},{"shards":"2","objects":1866}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104617+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:02:08.104617+0000","last_peered":"2024-09-10T16:02:08.104617+0000","last_clean":"2024-09-10T16:02:08.104617+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:02:08.104617+0000","last_undegraded":"2024-09-10T16:02:08.104617+0000","last_fullsized":"2024-09-10T16:02:08.104617+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'14866","reported_seq":14996,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:05.166236+0000","last_change":"2024-09-10T15:59:16.174113+0000","last_active":"2024-09-10T16:05:05.166236+0000","last_peered":"2024-09-10T16:05:05.166236+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:05:05.166236+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"43'4800","ondisk_log_start":"43'4800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25821184,"num_objects":6304,"num_object_clones":0,"num_object_copies":12608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4354,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6304,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12420,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":74},{"shards":"2","objects":1952}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'14611","reported_seq":14747,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:04.278349+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:05:04.278349+0000","last_peered":"2024-09-10T16:05:04.278349+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:05:04.278349+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"43'4600","ondisk_log_start":"43'4600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25505792,"num_objects":6227,"num_object_clones":0,"num_object_copies":12454,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6227,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11242,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1923},{"shards":"0,2","objects":113}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103316+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:02:08.103316+0000","last_peered":"2024-09-10T16:02:08.103316+0000","last_clean":"2024-09-10T16:02:08.103316+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:02:08.103316+0000","last_undegraded":"2024-09-10T16:02:08.103316+0000","last_fullsized":"2024-09-10T16:02:08.103316+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'14400","reported_seq":22036,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:04.289133+0000","last_change":"2024-09-10T16:03:29.605682+0000","last_active":"2024-09-10T16:05:04.289133+0000","last_peered":"2024-09-10T16:05:04.289133+0000","last_clean":"2024-09-10T15:59:14.115595+0000","last_became_active":"2024-09-10T15:59:16.244089+0000","last_became_peered":"2024-09-10T15:59:16.244089+0000","last_unstale":"2024-09-10T16:05:04.289133+0000","last_undegraded":"2024-09-10T16:03:29.603815+0000","last_fullsized":"2024-09-10T15:59:15.774437+0000","mapping_epoch":42,"log_start":"43'4300","ondisk_log_start":"43'4300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25387008,"num_objects":6198,"num_object_clones":0,"num_object_copies":12396,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":394,"num_objects_misplaced":66,"num_objects_unfound":0,"num_objects_dirty":6198,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10790,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3746,"num_bytes_recovered":15331328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5738},{"shards":"0,2","objects":66}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":95,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:29.877831+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:02:29.877831+0000","last_peered":"2024-09-10T16:02:29.877831+0000","last_clean":"2024-09-10T16:02:29.877831+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:02:29.877831+0000","last_undegraded":"2024-09-10T16:02:29.877831+0000","last_fullsized":"2024-09-10T16:02:29.877831+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'14789","reported_seq":14932,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:04.129052+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:05:04.129052+0000","last_peered":"2024-09-10T16:05:04.129052+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:05:04.129052+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"43'4700","ondisk_log_start":"43'4700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25636864,"num_objects":6259,"num_object_clones":0,"num_object_copies":12518,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6259,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11044,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1888},{"shards":"0,2","objects":108}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104463+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:02:08.104463+0000","last_peered":"2024-09-10T16:02:08.104463+0000","last_clean":"2024-09-10T16:02:08.104463+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:02:08.104463+0000","last_undegraded":"2024-09-10T16:02:08.104463+0000","last_fullsized":"2024-09-10T16:02:08.104463+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104448+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:02:08.104448+0000","last_peered":"2024-09-10T16:02:08.104448+0000","last_clean":"2024-09-10T16:02:08.104448+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:02:08.104448+0000","last_undegraded":"2024-09-10T16:02:08.104448+0000","last_fullsized":"2024-09-10T16:02:08.104448+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'14817","reported_seq":14960,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:04.268846+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:05:04.268846+0000","last_peered":"2024-09-10T16:05:04.268846+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:05:04.268846+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"43'4800","ondisk_log_start":"43'4800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26137542,"num_objects":6383,"num_object_clones":0,"num_object_copies":12766,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6383,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10980,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2060},{"shards":"0,2","objects":111}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":410452726,"num_objects":100218,"num_object_clones":0,"num_object_copies":200436,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43305,"num_objects_misplaced":66,"num_objects_unfound":0,"num_objects_dirty":100218,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":191888,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8018,"num_bytes_recovered":32825344,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1026691072,"data_stored":1026542021,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160914,"ondisk_log_size":160914,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986261,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":653528,"kb_used_data":337224,"kb_used_omap":4,"kb_used_meta":316283,"kb_avail":93718312,"statfs":{"total":96636764160,"available":95967551488,"internally_reserved":0,"allocated":345317376,"data_stored":345025143,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":323874474},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,0,2,2,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659206,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":595124,"kb_used_data":296864,"kb_used_omap":4,"kb_used_meta":298235,"kb_avail":93776716,"statfs":{"total":96636764160,"available":96027357184,"internally_reserved":0,"allocated":303988736,"data_stored":303659909,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":305393326},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,1,1,4,1,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182489,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":794308,"kb_used_data":371184,"kb_used_omap":4,"kb_used_meta":423099,"kb_avail":93577532,"statfs":{"total":96636764160,"available":95823392768,"internally_reserved":0,"allocated":380092416,"data_stored":379800178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":433254058},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,11,6],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":378376192,"data_stored":378340062,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":303562752,"data_stored":303490033,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":344752128,"data_stored":344711926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:05:06.130 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-10T16:05:06.130 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "45'14491", 'reported_seq': 14567, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:05:04.283999+0000', 'last_change': '2024-09-10T15:59:16.281558+0000', 'last_active': '2024-09-10T16:05:04.283999+0000', 'last_peered': '2024-09-10T16:05:04.283999+0000', 'last_clean': '2024-09-10T15:59:14.132456+0000', 'last_became_active': '2024-09-10T15:59:16.274407+0000', 'last_became_peered': '2024-09-10T15:59:16.274407+0000', 'last_unstale': '2024-09-10T16:05:04.283999+0000', 'last_undegraded': '2024-09-10T15:59:15.851279+0000', 'last_fullsized': '2024-09-10T15:59:15.774356+0000', 'mapping_epoch': 42, 'log_start': "43'4400", 'ondisk_log_start': "43'4400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'objects_scrubbed': 0, 'log_size': 10091, 'log_dups_size': 0, 'ondisk_log_size': 10091, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25317376, 'num_objects': 6181, 'num_object_clones': 0, 'num_object_copies': 12362, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4259, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6181, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14491, 'num_write_kb': 41344, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1922}, {'shards': '0,2', 'objects': 103}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:05:06.131 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-10T16:05:06.131 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "45'14798", 'reported_seq': 14937, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:05:04.250631+0000', 'last_change': '2024-09-10T15:59:16.308264+0000', 'last_active': '2024-09-10T16:05:04.250631+0000', 'last_peered': '2024-09-10T16:05:04.250631+0000', 'last_clean': '2024-09-10T15:59:14.111551+0000', 'last_became_active': '2024-09-10T15:59:16.300040+0000', 'last_became_peered': '2024-09-10T15:59:16.300040+0000', 'last_unstale': '2024-09-10T16:05:04.250631+0000', 'last_undegraded': '2024-09-10T15:59:15.878234+0000', 'last_fullsized': '2024-09-10T15:59:15.769328+0000', 'mapping_epoch': 42, 'log_start': "43'4700", 'ondisk_log_start': "43'4700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3442", 'last_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'objects_scrubbed': 3384, 'log_size': 10098, 'log_dups_size': 0, 'ondisk_log_size': 10098, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1292, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25653731, 'num_objects': 6264, 'num_object_clones': 0, 'num_object_copies': 12528, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4372, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6264, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 11410, 'num_write_kb': 28580, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1893}, {'shards': '0,2', 'objects': 106}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:05:06.131 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-10T16:05:06.131 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "45'14651", 'reported_seq': 14790, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:05:04.282453+0000', 'last_change': '2024-09-10T15:59:16.331899+0000', 'last_active': '2024-09-10T16:05:04.282453+0000', 'last_peered': '2024-09-10T16:05:04.282453+0000', 'last_clean': '2024-09-10T15:59:14.139436+0000', 'last_became_active': '2024-09-10T15:59:16.318715+0000', 'last_became_peered': '2024-09-10T15:59:16.318715+0000', 'last_unstale': '2024-09-10T16:05:04.282453+0000', 'last_undegraded': '2024-09-10T15:59:15.877524+0000', 'last_fullsized': '2024-09-10T15:59:15.769042+0000', 'mapping_epoch': 42, 'log_start': "43'4600", 'ondisk_log_start': "43'4600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3537", 'last_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'objects_scrubbed': 3497, 'log_size': 10051, 'log_dups_size': 0, 'ondisk_log_size': 10051, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1224, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26342342, 'num_objects': 6433, 'num_object_clones': 0, 'num_object_copies': 12866, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4237, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6433, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 11150, 'num_write_kb': 28172, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2196}, {'shards': '0,2', 'objects': 130}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:05:06.131 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-10T16:05:06.131 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "45'14276", 'reported_seq': 14412, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:05:05.133357+0000', 'last_change': '2024-09-10T15:59:16.175680+0000', 'last_active': '2024-09-10T16:05:05.133357+0000', 'last_peered': '2024-09-10T16:05:05.133357+0000', 'last_clean': '2024-09-10T15:59:14.088187+0000', 'last_became_active': '2024-09-10T15:59:16.165810+0000', 'last_became_peered': '2024-09-10T15:59:16.165810+0000', 'last_unstale': '2024-09-10T16:05:05.133357+0000', 'last_undegraded': '2024-09-10T15:59:15.822098+0000', 'last_fullsized': '2024-09-10T15:59:15.769123+0000', 'mapping_epoch': 42, 'log_start': "43'4200", 'ondisk_log_start': "43'4200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2218", 'last_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'objects_scrubbed': 2175, 'log_size': 10076, 'log_dups_size': 0, 'ondisk_log_size': 10076, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 825, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25317859, 'num_objects': 6182, 'num_object_clones': 0, 'num_object_copies': 12364, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4124, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6182, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 12095, 'num_write_kb': 32204, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 79}, {'shards': '2', 'objects': 2059}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:05:06.131 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-10T16:05:06.131 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "45'14611", 'reported_seq': 14741, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:05:04.259164+0000', 'last_change': '2024-09-10T15:59:16.277742+0000', 'last_active': '2024-09-10T16:05:04.259164+0000', 'last_peered': '2024-09-10T16:05:04.259164+0000', 'last_clean': '2024-09-10T15:59:14.129081+0000', 'last_became_active': '2024-09-10T15:59:16.269835+0000', 'last_became_peered': '2024-09-10T15:59:16.269835+0000', 'last_unstale': '2024-09-10T16:05:04.259164+0000', 'last_undegraded': '2024-09-10T15:59:15.860857+0000', 'last_fullsized': '2024-09-10T15:59:15.771410+0000', 'mapping_epoch': 42, 'log_start': "43'4600", 'ondisk_log_start': "43'4600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3557", 'last_scrub_stamp': '2024-09-10T15:59:02.493678+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:02.493678+0000', 'objects_scrubbed': 3488, 'log_size': 10011, 'log_dups_size': 0, 'ondisk_log_size': 10011, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1472, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25800704, 'num_objects': 6299, 'num_object_clones': 0, 'num_object_copies': 12598, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4257, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6299, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11115, 'num_write_kb': 27852, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2044}, {'shards': '0,2', 'objects': 103}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:05:06.131 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-10T16:05:06.131 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "45'14612", 'reported_seq': 14748, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:05:05.278101+0000', 'last_change': '2024-09-10T15:59:16.284311+0000', 'last_active': '2024-09-10T16:05:05.278101+0000', 'last_peered': '2024-09-10T16:05:05.278101+0000', 'last_clean': '2024-09-10T15:59:14.116041+0000', 'last_became_active': '2024-09-10T15:59:16.274543+0000', 'last_became_peered': '2024-09-10T15:59:16.274543+0000', 'last_unstale': '2024-09-10T16:05:05.278101+0000', 'last_undegraded': '2024-09-10T15:59:15.812484+0000', 'last_fullsized': '2024-09-10T15:59:15.770885+0000', 'mapping_epoch': 42, 'log_start': "43'4600", 'ondisk_log_start': "43'4600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2356", 'last_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'objects_scrubbed': 2316, 'log_size': 10012, 'log_dups_size': 0, 'ondisk_log_size': 10012, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 783, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25268707, 'num_objects': 6170, 'num_object_clones': 0, 'num_object_copies': 12340, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6170, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 12292, 'num_write_kb': 32292, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 85}, {'shards': '2', 'objects': 1866}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:05:06.131 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-10T16:05:06.132 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "45'14866", 'reported_seq': 14996, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:05:05.166236+0000', 'last_change': '2024-09-10T15:59:16.174113+0000', 'last_active': '2024-09-10T16:05:05.166236+0000', 'last_peered': '2024-09-10T16:05:05.166236+0000', 'last_clean': '2024-09-10T15:59:14.110654+0000', 'last_became_active': '2024-09-10T15:59:16.164645+0000', 'last_became_peered': '2024-09-10T15:59:16.164645+0000', 'last_unstale': '2024-09-10T16:05:05.166236+0000', 'last_undegraded': '2024-09-10T15:59:15.803593+0000', 'last_fullsized': '2024-09-10T15:59:15.769083+0000', 'mapping_epoch': 42, 'log_start': "43'4800", 'ondisk_log_start': "43'4800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2486", 'last_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'objects_scrubbed': 2440, 'log_size': 10066, 'log_dups_size': 0, 'ondisk_log_size': 10066, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 792, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25821184, 'num_objects': 6304, 'num_object_clones': 0, 'num_object_copies': 12608, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4354, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6304, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12420, 'num_write_kb': 32568, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 74}, {'shards': '2', 'objects': 1952}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:05:06.132 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-10T16:05:06.132 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "45'14611", 'reported_seq': 14747, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:05:04.278349+0000', 'last_change': '2024-09-10T15:59:16.315641+0000', 'last_active': '2024-09-10T16:05:04.278349+0000', 'last_peered': '2024-09-10T16:05:04.278349+0000', 'last_clean': '2024-09-10T15:59:14.117001+0000', 'last_became_active': '2024-09-10T15:59:16.302245+0000', 'last_became_peered': '2024-09-10T15:59:16.302245+0000', 'last_unstale': '2024-09-10T16:05:04.278349+0000', 'last_undegraded': '2024-09-10T15:59:15.845012+0000', 'last_fullsized': '2024-09-10T15:59:15.770337+0000', 'mapping_epoch': 42, 'log_start': "43'4600", 'ondisk_log_start': "43'4600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3404", 'last_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'objects_scrubbed': 3367, 'log_size': 10011, 'log_dups_size': 0, 'ondisk_log_size': 10011, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 983, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25505792, 'num_objects': 6227, 'num_object_clones': 0, 'num_object_copies': 12454, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6227, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11242, 'num_write_kb': 28204, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1923}, {'shards': '0,2', 'objects': 113}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:05:06.132 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-10T16:05:06.132 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "45'14400", 'reported_seq': 22036, 'reported_epoch': 45, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:05:04.289133+0000', 'last_change': '2024-09-10T16:03:29.605682+0000', 'last_active': '2024-09-10T16:05:04.289133+0000', 'last_peered': '2024-09-10T16:05:04.289133+0000', 'last_clean': '2024-09-10T15:59:14.115595+0000', 'last_became_active': '2024-09-10T15:59:16.244089+0000', 'last_became_peered': '2024-09-10T15:59:16.244089+0000', 'last_unstale': '2024-09-10T16:05:04.289133+0000', 'last_undegraded': '2024-09-10T16:03:29.603815+0000', 'last_fullsized': '2024-09-10T15:59:15.774437+0000', 'mapping_epoch': 42, 'log_start': "43'4300", 'ondisk_log_start': "43'4300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3653", 'last_scrub_stamp': '2024-09-10T15:59:05.298468+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:05.298468+0000', 'objects_scrubbed': 3602, 'log_size': 10100, 'log_dups_size': 0, 'ondisk_log_size': 10100, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1289, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25387008, 'num_objects': 6198, 'num_object_clones': 0, 'num_object_copies': 12396, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 394, 'num_objects_misplaced': 66, 'num_objects_unfound': 0, 'num_objects_dirty': 6198, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10790, 'num_write_kb': 26772, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3746, 'num_bytes_recovered': 15331328, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5738}, {'shards': '0,2', 'objects': 66}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:05:06.132 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-10T16:05:06.132 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "45'14789", 'reported_seq': 14932, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:05:04.129052+0000', 'last_change': '2024-09-10T15:59:16.311291+0000', 'last_active': '2024-09-10T16:05:04.129052+0000', 'last_peered': '2024-09-10T16:05:04.129052+0000', 'last_clean': '2024-09-10T15:59:14.116363+0000', 'last_became_active': '2024-09-10T15:59:16.301244+0000', 'last_became_peered': '2024-09-10T15:59:16.301244+0000', 'last_unstale': '2024-09-10T16:05:04.129052+0000', 'last_undegraded': '2024-09-10T15:59:15.883434+0000', 'last_fullsized': '2024-09-10T15:59:15.772756+0000', 'mapping_epoch': 42, 'log_start': "43'4700", 'ondisk_log_start': "43'4700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3789", 'last_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'objects_scrubbed': 3741, 'log_size': 10089, 'log_dups_size': 0, 'ondisk_log_size': 10089, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1440, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25636864, 'num_objects': 6259, 'num_object_clones': 0, 'num_object_copies': 12518, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4373, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6259, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11044, 'num_write_kb': 27124, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1888}, {'shards': '0,2', 'objects': 108}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:05:06.132 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-10T16:05:06.132 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "45'14817", 'reported_seq': 14960, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:05:04.268846+0000', 'last_change': '2024-09-10T15:59:16.313409+0000', 'last_active': '2024-09-10T16:05:04.268846+0000', 'last_peered': '2024-09-10T16:05:04.268846+0000', 'last_clean': '2024-09-10T15:59:14.129780+0000', 'last_became_active': '2024-09-10T15:59:16.300932+0000', 'last_became_peered': '2024-09-10T15:59:16.300932+0000', 'last_unstale': '2024-09-10T16:05:04.268846+0000', 'last_undegraded': '2024-09-10T15:59:15.887823+0000', 'last_fullsized': '2024-09-10T15:59:15.768513+0000', 'mapping_epoch': 42, 'log_start': "43'4800", 'ondisk_log_start': "43'4800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3880", 'last_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'objects_scrubbed': 3827, 'log_size': 10017, 'log_dups_size': 0, 'ondisk_log_size': 10017, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1397, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26137542, 'num_objects': 6383, 'num_object_clones': 0, 'num_object_copies': 12766, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4325, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6383, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10980, 'num_write_kb': 27072, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2060}, {'shards': '0,2', 'objects': 111}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:05:06.133 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T16:05:06.807 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:05:06.808 INFO:teuthology.orchestra.run.smithi115.stdout:{"fsid":"7effee29-df05-4351-9741-ee203564ed81","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 43305/200440 objects degraded (21.605%), 11 pgs degraded, 11 pgs undersized","count":22},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":470,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983954,"num_in_osds":3,"osd_in_since":1725983947,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":100220,"data_bytes":410452745,"bytes_used":2091991040,"bytes_avail":287818301440,"bytes_total":289910292480,"degraded_objects":43305,"degraded_total":200440,"degraded_ratio":0.21604969068050286,"misplaced_objects":66,"misplaced_total":200440,"misplaced_ratio":0.00032927559369387349,"recovering_objects_per_sec":38,"recovering_bytes_per_sec":159654,"recovering_keys_per_sec":0,"num_objects_recovered":468,"num_bytes_recovered":1916928,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":557},"fsmap":{"epoch":1,"btime":"2024-09-10T15:57:14:967120+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T16:00:13.560776+0000","services":{}},"progress_events":{}} 2024-09-10T16:05:06.836 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-10T16:05:06.837 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:05:07.429 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:05:07.431 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:05:07.452 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":273,"stamp":"2024-09-10T16:05:05.678164+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":410452745,"num_objects":100220,"num_object_clones":0,"num_object_copies":200440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43305,"num_objects_misplaced":66,"num_objects_unfound":0,"num_objects_dirty":100220,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":191890,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8020,"num_bytes_recovered":32825363,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160917,"ondisk_log_size":160917,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2042960,"kb_used_data":1005272,"kb_used_omap":12,"kb_used_meta":1037619,"kb_avail":281072560,"statfs":{"total":289910292480,"available":287818301440,"internally_reserved":0,"allocated":1029398528,"data_stored":1028485230,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1062521858},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,3,1,3,6,13,7],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-27414528,"num_objects":-6693,"num_object_clones":0,"num_object_copies":-13386,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-468,"num_objects_unfound":0,"num_objects_dirty":-6693,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6693,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":468,"num_bytes_recovered":1916928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006743"},"pg_stats":[{"pgid":"2.f","version":"45'14491","reported_seq":14567,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:04.283999+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:05:04.283999+0000","last_peered":"2024-09-10T16:05:04.283999+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:05:04.283999+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"43'4400","ondisk_log_start":"43'4400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25317376,"num_objects":6181,"num_object_clones":0,"num_object_copies":12362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6181,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14491,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1922},{"shards":"0,2","objects":103}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'14639","reported_seq":14752,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:05:05.297237+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:05:05.297237+0000","last_peered":"2024-09-10T16:05:05.297237+0000","last_clean":"2024-09-10T16:05:05.297237+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:05:05.297237+0000","last_undegraded":"2024-09-10T16:05:05.297237+0000","last_fullsized":"2024-09-10T16:05:05.297237+0000","mapping_epoch":35,"log_start":"43'4600","ondisk_log_start":"43'4600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25416646,"num_objects":6207,"num_object_clones":0,"num_object_copies":12414,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6207,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":12744,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'14522","reported_seq":23201,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:05:02.030442+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:05:02.030442+0000","last_peered":"2024-09-10T16:05:02.030442+0000","last_clean":"2024-09-10T16:05:02.030442+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:05:02.030442+0000","last_undegraded":"2024-09-10T16:05:02.030442+0000","last_fullsized":"2024-09-10T16:05:02.030442+0000","mapping_epoch":44,"log_start":"43'4500","ondisk_log_start":"43'4500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25702424,"num_objects":6276,"num_object_clones":0,"num_object_copies":12552,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6276,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12526,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'14488","reported_seq":14570,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:05:05.305474+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:05:05.305474+0000","last_peered":"2024-09-10T16:05:05.305474+0000","last_clean":"2024-09-10T16:05:05.305474+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:05:05.305474+0000","last_undegraded":"2024-09-10T16:05:05.305474+0000","last_fullsized":"2024-09-10T16:05:05.305474+0000","mapping_epoch":35,"log_start":"43'4400","ondisk_log_start":"43'4400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25280512,"num_objects":6172,"num_object_clones":0,"num_object_copies":12344,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6172,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14488,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'14650","reported_seq":14781,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:05:04.241757+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:05:04.241757+0000","last_peered":"2024-09-10T16:05:04.241757+0000","last_clean":"2024-09-10T16:05:04.241757+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:05:04.241757+0000","last_undegraded":"2024-09-10T16:05:04.241757+0000","last_fullsized":"2024-09-10T16:05:04.241757+0000","mapping_epoch":34,"log_start":"43'4600","ondisk_log_start":"43'4600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25563619,"num_objects":6242,"num_object_clones":0,"num_object_copies":12484,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6242,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11361,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'14798","reported_seq":14937,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:04.250631+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:05:04.250631+0000","last_peered":"2024-09-10T16:05:04.250631+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:05:04.250631+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"43'4700","ondisk_log_start":"43'4700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25653731,"num_objects":6264,"num_object_clones":0,"num_object_copies":12528,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6264,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11410,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1893},{"shards":"0,2","objects":106}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'14651","reported_seq":14790,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:04.282453+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:05:04.282453+0000","last_peered":"2024-09-10T16:05:04.282453+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:05:04.282453+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"43'4600","ondisk_log_start":"43'4600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26342342,"num_objects":6433,"num_object_clones":0,"num_object_copies":12866,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6433,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":11150,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2196},{"shards":"0,2","objects":130}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'14893","reported_seq":15024,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:05:04.200340+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:05:04.200340+0000","last_peered":"2024-09-10T16:05:04.200340+0000","last_clean":"2024-09-10T16:05:04.200340+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:05:04.200340+0000","last_undegraded":"2024-09-10T16:05:04.200340+0000","last_fullsized":"2024-09-10T16:05:04.200340+0000","mapping_epoch":34,"log_start":"43'4800","ondisk_log_start":"43'4800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26300416,"num_objects":6421,"num_object_clones":0,"num_object_copies":12842,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6421,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11740,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104261+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:02:08.104261+0000","last_peered":"2024-09-10T16:02:08.104261+0000","last_clean":"2024-09-10T16:02:08.104261+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:02:08.104261+0000","last_undegraded":"2024-09-10T16:02:08.104261+0000","last_fullsized":"2024-09-10T16:02:08.104261+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'14276","reported_seq":14412,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:05.133357+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:05:05.133357+0000","last_peered":"2024-09-10T16:05:05.133357+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:05:05.133357+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"43'4200","ondisk_log_start":"43'4200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25317859,"num_objects":6182,"num_object_clones":0,"num_object_copies":12364,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6182,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12095,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":79},{"shards":"2","objects":2059}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":95,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:29.860108+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:02:29.860108+0000","last_peered":"2024-09-10T16:02:29.860108+0000","last_clean":"2024-09-10T16:02:29.860108+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:02:29.860108+0000","last_undegraded":"2024-09-10T16:02:29.860108+0000","last_fullsized":"2024-09-10T16:02:29.860108+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'14611","reported_seq":14741,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:04.259164+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:05:04.259164+0000","last_peered":"2024-09-10T16:05:04.259164+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:05:04.259164+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"43'4600","ondisk_log_start":"43'4600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25800704,"num_objects":6299,"num_object_clones":0,"num_object_copies":12598,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6299,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11115,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2044},{"shards":"0,2","objects":103}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":105,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:12.405385+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:02:12.405385+0000","last_peered":"2024-09-10T16:02:12.405385+0000","last_clean":"2024-09-10T16:02:12.405385+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:02:12.405385+0000","last_undegraded":"2024-09-10T16:02:12.405385+0000","last_fullsized":"2024-09-10T16:02:12.405385+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'14612","reported_seq":14748,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:05.278101+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:05:05.278101+0000","last_peered":"2024-09-10T16:05:05.278101+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:05:05.278101+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"43'4600","ondisk_log_start":"43'4600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25268707,"num_objects":6170,"num_object_clones":0,"num_object_copies":12340,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6170,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12292,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":85},{"shards":"2","objects":1866}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104617+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:02:08.104617+0000","last_peered":"2024-09-10T16:02:08.104617+0000","last_clean":"2024-09-10T16:02:08.104617+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:02:08.104617+0000","last_undegraded":"2024-09-10T16:02:08.104617+0000","last_fullsized":"2024-09-10T16:02:08.104617+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'14866","reported_seq":14996,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:05.166236+0000","last_change":"2024-09-10T15:59:16.174113+0000","last_active":"2024-09-10T16:05:05.166236+0000","last_peered":"2024-09-10T16:05:05.166236+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:05:05.166236+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"43'4800","ondisk_log_start":"43'4800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25821184,"num_objects":6304,"num_object_clones":0,"num_object_copies":12608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4354,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6304,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12420,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":74},{"shards":"2","objects":1952}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'14611","reported_seq":14747,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:04.278349+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:05:04.278349+0000","last_peered":"2024-09-10T16:05:04.278349+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:05:04.278349+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"43'4600","ondisk_log_start":"43'4600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25505792,"num_objects":6227,"num_object_clones":0,"num_object_copies":12454,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6227,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11242,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1923},{"shards":"0,2","objects":113}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103316+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:02:08.103316+0000","last_peered":"2024-09-10T16:02:08.103316+0000","last_clean":"2024-09-10T16:02:08.103316+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:02:08.103316+0000","last_undegraded":"2024-09-10T16:02:08.103316+0000","last_fullsized":"2024-09-10T16:02:08.103316+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'14400","reported_seq":22036,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:04.289133+0000","last_change":"2024-09-10T16:03:29.605682+0000","last_active":"2024-09-10T16:05:04.289133+0000","last_peered":"2024-09-10T16:05:04.289133+0000","last_clean":"2024-09-10T15:59:14.115595+0000","last_became_active":"2024-09-10T15:59:16.244089+0000","last_became_peered":"2024-09-10T15:59:16.244089+0000","last_unstale":"2024-09-10T16:05:04.289133+0000","last_undegraded":"2024-09-10T16:03:29.603815+0000","last_fullsized":"2024-09-10T15:59:15.774437+0000","mapping_epoch":42,"log_start":"43'4300","ondisk_log_start":"43'4300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25387008,"num_objects":6198,"num_object_clones":0,"num_object_copies":12396,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":394,"num_objects_misplaced":66,"num_objects_unfound":0,"num_objects_dirty":6198,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10790,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3746,"num_bytes_recovered":15331328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5738},{"shards":"0,2","objects":66}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":95,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:29.877831+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:02:29.877831+0000","last_peered":"2024-09-10T16:02:29.877831+0000","last_clean":"2024-09-10T16:02:29.877831+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:02:29.877831+0000","last_undegraded":"2024-09-10T16:02:29.877831+0000","last_fullsized":"2024-09-10T16:02:29.877831+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'14789","reported_seq":14932,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:04.129052+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:05:04.129052+0000","last_peered":"2024-09-10T16:05:04.129052+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:05:04.129052+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"43'4700","ondisk_log_start":"43'4700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25636864,"num_objects":6259,"num_object_clones":0,"num_object_copies":12518,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6259,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11044,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1888},{"shards":"0,2","objects":108}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104463+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:02:08.104463+0000","last_peered":"2024-09-10T16:02:08.104463+0000","last_clean":"2024-09-10T16:02:08.104463+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:02:08.104463+0000","last_undegraded":"2024-09-10T16:02:08.104463+0000","last_fullsized":"2024-09-10T16:02:08.104463+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104448+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:02:08.104448+0000","last_peered":"2024-09-10T16:02:08.104448+0000","last_clean":"2024-09-10T16:02:08.104448+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:02:08.104448+0000","last_undegraded":"2024-09-10T16:02:08.104448+0000","last_fullsized":"2024-09-10T16:02:08.104448+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'14817","reported_seq":14960,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:04.268846+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:05:04.268846+0000","last_peered":"2024-09-10T16:05:04.268846+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:05:04.268846+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"43'4800","ondisk_log_start":"43'4800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26137542,"num_objects":6383,"num_object_clones":0,"num_object_copies":12766,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6383,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10980,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2060},{"shards":"0,2","objects":111}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":410452726,"num_objects":100218,"num_object_clones":0,"num_object_copies":200436,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43305,"num_objects_misplaced":66,"num_objects_unfound":0,"num_objects_dirty":100218,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":191888,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8018,"num_bytes_recovered":32825344,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1026691072,"data_stored":1026542021,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160914,"ondisk_log_size":160914,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986261,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":653528,"kb_used_data":337224,"kb_used_omap":4,"kb_used_meta":316283,"kb_avail":93718312,"statfs":{"total":96636764160,"available":95967551488,"internally_reserved":0,"allocated":345317376,"data_stored":345025143,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":323874474},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,0,2,2,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659206,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":595124,"kb_used_data":296864,"kb_used_omap":4,"kb_used_meta":298235,"kb_avail":93776716,"statfs":{"total":96636764160,"available":96027357184,"internally_reserved":0,"allocated":303988736,"data_stored":303659909,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":305393326},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,1,1,4,1,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182489,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":794308,"kb_used_data":371184,"kb_used_omap":4,"kb_used_meta":423099,"kb_avail":93577532,"statfs":{"total":96636764160,"available":95823392768,"internally_reserved":0,"allocated":380092416,"data_stored":379800178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":433254058},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,11,6],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":378376192,"data_stored":378340062,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":303562752,"data_stored":303490033,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":344752128,"data_stored":344711926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:05:10.454 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:05:11.006 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:05:11.009 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:05:11.033 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":275,"stamp":"2024-09-10T16:05:09.679707+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":404079369,"num_objects":98664,"num_object_clones":0,"num_object_copies":197328,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43214,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98664,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":193446,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8177,"num_bytes_recovered":33095699,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160773,"ondisk_log_size":160773,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2031452,"kb_used_data":997932,"kb_used_omap":12,"kb_used_meta":1033459,"kb_avail":281084068,"statfs":{"total":289910292480,"available":287830085632,"internally_reserved":0,"allocated":1021882368,"data_stored":1020969070,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1058262018},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,0,4,8,8,10,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-24502272,"num_objects":-5982,"num_object_clones":0,"num_object_copies":-11964,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-91,"num_objects_misplaced":-376,"num_objects_unfound":0,"num_objects_dirty":-5982,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5982,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":467,"num_bytes_recovered":1540096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006858"},"pg_stats":[{"pgid":"2.f","version":"45'14628","reported_seq":14704,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:09.253590+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:05:09.253590+0000","last_peered":"2024-09-10T16:05:09.253590+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:05:09.253590+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"43'4600","ondisk_log_start":"43'4600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24756224,"num_objects":6044,"num_object_clones":0,"num_object_copies":12088,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6044,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14628,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1785}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'14639","reported_seq":14752,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:05:05.297237+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:05:05.297237+0000","last_peered":"2024-09-10T16:05:05.297237+0000","last_clean":"2024-09-10T16:05:05.297237+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:05:05.297237+0000","last_undegraded":"2024-09-10T16:05:05.297237+0000","last_fullsized":"2024-09-10T16:05:05.297237+0000","mapping_epoch":35,"log_start":"43'4600","ondisk_log_start":"43'4600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25416646,"num_objects":6207,"num_object_clones":0,"num_object_copies":12414,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6207,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":12744,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'14638","reported_seq":23317,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:05:07.126726+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:05:07.126726+0000","last_peered":"2024-09-10T16:05:07.126726+0000","last_clean":"2024-09-10T16:05:07.126726+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:05:07.126726+0000","last_undegraded":"2024-09-10T16:05:07.126726+0000","last_fullsized":"2024-09-10T16:05:07.126726+0000","mapping_epoch":44,"log_start":"43'4600","ondisk_log_start":"43'4600","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25227288,"num_objects":6160,"num_object_clones":0,"num_object_copies":12320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6160,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12642,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'14488","reported_seq":14570,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:05:05.305474+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:05:05.305474+0000","last_peered":"2024-09-10T16:05:05.305474+0000","last_clean":"2024-09-10T16:05:05.305474+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:05:05.305474+0000","last_undegraded":"2024-09-10T16:05:05.305474+0000","last_fullsized":"2024-09-10T16:05:05.305474+0000","mapping_epoch":35,"log_start":"43'4400","ondisk_log_start":"43'4400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25280512,"num_objects":6172,"num_object_clones":0,"num_object_copies":12344,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6172,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14488,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'14776","reported_seq":14907,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.212879+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:05:09.212879+0000","last_peered":"2024-09-10T16:05:09.212879+0000","last_clean":"2024-09-10T16:05:09.212879+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:05:09.212879+0000","last_undegraded":"2024-09-10T16:05:09.212879+0000","last_fullsized":"2024-09-10T16:05:09.212879+0000","mapping_epoch":34,"log_start":"43'4700","ondisk_log_start":"43'4700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25047523,"num_objects":6116,"num_object_clones":0,"num_object_copies":12232,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6116,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11487,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'14942","reported_seq":15081,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:09.310517+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:05:09.310517+0000","last_peered":"2024-09-10T16:05:09.310517+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:05:09.310517+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"43'4900","ondisk_log_start":"43'4900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25063907,"num_objects":6120,"num_object_clones":0,"num_object_copies":12240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6120,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11554,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1749},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'14809","reported_seq":14948,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:09.268037+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:05:09.268037+0000","last_peered":"2024-09-10T16:05:09.268037+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:05:09.268037+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"43'4800","ondisk_log_start":"43'4800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25695174,"num_objects":6275,"num_object_clones":0,"num_object_copies":12550,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6275,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":11308,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2038},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'15045","reported_seq":15174,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.284726+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:05:09.284726+0000","last_peered":"2024-09-10T16:05:09.284726+0000","last_clean":"2024-09-10T16:05:09.284726+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:05:09.284726+0000","last_undegraded":"2024-09-10T16:05:09.284726+0000","last_fullsized":"2024-09-10T16:05:09.284726+0000","mapping_epoch":34,"log_start":"43'5000","ondisk_log_start":"43'5000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25677824,"num_objects":6269,"num_object_clones":0,"num_object_copies":12538,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6269,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11892,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104261+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:02:08.104261+0000","last_peered":"2024-09-10T16:02:08.104261+0000","last_clean":"2024-09-10T16:02:08.104261+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:02:08.104261+0000","last_undegraded":"2024-09-10T16:02:08.104261+0000","last_fullsized":"2024-09-10T16:02:08.104261+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'14276","reported_seq":14412,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:05.133357+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:05:05.133357+0000","last_peered":"2024-09-10T16:05:05.133357+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:05:05.133357+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"43'4200","ondisk_log_start":"43'4200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25317859,"num_objects":6182,"num_object_clones":0,"num_object_copies":12364,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6182,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12095,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":79},{"shards":"2","objects":2059}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":95,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:29.860108+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:02:29.860108+0000","last_peered":"2024-09-10T16:02:29.860108+0000","last_clean":"2024-09-10T16:02:29.860108+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:02:29.860108+0000","last_undegraded":"2024-09-10T16:02:29.860108+0000","last_fullsized":"2024-09-10T16:02:29.860108+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'14746","reported_seq":14877,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:09.308228+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:05:09.308228+0000","last_peered":"2024-09-10T16:05:09.308228+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:05:09.308228+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"43'4700","ondisk_log_start":"43'4700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25247744,"num_objects":6164,"num_object_clones":0,"num_object_copies":12328,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6164,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11250,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1909}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":105,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:12.405385+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:02:12.405385+0000","last_peered":"2024-09-10T16:02:12.405385+0000","last_clean":"2024-09-10T16:02:12.405385+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:02:12.405385+0000","last_undegraded":"2024-09-10T16:02:12.405385+0000","last_fullsized":"2024-09-10T16:02:12.405385+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'14612","reported_seq":14748,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:05.278101+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:05:05.278101+0000","last_peered":"2024-09-10T16:05:05.278101+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:05:05.278101+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"43'4600","ondisk_log_start":"43'4600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25268707,"num_objects":6170,"num_object_clones":0,"num_object_copies":12340,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6170,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12292,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":85},{"shards":"2","objects":1866}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104617+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:02:08.104617+0000","last_peered":"2024-09-10T16:02:08.104617+0000","last_clean":"2024-09-10T16:02:08.104617+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:02:08.104617+0000","last_undegraded":"2024-09-10T16:02:08.104617+0000","last_fullsized":"2024-09-10T16:02:08.104617+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'14866","reported_seq":14996,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:05.166236+0000","last_change":"2024-09-10T15:59:16.174113+0000","last_active":"2024-09-10T16:05:05.166236+0000","last_peered":"2024-09-10T16:05:05.166236+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:05:05.166236+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"43'4800","ondisk_log_start":"43'4800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25821184,"num_objects":6304,"num_object_clones":0,"num_object_copies":12608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4354,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6304,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12420,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":74},{"shards":"2","objects":1952}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'14766","reported_seq":14902,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:09.318844+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:05:09.318844+0000","last_peered":"2024-09-10T16:05:09.318844+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:05:09.318844+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"43'4700","ondisk_log_start":"43'4700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24870912,"num_objects":6072,"num_object_clones":0,"num_object_copies":12144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6072,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11397,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1768}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103316+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:02:08.103316+0000","last_peered":"2024-09-10T16:02:08.103316+0000","last_clean":"2024-09-10T16:02:08.103316+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:02:08.103316+0000","last_undegraded":"2024-09-10T16:02:08.103316+0000","last_fullsized":"2024-09-10T16:02:08.103316+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'14545","reported_seq":22496,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:09.318878+0000","last_change":"2024-09-10T16:03:29.605682+0000","last_active":"2024-09-10T16:05:09.318878+0000","last_peered":"2024-09-10T16:05:09.318878+0000","last_clean":"2024-09-10T15:59:14.115595+0000","last_became_active":"2024-09-10T15:59:16.244089+0000","last_became_peered":"2024-09-10T15:59:16.244089+0000","last_unstale":"2024-09-10T16:05:09.318878+0000","last_undegraded":"2024-09-10T16:03:29.603815+0000","last_fullsized":"2024-09-10T15:59:15.774437+0000","mapping_epoch":42,"log_start":"43'4500","ondisk_log_start":"43'4500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24793088,"num_objects":6053,"num_object_clones":0,"num_object_copies":12106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":303,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6053,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10935,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3903,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5750}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":95,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:29.877831+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:02:29.877831+0000","last_peered":"2024-09-10T16:02:29.877831+0000","last_clean":"2024-09-10T16:02:29.877831+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:02:29.877831+0000","last_undegraded":"2024-09-10T16:02:29.877831+0000","last_fullsized":"2024-09-10T16:02:29.877831+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'14930","reported_seq":15073,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:09.253687+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:05:09.253687+0000","last_peered":"2024-09-10T16:05:09.253687+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:05:09.253687+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"43'4900","ondisk_log_start":"43'4900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25059328,"num_objects":6118,"num_object_clones":0,"num_object_copies":12236,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6118,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11185,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1747}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104463+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:02:08.104463+0000","last_peered":"2024-09-10T16:02:08.104463+0000","last_clean":"2024-09-10T16:02:08.104463+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:02:08.104463+0000","last_undegraded":"2024-09-10T16:02:08.104463+0000","last_fullsized":"2024-09-10T16:02:08.104463+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104448+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:02:08.104448+0000","last_peered":"2024-09-10T16:02:08.104448+0000","last_clean":"2024-09-10T16:02:08.104448+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:02:08.104448+0000","last_undegraded":"2024-09-10T16:02:08.104448+0000","last_fullsized":"2024-09-10T16:02:08.104448+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'14964","reported_seq":15107,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:09.267271+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:05:09.267271+0000","last_peered":"2024-09-10T16:05:09.267271+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:05:09.267271+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"43'4900","ondisk_log_start":"43'4900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25535430,"num_objects":6236,"num_object_clones":0,"num_object_copies":12472,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6236,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11127,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1913},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":404079350,"num_objects":98662,"num_object_clones":0,"num_object_copies":197324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43214,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98662,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":193444,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8175,"num_bytes_recovered":33095680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1019113472,"data_stored":1018964421,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160770,"ondisk_log_size":160770,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986261,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":653528,"kb_used_data":337224,"kb_used_omap":4,"kb_used_meta":316283,"kb_avail":93718312,"statfs":{"total":96636764160,"available":95967551488,"internally_reserved":0,"allocated":345317376,"data_stored":345025143,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":323874474},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,0,2,2,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659207,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":602980,"kb_used_data":296404,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":93768860,"statfs":{"total":96636764160,"available":96019312640,"internally_reserved":0,"allocated":303517696,"data_stored":303188869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":313913006},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,2,3,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182490,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":774944,"kb_used_data":364304,"kb_used_omap":4,"kb_used_meta":410619,"kb_avail":93596896,"statfs":{"total":96636764160,"available":95843221504,"internally_reserved":0,"allocated":373047296,"data_stored":372755058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":420474538},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,1,4,3,5,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":371322880,"data_stored":371286750,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":303038464,"data_stored":302965745,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":344752128,"data_stored":344711926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:05:11.035 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:05:11.598 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:05:11.599 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:05:11.622 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":275,"stamp":"2024-09-10T16:05:09.679707+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":404079369,"num_objects":98664,"num_object_clones":0,"num_object_copies":197328,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43214,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98664,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":193446,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8177,"num_bytes_recovered":33095699,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160773,"ondisk_log_size":160773,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2031452,"kb_used_data":997932,"kb_used_omap":12,"kb_used_meta":1033459,"kb_avail":281084068,"statfs":{"total":289910292480,"available":287830085632,"internally_reserved":0,"allocated":1021882368,"data_stored":1020969070,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1058262018},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,0,4,8,8,10,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-24502272,"num_objects":-5982,"num_object_clones":0,"num_object_copies":-11964,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-91,"num_objects_misplaced":-376,"num_objects_unfound":0,"num_objects_dirty":-5982,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5982,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":467,"num_bytes_recovered":1540096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006858"},"pg_stats":[{"pgid":"2.f","version":"45'14628","reported_seq":14704,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:09.253590+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:05:09.253590+0000","last_peered":"2024-09-10T16:05:09.253590+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:05:09.253590+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"43'4600","ondisk_log_start":"43'4600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24756224,"num_objects":6044,"num_object_clones":0,"num_object_copies":12088,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6044,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14628,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1785}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'14639","reported_seq":14752,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:05:05.297237+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:05:05.297237+0000","last_peered":"2024-09-10T16:05:05.297237+0000","last_clean":"2024-09-10T16:05:05.297237+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:05:05.297237+0000","last_undegraded":"2024-09-10T16:05:05.297237+0000","last_fullsized":"2024-09-10T16:05:05.297237+0000","mapping_epoch":35,"log_start":"43'4600","ondisk_log_start":"43'4600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25416646,"num_objects":6207,"num_object_clones":0,"num_object_copies":12414,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6207,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":12744,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'14638","reported_seq":23317,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:05:07.126726+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:05:07.126726+0000","last_peered":"2024-09-10T16:05:07.126726+0000","last_clean":"2024-09-10T16:05:07.126726+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:05:07.126726+0000","last_undegraded":"2024-09-10T16:05:07.126726+0000","last_fullsized":"2024-09-10T16:05:07.126726+0000","mapping_epoch":44,"log_start":"43'4600","ondisk_log_start":"43'4600","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25227288,"num_objects":6160,"num_object_clones":0,"num_object_copies":12320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6160,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12642,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'14488","reported_seq":14570,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:05:05.305474+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:05:05.305474+0000","last_peered":"2024-09-10T16:05:05.305474+0000","last_clean":"2024-09-10T16:05:05.305474+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:05:05.305474+0000","last_undegraded":"2024-09-10T16:05:05.305474+0000","last_fullsized":"2024-09-10T16:05:05.305474+0000","mapping_epoch":35,"log_start":"43'4400","ondisk_log_start":"43'4400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25280512,"num_objects":6172,"num_object_clones":0,"num_object_copies":12344,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6172,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14488,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'14776","reported_seq":14907,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.212879+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:05:09.212879+0000","last_peered":"2024-09-10T16:05:09.212879+0000","last_clean":"2024-09-10T16:05:09.212879+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:05:09.212879+0000","last_undegraded":"2024-09-10T16:05:09.212879+0000","last_fullsized":"2024-09-10T16:05:09.212879+0000","mapping_epoch":34,"log_start":"43'4700","ondisk_log_start":"43'4700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25047523,"num_objects":6116,"num_object_clones":0,"num_object_copies":12232,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6116,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11487,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'14942","reported_seq":15081,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:09.310517+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:05:09.310517+0000","last_peered":"2024-09-10T16:05:09.310517+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:05:09.310517+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"43'4900","ondisk_log_start":"43'4900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25063907,"num_objects":6120,"num_object_clones":0,"num_object_copies":12240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6120,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11554,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1749},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'14809","reported_seq":14948,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:09.268037+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:05:09.268037+0000","last_peered":"2024-09-10T16:05:09.268037+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:05:09.268037+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"43'4800","ondisk_log_start":"43'4800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25695174,"num_objects":6275,"num_object_clones":0,"num_object_copies":12550,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6275,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":11308,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2038},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'15045","reported_seq":15174,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.284726+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:05:09.284726+0000","last_peered":"2024-09-10T16:05:09.284726+0000","last_clean":"2024-09-10T16:05:09.284726+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:05:09.284726+0000","last_undegraded":"2024-09-10T16:05:09.284726+0000","last_fullsized":"2024-09-10T16:05:09.284726+0000","mapping_epoch":34,"log_start":"43'5000","ondisk_log_start":"43'5000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25677824,"num_objects":6269,"num_object_clones":0,"num_object_copies":12538,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6269,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11892,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104261+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:02:08.104261+0000","last_peered":"2024-09-10T16:02:08.104261+0000","last_clean":"2024-09-10T16:02:08.104261+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:02:08.104261+0000","last_undegraded":"2024-09-10T16:02:08.104261+0000","last_fullsized":"2024-09-10T16:02:08.104261+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'14276","reported_seq":14412,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:05.133357+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:05:05.133357+0000","last_peered":"2024-09-10T16:05:05.133357+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:05:05.133357+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"43'4200","ondisk_log_start":"43'4200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25317859,"num_objects":6182,"num_object_clones":0,"num_object_copies":12364,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6182,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12095,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":79},{"shards":"2","objects":2059}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":95,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:29.860108+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:02:29.860108+0000","last_peered":"2024-09-10T16:02:29.860108+0000","last_clean":"2024-09-10T16:02:29.860108+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:02:29.860108+0000","last_undegraded":"2024-09-10T16:02:29.860108+0000","last_fullsized":"2024-09-10T16:02:29.860108+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'14746","reported_seq":14877,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:09.308228+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:05:09.308228+0000","last_peered":"2024-09-10T16:05:09.308228+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:05:09.308228+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"43'4700","ondisk_log_start":"43'4700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25247744,"num_objects":6164,"num_object_clones":0,"num_object_copies":12328,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6164,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11250,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1909}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":105,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:12.405385+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:02:12.405385+0000","last_peered":"2024-09-10T16:02:12.405385+0000","last_clean":"2024-09-10T16:02:12.405385+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:02:12.405385+0000","last_undegraded":"2024-09-10T16:02:12.405385+0000","last_fullsized":"2024-09-10T16:02:12.405385+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'14612","reported_seq":14748,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:05.278101+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:05:05.278101+0000","last_peered":"2024-09-10T16:05:05.278101+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:05:05.278101+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"43'4600","ondisk_log_start":"43'4600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25268707,"num_objects":6170,"num_object_clones":0,"num_object_copies":12340,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6170,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12292,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":85},{"shards":"2","objects":1866}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104617+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:02:08.104617+0000","last_peered":"2024-09-10T16:02:08.104617+0000","last_clean":"2024-09-10T16:02:08.104617+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:02:08.104617+0000","last_undegraded":"2024-09-10T16:02:08.104617+0000","last_fullsized":"2024-09-10T16:02:08.104617+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'14866","reported_seq":14996,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:05.166236+0000","last_change":"2024-09-10T15:59:16.174113+0000","last_active":"2024-09-10T16:05:05.166236+0000","last_peered":"2024-09-10T16:05:05.166236+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:05:05.166236+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"43'4800","ondisk_log_start":"43'4800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25821184,"num_objects":6304,"num_object_clones":0,"num_object_copies":12608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4354,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6304,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12420,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":74},{"shards":"2","objects":1952}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'14766","reported_seq":14902,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:09.318844+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:05:09.318844+0000","last_peered":"2024-09-10T16:05:09.318844+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:05:09.318844+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"43'4700","ondisk_log_start":"43'4700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24870912,"num_objects":6072,"num_object_clones":0,"num_object_copies":12144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6072,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11397,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1768}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103316+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:02:08.103316+0000","last_peered":"2024-09-10T16:02:08.103316+0000","last_clean":"2024-09-10T16:02:08.103316+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:02:08.103316+0000","last_undegraded":"2024-09-10T16:02:08.103316+0000","last_fullsized":"2024-09-10T16:02:08.103316+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'14545","reported_seq":22496,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:09.318878+0000","last_change":"2024-09-10T16:03:29.605682+0000","last_active":"2024-09-10T16:05:09.318878+0000","last_peered":"2024-09-10T16:05:09.318878+0000","last_clean":"2024-09-10T15:59:14.115595+0000","last_became_active":"2024-09-10T15:59:16.244089+0000","last_became_peered":"2024-09-10T15:59:16.244089+0000","last_unstale":"2024-09-10T16:05:09.318878+0000","last_undegraded":"2024-09-10T16:03:29.603815+0000","last_fullsized":"2024-09-10T15:59:15.774437+0000","mapping_epoch":42,"log_start":"43'4500","ondisk_log_start":"43'4500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24793088,"num_objects":6053,"num_object_clones":0,"num_object_copies":12106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":303,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6053,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10935,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3903,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5750}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":95,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:29.877831+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:02:29.877831+0000","last_peered":"2024-09-10T16:02:29.877831+0000","last_clean":"2024-09-10T16:02:29.877831+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:02:29.877831+0000","last_undegraded":"2024-09-10T16:02:29.877831+0000","last_fullsized":"2024-09-10T16:02:29.877831+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'14930","reported_seq":15073,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:09.253687+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:05:09.253687+0000","last_peered":"2024-09-10T16:05:09.253687+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:05:09.253687+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"43'4900","ondisk_log_start":"43'4900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25059328,"num_objects":6118,"num_object_clones":0,"num_object_copies":12236,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6118,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11185,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1747}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104463+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:02:08.104463+0000","last_peered":"2024-09-10T16:02:08.104463+0000","last_clean":"2024-09-10T16:02:08.104463+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:02:08.104463+0000","last_undegraded":"2024-09-10T16:02:08.104463+0000","last_fullsized":"2024-09-10T16:02:08.104463+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104448+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:02:08.104448+0000","last_peered":"2024-09-10T16:02:08.104448+0000","last_clean":"2024-09-10T16:02:08.104448+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:02:08.104448+0000","last_undegraded":"2024-09-10T16:02:08.104448+0000","last_fullsized":"2024-09-10T16:02:08.104448+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'14964","reported_seq":15107,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:09.267271+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:05:09.267271+0000","last_peered":"2024-09-10T16:05:09.267271+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:05:09.267271+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"43'4900","ondisk_log_start":"43'4900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25535430,"num_objects":6236,"num_object_clones":0,"num_object_copies":12472,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6236,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11127,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1913},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":404079350,"num_objects":98662,"num_object_clones":0,"num_object_copies":197324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43214,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98662,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":193444,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8175,"num_bytes_recovered":33095680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1019113472,"data_stored":1018964421,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160770,"ondisk_log_size":160770,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986261,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":653528,"kb_used_data":337224,"kb_used_omap":4,"kb_used_meta":316283,"kb_avail":93718312,"statfs":{"total":96636764160,"available":95967551488,"internally_reserved":0,"allocated":345317376,"data_stored":345025143,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":323874474},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,0,2,2,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659207,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":602980,"kb_used_data":296404,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":93768860,"statfs":{"total":96636764160,"available":96019312640,"internally_reserved":0,"allocated":303517696,"data_stored":303188869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":313913006},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,2,3,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182490,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":774944,"kb_used_data":364304,"kb_used_omap":4,"kb_used_meta":410619,"kb_avail":93596896,"statfs":{"total":96636764160,"available":95843221504,"internally_reserved":0,"allocated":373047296,"data_stored":372755058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":420474538},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,1,4,3,5,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":371322880,"data_stored":371286750,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":303038464,"data_stored":302965745,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":344752128,"data_stored":344711926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:05:11.624 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-10T16:05:11.624 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "45'14628", 'reported_seq': 14704, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:05:09.253590+0000', 'last_change': '2024-09-10T15:59:16.281558+0000', 'last_active': '2024-09-10T16:05:09.253590+0000', 'last_peered': '2024-09-10T16:05:09.253590+0000', 'last_clean': '2024-09-10T15:59:14.132456+0000', 'last_became_active': '2024-09-10T15:59:16.274407+0000', 'last_became_peered': '2024-09-10T15:59:16.274407+0000', 'last_unstale': '2024-09-10T16:05:09.253590+0000', 'last_undegraded': '2024-09-10T15:59:15.851279+0000', 'last_fullsized': '2024-09-10T15:59:15.774356+0000', 'mapping_epoch': 42, 'log_start': "43'4600", 'ondisk_log_start': "43'4600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'objects_scrubbed': 0, 'log_size': 10028, 'log_dups_size': 0, 'ondisk_log_size': 10028, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24756224, 'num_objects': 6044, 'num_object_clones': 0, 'num_object_copies': 12088, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4259, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6044, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14628, 'num_write_kb': 41344, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1785}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:05:11.625 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-10T16:05:11.625 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "45'14942", 'reported_seq': 15081, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:05:09.310517+0000', 'last_change': '2024-09-10T15:59:16.308264+0000', 'last_active': '2024-09-10T16:05:09.310517+0000', 'last_peered': '2024-09-10T16:05:09.310517+0000', 'last_clean': '2024-09-10T15:59:14.111551+0000', 'last_became_active': '2024-09-10T15:59:16.300040+0000', 'last_became_peered': '2024-09-10T15:59:16.300040+0000', 'last_unstale': '2024-09-10T16:05:09.310517+0000', 'last_undegraded': '2024-09-10T15:59:15.878234+0000', 'last_fullsized': '2024-09-10T15:59:15.769328+0000', 'mapping_epoch': 42, 'log_start': "43'4900", 'ondisk_log_start': "43'4900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3442", 'last_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'objects_scrubbed': 3384, 'log_size': 10042, 'log_dups_size': 0, 'ondisk_log_size': 10042, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1292, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25063907, 'num_objects': 6120, 'num_object_clones': 0, 'num_object_copies': 12240, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4372, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6120, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 11554, 'num_write_kb': 28580, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1749}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:05:11.625 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-10T16:05:11.625 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "45'14809", 'reported_seq': 14948, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:05:09.268037+0000', 'last_change': '2024-09-10T15:59:16.331899+0000', 'last_active': '2024-09-10T16:05:09.268037+0000', 'last_peered': '2024-09-10T16:05:09.268037+0000', 'last_clean': '2024-09-10T15:59:14.139436+0000', 'last_became_active': '2024-09-10T15:59:16.318715+0000', 'last_became_peered': '2024-09-10T15:59:16.318715+0000', 'last_unstale': '2024-09-10T16:05:09.268037+0000', 'last_undegraded': '2024-09-10T15:59:15.877524+0000', 'last_fullsized': '2024-09-10T15:59:15.769042+0000', 'mapping_epoch': 42, 'log_start': "43'4800", 'ondisk_log_start': "43'4800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3537", 'last_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'objects_scrubbed': 3497, 'log_size': 10009, 'log_dups_size': 0, 'ondisk_log_size': 10009, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1224, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25695174, 'num_objects': 6275, 'num_object_clones': 0, 'num_object_copies': 12550, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4237, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6275, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 11308, 'num_write_kb': 28172, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2038}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:05:11.626 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-10T16:05:11.626 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "45'14276", 'reported_seq': 14412, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:05:05.133357+0000', 'last_change': '2024-09-10T15:59:16.175680+0000', 'last_active': '2024-09-10T16:05:05.133357+0000', 'last_peered': '2024-09-10T16:05:05.133357+0000', 'last_clean': '2024-09-10T15:59:14.088187+0000', 'last_became_active': '2024-09-10T15:59:16.165810+0000', 'last_became_peered': '2024-09-10T15:59:16.165810+0000', 'last_unstale': '2024-09-10T16:05:05.133357+0000', 'last_undegraded': '2024-09-10T15:59:15.822098+0000', 'last_fullsized': '2024-09-10T15:59:15.769123+0000', 'mapping_epoch': 42, 'log_start': "43'4200", 'ondisk_log_start': "43'4200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2218", 'last_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'objects_scrubbed': 2175, 'log_size': 10076, 'log_dups_size': 0, 'ondisk_log_size': 10076, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 825, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25317859, 'num_objects': 6182, 'num_object_clones': 0, 'num_object_copies': 12364, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4124, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6182, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 12095, 'num_write_kb': 32204, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 79}, {'shards': '2', 'objects': 2059}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:05:11.626 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-10T16:05:11.626 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "45'14746", 'reported_seq': 14877, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:05:09.308228+0000', 'last_change': '2024-09-10T15:59:16.277742+0000', 'last_active': '2024-09-10T16:05:09.308228+0000', 'last_peered': '2024-09-10T16:05:09.308228+0000', 'last_clean': '2024-09-10T15:59:14.129081+0000', 'last_became_active': '2024-09-10T15:59:16.269835+0000', 'last_became_peered': '2024-09-10T15:59:16.269835+0000', 'last_unstale': '2024-09-10T16:05:09.308228+0000', 'last_undegraded': '2024-09-10T15:59:15.860857+0000', 'last_fullsized': '2024-09-10T15:59:15.771410+0000', 'mapping_epoch': 42, 'log_start': "43'4700", 'ondisk_log_start': "43'4700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3557", 'last_scrub_stamp': '2024-09-10T15:59:02.493678+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:02.493678+0000', 'objects_scrubbed': 3488, 'log_size': 10046, 'log_dups_size': 0, 'ondisk_log_size': 10046, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1472, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25247744, 'num_objects': 6164, 'num_object_clones': 0, 'num_object_copies': 12328, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4257, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6164, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11250, 'num_write_kb': 27852, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1909}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:05:11.627 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-10T16:05:11.627 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "45'14612", 'reported_seq': 14748, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:05:05.278101+0000', 'last_change': '2024-09-10T15:59:16.284311+0000', 'last_active': '2024-09-10T16:05:05.278101+0000', 'last_peered': '2024-09-10T16:05:05.278101+0000', 'last_clean': '2024-09-10T15:59:14.116041+0000', 'last_became_active': '2024-09-10T15:59:16.274543+0000', 'last_became_peered': '2024-09-10T15:59:16.274543+0000', 'last_unstale': '2024-09-10T16:05:05.278101+0000', 'last_undegraded': '2024-09-10T15:59:15.812484+0000', 'last_fullsized': '2024-09-10T15:59:15.770885+0000', 'mapping_epoch': 42, 'log_start': "43'4600", 'ondisk_log_start': "43'4600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2356", 'last_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'objects_scrubbed': 2316, 'log_size': 10012, 'log_dups_size': 0, 'ondisk_log_size': 10012, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 783, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25268707, 'num_objects': 6170, 'num_object_clones': 0, 'num_object_copies': 12340, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6170, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 12292, 'num_write_kb': 32292, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 85}, {'shards': '2', 'objects': 1866}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:05:11.627 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-10T16:05:11.627 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "45'14866", 'reported_seq': 14996, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:05:05.166236+0000', 'last_change': '2024-09-10T15:59:16.174113+0000', 'last_active': '2024-09-10T16:05:05.166236+0000', 'last_peered': '2024-09-10T16:05:05.166236+0000', 'last_clean': '2024-09-10T15:59:14.110654+0000', 'last_became_active': '2024-09-10T15:59:16.164645+0000', 'last_became_peered': '2024-09-10T15:59:16.164645+0000', 'last_unstale': '2024-09-10T16:05:05.166236+0000', 'last_undegraded': '2024-09-10T15:59:15.803593+0000', 'last_fullsized': '2024-09-10T15:59:15.769083+0000', 'mapping_epoch': 42, 'log_start': "43'4800", 'ondisk_log_start': "43'4800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2486", 'last_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'objects_scrubbed': 2440, 'log_size': 10066, 'log_dups_size': 0, 'ondisk_log_size': 10066, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 792, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25821184, 'num_objects': 6304, 'num_object_clones': 0, 'num_object_copies': 12608, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4354, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6304, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12420, 'num_write_kb': 32568, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 74}, {'shards': '2', 'objects': 1952}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:05:11.627 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-10T16:05:11.628 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "45'14766", 'reported_seq': 14902, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:05:09.318844+0000', 'last_change': '2024-09-10T15:59:16.315641+0000', 'last_active': '2024-09-10T16:05:09.318844+0000', 'last_peered': '2024-09-10T16:05:09.318844+0000', 'last_clean': '2024-09-10T15:59:14.117001+0000', 'last_became_active': '2024-09-10T15:59:16.302245+0000', 'last_became_peered': '2024-09-10T15:59:16.302245+0000', 'last_unstale': '2024-09-10T16:05:09.318844+0000', 'last_undegraded': '2024-09-10T15:59:15.845012+0000', 'last_fullsized': '2024-09-10T15:59:15.770337+0000', 'mapping_epoch': 42, 'log_start': "43'4700", 'ondisk_log_start': "43'4700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3404", 'last_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'objects_scrubbed': 3367, 'log_size': 10066, 'log_dups_size': 0, 'ondisk_log_size': 10066, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 983, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24870912, 'num_objects': 6072, 'num_object_clones': 0, 'num_object_copies': 12144, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6072, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11397, 'num_write_kb': 28204, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1768}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:05:11.628 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-10T16:05:11.628 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "45'14545", 'reported_seq': 22496, 'reported_epoch': 45, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:05:09.318878+0000', 'last_change': '2024-09-10T16:03:29.605682+0000', 'last_active': '2024-09-10T16:05:09.318878+0000', 'last_peered': '2024-09-10T16:05:09.318878+0000', 'last_clean': '2024-09-10T15:59:14.115595+0000', 'last_became_active': '2024-09-10T15:59:16.244089+0000', 'last_became_peered': '2024-09-10T15:59:16.244089+0000', 'last_unstale': '2024-09-10T16:05:09.318878+0000', 'last_undegraded': '2024-09-10T16:03:29.603815+0000', 'last_fullsized': '2024-09-10T15:59:15.774437+0000', 'mapping_epoch': 42, 'log_start': "43'4500", 'ondisk_log_start': "43'4500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3653", 'last_scrub_stamp': '2024-09-10T15:59:05.298468+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:05.298468+0000', 'objects_scrubbed': 3602, 'log_size': 10045, 'log_dups_size': 0, 'ondisk_log_size': 10045, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1289, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24793088, 'num_objects': 6053, 'num_object_clones': 0, 'num_object_copies': 12106, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 303, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6053, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10935, 'num_write_kb': 26772, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3903, 'num_bytes_recovered': 15601664, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5750}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:05:11.628 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-10T16:05:11.629 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "45'14930", 'reported_seq': 15073, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:05:09.253687+0000', 'last_change': '2024-09-10T15:59:16.311291+0000', 'last_active': '2024-09-10T16:05:09.253687+0000', 'last_peered': '2024-09-10T16:05:09.253687+0000', 'last_clean': '2024-09-10T15:59:14.116363+0000', 'last_became_active': '2024-09-10T15:59:16.301244+0000', 'last_became_peered': '2024-09-10T15:59:16.301244+0000', 'last_unstale': '2024-09-10T16:05:09.253687+0000', 'last_undegraded': '2024-09-10T15:59:15.883434+0000', 'last_fullsized': '2024-09-10T15:59:15.772756+0000', 'mapping_epoch': 42, 'log_start': "43'4900", 'ondisk_log_start': "43'4900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3789", 'last_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'objects_scrubbed': 3741, 'log_size': 10030, 'log_dups_size': 0, 'ondisk_log_size': 10030, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1440, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25059328, 'num_objects': 6118, 'num_object_clones': 0, 'num_object_copies': 12236, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4373, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6118, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11185, 'num_write_kb': 27124, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1747}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:05:11.629 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-10T16:05:11.629 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "45'14964", 'reported_seq': 15107, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:05:09.267271+0000', 'last_change': '2024-09-10T15:59:16.313409+0000', 'last_active': '2024-09-10T16:05:09.267271+0000', 'last_peered': '2024-09-10T16:05:09.267271+0000', 'last_clean': '2024-09-10T15:59:14.129780+0000', 'last_became_active': '2024-09-10T15:59:16.300932+0000', 'last_became_peered': '2024-09-10T15:59:16.300932+0000', 'last_unstale': '2024-09-10T16:05:09.267271+0000', 'last_undegraded': '2024-09-10T15:59:15.887823+0000', 'last_fullsized': '2024-09-10T15:59:15.768513+0000', 'mapping_epoch': 42, 'log_start': "43'4900", 'ondisk_log_start': "43'4900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3880", 'last_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'objects_scrubbed': 3827, 'log_size': 10064, 'log_dups_size': 0, 'ondisk_log_size': 10064, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1397, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25535430, 'num_objects': 6236, 'num_object_clones': 0, 'num_object_copies': 12472, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4325, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6236, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11127, 'num_write_kb': 27072, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1913}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:05:11.630 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T16:05:12.326 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:05:12.326 INFO:teuthology.orchestra.run.smithi115.stdout:{"fsid":"7effee29-df05-4351-9741-ee203564ed81","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 43214/195892 objects degraded (22.060%), 11 pgs degraded, 11 pgs undersized","count":22},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":476,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983954,"num_in_osds":3,"osd_in_since":1725983947,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":97946,"data_bytes":401138441,"bytes_used":2079748096,"bytes_avail":287830544384,"bytes_total":289910292480,"degraded_objects":43214,"degraded_total":195892,"degraded_ratio":0.2206011475711106,"recovering_objects_per_sec":26,"recovering_bytes_per_sec":76415,"recovering_keys_per_sec":0,"num_objects_recovered":315,"num_bytes_recovered":917504,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":437},"fsmap":{"epoch":1,"btime":"2024-09-10T15:57:14:967120+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T16:00:13.560776+0000","services":{}},"progress_events":{}} 2024-09-10T16:05:12.351 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-10T16:05:12.352 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:05:12.903 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:05:12.906 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:05:12.924 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":276,"stamp":"2024-09-10T16:05:11.680352+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":401138441,"num_objects":97946,"num_object_clones":0,"num_object_copies":195892,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43214,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":97946,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":194164,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8177,"num_bytes_recovered":33095699,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160691,"ondisk_log_size":160691,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2031004,"kb_used_data":993324,"kb_used_omap":12,"kb_used_meta":1037619,"kb_avail":281084516,"statfs":{"total":289910292480,"available":287830544384,"internally_reserved":0,"allocated":1017163776,"data_stored":1016250478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1062521858},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,0,7,10,8,9,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-21520384,"num_objects":-5254,"num_object_clones":0,"num_object_copies":-10508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-91,"num_objects_misplaced":-224,"num_objects_unfound":0,"num_objects_dirty":-5254,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5254,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":315,"num_bytes_recovered":917504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006853"},"pg_stats":[{"pgid":"2.f","version":"45'14628","reported_seq":14704,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:09.253590+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:05:09.253590+0000","last_peered":"2024-09-10T16:05:09.253590+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:05:09.253590+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"43'4600","ondisk_log_start":"43'4600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24756224,"num_objects":6044,"num_object_clones":0,"num_object_copies":12088,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6044,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14628,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1785}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'14781","reported_seq":14895,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:05:10.184446+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:05:10.184446+0000","last_peered":"2024-09-10T16:05:10.184446+0000","last_clean":"2024-09-10T16:05:10.184446+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:05:10.184446+0000","last_undegraded":"2024-09-10T16:05:10.184446+0000","last_fullsized":"2024-09-10T16:05:10.184446+0000","mapping_epoch":35,"log_start":"43'4700","ondisk_log_start":"43'4700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24835014,"num_objects":6065,"num_object_clones":0,"num_object_copies":12130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6065,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":12886,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'14638","reported_seq":23317,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:05:07.126726+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:05:07.126726+0000","last_peered":"2024-09-10T16:05:07.126726+0000","last_clean":"2024-09-10T16:05:07.126726+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:05:07.126726+0000","last_undegraded":"2024-09-10T16:05:07.126726+0000","last_fullsized":"2024-09-10T16:05:07.126726+0000","mapping_epoch":44,"log_start":"43'4600","ondisk_log_start":"43'4600","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25227288,"num_objects":6160,"num_object_clones":0,"num_object_copies":12320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6160,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12642,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'14630","reported_seq":14712,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:05:10.217500+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:05:10.217500+0000","last_peered":"2024-09-10T16:05:10.217500+0000","last_clean":"2024-09-10T16:05:10.217500+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:05:10.217500+0000","last_undegraded":"2024-09-10T16:05:10.217500+0000","last_fullsized":"2024-09-10T16:05:10.217500+0000","mapping_epoch":35,"log_start":"43'4600","ondisk_log_start":"43'4600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24698880,"num_objects":6030,"num_object_clones":0,"num_object_copies":12060,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6030,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14630,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'14776","reported_seq":14907,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.212879+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:05:09.212879+0000","last_peered":"2024-09-10T16:05:09.212879+0000","last_clean":"2024-09-10T16:05:09.212879+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:05:09.212879+0000","last_undegraded":"2024-09-10T16:05:09.212879+0000","last_fullsized":"2024-09-10T16:05:09.212879+0000","mapping_epoch":34,"log_start":"43'4700","ondisk_log_start":"43'4700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25047523,"num_objects":6116,"num_object_clones":0,"num_object_copies":12232,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6116,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11487,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'14942","reported_seq":15081,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:09.310517+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:05:09.310517+0000","last_peered":"2024-09-10T16:05:09.310517+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:05:09.310517+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"43'4900","ondisk_log_start":"43'4900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25063907,"num_objects":6120,"num_object_clones":0,"num_object_copies":12240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6120,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11554,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1749},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'14809","reported_seq":14948,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:09.268037+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:05:09.268037+0000","last_peered":"2024-09-10T16:05:09.268037+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:05:09.268037+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"43'4800","ondisk_log_start":"43'4800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25695174,"num_objects":6275,"num_object_clones":0,"num_object_copies":12550,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6275,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":11308,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2038},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'15045","reported_seq":15174,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.284726+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:05:09.284726+0000","last_peered":"2024-09-10T16:05:09.284726+0000","last_clean":"2024-09-10T16:05:09.284726+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:05:09.284726+0000","last_undegraded":"2024-09-10T16:05:09.284726+0000","last_fullsized":"2024-09-10T16:05:09.284726+0000","mapping_epoch":34,"log_start":"43'5000","ondisk_log_start":"43'5000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25677824,"num_objects":6269,"num_object_clones":0,"num_object_copies":12538,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6269,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11892,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104261+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:02:08.104261+0000","last_peered":"2024-09-10T16:02:08.104261+0000","last_clean":"2024-09-10T16:02:08.104261+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:02:08.104261+0000","last_undegraded":"2024-09-10T16:02:08.104261+0000","last_fullsized":"2024-09-10T16:02:08.104261+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'14427","reported_seq":14568,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:10.319443+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:05:10.319443+0000","last_peered":"2024-09-10T16:05:10.319443+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:05:10.319443+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"43'4400","ondisk_log_start":"43'4400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24699363,"num_objects":6031,"num_object_clones":0,"num_object_copies":12062,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6031,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12246,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":1908}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":95,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:29.860108+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:02:29.860108+0000","last_peered":"2024-09-10T16:02:29.860108+0000","last_clean":"2024-09-10T16:02:29.860108+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:02:29.860108+0000","last_undegraded":"2024-09-10T16:02:29.860108+0000","last_fullsized":"2024-09-10T16:02:29.860108+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'14746","reported_seq":14877,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:09.308228+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:05:09.308228+0000","last_peered":"2024-09-10T16:05:09.308228+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:05:09.308228+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"43'4700","ondisk_log_start":"43'4700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25247744,"num_objects":6164,"num_object_clones":0,"num_object_copies":12328,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6164,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11250,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1909}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":105,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:12.405385+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:02:12.405385+0000","last_peered":"2024-09-10T16:02:12.405385+0000","last_clean":"2024-09-10T16:02:12.405385+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:02:12.405385+0000","last_undegraded":"2024-09-10T16:02:12.405385+0000","last_fullsized":"2024-09-10T16:02:12.405385+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'14755","reported_seq":14890,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:10.283126+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:05:10.283126+0000","last_peered":"2024-09-10T16:05:10.283126+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:05:10.283126+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"43'4700","ondisk_log_start":"43'4700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24682979,"num_objects":6027,"num_object_clones":0,"num_object_copies":12054,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6027,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12435,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":1723}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104617+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:02:08.104617+0000","last_peered":"2024-09-10T16:02:08.104617+0000","last_clean":"2024-09-10T16:02:08.104617+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:02:08.104617+0000","last_undegraded":"2024-09-10T16:02:08.104617+0000","last_fullsized":"2024-09-10T16:02:08.104617+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'15006","reported_seq":15136,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:10.222936+0000","last_change":"2024-09-10T15:59:16.174113+0000","last_active":"2024-09-10T16:05:10.222936+0000","last_peered":"2024-09-10T16:05:10.222936+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:05:10.222936+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"43'5000","ondisk_log_start":"43'5000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25247744,"num_objects":6164,"num_object_clones":0,"num_object_copies":12328,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4354,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6164,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12560,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1812}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'14766","reported_seq":14902,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:09.318844+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:05:09.318844+0000","last_peered":"2024-09-10T16:05:09.318844+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:05:09.318844+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"43'4700","ondisk_log_start":"43'4700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24870912,"num_objects":6072,"num_object_clones":0,"num_object_copies":12144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6072,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11397,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1768}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103316+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:02:08.103316+0000","last_peered":"2024-09-10T16:02:08.103316+0000","last_clean":"2024-09-10T16:02:08.103316+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:02:08.103316+0000","last_undegraded":"2024-09-10T16:02:08.103316+0000","last_fullsized":"2024-09-10T16:02:08.103316+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'14545","reported_seq":22496,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:09.318878+0000","last_change":"2024-09-10T16:03:29.605682+0000","last_active":"2024-09-10T16:05:09.318878+0000","last_peered":"2024-09-10T16:05:09.318878+0000","last_clean":"2024-09-10T15:59:14.115595+0000","last_became_active":"2024-09-10T15:59:16.244089+0000","last_became_peered":"2024-09-10T15:59:16.244089+0000","last_unstale":"2024-09-10T16:05:09.318878+0000","last_undegraded":"2024-09-10T16:03:29.603815+0000","last_fullsized":"2024-09-10T15:59:15.774437+0000","mapping_epoch":42,"log_start":"43'4500","ondisk_log_start":"43'4500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24793088,"num_objects":6053,"num_object_clones":0,"num_object_copies":12106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":303,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6053,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10935,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3903,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5750}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":95,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:29.877831+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:02:29.877831+0000","last_peered":"2024-09-10T16:02:29.877831+0000","last_clean":"2024-09-10T16:02:29.877831+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:02:29.877831+0000","last_undegraded":"2024-09-10T16:02:29.877831+0000","last_fullsized":"2024-09-10T16:02:29.877831+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'14930","reported_seq":15073,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:09.253687+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:05:09.253687+0000","last_peered":"2024-09-10T16:05:09.253687+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:05:09.253687+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"43'4900","ondisk_log_start":"43'4900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25059328,"num_objects":6118,"num_object_clones":0,"num_object_copies":12236,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6118,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11185,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1747}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104463+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:02:08.104463+0000","last_peered":"2024-09-10T16:02:08.104463+0000","last_clean":"2024-09-10T16:02:08.104463+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:02:08.104463+0000","last_undegraded":"2024-09-10T16:02:08.104463+0000","last_fullsized":"2024-09-10T16:02:08.104463+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104448+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:02:08.104448+0000","last_peered":"2024-09-10T16:02:08.104448+0000","last_clean":"2024-09-10T16:02:08.104448+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:02:08.104448+0000","last_undegraded":"2024-09-10T16:02:08.104448+0000","last_fullsized":"2024-09-10T16:02:08.104448+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'14964","reported_seq":15107,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:09.267271+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:05:09.267271+0000","last_peered":"2024-09-10T16:05:09.267271+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:05:09.267271+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"43'4900","ondisk_log_start":"43'4900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25535430,"num_objects":6236,"num_object_clones":0,"num_object_copies":12472,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6236,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11127,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1913},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":401138422,"num_objects":97944,"num_object_clones":0,"num_object_copies":195888,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43214,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":97944,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":194162,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8175,"num_bytes_recovered":33095680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1014444032,"data_stored":1014294981,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160688,"ondisk_log_size":160688,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986262,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":653080,"kb_used_data":332616,"kb_used_omap":4,"kb_used_meta":320443,"kb_avail":93718760,"statfs":{"total":96636764160,"available":95968010240,"internally_reserved":0,"allocated":340598784,"data_stored":340306551,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":328134314},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,3,4,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659207,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":602980,"kb_used_data":296404,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":93768860,"statfs":{"total":96636764160,"available":96019312640,"internally_reserved":0,"allocated":303517696,"data_stored":303188869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":313913006},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,2,3,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182490,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":774944,"kb_used_data":364304,"kb_used_omap":4,"kb_used_meta":410619,"kb_avail":93596896,"statfs":{"total":96636764160,"available":95843221504,"internally_reserved":0,"allocated":373047296,"data_stored":372755058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":420474538},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,1,4,3,5,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":371322880,"data_stored":371286750,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":303038464,"data_stored":302965745,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":340082688,"data_stored":340042486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:05:15.932 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:05:16.511 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:05:16.513 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:05:16.535 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":278,"stamp":"2024-09-10T16:05:15.681595+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":391795465,"num_objects":95665,"num_object_clones":0,"num_object_copies":191330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43040,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":95665,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":196445,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8351,"num_bytes_recovered":33095699,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160872,"ondisk_log_size":160872,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2043504,"kb_used_data":975292,"kb_used_omap":12,"kb_used_meta":1068147,"kb_avail":281072016,"statfs":{"total":289910292480,"available":287817744384,"internally_reserved":0,"allocated":998699008,"data_stored":997785710,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1093782530},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,9,9,5,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-27365376,"num_objects":-6681,"num_object_clones":0,"num_object_copies":-13362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-265,"num_objects_misplaced":-224,"num_objects_unfound":0,"num_objects_dirty":-6681,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6681,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":489,"num_bytes_recovered":917504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006732"},"pg_stats":[{"pgid":"2.f","version":"45'14784","reported_seq":14860,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:14.304495+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:05:14.304495+0000","last_peered":"2024-09-10T16:05:14.304495+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:05:14.304495+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"43'4700","ondisk_log_start":"43'4700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24117248,"num_objects":5888,"num_object_clones":0,"num_object_copies":11776,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5888,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14784,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1629}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'14903","reported_seq":15017,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:05:15.295976+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:05:15.295976+0000","last_peered":"2024-09-10T16:05:15.295976+0000","last_clean":"2024-09-10T16:05:15.295976+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:05:15.295976+0000","last_undegraded":"2024-09-10T16:05:15.295976+0000","last_fullsized":"2024-09-10T16:05:15.295976+0000","mapping_epoch":35,"log_start":"43'4900","ondisk_log_start":"43'4900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24335302,"num_objects":5943,"num_object_clones":0,"num_object_copies":11886,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5943,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13008,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'14779","reported_seq":23458,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:05:12.168765+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:05:12.168765+0000","last_peered":"2024-09-10T16:05:12.168765+0000","last_clean":"2024-09-10T16:05:12.168765+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:05:12.168765+0000","last_undegraded":"2024-09-10T16:05:12.168765+0000","last_fullsized":"2024-09-10T16:05:12.168765+0000","mapping_epoch":44,"log_start":"43'4700","ondisk_log_start":"43'4700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24649752,"num_objects":6019,"num_object_clones":0,"num_object_copies":12038,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6019,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12783,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'14757","reported_seq":14839,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:05:15.279874+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:05:15.279874+0000","last_peered":"2024-09-10T16:05:15.279874+0000","last_clean":"2024-09-10T16:05:15.279874+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:05:15.279874+0000","last_undegraded":"2024-09-10T16:05:15.279874+0000","last_fullsized":"2024-09-10T16:05:15.279874+0000","mapping_epoch":35,"log_start":"43'4700","ondisk_log_start":"43'4700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24178688,"num_objects":5903,"num_object_clones":0,"num_object_copies":11806,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5903,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14757,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'14911","reported_seq":15042,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:05:14.257268+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:05:14.257268+0000","last_peered":"2024-09-10T16:05:14.257268+0000","last_clean":"2024-09-10T16:05:14.257268+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:05:14.257268+0000","last_undegraded":"2024-09-10T16:05:14.257268+0000","last_fullsized":"2024-09-10T16:05:14.257268+0000","mapping_epoch":34,"log_start":"43'4900","ondisk_log_start":"43'4900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24494563,"num_objects":5981,"num_object_clones":0,"num_object_copies":11962,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5981,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11622,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'15080","reported_seq":15219,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:14.338112+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:05:14.338112+0000","last_peered":"2024-09-10T16:05:14.338112+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:05:14.338112+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"43'5000","ondisk_log_start":"43'5000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24498659,"num_objects":5982,"num_object_clones":0,"num_object_copies":11964,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5982,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11692,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1611},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'14951","reported_seq":15087,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:14.305671+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:05:14.305671+0000","last_peered":"2024-09-10T16:05:14.305671+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:05:14.305671+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"43'4900","ondisk_log_start":"43'4900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25113542,"num_objects":6133,"num_object_clones":0,"num_object_copies":12266,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6133,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":11450,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1896},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'15185","reported_seq":15316,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:05:14.323753+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:05:14.323753+0000","last_peered":"2024-09-10T16:05:14.323753+0000","last_clean":"2024-09-10T16:05:14.323753+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:05:14.323753+0000","last_undegraded":"2024-09-10T16:05:14.323753+0000","last_fullsized":"2024-09-10T16:05:14.323753+0000","mapping_epoch":34,"log_start":"43'5100","ondisk_log_start":"43'5100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25104384,"num_objects":6129,"num_object_clones":0,"num_object_copies":12258,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6129,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12032,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104261+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:02:08.104261+0000","last_peered":"2024-09-10T16:02:08.104261+0000","last_clean":"2024-09-10T16:02:08.104261+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:02:08.104261+0000","last_undegraded":"2024-09-10T16:02:08.104261+0000","last_fullsized":"2024-09-10T16:02:08.104261+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'14562","reported_seq":14703,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:15.273446+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:05:15.273446+0000","last_peered":"2024-09-10T16:05:15.273446+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:05:15.273446+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"43'4500","ondisk_log_start":"43'4500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24146403,"num_objects":5896,"num_object_clones":0,"num_object_copies":11792,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5896,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12381,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":1773}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":95,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:29.860108+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:02:29.860108+0000","last_peered":"2024-09-10T16:02:29.860108+0000","last_clean":"2024-09-10T16:02:29.860108+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:02:29.860108+0000","last_undegraded":"2024-09-10T16:02:29.860108+0000","last_fullsized":"2024-09-10T16:02:29.860108+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'14890","reported_seq":15017,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:14.340630+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:05:14.340630+0000","last_peered":"2024-09-10T16:05:14.340630+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:05:14.340630+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"43'4800","ondisk_log_start":"43'4800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24657920,"num_objects":6020,"num_object_clones":0,"num_object_copies":12040,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6020,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11394,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1765}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":105,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:12.405385+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:02:12.405385+0000","last_peered":"2024-09-10T16:02:12.405385+0000","last_clean":"2024-09-10T16:02:12.405385+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:02:12.405385+0000","last_undegraded":"2024-09-10T16:02:12.405385+0000","last_fullsized":"2024-09-10T16:02:12.405385+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'14905","reported_seq":15041,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:15.293832+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:05:15.293832+0000","last_peered":"2024-09-10T16:05:15.293832+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:05:15.293832+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"43'4900","ondisk_log_start":"43'4900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24068579,"num_objects":5877,"num_object_clones":0,"num_object_copies":11754,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5877,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12585,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":1573}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104617+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:02:08.104617+0000","last_peered":"2024-09-10T16:02:08.104617+0000","last_clean":"2024-09-10T16:02:08.104617+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:02:08.104617+0000","last_undegraded":"2024-09-10T16:02:08.104617+0000","last_fullsized":"2024-09-10T16:02:08.104617+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'15170","reported_seq":15300,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:15.320591+0000","last_change":"2024-09-10T15:59:16.174113+0000","last_active":"2024-09-10T16:05:15.320591+0000","last_peered":"2024-09-10T16:05:15.320591+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:05:15.320591+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"43'5100","ondisk_log_start":"43'5100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24576000,"num_objects":6000,"num_object_clones":0,"num_object_copies":12000,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4354,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6000,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12724,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1648}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'14916","reported_seq":15052,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:14.305698+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:05:14.305698+0000","last_peered":"2024-09-10T16:05:14.305698+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:05:14.305698+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"43'4900","ondisk_log_start":"43'4900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24256512,"num_objects":5922,"num_object_clones":0,"num_object_copies":11844,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5922,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11547,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1618}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103316+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:02:08.103316+0000","last_peered":"2024-09-10T16:02:08.103316+0000","last_clean":"2024-09-10T16:02:08.103316+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:02:08.103316+0000","last_undegraded":"2024-09-10T16:02:08.103316+0000","last_fullsized":"2024-09-10T16:02:08.103316+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'14693","reported_seq":22992,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:14.363742+0000","last_change":"2024-09-10T16:03:29.605682+0000","last_active":"2024-09-10T16:05:14.363742+0000","last_peered":"2024-09-10T16:05:14.363742+0000","last_clean":"2024-09-10T15:59:14.115595+0000","last_became_active":"2024-09-10T15:59:16.244089+0000","last_became_peered":"2024-09-10T15:59:16.244089+0000","last_unstale":"2024-09-10T16:05:14.363742+0000","last_undegraded":"2024-09-10T16:03:29.603815+0000","last_fullsized":"2024-09-10T15:59:15.774437+0000","mapping_epoch":42,"log_start":"43'4600","ondisk_log_start":"43'4600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24186880,"num_objects":5905,"num_object_clones":0,"num_object_copies":11810,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":129,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5905,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11083,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4077,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5776}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":95,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:29.877831+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:02:29.877831+0000","last_peered":"2024-09-10T16:02:29.877831+0000","last_clean":"2024-09-10T16:02:29.877831+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:02:29.877831+0000","last_undegraded":"2024-09-10T16:02:29.877831+0000","last_fullsized":"2024-09-10T16:02:29.877831+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'15076","reported_seq":15217,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:14.251753+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:05:14.251753+0000","last_peered":"2024-09-10T16:05:14.251753+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:05:14.251753+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"43'5000","ondisk_log_start":"43'5000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24461312,"num_objects":5972,"num_object_clones":0,"num_object_copies":11944,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5972,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11331,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1601}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104463+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:02:08.104463+0000","last_peered":"2024-09-10T16:02:08.104463+0000","last_clean":"2024-09-10T16:02:08.104463+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:02:08.104463+0000","last_undegraded":"2024-09-10T16:02:08.104463+0000","last_fullsized":"2024-09-10T16:02:08.104463+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104448+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:02:08.104448+0000","last_peered":"2024-09-10T16:02:08.104448+0000","last_clean":"2024-09-10T16:02:08.104448+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:02:08.104448+0000","last_undegraded":"2024-09-10T16:02:08.104448+0000","last_fullsized":"2024-09-10T16:02:08.104448+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'15107","reported_seq":15252,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:14.311785+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:05:14.311785+0000","last_peered":"2024-09-10T16:05:14.311785+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:05:14.311785+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"43'5100","ondisk_log_start":"43'5100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24949702,"num_objects":6093,"num_object_clones":0,"num_object_copies":12186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6093,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11270,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1770},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":391795446,"num_objects":95663,"num_object_clones":0,"num_object_copies":191326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43040,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":95663,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":196443,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8349,"num_bytes_recovered":33095680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":996777984,"data_stored":996628933,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160869,"ondisk_log_size":160869,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986263,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":652768,"kb_used_data":328144,"kb_used_omap":4,"kb_used_meta":324603,"kb_avail":93719072,"statfs":{"total":96636764160,"available":95968329728,"internally_reserved":0,"allocated":336019456,"data_stored":335727223,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":332394154},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,0,5,1,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659208,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":619632,"kb_used_data":290844,"kb_used_omap":4,"kb_used_meta":328763,"kb_avail":93752208,"statfs":{"total":96636764160,"available":96002260992,"internally_reserved":0,"allocated":297824256,"data_stored":297495429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":336653998},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,3,2,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182491,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":771104,"kb_used_data":356304,"kb_used_omap":4,"kb_used_meta":414779,"kb_avail":93600736,"statfs":{"total":96636764160,"available":95847153664,"internally_reserved":0,"allocated":364855296,"data_stored":364563058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":424734378},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,7,1,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":364404736,"data_stored":364368606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":296935424,"data_stored":296862705,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":335437824,"data_stored":335397622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:05:16.536 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:05:17.123 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:05:17.124 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:05:17.147 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":278,"stamp":"2024-09-10T16:05:15.681595+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":391795465,"num_objects":95665,"num_object_clones":0,"num_object_copies":191330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43040,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":95665,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":196445,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8351,"num_bytes_recovered":33095699,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160872,"ondisk_log_size":160872,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2043504,"kb_used_data":975292,"kb_used_omap":12,"kb_used_meta":1068147,"kb_avail":281072016,"statfs":{"total":289910292480,"available":287817744384,"internally_reserved":0,"allocated":998699008,"data_stored":997785710,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1093782530},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,9,9,5,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-27365376,"num_objects":-6681,"num_object_clones":0,"num_object_copies":-13362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-265,"num_objects_misplaced":-224,"num_objects_unfound":0,"num_objects_dirty":-6681,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6681,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":489,"num_bytes_recovered":917504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006732"},"pg_stats":[{"pgid":"2.f","version":"45'14784","reported_seq":14860,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:14.304495+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:05:14.304495+0000","last_peered":"2024-09-10T16:05:14.304495+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:05:14.304495+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"43'4700","ondisk_log_start":"43'4700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24117248,"num_objects":5888,"num_object_clones":0,"num_object_copies":11776,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5888,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14784,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1629}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'14903","reported_seq":15017,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:05:15.295976+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:05:15.295976+0000","last_peered":"2024-09-10T16:05:15.295976+0000","last_clean":"2024-09-10T16:05:15.295976+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:05:15.295976+0000","last_undegraded":"2024-09-10T16:05:15.295976+0000","last_fullsized":"2024-09-10T16:05:15.295976+0000","mapping_epoch":35,"log_start":"43'4900","ondisk_log_start":"43'4900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24335302,"num_objects":5943,"num_object_clones":0,"num_object_copies":11886,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5943,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13008,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'14779","reported_seq":23458,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:05:12.168765+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:05:12.168765+0000","last_peered":"2024-09-10T16:05:12.168765+0000","last_clean":"2024-09-10T16:05:12.168765+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:05:12.168765+0000","last_undegraded":"2024-09-10T16:05:12.168765+0000","last_fullsized":"2024-09-10T16:05:12.168765+0000","mapping_epoch":44,"log_start":"43'4700","ondisk_log_start":"43'4700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24649752,"num_objects":6019,"num_object_clones":0,"num_object_copies":12038,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6019,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12783,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'14757","reported_seq":14839,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:05:15.279874+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:05:15.279874+0000","last_peered":"2024-09-10T16:05:15.279874+0000","last_clean":"2024-09-10T16:05:15.279874+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:05:15.279874+0000","last_undegraded":"2024-09-10T16:05:15.279874+0000","last_fullsized":"2024-09-10T16:05:15.279874+0000","mapping_epoch":35,"log_start":"43'4700","ondisk_log_start":"43'4700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24178688,"num_objects":5903,"num_object_clones":0,"num_object_copies":11806,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5903,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14757,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'14911","reported_seq":15042,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:05:14.257268+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:05:14.257268+0000","last_peered":"2024-09-10T16:05:14.257268+0000","last_clean":"2024-09-10T16:05:14.257268+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:05:14.257268+0000","last_undegraded":"2024-09-10T16:05:14.257268+0000","last_fullsized":"2024-09-10T16:05:14.257268+0000","mapping_epoch":34,"log_start":"43'4900","ondisk_log_start":"43'4900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24494563,"num_objects":5981,"num_object_clones":0,"num_object_copies":11962,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5981,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11622,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'15080","reported_seq":15219,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:14.338112+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:05:14.338112+0000","last_peered":"2024-09-10T16:05:14.338112+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:05:14.338112+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"43'5000","ondisk_log_start":"43'5000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24498659,"num_objects":5982,"num_object_clones":0,"num_object_copies":11964,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5982,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11692,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1611},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'14951","reported_seq":15087,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:14.305671+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:05:14.305671+0000","last_peered":"2024-09-10T16:05:14.305671+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:05:14.305671+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"43'4900","ondisk_log_start":"43'4900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25113542,"num_objects":6133,"num_object_clones":0,"num_object_copies":12266,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6133,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":11450,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1896},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'15185","reported_seq":15316,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:05:14.323753+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:05:14.323753+0000","last_peered":"2024-09-10T16:05:14.323753+0000","last_clean":"2024-09-10T16:05:14.323753+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:05:14.323753+0000","last_undegraded":"2024-09-10T16:05:14.323753+0000","last_fullsized":"2024-09-10T16:05:14.323753+0000","mapping_epoch":34,"log_start":"43'5100","ondisk_log_start":"43'5100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25104384,"num_objects":6129,"num_object_clones":0,"num_object_copies":12258,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6129,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12032,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104261+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:02:08.104261+0000","last_peered":"2024-09-10T16:02:08.104261+0000","last_clean":"2024-09-10T16:02:08.104261+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:02:08.104261+0000","last_undegraded":"2024-09-10T16:02:08.104261+0000","last_fullsized":"2024-09-10T16:02:08.104261+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'14562","reported_seq":14703,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:15.273446+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:05:15.273446+0000","last_peered":"2024-09-10T16:05:15.273446+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:05:15.273446+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"43'4500","ondisk_log_start":"43'4500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24146403,"num_objects":5896,"num_object_clones":0,"num_object_copies":11792,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5896,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12381,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":1773}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":95,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:29.860108+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:02:29.860108+0000","last_peered":"2024-09-10T16:02:29.860108+0000","last_clean":"2024-09-10T16:02:29.860108+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:02:29.860108+0000","last_undegraded":"2024-09-10T16:02:29.860108+0000","last_fullsized":"2024-09-10T16:02:29.860108+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'14890","reported_seq":15017,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:14.340630+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:05:14.340630+0000","last_peered":"2024-09-10T16:05:14.340630+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:05:14.340630+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"43'4800","ondisk_log_start":"43'4800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24657920,"num_objects":6020,"num_object_clones":0,"num_object_copies":12040,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6020,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11394,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1765}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":105,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:12.405385+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:02:12.405385+0000","last_peered":"2024-09-10T16:02:12.405385+0000","last_clean":"2024-09-10T16:02:12.405385+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:02:12.405385+0000","last_undegraded":"2024-09-10T16:02:12.405385+0000","last_fullsized":"2024-09-10T16:02:12.405385+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'14905","reported_seq":15041,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:15.293832+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:05:15.293832+0000","last_peered":"2024-09-10T16:05:15.293832+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:05:15.293832+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"43'4900","ondisk_log_start":"43'4900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24068579,"num_objects":5877,"num_object_clones":0,"num_object_copies":11754,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5877,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12585,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":1573}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104617+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:02:08.104617+0000","last_peered":"2024-09-10T16:02:08.104617+0000","last_clean":"2024-09-10T16:02:08.104617+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:02:08.104617+0000","last_undegraded":"2024-09-10T16:02:08.104617+0000","last_fullsized":"2024-09-10T16:02:08.104617+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'15170","reported_seq":15300,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:15.320591+0000","last_change":"2024-09-10T15:59:16.174113+0000","last_active":"2024-09-10T16:05:15.320591+0000","last_peered":"2024-09-10T16:05:15.320591+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:05:15.320591+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"43'5100","ondisk_log_start":"43'5100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24576000,"num_objects":6000,"num_object_clones":0,"num_object_copies":12000,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4354,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6000,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12724,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1648}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'14916","reported_seq":15052,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:14.305698+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:05:14.305698+0000","last_peered":"2024-09-10T16:05:14.305698+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:05:14.305698+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"43'4900","ondisk_log_start":"43'4900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24256512,"num_objects":5922,"num_object_clones":0,"num_object_copies":11844,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5922,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11547,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1618}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103316+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:02:08.103316+0000","last_peered":"2024-09-10T16:02:08.103316+0000","last_clean":"2024-09-10T16:02:08.103316+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:02:08.103316+0000","last_undegraded":"2024-09-10T16:02:08.103316+0000","last_fullsized":"2024-09-10T16:02:08.103316+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'14693","reported_seq":22992,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:14.363742+0000","last_change":"2024-09-10T16:03:29.605682+0000","last_active":"2024-09-10T16:05:14.363742+0000","last_peered":"2024-09-10T16:05:14.363742+0000","last_clean":"2024-09-10T15:59:14.115595+0000","last_became_active":"2024-09-10T15:59:16.244089+0000","last_became_peered":"2024-09-10T15:59:16.244089+0000","last_unstale":"2024-09-10T16:05:14.363742+0000","last_undegraded":"2024-09-10T16:03:29.603815+0000","last_fullsized":"2024-09-10T15:59:15.774437+0000","mapping_epoch":42,"log_start":"43'4600","ondisk_log_start":"43'4600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24186880,"num_objects":5905,"num_object_clones":0,"num_object_copies":11810,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":129,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5905,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11083,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4077,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5776}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":95,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:29.877831+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:02:29.877831+0000","last_peered":"2024-09-10T16:02:29.877831+0000","last_clean":"2024-09-10T16:02:29.877831+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:02:29.877831+0000","last_undegraded":"2024-09-10T16:02:29.877831+0000","last_fullsized":"2024-09-10T16:02:29.877831+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'15076","reported_seq":15217,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:14.251753+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:05:14.251753+0000","last_peered":"2024-09-10T16:05:14.251753+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:05:14.251753+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"43'5000","ondisk_log_start":"43'5000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24461312,"num_objects":5972,"num_object_clones":0,"num_object_copies":11944,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5972,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11331,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1601}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104463+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:02:08.104463+0000","last_peered":"2024-09-10T16:02:08.104463+0000","last_clean":"2024-09-10T16:02:08.104463+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:02:08.104463+0000","last_undegraded":"2024-09-10T16:02:08.104463+0000","last_fullsized":"2024-09-10T16:02:08.104463+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104448+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:02:08.104448+0000","last_peered":"2024-09-10T16:02:08.104448+0000","last_clean":"2024-09-10T16:02:08.104448+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:02:08.104448+0000","last_undegraded":"2024-09-10T16:02:08.104448+0000","last_fullsized":"2024-09-10T16:02:08.104448+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'15107","reported_seq":15252,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:14.311785+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:05:14.311785+0000","last_peered":"2024-09-10T16:05:14.311785+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:05:14.311785+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"43'5100","ondisk_log_start":"43'5100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24949702,"num_objects":6093,"num_object_clones":0,"num_object_copies":12186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6093,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11270,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1770},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":391795446,"num_objects":95663,"num_object_clones":0,"num_object_copies":191326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43040,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":95663,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":196443,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8349,"num_bytes_recovered":33095680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":996777984,"data_stored":996628933,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160869,"ondisk_log_size":160869,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986263,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":652768,"kb_used_data":328144,"kb_used_omap":4,"kb_used_meta":324603,"kb_avail":93719072,"statfs":{"total":96636764160,"available":95968329728,"internally_reserved":0,"allocated":336019456,"data_stored":335727223,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":332394154},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,0,5,1,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659208,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":619632,"kb_used_data":290844,"kb_used_omap":4,"kb_used_meta":328763,"kb_avail":93752208,"statfs":{"total":96636764160,"available":96002260992,"internally_reserved":0,"allocated":297824256,"data_stored":297495429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":336653998},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,3,2,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182491,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":771104,"kb_used_data":356304,"kb_used_omap":4,"kb_used_meta":414779,"kb_avail":93600736,"statfs":{"total":96636764160,"available":95847153664,"internally_reserved":0,"allocated":364855296,"data_stored":364563058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":424734378},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,7,1,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":364404736,"data_stored":364368606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":296935424,"data_stored":296862705,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":335437824,"data_stored":335397622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:05:17.148 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-10T16:05:17.148 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "45'14784", 'reported_seq': 14860, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:05:14.304495+0000', 'last_change': '2024-09-10T15:59:16.281558+0000', 'last_active': '2024-09-10T16:05:14.304495+0000', 'last_peered': '2024-09-10T16:05:14.304495+0000', 'last_clean': '2024-09-10T15:59:14.132456+0000', 'last_became_active': '2024-09-10T15:59:16.274407+0000', 'last_became_peered': '2024-09-10T15:59:16.274407+0000', 'last_unstale': '2024-09-10T16:05:14.304495+0000', 'last_undegraded': '2024-09-10T15:59:15.851279+0000', 'last_fullsized': '2024-09-10T15:59:15.774356+0000', 'mapping_epoch': 42, 'log_start': "43'4700", 'ondisk_log_start': "43'4700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'objects_scrubbed': 0, 'log_size': 10084, 'log_dups_size': 0, 'ondisk_log_size': 10084, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24117248, 'num_objects': 5888, 'num_object_clones': 0, 'num_object_copies': 11776, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4259, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5888, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14784, 'num_write_kb': 41344, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1629}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:05:17.148 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-10T16:05:17.148 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "45'15080", 'reported_seq': 15219, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:05:14.338112+0000', 'last_change': '2024-09-10T15:59:16.308264+0000', 'last_active': '2024-09-10T16:05:14.338112+0000', 'last_peered': '2024-09-10T16:05:14.338112+0000', 'last_clean': '2024-09-10T15:59:14.111551+0000', 'last_became_active': '2024-09-10T15:59:16.300040+0000', 'last_became_peered': '2024-09-10T15:59:16.300040+0000', 'last_unstale': '2024-09-10T16:05:14.338112+0000', 'last_undegraded': '2024-09-10T15:59:15.878234+0000', 'last_fullsized': '2024-09-10T15:59:15.769328+0000', 'mapping_epoch': 42, 'log_start': "43'5000", 'ondisk_log_start': "43'5000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3442", 'last_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'objects_scrubbed': 3384, 'log_size': 10080, 'log_dups_size': 0, 'ondisk_log_size': 10080, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1292, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24498659, 'num_objects': 5982, 'num_object_clones': 0, 'num_object_copies': 11964, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4372, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5982, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 11692, 'num_write_kb': 28580, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1611}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:05:17.149 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-10T16:05:17.149 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "45'14951", 'reported_seq': 15087, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:05:14.305671+0000', 'last_change': '2024-09-10T15:59:16.331899+0000', 'last_active': '2024-09-10T16:05:14.305671+0000', 'last_peered': '2024-09-10T16:05:14.305671+0000', 'last_clean': '2024-09-10T15:59:14.139436+0000', 'last_became_active': '2024-09-10T15:59:16.318715+0000', 'last_became_peered': '2024-09-10T15:59:16.318715+0000', 'last_unstale': '2024-09-10T16:05:14.305671+0000', 'last_undegraded': '2024-09-10T15:59:15.877524+0000', 'last_fullsized': '2024-09-10T15:59:15.769042+0000', 'mapping_epoch': 42, 'log_start': "43'4900", 'ondisk_log_start': "43'4900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3537", 'last_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'objects_scrubbed': 3497, 'log_size': 10051, 'log_dups_size': 0, 'ondisk_log_size': 10051, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1224, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25113542, 'num_objects': 6133, 'num_object_clones': 0, 'num_object_copies': 12266, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4237, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6133, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 11450, 'num_write_kb': 28172, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1896}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:05:17.149 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-10T16:05:17.149 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "45'14562", 'reported_seq': 14703, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:05:15.273446+0000', 'last_change': '2024-09-10T15:59:16.175680+0000', 'last_active': '2024-09-10T16:05:15.273446+0000', 'last_peered': '2024-09-10T16:05:15.273446+0000', 'last_clean': '2024-09-10T15:59:14.088187+0000', 'last_became_active': '2024-09-10T15:59:16.165810+0000', 'last_became_peered': '2024-09-10T15:59:16.165810+0000', 'last_unstale': '2024-09-10T16:05:15.273446+0000', 'last_undegraded': '2024-09-10T15:59:15.822098+0000', 'last_fullsized': '2024-09-10T15:59:15.769123+0000', 'mapping_epoch': 42, 'log_start': "43'4500", 'ondisk_log_start': "43'4500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2218", 'last_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'objects_scrubbed': 2175, 'log_size': 10062, 'log_dups_size': 0, 'ondisk_log_size': 10062, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 825, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24146403, 'num_objects': 5896, 'num_object_clones': 0, 'num_object_copies': 11792, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4124, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5896, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 12381, 'num_write_kb': 32204, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': 1773}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:05:17.149 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-10T16:05:17.149 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "45'14890", 'reported_seq': 15017, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:05:14.340630+0000', 'last_change': '2024-09-10T15:59:16.277742+0000', 'last_active': '2024-09-10T16:05:14.340630+0000', 'last_peered': '2024-09-10T16:05:14.340630+0000', 'last_clean': '2024-09-10T15:59:14.129081+0000', 'last_became_active': '2024-09-10T15:59:16.269835+0000', 'last_became_peered': '2024-09-10T15:59:16.269835+0000', 'last_unstale': '2024-09-10T16:05:14.340630+0000', 'last_undegraded': '2024-09-10T15:59:15.860857+0000', 'last_fullsized': '2024-09-10T15:59:15.771410+0000', 'mapping_epoch': 42, 'log_start': "43'4800", 'ondisk_log_start': "43'4800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3557", 'last_scrub_stamp': '2024-09-10T15:59:02.493678+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:02.493678+0000', 'objects_scrubbed': 3488, 'log_size': 10090, 'log_dups_size': 0, 'ondisk_log_size': 10090, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1472, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24657920, 'num_objects': 6020, 'num_object_clones': 0, 'num_object_copies': 12040, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4257, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6020, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11394, 'num_write_kb': 27852, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1765}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:05:17.149 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-10T16:05:17.149 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "45'14905", 'reported_seq': 15041, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:05:15.293832+0000', 'last_change': '2024-09-10T15:59:16.284311+0000', 'last_active': '2024-09-10T16:05:15.293832+0000', 'last_peered': '2024-09-10T16:05:15.293832+0000', 'last_clean': '2024-09-10T15:59:14.116041+0000', 'last_became_active': '2024-09-10T15:59:16.274543+0000', 'last_became_peered': '2024-09-10T15:59:16.274543+0000', 'last_unstale': '2024-09-10T16:05:15.293832+0000', 'last_undegraded': '2024-09-10T15:59:15.812484+0000', 'last_fullsized': '2024-09-10T15:59:15.770885+0000', 'mapping_epoch': 42, 'log_start': "43'4900", 'ondisk_log_start': "43'4900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2356", 'last_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'objects_scrubbed': 2316, 'log_size': 10005, 'log_dups_size': 0, 'ondisk_log_size': 10005, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 783, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24068579, 'num_objects': 5877, 'num_object_clones': 0, 'num_object_copies': 11754, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5877, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 12585, 'num_write_kb': 32292, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': 1573}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:05:17.149 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-10T16:05:17.149 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "45'15170", 'reported_seq': 15300, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:05:15.320591+0000', 'last_change': '2024-09-10T15:59:16.174113+0000', 'last_active': '2024-09-10T16:05:15.320591+0000', 'last_peered': '2024-09-10T16:05:15.320591+0000', 'last_clean': '2024-09-10T15:59:14.110654+0000', 'last_became_active': '2024-09-10T15:59:16.164645+0000', 'last_became_peered': '2024-09-10T15:59:16.164645+0000', 'last_unstale': '2024-09-10T16:05:15.320591+0000', 'last_undegraded': '2024-09-10T15:59:15.803593+0000', 'last_fullsized': '2024-09-10T15:59:15.769083+0000', 'mapping_epoch': 42, 'log_start': "43'5100", 'ondisk_log_start': "43'5100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2486", 'last_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'objects_scrubbed': 2440, 'log_size': 10070, 'log_dups_size': 0, 'ondisk_log_size': 10070, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 792, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24576000, 'num_objects': 6000, 'num_object_clones': 0, 'num_object_copies': 12000, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4354, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6000, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12724, 'num_write_kb': 32568, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 1648}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:05:17.150 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-10T16:05:17.150 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "45'14916", 'reported_seq': 15052, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:05:14.305698+0000', 'last_change': '2024-09-10T15:59:16.315641+0000', 'last_active': '2024-09-10T16:05:14.305698+0000', 'last_peered': '2024-09-10T16:05:14.305698+0000', 'last_clean': '2024-09-10T15:59:14.117001+0000', 'last_became_active': '2024-09-10T15:59:16.302245+0000', 'last_became_peered': '2024-09-10T15:59:16.302245+0000', 'last_unstale': '2024-09-10T16:05:14.305698+0000', 'last_undegraded': '2024-09-10T15:59:15.845012+0000', 'last_fullsized': '2024-09-10T15:59:15.770337+0000', 'mapping_epoch': 42, 'log_start': "43'4900", 'ondisk_log_start': "43'4900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3404", 'last_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'objects_scrubbed': 3367, 'log_size': 10016, 'log_dups_size': 0, 'ondisk_log_size': 10016, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 983, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24256512, 'num_objects': 5922, 'num_object_clones': 0, 'num_object_copies': 11844, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5922, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11547, 'num_write_kb': 28204, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1618}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:05:17.150 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-10T16:05:17.150 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "45'14693", 'reported_seq': 22992, 'reported_epoch': 45, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:05:14.363742+0000', 'last_change': '2024-09-10T16:03:29.605682+0000', 'last_active': '2024-09-10T16:05:14.363742+0000', 'last_peered': '2024-09-10T16:05:14.363742+0000', 'last_clean': '2024-09-10T15:59:14.115595+0000', 'last_became_active': '2024-09-10T15:59:16.244089+0000', 'last_became_peered': '2024-09-10T15:59:16.244089+0000', 'last_unstale': '2024-09-10T16:05:14.363742+0000', 'last_undegraded': '2024-09-10T16:03:29.603815+0000', 'last_fullsized': '2024-09-10T15:59:15.774437+0000', 'mapping_epoch': 42, 'log_start': "43'4600", 'ondisk_log_start': "43'4600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3653", 'last_scrub_stamp': '2024-09-10T15:59:05.298468+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:05.298468+0000', 'objects_scrubbed': 3602, 'log_size': 10093, 'log_dups_size': 0, 'ondisk_log_size': 10093, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1289, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24186880, 'num_objects': 5905, 'num_object_clones': 0, 'num_object_copies': 11810, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 129, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5905, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11083, 'num_write_kb': 26772, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 4077, 'num_bytes_recovered': 15601664, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5776}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:05:17.150 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-10T16:05:17.150 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "45'15076", 'reported_seq': 15217, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:05:14.251753+0000', 'last_change': '2024-09-10T15:59:16.311291+0000', 'last_active': '2024-09-10T16:05:14.251753+0000', 'last_peered': '2024-09-10T16:05:14.251753+0000', 'last_clean': '2024-09-10T15:59:14.116363+0000', 'last_became_active': '2024-09-10T15:59:16.301244+0000', 'last_became_peered': '2024-09-10T15:59:16.301244+0000', 'last_unstale': '2024-09-10T16:05:14.251753+0000', 'last_undegraded': '2024-09-10T15:59:15.883434+0000', 'last_fullsized': '2024-09-10T15:59:15.772756+0000', 'mapping_epoch': 42, 'log_start': "43'5000", 'ondisk_log_start': "43'5000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3789", 'last_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'objects_scrubbed': 3741, 'log_size': 10076, 'log_dups_size': 0, 'ondisk_log_size': 10076, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1440, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24461312, 'num_objects': 5972, 'num_object_clones': 0, 'num_object_copies': 11944, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4373, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5972, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11331, 'num_write_kb': 27124, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1601}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:05:17.150 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-10T16:05:17.150 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "45'15107", 'reported_seq': 15252, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:05:14.311785+0000', 'last_change': '2024-09-10T15:59:16.313409+0000', 'last_active': '2024-09-10T16:05:14.311785+0000', 'last_peered': '2024-09-10T16:05:14.311785+0000', 'last_clean': '2024-09-10T15:59:14.129780+0000', 'last_became_active': '2024-09-10T15:59:16.300932+0000', 'last_became_peered': '2024-09-10T15:59:16.300932+0000', 'last_unstale': '2024-09-10T16:05:14.311785+0000', 'last_undegraded': '2024-09-10T15:59:15.887823+0000', 'last_fullsized': '2024-09-10T15:59:15.768513+0000', 'mapping_epoch': 42, 'log_start': "43'5100", 'ondisk_log_start': "43'5100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3880", 'last_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'objects_scrubbed': 3827, 'log_size': 10007, 'log_dups_size': 0, 'ondisk_log_size': 10007, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1397, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24949702, 'num_objects': 6093, 'num_object_clones': 0, 'num_object_copies': 12186, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4325, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6093, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11270, 'num_write_kb': 27072, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1770}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:05:17.150 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T16:05:17.849 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:05:17.849 INFO:teuthology.orchestra.run.smithi115.stdout:{"fsid":"7effee29-df05-4351-9741-ee203564ed81","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 43040/190984 objects degraded (22.536%), 11 pgs degraded, 11 pgs undersized","count":22},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":481,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983954,"num_in_osds":3,"osd_in_since":1725983947,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":95492,"data_bytes":391086857,"bytes_used":2076651520,"bytes_avail":287833640960,"bytes_total":289910292480,"degraded_objects":43040,"degraded_total":190984,"degraded_ratio":0.22535919239308005,"recovering_objects_per_sec":27,"recovering_bytes_per_sec":22520,"recovering_keys_per_sec":0,"num_objects_recovered":331,"num_bytes_recovered":270336,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":393},"fsmap":{"epoch":1,"btime":"2024-09-10T15:57:14:967120+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T16:00:13.560776+0000","services":{}},"progress_events":{}} 2024-09-10T16:05:17.869 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-10T16:05:17.869 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:05:18.415 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:05:18.415 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:05:18.443 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":279,"stamp":"2024-09-10T16:05:17.682389+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":391086857,"num_objects":95492,"num_object_clones":0,"num_object_copies":190984,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43040,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":95492,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":196618,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8351,"num_bytes_recovered":33095699,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160845,"ondisk_log_size":160845,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2027980,"kb_used_data":968088,"kb_used_omap":12,"kb_used_meta":1059827,"kb_avail":281087540,"statfs":{"total":289910292480,"available":287833640960,"internally_reserved":0,"allocated":991322112,"data_stored":990408814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1085262850},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,4,9,8,5,7],"upper_bound":128},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19365888,"num_objects":-4728,"num_object_clones":0,"num_object_copies":-9456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-265,"num_objects_misplaced":-66,"num_objects_unfound":0,"num_objects_dirty":-4728,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4728,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":331,"num_bytes_recovered":270336,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004224"},"pg_stats":[{"pgid":"2.f","version":"45'14784","reported_seq":14860,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:14.304495+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:05:14.304495+0000","last_peered":"2024-09-10T16:05:14.304495+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:05:14.304495+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"43'4700","ondisk_log_start":"43'4700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24117248,"num_objects":5888,"num_object_clones":0,"num_object_copies":11776,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5888,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14784,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1629}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'14903","reported_seq":15017,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:05:15.295976+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:05:15.295976+0000","last_peered":"2024-09-10T16:05:15.295976+0000","last_clean":"2024-09-10T16:05:15.295976+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:05:15.295976+0000","last_undegraded":"2024-09-10T16:05:15.295976+0000","last_fullsized":"2024-09-10T16:05:15.295976+0000","mapping_epoch":35,"log_start":"43'4900","ondisk_log_start":"43'4900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24335302,"num_objects":5943,"num_object_clones":0,"num_object_copies":11886,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5943,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13008,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'14952","reported_seq":23631,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:05:17.264935+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:05:17.264935+0000","last_peered":"2024-09-10T16:05:17.264935+0000","last_clean":"2024-09-10T16:05:17.264935+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:05:17.264935+0000","last_undegraded":"2024-09-10T16:05:17.264935+0000","last_fullsized":"2024-09-10T16:05:17.264935+0000","mapping_epoch":44,"log_start":"43'4900","ondisk_log_start":"43'4900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23941144,"num_objects":5846,"num_object_clones":0,"num_object_copies":11692,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5846,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12956,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'14757","reported_seq":14839,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:05:15.279874+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:05:15.279874+0000","last_peered":"2024-09-10T16:05:15.279874+0000","last_clean":"2024-09-10T16:05:15.279874+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:05:15.279874+0000","last_undegraded":"2024-09-10T16:05:15.279874+0000","last_fullsized":"2024-09-10T16:05:15.279874+0000","mapping_epoch":35,"log_start":"43'4700","ondisk_log_start":"43'4700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24178688,"num_objects":5903,"num_object_clones":0,"num_object_copies":11806,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5903,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14757,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'14911","reported_seq":15042,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:05:14.257268+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:05:14.257268+0000","last_peered":"2024-09-10T16:05:14.257268+0000","last_clean":"2024-09-10T16:05:14.257268+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:05:14.257268+0000","last_undegraded":"2024-09-10T16:05:14.257268+0000","last_fullsized":"2024-09-10T16:05:14.257268+0000","mapping_epoch":34,"log_start":"43'4900","ondisk_log_start":"43'4900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24494563,"num_objects":5981,"num_object_clones":0,"num_object_copies":11962,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5981,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11622,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'15080","reported_seq":15219,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:14.338112+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:05:14.338112+0000","last_peered":"2024-09-10T16:05:14.338112+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:05:14.338112+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"43'5000","ondisk_log_start":"43'5000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24498659,"num_objects":5982,"num_object_clones":0,"num_object_copies":11964,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5982,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11692,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1611},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'14951","reported_seq":15087,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:14.305671+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:05:14.305671+0000","last_peered":"2024-09-10T16:05:14.305671+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:05:14.305671+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"43'4900","ondisk_log_start":"43'4900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25113542,"num_objects":6133,"num_object_clones":0,"num_object_copies":12266,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6133,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":11450,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1896},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'15185","reported_seq":15316,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:05:14.323753+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:05:14.323753+0000","last_peered":"2024-09-10T16:05:14.323753+0000","last_clean":"2024-09-10T16:05:14.323753+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:05:14.323753+0000","last_undegraded":"2024-09-10T16:05:14.323753+0000","last_fullsized":"2024-09-10T16:05:14.323753+0000","mapping_epoch":34,"log_start":"43'5100","ondisk_log_start":"43'5100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25104384,"num_objects":6129,"num_object_clones":0,"num_object_copies":12258,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6129,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12032,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":71,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:03:15.844822+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:03:15.844822+0000","last_peered":"2024-09-10T16:03:15.844822+0000","last_clean":"2024-09-10T16:03:15.844822+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:03:15.844822+0000","last_undegraded":"2024-09-10T16:03:15.844822+0000","last_fullsized":"2024-09-10T16:03:15.844822+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'14562","reported_seq":14703,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:15.273446+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:05:15.273446+0000","last_peered":"2024-09-10T16:05:15.273446+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:05:15.273446+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"43'4500","ondisk_log_start":"43'4500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24146403,"num_objects":5896,"num_object_clones":0,"num_object_copies":11792,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5896,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12381,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":1773}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":95,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:29.860108+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:02:29.860108+0000","last_peered":"2024-09-10T16:02:29.860108+0000","last_clean":"2024-09-10T16:02:29.860108+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:02:29.860108+0000","last_undegraded":"2024-09-10T16:02:29.860108+0000","last_fullsized":"2024-09-10T16:02:29.860108+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'14890","reported_seq":15017,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:14.340630+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:05:14.340630+0000","last_peered":"2024-09-10T16:05:14.340630+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:05:14.340630+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"43'4800","ondisk_log_start":"43'4800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24657920,"num_objects":6020,"num_object_clones":0,"num_object_copies":12040,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6020,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11394,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1765}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":105,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:12.405385+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:02:12.405385+0000","last_peered":"2024-09-10T16:02:12.405385+0000","last_clean":"2024-09-10T16:02:12.405385+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:02:12.405385+0000","last_undegraded":"2024-09-10T16:02:12.405385+0000","last_fullsized":"2024-09-10T16:02:12.405385+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'14905","reported_seq":15041,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:15.293832+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:05:15.293832+0000","last_peered":"2024-09-10T16:05:15.293832+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:05:15.293832+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"43'4900","ondisk_log_start":"43'4900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24068579,"num_objects":5877,"num_object_clones":0,"num_object_copies":11754,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5877,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12585,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":1573}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104617+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:02:08.104617+0000","last_peered":"2024-09-10T16:02:08.104617+0000","last_clean":"2024-09-10T16:02:08.104617+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:02:08.104617+0000","last_undegraded":"2024-09-10T16:02:08.104617+0000","last_fullsized":"2024-09-10T16:02:08.104617+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'15170","reported_seq":15300,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:15.320591+0000","last_change":"2024-09-10T15:59:16.174113+0000","last_active":"2024-09-10T16:05:15.320591+0000","last_peered":"2024-09-10T16:05:15.320591+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:05:15.320591+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"43'5100","ondisk_log_start":"43'5100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24576000,"num_objects":6000,"num_object_clones":0,"num_object_copies":12000,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4354,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6000,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12724,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1648}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'14916","reported_seq":15052,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:14.305698+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:05:14.305698+0000","last_peered":"2024-09-10T16:05:14.305698+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:05:14.305698+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"43'4900","ondisk_log_start":"43'4900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24256512,"num_objects":5922,"num_object_clones":0,"num_object_copies":11844,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5922,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11547,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1618}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103316+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:02:08.103316+0000","last_peered":"2024-09-10T16:02:08.103316+0000","last_clean":"2024-09-10T16:02:08.103316+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:02:08.103316+0000","last_undegraded":"2024-09-10T16:02:08.103316+0000","last_fullsized":"2024-09-10T16:02:08.103316+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'14693","reported_seq":22992,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:14.363742+0000","last_change":"2024-09-10T16:03:29.605682+0000","last_active":"2024-09-10T16:05:14.363742+0000","last_peered":"2024-09-10T16:05:14.363742+0000","last_clean":"2024-09-10T15:59:14.115595+0000","last_became_active":"2024-09-10T15:59:16.244089+0000","last_became_peered":"2024-09-10T15:59:16.244089+0000","last_unstale":"2024-09-10T16:05:14.363742+0000","last_undegraded":"2024-09-10T16:03:29.603815+0000","last_fullsized":"2024-09-10T15:59:15.774437+0000","mapping_epoch":42,"log_start":"43'4600","ondisk_log_start":"43'4600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24186880,"num_objects":5905,"num_object_clones":0,"num_object_copies":11810,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":129,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5905,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11083,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4077,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5776}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":95,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:29.877831+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:02:29.877831+0000","last_peered":"2024-09-10T16:02:29.877831+0000","last_clean":"2024-09-10T16:02:29.877831+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:02:29.877831+0000","last_undegraded":"2024-09-10T16:02:29.877831+0000","last_fullsized":"2024-09-10T16:02:29.877831+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'15076","reported_seq":15217,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:14.251753+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:05:14.251753+0000","last_peered":"2024-09-10T16:05:14.251753+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:05:14.251753+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"43'5000","ondisk_log_start":"43'5000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24461312,"num_objects":5972,"num_object_clones":0,"num_object_copies":11944,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5972,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11331,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1601}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104463+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:02:08.104463+0000","last_peered":"2024-09-10T16:02:08.104463+0000","last_clean":"2024-09-10T16:02:08.104463+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:02:08.104463+0000","last_undegraded":"2024-09-10T16:02:08.104463+0000","last_fullsized":"2024-09-10T16:02:08.104463+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104448+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:02:08.104448+0000","last_peered":"2024-09-10T16:02:08.104448+0000","last_clean":"2024-09-10T16:02:08.104448+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:02:08.104448+0000","last_undegraded":"2024-09-10T16:02:08.104448+0000","last_fullsized":"2024-09-10T16:02:08.104448+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'15107","reported_seq":15252,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:14.311785+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:05:14.311785+0000","last_peered":"2024-09-10T16:05:14.311785+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:05:14.311785+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"43'5100","ondisk_log_start":"43'5100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24949702,"num_objects":6093,"num_object_clones":0,"num_object_copies":12186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6093,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11270,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1770},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":391086838,"num_objects":95490,"num_object_clones":0,"num_object_copies":190980,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43040,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":95490,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":196616,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8349,"num_bytes_recovered":33095680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":989470720,"data_stored":989321669,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160842,"ondisk_log_size":160842,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986263,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":652768,"kb_used_data":328144,"kb_used_omap":4,"kb_used_meta":324603,"kb_avail":93719072,"statfs":{"total":96636764160,"available":95968329728,"internally_reserved":0,"allocated":336019456,"data_stored":335727223,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":332394154},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,0,5,1,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659209,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":604108,"kb_used_data":283640,"kb_used_omap":4,"kb_used_meta":320443,"kb_avail":93767732,"statfs":{"total":96636764160,"available":96018157568,"internally_reserved":0,"allocated":290447360,"data_stored":290118533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":328134318},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,2,2,2,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182491,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":771104,"kb_used_data":356304,"kb_used_omap":4,"kb_used_meta":414779,"kb_avail":93600736,"statfs":{"total":96636764160,"available":95847153664,"internally_reserved":0,"allocated":364855296,"data_stored":364563058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":424734378},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,7,1,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":364404736,"data_stored":364368606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":289628160,"data_stored":289555441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":335437824,"data_stored":335397622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:05:21.446 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:05:21.979 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:05:21.980 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:05:22.003 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":283,"stamp":"2024-09-10T16:05:21.683507+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":383247113,"num_objects":93578,"num_object_clones":0,"num_object_copies":187156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42950,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":93578,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":198532,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8441,"num_bytes_recovered":33095699,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160759,"ondisk_log_size":160759,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2013144,"kb_used_data":957404,"kb_used_omap":12,"kb_used_meta":1055667,"kb_avail":281102376,"statfs":{"total":289910292480,"available":287848833024,"internally_reserved":0,"allocated":980381696,"data_stored":979452732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1081003010},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,4,2,8,10,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-17313792,"num_objects":-4227,"num_object_clones":0,"num_object_copies":-8454,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-264,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4227,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4227,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":264,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002550"},"pg_stats":[{"pgid":"2.f","version":"46'14918","reported_seq":14996,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:19.314042+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:05:19.314042+0000","last_peered":"2024-09-10T16:05:19.314042+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:05:19.314042+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"43'4900","ondisk_log_start":"43'4900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23568384,"num_objects":5754,"num_object_clones":0,"num_object_copies":11508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14918,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1495}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'15030","reported_seq":15148,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:20.302012+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:05:20.302012+0000","last_peered":"2024-09-10T16:05:20.302012+0000","last_clean":"2024-09-10T16:05:20.302012+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:05:20.302012+0000","last_undegraded":"2024-09-10T16:05:20.302012+0000","last_fullsized":"2024-09-10T16:05:20.302012+0000","mapping_epoch":35,"log_start":"43'5000","ondisk_log_start":"43'5000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23815110,"num_objects":5816,"num_object_clones":0,"num_object_copies":11632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5816,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13135,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'14952","reported_seq":23631,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:05:17.264935+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:05:17.264935+0000","last_peered":"2024-09-10T16:05:17.264935+0000","last_clean":"2024-09-10T16:05:17.264935+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:05:17.264935+0000","last_undegraded":"2024-09-10T16:05:17.264935+0000","last_fullsized":"2024-09-10T16:05:17.264935+0000","mapping_epoch":44,"log_start":"43'4900","ondisk_log_start":"43'4900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23941144,"num_objects":5846,"num_object_clones":0,"num_object_copies":11692,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5846,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12956,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'14886","reported_seq":14972,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:20.262096+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:05:20.262096+0000","last_peered":"2024-09-10T16:05:20.262096+0000","last_clean":"2024-09-10T16:05:20.262096+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:05:20.262096+0000","last_undegraded":"2024-09-10T16:05:20.262096+0000","last_fullsized":"2024-09-10T16:05:20.262096+0000","mapping_epoch":35,"log_start":"43'4800","ondisk_log_start":"43'4800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23650304,"num_objects":5774,"num_object_clones":0,"num_object_copies":11548,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5774,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14886,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"46'15047","reported_seq":15180,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.193698+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:05:19.193698+0000","last_peered":"2024-09-10T16:05:19.193698+0000","last_clean":"2024-09-10T16:05:19.193698+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:05:19.193698+0000","last_undegraded":"2024-09-10T16:05:19.193698+0000","last_fullsized":"2024-09-10T16:05:19.193698+0000","mapping_epoch":34,"log_start":"43'5000","ondisk_log_start":"43'5000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23937507,"num_objects":5845,"num_object_clones":0,"num_object_copies":11690,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5845,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11758,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"46'15224","reported_seq":15365,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:19.335536+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:05:19.335536+0000","last_peered":"2024-09-10T16:05:19.335536+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:05:19.335536+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"43'5200","ondisk_log_start":"43'5200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23908835,"num_objects":5838,"num_object_clones":0,"num_object_copies":11676,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5838,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11836,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1467},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"46'15078","reported_seq":15219,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:19.322762+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:05:19.322762+0000","last_peered":"2024-09-10T16:05:19.322762+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:05:19.322762+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"43'5000","ondisk_log_start":"43'5000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24593350,"num_objects":6006,"num_object_clones":0,"num_object_copies":12012,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6006,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":11577,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1769},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"46'15331","reported_seq":15464,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.354786+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:05:19.354786+0000","last_peered":"2024-09-10T16:05:19.354786+0000","last_clean":"2024-09-10T16:05:19.354786+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:05:19.354786+0000","last_undegraded":"2024-09-10T16:05:19.354786+0000","last_fullsized":"2024-09-10T16:05:19.354786+0000","mapping_epoch":34,"log_start":"43'5300","ondisk_log_start":"43'5300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24506368,"num_objects":5983,"num_object_clones":0,"num_object_copies":11966,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5983,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12178,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":71,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:03:15.844822+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:03:15.844822+0000","last_peered":"2024-09-10T16:03:15.844822+0000","last_clean":"2024-09-10T16:03:15.844822+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:03:15.844822+0000","last_undegraded":"2024-09-10T16:03:15.844822+0000","last_fullsized":"2024-09-10T16:03:15.844822+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'14680","reported_seq":14825,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:20.256986+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:05:20.256986+0000","last_peered":"2024-09-10T16:05:20.256986+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:05:20.256986+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"43'4600","ondisk_log_start":"43'4600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23663075,"num_objects":5778,"num_object_clones":0,"num_object_copies":11556,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5778,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12499,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":1655}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":99,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.875776+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:05:19.875776+0000","last_peered":"2024-09-10T16:05:19.875776+0000","last_clean":"2024-09-10T16:05:19.875776+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:05:19.875776+0000","last_undegraded":"2024-09-10T16:05:19.875776+0000","last_fullsized":"2024-09-10T16:05:19.875776+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"46'15037","reported_seq":15171,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:19.335505+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:05:19.335505+0000","last_peered":"2024-09-10T16:05:19.335505+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:05:19.335505+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"43'5000","ondisk_log_start":"43'5000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24055808,"num_objects":5873,"num_object_clones":0,"num_object_copies":11746,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5873,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11541,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1618}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":107,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T16:05:18.859169+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:05:18.859169+0000","last_peered":"2024-09-10T16:05:18.859169+0000","last_clean":"2024-09-10T16:05:18.859169+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:05:18.859169+0000","last_undegraded":"2024-09-10T16:05:18.859169+0000","last_fullsized":"2024-09-10T16:05:18.859169+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'15024","reported_seq":15164,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:20.292026+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:05:20.292026+0000","last_peered":"2024-09-10T16:05:20.292026+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:05:20.292026+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"43'5000","ondisk_log_start":"43'5000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23581155,"num_objects":5758,"num_object_clones":0,"num_object_copies":11516,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5758,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12704,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":1454}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104617+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:02:08.104617+0000","last_peered":"2024-09-10T16:02:08.104617+0000","last_clean":"2024-09-10T16:02:08.104617+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:02:08.104617+0000","last_undegraded":"2024-09-10T16:02:08.104617+0000","last_fullsized":"2024-09-10T16:02:08.104617+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'15304","reported_seq":15618,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:20.337755+0000","last_change":"2024-09-10T16:05:18.271152+0000","last_active":"2024-09-10T16:05:20.337755+0000","last_peered":"2024-09-10T16:05:20.337755+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:05:20.337755+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"43'5300","ondisk_log_start":"43'5300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24027136,"num_objects":5866,"num_object_clones":0,"num_object_copies":11732,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4264,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5866,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12858,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":90,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1602}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"46'15054","reported_seq":15192,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:19.346258+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:05:19.346258+0000","last_peered":"2024-09-10T16:05:19.346258+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:05:19.346258+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"43'5000","ondisk_log_start":"43'5000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23691264,"num_objects":5784,"num_object_clones":0,"num_object_copies":11568,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5784,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11685,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1480}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103316+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:02:08.103316+0000","last_peered":"2024-09-10T16:02:08.103316+0000","last_clean":"2024-09-10T16:02:08.103316+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:02:08.103316+0000","last_undegraded":"2024-09-10T16:02:08.103316+0000","last_fullsized":"2024-09-10T16:02:08.103316+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'14693","reported_seq":22992,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:14.363742+0000","last_change":"2024-09-10T16:03:29.605682+0000","last_active":"2024-09-10T16:05:14.363742+0000","last_peered":"2024-09-10T16:05:14.363742+0000","last_clean":"2024-09-10T15:59:14.115595+0000","last_became_active":"2024-09-10T15:59:16.244089+0000","last_became_peered":"2024-09-10T15:59:16.244089+0000","last_unstale":"2024-09-10T16:05:14.363742+0000","last_undegraded":"2024-09-10T16:03:29.603815+0000","last_fullsized":"2024-09-10T15:59:15.774437+0000","mapping_epoch":42,"log_start":"43'4600","ondisk_log_start":"43'4600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24186880,"num_objects":5905,"num_object_clones":0,"num_object_copies":11810,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":129,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5905,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11083,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4077,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5776}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":99,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.876013+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:05:19.876013+0000","last_peered":"2024-09-10T16:05:19.876013+0000","last_clean":"2024-09-10T16:05:19.876013+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:05:19.876013+0000","last_undegraded":"2024-09-10T16:05:19.876013+0000","last_fullsized":"2024-09-10T16:05:19.876013+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"46'15240","reported_seq":15385,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:19.282342+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:05:19.282342+0000","last_peered":"2024-09-10T16:05:19.282342+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:05:19.282342+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"43'5200","ondisk_log_start":"43'5200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23789568,"num_objects":5808,"num_object_clones":0,"num_object_copies":11616,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5808,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11495,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1437}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104463+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:02:08.104463+0000","last_peered":"2024-09-10T16:02:08.104463+0000","last_clean":"2024-09-10T16:02:08.104463+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:02:08.104463+0000","last_undegraded":"2024-09-10T16:02:08.104463+0000","last_fullsized":"2024-09-10T16:02:08.104463+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104448+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:02:08.104448+0000","last_peered":"2024-09-10T16:02:08.104448+0000","last_clean":"2024-09-10T16:02:08.104448+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:02:08.104448+0000","last_undegraded":"2024-09-10T16:02:08.104448+0000","last_fullsized":"2024-09-10T16:02:08.104448+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"46'15258","reported_seq":15405,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:19.363249+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:05:19.363249+0000","last_peered":"2024-09-10T16:05:19.363249+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:05:19.363249+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"43'5200","ondisk_log_start":"43'5200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24331206,"num_objects":5942,"num_object_clones":0,"num_object_copies":11884,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5942,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11421,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1619},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":383247094,"num_objects":93576,"num_object_clones":0,"num_object_copies":187152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42950,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":93576,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":198530,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8439,"num_bytes_recovered":33095680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":978468864,"data_stored":978319813,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160756,"ondisk_log_size":160756,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986264,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":652984,"kb_used_data":324192,"kb_used_omap":4,"kb_used_meta":328763,"kb_avail":93718856,"statfs":{"total":96636764160,"available":95968108544,"internally_reserved":0,"allocated":331972608,"data_stored":331669924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":336653994},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,3,0,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659209,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":604108,"kb_used_data":283640,"kb_used_omap":4,"kb_used_meta":320443,"kb_avail":93767732,"statfs":{"total":96636764160,"available":96018157568,"internally_reserved":0,"allocated":290447360,"data_stored":290118533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":328134318},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,2,2,2,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182492,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":756052,"kb_used_data":349572,"kb_used_omap":4,"kb_used_meta":406459,"kb_avail":93615788,"statfs":{"total":96636764160,"available":95862566912,"internally_reserved":0,"allocated":357961728,"data_stored":357664275,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":416214698},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,3,8],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":357490688,"data_stored":357454558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":289628160,"data_stored":289555441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":331350016,"data_stored":331309814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:05:22.004 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:05:22.548 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:05:22.550 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:05:22.574 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":283,"stamp":"2024-09-10T16:05:21.683507+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":383247113,"num_objects":93578,"num_object_clones":0,"num_object_copies":187156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42950,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":93578,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":198532,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8441,"num_bytes_recovered":33095699,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160759,"ondisk_log_size":160759,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2013144,"kb_used_data":957404,"kb_used_omap":12,"kb_used_meta":1055667,"kb_avail":281102376,"statfs":{"total":289910292480,"available":287848833024,"internally_reserved":0,"allocated":980381696,"data_stored":979452732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1081003010},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,4,2,8,10,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-17313792,"num_objects":-4227,"num_object_clones":0,"num_object_copies":-8454,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-264,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4227,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4227,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":264,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002550"},"pg_stats":[{"pgid":"2.f","version":"46'14918","reported_seq":14996,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:19.314042+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:05:19.314042+0000","last_peered":"2024-09-10T16:05:19.314042+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:05:19.314042+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"43'4900","ondisk_log_start":"43'4900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23568384,"num_objects":5754,"num_object_clones":0,"num_object_copies":11508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14918,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1495}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'15030","reported_seq":15148,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:20.302012+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:05:20.302012+0000","last_peered":"2024-09-10T16:05:20.302012+0000","last_clean":"2024-09-10T16:05:20.302012+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:05:20.302012+0000","last_undegraded":"2024-09-10T16:05:20.302012+0000","last_fullsized":"2024-09-10T16:05:20.302012+0000","mapping_epoch":35,"log_start":"43'5000","ondisk_log_start":"43'5000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23815110,"num_objects":5816,"num_object_clones":0,"num_object_copies":11632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5816,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13135,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'14952","reported_seq":23631,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:05:17.264935+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:05:17.264935+0000","last_peered":"2024-09-10T16:05:17.264935+0000","last_clean":"2024-09-10T16:05:17.264935+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:05:17.264935+0000","last_undegraded":"2024-09-10T16:05:17.264935+0000","last_fullsized":"2024-09-10T16:05:17.264935+0000","mapping_epoch":44,"log_start":"43'4900","ondisk_log_start":"43'4900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23941144,"num_objects":5846,"num_object_clones":0,"num_object_copies":11692,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5846,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12956,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'14886","reported_seq":14972,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:20.262096+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:05:20.262096+0000","last_peered":"2024-09-10T16:05:20.262096+0000","last_clean":"2024-09-10T16:05:20.262096+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:05:20.262096+0000","last_undegraded":"2024-09-10T16:05:20.262096+0000","last_fullsized":"2024-09-10T16:05:20.262096+0000","mapping_epoch":35,"log_start":"43'4800","ondisk_log_start":"43'4800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23650304,"num_objects":5774,"num_object_clones":0,"num_object_copies":11548,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5774,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14886,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"46'15047","reported_seq":15180,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.193698+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:05:19.193698+0000","last_peered":"2024-09-10T16:05:19.193698+0000","last_clean":"2024-09-10T16:05:19.193698+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:05:19.193698+0000","last_undegraded":"2024-09-10T16:05:19.193698+0000","last_fullsized":"2024-09-10T16:05:19.193698+0000","mapping_epoch":34,"log_start":"43'5000","ondisk_log_start":"43'5000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23937507,"num_objects":5845,"num_object_clones":0,"num_object_copies":11690,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5845,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11758,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"46'15224","reported_seq":15365,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:19.335536+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:05:19.335536+0000","last_peered":"2024-09-10T16:05:19.335536+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:05:19.335536+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"43'5200","ondisk_log_start":"43'5200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23908835,"num_objects":5838,"num_object_clones":0,"num_object_copies":11676,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5838,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11836,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1467},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"46'15078","reported_seq":15219,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:19.322762+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:05:19.322762+0000","last_peered":"2024-09-10T16:05:19.322762+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:05:19.322762+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"43'5000","ondisk_log_start":"43'5000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24593350,"num_objects":6006,"num_object_clones":0,"num_object_copies":12012,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6006,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":11577,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1769},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"46'15331","reported_seq":15464,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.354786+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:05:19.354786+0000","last_peered":"2024-09-10T16:05:19.354786+0000","last_clean":"2024-09-10T16:05:19.354786+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:05:19.354786+0000","last_undegraded":"2024-09-10T16:05:19.354786+0000","last_fullsized":"2024-09-10T16:05:19.354786+0000","mapping_epoch":34,"log_start":"43'5300","ondisk_log_start":"43'5300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24506368,"num_objects":5983,"num_object_clones":0,"num_object_copies":11966,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5983,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12178,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":71,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:03:15.844822+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:03:15.844822+0000","last_peered":"2024-09-10T16:03:15.844822+0000","last_clean":"2024-09-10T16:03:15.844822+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:03:15.844822+0000","last_undegraded":"2024-09-10T16:03:15.844822+0000","last_fullsized":"2024-09-10T16:03:15.844822+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'14680","reported_seq":14825,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:20.256986+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:05:20.256986+0000","last_peered":"2024-09-10T16:05:20.256986+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:05:20.256986+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"43'4600","ondisk_log_start":"43'4600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23663075,"num_objects":5778,"num_object_clones":0,"num_object_copies":11556,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5778,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12499,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":1655}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":99,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.875776+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:05:19.875776+0000","last_peered":"2024-09-10T16:05:19.875776+0000","last_clean":"2024-09-10T16:05:19.875776+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:05:19.875776+0000","last_undegraded":"2024-09-10T16:05:19.875776+0000","last_fullsized":"2024-09-10T16:05:19.875776+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"46'15037","reported_seq":15171,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:19.335505+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:05:19.335505+0000","last_peered":"2024-09-10T16:05:19.335505+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:05:19.335505+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"43'5000","ondisk_log_start":"43'5000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24055808,"num_objects":5873,"num_object_clones":0,"num_object_copies":11746,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5873,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11541,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1618}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":107,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T16:05:18.859169+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:05:18.859169+0000","last_peered":"2024-09-10T16:05:18.859169+0000","last_clean":"2024-09-10T16:05:18.859169+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:05:18.859169+0000","last_undegraded":"2024-09-10T16:05:18.859169+0000","last_fullsized":"2024-09-10T16:05:18.859169+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'15024","reported_seq":15164,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:20.292026+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:05:20.292026+0000","last_peered":"2024-09-10T16:05:20.292026+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:05:20.292026+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"43'5000","ondisk_log_start":"43'5000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23581155,"num_objects":5758,"num_object_clones":0,"num_object_copies":11516,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5758,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12704,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":1454}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104617+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:02:08.104617+0000","last_peered":"2024-09-10T16:02:08.104617+0000","last_clean":"2024-09-10T16:02:08.104617+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:02:08.104617+0000","last_undegraded":"2024-09-10T16:02:08.104617+0000","last_fullsized":"2024-09-10T16:02:08.104617+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'15304","reported_seq":15618,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:20.337755+0000","last_change":"2024-09-10T16:05:18.271152+0000","last_active":"2024-09-10T16:05:20.337755+0000","last_peered":"2024-09-10T16:05:20.337755+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:05:20.337755+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"43'5300","ondisk_log_start":"43'5300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24027136,"num_objects":5866,"num_object_clones":0,"num_object_copies":11732,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4264,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5866,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12858,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":90,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1602}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"46'15054","reported_seq":15192,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:19.346258+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:05:19.346258+0000","last_peered":"2024-09-10T16:05:19.346258+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:05:19.346258+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"43'5000","ondisk_log_start":"43'5000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23691264,"num_objects":5784,"num_object_clones":0,"num_object_copies":11568,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5784,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11685,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1480}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.103316+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:02:08.103316+0000","last_peered":"2024-09-10T16:02:08.103316+0000","last_clean":"2024-09-10T16:02:08.103316+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:02:08.103316+0000","last_undegraded":"2024-09-10T16:02:08.103316+0000","last_fullsized":"2024-09-10T16:02:08.103316+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'14693","reported_seq":22992,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:14.363742+0000","last_change":"2024-09-10T16:03:29.605682+0000","last_active":"2024-09-10T16:05:14.363742+0000","last_peered":"2024-09-10T16:05:14.363742+0000","last_clean":"2024-09-10T15:59:14.115595+0000","last_became_active":"2024-09-10T15:59:16.244089+0000","last_became_peered":"2024-09-10T15:59:16.244089+0000","last_unstale":"2024-09-10T16:05:14.363742+0000","last_undegraded":"2024-09-10T16:03:29.603815+0000","last_fullsized":"2024-09-10T15:59:15.774437+0000","mapping_epoch":42,"log_start":"43'4600","ondisk_log_start":"43'4600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24186880,"num_objects":5905,"num_object_clones":0,"num_object_copies":11810,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":129,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5905,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11083,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4077,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5776}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":99,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.876013+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:05:19.876013+0000","last_peered":"2024-09-10T16:05:19.876013+0000","last_clean":"2024-09-10T16:05:19.876013+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:05:19.876013+0000","last_undegraded":"2024-09-10T16:05:19.876013+0000","last_fullsized":"2024-09-10T16:05:19.876013+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"46'15240","reported_seq":15385,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:19.282342+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:05:19.282342+0000","last_peered":"2024-09-10T16:05:19.282342+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:05:19.282342+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"43'5200","ondisk_log_start":"43'5200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23789568,"num_objects":5808,"num_object_clones":0,"num_object_copies":11616,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5808,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11495,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1437}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104463+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:02:08.104463+0000","last_peered":"2024-09-10T16:02:08.104463+0000","last_clean":"2024-09-10T16:02:08.104463+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:02:08.104463+0000","last_undegraded":"2024-09-10T16:02:08.104463+0000","last_fullsized":"2024-09-10T16:02:08.104463+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T16:02:08.104448+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:02:08.104448+0000","last_peered":"2024-09-10T16:02:08.104448+0000","last_clean":"2024-09-10T16:02:08.104448+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:02:08.104448+0000","last_undegraded":"2024-09-10T16:02:08.104448+0000","last_fullsized":"2024-09-10T16:02:08.104448+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"46'15258","reported_seq":15405,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:19.363249+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:05:19.363249+0000","last_peered":"2024-09-10T16:05:19.363249+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:05:19.363249+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"43'5200","ondisk_log_start":"43'5200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24331206,"num_objects":5942,"num_object_clones":0,"num_object_copies":11884,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5942,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11421,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1619},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":383247094,"num_objects":93576,"num_object_clones":0,"num_object_copies":187152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42950,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":93576,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":198530,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8439,"num_bytes_recovered":33095680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":978468864,"data_stored":978319813,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160756,"ondisk_log_size":160756,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986264,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":652984,"kb_used_data":324192,"kb_used_omap":4,"kb_used_meta":328763,"kb_avail":93718856,"statfs":{"total":96636764160,"available":95968108544,"internally_reserved":0,"allocated":331972608,"data_stored":331669924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":336653994},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,3,0,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659209,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":604108,"kb_used_data":283640,"kb_used_omap":4,"kb_used_meta":320443,"kb_avail":93767732,"statfs":{"total":96636764160,"available":96018157568,"internally_reserved":0,"allocated":290447360,"data_stored":290118533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":328134318},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,2,2,2,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182492,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":756052,"kb_used_data":349572,"kb_used_omap":4,"kb_used_meta":406459,"kb_avail":93615788,"statfs":{"total":96636764160,"available":95862566912,"internally_reserved":0,"allocated":357961728,"data_stored":357664275,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":416214698},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,3,8],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":357490688,"data_stored":357454558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":289628160,"data_stored":289555441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":331350016,"data_stored":331309814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:05:22.576 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-10T16:05:22.576 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "46'14918", 'reported_seq': 14996, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:05:19.314042+0000', 'last_change': '2024-09-10T15:59:16.281558+0000', 'last_active': '2024-09-10T16:05:19.314042+0000', 'last_peered': '2024-09-10T16:05:19.314042+0000', 'last_clean': '2024-09-10T15:59:14.132456+0000', 'last_became_active': '2024-09-10T15:59:16.274407+0000', 'last_became_peered': '2024-09-10T15:59:16.274407+0000', 'last_unstale': '2024-09-10T16:05:19.314042+0000', 'last_undegraded': '2024-09-10T15:59:15.851279+0000', 'last_fullsized': '2024-09-10T15:59:15.774356+0000', 'mapping_epoch': 42, 'log_start': "43'4900", 'ondisk_log_start': "43'4900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'objects_scrubbed': 0, 'log_size': 10018, 'log_dups_size': 0, 'ondisk_log_size': 10018, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23568384, 'num_objects': 5754, 'num_object_clones': 0, 'num_object_copies': 11508, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4259, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5754, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14918, 'num_write_kb': 41344, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1495}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:05:22.576 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-10T16:05:22.576 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "46'15224", 'reported_seq': 15365, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:05:19.335536+0000', 'last_change': '2024-09-10T15:59:16.308264+0000', 'last_active': '2024-09-10T16:05:19.335536+0000', 'last_peered': '2024-09-10T16:05:19.335536+0000', 'last_clean': '2024-09-10T15:59:14.111551+0000', 'last_became_active': '2024-09-10T15:59:16.300040+0000', 'last_became_peered': '2024-09-10T15:59:16.300040+0000', 'last_unstale': '2024-09-10T16:05:19.335536+0000', 'last_undegraded': '2024-09-10T15:59:15.878234+0000', 'last_fullsized': '2024-09-10T15:59:15.769328+0000', 'mapping_epoch': 42, 'log_start': "43'5200", 'ondisk_log_start': "43'5200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3442", 'last_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'objects_scrubbed': 3384, 'log_size': 10024, 'log_dups_size': 0, 'ondisk_log_size': 10024, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1292, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23908835, 'num_objects': 5838, 'num_object_clones': 0, 'num_object_copies': 11676, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4372, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5838, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 11836, 'num_write_kb': 28580, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1467}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:05:22.576 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-10T16:05:22.576 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "46'15078", 'reported_seq': 15219, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:05:19.322762+0000', 'last_change': '2024-09-10T15:59:16.331899+0000', 'last_active': '2024-09-10T16:05:19.322762+0000', 'last_peered': '2024-09-10T16:05:19.322762+0000', 'last_clean': '2024-09-10T15:59:14.139436+0000', 'last_became_active': '2024-09-10T15:59:16.318715+0000', 'last_became_peered': '2024-09-10T15:59:16.318715+0000', 'last_unstale': '2024-09-10T16:05:19.322762+0000', 'last_undegraded': '2024-09-10T15:59:15.877524+0000', 'last_fullsized': '2024-09-10T15:59:15.769042+0000', 'mapping_epoch': 42, 'log_start': "43'5000", 'ondisk_log_start': "43'5000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3537", 'last_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'objects_scrubbed': 3497, 'log_size': 10078, 'log_dups_size': 0, 'ondisk_log_size': 10078, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1224, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24593350, 'num_objects': 6006, 'num_object_clones': 0, 'num_object_copies': 12012, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4237, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6006, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 11577, 'num_write_kb': 28172, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1769}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:05:22.576 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-10T16:05:22.577 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "47'14680", 'reported_seq': 14825, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:05:20.256986+0000', 'last_change': '2024-09-10T15:59:16.175680+0000', 'last_active': '2024-09-10T16:05:20.256986+0000', 'last_peered': '2024-09-10T16:05:20.256986+0000', 'last_clean': '2024-09-10T15:59:14.088187+0000', 'last_became_active': '2024-09-10T15:59:16.165810+0000', 'last_became_peered': '2024-09-10T15:59:16.165810+0000', 'last_unstale': '2024-09-10T16:05:20.256986+0000', 'last_undegraded': '2024-09-10T15:59:15.822098+0000', 'last_fullsized': '2024-09-10T15:59:15.769123+0000', 'mapping_epoch': 42, 'log_start': "43'4600", 'ondisk_log_start': "43'4600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2218", 'last_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'objects_scrubbed': 2175, 'log_size': 10080, 'log_dups_size': 0, 'ondisk_log_size': 10080, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 825, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23663075, 'num_objects': 5778, 'num_object_clones': 0, 'num_object_copies': 11556, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4124, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5778, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 12499, 'num_write_kb': 32204, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': 1655}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:05:22.577 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-10T16:05:22.577 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "46'15037", 'reported_seq': 15171, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:05:19.335505+0000', 'last_change': '2024-09-10T15:59:16.277742+0000', 'last_active': '2024-09-10T16:05:19.335505+0000', 'last_peered': '2024-09-10T16:05:19.335505+0000', 'last_clean': '2024-09-10T15:59:14.129081+0000', 'last_became_active': '2024-09-10T15:59:16.269835+0000', 'last_became_peered': '2024-09-10T15:59:16.269835+0000', 'last_unstale': '2024-09-10T16:05:19.335505+0000', 'last_undegraded': '2024-09-10T15:59:15.860857+0000', 'last_fullsized': '2024-09-10T15:59:15.771410+0000', 'mapping_epoch': 42, 'log_start': "43'5000", 'ondisk_log_start': "43'5000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3557", 'last_scrub_stamp': '2024-09-10T15:59:02.493678+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:02.493678+0000', 'objects_scrubbed': 3488, 'log_size': 10037, 'log_dups_size': 0, 'ondisk_log_size': 10037, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1472, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24055808, 'num_objects': 5873, 'num_object_clones': 0, 'num_object_copies': 11746, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4257, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5873, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11541, 'num_write_kb': 27852, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1618}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:05:22.577 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-10T16:05:22.577 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "47'15024", 'reported_seq': 15164, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:05:20.292026+0000', 'last_change': '2024-09-10T15:59:16.284311+0000', 'last_active': '2024-09-10T16:05:20.292026+0000', 'last_peered': '2024-09-10T16:05:20.292026+0000', 'last_clean': '2024-09-10T15:59:14.116041+0000', 'last_became_active': '2024-09-10T15:59:16.274543+0000', 'last_became_peered': '2024-09-10T15:59:16.274543+0000', 'last_unstale': '2024-09-10T16:05:20.292026+0000', 'last_undegraded': '2024-09-10T15:59:15.812484+0000', 'last_fullsized': '2024-09-10T15:59:15.770885+0000', 'mapping_epoch': 42, 'log_start': "43'5000", 'ondisk_log_start': "43'5000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2356", 'last_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'objects_scrubbed': 2316, 'log_size': 10024, 'log_dups_size': 0, 'ondisk_log_size': 10024, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 783, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23581155, 'num_objects': 5758, 'num_object_clones': 0, 'num_object_copies': 11516, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5758, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 12704, 'num_write_kb': 32292, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': 1454}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:05:22.577 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-10T16:05:22.577 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "47'15304", 'reported_seq': 15618, 'reported_epoch': 47, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:05:20.337755+0000', 'last_change': '2024-09-10T16:05:18.271152+0000', 'last_active': '2024-09-10T16:05:20.337755+0000', 'last_peered': '2024-09-10T16:05:20.337755+0000', 'last_clean': '2024-09-10T15:59:14.110654+0000', 'last_became_active': '2024-09-10T15:59:16.164645+0000', 'last_became_peered': '2024-09-10T15:59:16.164645+0000', 'last_unstale': '2024-09-10T16:05:20.337755+0000', 'last_undegraded': '2024-09-10T15:59:15.803593+0000', 'last_fullsized': '2024-09-10T15:59:15.769083+0000', 'mapping_epoch': 42, 'log_start': "43'5300", 'ondisk_log_start': "43'5300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2486", 'last_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'objects_scrubbed': 2440, 'log_size': 10004, 'log_dups_size': 0, 'ondisk_log_size': 10004, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 792, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24027136, 'num_objects': 5866, 'num_object_clones': 0, 'num_object_copies': 11732, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4264, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5866, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12858, 'num_write_kb': 32568, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 90, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 1602}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:05:22.577 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-10T16:05:22.577 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "46'15054", 'reported_seq': 15192, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:05:19.346258+0000', 'last_change': '2024-09-10T15:59:16.315641+0000', 'last_active': '2024-09-10T16:05:19.346258+0000', 'last_peered': '2024-09-10T16:05:19.346258+0000', 'last_clean': '2024-09-10T15:59:14.117001+0000', 'last_became_active': '2024-09-10T15:59:16.302245+0000', 'last_became_peered': '2024-09-10T15:59:16.302245+0000', 'last_unstale': '2024-09-10T16:05:19.346258+0000', 'last_undegraded': '2024-09-10T15:59:15.845012+0000', 'last_fullsized': '2024-09-10T15:59:15.770337+0000', 'mapping_epoch': 42, 'log_start': "43'5000", 'ondisk_log_start': "43'5000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3404", 'last_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'objects_scrubbed': 3367, 'log_size': 10054, 'log_dups_size': 0, 'ondisk_log_size': 10054, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 983, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23691264, 'num_objects': 5784, 'num_object_clones': 0, 'num_object_copies': 11568, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5784, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11685, 'num_write_kb': 28204, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1480}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:05:22.577 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-10T16:05:22.577 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "45'14693", 'reported_seq': 22992, 'reported_epoch': 45, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:05:14.363742+0000', 'last_change': '2024-09-10T16:03:29.605682+0000', 'last_active': '2024-09-10T16:05:14.363742+0000', 'last_peered': '2024-09-10T16:05:14.363742+0000', 'last_clean': '2024-09-10T15:59:14.115595+0000', 'last_became_active': '2024-09-10T15:59:16.244089+0000', 'last_became_peered': '2024-09-10T15:59:16.244089+0000', 'last_unstale': '2024-09-10T16:05:14.363742+0000', 'last_undegraded': '2024-09-10T16:03:29.603815+0000', 'last_fullsized': '2024-09-10T15:59:15.774437+0000', 'mapping_epoch': 42, 'log_start': "43'4600", 'ondisk_log_start': "43'4600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3653", 'last_scrub_stamp': '2024-09-10T15:59:05.298468+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:05.298468+0000', 'objects_scrubbed': 3602, 'log_size': 10093, 'log_dups_size': 0, 'ondisk_log_size': 10093, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1289, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24186880, 'num_objects': 5905, 'num_object_clones': 0, 'num_object_copies': 11810, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 129, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5905, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11083, 'num_write_kb': 26772, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 4077, 'num_bytes_recovered': 15601664, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5776}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:05:22.578 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-10T16:05:22.578 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "46'15240", 'reported_seq': 15385, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:05:19.282342+0000', 'last_change': '2024-09-10T15:59:16.311291+0000', 'last_active': '2024-09-10T16:05:19.282342+0000', 'last_peered': '2024-09-10T16:05:19.282342+0000', 'last_clean': '2024-09-10T15:59:14.116363+0000', 'last_became_active': '2024-09-10T15:59:16.301244+0000', 'last_became_peered': '2024-09-10T15:59:16.301244+0000', 'last_unstale': '2024-09-10T16:05:19.282342+0000', 'last_undegraded': '2024-09-10T15:59:15.883434+0000', 'last_fullsized': '2024-09-10T15:59:15.772756+0000', 'mapping_epoch': 42, 'log_start': "43'5200", 'ondisk_log_start': "43'5200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3789", 'last_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'objects_scrubbed': 3741, 'log_size': 10040, 'log_dups_size': 0, 'ondisk_log_size': 10040, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1440, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23789568, 'num_objects': 5808, 'num_object_clones': 0, 'num_object_copies': 11616, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4373, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5808, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11495, 'num_write_kb': 27124, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1437}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:05:22.578 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-10T16:05:22.578 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "46'15258", 'reported_seq': 15405, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:05:19.363249+0000', 'last_change': '2024-09-10T15:59:16.313409+0000', 'last_active': '2024-09-10T16:05:19.363249+0000', 'last_peered': '2024-09-10T16:05:19.363249+0000', 'last_clean': '2024-09-10T15:59:14.129780+0000', 'last_became_active': '2024-09-10T15:59:16.300932+0000', 'last_became_peered': '2024-09-10T15:59:16.300932+0000', 'last_unstale': '2024-09-10T16:05:19.363249+0000', 'last_undegraded': '2024-09-10T15:59:15.887823+0000', 'last_fullsized': '2024-09-10T15:59:15.768513+0000', 'mapping_epoch': 42, 'log_start': "43'5200", 'ondisk_log_start': "43'5200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3880", 'last_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'objects_scrubbed': 3827, 'log_size': 10058, 'log_dups_size': 0, 'ondisk_log_size': 10058, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1397, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24331206, 'num_objects': 5942, 'num_object_clones': 0, 'num_object_copies': 11884, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4325, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5942, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11421, 'num_write_kb': 27072, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1619}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:05:22.578 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T16:05:23.294 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:05:23.294 INFO:teuthology.orchestra.run.smithi115.stdout:{"fsid":"7effee29-df05-4351-9741-ee203564ed81","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 42950/187156 objects degraded (22.949%), 11 pgs degraded, 11 pgs undersized","count":22},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":487,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":47,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983954,"num_in_osds":3,"osd_in_since":1725983947,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":2}],"num_pgs":24,"num_pools":2,"num_objects":93578,"data_bytes":383247113,"bytes_used":2061459456,"bytes_avail":287848833024,"bytes_total":289910292480,"degraded_objects":42950,"degraded_total":187156,"degraded_ratio":0.229487700100451,"recovering_objects_per_sec":32,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":264,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":528},"fsmap":{"epoch":1,"btime":"2024-09-10T15:57:14:967120+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T16:00:13.560776+0000","services":{}},"progress_events":{}} 2024-09-10T16:05:23.323 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-10T16:05:23.323 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:05:23.914 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:05:23.915 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:05:23.943 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":284,"stamp":"2024-09-10T16:05:23.684265+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":381924105,"num_objects":93255,"num_object_clones":0,"num_object_copies":186510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42821,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":93255,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":198855,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8570,"num_bytes_recovered":33095699,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160782,"ondisk_log_size":160782,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2011200,"kb_used_data":951304,"kb_used_omap":12,"kb_used_meta":1059827,"kb_avail":281104320,"statfs":{"total":289910292480,"available":287850823680,"internally_reserved":0,"allocated":974135296,"data_stored":973203107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1085262850},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,0,6,8,1,0,0,8,12],"upper_bound":2048},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-9871360,"num_objects":-2410,"num_object_clones":0,"num_object_copies":-4820,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-219,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-2410,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2410,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":219,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002670"},"pg_stats":[{"pgid":"2.f","version":"46'14918","reported_seq":14996,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:19.314042+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:05:19.314042+0000","last_peered":"2024-09-10T16:05:19.314042+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:05:19.314042+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"43'4900","ondisk_log_start":"43'4900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23568384,"num_objects":5754,"num_object_clones":0,"num_object_copies":11508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14918,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1495}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'15030","reported_seq":15148,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:20.302012+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:05:20.302012+0000","last_peered":"2024-09-10T16:05:20.302012+0000","last_clean":"2024-09-10T16:05:20.302012+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:05:20.302012+0000","last_undegraded":"2024-09-10T16:05:20.302012+0000","last_fullsized":"2024-09-10T16:05:20.302012+0000","mapping_epoch":35,"log_start":"43'5000","ondisk_log_start":"43'5000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23815110,"num_objects":5816,"num_object_clones":0,"num_object_copies":11632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5816,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13135,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'15083","reported_seq":23766,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:22.328783+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:05:22.328783+0000","last_peered":"2024-09-10T16:05:22.328783+0000","last_clean":"2024-09-10T16:05:22.328783+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:05:22.328783+0000","last_undegraded":"2024-09-10T16:05:22.328783+0000","last_fullsized":"2024-09-10T16:05:22.328783+0000","mapping_epoch":44,"log_start":"43'5000","ondisk_log_start":"43'5000","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23404568,"num_objects":5715,"num_object_clones":0,"num_object_copies":11430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5715,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13087,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'14886","reported_seq":14972,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:20.262096+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:05:20.262096+0000","last_peered":"2024-09-10T16:05:20.262096+0000","last_clean":"2024-09-10T16:05:20.262096+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:05:20.262096+0000","last_undegraded":"2024-09-10T16:05:20.262096+0000","last_fullsized":"2024-09-10T16:05:20.262096+0000","mapping_epoch":35,"log_start":"43'4800","ondisk_log_start":"43'4800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23650304,"num_objects":5774,"num_object_clones":0,"num_object_copies":11548,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5774,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14886,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"46'15047","reported_seq":15180,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.193698+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:05:19.193698+0000","last_peered":"2024-09-10T16:05:19.193698+0000","last_clean":"2024-09-10T16:05:19.193698+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:05:19.193698+0000","last_undegraded":"2024-09-10T16:05:19.193698+0000","last_fullsized":"2024-09-10T16:05:19.193698+0000","mapping_epoch":34,"log_start":"43'5000","ondisk_log_start":"43'5000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23937507,"num_objects":5845,"num_object_clones":0,"num_object_copies":11690,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5845,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11758,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"46'15224","reported_seq":15365,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:19.335536+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:05:19.335536+0000","last_peered":"2024-09-10T16:05:19.335536+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:05:19.335536+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"43'5200","ondisk_log_start":"43'5200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23908835,"num_objects":5838,"num_object_clones":0,"num_object_copies":11676,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5838,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11836,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1467},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"46'15078","reported_seq":15219,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:19.322762+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:05:19.322762+0000","last_peered":"2024-09-10T16:05:19.322762+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:05:19.322762+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"43'5000","ondisk_log_start":"43'5000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24593350,"num_objects":6006,"num_object_clones":0,"num_object_copies":12012,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6006,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":11577,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1769},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"46'15331","reported_seq":15464,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.354786+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:05:19.354786+0000","last_peered":"2024-09-10T16:05:19.354786+0000","last_clean":"2024-09-10T16:05:19.354786+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:05:19.354786+0000","last_undegraded":"2024-09-10T16:05:19.354786+0000","last_fullsized":"2024-09-10T16:05:19.354786+0000","mapping_epoch":34,"log_start":"43'5300","ondisk_log_start":"43'5300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24506368,"num_objects":5983,"num_object_clones":0,"num_object_copies":11966,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5983,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12178,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.876895+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:05:19.876895+0000","last_peered":"2024-09-10T16:05:19.876895+0000","last_clean":"2024-09-10T16:05:19.876895+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:05:19.876895+0000","last_undegraded":"2024-09-10T16:05:19.876895+0000","last_fullsized":"2024-09-10T16:05:19.876895+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'14680","reported_seq":14825,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:20.256986+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:05:20.256986+0000","last_peered":"2024-09-10T16:05:20.256986+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:05:20.256986+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"43'4600","ondisk_log_start":"43'4600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23663075,"num_objects":5778,"num_object_clones":0,"num_object_copies":11556,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5778,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12499,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":1655}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":99,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.875776+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:05:19.875776+0000","last_peered":"2024-09-10T16:05:19.875776+0000","last_clean":"2024-09-10T16:05:19.875776+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:05:19.875776+0000","last_undegraded":"2024-09-10T16:05:19.875776+0000","last_fullsized":"2024-09-10T16:05:19.875776+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"46'15037","reported_seq":15171,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:19.335505+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:05:19.335505+0000","last_peered":"2024-09-10T16:05:19.335505+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:05:19.335505+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"43'5000","ondisk_log_start":"43'5000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24055808,"num_objects":5873,"num_object_clones":0,"num_object_copies":11746,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5873,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11541,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1618}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":107,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T16:05:18.859169+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:05:18.859169+0000","last_peered":"2024-09-10T16:05:18.859169+0000","last_clean":"2024-09-10T16:05:18.859169+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:05:18.859169+0000","last_undegraded":"2024-09-10T16:05:18.859169+0000","last_fullsized":"2024-09-10T16:05:18.859169+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'15024","reported_seq":15164,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:20.292026+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:05:20.292026+0000","last_peered":"2024-09-10T16:05:20.292026+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:05:20.292026+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"43'5000","ondisk_log_start":"43'5000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23581155,"num_objects":5758,"num_object_clones":0,"num_object_copies":11516,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5758,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12704,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":1454}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":84,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877110+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:05:19.877110+0000","last_peered":"2024-09-10T16:05:19.877110+0000","last_clean":"2024-09-10T16:05:19.877110+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:05:19.877110+0000","last_undegraded":"2024-09-10T16:05:19.877110+0000","last_fullsized":"2024-09-10T16:05:19.877110+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'15304","reported_seq":15618,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:20.337755+0000","last_change":"2024-09-10T16:05:18.271152+0000","last_active":"2024-09-10T16:05:20.337755+0000","last_peered":"2024-09-10T16:05:20.337755+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:05:20.337755+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"43'5300","ondisk_log_start":"43'5300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24027136,"num_objects":5866,"num_object_clones":0,"num_object_copies":11732,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4264,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5866,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12858,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":90,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1602}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"46'15054","reported_seq":15192,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:19.346258+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:05:19.346258+0000","last_peered":"2024-09-10T16:05:19.346258+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:05:19.346258+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"43'5000","ondisk_log_start":"43'5000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23691264,"num_objects":5784,"num_object_clones":0,"num_object_copies":11568,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5784,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11685,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1480}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877617+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:05:19.877617+0000","last_peered":"2024-09-10T16:05:19.877617+0000","last_clean":"2024-09-10T16:05:19.877617+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:05:19.877617+0000","last_undegraded":"2024-09-10T16:05:19.877617+0000","last_fullsized":"2024-09-10T16:05:19.877617+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'14885","reported_seq":23456,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:22.391875+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:05:22.391875+0000","last_peered":"2024-09-10T16:05:22.391875+0000","last_clean":"2024-09-10T16:05:22.391875+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:05:22.391875+0000","last_undegraded":"2024-09-10T16:05:22.391875+0000","last_fullsized":"2024-09-10T16:05:22.391875+0000","mapping_epoch":46,"log_start":"43'4800","ondisk_log_start":"43'4800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23400448,"num_objects":5713,"num_object_clones":0,"num_object_copies":11426,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5713,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11275,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":99,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.876013+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:05:19.876013+0000","last_peered":"2024-09-10T16:05:19.876013+0000","last_clean":"2024-09-10T16:05:19.876013+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:05:19.876013+0000","last_undegraded":"2024-09-10T16:05:19.876013+0000","last_fullsized":"2024-09-10T16:05:19.876013+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"46'15240","reported_seq":15385,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:19.282342+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:05:19.282342+0000","last_peered":"2024-09-10T16:05:19.282342+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:05:19.282342+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"43'5200","ondisk_log_start":"43'5200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23789568,"num_objects":5808,"num_object_clones":0,"num_object_copies":11616,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5808,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11495,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1437}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877315+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:05:19.877315+0000","last_peered":"2024-09-10T16:05:19.877315+0000","last_clean":"2024-09-10T16:05:19.877315+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:05:19.877315+0000","last_undegraded":"2024-09-10T16:05:19.877315+0000","last_fullsized":"2024-09-10T16:05:19.877315+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877229+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:05:19.877229+0000","last_peered":"2024-09-10T16:05:19.877229+0000","last_clean":"2024-09-10T16:05:19.877229+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:05:19.877229+0000","last_undegraded":"2024-09-10T16:05:19.877229+0000","last_fullsized":"2024-09-10T16:05:19.877229+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"46'15258","reported_seq":15405,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:19.363249+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:05:19.363249+0000","last_peered":"2024-09-10T16:05:19.363249+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:05:19.363249+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"43'5200","ondisk_log_start":"43'5200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24331206,"num_objects":5942,"num_object_clones":0,"num_object_copies":11884,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5942,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11421,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1619},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":381924086,"num_objects":93253,"num_object_clones":0,"num_object_copies":186506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42821,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":93253,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":198853,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8568,"num_bytes_recovered":33095680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":972181504,"data_stored":972039679,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160779,"ondisk_log_size":160779,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986264,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":652984,"kb_used_data":324192,"kb_used_omap":4,"kb_used_meta":328763,"kb_avail":93718856,"statfs":{"total":96636764160,"available":95968108544,"internally_reserved":0,"allocated":331972608,"data_stored":331669924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":336653994},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,3,0,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659210,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":602164,"kb_used_data":277540,"kb_used_omap":4,"kb_used_meta":324603,"kb_avail":93769676,"statfs":{"total":96636764160,"available":96020148224,"internally_reserved":0,"allocated":284200960,"data_stored":283868908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":332394158},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,8,12],"upper_bound":2048},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182492,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":756052,"kb_used_data":349572,"kb_used_omap":4,"kb_used_meta":406459,"kb_avail":93615788,"statfs":{"total":96636764160,"available":95862566912,"internally_reserved":0,"allocated":357961728,"data_stored":357664275,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":416214698},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,3,8],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":357490688,"data_stored":357454558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":283340800,"data_stored":283275307,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":331350016,"data_stored":331309814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:05:26.946 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:05:27.514 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:05:27.518 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:05:27.546 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":285,"stamp":"2024-09-10T16:05:25.685111+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":374153993,"num_objects":91358,"num_object_clones":0,"num_object_copies":182716,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42730,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":91358,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":200752,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8661,"num_bytes_recovered":33095699,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161079,"ondisk_log_size":161079,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1991928,"kb_used_data":940356,"kb_used_omap":12,"kb_used_meta":1051507,"kb_avail":281123592,"statfs":{"total":289910292480,"available":287870558208,"internally_reserved":0,"allocated":962924544,"data_stored":961987119,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1076743170},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,4,3,8,5,4,0,8,12],"upper_bound":2048},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-16932864,"num_objects":-4134,"num_object_clones":0,"num_object_copies":-8268,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-310,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4134,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4134,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":310,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002722"},"pg_stats":[{"pgid":"2.f","version":"47'15050","reported_seq":15129,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:24.417372+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:05:24.417372+0000","last_peered":"2024-09-10T16:05:24.417372+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:05:24.417372+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"43'5000","ondisk_log_start":"43'5000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23027712,"num_objects":5622,"num_object_clones":0,"num_object_copies":11244,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5622,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15050,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1363}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'15164","reported_seq":15282,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:25.309452+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:05:25.309452+0000","last_peered":"2024-09-10T16:05:25.309452+0000","last_clean":"2024-09-10T16:05:25.309452+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:05:25.309452+0000","last_undegraded":"2024-09-10T16:05:25.309452+0000","last_fullsized":"2024-09-10T16:05:25.309452+0000","mapping_epoch":35,"log_start":"43'5100","ondisk_log_start":"43'5100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23266246,"num_objects":5682,"num_object_clones":0,"num_object_copies":11364,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5682,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13269,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'15083","reported_seq":23766,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:22.328783+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:05:22.328783+0000","last_peered":"2024-09-10T16:05:22.328783+0000","last_clean":"2024-09-10T16:05:22.328783+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:05:22.328783+0000","last_undegraded":"2024-09-10T16:05:22.328783+0000","last_fullsized":"2024-09-10T16:05:22.328783+0000","mapping_epoch":44,"log_start":"43'5000","ondisk_log_start":"43'5000","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23404568,"num_objects":5715,"num_object_clones":0,"num_object_copies":11430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5715,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13087,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'15071","reported_seq":15157,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:25.319140+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:05:25.319140+0000","last_peered":"2024-09-10T16:05:25.319140+0000","last_clean":"2024-09-10T16:05:25.319140+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:05:25.319140+0000","last_undegraded":"2024-09-10T16:05:25.319140+0000","last_fullsized":"2024-09-10T16:05:25.319140+0000","mapping_epoch":35,"log_start":"43'5000","ondisk_log_start":"43'5000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22892544,"num_objects":5589,"num_object_clones":0,"num_object_copies":11178,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5589,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15071,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'15178","reported_seq":15313,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:24.393595+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:05:24.393595+0000","last_peered":"2024-09-10T16:05:24.393595+0000","last_clean":"2024-09-10T16:05:24.393595+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:05:24.393595+0000","last_undegraded":"2024-09-10T16:05:24.393595+0000","last_fullsized":"2024-09-10T16:05:24.393595+0000","mapping_epoch":34,"log_start":"43'5100","ondisk_log_start":"43'5100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23400931,"num_objects":5714,"num_object_clones":0,"num_object_copies":11428,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5714,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11889,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'15353","reported_seq":15496,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:24.395494+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:05:24.395494+0000","last_peered":"2024-09-10T16:05:24.395494+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:05:24.395494+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"43'5300","ondisk_log_start":"43'5300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23380451,"num_objects":5709,"num_object_clones":0,"num_object_copies":11418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5709,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11965,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1338},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'15204","reported_seq":15347,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:24.377808+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:05:24.377808+0000","last_peered":"2024-09-10T16:05:24.377808+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:05:24.377808+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"43'5200","ondisk_log_start":"43'5200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24077254,"num_objects":5880,"num_object_clones":0,"num_object_copies":11760,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5880,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":11703,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1643},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'15474","reported_seq":15609,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:24.400848+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:05:24.400848+0000","last_peered":"2024-09-10T16:05:24.400848+0000","last_clean":"2024-09-10T16:05:24.400848+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:05:24.400848+0000","last_undegraded":"2024-09-10T16:05:24.400848+0000","last_fullsized":"2024-09-10T16:05:24.400848+0000","mapping_epoch":34,"log_start":"43'5400","ondisk_log_start":"43'5400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23920640,"num_objects":5840,"num_object_clones":0,"num_object_copies":11680,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5840,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12321,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.876895+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:05:19.876895+0000","last_peered":"2024-09-10T16:05:19.876895+0000","last_clean":"2024-09-10T16:05:19.876895+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:05:19.876895+0000","last_undegraded":"2024-09-10T16:05:19.876895+0000","last_fullsized":"2024-09-10T16:05:19.876895+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'14796","reported_seq":14941,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:25.340613+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:05:25.340613+0000","last_peered":"2024-09-10T16:05:25.340613+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:05:25.340613+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"43'4700","ondisk_log_start":"43'4700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23187939,"num_objects":5662,"num_object_clones":0,"num_object_copies":11324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5662,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12615,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":1539}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":99,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.875776+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:05:19.875776+0000","last_peered":"2024-09-10T16:05:19.875776+0000","last_clean":"2024-09-10T16:05:19.875776+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:05:19.875776+0000","last_undegraded":"2024-09-10T16:05:19.875776+0000","last_fullsized":"2024-09-10T16:05:19.875776+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'15155","reported_seq":15291,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:24.322985+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:05:24.322985+0000","last_peered":"2024-09-10T16:05:24.322985+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:05:24.322985+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"43'5100","ondisk_log_start":"43'5100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23572480,"num_objects":5755,"num_object_clones":0,"num_object_copies":11510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5755,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11659,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1500}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":109,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.871983+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:05:19.871983+0000","last_peered":"2024-09-10T16:05:19.871983+0000","last_clean":"2024-09-10T16:05:19.871983+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:05:19.871983+0000","last_undegraded":"2024-09-10T16:05:19.871983+0000","last_fullsized":"2024-09-10T16:05:19.871983+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'15152","reported_seq":15292,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:25.282220+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:05:25.282220+0000","last_peered":"2024-09-10T16:05:25.282220+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:05:25.282220+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"43'5100","ondisk_log_start":"43'5100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23056867,"num_objects":5630,"num_object_clones":0,"num_object_copies":11260,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5630,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12832,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":1326}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":84,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877110+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:05:19.877110+0000","last_peered":"2024-09-10T16:05:19.877110+0000","last_clean":"2024-09-10T16:05:19.877110+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:05:19.877110+0000","last_undegraded":"2024-09-10T16:05:19.877110+0000","last_fullsized":"2024-09-10T16:05:19.877110+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'15449","reported_seq":15946,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:25.334062+0000","last_change":"2024-09-10T16:05:18.271152+0000","last_active":"2024-09-10T16:05:25.334062+0000","last_peered":"2024-09-10T16:05:25.334062+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:05:25.334062+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"43'5400","ondisk_log_start":"43'5400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23433216,"num_objects":5721,"num_object_clones":0,"num_object_copies":11442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4173,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5721,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13003,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":181,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1548}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'15193","reported_seq":15333,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:24.415859+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:05:24.415859+0000","last_peered":"2024-09-10T16:05:24.415859+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:05:24.415859+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"43'5100","ondisk_log_start":"43'5100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23121920,"num_objects":5645,"num_object_clones":0,"num_object_copies":11290,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5645,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11824,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1341}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877617+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:05:19.877617+0000","last_peered":"2024-09-10T16:05:19.877617+0000","last_clean":"2024-09-10T16:05:19.877617+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:05:19.877617+0000","last_undegraded":"2024-09-10T16:05:19.877617+0000","last_fullsized":"2024-09-10T16:05:19.877617+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'14885","reported_seq":23456,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:22.391875+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:05:22.391875+0000","last_peered":"2024-09-10T16:05:22.391875+0000","last_clean":"2024-09-10T16:05:22.391875+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:05:22.391875+0000","last_undegraded":"2024-09-10T16:05:22.391875+0000","last_fullsized":"2024-09-10T16:05:22.391875+0000","mapping_epoch":46,"log_start":"43'4800","ondisk_log_start":"43'4800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23400448,"num_objects":5713,"num_object_clones":0,"num_object_copies":11426,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5713,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11275,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":99,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.876013+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:05:19.876013+0000","last_peered":"2024-09-10T16:05:19.876013+0000","last_clean":"2024-09-10T16:05:19.876013+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:05:19.876013+0000","last_undegraded":"2024-09-10T16:05:19.876013+0000","last_fullsized":"2024-09-10T16:05:19.876013+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'15373","reported_seq":15520,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:24.363412+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:05:24.363412+0000","last_peered":"2024-09-10T16:05:24.363412+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:05:24.363412+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"43'5300","ondisk_log_start":"43'5300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23244800,"num_objects":5675,"num_object_clones":0,"num_object_copies":11350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5675,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11628,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877315+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:05:19.877315+0000","last_peered":"2024-09-10T16:05:19.877315+0000","last_clean":"2024-09-10T16:05:19.877315+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:05:19.877315+0000","last_undegraded":"2024-09-10T16:05:19.877315+0000","last_fullsized":"2024-09-10T16:05:19.877315+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877229+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:05:19.877229+0000","last_peered":"2024-09-10T16:05:19.877229+0000","last_clean":"2024-09-10T16:05:19.877229+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:05:19.877229+0000","last_undegraded":"2024-09-10T16:05:19.877229+0000","last_fullsized":"2024-09-10T16:05:19.877229+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'15396","reported_seq":15544,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:24.401147+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:05:24.401147+0000","last_peered":"2024-09-10T16:05:24.401147+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:05:24.401147+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"43'5300","ondisk_log_start":"43'5300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23765958,"num_objects":5804,"num_object_clones":0,"num_object_copies":11608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5804,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11559,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1481},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":374153974,"num_objects":91356,"num_object_clones":0,"num_object_copies":182712,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42730,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":91356,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":200750,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8659,"num_bytes_recovered":33095680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":960872448,"data_stored":960730623,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161076,"ondisk_log_size":161076,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986265,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":635888,"kb_used_data":319584,"kb_used_omap":4,"kb_used_meta":316283,"kb_avail":93735952,"statfs":{"total":96636764160,"available":95985614848,"internally_reserved":0,"allocated":327254016,"data_stored":326951332,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":323874474},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,0,3,4,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659210,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":602164,"kb_used_data":277540,"kb_used_omap":4,"kb_used_meta":324603,"kb_avail":93769676,"statfs":{"total":96636764160,"available":96020148224,"internally_reserved":0,"allocated":284200960,"data_stored":283868908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":332394158},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,8,12],"upper_bound":2048},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182493,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":753876,"kb_used_data":343232,"kb_used_omap":4,"kb_used_meta":410619,"kb_avail":93617964,"statfs":{"total":96636764160,"available":95864795136,"internally_reserved":0,"allocated":351469568,"data_stored":351166879,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":420474538},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,3,5,1,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":350896128,"data_stored":350859998,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":283340800,"data_stored":283275307,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":326635520,"data_stored":326595318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:05:27.547 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:05:28.133 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:05:28.134 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:05:28.167 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":286,"stamp":"2024-09-10T16:05:27.685700+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":373007113,"num_objects":91078,"num_object_clones":0,"num_object_copies":182156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42730,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":91078,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":201032,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8661,"num_bytes_recovered":33095699,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160959,"ondisk_log_size":160959,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1993456,"kb_used_data":933564,"kb_used_omap":12,"kb_used_meta":1059827,"kb_avail":281122064,"statfs":{"total":289910292480,"available":287868993536,"internally_reserved":0,"allocated":955969536,"data_stored":955032111,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1085262850},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,5,5,13,9,7,1],"upper_bound":512},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18079744,"num_objects":-4414,"num_object_clones":0,"num_object_copies":-8828,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-310,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4414,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4414,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":310,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.828008"},"pg_stats":[{"pgid":"2.f","version":"47'15050","reported_seq":15129,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:24.417372+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:05:24.417372+0000","last_peered":"2024-09-10T16:05:24.417372+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:05:24.417372+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"43'5000","ondisk_log_start":"43'5000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23027712,"num_objects":5622,"num_object_clones":0,"num_object_copies":11244,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5622,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15050,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1363}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'15164","reported_seq":15282,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:25.309452+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:05:25.309452+0000","last_peered":"2024-09-10T16:05:25.309452+0000","last_clean":"2024-09-10T16:05:25.309452+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:05:25.309452+0000","last_undegraded":"2024-09-10T16:05:25.309452+0000","last_fullsized":"2024-09-10T16:05:25.309452+0000","mapping_epoch":35,"log_start":"43'5100","ondisk_log_start":"43'5100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23266246,"num_objects":5682,"num_object_clones":0,"num_object_copies":11364,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5682,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13269,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'15210","reported_seq":23893,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:27.482711+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:05:27.482711+0000","last_peered":"2024-09-10T16:05:27.482711+0000","last_clean":"2024-09-10T16:05:27.482711+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:05:27.482711+0000","last_undegraded":"2024-09-10T16:05:27.482711+0000","last_fullsized":"2024-09-10T16:05:27.482711+0000","mapping_epoch":44,"log_start":"43'5200","ondisk_log_start":"43'5200","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22884376,"num_objects":5588,"num_object_clones":0,"num_object_copies":11176,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5588,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13214,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'15071","reported_seq":15157,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:25.319140+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:05:25.319140+0000","last_peered":"2024-09-10T16:05:25.319140+0000","last_clean":"2024-09-10T16:05:25.319140+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:05:25.319140+0000","last_undegraded":"2024-09-10T16:05:25.319140+0000","last_fullsized":"2024-09-10T16:05:25.319140+0000","mapping_epoch":35,"log_start":"43'5000","ondisk_log_start":"43'5000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22892544,"num_objects":5589,"num_object_clones":0,"num_object_copies":11178,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5589,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15071,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'15178","reported_seq":15313,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:24.393595+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:05:24.393595+0000","last_peered":"2024-09-10T16:05:24.393595+0000","last_clean":"2024-09-10T16:05:24.393595+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:05:24.393595+0000","last_undegraded":"2024-09-10T16:05:24.393595+0000","last_fullsized":"2024-09-10T16:05:24.393595+0000","mapping_epoch":34,"log_start":"43'5100","ondisk_log_start":"43'5100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23400931,"num_objects":5714,"num_object_clones":0,"num_object_copies":11428,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5714,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11889,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'15353","reported_seq":15496,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:24.395494+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:05:24.395494+0000","last_peered":"2024-09-10T16:05:24.395494+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:05:24.395494+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"43'5300","ondisk_log_start":"43'5300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23380451,"num_objects":5709,"num_object_clones":0,"num_object_copies":11418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5709,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11965,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1338},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'15204","reported_seq":15347,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:24.377808+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:05:24.377808+0000","last_peered":"2024-09-10T16:05:24.377808+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:05:24.377808+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"43'5200","ondisk_log_start":"43'5200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24077254,"num_objects":5880,"num_object_clones":0,"num_object_copies":11760,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5880,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":11703,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1643},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'15474","reported_seq":15609,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:24.400848+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:05:24.400848+0000","last_peered":"2024-09-10T16:05:24.400848+0000","last_clean":"2024-09-10T16:05:24.400848+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:05:24.400848+0000","last_undegraded":"2024-09-10T16:05:24.400848+0000","last_fullsized":"2024-09-10T16:05:24.400848+0000","mapping_epoch":34,"log_start":"43'5400","ondisk_log_start":"43'5400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23920640,"num_objects":5840,"num_object_clones":0,"num_object_copies":11680,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5840,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12321,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.876895+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:05:19.876895+0000","last_peered":"2024-09-10T16:05:19.876895+0000","last_clean":"2024-09-10T16:05:19.876895+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:05:19.876895+0000","last_undegraded":"2024-09-10T16:05:19.876895+0000","last_fullsized":"2024-09-10T16:05:19.876895+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'14796","reported_seq":14941,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:25.340613+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:05:25.340613+0000","last_peered":"2024-09-10T16:05:25.340613+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:05:25.340613+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"43'4700","ondisk_log_start":"43'4700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23187939,"num_objects":5662,"num_object_clones":0,"num_object_copies":11324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5662,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12615,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":1539}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":99,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.875776+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:05:19.875776+0000","last_peered":"2024-09-10T16:05:19.875776+0000","last_clean":"2024-09-10T16:05:19.875776+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:05:19.875776+0000","last_undegraded":"2024-09-10T16:05:19.875776+0000","last_fullsized":"2024-09-10T16:05:19.875776+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'15155","reported_seq":15291,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:24.322985+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:05:24.322985+0000","last_peered":"2024-09-10T16:05:24.322985+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:05:24.322985+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"43'5100","ondisk_log_start":"43'5100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23572480,"num_objects":5755,"num_object_clones":0,"num_object_copies":11510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5755,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11659,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1500}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":109,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.871983+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:05:19.871983+0000","last_peered":"2024-09-10T16:05:19.871983+0000","last_clean":"2024-09-10T16:05:19.871983+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:05:19.871983+0000","last_undegraded":"2024-09-10T16:05:19.871983+0000","last_fullsized":"2024-09-10T16:05:19.871983+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'15152","reported_seq":15292,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:25.282220+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:05:25.282220+0000","last_peered":"2024-09-10T16:05:25.282220+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:05:25.282220+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"43'5100","ondisk_log_start":"43'5100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23056867,"num_objects":5630,"num_object_clones":0,"num_object_copies":11260,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5630,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12832,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":1326}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":84,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877110+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:05:19.877110+0000","last_peered":"2024-09-10T16:05:19.877110+0000","last_clean":"2024-09-10T16:05:19.877110+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:05:19.877110+0000","last_undegraded":"2024-09-10T16:05:19.877110+0000","last_fullsized":"2024-09-10T16:05:19.877110+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'15449","reported_seq":15946,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:25.334062+0000","last_change":"2024-09-10T16:05:18.271152+0000","last_active":"2024-09-10T16:05:25.334062+0000","last_peered":"2024-09-10T16:05:25.334062+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:05:25.334062+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"43'5400","ondisk_log_start":"43'5400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23433216,"num_objects":5721,"num_object_clones":0,"num_object_copies":11442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4173,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5721,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13003,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":181,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1548}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'15193","reported_seq":15333,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:24.415859+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:05:24.415859+0000","last_peered":"2024-09-10T16:05:24.415859+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:05:24.415859+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"43'5100","ondisk_log_start":"43'5100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23121920,"num_objects":5645,"num_object_clones":0,"num_object_copies":11290,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5645,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11824,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1341}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877617+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:05:19.877617+0000","last_peered":"2024-09-10T16:05:19.877617+0000","last_clean":"2024-09-10T16:05:19.877617+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:05:19.877617+0000","last_undegraded":"2024-09-10T16:05:19.877617+0000","last_fullsized":"2024-09-10T16:05:19.877617+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'15038","reported_seq":23609,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:27.416392+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:05:27.416392+0000","last_peered":"2024-09-10T16:05:27.416392+0000","last_clean":"2024-09-10T16:05:27.416392+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:05:27.416392+0000","last_undegraded":"2024-09-10T16:05:27.416392+0000","last_fullsized":"2024-09-10T16:05:27.416392+0000","mapping_epoch":46,"log_start":"43'5000","ondisk_log_start":"43'5000","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22773760,"num_objects":5560,"num_object_clones":0,"num_object_copies":11120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5560,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11428,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":99,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.876013+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:05:19.876013+0000","last_peered":"2024-09-10T16:05:19.876013+0000","last_clean":"2024-09-10T16:05:19.876013+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:05:19.876013+0000","last_undegraded":"2024-09-10T16:05:19.876013+0000","last_fullsized":"2024-09-10T16:05:19.876013+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'15373","reported_seq":15520,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:24.363412+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:05:24.363412+0000","last_peered":"2024-09-10T16:05:24.363412+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:05:24.363412+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"43'5300","ondisk_log_start":"43'5300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23244800,"num_objects":5675,"num_object_clones":0,"num_object_copies":11350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5675,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11628,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877315+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:05:19.877315+0000","last_peered":"2024-09-10T16:05:19.877315+0000","last_clean":"2024-09-10T16:05:19.877315+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:05:19.877315+0000","last_undegraded":"2024-09-10T16:05:19.877315+0000","last_fullsized":"2024-09-10T16:05:19.877315+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877229+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:05:19.877229+0000","last_peered":"2024-09-10T16:05:19.877229+0000","last_clean":"2024-09-10T16:05:19.877229+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:05:19.877229+0000","last_undegraded":"2024-09-10T16:05:19.877229+0000","last_fullsized":"2024-09-10T16:05:19.877229+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'15396","reported_seq":15544,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:24.401147+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:05:24.401147+0000","last_peered":"2024-09-10T16:05:24.401147+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:05:24.401147+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"43'5300","ondisk_log_start":"43'5300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23765958,"num_objects":5804,"num_object_clones":0,"num_object_copies":11608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5804,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11559,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1481},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":373007094,"num_objects":91076,"num_object_clones":0,"num_object_copies":182152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42730,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":91076,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":201030,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8659,"num_bytes_recovered":33095680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":953745408,"data_stored":953603583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160956,"ondisk_log_size":160956,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986265,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":635888,"kb_used_data":319584,"kb_used_omap":4,"kb_used_meta":316283,"kb_avail":93735952,"statfs":{"total":96636764160,"available":95985614848,"internally_reserved":0,"allocated":327254016,"data_stored":326951332,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":323874474},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,0,3,4,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659211,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":603692,"kb_used_data":270748,"kb_used_omap":4,"kb_used_meta":332923,"kb_avail":93768148,"statfs":{"total":96636764160,"available":96018583552,"internally_reserved":0,"allocated":277245952,"data_stored":276913900,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":340913838},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,2,5,4,3,1],"upper_bound":512},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182493,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":753876,"kb_used_data":343232,"kb_used_omap":4,"kb_used_meta":410619,"kb_avail":93617964,"statfs":{"total":96636764160,"available":95864795136,"internally_reserved":0,"allocated":351469568,"data_stored":351166879,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":420474538},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,3,5,1,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":350896128,"data_stored":350859998,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":276213760,"data_stored":276148267,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":326635520,"data_stored":326595318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:05:28.169 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-10T16:05:28.169 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "47'15050", 'reported_seq': 15129, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:05:24.417372+0000', 'last_change': '2024-09-10T15:59:16.281558+0000', 'last_active': '2024-09-10T16:05:24.417372+0000', 'last_peered': '2024-09-10T16:05:24.417372+0000', 'last_clean': '2024-09-10T15:59:14.132456+0000', 'last_became_active': '2024-09-10T15:59:16.274407+0000', 'last_became_peered': '2024-09-10T15:59:16.274407+0000', 'last_unstale': '2024-09-10T16:05:24.417372+0000', 'last_undegraded': '2024-09-10T15:59:15.851279+0000', 'last_fullsized': '2024-09-10T15:59:15.774356+0000', 'mapping_epoch': 42, 'log_start': "43'5000", 'ondisk_log_start': "43'5000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'objects_scrubbed': 0, 'log_size': 10050, 'log_dups_size': 0, 'ondisk_log_size': 10050, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23027712, 'num_objects': 5622, 'num_object_clones': 0, 'num_object_copies': 11244, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4259, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5622, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15050, 'num_write_kb': 41344, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1363}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:05:28.169 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-10T16:05:28.169 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "47'15353", 'reported_seq': 15496, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:05:24.395494+0000', 'last_change': '2024-09-10T15:59:16.308264+0000', 'last_active': '2024-09-10T16:05:24.395494+0000', 'last_peered': '2024-09-10T16:05:24.395494+0000', 'last_clean': '2024-09-10T15:59:14.111551+0000', 'last_became_active': '2024-09-10T15:59:16.300040+0000', 'last_became_peered': '2024-09-10T15:59:16.300040+0000', 'last_unstale': '2024-09-10T16:05:24.395494+0000', 'last_undegraded': '2024-09-10T15:59:15.878234+0000', 'last_fullsized': '2024-09-10T15:59:15.769328+0000', 'mapping_epoch': 42, 'log_start': "43'5300", 'ondisk_log_start': "43'5300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3442", 'last_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'objects_scrubbed': 3384, 'log_size': 10053, 'log_dups_size': 0, 'ondisk_log_size': 10053, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1292, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23380451, 'num_objects': 5709, 'num_object_clones': 0, 'num_object_copies': 11418, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4372, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5709, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 11965, 'num_write_kb': 28580, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1338}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:05:28.169 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-10T16:05:28.169 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "47'15204", 'reported_seq': 15347, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:05:24.377808+0000', 'last_change': '2024-09-10T15:59:16.331899+0000', 'last_active': '2024-09-10T16:05:24.377808+0000', 'last_peered': '2024-09-10T16:05:24.377808+0000', 'last_clean': '2024-09-10T15:59:14.139436+0000', 'last_became_active': '2024-09-10T15:59:16.318715+0000', 'last_became_peered': '2024-09-10T15:59:16.318715+0000', 'last_unstale': '2024-09-10T16:05:24.377808+0000', 'last_undegraded': '2024-09-10T15:59:15.877524+0000', 'last_fullsized': '2024-09-10T15:59:15.769042+0000', 'mapping_epoch': 42, 'log_start': "43'5200", 'ondisk_log_start': "43'5200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3537", 'last_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'objects_scrubbed': 3497, 'log_size': 10004, 'log_dups_size': 0, 'ondisk_log_size': 10004, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1224, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24077254, 'num_objects': 5880, 'num_object_clones': 0, 'num_object_copies': 11760, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4237, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5880, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 11703, 'num_write_kb': 28172, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1643}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:05:28.169 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-10T16:05:28.169 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "47'14796", 'reported_seq': 14941, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:05:25.340613+0000', 'last_change': '2024-09-10T15:59:16.175680+0000', 'last_active': '2024-09-10T16:05:25.340613+0000', 'last_peered': '2024-09-10T16:05:25.340613+0000', 'last_clean': '2024-09-10T15:59:14.088187+0000', 'last_became_active': '2024-09-10T15:59:16.165810+0000', 'last_became_peered': '2024-09-10T15:59:16.165810+0000', 'last_unstale': '2024-09-10T16:05:25.340613+0000', 'last_undegraded': '2024-09-10T15:59:15.822098+0000', 'last_fullsized': '2024-09-10T15:59:15.769123+0000', 'mapping_epoch': 42, 'log_start': "43'4700", 'ondisk_log_start': "43'4700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2218", 'last_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'objects_scrubbed': 2175, 'log_size': 10096, 'log_dups_size': 0, 'ondisk_log_size': 10096, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 825, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23187939, 'num_objects': 5662, 'num_object_clones': 0, 'num_object_copies': 11324, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4124, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5662, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 12615, 'num_write_kb': 32204, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': 1539}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:05:28.170 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-10T16:05:28.170 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "47'15155", 'reported_seq': 15291, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:05:24.322985+0000', 'last_change': '2024-09-10T15:59:16.277742+0000', 'last_active': '2024-09-10T16:05:24.322985+0000', 'last_peered': '2024-09-10T16:05:24.322985+0000', 'last_clean': '2024-09-10T15:59:14.129081+0000', 'last_became_active': '2024-09-10T15:59:16.269835+0000', 'last_became_peered': '2024-09-10T15:59:16.269835+0000', 'last_unstale': '2024-09-10T16:05:24.322985+0000', 'last_undegraded': '2024-09-10T15:59:15.860857+0000', 'last_fullsized': '2024-09-10T15:59:15.771410+0000', 'mapping_epoch': 42, 'log_start': "43'5100", 'ondisk_log_start': "43'5100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3557", 'last_scrub_stamp': '2024-09-10T15:59:02.493678+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:02.493678+0000', 'objects_scrubbed': 3488, 'log_size': 10055, 'log_dups_size': 0, 'ondisk_log_size': 10055, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1472, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23572480, 'num_objects': 5755, 'num_object_clones': 0, 'num_object_copies': 11510, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4257, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5755, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11659, 'num_write_kb': 27852, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1500}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:05:28.170 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-10T16:05:28.170 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "47'15152", 'reported_seq': 15292, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:05:25.282220+0000', 'last_change': '2024-09-10T15:59:16.284311+0000', 'last_active': '2024-09-10T16:05:25.282220+0000', 'last_peered': '2024-09-10T16:05:25.282220+0000', 'last_clean': '2024-09-10T15:59:14.116041+0000', 'last_became_active': '2024-09-10T15:59:16.274543+0000', 'last_became_peered': '2024-09-10T15:59:16.274543+0000', 'last_unstale': '2024-09-10T16:05:25.282220+0000', 'last_undegraded': '2024-09-10T15:59:15.812484+0000', 'last_fullsized': '2024-09-10T15:59:15.770885+0000', 'mapping_epoch': 42, 'log_start': "43'5100", 'ondisk_log_start': "43'5100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2356", 'last_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'objects_scrubbed': 2316, 'log_size': 10052, 'log_dups_size': 0, 'ondisk_log_size': 10052, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 783, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23056867, 'num_objects': 5630, 'num_object_clones': 0, 'num_object_copies': 11260, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5630, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 12832, 'num_write_kb': 32292, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': 1326}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:05:28.170 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-10T16:05:28.170 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "47'15449", 'reported_seq': 15946, 'reported_epoch': 47, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:05:25.334062+0000', 'last_change': '2024-09-10T16:05:18.271152+0000', 'last_active': '2024-09-10T16:05:25.334062+0000', 'last_peered': '2024-09-10T16:05:25.334062+0000', 'last_clean': '2024-09-10T15:59:14.110654+0000', 'last_became_active': '2024-09-10T15:59:16.164645+0000', 'last_became_peered': '2024-09-10T15:59:16.164645+0000', 'last_unstale': '2024-09-10T16:05:25.334062+0000', 'last_undegraded': '2024-09-10T15:59:15.803593+0000', 'last_fullsized': '2024-09-10T15:59:15.769083+0000', 'mapping_epoch': 42, 'log_start': "43'5400", 'ondisk_log_start': "43'5400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2486", 'last_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'objects_scrubbed': 2440, 'log_size': 10049, 'log_dups_size': 0, 'ondisk_log_size': 10049, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 792, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23433216, 'num_objects': 5721, 'num_object_clones': 0, 'num_object_copies': 11442, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4173, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5721, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13003, 'num_write_kb': 32568, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 181, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 1548}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:05:28.170 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-10T16:05:28.170 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "47'15193", 'reported_seq': 15333, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:05:24.415859+0000', 'last_change': '2024-09-10T15:59:16.315641+0000', 'last_active': '2024-09-10T16:05:24.415859+0000', 'last_peered': '2024-09-10T16:05:24.415859+0000', 'last_clean': '2024-09-10T15:59:14.117001+0000', 'last_became_active': '2024-09-10T15:59:16.302245+0000', 'last_became_peered': '2024-09-10T15:59:16.302245+0000', 'last_unstale': '2024-09-10T16:05:24.415859+0000', 'last_undegraded': '2024-09-10T15:59:15.845012+0000', 'last_fullsized': '2024-09-10T15:59:15.770337+0000', 'mapping_epoch': 42, 'log_start': "43'5100", 'ondisk_log_start': "43'5100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3404", 'last_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'objects_scrubbed': 3367, 'log_size': 10093, 'log_dups_size': 0, 'ondisk_log_size': 10093, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 983, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23121920, 'num_objects': 5645, 'num_object_clones': 0, 'num_object_copies': 11290, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5645, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11824, 'num_write_kb': 28204, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1341}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:05:28.170 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-10T16:05:28.170 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "47'15373", 'reported_seq': 15520, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:05:24.363412+0000', 'last_change': '2024-09-10T15:59:16.311291+0000', 'last_active': '2024-09-10T16:05:24.363412+0000', 'last_peered': '2024-09-10T16:05:24.363412+0000', 'last_clean': '2024-09-10T15:59:14.116363+0000', 'last_became_active': '2024-09-10T15:59:16.301244+0000', 'last_became_peered': '2024-09-10T15:59:16.301244+0000', 'last_unstale': '2024-09-10T16:05:24.363412+0000', 'last_undegraded': '2024-09-10T15:59:15.883434+0000', 'last_fullsized': '2024-09-10T15:59:15.772756+0000', 'mapping_epoch': 42, 'log_start': "43'5300", 'ondisk_log_start': "43'5300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3789", 'last_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'objects_scrubbed': 3741, 'log_size': 10073, 'log_dups_size': 0, 'ondisk_log_size': 10073, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1440, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23244800, 'num_objects': 5675, 'num_object_clones': 0, 'num_object_copies': 11350, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4373, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5675, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11628, 'num_write_kb': 27124, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1304}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:05:28.171 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-10T16:05:28.171 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "47'15396", 'reported_seq': 15544, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:05:24.401147+0000', 'last_change': '2024-09-10T15:59:16.313409+0000', 'last_active': '2024-09-10T16:05:24.401147+0000', 'last_peered': '2024-09-10T16:05:24.401147+0000', 'last_clean': '2024-09-10T15:59:14.129780+0000', 'last_became_active': '2024-09-10T15:59:16.300932+0000', 'last_became_peered': '2024-09-10T15:59:16.300932+0000', 'last_unstale': '2024-09-10T16:05:24.401147+0000', 'last_undegraded': '2024-09-10T15:59:15.887823+0000', 'last_fullsized': '2024-09-10T15:59:15.768513+0000', 'mapping_epoch': 42, 'log_start': "43'5300", 'ondisk_log_start': "43'5300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3880", 'last_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'objects_scrubbed': 3827, 'log_size': 10096, 'log_dups_size': 0, 'ondisk_log_size': 10096, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1397, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23765958, 'num_objects': 5804, 'num_object_clones': 0, 'num_object_copies': 11608, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4325, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5804, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11559, 'num_write_kb': 27072, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1481}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:05:28.171 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T16:05:28.875 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:05:28.875 INFO:teuthology.orchestra.run.smithi115.stdout:{"fsid":"7effee29-df05-4351-9741-ee203564ed81","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 42730/182156 objects degraded (23.458%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":492,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":47,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983954,"num_in_osds":3,"osd_in_since":1725983947,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":91078,"data_bytes":373007113,"bytes_used":2041298944,"bytes_avail":287868993536,"bytes_total":289910292480,"degraded_objects":42730,"degraded_total":182156,"degraded_ratio":0.23457915193570347,"recovering_objects_per_sec":35,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":310,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":499},"fsmap":{"epoch":1,"btime":"2024-09-10T15:57:14:967120+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T16:00:13.560776+0000","services":{}},"progress_events":{}} 2024-09-10T16:05:28.911 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-10T16:05:28.911 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:05:29.473 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:05:29.473 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:05:29.497 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":286,"stamp":"2024-09-10T16:05:27.685700+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":373007113,"num_objects":91078,"num_object_clones":0,"num_object_copies":182156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42730,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":91078,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":201032,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8661,"num_bytes_recovered":33095699,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160959,"ondisk_log_size":160959,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1993456,"kb_used_data":933564,"kb_used_omap":12,"kb_used_meta":1059827,"kb_avail":281122064,"statfs":{"total":289910292480,"available":287868993536,"internally_reserved":0,"allocated":955969536,"data_stored":955032111,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1085262850},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,5,5,13,9,7,1],"upper_bound":512},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18079744,"num_objects":-4414,"num_object_clones":0,"num_object_copies":-8828,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-310,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4414,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4414,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":310,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.828008"},"pg_stats":[{"pgid":"2.f","version":"47'15050","reported_seq":15129,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:24.417372+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:05:24.417372+0000","last_peered":"2024-09-10T16:05:24.417372+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:05:24.417372+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"43'5000","ondisk_log_start":"43'5000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23027712,"num_objects":5622,"num_object_clones":0,"num_object_copies":11244,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5622,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15050,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1363}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'15164","reported_seq":15282,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:25.309452+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:05:25.309452+0000","last_peered":"2024-09-10T16:05:25.309452+0000","last_clean":"2024-09-10T16:05:25.309452+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:05:25.309452+0000","last_undegraded":"2024-09-10T16:05:25.309452+0000","last_fullsized":"2024-09-10T16:05:25.309452+0000","mapping_epoch":35,"log_start":"43'5100","ondisk_log_start":"43'5100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23266246,"num_objects":5682,"num_object_clones":0,"num_object_copies":11364,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5682,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13269,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'15210","reported_seq":23893,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:27.482711+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:05:27.482711+0000","last_peered":"2024-09-10T16:05:27.482711+0000","last_clean":"2024-09-10T16:05:27.482711+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:05:27.482711+0000","last_undegraded":"2024-09-10T16:05:27.482711+0000","last_fullsized":"2024-09-10T16:05:27.482711+0000","mapping_epoch":44,"log_start":"43'5200","ondisk_log_start":"43'5200","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22884376,"num_objects":5588,"num_object_clones":0,"num_object_copies":11176,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5588,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13214,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'15071","reported_seq":15157,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:25.319140+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:05:25.319140+0000","last_peered":"2024-09-10T16:05:25.319140+0000","last_clean":"2024-09-10T16:05:25.319140+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:05:25.319140+0000","last_undegraded":"2024-09-10T16:05:25.319140+0000","last_fullsized":"2024-09-10T16:05:25.319140+0000","mapping_epoch":35,"log_start":"43'5000","ondisk_log_start":"43'5000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22892544,"num_objects":5589,"num_object_clones":0,"num_object_copies":11178,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5589,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15071,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'15178","reported_seq":15313,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:24.393595+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:05:24.393595+0000","last_peered":"2024-09-10T16:05:24.393595+0000","last_clean":"2024-09-10T16:05:24.393595+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:05:24.393595+0000","last_undegraded":"2024-09-10T16:05:24.393595+0000","last_fullsized":"2024-09-10T16:05:24.393595+0000","mapping_epoch":34,"log_start":"43'5100","ondisk_log_start":"43'5100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23400931,"num_objects":5714,"num_object_clones":0,"num_object_copies":11428,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5714,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11889,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'15353","reported_seq":15496,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:24.395494+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:05:24.395494+0000","last_peered":"2024-09-10T16:05:24.395494+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:05:24.395494+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"43'5300","ondisk_log_start":"43'5300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23380451,"num_objects":5709,"num_object_clones":0,"num_object_copies":11418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5709,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11965,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1338},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'15204","reported_seq":15347,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:24.377808+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:05:24.377808+0000","last_peered":"2024-09-10T16:05:24.377808+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:05:24.377808+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"43'5200","ondisk_log_start":"43'5200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24077254,"num_objects":5880,"num_object_clones":0,"num_object_copies":11760,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5880,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":11703,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1643},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'15474","reported_seq":15609,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:24.400848+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:05:24.400848+0000","last_peered":"2024-09-10T16:05:24.400848+0000","last_clean":"2024-09-10T16:05:24.400848+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:05:24.400848+0000","last_undegraded":"2024-09-10T16:05:24.400848+0000","last_fullsized":"2024-09-10T16:05:24.400848+0000","mapping_epoch":34,"log_start":"43'5400","ondisk_log_start":"43'5400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23920640,"num_objects":5840,"num_object_clones":0,"num_object_copies":11680,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5840,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12321,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.876895+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:05:19.876895+0000","last_peered":"2024-09-10T16:05:19.876895+0000","last_clean":"2024-09-10T16:05:19.876895+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:05:19.876895+0000","last_undegraded":"2024-09-10T16:05:19.876895+0000","last_fullsized":"2024-09-10T16:05:19.876895+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'14796","reported_seq":14941,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:25.340613+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:05:25.340613+0000","last_peered":"2024-09-10T16:05:25.340613+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:05:25.340613+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"43'4700","ondisk_log_start":"43'4700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23187939,"num_objects":5662,"num_object_clones":0,"num_object_copies":11324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5662,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12615,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":1539}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":99,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.875776+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:05:19.875776+0000","last_peered":"2024-09-10T16:05:19.875776+0000","last_clean":"2024-09-10T16:05:19.875776+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:05:19.875776+0000","last_undegraded":"2024-09-10T16:05:19.875776+0000","last_fullsized":"2024-09-10T16:05:19.875776+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'15155","reported_seq":15291,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:24.322985+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:05:24.322985+0000","last_peered":"2024-09-10T16:05:24.322985+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:05:24.322985+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"43'5100","ondisk_log_start":"43'5100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23572480,"num_objects":5755,"num_object_clones":0,"num_object_copies":11510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5755,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11659,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1500}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":109,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.871983+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:05:19.871983+0000","last_peered":"2024-09-10T16:05:19.871983+0000","last_clean":"2024-09-10T16:05:19.871983+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:05:19.871983+0000","last_undegraded":"2024-09-10T16:05:19.871983+0000","last_fullsized":"2024-09-10T16:05:19.871983+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'15152","reported_seq":15292,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:25.282220+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:05:25.282220+0000","last_peered":"2024-09-10T16:05:25.282220+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:05:25.282220+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"43'5100","ondisk_log_start":"43'5100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23056867,"num_objects":5630,"num_object_clones":0,"num_object_copies":11260,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5630,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12832,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":1326}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":84,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877110+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:05:19.877110+0000","last_peered":"2024-09-10T16:05:19.877110+0000","last_clean":"2024-09-10T16:05:19.877110+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:05:19.877110+0000","last_undegraded":"2024-09-10T16:05:19.877110+0000","last_fullsized":"2024-09-10T16:05:19.877110+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'15449","reported_seq":15946,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:25.334062+0000","last_change":"2024-09-10T16:05:18.271152+0000","last_active":"2024-09-10T16:05:25.334062+0000","last_peered":"2024-09-10T16:05:25.334062+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:05:25.334062+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"43'5400","ondisk_log_start":"43'5400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23433216,"num_objects":5721,"num_object_clones":0,"num_object_copies":11442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4173,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5721,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13003,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":181,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1548}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'15193","reported_seq":15333,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:24.415859+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:05:24.415859+0000","last_peered":"2024-09-10T16:05:24.415859+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:05:24.415859+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"43'5100","ondisk_log_start":"43'5100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23121920,"num_objects":5645,"num_object_clones":0,"num_object_copies":11290,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5645,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11824,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1341}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877617+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:05:19.877617+0000","last_peered":"2024-09-10T16:05:19.877617+0000","last_clean":"2024-09-10T16:05:19.877617+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:05:19.877617+0000","last_undegraded":"2024-09-10T16:05:19.877617+0000","last_fullsized":"2024-09-10T16:05:19.877617+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'15038","reported_seq":23609,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:27.416392+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:05:27.416392+0000","last_peered":"2024-09-10T16:05:27.416392+0000","last_clean":"2024-09-10T16:05:27.416392+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:05:27.416392+0000","last_undegraded":"2024-09-10T16:05:27.416392+0000","last_fullsized":"2024-09-10T16:05:27.416392+0000","mapping_epoch":46,"log_start":"43'5000","ondisk_log_start":"43'5000","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22773760,"num_objects":5560,"num_object_clones":0,"num_object_copies":11120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5560,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11428,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":99,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.876013+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:05:19.876013+0000","last_peered":"2024-09-10T16:05:19.876013+0000","last_clean":"2024-09-10T16:05:19.876013+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:05:19.876013+0000","last_undegraded":"2024-09-10T16:05:19.876013+0000","last_fullsized":"2024-09-10T16:05:19.876013+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'15373","reported_seq":15520,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:24.363412+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:05:24.363412+0000","last_peered":"2024-09-10T16:05:24.363412+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:05:24.363412+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"43'5300","ondisk_log_start":"43'5300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23244800,"num_objects":5675,"num_object_clones":0,"num_object_copies":11350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5675,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11628,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877315+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:05:19.877315+0000","last_peered":"2024-09-10T16:05:19.877315+0000","last_clean":"2024-09-10T16:05:19.877315+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:05:19.877315+0000","last_undegraded":"2024-09-10T16:05:19.877315+0000","last_fullsized":"2024-09-10T16:05:19.877315+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877229+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:05:19.877229+0000","last_peered":"2024-09-10T16:05:19.877229+0000","last_clean":"2024-09-10T16:05:19.877229+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:05:19.877229+0000","last_undegraded":"2024-09-10T16:05:19.877229+0000","last_fullsized":"2024-09-10T16:05:19.877229+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'15396","reported_seq":15544,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:24.401147+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:05:24.401147+0000","last_peered":"2024-09-10T16:05:24.401147+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:05:24.401147+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"43'5300","ondisk_log_start":"43'5300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23765958,"num_objects":5804,"num_object_clones":0,"num_object_copies":11608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5804,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11559,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1481},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":373007094,"num_objects":91076,"num_object_clones":0,"num_object_copies":182152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42730,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":91076,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":201030,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8659,"num_bytes_recovered":33095680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":953745408,"data_stored":953603583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160956,"ondisk_log_size":160956,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986265,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":635888,"kb_used_data":319584,"kb_used_omap":4,"kb_used_meta":316283,"kb_avail":93735952,"statfs":{"total":96636764160,"available":95985614848,"internally_reserved":0,"allocated":327254016,"data_stored":326951332,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":323874474},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,0,3,4,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659211,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":603692,"kb_used_data":270748,"kb_used_omap":4,"kb_used_meta":332923,"kb_avail":93768148,"statfs":{"total":96636764160,"available":96018583552,"internally_reserved":0,"allocated":277245952,"data_stored":276913900,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":340913838},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,2,5,4,3,1],"upper_bound":512},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182493,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":753876,"kb_used_data":343232,"kb_used_omap":4,"kb_used_meta":410619,"kb_avail":93617964,"statfs":{"total":96636764160,"available":95864795136,"internally_reserved":0,"allocated":351469568,"data_stored":351166879,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":420474538},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,3,5,1,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":350896128,"data_stored":350859998,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":276213760,"data_stored":276148267,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":326635520,"data_stored":326595318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:05:32.500 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:05:33.077 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:05:33.080 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:05:33.104 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":288,"stamp":"2024-09-10T16:05:31.688246+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":364696329,"num_objects":89049,"num_object_clones":0,"num_object_copies":178098,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42632,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":89049,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":203061,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8759,"num_bytes_recovered":33095699,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160688,"ondisk_log_size":160688,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1991272,"kb_used_data":923048,"kb_used_omap":12,"kb_used_meta":1068147,"kb_avail":281124248,"statfs":{"total":289910292480,"available":287871229952,"internally_reserved":0,"allocated":945201152,"data_stored":944263727,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1093782530},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,1,1,4,8,10,6,1],"upper_bound":512},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-21118976,"num_objects":-5156,"num_object_clones":0,"num_object_copies":-10312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5156,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5156,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":408,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"11.817011"},"pg_stats":[{"pgid":"2.f","version":"47'15195","reported_seq":15274,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:29.474536+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:05:29.474536+0000","last_peered":"2024-09-10T16:05:29.474536+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:05:29.474536+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"43'5100","ondisk_log_start":"43'5100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22433792,"num_objects":5477,"num_object_clones":0,"num_object_copies":10954,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5477,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15195,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1218}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'15321","reported_seq":15439,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:30.304030+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:05:30.304030+0000","last_peered":"2024-09-10T16:05:30.304030+0000","last_clean":"2024-09-10T16:05:30.304030+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:05:30.304030+0000","last_undegraded":"2024-09-10T16:05:30.304030+0000","last_fullsized":"2024-09-10T16:05:30.304030+0000","mapping_epoch":35,"log_start":"43'5300","ondisk_log_start":"43'5300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22623174,"num_objects":5525,"num_object_clones":0,"num_object_copies":11050,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5525,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13426,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'15210","reported_seq":23893,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:27.482711+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:05:27.482711+0000","last_peered":"2024-09-10T16:05:27.482711+0000","last_clean":"2024-09-10T16:05:27.482711+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:05:27.482711+0000","last_undegraded":"2024-09-10T16:05:27.482711+0000","last_fullsized":"2024-09-10T16:05:27.482711+0000","mapping_epoch":44,"log_start":"43'5200","ondisk_log_start":"43'5200","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22884376,"num_objects":5588,"num_object_clones":0,"num_object_copies":11176,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5588,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13214,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'15220","reported_seq":15306,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:30.368638+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:05:30.368638+0000","last_peered":"2024-09-10T16:05:30.368638+0000","last_clean":"2024-09-10T16:05:30.368638+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:05:30.368638+0000","last_undegraded":"2024-09-10T16:05:30.368638+0000","last_fullsized":"2024-09-10T16:05:30.368638+0000","mapping_epoch":35,"log_start":"43'5200","ondisk_log_start":"43'5200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22282240,"num_objects":5440,"num_object_clones":0,"num_object_copies":10880,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5440,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15220,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'15336","reported_seq":15471,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:29.298313+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:05:29.298313+0000","last_peered":"2024-09-10T16:05:29.298313+0000","last_clean":"2024-09-10T16:05:29.298313+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:05:29.298313+0000","last_undegraded":"2024-09-10T16:05:29.298313+0000","last_fullsized":"2024-09-10T16:05:29.298313+0000","mapping_epoch":34,"log_start":"43'5300","ondisk_log_start":"43'5300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22753763,"num_objects":5556,"num_object_clones":0,"num_object_copies":11112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5556,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12047,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'15504","reported_seq":15646,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:29.472434+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:05:29.472434+0000","last_peered":"2024-09-10T16:05:29.472434+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:05:29.472434+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"43'5500","ondisk_log_start":"43'5500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22761955,"num_objects":5558,"num_object_clones":0,"num_object_copies":11116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5558,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12116,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1187},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'15347","reported_seq":15490,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:29.439838+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:05:29.439838+0000","last_peered":"2024-09-10T16:05:29.439838+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:05:29.439838+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"43'5300","ondisk_log_start":"43'5300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23491526,"num_objects":5737,"num_object_clones":0,"num_object_copies":11474,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5737,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":11846,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1500},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'15632","reported_seq":15767,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:29.394660+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:05:29.394660+0000","last_peered":"2024-09-10T16:05:29.394660+0000","last_clean":"2024-09-10T16:05:29.394660+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:05:29.394660+0000","last_undegraded":"2024-09-10T16:05:29.394660+0000","last_fullsized":"2024-09-10T16:05:29.394660+0000","mapping_epoch":34,"log_start":"43'5600","ondisk_log_start":"43'5600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23273472,"num_objects":5682,"num_object_clones":0,"num_object_copies":11364,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5682,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12479,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.876895+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:05:19.876895+0000","last_peered":"2024-09-10T16:05:19.876895+0000","last_clean":"2024-09-10T16:05:19.876895+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:05:19.876895+0000","last_undegraded":"2024-09-10T16:05:19.876895+0000","last_fullsized":"2024-09-10T16:05:19.876895+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'14934","reported_seq":15079,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:30.353205+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:05:30.353205+0000","last_peered":"2024-09-10T16:05:30.353205+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:05:30.353205+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"43'4900","ondisk_log_start":"43'4900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22622691,"num_objects":5524,"num_object_clones":0,"num_object_copies":11048,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5524,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12753,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":1401}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":99,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.875776+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:05:19.875776+0000","last_peered":"2024-09-10T16:05:19.875776+0000","last_clean":"2024-09-10T16:05:19.875776+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:05:19.875776+0000","last_undegraded":"2024-09-10T16:05:19.875776+0000","last_fullsized":"2024-09-10T16:05:19.875776+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'15282","reported_seq":15418,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:29.404056+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:05:29.404056+0000","last_peered":"2024-09-10T16:05:29.404056+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:05:29.404056+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"43'5200","ondisk_log_start":"43'5200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23052288,"num_objects":5628,"num_object_clones":0,"num_object_copies":11256,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5628,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11786,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1373}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":109,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.871983+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:05:19.871983+0000","last_peered":"2024-09-10T16:05:19.871983+0000","last_clean":"2024-09-10T16:05:19.871983+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:05:19.871983+0000","last_undegraded":"2024-09-10T16:05:19.871983+0000","last_fullsized":"2024-09-10T16:05:19.871983+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'15279","reported_seq":15417,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:30.376659+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:05:30.376659+0000","last_peered":"2024-09-10T16:05:30.376659+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:05:30.376659+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"43'5200","ondisk_log_start":"43'5200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22536675,"num_objects":5503,"num_object_clones":0,"num_object_copies":11006,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5503,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12959,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":1199}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":84,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877110+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:05:19.877110+0000","last_peered":"2024-09-10T16:05:19.877110+0000","last_clean":"2024-09-10T16:05:19.877110+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:05:19.877110+0000","last_undegraded":"2024-09-10T16:05:19.877110+0000","last_fullsized":"2024-09-10T16:05:19.877110+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'15583","reported_seq":16276,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:30.352835+0000","last_change":"2024-09-10T16:05:18.271152+0000","last_active":"2024-09-10T16:05:30.352835+0000","last_peered":"2024-09-10T16:05:30.352835+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:05:30.352835+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"43'5500","ondisk_log_start":"43'5500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22884352,"num_objects":5587,"num_object_clones":0,"num_object_copies":11174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4075,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5587,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13137,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":279,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1512}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'15325","reported_seq":15465,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:29.234871+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:05:29.234871+0000","last_peered":"2024-09-10T16:05:29.234871+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:05:29.234871+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"43'5300","ondisk_log_start":"43'5300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22581248,"num_objects":5513,"num_object_clones":0,"num_object_copies":11026,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5513,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11956,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1209}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877617+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:05:19.877617+0000","last_peered":"2024-09-10T16:05:19.877617+0000","last_clean":"2024-09-10T16:05:19.877617+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:05:19.877617+0000","last_undegraded":"2024-09-10T16:05:19.877617+0000","last_fullsized":"2024-09-10T16:05:19.877617+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'15038","reported_seq":23609,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:27.416392+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:05:27.416392+0000","last_peered":"2024-09-10T16:05:27.416392+0000","last_clean":"2024-09-10T16:05:27.416392+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:05:27.416392+0000","last_undegraded":"2024-09-10T16:05:27.416392+0000","last_fullsized":"2024-09-10T16:05:27.416392+0000","mapping_epoch":46,"log_start":"43'5000","ondisk_log_start":"43'5000","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22773760,"num_objects":5560,"num_object_clones":0,"num_object_copies":11120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5560,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11428,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":99,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.876013+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:05:19.876013+0000","last_peered":"2024-09-10T16:05:19.876013+0000","last_clean":"2024-09-10T16:05:19.876013+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:05:19.876013+0000","last_undegraded":"2024-09-10T16:05:19.876013+0000","last_fullsized":"2024-09-10T16:05:19.876013+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'15525","reported_seq":15672,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:29.415329+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:05:29.415329+0000","last_peered":"2024-09-10T16:05:29.415329+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:05:29.415329+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"43'5500","ondisk_log_start":"43'5500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22622208,"num_objects":5523,"num_object_clones":0,"num_object_copies":11046,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5523,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11780,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1152}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877315+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:05:19.877315+0000","last_peered":"2024-09-10T16:05:19.877315+0000","last_clean":"2024-09-10T16:05:19.877315+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:05:19.877315+0000","last_undegraded":"2024-09-10T16:05:19.877315+0000","last_fullsized":"2024-09-10T16:05:19.877315+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877229+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:05:19.877229+0000","last_peered":"2024-09-10T16:05:19.877229+0000","last_clean":"2024-09-10T16:05:19.877229+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:05:19.877229+0000","last_undegraded":"2024-09-10T16:05:19.877229+0000","last_fullsized":"2024-09-10T16:05:19.877229+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'15554","reported_seq":15704,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:29.395029+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:05:29.395029+0000","last_peered":"2024-09-10T16:05:29.395029+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:05:29.395029+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"43'5500","ondisk_log_start":"43'5500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23118790,"num_objects":5646,"num_object_clones":0,"num_object_copies":11292,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5646,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11717,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1323},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":364696310,"num_objects":89047,"num_object_clones":0,"num_object_copies":178094,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42632,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":89047,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":203059,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8757,"num_bytes_recovered":33095680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":941645824,"data_stored":941503999,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160685,"ondisk_log_size":160685,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986266,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":635424,"kb_used_data":314952,"kb_used_omap":4,"kb_used_meta":320443,"kb_avail":93736416,"statfs":{"total":96636764160,"available":95986089984,"internally_reserved":0,"allocated":322510848,"data_stored":322208164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":328134314},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659211,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":603692,"kb_used_data":270748,"kb_used_omap":4,"kb_used_meta":332923,"kb_avail":93768148,"statfs":{"total":96636764160,"available":96018583552,"internally_reserved":0,"allocated":277245952,"data_stored":276913900,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":340913838},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,2,5,4,3,1],"upper_bound":512},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182494,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":752156,"kb_used_data":337348,"kb_used_omap":4,"kb_used_meta":414779,"kb_avail":93619684,"statfs":{"total":96636764160,"available":95866556416,"internally_reserved":0,"allocated":345444352,"data_stored":345141663,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":424734378},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,1,0,2,1,6,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":343629824,"data_stored":343593694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":276213760,"data_stored":276148267,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":321802240,"data_stored":321762038,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:05:33.106 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:05:33.695 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:05:33.697 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:05:33.716 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":289,"stamp":"2024-09-10T16:05:33.689060+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":363500297,"num_objects":88757,"num_object_clones":0,"num_object_copies":177514,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42632,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":88757,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":203353,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8759,"num_bytes_recovered":33095699,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160780,"ondisk_log_size":160780,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1972152,"kb_used_data":916408,"kb_used_omap":12,"kb_used_meta":1055667,"kb_avail":281143368,"statfs":{"total":289910292480,"available":287890808832,"internally_reserved":0,"allocated":938401792,"data_stored":937464367,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1081003010},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,2,1,4,9,11,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19746816,"num_objects":-4821,"num_object_clones":0,"num_object_copies":-9642,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-318,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4821,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4821,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":318,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005552"},"pg_stats":[{"pgid":"2.f","version":"47'15195","reported_seq":15274,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:29.474536+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:05:29.474536+0000","last_peered":"2024-09-10T16:05:29.474536+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:05:29.474536+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"43'5100","ondisk_log_start":"43'5100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22433792,"num_objects":5477,"num_object_clones":0,"num_object_copies":10954,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5477,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15195,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1218}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'15321","reported_seq":15439,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:30.304030+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:05:30.304030+0000","last_peered":"2024-09-10T16:05:30.304030+0000","last_clean":"2024-09-10T16:05:30.304030+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:05:30.304030+0000","last_undegraded":"2024-09-10T16:05:30.304030+0000","last_fullsized":"2024-09-10T16:05:30.304030+0000","mapping_epoch":35,"log_start":"43'5300","ondisk_log_start":"43'5300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22623174,"num_objects":5525,"num_object_clones":0,"num_object_copies":11050,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5525,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13426,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'15351","reported_seq":24034,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:32.432961+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:05:32.432961+0000","last_peered":"2024-09-10T16:05:32.432961+0000","last_clean":"2024-09-10T16:05:32.432961+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:05:32.432961+0000","last_undegraded":"2024-09-10T16:05:32.432961+0000","last_fullsized":"2024-09-10T16:05:32.432961+0000","mapping_epoch":44,"log_start":"43'5300","ondisk_log_start":"43'5300","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22306840,"num_objects":5447,"num_object_clones":0,"num_object_copies":10894,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5447,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13355,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'15220","reported_seq":15306,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:30.368638+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:05:30.368638+0000","last_peered":"2024-09-10T16:05:30.368638+0000","last_clean":"2024-09-10T16:05:30.368638+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:05:30.368638+0000","last_undegraded":"2024-09-10T16:05:30.368638+0000","last_fullsized":"2024-09-10T16:05:30.368638+0000","mapping_epoch":35,"log_start":"43'5200","ondisk_log_start":"43'5200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22282240,"num_objects":5440,"num_object_clones":0,"num_object_copies":10880,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5440,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15220,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'15336","reported_seq":15471,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:29.298313+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:05:29.298313+0000","last_peered":"2024-09-10T16:05:29.298313+0000","last_clean":"2024-09-10T16:05:29.298313+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:05:29.298313+0000","last_undegraded":"2024-09-10T16:05:29.298313+0000","last_fullsized":"2024-09-10T16:05:29.298313+0000","mapping_epoch":34,"log_start":"43'5300","ondisk_log_start":"43'5300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22753763,"num_objects":5556,"num_object_clones":0,"num_object_copies":11112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5556,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12047,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'15504","reported_seq":15646,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:29.472434+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:05:29.472434+0000","last_peered":"2024-09-10T16:05:29.472434+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:05:29.472434+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"43'5500","ondisk_log_start":"43'5500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22761955,"num_objects":5558,"num_object_clones":0,"num_object_copies":11116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5558,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12116,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1187},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'15347","reported_seq":15490,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:29.439838+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:05:29.439838+0000","last_peered":"2024-09-10T16:05:29.439838+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:05:29.439838+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"43'5300","ondisk_log_start":"43'5300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23491526,"num_objects":5737,"num_object_clones":0,"num_object_copies":11474,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5737,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":11846,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1500},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'15632","reported_seq":15767,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:29.394660+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:05:29.394660+0000","last_peered":"2024-09-10T16:05:29.394660+0000","last_clean":"2024-09-10T16:05:29.394660+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:05:29.394660+0000","last_undegraded":"2024-09-10T16:05:29.394660+0000","last_fullsized":"2024-09-10T16:05:29.394660+0000","mapping_epoch":34,"log_start":"43'5600","ondisk_log_start":"43'5600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23273472,"num_objects":5682,"num_object_clones":0,"num_object_copies":11364,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5682,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12479,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.876895+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:05:19.876895+0000","last_peered":"2024-09-10T16:05:19.876895+0000","last_clean":"2024-09-10T16:05:19.876895+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:05:19.876895+0000","last_undegraded":"2024-09-10T16:05:19.876895+0000","last_fullsized":"2024-09-10T16:05:19.876895+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'14934","reported_seq":15079,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:30.353205+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:05:30.353205+0000","last_peered":"2024-09-10T16:05:30.353205+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:05:30.353205+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"43'4900","ondisk_log_start":"43'4900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22622691,"num_objects":5524,"num_object_clones":0,"num_object_copies":11048,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5524,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12753,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":1401}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":99,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.875776+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:05:19.875776+0000","last_peered":"2024-09-10T16:05:19.875776+0000","last_clean":"2024-09-10T16:05:19.875776+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:05:19.875776+0000","last_undegraded":"2024-09-10T16:05:19.875776+0000","last_fullsized":"2024-09-10T16:05:19.875776+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'15282","reported_seq":15418,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:29.404056+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:05:29.404056+0000","last_peered":"2024-09-10T16:05:29.404056+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:05:29.404056+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"43'5200","ondisk_log_start":"43'5200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23052288,"num_objects":5628,"num_object_clones":0,"num_object_copies":11256,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5628,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11786,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1373}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":109,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.871983+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:05:19.871983+0000","last_peered":"2024-09-10T16:05:19.871983+0000","last_clean":"2024-09-10T16:05:19.871983+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:05:19.871983+0000","last_undegraded":"2024-09-10T16:05:19.871983+0000","last_fullsized":"2024-09-10T16:05:19.871983+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'15279","reported_seq":15417,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:30.376659+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:05:30.376659+0000","last_peered":"2024-09-10T16:05:30.376659+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:05:30.376659+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"43'5200","ondisk_log_start":"43'5200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22536675,"num_objects":5503,"num_object_clones":0,"num_object_copies":11006,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5503,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12959,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":1199}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877148+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:05:19.877148+0000","last_peered":"2024-09-10T16:05:19.877148+0000","last_clean":"2024-09-10T16:05:19.877148+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:05:19.877148+0000","last_undegraded":"2024-09-10T16:05:19.877148+0000","last_fullsized":"2024-09-10T16:05:19.877148+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'15583","reported_seq":16276,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:30.352835+0000","last_change":"2024-09-10T16:05:18.271152+0000","last_active":"2024-09-10T16:05:30.352835+0000","last_peered":"2024-09-10T16:05:30.352835+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:05:30.352835+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"43'5500","ondisk_log_start":"43'5500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22884352,"num_objects":5587,"num_object_clones":0,"num_object_copies":11174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4075,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5587,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13137,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":279,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1512}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'15325","reported_seq":15465,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:29.234871+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:05:29.234871+0000","last_peered":"2024-09-10T16:05:29.234871+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:05:29.234871+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"43'5300","ondisk_log_start":"43'5300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22581248,"num_objects":5513,"num_object_clones":0,"num_object_copies":11026,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5513,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11956,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1209}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877654+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:05:19.877654+0000","last_peered":"2024-09-10T16:05:19.877654+0000","last_clean":"2024-09-10T16:05:19.877654+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:05:19.877654+0000","last_undegraded":"2024-09-10T16:05:19.877654+0000","last_fullsized":"2024-09-10T16:05:19.877654+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'15189","reported_seq":23760,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:32.538810+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:05:32.538810+0000","last_peered":"2024-09-10T16:05:32.538810+0000","last_clean":"2024-09-10T16:05:32.538810+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:05:32.538810+0000","last_undegraded":"2024-09-10T16:05:32.538810+0000","last_fullsized":"2024-09-10T16:05:32.538810+0000","mapping_epoch":46,"log_start":"43'5100","ondisk_log_start":"43'5100","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22155264,"num_objects":5409,"num_object_clones":0,"num_object_copies":10818,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5409,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11579,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":99,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.876013+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:05:19.876013+0000","last_peered":"2024-09-10T16:05:19.876013+0000","last_clean":"2024-09-10T16:05:19.876013+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:05:19.876013+0000","last_undegraded":"2024-09-10T16:05:19.876013+0000","last_fullsized":"2024-09-10T16:05:19.876013+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'15525","reported_seq":15672,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:29.415329+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:05:29.415329+0000","last_peered":"2024-09-10T16:05:29.415329+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:05:29.415329+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"43'5500","ondisk_log_start":"43'5500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22622208,"num_objects":5523,"num_object_clones":0,"num_object_copies":11046,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5523,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11780,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1152}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877349+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:05:19.877349+0000","last_peered":"2024-09-10T16:05:19.877349+0000","last_clean":"2024-09-10T16:05:19.877349+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:05:19.877349+0000","last_undegraded":"2024-09-10T16:05:19.877349+0000","last_fullsized":"2024-09-10T16:05:19.877349+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877229+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:05:19.877229+0000","last_peered":"2024-09-10T16:05:19.877229+0000","last_clean":"2024-09-10T16:05:19.877229+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:05:19.877229+0000","last_undegraded":"2024-09-10T16:05:19.877229+0000","last_fullsized":"2024-09-10T16:05:19.877229+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'15554","reported_seq":15704,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:29.395029+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:05:29.395029+0000","last_peered":"2024-09-10T16:05:29.395029+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:05:29.395029+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"43'5500","ondisk_log_start":"43'5500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23118790,"num_objects":5646,"num_object_clones":0,"num_object_copies":11292,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5646,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11717,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1323},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":363500278,"num_objects":88755,"num_object_clones":0,"num_object_copies":177510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42632,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":88755,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":203351,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8757,"num_bytes_recovered":33095680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":934600704,"data_stored":934458879,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160777,"ondisk_log_size":160777,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986266,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":635424,"kb_used_data":314952,"kb_used_omap":4,"kb_used_meta":320443,"kb_avail":93736416,"statfs":{"total":96636764160,"available":95986089984,"internally_reserved":0,"allocated":322510848,"data_stored":322208164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":328134314},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659212,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":584572,"kb_used_data":264108,"kb_used_omap":4,"kb_used_meta":320443,"kb_avail":93787268,"statfs":{"total":96636764160,"available":96038162432,"internally_reserved":0,"allocated":270446592,"data_stored":270114540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":328134318},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,2,6,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182494,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":752156,"kb_used_data":337348,"kb_used_omap":4,"kb_used_meta":414779,"kb_avail":93619684,"statfs":{"total":96636764160,"available":95866556416,"internally_reserved":0,"allocated":345444352,"data_stored":345141663,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":424734378},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,1,0,2,1,6,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":343629824,"data_stored":343593694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":269168640,"data_stored":269103147,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":321802240,"data_stored":321762038,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:05:33.717 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-10T16:05:33.717 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "47'15195", 'reported_seq': 15274, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:05:29.474536+0000', 'last_change': '2024-09-10T15:59:16.281558+0000', 'last_active': '2024-09-10T16:05:29.474536+0000', 'last_peered': '2024-09-10T16:05:29.474536+0000', 'last_clean': '2024-09-10T15:59:14.132456+0000', 'last_became_active': '2024-09-10T15:59:16.274407+0000', 'last_became_peered': '2024-09-10T15:59:16.274407+0000', 'last_unstale': '2024-09-10T16:05:29.474536+0000', 'last_undegraded': '2024-09-10T15:59:15.851279+0000', 'last_fullsized': '2024-09-10T15:59:15.774356+0000', 'mapping_epoch': 42, 'log_start': "43'5100", 'ondisk_log_start': "43'5100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'objects_scrubbed': 0, 'log_size': 10095, 'log_dups_size': 0, 'ondisk_log_size': 10095, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22433792, 'num_objects': 5477, 'num_object_clones': 0, 'num_object_copies': 10954, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4259, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5477, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15195, 'num_write_kb': 41344, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1218}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:05:33.718 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-10T16:05:33.718 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "47'15504", 'reported_seq': 15646, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:05:29.472434+0000', 'last_change': '2024-09-10T15:59:16.308264+0000', 'last_active': '2024-09-10T16:05:29.472434+0000', 'last_peered': '2024-09-10T16:05:29.472434+0000', 'last_clean': '2024-09-10T15:59:14.111551+0000', 'last_became_active': '2024-09-10T15:59:16.300040+0000', 'last_became_peered': '2024-09-10T15:59:16.300040+0000', 'last_unstale': '2024-09-10T16:05:29.472434+0000', 'last_undegraded': '2024-09-10T15:59:15.878234+0000', 'last_fullsized': '2024-09-10T15:59:15.769328+0000', 'mapping_epoch': 42, 'log_start': "43'5500", 'ondisk_log_start': "43'5500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3442", 'last_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'objects_scrubbed': 3384, 'log_size': 10004, 'log_dups_size': 0, 'ondisk_log_size': 10004, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1292, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22761955, 'num_objects': 5558, 'num_object_clones': 0, 'num_object_copies': 11116, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4372, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5558, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 12116, 'num_write_kb': 28580, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1187}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:05:33.718 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-10T16:05:33.718 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "47'15347", 'reported_seq': 15490, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:05:29.439838+0000', 'last_change': '2024-09-10T15:59:16.331899+0000', 'last_active': '2024-09-10T16:05:29.439838+0000', 'last_peered': '2024-09-10T16:05:29.439838+0000', 'last_clean': '2024-09-10T15:59:14.139436+0000', 'last_became_active': '2024-09-10T15:59:16.318715+0000', 'last_became_peered': '2024-09-10T15:59:16.318715+0000', 'last_unstale': '2024-09-10T16:05:29.439838+0000', 'last_undegraded': '2024-09-10T15:59:15.877524+0000', 'last_fullsized': '2024-09-10T15:59:15.769042+0000', 'mapping_epoch': 42, 'log_start': "43'5300", 'ondisk_log_start': "43'5300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3537", 'last_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'objects_scrubbed': 3497, 'log_size': 10047, 'log_dups_size': 0, 'ondisk_log_size': 10047, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1224, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23491526, 'num_objects': 5737, 'num_object_clones': 0, 'num_object_copies': 11474, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4237, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5737, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 11846, 'num_write_kb': 28172, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1500}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:05:33.718 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-10T16:05:33.718 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "47'14934", 'reported_seq': 15079, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:05:30.353205+0000', 'last_change': '2024-09-10T15:59:16.175680+0000', 'last_active': '2024-09-10T16:05:30.353205+0000', 'last_peered': '2024-09-10T16:05:30.353205+0000', 'last_clean': '2024-09-10T15:59:14.088187+0000', 'last_became_active': '2024-09-10T15:59:16.165810+0000', 'last_became_peered': '2024-09-10T15:59:16.165810+0000', 'last_unstale': '2024-09-10T16:05:30.353205+0000', 'last_undegraded': '2024-09-10T15:59:15.822098+0000', 'last_fullsized': '2024-09-10T15:59:15.769123+0000', 'mapping_epoch': 42, 'log_start': "43'4900", 'ondisk_log_start': "43'4900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2218", 'last_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'objects_scrubbed': 2175, 'log_size': 10034, 'log_dups_size': 0, 'ondisk_log_size': 10034, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 825, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22622691, 'num_objects': 5524, 'num_object_clones': 0, 'num_object_copies': 11048, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4124, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5524, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 12753, 'num_write_kb': 32204, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': 1401}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:05:33.718 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-10T16:05:33.718 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "47'15282", 'reported_seq': 15418, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:05:29.404056+0000', 'last_change': '2024-09-10T15:59:16.277742+0000', 'last_active': '2024-09-10T16:05:29.404056+0000', 'last_peered': '2024-09-10T16:05:29.404056+0000', 'last_clean': '2024-09-10T15:59:14.129081+0000', 'last_became_active': '2024-09-10T15:59:16.269835+0000', 'last_became_peered': '2024-09-10T15:59:16.269835+0000', 'last_unstale': '2024-09-10T16:05:29.404056+0000', 'last_undegraded': '2024-09-10T15:59:15.860857+0000', 'last_fullsized': '2024-09-10T15:59:15.771410+0000', 'mapping_epoch': 42, 'log_start': "43'5200", 'ondisk_log_start': "43'5200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3557", 'last_scrub_stamp': '2024-09-10T15:59:02.493678+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:02.493678+0000', 'objects_scrubbed': 3488, 'log_size': 10082, 'log_dups_size': 0, 'ondisk_log_size': 10082, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1472, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23052288, 'num_objects': 5628, 'num_object_clones': 0, 'num_object_copies': 11256, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4257, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5628, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11786, 'num_write_kb': 27852, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1373}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:05:33.718 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-10T16:05:33.718 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "47'15279", 'reported_seq': 15417, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:05:30.376659+0000', 'last_change': '2024-09-10T15:59:16.284311+0000', 'last_active': '2024-09-10T16:05:30.376659+0000', 'last_peered': '2024-09-10T16:05:30.376659+0000', 'last_clean': '2024-09-10T15:59:14.116041+0000', 'last_became_active': '2024-09-10T15:59:16.274543+0000', 'last_became_peered': '2024-09-10T15:59:16.274543+0000', 'last_unstale': '2024-09-10T16:05:30.376659+0000', 'last_undegraded': '2024-09-10T15:59:15.812484+0000', 'last_fullsized': '2024-09-10T15:59:15.770885+0000', 'mapping_epoch': 42, 'log_start': "43'5200", 'ondisk_log_start': "43'5200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2356", 'last_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'objects_scrubbed': 2316, 'log_size': 10079, 'log_dups_size': 0, 'ondisk_log_size': 10079, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 783, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22536675, 'num_objects': 5503, 'num_object_clones': 0, 'num_object_copies': 11006, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5503, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 12959, 'num_write_kb': 32292, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': 1199}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:05:33.719 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-10T16:05:33.719 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "47'15583", 'reported_seq': 16276, 'reported_epoch': 47, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:05:30.352835+0000', 'last_change': '2024-09-10T16:05:18.271152+0000', 'last_active': '2024-09-10T16:05:30.352835+0000', 'last_peered': '2024-09-10T16:05:30.352835+0000', 'last_clean': '2024-09-10T15:59:14.110654+0000', 'last_became_active': '2024-09-10T15:59:16.164645+0000', 'last_became_peered': '2024-09-10T15:59:16.164645+0000', 'last_unstale': '2024-09-10T16:05:30.352835+0000', 'last_undegraded': '2024-09-10T15:59:15.803593+0000', 'last_fullsized': '2024-09-10T15:59:15.769083+0000', 'mapping_epoch': 42, 'log_start': "43'5500", 'ondisk_log_start': "43'5500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2486", 'last_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'objects_scrubbed': 2440, 'log_size': 10083, 'log_dups_size': 0, 'ondisk_log_size': 10083, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 792, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22884352, 'num_objects': 5587, 'num_object_clones': 0, 'num_object_copies': 11174, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4075, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5587, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13137, 'num_write_kb': 32568, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 279, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 1512}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:05:33.719 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-10T16:05:33.719 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "47'15325", 'reported_seq': 15465, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:05:29.234871+0000', 'last_change': '2024-09-10T15:59:16.315641+0000', 'last_active': '2024-09-10T16:05:29.234871+0000', 'last_peered': '2024-09-10T16:05:29.234871+0000', 'last_clean': '2024-09-10T15:59:14.117001+0000', 'last_became_active': '2024-09-10T15:59:16.302245+0000', 'last_became_peered': '2024-09-10T15:59:16.302245+0000', 'last_unstale': '2024-09-10T16:05:29.234871+0000', 'last_undegraded': '2024-09-10T15:59:15.845012+0000', 'last_fullsized': '2024-09-10T15:59:15.770337+0000', 'mapping_epoch': 42, 'log_start': "43'5300", 'ondisk_log_start': "43'5300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3404", 'last_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'objects_scrubbed': 3367, 'log_size': 10025, 'log_dups_size': 0, 'ondisk_log_size': 10025, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 983, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22581248, 'num_objects': 5513, 'num_object_clones': 0, 'num_object_copies': 11026, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5513, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11956, 'num_write_kb': 28204, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1209}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:05:33.719 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-10T16:05:33.719 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "47'15525", 'reported_seq': 15672, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:05:29.415329+0000', 'last_change': '2024-09-10T15:59:16.311291+0000', 'last_active': '2024-09-10T16:05:29.415329+0000', 'last_peered': '2024-09-10T16:05:29.415329+0000', 'last_clean': '2024-09-10T15:59:14.116363+0000', 'last_became_active': '2024-09-10T15:59:16.301244+0000', 'last_became_peered': '2024-09-10T15:59:16.301244+0000', 'last_unstale': '2024-09-10T16:05:29.415329+0000', 'last_undegraded': '2024-09-10T15:59:15.883434+0000', 'last_fullsized': '2024-09-10T15:59:15.772756+0000', 'mapping_epoch': 42, 'log_start': "43'5500", 'ondisk_log_start': "43'5500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3789", 'last_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'objects_scrubbed': 3741, 'log_size': 10025, 'log_dups_size': 0, 'ondisk_log_size': 10025, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1440, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22622208, 'num_objects': 5523, 'num_object_clones': 0, 'num_object_copies': 11046, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4373, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5523, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11780, 'num_write_kb': 27124, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1152}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:05:33.719 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-10T16:05:33.719 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "47'15554", 'reported_seq': 15704, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:05:29.395029+0000', 'last_change': '2024-09-10T15:59:16.313409+0000', 'last_active': '2024-09-10T16:05:29.395029+0000', 'last_peered': '2024-09-10T16:05:29.395029+0000', 'last_clean': '2024-09-10T15:59:14.129780+0000', 'last_became_active': '2024-09-10T15:59:16.300932+0000', 'last_became_peered': '2024-09-10T15:59:16.300932+0000', 'last_unstale': '2024-09-10T16:05:29.395029+0000', 'last_undegraded': '2024-09-10T15:59:15.887823+0000', 'last_fullsized': '2024-09-10T15:59:15.768513+0000', 'mapping_epoch': 42, 'log_start': "43'5500", 'ondisk_log_start': "43'5500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3880", 'last_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'objects_scrubbed': 3827, 'log_size': 10054, 'log_dups_size': 0, 'ondisk_log_size': 10054, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1397, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23118790, 'num_objects': 5646, 'num_object_clones': 0, 'num_object_copies': 11292, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4325, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5646, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11717, 'num_write_kb': 27072, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1323}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:05:33.719 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T16:05:34.454 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:05:34.455 INFO:teuthology.orchestra.run.smithi115.stdout:{"fsid":"7effee29-df05-4351-9741-ee203564ed81","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 42632/177514 objects degraded (24.016%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":498,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":47,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983954,"num_in_osds":3,"osd_in_since":1725983947,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":88757,"data_bytes":363500297,"bytes_used":2019483648,"bytes_avail":287890808832,"bytes_total":289910292480,"degraded_objects":42632,"degraded_total":177514,"degraded_ratio":0.24016133938731593,"recovering_objects_per_sec":26,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":318,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":401},"fsmap":{"epoch":1,"btime":"2024-09-10T15:57:14:967120+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T16:00:13.560776+0000","services":{}},"progress_events":{}} 2024-09-10T16:05:34.478 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-10T16:05:34.478 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:05:35.044 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:05:35.047 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:05:35.070 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":289,"stamp":"2024-09-10T16:05:33.689060+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":363500297,"num_objects":88757,"num_object_clones":0,"num_object_copies":177514,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42632,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":88757,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":203353,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8759,"num_bytes_recovered":33095699,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160780,"ondisk_log_size":160780,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1972152,"kb_used_data":916408,"kb_used_omap":12,"kb_used_meta":1055667,"kb_avail":281143368,"statfs":{"total":289910292480,"available":287890808832,"internally_reserved":0,"allocated":938401792,"data_stored":937464367,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1081003010},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,2,1,4,9,11,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19746816,"num_objects":-4821,"num_object_clones":0,"num_object_copies":-9642,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-318,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4821,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4821,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":318,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005552"},"pg_stats":[{"pgid":"2.f","version":"47'15195","reported_seq":15274,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:29.474536+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:05:29.474536+0000","last_peered":"2024-09-10T16:05:29.474536+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:05:29.474536+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"43'5100","ondisk_log_start":"43'5100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22433792,"num_objects":5477,"num_object_clones":0,"num_object_copies":10954,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5477,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15195,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1218}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'15321","reported_seq":15439,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:30.304030+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:05:30.304030+0000","last_peered":"2024-09-10T16:05:30.304030+0000","last_clean":"2024-09-10T16:05:30.304030+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:05:30.304030+0000","last_undegraded":"2024-09-10T16:05:30.304030+0000","last_fullsized":"2024-09-10T16:05:30.304030+0000","mapping_epoch":35,"log_start":"43'5300","ondisk_log_start":"43'5300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22623174,"num_objects":5525,"num_object_clones":0,"num_object_copies":11050,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5525,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13426,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'15351","reported_seq":24034,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:32.432961+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:05:32.432961+0000","last_peered":"2024-09-10T16:05:32.432961+0000","last_clean":"2024-09-10T16:05:32.432961+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:05:32.432961+0000","last_undegraded":"2024-09-10T16:05:32.432961+0000","last_fullsized":"2024-09-10T16:05:32.432961+0000","mapping_epoch":44,"log_start":"43'5300","ondisk_log_start":"43'5300","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22306840,"num_objects":5447,"num_object_clones":0,"num_object_copies":10894,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5447,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13355,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'15220","reported_seq":15306,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:30.368638+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:05:30.368638+0000","last_peered":"2024-09-10T16:05:30.368638+0000","last_clean":"2024-09-10T16:05:30.368638+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:05:30.368638+0000","last_undegraded":"2024-09-10T16:05:30.368638+0000","last_fullsized":"2024-09-10T16:05:30.368638+0000","mapping_epoch":35,"log_start":"43'5200","ondisk_log_start":"43'5200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22282240,"num_objects":5440,"num_object_clones":0,"num_object_copies":10880,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5440,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15220,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'15336","reported_seq":15471,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:29.298313+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:05:29.298313+0000","last_peered":"2024-09-10T16:05:29.298313+0000","last_clean":"2024-09-10T16:05:29.298313+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:05:29.298313+0000","last_undegraded":"2024-09-10T16:05:29.298313+0000","last_fullsized":"2024-09-10T16:05:29.298313+0000","mapping_epoch":34,"log_start":"43'5300","ondisk_log_start":"43'5300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22753763,"num_objects":5556,"num_object_clones":0,"num_object_copies":11112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5556,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12047,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'15504","reported_seq":15646,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:29.472434+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:05:29.472434+0000","last_peered":"2024-09-10T16:05:29.472434+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:05:29.472434+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"43'5500","ondisk_log_start":"43'5500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22761955,"num_objects":5558,"num_object_clones":0,"num_object_copies":11116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5558,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12116,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1187},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'15347","reported_seq":15490,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:29.439838+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:05:29.439838+0000","last_peered":"2024-09-10T16:05:29.439838+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:05:29.439838+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"43'5300","ondisk_log_start":"43'5300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23491526,"num_objects":5737,"num_object_clones":0,"num_object_copies":11474,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5737,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":11846,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1500},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'15632","reported_seq":15767,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:29.394660+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:05:29.394660+0000","last_peered":"2024-09-10T16:05:29.394660+0000","last_clean":"2024-09-10T16:05:29.394660+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:05:29.394660+0000","last_undegraded":"2024-09-10T16:05:29.394660+0000","last_fullsized":"2024-09-10T16:05:29.394660+0000","mapping_epoch":34,"log_start":"43'5600","ondisk_log_start":"43'5600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23273472,"num_objects":5682,"num_object_clones":0,"num_object_copies":11364,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5682,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12479,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.876895+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:05:19.876895+0000","last_peered":"2024-09-10T16:05:19.876895+0000","last_clean":"2024-09-10T16:05:19.876895+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:05:19.876895+0000","last_undegraded":"2024-09-10T16:05:19.876895+0000","last_fullsized":"2024-09-10T16:05:19.876895+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'14934","reported_seq":15079,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:30.353205+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:05:30.353205+0000","last_peered":"2024-09-10T16:05:30.353205+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:05:30.353205+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"43'4900","ondisk_log_start":"43'4900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22622691,"num_objects":5524,"num_object_clones":0,"num_object_copies":11048,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5524,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12753,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":1401}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":99,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.875776+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:05:19.875776+0000","last_peered":"2024-09-10T16:05:19.875776+0000","last_clean":"2024-09-10T16:05:19.875776+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:05:19.875776+0000","last_undegraded":"2024-09-10T16:05:19.875776+0000","last_fullsized":"2024-09-10T16:05:19.875776+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'15282","reported_seq":15418,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:29.404056+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:05:29.404056+0000","last_peered":"2024-09-10T16:05:29.404056+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:05:29.404056+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"43'5200","ondisk_log_start":"43'5200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23052288,"num_objects":5628,"num_object_clones":0,"num_object_copies":11256,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5628,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11786,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1373}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":109,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.871983+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:05:19.871983+0000","last_peered":"2024-09-10T16:05:19.871983+0000","last_clean":"2024-09-10T16:05:19.871983+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:05:19.871983+0000","last_undegraded":"2024-09-10T16:05:19.871983+0000","last_fullsized":"2024-09-10T16:05:19.871983+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'15279","reported_seq":15417,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:30.376659+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:05:30.376659+0000","last_peered":"2024-09-10T16:05:30.376659+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:05:30.376659+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"43'5200","ondisk_log_start":"43'5200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22536675,"num_objects":5503,"num_object_clones":0,"num_object_copies":11006,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5503,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12959,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":1199}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877148+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:05:19.877148+0000","last_peered":"2024-09-10T16:05:19.877148+0000","last_clean":"2024-09-10T16:05:19.877148+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:05:19.877148+0000","last_undegraded":"2024-09-10T16:05:19.877148+0000","last_fullsized":"2024-09-10T16:05:19.877148+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'15583","reported_seq":16276,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:30.352835+0000","last_change":"2024-09-10T16:05:18.271152+0000","last_active":"2024-09-10T16:05:30.352835+0000","last_peered":"2024-09-10T16:05:30.352835+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:05:30.352835+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"43'5500","ondisk_log_start":"43'5500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22884352,"num_objects":5587,"num_object_clones":0,"num_object_copies":11174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4075,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5587,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13137,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":279,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1512}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'15325","reported_seq":15465,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:29.234871+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:05:29.234871+0000","last_peered":"2024-09-10T16:05:29.234871+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:05:29.234871+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"43'5300","ondisk_log_start":"43'5300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22581248,"num_objects":5513,"num_object_clones":0,"num_object_copies":11026,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5513,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11956,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1209}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877654+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:05:19.877654+0000","last_peered":"2024-09-10T16:05:19.877654+0000","last_clean":"2024-09-10T16:05:19.877654+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:05:19.877654+0000","last_undegraded":"2024-09-10T16:05:19.877654+0000","last_fullsized":"2024-09-10T16:05:19.877654+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'15189","reported_seq":23760,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:32.538810+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:05:32.538810+0000","last_peered":"2024-09-10T16:05:32.538810+0000","last_clean":"2024-09-10T16:05:32.538810+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:05:32.538810+0000","last_undegraded":"2024-09-10T16:05:32.538810+0000","last_fullsized":"2024-09-10T16:05:32.538810+0000","mapping_epoch":46,"log_start":"43'5100","ondisk_log_start":"43'5100","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22155264,"num_objects":5409,"num_object_clones":0,"num_object_copies":10818,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5409,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11579,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":99,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.876013+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:05:19.876013+0000","last_peered":"2024-09-10T16:05:19.876013+0000","last_clean":"2024-09-10T16:05:19.876013+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:05:19.876013+0000","last_undegraded":"2024-09-10T16:05:19.876013+0000","last_fullsized":"2024-09-10T16:05:19.876013+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'15525","reported_seq":15672,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:29.415329+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:05:29.415329+0000","last_peered":"2024-09-10T16:05:29.415329+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:05:29.415329+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"43'5500","ondisk_log_start":"43'5500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22622208,"num_objects":5523,"num_object_clones":0,"num_object_copies":11046,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5523,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11780,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1152}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877349+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:05:19.877349+0000","last_peered":"2024-09-10T16:05:19.877349+0000","last_clean":"2024-09-10T16:05:19.877349+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:05:19.877349+0000","last_undegraded":"2024-09-10T16:05:19.877349+0000","last_fullsized":"2024-09-10T16:05:19.877349+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877229+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:05:19.877229+0000","last_peered":"2024-09-10T16:05:19.877229+0000","last_clean":"2024-09-10T16:05:19.877229+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:05:19.877229+0000","last_undegraded":"2024-09-10T16:05:19.877229+0000","last_fullsized":"2024-09-10T16:05:19.877229+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'15554","reported_seq":15704,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:29.395029+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:05:29.395029+0000","last_peered":"2024-09-10T16:05:29.395029+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:05:29.395029+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"43'5500","ondisk_log_start":"43'5500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23118790,"num_objects":5646,"num_object_clones":0,"num_object_copies":11292,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5646,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11717,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1323},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":363500278,"num_objects":88755,"num_object_clones":0,"num_object_copies":177510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42632,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":88755,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":203351,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8757,"num_bytes_recovered":33095680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":934600704,"data_stored":934458879,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160777,"ondisk_log_size":160777,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986266,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":635424,"kb_used_data":314952,"kb_used_omap":4,"kb_used_meta":320443,"kb_avail":93736416,"statfs":{"total":96636764160,"available":95986089984,"internally_reserved":0,"allocated":322510848,"data_stored":322208164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":328134314},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659212,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":584572,"kb_used_data":264108,"kb_used_omap":4,"kb_used_meta":320443,"kb_avail":93787268,"statfs":{"total":96636764160,"available":96038162432,"internally_reserved":0,"allocated":270446592,"data_stored":270114540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":328134318},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,2,6,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182494,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":752156,"kb_used_data":337348,"kb_used_omap":4,"kb_used_meta":414779,"kb_avail":93619684,"statfs":{"total":96636764160,"available":95866556416,"internally_reserved":0,"allocated":345444352,"data_stored":345141663,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":424734378},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,1,0,2,1,6,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":343629824,"data_stored":343593694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":269168640,"data_stored":269103147,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":321802240,"data_stored":321762038,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:05:38.072 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:05:38.621 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:05:38.622 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:05:38.645 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":291,"stamp":"2024-09-10T16:05:37.690519+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":354013961,"num_objects":86441,"num_object_clones":0,"num_object_copies":172882,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":86441,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":205669,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8873,"num_bytes_recovered":33095699,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160996,"ondisk_log_size":160996,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1969092,"kb_used_data":896700,"kb_used_omap":12,"kb_used_meta":1072307,"kb_avail":281146428,"statfs":{"total":289910292480,"available":287893942272,"internally_reserved":0,"allocated":918220800,"data_stored":917283375,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1098042370},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,2,4,5,11,6,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-20140032,"num_objects":-4917,"num_object_clones":0,"num_object_copies":-9834,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-212,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4917,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4917,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":212,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005408"},"pg_stats":[{"pgid":"2.f","version":"47'15337","reported_seq":15416,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:34.473803+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:05:34.473803+0000","last_peered":"2024-09-10T16:05:34.473803+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:05:34.473803+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"43'5300","ondisk_log_start":"43'5300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21852160,"num_objects":5335,"num_object_clones":0,"num_object_copies":10670,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5335,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15337,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1076}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'15476","reported_seq":15594,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:35.292501+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:05:35.292501+0000","last_peered":"2024-09-10T16:05:35.292501+0000","last_clean":"2024-09-10T16:05:35.292501+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:05:35.292501+0000","last_undegraded":"2024-09-10T16:05:35.292501+0000","last_fullsized":"2024-09-10T16:05:35.292501+0000","mapping_epoch":35,"log_start":"43'5400","ondisk_log_start":"43'5400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21988294,"num_objects":5370,"num_object_clones":0,"num_object_copies":10740,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5370,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13581,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'15485","reported_seq":24168,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:37.514106+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:05:37.514106+0000","last_peered":"2024-09-10T16:05:37.514106+0000","last_clean":"2024-09-10T16:05:37.514106+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:05:37.514106+0000","last_undegraded":"2024-09-10T16:05:37.514106+0000","last_fullsized":"2024-09-10T16:05:37.514106+0000","mapping_epoch":44,"log_start":"43'5400","ondisk_log_start":"43'5400","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21757976,"num_objects":5313,"num_object_clones":0,"num_object_copies":10626,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5313,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13489,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'15371","reported_seq":15457,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:35.389458+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:05:35.389458+0000","last_peered":"2024-09-10T16:05:35.389458+0000","last_clean":"2024-09-10T16:05:35.389458+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:05:35.389458+0000","last_undegraded":"2024-09-10T16:05:35.389458+0000","last_fullsized":"2024-09-10T16:05:35.389458+0000","mapping_epoch":35,"log_start":"43'5300","ondisk_log_start":"43'5300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21663744,"num_objects":5289,"num_object_clones":0,"num_object_copies":10578,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5289,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15371,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'15486","reported_seq":15621,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:34.482966+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:05:34.482966+0000","last_peered":"2024-09-10T16:05:34.482966+0000","last_clean":"2024-09-10T16:05:34.482966+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:05:34.482966+0000","last_undegraded":"2024-09-10T16:05:34.482966+0000","last_fullsized":"2024-09-10T16:05:34.482966+0000","mapping_epoch":34,"log_start":"43'5400","ondisk_log_start":"43'5400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22139363,"num_objects":5406,"num_object_clones":0,"num_object_copies":10812,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5406,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12197,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'15652","reported_seq":15795,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:34.491590+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:05:34.491590+0000","last_peered":"2024-09-10T16:05:34.491590+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:05:34.491590+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"43'5600","ondisk_log_start":"43'5600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22155747,"num_objects":5410,"num_object_clones":0,"num_object_copies":10820,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5410,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12264,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1039},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'15486","reported_seq":15629,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:34.417892+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:05:34.417892+0000","last_peered":"2024-09-10T16:05:34.417892+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:05:34.417892+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"43'5400","ondisk_log_start":"43'5400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22922182,"num_objects":5598,"num_object_clones":0,"num_object_copies":11196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5598,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":11985,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1361},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'15793","reported_seq":15928,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:34.414605+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:05:34.414605+0000","last_peered":"2024-09-10T16:05:34.414605+0000","last_clean":"2024-09-10T16:05:34.414605+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:05:34.414605+0000","last_undegraded":"2024-09-10T16:05:34.414605+0000","last_fullsized":"2024-09-10T16:05:34.414605+0000","mapping_epoch":34,"log_start":"43'5700","ondisk_log_start":"43'5700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22614016,"num_objects":5521,"num_object_clones":0,"num_object_copies":11042,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5521,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12640,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.876895+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:05:19.876895+0000","last_peered":"2024-09-10T16:05:19.876895+0000","last_clean":"2024-09-10T16:05:19.876895+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:05:19.876895+0000","last_undegraded":"2024-09-10T16:05:19.876895+0000","last_fullsized":"2024-09-10T16:05:19.876895+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'15095","reported_seq":15240,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:35.326539+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:05:35.326539+0000","last_peered":"2024-09-10T16:05:35.326539+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:05:35.326539+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"43'5000","ondisk_log_start":"43'5000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21963235,"num_objects":5363,"num_object_clones":0,"num_object_copies":10726,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5363,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12914,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":1240}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":99,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.875776+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:05:19.875776+0000","last_peered":"2024-09-10T16:05:19.875776+0000","last_clean":"2024-09-10T16:05:19.875776+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:05:19.875776+0000","last_undegraded":"2024-09-10T16:05:19.875776+0000","last_fullsized":"2024-09-10T16:05:19.875776+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'15410","reported_seq":15546,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:34.347357+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:05:34.347357+0000","last_peered":"2024-09-10T16:05:34.347357+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:05:34.347357+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"43'5400","ondisk_log_start":"43'5400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22528000,"num_objects":5500,"num_object_clones":0,"num_object_copies":11000,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5500,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11914,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1245}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":109,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.871983+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:05:19.871983+0000","last_peered":"2024-09-10T16:05:19.871983+0000","last_clean":"2024-09-10T16:05:19.871983+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:05:19.871983+0000","last_undegraded":"2024-09-10T16:05:19.871983+0000","last_fullsized":"2024-09-10T16:05:19.871983+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'15425","reported_seq":15565,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:35.382110+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:05:35.382110+0000","last_peered":"2024-09-10T16:05:35.382110+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:05:35.382110+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"43'5400","ondisk_log_start":"43'5400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21938659,"num_objects":5357,"num_object_clones":0,"num_object_copies":10714,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5357,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13105,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":1053}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877148+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:05:19.877148+0000","last_peered":"2024-09-10T16:05:19.877148+0000","last_clean":"2024-09-10T16:05:19.877148+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:05:19.877148+0000","last_undegraded":"2024-09-10T16:05:19.877148+0000","last_fullsized":"2024-09-10T16:05:19.877148+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'15701","reported_seq":16622,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:35.383186+0000","last_change":"2024-09-10T16:05:18.271152+0000","last_active":"2024-09-10T16:05:35.383186+0000","last_peered":"2024-09-10T16:05:35.383186+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:05:35.383186+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"43'5700","ondisk_log_start":"43'5700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22401024,"num_objects":5469,"num_object_clones":0,"num_object_copies":10938,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3961,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5469,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13255,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":393,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1508}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'15472","reported_seq":15612,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:34.417087+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:05:34.417087+0000","last_peered":"2024-09-10T16:05:34.417087+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:05:34.417087+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"43'5400","ondisk_log_start":"43'5400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21979136,"num_objects":5366,"num_object_clones":0,"num_object_copies":10732,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5366,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12103,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1062}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877654+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:05:19.877654+0000","last_peered":"2024-09-10T16:05:19.877654+0000","last_clean":"2024-09-10T16:05:19.877654+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:05:19.877654+0000","last_undegraded":"2024-09-10T16:05:19.877654+0000","last_fullsized":"2024-09-10T16:05:19.877654+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'15338","reported_seq":23909,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:37.653861+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:05:37.653861+0000","last_peered":"2024-09-10T16:05:37.653861+0000","last_clean":"2024-09-10T16:05:37.653861+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:05:37.653861+0000","last_undegraded":"2024-09-10T16:05:37.653861+0000","last_fullsized":"2024-09-10T16:05:37.653861+0000","mapping_epoch":46,"log_start":"43'5300","ondisk_log_start":"43'5300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21544960,"num_objects":5260,"num_object_clones":0,"num_object_copies":10520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11728,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":99,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.876013+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:05:19.876013+0000","last_peered":"2024-09-10T16:05:19.876013+0000","last_clean":"2024-09-10T16:05:19.876013+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:05:19.876013+0000","last_undegraded":"2024-09-10T16:05:19.876013+0000","last_fullsized":"2024-09-10T16:05:19.876013+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'15671","reported_seq":15818,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:34.446078+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:05:34.446078+0000","last_peered":"2024-09-10T16:05:34.446078+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:05:34.446078+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"43'5600","ondisk_log_start":"43'5600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22024192,"num_objects":5377,"num_object_clones":0,"num_object_copies":10754,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5377,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11926,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1006}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877349+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:05:19.877349+0000","last_peered":"2024-09-10T16:05:19.877349+0000","last_clean":"2024-09-10T16:05:19.877349+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:05:19.877349+0000","last_undegraded":"2024-09-10T16:05:19.877349+0000","last_fullsized":"2024-09-10T16:05:19.877349+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877229+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:05:19.877229+0000","last_peered":"2024-09-10T16:05:19.877229+0000","last_clean":"2024-09-10T16:05:19.877229+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:05:19.877229+0000","last_undegraded":"2024-09-10T16:05:19.877229+0000","last_fullsized":"2024-09-10T16:05:19.877229+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'15695","reported_seq":15845,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:34.472146+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:05:34.472146+0000","last_peered":"2024-09-10T16:05:34.472146+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:05:34.472146+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"43'5600","ondisk_log_start":"43'5600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22541254,"num_objects":5505,"num_object_clones":0,"num_object_copies":11010,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5505,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11858,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1182},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":354013942,"num_objects":86439,"num_object_clones":0,"num_object_copies":172878,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":86439,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":205667,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8871,"num_bytes_recovered":33095680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":915730432,"data_stored":915588607,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160993,"ondisk_log_size":160993,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986267,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":635040,"kb_used_data":310408,"kb_used_omap":4,"kb_used_meta":324603,"kb_avail":93736800,"statfs":{"total":96636764160,"available":95986483200,"internally_reserved":0,"allocated":317857792,"data_stored":317555108,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":332394154},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,1,1,4,4,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659213,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":581904,"kb_used_data":257280,"kb_used_omap":4,"kb_used_meta":324603,"kb_avail":93789936,"statfs":{"total":96636764160,"available":96040894464,"internally_reserved":0,"allocated":263454720,"data_stored":263122668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":332394158},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,2,0,5],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182495,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":752148,"kb_used_data":329012,"kb_used_omap":4,"kb_used_meta":423099,"kb_avail":93619692,"statfs":{"total":96636764160,"available":95866564608,"internally_reserved":0,"allocated":336908288,"data_stored":336605599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":433254058},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,4,2,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":336388096,"data_stored":336351966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":262197248,"data_stored":262131755,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":317145088,"data_stored":317104886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:05:38.646 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:05:39.197 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:05:39.200 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:05:39.223 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":291,"stamp":"2024-09-10T16:05:37.690519+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":354013961,"num_objects":86441,"num_object_clones":0,"num_object_copies":172882,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":86441,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":205669,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8873,"num_bytes_recovered":33095699,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160996,"ondisk_log_size":160996,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1969092,"kb_used_data":896700,"kb_used_omap":12,"kb_used_meta":1072307,"kb_avail":281146428,"statfs":{"total":289910292480,"available":287893942272,"internally_reserved":0,"allocated":918220800,"data_stored":917283375,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1098042370},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,2,4,5,11,6,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-20140032,"num_objects":-4917,"num_object_clones":0,"num_object_copies":-9834,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-212,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4917,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4917,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":212,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005408"},"pg_stats":[{"pgid":"2.f","version":"47'15337","reported_seq":15416,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:34.473803+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:05:34.473803+0000","last_peered":"2024-09-10T16:05:34.473803+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:05:34.473803+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"43'5300","ondisk_log_start":"43'5300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21852160,"num_objects":5335,"num_object_clones":0,"num_object_copies":10670,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5335,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15337,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1076}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'15476","reported_seq":15594,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:35.292501+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:05:35.292501+0000","last_peered":"2024-09-10T16:05:35.292501+0000","last_clean":"2024-09-10T16:05:35.292501+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:05:35.292501+0000","last_undegraded":"2024-09-10T16:05:35.292501+0000","last_fullsized":"2024-09-10T16:05:35.292501+0000","mapping_epoch":35,"log_start":"43'5400","ondisk_log_start":"43'5400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21988294,"num_objects":5370,"num_object_clones":0,"num_object_copies":10740,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5370,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13581,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'15485","reported_seq":24168,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:37.514106+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:05:37.514106+0000","last_peered":"2024-09-10T16:05:37.514106+0000","last_clean":"2024-09-10T16:05:37.514106+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:05:37.514106+0000","last_undegraded":"2024-09-10T16:05:37.514106+0000","last_fullsized":"2024-09-10T16:05:37.514106+0000","mapping_epoch":44,"log_start":"43'5400","ondisk_log_start":"43'5400","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21757976,"num_objects":5313,"num_object_clones":0,"num_object_copies":10626,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5313,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13489,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'15371","reported_seq":15457,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:35.389458+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:05:35.389458+0000","last_peered":"2024-09-10T16:05:35.389458+0000","last_clean":"2024-09-10T16:05:35.389458+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:05:35.389458+0000","last_undegraded":"2024-09-10T16:05:35.389458+0000","last_fullsized":"2024-09-10T16:05:35.389458+0000","mapping_epoch":35,"log_start":"43'5300","ondisk_log_start":"43'5300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21663744,"num_objects":5289,"num_object_clones":0,"num_object_copies":10578,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5289,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15371,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'15486","reported_seq":15621,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:34.482966+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:05:34.482966+0000","last_peered":"2024-09-10T16:05:34.482966+0000","last_clean":"2024-09-10T16:05:34.482966+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:05:34.482966+0000","last_undegraded":"2024-09-10T16:05:34.482966+0000","last_fullsized":"2024-09-10T16:05:34.482966+0000","mapping_epoch":34,"log_start":"43'5400","ondisk_log_start":"43'5400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22139363,"num_objects":5406,"num_object_clones":0,"num_object_copies":10812,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5406,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12197,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'15652","reported_seq":15795,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:34.491590+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:05:34.491590+0000","last_peered":"2024-09-10T16:05:34.491590+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:05:34.491590+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"43'5600","ondisk_log_start":"43'5600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22155747,"num_objects":5410,"num_object_clones":0,"num_object_copies":10820,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5410,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12264,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1039},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'15486","reported_seq":15629,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:34.417892+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:05:34.417892+0000","last_peered":"2024-09-10T16:05:34.417892+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:05:34.417892+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"43'5400","ondisk_log_start":"43'5400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22922182,"num_objects":5598,"num_object_clones":0,"num_object_copies":11196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5598,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":11985,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1361},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'15793","reported_seq":15928,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:34.414605+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:05:34.414605+0000","last_peered":"2024-09-10T16:05:34.414605+0000","last_clean":"2024-09-10T16:05:34.414605+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:05:34.414605+0000","last_undegraded":"2024-09-10T16:05:34.414605+0000","last_fullsized":"2024-09-10T16:05:34.414605+0000","mapping_epoch":34,"log_start":"43'5700","ondisk_log_start":"43'5700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22614016,"num_objects":5521,"num_object_clones":0,"num_object_copies":11042,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5521,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12640,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.876895+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:05:19.876895+0000","last_peered":"2024-09-10T16:05:19.876895+0000","last_clean":"2024-09-10T16:05:19.876895+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:05:19.876895+0000","last_undegraded":"2024-09-10T16:05:19.876895+0000","last_fullsized":"2024-09-10T16:05:19.876895+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'15095","reported_seq":15240,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:35.326539+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:05:35.326539+0000","last_peered":"2024-09-10T16:05:35.326539+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:05:35.326539+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"43'5000","ondisk_log_start":"43'5000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21963235,"num_objects":5363,"num_object_clones":0,"num_object_copies":10726,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5363,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12914,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":1240}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":99,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.875776+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:05:19.875776+0000","last_peered":"2024-09-10T16:05:19.875776+0000","last_clean":"2024-09-10T16:05:19.875776+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:05:19.875776+0000","last_undegraded":"2024-09-10T16:05:19.875776+0000","last_fullsized":"2024-09-10T16:05:19.875776+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'15410","reported_seq":15546,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:34.347357+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:05:34.347357+0000","last_peered":"2024-09-10T16:05:34.347357+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:05:34.347357+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"43'5400","ondisk_log_start":"43'5400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22528000,"num_objects":5500,"num_object_clones":0,"num_object_copies":11000,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5500,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11914,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1245}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":109,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.871983+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:05:19.871983+0000","last_peered":"2024-09-10T16:05:19.871983+0000","last_clean":"2024-09-10T16:05:19.871983+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:05:19.871983+0000","last_undegraded":"2024-09-10T16:05:19.871983+0000","last_fullsized":"2024-09-10T16:05:19.871983+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'15425","reported_seq":15565,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:35.382110+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:05:35.382110+0000","last_peered":"2024-09-10T16:05:35.382110+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:05:35.382110+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"43'5400","ondisk_log_start":"43'5400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21938659,"num_objects":5357,"num_object_clones":0,"num_object_copies":10714,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5357,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13105,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":1053}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877148+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:05:19.877148+0000","last_peered":"2024-09-10T16:05:19.877148+0000","last_clean":"2024-09-10T16:05:19.877148+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:05:19.877148+0000","last_undegraded":"2024-09-10T16:05:19.877148+0000","last_fullsized":"2024-09-10T16:05:19.877148+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'15701","reported_seq":16622,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:35.383186+0000","last_change":"2024-09-10T16:05:18.271152+0000","last_active":"2024-09-10T16:05:35.383186+0000","last_peered":"2024-09-10T16:05:35.383186+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:05:35.383186+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"43'5700","ondisk_log_start":"43'5700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22401024,"num_objects":5469,"num_object_clones":0,"num_object_copies":10938,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3961,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5469,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13255,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":393,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1508}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'15472","reported_seq":15612,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:34.417087+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:05:34.417087+0000","last_peered":"2024-09-10T16:05:34.417087+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:05:34.417087+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"43'5400","ondisk_log_start":"43'5400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21979136,"num_objects":5366,"num_object_clones":0,"num_object_copies":10732,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5366,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12103,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1062}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877654+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:05:19.877654+0000","last_peered":"2024-09-10T16:05:19.877654+0000","last_clean":"2024-09-10T16:05:19.877654+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:05:19.877654+0000","last_undegraded":"2024-09-10T16:05:19.877654+0000","last_fullsized":"2024-09-10T16:05:19.877654+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'15338","reported_seq":23909,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:37.653861+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:05:37.653861+0000","last_peered":"2024-09-10T16:05:37.653861+0000","last_clean":"2024-09-10T16:05:37.653861+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:05:37.653861+0000","last_undegraded":"2024-09-10T16:05:37.653861+0000","last_fullsized":"2024-09-10T16:05:37.653861+0000","mapping_epoch":46,"log_start":"43'5300","ondisk_log_start":"43'5300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21544960,"num_objects":5260,"num_object_clones":0,"num_object_copies":10520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11728,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":99,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.876013+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:05:19.876013+0000","last_peered":"2024-09-10T16:05:19.876013+0000","last_clean":"2024-09-10T16:05:19.876013+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:05:19.876013+0000","last_undegraded":"2024-09-10T16:05:19.876013+0000","last_fullsized":"2024-09-10T16:05:19.876013+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'15671","reported_seq":15818,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:34.446078+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:05:34.446078+0000","last_peered":"2024-09-10T16:05:34.446078+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:05:34.446078+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"43'5600","ondisk_log_start":"43'5600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22024192,"num_objects":5377,"num_object_clones":0,"num_object_copies":10754,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5377,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11926,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1006}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877349+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:05:19.877349+0000","last_peered":"2024-09-10T16:05:19.877349+0000","last_clean":"2024-09-10T16:05:19.877349+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:05:19.877349+0000","last_undegraded":"2024-09-10T16:05:19.877349+0000","last_fullsized":"2024-09-10T16:05:19.877349+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877229+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:05:19.877229+0000","last_peered":"2024-09-10T16:05:19.877229+0000","last_clean":"2024-09-10T16:05:19.877229+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:05:19.877229+0000","last_undegraded":"2024-09-10T16:05:19.877229+0000","last_fullsized":"2024-09-10T16:05:19.877229+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'15695","reported_seq":15845,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:34.472146+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:05:34.472146+0000","last_peered":"2024-09-10T16:05:34.472146+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:05:34.472146+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"43'5600","ondisk_log_start":"43'5600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22541254,"num_objects":5505,"num_object_clones":0,"num_object_copies":11010,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5505,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11858,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1182},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":354013942,"num_objects":86439,"num_object_clones":0,"num_object_copies":172878,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":86439,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":205667,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8871,"num_bytes_recovered":33095680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":915730432,"data_stored":915588607,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160993,"ondisk_log_size":160993,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986267,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":635040,"kb_used_data":310408,"kb_used_omap":4,"kb_used_meta":324603,"kb_avail":93736800,"statfs":{"total":96636764160,"available":95986483200,"internally_reserved":0,"allocated":317857792,"data_stored":317555108,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":332394154},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,1,1,4,4,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659213,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":581904,"kb_used_data":257280,"kb_used_omap":4,"kb_used_meta":324603,"kb_avail":93789936,"statfs":{"total":96636764160,"available":96040894464,"internally_reserved":0,"allocated":263454720,"data_stored":263122668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":332394158},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,2,0,5],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182495,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":752148,"kb_used_data":329012,"kb_used_omap":4,"kb_used_meta":423099,"kb_avail":93619692,"statfs":{"total":96636764160,"available":95866564608,"internally_reserved":0,"allocated":336908288,"data_stored":336605599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":433254058},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,4,2,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":336388096,"data_stored":336351966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":262197248,"data_stored":262131755,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":317145088,"data_stored":317104886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:05:39.224 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-10T16:05:39.224 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "47'15337", 'reported_seq': 15416, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:05:34.473803+0000', 'last_change': '2024-09-10T15:59:16.281558+0000', 'last_active': '2024-09-10T16:05:34.473803+0000', 'last_peered': '2024-09-10T16:05:34.473803+0000', 'last_clean': '2024-09-10T15:59:14.132456+0000', 'last_became_active': '2024-09-10T15:59:16.274407+0000', 'last_became_peered': '2024-09-10T15:59:16.274407+0000', 'last_unstale': '2024-09-10T16:05:34.473803+0000', 'last_undegraded': '2024-09-10T15:59:15.851279+0000', 'last_fullsized': '2024-09-10T15:59:15.774356+0000', 'mapping_epoch': 42, 'log_start': "43'5300", 'ondisk_log_start': "43'5300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'objects_scrubbed': 0, 'log_size': 10037, 'log_dups_size': 0, 'ondisk_log_size': 10037, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21852160, 'num_objects': 5335, 'num_object_clones': 0, 'num_object_copies': 10670, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4259, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5335, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15337, 'num_write_kb': 41344, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1076}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:05:39.224 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-10T16:05:39.224 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "47'15652", 'reported_seq': 15795, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:05:34.491590+0000', 'last_change': '2024-09-10T15:59:16.308264+0000', 'last_active': '2024-09-10T16:05:34.491590+0000', 'last_peered': '2024-09-10T16:05:34.491590+0000', 'last_clean': '2024-09-10T15:59:14.111551+0000', 'last_became_active': '2024-09-10T15:59:16.300040+0000', 'last_became_peered': '2024-09-10T15:59:16.300040+0000', 'last_unstale': '2024-09-10T16:05:34.491590+0000', 'last_undegraded': '2024-09-10T15:59:15.878234+0000', 'last_fullsized': '2024-09-10T15:59:15.769328+0000', 'mapping_epoch': 42, 'log_start': "43'5600", 'ondisk_log_start': "43'5600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3442", 'last_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'objects_scrubbed': 3384, 'log_size': 10052, 'log_dups_size': 0, 'ondisk_log_size': 10052, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1292, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22155747, 'num_objects': 5410, 'num_object_clones': 0, 'num_object_copies': 10820, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4372, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5410, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 12264, 'num_write_kb': 28580, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1039}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:05:39.225 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-10T16:05:39.225 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "47'15486", 'reported_seq': 15629, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:05:34.417892+0000', 'last_change': '2024-09-10T15:59:16.331899+0000', 'last_active': '2024-09-10T16:05:34.417892+0000', 'last_peered': '2024-09-10T16:05:34.417892+0000', 'last_clean': '2024-09-10T15:59:14.139436+0000', 'last_became_active': '2024-09-10T15:59:16.318715+0000', 'last_became_peered': '2024-09-10T15:59:16.318715+0000', 'last_unstale': '2024-09-10T16:05:34.417892+0000', 'last_undegraded': '2024-09-10T15:59:15.877524+0000', 'last_fullsized': '2024-09-10T15:59:15.769042+0000', 'mapping_epoch': 42, 'log_start': "43'5400", 'ondisk_log_start': "43'5400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3537", 'last_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'objects_scrubbed': 3497, 'log_size': 10086, 'log_dups_size': 0, 'ondisk_log_size': 10086, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1224, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22922182, 'num_objects': 5598, 'num_object_clones': 0, 'num_object_copies': 11196, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4237, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5598, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 11985, 'num_write_kb': 28172, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1361}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:05:39.225 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-10T16:05:39.225 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "47'15095", 'reported_seq': 15240, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:05:35.326539+0000', 'last_change': '2024-09-10T15:59:16.175680+0000', 'last_active': '2024-09-10T16:05:35.326539+0000', 'last_peered': '2024-09-10T16:05:35.326539+0000', 'last_clean': '2024-09-10T15:59:14.088187+0000', 'last_became_active': '2024-09-10T15:59:16.165810+0000', 'last_became_peered': '2024-09-10T15:59:16.165810+0000', 'last_unstale': '2024-09-10T16:05:35.326539+0000', 'last_undegraded': '2024-09-10T15:59:15.822098+0000', 'last_fullsized': '2024-09-10T15:59:15.769123+0000', 'mapping_epoch': 42, 'log_start': "43'5000", 'ondisk_log_start': "43'5000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2218", 'last_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'objects_scrubbed': 2175, 'log_size': 10095, 'log_dups_size': 0, 'ondisk_log_size': 10095, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 825, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21963235, 'num_objects': 5363, 'num_object_clones': 0, 'num_object_copies': 10726, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4124, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5363, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 12914, 'num_write_kb': 32204, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': 1240}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:05:39.225 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-10T16:05:39.225 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "47'15410", 'reported_seq': 15546, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:05:34.347357+0000', 'last_change': '2024-09-10T15:59:16.277742+0000', 'last_active': '2024-09-10T16:05:34.347357+0000', 'last_peered': '2024-09-10T16:05:34.347357+0000', 'last_clean': '2024-09-10T15:59:14.129081+0000', 'last_became_active': '2024-09-10T15:59:16.269835+0000', 'last_became_peered': '2024-09-10T15:59:16.269835+0000', 'last_unstale': '2024-09-10T16:05:34.347357+0000', 'last_undegraded': '2024-09-10T15:59:15.860857+0000', 'last_fullsized': '2024-09-10T15:59:15.771410+0000', 'mapping_epoch': 42, 'log_start': "43'5400", 'ondisk_log_start': "43'5400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3557", 'last_scrub_stamp': '2024-09-10T15:59:02.493678+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:02.493678+0000', 'objects_scrubbed': 3488, 'log_size': 10010, 'log_dups_size': 0, 'ondisk_log_size': 10010, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1472, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22528000, 'num_objects': 5500, 'num_object_clones': 0, 'num_object_copies': 11000, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4257, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5500, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11914, 'num_write_kb': 27852, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1245}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:05:39.225 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-10T16:05:39.225 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "47'15425", 'reported_seq': 15565, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:05:35.382110+0000', 'last_change': '2024-09-10T15:59:16.284311+0000', 'last_active': '2024-09-10T16:05:35.382110+0000', 'last_peered': '2024-09-10T16:05:35.382110+0000', 'last_clean': '2024-09-10T15:59:14.116041+0000', 'last_became_active': '2024-09-10T15:59:16.274543+0000', 'last_became_peered': '2024-09-10T15:59:16.274543+0000', 'last_unstale': '2024-09-10T16:05:35.382110+0000', 'last_undegraded': '2024-09-10T15:59:15.812484+0000', 'last_fullsized': '2024-09-10T15:59:15.770885+0000', 'mapping_epoch': 42, 'log_start': "43'5400", 'ondisk_log_start': "43'5400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2356", 'last_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'objects_scrubbed': 2316, 'log_size': 10025, 'log_dups_size': 0, 'ondisk_log_size': 10025, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 783, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21938659, 'num_objects': 5357, 'num_object_clones': 0, 'num_object_copies': 10714, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5357, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 13105, 'num_write_kb': 32292, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': 1053}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:05:39.225 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-10T16:05:39.225 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "47'15701", 'reported_seq': 16622, 'reported_epoch': 47, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:05:35.383186+0000', 'last_change': '2024-09-10T16:05:18.271152+0000', 'last_active': '2024-09-10T16:05:35.383186+0000', 'last_peered': '2024-09-10T16:05:35.383186+0000', 'last_clean': '2024-09-10T15:59:14.110654+0000', 'last_became_active': '2024-09-10T15:59:16.164645+0000', 'last_became_peered': '2024-09-10T15:59:16.164645+0000', 'last_unstale': '2024-09-10T16:05:35.383186+0000', 'last_undegraded': '2024-09-10T15:59:15.803593+0000', 'last_fullsized': '2024-09-10T15:59:15.769083+0000', 'mapping_epoch': 42, 'log_start': "43'5700", 'ondisk_log_start': "43'5700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2486", 'last_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'objects_scrubbed': 2440, 'log_size': 10001, 'log_dups_size': 0, 'ondisk_log_size': 10001, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 792, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22401024, 'num_objects': 5469, 'num_object_clones': 0, 'num_object_copies': 10938, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3961, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5469, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13255, 'num_write_kb': 32568, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 393, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 1508}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:05:39.226 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-10T16:05:39.226 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "47'15472", 'reported_seq': 15612, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:05:34.417087+0000', 'last_change': '2024-09-10T15:59:16.315641+0000', 'last_active': '2024-09-10T16:05:34.417087+0000', 'last_peered': '2024-09-10T16:05:34.417087+0000', 'last_clean': '2024-09-10T15:59:14.117001+0000', 'last_became_active': '2024-09-10T15:59:16.302245+0000', 'last_became_peered': '2024-09-10T15:59:16.302245+0000', 'last_unstale': '2024-09-10T16:05:34.417087+0000', 'last_undegraded': '2024-09-10T15:59:15.845012+0000', 'last_fullsized': '2024-09-10T15:59:15.770337+0000', 'mapping_epoch': 42, 'log_start': "43'5400", 'ondisk_log_start': "43'5400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3404", 'last_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'objects_scrubbed': 3367, 'log_size': 10072, 'log_dups_size': 0, 'ondisk_log_size': 10072, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 983, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21979136, 'num_objects': 5366, 'num_object_clones': 0, 'num_object_copies': 10732, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5366, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12103, 'num_write_kb': 28204, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1062}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:05:39.226 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-10T16:05:39.226 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "47'15671", 'reported_seq': 15818, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:05:34.446078+0000', 'last_change': '2024-09-10T15:59:16.311291+0000', 'last_active': '2024-09-10T16:05:34.446078+0000', 'last_peered': '2024-09-10T16:05:34.446078+0000', 'last_clean': '2024-09-10T15:59:14.116363+0000', 'last_became_active': '2024-09-10T15:59:16.301244+0000', 'last_became_peered': '2024-09-10T15:59:16.301244+0000', 'last_unstale': '2024-09-10T16:05:34.446078+0000', 'last_undegraded': '2024-09-10T15:59:15.883434+0000', 'last_fullsized': '2024-09-10T15:59:15.772756+0000', 'mapping_epoch': 42, 'log_start': "43'5600", 'ondisk_log_start': "43'5600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3789", 'last_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'objects_scrubbed': 3741, 'log_size': 10071, 'log_dups_size': 0, 'ondisk_log_size': 10071, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1440, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22024192, 'num_objects': 5377, 'num_object_clones': 0, 'num_object_copies': 10754, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4373, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5377, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11926, 'num_write_kb': 27124, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1006}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:05:39.226 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-10T16:05:39.226 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "47'15695", 'reported_seq': 15845, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:05:34.472146+0000', 'last_change': '2024-09-10T15:59:16.313409+0000', 'last_active': '2024-09-10T16:05:34.472146+0000', 'last_peered': '2024-09-10T16:05:34.472146+0000', 'last_clean': '2024-09-10T15:59:14.129780+0000', 'last_became_active': '2024-09-10T15:59:16.300932+0000', 'last_became_peered': '2024-09-10T15:59:16.300932+0000', 'last_unstale': '2024-09-10T16:05:34.472146+0000', 'last_undegraded': '2024-09-10T15:59:15.887823+0000', 'last_fullsized': '2024-09-10T15:59:15.768513+0000', 'mapping_epoch': 42, 'log_start': "43'5600", 'ondisk_log_start': "43'5600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3880", 'last_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'objects_scrubbed': 3827, 'log_size': 10095, 'log_dups_size': 0, 'ondisk_log_size': 10095, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1397, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22541254, 'num_objects': 5505, 'num_object_clones': 0, 'num_object_copies': 11010, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4325, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5505, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11858, 'num_write_kb': 27072, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1182}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:05:39.226 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T16:05:39.911 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:05:39.911 INFO:teuthology.orchestra.run.smithi115.stdout:{"fsid":"7effee29-df05-4351-9741-ee203564ed81","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 42518/172882 objects degraded (24.594%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":503,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":47,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983954,"num_in_osds":3,"osd_in_since":1725983947,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":86441,"data_bytes":354013961,"bytes_used":2016350208,"bytes_avail":287893942272,"bytes_total":289910292480,"degraded_objects":42518,"degraded_total":172882,"degraded_ratio":0.24593653474624311,"recovering_objects_per_sec":17,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":212,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":409},"fsmap":{"epoch":1,"btime":"2024-09-10T15:57:14:967120+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T16:00:13.560776+0000","services":{}},"progress_events":{}} 2024-09-10T16:05:39.933 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-10T16:05:39.934 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:05:40.513 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:05:40.515 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:05:40.533 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":292,"stamp":"2024-09-10T16:05:39.691291+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":348746505,"num_objects":85155,"num_object_clones":0,"num_object_copies":170310,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":85155,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":206955,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8873,"num_bytes_recovered":33095699,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160782,"ondisk_log_size":160782,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1949640,"kb_used_data":889740,"kb_used_omap":12,"kb_used_meta":1059827,"kb_avail":281165880,"statfs":{"total":289910292480,"available":287913861120,"internally_reserved":0,"allocated":911093760,"data_stored":910156335,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1085262850},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,2,4,1,14,12,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-24260608,"num_objects":-5923,"num_object_clones":0,"num_object_copies":-11846,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-212,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5923,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5923,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":212,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005590"},"pg_stats":[{"pgid":"2.f","version":"47'15479","reported_seq":15559,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:39.549937+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:05:39.549937+0000","last_peered":"2024-09-10T16:05:39.549937+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:05:39.549937+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"43'5400","ondisk_log_start":"43'5400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21270528,"num_objects":5193,"num_object_clones":0,"num_object_copies":10386,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5193,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15479,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":934}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'15476","reported_seq":15594,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:35.292501+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:05:35.292501+0000","last_peered":"2024-09-10T16:05:35.292501+0000","last_clean":"2024-09-10T16:05:35.292501+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:05:35.292501+0000","last_undegraded":"2024-09-10T16:05:35.292501+0000","last_fullsized":"2024-09-10T16:05:35.292501+0000","mapping_epoch":35,"log_start":"43'5400","ondisk_log_start":"43'5400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21988294,"num_objects":5370,"num_object_clones":0,"num_object_copies":10740,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5370,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13581,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'15485","reported_seq":24168,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:37.514106+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:05:37.514106+0000","last_peered":"2024-09-10T16:05:37.514106+0000","last_clean":"2024-09-10T16:05:37.514106+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:05:37.514106+0000","last_undegraded":"2024-09-10T16:05:37.514106+0000","last_fullsized":"2024-09-10T16:05:37.514106+0000","mapping_epoch":44,"log_start":"43'5400","ondisk_log_start":"43'5400","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21757976,"num_objects":5313,"num_object_clones":0,"num_object_copies":10626,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5313,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13489,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'15371","reported_seq":15457,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:35.389458+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:05:35.389458+0000","last_peered":"2024-09-10T16:05:35.389458+0000","last_clean":"2024-09-10T16:05:35.389458+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:05:35.389458+0000","last_undegraded":"2024-09-10T16:05:35.389458+0000","last_fullsized":"2024-09-10T16:05:35.389458+0000","mapping_epoch":35,"log_start":"43'5300","ondisk_log_start":"43'5300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21663744,"num_objects":5289,"num_object_clones":0,"num_object_copies":10578,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5289,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15371,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'15603","reported_seq":15738,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:39.373753+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:05:39.373753+0000","last_peered":"2024-09-10T16:05:39.373753+0000","last_clean":"2024-09-10T16:05:39.373753+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:05:39.373753+0000","last_undegraded":"2024-09-10T16:05:39.373753+0000","last_fullsized":"2024-09-10T16:05:39.373753+0000","mapping_epoch":34,"log_start":"43'5600","ondisk_log_start":"43'5600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21660131,"num_objects":5289,"num_object_clones":0,"num_object_copies":10578,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5289,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12314,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'15802","reported_seq":15945,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:39.521465+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:05:39.521465+0000","last_peered":"2024-09-10T16:05:39.521465+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:05:39.521465+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"43'5800","ondisk_log_start":"43'5800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21541347,"num_objects":5260,"num_object_clones":0,"num_object_copies":10520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5260,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12414,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":889},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'15647","reported_seq":15790,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:39.485489+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:05:39.485489+0000","last_peered":"2024-09-10T16:05:39.485489+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:05:39.485489+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"43'5600","ondisk_log_start":"43'5600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22262726,"num_objects":5437,"num_object_clones":0,"num_object_copies":10874,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5437,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":12146,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1200},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'15943","reported_seq":16077,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:39.540704+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:05:39.540704+0000","last_peered":"2024-09-10T16:05:39.540704+0000","last_clean":"2024-09-10T16:05:39.540704+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:05:39.540704+0000","last_undegraded":"2024-09-10T16:05:39.540704+0000","last_fullsized":"2024-09-10T16:05:39.540704+0000","mapping_epoch":34,"log_start":"43'5900","ondisk_log_start":"43'5900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21999616,"num_objects":5371,"num_object_clones":0,"num_object_copies":10742,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5371,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12790,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.876895+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:05:19.876895+0000","last_peered":"2024-09-10T16:05:19.876895+0000","last_clean":"2024-09-10T16:05:19.876895+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:05:19.876895+0000","last_undegraded":"2024-09-10T16:05:19.876895+0000","last_fullsized":"2024-09-10T16:05:19.876895+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'15095","reported_seq":15240,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:35.326539+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:05:35.326539+0000","last_peered":"2024-09-10T16:05:35.326539+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:05:35.326539+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"43'5000","ondisk_log_start":"43'5000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21963235,"num_objects":5363,"num_object_clones":0,"num_object_copies":10726,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5363,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12914,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":1240}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":99,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.875776+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:05:19.875776+0000","last_peered":"2024-09-10T16:05:19.875776+0000","last_clean":"2024-09-10T16:05:19.875776+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:05:19.875776+0000","last_undegraded":"2024-09-10T16:05:19.875776+0000","last_fullsized":"2024-09-10T16:05:19.875776+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'15570","reported_seq":15706,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:39.484259+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:05:39.484259+0000","last_peered":"2024-09-10T16:05:39.484259+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:05:39.484259+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"43'5500","ondisk_log_start":"43'5500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21872640,"num_objects":5340,"num_object_clones":0,"num_object_copies":10680,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5340,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12074,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1085}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":109,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.871983+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:05:19.871983+0000","last_peered":"2024-09-10T16:05:19.871983+0000","last_clean":"2024-09-10T16:05:19.871983+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:05:19.871983+0000","last_undegraded":"2024-09-10T16:05:19.871983+0000","last_fullsized":"2024-09-10T16:05:19.871983+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'15425","reported_seq":15565,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:35.382110+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:05:35.382110+0000","last_peered":"2024-09-10T16:05:35.382110+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:05:35.382110+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"43'5400","ondisk_log_start":"43'5400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21938659,"num_objects":5357,"num_object_clones":0,"num_object_copies":10714,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5357,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13105,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":1053}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877148+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:05:19.877148+0000","last_peered":"2024-09-10T16:05:19.877148+0000","last_clean":"2024-09-10T16:05:19.877148+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:05:19.877148+0000","last_undegraded":"2024-09-10T16:05:19.877148+0000","last_fullsized":"2024-09-10T16:05:19.877148+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'15701","reported_seq":16622,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:35.383186+0000","last_change":"2024-09-10T16:05:18.271152+0000","last_active":"2024-09-10T16:05:35.383186+0000","last_peered":"2024-09-10T16:05:35.383186+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:05:35.383186+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"43'5700","ondisk_log_start":"43'5700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22401024,"num_objects":5469,"num_object_clones":0,"num_object_copies":10938,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3961,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5469,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13255,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":393,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1508}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'15604","reported_seq":15742,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:39.553773+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:05:39.553773+0000","last_peered":"2024-09-10T16:05:39.553773+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:05:39.553773+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"43'5600","ondisk_log_start":"43'5600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21438464,"num_objects":5234,"num_object_clones":0,"num_object_copies":10468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5234,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12235,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":930}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877654+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:05:19.877654+0000","last_peered":"2024-09-10T16:05:19.877654+0000","last_clean":"2024-09-10T16:05:19.877654+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:05:19.877654+0000","last_undegraded":"2024-09-10T16:05:19.877654+0000","last_fullsized":"2024-09-10T16:05:19.877654+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'15338","reported_seq":23909,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:37.653861+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:05:37.653861+0000","last_peered":"2024-09-10T16:05:37.653861+0000","last_clean":"2024-09-10T16:05:37.653861+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:05:37.653861+0000","last_undegraded":"2024-09-10T16:05:37.653861+0000","last_fullsized":"2024-09-10T16:05:37.653861+0000","mapping_epoch":46,"log_start":"43'5300","ondisk_log_start":"43'5300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21544960,"num_objects":5260,"num_object_clones":0,"num_object_copies":10520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11728,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":99,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.876013+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:05:19.876013+0000","last_peered":"2024-09-10T16:05:19.876013+0000","last_clean":"2024-09-10T16:05:19.876013+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:05:19.876013+0000","last_undegraded":"2024-09-10T16:05:19.876013+0000","last_fullsized":"2024-09-10T16:05:19.876013+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'15798","reported_seq":15945,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:39.514210+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:05:39.514210+0000","last_peered":"2024-09-10T16:05:39.514210+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:05:39.514210+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"43'5700","ondisk_log_start":"43'5700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21504000,"num_objects":5250,"num_object_clones":0,"num_object_copies":10500,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5250,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12053,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":879}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877349+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:05:19.877349+0000","last_peered":"2024-09-10T16:05:19.877349+0000","last_clean":"2024-09-10T16:05:19.877349+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:05:19.877349+0000","last_undegraded":"2024-09-10T16:05:19.877349+0000","last_fullsized":"2024-09-10T16:05:19.877349+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877229+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:05:19.877229+0000","last_peered":"2024-09-10T16:05:19.877229+0000","last_clean":"2024-09-10T16:05:19.877229+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:05:19.877229+0000","last_undegraded":"2024-09-10T16:05:19.877229+0000","last_fullsized":"2024-09-10T16:05:19.877229+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'15842","reported_seq":15992,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:39.433158+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:05:39.433158+0000","last_peered":"2024-09-10T16:05:39.433158+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:05:39.433158+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"43'5800","ondisk_log_start":"43'5800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21939142,"num_objects":5358,"num_object_clones":0,"num_object_copies":10716,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5358,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12005,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1035},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":348746486,"num_objects":85153,"num_object_clones":0,"num_object_copies":170306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":85153,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":206953,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8871,"num_bytes_recovered":33095680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":908615680,"data_stored":908473855,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160779,"ondisk_log_size":160779,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986267,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":635040,"kb_used_data":310408,"kb_used_omap":4,"kb_used_meta":324603,"kb_avail":93736800,"statfs":{"total":96636764160,"available":95986483200,"internally_reserved":0,"allocated":317857792,"data_stored":317555108,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":332394154},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,1,1,4,4,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659213,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":581904,"kb_used_data":257280,"kb_used_omap":4,"kb_used_meta":324603,"kb_avail":93789936,"statfs":{"total":96636764160,"available":96040894464,"internally_reserved":0,"allocated":263454720,"data_stored":263122668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":332394158},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,2,0,5],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182496,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":732696,"kb_used_data":322052,"kb_used_omap":4,"kb_used_meta":410619,"kb_avail":93639144,"statfs":{"total":96636764160,"available":95886483456,"internally_reserved":0,"allocated":329781248,"data_stored":329478559,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":420474538},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,0,5,8],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":329273344,"data_stored":329237214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":262197248,"data_stored":262131755,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":317145088,"data_stored":317104886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:05:43.535 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:05:44.111 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:05:44.111 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:05:44.133 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":294,"stamp":"2024-09-10T16:05:43.692868+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":344662793,"num_objects":84158,"num_object_clones":0,"num_object_copies":168316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42422,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":84158,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":207952,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8969,"num_bytes_recovered":33095699,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160679,"ondisk_log_size":160679,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1934320,"kb_used_data":878572,"kb_used_omap":12,"kb_used_meta":1055667,"kb_avail":281181200,"statfs":{"total":289910292480,"available":287929548800,"internally_reserved":0,"allocated":899657728,"data_stored":898720303,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1081003010},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,6,4,10,13],"upper_bound":128},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-20033536,"num_objects":-4891,"num_object_clones":0,"num_object_copies":-9782,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-210,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4891,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4891,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":210,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004622"},"pg_stats":[{"pgid":"2.f","version":"47'15479","reported_seq":15559,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:39.549937+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:05:39.549937+0000","last_peered":"2024-09-10T16:05:39.549937+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:05:39.549937+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"43'5400","ondisk_log_start":"43'5400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21270528,"num_objects":5193,"num_object_clones":0,"num_object_copies":10386,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5193,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15479,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":934}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'15621","reported_seq":15739,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:40.377166+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:05:40.377166+0000","last_peered":"2024-09-10T16:05:40.377166+0000","last_clean":"2024-09-10T16:05:40.377166+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:05:40.377166+0000","last_undegraded":"2024-09-10T16:05:40.377166+0000","last_fullsized":"2024-09-10T16:05:40.377166+0000","mapping_epoch":35,"log_start":"43'5600","ondisk_log_start":"43'5600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21394374,"num_objects":5225,"num_object_clones":0,"num_object_copies":10450,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5225,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13726,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'15617","reported_seq":24300,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:42.550983+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:05:42.550983+0000","last_peered":"2024-09-10T16:05:42.550983+0000","last_clean":"2024-09-10T16:05:42.550983+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:05:42.550983+0000","last_undegraded":"2024-09-10T16:05:42.550983+0000","last_fullsized":"2024-09-10T16:05:42.550983+0000","mapping_epoch":44,"log_start":"43'5600","ondisk_log_start":"43'5600","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21217304,"num_objects":5181,"num_object_clones":0,"num_object_copies":10362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5181,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13621,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'15525","reported_seq":15611,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:40.329870+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:05:40.329870+0000","last_peered":"2024-09-10T16:05:40.329870+0000","last_clean":"2024-09-10T16:05:40.329870+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:05:40.329870+0000","last_undegraded":"2024-09-10T16:05:40.329870+0000","last_fullsized":"2024-09-10T16:05:40.329870+0000","mapping_epoch":35,"log_start":"43'5500","ondisk_log_start":"43'5500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21032960,"num_objects":5135,"num_object_clones":0,"num_object_copies":10270,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5135,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15525,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'15603","reported_seq":15738,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:39.373753+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:05:39.373753+0000","last_peered":"2024-09-10T16:05:39.373753+0000","last_clean":"2024-09-10T16:05:39.373753+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:05:39.373753+0000","last_undegraded":"2024-09-10T16:05:39.373753+0000","last_fullsized":"2024-09-10T16:05:39.373753+0000","mapping_epoch":34,"log_start":"43'5600","ondisk_log_start":"43'5600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21660131,"num_objects":5289,"num_object_clones":0,"num_object_copies":10578,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5289,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12314,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'15802","reported_seq":15945,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:39.521465+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:05:39.521465+0000","last_peered":"2024-09-10T16:05:39.521465+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:05:39.521465+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"43'5800","ondisk_log_start":"43'5800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21541347,"num_objects":5260,"num_object_clones":0,"num_object_copies":10520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5260,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12414,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":889},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'15647","reported_seq":15790,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:39.485489+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:05:39.485489+0000","last_peered":"2024-09-10T16:05:39.485489+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:05:39.485489+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"43'5600","ondisk_log_start":"43'5600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22262726,"num_objects":5437,"num_object_clones":0,"num_object_copies":10874,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5437,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":12146,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1200},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'15943","reported_seq":16077,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:39.540704+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:05:39.540704+0000","last_peered":"2024-09-10T16:05:39.540704+0000","last_clean":"2024-09-10T16:05:39.540704+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:05:39.540704+0000","last_undegraded":"2024-09-10T16:05:39.540704+0000","last_fullsized":"2024-09-10T16:05:39.540704+0000","mapping_epoch":34,"log_start":"43'5900","ondisk_log_start":"43'5900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21999616,"num_objects":5371,"num_object_clones":0,"num_object_copies":10742,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5371,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12790,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.876895+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:05:19.876895+0000","last_peered":"2024-09-10T16:05:19.876895+0000","last_clean":"2024-09-10T16:05:19.876895+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:05:19.876895+0000","last_undegraded":"2024-09-10T16:05:19.876895+0000","last_fullsized":"2024-09-10T16:05:19.876895+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'15224","reported_seq":15369,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:40.360862+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:05:40.360862+0000","last_peered":"2024-09-10T16:05:40.360862+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:05:40.360862+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"43'5200","ondisk_log_start":"43'5200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21434851,"num_objects":5234,"num_object_clones":0,"num_object_copies":10468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5234,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13043,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":1111}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":99,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.875776+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:05:19.875776+0000","last_peered":"2024-09-10T16:05:19.875776+0000","last_clean":"2024-09-10T16:05:19.875776+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:05:19.875776+0000","last_undegraded":"2024-09-10T16:05:19.875776+0000","last_fullsized":"2024-09-10T16:05:19.875776+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'15570","reported_seq":15706,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:39.484259+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:05:39.484259+0000","last_peered":"2024-09-10T16:05:39.484259+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:05:39.484259+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"43'5500","ondisk_log_start":"43'5500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21872640,"num_objects":5340,"num_object_clones":0,"num_object_copies":10680,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5340,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12074,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1085}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":109,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.871983+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:05:19.871983+0000","last_peered":"2024-09-10T16:05:19.871983+0000","last_clean":"2024-09-10T16:05:19.871983+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:05:19.871983+0000","last_undegraded":"2024-09-10T16:05:19.871983+0000","last_fullsized":"2024-09-10T16:05:19.871983+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'15569","reported_seq":15709,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:40.371024+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:05:40.371024+0000","last_peered":"2024-09-10T16:05:40.371024+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:05:40.371024+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"43'5500","ondisk_log_start":"43'5500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21348835,"num_objects":5213,"num_object_clones":0,"num_object_copies":10426,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5213,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13249,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":909}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877148+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:05:19.877148+0000","last_peered":"2024-09-10T16:05:19.877148+0000","last_clean":"2024-09-10T16:05:19.877148+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:05:19.877148+0000","last_undegraded":"2024-09-10T16:05:19.877148+0000","last_fullsized":"2024-09-10T16:05:19.877148+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'15848","reported_seq":16961,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:40.428026+0000","last_change":"2024-09-10T16:05:18.271152+0000","last_active":"2024-09-10T16:05:40.428026+0000","last_peered":"2024-09-10T16:05:40.428026+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:05:40.428026+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"43'5800","ondisk_log_start":"43'5800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21798912,"num_objects":5322,"num_object_clones":0,"num_object_copies":10644,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3865,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5322,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13402,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":489,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1457}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'15604","reported_seq":15742,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:39.553773+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:05:39.553773+0000","last_peered":"2024-09-10T16:05:39.553773+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:05:39.553773+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"43'5600","ondisk_log_start":"43'5600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21438464,"num_objects":5234,"num_object_clones":0,"num_object_copies":10468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5234,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12235,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":930}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877654+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:05:19.877654+0000","last_peered":"2024-09-10T16:05:19.877654+0000","last_clean":"2024-09-10T16:05:19.877654+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:05:19.877654+0000","last_undegraded":"2024-09-10T16:05:19.877654+0000","last_fullsized":"2024-09-10T16:05:19.877654+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'15484","reported_seq":24055,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:42.707743+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:05:42.707743+0000","last_peered":"2024-09-10T16:05:42.707743+0000","last_clean":"2024-09-10T16:05:42.707743+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:05:42.707743+0000","last_undegraded":"2024-09-10T16:05:42.707743+0000","last_fullsized":"2024-09-10T16:05:42.707743+0000","mapping_epoch":46,"log_start":"43'5400","ondisk_log_start":"43'5400","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20946944,"num_objects":5114,"num_object_clones":0,"num_object_copies":10228,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5114,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11874,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":99,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.876013+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:05:19.876013+0000","last_peered":"2024-09-10T16:05:19.876013+0000","last_clean":"2024-09-10T16:05:19.876013+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:05:19.876013+0000","last_undegraded":"2024-09-10T16:05:19.876013+0000","last_fullsized":"2024-09-10T16:05:19.876013+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'15798","reported_seq":15945,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:39.514210+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:05:39.514210+0000","last_peered":"2024-09-10T16:05:39.514210+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:05:39.514210+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"43'5700","ondisk_log_start":"43'5700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21504000,"num_objects":5250,"num_object_clones":0,"num_object_copies":10500,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5250,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12053,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":879}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877349+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:05:19.877349+0000","last_peered":"2024-09-10T16:05:19.877349+0000","last_clean":"2024-09-10T16:05:19.877349+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:05:19.877349+0000","last_undegraded":"2024-09-10T16:05:19.877349+0000","last_fullsized":"2024-09-10T16:05:19.877349+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877229+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:05:19.877229+0000","last_peered":"2024-09-10T16:05:19.877229+0000","last_clean":"2024-09-10T16:05:19.877229+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:05:19.877229+0000","last_undegraded":"2024-09-10T16:05:19.877229+0000","last_fullsized":"2024-09-10T16:05:19.877229+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'15842","reported_seq":15992,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:39.433158+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:05:39.433158+0000","last_peered":"2024-09-10T16:05:39.433158+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:05:39.433158+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"43'5800","ondisk_log_start":"43'5800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21939142,"num_objects":5358,"num_object_clones":0,"num_object_copies":10716,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5358,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12005,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1035},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":344662774,"num_objects":84156,"num_object_clones":0,"num_object_copies":168312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42422,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":84156,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":207950,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8967,"num_bytes_recovered":33095680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":897044480,"data_stored":896902655,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160676,"ondisk_log_size":160676,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986268,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":634692,"kb_used_data":305904,"kb_used_omap":4,"kb_used_meta":328763,"kb_avail":93737148,"statfs":{"total":96636764160,"available":95986839552,"internally_reserved":0,"allocated":313245696,"data_stored":312943012,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":336653994},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,3,1,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659214,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":566932,"kb_used_data":250616,"kb_used_omap":4,"kb_used_meta":316283,"kb_avail":93804908,"statfs":{"total":96636764160,"available":96056225792,"internally_reserved":0,"allocated":256630784,"data_stored":256298732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":323874478},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,3,4,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182496,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":732696,"kb_used_data":322052,"kb_used_omap":4,"kb_used_meta":410619,"kb_avail":93639144,"statfs":{"total":96636764160,"available":95886483456,"internally_reserved":0,"allocated":329781248,"data_stored":329478559,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":420474538},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,0,5,8],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":329273344,"data_stored":329237214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":255201280,"data_stored":255135787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":312569856,"data_stored":312529654,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:05:44.134 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:05:44.726 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:05:44.728 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:05:44.752 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":294,"stamp":"2024-09-10T16:05:43.692868+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":344662793,"num_objects":84158,"num_object_clones":0,"num_object_copies":168316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42422,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":84158,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":207952,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8969,"num_bytes_recovered":33095699,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160679,"ondisk_log_size":160679,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1934320,"kb_used_data":878572,"kb_used_omap":12,"kb_used_meta":1055667,"kb_avail":281181200,"statfs":{"total":289910292480,"available":287929548800,"internally_reserved":0,"allocated":899657728,"data_stored":898720303,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1081003010},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,6,4,10,13],"upper_bound":128},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-20033536,"num_objects":-4891,"num_object_clones":0,"num_object_copies":-9782,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-210,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4891,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4891,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":210,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004622"},"pg_stats":[{"pgid":"2.f","version":"47'15479","reported_seq":15559,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:39.549937+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:05:39.549937+0000","last_peered":"2024-09-10T16:05:39.549937+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:05:39.549937+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"43'5400","ondisk_log_start":"43'5400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21270528,"num_objects":5193,"num_object_clones":0,"num_object_copies":10386,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5193,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15479,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":934}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'15621","reported_seq":15739,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:40.377166+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:05:40.377166+0000","last_peered":"2024-09-10T16:05:40.377166+0000","last_clean":"2024-09-10T16:05:40.377166+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:05:40.377166+0000","last_undegraded":"2024-09-10T16:05:40.377166+0000","last_fullsized":"2024-09-10T16:05:40.377166+0000","mapping_epoch":35,"log_start":"43'5600","ondisk_log_start":"43'5600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21394374,"num_objects":5225,"num_object_clones":0,"num_object_copies":10450,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5225,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13726,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'15617","reported_seq":24300,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:42.550983+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:05:42.550983+0000","last_peered":"2024-09-10T16:05:42.550983+0000","last_clean":"2024-09-10T16:05:42.550983+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:05:42.550983+0000","last_undegraded":"2024-09-10T16:05:42.550983+0000","last_fullsized":"2024-09-10T16:05:42.550983+0000","mapping_epoch":44,"log_start":"43'5600","ondisk_log_start":"43'5600","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21217304,"num_objects":5181,"num_object_clones":0,"num_object_copies":10362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5181,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13621,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'15525","reported_seq":15611,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:40.329870+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:05:40.329870+0000","last_peered":"2024-09-10T16:05:40.329870+0000","last_clean":"2024-09-10T16:05:40.329870+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:05:40.329870+0000","last_undegraded":"2024-09-10T16:05:40.329870+0000","last_fullsized":"2024-09-10T16:05:40.329870+0000","mapping_epoch":35,"log_start":"43'5500","ondisk_log_start":"43'5500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21032960,"num_objects":5135,"num_object_clones":0,"num_object_copies":10270,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5135,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15525,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'15603","reported_seq":15738,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:39.373753+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:05:39.373753+0000","last_peered":"2024-09-10T16:05:39.373753+0000","last_clean":"2024-09-10T16:05:39.373753+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:05:39.373753+0000","last_undegraded":"2024-09-10T16:05:39.373753+0000","last_fullsized":"2024-09-10T16:05:39.373753+0000","mapping_epoch":34,"log_start":"43'5600","ondisk_log_start":"43'5600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21660131,"num_objects":5289,"num_object_clones":0,"num_object_copies":10578,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5289,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12314,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'15802","reported_seq":15945,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:39.521465+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:05:39.521465+0000","last_peered":"2024-09-10T16:05:39.521465+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:05:39.521465+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"43'5800","ondisk_log_start":"43'5800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21541347,"num_objects":5260,"num_object_clones":0,"num_object_copies":10520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5260,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12414,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":889},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'15647","reported_seq":15790,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:39.485489+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:05:39.485489+0000","last_peered":"2024-09-10T16:05:39.485489+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:05:39.485489+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"43'5600","ondisk_log_start":"43'5600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22262726,"num_objects":5437,"num_object_clones":0,"num_object_copies":10874,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5437,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":12146,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1200},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'15943","reported_seq":16077,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:39.540704+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:05:39.540704+0000","last_peered":"2024-09-10T16:05:39.540704+0000","last_clean":"2024-09-10T16:05:39.540704+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:05:39.540704+0000","last_undegraded":"2024-09-10T16:05:39.540704+0000","last_fullsized":"2024-09-10T16:05:39.540704+0000","mapping_epoch":34,"log_start":"43'5900","ondisk_log_start":"43'5900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21999616,"num_objects":5371,"num_object_clones":0,"num_object_copies":10742,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5371,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12790,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.876895+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:05:19.876895+0000","last_peered":"2024-09-10T16:05:19.876895+0000","last_clean":"2024-09-10T16:05:19.876895+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:05:19.876895+0000","last_undegraded":"2024-09-10T16:05:19.876895+0000","last_fullsized":"2024-09-10T16:05:19.876895+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'15224","reported_seq":15369,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:40.360862+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:05:40.360862+0000","last_peered":"2024-09-10T16:05:40.360862+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:05:40.360862+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"43'5200","ondisk_log_start":"43'5200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21434851,"num_objects":5234,"num_object_clones":0,"num_object_copies":10468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5234,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13043,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":1111}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":99,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.875776+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:05:19.875776+0000","last_peered":"2024-09-10T16:05:19.875776+0000","last_clean":"2024-09-10T16:05:19.875776+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:05:19.875776+0000","last_undegraded":"2024-09-10T16:05:19.875776+0000","last_fullsized":"2024-09-10T16:05:19.875776+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'15570","reported_seq":15706,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:39.484259+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:05:39.484259+0000","last_peered":"2024-09-10T16:05:39.484259+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:05:39.484259+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"43'5500","ondisk_log_start":"43'5500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21872640,"num_objects":5340,"num_object_clones":0,"num_object_copies":10680,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5340,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12074,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1085}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":109,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.871983+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:05:19.871983+0000","last_peered":"2024-09-10T16:05:19.871983+0000","last_clean":"2024-09-10T16:05:19.871983+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:05:19.871983+0000","last_undegraded":"2024-09-10T16:05:19.871983+0000","last_fullsized":"2024-09-10T16:05:19.871983+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'15569","reported_seq":15709,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:40.371024+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:05:40.371024+0000","last_peered":"2024-09-10T16:05:40.371024+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:05:40.371024+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"43'5500","ondisk_log_start":"43'5500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21348835,"num_objects":5213,"num_object_clones":0,"num_object_copies":10426,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5213,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13249,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":909}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877148+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:05:19.877148+0000","last_peered":"2024-09-10T16:05:19.877148+0000","last_clean":"2024-09-10T16:05:19.877148+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:05:19.877148+0000","last_undegraded":"2024-09-10T16:05:19.877148+0000","last_fullsized":"2024-09-10T16:05:19.877148+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'15848","reported_seq":16961,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:40.428026+0000","last_change":"2024-09-10T16:05:18.271152+0000","last_active":"2024-09-10T16:05:40.428026+0000","last_peered":"2024-09-10T16:05:40.428026+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:05:40.428026+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"43'5800","ondisk_log_start":"43'5800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21798912,"num_objects":5322,"num_object_clones":0,"num_object_copies":10644,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3865,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5322,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13402,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":489,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1457}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'15604","reported_seq":15742,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:39.553773+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:05:39.553773+0000","last_peered":"2024-09-10T16:05:39.553773+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:05:39.553773+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"43'5600","ondisk_log_start":"43'5600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21438464,"num_objects":5234,"num_object_clones":0,"num_object_copies":10468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5234,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12235,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":930}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877654+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:05:19.877654+0000","last_peered":"2024-09-10T16:05:19.877654+0000","last_clean":"2024-09-10T16:05:19.877654+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:05:19.877654+0000","last_undegraded":"2024-09-10T16:05:19.877654+0000","last_fullsized":"2024-09-10T16:05:19.877654+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'15484","reported_seq":24055,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:42.707743+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:05:42.707743+0000","last_peered":"2024-09-10T16:05:42.707743+0000","last_clean":"2024-09-10T16:05:42.707743+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:05:42.707743+0000","last_undegraded":"2024-09-10T16:05:42.707743+0000","last_fullsized":"2024-09-10T16:05:42.707743+0000","mapping_epoch":46,"log_start":"43'5400","ondisk_log_start":"43'5400","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20946944,"num_objects":5114,"num_object_clones":0,"num_object_copies":10228,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5114,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11874,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":99,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.876013+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:05:19.876013+0000","last_peered":"2024-09-10T16:05:19.876013+0000","last_clean":"2024-09-10T16:05:19.876013+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:05:19.876013+0000","last_undegraded":"2024-09-10T16:05:19.876013+0000","last_fullsized":"2024-09-10T16:05:19.876013+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'15798","reported_seq":15945,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:39.514210+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:05:39.514210+0000","last_peered":"2024-09-10T16:05:39.514210+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:05:39.514210+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"43'5700","ondisk_log_start":"43'5700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21504000,"num_objects":5250,"num_object_clones":0,"num_object_copies":10500,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5250,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12053,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":879}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877349+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:05:19.877349+0000","last_peered":"2024-09-10T16:05:19.877349+0000","last_clean":"2024-09-10T16:05:19.877349+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:05:19.877349+0000","last_undegraded":"2024-09-10T16:05:19.877349+0000","last_fullsized":"2024-09-10T16:05:19.877349+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877229+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:05:19.877229+0000","last_peered":"2024-09-10T16:05:19.877229+0000","last_clean":"2024-09-10T16:05:19.877229+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:05:19.877229+0000","last_undegraded":"2024-09-10T16:05:19.877229+0000","last_fullsized":"2024-09-10T16:05:19.877229+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'15842","reported_seq":15992,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:39.433158+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:05:39.433158+0000","last_peered":"2024-09-10T16:05:39.433158+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:05:39.433158+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"43'5800","ondisk_log_start":"43'5800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21939142,"num_objects":5358,"num_object_clones":0,"num_object_copies":10716,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5358,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12005,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1035},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":344662774,"num_objects":84156,"num_object_clones":0,"num_object_copies":168312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42422,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":84156,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":207950,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8967,"num_bytes_recovered":33095680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":897044480,"data_stored":896902655,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160676,"ondisk_log_size":160676,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986268,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":634692,"kb_used_data":305904,"kb_used_omap":4,"kb_used_meta":328763,"kb_avail":93737148,"statfs":{"total":96636764160,"available":95986839552,"internally_reserved":0,"allocated":313245696,"data_stored":312943012,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":336653994},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,3,1,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659214,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":566932,"kb_used_data":250616,"kb_used_omap":4,"kb_used_meta":316283,"kb_avail":93804908,"statfs":{"total":96636764160,"available":96056225792,"internally_reserved":0,"allocated":256630784,"data_stored":256298732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":323874478},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,3,4,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182496,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":732696,"kb_used_data":322052,"kb_used_omap":4,"kb_used_meta":410619,"kb_avail":93639144,"statfs":{"total":96636764160,"available":95886483456,"internally_reserved":0,"allocated":329781248,"data_stored":329478559,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":420474538},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,0,5,8],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":329273344,"data_stored":329237214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":255201280,"data_stored":255135787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":312569856,"data_stored":312529654,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:05:44.754 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-10T16:05:44.754 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "47'15479", 'reported_seq': 15559, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:05:39.549937+0000', 'last_change': '2024-09-10T15:59:16.281558+0000', 'last_active': '2024-09-10T16:05:39.549937+0000', 'last_peered': '2024-09-10T16:05:39.549937+0000', 'last_clean': '2024-09-10T15:59:14.132456+0000', 'last_became_active': '2024-09-10T15:59:16.274407+0000', 'last_became_peered': '2024-09-10T15:59:16.274407+0000', 'last_unstale': '2024-09-10T16:05:39.549937+0000', 'last_undegraded': '2024-09-10T15:59:15.851279+0000', 'last_fullsized': '2024-09-10T15:59:15.774356+0000', 'mapping_epoch': 42, 'log_start': "43'5400", 'ondisk_log_start': "43'5400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'objects_scrubbed': 0, 'log_size': 10079, 'log_dups_size': 0, 'ondisk_log_size': 10079, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21270528, 'num_objects': 5193, 'num_object_clones': 0, 'num_object_copies': 10386, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4259, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5193, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15479, 'num_write_kb': 41344, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 934}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:05:44.754 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-10T16:05:44.754 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "47'15802", 'reported_seq': 15945, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:05:39.521465+0000', 'last_change': '2024-09-10T15:59:16.308264+0000', 'last_active': '2024-09-10T16:05:39.521465+0000', 'last_peered': '2024-09-10T16:05:39.521465+0000', 'last_clean': '2024-09-10T15:59:14.111551+0000', 'last_became_active': '2024-09-10T15:59:16.300040+0000', 'last_became_peered': '2024-09-10T15:59:16.300040+0000', 'last_unstale': '2024-09-10T16:05:39.521465+0000', 'last_undegraded': '2024-09-10T15:59:15.878234+0000', 'last_fullsized': '2024-09-10T15:59:15.769328+0000', 'mapping_epoch': 42, 'log_start': "43'5800", 'ondisk_log_start': "43'5800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3442", 'last_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'objects_scrubbed': 3384, 'log_size': 10002, 'log_dups_size': 0, 'ondisk_log_size': 10002, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1292, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21541347, 'num_objects': 5260, 'num_object_clones': 0, 'num_object_copies': 10520, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4372, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5260, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 12414, 'num_write_kb': 28580, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 889}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:05:44.754 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-10T16:05:44.754 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "47'15647", 'reported_seq': 15790, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:05:39.485489+0000', 'last_change': '2024-09-10T15:59:16.331899+0000', 'last_active': '2024-09-10T16:05:39.485489+0000', 'last_peered': '2024-09-10T16:05:39.485489+0000', 'last_clean': '2024-09-10T15:59:14.139436+0000', 'last_became_active': '2024-09-10T15:59:16.318715+0000', 'last_became_peered': '2024-09-10T15:59:16.318715+0000', 'last_unstale': '2024-09-10T16:05:39.485489+0000', 'last_undegraded': '2024-09-10T15:59:15.877524+0000', 'last_fullsized': '2024-09-10T15:59:15.769042+0000', 'mapping_epoch': 42, 'log_start': "43'5600", 'ondisk_log_start': "43'5600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3537", 'last_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'objects_scrubbed': 3497, 'log_size': 10047, 'log_dups_size': 0, 'ondisk_log_size': 10047, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1224, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22262726, 'num_objects': 5437, 'num_object_clones': 0, 'num_object_copies': 10874, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4237, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5437, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 12146, 'num_write_kb': 28172, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1200}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:05:44.754 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-10T16:05:44.754 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "47'15224", 'reported_seq': 15369, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:05:40.360862+0000', 'last_change': '2024-09-10T15:59:16.175680+0000', 'last_active': '2024-09-10T16:05:40.360862+0000', 'last_peered': '2024-09-10T16:05:40.360862+0000', 'last_clean': '2024-09-10T15:59:14.088187+0000', 'last_became_active': '2024-09-10T15:59:16.165810+0000', 'last_became_peered': '2024-09-10T15:59:16.165810+0000', 'last_unstale': '2024-09-10T16:05:40.360862+0000', 'last_undegraded': '2024-09-10T15:59:15.822098+0000', 'last_fullsized': '2024-09-10T15:59:15.769123+0000', 'mapping_epoch': 42, 'log_start': "43'5200", 'ondisk_log_start': "43'5200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2218", 'last_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'objects_scrubbed': 2175, 'log_size': 10024, 'log_dups_size': 0, 'ondisk_log_size': 10024, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 825, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21434851, 'num_objects': 5234, 'num_object_clones': 0, 'num_object_copies': 10468, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4124, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5234, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 13043, 'num_write_kb': 32204, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': 1111}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:05:44.755 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-10T16:05:44.755 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "47'15570", 'reported_seq': 15706, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:05:39.484259+0000', 'last_change': '2024-09-10T15:59:16.277742+0000', 'last_active': '2024-09-10T16:05:39.484259+0000', 'last_peered': '2024-09-10T16:05:39.484259+0000', 'last_clean': '2024-09-10T15:59:14.129081+0000', 'last_became_active': '2024-09-10T15:59:16.269835+0000', 'last_became_peered': '2024-09-10T15:59:16.269835+0000', 'last_unstale': '2024-09-10T16:05:39.484259+0000', 'last_undegraded': '2024-09-10T15:59:15.860857+0000', 'last_fullsized': '2024-09-10T15:59:15.771410+0000', 'mapping_epoch': 42, 'log_start': "43'5500", 'ondisk_log_start': "43'5500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3557", 'last_scrub_stamp': '2024-09-10T15:59:02.493678+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:02.493678+0000', 'objects_scrubbed': 3488, 'log_size': 10070, 'log_dups_size': 0, 'ondisk_log_size': 10070, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1472, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21872640, 'num_objects': 5340, 'num_object_clones': 0, 'num_object_copies': 10680, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4257, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5340, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12074, 'num_write_kb': 27852, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1085}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:05:44.755 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-10T16:05:44.755 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "47'15569", 'reported_seq': 15709, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:05:40.371024+0000', 'last_change': '2024-09-10T15:59:16.284311+0000', 'last_active': '2024-09-10T16:05:40.371024+0000', 'last_peered': '2024-09-10T16:05:40.371024+0000', 'last_clean': '2024-09-10T15:59:14.116041+0000', 'last_became_active': '2024-09-10T15:59:16.274543+0000', 'last_became_peered': '2024-09-10T15:59:16.274543+0000', 'last_unstale': '2024-09-10T16:05:40.371024+0000', 'last_undegraded': '2024-09-10T15:59:15.812484+0000', 'last_fullsized': '2024-09-10T15:59:15.770885+0000', 'mapping_epoch': 42, 'log_start': "43'5500", 'ondisk_log_start': "43'5500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2356", 'last_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'objects_scrubbed': 2316, 'log_size': 10069, 'log_dups_size': 0, 'ondisk_log_size': 10069, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 783, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21348835, 'num_objects': 5213, 'num_object_clones': 0, 'num_object_copies': 10426, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5213, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 13249, 'num_write_kb': 32292, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': 909}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:05:44.755 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-10T16:05:44.755 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "47'15848", 'reported_seq': 16961, 'reported_epoch': 47, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:05:40.428026+0000', 'last_change': '2024-09-10T16:05:18.271152+0000', 'last_active': '2024-09-10T16:05:40.428026+0000', 'last_peered': '2024-09-10T16:05:40.428026+0000', 'last_clean': '2024-09-10T15:59:14.110654+0000', 'last_became_active': '2024-09-10T15:59:16.164645+0000', 'last_became_peered': '2024-09-10T15:59:16.164645+0000', 'last_unstale': '2024-09-10T16:05:40.428026+0000', 'last_undegraded': '2024-09-10T15:59:15.803593+0000', 'last_fullsized': '2024-09-10T15:59:15.769083+0000', 'mapping_epoch': 42, 'log_start': "43'5800", 'ondisk_log_start': "43'5800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2486", 'last_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'objects_scrubbed': 2440, 'log_size': 10048, 'log_dups_size': 0, 'ondisk_log_size': 10048, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 792, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21798912, 'num_objects': 5322, 'num_object_clones': 0, 'num_object_copies': 10644, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3865, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5322, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13402, 'num_write_kb': 32568, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 489, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 1457}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:05:44.755 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-10T16:05:44.755 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "47'15604", 'reported_seq': 15742, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:05:39.553773+0000', 'last_change': '2024-09-10T15:59:16.315641+0000', 'last_active': '2024-09-10T16:05:39.553773+0000', 'last_peered': '2024-09-10T16:05:39.553773+0000', 'last_clean': '2024-09-10T15:59:14.117001+0000', 'last_became_active': '2024-09-10T15:59:16.302245+0000', 'last_became_peered': '2024-09-10T15:59:16.302245+0000', 'last_unstale': '2024-09-10T16:05:39.553773+0000', 'last_undegraded': '2024-09-10T15:59:15.845012+0000', 'last_fullsized': '2024-09-10T15:59:15.770337+0000', 'mapping_epoch': 42, 'log_start': "43'5600", 'ondisk_log_start': "43'5600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3404", 'last_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'objects_scrubbed': 3367, 'log_size': 10004, 'log_dups_size': 0, 'ondisk_log_size': 10004, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 983, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21438464, 'num_objects': 5234, 'num_object_clones': 0, 'num_object_copies': 10468, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5234, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12235, 'num_write_kb': 28204, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 930}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:05:44.755 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-10T16:05:44.755 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "47'15798", 'reported_seq': 15945, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:05:39.514210+0000', 'last_change': '2024-09-10T15:59:16.311291+0000', 'last_active': '2024-09-10T16:05:39.514210+0000', 'last_peered': '2024-09-10T16:05:39.514210+0000', 'last_clean': '2024-09-10T15:59:14.116363+0000', 'last_became_active': '2024-09-10T15:59:16.301244+0000', 'last_became_peered': '2024-09-10T15:59:16.301244+0000', 'last_unstale': '2024-09-10T16:05:39.514210+0000', 'last_undegraded': '2024-09-10T15:59:15.883434+0000', 'last_fullsized': '2024-09-10T15:59:15.772756+0000', 'mapping_epoch': 42, 'log_start': "43'5700", 'ondisk_log_start': "43'5700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3789", 'last_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'objects_scrubbed': 3741, 'log_size': 10098, 'log_dups_size': 0, 'ondisk_log_size': 10098, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1440, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21504000, 'num_objects': 5250, 'num_object_clones': 0, 'num_object_copies': 10500, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4373, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5250, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12053, 'num_write_kb': 27124, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 879}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:05:44.756 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-10T16:05:44.756 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "47'15842", 'reported_seq': 15992, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:05:39.433158+0000', 'last_change': '2024-09-10T15:59:16.313409+0000', 'last_active': '2024-09-10T16:05:39.433158+0000', 'last_peered': '2024-09-10T16:05:39.433158+0000', 'last_clean': '2024-09-10T15:59:14.129780+0000', 'last_became_active': '2024-09-10T15:59:16.300932+0000', 'last_became_peered': '2024-09-10T15:59:16.300932+0000', 'last_unstale': '2024-09-10T16:05:39.433158+0000', 'last_undegraded': '2024-09-10T15:59:15.887823+0000', 'last_fullsized': '2024-09-10T15:59:15.768513+0000', 'mapping_epoch': 42, 'log_start': "43'5800", 'ondisk_log_start': "43'5800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3880", 'last_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'objects_scrubbed': 3827, 'log_size': 10042, 'log_dups_size': 0, 'ondisk_log_size': 10042, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1397, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21939142, 'num_objects': 5358, 'num_object_clones': 0, 'num_object_copies': 10716, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4325, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5358, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12005, 'num_write_kb': 27072, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1035}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:05:44.756 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T16:05:45.439 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:05:45.440 INFO:teuthology.orchestra.run.smithi115.stdout:{"fsid":"7effee29-df05-4351-9741-ee203564ed81","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 42422/168316 objects degraded (25.204%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":509,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":47,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983954,"num_in_osds":3,"osd_in_since":1725983947,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":84158,"data_bytes":344662793,"bytes_used":1980743680,"bytes_avail":287929548800,"bytes_total":289910292480,"degraded_objects":42422,"degraded_total":168316,"degraded_ratio":0.25203783359870718,"recovering_objects_per_sec":17,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":210,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":407},"fsmap":{"epoch":1,"btime":"2024-09-10T15:57:14:967120+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T16:00:13.560776+0000","services":{}},"progress_events":{}} 2024-09-10T16:05:45.470 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-10T16:05:45.471 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:05:46.061 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:05:46.062 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:05:46.090 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":295,"stamp":"2024-09-10T16:05:45.693590+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":336397065,"num_objects":82140,"num_object_clones":0,"num_object_copies":164280,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42322,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":82140,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":209970,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9069,"num_bytes_recovered":33095699,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160897,"ondisk_log_size":160897,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1915508,"kb_used_data":868088,"kb_used_omap":12,"kb_used_meta":1047347,"kb_avail":281200012,"statfs":{"total":289910292480,"available":287948812288,"internally_reserved":0,"allocated":888922112,"data_stored":887984687,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1072483330},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,2,5,7,7],"upper_bound":128},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-27103232,"num_objects":-6617,"num_object_clones":0,"num_object_copies":-13234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-310,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-6617,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6617,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":310,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004529"},"pg_stats":[{"pgid":"2.f","version":"47'15608","reported_seq":15688,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:44.470543+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:05:44.470543+0000","last_peered":"2024-09-10T16:05:44.470543+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:05:44.470543+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"43'5600","ondisk_log_start":"43'5600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20742144,"num_objects":5064,"num_object_clones":0,"num_object_copies":10128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5064,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15608,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":805}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'15765","reported_seq":15883,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:45.439477+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:05:45.439477+0000","last_peered":"2024-09-10T16:05:45.439477+0000","last_clean":"2024-09-10T16:05:45.439477+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:05:45.439477+0000","last_undegraded":"2024-09-10T16:05:45.439477+0000","last_fullsized":"2024-09-10T16:05:45.439477+0000","mapping_epoch":35,"log_start":"43'5700","ondisk_log_start":"43'5700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20804550,"num_objects":5081,"num_object_clones":0,"num_object_copies":10162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5081,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13870,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'15617","reported_seq":24300,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:42.550983+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:05:42.550983+0000","last_peered":"2024-09-10T16:05:42.550983+0000","last_clean":"2024-09-10T16:05:42.550983+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:05:42.550983+0000","last_undegraded":"2024-09-10T16:05:42.550983+0000","last_fullsized":"2024-09-10T16:05:42.550983+0000","mapping_epoch":44,"log_start":"43'5600","ondisk_log_start":"43'5600","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21217304,"num_objects":5181,"num_object_clones":0,"num_object_copies":10362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5181,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13621,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'15679","reported_seq":15765,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:45.307160+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:05:45.307160+0000","last_peered":"2024-09-10T16:05:45.307160+0000","last_clean":"2024-09-10T16:05:45.307160+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:05:45.307160+0000","last_undegraded":"2024-09-10T16:05:45.307160+0000","last_fullsized":"2024-09-10T16:05:45.307160+0000","mapping_epoch":35,"log_start":"43'5600","ondisk_log_start":"43'5600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20402176,"num_objects":4981,"num_object_clones":0,"num_object_copies":9962,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4981,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15679,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'15771","reported_seq":15906,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:44.518717+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:05:44.518717+0000","last_peered":"2024-09-10T16:05:44.518717+0000","last_clean":"2024-09-10T16:05:44.518717+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:05:44.518717+0000","last_undegraded":"2024-09-10T16:05:44.518717+0000","last_fullsized":"2024-09-10T16:05:44.518717+0000","mapping_epoch":34,"log_start":"43'5700","ondisk_log_start":"43'5700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20972003,"num_objects":5121,"num_object_clones":0,"num_object_copies":10242,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5121,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12482,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'15940","reported_seq":16083,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:44.532107+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:05:44.532107+0000","last_peered":"2024-09-10T16:05:44.532107+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:05:44.532107+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"43'5900","ondisk_log_start":"43'5900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20976099,"num_objects":5122,"num_object_clones":0,"num_object_copies":10244,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5122,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12552,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":751},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'15787","reported_seq":15928,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:44.451315+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:05:44.451315+0000","last_peered":"2024-09-10T16:05:44.451315+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:05:44.451315+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"43'5700","ondisk_log_start":"43'5700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21689286,"num_objects":5297,"num_object_clones":0,"num_object_copies":10594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5297,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":12286,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1060},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'16087","reported_seq":16222,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:44.577475+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:05:44.577475+0000","last_peered":"2024-09-10T16:05:44.577475+0000","last_clean":"2024-09-10T16:05:44.577475+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:05:44.577475+0000","last_undegraded":"2024-09-10T16:05:44.577475+0000","last_fullsized":"2024-09-10T16:05:44.577475+0000","mapping_epoch":34,"log_start":"43'6000","ondisk_log_start":"43'6000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21409792,"num_objects":5227,"num_object_clones":0,"num_object_copies":10454,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5227,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12934,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.876895+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:05:19.876895+0000","last_peered":"2024-09-10T16:05:19.876895+0000","last_clean":"2024-09-10T16:05:19.876895+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:05:19.876895+0000","last_undegraded":"2024-09-10T16:05:19.876895+0000","last_fullsized":"2024-09-10T16:05:19.876895+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'15347","reported_seq":15491,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:45.399785+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:05:45.399785+0000","last_peered":"2024-09-10T16:05:45.399785+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:05:45.399785+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"43'5300","ondisk_log_start":"43'5300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20931043,"num_objects":5111,"num_object_clones":0,"num_object_copies":10222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5111,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13166,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":988}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":99,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.875776+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:05:19.875776+0000","last_peered":"2024-09-10T16:05:19.875776+0000","last_clean":"2024-09-10T16:05:19.875776+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:05:19.875776+0000","last_undegraded":"2024-09-10T16:05:19.875776+0000","last_fullsized":"2024-09-10T16:05:19.875776+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'15703","reported_seq":15839,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:44.532728+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:05:44.532728+0000","last_peered":"2024-09-10T16:05:44.532728+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:05:44.532728+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"43'5700","ondisk_log_start":"43'5700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21327872,"num_objects":5207,"num_object_clones":0,"num_object_copies":10414,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5207,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12207,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":952}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":109,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.871983+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:05:19.871983+0000","last_peered":"2024-09-10T16:05:19.871983+0000","last_clean":"2024-09-10T16:05:19.871983+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:05:19.871983+0000","last_undegraded":"2024-09-10T16:05:19.871983+0000","last_fullsized":"2024-09-10T16:05:19.871983+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'15720","reported_seq":15860,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:45.444720+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:05:45.444720+0000","last_peered":"2024-09-10T16:05:45.444720+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:05:45.444720+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"43'5700","ondisk_log_start":"43'5700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20730339,"num_objects":5062,"num_object_clones":0,"num_object_copies":10124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5062,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13400,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":758}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877148+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:05:19.877148+0000","last_peered":"2024-09-10T16:05:19.877148+0000","last_clean":"2024-09-10T16:05:19.877148+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:05:19.877148+0000","last_undegraded":"2024-09-10T16:05:19.877148+0000","last_fullsized":"2024-09-10T16:05:19.877148+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'16000","reported_seq":17313,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:45.395020+0000","last_change":"2024-09-10T16:05:18.271152+0000","last_active":"2024-09-10T16:05:45.395020+0000","last_peered":"2024-09-10T16:05:45.395020+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:05:45.395020+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"43'5900","ondisk_log_start":"43'5900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21176320,"num_objects":5170,"num_object_clones":0,"num_object_copies":10340,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3765,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5170,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13554,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":589,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1405}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'15740","reported_seq":15880,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:44.522997+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:05:44.522997+0000","last_peered":"2024-09-10T16:05:44.522997+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:05:44.522997+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"43'5700","ondisk_log_start":"43'5700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20881408,"num_objects":5098,"num_object_clones":0,"num_object_copies":10196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5098,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12371,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":794}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877654+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:05:19.877654+0000","last_peered":"2024-09-10T16:05:19.877654+0000","last_clean":"2024-09-10T16:05:19.877654+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:05:19.877654+0000","last_undegraded":"2024-09-10T16:05:19.877654+0000","last_fullsized":"2024-09-10T16:05:19.877654+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'15484","reported_seq":24055,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:42.707743+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:05:42.707743+0000","last_peered":"2024-09-10T16:05:42.707743+0000","last_clean":"2024-09-10T16:05:42.707743+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:05:42.707743+0000","last_undegraded":"2024-09-10T16:05:42.707743+0000","last_fullsized":"2024-09-10T16:05:42.707743+0000","mapping_epoch":46,"log_start":"43'5400","ondisk_log_start":"43'5400","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20946944,"num_objects":5114,"num_object_clones":0,"num_object_copies":10228,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5114,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11874,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":99,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.876013+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:05:19.876013+0000","last_peered":"2024-09-10T16:05:19.876013+0000","last_clean":"2024-09-10T16:05:19.876013+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:05:19.876013+0000","last_undegraded":"2024-09-10T16:05:19.876013+0000","last_fullsized":"2024-09-10T16:05:19.876013+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'15960","reported_seq":16107,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:44.535090+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:05:44.535090+0000","last_peered":"2024-09-10T16:05:44.535090+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:05:44.535090+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"43'5900","ondisk_log_start":"43'5900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20840448,"num_objects":5088,"num_object_clones":0,"num_object_copies":10176,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5088,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12215,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":717}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877349+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:05:19.877349+0000","last_peered":"2024-09-10T16:05:19.877349+0000","last_clean":"2024-09-10T16:05:19.877349+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:05:19.877349+0000","last_undegraded":"2024-09-10T16:05:19.877349+0000","last_fullsized":"2024-09-10T16:05:19.877349+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877229+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:05:19.877229+0000","last_peered":"2024-09-10T16:05:19.877229+0000","last_clean":"2024-09-10T16:05:19.877229+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:05:19.877229+0000","last_undegraded":"2024-09-10T16:05:19.877229+0000","last_fullsized":"2024-09-10T16:05:19.877229+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'15986","reported_seq":16136,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:44.478067+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:05:44.478067+0000","last_peered":"2024-09-10T16:05:44.478067+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:05:44.478067+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"43'5900","ondisk_log_start":"43'5900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21349318,"num_objects":5214,"num_object_clones":0,"num_object_copies":10428,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5214,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12149,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":891},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":336397046,"num_objects":82138,"num_object_clones":0,"num_object_copies":164276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42322,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":82138,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":209968,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9067,"num_bytes_recovered":33095680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":885137408,"data_stored":884995583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160894,"ondisk_log_size":160894,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986269,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":617516,"kb_used_data":301212,"kb_used_omap":4,"kb_used_meta":316283,"kb_avail":93754324,"statfs":{"total":96636764160,"available":96004427776,"internally_reserved":0,"allocated":308441088,"data_stored":308138404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":323874474},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,0,0,1,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659214,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":566932,"kb_used_data":250616,"kb_used_omap":4,"kb_used_meta":316283,"kb_avail":93804908,"statfs":{"total":96636764160,"available":96056225792,"internally_reserved":0,"allocated":256630784,"data_stored":256298732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":323874478},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,3,4,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182497,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":731060,"kb_used_data":316260,"kb_used_omap":4,"kb_used_meta":414779,"kb_avail":93640780,"statfs":{"total":96636764160,"available":95888158720,"internally_reserved":0,"allocated":323850240,"data_stored":323547551,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":424734378},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,2,2,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":322162688,"data_stored":322126558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":255201280,"data_stored":255135787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":307773440,"data_stored":307733238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:05:49.092 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:05:49.649 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:05:49.651 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:05:49.679 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":296,"stamp":"2024-09-10T16:05:47.694345+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":336397065,"num_objects":82140,"num_object_clones":0,"num_object_copies":164280,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42322,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":82140,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":209970,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9069,"num_bytes_recovered":33095699,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160897,"ondisk_log_size":160897,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1915508,"kb_used_data":868088,"kb_used_omap":12,"kb_used_meta":1047347,"kb_avail":281200012,"statfs":{"total":289910292480,"available":287948812288,"internally_reserved":0,"allocated":888922112,"data_stored":887984687,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1072483330},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,2,5,7,7],"upper_bound":128},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18776064,"num_objects":-4584,"num_object_clones":0,"num_object_copies":-9168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-196,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4584,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4584,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":196,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004490"},"pg_stats":[{"pgid":"2.f","version":"47'15608","reported_seq":15688,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:44.470543+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:05:44.470543+0000","last_peered":"2024-09-10T16:05:44.470543+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:05:44.470543+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"43'5600","ondisk_log_start":"43'5600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20742144,"num_objects":5064,"num_object_clones":0,"num_object_copies":10128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5064,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15608,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":805}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'15765","reported_seq":15883,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:45.439477+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:05:45.439477+0000","last_peered":"2024-09-10T16:05:45.439477+0000","last_clean":"2024-09-10T16:05:45.439477+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:05:45.439477+0000","last_undegraded":"2024-09-10T16:05:45.439477+0000","last_fullsized":"2024-09-10T16:05:45.439477+0000","mapping_epoch":35,"log_start":"43'5700","ondisk_log_start":"43'5700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20804550,"num_objects":5081,"num_object_clones":0,"num_object_copies":10162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5081,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13870,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'15617","reported_seq":24300,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:42.550983+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:05:42.550983+0000","last_peered":"2024-09-10T16:05:42.550983+0000","last_clean":"2024-09-10T16:05:42.550983+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:05:42.550983+0000","last_undegraded":"2024-09-10T16:05:42.550983+0000","last_fullsized":"2024-09-10T16:05:42.550983+0000","mapping_epoch":44,"log_start":"43'5600","ondisk_log_start":"43'5600","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21217304,"num_objects":5181,"num_object_clones":0,"num_object_copies":10362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5181,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13621,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'15679","reported_seq":15765,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:45.307160+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:05:45.307160+0000","last_peered":"2024-09-10T16:05:45.307160+0000","last_clean":"2024-09-10T16:05:45.307160+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:05:45.307160+0000","last_undegraded":"2024-09-10T16:05:45.307160+0000","last_fullsized":"2024-09-10T16:05:45.307160+0000","mapping_epoch":35,"log_start":"43'5600","ondisk_log_start":"43'5600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20402176,"num_objects":4981,"num_object_clones":0,"num_object_copies":9962,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4981,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15679,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'15771","reported_seq":15906,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:44.518717+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:05:44.518717+0000","last_peered":"2024-09-10T16:05:44.518717+0000","last_clean":"2024-09-10T16:05:44.518717+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:05:44.518717+0000","last_undegraded":"2024-09-10T16:05:44.518717+0000","last_fullsized":"2024-09-10T16:05:44.518717+0000","mapping_epoch":34,"log_start":"43'5700","ondisk_log_start":"43'5700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20972003,"num_objects":5121,"num_object_clones":0,"num_object_copies":10242,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5121,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12482,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'15940","reported_seq":16083,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:44.532107+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:05:44.532107+0000","last_peered":"2024-09-10T16:05:44.532107+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:05:44.532107+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"43'5900","ondisk_log_start":"43'5900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20976099,"num_objects":5122,"num_object_clones":0,"num_object_copies":10244,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5122,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12552,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":751},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'15787","reported_seq":15928,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:44.451315+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:05:44.451315+0000","last_peered":"2024-09-10T16:05:44.451315+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:05:44.451315+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"43'5700","ondisk_log_start":"43'5700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21689286,"num_objects":5297,"num_object_clones":0,"num_object_copies":10594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5297,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":12286,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1060},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'16087","reported_seq":16222,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:44.577475+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:05:44.577475+0000","last_peered":"2024-09-10T16:05:44.577475+0000","last_clean":"2024-09-10T16:05:44.577475+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:05:44.577475+0000","last_undegraded":"2024-09-10T16:05:44.577475+0000","last_fullsized":"2024-09-10T16:05:44.577475+0000","mapping_epoch":34,"log_start":"43'6000","ondisk_log_start":"43'6000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21409792,"num_objects":5227,"num_object_clones":0,"num_object_copies":10454,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5227,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12934,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.876895+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:05:19.876895+0000","last_peered":"2024-09-10T16:05:19.876895+0000","last_clean":"2024-09-10T16:05:19.876895+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:05:19.876895+0000","last_undegraded":"2024-09-10T16:05:19.876895+0000","last_fullsized":"2024-09-10T16:05:19.876895+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'15347","reported_seq":15491,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:45.399785+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:05:45.399785+0000","last_peered":"2024-09-10T16:05:45.399785+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:05:45.399785+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"43'5300","ondisk_log_start":"43'5300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20931043,"num_objects":5111,"num_object_clones":0,"num_object_copies":10222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5111,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13166,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":988}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":99,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.875776+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:05:19.875776+0000","last_peered":"2024-09-10T16:05:19.875776+0000","last_clean":"2024-09-10T16:05:19.875776+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:05:19.875776+0000","last_undegraded":"2024-09-10T16:05:19.875776+0000","last_fullsized":"2024-09-10T16:05:19.875776+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'15703","reported_seq":15839,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:44.532728+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:05:44.532728+0000","last_peered":"2024-09-10T16:05:44.532728+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:05:44.532728+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"43'5700","ondisk_log_start":"43'5700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21327872,"num_objects":5207,"num_object_clones":0,"num_object_copies":10414,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5207,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12207,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":952}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":109,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.871983+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:05:19.871983+0000","last_peered":"2024-09-10T16:05:19.871983+0000","last_clean":"2024-09-10T16:05:19.871983+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:05:19.871983+0000","last_undegraded":"2024-09-10T16:05:19.871983+0000","last_fullsized":"2024-09-10T16:05:19.871983+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'15720","reported_seq":15860,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:45.444720+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:05:45.444720+0000","last_peered":"2024-09-10T16:05:45.444720+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:05:45.444720+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"43'5700","ondisk_log_start":"43'5700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20730339,"num_objects":5062,"num_object_clones":0,"num_object_copies":10124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5062,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13400,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":758}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877148+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:05:19.877148+0000","last_peered":"2024-09-10T16:05:19.877148+0000","last_clean":"2024-09-10T16:05:19.877148+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:05:19.877148+0000","last_undegraded":"2024-09-10T16:05:19.877148+0000","last_fullsized":"2024-09-10T16:05:19.877148+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'16000","reported_seq":17313,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:45.395020+0000","last_change":"2024-09-10T16:05:18.271152+0000","last_active":"2024-09-10T16:05:45.395020+0000","last_peered":"2024-09-10T16:05:45.395020+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:05:45.395020+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"43'5900","ondisk_log_start":"43'5900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21176320,"num_objects":5170,"num_object_clones":0,"num_object_copies":10340,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3765,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5170,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13554,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":589,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1405}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'15740","reported_seq":15880,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:44.522997+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:05:44.522997+0000","last_peered":"2024-09-10T16:05:44.522997+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:05:44.522997+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"43'5700","ondisk_log_start":"43'5700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20881408,"num_objects":5098,"num_object_clones":0,"num_object_copies":10196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5098,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12371,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":794}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877654+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:05:19.877654+0000","last_peered":"2024-09-10T16:05:19.877654+0000","last_clean":"2024-09-10T16:05:19.877654+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:05:19.877654+0000","last_undegraded":"2024-09-10T16:05:19.877654+0000","last_fullsized":"2024-09-10T16:05:19.877654+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'15484","reported_seq":24055,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:42.707743+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:05:42.707743+0000","last_peered":"2024-09-10T16:05:42.707743+0000","last_clean":"2024-09-10T16:05:42.707743+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:05:42.707743+0000","last_undegraded":"2024-09-10T16:05:42.707743+0000","last_fullsized":"2024-09-10T16:05:42.707743+0000","mapping_epoch":46,"log_start":"43'5400","ondisk_log_start":"43'5400","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20946944,"num_objects":5114,"num_object_clones":0,"num_object_copies":10228,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5114,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11874,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":99,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.876013+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:05:19.876013+0000","last_peered":"2024-09-10T16:05:19.876013+0000","last_clean":"2024-09-10T16:05:19.876013+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:05:19.876013+0000","last_undegraded":"2024-09-10T16:05:19.876013+0000","last_fullsized":"2024-09-10T16:05:19.876013+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'15960","reported_seq":16107,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:44.535090+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:05:44.535090+0000","last_peered":"2024-09-10T16:05:44.535090+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:05:44.535090+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"43'5900","ondisk_log_start":"43'5900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20840448,"num_objects":5088,"num_object_clones":0,"num_object_copies":10176,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5088,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12215,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":717}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877349+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:05:19.877349+0000","last_peered":"2024-09-10T16:05:19.877349+0000","last_clean":"2024-09-10T16:05:19.877349+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:05:19.877349+0000","last_undegraded":"2024-09-10T16:05:19.877349+0000","last_fullsized":"2024-09-10T16:05:19.877349+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877229+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:05:19.877229+0000","last_peered":"2024-09-10T16:05:19.877229+0000","last_clean":"2024-09-10T16:05:19.877229+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:05:19.877229+0000","last_undegraded":"2024-09-10T16:05:19.877229+0000","last_fullsized":"2024-09-10T16:05:19.877229+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'15986","reported_seq":16136,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:44.478067+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:05:44.478067+0000","last_peered":"2024-09-10T16:05:44.478067+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:05:44.478067+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"43'5900","ondisk_log_start":"43'5900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21349318,"num_objects":5214,"num_object_clones":0,"num_object_copies":10428,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5214,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12149,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":891},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":336397046,"num_objects":82138,"num_object_clones":0,"num_object_copies":164276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42322,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":82138,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":209968,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9067,"num_bytes_recovered":33095680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":885137408,"data_stored":884995583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160894,"ondisk_log_size":160894,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986269,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":617516,"kb_used_data":301212,"kb_used_omap":4,"kb_used_meta":316283,"kb_avail":93754324,"statfs":{"total":96636764160,"available":96004427776,"internally_reserved":0,"allocated":308441088,"data_stored":308138404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":323874474},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,0,0,1,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659214,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":566932,"kb_used_data":250616,"kb_used_omap":4,"kb_used_meta":316283,"kb_avail":93804908,"statfs":{"total":96636764160,"available":96056225792,"internally_reserved":0,"allocated":256630784,"data_stored":256298732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":323874478},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,3,4,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182497,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":731060,"kb_used_data":316260,"kb_used_omap":4,"kb_used_meta":414779,"kb_avail":93640780,"statfs":{"total":96636764160,"available":95888158720,"internally_reserved":0,"allocated":323850240,"data_stored":323547551,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":424734378},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,2,2,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":322162688,"data_stored":322126558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":255201280,"data_stored":255135787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":307773440,"data_stored":307733238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:05:49.680 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:05:50.228 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:05:50.229 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:05:50.248 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":297,"stamp":"2024-09-10T16:05:49.694899+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":329982729,"num_objects":80574,"num_object_clones":0,"num_object_copies":161148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42322,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":80574,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":211536,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9069,"num_bytes_recovered":33095699,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160863,"ondisk_log_size":160863,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1910448,"kb_used_data":854708,"kb_used_omap":12,"kb_used_meta":1055667,"kb_avail":281205072,"statfs":{"total":289910292480,"available":287953993728,"internally_reserved":0,"allocated":875220992,"data_stored":874283567,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1081003010},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,3,6,8,8,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-24031232,"num_objects":-5867,"num_object_clones":0,"num_object_copies":-11734,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-196,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5867,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5867,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":196,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004379"},"pg_stats":[{"pgid":"2.f","version":"47'15771","reported_seq":15850,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:49.579939+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:05:49.579939+0000","last_peered":"2024-09-10T16:05:49.579939+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:05:49.579939+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"43'5700","ondisk_log_start":"43'5700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20074496,"num_objects":4901,"num_object_clones":0,"num_object_copies":9802,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4901,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15771,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":642}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'15765","reported_seq":15883,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:45.439477+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:05:45.439477+0000","last_peered":"2024-09-10T16:05:45.439477+0000","last_clean":"2024-09-10T16:05:45.439477+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:05:45.439477+0000","last_undegraded":"2024-09-10T16:05:45.439477+0000","last_fullsized":"2024-09-10T16:05:45.439477+0000","mapping_epoch":35,"log_start":"43'5700","ondisk_log_start":"43'5700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20804550,"num_objects":5081,"num_object_clones":0,"num_object_copies":10162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5081,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13870,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'15741","reported_seq":24424,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:47.665989+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:05:47.665989+0000","last_peered":"2024-09-10T16:05:47.665989+0000","last_clean":"2024-09-10T16:05:47.665989+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:05:47.665989+0000","last_undegraded":"2024-09-10T16:05:47.665989+0000","last_fullsized":"2024-09-10T16:05:47.665989+0000","mapping_epoch":44,"log_start":"43'5700","ondisk_log_start":"43'5700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20709400,"num_objects":5057,"num_object_clones":0,"num_object_copies":10114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5057,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13745,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'15679","reported_seq":15765,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:45.307160+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:05:45.307160+0000","last_peered":"2024-09-10T16:05:45.307160+0000","last_clean":"2024-09-10T16:05:45.307160+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:05:45.307160+0000","last_undegraded":"2024-09-10T16:05:45.307160+0000","last_fullsized":"2024-09-10T16:05:45.307160+0000","mapping_epoch":35,"log_start":"43'5600","ondisk_log_start":"43'5600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20402176,"num_objects":4981,"num_object_clones":0,"num_object_copies":9962,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4981,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15679,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'15918","reported_seq":16053,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:49.524123+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:05:49.524123+0000","last_peered":"2024-09-10T16:05:49.524123+0000","last_clean":"2024-09-10T16:05:49.524123+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:05:49.524123+0000","last_undegraded":"2024-09-10T16:05:49.524123+0000","last_fullsized":"2024-09-10T16:05:49.524123+0000","mapping_epoch":34,"log_start":"43'5900","ondisk_log_start":"43'5900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20369891,"num_objects":4974,"num_object_clones":0,"num_object_copies":9948,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4974,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12629,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'16097","reported_seq":16240,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:49.485074+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:05:49.485074+0000","last_peered":"2024-09-10T16:05:49.485074+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:05:49.485074+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"43'6000","ondisk_log_start":"43'6000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20333027,"num_objects":4965,"num_object_clones":0,"num_object_copies":9930,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4965,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12709,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":594},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'15931","reported_seq":16074,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:49.571863+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:05:49.571863+0000","last_peered":"2024-09-10T16:05:49.571863+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:05:49.571863+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"43'5900","ondisk_log_start":"43'5900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21099462,"num_objects":5153,"num_object_clones":0,"num_object_copies":10306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5153,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":12430,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":916},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'16220","reported_seq":16355,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:49.542093+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:05:49.542093+0000","last_peered":"2024-09-10T16:05:49.542093+0000","last_clean":"2024-09-10T16:05:49.542093+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:05:49.542093+0000","last_undegraded":"2024-09-10T16:05:49.542093+0000","last_fullsized":"2024-09-10T16:05:49.542093+0000","mapping_epoch":34,"log_start":"43'6200","ondisk_log_start":"43'6200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20865024,"num_objects":5094,"num_object_clones":0,"num_object_copies":10188,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5094,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13067,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.876895+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:05:19.876895+0000","last_peered":"2024-09-10T16:05:19.876895+0000","last_clean":"2024-09-10T16:05:19.876895+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:05:19.876895+0000","last_undegraded":"2024-09-10T16:05:19.876895+0000","last_fullsized":"2024-09-10T16:05:19.876895+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'15347","reported_seq":15491,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:45.399785+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:05:45.399785+0000","last_peered":"2024-09-10T16:05:45.399785+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:05:45.399785+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"43'5300","ondisk_log_start":"43'5300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20931043,"num_objects":5111,"num_object_clones":0,"num_object_copies":10222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5111,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13166,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":988}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":99,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.875776+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:05:19.875776+0000","last_peered":"2024-09-10T16:05:19.875776+0000","last_clean":"2024-09-10T16:05:19.875776+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:05:19.875776+0000","last_undegraded":"2024-09-10T16:05:19.875776+0000","last_fullsized":"2024-09-10T16:05:19.875776+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'15848","reported_seq":15984,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:49.563727+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:05:49.563727+0000","last_peered":"2024-09-10T16:05:49.563727+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:05:49.563727+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"43'5800","ondisk_log_start":"43'5800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20733952,"num_objects":5062,"num_object_clones":0,"num_object_copies":10124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5062,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12352,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":807}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":109,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.871983+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:05:19.871983+0000","last_peered":"2024-09-10T16:05:19.871983+0000","last_clean":"2024-09-10T16:05:19.871983+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:05:19.871983+0000","last_undegraded":"2024-09-10T16:05:19.871983+0000","last_fullsized":"2024-09-10T16:05:19.871983+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'15720","reported_seq":15860,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:45.444720+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:05:45.444720+0000","last_peered":"2024-09-10T16:05:45.444720+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:05:45.444720+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"43'5700","ondisk_log_start":"43'5700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20730339,"num_objects":5062,"num_object_clones":0,"num_object_copies":10124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5062,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13400,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":758}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877148+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:05:19.877148+0000","last_peered":"2024-09-10T16:05:19.877148+0000","last_clean":"2024-09-10T16:05:19.877148+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:05:19.877148+0000","last_undegraded":"2024-09-10T16:05:19.877148+0000","last_fullsized":"2024-09-10T16:05:19.877148+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'16000","reported_seq":17313,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:45.395020+0000","last_change":"2024-09-10T16:05:18.271152+0000","last_active":"2024-09-10T16:05:45.395020+0000","last_peered":"2024-09-10T16:05:45.395020+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:05:45.395020+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"43'5900","ondisk_log_start":"43'5900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21176320,"num_objects":5170,"num_object_clones":0,"num_object_copies":10340,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3765,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5170,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13554,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":589,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1405}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'15871","reported_seq":16010,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:49.490886+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:05:49.490886+0000","last_peered":"2024-09-10T16:05:49.490886+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:05:49.490886+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"43'5800","ondisk_log_start":"43'5800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20344832,"num_objects":4967,"num_object_clones":0,"num_object_copies":9934,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4967,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12502,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":663}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877654+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:05:19.877654+0000","last_peered":"2024-09-10T16:05:19.877654+0000","last_clean":"2024-09-10T16:05:19.877654+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:05:19.877654+0000","last_undegraded":"2024-09-10T16:05:19.877654+0000","last_fullsized":"2024-09-10T16:05:19.877654+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'15611","reported_seq":24182,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:47.615147+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:05:47.615147+0000","last_peered":"2024-09-10T16:05:47.615147+0000","last_clean":"2024-09-10T16:05:47.615147+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:05:47.615147+0000","last_undegraded":"2024-09-10T16:05:47.615147+0000","last_fullsized":"2024-09-10T16:05:47.615147+0000","mapping_epoch":46,"log_start":"43'5600","ondisk_log_start":"43'5600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20426752,"num_objects":4987,"num_object_clones":0,"num_object_copies":9974,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4987,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12001,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":99,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.876013+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:05:19.876013+0000","last_peered":"2024-09-10T16:05:19.876013+0000","last_clean":"2024-09-10T16:05:19.876013+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:05:19.876013+0000","last_undegraded":"2024-09-10T16:05:19.876013+0000","last_fullsized":"2024-09-10T16:05:19.876013+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'16100","reported_seq":16247,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:49.529186+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:05:49.529186+0000","last_peered":"2024-09-10T16:05:49.529186+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:05:49.529186+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"43'6000","ondisk_log_start":"43'6000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20267008,"num_objects":4948,"num_object_clones":0,"num_object_copies":9896,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4948,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12355,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":577}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877349+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:05:19.877349+0000","last_peered":"2024-09-10T16:05:19.877349+0000","last_clean":"2024-09-10T16:05:19.877349+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:05:19.877349+0000","last_undegraded":"2024-09-10T16:05:19.877349+0000","last_fullsized":"2024-09-10T16:05:19.877349+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877269+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:05:19.877269+0000","last_peered":"2024-09-10T16:05:19.877269+0000","last_clean":"2024-09-10T16:05:19.877269+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:05:19.877269+0000","last_undegraded":"2024-09-10T16:05:19.877269+0000","last_fullsized":"2024-09-10T16:05:19.877269+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'16141","reported_seq":16286,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:49.588204+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:05:49.588204+0000","last_peered":"2024-09-10T16:05:49.588204+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:05:49.588204+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"43'6100","ondisk_log_start":"43'6100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20714438,"num_objects":5059,"num_object_clones":0,"num_object_copies":10118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5059,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12304,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":736},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":329982710,"num_objects":80572,"num_object_clones":0,"num_object_copies":161144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42322,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":80572,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":211534,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9067,"num_bytes_recovered":33095680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":871178240,"data_stored":871036415,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160860,"ondisk_log_size":160860,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986269,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":617516,"kb_used_data":301212,"kb_used_omap":4,"kb_used_meta":316283,"kb_avail":93754324,"statfs":{"total":96636764160,"available":96004427776,"internally_reserved":0,"allocated":308441088,"data_stored":308138404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":323874474},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,0,0,1,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659215,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":564552,"kb_used_data":244080,"kb_used_omap":4,"kb_used_meta":320443,"kb_avail":93807288,"statfs":{"total":96636764160,"available":96058662912,"internally_reserved":0,"allocated":249937920,"data_stored":249605868,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":328134318},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,2,4,5,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182498,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":728380,"kb_used_data":309416,"kb_used_omap":4,"kb_used_meta":418939,"kb_avail":93643460,"statfs":{"total":96636764160,"available":95890903040,"internally_reserved":0,"allocated":316841984,"data_stored":316539295,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":428994218},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,2,2,5,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":315080704,"data_stored":315044574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":248324096,"data_stored":248258603,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":307773440,"data_stored":307733238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:05:50.249 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-10T16:05:50.249 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "47'15771", 'reported_seq': 15850, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:05:49.579939+0000', 'last_change': '2024-09-10T15:59:16.281558+0000', 'last_active': '2024-09-10T16:05:49.579939+0000', 'last_peered': '2024-09-10T16:05:49.579939+0000', 'last_clean': '2024-09-10T15:59:14.132456+0000', 'last_became_active': '2024-09-10T15:59:16.274407+0000', 'last_became_peered': '2024-09-10T15:59:16.274407+0000', 'last_unstale': '2024-09-10T16:05:49.579939+0000', 'last_undegraded': '2024-09-10T15:59:15.851279+0000', 'last_fullsized': '2024-09-10T15:59:15.774356+0000', 'mapping_epoch': 42, 'log_start': "43'5700", 'ondisk_log_start': "43'5700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'objects_scrubbed': 0, 'log_size': 10071, 'log_dups_size': 0, 'ondisk_log_size': 10071, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20074496, 'num_objects': 4901, 'num_object_clones': 0, 'num_object_copies': 9802, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4259, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4901, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15771, 'num_write_kb': 41344, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 642}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:05:50.249 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-10T16:05:50.249 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "47'16097", 'reported_seq': 16240, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:05:49.485074+0000', 'last_change': '2024-09-10T15:59:16.308264+0000', 'last_active': '2024-09-10T16:05:49.485074+0000', 'last_peered': '2024-09-10T16:05:49.485074+0000', 'last_clean': '2024-09-10T15:59:14.111551+0000', 'last_became_active': '2024-09-10T15:59:16.300040+0000', 'last_became_peered': '2024-09-10T15:59:16.300040+0000', 'last_unstale': '2024-09-10T16:05:49.485074+0000', 'last_undegraded': '2024-09-10T15:59:15.878234+0000', 'last_fullsized': '2024-09-10T15:59:15.769328+0000', 'mapping_epoch': 42, 'log_start': "43'6000", 'ondisk_log_start': "43'6000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3442", 'last_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'objects_scrubbed': 3384, 'log_size': 10097, 'log_dups_size': 0, 'ondisk_log_size': 10097, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1292, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20333027, 'num_objects': 4965, 'num_object_clones': 0, 'num_object_copies': 9930, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4372, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4965, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 12709, 'num_write_kb': 28580, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 594}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:05:50.250 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-10T16:05:50.250 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "47'15931", 'reported_seq': 16074, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:05:49.571863+0000', 'last_change': '2024-09-10T15:59:16.331899+0000', 'last_active': '2024-09-10T16:05:49.571863+0000', 'last_peered': '2024-09-10T16:05:49.571863+0000', 'last_clean': '2024-09-10T15:59:14.139436+0000', 'last_became_active': '2024-09-10T15:59:16.318715+0000', 'last_became_peered': '2024-09-10T15:59:16.318715+0000', 'last_unstale': '2024-09-10T16:05:49.571863+0000', 'last_undegraded': '2024-09-10T15:59:15.877524+0000', 'last_fullsized': '2024-09-10T15:59:15.769042+0000', 'mapping_epoch': 42, 'log_start': "43'5900", 'ondisk_log_start': "43'5900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3537", 'last_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'objects_scrubbed': 3497, 'log_size': 10031, 'log_dups_size': 0, 'ondisk_log_size': 10031, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1224, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21099462, 'num_objects': 5153, 'num_object_clones': 0, 'num_object_copies': 10306, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4237, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5153, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 12430, 'num_write_kb': 28172, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 916}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:05:50.250 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-10T16:05:50.250 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "47'15347", 'reported_seq': 15491, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:05:45.399785+0000', 'last_change': '2024-09-10T15:59:16.175680+0000', 'last_active': '2024-09-10T16:05:45.399785+0000', 'last_peered': '2024-09-10T16:05:45.399785+0000', 'last_clean': '2024-09-10T15:59:14.088187+0000', 'last_became_active': '2024-09-10T15:59:16.165810+0000', 'last_became_peered': '2024-09-10T15:59:16.165810+0000', 'last_unstale': '2024-09-10T16:05:45.399785+0000', 'last_undegraded': '2024-09-10T15:59:15.822098+0000', 'last_fullsized': '2024-09-10T15:59:15.769123+0000', 'mapping_epoch': 42, 'log_start': "43'5300", 'ondisk_log_start': "43'5300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2218", 'last_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'objects_scrubbed': 2175, 'log_size': 10047, 'log_dups_size': 0, 'ondisk_log_size': 10047, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 825, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20931043, 'num_objects': 5111, 'num_object_clones': 0, 'num_object_copies': 10222, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4124, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5111, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 13166, 'num_write_kb': 32204, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': 988}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:05:50.250 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-10T16:05:50.250 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "47'15848", 'reported_seq': 15984, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:05:49.563727+0000', 'last_change': '2024-09-10T15:59:16.277742+0000', 'last_active': '2024-09-10T16:05:49.563727+0000', 'last_peered': '2024-09-10T16:05:49.563727+0000', 'last_clean': '2024-09-10T15:59:14.129081+0000', 'last_became_active': '2024-09-10T15:59:16.269835+0000', 'last_became_peered': '2024-09-10T15:59:16.269835+0000', 'last_unstale': '2024-09-10T16:05:49.563727+0000', 'last_undegraded': '2024-09-10T15:59:15.860857+0000', 'last_fullsized': '2024-09-10T15:59:15.771410+0000', 'mapping_epoch': 42, 'log_start': "43'5800", 'ondisk_log_start': "43'5800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3557", 'last_scrub_stamp': '2024-09-10T15:59:02.493678+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:02.493678+0000', 'objects_scrubbed': 3488, 'log_size': 10048, 'log_dups_size': 0, 'ondisk_log_size': 10048, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1472, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20733952, 'num_objects': 5062, 'num_object_clones': 0, 'num_object_copies': 10124, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4257, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5062, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12352, 'num_write_kb': 27852, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 807}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:05:50.250 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-10T16:05:50.250 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "47'15720", 'reported_seq': 15860, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:05:45.444720+0000', 'last_change': '2024-09-10T15:59:16.284311+0000', 'last_active': '2024-09-10T16:05:45.444720+0000', 'last_peered': '2024-09-10T16:05:45.444720+0000', 'last_clean': '2024-09-10T15:59:14.116041+0000', 'last_became_active': '2024-09-10T15:59:16.274543+0000', 'last_became_peered': '2024-09-10T15:59:16.274543+0000', 'last_unstale': '2024-09-10T16:05:45.444720+0000', 'last_undegraded': '2024-09-10T15:59:15.812484+0000', 'last_fullsized': '2024-09-10T15:59:15.770885+0000', 'mapping_epoch': 42, 'log_start': "43'5700", 'ondisk_log_start': "43'5700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2356", 'last_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'objects_scrubbed': 2316, 'log_size': 10020, 'log_dups_size': 0, 'ondisk_log_size': 10020, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 783, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20730339, 'num_objects': 5062, 'num_object_clones': 0, 'num_object_copies': 10124, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5062, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 13400, 'num_write_kb': 32292, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': 758}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:05:50.250 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-10T16:05:50.250 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "47'16000", 'reported_seq': 17313, 'reported_epoch': 47, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:05:45.395020+0000', 'last_change': '2024-09-10T16:05:18.271152+0000', 'last_active': '2024-09-10T16:05:45.395020+0000', 'last_peered': '2024-09-10T16:05:45.395020+0000', 'last_clean': '2024-09-10T15:59:14.110654+0000', 'last_became_active': '2024-09-10T15:59:16.164645+0000', 'last_became_peered': '2024-09-10T15:59:16.164645+0000', 'last_unstale': '2024-09-10T16:05:45.395020+0000', 'last_undegraded': '2024-09-10T15:59:15.803593+0000', 'last_fullsized': '2024-09-10T15:59:15.769083+0000', 'mapping_epoch': 42, 'log_start': "43'5900", 'ondisk_log_start': "43'5900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2486", 'last_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'objects_scrubbed': 2440, 'log_size': 10100, 'log_dups_size': 0, 'ondisk_log_size': 10100, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 792, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21176320, 'num_objects': 5170, 'num_object_clones': 0, 'num_object_copies': 10340, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3765, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5170, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13554, 'num_write_kb': 32568, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 589, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 1405}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:05:50.251 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-10T16:05:50.251 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "47'15871", 'reported_seq': 16010, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:05:49.490886+0000', 'last_change': '2024-09-10T15:59:16.315641+0000', 'last_active': '2024-09-10T16:05:49.490886+0000', 'last_peered': '2024-09-10T16:05:49.490886+0000', 'last_clean': '2024-09-10T15:59:14.117001+0000', 'last_became_active': '2024-09-10T15:59:16.302245+0000', 'last_became_peered': '2024-09-10T15:59:16.302245+0000', 'last_unstale': '2024-09-10T16:05:49.490886+0000', 'last_undegraded': '2024-09-10T15:59:15.845012+0000', 'last_fullsized': '2024-09-10T15:59:15.770337+0000', 'mapping_epoch': 42, 'log_start': "43'5800", 'ondisk_log_start': "43'5800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3404", 'last_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'objects_scrubbed': 3367, 'log_size': 10071, 'log_dups_size': 0, 'ondisk_log_size': 10071, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 983, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20344832, 'num_objects': 4967, 'num_object_clones': 0, 'num_object_copies': 9934, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4967, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12502, 'num_write_kb': 28204, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 663}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:05:50.251 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-10T16:05:50.251 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "47'16100", 'reported_seq': 16247, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:05:49.529186+0000', 'last_change': '2024-09-10T15:59:16.311291+0000', 'last_active': '2024-09-10T16:05:49.529186+0000', 'last_peered': '2024-09-10T16:05:49.529186+0000', 'last_clean': '2024-09-10T15:59:14.116363+0000', 'last_became_active': '2024-09-10T15:59:16.301244+0000', 'last_became_peered': '2024-09-10T15:59:16.301244+0000', 'last_unstale': '2024-09-10T16:05:49.529186+0000', 'last_undegraded': '2024-09-10T15:59:15.883434+0000', 'last_fullsized': '2024-09-10T15:59:15.772756+0000', 'mapping_epoch': 42, 'log_start': "43'6000", 'ondisk_log_start': "43'6000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3789", 'last_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'objects_scrubbed': 3741, 'log_size': 10100, 'log_dups_size': 0, 'ondisk_log_size': 10100, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1440, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20267008, 'num_objects': 4948, 'num_object_clones': 0, 'num_object_copies': 9896, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4373, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4948, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12355, 'num_write_kb': 27124, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 577}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:05:50.251 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-10T16:05:50.251 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "47'16141", 'reported_seq': 16286, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:05:49.588204+0000', 'last_change': '2024-09-10T15:59:16.313409+0000', 'last_active': '2024-09-10T16:05:49.588204+0000', 'last_peered': '2024-09-10T16:05:49.588204+0000', 'last_clean': '2024-09-10T15:59:14.129780+0000', 'last_became_active': '2024-09-10T15:59:16.300932+0000', 'last_became_peered': '2024-09-10T15:59:16.300932+0000', 'last_unstale': '2024-09-10T16:05:49.588204+0000', 'last_undegraded': '2024-09-10T15:59:15.887823+0000', 'last_fullsized': '2024-09-10T15:59:15.768513+0000', 'mapping_epoch': 42, 'log_start': "43'6100", 'ondisk_log_start': "43'6100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3880", 'last_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'objects_scrubbed': 3827, 'log_size': 10041, 'log_dups_size': 0, 'ondisk_log_size': 10041, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1397, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20714438, 'num_objects': 5059, 'num_object_clones': 0, 'num_object_copies': 10118, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4325, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5059, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12304, 'num_write_kb': 27072, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 736}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:05:50.251 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T16:05:50.935 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:05:50.935 INFO:teuthology.orchestra.run.smithi115.stdout:{"fsid":"7effee29-df05-4351-9741-ee203564ed81","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 42322/161148 objects degraded (26.263%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":514,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":47,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983954,"num_in_osds":3,"osd_in_since":1725983947,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":80574,"data_bytes":329982729,"bytes_used":1956298752,"bytes_avail":287953993728,"bytes_total":289910292480,"degraded_objects":42322,"degraded_total":161148,"degraded_ratio":0.26262814307344801,"recovering_objects_per_sec":16,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":196,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":488},"fsmap":{"epoch":1,"btime":"2024-09-10T15:57:14:967120+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T16:00:13.560776+0000","services":{}},"progress_events":{}} 2024-09-10T16:05:50.962 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-10T16:05:50.962 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:05:51.523 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:05:51.524 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:05:51.554 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":297,"stamp":"2024-09-10T16:05:49.694899+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":329982729,"num_objects":80574,"num_object_clones":0,"num_object_copies":161148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42322,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":80574,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":211536,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9069,"num_bytes_recovered":33095699,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160863,"ondisk_log_size":160863,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1910448,"kb_used_data":854708,"kb_used_omap":12,"kb_used_meta":1055667,"kb_avail":281205072,"statfs":{"total":289910292480,"available":287953993728,"internally_reserved":0,"allocated":875220992,"data_stored":874283567,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1081003010},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,3,6,8,8,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-24031232,"num_objects":-5867,"num_object_clones":0,"num_object_copies":-11734,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-196,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5867,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5867,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":196,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004379"},"pg_stats":[{"pgid":"2.f","version":"47'15771","reported_seq":15850,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:49.579939+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:05:49.579939+0000","last_peered":"2024-09-10T16:05:49.579939+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:05:49.579939+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"43'5700","ondisk_log_start":"43'5700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20074496,"num_objects":4901,"num_object_clones":0,"num_object_copies":9802,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4901,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15771,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":642}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'15765","reported_seq":15883,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:45.439477+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:05:45.439477+0000","last_peered":"2024-09-10T16:05:45.439477+0000","last_clean":"2024-09-10T16:05:45.439477+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:05:45.439477+0000","last_undegraded":"2024-09-10T16:05:45.439477+0000","last_fullsized":"2024-09-10T16:05:45.439477+0000","mapping_epoch":35,"log_start":"43'5700","ondisk_log_start":"43'5700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20804550,"num_objects":5081,"num_object_clones":0,"num_object_copies":10162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5081,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13870,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'15741","reported_seq":24424,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:47.665989+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:05:47.665989+0000","last_peered":"2024-09-10T16:05:47.665989+0000","last_clean":"2024-09-10T16:05:47.665989+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:05:47.665989+0000","last_undegraded":"2024-09-10T16:05:47.665989+0000","last_fullsized":"2024-09-10T16:05:47.665989+0000","mapping_epoch":44,"log_start":"43'5700","ondisk_log_start":"43'5700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20709400,"num_objects":5057,"num_object_clones":0,"num_object_copies":10114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5057,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13745,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'15679","reported_seq":15765,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:45.307160+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:05:45.307160+0000","last_peered":"2024-09-10T16:05:45.307160+0000","last_clean":"2024-09-10T16:05:45.307160+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:05:45.307160+0000","last_undegraded":"2024-09-10T16:05:45.307160+0000","last_fullsized":"2024-09-10T16:05:45.307160+0000","mapping_epoch":35,"log_start":"43'5600","ondisk_log_start":"43'5600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20402176,"num_objects":4981,"num_object_clones":0,"num_object_copies":9962,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4981,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15679,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'15918","reported_seq":16053,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:49.524123+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:05:49.524123+0000","last_peered":"2024-09-10T16:05:49.524123+0000","last_clean":"2024-09-10T16:05:49.524123+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:05:49.524123+0000","last_undegraded":"2024-09-10T16:05:49.524123+0000","last_fullsized":"2024-09-10T16:05:49.524123+0000","mapping_epoch":34,"log_start":"43'5900","ondisk_log_start":"43'5900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20369891,"num_objects":4974,"num_object_clones":0,"num_object_copies":9948,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4974,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12629,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'16097","reported_seq":16240,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:49.485074+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:05:49.485074+0000","last_peered":"2024-09-10T16:05:49.485074+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:05:49.485074+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"43'6000","ondisk_log_start":"43'6000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20333027,"num_objects":4965,"num_object_clones":0,"num_object_copies":9930,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4965,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12709,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":594},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'15931","reported_seq":16074,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:49.571863+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:05:49.571863+0000","last_peered":"2024-09-10T16:05:49.571863+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:05:49.571863+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"43'5900","ondisk_log_start":"43'5900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21099462,"num_objects":5153,"num_object_clones":0,"num_object_copies":10306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5153,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":12430,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":916},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'16220","reported_seq":16355,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:49.542093+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:05:49.542093+0000","last_peered":"2024-09-10T16:05:49.542093+0000","last_clean":"2024-09-10T16:05:49.542093+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:05:49.542093+0000","last_undegraded":"2024-09-10T16:05:49.542093+0000","last_fullsized":"2024-09-10T16:05:49.542093+0000","mapping_epoch":34,"log_start":"43'6200","ondisk_log_start":"43'6200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20865024,"num_objects":5094,"num_object_clones":0,"num_object_copies":10188,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5094,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13067,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.876895+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:05:19.876895+0000","last_peered":"2024-09-10T16:05:19.876895+0000","last_clean":"2024-09-10T16:05:19.876895+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:05:19.876895+0000","last_undegraded":"2024-09-10T16:05:19.876895+0000","last_fullsized":"2024-09-10T16:05:19.876895+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'15347","reported_seq":15491,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:45.399785+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:05:45.399785+0000","last_peered":"2024-09-10T16:05:45.399785+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:05:45.399785+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"43'5300","ondisk_log_start":"43'5300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20931043,"num_objects":5111,"num_object_clones":0,"num_object_copies":10222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5111,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13166,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":988}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":99,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.875776+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:05:19.875776+0000","last_peered":"2024-09-10T16:05:19.875776+0000","last_clean":"2024-09-10T16:05:19.875776+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:05:19.875776+0000","last_undegraded":"2024-09-10T16:05:19.875776+0000","last_fullsized":"2024-09-10T16:05:19.875776+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'15848","reported_seq":15984,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:49.563727+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:05:49.563727+0000","last_peered":"2024-09-10T16:05:49.563727+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:05:49.563727+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"43'5800","ondisk_log_start":"43'5800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20733952,"num_objects":5062,"num_object_clones":0,"num_object_copies":10124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5062,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12352,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":807}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":109,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.871983+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:05:19.871983+0000","last_peered":"2024-09-10T16:05:19.871983+0000","last_clean":"2024-09-10T16:05:19.871983+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:05:19.871983+0000","last_undegraded":"2024-09-10T16:05:19.871983+0000","last_fullsized":"2024-09-10T16:05:19.871983+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'15720","reported_seq":15860,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:45.444720+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:05:45.444720+0000","last_peered":"2024-09-10T16:05:45.444720+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:05:45.444720+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"43'5700","ondisk_log_start":"43'5700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20730339,"num_objects":5062,"num_object_clones":0,"num_object_copies":10124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5062,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13400,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":758}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877148+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:05:19.877148+0000","last_peered":"2024-09-10T16:05:19.877148+0000","last_clean":"2024-09-10T16:05:19.877148+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:05:19.877148+0000","last_undegraded":"2024-09-10T16:05:19.877148+0000","last_fullsized":"2024-09-10T16:05:19.877148+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'16000","reported_seq":17313,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:45.395020+0000","last_change":"2024-09-10T16:05:18.271152+0000","last_active":"2024-09-10T16:05:45.395020+0000","last_peered":"2024-09-10T16:05:45.395020+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:05:45.395020+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"43'5900","ondisk_log_start":"43'5900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21176320,"num_objects":5170,"num_object_clones":0,"num_object_copies":10340,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3765,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5170,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13554,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":589,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1405}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'15871","reported_seq":16010,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:49.490886+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:05:49.490886+0000","last_peered":"2024-09-10T16:05:49.490886+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:05:49.490886+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"43'5800","ondisk_log_start":"43'5800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20344832,"num_objects":4967,"num_object_clones":0,"num_object_copies":9934,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4967,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12502,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":663}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877654+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:05:19.877654+0000","last_peered":"2024-09-10T16:05:19.877654+0000","last_clean":"2024-09-10T16:05:19.877654+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:05:19.877654+0000","last_undegraded":"2024-09-10T16:05:19.877654+0000","last_fullsized":"2024-09-10T16:05:19.877654+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'15611","reported_seq":24182,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:47.615147+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:05:47.615147+0000","last_peered":"2024-09-10T16:05:47.615147+0000","last_clean":"2024-09-10T16:05:47.615147+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:05:47.615147+0000","last_undegraded":"2024-09-10T16:05:47.615147+0000","last_fullsized":"2024-09-10T16:05:47.615147+0000","mapping_epoch":46,"log_start":"43'5600","ondisk_log_start":"43'5600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20426752,"num_objects":4987,"num_object_clones":0,"num_object_copies":9974,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4987,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12001,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":99,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.876013+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:05:19.876013+0000","last_peered":"2024-09-10T16:05:19.876013+0000","last_clean":"2024-09-10T16:05:19.876013+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:05:19.876013+0000","last_undegraded":"2024-09-10T16:05:19.876013+0000","last_fullsized":"2024-09-10T16:05:19.876013+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'16100","reported_seq":16247,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:49.529186+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:05:49.529186+0000","last_peered":"2024-09-10T16:05:49.529186+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:05:49.529186+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"43'6000","ondisk_log_start":"43'6000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20267008,"num_objects":4948,"num_object_clones":0,"num_object_copies":9896,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4948,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12355,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":577}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877349+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:05:19.877349+0000","last_peered":"2024-09-10T16:05:19.877349+0000","last_clean":"2024-09-10T16:05:19.877349+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:05:19.877349+0000","last_undegraded":"2024-09-10T16:05:19.877349+0000","last_fullsized":"2024-09-10T16:05:19.877349+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877269+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:05:19.877269+0000","last_peered":"2024-09-10T16:05:19.877269+0000","last_clean":"2024-09-10T16:05:19.877269+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:05:19.877269+0000","last_undegraded":"2024-09-10T16:05:19.877269+0000","last_fullsized":"2024-09-10T16:05:19.877269+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'16141","reported_seq":16286,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:49.588204+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:05:49.588204+0000","last_peered":"2024-09-10T16:05:49.588204+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:05:49.588204+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"43'6100","ondisk_log_start":"43'6100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20714438,"num_objects":5059,"num_object_clones":0,"num_object_copies":10118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5059,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12304,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":736},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":329982710,"num_objects":80572,"num_object_clones":0,"num_object_copies":161144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42322,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":80572,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":211534,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9067,"num_bytes_recovered":33095680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":871178240,"data_stored":871036415,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160860,"ondisk_log_size":160860,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986269,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":617516,"kb_used_data":301212,"kb_used_omap":4,"kb_used_meta":316283,"kb_avail":93754324,"statfs":{"total":96636764160,"available":96004427776,"internally_reserved":0,"allocated":308441088,"data_stored":308138404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":323874474},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,0,0,1,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659215,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":564552,"kb_used_data":244080,"kb_used_omap":4,"kb_used_meta":320443,"kb_avail":93807288,"statfs":{"total":96636764160,"available":96058662912,"internally_reserved":0,"allocated":249937920,"data_stored":249605868,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":328134318},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,2,4,5,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182498,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":728380,"kb_used_data":309416,"kb_used_omap":4,"kb_used_meta":418939,"kb_avail":93643460,"statfs":{"total":96636764160,"available":95890903040,"internally_reserved":0,"allocated":316841984,"data_stored":316539295,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":428994218},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,2,2,5,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":315080704,"data_stored":315044574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":248324096,"data_stored":248258603,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":307773440,"data_stored":307733238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:05:54.557 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:05:55.108 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:05:55.110 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:05:55.133 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":299,"stamp":"2024-09-10T16:05:53.696264+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":325927689,"num_objects":79584,"num_object_clones":0,"num_object_copies":159168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42215,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":79584,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":212526,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9176,"num_bytes_recovered":33095699,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160853,"ondisk_log_size":160853,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1911588,"kb_used_data":843372,"kb_used_omap":12,"kb_used_meta":1068147,"kb_avail":281203932,"statfs":{"total":289910292480,"available":287952826368,"internally_reserved":0,"allocated":863612928,"data_stored":862675503,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1093782530},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,4,9,10,7,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19873792,"num_objects":-4852,"num_object_clones":0,"num_object_copies":-9704,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-207,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4852,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4852,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":207,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004228"},"pg_stats":[{"pgid":"2.f","version":"47'15771","reported_seq":15850,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:49.579939+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:05:49.579939+0000","last_peered":"2024-09-10T16:05:49.579939+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:05:49.579939+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"43'5700","ondisk_log_start":"43'5700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20074496,"num_objects":4901,"num_object_clones":0,"num_object_copies":9802,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4901,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15771,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":642}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'15908","reported_seq":16026,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:50.437646+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:05:50.437646+0000","last_peered":"2024-09-10T16:05:50.437646+0000","last_clean":"2024-09-10T16:05:50.437646+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:05:50.437646+0000","last_undegraded":"2024-09-10T16:05:50.437646+0000","last_fullsized":"2024-09-10T16:05:50.437646+0000","mapping_epoch":35,"log_start":"43'5900","ondisk_log_start":"43'5900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20218822,"num_objects":4938,"num_object_clones":0,"num_object_copies":9876,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4938,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":14013,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'15880","reported_seq":24563,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:52.640106+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:05:52.640106+0000","last_peered":"2024-09-10T16:05:52.640106+0000","last_clean":"2024-09-10T16:05:52.640106+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:05:52.640106+0000","last_undegraded":"2024-09-10T16:05:52.640106+0000","last_fullsized":"2024-09-10T16:05:52.640106+0000","mapping_epoch":44,"log_start":"43'5800","ondisk_log_start":"43'5800","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20140056,"num_objects":4918,"num_object_clones":0,"num_object_copies":9836,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4918,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13884,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'15810","reported_seq":15896,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:50.426645+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:05:50.426645+0000","last_peered":"2024-09-10T16:05:50.426645+0000","last_clean":"2024-09-10T16:05:50.426645+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:05:50.426645+0000","last_undegraded":"2024-09-10T16:05:50.426645+0000","last_fullsized":"2024-09-10T16:05:50.426645+0000","mapping_epoch":35,"log_start":"43'5800","ondisk_log_start":"43'5800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19865600,"num_objects":4850,"num_object_clones":0,"num_object_copies":9700,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4850,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15810,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'15918","reported_seq":16053,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:49.524123+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:05:49.524123+0000","last_peered":"2024-09-10T16:05:49.524123+0000","last_clean":"2024-09-10T16:05:49.524123+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:05:49.524123+0000","last_undegraded":"2024-09-10T16:05:49.524123+0000","last_fullsized":"2024-09-10T16:05:49.524123+0000","mapping_epoch":34,"log_start":"43'5900","ondisk_log_start":"43'5900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20369891,"num_objects":4974,"num_object_clones":0,"num_object_copies":9948,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4974,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12629,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'16097","reported_seq":16240,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:49.485074+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:05:49.485074+0000","last_peered":"2024-09-10T16:05:49.485074+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:05:49.485074+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"43'6000","ondisk_log_start":"43'6000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20333027,"num_objects":4965,"num_object_clones":0,"num_object_copies":9930,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4965,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12709,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":594},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'15931","reported_seq":16074,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:49.571863+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:05:49.571863+0000","last_peered":"2024-09-10T16:05:49.571863+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:05:49.571863+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"43'5900","ondisk_log_start":"43'5900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21099462,"num_objects":5153,"num_object_clones":0,"num_object_copies":10306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5153,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":12430,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":916},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'16220","reported_seq":16355,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:49.542093+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:05:49.542093+0000","last_peered":"2024-09-10T16:05:49.542093+0000","last_clean":"2024-09-10T16:05:49.542093+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:05:49.542093+0000","last_undegraded":"2024-09-10T16:05:49.542093+0000","last_fullsized":"2024-09-10T16:05:49.542093+0000","mapping_epoch":34,"log_start":"43'6200","ondisk_log_start":"43'6200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20865024,"num_objects":5094,"num_object_clones":0,"num_object_copies":10188,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5094,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13067,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.876895+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:05:19.876895+0000","last_peered":"2024-09-10T16:05:19.876895+0000","last_clean":"2024-09-10T16:05:19.876895+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:05:19.876895+0000","last_undegraded":"2024-09-10T16:05:19.876895+0000","last_fullsized":"2024-09-10T16:05:19.876895+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'15493","reported_seq":15637,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:50.476859+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:05:50.476859+0000","last_peered":"2024-09-10T16:05:50.476859+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:05:50.476859+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"43'5400","ondisk_log_start":"43'5400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20333027,"num_objects":4965,"num_object_clones":0,"num_object_copies":9930,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4965,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13312,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":842}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":99,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.875776+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:05:19.875776+0000","last_peered":"2024-09-10T16:05:19.875776+0000","last_clean":"2024-09-10T16:05:19.875776+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:05:19.875776+0000","last_undegraded":"2024-09-10T16:05:19.875776+0000","last_fullsized":"2024-09-10T16:05:19.875776+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'15848","reported_seq":15984,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:49.563727+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:05:49.563727+0000","last_peered":"2024-09-10T16:05:49.563727+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:05:49.563727+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"43'5800","ondisk_log_start":"43'5800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20733952,"num_objects":5062,"num_object_clones":0,"num_object_copies":10124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5062,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12352,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":807}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":109,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.871983+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:05:19.871983+0000","last_peered":"2024-09-10T16:05:19.871983+0000","last_clean":"2024-09-10T16:05:19.871983+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:05:19.871983+0000","last_undegraded":"2024-09-10T16:05:19.871983+0000","last_fullsized":"2024-09-10T16:05:19.871983+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'15853","reported_seq":15993,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:50.407900+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:05:50.407900+0000","last_peered":"2024-09-10T16:05:50.407900+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:05:50.407900+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"43'5800","ondisk_log_start":"43'5800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20185571,"num_objects":4929,"num_object_clones":0,"num_object_copies":9858,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4929,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13533,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":625}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877148+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:05:19.877148+0000","last_peered":"2024-09-10T16:05:19.877148+0000","last_clean":"2024-09-10T16:05:19.877148+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:05:19.877148+0000","last_undegraded":"2024-09-10T16:05:19.877148+0000","last_fullsized":"2024-09-10T16:05:19.877148+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'16149","reported_seq":17676,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:50.460641+0000","last_change":"2024-09-10T16:05:18.271152+0000","last_active":"2024-09-10T16:05:50.460641+0000","last_peered":"2024-09-10T16:05:50.460641+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:05:50.460641+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"43'6100","ondisk_log_start":"43'6100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20566016,"num_objects":5021,"num_object_clones":0,"num_object_copies":10042,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3658,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5021,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13703,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":696,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1363}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'15871","reported_seq":16010,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:49.490886+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:05:49.490886+0000","last_peered":"2024-09-10T16:05:49.490886+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:05:49.490886+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"43'5800","ondisk_log_start":"43'5800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20344832,"num_objects":4967,"num_object_clones":0,"num_object_copies":9934,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4967,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12502,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":663}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877654+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:05:19.877654+0000","last_peered":"2024-09-10T16:05:19.877654+0000","last_clean":"2024-09-10T16:05:19.877654+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:05:19.877654+0000","last_undegraded":"2024-09-10T16:05:19.877654+0000","last_fullsized":"2024-09-10T16:05:19.877654+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'15760","reported_seq":24331,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:52.725983+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:05:52.725983+0000","last_peered":"2024-09-10T16:05:52.725983+0000","last_clean":"2024-09-10T16:05:52.725983+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:05:52.725983+0000","last_undegraded":"2024-09-10T16:05:52.725983+0000","last_fullsized":"2024-09-10T16:05:52.725983+0000","mapping_epoch":46,"log_start":"43'5700","ondisk_log_start":"43'5700","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19816448,"num_objects":4838,"num_object_clones":0,"num_object_copies":9676,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4838,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12150,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":99,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.876013+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:05:19.876013+0000","last_peered":"2024-09-10T16:05:19.876013+0000","last_clean":"2024-09-10T16:05:19.876013+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:05:19.876013+0000","last_undegraded":"2024-09-10T16:05:19.876013+0000","last_fullsized":"2024-09-10T16:05:19.876013+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'16100","reported_seq":16247,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:49.529186+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:05:49.529186+0000","last_peered":"2024-09-10T16:05:49.529186+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:05:49.529186+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"43'6000","ondisk_log_start":"43'6000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20267008,"num_objects":4948,"num_object_clones":0,"num_object_copies":9896,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4948,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12355,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":577}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877349+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:05:19.877349+0000","last_peered":"2024-09-10T16:05:19.877349+0000","last_clean":"2024-09-10T16:05:19.877349+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:05:19.877349+0000","last_undegraded":"2024-09-10T16:05:19.877349+0000","last_fullsized":"2024-09-10T16:05:19.877349+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877269+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:05:19.877269+0000","last_peered":"2024-09-10T16:05:19.877269+0000","last_clean":"2024-09-10T16:05:19.877269+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:05:19.877269+0000","last_undegraded":"2024-09-10T16:05:19.877269+0000","last_fullsized":"2024-09-10T16:05:19.877269+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'16141","reported_seq":16286,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:49.588204+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:05:49.588204+0000","last_peered":"2024-09-10T16:05:49.588204+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:05:49.588204+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"43'6100","ondisk_log_start":"43'6100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20714438,"num_objects":5059,"num_object_clones":0,"num_object_copies":10118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5059,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12304,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":736},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":325927670,"num_objects":79582,"num_object_clones":0,"num_object_copies":159164,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42215,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":79582,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":212524,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9174,"num_bytes_recovered":33095680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":859459584,"data_stored":859317759,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160850,"ondisk_log_size":160850,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986270,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":617220,"kb_used_data":296752,"kb_used_omap":4,"kb_used_meta":320443,"kb_avail":93754620,"statfs":{"total":96636764160,"available":96004730880,"internally_reserved":0,"allocated":303874048,"data_stored":303571364,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":328134314},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,4,5],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659216,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":565988,"kb_used_data":237204,"kb_used_omap":4,"kb_used_meta":328763,"kb_avail":93805852,"statfs":{"total":96636764160,"available":96057192448,"internally_reserved":0,"allocated":242896896,"data_stored":242564844,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":336653998},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,3,3,3,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182498,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":728380,"kb_used_data":309416,"kb_used_omap":4,"kb_used_meta":418939,"kb_avail":93643460,"statfs":{"total":96636764160,"available":95890903040,"internally_reserved":0,"allocated":316841984,"data_stored":316539295,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":428994218},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,2,2,5,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":315080704,"data_stored":315044574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":241229824,"data_stored":241164331,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":303149056,"data_stored":303108854,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:05:55.135 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:05:55.685 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:05:55.686 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:05:55.708 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":299,"stamp":"2024-09-10T16:05:53.696264+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":325927689,"num_objects":79584,"num_object_clones":0,"num_object_copies":159168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42215,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":79584,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":212526,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9176,"num_bytes_recovered":33095699,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160853,"ondisk_log_size":160853,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1911588,"kb_used_data":843372,"kb_used_omap":12,"kb_used_meta":1068147,"kb_avail":281203932,"statfs":{"total":289910292480,"available":287952826368,"internally_reserved":0,"allocated":863612928,"data_stored":862675503,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1093782530},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,4,9,10,7,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19873792,"num_objects":-4852,"num_object_clones":0,"num_object_copies":-9704,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-207,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4852,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4852,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":207,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004228"},"pg_stats":[{"pgid":"2.f","version":"47'15771","reported_seq":15850,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:49.579939+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:05:49.579939+0000","last_peered":"2024-09-10T16:05:49.579939+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:05:49.579939+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"43'5700","ondisk_log_start":"43'5700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20074496,"num_objects":4901,"num_object_clones":0,"num_object_copies":9802,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4901,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15771,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":642}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'15908","reported_seq":16026,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:50.437646+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:05:50.437646+0000","last_peered":"2024-09-10T16:05:50.437646+0000","last_clean":"2024-09-10T16:05:50.437646+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:05:50.437646+0000","last_undegraded":"2024-09-10T16:05:50.437646+0000","last_fullsized":"2024-09-10T16:05:50.437646+0000","mapping_epoch":35,"log_start":"43'5900","ondisk_log_start":"43'5900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20218822,"num_objects":4938,"num_object_clones":0,"num_object_copies":9876,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4938,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":14013,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'15880","reported_seq":24563,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:52.640106+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:05:52.640106+0000","last_peered":"2024-09-10T16:05:52.640106+0000","last_clean":"2024-09-10T16:05:52.640106+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:05:52.640106+0000","last_undegraded":"2024-09-10T16:05:52.640106+0000","last_fullsized":"2024-09-10T16:05:52.640106+0000","mapping_epoch":44,"log_start":"43'5800","ondisk_log_start":"43'5800","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20140056,"num_objects":4918,"num_object_clones":0,"num_object_copies":9836,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4918,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13884,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'15810","reported_seq":15896,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:50.426645+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:05:50.426645+0000","last_peered":"2024-09-10T16:05:50.426645+0000","last_clean":"2024-09-10T16:05:50.426645+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:05:50.426645+0000","last_undegraded":"2024-09-10T16:05:50.426645+0000","last_fullsized":"2024-09-10T16:05:50.426645+0000","mapping_epoch":35,"log_start":"43'5800","ondisk_log_start":"43'5800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19865600,"num_objects":4850,"num_object_clones":0,"num_object_copies":9700,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4850,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15810,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'15918","reported_seq":16053,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:49.524123+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:05:49.524123+0000","last_peered":"2024-09-10T16:05:49.524123+0000","last_clean":"2024-09-10T16:05:49.524123+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:05:49.524123+0000","last_undegraded":"2024-09-10T16:05:49.524123+0000","last_fullsized":"2024-09-10T16:05:49.524123+0000","mapping_epoch":34,"log_start":"43'5900","ondisk_log_start":"43'5900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20369891,"num_objects":4974,"num_object_clones":0,"num_object_copies":9948,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4974,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12629,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'16097","reported_seq":16240,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:49.485074+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:05:49.485074+0000","last_peered":"2024-09-10T16:05:49.485074+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:05:49.485074+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"43'6000","ondisk_log_start":"43'6000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20333027,"num_objects":4965,"num_object_clones":0,"num_object_copies":9930,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4965,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12709,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":594},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'15931","reported_seq":16074,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:49.571863+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:05:49.571863+0000","last_peered":"2024-09-10T16:05:49.571863+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:05:49.571863+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"43'5900","ondisk_log_start":"43'5900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21099462,"num_objects":5153,"num_object_clones":0,"num_object_copies":10306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5153,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":12430,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":916},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'16220","reported_seq":16355,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:49.542093+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:05:49.542093+0000","last_peered":"2024-09-10T16:05:49.542093+0000","last_clean":"2024-09-10T16:05:49.542093+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:05:49.542093+0000","last_undegraded":"2024-09-10T16:05:49.542093+0000","last_fullsized":"2024-09-10T16:05:49.542093+0000","mapping_epoch":34,"log_start":"43'6200","ondisk_log_start":"43'6200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20865024,"num_objects":5094,"num_object_clones":0,"num_object_copies":10188,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5094,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13067,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.876895+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:05:19.876895+0000","last_peered":"2024-09-10T16:05:19.876895+0000","last_clean":"2024-09-10T16:05:19.876895+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:05:19.876895+0000","last_undegraded":"2024-09-10T16:05:19.876895+0000","last_fullsized":"2024-09-10T16:05:19.876895+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'15493","reported_seq":15637,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:50.476859+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:05:50.476859+0000","last_peered":"2024-09-10T16:05:50.476859+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:05:50.476859+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"43'5400","ondisk_log_start":"43'5400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20333027,"num_objects":4965,"num_object_clones":0,"num_object_copies":9930,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4965,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13312,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":842}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":99,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.875776+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:05:19.875776+0000","last_peered":"2024-09-10T16:05:19.875776+0000","last_clean":"2024-09-10T16:05:19.875776+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:05:19.875776+0000","last_undegraded":"2024-09-10T16:05:19.875776+0000","last_fullsized":"2024-09-10T16:05:19.875776+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'15848","reported_seq":15984,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:49.563727+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:05:49.563727+0000","last_peered":"2024-09-10T16:05:49.563727+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:05:49.563727+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"43'5800","ondisk_log_start":"43'5800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20733952,"num_objects":5062,"num_object_clones":0,"num_object_copies":10124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5062,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12352,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":807}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":109,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.871983+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:05:19.871983+0000","last_peered":"2024-09-10T16:05:19.871983+0000","last_clean":"2024-09-10T16:05:19.871983+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:05:19.871983+0000","last_undegraded":"2024-09-10T16:05:19.871983+0000","last_fullsized":"2024-09-10T16:05:19.871983+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'15853","reported_seq":15993,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:50.407900+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:05:50.407900+0000","last_peered":"2024-09-10T16:05:50.407900+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:05:50.407900+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"43'5800","ondisk_log_start":"43'5800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20185571,"num_objects":4929,"num_object_clones":0,"num_object_copies":9858,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4929,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13533,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":625}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877148+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:05:19.877148+0000","last_peered":"2024-09-10T16:05:19.877148+0000","last_clean":"2024-09-10T16:05:19.877148+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:05:19.877148+0000","last_undegraded":"2024-09-10T16:05:19.877148+0000","last_fullsized":"2024-09-10T16:05:19.877148+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'16149","reported_seq":17676,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:50.460641+0000","last_change":"2024-09-10T16:05:18.271152+0000","last_active":"2024-09-10T16:05:50.460641+0000","last_peered":"2024-09-10T16:05:50.460641+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:05:50.460641+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"43'6100","ondisk_log_start":"43'6100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20566016,"num_objects":5021,"num_object_clones":0,"num_object_copies":10042,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3658,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5021,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13703,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":696,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1363}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'15871","reported_seq":16010,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:49.490886+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:05:49.490886+0000","last_peered":"2024-09-10T16:05:49.490886+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:05:49.490886+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"43'5800","ondisk_log_start":"43'5800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20344832,"num_objects":4967,"num_object_clones":0,"num_object_copies":9934,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4967,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12502,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":663}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877654+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:05:19.877654+0000","last_peered":"2024-09-10T16:05:19.877654+0000","last_clean":"2024-09-10T16:05:19.877654+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:05:19.877654+0000","last_undegraded":"2024-09-10T16:05:19.877654+0000","last_fullsized":"2024-09-10T16:05:19.877654+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'15760","reported_seq":24331,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:52.725983+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:05:52.725983+0000","last_peered":"2024-09-10T16:05:52.725983+0000","last_clean":"2024-09-10T16:05:52.725983+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:05:52.725983+0000","last_undegraded":"2024-09-10T16:05:52.725983+0000","last_fullsized":"2024-09-10T16:05:52.725983+0000","mapping_epoch":46,"log_start":"43'5700","ondisk_log_start":"43'5700","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19816448,"num_objects":4838,"num_object_clones":0,"num_object_copies":9676,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4838,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12150,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":99,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.876013+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:05:19.876013+0000","last_peered":"2024-09-10T16:05:19.876013+0000","last_clean":"2024-09-10T16:05:19.876013+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:05:19.876013+0000","last_undegraded":"2024-09-10T16:05:19.876013+0000","last_fullsized":"2024-09-10T16:05:19.876013+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'16100","reported_seq":16247,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:49.529186+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:05:49.529186+0000","last_peered":"2024-09-10T16:05:49.529186+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:05:49.529186+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"43'6000","ondisk_log_start":"43'6000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20267008,"num_objects":4948,"num_object_clones":0,"num_object_copies":9896,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4948,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12355,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":577}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877349+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:05:19.877349+0000","last_peered":"2024-09-10T16:05:19.877349+0000","last_clean":"2024-09-10T16:05:19.877349+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:05:19.877349+0000","last_undegraded":"2024-09-10T16:05:19.877349+0000","last_fullsized":"2024-09-10T16:05:19.877349+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877269+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:05:19.877269+0000","last_peered":"2024-09-10T16:05:19.877269+0000","last_clean":"2024-09-10T16:05:19.877269+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:05:19.877269+0000","last_undegraded":"2024-09-10T16:05:19.877269+0000","last_fullsized":"2024-09-10T16:05:19.877269+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'16141","reported_seq":16286,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:49.588204+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:05:49.588204+0000","last_peered":"2024-09-10T16:05:49.588204+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:05:49.588204+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"43'6100","ondisk_log_start":"43'6100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20714438,"num_objects":5059,"num_object_clones":0,"num_object_copies":10118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5059,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12304,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":736},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":325927670,"num_objects":79582,"num_object_clones":0,"num_object_copies":159164,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42215,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":79582,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":212524,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9174,"num_bytes_recovered":33095680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":859459584,"data_stored":859317759,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160850,"ondisk_log_size":160850,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986270,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":617220,"kb_used_data":296752,"kb_used_omap":4,"kb_used_meta":320443,"kb_avail":93754620,"statfs":{"total":96636764160,"available":96004730880,"internally_reserved":0,"allocated":303874048,"data_stored":303571364,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":328134314},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,4,5],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659216,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":565988,"kb_used_data":237204,"kb_used_omap":4,"kb_used_meta":328763,"kb_avail":93805852,"statfs":{"total":96636764160,"available":96057192448,"internally_reserved":0,"allocated":242896896,"data_stored":242564844,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":336653998},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,3,3,3,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182498,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":728380,"kb_used_data":309416,"kb_used_omap":4,"kb_used_meta":418939,"kb_avail":93643460,"statfs":{"total":96636764160,"available":95890903040,"internally_reserved":0,"allocated":316841984,"data_stored":316539295,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":428994218},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,2,2,5,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":315080704,"data_stored":315044574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":241229824,"data_stored":241164331,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":303149056,"data_stored":303108854,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:05:55.710 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-10T16:05:55.710 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "47'15771", 'reported_seq': 15850, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:05:49.579939+0000', 'last_change': '2024-09-10T15:59:16.281558+0000', 'last_active': '2024-09-10T16:05:49.579939+0000', 'last_peered': '2024-09-10T16:05:49.579939+0000', 'last_clean': '2024-09-10T15:59:14.132456+0000', 'last_became_active': '2024-09-10T15:59:16.274407+0000', 'last_became_peered': '2024-09-10T15:59:16.274407+0000', 'last_unstale': '2024-09-10T16:05:49.579939+0000', 'last_undegraded': '2024-09-10T15:59:15.851279+0000', 'last_fullsized': '2024-09-10T15:59:15.774356+0000', 'mapping_epoch': 42, 'log_start': "43'5700", 'ondisk_log_start': "43'5700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'objects_scrubbed': 0, 'log_size': 10071, 'log_dups_size': 0, 'ondisk_log_size': 10071, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20074496, 'num_objects': 4901, 'num_object_clones': 0, 'num_object_copies': 9802, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4259, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4901, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15771, 'num_write_kb': 41344, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 642}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:05:55.710 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-10T16:05:55.710 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "47'16097", 'reported_seq': 16240, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:05:49.485074+0000', 'last_change': '2024-09-10T15:59:16.308264+0000', 'last_active': '2024-09-10T16:05:49.485074+0000', 'last_peered': '2024-09-10T16:05:49.485074+0000', 'last_clean': '2024-09-10T15:59:14.111551+0000', 'last_became_active': '2024-09-10T15:59:16.300040+0000', 'last_became_peered': '2024-09-10T15:59:16.300040+0000', 'last_unstale': '2024-09-10T16:05:49.485074+0000', 'last_undegraded': '2024-09-10T15:59:15.878234+0000', 'last_fullsized': '2024-09-10T15:59:15.769328+0000', 'mapping_epoch': 42, 'log_start': "43'6000", 'ondisk_log_start': "43'6000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3442", 'last_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'objects_scrubbed': 3384, 'log_size': 10097, 'log_dups_size': 0, 'ondisk_log_size': 10097, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1292, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20333027, 'num_objects': 4965, 'num_object_clones': 0, 'num_object_copies': 9930, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4372, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4965, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 12709, 'num_write_kb': 28580, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 594}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:05:55.710 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-10T16:05:55.710 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "47'15931", 'reported_seq': 16074, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:05:49.571863+0000', 'last_change': '2024-09-10T15:59:16.331899+0000', 'last_active': '2024-09-10T16:05:49.571863+0000', 'last_peered': '2024-09-10T16:05:49.571863+0000', 'last_clean': '2024-09-10T15:59:14.139436+0000', 'last_became_active': '2024-09-10T15:59:16.318715+0000', 'last_became_peered': '2024-09-10T15:59:16.318715+0000', 'last_unstale': '2024-09-10T16:05:49.571863+0000', 'last_undegraded': '2024-09-10T15:59:15.877524+0000', 'last_fullsized': '2024-09-10T15:59:15.769042+0000', 'mapping_epoch': 42, 'log_start': "43'5900", 'ondisk_log_start': "43'5900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3537", 'last_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'objects_scrubbed': 3497, 'log_size': 10031, 'log_dups_size': 0, 'ondisk_log_size': 10031, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1224, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21099462, 'num_objects': 5153, 'num_object_clones': 0, 'num_object_copies': 10306, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4237, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5153, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 12430, 'num_write_kb': 28172, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 916}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:05:55.710 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-10T16:05:55.711 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "47'15493", 'reported_seq': 15637, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:05:50.476859+0000', 'last_change': '2024-09-10T15:59:16.175680+0000', 'last_active': '2024-09-10T16:05:50.476859+0000', 'last_peered': '2024-09-10T16:05:50.476859+0000', 'last_clean': '2024-09-10T15:59:14.088187+0000', 'last_became_active': '2024-09-10T15:59:16.165810+0000', 'last_became_peered': '2024-09-10T15:59:16.165810+0000', 'last_unstale': '2024-09-10T16:05:50.476859+0000', 'last_undegraded': '2024-09-10T15:59:15.822098+0000', 'last_fullsized': '2024-09-10T15:59:15.769123+0000', 'mapping_epoch': 42, 'log_start': "43'5400", 'ondisk_log_start': "43'5400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2218", 'last_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'objects_scrubbed': 2175, 'log_size': 10093, 'log_dups_size': 0, 'ondisk_log_size': 10093, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 825, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20333027, 'num_objects': 4965, 'num_object_clones': 0, 'num_object_copies': 9930, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4124, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4965, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 13312, 'num_write_kb': 32204, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': 842}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:05:55.711 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-10T16:05:55.711 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "47'15848", 'reported_seq': 15984, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:05:49.563727+0000', 'last_change': '2024-09-10T15:59:16.277742+0000', 'last_active': '2024-09-10T16:05:49.563727+0000', 'last_peered': '2024-09-10T16:05:49.563727+0000', 'last_clean': '2024-09-10T15:59:14.129081+0000', 'last_became_active': '2024-09-10T15:59:16.269835+0000', 'last_became_peered': '2024-09-10T15:59:16.269835+0000', 'last_unstale': '2024-09-10T16:05:49.563727+0000', 'last_undegraded': '2024-09-10T15:59:15.860857+0000', 'last_fullsized': '2024-09-10T15:59:15.771410+0000', 'mapping_epoch': 42, 'log_start': "43'5800", 'ondisk_log_start': "43'5800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3557", 'last_scrub_stamp': '2024-09-10T15:59:02.493678+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:02.493678+0000', 'objects_scrubbed': 3488, 'log_size': 10048, 'log_dups_size': 0, 'ondisk_log_size': 10048, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1472, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20733952, 'num_objects': 5062, 'num_object_clones': 0, 'num_object_copies': 10124, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4257, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5062, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12352, 'num_write_kb': 27852, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 807}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:05:55.711 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-10T16:05:55.711 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "47'15853", 'reported_seq': 15993, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:05:50.407900+0000', 'last_change': '2024-09-10T15:59:16.284311+0000', 'last_active': '2024-09-10T16:05:50.407900+0000', 'last_peered': '2024-09-10T16:05:50.407900+0000', 'last_clean': '2024-09-10T15:59:14.116041+0000', 'last_became_active': '2024-09-10T15:59:16.274543+0000', 'last_became_peered': '2024-09-10T15:59:16.274543+0000', 'last_unstale': '2024-09-10T16:05:50.407900+0000', 'last_undegraded': '2024-09-10T15:59:15.812484+0000', 'last_fullsized': '2024-09-10T15:59:15.770885+0000', 'mapping_epoch': 42, 'log_start': "43'5800", 'ondisk_log_start': "43'5800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2356", 'last_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'objects_scrubbed': 2316, 'log_size': 10053, 'log_dups_size': 0, 'ondisk_log_size': 10053, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 783, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20185571, 'num_objects': 4929, 'num_object_clones': 0, 'num_object_copies': 9858, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4929, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 13533, 'num_write_kb': 32292, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': 625}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:05:55.711 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-10T16:05:55.711 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "47'16149", 'reported_seq': 17676, 'reported_epoch': 47, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:05:50.460641+0000', 'last_change': '2024-09-10T16:05:18.271152+0000', 'last_active': '2024-09-10T16:05:50.460641+0000', 'last_peered': '2024-09-10T16:05:50.460641+0000', 'last_clean': '2024-09-10T15:59:14.110654+0000', 'last_became_active': '2024-09-10T15:59:16.164645+0000', 'last_became_peered': '2024-09-10T15:59:16.164645+0000', 'last_unstale': '2024-09-10T16:05:50.460641+0000', 'last_undegraded': '2024-09-10T15:59:15.803593+0000', 'last_fullsized': '2024-09-10T15:59:15.769083+0000', 'mapping_epoch': 42, 'log_start': "43'6100", 'ondisk_log_start': "43'6100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2486", 'last_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'objects_scrubbed': 2440, 'log_size': 10049, 'log_dups_size': 0, 'ondisk_log_size': 10049, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 792, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20566016, 'num_objects': 5021, 'num_object_clones': 0, 'num_object_copies': 10042, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3658, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5021, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13703, 'num_write_kb': 32568, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 696, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 1363}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:05:55.711 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-10T16:05:55.711 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "47'15871", 'reported_seq': 16010, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:05:49.490886+0000', 'last_change': '2024-09-10T15:59:16.315641+0000', 'last_active': '2024-09-10T16:05:49.490886+0000', 'last_peered': '2024-09-10T16:05:49.490886+0000', 'last_clean': '2024-09-10T15:59:14.117001+0000', 'last_became_active': '2024-09-10T15:59:16.302245+0000', 'last_became_peered': '2024-09-10T15:59:16.302245+0000', 'last_unstale': '2024-09-10T16:05:49.490886+0000', 'last_undegraded': '2024-09-10T15:59:15.845012+0000', 'last_fullsized': '2024-09-10T15:59:15.770337+0000', 'mapping_epoch': 42, 'log_start': "43'5800", 'ondisk_log_start': "43'5800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3404", 'last_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'objects_scrubbed': 3367, 'log_size': 10071, 'log_dups_size': 0, 'ondisk_log_size': 10071, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 983, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20344832, 'num_objects': 4967, 'num_object_clones': 0, 'num_object_copies': 9934, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4967, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12502, 'num_write_kb': 28204, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 663}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:05:55.711 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-10T16:05:55.712 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "47'16100", 'reported_seq': 16247, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:05:49.529186+0000', 'last_change': '2024-09-10T15:59:16.311291+0000', 'last_active': '2024-09-10T16:05:49.529186+0000', 'last_peered': '2024-09-10T16:05:49.529186+0000', 'last_clean': '2024-09-10T15:59:14.116363+0000', 'last_became_active': '2024-09-10T15:59:16.301244+0000', 'last_became_peered': '2024-09-10T15:59:16.301244+0000', 'last_unstale': '2024-09-10T16:05:49.529186+0000', 'last_undegraded': '2024-09-10T15:59:15.883434+0000', 'last_fullsized': '2024-09-10T15:59:15.772756+0000', 'mapping_epoch': 42, 'log_start': "43'6000", 'ondisk_log_start': "43'6000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3789", 'last_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'objects_scrubbed': 3741, 'log_size': 10100, 'log_dups_size': 0, 'ondisk_log_size': 10100, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1440, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20267008, 'num_objects': 4948, 'num_object_clones': 0, 'num_object_copies': 9896, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4373, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4948, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12355, 'num_write_kb': 27124, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 577}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:05:55.712 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-10T16:05:55.712 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "47'16141", 'reported_seq': 16286, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:05:49.588204+0000', 'last_change': '2024-09-10T15:59:16.313409+0000', 'last_active': '2024-09-10T16:05:49.588204+0000', 'last_peered': '2024-09-10T16:05:49.588204+0000', 'last_clean': '2024-09-10T15:59:14.129780+0000', 'last_became_active': '2024-09-10T15:59:16.300932+0000', 'last_became_peered': '2024-09-10T15:59:16.300932+0000', 'last_unstale': '2024-09-10T16:05:49.588204+0000', 'last_undegraded': '2024-09-10T15:59:15.887823+0000', 'last_fullsized': '2024-09-10T15:59:15.768513+0000', 'mapping_epoch': 42, 'log_start': "43'6100", 'ondisk_log_start': "43'6100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3880", 'last_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'objects_scrubbed': 3827, 'log_size': 10041, 'log_dups_size': 0, 'ondisk_log_size': 10041, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1397, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20714438, 'num_objects': 5059, 'num_object_clones': 0, 'num_object_copies': 10118, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4325, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5059, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12304, 'num_write_kb': 27072, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 736}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:05:55.712 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T16:05:56.464 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:05:56.464 INFO:teuthology.orchestra.run.smithi115.stdout:{"fsid":"7effee29-df05-4351-9741-ee203564ed81","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 42110/155076 objects degraded (27.154%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":520,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":47,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983954,"num_in_osds":3,"osd_in_since":1725983947,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":77538,"data_bytes":317547273,"bytes_used":1959813120,"bytes_avail":287950479360,"bytes_total":289910292480,"degraded_objects":42110,"degraded_total":155076,"degraded_ratio":0.27154427506512935,"recovering_objects_per_sec":25,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":312,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":551},"fsmap":{"epoch":1,"btime":"2024-09-10T15:57:14:967120+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T16:00:13.560776+0000","services":{}},"progress_events":{}} 2024-09-10T16:05:56.487 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-10T16:05:56.487 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:05:57.025 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:05:57.028 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:05:57.051 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":300,"stamp":"2024-09-10T16:05:55.697066+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":317547273,"num_objects":77538,"num_object_clones":0,"num_object_copies":155076,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42110,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":77538,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":214572,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9281,"num_bytes_recovered":33095699,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160999,"ondisk_log_size":160999,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1913880,"kb_used_data":831768,"kb_used_omap":12,"kb_used_meta":1082035,"kb_avail":281201640,"statfs":{"total":289910292480,"available":287950479360,"internally_reserved":0,"allocated":851730432,"data_stored":850793007,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1108003842},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,2,4,9,6,9],"upper_bound":128},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-27115520,"num_objects":-6620,"num_object_clones":0,"num_object_copies":-13240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-312,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-6620,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6620,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":312,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004197"},"pg_stats":[{"pgid":"2.f","version":"47'15895","reported_seq":15975,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:54.605269+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:05:54.605269+0000","last_peered":"2024-09-10T16:05:54.605269+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:05:54.605269+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"43'5800","ondisk_log_start":"43'5800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19566592,"num_objects":4777,"num_object_clones":0,"num_object_copies":9554,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4777,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15895,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":518}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'16029","reported_seq":16147,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:55.480200+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:05:55.480200+0000","last_peered":"2024-09-10T16:05:55.480200+0000","last_clean":"2024-09-10T16:05:55.480200+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:05:55.480200+0000","last_undegraded":"2024-09-10T16:05:55.480200+0000","last_fullsized":"2024-09-10T16:05:55.480200+0000","mapping_epoch":35,"log_start":"43'6000","ondisk_log_start":"43'6000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19723206,"num_objects":4817,"num_object_clones":0,"num_object_copies":9634,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4817,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":14134,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'15880","reported_seq":24563,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:52.640106+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:05:52.640106+0000","last_peered":"2024-09-10T16:05:52.640106+0000","last_clean":"2024-09-10T16:05:52.640106+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:05:52.640106+0000","last_undegraded":"2024-09-10T16:05:52.640106+0000","last_fullsized":"2024-09-10T16:05:52.640106+0000","mapping_epoch":44,"log_start":"43'5800","ondisk_log_start":"43'5800","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20140056,"num_objects":4918,"num_object_clones":0,"num_object_copies":9836,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4918,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13884,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'15954","reported_seq":16040,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:55.479778+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:05:55.479778+0000","last_peered":"2024-09-10T16:05:55.479778+0000","last_clean":"2024-09-10T16:05:55.479778+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:05:55.479778+0000","last_undegraded":"2024-09-10T16:05:55.479778+0000","last_fullsized":"2024-09-10T16:05:55.479778+0000","mapping_epoch":35,"log_start":"43'5900","ondisk_log_start":"43'5900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19275776,"num_objects":4706,"num_object_clones":0,"num_object_copies":9412,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4706,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15954,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'16069","reported_seq":16203,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:54.695589+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:05:54.695589+0000","last_peered":"2024-09-10T16:05:54.695589+0000","last_clean":"2024-09-10T16:05:54.695589+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:05:54.695589+0000","last_undegraded":"2024-09-10T16:05:54.695589+0000","last_fullsized":"2024-09-10T16:05:54.695589+0000","mapping_epoch":34,"log_start":"43'6000","ondisk_log_start":"43'6000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19751395,"num_objects":4823,"num_object_clones":0,"num_object_copies":9646,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4823,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12780,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'16225","reported_seq":16368,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:54.640637+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:05:54.640637+0000","last_peered":"2024-09-10T16:05:54.640637+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:05:54.640637+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"43'6200","ondisk_log_start":"43'6200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19808739,"num_objects":4837,"num_object_clones":0,"num_object_copies":9674,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4837,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12837,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":466},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'16073","reported_seq":16216,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:54.620304+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:05:54.620304+0000","last_peered":"2024-09-10T16:05:54.620304+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:05:54.620304+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"43'6000","ondisk_log_start":"43'6000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20517830,"num_objects":5011,"num_object_clones":0,"num_object_copies":10022,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5011,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":12572,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":774},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'16387","reported_seq":16522,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:54.629366+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:05:54.629366+0000","last_peered":"2024-09-10T16:05:54.629366+0000","last_clean":"2024-09-10T16:05:54.629366+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:05:54.629366+0000","last_undegraded":"2024-09-10T16:05:54.629366+0000","last_fullsized":"2024-09-10T16:05:54.629366+0000","mapping_epoch":34,"log_start":"43'6300","ondisk_log_start":"43'6300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20180992,"num_objects":4927,"num_object_clones":0,"num_object_copies":9854,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4927,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13234,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.876895+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:05:19.876895+0000","last_peered":"2024-09-10T16:05:19.876895+0000","last_clean":"2024-09-10T16:05:19.876895+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:05:19.876895+0000","last_undegraded":"2024-09-10T16:05:19.876895+0000","last_fullsized":"2024-09-10T16:05:19.876895+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'15649","reported_seq":15794,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:55.356326+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:05:55.356326+0000","last_peered":"2024-09-10T16:05:55.356326+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:05:55.356326+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"43'5600","ondisk_log_start":"43'5600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19694051,"num_objects":4809,"num_object_clones":0,"num_object_copies":9618,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4809,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13468,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":686}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":99,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.875776+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:05:19.875776+0000","last_peered":"2024-09-10T16:05:19.875776+0000","last_clean":"2024-09-10T16:05:19.875776+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:05:19.875776+0000","last_undegraded":"2024-09-10T16:05:19.875776+0000","last_fullsized":"2024-09-10T16:05:19.875776+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'15992","reported_seq":16128,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:54.569221+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:05:54.569221+0000","last_peered":"2024-09-10T16:05:54.569221+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:05:54.569221+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"43'5900","ondisk_log_start":"43'5900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20144128,"num_objects":4918,"num_object_clones":0,"num_object_copies":9836,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4918,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12496,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":663}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":109,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.871983+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:05:19.871983+0000","last_peered":"2024-09-10T16:05:19.871983+0000","last_clean":"2024-09-10T16:05:19.871983+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:05:19.871983+0000","last_undegraded":"2024-09-10T16:05:19.871983+0000","last_fullsized":"2024-09-10T16:05:19.871983+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'16014","reported_seq":16154,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:55.408984+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:05:55.408984+0000","last_peered":"2024-09-10T16:05:55.408984+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:05:55.408984+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"43'6000","ondisk_log_start":"43'6000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19526115,"num_objects":4768,"num_object_clones":0,"num_object_copies":9536,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4768,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13694,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":464}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877148+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:05:19.877148+0000","last_peered":"2024-09-10T16:05:19.877148+0000","last_clean":"2024-09-10T16:05:19.877148+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:05:19.877148+0000","last_undegraded":"2024-09-10T16:05:19.877148+0000","last_fullsized":"2024-09-10T16:05:19.877148+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'16285","reported_seq":18022,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:55.471260+0000","last_change":"2024-09-10T16:05:18.271152+0000","last_active":"2024-09-10T16:05:55.471260+0000","last_peered":"2024-09-10T16:05:55.471260+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:05:55.471260+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"43'6200","ondisk_log_start":"43'6200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20008960,"num_objects":4885,"num_object_clones":0,"num_object_copies":9770,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3553,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4885,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13839,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":801,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1332}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'16027","reported_seq":16164,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:54.676806+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:05:54.676806+0000","last_peered":"2024-09-10T16:05:54.676806+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:05:54.676806+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"43'6000","ondisk_log_start":"43'6000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19705856,"num_objects":4811,"num_object_clones":0,"num_object_copies":9622,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4811,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12658,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":507}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877654+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:05:19.877654+0000","last_peered":"2024-09-10T16:05:19.877654+0000","last_clean":"2024-09-10T16:05:19.877654+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:05:19.877654+0000","last_undegraded":"2024-09-10T16:05:19.877654+0000","last_fullsized":"2024-09-10T16:05:19.877654+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'15760","reported_seq":24331,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:52.725983+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:05:52.725983+0000","last_peered":"2024-09-10T16:05:52.725983+0000","last_clean":"2024-09-10T16:05:52.725983+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:05:52.725983+0000","last_undegraded":"2024-09-10T16:05:52.725983+0000","last_fullsized":"2024-09-10T16:05:52.725983+0000","mapping_epoch":46,"log_start":"43'5700","ondisk_log_start":"43'5700","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19816448,"num_objects":4838,"num_object_clones":0,"num_object_copies":9676,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4838,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12150,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":99,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.876013+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:05:19.876013+0000","last_peered":"2024-09-10T16:05:19.876013+0000","last_clean":"2024-09-10T16:05:19.876013+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:05:19.876013+0000","last_undegraded":"2024-09-10T16:05:19.876013+0000","last_fullsized":"2024-09-10T16:05:19.876013+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'16262","reported_seq":16409,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:54.676515+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:05:54.676515+0000","last_peered":"2024-09-10T16:05:54.676515+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:05:54.676515+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"43'6200","ondisk_log_start":"43'6200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19603456,"num_objects":4786,"num_object_clones":0,"num_object_copies":9572,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4786,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12517,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":415}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877349+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:05:19.877349+0000","last_peered":"2024-09-10T16:05:19.877349+0000","last_clean":"2024-09-10T16:05:19.877349+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:05:19.877349+0000","last_undegraded":"2024-09-10T16:05:19.877349+0000","last_fullsized":"2024-09-10T16:05:19.877349+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877269+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:05:19.877269+0000","last_peered":"2024-09-10T16:05:19.877269+0000","last_clean":"2024-09-10T16:05:19.877269+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:05:19.877269+0000","last_undegraded":"2024-09-10T16:05:19.877269+0000","last_fullsized":"2024-09-10T16:05:19.877269+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'16295","reported_seq":16445,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:54.619479+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:05:54.619479+0000","last_peered":"2024-09-10T16:05:54.619479+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:05:54.619479+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"43'6200","ondisk_log_start":"43'6200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20083654,"num_objects":4905,"num_object_clones":0,"num_object_copies":9810,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4905,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12458,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":582},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":317547254,"num_objects":77536,"num_object_clones":0,"num_object_copies":155072,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42110,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":77536,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":214570,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9279,"num_bytes_recovered":33095680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":847536128,"data_stored":847394303,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160996,"ondisk_log_size":160996,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986271,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":616716,"kb_used_data":292088,"kb_used_omap":4,"kb_used_meta":324603,"kb_avail":93755124,"statfs":{"total":96636764160,"available":96005246976,"internally_reserved":0,"allocated":299098112,"data_stored":298795428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":332394154},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,0,2,0,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659216,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":565988,"kb_used_data":237204,"kb_used_omap":4,"kb_used_meta":328763,"kb_avail":93805852,"statfs":{"total":96636764160,"available":96057192448,"internally_reserved":0,"allocated":242896896,"data_stored":242564844,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":336653998},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,3,3,3,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182499,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":731176,"kb_used_data":302476,"kb_used_omap":4,"kb_used_meta":428667,"kb_avail":93640664,"statfs":{"total":96636764160,"available":95888039936,"internally_reserved":0,"allocated":309735424,"data_stored":309432735,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":438955690},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,1,4,3,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":307941376,"data_stored":307905246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":241229824,"data_stored":241164331,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":298364928,"data_stored":298324726,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:06:00.053 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:06:00.627 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:06:00.628 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:06:00.654 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":302,"stamp":"2024-09-10T16:05:59.698342+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":316429065,"num_objects":77265,"num_object_clones":0,"num_object_copies":154530,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42110,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":77265,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":214845,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9281,"num_bytes_recovered":33095699,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160972,"ondisk_log_size":160972,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1912660,"kb_used_data":824980,"kb_used_omap":12,"kb_used_meta":1087603,"kb_avail":281202860,"statfs":{"total":289910292480,"available":287951728640,"internally_reserved":0,"allocated":844779520,"data_stored":843842095,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1113705474},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,0,3,11,7,9,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19968000,"num_objects":-4875,"num_object_clones":0,"num_object_copies":-9750,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-212,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4875,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4875,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":212,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003996"},"pg_stats":[{"pgid":"2.f","version":"47'15895","reported_seq":15975,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:54.605269+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:05:54.605269+0000","last_peered":"2024-09-10T16:05:54.605269+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:05:54.605269+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"43'5800","ondisk_log_start":"43'5800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19566592,"num_objects":4777,"num_object_clones":0,"num_object_copies":9554,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4777,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15895,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":518}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'16029","reported_seq":16147,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:55.480200+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:05:55.480200+0000","last_peered":"2024-09-10T16:05:55.480200+0000","last_clean":"2024-09-10T16:05:55.480200+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:05:55.480200+0000","last_undegraded":"2024-09-10T16:05:55.480200+0000","last_fullsized":"2024-09-10T16:05:55.480200+0000","mapping_epoch":35,"log_start":"43'6000","ondisk_log_start":"43'6000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19723206,"num_objects":4817,"num_object_clones":0,"num_object_copies":9634,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4817,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":14134,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'16019","reported_seq":24702,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:57.812958+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:05:57.812958+0000","last_peered":"2024-09-10T16:05:57.812958+0000","last_clean":"2024-09-10T16:05:57.812958+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:05:57.812958+0000","last_undegraded":"2024-09-10T16:05:57.812958+0000","last_fullsized":"2024-09-10T16:05:57.812958+0000","mapping_epoch":44,"log_start":"43'6000","ondisk_log_start":"43'6000","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19570712,"num_objects":4779,"num_object_clones":0,"num_object_copies":9558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4779,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14023,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'15954","reported_seq":16040,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:55.479778+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:05:55.479778+0000","last_peered":"2024-09-10T16:05:55.479778+0000","last_clean":"2024-09-10T16:05:55.479778+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:05:55.479778+0000","last_undegraded":"2024-09-10T16:05:55.479778+0000","last_fullsized":"2024-09-10T16:05:55.479778+0000","mapping_epoch":35,"log_start":"43'5900","ondisk_log_start":"43'5900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19275776,"num_objects":4706,"num_object_clones":0,"num_object_copies":9412,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4706,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15954,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'16069","reported_seq":16203,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:54.695589+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:05:54.695589+0000","last_peered":"2024-09-10T16:05:54.695589+0000","last_clean":"2024-09-10T16:05:54.695589+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:05:54.695589+0000","last_undegraded":"2024-09-10T16:05:54.695589+0000","last_fullsized":"2024-09-10T16:05:54.695589+0000","mapping_epoch":34,"log_start":"43'6000","ondisk_log_start":"43'6000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19751395,"num_objects":4823,"num_object_clones":0,"num_object_copies":9646,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4823,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12780,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'16225","reported_seq":16368,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:54.640637+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:05:54.640637+0000","last_peered":"2024-09-10T16:05:54.640637+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:05:54.640637+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"43'6200","ondisk_log_start":"43'6200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19808739,"num_objects":4837,"num_object_clones":0,"num_object_copies":9674,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4837,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12837,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":466},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'16073","reported_seq":16216,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:54.620304+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:05:54.620304+0000","last_peered":"2024-09-10T16:05:54.620304+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:05:54.620304+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"43'6000","ondisk_log_start":"43'6000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20517830,"num_objects":5011,"num_object_clones":0,"num_object_copies":10022,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5011,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":12572,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":774},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'16387","reported_seq":16522,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:54.629366+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:05:54.629366+0000","last_peered":"2024-09-10T16:05:54.629366+0000","last_clean":"2024-09-10T16:05:54.629366+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:05:54.629366+0000","last_undegraded":"2024-09-10T16:05:54.629366+0000","last_fullsized":"2024-09-10T16:05:54.629366+0000","mapping_epoch":34,"log_start":"43'6300","ondisk_log_start":"43'6300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20180992,"num_objects":4927,"num_object_clones":0,"num_object_copies":9854,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4927,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13234,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.876895+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:05:19.876895+0000","last_peered":"2024-09-10T16:05:19.876895+0000","last_clean":"2024-09-10T16:05:19.876895+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:05:19.876895+0000","last_undegraded":"2024-09-10T16:05:19.876895+0000","last_fullsized":"2024-09-10T16:05:19.876895+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'15649","reported_seq":15794,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:55.356326+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:05:55.356326+0000","last_peered":"2024-09-10T16:05:55.356326+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:05:55.356326+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"43'5600","ondisk_log_start":"43'5600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19694051,"num_objects":4809,"num_object_clones":0,"num_object_copies":9618,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4809,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13468,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":686}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":99,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.875776+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:05:19.875776+0000","last_peered":"2024-09-10T16:05:19.875776+0000","last_clean":"2024-09-10T16:05:19.875776+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:05:19.875776+0000","last_undegraded":"2024-09-10T16:05:19.875776+0000","last_fullsized":"2024-09-10T16:05:19.875776+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'15992","reported_seq":16128,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:54.569221+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:05:54.569221+0000","last_peered":"2024-09-10T16:05:54.569221+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:05:54.569221+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"43'5900","ondisk_log_start":"43'5900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20144128,"num_objects":4918,"num_object_clones":0,"num_object_copies":9836,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4918,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12496,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":663}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":109,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.871983+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:05:19.871983+0000","last_peered":"2024-09-10T16:05:19.871983+0000","last_clean":"2024-09-10T16:05:19.871983+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:05:19.871983+0000","last_undegraded":"2024-09-10T16:05:19.871983+0000","last_fullsized":"2024-09-10T16:05:19.871983+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'16014","reported_seq":16154,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:55.408984+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:05:55.408984+0000","last_peered":"2024-09-10T16:05:55.408984+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:05:55.408984+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"43'6000","ondisk_log_start":"43'6000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19526115,"num_objects":4768,"num_object_clones":0,"num_object_copies":9536,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4768,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13694,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":464}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877148+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:05:19.877148+0000","last_peered":"2024-09-10T16:05:19.877148+0000","last_clean":"2024-09-10T16:05:19.877148+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:05:19.877148+0000","last_undegraded":"2024-09-10T16:05:19.877148+0000","last_fullsized":"2024-09-10T16:05:19.877148+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'16285","reported_seq":18022,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:55.471260+0000","last_change":"2024-09-10T16:05:18.271152+0000","last_active":"2024-09-10T16:05:55.471260+0000","last_peered":"2024-09-10T16:05:55.471260+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:05:55.471260+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"43'6200","ondisk_log_start":"43'6200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20008960,"num_objects":4885,"num_object_clones":0,"num_object_copies":9770,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3553,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4885,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13839,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":801,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1332}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'16027","reported_seq":16164,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:54.676806+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:05:54.676806+0000","last_peered":"2024-09-10T16:05:54.676806+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:05:54.676806+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"43'6000","ondisk_log_start":"43'6000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19705856,"num_objects":4811,"num_object_clones":0,"num_object_copies":9622,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4811,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12658,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":507}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877654+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:05:19.877654+0000","last_peered":"2024-09-10T16:05:19.877654+0000","last_clean":"2024-09-10T16:05:19.877654+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:05:19.877654+0000","last_undegraded":"2024-09-10T16:05:19.877654+0000","last_fullsized":"2024-09-10T16:05:19.877654+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'15894","reported_seq":24465,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:57.656239+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:05:57.656239+0000","last_peered":"2024-09-10T16:05:57.656239+0000","last_clean":"2024-09-10T16:05:57.656239+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:05:57.656239+0000","last_undegraded":"2024-09-10T16:05:57.656239+0000","last_fullsized":"2024-09-10T16:05:57.656239+0000","mapping_epoch":46,"log_start":"43'5800","ondisk_log_start":"43'5800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19267584,"num_objects":4704,"num_object_clones":0,"num_object_copies":9408,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4704,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12284,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":99,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.876013+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:05:19.876013+0000","last_peered":"2024-09-10T16:05:19.876013+0000","last_clean":"2024-09-10T16:05:19.876013+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:05:19.876013+0000","last_undegraded":"2024-09-10T16:05:19.876013+0000","last_fullsized":"2024-09-10T16:05:19.876013+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'16262","reported_seq":16409,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:54.676515+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:05:54.676515+0000","last_peered":"2024-09-10T16:05:54.676515+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:05:54.676515+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"43'6200","ondisk_log_start":"43'6200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19603456,"num_objects":4786,"num_object_clones":0,"num_object_copies":9572,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4786,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12517,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":415}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877349+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:05:19.877349+0000","last_peered":"2024-09-10T16:05:19.877349+0000","last_clean":"2024-09-10T16:05:19.877349+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:05:19.877349+0000","last_undegraded":"2024-09-10T16:05:19.877349+0000","last_fullsized":"2024-09-10T16:05:19.877349+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877269+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:05:19.877269+0000","last_peered":"2024-09-10T16:05:19.877269+0000","last_clean":"2024-09-10T16:05:19.877269+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:05:19.877269+0000","last_undegraded":"2024-09-10T16:05:19.877269+0000","last_fullsized":"2024-09-10T16:05:19.877269+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'16295","reported_seq":16445,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:54.619479+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:05:54.619479+0000","last_peered":"2024-09-10T16:05:54.619479+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:05:54.619479+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"43'6200","ondisk_log_start":"43'6200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20083654,"num_objects":4905,"num_object_clones":0,"num_object_copies":9810,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4905,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12458,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":582},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":316429046,"num_objects":77263,"num_object_clones":0,"num_object_copies":154526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42110,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":77263,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":214843,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9279,"num_bytes_recovered":33095680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":840552448,"data_stored":840410623,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160969,"ondisk_log_size":160969,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986271,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":616716,"kb_used_data":292088,"kb_used_omap":4,"kb_used_meta":324603,"kb_avail":93755124,"statfs":{"total":96636764160,"available":96005246976,"internally_reserved":0,"allocated":299098112,"data_stored":298795428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":332394154},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,0,2,0,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659217,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":564768,"kb_used_data":230416,"kb_used_omap":4,"kb_used_meta":334331,"kb_avail":93807072,"statfs":{"total":96636764160,"available":96058441728,"internally_reserved":0,"allocated":235945984,"data_stored":235613932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":342355630},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,2,5,4,2,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182499,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":731176,"kb_used_data":302476,"kb_used_omap":4,"kb_used_meta":428667,"kb_avail":93640664,"statfs":{"total":96636764160,"available":95888039936,"internally_reserved":0,"allocated":309735424,"data_stored":309432735,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":438955690},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,1,4,3,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":307941376,"data_stored":307905246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":234246144,"data_stored":234180651,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":298364928,"data_stored":298324726,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:06:00.656 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:06:01.208 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:06:01.209 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:06:01.233 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":302,"stamp":"2024-09-10T16:05:59.698342+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":316429065,"num_objects":77265,"num_object_clones":0,"num_object_copies":154530,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42110,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":77265,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":214845,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9281,"num_bytes_recovered":33095699,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160972,"ondisk_log_size":160972,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1912660,"kb_used_data":824980,"kb_used_omap":12,"kb_used_meta":1087603,"kb_avail":281202860,"statfs":{"total":289910292480,"available":287951728640,"internally_reserved":0,"allocated":844779520,"data_stored":843842095,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1113705474},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,0,3,11,7,9,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19968000,"num_objects":-4875,"num_object_clones":0,"num_object_copies":-9750,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-212,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4875,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4875,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":212,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003996"},"pg_stats":[{"pgid":"2.f","version":"47'15895","reported_seq":15975,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:54.605269+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:05:54.605269+0000","last_peered":"2024-09-10T16:05:54.605269+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:05:54.605269+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"43'5800","ondisk_log_start":"43'5800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19566592,"num_objects":4777,"num_object_clones":0,"num_object_copies":9554,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4777,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15895,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":518}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'16029","reported_seq":16147,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:55.480200+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:05:55.480200+0000","last_peered":"2024-09-10T16:05:55.480200+0000","last_clean":"2024-09-10T16:05:55.480200+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:05:55.480200+0000","last_undegraded":"2024-09-10T16:05:55.480200+0000","last_fullsized":"2024-09-10T16:05:55.480200+0000","mapping_epoch":35,"log_start":"43'6000","ondisk_log_start":"43'6000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19723206,"num_objects":4817,"num_object_clones":0,"num_object_copies":9634,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4817,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":14134,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'16019","reported_seq":24702,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:57.812958+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:05:57.812958+0000","last_peered":"2024-09-10T16:05:57.812958+0000","last_clean":"2024-09-10T16:05:57.812958+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:05:57.812958+0000","last_undegraded":"2024-09-10T16:05:57.812958+0000","last_fullsized":"2024-09-10T16:05:57.812958+0000","mapping_epoch":44,"log_start":"43'6000","ondisk_log_start":"43'6000","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19570712,"num_objects":4779,"num_object_clones":0,"num_object_copies":9558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4779,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14023,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'15954","reported_seq":16040,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:55.479778+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:05:55.479778+0000","last_peered":"2024-09-10T16:05:55.479778+0000","last_clean":"2024-09-10T16:05:55.479778+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:05:55.479778+0000","last_undegraded":"2024-09-10T16:05:55.479778+0000","last_fullsized":"2024-09-10T16:05:55.479778+0000","mapping_epoch":35,"log_start":"43'5900","ondisk_log_start":"43'5900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19275776,"num_objects":4706,"num_object_clones":0,"num_object_copies":9412,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4706,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15954,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'16069","reported_seq":16203,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:54.695589+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:05:54.695589+0000","last_peered":"2024-09-10T16:05:54.695589+0000","last_clean":"2024-09-10T16:05:54.695589+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:05:54.695589+0000","last_undegraded":"2024-09-10T16:05:54.695589+0000","last_fullsized":"2024-09-10T16:05:54.695589+0000","mapping_epoch":34,"log_start":"43'6000","ondisk_log_start":"43'6000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19751395,"num_objects":4823,"num_object_clones":0,"num_object_copies":9646,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4823,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12780,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'16225","reported_seq":16368,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:54.640637+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:05:54.640637+0000","last_peered":"2024-09-10T16:05:54.640637+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:05:54.640637+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"43'6200","ondisk_log_start":"43'6200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19808739,"num_objects":4837,"num_object_clones":0,"num_object_copies":9674,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4837,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12837,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":466},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'16073","reported_seq":16216,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:54.620304+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:05:54.620304+0000","last_peered":"2024-09-10T16:05:54.620304+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:05:54.620304+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"43'6000","ondisk_log_start":"43'6000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20517830,"num_objects":5011,"num_object_clones":0,"num_object_copies":10022,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5011,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":12572,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":774},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'16387","reported_seq":16522,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:54.629366+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:05:54.629366+0000","last_peered":"2024-09-10T16:05:54.629366+0000","last_clean":"2024-09-10T16:05:54.629366+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:05:54.629366+0000","last_undegraded":"2024-09-10T16:05:54.629366+0000","last_fullsized":"2024-09-10T16:05:54.629366+0000","mapping_epoch":34,"log_start":"43'6300","ondisk_log_start":"43'6300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20180992,"num_objects":4927,"num_object_clones":0,"num_object_copies":9854,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4927,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13234,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.876895+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:05:19.876895+0000","last_peered":"2024-09-10T16:05:19.876895+0000","last_clean":"2024-09-10T16:05:19.876895+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:05:19.876895+0000","last_undegraded":"2024-09-10T16:05:19.876895+0000","last_fullsized":"2024-09-10T16:05:19.876895+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'15649","reported_seq":15794,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:55.356326+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:05:55.356326+0000","last_peered":"2024-09-10T16:05:55.356326+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:05:55.356326+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"43'5600","ondisk_log_start":"43'5600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19694051,"num_objects":4809,"num_object_clones":0,"num_object_copies":9618,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4809,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13468,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":686}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":99,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.875776+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:05:19.875776+0000","last_peered":"2024-09-10T16:05:19.875776+0000","last_clean":"2024-09-10T16:05:19.875776+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:05:19.875776+0000","last_undegraded":"2024-09-10T16:05:19.875776+0000","last_fullsized":"2024-09-10T16:05:19.875776+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'15992","reported_seq":16128,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:54.569221+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:05:54.569221+0000","last_peered":"2024-09-10T16:05:54.569221+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:05:54.569221+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"43'5900","ondisk_log_start":"43'5900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20144128,"num_objects":4918,"num_object_clones":0,"num_object_copies":9836,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4918,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12496,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":663}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":109,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.871983+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:05:19.871983+0000","last_peered":"2024-09-10T16:05:19.871983+0000","last_clean":"2024-09-10T16:05:19.871983+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:05:19.871983+0000","last_undegraded":"2024-09-10T16:05:19.871983+0000","last_fullsized":"2024-09-10T16:05:19.871983+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'16014","reported_seq":16154,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:55.408984+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:05:55.408984+0000","last_peered":"2024-09-10T16:05:55.408984+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:05:55.408984+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"43'6000","ondisk_log_start":"43'6000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19526115,"num_objects":4768,"num_object_clones":0,"num_object_copies":9536,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4768,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13694,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":464}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877148+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:05:19.877148+0000","last_peered":"2024-09-10T16:05:19.877148+0000","last_clean":"2024-09-10T16:05:19.877148+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:05:19.877148+0000","last_undegraded":"2024-09-10T16:05:19.877148+0000","last_fullsized":"2024-09-10T16:05:19.877148+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'16285","reported_seq":18022,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:55.471260+0000","last_change":"2024-09-10T16:05:18.271152+0000","last_active":"2024-09-10T16:05:55.471260+0000","last_peered":"2024-09-10T16:05:55.471260+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:05:55.471260+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"43'6200","ondisk_log_start":"43'6200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20008960,"num_objects":4885,"num_object_clones":0,"num_object_copies":9770,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3553,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4885,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13839,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":801,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1332}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'16027","reported_seq":16164,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:54.676806+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:05:54.676806+0000","last_peered":"2024-09-10T16:05:54.676806+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:05:54.676806+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"43'6000","ondisk_log_start":"43'6000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19705856,"num_objects":4811,"num_object_clones":0,"num_object_copies":9622,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4811,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12658,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":507}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877654+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:05:19.877654+0000","last_peered":"2024-09-10T16:05:19.877654+0000","last_clean":"2024-09-10T16:05:19.877654+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:05:19.877654+0000","last_undegraded":"2024-09-10T16:05:19.877654+0000","last_fullsized":"2024-09-10T16:05:19.877654+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'15894","reported_seq":24465,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:57.656239+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:05:57.656239+0000","last_peered":"2024-09-10T16:05:57.656239+0000","last_clean":"2024-09-10T16:05:57.656239+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:05:57.656239+0000","last_undegraded":"2024-09-10T16:05:57.656239+0000","last_fullsized":"2024-09-10T16:05:57.656239+0000","mapping_epoch":46,"log_start":"43'5800","ondisk_log_start":"43'5800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19267584,"num_objects":4704,"num_object_clones":0,"num_object_copies":9408,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4704,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12284,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":99,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.876013+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:05:19.876013+0000","last_peered":"2024-09-10T16:05:19.876013+0000","last_clean":"2024-09-10T16:05:19.876013+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:05:19.876013+0000","last_undegraded":"2024-09-10T16:05:19.876013+0000","last_fullsized":"2024-09-10T16:05:19.876013+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'16262","reported_seq":16409,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:54.676515+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:05:54.676515+0000","last_peered":"2024-09-10T16:05:54.676515+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:05:54.676515+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"43'6200","ondisk_log_start":"43'6200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19603456,"num_objects":4786,"num_object_clones":0,"num_object_copies":9572,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4786,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12517,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":415}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877349+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:05:19.877349+0000","last_peered":"2024-09-10T16:05:19.877349+0000","last_clean":"2024-09-10T16:05:19.877349+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:05:19.877349+0000","last_undegraded":"2024-09-10T16:05:19.877349+0000","last_fullsized":"2024-09-10T16:05:19.877349+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877269+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:05:19.877269+0000","last_peered":"2024-09-10T16:05:19.877269+0000","last_clean":"2024-09-10T16:05:19.877269+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:05:19.877269+0000","last_undegraded":"2024-09-10T16:05:19.877269+0000","last_fullsized":"2024-09-10T16:05:19.877269+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'16295","reported_seq":16445,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:54.619479+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:05:54.619479+0000","last_peered":"2024-09-10T16:05:54.619479+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:05:54.619479+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"43'6200","ondisk_log_start":"43'6200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20083654,"num_objects":4905,"num_object_clones":0,"num_object_copies":9810,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4905,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12458,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":582},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":316429046,"num_objects":77263,"num_object_clones":0,"num_object_copies":154526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42110,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":77263,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":214843,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9279,"num_bytes_recovered":33095680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":840552448,"data_stored":840410623,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160969,"ondisk_log_size":160969,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986271,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":616716,"kb_used_data":292088,"kb_used_omap":4,"kb_used_meta":324603,"kb_avail":93755124,"statfs":{"total":96636764160,"available":96005246976,"internally_reserved":0,"allocated":299098112,"data_stored":298795428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":332394154},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,0,2,0,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659217,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":564768,"kb_used_data":230416,"kb_used_omap":4,"kb_used_meta":334331,"kb_avail":93807072,"statfs":{"total":96636764160,"available":96058441728,"internally_reserved":0,"allocated":235945984,"data_stored":235613932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":342355630},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,2,5,4,2,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182499,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":731176,"kb_used_data":302476,"kb_used_omap":4,"kb_used_meta":428667,"kb_avail":93640664,"statfs":{"total":96636764160,"available":95888039936,"internally_reserved":0,"allocated":309735424,"data_stored":309432735,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":438955690},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,1,4,3,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":307941376,"data_stored":307905246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":234246144,"data_stored":234180651,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":298364928,"data_stored":298324726,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:06:01.234 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-10T16:06:01.234 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "47'15895", 'reported_seq': 15975, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:05:54.605269+0000', 'last_change': '2024-09-10T15:59:16.281558+0000', 'last_active': '2024-09-10T16:05:54.605269+0000', 'last_peered': '2024-09-10T16:05:54.605269+0000', 'last_clean': '2024-09-10T15:59:14.132456+0000', 'last_became_active': '2024-09-10T15:59:16.274407+0000', 'last_became_peered': '2024-09-10T15:59:16.274407+0000', 'last_unstale': '2024-09-10T16:05:54.605269+0000', 'last_undegraded': '2024-09-10T15:59:15.851279+0000', 'last_fullsized': '2024-09-10T15:59:15.774356+0000', 'mapping_epoch': 42, 'log_start': "43'5800", 'ondisk_log_start': "43'5800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'objects_scrubbed': 0, 'log_size': 10095, 'log_dups_size': 0, 'ondisk_log_size': 10095, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19566592, 'num_objects': 4777, 'num_object_clones': 0, 'num_object_copies': 9554, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4259, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4777, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15895, 'num_write_kb': 41344, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 518}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:06:01.235 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-10T16:06:01.235 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "47'16225", 'reported_seq': 16368, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:05:54.640637+0000', 'last_change': '2024-09-10T15:59:16.308264+0000', 'last_active': '2024-09-10T16:05:54.640637+0000', 'last_peered': '2024-09-10T16:05:54.640637+0000', 'last_clean': '2024-09-10T15:59:14.111551+0000', 'last_became_active': '2024-09-10T15:59:16.300040+0000', 'last_became_peered': '2024-09-10T15:59:16.300040+0000', 'last_unstale': '2024-09-10T16:05:54.640637+0000', 'last_undegraded': '2024-09-10T15:59:15.878234+0000', 'last_fullsized': '2024-09-10T15:59:15.769328+0000', 'mapping_epoch': 42, 'log_start': "43'6200", 'ondisk_log_start': "43'6200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3442", 'last_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'objects_scrubbed': 3384, 'log_size': 10025, 'log_dups_size': 0, 'ondisk_log_size': 10025, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1292, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19808739, 'num_objects': 4837, 'num_object_clones': 0, 'num_object_copies': 9674, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4372, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4837, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 12837, 'num_write_kb': 28580, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 466}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:06:01.235 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-10T16:06:01.235 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "47'16073", 'reported_seq': 16216, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:05:54.620304+0000', 'last_change': '2024-09-10T15:59:16.331899+0000', 'last_active': '2024-09-10T16:05:54.620304+0000', 'last_peered': '2024-09-10T16:05:54.620304+0000', 'last_clean': '2024-09-10T15:59:14.139436+0000', 'last_became_active': '2024-09-10T15:59:16.318715+0000', 'last_became_peered': '2024-09-10T15:59:16.318715+0000', 'last_unstale': '2024-09-10T16:05:54.620304+0000', 'last_undegraded': '2024-09-10T15:59:15.877524+0000', 'last_fullsized': '2024-09-10T15:59:15.769042+0000', 'mapping_epoch': 42, 'log_start': "43'6000", 'ondisk_log_start': "43'6000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3537", 'last_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'objects_scrubbed': 3497, 'log_size': 10073, 'log_dups_size': 0, 'ondisk_log_size': 10073, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1224, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20517830, 'num_objects': 5011, 'num_object_clones': 0, 'num_object_copies': 10022, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4237, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5011, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 12572, 'num_write_kb': 28172, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 774}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:06:01.235 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-10T16:06:01.235 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "47'15649", 'reported_seq': 15794, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:05:55.356326+0000', 'last_change': '2024-09-10T15:59:16.175680+0000', 'last_active': '2024-09-10T16:05:55.356326+0000', 'last_peered': '2024-09-10T16:05:55.356326+0000', 'last_clean': '2024-09-10T15:59:14.088187+0000', 'last_became_active': '2024-09-10T15:59:16.165810+0000', 'last_became_peered': '2024-09-10T15:59:16.165810+0000', 'last_unstale': '2024-09-10T16:05:55.356326+0000', 'last_undegraded': '2024-09-10T15:59:15.822098+0000', 'last_fullsized': '2024-09-10T15:59:15.769123+0000', 'mapping_epoch': 42, 'log_start': "43'5600", 'ondisk_log_start': "43'5600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2218", 'last_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'objects_scrubbed': 2175, 'log_size': 10049, 'log_dups_size': 0, 'ondisk_log_size': 10049, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 825, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19694051, 'num_objects': 4809, 'num_object_clones': 0, 'num_object_copies': 9618, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4124, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4809, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 13468, 'num_write_kb': 32204, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': 686}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:06:01.235 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-10T16:06:01.235 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "47'15992", 'reported_seq': 16128, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:05:54.569221+0000', 'last_change': '2024-09-10T15:59:16.277742+0000', 'last_active': '2024-09-10T16:05:54.569221+0000', 'last_peered': '2024-09-10T16:05:54.569221+0000', 'last_clean': '2024-09-10T15:59:14.129081+0000', 'last_became_active': '2024-09-10T15:59:16.269835+0000', 'last_became_peered': '2024-09-10T15:59:16.269835+0000', 'last_unstale': '2024-09-10T16:05:54.569221+0000', 'last_undegraded': '2024-09-10T15:59:15.860857+0000', 'last_fullsized': '2024-09-10T15:59:15.771410+0000', 'mapping_epoch': 42, 'log_start': "43'5900", 'ondisk_log_start': "43'5900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3557", 'last_scrub_stamp': '2024-09-10T15:59:02.493678+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:02.493678+0000', 'objects_scrubbed': 3488, 'log_size': 10092, 'log_dups_size': 0, 'ondisk_log_size': 10092, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1472, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20144128, 'num_objects': 4918, 'num_object_clones': 0, 'num_object_copies': 9836, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4257, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4918, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12496, 'num_write_kb': 27852, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 663}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:06:01.235 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-10T16:06:01.235 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "47'16014", 'reported_seq': 16154, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:05:55.408984+0000', 'last_change': '2024-09-10T15:59:16.284311+0000', 'last_active': '2024-09-10T16:05:55.408984+0000', 'last_peered': '2024-09-10T16:05:55.408984+0000', 'last_clean': '2024-09-10T15:59:14.116041+0000', 'last_became_active': '2024-09-10T15:59:16.274543+0000', 'last_became_peered': '2024-09-10T15:59:16.274543+0000', 'last_unstale': '2024-09-10T16:05:55.408984+0000', 'last_undegraded': '2024-09-10T15:59:15.812484+0000', 'last_fullsized': '2024-09-10T15:59:15.770885+0000', 'mapping_epoch': 42, 'log_start': "43'6000", 'ondisk_log_start': "43'6000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2356", 'last_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'objects_scrubbed': 2316, 'log_size': 10014, 'log_dups_size': 0, 'ondisk_log_size': 10014, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 783, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19526115, 'num_objects': 4768, 'num_object_clones': 0, 'num_object_copies': 9536, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4768, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 13694, 'num_write_kb': 32292, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': 464}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:06:01.236 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-10T16:06:01.236 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "47'16285", 'reported_seq': 18022, 'reported_epoch': 47, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:05:55.471260+0000', 'last_change': '2024-09-10T16:05:18.271152+0000', 'last_active': '2024-09-10T16:05:55.471260+0000', 'last_peered': '2024-09-10T16:05:55.471260+0000', 'last_clean': '2024-09-10T15:59:14.110654+0000', 'last_became_active': '2024-09-10T15:59:16.164645+0000', 'last_became_peered': '2024-09-10T15:59:16.164645+0000', 'last_unstale': '2024-09-10T16:05:55.471260+0000', 'last_undegraded': '2024-09-10T15:59:15.803593+0000', 'last_fullsized': '2024-09-10T15:59:15.769083+0000', 'mapping_epoch': 42, 'log_start': "43'6200", 'ondisk_log_start': "43'6200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2486", 'last_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'objects_scrubbed': 2440, 'log_size': 10085, 'log_dups_size': 0, 'ondisk_log_size': 10085, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 792, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20008960, 'num_objects': 4885, 'num_object_clones': 0, 'num_object_copies': 9770, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3553, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4885, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13839, 'num_write_kb': 32568, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 801, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 1332}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:06:01.236 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-10T16:06:01.236 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "47'16027", 'reported_seq': 16164, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:05:54.676806+0000', 'last_change': '2024-09-10T15:59:16.315641+0000', 'last_active': '2024-09-10T16:05:54.676806+0000', 'last_peered': '2024-09-10T16:05:54.676806+0000', 'last_clean': '2024-09-10T15:59:14.117001+0000', 'last_became_active': '2024-09-10T15:59:16.302245+0000', 'last_became_peered': '2024-09-10T15:59:16.302245+0000', 'last_unstale': '2024-09-10T16:05:54.676806+0000', 'last_undegraded': '2024-09-10T15:59:15.845012+0000', 'last_fullsized': '2024-09-10T15:59:15.770337+0000', 'mapping_epoch': 42, 'log_start': "43'6000", 'ondisk_log_start': "43'6000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3404", 'last_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'objects_scrubbed': 3367, 'log_size': 10027, 'log_dups_size': 0, 'ondisk_log_size': 10027, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 983, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19705856, 'num_objects': 4811, 'num_object_clones': 0, 'num_object_copies': 9622, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4811, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12658, 'num_write_kb': 28204, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 507}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:06:01.236 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-10T16:06:01.236 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "47'16262", 'reported_seq': 16409, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:05:54.676515+0000', 'last_change': '2024-09-10T15:59:16.311291+0000', 'last_active': '2024-09-10T16:05:54.676515+0000', 'last_peered': '2024-09-10T16:05:54.676515+0000', 'last_clean': '2024-09-10T15:59:14.116363+0000', 'last_became_active': '2024-09-10T15:59:16.301244+0000', 'last_became_peered': '2024-09-10T15:59:16.301244+0000', 'last_unstale': '2024-09-10T16:05:54.676515+0000', 'last_undegraded': '2024-09-10T15:59:15.883434+0000', 'last_fullsized': '2024-09-10T15:59:15.772756+0000', 'mapping_epoch': 42, 'log_start': "43'6200", 'ondisk_log_start': "43'6200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3789", 'last_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'objects_scrubbed': 3741, 'log_size': 10062, 'log_dups_size': 0, 'ondisk_log_size': 10062, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1440, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19603456, 'num_objects': 4786, 'num_object_clones': 0, 'num_object_copies': 9572, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4373, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4786, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12517, 'num_write_kb': 27124, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 415}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:06:01.236 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-10T16:06:01.236 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "47'16295", 'reported_seq': 16445, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:05:54.619479+0000', 'last_change': '2024-09-10T15:59:16.313409+0000', 'last_active': '2024-09-10T16:05:54.619479+0000', 'last_peered': '2024-09-10T16:05:54.619479+0000', 'last_clean': '2024-09-10T15:59:14.129780+0000', 'last_became_active': '2024-09-10T15:59:16.300932+0000', 'last_became_peered': '2024-09-10T15:59:16.300932+0000', 'last_unstale': '2024-09-10T16:05:54.619479+0000', 'last_undegraded': '2024-09-10T15:59:15.887823+0000', 'last_fullsized': '2024-09-10T15:59:15.768513+0000', 'mapping_epoch': 42, 'log_start': "43'6200", 'ondisk_log_start': "43'6200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3880", 'last_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'objects_scrubbed': 3827, 'log_size': 10095, 'log_dups_size': 0, 'ondisk_log_size': 10095, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1397, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20083654, 'num_objects': 4905, 'num_object_clones': 0, 'num_object_copies': 9810, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4325, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4905, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12458, 'num_write_kb': 27072, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 582}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:06:01.236 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T16:06:01.916 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:06:01.916 INFO:teuthology.orchestra.run.smithi115.stdout:{"fsid":"7effee29-df05-4351-9741-ee203564ed81","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 42110/154530 objects degraded (27.250%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":525,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":47,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983954,"num_in_osds":3,"osd_in_since":1725983947,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":77265,"data_bytes":316429065,"bytes_used":1958563840,"bytes_avail":287951728640,"bytes_total":289910292480,"degraded_objects":42110,"degraded_total":154530,"degraded_ratio":0.27250372096033132,"recovering_objects_per_sec":17,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":212,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":406},"fsmap":{"epoch":1,"btime":"2024-09-10T15:57:14:967120+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T16:00:13.560776+0000","services":{}},"progress_events":{}} 2024-09-10T16:06:01.942 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-10T16:06:01.942 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:06:02.464 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:06:02.466 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:06:02.503 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":303,"stamp":"2024-09-10T16:06:01.698976+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":308114185,"num_objects":75235,"num_object_clones":0,"num_object_copies":150470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42006,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":75235,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":216875,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9385,"num_bytes_recovered":33095699,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160902,"ondisk_log_size":160902,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1927696,"kb_used_data":813644,"kb_used_omap":12,"kb_used_meta":1113971,"kb_avail":281187824,"statfs":{"total":289910292480,"available":287936331776,"internally_reserved":0,"allocated":833171456,"data_stored":832234031,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1140706306},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,5,0,4,10,13,4,6],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-21868544,"num_objects":-5339,"num_object_clones":0,"num_object_copies":-10678,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-316,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5339,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5339,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":316,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004077"},"pg_stats":[{"pgid":"2.f","version":"47'16032","reported_seq":16111,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:59.728502+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:05:59.728502+0000","last_peered":"2024-09-10T16:05:59.728502+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:05:59.728502+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"43'6000","ondisk_log_start":"43'6000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19005440,"num_objects":4640,"num_object_clones":0,"num_object_copies":9280,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4640,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16032,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":381}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'16181","reported_seq":16299,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:06:00.451019+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:06:00.451019+0000","last_peered":"2024-09-10T16:06:00.451019+0000","last_clean":"2024-09-10T16:06:00.451019+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:06:00.451019+0000","last_undegraded":"2024-09-10T16:06:00.451019+0000","last_fullsized":"2024-09-10T16:06:00.451019+0000","mapping_epoch":35,"log_start":"43'6100","ondisk_log_start":"43'6100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19100614,"num_objects":4665,"num_object_clones":0,"num_object_copies":9330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4665,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":14286,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'16019","reported_seq":24702,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:57.812958+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:05:57.812958+0000","last_peered":"2024-09-10T16:05:57.812958+0000","last_clean":"2024-09-10T16:05:57.812958+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:05:57.812958+0000","last_undegraded":"2024-09-10T16:05:57.812958+0000","last_fullsized":"2024-09-10T16:05:57.812958+0000","mapping_epoch":44,"log_start":"43'6000","ondisk_log_start":"43'6000","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19570712,"num_objects":4779,"num_object_clones":0,"num_object_copies":9558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4779,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14023,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'16099","reported_seq":16185,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:06:00.491461+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:06:00.491461+0000","last_peered":"2024-09-10T16:06:00.491461+0000","last_clean":"2024-09-10T16:06:00.491461+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:06:00.491461+0000","last_undegraded":"2024-09-10T16:06:00.491461+0000","last_fullsized":"2024-09-10T16:06:00.491461+0000","mapping_epoch":35,"log_start":"43'6000","ondisk_log_start":"43'6000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18681856,"num_objects":4561,"num_object_clones":0,"num_object_copies":9122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4561,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16099,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'16217","reported_seq":16352,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:59.702442+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:05:59.702442+0000","last_peered":"2024-09-10T16:05:59.702442+0000","last_clean":"2024-09-10T16:05:59.702442+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:05:59.702442+0000","last_undegraded":"2024-09-10T16:05:59.702442+0000","last_fullsized":"2024-09-10T16:05:59.702442+0000","mapping_epoch":34,"log_start":"43'6200","ondisk_log_start":"43'6200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19145187,"num_objects":4675,"num_object_clones":0,"num_object_copies":9350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4675,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12928,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'16381","reported_seq":16524,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:59.689729+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:05:59.689729+0000","last_peered":"2024-09-10T16:05:59.689729+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:05:59.689729+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"43'6300","ondisk_log_start":"43'6300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19169763,"num_objects":4681,"num_object_clones":0,"num_object_copies":9362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4681,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12993,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":310},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'16226","reported_seq":16369,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:59.560710+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:05:59.560710+0000","last_peered":"2024-09-10T16:05:59.560710+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:05:59.560710+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"43'6200","ondisk_log_start":"43'6200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19891142,"num_objects":4858,"num_object_clones":0,"num_object_copies":9716,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4858,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":12725,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":621},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'16519","reported_seq":16654,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:59.727181+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:05:59.727181+0000","last_peered":"2024-09-10T16:05:59.727181+0000","last_clean":"2024-09-10T16:05:59.727181+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:05:59.727181+0000","last_undegraded":"2024-09-10T16:05:59.727181+0000","last_fullsized":"2024-09-10T16:05:59.727181+0000","mapping_epoch":34,"log_start":"43'6500","ondisk_log_start":"43'6500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19640320,"num_objects":4795,"num_object_clones":0,"num_object_copies":9590,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4795,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13366,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.876895+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:05:19.876895+0000","last_peered":"2024-09-10T16:05:19.876895+0000","last_clean":"2024-09-10T16:05:19.876895+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:05:19.876895+0000","last_undegraded":"2024-09-10T16:05:19.876895+0000","last_fullsized":"2024-09-10T16:05:19.876895+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'15785","reported_seq":15930,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:00.477604+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:06:00.477604+0000","last_peered":"2024-09-10T16:06:00.477604+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:06:00.477604+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"43'5700","ondisk_log_start":"43'5700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19136995,"num_objects":4673,"num_object_clones":0,"num_object_copies":9346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4673,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13604,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":550}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":99,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.875776+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:05:19.875776+0000","last_peered":"2024-09-10T16:05:19.875776+0000","last_clean":"2024-09-10T16:05:19.875776+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:05:19.875776+0000","last_undegraded":"2024-09-10T16:05:19.875776+0000","last_fullsized":"2024-09-10T16:05:19.875776+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'16128","reported_seq":16264,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:59.677052+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:05:59.677052+0000","last_peered":"2024-09-10T16:05:59.677052+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:05:59.677052+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"43'6100","ondisk_log_start":"43'6100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19587072,"num_objects":4782,"num_object_clones":0,"num_object_copies":9564,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4782,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12632,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":527}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":109,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.871983+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:05:19.871983+0000","last_peered":"2024-09-10T16:05:19.871983+0000","last_clean":"2024-09-10T16:05:19.871983+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:05:19.871983+0000","last_undegraded":"2024-09-10T16:05:19.871983+0000","last_fullsized":"2024-09-10T16:05:19.871983+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'16154","reported_seq":16293,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:00.466462+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:06:00.466462+0000","last_peered":"2024-09-10T16:06:00.466462+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:06:00.466462+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"43'6100","ondisk_log_start":"43'6100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18952675,"num_objects":4628,"num_object_clones":0,"num_object_copies":9256,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4628,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13834,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":324}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877148+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:05:19.877148+0000","last_peered":"2024-09-10T16:05:19.877148+0000","last_clean":"2024-09-10T16:05:19.877148+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:05:19.877148+0000","last_undegraded":"2024-09-10T16:05:19.877148+0000","last_fullsized":"2024-09-10T16:05:19.877148+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'16441","reported_seq":18386,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:00.470099+0000","last_change":"2024-09-10T16:05:18.271152+0000","last_active":"2024-09-10T16:06:00.470099+0000","last_peered":"2024-09-10T16:06:00.470099+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:06:00.470099+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"43'6400","ondisk_log_start":"43'6400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19369984,"num_objects":4729,"num_object_clones":0,"num_object_copies":9458,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3449,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4729,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13995,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":905,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1280}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'16184","reported_seq":16324,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:59.694806+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:05:59.694806+0000","last_peered":"2024-09-10T16:05:59.694806+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:05:59.694806+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"43'6100","ondisk_log_start":"43'6100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19062784,"num_objects":4654,"num_object_clones":0,"num_object_copies":9308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4654,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12815,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":350}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877654+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:05:19.877654+0000","last_peered":"2024-09-10T16:05:19.877654+0000","last_clean":"2024-09-10T16:05:19.877654+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:05:19.877654+0000","last_undegraded":"2024-09-10T16:05:19.877654+0000","last_fullsized":"2024-09-10T16:05:19.877654+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'15894","reported_seq":24465,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:57.656239+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:05:57.656239+0000","last_peered":"2024-09-10T16:05:57.656239+0000","last_clean":"2024-09-10T16:05:57.656239+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:05:57.656239+0000","last_undegraded":"2024-09-10T16:05:57.656239+0000","last_fullsized":"2024-09-10T16:05:57.656239+0000","mapping_epoch":46,"log_start":"43'5800","ondisk_log_start":"43'5800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19267584,"num_objects":4704,"num_object_clones":0,"num_object_copies":9408,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4704,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12284,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":99,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.876013+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:05:19.876013+0000","last_peered":"2024-09-10T16:05:19.876013+0000","last_clean":"2024-09-10T16:05:19.876013+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:05:19.876013+0000","last_undegraded":"2024-09-10T16:05:19.876013+0000","last_fullsized":"2024-09-10T16:05:19.876013+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'16398","reported_seq":16544,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:59.735245+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:05:59.735245+0000","last_peered":"2024-09-10T16:05:59.735245+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:05:59.735245+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"43'6300","ondisk_log_start":"43'6300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19046400,"num_objects":4650,"num_object_clones":0,"num_object_copies":9300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4650,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12653,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":279}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877349+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:05:19.877349+0000","last_peered":"2024-09-10T16:05:19.877349+0000","last_clean":"2024-09-10T16:05:19.877349+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:05:19.877349+0000","last_undegraded":"2024-09-10T16:05:19.877349+0000","last_fullsized":"2024-09-10T16:05:19.877349+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877269+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:05:19.877269+0000","last_peered":"2024-09-10T16:05:19.877269+0000","last_clean":"2024-09-10T16:05:19.877269+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:05:19.877269+0000","last_undegraded":"2024-09-10T16:05:19.877269+0000","last_fullsized":"2024-09-10T16:05:19.877269+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'16441","reported_seq":16591,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:59.691795+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:05:59.691795+0000","last_peered":"2024-09-10T16:05:59.691795+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:05:59.691795+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"43'6400","ondisk_log_start":"43'6400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19485638,"num_objects":4759,"num_object_clones":0,"num_object_copies":9518,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4759,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12604,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":436},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":308114166,"num_objects":75233,"num_object_clones":0,"num_object_copies":150466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42006,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":75233,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":216873,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9383,"num_bytes_recovered":33095680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":828755968,"data_stored":828614143,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160899,"ondisk_log_size":160899,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986272,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":634452,"kb_used_data":287616,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93737388,"statfs":{"total":96636764160,"available":95987085312,"internally_reserved":0,"allocated":294518784,"data_stored":294216100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":355135146},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,4,0,2,0,5,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659217,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":564768,"kb_used_data":230416,"kb_used_omap":4,"kb_used_meta":334331,"kb_avail":93807072,"statfs":{"total":96636764160,"available":96058441728,"internally_reserved":0,"allocated":235945984,"data_stored":235613932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":342355630},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,2,5,4,2,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182500,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":728476,"kb_used_data":295612,"kb_used_omap":4,"kb_used_meta":432827,"kb_avail":93643364,"statfs":{"total":96636764160,"available":95890804736,"internally_reserved":0,"allocated":302706688,"data_stored":302403999,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":443215530},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,0,5,4,1,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":300847104,"data_stored":300810974,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":234246144,"data_stored":234180651,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":293662720,"data_stored":293622518,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:06:05.505 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:06:06.053 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:06:06.054 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:06:06.077 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":305,"stamp":"2024-09-10T16:06:05.700482+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":298627849,"num_objects":72919,"num_object_clones":0,"num_object_copies":145838,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41895,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72919,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":219191,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9496,"num_bytes_recovered":33095699,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160918,"ondisk_log_size":160918,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1904964,"kb_used_data":795080,"kb_used_omap":12,"kb_used_meta":1109811,"kb_avail":281210556,"statfs":{"total":289910292480,"available":287959609344,"internally_reserved":0,"allocated":814161920,"data_stored":813224495,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1136446466},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,5,7,9,6,7,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-27299840,"num_objects":-6665,"num_object_clones":0,"num_object_copies":-13330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-6665,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6665,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":320,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004217"},"pg_stats":[{"pgid":"2.f","version":"47'16182","reported_seq":16261,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:04.745686+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:06:04.745686+0000","last_peered":"2024-09-10T16:06:04.745686+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:06:04.745686+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"43'6100","ondisk_log_start":"43'6100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18391040,"num_objects":4490,"num_object_clones":0,"num_object_copies":8980,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4490,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16182,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":231}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'16312","reported_seq":16430,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:06:05.530234+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:06:05.530234+0000","last_peered":"2024-09-10T16:06:05.530234+0000","last_clean":"2024-09-10T16:06:05.530234+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:06:05.530234+0000","last_undegraded":"2024-09-10T16:06:05.530234+0000","last_fullsized":"2024-09-10T16:06:05.530234+0000","mapping_epoch":35,"log_start":"43'6300","ondisk_log_start":"43'6300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18564038,"num_objects":4534,"num_object_clones":0,"num_object_copies":9068,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4534,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":14417,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'16160","reported_seq":24843,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:06:02.912813+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:06:02.912813+0000","last_peered":"2024-09-10T16:06:02.912813+0000","last_clean":"2024-09-10T16:06:02.912813+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:06:02.912813+0000","last_undegraded":"2024-09-10T16:06:02.912813+0000","last_fullsized":"2024-09-10T16:06:02.912813+0000","mapping_epoch":44,"log_start":"43'6100","ondisk_log_start":"43'6100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18993176,"num_objects":4638,"num_object_clones":0,"num_object_copies":9276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4638,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14164,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'16258","reported_seq":16343,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:06:05.537069+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:06:05.537069+0000","last_peered":"2024-09-10T16:06:05.537069+0000","last_clean":"2024-09-10T16:06:05.537069+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:06:05.537069+0000","last_undegraded":"2024-09-10T16:06:05.537069+0000","last_fullsized":"2024-09-10T16:06:05.537069+0000","mapping_epoch":35,"log_start":"43'6200","ondisk_log_start":"43'6200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18030592,"num_objects":4402,"num_object_clones":0,"num_object_copies":8804,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4402,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16258,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'16357","reported_seq":16491,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:06:04.743046+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:06:04.743046+0000","last_peered":"2024-09-10T16:06:04.743046+0000","last_clean":"2024-09-10T16:06:04.743046+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:06:04.743046+0000","last_undegraded":"2024-09-10T16:06:04.743046+0000","last_fullsized":"2024-09-10T16:06:04.743046+0000","mapping_epoch":34,"log_start":"43'6300","ondisk_log_start":"43'6300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18571747,"num_objects":4535,"num_object_clones":0,"num_object_copies":9070,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4535,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13068,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'16531","reported_seq":16671,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:04.753895+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:06:04.753895+0000","last_peered":"2024-09-10T16:06:04.753895+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:06:04.753895+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"43'6500","ondisk_log_start":"43'6500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18555363,"num_objects":4531,"num_object_clones":0,"num_object_copies":9062,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4531,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13143,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":160},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'16383","reported_seq":16525,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:04.736127+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:06:04.736127+0000","last_peered":"2024-09-10T16:06:04.736127+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:06:04.736127+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"43'6300","ondisk_log_start":"43'6300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19248070,"num_objects":4701,"num_object_clones":0,"num_object_copies":9402,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4701,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":12882,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":464},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'16664","reported_seq":16799,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:06:04.748781+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:06:04.748781+0000","last_peered":"2024-09-10T16:06:04.748781+0000","last_clean":"2024-09-10T16:06:04.748781+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:06:04.748781+0000","last_undegraded":"2024-09-10T16:06:04.748781+0000","last_fullsized":"2024-09-10T16:06:04.748781+0000","mapping_epoch":34,"log_start":"43'6600","ondisk_log_start":"43'6600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19046400,"num_objects":4650,"num_object_clones":0,"num_object_copies":9300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4650,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13511,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.876895+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:05:19.876895+0000","last_peered":"2024-09-10T16:05:19.876895+0000","last_clean":"2024-09-10T16:05:19.876895+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:05:19.876895+0000","last_undegraded":"2024-09-10T16:05:19.876895+0000","last_fullsized":"2024-09-10T16:05:19.876895+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'15914","reported_seq":16059,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:05.540627+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:06:05.540627+0000","last_peered":"2024-09-10T16:06:05.540627+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:06:05.540627+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"43'5900","ondisk_log_start":"43'5900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18608611,"num_objects":4544,"num_object_clones":0,"num_object_copies":9088,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4544,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13733,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":421}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":99,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.875776+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:05:19.875776+0000","last_peered":"2024-09-10T16:05:19.875776+0000","last_clean":"2024-09-10T16:05:19.875776+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:05:19.875776+0000","last_undegraded":"2024-09-10T16:05:19.875776+0000","last_fullsized":"2024-09-10T16:05:19.875776+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'16263","reported_seq":16399,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:04.696844+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:06:04.696844+0000","last_peered":"2024-09-10T16:06:04.696844+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:06:04.696844+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"43'6200","ondisk_log_start":"43'6200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19034112,"num_objects":4647,"num_object_clones":0,"num_object_copies":9294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4647,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12767,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":392}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":109,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.871983+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:05:19.871983+0000","last_peered":"2024-09-10T16:05:19.871983+0000","last_clean":"2024-09-10T16:05:19.871983+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:05:19.871983+0000","last_undegraded":"2024-09-10T16:05:19.871983+0000","last_fullsized":"2024-09-10T16:05:19.871983+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'16288","reported_seq":16428,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:05.516035+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:06:05.516035+0000","last_peered":"2024-09-10T16:06:05.516035+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:06:05.516035+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"43'6200","ondisk_log_start":"43'6200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18403811,"num_objects":4494,"num_object_clones":0,"num_object_copies":8988,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4494,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13968,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":190}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877148+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:05:19.877148+0000","last_peered":"2024-09-10T16:05:19.877148+0000","last_clean":"2024-09-10T16:05:19.877148+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:05:19.877148+0000","last_undegraded":"2024-09-10T16:05:19.877148+0000","last_fullsized":"2024-09-10T16:05:19.877148+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'16589","reported_seq":18754,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:05.556952+0000","last_change":"2024-09-10T16:05:18.271152+0000","last_active":"2024-09-10T16:06:05.556952+0000","last_peered":"2024-09-10T16:06:05.556952+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:06:05.556952+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"43'6500","ondisk_log_start":"43'6500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18763776,"num_objects":4581,"num_object_clones":0,"num_object_copies":9162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3338,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4581,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14143,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1016,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1243}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'16329","reported_seq":16469,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:04.665681+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:06:04.665681+0000","last_peered":"2024-09-10T16:06:04.665681+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:06:04.665681+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"43'6300","ondisk_log_start":"43'6300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18468864,"num_objects":4509,"num_object_clones":0,"num_object_copies":9018,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4509,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12960,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":205}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877654+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:05:19.877654+0000","last_peered":"2024-09-10T16:05:19.877654+0000","last_clean":"2024-09-10T16:05:19.877654+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:05:19.877654+0000","last_undegraded":"2024-09-10T16:05:19.877654+0000","last_fullsized":"2024-09-10T16:05:19.877654+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'16044","reported_seq":24615,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:06:02.893799+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:06:02.893799+0000","last_peered":"2024-09-10T16:06:02.893799+0000","last_clean":"2024-09-10T16:06:02.893799+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:06:02.893799+0000","last_undegraded":"2024-09-10T16:06:02.893799+0000","last_fullsized":"2024-09-10T16:06:02.893799+0000","mapping_epoch":46,"log_start":"43'6000","ondisk_log_start":"43'6000","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18653184,"num_objects":4554,"num_object_clones":0,"num_object_copies":9108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4554,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12434,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":99,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.876013+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:05:19.876013+0000","last_peered":"2024-09-10T16:05:19.876013+0000","last_clean":"2024-09-10T16:05:19.876013+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:05:19.876013+0000","last_undegraded":"2024-09-10T16:05:19.876013+0000","last_fullsized":"2024-09-10T16:05:19.876013+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'16542","reported_seq":16689,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:04.754167+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:06:04.754167+0000","last_peered":"2024-09-10T16:06:04.754167+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:06:04.754167+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"43'6500","ondisk_log_start":"43'6500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18456576,"num_objects":4506,"num_object_clones":0,"num_object_copies":9012,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4506,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12797,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":135}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877349+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:05:19.877349+0000","last_peered":"2024-09-10T16:05:19.877349+0000","last_clean":"2024-09-10T16:05:19.877349+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:05:19.877349+0000","last_undegraded":"2024-09-10T16:05:19.877349+0000","last_fullsized":"2024-09-10T16:05:19.877349+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877269+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:05:19.877269+0000","last_peered":"2024-09-10T16:05:19.877269+0000","last_clean":"2024-09-10T16:05:19.877269+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:05:19.877269+0000","last_undegraded":"2024-09-10T16:05:19.877269+0000","last_fullsized":"2024-09-10T16:05:19.877269+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'16599","reported_seq":16749,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:04.745577+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:06:04.745577+0000","last_peered":"2024-09-10T16:06:04.745577+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:06:04.745577+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"43'6500","ondisk_log_start":"43'6500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18838470,"num_objects":4601,"num_object_clones":0,"num_object_copies":9202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4601,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12762,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":278},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":298627830,"num_objects":72917,"num_object_clones":0,"num_object_copies":145834,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41895,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72917,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":219189,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9494,"num_bytes_recovered":33095680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":809799680,"data_stored":809657855,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160915,"ondisk_log_size":160915,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986273,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":617556,"kb_used_data":283204,"kb_used_omap":4,"kb_used_meta":334331,"kb_avail":93754284,"statfs":{"total":96636764160,"available":96004386816,"internally_reserved":0,"allocated":290000896,"data_stored":289698212,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":342355626},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,3],"upper_bound":32},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659218,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":561944,"kb_used_data":223424,"kb_used_omap":4,"kb_used_meta":338491,"kb_avail":93809896,"statfs":{"total":96636764160,"available":96061333504,"internally_reserved":0,"allocated":228786176,"data_stored":228454124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":346615470},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,2,5,4,2,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182501,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":725464,"kb_used_data":288452,"kb_used_omap":4,"kb_used_meta":436987,"kb_avail":93646376,"statfs":{"total":96636764160,"available":95893889024,"internally_reserved":0,"allocated":295374848,"data_stored":295072159,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":447475370},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,4,3,1,2,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":293666816,"data_stored":293630686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":227086336,"data_stored":227020843,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":289046528,"data_stored":289006326,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:06:06.078 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:06:06.655 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:06:06.657 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:06:06.679 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":305,"stamp":"2024-09-10T16:06:05.700482+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":298627849,"num_objects":72919,"num_object_clones":0,"num_object_copies":145838,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41895,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72919,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":219191,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9496,"num_bytes_recovered":33095699,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160918,"ondisk_log_size":160918,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1904964,"kb_used_data":795080,"kb_used_omap":12,"kb_used_meta":1109811,"kb_avail":281210556,"statfs":{"total":289910292480,"available":287959609344,"internally_reserved":0,"allocated":814161920,"data_stored":813224495,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1136446466},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,5,7,9,6,7,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-27299840,"num_objects":-6665,"num_object_clones":0,"num_object_copies":-13330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-6665,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6665,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":320,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004217"},"pg_stats":[{"pgid":"2.f","version":"47'16182","reported_seq":16261,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:04.745686+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:06:04.745686+0000","last_peered":"2024-09-10T16:06:04.745686+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:06:04.745686+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"43'6100","ondisk_log_start":"43'6100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18391040,"num_objects":4490,"num_object_clones":0,"num_object_copies":8980,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4490,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16182,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":231}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'16312","reported_seq":16430,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:06:05.530234+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:06:05.530234+0000","last_peered":"2024-09-10T16:06:05.530234+0000","last_clean":"2024-09-10T16:06:05.530234+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:06:05.530234+0000","last_undegraded":"2024-09-10T16:06:05.530234+0000","last_fullsized":"2024-09-10T16:06:05.530234+0000","mapping_epoch":35,"log_start":"43'6300","ondisk_log_start":"43'6300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18564038,"num_objects":4534,"num_object_clones":0,"num_object_copies":9068,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4534,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":14417,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'16160","reported_seq":24843,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:06:02.912813+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:06:02.912813+0000","last_peered":"2024-09-10T16:06:02.912813+0000","last_clean":"2024-09-10T16:06:02.912813+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:06:02.912813+0000","last_undegraded":"2024-09-10T16:06:02.912813+0000","last_fullsized":"2024-09-10T16:06:02.912813+0000","mapping_epoch":44,"log_start":"43'6100","ondisk_log_start":"43'6100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18993176,"num_objects":4638,"num_object_clones":0,"num_object_copies":9276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4638,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14164,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'16258","reported_seq":16343,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:06:05.537069+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:06:05.537069+0000","last_peered":"2024-09-10T16:06:05.537069+0000","last_clean":"2024-09-10T16:06:05.537069+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:06:05.537069+0000","last_undegraded":"2024-09-10T16:06:05.537069+0000","last_fullsized":"2024-09-10T16:06:05.537069+0000","mapping_epoch":35,"log_start":"43'6200","ondisk_log_start":"43'6200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18030592,"num_objects":4402,"num_object_clones":0,"num_object_copies":8804,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4402,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16258,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'16357","reported_seq":16491,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:06:04.743046+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:06:04.743046+0000","last_peered":"2024-09-10T16:06:04.743046+0000","last_clean":"2024-09-10T16:06:04.743046+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:06:04.743046+0000","last_undegraded":"2024-09-10T16:06:04.743046+0000","last_fullsized":"2024-09-10T16:06:04.743046+0000","mapping_epoch":34,"log_start":"43'6300","ondisk_log_start":"43'6300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18571747,"num_objects":4535,"num_object_clones":0,"num_object_copies":9070,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4535,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13068,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'16531","reported_seq":16671,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:04.753895+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:06:04.753895+0000","last_peered":"2024-09-10T16:06:04.753895+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:06:04.753895+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"43'6500","ondisk_log_start":"43'6500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18555363,"num_objects":4531,"num_object_clones":0,"num_object_copies":9062,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4531,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13143,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":160},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'16383","reported_seq":16525,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:04.736127+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:06:04.736127+0000","last_peered":"2024-09-10T16:06:04.736127+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:06:04.736127+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"43'6300","ondisk_log_start":"43'6300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19248070,"num_objects":4701,"num_object_clones":0,"num_object_copies":9402,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4701,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":12882,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":464},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'16664","reported_seq":16799,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:06:04.748781+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:06:04.748781+0000","last_peered":"2024-09-10T16:06:04.748781+0000","last_clean":"2024-09-10T16:06:04.748781+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:06:04.748781+0000","last_undegraded":"2024-09-10T16:06:04.748781+0000","last_fullsized":"2024-09-10T16:06:04.748781+0000","mapping_epoch":34,"log_start":"43'6600","ondisk_log_start":"43'6600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19046400,"num_objects":4650,"num_object_clones":0,"num_object_copies":9300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4650,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13511,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.876895+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:05:19.876895+0000","last_peered":"2024-09-10T16:05:19.876895+0000","last_clean":"2024-09-10T16:05:19.876895+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:05:19.876895+0000","last_undegraded":"2024-09-10T16:05:19.876895+0000","last_fullsized":"2024-09-10T16:05:19.876895+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'15914","reported_seq":16059,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:05.540627+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:06:05.540627+0000","last_peered":"2024-09-10T16:06:05.540627+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:06:05.540627+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"43'5900","ondisk_log_start":"43'5900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18608611,"num_objects":4544,"num_object_clones":0,"num_object_copies":9088,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4544,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13733,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":421}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":99,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.875776+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:05:19.875776+0000","last_peered":"2024-09-10T16:05:19.875776+0000","last_clean":"2024-09-10T16:05:19.875776+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:05:19.875776+0000","last_undegraded":"2024-09-10T16:05:19.875776+0000","last_fullsized":"2024-09-10T16:05:19.875776+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'16263","reported_seq":16399,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:04.696844+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:06:04.696844+0000","last_peered":"2024-09-10T16:06:04.696844+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:06:04.696844+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"43'6200","ondisk_log_start":"43'6200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19034112,"num_objects":4647,"num_object_clones":0,"num_object_copies":9294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4647,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12767,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":392}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":109,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.871983+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:05:19.871983+0000","last_peered":"2024-09-10T16:05:19.871983+0000","last_clean":"2024-09-10T16:05:19.871983+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:05:19.871983+0000","last_undegraded":"2024-09-10T16:05:19.871983+0000","last_fullsized":"2024-09-10T16:05:19.871983+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'16288","reported_seq":16428,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:05.516035+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:06:05.516035+0000","last_peered":"2024-09-10T16:06:05.516035+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:06:05.516035+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"43'6200","ondisk_log_start":"43'6200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18403811,"num_objects":4494,"num_object_clones":0,"num_object_copies":8988,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4494,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13968,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":190}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877148+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:05:19.877148+0000","last_peered":"2024-09-10T16:05:19.877148+0000","last_clean":"2024-09-10T16:05:19.877148+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:05:19.877148+0000","last_undegraded":"2024-09-10T16:05:19.877148+0000","last_fullsized":"2024-09-10T16:05:19.877148+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'16589","reported_seq":18754,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:05.556952+0000","last_change":"2024-09-10T16:05:18.271152+0000","last_active":"2024-09-10T16:06:05.556952+0000","last_peered":"2024-09-10T16:06:05.556952+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:06:05.556952+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"43'6500","ondisk_log_start":"43'6500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18763776,"num_objects":4581,"num_object_clones":0,"num_object_copies":9162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3338,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4581,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14143,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1016,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1243}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'16329","reported_seq":16469,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:04.665681+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:06:04.665681+0000","last_peered":"2024-09-10T16:06:04.665681+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:06:04.665681+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"43'6300","ondisk_log_start":"43'6300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18468864,"num_objects":4509,"num_object_clones":0,"num_object_copies":9018,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4509,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12960,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":205}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877654+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:05:19.877654+0000","last_peered":"2024-09-10T16:05:19.877654+0000","last_clean":"2024-09-10T16:05:19.877654+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:05:19.877654+0000","last_undegraded":"2024-09-10T16:05:19.877654+0000","last_fullsized":"2024-09-10T16:05:19.877654+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'16044","reported_seq":24615,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:06:02.893799+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:06:02.893799+0000","last_peered":"2024-09-10T16:06:02.893799+0000","last_clean":"2024-09-10T16:06:02.893799+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:06:02.893799+0000","last_undegraded":"2024-09-10T16:06:02.893799+0000","last_fullsized":"2024-09-10T16:06:02.893799+0000","mapping_epoch":46,"log_start":"43'6000","ondisk_log_start":"43'6000","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18653184,"num_objects":4554,"num_object_clones":0,"num_object_copies":9108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4554,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12434,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":99,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.876013+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:05:19.876013+0000","last_peered":"2024-09-10T16:05:19.876013+0000","last_clean":"2024-09-10T16:05:19.876013+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:05:19.876013+0000","last_undegraded":"2024-09-10T16:05:19.876013+0000","last_fullsized":"2024-09-10T16:05:19.876013+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'16542","reported_seq":16689,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:04.754167+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:06:04.754167+0000","last_peered":"2024-09-10T16:06:04.754167+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:06:04.754167+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"43'6500","ondisk_log_start":"43'6500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18456576,"num_objects":4506,"num_object_clones":0,"num_object_copies":9012,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4506,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12797,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":135}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877349+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:05:19.877349+0000","last_peered":"2024-09-10T16:05:19.877349+0000","last_clean":"2024-09-10T16:05:19.877349+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:05:19.877349+0000","last_undegraded":"2024-09-10T16:05:19.877349+0000","last_fullsized":"2024-09-10T16:05:19.877349+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877269+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:05:19.877269+0000","last_peered":"2024-09-10T16:05:19.877269+0000","last_clean":"2024-09-10T16:05:19.877269+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:05:19.877269+0000","last_undegraded":"2024-09-10T16:05:19.877269+0000","last_fullsized":"2024-09-10T16:05:19.877269+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'16599","reported_seq":16749,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:04.745577+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:06:04.745577+0000","last_peered":"2024-09-10T16:06:04.745577+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:06:04.745577+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"43'6500","ondisk_log_start":"43'6500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18838470,"num_objects":4601,"num_object_clones":0,"num_object_copies":9202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4601,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12762,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":278},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":298627830,"num_objects":72917,"num_object_clones":0,"num_object_copies":145834,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41895,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72917,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":219189,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9494,"num_bytes_recovered":33095680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":809799680,"data_stored":809657855,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160915,"ondisk_log_size":160915,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986273,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":617556,"kb_used_data":283204,"kb_used_omap":4,"kb_used_meta":334331,"kb_avail":93754284,"statfs":{"total":96636764160,"available":96004386816,"internally_reserved":0,"allocated":290000896,"data_stored":289698212,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":342355626},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,3],"upper_bound":32},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659218,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":561944,"kb_used_data":223424,"kb_used_omap":4,"kb_used_meta":338491,"kb_avail":93809896,"statfs":{"total":96636764160,"available":96061333504,"internally_reserved":0,"allocated":228786176,"data_stored":228454124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":346615470},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,2,5,4,2,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182501,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":725464,"kb_used_data":288452,"kb_used_omap":4,"kb_used_meta":436987,"kb_avail":93646376,"statfs":{"total":96636764160,"available":95893889024,"internally_reserved":0,"allocated":295374848,"data_stored":295072159,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":447475370},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,4,3,1,2,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":293666816,"data_stored":293630686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":227086336,"data_stored":227020843,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":289046528,"data_stored":289006326,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:06:06.680 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-10T16:06:06.681 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "47'16182", 'reported_seq': 16261, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:06:04.745686+0000', 'last_change': '2024-09-10T15:59:16.281558+0000', 'last_active': '2024-09-10T16:06:04.745686+0000', 'last_peered': '2024-09-10T16:06:04.745686+0000', 'last_clean': '2024-09-10T15:59:14.132456+0000', 'last_became_active': '2024-09-10T15:59:16.274407+0000', 'last_became_peered': '2024-09-10T15:59:16.274407+0000', 'last_unstale': '2024-09-10T16:06:04.745686+0000', 'last_undegraded': '2024-09-10T15:59:15.851279+0000', 'last_fullsized': '2024-09-10T15:59:15.774356+0000', 'mapping_epoch': 42, 'log_start': "43'6100", 'ondisk_log_start': "43'6100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'objects_scrubbed': 0, 'log_size': 10082, 'log_dups_size': 0, 'ondisk_log_size': 10082, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18391040, 'num_objects': 4490, 'num_object_clones': 0, 'num_object_copies': 8980, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4259, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4490, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16182, 'num_write_kb': 41344, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 231}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:06:06.681 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-10T16:06:06.681 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "47'16531", 'reported_seq': 16671, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:06:04.753895+0000', 'last_change': '2024-09-10T15:59:16.308264+0000', 'last_active': '2024-09-10T16:06:04.753895+0000', 'last_peered': '2024-09-10T16:06:04.753895+0000', 'last_clean': '2024-09-10T15:59:14.111551+0000', 'last_became_active': '2024-09-10T15:59:16.300040+0000', 'last_became_peered': '2024-09-10T15:59:16.300040+0000', 'last_unstale': '2024-09-10T16:06:04.753895+0000', 'last_undegraded': '2024-09-10T15:59:15.878234+0000', 'last_fullsized': '2024-09-10T15:59:15.769328+0000', 'mapping_epoch': 42, 'log_start': "43'6500", 'ondisk_log_start': "43'6500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3442", 'last_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'objects_scrubbed': 3384, 'log_size': 10031, 'log_dups_size': 0, 'ondisk_log_size': 10031, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1292, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18555363, 'num_objects': 4531, 'num_object_clones': 0, 'num_object_copies': 9062, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4372, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4531, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 13143, 'num_write_kb': 28580, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 160}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:06:06.681 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-10T16:06:06.681 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "47'16383", 'reported_seq': 16525, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:06:04.736127+0000', 'last_change': '2024-09-10T15:59:16.331899+0000', 'last_active': '2024-09-10T16:06:04.736127+0000', 'last_peered': '2024-09-10T16:06:04.736127+0000', 'last_clean': '2024-09-10T15:59:14.139436+0000', 'last_became_active': '2024-09-10T15:59:16.318715+0000', 'last_became_peered': '2024-09-10T15:59:16.318715+0000', 'last_unstale': '2024-09-10T16:06:04.736127+0000', 'last_undegraded': '2024-09-10T15:59:15.877524+0000', 'last_fullsized': '2024-09-10T15:59:15.769042+0000', 'mapping_epoch': 42, 'log_start': "43'6300", 'ondisk_log_start': "43'6300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3537", 'last_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'objects_scrubbed': 3497, 'log_size': 10083, 'log_dups_size': 0, 'ondisk_log_size': 10083, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1224, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19248070, 'num_objects': 4701, 'num_object_clones': 0, 'num_object_copies': 9402, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4237, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4701, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 12882, 'num_write_kb': 28172, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 464}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:06:06.681 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-10T16:06:06.681 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "47'15914", 'reported_seq': 16059, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:06:05.540627+0000', 'last_change': '2024-09-10T15:59:16.175680+0000', 'last_active': '2024-09-10T16:06:05.540627+0000', 'last_peered': '2024-09-10T16:06:05.540627+0000', 'last_clean': '2024-09-10T15:59:14.088187+0000', 'last_became_active': '2024-09-10T15:59:16.165810+0000', 'last_became_peered': '2024-09-10T15:59:16.165810+0000', 'last_unstale': '2024-09-10T16:06:05.540627+0000', 'last_undegraded': '2024-09-10T15:59:15.822098+0000', 'last_fullsized': '2024-09-10T15:59:15.769123+0000', 'mapping_epoch': 42, 'log_start': "43'5900", 'ondisk_log_start': "43'5900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2218", 'last_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'objects_scrubbed': 2175, 'log_size': 10014, 'log_dups_size': 0, 'ondisk_log_size': 10014, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 825, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18608611, 'num_objects': 4544, 'num_object_clones': 0, 'num_object_copies': 9088, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4124, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4544, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 13733, 'num_write_kb': 32204, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': 421}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:06:06.681 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-10T16:06:06.681 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "47'16263", 'reported_seq': 16399, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:06:04.696844+0000', 'last_change': '2024-09-10T15:59:16.277742+0000', 'last_active': '2024-09-10T16:06:04.696844+0000', 'last_peered': '2024-09-10T16:06:04.696844+0000', 'last_clean': '2024-09-10T15:59:14.129081+0000', 'last_became_active': '2024-09-10T15:59:16.269835+0000', 'last_became_peered': '2024-09-10T15:59:16.269835+0000', 'last_unstale': '2024-09-10T16:06:04.696844+0000', 'last_undegraded': '2024-09-10T15:59:15.860857+0000', 'last_fullsized': '2024-09-10T15:59:15.771410+0000', 'mapping_epoch': 42, 'log_start': "43'6200", 'ondisk_log_start': "43'6200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3557", 'last_scrub_stamp': '2024-09-10T15:59:02.493678+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:02.493678+0000', 'objects_scrubbed': 3488, 'log_size': 10063, 'log_dups_size': 0, 'ondisk_log_size': 10063, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1472, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19034112, 'num_objects': 4647, 'num_object_clones': 0, 'num_object_copies': 9294, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4257, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4647, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12767, 'num_write_kb': 27852, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 392}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:06:06.681 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-10T16:06:06.682 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "47'16288", 'reported_seq': 16428, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:06:05.516035+0000', 'last_change': '2024-09-10T15:59:16.284311+0000', 'last_active': '2024-09-10T16:06:05.516035+0000', 'last_peered': '2024-09-10T16:06:05.516035+0000', 'last_clean': '2024-09-10T15:59:14.116041+0000', 'last_became_active': '2024-09-10T15:59:16.274543+0000', 'last_became_peered': '2024-09-10T15:59:16.274543+0000', 'last_unstale': '2024-09-10T16:06:05.516035+0000', 'last_undegraded': '2024-09-10T15:59:15.812484+0000', 'last_fullsized': '2024-09-10T15:59:15.770885+0000', 'mapping_epoch': 42, 'log_start': "43'6200", 'ondisk_log_start': "43'6200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2356", 'last_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'objects_scrubbed': 2316, 'log_size': 10088, 'log_dups_size': 0, 'ondisk_log_size': 10088, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 783, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18403811, 'num_objects': 4494, 'num_object_clones': 0, 'num_object_copies': 8988, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4494, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 13968, 'num_write_kb': 32292, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': 190}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:06:06.682 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-10T16:06:06.682 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "47'16589", 'reported_seq': 18754, 'reported_epoch': 47, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:06:05.556952+0000', 'last_change': '2024-09-10T16:05:18.271152+0000', 'last_active': '2024-09-10T16:06:05.556952+0000', 'last_peered': '2024-09-10T16:06:05.556952+0000', 'last_clean': '2024-09-10T15:59:14.110654+0000', 'last_became_active': '2024-09-10T15:59:16.164645+0000', 'last_became_peered': '2024-09-10T15:59:16.164645+0000', 'last_unstale': '2024-09-10T16:06:05.556952+0000', 'last_undegraded': '2024-09-10T15:59:15.803593+0000', 'last_fullsized': '2024-09-10T15:59:15.769083+0000', 'mapping_epoch': 42, 'log_start': "43'6500", 'ondisk_log_start': "43'6500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2486", 'last_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'objects_scrubbed': 2440, 'log_size': 10089, 'log_dups_size': 0, 'ondisk_log_size': 10089, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 792, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18763776, 'num_objects': 4581, 'num_object_clones': 0, 'num_object_copies': 9162, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3338, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4581, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14143, 'num_write_kb': 32568, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1016, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 1243}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:06:06.682 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-10T16:06:06.682 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "47'16329", 'reported_seq': 16469, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:06:04.665681+0000', 'last_change': '2024-09-10T15:59:16.315641+0000', 'last_active': '2024-09-10T16:06:04.665681+0000', 'last_peered': '2024-09-10T16:06:04.665681+0000', 'last_clean': '2024-09-10T15:59:14.117001+0000', 'last_became_active': '2024-09-10T15:59:16.302245+0000', 'last_became_peered': '2024-09-10T15:59:16.302245+0000', 'last_unstale': '2024-09-10T16:06:04.665681+0000', 'last_undegraded': '2024-09-10T15:59:15.845012+0000', 'last_fullsized': '2024-09-10T15:59:15.770337+0000', 'mapping_epoch': 42, 'log_start': "43'6300", 'ondisk_log_start': "43'6300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3404", 'last_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'objects_scrubbed': 3367, 'log_size': 10029, 'log_dups_size': 0, 'ondisk_log_size': 10029, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 983, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18468864, 'num_objects': 4509, 'num_object_clones': 0, 'num_object_copies': 9018, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4509, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12960, 'num_write_kb': 28204, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 205}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:06:06.682 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-10T16:06:06.682 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "47'16542", 'reported_seq': 16689, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:06:04.754167+0000', 'last_change': '2024-09-10T15:59:16.311291+0000', 'last_active': '2024-09-10T16:06:04.754167+0000', 'last_peered': '2024-09-10T16:06:04.754167+0000', 'last_clean': '2024-09-10T15:59:14.116363+0000', 'last_became_active': '2024-09-10T15:59:16.301244+0000', 'last_became_peered': '2024-09-10T15:59:16.301244+0000', 'last_unstale': '2024-09-10T16:06:04.754167+0000', 'last_undegraded': '2024-09-10T15:59:15.883434+0000', 'last_fullsized': '2024-09-10T15:59:15.772756+0000', 'mapping_epoch': 42, 'log_start': "43'6500", 'ondisk_log_start': "43'6500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3789", 'last_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'objects_scrubbed': 3741, 'log_size': 10042, 'log_dups_size': 0, 'ondisk_log_size': 10042, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1440, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18456576, 'num_objects': 4506, 'num_object_clones': 0, 'num_object_copies': 9012, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4373, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4506, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12797, 'num_write_kb': 27124, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 135}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:06:06.682 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-10T16:06:06.682 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "47'16599", 'reported_seq': 16749, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:06:04.745577+0000', 'last_change': '2024-09-10T15:59:16.313409+0000', 'last_active': '2024-09-10T16:06:04.745577+0000', 'last_peered': '2024-09-10T16:06:04.745577+0000', 'last_clean': '2024-09-10T15:59:14.129780+0000', 'last_became_active': '2024-09-10T15:59:16.300932+0000', 'last_became_peered': '2024-09-10T15:59:16.300932+0000', 'last_unstale': '2024-09-10T16:06:04.745577+0000', 'last_undegraded': '2024-09-10T15:59:15.887823+0000', 'last_fullsized': '2024-09-10T15:59:15.768513+0000', 'mapping_epoch': 42, 'log_start': "43'6500", 'ondisk_log_start': "43'6500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3880", 'last_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'objects_scrubbed': 3827, 'log_size': 10099, 'log_dups_size': 0, 'ondisk_log_size': 10099, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1397, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18838470, 'num_objects': 4601, 'num_object_clones': 0, 'num_object_copies': 9202, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4325, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4601, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12762, 'num_write_kb': 27072, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 278}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:06:06.683 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T16:06:07.369 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:06:07.370 INFO:teuthology.orchestra.run.smithi115.stdout:{"fsid":"7effee29-df05-4351-9741-ee203564ed81","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 41895/145838 objects degraded (28.727%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":531,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":47,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983954,"num_in_osds":3,"osd_in_since":1725983947,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":72919,"data_bytes":298627849,"bytes_used":1950683136,"bytes_avail":287959609344,"bytes_total":289910292480,"degraded_objects":41895,"degraded_total":145838,"degraded_ratio":0.28727080733416532,"recovering_objects_per_sec":26,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":320,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":555},"fsmap":{"epoch":1,"btime":"2024-09-10T15:57:14:967120+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T16:00:13.560776+0000","services":{}},"progress_events":{}} 2024-09-10T16:06:07.408 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-10T16:06:07.408 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:06:07.955 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:06:07.957 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:06:07.977 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":306,"stamp":"2024-09-10T16:06:07.701338+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":298627849,"num_objects":72919,"num_object_clones":0,"num_object_copies":145838,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41895,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72919,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":219191,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9496,"num_bytes_recovered":33095699,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160918,"ondisk_log_size":160918,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1904964,"kb_used_data":795080,"kb_used_omap":12,"kb_used_meta":1109811,"kb_avail":281210556,"statfs":{"total":289910292480,"available":287959609344,"internally_reserved":0,"allocated":814161920,"data_stored":813224495,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1136446466},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,5,7,9,6,7,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18919424,"num_objects":-4619,"num_object_clones":0,"num_object_copies":-9238,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-215,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4619,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4619,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":215,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004272"},"pg_stats":[{"pgid":"2.f","version":"47'16182","reported_seq":16261,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:04.745686+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:06:04.745686+0000","last_peered":"2024-09-10T16:06:04.745686+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:06:04.745686+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"43'6100","ondisk_log_start":"43'6100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18391040,"num_objects":4490,"num_object_clones":0,"num_object_copies":8980,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4490,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16182,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":231}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'16312","reported_seq":16430,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:06:05.530234+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:06:05.530234+0000","last_peered":"2024-09-10T16:06:05.530234+0000","last_clean":"2024-09-10T16:06:05.530234+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:06:05.530234+0000","last_undegraded":"2024-09-10T16:06:05.530234+0000","last_fullsized":"2024-09-10T16:06:05.530234+0000","mapping_epoch":35,"log_start":"43'6300","ondisk_log_start":"43'6300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18564038,"num_objects":4534,"num_object_clones":0,"num_object_copies":9068,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4534,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":14417,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'16160","reported_seq":24843,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:06:02.912813+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:06:02.912813+0000","last_peered":"2024-09-10T16:06:02.912813+0000","last_clean":"2024-09-10T16:06:02.912813+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:06:02.912813+0000","last_undegraded":"2024-09-10T16:06:02.912813+0000","last_fullsized":"2024-09-10T16:06:02.912813+0000","mapping_epoch":44,"log_start":"43'6100","ondisk_log_start":"43'6100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18993176,"num_objects":4638,"num_object_clones":0,"num_object_copies":9276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4638,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14164,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'16258","reported_seq":16343,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:06:05.537069+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:06:05.537069+0000","last_peered":"2024-09-10T16:06:05.537069+0000","last_clean":"2024-09-10T16:06:05.537069+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:06:05.537069+0000","last_undegraded":"2024-09-10T16:06:05.537069+0000","last_fullsized":"2024-09-10T16:06:05.537069+0000","mapping_epoch":35,"log_start":"43'6200","ondisk_log_start":"43'6200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18030592,"num_objects":4402,"num_object_clones":0,"num_object_copies":8804,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4402,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16258,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'16357","reported_seq":16491,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:06:04.743046+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:06:04.743046+0000","last_peered":"2024-09-10T16:06:04.743046+0000","last_clean":"2024-09-10T16:06:04.743046+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:06:04.743046+0000","last_undegraded":"2024-09-10T16:06:04.743046+0000","last_fullsized":"2024-09-10T16:06:04.743046+0000","mapping_epoch":34,"log_start":"43'6300","ondisk_log_start":"43'6300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18571747,"num_objects":4535,"num_object_clones":0,"num_object_copies":9070,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4535,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13068,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'16531","reported_seq":16671,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:04.753895+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:06:04.753895+0000","last_peered":"2024-09-10T16:06:04.753895+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:06:04.753895+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"43'6500","ondisk_log_start":"43'6500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18555363,"num_objects":4531,"num_object_clones":0,"num_object_copies":9062,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4531,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13143,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":160},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'16383","reported_seq":16525,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:04.736127+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:06:04.736127+0000","last_peered":"2024-09-10T16:06:04.736127+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:06:04.736127+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"43'6300","ondisk_log_start":"43'6300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19248070,"num_objects":4701,"num_object_clones":0,"num_object_copies":9402,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4701,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":12882,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":464},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'16664","reported_seq":16799,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:06:04.748781+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:06:04.748781+0000","last_peered":"2024-09-10T16:06:04.748781+0000","last_clean":"2024-09-10T16:06:04.748781+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:06:04.748781+0000","last_undegraded":"2024-09-10T16:06:04.748781+0000","last_fullsized":"2024-09-10T16:06:04.748781+0000","mapping_epoch":34,"log_start":"43'6600","ondisk_log_start":"43'6600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19046400,"num_objects":4650,"num_object_clones":0,"num_object_copies":9300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4650,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13511,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.876895+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:05:19.876895+0000","last_peered":"2024-09-10T16:05:19.876895+0000","last_clean":"2024-09-10T16:05:19.876895+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:05:19.876895+0000","last_undegraded":"2024-09-10T16:05:19.876895+0000","last_fullsized":"2024-09-10T16:05:19.876895+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'15914","reported_seq":16059,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:05.540627+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:06:05.540627+0000","last_peered":"2024-09-10T16:06:05.540627+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:06:05.540627+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"43'5900","ondisk_log_start":"43'5900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18608611,"num_objects":4544,"num_object_clones":0,"num_object_copies":9088,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4544,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13733,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":421}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":99,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.875776+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:05:19.875776+0000","last_peered":"2024-09-10T16:05:19.875776+0000","last_clean":"2024-09-10T16:05:19.875776+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:05:19.875776+0000","last_undegraded":"2024-09-10T16:05:19.875776+0000","last_fullsized":"2024-09-10T16:05:19.875776+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'16263","reported_seq":16399,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:04.696844+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:06:04.696844+0000","last_peered":"2024-09-10T16:06:04.696844+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:06:04.696844+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"43'6200","ondisk_log_start":"43'6200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19034112,"num_objects":4647,"num_object_clones":0,"num_object_copies":9294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4647,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12767,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":392}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":109,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.871983+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:05:19.871983+0000","last_peered":"2024-09-10T16:05:19.871983+0000","last_clean":"2024-09-10T16:05:19.871983+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:05:19.871983+0000","last_undegraded":"2024-09-10T16:05:19.871983+0000","last_fullsized":"2024-09-10T16:05:19.871983+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'16288","reported_seq":16428,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:05.516035+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:06:05.516035+0000","last_peered":"2024-09-10T16:06:05.516035+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:06:05.516035+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"43'6200","ondisk_log_start":"43'6200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18403811,"num_objects":4494,"num_object_clones":0,"num_object_copies":8988,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4494,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13968,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":190}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877148+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:05:19.877148+0000","last_peered":"2024-09-10T16:05:19.877148+0000","last_clean":"2024-09-10T16:05:19.877148+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:05:19.877148+0000","last_undegraded":"2024-09-10T16:05:19.877148+0000","last_fullsized":"2024-09-10T16:05:19.877148+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'16589","reported_seq":18754,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:05.556952+0000","last_change":"2024-09-10T16:05:18.271152+0000","last_active":"2024-09-10T16:06:05.556952+0000","last_peered":"2024-09-10T16:06:05.556952+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:06:05.556952+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"43'6500","ondisk_log_start":"43'6500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18763776,"num_objects":4581,"num_object_clones":0,"num_object_copies":9162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3338,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4581,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14143,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1016,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1243}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'16329","reported_seq":16469,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:04.665681+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:06:04.665681+0000","last_peered":"2024-09-10T16:06:04.665681+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:06:04.665681+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"43'6300","ondisk_log_start":"43'6300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18468864,"num_objects":4509,"num_object_clones":0,"num_object_copies":9018,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4509,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12960,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":205}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877654+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:05:19.877654+0000","last_peered":"2024-09-10T16:05:19.877654+0000","last_clean":"2024-09-10T16:05:19.877654+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:05:19.877654+0000","last_undegraded":"2024-09-10T16:05:19.877654+0000","last_fullsized":"2024-09-10T16:05:19.877654+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'16044","reported_seq":24615,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:06:02.893799+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:06:02.893799+0000","last_peered":"2024-09-10T16:06:02.893799+0000","last_clean":"2024-09-10T16:06:02.893799+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:06:02.893799+0000","last_undegraded":"2024-09-10T16:06:02.893799+0000","last_fullsized":"2024-09-10T16:06:02.893799+0000","mapping_epoch":46,"log_start":"43'6000","ondisk_log_start":"43'6000","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18653184,"num_objects":4554,"num_object_clones":0,"num_object_copies":9108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4554,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12434,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":99,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.876013+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:05:19.876013+0000","last_peered":"2024-09-10T16:05:19.876013+0000","last_clean":"2024-09-10T16:05:19.876013+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:05:19.876013+0000","last_undegraded":"2024-09-10T16:05:19.876013+0000","last_fullsized":"2024-09-10T16:05:19.876013+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'16542","reported_seq":16689,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:04.754167+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:06:04.754167+0000","last_peered":"2024-09-10T16:06:04.754167+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:06:04.754167+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"43'6500","ondisk_log_start":"43'6500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18456576,"num_objects":4506,"num_object_clones":0,"num_object_copies":9012,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4506,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12797,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":135}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877349+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:05:19.877349+0000","last_peered":"2024-09-10T16:05:19.877349+0000","last_clean":"2024-09-10T16:05:19.877349+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:05:19.877349+0000","last_undegraded":"2024-09-10T16:05:19.877349+0000","last_fullsized":"2024-09-10T16:05:19.877349+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877269+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:05:19.877269+0000","last_peered":"2024-09-10T16:05:19.877269+0000","last_clean":"2024-09-10T16:05:19.877269+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:05:19.877269+0000","last_undegraded":"2024-09-10T16:05:19.877269+0000","last_fullsized":"2024-09-10T16:05:19.877269+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'16599","reported_seq":16749,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:04.745577+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:06:04.745577+0000","last_peered":"2024-09-10T16:06:04.745577+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:06:04.745577+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"43'6500","ondisk_log_start":"43'6500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18838470,"num_objects":4601,"num_object_clones":0,"num_object_copies":9202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4601,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12762,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":278},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":298627830,"num_objects":72917,"num_object_clones":0,"num_object_copies":145834,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41895,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72917,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":219189,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9494,"num_bytes_recovered":33095680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":809799680,"data_stored":809657855,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160915,"ondisk_log_size":160915,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986273,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":617556,"kb_used_data":283204,"kb_used_omap":4,"kb_used_meta":334331,"kb_avail":93754284,"statfs":{"total":96636764160,"available":96004386816,"internally_reserved":0,"allocated":290000896,"data_stored":289698212,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":342355626},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,3],"upper_bound":32},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659218,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":561944,"kb_used_data":223424,"kb_used_omap":4,"kb_used_meta":338491,"kb_avail":93809896,"statfs":{"total":96636764160,"available":96061333504,"internally_reserved":0,"allocated":228786176,"data_stored":228454124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":346615470},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,2,5,4,2,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182501,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":725464,"kb_used_data":288452,"kb_used_omap":4,"kb_used_meta":436987,"kb_avail":93646376,"statfs":{"total":96636764160,"available":95893889024,"internally_reserved":0,"allocated":295374848,"data_stored":295072159,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":447475370},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,4,3,1,2,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":293666816,"data_stored":293630686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":227086336,"data_stored":227020843,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":289046528,"data_stored":289006326,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:06:10.980 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:06:11.523 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:06:11.524 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:06:11.549 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":307,"stamp":"2024-09-10T16:06:09.702105+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":297464585,"num_objects":72635,"num_object_clones":0,"num_object_copies":145270,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41895,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72635,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":219475,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9496,"num_bytes_recovered":33095699,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160902,"ondisk_log_size":160902,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1905072,"kb_used_data":786876,"kb_used_omap":12,"kb_used_meta":1118131,"kb_avail":281210448,"statfs":{"total":289910292480,"available":287959498752,"internally_reserved":0,"allocated":805761024,"data_stored":804823599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1144966146},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,6,5,10,6,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-20082688,"num_objects":-4903,"num_object_clones":0,"num_object_copies":-9806,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-215,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4903,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4903,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":215,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004354"},"pg_stats":[{"pgid":"2.f","version":"47'16182","reported_seq":16261,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:04.745686+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:06:04.745686+0000","last_peered":"2024-09-10T16:06:04.745686+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:06:04.745686+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"43'6100","ondisk_log_start":"43'6100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18391040,"num_objects":4490,"num_object_clones":0,"num_object_copies":8980,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4490,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16182,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":231}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'16312","reported_seq":16430,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:06:05.530234+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:06:05.530234+0000","last_peered":"2024-09-10T16:06:05.530234+0000","last_clean":"2024-09-10T16:06:05.530234+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:06:05.530234+0000","last_undegraded":"2024-09-10T16:06:05.530234+0000","last_fullsized":"2024-09-10T16:06:05.530234+0000","mapping_epoch":35,"log_start":"43'6300","ondisk_log_start":"43'6300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18564038,"num_objects":4534,"num_object_clones":0,"num_object_copies":9068,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4534,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":14417,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'16312","reported_seq":24994,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:06:07.965473+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:06:07.965473+0000","last_peered":"2024-09-10T16:06:07.965473+0000","last_clean":"2024-09-10T16:06:07.965473+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:06:07.965473+0000","last_undegraded":"2024-09-10T16:06:07.965473+0000","last_fullsized":"2024-09-10T16:06:07.965473+0000","mapping_epoch":44,"log_start":"43'6300","ondisk_log_start":"43'6300","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18370584,"num_objects":4486,"num_object_clones":0,"num_object_copies":8972,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4486,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14316,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'16258","reported_seq":16343,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:06:05.537069+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:06:05.537069+0000","last_peered":"2024-09-10T16:06:05.537069+0000","last_clean":"2024-09-10T16:06:05.537069+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:06:05.537069+0000","last_undegraded":"2024-09-10T16:06:05.537069+0000","last_fullsized":"2024-09-10T16:06:05.537069+0000","mapping_epoch":35,"log_start":"43'6200","ondisk_log_start":"43'6200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18030592,"num_objects":4402,"num_object_clones":0,"num_object_copies":8804,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4402,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16258,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'16357","reported_seq":16491,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:06:04.743046+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:06:04.743046+0000","last_peered":"2024-09-10T16:06:04.743046+0000","last_clean":"2024-09-10T16:06:04.743046+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:06:04.743046+0000","last_undegraded":"2024-09-10T16:06:04.743046+0000","last_fullsized":"2024-09-10T16:06:04.743046+0000","mapping_epoch":34,"log_start":"43'6300","ondisk_log_start":"43'6300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18571747,"num_objects":4535,"num_object_clones":0,"num_object_copies":9070,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4535,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13068,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'16531","reported_seq":16671,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:04.753895+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:06:04.753895+0000","last_peered":"2024-09-10T16:06:04.753895+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:06:04.753895+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"43'6500","ondisk_log_start":"43'6500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18555363,"num_objects":4531,"num_object_clones":0,"num_object_copies":9062,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4531,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13143,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":160},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'16383","reported_seq":16525,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:04.736127+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:06:04.736127+0000","last_peered":"2024-09-10T16:06:04.736127+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:06:04.736127+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"43'6300","ondisk_log_start":"43'6300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19248070,"num_objects":4701,"num_object_clones":0,"num_object_copies":9402,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4701,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":12882,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":464},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'16664","reported_seq":16799,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:06:04.748781+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:06:04.748781+0000","last_peered":"2024-09-10T16:06:04.748781+0000","last_clean":"2024-09-10T16:06:04.748781+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:06:04.748781+0000","last_undegraded":"2024-09-10T16:06:04.748781+0000","last_fullsized":"2024-09-10T16:06:04.748781+0000","mapping_epoch":34,"log_start":"43'6600","ondisk_log_start":"43'6600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19046400,"num_objects":4650,"num_object_clones":0,"num_object_copies":9300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4650,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13511,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.876895+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:05:19.876895+0000","last_peered":"2024-09-10T16:05:19.876895+0000","last_clean":"2024-09-10T16:05:19.876895+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:05:19.876895+0000","last_undegraded":"2024-09-10T16:05:19.876895+0000","last_fullsized":"2024-09-10T16:05:19.876895+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'15914","reported_seq":16059,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:05.540627+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:06:05.540627+0000","last_peered":"2024-09-10T16:06:05.540627+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:06:05.540627+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"43'5900","ondisk_log_start":"43'5900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18608611,"num_objects":4544,"num_object_clones":0,"num_object_copies":9088,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4544,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13733,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":421}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":99,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.875776+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:05:19.875776+0000","last_peered":"2024-09-10T16:05:19.875776+0000","last_clean":"2024-09-10T16:05:19.875776+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:05:19.875776+0000","last_undegraded":"2024-09-10T16:05:19.875776+0000","last_fullsized":"2024-09-10T16:05:19.875776+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'16263","reported_seq":16399,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:04.696844+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:06:04.696844+0000","last_peered":"2024-09-10T16:06:04.696844+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:06:04.696844+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"43'6200","ondisk_log_start":"43'6200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19034112,"num_objects":4647,"num_object_clones":0,"num_object_copies":9294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4647,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12767,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":392}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":109,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.871983+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:05:19.871983+0000","last_peered":"2024-09-10T16:05:19.871983+0000","last_clean":"2024-09-10T16:05:19.871983+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:05:19.871983+0000","last_undegraded":"2024-09-10T16:05:19.871983+0000","last_fullsized":"2024-09-10T16:05:19.871983+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'16288","reported_seq":16428,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:05.516035+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:06:05.516035+0000","last_peered":"2024-09-10T16:06:05.516035+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:06:05.516035+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"43'6200","ondisk_log_start":"43'6200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18403811,"num_objects":4494,"num_object_clones":0,"num_object_copies":8988,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4494,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13968,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":190}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877148+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:05:19.877148+0000","last_peered":"2024-09-10T16:05:19.877148+0000","last_clean":"2024-09-10T16:05:19.877148+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:05:19.877148+0000","last_undegraded":"2024-09-10T16:05:19.877148+0000","last_fullsized":"2024-09-10T16:05:19.877148+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'16589","reported_seq":18754,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:05.556952+0000","last_change":"2024-09-10T16:05:18.271152+0000","last_active":"2024-09-10T16:06:05.556952+0000","last_peered":"2024-09-10T16:06:05.556952+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:06:05.556952+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"43'6500","ondisk_log_start":"43'6500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18763776,"num_objects":4581,"num_object_clones":0,"num_object_copies":9162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3338,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4581,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14143,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1016,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1243}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'16329","reported_seq":16469,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:04.665681+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:06:04.665681+0000","last_peered":"2024-09-10T16:06:04.665681+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:06:04.665681+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"43'6300","ondisk_log_start":"43'6300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18468864,"num_objects":4509,"num_object_clones":0,"num_object_copies":9018,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4509,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12960,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":205}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877654+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:05:19.877654+0000","last_peered":"2024-09-10T16:05:19.877654+0000","last_clean":"2024-09-10T16:05:19.877654+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:05:19.877654+0000","last_undegraded":"2024-09-10T16:05:19.877654+0000","last_fullsized":"2024-09-10T16:05:19.877654+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'16176","reported_seq":24747,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:06:07.903268+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:06:07.903268+0000","last_peered":"2024-09-10T16:06:07.903268+0000","last_clean":"2024-09-10T16:06:07.903268+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:06:07.903268+0000","last_undegraded":"2024-09-10T16:06:07.903268+0000","last_fullsized":"2024-09-10T16:06:07.903268+0000","mapping_epoch":46,"log_start":"43'6100","ondisk_log_start":"43'6100","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18112512,"num_objects":4422,"num_object_clones":0,"num_object_copies":8844,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4422,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12566,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":99,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.876013+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:05:19.876013+0000","last_peered":"2024-09-10T16:05:19.876013+0000","last_clean":"2024-09-10T16:05:19.876013+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:05:19.876013+0000","last_undegraded":"2024-09-10T16:05:19.876013+0000","last_fullsized":"2024-09-10T16:05:19.876013+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'16542","reported_seq":16689,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:04.754167+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:06:04.754167+0000","last_peered":"2024-09-10T16:06:04.754167+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:06:04.754167+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"43'6500","ondisk_log_start":"43'6500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18456576,"num_objects":4506,"num_object_clones":0,"num_object_copies":9012,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4506,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12797,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":135}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877349+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:05:19.877349+0000","last_peered":"2024-09-10T16:05:19.877349+0000","last_clean":"2024-09-10T16:05:19.877349+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:05:19.877349+0000","last_undegraded":"2024-09-10T16:05:19.877349+0000","last_fullsized":"2024-09-10T16:05:19.877349+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877269+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:05:19.877269+0000","last_peered":"2024-09-10T16:05:19.877269+0000","last_clean":"2024-09-10T16:05:19.877269+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:05:19.877269+0000","last_undegraded":"2024-09-10T16:05:19.877269+0000","last_fullsized":"2024-09-10T16:05:19.877269+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'16599","reported_seq":16749,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:04.745577+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:06:04.745577+0000","last_peered":"2024-09-10T16:06:04.745577+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:06:04.745577+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"43'6500","ondisk_log_start":"43'6500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18838470,"num_objects":4601,"num_object_clones":0,"num_object_copies":9202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4601,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12762,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":278},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":297464566,"num_objects":72633,"num_object_clones":0,"num_object_copies":145266,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41895,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72633,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":219473,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9494,"num_bytes_recovered":33095680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":802668544,"data_stored":802526719,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160899,"ondisk_log_size":160899,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986273,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":617556,"kb_used_data":283204,"kb_used_omap":4,"kb_used_meta":334331,"kb_avail":93754284,"statfs":{"total":96636764160,"available":96004386816,"internally_reserved":0,"allocated":290000896,"data_stored":289698212,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":342355626},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,3],"upper_bound":32},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659219,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":562052,"kb_used_data":215220,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93809788,"statfs":{"total":96636764160,"available":96061222912,"internally_reserved":0,"allocated":220385280,"data_stored":220053228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":355135150},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,6,4],"upper_bound":64},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182501,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":725464,"kb_used_data":288452,"kb_used_omap":4,"kb_used_meta":436987,"kb_avail":93646376,"statfs":{"total":96636764160,"available":95893889024,"internally_reserved":0,"allocated":295374848,"data_stored":295072159,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":447475370},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,4,3,1,2,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":293666816,"data_stored":293630686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":219955200,"data_stored":219889707,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":289046528,"data_stored":289006326,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:06:11.552 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:06:12.104 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:06:12.105 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:06:12.128 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":308,"stamp":"2024-09-10T16:06:11.702926+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":289248009,"num_objects":70629,"num_object_clones":0,"num_object_copies":141258,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41790,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70629,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":221481,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9601,"num_bytes_recovered":33095699,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160908,"ondisk_log_size":160908,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1889408,"kb_used_data":775368,"kb_used_omap":12,"kb_used_meta":1113971,"kb_avail":281226112,"statfs":{"total":289910292480,"available":287975538688,"internally_reserved":0,"allocated":793976832,"data_stored":793039407,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1140706306},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,8,9,8,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-27181056,"num_objects":-6636,"num_object_clones":0,"num_object_copies":-13272,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-6636,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6636,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":320,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004584"},"pg_stats":[{"pgid":"2.f","version":"47'16320","reported_seq":16400,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:09.736133+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:06:09.736133+0000","last_peered":"2024-09-10T16:06:09.736133+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:06:09.736133+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"43'6300","ondisk_log_start":"43'6300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17825792,"num_objects":4352,"num_object_clones":0,"num_object_copies":8704,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4352,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16320,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":93}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'16449","reported_seq":16567,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:06:10.376911+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:06:10.376911+0000","last_peered":"2024-09-10T16:06:10.376911+0000","last_clean":"2024-09-10T16:06:10.376911+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:06:10.376911+0000","last_undegraded":"2024-09-10T16:06:10.376911+0000","last_fullsized":"2024-09-10T16:06:10.376911+0000","mapping_epoch":35,"log_start":"43'6400","ondisk_log_start":"43'6400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18002886,"num_objects":4397,"num_object_clones":0,"num_object_copies":8794,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4397,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":14554,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'16312","reported_seq":24994,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:06:07.965473+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:06:07.965473+0000","last_peered":"2024-09-10T16:06:07.965473+0000","last_clean":"2024-09-10T16:06:07.965473+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:06:07.965473+0000","last_undegraded":"2024-09-10T16:06:07.965473+0000","last_fullsized":"2024-09-10T16:06:07.965473+0000","mapping_epoch":44,"log_start":"43'6300","ondisk_log_start":"43'6300","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18370584,"num_objects":4486,"num_object_clones":0,"num_object_copies":8972,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4486,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14316,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'16397","reported_seq":16483,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:06:10.502594+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:06:10.502594+0000","last_peered":"2024-09-10T16:06:10.502594+0000","last_clean":"2024-09-10T16:06:10.502594+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:06:10.502594+0000","last_undegraded":"2024-09-10T16:06:10.502594+0000","last_fullsized":"2024-09-10T16:06:10.502594+0000","mapping_epoch":35,"log_start":"43'6300","ondisk_log_start":"43'6300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17461248,"num_objects":4263,"num_object_clones":0,"num_object_copies":8526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4263,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16397,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'16500","reported_seq":16635,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:06:09.785674+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:06:09.785674+0000","last_peered":"2024-09-10T16:06:09.785674+0000","last_clean":"2024-09-10T16:06:09.785674+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:06:09.785674+0000","last_undegraded":"2024-09-10T16:06:09.785674+0000","last_fullsized":"2024-09-10T16:06:09.785674+0000","mapping_epoch":34,"log_start":"43'6400","ondisk_log_start":"43'6400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17986019,"num_objects":4392,"num_object_clones":0,"num_object_copies":8784,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4392,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13211,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'16681","reported_seq":16824,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:09.779805+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:06:09.779805+0000","last_peered":"2024-09-10T16:06:09.779805+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:06:09.779805+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"43'6600","ondisk_log_start":"43'6600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17940963,"num_objects":4381,"num_object_clones":0,"num_object_copies":8762,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4381,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13293,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":10},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'16510","reported_seq":16653,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:09.733574+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:06:09.733574+0000","last_peered":"2024-09-10T16:06:09.733574+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:06:09.733574+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"43'6500","ondisk_log_start":"43'6500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18727878,"num_objects":4574,"num_object_clones":0,"num_object_copies":9148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4574,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13009,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":337},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'16784","reported_seq":16919,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:06:09.718425+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:06:09.718425+0000","last_peered":"2024-09-10T16:06:09.718425+0000","last_clean":"2024-09-10T16:06:09.718425+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:06:09.718425+0000","last_undegraded":"2024-09-10T16:06:09.718425+0000","last_fullsized":"2024-09-10T16:06:09.718425+0000","mapping_epoch":34,"log_start":"43'6700","ondisk_log_start":"43'6700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18554880,"num_objects":4530,"num_object_clones":0,"num_object_copies":9060,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4530,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13631,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.876895+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:05:19.876895+0000","last_peered":"2024-09-10T16:05:19.876895+0000","last_clean":"2024-09-10T16:05:19.876895+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:05:19.876895+0000","last_undegraded":"2024-09-10T16:05:19.876895+0000","last_fullsized":"2024-09-10T16:05:19.876895+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'16055","reported_seq":16198,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:10.515251+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:06:10.515251+0000","last_peered":"2024-09-10T16:06:10.515251+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:06:10.515251+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"43'6000","ondisk_log_start":"43'6000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18031075,"num_objects":4403,"num_object_clones":0,"num_object_copies":8806,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4403,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13874,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":280}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":99,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.875776+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:05:19.875776+0000","last_peered":"2024-09-10T16:05:19.875776+0000","last_clean":"2024-09-10T16:05:19.875776+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:05:19.875776+0000","last_undegraded":"2024-09-10T16:05:19.875776+0000","last_fullsized":"2024-09-10T16:05:19.875776+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'16427","reported_seq":16559,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:09.800606+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:06:09.800606+0000","last_peered":"2024-09-10T16:06:09.800606+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:06:09.800606+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"43'6400","ondisk_log_start":"43'6400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18362368,"num_objects":4483,"num_object_clones":0,"num_object_copies":8966,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4483,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12931,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":228}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":109,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.871983+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:05:19.871983+0000","last_peered":"2024-09-10T16:05:19.871983+0000","last_clean":"2024-09-10T16:05:19.871983+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:05:19.871983+0000","last_undegraded":"2024-09-10T16:05:19.871983+0000","last_fullsized":"2024-09-10T16:05:19.871983+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'16428","reported_seq":16568,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:10.542399+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:06:10.542399+0000","last_peered":"2024-09-10T16:06:10.542399+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:06:10.542399+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"43'6400","ondisk_log_start":"43'6400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17830371,"num_objects":4354,"num_object_clones":0,"num_object_copies":8708,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4354,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14108,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":50}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877148+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:05:19.877148+0000","last_peered":"2024-09-10T16:05:19.877148+0000","last_clean":"2024-09-10T16:05:19.877148+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:05:19.877148+0000","last_undegraded":"2024-09-10T16:05:19.877148+0000","last_fullsized":"2024-09-10T16:05:19.877148+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'16740","reported_seq":19117,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:10.556121+0000","last_change":"2024-09-10T16:05:18.271152+0000","last_active":"2024-09-10T16:06:10.556121+0000","last_peered":"2024-09-10T16:06:10.556121+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:06:10.556121+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"43'6700","ondisk_log_start":"43'6700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18145280,"num_objects":4430,"num_object_clones":0,"num_object_copies":8860,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3233,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4430,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14294,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1121,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1197}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'16484","reported_seq":16624,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:09.787441+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:06:09.787441+0000","last_peered":"2024-09-10T16:06:09.787441+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:06:09.787441+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"43'6400","ondisk_log_start":"43'6400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17833984,"num_objects":4354,"num_object_clones":0,"num_object_copies":8708,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4354,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13115,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":50}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877654+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:05:19.877654+0000","last_peered":"2024-09-10T16:05:19.877654+0000","last_clean":"2024-09-10T16:05:19.877654+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:05:19.877654+0000","last_undegraded":"2024-09-10T16:05:19.877654+0000","last_fullsized":"2024-09-10T16:05:19.877654+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'16176","reported_seq":24747,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:06:07.903268+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:06:07.903268+0000","last_peered":"2024-09-10T16:06:07.903268+0000","last_clean":"2024-09-10T16:06:07.903268+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:06:07.903268+0000","last_undegraded":"2024-09-10T16:06:07.903268+0000","last_fullsized":"2024-09-10T16:06:07.903268+0000","mapping_epoch":46,"log_start":"43'6100","ondisk_log_start":"43'6100","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18112512,"num_objects":4422,"num_object_clones":0,"num_object_copies":8844,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4422,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12566,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":99,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.876013+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:05:19.876013+0000","last_peered":"2024-09-10T16:05:19.876013+0000","last_clean":"2024-09-10T16:05:19.876013+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:05:19.876013+0000","last_undegraded":"2024-09-10T16:05:19.876013+0000","last_fullsized":"2024-09-10T16:05:19.876013+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'16694","reported_seq":16841,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:09.744079+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:06:09.744079+0000","last_peered":"2024-09-10T16:06:09.744079+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:06:09.744079+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"43'6600","ondisk_log_start":"43'6600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17833984,"num_objects":4354,"num_object_clones":0,"num_object_copies":8708,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4354,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12949,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-17}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877349+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:05:19.877349+0000","last_peered":"2024-09-10T16:05:19.877349+0000","last_clean":"2024-09-10T16:05:19.877349+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:05:19.877349+0000","last_undegraded":"2024-09-10T16:05:19.877349+0000","last_fullsized":"2024-09-10T16:05:19.877349+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877269+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:05:19.877269+0000","last_peered":"2024-09-10T16:05:19.877269+0000","last_clean":"2024-09-10T16:05:19.877269+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:05:19.877269+0000","last_undegraded":"2024-09-10T16:05:19.877269+0000","last_fullsized":"2024-09-10T16:05:19.877269+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'16748","reported_seq":16898,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:09.787556+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:06:09.787556+0000","last_peered":"2024-09-10T16:06:09.787556+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:06:09.787556+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"43'6700","ondisk_log_start":"43'6700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18228166,"num_objects":4452,"num_object_clones":0,"num_object_copies":8904,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4452,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12911,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":129},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":289247990,"num_objects":70627,"num_object_clones":0,"num_object_copies":141254,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41790,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70627,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":221479,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9599,"num_bytes_recovered":33095680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":791097344,"data_stored":790955519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160905,"ondisk_log_size":160905,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986274,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":617200,"kb_used_data":278688,"kb_used_omap":4,"kb_used_meta":338491,"kb_avail":93754640,"statfs":{"total":96636764160,"available":96004751360,"internally_reserved":0,"allocated":285376512,"data_stored":285073828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":346615466},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,2,1,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659219,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":562052,"kb_used_data":215220,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93809788,"statfs":{"total":96636764160,"available":96061222912,"internally_reserved":0,"allocated":220385280,"data_stored":220053228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":355135150},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,6,4],"upper_bound":64},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182502,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":710156,"kb_used_data":281460,"kb_used_omap":4,"kb_used_meta":428667,"kb_avail":93661684,"statfs":{"total":96636764160,"available":95909564416,"internally_reserved":0,"allocated":288215040,"data_stored":287912351,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":438955690},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,4,1,3,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":286658560,"data_stored":286622430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":219955200,"data_stored":219889707,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":284483584,"data_stored":284443382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:06:12.129 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-10T16:06:12.130 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "47'16320", 'reported_seq': 16400, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:06:09.736133+0000', 'last_change': '2024-09-10T15:59:16.281558+0000', 'last_active': '2024-09-10T16:06:09.736133+0000', 'last_peered': '2024-09-10T16:06:09.736133+0000', 'last_clean': '2024-09-10T15:59:14.132456+0000', 'last_became_active': '2024-09-10T15:59:16.274407+0000', 'last_became_peered': '2024-09-10T15:59:16.274407+0000', 'last_unstale': '2024-09-10T16:06:09.736133+0000', 'last_undegraded': '2024-09-10T15:59:15.851279+0000', 'last_fullsized': '2024-09-10T15:59:15.774356+0000', 'mapping_epoch': 42, 'log_start': "43'6300", 'ondisk_log_start': "43'6300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'objects_scrubbed': 0, 'log_size': 10020, 'log_dups_size': 0, 'ondisk_log_size': 10020, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17825792, 'num_objects': 4352, 'num_object_clones': 0, 'num_object_copies': 8704, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4259, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4352, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16320, 'num_write_kb': 41344, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 93}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:06:12.130 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-10T16:06:12.130 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "47'16681", 'reported_seq': 16824, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:06:09.779805+0000', 'last_change': '2024-09-10T15:59:16.308264+0000', 'last_active': '2024-09-10T16:06:09.779805+0000', 'last_peered': '2024-09-10T16:06:09.779805+0000', 'last_clean': '2024-09-10T15:59:14.111551+0000', 'last_became_active': '2024-09-10T15:59:16.300040+0000', 'last_became_peered': '2024-09-10T15:59:16.300040+0000', 'last_unstale': '2024-09-10T16:06:09.779805+0000', 'last_undegraded': '2024-09-10T15:59:15.878234+0000', 'last_fullsized': '2024-09-10T15:59:15.769328+0000', 'mapping_epoch': 42, 'log_start': "43'6600", 'ondisk_log_start': "43'6600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3442", 'last_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'objects_scrubbed': 3384, 'log_size': 10081, 'log_dups_size': 0, 'ondisk_log_size': 10081, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1292, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17940963, 'num_objects': 4381, 'num_object_clones': 0, 'num_object_copies': 8762, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4372, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4381, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 13293, 'num_write_kb': 28580, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 10}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:06:12.130 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-10T16:06:12.130 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "47'16510", 'reported_seq': 16653, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:06:09.733574+0000', 'last_change': '2024-09-10T15:59:16.331899+0000', 'last_active': '2024-09-10T16:06:09.733574+0000', 'last_peered': '2024-09-10T16:06:09.733574+0000', 'last_clean': '2024-09-10T15:59:14.139436+0000', 'last_became_active': '2024-09-10T15:59:16.318715+0000', 'last_became_peered': '2024-09-10T15:59:16.318715+0000', 'last_unstale': '2024-09-10T16:06:09.733574+0000', 'last_undegraded': '2024-09-10T15:59:15.877524+0000', 'last_fullsized': '2024-09-10T15:59:15.769042+0000', 'mapping_epoch': 42, 'log_start': "43'6500", 'ondisk_log_start': "43'6500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3537", 'last_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'objects_scrubbed': 3497, 'log_size': 10010, 'log_dups_size': 0, 'ondisk_log_size': 10010, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1224, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18727878, 'num_objects': 4574, 'num_object_clones': 0, 'num_object_copies': 9148, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4237, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4574, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 13009, 'num_write_kb': 28172, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 337}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:06:12.130 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-10T16:06:12.130 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "47'16055", 'reported_seq': 16198, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:06:10.515251+0000', 'last_change': '2024-09-10T15:59:16.175680+0000', 'last_active': '2024-09-10T16:06:10.515251+0000', 'last_peered': '2024-09-10T16:06:10.515251+0000', 'last_clean': '2024-09-10T15:59:14.088187+0000', 'last_became_active': '2024-09-10T15:59:16.165810+0000', 'last_became_peered': '2024-09-10T15:59:16.165810+0000', 'last_unstale': '2024-09-10T16:06:10.515251+0000', 'last_undegraded': '2024-09-10T15:59:15.822098+0000', 'last_fullsized': '2024-09-10T15:59:15.769123+0000', 'mapping_epoch': 42, 'log_start': "43'6000", 'ondisk_log_start': "43'6000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2218", 'last_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'objects_scrubbed': 2175, 'log_size': 10055, 'log_dups_size': 0, 'ondisk_log_size': 10055, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 825, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18031075, 'num_objects': 4403, 'num_object_clones': 0, 'num_object_copies': 8806, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4124, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4403, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 13874, 'num_write_kb': 32204, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': 280}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:06:12.130 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-10T16:06:12.130 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "47'16427", 'reported_seq': 16559, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:06:09.800606+0000', 'last_change': '2024-09-10T15:59:16.277742+0000', 'last_active': '2024-09-10T16:06:09.800606+0000', 'last_peered': '2024-09-10T16:06:09.800606+0000', 'last_clean': '2024-09-10T15:59:14.129081+0000', 'last_became_active': '2024-09-10T15:59:16.269835+0000', 'last_became_peered': '2024-09-10T15:59:16.269835+0000', 'last_unstale': '2024-09-10T16:06:09.800606+0000', 'last_undegraded': '2024-09-10T15:59:15.860857+0000', 'last_fullsized': '2024-09-10T15:59:15.771410+0000', 'mapping_epoch': 42, 'log_start': "43'6400", 'ondisk_log_start': "43'6400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3557", 'last_scrub_stamp': '2024-09-10T15:59:02.493678+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:02.493678+0000', 'objects_scrubbed': 3488, 'log_size': 10027, 'log_dups_size': 0, 'ondisk_log_size': 10027, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1472, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18362368, 'num_objects': 4483, 'num_object_clones': 0, 'num_object_copies': 8966, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4257, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4483, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12931, 'num_write_kb': 27852, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 228}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:06:12.130 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-10T16:06:12.131 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "47'16428", 'reported_seq': 16568, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:06:10.542399+0000', 'last_change': '2024-09-10T15:59:16.284311+0000', 'last_active': '2024-09-10T16:06:10.542399+0000', 'last_peered': '2024-09-10T16:06:10.542399+0000', 'last_clean': '2024-09-10T15:59:14.116041+0000', 'last_became_active': '2024-09-10T15:59:16.274543+0000', 'last_became_peered': '2024-09-10T15:59:16.274543+0000', 'last_unstale': '2024-09-10T16:06:10.542399+0000', 'last_undegraded': '2024-09-10T15:59:15.812484+0000', 'last_fullsized': '2024-09-10T15:59:15.770885+0000', 'mapping_epoch': 42, 'log_start': "43'6400", 'ondisk_log_start': "43'6400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2356", 'last_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'objects_scrubbed': 2316, 'log_size': 10028, 'log_dups_size': 0, 'ondisk_log_size': 10028, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 783, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17830371, 'num_objects': 4354, 'num_object_clones': 0, 'num_object_copies': 8708, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4354, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 14108, 'num_write_kb': 32292, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': 50}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:06:12.131 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-10T16:06:12.131 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "47'16740", 'reported_seq': 19117, 'reported_epoch': 47, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:06:10.556121+0000', 'last_change': '2024-09-10T16:05:18.271152+0000', 'last_active': '2024-09-10T16:06:10.556121+0000', 'last_peered': '2024-09-10T16:06:10.556121+0000', 'last_clean': '2024-09-10T15:59:14.110654+0000', 'last_became_active': '2024-09-10T15:59:16.164645+0000', 'last_became_peered': '2024-09-10T15:59:16.164645+0000', 'last_unstale': '2024-09-10T16:06:10.556121+0000', 'last_undegraded': '2024-09-10T15:59:15.803593+0000', 'last_fullsized': '2024-09-10T15:59:15.769083+0000', 'mapping_epoch': 42, 'log_start': "43'6700", 'ondisk_log_start': "43'6700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2486", 'last_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'objects_scrubbed': 2440, 'log_size': 10040, 'log_dups_size': 0, 'ondisk_log_size': 10040, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 792, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18145280, 'num_objects': 4430, 'num_object_clones': 0, 'num_object_copies': 8860, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3233, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4430, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14294, 'num_write_kb': 32568, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1121, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 1197}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:06:12.131 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-10T16:06:12.131 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "47'16484", 'reported_seq': 16624, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:06:09.787441+0000', 'last_change': '2024-09-10T15:59:16.315641+0000', 'last_active': '2024-09-10T16:06:09.787441+0000', 'last_peered': '2024-09-10T16:06:09.787441+0000', 'last_clean': '2024-09-10T15:59:14.117001+0000', 'last_became_active': '2024-09-10T15:59:16.302245+0000', 'last_became_peered': '2024-09-10T15:59:16.302245+0000', 'last_unstale': '2024-09-10T16:06:09.787441+0000', 'last_undegraded': '2024-09-10T15:59:15.845012+0000', 'last_fullsized': '2024-09-10T15:59:15.770337+0000', 'mapping_epoch': 42, 'log_start': "43'6400", 'ondisk_log_start': "43'6400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3404", 'last_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'objects_scrubbed': 3367, 'log_size': 10084, 'log_dups_size': 0, 'ondisk_log_size': 10084, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 983, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17833984, 'num_objects': 4354, 'num_object_clones': 0, 'num_object_copies': 8708, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4354, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13115, 'num_write_kb': 28204, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 50}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:06:12.131 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-10T16:06:12.131 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "47'16694", 'reported_seq': 16841, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:06:09.744079+0000', 'last_change': '2024-09-10T15:59:16.311291+0000', 'last_active': '2024-09-10T16:06:09.744079+0000', 'last_peered': '2024-09-10T16:06:09.744079+0000', 'last_clean': '2024-09-10T15:59:14.116363+0000', 'last_became_active': '2024-09-10T15:59:16.301244+0000', 'last_became_peered': '2024-09-10T15:59:16.301244+0000', 'last_unstale': '2024-09-10T16:06:09.744079+0000', 'last_undegraded': '2024-09-10T15:59:15.883434+0000', 'last_fullsized': '2024-09-10T15:59:15.772756+0000', 'mapping_epoch': 42, 'log_start': "43'6600", 'ondisk_log_start': "43'6600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3789", 'last_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'objects_scrubbed': 3741, 'log_size': 10094, 'log_dups_size': 0, 'ondisk_log_size': 10094, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1440, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17833984, 'num_objects': 4354, 'num_object_clones': 0, 'num_object_copies': 8708, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4373, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4354, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12949, 'num_write_kb': 27124, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -17}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:06:12.131 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-10T16:06:12.131 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "47'16748", 'reported_seq': 16898, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:06:09.787556+0000', 'last_change': '2024-09-10T15:59:16.313409+0000', 'last_active': '2024-09-10T16:06:09.787556+0000', 'last_peered': '2024-09-10T16:06:09.787556+0000', 'last_clean': '2024-09-10T15:59:14.129780+0000', 'last_became_active': '2024-09-10T15:59:16.300932+0000', 'last_became_peered': '2024-09-10T15:59:16.300932+0000', 'last_unstale': '2024-09-10T16:06:09.787556+0000', 'last_undegraded': '2024-09-10T15:59:15.887823+0000', 'last_fullsized': '2024-09-10T15:59:15.768513+0000', 'mapping_epoch': 42, 'log_start': "43'6700", 'ondisk_log_start': "43'6700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3880", 'last_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'objects_scrubbed': 3827, 'log_size': 10048, 'log_dups_size': 0, 'ondisk_log_size': 10048, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1397, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18228166, 'num_objects': 4452, 'num_object_clones': 0, 'num_object_copies': 8904, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4325, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4452, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12911, 'num_write_kb': 27072, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 129}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:06:12.132 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T16:06:12.813 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:06:12.813 INFO:teuthology.orchestra.run.smithi115.stdout:{"fsid":"7effee29-df05-4351-9741-ee203564ed81","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 41790/141258 objects degraded (29.584%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":536,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":47,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983954,"num_in_osds":3,"osd_in_since":1725983947,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":70629,"data_bytes":289248009,"bytes_used":1934753792,"bytes_avail":287975538688,"bytes_total":289910292480,"degraded_objects":41790,"degraded_total":141258,"degraded_ratio":0.29584165144628977,"recovering_objects_per_sec":26,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":320,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":552},"fsmap":{"epoch":1,"btime":"2024-09-10T15:57:14:967120+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T16:00:13.560776+0000","services":{}},"progress_events":{}} 2024-09-10T16:06:12.830 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-10T16:06:12.830 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:06:13.390 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:06:13.392 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:06:13.419 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":308,"stamp":"2024-09-10T16:06:11.702926+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":289248009,"num_objects":70629,"num_object_clones":0,"num_object_copies":141258,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41790,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70629,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":221481,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9601,"num_bytes_recovered":33095699,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160908,"ondisk_log_size":160908,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1889408,"kb_used_data":775368,"kb_used_omap":12,"kb_used_meta":1113971,"kb_avail":281226112,"statfs":{"total":289910292480,"available":287975538688,"internally_reserved":0,"allocated":793976832,"data_stored":793039407,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1140706306},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,8,9,8,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-27181056,"num_objects":-6636,"num_object_clones":0,"num_object_copies":-13272,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-6636,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6636,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":320,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004584"},"pg_stats":[{"pgid":"2.f","version":"47'16320","reported_seq":16400,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:09.736133+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:06:09.736133+0000","last_peered":"2024-09-10T16:06:09.736133+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:06:09.736133+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"43'6300","ondisk_log_start":"43'6300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17825792,"num_objects":4352,"num_object_clones":0,"num_object_copies":8704,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4352,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16320,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":93}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'16449","reported_seq":16567,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:06:10.376911+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:06:10.376911+0000","last_peered":"2024-09-10T16:06:10.376911+0000","last_clean":"2024-09-10T16:06:10.376911+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:06:10.376911+0000","last_undegraded":"2024-09-10T16:06:10.376911+0000","last_fullsized":"2024-09-10T16:06:10.376911+0000","mapping_epoch":35,"log_start":"43'6400","ondisk_log_start":"43'6400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18002886,"num_objects":4397,"num_object_clones":0,"num_object_copies":8794,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4397,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":14554,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'16312","reported_seq":24994,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:06:07.965473+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:06:07.965473+0000","last_peered":"2024-09-10T16:06:07.965473+0000","last_clean":"2024-09-10T16:06:07.965473+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:06:07.965473+0000","last_undegraded":"2024-09-10T16:06:07.965473+0000","last_fullsized":"2024-09-10T16:06:07.965473+0000","mapping_epoch":44,"log_start":"43'6300","ondisk_log_start":"43'6300","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18370584,"num_objects":4486,"num_object_clones":0,"num_object_copies":8972,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4486,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14316,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'16397","reported_seq":16483,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:06:10.502594+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:06:10.502594+0000","last_peered":"2024-09-10T16:06:10.502594+0000","last_clean":"2024-09-10T16:06:10.502594+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:06:10.502594+0000","last_undegraded":"2024-09-10T16:06:10.502594+0000","last_fullsized":"2024-09-10T16:06:10.502594+0000","mapping_epoch":35,"log_start":"43'6300","ondisk_log_start":"43'6300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17461248,"num_objects":4263,"num_object_clones":0,"num_object_copies":8526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4263,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16397,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'16500","reported_seq":16635,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:06:09.785674+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:06:09.785674+0000","last_peered":"2024-09-10T16:06:09.785674+0000","last_clean":"2024-09-10T16:06:09.785674+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:06:09.785674+0000","last_undegraded":"2024-09-10T16:06:09.785674+0000","last_fullsized":"2024-09-10T16:06:09.785674+0000","mapping_epoch":34,"log_start":"43'6400","ondisk_log_start":"43'6400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17986019,"num_objects":4392,"num_object_clones":0,"num_object_copies":8784,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4392,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13211,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'16681","reported_seq":16824,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:09.779805+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:06:09.779805+0000","last_peered":"2024-09-10T16:06:09.779805+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:06:09.779805+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"43'6600","ondisk_log_start":"43'6600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17940963,"num_objects":4381,"num_object_clones":0,"num_object_copies":8762,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4381,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13293,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":10},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'16510","reported_seq":16653,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:09.733574+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:06:09.733574+0000","last_peered":"2024-09-10T16:06:09.733574+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:06:09.733574+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"43'6500","ondisk_log_start":"43'6500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18727878,"num_objects":4574,"num_object_clones":0,"num_object_copies":9148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4574,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13009,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":337},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'16784","reported_seq":16919,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:06:09.718425+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:06:09.718425+0000","last_peered":"2024-09-10T16:06:09.718425+0000","last_clean":"2024-09-10T16:06:09.718425+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:06:09.718425+0000","last_undegraded":"2024-09-10T16:06:09.718425+0000","last_fullsized":"2024-09-10T16:06:09.718425+0000","mapping_epoch":34,"log_start":"43'6700","ondisk_log_start":"43'6700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18554880,"num_objects":4530,"num_object_clones":0,"num_object_copies":9060,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4530,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13631,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.876895+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:05:19.876895+0000","last_peered":"2024-09-10T16:05:19.876895+0000","last_clean":"2024-09-10T16:05:19.876895+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:05:19.876895+0000","last_undegraded":"2024-09-10T16:05:19.876895+0000","last_fullsized":"2024-09-10T16:05:19.876895+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'16055","reported_seq":16198,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:10.515251+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:06:10.515251+0000","last_peered":"2024-09-10T16:06:10.515251+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:06:10.515251+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"43'6000","ondisk_log_start":"43'6000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18031075,"num_objects":4403,"num_object_clones":0,"num_object_copies":8806,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4403,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13874,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":280}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":99,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.875776+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:05:19.875776+0000","last_peered":"2024-09-10T16:05:19.875776+0000","last_clean":"2024-09-10T16:05:19.875776+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:05:19.875776+0000","last_undegraded":"2024-09-10T16:05:19.875776+0000","last_fullsized":"2024-09-10T16:05:19.875776+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'16427","reported_seq":16559,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:09.800606+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:06:09.800606+0000","last_peered":"2024-09-10T16:06:09.800606+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:06:09.800606+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"43'6400","ondisk_log_start":"43'6400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18362368,"num_objects":4483,"num_object_clones":0,"num_object_copies":8966,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4483,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12931,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":228}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":109,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.871983+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:05:19.871983+0000","last_peered":"2024-09-10T16:05:19.871983+0000","last_clean":"2024-09-10T16:05:19.871983+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:05:19.871983+0000","last_undegraded":"2024-09-10T16:05:19.871983+0000","last_fullsized":"2024-09-10T16:05:19.871983+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'16428","reported_seq":16568,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:10.542399+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:06:10.542399+0000","last_peered":"2024-09-10T16:06:10.542399+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:06:10.542399+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"43'6400","ondisk_log_start":"43'6400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17830371,"num_objects":4354,"num_object_clones":0,"num_object_copies":8708,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4354,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14108,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":50}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877148+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:05:19.877148+0000","last_peered":"2024-09-10T16:05:19.877148+0000","last_clean":"2024-09-10T16:05:19.877148+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:05:19.877148+0000","last_undegraded":"2024-09-10T16:05:19.877148+0000","last_fullsized":"2024-09-10T16:05:19.877148+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'16740","reported_seq":19117,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:10.556121+0000","last_change":"2024-09-10T16:05:18.271152+0000","last_active":"2024-09-10T16:06:10.556121+0000","last_peered":"2024-09-10T16:06:10.556121+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:06:10.556121+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"43'6700","ondisk_log_start":"43'6700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18145280,"num_objects":4430,"num_object_clones":0,"num_object_copies":8860,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3233,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4430,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14294,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1121,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1197}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'16484","reported_seq":16624,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:09.787441+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:06:09.787441+0000","last_peered":"2024-09-10T16:06:09.787441+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:06:09.787441+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"43'6400","ondisk_log_start":"43'6400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17833984,"num_objects":4354,"num_object_clones":0,"num_object_copies":8708,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4354,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13115,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":50}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877654+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:05:19.877654+0000","last_peered":"2024-09-10T16:05:19.877654+0000","last_clean":"2024-09-10T16:05:19.877654+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:05:19.877654+0000","last_undegraded":"2024-09-10T16:05:19.877654+0000","last_fullsized":"2024-09-10T16:05:19.877654+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'16176","reported_seq":24747,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:06:07.903268+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:06:07.903268+0000","last_peered":"2024-09-10T16:06:07.903268+0000","last_clean":"2024-09-10T16:06:07.903268+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:06:07.903268+0000","last_undegraded":"2024-09-10T16:06:07.903268+0000","last_fullsized":"2024-09-10T16:06:07.903268+0000","mapping_epoch":46,"log_start":"43'6100","ondisk_log_start":"43'6100","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18112512,"num_objects":4422,"num_object_clones":0,"num_object_copies":8844,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4422,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12566,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":99,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.876013+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:05:19.876013+0000","last_peered":"2024-09-10T16:05:19.876013+0000","last_clean":"2024-09-10T16:05:19.876013+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:05:19.876013+0000","last_undegraded":"2024-09-10T16:05:19.876013+0000","last_fullsized":"2024-09-10T16:05:19.876013+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'16694","reported_seq":16841,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:09.744079+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:06:09.744079+0000","last_peered":"2024-09-10T16:06:09.744079+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:06:09.744079+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"43'6600","ondisk_log_start":"43'6600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17833984,"num_objects":4354,"num_object_clones":0,"num_object_copies":8708,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4354,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12949,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-17}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877349+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:05:19.877349+0000","last_peered":"2024-09-10T16:05:19.877349+0000","last_clean":"2024-09-10T16:05:19.877349+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:05:19.877349+0000","last_undegraded":"2024-09-10T16:05:19.877349+0000","last_fullsized":"2024-09-10T16:05:19.877349+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877269+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:05:19.877269+0000","last_peered":"2024-09-10T16:05:19.877269+0000","last_clean":"2024-09-10T16:05:19.877269+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:05:19.877269+0000","last_undegraded":"2024-09-10T16:05:19.877269+0000","last_fullsized":"2024-09-10T16:05:19.877269+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'16748","reported_seq":16898,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:09.787556+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:06:09.787556+0000","last_peered":"2024-09-10T16:06:09.787556+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:06:09.787556+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"43'6700","ondisk_log_start":"43'6700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18228166,"num_objects":4452,"num_object_clones":0,"num_object_copies":8904,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4452,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12911,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":129},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":289247990,"num_objects":70627,"num_object_clones":0,"num_object_copies":141254,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41790,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70627,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":221479,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9599,"num_bytes_recovered":33095680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":791097344,"data_stored":790955519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160905,"ondisk_log_size":160905,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986274,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":617200,"kb_used_data":278688,"kb_used_omap":4,"kb_used_meta":338491,"kb_avail":93754640,"statfs":{"total":96636764160,"available":96004751360,"internally_reserved":0,"allocated":285376512,"data_stored":285073828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":346615466},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,2,1,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659219,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":562052,"kb_used_data":215220,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93809788,"statfs":{"total":96636764160,"available":96061222912,"internally_reserved":0,"allocated":220385280,"data_stored":220053228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":355135150},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,6,4],"upper_bound":64},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182502,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":710156,"kb_used_data":281460,"kb_used_omap":4,"kb_used_meta":428667,"kb_avail":93661684,"statfs":{"total":96636764160,"available":95909564416,"internally_reserved":0,"allocated":288215040,"data_stored":287912351,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":438955690},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,4,1,3,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":286658560,"data_stored":286622430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":219955200,"data_stored":219889707,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":284483584,"data_stored":284443382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:06:16.422 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:06:16.963 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:06:16.964 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:06:16.986 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":310,"stamp":"2024-09-10T16:06:15.705210+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":279814921,"num_objects":68326,"num_object_clones":0,"num_object_copies":136652,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41686,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":68326,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":223784,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9705,"num_bytes_recovered":33095699,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160911,"ondisk_log_size":160911,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1871104,"kb_used_data":757056,"kb_used_omap":12,"kb_used_meta":1113971,"kb_avail":281244416,"statfs":{"total":289910292480,"available":287994281984,"internally_reserved":0,"allocated":775225344,"data_stored":774287919,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1140706306},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,4,1,4,5,5,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-27107328,"num_objects":-6618,"num_object_clones":0,"num_object_copies":-13236,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-6618,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6618,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":320,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005551"},"pg_stats":[{"pgid":"2.f","version":"47'16473","reported_seq":16553,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:14.804690+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:06:14.804690+0000","last_peered":"2024-09-10T16:06:14.804690+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:06:14.804690+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"43'6400","ondisk_log_start":"43'6400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17199104,"num_objects":4199,"num_object_clones":0,"num_object_copies":8398,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4199,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16473,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-60}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'16615","reported_seq":16733,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:06:15.552939+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:06:15.552939+0000","last_peered":"2024-09-10T16:06:15.552939+0000","last_clean":"2024-09-10T16:06:15.552939+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:06:15.552939+0000","last_undegraded":"2024-09-10T16:06:15.552939+0000","last_fullsized":"2024-09-10T16:06:15.552939+0000","mapping_epoch":35,"log_start":"43'6600","ondisk_log_start":"43'6600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17322950,"num_objects":4231,"num_object_clones":0,"num_object_copies":8462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4231,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":14720,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'16462","reported_seq":25145,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:06:12.953547+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:06:12.953547+0000","last_peered":"2024-09-10T16:06:12.953547+0000","last_clean":"2024-09-10T16:06:12.953547+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:06:12.953547+0000","last_undegraded":"2024-09-10T16:06:12.953547+0000","last_fullsized":"2024-09-10T16:06:12.953547+0000","mapping_epoch":44,"log_start":"43'6400","ondisk_log_start":"43'6400","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17756184,"num_objects":4336,"num_object_clones":0,"num_object_copies":8672,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4336,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14466,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'16521","reported_seq":16607,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:06:15.491163+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:06:15.491163+0000","last_peered":"2024-09-10T16:06:15.491163+0000","last_clean":"2024-09-10T16:06:15.491163+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:06:15.491163+0000","last_undegraded":"2024-09-10T16:06:15.491163+0000","last_fullsized":"2024-09-10T16:06:15.491163+0000","mapping_epoch":35,"log_start":"43'6500","ondisk_log_start":"43'6500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16953344,"num_objects":4139,"num_object_clones":0,"num_object_copies":8278,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4139,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16521,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'16650","reported_seq":16785,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:06:14.841319+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:06:14.841319+0000","last_peered":"2024-09-10T16:06:14.841319+0000","last_clean":"2024-09-10T16:06:14.841319+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:06:14.841319+0000","last_undegraded":"2024-09-10T16:06:14.841319+0000","last_fullsized":"2024-09-10T16:06:14.841319+0000","mapping_epoch":34,"log_start":"43'6600","ondisk_log_start":"43'6600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17371619,"num_objects":4242,"num_object_clones":0,"num_object_copies":8484,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4242,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13361,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'16823","reported_seq":16966,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:14.791473+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:06:14.791473+0000","last_peered":"2024-09-10T16:06:14.791473+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:06:14.791473+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"43'6800","ondisk_log_start":"43'6800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17359331,"num_objects":4239,"num_object_clones":0,"num_object_copies":8478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4239,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13435,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-132},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'16669","reported_seq":16812,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:14.694166+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:06:14.694166+0000","last_peered":"2024-09-10T16:06:14.694166+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:06:14.694166+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"43'6600","ondisk_log_start":"43'6600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18076614,"num_objects":4415,"num_object_clones":0,"num_object_copies":8830,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4415,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13168,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":178},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'16934","reported_seq":17068,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:06:14.841925+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:06:14.841925+0000","last_peered":"2024-09-10T16:06:14.841925+0000","last_clean":"2024-09-10T16:06:14.841925+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:06:14.841925+0000","last_undegraded":"2024-09-10T16:06:14.841925+0000","last_fullsized":"2024-09-10T16:06:14.841925+0000","mapping_epoch":34,"log_start":"43'6900","ondisk_log_start":"43'6900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17940480,"num_objects":4380,"num_object_clones":0,"num_object_copies":8760,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4380,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13781,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.876895+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:05:19.876895+0000","last_peered":"2024-09-10T16:05:19.876895+0000","last_clean":"2024-09-10T16:05:19.876895+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:05:19.876895+0000","last_undegraded":"2024-09-10T16:05:19.876895+0000","last_fullsized":"2024-09-10T16:05:19.876895+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'16205","reported_seq":16350,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:15.552461+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:06:15.552461+0000","last_peered":"2024-09-10T16:06:15.552461+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:06:15.552461+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"43'6200","ondisk_log_start":"43'6200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17416675,"num_objects":4253,"num_object_clones":0,"num_object_copies":8506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4253,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14024,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":130}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":99,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.875776+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:05:19.875776+0000","last_peered":"2024-09-10T16:05:19.875776+0000","last_clean":"2024-09-10T16:05:19.875776+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:05:19.875776+0000","last_undegraded":"2024-09-10T16:05:19.875776+0000","last_fullsized":"2024-09-10T16:05:19.875776+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'16554","reported_seq":16690,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:14.840663+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:06:14.840663+0000","last_peered":"2024-09-10T16:06:14.840663+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:06:14.840663+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"43'6500","ondisk_log_start":"43'6500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17842176,"num_objects":4356,"num_object_clones":0,"num_object_copies":8712,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4356,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13058,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":101}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":109,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.871983+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:05:19.871983+0000","last_peered":"2024-09-10T16:05:19.871983+0000","last_clean":"2024-09-10T16:05:19.871983+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:05:19.871983+0000","last_undegraded":"2024-09-10T16:05:19.871983+0000","last_fullsized":"2024-09-10T16:05:19.871983+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'16579","reported_seq":16719,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:15.553409+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:06:15.553409+0000","last_peered":"2024-09-10T16:06:15.553409+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:06:15.553409+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"43'6500","ondisk_log_start":"43'6500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17211875,"num_objects":4203,"num_object_clones":0,"num_object_copies":8406,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4203,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14259,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-101}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877148+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:05:19.877148+0000","last_peered":"2024-09-10T16:05:19.877148+0000","last_clean":"2024-09-10T16:05:19.877148+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:05:19.877148+0000","last_undegraded":"2024-09-10T16:05:19.877148+0000","last_fullsized":"2024-09-10T16:05:19.877148+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'16898","reported_seq":19483,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:15.557699+0000","last_change":"2024-09-10T16:05:18.271152+0000","last_active":"2024-09-10T16:06:15.557699+0000","last_peered":"2024-09-10T16:06:15.557699+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:06:15.557699+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"43'6800","ondisk_log_start":"43'6800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17498112,"num_objects":4272,"num_object_clones":0,"num_object_copies":8544,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3129,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4272,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14452,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1225,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1143}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'16590","reported_seq":16730,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:14.712623+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:06:14.712623+0000","last_peered":"2024-09-10T16:06:14.712623+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:06:14.712623+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"43'6500","ondisk_log_start":"43'6500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17399808,"num_objects":4248,"num_object_clones":0,"num_object_copies":8496,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4248,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13221,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-56}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877654+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:05:19.877654+0000","last_peered":"2024-09-10T16:05:19.877654+0000","last_clean":"2024-09-10T16:05:19.877654+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:05:19.877654+0000","last_undegraded":"2024-09-10T16:05:19.877654+0000","last_fullsized":"2024-09-10T16:05:19.877654+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'16296","reported_seq":24867,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:06:12.956358+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:06:12.956358+0000","last_peered":"2024-09-10T16:06:12.956358+0000","last_clean":"2024-09-10T16:06:12.956358+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:06:12.956358+0000","last_undegraded":"2024-09-10T16:06:12.956358+0000","last_fullsized":"2024-09-10T16:06:12.956358+0000","mapping_epoch":46,"log_start":"43'6200","ondisk_log_start":"43'6200","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17620992,"num_objects":4302,"num_object_clones":0,"num_object_copies":8604,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4302,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12686,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":99,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.876013+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:05:19.876013+0000","last_peered":"2024-09-10T16:05:19.876013+0000","last_clean":"2024-09-10T16:05:19.876013+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:05:19.876013+0000","last_undegraded":"2024-09-10T16:05:19.876013+0000","last_fullsized":"2024-09-10T16:05:19.876013+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'16841","reported_seq":16987,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:14.768116+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:06:14.768116+0000","last_peered":"2024-09-10T16:06:14.768116+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:06:14.768116+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"43'6800","ondisk_log_start":"43'6800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17231872,"num_objects":4207,"num_object_clones":0,"num_object_copies":8414,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4207,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13096,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-164}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877349+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:05:19.877349+0000","last_peered":"2024-09-10T16:05:19.877349+0000","last_clean":"2024-09-10T16:05:19.877349+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:05:19.877349+0000","last_undegraded":"2024-09-10T16:05:19.877349+0000","last_fullsized":"2024-09-10T16:05:19.877349+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877269+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:05:19.877269+0000","last_peered":"2024-09-10T16:05:19.877269+0000","last_clean":"2024-09-10T16:05:19.877269+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:05:19.877269+0000","last_undegraded":"2024-09-10T16:05:19.877269+0000","last_fullsized":"2024-09-10T16:05:19.877269+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'16898","reported_seq":17048,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:14.823895+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:06:14.823895+0000","last_peered":"2024-09-10T16:06:14.823895+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:06:14.823895+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"43'6800","ondisk_log_start":"43'6800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17613766,"num_objects":4302,"num_object_clones":0,"num_object_copies":8604,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4302,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13061,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-21},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":279814902,"num_objects":68324,"num_object_clones":0,"num_object_copies":136648,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41686,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":68324,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":223782,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9703,"num_bytes_recovered":33095680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":772087808,"data_stored":771945983,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160908,"ondisk_log_size":160908,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986275,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":616516,"kb_used_data":273840,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93755324,"statfs":{"total":96636764160,"available":96005451776,"internally_reserved":0,"allocated":280412160,"data_stored":280109476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":350875306},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2],"upper_bound":16},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659220,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":546960,"kb_used_data":208448,"kb_used_omap":4,"kb_used_meta":338491,"kb_avail":93824880,"statfs":{"total":96636764160,"available":96076677120,"internally_reserved":0,"allocated":213450752,"data_stored":213118700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":346615470},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,4,1,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182503,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":707628,"kb_used_data":274768,"kb_used_omap":4,"kb_used_meta":432827,"kb_avail":93664212,"statfs":{"total":96636764160,"available":95912153088,"internally_reserved":0,"allocated":281362432,"data_stored":281059743,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":443215530},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,4,0,1,1,4,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":279744512,"data_stored":279708382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":212779008,"data_stored":212713515,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":279564288,"data_stored":279524086,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:06:16.987 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:06:17.549 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:06:17.551 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:06:17.574 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":310,"stamp":"2024-09-10T16:06:15.705210+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":279814921,"num_objects":68326,"num_object_clones":0,"num_object_copies":136652,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41686,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":68326,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":223784,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9705,"num_bytes_recovered":33095699,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160911,"ondisk_log_size":160911,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1871104,"kb_used_data":757056,"kb_used_omap":12,"kb_used_meta":1113971,"kb_avail":281244416,"statfs":{"total":289910292480,"available":287994281984,"internally_reserved":0,"allocated":775225344,"data_stored":774287919,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1140706306},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,4,1,4,5,5,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-27107328,"num_objects":-6618,"num_object_clones":0,"num_object_copies":-13236,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-6618,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6618,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":320,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005551"},"pg_stats":[{"pgid":"2.f","version":"47'16473","reported_seq":16553,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:14.804690+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:06:14.804690+0000","last_peered":"2024-09-10T16:06:14.804690+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:06:14.804690+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"43'6400","ondisk_log_start":"43'6400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17199104,"num_objects":4199,"num_object_clones":0,"num_object_copies":8398,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4199,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16473,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-60}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'16615","reported_seq":16733,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:06:15.552939+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:06:15.552939+0000","last_peered":"2024-09-10T16:06:15.552939+0000","last_clean":"2024-09-10T16:06:15.552939+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:06:15.552939+0000","last_undegraded":"2024-09-10T16:06:15.552939+0000","last_fullsized":"2024-09-10T16:06:15.552939+0000","mapping_epoch":35,"log_start":"43'6600","ondisk_log_start":"43'6600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17322950,"num_objects":4231,"num_object_clones":0,"num_object_copies":8462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4231,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":14720,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'16462","reported_seq":25145,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:06:12.953547+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:06:12.953547+0000","last_peered":"2024-09-10T16:06:12.953547+0000","last_clean":"2024-09-10T16:06:12.953547+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:06:12.953547+0000","last_undegraded":"2024-09-10T16:06:12.953547+0000","last_fullsized":"2024-09-10T16:06:12.953547+0000","mapping_epoch":44,"log_start":"43'6400","ondisk_log_start":"43'6400","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17756184,"num_objects":4336,"num_object_clones":0,"num_object_copies":8672,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4336,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14466,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'16521","reported_seq":16607,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:06:15.491163+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:06:15.491163+0000","last_peered":"2024-09-10T16:06:15.491163+0000","last_clean":"2024-09-10T16:06:15.491163+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:06:15.491163+0000","last_undegraded":"2024-09-10T16:06:15.491163+0000","last_fullsized":"2024-09-10T16:06:15.491163+0000","mapping_epoch":35,"log_start":"43'6500","ondisk_log_start":"43'6500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16953344,"num_objects":4139,"num_object_clones":0,"num_object_copies":8278,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4139,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16521,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'16650","reported_seq":16785,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:06:14.841319+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:06:14.841319+0000","last_peered":"2024-09-10T16:06:14.841319+0000","last_clean":"2024-09-10T16:06:14.841319+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:06:14.841319+0000","last_undegraded":"2024-09-10T16:06:14.841319+0000","last_fullsized":"2024-09-10T16:06:14.841319+0000","mapping_epoch":34,"log_start":"43'6600","ondisk_log_start":"43'6600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17371619,"num_objects":4242,"num_object_clones":0,"num_object_copies":8484,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4242,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13361,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'16823","reported_seq":16966,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:14.791473+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:06:14.791473+0000","last_peered":"2024-09-10T16:06:14.791473+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:06:14.791473+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"43'6800","ondisk_log_start":"43'6800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17359331,"num_objects":4239,"num_object_clones":0,"num_object_copies":8478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4239,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13435,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-132},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'16669","reported_seq":16812,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:14.694166+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:06:14.694166+0000","last_peered":"2024-09-10T16:06:14.694166+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:06:14.694166+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"43'6600","ondisk_log_start":"43'6600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18076614,"num_objects":4415,"num_object_clones":0,"num_object_copies":8830,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4415,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13168,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":178},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'16934","reported_seq":17068,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:06:14.841925+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:06:14.841925+0000","last_peered":"2024-09-10T16:06:14.841925+0000","last_clean":"2024-09-10T16:06:14.841925+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:06:14.841925+0000","last_undegraded":"2024-09-10T16:06:14.841925+0000","last_fullsized":"2024-09-10T16:06:14.841925+0000","mapping_epoch":34,"log_start":"43'6900","ondisk_log_start":"43'6900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17940480,"num_objects":4380,"num_object_clones":0,"num_object_copies":8760,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4380,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13781,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.876895+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:05:19.876895+0000","last_peered":"2024-09-10T16:05:19.876895+0000","last_clean":"2024-09-10T16:05:19.876895+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:05:19.876895+0000","last_undegraded":"2024-09-10T16:05:19.876895+0000","last_fullsized":"2024-09-10T16:05:19.876895+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'16205","reported_seq":16350,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:15.552461+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:06:15.552461+0000","last_peered":"2024-09-10T16:06:15.552461+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:06:15.552461+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"43'6200","ondisk_log_start":"43'6200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17416675,"num_objects":4253,"num_object_clones":0,"num_object_copies":8506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4253,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14024,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":130}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":99,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.875776+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:05:19.875776+0000","last_peered":"2024-09-10T16:05:19.875776+0000","last_clean":"2024-09-10T16:05:19.875776+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:05:19.875776+0000","last_undegraded":"2024-09-10T16:05:19.875776+0000","last_fullsized":"2024-09-10T16:05:19.875776+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'16554","reported_seq":16690,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:14.840663+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:06:14.840663+0000","last_peered":"2024-09-10T16:06:14.840663+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:06:14.840663+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"43'6500","ondisk_log_start":"43'6500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17842176,"num_objects":4356,"num_object_clones":0,"num_object_copies":8712,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4356,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13058,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":101}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":109,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.871983+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:05:19.871983+0000","last_peered":"2024-09-10T16:05:19.871983+0000","last_clean":"2024-09-10T16:05:19.871983+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:05:19.871983+0000","last_undegraded":"2024-09-10T16:05:19.871983+0000","last_fullsized":"2024-09-10T16:05:19.871983+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'16579","reported_seq":16719,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:15.553409+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:06:15.553409+0000","last_peered":"2024-09-10T16:06:15.553409+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:06:15.553409+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"43'6500","ondisk_log_start":"43'6500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17211875,"num_objects":4203,"num_object_clones":0,"num_object_copies":8406,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4203,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14259,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-101}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877148+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:05:19.877148+0000","last_peered":"2024-09-10T16:05:19.877148+0000","last_clean":"2024-09-10T16:05:19.877148+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:05:19.877148+0000","last_undegraded":"2024-09-10T16:05:19.877148+0000","last_fullsized":"2024-09-10T16:05:19.877148+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'16898","reported_seq":19483,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:15.557699+0000","last_change":"2024-09-10T16:05:18.271152+0000","last_active":"2024-09-10T16:06:15.557699+0000","last_peered":"2024-09-10T16:06:15.557699+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:06:15.557699+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"43'6800","ondisk_log_start":"43'6800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17498112,"num_objects":4272,"num_object_clones":0,"num_object_copies":8544,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3129,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4272,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14452,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1225,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1143}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'16590","reported_seq":16730,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:14.712623+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:06:14.712623+0000","last_peered":"2024-09-10T16:06:14.712623+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:06:14.712623+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"43'6500","ondisk_log_start":"43'6500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17399808,"num_objects":4248,"num_object_clones":0,"num_object_copies":8496,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4248,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13221,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-56}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877654+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:05:19.877654+0000","last_peered":"2024-09-10T16:05:19.877654+0000","last_clean":"2024-09-10T16:05:19.877654+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:05:19.877654+0000","last_undegraded":"2024-09-10T16:05:19.877654+0000","last_fullsized":"2024-09-10T16:05:19.877654+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'16296","reported_seq":24867,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:06:12.956358+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:06:12.956358+0000","last_peered":"2024-09-10T16:06:12.956358+0000","last_clean":"2024-09-10T16:06:12.956358+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:06:12.956358+0000","last_undegraded":"2024-09-10T16:06:12.956358+0000","last_fullsized":"2024-09-10T16:06:12.956358+0000","mapping_epoch":46,"log_start":"43'6200","ondisk_log_start":"43'6200","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17620992,"num_objects":4302,"num_object_clones":0,"num_object_copies":8604,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4302,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12686,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":99,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.876013+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:05:19.876013+0000","last_peered":"2024-09-10T16:05:19.876013+0000","last_clean":"2024-09-10T16:05:19.876013+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:05:19.876013+0000","last_undegraded":"2024-09-10T16:05:19.876013+0000","last_fullsized":"2024-09-10T16:05:19.876013+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'16841","reported_seq":16987,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:14.768116+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:06:14.768116+0000","last_peered":"2024-09-10T16:06:14.768116+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:06:14.768116+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"43'6800","ondisk_log_start":"43'6800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17231872,"num_objects":4207,"num_object_clones":0,"num_object_copies":8414,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4207,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13096,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-164}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877349+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:05:19.877349+0000","last_peered":"2024-09-10T16:05:19.877349+0000","last_clean":"2024-09-10T16:05:19.877349+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:05:19.877349+0000","last_undegraded":"2024-09-10T16:05:19.877349+0000","last_fullsized":"2024-09-10T16:05:19.877349+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877269+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:05:19.877269+0000","last_peered":"2024-09-10T16:05:19.877269+0000","last_clean":"2024-09-10T16:05:19.877269+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:05:19.877269+0000","last_undegraded":"2024-09-10T16:05:19.877269+0000","last_fullsized":"2024-09-10T16:05:19.877269+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'16898","reported_seq":17048,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:14.823895+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:06:14.823895+0000","last_peered":"2024-09-10T16:06:14.823895+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:06:14.823895+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"43'6800","ondisk_log_start":"43'6800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17613766,"num_objects":4302,"num_object_clones":0,"num_object_copies":8604,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4302,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13061,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-21},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":279814902,"num_objects":68324,"num_object_clones":0,"num_object_copies":136648,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41686,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":68324,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":223782,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9703,"num_bytes_recovered":33095680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":772087808,"data_stored":771945983,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160908,"ondisk_log_size":160908,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986275,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":616516,"kb_used_data":273840,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93755324,"statfs":{"total":96636764160,"available":96005451776,"internally_reserved":0,"allocated":280412160,"data_stored":280109476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":350875306},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2],"upper_bound":16},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659220,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":546960,"kb_used_data":208448,"kb_used_omap":4,"kb_used_meta":338491,"kb_avail":93824880,"statfs":{"total":96636764160,"available":96076677120,"internally_reserved":0,"allocated":213450752,"data_stored":213118700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":346615470},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,4,1,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182503,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":707628,"kb_used_data":274768,"kb_used_omap":4,"kb_used_meta":432827,"kb_avail":93664212,"statfs":{"total":96636764160,"available":95912153088,"internally_reserved":0,"allocated":281362432,"data_stored":281059743,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":443215530},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,4,0,1,1,4,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":279744512,"data_stored":279708382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":212779008,"data_stored":212713515,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":279564288,"data_stored":279524086,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:06:17.575 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-10T16:06:17.575 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "47'16473", 'reported_seq': 16553, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:06:14.804690+0000', 'last_change': '2024-09-10T15:59:16.281558+0000', 'last_active': '2024-09-10T16:06:14.804690+0000', 'last_peered': '2024-09-10T16:06:14.804690+0000', 'last_clean': '2024-09-10T15:59:14.132456+0000', 'last_became_active': '2024-09-10T15:59:16.274407+0000', 'last_became_peered': '2024-09-10T15:59:16.274407+0000', 'last_unstale': '2024-09-10T16:06:14.804690+0000', 'last_undegraded': '2024-09-10T15:59:15.851279+0000', 'last_fullsized': '2024-09-10T15:59:15.774356+0000', 'mapping_epoch': 42, 'log_start': "43'6400", 'ondisk_log_start': "43'6400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'objects_scrubbed': 0, 'log_size': 10073, 'log_dups_size': 0, 'ondisk_log_size': 10073, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17199104, 'num_objects': 4199, 'num_object_clones': 0, 'num_object_copies': 8398, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4259, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4199, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16473, 'num_write_kb': 41344, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -60}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:06:17.575 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-10T16:06:17.576 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "47'16823", 'reported_seq': 16966, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:06:14.791473+0000', 'last_change': '2024-09-10T15:59:16.308264+0000', 'last_active': '2024-09-10T16:06:14.791473+0000', 'last_peered': '2024-09-10T16:06:14.791473+0000', 'last_clean': '2024-09-10T15:59:14.111551+0000', 'last_became_active': '2024-09-10T15:59:16.300040+0000', 'last_became_peered': '2024-09-10T15:59:16.300040+0000', 'last_unstale': '2024-09-10T16:06:14.791473+0000', 'last_undegraded': '2024-09-10T15:59:15.878234+0000', 'last_fullsized': '2024-09-10T15:59:15.769328+0000', 'mapping_epoch': 42, 'log_start': "43'6800", 'ondisk_log_start': "43'6800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3442", 'last_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'objects_scrubbed': 3384, 'log_size': 10023, 'log_dups_size': 0, 'ondisk_log_size': 10023, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1292, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17359331, 'num_objects': 4239, 'num_object_clones': 0, 'num_object_copies': 8478, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4372, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4239, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 13435, 'num_write_kb': 28580, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -132}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:06:17.576 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-10T16:06:17.576 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "47'16669", 'reported_seq': 16812, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:06:14.694166+0000', 'last_change': '2024-09-10T15:59:16.331899+0000', 'last_active': '2024-09-10T16:06:14.694166+0000', 'last_peered': '2024-09-10T16:06:14.694166+0000', 'last_clean': '2024-09-10T15:59:14.139436+0000', 'last_became_active': '2024-09-10T15:59:16.318715+0000', 'last_became_peered': '2024-09-10T15:59:16.318715+0000', 'last_unstale': '2024-09-10T16:06:14.694166+0000', 'last_undegraded': '2024-09-10T15:59:15.877524+0000', 'last_fullsized': '2024-09-10T15:59:15.769042+0000', 'mapping_epoch': 42, 'log_start': "43'6600", 'ondisk_log_start': "43'6600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3537", 'last_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'objects_scrubbed': 3497, 'log_size': 10069, 'log_dups_size': 0, 'ondisk_log_size': 10069, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1224, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18076614, 'num_objects': 4415, 'num_object_clones': 0, 'num_object_copies': 8830, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4237, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4415, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 13168, 'num_write_kb': 28172, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 178}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:06:17.576 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-10T16:06:17.576 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "47'16205", 'reported_seq': 16350, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:06:15.552461+0000', 'last_change': '2024-09-10T15:59:16.175680+0000', 'last_active': '2024-09-10T16:06:15.552461+0000', 'last_peered': '2024-09-10T16:06:15.552461+0000', 'last_clean': '2024-09-10T15:59:14.088187+0000', 'last_became_active': '2024-09-10T15:59:16.165810+0000', 'last_became_peered': '2024-09-10T15:59:16.165810+0000', 'last_unstale': '2024-09-10T16:06:15.552461+0000', 'last_undegraded': '2024-09-10T15:59:15.822098+0000', 'last_fullsized': '2024-09-10T15:59:15.769123+0000', 'mapping_epoch': 42, 'log_start': "43'6200", 'ondisk_log_start': "43'6200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2218", 'last_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'objects_scrubbed': 2175, 'log_size': 10005, 'log_dups_size': 0, 'ondisk_log_size': 10005, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 825, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17416675, 'num_objects': 4253, 'num_object_clones': 0, 'num_object_copies': 8506, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4124, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4253, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 14024, 'num_write_kb': 32204, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': 130}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:06:17.576 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-10T16:06:17.576 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "47'16554", 'reported_seq': 16690, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:06:14.840663+0000', 'last_change': '2024-09-10T15:59:16.277742+0000', 'last_active': '2024-09-10T16:06:14.840663+0000', 'last_peered': '2024-09-10T16:06:14.840663+0000', 'last_clean': '2024-09-10T15:59:14.129081+0000', 'last_became_active': '2024-09-10T15:59:16.269835+0000', 'last_became_peered': '2024-09-10T15:59:16.269835+0000', 'last_unstale': '2024-09-10T16:06:14.840663+0000', 'last_undegraded': '2024-09-10T15:59:15.860857+0000', 'last_fullsized': '2024-09-10T15:59:15.771410+0000', 'mapping_epoch': 42, 'log_start': "43'6500", 'ondisk_log_start': "43'6500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3557", 'last_scrub_stamp': '2024-09-10T15:59:02.493678+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:02.493678+0000', 'objects_scrubbed': 3488, 'log_size': 10054, 'log_dups_size': 0, 'ondisk_log_size': 10054, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1472, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17842176, 'num_objects': 4356, 'num_object_clones': 0, 'num_object_copies': 8712, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4257, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4356, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13058, 'num_write_kb': 27852, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 101}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:06:17.577 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-10T16:06:17.577 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "47'16579", 'reported_seq': 16719, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:06:15.553409+0000', 'last_change': '2024-09-10T15:59:16.284311+0000', 'last_active': '2024-09-10T16:06:15.553409+0000', 'last_peered': '2024-09-10T16:06:15.553409+0000', 'last_clean': '2024-09-10T15:59:14.116041+0000', 'last_became_active': '2024-09-10T15:59:16.274543+0000', 'last_became_peered': '2024-09-10T15:59:16.274543+0000', 'last_unstale': '2024-09-10T16:06:15.553409+0000', 'last_undegraded': '2024-09-10T15:59:15.812484+0000', 'last_fullsized': '2024-09-10T15:59:15.770885+0000', 'mapping_epoch': 42, 'log_start': "43'6500", 'ondisk_log_start': "43'6500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2356", 'last_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'objects_scrubbed': 2316, 'log_size': 10079, 'log_dups_size': 0, 'ondisk_log_size': 10079, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 783, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17211875, 'num_objects': 4203, 'num_object_clones': 0, 'num_object_copies': 8406, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4203, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 14259, 'num_write_kb': 32292, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': -101}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:06:17.577 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-10T16:06:17.577 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "47'16898", 'reported_seq': 19483, 'reported_epoch': 47, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:06:15.557699+0000', 'last_change': '2024-09-10T16:05:18.271152+0000', 'last_active': '2024-09-10T16:06:15.557699+0000', 'last_peered': '2024-09-10T16:06:15.557699+0000', 'last_clean': '2024-09-10T15:59:14.110654+0000', 'last_became_active': '2024-09-10T15:59:16.164645+0000', 'last_became_peered': '2024-09-10T15:59:16.164645+0000', 'last_unstale': '2024-09-10T16:06:15.557699+0000', 'last_undegraded': '2024-09-10T15:59:15.803593+0000', 'last_fullsized': '2024-09-10T15:59:15.769083+0000', 'mapping_epoch': 42, 'log_start': "43'6800", 'ondisk_log_start': "43'6800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2486", 'last_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'objects_scrubbed': 2440, 'log_size': 10098, 'log_dups_size': 0, 'ondisk_log_size': 10098, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 792, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17498112, 'num_objects': 4272, 'num_object_clones': 0, 'num_object_copies': 8544, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3129, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4272, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14452, 'num_write_kb': 32568, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1225, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 1143}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:06:17.577 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-10T16:06:17.577 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "47'16590", 'reported_seq': 16730, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:06:14.712623+0000', 'last_change': '2024-09-10T15:59:16.315641+0000', 'last_active': '2024-09-10T16:06:14.712623+0000', 'last_peered': '2024-09-10T16:06:14.712623+0000', 'last_clean': '2024-09-10T15:59:14.117001+0000', 'last_became_active': '2024-09-10T15:59:16.302245+0000', 'last_became_peered': '2024-09-10T15:59:16.302245+0000', 'last_unstale': '2024-09-10T16:06:14.712623+0000', 'last_undegraded': '2024-09-10T15:59:15.845012+0000', 'last_fullsized': '2024-09-10T15:59:15.770337+0000', 'mapping_epoch': 42, 'log_start': "43'6500", 'ondisk_log_start': "43'6500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3404", 'last_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'objects_scrubbed': 3367, 'log_size': 10090, 'log_dups_size': 0, 'ondisk_log_size': 10090, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 983, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17399808, 'num_objects': 4248, 'num_object_clones': 0, 'num_object_copies': 8496, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4248, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13221, 'num_write_kb': 28204, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -56}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:06:17.577 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-10T16:06:17.577 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "47'16841", 'reported_seq': 16987, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:06:14.768116+0000', 'last_change': '2024-09-10T15:59:16.311291+0000', 'last_active': '2024-09-10T16:06:14.768116+0000', 'last_peered': '2024-09-10T16:06:14.768116+0000', 'last_clean': '2024-09-10T15:59:14.116363+0000', 'last_became_active': '2024-09-10T15:59:16.301244+0000', 'last_became_peered': '2024-09-10T15:59:16.301244+0000', 'last_unstale': '2024-09-10T16:06:14.768116+0000', 'last_undegraded': '2024-09-10T15:59:15.883434+0000', 'last_fullsized': '2024-09-10T15:59:15.772756+0000', 'mapping_epoch': 42, 'log_start': "43'6800", 'ondisk_log_start': "43'6800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3789", 'last_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'objects_scrubbed': 3741, 'log_size': 10041, 'log_dups_size': 0, 'ondisk_log_size': 10041, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1440, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17231872, 'num_objects': 4207, 'num_object_clones': 0, 'num_object_copies': 8414, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4373, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4207, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13096, 'num_write_kb': 27124, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -164}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:06:17.577 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-10T16:06:17.577 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "47'16898", 'reported_seq': 17048, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:06:14.823895+0000', 'last_change': '2024-09-10T15:59:16.313409+0000', 'last_active': '2024-09-10T16:06:14.823895+0000', 'last_peered': '2024-09-10T16:06:14.823895+0000', 'last_clean': '2024-09-10T15:59:14.129780+0000', 'last_became_active': '2024-09-10T15:59:16.300932+0000', 'last_became_peered': '2024-09-10T15:59:16.300932+0000', 'last_unstale': '2024-09-10T16:06:14.823895+0000', 'last_undegraded': '2024-09-10T15:59:15.887823+0000', 'last_fullsized': '2024-09-10T15:59:15.768513+0000', 'mapping_epoch': 42, 'log_start': "43'6800", 'ondisk_log_start': "43'6800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3880", 'last_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'objects_scrubbed': 3827, 'log_size': 10098, 'log_dups_size': 0, 'ondisk_log_size': 10098, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1397, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17613766, 'num_objects': 4302, 'num_object_clones': 0, 'num_object_copies': 8604, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4325, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4302, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13061, 'num_write_kb': 27072, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -21}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:06:17.578 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T16:06:18.281 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:06:18.281 INFO:teuthology.orchestra.run.smithi115.stdout:{"fsid":"7effee29-df05-4351-9741-ee203564ed81","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 41686/136652 objects degraded (30.505%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":542,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":47,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983954,"num_in_osds":3,"osd_in_since":1725983947,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":68326,"data_bytes":279814921,"bytes_used":1916010496,"bytes_avail":287994281984,"bytes_total":289910292480,"degraded_objects":41686,"degraded_total":136652,"degraded_ratio":0.3050522495097035,"recovering_objects_per_sec":17,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":209,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":382},"fsmap":{"epoch":1,"btime":"2024-09-10T15:57:14:967120+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T16:00:13.560776+0000","services":{}},"progress_events":{}} 2024-09-10T16:06:18.304 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-10T16:06:18.304 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:06:18.872 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:06:18.875 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:06:18.893 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":311,"stamp":"2024-09-10T16:06:17.706061+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":279814921,"num_objects":68326,"num_object_clones":0,"num_object_copies":136652,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41686,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":68326,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":223784,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9705,"num_bytes_recovered":33095699,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160911,"ondisk_log_size":160911,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1871104,"kb_used_data":757056,"kb_used_omap":12,"kb_used_meta":1113971,"kb_avail":281244416,"statfs":{"total":289910292480,"available":287994281984,"internally_reserved":0,"allocated":775225344,"data_stored":774287919,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1140706306},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,4,1,4,5,5,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18812928,"num_objects":-4593,"num_object_clones":0,"num_object_copies":-9186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-209,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4593,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4593,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":209,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005579"},"pg_stats":[{"pgid":"2.f","version":"47'16473","reported_seq":16553,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:14.804690+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:06:14.804690+0000","last_peered":"2024-09-10T16:06:14.804690+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:06:14.804690+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"43'6400","ondisk_log_start":"43'6400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17199104,"num_objects":4199,"num_object_clones":0,"num_object_copies":8398,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4199,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16473,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-60}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'16615","reported_seq":16733,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:06:15.552939+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:06:15.552939+0000","last_peered":"2024-09-10T16:06:15.552939+0000","last_clean":"2024-09-10T16:06:15.552939+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:06:15.552939+0000","last_undegraded":"2024-09-10T16:06:15.552939+0000","last_fullsized":"2024-09-10T16:06:15.552939+0000","mapping_epoch":35,"log_start":"43'6600","ondisk_log_start":"43'6600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17322950,"num_objects":4231,"num_object_clones":0,"num_object_copies":8462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4231,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":14720,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'16462","reported_seq":25145,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:06:12.953547+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:06:12.953547+0000","last_peered":"2024-09-10T16:06:12.953547+0000","last_clean":"2024-09-10T16:06:12.953547+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:06:12.953547+0000","last_undegraded":"2024-09-10T16:06:12.953547+0000","last_fullsized":"2024-09-10T16:06:12.953547+0000","mapping_epoch":44,"log_start":"43'6400","ondisk_log_start":"43'6400","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17756184,"num_objects":4336,"num_object_clones":0,"num_object_copies":8672,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4336,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14466,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'16521","reported_seq":16607,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:06:15.491163+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:06:15.491163+0000","last_peered":"2024-09-10T16:06:15.491163+0000","last_clean":"2024-09-10T16:06:15.491163+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:06:15.491163+0000","last_undegraded":"2024-09-10T16:06:15.491163+0000","last_fullsized":"2024-09-10T16:06:15.491163+0000","mapping_epoch":35,"log_start":"43'6500","ondisk_log_start":"43'6500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16953344,"num_objects":4139,"num_object_clones":0,"num_object_copies":8278,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4139,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16521,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'16650","reported_seq":16785,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:06:14.841319+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:06:14.841319+0000","last_peered":"2024-09-10T16:06:14.841319+0000","last_clean":"2024-09-10T16:06:14.841319+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:06:14.841319+0000","last_undegraded":"2024-09-10T16:06:14.841319+0000","last_fullsized":"2024-09-10T16:06:14.841319+0000","mapping_epoch":34,"log_start":"43'6600","ondisk_log_start":"43'6600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17371619,"num_objects":4242,"num_object_clones":0,"num_object_copies":8484,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4242,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13361,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'16823","reported_seq":16966,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:14.791473+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:06:14.791473+0000","last_peered":"2024-09-10T16:06:14.791473+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:06:14.791473+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"43'6800","ondisk_log_start":"43'6800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17359331,"num_objects":4239,"num_object_clones":0,"num_object_copies":8478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4239,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13435,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-132},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'16669","reported_seq":16812,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:14.694166+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:06:14.694166+0000","last_peered":"2024-09-10T16:06:14.694166+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:06:14.694166+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"43'6600","ondisk_log_start":"43'6600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18076614,"num_objects":4415,"num_object_clones":0,"num_object_copies":8830,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4415,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13168,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":178},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'16934","reported_seq":17068,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:06:14.841925+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:06:14.841925+0000","last_peered":"2024-09-10T16:06:14.841925+0000","last_clean":"2024-09-10T16:06:14.841925+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:06:14.841925+0000","last_undegraded":"2024-09-10T16:06:14.841925+0000","last_fullsized":"2024-09-10T16:06:14.841925+0000","mapping_epoch":34,"log_start":"43'6900","ondisk_log_start":"43'6900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17940480,"num_objects":4380,"num_object_clones":0,"num_object_copies":8760,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4380,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13781,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.876895+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:05:19.876895+0000","last_peered":"2024-09-10T16:05:19.876895+0000","last_clean":"2024-09-10T16:05:19.876895+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:05:19.876895+0000","last_undegraded":"2024-09-10T16:05:19.876895+0000","last_fullsized":"2024-09-10T16:05:19.876895+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'16205","reported_seq":16350,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:15.552461+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:06:15.552461+0000","last_peered":"2024-09-10T16:06:15.552461+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:06:15.552461+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"43'6200","ondisk_log_start":"43'6200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17416675,"num_objects":4253,"num_object_clones":0,"num_object_copies":8506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4253,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14024,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":130}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":99,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.875776+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:05:19.875776+0000","last_peered":"2024-09-10T16:05:19.875776+0000","last_clean":"2024-09-10T16:05:19.875776+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:05:19.875776+0000","last_undegraded":"2024-09-10T16:05:19.875776+0000","last_fullsized":"2024-09-10T16:05:19.875776+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'16554","reported_seq":16690,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:14.840663+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:06:14.840663+0000","last_peered":"2024-09-10T16:06:14.840663+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:06:14.840663+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"43'6500","ondisk_log_start":"43'6500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17842176,"num_objects":4356,"num_object_clones":0,"num_object_copies":8712,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4356,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13058,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":101}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":109,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.871983+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:05:19.871983+0000","last_peered":"2024-09-10T16:05:19.871983+0000","last_clean":"2024-09-10T16:05:19.871983+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:05:19.871983+0000","last_undegraded":"2024-09-10T16:05:19.871983+0000","last_fullsized":"2024-09-10T16:05:19.871983+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'16579","reported_seq":16719,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:15.553409+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:06:15.553409+0000","last_peered":"2024-09-10T16:06:15.553409+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:06:15.553409+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"43'6500","ondisk_log_start":"43'6500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17211875,"num_objects":4203,"num_object_clones":0,"num_object_copies":8406,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4203,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14259,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-101}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877148+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:05:19.877148+0000","last_peered":"2024-09-10T16:05:19.877148+0000","last_clean":"2024-09-10T16:05:19.877148+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:05:19.877148+0000","last_undegraded":"2024-09-10T16:05:19.877148+0000","last_fullsized":"2024-09-10T16:05:19.877148+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'16898","reported_seq":19483,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:15.557699+0000","last_change":"2024-09-10T16:05:18.271152+0000","last_active":"2024-09-10T16:06:15.557699+0000","last_peered":"2024-09-10T16:06:15.557699+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:06:15.557699+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"43'6800","ondisk_log_start":"43'6800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17498112,"num_objects":4272,"num_object_clones":0,"num_object_copies":8544,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3129,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4272,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14452,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1225,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1143}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'16590","reported_seq":16730,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:14.712623+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:06:14.712623+0000","last_peered":"2024-09-10T16:06:14.712623+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:06:14.712623+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"43'6500","ondisk_log_start":"43'6500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17399808,"num_objects":4248,"num_object_clones":0,"num_object_copies":8496,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4248,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13221,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-56}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877654+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:05:19.877654+0000","last_peered":"2024-09-10T16:05:19.877654+0000","last_clean":"2024-09-10T16:05:19.877654+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:05:19.877654+0000","last_undegraded":"2024-09-10T16:05:19.877654+0000","last_fullsized":"2024-09-10T16:05:19.877654+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'16296","reported_seq":24867,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:06:12.956358+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:06:12.956358+0000","last_peered":"2024-09-10T16:06:12.956358+0000","last_clean":"2024-09-10T16:06:12.956358+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:06:12.956358+0000","last_undegraded":"2024-09-10T16:06:12.956358+0000","last_fullsized":"2024-09-10T16:06:12.956358+0000","mapping_epoch":46,"log_start":"43'6200","ondisk_log_start":"43'6200","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17620992,"num_objects":4302,"num_object_clones":0,"num_object_copies":8604,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4302,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12686,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":99,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.876013+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:05:19.876013+0000","last_peered":"2024-09-10T16:05:19.876013+0000","last_clean":"2024-09-10T16:05:19.876013+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:05:19.876013+0000","last_undegraded":"2024-09-10T16:05:19.876013+0000","last_fullsized":"2024-09-10T16:05:19.876013+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'16841","reported_seq":16987,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:14.768116+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:06:14.768116+0000","last_peered":"2024-09-10T16:06:14.768116+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:06:14.768116+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"43'6800","ondisk_log_start":"43'6800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17231872,"num_objects":4207,"num_object_clones":0,"num_object_copies":8414,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4207,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13096,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-164}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877349+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:05:19.877349+0000","last_peered":"2024-09-10T16:05:19.877349+0000","last_clean":"2024-09-10T16:05:19.877349+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:05:19.877349+0000","last_undegraded":"2024-09-10T16:05:19.877349+0000","last_fullsized":"2024-09-10T16:05:19.877349+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877269+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:05:19.877269+0000","last_peered":"2024-09-10T16:05:19.877269+0000","last_clean":"2024-09-10T16:05:19.877269+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:05:19.877269+0000","last_undegraded":"2024-09-10T16:05:19.877269+0000","last_fullsized":"2024-09-10T16:05:19.877269+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'16898","reported_seq":17048,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:14.823895+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:06:14.823895+0000","last_peered":"2024-09-10T16:06:14.823895+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:06:14.823895+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"43'6800","ondisk_log_start":"43'6800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17613766,"num_objects":4302,"num_object_clones":0,"num_object_copies":8604,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4302,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13061,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-21},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":279814902,"num_objects":68324,"num_object_clones":0,"num_object_copies":136648,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41686,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":68324,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":223782,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9703,"num_bytes_recovered":33095680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":772087808,"data_stored":771945983,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160908,"ondisk_log_size":160908,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986275,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":616516,"kb_used_data":273840,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93755324,"statfs":{"total":96636764160,"available":96005451776,"internally_reserved":0,"allocated":280412160,"data_stored":280109476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":350875306},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2],"upper_bound":16},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659220,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":546960,"kb_used_data":208448,"kb_used_omap":4,"kb_used_meta":338491,"kb_avail":93824880,"statfs":{"total":96636764160,"available":96076677120,"internally_reserved":0,"allocated":213450752,"data_stored":213118700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":346615470},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,4,1,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182503,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":707628,"kb_used_data":274768,"kb_used_omap":4,"kb_used_meta":432827,"kb_avail":93664212,"statfs":{"total":96636764160,"available":95912153088,"internally_reserved":0,"allocated":281362432,"data_stored":281059743,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":443215530},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,4,0,1,1,4,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":279744512,"data_stored":279708382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":212779008,"data_stored":212713515,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":279564288,"data_stored":279524086,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:06:21.896 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:06:22.422 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:06:22.423 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:06:22.445 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":313,"stamp":"2024-09-10T16:06:21.707473+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":270439177,"num_objects":66037,"num_object_clones":0,"num_object_copies":132074,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41577,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66037,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":226073,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9814,"num_bytes_recovered":33095699,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160800,"ondisk_log_size":160800,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1865000,"kb_used_data":738452,"kb_used_omap":12,"kb_used_meta":1126451,"kb_avail":281250520,"statfs":{"total":289910292480,"available":288000532480,"internally_reserved":0,"allocated":756174848,"data_stored":755237423,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1153485826},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,4,6,8,2,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-27025408,"num_objects":-6598,"num_object_clones":0,"num_object_copies":-13196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-318,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-6598,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6598,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":318,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005368"},"pg_stats":[{"pgid":"2.f","version":"47'16626","reported_seq":16705,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:19.794366+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:06:19.794366+0000","last_peered":"2024-09-10T16:06:19.794366+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:06:19.794366+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"43'6600","ondisk_log_start":"43'6600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16572416,"num_objects":4046,"num_object_clones":0,"num_object_copies":8092,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4046,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16626,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-213}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'16757","reported_seq":16875,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:06:20.527569+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:06:20.527569+0000","last_peered":"2024-09-10T16:06:20.527569+0000","last_clean":"2024-09-10T16:06:20.527569+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:06:20.527569+0000","last_undegraded":"2024-09-10T16:06:20.527569+0000","last_fullsized":"2024-09-10T16:06:20.527569+0000","mapping_epoch":35,"log_start":"43'6700","ondisk_log_start":"43'6700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16741318,"num_objects":4089,"num_object_clones":0,"num_object_copies":8178,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4089,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":14862,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'16605","reported_seq":25288,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:06:18.058769+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:06:18.058769+0000","last_peered":"2024-09-10T16:06:18.058769+0000","last_clean":"2024-09-10T16:06:18.058769+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:06:18.058769+0000","last_undegraded":"2024-09-10T16:06:18.058769+0000","last_fullsized":"2024-09-10T16:06:18.058769+0000","mapping_epoch":44,"log_start":"43'6600","ondisk_log_start":"43'6600","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17170456,"num_objects":4193,"num_object_clones":0,"num_object_copies":8386,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4193,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14609,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'16670","reported_seq":16756,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:06:20.410063+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:06:20.410063+0000","last_peered":"2024-09-10T16:06:20.410063+0000","last_clean":"2024-09-10T16:06:20.410063+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:06:20.410063+0000","last_undegraded":"2024-09-10T16:06:20.410063+0000","last_fullsized":"2024-09-10T16:06:20.410063+0000","mapping_epoch":35,"log_start":"43'6600","ondisk_log_start":"43'6600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16343040,"num_objects":3990,"num_object_clones":0,"num_object_copies":7980,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3990,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16670,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'16791","reported_seq":16926,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:06:19.777534+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:06:19.777534+0000","last_peered":"2024-09-10T16:06:19.777534+0000","last_clean":"2024-09-10T16:06:19.777534+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:06:19.777534+0000","last_undegraded":"2024-09-10T16:06:19.777534+0000","last_fullsized":"2024-09-10T16:06:19.777534+0000","mapping_epoch":34,"log_start":"43'6700","ondisk_log_start":"43'6700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16794083,"num_objects":4101,"num_object_clones":0,"num_object_copies":8202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4101,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13502,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'16960","reported_seq":17103,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:19.795178+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:06:19.795178+0000","last_peered":"2024-09-10T16:06:19.795178+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:06:19.795178+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"43'6900","ondisk_log_start":"43'6900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16798179,"num_objects":4102,"num_object_clones":0,"num_object_copies":8204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4102,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13572,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-269},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'16835","reported_seq":16976,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:19.815189+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:06:19.815189+0000","last_peered":"2024-09-10T16:06:19.815189+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:06:19.815189+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"43'6800","ondisk_log_start":"43'6800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17396678,"num_objects":4249,"num_object_clones":0,"num_object_copies":8498,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4249,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13334,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":12},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'17087","reported_seq":17222,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:06:19.828743+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:06:19.828743+0000","last_peered":"2024-09-10T16:06:19.828743+0000","last_clean":"2024-09-10T16:06:19.828743+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:06:19.828743+0000","last_undegraded":"2024-09-10T16:06:19.828743+0000","last_fullsized":"2024-09-10T16:06:19.828743+0000","mapping_epoch":34,"log_start":"43'7000","ondisk_log_start":"43'7000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17313792,"num_objects":4227,"num_object_clones":0,"num_object_copies":8454,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4227,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13934,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.876895+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:05:19.876895+0000","last_peered":"2024-09-10T16:05:19.876895+0000","last_clean":"2024-09-10T16:05:19.876895+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:05:19.876895+0000","last_undegraded":"2024-09-10T16:05:19.876895+0000","last_fullsized":"2024-09-10T16:05:19.876895+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'16331","reported_seq":16476,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:20.603612+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:06:20.603612+0000","last_peered":"2024-09-10T16:06:20.603612+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:06:20.603612+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"43'6300","ondisk_log_start":"43'6300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16900579,"num_objects":4127,"num_object_clones":0,"num_object_copies":8254,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4127,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14150,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":4}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":99,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.875776+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:05:19.875776+0000","last_peered":"2024-09-10T16:05:19.875776+0000","last_clean":"2024-09-10T16:05:19.875776+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:05:19.875776+0000","last_undegraded":"2024-09-10T16:05:19.875776+0000","last_fullsized":"2024-09-10T16:05:19.875776+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'16692","reported_seq":16828,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:19.837546+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:06:19.837546+0000","last_peered":"2024-09-10T16:06:19.837546+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:06:19.837546+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"43'6600","ondisk_log_start":"43'6600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17276928,"num_objects":4218,"num_object_clones":0,"num_object_copies":8436,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4218,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13196,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-37}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":109,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.871983+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:05:19.871983+0000","last_peered":"2024-09-10T16:05:19.871983+0000","last_clean":"2024-09-10T16:05:19.871983+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:05:19.871983+0000","last_undegraded":"2024-09-10T16:05:19.871983+0000","last_fullsized":"2024-09-10T16:05:19.871983+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'16729","reported_seq":16869,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:20.504267+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:06:20.504267+0000","last_peered":"2024-09-10T16:06:20.504267+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:06:20.504267+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"43'6700","ondisk_log_start":"43'6700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16597475,"num_objects":4053,"num_object_clones":0,"num_object_copies":8106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4053,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14409,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-251}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877148+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:05:19.877148+0000","last_peered":"2024-09-10T16:05:19.877148+0000","last_clean":"2024-09-10T16:05:19.877148+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:05:19.877148+0000","last_undegraded":"2024-09-10T16:05:19.877148+0000","last_fullsized":"2024-09-10T16:05:19.877148+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'17043","reported_seq":19844,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:20.586929+0000","last_change":"2024-09-10T16:05:18.271152+0000","last_active":"2024-09-10T16:06:20.586929+0000","last_peered":"2024-09-10T16:06:20.586929+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:06:20.586929+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"43'7000","ondisk_log_start":"43'7000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16904192,"num_objects":4127,"num_object_clones":0,"num_object_copies":8254,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3020,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4127,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14597,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1334,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1107}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'16724","reported_seq":16864,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:19.750142+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:06:19.750142+0000","last_peered":"2024-09-10T16:06:19.750142+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:06:19.750142+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"43'6700","ondisk_log_start":"43'6700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16850944,"num_objects":4114,"num_object_clones":0,"num_object_copies":8228,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4114,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13355,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-190}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877654+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:05:19.877654+0000","last_peered":"2024-09-10T16:05:19.877654+0000","last_clean":"2024-09-10T16:05:19.877654+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:05:19.877654+0000","last_undegraded":"2024-09-10T16:05:19.877654+0000","last_fullsized":"2024-09-10T16:05:19.877654+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'16446","reported_seq":25017,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:06:17.996461+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:06:17.996461+0000","last_peered":"2024-09-10T16:06:17.996461+0000","last_clean":"2024-09-10T16:06:17.996461+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:06:17.996461+0000","last_undegraded":"2024-09-10T16:06:17.996461+0000","last_fullsized":"2024-09-10T16:06:17.996461+0000","mapping_epoch":46,"log_start":"43'6400","ondisk_log_start":"43'6400","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17006592,"num_objects":4152,"num_object_clones":0,"num_object_copies":8304,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4152,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12836,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":99,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.876013+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:05:19.876013+0000","last_peered":"2024-09-10T16:05:19.876013+0000","last_clean":"2024-09-10T16:05:19.876013+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:05:19.876013+0000","last_undegraded":"2024-09-10T16:05:19.876013+0000","last_fullsized":"2024-09-10T16:05:19.876013+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'16974","reported_seq":17121,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:19.835536+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:06:19.835536+0000","last_peered":"2024-09-10T16:06:19.835536+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:06:19.835536+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"43'6900","ondisk_log_start":"43'6900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16687104,"num_objects":4074,"num_object_clones":0,"num_object_copies":8148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4074,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13229,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-297}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877349+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:05:19.877349+0000","last_peered":"2024-09-10T16:05:19.877349+0000","last_clean":"2024-09-10T16:05:19.877349+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:05:19.877349+0000","last_undegraded":"2024-09-10T16:05:19.877349+0000","last_fullsized":"2024-09-10T16:05:19.877349+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877269+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:05:19.877269+0000","last_peered":"2024-09-10T16:05:19.877269+0000","last_clean":"2024-09-10T16:05:19.877269+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:05:19.877269+0000","last_undegraded":"2024-09-10T16:05:19.877269+0000","last_fullsized":"2024-09-10T16:05:19.877269+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'17027","reported_seq":17177,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:19.793948+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:06:19.793948+0000","last_peered":"2024-09-10T16:06:19.793948+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:06:19.793948+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"43'7000","ondisk_log_start":"43'7000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17085382,"num_objects":4173,"num_object_clones":0,"num_object_copies":8346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4173,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13190,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-150},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":270439158,"num_objects":66035,"num_object_clones":0,"num_object_copies":132070,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41577,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66035,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":226071,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9812,"num_bytes_recovered":33095680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":753176576,"data_stored":753034751,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160797,"ondisk_log_size":160797,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986276,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":616076,"kb_used_data":269244,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93755764,"statfs":{"total":96636764160,"available":96005902336,"internally_reserved":0,"allocated":275705856,"data_stored":275403172,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":355135146},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2],"upper_bound":16},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659221,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":544088,"kb_used_data":201412,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93827752,"statfs":{"total":96636764160,"available":96079618048,"internally_reserved":0,"allocated":206245888,"data_stored":205913836,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":350875310},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,1,3,6,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182504,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":704836,"kb_used_data":267796,"kb_used_omap":4,"kb_used_meta":436987,"kb_avail":93667004,"statfs":{"total":96636764160,"available":95915012096,"internally_reserved":0,"allocated":274223104,"data_stored":273920415,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":447475370},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,3,2,1,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":272666624,"data_stored":272630494,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":205656064,"data_stored":205590571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":274853888,"data_stored":274813686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:06:22.447 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:06:22.988 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:06:22.989 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:06:23.010 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":313,"stamp":"2024-09-10T16:06:21.707473+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":270439177,"num_objects":66037,"num_object_clones":0,"num_object_copies":132074,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41577,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66037,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":226073,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9814,"num_bytes_recovered":33095699,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160800,"ondisk_log_size":160800,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1865000,"kb_used_data":738452,"kb_used_omap":12,"kb_used_meta":1126451,"kb_avail":281250520,"statfs":{"total":289910292480,"available":288000532480,"internally_reserved":0,"allocated":756174848,"data_stored":755237423,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1153485826},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,4,6,8,2,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-27025408,"num_objects":-6598,"num_object_clones":0,"num_object_copies":-13196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-318,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-6598,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6598,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":318,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005368"},"pg_stats":[{"pgid":"2.f","version":"47'16626","reported_seq":16705,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:19.794366+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:06:19.794366+0000","last_peered":"2024-09-10T16:06:19.794366+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:06:19.794366+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"43'6600","ondisk_log_start":"43'6600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16572416,"num_objects":4046,"num_object_clones":0,"num_object_copies":8092,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4046,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16626,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-213}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'16757","reported_seq":16875,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:06:20.527569+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:06:20.527569+0000","last_peered":"2024-09-10T16:06:20.527569+0000","last_clean":"2024-09-10T16:06:20.527569+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:06:20.527569+0000","last_undegraded":"2024-09-10T16:06:20.527569+0000","last_fullsized":"2024-09-10T16:06:20.527569+0000","mapping_epoch":35,"log_start":"43'6700","ondisk_log_start":"43'6700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16741318,"num_objects":4089,"num_object_clones":0,"num_object_copies":8178,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4089,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":14862,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'16605","reported_seq":25288,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:06:18.058769+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:06:18.058769+0000","last_peered":"2024-09-10T16:06:18.058769+0000","last_clean":"2024-09-10T16:06:18.058769+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:06:18.058769+0000","last_undegraded":"2024-09-10T16:06:18.058769+0000","last_fullsized":"2024-09-10T16:06:18.058769+0000","mapping_epoch":44,"log_start":"43'6600","ondisk_log_start":"43'6600","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17170456,"num_objects":4193,"num_object_clones":0,"num_object_copies":8386,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4193,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14609,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'16670","reported_seq":16756,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:06:20.410063+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:06:20.410063+0000","last_peered":"2024-09-10T16:06:20.410063+0000","last_clean":"2024-09-10T16:06:20.410063+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:06:20.410063+0000","last_undegraded":"2024-09-10T16:06:20.410063+0000","last_fullsized":"2024-09-10T16:06:20.410063+0000","mapping_epoch":35,"log_start":"43'6600","ondisk_log_start":"43'6600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16343040,"num_objects":3990,"num_object_clones":0,"num_object_copies":7980,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3990,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16670,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'16791","reported_seq":16926,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:06:19.777534+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:06:19.777534+0000","last_peered":"2024-09-10T16:06:19.777534+0000","last_clean":"2024-09-10T16:06:19.777534+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:06:19.777534+0000","last_undegraded":"2024-09-10T16:06:19.777534+0000","last_fullsized":"2024-09-10T16:06:19.777534+0000","mapping_epoch":34,"log_start":"43'6700","ondisk_log_start":"43'6700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16794083,"num_objects":4101,"num_object_clones":0,"num_object_copies":8202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4101,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13502,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'16960","reported_seq":17103,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:19.795178+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:06:19.795178+0000","last_peered":"2024-09-10T16:06:19.795178+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:06:19.795178+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"43'6900","ondisk_log_start":"43'6900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16798179,"num_objects":4102,"num_object_clones":0,"num_object_copies":8204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4102,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13572,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-269},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'16835","reported_seq":16976,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:19.815189+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:06:19.815189+0000","last_peered":"2024-09-10T16:06:19.815189+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:06:19.815189+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"43'6800","ondisk_log_start":"43'6800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17396678,"num_objects":4249,"num_object_clones":0,"num_object_copies":8498,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4249,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13334,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":12},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'17087","reported_seq":17222,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:06:19.828743+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:06:19.828743+0000","last_peered":"2024-09-10T16:06:19.828743+0000","last_clean":"2024-09-10T16:06:19.828743+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:06:19.828743+0000","last_undegraded":"2024-09-10T16:06:19.828743+0000","last_fullsized":"2024-09-10T16:06:19.828743+0000","mapping_epoch":34,"log_start":"43'7000","ondisk_log_start":"43'7000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17313792,"num_objects":4227,"num_object_clones":0,"num_object_copies":8454,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4227,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13934,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.876895+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:05:19.876895+0000","last_peered":"2024-09-10T16:05:19.876895+0000","last_clean":"2024-09-10T16:05:19.876895+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:05:19.876895+0000","last_undegraded":"2024-09-10T16:05:19.876895+0000","last_fullsized":"2024-09-10T16:05:19.876895+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'16331","reported_seq":16476,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:20.603612+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:06:20.603612+0000","last_peered":"2024-09-10T16:06:20.603612+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:06:20.603612+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"43'6300","ondisk_log_start":"43'6300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16900579,"num_objects":4127,"num_object_clones":0,"num_object_copies":8254,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4127,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14150,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":4}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":99,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.875776+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:05:19.875776+0000","last_peered":"2024-09-10T16:05:19.875776+0000","last_clean":"2024-09-10T16:05:19.875776+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:05:19.875776+0000","last_undegraded":"2024-09-10T16:05:19.875776+0000","last_fullsized":"2024-09-10T16:05:19.875776+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'16692","reported_seq":16828,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:19.837546+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:06:19.837546+0000","last_peered":"2024-09-10T16:06:19.837546+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:06:19.837546+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"43'6600","ondisk_log_start":"43'6600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17276928,"num_objects":4218,"num_object_clones":0,"num_object_copies":8436,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4218,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13196,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-37}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":109,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.871983+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:05:19.871983+0000","last_peered":"2024-09-10T16:05:19.871983+0000","last_clean":"2024-09-10T16:05:19.871983+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:05:19.871983+0000","last_undegraded":"2024-09-10T16:05:19.871983+0000","last_fullsized":"2024-09-10T16:05:19.871983+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'16729","reported_seq":16869,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:20.504267+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:06:20.504267+0000","last_peered":"2024-09-10T16:06:20.504267+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:06:20.504267+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"43'6700","ondisk_log_start":"43'6700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16597475,"num_objects":4053,"num_object_clones":0,"num_object_copies":8106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4053,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14409,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-251}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877148+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:05:19.877148+0000","last_peered":"2024-09-10T16:05:19.877148+0000","last_clean":"2024-09-10T16:05:19.877148+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:05:19.877148+0000","last_undegraded":"2024-09-10T16:05:19.877148+0000","last_fullsized":"2024-09-10T16:05:19.877148+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'17043","reported_seq":19844,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:20.586929+0000","last_change":"2024-09-10T16:05:18.271152+0000","last_active":"2024-09-10T16:06:20.586929+0000","last_peered":"2024-09-10T16:06:20.586929+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:06:20.586929+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"43'7000","ondisk_log_start":"43'7000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16904192,"num_objects":4127,"num_object_clones":0,"num_object_copies":8254,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3020,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4127,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14597,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1334,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1107}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'16724","reported_seq":16864,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:19.750142+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:06:19.750142+0000","last_peered":"2024-09-10T16:06:19.750142+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:06:19.750142+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"43'6700","ondisk_log_start":"43'6700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16850944,"num_objects":4114,"num_object_clones":0,"num_object_copies":8228,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4114,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13355,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-190}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877654+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:05:19.877654+0000","last_peered":"2024-09-10T16:05:19.877654+0000","last_clean":"2024-09-10T16:05:19.877654+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:05:19.877654+0000","last_undegraded":"2024-09-10T16:05:19.877654+0000","last_fullsized":"2024-09-10T16:05:19.877654+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'16446","reported_seq":25017,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:06:17.996461+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:06:17.996461+0000","last_peered":"2024-09-10T16:06:17.996461+0000","last_clean":"2024-09-10T16:06:17.996461+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:06:17.996461+0000","last_undegraded":"2024-09-10T16:06:17.996461+0000","last_fullsized":"2024-09-10T16:06:17.996461+0000","mapping_epoch":46,"log_start":"43'6400","ondisk_log_start":"43'6400","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17006592,"num_objects":4152,"num_object_clones":0,"num_object_copies":8304,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4152,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12836,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":99,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.876013+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:05:19.876013+0000","last_peered":"2024-09-10T16:05:19.876013+0000","last_clean":"2024-09-10T16:05:19.876013+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:05:19.876013+0000","last_undegraded":"2024-09-10T16:05:19.876013+0000","last_fullsized":"2024-09-10T16:05:19.876013+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'16974","reported_seq":17121,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:19.835536+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:06:19.835536+0000","last_peered":"2024-09-10T16:06:19.835536+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:06:19.835536+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"43'6900","ondisk_log_start":"43'6900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16687104,"num_objects":4074,"num_object_clones":0,"num_object_copies":8148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4074,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13229,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-297}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877349+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:05:19.877349+0000","last_peered":"2024-09-10T16:05:19.877349+0000","last_clean":"2024-09-10T16:05:19.877349+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:05:19.877349+0000","last_undegraded":"2024-09-10T16:05:19.877349+0000","last_fullsized":"2024-09-10T16:05:19.877349+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877269+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:05:19.877269+0000","last_peered":"2024-09-10T16:05:19.877269+0000","last_clean":"2024-09-10T16:05:19.877269+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:05:19.877269+0000","last_undegraded":"2024-09-10T16:05:19.877269+0000","last_fullsized":"2024-09-10T16:05:19.877269+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'17027","reported_seq":17177,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:19.793948+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:06:19.793948+0000","last_peered":"2024-09-10T16:06:19.793948+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:06:19.793948+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"43'7000","ondisk_log_start":"43'7000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17085382,"num_objects":4173,"num_object_clones":0,"num_object_copies":8346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4173,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13190,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-150},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":270439158,"num_objects":66035,"num_object_clones":0,"num_object_copies":132070,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41577,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66035,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":226071,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9812,"num_bytes_recovered":33095680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":753176576,"data_stored":753034751,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160797,"ondisk_log_size":160797,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986276,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":616076,"kb_used_data":269244,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93755764,"statfs":{"total":96636764160,"available":96005902336,"internally_reserved":0,"allocated":275705856,"data_stored":275403172,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":355135146},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2],"upper_bound":16},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659221,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":544088,"kb_used_data":201412,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93827752,"statfs":{"total":96636764160,"available":96079618048,"internally_reserved":0,"allocated":206245888,"data_stored":205913836,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":350875310},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,1,3,6,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182504,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":704836,"kb_used_data":267796,"kb_used_omap":4,"kb_used_meta":436987,"kb_avail":93667004,"statfs":{"total":96636764160,"available":95915012096,"internally_reserved":0,"allocated":274223104,"data_stored":273920415,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":447475370},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,3,2,1,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":272666624,"data_stored":272630494,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":205656064,"data_stored":205590571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":274853888,"data_stored":274813686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:06:23.012 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-10T16:06:23.012 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "47'16626", 'reported_seq': 16705, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:06:19.794366+0000', 'last_change': '2024-09-10T15:59:16.281558+0000', 'last_active': '2024-09-10T16:06:19.794366+0000', 'last_peered': '2024-09-10T16:06:19.794366+0000', 'last_clean': '2024-09-10T15:59:14.132456+0000', 'last_became_active': '2024-09-10T15:59:16.274407+0000', 'last_became_peered': '2024-09-10T15:59:16.274407+0000', 'last_unstale': '2024-09-10T16:06:19.794366+0000', 'last_undegraded': '2024-09-10T15:59:15.851279+0000', 'last_fullsized': '2024-09-10T15:59:15.774356+0000', 'mapping_epoch': 42, 'log_start': "43'6600", 'ondisk_log_start': "43'6600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'objects_scrubbed': 0, 'log_size': 10026, 'log_dups_size': 0, 'ondisk_log_size': 10026, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16572416, 'num_objects': 4046, 'num_object_clones': 0, 'num_object_copies': 8092, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4259, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4046, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16626, 'num_write_kb': 41344, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -213}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:06:23.012 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-10T16:06:23.012 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "47'16960", 'reported_seq': 17103, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:06:19.795178+0000', 'last_change': '2024-09-10T15:59:16.308264+0000', 'last_active': '2024-09-10T16:06:19.795178+0000', 'last_peered': '2024-09-10T16:06:19.795178+0000', 'last_clean': '2024-09-10T15:59:14.111551+0000', 'last_became_active': '2024-09-10T15:59:16.300040+0000', 'last_became_peered': '2024-09-10T15:59:16.300040+0000', 'last_unstale': '2024-09-10T16:06:19.795178+0000', 'last_undegraded': '2024-09-10T15:59:15.878234+0000', 'last_fullsized': '2024-09-10T15:59:15.769328+0000', 'mapping_epoch': 42, 'log_start': "43'6900", 'ondisk_log_start': "43'6900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3442", 'last_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'objects_scrubbed': 3384, 'log_size': 10060, 'log_dups_size': 0, 'ondisk_log_size': 10060, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1292, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16798179, 'num_objects': 4102, 'num_object_clones': 0, 'num_object_copies': 8204, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4372, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4102, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 13572, 'num_write_kb': 28580, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -269}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:06:23.012 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-10T16:06:23.012 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "47'16835", 'reported_seq': 16976, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:06:19.815189+0000', 'last_change': '2024-09-10T15:59:16.331899+0000', 'last_active': '2024-09-10T16:06:19.815189+0000', 'last_peered': '2024-09-10T16:06:19.815189+0000', 'last_clean': '2024-09-10T15:59:14.139436+0000', 'last_became_active': '2024-09-10T15:59:16.318715+0000', 'last_became_peered': '2024-09-10T15:59:16.318715+0000', 'last_unstale': '2024-09-10T16:06:19.815189+0000', 'last_undegraded': '2024-09-10T15:59:15.877524+0000', 'last_fullsized': '2024-09-10T15:59:15.769042+0000', 'mapping_epoch': 42, 'log_start': "43'6800", 'ondisk_log_start': "43'6800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3537", 'last_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'objects_scrubbed': 3497, 'log_size': 10035, 'log_dups_size': 0, 'ondisk_log_size': 10035, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1224, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17396678, 'num_objects': 4249, 'num_object_clones': 0, 'num_object_copies': 8498, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4237, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4249, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 13334, 'num_write_kb': 28172, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 12}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:06:23.012 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-10T16:06:23.012 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "47'16331", 'reported_seq': 16476, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:06:20.603612+0000', 'last_change': '2024-09-10T15:59:16.175680+0000', 'last_active': '2024-09-10T16:06:20.603612+0000', 'last_peered': '2024-09-10T16:06:20.603612+0000', 'last_clean': '2024-09-10T15:59:14.088187+0000', 'last_became_active': '2024-09-10T15:59:16.165810+0000', 'last_became_peered': '2024-09-10T15:59:16.165810+0000', 'last_unstale': '2024-09-10T16:06:20.603612+0000', 'last_undegraded': '2024-09-10T15:59:15.822098+0000', 'last_fullsized': '2024-09-10T15:59:15.769123+0000', 'mapping_epoch': 42, 'log_start': "43'6300", 'ondisk_log_start': "43'6300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2218", 'last_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'objects_scrubbed': 2175, 'log_size': 10031, 'log_dups_size': 0, 'ondisk_log_size': 10031, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 825, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16900579, 'num_objects': 4127, 'num_object_clones': 0, 'num_object_copies': 8254, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4124, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4127, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 14150, 'num_write_kb': 32204, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': 4}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:06:23.013 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-10T16:06:23.013 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "47'16692", 'reported_seq': 16828, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:06:19.837546+0000', 'last_change': '2024-09-10T15:59:16.277742+0000', 'last_active': '2024-09-10T16:06:19.837546+0000', 'last_peered': '2024-09-10T16:06:19.837546+0000', 'last_clean': '2024-09-10T15:59:14.129081+0000', 'last_became_active': '2024-09-10T15:59:16.269835+0000', 'last_became_peered': '2024-09-10T15:59:16.269835+0000', 'last_unstale': '2024-09-10T16:06:19.837546+0000', 'last_undegraded': '2024-09-10T15:59:15.860857+0000', 'last_fullsized': '2024-09-10T15:59:15.771410+0000', 'mapping_epoch': 42, 'log_start': "43'6600", 'ondisk_log_start': "43'6600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3557", 'last_scrub_stamp': '2024-09-10T15:59:02.493678+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:02.493678+0000', 'objects_scrubbed': 3488, 'log_size': 10092, 'log_dups_size': 0, 'ondisk_log_size': 10092, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1472, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17276928, 'num_objects': 4218, 'num_object_clones': 0, 'num_object_copies': 8436, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4257, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4218, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13196, 'num_write_kb': 27852, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -37}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:06:23.013 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-10T16:06:23.013 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "47'16729", 'reported_seq': 16869, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:06:20.504267+0000', 'last_change': '2024-09-10T15:59:16.284311+0000', 'last_active': '2024-09-10T16:06:20.504267+0000', 'last_peered': '2024-09-10T16:06:20.504267+0000', 'last_clean': '2024-09-10T15:59:14.116041+0000', 'last_became_active': '2024-09-10T15:59:16.274543+0000', 'last_became_peered': '2024-09-10T15:59:16.274543+0000', 'last_unstale': '2024-09-10T16:06:20.504267+0000', 'last_undegraded': '2024-09-10T15:59:15.812484+0000', 'last_fullsized': '2024-09-10T15:59:15.770885+0000', 'mapping_epoch': 42, 'log_start': "43'6700", 'ondisk_log_start': "43'6700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2356", 'last_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'objects_scrubbed': 2316, 'log_size': 10029, 'log_dups_size': 0, 'ondisk_log_size': 10029, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 783, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16597475, 'num_objects': 4053, 'num_object_clones': 0, 'num_object_copies': 8106, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4053, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 14409, 'num_write_kb': 32292, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': -251}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:06:23.013 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-10T16:06:23.013 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "47'17043", 'reported_seq': 19844, 'reported_epoch': 47, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:06:20.586929+0000', 'last_change': '2024-09-10T16:05:18.271152+0000', 'last_active': '2024-09-10T16:06:20.586929+0000', 'last_peered': '2024-09-10T16:06:20.586929+0000', 'last_clean': '2024-09-10T15:59:14.110654+0000', 'last_became_active': '2024-09-10T15:59:16.164645+0000', 'last_became_peered': '2024-09-10T15:59:16.164645+0000', 'last_unstale': '2024-09-10T16:06:20.586929+0000', 'last_undegraded': '2024-09-10T15:59:15.803593+0000', 'last_fullsized': '2024-09-10T15:59:15.769083+0000', 'mapping_epoch': 42, 'log_start': "43'7000", 'ondisk_log_start': "43'7000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2486", 'last_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'objects_scrubbed': 2440, 'log_size': 10043, 'log_dups_size': 0, 'ondisk_log_size': 10043, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 792, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16904192, 'num_objects': 4127, 'num_object_clones': 0, 'num_object_copies': 8254, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3020, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4127, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14597, 'num_write_kb': 32568, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1334, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 1107}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:06:23.013 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-10T16:06:23.013 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "47'16724", 'reported_seq': 16864, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:06:19.750142+0000', 'last_change': '2024-09-10T15:59:16.315641+0000', 'last_active': '2024-09-10T16:06:19.750142+0000', 'last_peered': '2024-09-10T16:06:19.750142+0000', 'last_clean': '2024-09-10T15:59:14.117001+0000', 'last_became_active': '2024-09-10T15:59:16.302245+0000', 'last_became_peered': '2024-09-10T15:59:16.302245+0000', 'last_unstale': '2024-09-10T16:06:19.750142+0000', 'last_undegraded': '2024-09-10T15:59:15.845012+0000', 'last_fullsized': '2024-09-10T15:59:15.770337+0000', 'mapping_epoch': 42, 'log_start': "43'6700", 'ondisk_log_start': "43'6700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3404", 'last_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'objects_scrubbed': 3367, 'log_size': 10024, 'log_dups_size': 0, 'ondisk_log_size': 10024, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 983, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16850944, 'num_objects': 4114, 'num_object_clones': 0, 'num_object_copies': 8228, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4114, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13355, 'num_write_kb': 28204, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -190}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:06:23.013 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-10T16:06:23.013 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "47'16974", 'reported_seq': 17121, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:06:19.835536+0000', 'last_change': '2024-09-10T15:59:16.311291+0000', 'last_active': '2024-09-10T16:06:19.835536+0000', 'last_peered': '2024-09-10T16:06:19.835536+0000', 'last_clean': '2024-09-10T15:59:14.116363+0000', 'last_became_active': '2024-09-10T15:59:16.301244+0000', 'last_became_peered': '2024-09-10T15:59:16.301244+0000', 'last_unstale': '2024-09-10T16:06:19.835536+0000', 'last_undegraded': '2024-09-10T15:59:15.883434+0000', 'last_fullsized': '2024-09-10T15:59:15.772756+0000', 'mapping_epoch': 42, 'log_start': "43'6900", 'ondisk_log_start': "43'6900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3789", 'last_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'objects_scrubbed': 3741, 'log_size': 10074, 'log_dups_size': 0, 'ondisk_log_size': 10074, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1440, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16687104, 'num_objects': 4074, 'num_object_clones': 0, 'num_object_copies': 8148, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4373, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4074, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13229, 'num_write_kb': 27124, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -297}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:06:23.014 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-10T16:06:23.014 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "47'17027", 'reported_seq': 17177, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:06:19.793948+0000', 'last_change': '2024-09-10T15:59:16.313409+0000', 'last_active': '2024-09-10T16:06:19.793948+0000', 'last_peered': '2024-09-10T16:06:19.793948+0000', 'last_clean': '2024-09-10T15:59:14.129780+0000', 'last_became_active': '2024-09-10T15:59:16.300932+0000', 'last_became_peered': '2024-09-10T15:59:16.300932+0000', 'last_unstale': '2024-09-10T16:06:19.793948+0000', 'last_undegraded': '2024-09-10T15:59:15.887823+0000', 'last_fullsized': '2024-09-10T15:59:15.768513+0000', 'mapping_epoch': 42, 'log_start': "43'7000", 'ondisk_log_start': "43'7000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3880", 'last_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'objects_scrubbed': 3827, 'log_size': 10027, 'log_dups_size': 0, 'ondisk_log_size': 10027, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1397, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17085382, 'num_objects': 4173, 'num_object_clones': 0, 'num_object_copies': 8346, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4325, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4173, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13190, 'num_write_kb': 27072, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -150}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:06:23.014 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T16:06:23.716 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:06:23.716 INFO:teuthology.orchestra.run.smithi115.stdout:{"fsid":"7effee29-df05-4351-9741-ee203564ed81","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 41577/132074 objects degraded (31.480%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":547,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":47,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983954,"num_in_osds":3,"osd_in_since":1725983947,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":66037,"data_bytes":270439177,"bytes_used":1909760000,"bytes_avail":288000532480,"bytes_total":289910292480,"degraded_objects":41577,"degraded_total":132074,"degraded_ratio":0.31480079349455609,"recovering_objects_per_sec":26,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":318,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":549},"fsmap":{"epoch":1,"btime":"2024-09-10T15:57:14:967120+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T16:00:13.560776+0000","services":{}},"progress_events":{}} 2024-09-10T16:06:23.735 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-10T16:06:23.735 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:06:24.275 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:06:24.278 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:06:24.302 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":314,"stamp":"2024-09-10T16:06:23.708106+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":269148937,"num_objects":65722,"num_object_clones":0,"num_object_copies":131444,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41577,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":65722,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":226388,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9814,"num_bytes_recovered":33095699,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160815,"ondisk_log_size":160815,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1862244,"kb_used_data":731524,"kb_used_omap":12,"kb_used_meta":1130611,"kb_avail":281253276,"statfs":{"total":289910292480,"available":288003354624,"internally_reserved":0,"allocated":749080576,"data_stored":748143151,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1157745666},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,8,5,5,1,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-20099072,"num_objects":-4907,"num_object_clones":0,"num_object_copies":-9814,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-213,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4907,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4907,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":213,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005179"},"pg_stats":[{"pgid":"2.f","version":"47'16626","reported_seq":16705,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:19.794366+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:06:19.794366+0000","last_peered":"2024-09-10T16:06:19.794366+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:06:19.794366+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"43'6600","ondisk_log_start":"43'6600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16572416,"num_objects":4046,"num_object_clones":0,"num_object_copies":8092,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4046,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16626,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-213}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'16757","reported_seq":16875,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:06:20.527569+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:06:20.527569+0000","last_peered":"2024-09-10T16:06:20.527569+0000","last_clean":"2024-09-10T16:06:20.527569+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:06:20.527569+0000","last_undegraded":"2024-09-10T16:06:20.527569+0000","last_fullsized":"2024-09-10T16:06:20.527569+0000","mapping_epoch":35,"log_start":"43'6700","ondisk_log_start":"43'6700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16741318,"num_objects":4089,"num_object_clones":0,"num_object_copies":8178,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4089,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":14862,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'16756","reported_seq":25439,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:06:23.052407+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:06:23.052407+0000","last_peered":"2024-09-10T16:06:23.052407+0000","last_clean":"2024-09-10T16:06:23.052407+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:06:23.052407+0000","last_undegraded":"2024-09-10T16:06:23.052407+0000","last_fullsized":"2024-09-10T16:06:23.052407+0000","mapping_epoch":44,"log_start":"43'6700","ondisk_log_start":"43'6700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16551960,"num_objects":4042,"num_object_clones":0,"num_object_copies":8084,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4042,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14760,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'16670","reported_seq":16756,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:06:20.410063+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:06:20.410063+0000","last_peered":"2024-09-10T16:06:20.410063+0000","last_clean":"2024-09-10T16:06:20.410063+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:06:20.410063+0000","last_undegraded":"2024-09-10T16:06:20.410063+0000","last_fullsized":"2024-09-10T16:06:20.410063+0000","mapping_epoch":35,"log_start":"43'6600","ondisk_log_start":"43'6600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16343040,"num_objects":3990,"num_object_clones":0,"num_object_copies":7980,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3990,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16670,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'16791","reported_seq":16926,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:06:19.777534+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:06:19.777534+0000","last_peered":"2024-09-10T16:06:19.777534+0000","last_clean":"2024-09-10T16:06:19.777534+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:06:19.777534+0000","last_undegraded":"2024-09-10T16:06:19.777534+0000","last_fullsized":"2024-09-10T16:06:19.777534+0000","mapping_epoch":34,"log_start":"43'6700","ondisk_log_start":"43'6700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16794083,"num_objects":4101,"num_object_clones":0,"num_object_copies":8202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4101,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13502,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'16960","reported_seq":17103,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:19.795178+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:06:19.795178+0000","last_peered":"2024-09-10T16:06:19.795178+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:06:19.795178+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"43'6900","ondisk_log_start":"43'6900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16798179,"num_objects":4102,"num_object_clones":0,"num_object_copies":8204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4102,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13572,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-269},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'16835","reported_seq":16976,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:19.815189+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:06:19.815189+0000","last_peered":"2024-09-10T16:06:19.815189+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:06:19.815189+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"43'6800","ondisk_log_start":"43'6800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17396678,"num_objects":4249,"num_object_clones":0,"num_object_copies":8498,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4249,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13334,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":12},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'17087","reported_seq":17222,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:06:19.828743+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:06:19.828743+0000","last_peered":"2024-09-10T16:06:19.828743+0000","last_clean":"2024-09-10T16:06:19.828743+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:06:19.828743+0000","last_undegraded":"2024-09-10T16:06:19.828743+0000","last_fullsized":"2024-09-10T16:06:19.828743+0000","mapping_epoch":34,"log_start":"43'7000","ondisk_log_start":"43'7000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17313792,"num_objects":4227,"num_object_clones":0,"num_object_copies":8454,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4227,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13934,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.876895+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:05:19.876895+0000","last_peered":"2024-09-10T16:05:19.876895+0000","last_clean":"2024-09-10T16:05:19.876895+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:05:19.876895+0000","last_undegraded":"2024-09-10T16:05:19.876895+0000","last_fullsized":"2024-09-10T16:05:19.876895+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'16331","reported_seq":16476,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:20.603612+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:06:20.603612+0000","last_peered":"2024-09-10T16:06:20.603612+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:06:20.603612+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"43'6300","ondisk_log_start":"43'6300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16900579,"num_objects":4127,"num_object_clones":0,"num_object_copies":8254,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4127,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14150,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":4}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":99,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.875776+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:05:19.875776+0000","last_peered":"2024-09-10T16:05:19.875776+0000","last_clean":"2024-09-10T16:05:19.875776+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:05:19.875776+0000","last_undegraded":"2024-09-10T16:05:19.875776+0000","last_fullsized":"2024-09-10T16:05:19.875776+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'16692","reported_seq":16828,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:19.837546+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:06:19.837546+0000","last_peered":"2024-09-10T16:06:19.837546+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:06:19.837546+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"43'6600","ondisk_log_start":"43'6600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17276928,"num_objects":4218,"num_object_clones":0,"num_object_copies":8436,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4218,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13196,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-37}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":109,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.871983+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:05:19.871983+0000","last_peered":"2024-09-10T16:05:19.871983+0000","last_clean":"2024-09-10T16:05:19.871983+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:05:19.871983+0000","last_undegraded":"2024-09-10T16:05:19.871983+0000","last_fullsized":"2024-09-10T16:05:19.871983+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'16729","reported_seq":16869,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:20.504267+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:06:20.504267+0000","last_peered":"2024-09-10T16:06:20.504267+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:06:20.504267+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"43'6700","ondisk_log_start":"43'6700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16597475,"num_objects":4053,"num_object_clones":0,"num_object_copies":8106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4053,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14409,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-251}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877148+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:05:19.877148+0000","last_peered":"2024-09-10T16:05:19.877148+0000","last_clean":"2024-09-10T16:05:19.877148+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:05:19.877148+0000","last_undegraded":"2024-09-10T16:05:19.877148+0000","last_fullsized":"2024-09-10T16:05:19.877148+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'17043","reported_seq":19844,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:20.586929+0000","last_change":"2024-09-10T16:05:18.271152+0000","last_active":"2024-09-10T16:06:20.586929+0000","last_peered":"2024-09-10T16:06:20.586929+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:06:20.586929+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"43'7000","ondisk_log_start":"43'7000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16904192,"num_objects":4127,"num_object_clones":0,"num_object_copies":8254,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3020,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4127,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14597,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1334,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1107}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'16724","reported_seq":16864,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:19.750142+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:06:19.750142+0000","last_peered":"2024-09-10T16:06:19.750142+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:06:19.750142+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"43'6700","ondisk_log_start":"43'6700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16850944,"num_objects":4114,"num_object_clones":0,"num_object_copies":8228,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4114,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13355,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-190}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877654+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:05:19.877654+0000","last_peered":"2024-09-10T16:05:19.877654+0000","last_clean":"2024-09-10T16:05:19.877654+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:05:19.877654+0000","last_undegraded":"2024-09-10T16:05:19.877654+0000","last_fullsized":"2024-09-10T16:05:19.877654+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'16610","reported_seq":25181,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:06:23.112658+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:06:23.112658+0000","last_peered":"2024-09-10T16:06:23.112658+0000","last_clean":"2024-09-10T16:06:23.112658+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:06:23.112658+0000","last_undegraded":"2024-09-10T16:06:23.112658+0000","last_fullsized":"2024-09-10T16:06:23.112658+0000","mapping_epoch":46,"log_start":"43'6600","ondisk_log_start":"43'6600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16334848,"num_objects":3988,"num_object_clones":0,"num_object_copies":7976,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3988,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13000,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":99,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.876013+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:05:19.876013+0000","last_peered":"2024-09-10T16:05:19.876013+0000","last_clean":"2024-09-10T16:05:19.876013+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:05:19.876013+0000","last_undegraded":"2024-09-10T16:05:19.876013+0000","last_fullsized":"2024-09-10T16:05:19.876013+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'16974","reported_seq":17121,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:19.835536+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:06:19.835536+0000","last_peered":"2024-09-10T16:06:19.835536+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:06:19.835536+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"43'6900","ondisk_log_start":"43'6900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16687104,"num_objects":4074,"num_object_clones":0,"num_object_copies":8148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4074,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13229,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-297}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877349+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:05:19.877349+0000","last_peered":"2024-09-10T16:05:19.877349+0000","last_clean":"2024-09-10T16:05:19.877349+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:05:19.877349+0000","last_undegraded":"2024-09-10T16:05:19.877349+0000","last_fullsized":"2024-09-10T16:05:19.877349+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877269+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:05:19.877269+0000","last_peered":"2024-09-10T16:05:19.877269+0000","last_clean":"2024-09-10T16:05:19.877269+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:05:19.877269+0000","last_undegraded":"2024-09-10T16:05:19.877269+0000","last_fullsized":"2024-09-10T16:05:19.877269+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'17027","reported_seq":17177,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:19.793948+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:06:19.793948+0000","last_peered":"2024-09-10T16:06:19.793948+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:06:19.793948+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"43'7000","ondisk_log_start":"43'7000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17085382,"num_objects":4173,"num_object_clones":0,"num_object_copies":8346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4173,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13190,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-150},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":269148918,"num_objects":65720,"num_object_clones":0,"num_object_copies":131440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41577,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":65720,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":226386,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9812,"num_bytes_recovered":33095680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":745861120,"data_stored":745719295,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160812,"ondisk_log_size":160812,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986276,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":616076,"kb_used_data":269244,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93755764,"statfs":{"total":96636764160,"available":96005902336,"internally_reserved":0,"allocated":275705856,"data_stored":275403172,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":355135146},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2],"upper_bound":16},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659222,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":541332,"kb_used_data":194484,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93830508,"statfs":{"total":96636764160,"available":96082440192,"internally_reserved":0,"allocated":199151616,"data_stored":198819564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":355135150},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,5,2,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182504,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":704836,"kb_used_data":267796,"kb_used_omap":4,"kb_used_meta":436987,"kb_avail":93667004,"statfs":{"total":96636764160,"available":95915012096,"internally_reserved":0,"allocated":274223104,"data_stored":273920415,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":447475370},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,3,2,1,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":272666624,"data_stored":272630494,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":198340608,"data_stored":198275115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":274853888,"data_stored":274813686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:06:27.304 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:06:27.863 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:06:27.865 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:06:27.885 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":316,"stamp":"2024-09-10T16:06:27.709539+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":261026569,"num_objects":63739,"num_object_clones":0,"num_object_copies":127478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41469,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63739,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":228371,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9922,"num_bytes_recovered":33095699,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160898,"ondisk_log_size":160898,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1842816,"kb_used_data":720436,"kb_used_omap":12,"kb_used_meta":1122291,"kb_avail":281272704,"statfs":{"total":289910292480,"available":288023248896,"internally_reserved":0,"allocated":737726464,"data_stored":736789039,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1149225986},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,7,6,5,1,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18788352,"num_objects":-4587,"num_object_clones":0,"num_object_copies":-9174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-217,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4587,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4587,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":217,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004329"},"pg_stats":[{"pgid":"2.f","version":"47'16759","reported_seq":16833,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:24.802942+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:06:24.802942+0000","last_peered":"2024-09-10T16:06:24.802942+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:06:24.802942+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"43'6700","ondisk_log_start":"43'6700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16027648,"num_objects":3913,"num_object_clones":0,"num_object_copies":7826,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3913,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16759,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-346}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'16884","reported_seq":17002,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:06:25.633742+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:06:25.633742+0000","last_peered":"2024-09-10T16:06:25.633742+0000","last_clean":"2024-09-10T16:06:25.633742+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:06:25.633742+0000","last_undegraded":"2024-09-10T16:06:25.633742+0000","last_fullsized":"2024-09-10T16:06:25.633742+0000","mapping_epoch":35,"log_start":"43'6800","ondisk_log_start":"43'6800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16221126,"num_objects":3962,"num_object_clones":0,"num_object_copies":7924,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3962,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":14989,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'16756","reported_seq":25439,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:06:23.052407+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:06:23.052407+0000","last_peered":"2024-09-10T16:06:23.052407+0000","last_clean":"2024-09-10T16:06:23.052407+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:06:23.052407+0000","last_undegraded":"2024-09-10T16:06:23.052407+0000","last_fullsized":"2024-09-10T16:06:23.052407+0000","mapping_epoch":44,"log_start":"43'6700","ondisk_log_start":"43'6700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16551960,"num_objects":4042,"num_object_clones":0,"num_object_copies":8084,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4042,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14760,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'16805","reported_seq":16891,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:06:25.617375+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:06:25.617375+0000","last_peered":"2024-09-10T16:06:25.617375+0000","last_clean":"2024-09-10T16:06:25.617375+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:06:25.617375+0000","last_undegraded":"2024-09-10T16:06:25.617375+0000","last_fullsized":"2024-09-10T16:06:25.617375+0000","mapping_epoch":35,"log_start":"43'6800","ondisk_log_start":"43'6800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15790080,"num_objects":3855,"num_object_clones":0,"num_object_copies":7710,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3855,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16805,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'16926","reported_seq":17061,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:06:24.833253+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:06:24.833253+0000","last_peered":"2024-09-10T16:06:24.833253+0000","last_clean":"2024-09-10T16:06:24.833253+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:06:24.833253+0000","last_undegraded":"2024-09-10T16:06:24.833253+0000","last_fullsized":"2024-09-10T16:06:24.833253+0000","mapping_epoch":34,"log_start":"43'6900","ondisk_log_start":"43'6900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16241123,"num_objects":3966,"num_object_clones":0,"num_object_copies":7932,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3966,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13637,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'17098","reported_seq":17240,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:24.829970+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:06:24.829970+0000","last_peered":"2024-09-10T16:06:24.829970+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:06:24.829970+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"43'7000","ondisk_log_start":"43'7000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16232931,"num_objects":3964,"num_object_clones":0,"num_object_copies":7928,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3964,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13710,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-407},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'17008","reported_seq":17151,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:24.835129+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:06:24.835129+0000","last_peered":"2024-09-10T16:06:24.835129+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:06:24.835129+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"43'7000","ondisk_log_start":"43'7000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16688070,"num_objects":4076,"num_object_clones":0,"num_object_copies":8152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4076,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13507,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-161},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'17220","reported_seq":17355,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:06:24.842355+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:06:24.842355+0000","last_peered":"2024-09-10T16:06:24.842355+0000","last_clean":"2024-09-10T16:06:24.842355+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:06:24.842355+0000","last_undegraded":"2024-09-10T16:06:24.842355+0000","last_fullsized":"2024-09-10T16:06:24.842355+0000","mapping_epoch":34,"log_start":"43'7200","ondisk_log_start":"43'7200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16769024,"num_objects":4094,"num_object_clones":0,"num_object_copies":8188,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4094,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14067,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.876895+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:05:19.876895+0000","last_peered":"2024-09-10T16:05:19.876895+0000","last_clean":"2024-09-10T16:05:19.876895+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:05:19.876895+0000","last_undegraded":"2024-09-10T16:05:19.876895+0000","last_fullsized":"2024-09-10T16:05:19.876895+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'16478","reported_seq":16623,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:25.644151+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:06:25.644151+0000","last_peered":"2024-09-10T16:06:25.644151+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:06:25.644151+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"43'6400","ondisk_log_start":"43'6400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16298467,"num_objects":3980,"num_object_clones":0,"num_object_copies":7960,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3980,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14297,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-143}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":99,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.875776+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:05:19.875776+0000","last_peered":"2024-09-10T16:05:19.875776+0000","last_clean":"2024-09-10T16:05:19.875776+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:05:19.875776+0000","last_undegraded":"2024-09-10T16:05:19.875776+0000","last_fullsized":"2024-09-10T16:05:19.875776+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'16843","reported_seq":16979,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:24.814446+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:06:24.814446+0000","last_peered":"2024-09-10T16:06:24.814446+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:06:24.814446+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"43'6800","ondisk_log_start":"43'6800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16658432,"num_objects":4067,"num_object_clones":0,"num_object_copies":8134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4067,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13347,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-188}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":110,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.872023+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:05:19.872023+0000","last_peered":"2024-09-10T16:05:19.872023+0000","last_clean":"2024-09-10T16:05:19.872023+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:05:19.872023+0000","last_undegraded":"2024-09-10T16:05:19.872023+0000","last_fullsized":"2024-09-10T16:05:19.872023+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'16873","reported_seq":17013,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:25.576662+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:06:25.576662+0000","last_peered":"2024-09-10T16:06:25.576662+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:06:25.576662+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"43'6800","ondisk_log_start":"43'6800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16007651,"num_objects":3909,"num_object_clones":0,"num_object_copies":7818,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3909,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14553,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-395}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877148+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:05:19.877148+0000","last_peered":"2024-09-10T16:05:19.877148+0000","last_clean":"2024-09-10T16:05:19.877148+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:05:19.877148+0000","last_undegraded":"2024-09-10T16:05:19.877148+0000","last_fullsized":"2024-09-10T16:05:19.877148+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'17188","reported_seq":20208,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:25.613510+0000","last_change":"2024-09-10T16:05:18.271152+0000","last_active":"2024-09-10T16:06:25.613510+0000","last_peered":"2024-09-10T16:06:25.613510+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:06:25.613510+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"43'7100","ondisk_log_start":"43'7100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16310272,"num_objects":3982,"num_object_clones":0,"num_object_copies":7964,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2912,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3982,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14742,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1442,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1070}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'16865","reported_seq":17002,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:24.860849+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:06:24.860849+0000","last_peered":"2024-09-10T16:06:24.860849+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:06:24.860849+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"43'6800","ondisk_log_start":"43'6800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16273408,"num_objects":3973,"num_object_clones":0,"num_object_copies":7946,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3973,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13496,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-331}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877654+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:05:19.877654+0000","last_peered":"2024-09-10T16:05:19.877654+0000","last_clean":"2024-09-10T16:05:19.877654+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:05:19.877654+0000","last_undegraded":"2024-09-10T16:05:19.877654+0000","last_fullsized":"2024-09-10T16:05:19.877654+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'16610","reported_seq":25181,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:06:23.112658+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:06:23.112658+0000","last_peered":"2024-09-10T16:06:23.112658+0000","last_clean":"2024-09-10T16:06:23.112658+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:06:23.112658+0000","last_undegraded":"2024-09-10T16:06:23.112658+0000","last_fullsized":"2024-09-10T16:06:23.112658+0000","mapping_epoch":46,"log_start":"43'6600","ondisk_log_start":"43'6600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16334848,"num_objects":3988,"num_object_clones":0,"num_object_copies":7976,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3988,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13000,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":99,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.876013+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:05:19.876013+0000","last_peered":"2024-09-10T16:05:19.876013+0000","last_clean":"2024-09-10T16:05:19.876013+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:05:19.876013+0000","last_undegraded":"2024-09-10T16:05:19.876013+0000","last_fullsized":"2024-09-10T16:05:19.876013+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'17097","reported_seq":17244,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:24.867982+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:06:24.867982+0000","last_peered":"2024-09-10T16:06:24.867982+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:06:24.867982+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"43'7000","ondisk_log_start":"43'7000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16183296,"num_objects":3951,"num_object_clones":0,"num_object_copies":7902,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3951,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13352,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-420}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877349+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:05:19.877349+0000","last_peered":"2024-09-10T16:05:19.877349+0000","last_clean":"2024-09-10T16:05:19.877349+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:05:19.877349+0000","last_undegraded":"2024-09-10T16:05:19.877349+0000","last_fullsized":"2024-09-10T16:05:19.877349+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877269+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:05:19.877269+0000","last_peered":"2024-09-10T16:05:19.877269+0000","last_clean":"2024-09-10T16:05:19.877269+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:05:19.877269+0000","last_undegraded":"2024-09-10T16:05:19.877269+0000","last_fullsized":"2024-09-10T16:05:19.877269+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'17185","reported_seq":17335,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:24.834396+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:06:24.834396+0000","last_peered":"2024-09-10T16:06:24.834396+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:06:24.834396+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"43'7100","ondisk_log_start":"43'7100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16438214,"num_objects":4015,"num_object_clones":0,"num_object_copies":8030,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4015,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13348,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-308},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":261026550,"num_objects":63737,"num_object_clones":0,"num_object_copies":127474,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41469,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63737,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":228369,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9920,"num_bytes_recovered":33095680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":734408704,"data_stored":734266879,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160895,"ondisk_log_size":160895,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986277,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":615844,"kb_used_data":264848,"kb_used_omap":4,"kb_used_meta":350971,"kb_avail":93755996,"statfs":{"total":96636764160,"available":96006139904,"internally_reserved":0,"allocated":271204352,"data_stored":270901668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":359394986},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,1,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659222,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":541332,"kb_used_data":194484,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93830508,"statfs":{"total":96636764160,"available":96082440192,"internally_reserved":0,"allocated":199151616,"data_stored":198819564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":355135150},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,5,2,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182505,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":685640,"kb_used_data":261104,"kb_used_omap":4,"kb_used_meta":424507,"kb_avail":93686200,"statfs":{"total":96636764160,"available":95934668800,"internally_reserved":0,"allocated":267370496,"data_stored":267067807,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":434695850},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,3,2,1,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":265801728,"data_stored":265765598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":198340608,"data_stored":198275115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":270266368,"data_stored":270226166,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:06:27.887 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:06:28.445 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:06:28.446 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:06:28.470 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":316,"stamp":"2024-09-10T16:06:27.709539+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":261026569,"num_objects":63739,"num_object_clones":0,"num_object_copies":127478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41469,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63739,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":228371,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9922,"num_bytes_recovered":33095699,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160898,"ondisk_log_size":160898,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1842816,"kb_used_data":720436,"kb_used_omap":12,"kb_used_meta":1122291,"kb_avail":281272704,"statfs":{"total":289910292480,"available":288023248896,"internally_reserved":0,"allocated":737726464,"data_stored":736789039,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1149225986},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,7,6,5,1,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18788352,"num_objects":-4587,"num_object_clones":0,"num_object_copies":-9174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-217,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4587,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4587,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":217,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004329"},"pg_stats":[{"pgid":"2.f","version":"47'16759","reported_seq":16833,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:24.802942+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:06:24.802942+0000","last_peered":"2024-09-10T16:06:24.802942+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:06:24.802942+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"43'6700","ondisk_log_start":"43'6700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16027648,"num_objects":3913,"num_object_clones":0,"num_object_copies":7826,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3913,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16759,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-346}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'16884","reported_seq":17002,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:06:25.633742+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:06:25.633742+0000","last_peered":"2024-09-10T16:06:25.633742+0000","last_clean":"2024-09-10T16:06:25.633742+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:06:25.633742+0000","last_undegraded":"2024-09-10T16:06:25.633742+0000","last_fullsized":"2024-09-10T16:06:25.633742+0000","mapping_epoch":35,"log_start":"43'6800","ondisk_log_start":"43'6800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16221126,"num_objects":3962,"num_object_clones":0,"num_object_copies":7924,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3962,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":14989,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'16756","reported_seq":25439,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:06:23.052407+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:06:23.052407+0000","last_peered":"2024-09-10T16:06:23.052407+0000","last_clean":"2024-09-10T16:06:23.052407+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:06:23.052407+0000","last_undegraded":"2024-09-10T16:06:23.052407+0000","last_fullsized":"2024-09-10T16:06:23.052407+0000","mapping_epoch":44,"log_start":"43'6700","ondisk_log_start":"43'6700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16551960,"num_objects":4042,"num_object_clones":0,"num_object_copies":8084,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4042,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14760,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'16805","reported_seq":16891,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:06:25.617375+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:06:25.617375+0000","last_peered":"2024-09-10T16:06:25.617375+0000","last_clean":"2024-09-10T16:06:25.617375+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:06:25.617375+0000","last_undegraded":"2024-09-10T16:06:25.617375+0000","last_fullsized":"2024-09-10T16:06:25.617375+0000","mapping_epoch":35,"log_start":"43'6800","ondisk_log_start":"43'6800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15790080,"num_objects":3855,"num_object_clones":0,"num_object_copies":7710,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3855,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16805,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'16926","reported_seq":17061,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:06:24.833253+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:06:24.833253+0000","last_peered":"2024-09-10T16:06:24.833253+0000","last_clean":"2024-09-10T16:06:24.833253+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:06:24.833253+0000","last_undegraded":"2024-09-10T16:06:24.833253+0000","last_fullsized":"2024-09-10T16:06:24.833253+0000","mapping_epoch":34,"log_start":"43'6900","ondisk_log_start":"43'6900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16241123,"num_objects":3966,"num_object_clones":0,"num_object_copies":7932,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3966,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13637,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'17098","reported_seq":17240,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:24.829970+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:06:24.829970+0000","last_peered":"2024-09-10T16:06:24.829970+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:06:24.829970+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"43'7000","ondisk_log_start":"43'7000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16232931,"num_objects":3964,"num_object_clones":0,"num_object_copies":7928,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3964,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13710,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-407},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'17008","reported_seq":17151,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:24.835129+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:06:24.835129+0000","last_peered":"2024-09-10T16:06:24.835129+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:06:24.835129+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"43'7000","ondisk_log_start":"43'7000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16688070,"num_objects":4076,"num_object_clones":0,"num_object_copies":8152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4076,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13507,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-161},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'17220","reported_seq":17355,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:06:24.842355+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:06:24.842355+0000","last_peered":"2024-09-10T16:06:24.842355+0000","last_clean":"2024-09-10T16:06:24.842355+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:06:24.842355+0000","last_undegraded":"2024-09-10T16:06:24.842355+0000","last_fullsized":"2024-09-10T16:06:24.842355+0000","mapping_epoch":34,"log_start":"43'7200","ondisk_log_start":"43'7200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16769024,"num_objects":4094,"num_object_clones":0,"num_object_copies":8188,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4094,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14067,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.876895+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:05:19.876895+0000","last_peered":"2024-09-10T16:05:19.876895+0000","last_clean":"2024-09-10T16:05:19.876895+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:05:19.876895+0000","last_undegraded":"2024-09-10T16:05:19.876895+0000","last_fullsized":"2024-09-10T16:05:19.876895+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'16478","reported_seq":16623,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:25.644151+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:06:25.644151+0000","last_peered":"2024-09-10T16:06:25.644151+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:06:25.644151+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"43'6400","ondisk_log_start":"43'6400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16298467,"num_objects":3980,"num_object_clones":0,"num_object_copies":7960,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3980,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14297,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-143}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":99,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.875776+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:05:19.875776+0000","last_peered":"2024-09-10T16:05:19.875776+0000","last_clean":"2024-09-10T16:05:19.875776+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:05:19.875776+0000","last_undegraded":"2024-09-10T16:05:19.875776+0000","last_fullsized":"2024-09-10T16:05:19.875776+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'16843","reported_seq":16979,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:24.814446+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:06:24.814446+0000","last_peered":"2024-09-10T16:06:24.814446+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:06:24.814446+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"43'6800","ondisk_log_start":"43'6800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16658432,"num_objects":4067,"num_object_clones":0,"num_object_copies":8134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4067,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13347,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-188}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":110,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.872023+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:05:19.872023+0000","last_peered":"2024-09-10T16:05:19.872023+0000","last_clean":"2024-09-10T16:05:19.872023+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:05:19.872023+0000","last_undegraded":"2024-09-10T16:05:19.872023+0000","last_fullsized":"2024-09-10T16:05:19.872023+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'16873","reported_seq":17013,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:25.576662+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:06:25.576662+0000","last_peered":"2024-09-10T16:06:25.576662+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:06:25.576662+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"43'6800","ondisk_log_start":"43'6800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16007651,"num_objects":3909,"num_object_clones":0,"num_object_copies":7818,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3909,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14553,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-395}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877148+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:05:19.877148+0000","last_peered":"2024-09-10T16:05:19.877148+0000","last_clean":"2024-09-10T16:05:19.877148+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:05:19.877148+0000","last_undegraded":"2024-09-10T16:05:19.877148+0000","last_fullsized":"2024-09-10T16:05:19.877148+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'17188","reported_seq":20208,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:25.613510+0000","last_change":"2024-09-10T16:05:18.271152+0000","last_active":"2024-09-10T16:06:25.613510+0000","last_peered":"2024-09-10T16:06:25.613510+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:06:25.613510+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"43'7100","ondisk_log_start":"43'7100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16310272,"num_objects":3982,"num_object_clones":0,"num_object_copies":7964,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2912,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3982,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14742,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1442,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1070}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'16865","reported_seq":17002,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:24.860849+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:06:24.860849+0000","last_peered":"2024-09-10T16:06:24.860849+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:06:24.860849+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"43'6800","ondisk_log_start":"43'6800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16273408,"num_objects":3973,"num_object_clones":0,"num_object_copies":7946,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3973,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13496,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-331}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877654+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:05:19.877654+0000","last_peered":"2024-09-10T16:05:19.877654+0000","last_clean":"2024-09-10T16:05:19.877654+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:05:19.877654+0000","last_undegraded":"2024-09-10T16:05:19.877654+0000","last_fullsized":"2024-09-10T16:05:19.877654+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'16610","reported_seq":25181,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:06:23.112658+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:06:23.112658+0000","last_peered":"2024-09-10T16:06:23.112658+0000","last_clean":"2024-09-10T16:06:23.112658+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:06:23.112658+0000","last_undegraded":"2024-09-10T16:06:23.112658+0000","last_fullsized":"2024-09-10T16:06:23.112658+0000","mapping_epoch":46,"log_start":"43'6600","ondisk_log_start":"43'6600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16334848,"num_objects":3988,"num_object_clones":0,"num_object_copies":7976,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3988,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13000,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":99,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.876013+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:05:19.876013+0000","last_peered":"2024-09-10T16:05:19.876013+0000","last_clean":"2024-09-10T16:05:19.876013+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:05:19.876013+0000","last_undegraded":"2024-09-10T16:05:19.876013+0000","last_fullsized":"2024-09-10T16:05:19.876013+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'17097","reported_seq":17244,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:24.867982+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:06:24.867982+0000","last_peered":"2024-09-10T16:06:24.867982+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:06:24.867982+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"43'7000","ondisk_log_start":"43'7000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16183296,"num_objects":3951,"num_object_clones":0,"num_object_copies":7902,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3951,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13352,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-420}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877349+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:05:19.877349+0000","last_peered":"2024-09-10T16:05:19.877349+0000","last_clean":"2024-09-10T16:05:19.877349+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:05:19.877349+0000","last_undegraded":"2024-09-10T16:05:19.877349+0000","last_fullsized":"2024-09-10T16:05:19.877349+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877269+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:05:19.877269+0000","last_peered":"2024-09-10T16:05:19.877269+0000","last_clean":"2024-09-10T16:05:19.877269+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:05:19.877269+0000","last_undegraded":"2024-09-10T16:05:19.877269+0000","last_fullsized":"2024-09-10T16:05:19.877269+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'17185","reported_seq":17335,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:24.834396+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:06:24.834396+0000","last_peered":"2024-09-10T16:06:24.834396+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:06:24.834396+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"43'7100","ondisk_log_start":"43'7100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16438214,"num_objects":4015,"num_object_clones":0,"num_object_copies":8030,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4015,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13348,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-308},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":261026550,"num_objects":63737,"num_object_clones":0,"num_object_copies":127474,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41469,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63737,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":228369,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9920,"num_bytes_recovered":33095680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":734408704,"data_stored":734266879,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160895,"ondisk_log_size":160895,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986277,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":615844,"kb_used_data":264848,"kb_used_omap":4,"kb_used_meta":350971,"kb_avail":93755996,"statfs":{"total":96636764160,"available":96006139904,"internally_reserved":0,"allocated":271204352,"data_stored":270901668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":359394986},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,1,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659222,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":541332,"kb_used_data":194484,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93830508,"statfs":{"total":96636764160,"available":96082440192,"internally_reserved":0,"allocated":199151616,"data_stored":198819564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":355135150},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,5,2,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182505,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":685640,"kb_used_data":261104,"kb_used_omap":4,"kb_used_meta":424507,"kb_avail":93686200,"statfs":{"total":96636764160,"available":95934668800,"internally_reserved":0,"allocated":267370496,"data_stored":267067807,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":434695850},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,3,2,1,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":265801728,"data_stored":265765598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":198340608,"data_stored":198275115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":270266368,"data_stored":270226166,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:06:28.471 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-10T16:06:28.471 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "47'16759", 'reported_seq': 16833, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:06:24.802942+0000', 'last_change': '2024-09-10T15:59:16.281558+0000', 'last_active': '2024-09-10T16:06:24.802942+0000', 'last_peered': '2024-09-10T16:06:24.802942+0000', 'last_clean': '2024-09-10T15:59:14.132456+0000', 'last_became_active': '2024-09-10T15:59:16.274407+0000', 'last_became_peered': '2024-09-10T15:59:16.274407+0000', 'last_unstale': '2024-09-10T16:06:24.802942+0000', 'last_undegraded': '2024-09-10T15:59:15.851279+0000', 'last_fullsized': '2024-09-10T15:59:15.774356+0000', 'mapping_epoch': 42, 'log_start': "43'6700", 'ondisk_log_start': "43'6700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'objects_scrubbed': 0, 'log_size': 10059, 'log_dups_size': 0, 'ondisk_log_size': 10059, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16027648, 'num_objects': 3913, 'num_object_clones': 0, 'num_object_copies': 7826, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4259, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3913, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16759, 'num_write_kb': 41344, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -346}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:06:28.471 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-10T16:06:28.472 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "47'17098", 'reported_seq': 17240, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:06:24.829970+0000', 'last_change': '2024-09-10T15:59:16.308264+0000', 'last_active': '2024-09-10T16:06:24.829970+0000', 'last_peered': '2024-09-10T16:06:24.829970+0000', 'last_clean': '2024-09-10T15:59:14.111551+0000', 'last_became_active': '2024-09-10T15:59:16.300040+0000', 'last_became_peered': '2024-09-10T15:59:16.300040+0000', 'last_unstale': '2024-09-10T16:06:24.829970+0000', 'last_undegraded': '2024-09-10T15:59:15.878234+0000', 'last_fullsized': '2024-09-10T15:59:15.769328+0000', 'mapping_epoch': 42, 'log_start': "43'7000", 'ondisk_log_start': "43'7000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3442", 'last_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'objects_scrubbed': 3384, 'log_size': 10098, 'log_dups_size': 0, 'ondisk_log_size': 10098, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1292, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16232931, 'num_objects': 3964, 'num_object_clones': 0, 'num_object_copies': 7928, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4372, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3964, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 13710, 'num_write_kb': 28580, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -407}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:06:28.472 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-10T16:06:28.472 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "47'17008", 'reported_seq': 17151, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:06:24.835129+0000', 'last_change': '2024-09-10T15:59:16.331899+0000', 'last_active': '2024-09-10T16:06:24.835129+0000', 'last_peered': '2024-09-10T16:06:24.835129+0000', 'last_clean': '2024-09-10T15:59:14.139436+0000', 'last_became_active': '2024-09-10T15:59:16.318715+0000', 'last_became_peered': '2024-09-10T15:59:16.318715+0000', 'last_unstale': '2024-09-10T16:06:24.835129+0000', 'last_undegraded': '2024-09-10T15:59:15.877524+0000', 'last_fullsized': '2024-09-10T15:59:15.769042+0000', 'mapping_epoch': 42, 'log_start': "43'7000", 'ondisk_log_start': "43'7000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3537", 'last_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'objects_scrubbed': 3497, 'log_size': 10008, 'log_dups_size': 0, 'ondisk_log_size': 10008, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1224, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16688070, 'num_objects': 4076, 'num_object_clones': 0, 'num_object_copies': 8152, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4237, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4076, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 13507, 'num_write_kb': 28172, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -161}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:06:28.472 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-10T16:06:28.473 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "47'16478", 'reported_seq': 16623, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:06:25.644151+0000', 'last_change': '2024-09-10T15:59:16.175680+0000', 'last_active': '2024-09-10T16:06:25.644151+0000', 'last_peered': '2024-09-10T16:06:25.644151+0000', 'last_clean': '2024-09-10T15:59:14.088187+0000', 'last_became_active': '2024-09-10T15:59:16.165810+0000', 'last_became_peered': '2024-09-10T15:59:16.165810+0000', 'last_unstale': '2024-09-10T16:06:25.644151+0000', 'last_undegraded': '2024-09-10T15:59:15.822098+0000', 'last_fullsized': '2024-09-10T15:59:15.769123+0000', 'mapping_epoch': 42, 'log_start': "43'6400", 'ondisk_log_start': "43'6400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2218", 'last_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'objects_scrubbed': 2175, 'log_size': 10078, 'log_dups_size': 0, 'ondisk_log_size': 10078, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 825, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16298467, 'num_objects': 3980, 'num_object_clones': 0, 'num_object_copies': 7960, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4124, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3980, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 14297, 'num_write_kb': 32204, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': -143}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:06:28.473 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-10T16:06:28.473 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "47'16843", 'reported_seq': 16979, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:06:24.814446+0000', 'last_change': '2024-09-10T15:59:16.277742+0000', 'last_active': '2024-09-10T16:06:24.814446+0000', 'last_peered': '2024-09-10T16:06:24.814446+0000', 'last_clean': '2024-09-10T15:59:14.129081+0000', 'last_became_active': '2024-09-10T15:59:16.269835+0000', 'last_became_peered': '2024-09-10T15:59:16.269835+0000', 'last_unstale': '2024-09-10T16:06:24.814446+0000', 'last_undegraded': '2024-09-10T15:59:15.860857+0000', 'last_fullsized': '2024-09-10T15:59:15.771410+0000', 'mapping_epoch': 42, 'log_start': "43'6800", 'ondisk_log_start': "43'6800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3557", 'last_scrub_stamp': '2024-09-10T15:59:02.493678+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:02.493678+0000', 'objects_scrubbed': 3488, 'log_size': 10043, 'log_dups_size': 0, 'ondisk_log_size': 10043, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1472, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16658432, 'num_objects': 4067, 'num_object_clones': 0, 'num_object_copies': 8134, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4257, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4067, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13347, 'num_write_kb': 27852, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -188}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:06:28.473 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-10T16:06:28.473 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "47'16873", 'reported_seq': 17013, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:06:25.576662+0000', 'last_change': '2024-09-10T15:59:16.284311+0000', 'last_active': '2024-09-10T16:06:25.576662+0000', 'last_peered': '2024-09-10T16:06:25.576662+0000', 'last_clean': '2024-09-10T15:59:14.116041+0000', 'last_became_active': '2024-09-10T15:59:16.274543+0000', 'last_became_peered': '2024-09-10T15:59:16.274543+0000', 'last_unstale': '2024-09-10T16:06:25.576662+0000', 'last_undegraded': '2024-09-10T15:59:15.812484+0000', 'last_fullsized': '2024-09-10T15:59:15.770885+0000', 'mapping_epoch': 42, 'log_start': "43'6800", 'ondisk_log_start': "43'6800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2356", 'last_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'objects_scrubbed': 2316, 'log_size': 10073, 'log_dups_size': 0, 'ondisk_log_size': 10073, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 783, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16007651, 'num_objects': 3909, 'num_object_clones': 0, 'num_object_copies': 7818, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3909, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 14553, 'num_write_kb': 32292, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': -395}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:06:28.474 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-10T16:06:28.474 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "47'17188", 'reported_seq': 20208, 'reported_epoch': 47, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:06:25.613510+0000', 'last_change': '2024-09-10T16:05:18.271152+0000', 'last_active': '2024-09-10T16:06:25.613510+0000', 'last_peered': '2024-09-10T16:06:25.613510+0000', 'last_clean': '2024-09-10T15:59:14.110654+0000', 'last_became_active': '2024-09-10T15:59:16.164645+0000', 'last_became_peered': '2024-09-10T15:59:16.164645+0000', 'last_unstale': '2024-09-10T16:06:25.613510+0000', 'last_undegraded': '2024-09-10T15:59:15.803593+0000', 'last_fullsized': '2024-09-10T15:59:15.769083+0000', 'mapping_epoch': 42, 'log_start': "43'7100", 'ondisk_log_start': "43'7100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2486", 'last_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'objects_scrubbed': 2440, 'log_size': 10088, 'log_dups_size': 0, 'ondisk_log_size': 10088, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 792, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16310272, 'num_objects': 3982, 'num_object_clones': 0, 'num_object_copies': 7964, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2912, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3982, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14742, 'num_write_kb': 32568, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1442, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 1070}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:06:28.474 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-10T16:06:28.474 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "47'16865", 'reported_seq': 17002, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:06:24.860849+0000', 'last_change': '2024-09-10T15:59:16.315641+0000', 'last_active': '2024-09-10T16:06:24.860849+0000', 'last_peered': '2024-09-10T16:06:24.860849+0000', 'last_clean': '2024-09-10T15:59:14.117001+0000', 'last_became_active': '2024-09-10T15:59:16.302245+0000', 'last_became_peered': '2024-09-10T15:59:16.302245+0000', 'last_unstale': '2024-09-10T16:06:24.860849+0000', 'last_undegraded': '2024-09-10T15:59:15.845012+0000', 'last_fullsized': '2024-09-10T15:59:15.770337+0000', 'mapping_epoch': 42, 'log_start': "43'6800", 'ondisk_log_start': "43'6800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3404", 'last_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'objects_scrubbed': 3367, 'log_size': 10065, 'log_dups_size': 0, 'ondisk_log_size': 10065, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 983, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16273408, 'num_objects': 3973, 'num_object_clones': 0, 'num_object_copies': 7946, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3973, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13496, 'num_write_kb': 28204, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -331}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:06:28.474 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-10T16:06:28.475 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "47'17097", 'reported_seq': 17244, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:06:24.867982+0000', 'last_change': '2024-09-10T15:59:16.311291+0000', 'last_active': '2024-09-10T16:06:24.867982+0000', 'last_peered': '2024-09-10T16:06:24.867982+0000', 'last_clean': '2024-09-10T15:59:14.116363+0000', 'last_became_active': '2024-09-10T15:59:16.301244+0000', 'last_became_peered': '2024-09-10T15:59:16.301244+0000', 'last_unstale': '2024-09-10T16:06:24.867982+0000', 'last_undegraded': '2024-09-10T15:59:15.883434+0000', 'last_fullsized': '2024-09-10T15:59:15.772756+0000', 'mapping_epoch': 42, 'log_start': "43'7000", 'ondisk_log_start': "43'7000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3789", 'last_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'objects_scrubbed': 3741, 'log_size': 10097, 'log_dups_size': 0, 'ondisk_log_size': 10097, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1440, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16183296, 'num_objects': 3951, 'num_object_clones': 0, 'num_object_copies': 7902, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4373, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3951, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13352, 'num_write_kb': 27124, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -420}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:06:28.475 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-10T16:06:28.475 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "47'17185", 'reported_seq': 17335, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:06:24.834396+0000', 'last_change': '2024-09-10T15:59:16.313409+0000', 'last_active': '2024-09-10T16:06:24.834396+0000', 'last_peered': '2024-09-10T16:06:24.834396+0000', 'last_clean': '2024-09-10T15:59:14.129780+0000', 'last_became_active': '2024-09-10T15:59:16.300932+0000', 'last_became_peered': '2024-09-10T15:59:16.300932+0000', 'last_unstale': '2024-09-10T16:06:24.834396+0000', 'last_undegraded': '2024-09-10T15:59:15.887823+0000', 'last_fullsized': '2024-09-10T15:59:15.768513+0000', 'mapping_epoch': 42, 'log_start': "43'7100", 'ondisk_log_start': "43'7100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3880", 'last_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'objects_scrubbed': 3827, 'log_size': 10085, 'log_dups_size': 0, 'ondisk_log_size': 10085, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1397, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16438214, 'num_objects': 4015, 'num_object_clones': 0, 'num_object_copies': 8030, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4325, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4015, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13348, 'num_write_kb': 27072, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -308}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:06:28.476 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T16:06:29.153 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:06:29.153 INFO:teuthology.orchestra.run.smithi115.stdout:{"fsid":"7effee29-df05-4351-9741-ee203564ed81","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 41469/127478 objects degraded (32.530%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":553,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":47,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983954,"num_in_osds":3,"osd_in_since":1725983947,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":63739,"data_bytes":261026569,"bytes_used":1887043584,"bytes_avail":288023248896,"bytes_total":289910292480,"degraded_objects":41469,"degraded_total":127478,"degraded_ratio":0.32530318956996501,"recovering_objects_per_sec":18,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":217,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":382},"fsmap":{"epoch":1,"btime":"2024-09-10T15:57:14:967120+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T16:00:13.560776+0000","services":{}},"progress_events":{}} 2024-09-10T16:06:29.177 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-10T16:06:29.177 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:06:29.722 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:06:29.723 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:06:29.747 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":317,"stamp":"2024-09-10T16:06:29.710891+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":259932937,"num_objects":63472,"num_object_clones":0,"num_object_copies":126944,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41469,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63472,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":228638,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9922,"num_bytes_recovered":33095699,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160865,"ondisk_log_size":160865,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1827580,"kb_used_data":713532,"kb_used_omap":12,"kb_used_meta":1113971,"kb_avail":281287940,"statfs":{"total":289910292480,"available":288038850560,"internally_reserved":0,"allocated":730656768,"data_stored":729719343,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1140706306},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,3,9,2,6,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19881984,"num_objects":-4854,"num_object_clones":0,"num_object_copies":-9708,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-217,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4854,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4854,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":217,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004830"},"pg_stats":[{"pgid":"2.f","version":"47'16759","reported_seq":16833,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:24.802942+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:06:24.802942+0000","last_peered":"2024-09-10T16:06:24.802942+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:06:24.802942+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"43'6700","ondisk_log_start":"43'6700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16027648,"num_objects":3913,"num_object_clones":0,"num_object_copies":7826,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3913,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16759,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-346}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'16884","reported_seq":17002,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:06:25.633742+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:06:25.633742+0000","last_peered":"2024-09-10T16:06:25.633742+0000","last_clean":"2024-09-10T16:06:25.633742+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:06:25.633742+0000","last_undegraded":"2024-09-10T16:06:25.633742+0000","last_fullsized":"2024-09-10T16:06:25.633742+0000","mapping_epoch":35,"log_start":"43'6800","ondisk_log_start":"43'6800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16221126,"num_objects":3962,"num_object_clones":0,"num_object_copies":7924,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3962,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":14989,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'16905","reported_seq":25588,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:06:28.149142+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:06:28.149142+0000","last_peered":"2024-09-10T16:06:28.149142+0000","last_clean":"2024-09-10T16:06:28.149142+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:06:28.149142+0000","last_undegraded":"2024-09-10T16:06:28.149142+0000","last_fullsized":"2024-09-10T16:06:28.149142+0000","mapping_epoch":44,"log_start":"43'6900","ondisk_log_start":"43'6900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15941656,"num_objects":3893,"num_object_clones":0,"num_object_copies":7786,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3893,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14909,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'16805","reported_seq":16891,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:06:25.617375+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:06:25.617375+0000","last_peered":"2024-09-10T16:06:25.617375+0000","last_clean":"2024-09-10T16:06:25.617375+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:06:25.617375+0000","last_undegraded":"2024-09-10T16:06:25.617375+0000","last_fullsized":"2024-09-10T16:06:25.617375+0000","mapping_epoch":35,"log_start":"43'6800","ondisk_log_start":"43'6800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15790080,"num_objects":3855,"num_object_clones":0,"num_object_copies":7710,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3855,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16805,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'16926","reported_seq":17061,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:06:24.833253+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:06:24.833253+0000","last_peered":"2024-09-10T16:06:24.833253+0000","last_clean":"2024-09-10T16:06:24.833253+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:06:24.833253+0000","last_undegraded":"2024-09-10T16:06:24.833253+0000","last_fullsized":"2024-09-10T16:06:24.833253+0000","mapping_epoch":34,"log_start":"43'6900","ondisk_log_start":"43'6900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16241123,"num_objects":3966,"num_object_clones":0,"num_object_copies":7932,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3966,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13637,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'17098","reported_seq":17240,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:24.829970+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:06:24.829970+0000","last_peered":"2024-09-10T16:06:24.829970+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:06:24.829970+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"43'7000","ondisk_log_start":"43'7000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16232931,"num_objects":3964,"num_object_clones":0,"num_object_copies":7928,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3964,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13710,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-407},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'17008","reported_seq":17151,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:24.835129+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:06:24.835129+0000","last_peered":"2024-09-10T16:06:24.835129+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:06:24.835129+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"43'7000","ondisk_log_start":"43'7000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16688070,"num_objects":4076,"num_object_clones":0,"num_object_copies":8152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4076,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13507,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-161},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'17220","reported_seq":17355,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:06:24.842355+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:06:24.842355+0000","last_peered":"2024-09-10T16:06:24.842355+0000","last_clean":"2024-09-10T16:06:24.842355+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:06:24.842355+0000","last_undegraded":"2024-09-10T16:06:24.842355+0000","last_fullsized":"2024-09-10T16:06:24.842355+0000","mapping_epoch":34,"log_start":"43'7200","ondisk_log_start":"43'7200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16769024,"num_objects":4094,"num_object_clones":0,"num_object_copies":8188,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4094,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14067,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.876895+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:05:19.876895+0000","last_peered":"2024-09-10T16:05:19.876895+0000","last_clean":"2024-09-10T16:05:19.876895+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:05:19.876895+0000","last_undegraded":"2024-09-10T16:05:19.876895+0000","last_fullsized":"2024-09-10T16:05:19.876895+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'16478","reported_seq":16623,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:25.644151+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:06:25.644151+0000","last_peered":"2024-09-10T16:06:25.644151+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:06:25.644151+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"43'6400","ondisk_log_start":"43'6400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16298467,"num_objects":3980,"num_object_clones":0,"num_object_copies":7960,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3980,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14297,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-143}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":99,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.875776+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:05:19.875776+0000","last_peered":"2024-09-10T16:05:19.875776+0000","last_clean":"2024-09-10T16:05:19.875776+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:05:19.875776+0000","last_undegraded":"2024-09-10T16:05:19.875776+0000","last_fullsized":"2024-09-10T16:05:19.875776+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'16843","reported_seq":16979,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:24.814446+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:06:24.814446+0000","last_peered":"2024-09-10T16:06:24.814446+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:06:24.814446+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"43'6800","ondisk_log_start":"43'6800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16658432,"num_objects":4067,"num_object_clones":0,"num_object_copies":8134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4067,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13347,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-188}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":110,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.872023+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:05:19.872023+0000","last_peered":"2024-09-10T16:05:19.872023+0000","last_clean":"2024-09-10T16:05:19.872023+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:05:19.872023+0000","last_undegraded":"2024-09-10T16:05:19.872023+0000","last_fullsized":"2024-09-10T16:05:19.872023+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'16873","reported_seq":17013,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:25.576662+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:06:25.576662+0000","last_peered":"2024-09-10T16:06:25.576662+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:06:25.576662+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"43'6800","ondisk_log_start":"43'6800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16007651,"num_objects":3909,"num_object_clones":0,"num_object_copies":7818,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3909,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14553,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-395}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877148+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:05:19.877148+0000","last_peered":"2024-09-10T16:05:19.877148+0000","last_clean":"2024-09-10T16:05:19.877148+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:05:19.877148+0000","last_undegraded":"2024-09-10T16:05:19.877148+0000","last_fullsized":"2024-09-10T16:05:19.877148+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'17188","reported_seq":20208,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:25.613510+0000","last_change":"2024-09-10T16:05:18.271152+0000","last_active":"2024-09-10T16:06:25.613510+0000","last_peered":"2024-09-10T16:06:25.613510+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:06:25.613510+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"43'7100","ondisk_log_start":"43'7100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16310272,"num_objects":3982,"num_object_clones":0,"num_object_copies":7964,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2912,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3982,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14742,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1442,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1070}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'16865","reported_seq":17002,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:24.860849+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:06:24.860849+0000","last_peered":"2024-09-10T16:06:24.860849+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:06:24.860849+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"43'6800","ondisk_log_start":"43'6800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16273408,"num_objects":3973,"num_object_clones":0,"num_object_copies":7946,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3973,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13496,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-331}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877654+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:05:19.877654+0000","last_peered":"2024-09-10T16:05:19.877654+0000","last_clean":"2024-09-10T16:05:19.877654+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:05:19.877654+0000","last_undegraded":"2024-09-10T16:05:19.877654+0000","last_fullsized":"2024-09-10T16:05:19.877654+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'16728","reported_seq":25299,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:06:28.112284+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:06:28.112284+0000","last_peered":"2024-09-10T16:06:28.112284+0000","last_clean":"2024-09-10T16:06:28.112284+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:06:28.112284+0000","last_undegraded":"2024-09-10T16:06:28.112284+0000","last_fullsized":"2024-09-10T16:06:28.112284+0000","mapping_epoch":46,"log_start":"43'6700","ondisk_log_start":"43'6700","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15851520,"num_objects":3870,"num_object_clones":0,"num_object_copies":7740,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3870,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13118,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":99,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.876013+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:05:19.876013+0000","last_peered":"2024-09-10T16:05:19.876013+0000","last_clean":"2024-09-10T16:05:19.876013+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:05:19.876013+0000","last_undegraded":"2024-09-10T16:05:19.876013+0000","last_fullsized":"2024-09-10T16:05:19.876013+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'17097","reported_seq":17244,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:24.867982+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:06:24.867982+0000","last_peered":"2024-09-10T16:06:24.867982+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:06:24.867982+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"43'7000","ondisk_log_start":"43'7000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16183296,"num_objects":3951,"num_object_clones":0,"num_object_copies":7902,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3951,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13352,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-420}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877349+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:05:19.877349+0000","last_peered":"2024-09-10T16:05:19.877349+0000","last_clean":"2024-09-10T16:05:19.877349+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:05:19.877349+0000","last_undegraded":"2024-09-10T16:05:19.877349+0000","last_fullsized":"2024-09-10T16:05:19.877349+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877269+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:05:19.877269+0000","last_peered":"2024-09-10T16:05:19.877269+0000","last_clean":"2024-09-10T16:05:19.877269+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:05:19.877269+0000","last_undegraded":"2024-09-10T16:05:19.877269+0000","last_fullsized":"2024-09-10T16:05:19.877269+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'17185","reported_seq":17335,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:24.834396+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:06:24.834396+0000","last_peered":"2024-09-10T16:06:24.834396+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:06:24.834396+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"43'7100","ondisk_log_start":"43'7100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16438214,"num_objects":4015,"num_object_clones":0,"num_object_copies":8030,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4015,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13348,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-308},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":259932918,"num_objects":63470,"num_object_clones":0,"num_object_copies":126940,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41469,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63470,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":228636,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9920,"num_bytes_recovered":33095680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":727371776,"data_stored":727229951,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160862,"ondisk_log_size":160862,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986277,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":615844,"kb_used_data":264848,"kb_used_omap":4,"kb_used_meta":350971,"kb_avail":93755996,"statfs":{"total":96636764160,"available":96006139904,"internally_reserved":0,"allocated":271204352,"data_stored":270901668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":359394986},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,1,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659223,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":526096,"kb_used_data":187580,"kb_used_omap":4,"kb_used_meta":338491,"kb_avail":93845744,"statfs":{"total":96636764160,"available":96098041856,"internally_reserved":0,"allocated":192081920,"data_stored":191749868,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":346615470},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,5,0,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182505,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":685640,"kb_used_data":261104,"kb_used_omap":4,"kb_used_meta":424507,"kb_avail":93686200,"statfs":{"total":96636764160,"available":95934668800,"internally_reserved":0,"allocated":267370496,"data_stored":267067807,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":434695850},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,3,2,1,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":265801728,"data_stored":265765598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":191303680,"data_stored":191238187,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":270266368,"data_stored":270226166,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:06:32.748 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:06:33.309 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:06:33.312 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:06:33.333 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":318,"stamp":"2024-09-10T16:06:31.711523+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":251577097,"num_objects":61432,"num_object_clones":0,"num_object_copies":122864,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41354,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61432,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":230678,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10037,"num_bytes_recovered":33095699,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160705,"ondisk_log_size":160705,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1811496,"kb_used_data":701616,"kb_used_omap":12,"kb_used_meta":1109811,"kb_avail":281304024,"statfs":{"total":289910292480,"available":288055320576,"internally_reserved":0,"allocated":718454784,"data_stored":717517359,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1136446466},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,1,0,4,9,3,15,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-27037696,"num_objects":-6601,"num_object_clones":0,"num_object_copies":-13202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-332,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-6601,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6601,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":332,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004772"},"pg_stats":[{"pgid":"2.f","version":"47'16884","reported_seq":16964,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:29.895985+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:06:29.895985+0000","last_peered":"2024-09-10T16:06:29.895985+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:06:29.895985+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"43'6800","ondisk_log_start":"43'6800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15515648,"num_objects":3788,"num_object_clones":0,"num_object_copies":7576,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3788,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16884,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-471}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'17040","reported_seq":17158,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:06:30.689749+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:06:30.689749+0000","last_peered":"2024-09-10T16:06:30.689749+0000","last_clean":"2024-09-10T16:06:30.689749+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:06:30.689749+0000","last_undegraded":"2024-09-10T16:06:30.689749+0000","last_fullsized":"2024-09-10T16:06:30.689749+0000","mapping_epoch":35,"log_start":"43'7000","ondisk_log_start":"43'7000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15582150,"num_objects":3806,"num_object_clones":0,"num_object_copies":7612,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3806,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":15145,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'16905","reported_seq":25588,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:06:28.149142+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:06:28.149142+0000","last_peered":"2024-09-10T16:06:28.149142+0000","last_clean":"2024-09-10T16:06:28.149142+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:06:28.149142+0000","last_undegraded":"2024-09-10T16:06:28.149142+0000","last_fullsized":"2024-09-10T16:06:28.149142+0000","mapping_epoch":44,"log_start":"43'6900","ondisk_log_start":"43'6900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15941656,"num_objects":3893,"num_object_clones":0,"num_object_copies":7786,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3893,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14909,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'16956","reported_seq":17042,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:06:30.540922+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:06:30.540922+0000","last_peered":"2024-09-10T16:06:30.540922+0000","last_clean":"2024-09-10T16:06:30.540922+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:06:30.540922+0000","last_undegraded":"2024-09-10T16:06:30.540922+0000","last_fullsized":"2024-09-10T16:06:30.540922+0000","mapping_epoch":35,"log_start":"43'6900","ondisk_log_start":"43'6900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15171584,"num_objects":3704,"num_object_clones":0,"num_object_copies":7408,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3704,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16956,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'17066","reported_seq":17201,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:06:29.875107+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:06:29.875107+0000","last_peered":"2024-09-10T16:06:29.875107+0000","last_clean":"2024-09-10T16:06:29.875107+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:06:29.875107+0000","last_undegraded":"2024-09-10T16:06:29.875107+0000","last_fullsized":"2024-09-10T16:06:29.875107+0000","mapping_epoch":34,"log_start":"43'7000","ondisk_log_start":"43'7000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15667683,"num_objects":3826,"num_object_clones":0,"num_object_copies":7652,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3826,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13777,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'17219","reported_seq":17362,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:29.857674+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:06:29.857674+0000","last_peered":"2024-09-10T16:06:29.857674+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:06:29.857674+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"43'7200","ondisk_log_start":"43'7200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15737315,"num_objects":3843,"num_object_clones":0,"num_object_copies":7686,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3843,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13831,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-528},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'17156","reported_seq":17297,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:29.804165+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:06:29.804165+0000","last_peered":"2024-09-10T16:06:29.804165+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:06:29.804165+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"43'7100","ondisk_log_start":"43'7100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16081862,"num_objects":3928,"num_object_clones":0,"num_object_copies":7856,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3928,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13655,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-309},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'17393","reported_seq":17528,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:06:29.733240+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:06:29.733240+0000","last_peered":"2024-09-10T16:06:29.733240+0000","last_clean":"2024-09-10T16:06:29.733240+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:06:29.733240+0000","last_undegraded":"2024-09-10T16:06:29.733240+0000","last_fullsized":"2024-09-10T16:06:29.733240+0000","mapping_epoch":34,"log_start":"43'7300","ondisk_log_start":"43'7300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16060416,"num_objects":3921,"num_object_clones":0,"num_object_copies":7842,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3921,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14240,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.876895+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:05:19.876895+0000","last_peered":"2024-09-10T16:05:19.876895+0000","last_clean":"2024-09-10T16:05:19.876895+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:05:19.876895+0000","last_undegraded":"2024-09-10T16:05:19.876895+0000","last_fullsized":"2024-09-10T16:05:19.876895+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'16626","reported_seq":16771,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:30.604611+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:06:30.604611+0000","last_peered":"2024-09-10T16:06:30.604611+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:06:30.604611+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"43'6600","ondisk_log_start":"43'6600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15692259,"num_objects":3832,"num_object_clones":0,"num_object_copies":7664,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3832,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14445,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-291}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":99,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.875776+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:05:19.875776+0000","last_peered":"2024-09-10T16:05:19.875776+0000","last_clean":"2024-09-10T16:05:19.875776+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:05:19.875776+0000","last_undegraded":"2024-09-10T16:05:19.875776+0000","last_fullsized":"2024-09-10T16:05:19.875776+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'16983","reported_seq":17119,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:29.817138+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:06:29.817138+0000","last_peered":"2024-09-10T16:06:29.817138+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:06:29.817138+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"43'6900","ondisk_log_start":"43'6900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16084992,"num_objects":3927,"num_object_clones":0,"num_object_copies":7854,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3927,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13487,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-328}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":110,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.872023+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:05:19.872023+0000","last_peered":"2024-09-10T16:05:19.872023+0000","last_clean":"2024-09-10T16:05:19.872023+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:05:19.872023+0000","last_undegraded":"2024-09-10T16:05:19.872023+0000","last_fullsized":"2024-09-10T16:05:19.872023+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'17032","reported_seq":17172,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:30.671750+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:06:30.671750+0000","last_peered":"2024-09-10T16:06:30.671750+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:06:30.671750+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"43'7000","ondisk_log_start":"43'7000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15356387,"num_objects":3750,"num_object_clones":0,"num_object_copies":7500,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3750,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14712,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-554}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877148+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:05:19.877148+0000","last_peered":"2024-09-10T16:05:19.877148+0000","last_clean":"2024-09-10T16:05:19.877148+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:05:19.877148+0000","last_undegraded":"2024-09-10T16:05:19.877148+0000","last_fullsized":"2024-09-10T16:05:19.877148+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'17327","reported_seq":20577,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:30.659755+0000","last_change":"2024-09-10T16:05:18.271152+0000","last_active":"2024-09-10T16:06:30.659755+0000","last_peered":"2024-09-10T16:06:30.659755+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:06:30.659755+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"43'7300","ondisk_log_start":"43'7300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15740928,"num_objects":3843,"num_object_clones":0,"num_object_copies":7686,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2797,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3843,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14881,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1557,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1046}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'17011","reported_seq":17151,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:29.979536+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:06:29.979536+0000","last_peered":"2024-09-10T16:06:29.979536+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:06:29.979536+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"43'7000","ondisk_log_start":"43'7000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15675392,"num_objects":3827,"num_object_clones":0,"num_object_copies":7654,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3827,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13642,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-477}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877654+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:05:19.877654+0000","last_peered":"2024-09-10T16:05:19.877654+0000","last_clean":"2024-09-10T16:05:19.877654+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:05:19.877654+0000","last_undegraded":"2024-09-10T16:05:19.877654+0000","last_fullsized":"2024-09-10T16:05:19.877654+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'16728","reported_seq":25299,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:06:28.112284+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:06:28.112284+0000","last_peered":"2024-09-10T16:06:28.112284+0000","last_clean":"2024-09-10T16:06:28.112284+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:06:28.112284+0000","last_undegraded":"2024-09-10T16:06:28.112284+0000","last_fullsized":"2024-09-10T16:06:28.112284+0000","mapping_epoch":46,"log_start":"43'6700","ondisk_log_start":"43'6700","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15851520,"num_objects":3870,"num_object_clones":0,"num_object_copies":7740,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3870,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13118,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":99,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.876013+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:05:19.876013+0000","last_peered":"2024-09-10T16:05:19.876013+0000","last_clean":"2024-09-10T16:05:19.876013+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:05:19.876013+0000","last_undegraded":"2024-09-10T16:05:19.876013+0000","last_fullsized":"2024-09-10T16:05:19.876013+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'17237","reported_seq":17384,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:29.766911+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:06:29.766911+0000","last_peered":"2024-09-10T16:06:29.766911+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:06:29.766911+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"43'7200","ondisk_log_start":"43'7200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15609856,"num_objects":3811,"num_object_clones":0,"num_object_copies":7622,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3811,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13492,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-560}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877349+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:05:19.877349+0000","last_peered":"2024-09-10T16:05:19.877349+0000","last_clean":"2024-09-10T16:05:19.877349+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:05:19.877349+0000","last_undegraded":"2024-09-10T16:05:19.877349+0000","last_fullsized":"2024-09-10T16:05:19.877349+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877269+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:05:19.877269+0000","last_peered":"2024-09-10T16:05:19.877269+0000","last_clean":"2024-09-10T16:05:19.877269+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:05:19.877269+0000","last_undegraded":"2024-09-10T16:05:19.877269+0000","last_fullsized":"2024-09-10T16:05:19.877269+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'17339","reported_seq":17488,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:29.780068+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:06:29.780068+0000","last_peered":"2024-09-10T16:06:29.780068+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:06:29.780068+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"43'7300","ondisk_log_start":"43'7300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15807430,"num_objects":3861,"num_object_clones":0,"num_object_copies":7722,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3861,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13502,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-462},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":251577078,"num_objects":61430,"num_object_clones":0,"num_object_copies":122860,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41354,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61430,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":230676,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10035,"num_bytes_recovered":33095680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":715206656,"data_stored":715064831,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160702,"ondisk_log_size":160702,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986278,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":598384,"kb_used_data":259872,"kb_used_omap":4,"kb_used_meta":338491,"kb_avail":93773456,"statfs":{"total":96636764160,"available":96024018944,"internally_reserved":0,"allocated":266108928,"data_stored":265806244,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":346615466},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,1,0,1,1,0,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659223,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":526096,"kb_used_data":187580,"kb_used_omap":4,"kb_used_meta":338491,"kb_avail":93845744,"statfs":{"total":96636764160,"available":96098041856,"internally_reserved":0,"allocated":192081920,"data_stored":191749868,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":346615470},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,5,0,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182506,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":687016,"kb_used_data":254164,"kb_used_omap":4,"kb_used_meta":432827,"kb_avail":93684824,"statfs":{"total":96636764160,"available":95933259776,"internally_reserved":0,"allocated":260263936,"data_stored":259961247,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":443215530},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,2,3,3,8],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":258613248,"data_stored":258577118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":191303680,"data_stored":191238187,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":265289728,"data_stored":265249526,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:06:33.334 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:06:33.894 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:06:33.894 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:06:33.923 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":319,"stamp":"2024-09-10T16:06:33.712272+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":250524425,"num_objects":61175,"num_object_clones":0,"num_object_copies":122350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41354,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61175,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":230935,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10037,"num_bytes_recovered":33095699,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160762,"ondisk_log_size":160762,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1808892,"kb_used_data":694848,"kb_used_omap":12,"kb_used_meta":1113971,"kb_avail":281306628,"statfs":{"total":289910292480,"available":288057987072,"internally_reserved":0,"allocated":711524352,"data_stored":710586927,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1140706306},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,1,2,5,7,5,13,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19914752,"num_objects":-4862,"num_object_clones":0,"num_object_copies":-9724,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-223,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4862,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4862,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":223,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004798"},"pg_stats":[{"pgid":"2.f","version":"47'16884","reported_seq":16964,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:29.895985+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:06:29.895985+0000","last_peered":"2024-09-10T16:06:29.895985+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:06:29.895985+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"43'6800","ondisk_log_start":"43'6800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15515648,"num_objects":3788,"num_object_clones":0,"num_object_copies":7576,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3788,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16884,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-471}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'17040","reported_seq":17158,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:06:30.689749+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:06:30.689749+0000","last_peered":"2024-09-10T16:06:30.689749+0000","last_clean":"2024-09-10T16:06:30.689749+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:06:30.689749+0000","last_undegraded":"2024-09-10T16:06:30.689749+0000","last_fullsized":"2024-09-10T16:06:30.689749+0000","mapping_epoch":35,"log_start":"43'7000","ondisk_log_start":"43'7000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15582150,"num_objects":3806,"num_object_clones":0,"num_object_copies":7612,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3806,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":15145,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'17047","reported_seq":25730,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:06:33.272029+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:06:33.272029+0000","last_peered":"2024-09-10T16:06:33.272029+0000","last_clean":"2024-09-10T16:06:33.272029+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:06:33.272029+0000","last_undegraded":"2024-09-10T16:06:33.272029+0000","last_fullsized":"2024-09-10T16:06:33.272029+0000","mapping_epoch":44,"log_start":"43'7000","ondisk_log_start":"43'7000","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15360024,"num_objects":3751,"num_object_clones":0,"num_object_copies":7502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3751,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15051,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'16956","reported_seq":17042,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:06:30.540922+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:06:30.540922+0000","last_peered":"2024-09-10T16:06:30.540922+0000","last_clean":"2024-09-10T16:06:30.540922+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:06:30.540922+0000","last_undegraded":"2024-09-10T16:06:30.540922+0000","last_fullsized":"2024-09-10T16:06:30.540922+0000","mapping_epoch":35,"log_start":"43'6900","ondisk_log_start":"43'6900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15171584,"num_objects":3704,"num_object_clones":0,"num_object_copies":7408,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3704,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16956,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'17066","reported_seq":17201,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:06:29.875107+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:06:29.875107+0000","last_peered":"2024-09-10T16:06:29.875107+0000","last_clean":"2024-09-10T16:06:29.875107+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:06:29.875107+0000","last_undegraded":"2024-09-10T16:06:29.875107+0000","last_fullsized":"2024-09-10T16:06:29.875107+0000","mapping_epoch":34,"log_start":"43'7000","ondisk_log_start":"43'7000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15667683,"num_objects":3826,"num_object_clones":0,"num_object_copies":7652,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3826,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13777,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'17219","reported_seq":17362,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:29.857674+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:06:29.857674+0000","last_peered":"2024-09-10T16:06:29.857674+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:06:29.857674+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"43'7200","ondisk_log_start":"43'7200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15737315,"num_objects":3843,"num_object_clones":0,"num_object_copies":7686,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3843,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13831,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-528},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'17156","reported_seq":17297,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:29.804165+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:06:29.804165+0000","last_peered":"2024-09-10T16:06:29.804165+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:06:29.804165+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"43'7100","ondisk_log_start":"43'7100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16081862,"num_objects":3928,"num_object_clones":0,"num_object_copies":7856,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3928,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13655,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-309},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'17393","reported_seq":17528,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:06:29.733240+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:06:29.733240+0000","last_peered":"2024-09-10T16:06:29.733240+0000","last_clean":"2024-09-10T16:06:29.733240+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:06:29.733240+0000","last_undegraded":"2024-09-10T16:06:29.733240+0000","last_fullsized":"2024-09-10T16:06:29.733240+0000","mapping_epoch":34,"log_start":"43'7300","ondisk_log_start":"43'7300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16060416,"num_objects":3921,"num_object_clones":0,"num_object_copies":7842,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3921,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14240,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.876895+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:05:19.876895+0000","last_peered":"2024-09-10T16:05:19.876895+0000","last_clean":"2024-09-10T16:05:19.876895+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:05:19.876895+0000","last_undegraded":"2024-09-10T16:05:19.876895+0000","last_fullsized":"2024-09-10T16:05:19.876895+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'16626","reported_seq":16771,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:30.604611+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:06:30.604611+0000","last_peered":"2024-09-10T16:06:30.604611+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:06:30.604611+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"43'6600","ondisk_log_start":"43'6600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15692259,"num_objects":3832,"num_object_clones":0,"num_object_copies":7664,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3832,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14445,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-291}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":99,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.875776+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:05:19.875776+0000","last_peered":"2024-09-10T16:05:19.875776+0000","last_clean":"2024-09-10T16:05:19.875776+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:05:19.875776+0000","last_undegraded":"2024-09-10T16:05:19.875776+0000","last_fullsized":"2024-09-10T16:05:19.875776+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'16983","reported_seq":17119,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:29.817138+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:06:29.817138+0000","last_peered":"2024-09-10T16:06:29.817138+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:06:29.817138+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"43'6900","ondisk_log_start":"43'6900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16084992,"num_objects":3927,"num_object_clones":0,"num_object_copies":7854,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3927,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13487,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-328}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":110,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.872023+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:05:19.872023+0000","last_peered":"2024-09-10T16:05:19.872023+0000","last_clean":"2024-09-10T16:05:19.872023+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:05:19.872023+0000","last_undegraded":"2024-09-10T16:05:19.872023+0000","last_fullsized":"2024-09-10T16:05:19.872023+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'17032","reported_seq":17172,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:30.671750+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:06:30.671750+0000","last_peered":"2024-09-10T16:06:30.671750+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:06:30.671750+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"43'7000","ondisk_log_start":"43'7000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15356387,"num_objects":3750,"num_object_clones":0,"num_object_copies":7500,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3750,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14712,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-554}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877148+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:05:19.877148+0000","last_peered":"2024-09-10T16:05:19.877148+0000","last_clean":"2024-09-10T16:05:19.877148+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:05:19.877148+0000","last_undegraded":"2024-09-10T16:05:19.877148+0000","last_fullsized":"2024-09-10T16:05:19.877148+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'17327","reported_seq":20577,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:30.659755+0000","last_change":"2024-09-10T16:05:18.271152+0000","last_active":"2024-09-10T16:06:30.659755+0000","last_peered":"2024-09-10T16:06:30.659755+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:06:30.659755+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"43'7300","ondisk_log_start":"43'7300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15740928,"num_objects":3843,"num_object_clones":0,"num_object_copies":7686,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2797,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3843,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14881,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1557,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1046}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'17011","reported_seq":17151,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:29.979536+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:06:29.979536+0000","last_peered":"2024-09-10T16:06:29.979536+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:06:29.979536+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"43'7000","ondisk_log_start":"43'7000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15675392,"num_objects":3827,"num_object_clones":0,"num_object_copies":7654,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3827,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13642,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-477}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877654+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:05:19.877654+0000","last_peered":"2024-09-10T16:05:19.877654+0000","last_clean":"2024-09-10T16:05:19.877654+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:05:19.877654+0000","last_undegraded":"2024-09-10T16:05:19.877654+0000","last_fullsized":"2024-09-10T16:05:19.877654+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'16843","reported_seq":25414,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:06:33.256732+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:06:33.256732+0000","last_peered":"2024-09-10T16:06:33.256732+0000","last_clean":"2024-09-10T16:06:33.256732+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:06:33.256732+0000","last_undegraded":"2024-09-10T16:06:33.256732+0000","last_fullsized":"2024-09-10T16:06:33.256732+0000","mapping_epoch":46,"log_start":"43'6800","ondisk_log_start":"43'6800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15380480,"num_objects":3755,"num_object_clones":0,"num_object_copies":7510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3755,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13233,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":99,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.876013+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:05:19.876013+0000","last_peered":"2024-09-10T16:05:19.876013+0000","last_clean":"2024-09-10T16:05:19.876013+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:05:19.876013+0000","last_undegraded":"2024-09-10T16:05:19.876013+0000","last_fullsized":"2024-09-10T16:05:19.876013+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'17237","reported_seq":17384,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:29.766911+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:06:29.766911+0000","last_peered":"2024-09-10T16:06:29.766911+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:06:29.766911+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"43'7200","ondisk_log_start":"43'7200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15609856,"num_objects":3811,"num_object_clones":0,"num_object_copies":7622,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3811,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13492,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-560}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877349+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:05:19.877349+0000","last_peered":"2024-09-10T16:05:19.877349+0000","last_clean":"2024-09-10T16:05:19.877349+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:05:19.877349+0000","last_undegraded":"2024-09-10T16:05:19.877349+0000","last_fullsized":"2024-09-10T16:05:19.877349+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877269+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:05:19.877269+0000","last_peered":"2024-09-10T16:05:19.877269+0000","last_clean":"2024-09-10T16:05:19.877269+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:05:19.877269+0000","last_undegraded":"2024-09-10T16:05:19.877269+0000","last_fullsized":"2024-09-10T16:05:19.877269+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'17339","reported_seq":17488,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:29.780068+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:06:29.780068+0000","last_peered":"2024-09-10T16:06:29.780068+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:06:29.780068+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"43'7300","ondisk_log_start":"43'7300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15807430,"num_objects":3861,"num_object_clones":0,"num_object_copies":7722,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3861,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13502,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-462},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":250524406,"num_objects":61173,"num_object_clones":0,"num_object_copies":122346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41354,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61173,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":230933,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10035,"num_bytes_recovered":33095680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":708026368,"data_stored":707884543,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160759,"ondisk_log_size":160759,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986278,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":598384,"kb_used_data":259872,"kb_used_omap":4,"kb_used_meta":338491,"kb_avail":93773456,"statfs":{"total":96636764160,"available":96024018944,"internally_reserved":0,"allocated":266108928,"data_stored":265806244,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":346615466},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,1,0,1,1,0,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659224,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":523492,"kb_used_data":180812,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93848348,"statfs":{"total":96636764160,"available":96100708352,"internally_reserved":0,"allocated":185151488,"data_stored":184819436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":350875310},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,2,3,2,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182506,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":687016,"kb_used_data":254164,"kb_used_omap":4,"kb_used_meta":432827,"kb_avail":93684824,"statfs":{"total":96636764160,"available":95933259776,"internally_reserved":0,"allocated":260263936,"data_stored":259961247,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":443215530},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,2,3,3,8],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":258613248,"data_stored":258577118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":184123392,"data_stored":184057899,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":265289728,"data_stored":265249526,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:06:33.924 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-10T16:06:33.924 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "47'16884", 'reported_seq': 16964, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:06:29.895985+0000', 'last_change': '2024-09-10T15:59:16.281558+0000', 'last_active': '2024-09-10T16:06:29.895985+0000', 'last_peered': '2024-09-10T16:06:29.895985+0000', 'last_clean': '2024-09-10T15:59:14.132456+0000', 'last_became_active': '2024-09-10T15:59:16.274407+0000', 'last_became_peered': '2024-09-10T15:59:16.274407+0000', 'last_unstale': '2024-09-10T16:06:29.895985+0000', 'last_undegraded': '2024-09-10T15:59:15.851279+0000', 'last_fullsized': '2024-09-10T15:59:15.774356+0000', 'mapping_epoch': 42, 'log_start': "43'6800", 'ondisk_log_start': "43'6800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'objects_scrubbed': 0, 'log_size': 10084, 'log_dups_size': 0, 'ondisk_log_size': 10084, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 15515648, 'num_objects': 3788, 'num_object_clones': 0, 'num_object_copies': 7576, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4259, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3788, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16884, 'num_write_kb': 41344, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -471}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:06:33.924 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-10T16:06:33.924 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "47'17219", 'reported_seq': 17362, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:06:29.857674+0000', 'last_change': '2024-09-10T15:59:16.308264+0000', 'last_active': '2024-09-10T16:06:29.857674+0000', 'last_peered': '2024-09-10T16:06:29.857674+0000', 'last_clean': '2024-09-10T15:59:14.111551+0000', 'last_became_active': '2024-09-10T15:59:16.300040+0000', 'last_became_peered': '2024-09-10T15:59:16.300040+0000', 'last_unstale': '2024-09-10T16:06:29.857674+0000', 'last_undegraded': '2024-09-10T15:59:15.878234+0000', 'last_fullsized': '2024-09-10T15:59:15.769328+0000', 'mapping_epoch': 42, 'log_start': "43'7200", 'ondisk_log_start': "43'7200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3442", 'last_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'objects_scrubbed': 3384, 'log_size': 10019, 'log_dups_size': 0, 'ondisk_log_size': 10019, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1292, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 15737315, 'num_objects': 3843, 'num_object_clones': 0, 'num_object_copies': 7686, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4372, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3843, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 13831, 'num_write_kb': 28580, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -528}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:06:33.925 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-10T16:06:33.925 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "47'17156", 'reported_seq': 17297, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:06:29.804165+0000', 'last_change': '2024-09-10T15:59:16.331899+0000', 'last_active': '2024-09-10T16:06:29.804165+0000', 'last_peered': '2024-09-10T16:06:29.804165+0000', 'last_clean': '2024-09-10T15:59:14.139436+0000', 'last_became_active': '2024-09-10T15:59:16.318715+0000', 'last_became_peered': '2024-09-10T15:59:16.318715+0000', 'last_unstale': '2024-09-10T16:06:29.804165+0000', 'last_undegraded': '2024-09-10T15:59:15.877524+0000', 'last_fullsized': '2024-09-10T15:59:15.769042+0000', 'mapping_epoch': 42, 'log_start': "43'7100", 'ondisk_log_start': "43'7100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3537", 'last_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'objects_scrubbed': 3497, 'log_size': 10056, 'log_dups_size': 0, 'ondisk_log_size': 10056, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1224, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16081862, 'num_objects': 3928, 'num_object_clones': 0, 'num_object_copies': 7856, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4237, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3928, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 13655, 'num_write_kb': 28172, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -309}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:06:33.925 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-10T16:06:33.925 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "47'16626", 'reported_seq': 16771, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:06:30.604611+0000', 'last_change': '2024-09-10T15:59:16.175680+0000', 'last_active': '2024-09-10T16:06:30.604611+0000', 'last_peered': '2024-09-10T16:06:30.604611+0000', 'last_clean': '2024-09-10T15:59:14.088187+0000', 'last_became_active': '2024-09-10T15:59:16.165810+0000', 'last_became_peered': '2024-09-10T15:59:16.165810+0000', 'last_unstale': '2024-09-10T16:06:30.604611+0000', 'last_undegraded': '2024-09-10T15:59:15.822098+0000', 'last_fullsized': '2024-09-10T15:59:15.769123+0000', 'mapping_epoch': 42, 'log_start': "43'6600", 'ondisk_log_start': "43'6600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2218", 'last_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'objects_scrubbed': 2175, 'log_size': 10026, 'log_dups_size': 0, 'ondisk_log_size': 10026, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 825, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 15692259, 'num_objects': 3832, 'num_object_clones': 0, 'num_object_copies': 7664, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4124, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3832, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 14445, 'num_write_kb': 32204, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': -291}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:06:33.925 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-10T16:06:33.925 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "47'16983", 'reported_seq': 17119, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:06:29.817138+0000', 'last_change': '2024-09-10T15:59:16.277742+0000', 'last_active': '2024-09-10T16:06:29.817138+0000', 'last_peered': '2024-09-10T16:06:29.817138+0000', 'last_clean': '2024-09-10T15:59:14.129081+0000', 'last_became_active': '2024-09-10T15:59:16.269835+0000', 'last_became_peered': '2024-09-10T15:59:16.269835+0000', 'last_unstale': '2024-09-10T16:06:29.817138+0000', 'last_undegraded': '2024-09-10T15:59:15.860857+0000', 'last_fullsized': '2024-09-10T15:59:15.771410+0000', 'mapping_epoch': 42, 'log_start': "43'6900", 'ondisk_log_start': "43'6900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3557", 'last_scrub_stamp': '2024-09-10T15:59:02.493678+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:02.493678+0000', 'objects_scrubbed': 3488, 'log_size': 10083, 'log_dups_size': 0, 'ondisk_log_size': 10083, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1472, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16084992, 'num_objects': 3927, 'num_object_clones': 0, 'num_object_copies': 7854, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4257, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3927, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13487, 'num_write_kb': 27852, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -328}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:06:33.925 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-10T16:06:33.925 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "47'17032", 'reported_seq': 17172, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:06:30.671750+0000', 'last_change': '2024-09-10T15:59:16.284311+0000', 'last_active': '2024-09-10T16:06:30.671750+0000', 'last_peered': '2024-09-10T16:06:30.671750+0000', 'last_clean': '2024-09-10T15:59:14.116041+0000', 'last_became_active': '2024-09-10T15:59:16.274543+0000', 'last_became_peered': '2024-09-10T15:59:16.274543+0000', 'last_unstale': '2024-09-10T16:06:30.671750+0000', 'last_undegraded': '2024-09-10T15:59:15.812484+0000', 'last_fullsized': '2024-09-10T15:59:15.770885+0000', 'mapping_epoch': 42, 'log_start': "43'7000", 'ondisk_log_start': "43'7000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2356", 'last_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'objects_scrubbed': 2316, 'log_size': 10032, 'log_dups_size': 0, 'ondisk_log_size': 10032, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 783, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 15356387, 'num_objects': 3750, 'num_object_clones': 0, 'num_object_copies': 7500, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3750, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 14712, 'num_write_kb': 32292, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': -554}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:06:33.925 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-10T16:06:33.925 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "47'17327", 'reported_seq': 20577, 'reported_epoch': 47, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:06:30.659755+0000', 'last_change': '2024-09-10T16:05:18.271152+0000', 'last_active': '2024-09-10T16:06:30.659755+0000', 'last_peered': '2024-09-10T16:06:30.659755+0000', 'last_clean': '2024-09-10T15:59:14.110654+0000', 'last_became_active': '2024-09-10T15:59:16.164645+0000', 'last_became_peered': '2024-09-10T15:59:16.164645+0000', 'last_unstale': '2024-09-10T16:06:30.659755+0000', 'last_undegraded': '2024-09-10T15:59:15.803593+0000', 'last_fullsized': '2024-09-10T15:59:15.769083+0000', 'mapping_epoch': 42, 'log_start': "43'7300", 'ondisk_log_start': "43'7300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2486", 'last_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'objects_scrubbed': 2440, 'log_size': 10027, 'log_dups_size': 0, 'ondisk_log_size': 10027, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 792, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 15740928, 'num_objects': 3843, 'num_object_clones': 0, 'num_object_copies': 7686, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2797, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3843, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14881, 'num_write_kb': 32568, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1557, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 1046}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:06:33.925 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-10T16:06:33.926 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "47'17011", 'reported_seq': 17151, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:06:29.979536+0000', 'last_change': '2024-09-10T15:59:16.315641+0000', 'last_active': '2024-09-10T16:06:29.979536+0000', 'last_peered': '2024-09-10T16:06:29.979536+0000', 'last_clean': '2024-09-10T15:59:14.117001+0000', 'last_became_active': '2024-09-10T15:59:16.302245+0000', 'last_became_peered': '2024-09-10T15:59:16.302245+0000', 'last_unstale': '2024-09-10T16:06:29.979536+0000', 'last_undegraded': '2024-09-10T15:59:15.845012+0000', 'last_fullsized': '2024-09-10T15:59:15.770337+0000', 'mapping_epoch': 42, 'log_start': "43'7000", 'ondisk_log_start': "43'7000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3404", 'last_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'objects_scrubbed': 3367, 'log_size': 10011, 'log_dups_size': 0, 'ondisk_log_size': 10011, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 983, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 15675392, 'num_objects': 3827, 'num_object_clones': 0, 'num_object_copies': 7654, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3827, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13642, 'num_write_kb': 28204, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -477}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:06:33.926 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-10T16:06:33.926 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "47'17237", 'reported_seq': 17384, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:06:29.766911+0000', 'last_change': '2024-09-10T15:59:16.311291+0000', 'last_active': '2024-09-10T16:06:29.766911+0000', 'last_peered': '2024-09-10T16:06:29.766911+0000', 'last_clean': '2024-09-10T15:59:14.116363+0000', 'last_became_active': '2024-09-10T15:59:16.301244+0000', 'last_became_peered': '2024-09-10T15:59:16.301244+0000', 'last_unstale': '2024-09-10T16:06:29.766911+0000', 'last_undegraded': '2024-09-10T15:59:15.883434+0000', 'last_fullsized': '2024-09-10T15:59:15.772756+0000', 'mapping_epoch': 42, 'log_start': "43'7200", 'ondisk_log_start': "43'7200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3789", 'last_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'objects_scrubbed': 3741, 'log_size': 10037, 'log_dups_size': 0, 'ondisk_log_size': 10037, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1440, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 15609856, 'num_objects': 3811, 'num_object_clones': 0, 'num_object_copies': 7622, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4373, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3811, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13492, 'num_write_kb': 27124, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -560}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:06:33.926 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-10T16:06:33.926 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "47'17339", 'reported_seq': 17488, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:06:29.780068+0000', 'last_change': '2024-09-10T15:59:16.313409+0000', 'last_active': '2024-09-10T16:06:29.780068+0000', 'last_peered': '2024-09-10T16:06:29.780068+0000', 'last_clean': '2024-09-10T15:59:14.129780+0000', 'last_became_active': '2024-09-10T15:59:16.300932+0000', 'last_became_peered': '2024-09-10T15:59:16.300932+0000', 'last_unstale': '2024-09-10T16:06:29.780068+0000', 'last_undegraded': '2024-09-10T15:59:15.887823+0000', 'last_fullsized': '2024-09-10T15:59:15.768513+0000', 'mapping_epoch': 42, 'log_start': "43'7300", 'ondisk_log_start': "43'7300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3880", 'last_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'objects_scrubbed': 3827, 'log_size': 10039, 'log_dups_size': 0, 'ondisk_log_size': 10039, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1397, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 15807430, 'num_objects': 3861, 'num_object_clones': 0, 'num_object_copies': 7722, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4325, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3861, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13502, 'num_write_kb': 27072, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -462}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:06:33.926 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T16:06:34.617 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:06:34.617 INFO:teuthology.orchestra.run.smithi115.stdout:{"fsid":"7effee29-df05-4351-9741-ee203564ed81","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 41354/122350 objects degraded (33.800%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":558,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":47,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983954,"num_in_osds":3,"osd_in_since":1725983947,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":61175,"data_bytes":250524425,"bytes_used":1852305408,"bytes_avail":288057987072,"bytes_total":289910292480,"degraded_objects":41354,"degraded_total":122350,"degraded_ratio":0.33799754801798121,"recovering_objects_per_sec":18,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":223,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":405},"fsmap":{"epoch":1,"btime":"2024-09-10T15:57:14:967120+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T16:00:13.560776+0000","services":{}},"progress_events":{}} 2024-09-10T16:06:34.645 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-10T16:06:34.645 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:06:35.204 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:06:35.207 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:06:35.226 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":319,"stamp":"2024-09-10T16:06:33.712272+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":250524425,"num_objects":61175,"num_object_clones":0,"num_object_copies":122350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41354,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61175,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":230935,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10037,"num_bytes_recovered":33095699,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160762,"ondisk_log_size":160762,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1808892,"kb_used_data":694848,"kb_used_omap":12,"kb_used_meta":1113971,"kb_avail":281306628,"statfs":{"total":289910292480,"available":288057987072,"internally_reserved":0,"allocated":711524352,"data_stored":710586927,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1140706306},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,1,2,5,7,5,13,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19914752,"num_objects":-4862,"num_object_clones":0,"num_object_copies":-9724,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-223,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4862,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4862,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":223,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004798"},"pg_stats":[{"pgid":"2.f","version":"47'16884","reported_seq":16964,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:29.895985+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:06:29.895985+0000","last_peered":"2024-09-10T16:06:29.895985+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:06:29.895985+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"43'6800","ondisk_log_start":"43'6800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15515648,"num_objects":3788,"num_object_clones":0,"num_object_copies":7576,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3788,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16884,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-471}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'17040","reported_seq":17158,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:06:30.689749+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:06:30.689749+0000","last_peered":"2024-09-10T16:06:30.689749+0000","last_clean":"2024-09-10T16:06:30.689749+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:06:30.689749+0000","last_undegraded":"2024-09-10T16:06:30.689749+0000","last_fullsized":"2024-09-10T16:06:30.689749+0000","mapping_epoch":35,"log_start":"43'7000","ondisk_log_start":"43'7000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15582150,"num_objects":3806,"num_object_clones":0,"num_object_copies":7612,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3806,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":15145,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'17047","reported_seq":25730,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:06:33.272029+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:06:33.272029+0000","last_peered":"2024-09-10T16:06:33.272029+0000","last_clean":"2024-09-10T16:06:33.272029+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:06:33.272029+0000","last_undegraded":"2024-09-10T16:06:33.272029+0000","last_fullsized":"2024-09-10T16:06:33.272029+0000","mapping_epoch":44,"log_start":"43'7000","ondisk_log_start":"43'7000","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15360024,"num_objects":3751,"num_object_clones":0,"num_object_copies":7502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3751,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15051,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'16956","reported_seq":17042,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:06:30.540922+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:06:30.540922+0000","last_peered":"2024-09-10T16:06:30.540922+0000","last_clean":"2024-09-10T16:06:30.540922+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:06:30.540922+0000","last_undegraded":"2024-09-10T16:06:30.540922+0000","last_fullsized":"2024-09-10T16:06:30.540922+0000","mapping_epoch":35,"log_start":"43'6900","ondisk_log_start":"43'6900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15171584,"num_objects":3704,"num_object_clones":0,"num_object_copies":7408,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3704,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16956,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'17066","reported_seq":17201,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:06:29.875107+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:06:29.875107+0000","last_peered":"2024-09-10T16:06:29.875107+0000","last_clean":"2024-09-10T16:06:29.875107+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:06:29.875107+0000","last_undegraded":"2024-09-10T16:06:29.875107+0000","last_fullsized":"2024-09-10T16:06:29.875107+0000","mapping_epoch":34,"log_start":"43'7000","ondisk_log_start":"43'7000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15667683,"num_objects":3826,"num_object_clones":0,"num_object_copies":7652,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3826,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13777,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'17219","reported_seq":17362,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:29.857674+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:06:29.857674+0000","last_peered":"2024-09-10T16:06:29.857674+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:06:29.857674+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"43'7200","ondisk_log_start":"43'7200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15737315,"num_objects":3843,"num_object_clones":0,"num_object_copies":7686,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3843,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13831,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-528},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'17156","reported_seq":17297,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:29.804165+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:06:29.804165+0000","last_peered":"2024-09-10T16:06:29.804165+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:06:29.804165+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"43'7100","ondisk_log_start":"43'7100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16081862,"num_objects":3928,"num_object_clones":0,"num_object_copies":7856,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3928,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13655,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-309},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'17393","reported_seq":17528,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:06:29.733240+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:06:29.733240+0000","last_peered":"2024-09-10T16:06:29.733240+0000","last_clean":"2024-09-10T16:06:29.733240+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:06:29.733240+0000","last_undegraded":"2024-09-10T16:06:29.733240+0000","last_fullsized":"2024-09-10T16:06:29.733240+0000","mapping_epoch":34,"log_start":"43'7300","ondisk_log_start":"43'7300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16060416,"num_objects":3921,"num_object_clones":0,"num_object_copies":7842,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3921,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14240,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.876895+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:05:19.876895+0000","last_peered":"2024-09-10T16:05:19.876895+0000","last_clean":"2024-09-10T16:05:19.876895+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:05:19.876895+0000","last_undegraded":"2024-09-10T16:05:19.876895+0000","last_fullsized":"2024-09-10T16:05:19.876895+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'16626","reported_seq":16771,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:30.604611+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:06:30.604611+0000","last_peered":"2024-09-10T16:06:30.604611+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:06:30.604611+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"43'6600","ondisk_log_start":"43'6600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15692259,"num_objects":3832,"num_object_clones":0,"num_object_copies":7664,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3832,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14445,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-291}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":99,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.875776+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:05:19.875776+0000","last_peered":"2024-09-10T16:05:19.875776+0000","last_clean":"2024-09-10T16:05:19.875776+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:05:19.875776+0000","last_undegraded":"2024-09-10T16:05:19.875776+0000","last_fullsized":"2024-09-10T16:05:19.875776+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'16983","reported_seq":17119,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:29.817138+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:06:29.817138+0000","last_peered":"2024-09-10T16:06:29.817138+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:06:29.817138+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"43'6900","ondisk_log_start":"43'6900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16084992,"num_objects":3927,"num_object_clones":0,"num_object_copies":7854,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3927,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13487,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-328}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":110,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.872023+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:05:19.872023+0000","last_peered":"2024-09-10T16:05:19.872023+0000","last_clean":"2024-09-10T16:05:19.872023+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:05:19.872023+0000","last_undegraded":"2024-09-10T16:05:19.872023+0000","last_fullsized":"2024-09-10T16:05:19.872023+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'17032","reported_seq":17172,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:30.671750+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:06:30.671750+0000","last_peered":"2024-09-10T16:06:30.671750+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:06:30.671750+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"43'7000","ondisk_log_start":"43'7000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15356387,"num_objects":3750,"num_object_clones":0,"num_object_copies":7500,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3750,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14712,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-554}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877148+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:05:19.877148+0000","last_peered":"2024-09-10T16:05:19.877148+0000","last_clean":"2024-09-10T16:05:19.877148+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:05:19.877148+0000","last_undegraded":"2024-09-10T16:05:19.877148+0000","last_fullsized":"2024-09-10T16:05:19.877148+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'17327","reported_seq":20577,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:30.659755+0000","last_change":"2024-09-10T16:05:18.271152+0000","last_active":"2024-09-10T16:06:30.659755+0000","last_peered":"2024-09-10T16:06:30.659755+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:06:30.659755+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"43'7300","ondisk_log_start":"43'7300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15740928,"num_objects":3843,"num_object_clones":0,"num_object_copies":7686,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2797,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3843,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14881,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1557,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1046}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'17011","reported_seq":17151,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:29.979536+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:06:29.979536+0000","last_peered":"2024-09-10T16:06:29.979536+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:06:29.979536+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"43'7000","ondisk_log_start":"43'7000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15675392,"num_objects":3827,"num_object_clones":0,"num_object_copies":7654,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3827,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13642,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-477}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877654+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:05:19.877654+0000","last_peered":"2024-09-10T16:05:19.877654+0000","last_clean":"2024-09-10T16:05:19.877654+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:05:19.877654+0000","last_undegraded":"2024-09-10T16:05:19.877654+0000","last_fullsized":"2024-09-10T16:05:19.877654+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'16843","reported_seq":25414,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:06:33.256732+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:06:33.256732+0000","last_peered":"2024-09-10T16:06:33.256732+0000","last_clean":"2024-09-10T16:06:33.256732+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:06:33.256732+0000","last_undegraded":"2024-09-10T16:06:33.256732+0000","last_fullsized":"2024-09-10T16:06:33.256732+0000","mapping_epoch":46,"log_start":"43'6800","ondisk_log_start":"43'6800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15380480,"num_objects":3755,"num_object_clones":0,"num_object_copies":7510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3755,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13233,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":99,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.876013+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:05:19.876013+0000","last_peered":"2024-09-10T16:05:19.876013+0000","last_clean":"2024-09-10T16:05:19.876013+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:05:19.876013+0000","last_undegraded":"2024-09-10T16:05:19.876013+0000","last_fullsized":"2024-09-10T16:05:19.876013+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'17237","reported_seq":17384,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:29.766911+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:06:29.766911+0000","last_peered":"2024-09-10T16:06:29.766911+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:06:29.766911+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"43'7200","ondisk_log_start":"43'7200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15609856,"num_objects":3811,"num_object_clones":0,"num_object_copies":7622,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3811,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13492,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-560}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877349+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:05:19.877349+0000","last_peered":"2024-09-10T16:05:19.877349+0000","last_clean":"2024-09-10T16:05:19.877349+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:05:19.877349+0000","last_undegraded":"2024-09-10T16:05:19.877349+0000","last_fullsized":"2024-09-10T16:05:19.877349+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877269+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:05:19.877269+0000","last_peered":"2024-09-10T16:05:19.877269+0000","last_clean":"2024-09-10T16:05:19.877269+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:05:19.877269+0000","last_undegraded":"2024-09-10T16:05:19.877269+0000","last_fullsized":"2024-09-10T16:05:19.877269+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'17339","reported_seq":17488,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:29.780068+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:06:29.780068+0000","last_peered":"2024-09-10T16:06:29.780068+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:06:29.780068+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"43'7300","ondisk_log_start":"43'7300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15807430,"num_objects":3861,"num_object_clones":0,"num_object_copies":7722,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3861,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13502,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-462},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":250524406,"num_objects":61173,"num_object_clones":0,"num_object_copies":122346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41354,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61173,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":230933,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10035,"num_bytes_recovered":33095680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":708026368,"data_stored":707884543,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160759,"ondisk_log_size":160759,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986278,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":598384,"kb_used_data":259872,"kb_used_omap":4,"kb_used_meta":338491,"kb_avail":93773456,"statfs":{"total":96636764160,"available":96024018944,"internally_reserved":0,"allocated":266108928,"data_stored":265806244,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":346615466},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,1,0,1,1,0,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659224,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":523492,"kb_used_data":180812,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93848348,"statfs":{"total":96636764160,"available":96100708352,"internally_reserved":0,"allocated":185151488,"data_stored":184819436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":350875310},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,2,3,2,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182506,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":687016,"kb_used_data":254164,"kb_used_omap":4,"kb_used_meta":432827,"kb_avail":93684824,"statfs":{"total":96636764160,"available":95933259776,"internally_reserved":0,"allocated":260263936,"data_stored":259961247,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":443215530},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,2,3,3,8],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":258613248,"data_stored":258577118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":184123392,"data_stored":184057899,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":265289728,"data_stored":265249526,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:06:38.228 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:06:38.812 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:06:38.813 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:06:38.840 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":321,"stamp":"2024-09-10T16:06:37.713749+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":242115337,"num_objects":59122,"num_object_clones":0,"num_object_copies":118244,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41236,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59122,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":232988,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10155,"num_bytes_recovered":33095699,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160815,"ondisk_log_size":160815,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1805440,"kb_used_data":683080,"kb_used_omap":12,"kb_used_meta":1122291,"kb_avail":281310080,"statfs":{"total":289910292480,"available":288061521920,"internally_reserved":0,"allocated":699473920,"data_stored":698536495,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1149225986},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,8,15,8,8,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18911232,"num_objects":-4617,"num_object_clones":0,"num_object_copies":-9234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-233,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4617,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4617,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":233,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005015"},"pg_stats":[{"pgid":"2.f","version":"47'17026","reported_seq":17106,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:34.895789+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:06:34.895789+0000","last_peered":"2024-09-10T16:06:34.895789+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:06:34.895789+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"43'7000","ondisk_log_start":"43'7000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14934016,"num_objects":3646,"num_object_clones":0,"num_object_copies":7292,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3646,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17026,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-613}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'17184","reported_seq":17302,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:06:35.700785+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:06:35.700785+0000","last_peered":"2024-09-10T16:06:35.700785+0000","last_clean":"2024-09-10T16:06:35.700785+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:06:35.700785+0000","last_undegraded":"2024-09-10T16:06:35.700785+0000","last_fullsized":"2024-09-10T16:06:35.700785+0000","mapping_epoch":35,"log_start":"43'7100","ondisk_log_start":"43'7100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14992326,"num_objects":3662,"num_object_clones":0,"num_object_copies":7324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3662,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":15289,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'17047","reported_seq":25730,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:06:33.272029+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:06:33.272029+0000","last_peered":"2024-09-10T16:06:33.272029+0000","last_clean":"2024-09-10T16:06:33.272029+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:06:33.272029+0000","last_undegraded":"2024-09-10T16:06:33.272029+0000","last_fullsized":"2024-09-10T16:06:33.272029+0000","mapping_epoch":44,"log_start":"43'7000","ondisk_log_start":"43'7000","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15360024,"num_objects":3751,"num_object_clones":0,"num_object_copies":7502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3751,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15051,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'17108","reported_seq":17194,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:06:35.669059+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:06:35.669059+0000","last_peered":"2024-09-10T16:06:35.669059+0000","last_clean":"2024-09-10T16:06:35.669059+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:06:35.669059+0000","last_undegraded":"2024-09-10T16:06:35.669059+0000","last_fullsized":"2024-09-10T16:06:35.669059+0000","mapping_epoch":35,"log_start":"43'7100","ondisk_log_start":"43'7100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14548992,"num_objects":3552,"num_object_clones":0,"num_object_copies":7104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3552,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17108,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'17211","reported_seq":17345,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:06:35.018167+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:06:35.018167+0000","last_peered":"2024-09-10T16:06:35.018167+0000","last_clean":"2024-09-10T16:06:35.018167+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:06:35.018167+0000","last_undegraded":"2024-09-10T16:06:35.018167+0000","last_fullsized":"2024-09-10T16:06:35.018167+0000","mapping_epoch":34,"log_start":"43'7200","ondisk_log_start":"43'7200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15073763,"num_objects":3681,"num_object_clones":0,"num_object_copies":7362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3681,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13922,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'17367","reported_seq":17510,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:34.789338+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:06:34.789338+0000","last_peered":"2024-09-10T16:06:34.789338+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:06:34.789338+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"43'7300","ondisk_log_start":"43'7300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15131107,"num_objects":3695,"num_object_clones":0,"num_object_copies":7390,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3695,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13979,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-676},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'17329","reported_seq":17469,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:34.974241+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:06:34.974241+0000","last_peered":"2024-09-10T16:06:34.974241+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:06:34.974241+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"43'7300","ondisk_log_start":"43'7300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15373254,"num_objects":3755,"num_object_clones":0,"num_object_copies":7510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3755,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13828,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-482},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'17542","reported_seq":17677,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:06:34.945386+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:06:34.945386+0000","last_peered":"2024-09-10T16:06:34.945386+0000","last_clean":"2024-09-10T16:06:34.945386+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:06:34.945386+0000","last_undegraded":"2024-09-10T16:06:34.945386+0000","last_fullsized":"2024-09-10T16:06:34.945386+0000","mapping_epoch":34,"log_start":"43'7500","ondisk_log_start":"43'7500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15450112,"num_objects":3772,"num_object_clones":0,"num_object_copies":7544,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3772,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14389,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.876895+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:05:19.876895+0000","last_peered":"2024-09-10T16:05:19.876895+0000","last_clean":"2024-09-10T16:05:19.876895+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:05:19.876895+0000","last_undegraded":"2024-09-10T16:05:19.876895+0000","last_fullsized":"2024-09-10T16:05:19.876895+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'16745","reported_seq":16890,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:35.650498+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:06:35.650498+0000","last_peered":"2024-09-10T16:06:35.650498+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:06:35.650498+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"43'6700","ondisk_log_start":"43'6700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15204835,"num_objects":3713,"num_object_clones":0,"num_object_copies":7426,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3713,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14564,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-410}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":99,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.875776+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:05:19.875776+0000","last_peered":"2024-09-10T16:05:19.875776+0000","last_clean":"2024-09-10T16:05:19.875776+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:05:19.875776+0000","last_undegraded":"2024-09-10T16:05:19.875776+0000","last_fullsized":"2024-09-10T16:05:19.875776+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'17154","reported_seq":17290,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:34.988787+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:06:34.988787+0000","last_peered":"2024-09-10T16:06:34.988787+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:06:34.988787+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"43'7100","ondisk_log_start":"43'7100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15384576,"num_objects":3756,"num_object_clones":0,"num_object_copies":7512,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3756,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13658,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-499}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":110,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.872023+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:05:19.872023+0000","last_peered":"2024-09-10T16:05:19.872023+0000","last_clean":"2024-09-10T16:05:19.872023+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:05:19.872023+0000","last_undegraded":"2024-09-10T16:05:19.872023+0000","last_fullsized":"2024-09-10T16:05:19.872023+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'17170","reported_seq":17310,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:35.558922+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:06:35.558922+0000","last_peered":"2024-09-10T16:06:35.558922+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:06:35.558922+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"43'7100","ondisk_log_start":"43'7100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14791139,"num_objects":3612,"num_object_clones":0,"num_object_copies":7224,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3612,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14850,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-692}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877148+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:05:19.877148+0000","last_peered":"2024-09-10T16:05:19.877148+0000","last_clean":"2024-09-10T16:05:19.877148+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:05:19.877148+0000","last_undegraded":"2024-09-10T16:05:19.877148+0000","last_fullsized":"2024-09-10T16:05:19.877148+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'17477","reported_seq":20963,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:35.659368+0000","last_change":"2024-09-10T16:05:18.271152+0000","last_active":"2024-09-10T16:06:35.659368+0000","last_peered":"2024-09-10T16:06:35.659368+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:06:35.659368+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"43'7400","ondisk_log_start":"43'7400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15126528,"num_objects":3693,"num_object_clones":0,"num_object_copies":7386,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2679,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3693,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15031,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1675,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1014}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'17156","reported_seq":17296,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:34.947800+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:06:34.947800+0000","last_peered":"2024-09-10T16:06:34.947800+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:06:34.947800+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"43'7100","ondisk_log_start":"43'7100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15081472,"num_objects":3682,"num_object_clones":0,"num_object_copies":7364,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3682,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13787,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-622}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877654+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:05:19.877654+0000","last_peered":"2024-09-10T16:05:19.877654+0000","last_clean":"2024-09-10T16:05:19.877654+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:05:19.877654+0000","last_undegraded":"2024-09-10T16:05:19.877654+0000","last_fullsized":"2024-09-10T16:05:19.877654+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'16843","reported_seq":25414,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:06:33.256732+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:06:33.256732+0000","last_peered":"2024-09-10T16:06:33.256732+0000","last_clean":"2024-09-10T16:06:33.256732+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:06:33.256732+0000","last_undegraded":"2024-09-10T16:06:33.256732+0000","last_fullsized":"2024-09-10T16:06:33.256732+0000","mapping_epoch":46,"log_start":"43'6800","ondisk_log_start":"43'6800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15380480,"num_objects":3755,"num_object_clones":0,"num_object_copies":7510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3755,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13233,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":99,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.876013+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:05:19.876013+0000","last_peered":"2024-09-10T16:05:19.876013+0000","last_clean":"2024-09-10T16:05:19.876013+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:05:19.876013+0000","last_undegraded":"2024-09-10T16:05:19.876013+0000","last_fullsized":"2024-09-10T16:05:19.876013+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'17396","reported_seq":17543,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:34.983876+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:06:34.983876+0000","last_peered":"2024-09-10T16:06:34.983876+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:06:34.983876+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"43'7300","ondisk_log_start":"43'7300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14958592,"num_objects":3652,"num_object_clones":0,"num_object_copies":7304,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3652,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13651,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-719}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877349+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:05:19.877349+0000","last_peered":"2024-09-10T16:05:19.877349+0000","last_clean":"2024-09-10T16:05:19.877349+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:05:19.877349+0000","last_undegraded":"2024-09-10T16:05:19.877349+0000","last_fullsized":"2024-09-10T16:05:19.877349+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877269+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:05:19.877269+0000","last_peered":"2024-09-10T16:05:19.877269+0000","last_clean":"2024-09-10T16:05:19.877269+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:05:19.877269+0000","last_undegraded":"2024-09-10T16:05:19.877269+0000","last_fullsized":"2024-09-10T16:05:19.877269+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'17457","reported_seq":17607,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:35.011358+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:06:35.011358+0000","last_peered":"2024-09-10T16:06:35.011358+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:06:35.011358+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"43'7400","ondisk_log_start":"43'7400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15324102,"num_objects":3743,"num_object_clones":0,"num_object_copies":7486,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3743,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13620,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-580},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":242115318,"num_objects":59120,"num_object_clones":0,"num_object_copies":118240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41236,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59120,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":232986,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10153,"num_bytes_recovered":33095680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":696131584,"data_stored":695989759,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160812,"ondisk_log_size":160812,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986279,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":597816,"kb_used_data":255144,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93774024,"statfs":{"total":96636764160,"available":96024600576,"internally_reserved":0,"allocated":261267456,"data_stored":260964772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":350875306},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,6,2,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659224,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":523492,"kb_used_data":180812,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93848348,"statfs":{"total":96636764160,"available":96100708352,"internally_reserved":0,"allocated":185151488,"data_stored":184819436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":350875310},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,2,3,2,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182507,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":684132,"kb_used_data":247124,"kb_used_omap":4,"kb_used_meta":436987,"kb_avail":93687708,"statfs":{"total":96636764160,"available":95936212992,"internally_reserved":0,"allocated":253054976,"data_stored":252752287,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":447475370},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,6,4,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":251453440,"data_stored":251417310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":184123392,"data_stored":184057899,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":260554752,"data_stored":260514550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:06:38.841 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:06:39.403 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:06:39.405 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:06:39.427 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":321,"stamp":"2024-09-10T16:06:37.713749+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":242115337,"num_objects":59122,"num_object_clones":0,"num_object_copies":118244,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41236,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59122,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":232988,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10155,"num_bytes_recovered":33095699,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160815,"ondisk_log_size":160815,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1805440,"kb_used_data":683080,"kb_used_omap":12,"kb_used_meta":1122291,"kb_avail":281310080,"statfs":{"total":289910292480,"available":288061521920,"internally_reserved":0,"allocated":699473920,"data_stored":698536495,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1149225986},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,8,15,8,8,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18911232,"num_objects":-4617,"num_object_clones":0,"num_object_copies":-9234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-233,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4617,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4617,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":233,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005015"},"pg_stats":[{"pgid":"2.f","version":"47'17026","reported_seq":17106,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:34.895789+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:06:34.895789+0000","last_peered":"2024-09-10T16:06:34.895789+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:06:34.895789+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"43'7000","ondisk_log_start":"43'7000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14934016,"num_objects":3646,"num_object_clones":0,"num_object_copies":7292,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3646,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17026,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-613}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'17184","reported_seq":17302,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:06:35.700785+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:06:35.700785+0000","last_peered":"2024-09-10T16:06:35.700785+0000","last_clean":"2024-09-10T16:06:35.700785+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:06:35.700785+0000","last_undegraded":"2024-09-10T16:06:35.700785+0000","last_fullsized":"2024-09-10T16:06:35.700785+0000","mapping_epoch":35,"log_start":"43'7100","ondisk_log_start":"43'7100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14992326,"num_objects":3662,"num_object_clones":0,"num_object_copies":7324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3662,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":15289,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'17047","reported_seq":25730,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:06:33.272029+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:06:33.272029+0000","last_peered":"2024-09-10T16:06:33.272029+0000","last_clean":"2024-09-10T16:06:33.272029+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:06:33.272029+0000","last_undegraded":"2024-09-10T16:06:33.272029+0000","last_fullsized":"2024-09-10T16:06:33.272029+0000","mapping_epoch":44,"log_start":"43'7000","ondisk_log_start":"43'7000","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15360024,"num_objects":3751,"num_object_clones":0,"num_object_copies":7502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3751,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15051,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'17108","reported_seq":17194,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:06:35.669059+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:06:35.669059+0000","last_peered":"2024-09-10T16:06:35.669059+0000","last_clean":"2024-09-10T16:06:35.669059+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:06:35.669059+0000","last_undegraded":"2024-09-10T16:06:35.669059+0000","last_fullsized":"2024-09-10T16:06:35.669059+0000","mapping_epoch":35,"log_start":"43'7100","ondisk_log_start":"43'7100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14548992,"num_objects":3552,"num_object_clones":0,"num_object_copies":7104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3552,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17108,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'17211","reported_seq":17345,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:06:35.018167+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:06:35.018167+0000","last_peered":"2024-09-10T16:06:35.018167+0000","last_clean":"2024-09-10T16:06:35.018167+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:06:35.018167+0000","last_undegraded":"2024-09-10T16:06:35.018167+0000","last_fullsized":"2024-09-10T16:06:35.018167+0000","mapping_epoch":34,"log_start":"43'7200","ondisk_log_start":"43'7200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15073763,"num_objects":3681,"num_object_clones":0,"num_object_copies":7362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3681,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13922,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'17367","reported_seq":17510,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:34.789338+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:06:34.789338+0000","last_peered":"2024-09-10T16:06:34.789338+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:06:34.789338+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"43'7300","ondisk_log_start":"43'7300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15131107,"num_objects":3695,"num_object_clones":0,"num_object_copies":7390,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3695,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13979,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-676},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'17329","reported_seq":17469,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:34.974241+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:06:34.974241+0000","last_peered":"2024-09-10T16:06:34.974241+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:06:34.974241+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"43'7300","ondisk_log_start":"43'7300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15373254,"num_objects":3755,"num_object_clones":0,"num_object_copies":7510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3755,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13828,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-482},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'17542","reported_seq":17677,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:06:34.945386+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:06:34.945386+0000","last_peered":"2024-09-10T16:06:34.945386+0000","last_clean":"2024-09-10T16:06:34.945386+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:06:34.945386+0000","last_undegraded":"2024-09-10T16:06:34.945386+0000","last_fullsized":"2024-09-10T16:06:34.945386+0000","mapping_epoch":34,"log_start":"43'7500","ondisk_log_start":"43'7500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15450112,"num_objects":3772,"num_object_clones":0,"num_object_copies":7544,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3772,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14389,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.876895+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:05:19.876895+0000","last_peered":"2024-09-10T16:05:19.876895+0000","last_clean":"2024-09-10T16:05:19.876895+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:05:19.876895+0000","last_undegraded":"2024-09-10T16:05:19.876895+0000","last_fullsized":"2024-09-10T16:05:19.876895+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'16745","reported_seq":16890,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:35.650498+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:06:35.650498+0000","last_peered":"2024-09-10T16:06:35.650498+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:06:35.650498+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"43'6700","ondisk_log_start":"43'6700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15204835,"num_objects":3713,"num_object_clones":0,"num_object_copies":7426,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3713,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14564,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-410}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":99,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.875776+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:05:19.875776+0000","last_peered":"2024-09-10T16:05:19.875776+0000","last_clean":"2024-09-10T16:05:19.875776+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:05:19.875776+0000","last_undegraded":"2024-09-10T16:05:19.875776+0000","last_fullsized":"2024-09-10T16:05:19.875776+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'17154","reported_seq":17290,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:34.988787+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:06:34.988787+0000","last_peered":"2024-09-10T16:06:34.988787+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:06:34.988787+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"43'7100","ondisk_log_start":"43'7100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15384576,"num_objects":3756,"num_object_clones":0,"num_object_copies":7512,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3756,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13658,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-499}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":110,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.872023+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:05:19.872023+0000","last_peered":"2024-09-10T16:05:19.872023+0000","last_clean":"2024-09-10T16:05:19.872023+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:05:19.872023+0000","last_undegraded":"2024-09-10T16:05:19.872023+0000","last_fullsized":"2024-09-10T16:05:19.872023+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'17170","reported_seq":17310,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:35.558922+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:06:35.558922+0000","last_peered":"2024-09-10T16:06:35.558922+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:06:35.558922+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"43'7100","ondisk_log_start":"43'7100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14791139,"num_objects":3612,"num_object_clones":0,"num_object_copies":7224,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3612,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14850,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-692}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877148+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:05:19.877148+0000","last_peered":"2024-09-10T16:05:19.877148+0000","last_clean":"2024-09-10T16:05:19.877148+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:05:19.877148+0000","last_undegraded":"2024-09-10T16:05:19.877148+0000","last_fullsized":"2024-09-10T16:05:19.877148+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'17477","reported_seq":20963,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:35.659368+0000","last_change":"2024-09-10T16:05:18.271152+0000","last_active":"2024-09-10T16:06:35.659368+0000","last_peered":"2024-09-10T16:06:35.659368+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:06:35.659368+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"43'7400","ondisk_log_start":"43'7400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15126528,"num_objects":3693,"num_object_clones":0,"num_object_copies":7386,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2679,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3693,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15031,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1675,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1014}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'17156","reported_seq":17296,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:34.947800+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:06:34.947800+0000","last_peered":"2024-09-10T16:06:34.947800+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:06:34.947800+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"43'7100","ondisk_log_start":"43'7100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15081472,"num_objects":3682,"num_object_clones":0,"num_object_copies":7364,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3682,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13787,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-622}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877654+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:05:19.877654+0000","last_peered":"2024-09-10T16:05:19.877654+0000","last_clean":"2024-09-10T16:05:19.877654+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:05:19.877654+0000","last_undegraded":"2024-09-10T16:05:19.877654+0000","last_fullsized":"2024-09-10T16:05:19.877654+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'16843","reported_seq":25414,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:06:33.256732+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:06:33.256732+0000","last_peered":"2024-09-10T16:06:33.256732+0000","last_clean":"2024-09-10T16:06:33.256732+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:06:33.256732+0000","last_undegraded":"2024-09-10T16:06:33.256732+0000","last_fullsized":"2024-09-10T16:06:33.256732+0000","mapping_epoch":46,"log_start":"43'6800","ondisk_log_start":"43'6800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15380480,"num_objects":3755,"num_object_clones":0,"num_object_copies":7510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3755,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13233,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":99,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.876013+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:05:19.876013+0000","last_peered":"2024-09-10T16:05:19.876013+0000","last_clean":"2024-09-10T16:05:19.876013+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:05:19.876013+0000","last_undegraded":"2024-09-10T16:05:19.876013+0000","last_fullsized":"2024-09-10T16:05:19.876013+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'17396","reported_seq":17543,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:34.983876+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:06:34.983876+0000","last_peered":"2024-09-10T16:06:34.983876+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:06:34.983876+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"43'7300","ondisk_log_start":"43'7300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14958592,"num_objects":3652,"num_object_clones":0,"num_object_copies":7304,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3652,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13651,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-719}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877349+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:05:19.877349+0000","last_peered":"2024-09-10T16:05:19.877349+0000","last_clean":"2024-09-10T16:05:19.877349+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:05:19.877349+0000","last_undegraded":"2024-09-10T16:05:19.877349+0000","last_fullsized":"2024-09-10T16:05:19.877349+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877269+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:05:19.877269+0000","last_peered":"2024-09-10T16:05:19.877269+0000","last_clean":"2024-09-10T16:05:19.877269+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:05:19.877269+0000","last_undegraded":"2024-09-10T16:05:19.877269+0000","last_fullsized":"2024-09-10T16:05:19.877269+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'17457","reported_seq":17607,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:35.011358+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:06:35.011358+0000","last_peered":"2024-09-10T16:06:35.011358+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:06:35.011358+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"43'7400","ondisk_log_start":"43'7400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15324102,"num_objects":3743,"num_object_clones":0,"num_object_copies":7486,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3743,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13620,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-580},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":242115318,"num_objects":59120,"num_object_clones":0,"num_object_copies":118240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41236,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59120,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":232986,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10153,"num_bytes_recovered":33095680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":696131584,"data_stored":695989759,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160812,"ondisk_log_size":160812,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986279,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":597816,"kb_used_data":255144,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93774024,"statfs":{"total":96636764160,"available":96024600576,"internally_reserved":0,"allocated":261267456,"data_stored":260964772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":350875306},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,6,2,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659224,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":523492,"kb_used_data":180812,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93848348,"statfs":{"total":96636764160,"available":96100708352,"internally_reserved":0,"allocated":185151488,"data_stored":184819436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":350875310},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,2,3,2,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182507,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":684132,"kb_used_data":247124,"kb_used_omap":4,"kb_used_meta":436987,"kb_avail":93687708,"statfs":{"total":96636764160,"available":95936212992,"internally_reserved":0,"allocated":253054976,"data_stored":252752287,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":447475370},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,6,4,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":251453440,"data_stored":251417310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":184123392,"data_stored":184057899,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":260554752,"data_stored":260514550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:06:39.429 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-10T16:06:39.429 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "47'17026", 'reported_seq': 17106, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:06:34.895789+0000', 'last_change': '2024-09-10T15:59:16.281558+0000', 'last_active': '2024-09-10T16:06:34.895789+0000', 'last_peered': '2024-09-10T16:06:34.895789+0000', 'last_clean': '2024-09-10T15:59:14.132456+0000', 'last_became_active': '2024-09-10T15:59:16.274407+0000', 'last_became_peered': '2024-09-10T15:59:16.274407+0000', 'last_unstale': '2024-09-10T16:06:34.895789+0000', 'last_undegraded': '2024-09-10T15:59:15.851279+0000', 'last_fullsized': '2024-09-10T15:59:15.774356+0000', 'mapping_epoch': 42, 'log_start': "43'7000", 'ondisk_log_start': "43'7000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'objects_scrubbed': 0, 'log_size': 10026, 'log_dups_size': 0, 'ondisk_log_size': 10026, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 14934016, 'num_objects': 3646, 'num_object_clones': 0, 'num_object_copies': 7292, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4259, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3646, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17026, 'num_write_kb': 41344, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -613}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:06:39.429 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-10T16:06:39.429 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "47'17367", 'reported_seq': 17510, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:06:34.789338+0000', 'last_change': '2024-09-10T15:59:16.308264+0000', 'last_active': '2024-09-10T16:06:34.789338+0000', 'last_peered': '2024-09-10T16:06:34.789338+0000', 'last_clean': '2024-09-10T15:59:14.111551+0000', 'last_became_active': '2024-09-10T15:59:16.300040+0000', 'last_became_peered': '2024-09-10T15:59:16.300040+0000', 'last_unstale': '2024-09-10T16:06:34.789338+0000', 'last_undegraded': '2024-09-10T15:59:15.878234+0000', 'last_fullsized': '2024-09-10T15:59:15.769328+0000', 'mapping_epoch': 42, 'log_start': "43'7300", 'ondisk_log_start': "43'7300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3442", 'last_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'objects_scrubbed': 3384, 'log_size': 10067, 'log_dups_size': 0, 'ondisk_log_size': 10067, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1292, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 15131107, 'num_objects': 3695, 'num_object_clones': 0, 'num_object_copies': 7390, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4372, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3695, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 13979, 'num_write_kb': 28580, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -676}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:06:39.429 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-10T16:06:39.429 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "47'17329", 'reported_seq': 17469, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:06:34.974241+0000', 'last_change': '2024-09-10T15:59:16.331899+0000', 'last_active': '2024-09-10T16:06:34.974241+0000', 'last_peered': '2024-09-10T16:06:34.974241+0000', 'last_clean': '2024-09-10T15:59:14.139436+0000', 'last_became_active': '2024-09-10T15:59:16.318715+0000', 'last_became_peered': '2024-09-10T15:59:16.318715+0000', 'last_unstale': '2024-09-10T16:06:34.974241+0000', 'last_undegraded': '2024-09-10T15:59:15.877524+0000', 'last_fullsized': '2024-09-10T15:59:15.769042+0000', 'mapping_epoch': 42, 'log_start': "43'7300", 'ondisk_log_start': "43'7300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3537", 'last_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'objects_scrubbed': 3497, 'log_size': 10029, 'log_dups_size': 0, 'ondisk_log_size': 10029, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1224, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 15373254, 'num_objects': 3755, 'num_object_clones': 0, 'num_object_copies': 7510, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4237, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3755, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 13828, 'num_write_kb': 28172, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -482}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:06:39.429 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-10T16:06:39.429 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "47'16745", 'reported_seq': 16890, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:06:35.650498+0000', 'last_change': '2024-09-10T15:59:16.175680+0000', 'last_active': '2024-09-10T16:06:35.650498+0000', 'last_peered': '2024-09-10T16:06:35.650498+0000', 'last_clean': '2024-09-10T15:59:14.088187+0000', 'last_became_active': '2024-09-10T15:59:16.165810+0000', 'last_became_peered': '2024-09-10T15:59:16.165810+0000', 'last_unstale': '2024-09-10T16:06:35.650498+0000', 'last_undegraded': '2024-09-10T15:59:15.822098+0000', 'last_fullsized': '2024-09-10T15:59:15.769123+0000', 'mapping_epoch': 42, 'log_start': "43'6700", 'ondisk_log_start': "43'6700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2218", 'last_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'objects_scrubbed': 2175, 'log_size': 10045, 'log_dups_size': 0, 'ondisk_log_size': 10045, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 825, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 15204835, 'num_objects': 3713, 'num_object_clones': 0, 'num_object_copies': 7426, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4124, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3713, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 14564, 'num_write_kb': 32204, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': -410}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:06:39.429 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-10T16:06:39.430 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "47'17154", 'reported_seq': 17290, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:06:34.988787+0000', 'last_change': '2024-09-10T15:59:16.277742+0000', 'last_active': '2024-09-10T16:06:34.988787+0000', 'last_peered': '2024-09-10T16:06:34.988787+0000', 'last_clean': '2024-09-10T15:59:14.129081+0000', 'last_became_active': '2024-09-10T15:59:16.269835+0000', 'last_became_peered': '2024-09-10T15:59:16.269835+0000', 'last_unstale': '2024-09-10T16:06:34.988787+0000', 'last_undegraded': '2024-09-10T15:59:15.860857+0000', 'last_fullsized': '2024-09-10T15:59:15.771410+0000', 'mapping_epoch': 42, 'log_start': "43'7100", 'ondisk_log_start': "43'7100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3557", 'last_scrub_stamp': '2024-09-10T15:59:02.493678+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:02.493678+0000', 'objects_scrubbed': 3488, 'log_size': 10054, 'log_dups_size': 0, 'ondisk_log_size': 10054, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1472, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 15384576, 'num_objects': 3756, 'num_object_clones': 0, 'num_object_copies': 7512, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4257, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3756, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13658, 'num_write_kb': 27852, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -499}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:06:39.430 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-10T16:06:39.430 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "47'17170", 'reported_seq': 17310, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:06:35.558922+0000', 'last_change': '2024-09-10T15:59:16.284311+0000', 'last_active': '2024-09-10T16:06:35.558922+0000', 'last_peered': '2024-09-10T16:06:35.558922+0000', 'last_clean': '2024-09-10T15:59:14.116041+0000', 'last_became_active': '2024-09-10T15:59:16.274543+0000', 'last_became_peered': '2024-09-10T15:59:16.274543+0000', 'last_unstale': '2024-09-10T16:06:35.558922+0000', 'last_undegraded': '2024-09-10T15:59:15.812484+0000', 'last_fullsized': '2024-09-10T15:59:15.770885+0000', 'mapping_epoch': 42, 'log_start': "43'7100", 'ondisk_log_start': "43'7100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2356", 'last_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'objects_scrubbed': 2316, 'log_size': 10070, 'log_dups_size': 0, 'ondisk_log_size': 10070, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 783, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 14791139, 'num_objects': 3612, 'num_object_clones': 0, 'num_object_copies': 7224, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3612, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 14850, 'num_write_kb': 32292, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': -692}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:06:39.430 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-10T16:06:39.430 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "47'17477", 'reported_seq': 20963, 'reported_epoch': 47, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:06:35.659368+0000', 'last_change': '2024-09-10T16:05:18.271152+0000', 'last_active': '2024-09-10T16:06:35.659368+0000', 'last_peered': '2024-09-10T16:06:35.659368+0000', 'last_clean': '2024-09-10T15:59:14.110654+0000', 'last_became_active': '2024-09-10T15:59:16.164645+0000', 'last_became_peered': '2024-09-10T15:59:16.164645+0000', 'last_unstale': '2024-09-10T16:06:35.659368+0000', 'last_undegraded': '2024-09-10T15:59:15.803593+0000', 'last_fullsized': '2024-09-10T15:59:15.769083+0000', 'mapping_epoch': 42, 'log_start': "43'7400", 'ondisk_log_start': "43'7400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2486", 'last_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'objects_scrubbed': 2440, 'log_size': 10077, 'log_dups_size': 0, 'ondisk_log_size': 10077, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 792, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 15126528, 'num_objects': 3693, 'num_object_clones': 0, 'num_object_copies': 7386, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2679, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3693, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15031, 'num_write_kb': 32568, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1675, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 1014}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:06:39.430 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-10T16:06:39.430 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "47'17156", 'reported_seq': 17296, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:06:34.947800+0000', 'last_change': '2024-09-10T15:59:16.315641+0000', 'last_active': '2024-09-10T16:06:34.947800+0000', 'last_peered': '2024-09-10T16:06:34.947800+0000', 'last_clean': '2024-09-10T15:59:14.117001+0000', 'last_became_active': '2024-09-10T15:59:16.302245+0000', 'last_became_peered': '2024-09-10T15:59:16.302245+0000', 'last_unstale': '2024-09-10T16:06:34.947800+0000', 'last_undegraded': '2024-09-10T15:59:15.845012+0000', 'last_fullsized': '2024-09-10T15:59:15.770337+0000', 'mapping_epoch': 42, 'log_start': "43'7100", 'ondisk_log_start': "43'7100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3404", 'last_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'objects_scrubbed': 3367, 'log_size': 10056, 'log_dups_size': 0, 'ondisk_log_size': 10056, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 983, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 15081472, 'num_objects': 3682, 'num_object_clones': 0, 'num_object_copies': 7364, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3682, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13787, 'num_write_kb': 28204, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -622}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:06:39.430 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-10T16:06:39.430 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "47'17396", 'reported_seq': 17543, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:06:34.983876+0000', 'last_change': '2024-09-10T15:59:16.311291+0000', 'last_active': '2024-09-10T16:06:34.983876+0000', 'last_peered': '2024-09-10T16:06:34.983876+0000', 'last_clean': '2024-09-10T15:59:14.116363+0000', 'last_became_active': '2024-09-10T15:59:16.301244+0000', 'last_became_peered': '2024-09-10T15:59:16.301244+0000', 'last_unstale': '2024-09-10T16:06:34.983876+0000', 'last_undegraded': '2024-09-10T15:59:15.883434+0000', 'last_fullsized': '2024-09-10T15:59:15.772756+0000', 'mapping_epoch': 42, 'log_start': "43'7300", 'ondisk_log_start': "43'7300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3789", 'last_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'objects_scrubbed': 3741, 'log_size': 10096, 'log_dups_size': 0, 'ondisk_log_size': 10096, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1440, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 14958592, 'num_objects': 3652, 'num_object_clones': 0, 'num_object_copies': 7304, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4373, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3652, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13651, 'num_write_kb': 27124, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -719}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:06:39.430 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-10T16:06:39.430 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "47'17457", 'reported_seq': 17607, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:06:35.011358+0000', 'last_change': '2024-09-10T15:59:16.313409+0000', 'last_active': '2024-09-10T16:06:35.011358+0000', 'last_peered': '2024-09-10T16:06:35.011358+0000', 'last_clean': '2024-09-10T15:59:14.129780+0000', 'last_became_active': '2024-09-10T15:59:16.300932+0000', 'last_became_peered': '2024-09-10T15:59:16.300932+0000', 'last_unstale': '2024-09-10T16:06:35.011358+0000', 'last_undegraded': '2024-09-10T15:59:15.887823+0000', 'last_fullsized': '2024-09-10T15:59:15.768513+0000', 'mapping_epoch': 42, 'log_start': "43'7400", 'ondisk_log_start': "43'7400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3880", 'last_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'objects_scrubbed': 3827, 'log_size': 10057, 'log_dups_size': 0, 'ondisk_log_size': 10057, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1397, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 15324102, 'num_objects': 3743, 'num_object_clones': 0, 'num_object_copies': 7486, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4325, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3743, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13620, 'num_write_kb': 27072, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -580}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:06:39.431 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T16:06:40.103 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:06:40.104 INFO:teuthology.orchestra.run.smithi115.stdout:{"fsid":"7effee29-df05-4351-9741-ee203564ed81","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 41236/118244 objects degraded (34.874%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":564,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":47,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983954,"num_in_osds":3,"osd_in_since":1725983947,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":59122,"data_bytes":242115337,"bytes_used":1848770560,"bytes_avail":288061521920,"bytes_total":289910292480,"degraded_objects":41236,"degraded_total":118244,"degraded_ratio":0.3487365109434728,"recovering_objects_per_sec":19,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":233,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":384},"fsmap":{"epoch":1,"btime":"2024-09-10T15:57:14:967120+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T16:00:13.560776+0000","services":{}},"progress_events":{}} 2024-09-10T16:06:40.125 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-10T16:06:40.126 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:06:40.708 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:06:40.710 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:06:40.733 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":322,"stamp":"2024-09-10T16:06:39.717972+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":240771849,"num_objects":58794,"num_object_clones":0,"num_object_copies":117588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41236,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58794,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":233316,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10155,"num_bytes_recovered":33095699,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160843,"ondisk_log_size":160843,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1806956,"kb_used_data":676268,"kb_used_omap":12,"kb_used_meta":1130611,"kb_avail":281308564,"statfs":{"total":289910292480,"available":288059969536,"internally_reserved":0,"allocated":692498432,"data_stored":691561007,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1157745666},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,8,14,13,8,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-20254720,"num_objects":-4945,"num_object_clones":0,"num_object_copies":-9890,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-233,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4945,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4945,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":233,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.008433"},"pg_stats":[{"pgid":"2.f","version":"47'17026","reported_seq":17106,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:34.895789+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:06:34.895789+0000","last_peered":"2024-09-10T16:06:34.895789+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:06:34.895789+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"43'7000","ondisk_log_start":"43'7000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14934016,"num_objects":3646,"num_object_clones":0,"num_object_copies":7292,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3646,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17026,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-613}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'17184","reported_seq":17302,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:06:35.700785+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:06:35.700785+0000","last_peered":"2024-09-10T16:06:35.700785+0000","last_clean":"2024-09-10T16:06:35.700785+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:06:35.700785+0000","last_undegraded":"2024-09-10T16:06:35.700785+0000","last_fullsized":"2024-09-10T16:06:35.700785+0000","mapping_epoch":35,"log_start":"43'7100","ondisk_log_start":"43'7100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14992326,"num_objects":3662,"num_object_clones":0,"num_object_copies":7324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3662,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":15289,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'17220","reported_seq":25903,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:06:38.354748+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:06:38.354748+0000","last_peered":"2024-09-10T16:06:38.354748+0000","last_clean":"2024-09-10T16:06:38.354748+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:06:38.354748+0000","last_undegraded":"2024-09-10T16:06:38.354748+0000","last_fullsized":"2024-09-10T16:06:38.354748+0000","mapping_epoch":44,"log_start":"43'7200","ondisk_log_start":"43'7200","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14651416,"num_objects":3578,"num_object_clones":0,"num_object_copies":7156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3578,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15224,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'17108","reported_seq":17194,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:06:35.669059+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:06:35.669059+0000","last_peered":"2024-09-10T16:06:35.669059+0000","last_clean":"2024-09-10T16:06:35.669059+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:06:35.669059+0000","last_undegraded":"2024-09-10T16:06:35.669059+0000","last_fullsized":"2024-09-10T16:06:35.669059+0000","mapping_epoch":35,"log_start":"43'7100","ondisk_log_start":"43'7100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14548992,"num_objects":3552,"num_object_clones":0,"num_object_copies":7104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3552,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17108,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'17211","reported_seq":17345,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:06:35.018167+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:06:35.018167+0000","last_peered":"2024-09-10T16:06:35.018167+0000","last_clean":"2024-09-10T16:06:35.018167+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:06:35.018167+0000","last_undegraded":"2024-09-10T16:06:35.018167+0000","last_fullsized":"2024-09-10T16:06:35.018167+0000","mapping_epoch":34,"log_start":"43'7200","ondisk_log_start":"43'7200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15073763,"num_objects":3681,"num_object_clones":0,"num_object_copies":7362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3681,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13922,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'17367","reported_seq":17510,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:34.789338+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:06:34.789338+0000","last_peered":"2024-09-10T16:06:34.789338+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:06:34.789338+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"43'7300","ondisk_log_start":"43'7300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15131107,"num_objects":3695,"num_object_clones":0,"num_object_copies":7390,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3695,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13979,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-676},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'17329","reported_seq":17469,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:34.974241+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:06:34.974241+0000","last_peered":"2024-09-10T16:06:34.974241+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:06:34.974241+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"43'7300","ondisk_log_start":"43'7300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15373254,"num_objects":3755,"num_object_clones":0,"num_object_copies":7510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3755,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13828,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-482},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'17542","reported_seq":17677,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:06:34.945386+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:06:34.945386+0000","last_peered":"2024-09-10T16:06:34.945386+0000","last_clean":"2024-09-10T16:06:34.945386+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:06:34.945386+0000","last_undegraded":"2024-09-10T16:06:34.945386+0000","last_fullsized":"2024-09-10T16:06:34.945386+0000","mapping_epoch":34,"log_start":"43'7500","ondisk_log_start":"43'7500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15450112,"num_objects":3772,"num_object_clones":0,"num_object_copies":7544,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3772,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14389,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.876895+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:05:19.876895+0000","last_peered":"2024-09-10T16:05:19.876895+0000","last_clean":"2024-09-10T16:05:19.876895+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:05:19.876895+0000","last_undegraded":"2024-09-10T16:05:19.876895+0000","last_fullsized":"2024-09-10T16:05:19.876895+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'16745","reported_seq":16890,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:35.650498+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:06:35.650498+0000","last_peered":"2024-09-10T16:06:35.650498+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:06:35.650498+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"43'6700","ondisk_log_start":"43'6700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15204835,"num_objects":3713,"num_object_clones":0,"num_object_copies":7426,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3713,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14564,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-410}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":99,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.875776+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:05:19.875776+0000","last_peered":"2024-09-10T16:05:19.875776+0000","last_clean":"2024-09-10T16:05:19.875776+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:05:19.875776+0000","last_undegraded":"2024-09-10T16:05:19.875776+0000","last_fullsized":"2024-09-10T16:05:19.875776+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'17154","reported_seq":17290,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:34.988787+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:06:34.988787+0000","last_peered":"2024-09-10T16:06:34.988787+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:06:34.988787+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"43'7100","ondisk_log_start":"43'7100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15384576,"num_objects":3756,"num_object_clones":0,"num_object_copies":7512,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3756,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13658,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-499}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":110,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.872023+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:05:19.872023+0000","last_peered":"2024-09-10T16:05:19.872023+0000","last_clean":"2024-09-10T16:05:19.872023+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:05:19.872023+0000","last_undegraded":"2024-09-10T16:05:19.872023+0000","last_fullsized":"2024-09-10T16:05:19.872023+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'17170","reported_seq":17310,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:35.558922+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:06:35.558922+0000","last_peered":"2024-09-10T16:06:35.558922+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:06:35.558922+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"43'7100","ondisk_log_start":"43'7100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14791139,"num_objects":3612,"num_object_clones":0,"num_object_copies":7224,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3612,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14850,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-692}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877148+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:05:19.877148+0000","last_peered":"2024-09-10T16:05:19.877148+0000","last_clean":"2024-09-10T16:05:19.877148+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:05:19.877148+0000","last_undegraded":"2024-09-10T16:05:19.877148+0000","last_fullsized":"2024-09-10T16:05:19.877148+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'17477","reported_seq":20963,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:35.659368+0000","last_change":"2024-09-10T16:05:18.271152+0000","last_active":"2024-09-10T16:06:35.659368+0000","last_peered":"2024-09-10T16:06:35.659368+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:06:35.659368+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"43'7400","ondisk_log_start":"43'7400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15126528,"num_objects":3693,"num_object_clones":0,"num_object_copies":7386,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2679,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3693,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15031,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1675,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1014}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'17156","reported_seq":17296,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:34.947800+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:06:34.947800+0000","last_peered":"2024-09-10T16:06:34.947800+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:06:34.947800+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"43'7100","ondisk_log_start":"43'7100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15081472,"num_objects":3682,"num_object_clones":0,"num_object_copies":7364,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3682,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13787,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-622}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877654+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:05:19.877654+0000","last_peered":"2024-09-10T16:05:19.877654+0000","last_clean":"2024-09-10T16:05:19.877654+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:05:19.877654+0000","last_undegraded":"2024-09-10T16:05:19.877654+0000","last_fullsized":"2024-09-10T16:05:19.877654+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'16998","reported_seq":25569,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:06:38.385323+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:06:38.385323+0000","last_peered":"2024-09-10T16:06:38.385323+0000","last_clean":"2024-09-10T16:06:38.385323+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:06:38.385323+0000","last_undegraded":"2024-09-10T16:06:38.385323+0000","last_fullsized":"2024-09-10T16:06:38.385323+0000","mapping_epoch":46,"log_start":"43'6900","ondisk_log_start":"43'6900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14745600,"num_objects":3600,"num_object_clones":0,"num_object_copies":7200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3600,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13388,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":99,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.876013+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:05:19.876013+0000","last_peered":"2024-09-10T16:05:19.876013+0000","last_clean":"2024-09-10T16:05:19.876013+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:05:19.876013+0000","last_undegraded":"2024-09-10T16:05:19.876013+0000","last_fullsized":"2024-09-10T16:05:19.876013+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'17396","reported_seq":17543,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:34.983876+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:06:34.983876+0000","last_peered":"2024-09-10T16:06:34.983876+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:06:34.983876+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"43'7300","ondisk_log_start":"43'7300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14958592,"num_objects":3652,"num_object_clones":0,"num_object_copies":7304,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3652,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13651,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-719}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877349+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:05:19.877349+0000","last_peered":"2024-09-10T16:05:19.877349+0000","last_clean":"2024-09-10T16:05:19.877349+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:05:19.877349+0000","last_undegraded":"2024-09-10T16:05:19.877349+0000","last_fullsized":"2024-09-10T16:05:19.877349+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877269+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:05:19.877269+0000","last_peered":"2024-09-10T16:05:19.877269+0000","last_clean":"2024-09-10T16:05:19.877269+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:05:19.877269+0000","last_undegraded":"2024-09-10T16:05:19.877269+0000","last_fullsized":"2024-09-10T16:05:19.877269+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'17457","reported_seq":17607,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:35.011358+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:06:35.011358+0000","last_peered":"2024-09-10T16:06:35.011358+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:06:35.011358+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"43'7400","ondisk_log_start":"43'7400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15324102,"num_objects":3743,"num_object_clones":0,"num_object_copies":7486,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3743,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13620,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-580},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":240771830,"num_objects":58792,"num_object_clones":0,"num_object_copies":117584,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41236,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58792,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":233314,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10153,"num_bytes_recovered":33095680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":688869376,"data_stored":688727551,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160840,"ondisk_log_size":160840,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986279,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":597816,"kb_used_data":255144,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93774024,"statfs":{"total":96636764160,"available":96024600576,"internally_reserved":0,"allocated":261267456,"data_stored":260964772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":350875306},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,6,2,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659225,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":525008,"kb_used_data":174000,"kb_used_omap":4,"kb_used_meta":350971,"kb_avail":93846832,"statfs":{"total":96636764160,"available":96099155968,"internally_reserved":0,"allocated":178176000,"data_stored":177843948,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":359394990},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,2,7,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182507,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":684132,"kb_used_data":247124,"kb_used_omap":4,"kb_used_meta":436987,"kb_avail":93687708,"statfs":{"total":96636764160,"available":95936212992,"internally_reserved":0,"allocated":253054976,"data_stored":252752287,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":447475370},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,6,4,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":251453440,"data_stored":251417310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":176861184,"data_stored":176795691,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":260554752,"data_stored":260514550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:06:43.735 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:06:44.300 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:06:44.300 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:06:44.320 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":324,"stamp":"2024-09-10T16:06:43.718899+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":231318281,"num_objects":56486,"num_object_clones":0,"num_object_copies":112972,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41108,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56486,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":235624,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10283,"num_bytes_recovered":33095699,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160751,"ondisk_log_size":160751,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1785620,"kb_used_data":657692,"kb_used_omap":12,"kb_used_meta":1127859,"kb_avail":281329900,"statfs":{"total":289910292480,"available":288081817600,"internally_reserved":0,"allocated":673476608,"data_stored":672539183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1154927618},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,4,10,8,6,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-20258816,"num_objects":-4946,"num_object_clones":0,"num_object_copies":-9892,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-246,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4946,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4946,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":246,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.007376"},"pg_stats":[{"pgid":"2.f","version":"47'17180","reported_seq":17258,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:39.979493+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:06:39.979493+0000","last_peered":"2024-09-10T16:06:39.979493+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:06:39.979493+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"43'7100","ondisk_log_start":"43'7100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14303232,"num_objects":3492,"num_object_clones":0,"num_object_copies":6984,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3492,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17180,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-767}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'17325","reported_seq":17443,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:06:40.615305+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:06:40.615305+0000","last_peered":"2024-09-10T16:06:40.615305+0000","last_clean":"2024-09-10T16:06:40.615305+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:06:40.615305+0000","last_undegraded":"2024-09-10T16:06:40.615305+0000","last_fullsized":"2024-09-10T16:06:40.615305+0000","mapping_epoch":35,"log_start":"43'7300","ondisk_log_start":"43'7300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14414790,"num_objects":3521,"num_object_clones":0,"num_object_copies":7042,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3521,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":15430,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'17376","reported_seq":26059,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:06:43.514950+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:06:43.514950+0000","last_peered":"2024-09-10T16:06:43.514950+0000","last_clean":"2024-09-10T16:06:43.514950+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:06:43.514950+0000","last_undegraded":"2024-09-10T16:06:43.514950+0000","last_fullsized":"2024-09-10T16:06:43.514950+0000","mapping_epoch":44,"log_start":"43'7300","ondisk_log_start":"43'7300","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14012440,"num_objects":3422,"num_object_clones":0,"num_object_copies":6844,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3422,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15380,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'17246","reported_seq":17332,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:06:40.632376+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:06:40.632376+0000","last_peered":"2024-09-10T16:06:40.632376+0000","last_clean":"2024-09-10T16:06:40.632376+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:06:40.632376+0000","last_undegraded":"2024-09-10T16:06:40.632376+0000","last_fullsized":"2024-09-10T16:06:40.632376+0000","mapping_epoch":35,"log_start":"43'7200","ondisk_log_start":"43'7200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13983744,"num_objects":3414,"num_object_clones":0,"num_object_copies":6828,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3414,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17246,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'17377","reported_seq":17512,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:06:39.971250+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:06:39.971250+0000","last_peered":"2024-09-10T16:06:39.971250+0000","last_clean":"2024-09-10T16:06:39.971250+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:06:39.971250+0000","last_undegraded":"2024-09-10T16:06:39.971250+0000","last_fullsized":"2024-09-10T16:06:39.971250+0000","mapping_epoch":34,"log_start":"43'7300","ondisk_log_start":"43'7300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14393827,"num_objects":3515,"num_object_clones":0,"num_object_copies":7030,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3515,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14088,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'17505","reported_seq":17648,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:39.961881+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:06:39.961881+0000","last_peered":"2024-09-10T16:06:39.961881+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:06:39.961881+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"43'7500","ondisk_log_start":"43'7500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14565859,"num_objects":3557,"num_object_clones":0,"num_object_copies":7114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3557,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14117,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-814},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'17444","reported_seq":17587,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:39.853082+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:06:39.853082+0000","last_peered":"2024-09-10T16:06:39.853082+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:06:39.853082+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"43'7400","ondisk_log_start":"43'7400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14902214,"num_objects":3640,"num_object_clones":0,"num_object_copies":7280,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3640,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13943,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-597},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'17686","reported_seq":17821,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:06:39.944275+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:06:39.944275+0000","last_peered":"2024-09-10T16:06:39.944275+0000","last_clean":"2024-09-10T16:06:39.944275+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:06:39.944275+0000","last_undegraded":"2024-09-10T16:06:39.944275+0000","last_fullsized":"2024-09-10T16:06:39.944275+0000","mapping_epoch":34,"log_start":"43'7600","ondisk_log_start":"43'7600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14860288,"num_objects":3628,"num_object_clones":0,"num_object_copies":7256,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3628,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14533,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.876895+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:05:19.876895+0000","last_peered":"2024-09-10T16:05:19.876895+0000","last_clean":"2024-09-10T16:05:19.876895+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:05:19.876895+0000","last_undegraded":"2024-09-10T16:05:19.876895+0000","last_fullsized":"2024-09-10T16:05:19.876895+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'16877","reported_seq":17021,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:40.479329+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:06:40.479329+0000","last_peered":"2024-09-10T16:06:40.479329+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:06:40.479329+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"43'6800","ondisk_log_start":"43'6800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14664163,"num_objects":3581,"num_object_clones":0,"num_object_copies":7162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3581,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14696,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-542}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":100,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.875849+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:05:19.875849+0000","last_peered":"2024-09-10T16:05:19.875849+0000","last_clean":"2024-09-10T16:05:19.875849+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:05:19.875849+0000","last_undegraded":"2024-09-10T16:05:19.875849+0000","last_fullsized":"2024-09-10T16:05:19.875849+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'17309","reported_seq":17444,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:40.018398+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:06:40.018398+0000","last_peered":"2024-09-10T16:06:40.018398+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:06:40.018398+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"43'7300","ondisk_log_start":"43'7300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14749696,"num_objects":3601,"num_object_clones":0,"num_object_copies":7202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3601,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13813,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-654}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":110,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.872023+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:05:19.872023+0000","last_peered":"2024-09-10T16:05:19.872023+0000","last_clean":"2024-09-10T16:05:19.872023+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:05:19.872023+0000","last_undegraded":"2024-09-10T16:05:19.872023+0000","last_fullsized":"2024-09-10T16:05:19.872023+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'17325","reported_seq":17464,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:40.656451+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:06:40.656451+0000","last_peered":"2024-09-10T16:06:40.656451+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:06:40.656451+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"43'7300","ondisk_log_start":"43'7300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14156259,"num_objects":3457,"num_object_clones":0,"num_object_copies":6914,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3457,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15005,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-847}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877148+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:05:19.877148+0000","last_peered":"2024-09-10T16:05:19.877148+0000","last_clean":"2024-09-10T16:05:19.877148+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:05:19.877148+0000","last_undegraded":"2024-09-10T16:05:19.877148+0000","last_fullsized":"2024-09-10T16:05:19.877148+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'17612","reported_seq":21354,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:40.709283+0000","last_change":"2024-09-10T16:05:18.271152+0000","last_active":"2024-09-10T16:06:40.709283+0000","last_peered":"2024-09-10T16:06:40.709283+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:06:40.709283+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"43'7600","ondisk_log_start":"43'7600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14573568,"num_objects":3558,"num_object_clones":0,"num_object_copies":7116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2551,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3558,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15166,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1803,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1007}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'17312","reported_seq":17450,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:40.054188+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:06:40.054188+0000","last_peered":"2024-09-10T16:06:40.054188+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:06:40.054188+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"43'7300","ondisk_log_start":"43'7300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14442496,"num_objects":3526,"num_object_clones":0,"num_object_copies":7052,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3526,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13943,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-778}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877654+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:05:19.877654+0000","last_peered":"2024-09-10T16:05:19.877654+0000","last_clean":"2024-09-10T16:05:19.877654+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:05:19.877654+0000","last_undegraded":"2024-09-10T16:05:19.877654+0000","last_fullsized":"2024-09-10T16:05:19.877654+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'17135","reported_seq":25706,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:06:43.435161+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:06:43.435161+0000","last_peered":"2024-09-10T16:06:43.435161+0000","last_clean":"2024-09-10T16:06:43.435161+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:06:43.435161+0000","last_undegraded":"2024-09-10T16:06:43.435161+0000","last_fullsized":"2024-09-10T16:06:43.435161+0000","mapping_epoch":46,"log_start":"43'7100","ondisk_log_start":"43'7100","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14184448,"num_objects":3463,"num_object_clones":0,"num_object_copies":6926,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3463,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13525,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":100,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.876043+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:05:19.876043+0000","last_peered":"2024-09-10T16:05:19.876043+0000","last_clean":"2024-09-10T16:05:19.876043+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:05:19.876043+0000","last_undegraded":"2024-09-10T16:05:19.876043+0000","last_fullsized":"2024-09-10T16:05:19.876043+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'17539","reported_seq":17687,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:40.001187+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:06:40.001187+0000","last_peered":"2024-09-10T16:06:40.001187+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:06:40.001187+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"43'7500","ondisk_log_start":"43'7500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14372864,"num_objects":3509,"num_object_clones":0,"num_object_copies":7018,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3509,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13794,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-862}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877349+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:05:19.877349+0000","last_peered":"2024-09-10T16:05:19.877349+0000","last_clean":"2024-09-10T16:05:19.877349+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:05:19.877349+0000","last_undegraded":"2024-09-10T16:05:19.877349+0000","last_fullsized":"2024-09-10T16:05:19.877349+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877269+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:05:19.877269+0000","last_peered":"2024-09-10T16:05:19.877269+0000","last_clean":"2024-09-10T16:05:19.877269+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:05:19.877269+0000","last_undegraded":"2024-09-10T16:05:19.877269+0000","last_fullsized":"2024-09-10T16:05:19.877269+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'17600","reported_seq":17750,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:40.027284+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:06:40.027284+0000","last_peered":"2024-09-10T16:06:40.027284+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:06:40.027284+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"43'7500","ondisk_log_start":"43'7500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14738374,"num_objects":3600,"num_object_clones":0,"num_object_copies":7200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3600,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13763,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-723},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":231318262,"num_objects":56484,"num_object_clones":0,"num_object_copies":112968,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41108,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56484,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":235622,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10281,"num_bytes_recovered":33095680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":669638656,"data_stored":669496831,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160748,"ondisk_log_size":160748,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986280,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":597348,"kb_used_data":250508,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93774492,"statfs":{"total":96636764160,"available":96025079808,"internally_reserved":0,"allocated":256520192,"data_stored":256217508,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":355135146},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,2,0,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659226,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":523636,"kb_used_data":167076,"kb_used_omap":4,"kb_used_meta":356539,"kb_avail":93848204,"statfs":{"total":96636764160,"available":96100560896,"internally_reserved":0,"allocated":171085824,"data_stored":170753772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":365096622},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,4,4,2,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182508,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":664636,"kb_used_data":240108,"kb_used_omap":4,"kb_used_meta":424507,"kb_avail":93707204,"statfs":{"total":96636764160,"available":95956176896,"internally_reserved":0,"allocated":245870592,"data_stored":245567903,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":434695850},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,4,4,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":244199424,"data_stored":244163294,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":169590784,"data_stored":169525291,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":255848448,"data_stored":255808246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:06:44.321 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:06:44.894 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:06:44.894 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:06:44.918 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":324,"stamp":"2024-09-10T16:06:43.718899+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":231318281,"num_objects":56486,"num_object_clones":0,"num_object_copies":112972,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41108,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56486,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":235624,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10283,"num_bytes_recovered":33095699,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160751,"ondisk_log_size":160751,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1785620,"kb_used_data":657692,"kb_used_omap":12,"kb_used_meta":1127859,"kb_avail":281329900,"statfs":{"total":289910292480,"available":288081817600,"internally_reserved":0,"allocated":673476608,"data_stored":672539183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1154927618},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,4,10,8,6,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-20258816,"num_objects":-4946,"num_object_clones":0,"num_object_copies":-9892,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-246,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4946,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4946,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":246,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.007376"},"pg_stats":[{"pgid":"2.f","version":"47'17180","reported_seq":17258,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:39.979493+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:06:39.979493+0000","last_peered":"2024-09-10T16:06:39.979493+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:06:39.979493+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"43'7100","ondisk_log_start":"43'7100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14303232,"num_objects":3492,"num_object_clones":0,"num_object_copies":6984,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3492,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17180,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-767}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'17325","reported_seq":17443,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:06:40.615305+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:06:40.615305+0000","last_peered":"2024-09-10T16:06:40.615305+0000","last_clean":"2024-09-10T16:06:40.615305+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:06:40.615305+0000","last_undegraded":"2024-09-10T16:06:40.615305+0000","last_fullsized":"2024-09-10T16:06:40.615305+0000","mapping_epoch":35,"log_start":"43'7300","ondisk_log_start":"43'7300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14414790,"num_objects":3521,"num_object_clones":0,"num_object_copies":7042,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3521,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":15430,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'17376","reported_seq":26059,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:06:43.514950+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:06:43.514950+0000","last_peered":"2024-09-10T16:06:43.514950+0000","last_clean":"2024-09-10T16:06:43.514950+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:06:43.514950+0000","last_undegraded":"2024-09-10T16:06:43.514950+0000","last_fullsized":"2024-09-10T16:06:43.514950+0000","mapping_epoch":44,"log_start":"43'7300","ondisk_log_start":"43'7300","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14012440,"num_objects":3422,"num_object_clones":0,"num_object_copies":6844,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3422,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15380,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'17246","reported_seq":17332,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:06:40.632376+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:06:40.632376+0000","last_peered":"2024-09-10T16:06:40.632376+0000","last_clean":"2024-09-10T16:06:40.632376+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:06:40.632376+0000","last_undegraded":"2024-09-10T16:06:40.632376+0000","last_fullsized":"2024-09-10T16:06:40.632376+0000","mapping_epoch":35,"log_start":"43'7200","ondisk_log_start":"43'7200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13983744,"num_objects":3414,"num_object_clones":0,"num_object_copies":6828,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3414,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17246,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'17377","reported_seq":17512,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:06:39.971250+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:06:39.971250+0000","last_peered":"2024-09-10T16:06:39.971250+0000","last_clean":"2024-09-10T16:06:39.971250+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:06:39.971250+0000","last_undegraded":"2024-09-10T16:06:39.971250+0000","last_fullsized":"2024-09-10T16:06:39.971250+0000","mapping_epoch":34,"log_start":"43'7300","ondisk_log_start":"43'7300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14393827,"num_objects":3515,"num_object_clones":0,"num_object_copies":7030,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3515,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14088,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'17505","reported_seq":17648,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:39.961881+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:06:39.961881+0000","last_peered":"2024-09-10T16:06:39.961881+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:06:39.961881+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"43'7500","ondisk_log_start":"43'7500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14565859,"num_objects":3557,"num_object_clones":0,"num_object_copies":7114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3557,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14117,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-814},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'17444","reported_seq":17587,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:39.853082+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:06:39.853082+0000","last_peered":"2024-09-10T16:06:39.853082+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:06:39.853082+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"43'7400","ondisk_log_start":"43'7400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14902214,"num_objects":3640,"num_object_clones":0,"num_object_copies":7280,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3640,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13943,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-597},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'17686","reported_seq":17821,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:06:39.944275+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:06:39.944275+0000","last_peered":"2024-09-10T16:06:39.944275+0000","last_clean":"2024-09-10T16:06:39.944275+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:06:39.944275+0000","last_undegraded":"2024-09-10T16:06:39.944275+0000","last_fullsized":"2024-09-10T16:06:39.944275+0000","mapping_epoch":34,"log_start":"43'7600","ondisk_log_start":"43'7600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14860288,"num_objects":3628,"num_object_clones":0,"num_object_copies":7256,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3628,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14533,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.876895+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:05:19.876895+0000","last_peered":"2024-09-10T16:05:19.876895+0000","last_clean":"2024-09-10T16:05:19.876895+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:05:19.876895+0000","last_undegraded":"2024-09-10T16:05:19.876895+0000","last_fullsized":"2024-09-10T16:05:19.876895+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'16877","reported_seq":17021,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:40.479329+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:06:40.479329+0000","last_peered":"2024-09-10T16:06:40.479329+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:06:40.479329+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"43'6800","ondisk_log_start":"43'6800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14664163,"num_objects":3581,"num_object_clones":0,"num_object_copies":7162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3581,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14696,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-542}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":100,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.875849+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:05:19.875849+0000","last_peered":"2024-09-10T16:05:19.875849+0000","last_clean":"2024-09-10T16:05:19.875849+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:05:19.875849+0000","last_undegraded":"2024-09-10T16:05:19.875849+0000","last_fullsized":"2024-09-10T16:05:19.875849+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'17309","reported_seq":17444,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:40.018398+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:06:40.018398+0000","last_peered":"2024-09-10T16:06:40.018398+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:06:40.018398+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"43'7300","ondisk_log_start":"43'7300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14749696,"num_objects":3601,"num_object_clones":0,"num_object_copies":7202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3601,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13813,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-654}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":110,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.872023+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:05:19.872023+0000","last_peered":"2024-09-10T16:05:19.872023+0000","last_clean":"2024-09-10T16:05:19.872023+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:05:19.872023+0000","last_undegraded":"2024-09-10T16:05:19.872023+0000","last_fullsized":"2024-09-10T16:05:19.872023+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'17325","reported_seq":17464,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:40.656451+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:06:40.656451+0000","last_peered":"2024-09-10T16:06:40.656451+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:06:40.656451+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"43'7300","ondisk_log_start":"43'7300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14156259,"num_objects":3457,"num_object_clones":0,"num_object_copies":6914,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3457,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15005,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-847}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877148+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:05:19.877148+0000","last_peered":"2024-09-10T16:05:19.877148+0000","last_clean":"2024-09-10T16:05:19.877148+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:05:19.877148+0000","last_undegraded":"2024-09-10T16:05:19.877148+0000","last_fullsized":"2024-09-10T16:05:19.877148+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'17612","reported_seq":21354,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:40.709283+0000","last_change":"2024-09-10T16:05:18.271152+0000","last_active":"2024-09-10T16:06:40.709283+0000","last_peered":"2024-09-10T16:06:40.709283+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:06:40.709283+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"43'7600","ondisk_log_start":"43'7600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14573568,"num_objects":3558,"num_object_clones":0,"num_object_copies":7116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2551,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3558,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15166,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1803,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1007}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'17312","reported_seq":17450,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:40.054188+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:06:40.054188+0000","last_peered":"2024-09-10T16:06:40.054188+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:06:40.054188+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"43'7300","ondisk_log_start":"43'7300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14442496,"num_objects":3526,"num_object_clones":0,"num_object_copies":7052,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3526,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13943,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-778}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877654+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:05:19.877654+0000","last_peered":"2024-09-10T16:05:19.877654+0000","last_clean":"2024-09-10T16:05:19.877654+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:05:19.877654+0000","last_undegraded":"2024-09-10T16:05:19.877654+0000","last_fullsized":"2024-09-10T16:05:19.877654+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'17135","reported_seq":25706,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:06:43.435161+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:06:43.435161+0000","last_peered":"2024-09-10T16:06:43.435161+0000","last_clean":"2024-09-10T16:06:43.435161+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:06:43.435161+0000","last_undegraded":"2024-09-10T16:06:43.435161+0000","last_fullsized":"2024-09-10T16:06:43.435161+0000","mapping_epoch":46,"log_start":"43'7100","ondisk_log_start":"43'7100","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14184448,"num_objects":3463,"num_object_clones":0,"num_object_copies":6926,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3463,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13525,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":100,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.876043+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:05:19.876043+0000","last_peered":"2024-09-10T16:05:19.876043+0000","last_clean":"2024-09-10T16:05:19.876043+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:05:19.876043+0000","last_undegraded":"2024-09-10T16:05:19.876043+0000","last_fullsized":"2024-09-10T16:05:19.876043+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'17539","reported_seq":17687,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:40.001187+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:06:40.001187+0000","last_peered":"2024-09-10T16:06:40.001187+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:06:40.001187+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"43'7500","ondisk_log_start":"43'7500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14372864,"num_objects":3509,"num_object_clones":0,"num_object_copies":7018,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3509,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13794,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-862}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877349+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:05:19.877349+0000","last_peered":"2024-09-10T16:05:19.877349+0000","last_clean":"2024-09-10T16:05:19.877349+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:05:19.877349+0000","last_undegraded":"2024-09-10T16:05:19.877349+0000","last_fullsized":"2024-09-10T16:05:19.877349+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877269+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:05:19.877269+0000","last_peered":"2024-09-10T16:05:19.877269+0000","last_clean":"2024-09-10T16:05:19.877269+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:05:19.877269+0000","last_undegraded":"2024-09-10T16:05:19.877269+0000","last_fullsized":"2024-09-10T16:05:19.877269+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'17600","reported_seq":17750,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:40.027284+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:06:40.027284+0000","last_peered":"2024-09-10T16:06:40.027284+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:06:40.027284+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"43'7500","ondisk_log_start":"43'7500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14738374,"num_objects":3600,"num_object_clones":0,"num_object_copies":7200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3600,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13763,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-723},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":231318262,"num_objects":56484,"num_object_clones":0,"num_object_copies":112968,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41108,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56484,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":235622,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10281,"num_bytes_recovered":33095680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":669638656,"data_stored":669496831,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160748,"ondisk_log_size":160748,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986280,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":597348,"kb_used_data":250508,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93774492,"statfs":{"total":96636764160,"available":96025079808,"internally_reserved":0,"allocated":256520192,"data_stored":256217508,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":355135146},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,2,0,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659226,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":523636,"kb_used_data":167076,"kb_used_omap":4,"kb_used_meta":356539,"kb_avail":93848204,"statfs":{"total":96636764160,"available":96100560896,"internally_reserved":0,"allocated":171085824,"data_stored":170753772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":365096622},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,4,4,2,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182508,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":664636,"kb_used_data":240108,"kb_used_omap":4,"kb_used_meta":424507,"kb_avail":93707204,"statfs":{"total":96636764160,"available":95956176896,"internally_reserved":0,"allocated":245870592,"data_stored":245567903,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":434695850},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,4,4,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":244199424,"data_stored":244163294,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":169590784,"data_stored":169525291,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":255848448,"data_stored":255808246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:06:44.919 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-10T16:06:44.919 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "47'17180", 'reported_seq': 17258, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:06:39.979493+0000', 'last_change': '2024-09-10T15:59:16.281558+0000', 'last_active': '2024-09-10T16:06:39.979493+0000', 'last_peered': '2024-09-10T16:06:39.979493+0000', 'last_clean': '2024-09-10T15:59:14.132456+0000', 'last_became_active': '2024-09-10T15:59:16.274407+0000', 'last_became_peered': '2024-09-10T15:59:16.274407+0000', 'last_unstale': '2024-09-10T16:06:39.979493+0000', 'last_undegraded': '2024-09-10T15:59:15.851279+0000', 'last_fullsized': '2024-09-10T15:59:15.774356+0000', 'mapping_epoch': 42, 'log_start': "43'7100", 'ondisk_log_start': "43'7100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'objects_scrubbed': 0, 'log_size': 10080, 'log_dups_size': 0, 'ondisk_log_size': 10080, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 14303232, 'num_objects': 3492, 'num_object_clones': 0, 'num_object_copies': 6984, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4259, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3492, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17180, 'num_write_kb': 41344, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -767}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:06:44.919 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-10T16:06:44.919 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "47'17505", 'reported_seq': 17648, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:06:39.961881+0000', 'last_change': '2024-09-10T15:59:16.308264+0000', 'last_active': '2024-09-10T16:06:39.961881+0000', 'last_peered': '2024-09-10T16:06:39.961881+0000', 'last_clean': '2024-09-10T15:59:14.111551+0000', 'last_became_active': '2024-09-10T15:59:16.300040+0000', 'last_became_peered': '2024-09-10T15:59:16.300040+0000', 'last_unstale': '2024-09-10T16:06:39.961881+0000', 'last_undegraded': '2024-09-10T15:59:15.878234+0000', 'last_fullsized': '2024-09-10T15:59:15.769328+0000', 'mapping_epoch': 42, 'log_start': "43'7500", 'ondisk_log_start': "43'7500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3442", 'last_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'objects_scrubbed': 3384, 'log_size': 10005, 'log_dups_size': 0, 'ondisk_log_size': 10005, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1292, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 14565859, 'num_objects': 3557, 'num_object_clones': 0, 'num_object_copies': 7114, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4372, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3557, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 14117, 'num_write_kb': 28580, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -814}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:06:44.919 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-10T16:06:44.919 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "47'17444", 'reported_seq': 17587, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:06:39.853082+0000', 'last_change': '2024-09-10T15:59:16.331899+0000', 'last_active': '2024-09-10T16:06:39.853082+0000', 'last_peered': '2024-09-10T16:06:39.853082+0000', 'last_clean': '2024-09-10T15:59:14.139436+0000', 'last_became_active': '2024-09-10T15:59:16.318715+0000', 'last_became_peered': '2024-09-10T15:59:16.318715+0000', 'last_unstale': '2024-09-10T16:06:39.853082+0000', 'last_undegraded': '2024-09-10T15:59:15.877524+0000', 'last_fullsized': '2024-09-10T15:59:15.769042+0000', 'mapping_epoch': 42, 'log_start': "43'7400", 'ondisk_log_start': "43'7400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3537", 'last_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'objects_scrubbed': 3497, 'log_size': 10044, 'log_dups_size': 0, 'ondisk_log_size': 10044, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1224, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 14902214, 'num_objects': 3640, 'num_object_clones': 0, 'num_object_copies': 7280, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4237, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3640, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 13943, 'num_write_kb': 28172, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -597}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:06:44.920 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-10T16:06:44.920 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "47'16877", 'reported_seq': 17021, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:06:40.479329+0000', 'last_change': '2024-09-10T15:59:16.175680+0000', 'last_active': '2024-09-10T16:06:40.479329+0000', 'last_peered': '2024-09-10T16:06:40.479329+0000', 'last_clean': '2024-09-10T15:59:14.088187+0000', 'last_became_active': '2024-09-10T15:59:16.165810+0000', 'last_became_peered': '2024-09-10T15:59:16.165810+0000', 'last_unstale': '2024-09-10T16:06:40.479329+0000', 'last_undegraded': '2024-09-10T15:59:15.822098+0000', 'last_fullsized': '2024-09-10T15:59:15.769123+0000', 'mapping_epoch': 42, 'log_start': "43'6800", 'ondisk_log_start': "43'6800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2218", 'last_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'objects_scrubbed': 2175, 'log_size': 10077, 'log_dups_size': 0, 'ondisk_log_size': 10077, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 825, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 14664163, 'num_objects': 3581, 'num_object_clones': 0, 'num_object_copies': 7162, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4124, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3581, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 14696, 'num_write_kb': 32204, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': -542}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:06:44.920 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-10T16:06:44.920 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "47'17309", 'reported_seq': 17444, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:06:40.018398+0000', 'last_change': '2024-09-10T15:59:16.277742+0000', 'last_active': '2024-09-10T16:06:40.018398+0000', 'last_peered': '2024-09-10T16:06:40.018398+0000', 'last_clean': '2024-09-10T15:59:14.129081+0000', 'last_became_active': '2024-09-10T15:59:16.269835+0000', 'last_became_peered': '2024-09-10T15:59:16.269835+0000', 'last_unstale': '2024-09-10T16:06:40.018398+0000', 'last_undegraded': '2024-09-10T15:59:15.860857+0000', 'last_fullsized': '2024-09-10T15:59:15.771410+0000', 'mapping_epoch': 42, 'log_start': "43'7300", 'ondisk_log_start': "43'7300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3557", 'last_scrub_stamp': '2024-09-10T15:59:02.493678+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:02.493678+0000', 'objects_scrubbed': 3488, 'log_size': 10009, 'log_dups_size': 0, 'ondisk_log_size': 10009, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1472, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 14749696, 'num_objects': 3601, 'num_object_clones': 0, 'num_object_copies': 7202, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4257, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3601, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13813, 'num_write_kb': 27852, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -654}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:06:44.920 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-10T16:06:44.920 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "47'17325", 'reported_seq': 17464, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:06:40.656451+0000', 'last_change': '2024-09-10T15:59:16.284311+0000', 'last_active': '2024-09-10T16:06:40.656451+0000', 'last_peered': '2024-09-10T16:06:40.656451+0000', 'last_clean': '2024-09-10T15:59:14.116041+0000', 'last_became_active': '2024-09-10T15:59:16.274543+0000', 'last_became_peered': '2024-09-10T15:59:16.274543+0000', 'last_unstale': '2024-09-10T16:06:40.656451+0000', 'last_undegraded': '2024-09-10T15:59:15.812484+0000', 'last_fullsized': '2024-09-10T15:59:15.770885+0000', 'mapping_epoch': 42, 'log_start': "43'7300", 'ondisk_log_start': "43'7300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2356", 'last_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'objects_scrubbed': 2316, 'log_size': 10025, 'log_dups_size': 0, 'ondisk_log_size': 10025, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 783, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 14156259, 'num_objects': 3457, 'num_object_clones': 0, 'num_object_copies': 6914, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3457, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 15005, 'num_write_kb': 32292, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': -847}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:06:44.920 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-10T16:06:44.920 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "47'17612", 'reported_seq': 21354, 'reported_epoch': 47, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:06:40.709283+0000', 'last_change': '2024-09-10T16:05:18.271152+0000', 'last_active': '2024-09-10T16:06:40.709283+0000', 'last_peered': '2024-09-10T16:06:40.709283+0000', 'last_clean': '2024-09-10T15:59:14.110654+0000', 'last_became_active': '2024-09-10T15:59:16.164645+0000', 'last_became_peered': '2024-09-10T15:59:16.164645+0000', 'last_unstale': '2024-09-10T16:06:40.709283+0000', 'last_undegraded': '2024-09-10T15:59:15.803593+0000', 'last_fullsized': '2024-09-10T15:59:15.769083+0000', 'mapping_epoch': 42, 'log_start': "43'7600", 'ondisk_log_start': "43'7600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2486", 'last_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'objects_scrubbed': 2440, 'log_size': 10012, 'log_dups_size': 0, 'ondisk_log_size': 10012, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 792, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 14573568, 'num_objects': 3558, 'num_object_clones': 0, 'num_object_copies': 7116, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2551, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3558, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15166, 'num_write_kb': 32568, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1803, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 1007}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:06:44.920 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-10T16:06:44.921 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "47'17312", 'reported_seq': 17450, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:06:40.054188+0000', 'last_change': '2024-09-10T15:59:16.315641+0000', 'last_active': '2024-09-10T16:06:40.054188+0000', 'last_peered': '2024-09-10T16:06:40.054188+0000', 'last_clean': '2024-09-10T15:59:14.117001+0000', 'last_became_active': '2024-09-10T15:59:16.302245+0000', 'last_became_peered': '2024-09-10T15:59:16.302245+0000', 'last_unstale': '2024-09-10T16:06:40.054188+0000', 'last_undegraded': '2024-09-10T15:59:15.845012+0000', 'last_fullsized': '2024-09-10T15:59:15.770337+0000', 'mapping_epoch': 42, 'log_start': "43'7300", 'ondisk_log_start': "43'7300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3404", 'last_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'objects_scrubbed': 3367, 'log_size': 10012, 'log_dups_size': 0, 'ondisk_log_size': 10012, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 983, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 14442496, 'num_objects': 3526, 'num_object_clones': 0, 'num_object_copies': 7052, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3526, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13943, 'num_write_kb': 28204, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -778}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:06:44.921 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-10T16:06:44.921 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "47'17539", 'reported_seq': 17687, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:06:40.001187+0000', 'last_change': '2024-09-10T15:59:16.311291+0000', 'last_active': '2024-09-10T16:06:40.001187+0000', 'last_peered': '2024-09-10T16:06:40.001187+0000', 'last_clean': '2024-09-10T15:59:14.116363+0000', 'last_became_active': '2024-09-10T15:59:16.301244+0000', 'last_became_peered': '2024-09-10T15:59:16.301244+0000', 'last_unstale': '2024-09-10T16:06:40.001187+0000', 'last_undegraded': '2024-09-10T15:59:15.883434+0000', 'last_fullsized': '2024-09-10T15:59:15.772756+0000', 'mapping_epoch': 42, 'log_start': "43'7500", 'ondisk_log_start': "43'7500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3789", 'last_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'objects_scrubbed': 3741, 'log_size': 10039, 'log_dups_size': 0, 'ondisk_log_size': 10039, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1440, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 14372864, 'num_objects': 3509, 'num_object_clones': 0, 'num_object_copies': 7018, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4373, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3509, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13794, 'num_write_kb': 27124, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -862}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:06:44.921 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-10T16:06:44.921 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "47'17600", 'reported_seq': 17750, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:06:40.027284+0000', 'last_change': '2024-09-10T15:59:16.313409+0000', 'last_active': '2024-09-10T16:06:40.027284+0000', 'last_peered': '2024-09-10T16:06:40.027284+0000', 'last_clean': '2024-09-10T15:59:14.129780+0000', 'last_became_active': '2024-09-10T15:59:16.300932+0000', 'last_became_peered': '2024-09-10T15:59:16.300932+0000', 'last_unstale': '2024-09-10T16:06:40.027284+0000', 'last_undegraded': '2024-09-10T15:59:15.887823+0000', 'last_fullsized': '2024-09-10T15:59:15.768513+0000', 'mapping_epoch': 42, 'log_start': "43'7500", 'ondisk_log_start': "43'7500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3880", 'last_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'objects_scrubbed': 3827, 'log_size': 10100, 'log_dups_size': 0, 'ondisk_log_size': 10100, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1397, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 14738374, 'num_objects': 3600, 'num_object_clones': 0, 'num_object_copies': 7200, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4325, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3600, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13763, 'num_write_kb': 27072, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -723}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:06:44.922 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T16:06:45.648 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:06:45.648 INFO:teuthology.orchestra.run.smithi115.stdout:{"fsid":"7effee29-df05-4351-9741-ee203564ed81","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 41108/112972 objects degraded (36.388%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":569,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":47,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983954,"num_in_osds":3,"osd_in_since":1725983947,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":56486,"data_bytes":231318281,"bytes_used":1828474880,"bytes_avail":288081817600,"bytes_total":289910292480,"degraded_objects":41108,"degraded_total":112972,"degraded_ratio":0.36387777502389973,"recovering_objects_per_sec":20,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":246,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":411},"fsmap":{"epoch":1,"btime":"2024-09-10T15:57:14:967120+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T16:00:13.560776+0000","services":{}},"progress_events":{}} 2024-09-10T16:06:45.670 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-10T16:06:45.670 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:06:46.196 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:06:46.199 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:06:46.227 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":325,"stamp":"2024-09-10T16:06:45.719528+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":225776393,"num_objects":55133,"num_object_clones":0,"num_object_copies":110266,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41108,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55133,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":236977,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10283,"num_bytes_recovered":33095699,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160704,"ondisk_log_size":160704,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1786880,"kb_used_data":650628,"kb_used_omap":12,"kb_used_meta":1136179,"kb_avail":281328640,"statfs":{"total":289910292480,"available":288080527360,"internally_reserved":0,"allocated":666243072,"data_stored":665305647,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1163447298},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,6,4,7,8,9,7],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-24748032,"num_objects":-6042,"num_object_clones":0,"num_object_copies":-12084,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-246,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-6042,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6042,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":246,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.007256"},"pg_stats":[{"pgid":"2.f","version":"47'17334","reported_seq":17413,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:45.084970+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:06:45.084970+0000","last_peered":"2024-09-10T16:06:45.084970+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:06:45.084970+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"43'7300","ondisk_log_start":"43'7300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13672448,"num_objects":3338,"num_object_clones":0,"num_object_copies":6676,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3338,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17334,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-921}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'17325","reported_seq":17443,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:06:40.615305+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:06:40.615305+0000","last_peered":"2024-09-10T16:06:40.615305+0000","last_clean":"2024-09-10T16:06:40.615305+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:06:40.615305+0000","last_undegraded":"2024-09-10T16:06:40.615305+0000","last_fullsized":"2024-09-10T16:06:40.615305+0000","mapping_epoch":35,"log_start":"43'7300","ondisk_log_start":"43'7300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14414790,"num_objects":3521,"num_object_clones":0,"num_object_copies":7042,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3521,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":15430,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'17376","reported_seq":26059,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:06:43.514950+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:06:43.514950+0000","last_peered":"2024-09-10T16:06:43.514950+0000","last_clean":"2024-09-10T16:06:43.514950+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:06:43.514950+0000","last_undegraded":"2024-09-10T16:06:43.514950+0000","last_fullsized":"2024-09-10T16:06:43.514950+0000","mapping_epoch":44,"log_start":"43'7300","ondisk_log_start":"43'7300","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14012440,"num_objects":3422,"num_object_clones":0,"num_object_copies":6844,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3422,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15380,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'17246","reported_seq":17332,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:06:40.632376+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:06:40.632376+0000","last_peered":"2024-09-10T16:06:40.632376+0000","last_clean":"2024-09-10T16:06:40.632376+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:06:40.632376+0000","last_undegraded":"2024-09-10T16:06:40.632376+0000","last_fullsized":"2024-09-10T16:06:40.632376+0000","mapping_epoch":35,"log_start":"43'7200","ondisk_log_start":"43'7200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13983744,"num_objects":3414,"num_object_clones":0,"num_object_copies":6828,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3414,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17246,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'17518","reported_seq":17653,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:06:44.972753+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:06:44.972753+0000","last_peered":"2024-09-10T16:06:44.972753+0000","last_clean":"2024-09-10T16:06:44.972753+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:06:44.972753+0000","last_undegraded":"2024-09-10T16:06:44.972753+0000","last_fullsized":"2024-09-10T16:06:44.972753+0000","mapping_epoch":34,"log_start":"43'7500","ondisk_log_start":"43'7500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13816291,"num_objects":3374,"num_object_clones":0,"num_object_copies":6748,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3374,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14229,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'17659","reported_seq":17802,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:45.012167+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:06:45.012167+0000","last_peered":"2024-09-10T16:06:45.012167+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:06:45.012167+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"43'7600","ondisk_log_start":"43'7600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13935075,"num_objects":3403,"num_object_clones":0,"num_object_copies":6806,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3403,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14271,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-968},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'17607","reported_seq":17746,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:45.043021+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:06:45.043021+0000","last_peered":"2024-09-10T16:06:45.043021+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:06:45.043021+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"43'7600","ondisk_log_start":"43'7600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14234566,"num_objects":3477,"num_object_clones":0,"num_object_copies":6954,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3477,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":14106,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-760},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'17828","reported_seq":17963,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:06:45.081205+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:06:45.081205+0000","last_peered":"2024-09-10T16:06:45.081205+0000","last_clean":"2024-09-10T16:06:45.081205+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:06:45.081205+0000","last_undegraded":"2024-09-10T16:06:45.081205+0000","last_fullsized":"2024-09-10T16:06:45.081205+0000","mapping_epoch":34,"log_start":"43'7800","ondisk_log_start":"43'7800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14278656,"num_objects":3486,"num_object_clones":0,"num_object_copies":6972,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3486,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14675,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.876895+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:05:19.876895+0000","last_peered":"2024-09-10T16:05:19.876895+0000","last_clean":"2024-09-10T16:05:19.876895+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:05:19.876895+0000","last_undegraded":"2024-09-10T16:05:19.876895+0000","last_fullsized":"2024-09-10T16:05:19.876895+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'16877","reported_seq":17021,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:40.479329+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:06:40.479329+0000","last_peered":"2024-09-10T16:06:40.479329+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:06:40.479329+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"43'6800","ondisk_log_start":"43'6800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14664163,"num_objects":3581,"num_object_clones":0,"num_object_copies":7162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3581,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14696,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-542}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":100,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.875849+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:05:19.875849+0000","last_peered":"2024-09-10T16:05:19.875849+0000","last_clean":"2024-09-10T16:05:19.875849+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:05:19.875849+0000","last_undegraded":"2024-09-10T16:05:19.875849+0000","last_fullsized":"2024-09-10T16:05:19.875849+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'17476","reported_seq":17612,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:45.008547+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:06:45.008547+0000","last_peered":"2024-09-10T16:06:45.008547+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:06:45.008547+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"43'7400","ondisk_log_start":"43'7400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14065664,"num_objects":3434,"num_object_clones":0,"num_object_copies":6868,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3434,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13980,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-821}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":110,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.872023+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:05:19.872023+0000","last_peered":"2024-09-10T16:05:19.872023+0000","last_clean":"2024-09-10T16:05:19.872023+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:05:19.872023+0000","last_undegraded":"2024-09-10T16:05:19.872023+0000","last_fullsized":"2024-09-10T16:05:19.872023+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'17325","reported_seq":17464,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:40.656451+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:06:40.656451+0000","last_peered":"2024-09-10T16:06:40.656451+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:06:40.656451+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"43'7300","ondisk_log_start":"43'7300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14156259,"num_objects":3457,"num_object_clones":0,"num_object_copies":6914,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3457,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15005,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-847}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877148+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:05:19.877148+0000","last_peered":"2024-09-10T16:05:19.877148+0000","last_clean":"2024-09-10T16:05:19.877148+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:05:19.877148+0000","last_undegraded":"2024-09-10T16:05:19.877148+0000","last_fullsized":"2024-09-10T16:05:19.877148+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'17612","reported_seq":21354,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:40.709283+0000","last_change":"2024-09-10T16:05:18.271152+0000","last_active":"2024-09-10T16:06:40.709283+0000","last_peered":"2024-09-10T16:06:40.709283+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:06:40.709283+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"43'7600","ondisk_log_start":"43'7600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14573568,"num_objects":3558,"num_object_clones":0,"num_object_copies":7116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2551,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3558,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15166,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1803,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1007}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'17449","reported_seq":17589,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:44.972116+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:06:44.972116+0000","last_peered":"2024-09-10T16:06:44.972116+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:06:44.972116+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"43'7400","ondisk_log_start":"43'7400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13881344,"num_objects":3389,"num_object_clones":0,"num_object_copies":6778,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3389,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14080,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-915}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877654+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:05:19.877654+0000","last_peered":"2024-09-10T16:05:19.877654+0000","last_clean":"2024-09-10T16:05:19.877654+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:05:19.877654+0000","last_undegraded":"2024-09-10T16:05:19.877654+0000","last_fullsized":"2024-09-10T16:05:19.877654+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'17135","reported_seq":25706,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:06:43.435161+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:06:43.435161+0000","last_peered":"2024-09-10T16:06:43.435161+0000","last_clean":"2024-09-10T16:06:43.435161+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:06:43.435161+0000","last_undegraded":"2024-09-10T16:06:43.435161+0000","last_fullsized":"2024-09-10T16:06:43.435161+0000","mapping_epoch":46,"log_start":"43'7100","ondisk_log_start":"43'7100","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14184448,"num_objects":3463,"num_object_clones":0,"num_object_copies":6926,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3463,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13525,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":100,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.876043+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:05:19.876043+0000","last_peered":"2024-09-10T16:05:19.876043+0000","last_clean":"2024-09-10T16:05:19.876043+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:05:19.876043+0000","last_undegraded":"2024-09-10T16:05:19.876043+0000","last_fullsized":"2024-09-10T16:05:19.876043+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'17691","reported_seq":17839,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:45.009548+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:06:45.009548+0000","last_peered":"2024-09-10T16:06:45.009548+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:06:45.009548+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"43'7600","ondisk_log_start":"43'7600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13750272,"num_objects":3357,"num_object_clones":0,"num_object_copies":6714,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3357,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13946,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1014}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877349+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:05:19.877349+0000","last_peered":"2024-09-10T16:05:19.877349+0000","last_clean":"2024-09-10T16:05:19.877349+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:05:19.877349+0000","last_undegraded":"2024-09-10T16:05:19.877349+0000","last_fullsized":"2024-09-10T16:05:19.877349+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877269+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:05:19.877269+0000","last_peered":"2024-09-10T16:05:19.877269+0000","last_clean":"2024-09-10T16:05:19.877269+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:05:19.877269+0000","last_undegraded":"2024-09-10T16:05:19.877269+0000","last_fullsized":"2024-09-10T16:05:19.877269+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'17743","reported_seq":17891,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:45.029678+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:06:45.029678+0000","last_peered":"2024-09-10T16:06:45.029678+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:06:45.029678+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"43'7700","ondisk_log_start":"43'7700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14152646,"num_objects":3457,"num_object_clones":0,"num_object_copies":6914,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3457,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13906,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-866},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":225776374,"num_objects":55131,"num_object_clones":0,"num_object_copies":110262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41108,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55131,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":236975,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10281,"num_bytes_recovered":33095680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":662478848,"data_stored":662337023,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160701,"ondisk_log_size":160701,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986280,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":597348,"kb_used_data":250508,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93774492,"statfs":{"total":96636764160,"available":96025079808,"internally_reserved":0,"allocated":256520192,"data_stored":256217508,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":355135146},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,2,0,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659226,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":523636,"kb_used_data":167076,"kb_used_omap":4,"kb_used_meta":356539,"kb_avail":93848204,"statfs":{"total":96636764160,"available":96100560896,"internally_reserved":0,"allocated":171085824,"data_stored":170753772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":365096622},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,4,4,2,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182509,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":665896,"kb_used_data":233044,"kb_used_omap":4,"kb_used_meta":432827,"kb_avail":93705944,"statfs":{"total":96636764160,"available":95954886656,"internally_reserved":0,"allocated":238637056,"data_stored":238334367,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":443215530},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,6,1,1,4,5,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":237039616,"data_stored":237003486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":169590784,"data_stored":169525291,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":255848448,"data_stored":255808246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:06:49.229 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:06:49.771 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:06:49.773 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:06:49.792 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":327,"stamp":"2024-09-10T16:06:49.720747+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":221721353,"num_objects":54143,"num_object_clones":0,"num_object_copies":108286,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40991,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54143,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":237967,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10400,"num_bytes_recovered":33095699,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160794,"ondisk_log_size":160794,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1783884,"kb_used_data":639280,"kb_used_omap":12,"kb_used_meta":1144499,"kb_avail":281331636,"statfs":{"total":289910292480,"available":288083595264,"internally_reserved":0,"allocated":654622720,"data_stored":653685295,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1171966978},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,10,1,6,6,8,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-20393984,"num_objects":-4979,"num_object_clones":0,"num_object_copies":-9958,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-245,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4979,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4979,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":245,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006998"},"pg_stats":[{"pgid":"2.f","version":"47'17334","reported_seq":17413,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:45.084970+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:06:45.084970+0000","last_peered":"2024-09-10T16:06:45.084970+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:06:45.084970+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"43'7300","ondisk_log_start":"43'7300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13672448,"num_objects":3338,"num_object_clones":0,"num_object_copies":6676,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3338,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17334,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-921}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'17461","reported_seq":17579,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:06:45.744485+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:06:45.744485+0000","last_peered":"2024-09-10T16:06:45.744485+0000","last_clean":"2024-09-10T16:06:45.744485+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:06:45.744485+0000","last_undegraded":"2024-09-10T16:06:45.744485+0000","last_fullsized":"2024-09-10T16:06:45.744485+0000","mapping_epoch":35,"log_start":"43'7400","ondisk_log_start":"43'7400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13857734,"num_objects":3385,"num_object_clones":0,"num_object_copies":6770,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3385,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":15566,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'17520","reported_seq":26203,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:06:48.642260+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:06:48.642260+0000","last_peered":"2024-09-10T16:06:48.642260+0000","last_clean":"2024-09-10T16:06:48.642260+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:06:48.642260+0000","last_undegraded":"2024-09-10T16:06:48.642260+0000","last_fullsized":"2024-09-10T16:06:48.642260+0000","mapping_epoch":44,"log_start":"43'7500","ondisk_log_start":"43'7500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13422616,"num_objects":3278,"num_object_clones":0,"num_object_copies":6556,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3278,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15524,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'17388","reported_seq":17474,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:06:45.676058+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:06:45.676058+0000","last_peered":"2024-09-10T16:06:45.676058+0000","last_clean":"2024-09-10T16:06:45.676058+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:06:45.676058+0000","last_undegraded":"2024-09-10T16:06:45.676058+0000","last_fullsized":"2024-09-10T16:06:45.676058+0000","mapping_epoch":35,"log_start":"43'7300","ondisk_log_start":"43'7300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13402112,"num_objects":3272,"num_object_clones":0,"num_object_copies":6544,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3272,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17388,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'17518","reported_seq":17653,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:06:44.972753+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:06:44.972753+0000","last_peered":"2024-09-10T16:06:44.972753+0000","last_clean":"2024-09-10T16:06:44.972753+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:06:44.972753+0000","last_undegraded":"2024-09-10T16:06:44.972753+0000","last_fullsized":"2024-09-10T16:06:44.972753+0000","mapping_epoch":34,"log_start":"43'7500","ondisk_log_start":"43'7500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13816291,"num_objects":3374,"num_object_clones":0,"num_object_copies":6748,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3374,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14229,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'17659","reported_seq":17802,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:45.012167+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:06:45.012167+0000","last_peered":"2024-09-10T16:06:45.012167+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:06:45.012167+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"43'7600","ondisk_log_start":"43'7600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13935075,"num_objects":3403,"num_object_clones":0,"num_object_copies":6806,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3403,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14271,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-968},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'17607","reported_seq":17746,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:45.043021+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:06:45.043021+0000","last_peered":"2024-09-10T16:06:45.043021+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:06:45.043021+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"43'7600","ondisk_log_start":"43'7600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14234566,"num_objects":3477,"num_object_clones":0,"num_object_copies":6954,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3477,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":14106,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-760},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'17828","reported_seq":17963,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:06:45.081205+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:06:45.081205+0000","last_peered":"2024-09-10T16:06:45.081205+0000","last_clean":"2024-09-10T16:06:45.081205+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:06:45.081205+0000","last_undegraded":"2024-09-10T16:06:45.081205+0000","last_fullsized":"2024-09-10T16:06:45.081205+0000","mapping_epoch":34,"log_start":"43'7800","ondisk_log_start":"43'7800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14278656,"num_objects":3486,"num_object_clones":0,"num_object_copies":6972,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3486,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14675,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.876895+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:05:19.876895+0000","last_peered":"2024-09-10T16:05:19.876895+0000","last_clean":"2024-09-10T16:05:19.876895+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:05:19.876895+0000","last_undegraded":"2024-09-10T16:05:19.876895+0000","last_fullsized":"2024-09-10T16:05:19.876895+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'17012","reported_seq":17157,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:45.701694+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:06:45.701694+0000","last_peered":"2024-09-10T16:06:45.701694+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:06:45.701694+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"43'7000","ondisk_log_start":"43'7000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14111203,"num_objects":3446,"num_object_clones":0,"num_object_copies":6892,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3446,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14831,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-677}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":100,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.875849+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:05:19.875849+0000","last_peered":"2024-09-10T16:05:19.875849+0000","last_clean":"2024-09-10T16:05:19.875849+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:05:19.875849+0000","last_undegraded":"2024-09-10T16:05:19.875849+0000","last_fullsized":"2024-09-10T16:05:19.875849+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'17476","reported_seq":17612,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:45.008547+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:06:45.008547+0000","last_peered":"2024-09-10T16:06:45.008547+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:06:45.008547+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"43'7400","ondisk_log_start":"43'7400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14065664,"num_objects":3434,"num_object_clones":0,"num_object_copies":6868,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3434,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13980,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-821}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":110,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.872023+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:05:19.872023+0000","last_peered":"2024-09-10T16:05:19.872023+0000","last_clean":"2024-09-10T16:05:19.872023+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:05:19.872023+0000","last_undegraded":"2024-09-10T16:05:19.872023+0000","last_fullsized":"2024-09-10T16:05:19.872023+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'17466","reported_seq":17607,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:45.685415+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:06:45.685415+0000","last_peered":"2024-09-10T16:06:45.685415+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:06:45.685415+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"43'7400","ondisk_log_start":"43'7400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13578723,"num_objects":3316,"num_object_clones":0,"num_object_copies":6632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3316,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15146,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-988}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877148+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:05:19.877148+0000","last_peered":"2024-09-10T16:05:19.877148+0000","last_clean":"2024-09-10T16:05:19.877148+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:05:19.877148+0000","last_undegraded":"2024-09-10T16:05:19.877148+0000","last_fullsized":"2024-09-10T16:05:19.877148+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'17768","reported_seq":21744,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:45.732440+0000","last_change":"2024-09-10T16:05:18.271152+0000","last_active":"2024-09-10T16:06:45.732440+0000","last_peered":"2024-09-10T16:06:45.732440+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:06:45.732440+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"43'7700","ondisk_log_start":"43'7700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13934592,"num_objects":3402,"num_object_clones":0,"num_object_copies":6804,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2434,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3402,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15322,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1920,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":968}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'17449","reported_seq":17589,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:44.972116+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:06:44.972116+0000","last_peered":"2024-09-10T16:06:44.972116+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:06:44.972116+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"43'7400","ondisk_log_start":"43'7400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13881344,"num_objects":3389,"num_object_clones":0,"num_object_copies":6778,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3389,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14080,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-915}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877654+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:05:19.877654+0000","last_peered":"2024-09-10T16:05:19.877654+0000","last_clean":"2024-09-10T16:05:19.877654+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:05:19.877654+0000","last_undegraded":"2024-09-10T16:05:19.877654+0000","last_fullsized":"2024-09-10T16:05:19.877654+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'17271","reported_seq":25842,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:06:48.550453+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:06:48.550453+0000","last_peered":"2024-09-10T16:06:48.550453+0000","last_clean":"2024-09-10T16:06:48.550453+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:06:48.550453+0000","last_undegraded":"2024-09-10T16:06:48.550453+0000","last_fullsized":"2024-09-10T16:06:48.550453+0000","mapping_epoch":46,"log_start":"43'7200","ondisk_log_start":"43'7200","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13627392,"num_objects":3327,"num_object_clones":0,"num_object_copies":6654,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3327,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13661,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":100,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.876043+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:05:19.876043+0000","last_peered":"2024-09-10T16:05:19.876043+0000","last_clean":"2024-09-10T16:05:19.876043+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:05:19.876043+0000","last_undegraded":"2024-09-10T16:05:19.876043+0000","last_fullsized":"2024-09-10T16:05:19.876043+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'17691","reported_seq":17839,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:45.009548+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:06:45.009548+0000","last_peered":"2024-09-10T16:06:45.009548+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:06:45.009548+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"43'7600","ondisk_log_start":"43'7600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13750272,"num_objects":3357,"num_object_clones":0,"num_object_copies":6714,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3357,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13946,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1014}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877349+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:05:19.877349+0000","last_peered":"2024-09-10T16:05:19.877349+0000","last_clean":"2024-09-10T16:05:19.877349+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:05:19.877349+0000","last_undegraded":"2024-09-10T16:05:19.877349+0000","last_fullsized":"2024-09-10T16:05:19.877349+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877269+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:05:19.877269+0000","last_peered":"2024-09-10T16:05:19.877269+0000","last_clean":"2024-09-10T16:05:19.877269+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:05:19.877269+0000","last_undegraded":"2024-09-10T16:05:19.877269+0000","last_fullsized":"2024-09-10T16:05:19.877269+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'17743","reported_seq":17891,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:45.029678+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:06:45.029678+0000","last_peered":"2024-09-10T16:06:45.029678+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:06:45.029678+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"43'7700","ondisk_log_start":"43'7700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14152646,"num_objects":3457,"num_object_clones":0,"num_object_copies":6914,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3457,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13906,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-866},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":221721334,"num_objects":54141,"num_object_clones":0,"num_object_copies":108282,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40991,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54141,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":237965,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10398,"num_bytes_recovered":33095680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":650522624,"data_stored":650380799,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160791,"ondisk_log_size":160791,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986281,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":596984,"kb_used_data":245992,"kb_used_omap":4,"kb_used_meta":350971,"kb_avail":93774856,"statfs":{"total":96636764160,"available":96025452544,"internally_reserved":0,"allocated":251895808,"data_stored":251593124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":359394986},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,0,3],"upper_bound":32},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659227,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":521004,"kb_used_data":160244,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93850836,"statfs":{"total":96636764160,"available":96103256064,"internally_reserved":0,"allocated":164089856,"data_stored":163757804,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":369356462},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,0,2,2,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182509,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":665896,"kb_used_data":233044,"kb_used_omap":4,"kb_used_meta":432827,"kb_avail":93705944,"statfs":{"total":96636764160,"available":95954886656,"internally_reserved":0,"allocated":238637056,"data_stored":238334367,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":443215530},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,6,1,1,4,5,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":237039616,"data_stored":237003486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":162320384,"data_stored":162254891,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":251162624,"data_stored":251122422,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:06:49.793 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:06:50.359 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:06:50.359 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:06:50.389 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":327,"stamp":"2024-09-10T16:06:49.720747+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":221721353,"num_objects":54143,"num_object_clones":0,"num_object_copies":108286,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40991,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54143,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":237967,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10400,"num_bytes_recovered":33095699,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160794,"ondisk_log_size":160794,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1783884,"kb_used_data":639280,"kb_used_omap":12,"kb_used_meta":1144499,"kb_avail":281331636,"statfs":{"total":289910292480,"available":288083595264,"internally_reserved":0,"allocated":654622720,"data_stored":653685295,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1171966978},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,10,1,6,6,8,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-20393984,"num_objects":-4979,"num_object_clones":0,"num_object_copies":-9958,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-245,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4979,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4979,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":245,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006998"},"pg_stats":[{"pgid":"2.f","version":"47'17334","reported_seq":17413,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:45.084970+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:06:45.084970+0000","last_peered":"2024-09-10T16:06:45.084970+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:06:45.084970+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"43'7300","ondisk_log_start":"43'7300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13672448,"num_objects":3338,"num_object_clones":0,"num_object_copies":6676,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3338,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17334,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-921}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'17461","reported_seq":17579,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:06:45.744485+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:06:45.744485+0000","last_peered":"2024-09-10T16:06:45.744485+0000","last_clean":"2024-09-10T16:06:45.744485+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:06:45.744485+0000","last_undegraded":"2024-09-10T16:06:45.744485+0000","last_fullsized":"2024-09-10T16:06:45.744485+0000","mapping_epoch":35,"log_start":"43'7400","ondisk_log_start":"43'7400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13857734,"num_objects":3385,"num_object_clones":0,"num_object_copies":6770,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3385,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":15566,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'17520","reported_seq":26203,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:06:48.642260+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:06:48.642260+0000","last_peered":"2024-09-10T16:06:48.642260+0000","last_clean":"2024-09-10T16:06:48.642260+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:06:48.642260+0000","last_undegraded":"2024-09-10T16:06:48.642260+0000","last_fullsized":"2024-09-10T16:06:48.642260+0000","mapping_epoch":44,"log_start":"43'7500","ondisk_log_start":"43'7500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13422616,"num_objects":3278,"num_object_clones":0,"num_object_copies":6556,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3278,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15524,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'17388","reported_seq":17474,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:06:45.676058+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:06:45.676058+0000","last_peered":"2024-09-10T16:06:45.676058+0000","last_clean":"2024-09-10T16:06:45.676058+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:06:45.676058+0000","last_undegraded":"2024-09-10T16:06:45.676058+0000","last_fullsized":"2024-09-10T16:06:45.676058+0000","mapping_epoch":35,"log_start":"43'7300","ondisk_log_start":"43'7300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13402112,"num_objects":3272,"num_object_clones":0,"num_object_copies":6544,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3272,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17388,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'17518","reported_seq":17653,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:06:44.972753+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:06:44.972753+0000","last_peered":"2024-09-10T16:06:44.972753+0000","last_clean":"2024-09-10T16:06:44.972753+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:06:44.972753+0000","last_undegraded":"2024-09-10T16:06:44.972753+0000","last_fullsized":"2024-09-10T16:06:44.972753+0000","mapping_epoch":34,"log_start":"43'7500","ondisk_log_start":"43'7500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13816291,"num_objects":3374,"num_object_clones":0,"num_object_copies":6748,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3374,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14229,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'17659","reported_seq":17802,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:45.012167+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:06:45.012167+0000","last_peered":"2024-09-10T16:06:45.012167+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:06:45.012167+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"43'7600","ondisk_log_start":"43'7600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13935075,"num_objects":3403,"num_object_clones":0,"num_object_copies":6806,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3403,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14271,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-968},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'17607","reported_seq":17746,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:45.043021+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:06:45.043021+0000","last_peered":"2024-09-10T16:06:45.043021+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:06:45.043021+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"43'7600","ondisk_log_start":"43'7600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14234566,"num_objects":3477,"num_object_clones":0,"num_object_copies":6954,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3477,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":14106,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-760},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'17828","reported_seq":17963,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:06:45.081205+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:06:45.081205+0000","last_peered":"2024-09-10T16:06:45.081205+0000","last_clean":"2024-09-10T16:06:45.081205+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:06:45.081205+0000","last_undegraded":"2024-09-10T16:06:45.081205+0000","last_fullsized":"2024-09-10T16:06:45.081205+0000","mapping_epoch":34,"log_start":"43'7800","ondisk_log_start":"43'7800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14278656,"num_objects":3486,"num_object_clones":0,"num_object_copies":6972,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3486,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14675,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.876895+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:05:19.876895+0000","last_peered":"2024-09-10T16:05:19.876895+0000","last_clean":"2024-09-10T16:05:19.876895+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:05:19.876895+0000","last_undegraded":"2024-09-10T16:05:19.876895+0000","last_fullsized":"2024-09-10T16:05:19.876895+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'17012","reported_seq":17157,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:45.701694+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:06:45.701694+0000","last_peered":"2024-09-10T16:06:45.701694+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:06:45.701694+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"43'7000","ondisk_log_start":"43'7000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14111203,"num_objects":3446,"num_object_clones":0,"num_object_copies":6892,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3446,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14831,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-677}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":100,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.875849+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:05:19.875849+0000","last_peered":"2024-09-10T16:05:19.875849+0000","last_clean":"2024-09-10T16:05:19.875849+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:05:19.875849+0000","last_undegraded":"2024-09-10T16:05:19.875849+0000","last_fullsized":"2024-09-10T16:05:19.875849+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'17476","reported_seq":17612,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:45.008547+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:06:45.008547+0000","last_peered":"2024-09-10T16:06:45.008547+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:06:45.008547+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"43'7400","ondisk_log_start":"43'7400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14065664,"num_objects":3434,"num_object_clones":0,"num_object_copies":6868,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3434,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13980,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-821}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":110,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.872023+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:05:19.872023+0000","last_peered":"2024-09-10T16:05:19.872023+0000","last_clean":"2024-09-10T16:05:19.872023+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:05:19.872023+0000","last_undegraded":"2024-09-10T16:05:19.872023+0000","last_fullsized":"2024-09-10T16:05:19.872023+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'17466","reported_seq":17607,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:45.685415+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:06:45.685415+0000","last_peered":"2024-09-10T16:06:45.685415+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:06:45.685415+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"43'7400","ondisk_log_start":"43'7400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13578723,"num_objects":3316,"num_object_clones":0,"num_object_copies":6632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3316,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15146,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-988}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877148+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:05:19.877148+0000","last_peered":"2024-09-10T16:05:19.877148+0000","last_clean":"2024-09-10T16:05:19.877148+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:05:19.877148+0000","last_undegraded":"2024-09-10T16:05:19.877148+0000","last_fullsized":"2024-09-10T16:05:19.877148+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'17768","reported_seq":21744,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:45.732440+0000","last_change":"2024-09-10T16:05:18.271152+0000","last_active":"2024-09-10T16:06:45.732440+0000","last_peered":"2024-09-10T16:06:45.732440+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:06:45.732440+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"43'7700","ondisk_log_start":"43'7700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13934592,"num_objects":3402,"num_object_clones":0,"num_object_copies":6804,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2434,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3402,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15322,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1920,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":968}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'17449","reported_seq":17589,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:44.972116+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:06:44.972116+0000","last_peered":"2024-09-10T16:06:44.972116+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:06:44.972116+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"43'7400","ondisk_log_start":"43'7400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13881344,"num_objects":3389,"num_object_clones":0,"num_object_copies":6778,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3389,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14080,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-915}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877654+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:05:19.877654+0000","last_peered":"2024-09-10T16:05:19.877654+0000","last_clean":"2024-09-10T16:05:19.877654+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:05:19.877654+0000","last_undegraded":"2024-09-10T16:05:19.877654+0000","last_fullsized":"2024-09-10T16:05:19.877654+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'17271","reported_seq":25842,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:06:48.550453+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:06:48.550453+0000","last_peered":"2024-09-10T16:06:48.550453+0000","last_clean":"2024-09-10T16:06:48.550453+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:06:48.550453+0000","last_undegraded":"2024-09-10T16:06:48.550453+0000","last_fullsized":"2024-09-10T16:06:48.550453+0000","mapping_epoch":46,"log_start":"43'7200","ondisk_log_start":"43'7200","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13627392,"num_objects":3327,"num_object_clones":0,"num_object_copies":6654,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3327,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13661,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":100,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.876043+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:05:19.876043+0000","last_peered":"2024-09-10T16:05:19.876043+0000","last_clean":"2024-09-10T16:05:19.876043+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:05:19.876043+0000","last_undegraded":"2024-09-10T16:05:19.876043+0000","last_fullsized":"2024-09-10T16:05:19.876043+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'17691","reported_seq":17839,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:45.009548+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:06:45.009548+0000","last_peered":"2024-09-10T16:06:45.009548+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:06:45.009548+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"43'7600","ondisk_log_start":"43'7600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13750272,"num_objects":3357,"num_object_clones":0,"num_object_copies":6714,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3357,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13946,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1014}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877349+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:05:19.877349+0000","last_peered":"2024-09-10T16:05:19.877349+0000","last_clean":"2024-09-10T16:05:19.877349+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:05:19.877349+0000","last_undegraded":"2024-09-10T16:05:19.877349+0000","last_fullsized":"2024-09-10T16:05:19.877349+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877269+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:05:19.877269+0000","last_peered":"2024-09-10T16:05:19.877269+0000","last_clean":"2024-09-10T16:05:19.877269+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:05:19.877269+0000","last_undegraded":"2024-09-10T16:05:19.877269+0000","last_fullsized":"2024-09-10T16:05:19.877269+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'17743","reported_seq":17891,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:45.029678+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:06:45.029678+0000","last_peered":"2024-09-10T16:06:45.029678+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:06:45.029678+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"43'7700","ondisk_log_start":"43'7700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14152646,"num_objects":3457,"num_object_clones":0,"num_object_copies":6914,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3457,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13906,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-866},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":221721334,"num_objects":54141,"num_object_clones":0,"num_object_copies":108282,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40991,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54141,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":237965,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10398,"num_bytes_recovered":33095680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":650522624,"data_stored":650380799,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160791,"ondisk_log_size":160791,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986281,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":596984,"kb_used_data":245992,"kb_used_omap":4,"kb_used_meta":350971,"kb_avail":93774856,"statfs":{"total":96636764160,"available":96025452544,"internally_reserved":0,"allocated":251895808,"data_stored":251593124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":359394986},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,0,3],"upper_bound":32},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659227,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":521004,"kb_used_data":160244,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93850836,"statfs":{"total":96636764160,"available":96103256064,"internally_reserved":0,"allocated":164089856,"data_stored":163757804,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":369356462},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,0,2,2,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182509,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":665896,"kb_used_data":233044,"kb_used_omap":4,"kb_used_meta":432827,"kb_avail":93705944,"statfs":{"total":96636764160,"available":95954886656,"internally_reserved":0,"allocated":238637056,"data_stored":238334367,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":443215530},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,6,1,1,4,5,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":237039616,"data_stored":237003486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":162320384,"data_stored":162254891,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":251162624,"data_stored":251122422,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:06:50.390 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-10T16:06:50.390 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "47'17334", 'reported_seq': 17413, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:06:45.084970+0000', 'last_change': '2024-09-10T15:59:16.281558+0000', 'last_active': '2024-09-10T16:06:45.084970+0000', 'last_peered': '2024-09-10T16:06:45.084970+0000', 'last_clean': '2024-09-10T15:59:14.132456+0000', 'last_became_active': '2024-09-10T15:59:16.274407+0000', 'last_became_peered': '2024-09-10T15:59:16.274407+0000', 'last_unstale': '2024-09-10T16:06:45.084970+0000', 'last_undegraded': '2024-09-10T15:59:15.851279+0000', 'last_fullsized': '2024-09-10T15:59:15.774356+0000', 'mapping_epoch': 42, 'log_start': "43'7300", 'ondisk_log_start': "43'7300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'objects_scrubbed': 0, 'log_size': 10034, 'log_dups_size': 0, 'ondisk_log_size': 10034, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 13672448, 'num_objects': 3338, 'num_object_clones': 0, 'num_object_copies': 6676, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4259, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3338, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17334, 'num_write_kb': 41344, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -921}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:06:50.391 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-10T16:06:50.391 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "47'17659", 'reported_seq': 17802, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:06:45.012167+0000', 'last_change': '2024-09-10T15:59:16.308264+0000', 'last_active': '2024-09-10T16:06:45.012167+0000', 'last_peered': '2024-09-10T16:06:45.012167+0000', 'last_clean': '2024-09-10T15:59:14.111551+0000', 'last_became_active': '2024-09-10T15:59:16.300040+0000', 'last_became_peered': '2024-09-10T15:59:16.300040+0000', 'last_unstale': '2024-09-10T16:06:45.012167+0000', 'last_undegraded': '2024-09-10T15:59:15.878234+0000', 'last_fullsized': '2024-09-10T15:59:15.769328+0000', 'mapping_epoch': 42, 'log_start': "43'7600", 'ondisk_log_start': "43'7600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3442", 'last_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'objects_scrubbed': 3384, 'log_size': 10059, 'log_dups_size': 0, 'ondisk_log_size': 10059, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1292, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 13935075, 'num_objects': 3403, 'num_object_clones': 0, 'num_object_copies': 6806, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4372, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3403, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 14271, 'num_write_kb': 28580, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -968}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:06:50.391 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-10T16:06:50.391 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "47'17607", 'reported_seq': 17746, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:06:45.043021+0000', 'last_change': '2024-09-10T15:59:16.331899+0000', 'last_active': '2024-09-10T16:06:45.043021+0000', 'last_peered': '2024-09-10T16:06:45.043021+0000', 'last_clean': '2024-09-10T15:59:14.139436+0000', 'last_became_active': '2024-09-10T15:59:16.318715+0000', 'last_became_peered': '2024-09-10T15:59:16.318715+0000', 'last_unstale': '2024-09-10T16:06:45.043021+0000', 'last_undegraded': '2024-09-10T15:59:15.877524+0000', 'last_fullsized': '2024-09-10T15:59:15.769042+0000', 'mapping_epoch': 42, 'log_start': "43'7600", 'ondisk_log_start': "43'7600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3537", 'last_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'objects_scrubbed': 3497, 'log_size': 10007, 'log_dups_size': 0, 'ondisk_log_size': 10007, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1224, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 14234566, 'num_objects': 3477, 'num_object_clones': 0, 'num_object_copies': 6954, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4237, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3477, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 14106, 'num_write_kb': 28172, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -760}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:06:50.391 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-10T16:06:50.391 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "47'17012", 'reported_seq': 17157, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:06:45.701694+0000', 'last_change': '2024-09-10T15:59:16.175680+0000', 'last_active': '2024-09-10T16:06:45.701694+0000', 'last_peered': '2024-09-10T16:06:45.701694+0000', 'last_clean': '2024-09-10T15:59:14.088187+0000', 'last_became_active': '2024-09-10T15:59:16.165810+0000', 'last_became_peered': '2024-09-10T15:59:16.165810+0000', 'last_unstale': '2024-09-10T16:06:45.701694+0000', 'last_undegraded': '2024-09-10T15:59:15.822098+0000', 'last_fullsized': '2024-09-10T15:59:15.769123+0000', 'mapping_epoch': 42, 'log_start': "43'7000", 'ondisk_log_start': "43'7000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2218", 'last_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'objects_scrubbed': 2175, 'log_size': 10012, 'log_dups_size': 0, 'ondisk_log_size': 10012, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 825, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 14111203, 'num_objects': 3446, 'num_object_clones': 0, 'num_object_copies': 6892, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4124, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3446, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 14831, 'num_write_kb': 32204, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': -677}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:06:50.391 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-10T16:06:50.391 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "47'17476", 'reported_seq': 17612, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:06:45.008547+0000', 'last_change': '2024-09-10T15:59:16.277742+0000', 'last_active': '2024-09-10T16:06:45.008547+0000', 'last_peered': '2024-09-10T16:06:45.008547+0000', 'last_clean': '2024-09-10T15:59:14.129081+0000', 'last_became_active': '2024-09-10T15:59:16.269835+0000', 'last_became_peered': '2024-09-10T15:59:16.269835+0000', 'last_unstale': '2024-09-10T16:06:45.008547+0000', 'last_undegraded': '2024-09-10T15:59:15.860857+0000', 'last_fullsized': '2024-09-10T15:59:15.771410+0000', 'mapping_epoch': 42, 'log_start': "43'7400", 'ondisk_log_start': "43'7400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3557", 'last_scrub_stamp': '2024-09-10T15:59:02.493678+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:02.493678+0000', 'objects_scrubbed': 3488, 'log_size': 10076, 'log_dups_size': 0, 'ondisk_log_size': 10076, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1472, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 14065664, 'num_objects': 3434, 'num_object_clones': 0, 'num_object_copies': 6868, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4257, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3434, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13980, 'num_write_kb': 27852, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -821}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:06:50.391 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-10T16:06:50.392 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "47'17466", 'reported_seq': 17607, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:06:45.685415+0000', 'last_change': '2024-09-10T15:59:16.284311+0000', 'last_active': '2024-09-10T16:06:45.685415+0000', 'last_peered': '2024-09-10T16:06:45.685415+0000', 'last_clean': '2024-09-10T15:59:14.116041+0000', 'last_became_active': '2024-09-10T15:59:16.274543+0000', 'last_became_peered': '2024-09-10T15:59:16.274543+0000', 'last_unstale': '2024-09-10T16:06:45.685415+0000', 'last_undegraded': '2024-09-10T15:59:15.812484+0000', 'last_fullsized': '2024-09-10T15:59:15.770885+0000', 'mapping_epoch': 42, 'log_start': "43'7400", 'ondisk_log_start': "43'7400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2356", 'last_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'objects_scrubbed': 2316, 'log_size': 10066, 'log_dups_size': 0, 'ondisk_log_size': 10066, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 783, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 13578723, 'num_objects': 3316, 'num_object_clones': 0, 'num_object_copies': 6632, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3316, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 15146, 'num_write_kb': 32292, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': -988}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:06:50.392 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-10T16:06:50.392 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "47'17768", 'reported_seq': 21744, 'reported_epoch': 47, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:06:45.732440+0000', 'last_change': '2024-09-10T16:05:18.271152+0000', 'last_active': '2024-09-10T16:06:45.732440+0000', 'last_peered': '2024-09-10T16:06:45.732440+0000', 'last_clean': '2024-09-10T15:59:14.110654+0000', 'last_became_active': '2024-09-10T15:59:16.164645+0000', 'last_became_peered': '2024-09-10T15:59:16.164645+0000', 'last_unstale': '2024-09-10T16:06:45.732440+0000', 'last_undegraded': '2024-09-10T15:59:15.803593+0000', 'last_fullsized': '2024-09-10T15:59:15.769083+0000', 'mapping_epoch': 42, 'log_start': "43'7700", 'ondisk_log_start': "43'7700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2486", 'last_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'objects_scrubbed': 2440, 'log_size': 10068, 'log_dups_size': 0, 'ondisk_log_size': 10068, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 792, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 13934592, 'num_objects': 3402, 'num_object_clones': 0, 'num_object_copies': 6804, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2434, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3402, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15322, 'num_write_kb': 32568, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1920, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 968}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:06:50.392 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-10T16:06:50.392 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "47'17449", 'reported_seq': 17589, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:06:44.972116+0000', 'last_change': '2024-09-10T15:59:16.315641+0000', 'last_active': '2024-09-10T16:06:44.972116+0000', 'last_peered': '2024-09-10T16:06:44.972116+0000', 'last_clean': '2024-09-10T15:59:14.117001+0000', 'last_became_active': '2024-09-10T15:59:16.302245+0000', 'last_became_peered': '2024-09-10T15:59:16.302245+0000', 'last_unstale': '2024-09-10T16:06:44.972116+0000', 'last_undegraded': '2024-09-10T15:59:15.845012+0000', 'last_fullsized': '2024-09-10T15:59:15.770337+0000', 'mapping_epoch': 42, 'log_start': "43'7400", 'ondisk_log_start': "43'7400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3404", 'last_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'objects_scrubbed': 3367, 'log_size': 10049, 'log_dups_size': 0, 'ondisk_log_size': 10049, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 983, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 13881344, 'num_objects': 3389, 'num_object_clones': 0, 'num_object_copies': 6778, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3389, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14080, 'num_write_kb': 28204, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -915}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:06:50.392 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-10T16:06:50.392 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "47'17691", 'reported_seq': 17839, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:06:45.009548+0000', 'last_change': '2024-09-10T15:59:16.311291+0000', 'last_active': '2024-09-10T16:06:45.009548+0000', 'last_peered': '2024-09-10T16:06:45.009548+0000', 'last_clean': '2024-09-10T15:59:14.116363+0000', 'last_became_active': '2024-09-10T15:59:16.301244+0000', 'last_became_peered': '2024-09-10T15:59:16.301244+0000', 'last_unstale': '2024-09-10T16:06:45.009548+0000', 'last_undegraded': '2024-09-10T15:59:15.883434+0000', 'last_fullsized': '2024-09-10T15:59:15.772756+0000', 'mapping_epoch': 42, 'log_start': "43'7600", 'ondisk_log_start': "43'7600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3789", 'last_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'objects_scrubbed': 3741, 'log_size': 10091, 'log_dups_size': 0, 'ondisk_log_size': 10091, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1440, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 13750272, 'num_objects': 3357, 'num_object_clones': 0, 'num_object_copies': 6714, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4373, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3357, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13946, 'num_write_kb': 27124, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1014}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:06:50.392 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-10T16:06:50.392 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "47'17743", 'reported_seq': 17891, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:06:45.029678+0000', 'last_change': '2024-09-10T15:59:16.313409+0000', 'last_active': '2024-09-10T16:06:45.029678+0000', 'last_peered': '2024-09-10T16:06:45.029678+0000', 'last_clean': '2024-09-10T15:59:14.129780+0000', 'last_became_active': '2024-09-10T15:59:16.300932+0000', 'last_became_peered': '2024-09-10T15:59:16.300932+0000', 'last_unstale': '2024-09-10T16:06:45.029678+0000', 'last_undegraded': '2024-09-10T15:59:15.887823+0000', 'last_fullsized': '2024-09-10T15:59:15.768513+0000', 'mapping_epoch': 42, 'log_start': "43'7700", 'ondisk_log_start': "43'7700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3880", 'last_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'objects_scrubbed': 3827, 'log_size': 10043, 'log_dups_size': 0, 'ondisk_log_size': 10043, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1397, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 14152646, 'num_objects': 3457, 'num_object_clones': 0, 'num_object_copies': 6914, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4325, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3457, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13906, 'num_write_kb': 27072, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -866}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:06:50.393 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T16:06:51.075 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:06:51.076 INFO:teuthology.orchestra.run.smithi115.stdout:{"fsid":"7effee29-df05-4351-9741-ee203564ed81","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 40991/108286 objects degraded (37.854%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":575,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":47,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983954,"num_in_osds":3,"osd_in_since":1725983947,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":54143,"data_bytes":221721353,"bytes_used":1826697216,"bytes_avail":288083595264,"bytes_total":289910292480,"degraded_objects":40991,"degraded_total":108286,"degraded_ratio":0.37854385608481245,"recovering_objects_per_sec":20,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":245,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":414},"fsmap":{"epoch":1,"btime":"2024-09-10T15:57:14:967120+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T16:00:13.560776+0000","services":{}},"progress_events":{}} 2024-09-10T16:06:51.097 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-10T16:06:51.097 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:06:51.651 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:06:51.652 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:06:51.673 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":327,"stamp":"2024-09-10T16:06:49.720747+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":221721353,"num_objects":54143,"num_object_clones":0,"num_object_copies":108286,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40991,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54143,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":237967,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10400,"num_bytes_recovered":33095699,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160794,"ondisk_log_size":160794,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1783884,"kb_used_data":639280,"kb_used_omap":12,"kb_used_meta":1144499,"kb_avail":281331636,"statfs":{"total":289910292480,"available":288083595264,"internally_reserved":0,"allocated":654622720,"data_stored":653685295,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1171966978},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,10,1,6,6,8,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-20393984,"num_objects":-4979,"num_object_clones":0,"num_object_copies":-9958,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-245,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4979,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4979,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":245,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006998"},"pg_stats":[{"pgid":"2.f","version":"47'17334","reported_seq":17413,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:45.084970+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:06:45.084970+0000","last_peered":"2024-09-10T16:06:45.084970+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:06:45.084970+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"43'7300","ondisk_log_start":"43'7300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13672448,"num_objects":3338,"num_object_clones":0,"num_object_copies":6676,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3338,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17334,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-921}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'17461","reported_seq":17579,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:06:45.744485+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:06:45.744485+0000","last_peered":"2024-09-10T16:06:45.744485+0000","last_clean":"2024-09-10T16:06:45.744485+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:06:45.744485+0000","last_undegraded":"2024-09-10T16:06:45.744485+0000","last_fullsized":"2024-09-10T16:06:45.744485+0000","mapping_epoch":35,"log_start":"43'7400","ondisk_log_start":"43'7400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13857734,"num_objects":3385,"num_object_clones":0,"num_object_copies":6770,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3385,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":15566,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'17520","reported_seq":26203,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:06:48.642260+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:06:48.642260+0000","last_peered":"2024-09-10T16:06:48.642260+0000","last_clean":"2024-09-10T16:06:48.642260+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:06:48.642260+0000","last_undegraded":"2024-09-10T16:06:48.642260+0000","last_fullsized":"2024-09-10T16:06:48.642260+0000","mapping_epoch":44,"log_start":"43'7500","ondisk_log_start":"43'7500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13422616,"num_objects":3278,"num_object_clones":0,"num_object_copies":6556,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3278,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15524,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'17388","reported_seq":17474,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:06:45.676058+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:06:45.676058+0000","last_peered":"2024-09-10T16:06:45.676058+0000","last_clean":"2024-09-10T16:06:45.676058+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:06:45.676058+0000","last_undegraded":"2024-09-10T16:06:45.676058+0000","last_fullsized":"2024-09-10T16:06:45.676058+0000","mapping_epoch":35,"log_start":"43'7300","ondisk_log_start":"43'7300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13402112,"num_objects":3272,"num_object_clones":0,"num_object_copies":6544,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3272,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17388,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'17518","reported_seq":17653,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:06:44.972753+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:06:44.972753+0000","last_peered":"2024-09-10T16:06:44.972753+0000","last_clean":"2024-09-10T16:06:44.972753+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:06:44.972753+0000","last_undegraded":"2024-09-10T16:06:44.972753+0000","last_fullsized":"2024-09-10T16:06:44.972753+0000","mapping_epoch":34,"log_start":"43'7500","ondisk_log_start":"43'7500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13816291,"num_objects":3374,"num_object_clones":0,"num_object_copies":6748,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3374,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14229,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'17659","reported_seq":17802,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:45.012167+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:06:45.012167+0000","last_peered":"2024-09-10T16:06:45.012167+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:06:45.012167+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"43'7600","ondisk_log_start":"43'7600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13935075,"num_objects":3403,"num_object_clones":0,"num_object_copies":6806,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3403,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14271,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-968},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'17607","reported_seq":17746,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:45.043021+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:06:45.043021+0000","last_peered":"2024-09-10T16:06:45.043021+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:06:45.043021+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"43'7600","ondisk_log_start":"43'7600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14234566,"num_objects":3477,"num_object_clones":0,"num_object_copies":6954,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3477,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":14106,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-760},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'17828","reported_seq":17963,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:06:45.081205+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:06:45.081205+0000","last_peered":"2024-09-10T16:06:45.081205+0000","last_clean":"2024-09-10T16:06:45.081205+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:06:45.081205+0000","last_undegraded":"2024-09-10T16:06:45.081205+0000","last_fullsized":"2024-09-10T16:06:45.081205+0000","mapping_epoch":34,"log_start":"43'7800","ondisk_log_start":"43'7800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14278656,"num_objects":3486,"num_object_clones":0,"num_object_copies":6972,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3486,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14675,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.876895+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:05:19.876895+0000","last_peered":"2024-09-10T16:05:19.876895+0000","last_clean":"2024-09-10T16:05:19.876895+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:05:19.876895+0000","last_undegraded":"2024-09-10T16:05:19.876895+0000","last_fullsized":"2024-09-10T16:05:19.876895+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'17012","reported_seq":17157,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:45.701694+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:06:45.701694+0000","last_peered":"2024-09-10T16:06:45.701694+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:06:45.701694+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"43'7000","ondisk_log_start":"43'7000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14111203,"num_objects":3446,"num_object_clones":0,"num_object_copies":6892,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3446,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14831,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-677}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":100,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.875849+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:05:19.875849+0000","last_peered":"2024-09-10T16:05:19.875849+0000","last_clean":"2024-09-10T16:05:19.875849+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:05:19.875849+0000","last_undegraded":"2024-09-10T16:05:19.875849+0000","last_fullsized":"2024-09-10T16:05:19.875849+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'17476","reported_seq":17612,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:45.008547+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:06:45.008547+0000","last_peered":"2024-09-10T16:06:45.008547+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:06:45.008547+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"43'7400","ondisk_log_start":"43'7400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14065664,"num_objects":3434,"num_object_clones":0,"num_object_copies":6868,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3434,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13980,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-821}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":110,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.872023+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:05:19.872023+0000","last_peered":"2024-09-10T16:05:19.872023+0000","last_clean":"2024-09-10T16:05:19.872023+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:05:19.872023+0000","last_undegraded":"2024-09-10T16:05:19.872023+0000","last_fullsized":"2024-09-10T16:05:19.872023+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'17466","reported_seq":17607,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:45.685415+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:06:45.685415+0000","last_peered":"2024-09-10T16:06:45.685415+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:06:45.685415+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"43'7400","ondisk_log_start":"43'7400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13578723,"num_objects":3316,"num_object_clones":0,"num_object_copies":6632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3316,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15146,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-988}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877148+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:05:19.877148+0000","last_peered":"2024-09-10T16:05:19.877148+0000","last_clean":"2024-09-10T16:05:19.877148+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:05:19.877148+0000","last_undegraded":"2024-09-10T16:05:19.877148+0000","last_fullsized":"2024-09-10T16:05:19.877148+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'17768","reported_seq":21744,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:45.732440+0000","last_change":"2024-09-10T16:05:18.271152+0000","last_active":"2024-09-10T16:06:45.732440+0000","last_peered":"2024-09-10T16:06:45.732440+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:06:45.732440+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"43'7700","ondisk_log_start":"43'7700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13934592,"num_objects":3402,"num_object_clones":0,"num_object_copies":6804,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2434,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3402,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15322,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1920,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":968}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'17449","reported_seq":17589,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:44.972116+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:06:44.972116+0000","last_peered":"2024-09-10T16:06:44.972116+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:06:44.972116+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"43'7400","ondisk_log_start":"43'7400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13881344,"num_objects":3389,"num_object_clones":0,"num_object_copies":6778,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3389,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14080,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-915}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877654+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:05:19.877654+0000","last_peered":"2024-09-10T16:05:19.877654+0000","last_clean":"2024-09-10T16:05:19.877654+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:05:19.877654+0000","last_undegraded":"2024-09-10T16:05:19.877654+0000","last_fullsized":"2024-09-10T16:05:19.877654+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'17271","reported_seq":25842,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:06:48.550453+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:06:48.550453+0000","last_peered":"2024-09-10T16:06:48.550453+0000","last_clean":"2024-09-10T16:06:48.550453+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:06:48.550453+0000","last_undegraded":"2024-09-10T16:06:48.550453+0000","last_fullsized":"2024-09-10T16:06:48.550453+0000","mapping_epoch":46,"log_start":"43'7200","ondisk_log_start":"43'7200","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13627392,"num_objects":3327,"num_object_clones":0,"num_object_copies":6654,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3327,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13661,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":100,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.876043+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:05:19.876043+0000","last_peered":"2024-09-10T16:05:19.876043+0000","last_clean":"2024-09-10T16:05:19.876043+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:05:19.876043+0000","last_undegraded":"2024-09-10T16:05:19.876043+0000","last_fullsized":"2024-09-10T16:05:19.876043+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'17691","reported_seq":17839,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:45.009548+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:06:45.009548+0000","last_peered":"2024-09-10T16:06:45.009548+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:06:45.009548+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"43'7600","ondisk_log_start":"43'7600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13750272,"num_objects":3357,"num_object_clones":0,"num_object_copies":6714,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3357,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13946,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1014}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877349+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:05:19.877349+0000","last_peered":"2024-09-10T16:05:19.877349+0000","last_clean":"2024-09-10T16:05:19.877349+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:05:19.877349+0000","last_undegraded":"2024-09-10T16:05:19.877349+0000","last_fullsized":"2024-09-10T16:05:19.877349+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877269+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:05:19.877269+0000","last_peered":"2024-09-10T16:05:19.877269+0000","last_clean":"2024-09-10T16:05:19.877269+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:05:19.877269+0000","last_undegraded":"2024-09-10T16:05:19.877269+0000","last_fullsized":"2024-09-10T16:05:19.877269+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'17743","reported_seq":17891,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:45.029678+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:06:45.029678+0000","last_peered":"2024-09-10T16:06:45.029678+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:06:45.029678+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"43'7700","ondisk_log_start":"43'7700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14152646,"num_objects":3457,"num_object_clones":0,"num_object_copies":6914,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3457,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13906,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-866},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":221721334,"num_objects":54141,"num_object_clones":0,"num_object_copies":108282,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40991,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54141,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":237965,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10398,"num_bytes_recovered":33095680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":650522624,"data_stored":650380799,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160791,"ondisk_log_size":160791,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986281,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":596984,"kb_used_data":245992,"kb_used_omap":4,"kb_used_meta":350971,"kb_avail":93774856,"statfs":{"total":96636764160,"available":96025452544,"internally_reserved":0,"allocated":251895808,"data_stored":251593124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":359394986},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,0,3],"upper_bound":32},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659227,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":521004,"kb_used_data":160244,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93850836,"statfs":{"total":96636764160,"available":96103256064,"internally_reserved":0,"allocated":164089856,"data_stored":163757804,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":369356462},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,0,2,2,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182509,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":665896,"kb_used_data":233044,"kb_used_omap":4,"kb_used_meta":432827,"kb_avail":93705944,"statfs":{"total":96636764160,"available":95954886656,"internally_reserved":0,"allocated":238637056,"data_stored":238334367,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":443215530},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,6,1,1,4,5,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":237039616,"data_stored":237003486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":162320384,"data_stored":162254891,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":251162624,"data_stored":251122422,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:06:54.676 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:06:55.254 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:06:55.255 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:06:55.278 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":329,"stamp":"2024-09-10T16:06:53.725735+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":212083465,"num_objects":51790,"num_object_clones":0,"num_object_copies":103580,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40880,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51790,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":240320,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10511,"num_bytes_recovered":33095699,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160647,"ondisk_log_size":160647,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1765036,"kb_used_data":619064,"kb_used_omap":12,"kb_used_meta":1145907,"kb_avail":281350484,"statfs":{"total":289910292480,"available":288102895616,"internally_reserved":0,"allocated":633921536,"data_stored":632984111,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1173408770},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,5,10,7,4,3,3],"upper_bound":512},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-20434944,"num_objects":-4989,"num_object_clones":0,"num_object_copies":-9978,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-228,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4989,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4989,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":228,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.007319"},"pg_stats":[{"pgid":"2.f","version":"47'17462","reported_seq":17541,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:50.145867+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:06:50.145867+0000","last_peered":"2024-09-10T16:06:50.145867+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:06:50.145867+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"43'7400","ondisk_log_start":"43'7400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13148160,"num_objects":3210,"num_object_clones":0,"num_object_copies":6420,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3210,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17462,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1049}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'17606","reported_seq":17724,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:06:50.759074+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:06:50.759074+0000","last_peered":"2024-09-10T16:06:50.759074+0000","last_clean":"2024-09-10T16:06:50.759074+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:06:50.759074+0000","last_undegraded":"2024-09-10T16:06:50.759074+0000","last_fullsized":"2024-09-10T16:06:50.759074+0000","mapping_epoch":35,"log_start":"43'7600","ondisk_log_start":"43'7600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13263814,"num_objects":3240,"num_object_clones":0,"num_object_copies":6480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3240,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":15711,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'17656","reported_seq":26340,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:06:53.616223+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:06:53.616223+0000","last_peered":"2024-09-10T16:06:53.616223+0000","last_clean":"2024-09-10T16:06:53.616223+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:06:53.616223+0000","last_undegraded":"2024-09-10T16:06:53.616223+0000","last_fullsized":"2024-09-10T16:06:53.616223+0000","mapping_epoch":44,"log_start":"43'7600","ondisk_log_start":"43'7600","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12865560,"num_objects":3142,"num_object_clones":0,"num_object_copies":6284,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3142,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15660,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'17517","reported_seq":17603,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:06:50.638678+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:06:50.638678+0000","last_peered":"2024-09-10T16:06:50.638678+0000","last_clean":"2024-09-10T16:06:50.638678+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:06:50.638678+0000","last_undegraded":"2024-09-10T16:06:50.638678+0000","last_fullsized":"2024-09-10T16:06:50.638678+0000","mapping_epoch":35,"log_start":"43'7500","ondisk_log_start":"43'7500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12873728,"num_objects":3143,"num_object_clones":0,"num_object_copies":6286,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3143,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17517,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'17680","reported_seq":17816,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:06:50.145853+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:06:50.145853+0000","last_peered":"2024-09-10T16:06:50.145853+0000","last_clean":"2024-09-10T16:06:50.145853+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:06:50.145853+0000","last_undegraded":"2024-09-10T16:06:50.145853+0000","last_fullsized":"2024-09-10T16:06:50.145853+0000","mapping_epoch":34,"log_start":"43'7600","ondisk_log_start":"43'7600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13152739,"num_objects":3212,"num_object_clones":0,"num_object_copies":6424,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3212,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14391,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'17809","reported_seq":17952,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:50.051950+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:06:50.051950+0000","last_peered":"2024-09-10T16:06:50.051950+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:06:50.051950+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"43'7800","ondisk_log_start":"43'7800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13320675,"num_objects":3253,"num_object_clones":0,"num_object_copies":6506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3253,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14421,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1118},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'17753","reported_seq":17896,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:50.090380+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:06:50.090380+0000","last_peered":"2024-09-10T16:06:50.090380+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:06:50.090380+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"43'7700","ondisk_log_start":"43'7700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13636550,"num_objects":3331,"num_object_clones":0,"num_object_copies":6662,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3331,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":14252,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-906},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'17998","reported_seq":18134,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:06:50.145978+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:06:50.145978+0000","last_peered":"2024-09-10T16:06:50.145978+0000","last_clean":"2024-09-10T16:06:50.145978+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:06:50.145978+0000","last_undegraded":"2024-09-10T16:06:50.145978+0000","last_fullsized":"2024-09-10T16:06:50.145978+0000","mapping_epoch":34,"log_start":"43'7900","ondisk_log_start":"43'7900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13582336,"num_objects":3316,"num_object_clones":0,"num_object_copies":6632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3316,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14845,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.876895+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:05:19.876895+0000","last_peered":"2024-09-10T16:05:19.876895+0000","last_clean":"2024-09-10T16:05:19.876895+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:05:19.876895+0000","last_undegraded":"2024-09-10T16:05:19.876895+0000","last_fullsized":"2024-09-10T16:05:19.876895+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'17142","reported_seq":17286,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:50.648154+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:06:50.648154+0000","last_peered":"2024-09-10T16:06:50.648154+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:06:50.648154+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"43'7100","ondisk_log_start":"43'7100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13578723,"num_objects":3316,"num_object_clones":0,"num_object_copies":6632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3316,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14961,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-807}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":100,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.875849+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:05:19.875849+0000","last_peered":"2024-09-10T16:05:19.875849+0000","last_clean":"2024-09-10T16:05:19.875849+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:05:19.875849+0000","last_undegraded":"2024-09-10T16:05:19.875849+0000","last_fullsized":"2024-09-10T16:05:19.875849+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'17620","reported_seq":17756,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:50.122620+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:06:50.122620+0000","last_peered":"2024-09-10T16:06:50.122620+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:06:50.122620+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"43'7600","ondisk_log_start":"43'7600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13475840,"num_objects":3290,"num_object_clones":0,"num_object_copies":6580,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3290,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14124,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-965}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":110,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.872023+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:05:19.872023+0000","last_peered":"2024-09-10T16:05:19.872023+0000","last_clean":"2024-09-10T16:05:19.872023+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:05:19.872023+0000","last_undegraded":"2024-09-10T16:05:19.872023+0000","last_fullsized":"2024-09-10T16:05:19.872023+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'17637","reported_seq":17777,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:50.723154+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:06:50.723154+0000","last_peered":"2024-09-10T16:06:50.723154+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:06:50.723154+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"43'7600","ondisk_log_start":"43'7600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12878307,"num_objects":3145,"num_object_clones":0,"num_object_copies":6290,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3145,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15317,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-1159}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877148+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:05:19.877148+0000","last_peered":"2024-09-10T16:05:19.877148+0000","last_clean":"2024-09-10T16:05:19.877148+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:05:19.877148+0000","last_undegraded":"2024-09-10T16:05:19.877148+0000","last_fullsized":"2024-09-10T16:05:19.877148+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'17923","reported_seq":22121,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:50.714321+0000","last_change":"2024-09-10T16:05:18.271152+0000","last_active":"2024-09-10T16:06:50.714321+0000","last_peered":"2024-09-10T16:06:50.714321+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:06:50.714321+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"43'7900","ondisk_log_start":"43'7900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13299712,"num_objects":3247,"num_object_clones":0,"num_object_copies":6494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3247,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15477,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2031,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":924}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'17586","reported_seq":17726,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:50.057240+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:06:50.057240+0000","last_peered":"2024-09-10T16:06:50.057240+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:06:50.057240+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"43'7500","ondisk_log_start":"43'7500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13320192,"num_objects":3252,"num_object_clones":0,"num_object_copies":6504,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3252,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14217,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1052}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877654+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:05:19.877654+0000","last_peered":"2024-09-10T16:05:19.877654+0000","last_clean":"2024-09-10T16:05:19.877654+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:05:19.877654+0000","last_undegraded":"2024-09-10T16:05:19.877654+0000","last_fullsized":"2024-09-10T16:05:19.877654+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'17409","reported_seq":25978,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:06:53.616143+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:06:53.616143+0000","last_peered":"2024-09-10T16:06:53.616143+0000","last_clean":"2024-09-10T16:06:53.616143+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:06:53.616143+0000","last_undegraded":"2024-09-10T16:06:53.616143+0000","last_fullsized":"2024-09-10T16:06:53.616143+0000","mapping_epoch":46,"log_start":"43'7400","ondisk_log_start":"43'7400","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13062144,"num_objects":3189,"num_object_clones":0,"num_object_copies":6378,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3189,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13799,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":100,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.876043+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:05:19.876043+0000","last_peered":"2024-09-10T16:05:19.876043+0000","last_clean":"2024-09-10T16:05:19.876043+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:05:19.876043+0000","last_undegraded":"2024-09-10T16:05:19.876043+0000","last_fullsized":"2024-09-10T16:05:19.876043+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'17844","reported_seq":17992,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:49.966775+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:06:49.966775+0000","last_peered":"2024-09-10T16:06:49.966775+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:06:49.966775+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"43'7800","ondisk_log_start":"43'7800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13123584,"num_objects":3204,"num_object_clones":0,"num_object_copies":6408,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3204,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14099,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1167}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877349+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:05:19.877349+0000","last_peered":"2024-09-10T16:05:19.877349+0000","last_clean":"2024-09-10T16:05:19.877349+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:05:19.877349+0000","last_undegraded":"2024-09-10T16:05:19.877349+0000","last_fullsized":"2024-09-10T16:05:19.877349+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877269+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:05:19.877269+0000","last_peered":"2024-09-10T16:05:19.877269+0000","last_clean":"2024-09-10T16:05:19.877269+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:05:19.877269+0000","last_undegraded":"2024-09-10T16:05:19.877269+0000","last_fullsized":"2024-09-10T16:05:19.877269+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'17902","reported_seq":18049,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:50.116663+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:06:50.116663+0000","last_peered":"2024-09-10T16:06:50.116663+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:06:50.116663+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"43'7900","ondisk_log_start":"43'7900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13501382,"num_objects":3298,"num_object_clones":0,"num_object_copies":6596,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3298,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14065,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1025},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":212083446,"num_objects":51788,"num_object_clones":0,"num_object_copies":103576,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40880,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51788,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":240318,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10509,"num_bytes_recovered":33095680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":631259136,"data_stored":631117311,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160644,"ondisk_log_size":160644,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986282,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":579716,"kb_used_data":241200,"kb_used_omap":4,"kb_used_meta":338491,"kb_avail":93792124,"statfs":{"total":96636764160,"available":96043134976,"internally_reserved":0,"allocated":246988800,"data_stored":246686116,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":346615466},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,2,3,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659228,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":504368,"kb_used_data":151968,"kb_used_omap":4,"kb_used_meta":352379,"kb_avail":93867472,"statfs":{"total":96636764160,"available":96120291328,"internally_reserved":0,"allocated":155615232,"data_stored":155283180,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":360836782},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,4,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182510,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":680952,"kb_used_data":225896,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93690888,"statfs":{"total":96636764160,"available":95939469312,"internally_reserved":0,"allocated":231317504,"data_stored":231014815,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":465956522},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,4,4,2,3,3,3],"upper_bound":512},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":229830656,"data_stored":229794526,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":155189248,"data_stored":155123755,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":246239232,"data_stored":246199030,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:06:55.280 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:06:55.849 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:06:55.850 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:06:55.877 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":330,"stamp":"2024-09-10T16:06:55.726542+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":206545673,"num_objects":50438,"num_object_clones":0,"num_object_copies":100876,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40880,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50438,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":241672,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10511,"num_bytes_recovered":33095699,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160699,"ondisk_log_size":160699,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1745584,"kb_used_data":612088,"kb_used_omap":12,"kb_used_meta":1133427,"kb_avail":281369936,"statfs":{"total":289910292480,"available":288122814464,"internally_reserved":0,"allocated":626778112,"data_stored":625840687,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1160629250},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,2,3,7,9,8],"upper_bound":128},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-24772608,"num_objects":-6048,"num_object_clones":0,"num_object_copies":-12096,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-228,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-6048,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6048,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":228,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.007643"},"pg_stats":[{"pgid":"2.f","version":"47'17618","reported_seq":17697,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:55.219450+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:06:55.219450+0000","last_peered":"2024-09-10T16:06:55.219450+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:06:55.219450+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"43'7600","ondisk_log_start":"43'7600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12509184,"num_objects":3054,"num_object_clones":0,"num_object_copies":6108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3054,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17618,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1205}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'17606","reported_seq":17724,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:06:50.759074+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:06:50.759074+0000","last_peered":"2024-09-10T16:06:50.759074+0000","last_clean":"2024-09-10T16:06:50.759074+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:06:50.759074+0000","last_undegraded":"2024-09-10T16:06:50.759074+0000","last_fullsized":"2024-09-10T16:06:50.759074+0000","mapping_epoch":35,"log_start":"43'7600","ondisk_log_start":"43'7600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13263814,"num_objects":3240,"num_object_clones":0,"num_object_copies":6480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3240,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":15711,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'17656","reported_seq":26340,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:06:53.616223+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:06:53.616223+0000","last_peered":"2024-09-10T16:06:53.616223+0000","last_clean":"2024-09-10T16:06:53.616223+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:06:53.616223+0000","last_undegraded":"2024-09-10T16:06:53.616223+0000","last_fullsized":"2024-09-10T16:06:53.616223+0000","mapping_epoch":44,"log_start":"43'7600","ondisk_log_start":"43'7600","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12865560,"num_objects":3142,"num_object_clones":0,"num_object_copies":6284,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3142,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15660,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'17517","reported_seq":17603,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:06:50.638678+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:06:50.638678+0000","last_peered":"2024-09-10T16:06:50.638678+0000","last_clean":"2024-09-10T16:06:50.638678+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:06:50.638678+0000","last_undegraded":"2024-09-10T16:06:50.638678+0000","last_fullsized":"2024-09-10T16:06:50.638678+0000","mapping_epoch":35,"log_start":"43'7500","ondisk_log_start":"43'7500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12873728,"num_objects":3143,"num_object_clones":0,"num_object_copies":6286,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3143,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17517,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'17847","reported_seq":17981,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:06:55.195977+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:06:55.195977+0000","last_peered":"2024-09-10T16:06:55.195977+0000","last_clean":"2024-09-10T16:06:55.195977+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:06:55.195977+0000","last_undegraded":"2024-09-10T16:06:55.195977+0000","last_fullsized":"2024-09-10T16:06:55.195977+0000","mapping_epoch":34,"log_start":"43'7800","ondisk_log_start":"43'7800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12468707,"num_objects":3045,"num_object_clones":0,"num_object_copies":6090,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3045,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14558,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'17945","reported_seq":18088,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:55.168178+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:06:55.168178+0000","last_peered":"2024-09-10T16:06:55.168178+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:06:55.168178+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"43'7900","ondisk_log_start":"43'7900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12763619,"num_objects":3117,"num_object_clones":0,"num_object_copies":6234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3117,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14557,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1254},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'17892","reported_seq":18035,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:55.089578+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:06:55.089578+0000","last_peered":"2024-09-10T16:06:55.089578+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:06:55.089578+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"43'7800","ondisk_log_start":"43'7800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13067206,"num_objects":3192,"num_object_clones":0,"num_object_copies":6384,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3192,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":14391,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1045},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'18140","reported_seq":18276,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:06:55.181346+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:06:55.181346+0000","last_peered":"2024-09-10T16:06:55.181346+0000","last_clean":"2024-09-10T16:06:55.181346+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:06:55.181346+0000","last_undegraded":"2024-09-10T16:06:55.181346+0000","last_fullsized":"2024-09-10T16:06:55.181346+0000","mapping_epoch":34,"log_start":"43'8100","ondisk_log_start":"43'8100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13000704,"num_objects":3174,"num_object_clones":0,"num_object_copies":6348,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3174,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14987,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.876895+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:05:19.876895+0000","last_peered":"2024-09-10T16:05:19.876895+0000","last_clean":"2024-09-10T16:05:19.876895+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:05:19.876895+0000","last_undegraded":"2024-09-10T16:05:19.876895+0000","last_fullsized":"2024-09-10T16:05:19.876895+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'17142","reported_seq":17286,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:50.648154+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:06:50.648154+0000","last_peered":"2024-09-10T16:06:50.648154+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:06:50.648154+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"43'7100","ondisk_log_start":"43'7100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13578723,"num_objects":3316,"num_object_clones":0,"num_object_copies":6632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3316,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14961,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-807}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":100,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.875849+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:05:19.875849+0000","last_peered":"2024-09-10T16:05:19.875849+0000","last_clean":"2024-09-10T16:05:19.875849+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:05:19.875849+0000","last_undegraded":"2024-09-10T16:05:19.875849+0000","last_fullsized":"2024-09-10T16:05:19.875849+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'17778","reported_seq":17914,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:55.140385+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:06:55.140385+0000","last_peered":"2024-09-10T16:06:55.140385+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:06:55.140385+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"43'7700","ondisk_log_start":"43'7700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12828672,"num_objects":3132,"num_object_clones":0,"num_object_copies":6264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3132,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14282,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1123}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":110,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.872023+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:05:19.872023+0000","last_peered":"2024-09-10T16:05:19.872023+0000","last_clean":"2024-09-10T16:05:19.872023+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:05:19.872023+0000","last_undegraded":"2024-09-10T16:05:19.872023+0000","last_fullsized":"2024-09-10T16:05:19.872023+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'17637","reported_seq":17777,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:50.723154+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:06:50.723154+0000","last_peered":"2024-09-10T16:06:50.723154+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:06:50.723154+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"43'7600","ondisk_log_start":"43'7600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12878307,"num_objects":3145,"num_object_clones":0,"num_object_copies":6290,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3145,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15317,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-1159}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877148+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:05:19.877148+0000","last_peered":"2024-09-10T16:05:19.877148+0000","last_clean":"2024-09-10T16:05:19.877148+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:05:19.877148+0000","last_undegraded":"2024-09-10T16:05:19.877148+0000","last_fullsized":"2024-09-10T16:05:19.877148+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'17923","reported_seq":22121,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:50.714321+0000","last_change":"2024-09-10T16:05:18.271152+0000","last_active":"2024-09-10T16:06:50.714321+0000","last_peered":"2024-09-10T16:06:50.714321+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:06:50.714321+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"43'7900","ondisk_log_start":"43'7900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13299712,"num_objects":3247,"num_object_clones":0,"num_object_copies":6494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3247,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15477,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2031,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":924}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'17756","reported_seq":17896,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:55.219348+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:06:55.219348+0000","last_peered":"2024-09-10T16:06:55.219348+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:06:55.219348+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"43'7700","ondisk_log_start":"43'7700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12623872,"num_objects":3082,"num_object_clones":0,"num_object_copies":6164,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3082,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14387,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1222}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877654+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:05:19.877654+0000","last_peered":"2024-09-10T16:05:19.877654+0000","last_clean":"2024-09-10T16:05:19.877654+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:05:19.877654+0000","last_undegraded":"2024-09-10T16:05:19.877654+0000","last_fullsized":"2024-09-10T16:05:19.877654+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'17409","reported_seq":25978,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:06:53.616143+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:06:53.616143+0000","last_peered":"2024-09-10T16:06:53.616143+0000","last_clean":"2024-09-10T16:06:53.616143+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:06:53.616143+0000","last_undegraded":"2024-09-10T16:06:53.616143+0000","last_fullsized":"2024-09-10T16:06:53.616143+0000","mapping_epoch":46,"log_start":"43'7400","ondisk_log_start":"43'7400","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13062144,"num_objects":3189,"num_object_clones":0,"num_object_copies":6378,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3189,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13799,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":100,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.876043+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:05:19.876043+0000","last_peered":"2024-09-10T16:05:19.876043+0000","last_clean":"2024-09-10T16:05:19.876043+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:05:19.876043+0000","last_undegraded":"2024-09-10T16:05:19.876043+0000","last_fullsized":"2024-09-10T16:05:19.876043+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'17984","reported_seq":18132,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:55.173070+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:06:55.173070+0000","last_peered":"2024-09-10T16:06:55.173070+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:06:55.173070+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"43'7900","ondisk_log_start":"43'7900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12550144,"num_objects":3064,"num_object_clones":0,"num_object_copies":6128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3064,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14239,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1307}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877349+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:05:19.877349+0000","last_peered":"2024-09-10T16:05:19.877349+0000","last_clean":"2024-09-10T16:05:19.877349+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:05:19.877349+0000","last_undegraded":"2024-09-10T16:05:19.877349+0000","last_fullsized":"2024-09-10T16:05:19.877349+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877269+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:05:19.877269+0000","last_peered":"2024-09-10T16:05:19.877269+0000","last_clean":"2024-09-10T16:05:19.877269+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:05:19.877269+0000","last_undegraded":"2024-09-10T16:05:19.877269+0000","last_fullsized":"2024-09-10T16:05:19.877269+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'18046","reported_seq":18196,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:55.192900+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:06:55.192900+0000","last_peered":"2024-09-10T16:06:55.192900+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:06:55.192900+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"43'8000","ondisk_log_start":"43'8000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12911558,"num_objects":3154,"num_object_clones":0,"num_object_copies":6308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3154,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14209,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1169},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":206545654,"num_objects":50436,"num_object_clones":0,"num_object_copies":100872,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40880,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50436,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":241670,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10509,"num_bytes_recovered":33095680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":624009216,"data_stored":623867391,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160696,"ondisk_log_size":160696,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986282,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":579716,"kb_used_data":241200,"kb_used_omap":4,"kb_used_meta":338491,"kb_avail":93792124,"statfs":{"total":96636764160,"available":96043134976,"internally_reserved":0,"allocated":246988800,"data_stored":246686116,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":346615466},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,2,3,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659228,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":504368,"kb_used_data":151968,"kb_used_omap":4,"kb_used_meta":352379,"kb_avail":93867472,"statfs":{"total":96636764160,"available":96120291328,"internally_reserved":0,"allocated":155615232,"data_stored":155283180,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":360836782},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,4,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182511,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":661500,"kb_used_data":218920,"kb_used_omap":4,"kb_used_meta":442555,"kb_avail":93710340,"statfs":{"total":96636764160,"available":95959388160,"internally_reserved":0,"allocated":224174080,"data_stored":223871391,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":453177002},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,2,2,1,4,7],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":222580736,"data_stored":222544606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":155189248,"data_stored":155123755,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":246239232,"data_stored":246199030,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:06:55.878 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-10T16:06:55.879 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "47'17618", 'reported_seq': 17697, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:06:55.219450+0000', 'last_change': '2024-09-10T15:59:16.281558+0000', 'last_active': '2024-09-10T16:06:55.219450+0000', 'last_peered': '2024-09-10T16:06:55.219450+0000', 'last_clean': '2024-09-10T15:59:14.132456+0000', 'last_became_active': '2024-09-10T15:59:16.274407+0000', 'last_became_peered': '2024-09-10T15:59:16.274407+0000', 'last_unstale': '2024-09-10T16:06:55.219450+0000', 'last_undegraded': '2024-09-10T15:59:15.851279+0000', 'last_fullsized': '2024-09-10T15:59:15.774356+0000', 'mapping_epoch': 42, 'log_start': "43'7600", 'ondisk_log_start': "43'7600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'objects_scrubbed': 0, 'log_size': 10018, 'log_dups_size': 0, 'ondisk_log_size': 10018, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 12509184, 'num_objects': 3054, 'num_object_clones': 0, 'num_object_copies': 6108, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4259, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3054, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17618, 'num_write_kb': 41344, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1205}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:06:55.879 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-10T16:06:55.879 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "47'17945", 'reported_seq': 18088, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:06:55.168178+0000', 'last_change': '2024-09-10T15:59:16.308264+0000', 'last_active': '2024-09-10T16:06:55.168178+0000', 'last_peered': '2024-09-10T16:06:55.168178+0000', 'last_clean': '2024-09-10T15:59:14.111551+0000', 'last_became_active': '2024-09-10T15:59:16.300040+0000', 'last_became_peered': '2024-09-10T15:59:16.300040+0000', 'last_unstale': '2024-09-10T16:06:55.168178+0000', 'last_undegraded': '2024-09-10T15:59:15.878234+0000', 'last_fullsized': '2024-09-10T15:59:15.769328+0000', 'mapping_epoch': 42, 'log_start': "43'7900", 'ondisk_log_start': "43'7900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3442", 'last_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'objects_scrubbed': 3384, 'log_size': 10045, 'log_dups_size': 0, 'ondisk_log_size': 10045, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1292, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 12763619, 'num_objects': 3117, 'num_object_clones': 0, 'num_object_copies': 6234, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4372, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3117, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 14557, 'num_write_kb': 28580, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1254}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:06:55.879 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-10T16:06:55.879 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "47'17892", 'reported_seq': 18035, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:06:55.089578+0000', 'last_change': '2024-09-10T15:59:16.331899+0000', 'last_active': '2024-09-10T16:06:55.089578+0000', 'last_peered': '2024-09-10T16:06:55.089578+0000', 'last_clean': '2024-09-10T15:59:14.139436+0000', 'last_became_active': '2024-09-10T15:59:16.318715+0000', 'last_became_peered': '2024-09-10T15:59:16.318715+0000', 'last_unstale': '2024-09-10T16:06:55.089578+0000', 'last_undegraded': '2024-09-10T15:59:15.877524+0000', 'last_fullsized': '2024-09-10T15:59:15.769042+0000', 'mapping_epoch': 42, 'log_start': "43'7800", 'ondisk_log_start': "43'7800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3537", 'last_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'objects_scrubbed': 3497, 'log_size': 10092, 'log_dups_size': 0, 'ondisk_log_size': 10092, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1224, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 13067206, 'num_objects': 3192, 'num_object_clones': 0, 'num_object_copies': 6384, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4237, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3192, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 14391, 'num_write_kb': 28172, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1045}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:06:55.879 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-10T16:06:55.879 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "47'17142", 'reported_seq': 17286, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:06:50.648154+0000', 'last_change': '2024-09-10T15:59:16.175680+0000', 'last_active': '2024-09-10T16:06:50.648154+0000', 'last_peered': '2024-09-10T16:06:50.648154+0000', 'last_clean': '2024-09-10T15:59:14.088187+0000', 'last_became_active': '2024-09-10T15:59:16.165810+0000', 'last_became_peered': '2024-09-10T15:59:16.165810+0000', 'last_unstale': '2024-09-10T16:06:50.648154+0000', 'last_undegraded': '2024-09-10T15:59:15.822098+0000', 'last_fullsized': '2024-09-10T15:59:15.769123+0000', 'mapping_epoch': 42, 'log_start': "43'7100", 'ondisk_log_start': "43'7100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2218", 'last_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'objects_scrubbed': 2175, 'log_size': 10042, 'log_dups_size': 0, 'ondisk_log_size': 10042, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 825, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 13578723, 'num_objects': 3316, 'num_object_clones': 0, 'num_object_copies': 6632, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4124, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3316, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 14961, 'num_write_kb': 32204, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': -807}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:06:55.879 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-10T16:06:55.879 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "47'17778", 'reported_seq': 17914, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:06:55.140385+0000', 'last_change': '2024-09-10T15:59:16.277742+0000', 'last_active': '2024-09-10T16:06:55.140385+0000', 'last_peered': '2024-09-10T16:06:55.140385+0000', 'last_clean': '2024-09-10T15:59:14.129081+0000', 'last_became_active': '2024-09-10T15:59:16.269835+0000', 'last_became_peered': '2024-09-10T15:59:16.269835+0000', 'last_unstale': '2024-09-10T16:06:55.140385+0000', 'last_undegraded': '2024-09-10T15:59:15.860857+0000', 'last_fullsized': '2024-09-10T15:59:15.771410+0000', 'mapping_epoch': 42, 'log_start': "43'7700", 'ondisk_log_start': "43'7700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3557", 'last_scrub_stamp': '2024-09-10T15:59:02.493678+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:02.493678+0000', 'objects_scrubbed': 3488, 'log_size': 10078, 'log_dups_size': 0, 'ondisk_log_size': 10078, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1472, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 12828672, 'num_objects': 3132, 'num_object_clones': 0, 'num_object_copies': 6264, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4257, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3132, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14282, 'num_write_kb': 27852, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1123}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:06:55.879 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-10T16:06:55.880 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "47'17637", 'reported_seq': 17777, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:06:50.723154+0000', 'last_change': '2024-09-10T15:59:16.284311+0000', 'last_active': '2024-09-10T16:06:50.723154+0000', 'last_peered': '2024-09-10T16:06:50.723154+0000', 'last_clean': '2024-09-10T15:59:14.116041+0000', 'last_became_active': '2024-09-10T15:59:16.274543+0000', 'last_became_peered': '2024-09-10T15:59:16.274543+0000', 'last_unstale': '2024-09-10T16:06:50.723154+0000', 'last_undegraded': '2024-09-10T15:59:15.812484+0000', 'last_fullsized': '2024-09-10T15:59:15.770885+0000', 'mapping_epoch': 42, 'log_start': "43'7600", 'ondisk_log_start': "43'7600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2356", 'last_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'objects_scrubbed': 2316, 'log_size': 10037, 'log_dups_size': 0, 'ondisk_log_size': 10037, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 783, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 12878307, 'num_objects': 3145, 'num_object_clones': 0, 'num_object_copies': 6290, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3145, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 15317, 'num_write_kb': 32292, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': -1159}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:06:55.880 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-10T16:06:55.880 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "47'17923", 'reported_seq': 22121, 'reported_epoch': 47, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:06:50.714321+0000', 'last_change': '2024-09-10T16:05:18.271152+0000', 'last_active': '2024-09-10T16:06:50.714321+0000', 'last_peered': '2024-09-10T16:06:50.714321+0000', 'last_clean': '2024-09-10T15:59:14.110654+0000', 'last_became_active': '2024-09-10T15:59:16.164645+0000', 'last_became_peered': '2024-09-10T15:59:16.164645+0000', 'last_unstale': '2024-09-10T16:06:50.714321+0000', 'last_undegraded': '2024-09-10T15:59:15.803593+0000', 'last_fullsized': '2024-09-10T15:59:15.769083+0000', 'mapping_epoch': 42, 'log_start': "43'7900", 'ondisk_log_start': "43'7900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2486", 'last_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'objects_scrubbed': 2440, 'log_size': 10023, 'log_dups_size': 0, 'ondisk_log_size': 10023, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 792, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 13299712, 'num_objects': 3247, 'num_object_clones': 0, 'num_object_copies': 6494, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2323, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3247, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15477, 'num_write_kb': 32568, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2031, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 924}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:06:55.880 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-10T16:06:55.880 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "47'17756", 'reported_seq': 17896, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:06:55.219348+0000', 'last_change': '2024-09-10T15:59:16.315641+0000', 'last_active': '2024-09-10T16:06:55.219348+0000', 'last_peered': '2024-09-10T16:06:55.219348+0000', 'last_clean': '2024-09-10T15:59:14.117001+0000', 'last_became_active': '2024-09-10T15:59:16.302245+0000', 'last_became_peered': '2024-09-10T15:59:16.302245+0000', 'last_unstale': '2024-09-10T16:06:55.219348+0000', 'last_undegraded': '2024-09-10T15:59:15.845012+0000', 'last_fullsized': '2024-09-10T15:59:15.770337+0000', 'mapping_epoch': 42, 'log_start': "43'7700", 'ondisk_log_start': "43'7700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3404", 'last_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'objects_scrubbed': 3367, 'log_size': 10056, 'log_dups_size': 0, 'ondisk_log_size': 10056, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 983, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 12623872, 'num_objects': 3082, 'num_object_clones': 0, 'num_object_copies': 6164, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3082, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14387, 'num_write_kb': 28204, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1222}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:06:55.880 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-10T16:06:55.880 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "47'17984", 'reported_seq': 18132, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:06:55.173070+0000', 'last_change': '2024-09-10T15:59:16.311291+0000', 'last_active': '2024-09-10T16:06:55.173070+0000', 'last_peered': '2024-09-10T16:06:55.173070+0000', 'last_clean': '2024-09-10T15:59:14.116363+0000', 'last_became_active': '2024-09-10T15:59:16.301244+0000', 'last_became_peered': '2024-09-10T15:59:16.301244+0000', 'last_unstale': '2024-09-10T16:06:55.173070+0000', 'last_undegraded': '2024-09-10T15:59:15.883434+0000', 'last_fullsized': '2024-09-10T15:59:15.772756+0000', 'mapping_epoch': 42, 'log_start': "43'7900", 'ondisk_log_start': "43'7900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3789", 'last_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'objects_scrubbed': 3741, 'log_size': 10084, 'log_dups_size': 0, 'ondisk_log_size': 10084, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1440, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 12550144, 'num_objects': 3064, 'num_object_clones': 0, 'num_object_copies': 6128, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4373, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3064, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14239, 'num_write_kb': 27124, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1307}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:06:55.880 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-10T16:06:55.880 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "47'18046", 'reported_seq': 18196, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:06:55.192900+0000', 'last_change': '2024-09-10T15:59:16.313409+0000', 'last_active': '2024-09-10T16:06:55.192900+0000', 'last_peered': '2024-09-10T16:06:55.192900+0000', 'last_clean': '2024-09-10T15:59:14.129780+0000', 'last_became_active': '2024-09-10T15:59:16.300932+0000', 'last_became_peered': '2024-09-10T15:59:16.300932+0000', 'last_unstale': '2024-09-10T16:06:55.192900+0000', 'last_undegraded': '2024-09-10T15:59:15.887823+0000', 'last_fullsized': '2024-09-10T15:59:15.768513+0000', 'mapping_epoch': 42, 'log_start': "43'8000", 'ondisk_log_start': "43'8000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3880", 'last_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'objects_scrubbed': 3827, 'log_size': 10046, 'log_dups_size': 0, 'ondisk_log_size': 10046, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1397, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 12911558, 'num_objects': 3154, 'num_object_clones': 0, 'num_object_copies': 6308, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4325, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3154, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14209, 'num_write_kb': 27072, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1169}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:06:55.881 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T16:06:56.614 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:06:56.615 INFO:teuthology.orchestra.run.smithi115.stdout:{"fsid":"7effee29-df05-4351-9741-ee203564ed81","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 40880/100876 objects degraded (40.525%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":580,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":47,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983954,"num_in_osds":3,"osd_in_since":1725983947,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":50438,"data_bytes":206545673,"bytes_used":1787478016,"bytes_avail":288122814464,"bytes_total":289910292480,"degraded_objects":40880,"degraded_total":100876,"degraded_ratio":0.40525000991316079,"recovering_objects_per_sec":18,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":228,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":503},"fsmap":{"epoch":1,"btime":"2024-09-10T15:57:14:967120+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T16:00:13.560776+0000","services":{}},"progress_events":{}} 2024-09-10T16:06:56.641 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-10T16:06:56.641 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:06:57.192 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:06:57.195 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:06:57.217 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":330,"stamp":"2024-09-10T16:06:55.726542+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":206545673,"num_objects":50438,"num_object_clones":0,"num_object_copies":100876,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40880,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50438,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":241672,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10511,"num_bytes_recovered":33095699,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160699,"ondisk_log_size":160699,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1745584,"kb_used_data":612088,"kb_used_omap":12,"kb_used_meta":1133427,"kb_avail":281369936,"statfs":{"total":289910292480,"available":288122814464,"internally_reserved":0,"allocated":626778112,"data_stored":625840687,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1160629250},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,2,3,7,9,8],"upper_bound":128},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-24772608,"num_objects":-6048,"num_object_clones":0,"num_object_copies":-12096,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-228,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-6048,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6048,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":228,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.007643"},"pg_stats":[{"pgid":"2.f","version":"47'17618","reported_seq":17697,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:55.219450+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:06:55.219450+0000","last_peered":"2024-09-10T16:06:55.219450+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:06:55.219450+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"43'7600","ondisk_log_start":"43'7600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12509184,"num_objects":3054,"num_object_clones":0,"num_object_copies":6108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3054,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17618,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1205}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'17606","reported_seq":17724,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:06:50.759074+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:06:50.759074+0000","last_peered":"2024-09-10T16:06:50.759074+0000","last_clean":"2024-09-10T16:06:50.759074+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:06:50.759074+0000","last_undegraded":"2024-09-10T16:06:50.759074+0000","last_fullsized":"2024-09-10T16:06:50.759074+0000","mapping_epoch":35,"log_start":"43'7600","ondisk_log_start":"43'7600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13263814,"num_objects":3240,"num_object_clones":0,"num_object_copies":6480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3240,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":15711,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'17656","reported_seq":26340,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:06:53.616223+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:06:53.616223+0000","last_peered":"2024-09-10T16:06:53.616223+0000","last_clean":"2024-09-10T16:06:53.616223+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:06:53.616223+0000","last_undegraded":"2024-09-10T16:06:53.616223+0000","last_fullsized":"2024-09-10T16:06:53.616223+0000","mapping_epoch":44,"log_start":"43'7600","ondisk_log_start":"43'7600","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12865560,"num_objects":3142,"num_object_clones":0,"num_object_copies":6284,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3142,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15660,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'17517","reported_seq":17603,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:06:50.638678+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:06:50.638678+0000","last_peered":"2024-09-10T16:06:50.638678+0000","last_clean":"2024-09-10T16:06:50.638678+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:06:50.638678+0000","last_undegraded":"2024-09-10T16:06:50.638678+0000","last_fullsized":"2024-09-10T16:06:50.638678+0000","mapping_epoch":35,"log_start":"43'7500","ondisk_log_start":"43'7500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12873728,"num_objects":3143,"num_object_clones":0,"num_object_copies":6286,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3143,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17517,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'17847","reported_seq":17981,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:06:55.195977+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:06:55.195977+0000","last_peered":"2024-09-10T16:06:55.195977+0000","last_clean":"2024-09-10T16:06:55.195977+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:06:55.195977+0000","last_undegraded":"2024-09-10T16:06:55.195977+0000","last_fullsized":"2024-09-10T16:06:55.195977+0000","mapping_epoch":34,"log_start":"43'7800","ondisk_log_start":"43'7800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12468707,"num_objects":3045,"num_object_clones":0,"num_object_copies":6090,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3045,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14558,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'17945","reported_seq":18088,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:55.168178+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:06:55.168178+0000","last_peered":"2024-09-10T16:06:55.168178+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:06:55.168178+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"43'7900","ondisk_log_start":"43'7900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12763619,"num_objects":3117,"num_object_clones":0,"num_object_copies":6234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3117,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14557,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1254},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'17892","reported_seq":18035,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:55.089578+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:06:55.089578+0000","last_peered":"2024-09-10T16:06:55.089578+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:06:55.089578+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"43'7800","ondisk_log_start":"43'7800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13067206,"num_objects":3192,"num_object_clones":0,"num_object_copies":6384,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3192,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":14391,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1045},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'18140","reported_seq":18276,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:06:55.181346+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:06:55.181346+0000","last_peered":"2024-09-10T16:06:55.181346+0000","last_clean":"2024-09-10T16:06:55.181346+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:06:55.181346+0000","last_undegraded":"2024-09-10T16:06:55.181346+0000","last_fullsized":"2024-09-10T16:06:55.181346+0000","mapping_epoch":34,"log_start":"43'8100","ondisk_log_start":"43'8100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13000704,"num_objects":3174,"num_object_clones":0,"num_object_copies":6348,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3174,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14987,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.876895+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:05:19.876895+0000","last_peered":"2024-09-10T16:05:19.876895+0000","last_clean":"2024-09-10T16:05:19.876895+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:05:19.876895+0000","last_undegraded":"2024-09-10T16:05:19.876895+0000","last_fullsized":"2024-09-10T16:05:19.876895+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'17142","reported_seq":17286,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:50.648154+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:06:50.648154+0000","last_peered":"2024-09-10T16:06:50.648154+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:06:50.648154+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"43'7100","ondisk_log_start":"43'7100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13578723,"num_objects":3316,"num_object_clones":0,"num_object_copies":6632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3316,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14961,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-807}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":100,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.875849+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:05:19.875849+0000","last_peered":"2024-09-10T16:05:19.875849+0000","last_clean":"2024-09-10T16:05:19.875849+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:05:19.875849+0000","last_undegraded":"2024-09-10T16:05:19.875849+0000","last_fullsized":"2024-09-10T16:05:19.875849+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'17778","reported_seq":17914,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:55.140385+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:06:55.140385+0000","last_peered":"2024-09-10T16:06:55.140385+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:06:55.140385+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"43'7700","ondisk_log_start":"43'7700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12828672,"num_objects":3132,"num_object_clones":0,"num_object_copies":6264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3132,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14282,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1123}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":110,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.872023+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:05:19.872023+0000","last_peered":"2024-09-10T16:05:19.872023+0000","last_clean":"2024-09-10T16:05:19.872023+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:05:19.872023+0000","last_undegraded":"2024-09-10T16:05:19.872023+0000","last_fullsized":"2024-09-10T16:05:19.872023+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'17637","reported_seq":17777,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:50.723154+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:06:50.723154+0000","last_peered":"2024-09-10T16:06:50.723154+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:06:50.723154+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"43'7600","ondisk_log_start":"43'7600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12878307,"num_objects":3145,"num_object_clones":0,"num_object_copies":6290,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3145,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15317,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-1159}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877148+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:05:19.877148+0000","last_peered":"2024-09-10T16:05:19.877148+0000","last_clean":"2024-09-10T16:05:19.877148+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:05:19.877148+0000","last_undegraded":"2024-09-10T16:05:19.877148+0000","last_fullsized":"2024-09-10T16:05:19.877148+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'17923","reported_seq":22121,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:50.714321+0000","last_change":"2024-09-10T16:05:18.271152+0000","last_active":"2024-09-10T16:06:50.714321+0000","last_peered":"2024-09-10T16:06:50.714321+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:06:50.714321+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"43'7900","ondisk_log_start":"43'7900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13299712,"num_objects":3247,"num_object_clones":0,"num_object_copies":6494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3247,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15477,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2031,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":924}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'17756","reported_seq":17896,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:55.219348+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:06:55.219348+0000","last_peered":"2024-09-10T16:06:55.219348+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:06:55.219348+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"43'7700","ondisk_log_start":"43'7700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12623872,"num_objects":3082,"num_object_clones":0,"num_object_copies":6164,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3082,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14387,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1222}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877654+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:05:19.877654+0000","last_peered":"2024-09-10T16:05:19.877654+0000","last_clean":"2024-09-10T16:05:19.877654+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:05:19.877654+0000","last_undegraded":"2024-09-10T16:05:19.877654+0000","last_fullsized":"2024-09-10T16:05:19.877654+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'17409","reported_seq":25978,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:06:53.616143+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:06:53.616143+0000","last_peered":"2024-09-10T16:06:53.616143+0000","last_clean":"2024-09-10T16:06:53.616143+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:06:53.616143+0000","last_undegraded":"2024-09-10T16:06:53.616143+0000","last_fullsized":"2024-09-10T16:06:53.616143+0000","mapping_epoch":46,"log_start":"43'7400","ondisk_log_start":"43'7400","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13062144,"num_objects":3189,"num_object_clones":0,"num_object_copies":6378,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3189,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13799,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":100,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.876043+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:05:19.876043+0000","last_peered":"2024-09-10T16:05:19.876043+0000","last_clean":"2024-09-10T16:05:19.876043+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:05:19.876043+0000","last_undegraded":"2024-09-10T16:05:19.876043+0000","last_fullsized":"2024-09-10T16:05:19.876043+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'17984","reported_seq":18132,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:55.173070+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:06:55.173070+0000","last_peered":"2024-09-10T16:06:55.173070+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:06:55.173070+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"43'7900","ondisk_log_start":"43'7900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12550144,"num_objects":3064,"num_object_clones":0,"num_object_copies":6128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3064,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14239,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1307}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877349+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:05:19.877349+0000","last_peered":"2024-09-10T16:05:19.877349+0000","last_clean":"2024-09-10T16:05:19.877349+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:05:19.877349+0000","last_undegraded":"2024-09-10T16:05:19.877349+0000","last_fullsized":"2024-09-10T16:05:19.877349+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877269+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:05:19.877269+0000","last_peered":"2024-09-10T16:05:19.877269+0000","last_clean":"2024-09-10T16:05:19.877269+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:05:19.877269+0000","last_undegraded":"2024-09-10T16:05:19.877269+0000","last_fullsized":"2024-09-10T16:05:19.877269+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'18046","reported_seq":18196,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:55.192900+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:06:55.192900+0000","last_peered":"2024-09-10T16:06:55.192900+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:06:55.192900+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"43'8000","ondisk_log_start":"43'8000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12911558,"num_objects":3154,"num_object_clones":0,"num_object_copies":6308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3154,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14209,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1169},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":206545654,"num_objects":50436,"num_object_clones":0,"num_object_copies":100872,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40880,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50436,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":241670,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10509,"num_bytes_recovered":33095680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":624009216,"data_stored":623867391,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160696,"ondisk_log_size":160696,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986282,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":579716,"kb_used_data":241200,"kb_used_omap":4,"kb_used_meta":338491,"kb_avail":93792124,"statfs":{"total":96636764160,"available":96043134976,"internally_reserved":0,"allocated":246988800,"data_stored":246686116,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":346615466},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,2,3,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659228,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":504368,"kb_used_data":151968,"kb_used_omap":4,"kb_used_meta":352379,"kb_avail":93867472,"statfs":{"total":96636764160,"available":96120291328,"internally_reserved":0,"allocated":155615232,"data_stored":155283180,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":360836782},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,4,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182511,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":661500,"kb_used_data":218920,"kb_used_omap":4,"kb_used_meta":442555,"kb_avail":93710340,"statfs":{"total":96636764160,"available":95959388160,"internally_reserved":0,"allocated":224174080,"data_stored":223871391,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":453177002},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,2,2,1,4,7],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":222580736,"data_stored":222544606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":155189248,"data_stored":155123755,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":246239232,"data_stored":246199030,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:07:00.220 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:07:00.764 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:07:00.765 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:07:00.794 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":332,"stamp":"2024-09-10T16:06:59.727947+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":202433289,"num_objects":49434,"num_object_clones":0,"num_object_copies":98868,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40754,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49434,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":242676,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10637,"num_bytes_recovered":33095699,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161003,"ondisk_log_size":161003,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1746624,"kb_used_data":600644,"kb_used_omap":12,"kb_used_meta":1145907,"kb_avail":281368896,"statfs":{"total":289910292480,"available":288121749504,"internally_reserved":0,"allocated":615059456,"data_stored":614122031,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1173408770},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,2,5,6,7,15],"upper_bound":128},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-20434944,"num_objects":-4989,"num_object_clones":0,"num_object_copies":-9978,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4989,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4989,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":237,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.007671"},"pg_stats":[{"pgid":"2.f","version":"47'17618","reported_seq":17697,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:55.219450+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:06:55.219450+0000","last_peered":"2024-09-10T16:06:55.219450+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:06:55.219450+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"43'7600","ondisk_log_start":"43'7600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12509184,"num_objects":3054,"num_object_clones":0,"num_object_copies":6108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3054,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17618,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1205}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'17755","reported_seq":17872,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:06:55.784537+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:06:55.784537+0000","last_peered":"2024-09-10T16:06:55.784537+0000","last_clean":"2024-09-10T16:06:55.784537+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:06:55.784537+0000","last_undegraded":"2024-09-10T16:06:55.784537+0000","last_fullsized":"2024-09-10T16:06:55.784537+0000","mapping_epoch":35,"log_start":"43'7700","ondisk_log_start":"43'7700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12653510,"num_objects":3091,"num_object_clones":0,"num_object_copies":6182,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3091,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":15860,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'17794","reported_seq":26478,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:06:58.651659+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:06:58.651659+0000","last_peered":"2024-09-10T16:06:58.651659+0000","last_clean":"2024-09-10T16:06:58.651659+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:06:58.651659+0000","last_undegraded":"2024-09-10T16:06:58.651659+0000","last_fullsized":"2024-09-10T16:06:58.651659+0000","mapping_epoch":44,"log_start":"43'7700","ondisk_log_start":"43'7700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12300312,"num_objects":3004,"num_object_clones":0,"num_object_copies":6008,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3004,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15798,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'17644","reported_seq":17729,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:06:55.658196+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:06:55.658196+0000","last_peered":"2024-09-10T16:06:55.658196+0000","last_clean":"2024-09-10T16:06:55.658196+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:06:55.658196+0000","last_undegraded":"2024-09-10T16:06:55.658196+0000","last_fullsized":"2024-09-10T16:06:55.658196+0000","mapping_epoch":35,"log_start":"43'7600","ondisk_log_start":"43'7600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12353536,"num_objects":3016,"num_object_clones":0,"num_object_copies":6032,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3016,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17644,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'17847","reported_seq":17981,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:06:55.195977+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:06:55.195977+0000","last_peered":"2024-09-10T16:06:55.195977+0000","last_clean":"2024-09-10T16:06:55.195977+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:06:55.195977+0000","last_undegraded":"2024-09-10T16:06:55.195977+0000","last_fullsized":"2024-09-10T16:06:55.195977+0000","mapping_epoch":34,"log_start":"43'7800","ondisk_log_start":"43'7800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12468707,"num_objects":3045,"num_object_clones":0,"num_object_copies":6090,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3045,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14558,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'17945","reported_seq":18088,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:55.168178+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:06:55.168178+0000","last_peered":"2024-09-10T16:06:55.168178+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:06:55.168178+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"43'7900","ondisk_log_start":"43'7900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12763619,"num_objects":3117,"num_object_clones":0,"num_object_copies":6234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3117,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14557,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1254},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'17892","reported_seq":18035,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:55.089578+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:06:55.089578+0000","last_peered":"2024-09-10T16:06:55.089578+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:06:55.089578+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"43'7800","ondisk_log_start":"43'7800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13067206,"num_objects":3192,"num_object_clones":0,"num_object_copies":6384,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3192,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":14391,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1045},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'18140","reported_seq":18276,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:06:55.181346+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:06:55.181346+0000","last_peered":"2024-09-10T16:06:55.181346+0000","last_clean":"2024-09-10T16:06:55.181346+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:06:55.181346+0000","last_undegraded":"2024-09-10T16:06:55.181346+0000","last_fullsized":"2024-09-10T16:06:55.181346+0000","mapping_epoch":34,"log_start":"43'8100","ondisk_log_start":"43'8100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13000704,"num_objects":3174,"num_object_clones":0,"num_object_copies":6348,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3174,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14987,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.876895+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:05:19.876895+0000","last_peered":"2024-09-10T16:05:19.876895+0000","last_clean":"2024-09-10T16:05:19.876895+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:05:19.876895+0000","last_undegraded":"2024-09-10T16:05:19.876895+0000","last_fullsized":"2024-09-10T16:05:19.876895+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'17280","reported_seq":17426,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:55.657484+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:06:55.657484+0000","last_peered":"2024-09-10T16:06:55.657484+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:06:55.657484+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"43'7200","ondisk_log_start":"43'7200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13013475,"num_objects":3178,"num_object_clones":0,"num_object_copies":6356,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3178,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15099,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-945}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":100,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.875849+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:05:19.875849+0000","last_peered":"2024-09-10T16:05:19.875849+0000","last_clean":"2024-09-10T16:05:19.875849+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:05:19.875849+0000","last_undegraded":"2024-09-10T16:05:19.875849+0000","last_fullsized":"2024-09-10T16:05:19.875849+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'17778","reported_seq":17914,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:55.140385+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:06:55.140385+0000","last_peered":"2024-09-10T16:06:55.140385+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:06:55.140385+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"43'7700","ondisk_log_start":"43'7700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12828672,"num_objects":3132,"num_object_clones":0,"num_object_copies":6264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3132,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14282,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1123}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":110,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.872023+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:05:19.872023+0000","last_peered":"2024-09-10T16:05:19.872023+0000","last_clean":"2024-09-10T16:05:19.872023+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:05:19.872023+0000","last_undegraded":"2024-09-10T16:05:19.872023+0000","last_fullsized":"2024-09-10T16:05:19.872023+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'17769","reported_seq":17910,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:55.784106+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:06:55.784106+0000","last_peered":"2024-09-10T16:06:55.784106+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:06:55.784106+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"43'7700","ondisk_log_start":"43'7700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12337635,"num_objects":3013,"num_object_clones":0,"num_object_copies":6026,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3013,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15449,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-1291}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877148+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:05:19.877148+0000","last_peered":"2024-09-10T16:05:19.877148+0000","last_clean":"2024-09-10T16:05:19.877148+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:05:19.877148+0000","last_undegraded":"2024-09-10T16:05:19.877148+0000","last_fullsized":"2024-09-10T16:05:19.877148+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'18080","reported_seq":22530,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:55.786761+0000","last_change":"2024-09-10T16:05:18.271152+0000","last_active":"2024-09-10T16:06:55.786761+0000","last_peered":"2024-09-10T16:06:55.786761+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:06:55.786761+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"43'8000","ondisk_log_start":"43'8000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12656640,"num_objects":3090,"num_object_clones":0,"num_object_copies":6180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2197,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3090,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15634,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2157,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":893}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'17756","reported_seq":17896,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:55.219348+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:06:55.219348+0000","last_peered":"2024-09-10T16:06:55.219348+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:06:55.219348+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"43'7700","ondisk_log_start":"43'7700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12623872,"num_objects":3082,"num_object_clones":0,"num_object_copies":6164,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3082,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14387,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1222}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877654+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:05:19.877654+0000","last_peered":"2024-09-10T16:05:19.877654+0000","last_clean":"2024-09-10T16:05:19.877654+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:05:19.877654+0000","last_undegraded":"2024-09-10T16:05:19.877654+0000","last_fullsized":"2024-09-10T16:05:19.877654+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'17572","reported_seq":26142,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:06:58.756072+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:06:58.756072+0000","last_peered":"2024-09-10T16:06:58.756072+0000","last_clean":"2024-09-10T16:06:58.756072+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:06:58.756072+0000","last_undegraded":"2024-09-10T16:06:58.756072+0000","last_fullsized":"2024-09-10T16:06:58.756072+0000","mapping_epoch":46,"log_start":"43'7500","ondisk_log_start":"43'7500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12394496,"num_objects":3026,"num_object_clones":0,"num_object_copies":6052,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3026,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13962,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":100,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.876043+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:05:19.876043+0000","last_peered":"2024-09-10T16:05:19.876043+0000","last_clean":"2024-09-10T16:05:19.876043+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:05:19.876043+0000","last_undegraded":"2024-09-10T16:05:19.876043+0000","last_fullsized":"2024-09-10T16:05:19.876043+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'17984","reported_seq":18132,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:55.173070+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:06:55.173070+0000","last_peered":"2024-09-10T16:06:55.173070+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:06:55.173070+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"43'7900","ondisk_log_start":"43'7900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12550144,"num_objects":3064,"num_object_clones":0,"num_object_copies":6128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3064,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14239,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1307}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877349+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:05:19.877349+0000","last_peered":"2024-09-10T16:05:19.877349+0000","last_clean":"2024-09-10T16:05:19.877349+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:05:19.877349+0000","last_undegraded":"2024-09-10T16:05:19.877349+0000","last_fullsized":"2024-09-10T16:05:19.877349+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877269+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:05:19.877269+0000","last_peered":"2024-09-10T16:05:19.877269+0000","last_clean":"2024-09-10T16:05:19.877269+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:05:19.877269+0000","last_undegraded":"2024-09-10T16:05:19.877269+0000","last_fullsized":"2024-09-10T16:05:19.877269+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'18046","reported_seq":18196,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:55.192900+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:06:55.192900+0000","last_peered":"2024-09-10T16:06:55.192900+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:06:55.192900+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"43'8000","ondisk_log_start":"43'8000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12911558,"num_objects":3154,"num_object_clones":0,"num_object_copies":6308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3154,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14209,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1169},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":202433270,"num_objects":49432,"num_object_clones":0,"num_object_copies":98864,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40754,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49432,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":242674,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10635,"num_bytes_recovered":33095680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":612208640,"data_stored":612066815,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161000,"ondisk_log_size":161000,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986283,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":579316,"kb_used_data":236636,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93792524,"statfs":{"total":96636764160,"available":96043544576,"internally_reserved":0,"allocated":242315264,"data_stored":242012580,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":350875306},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,1,1,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659229,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":505808,"kb_used_data":145088,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93866032,"statfs":{"total":96636764160,"available":96118816768,"internally_reserved":0,"allocated":148570112,"data_stored":148238060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":369356462},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,4,2,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182511,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":661500,"kb_used_data":218920,"kb_used_omap":4,"kb_used_meta":442555,"kb_avail":93710340,"statfs":{"total":96636764160,"available":95959388160,"internally_reserved":0,"allocated":224174080,"data_stored":223871391,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":453177002},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,2,2,1,4,7],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":222580736,"data_stored":222544606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":148074496,"data_stored":148009003,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":241553408,"data_stored":241513206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:07:00.795 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:07:01.342 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:07:01.344 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:07:01.366 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":332,"stamp":"2024-09-10T16:06:59.727947+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":202433289,"num_objects":49434,"num_object_clones":0,"num_object_copies":98868,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40754,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49434,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":242676,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10637,"num_bytes_recovered":33095699,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161003,"ondisk_log_size":161003,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1746624,"kb_used_data":600644,"kb_used_omap":12,"kb_used_meta":1145907,"kb_avail":281368896,"statfs":{"total":289910292480,"available":288121749504,"internally_reserved":0,"allocated":615059456,"data_stored":614122031,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1173408770},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,2,5,6,7,15],"upper_bound":128},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-20434944,"num_objects":-4989,"num_object_clones":0,"num_object_copies":-9978,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4989,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4989,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":237,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.007671"},"pg_stats":[{"pgid":"2.f","version":"47'17618","reported_seq":17697,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:55.219450+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:06:55.219450+0000","last_peered":"2024-09-10T16:06:55.219450+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:06:55.219450+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"43'7600","ondisk_log_start":"43'7600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12509184,"num_objects":3054,"num_object_clones":0,"num_object_copies":6108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3054,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17618,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1205}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'17755","reported_seq":17872,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:06:55.784537+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:06:55.784537+0000","last_peered":"2024-09-10T16:06:55.784537+0000","last_clean":"2024-09-10T16:06:55.784537+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:06:55.784537+0000","last_undegraded":"2024-09-10T16:06:55.784537+0000","last_fullsized":"2024-09-10T16:06:55.784537+0000","mapping_epoch":35,"log_start":"43'7700","ondisk_log_start":"43'7700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12653510,"num_objects":3091,"num_object_clones":0,"num_object_copies":6182,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3091,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":15860,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'17794","reported_seq":26478,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:06:58.651659+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:06:58.651659+0000","last_peered":"2024-09-10T16:06:58.651659+0000","last_clean":"2024-09-10T16:06:58.651659+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:06:58.651659+0000","last_undegraded":"2024-09-10T16:06:58.651659+0000","last_fullsized":"2024-09-10T16:06:58.651659+0000","mapping_epoch":44,"log_start":"43'7700","ondisk_log_start":"43'7700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12300312,"num_objects":3004,"num_object_clones":0,"num_object_copies":6008,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3004,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15798,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'17644","reported_seq":17729,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:06:55.658196+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:06:55.658196+0000","last_peered":"2024-09-10T16:06:55.658196+0000","last_clean":"2024-09-10T16:06:55.658196+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:06:55.658196+0000","last_undegraded":"2024-09-10T16:06:55.658196+0000","last_fullsized":"2024-09-10T16:06:55.658196+0000","mapping_epoch":35,"log_start":"43'7600","ondisk_log_start":"43'7600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12353536,"num_objects":3016,"num_object_clones":0,"num_object_copies":6032,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3016,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17644,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'17847","reported_seq":17981,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:06:55.195977+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:06:55.195977+0000","last_peered":"2024-09-10T16:06:55.195977+0000","last_clean":"2024-09-10T16:06:55.195977+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:06:55.195977+0000","last_undegraded":"2024-09-10T16:06:55.195977+0000","last_fullsized":"2024-09-10T16:06:55.195977+0000","mapping_epoch":34,"log_start":"43'7800","ondisk_log_start":"43'7800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12468707,"num_objects":3045,"num_object_clones":0,"num_object_copies":6090,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3045,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14558,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'17945","reported_seq":18088,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:55.168178+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:06:55.168178+0000","last_peered":"2024-09-10T16:06:55.168178+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:06:55.168178+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"43'7900","ondisk_log_start":"43'7900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12763619,"num_objects":3117,"num_object_clones":0,"num_object_copies":6234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3117,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14557,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1254},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'17892","reported_seq":18035,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:55.089578+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:06:55.089578+0000","last_peered":"2024-09-10T16:06:55.089578+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:06:55.089578+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"43'7800","ondisk_log_start":"43'7800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13067206,"num_objects":3192,"num_object_clones":0,"num_object_copies":6384,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3192,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":14391,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1045},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'18140","reported_seq":18276,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:06:55.181346+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:06:55.181346+0000","last_peered":"2024-09-10T16:06:55.181346+0000","last_clean":"2024-09-10T16:06:55.181346+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:06:55.181346+0000","last_undegraded":"2024-09-10T16:06:55.181346+0000","last_fullsized":"2024-09-10T16:06:55.181346+0000","mapping_epoch":34,"log_start":"43'8100","ondisk_log_start":"43'8100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13000704,"num_objects":3174,"num_object_clones":0,"num_object_copies":6348,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3174,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14987,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.876895+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:05:19.876895+0000","last_peered":"2024-09-10T16:05:19.876895+0000","last_clean":"2024-09-10T16:05:19.876895+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:05:19.876895+0000","last_undegraded":"2024-09-10T16:05:19.876895+0000","last_fullsized":"2024-09-10T16:05:19.876895+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'17280","reported_seq":17426,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:55.657484+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:06:55.657484+0000","last_peered":"2024-09-10T16:06:55.657484+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:06:55.657484+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"43'7200","ondisk_log_start":"43'7200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13013475,"num_objects":3178,"num_object_clones":0,"num_object_copies":6356,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3178,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15099,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-945}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":100,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.875849+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:05:19.875849+0000","last_peered":"2024-09-10T16:05:19.875849+0000","last_clean":"2024-09-10T16:05:19.875849+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:05:19.875849+0000","last_undegraded":"2024-09-10T16:05:19.875849+0000","last_fullsized":"2024-09-10T16:05:19.875849+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'17778","reported_seq":17914,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:55.140385+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:06:55.140385+0000","last_peered":"2024-09-10T16:06:55.140385+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:06:55.140385+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"43'7700","ondisk_log_start":"43'7700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12828672,"num_objects":3132,"num_object_clones":0,"num_object_copies":6264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3132,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14282,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1123}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":110,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.872023+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:05:19.872023+0000","last_peered":"2024-09-10T16:05:19.872023+0000","last_clean":"2024-09-10T16:05:19.872023+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:05:19.872023+0000","last_undegraded":"2024-09-10T16:05:19.872023+0000","last_fullsized":"2024-09-10T16:05:19.872023+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'17769","reported_seq":17910,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:55.784106+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:06:55.784106+0000","last_peered":"2024-09-10T16:06:55.784106+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:06:55.784106+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"43'7700","ondisk_log_start":"43'7700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12337635,"num_objects":3013,"num_object_clones":0,"num_object_copies":6026,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3013,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15449,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-1291}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877148+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:05:19.877148+0000","last_peered":"2024-09-10T16:05:19.877148+0000","last_clean":"2024-09-10T16:05:19.877148+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:05:19.877148+0000","last_undegraded":"2024-09-10T16:05:19.877148+0000","last_fullsized":"2024-09-10T16:05:19.877148+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'18080","reported_seq":22530,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:55.786761+0000","last_change":"2024-09-10T16:05:18.271152+0000","last_active":"2024-09-10T16:06:55.786761+0000","last_peered":"2024-09-10T16:06:55.786761+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:06:55.786761+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"43'8000","ondisk_log_start":"43'8000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12656640,"num_objects":3090,"num_object_clones":0,"num_object_copies":6180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2197,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3090,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15634,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2157,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":893}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'17756","reported_seq":17896,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:55.219348+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:06:55.219348+0000","last_peered":"2024-09-10T16:06:55.219348+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:06:55.219348+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"43'7700","ondisk_log_start":"43'7700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12623872,"num_objects":3082,"num_object_clones":0,"num_object_copies":6164,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3082,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14387,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1222}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877654+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:05:19.877654+0000","last_peered":"2024-09-10T16:05:19.877654+0000","last_clean":"2024-09-10T16:05:19.877654+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:05:19.877654+0000","last_undegraded":"2024-09-10T16:05:19.877654+0000","last_fullsized":"2024-09-10T16:05:19.877654+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'17572","reported_seq":26142,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:06:58.756072+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:06:58.756072+0000","last_peered":"2024-09-10T16:06:58.756072+0000","last_clean":"2024-09-10T16:06:58.756072+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:06:58.756072+0000","last_undegraded":"2024-09-10T16:06:58.756072+0000","last_fullsized":"2024-09-10T16:06:58.756072+0000","mapping_epoch":46,"log_start":"43'7500","ondisk_log_start":"43'7500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12394496,"num_objects":3026,"num_object_clones":0,"num_object_copies":6052,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3026,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13962,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":100,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.876043+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:05:19.876043+0000","last_peered":"2024-09-10T16:05:19.876043+0000","last_clean":"2024-09-10T16:05:19.876043+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:05:19.876043+0000","last_undegraded":"2024-09-10T16:05:19.876043+0000","last_fullsized":"2024-09-10T16:05:19.876043+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'17984","reported_seq":18132,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:55.173070+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:06:55.173070+0000","last_peered":"2024-09-10T16:06:55.173070+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:06:55.173070+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"43'7900","ondisk_log_start":"43'7900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12550144,"num_objects":3064,"num_object_clones":0,"num_object_copies":6128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3064,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14239,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1307}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877349+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:05:19.877349+0000","last_peered":"2024-09-10T16:05:19.877349+0000","last_clean":"2024-09-10T16:05:19.877349+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:05:19.877349+0000","last_undegraded":"2024-09-10T16:05:19.877349+0000","last_fullsized":"2024-09-10T16:05:19.877349+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877269+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:05:19.877269+0000","last_peered":"2024-09-10T16:05:19.877269+0000","last_clean":"2024-09-10T16:05:19.877269+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:05:19.877269+0000","last_undegraded":"2024-09-10T16:05:19.877269+0000","last_fullsized":"2024-09-10T16:05:19.877269+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'18046","reported_seq":18196,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:06:55.192900+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:06:55.192900+0000","last_peered":"2024-09-10T16:06:55.192900+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:06:55.192900+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"43'8000","ondisk_log_start":"43'8000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12911558,"num_objects":3154,"num_object_clones":0,"num_object_copies":6308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3154,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14209,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1169},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":202433270,"num_objects":49432,"num_object_clones":0,"num_object_copies":98864,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40754,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49432,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":242674,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10635,"num_bytes_recovered":33095680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":612208640,"data_stored":612066815,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161000,"ondisk_log_size":161000,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986283,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":579316,"kb_used_data":236636,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93792524,"statfs":{"total":96636764160,"available":96043544576,"internally_reserved":0,"allocated":242315264,"data_stored":242012580,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":350875306},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,1,1,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659229,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":505808,"kb_used_data":145088,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93866032,"statfs":{"total":96636764160,"available":96118816768,"internally_reserved":0,"allocated":148570112,"data_stored":148238060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":369356462},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,4,2,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182511,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":661500,"kb_used_data":218920,"kb_used_omap":4,"kb_used_meta":442555,"kb_avail":93710340,"statfs":{"total":96636764160,"available":95959388160,"internally_reserved":0,"allocated":224174080,"data_stored":223871391,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":453177002},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,2,2,1,4,7],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":222580736,"data_stored":222544606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":148074496,"data_stored":148009003,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":241553408,"data_stored":241513206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:07:01.367 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-10T16:07:01.367 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "47'17618", 'reported_seq': 17697, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:06:55.219450+0000', 'last_change': '2024-09-10T15:59:16.281558+0000', 'last_active': '2024-09-10T16:06:55.219450+0000', 'last_peered': '2024-09-10T16:06:55.219450+0000', 'last_clean': '2024-09-10T15:59:14.132456+0000', 'last_became_active': '2024-09-10T15:59:16.274407+0000', 'last_became_peered': '2024-09-10T15:59:16.274407+0000', 'last_unstale': '2024-09-10T16:06:55.219450+0000', 'last_undegraded': '2024-09-10T15:59:15.851279+0000', 'last_fullsized': '2024-09-10T15:59:15.774356+0000', 'mapping_epoch': 42, 'log_start': "43'7600", 'ondisk_log_start': "43'7600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'objects_scrubbed': 0, 'log_size': 10018, 'log_dups_size': 0, 'ondisk_log_size': 10018, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 12509184, 'num_objects': 3054, 'num_object_clones': 0, 'num_object_copies': 6108, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4259, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3054, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17618, 'num_write_kb': 41344, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1205}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:07:01.367 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-10T16:07:01.367 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "47'17945", 'reported_seq': 18088, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:06:55.168178+0000', 'last_change': '2024-09-10T15:59:16.308264+0000', 'last_active': '2024-09-10T16:06:55.168178+0000', 'last_peered': '2024-09-10T16:06:55.168178+0000', 'last_clean': '2024-09-10T15:59:14.111551+0000', 'last_became_active': '2024-09-10T15:59:16.300040+0000', 'last_became_peered': '2024-09-10T15:59:16.300040+0000', 'last_unstale': '2024-09-10T16:06:55.168178+0000', 'last_undegraded': '2024-09-10T15:59:15.878234+0000', 'last_fullsized': '2024-09-10T15:59:15.769328+0000', 'mapping_epoch': 42, 'log_start': "43'7900", 'ondisk_log_start': "43'7900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3442", 'last_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'objects_scrubbed': 3384, 'log_size': 10045, 'log_dups_size': 0, 'ondisk_log_size': 10045, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1292, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 12763619, 'num_objects': 3117, 'num_object_clones': 0, 'num_object_copies': 6234, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4372, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3117, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 14557, 'num_write_kb': 28580, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1254}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:07:01.367 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-10T16:07:01.367 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "47'17892", 'reported_seq': 18035, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:06:55.089578+0000', 'last_change': '2024-09-10T15:59:16.331899+0000', 'last_active': '2024-09-10T16:06:55.089578+0000', 'last_peered': '2024-09-10T16:06:55.089578+0000', 'last_clean': '2024-09-10T15:59:14.139436+0000', 'last_became_active': '2024-09-10T15:59:16.318715+0000', 'last_became_peered': '2024-09-10T15:59:16.318715+0000', 'last_unstale': '2024-09-10T16:06:55.089578+0000', 'last_undegraded': '2024-09-10T15:59:15.877524+0000', 'last_fullsized': '2024-09-10T15:59:15.769042+0000', 'mapping_epoch': 42, 'log_start': "43'7800", 'ondisk_log_start': "43'7800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3537", 'last_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'objects_scrubbed': 3497, 'log_size': 10092, 'log_dups_size': 0, 'ondisk_log_size': 10092, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1224, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 13067206, 'num_objects': 3192, 'num_object_clones': 0, 'num_object_copies': 6384, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4237, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3192, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 14391, 'num_write_kb': 28172, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1045}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:07:01.367 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-10T16:07:01.368 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "47'17280", 'reported_seq': 17426, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:06:55.657484+0000', 'last_change': '2024-09-10T15:59:16.175680+0000', 'last_active': '2024-09-10T16:06:55.657484+0000', 'last_peered': '2024-09-10T16:06:55.657484+0000', 'last_clean': '2024-09-10T15:59:14.088187+0000', 'last_became_active': '2024-09-10T15:59:16.165810+0000', 'last_became_peered': '2024-09-10T15:59:16.165810+0000', 'last_unstale': '2024-09-10T16:06:55.657484+0000', 'last_undegraded': '2024-09-10T15:59:15.822098+0000', 'last_fullsized': '2024-09-10T15:59:15.769123+0000', 'mapping_epoch': 42, 'log_start': "43'7200", 'ondisk_log_start': "43'7200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2218", 'last_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'objects_scrubbed': 2175, 'log_size': 10080, 'log_dups_size': 0, 'ondisk_log_size': 10080, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 825, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 13013475, 'num_objects': 3178, 'num_object_clones': 0, 'num_object_copies': 6356, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4124, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3178, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 15099, 'num_write_kb': 32204, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': -945}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:07:01.368 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-10T16:07:01.368 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "47'17778", 'reported_seq': 17914, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:06:55.140385+0000', 'last_change': '2024-09-10T15:59:16.277742+0000', 'last_active': '2024-09-10T16:06:55.140385+0000', 'last_peered': '2024-09-10T16:06:55.140385+0000', 'last_clean': '2024-09-10T15:59:14.129081+0000', 'last_became_active': '2024-09-10T15:59:16.269835+0000', 'last_became_peered': '2024-09-10T15:59:16.269835+0000', 'last_unstale': '2024-09-10T16:06:55.140385+0000', 'last_undegraded': '2024-09-10T15:59:15.860857+0000', 'last_fullsized': '2024-09-10T15:59:15.771410+0000', 'mapping_epoch': 42, 'log_start': "43'7700", 'ondisk_log_start': "43'7700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3557", 'last_scrub_stamp': '2024-09-10T15:59:02.493678+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:02.493678+0000', 'objects_scrubbed': 3488, 'log_size': 10078, 'log_dups_size': 0, 'ondisk_log_size': 10078, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1472, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 12828672, 'num_objects': 3132, 'num_object_clones': 0, 'num_object_copies': 6264, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4257, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3132, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14282, 'num_write_kb': 27852, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1123}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:07:01.368 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-10T16:07:01.368 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "47'17769", 'reported_seq': 17910, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:06:55.784106+0000', 'last_change': '2024-09-10T15:59:16.284311+0000', 'last_active': '2024-09-10T16:06:55.784106+0000', 'last_peered': '2024-09-10T16:06:55.784106+0000', 'last_clean': '2024-09-10T15:59:14.116041+0000', 'last_became_active': '2024-09-10T15:59:16.274543+0000', 'last_became_peered': '2024-09-10T15:59:16.274543+0000', 'last_unstale': '2024-09-10T16:06:55.784106+0000', 'last_undegraded': '2024-09-10T15:59:15.812484+0000', 'last_fullsized': '2024-09-10T15:59:15.770885+0000', 'mapping_epoch': 42, 'log_start': "43'7700", 'ondisk_log_start': "43'7700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2356", 'last_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'objects_scrubbed': 2316, 'log_size': 10069, 'log_dups_size': 0, 'ondisk_log_size': 10069, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 783, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 12337635, 'num_objects': 3013, 'num_object_clones': 0, 'num_object_copies': 6026, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3013, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 15449, 'num_write_kb': 32292, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': -1291}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:07:01.368 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-10T16:07:01.368 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "47'18080", 'reported_seq': 22530, 'reported_epoch': 47, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:06:55.786761+0000', 'last_change': '2024-09-10T16:05:18.271152+0000', 'last_active': '2024-09-10T16:06:55.786761+0000', 'last_peered': '2024-09-10T16:06:55.786761+0000', 'last_clean': '2024-09-10T15:59:14.110654+0000', 'last_became_active': '2024-09-10T15:59:16.164645+0000', 'last_became_peered': '2024-09-10T15:59:16.164645+0000', 'last_unstale': '2024-09-10T16:06:55.786761+0000', 'last_undegraded': '2024-09-10T15:59:15.803593+0000', 'last_fullsized': '2024-09-10T15:59:15.769083+0000', 'mapping_epoch': 42, 'log_start': "43'8000", 'ondisk_log_start': "43'8000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2486", 'last_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'objects_scrubbed': 2440, 'log_size': 10080, 'log_dups_size': 0, 'ondisk_log_size': 10080, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 792, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 12656640, 'num_objects': 3090, 'num_object_clones': 0, 'num_object_copies': 6180, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2197, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3090, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15634, 'num_write_kb': 32568, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2157, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 893}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:07:01.368 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-10T16:07:01.368 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "47'17756", 'reported_seq': 17896, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:06:55.219348+0000', 'last_change': '2024-09-10T15:59:16.315641+0000', 'last_active': '2024-09-10T16:06:55.219348+0000', 'last_peered': '2024-09-10T16:06:55.219348+0000', 'last_clean': '2024-09-10T15:59:14.117001+0000', 'last_became_active': '2024-09-10T15:59:16.302245+0000', 'last_became_peered': '2024-09-10T15:59:16.302245+0000', 'last_unstale': '2024-09-10T16:06:55.219348+0000', 'last_undegraded': '2024-09-10T15:59:15.845012+0000', 'last_fullsized': '2024-09-10T15:59:15.770337+0000', 'mapping_epoch': 42, 'log_start': "43'7700", 'ondisk_log_start': "43'7700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3404", 'last_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'objects_scrubbed': 3367, 'log_size': 10056, 'log_dups_size': 0, 'ondisk_log_size': 10056, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 983, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 12623872, 'num_objects': 3082, 'num_object_clones': 0, 'num_object_copies': 6164, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3082, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14387, 'num_write_kb': 28204, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1222}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:07:01.368 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-10T16:07:01.369 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "47'17984", 'reported_seq': 18132, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:06:55.173070+0000', 'last_change': '2024-09-10T15:59:16.311291+0000', 'last_active': '2024-09-10T16:06:55.173070+0000', 'last_peered': '2024-09-10T16:06:55.173070+0000', 'last_clean': '2024-09-10T15:59:14.116363+0000', 'last_became_active': '2024-09-10T15:59:16.301244+0000', 'last_became_peered': '2024-09-10T15:59:16.301244+0000', 'last_unstale': '2024-09-10T16:06:55.173070+0000', 'last_undegraded': '2024-09-10T15:59:15.883434+0000', 'last_fullsized': '2024-09-10T15:59:15.772756+0000', 'mapping_epoch': 42, 'log_start': "43'7900", 'ondisk_log_start': "43'7900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3789", 'last_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'objects_scrubbed': 3741, 'log_size': 10084, 'log_dups_size': 0, 'ondisk_log_size': 10084, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1440, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 12550144, 'num_objects': 3064, 'num_object_clones': 0, 'num_object_copies': 6128, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4373, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3064, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14239, 'num_write_kb': 27124, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1307}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:07:01.369 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-10T16:07:01.369 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "47'18046", 'reported_seq': 18196, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:06:55.192900+0000', 'last_change': '2024-09-10T15:59:16.313409+0000', 'last_active': '2024-09-10T16:06:55.192900+0000', 'last_peered': '2024-09-10T16:06:55.192900+0000', 'last_clean': '2024-09-10T15:59:14.129780+0000', 'last_became_active': '2024-09-10T15:59:16.300932+0000', 'last_became_peered': '2024-09-10T15:59:16.300932+0000', 'last_unstale': '2024-09-10T16:06:55.192900+0000', 'last_undegraded': '2024-09-10T15:59:15.887823+0000', 'last_fullsized': '2024-09-10T15:59:15.768513+0000', 'mapping_epoch': 42, 'log_start': "43'8000", 'ondisk_log_start': "43'8000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3880", 'last_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'objects_scrubbed': 3827, 'log_size': 10046, 'log_dups_size': 0, 'ondisk_log_size': 10046, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1397, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 12911558, 'num_objects': 3154, 'num_object_clones': 0, 'num_object_copies': 6308, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4325, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3154, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14209, 'num_write_kb': 27072, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1169}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:07:01.370 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T16:07:02.066 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:07:02.066 INFO:teuthology.orchestra.run.smithi115.stdout:{"fsid":"7effee29-df05-4351-9741-ee203564ed81","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 40625/94834 objects degraded (42.838%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":586,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":47,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983954,"num_in_osds":3,"osd_in_since":1725983947,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":47417,"data_bytes":194171657,"bytes_used":1788280832,"bytes_avail":288122011648,"bytes_total":289910292480,"degraded_objects":40625,"degraded_total":94834,"degraded_ratio":0.42838011683573396,"recovering_objects_per_sec":30,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":366,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":560},"fsmap":{"epoch":1,"btime":"2024-09-10T15:57:14:967120+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T16:00:13.560776+0000","services":{}},"progress_events":{}} 2024-09-10T16:07:02.087 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-10T16:07:02.087 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:07:02.645 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:07:02.645 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:07:02.676 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":333,"stamp":"2024-09-10T16:07:01.728665+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":194171657,"num_objects":47417,"num_object_clones":0,"num_object_copies":94834,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40625,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47417,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":244693,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10766,"num_bytes_recovered":33095699,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160920,"ondisk_log_size":160920,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1746368,"kb_used_data":587920,"kb_used_omap":12,"kb_used_meta":1158387,"kb_avail":281369152,"statfs":{"total":289910292480,"available":288122011648,"internally_reserved":0,"allocated":602030080,"data_stored":601092655,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1186188290},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,3,8,8,8,14],"upper_bound":128},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-27549696,"num_objects":-6726,"num_object_clones":0,"num_object_copies":-13452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-366,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-6726,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6726,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":366,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.007917"},"pg_stats":[{"pgid":"2.f","version":"47'17765","reported_seq":17845,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:00.138180+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:07:00.138180+0000","last_peered":"2024-09-10T16:07:00.138180+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:07:00.138180+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"43'7700","ondisk_log_start":"43'7700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11907072,"num_objects":2907,"num_object_clones":0,"num_object_copies":5814,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2907,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17765,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1352}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'17885","reported_seq":18003,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:07:00.787957+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:07:00.787957+0000","last_peered":"2024-09-10T16:07:00.787957+0000","last_clean":"2024-09-10T16:07:00.787957+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:07:00.787957+0000","last_undegraded":"2024-09-10T16:07:00.787957+0000","last_fullsized":"2024-09-10T16:07:00.787957+0000","mapping_epoch":35,"log_start":"43'7800","ondisk_log_start":"43'7800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12121030,"num_objects":2961,"num_object_clones":0,"num_object_copies":5922,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2961,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":15990,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'17794","reported_seq":26478,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:06:58.651659+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:06:58.651659+0000","last_peered":"2024-09-10T16:06:58.651659+0000","last_clean":"2024-09-10T16:06:58.651659+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:06:58.651659+0000","last_undegraded":"2024-09-10T16:06:58.651659+0000","last_fullsized":"2024-09-10T16:06:58.651659+0000","mapping_epoch":44,"log_start":"43'7700","ondisk_log_start":"43'7700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12300312,"num_objects":3004,"num_object_clones":0,"num_object_copies":6008,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3004,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15798,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'17779","reported_seq":17865,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:07:00.459570+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:07:00.459570+0000","last_peered":"2024-09-10T16:07:00.459570+0000","last_clean":"2024-09-10T16:07:00.459570+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:07:00.459570+0000","last_undegraded":"2024-09-10T16:07:00.459570+0000","last_fullsized":"2024-09-10T16:07:00.459570+0000","mapping_epoch":35,"log_start":"43'7700","ondisk_log_start":"43'7700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11800576,"num_objects":2881,"num_object_clones":0,"num_object_copies":5762,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2881,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17779,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'18000","reported_seq":18135,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:07:00.255605+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:07:00.255605+0000","last_peered":"2024-09-10T16:07:00.255605+0000","last_clean":"2024-09-10T16:07:00.255605+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:07:00.255605+0000","last_undegraded":"2024-09-10T16:07:00.255605+0000","last_fullsized":"2024-09-10T16:07:00.255605+0000","mapping_epoch":34,"log_start":"43'7900","ondisk_log_start":"43'7900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11842019,"num_objects":2892,"num_object_clones":0,"num_object_copies":5784,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2892,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14711,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'18095","reported_seq":18238,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:00.236078+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:07:00.236078+0000","last_peered":"2024-09-10T16:07:00.236078+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:07:00.236078+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"43'8000","ondisk_log_start":"43'8000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12149219,"num_objects":2967,"num_object_clones":0,"num_object_copies":5934,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2967,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14707,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1404},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'18054","reported_seq":18198,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:00.257308+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:07:00.257308+0000","last_peered":"2024-09-10T16:07:00.257308+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:07:00.257308+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"43'8000","ondisk_log_start":"43'8000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12403654,"num_objects":3030,"num_object_clones":0,"num_object_copies":6060,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3030,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":14553,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1207},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'18284","reported_seq":18420,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:07:00.206823+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:07:00.206823+0000","last_peered":"2024-09-10T16:07:00.206823+0000","last_clean":"2024-09-10T16:07:00.206823+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:07:00.206823+0000","last_undegraded":"2024-09-10T16:07:00.206823+0000","last_fullsized":"2024-09-10T16:07:00.206823+0000","mapping_epoch":34,"log_start":"43'8200","ondisk_log_start":"43'8200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12410880,"num_objects":3030,"num_object_clones":0,"num_object_copies":6060,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3030,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15131,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.876895+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:05:19.876895+0000","last_peered":"2024-09-10T16:05:19.876895+0000","last_clean":"2024-09-10T16:05:19.876895+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:05:19.876895+0000","last_undegraded":"2024-09-10T16:05:19.876895+0000","last_fullsized":"2024-09-10T16:05:19.876895+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'17403","reported_seq":17549,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:00.697876+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:07:00.697876+0000","last_peered":"2024-09-10T16:07:00.697876+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:07:00.697876+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"43'7400","ondisk_log_start":"43'7400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12509667,"num_objects":3055,"num_object_clones":0,"num_object_copies":6110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3055,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15222,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-1068}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":100,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.875849+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:05:19.875849+0000","last_peered":"2024-09-10T16:05:19.875849+0000","last_clean":"2024-09-10T16:05:19.875849+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:05:19.875849+0000","last_undegraded":"2024-09-10T16:05:19.875849+0000","last_fullsized":"2024-09-10T16:05:19.875849+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'17915","reported_seq":18051,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:00.162135+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:07:00.162135+0000","last_peered":"2024-09-10T16:07:00.162135+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:07:00.162135+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"43'7900","ondisk_log_start":"43'7900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12267520,"num_objects":2995,"num_object_clones":0,"num_object_copies":5990,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2995,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14419,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1260}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":110,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.872023+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:05:19.872023+0000","last_peered":"2024-09-10T16:05:19.872023+0000","last_clean":"2024-09-10T16:05:19.872023+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:05:19.872023+0000","last_undegraded":"2024-09-10T16:05:19.872023+0000","last_fullsized":"2024-09-10T16:05:19.872023+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'17909","reported_seq":18050,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:00.735254+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:07:00.735254+0000","last_peered":"2024-09-10T16:07:00.735254+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:07:00.735254+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"43'7900","ondisk_log_start":"43'7900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11764195,"num_objects":2873,"num_object_clones":0,"num_object_copies":5746,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2873,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15589,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-1431}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877148+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:05:19.877148+0000","last_peered":"2024-09-10T16:05:19.877148+0000","last_clean":"2024-09-10T16:05:19.877148+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:05:19.877148+0000","last_undegraded":"2024-09-10T16:05:19.877148+0000","last_fullsized":"2024-09-10T16:05:19.877148+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'18221","reported_seq":22929,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:00.773977+0000","last_change":"2024-09-10T16:05:18.271152+0000","last_active":"2024-09-10T16:07:00.773977+0000","last_peered":"2024-09-10T16:07:00.773977+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:07:00.773977+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"43'8200","ondisk_log_start":"43'8200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12079104,"num_objects":2949,"num_object_clones":0,"num_object_copies":5898,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2068,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2949,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15775,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2286,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":881}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'17903","reported_seq":18042,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:00.246677+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:07:00.246677+0000","last_peered":"2024-09-10T16:07:00.246677+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:07:00.246677+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"43'7900","ondisk_log_start":"43'7900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12021760,"num_objects":2935,"num_object_clones":0,"num_object_copies":5870,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2935,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14534,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1369}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877654+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:05:19.877654+0000","last_peered":"2024-09-10T16:05:19.877654+0000","last_clean":"2024-09-10T16:05:19.877654+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:05:19.877654+0000","last_undegraded":"2024-09-10T16:05:19.877654+0000","last_fullsized":"2024-09-10T16:05:19.877654+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'17572","reported_seq":26142,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:06:58.756072+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:06:58.756072+0000","last_peered":"2024-09-10T16:06:58.756072+0000","last_clean":"2024-09-10T16:06:58.756072+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:06:58.756072+0000","last_undegraded":"2024-09-10T16:06:58.756072+0000","last_fullsized":"2024-09-10T16:06:58.756072+0000","mapping_epoch":46,"log_start":"43'7500","ondisk_log_start":"43'7500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12394496,"num_objects":3026,"num_object_clones":0,"num_object_copies":6052,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3026,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13962,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":100,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.876043+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:05:19.876043+0000","last_peered":"2024-09-10T16:05:19.876043+0000","last_clean":"2024-09-10T16:05:19.876043+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:05:19.876043+0000","last_undegraded":"2024-09-10T16:05:19.876043+0000","last_fullsized":"2024-09-10T16:05:19.876043+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'18148","reported_seq":18295,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:00.213912+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:07:00.213912+0000","last_peered":"2024-09-10T16:07:00.213912+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:07:00.213912+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"43'8100","ondisk_log_start":"43'8100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11878400,"num_objects":2900,"num_object_clones":0,"num_object_copies":5800,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2900,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14403,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1471}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877349+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:05:19.877349+0000","last_peered":"2024-09-10T16:05:19.877349+0000","last_clean":"2024-09-10T16:05:19.877349+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:05:19.877349+0000","last_undegraded":"2024-09-10T16:05:19.877349+0000","last_fullsized":"2024-09-10T16:05:19.877349+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877269+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:05:19.877269+0000","last_peered":"2024-09-10T16:05:19.877269+0000","last_clean":"2024-09-10T16:05:19.877269+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:05:19.877269+0000","last_undegraded":"2024-09-10T16:05:19.877269+0000","last_fullsized":"2024-09-10T16:05:19.877269+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'18190","reported_seq":18340,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:00.171817+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:07:00.171817+0000","last_peered":"2024-09-10T16:07:00.171817+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:07:00.171817+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"43'8100","ondisk_log_start":"43'8100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12321734,"num_objects":3010,"num_object_clones":0,"num_object_copies":6020,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3010,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14353,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1313},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":194171638,"num_objects":47415,"num_object_clones":0,"num_object_copies":94830,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40625,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47415,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":244691,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10764,"num_bytes_recovered":33095680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":600322048,"data_stored":600180223,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160917,"ondisk_log_size":160917,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986284,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":579056,"kb_used_data":232224,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93792784,"statfs":{"total":96636764160,"available":96043810816,"internally_reserved":0,"allocated":237797376,"data_stored":237494692,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":355135146},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,7,1,1,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659229,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":505808,"kb_used_data":145088,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93866032,"statfs":{"total":96636764160,"available":96118816768,"internally_reserved":0,"allocated":148570112,"data_stored":148238060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":369356462},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,4,2,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182512,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":661504,"kb_used_data":210608,"kb_used_omap":4,"kb_used_meta":450875,"kb_avail":93710336,"statfs":{"total":96636764160,"available":95959384064,"internally_reserved":0,"allocated":215662592,"data_stored":215359903,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":461696682},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,3,0,3,5,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":215212032,"data_stored":215175902,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":148074496,"data_stored":148009003,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":237035520,"data_stored":236995318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:07:05.679 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:07:06.298 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:07:06.300 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:07:06.330 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":335,"stamp":"2024-09-10T16:07:05.730125+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":187368201,"num_objects":45756,"num_object_clones":0,"num_object_copies":91512,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40625,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45756,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":246354,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10766,"num_bytes_recovered":33095699,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160781,"ondisk_log_size":160781,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1740656,"kb_used_data":573888,"kb_used_omap":12,"kb_used_meta":1166707,"kb_avail":281374864,"statfs":{"total":289910292480,"available":288127860736,"internally_reserved":0,"allocated":587661312,"data_stored":586723887,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1194707970},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,1,3,9,4,12,17],"upper_bound":128},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-24715264,"num_objects":-6034,"num_object_clones":0,"num_object_copies":-12068,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-255,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-6034,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6034,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":255,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004390"},"pg_stats":[{"pgid":"2.f","version":"47'17935","reported_seq":18014,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:05.320638+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:07:05.320638+0000","last_peered":"2024-09-10T16:07:05.320638+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:07:05.320638+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"43'7900","ondisk_log_start":"43'7900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11210752,"num_objects":2737,"num_object_clones":0,"num_object_copies":5474,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2737,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17935,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1522}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'17885","reported_seq":18003,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:07:00.787957+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:07:00.787957+0000","last_peered":"2024-09-10T16:07:00.787957+0000","last_clean":"2024-09-10T16:07:00.787957+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:07:00.787957+0000","last_undegraded":"2024-09-10T16:07:00.787957+0000","last_fullsized":"2024-09-10T16:07:00.787957+0000","mapping_epoch":35,"log_start":"43'7800","ondisk_log_start":"43'7800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12121030,"num_objects":2961,"num_object_clones":0,"num_object_copies":5922,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2961,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":15990,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'17947","reported_seq":26631,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:07:03.824630+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:07:03.824630+0000","last_peered":"2024-09-10T16:07:03.824630+0000","last_clean":"2024-09-10T16:07:03.824630+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:07:03.824630+0000","last_undegraded":"2024-09-10T16:07:03.824630+0000","last_fullsized":"2024-09-10T16:07:03.824630+0000","mapping_epoch":44,"log_start":"43'7900","ondisk_log_start":"43'7900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11673624,"num_objects":2851,"num_object_clones":0,"num_object_copies":5702,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2851,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15951,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'17779","reported_seq":17865,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:07:00.459570+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:07:00.459570+0000","last_peered":"2024-09-10T16:07:00.459570+0000","last_clean":"2024-09-10T16:07:00.459570+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:07:00.459570+0000","last_undegraded":"2024-09-10T16:07:00.459570+0000","last_fullsized":"2024-09-10T16:07:00.459570+0000","mapping_epoch":35,"log_start":"43'7700","ondisk_log_start":"43'7700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11800576,"num_objects":2881,"num_object_clones":0,"num_object_copies":5762,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2881,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17779,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'18140","reported_seq":18276,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:07:05.186978+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:07:05.186978+0000","last_peered":"2024-09-10T16:07:05.186978+0000","last_clean":"2024-09-10T16:07:05.186978+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:07:05.186978+0000","last_undegraded":"2024-09-10T16:07:05.186978+0000","last_fullsized":"2024-09-10T16:07:05.186978+0000","mapping_epoch":34,"log_start":"43'8100","ondisk_log_start":"43'8100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11268579,"num_objects":2752,"num_object_clones":0,"num_object_copies":5504,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2752,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14851,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'18246","reported_seq":18386,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:05.264786+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:07:05.264786+0000","last_peered":"2024-09-10T16:07:05.264786+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:07:05.264786+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"43'8200","ondisk_log_start":"43'8200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11530723,"num_objects":2816,"num_object_clones":0,"num_object_copies":5632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2816,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14858,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1555},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'18200","reported_seq":18344,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:05.264764+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:07:05.264764+0000","last_peered":"2024-09-10T16:07:05.264764+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:07:05.264764+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"43'8100","ondisk_log_start":"43'8100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11805638,"num_objects":2884,"num_object_clones":0,"num_object_copies":5768,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2884,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":14699,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1353},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'18441","reported_seq":18577,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:07:05.232274+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:07:05.232274+0000","last_peered":"2024-09-10T16:07:05.232274+0000","last_clean":"2024-09-10T16:07:05.232274+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:07:05.232274+0000","last_undegraded":"2024-09-10T16:07:05.232274+0000","last_fullsized":"2024-09-10T16:07:05.232274+0000","mapping_epoch":34,"log_start":"43'8400","ondisk_log_start":"43'8400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11767808,"num_objects":2873,"num_object_clones":0,"num_object_copies":5746,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2873,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15288,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.876895+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:05:19.876895+0000","last_peered":"2024-09-10T16:05:19.876895+0000","last_clean":"2024-09-10T16:05:19.876895+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:05:19.876895+0000","last_undegraded":"2024-09-10T16:05:19.876895+0000","last_fullsized":"2024-09-10T16:05:19.876895+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'17403","reported_seq":17549,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:00.697876+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:07:00.697876+0000","last_peered":"2024-09-10T16:07:00.697876+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:07:00.697876+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"43'7400","ondisk_log_start":"43'7400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12509667,"num_objects":3055,"num_object_clones":0,"num_object_copies":6110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3055,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15222,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-1068}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":100,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.875849+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:05:19.875849+0000","last_peered":"2024-09-10T16:05:19.875849+0000","last_clean":"2024-09-10T16:05:19.875849+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:05:19.875849+0000","last_undegraded":"2024-09-10T16:05:19.875849+0000","last_fullsized":"2024-09-10T16:05:19.875849+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'18069","reported_seq":18204,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:05.301588+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:07:05.301588+0000","last_peered":"2024-09-10T16:07:05.301588+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:07:05.301588+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"43'8000","ondisk_log_start":"43'8000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11636736,"num_objects":2841,"num_object_clones":0,"num_object_copies":5682,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2841,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14573,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1414}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":110,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.872023+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:05:19.872023+0000","last_peered":"2024-09-10T16:05:19.872023+0000","last_clean":"2024-09-10T16:05:19.872023+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:05:19.872023+0000","last_undegraded":"2024-09-10T16:05:19.872023+0000","last_fullsized":"2024-09-10T16:05:19.872023+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'17909","reported_seq":18050,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:00.735254+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:07:00.735254+0000","last_peered":"2024-09-10T16:07:00.735254+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:07:00.735254+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"43'7900","ondisk_log_start":"43'7900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11764195,"num_objects":2873,"num_object_clones":0,"num_object_copies":5746,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2873,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15589,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-1431}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877148+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:05:19.877148+0000","last_peered":"2024-09-10T16:05:19.877148+0000","last_clean":"2024-09-10T16:05:19.877148+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:05:19.877148+0000","last_undegraded":"2024-09-10T16:05:19.877148+0000","last_fullsized":"2024-09-10T16:05:19.877148+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'18221","reported_seq":22929,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:00.773977+0000","last_change":"2024-09-10T16:05:18.271152+0000","last_active":"2024-09-10T16:07:00.773977+0000","last_peered":"2024-09-10T16:07:00.773977+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:07:00.773977+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"43'8200","ondisk_log_start":"43'8200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12079104,"num_objects":2949,"num_object_clones":0,"num_object_copies":5898,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2068,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2949,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15775,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2286,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":881}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'18045","reported_seq":18186,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:05.194117+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:07:05.194117+0000","last_peered":"2024-09-10T16:07:05.194117+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:07:05.194117+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"43'8000","ondisk_log_start":"43'8000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11440128,"num_objects":2793,"num_object_clones":0,"num_object_copies":5586,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2793,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14676,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1511}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877654+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:05:19.877654+0000","last_peered":"2024-09-10T16:05:19.877654+0000","last_clean":"2024-09-10T16:05:19.877654+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:05:19.877654+0000","last_undegraded":"2024-09-10T16:05:19.877654+0000","last_fullsized":"2024-09-10T16:05:19.877654+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'17730","reported_seq":26301,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:07:03.815350+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:07:03.815350+0000","last_peered":"2024-09-10T16:07:03.815350+0000","last_clean":"2024-09-10T16:07:03.815350+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:07:03.815350+0000","last_undegraded":"2024-09-10T16:07:03.815350+0000","last_fullsized":"2024-09-10T16:07:03.815350+0000","mapping_epoch":46,"log_start":"43'7700","ondisk_log_start":"43'7700","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11747328,"num_objects":2868,"num_object_clones":0,"num_object_copies":5736,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2868,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14120,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":100,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.876043+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:05:19.876043+0000","last_peered":"2024-09-10T16:05:19.876043+0000","last_clean":"2024-09-10T16:05:19.876043+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:05:19.876043+0000","last_undegraded":"2024-09-10T16:05:19.876043+0000","last_fullsized":"2024-09-10T16:05:19.876043+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'18293","reported_seq":18441,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:05.286824+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:07:05.286824+0000","last_peered":"2024-09-10T16:07:05.286824+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:07:05.286824+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"43'8200","ondisk_log_start":"43'8200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11284480,"num_objects":2755,"num_object_clones":0,"num_object_copies":5510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2755,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14548,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1616}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877349+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:05:19.877349+0000","last_peered":"2024-09-10T16:05:19.877349+0000","last_clean":"2024-09-10T16:05:19.877349+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:05:19.877349+0000","last_undegraded":"2024-09-10T16:05:19.877349+0000","last_fullsized":"2024-09-10T16:05:19.877349+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877269+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:05:19.877269+0000","last_peered":"2024-09-10T16:05:19.877269+0000","last_clean":"2024-09-10T16:05:19.877269+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:05:19.877269+0000","last_undegraded":"2024-09-10T16:05:19.877269+0000","last_fullsized":"2024-09-10T16:05:19.877269+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'18335","reported_seq":18485,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:05.188415+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:07:05.188415+0000","last_peered":"2024-09-10T16:07:05.188415+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:07:05.188415+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"43'8300","ondisk_log_start":"43'8300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11727814,"num_objects":2865,"num_object_clones":0,"num_object_copies":5730,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2865,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14498,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1458},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":187368182,"num_objects":45754,"num_object_clones":0,"num_object_copies":91508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40625,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45754,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":246352,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10764,"num_bytes_recovered":33095680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":585838592,"data_stored":585696767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160778,"ondisk_log_size":160778,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986284,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":579056,"kb_used_data":232224,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93792784,"statfs":{"total":96636764160,"available":96043810816,"internally_reserved":0,"allocated":237797376,"data_stored":237494692,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":355135146},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,7,1,1,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659230,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":503068,"kb_used_data":138188,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93868772,"statfs":{"total":96636764160,"available":96121622528,"internally_reserved":0,"allocated":141504512,"data_stored":141172460,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":373616302},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,2,7,7],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182513,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":658532,"kb_used_data":203476,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93713308,"statfs":{"total":96636764160,"available":95962427392,"internally_reserved":0,"allocated":208359424,"data_stored":208056735,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":465956522},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,1,1,1,1,4,7],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":207917056,"data_stored":207880926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":140886016,"data_stored":140820523,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":237035520,"data_stored":236995318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:07:06.332 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:07:06.878 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:07:06.878 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:07:06.901 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":335,"stamp":"2024-09-10T16:07:05.730125+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":187368201,"num_objects":45756,"num_object_clones":0,"num_object_copies":91512,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40625,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45756,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":246354,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10766,"num_bytes_recovered":33095699,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160781,"ondisk_log_size":160781,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1740656,"kb_used_data":573888,"kb_used_omap":12,"kb_used_meta":1166707,"kb_avail":281374864,"statfs":{"total":289910292480,"available":288127860736,"internally_reserved":0,"allocated":587661312,"data_stored":586723887,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1194707970},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,1,3,9,4,12,17],"upper_bound":128},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-24715264,"num_objects":-6034,"num_object_clones":0,"num_object_copies":-12068,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-255,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-6034,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6034,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":255,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004390"},"pg_stats":[{"pgid":"2.f","version":"47'17935","reported_seq":18014,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:05.320638+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:07:05.320638+0000","last_peered":"2024-09-10T16:07:05.320638+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:07:05.320638+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"43'7900","ondisk_log_start":"43'7900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11210752,"num_objects":2737,"num_object_clones":0,"num_object_copies":5474,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2737,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17935,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1522}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'17885","reported_seq":18003,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:07:00.787957+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:07:00.787957+0000","last_peered":"2024-09-10T16:07:00.787957+0000","last_clean":"2024-09-10T16:07:00.787957+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:07:00.787957+0000","last_undegraded":"2024-09-10T16:07:00.787957+0000","last_fullsized":"2024-09-10T16:07:00.787957+0000","mapping_epoch":35,"log_start":"43'7800","ondisk_log_start":"43'7800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12121030,"num_objects":2961,"num_object_clones":0,"num_object_copies":5922,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2961,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":15990,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'17947","reported_seq":26631,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:07:03.824630+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:07:03.824630+0000","last_peered":"2024-09-10T16:07:03.824630+0000","last_clean":"2024-09-10T16:07:03.824630+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:07:03.824630+0000","last_undegraded":"2024-09-10T16:07:03.824630+0000","last_fullsized":"2024-09-10T16:07:03.824630+0000","mapping_epoch":44,"log_start":"43'7900","ondisk_log_start":"43'7900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11673624,"num_objects":2851,"num_object_clones":0,"num_object_copies":5702,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2851,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15951,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'17779","reported_seq":17865,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:07:00.459570+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:07:00.459570+0000","last_peered":"2024-09-10T16:07:00.459570+0000","last_clean":"2024-09-10T16:07:00.459570+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:07:00.459570+0000","last_undegraded":"2024-09-10T16:07:00.459570+0000","last_fullsized":"2024-09-10T16:07:00.459570+0000","mapping_epoch":35,"log_start":"43'7700","ondisk_log_start":"43'7700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11800576,"num_objects":2881,"num_object_clones":0,"num_object_copies":5762,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2881,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17779,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'18140","reported_seq":18276,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:07:05.186978+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:07:05.186978+0000","last_peered":"2024-09-10T16:07:05.186978+0000","last_clean":"2024-09-10T16:07:05.186978+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:07:05.186978+0000","last_undegraded":"2024-09-10T16:07:05.186978+0000","last_fullsized":"2024-09-10T16:07:05.186978+0000","mapping_epoch":34,"log_start":"43'8100","ondisk_log_start":"43'8100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11268579,"num_objects":2752,"num_object_clones":0,"num_object_copies":5504,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2752,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14851,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'18246","reported_seq":18386,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:05.264786+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:07:05.264786+0000","last_peered":"2024-09-10T16:07:05.264786+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:07:05.264786+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"43'8200","ondisk_log_start":"43'8200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11530723,"num_objects":2816,"num_object_clones":0,"num_object_copies":5632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2816,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14858,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1555},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'18200","reported_seq":18344,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:05.264764+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:07:05.264764+0000","last_peered":"2024-09-10T16:07:05.264764+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:07:05.264764+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"43'8100","ondisk_log_start":"43'8100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11805638,"num_objects":2884,"num_object_clones":0,"num_object_copies":5768,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2884,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":14699,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1353},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'18441","reported_seq":18577,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:07:05.232274+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:07:05.232274+0000","last_peered":"2024-09-10T16:07:05.232274+0000","last_clean":"2024-09-10T16:07:05.232274+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:07:05.232274+0000","last_undegraded":"2024-09-10T16:07:05.232274+0000","last_fullsized":"2024-09-10T16:07:05.232274+0000","mapping_epoch":34,"log_start":"43'8400","ondisk_log_start":"43'8400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11767808,"num_objects":2873,"num_object_clones":0,"num_object_copies":5746,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2873,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15288,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.876895+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:05:19.876895+0000","last_peered":"2024-09-10T16:05:19.876895+0000","last_clean":"2024-09-10T16:05:19.876895+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:05:19.876895+0000","last_undegraded":"2024-09-10T16:05:19.876895+0000","last_fullsized":"2024-09-10T16:05:19.876895+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'17403","reported_seq":17549,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:00.697876+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:07:00.697876+0000","last_peered":"2024-09-10T16:07:00.697876+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:07:00.697876+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"43'7400","ondisk_log_start":"43'7400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12509667,"num_objects":3055,"num_object_clones":0,"num_object_copies":6110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3055,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15222,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-1068}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":100,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.875849+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:05:19.875849+0000","last_peered":"2024-09-10T16:05:19.875849+0000","last_clean":"2024-09-10T16:05:19.875849+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:05:19.875849+0000","last_undegraded":"2024-09-10T16:05:19.875849+0000","last_fullsized":"2024-09-10T16:05:19.875849+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'18069","reported_seq":18204,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:05.301588+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:07:05.301588+0000","last_peered":"2024-09-10T16:07:05.301588+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:07:05.301588+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"43'8000","ondisk_log_start":"43'8000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11636736,"num_objects":2841,"num_object_clones":0,"num_object_copies":5682,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2841,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14573,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1414}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":110,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.872023+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:05:19.872023+0000","last_peered":"2024-09-10T16:05:19.872023+0000","last_clean":"2024-09-10T16:05:19.872023+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:05:19.872023+0000","last_undegraded":"2024-09-10T16:05:19.872023+0000","last_fullsized":"2024-09-10T16:05:19.872023+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'17909","reported_seq":18050,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:00.735254+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:07:00.735254+0000","last_peered":"2024-09-10T16:07:00.735254+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:07:00.735254+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"43'7900","ondisk_log_start":"43'7900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11764195,"num_objects":2873,"num_object_clones":0,"num_object_copies":5746,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2873,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15589,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-1431}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877148+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:05:19.877148+0000","last_peered":"2024-09-10T16:05:19.877148+0000","last_clean":"2024-09-10T16:05:19.877148+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:05:19.877148+0000","last_undegraded":"2024-09-10T16:05:19.877148+0000","last_fullsized":"2024-09-10T16:05:19.877148+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'18221","reported_seq":22929,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:00.773977+0000","last_change":"2024-09-10T16:05:18.271152+0000","last_active":"2024-09-10T16:07:00.773977+0000","last_peered":"2024-09-10T16:07:00.773977+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:07:00.773977+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"43'8200","ondisk_log_start":"43'8200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12079104,"num_objects":2949,"num_object_clones":0,"num_object_copies":5898,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2068,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2949,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15775,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2286,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":881}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'18045","reported_seq":18186,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:05.194117+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:07:05.194117+0000","last_peered":"2024-09-10T16:07:05.194117+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:07:05.194117+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"43'8000","ondisk_log_start":"43'8000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11440128,"num_objects":2793,"num_object_clones":0,"num_object_copies":5586,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2793,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14676,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1511}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877654+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:05:19.877654+0000","last_peered":"2024-09-10T16:05:19.877654+0000","last_clean":"2024-09-10T16:05:19.877654+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:05:19.877654+0000","last_undegraded":"2024-09-10T16:05:19.877654+0000","last_fullsized":"2024-09-10T16:05:19.877654+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'17730","reported_seq":26301,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:07:03.815350+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:07:03.815350+0000","last_peered":"2024-09-10T16:07:03.815350+0000","last_clean":"2024-09-10T16:07:03.815350+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:07:03.815350+0000","last_undegraded":"2024-09-10T16:07:03.815350+0000","last_fullsized":"2024-09-10T16:07:03.815350+0000","mapping_epoch":46,"log_start":"43'7700","ondisk_log_start":"43'7700","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11747328,"num_objects":2868,"num_object_clones":0,"num_object_copies":5736,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2868,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14120,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":100,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.876043+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:05:19.876043+0000","last_peered":"2024-09-10T16:05:19.876043+0000","last_clean":"2024-09-10T16:05:19.876043+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:05:19.876043+0000","last_undegraded":"2024-09-10T16:05:19.876043+0000","last_fullsized":"2024-09-10T16:05:19.876043+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'18293","reported_seq":18441,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:05.286824+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:07:05.286824+0000","last_peered":"2024-09-10T16:07:05.286824+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:07:05.286824+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"43'8200","ondisk_log_start":"43'8200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11284480,"num_objects":2755,"num_object_clones":0,"num_object_copies":5510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2755,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14548,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1616}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877349+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:05:19.877349+0000","last_peered":"2024-09-10T16:05:19.877349+0000","last_clean":"2024-09-10T16:05:19.877349+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:05:19.877349+0000","last_undegraded":"2024-09-10T16:05:19.877349+0000","last_fullsized":"2024-09-10T16:05:19.877349+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877269+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:05:19.877269+0000","last_peered":"2024-09-10T16:05:19.877269+0000","last_clean":"2024-09-10T16:05:19.877269+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:05:19.877269+0000","last_undegraded":"2024-09-10T16:05:19.877269+0000","last_fullsized":"2024-09-10T16:05:19.877269+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'18335","reported_seq":18485,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:05.188415+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:07:05.188415+0000","last_peered":"2024-09-10T16:07:05.188415+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:07:05.188415+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"43'8300","ondisk_log_start":"43'8300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11727814,"num_objects":2865,"num_object_clones":0,"num_object_copies":5730,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2865,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14498,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1458},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":187368182,"num_objects":45754,"num_object_clones":0,"num_object_copies":91508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40625,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45754,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":246352,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10764,"num_bytes_recovered":33095680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":585838592,"data_stored":585696767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160778,"ondisk_log_size":160778,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986284,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":579056,"kb_used_data":232224,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93792784,"statfs":{"total":96636764160,"available":96043810816,"internally_reserved":0,"allocated":237797376,"data_stored":237494692,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":355135146},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,7,1,1,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659230,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":503068,"kb_used_data":138188,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93868772,"statfs":{"total":96636764160,"available":96121622528,"internally_reserved":0,"allocated":141504512,"data_stored":141172460,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":373616302},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,2,7,7],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182513,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":658532,"kb_used_data":203476,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93713308,"statfs":{"total":96636764160,"available":95962427392,"internally_reserved":0,"allocated":208359424,"data_stored":208056735,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":465956522},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,1,1,1,1,4,7],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":207917056,"data_stored":207880926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":140886016,"data_stored":140820523,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":237035520,"data_stored":236995318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:07:06.902 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-10T16:07:06.902 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "47'17935", 'reported_seq': 18014, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:07:05.320638+0000', 'last_change': '2024-09-10T15:59:16.281558+0000', 'last_active': '2024-09-10T16:07:05.320638+0000', 'last_peered': '2024-09-10T16:07:05.320638+0000', 'last_clean': '2024-09-10T15:59:14.132456+0000', 'last_became_active': '2024-09-10T15:59:16.274407+0000', 'last_became_peered': '2024-09-10T15:59:16.274407+0000', 'last_unstale': '2024-09-10T16:07:05.320638+0000', 'last_undegraded': '2024-09-10T15:59:15.851279+0000', 'last_fullsized': '2024-09-10T15:59:15.774356+0000', 'mapping_epoch': 42, 'log_start': "43'7900", 'ondisk_log_start': "43'7900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'objects_scrubbed': 0, 'log_size': 10035, 'log_dups_size': 0, 'ondisk_log_size': 10035, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 11210752, 'num_objects': 2737, 'num_object_clones': 0, 'num_object_copies': 5474, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4259, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2737, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17935, 'num_write_kb': 41344, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1522}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:07:06.902 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-10T16:07:06.902 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "47'18246", 'reported_seq': 18386, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:07:05.264786+0000', 'last_change': '2024-09-10T15:59:16.308264+0000', 'last_active': '2024-09-10T16:07:05.264786+0000', 'last_peered': '2024-09-10T16:07:05.264786+0000', 'last_clean': '2024-09-10T15:59:14.111551+0000', 'last_became_active': '2024-09-10T15:59:16.300040+0000', 'last_became_peered': '2024-09-10T15:59:16.300040+0000', 'last_unstale': '2024-09-10T16:07:05.264786+0000', 'last_undegraded': '2024-09-10T15:59:15.878234+0000', 'last_fullsized': '2024-09-10T15:59:15.769328+0000', 'mapping_epoch': 42, 'log_start': "43'8200", 'ondisk_log_start': "43'8200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3442", 'last_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'objects_scrubbed': 3384, 'log_size': 10046, 'log_dups_size': 0, 'ondisk_log_size': 10046, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1292, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 11530723, 'num_objects': 2816, 'num_object_clones': 0, 'num_object_copies': 5632, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4372, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2816, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 14858, 'num_write_kb': 28580, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1555}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:07:06.902 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-10T16:07:06.903 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "47'18200", 'reported_seq': 18344, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:07:05.264764+0000', 'last_change': '2024-09-10T15:59:16.331899+0000', 'last_active': '2024-09-10T16:07:05.264764+0000', 'last_peered': '2024-09-10T16:07:05.264764+0000', 'last_clean': '2024-09-10T15:59:14.139436+0000', 'last_became_active': '2024-09-10T15:59:16.318715+0000', 'last_became_peered': '2024-09-10T15:59:16.318715+0000', 'last_unstale': '2024-09-10T16:07:05.264764+0000', 'last_undegraded': '2024-09-10T15:59:15.877524+0000', 'last_fullsized': '2024-09-10T15:59:15.769042+0000', 'mapping_epoch': 42, 'log_start': "43'8100", 'ondisk_log_start': "43'8100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3537", 'last_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'objects_scrubbed': 3497, 'log_size': 10100, 'log_dups_size': 0, 'ondisk_log_size': 10100, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1224, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 11805638, 'num_objects': 2884, 'num_object_clones': 0, 'num_object_copies': 5768, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4237, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2884, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 14699, 'num_write_kb': 28172, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1353}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:07:06.903 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-10T16:07:06.903 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "47'17403", 'reported_seq': 17549, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:07:00.697876+0000', 'last_change': '2024-09-10T15:59:16.175680+0000', 'last_active': '2024-09-10T16:07:00.697876+0000', 'last_peered': '2024-09-10T16:07:00.697876+0000', 'last_clean': '2024-09-10T15:59:14.088187+0000', 'last_became_active': '2024-09-10T15:59:16.165810+0000', 'last_became_peered': '2024-09-10T15:59:16.165810+0000', 'last_unstale': '2024-09-10T16:07:00.697876+0000', 'last_undegraded': '2024-09-10T15:59:15.822098+0000', 'last_fullsized': '2024-09-10T15:59:15.769123+0000', 'mapping_epoch': 42, 'log_start': "43'7400", 'ondisk_log_start': "43'7400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2218", 'last_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'objects_scrubbed': 2175, 'log_size': 10003, 'log_dups_size': 0, 'ondisk_log_size': 10003, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 825, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 12509667, 'num_objects': 3055, 'num_object_clones': 0, 'num_object_copies': 6110, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4124, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3055, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 15222, 'num_write_kb': 32204, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': -1068}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:07:06.903 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-10T16:07:06.903 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "47'18069", 'reported_seq': 18204, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:07:05.301588+0000', 'last_change': '2024-09-10T15:59:16.277742+0000', 'last_active': '2024-09-10T16:07:05.301588+0000', 'last_peered': '2024-09-10T16:07:05.301588+0000', 'last_clean': '2024-09-10T15:59:14.129081+0000', 'last_became_active': '2024-09-10T15:59:16.269835+0000', 'last_became_peered': '2024-09-10T15:59:16.269835+0000', 'last_unstale': '2024-09-10T16:07:05.301588+0000', 'last_undegraded': '2024-09-10T15:59:15.860857+0000', 'last_fullsized': '2024-09-10T15:59:15.771410+0000', 'mapping_epoch': 42, 'log_start': "43'8000", 'ondisk_log_start': "43'8000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3557", 'last_scrub_stamp': '2024-09-10T15:59:02.493678+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:02.493678+0000', 'objects_scrubbed': 3488, 'log_size': 10069, 'log_dups_size': 0, 'ondisk_log_size': 10069, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1472, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 11636736, 'num_objects': 2841, 'num_object_clones': 0, 'num_object_copies': 5682, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4257, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2841, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14573, 'num_write_kb': 27852, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1414}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:07:06.903 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-10T16:07:06.903 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "47'17909", 'reported_seq': 18050, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:07:00.735254+0000', 'last_change': '2024-09-10T15:59:16.284311+0000', 'last_active': '2024-09-10T16:07:00.735254+0000', 'last_peered': '2024-09-10T16:07:00.735254+0000', 'last_clean': '2024-09-10T15:59:14.116041+0000', 'last_became_active': '2024-09-10T15:59:16.274543+0000', 'last_became_peered': '2024-09-10T15:59:16.274543+0000', 'last_unstale': '2024-09-10T16:07:00.735254+0000', 'last_undegraded': '2024-09-10T15:59:15.812484+0000', 'last_fullsized': '2024-09-10T15:59:15.770885+0000', 'mapping_epoch': 42, 'log_start': "43'7900", 'ondisk_log_start': "43'7900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2356", 'last_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'objects_scrubbed': 2316, 'log_size': 10009, 'log_dups_size': 0, 'ondisk_log_size': 10009, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 783, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 11764195, 'num_objects': 2873, 'num_object_clones': 0, 'num_object_copies': 5746, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2873, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 15589, 'num_write_kb': 32292, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': -1431}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:07:06.903 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-10T16:07:06.903 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "47'18221", 'reported_seq': 22929, 'reported_epoch': 47, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:07:00.773977+0000', 'last_change': '2024-09-10T16:05:18.271152+0000', 'last_active': '2024-09-10T16:07:00.773977+0000', 'last_peered': '2024-09-10T16:07:00.773977+0000', 'last_clean': '2024-09-10T15:59:14.110654+0000', 'last_became_active': '2024-09-10T15:59:16.164645+0000', 'last_became_peered': '2024-09-10T15:59:16.164645+0000', 'last_unstale': '2024-09-10T16:07:00.773977+0000', 'last_undegraded': '2024-09-10T15:59:15.803593+0000', 'last_fullsized': '2024-09-10T15:59:15.769083+0000', 'mapping_epoch': 42, 'log_start': "43'8200", 'ondisk_log_start': "43'8200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2486", 'last_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'objects_scrubbed': 2440, 'log_size': 10021, 'log_dups_size': 0, 'ondisk_log_size': 10021, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 792, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 12079104, 'num_objects': 2949, 'num_object_clones': 0, 'num_object_copies': 5898, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2068, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2949, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15775, 'num_write_kb': 32568, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2286, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 881}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:07:06.903 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-10T16:07:06.903 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "47'18045", 'reported_seq': 18186, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:07:05.194117+0000', 'last_change': '2024-09-10T15:59:16.315641+0000', 'last_active': '2024-09-10T16:07:05.194117+0000', 'last_peered': '2024-09-10T16:07:05.194117+0000', 'last_clean': '2024-09-10T15:59:14.117001+0000', 'last_became_active': '2024-09-10T15:59:16.302245+0000', 'last_became_peered': '2024-09-10T15:59:16.302245+0000', 'last_unstale': '2024-09-10T16:07:05.194117+0000', 'last_undegraded': '2024-09-10T15:59:15.845012+0000', 'last_fullsized': '2024-09-10T15:59:15.770337+0000', 'mapping_epoch': 42, 'log_start': "43'8000", 'ondisk_log_start': "43'8000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3404", 'last_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'objects_scrubbed': 3367, 'log_size': 10045, 'log_dups_size': 0, 'ondisk_log_size': 10045, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 983, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 11440128, 'num_objects': 2793, 'num_object_clones': 0, 'num_object_copies': 5586, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2793, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14676, 'num_write_kb': 28204, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1511}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:07:06.904 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-10T16:07:06.904 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "47'18293", 'reported_seq': 18441, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:07:05.286824+0000', 'last_change': '2024-09-10T15:59:16.311291+0000', 'last_active': '2024-09-10T16:07:05.286824+0000', 'last_peered': '2024-09-10T16:07:05.286824+0000', 'last_clean': '2024-09-10T15:59:14.116363+0000', 'last_became_active': '2024-09-10T15:59:16.301244+0000', 'last_became_peered': '2024-09-10T15:59:16.301244+0000', 'last_unstale': '2024-09-10T16:07:05.286824+0000', 'last_undegraded': '2024-09-10T15:59:15.883434+0000', 'last_fullsized': '2024-09-10T15:59:15.772756+0000', 'mapping_epoch': 42, 'log_start': "43'8200", 'ondisk_log_start': "43'8200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3789", 'last_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'objects_scrubbed': 3741, 'log_size': 10093, 'log_dups_size': 0, 'ondisk_log_size': 10093, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1440, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 11284480, 'num_objects': 2755, 'num_object_clones': 0, 'num_object_copies': 5510, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4373, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2755, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14548, 'num_write_kb': 27124, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1616}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:07:06.904 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-10T16:07:06.904 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "47'18335", 'reported_seq': 18485, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:07:05.188415+0000', 'last_change': '2024-09-10T15:59:16.313409+0000', 'last_active': '2024-09-10T16:07:05.188415+0000', 'last_peered': '2024-09-10T16:07:05.188415+0000', 'last_clean': '2024-09-10T15:59:14.129780+0000', 'last_became_active': '2024-09-10T15:59:16.300932+0000', 'last_became_peered': '2024-09-10T15:59:16.300932+0000', 'last_unstale': '2024-09-10T16:07:05.188415+0000', 'last_undegraded': '2024-09-10T15:59:15.887823+0000', 'last_fullsized': '2024-09-10T15:59:15.768513+0000', 'mapping_epoch': 42, 'log_start': "43'8300", 'ondisk_log_start': "43'8300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3880", 'last_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'objects_scrubbed': 3827, 'log_size': 10035, 'log_dups_size': 0, 'ondisk_log_size': 10035, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1397, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 11727814, 'num_objects': 2865, 'num_object_clones': 0, 'num_object_copies': 5730, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4325, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2865, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14498, 'num_write_kb': 27072, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1458}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:07:06.904 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T16:07:07.630 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:07:07.630 INFO:teuthology.orchestra.run.smithi115.stdout:{"fsid":"7effee29-df05-4351-9741-ee203564ed81","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 40625/91512 objects degraded (44.393%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":591,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":47,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983954,"num_in_osds":3,"osd_in_since":1725983947,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":45756,"data_bytes":187368201,"bytes_used":1782431744,"bytes_avail":288127860736,"bytes_total":289910292480,"degraded_objects":40625,"degraded_total":91512,"degraded_ratio":0.44393085059882859,"recovering_objects_per_sec":21,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":255,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":502},"fsmap":{"epoch":1,"btime":"2024-09-10T15:57:14:967120+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T16:00:13.560776+0000","services":{}},"progress_events":{}} 2024-09-10T16:07:07.659 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-10T16:07:07.660 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:07:08.226 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:07:08.226 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:07:08.256 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":336,"stamp":"2024-09-10T16:07:07.730788+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":184550153,"num_objects":45068,"num_object_clones":0,"num_object_copies":90136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40481,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45068,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":247042,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10910,"num_bytes_recovered":33095699,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160769,"ondisk_log_size":160769,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1740324,"kb_used_data":569396,"kb_used_omap":12,"kb_used_meta":1170867,"kb_avail":281375196,"statfs":{"total":289910292480,"available":288128200704,"internally_reserved":0,"allocated":583061504,"data_stored":582124079,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1198967810},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,1,3,4,3,13,14],"upper_bound":128},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-21995520,"num_objects":-5370,"num_object_clones":0,"num_object_copies":-10740,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-399,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5370,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5370,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":399,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004245"},"pg_stats":[{"pgid":"2.f","version":"47'17935","reported_seq":18014,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:05.320638+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:07:05.320638+0000","last_peered":"2024-09-10T16:07:05.320638+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:07:05.320638+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"43'7900","ondisk_log_start":"43'7900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11210752,"num_objects":2737,"num_object_clones":0,"num_object_copies":5474,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2737,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17935,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1522}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'18029","reported_seq":18147,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:07:05.741339+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:07:05.741339+0000","last_peered":"2024-09-10T16:07:05.741339+0000","last_clean":"2024-09-10T16:07:05.741339+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:07:05.741339+0000","last_undegraded":"2024-09-10T16:07:05.741339+0000","last_fullsized":"2024-09-10T16:07:05.741339+0000","mapping_epoch":35,"log_start":"43'8000","ondisk_log_start":"43'8000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11531206,"num_objects":2817,"num_object_clones":0,"num_object_copies":5634,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2817,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16134,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'17947","reported_seq":26631,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:07:03.824630+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:07:03.824630+0000","last_peered":"2024-09-10T16:07:03.824630+0000","last_clean":"2024-09-10T16:07:03.824630+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:07:03.824630+0000","last_undegraded":"2024-09-10T16:07:03.824630+0000","last_fullsized":"2024-09-10T16:07:03.824630+0000","mapping_epoch":44,"log_start":"43'7900","ondisk_log_start":"43'7900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11673624,"num_objects":2851,"num_object_clones":0,"num_object_copies":5702,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2851,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15951,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'17928","reported_seq":18014,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:07:05.800449+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:07:05.800449+0000","last_peered":"2024-09-10T16:07:05.800449+0000","last_clean":"2024-09-10T16:07:05.800449+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:07:05.800449+0000","last_undegraded":"2024-09-10T16:07:05.800449+0000","last_fullsized":"2024-09-10T16:07:05.800449+0000","mapping_epoch":35,"log_start":"43'7900","ondisk_log_start":"43'7900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11190272,"num_objects":2732,"num_object_clones":0,"num_object_copies":5464,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2732,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17928,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'18140","reported_seq":18276,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:07:05.186978+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:07:05.186978+0000","last_peered":"2024-09-10T16:07:05.186978+0000","last_clean":"2024-09-10T16:07:05.186978+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:07:05.186978+0000","last_undegraded":"2024-09-10T16:07:05.186978+0000","last_fullsized":"2024-09-10T16:07:05.186978+0000","mapping_epoch":34,"log_start":"43'8100","ondisk_log_start":"43'8100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11268579,"num_objects":2752,"num_object_clones":0,"num_object_copies":5504,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2752,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14851,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'18246","reported_seq":18386,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:05.264786+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:07:05.264786+0000","last_peered":"2024-09-10T16:07:05.264786+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:07:05.264786+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"43'8200","ondisk_log_start":"43'8200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11530723,"num_objects":2816,"num_object_clones":0,"num_object_copies":5632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2816,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14858,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1555},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'18200","reported_seq":18344,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:05.264764+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:07:05.264764+0000","last_peered":"2024-09-10T16:07:05.264764+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:07:05.264764+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"43'8100","ondisk_log_start":"43'8100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11805638,"num_objects":2884,"num_object_clones":0,"num_object_copies":5768,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2884,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":14699,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1353},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'18441","reported_seq":18577,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:07:05.232274+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:07:05.232274+0000","last_peered":"2024-09-10T16:07:05.232274+0000","last_clean":"2024-09-10T16:07:05.232274+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:07:05.232274+0000","last_undegraded":"2024-09-10T16:07:05.232274+0000","last_fullsized":"2024-09-10T16:07:05.232274+0000","mapping_epoch":34,"log_start":"43'8400","ondisk_log_start":"43'8400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11767808,"num_objects":2873,"num_object_clones":0,"num_object_copies":5746,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2873,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15288,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.876895+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:05:19.876895+0000","last_peered":"2024-09-10T16:05:19.876895+0000","last_clean":"2024-09-10T16:05:19.876895+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:05:19.876895+0000","last_undegraded":"2024-09-10T16:05:19.876895+0000","last_fullsized":"2024-09-10T16:05:19.876895+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'17544","reported_seq":17690,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:05.785221+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:07:05.785221+0000","last_peered":"2024-09-10T16:07:05.785221+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:07:05.785221+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"43'7500","ondisk_log_start":"43'7500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11932131,"num_objects":2914,"num_object_clones":0,"num_object_copies":5828,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2914,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15363,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-1209}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":100,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.875849+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:05:19.875849+0000","last_peered":"2024-09-10T16:05:19.875849+0000","last_clean":"2024-09-10T16:05:19.875849+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:05:19.875849+0000","last_undegraded":"2024-09-10T16:05:19.875849+0000","last_fullsized":"2024-09-10T16:05:19.875849+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'18069","reported_seq":18204,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:05.301588+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:07:05.301588+0000","last_peered":"2024-09-10T16:07:05.301588+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:07:05.301588+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"43'8000","ondisk_log_start":"43'8000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11636736,"num_objects":2841,"num_object_clones":0,"num_object_copies":5682,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2841,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14573,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1414}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":110,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.872023+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:05:19.872023+0000","last_peered":"2024-09-10T16:05:19.872023+0000","last_clean":"2024-09-10T16:05:19.872023+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:05:19.872023+0000","last_undegraded":"2024-09-10T16:05:19.872023+0000","last_fullsized":"2024-09-10T16:05:19.872023+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'18030","reported_seq":18171,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:05.757044+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:07:05.757044+0000","last_peered":"2024-09-10T16:07:05.757044+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:07:05.757044+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"43'8000","ondisk_log_start":"43'8000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11268579,"num_objects":2752,"num_object_clones":0,"num_object_copies":5504,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2752,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15710,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-1552}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877148+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:05:19.877148+0000","last_peered":"2024-09-10T16:05:19.877148+0000","last_clean":"2024-09-10T16:05:19.877148+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:05:19.877148+0000","last_undegraded":"2024-09-10T16:05:19.877148+0000","last_fullsized":"2024-09-10T16:05:19.877148+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'18354","reported_seq":23350,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:05.811621+0000","last_change":"2024-09-10T16:05:18.271152+0000","last_active":"2024-09-10T16:07:05.811621+0000","last_peered":"2024-09-10T16:07:05.811621+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:07:05.811621+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"43'8300","ondisk_log_start":"43'8300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11534336,"num_objects":2816,"num_object_clones":0,"num_object_copies":5632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1924,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2816,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15908,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2430,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":892}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'18045","reported_seq":18186,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:05.194117+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:07:05.194117+0000","last_peered":"2024-09-10T16:07:05.194117+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:07:05.194117+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"43'8000","ondisk_log_start":"43'8000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11440128,"num_objects":2793,"num_object_clones":0,"num_object_copies":5586,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2793,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14676,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1511}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877654+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:05:19.877654+0000","last_peered":"2024-09-10T16:05:19.877654+0000","last_clean":"2024-09-10T16:05:19.877654+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:05:19.877654+0000","last_undegraded":"2024-09-10T16:05:19.877654+0000","last_fullsized":"2024-09-10T16:05:19.877654+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'17730","reported_seq":26301,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:07:03.815350+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:07:03.815350+0000","last_peered":"2024-09-10T16:07:03.815350+0000","last_clean":"2024-09-10T16:07:03.815350+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:07:03.815350+0000","last_undegraded":"2024-09-10T16:07:03.815350+0000","last_fullsized":"2024-09-10T16:07:03.815350+0000","mapping_epoch":46,"log_start":"43'7700","ondisk_log_start":"43'7700","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11747328,"num_objects":2868,"num_object_clones":0,"num_object_copies":5736,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2868,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14120,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":100,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.876043+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:05:19.876043+0000","last_peered":"2024-09-10T16:05:19.876043+0000","last_clean":"2024-09-10T16:05:19.876043+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:05:19.876043+0000","last_undegraded":"2024-09-10T16:05:19.876043+0000","last_fullsized":"2024-09-10T16:05:19.876043+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'18293","reported_seq":18441,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:05.286824+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:07:05.286824+0000","last_peered":"2024-09-10T16:07:05.286824+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:07:05.286824+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"43'8200","ondisk_log_start":"43'8200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11284480,"num_objects":2755,"num_object_clones":0,"num_object_copies":5510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2755,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14548,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1616}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877349+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:05:19.877349+0000","last_peered":"2024-09-10T16:05:19.877349+0000","last_clean":"2024-09-10T16:05:19.877349+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:05:19.877349+0000","last_undegraded":"2024-09-10T16:05:19.877349+0000","last_fullsized":"2024-09-10T16:05:19.877349+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877269+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:05:19.877269+0000","last_peered":"2024-09-10T16:05:19.877269+0000","last_clean":"2024-09-10T16:05:19.877269+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:05:19.877269+0000","last_undegraded":"2024-09-10T16:05:19.877269+0000","last_fullsized":"2024-09-10T16:05:19.877269+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'18335","reported_seq":18485,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:05.188415+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:07:05.188415+0000","last_peered":"2024-09-10T16:07:05.188415+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:07:05.188415+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"43'8300","ondisk_log_start":"43'8300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11727814,"num_objects":2865,"num_object_clones":0,"num_object_copies":5730,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2865,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14498,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1458},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":184550134,"num_objects":45066,"num_object_clones":0,"num_object_copies":90132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40481,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45066,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":247040,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10908,"num_bytes_recovered":33095680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":581201920,"data_stored":581060095,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160766,"ondisk_log_size":160766,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986285,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":578724,"kb_used_data":227732,"kb_used_omap":4,"kb_used_meta":350971,"kb_avail":93793116,"statfs":{"total":96636764160,"available":96044150784,"internally_reserved":0,"allocated":233197568,"data_stored":232894884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":359394986},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,0,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659230,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":503068,"kb_used_data":138188,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93868772,"statfs":{"total":96636764160,"available":96121622528,"internally_reserved":0,"allocated":141504512,"data_stored":141172460,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":373616302},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,2,7,7],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182513,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":658532,"kb_used_data":203476,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93713308,"statfs":{"total":96636764160,"available":95962427392,"internally_reserved":0,"allocated":208359424,"data_stored":208056735,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":465956522},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,1,1,1,1,4,7],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":207917056,"data_stored":207880926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":140886016,"data_stored":140820523,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":232398848,"data_stored":232358646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:07:11.260 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:07:11.801 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:07:11.804 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:07:11.827 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":338,"stamp":"2024-09-10T16:07:11.731974+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":175145737,"num_objects":42772,"num_object_clones":0,"num_object_copies":85544,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40342,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42772,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":249338,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11049,"num_bytes_recovered":33095699,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161065,"ondisk_log_size":161065,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1688884,"kb_used_data":551216,"kb_used_omap":12,"kb_used_meta":1137587,"kb_avail":281426636,"statfs":{"total":289910292480,"available":288180875264,"internally_reserved":0,"allocated":564445184,"data_stored":563507759,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1164889090},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,5,4,8,10,9],"upper_bound":256},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-27287552,"num_objects":-6662,"num_object_clones":0,"num_object_copies":-13324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-412,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-6662,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6662,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":412,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004026"},"pg_stats":[{"pgid":"2.f","version":"47'18076","reported_seq":18157,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:10.293198+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:07:10.293198+0000","last_peered":"2024-09-10T16:07:10.293198+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:07:10.293198+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"43'8000","ondisk_log_start":"43'8000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10633216,"num_objects":2596,"num_object_clones":0,"num_object_copies":5192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2596,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18076,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1663}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'18180","reported_seq":18298,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:07:10.791532+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:07:10.791532+0000","last_peered":"2024-09-10T16:07:10.791532+0000","last_clean":"2024-09-10T16:07:10.791532+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:07:10.791532+0000","last_undegraded":"2024-09-10T16:07:10.791532+0000","last_fullsized":"2024-09-10T16:07:10.791532+0000","mapping_epoch":35,"log_start":"43'8100","ondisk_log_start":"43'8100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10912710,"num_objects":2666,"num_object_clones":0,"num_object_copies":5332,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2666,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16285,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'18091","reported_seq":26775,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:07:08.888002+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:07:08.888002+0000","last_peered":"2024-09-10T16:07:08.888002+0000","last_clean":"2024-09-10T16:07:08.888002+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:07:08.888002+0000","last_undegraded":"2024-09-10T16:07:08.888002+0000","last_fullsized":"2024-09-10T16:07:08.888002+0000","mapping_epoch":44,"log_start":"43'8000","ondisk_log_start":"43'8000","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11083800,"num_objects":2707,"num_object_clones":0,"num_object_copies":5414,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2707,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16095,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'18052","reported_seq":18138,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:07:10.728479+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:07:10.728479+0000","last_peered":"2024-09-10T16:07:10.728479+0000","last_clean":"2024-09-10T16:07:10.728479+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:07:10.728479+0000","last_undegraded":"2024-09-10T16:07:10.728479+0000","last_fullsized":"2024-09-10T16:07:10.728479+0000","mapping_epoch":35,"log_start":"43'8000","ondisk_log_start":"43'8000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10682368,"num_objects":2608,"num_object_clones":0,"num_object_copies":5216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2608,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18052,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'18287","reported_seq":18423,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:07:10.312262+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:07:10.312262+0000","last_peered":"2024-09-10T16:07:10.312262+0000","last_clean":"2024-09-10T16:07:10.312262+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:07:10.312262+0000","last_undegraded":"2024-09-10T16:07:10.312262+0000","last_fullsized":"2024-09-10T16:07:10.312262+0000","mapping_epoch":34,"log_start":"43'8200","ondisk_log_start":"43'8200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10666467,"num_objects":2605,"num_object_clones":0,"num_object_copies":5210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2605,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14998,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'18385","reported_seq":18529,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:10.212422+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:07:10.212422+0000","last_peered":"2024-09-10T16:07:10.212422+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:07:10.212422+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"43'8300","ondisk_log_start":"43'8300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10961379,"num_objects":2677,"num_object_clones":0,"num_object_copies":5354,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2677,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14997,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1694},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'18344","reported_seq":18488,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:10.302364+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:07:10.302364+0000","last_peered":"2024-09-10T16:07:10.302364+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:07:10.302364+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"43'8300","ondisk_log_start":"43'8300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11215814,"num_objects":2740,"num_object_clones":0,"num_object_copies":5480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2740,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":14843,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1497},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'18589","reported_seq":18725,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:07:10.312314+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:07:10.312314+0000","last_peered":"2024-09-10T16:07:10.312314+0000","last_clean":"2024-09-10T16:07:10.312314+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:07:10.312314+0000","last_undegraded":"2024-09-10T16:07:10.312314+0000","last_fullsized":"2024-09-10T16:07:10.312314+0000","mapping_epoch":34,"log_start":"43'8500","ondisk_log_start":"43'8500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11161600,"num_objects":2725,"num_object_clones":0,"num_object_copies":5450,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2725,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15436,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.876895+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:05:19.876895+0000","last_peered":"2024-09-10T16:05:19.876895+0000","last_clean":"2024-09-10T16:05:19.876895+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:05:19.876895+0000","last_undegraded":"2024-09-10T16:05:19.876895+0000","last_fullsized":"2024-09-10T16:05:19.876895+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'17678","reported_seq":17824,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:10.719777+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:07:10.719777+0000","last_peered":"2024-09-10T16:07:10.719777+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:07:10.719777+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"43'7600","ondisk_log_start":"43'7600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11383267,"num_objects":2780,"num_object_clones":0,"num_object_copies":5560,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2780,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15497,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-1343}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":100,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.875849+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:05:19.875849+0000","last_peered":"2024-09-10T16:05:19.875849+0000","last_clean":"2024-09-10T16:05:19.875849+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:05:19.875849+0000","last_undegraded":"2024-09-10T16:05:19.875849+0000","last_fullsized":"2024-09-10T16:05:19.875849+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'18209","reported_seq":18345,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:10.217419+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:07:10.217419+0000","last_peered":"2024-09-10T16:07:10.217419+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:07:10.217419+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"43'8200","ondisk_log_start":"43'8200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11063296,"num_objects":2701,"num_object_clones":0,"num_object_copies":5402,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2701,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14713,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1554}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":110,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.872023+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:05:19.872023+0000","last_peered":"2024-09-10T16:05:19.872023+0000","last_clean":"2024-09-10T16:05:19.872023+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:05:19.872023+0000","last_undegraded":"2024-09-10T16:05:19.872023+0000","last_fullsized":"2024-09-10T16:05:19.872023+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'18186","reported_seq":18323,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:10.828522+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:07:10.828522+0000","last_peered":"2024-09-10T16:07:10.828522+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:07:10.828522+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"43'8100","ondisk_log_start":"43'8100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10629603,"num_objects":2596,"num_object_clones":0,"num_object_copies":5192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2596,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15866,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-1708}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877148+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:05:19.877148+0000","last_peered":"2024-09-10T16:05:19.877148+0000","last_clean":"2024-09-10T16:05:19.877148+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:05:19.877148+0000","last_undegraded":"2024-09-10T16:05:19.877148+0000","last_fullsized":"2024-09-10T16:05:19.877148+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'18501","reported_seq":23775,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:10.839751+0000","last_change":"2024-09-10T16:05:18.271152+0000","last_active":"2024-09-10T16:07:10.839751+0000","last_peered":"2024-09-10T16:07:10.839751+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:07:10.839751+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"43'8500","ondisk_log_start":"43'8500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10932224,"num_objects":2669,"num_object_clones":0,"num_object_copies":5338,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1785,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2669,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16055,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2569,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":884}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'18187","reported_seq":18328,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:10.294249+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:07:10.294249+0000","last_peered":"2024-09-10T16:07:10.294249+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:07:10.294249+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"43'8100","ondisk_log_start":"43'8100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10858496,"num_objects":2651,"num_object_clones":0,"num_object_copies":5302,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2651,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14818,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1653}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877654+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:05:19.877654+0000","last_peered":"2024-09-10T16:05:19.877654+0000","last_clean":"2024-09-10T16:05:19.877654+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:05:19.877654+0000","last_undegraded":"2024-09-10T16:05:19.877654+0000","last_fullsized":"2024-09-10T16:05:19.877654+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'17885","reported_seq":26456,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:07:08.919954+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:07:08.919954+0000","last_peered":"2024-09-10T16:07:08.919954+0000","last_clean":"2024-09-10T16:07:08.919954+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:07:08.919954+0000","last_undegraded":"2024-09-10T16:07:08.919954+0000","last_fullsized":"2024-09-10T16:07:08.919954+0000","mapping_epoch":46,"log_start":"43'7800","ondisk_log_start":"43'7800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11112448,"num_objects":2713,"num_object_clones":0,"num_object_copies":5426,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2713,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14275,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":100,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.876043+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:05:19.876043+0000","last_peered":"2024-09-10T16:05:19.876043+0000","last_clean":"2024-09-10T16:05:19.876043+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:05:19.876043+0000","last_undegraded":"2024-09-10T16:05:19.876043+0000","last_fullsized":"2024-09-10T16:05:19.876043+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'18431","reported_seq":18575,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:10.293756+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:07:10.293756+0000","last_peered":"2024-09-10T16:07:10.293756+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:07:10.293756+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"43'8400","ondisk_log_start":"43'8400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10719232,"num_objects":2617,"num_object_clones":0,"num_object_copies":5234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2617,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14686,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1754}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877349+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:05:19.877349+0000","last_peered":"2024-09-10T16:05:19.877349+0000","last_clean":"2024-09-10T16:05:19.877349+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:05:19.877349+0000","last_undegraded":"2024-09-10T16:05:19.877349+0000","last_fullsized":"2024-09-10T16:05:19.877349+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877269+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:05:19.877269+0000","last_peered":"2024-09-10T16:05:19.877269+0000","last_clean":"2024-09-10T16:05:19.877269+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:05:19.877269+0000","last_undegraded":"2024-09-10T16:05:19.877269+0000","last_fullsized":"2024-09-10T16:05:19.877269+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'18481","reported_seq":18631,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:10.298339+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:07:10.298339+0000","last_peered":"2024-09-10T16:07:10.298339+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:07:10.298339+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"43'8400","ondisk_log_start":"43'8400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11129798,"num_objects":2719,"num_object_clones":0,"num_object_copies":5438,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2719,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14644,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1604},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":175145718,"num_objects":42770,"num_object_clones":0,"num_object_copies":85540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40342,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42770,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":249336,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11047,"num_bytes_recovered":33095680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":562343936,"data_stored":562202111,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161062,"ondisk_log_size":161062,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986286,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":561656,"kb_used_data":223144,"kb_used_omap":4,"kb_used_meta":338491,"kb_avail":93810184,"statfs":{"total":96636764160,"available":96061628416,"internally_reserved":0,"allocated":228499456,"data_stored":228196772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":346615466},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,2,3,1,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659231,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487872,"kb_used_data":131296,"kb_used_omap":4,"kb_used_meta":356539,"kb_avail":93883968,"statfs":{"total":96636764160,"available":96137183232,"internally_reserved":0,"allocated":134447104,"data_stored":134115052,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":365096622},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,0,2,4,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182514,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":639356,"kb_used_data":196776,"kb_used_omap":4,"kb_used_meta":442555,"kb_avail":93732484,"statfs":{"total":96636764160,"available":95982063616,"internally_reserved":0,"allocated":201498624,"data_stored":201195935,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":453177002},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,2,2,3,5,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":200867840,"data_stored":200831710,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":133738496,"data_stored":133673003,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":227737600,"data_stored":227697398,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:07:11.829 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:07:12.418 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:07:12.418 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:07:12.451 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":338,"stamp":"2024-09-10T16:07:11.731974+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":175145737,"num_objects":42772,"num_object_clones":0,"num_object_copies":85544,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40342,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42772,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":249338,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11049,"num_bytes_recovered":33095699,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161065,"ondisk_log_size":161065,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1688884,"kb_used_data":551216,"kb_used_omap":12,"kb_used_meta":1137587,"kb_avail":281426636,"statfs":{"total":289910292480,"available":288180875264,"internally_reserved":0,"allocated":564445184,"data_stored":563507759,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1164889090},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,5,4,8,10,9],"upper_bound":256},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-27287552,"num_objects":-6662,"num_object_clones":0,"num_object_copies":-13324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-412,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-6662,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6662,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":412,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004026"},"pg_stats":[{"pgid":"2.f","version":"47'18076","reported_seq":18157,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:10.293198+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:07:10.293198+0000","last_peered":"2024-09-10T16:07:10.293198+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:07:10.293198+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"43'8000","ondisk_log_start":"43'8000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10633216,"num_objects":2596,"num_object_clones":0,"num_object_copies":5192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2596,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18076,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1663}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'18180","reported_seq":18298,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:07:10.791532+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:07:10.791532+0000","last_peered":"2024-09-10T16:07:10.791532+0000","last_clean":"2024-09-10T16:07:10.791532+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:07:10.791532+0000","last_undegraded":"2024-09-10T16:07:10.791532+0000","last_fullsized":"2024-09-10T16:07:10.791532+0000","mapping_epoch":35,"log_start":"43'8100","ondisk_log_start":"43'8100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10912710,"num_objects":2666,"num_object_clones":0,"num_object_copies":5332,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2666,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16285,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'18091","reported_seq":26775,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:07:08.888002+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:07:08.888002+0000","last_peered":"2024-09-10T16:07:08.888002+0000","last_clean":"2024-09-10T16:07:08.888002+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:07:08.888002+0000","last_undegraded":"2024-09-10T16:07:08.888002+0000","last_fullsized":"2024-09-10T16:07:08.888002+0000","mapping_epoch":44,"log_start":"43'8000","ondisk_log_start":"43'8000","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11083800,"num_objects":2707,"num_object_clones":0,"num_object_copies":5414,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2707,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16095,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'18052","reported_seq":18138,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:07:10.728479+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:07:10.728479+0000","last_peered":"2024-09-10T16:07:10.728479+0000","last_clean":"2024-09-10T16:07:10.728479+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:07:10.728479+0000","last_undegraded":"2024-09-10T16:07:10.728479+0000","last_fullsized":"2024-09-10T16:07:10.728479+0000","mapping_epoch":35,"log_start":"43'8000","ondisk_log_start":"43'8000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10682368,"num_objects":2608,"num_object_clones":0,"num_object_copies":5216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2608,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18052,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'18287","reported_seq":18423,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:07:10.312262+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:07:10.312262+0000","last_peered":"2024-09-10T16:07:10.312262+0000","last_clean":"2024-09-10T16:07:10.312262+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:07:10.312262+0000","last_undegraded":"2024-09-10T16:07:10.312262+0000","last_fullsized":"2024-09-10T16:07:10.312262+0000","mapping_epoch":34,"log_start":"43'8200","ondisk_log_start":"43'8200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10666467,"num_objects":2605,"num_object_clones":0,"num_object_copies":5210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2605,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14998,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'18385","reported_seq":18529,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:10.212422+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:07:10.212422+0000","last_peered":"2024-09-10T16:07:10.212422+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:07:10.212422+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"43'8300","ondisk_log_start":"43'8300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10961379,"num_objects":2677,"num_object_clones":0,"num_object_copies":5354,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2677,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14997,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1694},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'18344","reported_seq":18488,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:10.302364+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:07:10.302364+0000","last_peered":"2024-09-10T16:07:10.302364+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:07:10.302364+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"43'8300","ondisk_log_start":"43'8300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11215814,"num_objects":2740,"num_object_clones":0,"num_object_copies":5480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2740,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":14843,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1497},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'18589","reported_seq":18725,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:07:10.312314+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:07:10.312314+0000","last_peered":"2024-09-10T16:07:10.312314+0000","last_clean":"2024-09-10T16:07:10.312314+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:07:10.312314+0000","last_undegraded":"2024-09-10T16:07:10.312314+0000","last_fullsized":"2024-09-10T16:07:10.312314+0000","mapping_epoch":34,"log_start":"43'8500","ondisk_log_start":"43'8500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11161600,"num_objects":2725,"num_object_clones":0,"num_object_copies":5450,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2725,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15436,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.876895+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:05:19.876895+0000","last_peered":"2024-09-10T16:05:19.876895+0000","last_clean":"2024-09-10T16:05:19.876895+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:05:19.876895+0000","last_undegraded":"2024-09-10T16:05:19.876895+0000","last_fullsized":"2024-09-10T16:05:19.876895+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'17678","reported_seq":17824,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:10.719777+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:07:10.719777+0000","last_peered":"2024-09-10T16:07:10.719777+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:07:10.719777+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"43'7600","ondisk_log_start":"43'7600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11383267,"num_objects":2780,"num_object_clones":0,"num_object_copies":5560,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2780,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15497,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-1343}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":100,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.875849+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:05:19.875849+0000","last_peered":"2024-09-10T16:05:19.875849+0000","last_clean":"2024-09-10T16:05:19.875849+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:05:19.875849+0000","last_undegraded":"2024-09-10T16:05:19.875849+0000","last_fullsized":"2024-09-10T16:05:19.875849+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'18209","reported_seq":18345,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:10.217419+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:07:10.217419+0000","last_peered":"2024-09-10T16:07:10.217419+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:07:10.217419+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"43'8200","ondisk_log_start":"43'8200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11063296,"num_objects":2701,"num_object_clones":0,"num_object_copies":5402,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2701,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14713,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1554}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":110,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.872023+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:05:19.872023+0000","last_peered":"2024-09-10T16:05:19.872023+0000","last_clean":"2024-09-10T16:05:19.872023+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:05:19.872023+0000","last_undegraded":"2024-09-10T16:05:19.872023+0000","last_fullsized":"2024-09-10T16:05:19.872023+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'18186","reported_seq":18323,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:10.828522+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:07:10.828522+0000","last_peered":"2024-09-10T16:07:10.828522+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:07:10.828522+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"43'8100","ondisk_log_start":"43'8100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10629603,"num_objects":2596,"num_object_clones":0,"num_object_copies":5192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2596,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15866,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-1708}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877148+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:05:19.877148+0000","last_peered":"2024-09-10T16:05:19.877148+0000","last_clean":"2024-09-10T16:05:19.877148+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:05:19.877148+0000","last_undegraded":"2024-09-10T16:05:19.877148+0000","last_fullsized":"2024-09-10T16:05:19.877148+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'18501","reported_seq":23775,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:10.839751+0000","last_change":"2024-09-10T16:05:18.271152+0000","last_active":"2024-09-10T16:07:10.839751+0000","last_peered":"2024-09-10T16:07:10.839751+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:07:10.839751+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"43'8500","ondisk_log_start":"43'8500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10932224,"num_objects":2669,"num_object_clones":0,"num_object_copies":5338,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1785,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2669,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16055,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2569,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":884}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'18187","reported_seq":18328,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:10.294249+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:07:10.294249+0000","last_peered":"2024-09-10T16:07:10.294249+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:07:10.294249+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"43'8100","ondisk_log_start":"43'8100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10858496,"num_objects":2651,"num_object_clones":0,"num_object_copies":5302,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2651,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14818,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1653}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877654+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:05:19.877654+0000","last_peered":"2024-09-10T16:05:19.877654+0000","last_clean":"2024-09-10T16:05:19.877654+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:05:19.877654+0000","last_undegraded":"2024-09-10T16:05:19.877654+0000","last_fullsized":"2024-09-10T16:05:19.877654+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'17885","reported_seq":26456,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:07:08.919954+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:07:08.919954+0000","last_peered":"2024-09-10T16:07:08.919954+0000","last_clean":"2024-09-10T16:07:08.919954+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:07:08.919954+0000","last_undegraded":"2024-09-10T16:07:08.919954+0000","last_fullsized":"2024-09-10T16:07:08.919954+0000","mapping_epoch":46,"log_start":"43'7800","ondisk_log_start":"43'7800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11112448,"num_objects":2713,"num_object_clones":0,"num_object_copies":5426,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2713,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14275,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":100,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.876043+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:05:19.876043+0000","last_peered":"2024-09-10T16:05:19.876043+0000","last_clean":"2024-09-10T16:05:19.876043+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:05:19.876043+0000","last_undegraded":"2024-09-10T16:05:19.876043+0000","last_fullsized":"2024-09-10T16:05:19.876043+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'18431","reported_seq":18575,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:10.293756+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:07:10.293756+0000","last_peered":"2024-09-10T16:07:10.293756+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:07:10.293756+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"43'8400","ondisk_log_start":"43'8400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10719232,"num_objects":2617,"num_object_clones":0,"num_object_copies":5234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2617,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14686,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1754}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877349+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:05:19.877349+0000","last_peered":"2024-09-10T16:05:19.877349+0000","last_clean":"2024-09-10T16:05:19.877349+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:05:19.877349+0000","last_undegraded":"2024-09-10T16:05:19.877349+0000","last_fullsized":"2024-09-10T16:05:19.877349+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877269+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:05:19.877269+0000","last_peered":"2024-09-10T16:05:19.877269+0000","last_clean":"2024-09-10T16:05:19.877269+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:05:19.877269+0000","last_undegraded":"2024-09-10T16:05:19.877269+0000","last_fullsized":"2024-09-10T16:05:19.877269+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'18481","reported_seq":18631,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:10.298339+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:07:10.298339+0000","last_peered":"2024-09-10T16:07:10.298339+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:07:10.298339+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"43'8400","ondisk_log_start":"43'8400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11129798,"num_objects":2719,"num_object_clones":0,"num_object_copies":5438,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2719,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14644,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1604},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":175145718,"num_objects":42770,"num_object_clones":0,"num_object_copies":85540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40342,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42770,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":249336,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11047,"num_bytes_recovered":33095680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":562343936,"data_stored":562202111,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161062,"ondisk_log_size":161062,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986286,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":561656,"kb_used_data":223144,"kb_used_omap":4,"kb_used_meta":338491,"kb_avail":93810184,"statfs":{"total":96636764160,"available":96061628416,"internally_reserved":0,"allocated":228499456,"data_stored":228196772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":346615466},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,2,3,1,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659231,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487872,"kb_used_data":131296,"kb_used_omap":4,"kb_used_meta":356539,"kb_avail":93883968,"statfs":{"total":96636764160,"available":96137183232,"internally_reserved":0,"allocated":134447104,"data_stored":134115052,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":365096622},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,0,2,4,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182514,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":639356,"kb_used_data":196776,"kb_used_omap":4,"kb_used_meta":442555,"kb_avail":93732484,"statfs":{"total":96636764160,"available":95982063616,"internally_reserved":0,"allocated":201498624,"data_stored":201195935,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":453177002},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,2,2,3,5,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":200867840,"data_stored":200831710,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":133738496,"data_stored":133673003,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":227737600,"data_stored":227697398,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:07:12.452 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-10T16:07:12.452 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "47'18076", 'reported_seq': 18157, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:07:10.293198+0000', 'last_change': '2024-09-10T15:59:16.281558+0000', 'last_active': '2024-09-10T16:07:10.293198+0000', 'last_peered': '2024-09-10T16:07:10.293198+0000', 'last_clean': '2024-09-10T15:59:14.132456+0000', 'last_became_active': '2024-09-10T15:59:16.274407+0000', 'last_became_peered': '2024-09-10T15:59:16.274407+0000', 'last_unstale': '2024-09-10T16:07:10.293198+0000', 'last_undegraded': '2024-09-10T15:59:15.851279+0000', 'last_fullsized': '2024-09-10T15:59:15.774356+0000', 'mapping_epoch': 42, 'log_start': "43'8000", 'ondisk_log_start': "43'8000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'objects_scrubbed': 0, 'log_size': 10076, 'log_dups_size': 0, 'ondisk_log_size': 10076, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 10633216, 'num_objects': 2596, 'num_object_clones': 0, 'num_object_copies': 5192, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4259, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2596, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18076, 'num_write_kb': 41344, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1663}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:07:12.453 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-10T16:07:12.453 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "47'18385", 'reported_seq': 18529, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:07:10.212422+0000', 'last_change': '2024-09-10T15:59:16.308264+0000', 'last_active': '2024-09-10T16:07:10.212422+0000', 'last_peered': '2024-09-10T16:07:10.212422+0000', 'last_clean': '2024-09-10T15:59:14.111551+0000', 'last_became_active': '2024-09-10T15:59:16.300040+0000', 'last_became_peered': '2024-09-10T15:59:16.300040+0000', 'last_unstale': '2024-09-10T16:07:10.212422+0000', 'last_undegraded': '2024-09-10T15:59:15.878234+0000', 'last_fullsized': '2024-09-10T15:59:15.769328+0000', 'mapping_epoch': 42, 'log_start': "43'8300", 'ondisk_log_start': "43'8300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3442", 'last_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'objects_scrubbed': 3384, 'log_size': 10085, 'log_dups_size': 0, 'ondisk_log_size': 10085, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1292, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 10961379, 'num_objects': 2677, 'num_object_clones': 0, 'num_object_copies': 5354, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4372, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2677, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 14997, 'num_write_kb': 28580, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1694}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:07:12.453 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-10T16:07:12.453 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "47'18344", 'reported_seq': 18488, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:07:10.302364+0000', 'last_change': '2024-09-10T15:59:16.331899+0000', 'last_active': '2024-09-10T16:07:10.302364+0000', 'last_peered': '2024-09-10T16:07:10.302364+0000', 'last_clean': '2024-09-10T15:59:14.139436+0000', 'last_became_active': '2024-09-10T15:59:16.318715+0000', 'last_became_peered': '2024-09-10T15:59:16.318715+0000', 'last_unstale': '2024-09-10T16:07:10.302364+0000', 'last_undegraded': '2024-09-10T15:59:15.877524+0000', 'last_fullsized': '2024-09-10T15:59:15.769042+0000', 'mapping_epoch': 42, 'log_start': "43'8300", 'ondisk_log_start': "43'8300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3537", 'last_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'objects_scrubbed': 3497, 'log_size': 10044, 'log_dups_size': 0, 'ondisk_log_size': 10044, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1224, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 11215814, 'num_objects': 2740, 'num_object_clones': 0, 'num_object_copies': 5480, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4237, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2740, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 14843, 'num_write_kb': 28172, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1497}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:07:12.453 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-10T16:07:12.453 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "47'17678", 'reported_seq': 17824, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:07:10.719777+0000', 'last_change': '2024-09-10T15:59:16.175680+0000', 'last_active': '2024-09-10T16:07:10.719777+0000', 'last_peered': '2024-09-10T16:07:10.719777+0000', 'last_clean': '2024-09-10T15:59:14.088187+0000', 'last_became_active': '2024-09-10T15:59:16.165810+0000', 'last_became_peered': '2024-09-10T15:59:16.165810+0000', 'last_unstale': '2024-09-10T16:07:10.719777+0000', 'last_undegraded': '2024-09-10T15:59:15.822098+0000', 'last_fullsized': '2024-09-10T15:59:15.769123+0000', 'mapping_epoch': 42, 'log_start': "43'7600", 'ondisk_log_start': "43'7600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2218", 'last_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'objects_scrubbed': 2175, 'log_size': 10078, 'log_dups_size': 0, 'ondisk_log_size': 10078, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 825, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 11383267, 'num_objects': 2780, 'num_object_clones': 0, 'num_object_copies': 5560, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4124, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2780, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 15497, 'num_write_kb': 32204, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': -1343}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:07:12.453 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-10T16:07:12.453 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "47'18209", 'reported_seq': 18345, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:07:10.217419+0000', 'last_change': '2024-09-10T15:59:16.277742+0000', 'last_active': '2024-09-10T16:07:10.217419+0000', 'last_peered': '2024-09-10T16:07:10.217419+0000', 'last_clean': '2024-09-10T15:59:14.129081+0000', 'last_became_active': '2024-09-10T15:59:16.269835+0000', 'last_became_peered': '2024-09-10T15:59:16.269835+0000', 'last_unstale': '2024-09-10T16:07:10.217419+0000', 'last_undegraded': '2024-09-10T15:59:15.860857+0000', 'last_fullsized': '2024-09-10T15:59:15.771410+0000', 'mapping_epoch': 42, 'log_start': "43'8200", 'ondisk_log_start': "43'8200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3557", 'last_scrub_stamp': '2024-09-10T15:59:02.493678+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:02.493678+0000', 'objects_scrubbed': 3488, 'log_size': 10009, 'log_dups_size': 0, 'ondisk_log_size': 10009, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1472, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 11063296, 'num_objects': 2701, 'num_object_clones': 0, 'num_object_copies': 5402, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4257, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2701, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14713, 'num_write_kb': 27852, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1554}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:07:12.453 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-10T16:07:12.453 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "47'18186", 'reported_seq': 18323, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:07:10.828522+0000', 'last_change': '2024-09-10T15:59:16.284311+0000', 'last_active': '2024-09-10T16:07:10.828522+0000', 'last_peered': '2024-09-10T16:07:10.828522+0000', 'last_clean': '2024-09-10T15:59:14.116041+0000', 'last_became_active': '2024-09-10T15:59:16.274543+0000', 'last_became_peered': '2024-09-10T15:59:16.274543+0000', 'last_unstale': '2024-09-10T16:07:10.828522+0000', 'last_undegraded': '2024-09-10T15:59:15.812484+0000', 'last_fullsized': '2024-09-10T15:59:15.770885+0000', 'mapping_epoch': 42, 'log_start': "43'8100", 'ondisk_log_start': "43'8100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2356", 'last_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'objects_scrubbed': 2316, 'log_size': 10086, 'log_dups_size': 0, 'ondisk_log_size': 10086, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 783, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 10629603, 'num_objects': 2596, 'num_object_clones': 0, 'num_object_copies': 5192, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2596, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 15866, 'num_write_kb': 32292, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': -1708}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:07:12.453 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-10T16:07:12.454 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "47'18501", 'reported_seq': 23775, 'reported_epoch': 47, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:07:10.839751+0000', 'last_change': '2024-09-10T16:05:18.271152+0000', 'last_active': '2024-09-10T16:07:10.839751+0000', 'last_peered': '2024-09-10T16:07:10.839751+0000', 'last_clean': '2024-09-10T15:59:14.110654+0000', 'last_became_active': '2024-09-10T15:59:16.164645+0000', 'last_became_peered': '2024-09-10T15:59:16.164645+0000', 'last_unstale': '2024-09-10T16:07:10.839751+0000', 'last_undegraded': '2024-09-10T15:59:15.803593+0000', 'last_fullsized': '2024-09-10T15:59:15.769083+0000', 'mapping_epoch': 42, 'log_start': "43'8500", 'ondisk_log_start': "43'8500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2486", 'last_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'objects_scrubbed': 2440, 'log_size': 10001, 'log_dups_size': 0, 'ondisk_log_size': 10001, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 792, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 10932224, 'num_objects': 2669, 'num_object_clones': 0, 'num_object_copies': 5338, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1785, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2669, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16055, 'num_write_kb': 32568, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2569, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 884}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:07:12.454 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-10T16:07:12.454 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "47'18187", 'reported_seq': 18328, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:07:10.294249+0000', 'last_change': '2024-09-10T15:59:16.315641+0000', 'last_active': '2024-09-10T16:07:10.294249+0000', 'last_peered': '2024-09-10T16:07:10.294249+0000', 'last_clean': '2024-09-10T15:59:14.117001+0000', 'last_became_active': '2024-09-10T15:59:16.302245+0000', 'last_became_peered': '2024-09-10T15:59:16.302245+0000', 'last_unstale': '2024-09-10T16:07:10.294249+0000', 'last_undegraded': '2024-09-10T15:59:15.845012+0000', 'last_fullsized': '2024-09-10T15:59:15.770337+0000', 'mapping_epoch': 42, 'log_start': "43'8100", 'ondisk_log_start': "43'8100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3404", 'last_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'objects_scrubbed': 3367, 'log_size': 10087, 'log_dups_size': 0, 'ondisk_log_size': 10087, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 983, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 10858496, 'num_objects': 2651, 'num_object_clones': 0, 'num_object_copies': 5302, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2651, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14818, 'num_write_kb': 28204, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1653}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:07:12.454 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-10T16:07:12.454 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "47'18431", 'reported_seq': 18575, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:07:10.293756+0000', 'last_change': '2024-09-10T15:59:16.311291+0000', 'last_active': '2024-09-10T16:07:10.293756+0000', 'last_peered': '2024-09-10T16:07:10.293756+0000', 'last_clean': '2024-09-10T15:59:14.116363+0000', 'last_became_active': '2024-09-10T15:59:16.301244+0000', 'last_became_peered': '2024-09-10T15:59:16.301244+0000', 'last_unstale': '2024-09-10T16:07:10.293756+0000', 'last_undegraded': '2024-09-10T15:59:15.883434+0000', 'last_fullsized': '2024-09-10T15:59:15.772756+0000', 'mapping_epoch': 42, 'log_start': "43'8400", 'ondisk_log_start': "43'8400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3789", 'last_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'objects_scrubbed': 3741, 'log_size': 10031, 'log_dups_size': 0, 'ondisk_log_size': 10031, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1440, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 10719232, 'num_objects': 2617, 'num_object_clones': 0, 'num_object_copies': 5234, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4373, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2617, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14686, 'num_write_kb': 27124, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1754}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:07:12.454 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-10T16:07:12.454 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "47'18481", 'reported_seq': 18631, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:07:10.298339+0000', 'last_change': '2024-09-10T15:59:16.313409+0000', 'last_active': '2024-09-10T16:07:10.298339+0000', 'last_peered': '2024-09-10T16:07:10.298339+0000', 'last_clean': '2024-09-10T15:59:14.129780+0000', 'last_became_active': '2024-09-10T15:59:16.300932+0000', 'last_became_peered': '2024-09-10T15:59:16.300932+0000', 'last_unstale': '2024-09-10T16:07:10.298339+0000', 'last_undegraded': '2024-09-10T15:59:15.887823+0000', 'last_fullsized': '2024-09-10T15:59:15.768513+0000', 'mapping_epoch': 42, 'log_start': "43'8400", 'ondisk_log_start': "43'8400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3880", 'last_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'objects_scrubbed': 3827, 'log_size': 10081, 'log_dups_size': 0, 'ondisk_log_size': 10081, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1397, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 11129798, 'num_objects': 2719, 'num_object_clones': 0, 'num_object_copies': 5438, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4325, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2719, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14644, 'num_write_kb': 27072, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1604}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:07:12.454 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T16:07:13.156 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:07:13.157 INFO:teuthology.orchestra.run.smithi115.stdout:{"fsid":"7effee29-df05-4351-9741-ee203564ed81","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 40342/85544 objects degraded (47.159%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":597,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":47,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983954,"num_in_osds":3,"osd_in_since":1725983947,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":42772,"data_bytes":175145737,"bytes_used":1729417216,"bytes_avail":288180875264,"bytes_total":289910292480,"degraded_objects":40342,"degraded_total":85544,"degraded_ratio":0.47159356588422324,"recovering_objects_per_sec":34,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":412,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":554},"fsmap":{"epoch":1,"btime":"2024-09-10T15:57:14:967120+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T16:00:13.560776+0000","services":{}},"progress_events":{}} 2024-09-10T16:07:13.189 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-10T16:07:13.190 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:07:13.753 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:07:13.755 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:07:13.776 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":339,"stamp":"2024-09-10T16:07:13.734853+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":175145737,"num_objects":42772,"num_object_clones":0,"num_object_copies":85544,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40342,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42772,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":249338,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11049,"num_bytes_recovered":33095699,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161065,"ondisk_log_size":161065,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1688884,"kb_used_data":551216,"kb_used_omap":12,"kb_used_meta":1137587,"kb_avail":281426636,"statfs":{"total":289910292480,"available":288180875264,"internally_reserved":0,"allocated":564445184,"data_stored":563507759,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1164889090},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,5,4,8,10,9],"upper_bound":256},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19025920,"num_objects":-4645,"num_object_clones":0,"num_object_copies":-9290,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-283,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4645,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4645,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":283,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006187"},"pg_stats":[{"pgid":"2.f","version":"47'18076","reported_seq":18157,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:10.293198+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:07:10.293198+0000","last_peered":"2024-09-10T16:07:10.293198+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:07:10.293198+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"43'8000","ondisk_log_start":"43'8000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10633216,"num_objects":2596,"num_object_clones":0,"num_object_copies":5192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2596,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18076,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1663}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'18180","reported_seq":18298,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:07:10.791532+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:07:10.791532+0000","last_peered":"2024-09-10T16:07:10.791532+0000","last_clean":"2024-09-10T16:07:10.791532+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:07:10.791532+0000","last_undegraded":"2024-09-10T16:07:10.791532+0000","last_fullsized":"2024-09-10T16:07:10.791532+0000","mapping_epoch":35,"log_start":"43'8100","ondisk_log_start":"43'8100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10912710,"num_objects":2666,"num_object_clones":0,"num_object_copies":5332,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2666,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16285,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'18091","reported_seq":26775,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:07:08.888002+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:07:08.888002+0000","last_peered":"2024-09-10T16:07:08.888002+0000","last_clean":"2024-09-10T16:07:08.888002+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:07:08.888002+0000","last_undegraded":"2024-09-10T16:07:08.888002+0000","last_fullsized":"2024-09-10T16:07:08.888002+0000","mapping_epoch":44,"log_start":"43'8000","ondisk_log_start":"43'8000","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11083800,"num_objects":2707,"num_object_clones":0,"num_object_copies":5414,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2707,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16095,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'18052","reported_seq":18138,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:07:10.728479+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:07:10.728479+0000","last_peered":"2024-09-10T16:07:10.728479+0000","last_clean":"2024-09-10T16:07:10.728479+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:07:10.728479+0000","last_undegraded":"2024-09-10T16:07:10.728479+0000","last_fullsized":"2024-09-10T16:07:10.728479+0000","mapping_epoch":35,"log_start":"43'8000","ondisk_log_start":"43'8000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10682368,"num_objects":2608,"num_object_clones":0,"num_object_copies":5216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2608,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18052,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'18287","reported_seq":18423,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:07:10.312262+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:07:10.312262+0000","last_peered":"2024-09-10T16:07:10.312262+0000","last_clean":"2024-09-10T16:07:10.312262+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:07:10.312262+0000","last_undegraded":"2024-09-10T16:07:10.312262+0000","last_fullsized":"2024-09-10T16:07:10.312262+0000","mapping_epoch":34,"log_start":"43'8200","ondisk_log_start":"43'8200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10666467,"num_objects":2605,"num_object_clones":0,"num_object_copies":5210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2605,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14998,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'18385","reported_seq":18529,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:10.212422+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:07:10.212422+0000","last_peered":"2024-09-10T16:07:10.212422+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:07:10.212422+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"43'8300","ondisk_log_start":"43'8300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10961379,"num_objects":2677,"num_object_clones":0,"num_object_copies":5354,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2677,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14997,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1694},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'18344","reported_seq":18488,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:10.302364+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:07:10.302364+0000","last_peered":"2024-09-10T16:07:10.302364+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:07:10.302364+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"43'8300","ondisk_log_start":"43'8300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11215814,"num_objects":2740,"num_object_clones":0,"num_object_copies":5480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2740,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":14843,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1497},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'18589","reported_seq":18725,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:07:10.312314+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:07:10.312314+0000","last_peered":"2024-09-10T16:07:10.312314+0000","last_clean":"2024-09-10T16:07:10.312314+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:07:10.312314+0000","last_undegraded":"2024-09-10T16:07:10.312314+0000","last_fullsized":"2024-09-10T16:07:10.312314+0000","mapping_epoch":34,"log_start":"43'8500","ondisk_log_start":"43'8500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11161600,"num_objects":2725,"num_object_clones":0,"num_object_copies":5450,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2725,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15436,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.876895+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:05:19.876895+0000","last_peered":"2024-09-10T16:05:19.876895+0000","last_clean":"2024-09-10T16:05:19.876895+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:05:19.876895+0000","last_undegraded":"2024-09-10T16:05:19.876895+0000","last_fullsized":"2024-09-10T16:05:19.876895+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'17678","reported_seq":17824,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:10.719777+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:07:10.719777+0000","last_peered":"2024-09-10T16:07:10.719777+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:07:10.719777+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"43'7600","ondisk_log_start":"43'7600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11383267,"num_objects":2780,"num_object_clones":0,"num_object_copies":5560,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2780,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15497,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-1343}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":100,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.875849+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:05:19.875849+0000","last_peered":"2024-09-10T16:05:19.875849+0000","last_clean":"2024-09-10T16:05:19.875849+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:05:19.875849+0000","last_undegraded":"2024-09-10T16:05:19.875849+0000","last_fullsized":"2024-09-10T16:05:19.875849+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'18209","reported_seq":18345,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:10.217419+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:07:10.217419+0000","last_peered":"2024-09-10T16:07:10.217419+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:07:10.217419+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"43'8200","ondisk_log_start":"43'8200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11063296,"num_objects":2701,"num_object_clones":0,"num_object_copies":5402,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2701,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14713,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1554}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":110,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.872023+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:05:19.872023+0000","last_peered":"2024-09-10T16:05:19.872023+0000","last_clean":"2024-09-10T16:05:19.872023+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:05:19.872023+0000","last_undegraded":"2024-09-10T16:05:19.872023+0000","last_fullsized":"2024-09-10T16:05:19.872023+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'18186","reported_seq":18323,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:10.828522+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:07:10.828522+0000","last_peered":"2024-09-10T16:07:10.828522+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:07:10.828522+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"43'8100","ondisk_log_start":"43'8100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10629603,"num_objects":2596,"num_object_clones":0,"num_object_copies":5192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2596,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15866,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-1708}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877148+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:05:19.877148+0000","last_peered":"2024-09-10T16:05:19.877148+0000","last_clean":"2024-09-10T16:05:19.877148+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:05:19.877148+0000","last_undegraded":"2024-09-10T16:05:19.877148+0000","last_fullsized":"2024-09-10T16:05:19.877148+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'18501","reported_seq":23775,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:10.839751+0000","last_change":"2024-09-10T16:05:18.271152+0000","last_active":"2024-09-10T16:07:10.839751+0000","last_peered":"2024-09-10T16:07:10.839751+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:07:10.839751+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"43'8500","ondisk_log_start":"43'8500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10932224,"num_objects":2669,"num_object_clones":0,"num_object_copies":5338,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1785,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2669,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16055,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2569,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":884}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'18187","reported_seq":18328,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:10.294249+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:07:10.294249+0000","last_peered":"2024-09-10T16:07:10.294249+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:07:10.294249+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"43'8100","ondisk_log_start":"43'8100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10858496,"num_objects":2651,"num_object_clones":0,"num_object_copies":5302,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2651,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14818,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1653}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877654+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:05:19.877654+0000","last_peered":"2024-09-10T16:05:19.877654+0000","last_clean":"2024-09-10T16:05:19.877654+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:05:19.877654+0000","last_undegraded":"2024-09-10T16:05:19.877654+0000","last_fullsized":"2024-09-10T16:05:19.877654+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'17885","reported_seq":26456,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:07:08.919954+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:07:08.919954+0000","last_peered":"2024-09-10T16:07:08.919954+0000","last_clean":"2024-09-10T16:07:08.919954+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:07:08.919954+0000","last_undegraded":"2024-09-10T16:07:08.919954+0000","last_fullsized":"2024-09-10T16:07:08.919954+0000","mapping_epoch":46,"log_start":"43'7800","ondisk_log_start":"43'7800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11112448,"num_objects":2713,"num_object_clones":0,"num_object_copies":5426,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2713,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14275,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":100,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.876043+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:05:19.876043+0000","last_peered":"2024-09-10T16:05:19.876043+0000","last_clean":"2024-09-10T16:05:19.876043+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:05:19.876043+0000","last_undegraded":"2024-09-10T16:05:19.876043+0000","last_fullsized":"2024-09-10T16:05:19.876043+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'18431","reported_seq":18575,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:10.293756+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:07:10.293756+0000","last_peered":"2024-09-10T16:07:10.293756+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:07:10.293756+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"43'8400","ondisk_log_start":"43'8400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10719232,"num_objects":2617,"num_object_clones":0,"num_object_copies":5234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2617,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14686,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1754}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877349+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:05:19.877349+0000","last_peered":"2024-09-10T16:05:19.877349+0000","last_clean":"2024-09-10T16:05:19.877349+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:05:19.877349+0000","last_undegraded":"2024-09-10T16:05:19.877349+0000","last_fullsized":"2024-09-10T16:05:19.877349+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877269+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:05:19.877269+0000","last_peered":"2024-09-10T16:05:19.877269+0000","last_clean":"2024-09-10T16:05:19.877269+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:05:19.877269+0000","last_undegraded":"2024-09-10T16:05:19.877269+0000","last_fullsized":"2024-09-10T16:05:19.877269+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'18481","reported_seq":18631,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:10.298339+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:07:10.298339+0000","last_peered":"2024-09-10T16:07:10.298339+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:07:10.298339+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"43'8400","ondisk_log_start":"43'8400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11129798,"num_objects":2719,"num_object_clones":0,"num_object_copies":5438,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2719,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14644,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1604},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":175145718,"num_objects":42770,"num_object_clones":0,"num_object_copies":85540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40342,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42770,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":249336,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11047,"num_bytes_recovered":33095680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":562343936,"data_stored":562202111,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161062,"ondisk_log_size":161062,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986286,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":561656,"kb_used_data":223144,"kb_used_omap":4,"kb_used_meta":338491,"kb_avail":93810184,"statfs":{"total":96636764160,"available":96061628416,"internally_reserved":0,"allocated":228499456,"data_stored":228196772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":346615466},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,2,3,1,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659231,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487872,"kb_used_data":131296,"kb_used_omap":4,"kb_used_meta":356539,"kb_avail":93883968,"statfs":{"total":96636764160,"available":96137183232,"internally_reserved":0,"allocated":134447104,"data_stored":134115052,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":365096622},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,0,2,4,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182514,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":639356,"kb_used_data":196776,"kb_used_omap":4,"kb_used_meta":442555,"kb_avail":93732484,"statfs":{"total":96636764160,"available":95982063616,"internally_reserved":0,"allocated":201498624,"data_stored":201195935,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":453177002},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,2,2,3,5,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":200867840,"data_stored":200831710,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":133738496,"data_stored":133673003,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":227737600,"data_stored":227697398,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:07:16.778 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:07:17.357 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:07:17.357 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:07:17.382 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":340,"stamp":"2024-09-10T16:07:15.735277+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":168772361,"num_objects":41216,"num_object_clones":0,"num_object_copies":82432,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40342,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41216,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":250894,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11049,"num_bytes_recovered":33095699,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160621,"ondisk_log_size":160621,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1687456,"kb_used_data":537320,"kb_used_omap":12,"kb_used_meta":1150067,"kb_avail":281428064,"statfs":{"total":289910292480,"available":288182337536,"internally_reserved":0,"allocated":550215680,"data_stored":549278255,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1177668610},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,7,4,2,12,9,6],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-25399296,"num_objects":-6201,"num_object_clones":0,"num_object_copies":-12402,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-283,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-6201,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6201,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":283,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005835"},"pg_stats":[{"pgid":"2.f","version":"47'18217","reported_seq":18298,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:15.325114+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:07:15.325114+0000","last_peered":"2024-09-10T16:07:15.325114+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:07:15.325114+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"43'8200","ondisk_log_start":"43'8200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10055680,"num_objects":2455,"num_object_clones":0,"num_object_copies":4910,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2455,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18217,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1804}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'18180","reported_seq":18298,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:07:10.791532+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:07:10.791532+0000","last_peered":"2024-09-10T16:07:10.791532+0000","last_clean":"2024-09-10T16:07:10.791532+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:07:10.791532+0000","last_undegraded":"2024-09-10T16:07:10.791532+0000","last_fullsized":"2024-09-10T16:07:10.791532+0000","mapping_epoch":35,"log_start":"43'8100","ondisk_log_start":"43'8100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10912710,"num_objects":2666,"num_object_clones":0,"num_object_copies":5332,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2666,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16285,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'18227","reported_seq":26911,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:07:13.966405+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:07:13.966405+0000","last_peered":"2024-09-10T16:07:13.966405+0000","last_clean":"2024-09-10T16:07:13.966405+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:07:13.966405+0000","last_undegraded":"2024-09-10T16:07:13.966405+0000","last_fullsized":"2024-09-10T16:07:13.966405+0000","mapping_epoch":44,"log_start":"43'8200","ondisk_log_start":"43'8200","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10526744,"num_objects":2571,"num_object_clones":0,"num_object_copies":5142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2571,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16231,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'18052","reported_seq":18138,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:07:10.728479+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:07:10.728479+0000","last_peered":"2024-09-10T16:07:10.728479+0000","last_clean":"2024-09-10T16:07:10.728479+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:07:10.728479+0000","last_undegraded":"2024-09-10T16:07:10.728479+0000","last_fullsized":"2024-09-10T16:07:10.728479+0000","mapping_epoch":35,"log_start":"43'8000","ondisk_log_start":"43'8000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10682368,"num_objects":2608,"num_object_clones":0,"num_object_copies":5216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2608,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18052,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'18408","reported_seq":18544,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:07:15.251518+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:07:15.251518+0000","last_peered":"2024-09-10T16:07:15.251518+0000","last_clean":"2024-09-10T16:07:15.251518+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:07:15.251518+0000","last_undegraded":"2024-09-10T16:07:15.251518+0000","last_fullsized":"2024-09-10T16:07:15.251518+0000","mapping_epoch":34,"log_start":"43'8400","ondisk_log_start":"43'8400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10170851,"num_objects":2484,"num_object_clones":0,"num_object_copies":4968,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2484,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15119,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'18525","reported_seq":18668,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:15.323983+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:07:15.323983+0000","last_peered":"2024-09-10T16:07:15.323983+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:07:15.323983+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"43'8500","ondisk_log_start":"43'8500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10387939,"num_objects":2537,"num_object_clones":0,"num_object_copies":5074,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2537,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15137,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1834},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'18520","reported_seq":18664,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:15.344777+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:07:15.344777+0000","last_peered":"2024-09-10T16:07:15.344777+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:07:15.344777+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"43'8500","ondisk_log_start":"43'8500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10494918,"num_objects":2564,"num_object_clones":0,"num_object_copies":5128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2564,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":15019,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1673},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'18731","reported_seq":18864,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:07:15.295019+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:07:15.295019+0000","last_peered":"2024-09-10T16:07:15.295019+0000","last_clean":"2024-09-10T16:07:15.295019+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:07:15.295019+0000","last_undegraded":"2024-09-10T16:07:15.295019+0000","last_fullsized":"2024-09-10T16:07:15.295019+0000","mapping_epoch":34,"log_start":"43'8700","ondisk_log_start":"43'8700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10579968,"num_objects":2583,"num_object_clones":0,"num_object_copies":5166,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2583,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15578,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.876895+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:05:19.876895+0000","last_peered":"2024-09-10T16:05:19.876895+0000","last_clean":"2024-09-10T16:05:19.876895+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:05:19.876895+0000","last_undegraded":"2024-09-10T16:05:19.876895+0000","last_fullsized":"2024-09-10T16:05:19.876895+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'17678","reported_seq":17824,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:10.719777+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:07:10.719777+0000","last_peered":"2024-09-10T16:07:10.719777+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:07:10.719777+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"43'7600","ondisk_log_start":"43'7600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11383267,"num_objects":2780,"num_object_clones":0,"num_object_copies":5560,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2780,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15497,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-1343}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":100,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.875849+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:05:19.875849+0000","last_peered":"2024-09-10T16:05:19.875849+0000","last_clean":"2024-09-10T16:05:19.875849+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:05:19.875849+0000","last_undegraded":"2024-09-10T16:05:19.875849+0000","last_fullsized":"2024-09-10T16:05:19.875849+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'18360","reported_seq":18494,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:15.365225+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:07:15.365225+0000","last_peered":"2024-09-10T16:07:15.365225+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:07:15.365225+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"43'8300","ondisk_log_start":"43'8300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10444800,"num_objects":2550,"num_object_clones":0,"num_object_copies":5100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2550,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14864,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1705}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":110,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.872023+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:05:19.872023+0000","last_peered":"2024-09-10T16:05:19.872023+0000","last_clean":"2024-09-10T16:05:19.872023+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:05:19.872023+0000","last_undegraded":"2024-09-10T16:05:19.872023+0000","last_fullsized":"2024-09-10T16:05:19.872023+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'18186","reported_seq":18323,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:10.828522+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:07:10.828522+0000","last_peered":"2024-09-10T16:07:10.828522+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:07:10.828522+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"43'8100","ondisk_log_start":"43'8100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10629603,"num_objects":2596,"num_object_clones":0,"num_object_copies":5192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2596,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15866,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-1708}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877148+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:05:19.877148+0000","last_peered":"2024-09-10T16:05:19.877148+0000","last_clean":"2024-09-10T16:05:19.877148+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:05:19.877148+0000","last_undegraded":"2024-09-10T16:05:19.877148+0000","last_fullsized":"2024-09-10T16:05:19.877148+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'18501","reported_seq":23775,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:10.839751+0000","last_change":"2024-09-10T16:05:18.271152+0000","last_active":"2024-09-10T16:07:10.839751+0000","last_peered":"2024-09-10T16:07:10.839751+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:07:10.839751+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"43'8500","ondisk_log_start":"43'8500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10932224,"num_objects":2669,"num_object_clones":0,"num_object_copies":5338,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1785,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2669,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16055,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2569,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":884}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'18327","reported_seq":18468,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:15.324212+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:07:15.324212+0000","last_peered":"2024-09-10T16:07:15.324212+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:07:15.324212+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"43'8300","ondisk_log_start":"43'8300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10285056,"num_objects":2511,"num_object_clones":0,"num_object_copies":5022,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2511,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14958,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1793}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877654+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:05:19.877654+0000","last_peered":"2024-09-10T16:05:19.877654+0000","last_clean":"2024-09-10T16:05:19.877654+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:05:19.877654+0000","last_undegraded":"2024-09-10T16:05:19.877654+0000","last_fullsized":"2024-09-10T16:05:19.877654+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'18034","reported_seq":26605,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:07:13.952858+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:07:13.952858+0000","last_peered":"2024-09-10T16:07:13.952858+0000","last_clean":"2024-09-10T16:07:13.952858+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:07:13.952858+0000","last_undegraded":"2024-09-10T16:07:13.952858+0000","last_fullsized":"2024-09-10T16:07:13.952858+0000","mapping_epoch":46,"log_start":"43'8000","ondisk_log_start":"43'8000","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10502144,"num_objects":2564,"num_object_clones":0,"num_object_copies":5128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2564,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14424,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":100,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.876043+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:05:19.876043+0000","last_peered":"2024-09-10T16:05:19.876043+0000","last_clean":"2024-09-10T16:05:19.876043+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:05:19.876043+0000","last_undegraded":"2024-09-10T16:05:19.876043+0000","last_fullsized":"2024-09-10T16:05:19.876043+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'18562","reported_seq":18708,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:15.315002+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:07:15.315002+0000","last_peered":"2024-09-10T16:07:15.315002+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:07:15.315002+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"43'8500","ondisk_log_start":"43'8500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10182656,"num_objects":2486,"num_object_clones":0,"num_object_copies":4972,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2486,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14817,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1885}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877349+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:05:19.877349+0000","last_peered":"2024-09-10T16:05:19.877349+0000","last_clean":"2024-09-10T16:05:19.877349+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:05:19.877349+0000","last_undegraded":"2024-09-10T16:05:19.877349+0000","last_fullsized":"2024-09-10T16:05:19.877349+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877269+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:05:19.877269+0000","last_peered":"2024-09-10T16:05:19.877269+0000","last_clean":"2024-09-10T16:05:19.877269+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:05:19.877269+0000","last_undegraded":"2024-09-10T16:05:19.877269+0000","last_fullsized":"2024-09-10T16:05:19.877269+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'18610","reported_seq":18760,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:15.330095+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:07:15.330095+0000","last_peered":"2024-09-10T16:07:15.330095+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:07:15.330095+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"43'8600","ondisk_log_start":"43'8600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10601414,"num_objects":2590,"num_object_clones":0,"num_object_copies":5180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2590,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14773,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1733},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":168772342,"num_objects":41214,"num_object_clones":0,"num_object_copies":82428,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40342,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41214,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":250892,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11047,"num_bytes_recovered":33095680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":548126720,"data_stored":547984895,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160618,"ondisk_log_size":160618,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986286,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":561656,"kb_used_data":223144,"kb_used_omap":4,"kb_used_meta":338491,"kb_avail":93810184,"statfs":{"total":96636764160,"available":96061628416,"internally_reserved":0,"allocated":228499456,"data_stored":228196772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":346615466},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,2,3,1,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659232,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":484916,"kb_used_data":124188,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93886924,"statfs":{"total":96636764160,"available":96140210176,"internally_reserved":0,"allocated":127168512,"data_stored":126836460,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":369356462},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,0,7,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182515,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":640884,"kb_used_data":189988,"kb_used_omap":4,"kb_used_meta":450875,"kb_avail":93730956,"statfs":{"total":96636764160,"available":95980498944,"internally_reserved":0,"allocated":194547712,"data_stored":194245023,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":461696682},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,7,1,0,2,3,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":193871872,"data_stored":193835742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":126517248,"data_stored":126451755,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":227737600,"data_stored":227697398,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:07:17.383 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:07:17.947 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:07:17.949 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:07:17.973 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":341,"stamp":"2024-09-10T16:07:17.737545+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":165782281,"num_objects":40486,"num_object_clones":0,"num_object_copies":80972,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40203,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40486,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":251624,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11188,"num_bytes_recovered":33095699,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160451,"ondisk_log_size":160451,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1687012,"kb_used_data":532712,"kb_used_omap":12,"kb_used_meta":1154227,"kb_avail":281428508,"statfs":{"total":289910292480,"available":288182792192,"internally_reserved":0,"allocated":545497088,"data_stored":544559663,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1181928450},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,9,5,2,12,8,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-21585920,"num_objects":-5270,"num_object_clones":0,"num_object_copies":-10540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-422,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5270,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5270,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":422,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.007420"},"pg_stats":[{"pgid":"2.f","version":"47'18217","reported_seq":18298,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:15.325114+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:07:15.325114+0000","last_peered":"2024-09-10T16:07:15.325114+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:07:15.325114+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"43'8200","ondisk_log_start":"43'8200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10055680,"num_objects":2455,"num_object_clones":0,"num_object_copies":4910,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2455,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18217,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1804}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'18318","reported_seq":18436,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:07:15.729716+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:07:15.729716+0000","last_peered":"2024-09-10T16:07:15.729716+0000","last_clean":"2024-09-10T16:07:15.729716+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:07:15.729716+0000","last_undegraded":"2024-09-10T16:07:15.729716+0000","last_fullsized":"2024-09-10T16:07:15.729716+0000","mapping_epoch":35,"log_start":"43'8300","ondisk_log_start":"43'8300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10347462,"num_objects":2528,"num_object_clones":0,"num_object_copies":5056,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2528,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16423,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'18227","reported_seq":26911,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:07:13.966405+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:07:13.966405+0000","last_peered":"2024-09-10T16:07:13.966405+0000","last_clean":"2024-09-10T16:07:13.966405+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:07:13.966405+0000","last_undegraded":"2024-09-10T16:07:13.966405+0000","last_fullsized":"2024-09-10T16:07:13.966405+0000","mapping_epoch":44,"log_start":"43'8200","ondisk_log_start":"43'8200","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10526744,"num_objects":2571,"num_object_clones":0,"num_object_copies":5142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2571,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16231,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'18204","reported_seq":18290,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:07:15.801159+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:07:15.801159+0000","last_peered":"2024-09-10T16:07:15.801159+0000","last_clean":"2024-09-10T16:07:15.801159+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:07:15.801159+0000","last_undegraded":"2024-09-10T16:07:15.801159+0000","last_fullsized":"2024-09-10T16:07:15.801159+0000","mapping_epoch":35,"log_start":"43'8200","ondisk_log_start":"43'8200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10059776,"num_objects":2456,"num_object_clones":0,"num_object_copies":4912,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2456,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18204,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'18408","reported_seq":18544,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:07:15.251518+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:07:15.251518+0000","last_peered":"2024-09-10T16:07:15.251518+0000","last_clean":"2024-09-10T16:07:15.251518+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:07:15.251518+0000","last_undegraded":"2024-09-10T16:07:15.251518+0000","last_fullsized":"2024-09-10T16:07:15.251518+0000","mapping_epoch":34,"log_start":"43'8400","ondisk_log_start":"43'8400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10170851,"num_objects":2484,"num_object_clones":0,"num_object_copies":4968,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2484,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15119,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'18525","reported_seq":18668,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:15.323983+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:07:15.323983+0000","last_peered":"2024-09-10T16:07:15.323983+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:07:15.323983+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"43'8500","ondisk_log_start":"43'8500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10387939,"num_objects":2537,"num_object_clones":0,"num_object_copies":5074,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2537,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15137,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1834},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'18520","reported_seq":18664,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:15.344777+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:07:15.344777+0000","last_peered":"2024-09-10T16:07:15.344777+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:07:15.344777+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"43'8500","ondisk_log_start":"43'8500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10494918,"num_objects":2564,"num_object_clones":0,"num_object_copies":5128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2564,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":15019,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1673},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'18731","reported_seq":18864,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:07:15.295019+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:07:15.295019+0000","last_peered":"2024-09-10T16:07:15.295019+0000","last_clean":"2024-09-10T16:07:15.295019+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:07:15.295019+0000","last_undegraded":"2024-09-10T16:07:15.295019+0000","last_fullsized":"2024-09-10T16:07:15.295019+0000","mapping_epoch":34,"log_start":"43'8700","ondisk_log_start":"43'8700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10579968,"num_objects":2583,"num_object_clones":0,"num_object_copies":5166,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2583,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15578,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.876895+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:05:19.876895+0000","last_peered":"2024-09-10T16:05:19.876895+0000","last_clean":"2024-09-10T16:05:19.876895+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:05:19.876895+0000","last_undegraded":"2024-09-10T16:05:19.876895+0000","last_fullsized":"2024-09-10T16:05:19.876895+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'17836","reported_seq":17982,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:15.748935+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:07:15.748935+0000","last_peered":"2024-09-10T16:07:15.748935+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:07:15.748935+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"43'7800","ondisk_log_start":"43'7800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10736099,"num_objects":2622,"num_object_clones":0,"num_object_copies":5244,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2622,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15655,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-1501}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":100,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.875849+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:05:19.875849+0000","last_peered":"2024-09-10T16:05:19.875849+0000","last_clean":"2024-09-10T16:05:19.875849+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:05:19.875849+0000","last_undegraded":"2024-09-10T16:05:19.875849+0000","last_fullsized":"2024-09-10T16:05:19.875849+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'18360","reported_seq":18494,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:15.365225+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:07:15.365225+0000","last_peered":"2024-09-10T16:07:15.365225+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:07:15.365225+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"43'8300","ondisk_log_start":"43'8300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10444800,"num_objects":2550,"num_object_clones":0,"num_object_copies":5100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2550,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14864,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1705}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":110,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.872023+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:05:19.872023+0000","last_peered":"2024-09-10T16:05:19.872023+0000","last_clean":"2024-09-10T16:05:19.872023+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:05:19.872023+0000","last_undegraded":"2024-09-10T16:05:19.872023+0000","last_fullsized":"2024-09-10T16:05:19.872023+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'18325","reported_seq":18466,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:15.748857+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:07:15.748857+0000","last_peered":"2024-09-10T16:07:15.748857+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:07:15.748857+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"43'8300","ondisk_log_start":"43'8300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10060259,"num_objects":2457,"num_object_clones":0,"num_object_copies":4914,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2457,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16005,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-1847}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877148+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:05:19.877148+0000","last_peered":"2024-09-10T16:05:19.877148+0000","last_clean":"2024-09-10T16:05:19.877148+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:05:19.877148+0000","last_undegraded":"2024-09-10T16:05:19.877148+0000","last_fullsized":"2024-09-10T16:05:19.877148+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'18644","reported_seq":24196,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:15.829501+0000","last_change":"2024-09-10T16:05:18.271152+0000","last_active":"2024-09-10T16:07:15.829501+0000","last_peered":"2024-09-10T16:07:15.829501+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:07:15.829501+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"43'8600","ondisk_log_start":"43'8600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10346496,"num_objects":2526,"num_object_clones":0,"num_object_copies":5052,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1646,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2526,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16198,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2708,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":880}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'18327","reported_seq":18468,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:15.324212+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:07:15.324212+0000","last_peered":"2024-09-10T16:07:15.324212+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:07:15.324212+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"43'8300","ondisk_log_start":"43'8300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10285056,"num_objects":2511,"num_object_clones":0,"num_object_copies":5022,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2511,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14958,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1793}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877654+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:05:19.877654+0000","last_peered":"2024-09-10T16:05:19.877654+0000","last_clean":"2024-09-10T16:05:19.877654+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:05:19.877654+0000","last_undegraded":"2024-09-10T16:05:19.877654+0000","last_fullsized":"2024-09-10T16:05:19.877654+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'18034","reported_seq":26605,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:07:13.952858+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:07:13.952858+0000","last_peered":"2024-09-10T16:07:13.952858+0000","last_clean":"2024-09-10T16:07:13.952858+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:07:13.952858+0000","last_undegraded":"2024-09-10T16:07:13.952858+0000","last_fullsized":"2024-09-10T16:07:13.952858+0000","mapping_epoch":46,"log_start":"43'8000","ondisk_log_start":"43'8000","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10502144,"num_objects":2564,"num_object_clones":0,"num_object_copies":5128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2564,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14424,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":100,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.876043+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:05:19.876043+0000","last_peered":"2024-09-10T16:05:19.876043+0000","last_clean":"2024-09-10T16:05:19.876043+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:05:19.876043+0000","last_undegraded":"2024-09-10T16:05:19.876043+0000","last_fullsized":"2024-09-10T16:05:19.876043+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'18562","reported_seq":18708,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:15.315002+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:07:15.315002+0000","last_peered":"2024-09-10T16:07:15.315002+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:07:15.315002+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"43'8500","ondisk_log_start":"43'8500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10182656,"num_objects":2486,"num_object_clones":0,"num_object_copies":4972,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2486,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14817,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1885}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877349+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:05:19.877349+0000","last_peered":"2024-09-10T16:05:19.877349+0000","last_clean":"2024-09-10T16:05:19.877349+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:05:19.877349+0000","last_undegraded":"2024-09-10T16:05:19.877349+0000","last_fullsized":"2024-09-10T16:05:19.877349+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877269+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:05:19.877269+0000","last_peered":"2024-09-10T16:05:19.877269+0000","last_clean":"2024-09-10T16:05:19.877269+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:05:19.877269+0000","last_undegraded":"2024-09-10T16:05:19.877269+0000","last_fullsized":"2024-09-10T16:05:19.877269+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'18610","reported_seq":18760,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:15.330095+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:07:15.330095+0000","last_peered":"2024-09-10T16:07:15.330095+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:07:15.330095+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"43'8600","ondisk_log_start":"43'8600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10601414,"num_objects":2590,"num_object_clones":0,"num_object_copies":5180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2590,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14773,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1733},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":165782262,"num_objects":40484,"num_object_clones":0,"num_object_copies":80968,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40203,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40484,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":251622,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11186,"num_bytes_recovered":33095680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":543444992,"data_stored":543303167,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160448,"ondisk_log_size":160448,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986287,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":561212,"kb_used_data":218536,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93810628,"statfs":{"total":96636764160,"available":96062083072,"internally_reserved":0,"allocated":223780864,"data_stored":223478180,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":350875306},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,2,2,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659232,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":484916,"kb_used_data":124188,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93886924,"statfs":{"total":96636764160,"available":96140210176,"internally_reserved":0,"allocated":127168512,"data_stored":126836460,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":369356462},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,0,7,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182515,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":640884,"kb_used_data":189988,"kb_used_omap":4,"kb_used_meta":450875,"kb_avail":93730956,"statfs":{"total":96636764160,"available":95980498944,"internally_reserved":0,"allocated":194547712,"data_stored":194245023,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":461696682},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,7,1,0,2,3,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":193871872,"data_stored":193835742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":126517248,"data_stored":126451755,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":223055872,"data_stored":223015670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:07:17.974 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-10T16:07:17.974 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "47'18217", 'reported_seq': 18298, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:07:15.325114+0000', 'last_change': '2024-09-10T15:59:16.281558+0000', 'last_active': '2024-09-10T16:07:15.325114+0000', 'last_peered': '2024-09-10T16:07:15.325114+0000', 'last_clean': '2024-09-10T15:59:14.132456+0000', 'last_became_active': '2024-09-10T15:59:16.274407+0000', 'last_became_peered': '2024-09-10T15:59:16.274407+0000', 'last_unstale': '2024-09-10T16:07:15.325114+0000', 'last_undegraded': '2024-09-10T15:59:15.851279+0000', 'last_fullsized': '2024-09-10T15:59:15.774356+0000', 'mapping_epoch': 42, 'log_start': "43'8200", 'ondisk_log_start': "43'8200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'objects_scrubbed': 0, 'log_size': 10017, 'log_dups_size': 0, 'ondisk_log_size': 10017, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 10055680, 'num_objects': 2455, 'num_object_clones': 0, 'num_object_copies': 4910, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4259, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2455, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18217, 'num_write_kb': 41344, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1804}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:07:17.974 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-10T16:07:17.974 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "47'18525", 'reported_seq': 18668, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:07:15.323983+0000', 'last_change': '2024-09-10T15:59:16.308264+0000', 'last_active': '2024-09-10T16:07:15.323983+0000', 'last_peered': '2024-09-10T16:07:15.323983+0000', 'last_clean': '2024-09-10T15:59:14.111551+0000', 'last_became_active': '2024-09-10T15:59:16.300040+0000', 'last_became_peered': '2024-09-10T15:59:16.300040+0000', 'last_unstale': '2024-09-10T16:07:15.323983+0000', 'last_undegraded': '2024-09-10T15:59:15.878234+0000', 'last_fullsized': '2024-09-10T15:59:15.769328+0000', 'mapping_epoch': 42, 'log_start': "43'8500", 'ondisk_log_start': "43'8500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3442", 'last_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'objects_scrubbed': 3384, 'log_size': 10025, 'log_dups_size': 0, 'ondisk_log_size': 10025, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1292, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 10387939, 'num_objects': 2537, 'num_object_clones': 0, 'num_object_copies': 5074, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4372, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2537, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 15137, 'num_write_kb': 28580, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1834}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:07:17.975 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-10T16:07:17.975 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "47'18520", 'reported_seq': 18664, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:07:15.344777+0000', 'last_change': '2024-09-10T15:59:16.331899+0000', 'last_active': '2024-09-10T16:07:15.344777+0000', 'last_peered': '2024-09-10T16:07:15.344777+0000', 'last_clean': '2024-09-10T15:59:14.139436+0000', 'last_became_active': '2024-09-10T15:59:16.318715+0000', 'last_became_peered': '2024-09-10T15:59:16.318715+0000', 'last_unstale': '2024-09-10T16:07:15.344777+0000', 'last_undegraded': '2024-09-10T15:59:15.877524+0000', 'last_fullsized': '2024-09-10T15:59:15.769042+0000', 'mapping_epoch': 42, 'log_start': "43'8500", 'ondisk_log_start': "43'8500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3537", 'last_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'objects_scrubbed': 3497, 'log_size': 10020, 'log_dups_size': 0, 'ondisk_log_size': 10020, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1224, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 10494918, 'num_objects': 2564, 'num_object_clones': 0, 'num_object_copies': 5128, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4237, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2564, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 15019, 'num_write_kb': 28172, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1673}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:07:17.975 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-10T16:07:17.975 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "47'17836", 'reported_seq': 17982, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:07:15.748935+0000', 'last_change': '2024-09-10T15:59:16.175680+0000', 'last_active': '2024-09-10T16:07:15.748935+0000', 'last_peered': '2024-09-10T16:07:15.748935+0000', 'last_clean': '2024-09-10T15:59:14.088187+0000', 'last_became_active': '2024-09-10T15:59:16.165810+0000', 'last_became_peered': '2024-09-10T15:59:16.165810+0000', 'last_unstale': '2024-09-10T16:07:15.748935+0000', 'last_undegraded': '2024-09-10T15:59:15.822098+0000', 'last_fullsized': '2024-09-10T15:59:15.769123+0000', 'mapping_epoch': 42, 'log_start': "43'7800", 'ondisk_log_start': "43'7800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2218", 'last_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'objects_scrubbed': 2175, 'log_size': 10036, 'log_dups_size': 0, 'ondisk_log_size': 10036, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 825, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 10736099, 'num_objects': 2622, 'num_object_clones': 0, 'num_object_copies': 5244, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4124, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2622, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 15655, 'num_write_kb': 32204, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': -1501}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:07:17.975 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-10T16:07:17.975 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "47'18360", 'reported_seq': 18494, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:07:15.365225+0000', 'last_change': '2024-09-10T15:59:16.277742+0000', 'last_active': '2024-09-10T16:07:15.365225+0000', 'last_peered': '2024-09-10T16:07:15.365225+0000', 'last_clean': '2024-09-10T15:59:14.129081+0000', 'last_became_active': '2024-09-10T15:59:16.269835+0000', 'last_became_peered': '2024-09-10T15:59:16.269835+0000', 'last_unstale': '2024-09-10T16:07:15.365225+0000', 'last_undegraded': '2024-09-10T15:59:15.860857+0000', 'last_fullsized': '2024-09-10T15:59:15.771410+0000', 'mapping_epoch': 42, 'log_start': "43'8300", 'ondisk_log_start': "43'8300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3557", 'last_scrub_stamp': '2024-09-10T15:59:02.493678+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:02.493678+0000', 'objects_scrubbed': 3488, 'log_size': 10060, 'log_dups_size': 0, 'ondisk_log_size': 10060, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1472, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 10444800, 'num_objects': 2550, 'num_object_clones': 0, 'num_object_copies': 5100, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4257, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2550, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14864, 'num_write_kb': 27852, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1705}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:07:17.975 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-10T16:07:17.975 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "47'18325", 'reported_seq': 18466, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:07:15.748857+0000', 'last_change': '2024-09-10T15:59:16.284311+0000', 'last_active': '2024-09-10T16:07:15.748857+0000', 'last_peered': '2024-09-10T16:07:15.748857+0000', 'last_clean': '2024-09-10T15:59:14.116041+0000', 'last_became_active': '2024-09-10T15:59:16.274543+0000', 'last_became_peered': '2024-09-10T15:59:16.274543+0000', 'last_unstale': '2024-09-10T16:07:15.748857+0000', 'last_undegraded': '2024-09-10T15:59:15.812484+0000', 'last_fullsized': '2024-09-10T15:59:15.770885+0000', 'mapping_epoch': 42, 'log_start': "43'8300", 'ondisk_log_start': "43'8300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2356", 'last_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'objects_scrubbed': 2316, 'log_size': 10025, 'log_dups_size': 0, 'ondisk_log_size': 10025, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 783, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 10060259, 'num_objects': 2457, 'num_object_clones': 0, 'num_object_copies': 4914, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2457, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 16005, 'num_write_kb': 32292, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': -1847}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:07:17.975 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-10T16:07:17.975 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "47'18644", 'reported_seq': 24196, 'reported_epoch': 47, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:07:15.829501+0000', 'last_change': '2024-09-10T16:05:18.271152+0000', 'last_active': '2024-09-10T16:07:15.829501+0000', 'last_peered': '2024-09-10T16:07:15.829501+0000', 'last_clean': '2024-09-10T15:59:14.110654+0000', 'last_became_active': '2024-09-10T15:59:16.164645+0000', 'last_became_peered': '2024-09-10T15:59:16.164645+0000', 'last_unstale': '2024-09-10T16:07:15.829501+0000', 'last_undegraded': '2024-09-10T15:59:15.803593+0000', 'last_fullsized': '2024-09-10T15:59:15.769083+0000', 'mapping_epoch': 42, 'log_start': "43'8600", 'ondisk_log_start': "43'8600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2486", 'last_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'objects_scrubbed': 2440, 'log_size': 10044, 'log_dups_size': 0, 'ondisk_log_size': 10044, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 792, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 10346496, 'num_objects': 2526, 'num_object_clones': 0, 'num_object_copies': 5052, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1646, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2526, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16198, 'num_write_kb': 32568, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2708, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 880}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:07:17.976 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-10T16:07:17.976 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "47'18327", 'reported_seq': 18468, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:07:15.324212+0000', 'last_change': '2024-09-10T15:59:16.315641+0000', 'last_active': '2024-09-10T16:07:15.324212+0000', 'last_peered': '2024-09-10T16:07:15.324212+0000', 'last_clean': '2024-09-10T15:59:14.117001+0000', 'last_became_active': '2024-09-10T15:59:16.302245+0000', 'last_became_peered': '2024-09-10T15:59:16.302245+0000', 'last_unstale': '2024-09-10T16:07:15.324212+0000', 'last_undegraded': '2024-09-10T15:59:15.845012+0000', 'last_fullsized': '2024-09-10T15:59:15.770337+0000', 'mapping_epoch': 42, 'log_start': "43'8300", 'ondisk_log_start': "43'8300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3404", 'last_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'objects_scrubbed': 3367, 'log_size': 10027, 'log_dups_size': 0, 'ondisk_log_size': 10027, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 983, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 10285056, 'num_objects': 2511, 'num_object_clones': 0, 'num_object_copies': 5022, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2511, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14958, 'num_write_kb': 28204, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1793}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:07:17.976 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-10T16:07:17.976 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "47'18562", 'reported_seq': 18708, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:07:15.315002+0000', 'last_change': '2024-09-10T15:59:16.311291+0000', 'last_active': '2024-09-10T16:07:15.315002+0000', 'last_peered': '2024-09-10T16:07:15.315002+0000', 'last_clean': '2024-09-10T15:59:14.116363+0000', 'last_became_active': '2024-09-10T15:59:16.301244+0000', 'last_became_peered': '2024-09-10T15:59:16.301244+0000', 'last_unstale': '2024-09-10T16:07:15.315002+0000', 'last_undegraded': '2024-09-10T15:59:15.883434+0000', 'last_fullsized': '2024-09-10T15:59:15.772756+0000', 'mapping_epoch': 42, 'log_start': "43'8500", 'ondisk_log_start': "43'8500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3789", 'last_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'objects_scrubbed': 3741, 'log_size': 10062, 'log_dups_size': 0, 'ondisk_log_size': 10062, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1440, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 10182656, 'num_objects': 2486, 'num_object_clones': 0, 'num_object_copies': 4972, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4373, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2486, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14817, 'num_write_kb': 27124, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1885}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:07:17.976 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-10T16:07:17.976 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "47'18610", 'reported_seq': 18760, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:07:15.330095+0000', 'last_change': '2024-09-10T15:59:16.313409+0000', 'last_active': '2024-09-10T16:07:15.330095+0000', 'last_peered': '2024-09-10T16:07:15.330095+0000', 'last_clean': '2024-09-10T15:59:14.129780+0000', 'last_became_active': '2024-09-10T15:59:16.300932+0000', 'last_became_peered': '2024-09-10T15:59:16.300932+0000', 'last_unstale': '2024-09-10T16:07:15.330095+0000', 'last_undegraded': '2024-09-10T15:59:15.887823+0000', 'last_fullsized': '2024-09-10T15:59:15.768513+0000', 'mapping_epoch': 42, 'log_start': "43'8600", 'ondisk_log_start': "43'8600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3880", 'last_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'objects_scrubbed': 3827, 'log_size': 10010, 'log_dups_size': 0, 'ondisk_log_size': 10010, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1397, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 10601414, 'num_objects': 2590, 'num_object_clones': 0, 'num_object_copies': 5180, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4325, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2590, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14773, 'num_write_kb': 27072, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1733}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:07:17.976 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T16:07:18.662 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:07:18.662 INFO:teuthology.orchestra.run.smithi115.stdout:{"fsid":"7effee29-df05-4351-9741-ee203564ed81","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 40203/80972 objects degraded (49.650%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":602,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":47,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983954,"num_in_osds":3,"osd_in_since":1725983947,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":40486,"data_bytes":165782281,"bytes_used":1727500288,"bytes_avail":288182792192,"bytes_total":289910292480,"degraded_objects":40203,"degraded_total":80972,"degraded_ratio":0.49650496467914834,"recovering_objects_per_sec":35,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":422,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":438},"fsmap":{"epoch":1,"btime":"2024-09-10T15:57:14:967120+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T16:00:13.560776+0000","services":{}},"progress_events":{}} 2024-09-10T16:07:18.687 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-10T16:07:18.687 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:07:19.272 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:07:19.272 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:07:19.301 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":341,"stamp":"2024-09-10T16:07:17.737545+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":165782281,"num_objects":40486,"num_object_clones":0,"num_object_copies":80972,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40203,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40486,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":251624,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11188,"num_bytes_recovered":33095699,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160451,"ondisk_log_size":160451,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1687012,"kb_used_data":532712,"kb_used_omap":12,"kb_used_meta":1154227,"kb_avail":281428508,"statfs":{"total":289910292480,"available":288182792192,"internally_reserved":0,"allocated":545497088,"data_stored":544559663,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1181928450},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,9,5,2,12,8,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-21585920,"num_objects":-5270,"num_object_clones":0,"num_object_copies":-10540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-422,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5270,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5270,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":422,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.007420"},"pg_stats":[{"pgid":"2.f","version":"47'18217","reported_seq":18298,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:15.325114+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:07:15.325114+0000","last_peered":"2024-09-10T16:07:15.325114+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:07:15.325114+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"43'8200","ondisk_log_start":"43'8200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10055680,"num_objects":2455,"num_object_clones":0,"num_object_copies":4910,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2455,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18217,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1804}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'18318","reported_seq":18436,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:07:15.729716+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:07:15.729716+0000","last_peered":"2024-09-10T16:07:15.729716+0000","last_clean":"2024-09-10T16:07:15.729716+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:07:15.729716+0000","last_undegraded":"2024-09-10T16:07:15.729716+0000","last_fullsized":"2024-09-10T16:07:15.729716+0000","mapping_epoch":35,"log_start":"43'8300","ondisk_log_start":"43'8300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10347462,"num_objects":2528,"num_object_clones":0,"num_object_copies":5056,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2528,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16423,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'18227","reported_seq":26911,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:07:13.966405+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:07:13.966405+0000","last_peered":"2024-09-10T16:07:13.966405+0000","last_clean":"2024-09-10T16:07:13.966405+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:07:13.966405+0000","last_undegraded":"2024-09-10T16:07:13.966405+0000","last_fullsized":"2024-09-10T16:07:13.966405+0000","mapping_epoch":44,"log_start":"43'8200","ondisk_log_start":"43'8200","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10526744,"num_objects":2571,"num_object_clones":0,"num_object_copies":5142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2571,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16231,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'18204","reported_seq":18290,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:07:15.801159+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:07:15.801159+0000","last_peered":"2024-09-10T16:07:15.801159+0000","last_clean":"2024-09-10T16:07:15.801159+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:07:15.801159+0000","last_undegraded":"2024-09-10T16:07:15.801159+0000","last_fullsized":"2024-09-10T16:07:15.801159+0000","mapping_epoch":35,"log_start":"43'8200","ondisk_log_start":"43'8200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10059776,"num_objects":2456,"num_object_clones":0,"num_object_copies":4912,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2456,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18204,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'18408","reported_seq":18544,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:07:15.251518+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:07:15.251518+0000","last_peered":"2024-09-10T16:07:15.251518+0000","last_clean":"2024-09-10T16:07:15.251518+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:07:15.251518+0000","last_undegraded":"2024-09-10T16:07:15.251518+0000","last_fullsized":"2024-09-10T16:07:15.251518+0000","mapping_epoch":34,"log_start":"43'8400","ondisk_log_start":"43'8400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10170851,"num_objects":2484,"num_object_clones":0,"num_object_copies":4968,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2484,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15119,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'18525","reported_seq":18668,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:15.323983+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:07:15.323983+0000","last_peered":"2024-09-10T16:07:15.323983+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:07:15.323983+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"43'8500","ondisk_log_start":"43'8500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10387939,"num_objects":2537,"num_object_clones":0,"num_object_copies":5074,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2537,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15137,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1834},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'18520","reported_seq":18664,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:15.344777+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:07:15.344777+0000","last_peered":"2024-09-10T16:07:15.344777+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:07:15.344777+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"43'8500","ondisk_log_start":"43'8500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10494918,"num_objects":2564,"num_object_clones":0,"num_object_copies":5128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2564,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":15019,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1673},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'18731","reported_seq":18864,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:07:15.295019+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:07:15.295019+0000","last_peered":"2024-09-10T16:07:15.295019+0000","last_clean":"2024-09-10T16:07:15.295019+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:07:15.295019+0000","last_undegraded":"2024-09-10T16:07:15.295019+0000","last_fullsized":"2024-09-10T16:07:15.295019+0000","mapping_epoch":34,"log_start":"43'8700","ondisk_log_start":"43'8700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10579968,"num_objects":2583,"num_object_clones":0,"num_object_copies":5166,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2583,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15578,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.876895+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:05:19.876895+0000","last_peered":"2024-09-10T16:05:19.876895+0000","last_clean":"2024-09-10T16:05:19.876895+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:05:19.876895+0000","last_undegraded":"2024-09-10T16:05:19.876895+0000","last_fullsized":"2024-09-10T16:05:19.876895+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'17836","reported_seq":17982,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:15.748935+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:07:15.748935+0000","last_peered":"2024-09-10T16:07:15.748935+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:07:15.748935+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"43'7800","ondisk_log_start":"43'7800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10736099,"num_objects":2622,"num_object_clones":0,"num_object_copies":5244,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2622,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15655,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-1501}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":100,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.875849+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:05:19.875849+0000","last_peered":"2024-09-10T16:05:19.875849+0000","last_clean":"2024-09-10T16:05:19.875849+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:05:19.875849+0000","last_undegraded":"2024-09-10T16:05:19.875849+0000","last_fullsized":"2024-09-10T16:05:19.875849+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'18360","reported_seq":18494,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:15.365225+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:07:15.365225+0000","last_peered":"2024-09-10T16:07:15.365225+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:07:15.365225+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"43'8300","ondisk_log_start":"43'8300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10444800,"num_objects":2550,"num_object_clones":0,"num_object_copies":5100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2550,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14864,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1705}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":110,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.872023+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:05:19.872023+0000","last_peered":"2024-09-10T16:05:19.872023+0000","last_clean":"2024-09-10T16:05:19.872023+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:05:19.872023+0000","last_undegraded":"2024-09-10T16:05:19.872023+0000","last_fullsized":"2024-09-10T16:05:19.872023+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'18325","reported_seq":18466,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:15.748857+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:07:15.748857+0000","last_peered":"2024-09-10T16:07:15.748857+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:07:15.748857+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"43'8300","ondisk_log_start":"43'8300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10060259,"num_objects":2457,"num_object_clones":0,"num_object_copies":4914,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2457,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16005,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-1847}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877148+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:05:19.877148+0000","last_peered":"2024-09-10T16:05:19.877148+0000","last_clean":"2024-09-10T16:05:19.877148+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:05:19.877148+0000","last_undegraded":"2024-09-10T16:05:19.877148+0000","last_fullsized":"2024-09-10T16:05:19.877148+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'18644","reported_seq":24196,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:15.829501+0000","last_change":"2024-09-10T16:05:18.271152+0000","last_active":"2024-09-10T16:07:15.829501+0000","last_peered":"2024-09-10T16:07:15.829501+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:07:15.829501+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"43'8600","ondisk_log_start":"43'8600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10346496,"num_objects":2526,"num_object_clones":0,"num_object_copies":5052,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1646,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2526,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16198,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2708,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":880}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'18327","reported_seq":18468,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:15.324212+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:07:15.324212+0000","last_peered":"2024-09-10T16:07:15.324212+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:07:15.324212+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"43'8300","ondisk_log_start":"43'8300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10285056,"num_objects":2511,"num_object_clones":0,"num_object_copies":5022,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2511,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14958,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1793}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877654+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:05:19.877654+0000","last_peered":"2024-09-10T16:05:19.877654+0000","last_clean":"2024-09-10T16:05:19.877654+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:05:19.877654+0000","last_undegraded":"2024-09-10T16:05:19.877654+0000","last_fullsized":"2024-09-10T16:05:19.877654+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'18034","reported_seq":26605,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:07:13.952858+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:07:13.952858+0000","last_peered":"2024-09-10T16:07:13.952858+0000","last_clean":"2024-09-10T16:07:13.952858+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:07:13.952858+0000","last_undegraded":"2024-09-10T16:07:13.952858+0000","last_fullsized":"2024-09-10T16:07:13.952858+0000","mapping_epoch":46,"log_start":"43'8000","ondisk_log_start":"43'8000","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10502144,"num_objects":2564,"num_object_clones":0,"num_object_copies":5128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2564,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14424,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":100,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.876043+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:05:19.876043+0000","last_peered":"2024-09-10T16:05:19.876043+0000","last_clean":"2024-09-10T16:05:19.876043+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:05:19.876043+0000","last_undegraded":"2024-09-10T16:05:19.876043+0000","last_fullsized":"2024-09-10T16:05:19.876043+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'18562","reported_seq":18708,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:15.315002+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:07:15.315002+0000","last_peered":"2024-09-10T16:07:15.315002+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:07:15.315002+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"43'8500","ondisk_log_start":"43'8500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10182656,"num_objects":2486,"num_object_clones":0,"num_object_copies":4972,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2486,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14817,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1885}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877349+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:05:19.877349+0000","last_peered":"2024-09-10T16:05:19.877349+0000","last_clean":"2024-09-10T16:05:19.877349+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:05:19.877349+0000","last_undegraded":"2024-09-10T16:05:19.877349+0000","last_fullsized":"2024-09-10T16:05:19.877349+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877269+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:05:19.877269+0000","last_peered":"2024-09-10T16:05:19.877269+0000","last_clean":"2024-09-10T16:05:19.877269+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:05:19.877269+0000","last_undegraded":"2024-09-10T16:05:19.877269+0000","last_fullsized":"2024-09-10T16:05:19.877269+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'18610","reported_seq":18760,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:15.330095+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:07:15.330095+0000","last_peered":"2024-09-10T16:07:15.330095+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:07:15.330095+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"43'8600","ondisk_log_start":"43'8600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10601414,"num_objects":2590,"num_object_clones":0,"num_object_copies":5180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2590,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14773,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1733},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":165782262,"num_objects":40484,"num_object_clones":0,"num_object_copies":80968,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40203,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40484,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":251622,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11186,"num_bytes_recovered":33095680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":543444992,"data_stored":543303167,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160448,"ondisk_log_size":160448,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986287,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":561212,"kb_used_data":218536,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93810628,"statfs":{"total":96636764160,"available":96062083072,"internally_reserved":0,"allocated":223780864,"data_stored":223478180,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":350875306},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,2,2,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659232,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":484916,"kb_used_data":124188,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93886924,"statfs":{"total":96636764160,"available":96140210176,"internally_reserved":0,"allocated":127168512,"data_stored":126836460,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":369356462},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,0,7,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182515,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":640884,"kb_used_data":189988,"kb_used_omap":4,"kb_used_meta":450875,"kb_avail":93730956,"statfs":{"total":96636764160,"available":95980498944,"internally_reserved":0,"allocated":194547712,"data_stored":194245023,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":461696682},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,7,1,0,2,3,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":193871872,"data_stored":193835742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":126517248,"data_stored":126451755,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":223055872,"data_stored":223015670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:07:22.303 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:07:22.881 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:07:22.882 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:07:22.910 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":343,"stamp":"2024-09-10T16:07:21.738441+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":156185353,"num_objects":38143,"num_object_clones":0,"num_object_copies":76286,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40073,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38143,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":253967,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11318,"num_bytes_recovered":33095699,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161094,"ondisk_log_size":161094,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1684996,"kb_used_data":514068,"kb_used_omap":12,"kb_used_meta":1170867,"kb_avail":281430524,"statfs":{"total":289910292480,"available":288184856576,"internally_reserved":0,"allocated":526405632,"data_stored":525468207,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1198967810},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,4,10,7,11],"upper_bound":128},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-27140096,"num_objects":-6626,"num_object_clones":0,"num_object_copies":-13252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-6626,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6626,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":408,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.007031"},"pg_stats":[{"pgid":"2.f","version":"47'18360","reported_seq":18441,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:20.422293+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:07:20.422293+0000","last_peered":"2024-09-10T16:07:20.422293+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:07:20.422293+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"43'8300","ondisk_log_start":"43'8300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9469952,"num_objects":2312,"num_object_clones":0,"num_object_copies":4624,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2312,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18360,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1947}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'18480","reported_seq":18599,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:07:20.826421+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:07:20.826421+0000","last_peered":"2024-09-10T16:07:20.826421+0000","last_clean":"2024-09-10T16:07:20.826421+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:07:20.826421+0000","last_undegraded":"2024-09-10T16:07:20.826421+0000","last_fullsized":"2024-09-10T16:07:20.826421+0000","mapping_epoch":35,"log_start":"43'8400","ondisk_log_start":"43'8400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9683910,"num_objects":2366,"num_object_clones":0,"num_object_copies":4732,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2366,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16585,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'18367","reported_seq":27051,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:07:19.000308+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:07:19.000308+0000","last_peered":"2024-09-10T16:07:19.000308+0000","last_clean":"2024-09-10T16:07:19.000308+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:07:19.000308+0000","last_undegraded":"2024-09-10T16:07:19.000308+0000","last_fullsized":"2024-09-10T16:07:19.000308+0000","mapping_epoch":44,"log_start":"43'8300","ondisk_log_start":"43'8300","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9953304,"num_objects":2431,"num_object_clones":0,"num_object_copies":4862,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2431,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16371,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'18353","reported_seq":18440,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:07:20.743993+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:07:20.743993+0000","last_peered":"2024-09-10T16:07:20.743993+0000","last_clean":"2024-09-10T16:07:20.743993+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:07:20.743993+0000","last_undegraded":"2024-09-10T16:07:20.743993+0000","last_fullsized":"2024-09-10T16:07:20.743993+0000","mapping_epoch":35,"log_start":"43'8300","ondisk_log_start":"43'8300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9449472,"num_objects":2307,"num_object_clones":0,"num_object_copies":4614,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2307,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18353,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'18557","reported_seq":18693,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:07:20.242536+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:07:20.242536+0000","last_peered":"2024-09-10T16:07:20.242536+0000","last_clean":"2024-09-10T16:07:20.242536+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:07:20.242536+0000","last_undegraded":"2024-09-10T16:07:20.242536+0000","last_fullsized":"2024-09-10T16:07:20.242536+0000","mapping_epoch":34,"log_start":"43'8500","ondisk_log_start":"43'8500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9560547,"num_objects":2335,"num_object_clones":0,"num_object_copies":4670,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2335,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15268,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'18669","reported_seq":18813,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:20.348393+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:07:20.348393+0000","last_peered":"2024-09-10T16:07:20.348393+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:07:20.348393+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"43'8600","ondisk_log_start":"43'8600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9798115,"num_objects":2393,"num_object_clones":0,"num_object_copies":4786,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2393,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15281,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1978},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'18660","reported_seq":18804,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:20.395097+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:07:20.395097+0000","last_peered":"2024-09-10T16:07:20.395097+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:07:20.395097+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"43'8600","ondisk_log_start":"43'8600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9921478,"num_objects":2424,"num_object_clones":0,"num_object_copies":4848,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2424,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":15159,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1813},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'18889","reported_seq":19025,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:07:20.410944+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:07:20.410944+0000","last_peered":"2024-09-10T16:07:20.410944+0000","last_clean":"2024-09-10T16:07:20.410944+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:07:20.410944+0000","last_undegraded":"2024-09-10T16:07:20.410944+0000","last_fullsized":"2024-09-10T16:07:20.410944+0000","mapping_epoch":34,"log_start":"43'8800","ondisk_log_start":"43'8800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9932800,"num_objects":2425,"num_object_clones":0,"num_object_copies":4850,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2425,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15736,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.876943+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:05:19.876943+0000","last_peered":"2024-09-10T16:05:19.876943+0000","last_clean":"2024-09-10T16:05:19.876943+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:05:19.876943+0000","last_undegraded":"2024-09-10T16:05:19.876943+0000","last_fullsized":"2024-09-10T16:05:19.876943+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'17979","reported_seq":18125,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:20.781437+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:07:20.781437+0000","last_peered":"2024-09-10T16:07:20.781437+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:07:20.781437+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"43'7900","ondisk_log_start":"43'7900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10150371,"num_objects":2479,"num_object_clones":0,"num_object_copies":4958,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2479,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15798,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-1644}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":100,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.875849+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:05:19.875849+0000","last_peered":"2024-09-10T16:05:19.875849+0000","last_clean":"2024-09-10T16:05:19.875849+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:05:19.875849+0000","last_undegraded":"2024-09-10T16:05:19.875849+0000","last_fullsized":"2024-09-10T16:05:19.875849+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'18525","reported_seq":18658,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:20.348958+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:07:20.348958+0000","last_peered":"2024-09-10T16:07:20.348958+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:07:20.348958+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"43'8500","ondisk_log_start":"43'8500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9768960,"num_objects":2385,"num_object_clones":0,"num_object_copies":4770,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2385,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15029,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1870}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":110,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.872023+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:05:19.872023+0000","last_peered":"2024-09-10T16:05:19.872023+0000","last_clean":"2024-09-10T16:05:19.872023+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:05:19.872023+0000","last_undegraded":"2024-09-10T16:05:19.872023+0000","last_fullsized":"2024-09-10T16:05:19.872023+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'18459","reported_seq":18600,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:20.803567+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:07:20.803567+0000","last_peered":"2024-09-10T16:07:20.803567+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:07:20.803567+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"43'8400","ondisk_log_start":"43'8400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9511395,"num_objects":2323,"num_object_clones":0,"num_object_copies":4646,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2323,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16139,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-1981}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877148+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:05:19.877148+0000","last_peered":"2024-09-10T16:05:19.877148+0000","last_clean":"2024-09-10T16:05:19.877148+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:05:19.877148+0000","last_undegraded":"2024-09-10T16:05:19.877148+0000","last_fullsized":"2024-09-10T16:05:19.877148+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'18782","reported_seq":24594,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:20.839078+0000","last_change":"2024-09-10T16:05:18.271152+0000","last_active":"2024-09-10T16:07:20.839078+0000","last_peered":"2024-09-10T16:07:20.839078+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:07:20.839078+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"43'8700","ondisk_log_start":"43'8700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9781248,"num_objects":2388,"num_object_clones":0,"num_object_copies":4776,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1516,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2388,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16336,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2838,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":872}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'18474","reported_seq":18615,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:20.420044+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:07:20.420044+0000","last_peered":"2024-09-10T16:07:20.420044+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:07:20.420044+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"43'8400","ondisk_log_start":"43'8400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9682944,"num_objects":2364,"num_object_clones":0,"num_object_copies":4728,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2364,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15105,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1940}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877654+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:05:19.877654+0000","last_peered":"2024-09-10T16:05:19.877654+0000","last_clean":"2024-09-10T16:05:19.877654+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:05:19.877654+0000","last_undegraded":"2024-09-10T16:05:19.877654+0000","last_fullsized":"2024-09-10T16:05:19.877654+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'18194","reported_seq":26765,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:07:19.070205+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:07:19.070205+0000","last_peered":"2024-09-10T16:07:19.070205+0000","last_clean":"2024-09-10T16:07:19.070205+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:07:19.070205+0000","last_undegraded":"2024-09-10T16:07:19.070205+0000","last_fullsized":"2024-09-10T16:07:19.070205+0000","mapping_epoch":46,"log_start":"43'8100","ondisk_log_start":"43'8100","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9846784,"num_objects":2404,"num_object_clones":0,"num_object_copies":4808,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2404,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14584,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":100,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.876043+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:05:19.876043+0000","last_peered":"2024-09-10T16:05:19.876043+0000","last_clean":"2024-09-10T16:05:19.876043+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:05:19.876043+0000","last_undegraded":"2024-09-10T16:05:19.876043+0000","last_fullsized":"2024-09-10T16:05:19.876043+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'18685","reported_seq":18833,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:20.386336+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:07:20.386336+0000","last_peered":"2024-09-10T16:07:20.386336+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:07:20.386336+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"43'8600","ondisk_log_start":"43'8600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9678848,"num_objects":2363,"num_object_clones":0,"num_object_copies":4726,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2363,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14940,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2008}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877349+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:05:19.877349+0000","last_peered":"2024-09-10T16:05:19.877349+0000","last_clean":"2024-09-10T16:05:19.877349+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:05:19.877349+0000","last_undegraded":"2024-09-10T16:05:19.877349+0000","last_fullsized":"2024-09-10T16:05:19.877349+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877269+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:05:19.877269+0000","last_peered":"2024-09-10T16:05:19.877269+0000","last_clean":"2024-09-10T16:05:19.877269+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:05:19.877269+0000","last_undegraded":"2024-09-10T16:05:19.877269+0000","last_fullsized":"2024-09-10T16:05:19.877269+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'18758","reported_seq":18907,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:20.375001+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:07:20.375001+0000","last_peered":"2024-09-10T16:07:20.375001+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:07:20.375001+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"43'8700","ondisk_log_start":"43'8700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9995206,"num_objects":2442,"num_object_clones":0,"num_object_copies":4884,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2442,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14921,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1881},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":156185334,"num_objects":38141,"num_object_clones":0,"num_object_copies":76282,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40073,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38141,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":253965,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11316,"num_bytes_recovered":33095680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":524214272,"data_stored":524072447,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161091,"ondisk_log_size":161091,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986288,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":560724,"kb_used_data":213892,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93811116,"statfs":{"total":96636764160,"available":96062582784,"internally_reserved":0,"allocated":219025408,"data_stored":218722724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":355135146},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,0,0,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659233,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":486312,"kb_used_data":117272,"kb_used_omap":4,"kb_used_meta":369019,"kb_avail":93885528,"statfs":{"total":96636764160,"available":96138780672,"internally_reserved":0,"allocated":120086528,"data_stored":119754476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":377876142},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,6,3,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182516,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":637960,"kb_used_data":182904,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93733880,"statfs":{"total":96636764160,"available":95983493120,"internally_reserved":0,"allocated":187293696,"data_stored":186991007,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":465956522},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,4,4,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":186531840,"data_stored":186495710,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":119394304,"data_stored":119328811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":218288128,"data_stored":218247926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:07:22.911 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:07:23.441 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:07:23.443 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:07:23.463 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":343,"stamp":"2024-09-10T16:07:21.738441+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":156185353,"num_objects":38143,"num_object_clones":0,"num_object_copies":76286,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40073,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38143,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":253967,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11318,"num_bytes_recovered":33095699,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161094,"ondisk_log_size":161094,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1684996,"kb_used_data":514068,"kb_used_omap":12,"kb_used_meta":1170867,"kb_avail":281430524,"statfs":{"total":289910292480,"available":288184856576,"internally_reserved":0,"allocated":526405632,"data_stored":525468207,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1198967810},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,4,10,7,11],"upper_bound":128},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-27140096,"num_objects":-6626,"num_object_clones":0,"num_object_copies":-13252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-6626,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6626,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":408,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.007031"},"pg_stats":[{"pgid":"2.f","version":"47'18360","reported_seq":18441,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:20.422293+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:07:20.422293+0000","last_peered":"2024-09-10T16:07:20.422293+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:07:20.422293+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"43'8300","ondisk_log_start":"43'8300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9469952,"num_objects":2312,"num_object_clones":0,"num_object_copies":4624,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2312,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18360,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1947}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'18480","reported_seq":18599,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:07:20.826421+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:07:20.826421+0000","last_peered":"2024-09-10T16:07:20.826421+0000","last_clean":"2024-09-10T16:07:20.826421+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:07:20.826421+0000","last_undegraded":"2024-09-10T16:07:20.826421+0000","last_fullsized":"2024-09-10T16:07:20.826421+0000","mapping_epoch":35,"log_start":"43'8400","ondisk_log_start":"43'8400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9683910,"num_objects":2366,"num_object_clones":0,"num_object_copies":4732,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2366,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16585,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'18367","reported_seq":27051,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:07:19.000308+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:07:19.000308+0000","last_peered":"2024-09-10T16:07:19.000308+0000","last_clean":"2024-09-10T16:07:19.000308+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:07:19.000308+0000","last_undegraded":"2024-09-10T16:07:19.000308+0000","last_fullsized":"2024-09-10T16:07:19.000308+0000","mapping_epoch":44,"log_start":"43'8300","ondisk_log_start":"43'8300","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9953304,"num_objects":2431,"num_object_clones":0,"num_object_copies":4862,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2431,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16371,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'18353","reported_seq":18440,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:07:20.743993+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:07:20.743993+0000","last_peered":"2024-09-10T16:07:20.743993+0000","last_clean":"2024-09-10T16:07:20.743993+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:07:20.743993+0000","last_undegraded":"2024-09-10T16:07:20.743993+0000","last_fullsized":"2024-09-10T16:07:20.743993+0000","mapping_epoch":35,"log_start":"43'8300","ondisk_log_start":"43'8300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9449472,"num_objects":2307,"num_object_clones":0,"num_object_copies":4614,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2307,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18353,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'18557","reported_seq":18693,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:07:20.242536+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:07:20.242536+0000","last_peered":"2024-09-10T16:07:20.242536+0000","last_clean":"2024-09-10T16:07:20.242536+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:07:20.242536+0000","last_undegraded":"2024-09-10T16:07:20.242536+0000","last_fullsized":"2024-09-10T16:07:20.242536+0000","mapping_epoch":34,"log_start":"43'8500","ondisk_log_start":"43'8500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9560547,"num_objects":2335,"num_object_clones":0,"num_object_copies":4670,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2335,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15268,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'18669","reported_seq":18813,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:20.348393+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:07:20.348393+0000","last_peered":"2024-09-10T16:07:20.348393+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:07:20.348393+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"43'8600","ondisk_log_start":"43'8600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9798115,"num_objects":2393,"num_object_clones":0,"num_object_copies":4786,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2393,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15281,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1978},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'18660","reported_seq":18804,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:20.395097+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:07:20.395097+0000","last_peered":"2024-09-10T16:07:20.395097+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:07:20.395097+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"43'8600","ondisk_log_start":"43'8600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9921478,"num_objects":2424,"num_object_clones":0,"num_object_copies":4848,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2424,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":15159,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1813},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'18889","reported_seq":19025,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:07:20.410944+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:07:20.410944+0000","last_peered":"2024-09-10T16:07:20.410944+0000","last_clean":"2024-09-10T16:07:20.410944+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:07:20.410944+0000","last_undegraded":"2024-09-10T16:07:20.410944+0000","last_fullsized":"2024-09-10T16:07:20.410944+0000","mapping_epoch":34,"log_start":"43'8800","ondisk_log_start":"43'8800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9932800,"num_objects":2425,"num_object_clones":0,"num_object_copies":4850,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2425,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15736,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.876943+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:05:19.876943+0000","last_peered":"2024-09-10T16:05:19.876943+0000","last_clean":"2024-09-10T16:05:19.876943+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:05:19.876943+0000","last_undegraded":"2024-09-10T16:05:19.876943+0000","last_fullsized":"2024-09-10T16:05:19.876943+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'17979","reported_seq":18125,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:20.781437+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:07:20.781437+0000","last_peered":"2024-09-10T16:07:20.781437+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:07:20.781437+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"43'7900","ondisk_log_start":"43'7900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10150371,"num_objects":2479,"num_object_clones":0,"num_object_copies":4958,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2479,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15798,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-1644}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":100,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.875849+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:05:19.875849+0000","last_peered":"2024-09-10T16:05:19.875849+0000","last_clean":"2024-09-10T16:05:19.875849+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:05:19.875849+0000","last_undegraded":"2024-09-10T16:05:19.875849+0000","last_fullsized":"2024-09-10T16:05:19.875849+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'18525","reported_seq":18658,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:20.348958+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:07:20.348958+0000","last_peered":"2024-09-10T16:07:20.348958+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:07:20.348958+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"43'8500","ondisk_log_start":"43'8500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9768960,"num_objects":2385,"num_object_clones":0,"num_object_copies":4770,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2385,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15029,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1870}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":110,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.872023+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:05:19.872023+0000","last_peered":"2024-09-10T16:05:19.872023+0000","last_clean":"2024-09-10T16:05:19.872023+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:05:19.872023+0000","last_undegraded":"2024-09-10T16:05:19.872023+0000","last_fullsized":"2024-09-10T16:05:19.872023+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'18459","reported_seq":18600,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:20.803567+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:07:20.803567+0000","last_peered":"2024-09-10T16:07:20.803567+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:07:20.803567+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"43'8400","ondisk_log_start":"43'8400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9511395,"num_objects":2323,"num_object_clones":0,"num_object_copies":4646,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2323,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16139,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-1981}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877148+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:05:19.877148+0000","last_peered":"2024-09-10T16:05:19.877148+0000","last_clean":"2024-09-10T16:05:19.877148+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:05:19.877148+0000","last_undegraded":"2024-09-10T16:05:19.877148+0000","last_fullsized":"2024-09-10T16:05:19.877148+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'18782","reported_seq":24594,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:20.839078+0000","last_change":"2024-09-10T16:05:18.271152+0000","last_active":"2024-09-10T16:07:20.839078+0000","last_peered":"2024-09-10T16:07:20.839078+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:07:20.839078+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"43'8700","ondisk_log_start":"43'8700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9781248,"num_objects":2388,"num_object_clones":0,"num_object_copies":4776,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1516,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2388,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16336,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2838,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":872}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'18474","reported_seq":18615,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:20.420044+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:07:20.420044+0000","last_peered":"2024-09-10T16:07:20.420044+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:07:20.420044+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"43'8400","ondisk_log_start":"43'8400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9682944,"num_objects":2364,"num_object_clones":0,"num_object_copies":4728,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2364,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15105,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1940}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877654+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:05:19.877654+0000","last_peered":"2024-09-10T16:05:19.877654+0000","last_clean":"2024-09-10T16:05:19.877654+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:05:19.877654+0000","last_undegraded":"2024-09-10T16:05:19.877654+0000","last_fullsized":"2024-09-10T16:05:19.877654+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'18194","reported_seq":26765,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:07:19.070205+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:07:19.070205+0000","last_peered":"2024-09-10T16:07:19.070205+0000","last_clean":"2024-09-10T16:07:19.070205+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:07:19.070205+0000","last_undegraded":"2024-09-10T16:07:19.070205+0000","last_fullsized":"2024-09-10T16:07:19.070205+0000","mapping_epoch":46,"log_start":"43'8100","ondisk_log_start":"43'8100","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9846784,"num_objects":2404,"num_object_clones":0,"num_object_copies":4808,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2404,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14584,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":100,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.876043+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:05:19.876043+0000","last_peered":"2024-09-10T16:05:19.876043+0000","last_clean":"2024-09-10T16:05:19.876043+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:05:19.876043+0000","last_undegraded":"2024-09-10T16:05:19.876043+0000","last_fullsized":"2024-09-10T16:05:19.876043+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'18685","reported_seq":18833,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:20.386336+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:07:20.386336+0000","last_peered":"2024-09-10T16:07:20.386336+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:07:20.386336+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"43'8600","ondisk_log_start":"43'8600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9678848,"num_objects":2363,"num_object_clones":0,"num_object_copies":4726,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2363,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14940,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2008}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877349+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:05:19.877349+0000","last_peered":"2024-09-10T16:05:19.877349+0000","last_clean":"2024-09-10T16:05:19.877349+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:05:19.877349+0000","last_undegraded":"2024-09-10T16:05:19.877349+0000","last_fullsized":"2024-09-10T16:05:19.877349+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877269+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:05:19.877269+0000","last_peered":"2024-09-10T16:05:19.877269+0000","last_clean":"2024-09-10T16:05:19.877269+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:05:19.877269+0000","last_undegraded":"2024-09-10T16:05:19.877269+0000","last_fullsized":"2024-09-10T16:05:19.877269+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'18758","reported_seq":18907,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:20.375001+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:07:20.375001+0000","last_peered":"2024-09-10T16:07:20.375001+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:07:20.375001+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"43'8700","ondisk_log_start":"43'8700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9995206,"num_objects":2442,"num_object_clones":0,"num_object_copies":4884,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2442,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14921,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1881},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":156185334,"num_objects":38141,"num_object_clones":0,"num_object_copies":76282,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40073,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38141,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":253965,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11316,"num_bytes_recovered":33095680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":524214272,"data_stored":524072447,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161091,"ondisk_log_size":161091,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986288,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":560724,"kb_used_data":213892,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93811116,"statfs":{"total":96636764160,"available":96062582784,"internally_reserved":0,"allocated":219025408,"data_stored":218722724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":355135146},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,0,0,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659233,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":486312,"kb_used_data":117272,"kb_used_omap":4,"kb_used_meta":369019,"kb_avail":93885528,"statfs":{"total":96636764160,"available":96138780672,"internally_reserved":0,"allocated":120086528,"data_stored":119754476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":377876142},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,6,3,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182516,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":637960,"kb_used_data":182904,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93733880,"statfs":{"total":96636764160,"available":95983493120,"internally_reserved":0,"allocated":187293696,"data_stored":186991007,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":465956522},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,4,4,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":186531840,"data_stored":186495710,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":119394304,"data_stored":119328811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":218288128,"data_stored":218247926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:07:23.464 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-10T16:07:23.464 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "47'18360", 'reported_seq': 18441, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:07:20.422293+0000', 'last_change': '2024-09-10T15:59:16.281558+0000', 'last_active': '2024-09-10T16:07:20.422293+0000', 'last_peered': '2024-09-10T16:07:20.422293+0000', 'last_clean': '2024-09-10T15:59:14.132456+0000', 'last_became_active': '2024-09-10T15:59:16.274407+0000', 'last_became_peered': '2024-09-10T15:59:16.274407+0000', 'last_unstale': '2024-09-10T16:07:20.422293+0000', 'last_undegraded': '2024-09-10T15:59:15.851279+0000', 'last_fullsized': '2024-09-10T15:59:15.774356+0000', 'mapping_epoch': 42, 'log_start': "43'8300", 'ondisk_log_start': "43'8300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'objects_scrubbed': 0, 'log_size': 10060, 'log_dups_size': 0, 'ondisk_log_size': 10060, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 9469952, 'num_objects': 2312, 'num_object_clones': 0, 'num_object_copies': 4624, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4259, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2312, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18360, 'num_write_kb': 41344, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1947}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:07:23.464 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-10T16:07:23.464 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "47'18669", 'reported_seq': 18813, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:07:20.348393+0000', 'last_change': '2024-09-10T15:59:16.308264+0000', 'last_active': '2024-09-10T16:07:20.348393+0000', 'last_peered': '2024-09-10T16:07:20.348393+0000', 'last_clean': '2024-09-10T15:59:14.111551+0000', 'last_became_active': '2024-09-10T15:59:16.300040+0000', 'last_became_peered': '2024-09-10T15:59:16.300040+0000', 'last_unstale': '2024-09-10T16:07:20.348393+0000', 'last_undegraded': '2024-09-10T15:59:15.878234+0000', 'last_fullsized': '2024-09-10T15:59:15.769328+0000', 'mapping_epoch': 42, 'log_start': "43'8600", 'ondisk_log_start': "43'8600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3442", 'last_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'objects_scrubbed': 3384, 'log_size': 10069, 'log_dups_size': 0, 'ondisk_log_size': 10069, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1292, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 9798115, 'num_objects': 2393, 'num_object_clones': 0, 'num_object_copies': 4786, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4372, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2393, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 15281, 'num_write_kb': 28580, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1978}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:07:23.465 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-10T16:07:23.465 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "47'18660", 'reported_seq': 18804, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:07:20.395097+0000', 'last_change': '2024-09-10T15:59:16.331899+0000', 'last_active': '2024-09-10T16:07:20.395097+0000', 'last_peered': '2024-09-10T16:07:20.395097+0000', 'last_clean': '2024-09-10T15:59:14.139436+0000', 'last_became_active': '2024-09-10T15:59:16.318715+0000', 'last_became_peered': '2024-09-10T15:59:16.318715+0000', 'last_unstale': '2024-09-10T16:07:20.395097+0000', 'last_undegraded': '2024-09-10T15:59:15.877524+0000', 'last_fullsized': '2024-09-10T15:59:15.769042+0000', 'mapping_epoch': 42, 'log_start': "43'8600", 'ondisk_log_start': "43'8600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3537", 'last_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'objects_scrubbed': 3497, 'log_size': 10060, 'log_dups_size': 0, 'ondisk_log_size': 10060, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1224, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 9921478, 'num_objects': 2424, 'num_object_clones': 0, 'num_object_copies': 4848, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4237, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2424, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 15159, 'num_write_kb': 28172, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1813}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:07:23.465 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-10T16:07:23.465 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "47'17979", 'reported_seq': 18125, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:07:20.781437+0000', 'last_change': '2024-09-10T15:59:16.175680+0000', 'last_active': '2024-09-10T16:07:20.781437+0000', 'last_peered': '2024-09-10T16:07:20.781437+0000', 'last_clean': '2024-09-10T15:59:14.088187+0000', 'last_became_active': '2024-09-10T15:59:16.165810+0000', 'last_became_peered': '2024-09-10T15:59:16.165810+0000', 'last_unstale': '2024-09-10T16:07:20.781437+0000', 'last_undegraded': '2024-09-10T15:59:15.822098+0000', 'last_fullsized': '2024-09-10T15:59:15.769123+0000', 'mapping_epoch': 42, 'log_start': "43'7900", 'ondisk_log_start': "43'7900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2218", 'last_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'objects_scrubbed': 2175, 'log_size': 10079, 'log_dups_size': 0, 'ondisk_log_size': 10079, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 825, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 10150371, 'num_objects': 2479, 'num_object_clones': 0, 'num_object_copies': 4958, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4124, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2479, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 15798, 'num_write_kb': 32204, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': -1644}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:07:23.465 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-10T16:07:23.465 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "47'18525", 'reported_seq': 18658, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:07:20.348958+0000', 'last_change': '2024-09-10T15:59:16.277742+0000', 'last_active': '2024-09-10T16:07:20.348958+0000', 'last_peered': '2024-09-10T16:07:20.348958+0000', 'last_clean': '2024-09-10T15:59:14.129081+0000', 'last_became_active': '2024-09-10T15:59:16.269835+0000', 'last_became_peered': '2024-09-10T15:59:16.269835+0000', 'last_unstale': '2024-09-10T16:07:20.348958+0000', 'last_undegraded': '2024-09-10T15:59:15.860857+0000', 'last_fullsized': '2024-09-10T15:59:15.771410+0000', 'mapping_epoch': 42, 'log_start': "43'8500", 'ondisk_log_start': "43'8500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3557", 'last_scrub_stamp': '2024-09-10T15:59:02.493678+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:02.493678+0000', 'objects_scrubbed': 3488, 'log_size': 10025, 'log_dups_size': 0, 'ondisk_log_size': 10025, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1472, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 9768960, 'num_objects': 2385, 'num_object_clones': 0, 'num_object_copies': 4770, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4257, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2385, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15029, 'num_write_kb': 27852, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1870}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:07:23.465 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-10T16:07:23.465 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "47'18459", 'reported_seq': 18600, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:07:20.803567+0000', 'last_change': '2024-09-10T15:59:16.284311+0000', 'last_active': '2024-09-10T16:07:20.803567+0000', 'last_peered': '2024-09-10T16:07:20.803567+0000', 'last_clean': '2024-09-10T15:59:14.116041+0000', 'last_became_active': '2024-09-10T15:59:16.274543+0000', 'last_became_peered': '2024-09-10T15:59:16.274543+0000', 'last_unstale': '2024-09-10T16:07:20.803567+0000', 'last_undegraded': '2024-09-10T15:59:15.812484+0000', 'last_fullsized': '2024-09-10T15:59:15.770885+0000', 'mapping_epoch': 42, 'log_start': "43'8400", 'ondisk_log_start': "43'8400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2356", 'last_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'objects_scrubbed': 2316, 'log_size': 10059, 'log_dups_size': 0, 'ondisk_log_size': 10059, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 783, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 9511395, 'num_objects': 2323, 'num_object_clones': 0, 'num_object_copies': 4646, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2323, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 16139, 'num_write_kb': 32292, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': -1981}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:07:23.465 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-10T16:07:23.465 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "47'18782", 'reported_seq': 24594, 'reported_epoch': 47, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:07:20.839078+0000', 'last_change': '2024-09-10T16:05:18.271152+0000', 'last_active': '2024-09-10T16:07:20.839078+0000', 'last_peered': '2024-09-10T16:07:20.839078+0000', 'last_clean': '2024-09-10T15:59:14.110654+0000', 'last_became_active': '2024-09-10T15:59:16.164645+0000', 'last_became_peered': '2024-09-10T15:59:16.164645+0000', 'last_unstale': '2024-09-10T16:07:20.839078+0000', 'last_undegraded': '2024-09-10T15:59:15.803593+0000', 'last_fullsized': '2024-09-10T15:59:15.769083+0000', 'mapping_epoch': 42, 'log_start': "43'8700", 'ondisk_log_start': "43'8700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2486", 'last_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'objects_scrubbed': 2440, 'log_size': 10082, 'log_dups_size': 0, 'ondisk_log_size': 10082, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 792, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 9781248, 'num_objects': 2388, 'num_object_clones': 0, 'num_object_copies': 4776, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1516, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2388, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16336, 'num_write_kb': 32568, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2838, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 872}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:07:23.465 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-10T16:07:23.466 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "47'18474", 'reported_seq': 18615, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:07:20.420044+0000', 'last_change': '2024-09-10T15:59:16.315641+0000', 'last_active': '2024-09-10T16:07:20.420044+0000', 'last_peered': '2024-09-10T16:07:20.420044+0000', 'last_clean': '2024-09-10T15:59:14.117001+0000', 'last_became_active': '2024-09-10T15:59:16.302245+0000', 'last_became_peered': '2024-09-10T15:59:16.302245+0000', 'last_unstale': '2024-09-10T16:07:20.420044+0000', 'last_undegraded': '2024-09-10T15:59:15.845012+0000', 'last_fullsized': '2024-09-10T15:59:15.770337+0000', 'mapping_epoch': 42, 'log_start': "43'8400", 'ondisk_log_start': "43'8400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3404", 'last_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'objects_scrubbed': 3367, 'log_size': 10074, 'log_dups_size': 0, 'ondisk_log_size': 10074, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 983, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 9682944, 'num_objects': 2364, 'num_object_clones': 0, 'num_object_copies': 4728, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2364, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15105, 'num_write_kb': 28204, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1940}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:07:23.466 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-10T16:07:23.466 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "47'18685", 'reported_seq': 18833, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:07:20.386336+0000', 'last_change': '2024-09-10T15:59:16.311291+0000', 'last_active': '2024-09-10T16:07:20.386336+0000', 'last_peered': '2024-09-10T16:07:20.386336+0000', 'last_clean': '2024-09-10T15:59:14.116363+0000', 'last_became_active': '2024-09-10T15:59:16.301244+0000', 'last_became_peered': '2024-09-10T15:59:16.301244+0000', 'last_unstale': '2024-09-10T16:07:20.386336+0000', 'last_undegraded': '2024-09-10T15:59:15.883434+0000', 'last_fullsized': '2024-09-10T15:59:15.772756+0000', 'mapping_epoch': 42, 'log_start': "43'8600", 'ondisk_log_start': "43'8600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3789", 'last_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'objects_scrubbed': 3741, 'log_size': 10085, 'log_dups_size': 0, 'ondisk_log_size': 10085, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1440, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 9678848, 'num_objects': 2363, 'num_object_clones': 0, 'num_object_copies': 4726, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4373, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2363, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14940, 'num_write_kb': 27124, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2008}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:07:23.466 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-10T16:07:23.466 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "47'18758", 'reported_seq': 18907, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:07:20.375001+0000', 'last_change': '2024-09-10T15:59:16.313409+0000', 'last_active': '2024-09-10T16:07:20.375001+0000', 'last_peered': '2024-09-10T16:07:20.375001+0000', 'last_clean': '2024-09-10T15:59:14.129780+0000', 'last_became_active': '2024-09-10T15:59:16.300932+0000', 'last_became_peered': '2024-09-10T15:59:16.300932+0000', 'last_unstale': '2024-09-10T16:07:20.375001+0000', 'last_undegraded': '2024-09-10T15:59:15.887823+0000', 'last_fullsized': '2024-09-10T15:59:15.768513+0000', 'mapping_epoch': 42, 'log_start': "43'8700", 'ondisk_log_start': "43'8700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3880", 'last_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'objects_scrubbed': 3827, 'log_size': 10058, 'log_dups_size': 0, 'ondisk_log_size': 10058, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1397, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 9995206, 'num_objects': 2442, 'num_object_clones': 0, 'num_object_copies': 4884, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4325, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2442, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14921, 'num_write_kb': 27072, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1881}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:07:23.466 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T16:07:24.138 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:07:24.139 INFO:teuthology.orchestra.run.smithi115.stdout:{"fsid":"7effee29-df05-4351-9741-ee203564ed81","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 40073/76286 objects degraded (52.530%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":608,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":47,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983954,"num_in_osds":3,"osd_in_since":1725983947,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":38143,"data_bytes":156185353,"bytes_used":1725435904,"bytes_avail":288184856576,"bytes_total":289910292480,"degraded_objects":40073,"degraded_total":76286,"degraded_ratio":0.5252995307133681,"recovering_objects_per_sec":22,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":269,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":385},"fsmap":{"epoch":1,"btime":"2024-09-10T15:57:14:967120+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T16:00:13.560776+0000","services":{}},"progress_events":{}} 2024-09-10T16:07:24.156 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-10T16:07:24.156 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:07:24.742 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:07:24.744 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:07:24.767 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":344,"stamp":"2024-09-10T16:07:23.738893+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":156185353,"num_objects":38143,"num_object_clones":0,"num_object_copies":76286,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40073,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38143,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":253967,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11318,"num_bytes_recovered":33095699,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161094,"ondisk_log_size":161094,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1684996,"kb_used_data":514068,"kb_used_omap":12,"kb_used_meta":1170867,"kb_avail":281430524,"statfs":{"total":289910292480,"available":288184856576,"internally_reserved":0,"allocated":526405632,"data_stored":525468207,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1198967810},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,4,10,7,11],"upper_bound":128},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18960384,"num_objects":-4629,"num_object_clones":0,"num_object_copies":-9258,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-269,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4629,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4629,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":269,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006919"},"pg_stats":[{"pgid":"2.f","version":"47'18360","reported_seq":18441,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:20.422293+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:07:20.422293+0000","last_peered":"2024-09-10T16:07:20.422293+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:07:20.422293+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"43'8300","ondisk_log_start":"43'8300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9469952,"num_objects":2312,"num_object_clones":0,"num_object_copies":4624,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2312,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18360,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1947}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'18480","reported_seq":18599,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:07:20.826421+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:07:20.826421+0000","last_peered":"2024-09-10T16:07:20.826421+0000","last_clean":"2024-09-10T16:07:20.826421+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:07:20.826421+0000","last_undegraded":"2024-09-10T16:07:20.826421+0000","last_fullsized":"2024-09-10T16:07:20.826421+0000","mapping_epoch":35,"log_start":"43'8400","ondisk_log_start":"43'8400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9683910,"num_objects":2366,"num_object_clones":0,"num_object_copies":4732,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2366,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16585,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'18367","reported_seq":27051,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:07:19.000308+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:07:19.000308+0000","last_peered":"2024-09-10T16:07:19.000308+0000","last_clean":"2024-09-10T16:07:19.000308+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:07:19.000308+0000","last_undegraded":"2024-09-10T16:07:19.000308+0000","last_fullsized":"2024-09-10T16:07:19.000308+0000","mapping_epoch":44,"log_start":"43'8300","ondisk_log_start":"43'8300","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9953304,"num_objects":2431,"num_object_clones":0,"num_object_copies":4862,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2431,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16371,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'18353","reported_seq":18440,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:07:20.743993+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:07:20.743993+0000","last_peered":"2024-09-10T16:07:20.743993+0000","last_clean":"2024-09-10T16:07:20.743993+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:07:20.743993+0000","last_undegraded":"2024-09-10T16:07:20.743993+0000","last_fullsized":"2024-09-10T16:07:20.743993+0000","mapping_epoch":35,"log_start":"43'8300","ondisk_log_start":"43'8300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9449472,"num_objects":2307,"num_object_clones":0,"num_object_copies":4614,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2307,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18353,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'18557","reported_seq":18693,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:07:20.242536+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:07:20.242536+0000","last_peered":"2024-09-10T16:07:20.242536+0000","last_clean":"2024-09-10T16:07:20.242536+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:07:20.242536+0000","last_undegraded":"2024-09-10T16:07:20.242536+0000","last_fullsized":"2024-09-10T16:07:20.242536+0000","mapping_epoch":34,"log_start":"43'8500","ondisk_log_start":"43'8500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9560547,"num_objects":2335,"num_object_clones":0,"num_object_copies":4670,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2335,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15268,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'18669","reported_seq":18813,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:20.348393+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:07:20.348393+0000","last_peered":"2024-09-10T16:07:20.348393+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:07:20.348393+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"43'8600","ondisk_log_start":"43'8600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9798115,"num_objects":2393,"num_object_clones":0,"num_object_copies":4786,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2393,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15281,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1978},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'18660","reported_seq":18804,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:20.395097+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:07:20.395097+0000","last_peered":"2024-09-10T16:07:20.395097+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:07:20.395097+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"43'8600","ondisk_log_start":"43'8600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9921478,"num_objects":2424,"num_object_clones":0,"num_object_copies":4848,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2424,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":15159,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1813},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'18889","reported_seq":19025,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:07:20.410944+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:07:20.410944+0000","last_peered":"2024-09-10T16:07:20.410944+0000","last_clean":"2024-09-10T16:07:20.410944+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:07:20.410944+0000","last_undegraded":"2024-09-10T16:07:20.410944+0000","last_fullsized":"2024-09-10T16:07:20.410944+0000","mapping_epoch":34,"log_start":"43'8800","ondisk_log_start":"43'8800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9932800,"num_objects":2425,"num_object_clones":0,"num_object_copies":4850,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2425,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15736,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.876943+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:05:19.876943+0000","last_peered":"2024-09-10T16:05:19.876943+0000","last_clean":"2024-09-10T16:05:19.876943+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:05:19.876943+0000","last_undegraded":"2024-09-10T16:05:19.876943+0000","last_fullsized":"2024-09-10T16:05:19.876943+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'17979","reported_seq":18125,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:20.781437+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:07:20.781437+0000","last_peered":"2024-09-10T16:07:20.781437+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:07:20.781437+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"43'7900","ondisk_log_start":"43'7900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10150371,"num_objects":2479,"num_object_clones":0,"num_object_copies":4958,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2479,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15798,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-1644}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":100,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.875849+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:05:19.875849+0000","last_peered":"2024-09-10T16:05:19.875849+0000","last_clean":"2024-09-10T16:05:19.875849+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:05:19.875849+0000","last_undegraded":"2024-09-10T16:05:19.875849+0000","last_fullsized":"2024-09-10T16:05:19.875849+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'18525","reported_seq":18658,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:20.348958+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:07:20.348958+0000","last_peered":"2024-09-10T16:07:20.348958+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:07:20.348958+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"43'8500","ondisk_log_start":"43'8500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9768960,"num_objects":2385,"num_object_clones":0,"num_object_copies":4770,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2385,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15029,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1870}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":110,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.872023+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:05:19.872023+0000","last_peered":"2024-09-10T16:05:19.872023+0000","last_clean":"2024-09-10T16:05:19.872023+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:05:19.872023+0000","last_undegraded":"2024-09-10T16:05:19.872023+0000","last_fullsized":"2024-09-10T16:05:19.872023+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'18459","reported_seq":18600,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:20.803567+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:07:20.803567+0000","last_peered":"2024-09-10T16:07:20.803567+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:07:20.803567+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"43'8400","ondisk_log_start":"43'8400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9511395,"num_objects":2323,"num_object_clones":0,"num_object_copies":4646,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2323,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16139,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-1981}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877148+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:05:19.877148+0000","last_peered":"2024-09-10T16:05:19.877148+0000","last_clean":"2024-09-10T16:05:19.877148+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:05:19.877148+0000","last_undegraded":"2024-09-10T16:05:19.877148+0000","last_fullsized":"2024-09-10T16:05:19.877148+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'18782","reported_seq":24594,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:20.839078+0000","last_change":"2024-09-10T16:05:18.271152+0000","last_active":"2024-09-10T16:07:20.839078+0000","last_peered":"2024-09-10T16:07:20.839078+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:07:20.839078+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"43'8700","ondisk_log_start":"43'8700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9781248,"num_objects":2388,"num_object_clones":0,"num_object_copies":4776,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1516,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2388,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16336,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2838,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":872}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'18474","reported_seq":18615,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:20.420044+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:07:20.420044+0000","last_peered":"2024-09-10T16:07:20.420044+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:07:20.420044+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"43'8400","ondisk_log_start":"43'8400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9682944,"num_objects":2364,"num_object_clones":0,"num_object_copies":4728,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2364,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15105,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1940}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877654+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:05:19.877654+0000","last_peered":"2024-09-10T16:05:19.877654+0000","last_clean":"2024-09-10T16:05:19.877654+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:05:19.877654+0000","last_undegraded":"2024-09-10T16:05:19.877654+0000","last_fullsized":"2024-09-10T16:05:19.877654+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'18194","reported_seq":26765,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:07:19.070205+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:07:19.070205+0000","last_peered":"2024-09-10T16:07:19.070205+0000","last_clean":"2024-09-10T16:07:19.070205+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:07:19.070205+0000","last_undegraded":"2024-09-10T16:07:19.070205+0000","last_fullsized":"2024-09-10T16:07:19.070205+0000","mapping_epoch":46,"log_start":"43'8100","ondisk_log_start":"43'8100","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9846784,"num_objects":2404,"num_object_clones":0,"num_object_copies":4808,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2404,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14584,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":100,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.876043+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:05:19.876043+0000","last_peered":"2024-09-10T16:05:19.876043+0000","last_clean":"2024-09-10T16:05:19.876043+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:05:19.876043+0000","last_undegraded":"2024-09-10T16:05:19.876043+0000","last_fullsized":"2024-09-10T16:05:19.876043+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'18685","reported_seq":18833,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:20.386336+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:07:20.386336+0000","last_peered":"2024-09-10T16:07:20.386336+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:07:20.386336+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"43'8600","ondisk_log_start":"43'8600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9678848,"num_objects":2363,"num_object_clones":0,"num_object_copies":4726,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2363,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14940,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2008}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877349+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:05:19.877349+0000","last_peered":"2024-09-10T16:05:19.877349+0000","last_clean":"2024-09-10T16:05:19.877349+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:05:19.877349+0000","last_undegraded":"2024-09-10T16:05:19.877349+0000","last_fullsized":"2024-09-10T16:05:19.877349+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877269+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:05:19.877269+0000","last_peered":"2024-09-10T16:05:19.877269+0000","last_clean":"2024-09-10T16:05:19.877269+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:05:19.877269+0000","last_undegraded":"2024-09-10T16:05:19.877269+0000","last_fullsized":"2024-09-10T16:05:19.877269+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'18758","reported_seq":18907,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:20.375001+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:07:20.375001+0000","last_peered":"2024-09-10T16:07:20.375001+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:07:20.375001+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"43'8700","ondisk_log_start":"43'8700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9995206,"num_objects":2442,"num_object_clones":0,"num_object_copies":4884,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2442,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14921,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1881},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":156185334,"num_objects":38141,"num_object_clones":0,"num_object_copies":76282,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40073,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38141,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":253965,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11316,"num_bytes_recovered":33095680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":524214272,"data_stored":524072447,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161091,"ondisk_log_size":161091,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986288,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":560724,"kb_used_data":213892,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93811116,"statfs":{"total":96636764160,"available":96062582784,"internally_reserved":0,"allocated":219025408,"data_stored":218722724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":355135146},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,0,0,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659233,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":486312,"kb_used_data":117272,"kb_used_omap":4,"kb_used_meta":369019,"kb_avail":93885528,"statfs":{"total":96636764160,"available":96138780672,"internally_reserved":0,"allocated":120086528,"data_stored":119754476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":377876142},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,6,3,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182516,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":637960,"kb_used_data":182904,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93733880,"statfs":{"total":96636764160,"available":95983493120,"internally_reserved":0,"allocated":187293696,"data_stored":186991007,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":465956522},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,4,4,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":186531840,"data_stored":186495710,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":119394304,"data_stored":119328811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":218288128,"data_stored":218247926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:07:27.769 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:07:28.337 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:07:28.340 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:07:28.363 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":346,"stamp":"2024-09-10T16:07:27.739799+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":146658057,"num_objects":35817,"num_object_clones":0,"num_object_copies":71634,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39932,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35817,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":256293,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11459,"num_bytes_recovered":33095699,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160620,"ondisk_log_size":160620,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1645704,"kb_used_data":495556,"kb_used_omap":12,"kb_used_meta":1150067,"kb_avail":281469816,"statfs":{"total":289910292480,"available":288225091584,"internally_reserved":0,"allocated":507449344,"data_stored":506511919,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1177668610},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,0,1,9,8,13,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-22114304,"num_objects":-5399,"num_object_clones":0,"num_object_copies":-10798,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-410,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5399,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5399,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":410,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004522"},"pg_stats":[{"pgid":"2.f","version":"47'18487","reported_seq":18568,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:25.371415+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:07:25.371415+0000","last_peered":"2024-09-10T16:07:25.371415+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:07:25.371415+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"43'8400","ondisk_log_start":"43'8400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8949760,"num_objects":2185,"num_object_clones":0,"num_object_copies":4370,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2185,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18487,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2074}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'18615","reported_seq":18734,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:07:25.681168+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:07:25.681168+0000","last_peered":"2024-09-10T16:07:25.681168+0000","last_clean":"2024-09-10T16:07:25.681168+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:07:25.681168+0000","last_undegraded":"2024-09-10T16:07:25.681168+0000","last_fullsized":"2024-09-10T16:07:25.681168+0000","mapping_epoch":35,"log_start":"43'8600","ondisk_log_start":"43'8600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9130950,"num_objects":2231,"num_object_clones":0,"num_object_copies":4462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2231,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16720,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'18518","reported_seq":27201,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:07:24.078148+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:07:24.078148+0000","last_peered":"2024-09-10T16:07:24.078148+0000","last_clean":"2024-09-10T16:07:24.078148+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:07:24.078148+0000","last_undegraded":"2024-09-10T16:07:24.078148+0000","last_fullsized":"2024-09-10T16:07:24.078148+0000","mapping_epoch":44,"log_start":"43'8500","ondisk_log_start":"43'8500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9334808,"num_objects":2280,"num_object_clones":0,"num_object_copies":4560,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2280,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16522,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'18498","reported_seq":18585,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:07:25.785793+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:07:25.785793+0000","last_peered":"2024-09-10T16:07:25.785793+0000","last_clean":"2024-09-10T16:07:25.785793+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:07:25.785793+0000","last_undegraded":"2024-09-10T16:07:25.785793+0000","last_fullsized":"2024-09-10T16:07:25.785793+0000","mapping_epoch":35,"log_start":"43'8400","ondisk_log_start":"43'8400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8855552,"num_objects":2162,"num_object_clones":0,"num_object_copies":4324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2162,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18498,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'18704","reported_seq":18840,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:07:25.458955+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:07:25.458955+0000","last_peered":"2024-09-10T16:07:25.458955+0000","last_clean":"2024-09-10T16:07:25.458955+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:07:25.458955+0000","last_undegraded":"2024-09-10T16:07:25.458955+0000","last_fullsized":"2024-09-10T16:07:25.458955+0000","mapping_epoch":34,"log_start":"43'8700","ondisk_log_start":"43'8700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8958435,"num_objects":2188,"num_object_clones":0,"num_object_copies":4376,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2188,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15415,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'18821","reported_seq":18965,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:25.436278+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:07:25.436278+0000","last_peered":"2024-09-10T16:07:25.436278+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:07:25.436278+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"43'8800","ondisk_log_start":"43'8800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9175523,"num_objects":2241,"num_object_clones":0,"num_object_copies":4482,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2241,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15433,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2130},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'18814","reported_seq":18958,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:25.345976+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:07:25.345976+0000","last_peered":"2024-09-10T16:07:25.345976+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:07:25.345976+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"43'8800","ondisk_log_start":"43'8800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9290694,"num_objects":2270,"num_object_clones":0,"num_object_copies":4540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2270,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":15313,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1967},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'19038","reported_seq":19174,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:07:25.454108+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:07:25.454108+0000","last_peered":"2024-09-10T16:07:25.454108+0000","last_clean":"2024-09-10T16:07:25.454108+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:07:25.454108+0000","last_undegraded":"2024-09-10T16:07:25.454108+0000","last_fullsized":"2024-09-10T16:07:25.454108+0000","mapping_epoch":34,"log_start":"43'9000","ondisk_log_start":"43'9000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9322496,"num_objects":2276,"num_object_clones":0,"num_object_copies":4552,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2276,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15885,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.876943+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:05:19.876943+0000","last_peered":"2024-09-10T16:05:19.876943+0000","last_clean":"2024-09-10T16:05:19.876943+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:05:19.876943+0000","last_undegraded":"2024-09-10T16:05:19.876943+0000","last_fullsized":"2024-09-10T16:05:19.876943+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'18133","reported_seq":18279,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:25.782849+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:07:25.782849+0000","last_peered":"2024-09-10T16:07:25.782849+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:07:25.782849+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"43'8100","ondisk_log_start":"43'8100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9519587,"num_objects":2325,"num_object_clones":0,"num_object_copies":4650,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2325,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15952,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-1798}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":100,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.875849+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:05:19.875849+0000","last_peered":"2024-09-10T16:05:19.875849+0000","last_clean":"2024-09-10T16:05:19.875849+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:05:19.875849+0000","last_undegraded":"2024-09-10T16:05:19.875849+0000","last_fullsized":"2024-09-10T16:05:19.875849+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'18652","reported_seq":18787,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:25.364573+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:07:25.364573+0000","last_peered":"2024-09-10T16:07:25.364573+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:07:25.364573+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"43'8600","ondisk_log_start":"43'8600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9248768,"num_objects":2258,"num_object_clones":0,"num_object_copies":4516,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2258,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15156,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1997}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":110,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.872023+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:05:19.872023+0000","last_peered":"2024-09-10T16:05:19.872023+0000","last_clean":"2024-09-10T16:05:19.872023+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:05:19.872023+0000","last_undegraded":"2024-09-10T16:05:19.872023+0000","last_fullsized":"2024-09-10T16:05:19.872023+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'18592","reported_seq":18733,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:25.833244+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:07:25.833244+0000","last_peered":"2024-09-10T16:07:25.833244+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:07:25.833244+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"43'8500","ondisk_log_start":"43'8500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8966627,"num_objects":2190,"num_object_clones":0,"num_object_copies":4380,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2190,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16272,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-2114}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877148+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:05:19.877148+0000","last_peered":"2024-09-10T16:05:19.877148+0000","last_clean":"2024-09-10T16:05:19.877148+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:05:19.877148+0000","last_undegraded":"2024-09-10T16:05:19.877148+0000","last_fullsized":"2024-09-10T16:05:19.877148+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'18917","reported_seq":25011,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:25.838174+0000","last_change":"2024-09-10T16:05:18.271152+0000","last_active":"2024-09-10T16:07:25.838174+0000","last_peered":"2024-09-10T16:07:25.838174+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:07:25.838174+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"43'8900","ondisk_log_start":"43'8900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9228288,"num_objects":2253,"num_object_clones":0,"num_object_copies":4506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1375,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2253,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16471,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2979,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":878}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'18622","reported_seq":18763,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:25.346363+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:07:25.346363+0000","last_peered":"2024-09-10T16:07:25.346363+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:07:25.346363+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"43'8600","ondisk_log_start":"43'8600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9076736,"num_objects":2216,"num_object_clones":0,"num_object_copies":4432,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2216,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15253,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2088}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877654+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:05:19.877654+0000","last_peered":"2024-09-10T16:05:19.877654+0000","last_clean":"2024-09-10T16:05:19.877654+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:05:19.877654+0000","last_undegraded":"2024-09-10T16:05:19.877654+0000","last_fullsized":"2024-09-10T16:05:19.877654+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'18350","reported_seq":26920,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:07:24.180022+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:07:24.180022+0000","last_peered":"2024-09-10T16:07:24.180022+0000","last_clean":"2024-09-10T16:07:24.180022+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:07:24.180022+0000","last_undegraded":"2024-09-10T16:07:24.180022+0000","last_fullsized":"2024-09-10T16:07:24.180022+0000","mapping_epoch":46,"log_start":"43'8300","ondisk_log_start":"43'8300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9207808,"num_objects":2248,"num_object_clones":0,"num_object_copies":4496,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2248,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14740,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":100,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.876043+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:05:19.876043+0000","last_peered":"2024-09-10T16:05:19.876043+0000","last_clean":"2024-09-10T16:05:19.876043+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:05:19.876043+0000","last_undegraded":"2024-09-10T16:05:19.876043+0000","last_fullsized":"2024-09-10T16:05:19.876043+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'18841","reported_seq":18989,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:25.408318+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:07:25.408318+0000","last_peered":"2024-09-10T16:07:25.408318+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:07:25.408318+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"43'8800","ondisk_log_start":"43'8800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9039872,"num_objects":2207,"num_object_clones":0,"num_object_copies":4414,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2207,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15096,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2164}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877349+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:05:19.877349+0000","last_peered":"2024-09-10T16:05:19.877349+0000","last_clean":"2024-09-10T16:05:19.877349+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:05:19.877349+0000","last_undegraded":"2024-09-10T16:05:19.877349+0000","last_fullsized":"2024-09-10T16:05:19.877349+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877269+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:05:19.877269+0000","last_peered":"2024-09-10T16:05:19.877269+0000","last_clean":"2024-09-10T16:05:19.877269+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:05:19.877269+0000","last_undegraded":"2024-09-10T16:05:19.877269+0000","last_fullsized":"2024-09-10T16:05:19.877269+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'18915","reported_seq":19065,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:25.440029+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:07:25.440029+0000","last_peered":"2024-09-10T16:07:25.440029+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:07:25.440029+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"43'8900","ondisk_log_start":"43'8900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9352134,"num_objects":2285,"num_object_clones":0,"num_object_copies":4570,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2285,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15078,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2038},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":146658038,"num_objects":35815,"num_object_clones":0,"num_object_copies":71630,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39932,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35815,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":256291,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11457,"num_bytes_recovered":33095680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":505049088,"data_stored":504907263,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160617,"ondisk_log_size":160617,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986289,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":560320,"kb_used_data":209324,"kb_used_omap":4,"kb_used_meta":350971,"kb_avail":93811520,"statfs":{"total":96636764160,"available":96062996480,"internally_reserved":0,"allocated":214347776,"data_stored":214045092,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":359394986},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,0,2,3,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659234,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":466968,"kb_used_data":110408,"kb_used_omap":4,"kb_used_meta":356539,"kb_avail":93904872,"statfs":{"total":96636764160,"available":96158588928,"internally_reserved":0,"allocated":113057792,"data_stored":112725740,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":365096622},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,0,0,2,4,7],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182517,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":618416,"kb_used_data":175824,"kb_used_omap":4,"kb_used_meta":442555,"kb_avail":93753424,"statfs":{"total":96636764160,"available":96003506176,"internally_reserved":0,"allocated":180043776,"data_stored":179741087,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":453177002},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,7,2,3,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":179335168,"data_stored":179299038,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":112103424,"data_stored":112037931,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":213610496,"data_stored":213570294,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:07:28.365 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:07:28.938 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:07:28.941 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:07:28.960 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":346,"stamp":"2024-09-10T16:07:27.739799+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":146658057,"num_objects":35817,"num_object_clones":0,"num_object_copies":71634,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39932,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35817,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":256293,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11459,"num_bytes_recovered":33095699,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160620,"ondisk_log_size":160620,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1645704,"kb_used_data":495556,"kb_used_omap":12,"kb_used_meta":1150067,"kb_avail":281469816,"statfs":{"total":289910292480,"available":288225091584,"internally_reserved":0,"allocated":507449344,"data_stored":506511919,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1177668610},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,0,1,9,8,13,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-22114304,"num_objects":-5399,"num_object_clones":0,"num_object_copies":-10798,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-410,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5399,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5399,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":410,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004522"},"pg_stats":[{"pgid":"2.f","version":"47'18487","reported_seq":18568,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:25.371415+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:07:25.371415+0000","last_peered":"2024-09-10T16:07:25.371415+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:07:25.371415+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"43'8400","ondisk_log_start":"43'8400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8949760,"num_objects":2185,"num_object_clones":0,"num_object_copies":4370,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2185,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18487,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2074}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'18615","reported_seq":18734,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:07:25.681168+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:07:25.681168+0000","last_peered":"2024-09-10T16:07:25.681168+0000","last_clean":"2024-09-10T16:07:25.681168+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:07:25.681168+0000","last_undegraded":"2024-09-10T16:07:25.681168+0000","last_fullsized":"2024-09-10T16:07:25.681168+0000","mapping_epoch":35,"log_start":"43'8600","ondisk_log_start":"43'8600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9130950,"num_objects":2231,"num_object_clones":0,"num_object_copies":4462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2231,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16720,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'18518","reported_seq":27201,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:07:24.078148+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:07:24.078148+0000","last_peered":"2024-09-10T16:07:24.078148+0000","last_clean":"2024-09-10T16:07:24.078148+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:07:24.078148+0000","last_undegraded":"2024-09-10T16:07:24.078148+0000","last_fullsized":"2024-09-10T16:07:24.078148+0000","mapping_epoch":44,"log_start":"43'8500","ondisk_log_start":"43'8500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9334808,"num_objects":2280,"num_object_clones":0,"num_object_copies":4560,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2280,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16522,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'18498","reported_seq":18585,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:07:25.785793+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:07:25.785793+0000","last_peered":"2024-09-10T16:07:25.785793+0000","last_clean":"2024-09-10T16:07:25.785793+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:07:25.785793+0000","last_undegraded":"2024-09-10T16:07:25.785793+0000","last_fullsized":"2024-09-10T16:07:25.785793+0000","mapping_epoch":35,"log_start":"43'8400","ondisk_log_start":"43'8400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8855552,"num_objects":2162,"num_object_clones":0,"num_object_copies":4324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2162,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18498,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'18704","reported_seq":18840,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:07:25.458955+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:07:25.458955+0000","last_peered":"2024-09-10T16:07:25.458955+0000","last_clean":"2024-09-10T16:07:25.458955+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:07:25.458955+0000","last_undegraded":"2024-09-10T16:07:25.458955+0000","last_fullsized":"2024-09-10T16:07:25.458955+0000","mapping_epoch":34,"log_start":"43'8700","ondisk_log_start":"43'8700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8958435,"num_objects":2188,"num_object_clones":0,"num_object_copies":4376,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2188,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15415,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'18821","reported_seq":18965,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:25.436278+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:07:25.436278+0000","last_peered":"2024-09-10T16:07:25.436278+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:07:25.436278+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"43'8800","ondisk_log_start":"43'8800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9175523,"num_objects":2241,"num_object_clones":0,"num_object_copies":4482,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2241,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15433,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2130},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'18814","reported_seq":18958,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:25.345976+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:07:25.345976+0000","last_peered":"2024-09-10T16:07:25.345976+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:07:25.345976+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"43'8800","ondisk_log_start":"43'8800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9290694,"num_objects":2270,"num_object_clones":0,"num_object_copies":4540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2270,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":15313,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1967},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'19038","reported_seq":19174,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:07:25.454108+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:07:25.454108+0000","last_peered":"2024-09-10T16:07:25.454108+0000","last_clean":"2024-09-10T16:07:25.454108+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:07:25.454108+0000","last_undegraded":"2024-09-10T16:07:25.454108+0000","last_fullsized":"2024-09-10T16:07:25.454108+0000","mapping_epoch":34,"log_start":"43'9000","ondisk_log_start":"43'9000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9322496,"num_objects":2276,"num_object_clones":0,"num_object_copies":4552,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2276,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15885,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.876943+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:05:19.876943+0000","last_peered":"2024-09-10T16:05:19.876943+0000","last_clean":"2024-09-10T16:05:19.876943+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:05:19.876943+0000","last_undegraded":"2024-09-10T16:05:19.876943+0000","last_fullsized":"2024-09-10T16:05:19.876943+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'18133","reported_seq":18279,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:25.782849+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:07:25.782849+0000","last_peered":"2024-09-10T16:07:25.782849+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:07:25.782849+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"43'8100","ondisk_log_start":"43'8100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9519587,"num_objects":2325,"num_object_clones":0,"num_object_copies":4650,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2325,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15952,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-1798}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":100,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.875849+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:05:19.875849+0000","last_peered":"2024-09-10T16:05:19.875849+0000","last_clean":"2024-09-10T16:05:19.875849+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:05:19.875849+0000","last_undegraded":"2024-09-10T16:05:19.875849+0000","last_fullsized":"2024-09-10T16:05:19.875849+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'18652","reported_seq":18787,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:25.364573+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:07:25.364573+0000","last_peered":"2024-09-10T16:07:25.364573+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:07:25.364573+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"43'8600","ondisk_log_start":"43'8600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9248768,"num_objects":2258,"num_object_clones":0,"num_object_copies":4516,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2258,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15156,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1997}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":110,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.872023+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:05:19.872023+0000","last_peered":"2024-09-10T16:05:19.872023+0000","last_clean":"2024-09-10T16:05:19.872023+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:05:19.872023+0000","last_undegraded":"2024-09-10T16:05:19.872023+0000","last_fullsized":"2024-09-10T16:05:19.872023+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'18592","reported_seq":18733,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:25.833244+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:07:25.833244+0000","last_peered":"2024-09-10T16:07:25.833244+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:07:25.833244+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"43'8500","ondisk_log_start":"43'8500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8966627,"num_objects":2190,"num_object_clones":0,"num_object_copies":4380,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2190,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16272,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-2114}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877148+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:05:19.877148+0000","last_peered":"2024-09-10T16:05:19.877148+0000","last_clean":"2024-09-10T16:05:19.877148+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:05:19.877148+0000","last_undegraded":"2024-09-10T16:05:19.877148+0000","last_fullsized":"2024-09-10T16:05:19.877148+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'18917","reported_seq":25011,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:25.838174+0000","last_change":"2024-09-10T16:05:18.271152+0000","last_active":"2024-09-10T16:07:25.838174+0000","last_peered":"2024-09-10T16:07:25.838174+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:07:25.838174+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"43'8900","ondisk_log_start":"43'8900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9228288,"num_objects":2253,"num_object_clones":0,"num_object_copies":4506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1375,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2253,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16471,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2979,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":878}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'18622","reported_seq":18763,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:25.346363+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:07:25.346363+0000","last_peered":"2024-09-10T16:07:25.346363+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:07:25.346363+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"43'8600","ondisk_log_start":"43'8600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9076736,"num_objects":2216,"num_object_clones":0,"num_object_copies":4432,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2216,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15253,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2088}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877654+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:05:19.877654+0000","last_peered":"2024-09-10T16:05:19.877654+0000","last_clean":"2024-09-10T16:05:19.877654+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:05:19.877654+0000","last_undegraded":"2024-09-10T16:05:19.877654+0000","last_fullsized":"2024-09-10T16:05:19.877654+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'18350","reported_seq":26920,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:07:24.180022+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:07:24.180022+0000","last_peered":"2024-09-10T16:07:24.180022+0000","last_clean":"2024-09-10T16:07:24.180022+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:07:24.180022+0000","last_undegraded":"2024-09-10T16:07:24.180022+0000","last_fullsized":"2024-09-10T16:07:24.180022+0000","mapping_epoch":46,"log_start":"43'8300","ondisk_log_start":"43'8300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9207808,"num_objects":2248,"num_object_clones":0,"num_object_copies":4496,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2248,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14740,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":100,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.876043+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:05:19.876043+0000","last_peered":"2024-09-10T16:05:19.876043+0000","last_clean":"2024-09-10T16:05:19.876043+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:05:19.876043+0000","last_undegraded":"2024-09-10T16:05:19.876043+0000","last_fullsized":"2024-09-10T16:05:19.876043+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'18841","reported_seq":18989,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:25.408318+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:07:25.408318+0000","last_peered":"2024-09-10T16:07:25.408318+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:07:25.408318+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"43'8800","ondisk_log_start":"43'8800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9039872,"num_objects":2207,"num_object_clones":0,"num_object_copies":4414,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2207,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15096,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2164}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877349+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:05:19.877349+0000","last_peered":"2024-09-10T16:05:19.877349+0000","last_clean":"2024-09-10T16:05:19.877349+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:05:19.877349+0000","last_undegraded":"2024-09-10T16:05:19.877349+0000","last_fullsized":"2024-09-10T16:05:19.877349+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877269+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:05:19.877269+0000","last_peered":"2024-09-10T16:05:19.877269+0000","last_clean":"2024-09-10T16:05:19.877269+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:05:19.877269+0000","last_undegraded":"2024-09-10T16:05:19.877269+0000","last_fullsized":"2024-09-10T16:05:19.877269+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'18915","reported_seq":19065,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:25.440029+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:07:25.440029+0000","last_peered":"2024-09-10T16:07:25.440029+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:07:25.440029+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"43'8900","ondisk_log_start":"43'8900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9352134,"num_objects":2285,"num_object_clones":0,"num_object_copies":4570,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2285,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15078,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2038},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":146658038,"num_objects":35815,"num_object_clones":0,"num_object_copies":71630,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39932,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35815,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":256291,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11457,"num_bytes_recovered":33095680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":505049088,"data_stored":504907263,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160617,"ondisk_log_size":160617,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986289,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":560320,"kb_used_data":209324,"kb_used_omap":4,"kb_used_meta":350971,"kb_avail":93811520,"statfs":{"total":96636764160,"available":96062996480,"internally_reserved":0,"allocated":214347776,"data_stored":214045092,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":359394986},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,0,2,3,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659234,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":466968,"kb_used_data":110408,"kb_used_omap":4,"kb_used_meta":356539,"kb_avail":93904872,"statfs":{"total":96636764160,"available":96158588928,"internally_reserved":0,"allocated":113057792,"data_stored":112725740,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":365096622},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,0,0,2,4,7],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182517,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":618416,"kb_used_data":175824,"kb_used_omap":4,"kb_used_meta":442555,"kb_avail":93753424,"statfs":{"total":96636764160,"available":96003506176,"internally_reserved":0,"allocated":180043776,"data_stored":179741087,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":453177002},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,7,2,3,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":179335168,"data_stored":179299038,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":112103424,"data_stored":112037931,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":213610496,"data_stored":213570294,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:07:28.961 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-10T16:07:28.961 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "47'18487", 'reported_seq': 18568, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:07:25.371415+0000', 'last_change': '2024-09-10T15:59:16.281558+0000', 'last_active': '2024-09-10T16:07:25.371415+0000', 'last_peered': '2024-09-10T16:07:25.371415+0000', 'last_clean': '2024-09-10T15:59:14.132456+0000', 'last_became_active': '2024-09-10T15:59:16.274407+0000', 'last_became_peered': '2024-09-10T15:59:16.274407+0000', 'last_unstale': '2024-09-10T16:07:25.371415+0000', 'last_undegraded': '2024-09-10T15:59:15.851279+0000', 'last_fullsized': '2024-09-10T15:59:15.774356+0000', 'mapping_epoch': 42, 'log_start': "43'8400", 'ondisk_log_start': "43'8400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'objects_scrubbed': 0, 'log_size': 10087, 'log_dups_size': 0, 'ondisk_log_size': 10087, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 8949760, 'num_objects': 2185, 'num_object_clones': 0, 'num_object_copies': 4370, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4259, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2185, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18487, 'num_write_kb': 41344, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2074}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:07:28.961 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-10T16:07:28.961 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "47'18821", 'reported_seq': 18965, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:07:25.436278+0000', 'last_change': '2024-09-10T15:59:16.308264+0000', 'last_active': '2024-09-10T16:07:25.436278+0000', 'last_peered': '2024-09-10T16:07:25.436278+0000', 'last_clean': '2024-09-10T15:59:14.111551+0000', 'last_became_active': '2024-09-10T15:59:16.300040+0000', 'last_became_peered': '2024-09-10T15:59:16.300040+0000', 'last_unstale': '2024-09-10T16:07:25.436278+0000', 'last_undegraded': '2024-09-10T15:59:15.878234+0000', 'last_fullsized': '2024-09-10T15:59:15.769328+0000', 'mapping_epoch': 42, 'log_start': "43'8800", 'ondisk_log_start': "43'8800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3442", 'last_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'objects_scrubbed': 3384, 'log_size': 10021, 'log_dups_size': 0, 'ondisk_log_size': 10021, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1292, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 9175523, 'num_objects': 2241, 'num_object_clones': 0, 'num_object_copies': 4482, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4372, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2241, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 15433, 'num_write_kb': 28580, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2130}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:07:28.962 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-10T16:07:28.962 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "47'18814", 'reported_seq': 18958, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:07:25.345976+0000', 'last_change': '2024-09-10T15:59:16.331899+0000', 'last_active': '2024-09-10T16:07:25.345976+0000', 'last_peered': '2024-09-10T16:07:25.345976+0000', 'last_clean': '2024-09-10T15:59:14.139436+0000', 'last_became_active': '2024-09-10T15:59:16.318715+0000', 'last_became_peered': '2024-09-10T15:59:16.318715+0000', 'last_unstale': '2024-09-10T16:07:25.345976+0000', 'last_undegraded': '2024-09-10T15:59:15.877524+0000', 'last_fullsized': '2024-09-10T15:59:15.769042+0000', 'mapping_epoch': 42, 'log_start': "43'8800", 'ondisk_log_start': "43'8800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3537", 'last_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'objects_scrubbed': 3497, 'log_size': 10014, 'log_dups_size': 0, 'ondisk_log_size': 10014, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1224, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 9290694, 'num_objects': 2270, 'num_object_clones': 0, 'num_object_copies': 4540, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4237, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2270, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 15313, 'num_write_kb': 28172, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1967}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:07:28.962 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-10T16:07:28.962 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "47'18133", 'reported_seq': 18279, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:07:25.782849+0000', 'last_change': '2024-09-10T15:59:16.175680+0000', 'last_active': '2024-09-10T16:07:25.782849+0000', 'last_peered': '2024-09-10T16:07:25.782849+0000', 'last_clean': '2024-09-10T15:59:14.088187+0000', 'last_became_active': '2024-09-10T15:59:16.165810+0000', 'last_became_peered': '2024-09-10T15:59:16.165810+0000', 'last_unstale': '2024-09-10T16:07:25.782849+0000', 'last_undegraded': '2024-09-10T15:59:15.822098+0000', 'last_fullsized': '2024-09-10T15:59:15.769123+0000', 'mapping_epoch': 42, 'log_start': "43'8100", 'ondisk_log_start': "43'8100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2218", 'last_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'objects_scrubbed': 2175, 'log_size': 10033, 'log_dups_size': 0, 'ondisk_log_size': 10033, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 825, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 9519587, 'num_objects': 2325, 'num_object_clones': 0, 'num_object_copies': 4650, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4124, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2325, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 15952, 'num_write_kb': 32204, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': -1798}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:07:28.962 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-10T16:07:28.962 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "47'18652", 'reported_seq': 18787, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:07:25.364573+0000', 'last_change': '2024-09-10T15:59:16.277742+0000', 'last_active': '2024-09-10T16:07:25.364573+0000', 'last_peered': '2024-09-10T16:07:25.364573+0000', 'last_clean': '2024-09-10T15:59:14.129081+0000', 'last_became_active': '2024-09-10T15:59:16.269835+0000', 'last_became_peered': '2024-09-10T15:59:16.269835+0000', 'last_unstale': '2024-09-10T16:07:25.364573+0000', 'last_undegraded': '2024-09-10T15:59:15.860857+0000', 'last_fullsized': '2024-09-10T15:59:15.771410+0000', 'mapping_epoch': 42, 'log_start': "43'8600", 'ondisk_log_start': "43'8600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3557", 'last_scrub_stamp': '2024-09-10T15:59:02.493678+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:02.493678+0000', 'objects_scrubbed': 3488, 'log_size': 10052, 'log_dups_size': 0, 'ondisk_log_size': 10052, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1472, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 9248768, 'num_objects': 2258, 'num_object_clones': 0, 'num_object_copies': 4516, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4257, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2258, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15156, 'num_write_kb': 27852, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1997}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:07:28.962 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-10T16:07:28.962 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "47'18592", 'reported_seq': 18733, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:07:25.833244+0000', 'last_change': '2024-09-10T15:59:16.284311+0000', 'last_active': '2024-09-10T16:07:25.833244+0000', 'last_peered': '2024-09-10T16:07:25.833244+0000', 'last_clean': '2024-09-10T15:59:14.116041+0000', 'last_became_active': '2024-09-10T15:59:16.274543+0000', 'last_became_peered': '2024-09-10T15:59:16.274543+0000', 'last_unstale': '2024-09-10T16:07:25.833244+0000', 'last_undegraded': '2024-09-10T15:59:15.812484+0000', 'last_fullsized': '2024-09-10T15:59:15.770885+0000', 'mapping_epoch': 42, 'log_start': "43'8500", 'ondisk_log_start': "43'8500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2356", 'last_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'objects_scrubbed': 2316, 'log_size': 10092, 'log_dups_size': 0, 'ondisk_log_size': 10092, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 783, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 8966627, 'num_objects': 2190, 'num_object_clones': 0, 'num_object_copies': 4380, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2190, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 16272, 'num_write_kb': 32292, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': -2114}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:07:28.962 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-10T16:07:28.962 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "47'18917", 'reported_seq': 25011, 'reported_epoch': 47, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:07:25.838174+0000', 'last_change': '2024-09-10T16:05:18.271152+0000', 'last_active': '2024-09-10T16:07:25.838174+0000', 'last_peered': '2024-09-10T16:07:25.838174+0000', 'last_clean': '2024-09-10T15:59:14.110654+0000', 'last_became_active': '2024-09-10T15:59:16.164645+0000', 'last_became_peered': '2024-09-10T15:59:16.164645+0000', 'last_unstale': '2024-09-10T16:07:25.838174+0000', 'last_undegraded': '2024-09-10T15:59:15.803593+0000', 'last_fullsized': '2024-09-10T15:59:15.769083+0000', 'mapping_epoch': 42, 'log_start': "43'8900", 'ondisk_log_start': "43'8900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2486", 'last_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'objects_scrubbed': 2440, 'log_size': 10017, 'log_dups_size': 0, 'ondisk_log_size': 10017, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 792, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 9228288, 'num_objects': 2253, 'num_object_clones': 0, 'num_object_copies': 4506, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1375, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2253, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16471, 'num_write_kb': 32568, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2979, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 878}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:07:28.962 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-10T16:07:28.963 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "47'18622", 'reported_seq': 18763, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:07:25.346363+0000', 'last_change': '2024-09-10T15:59:16.315641+0000', 'last_active': '2024-09-10T16:07:25.346363+0000', 'last_peered': '2024-09-10T16:07:25.346363+0000', 'last_clean': '2024-09-10T15:59:14.117001+0000', 'last_became_active': '2024-09-10T15:59:16.302245+0000', 'last_became_peered': '2024-09-10T15:59:16.302245+0000', 'last_unstale': '2024-09-10T16:07:25.346363+0000', 'last_undegraded': '2024-09-10T15:59:15.845012+0000', 'last_fullsized': '2024-09-10T15:59:15.770337+0000', 'mapping_epoch': 42, 'log_start': "43'8600", 'ondisk_log_start': "43'8600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3404", 'last_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'objects_scrubbed': 3367, 'log_size': 10022, 'log_dups_size': 0, 'ondisk_log_size': 10022, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 983, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 9076736, 'num_objects': 2216, 'num_object_clones': 0, 'num_object_copies': 4432, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2216, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15253, 'num_write_kb': 28204, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2088}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:07:28.963 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-10T16:07:28.963 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "47'18841", 'reported_seq': 18989, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:07:25.408318+0000', 'last_change': '2024-09-10T15:59:16.311291+0000', 'last_active': '2024-09-10T16:07:25.408318+0000', 'last_peered': '2024-09-10T16:07:25.408318+0000', 'last_clean': '2024-09-10T15:59:14.116363+0000', 'last_became_active': '2024-09-10T15:59:16.301244+0000', 'last_became_peered': '2024-09-10T15:59:16.301244+0000', 'last_unstale': '2024-09-10T16:07:25.408318+0000', 'last_undegraded': '2024-09-10T15:59:15.883434+0000', 'last_fullsized': '2024-09-10T15:59:15.772756+0000', 'mapping_epoch': 42, 'log_start': "43'8800", 'ondisk_log_start': "43'8800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3789", 'last_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'objects_scrubbed': 3741, 'log_size': 10041, 'log_dups_size': 0, 'ondisk_log_size': 10041, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1440, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 9039872, 'num_objects': 2207, 'num_object_clones': 0, 'num_object_copies': 4414, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4373, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2207, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15096, 'num_write_kb': 27124, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2164}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:07:28.963 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-10T16:07:28.963 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "47'18915", 'reported_seq': 19065, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:07:25.440029+0000', 'last_change': '2024-09-10T15:59:16.313409+0000', 'last_active': '2024-09-10T16:07:25.440029+0000', 'last_peered': '2024-09-10T16:07:25.440029+0000', 'last_clean': '2024-09-10T15:59:14.129780+0000', 'last_became_active': '2024-09-10T15:59:16.300932+0000', 'last_became_peered': '2024-09-10T15:59:16.300932+0000', 'last_unstale': '2024-09-10T16:07:25.440029+0000', 'last_undegraded': '2024-09-10T15:59:15.887823+0000', 'last_fullsized': '2024-09-10T15:59:15.768513+0000', 'mapping_epoch': 42, 'log_start': "43'8900", 'ondisk_log_start': "43'8900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3880", 'last_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'objects_scrubbed': 3827, 'log_size': 10015, 'log_dups_size': 0, 'ondisk_log_size': 10015, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1397, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 9352134, 'num_objects': 2285, 'num_object_clones': 0, 'num_object_copies': 4570, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4325, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2285, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15078, 'num_write_kb': 27072, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2038}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:07:28.963 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T16:07:29.683 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:07:29.683 INFO:teuthology.orchestra.run.smithi115.stdout:{"fsid":"7effee29-df05-4351-9741-ee203564ed81","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 39932/71634 objects degraded (55.744%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":613,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":47,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983954,"num_in_osds":3,"osd_in_since":1725983947,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":35817,"data_bytes":146658057,"bytes_used":1685200896,"bytes_avail":288225091584,"bytes_total":289910292480,"degraded_objects":39932,"degraded_total":71634,"degraded_ratio":0.5574447887874473,"recovering_objects_per_sec":34,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":410,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":449},"fsmap":{"epoch":1,"btime":"2024-09-10T15:57:14:967120+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T16:00:13.560776+0000","services":{}},"progress_events":{}} 2024-09-10T16:07:29.705 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-10T16:07:29.706 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:07:30.230 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:07:30.232 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:07:30.257 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":347,"stamp":"2024-09-10T16:07:29.740245+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":145576713,"num_objects":35553,"num_object_clones":0,"num_object_copies":71106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39932,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35553,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":256557,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11459,"num_bytes_recovered":33095699,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160684,"ondisk_log_size":160684,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1660904,"kb_used_data":488544,"kb_used_omap":12,"kb_used_meta":1172275,"kb_avail":281454616,"statfs":{"total":289910292480,"available":288209526784,"internally_reserved":0,"allocated":500269056,"data_stored":499331631,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1200409602},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,1,1,2,10,6,8,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-20205568,"num_objects":-4933,"num_object_clones":0,"num_object_copies":-9866,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-271,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4933,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4933,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":271,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002699"},"pg_stats":[{"pgid":"2.f","version":"47'18487","reported_seq":18568,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:25.371415+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:07:25.371415+0000","last_peered":"2024-09-10T16:07:25.371415+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:07:25.371415+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"43'8400","ondisk_log_start":"43'8400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8949760,"num_objects":2185,"num_object_clones":0,"num_object_copies":4370,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2185,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18487,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2074}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'18615","reported_seq":18734,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:07:25.681168+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:07:25.681168+0000","last_peered":"2024-09-10T16:07:25.681168+0000","last_clean":"2024-09-10T16:07:25.681168+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:07:25.681168+0000","last_undegraded":"2024-09-10T16:07:25.681168+0000","last_fullsized":"2024-09-10T16:07:25.681168+0000","mapping_epoch":35,"log_start":"43'8600","ondisk_log_start":"43'8600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9130950,"num_objects":2231,"num_object_clones":0,"num_object_copies":4462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2231,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16720,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'18662","reported_seq":27346,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:07:29.166034+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:07:29.166034+0000","last_peered":"2024-09-10T16:07:29.166034+0000","last_clean":"2024-09-10T16:07:29.166034+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:07:29.166034+0000","last_undegraded":"2024-09-10T16:07:29.166034+0000","last_fullsized":"2024-09-10T16:07:29.166034+0000","mapping_epoch":44,"log_start":"43'8600","ondisk_log_start":"43'8600","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8744984,"num_objects":2136,"num_object_clones":0,"num_object_copies":4272,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2136,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16666,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'18498","reported_seq":18585,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:07:25.785793+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:07:25.785793+0000","last_peered":"2024-09-10T16:07:25.785793+0000","last_clean":"2024-09-10T16:07:25.785793+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:07:25.785793+0000","last_undegraded":"2024-09-10T16:07:25.785793+0000","last_fullsized":"2024-09-10T16:07:25.785793+0000","mapping_epoch":35,"log_start":"43'8400","ondisk_log_start":"43'8400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8855552,"num_objects":2162,"num_object_clones":0,"num_object_copies":4324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2162,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18498,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'18704","reported_seq":18840,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:07:25.458955+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:07:25.458955+0000","last_peered":"2024-09-10T16:07:25.458955+0000","last_clean":"2024-09-10T16:07:25.458955+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:07:25.458955+0000","last_undegraded":"2024-09-10T16:07:25.458955+0000","last_fullsized":"2024-09-10T16:07:25.458955+0000","mapping_epoch":34,"log_start":"43'8700","ondisk_log_start":"43'8700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8958435,"num_objects":2188,"num_object_clones":0,"num_object_copies":4376,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2188,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15415,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'18821","reported_seq":18965,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:25.436278+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:07:25.436278+0000","last_peered":"2024-09-10T16:07:25.436278+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:07:25.436278+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"43'8800","ondisk_log_start":"43'8800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9175523,"num_objects":2241,"num_object_clones":0,"num_object_copies":4482,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2241,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15433,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2130},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'18814","reported_seq":18958,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:25.345976+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:07:25.345976+0000","last_peered":"2024-09-10T16:07:25.345976+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:07:25.345976+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"43'8800","ondisk_log_start":"43'8800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9290694,"num_objects":2270,"num_object_clones":0,"num_object_copies":4540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2270,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":15313,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1967},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'19038","reported_seq":19174,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:07:25.454108+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:07:25.454108+0000","last_peered":"2024-09-10T16:07:25.454108+0000","last_clean":"2024-09-10T16:07:25.454108+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:07:25.454108+0000","last_undegraded":"2024-09-10T16:07:25.454108+0000","last_fullsized":"2024-09-10T16:07:25.454108+0000","mapping_epoch":34,"log_start":"43'9000","ondisk_log_start":"43'9000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9322496,"num_objects":2276,"num_object_clones":0,"num_object_copies":4552,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2276,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15885,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.876943+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:05:19.876943+0000","last_peered":"2024-09-10T16:05:19.876943+0000","last_clean":"2024-09-10T16:05:19.876943+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:05:19.876943+0000","last_undegraded":"2024-09-10T16:05:19.876943+0000","last_fullsized":"2024-09-10T16:05:19.876943+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'18133","reported_seq":18279,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:25.782849+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:07:25.782849+0000","last_peered":"2024-09-10T16:07:25.782849+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:07:25.782849+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"43'8100","ondisk_log_start":"43'8100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9519587,"num_objects":2325,"num_object_clones":0,"num_object_copies":4650,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2325,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15952,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-1798}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":100,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.875849+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:05:19.875849+0000","last_peered":"2024-09-10T16:05:19.875849+0000","last_clean":"2024-09-10T16:05:19.875849+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:05:19.875849+0000","last_undegraded":"2024-09-10T16:05:19.875849+0000","last_fullsized":"2024-09-10T16:05:19.875849+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'18652","reported_seq":18787,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:25.364573+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:07:25.364573+0000","last_peered":"2024-09-10T16:07:25.364573+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:07:25.364573+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"43'8600","ondisk_log_start":"43'8600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9248768,"num_objects":2258,"num_object_clones":0,"num_object_copies":4516,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2258,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15156,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1997}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":110,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.872023+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:05:19.872023+0000","last_peered":"2024-09-10T16:05:19.872023+0000","last_clean":"2024-09-10T16:05:19.872023+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:05:19.872023+0000","last_undegraded":"2024-09-10T16:05:19.872023+0000","last_fullsized":"2024-09-10T16:05:19.872023+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'18592","reported_seq":18733,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:25.833244+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:07:25.833244+0000","last_peered":"2024-09-10T16:07:25.833244+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:07:25.833244+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"43'8500","ondisk_log_start":"43'8500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8966627,"num_objects":2190,"num_object_clones":0,"num_object_copies":4380,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2190,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16272,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-2114}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877148+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:05:19.877148+0000","last_peered":"2024-09-10T16:05:19.877148+0000","last_clean":"2024-09-10T16:05:19.877148+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:05:19.877148+0000","last_undegraded":"2024-09-10T16:05:19.877148+0000","last_fullsized":"2024-09-10T16:05:19.877148+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'18917","reported_seq":25011,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:25.838174+0000","last_change":"2024-09-10T16:05:18.271152+0000","last_active":"2024-09-10T16:07:25.838174+0000","last_peered":"2024-09-10T16:07:25.838174+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:07:25.838174+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"43'8900","ondisk_log_start":"43'8900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9228288,"num_objects":2253,"num_object_clones":0,"num_object_copies":4506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1375,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2253,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16471,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2979,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":878}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'18622","reported_seq":18763,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:25.346363+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:07:25.346363+0000","last_peered":"2024-09-10T16:07:25.346363+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:07:25.346363+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"43'8600","ondisk_log_start":"43'8600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9076736,"num_objects":2216,"num_object_clones":0,"num_object_copies":4432,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2216,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15253,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2088}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877654+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:05:19.877654+0000","last_peered":"2024-09-10T16:05:19.877654+0000","last_clean":"2024-09-10T16:05:19.877654+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:05:19.877654+0000","last_undegraded":"2024-09-10T16:05:19.877654+0000","last_fullsized":"2024-09-10T16:05:19.877654+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'18470","reported_seq":27041,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:07:29.242778+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:07:29.242778+0000","last_peered":"2024-09-10T16:07:29.242778+0000","last_clean":"2024-09-10T16:07:29.242778+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:07:29.242778+0000","last_undegraded":"2024-09-10T16:07:29.242778+0000","last_fullsized":"2024-09-10T16:07:29.242778+0000","mapping_epoch":46,"log_start":"43'8400","ondisk_log_start":"43'8400","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8716288,"num_objects":2128,"num_object_clones":0,"num_object_copies":4256,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2128,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14860,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":100,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.876043+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:05:19.876043+0000","last_peered":"2024-09-10T16:05:19.876043+0000","last_clean":"2024-09-10T16:05:19.876043+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:05:19.876043+0000","last_undegraded":"2024-09-10T16:05:19.876043+0000","last_fullsized":"2024-09-10T16:05:19.876043+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'18841","reported_seq":18989,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:25.408318+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:07:25.408318+0000","last_peered":"2024-09-10T16:07:25.408318+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:07:25.408318+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"43'8800","ondisk_log_start":"43'8800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9039872,"num_objects":2207,"num_object_clones":0,"num_object_copies":4414,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2207,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15096,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2164}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877349+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:05:19.877349+0000","last_peered":"2024-09-10T16:05:19.877349+0000","last_clean":"2024-09-10T16:05:19.877349+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:05:19.877349+0000","last_undegraded":"2024-09-10T16:05:19.877349+0000","last_fullsized":"2024-09-10T16:05:19.877349+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877269+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:05:19.877269+0000","last_peered":"2024-09-10T16:05:19.877269+0000","last_clean":"2024-09-10T16:05:19.877269+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:05:19.877269+0000","last_undegraded":"2024-09-10T16:05:19.877269+0000","last_fullsized":"2024-09-10T16:05:19.877269+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'18915","reported_seq":19065,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:25.440029+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:07:25.440029+0000","last_peered":"2024-09-10T16:07:25.440029+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:07:25.440029+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"43'8900","ondisk_log_start":"43'8900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9352134,"num_objects":2285,"num_object_clones":0,"num_object_copies":4570,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2285,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15078,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2038},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":145576694,"num_objects":35551,"num_object_clones":0,"num_object_copies":71102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39932,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35551,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":256555,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11457,"num_bytes_recovered":33095680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":497856512,"data_stored":497714687,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160681,"ondisk_log_size":160681,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986289,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":560320,"kb_used_data":209324,"kb_used_omap":4,"kb_used_meta":350971,"kb_avail":93811520,"statfs":{"total":96636764160,"available":96062996480,"internally_reserved":0,"allocated":214347776,"data_stored":214045092,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":359394986},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,0,2,3,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659235,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":482168,"kb_used_data":103396,"kb_used_omap":4,"kb_used_meta":378747,"kb_avail":93889672,"statfs":{"total":96636764160,"available":96143024128,"internally_reserved":0,"allocated":105877504,"data_stored":105545452,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":387837614},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,1,1,1,3,2,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182517,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":618416,"kb_used_data":175824,"kb_used_omap":4,"kb_used_meta":442555,"kb_avail":93753424,"statfs":{"total":96636764160,"available":96003506176,"internally_reserved":0,"allocated":180043776,"data_stored":179741087,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":453177002},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,7,2,3,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":179335168,"data_stored":179299038,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":104910848,"data_stored":104845355,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":213610496,"data_stored":213570294,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:07:33.260 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:07:33.826 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:07:33.829 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:07:33.852 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":349,"stamp":"2024-09-10T16:07:33.741154+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":137138953,"num_objects":33493,"num_object_clones":0,"num_object_copies":66986,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39791,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33493,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":258617,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11600,"num_bytes_recovered":33095699,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160944,"ondisk_log_size":160944,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1645276,"kb_used_data":477092,"kb_used_omap":12,"kb_used_meta":1168115,"kb_avail":281470244,"statfs":{"total":289910292480,"available":288225529856,"internally_reserved":0,"allocated":488542208,"data_stored":487604783,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1196149762},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,1,1,1,15,8,11,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19046400,"num_objects":-4650,"num_object_clones":0,"num_object_copies":-9300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-282,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4650,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4650,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":282,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002712"},"pg_stats":[{"pgid":"2.f","version":"47'18616","reported_seq":18697,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:30.437828+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:07:30.437828+0000","last_peered":"2024-09-10T16:07:30.437828+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:07:30.437828+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"43'8600","ondisk_log_start":"43'8600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8421376,"num_objects":2056,"num_object_clones":0,"num_object_copies":4112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2056,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18616,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2203}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'18768","reported_seq":18887,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:07:30.842095+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:07:30.842095+0000","last_peered":"2024-09-10T16:07:30.842095+0000","last_clean":"2024-09-10T16:07:30.842095+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:07:30.842095+0000","last_undegraded":"2024-09-10T16:07:30.842095+0000","last_fullsized":"2024-09-10T16:07:30.842095+0000","mapping_epoch":35,"log_start":"43'8700","ondisk_log_start":"43'8700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8504262,"num_objects":2078,"num_object_clones":0,"num_object_copies":4156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2078,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16873,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'18662","reported_seq":27346,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:07:29.166034+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:07:29.166034+0000","last_peered":"2024-09-10T16:07:29.166034+0000","last_clean":"2024-09-10T16:07:29.166034+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:07:29.166034+0000","last_undegraded":"2024-09-10T16:07:29.166034+0000","last_fullsized":"2024-09-10T16:07:29.166034+0000","mapping_epoch":44,"log_start":"43'8600","ondisk_log_start":"43'8600","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8744984,"num_objects":2136,"num_object_clones":0,"num_object_copies":4272,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2136,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16666,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'18645","reported_seq":18732,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:07:30.783234+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:07:30.783234+0000","last_peered":"2024-09-10T16:07:30.783234+0000","last_clean":"2024-09-10T16:07:30.783234+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:07:30.783234+0000","last_undegraded":"2024-09-10T16:07:30.783234+0000","last_fullsized":"2024-09-10T16:07:30.783234+0000","mapping_epoch":35,"log_start":"43'8600","ondisk_log_start":"43'8600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8253440,"num_objects":2015,"num_object_clones":0,"num_object_copies":4030,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2015,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18645,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'18839","reported_seq":18975,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:07:30.497448+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:07:30.497448+0000","last_peered":"2024-09-10T16:07:30.497448+0000","last_clean":"2024-09-10T16:07:30.497448+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:07:30.497448+0000","last_undegraded":"2024-09-10T16:07:30.497448+0000","last_fullsized":"2024-09-10T16:07:30.497448+0000","mapping_epoch":34,"log_start":"43'8800","ondisk_log_start":"43'8800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8405475,"num_objects":2053,"num_object_clones":0,"num_object_copies":4106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2053,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15550,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'18958","reported_seq":19102,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:30.433007+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:07:30.433007+0000","last_peered":"2024-09-10T16:07:30.433007+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:07:30.433007+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"43'8900","ondisk_log_start":"43'8900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8614371,"num_objects":2104,"num_object_clones":0,"num_object_copies":4208,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2104,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15570,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2267},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'18962","reported_seq":19106,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:30.382656+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:07:30.382656+0000","last_peered":"2024-09-10T16:07:30.382656+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:07:30.382656+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"43'8900","ondisk_log_start":"43'8900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8684486,"num_objects":2122,"num_object_clones":0,"num_object_copies":4244,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2122,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":15461,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2115},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'19174","reported_seq":19309,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:07:30.478848+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:07:30.478848+0000","last_peered":"2024-09-10T16:07:30.478848+0000","last_clean":"2024-09-10T16:07:30.478848+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:07:30.478848+0000","last_undegraded":"2024-09-10T16:07:30.478848+0000","last_fullsized":"2024-09-10T16:07:30.478848+0000","mapping_epoch":34,"log_start":"43'9100","ondisk_log_start":"43'9100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8765440,"num_objects":2140,"num_object_clones":0,"num_object_copies":4280,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2140,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16021,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.876943+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:05:19.876943+0000","last_peered":"2024-09-10T16:05:19.876943+0000","last_clean":"2024-09-10T16:05:19.876943+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:05:19.876943+0000","last_undegraded":"2024-09-10T16:05:19.876943+0000","last_fullsized":"2024-09-10T16:05:19.876943+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'18277","reported_seq":18423,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:30.834178+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:07:30.834178+0000","last_peered":"2024-09-10T16:07:30.834178+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:07:30.834178+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"43'8200","ondisk_log_start":"43'8200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8929763,"num_objects":2181,"num_object_clones":0,"num_object_copies":4362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2181,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16096,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-1942}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":100,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.875849+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:05:19.875849+0000","last_peered":"2024-09-10T16:05:19.875849+0000","last_clean":"2024-09-10T16:05:19.875849+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:05:19.875849+0000","last_undegraded":"2024-09-10T16:05:19.875849+0000","last_fullsized":"2024-09-10T16:05:19.875849+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'18806","reported_seq":18943,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:30.434065+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:07:30.434065+0000","last_peered":"2024-09-10T16:07:30.434065+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:07:30.434065+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"43'8800","ondisk_log_start":"43'8800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8617984,"num_objects":2104,"num_object_clones":0,"num_object_copies":4208,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2104,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15310,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2151}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":110,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.872023+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:05:19.872023+0000","last_peered":"2024-09-10T16:05:19.872023+0000","last_clean":"2024-09-10T16:05:19.872023+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:05:19.872023+0000","last_undegraded":"2024-09-10T16:05:19.872023+0000","last_fullsized":"2024-09-10T16:05:19.872023+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'18734","reported_seq":18875,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:30.864306+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:07:30.864306+0000","last_peered":"2024-09-10T16:07:30.864306+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:07:30.864306+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"43'8700","ondisk_log_start":"43'8700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8384995,"num_objects":2048,"num_object_clones":0,"num_object_copies":4096,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2048,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16414,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-2256}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877148+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:05:19.877148+0000","last_peered":"2024-09-10T16:05:19.877148+0000","last_clean":"2024-09-10T16:05:19.877148+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:05:19.877148+0000","last_undegraded":"2024-09-10T16:05:19.877148+0000","last_fullsized":"2024-09-10T16:05:19.877148+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'19076","reported_seq":25452,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:30.870587+0000","last_change":"2024-09-10T16:05:18.271152+0000","last_active":"2024-09-10T16:07:30.870587+0000","last_peered":"2024-09-10T16:07:30.870587+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:07:30.870587+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"43'9000","ondisk_log_start":"43'9000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8577024,"num_objects":2094,"num_object_clones":0,"num_object_copies":4188,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1234,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2094,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16630,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3120,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":860}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'18760","reported_seq":18901,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:30.389592+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:07:30.389592+0000","last_peered":"2024-09-10T16:07:30.389592+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:07:30.389592+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"43'8700","ondisk_log_start":"43'8700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8511488,"num_objects":2078,"num_object_clones":0,"num_object_copies":4156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2078,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15391,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2226}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877654+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:05:19.877654+0000","last_peered":"2024-09-10T16:05:19.877654+0000","last_clean":"2024-09-10T16:05:19.877654+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:05:19.877654+0000","last_undegraded":"2024-09-10T16:05:19.877654+0000","last_fullsized":"2024-09-10T16:05:19.877654+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'18470","reported_seq":27041,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:07:29.242778+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:07:29.242778+0000","last_peered":"2024-09-10T16:07:29.242778+0000","last_clean":"2024-09-10T16:07:29.242778+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:07:29.242778+0000","last_undegraded":"2024-09-10T16:07:29.242778+0000","last_fullsized":"2024-09-10T16:07:29.242778+0000","mapping_epoch":46,"log_start":"43'8400","ondisk_log_start":"43'8400","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8716288,"num_objects":2128,"num_object_clones":0,"num_object_copies":4256,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2128,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14860,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":100,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.876043+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:05:19.876043+0000","last_peered":"2024-09-10T16:05:19.876043+0000","last_clean":"2024-09-10T16:05:19.876043+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:05:19.876043+0000","last_undegraded":"2024-09-10T16:05:19.876043+0000","last_fullsized":"2024-09-10T16:05:19.876043+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'19000","reported_seq":19148,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:30.460688+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:07:30.460688+0000","last_peered":"2024-09-10T16:07:30.460688+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:07:30.460688+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"43'8900","ondisk_log_start":"43'8900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":2048,"num_object_clones":0,"num_object_copies":4096,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2048,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15255,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2323}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877349+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:05:19.877349+0000","last_peered":"2024-09-10T16:05:19.877349+0000","last_clean":"2024-09-10T16:05:19.877349+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:05:19.877349+0000","last_undegraded":"2024-09-10T16:05:19.877349+0000","last_fullsized":"2024-09-10T16:05:19.877349+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877269+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:05:19.877269+0000","last_peered":"2024-09-10T16:05:19.877269+0000","last_clean":"2024-09-10T16:05:19.877269+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:05:19.877269+0000","last_undegraded":"2024-09-10T16:05:19.877269+0000","last_fullsized":"2024-09-10T16:05:19.877269+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'19094","reported_seq":19243,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:30.423962+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:07:30.423962+0000","last_peered":"2024-09-10T16:07:30.423962+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:07:30.423962+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"43'9000","ondisk_log_start":"43'9000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8618950,"num_objects":2106,"num_object_clones":0,"num_object_copies":4212,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2106,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15257,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2217},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":137138934,"num_objects":33491,"num_object_clones":0,"num_object_copies":66982,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39791,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33491,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":258615,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11598,"num_bytes_recovered":33095680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":486002688,"data_stored":485860863,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160941,"ondisk_log_size":160941,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986290,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":543412,"kb_used_data":204896,"kb_used_omap":4,"kb_used_meta":338491,"kb_avail":93828428,"statfs":{"total":96636764160,"available":96080310272,"internally_reserved":0,"allocated":209813504,"data_stored":209510820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":346615466},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,5,4,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659235,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":482168,"kb_used_data":103396,"kb_used_omap":4,"kb_used_meta":378747,"kb_avail":93889672,"statfs":{"total":96636764160,"available":96143024128,"internally_reserved":0,"allocated":105877504,"data_stored":105545452,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":387837614},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,1,1,1,3,2,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182518,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":619696,"kb_used_data":168800,"kb_used_omap":4,"kb_used_meta":450875,"kb_avail":93752144,"statfs":{"total":96636764160,"available":96002195456,"internally_reserved":0,"allocated":172851200,"data_stored":172548511,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":461696682},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,7,2,3,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":172220416,"data_stored":172184286,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":104910848,"data_stored":104845355,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":208871424,"data_stored":208831222,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:07:33.853 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:07:34.418 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:07:34.419 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:07:34.440 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":349,"stamp":"2024-09-10T16:07:33.741154+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":137138953,"num_objects":33493,"num_object_clones":0,"num_object_copies":66986,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39791,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33493,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":258617,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11600,"num_bytes_recovered":33095699,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160944,"ondisk_log_size":160944,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1645276,"kb_used_data":477092,"kb_used_omap":12,"kb_used_meta":1168115,"kb_avail":281470244,"statfs":{"total":289910292480,"available":288225529856,"internally_reserved":0,"allocated":488542208,"data_stored":487604783,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1196149762},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,1,1,1,15,8,11,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19046400,"num_objects":-4650,"num_object_clones":0,"num_object_copies":-9300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-282,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4650,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4650,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":282,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002712"},"pg_stats":[{"pgid":"2.f","version":"47'18616","reported_seq":18697,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:30.437828+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:07:30.437828+0000","last_peered":"2024-09-10T16:07:30.437828+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:07:30.437828+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"43'8600","ondisk_log_start":"43'8600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8421376,"num_objects":2056,"num_object_clones":0,"num_object_copies":4112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2056,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18616,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2203}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'18768","reported_seq":18887,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:07:30.842095+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:07:30.842095+0000","last_peered":"2024-09-10T16:07:30.842095+0000","last_clean":"2024-09-10T16:07:30.842095+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:07:30.842095+0000","last_undegraded":"2024-09-10T16:07:30.842095+0000","last_fullsized":"2024-09-10T16:07:30.842095+0000","mapping_epoch":35,"log_start":"43'8700","ondisk_log_start":"43'8700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8504262,"num_objects":2078,"num_object_clones":0,"num_object_copies":4156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2078,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16873,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'18662","reported_seq":27346,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:07:29.166034+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:07:29.166034+0000","last_peered":"2024-09-10T16:07:29.166034+0000","last_clean":"2024-09-10T16:07:29.166034+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:07:29.166034+0000","last_undegraded":"2024-09-10T16:07:29.166034+0000","last_fullsized":"2024-09-10T16:07:29.166034+0000","mapping_epoch":44,"log_start":"43'8600","ondisk_log_start":"43'8600","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8744984,"num_objects":2136,"num_object_clones":0,"num_object_copies":4272,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2136,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16666,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'18645","reported_seq":18732,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:07:30.783234+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:07:30.783234+0000","last_peered":"2024-09-10T16:07:30.783234+0000","last_clean":"2024-09-10T16:07:30.783234+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:07:30.783234+0000","last_undegraded":"2024-09-10T16:07:30.783234+0000","last_fullsized":"2024-09-10T16:07:30.783234+0000","mapping_epoch":35,"log_start":"43'8600","ondisk_log_start":"43'8600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8253440,"num_objects":2015,"num_object_clones":0,"num_object_copies":4030,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2015,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18645,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'18839","reported_seq":18975,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:07:30.497448+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:07:30.497448+0000","last_peered":"2024-09-10T16:07:30.497448+0000","last_clean":"2024-09-10T16:07:30.497448+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:07:30.497448+0000","last_undegraded":"2024-09-10T16:07:30.497448+0000","last_fullsized":"2024-09-10T16:07:30.497448+0000","mapping_epoch":34,"log_start":"43'8800","ondisk_log_start":"43'8800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8405475,"num_objects":2053,"num_object_clones":0,"num_object_copies":4106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2053,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15550,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'18958","reported_seq":19102,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:30.433007+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:07:30.433007+0000","last_peered":"2024-09-10T16:07:30.433007+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:07:30.433007+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"43'8900","ondisk_log_start":"43'8900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8614371,"num_objects":2104,"num_object_clones":0,"num_object_copies":4208,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2104,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15570,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2267},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'18962","reported_seq":19106,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:30.382656+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:07:30.382656+0000","last_peered":"2024-09-10T16:07:30.382656+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:07:30.382656+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"43'8900","ondisk_log_start":"43'8900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8684486,"num_objects":2122,"num_object_clones":0,"num_object_copies":4244,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2122,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":15461,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2115},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'19174","reported_seq":19309,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:07:30.478848+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:07:30.478848+0000","last_peered":"2024-09-10T16:07:30.478848+0000","last_clean":"2024-09-10T16:07:30.478848+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:07:30.478848+0000","last_undegraded":"2024-09-10T16:07:30.478848+0000","last_fullsized":"2024-09-10T16:07:30.478848+0000","mapping_epoch":34,"log_start":"43'9100","ondisk_log_start":"43'9100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8765440,"num_objects":2140,"num_object_clones":0,"num_object_copies":4280,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2140,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16021,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.876943+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:05:19.876943+0000","last_peered":"2024-09-10T16:05:19.876943+0000","last_clean":"2024-09-10T16:05:19.876943+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:05:19.876943+0000","last_undegraded":"2024-09-10T16:05:19.876943+0000","last_fullsized":"2024-09-10T16:05:19.876943+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'18277","reported_seq":18423,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:30.834178+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:07:30.834178+0000","last_peered":"2024-09-10T16:07:30.834178+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:07:30.834178+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"43'8200","ondisk_log_start":"43'8200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8929763,"num_objects":2181,"num_object_clones":0,"num_object_copies":4362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2181,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16096,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-1942}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":100,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.875849+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:05:19.875849+0000","last_peered":"2024-09-10T16:05:19.875849+0000","last_clean":"2024-09-10T16:05:19.875849+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:05:19.875849+0000","last_undegraded":"2024-09-10T16:05:19.875849+0000","last_fullsized":"2024-09-10T16:05:19.875849+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'18806","reported_seq":18943,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:30.434065+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:07:30.434065+0000","last_peered":"2024-09-10T16:07:30.434065+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:07:30.434065+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"43'8800","ondisk_log_start":"43'8800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8617984,"num_objects":2104,"num_object_clones":0,"num_object_copies":4208,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2104,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15310,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2151}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":110,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.872023+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:05:19.872023+0000","last_peered":"2024-09-10T16:05:19.872023+0000","last_clean":"2024-09-10T16:05:19.872023+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:05:19.872023+0000","last_undegraded":"2024-09-10T16:05:19.872023+0000","last_fullsized":"2024-09-10T16:05:19.872023+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'18734","reported_seq":18875,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:30.864306+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:07:30.864306+0000","last_peered":"2024-09-10T16:07:30.864306+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:07:30.864306+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"43'8700","ondisk_log_start":"43'8700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8384995,"num_objects":2048,"num_object_clones":0,"num_object_copies":4096,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2048,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16414,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-2256}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877148+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:05:19.877148+0000","last_peered":"2024-09-10T16:05:19.877148+0000","last_clean":"2024-09-10T16:05:19.877148+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:05:19.877148+0000","last_undegraded":"2024-09-10T16:05:19.877148+0000","last_fullsized":"2024-09-10T16:05:19.877148+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'19076","reported_seq":25452,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:30.870587+0000","last_change":"2024-09-10T16:05:18.271152+0000","last_active":"2024-09-10T16:07:30.870587+0000","last_peered":"2024-09-10T16:07:30.870587+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:07:30.870587+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"43'9000","ondisk_log_start":"43'9000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8577024,"num_objects":2094,"num_object_clones":0,"num_object_copies":4188,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1234,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2094,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16630,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3120,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":860}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'18760","reported_seq":18901,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:30.389592+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:07:30.389592+0000","last_peered":"2024-09-10T16:07:30.389592+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:07:30.389592+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"43'8700","ondisk_log_start":"43'8700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8511488,"num_objects":2078,"num_object_clones":0,"num_object_copies":4156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2078,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15391,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2226}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877654+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:05:19.877654+0000","last_peered":"2024-09-10T16:05:19.877654+0000","last_clean":"2024-09-10T16:05:19.877654+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:05:19.877654+0000","last_undegraded":"2024-09-10T16:05:19.877654+0000","last_fullsized":"2024-09-10T16:05:19.877654+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'18470","reported_seq":27041,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:07:29.242778+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:07:29.242778+0000","last_peered":"2024-09-10T16:07:29.242778+0000","last_clean":"2024-09-10T16:07:29.242778+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:07:29.242778+0000","last_undegraded":"2024-09-10T16:07:29.242778+0000","last_fullsized":"2024-09-10T16:07:29.242778+0000","mapping_epoch":46,"log_start":"43'8400","ondisk_log_start":"43'8400","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8716288,"num_objects":2128,"num_object_clones":0,"num_object_copies":4256,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2128,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14860,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":100,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.876043+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:05:19.876043+0000","last_peered":"2024-09-10T16:05:19.876043+0000","last_clean":"2024-09-10T16:05:19.876043+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:05:19.876043+0000","last_undegraded":"2024-09-10T16:05:19.876043+0000","last_fullsized":"2024-09-10T16:05:19.876043+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'19000","reported_seq":19148,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:30.460688+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:07:30.460688+0000","last_peered":"2024-09-10T16:07:30.460688+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:07:30.460688+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"43'8900","ondisk_log_start":"43'8900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":2048,"num_object_clones":0,"num_object_copies":4096,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2048,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15255,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2323}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877349+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:05:19.877349+0000","last_peered":"2024-09-10T16:05:19.877349+0000","last_clean":"2024-09-10T16:05:19.877349+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:05:19.877349+0000","last_undegraded":"2024-09-10T16:05:19.877349+0000","last_fullsized":"2024-09-10T16:05:19.877349+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877269+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:05:19.877269+0000","last_peered":"2024-09-10T16:05:19.877269+0000","last_clean":"2024-09-10T16:05:19.877269+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:05:19.877269+0000","last_undegraded":"2024-09-10T16:05:19.877269+0000","last_fullsized":"2024-09-10T16:05:19.877269+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'19094","reported_seq":19243,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:30.423962+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:07:30.423962+0000","last_peered":"2024-09-10T16:07:30.423962+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:07:30.423962+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"43'9000","ondisk_log_start":"43'9000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8618950,"num_objects":2106,"num_object_clones":0,"num_object_copies":4212,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2106,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15257,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2217},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":137138934,"num_objects":33491,"num_object_clones":0,"num_object_copies":66982,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39791,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33491,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":258615,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11598,"num_bytes_recovered":33095680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":486002688,"data_stored":485860863,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160941,"ondisk_log_size":160941,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986290,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":543412,"kb_used_data":204896,"kb_used_omap":4,"kb_used_meta":338491,"kb_avail":93828428,"statfs":{"total":96636764160,"available":96080310272,"internally_reserved":0,"allocated":209813504,"data_stored":209510820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":346615466},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,5,4,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659235,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":482168,"kb_used_data":103396,"kb_used_omap":4,"kb_used_meta":378747,"kb_avail":93889672,"statfs":{"total":96636764160,"available":96143024128,"internally_reserved":0,"allocated":105877504,"data_stored":105545452,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":387837614},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,1,1,1,3,2,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182518,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":619696,"kb_used_data":168800,"kb_used_omap":4,"kb_used_meta":450875,"kb_avail":93752144,"statfs":{"total":96636764160,"available":96002195456,"internally_reserved":0,"allocated":172851200,"data_stored":172548511,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":461696682},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,7,2,3,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":172220416,"data_stored":172184286,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":104910848,"data_stored":104845355,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":208871424,"data_stored":208831222,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:07:34.441 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-10T16:07:34.441 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "47'18616", 'reported_seq': 18697, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:07:30.437828+0000', 'last_change': '2024-09-10T15:59:16.281558+0000', 'last_active': '2024-09-10T16:07:30.437828+0000', 'last_peered': '2024-09-10T16:07:30.437828+0000', 'last_clean': '2024-09-10T15:59:14.132456+0000', 'last_became_active': '2024-09-10T15:59:16.274407+0000', 'last_became_peered': '2024-09-10T15:59:16.274407+0000', 'last_unstale': '2024-09-10T16:07:30.437828+0000', 'last_undegraded': '2024-09-10T15:59:15.851279+0000', 'last_fullsized': '2024-09-10T15:59:15.774356+0000', 'mapping_epoch': 42, 'log_start': "43'8600", 'ondisk_log_start': "43'8600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'objects_scrubbed': 0, 'log_size': 10016, 'log_dups_size': 0, 'ondisk_log_size': 10016, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 8421376, 'num_objects': 2056, 'num_object_clones': 0, 'num_object_copies': 4112, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4259, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2056, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18616, 'num_write_kb': 41344, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2203}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:07:34.441 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-10T16:07:34.441 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "47'18958", 'reported_seq': 19102, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:07:30.433007+0000', 'last_change': '2024-09-10T15:59:16.308264+0000', 'last_active': '2024-09-10T16:07:30.433007+0000', 'last_peered': '2024-09-10T16:07:30.433007+0000', 'last_clean': '2024-09-10T15:59:14.111551+0000', 'last_became_active': '2024-09-10T15:59:16.300040+0000', 'last_became_peered': '2024-09-10T15:59:16.300040+0000', 'last_unstale': '2024-09-10T16:07:30.433007+0000', 'last_undegraded': '2024-09-10T15:59:15.878234+0000', 'last_fullsized': '2024-09-10T15:59:15.769328+0000', 'mapping_epoch': 42, 'log_start': "43'8900", 'ondisk_log_start': "43'8900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3442", 'last_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'objects_scrubbed': 3384, 'log_size': 10058, 'log_dups_size': 0, 'ondisk_log_size': 10058, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1292, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 8614371, 'num_objects': 2104, 'num_object_clones': 0, 'num_object_copies': 4208, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4372, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2104, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 15570, 'num_write_kb': 28580, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2267}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:07:34.441 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-10T16:07:34.442 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "47'18962", 'reported_seq': 19106, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:07:30.382656+0000', 'last_change': '2024-09-10T15:59:16.331899+0000', 'last_active': '2024-09-10T16:07:30.382656+0000', 'last_peered': '2024-09-10T16:07:30.382656+0000', 'last_clean': '2024-09-10T15:59:14.139436+0000', 'last_became_active': '2024-09-10T15:59:16.318715+0000', 'last_became_peered': '2024-09-10T15:59:16.318715+0000', 'last_unstale': '2024-09-10T16:07:30.382656+0000', 'last_undegraded': '2024-09-10T15:59:15.877524+0000', 'last_fullsized': '2024-09-10T15:59:15.769042+0000', 'mapping_epoch': 42, 'log_start': "43'8900", 'ondisk_log_start': "43'8900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3537", 'last_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'objects_scrubbed': 3497, 'log_size': 10062, 'log_dups_size': 0, 'ondisk_log_size': 10062, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1224, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 8684486, 'num_objects': 2122, 'num_object_clones': 0, 'num_object_copies': 4244, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4237, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2122, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 15461, 'num_write_kb': 28172, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2115}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:07:34.442 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-10T16:07:34.442 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "47'18277", 'reported_seq': 18423, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:07:30.834178+0000', 'last_change': '2024-09-10T15:59:16.175680+0000', 'last_active': '2024-09-10T16:07:30.834178+0000', 'last_peered': '2024-09-10T16:07:30.834178+0000', 'last_clean': '2024-09-10T15:59:14.088187+0000', 'last_became_active': '2024-09-10T15:59:16.165810+0000', 'last_became_peered': '2024-09-10T15:59:16.165810+0000', 'last_unstale': '2024-09-10T16:07:30.834178+0000', 'last_undegraded': '2024-09-10T15:59:15.822098+0000', 'last_fullsized': '2024-09-10T15:59:15.769123+0000', 'mapping_epoch': 42, 'log_start': "43'8200", 'ondisk_log_start': "43'8200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2218", 'last_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'objects_scrubbed': 2175, 'log_size': 10077, 'log_dups_size': 0, 'ondisk_log_size': 10077, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 825, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 8929763, 'num_objects': 2181, 'num_object_clones': 0, 'num_object_copies': 4362, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4124, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2181, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 16096, 'num_write_kb': 32204, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': -1942}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:07:34.442 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-10T16:07:34.442 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "47'18806", 'reported_seq': 18943, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:07:30.434065+0000', 'last_change': '2024-09-10T15:59:16.277742+0000', 'last_active': '2024-09-10T16:07:30.434065+0000', 'last_peered': '2024-09-10T16:07:30.434065+0000', 'last_clean': '2024-09-10T15:59:14.129081+0000', 'last_became_active': '2024-09-10T15:59:16.269835+0000', 'last_became_peered': '2024-09-10T15:59:16.269835+0000', 'last_unstale': '2024-09-10T16:07:30.434065+0000', 'last_undegraded': '2024-09-10T15:59:15.860857+0000', 'last_fullsized': '2024-09-10T15:59:15.771410+0000', 'mapping_epoch': 42, 'log_start': "43'8800", 'ondisk_log_start': "43'8800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3557", 'last_scrub_stamp': '2024-09-10T15:59:02.493678+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:02.493678+0000', 'objects_scrubbed': 3488, 'log_size': 10006, 'log_dups_size': 0, 'ondisk_log_size': 10006, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1472, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 8617984, 'num_objects': 2104, 'num_object_clones': 0, 'num_object_copies': 4208, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4257, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2104, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15310, 'num_write_kb': 27852, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2151}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:07:34.442 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-10T16:07:34.442 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "47'18734", 'reported_seq': 18875, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:07:30.864306+0000', 'last_change': '2024-09-10T15:59:16.284311+0000', 'last_active': '2024-09-10T16:07:30.864306+0000', 'last_peered': '2024-09-10T16:07:30.864306+0000', 'last_clean': '2024-09-10T15:59:14.116041+0000', 'last_became_active': '2024-09-10T15:59:16.274543+0000', 'last_became_peered': '2024-09-10T15:59:16.274543+0000', 'last_unstale': '2024-09-10T16:07:30.864306+0000', 'last_undegraded': '2024-09-10T15:59:15.812484+0000', 'last_fullsized': '2024-09-10T15:59:15.770885+0000', 'mapping_epoch': 42, 'log_start': "43'8700", 'ondisk_log_start': "43'8700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2356", 'last_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'objects_scrubbed': 2316, 'log_size': 10034, 'log_dups_size': 0, 'ondisk_log_size': 10034, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 783, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 8384995, 'num_objects': 2048, 'num_object_clones': 0, 'num_object_copies': 4096, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2048, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 16414, 'num_write_kb': 32292, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': -2256}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:07:34.442 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-10T16:07:34.442 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "47'19076", 'reported_seq': 25452, 'reported_epoch': 47, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:07:30.870587+0000', 'last_change': '2024-09-10T16:05:18.271152+0000', 'last_active': '2024-09-10T16:07:30.870587+0000', 'last_peered': '2024-09-10T16:07:30.870587+0000', 'last_clean': '2024-09-10T15:59:14.110654+0000', 'last_became_active': '2024-09-10T15:59:16.164645+0000', 'last_became_peered': '2024-09-10T15:59:16.164645+0000', 'last_unstale': '2024-09-10T16:07:30.870587+0000', 'last_undegraded': '2024-09-10T15:59:15.803593+0000', 'last_fullsized': '2024-09-10T15:59:15.769083+0000', 'mapping_epoch': 42, 'log_start': "43'9000", 'ondisk_log_start': "43'9000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2486", 'last_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'objects_scrubbed': 2440, 'log_size': 10076, 'log_dups_size': 0, 'ondisk_log_size': 10076, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 792, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 8577024, 'num_objects': 2094, 'num_object_clones': 0, 'num_object_copies': 4188, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1234, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2094, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16630, 'num_write_kb': 32568, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3120, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 860}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:07:34.442 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-10T16:07:34.443 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "47'18760", 'reported_seq': 18901, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:07:30.389592+0000', 'last_change': '2024-09-10T15:59:16.315641+0000', 'last_active': '2024-09-10T16:07:30.389592+0000', 'last_peered': '2024-09-10T16:07:30.389592+0000', 'last_clean': '2024-09-10T15:59:14.117001+0000', 'last_became_active': '2024-09-10T15:59:16.302245+0000', 'last_became_peered': '2024-09-10T15:59:16.302245+0000', 'last_unstale': '2024-09-10T16:07:30.389592+0000', 'last_undegraded': '2024-09-10T15:59:15.845012+0000', 'last_fullsized': '2024-09-10T15:59:15.770337+0000', 'mapping_epoch': 42, 'log_start': "43'8700", 'ondisk_log_start': "43'8700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3404", 'last_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'objects_scrubbed': 3367, 'log_size': 10060, 'log_dups_size': 0, 'ondisk_log_size': 10060, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 983, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 8511488, 'num_objects': 2078, 'num_object_clones': 0, 'num_object_copies': 4156, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2078, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15391, 'num_write_kb': 28204, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2226}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:07:34.443 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-10T16:07:34.443 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "47'19000", 'reported_seq': 19148, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:07:30.460688+0000', 'last_change': '2024-09-10T15:59:16.311291+0000', 'last_active': '2024-09-10T16:07:30.460688+0000', 'last_peered': '2024-09-10T16:07:30.460688+0000', 'last_clean': '2024-09-10T15:59:14.116363+0000', 'last_became_active': '2024-09-10T15:59:16.301244+0000', 'last_became_peered': '2024-09-10T15:59:16.301244+0000', 'last_unstale': '2024-09-10T16:07:30.460688+0000', 'last_undegraded': '2024-09-10T15:59:15.883434+0000', 'last_fullsized': '2024-09-10T15:59:15.772756+0000', 'mapping_epoch': 42, 'log_start': "43'8900", 'ondisk_log_start': "43'8900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3789", 'last_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'objects_scrubbed': 3741, 'log_size': 10100, 'log_dups_size': 0, 'ondisk_log_size': 10100, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1440, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 8388608, 'num_objects': 2048, 'num_object_clones': 0, 'num_object_copies': 4096, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4373, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2048, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15255, 'num_write_kb': 27124, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2323}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:07:34.443 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-10T16:07:34.443 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "47'19094", 'reported_seq': 19243, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:07:30.423962+0000', 'last_change': '2024-09-10T15:59:16.313409+0000', 'last_active': '2024-09-10T16:07:30.423962+0000', 'last_peered': '2024-09-10T16:07:30.423962+0000', 'last_clean': '2024-09-10T15:59:14.129780+0000', 'last_became_active': '2024-09-10T15:59:16.300932+0000', 'last_became_peered': '2024-09-10T15:59:16.300932+0000', 'last_unstale': '2024-09-10T16:07:30.423962+0000', 'last_undegraded': '2024-09-10T15:59:15.887823+0000', 'last_fullsized': '2024-09-10T15:59:15.768513+0000', 'mapping_epoch': 42, 'log_start': "43'9000", 'ondisk_log_start': "43'9000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3880", 'last_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'objects_scrubbed': 3827, 'log_size': 10094, 'log_dups_size': 0, 'ondisk_log_size': 10094, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1397, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 8618950, 'num_objects': 2106, 'num_object_clones': 0, 'num_object_copies': 4212, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4325, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2106, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15257, 'num_write_kb': 27072, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2217}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:07:34.443 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T16:07:35.145 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:07:35.146 INFO:teuthology.orchestra.run.smithi115.stdout:{"fsid":"7effee29-df05-4351-9741-ee203564ed81","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 39791/66986 objects degraded (59.402%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":619,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":47,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983954,"num_in_osds":3,"osd_in_since":1725983947,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":33493,"data_bytes":137138953,"bytes_used":1684762624,"bytes_avail":288225529856,"bytes_total":289910292480,"degraded_objects":39791,"degraded_total":66986,"degraded_ratio":0.59401964589615741,"recovering_objects_per_sec":23,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":282,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":387},"fsmap":{"epoch":1,"btime":"2024-09-10T15:57:14:967120+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T16:00:13.560776+0000","services":{}},"progress_events":{}} 2024-09-10T16:07:35.184 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-10T16:07:35.184 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:07:35.731 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:07:35.732 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:07:35.752 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":349,"stamp":"2024-09-10T16:07:33.741154+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":137138953,"num_objects":33493,"num_object_clones":0,"num_object_copies":66986,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39791,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33493,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":258617,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11600,"num_bytes_recovered":33095699,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160944,"ondisk_log_size":160944,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1645276,"kb_used_data":477092,"kb_used_omap":12,"kb_used_meta":1168115,"kb_avail":281470244,"statfs":{"total":289910292480,"available":288225529856,"internally_reserved":0,"allocated":488542208,"data_stored":487604783,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1196149762},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,1,1,1,15,8,11,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19046400,"num_objects":-4650,"num_object_clones":0,"num_object_copies":-9300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-282,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4650,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4650,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":282,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002712"},"pg_stats":[{"pgid":"2.f","version":"47'18616","reported_seq":18697,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:30.437828+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:07:30.437828+0000","last_peered":"2024-09-10T16:07:30.437828+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:07:30.437828+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"43'8600","ondisk_log_start":"43'8600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8421376,"num_objects":2056,"num_object_clones":0,"num_object_copies":4112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2056,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18616,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2203}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'18768","reported_seq":18887,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:07:30.842095+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:07:30.842095+0000","last_peered":"2024-09-10T16:07:30.842095+0000","last_clean":"2024-09-10T16:07:30.842095+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:07:30.842095+0000","last_undegraded":"2024-09-10T16:07:30.842095+0000","last_fullsized":"2024-09-10T16:07:30.842095+0000","mapping_epoch":35,"log_start":"43'8700","ondisk_log_start":"43'8700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8504262,"num_objects":2078,"num_object_clones":0,"num_object_copies":4156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2078,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16873,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'18662","reported_seq":27346,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:07:29.166034+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:07:29.166034+0000","last_peered":"2024-09-10T16:07:29.166034+0000","last_clean":"2024-09-10T16:07:29.166034+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:07:29.166034+0000","last_undegraded":"2024-09-10T16:07:29.166034+0000","last_fullsized":"2024-09-10T16:07:29.166034+0000","mapping_epoch":44,"log_start":"43'8600","ondisk_log_start":"43'8600","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8744984,"num_objects":2136,"num_object_clones":0,"num_object_copies":4272,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2136,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16666,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'18645","reported_seq":18732,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:07:30.783234+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:07:30.783234+0000","last_peered":"2024-09-10T16:07:30.783234+0000","last_clean":"2024-09-10T16:07:30.783234+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:07:30.783234+0000","last_undegraded":"2024-09-10T16:07:30.783234+0000","last_fullsized":"2024-09-10T16:07:30.783234+0000","mapping_epoch":35,"log_start":"43'8600","ondisk_log_start":"43'8600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8253440,"num_objects":2015,"num_object_clones":0,"num_object_copies":4030,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2015,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18645,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'18839","reported_seq":18975,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:07:30.497448+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:07:30.497448+0000","last_peered":"2024-09-10T16:07:30.497448+0000","last_clean":"2024-09-10T16:07:30.497448+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:07:30.497448+0000","last_undegraded":"2024-09-10T16:07:30.497448+0000","last_fullsized":"2024-09-10T16:07:30.497448+0000","mapping_epoch":34,"log_start":"43'8800","ondisk_log_start":"43'8800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8405475,"num_objects":2053,"num_object_clones":0,"num_object_copies":4106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2053,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15550,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'18958","reported_seq":19102,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:30.433007+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:07:30.433007+0000","last_peered":"2024-09-10T16:07:30.433007+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:07:30.433007+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"43'8900","ondisk_log_start":"43'8900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8614371,"num_objects":2104,"num_object_clones":0,"num_object_copies":4208,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2104,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15570,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2267},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'18962","reported_seq":19106,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:30.382656+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:07:30.382656+0000","last_peered":"2024-09-10T16:07:30.382656+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:07:30.382656+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"43'8900","ondisk_log_start":"43'8900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8684486,"num_objects":2122,"num_object_clones":0,"num_object_copies":4244,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2122,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":15461,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2115},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'19174","reported_seq":19309,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:07:30.478848+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:07:30.478848+0000","last_peered":"2024-09-10T16:07:30.478848+0000","last_clean":"2024-09-10T16:07:30.478848+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:07:30.478848+0000","last_undegraded":"2024-09-10T16:07:30.478848+0000","last_fullsized":"2024-09-10T16:07:30.478848+0000","mapping_epoch":34,"log_start":"43'9100","ondisk_log_start":"43'9100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8765440,"num_objects":2140,"num_object_clones":0,"num_object_copies":4280,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2140,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16021,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.876943+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:05:19.876943+0000","last_peered":"2024-09-10T16:05:19.876943+0000","last_clean":"2024-09-10T16:05:19.876943+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:05:19.876943+0000","last_undegraded":"2024-09-10T16:05:19.876943+0000","last_fullsized":"2024-09-10T16:05:19.876943+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'18277","reported_seq":18423,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:30.834178+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:07:30.834178+0000","last_peered":"2024-09-10T16:07:30.834178+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:07:30.834178+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"43'8200","ondisk_log_start":"43'8200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8929763,"num_objects":2181,"num_object_clones":0,"num_object_copies":4362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2181,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16096,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-1942}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":100,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.875849+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:05:19.875849+0000","last_peered":"2024-09-10T16:05:19.875849+0000","last_clean":"2024-09-10T16:05:19.875849+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:05:19.875849+0000","last_undegraded":"2024-09-10T16:05:19.875849+0000","last_fullsized":"2024-09-10T16:05:19.875849+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'18806","reported_seq":18943,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:30.434065+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:07:30.434065+0000","last_peered":"2024-09-10T16:07:30.434065+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:07:30.434065+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"43'8800","ondisk_log_start":"43'8800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8617984,"num_objects":2104,"num_object_clones":0,"num_object_copies":4208,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2104,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15310,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2151}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":110,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.872023+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:05:19.872023+0000","last_peered":"2024-09-10T16:05:19.872023+0000","last_clean":"2024-09-10T16:05:19.872023+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:05:19.872023+0000","last_undegraded":"2024-09-10T16:05:19.872023+0000","last_fullsized":"2024-09-10T16:05:19.872023+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'18734","reported_seq":18875,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:30.864306+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:07:30.864306+0000","last_peered":"2024-09-10T16:07:30.864306+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:07:30.864306+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"43'8700","ondisk_log_start":"43'8700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8384995,"num_objects":2048,"num_object_clones":0,"num_object_copies":4096,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2048,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16414,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-2256}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877148+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:05:19.877148+0000","last_peered":"2024-09-10T16:05:19.877148+0000","last_clean":"2024-09-10T16:05:19.877148+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:05:19.877148+0000","last_undegraded":"2024-09-10T16:05:19.877148+0000","last_fullsized":"2024-09-10T16:05:19.877148+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'19076","reported_seq":25452,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:30.870587+0000","last_change":"2024-09-10T16:05:18.271152+0000","last_active":"2024-09-10T16:07:30.870587+0000","last_peered":"2024-09-10T16:07:30.870587+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:07:30.870587+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"43'9000","ondisk_log_start":"43'9000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8577024,"num_objects":2094,"num_object_clones":0,"num_object_copies":4188,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1234,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2094,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16630,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3120,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":860}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'18760","reported_seq":18901,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:30.389592+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:07:30.389592+0000","last_peered":"2024-09-10T16:07:30.389592+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:07:30.389592+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"43'8700","ondisk_log_start":"43'8700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8511488,"num_objects":2078,"num_object_clones":0,"num_object_copies":4156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2078,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15391,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2226}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877654+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:05:19.877654+0000","last_peered":"2024-09-10T16:05:19.877654+0000","last_clean":"2024-09-10T16:05:19.877654+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:05:19.877654+0000","last_undegraded":"2024-09-10T16:05:19.877654+0000","last_fullsized":"2024-09-10T16:05:19.877654+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'18470","reported_seq":27041,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:07:29.242778+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:07:29.242778+0000","last_peered":"2024-09-10T16:07:29.242778+0000","last_clean":"2024-09-10T16:07:29.242778+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:07:29.242778+0000","last_undegraded":"2024-09-10T16:07:29.242778+0000","last_fullsized":"2024-09-10T16:07:29.242778+0000","mapping_epoch":46,"log_start":"43'8400","ondisk_log_start":"43'8400","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8716288,"num_objects":2128,"num_object_clones":0,"num_object_copies":4256,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2128,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14860,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":100,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.876043+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:05:19.876043+0000","last_peered":"2024-09-10T16:05:19.876043+0000","last_clean":"2024-09-10T16:05:19.876043+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:05:19.876043+0000","last_undegraded":"2024-09-10T16:05:19.876043+0000","last_fullsized":"2024-09-10T16:05:19.876043+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'19000","reported_seq":19148,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:30.460688+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:07:30.460688+0000","last_peered":"2024-09-10T16:07:30.460688+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:07:30.460688+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"43'8900","ondisk_log_start":"43'8900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":2048,"num_object_clones":0,"num_object_copies":4096,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2048,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15255,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2323}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877349+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:05:19.877349+0000","last_peered":"2024-09-10T16:05:19.877349+0000","last_clean":"2024-09-10T16:05:19.877349+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:05:19.877349+0000","last_undegraded":"2024-09-10T16:05:19.877349+0000","last_fullsized":"2024-09-10T16:05:19.877349+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877269+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:05:19.877269+0000","last_peered":"2024-09-10T16:05:19.877269+0000","last_clean":"2024-09-10T16:05:19.877269+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:05:19.877269+0000","last_undegraded":"2024-09-10T16:05:19.877269+0000","last_fullsized":"2024-09-10T16:05:19.877269+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'19094","reported_seq":19243,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:30.423962+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:07:30.423962+0000","last_peered":"2024-09-10T16:07:30.423962+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:07:30.423962+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"43'9000","ondisk_log_start":"43'9000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8618950,"num_objects":2106,"num_object_clones":0,"num_object_copies":4212,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2106,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15257,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2217},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":137138934,"num_objects":33491,"num_object_clones":0,"num_object_copies":66982,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39791,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33491,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":258615,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11598,"num_bytes_recovered":33095680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":486002688,"data_stored":485860863,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160941,"ondisk_log_size":160941,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986290,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":543412,"kb_used_data":204896,"kb_used_omap":4,"kb_used_meta":338491,"kb_avail":93828428,"statfs":{"total":96636764160,"available":96080310272,"internally_reserved":0,"allocated":209813504,"data_stored":209510820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":346615466},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,5,4,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659235,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":482168,"kb_used_data":103396,"kb_used_omap":4,"kb_used_meta":378747,"kb_avail":93889672,"statfs":{"total":96636764160,"available":96143024128,"internally_reserved":0,"allocated":105877504,"data_stored":105545452,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":387837614},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,1,1,1,3,2,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182518,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":619696,"kb_used_data":168800,"kb_used_omap":4,"kb_used_meta":450875,"kb_avail":93752144,"statfs":{"total":96636764160,"available":96002195456,"internally_reserved":0,"allocated":172851200,"data_stored":172548511,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":461696682},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,7,2,3,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":172220416,"data_stored":172184286,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":104910848,"data_stored":104845355,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":208871424,"data_stored":208831222,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:07:38.754 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:07:39.327 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:07:39.328 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:07:39.350 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":351,"stamp":"2024-09-10T16:07:37.742607+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":127582985,"num_objects":31160,"num_object_clones":0,"num_object_copies":62320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39658,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31160,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":260950,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11733,"num_bytes_recovered":33095699,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160677,"ondisk_log_size":160677,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1644684,"kb_used_data":458448,"kb_used_omap":12,"kb_used_meta":1186163,"kb_avail":281470836,"statfs":{"total":289910292480,"available":288226136064,"internally_reserved":0,"allocated":469450752,"data_stored":468513327,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1214630914},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,2,3,6,7,4,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-21950464,"num_objects":-5359,"num_object_clones":0,"num_object_copies":-10718,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-415,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5359,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5359,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":415,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003290"},"pg_stats":[{"pgid":"2.f","version":"47'18752","reported_seq":18833,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:35.366669+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:07:35.366669+0000","last_peered":"2024-09-10T16:07:35.366669+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:07:35.366669+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"43'8700","ondisk_log_start":"43'8700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7864320,"num_objects":1920,"num_object_clones":0,"num_object_copies":3840,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1920,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18752,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2339}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'18923","reported_seq":19042,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:07:35.869334+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:07:35.869334+0000","last_peered":"2024-09-10T16:07:35.869334+0000","last_clean":"2024-09-10T16:07:35.869334+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:07:35.869334+0000","last_undegraded":"2024-09-10T16:07:35.869334+0000","last_fullsized":"2024-09-10T16:07:35.869334+0000","mapping_epoch":35,"log_start":"43'8900","ondisk_log_start":"43'8900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7869382,"num_objects":1923,"num_object_clones":0,"num_object_copies":3846,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1923,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17028,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'18818","reported_seq":27502,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:07:34.303428+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:07:34.303428+0000","last_peered":"2024-09-10T16:07:34.303428+0000","last_clean":"2024-09-10T16:07:34.303428+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:07:34.303428+0000","last_undegraded":"2024-09-10T16:07:34.303428+0000","last_fullsized":"2024-09-10T16:07:34.303428+0000","mapping_epoch":44,"log_start":"43'8800","ondisk_log_start":"43'8800","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8106008,"num_objects":1980,"num_object_clones":0,"num_object_copies":3960,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1980,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16822,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'18779","reported_seq":18866,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:07:35.629954+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:07:35.629954+0000","last_peered":"2024-09-10T16:07:35.629954+0000","last_clean":"2024-09-10T16:07:35.629954+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:07:35.629954+0000","last_undegraded":"2024-09-10T16:07:35.629954+0000","last_fullsized":"2024-09-10T16:07:35.629954+0000","mapping_epoch":35,"log_start":"43'8700","ondisk_log_start":"43'8700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7704576,"num_objects":1881,"num_object_clones":0,"num_object_copies":3762,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1881,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18779,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'18973","reported_seq":19109,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:07:35.533392+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:07:35.533392+0000","last_peered":"2024-09-10T16:07:35.533392+0000","last_clean":"2024-09-10T16:07:35.533392+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:07:35.533392+0000","last_undegraded":"2024-09-10T16:07:35.533392+0000","last_fullsized":"2024-09-10T16:07:35.533392+0000","mapping_epoch":34,"log_start":"43'8900","ondisk_log_start":"43'8900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7856611,"num_objects":1919,"num_object_clones":0,"num_object_copies":3838,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1919,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15684,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'19081","reported_seq":19225,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:35.413817+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:07:35.413817+0000","last_peered":"2024-09-10T16:07:35.413817+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:07:35.413817+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"43'9000","ondisk_log_start":"43'9000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8110563,"num_objects":1981,"num_object_clones":0,"num_object_copies":3962,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1981,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15693,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2390},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'19110","reported_seq":19253,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:35.490527+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:07:35.490527+0000","last_peered":"2024-09-10T16:07:35.490527+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:07:35.490527+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"43'9100","ondisk_log_start":"43'9100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8078278,"num_objects":1974,"num_object_clones":0,"num_object_copies":3948,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1974,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":15609,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2263},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'19324","reported_seq":19460,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:07:35.554988+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:07:35.554988+0000","last_peered":"2024-09-10T16:07:35.554988+0000","last_clean":"2024-09-10T16:07:35.554988+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:07:35.554988+0000","last_undegraded":"2024-09-10T16:07:35.554988+0000","last_fullsized":"2024-09-10T16:07:35.554988+0000","mapping_epoch":34,"log_start":"43'9300","ondisk_log_start":"43'9300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8151040,"num_objects":1990,"num_object_clones":0,"num_object_copies":3980,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1990,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16171,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.876943+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:05:19.876943+0000","last_peered":"2024-09-10T16:05:19.876943+0000","last_clean":"2024-09-10T16:05:19.876943+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:05:19.876943+0000","last_undegraded":"2024-09-10T16:05:19.876943+0000","last_fullsized":"2024-09-10T16:05:19.876943+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'18430","reported_seq":18576,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:35.782286+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:07:35.782286+0000","last_peered":"2024-09-10T16:07:35.782286+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:07:35.782286+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"43'8400","ondisk_log_start":"43'8400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8303075,"num_objects":2028,"num_object_clones":0,"num_object_copies":4056,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2028,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16249,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-2095}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":100,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.875849+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:05:19.875849+0000","last_peered":"2024-09-10T16:05:19.875849+0000","last_clean":"2024-09-10T16:05:19.875849+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:05:19.875849+0000","last_undegraded":"2024-09-10T16:05:19.875849+0000","last_fullsized":"2024-09-10T16:05:19.875849+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'18957","reported_seq":19094,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:35.559632+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:07:35.559632+0000","last_peered":"2024-09-10T16:07:35.559632+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:07:35.559632+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"43'8900","ondisk_log_start":"43'8900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7999488,"num_objects":1953,"num_object_clones":0,"num_object_copies":3906,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1953,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15461,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2302}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":110,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.872023+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:05:19.872023+0000","last_peered":"2024-09-10T16:05:19.872023+0000","last_clean":"2024-09-10T16:05:19.872023+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:05:19.872023+0000","last_undegraded":"2024-09-10T16:05:19.872023+0000","last_fullsized":"2024-09-10T16:05:19.872023+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'18859","reported_seq":19000,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:35.750728+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:07:35.750728+0000","last_peered":"2024-09-10T16:07:35.750728+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:07:35.750728+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"43'8800","ondisk_log_start":"43'8800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7872995,"num_objects":1923,"num_object_clones":0,"num_object_copies":3846,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1923,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16539,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-2381}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877148+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:05:19.877148+0000","last_peered":"2024-09-10T16:05:19.877148+0000","last_clean":"2024-09-10T16:05:19.877148+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:05:19.877148+0000","last_undegraded":"2024-09-10T16:05:19.877148+0000","last_fullsized":"2024-09-10T16:05:19.877148+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'19242","reported_seq":25884,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:35.843137+0000","last_change":"2024-09-10T16:05:18.271152+0000","last_active":"2024-09-10T16:07:35.843137+0000","last_peered":"2024-09-10T16:07:35.843137+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:07:35.843137+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"43'9200","ondisk_log_start":"43'9200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7897088,"num_objects":1928,"num_object_clones":0,"num_object_copies":3856,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1101,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1928,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16796,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3253,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":827}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'18907","reported_seq":19048,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:35.462531+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:07:35.462531+0000","last_peered":"2024-09-10T16:07:35.462531+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:07:35.462531+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"43'8900","ondisk_log_start":"43'8900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7909376,"num_objects":1931,"num_object_clones":0,"num_object_copies":3862,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1931,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15538,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2373}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877654+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:05:19.877654+0000","last_peered":"2024-09-10T16:05:19.877654+0000","last_clean":"2024-09-10T16:05:19.877654+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:05:19.877654+0000","last_undegraded":"2024-09-10T16:05:19.877654+0000","last_fullsized":"2024-09-10T16:05:19.877654+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'18627","reported_seq":27198,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:07:34.272427+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:07:34.272427+0000","last_peered":"2024-09-10T16:07:34.272427+0000","last_clean":"2024-09-10T16:07:34.272427+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:07:34.272427+0000","last_undegraded":"2024-09-10T16:07:34.272427+0000","last_fullsized":"2024-09-10T16:07:34.272427+0000","mapping_epoch":46,"log_start":"43'8600","ondisk_log_start":"43'8600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8073216,"num_objects":1971,"num_object_clones":0,"num_object_copies":3942,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1971,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15017,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":100,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.876043+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:05:19.876043+0000","last_peered":"2024-09-10T16:05:19.876043+0000","last_clean":"2024-09-10T16:05:19.876043+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:05:19.876043+0000","last_undegraded":"2024-09-10T16:05:19.876043+0000","last_fullsized":"2024-09-10T16:05:19.876043+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'19158","reported_seq":19306,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:35.507972+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:07:35.507972+0000","last_peered":"2024-09-10T16:07:35.507972+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:07:35.507972+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"43'9100","ondisk_log_start":"43'9100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7741440,"num_objects":1890,"num_object_clones":0,"num_object_copies":3780,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1890,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15413,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2481}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877349+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:05:19.877349+0000","last_peered":"2024-09-10T16:05:19.877349+0000","last_clean":"2024-09-10T16:05:19.877349+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:05:19.877349+0000","last_undegraded":"2024-09-10T16:05:19.877349+0000","last_fullsized":"2024-09-10T16:05:19.877349+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877269+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:05:19.877269+0000","last_peered":"2024-09-10T16:05:19.877269+0000","last_clean":"2024-09-10T16:05:19.877269+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:05:19.877269+0000","last_undegraded":"2024-09-10T16:05:19.877269+0000","last_fullsized":"2024-09-10T16:05:19.877269+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'19234","reported_seq":19384,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:35.521060+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:07:35.521060+0000","last_peered":"2024-09-10T16:07:35.521060+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:07:35.521060+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"43'9200","ondisk_log_start":"43'9200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8045510,"num_objects":1966,"num_object_clones":0,"num_object_copies":3932,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1966,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15397,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2357},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":127582966,"num_objects":31158,"num_object_clones":0,"num_object_copies":62316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39658,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31158,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":260948,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11731,"num_bytes_recovered":33095680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":466804736,"data_stored":466662911,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160674,"ondisk_log_size":160674,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986291,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":560768,"kb_used_data":200048,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93811072,"statfs":{"total":96636764160,"available":96062537728,"internally_reserved":0,"allocated":204849152,"data_stored":204546468,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369356458},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,0,1,0,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659236,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":466840,"kb_used_data":96388,"kb_used_omap":4,"kb_used_meta":370427,"kb_avail":93905000,"statfs":{"total":96636764160,"available":96158720000,"internally_reserved":0,"allocated":98701312,"data_stored":98369260,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":379317934},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,1,2,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182519,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":617076,"kb_used_data":162012,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93754764,"statfs":{"total":96636764160,"available":96004878336,"internally_reserved":0,"allocated":165900288,"data_stored":165597599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":465956522},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,0,1,4,5,2,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":165122048,"data_stored":165085918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":97579008,"data_stored":97513515,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":204103680,"data_stored":204063478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:07:39.351 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:07:39.900 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:07:39.903 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:07:39.927 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":352,"stamp":"2024-09-10T16:07:39.743134+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":126313225,"num_objects":30850,"num_object_clones":0,"num_object_copies":61700,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39658,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30850,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":261260,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11733,"num_bytes_recovered":33095699,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160787,"ondisk_log_size":160787,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1641824,"kb_used_data":451420,"kb_used_omap":12,"kb_used_meta":1190323,"kb_avail":281473696,"statfs":{"total":289910292480,"available":288229064704,"internally_reserved":0,"allocated":462254080,"data_stored":461316655,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1218890754},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[5,0,4,1,8,9,4,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-20344832,"num_objects":-4967,"num_object_clones":0,"num_object_copies":-9934,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-274,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4967,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4967,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":274,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003335"},"pg_stats":[{"pgid":"2.f","version":"47'18752","reported_seq":18833,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:35.366669+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:07:35.366669+0000","last_peered":"2024-09-10T16:07:35.366669+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:07:35.366669+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"43'8700","ondisk_log_start":"43'8700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7864320,"num_objects":1920,"num_object_clones":0,"num_object_copies":3840,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1920,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18752,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2339}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'18923","reported_seq":19042,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:07:35.869334+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:07:35.869334+0000","last_peered":"2024-09-10T16:07:35.869334+0000","last_clean":"2024-09-10T16:07:35.869334+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:07:35.869334+0000","last_undegraded":"2024-09-10T16:07:35.869334+0000","last_fullsized":"2024-09-10T16:07:35.869334+0000","mapping_epoch":35,"log_start":"43'8900","ondisk_log_start":"43'8900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7869382,"num_objects":1923,"num_object_clones":0,"num_object_copies":3846,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1923,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17028,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'18982","reported_seq":27666,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:07:39.162800+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:07:39.162800+0000","last_peered":"2024-09-10T16:07:39.162800+0000","last_clean":"2024-09-10T16:07:39.162800+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:07:39.162800+0000","last_undegraded":"2024-09-10T16:07:39.162800+0000","last_fullsized":"2024-09-10T16:07:39.162800+0000","mapping_epoch":44,"log_start":"43'8900","ondisk_log_start":"43'8900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7434264,"num_objects":1816,"num_object_clones":0,"num_object_copies":3632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1816,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16986,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'18779","reported_seq":18866,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:07:35.629954+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:07:35.629954+0000","last_peered":"2024-09-10T16:07:35.629954+0000","last_clean":"2024-09-10T16:07:35.629954+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:07:35.629954+0000","last_undegraded":"2024-09-10T16:07:35.629954+0000","last_fullsized":"2024-09-10T16:07:35.629954+0000","mapping_epoch":35,"log_start":"43'8700","ondisk_log_start":"43'8700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7704576,"num_objects":1881,"num_object_clones":0,"num_object_copies":3762,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1881,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18779,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'18973","reported_seq":19109,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:07:35.533392+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:07:35.533392+0000","last_peered":"2024-09-10T16:07:35.533392+0000","last_clean":"2024-09-10T16:07:35.533392+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:07:35.533392+0000","last_undegraded":"2024-09-10T16:07:35.533392+0000","last_fullsized":"2024-09-10T16:07:35.533392+0000","mapping_epoch":34,"log_start":"43'8900","ondisk_log_start":"43'8900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7856611,"num_objects":1919,"num_object_clones":0,"num_object_copies":3838,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1919,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15684,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'19081","reported_seq":19225,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:35.413817+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:07:35.413817+0000","last_peered":"2024-09-10T16:07:35.413817+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:07:35.413817+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"43'9000","ondisk_log_start":"43'9000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8110563,"num_objects":1981,"num_object_clones":0,"num_object_copies":3962,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1981,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15693,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2390},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'19110","reported_seq":19253,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:35.490527+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:07:35.490527+0000","last_peered":"2024-09-10T16:07:35.490527+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:07:35.490527+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"43'9100","ondisk_log_start":"43'9100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8078278,"num_objects":1974,"num_object_clones":0,"num_object_copies":3948,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1974,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":15609,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2263},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'19324","reported_seq":19460,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:07:35.554988+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:07:35.554988+0000","last_peered":"2024-09-10T16:07:35.554988+0000","last_clean":"2024-09-10T16:07:35.554988+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:07:35.554988+0000","last_undegraded":"2024-09-10T16:07:35.554988+0000","last_fullsized":"2024-09-10T16:07:35.554988+0000","mapping_epoch":34,"log_start":"43'9300","ondisk_log_start":"43'9300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8151040,"num_objects":1990,"num_object_clones":0,"num_object_copies":3980,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1990,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16171,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.876943+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:05:19.876943+0000","last_peered":"2024-09-10T16:05:19.876943+0000","last_clean":"2024-09-10T16:05:19.876943+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:05:19.876943+0000","last_undegraded":"2024-09-10T16:05:19.876943+0000","last_fullsized":"2024-09-10T16:05:19.876943+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'18430","reported_seq":18576,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:35.782286+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:07:35.782286+0000","last_peered":"2024-09-10T16:07:35.782286+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:07:35.782286+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"43'8400","ondisk_log_start":"43'8400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8303075,"num_objects":2028,"num_object_clones":0,"num_object_copies":4056,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2028,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16249,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-2095}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":100,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.875849+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:05:19.875849+0000","last_peered":"2024-09-10T16:05:19.875849+0000","last_clean":"2024-09-10T16:05:19.875849+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:05:19.875849+0000","last_undegraded":"2024-09-10T16:05:19.875849+0000","last_fullsized":"2024-09-10T16:05:19.875849+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'18957","reported_seq":19094,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:35.559632+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:07:35.559632+0000","last_peered":"2024-09-10T16:07:35.559632+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:07:35.559632+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"43'8900","ondisk_log_start":"43'8900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7999488,"num_objects":1953,"num_object_clones":0,"num_object_copies":3906,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1953,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15461,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2302}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":110,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.872023+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:05:19.872023+0000","last_peered":"2024-09-10T16:05:19.872023+0000","last_clean":"2024-09-10T16:05:19.872023+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:05:19.872023+0000","last_undegraded":"2024-09-10T16:05:19.872023+0000","last_fullsized":"2024-09-10T16:05:19.872023+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'18859","reported_seq":19000,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:35.750728+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:07:35.750728+0000","last_peered":"2024-09-10T16:07:35.750728+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:07:35.750728+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"43'8800","ondisk_log_start":"43'8800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7872995,"num_objects":1923,"num_object_clones":0,"num_object_copies":3846,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1923,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16539,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-2381}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":86,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:32.574539+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:05:32.574539+0000","last_peered":"2024-09-10T16:05:32.574539+0000","last_clean":"2024-09-10T16:05:32.574539+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:05:32.574539+0000","last_undegraded":"2024-09-10T16:05:32.574539+0000","last_fullsized":"2024-09-10T16:05:32.574539+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'19242","reported_seq":25884,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:35.843137+0000","last_change":"2024-09-10T16:05:18.271152+0000","last_active":"2024-09-10T16:07:35.843137+0000","last_peered":"2024-09-10T16:07:35.843137+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:07:35.843137+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"43'9200","ondisk_log_start":"43'9200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7897088,"num_objects":1928,"num_object_clones":0,"num_object_copies":3856,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1101,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1928,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16796,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3253,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":827}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'18907","reported_seq":19048,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:35.462531+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:07:35.462531+0000","last_peered":"2024-09-10T16:07:35.462531+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:07:35.462531+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"43'8900","ondisk_log_start":"43'8900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7909376,"num_objects":1931,"num_object_clones":0,"num_object_copies":3862,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1931,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15538,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2373}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:32.574559+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:05:32.574559+0000","last_peered":"2024-09-10T16:05:32.574559+0000","last_clean":"2024-09-10T16:05:32.574559+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:05:32.574559+0000","last_undegraded":"2024-09-10T16:05:32.574559+0000","last_fullsized":"2024-09-10T16:05:32.574559+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'18773","reported_seq":27345,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:07:39.253091+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:07:39.253091+0000","last_peered":"2024-09-10T16:07:39.253091+0000","last_clean":"2024-09-10T16:07:39.253091+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:07:39.253091+0000","last_undegraded":"2024-09-10T16:07:39.253091+0000","last_fullsized":"2024-09-10T16:07:39.253091+0000","mapping_epoch":46,"log_start":"43'8700","ondisk_log_start":"43'8700","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7475200,"num_objects":1825,"num_object_clones":0,"num_object_copies":3650,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1825,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15163,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":100,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.876043+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:05:19.876043+0000","last_peered":"2024-09-10T16:05:19.876043+0000","last_clean":"2024-09-10T16:05:19.876043+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:05:19.876043+0000","last_undegraded":"2024-09-10T16:05:19.876043+0000","last_fullsized":"2024-09-10T16:05:19.876043+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'19158","reported_seq":19306,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:35.507972+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:07:35.507972+0000","last_peered":"2024-09-10T16:07:35.507972+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:07:35.507972+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"43'9100","ondisk_log_start":"43'9100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7741440,"num_objects":1890,"num_object_clones":0,"num_object_copies":3780,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1890,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15413,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2481}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:32.574572+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:05:32.574572+0000","last_peered":"2024-09-10T16:05:32.574572+0000","last_clean":"2024-09-10T16:05:32.574572+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:05:32.574572+0000","last_undegraded":"2024-09-10T16:05:32.574572+0000","last_fullsized":"2024-09-10T16:05:32.574572+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877269+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:05:19.877269+0000","last_peered":"2024-09-10T16:05:19.877269+0000","last_clean":"2024-09-10T16:05:19.877269+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:05:19.877269+0000","last_undegraded":"2024-09-10T16:05:19.877269+0000","last_fullsized":"2024-09-10T16:05:19.877269+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'19234","reported_seq":19384,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:35.521060+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:07:35.521060+0000","last_peered":"2024-09-10T16:07:35.521060+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:07:35.521060+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"43'9200","ondisk_log_start":"43'9200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8045510,"num_objects":1966,"num_object_clones":0,"num_object_copies":3932,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1966,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15397,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2357},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":126313206,"num_objects":30848,"num_object_clones":0,"num_object_copies":61696,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39658,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30848,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":261258,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11731,"num_bytes_recovered":33095680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":459493376,"data_stored":459351551,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160784,"ondisk_log_size":160784,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986291,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":560768,"kb_used_data":200048,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93811072,"statfs":{"total":96636764160,"available":96062537728,"internally_reserved":0,"allocated":204849152,"data_stored":204546468,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369356458},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,0,1,0,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659237,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":463980,"kb_used_data":89360,"kb_used_omap":4,"kb_used_meta":374587,"kb_avail":93907860,"statfs":{"total":96636764160,"available":96161648640,"internally_reserved":0,"allocated":91504640,"data_stored":91172588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":383577774},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,2,0,3,4,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182519,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":617076,"kb_used_data":162012,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93754764,"statfs":{"total":96636764160,"available":96004878336,"internally_reserved":0,"allocated":165900288,"data_stored":165597599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":465956522},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,0,1,4,5,2,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":165122048,"data_stored":165085918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90267648,"data_stored":90202155,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":204103680,"data_stored":204063478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:07:39.929 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-10T16:07:39.929 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "47'18752", 'reported_seq': 18833, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:07:35.366669+0000', 'last_change': '2024-09-10T15:59:16.281558+0000', 'last_active': '2024-09-10T16:07:35.366669+0000', 'last_peered': '2024-09-10T16:07:35.366669+0000', 'last_clean': '2024-09-10T15:59:14.132456+0000', 'last_became_active': '2024-09-10T15:59:16.274407+0000', 'last_became_peered': '2024-09-10T15:59:16.274407+0000', 'last_unstale': '2024-09-10T16:07:35.366669+0000', 'last_undegraded': '2024-09-10T15:59:15.851279+0000', 'last_fullsized': '2024-09-10T15:59:15.774356+0000', 'mapping_epoch': 42, 'log_start': "43'8700", 'ondisk_log_start': "43'8700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'objects_scrubbed': 0, 'log_size': 10052, 'log_dups_size': 0, 'ondisk_log_size': 10052, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 7864320, 'num_objects': 1920, 'num_object_clones': 0, 'num_object_copies': 3840, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4259, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1920, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18752, 'num_write_kb': 41344, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2339}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:07:39.929 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-10T16:07:39.929 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "47'19081", 'reported_seq': 19225, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:07:35.413817+0000', 'last_change': '2024-09-10T15:59:16.308264+0000', 'last_active': '2024-09-10T16:07:35.413817+0000', 'last_peered': '2024-09-10T16:07:35.413817+0000', 'last_clean': '2024-09-10T15:59:14.111551+0000', 'last_became_active': '2024-09-10T15:59:16.300040+0000', 'last_became_peered': '2024-09-10T15:59:16.300040+0000', 'last_unstale': '2024-09-10T16:07:35.413817+0000', 'last_undegraded': '2024-09-10T15:59:15.878234+0000', 'last_fullsized': '2024-09-10T15:59:15.769328+0000', 'mapping_epoch': 42, 'log_start': "43'9000", 'ondisk_log_start': "43'9000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3442", 'last_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'objects_scrubbed': 3384, 'log_size': 10081, 'log_dups_size': 0, 'ondisk_log_size': 10081, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1292, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 8110563, 'num_objects': 1981, 'num_object_clones': 0, 'num_object_copies': 3962, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4372, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1981, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 15693, 'num_write_kb': 28580, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2390}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:07:39.929 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-10T16:07:39.929 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "47'19110", 'reported_seq': 19253, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:07:35.490527+0000', 'last_change': '2024-09-10T15:59:16.331899+0000', 'last_active': '2024-09-10T16:07:35.490527+0000', 'last_peered': '2024-09-10T16:07:35.490527+0000', 'last_clean': '2024-09-10T15:59:14.139436+0000', 'last_became_active': '2024-09-10T15:59:16.318715+0000', 'last_became_peered': '2024-09-10T15:59:16.318715+0000', 'last_unstale': '2024-09-10T16:07:35.490527+0000', 'last_undegraded': '2024-09-10T15:59:15.877524+0000', 'last_fullsized': '2024-09-10T15:59:15.769042+0000', 'mapping_epoch': 42, 'log_start': "43'9100", 'ondisk_log_start': "43'9100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3537", 'last_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'objects_scrubbed': 3497, 'log_size': 10010, 'log_dups_size': 0, 'ondisk_log_size': 10010, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1224, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 8078278, 'num_objects': 1974, 'num_object_clones': 0, 'num_object_copies': 3948, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4237, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1974, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 15609, 'num_write_kb': 28172, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2263}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:07:39.929 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-10T16:07:39.929 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "47'18430", 'reported_seq': 18576, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:07:35.782286+0000', 'last_change': '2024-09-10T15:59:16.175680+0000', 'last_active': '2024-09-10T16:07:35.782286+0000', 'last_peered': '2024-09-10T16:07:35.782286+0000', 'last_clean': '2024-09-10T15:59:14.088187+0000', 'last_became_active': '2024-09-10T15:59:16.165810+0000', 'last_became_peered': '2024-09-10T15:59:16.165810+0000', 'last_unstale': '2024-09-10T16:07:35.782286+0000', 'last_undegraded': '2024-09-10T15:59:15.822098+0000', 'last_fullsized': '2024-09-10T15:59:15.769123+0000', 'mapping_epoch': 42, 'log_start': "43'8400", 'ondisk_log_start': "43'8400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2218", 'last_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'objects_scrubbed': 2175, 'log_size': 10030, 'log_dups_size': 0, 'ondisk_log_size': 10030, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 825, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 8303075, 'num_objects': 2028, 'num_object_clones': 0, 'num_object_copies': 4056, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4124, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2028, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 16249, 'num_write_kb': 32204, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': -2095}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:07:39.929 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-10T16:07:39.929 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "47'18957", 'reported_seq': 19094, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:07:35.559632+0000', 'last_change': '2024-09-10T15:59:16.277742+0000', 'last_active': '2024-09-10T16:07:35.559632+0000', 'last_peered': '2024-09-10T16:07:35.559632+0000', 'last_clean': '2024-09-10T15:59:14.129081+0000', 'last_became_active': '2024-09-10T15:59:16.269835+0000', 'last_became_peered': '2024-09-10T15:59:16.269835+0000', 'last_unstale': '2024-09-10T16:07:35.559632+0000', 'last_undegraded': '2024-09-10T15:59:15.860857+0000', 'last_fullsized': '2024-09-10T15:59:15.771410+0000', 'mapping_epoch': 42, 'log_start': "43'8900", 'ondisk_log_start': "43'8900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3557", 'last_scrub_stamp': '2024-09-10T15:59:02.493678+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:02.493678+0000', 'objects_scrubbed': 3488, 'log_size': 10057, 'log_dups_size': 0, 'ondisk_log_size': 10057, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1472, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 7999488, 'num_objects': 1953, 'num_object_clones': 0, 'num_object_copies': 3906, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4257, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1953, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15461, 'num_write_kb': 27852, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2302}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:07:39.929 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-10T16:07:39.930 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "47'18859", 'reported_seq': 19000, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:07:35.750728+0000', 'last_change': '2024-09-10T15:59:16.284311+0000', 'last_active': '2024-09-10T16:07:35.750728+0000', 'last_peered': '2024-09-10T16:07:35.750728+0000', 'last_clean': '2024-09-10T15:59:14.116041+0000', 'last_became_active': '2024-09-10T15:59:16.274543+0000', 'last_became_peered': '2024-09-10T15:59:16.274543+0000', 'last_unstale': '2024-09-10T16:07:35.750728+0000', 'last_undegraded': '2024-09-10T15:59:15.812484+0000', 'last_fullsized': '2024-09-10T15:59:15.770885+0000', 'mapping_epoch': 42, 'log_start': "43'8800", 'ondisk_log_start': "43'8800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2356", 'last_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'objects_scrubbed': 2316, 'log_size': 10059, 'log_dups_size': 0, 'ondisk_log_size': 10059, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 783, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 7872995, 'num_objects': 1923, 'num_object_clones': 0, 'num_object_copies': 3846, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1923, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 16539, 'num_write_kb': 32292, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': -2381}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:07:39.930 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-10T16:07:39.930 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "47'19242", 'reported_seq': 25884, 'reported_epoch': 47, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:07:35.843137+0000', 'last_change': '2024-09-10T16:05:18.271152+0000', 'last_active': '2024-09-10T16:07:35.843137+0000', 'last_peered': '2024-09-10T16:07:35.843137+0000', 'last_clean': '2024-09-10T15:59:14.110654+0000', 'last_became_active': '2024-09-10T15:59:16.164645+0000', 'last_became_peered': '2024-09-10T15:59:16.164645+0000', 'last_unstale': '2024-09-10T16:07:35.843137+0000', 'last_undegraded': '2024-09-10T15:59:15.803593+0000', 'last_fullsized': '2024-09-10T15:59:15.769083+0000', 'mapping_epoch': 42, 'log_start': "43'9200", 'ondisk_log_start': "43'9200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2486", 'last_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'objects_scrubbed': 2440, 'log_size': 10042, 'log_dups_size': 0, 'ondisk_log_size': 10042, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 792, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 7897088, 'num_objects': 1928, 'num_object_clones': 0, 'num_object_copies': 3856, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1101, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1928, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16796, 'num_write_kb': 32568, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3253, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 827}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:07:39.930 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-10T16:07:39.930 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "47'18907", 'reported_seq': 19048, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:07:35.462531+0000', 'last_change': '2024-09-10T15:59:16.315641+0000', 'last_active': '2024-09-10T16:07:35.462531+0000', 'last_peered': '2024-09-10T16:07:35.462531+0000', 'last_clean': '2024-09-10T15:59:14.117001+0000', 'last_became_active': '2024-09-10T15:59:16.302245+0000', 'last_became_peered': '2024-09-10T15:59:16.302245+0000', 'last_unstale': '2024-09-10T16:07:35.462531+0000', 'last_undegraded': '2024-09-10T15:59:15.845012+0000', 'last_fullsized': '2024-09-10T15:59:15.770337+0000', 'mapping_epoch': 42, 'log_start': "43'8900", 'ondisk_log_start': "43'8900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3404", 'last_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'objects_scrubbed': 3367, 'log_size': 10007, 'log_dups_size': 0, 'ondisk_log_size': 10007, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 983, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 7909376, 'num_objects': 1931, 'num_object_clones': 0, 'num_object_copies': 3862, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1931, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15538, 'num_write_kb': 28204, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2373}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:07:39.930 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-10T16:07:39.930 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "47'19158", 'reported_seq': 19306, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:07:35.507972+0000', 'last_change': '2024-09-10T15:59:16.311291+0000', 'last_active': '2024-09-10T16:07:35.507972+0000', 'last_peered': '2024-09-10T16:07:35.507972+0000', 'last_clean': '2024-09-10T15:59:14.116363+0000', 'last_became_active': '2024-09-10T15:59:16.301244+0000', 'last_became_peered': '2024-09-10T15:59:16.301244+0000', 'last_unstale': '2024-09-10T16:07:35.507972+0000', 'last_undegraded': '2024-09-10T15:59:15.883434+0000', 'last_fullsized': '2024-09-10T15:59:15.772756+0000', 'mapping_epoch': 42, 'log_start': "43'9100", 'ondisk_log_start': "43'9100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3789", 'last_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'objects_scrubbed': 3741, 'log_size': 10058, 'log_dups_size': 0, 'ondisk_log_size': 10058, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1440, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 7741440, 'num_objects': 1890, 'num_object_clones': 0, 'num_object_copies': 3780, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4373, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1890, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15413, 'num_write_kb': 27124, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2481}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:07:39.930 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-10T16:07:39.930 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "47'19234", 'reported_seq': 19384, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:07:35.521060+0000', 'last_change': '2024-09-10T15:59:16.313409+0000', 'last_active': '2024-09-10T16:07:35.521060+0000', 'last_peered': '2024-09-10T16:07:35.521060+0000', 'last_clean': '2024-09-10T15:59:14.129780+0000', 'last_became_active': '2024-09-10T15:59:16.300932+0000', 'last_became_peered': '2024-09-10T15:59:16.300932+0000', 'last_unstale': '2024-09-10T16:07:35.521060+0000', 'last_undegraded': '2024-09-10T15:59:15.887823+0000', 'last_fullsized': '2024-09-10T15:59:15.768513+0000', 'mapping_epoch': 42, 'log_start': "43'9200", 'ondisk_log_start': "43'9200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3880", 'last_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'objects_scrubbed': 3827, 'log_size': 10034, 'log_dups_size': 0, 'ondisk_log_size': 10034, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1397, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 8045510, 'num_objects': 1966, 'num_object_clones': 0, 'num_object_copies': 3932, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4325, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1966, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15397, 'num_write_kb': 27072, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2357}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:07:39.930 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T16:07:40.649 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:07:40.649 INFO:teuthology.orchestra.run.smithi115.stdout:{"fsid":"7effee29-df05-4351-9741-ee203564ed81","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 39658/61700 objects degraded (64.276%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":624,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":47,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983954,"num_in_osds":3,"osd_in_since":1725983947,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":30850,"data_bytes":126313225,"bytes_used":1681227776,"bytes_avail":288229064704,"bytes_total":289910292480,"degraded_objects":39658,"degraded_total":61700,"degraded_ratio":0.64275526742301448,"recovering_objects_per_sec":22,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":274,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":413},"fsmap":{"epoch":1,"btime":"2024-09-10T15:57:14:967120+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T16:00:13.560776+0000","services":{}},"progress_events":{}} 2024-09-10T16:07:40.675 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-10T16:07:40.676 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:07:41.229 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:07:41.231 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:07:41.253 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":352,"stamp":"2024-09-10T16:07:39.743134+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":126313225,"num_objects":30850,"num_object_clones":0,"num_object_copies":61700,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39658,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30850,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":261260,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11733,"num_bytes_recovered":33095699,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160787,"ondisk_log_size":160787,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1641824,"kb_used_data":451420,"kb_used_omap":12,"kb_used_meta":1190323,"kb_avail":281473696,"statfs":{"total":289910292480,"available":288229064704,"internally_reserved":0,"allocated":462254080,"data_stored":461316655,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1218890754},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[5,0,4,1,8,9,4,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-20344832,"num_objects":-4967,"num_object_clones":0,"num_object_copies":-9934,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-274,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4967,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4967,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":274,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003335"},"pg_stats":[{"pgid":"2.f","version":"47'18752","reported_seq":18833,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:35.366669+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:07:35.366669+0000","last_peered":"2024-09-10T16:07:35.366669+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:07:35.366669+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"43'8700","ondisk_log_start":"43'8700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7864320,"num_objects":1920,"num_object_clones":0,"num_object_copies":3840,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1920,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18752,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2339}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'18923","reported_seq":19042,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:07:35.869334+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:07:35.869334+0000","last_peered":"2024-09-10T16:07:35.869334+0000","last_clean":"2024-09-10T16:07:35.869334+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:07:35.869334+0000","last_undegraded":"2024-09-10T16:07:35.869334+0000","last_fullsized":"2024-09-10T16:07:35.869334+0000","mapping_epoch":35,"log_start":"43'8900","ondisk_log_start":"43'8900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7869382,"num_objects":1923,"num_object_clones":0,"num_object_copies":3846,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1923,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17028,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'18982","reported_seq":27666,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:07:39.162800+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:07:39.162800+0000","last_peered":"2024-09-10T16:07:39.162800+0000","last_clean":"2024-09-10T16:07:39.162800+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:07:39.162800+0000","last_undegraded":"2024-09-10T16:07:39.162800+0000","last_fullsized":"2024-09-10T16:07:39.162800+0000","mapping_epoch":44,"log_start":"43'8900","ondisk_log_start":"43'8900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7434264,"num_objects":1816,"num_object_clones":0,"num_object_copies":3632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1816,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16986,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'18779","reported_seq":18866,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:07:35.629954+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:07:35.629954+0000","last_peered":"2024-09-10T16:07:35.629954+0000","last_clean":"2024-09-10T16:07:35.629954+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:07:35.629954+0000","last_undegraded":"2024-09-10T16:07:35.629954+0000","last_fullsized":"2024-09-10T16:07:35.629954+0000","mapping_epoch":35,"log_start":"43'8700","ondisk_log_start":"43'8700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7704576,"num_objects":1881,"num_object_clones":0,"num_object_copies":3762,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1881,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18779,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'18973","reported_seq":19109,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:07:35.533392+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:07:35.533392+0000","last_peered":"2024-09-10T16:07:35.533392+0000","last_clean":"2024-09-10T16:07:35.533392+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:07:35.533392+0000","last_undegraded":"2024-09-10T16:07:35.533392+0000","last_fullsized":"2024-09-10T16:07:35.533392+0000","mapping_epoch":34,"log_start":"43'8900","ondisk_log_start":"43'8900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7856611,"num_objects":1919,"num_object_clones":0,"num_object_copies":3838,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1919,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15684,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'19081","reported_seq":19225,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:35.413817+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:07:35.413817+0000","last_peered":"2024-09-10T16:07:35.413817+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:07:35.413817+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"43'9000","ondisk_log_start":"43'9000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8110563,"num_objects":1981,"num_object_clones":0,"num_object_copies":3962,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1981,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15693,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2390},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'19110","reported_seq":19253,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:35.490527+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:07:35.490527+0000","last_peered":"2024-09-10T16:07:35.490527+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:07:35.490527+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"43'9100","ondisk_log_start":"43'9100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8078278,"num_objects":1974,"num_object_clones":0,"num_object_copies":3948,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1974,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":15609,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2263},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'19324","reported_seq":19460,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:07:35.554988+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:07:35.554988+0000","last_peered":"2024-09-10T16:07:35.554988+0000","last_clean":"2024-09-10T16:07:35.554988+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:07:35.554988+0000","last_undegraded":"2024-09-10T16:07:35.554988+0000","last_fullsized":"2024-09-10T16:07:35.554988+0000","mapping_epoch":34,"log_start":"43'9300","ondisk_log_start":"43'9300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8151040,"num_objects":1990,"num_object_clones":0,"num_object_copies":3980,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1990,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16171,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.876943+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:05:19.876943+0000","last_peered":"2024-09-10T16:05:19.876943+0000","last_clean":"2024-09-10T16:05:19.876943+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:05:19.876943+0000","last_undegraded":"2024-09-10T16:05:19.876943+0000","last_fullsized":"2024-09-10T16:05:19.876943+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'18430","reported_seq":18576,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:35.782286+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:07:35.782286+0000","last_peered":"2024-09-10T16:07:35.782286+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:07:35.782286+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"43'8400","ondisk_log_start":"43'8400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8303075,"num_objects":2028,"num_object_clones":0,"num_object_copies":4056,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2028,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16249,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-2095}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":100,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.875849+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:05:19.875849+0000","last_peered":"2024-09-10T16:05:19.875849+0000","last_clean":"2024-09-10T16:05:19.875849+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:05:19.875849+0000","last_undegraded":"2024-09-10T16:05:19.875849+0000","last_fullsized":"2024-09-10T16:05:19.875849+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'18957","reported_seq":19094,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:35.559632+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:07:35.559632+0000","last_peered":"2024-09-10T16:07:35.559632+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:07:35.559632+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"43'8900","ondisk_log_start":"43'8900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7999488,"num_objects":1953,"num_object_clones":0,"num_object_copies":3906,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1953,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15461,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2302}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":110,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.872023+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:05:19.872023+0000","last_peered":"2024-09-10T16:05:19.872023+0000","last_clean":"2024-09-10T16:05:19.872023+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:05:19.872023+0000","last_undegraded":"2024-09-10T16:05:19.872023+0000","last_fullsized":"2024-09-10T16:05:19.872023+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'18859","reported_seq":19000,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:35.750728+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:07:35.750728+0000","last_peered":"2024-09-10T16:07:35.750728+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:07:35.750728+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"43'8800","ondisk_log_start":"43'8800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7872995,"num_objects":1923,"num_object_clones":0,"num_object_copies":3846,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1923,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16539,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-2381}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":86,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:32.574539+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:05:32.574539+0000","last_peered":"2024-09-10T16:05:32.574539+0000","last_clean":"2024-09-10T16:05:32.574539+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:05:32.574539+0000","last_undegraded":"2024-09-10T16:05:32.574539+0000","last_fullsized":"2024-09-10T16:05:32.574539+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'19242","reported_seq":25884,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:35.843137+0000","last_change":"2024-09-10T16:05:18.271152+0000","last_active":"2024-09-10T16:07:35.843137+0000","last_peered":"2024-09-10T16:07:35.843137+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:07:35.843137+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"43'9200","ondisk_log_start":"43'9200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7897088,"num_objects":1928,"num_object_clones":0,"num_object_copies":3856,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1101,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1928,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16796,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3253,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":827}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'18907","reported_seq":19048,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:35.462531+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:07:35.462531+0000","last_peered":"2024-09-10T16:07:35.462531+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:07:35.462531+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"43'8900","ondisk_log_start":"43'8900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7909376,"num_objects":1931,"num_object_clones":0,"num_object_copies":3862,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1931,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15538,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2373}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:32.574559+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:05:32.574559+0000","last_peered":"2024-09-10T16:05:32.574559+0000","last_clean":"2024-09-10T16:05:32.574559+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:05:32.574559+0000","last_undegraded":"2024-09-10T16:05:32.574559+0000","last_fullsized":"2024-09-10T16:05:32.574559+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'18773","reported_seq":27345,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:07:39.253091+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:07:39.253091+0000","last_peered":"2024-09-10T16:07:39.253091+0000","last_clean":"2024-09-10T16:07:39.253091+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:07:39.253091+0000","last_undegraded":"2024-09-10T16:07:39.253091+0000","last_fullsized":"2024-09-10T16:07:39.253091+0000","mapping_epoch":46,"log_start":"43'8700","ondisk_log_start":"43'8700","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7475200,"num_objects":1825,"num_object_clones":0,"num_object_copies":3650,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1825,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15163,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":100,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.876043+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:05:19.876043+0000","last_peered":"2024-09-10T16:05:19.876043+0000","last_clean":"2024-09-10T16:05:19.876043+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:05:19.876043+0000","last_undegraded":"2024-09-10T16:05:19.876043+0000","last_fullsized":"2024-09-10T16:05:19.876043+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'19158","reported_seq":19306,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:35.507972+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:07:35.507972+0000","last_peered":"2024-09-10T16:07:35.507972+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:07:35.507972+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"43'9100","ondisk_log_start":"43'9100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7741440,"num_objects":1890,"num_object_clones":0,"num_object_copies":3780,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1890,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15413,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2481}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:32.574572+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:05:32.574572+0000","last_peered":"2024-09-10T16:05:32.574572+0000","last_clean":"2024-09-10T16:05:32.574572+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:05:32.574572+0000","last_undegraded":"2024-09-10T16:05:32.574572+0000","last_fullsized":"2024-09-10T16:05:32.574572+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877269+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:05:19.877269+0000","last_peered":"2024-09-10T16:05:19.877269+0000","last_clean":"2024-09-10T16:05:19.877269+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:05:19.877269+0000","last_undegraded":"2024-09-10T16:05:19.877269+0000","last_fullsized":"2024-09-10T16:05:19.877269+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'19234","reported_seq":19384,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:35.521060+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:07:35.521060+0000","last_peered":"2024-09-10T16:07:35.521060+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:07:35.521060+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"43'9200","ondisk_log_start":"43'9200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8045510,"num_objects":1966,"num_object_clones":0,"num_object_copies":3932,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1966,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15397,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2357},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":126313206,"num_objects":30848,"num_object_clones":0,"num_object_copies":61696,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39658,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30848,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":261258,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11731,"num_bytes_recovered":33095680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":459493376,"data_stored":459351551,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160784,"ondisk_log_size":160784,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986291,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":560768,"kb_used_data":200048,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93811072,"statfs":{"total":96636764160,"available":96062537728,"internally_reserved":0,"allocated":204849152,"data_stored":204546468,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369356458},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,0,1,0,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659237,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":463980,"kb_used_data":89360,"kb_used_omap":4,"kb_used_meta":374587,"kb_avail":93907860,"statfs":{"total":96636764160,"available":96161648640,"internally_reserved":0,"allocated":91504640,"data_stored":91172588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":383577774},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,2,0,3,4,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182519,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":617076,"kb_used_data":162012,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93754764,"statfs":{"total":96636764160,"available":96004878336,"internally_reserved":0,"allocated":165900288,"data_stored":165597599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":465956522},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,0,1,4,5,2,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":165122048,"data_stored":165085918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90267648,"data_stored":90202155,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":204103680,"data_stored":204063478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:07:44.255 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:07:44.872 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:07:44.874 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:07:44.900 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":354,"stamp":"2024-09-10T16:07:43.744312+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":117875465,"num_objects":28790,"num_object_clones":0,"num_object_copies":57580,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39513,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28790,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":263320,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11878,"num_bytes_recovered":33095699,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160847,"ondisk_log_size":160847,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1621688,"kb_used_data":439600,"kb_used_omap":12,"kb_used_meta":1182003,"kb_avail":281493832,"statfs":{"total":289910292480,"available":288249683968,"internally_reserved":0,"allocated":450150400,"data_stored":449212975,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1210371074},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,5,3,5,9,5,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19263488,"num_objects":-4703,"num_object_clones":0,"num_object_copies":-9406,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-278,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4703,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4703,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":278,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003629"},"pg_stats":[{"pgid":"2.f","version":"47'18910","reported_seq":18988,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:40.552835+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:07:40.552835+0000","last_peered":"2024-09-10T16:07:40.552835+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:07:40.552835+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"43'8900","ondisk_log_start":"43'8900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7217152,"num_objects":1762,"num_object_clones":0,"num_object_copies":3524,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1762,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18910,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2497}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'19065","reported_seq":19184,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:07:40.828722+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:07:40.828722+0000","last_peered":"2024-09-10T16:07:40.828722+0000","last_clean":"2024-09-10T16:07:40.828722+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:07:40.828722+0000","last_undegraded":"2024-09-10T16:07:40.828722+0000","last_fullsized":"2024-09-10T16:07:40.828722+0000","mapping_epoch":35,"log_start":"43'9000","ondisk_log_start":"43'9000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7287750,"num_objects":1781,"num_object_clones":0,"num_object_copies":3562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1781,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17170,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'18982","reported_seq":27666,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:07:39.162800+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:07:39.162800+0000","last_peered":"2024-09-10T16:07:39.162800+0000","last_clean":"2024-09-10T16:07:39.162800+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:07:39.162800+0000","last_undegraded":"2024-09-10T16:07:39.162800+0000","last_fullsized":"2024-09-10T16:07:39.162800+0000","mapping_epoch":44,"log_start":"43'8900","ondisk_log_start":"43'8900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7434264,"num_objects":1816,"num_object_clones":0,"num_object_copies":3632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1816,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16986,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'18908","reported_seq":18995,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:07:40.804889+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:07:40.804889+0000","last_peered":"2024-09-10T16:07:40.804889+0000","last_clean":"2024-09-10T16:07:40.804889+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:07:40.804889+0000","last_undegraded":"2024-09-10T16:07:40.804889+0000","last_fullsized":"2024-09-10T16:07:40.804889+0000","mapping_epoch":35,"log_start":"43'8900","ondisk_log_start":"43'8900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7176192,"num_objects":1752,"num_object_clones":0,"num_object_copies":3504,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1752,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18908,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'19125","reported_seq":19261,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:07:40.500710+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:07:40.500710+0000","last_peered":"2024-09-10T16:07:40.500710+0000","last_clean":"2024-09-10T16:07:40.500710+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:07:40.500710+0000","last_undegraded":"2024-09-10T16:07:40.500710+0000","last_fullsized":"2024-09-10T16:07:40.500710+0000","mapping_epoch":34,"log_start":"43'9100","ondisk_log_start":"43'9100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7234019,"num_objects":1767,"num_object_clones":0,"num_object_copies":3534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1767,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15836,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'19230","reported_seq":19374,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:40.440299+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:07:40.440299+0000","last_peered":"2024-09-10T16:07:40.440299+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:07:40.440299+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"43'9200","ondisk_log_start":"43'9200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7500259,"num_objects":1832,"num_object_clones":0,"num_object_copies":3664,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1832,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15842,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2539},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'19253","reported_seq":19396,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:40.566431+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:07:40.566431+0000","last_peered":"2024-09-10T16:07:40.566431+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:07:40.566431+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"43'9200","ondisk_log_start":"43'9200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7492550,"num_objects":1831,"num_object_clones":0,"num_object_copies":3662,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1831,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":15752,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2406},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'19496","reported_seq":19632,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:07:40.467925+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:07:40.467925+0000","last_peered":"2024-09-10T16:07:40.467925+0000","last_clean":"2024-09-10T16:07:40.467925+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:07:40.467925+0000","last_undegraded":"2024-09-10T16:07:40.467925+0000","last_fullsized":"2024-09-10T16:07:40.467925+0000","mapping_epoch":34,"log_start":"43'9400","ondisk_log_start":"43'9400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7446528,"num_objects":1818,"num_object_clones":0,"num_object_copies":3636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1818,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16343,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.876943+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:05:19.876943+0000","last_peered":"2024-09-10T16:05:19.876943+0000","last_clean":"2024-09-10T16:05:19.876943+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:05:19.876943+0000","last_undegraded":"2024-09-10T16:05:19.876943+0000","last_fullsized":"2024-09-10T16:05:19.876943+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'18578","reported_seq":18724,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:40.838178+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:07:40.838178+0000","last_peered":"2024-09-10T16:07:40.838178+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:07:40.838178+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"43'8500","ondisk_log_start":"43'8500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7696867,"num_objects":1880,"num_object_clones":0,"num_object_copies":3760,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1880,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16397,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-2243}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":100,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.875849+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:05:19.875849+0000","last_peered":"2024-09-10T16:05:19.875849+0000","last_clean":"2024-09-10T16:05:19.875849+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:05:19.875849+0000","last_undegraded":"2024-09-10T16:05:19.875849+0000","last_fullsized":"2024-09-10T16:05:19.875849+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'19100","reported_seq":19237,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:40.539100+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:07:40.539100+0000","last_peered":"2024-09-10T16:07:40.539100+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:07:40.539100+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"43'9000","ondisk_log_start":"43'9000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7413760,"num_objects":1810,"num_object_clones":0,"num_object_copies":3620,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1810,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15604,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2445}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":110,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.872023+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:05:19.872023+0000","last_peered":"2024-09-10T16:05:19.872023+0000","last_clean":"2024-09-10T16:05:19.872023+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:05:19.872023+0000","last_undegraded":"2024-09-10T16:05:19.872023+0000","last_fullsized":"2024-09-10T16:05:19.872023+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'19006","reported_seq":19147,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:40.870415+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:07:40.870415+0000","last_peered":"2024-09-10T16:07:40.870415+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:07:40.870415+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"43'9000","ondisk_log_start":"43'9000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7270883,"num_objects":1776,"num_object_clones":0,"num_object_copies":3552,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1776,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16686,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-2528}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":86,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:32.574539+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:05:32.574539+0000","last_peered":"2024-09-10T16:05:32.574539+0000","last_clean":"2024-09-10T16:05:32.574539+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:05:32.574539+0000","last_undegraded":"2024-09-10T16:05:32.574539+0000","last_fullsized":"2024-09-10T16:05:32.574539+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'19389","reported_seq":26321,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:40.884745+0000","last_change":"2024-09-10T16:05:18.271152+0000","last_active":"2024-09-10T16:07:40.884745+0000","last_peered":"2024-09-10T16:07:40.884745+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:07:40.884745+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"43'9300","ondisk_log_start":"43'9300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7294976,"num_objects":1781,"num_object_clones":0,"num_object_copies":3562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":956,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1781,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16943,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3398,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":825}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'19045","reported_seq":19186,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:40.489610+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:07:40.489610+0000","last_peered":"2024-09-10T16:07:40.489610+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:07:40.489610+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"43'9000","ondisk_log_start":"43'9000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7344128,"num_objects":1793,"num_object_clones":0,"num_object_copies":3586,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1793,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15676,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2511}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:32.574559+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:05:32.574559+0000","last_peered":"2024-09-10T16:05:32.574559+0000","last_clean":"2024-09-10T16:05:32.574559+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:05:32.574559+0000","last_undegraded":"2024-09-10T16:05:32.574559+0000","last_fullsized":"2024-09-10T16:05:32.574559+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'18773","reported_seq":27345,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:07:39.253091+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:07:39.253091+0000","last_peered":"2024-09-10T16:07:39.253091+0000","last_clean":"2024-09-10T16:07:39.253091+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:07:39.253091+0000","last_undegraded":"2024-09-10T16:07:39.253091+0000","last_fullsized":"2024-09-10T16:07:39.253091+0000","mapping_epoch":46,"log_start":"43'8700","ondisk_log_start":"43'8700","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7475200,"num_objects":1825,"num_object_clones":0,"num_object_copies":3650,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1825,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15163,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":100,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.876043+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:05:19.876043+0000","last_peered":"2024-09-10T16:05:19.876043+0000","last_clean":"2024-09-10T16:05:19.876043+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:05:19.876043+0000","last_undegraded":"2024-09-10T16:05:19.876043+0000","last_fullsized":"2024-09-10T16:05:19.876043+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'19305","reported_seq":19453,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:40.556026+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:07:40.556026+0000","last_peered":"2024-09-10T16:07:40.556026+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:07:40.556026+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"43'9300","ondisk_log_start":"43'9300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7139328,"num_objects":1743,"num_object_clones":0,"num_object_copies":3486,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1743,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15560,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2628}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:32.574572+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:05:32.574572+0000","last_peered":"2024-09-10T16:05:32.574572+0000","last_clean":"2024-09-10T16:05:32.574572+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:05:32.574572+0000","last_undegraded":"2024-09-10T16:05:32.574572+0000","last_fullsized":"2024-09-10T16:05:32.574572+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877269+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:05:19.877269+0000","last_peered":"2024-09-10T16:05:19.877269+0000","last_clean":"2024-09-10T16:05:19.877269+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:05:19.877269+0000","last_undegraded":"2024-09-10T16:05:19.877269+0000","last_fullsized":"2024-09-10T16:05:19.877269+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'19379","reported_seq":19528,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:40.471398+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:07:40.471398+0000","last_peered":"2024-09-10T16:07:40.471398+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:07:40.471398+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"43'9300","ondisk_log_start":"43'9300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7451590,"num_objects":1821,"num_object_clones":0,"num_object_copies":3642,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1821,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15542,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2502},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":117875446,"num_objects":28788,"num_object_clones":0,"num_object_copies":57576,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39513,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28788,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":263318,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11876,"num_bytes_recovered":33095680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":447410176,"data_stored":447268351,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160844,"ondisk_log_size":160844,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986292,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":560144,"kb_used_data":195260,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93811696,"statfs":{"total":96636764160,"available":96063176704,"internally_reserved":0,"allocated":199946240,"data_stored":199643556,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,1,1,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659237,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":463980,"kb_used_data":89360,"kb_used_omap":4,"kb_used_meta":374587,"kb_avail":93907860,"statfs":{"total":96636764160,"available":96161648640,"internally_reserved":0,"allocated":91504640,"data_stored":91172588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":383577774},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,2,0,3,4,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182520,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":597564,"kb_used_data":154980,"kb_used_omap":4,"kb_used_meta":442555,"kb_avail":93774276,"statfs":{"total":96636764160,"available":96024858624,"internally_reserved":0,"allocated":158699520,"data_stored":158396831,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":453177002},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,1,1,4,3,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":157941760,"data_stored":157905630,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90267648,"data_stored":90202155,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":199200768,"data_stored":199160566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:07:44.901 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:07:45.476 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:07:45.477 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:07:45.507 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":354,"stamp":"2024-09-10T16:07:43.744312+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":117875465,"num_objects":28790,"num_object_clones":0,"num_object_copies":57580,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39513,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28790,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":263320,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11878,"num_bytes_recovered":33095699,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160847,"ondisk_log_size":160847,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1621688,"kb_used_data":439600,"kb_used_omap":12,"kb_used_meta":1182003,"kb_avail":281493832,"statfs":{"total":289910292480,"available":288249683968,"internally_reserved":0,"allocated":450150400,"data_stored":449212975,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1210371074},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,5,3,5,9,5,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19263488,"num_objects":-4703,"num_object_clones":0,"num_object_copies":-9406,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-278,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4703,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4703,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":278,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003629"},"pg_stats":[{"pgid":"2.f","version":"47'18910","reported_seq":18988,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:40.552835+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:07:40.552835+0000","last_peered":"2024-09-10T16:07:40.552835+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:07:40.552835+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"43'8900","ondisk_log_start":"43'8900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7217152,"num_objects":1762,"num_object_clones":0,"num_object_copies":3524,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1762,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18910,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2497}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'19065","reported_seq":19184,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:07:40.828722+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:07:40.828722+0000","last_peered":"2024-09-10T16:07:40.828722+0000","last_clean":"2024-09-10T16:07:40.828722+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:07:40.828722+0000","last_undegraded":"2024-09-10T16:07:40.828722+0000","last_fullsized":"2024-09-10T16:07:40.828722+0000","mapping_epoch":35,"log_start":"43'9000","ondisk_log_start":"43'9000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7287750,"num_objects":1781,"num_object_clones":0,"num_object_copies":3562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1781,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17170,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'18982","reported_seq":27666,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:07:39.162800+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:07:39.162800+0000","last_peered":"2024-09-10T16:07:39.162800+0000","last_clean":"2024-09-10T16:07:39.162800+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:07:39.162800+0000","last_undegraded":"2024-09-10T16:07:39.162800+0000","last_fullsized":"2024-09-10T16:07:39.162800+0000","mapping_epoch":44,"log_start":"43'8900","ondisk_log_start":"43'8900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7434264,"num_objects":1816,"num_object_clones":0,"num_object_copies":3632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1816,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16986,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'18908","reported_seq":18995,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:07:40.804889+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:07:40.804889+0000","last_peered":"2024-09-10T16:07:40.804889+0000","last_clean":"2024-09-10T16:07:40.804889+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:07:40.804889+0000","last_undegraded":"2024-09-10T16:07:40.804889+0000","last_fullsized":"2024-09-10T16:07:40.804889+0000","mapping_epoch":35,"log_start":"43'8900","ondisk_log_start":"43'8900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7176192,"num_objects":1752,"num_object_clones":0,"num_object_copies":3504,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1752,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18908,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'19125","reported_seq":19261,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:07:40.500710+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:07:40.500710+0000","last_peered":"2024-09-10T16:07:40.500710+0000","last_clean":"2024-09-10T16:07:40.500710+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:07:40.500710+0000","last_undegraded":"2024-09-10T16:07:40.500710+0000","last_fullsized":"2024-09-10T16:07:40.500710+0000","mapping_epoch":34,"log_start":"43'9100","ondisk_log_start":"43'9100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7234019,"num_objects":1767,"num_object_clones":0,"num_object_copies":3534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1767,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15836,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'19230","reported_seq":19374,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:40.440299+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:07:40.440299+0000","last_peered":"2024-09-10T16:07:40.440299+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:07:40.440299+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"43'9200","ondisk_log_start":"43'9200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7500259,"num_objects":1832,"num_object_clones":0,"num_object_copies":3664,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1832,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15842,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2539},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'19253","reported_seq":19396,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:40.566431+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:07:40.566431+0000","last_peered":"2024-09-10T16:07:40.566431+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:07:40.566431+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"43'9200","ondisk_log_start":"43'9200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7492550,"num_objects":1831,"num_object_clones":0,"num_object_copies":3662,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1831,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":15752,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2406},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'19496","reported_seq":19632,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:07:40.467925+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:07:40.467925+0000","last_peered":"2024-09-10T16:07:40.467925+0000","last_clean":"2024-09-10T16:07:40.467925+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:07:40.467925+0000","last_undegraded":"2024-09-10T16:07:40.467925+0000","last_fullsized":"2024-09-10T16:07:40.467925+0000","mapping_epoch":34,"log_start":"43'9400","ondisk_log_start":"43'9400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7446528,"num_objects":1818,"num_object_clones":0,"num_object_copies":3636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1818,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16343,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.876943+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:05:19.876943+0000","last_peered":"2024-09-10T16:05:19.876943+0000","last_clean":"2024-09-10T16:05:19.876943+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:05:19.876943+0000","last_undegraded":"2024-09-10T16:05:19.876943+0000","last_fullsized":"2024-09-10T16:05:19.876943+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'18578","reported_seq":18724,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:40.838178+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:07:40.838178+0000","last_peered":"2024-09-10T16:07:40.838178+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:07:40.838178+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"43'8500","ondisk_log_start":"43'8500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7696867,"num_objects":1880,"num_object_clones":0,"num_object_copies":3760,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1880,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16397,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-2243}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":100,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.875849+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:05:19.875849+0000","last_peered":"2024-09-10T16:05:19.875849+0000","last_clean":"2024-09-10T16:05:19.875849+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:05:19.875849+0000","last_undegraded":"2024-09-10T16:05:19.875849+0000","last_fullsized":"2024-09-10T16:05:19.875849+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'19100","reported_seq":19237,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:40.539100+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:07:40.539100+0000","last_peered":"2024-09-10T16:07:40.539100+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:07:40.539100+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"43'9000","ondisk_log_start":"43'9000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7413760,"num_objects":1810,"num_object_clones":0,"num_object_copies":3620,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1810,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15604,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2445}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":110,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.872023+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:05:19.872023+0000","last_peered":"2024-09-10T16:05:19.872023+0000","last_clean":"2024-09-10T16:05:19.872023+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:05:19.872023+0000","last_undegraded":"2024-09-10T16:05:19.872023+0000","last_fullsized":"2024-09-10T16:05:19.872023+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'19006","reported_seq":19147,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:40.870415+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:07:40.870415+0000","last_peered":"2024-09-10T16:07:40.870415+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:07:40.870415+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"43'9000","ondisk_log_start":"43'9000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7270883,"num_objects":1776,"num_object_clones":0,"num_object_copies":3552,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1776,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16686,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-2528}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":86,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:32.574539+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:05:32.574539+0000","last_peered":"2024-09-10T16:05:32.574539+0000","last_clean":"2024-09-10T16:05:32.574539+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:05:32.574539+0000","last_undegraded":"2024-09-10T16:05:32.574539+0000","last_fullsized":"2024-09-10T16:05:32.574539+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'19389","reported_seq":26321,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:40.884745+0000","last_change":"2024-09-10T16:05:18.271152+0000","last_active":"2024-09-10T16:07:40.884745+0000","last_peered":"2024-09-10T16:07:40.884745+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:07:40.884745+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"43'9300","ondisk_log_start":"43'9300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7294976,"num_objects":1781,"num_object_clones":0,"num_object_copies":3562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":956,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1781,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16943,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3398,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":825}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'19045","reported_seq":19186,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:40.489610+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:07:40.489610+0000","last_peered":"2024-09-10T16:07:40.489610+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:07:40.489610+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"43'9000","ondisk_log_start":"43'9000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7344128,"num_objects":1793,"num_object_clones":0,"num_object_copies":3586,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1793,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15676,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2511}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:32.574559+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:05:32.574559+0000","last_peered":"2024-09-10T16:05:32.574559+0000","last_clean":"2024-09-10T16:05:32.574559+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:05:32.574559+0000","last_undegraded":"2024-09-10T16:05:32.574559+0000","last_fullsized":"2024-09-10T16:05:32.574559+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'18773","reported_seq":27345,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:07:39.253091+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:07:39.253091+0000","last_peered":"2024-09-10T16:07:39.253091+0000","last_clean":"2024-09-10T16:07:39.253091+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:07:39.253091+0000","last_undegraded":"2024-09-10T16:07:39.253091+0000","last_fullsized":"2024-09-10T16:07:39.253091+0000","mapping_epoch":46,"log_start":"43'8700","ondisk_log_start":"43'8700","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7475200,"num_objects":1825,"num_object_clones":0,"num_object_copies":3650,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1825,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15163,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":100,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.876043+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:05:19.876043+0000","last_peered":"2024-09-10T16:05:19.876043+0000","last_clean":"2024-09-10T16:05:19.876043+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:05:19.876043+0000","last_undegraded":"2024-09-10T16:05:19.876043+0000","last_fullsized":"2024-09-10T16:05:19.876043+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'19305","reported_seq":19453,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:40.556026+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:07:40.556026+0000","last_peered":"2024-09-10T16:07:40.556026+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:07:40.556026+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"43'9300","ondisk_log_start":"43'9300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7139328,"num_objects":1743,"num_object_clones":0,"num_object_copies":3486,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1743,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15560,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2628}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:32.574572+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:05:32.574572+0000","last_peered":"2024-09-10T16:05:32.574572+0000","last_clean":"2024-09-10T16:05:32.574572+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:05:32.574572+0000","last_undegraded":"2024-09-10T16:05:32.574572+0000","last_fullsized":"2024-09-10T16:05:32.574572+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877269+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:05:19.877269+0000","last_peered":"2024-09-10T16:05:19.877269+0000","last_clean":"2024-09-10T16:05:19.877269+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:05:19.877269+0000","last_undegraded":"2024-09-10T16:05:19.877269+0000","last_fullsized":"2024-09-10T16:05:19.877269+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'19379","reported_seq":19528,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:40.471398+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:07:40.471398+0000","last_peered":"2024-09-10T16:07:40.471398+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:07:40.471398+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"43'9300","ondisk_log_start":"43'9300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7451590,"num_objects":1821,"num_object_clones":0,"num_object_copies":3642,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1821,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15542,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2502},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":117875446,"num_objects":28788,"num_object_clones":0,"num_object_copies":57576,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39513,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28788,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":263318,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11876,"num_bytes_recovered":33095680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":447410176,"data_stored":447268351,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160844,"ondisk_log_size":160844,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986292,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":560144,"kb_used_data":195260,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93811696,"statfs":{"total":96636764160,"available":96063176704,"internally_reserved":0,"allocated":199946240,"data_stored":199643556,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,1,1,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659237,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":463980,"kb_used_data":89360,"kb_used_omap":4,"kb_used_meta":374587,"kb_avail":93907860,"statfs":{"total":96636764160,"available":96161648640,"internally_reserved":0,"allocated":91504640,"data_stored":91172588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":383577774},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,2,0,3,4,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182520,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":597564,"kb_used_data":154980,"kb_used_omap":4,"kb_used_meta":442555,"kb_avail":93774276,"statfs":{"total":96636764160,"available":96024858624,"internally_reserved":0,"allocated":158699520,"data_stored":158396831,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":453177002},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,1,1,4,3,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":157941760,"data_stored":157905630,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90267648,"data_stored":90202155,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":199200768,"data_stored":199160566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:07:45.508 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-10T16:07:45.509 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "47'18910", 'reported_seq': 18988, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:07:40.552835+0000', 'last_change': '2024-09-10T15:59:16.281558+0000', 'last_active': '2024-09-10T16:07:40.552835+0000', 'last_peered': '2024-09-10T16:07:40.552835+0000', 'last_clean': '2024-09-10T15:59:14.132456+0000', 'last_became_active': '2024-09-10T15:59:16.274407+0000', 'last_became_peered': '2024-09-10T15:59:16.274407+0000', 'last_unstale': '2024-09-10T16:07:40.552835+0000', 'last_undegraded': '2024-09-10T15:59:15.851279+0000', 'last_fullsized': '2024-09-10T15:59:15.774356+0000', 'mapping_epoch': 42, 'log_start': "43'8900", 'ondisk_log_start': "43'8900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'objects_scrubbed': 0, 'log_size': 10010, 'log_dups_size': 0, 'ondisk_log_size': 10010, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 7217152, 'num_objects': 1762, 'num_object_clones': 0, 'num_object_copies': 3524, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4259, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1762, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18910, 'num_write_kb': 41344, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2497}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:07:45.509 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-10T16:07:45.509 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "47'19230", 'reported_seq': 19374, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:07:40.440299+0000', 'last_change': '2024-09-10T15:59:16.308264+0000', 'last_active': '2024-09-10T16:07:40.440299+0000', 'last_peered': '2024-09-10T16:07:40.440299+0000', 'last_clean': '2024-09-10T15:59:14.111551+0000', 'last_became_active': '2024-09-10T15:59:16.300040+0000', 'last_became_peered': '2024-09-10T15:59:16.300040+0000', 'last_unstale': '2024-09-10T16:07:40.440299+0000', 'last_undegraded': '2024-09-10T15:59:15.878234+0000', 'last_fullsized': '2024-09-10T15:59:15.769328+0000', 'mapping_epoch': 42, 'log_start': "43'9200", 'ondisk_log_start': "43'9200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3442", 'last_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'objects_scrubbed': 3384, 'log_size': 10030, 'log_dups_size': 0, 'ondisk_log_size': 10030, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1292, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 7500259, 'num_objects': 1832, 'num_object_clones': 0, 'num_object_copies': 3664, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4372, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1832, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 15842, 'num_write_kb': 28580, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2539}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:07:45.509 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-10T16:07:45.509 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "47'19253", 'reported_seq': 19396, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:07:40.566431+0000', 'last_change': '2024-09-10T15:59:16.331899+0000', 'last_active': '2024-09-10T16:07:40.566431+0000', 'last_peered': '2024-09-10T16:07:40.566431+0000', 'last_clean': '2024-09-10T15:59:14.139436+0000', 'last_became_active': '2024-09-10T15:59:16.318715+0000', 'last_became_peered': '2024-09-10T15:59:16.318715+0000', 'last_unstale': '2024-09-10T16:07:40.566431+0000', 'last_undegraded': '2024-09-10T15:59:15.877524+0000', 'last_fullsized': '2024-09-10T15:59:15.769042+0000', 'mapping_epoch': 42, 'log_start': "43'9200", 'ondisk_log_start': "43'9200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3537", 'last_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'objects_scrubbed': 3497, 'log_size': 10053, 'log_dups_size': 0, 'ondisk_log_size': 10053, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1224, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 7492550, 'num_objects': 1831, 'num_object_clones': 0, 'num_object_copies': 3662, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4237, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1831, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 15752, 'num_write_kb': 28172, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2406}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:07:45.509 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-10T16:07:45.509 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "47'18578", 'reported_seq': 18724, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:07:40.838178+0000', 'last_change': '2024-09-10T15:59:16.175680+0000', 'last_active': '2024-09-10T16:07:40.838178+0000', 'last_peered': '2024-09-10T16:07:40.838178+0000', 'last_clean': '2024-09-10T15:59:14.088187+0000', 'last_became_active': '2024-09-10T15:59:16.165810+0000', 'last_became_peered': '2024-09-10T15:59:16.165810+0000', 'last_unstale': '2024-09-10T16:07:40.838178+0000', 'last_undegraded': '2024-09-10T15:59:15.822098+0000', 'last_fullsized': '2024-09-10T15:59:15.769123+0000', 'mapping_epoch': 42, 'log_start': "43'8500", 'ondisk_log_start': "43'8500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2218", 'last_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'objects_scrubbed': 2175, 'log_size': 10078, 'log_dups_size': 0, 'ondisk_log_size': 10078, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 825, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 7696867, 'num_objects': 1880, 'num_object_clones': 0, 'num_object_copies': 3760, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4124, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1880, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 16397, 'num_write_kb': 32204, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': -2243}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:07:45.509 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-10T16:07:45.509 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "47'19100", 'reported_seq': 19237, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:07:40.539100+0000', 'last_change': '2024-09-10T15:59:16.277742+0000', 'last_active': '2024-09-10T16:07:40.539100+0000', 'last_peered': '2024-09-10T16:07:40.539100+0000', 'last_clean': '2024-09-10T15:59:14.129081+0000', 'last_became_active': '2024-09-10T15:59:16.269835+0000', 'last_became_peered': '2024-09-10T15:59:16.269835+0000', 'last_unstale': '2024-09-10T16:07:40.539100+0000', 'last_undegraded': '2024-09-10T15:59:15.860857+0000', 'last_fullsized': '2024-09-10T15:59:15.771410+0000', 'mapping_epoch': 42, 'log_start': "43'9000", 'ondisk_log_start': "43'9000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3557", 'last_scrub_stamp': '2024-09-10T15:59:02.493678+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:02.493678+0000', 'objects_scrubbed': 3488, 'log_size': 10100, 'log_dups_size': 0, 'ondisk_log_size': 10100, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1472, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 7413760, 'num_objects': 1810, 'num_object_clones': 0, 'num_object_copies': 3620, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4257, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1810, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15604, 'num_write_kb': 27852, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2445}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:07:45.510 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-10T16:07:45.510 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "47'19006", 'reported_seq': 19147, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:07:40.870415+0000', 'last_change': '2024-09-10T15:59:16.284311+0000', 'last_active': '2024-09-10T16:07:40.870415+0000', 'last_peered': '2024-09-10T16:07:40.870415+0000', 'last_clean': '2024-09-10T15:59:14.116041+0000', 'last_became_active': '2024-09-10T15:59:16.274543+0000', 'last_became_peered': '2024-09-10T15:59:16.274543+0000', 'last_unstale': '2024-09-10T16:07:40.870415+0000', 'last_undegraded': '2024-09-10T15:59:15.812484+0000', 'last_fullsized': '2024-09-10T15:59:15.770885+0000', 'mapping_epoch': 42, 'log_start': "43'9000", 'ondisk_log_start': "43'9000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2356", 'last_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'objects_scrubbed': 2316, 'log_size': 10006, 'log_dups_size': 0, 'ondisk_log_size': 10006, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 783, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 7270883, 'num_objects': 1776, 'num_object_clones': 0, 'num_object_copies': 3552, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1776, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 16686, 'num_write_kb': 32292, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': -2528}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:07:45.510 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-10T16:07:45.510 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "47'19389", 'reported_seq': 26321, 'reported_epoch': 47, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:07:40.884745+0000', 'last_change': '2024-09-10T16:05:18.271152+0000', 'last_active': '2024-09-10T16:07:40.884745+0000', 'last_peered': '2024-09-10T16:07:40.884745+0000', 'last_clean': '2024-09-10T15:59:14.110654+0000', 'last_became_active': '2024-09-10T15:59:16.164645+0000', 'last_became_peered': '2024-09-10T15:59:16.164645+0000', 'last_unstale': '2024-09-10T16:07:40.884745+0000', 'last_undegraded': '2024-09-10T15:59:15.803593+0000', 'last_fullsized': '2024-09-10T15:59:15.769083+0000', 'mapping_epoch': 42, 'log_start': "43'9300", 'ondisk_log_start': "43'9300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2486", 'last_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'objects_scrubbed': 2440, 'log_size': 10089, 'log_dups_size': 0, 'ondisk_log_size': 10089, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 792, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 7294976, 'num_objects': 1781, 'num_object_clones': 0, 'num_object_copies': 3562, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 956, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1781, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16943, 'num_write_kb': 32568, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3398, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 825}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:07:45.510 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-10T16:07:45.510 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "47'19045", 'reported_seq': 19186, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:07:40.489610+0000', 'last_change': '2024-09-10T15:59:16.315641+0000', 'last_active': '2024-09-10T16:07:40.489610+0000', 'last_peered': '2024-09-10T16:07:40.489610+0000', 'last_clean': '2024-09-10T15:59:14.117001+0000', 'last_became_active': '2024-09-10T15:59:16.302245+0000', 'last_became_peered': '2024-09-10T15:59:16.302245+0000', 'last_unstale': '2024-09-10T16:07:40.489610+0000', 'last_undegraded': '2024-09-10T15:59:15.845012+0000', 'last_fullsized': '2024-09-10T15:59:15.770337+0000', 'mapping_epoch': 42, 'log_start': "43'9000", 'ondisk_log_start': "43'9000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3404", 'last_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'objects_scrubbed': 3367, 'log_size': 10045, 'log_dups_size': 0, 'ondisk_log_size': 10045, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 983, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 7344128, 'num_objects': 1793, 'num_object_clones': 0, 'num_object_copies': 3586, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1793, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15676, 'num_write_kb': 28204, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2511}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:07:45.510 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-10T16:07:45.510 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "47'19305", 'reported_seq': 19453, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:07:40.556026+0000', 'last_change': '2024-09-10T15:59:16.311291+0000', 'last_active': '2024-09-10T16:07:40.556026+0000', 'last_peered': '2024-09-10T16:07:40.556026+0000', 'last_clean': '2024-09-10T15:59:14.116363+0000', 'last_became_active': '2024-09-10T15:59:16.301244+0000', 'last_became_peered': '2024-09-10T15:59:16.301244+0000', 'last_unstale': '2024-09-10T16:07:40.556026+0000', 'last_undegraded': '2024-09-10T15:59:15.883434+0000', 'last_fullsized': '2024-09-10T15:59:15.772756+0000', 'mapping_epoch': 42, 'log_start': "43'9300", 'ondisk_log_start': "43'9300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3789", 'last_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'objects_scrubbed': 3741, 'log_size': 10005, 'log_dups_size': 0, 'ondisk_log_size': 10005, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1440, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 7139328, 'num_objects': 1743, 'num_object_clones': 0, 'num_object_copies': 3486, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4373, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1743, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15560, 'num_write_kb': 27124, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2628}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:07:45.510 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-10T16:07:45.510 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "47'19379", 'reported_seq': 19528, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:07:40.471398+0000', 'last_change': '2024-09-10T15:59:16.313409+0000', 'last_active': '2024-09-10T16:07:40.471398+0000', 'last_peered': '2024-09-10T16:07:40.471398+0000', 'last_clean': '2024-09-10T15:59:14.129780+0000', 'last_became_active': '2024-09-10T15:59:16.300932+0000', 'last_became_peered': '2024-09-10T15:59:16.300932+0000', 'last_unstale': '2024-09-10T16:07:40.471398+0000', 'last_undegraded': '2024-09-10T15:59:15.887823+0000', 'last_fullsized': '2024-09-10T15:59:15.768513+0000', 'mapping_epoch': 42, 'log_start': "43'9300", 'ondisk_log_start': "43'9300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3880", 'last_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'objects_scrubbed': 3827, 'log_size': 10079, 'log_dups_size': 0, 'ondisk_log_size': 10079, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1397, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 7451590, 'num_objects': 1821, 'num_object_clones': 0, 'num_object_copies': 3642, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4325, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1821, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15542, 'num_write_kb': 27072, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2502}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:07:45.511 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T16:07:46.186 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:07:46.186 INFO:teuthology.orchestra.run.smithi115.stdout:{"fsid":"7effee29-df05-4351-9741-ee203564ed81","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 39513/54360 objects degraded (72.688%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":630,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":47,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983954,"num_in_osds":3,"osd_in_since":1725983947,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":27180,"data_bytes":111280905,"bytes_used":1681989632,"bytes_avail":288228302848,"bytes_total":289910292480,"degraded_objects":39513,"degraded_total":54360,"degraded_ratio":0.7268763796909492,"recovering_objects_per_sec":23,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":278,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":525},"fsmap":{"epoch":1,"btime":"2024-09-10T15:57:14:967120+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T16:00:13.560776+0000","services":{}},"progress_events":{}} 2024-09-10T16:07:46.208 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-10T16:07:46.208 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:07:46.729 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:07:46.729 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:07:46.754 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":355,"stamp":"2024-09-10T16:07:45.744841+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":111280905,"num_objects":27180,"num_object_clones":0,"num_object_copies":54360,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39513,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27180,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":264930,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11878,"num_bytes_recovered":33095699,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160757,"ondisk_log_size":160757,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1642568,"kb_used_data":425812,"kb_used_omap":12,"kb_used_meta":1216691,"kb_avail":281472952,"statfs":{"total":289910292480,"available":288228302848,"internally_reserved":0,"allocated":436031488,"data_stored":435094063,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1245891586},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,4,2,8,6,6,9],"upper_bound":128},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-25858048,"num_objects":-6313,"num_object_clones":0,"num_object_copies":-12626,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-278,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-6313,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6313,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":278,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003687"},"pg_stats":[{"pgid":"2.f","version":"47'19064","reported_seq":19144,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:45.507201+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:07:45.507201+0000","last_peered":"2024-09-10T16:07:45.507201+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:07:45.507201+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"43'9000","ondisk_log_start":"43'9000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6586368,"num_objects":1608,"num_object_clones":0,"num_object_copies":3216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1608,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19064,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2651}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'19065","reported_seq":19184,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:07:40.828722+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:07:40.828722+0000","last_peered":"2024-09-10T16:07:40.828722+0000","last_clean":"2024-09-10T16:07:40.828722+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:07:40.828722+0000","last_undegraded":"2024-09-10T16:07:40.828722+0000","last_fullsized":"2024-09-10T16:07:40.828722+0000","mapping_epoch":35,"log_start":"43'9000","ondisk_log_start":"43'9000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7287750,"num_objects":1781,"num_object_clones":0,"num_object_copies":3562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1781,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17170,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'19136","reported_seq":27820,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:07:44.312480+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:07:44.312480+0000","last_peered":"2024-09-10T16:07:44.312480+0000","last_clean":"2024-09-10T16:07:44.312480+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:07:44.312480+0000","last_undegraded":"2024-09-10T16:07:44.312480+0000","last_fullsized":"2024-09-10T16:07:44.312480+0000","mapping_epoch":44,"log_start":"43'9100","ondisk_log_start":"43'9100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6803480,"num_objects":1662,"num_object_clones":0,"num_object_copies":3324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1662,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17140,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'18908","reported_seq":18995,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:07:40.804889+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:07:40.804889+0000","last_peered":"2024-09-10T16:07:40.804889+0000","last_clean":"2024-09-10T16:07:40.804889+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:07:40.804889+0000","last_undegraded":"2024-09-10T16:07:40.804889+0000","last_fullsized":"2024-09-10T16:07:40.804889+0000","mapping_epoch":35,"log_start":"43'8900","ondisk_log_start":"43'8900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7176192,"num_objects":1752,"num_object_clones":0,"num_object_copies":3504,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1752,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18908,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'19264","reported_seq":19400,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:07:45.572409+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:07:45.572409+0000","last_peered":"2024-09-10T16:07:45.572409+0000","last_clean":"2024-09-10T16:07:45.572409+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:07:45.572409+0000","last_undegraded":"2024-09-10T16:07:45.572409+0000","last_fullsized":"2024-09-10T16:07:45.572409+0000","mapping_epoch":34,"log_start":"43'9200","ondisk_log_start":"43'9200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6664675,"num_objects":1628,"num_object_clones":0,"num_object_copies":3256,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1628,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15975,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'19388","reported_seq":19532,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:45.545199+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:07:45.545199+0000","last_peered":"2024-09-10T16:07:45.545199+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:07:45.545199+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"43'9300","ondisk_log_start":"43'9300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6853091,"num_objects":1674,"num_object_clones":0,"num_object_copies":3348,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1674,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16000,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2697},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'19406","reported_seq":19550,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:45.479369+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:07:45.479369+0000","last_peered":"2024-09-10T16:07:45.479369+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:07:45.479369+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"43'9400","ondisk_log_start":"43'9400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6865862,"num_objects":1678,"num_object_clones":0,"num_object_copies":3356,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1678,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":15905,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2559},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'19643","reported_seq":19779,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:07:45.629615+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:07:45.629615+0000","last_peered":"2024-09-10T16:07:45.629615+0000","last_clean":"2024-09-10T16:07:45.629615+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:07:45.629615+0000","last_undegraded":"2024-09-10T16:07:45.629615+0000","last_fullsized":"2024-09-10T16:07:45.629615+0000","mapping_epoch":34,"log_start":"43'9600","ondisk_log_start":"43'9600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6844416,"num_objects":1671,"num_object_clones":0,"num_object_copies":3342,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1671,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16490,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.876943+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:05:19.876943+0000","last_peered":"2024-09-10T16:05:19.876943+0000","last_clean":"2024-09-10T16:05:19.876943+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:05:19.876943+0000","last_undegraded":"2024-09-10T16:05:19.876943+0000","last_fullsized":"2024-09-10T16:05:19.876943+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'18578","reported_seq":18724,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:40.838178+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:07:40.838178+0000","last_peered":"2024-09-10T16:07:40.838178+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:07:40.838178+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"43'8500","ondisk_log_start":"43'8500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7696867,"num_objects":1880,"num_object_clones":0,"num_object_copies":3760,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1880,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16397,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-2243}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":100,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.875849+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:05:19.875849+0000","last_peered":"2024-09-10T16:05:19.875849+0000","last_clean":"2024-09-10T16:05:19.875849+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:05:19.875849+0000","last_undegraded":"2024-09-10T16:05:19.875849+0000","last_fullsized":"2024-09-10T16:05:19.875849+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'19250","reported_seq":19387,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:45.622522+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:07:45.622522+0000","last_peered":"2024-09-10T16:07:45.622522+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:07:45.622522+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"43'9200","ondisk_log_start":"43'9200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6799360,"num_objects":1660,"num_object_clones":0,"num_object_copies":3320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1660,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15754,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2595}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":110,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.872023+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:05:19.872023+0000","last_peered":"2024-09-10T16:05:19.872023+0000","last_clean":"2024-09-10T16:05:19.872023+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:05:19.872023+0000","last_undegraded":"2024-09-10T16:05:19.872023+0000","last_fullsized":"2024-09-10T16:05:19.872023+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'19006","reported_seq":19147,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:40.870415+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:07:40.870415+0000","last_peered":"2024-09-10T16:07:40.870415+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:07:40.870415+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"43'9000","ondisk_log_start":"43'9000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7270883,"num_objects":1776,"num_object_clones":0,"num_object_copies":3552,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1776,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16686,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-2528}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":86,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:32.574539+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:05:32.574539+0000","last_peered":"2024-09-10T16:05:32.574539+0000","last_clean":"2024-09-10T16:05:32.574539+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:05:32.574539+0000","last_undegraded":"2024-09-10T16:05:32.574539+0000","last_fullsized":"2024-09-10T16:05:32.574539+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'19389","reported_seq":26321,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:40.884745+0000","last_change":"2024-09-10T16:05:18.271152+0000","last_active":"2024-09-10T16:07:40.884745+0000","last_peered":"2024-09-10T16:07:40.884745+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:07:40.884745+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"43'9300","ondisk_log_start":"43'9300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7294976,"num_objects":1781,"num_object_clones":0,"num_object_copies":3562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":956,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1781,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16943,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3398,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":825}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'19182","reported_seq":19323,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:45.569537+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:07:45.569537+0000","last_peered":"2024-09-10T16:07:45.569537+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:07:45.569537+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"43'9100","ondisk_log_start":"43'9100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6782976,"num_objects":1656,"num_object_clones":0,"num_object_copies":3312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1656,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15813,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2648}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:32.574559+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:05:32.574559+0000","last_peered":"2024-09-10T16:05:32.574559+0000","last_clean":"2024-09-10T16:05:32.574559+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:05:32.574559+0000","last_undegraded":"2024-09-10T16:05:32.574559+0000","last_fullsized":"2024-09-10T16:05:32.574559+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'18915","reported_seq":27487,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:07:44.391404+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:07:44.391404+0000","last_peered":"2024-09-10T16:07:44.391404+0000","last_clean":"2024-09-10T16:07:44.391404+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:07:44.391404+0000","last_undegraded":"2024-09-10T16:07:44.391404+0000","last_fullsized":"2024-09-10T16:07:44.391404+0000","mapping_epoch":46,"log_start":"43'8900","ondisk_log_start":"43'8900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6893568,"num_objects":1683,"num_object_clones":0,"num_object_copies":3366,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1683,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15305,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":100,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.876043+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:05:19.876043+0000","last_peered":"2024-09-10T16:05:19.876043+0000","last_clean":"2024-09-10T16:05:19.876043+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:05:19.876043+0000","last_undegraded":"2024-09-10T16:05:19.876043+0000","last_fullsized":"2024-09-10T16:05:19.876043+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'19446","reported_seq":19593,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:45.597831+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:07:45.597831+0000","last_peered":"2024-09-10T16:07:45.597831+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:07:45.597831+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"43'9400","ondisk_log_start":"43'9400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6561792,"num_objects":1602,"num_object_clones":0,"num_object_copies":3204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1602,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15701,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2769}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:32.574572+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:05:32.574572+0000","last_peered":"2024-09-10T16:05:32.574572+0000","last_clean":"2024-09-10T16:05:32.574572+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:05:32.574572+0000","last_undegraded":"2024-09-10T16:05:32.574572+0000","last_fullsized":"2024-09-10T16:05:32.574572+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877269+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:05:19.877269+0000","last_peered":"2024-09-10T16:05:19.877269+0000","last_clean":"2024-09-10T16:05:19.877269+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:05:19.877269+0000","last_undegraded":"2024-09-10T16:05:19.877269+0000","last_fullsized":"2024-09-10T16:05:19.877269+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'19514","reported_seq":19665,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:45.562273+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:07:45.562273+0000","last_peered":"2024-09-10T16:07:45.562273+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:07:45.562273+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"43'9500","ondisk_log_start":"43'9500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6898630,"num_objects":1686,"num_object_clones":0,"num_object_copies":3372,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1686,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15677,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2637},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":111280886,"num_objects":27178,"num_object_clones":0,"num_object_copies":54356,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39513,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27178,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":264928,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11876,"num_bytes_recovered":33095680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":432918528,"data_stored":432776703,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160754,"ondisk_log_size":160754,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986292,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":560144,"kb_used_data":195260,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93811696,"statfs":{"total":96636764160,"available":96063176704,"internally_reserved":0,"allocated":199946240,"data_stored":199643556,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,1,1,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659238,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":465388,"kb_used_data":82460,"kb_used_omap":4,"kb_used_meta":382907,"kb_avail":93906452,"statfs":{"total":96636764160,"available":96160206848,"internally_reserved":0,"allocated":84439040,"data_stored":84106988,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":392097454},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,2,2,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182521,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":617036,"kb_used_data":148092,"kb_used_omap":4,"kb_used_meta":468923,"kb_avail":93754804,"statfs":{"total":96636764160,"available":96004919296,"internally_reserved":0,"allocated":151646208,"data_stored":151343519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":480177834},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,4,1,4,3,3,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":150822912,"data_stored":150786782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":82894848,"data_stored":82829355,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":199200768,"data_stored":199160566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:07:49.756 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:07:50.310 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:07:50.311 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:07:50.331 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":357,"stamp":"2024-09-10T16:07:49.745782+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":107049737,"num_objects":26147,"num_object_clones":0,"num_object_copies":52294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39357,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26147,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":265963,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12034,"num_bytes_recovered":33095699,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160790,"ondisk_log_size":160790,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1625632,"kb_used_data":413036,"kb_used_omap":12,"kb_used_meta":1212531,"kb_avail":281489888,"statfs":{"total":289910292480,"available":288245645312,"internally_reserved":0,"allocated":422948864,"data_stored":422011439,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1241631746},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,5,1,4,4,6,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-20533248,"num_objects":-5013,"num_object_clones":0,"num_object_copies":-10026,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-301,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5013,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5013,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":301,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003174"},"pg_stats":[{"pgid":"2.f","version":"47'19064","reported_seq":19144,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:45.507201+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:07:45.507201+0000","last_peered":"2024-09-10T16:07:45.507201+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:07:45.507201+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"43'9000","ondisk_log_start":"43'9000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6586368,"num_objects":1608,"num_object_clones":0,"num_object_copies":3216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1608,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19064,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2651}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'19206","reported_seq":19325,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:07:45.863482+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:07:45.863482+0000","last_peered":"2024-09-10T16:07:45.863482+0000","last_clean":"2024-09-10T16:07:45.863482+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:07:45.863482+0000","last_undegraded":"2024-09-10T16:07:45.863482+0000","last_fullsized":"2024-09-10T16:07:45.863482+0000","mapping_epoch":35,"log_start":"43'9200","ondisk_log_start":"43'9200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6710214,"num_objects":1640,"num_object_clones":0,"num_object_copies":3280,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1640,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17311,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'19275","reported_seq":27959,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:07:49.457707+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:07:49.457707+0000","last_peered":"2024-09-10T16:07:49.457707+0000","last_clean":"2024-09-10T16:07:49.457707+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:07:49.457707+0000","last_undegraded":"2024-09-10T16:07:49.457707+0000","last_fullsized":"2024-09-10T16:07:49.457707+0000","mapping_epoch":44,"log_start":"43'9200","ondisk_log_start":"43'9200","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6234136,"num_objects":1523,"num_object_clones":0,"num_object_copies":3046,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1523,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17279,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'19050","reported_seq":19137,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:07:45.907247+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:07:45.907247+0000","last_peered":"2024-09-10T16:07:45.907247+0000","last_clean":"2024-09-10T16:07:45.907247+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:07:45.907247+0000","last_undegraded":"2024-09-10T16:07:45.907247+0000","last_fullsized":"2024-09-10T16:07:45.907247+0000","mapping_epoch":35,"log_start":"43'9000","ondisk_log_start":"43'9000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6594560,"num_objects":1610,"num_object_clones":0,"num_object_copies":3220,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1610,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19050,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'19264","reported_seq":19400,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:07:45.572409+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:07:45.572409+0000","last_peered":"2024-09-10T16:07:45.572409+0000","last_clean":"2024-09-10T16:07:45.572409+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:07:45.572409+0000","last_undegraded":"2024-09-10T16:07:45.572409+0000","last_fullsized":"2024-09-10T16:07:45.572409+0000","mapping_epoch":34,"log_start":"43'9200","ondisk_log_start":"43'9200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6664675,"num_objects":1628,"num_object_clones":0,"num_object_copies":3256,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1628,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15975,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'19388","reported_seq":19532,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:45.545199+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:07:45.545199+0000","last_peered":"2024-09-10T16:07:45.545199+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:07:45.545199+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"43'9300","ondisk_log_start":"43'9300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6853091,"num_objects":1674,"num_object_clones":0,"num_object_copies":3348,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1674,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16000,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2697},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'19406","reported_seq":19550,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:45.479369+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:07:45.479369+0000","last_peered":"2024-09-10T16:07:45.479369+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:07:45.479369+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"43'9400","ondisk_log_start":"43'9400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6865862,"num_objects":1678,"num_object_clones":0,"num_object_copies":3356,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1678,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":15905,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2559},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'19643","reported_seq":19779,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:07:45.629615+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:07:45.629615+0000","last_peered":"2024-09-10T16:07:45.629615+0000","last_clean":"2024-09-10T16:07:45.629615+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:07:45.629615+0000","last_undegraded":"2024-09-10T16:07:45.629615+0000","last_fullsized":"2024-09-10T16:07:45.629615+0000","mapping_epoch":34,"log_start":"43'9600","ondisk_log_start":"43'9600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6844416,"num_objects":1671,"num_object_clones":0,"num_object_copies":3342,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1671,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16490,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.876943+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:05:19.876943+0000","last_peered":"2024-09-10T16:05:19.876943+0000","last_clean":"2024-09-10T16:05:19.876943+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:05:19.876943+0000","last_undegraded":"2024-09-10T16:05:19.876943+0000","last_fullsized":"2024-09-10T16:05:19.876943+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'18733","reported_seq":18879,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:45.849871+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:07:45.849871+0000","last_peered":"2024-09-10T16:07:45.849871+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:07:45.849871+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"43'8700","ondisk_log_start":"43'8700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7061987,"num_objects":1725,"num_object_clones":0,"num_object_copies":3450,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1725,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16552,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-2398}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":100,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.875849+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:05:19.875849+0000","last_peered":"2024-09-10T16:05:19.875849+0000","last_clean":"2024-09-10T16:05:19.875849+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:05:19.875849+0000","last_undegraded":"2024-09-10T16:05:19.875849+0000","last_fullsized":"2024-09-10T16:05:19.875849+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'19250","reported_seq":19387,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:45.622522+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:07:45.622522+0000","last_peered":"2024-09-10T16:07:45.622522+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:07:45.622522+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"43'9200","ondisk_log_start":"43'9200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6799360,"num_objects":1660,"num_object_clones":0,"num_object_copies":3320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1660,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15754,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2595}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":110,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.872023+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:05:19.872023+0000","last_peered":"2024-09-10T16:05:19.872023+0000","last_clean":"2024-09-10T16:05:19.872023+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:05:19.872023+0000","last_undegraded":"2024-09-10T16:05:19.872023+0000","last_fullsized":"2024-09-10T16:05:19.872023+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'19171","reported_seq":19312,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:45.906393+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:07:45.906393+0000","last_peered":"2024-09-10T16:07:45.906393+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:07:45.906393+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"43'9100","ondisk_log_start":"43'9100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6595043,"num_objects":1611,"num_object_clones":0,"num_object_copies":3222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1611,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16851,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-2693}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":86,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:32.574539+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:05:32.574539+0000","last_peered":"2024-09-10T16:05:32.574539+0000","last_clean":"2024-09-10T16:05:32.574539+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:05:32.574539+0000","last_undegraded":"2024-09-10T16:05:32.574539+0000","last_fullsized":"2024-09-10T16:05:32.574539+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'19529","reported_seq":26773,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:45.883460+0000","last_change":"2024-09-10T16:05:18.271152+0000","last_active":"2024-09-10T16:07:45.883460+0000","last_peered":"2024-09-10T16:07:45.883460+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:07:45.883460+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"43'9500","ondisk_log_start":"43'9500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6721536,"num_objects":1641,"num_object_clones":0,"num_object_copies":3282,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":800,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1641,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17083,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3554,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":841}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'19182","reported_seq":19323,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:45.569537+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:07:45.569537+0000","last_peered":"2024-09-10T16:07:45.569537+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:07:45.569537+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"43'9100","ondisk_log_start":"43'9100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6782976,"num_objects":1656,"num_object_clones":0,"num_object_copies":3312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1656,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15813,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2648}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:32.574559+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:05:32.574559+0000","last_peered":"2024-09-10T16:05:32.574559+0000","last_clean":"2024-09-10T16:05:32.574559+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:05:32.574559+0000","last_undegraded":"2024-09-10T16:05:32.574559+0000","last_fullsized":"2024-09-10T16:05:32.574559+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'19066","reported_seq":27638,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:07:49.418301+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:07:49.418301+0000","last_peered":"2024-09-10T16:07:49.418301+0000","last_clean":"2024-09-10T16:07:49.418301+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:07:49.418301+0000","last_undegraded":"2024-09-10T16:07:49.418301+0000","last_fullsized":"2024-09-10T16:07:49.418301+0000","mapping_epoch":46,"log_start":"43'9000","ondisk_log_start":"43'9000","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6275072,"num_objects":1532,"num_object_clones":0,"num_object_copies":3064,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1532,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15456,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":100,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.876043+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:05:19.876043+0000","last_peered":"2024-09-10T16:05:19.876043+0000","last_clean":"2024-09-10T16:05:19.876043+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:05:19.876043+0000","last_undegraded":"2024-09-10T16:05:19.876043+0000","last_fullsized":"2024-09-10T16:05:19.876043+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'19446","reported_seq":19593,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:45.597831+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:07:45.597831+0000","last_peered":"2024-09-10T16:07:45.597831+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:07:45.597831+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"43'9400","ondisk_log_start":"43'9400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6561792,"num_objects":1602,"num_object_clones":0,"num_object_copies":3204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1602,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15701,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2769}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:32.574572+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:05:32.574572+0000","last_peered":"2024-09-10T16:05:32.574572+0000","last_clean":"2024-09-10T16:05:32.574572+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:05:32.574572+0000","last_undegraded":"2024-09-10T16:05:32.574572+0000","last_fullsized":"2024-09-10T16:05:32.574572+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877269+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:05:19.877269+0000","last_peered":"2024-09-10T16:05:19.877269+0000","last_clean":"2024-09-10T16:05:19.877269+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:05:19.877269+0000","last_undegraded":"2024-09-10T16:05:19.877269+0000","last_fullsized":"2024-09-10T16:05:19.877269+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'19514","reported_seq":19665,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:45.562273+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:07:45.562273+0000","last_peered":"2024-09-10T16:07:45.562273+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:07:45.562273+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"43'9500","ondisk_log_start":"43'9500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6898630,"num_objects":1686,"num_object_clones":0,"num_object_copies":3372,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1686,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15677,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2637},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":107049718,"num_objects":26145,"num_object_clones":0,"num_object_copies":52290,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39357,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26145,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":265961,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12032,"num_bytes_recovered":33095680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":420802560,"data_stored":420660735,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160787,"ondisk_log_size":160787,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986293,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":559668,"kb_used_data":190628,"kb_used_omap":4,"kb_used_meta":369019,"kb_avail":93812172,"statfs":{"total":96636764160,"available":96063664128,"internally_reserved":0,"allocated":195203072,"data_stored":194900388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":377876138},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,0,1,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659239,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":448928,"kb_used_data":74316,"kb_used_omap":4,"kb_used_meta":374587,"kb_avail":93922912,"statfs":{"total":96636764160,"available":96177061888,"internally_reserved":0,"allocated":76099584,"data_stored":75767532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":383577774},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182521,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":617036,"kb_used_data":148092,"kb_used_omap":4,"kb_used_meta":468923,"kb_avail":93754804,"statfs":{"total":96636764160,"available":96004919296,"internally_reserved":0,"allocated":151646208,"data_stored":151343519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":480177834},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,4,1,4,3,3,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":150822912,"data_stored":150786782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":75608064,"data_stored":75542571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":194371584,"data_stored":194331382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:07:50.332 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:07:50.876 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:07:50.877 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:07:50.901 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":357,"stamp":"2024-09-10T16:07:49.745782+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":107049737,"num_objects":26147,"num_object_clones":0,"num_object_copies":52294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39357,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26147,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":265963,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12034,"num_bytes_recovered":33095699,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160790,"ondisk_log_size":160790,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1625632,"kb_used_data":413036,"kb_used_omap":12,"kb_used_meta":1212531,"kb_avail":281489888,"statfs":{"total":289910292480,"available":288245645312,"internally_reserved":0,"allocated":422948864,"data_stored":422011439,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1241631746},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,5,1,4,4,6,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-20533248,"num_objects":-5013,"num_object_clones":0,"num_object_copies":-10026,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-301,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5013,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5013,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":301,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003174"},"pg_stats":[{"pgid":"2.f","version":"47'19064","reported_seq":19144,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:45.507201+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:07:45.507201+0000","last_peered":"2024-09-10T16:07:45.507201+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:07:45.507201+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"43'9000","ondisk_log_start":"43'9000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6586368,"num_objects":1608,"num_object_clones":0,"num_object_copies":3216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1608,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19064,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2651}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'19206","reported_seq":19325,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:07:45.863482+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:07:45.863482+0000","last_peered":"2024-09-10T16:07:45.863482+0000","last_clean":"2024-09-10T16:07:45.863482+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:07:45.863482+0000","last_undegraded":"2024-09-10T16:07:45.863482+0000","last_fullsized":"2024-09-10T16:07:45.863482+0000","mapping_epoch":35,"log_start":"43'9200","ondisk_log_start":"43'9200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6710214,"num_objects":1640,"num_object_clones":0,"num_object_copies":3280,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1640,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17311,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'19275","reported_seq":27959,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:07:49.457707+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:07:49.457707+0000","last_peered":"2024-09-10T16:07:49.457707+0000","last_clean":"2024-09-10T16:07:49.457707+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:07:49.457707+0000","last_undegraded":"2024-09-10T16:07:49.457707+0000","last_fullsized":"2024-09-10T16:07:49.457707+0000","mapping_epoch":44,"log_start":"43'9200","ondisk_log_start":"43'9200","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6234136,"num_objects":1523,"num_object_clones":0,"num_object_copies":3046,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1523,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17279,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'19050","reported_seq":19137,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:07:45.907247+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:07:45.907247+0000","last_peered":"2024-09-10T16:07:45.907247+0000","last_clean":"2024-09-10T16:07:45.907247+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:07:45.907247+0000","last_undegraded":"2024-09-10T16:07:45.907247+0000","last_fullsized":"2024-09-10T16:07:45.907247+0000","mapping_epoch":35,"log_start":"43'9000","ondisk_log_start":"43'9000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6594560,"num_objects":1610,"num_object_clones":0,"num_object_copies":3220,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1610,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19050,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'19264","reported_seq":19400,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:07:45.572409+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:07:45.572409+0000","last_peered":"2024-09-10T16:07:45.572409+0000","last_clean":"2024-09-10T16:07:45.572409+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:07:45.572409+0000","last_undegraded":"2024-09-10T16:07:45.572409+0000","last_fullsized":"2024-09-10T16:07:45.572409+0000","mapping_epoch":34,"log_start":"43'9200","ondisk_log_start":"43'9200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6664675,"num_objects":1628,"num_object_clones":0,"num_object_copies":3256,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1628,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15975,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'19388","reported_seq":19532,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:45.545199+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:07:45.545199+0000","last_peered":"2024-09-10T16:07:45.545199+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:07:45.545199+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"43'9300","ondisk_log_start":"43'9300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6853091,"num_objects":1674,"num_object_clones":0,"num_object_copies":3348,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1674,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16000,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2697},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'19406","reported_seq":19550,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:45.479369+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:07:45.479369+0000","last_peered":"2024-09-10T16:07:45.479369+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:07:45.479369+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"43'9400","ondisk_log_start":"43'9400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6865862,"num_objects":1678,"num_object_clones":0,"num_object_copies":3356,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1678,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":15905,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2559},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'19643","reported_seq":19779,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:07:45.629615+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:07:45.629615+0000","last_peered":"2024-09-10T16:07:45.629615+0000","last_clean":"2024-09-10T16:07:45.629615+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:07:45.629615+0000","last_undegraded":"2024-09-10T16:07:45.629615+0000","last_fullsized":"2024-09-10T16:07:45.629615+0000","mapping_epoch":34,"log_start":"43'9600","ondisk_log_start":"43'9600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6844416,"num_objects":1671,"num_object_clones":0,"num_object_copies":3342,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1671,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16490,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.876943+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:05:19.876943+0000","last_peered":"2024-09-10T16:05:19.876943+0000","last_clean":"2024-09-10T16:05:19.876943+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:05:19.876943+0000","last_undegraded":"2024-09-10T16:05:19.876943+0000","last_fullsized":"2024-09-10T16:05:19.876943+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'18733","reported_seq":18879,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:45.849871+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:07:45.849871+0000","last_peered":"2024-09-10T16:07:45.849871+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:07:45.849871+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"43'8700","ondisk_log_start":"43'8700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7061987,"num_objects":1725,"num_object_clones":0,"num_object_copies":3450,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1725,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16552,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-2398}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":100,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.875849+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:05:19.875849+0000","last_peered":"2024-09-10T16:05:19.875849+0000","last_clean":"2024-09-10T16:05:19.875849+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:05:19.875849+0000","last_undegraded":"2024-09-10T16:05:19.875849+0000","last_fullsized":"2024-09-10T16:05:19.875849+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'19250","reported_seq":19387,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:45.622522+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:07:45.622522+0000","last_peered":"2024-09-10T16:07:45.622522+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:07:45.622522+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"43'9200","ondisk_log_start":"43'9200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6799360,"num_objects":1660,"num_object_clones":0,"num_object_copies":3320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1660,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15754,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2595}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":110,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.872023+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:05:19.872023+0000","last_peered":"2024-09-10T16:05:19.872023+0000","last_clean":"2024-09-10T16:05:19.872023+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:05:19.872023+0000","last_undegraded":"2024-09-10T16:05:19.872023+0000","last_fullsized":"2024-09-10T16:05:19.872023+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'19171","reported_seq":19312,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:45.906393+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:07:45.906393+0000","last_peered":"2024-09-10T16:07:45.906393+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:07:45.906393+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"43'9100","ondisk_log_start":"43'9100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6595043,"num_objects":1611,"num_object_clones":0,"num_object_copies":3222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1611,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16851,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-2693}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":86,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:32.574539+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:05:32.574539+0000","last_peered":"2024-09-10T16:05:32.574539+0000","last_clean":"2024-09-10T16:05:32.574539+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:05:32.574539+0000","last_undegraded":"2024-09-10T16:05:32.574539+0000","last_fullsized":"2024-09-10T16:05:32.574539+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'19529","reported_seq":26773,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:45.883460+0000","last_change":"2024-09-10T16:05:18.271152+0000","last_active":"2024-09-10T16:07:45.883460+0000","last_peered":"2024-09-10T16:07:45.883460+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:07:45.883460+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"43'9500","ondisk_log_start":"43'9500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6721536,"num_objects":1641,"num_object_clones":0,"num_object_copies":3282,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":800,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1641,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17083,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3554,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":841}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'19182","reported_seq":19323,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:45.569537+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:07:45.569537+0000","last_peered":"2024-09-10T16:07:45.569537+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:07:45.569537+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"43'9100","ondisk_log_start":"43'9100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6782976,"num_objects":1656,"num_object_clones":0,"num_object_copies":3312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1656,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15813,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2648}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:32.574559+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:05:32.574559+0000","last_peered":"2024-09-10T16:05:32.574559+0000","last_clean":"2024-09-10T16:05:32.574559+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:05:32.574559+0000","last_undegraded":"2024-09-10T16:05:32.574559+0000","last_fullsized":"2024-09-10T16:05:32.574559+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'19066","reported_seq":27638,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:07:49.418301+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:07:49.418301+0000","last_peered":"2024-09-10T16:07:49.418301+0000","last_clean":"2024-09-10T16:07:49.418301+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:07:49.418301+0000","last_undegraded":"2024-09-10T16:07:49.418301+0000","last_fullsized":"2024-09-10T16:07:49.418301+0000","mapping_epoch":46,"log_start":"43'9000","ondisk_log_start":"43'9000","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6275072,"num_objects":1532,"num_object_clones":0,"num_object_copies":3064,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1532,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15456,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":100,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.876043+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:05:19.876043+0000","last_peered":"2024-09-10T16:05:19.876043+0000","last_clean":"2024-09-10T16:05:19.876043+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:05:19.876043+0000","last_undegraded":"2024-09-10T16:05:19.876043+0000","last_fullsized":"2024-09-10T16:05:19.876043+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'19446","reported_seq":19593,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:45.597831+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:07:45.597831+0000","last_peered":"2024-09-10T16:07:45.597831+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:07:45.597831+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"43'9400","ondisk_log_start":"43'9400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6561792,"num_objects":1602,"num_object_clones":0,"num_object_copies":3204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1602,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15701,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2769}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:32.574572+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:05:32.574572+0000","last_peered":"2024-09-10T16:05:32.574572+0000","last_clean":"2024-09-10T16:05:32.574572+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:05:32.574572+0000","last_undegraded":"2024-09-10T16:05:32.574572+0000","last_fullsized":"2024-09-10T16:05:32.574572+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877269+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:05:19.877269+0000","last_peered":"2024-09-10T16:05:19.877269+0000","last_clean":"2024-09-10T16:05:19.877269+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:05:19.877269+0000","last_undegraded":"2024-09-10T16:05:19.877269+0000","last_fullsized":"2024-09-10T16:05:19.877269+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'19514","reported_seq":19665,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:45.562273+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:07:45.562273+0000","last_peered":"2024-09-10T16:07:45.562273+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:07:45.562273+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"43'9500","ondisk_log_start":"43'9500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6898630,"num_objects":1686,"num_object_clones":0,"num_object_copies":3372,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1686,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15677,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2637},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":107049718,"num_objects":26145,"num_object_clones":0,"num_object_copies":52290,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39357,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26145,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":265961,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12032,"num_bytes_recovered":33095680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":420802560,"data_stored":420660735,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160787,"ondisk_log_size":160787,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986293,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":559668,"kb_used_data":190628,"kb_used_omap":4,"kb_used_meta":369019,"kb_avail":93812172,"statfs":{"total":96636764160,"available":96063664128,"internally_reserved":0,"allocated":195203072,"data_stored":194900388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":377876138},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,0,1,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659239,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":448928,"kb_used_data":74316,"kb_used_omap":4,"kb_used_meta":374587,"kb_avail":93922912,"statfs":{"total":96636764160,"available":96177061888,"internally_reserved":0,"allocated":76099584,"data_stored":75767532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":383577774},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182521,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":617036,"kb_used_data":148092,"kb_used_omap":4,"kb_used_meta":468923,"kb_avail":93754804,"statfs":{"total":96636764160,"available":96004919296,"internally_reserved":0,"allocated":151646208,"data_stored":151343519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":480177834},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,4,1,4,3,3,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":150822912,"data_stored":150786782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":75608064,"data_stored":75542571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":194371584,"data_stored":194331382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:07:50.903 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-10T16:07:50.903 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "47'19064", 'reported_seq': 19144, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:07:45.507201+0000', 'last_change': '2024-09-10T15:59:16.281558+0000', 'last_active': '2024-09-10T16:07:45.507201+0000', 'last_peered': '2024-09-10T16:07:45.507201+0000', 'last_clean': '2024-09-10T15:59:14.132456+0000', 'last_became_active': '2024-09-10T15:59:16.274407+0000', 'last_became_peered': '2024-09-10T15:59:16.274407+0000', 'last_unstale': '2024-09-10T16:07:45.507201+0000', 'last_undegraded': '2024-09-10T15:59:15.851279+0000', 'last_fullsized': '2024-09-10T15:59:15.774356+0000', 'mapping_epoch': 42, 'log_start': "43'9000", 'ondisk_log_start': "43'9000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'objects_scrubbed': 0, 'log_size': 10064, 'log_dups_size': 0, 'ondisk_log_size': 10064, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 6586368, 'num_objects': 1608, 'num_object_clones': 0, 'num_object_copies': 3216, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4259, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1608, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 19064, 'num_write_kb': 41344, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2651}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:07:50.903 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-10T16:07:50.903 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "47'19388", 'reported_seq': 19532, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:07:45.545199+0000', 'last_change': '2024-09-10T15:59:16.308264+0000', 'last_active': '2024-09-10T16:07:45.545199+0000', 'last_peered': '2024-09-10T16:07:45.545199+0000', 'last_clean': '2024-09-10T15:59:14.111551+0000', 'last_became_active': '2024-09-10T15:59:16.300040+0000', 'last_became_peered': '2024-09-10T15:59:16.300040+0000', 'last_unstale': '2024-09-10T16:07:45.545199+0000', 'last_undegraded': '2024-09-10T15:59:15.878234+0000', 'last_fullsized': '2024-09-10T15:59:15.769328+0000', 'mapping_epoch': 42, 'log_start': "43'9300", 'ondisk_log_start': "43'9300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3442", 'last_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'objects_scrubbed': 3384, 'log_size': 10088, 'log_dups_size': 0, 'ondisk_log_size': 10088, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1292, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 6853091, 'num_objects': 1674, 'num_object_clones': 0, 'num_object_copies': 3348, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4372, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1674, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 16000, 'num_write_kb': 28580, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2697}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:07:50.903 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-10T16:07:50.903 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "47'19406", 'reported_seq': 19550, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:07:45.479369+0000', 'last_change': '2024-09-10T15:59:16.331899+0000', 'last_active': '2024-09-10T16:07:45.479369+0000', 'last_peered': '2024-09-10T16:07:45.479369+0000', 'last_clean': '2024-09-10T15:59:14.139436+0000', 'last_became_active': '2024-09-10T15:59:16.318715+0000', 'last_became_peered': '2024-09-10T15:59:16.318715+0000', 'last_unstale': '2024-09-10T16:07:45.479369+0000', 'last_undegraded': '2024-09-10T15:59:15.877524+0000', 'last_fullsized': '2024-09-10T15:59:15.769042+0000', 'mapping_epoch': 42, 'log_start': "43'9400", 'ondisk_log_start': "43'9400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3537", 'last_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'objects_scrubbed': 3497, 'log_size': 10006, 'log_dups_size': 0, 'ondisk_log_size': 10006, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1224, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 6865862, 'num_objects': 1678, 'num_object_clones': 0, 'num_object_copies': 3356, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4237, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1678, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 15905, 'num_write_kb': 28172, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2559}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:07:50.903 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-10T16:07:50.904 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "47'18733", 'reported_seq': 18879, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:07:45.849871+0000', 'last_change': '2024-09-10T15:59:16.175680+0000', 'last_active': '2024-09-10T16:07:45.849871+0000', 'last_peered': '2024-09-10T16:07:45.849871+0000', 'last_clean': '2024-09-10T15:59:14.088187+0000', 'last_became_active': '2024-09-10T15:59:16.165810+0000', 'last_became_peered': '2024-09-10T15:59:16.165810+0000', 'last_unstale': '2024-09-10T16:07:45.849871+0000', 'last_undegraded': '2024-09-10T15:59:15.822098+0000', 'last_fullsized': '2024-09-10T15:59:15.769123+0000', 'mapping_epoch': 42, 'log_start': "43'8700", 'ondisk_log_start': "43'8700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2218", 'last_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'objects_scrubbed': 2175, 'log_size': 10033, 'log_dups_size': 0, 'ondisk_log_size': 10033, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 825, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 7061987, 'num_objects': 1725, 'num_object_clones': 0, 'num_object_copies': 3450, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4124, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1725, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 16552, 'num_write_kb': 32204, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': -2398}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:07:50.904 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-10T16:07:50.904 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "47'19250", 'reported_seq': 19387, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:07:45.622522+0000', 'last_change': '2024-09-10T15:59:16.277742+0000', 'last_active': '2024-09-10T16:07:45.622522+0000', 'last_peered': '2024-09-10T16:07:45.622522+0000', 'last_clean': '2024-09-10T15:59:14.129081+0000', 'last_became_active': '2024-09-10T15:59:16.269835+0000', 'last_became_peered': '2024-09-10T15:59:16.269835+0000', 'last_unstale': '2024-09-10T16:07:45.622522+0000', 'last_undegraded': '2024-09-10T15:59:15.860857+0000', 'last_fullsized': '2024-09-10T15:59:15.771410+0000', 'mapping_epoch': 42, 'log_start': "43'9200", 'ondisk_log_start': "43'9200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3557", 'last_scrub_stamp': '2024-09-10T15:59:02.493678+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:02.493678+0000', 'objects_scrubbed': 3488, 'log_size': 10050, 'log_dups_size': 0, 'ondisk_log_size': 10050, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1472, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 6799360, 'num_objects': 1660, 'num_object_clones': 0, 'num_object_copies': 3320, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4257, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1660, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15754, 'num_write_kb': 27852, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2595}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:07:50.904 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-10T16:07:50.904 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "47'19171", 'reported_seq': 19312, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:07:45.906393+0000', 'last_change': '2024-09-10T15:59:16.284311+0000', 'last_active': '2024-09-10T16:07:45.906393+0000', 'last_peered': '2024-09-10T16:07:45.906393+0000', 'last_clean': '2024-09-10T15:59:14.116041+0000', 'last_became_active': '2024-09-10T15:59:16.274543+0000', 'last_became_peered': '2024-09-10T15:59:16.274543+0000', 'last_unstale': '2024-09-10T16:07:45.906393+0000', 'last_undegraded': '2024-09-10T15:59:15.812484+0000', 'last_fullsized': '2024-09-10T15:59:15.770885+0000', 'mapping_epoch': 42, 'log_start': "43'9100", 'ondisk_log_start': "43'9100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2356", 'last_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'objects_scrubbed': 2316, 'log_size': 10071, 'log_dups_size': 0, 'ondisk_log_size': 10071, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 783, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 6595043, 'num_objects': 1611, 'num_object_clones': 0, 'num_object_copies': 3222, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1611, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 16851, 'num_write_kb': 32292, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': -2693}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:07:50.904 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-10T16:07:50.904 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "47'19529", 'reported_seq': 26773, 'reported_epoch': 47, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:07:45.883460+0000', 'last_change': '2024-09-10T16:05:18.271152+0000', 'last_active': '2024-09-10T16:07:45.883460+0000', 'last_peered': '2024-09-10T16:07:45.883460+0000', 'last_clean': '2024-09-10T15:59:14.110654+0000', 'last_became_active': '2024-09-10T15:59:16.164645+0000', 'last_became_peered': '2024-09-10T15:59:16.164645+0000', 'last_unstale': '2024-09-10T16:07:45.883460+0000', 'last_undegraded': '2024-09-10T15:59:15.803593+0000', 'last_fullsized': '2024-09-10T15:59:15.769083+0000', 'mapping_epoch': 42, 'log_start': "43'9500", 'ondisk_log_start': "43'9500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2486", 'last_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'objects_scrubbed': 2440, 'log_size': 10029, 'log_dups_size': 0, 'ondisk_log_size': 10029, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 792, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 6721536, 'num_objects': 1641, 'num_object_clones': 0, 'num_object_copies': 3282, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 800, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1641, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17083, 'num_write_kb': 32568, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3554, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 841}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:07:50.904 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-10T16:07:50.904 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "47'19182", 'reported_seq': 19323, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:07:45.569537+0000', 'last_change': '2024-09-10T15:59:16.315641+0000', 'last_active': '2024-09-10T16:07:45.569537+0000', 'last_peered': '2024-09-10T16:07:45.569537+0000', 'last_clean': '2024-09-10T15:59:14.117001+0000', 'last_became_active': '2024-09-10T15:59:16.302245+0000', 'last_became_peered': '2024-09-10T15:59:16.302245+0000', 'last_unstale': '2024-09-10T16:07:45.569537+0000', 'last_undegraded': '2024-09-10T15:59:15.845012+0000', 'last_fullsized': '2024-09-10T15:59:15.770337+0000', 'mapping_epoch': 42, 'log_start': "43'9100", 'ondisk_log_start': "43'9100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3404", 'last_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'objects_scrubbed': 3367, 'log_size': 10082, 'log_dups_size': 0, 'ondisk_log_size': 10082, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 983, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 6782976, 'num_objects': 1656, 'num_object_clones': 0, 'num_object_copies': 3312, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1656, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15813, 'num_write_kb': 28204, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2648}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:07:50.904 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-10T16:07:50.905 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "47'19446", 'reported_seq': 19593, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:07:45.597831+0000', 'last_change': '2024-09-10T15:59:16.311291+0000', 'last_active': '2024-09-10T16:07:45.597831+0000', 'last_peered': '2024-09-10T16:07:45.597831+0000', 'last_clean': '2024-09-10T15:59:14.116363+0000', 'last_became_active': '2024-09-10T15:59:16.301244+0000', 'last_became_peered': '2024-09-10T15:59:16.301244+0000', 'last_unstale': '2024-09-10T16:07:45.597831+0000', 'last_undegraded': '2024-09-10T15:59:15.883434+0000', 'last_fullsized': '2024-09-10T15:59:15.772756+0000', 'mapping_epoch': 42, 'log_start': "43'9400", 'ondisk_log_start': "43'9400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3789", 'last_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'objects_scrubbed': 3741, 'log_size': 10046, 'log_dups_size': 0, 'ondisk_log_size': 10046, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1440, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 6561792, 'num_objects': 1602, 'num_object_clones': 0, 'num_object_copies': 3204, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4373, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1602, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15701, 'num_write_kb': 27124, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2769}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:07:50.905 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-10T16:07:50.905 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "47'19514", 'reported_seq': 19665, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:07:45.562273+0000', 'last_change': '2024-09-10T15:59:16.313409+0000', 'last_active': '2024-09-10T16:07:45.562273+0000', 'last_peered': '2024-09-10T16:07:45.562273+0000', 'last_clean': '2024-09-10T15:59:14.129780+0000', 'last_became_active': '2024-09-10T15:59:16.300932+0000', 'last_became_peered': '2024-09-10T15:59:16.300932+0000', 'last_unstale': '2024-09-10T16:07:45.562273+0000', 'last_undegraded': '2024-09-10T15:59:15.887823+0000', 'last_fullsized': '2024-09-10T15:59:15.768513+0000', 'mapping_epoch': 42, 'log_start': "43'9500", 'ondisk_log_start': "43'9500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3880", 'last_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'objects_scrubbed': 3827, 'log_size': 10014, 'log_dups_size': 0, 'ondisk_log_size': 10014, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1397, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 6898630, 'num_objects': 1686, 'num_object_clones': 0, 'num_object_copies': 3372, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4325, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1686, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15677, 'num_write_kb': 27072, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2637}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:07:50.905 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T16:07:51.617 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:07:51.617 INFO:teuthology.orchestra.run.smithi115.stdout:{"fsid":"7effee29-df05-4351-9741-ee203564ed81","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 39357/52294 objects degraded (75.261%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":635,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":47,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983954,"num_in_osds":3,"osd_in_since":1725983947,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":26147,"data_bytes":107049737,"bytes_used":1664647168,"bytes_avail":288245645312,"bytes_total":289910292480,"degraded_objects":39357,"degraded_total":52294,"degraded_ratio":0.75261024209278315,"recovering_objects_per_sec":25,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":301,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":417},"fsmap":{"epoch":1,"btime":"2024-09-10T15:57:14:967120+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T16:00:13.560776+0000","services":{}},"progress_events":{}} 2024-09-10T16:07:51.638 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-10T16:07:51.638 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:07:52.221 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:07:52.221 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:07:52.249 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":358,"stamp":"2024-09-10T16:07:51.746222+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":98665225,"num_objects":24100,"num_object_clones":0,"num_object_copies":48200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39184,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24100,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":268010,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12207,"num_bytes_recovered":33095699,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160637,"ondisk_log_size":160637,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1605736,"kb_used_data":401456,"kb_used_omap":12,"kb_used_meta":1204211,"kb_avail":281509784,"statfs":{"total":289910292480,"available":288266018816,"internally_reserved":0,"allocated":411090944,"data_stored":410153519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1233112066},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,2,3,7,2,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-27648000,"num_objects":-6750,"num_object_clones":0,"num_object_copies":-13500,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-474,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-6750,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6750,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":474,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003087"},"pg_stats":[{"pgid":"2.f","version":"47'19198","reported_seq":19279,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:50.558545+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:07:50.558545+0000","last_peered":"2024-09-10T16:07:50.558545+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:07:50.558545+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"43'9100","ondisk_log_start":"43'9100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6037504,"num_objects":1474,"num_object_clones":0,"num_object_copies":2948,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1474,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19198,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2785}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'19335","reported_seq":19454,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:07:50.925587+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:07:50.925587+0000","last_peered":"2024-09-10T16:07:50.925587+0000","last_clean":"2024-09-10T16:07:50.925587+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:07:50.925587+0000","last_undegraded":"2024-09-10T16:07:50.925587+0000","last_fullsized":"2024-09-10T16:07:50.925587+0000","mapping_epoch":35,"log_start":"43'9300","ondisk_log_start":"43'9300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6181830,"num_objects":1511,"num_object_clones":0,"num_object_copies":3022,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1511,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17440,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'19275","reported_seq":27959,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:07:49.457707+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:07:49.457707+0000","last_peered":"2024-09-10T16:07:49.457707+0000","last_clean":"2024-09-10T16:07:49.457707+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:07:49.457707+0000","last_undegraded":"2024-09-10T16:07:49.457707+0000","last_fullsized":"2024-09-10T16:07:49.457707+0000","mapping_epoch":44,"log_start":"43'9200","ondisk_log_start":"43'9200","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6234136,"num_objects":1523,"num_object_clones":0,"num_object_copies":3046,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1523,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17279,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'19208","reported_seq":19295,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:07:50.913405+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:07:50.913405+0000","last_peered":"2024-09-10T16:07:50.913405+0000","last_clean":"2024-09-10T16:07:50.913405+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:07:50.913405+0000","last_undegraded":"2024-09-10T16:07:50.913405+0000","last_fullsized":"2024-09-10T16:07:50.913405+0000","mapping_epoch":35,"log_start":"43'9200","ondisk_log_start":"43'9200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5947392,"num_objects":1452,"num_object_clones":0,"num_object_copies":2904,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1452,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19208,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'19401","reported_seq":19537,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:07:50.631815+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:07:50.631815+0000","last_peered":"2024-09-10T16:07:50.631815+0000","last_clean":"2024-09-10T16:07:50.631815+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:07:50.631815+0000","last_undegraded":"2024-09-10T16:07:50.631815+0000","last_fullsized":"2024-09-10T16:07:50.631815+0000","mapping_epoch":34,"log_start":"43'9400","ondisk_log_start":"43'9400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6103523,"num_objects":1491,"num_object_clones":0,"num_object_copies":2982,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1491,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16112,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'19514","reported_seq":19658,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:50.516497+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:07:50.516497+0000","last_peered":"2024-09-10T16:07:50.516497+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:07:50.516497+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"43'9500","ondisk_log_start":"43'9500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6336995,"num_objects":1548,"num_object_clones":0,"num_object_copies":3096,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1548,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16126,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2823},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'19549","reported_seq":19691,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:50.566197+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:07:50.566197+0000","last_peered":"2024-09-10T16:07:50.566197+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:07:50.566197+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"43'9500","ondisk_log_start":"43'9500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6280134,"num_objects":1535,"num_object_clones":0,"num_object_copies":3070,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1535,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16048,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2702},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'19784","reported_seq":19920,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:07:50.598666+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:07:50.598666+0000","last_peered":"2024-09-10T16:07:50.598666+0000","last_clean":"2024-09-10T16:07:50.598666+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:07:50.598666+0000","last_undegraded":"2024-09-10T16:07:50.598666+0000","last_fullsized":"2024-09-10T16:07:50.598666+0000","mapping_epoch":34,"log_start":"43'9700","ondisk_log_start":"43'9700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6266880,"num_objects":1530,"num_object_clones":0,"num_object_copies":3060,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1530,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16631,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.876943+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:05:19.876943+0000","last_peered":"2024-09-10T16:05:19.876943+0000","last_clean":"2024-09-10T16:05:19.876943+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:05:19.876943+0000","last_undegraded":"2024-09-10T16:05:19.876943+0000","last_fullsized":"2024-09-10T16:05:19.876943+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'18911","reported_seq":19057,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:50.912425+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:07:50.912425+0000","last_peered":"2024-09-10T16:07:50.912425+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:07:50.912425+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"43'8900","ondisk_log_start":"43'8900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6332899,"num_objects":1547,"num_object_clones":0,"num_object_copies":3094,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1547,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16730,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-2576}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":100,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.875849+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:05:19.875849+0000","last_peered":"2024-09-10T16:05:19.875849+0000","last_clean":"2024-09-10T16:05:19.875849+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:05:19.875849+0000","last_undegraded":"2024-09-10T16:05:19.875849+0000","last_fullsized":"2024-09-10T16:05:19.875849+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'19403","reported_seq":19539,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:50.604726+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:07:50.604726+0000","last_peered":"2024-09-10T16:07:50.604726+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:07:50.604726+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"43'9400","ondisk_log_start":"43'9400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6172672,"num_objects":1507,"num_object_clones":0,"num_object_copies":3014,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1507,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15907,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2748}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":110,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.872023+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:05:19.872023+0000","last_peered":"2024-09-10T16:05:19.872023+0000","last_clean":"2024-09-10T16:05:19.872023+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:05:19.872023+0000","last_undegraded":"2024-09-10T16:05:19.872023+0000","last_fullsized":"2024-09-10T16:05:19.872023+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'19313","reported_seq":19454,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:50.878314+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:07:50.878314+0000","last_peered":"2024-09-10T16:07:50.878314+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:07:50.878314+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"43'9300","ondisk_log_start":"43'9300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6013411,"num_objects":1469,"num_object_clones":0,"num_object_copies":2938,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1469,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16993,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-2835}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":86,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:32.574539+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:05:32.574539+0000","last_peered":"2024-09-10T16:05:32.574539+0000","last_clean":"2024-09-10T16:05:32.574539+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:05:32.574539+0000","last_undegraded":"2024-09-10T16:05:32.574539+0000","last_fullsized":"2024-09-10T16:05:32.574539+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'19670","reported_seq":27260,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:50.913235+0000","last_change":"2024-09-10T16:05:18.271152+0000","last_active":"2024-09-10T16:07:50.913235+0000","last_peered":"2024-09-10T16:07:50.913235+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:07:50.913235+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"43'9600","ondisk_log_start":"43'9600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6144000,"num_objects":1500,"num_object_clones":0,"num_object_copies":3000,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":627,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1500,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17224,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3727,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":873}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'19344","reported_seq":19485,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:50.643295+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:07:50.643295+0000","last_peered":"2024-09-10T16:07:50.643295+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:07:50.643295+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"43'9300","ondisk_log_start":"43'9300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6119424,"num_objects":1494,"num_object_clones":0,"num_object_copies":2988,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1494,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15975,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2810}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:32.574559+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:05:32.574559+0000","last_peered":"2024-09-10T16:05:32.574559+0000","last_clean":"2024-09-10T16:05:32.574559+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:05:32.574559+0000","last_undegraded":"2024-09-10T16:05:32.574559+0000","last_fullsized":"2024-09-10T16:05:32.574559+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'19066","reported_seq":27638,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:07:49.418301+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:07:49.418301+0000","last_peered":"2024-09-10T16:07:49.418301+0000","last_clean":"2024-09-10T16:07:49.418301+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:07:49.418301+0000","last_undegraded":"2024-09-10T16:07:49.418301+0000","last_fullsized":"2024-09-10T16:07:49.418301+0000","mapping_epoch":46,"log_start":"43'9000","ondisk_log_start":"43'9000","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6275072,"num_objects":1532,"num_object_clones":0,"num_object_copies":3064,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1532,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15456,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":100,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.876043+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:05:19.876043+0000","last_peered":"2024-09-10T16:05:19.876043+0000","last_clean":"2024-09-10T16:05:19.876043+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:05:19.876043+0000","last_undegraded":"2024-09-10T16:05:19.876043+0000","last_fullsized":"2024-09-10T16:05:19.876043+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'19603","reported_seq":19751,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:50.639139+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:07:50.639139+0000","last_peered":"2024-09-10T16:07:50.639139+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:07:50.639139+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"43'9600","ondisk_log_start":"43'9600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5918720,"num_objects":1445,"num_object_clones":0,"num_object_copies":2890,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1445,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15858,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2926}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:32.574572+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:05:32.574572+0000","last_peered":"2024-09-10T16:05:32.574572+0000","last_clean":"2024-09-10T16:05:32.574572+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:05:32.574572+0000","last_undegraded":"2024-09-10T16:05:32.574572+0000","last_fullsized":"2024-09-10T16:05:32.574572+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877269+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:05:19.877269+0000","last_peered":"2024-09-10T16:05:19.877269+0000","last_clean":"2024-09-10T16:05:19.877269+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:05:19.877269+0000","last_undegraded":"2024-09-10T16:05:19.877269+0000","last_fullsized":"2024-09-10T16:05:19.877269+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'19660","reported_seq":19811,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:50.513245+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:07:50.513245+0000","last_peered":"2024-09-10T16:07:50.513245+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:07:50.513245+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"43'9600","ondisk_log_start":"43'9600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6300614,"num_objects":1540,"num_object_clones":0,"num_object_copies":3080,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1540,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15823,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2783},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":98665206,"num_objects":24098,"num_object_clones":0,"num_object_copies":48196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39184,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24098,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":268008,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12205,"num_bytes_recovered":33095680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":408952832,"data_stored":408811007,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160634,"ondisk_log_size":160634,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986294,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":542564,"kb_used_data":186004,"kb_used_omap":4,"kb_used_meta":356539,"kb_avail":93829276,"statfs":{"total":96636764160,"available":96081178624,"internally_reserved":0,"allocated":190468096,"data_stored":190165412,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":365096618},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,0,0,2,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659239,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":448928,"kb_used_data":74316,"kb_used_omap":4,"kb_used_meta":374587,"kb_avail":93922912,"statfs":{"total":96636764160,"available":96177061888,"internally_reserved":0,"allocated":76099584,"data_stored":75767532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":383577774},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182522,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":614244,"kb_used_data":141136,"kb_used_omap":4,"kb_used_meta":473083,"kb_avail":93757596,"statfs":{"total":96636764160,"available":96007778304,"internally_reserved":0,"allocated":144523264,"data_stored":144220575,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":484437674},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,3,4,1,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":143732736,"data_stored":143696606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":75608064,"data_stored":75542571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":189612032,"data_stored":189571830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:07:55.251 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:07:55.821 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:07:55.824 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:07:55.844 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":360,"stamp":"2024-09-10T16:07:55.747122+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":91865865,"num_objects":22440,"num_object_clones":0,"num_object_copies":44880,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39184,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22440,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":269670,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12207,"num_bytes_recovered":33095699,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160597,"ondisk_log_size":160597,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1583188,"kb_used_data":387208,"kb_used_omap":12,"kb_used_meta":1195891,"kb_avail":281532332,"statfs":{"total":289910292480,"available":288289107968,"internally_reserved":0,"allocated":396500992,"data_stored":395563567,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1224592386},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,1,0,3,4,7,7,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-26009600,"num_objects":-6350,"num_object_clones":0,"num_object_copies":-12700,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-329,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-6350,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6350,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":329,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002810"},"pg_stats":[{"pgid":"2.f","version":"47'19353","reported_seq":19434,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:55.677851+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:07:55.677851+0000","last_peered":"2024-09-10T16:07:55.677851+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:07:55.677851+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"43'9300","ondisk_log_start":"43'9300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5402624,"num_objects":1319,"num_object_clones":0,"num_object_copies":2638,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1319,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19353,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2940}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'19335","reported_seq":19454,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:07:50.925587+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:07:50.925587+0000","last_peered":"2024-09-10T16:07:50.925587+0000","last_clean":"2024-09-10T16:07:50.925587+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:07:50.925587+0000","last_undegraded":"2024-09-10T16:07:50.925587+0000","last_fullsized":"2024-09-10T16:07:50.925587+0000","mapping_epoch":35,"log_start":"43'9300","ondisk_log_start":"43'9300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6181830,"num_objects":1511,"num_object_clones":0,"num_object_copies":3022,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1511,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17440,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'19418","reported_seq":28102,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:07:54.538580+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:07:54.538580+0000","last_peered":"2024-09-10T16:07:54.538580+0000","last_clean":"2024-09-10T16:07:54.538580+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:07:54.538580+0000","last_undegraded":"2024-09-10T16:07:54.538580+0000","last_fullsized":"2024-09-10T16:07:54.538580+0000","mapping_epoch":44,"log_start":"43'9400","ondisk_log_start":"43'9400","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5648408,"num_objects":1380,"num_object_clones":0,"num_object_copies":2760,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1380,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17422,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'19208","reported_seq":19295,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:07:50.913405+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:07:50.913405+0000","last_peered":"2024-09-10T16:07:50.913405+0000","last_clean":"2024-09-10T16:07:50.913405+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:07:50.913405+0000","last_undegraded":"2024-09-10T16:07:50.913405+0000","last_fullsized":"2024-09-10T16:07:50.913405+0000","mapping_epoch":35,"log_start":"43'9200","ondisk_log_start":"43'9200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5947392,"num_objects":1452,"num_object_clones":0,"num_object_copies":2904,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1452,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19208,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'19542","reported_seq":19678,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:07:55.650822+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:07:55.650822+0000","last_peered":"2024-09-10T16:07:55.650822+0000","last_clean":"2024-09-10T16:07:55.650822+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:07:55.650822+0000","last_undegraded":"2024-09-10T16:07:55.650822+0000","last_fullsized":"2024-09-10T16:07:55.650822+0000","mapping_epoch":34,"log_start":"43'9500","ondisk_log_start":"43'9500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5525987,"num_objects":1350,"num_object_clones":0,"num_object_copies":2700,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1350,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16253,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'19683","reported_seq":19824,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:55.503652+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:07:55.503652+0000","last_peered":"2024-09-10T16:07:55.503652+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:07:55.503652+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"43'9600","ondisk_log_start":"43'9600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5644771,"num_objects":1379,"num_object_clones":0,"num_object_copies":2758,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1379,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16295,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2992},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'19714","reported_seq":19858,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:55.617648+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:07:55.617648+0000","last_peered":"2024-09-10T16:07:55.617648+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:07:55.617648+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"43'9700","ondisk_log_start":"43'9700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5604294,"num_objects":1370,"num_object_clones":0,"num_object_copies":2740,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1370,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16213,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2867},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'19944","reported_seq":20080,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:07:55.540396+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:07:55.540396+0000","last_peered":"2024-09-10T16:07:55.540396+0000","last_clean":"2024-09-10T16:07:55.540396+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:07:55.540396+0000","last_undegraded":"2024-09-10T16:07:55.540396+0000","last_fullsized":"2024-09-10T16:07:55.540396+0000","mapping_epoch":34,"log_start":"43'9900","ondisk_log_start":"43'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5611520,"num_objects":1370,"num_object_clones":0,"num_object_copies":2740,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1370,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16791,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.876943+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:05:19.876943+0000","last_peered":"2024-09-10T16:05:19.876943+0000","last_clean":"2024-09-10T16:05:19.876943+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:05:19.876943+0000","last_undegraded":"2024-09-10T16:05:19.876943+0000","last_fullsized":"2024-09-10T16:05:19.876943+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'18911","reported_seq":19057,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:50.912425+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:07:50.912425+0000","last_peered":"2024-09-10T16:07:50.912425+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:07:50.912425+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"43'8900","ondisk_log_start":"43'8900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6332899,"num_objects":1547,"num_object_clones":0,"num_object_copies":3094,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1547,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16730,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-2576}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":100,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.875849+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:05:19.875849+0000","last_peered":"2024-09-10T16:05:19.875849+0000","last_clean":"2024-09-10T16:05:19.875849+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:05:19.875849+0000","last_undegraded":"2024-09-10T16:05:19.875849+0000","last_fullsized":"2024-09-10T16:05:19.875849+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'19560","reported_seq":19692,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:55.641441+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:07:55.641441+0000","last_peered":"2024-09-10T16:07:55.641441+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:07:55.641441+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"43'9500","ondisk_log_start":"43'9500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5529600,"num_objects":1350,"num_object_clones":0,"num_object_copies":2700,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1350,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16064,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2905}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":110,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.872023+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:05:19.872023+0000","last_peered":"2024-09-10T16:05:19.872023+0000","last_clean":"2024-09-10T16:05:19.872023+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:05:19.872023+0000","last_undegraded":"2024-09-10T16:05:19.872023+0000","last_fullsized":"2024-09-10T16:05:19.872023+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'19313","reported_seq":19454,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:50.878314+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:07:50.878314+0000","last_peered":"2024-09-10T16:07:50.878314+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:07:50.878314+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"43'9300","ondisk_log_start":"43'9300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6013411,"num_objects":1469,"num_object_clones":0,"num_object_copies":2938,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1469,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16993,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-2835}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":86,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:32.574539+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:05:32.574539+0000","last_peered":"2024-09-10T16:05:32.574539+0000","last_clean":"2024-09-10T16:05:32.574539+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:05:32.574539+0000","last_undegraded":"2024-09-10T16:05:32.574539+0000","last_fullsized":"2024-09-10T16:05:32.574539+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'19670","reported_seq":27260,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:50.913235+0000","last_change":"2024-09-10T16:05:18.271152+0000","last_active":"2024-09-10T16:07:50.913235+0000","last_peered":"2024-09-10T16:07:50.913235+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:07:50.913235+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"43'9600","ondisk_log_start":"43'9600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6144000,"num_objects":1500,"num_object_clones":0,"num_object_copies":3000,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":627,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1500,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17224,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3727,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":873}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'19474","reported_seq":19615,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:55.639009+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:07:55.639009+0000","last_peered":"2024-09-10T16:07:55.639009+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:07:55.639009+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"43'9400","ondisk_log_start":"43'9400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5586944,"num_objects":1364,"num_object_clones":0,"num_object_copies":2728,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1364,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16105,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2940}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:32.574559+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:05:32.574559+0000","last_peered":"2024-09-10T16:05:32.574559+0000","last_clean":"2024-09-10T16:05:32.574559+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:05:32.574559+0000","last_undegraded":"2024-09-10T16:05:32.574559+0000","last_fullsized":"2024-09-10T16:05:32.574559+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'19201","reported_seq":27773,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:07:54.501151+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:07:54.501151+0000","last_peered":"2024-09-10T16:07:54.501151+0000","last_clean":"2024-09-10T16:07:54.501151+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:07:54.501151+0000","last_undegraded":"2024-09-10T16:07:54.501151+0000","last_fullsized":"2024-09-10T16:07:54.501151+0000","mapping_epoch":46,"log_start":"43'9200","ondisk_log_start":"43'9200","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5722112,"num_objects":1397,"num_object_clones":0,"num_object_copies":2794,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1397,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15591,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":100,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.876043+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:05:19.876043+0000","last_peered":"2024-09-10T16:05:19.876043+0000","last_clean":"2024-09-10T16:05:19.876043+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:05:19.876043+0000","last_undegraded":"2024-09-10T16:05:19.876043+0000","last_fullsized":"2024-09-10T16:05:19.876043+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'19758","reported_seq":19906,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:55.575904+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:07:55.575904+0000","last_peered":"2024-09-10T16:07:55.575904+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:07:55.575904+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"43'9700","ondisk_log_start":"43'9700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5283840,"num_objects":1290,"num_object_clones":0,"num_object_copies":2580,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1290,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16013,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3081}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:32.574572+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:05:32.574572+0000","last_peered":"2024-09-10T16:05:32.574572+0000","last_clean":"2024-09-10T16:05:32.574572+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:05:32.574572+0000","last_undegraded":"2024-09-10T16:05:32.574572+0000","last_fullsized":"2024-09-10T16:05:32.574572+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877269+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:05:19.877269+0000","last_peered":"2024-09-10T16:05:19.877269+0000","last_clean":"2024-09-10T16:05:19.877269+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:05:19.877269+0000","last_undegraded":"2024-09-10T16:05:19.877269+0000","last_fullsized":"2024-09-10T16:05:19.877269+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'19810","reported_seq":19961,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:55.561240+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:07:55.561240+0000","last_peered":"2024-09-10T16:07:55.561240+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:07:55.561240+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"43'9800","ondisk_log_start":"43'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5686214,"num_objects":1390,"num_object_clones":0,"num_object_copies":2780,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1390,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15973,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2933},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":91865846,"num_objects":22438,"num_object_clones":0,"num_object_copies":44876,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39184,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22438,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":269668,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12205,"num_bytes_recovered":33095680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":394350592,"data_stored":394208767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160594,"ondisk_log_size":160594,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986294,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":542564,"kb_used_data":186004,"kb_used_omap":4,"kb_used_meta":356539,"kb_avail":93829276,"statfs":{"total":96636764160,"available":96081178624,"internally_reserved":0,"allocated":190468096,"data_stored":190165412,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":365096618},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,0,0,2,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659240,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":446124,"kb_used_data":67344,"kb_used_omap":4,"kb_used_meta":378747,"kb_avail":93925716,"statfs":{"total":96636764160,"available":96179933184,"internally_reserved":0,"allocated":68960256,"data_stored":68628204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":387837614},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,0,0,2,1,1,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182523,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":594500,"kb_used_data":133860,"kb_used_omap":4,"kb_used_meta":460603,"kb_avail":93777340,"statfs":{"total":96636764160,"available":96027996160,"internally_reserved":0,"allocated":137072640,"data_stored":136769951,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":471658154},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,3,4,4,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":136298496,"data_stored":136262366,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":68440064,"data_stored":68374571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":189612032,"data_stored":189571830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:07:55.846 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:07:56.414 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:07:56.415 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:07:56.436 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":360,"stamp":"2024-09-10T16:07:55.747122+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":91865865,"num_objects":22440,"num_object_clones":0,"num_object_copies":44880,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39184,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22440,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":269670,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12207,"num_bytes_recovered":33095699,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160597,"ondisk_log_size":160597,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1583188,"kb_used_data":387208,"kb_used_omap":12,"kb_used_meta":1195891,"kb_avail":281532332,"statfs":{"total":289910292480,"available":288289107968,"internally_reserved":0,"allocated":396500992,"data_stored":395563567,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1224592386},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,1,0,3,4,7,7,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-26009600,"num_objects":-6350,"num_object_clones":0,"num_object_copies":-12700,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-329,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-6350,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6350,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":329,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002810"},"pg_stats":[{"pgid":"2.f","version":"47'19353","reported_seq":19434,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:55.677851+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:07:55.677851+0000","last_peered":"2024-09-10T16:07:55.677851+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:07:55.677851+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"43'9300","ondisk_log_start":"43'9300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5402624,"num_objects":1319,"num_object_clones":0,"num_object_copies":2638,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1319,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19353,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2940}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'19335","reported_seq":19454,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:07:50.925587+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:07:50.925587+0000","last_peered":"2024-09-10T16:07:50.925587+0000","last_clean":"2024-09-10T16:07:50.925587+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:07:50.925587+0000","last_undegraded":"2024-09-10T16:07:50.925587+0000","last_fullsized":"2024-09-10T16:07:50.925587+0000","mapping_epoch":35,"log_start":"43'9300","ondisk_log_start":"43'9300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6181830,"num_objects":1511,"num_object_clones":0,"num_object_copies":3022,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1511,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17440,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'19418","reported_seq":28102,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:07:54.538580+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:07:54.538580+0000","last_peered":"2024-09-10T16:07:54.538580+0000","last_clean":"2024-09-10T16:07:54.538580+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:07:54.538580+0000","last_undegraded":"2024-09-10T16:07:54.538580+0000","last_fullsized":"2024-09-10T16:07:54.538580+0000","mapping_epoch":44,"log_start":"43'9400","ondisk_log_start":"43'9400","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5648408,"num_objects":1380,"num_object_clones":0,"num_object_copies":2760,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1380,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17422,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'19208","reported_seq":19295,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:07:50.913405+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:07:50.913405+0000","last_peered":"2024-09-10T16:07:50.913405+0000","last_clean":"2024-09-10T16:07:50.913405+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:07:50.913405+0000","last_undegraded":"2024-09-10T16:07:50.913405+0000","last_fullsized":"2024-09-10T16:07:50.913405+0000","mapping_epoch":35,"log_start":"43'9200","ondisk_log_start":"43'9200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5947392,"num_objects":1452,"num_object_clones":0,"num_object_copies":2904,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1452,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19208,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'19542","reported_seq":19678,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:07:55.650822+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:07:55.650822+0000","last_peered":"2024-09-10T16:07:55.650822+0000","last_clean":"2024-09-10T16:07:55.650822+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:07:55.650822+0000","last_undegraded":"2024-09-10T16:07:55.650822+0000","last_fullsized":"2024-09-10T16:07:55.650822+0000","mapping_epoch":34,"log_start":"43'9500","ondisk_log_start":"43'9500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5525987,"num_objects":1350,"num_object_clones":0,"num_object_copies":2700,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1350,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16253,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'19683","reported_seq":19824,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:55.503652+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:07:55.503652+0000","last_peered":"2024-09-10T16:07:55.503652+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:07:55.503652+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"43'9600","ondisk_log_start":"43'9600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5644771,"num_objects":1379,"num_object_clones":0,"num_object_copies":2758,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1379,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16295,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2992},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'19714","reported_seq":19858,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:55.617648+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:07:55.617648+0000","last_peered":"2024-09-10T16:07:55.617648+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:07:55.617648+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"43'9700","ondisk_log_start":"43'9700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5604294,"num_objects":1370,"num_object_clones":0,"num_object_copies":2740,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1370,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16213,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2867},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'19944","reported_seq":20080,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:07:55.540396+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:07:55.540396+0000","last_peered":"2024-09-10T16:07:55.540396+0000","last_clean":"2024-09-10T16:07:55.540396+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:07:55.540396+0000","last_undegraded":"2024-09-10T16:07:55.540396+0000","last_fullsized":"2024-09-10T16:07:55.540396+0000","mapping_epoch":34,"log_start":"43'9900","ondisk_log_start":"43'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5611520,"num_objects":1370,"num_object_clones":0,"num_object_copies":2740,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1370,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16791,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.876943+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:05:19.876943+0000","last_peered":"2024-09-10T16:05:19.876943+0000","last_clean":"2024-09-10T16:05:19.876943+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:05:19.876943+0000","last_undegraded":"2024-09-10T16:05:19.876943+0000","last_fullsized":"2024-09-10T16:05:19.876943+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'18911","reported_seq":19057,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:50.912425+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:07:50.912425+0000","last_peered":"2024-09-10T16:07:50.912425+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:07:50.912425+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"43'8900","ondisk_log_start":"43'8900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6332899,"num_objects":1547,"num_object_clones":0,"num_object_copies":3094,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1547,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16730,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-2576}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":100,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.875849+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:05:19.875849+0000","last_peered":"2024-09-10T16:05:19.875849+0000","last_clean":"2024-09-10T16:05:19.875849+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:05:19.875849+0000","last_undegraded":"2024-09-10T16:05:19.875849+0000","last_fullsized":"2024-09-10T16:05:19.875849+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'19560","reported_seq":19692,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:55.641441+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:07:55.641441+0000","last_peered":"2024-09-10T16:07:55.641441+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:07:55.641441+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"43'9500","ondisk_log_start":"43'9500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5529600,"num_objects":1350,"num_object_clones":0,"num_object_copies":2700,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1350,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16064,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2905}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":110,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.872023+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:05:19.872023+0000","last_peered":"2024-09-10T16:05:19.872023+0000","last_clean":"2024-09-10T16:05:19.872023+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:05:19.872023+0000","last_undegraded":"2024-09-10T16:05:19.872023+0000","last_fullsized":"2024-09-10T16:05:19.872023+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'19313","reported_seq":19454,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:50.878314+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:07:50.878314+0000","last_peered":"2024-09-10T16:07:50.878314+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:07:50.878314+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"43'9300","ondisk_log_start":"43'9300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6013411,"num_objects":1469,"num_object_clones":0,"num_object_copies":2938,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1469,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16993,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-2835}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":86,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:32.574539+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:05:32.574539+0000","last_peered":"2024-09-10T16:05:32.574539+0000","last_clean":"2024-09-10T16:05:32.574539+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:05:32.574539+0000","last_undegraded":"2024-09-10T16:05:32.574539+0000","last_fullsized":"2024-09-10T16:05:32.574539+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'19670","reported_seq":27260,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:50.913235+0000","last_change":"2024-09-10T16:05:18.271152+0000","last_active":"2024-09-10T16:07:50.913235+0000","last_peered":"2024-09-10T16:07:50.913235+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:07:50.913235+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"43'9600","ondisk_log_start":"43'9600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6144000,"num_objects":1500,"num_object_clones":0,"num_object_copies":3000,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":627,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1500,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17224,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3727,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":873}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'19474","reported_seq":19615,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:55.639009+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:07:55.639009+0000","last_peered":"2024-09-10T16:07:55.639009+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:07:55.639009+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"43'9400","ondisk_log_start":"43'9400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5586944,"num_objects":1364,"num_object_clones":0,"num_object_copies":2728,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1364,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16105,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2940}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:32.574559+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:05:32.574559+0000","last_peered":"2024-09-10T16:05:32.574559+0000","last_clean":"2024-09-10T16:05:32.574559+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:05:32.574559+0000","last_undegraded":"2024-09-10T16:05:32.574559+0000","last_fullsized":"2024-09-10T16:05:32.574559+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'19201","reported_seq":27773,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:07:54.501151+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:07:54.501151+0000","last_peered":"2024-09-10T16:07:54.501151+0000","last_clean":"2024-09-10T16:07:54.501151+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:07:54.501151+0000","last_undegraded":"2024-09-10T16:07:54.501151+0000","last_fullsized":"2024-09-10T16:07:54.501151+0000","mapping_epoch":46,"log_start":"43'9200","ondisk_log_start":"43'9200","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5722112,"num_objects":1397,"num_object_clones":0,"num_object_copies":2794,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1397,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15591,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":100,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.876043+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:05:19.876043+0000","last_peered":"2024-09-10T16:05:19.876043+0000","last_clean":"2024-09-10T16:05:19.876043+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:05:19.876043+0000","last_undegraded":"2024-09-10T16:05:19.876043+0000","last_fullsized":"2024-09-10T16:05:19.876043+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'19758","reported_seq":19906,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:55.575904+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:07:55.575904+0000","last_peered":"2024-09-10T16:07:55.575904+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:07:55.575904+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"43'9700","ondisk_log_start":"43'9700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5283840,"num_objects":1290,"num_object_clones":0,"num_object_copies":2580,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1290,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16013,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3081}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:32.574572+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:05:32.574572+0000","last_peered":"2024-09-10T16:05:32.574572+0000","last_clean":"2024-09-10T16:05:32.574572+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:05:32.574572+0000","last_undegraded":"2024-09-10T16:05:32.574572+0000","last_fullsized":"2024-09-10T16:05:32.574572+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877269+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:05:19.877269+0000","last_peered":"2024-09-10T16:05:19.877269+0000","last_clean":"2024-09-10T16:05:19.877269+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:05:19.877269+0000","last_undegraded":"2024-09-10T16:05:19.877269+0000","last_fullsized":"2024-09-10T16:05:19.877269+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'19810","reported_seq":19961,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:55.561240+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:07:55.561240+0000","last_peered":"2024-09-10T16:07:55.561240+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:07:55.561240+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"43'9800","ondisk_log_start":"43'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5686214,"num_objects":1390,"num_object_clones":0,"num_object_copies":2780,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1390,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15973,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2933},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":91865846,"num_objects":22438,"num_object_clones":0,"num_object_copies":44876,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39184,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22438,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":269668,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12205,"num_bytes_recovered":33095680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":394350592,"data_stored":394208767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160594,"ondisk_log_size":160594,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986294,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":542564,"kb_used_data":186004,"kb_used_omap":4,"kb_used_meta":356539,"kb_avail":93829276,"statfs":{"total":96636764160,"available":96081178624,"internally_reserved":0,"allocated":190468096,"data_stored":190165412,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":365096618},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,0,0,2,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659240,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":446124,"kb_used_data":67344,"kb_used_omap":4,"kb_used_meta":378747,"kb_avail":93925716,"statfs":{"total":96636764160,"available":96179933184,"internally_reserved":0,"allocated":68960256,"data_stored":68628204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":387837614},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,0,0,2,1,1,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182523,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":594500,"kb_used_data":133860,"kb_used_omap":4,"kb_used_meta":460603,"kb_avail":93777340,"statfs":{"total":96636764160,"available":96027996160,"internally_reserved":0,"allocated":137072640,"data_stored":136769951,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":471658154},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,3,4,4,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":136298496,"data_stored":136262366,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":68440064,"data_stored":68374571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":189612032,"data_stored":189571830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:07:56.437 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-10T16:07:56.437 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "47'19353", 'reported_seq': 19434, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:07:55.677851+0000', 'last_change': '2024-09-10T15:59:16.281558+0000', 'last_active': '2024-09-10T16:07:55.677851+0000', 'last_peered': '2024-09-10T16:07:55.677851+0000', 'last_clean': '2024-09-10T15:59:14.132456+0000', 'last_became_active': '2024-09-10T15:59:16.274407+0000', 'last_became_peered': '2024-09-10T15:59:16.274407+0000', 'last_unstale': '2024-09-10T16:07:55.677851+0000', 'last_undegraded': '2024-09-10T15:59:15.851279+0000', 'last_fullsized': '2024-09-10T15:59:15.774356+0000', 'mapping_epoch': 42, 'log_start': "43'9300", 'ondisk_log_start': "43'9300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'objects_scrubbed': 0, 'log_size': 10053, 'log_dups_size': 0, 'ondisk_log_size': 10053, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 5402624, 'num_objects': 1319, 'num_object_clones': 0, 'num_object_copies': 2638, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4259, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1319, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 19353, 'num_write_kb': 41344, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2940}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:07:56.437 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-10T16:07:56.438 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "47'19683", 'reported_seq': 19824, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:07:55.503652+0000', 'last_change': '2024-09-10T15:59:16.308264+0000', 'last_active': '2024-09-10T16:07:55.503652+0000', 'last_peered': '2024-09-10T16:07:55.503652+0000', 'last_clean': '2024-09-10T15:59:14.111551+0000', 'last_became_active': '2024-09-10T15:59:16.300040+0000', 'last_became_peered': '2024-09-10T15:59:16.300040+0000', 'last_unstale': '2024-09-10T16:07:55.503652+0000', 'last_undegraded': '2024-09-10T15:59:15.878234+0000', 'last_fullsized': '2024-09-10T15:59:15.769328+0000', 'mapping_epoch': 42, 'log_start': "43'9600", 'ondisk_log_start': "43'9600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3442", 'last_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'objects_scrubbed': 3384, 'log_size': 10083, 'log_dups_size': 0, 'ondisk_log_size': 10083, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1292, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 5644771, 'num_objects': 1379, 'num_object_clones': 0, 'num_object_copies': 2758, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4372, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1379, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 16295, 'num_write_kb': 28580, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2992}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:07:56.438 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-10T16:07:56.438 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "47'19714", 'reported_seq': 19858, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:07:55.617648+0000', 'last_change': '2024-09-10T15:59:16.331899+0000', 'last_active': '2024-09-10T16:07:55.617648+0000', 'last_peered': '2024-09-10T16:07:55.617648+0000', 'last_clean': '2024-09-10T15:59:14.139436+0000', 'last_became_active': '2024-09-10T15:59:16.318715+0000', 'last_became_peered': '2024-09-10T15:59:16.318715+0000', 'last_unstale': '2024-09-10T16:07:55.617648+0000', 'last_undegraded': '2024-09-10T15:59:15.877524+0000', 'last_fullsized': '2024-09-10T15:59:15.769042+0000', 'mapping_epoch': 42, 'log_start': "43'9700", 'ondisk_log_start': "43'9700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3537", 'last_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'objects_scrubbed': 3497, 'log_size': 10014, 'log_dups_size': 0, 'ondisk_log_size': 10014, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1224, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 5604294, 'num_objects': 1370, 'num_object_clones': 0, 'num_object_copies': 2740, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4237, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1370, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 16213, 'num_write_kb': 28172, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2867}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:07:56.438 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-10T16:07:56.438 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "47'18911", 'reported_seq': 19057, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:07:50.912425+0000', 'last_change': '2024-09-10T15:59:16.175680+0000', 'last_active': '2024-09-10T16:07:50.912425+0000', 'last_peered': '2024-09-10T16:07:50.912425+0000', 'last_clean': '2024-09-10T15:59:14.088187+0000', 'last_became_active': '2024-09-10T15:59:16.165810+0000', 'last_became_peered': '2024-09-10T15:59:16.165810+0000', 'last_unstale': '2024-09-10T16:07:50.912425+0000', 'last_undegraded': '2024-09-10T15:59:15.822098+0000', 'last_fullsized': '2024-09-10T15:59:15.769123+0000', 'mapping_epoch': 42, 'log_start': "43'8900", 'ondisk_log_start': "43'8900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2218", 'last_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'objects_scrubbed': 2175, 'log_size': 10011, 'log_dups_size': 0, 'ondisk_log_size': 10011, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 825, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 6332899, 'num_objects': 1547, 'num_object_clones': 0, 'num_object_copies': 3094, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4124, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1547, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 16730, 'num_write_kb': 32204, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': -2576}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:07:56.438 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-10T16:07:56.438 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "47'19560", 'reported_seq': 19692, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:07:55.641441+0000', 'last_change': '2024-09-10T15:59:16.277742+0000', 'last_active': '2024-09-10T16:07:55.641441+0000', 'last_peered': '2024-09-10T16:07:55.641441+0000', 'last_clean': '2024-09-10T15:59:14.129081+0000', 'last_became_active': '2024-09-10T15:59:16.269835+0000', 'last_became_peered': '2024-09-10T15:59:16.269835+0000', 'last_unstale': '2024-09-10T16:07:55.641441+0000', 'last_undegraded': '2024-09-10T15:59:15.860857+0000', 'last_fullsized': '2024-09-10T15:59:15.771410+0000', 'mapping_epoch': 42, 'log_start': "43'9500", 'ondisk_log_start': "43'9500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3557", 'last_scrub_stamp': '2024-09-10T15:59:02.493678+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:02.493678+0000', 'objects_scrubbed': 3488, 'log_size': 10060, 'log_dups_size': 0, 'ondisk_log_size': 10060, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1472, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 5529600, 'num_objects': 1350, 'num_object_clones': 0, 'num_object_copies': 2700, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4257, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1350, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16064, 'num_write_kb': 27852, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2905}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:07:56.438 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-10T16:07:56.438 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "47'19313", 'reported_seq': 19454, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:07:50.878314+0000', 'last_change': '2024-09-10T15:59:16.284311+0000', 'last_active': '2024-09-10T16:07:50.878314+0000', 'last_peered': '2024-09-10T16:07:50.878314+0000', 'last_clean': '2024-09-10T15:59:14.116041+0000', 'last_became_active': '2024-09-10T15:59:16.274543+0000', 'last_became_peered': '2024-09-10T15:59:16.274543+0000', 'last_unstale': '2024-09-10T16:07:50.878314+0000', 'last_undegraded': '2024-09-10T15:59:15.812484+0000', 'last_fullsized': '2024-09-10T15:59:15.770885+0000', 'mapping_epoch': 42, 'log_start': "43'9300", 'ondisk_log_start': "43'9300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2356", 'last_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'objects_scrubbed': 2316, 'log_size': 10013, 'log_dups_size': 0, 'ondisk_log_size': 10013, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 783, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 6013411, 'num_objects': 1469, 'num_object_clones': 0, 'num_object_copies': 2938, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1469, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 16993, 'num_write_kb': 32292, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': -2835}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:07:56.438 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-10T16:07:56.439 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "47'19670", 'reported_seq': 27260, 'reported_epoch': 47, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:07:50.913235+0000', 'last_change': '2024-09-10T16:05:18.271152+0000', 'last_active': '2024-09-10T16:07:50.913235+0000', 'last_peered': '2024-09-10T16:07:50.913235+0000', 'last_clean': '2024-09-10T15:59:14.110654+0000', 'last_became_active': '2024-09-10T15:59:16.164645+0000', 'last_became_peered': '2024-09-10T15:59:16.164645+0000', 'last_unstale': '2024-09-10T16:07:50.913235+0000', 'last_undegraded': '2024-09-10T15:59:15.803593+0000', 'last_fullsized': '2024-09-10T15:59:15.769083+0000', 'mapping_epoch': 42, 'log_start': "43'9600", 'ondisk_log_start': "43'9600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2486", 'last_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'objects_scrubbed': 2440, 'log_size': 10070, 'log_dups_size': 0, 'ondisk_log_size': 10070, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 792, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 6144000, 'num_objects': 1500, 'num_object_clones': 0, 'num_object_copies': 3000, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 627, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1500, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17224, 'num_write_kb': 32568, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3727, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 873}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:07:56.439 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-10T16:07:56.439 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "47'19474", 'reported_seq': 19615, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:07:55.639009+0000', 'last_change': '2024-09-10T15:59:16.315641+0000', 'last_active': '2024-09-10T16:07:55.639009+0000', 'last_peered': '2024-09-10T16:07:55.639009+0000', 'last_clean': '2024-09-10T15:59:14.117001+0000', 'last_became_active': '2024-09-10T15:59:16.302245+0000', 'last_became_peered': '2024-09-10T15:59:16.302245+0000', 'last_unstale': '2024-09-10T16:07:55.639009+0000', 'last_undegraded': '2024-09-10T15:59:15.845012+0000', 'last_fullsized': '2024-09-10T15:59:15.770337+0000', 'mapping_epoch': 42, 'log_start': "43'9400", 'ondisk_log_start': "43'9400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3404", 'last_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'objects_scrubbed': 3367, 'log_size': 10074, 'log_dups_size': 0, 'ondisk_log_size': 10074, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 983, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 5586944, 'num_objects': 1364, 'num_object_clones': 0, 'num_object_copies': 2728, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1364, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16105, 'num_write_kb': 28204, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2940}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:07:56.439 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-10T16:07:56.439 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "47'19758", 'reported_seq': 19906, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:07:55.575904+0000', 'last_change': '2024-09-10T15:59:16.311291+0000', 'last_active': '2024-09-10T16:07:55.575904+0000', 'last_peered': '2024-09-10T16:07:55.575904+0000', 'last_clean': '2024-09-10T15:59:14.116363+0000', 'last_became_active': '2024-09-10T15:59:16.301244+0000', 'last_became_peered': '2024-09-10T15:59:16.301244+0000', 'last_unstale': '2024-09-10T16:07:55.575904+0000', 'last_undegraded': '2024-09-10T15:59:15.883434+0000', 'last_fullsized': '2024-09-10T15:59:15.772756+0000', 'mapping_epoch': 42, 'log_start': "43'9700", 'ondisk_log_start': "43'9700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3789", 'last_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'objects_scrubbed': 3741, 'log_size': 10058, 'log_dups_size': 0, 'ondisk_log_size': 10058, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1440, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 5283840, 'num_objects': 1290, 'num_object_clones': 0, 'num_object_copies': 2580, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4373, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1290, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16013, 'num_write_kb': 27124, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3081}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:07:56.439 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-10T16:07:56.439 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "47'19810", 'reported_seq': 19961, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:07:55.561240+0000', 'last_change': '2024-09-10T15:59:16.313409+0000', 'last_active': '2024-09-10T16:07:55.561240+0000', 'last_peered': '2024-09-10T16:07:55.561240+0000', 'last_clean': '2024-09-10T15:59:14.129780+0000', 'last_became_active': '2024-09-10T15:59:16.300932+0000', 'last_became_peered': '2024-09-10T15:59:16.300932+0000', 'last_unstale': '2024-09-10T16:07:55.561240+0000', 'last_undegraded': '2024-09-10T15:59:15.887823+0000', 'last_fullsized': '2024-09-10T15:59:15.768513+0000', 'mapping_epoch': 42, 'log_start': "43'9800", 'ondisk_log_start': "43'9800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3880", 'last_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'objects_scrubbed': 3827, 'log_size': 10010, 'log_dups_size': 0, 'ondisk_log_size': 10010, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1397, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 5686214, 'num_objects': 1390, 'num_object_clones': 0, 'num_object_copies': 2780, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4325, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1390, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15973, 'num_write_kb': 27072, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2933}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:07:56.439 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T16:07:57.133 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:07:57.133 INFO:teuthology.orchestra.run.smithi115.stdout:{"fsid":"7effee29-df05-4351-9741-ee203564ed81","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 39184/44880 objects degraded (87.308%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":641,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":47,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983954,"num_in_osds":3,"osd_in_since":1725983947,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":22440,"data_bytes":91865865,"bytes_used":1621184512,"bytes_avail":288289107968,"bytes_total":289910292480,"degraded_objects":39184,"degraded_total":44880,"degraded_ratio":0.87308377896613198,"recovering_objects_per_sec":27,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":329,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":529},"fsmap":{"epoch":1,"btime":"2024-09-10T15:57:14:967120+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T16:00:13.560776+0000","services":{}},"progress_events":{}} 2024-09-10T16:07:57.158 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-10T16:07:57.158 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:07:57.720 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:07:57.723 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:07:57.746 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":360,"stamp":"2024-09-10T16:07:55.747122+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":91865865,"num_objects":22440,"num_object_clones":0,"num_object_copies":44880,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39184,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22440,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":269670,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12207,"num_bytes_recovered":33095699,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160597,"ondisk_log_size":160597,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1583188,"kb_used_data":387208,"kb_used_omap":12,"kb_used_meta":1195891,"kb_avail":281532332,"statfs":{"total":289910292480,"available":288289107968,"internally_reserved":0,"allocated":396500992,"data_stored":395563567,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1224592386},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,1,0,3,4,7,7,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-26009600,"num_objects":-6350,"num_object_clones":0,"num_object_copies":-12700,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-329,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-6350,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6350,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":329,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002810"},"pg_stats":[{"pgid":"2.f","version":"47'19353","reported_seq":19434,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:55.677851+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:07:55.677851+0000","last_peered":"2024-09-10T16:07:55.677851+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:07:55.677851+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"43'9300","ondisk_log_start":"43'9300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5402624,"num_objects":1319,"num_object_clones":0,"num_object_copies":2638,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1319,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19353,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2940}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'19335","reported_seq":19454,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:07:50.925587+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:07:50.925587+0000","last_peered":"2024-09-10T16:07:50.925587+0000","last_clean":"2024-09-10T16:07:50.925587+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:07:50.925587+0000","last_undegraded":"2024-09-10T16:07:50.925587+0000","last_fullsized":"2024-09-10T16:07:50.925587+0000","mapping_epoch":35,"log_start":"43'9300","ondisk_log_start":"43'9300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6181830,"num_objects":1511,"num_object_clones":0,"num_object_copies":3022,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1511,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17440,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'19418","reported_seq":28102,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:07:54.538580+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:07:54.538580+0000","last_peered":"2024-09-10T16:07:54.538580+0000","last_clean":"2024-09-10T16:07:54.538580+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:07:54.538580+0000","last_undegraded":"2024-09-10T16:07:54.538580+0000","last_fullsized":"2024-09-10T16:07:54.538580+0000","mapping_epoch":44,"log_start":"43'9400","ondisk_log_start":"43'9400","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5648408,"num_objects":1380,"num_object_clones":0,"num_object_copies":2760,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1380,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17422,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'19208","reported_seq":19295,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:07:50.913405+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:07:50.913405+0000","last_peered":"2024-09-10T16:07:50.913405+0000","last_clean":"2024-09-10T16:07:50.913405+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:07:50.913405+0000","last_undegraded":"2024-09-10T16:07:50.913405+0000","last_fullsized":"2024-09-10T16:07:50.913405+0000","mapping_epoch":35,"log_start":"43'9200","ondisk_log_start":"43'9200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5947392,"num_objects":1452,"num_object_clones":0,"num_object_copies":2904,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1452,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19208,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'19542","reported_seq":19678,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:07:55.650822+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:07:55.650822+0000","last_peered":"2024-09-10T16:07:55.650822+0000","last_clean":"2024-09-10T16:07:55.650822+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:07:55.650822+0000","last_undegraded":"2024-09-10T16:07:55.650822+0000","last_fullsized":"2024-09-10T16:07:55.650822+0000","mapping_epoch":34,"log_start":"43'9500","ondisk_log_start":"43'9500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5525987,"num_objects":1350,"num_object_clones":0,"num_object_copies":2700,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1350,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16253,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'19683","reported_seq":19824,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:55.503652+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:07:55.503652+0000","last_peered":"2024-09-10T16:07:55.503652+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:07:55.503652+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"43'9600","ondisk_log_start":"43'9600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5644771,"num_objects":1379,"num_object_clones":0,"num_object_copies":2758,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1379,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16295,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2992},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'19714","reported_seq":19858,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:55.617648+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:07:55.617648+0000","last_peered":"2024-09-10T16:07:55.617648+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:07:55.617648+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"43'9700","ondisk_log_start":"43'9700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5604294,"num_objects":1370,"num_object_clones":0,"num_object_copies":2740,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1370,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16213,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2867},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'19944","reported_seq":20080,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:07:55.540396+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:07:55.540396+0000","last_peered":"2024-09-10T16:07:55.540396+0000","last_clean":"2024-09-10T16:07:55.540396+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:07:55.540396+0000","last_undegraded":"2024-09-10T16:07:55.540396+0000","last_fullsized":"2024-09-10T16:07:55.540396+0000","mapping_epoch":34,"log_start":"43'9900","ondisk_log_start":"43'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5611520,"num_objects":1370,"num_object_clones":0,"num_object_copies":2740,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1370,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16791,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.876943+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:05:19.876943+0000","last_peered":"2024-09-10T16:05:19.876943+0000","last_clean":"2024-09-10T16:05:19.876943+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:05:19.876943+0000","last_undegraded":"2024-09-10T16:05:19.876943+0000","last_fullsized":"2024-09-10T16:05:19.876943+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'18911","reported_seq":19057,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:50.912425+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:07:50.912425+0000","last_peered":"2024-09-10T16:07:50.912425+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:07:50.912425+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"43'8900","ondisk_log_start":"43'8900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6332899,"num_objects":1547,"num_object_clones":0,"num_object_copies":3094,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1547,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16730,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-2576}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":100,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.875849+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:05:19.875849+0000","last_peered":"2024-09-10T16:05:19.875849+0000","last_clean":"2024-09-10T16:05:19.875849+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:05:19.875849+0000","last_undegraded":"2024-09-10T16:05:19.875849+0000","last_fullsized":"2024-09-10T16:05:19.875849+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'19560","reported_seq":19692,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:55.641441+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:07:55.641441+0000","last_peered":"2024-09-10T16:07:55.641441+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:07:55.641441+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"43'9500","ondisk_log_start":"43'9500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5529600,"num_objects":1350,"num_object_clones":0,"num_object_copies":2700,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1350,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16064,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2905}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":110,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.872023+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:05:19.872023+0000","last_peered":"2024-09-10T16:05:19.872023+0000","last_clean":"2024-09-10T16:05:19.872023+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:05:19.872023+0000","last_undegraded":"2024-09-10T16:05:19.872023+0000","last_fullsized":"2024-09-10T16:05:19.872023+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'19313","reported_seq":19454,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:50.878314+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:07:50.878314+0000","last_peered":"2024-09-10T16:07:50.878314+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:07:50.878314+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"43'9300","ondisk_log_start":"43'9300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6013411,"num_objects":1469,"num_object_clones":0,"num_object_copies":2938,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1469,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16993,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-2835}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":86,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:32.574539+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:05:32.574539+0000","last_peered":"2024-09-10T16:05:32.574539+0000","last_clean":"2024-09-10T16:05:32.574539+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:05:32.574539+0000","last_undegraded":"2024-09-10T16:05:32.574539+0000","last_fullsized":"2024-09-10T16:05:32.574539+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'19670","reported_seq":27260,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:50.913235+0000","last_change":"2024-09-10T16:05:18.271152+0000","last_active":"2024-09-10T16:07:50.913235+0000","last_peered":"2024-09-10T16:07:50.913235+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:07:50.913235+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"43'9600","ondisk_log_start":"43'9600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6144000,"num_objects":1500,"num_object_clones":0,"num_object_copies":3000,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":627,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1500,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17224,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3727,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":873}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'19474","reported_seq":19615,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:55.639009+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:07:55.639009+0000","last_peered":"2024-09-10T16:07:55.639009+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:07:55.639009+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"43'9400","ondisk_log_start":"43'9400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5586944,"num_objects":1364,"num_object_clones":0,"num_object_copies":2728,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1364,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16105,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2940}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:32.574559+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:05:32.574559+0000","last_peered":"2024-09-10T16:05:32.574559+0000","last_clean":"2024-09-10T16:05:32.574559+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:05:32.574559+0000","last_undegraded":"2024-09-10T16:05:32.574559+0000","last_fullsized":"2024-09-10T16:05:32.574559+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'19201","reported_seq":27773,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:07:54.501151+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:07:54.501151+0000","last_peered":"2024-09-10T16:07:54.501151+0000","last_clean":"2024-09-10T16:07:54.501151+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:07:54.501151+0000","last_undegraded":"2024-09-10T16:07:54.501151+0000","last_fullsized":"2024-09-10T16:07:54.501151+0000","mapping_epoch":46,"log_start":"43'9200","ondisk_log_start":"43'9200","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5722112,"num_objects":1397,"num_object_clones":0,"num_object_copies":2794,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1397,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15591,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":100,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.876043+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:05:19.876043+0000","last_peered":"2024-09-10T16:05:19.876043+0000","last_clean":"2024-09-10T16:05:19.876043+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:05:19.876043+0000","last_undegraded":"2024-09-10T16:05:19.876043+0000","last_fullsized":"2024-09-10T16:05:19.876043+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'19758","reported_seq":19906,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:55.575904+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:07:55.575904+0000","last_peered":"2024-09-10T16:07:55.575904+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:07:55.575904+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"43'9700","ondisk_log_start":"43'9700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5283840,"num_objects":1290,"num_object_clones":0,"num_object_copies":2580,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1290,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16013,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3081}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:32.574572+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:05:32.574572+0000","last_peered":"2024-09-10T16:05:32.574572+0000","last_clean":"2024-09-10T16:05:32.574572+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:05:32.574572+0000","last_undegraded":"2024-09-10T16:05:32.574572+0000","last_fullsized":"2024-09-10T16:05:32.574572+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.877269+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:05:19.877269+0000","last_peered":"2024-09-10T16:05:19.877269+0000","last_clean":"2024-09-10T16:05:19.877269+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:05:19.877269+0000","last_undegraded":"2024-09-10T16:05:19.877269+0000","last_fullsized":"2024-09-10T16:05:19.877269+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'19810","reported_seq":19961,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:55.561240+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:07:55.561240+0000","last_peered":"2024-09-10T16:07:55.561240+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:07:55.561240+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"43'9800","ondisk_log_start":"43'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5686214,"num_objects":1390,"num_object_clones":0,"num_object_copies":2780,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1390,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15973,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2933},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":91865846,"num_objects":22438,"num_object_clones":0,"num_object_copies":44876,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39184,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22438,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":269668,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12205,"num_bytes_recovered":33095680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":394350592,"data_stored":394208767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160594,"ondisk_log_size":160594,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986294,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":542564,"kb_used_data":186004,"kb_used_omap":4,"kb_used_meta":356539,"kb_avail":93829276,"statfs":{"total":96636764160,"available":96081178624,"internally_reserved":0,"allocated":190468096,"data_stored":190165412,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":365096618},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,0,0,2,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659240,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":446124,"kb_used_data":67344,"kb_used_omap":4,"kb_used_meta":378747,"kb_avail":93925716,"statfs":{"total":96636764160,"available":96179933184,"internally_reserved":0,"allocated":68960256,"data_stored":68628204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":387837614},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,0,0,2,1,1,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182523,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":594500,"kb_used_data":133860,"kb_used_omap":4,"kb_used_meta":460603,"kb_avail":93777340,"statfs":{"total":96636764160,"available":96027996160,"internally_reserved":0,"allocated":137072640,"data_stored":136769951,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":471658154},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,3,4,4,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":136298496,"data_stored":136262366,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":68440064,"data_stored":68374571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":189612032,"data_stored":189571830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:08:00.749 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:08:01.283 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:08:01.284 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:08:01.310 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":362,"stamp":"2024-09-10T16:07:59.747971+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":87929609,"num_objects":21479,"num_object_clones":0,"num_object_copies":42958,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39009,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21479,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":270631,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12382,"num_bytes_recovered":33095699,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160858,"ondisk_log_size":160858,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1580072,"kb_used_data":375772,"kb_used_omap":12,"kb_used_meta":1204211,"kb_avail":281535448,"statfs":{"total":289910292480,"available":288292298752,"internally_reserved":0,"allocated":384790528,"data_stored":383853103,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1233112066},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,8,9,6,11,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-20307968,"num_objects":-4958,"num_object_clones":0,"num_object_copies":-9916,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-348,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4958,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4958,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":348,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002640"},"pg_stats":[{"pgid":"2.f","version":"47'19353","reported_seq":19434,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:55.677851+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:07:55.677851+0000","last_peered":"2024-09-10T16:07:55.677851+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:07:55.677851+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"43'9300","ondisk_log_start":"43'9300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5402624,"num_objects":1319,"num_object_clones":0,"num_object_copies":2638,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1319,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19353,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2940}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'19472","reported_seq":19591,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:07:55.849151+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:07:55.849151+0000","last_peered":"2024-09-10T16:07:55.849151+0000","last_clean":"2024-09-10T16:07:55.849151+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:07:55.849151+0000","last_undegraded":"2024-09-10T16:07:55.849151+0000","last_fullsized":"2024-09-10T16:07:55.849151+0000","mapping_epoch":35,"log_start":"43'9400","ondisk_log_start":"43'9400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5620678,"num_objects":1374,"num_object_clones":0,"num_object_copies":2748,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1374,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17577,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'19557","reported_seq":28241,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:07:59.551716+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:07:59.551716+0000","last_peered":"2024-09-10T16:07:59.551716+0000","last_clean":"2024-09-10T16:07:59.551716+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:07:59.551716+0000","last_undegraded":"2024-09-10T16:07:59.551716+0000","last_fullsized":"2024-09-10T16:07:59.551716+0000","mapping_epoch":44,"log_start":"43'9500","ondisk_log_start":"43'9500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5079064,"num_objects":1241,"num_object_clones":0,"num_object_copies":2482,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1241,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17561,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'19360","reported_seq":19447,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:07:55.904128+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:07:55.904128+0000","last_peered":"2024-09-10T16:07:55.904128+0000","last_clean":"2024-09-10T16:07:55.904128+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:07:55.904128+0000","last_undegraded":"2024-09-10T16:07:55.904128+0000","last_fullsized":"2024-09-10T16:07:55.904128+0000","mapping_epoch":35,"log_start":"43'9300","ondisk_log_start":"43'9300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5324800,"num_objects":1300,"num_object_clones":0,"num_object_copies":2600,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1300,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19360,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'19542","reported_seq":19678,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:07:55.650822+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:07:55.650822+0000","last_peered":"2024-09-10T16:07:55.650822+0000","last_clean":"2024-09-10T16:07:55.650822+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:07:55.650822+0000","last_undegraded":"2024-09-10T16:07:55.650822+0000","last_fullsized":"2024-09-10T16:07:55.650822+0000","mapping_epoch":34,"log_start":"43'9500","ondisk_log_start":"43'9500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5525987,"num_objects":1350,"num_object_clones":0,"num_object_copies":2700,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1350,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16253,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'19683","reported_seq":19824,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:55.503652+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:07:55.503652+0000","last_peered":"2024-09-10T16:07:55.503652+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:07:55.503652+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"43'9600","ondisk_log_start":"43'9600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5644771,"num_objects":1379,"num_object_clones":0,"num_object_copies":2758,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1379,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16295,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2992},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'19714","reported_seq":19858,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:55.617648+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:07:55.617648+0000","last_peered":"2024-09-10T16:07:55.617648+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:07:55.617648+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"43'9700","ondisk_log_start":"43'9700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5604294,"num_objects":1370,"num_object_clones":0,"num_object_copies":2740,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1370,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16213,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2867},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'19944","reported_seq":20080,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:07:55.540396+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:07:55.540396+0000","last_peered":"2024-09-10T16:07:55.540396+0000","last_clean":"2024-09-10T16:07:55.540396+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:07:55.540396+0000","last_undegraded":"2024-09-10T16:07:55.540396+0000","last_fullsized":"2024-09-10T16:07:55.540396+0000","mapping_epoch":34,"log_start":"43'9900","ondisk_log_start":"43'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5611520,"num_objects":1370,"num_object_clones":0,"num_object_copies":2740,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1370,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16791,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.876943+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:05:19.876943+0000","last_peered":"2024-09-10T16:05:19.876943+0000","last_clean":"2024-09-10T16:05:19.876943+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:05:19.876943+0000","last_undegraded":"2024-09-10T16:05:19.876943+0000","last_fullsized":"2024-09-10T16:05:19.876943+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'19058","reported_seq":19204,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:55.870136+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:07:55.870136+0000","last_peered":"2024-09-10T16:07:55.870136+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:07:55.870136+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"43'9000","ondisk_log_start":"43'9000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5730787,"num_objects":1400,"num_object_clones":0,"num_object_copies":2800,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1400,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16877,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-2723}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":100,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.875849+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:05:19.875849+0000","last_peered":"2024-09-10T16:05:19.875849+0000","last_clean":"2024-09-10T16:05:19.875849+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:05:19.875849+0000","last_undegraded":"2024-09-10T16:05:19.875849+0000","last_fullsized":"2024-09-10T16:05:19.875849+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'19560","reported_seq":19692,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:55.641441+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:07:55.641441+0000","last_peered":"2024-09-10T16:07:55.641441+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:07:55.641441+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"43'9500","ondisk_log_start":"43'9500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5529600,"num_objects":1350,"num_object_clones":0,"num_object_copies":2700,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1350,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16064,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2905}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":110,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.872023+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:05:19.872023+0000","last_peered":"2024-09-10T16:05:19.872023+0000","last_clean":"2024-09-10T16:05:19.872023+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:05:19.872023+0000","last_undegraded":"2024-09-10T16:05:19.872023+0000","last_fullsized":"2024-09-10T16:05:19.872023+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'19444","reported_seq":19585,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:55.870097+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:07:55.870097+0000","last_peered":"2024-09-10T16:07:55.870097+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:07:55.870097+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"43'9400","ondisk_log_start":"43'9400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5476835,"num_objects":1338,"num_object_clones":0,"num_object_copies":2676,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1338,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17124,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-2966}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":86,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:32.574539+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:05:32.574539+0000","last_peered":"2024-09-10T16:05:32.574539+0000","last_clean":"2024-09-10T16:05:32.574539+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:05:32.574539+0000","last_undegraded":"2024-09-10T16:05:32.574539+0000","last_fullsized":"2024-09-10T16:05:32.574539+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'19792","reported_seq":27732,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:55.915934+0000","last_change":"2024-09-10T16:05:18.271152+0000","last_active":"2024-09-10T16:07:55.915934+0000","last_peered":"2024-09-10T16:07:55.915934+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:07:55.915934+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"43'9700","ondisk_log_start":"43'9700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5644288,"num_objects":1378,"num_object_clones":0,"num_object_copies":2756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":452,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1378,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17346,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3902,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":926}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'19474","reported_seq":19615,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:55.639009+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:07:55.639009+0000","last_peered":"2024-09-10T16:07:55.639009+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:07:55.639009+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"43'9400","ondisk_log_start":"43'9400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5586944,"num_objects":1364,"num_object_clones":0,"num_object_copies":2728,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1364,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16105,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2940}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:32.574559+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:05:32.574559+0000","last_peered":"2024-09-10T16:05:32.574559+0000","last_clean":"2024-09-10T16:05:32.574559+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:05:32.574559+0000","last_undegraded":"2024-09-10T16:05:32.574559+0000","last_fullsized":"2024-09-10T16:05:32.574559+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'19334","reported_seq":27906,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:07:59.556248+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:07:59.556248+0000","last_peered":"2024-09-10T16:07:59.556248+0000","last_clean":"2024-09-10T16:07:59.556248+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:07:59.556248+0000","last_undegraded":"2024-09-10T16:07:59.556248+0000","last_fullsized":"2024-09-10T16:07:59.556248+0000","mapping_epoch":46,"log_start":"43'9300","ondisk_log_start":"43'9300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5177344,"num_objects":1264,"num_object_clones":0,"num_object_copies":2528,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1264,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15724,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":100,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.876043+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:05:19.876043+0000","last_peered":"2024-09-10T16:05:19.876043+0000","last_clean":"2024-09-10T16:05:19.876043+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:05:19.876043+0000","last_undegraded":"2024-09-10T16:05:19.876043+0000","last_fullsized":"2024-09-10T16:05:19.876043+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'19758","reported_seq":19906,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:55.575904+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:07:55.575904+0000","last_peered":"2024-09-10T16:07:55.575904+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:07:55.575904+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"43'9700","ondisk_log_start":"43'9700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5283840,"num_objects":1290,"num_object_clones":0,"num_object_copies":2580,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1290,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16013,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3081}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:32.574572+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:05:32.574572+0000","last_peered":"2024-09-10T16:05:32.574572+0000","last_clean":"2024-09-10T16:05:32.574572+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:05:32.574572+0000","last_undegraded":"2024-09-10T16:05:32.574572+0000","last_fullsized":"2024-09-10T16:05:32.574572+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":81,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:47.753676+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:05:47.753676+0000","last_peered":"2024-09-10T16:05:47.753676+0000","last_clean":"2024-09-10T16:05:47.753676+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:05:47.753676+0000","last_undegraded":"2024-09-10T16:05:47.753676+0000","last_fullsized":"2024-09-10T16:05:47.753676+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'19810","reported_seq":19961,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:07:55.561240+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:07:55.561240+0000","last_peered":"2024-09-10T16:07:55.561240+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:07:55.561240+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"43'9800","ondisk_log_start":"43'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5686214,"num_objects":1390,"num_object_clones":0,"num_object_copies":2780,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1390,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15973,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2933},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":87929590,"num_objects":21477,"num_object_clones":0,"num_object_copies":42954,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39009,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21477,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":270629,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12380,"num_bytes_recovered":33095680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":382468096,"data_stored":382326271,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160855,"ondisk_log_size":160855,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986295,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":542140,"kb_used_data":181420,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93829700,"statfs":{"total":96636764160,"available":96081612800,"internally_reserved":0,"allocated":185774080,"data_stored":185471396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369356458},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,3,1,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659241,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":443432,"kb_used_data":60492,"kb_used_omap":4,"kb_used_meta":382907,"kb_avail":93928408,"statfs":{"total":96636764160,"available":96182689792,"internally_reserved":0,"allocated":61943808,"data_stored":61611756,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":392097454},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,5,3,1,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182523,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":594500,"kb_used_data":133860,"kb_used_omap":4,"kb_used_meta":460603,"kb_avail":93777340,"statfs":{"total":96636764160,"available":96027996160,"internally_reserved":0,"allocated":137072640,"data_stored":136769951,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":471658154},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,3,4,4,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":136298496,"data_stored":136262366,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61251584,"data_stored":61186091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":184918016,"data_stored":184877814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:08:01.311 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:08:01.844 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:08:01.848 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:08:01.866 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":363,"stamp":"2024-09-10T16:08:01.748407+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":79303433,"num_objects":19373,"num_object_clones":0,"num_object_copies":38746,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38846,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19373,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":272737,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12545,"num_bytes_recovered":33095699,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160664,"ondisk_log_size":160664,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1576688,"kb_used_data":364080,"kb_used_omap":12,"kb_used_meta":1212531,"kb_avail":281538832,"statfs":{"total":289910292480,"available":288295763968,"internally_reserved":0,"allocated":372817920,"data_stored":371880495,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1241631746},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[5,2,3,7,9,4,11,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-27746304,"num_objects":-6774,"num_object_clones":0,"num_object_copies":-13548,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-511,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-6774,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6774,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":511,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002624"},"pg_stats":[{"pgid":"2.f","version":"47'19505","reported_seq":19584,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:00.779227+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:08:00.779227+0000","last_peered":"2024-09-10T16:08:00.779227+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:08:00.779227+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"43'9500","ondisk_log_start":"43'9500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4780032,"num_objects":1167,"num_object_clones":0,"num_object_copies":2334,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1167,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19505,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3092}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'19624","reported_seq":19743,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:08:00.835597+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:08:00.835597+0000","last_peered":"2024-09-10T16:08:00.835597+0000","last_clean":"2024-09-10T16:08:00.835597+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:08:00.835597+0000","last_undegraded":"2024-09-10T16:08:00.835597+0000","last_fullsized":"2024-09-10T16:08:00.835597+0000","mapping_epoch":35,"log_start":"43'9600","ondisk_log_start":"43'9600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4998086,"num_objects":1222,"num_object_clones":0,"num_object_copies":2444,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1222,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17729,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'19557","reported_seq":28241,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:07:59.551716+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:07:59.551716+0000","last_peered":"2024-09-10T16:07:59.551716+0000","last_clean":"2024-09-10T16:07:59.551716+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:07:59.551716+0000","last_undegraded":"2024-09-10T16:07:59.551716+0000","last_fullsized":"2024-09-10T16:07:59.551716+0000","mapping_epoch":44,"log_start":"43'9500","ondisk_log_start":"43'9500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5079064,"num_objects":1241,"num_object_clones":0,"num_object_copies":2482,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1241,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17561,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'19511","reported_seq":19598,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:08:00.878101+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:08:00.878101+0000","last_peered":"2024-09-10T16:08:00.878101+0000","last_clean":"2024-09-10T16:08:00.878101+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:08:00.878101+0000","last_undegraded":"2024-09-10T16:08:00.878101+0000","last_fullsized":"2024-09-10T16:08:00.878101+0000","mapping_epoch":35,"log_start":"43'9500","ondisk_log_start":"43'9500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4706304,"num_objects":1149,"num_object_clones":0,"num_object_copies":2298,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1149,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19511,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'19702","reported_seq":19838,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:08:00.754516+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:08:00.754516+0000","last_peered":"2024-09-10T16:08:00.754516+0000","last_clean":"2024-09-10T16:08:00.754516+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:08:00.754516+0000","last_undegraded":"2024-09-10T16:08:00.754516+0000","last_fullsized":"2024-09-10T16:08:00.754516+0000","mapping_epoch":34,"log_start":"43'9700","ondisk_log_start":"43'9700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4870627,"num_objects":1190,"num_object_clones":0,"num_object_copies":2380,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1190,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16413,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'19847","reported_seq":19991,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:00.693120+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:08:00.693120+0000","last_peered":"2024-09-10T16:08:00.693120+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:08:00.693120+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"43'9800","ondisk_log_start":"43'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4973027,"num_objects":1215,"num_object_clones":0,"num_object_copies":2430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1215,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16459,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3156},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'19862","reported_seq":20005,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:00.742997+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:08:00.742997+0000","last_peered":"2024-09-10T16:08:00.742997+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:08:00.742997+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"43'9800","ondisk_log_start":"43'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4998086,"num_objects":1222,"num_object_clones":0,"num_object_copies":2444,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1222,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16361,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3015},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'20088","reported_seq":20224,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:08:00.729636+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:08:00.729636+0000","last_peered":"2024-09-10T16:08:00.729636+0000","last_clean":"2024-09-10T16:08:00.729636+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:08:00.729636+0000","last_undegraded":"2024-09-10T16:08:00.729636+0000","last_fullsized":"2024-09-10T16:08:00.729636+0000","mapping_epoch":34,"log_start":"44'10000","ondisk_log_start":"44'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5021696,"num_objects":1226,"num_object_clones":0,"num_object_copies":2452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1226,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16935,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.876943+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:05:19.876943+0000","last_peered":"2024-09-10T16:05:19.876943+0000","last_clean":"2024-09-10T16:05:19.876943+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:05:19.876943+0000","last_undegraded":"2024-09-10T16:05:19.876943+0000","last_fullsized":"2024-09-10T16:05:19.876943+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'19213","reported_seq":19359,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:00.918154+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:08:00.918154+0000","last_peered":"2024-09-10T16:08:00.918154+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:08:00.918154+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"43'9200","ondisk_log_start":"43'9200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5095907,"num_objects":1245,"num_object_clones":0,"num_object_copies":2490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1245,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17032,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-2878}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":100,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.875849+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:05:19.875849+0000","last_peered":"2024-09-10T16:05:19.875849+0000","last_clean":"2024-09-10T16:05:19.875849+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:05:19.875849+0000","last_undegraded":"2024-09-10T16:05:19.875849+0000","last_fullsized":"2024-09-10T16:05:19.875849+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'19693","reported_seq":19830,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:00.480004+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:08:00.480004+0000","last_peered":"2024-09-10T16:08:00.480004+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:08:00.480004+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"43'9600","ondisk_log_start":"43'9600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4984832,"num_objects":1217,"num_object_clones":0,"num_object_copies":2434,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1217,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16197,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3038}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":110,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.872023+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:05:19.872023+0000","last_peered":"2024-09-10T16:05:19.872023+0000","last_clean":"2024-09-10T16:05:19.872023+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:05:19.872023+0000","last_undegraded":"2024-09-10T16:05:19.872023+0000","last_fullsized":"2024-09-10T16:05:19.872023+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'19605","reported_seq":19746,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:00.907052+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:08:00.907052+0000","last_peered":"2024-09-10T16:08:00.907052+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:08:00.907052+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"43'9600","ondisk_log_start":"43'9600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4817379,"num_objects":1177,"num_object_clones":0,"num_object_copies":2354,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1177,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17285,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-3127}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":86,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:32.574539+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:05:32.574539+0000","last_peered":"2024-09-10T16:05:32.574539+0000","last_clean":"2024-09-10T16:05:32.574539+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:05:32.574539+0000","last_undegraded":"2024-09-10T16:05:32.574539+0000","last_fullsized":"2024-09-10T16:05:32.574539+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'19940","reported_seq":28206,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:00.922598+0000","last_change":"2024-09-10T16:05:18.271152+0000","last_active":"2024-09-10T16:08:00.922598+0000","last_peered":"2024-09-10T16:08:00.922598+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:08:00.922598+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"43'9900","ondisk_log_start":"43'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5038080,"num_objects":1230,"num_object_clones":0,"num_object_copies":2460,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":289,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1230,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17494,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4065,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":941}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'19617","reported_seq":19758,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:00.759770+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:08:00.759770+0000","last_peered":"2024-09-10T16:08:00.759770+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:08:00.759770+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"43'9600","ondisk_log_start":"43'9600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5001216,"num_objects":1221,"num_object_clones":0,"num_object_copies":2442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1221,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16248,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3083}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:32.574559+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:05:32.574559+0000","last_peered":"2024-09-10T16:05:32.574559+0000","last_clean":"2024-09-10T16:05:32.574559+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:05:32.574559+0000","last_undegraded":"2024-09-10T16:05:32.574559+0000","last_fullsized":"2024-09-10T16:05:32.574559+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'19334","reported_seq":27906,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:07:59.556248+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:07:59.556248+0000","last_peered":"2024-09-10T16:07:59.556248+0000","last_clean":"2024-09-10T16:07:59.556248+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:07:59.556248+0000","last_undegraded":"2024-09-10T16:07:59.556248+0000","last_fullsized":"2024-09-10T16:07:59.556248+0000","mapping_epoch":46,"log_start":"43'9300","ondisk_log_start":"43'9300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5177344,"num_objects":1264,"num_object_clones":0,"num_object_copies":2528,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1264,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15724,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":100,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.876043+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:05:19.876043+0000","last_peered":"2024-09-10T16:05:19.876043+0000","last_clean":"2024-09-10T16:05:19.876043+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:05:19.876043+0000","last_undegraded":"2024-09-10T16:05:19.876043+0000","last_fullsized":"2024-09-10T16:05:19.876043+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'19898","reported_seq":20046,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:00.709848+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:08:00.709848+0000","last_peered":"2024-09-10T16:08:00.709848+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:08:00.709848+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"43'9800","ondisk_log_start":"43'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4710400,"num_objects":1150,"num_object_clones":0,"num_object_copies":2300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1150,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16153,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3221}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:32.574572+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:05:32.574572+0000","last_peered":"2024-09-10T16:05:32.574572+0000","last_clean":"2024-09-10T16:05:32.574572+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:05:32.574572+0000","last_undegraded":"2024-09-10T16:05:32.574572+0000","last_fullsized":"2024-09-10T16:05:32.574572+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":81,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:47.753676+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:05:47.753676+0000","last_peered":"2024-09-10T16:05:47.753676+0000","last_clean":"2024-09-10T16:05:47.753676+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:05:47.753676+0000","last_undegraded":"2024-09-10T16:05:47.753676+0000","last_fullsized":"2024-09-10T16:05:47.753676+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'19965","reported_seq":20116,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:00.695389+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:08:00.695389+0000","last_peered":"2024-09-10T16:08:00.695389+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:08:00.695389+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"44'9900","ondisk_log_start":"44'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5051334,"num_objects":1235,"num_object_clones":0,"num_object_copies":2470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1235,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16128,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3088},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":79303414,"num_objects":19371,"num_object_clones":0,"num_object_copies":38742,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38846,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19371,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":272735,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12543,"num_bytes_recovered":33095680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":370180096,"data_stored":370038271,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160661,"ondisk_log_size":160661,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986296,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":541688,"kb_used_data":176808,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93830152,"statfs":{"total":96636764160,"available":96082075648,"internally_reserved":0,"allocated":181051392,"data_stored":180748708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,3,2,1,1,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659241,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":443432,"kb_used_data":60492,"kb_used_omap":4,"kb_used_meta":382907,"kb_avail":93928408,"statfs":{"total":96636764160,"available":96182689792,"internally_reserved":0,"allocated":61943808,"data_stored":61611756,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":392097454},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,5,3,1,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182524,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":591568,"kb_used_data":126780,"kb_used_omap":4,"kb_used_meta":464763,"kb_avail":93780272,"statfs":{"total":96636764160,"available":96030998528,"internally_reserved":0,"allocated":129822720,"data_stored":129520031,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":475917994},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[5,1,0,0,5,2,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":128933888,"data_stored":128897758,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61251584,"data_stored":61186091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":179994624,"data_stored":179954422,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:08:01.867 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-10T16:08:01.867 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "47'19505", 'reported_seq': 19584, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:08:00.779227+0000', 'last_change': '2024-09-10T15:59:16.281558+0000', 'last_active': '2024-09-10T16:08:00.779227+0000', 'last_peered': '2024-09-10T16:08:00.779227+0000', 'last_clean': '2024-09-10T15:59:14.132456+0000', 'last_became_active': '2024-09-10T15:59:16.274407+0000', 'last_became_peered': '2024-09-10T15:59:16.274407+0000', 'last_unstale': '2024-09-10T16:08:00.779227+0000', 'last_undegraded': '2024-09-10T15:59:15.851279+0000', 'last_fullsized': '2024-09-10T15:59:15.774356+0000', 'mapping_epoch': 42, 'log_start': "43'9500", 'ondisk_log_start': "43'9500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'objects_scrubbed': 0, 'log_size': 10005, 'log_dups_size': 0, 'ondisk_log_size': 10005, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 4780032, 'num_objects': 1167, 'num_object_clones': 0, 'num_object_copies': 2334, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4259, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1167, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 19505, 'num_write_kb': 41344, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3092}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:08:01.868 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-10T16:08:01.868 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "47'19847", 'reported_seq': 19991, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:08:00.693120+0000', 'last_change': '2024-09-10T15:59:16.308264+0000', 'last_active': '2024-09-10T16:08:00.693120+0000', 'last_peered': '2024-09-10T16:08:00.693120+0000', 'last_clean': '2024-09-10T15:59:14.111551+0000', 'last_became_active': '2024-09-10T15:59:16.300040+0000', 'last_became_peered': '2024-09-10T15:59:16.300040+0000', 'last_unstale': '2024-09-10T16:08:00.693120+0000', 'last_undegraded': '2024-09-10T15:59:15.878234+0000', 'last_fullsized': '2024-09-10T15:59:15.769328+0000', 'mapping_epoch': 42, 'log_start': "43'9800", 'ondisk_log_start': "43'9800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3442", 'last_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'objects_scrubbed': 3384, 'log_size': 10047, 'log_dups_size': 0, 'ondisk_log_size': 10047, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1292, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 4973027, 'num_objects': 1215, 'num_object_clones': 0, 'num_object_copies': 2430, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4372, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1215, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 16459, 'num_write_kb': 28580, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3156}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:08:01.868 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-10T16:08:01.868 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "47'19862", 'reported_seq': 20005, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:08:00.742997+0000', 'last_change': '2024-09-10T15:59:16.331899+0000', 'last_active': '2024-09-10T16:08:00.742997+0000', 'last_peered': '2024-09-10T16:08:00.742997+0000', 'last_clean': '2024-09-10T15:59:14.139436+0000', 'last_became_active': '2024-09-10T15:59:16.318715+0000', 'last_became_peered': '2024-09-10T15:59:16.318715+0000', 'last_unstale': '2024-09-10T16:08:00.742997+0000', 'last_undegraded': '2024-09-10T15:59:15.877524+0000', 'last_fullsized': '2024-09-10T15:59:15.769042+0000', 'mapping_epoch': 42, 'log_start': "43'9800", 'ondisk_log_start': "43'9800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3537", 'last_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'objects_scrubbed': 3497, 'log_size': 10062, 'log_dups_size': 0, 'ondisk_log_size': 10062, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1224, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 4998086, 'num_objects': 1222, 'num_object_clones': 0, 'num_object_copies': 2444, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4237, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1222, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 16361, 'num_write_kb': 28172, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3015}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:08:01.868 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-10T16:08:01.868 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "47'19213", 'reported_seq': 19359, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:08:00.918154+0000', 'last_change': '2024-09-10T15:59:16.175680+0000', 'last_active': '2024-09-10T16:08:00.918154+0000', 'last_peered': '2024-09-10T16:08:00.918154+0000', 'last_clean': '2024-09-10T15:59:14.088187+0000', 'last_became_active': '2024-09-10T15:59:16.165810+0000', 'last_became_peered': '2024-09-10T15:59:16.165810+0000', 'last_unstale': '2024-09-10T16:08:00.918154+0000', 'last_undegraded': '2024-09-10T15:59:15.822098+0000', 'last_fullsized': '2024-09-10T15:59:15.769123+0000', 'mapping_epoch': 42, 'log_start': "43'9200", 'ondisk_log_start': "43'9200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2218", 'last_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'objects_scrubbed': 2175, 'log_size': 10013, 'log_dups_size': 0, 'ondisk_log_size': 10013, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 825, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 5095907, 'num_objects': 1245, 'num_object_clones': 0, 'num_object_copies': 2490, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4124, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1245, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 17032, 'num_write_kb': 32204, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': -2878}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:08:01.868 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-10T16:08:01.868 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "47'19693", 'reported_seq': 19830, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:08:00.480004+0000', 'last_change': '2024-09-10T15:59:16.277742+0000', 'last_active': '2024-09-10T16:08:00.480004+0000', 'last_peered': '2024-09-10T16:08:00.480004+0000', 'last_clean': '2024-09-10T15:59:14.129081+0000', 'last_became_active': '2024-09-10T15:59:16.269835+0000', 'last_became_peered': '2024-09-10T15:59:16.269835+0000', 'last_unstale': '2024-09-10T16:08:00.480004+0000', 'last_undegraded': '2024-09-10T15:59:15.860857+0000', 'last_fullsized': '2024-09-10T15:59:15.771410+0000', 'mapping_epoch': 42, 'log_start': "43'9600", 'ondisk_log_start': "43'9600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3557", 'last_scrub_stamp': '2024-09-10T15:59:02.493678+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:02.493678+0000', 'objects_scrubbed': 3488, 'log_size': 10093, 'log_dups_size': 0, 'ondisk_log_size': 10093, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1472, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 4984832, 'num_objects': 1217, 'num_object_clones': 0, 'num_object_copies': 2434, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4257, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1217, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16197, 'num_write_kb': 27852, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3038}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:08:01.868 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-10T16:08:01.868 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "47'19605", 'reported_seq': 19746, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:08:00.907052+0000', 'last_change': '2024-09-10T15:59:16.284311+0000', 'last_active': '2024-09-10T16:08:00.907052+0000', 'last_peered': '2024-09-10T16:08:00.907052+0000', 'last_clean': '2024-09-10T15:59:14.116041+0000', 'last_became_active': '2024-09-10T15:59:16.274543+0000', 'last_became_peered': '2024-09-10T15:59:16.274543+0000', 'last_unstale': '2024-09-10T16:08:00.907052+0000', 'last_undegraded': '2024-09-10T15:59:15.812484+0000', 'last_fullsized': '2024-09-10T15:59:15.770885+0000', 'mapping_epoch': 42, 'log_start': "43'9600", 'ondisk_log_start': "43'9600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2356", 'last_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'objects_scrubbed': 2316, 'log_size': 10005, 'log_dups_size': 0, 'ondisk_log_size': 10005, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 783, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 4817379, 'num_objects': 1177, 'num_object_clones': 0, 'num_object_copies': 2354, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1177, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 17285, 'num_write_kb': 32292, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': -3127}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:08:01.869 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-10T16:08:01.869 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "47'19940", 'reported_seq': 28206, 'reported_epoch': 47, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:08:00.922598+0000', 'last_change': '2024-09-10T16:05:18.271152+0000', 'last_active': '2024-09-10T16:08:00.922598+0000', 'last_peered': '2024-09-10T16:08:00.922598+0000', 'last_clean': '2024-09-10T15:59:14.110654+0000', 'last_became_active': '2024-09-10T15:59:16.164645+0000', 'last_became_peered': '2024-09-10T15:59:16.164645+0000', 'last_unstale': '2024-09-10T16:08:00.922598+0000', 'last_undegraded': '2024-09-10T15:59:15.803593+0000', 'last_fullsized': '2024-09-10T15:59:15.769083+0000', 'mapping_epoch': 42, 'log_start': "43'9900", 'ondisk_log_start': "43'9900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2486", 'last_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'objects_scrubbed': 2440, 'log_size': 10040, 'log_dups_size': 0, 'ondisk_log_size': 10040, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 792, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 5038080, 'num_objects': 1230, 'num_object_clones': 0, 'num_object_copies': 2460, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 289, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1230, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17494, 'num_write_kb': 32568, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 4065, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 941}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:08:01.869 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-10T16:08:01.869 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "47'19617", 'reported_seq': 19758, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:08:00.759770+0000', 'last_change': '2024-09-10T15:59:16.315641+0000', 'last_active': '2024-09-10T16:08:00.759770+0000', 'last_peered': '2024-09-10T16:08:00.759770+0000', 'last_clean': '2024-09-10T15:59:14.117001+0000', 'last_became_active': '2024-09-10T15:59:16.302245+0000', 'last_became_peered': '2024-09-10T15:59:16.302245+0000', 'last_unstale': '2024-09-10T16:08:00.759770+0000', 'last_undegraded': '2024-09-10T15:59:15.845012+0000', 'last_fullsized': '2024-09-10T15:59:15.770337+0000', 'mapping_epoch': 42, 'log_start': "43'9600", 'ondisk_log_start': "43'9600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3404", 'last_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'objects_scrubbed': 3367, 'log_size': 10017, 'log_dups_size': 0, 'ondisk_log_size': 10017, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 983, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 5001216, 'num_objects': 1221, 'num_object_clones': 0, 'num_object_copies': 2442, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1221, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16248, 'num_write_kb': 28204, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3083}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:08:01.869 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-10T16:08:01.869 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "47'19898", 'reported_seq': 20046, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:08:00.709848+0000', 'last_change': '2024-09-10T15:59:16.311291+0000', 'last_active': '2024-09-10T16:08:00.709848+0000', 'last_peered': '2024-09-10T16:08:00.709848+0000', 'last_clean': '2024-09-10T15:59:14.116363+0000', 'last_became_active': '2024-09-10T15:59:16.301244+0000', 'last_became_peered': '2024-09-10T15:59:16.301244+0000', 'last_unstale': '2024-09-10T16:08:00.709848+0000', 'last_undegraded': '2024-09-10T15:59:15.883434+0000', 'last_fullsized': '2024-09-10T15:59:15.772756+0000', 'mapping_epoch': 42, 'log_start': "43'9800", 'ondisk_log_start': "43'9800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3789", 'last_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'objects_scrubbed': 3741, 'log_size': 10098, 'log_dups_size': 0, 'ondisk_log_size': 10098, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1440, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 4710400, 'num_objects': 1150, 'num_object_clones': 0, 'num_object_copies': 2300, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4373, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1150, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16153, 'num_write_kb': 27124, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3221}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:08:01.869 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-10T16:08:01.869 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "47'19965", 'reported_seq': 20116, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:08:00.695389+0000', 'last_change': '2024-09-10T15:59:16.313409+0000', 'last_active': '2024-09-10T16:08:00.695389+0000', 'last_peered': '2024-09-10T16:08:00.695389+0000', 'last_clean': '2024-09-10T15:59:14.129780+0000', 'last_became_active': '2024-09-10T15:59:16.300932+0000', 'last_became_peered': '2024-09-10T15:59:16.300932+0000', 'last_unstale': '2024-09-10T16:08:00.695389+0000', 'last_undegraded': '2024-09-10T15:59:15.887823+0000', 'last_fullsized': '2024-09-10T15:59:15.768513+0000', 'mapping_epoch': 42, 'log_start': "44'9900", 'ondisk_log_start': "44'9900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3880", 'last_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'objects_scrubbed': 3827, 'log_size': 10065, 'log_dups_size': 0, 'ondisk_log_size': 10065, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1397, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 5051334, 'num_objects': 1235, 'num_object_clones': 0, 'num_object_copies': 2470, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4325, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1235, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16128, 'num_write_kb': 27072, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3088}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:08:01.870 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T16:08:02.578 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:08:02.578 INFO:teuthology.orchestra.run.smithi115.stdout:{"fsid":"7effee29-df05-4351-9741-ee203564ed81","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 38846/38746 objects degraded (100.258%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":646,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":47,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983954,"num_in_osds":3,"osd_in_since":1725983947,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":19373,"data_bytes":79303433,"bytes_used":1614528512,"bytes_avail":288295763968,"bytes_total":289910292480,"degraded_objects":38846,"degraded_total":38746,"degraded_ratio":1.0025809115779694,"recovering_objects_per_sec":42,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":511,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":564},"fsmap":{"epoch":1,"btime":"2024-09-10T15:57:14:967120+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T16:00:13.560776+0000","services":{}},"progress_events":{}} 2024-09-10T16:08:02.601 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-10T16:08:02.601 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:08:03.163 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:08:03.165 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:08:03.193 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":363,"stamp":"2024-09-10T16:08:01.748407+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":79303433,"num_objects":19373,"num_object_clones":0,"num_object_copies":38746,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38846,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19373,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":272737,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12545,"num_bytes_recovered":33095699,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160664,"ondisk_log_size":160664,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1576688,"kb_used_data":364080,"kb_used_omap":12,"kb_used_meta":1212531,"kb_avail":281538832,"statfs":{"total":289910292480,"available":288295763968,"internally_reserved":0,"allocated":372817920,"data_stored":371880495,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1241631746},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[5,2,3,7,9,4,11,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-27746304,"num_objects":-6774,"num_object_clones":0,"num_object_copies":-13548,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-511,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-6774,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6774,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":511,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002624"},"pg_stats":[{"pgid":"2.f","version":"47'19505","reported_seq":19584,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:00.779227+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:08:00.779227+0000","last_peered":"2024-09-10T16:08:00.779227+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:08:00.779227+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"43'9500","ondisk_log_start":"43'9500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4780032,"num_objects":1167,"num_object_clones":0,"num_object_copies":2334,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1167,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19505,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3092}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'19624","reported_seq":19743,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:08:00.835597+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:08:00.835597+0000","last_peered":"2024-09-10T16:08:00.835597+0000","last_clean":"2024-09-10T16:08:00.835597+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:08:00.835597+0000","last_undegraded":"2024-09-10T16:08:00.835597+0000","last_fullsized":"2024-09-10T16:08:00.835597+0000","mapping_epoch":35,"log_start":"43'9600","ondisk_log_start":"43'9600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4998086,"num_objects":1222,"num_object_clones":0,"num_object_copies":2444,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1222,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17729,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'19557","reported_seq":28241,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:07:59.551716+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:07:59.551716+0000","last_peered":"2024-09-10T16:07:59.551716+0000","last_clean":"2024-09-10T16:07:59.551716+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:07:59.551716+0000","last_undegraded":"2024-09-10T16:07:59.551716+0000","last_fullsized":"2024-09-10T16:07:59.551716+0000","mapping_epoch":44,"log_start":"43'9500","ondisk_log_start":"43'9500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5079064,"num_objects":1241,"num_object_clones":0,"num_object_copies":2482,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1241,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17561,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'19511","reported_seq":19598,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:08:00.878101+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:08:00.878101+0000","last_peered":"2024-09-10T16:08:00.878101+0000","last_clean":"2024-09-10T16:08:00.878101+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:08:00.878101+0000","last_undegraded":"2024-09-10T16:08:00.878101+0000","last_fullsized":"2024-09-10T16:08:00.878101+0000","mapping_epoch":35,"log_start":"43'9500","ondisk_log_start":"43'9500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4706304,"num_objects":1149,"num_object_clones":0,"num_object_copies":2298,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1149,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19511,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'19702","reported_seq":19838,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:08:00.754516+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:08:00.754516+0000","last_peered":"2024-09-10T16:08:00.754516+0000","last_clean":"2024-09-10T16:08:00.754516+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:08:00.754516+0000","last_undegraded":"2024-09-10T16:08:00.754516+0000","last_fullsized":"2024-09-10T16:08:00.754516+0000","mapping_epoch":34,"log_start":"43'9700","ondisk_log_start":"43'9700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4870627,"num_objects":1190,"num_object_clones":0,"num_object_copies":2380,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1190,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16413,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'19847","reported_seq":19991,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:00.693120+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:08:00.693120+0000","last_peered":"2024-09-10T16:08:00.693120+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:08:00.693120+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"43'9800","ondisk_log_start":"43'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4973027,"num_objects":1215,"num_object_clones":0,"num_object_copies":2430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1215,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16459,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3156},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'19862","reported_seq":20005,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:00.742997+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:08:00.742997+0000","last_peered":"2024-09-10T16:08:00.742997+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:08:00.742997+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"43'9800","ondisk_log_start":"43'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4998086,"num_objects":1222,"num_object_clones":0,"num_object_copies":2444,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1222,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16361,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3015},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'20088","reported_seq":20224,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:08:00.729636+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:08:00.729636+0000","last_peered":"2024-09-10T16:08:00.729636+0000","last_clean":"2024-09-10T16:08:00.729636+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:08:00.729636+0000","last_undegraded":"2024-09-10T16:08:00.729636+0000","last_fullsized":"2024-09-10T16:08:00.729636+0000","mapping_epoch":34,"log_start":"44'10000","ondisk_log_start":"44'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5021696,"num_objects":1226,"num_object_clones":0,"num_object_copies":2452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1226,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16935,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.876943+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:05:19.876943+0000","last_peered":"2024-09-10T16:05:19.876943+0000","last_clean":"2024-09-10T16:05:19.876943+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:05:19.876943+0000","last_undegraded":"2024-09-10T16:05:19.876943+0000","last_fullsized":"2024-09-10T16:05:19.876943+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'19213","reported_seq":19359,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:00.918154+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:08:00.918154+0000","last_peered":"2024-09-10T16:08:00.918154+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:08:00.918154+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"43'9200","ondisk_log_start":"43'9200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5095907,"num_objects":1245,"num_object_clones":0,"num_object_copies":2490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1245,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17032,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-2878}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":100,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.875849+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:05:19.875849+0000","last_peered":"2024-09-10T16:05:19.875849+0000","last_clean":"2024-09-10T16:05:19.875849+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:05:19.875849+0000","last_undegraded":"2024-09-10T16:05:19.875849+0000","last_fullsized":"2024-09-10T16:05:19.875849+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'19693","reported_seq":19830,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:00.480004+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:08:00.480004+0000","last_peered":"2024-09-10T16:08:00.480004+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:08:00.480004+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"43'9600","ondisk_log_start":"43'9600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4984832,"num_objects":1217,"num_object_clones":0,"num_object_copies":2434,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1217,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16197,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3038}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":110,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.872023+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:05:19.872023+0000","last_peered":"2024-09-10T16:05:19.872023+0000","last_clean":"2024-09-10T16:05:19.872023+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:05:19.872023+0000","last_undegraded":"2024-09-10T16:05:19.872023+0000","last_fullsized":"2024-09-10T16:05:19.872023+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'19605","reported_seq":19746,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:00.907052+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:08:00.907052+0000","last_peered":"2024-09-10T16:08:00.907052+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:08:00.907052+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"43'9600","ondisk_log_start":"43'9600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4817379,"num_objects":1177,"num_object_clones":0,"num_object_copies":2354,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1177,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17285,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-3127}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":86,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:32.574539+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:05:32.574539+0000","last_peered":"2024-09-10T16:05:32.574539+0000","last_clean":"2024-09-10T16:05:32.574539+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:05:32.574539+0000","last_undegraded":"2024-09-10T16:05:32.574539+0000","last_fullsized":"2024-09-10T16:05:32.574539+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'19940","reported_seq":28206,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:00.922598+0000","last_change":"2024-09-10T16:05:18.271152+0000","last_active":"2024-09-10T16:08:00.922598+0000","last_peered":"2024-09-10T16:08:00.922598+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:08:00.922598+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"43'9900","ondisk_log_start":"43'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5038080,"num_objects":1230,"num_object_clones":0,"num_object_copies":2460,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":289,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1230,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17494,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4065,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":941}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'19617","reported_seq":19758,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:00.759770+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:08:00.759770+0000","last_peered":"2024-09-10T16:08:00.759770+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:08:00.759770+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"43'9600","ondisk_log_start":"43'9600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5001216,"num_objects":1221,"num_object_clones":0,"num_object_copies":2442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1221,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16248,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3083}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:32.574559+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:05:32.574559+0000","last_peered":"2024-09-10T16:05:32.574559+0000","last_clean":"2024-09-10T16:05:32.574559+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:05:32.574559+0000","last_undegraded":"2024-09-10T16:05:32.574559+0000","last_fullsized":"2024-09-10T16:05:32.574559+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'19334","reported_seq":27906,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:07:59.556248+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:07:59.556248+0000","last_peered":"2024-09-10T16:07:59.556248+0000","last_clean":"2024-09-10T16:07:59.556248+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:07:59.556248+0000","last_undegraded":"2024-09-10T16:07:59.556248+0000","last_fullsized":"2024-09-10T16:07:59.556248+0000","mapping_epoch":46,"log_start":"43'9300","ondisk_log_start":"43'9300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5177344,"num_objects":1264,"num_object_clones":0,"num_object_copies":2528,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1264,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15724,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":100,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.876043+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:05:19.876043+0000","last_peered":"2024-09-10T16:05:19.876043+0000","last_clean":"2024-09-10T16:05:19.876043+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:05:19.876043+0000","last_undegraded":"2024-09-10T16:05:19.876043+0000","last_fullsized":"2024-09-10T16:05:19.876043+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'19898","reported_seq":20046,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:00.709848+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:08:00.709848+0000","last_peered":"2024-09-10T16:08:00.709848+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:08:00.709848+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"43'9800","ondisk_log_start":"43'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4710400,"num_objects":1150,"num_object_clones":0,"num_object_copies":2300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1150,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16153,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3221}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:32.574572+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:05:32.574572+0000","last_peered":"2024-09-10T16:05:32.574572+0000","last_clean":"2024-09-10T16:05:32.574572+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:05:32.574572+0000","last_undegraded":"2024-09-10T16:05:32.574572+0000","last_fullsized":"2024-09-10T16:05:32.574572+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":81,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:47.753676+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:05:47.753676+0000","last_peered":"2024-09-10T16:05:47.753676+0000","last_clean":"2024-09-10T16:05:47.753676+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:05:47.753676+0000","last_undegraded":"2024-09-10T16:05:47.753676+0000","last_fullsized":"2024-09-10T16:05:47.753676+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'19965","reported_seq":20116,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:00.695389+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:08:00.695389+0000","last_peered":"2024-09-10T16:08:00.695389+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:08:00.695389+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"44'9900","ondisk_log_start":"44'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5051334,"num_objects":1235,"num_object_clones":0,"num_object_copies":2470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1235,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16128,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3088},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":79303414,"num_objects":19371,"num_object_clones":0,"num_object_copies":38742,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38846,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19371,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":272735,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12543,"num_bytes_recovered":33095680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":370180096,"data_stored":370038271,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160661,"ondisk_log_size":160661,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986296,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":541688,"kb_used_data":176808,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93830152,"statfs":{"total":96636764160,"available":96082075648,"internally_reserved":0,"allocated":181051392,"data_stored":180748708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,3,2,1,1,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659241,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":443432,"kb_used_data":60492,"kb_used_omap":4,"kb_used_meta":382907,"kb_avail":93928408,"statfs":{"total":96636764160,"available":96182689792,"internally_reserved":0,"allocated":61943808,"data_stored":61611756,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":392097454},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,5,3,1,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182524,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":591568,"kb_used_data":126780,"kb_used_omap":4,"kb_used_meta":464763,"kb_avail":93780272,"statfs":{"total":96636764160,"available":96030998528,"internally_reserved":0,"allocated":129822720,"data_stored":129520031,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":475917994},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[5,1,0,0,5,2,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":128933888,"data_stored":128897758,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61251584,"data_stored":61186091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":179994624,"data_stored":179954422,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:08:06.195 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:08:06.764 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:08:06.765 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:08:06.795 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":365,"stamp":"2024-09-10T16:08:05.749314+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":78148361,"num_objects":19091,"num_object_clones":0,"num_object_copies":38182,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38846,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19091,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":273019,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12545,"num_bytes_recovered":33095699,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160646,"ondisk_log_size":160646,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1561292,"kb_used_data":357012,"kb_used_omap":12,"kb_used_meta":1204211,"kb_avail":281554228,"statfs":{"total":289910292480,"available":288311529472,"internally_reserved":0,"allocated":365580288,"data_stored":364642863,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1233112066},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[5,2,3,3,9,3,10,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-20516864,"num_objects":-5009,"num_object_clones":0,"num_object_copies":-10018,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-338,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5009,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5009,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":338,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002606"},"pg_stats":[{"pgid":"2.f","version":"47'19505","reported_seq":19584,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:00.779227+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:08:00.779227+0000","last_peered":"2024-09-10T16:08:00.779227+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:08:00.779227+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"43'9500","ondisk_log_start":"43'9500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4780032,"num_objects":1167,"num_object_clones":0,"num_object_copies":2334,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1167,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19505,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3092}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'19624","reported_seq":19743,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:08:00.835597+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:08:00.835597+0000","last_peered":"2024-09-10T16:08:00.835597+0000","last_clean":"2024-09-10T16:08:00.835597+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:08:00.835597+0000","last_undegraded":"2024-09-10T16:08:00.835597+0000","last_fullsized":"2024-09-10T16:08:00.835597+0000","mapping_epoch":35,"log_start":"43'9600","ondisk_log_start":"43'9600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4998086,"num_objects":1222,"num_object_clones":0,"num_object_copies":2444,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1222,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17729,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'19707","reported_seq":28391,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:08:04.588994+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:08:04.588994+0000","last_peered":"2024-09-10T16:08:04.588994+0000","last_clean":"2024-09-10T16:08:04.588994+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:08:04.588994+0000","last_undegraded":"2024-09-10T16:08:04.588994+0000","last_fullsized":"2024-09-10T16:08:04.588994+0000","mapping_epoch":44,"log_start":"43'9700","ondisk_log_start":"43'9700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4464664,"num_objects":1091,"num_object_clones":0,"num_object_copies":2182,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1091,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17711,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'19511","reported_seq":19598,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:08:00.878101+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:08:00.878101+0000","last_peered":"2024-09-10T16:08:00.878101+0000","last_clean":"2024-09-10T16:08:00.878101+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:08:00.878101+0000","last_undegraded":"2024-09-10T16:08:00.878101+0000","last_fullsized":"2024-09-10T16:08:00.878101+0000","mapping_epoch":35,"log_start":"43'9500","ondisk_log_start":"43'9500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4706304,"num_objects":1149,"num_object_clones":0,"num_object_copies":2298,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1149,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19511,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'19702","reported_seq":19838,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:08:00.754516+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:08:00.754516+0000","last_peered":"2024-09-10T16:08:00.754516+0000","last_clean":"2024-09-10T16:08:00.754516+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:08:00.754516+0000","last_undegraded":"2024-09-10T16:08:00.754516+0000","last_fullsized":"2024-09-10T16:08:00.754516+0000","mapping_epoch":34,"log_start":"43'9700","ondisk_log_start":"43'9700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4870627,"num_objects":1190,"num_object_clones":0,"num_object_copies":2380,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1190,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16413,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'19847","reported_seq":19991,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:00.693120+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:08:00.693120+0000","last_peered":"2024-09-10T16:08:00.693120+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:08:00.693120+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"43'9800","ondisk_log_start":"43'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4973027,"num_objects":1215,"num_object_clones":0,"num_object_copies":2430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1215,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16459,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3156},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'19862","reported_seq":20005,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:00.742997+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:08:00.742997+0000","last_peered":"2024-09-10T16:08:00.742997+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:08:00.742997+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"43'9800","ondisk_log_start":"43'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4998086,"num_objects":1222,"num_object_clones":0,"num_object_copies":2444,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1222,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16361,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3015},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'20088","reported_seq":20224,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:08:00.729636+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:08:00.729636+0000","last_peered":"2024-09-10T16:08:00.729636+0000","last_clean":"2024-09-10T16:08:00.729636+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:08:00.729636+0000","last_undegraded":"2024-09-10T16:08:00.729636+0000","last_fullsized":"2024-09-10T16:08:00.729636+0000","mapping_epoch":34,"log_start":"44'10000","ondisk_log_start":"44'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5021696,"num_objects":1226,"num_object_clones":0,"num_object_copies":2452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1226,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16935,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.876943+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:05:19.876943+0000","last_peered":"2024-09-10T16:05:19.876943+0000","last_clean":"2024-09-10T16:05:19.876943+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:05:19.876943+0000","last_undegraded":"2024-09-10T16:05:19.876943+0000","last_fullsized":"2024-09-10T16:05:19.876943+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'19213","reported_seq":19359,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:00.918154+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:08:00.918154+0000","last_peered":"2024-09-10T16:08:00.918154+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:08:00.918154+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"43'9200","ondisk_log_start":"43'9200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5095907,"num_objects":1245,"num_object_clones":0,"num_object_copies":2490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1245,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17032,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-2878}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":100,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.875849+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:05:19.875849+0000","last_peered":"2024-09-10T16:05:19.875849+0000","last_clean":"2024-09-10T16:05:19.875849+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:05:19.875849+0000","last_undegraded":"2024-09-10T16:05:19.875849+0000","last_fullsized":"2024-09-10T16:05:19.875849+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'19693","reported_seq":19830,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:00.480004+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:08:00.480004+0000","last_peered":"2024-09-10T16:08:00.480004+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:08:00.480004+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"43'9600","ondisk_log_start":"43'9600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4984832,"num_objects":1217,"num_object_clones":0,"num_object_copies":2434,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1217,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16197,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3038}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":110,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.872023+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:05:19.872023+0000","last_peered":"2024-09-10T16:05:19.872023+0000","last_clean":"2024-09-10T16:05:19.872023+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:05:19.872023+0000","last_undegraded":"2024-09-10T16:05:19.872023+0000","last_fullsized":"2024-09-10T16:05:19.872023+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'19605","reported_seq":19746,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:00.907052+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:08:00.907052+0000","last_peered":"2024-09-10T16:08:00.907052+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:08:00.907052+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"43'9600","ondisk_log_start":"43'9600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4817379,"num_objects":1177,"num_object_clones":0,"num_object_copies":2354,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1177,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17285,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-3127}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":86,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:32.574539+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:05:32.574539+0000","last_peered":"2024-09-10T16:05:32.574539+0000","last_clean":"2024-09-10T16:05:32.574539+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:05:32.574539+0000","last_undegraded":"2024-09-10T16:05:32.574539+0000","last_fullsized":"2024-09-10T16:05:32.574539+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'19940","reported_seq":28206,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:00.922598+0000","last_change":"2024-09-10T16:05:18.271152+0000","last_active":"2024-09-10T16:08:00.922598+0000","last_peered":"2024-09-10T16:08:00.922598+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:08:00.922598+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"43'9900","ondisk_log_start":"43'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5038080,"num_objects":1230,"num_object_clones":0,"num_object_copies":2460,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":289,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1230,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17494,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4065,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":941}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'19617","reported_seq":19758,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:00.759770+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:08:00.759770+0000","last_peered":"2024-09-10T16:08:00.759770+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:08:00.759770+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"43'9600","ondisk_log_start":"43'9600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5001216,"num_objects":1221,"num_object_clones":0,"num_object_copies":2442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1221,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16248,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3083}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:32.574559+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:05:32.574559+0000","last_peered":"2024-09-10T16:05:32.574559+0000","last_clean":"2024-09-10T16:05:32.574559+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:05:32.574559+0000","last_undegraded":"2024-09-10T16:05:32.574559+0000","last_fullsized":"2024-09-10T16:05:32.574559+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'19466","reported_seq":28038,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:08:04.671631+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:08:04.671631+0000","last_peered":"2024-09-10T16:08:04.671631+0000","last_clean":"2024-09-10T16:08:04.671631+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:08:04.671631+0000","last_undegraded":"2024-09-10T16:08:04.671631+0000","last_fullsized":"2024-09-10T16:08:04.671631+0000","mapping_epoch":46,"log_start":"43'9400","ondisk_log_start":"43'9400","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4636672,"num_objects":1132,"num_object_clones":0,"num_object_copies":2264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1132,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15856,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":100,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.876043+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:05:19.876043+0000","last_peered":"2024-09-10T16:05:19.876043+0000","last_clean":"2024-09-10T16:05:19.876043+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:05:19.876043+0000","last_undegraded":"2024-09-10T16:05:19.876043+0000","last_fullsized":"2024-09-10T16:05:19.876043+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'19898","reported_seq":20046,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:00.709848+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:08:00.709848+0000","last_peered":"2024-09-10T16:08:00.709848+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:08:00.709848+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"43'9800","ondisk_log_start":"43'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4710400,"num_objects":1150,"num_object_clones":0,"num_object_copies":2300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1150,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16153,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3221}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:32.574572+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:05:32.574572+0000","last_peered":"2024-09-10T16:05:32.574572+0000","last_clean":"2024-09-10T16:05:32.574572+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:05:32.574572+0000","last_undegraded":"2024-09-10T16:05:32.574572+0000","last_fullsized":"2024-09-10T16:05:32.574572+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":81,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:47.753676+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:05:47.753676+0000","last_peered":"2024-09-10T16:05:47.753676+0000","last_clean":"2024-09-10T16:05:47.753676+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:05:47.753676+0000","last_undegraded":"2024-09-10T16:05:47.753676+0000","last_fullsized":"2024-09-10T16:05:47.753676+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'19965","reported_seq":20116,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:00.695389+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:08:00.695389+0000","last_peered":"2024-09-10T16:08:00.695389+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:08:00.695389+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"44'9900","ondisk_log_start":"44'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5051334,"num_objects":1235,"num_object_clones":0,"num_object_copies":2470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1235,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16128,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3088},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":78148342,"num_objects":19089,"num_object_clones":0,"num_object_copies":38178,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38846,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19089,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":273017,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12543,"num_bytes_recovered":33095680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":362967040,"data_stored":362825215,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160643,"ondisk_log_size":160643,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986296,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":541688,"kb_used_data":176808,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93830152,"statfs":{"total":96636764160,"available":96082075648,"internally_reserved":0,"allocated":181051392,"data_stored":180748708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,3,2,1,1,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659242,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":428036,"kb_used_data":53424,"kb_used_omap":4,"kb_used_meta":374587,"kb_avail":93943804,"statfs":{"total":96636764160,"available":96198455296,"internally_reserved":0,"allocated":54706176,"data_stored":54374124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":383577774},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,3,0,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182524,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":591568,"kb_used_data":126780,"kb_used_omap":4,"kb_used_meta":464763,"kb_avail":93780272,"statfs":{"total":96636764160,"available":96030998528,"internally_reserved":0,"allocated":129822720,"data_stored":129520031,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":475917994},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[5,1,0,0,5,2,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":128933888,"data_stored":128897758,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":54038528,"data_stored":53973035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":179994624,"data_stored":179954422,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:08:06.796 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:08:07.344 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:08:07.346 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:08:07.378 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":365,"stamp":"2024-09-10T16:08:05.749314+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":78148361,"num_objects":19091,"num_object_clones":0,"num_object_copies":38182,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38846,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19091,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":273019,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12545,"num_bytes_recovered":33095699,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160646,"ondisk_log_size":160646,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1561292,"kb_used_data":357012,"kb_used_omap":12,"kb_used_meta":1204211,"kb_avail":281554228,"statfs":{"total":289910292480,"available":288311529472,"internally_reserved":0,"allocated":365580288,"data_stored":364642863,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1233112066},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[5,2,3,3,9,3,10,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-20516864,"num_objects":-5009,"num_object_clones":0,"num_object_copies":-10018,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-338,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5009,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5009,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":338,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002606"},"pg_stats":[{"pgid":"2.f","version":"47'19505","reported_seq":19584,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:00.779227+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:08:00.779227+0000","last_peered":"2024-09-10T16:08:00.779227+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:08:00.779227+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"43'9500","ondisk_log_start":"43'9500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4780032,"num_objects":1167,"num_object_clones":0,"num_object_copies":2334,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1167,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19505,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3092}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'19624","reported_seq":19743,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:08:00.835597+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:08:00.835597+0000","last_peered":"2024-09-10T16:08:00.835597+0000","last_clean":"2024-09-10T16:08:00.835597+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:08:00.835597+0000","last_undegraded":"2024-09-10T16:08:00.835597+0000","last_fullsized":"2024-09-10T16:08:00.835597+0000","mapping_epoch":35,"log_start":"43'9600","ondisk_log_start":"43'9600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4998086,"num_objects":1222,"num_object_clones":0,"num_object_copies":2444,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1222,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17729,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'19707","reported_seq":28391,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:08:04.588994+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:08:04.588994+0000","last_peered":"2024-09-10T16:08:04.588994+0000","last_clean":"2024-09-10T16:08:04.588994+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:08:04.588994+0000","last_undegraded":"2024-09-10T16:08:04.588994+0000","last_fullsized":"2024-09-10T16:08:04.588994+0000","mapping_epoch":44,"log_start":"43'9700","ondisk_log_start":"43'9700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4464664,"num_objects":1091,"num_object_clones":0,"num_object_copies":2182,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1091,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17711,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'19511","reported_seq":19598,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:08:00.878101+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:08:00.878101+0000","last_peered":"2024-09-10T16:08:00.878101+0000","last_clean":"2024-09-10T16:08:00.878101+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:08:00.878101+0000","last_undegraded":"2024-09-10T16:08:00.878101+0000","last_fullsized":"2024-09-10T16:08:00.878101+0000","mapping_epoch":35,"log_start":"43'9500","ondisk_log_start":"43'9500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4706304,"num_objects":1149,"num_object_clones":0,"num_object_copies":2298,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1149,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19511,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'19702","reported_seq":19838,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:08:00.754516+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:08:00.754516+0000","last_peered":"2024-09-10T16:08:00.754516+0000","last_clean":"2024-09-10T16:08:00.754516+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:08:00.754516+0000","last_undegraded":"2024-09-10T16:08:00.754516+0000","last_fullsized":"2024-09-10T16:08:00.754516+0000","mapping_epoch":34,"log_start":"43'9700","ondisk_log_start":"43'9700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4870627,"num_objects":1190,"num_object_clones":0,"num_object_copies":2380,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1190,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16413,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'19847","reported_seq":19991,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:00.693120+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:08:00.693120+0000","last_peered":"2024-09-10T16:08:00.693120+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:08:00.693120+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"43'9800","ondisk_log_start":"43'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4973027,"num_objects":1215,"num_object_clones":0,"num_object_copies":2430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1215,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16459,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3156},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'19862","reported_seq":20005,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:00.742997+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:08:00.742997+0000","last_peered":"2024-09-10T16:08:00.742997+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:08:00.742997+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"43'9800","ondisk_log_start":"43'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4998086,"num_objects":1222,"num_object_clones":0,"num_object_copies":2444,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1222,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16361,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3015},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'20088","reported_seq":20224,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:08:00.729636+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:08:00.729636+0000","last_peered":"2024-09-10T16:08:00.729636+0000","last_clean":"2024-09-10T16:08:00.729636+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:08:00.729636+0000","last_undegraded":"2024-09-10T16:08:00.729636+0000","last_fullsized":"2024-09-10T16:08:00.729636+0000","mapping_epoch":34,"log_start":"44'10000","ondisk_log_start":"44'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5021696,"num_objects":1226,"num_object_clones":0,"num_object_copies":2452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1226,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16935,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.876943+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:05:19.876943+0000","last_peered":"2024-09-10T16:05:19.876943+0000","last_clean":"2024-09-10T16:05:19.876943+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:05:19.876943+0000","last_undegraded":"2024-09-10T16:05:19.876943+0000","last_fullsized":"2024-09-10T16:05:19.876943+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'19213","reported_seq":19359,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:00.918154+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:08:00.918154+0000","last_peered":"2024-09-10T16:08:00.918154+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:08:00.918154+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"43'9200","ondisk_log_start":"43'9200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5095907,"num_objects":1245,"num_object_clones":0,"num_object_copies":2490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1245,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17032,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-2878}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":100,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.875849+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:05:19.875849+0000","last_peered":"2024-09-10T16:05:19.875849+0000","last_clean":"2024-09-10T16:05:19.875849+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:05:19.875849+0000","last_undegraded":"2024-09-10T16:05:19.875849+0000","last_fullsized":"2024-09-10T16:05:19.875849+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'19693","reported_seq":19830,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:00.480004+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:08:00.480004+0000","last_peered":"2024-09-10T16:08:00.480004+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:08:00.480004+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"43'9600","ondisk_log_start":"43'9600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4984832,"num_objects":1217,"num_object_clones":0,"num_object_copies":2434,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1217,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16197,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3038}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":110,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.872023+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:05:19.872023+0000","last_peered":"2024-09-10T16:05:19.872023+0000","last_clean":"2024-09-10T16:05:19.872023+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:05:19.872023+0000","last_undegraded":"2024-09-10T16:05:19.872023+0000","last_fullsized":"2024-09-10T16:05:19.872023+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'19605","reported_seq":19746,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:00.907052+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:08:00.907052+0000","last_peered":"2024-09-10T16:08:00.907052+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:08:00.907052+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"43'9600","ondisk_log_start":"43'9600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4817379,"num_objects":1177,"num_object_clones":0,"num_object_copies":2354,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1177,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17285,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-3127}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":86,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:32.574539+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:05:32.574539+0000","last_peered":"2024-09-10T16:05:32.574539+0000","last_clean":"2024-09-10T16:05:32.574539+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:05:32.574539+0000","last_undegraded":"2024-09-10T16:05:32.574539+0000","last_fullsized":"2024-09-10T16:05:32.574539+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'19940","reported_seq":28206,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:00.922598+0000","last_change":"2024-09-10T16:05:18.271152+0000","last_active":"2024-09-10T16:08:00.922598+0000","last_peered":"2024-09-10T16:08:00.922598+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:08:00.922598+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"43'9900","ondisk_log_start":"43'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5038080,"num_objects":1230,"num_object_clones":0,"num_object_copies":2460,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":289,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1230,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17494,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4065,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":941}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'19617","reported_seq":19758,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:00.759770+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:08:00.759770+0000","last_peered":"2024-09-10T16:08:00.759770+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:08:00.759770+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"43'9600","ondisk_log_start":"43'9600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5001216,"num_objects":1221,"num_object_clones":0,"num_object_copies":2442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1221,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16248,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3083}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:32.574559+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:05:32.574559+0000","last_peered":"2024-09-10T16:05:32.574559+0000","last_clean":"2024-09-10T16:05:32.574559+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:05:32.574559+0000","last_undegraded":"2024-09-10T16:05:32.574559+0000","last_fullsized":"2024-09-10T16:05:32.574559+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'19466","reported_seq":28038,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:08:04.671631+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:08:04.671631+0000","last_peered":"2024-09-10T16:08:04.671631+0000","last_clean":"2024-09-10T16:08:04.671631+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:08:04.671631+0000","last_undegraded":"2024-09-10T16:08:04.671631+0000","last_fullsized":"2024-09-10T16:08:04.671631+0000","mapping_epoch":46,"log_start":"43'9400","ondisk_log_start":"43'9400","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4636672,"num_objects":1132,"num_object_clones":0,"num_object_copies":2264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1132,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15856,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":100,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.876043+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:05:19.876043+0000","last_peered":"2024-09-10T16:05:19.876043+0000","last_clean":"2024-09-10T16:05:19.876043+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:05:19.876043+0000","last_undegraded":"2024-09-10T16:05:19.876043+0000","last_fullsized":"2024-09-10T16:05:19.876043+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'19898","reported_seq":20046,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:00.709848+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:08:00.709848+0000","last_peered":"2024-09-10T16:08:00.709848+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:08:00.709848+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"43'9800","ondisk_log_start":"43'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4710400,"num_objects":1150,"num_object_clones":0,"num_object_copies":2300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1150,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16153,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3221}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:32.574572+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:05:32.574572+0000","last_peered":"2024-09-10T16:05:32.574572+0000","last_clean":"2024-09-10T16:05:32.574572+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:05:32.574572+0000","last_undegraded":"2024-09-10T16:05:32.574572+0000","last_fullsized":"2024-09-10T16:05:32.574572+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":81,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:47.753676+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:05:47.753676+0000","last_peered":"2024-09-10T16:05:47.753676+0000","last_clean":"2024-09-10T16:05:47.753676+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:05:47.753676+0000","last_undegraded":"2024-09-10T16:05:47.753676+0000","last_fullsized":"2024-09-10T16:05:47.753676+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'19965","reported_seq":20116,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:00.695389+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:08:00.695389+0000","last_peered":"2024-09-10T16:08:00.695389+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:08:00.695389+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"44'9900","ondisk_log_start":"44'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5051334,"num_objects":1235,"num_object_clones":0,"num_object_copies":2470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1235,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16128,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3088},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":78148342,"num_objects":19089,"num_object_clones":0,"num_object_copies":38178,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38846,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19089,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":273017,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12543,"num_bytes_recovered":33095680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":362967040,"data_stored":362825215,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160643,"ondisk_log_size":160643,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986296,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":541688,"kb_used_data":176808,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93830152,"statfs":{"total":96636764160,"available":96082075648,"internally_reserved":0,"allocated":181051392,"data_stored":180748708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,3,2,1,1,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659242,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":428036,"kb_used_data":53424,"kb_used_omap":4,"kb_used_meta":374587,"kb_avail":93943804,"statfs":{"total":96636764160,"available":96198455296,"internally_reserved":0,"allocated":54706176,"data_stored":54374124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":383577774},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,3,0,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182524,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":591568,"kb_used_data":126780,"kb_used_omap":4,"kb_used_meta":464763,"kb_avail":93780272,"statfs":{"total":96636764160,"available":96030998528,"internally_reserved":0,"allocated":129822720,"data_stored":129520031,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":475917994},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[5,1,0,0,5,2,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":128933888,"data_stored":128897758,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":54038528,"data_stored":53973035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":179994624,"data_stored":179954422,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:08:07.379 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-10T16:08:07.380 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "47'19505", 'reported_seq': 19584, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:08:00.779227+0000', 'last_change': '2024-09-10T15:59:16.281558+0000', 'last_active': '2024-09-10T16:08:00.779227+0000', 'last_peered': '2024-09-10T16:08:00.779227+0000', 'last_clean': '2024-09-10T15:59:14.132456+0000', 'last_became_active': '2024-09-10T15:59:16.274407+0000', 'last_became_peered': '2024-09-10T15:59:16.274407+0000', 'last_unstale': '2024-09-10T16:08:00.779227+0000', 'last_undegraded': '2024-09-10T15:59:15.851279+0000', 'last_fullsized': '2024-09-10T15:59:15.774356+0000', 'mapping_epoch': 42, 'log_start': "43'9500", 'ondisk_log_start': "43'9500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'objects_scrubbed': 0, 'log_size': 10005, 'log_dups_size': 0, 'ondisk_log_size': 10005, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 4780032, 'num_objects': 1167, 'num_object_clones': 0, 'num_object_copies': 2334, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4259, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1167, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 19505, 'num_write_kb': 41344, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3092}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:08:07.380 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-10T16:08:07.380 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "47'19847", 'reported_seq': 19991, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:08:00.693120+0000', 'last_change': '2024-09-10T15:59:16.308264+0000', 'last_active': '2024-09-10T16:08:00.693120+0000', 'last_peered': '2024-09-10T16:08:00.693120+0000', 'last_clean': '2024-09-10T15:59:14.111551+0000', 'last_became_active': '2024-09-10T15:59:16.300040+0000', 'last_became_peered': '2024-09-10T15:59:16.300040+0000', 'last_unstale': '2024-09-10T16:08:00.693120+0000', 'last_undegraded': '2024-09-10T15:59:15.878234+0000', 'last_fullsized': '2024-09-10T15:59:15.769328+0000', 'mapping_epoch': 42, 'log_start': "43'9800", 'ondisk_log_start': "43'9800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3442", 'last_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'objects_scrubbed': 3384, 'log_size': 10047, 'log_dups_size': 0, 'ondisk_log_size': 10047, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1292, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 4973027, 'num_objects': 1215, 'num_object_clones': 0, 'num_object_copies': 2430, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4372, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1215, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 16459, 'num_write_kb': 28580, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3156}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:08:07.380 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-10T16:08:07.380 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "47'19862", 'reported_seq': 20005, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:08:00.742997+0000', 'last_change': '2024-09-10T15:59:16.331899+0000', 'last_active': '2024-09-10T16:08:00.742997+0000', 'last_peered': '2024-09-10T16:08:00.742997+0000', 'last_clean': '2024-09-10T15:59:14.139436+0000', 'last_became_active': '2024-09-10T15:59:16.318715+0000', 'last_became_peered': '2024-09-10T15:59:16.318715+0000', 'last_unstale': '2024-09-10T16:08:00.742997+0000', 'last_undegraded': '2024-09-10T15:59:15.877524+0000', 'last_fullsized': '2024-09-10T15:59:15.769042+0000', 'mapping_epoch': 42, 'log_start': "43'9800", 'ondisk_log_start': "43'9800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3537", 'last_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'objects_scrubbed': 3497, 'log_size': 10062, 'log_dups_size': 0, 'ondisk_log_size': 10062, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1224, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 4998086, 'num_objects': 1222, 'num_object_clones': 0, 'num_object_copies': 2444, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4237, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1222, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 16361, 'num_write_kb': 28172, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3015}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:08:07.380 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-10T16:08:07.380 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "47'19213", 'reported_seq': 19359, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:08:00.918154+0000', 'last_change': '2024-09-10T15:59:16.175680+0000', 'last_active': '2024-09-10T16:08:00.918154+0000', 'last_peered': '2024-09-10T16:08:00.918154+0000', 'last_clean': '2024-09-10T15:59:14.088187+0000', 'last_became_active': '2024-09-10T15:59:16.165810+0000', 'last_became_peered': '2024-09-10T15:59:16.165810+0000', 'last_unstale': '2024-09-10T16:08:00.918154+0000', 'last_undegraded': '2024-09-10T15:59:15.822098+0000', 'last_fullsized': '2024-09-10T15:59:15.769123+0000', 'mapping_epoch': 42, 'log_start': "43'9200", 'ondisk_log_start': "43'9200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2218", 'last_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'objects_scrubbed': 2175, 'log_size': 10013, 'log_dups_size': 0, 'ondisk_log_size': 10013, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 825, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 5095907, 'num_objects': 1245, 'num_object_clones': 0, 'num_object_copies': 2490, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4124, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1245, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 17032, 'num_write_kb': 32204, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': -2878}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:08:07.380 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-10T16:08:07.380 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "47'19693", 'reported_seq': 19830, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:08:00.480004+0000', 'last_change': '2024-09-10T15:59:16.277742+0000', 'last_active': '2024-09-10T16:08:00.480004+0000', 'last_peered': '2024-09-10T16:08:00.480004+0000', 'last_clean': '2024-09-10T15:59:14.129081+0000', 'last_became_active': '2024-09-10T15:59:16.269835+0000', 'last_became_peered': '2024-09-10T15:59:16.269835+0000', 'last_unstale': '2024-09-10T16:08:00.480004+0000', 'last_undegraded': '2024-09-10T15:59:15.860857+0000', 'last_fullsized': '2024-09-10T15:59:15.771410+0000', 'mapping_epoch': 42, 'log_start': "43'9600", 'ondisk_log_start': "43'9600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3557", 'last_scrub_stamp': '2024-09-10T15:59:02.493678+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:02.493678+0000', 'objects_scrubbed': 3488, 'log_size': 10093, 'log_dups_size': 0, 'ondisk_log_size': 10093, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1472, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 4984832, 'num_objects': 1217, 'num_object_clones': 0, 'num_object_copies': 2434, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4257, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1217, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16197, 'num_write_kb': 27852, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3038}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:08:07.381 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-10T16:08:07.381 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "47'19605", 'reported_seq': 19746, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:08:00.907052+0000', 'last_change': '2024-09-10T15:59:16.284311+0000', 'last_active': '2024-09-10T16:08:00.907052+0000', 'last_peered': '2024-09-10T16:08:00.907052+0000', 'last_clean': '2024-09-10T15:59:14.116041+0000', 'last_became_active': '2024-09-10T15:59:16.274543+0000', 'last_became_peered': '2024-09-10T15:59:16.274543+0000', 'last_unstale': '2024-09-10T16:08:00.907052+0000', 'last_undegraded': '2024-09-10T15:59:15.812484+0000', 'last_fullsized': '2024-09-10T15:59:15.770885+0000', 'mapping_epoch': 42, 'log_start': "43'9600", 'ondisk_log_start': "43'9600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2356", 'last_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'objects_scrubbed': 2316, 'log_size': 10005, 'log_dups_size': 0, 'ondisk_log_size': 10005, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 783, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 4817379, 'num_objects': 1177, 'num_object_clones': 0, 'num_object_copies': 2354, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1177, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 17285, 'num_write_kb': 32292, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': -3127}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:08:07.381 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-10T16:08:07.381 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "47'19940", 'reported_seq': 28206, 'reported_epoch': 47, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:08:00.922598+0000', 'last_change': '2024-09-10T16:05:18.271152+0000', 'last_active': '2024-09-10T16:08:00.922598+0000', 'last_peered': '2024-09-10T16:08:00.922598+0000', 'last_clean': '2024-09-10T15:59:14.110654+0000', 'last_became_active': '2024-09-10T15:59:16.164645+0000', 'last_became_peered': '2024-09-10T15:59:16.164645+0000', 'last_unstale': '2024-09-10T16:08:00.922598+0000', 'last_undegraded': '2024-09-10T15:59:15.803593+0000', 'last_fullsized': '2024-09-10T15:59:15.769083+0000', 'mapping_epoch': 42, 'log_start': "43'9900", 'ondisk_log_start': "43'9900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2486", 'last_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'objects_scrubbed': 2440, 'log_size': 10040, 'log_dups_size': 0, 'ondisk_log_size': 10040, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 792, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 5038080, 'num_objects': 1230, 'num_object_clones': 0, 'num_object_copies': 2460, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 289, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1230, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17494, 'num_write_kb': 32568, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 4065, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 941}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:08:07.381 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-10T16:08:07.381 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "47'19617", 'reported_seq': 19758, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:08:00.759770+0000', 'last_change': '2024-09-10T15:59:16.315641+0000', 'last_active': '2024-09-10T16:08:00.759770+0000', 'last_peered': '2024-09-10T16:08:00.759770+0000', 'last_clean': '2024-09-10T15:59:14.117001+0000', 'last_became_active': '2024-09-10T15:59:16.302245+0000', 'last_became_peered': '2024-09-10T15:59:16.302245+0000', 'last_unstale': '2024-09-10T16:08:00.759770+0000', 'last_undegraded': '2024-09-10T15:59:15.845012+0000', 'last_fullsized': '2024-09-10T15:59:15.770337+0000', 'mapping_epoch': 42, 'log_start': "43'9600", 'ondisk_log_start': "43'9600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3404", 'last_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'objects_scrubbed': 3367, 'log_size': 10017, 'log_dups_size': 0, 'ondisk_log_size': 10017, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 983, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 5001216, 'num_objects': 1221, 'num_object_clones': 0, 'num_object_copies': 2442, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1221, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16248, 'num_write_kb': 28204, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3083}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:08:07.381 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-10T16:08:07.381 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "47'19898", 'reported_seq': 20046, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:08:00.709848+0000', 'last_change': '2024-09-10T15:59:16.311291+0000', 'last_active': '2024-09-10T16:08:00.709848+0000', 'last_peered': '2024-09-10T16:08:00.709848+0000', 'last_clean': '2024-09-10T15:59:14.116363+0000', 'last_became_active': '2024-09-10T15:59:16.301244+0000', 'last_became_peered': '2024-09-10T15:59:16.301244+0000', 'last_unstale': '2024-09-10T16:08:00.709848+0000', 'last_undegraded': '2024-09-10T15:59:15.883434+0000', 'last_fullsized': '2024-09-10T15:59:15.772756+0000', 'mapping_epoch': 42, 'log_start': "43'9800", 'ondisk_log_start': "43'9800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3789", 'last_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'objects_scrubbed': 3741, 'log_size': 10098, 'log_dups_size': 0, 'ondisk_log_size': 10098, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1440, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 4710400, 'num_objects': 1150, 'num_object_clones': 0, 'num_object_copies': 2300, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4373, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1150, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16153, 'num_write_kb': 27124, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3221}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:08:07.381 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-10T16:08:07.381 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "47'19965", 'reported_seq': 20116, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:08:00.695389+0000', 'last_change': '2024-09-10T15:59:16.313409+0000', 'last_active': '2024-09-10T16:08:00.695389+0000', 'last_peered': '2024-09-10T16:08:00.695389+0000', 'last_clean': '2024-09-10T15:59:14.129780+0000', 'last_became_active': '2024-09-10T15:59:16.300932+0000', 'last_became_peered': '2024-09-10T15:59:16.300932+0000', 'last_unstale': '2024-09-10T16:08:00.695389+0000', 'last_undegraded': '2024-09-10T15:59:15.887823+0000', 'last_fullsized': '2024-09-10T15:59:15.768513+0000', 'mapping_epoch': 42, 'log_start': "44'9900", 'ondisk_log_start': "44'9900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3880", 'last_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'objects_scrubbed': 3827, 'log_size': 10065, 'log_dups_size': 0, 'ondisk_log_size': 10065, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1397, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 5051334, 'num_objects': 1235, 'num_object_clones': 0, 'num_object_copies': 2470, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4325, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1235, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16128, 'num_write_kb': 27072, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3088}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:08:07.382 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T16:08:08.080 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:08:08.080 INFO:teuthology.orchestra.run.smithi115.stdout:{"fsid":"7effee29-df05-4351-9741-ee203564ed81","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 38846/38182 objects degraded (101.739%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":652,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":47,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983954,"num_in_osds":3,"osd_in_since":1725983947,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":19091,"data_bytes":78148361,"bytes_used":1598763008,"bytes_avail":288311529472,"bytes_total":289910292480,"degraded_objects":38846,"degraded_total":38182,"degraded_ratio":1.0173903933790791,"recovering_objects_per_sec":28,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":338,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":417},"fsmap":{"epoch":1,"btime":"2024-09-10T15:57:14:967120+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T16:00:13.560776+0000","services":{}},"progress_events":{}} 2024-09-10T16:08:08.108 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-10T16:08:08.108 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:08:08.664 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:08:08.666 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:08:08.688 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":366,"stamp":"2024-09-10T16:08:07.749856+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":69632777,"num_objects":17012,"num_object_clones":0,"num_object_copies":34024,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38671,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17012,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":275098,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12720,"num_bytes_recovered":33095699,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160725,"ondisk_log_size":160725,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1561916,"kb_used_data":345152,"kb_used_omap":12,"kb_used_meta":1216691,"kb_avail":281553604,"statfs":{"total":289910292480,"available":288310890496,"internally_reserved":0,"allocated":353435648,"data_stored":352498223,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1245891586},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,3,2,8,6,14],"upper_bound":128},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-22233088,"num_objects":-5428,"num_object_clones":0,"num_object_copies":-10856,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-513,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5428,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5428,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":513,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002734"},"pg_stats":[{"pgid":"2.f","version":"47'19646","reported_seq":19727,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:05.740840+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:08:05.740840+0000","last_peered":"2024-09-10T16:08:05.740840+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:08:05.740840+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"43'9600","ondisk_log_start":"43'9600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4202496,"num_objects":1026,"num_object_clones":0,"num_object_copies":2052,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1026,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19646,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3233}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'19785","reported_seq":19904,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:08:05.920385+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:08:05.920385+0000","last_peered":"2024-09-10T16:08:05.920385+0000","last_clean":"2024-09-10T16:08:05.920385+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:08:05.920385+0000","last_undegraded":"2024-09-10T16:08:05.920385+0000","last_fullsized":"2024-09-10T16:08:05.920385+0000","mapping_epoch":35,"log_start":"43'9700","ondisk_log_start":"43'9700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4338630,"num_objects":1061,"num_object_clones":0,"num_object_copies":2122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1061,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17890,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'19707","reported_seq":28391,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:08:04.588994+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:08:04.588994+0000","last_peered":"2024-09-10T16:08:04.588994+0000","last_clean":"2024-09-10T16:08:04.588994+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:08:04.588994+0000","last_undegraded":"2024-09-10T16:08:04.588994+0000","last_fullsized":"2024-09-10T16:08:04.588994+0000","mapping_epoch":44,"log_start":"43'9700","ondisk_log_start":"43'9700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4464664,"num_objects":1091,"num_object_clones":0,"num_object_copies":2182,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1091,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17711,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'19641","reported_seq":19728,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:08:05.953267+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:08:05.953267+0000","last_peered":"2024-09-10T16:08:05.953267+0000","last_clean":"2024-09-10T16:08:05.953267+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:08:05.953267+0000","last_undegraded":"2024-09-10T16:08:05.953267+0000","last_fullsized":"2024-09-10T16:08:05.953267+0000","mapping_epoch":35,"log_start":"43'9600","ondisk_log_start":"43'9600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4173824,"num_objects":1019,"num_object_clones":0,"num_object_copies":2038,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1019,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19641,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'19828","reported_seq":19964,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:08:05.787113+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:08:05.787113+0000","last_peered":"2024-09-10T16:08:05.787113+0000","last_clean":"2024-09-10T16:08:05.787113+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:08:05.787113+0000","last_undegraded":"2024-09-10T16:08:05.787113+0000","last_fullsized":"2024-09-10T16:08:05.787113+0000","mapping_epoch":34,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4354531,"num_objects":1064,"num_object_clones":0,"num_object_copies":2128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1064,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16539,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'19999","reported_seq":20143,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:05.759182+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:08:05.759182+0000","last_peered":"2024-09-10T16:08:05.759182+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:08:05.759182+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4350435,"num_objects":1063,"num_object_clones":0,"num_object_copies":2126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1063,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16611,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3308},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20005","reported_seq":20149,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:05.753782+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:08:05.753782+0000","last_peered":"2024-09-10T16:08:05.753782+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:08:05.753782+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4412358,"num_objects":1079,"num_object_clones":0,"num_object_copies":2158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1079,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16504,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3158},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'20254","reported_seq":20390,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:08:05.798959+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:08:05.798959+0000","last_peered":"2024-09-10T16:08:05.798959+0000","last_clean":"2024-09-10T16:08:05.798959+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:08:05.798959+0000","last_undegraded":"2024-09-10T16:08:05.798959+0000","last_fullsized":"2024-09-10T16:08:05.798959+0000","mapping_epoch":34,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4341760,"num_objects":1060,"num_object_clones":0,"num_object_copies":2120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1060,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17101,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.876943+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:05:19.876943+0000","last_peered":"2024-09-10T16:05:19.876943+0000","last_clean":"2024-09-10T16:05:19.876943+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:05:19.876943+0000","last_undegraded":"2024-09-10T16:05:19.876943+0000","last_fullsized":"2024-09-10T16:05:19.876943+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'19372","reported_seq":19518,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:05.958811+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:08:05.958811+0000","last_peered":"2024-09-10T16:08:05.958811+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:08:05.958811+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"43'9300","ondisk_log_start":"43'9300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4444643,"num_objects":1086,"num_object_clones":0,"num_object_copies":2172,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1086,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17191,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-3037}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":100,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.875849+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:05:19.875849+0000","last_peered":"2024-09-10T16:05:19.875849+0000","last_clean":"2024-09-10T16:05:19.875849+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:05:19.875849+0000","last_undegraded":"2024-09-10T16:05:19.875849+0000","last_fullsized":"2024-09-10T16:05:19.875849+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'19856","reported_seq":19993,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:05.788804+0000","last_change":"2024-09-10T15:59:16.277742+0000","last_active":"2024-09-10T16:08:05.788804+0000","last_peered":"2024-09-10T16:08:05.788804+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:08:05.788804+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"44'9800","ondisk_log_start":"44'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4317184,"num_objects":1054,"num_object_clones":0,"num_object_copies":2108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1054,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16360,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3201}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":110,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.872023+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:05:19.872023+0000","last_peered":"2024-09-10T16:05:19.872023+0000","last_clean":"2024-09-10T16:05:19.872023+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:05:19.872023+0000","last_undegraded":"2024-09-10T16:05:19.872023+0000","last_fullsized":"2024-09-10T16:05:19.872023+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'19738","reported_seq":19879,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:05.893168+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:08:05.893168+0000","last_peered":"2024-09-10T16:08:05.893168+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:08:05.893168+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"44'9700","ondisk_log_start":"44'9700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4272611,"num_objects":1044,"num_object_clones":0,"num_object_copies":2088,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1044,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17418,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-3260}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":86,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:32.574539+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:05:32.574539+0000","last_peered":"2024-09-10T16:05:32.574539+0000","last_clean":"2024-09-10T16:05:32.574539+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:05:32.574539+0000","last_undegraded":"2024-09-10T16:05:32.574539+0000","last_fullsized":"2024-09-10T16:05:32.574539+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20108","reported_seq":28722,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:05.946637+0000","last_change":"2024-09-10T16:05:18.271152+0000","last_active":"2024-09-10T16:08:05.946637+0000","last_peered":"2024-09-10T16:08:05.946637+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T15:59:16.164645+0000","last_became_peered":"2024-09-10T15:59:16.164645+0000","last_unstale":"2024-09-10T16:08:05.946637+0000","last_undegraded":"2024-09-10T15:59:15.803593+0000","last_fullsized":"2024-09-10T15:59:15.769083+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4349952,"num_objects":1062,"num_object_clones":0,"num_object_copies":2124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":114,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1062,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17662,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4240,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":948}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'19765","reported_seq":19906,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:05.693932+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:08:05.693932+0000","last_peered":"2024-09-10T16:08:05.693932+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:08:05.693932+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"43'9700","ondisk_log_start":"43'9700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4395008,"num_objects":1073,"num_object_clones":0,"num_object_copies":2146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1073,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16396,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3231}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:32.574559+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:05:32.574559+0000","last_peered":"2024-09-10T16:05:32.574559+0000","last_clean":"2024-09-10T16:05:32.574559+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:05:32.574559+0000","last_undegraded":"2024-09-10T16:05:32.574559+0000","last_fullsized":"2024-09-10T16:05:32.574559+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'19466","reported_seq":28038,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:08:04.671631+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:08:04.671631+0000","last_peered":"2024-09-10T16:08:04.671631+0000","last_clean":"2024-09-10T16:08:04.671631+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:08:04.671631+0000","last_undegraded":"2024-09-10T16:08:04.671631+0000","last_fullsized":"2024-09-10T16:08:04.671631+0000","mapping_epoch":46,"log_start":"43'9400","ondisk_log_start":"43'9400","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4636672,"num_objects":1132,"num_object_clones":0,"num_object_copies":2264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1132,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15856,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":100,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.876043+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:05:19.876043+0000","last_peered":"2024-09-10T16:05:19.876043+0000","last_clean":"2024-09-10T16:05:19.876043+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:05:19.876043+0000","last_undegraded":"2024-09-10T16:05:19.876043+0000","last_fullsized":"2024-09-10T16:05:19.876043+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'20041","reported_seq":20189,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:05.753162+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:08:05.753162+0000","last_peered":"2024-09-10T16:08:05.753162+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:08:05.753162+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4124672,"num_objects":1007,"num_object_clones":0,"num_object_copies":2014,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1007,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16296,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3364}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:32.574572+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:05:32.574572+0000","last_peered":"2024-09-10T16:05:32.574572+0000","last_clean":"2024-09-10T16:05:32.574572+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:05:32.574572+0000","last_undegraded":"2024-09-10T16:05:32.574572+0000","last_fullsized":"2024-09-10T16:05:32.574572+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":81,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:47.753676+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:05:47.753676+0000","last_peered":"2024-09-10T16:05:47.753676+0000","last_clean":"2024-09-10T16:05:47.753676+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:05:47.753676+0000","last_undegraded":"2024-09-10T16:05:47.753676+0000","last_fullsized":"2024-09-10T16:05:47.753676+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'20111","reported_seq":20262,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:05.789129+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:08:05.789129+0000","last_peered":"2024-09-10T16:08:05.789129+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:08:05.789129+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4453318,"num_objects":1089,"num_object_clones":0,"num_object_copies":2178,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1089,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16274,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3234},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":69632758,"num_objects":17010,"num_object_clones":0,"num_object_copies":34020,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38671,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17010,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":275096,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12718,"num_bytes_recovered":33095680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":350924800,"data_stored":350782975,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160722,"ondisk_log_size":160722,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986297,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":541064,"kb_used_data":172016,"kb_used_omap":4,"kb_used_meta":369019,"kb_avail":93830776,"statfs":{"total":96636764160,"available":96082714624,"internally_reserved":0,"allocated":176144384,"data_stored":175841700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":377876138},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,4,2,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659242,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":428036,"kb_used_data":53424,"kb_used_omap":4,"kb_used_meta":374587,"kb_avail":93943804,"statfs":{"total":96636764160,"available":96198455296,"internally_reserved":0,"allocated":54706176,"data_stored":54374124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":383577774},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,3,0,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182525,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":592816,"kb_used_data":119712,"kb_used_omap":4,"kb_used_meta":473083,"kb_avail":93779024,"statfs":{"total":96636764160,"available":96029720576,"internally_reserved":0,"allocated":122585088,"data_stored":122282399,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":484437674},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,3,0,1,4,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":121778176,"data_stored":121742046,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":54038528,"data_stored":53973035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":175108096,"data_stored":175067894,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:08:11.692 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:08:12.246 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:08:12.247 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:08:12.266 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":370,"stamp":"2024-09-10T16:08:11.750753+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":60924681,"num_objects":14886,"num_object_clones":0,"num_object_copies":29772,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38489,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14886,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":277224,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12903,"num_bytes_recovered":33095699,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160851,"ondisk_log_size":160851,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1540864,"kb_used_data":326856,"kb_used_omap":12,"kb_used_meta":1213939,"kb_avail":281574656,"statfs":{"total":289910292480,"available":288332447744,"internally_reserved":0,"allocated":334700544,"data_stored":333756252,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1243073538},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,6,7,12,13,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18378752,"num_objects":-4487,"num_object_clones":0,"num_object_copies":-8974,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-357,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4487,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4487,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":358,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001859"},"pg_stats":[{"pgid":"2.f","version":"48'19775","reported_seq":19858,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:10.310107+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:08:10.310107+0000","last_peered":"2024-09-10T16:08:10.310107+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:08:10.310107+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3674112,"num_objects":897,"num_object_clones":0,"num_object_copies":1794,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":897,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19775,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3362}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"48'19910","reported_seq":20033,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.838378+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:08:10.838378+0000","last_peered":"2024-09-10T16:08:10.838378+0000","last_clean":"2024-09-10T16:08:10.838378+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:08:10.838378+0000","last_undegraded":"2024-09-10T16:08:10.838378+0000","last_fullsized":"2024-09-10T16:08:10.838378+0000","mapping_epoch":35,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3826630,"num_objects":936,"num_object_clones":0,"num_object_copies":1872,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":936,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18015,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'19858","reported_seq":28542,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:08:09.698363+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:08:09.698363+0000","last_peered":"2024-09-10T16:08:09.698363+0000","last_clean":"2024-09-10T16:08:09.698363+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:08:09.698363+0000","last_undegraded":"2024-09-10T16:08:09.698363+0000","last_fullsized":"2024-09-10T16:08:09.698363+0000","mapping_epoch":44,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3846168,"num_objects":940,"num_object_clones":0,"num_object_copies":1880,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":940,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17862,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19775","reported_seq":19866,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.974990+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:08:10.974990+0000","last_peered":"2024-09-10T16:08:10.974990+0000","last_clean":"2024-09-10T16:08:10.974990+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:08:10.974990+0000","last_undegraded":"2024-09-10T16:08:10.974990+0000","last_fullsized":"2024-09-10T16:08:10.974990+0000","mapping_epoch":35,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3624960,"num_objects":885,"num_object_clones":0,"num_object_copies":1770,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":885,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19775,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"48'19958","reported_seq":20096,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.281160+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:08:10.281160+0000","last_peered":"2024-09-10T16:08:10.281160+0000","last_clean":"2024-09-10T16:08:10.281160+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:08:10.281160+0000","last_undegraded":"2024-09-10T16:08:10.281160+0000","last_fullsized":"2024-09-10T16:08:10.281160+0000","mapping_epoch":34,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3822051,"num_objects":934,"num_object_clones":0,"num_object_copies":1868,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":934,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16669,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"48'20136","reported_seq":20282,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:10.301350+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:08:10.301350+0000","last_peered":"2024-09-10T16:08:10.301350+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:08:10.301350+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3789283,"num_objects":926,"num_object_clones":0,"num_object_copies":1852,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":926,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16748,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3445},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"48'20148","reported_seq":20294,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:10.253604+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:08:10.253604+0000","last_peered":"2024-09-10T16:08:10.253604+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:08:10.253604+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3826630,"num_objects":936,"num_object_clones":0,"num_object_copies":1872,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":936,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16647,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3301},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"48'20381","reported_seq":20519,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.180438+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:08:10.180438+0000","last_peered":"2024-09-10T16:08:10.180438+0000","last_clean":"2024-09-10T16:08:10.180438+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:08:10.180438+0000","last_undegraded":"2024-09-10T16:08:10.180438+0000","last_fullsized":"2024-09-10T16:08:10.180438+0000","mapping_epoch":34,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3821568,"num_objects":933,"num_object_clones":0,"num_object_copies":1866,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":933,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17228,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.876943+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:05:19.876943+0000","last_peered":"2024-09-10T16:05:19.876943+0000","last_clean":"2024-09-10T16:05:19.876943+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:05:19.876943+0000","last_undegraded":"2024-09-10T16:05:19.876943+0000","last_fullsized":"2024-09-10T16:05:19.876943+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19512","reported_seq":19662,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:10.988641+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:08:10.988641+0000","last_peered":"2024-09-10T16:08:10.988641+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:08:10.988641+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"43'9500","ondisk_log_start":"43'9500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3871203,"num_objects":946,"num_object_clones":0,"num_object_copies":1892,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":946,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17331,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-3177}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":104,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.838197+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:08:10.838197+0000","last_peered":"2024-09-10T16:08:10.838197+0000","last_clean":"2024-09-10T16:08:10.838197+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:08:10.838197+0000","last_undegraded":"2024-09-10T16:08:10.838197+0000","last_fullsized":"2024-09-10T16:08:10.838197+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"48'19986","reported_seq":20262,"reported_epoch":48,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:10.808603+0000","last_change":"2024-09-10T16:08:09.280891+0000","last_active":"2024-09-10T16:08:10.808603+0000","last_peered":"2024-09-10T16:08:10.808603+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:08:10.808603+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3784704,"num_objects":924,"num_object_clones":0,"num_object_copies":1848,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4189,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":924,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16490,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":68,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3265}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":112,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T16:08:09.841634+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:08:09.841634+0000","last_peered":"2024-09-10T16:08:09.841634+0000","last_clean":"2024-09-10T16:08:09.841634+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:08:09.841634+0000","last_undegraded":"2024-09-10T16:08:09.841634+0000","last_fullsized":"2024-09-10T16:08:09.841634+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"48'19852","reported_seq":19997,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:10.838287+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:08:10.838287+0000","last_peered":"2024-09-10T16:08:10.838287+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:08:10.838287+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3805667,"num_objects":930,"num_object_clones":0,"num_object_copies":1860,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":930,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17532,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-3374}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":86,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:32.574539+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:05:32.574539+0000","last_peered":"2024-09-10T16:05:32.574539+0000","last_clean":"2024-09-10T16:05:32.574539+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:05:32.574539+0000","last_undegraded":"2024-09-10T16:05:32.574539+0000","last_fullsized":"2024-09-10T16:05:32.574539+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'20255","reported_seq":29104,"reported_epoch":49,"state":"active+recovery_wait","last_fresh":"2024-09-10T16:08:10.992801+0000","last_change":"2024-09-10T16:08:10.929169+0000","last_active":"2024-09-10T16:08:10.992801+0000","last_peered":"2024-09-10T16:08:10.992801+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T16:08:10.860438+0000","last_became_peered":"2024-09-10T16:08:10.860438+0000","last_unstale":"2024-09-10T16:08:10.992801+0000","last_undegraded":"2024-09-10T16:08:10.992801+0000","last_fullsized":"2024-09-10T16:08:10.992801+0000","mapping_epoch":48,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3747840,"num_objects":915,"num_object_clones":0,"num_object_copies":1830,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":915,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17809,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","0","1"],"object_location_counts":[{"shards":"1,2","objects":915}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"48'19893","reported_seq":20036,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:10.273364+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:08:10.273364+0000","last_peered":"2024-09-10T16:08:10.273364+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:08:10.273364+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3870720,"num_objects":945,"num_object_clones":0,"num_object_copies":1890,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":945,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16524,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3359}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:32.574559+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:05:32.574559+0000","last_peered":"2024-09-10T16:05:32.574559+0000","last_clean":"2024-09-10T16:05:32.574559+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:05:32.574559+0000","last_undegraded":"2024-09-10T16:05:32.574559+0000","last_fullsized":"2024-09-10T16:05:32.574559+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'19616","reported_seq":28188,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:08:09.643097+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:08:09.643097+0000","last_peered":"2024-09-10T16:08:09.643097+0000","last_clean":"2024-09-10T16:08:09.643097+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:08:09.643097+0000","last_undegraded":"2024-09-10T16:08:09.643097+0000","last_fullsized":"2024-09-10T16:08:09.643097+0000","mapping_epoch":46,"log_start":"43'9600","ondisk_log_start":"43'9600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4022272,"num_objects":982,"num_object_clones":0,"num_object_copies":1964,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":982,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16006,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":104,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.838936+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:08:10.838936+0000","last_peered":"2024-09-10T16:08:10.838936+0000","last_clean":"2024-09-10T16:08:10.838936+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:08:10.838936+0000","last_undegraded":"2024-09-10T16:08:10.838936+0000","last_fullsized":"2024-09-10T16:08:10.838936+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"48'20159","reported_seq":20309,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:10.194979+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:08:10.194979+0000","last_peered":"2024-09-10T16:08:10.194979+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:08:10.194979+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3641344,"num_objects":889,"num_object_clones":0,"num_object_copies":1778,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":889,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16414,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3482}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:32.574572+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:05:32.574572+0000","last_peered":"2024-09-10T16:05:32.574572+0000","last_clean":"2024-09-10T16:05:32.574572+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:05:32.574572+0000","last_undegraded":"2024-09-10T16:05:32.574572+0000","last_fullsized":"2024-09-10T16:05:32.574572+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":81,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:47.753676+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:05:47.753676+0000","last_peered":"2024-09-10T16:05:47.753676+0000","last_clean":"2024-09-10T16:05:47.753676+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:05:47.753676+0000","last_undegraded":"2024-09-10T16:05:47.753676+0000","last_fullsized":"2024-09-10T16:05:47.753676+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"48'20234","reported_seq":20387,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:10.325389+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:08:10.325389+0000","last_peered":"2024-09-10T16:08:10.325389+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:08:10.325389+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3949510,"num_objects":966,"num_object_clones":0,"num_object_copies":1932,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":966,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16397,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3357},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":60924662,"num_objects":14884,"num_object_clones":0,"num_object_copies":29768,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38489,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14884,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":277222,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12901,"num_bytes_recovered":33095680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":332972032,"data_stored":332837433,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160848,"ondisk_log_size":160848,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986298,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":523944,"kb_used_data":167380,"kb_used_omap":4,"kb_used_meta":356539,"kb_avail":93847896,"statfs":{"total":96636764160,"available":96100245504,"internally_reserved":0,"allocated":171397120,"data_stored":171089196,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":365096618},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,6,3,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659243,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":443144,"kb_used_data":46328,"kb_used_omap":4,"kb_used_meta":396795,"kb_avail":93928696,"statfs":{"total":96636764160,"available":96182984704,"internally_reserved":0,"allocated":47439872,"data_stored":47111433,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":406318766},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,6,0,5,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182526,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":573776,"kb_used_data":113148,"kb_used_omap":4,"kb_used_meta":460603,"kb_avail":93798064,"statfs":{"total":96636764160,"available":96049217536,"internally_reserved":0,"allocated":115863552,"data_stored":115555623,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":471658154},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,1,4,6,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":115458048,"data_stored":115421918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":46657536,"data_stored":46599269,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":170856448,"data_stored":170816246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:08:12.268 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:08:12.839 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:08:12.842 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:08:12.863 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":370,"stamp":"2024-09-10T16:08:11.750753+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":60924681,"num_objects":14886,"num_object_clones":0,"num_object_copies":29772,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38489,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14886,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":277224,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12903,"num_bytes_recovered":33095699,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160851,"ondisk_log_size":160851,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1540864,"kb_used_data":326856,"kb_used_omap":12,"kb_used_meta":1213939,"kb_avail":281574656,"statfs":{"total":289910292480,"available":288332447744,"internally_reserved":0,"allocated":334700544,"data_stored":333756252,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1243073538},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,6,7,12,13,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18378752,"num_objects":-4487,"num_object_clones":0,"num_object_copies":-8974,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-357,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4487,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4487,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":358,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001859"},"pg_stats":[{"pgid":"2.f","version":"48'19775","reported_seq":19858,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:10.310107+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:08:10.310107+0000","last_peered":"2024-09-10T16:08:10.310107+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:08:10.310107+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3674112,"num_objects":897,"num_object_clones":0,"num_object_copies":1794,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":897,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19775,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3362}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"48'19910","reported_seq":20033,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.838378+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:08:10.838378+0000","last_peered":"2024-09-10T16:08:10.838378+0000","last_clean":"2024-09-10T16:08:10.838378+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:08:10.838378+0000","last_undegraded":"2024-09-10T16:08:10.838378+0000","last_fullsized":"2024-09-10T16:08:10.838378+0000","mapping_epoch":35,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3826630,"num_objects":936,"num_object_clones":0,"num_object_copies":1872,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":936,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18015,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'19858","reported_seq":28542,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:08:09.698363+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:08:09.698363+0000","last_peered":"2024-09-10T16:08:09.698363+0000","last_clean":"2024-09-10T16:08:09.698363+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:08:09.698363+0000","last_undegraded":"2024-09-10T16:08:09.698363+0000","last_fullsized":"2024-09-10T16:08:09.698363+0000","mapping_epoch":44,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3846168,"num_objects":940,"num_object_clones":0,"num_object_copies":1880,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":940,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17862,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19775","reported_seq":19866,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.974990+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:08:10.974990+0000","last_peered":"2024-09-10T16:08:10.974990+0000","last_clean":"2024-09-10T16:08:10.974990+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:08:10.974990+0000","last_undegraded":"2024-09-10T16:08:10.974990+0000","last_fullsized":"2024-09-10T16:08:10.974990+0000","mapping_epoch":35,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3624960,"num_objects":885,"num_object_clones":0,"num_object_copies":1770,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":885,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19775,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"48'19958","reported_seq":20096,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.281160+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:08:10.281160+0000","last_peered":"2024-09-10T16:08:10.281160+0000","last_clean":"2024-09-10T16:08:10.281160+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:08:10.281160+0000","last_undegraded":"2024-09-10T16:08:10.281160+0000","last_fullsized":"2024-09-10T16:08:10.281160+0000","mapping_epoch":34,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3822051,"num_objects":934,"num_object_clones":0,"num_object_copies":1868,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":934,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16669,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"48'20136","reported_seq":20282,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:10.301350+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:08:10.301350+0000","last_peered":"2024-09-10T16:08:10.301350+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:08:10.301350+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3789283,"num_objects":926,"num_object_clones":0,"num_object_copies":1852,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":926,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16748,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3445},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"48'20148","reported_seq":20294,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:10.253604+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:08:10.253604+0000","last_peered":"2024-09-10T16:08:10.253604+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:08:10.253604+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3826630,"num_objects":936,"num_object_clones":0,"num_object_copies":1872,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":936,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16647,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3301},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"48'20381","reported_seq":20519,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.180438+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:08:10.180438+0000","last_peered":"2024-09-10T16:08:10.180438+0000","last_clean":"2024-09-10T16:08:10.180438+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:08:10.180438+0000","last_undegraded":"2024-09-10T16:08:10.180438+0000","last_fullsized":"2024-09-10T16:08:10.180438+0000","mapping_epoch":34,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3821568,"num_objects":933,"num_object_clones":0,"num_object_copies":1866,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":933,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17228,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.876943+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:05:19.876943+0000","last_peered":"2024-09-10T16:05:19.876943+0000","last_clean":"2024-09-10T16:05:19.876943+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:05:19.876943+0000","last_undegraded":"2024-09-10T16:05:19.876943+0000","last_fullsized":"2024-09-10T16:05:19.876943+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19512","reported_seq":19662,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:10.988641+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:08:10.988641+0000","last_peered":"2024-09-10T16:08:10.988641+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:08:10.988641+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"43'9500","ondisk_log_start":"43'9500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3871203,"num_objects":946,"num_object_clones":0,"num_object_copies":1892,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":946,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17331,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-3177}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":104,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.838197+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:08:10.838197+0000","last_peered":"2024-09-10T16:08:10.838197+0000","last_clean":"2024-09-10T16:08:10.838197+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:08:10.838197+0000","last_undegraded":"2024-09-10T16:08:10.838197+0000","last_fullsized":"2024-09-10T16:08:10.838197+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"48'19986","reported_seq":20262,"reported_epoch":48,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:10.808603+0000","last_change":"2024-09-10T16:08:09.280891+0000","last_active":"2024-09-10T16:08:10.808603+0000","last_peered":"2024-09-10T16:08:10.808603+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:08:10.808603+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3784704,"num_objects":924,"num_object_clones":0,"num_object_copies":1848,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4189,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":924,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16490,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":68,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3265}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":112,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T16:08:09.841634+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:08:09.841634+0000","last_peered":"2024-09-10T16:08:09.841634+0000","last_clean":"2024-09-10T16:08:09.841634+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:08:09.841634+0000","last_undegraded":"2024-09-10T16:08:09.841634+0000","last_fullsized":"2024-09-10T16:08:09.841634+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"48'19852","reported_seq":19997,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:10.838287+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:08:10.838287+0000","last_peered":"2024-09-10T16:08:10.838287+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:08:10.838287+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3805667,"num_objects":930,"num_object_clones":0,"num_object_copies":1860,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":930,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17532,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-3374}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":86,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:32.574539+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:05:32.574539+0000","last_peered":"2024-09-10T16:05:32.574539+0000","last_clean":"2024-09-10T16:05:32.574539+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:05:32.574539+0000","last_undegraded":"2024-09-10T16:05:32.574539+0000","last_fullsized":"2024-09-10T16:05:32.574539+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'20255","reported_seq":29104,"reported_epoch":49,"state":"active+recovery_wait","last_fresh":"2024-09-10T16:08:10.992801+0000","last_change":"2024-09-10T16:08:10.929169+0000","last_active":"2024-09-10T16:08:10.992801+0000","last_peered":"2024-09-10T16:08:10.992801+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T16:08:10.860438+0000","last_became_peered":"2024-09-10T16:08:10.860438+0000","last_unstale":"2024-09-10T16:08:10.992801+0000","last_undegraded":"2024-09-10T16:08:10.992801+0000","last_fullsized":"2024-09-10T16:08:10.992801+0000","mapping_epoch":48,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3747840,"num_objects":915,"num_object_clones":0,"num_object_copies":1830,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":915,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17809,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","0","1"],"object_location_counts":[{"shards":"1,2","objects":915}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"48'19893","reported_seq":20036,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:10.273364+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:08:10.273364+0000","last_peered":"2024-09-10T16:08:10.273364+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:08:10.273364+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3870720,"num_objects":945,"num_object_clones":0,"num_object_copies":1890,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":945,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16524,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3359}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:32.574559+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:05:32.574559+0000","last_peered":"2024-09-10T16:05:32.574559+0000","last_clean":"2024-09-10T16:05:32.574559+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:05:32.574559+0000","last_undegraded":"2024-09-10T16:05:32.574559+0000","last_fullsized":"2024-09-10T16:05:32.574559+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'19616","reported_seq":28188,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:08:09.643097+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:08:09.643097+0000","last_peered":"2024-09-10T16:08:09.643097+0000","last_clean":"2024-09-10T16:08:09.643097+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:08:09.643097+0000","last_undegraded":"2024-09-10T16:08:09.643097+0000","last_fullsized":"2024-09-10T16:08:09.643097+0000","mapping_epoch":46,"log_start":"43'9600","ondisk_log_start":"43'9600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4022272,"num_objects":982,"num_object_clones":0,"num_object_copies":1964,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":982,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16006,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":104,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.838936+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:08:10.838936+0000","last_peered":"2024-09-10T16:08:10.838936+0000","last_clean":"2024-09-10T16:08:10.838936+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:08:10.838936+0000","last_undegraded":"2024-09-10T16:08:10.838936+0000","last_fullsized":"2024-09-10T16:08:10.838936+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"48'20159","reported_seq":20309,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:10.194979+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:08:10.194979+0000","last_peered":"2024-09-10T16:08:10.194979+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:08:10.194979+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3641344,"num_objects":889,"num_object_clones":0,"num_object_copies":1778,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":889,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16414,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3482}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:32.574572+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:05:32.574572+0000","last_peered":"2024-09-10T16:05:32.574572+0000","last_clean":"2024-09-10T16:05:32.574572+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:05:32.574572+0000","last_undegraded":"2024-09-10T16:05:32.574572+0000","last_fullsized":"2024-09-10T16:05:32.574572+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":81,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:47.753676+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:05:47.753676+0000","last_peered":"2024-09-10T16:05:47.753676+0000","last_clean":"2024-09-10T16:05:47.753676+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:05:47.753676+0000","last_undegraded":"2024-09-10T16:05:47.753676+0000","last_fullsized":"2024-09-10T16:05:47.753676+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"48'20234","reported_seq":20387,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:10.325389+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:08:10.325389+0000","last_peered":"2024-09-10T16:08:10.325389+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:08:10.325389+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3949510,"num_objects":966,"num_object_clones":0,"num_object_copies":1932,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":966,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16397,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3357},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":60924662,"num_objects":14884,"num_object_clones":0,"num_object_copies":29768,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38489,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14884,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":277222,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12901,"num_bytes_recovered":33095680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":332972032,"data_stored":332837433,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160848,"ondisk_log_size":160848,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986298,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":523944,"kb_used_data":167380,"kb_used_omap":4,"kb_used_meta":356539,"kb_avail":93847896,"statfs":{"total":96636764160,"available":96100245504,"internally_reserved":0,"allocated":171397120,"data_stored":171089196,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":365096618},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,6,3,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659243,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":443144,"kb_used_data":46328,"kb_used_omap":4,"kb_used_meta":396795,"kb_avail":93928696,"statfs":{"total":96636764160,"available":96182984704,"internally_reserved":0,"allocated":47439872,"data_stored":47111433,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":406318766},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,6,0,5,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182526,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":573776,"kb_used_data":113148,"kb_used_omap":4,"kb_used_meta":460603,"kb_avail":93798064,"statfs":{"total":96636764160,"available":96049217536,"internally_reserved":0,"allocated":115863552,"data_stored":115555623,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":471658154},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,1,4,6,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":115458048,"data_stored":115421918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":46657536,"data_stored":46599269,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":170856448,"data_stored":170816246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:08:12.864 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-10T16:08:12.864 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "48'19775", 'reported_seq': 19858, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:08:10.310107+0000', 'last_change': '2024-09-10T15:59:16.281558+0000', 'last_active': '2024-09-10T16:08:10.310107+0000', 'last_peered': '2024-09-10T16:08:10.310107+0000', 'last_clean': '2024-09-10T15:59:14.132456+0000', 'last_became_active': '2024-09-10T15:59:16.274407+0000', 'last_became_peered': '2024-09-10T15:59:16.274407+0000', 'last_unstale': '2024-09-10T16:08:10.310107+0000', 'last_undegraded': '2024-09-10T15:59:15.851279+0000', 'last_fullsized': '2024-09-10T15:59:15.774356+0000', 'mapping_epoch': 42, 'log_start': "45'9700", 'ondisk_log_start': "45'9700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'objects_scrubbed': 0, 'log_size': 10075, 'log_dups_size': 0, 'ondisk_log_size': 10075, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 3674112, 'num_objects': 897, 'num_object_clones': 0, 'num_object_copies': 1794, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4259, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 897, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 19775, 'num_write_kb': 41344, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3362}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:08:12.864 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-10T16:08:12.864 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "48'20136", 'reported_seq': 20282, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:08:10.301350+0000', 'last_change': '2024-09-10T15:59:16.308264+0000', 'last_active': '2024-09-10T16:08:10.301350+0000', 'last_peered': '2024-09-10T16:08:10.301350+0000', 'last_clean': '2024-09-10T15:59:14.111551+0000', 'last_became_active': '2024-09-10T15:59:16.300040+0000', 'last_became_peered': '2024-09-10T15:59:16.300040+0000', 'last_unstale': '2024-09-10T16:08:10.301350+0000', 'last_undegraded': '2024-09-10T15:59:15.878234+0000', 'last_fullsized': '2024-09-10T15:59:15.769328+0000', 'mapping_epoch': 42, 'log_start': "45'10100", 'ondisk_log_start': "45'10100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3442", 'last_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'objects_scrubbed': 3384, 'log_size': 10036, 'log_dups_size': 0, 'ondisk_log_size': 10036, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1292, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 3789283, 'num_objects': 926, 'num_object_clones': 0, 'num_object_copies': 1852, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4372, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 926, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 16748, 'num_write_kb': 28580, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3445}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:08:12.864 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-10T16:08:12.864 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "48'20148", 'reported_seq': 20294, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:08:10.253604+0000', 'last_change': '2024-09-10T15:59:16.331899+0000', 'last_active': '2024-09-10T16:08:10.253604+0000', 'last_peered': '2024-09-10T16:08:10.253604+0000', 'last_clean': '2024-09-10T15:59:14.139436+0000', 'last_became_active': '2024-09-10T15:59:16.318715+0000', 'last_became_peered': '2024-09-10T15:59:16.318715+0000', 'last_unstale': '2024-09-10T16:08:10.253604+0000', 'last_undegraded': '2024-09-10T15:59:15.877524+0000', 'last_fullsized': '2024-09-10T15:59:15.769042+0000', 'mapping_epoch': 42, 'log_start': "45'10100", 'ondisk_log_start': "45'10100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3537", 'last_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'objects_scrubbed': 3497, 'log_size': 10048, 'log_dups_size': 0, 'ondisk_log_size': 10048, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1224, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 3826630, 'num_objects': 936, 'num_object_clones': 0, 'num_object_copies': 1872, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4237, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 936, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 16647, 'num_write_kb': 28172, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3301}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:08:12.865 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-10T16:08:12.865 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "49'19512", 'reported_seq': 19662, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:08:10.988641+0000', 'last_change': '2024-09-10T15:59:16.175680+0000', 'last_active': '2024-09-10T16:08:10.988641+0000', 'last_peered': '2024-09-10T16:08:10.988641+0000', 'last_clean': '2024-09-10T15:59:14.088187+0000', 'last_became_active': '2024-09-10T15:59:16.165810+0000', 'last_became_peered': '2024-09-10T15:59:16.165810+0000', 'last_unstale': '2024-09-10T16:08:10.988641+0000', 'last_undegraded': '2024-09-10T15:59:15.822098+0000', 'last_fullsized': '2024-09-10T15:59:15.769123+0000', 'mapping_epoch': 42, 'log_start': "43'9500", 'ondisk_log_start': "43'9500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2218", 'last_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'objects_scrubbed': 2175, 'log_size': 10012, 'log_dups_size': 0, 'ondisk_log_size': 10012, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 825, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 3871203, 'num_objects': 946, 'num_object_clones': 0, 'num_object_copies': 1892, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4124, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 946, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 17331, 'num_write_kb': 32204, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': -3177}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:08:12.865 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-10T16:08:12.865 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "48'19986", 'reported_seq': 20262, 'reported_epoch': 48, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:08:10.808603+0000', 'last_change': '2024-09-10T16:08:09.280891+0000', 'last_active': '2024-09-10T16:08:10.808603+0000', 'last_peered': '2024-09-10T16:08:10.808603+0000', 'last_clean': '2024-09-10T15:59:14.129081+0000', 'last_became_active': '2024-09-10T15:59:16.269835+0000', 'last_became_peered': '2024-09-10T15:59:16.269835+0000', 'last_unstale': '2024-09-10T16:08:10.808603+0000', 'last_undegraded': '2024-09-10T15:59:15.860857+0000', 'last_fullsized': '2024-09-10T15:59:15.771410+0000', 'mapping_epoch': 42, 'log_start': "45'9900", 'ondisk_log_start': "45'9900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3557", 'last_scrub_stamp': '2024-09-10T15:59:02.493678+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:02.493678+0000', 'objects_scrubbed': 3488, 'log_size': 10086, 'log_dups_size': 0, 'ondisk_log_size': 10086, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1472, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 3784704, 'num_objects': 924, 'num_object_clones': 0, 'num_object_copies': 1848, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4189, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 924, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16490, 'num_write_kb': 27852, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 68, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3265}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:08:12.865 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-10T16:08:12.865 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "48'19852", 'reported_seq': 19997, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:08:10.838287+0000', 'last_change': '2024-09-10T15:59:16.284311+0000', 'last_active': '2024-09-10T16:08:10.838287+0000', 'last_peered': '2024-09-10T16:08:10.838287+0000', 'last_clean': '2024-09-10T15:59:14.116041+0000', 'last_became_active': '2024-09-10T15:59:16.274543+0000', 'last_became_peered': '2024-09-10T15:59:16.274543+0000', 'last_unstale': '2024-09-10T16:08:10.838287+0000', 'last_undegraded': '2024-09-10T15:59:15.812484+0000', 'last_fullsized': '2024-09-10T15:59:15.770885+0000', 'mapping_epoch': 42, 'log_start': "45'9800", 'ondisk_log_start': "45'9800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2356", 'last_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'objects_scrubbed': 2316, 'log_size': 10052, 'log_dups_size': 0, 'ondisk_log_size': 10052, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 783, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 3805667, 'num_objects': 930, 'num_object_clones': 0, 'num_object_copies': 1860, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 930, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 17532, 'num_write_kb': 32292, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': -3374}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:08:12.865 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-10T16:08:12.865 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "49'20255", 'reported_seq': 29104, 'reported_epoch': 49, 'state': 'active+recovery_wait', 'last_fresh': '2024-09-10T16:08:10.992801+0000', 'last_change': '2024-09-10T16:08:10.929169+0000', 'last_active': '2024-09-10T16:08:10.992801+0000', 'last_peered': '2024-09-10T16:08:10.992801+0000', 'last_clean': '2024-09-10T15:59:14.110654+0000', 'last_became_active': '2024-09-10T16:08:10.860438+0000', 'last_became_peered': '2024-09-10T16:08:10.860438+0000', 'last_unstale': '2024-09-10T16:08:10.992801+0000', 'last_undegraded': '2024-09-10T16:08:10.992801+0000', 'last_fullsized': '2024-09-10T16:08:10.992801+0000', 'mapping_epoch': 48, 'log_start': "45'10200", 'ondisk_log_start': "45'10200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2486", 'last_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'objects_scrubbed': 2440, 'log_size': 10055, 'log_dups_size': 0, 'ondisk_log_size': 10055, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 792, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 3747840, 'num_objects': 915, 'num_object_clones': 0, 'num_object_copies': 1830, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 915, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17809, 'num_write_kb': 32568, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 4355, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': ['2', '0', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 915}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:08:12.865 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-10T16:08:12.865 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "48'19893", 'reported_seq': 20036, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:08:10.273364+0000', 'last_change': '2024-09-10T15:59:16.315641+0000', 'last_active': '2024-09-10T16:08:10.273364+0000', 'last_peered': '2024-09-10T16:08:10.273364+0000', 'last_clean': '2024-09-10T15:59:14.117001+0000', 'last_became_active': '2024-09-10T15:59:16.302245+0000', 'last_became_peered': '2024-09-10T15:59:16.302245+0000', 'last_unstale': '2024-09-10T16:08:10.273364+0000', 'last_undegraded': '2024-09-10T15:59:15.845012+0000', 'last_fullsized': '2024-09-10T15:59:15.770337+0000', 'mapping_epoch': 42, 'log_start': "45'9800", 'ondisk_log_start': "45'9800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3404", 'last_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'objects_scrubbed': 3367, 'log_size': 10093, 'log_dups_size': 0, 'ondisk_log_size': 10093, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 983, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 3870720, 'num_objects': 945, 'num_object_clones': 0, 'num_object_copies': 1890, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 945, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16524, 'num_write_kb': 28204, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3359}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:08:12.866 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-10T16:08:12.866 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "48'20159", 'reported_seq': 20309, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:08:10.194979+0000', 'last_change': '2024-09-10T15:59:16.311291+0000', 'last_active': '2024-09-10T16:08:10.194979+0000', 'last_peered': '2024-09-10T16:08:10.194979+0000', 'last_clean': '2024-09-10T15:59:14.116363+0000', 'last_became_active': '2024-09-10T15:59:16.301244+0000', 'last_became_peered': '2024-09-10T15:59:16.301244+0000', 'last_unstale': '2024-09-10T16:08:10.194979+0000', 'last_undegraded': '2024-09-10T15:59:15.883434+0000', 'last_fullsized': '2024-09-10T15:59:15.772756+0000', 'mapping_epoch': 42, 'log_start': "45'10100", 'ondisk_log_start': "45'10100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3789", 'last_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'objects_scrubbed': 3741, 'log_size': 10059, 'log_dups_size': 0, 'ondisk_log_size': 10059, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1440, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 3641344, 'num_objects': 889, 'num_object_clones': 0, 'num_object_copies': 1778, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4373, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 889, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16414, 'num_write_kb': 27124, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3482}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:08:12.866 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-10T16:08:12.866 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "48'20234", 'reported_seq': 20387, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:08:10.325389+0000', 'last_change': '2024-09-10T15:59:16.313409+0000', 'last_active': '2024-09-10T16:08:10.325389+0000', 'last_peered': '2024-09-10T16:08:10.325389+0000', 'last_clean': '2024-09-10T15:59:14.129780+0000', 'last_became_active': '2024-09-10T15:59:16.300932+0000', 'last_became_peered': '2024-09-10T15:59:16.300932+0000', 'last_unstale': '2024-09-10T16:08:10.325389+0000', 'last_undegraded': '2024-09-10T15:59:15.887823+0000', 'last_fullsized': '2024-09-10T15:59:15.768513+0000', 'mapping_epoch': 42, 'log_start': "45'10200", 'ondisk_log_start': "45'10200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3880", 'last_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'objects_scrubbed': 3827, 'log_size': 10034, 'log_dups_size': 0, 'ondisk_log_size': 10034, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1397, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 3949510, 'num_objects': 966, 'num_object_clones': 0, 'num_object_copies': 1932, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4325, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 966, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16397, 'num_write_kb': 27072, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3357}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:08:12.866 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T16:08:13.533 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:08:13.533 INFO:teuthology.orchestra.run.smithi115.stdout:{"fsid":"7effee29-df05-4351-9741-ee203564ed81","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 38489/29772 objects degraded (129.279%), 9 pgs degraded, 9 pgs undersized","count":18},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":657,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":49,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983954,"num_in_osds":3,"osd_in_since":1725983947,"num_remapped_pgs":9},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":8},{"state_name":"active+recovering+undersized+degraded+remapped","count":1},{"state_name":"active+recovery_wait","count":1}],"num_pgs":24,"num_pools":2,"num_objects":14886,"data_bytes":60924681,"bytes_used":1577844736,"bytes_avail":288332447744,"bytes_total":289910292480,"degraded_objects":38489,"degraded_total":29772,"degraded_ratio":1.2927918849926108,"recovering_objects_per_sec":44,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":358,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":560},"fsmap":{"epoch":1,"btime":"2024-09-10T15:57:14:967120+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T16:00:13.560776+0000","services":{}},"progress_events":{}} 2024-09-10T16:08:13.559 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-10T16:08:13.559 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:08:14.149 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:08:14.152 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:08:14.177 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":371,"stamp":"2024-09-10T16:08:13.751185+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":60924681,"num_objects":14886,"num_object_clones":0,"num_object_copies":29772,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38489,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14886,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":277224,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12903,"num_bytes_recovered":33095699,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160851,"ondisk_log_size":160851,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1540864,"kb_used_data":326856,"kb_used_omap":12,"kb_used_meta":1213939,"kb_avail":281574656,"statfs":{"total":289910292480,"available":288332447744,"internally_reserved":0,"allocated":334700544,"data_stored":333756252,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1243073538},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,6,7,12,13,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-17223680,"num_objects":-4205,"num_object_clones":0,"num_object_copies":-8410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-357,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4205,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4205,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":358,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001870"},"pg_stats":[{"pgid":"2.f","version":"48'19775","reported_seq":19858,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:10.310107+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:08:10.310107+0000","last_peered":"2024-09-10T16:08:10.310107+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:08:10.310107+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3674112,"num_objects":897,"num_object_clones":0,"num_object_copies":1794,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":897,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19775,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3362}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"48'19910","reported_seq":20033,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.838378+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:08:10.838378+0000","last_peered":"2024-09-10T16:08:10.838378+0000","last_clean":"2024-09-10T16:08:10.838378+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:08:10.838378+0000","last_undegraded":"2024-09-10T16:08:10.838378+0000","last_fullsized":"2024-09-10T16:08:10.838378+0000","mapping_epoch":35,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3826630,"num_objects":936,"num_object_clones":0,"num_object_copies":1872,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":936,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18015,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'19858","reported_seq":28542,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:08:09.698363+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:08:09.698363+0000","last_peered":"2024-09-10T16:08:09.698363+0000","last_clean":"2024-09-10T16:08:09.698363+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:08:09.698363+0000","last_undegraded":"2024-09-10T16:08:09.698363+0000","last_fullsized":"2024-09-10T16:08:09.698363+0000","mapping_epoch":44,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3846168,"num_objects":940,"num_object_clones":0,"num_object_copies":1880,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":940,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17862,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19775","reported_seq":19866,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.974990+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:08:10.974990+0000","last_peered":"2024-09-10T16:08:10.974990+0000","last_clean":"2024-09-10T16:08:10.974990+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:08:10.974990+0000","last_undegraded":"2024-09-10T16:08:10.974990+0000","last_fullsized":"2024-09-10T16:08:10.974990+0000","mapping_epoch":35,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3624960,"num_objects":885,"num_object_clones":0,"num_object_copies":1770,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":885,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19775,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"48'19958","reported_seq":20096,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.281160+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:08:10.281160+0000","last_peered":"2024-09-10T16:08:10.281160+0000","last_clean":"2024-09-10T16:08:10.281160+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:08:10.281160+0000","last_undegraded":"2024-09-10T16:08:10.281160+0000","last_fullsized":"2024-09-10T16:08:10.281160+0000","mapping_epoch":34,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3822051,"num_objects":934,"num_object_clones":0,"num_object_copies":1868,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":934,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16669,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"48'20136","reported_seq":20282,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:10.301350+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:08:10.301350+0000","last_peered":"2024-09-10T16:08:10.301350+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:08:10.301350+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3789283,"num_objects":926,"num_object_clones":0,"num_object_copies":1852,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":926,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16748,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3445},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"48'20148","reported_seq":20294,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:10.253604+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:08:10.253604+0000","last_peered":"2024-09-10T16:08:10.253604+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:08:10.253604+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3826630,"num_objects":936,"num_object_clones":0,"num_object_copies":1872,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":936,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16647,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3301},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"48'20381","reported_seq":20519,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.180438+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:08:10.180438+0000","last_peered":"2024-09-10T16:08:10.180438+0000","last_clean":"2024-09-10T16:08:10.180438+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:08:10.180438+0000","last_undegraded":"2024-09-10T16:08:10.180438+0000","last_fullsized":"2024-09-10T16:08:10.180438+0000","mapping_epoch":34,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3821568,"num_objects":933,"num_object_clones":0,"num_object_copies":1866,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":933,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17228,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:19.876943+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:05:19.876943+0000","last_peered":"2024-09-10T16:05:19.876943+0000","last_clean":"2024-09-10T16:05:19.876943+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:05:19.876943+0000","last_undegraded":"2024-09-10T16:05:19.876943+0000","last_fullsized":"2024-09-10T16:05:19.876943+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19512","reported_seq":19662,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:10.988641+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:08:10.988641+0000","last_peered":"2024-09-10T16:08:10.988641+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:08:10.988641+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"43'9500","ondisk_log_start":"43'9500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3871203,"num_objects":946,"num_object_clones":0,"num_object_copies":1892,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":946,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17331,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-3177}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":104,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.838197+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:08:10.838197+0000","last_peered":"2024-09-10T16:08:10.838197+0000","last_clean":"2024-09-10T16:08:10.838197+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:08:10.838197+0000","last_undegraded":"2024-09-10T16:08:10.838197+0000","last_fullsized":"2024-09-10T16:08:10.838197+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"48'19986","reported_seq":20262,"reported_epoch":48,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:10.808603+0000","last_change":"2024-09-10T16:08:09.280891+0000","last_active":"2024-09-10T16:08:10.808603+0000","last_peered":"2024-09-10T16:08:10.808603+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:08:10.808603+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3784704,"num_objects":924,"num_object_clones":0,"num_object_copies":1848,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4189,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":924,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16490,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":68,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3265}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":112,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T16:08:09.841634+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:08:09.841634+0000","last_peered":"2024-09-10T16:08:09.841634+0000","last_clean":"2024-09-10T16:08:09.841634+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:08:09.841634+0000","last_undegraded":"2024-09-10T16:08:09.841634+0000","last_fullsized":"2024-09-10T16:08:09.841634+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"48'19852","reported_seq":19997,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:10.838287+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:08:10.838287+0000","last_peered":"2024-09-10T16:08:10.838287+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:08:10.838287+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3805667,"num_objects":930,"num_object_clones":0,"num_object_copies":1860,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":930,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17532,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-3374}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":86,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:32.574539+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:05:32.574539+0000","last_peered":"2024-09-10T16:05:32.574539+0000","last_clean":"2024-09-10T16:05:32.574539+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:05:32.574539+0000","last_undegraded":"2024-09-10T16:05:32.574539+0000","last_fullsized":"2024-09-10T16:05:32.574539+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'20255","reported_seq":29104,"reported_epoch":49,"state":"active+recovery_wait","last_fresh":"2024-09-10T16:08:10.992801+0000","last_change":"2024-09-10T16:08:10.929169+0000","last_active":"2024-09-10T16:08:10.992801+0000","last_peered":"2024-09-10T16:08:10.992801+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T16:08:10.860438+0000","last_became_peered":"2024-09-10T16:08:10.860438+0000","last_unstale":"2024-09-10T16:08:10.992801+0000","last_undegraded":"2024-09-10T16:08:10.992801+0000","last_fullsized":"2024-09-10T16:08:10.992801+0000","mapping_epoch":48,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3747840,"num_objects":915,"num_object_clones":0,"num_object_copies":1830,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":915,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17809,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","0","1"],"object_location_counts":[{"shards":"1,2","objects":915}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"48'19893","reported_seq":20036,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:10.273364+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:08:10.273364+0000","last_peered":"2024-09-10T16:08:10.273364+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:08:10.273364+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3870720,"num_objects":945,"num_object_clones":0,"num_object_copies":1890,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":945,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16524,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3359}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:32.574559+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:05:32.574559+0000","last_peered":"2024-09-10T16:05:32.574559+0000","last_clean":"2024-09-10T16:05:32.574559+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:05:32.574559+0000","last_undegraded":"2024-09-10T16:05:32.574559+0000","last_fullsized":"2024-09-10T16:05:32.574559+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'19616","reported_seq":28188,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:08:09.643097+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:08:09.643097+0000","last_peered":"2024-09-10T16:08:09.643097+0000","last_clean":"2024-09-10T16:08:09.643097+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:08:09.643097+0000","last_undegraded":"2024-09-10T16:08:09.643097+0000","last_fullsized":"2024-09-10T16:08:09.643097+0000","mapping_epoch":46,"log_start":"43'9600","ondisk_log_start":"43'9600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4022272,"num_objects":982,"num_object_clones":0,"num_object_copies":1964,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":982,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16006,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":104,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.838936+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:08:10.838936+0000","last_peered":"2024-09-10T16:08:10.838936+0000","last_clean":"2024-09-10T16:08:10.838936+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:08:10.838936+0000","last_undegraded":"2024-09-10T16:08:10.838936+0000","last_fullsized":"2024-09-10T16:08:10.838936+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"48'20159","reported_seq":20309,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:10.194979+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:08:10.194979+0000","last_peered":"2024-09-10T16:08:10.194979+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:08:10.194979+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3641344,"num_objects":889,"num_object_clones":0,"num_object_copies":1778,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":889,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16414,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3482}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:32.574572+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:05:32.574572+0000","last_peered":"2024-09-10T16:05:32.574572+0000","last_clean":"2024-09-10T16:05:32.574572+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:05:32.574572+0000","last_undegraded":"2024-09-10T16:05:32.574572+0000","last_fullsized":"2024-09-10T16:05:32.574572+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":81,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T16:05:47.753676+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:05:47.753676+0000","last_peered":"2024-09-10T16:05:47.753676+0000","last_clean":"2024-09-10T16:05:47.753676+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:05:47.753676+0000","last_undegraded":"2024-09-10T16:05:47.753676+0000","last_fullsized":"2024-09-10T16:05:47.753676+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"48'20234","reported_seq":20387,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:10.325389+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:08:10.325389+0000","last_peered":"2024-09-10T16:08:10.325389+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:08:10.325389+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3949510,"num_objects":966,"num_object_clones":0,"num_object_copies":1932,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":966,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16397,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3357},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":60924662,"num_objects":14884,"num_object_clones":0,"num_object_copies":29768,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38489,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14884,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":277222,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12901,"num_bytes_recovered":33095680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":332972032,"data_stored":332837433,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160848,"ondisk_log_size":160848,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986298,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":523944,"kb_used_data":167380,"kb_used_omap":4,"kb_used_meta":356539,"kb_avail":93847896,"statfs":{"total":96636764160,"available":96100245504,"internally_reserved":0,"allocated":171397120,"data_stored":171089196,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":365096618},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,6,3,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659243,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":443144,"kb_used_data":46328,"kb_used_omap":4,"kb_used_meta":396795,"kb_avail":93928696,"statfs":{"total":96636764160,"available":96182984704,"internally_reserved":0,"allocated":47439872,"data_stored":47111433,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":406318766},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,6,0,5,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182526,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":573776,"kb_used_data":113148,"kb_used_omap":4,"kb_used_meta":460603,"kb_avail":93798064,"statfs":{"total":96636764160,"available":96049217536,"internally_reserved":0,"allocated":115863552,"data_stored":115555623,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":471658154},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,1,4,6,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":115458048,"data_stored":115421918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":46657536,"data_stored":46599269,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":170856448,"data_stored":170816246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:08:17.180 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:08:17.748 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:08:17.750 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:08:17.774 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":372,"stamp":"2024-09-10T16:08:15.751582+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":59843337,"num_objects":14622,"num_object_clones":0,"num_object_copies":29244,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38489,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14622,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":277488,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12903,"num_bytes_recovered":33095699,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160915,"ondisk_log_size":160915,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1544664,"kb_used_data":320916,"kb_used_omap":12,"kb_used_meta":1223667,"kb_avail":281570856,"statfs":{"total":289910292480,"available":288328556544,"internally_reserved":0,"allocated":328617984,"data_stored":327666804,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1253035010},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,3,9,9,8,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-9789440,"num_objects":-2390,"num_object_clones":0,"num_object_copies":-4780,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-182,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-2390,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2390,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":183,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001726"},"pg_stats":[{"pgid":"2.f","version":"48'19775","reported_seq":19858,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:10.310107+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:08:10.310107+0000","last_peered":"2024-09-10T16:08:10.310107+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:08:10.310107+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3674112,"num_objects":897,"num_object_clones":0,"num_object_copies":1794,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":897,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19775,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3362}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"48'19910","reported_seq":20033,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.838378+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:08:10.838378+0000","last_peered":"2024-09-10T16:08:10.838378+0000","last_clean":"2024-09-10T16:08:10.838378+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:08:10.838378+0000","last_undegraded":"2024-09-10T16:08:10.838378+0000","last_fullsized":"2024-09-10T16:08:10.838378+0000","mapping_epoch":35,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3826630,"num_objects":936,"num_object_clones":0,"num_object_copies":1872,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":936,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18015,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19985","reported_seq":28673,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:14.694212+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:08:14.694212+0000","last_peered":"2024-09-10T16:08:14.694212+0000","last_clean":"2024-09-10T16:08:14.694212+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:08:14.694212+0000","last_undegraded":"2024-09-10T16:08:14.694212+0000","last_fullsized":"2024-09-10T16:08:14.694212+0000","mapping_epoch":44,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3325976,"num_objects":813,"num_object_clones":0,"num_object_copies":1626,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":813,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17989,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19775","reported_seq":19866,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.974990+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:08:10.974990+0000","last_peered":"2024-09-10T16:08:10.974990+0000","last_clean":"2024-09-10T16:08:10.974990+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:08:10.974990+0000","last_undegraded":"2024-09-10T16:08:10.974990+0000","last_fullsized":"2024-09-10T16:08:10.974990+0000","mapping_epoch":35,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3624960,"num_objects":885,"num_object_clones":0,"num_object_copies":1770,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":885,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19775,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"48'19958","reported_seq":20096,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.281160+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:08:10.281160+0000","last_peered":"2024-09-10T16:08:10.281160+0000","last_clean":"2024-09-10T16:08:10.281160+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:08:10.281160+0000","last_undegraded":"2024-09-10T16:08:10.281160+0000","last_fullsized":"2024-09-10T16:08:10.281160+0000","mapping_epoch":34,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3822051,"num_objects":934,"num_object_clones":0,"num_object_copies":1868,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":934,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16669,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"48'20136","reported_seq":20282,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:10.301350+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:08:10.301350+0000","last_peered":"2024-09-10T16:08:10.301350+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:08:10.301350+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3789283,"num_objects":926,"num_object_clones":0,"num_object_copies":1852,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":926,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16748,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3445},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"48'20148","reported_seq":20294,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:10.253604+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:08:10.253604+0000","last_peered":"2024-09-10T16:08:10.253604+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:08:10.253604+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3826630,"num_objects":936,"num_object_clones":0,"num_object_copies":1872,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":936,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16647,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3301},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"48'20381","reported_seq":20519,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.180438+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:08:10.180438+0000","last_peered":"2024-09-10T16:08:10.180438+0000","last_clean":"2024-09-10T16:08:10.180438+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:08:10.180438+0000","last_undegraded":"2024-09-10T16:08:10.180438+0000","last_fullsized":"2024-09-10T16:08:10.180438+0000","mapping_epoch":34,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3821568,"num_objects":933,"num_object_clones":0,"num_object_copies":1866,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":933,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17228,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.838857+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:08:10.838857+0000","last_peered":"2024-09-10T16:08:10.838857+0000","last_clean":"2024-09-10T16:08:10.838857+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:08:10.838857+0000","last_undegraded":"2024-09-10T16:08:10.838857+0000","last_fullsized":"2024-09-10T16:08:10.838857+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19512","reported_seq":19662,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:10.988641+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:08:10.988641+0000","last_peered":"2024-09-10T16:08:10.988641+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:08:10.988641+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"43'9500","ondisk_log_start":"43'9500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3871203,"num_objects":946,"num_object_clones":0,"num_object_copies":1892,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":946,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17331,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-3177}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":104,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.838197+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:08:10.838197+0000","last_peered":"2024-09-10T16:08:10.838197+0000","last_clean":"2024-09-10T16:08:10.838197+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:08:10.838197+0000","last_undegraded":"2024-09-10T16:08:10.838197+0000","last_fullsized":"2024-09-10T16:08:10.838197+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"48'19986","reported_seq":20262,"reported_epoch":48,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:10.808603+0000","last_change":"2024-09-10T16:08:09.280891+0000","last_active":"2024-09-10T16:08:10.808603+0000","last_peered":"2024-09-10T16:08:10.808603+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:08:10.808603+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3784704,"num_objects":924,"num_object_clones":0,"num_object_copies":1848,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4189,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":924,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16490,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":68,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3265}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":112,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T16:08:09.841634+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:08:09.841634+0000","last_peered":"2024-09-10T16:08:09.841634+0000","last_clean":"2024-09-10T16:08:09.841634+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:08:09.841634+0000","last_undegraded":"2024-09-10T16:08:09.841634+0000","last_fullsized":"2024-09-10T16:08:09.841634+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"48'19852","reported_seq":19997,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:10.838287+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:08:10.838287+0000","last_peered":"2024-09-10T16:08:10.838287+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:08:10.838287+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3805667,"num_objects":930,"num_object_clones":0,"num_object_copies":1860,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":930,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17532,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-3374}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":90,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.837937+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:08:10.837937+0000","last_peered":"2024-09-10T16:08:10.837937+0000","last_clean":"2024-09-10T16:08:10.837937+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:08:10.837937+0000","last_undegraded":"2024-09-10T16:08:10.837937+0000","last_fullsized":"2024-09-10T16:08:10.837937+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'20255","reported_seq":29104,"reported_epoch":49,"state":"active+recovery_wait","last_fresh":"2024-09-10T16:08:10.992801+0000","last_change":"2024-09-10T16:08:10.929169+0000","last_active":"2024-09-10T16:08:10.992801+0000","last_peered":"2024-09-10T16:08:10.992801+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T16:08:10.860438+0000","last_became_peered":"2024-09-10T16:08:10.860438+0000","last_unstale":"2024-09-10T16:08:10.992801+0000","last_undegraded":"2024-09-10T16:08:10.992801+0000","last_fullsized":"2024-09-10T16:08:10.992801+0000","mapping_epoch":48,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3747840,"num_objects":915,"num_object_clones":0,"num_object_copies":1830,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":915,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17809,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","0","1"],"object_location_counts":[{"shards":"1,2","objects":915}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"48'19893","reported_seq":20036,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:10.273364+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:08:10.273364+0000","last_peered":"2024-09-10T16:08:10.273364+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:08:10.273364+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3870720,"num_objects":945,"num_object_clones":0,"num_object_copies":1890,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":945,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16524,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3359}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.838546+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:08:10.838546+0000","last_peered":"2024-09-10T16:08:10.838546+0000","last_clean":"2024-09-10T16:08:10.838546+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:08:10.838546+0000","last_undegraded":"2024-09-10T16:08:10.838546+0000","last_fullsized":"2024-09-10T16:08:10.838546+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19753","reported_seq":28329,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:14.743651+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:08:14.743651+0000","last_peered":"2024-09-10T16:08:14.743651+0000","last_clean":"2024-09-10T16:08:14.743651+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:08:14.743651+0000","last_undegraded":"2024-09-10T16:08:14.743651+0000","last_fullsized":"2024-09-10T16:08:14.743651+0000","mapping_epoch":46,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3461120,"num_objects":845,"num_object_clones":0,"num_object_copies":1690,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":845,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16143,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":104,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.838936+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:08:10.838936+0000","last_peered":"2024-09-10T16:08:10.838936+0000","last_clean":"2024-09-10T16:08:10.838936+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:08:10.838936+0000","last_undegraded":"2024-09-10T16:08:10.838936+0000","last_fullsized":"2024-09-10T16:08:10.838936+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"48'20159","reported_seq":20309,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:10.194979+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:08:10.194979+0000","last_peered":"2024-09-10T16:08:10.194979+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:08:10.194979+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3641344,"num_objects":889,"num_object_clones":0,"num_object_copies":1778,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":889,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16414,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3482}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.839114+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:08:10.839114+0000","last_peered":"2024-09-10T16:08:10.839114+0000","last_clean":"2024-09-10T16:08:10.839114+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:08:10.839114+0000","last_undegraded":"2024-09-10T16:08:10.839114+0000","last_fullsized":"2024-09-10T16:08:10.839114+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":85,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.838633+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:08:10.838633+0000","last_peered":"2024-09-10T16:08:10.838633+0000","last_clean":"2024-09-10T16:08:10.838633+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:08:10.838633+0000","last_undegraded":"2024-09-10T16:08:10.838633+0000","last_fullsized":"2024-09-10T16:08:10.838633+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"48'20234","reported_seq":20387,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:10.325389+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:08:10.325389+0000","last_peered":"2024-09-10T16:08:10.325389+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:08:10.325389+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3949510,"num_objects":966,"num_object_clones":0,"num_object_copies":1932,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":966,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16397,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3357},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":59843318,"num_objects":14620,"num_object_clones":0,"num_object_copies":29240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38489,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14620,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":277486,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12901,"num_bytes_recovered":33095680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":326664192,"data_stored":326529593,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160912,"ondisk_log_size":160912,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986298,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":523944,"kb_used_data":167380,"kb_used_omap":4,"kb_used_meta":356539,"kb_avail":93847896,"statfs":{"total":96636764160,"available":96100245504,"internally_reserved":0,"allocated":171397120,"data_stored":171089196,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":365096618},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,6,3,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659244,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":446944,"kb_used_data":40388,"kb_used_omap":4,"kb_used_meta":406523,"kb_avail":93924896,"statfs":{"total":96636764160,"available":96179093504,"internally_reserved":0,"allocated":41357312,"data_stored":41021985,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":416280238},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,2,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182526,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":573776,"kb_used_data":113148,"kb_used_omap":4,"kb_used_meta":460603,"kb_avail":93798064,"statfs":{"total":96636764160,"available":96049217536,"internally_reserved":0,"allocated":115863552,"data_stored":115555623,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":471658154},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,1,4,6,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":115458048,"data_stored":115421918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40349696,"data_stored":40291429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":170856448,"data_stored":170816246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:08:17.775 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:08:18.337 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:08:18.337 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:08:18.363 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":373,"stamp":"2024-09-10T16:08:17.754575+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":51688201,"num_objects":12631,"num_object_clones":0,"num_object_copies":25262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38417,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12631,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":279479,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12975,"num_bytes_recovered":33095699,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160906,"ondisk_log_size":160906,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1538304,"kb_used_data":310388,"kb_used_omap":12,"kb_used_meta":1227827,"kb_avail":281577216,"statfs":{"total":289910292480,"available":288335069184,"internally_reserved":0,"allocated":317837312,"data_stored":316875610,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1257294850},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,5,4,4,7,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-16711680,"num_objects":-4080,"num_object_clones":0,"num_object_copies":-8160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-254,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4080,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4080,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":255,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.004256"},"pg_stats":[{"pgid":"2.f","version":"49'19905","reported_seq":19990,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:15.723842+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:08:15.723842+0000","last_peered":"2024-09-10T16:08:15.723842+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:08:15.723842+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3141632,"num_objects":767,"num_object_clones":0,"num_object_copies":1534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":767,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19905,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3492}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20060","reported_seq":20183,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:15.964932+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:08:15.964932+0000","last_peered":"2024-09-10T16:08:15.964932+0000","last_clean":"2024-09-10T16:08:15.964932+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:08:15.964932+0000","last_undegraded":"2024-09-10T16:08:15.964932+0000","last_fullsized":"2024-09-10T16:08:15.964932+0000","mapping_epoch":35,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3212230,"num_objects":786,"num_object_clones":0,"num_object_copies":1572,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":786,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18165,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19985","reported_seq":28673,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:14.694212+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:08:14.694212+0000","last_peered":"2024-09-10T16:08:14.694212+0000","last_clean":"2024-09-10T16:08:14.694212+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:08:14.694212+0000","last_undegraded":"2024-09-10T16:08:14.694212+0000","last_fullsized":"2024-09-10T16:08:14.694212+0000","mapping_epoch":44,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3325976,"num_objects":813,"num_object_clones":0,"num_object_copies":1626,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":813,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17989,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19902","reported_seq":19993,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:15.997632+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:08:15.997632+0000","last_peered":"2024-09-10T16:08:15.997632+0000","last_clean":"2024-09-10T16:08:15.997632+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:08:15.997632+0000","last_undegraded":"2024-09-10T16:08:15.997632+0000","last_fullsized":"2024-09-10T16:08:15.997632+0000","mapping_epoch":35,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3104768,"num_objects":758,"num_object_clones":0,"num_object_copies":1516,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":758,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19902,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20103","reported_seq":20243,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:15.798072+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:08:15.798072+0000","last_peered":"2024-09-10T16:08:15.798072+0000","last_clean":"2024-09-10T16:08:15.798072+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:08:15.798072+0000","last_undegraded":"2024-09-10T16:08:15.798072+0000","last_fullsized":"2024-09-10T16:08:15.798072+0000","mapping_epoch":34,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3228131,"num_objects":789,"num_object_clones":0,"num_object_copies":1578,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":789,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16814,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20296","reported_seq":20444,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:15.772208+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:08:15.772208+0000","last_peered":"2024-09-10T16:08:15.772208+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:08:15.772208+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3133923,"num_objects":766,"num_object_clones":0,"num_object_copies":1532,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":766,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16908,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3605},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'20295","reported_seq":20442,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:15.720757+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:08:15.720757+0000","last_peered":"2024-09-10T16:08:15.720757+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:08:15.720757+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3224518,"num_objects":789,"num_object_clones":0,"num_object_copies":1578,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":789,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16794,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3448},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20529","reported_seq":20669,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:15.791179+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:08:15.791179+0000","last_peered":"2024-09-10T16:08:15.791179+0000","last_clean":"2024-09-10T16:08:15.791179+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:08:15.791179+0000","last_undegraded":"2024-09-10T16:08:15.791179+0000","last_fullsized":"2024-09-10T16:08:15.791179+0000","mapping_epoch":34,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3215360,"num_objects":785,"num_object_clones":0,"num_object_copies":1570,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":785,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17376,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.838857+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:08:10.838857+0000","last_peered":"2024-09-10T16:08:10.838857+0000","last_clean":"2024-09-10T16:08:10.838857+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:08:10.838857+0000","last_undegraded":"2024-09-10T16:08:10.838857+0000","last_fullsized":"2024-09-10T16:08:10.838857+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19660","reported_seq":19810,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:15.847575+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:08:15.847575+0000","last_peered":"2024-09-10T16:08:15.847575+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:08:15.847575+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3264995,"num_objects":798,"num_object_clones":0,"num_object_copies":1596,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":798,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17479,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-3325}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":104,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.838197+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:08:10.838197+0000","last_peered":"2024-09-10T16:08:10.838197+0000","last_clean":"2024-09-10T16:08:10.838197+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:08:10.838197+0000","last_undegraded":"2024-09-10T16:08:10.838197+0000","last_fullsized":"2024-09-10T16:08:10.838197+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20130","reported_seq":20551,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:15.810658+0000","last_change":"2024-09-10T16:08:09.280891+0000","last_active":"2024-09-10T16:08:15.810658+0000","last_peered":"2024-09-10T16:08:15.810658+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:08:15.810658+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3194880,"num_objects":780,"num_object_clones":0,"num_object_copies":1560,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4117,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":780,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16634,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":140,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3337}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":114,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.840262+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:08:10.840262+0000","last_peered":"2024-09-10T16:08:10.840262+0000","last_clean":"2024-09-10T16:08:10.840262+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:08:10.840262+0000","last_undegraded":"2024-09-10T16:08:10.840262+0000","last_fullsized":"2024-09-10T16:08:10.840262+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'19986","reported_seq":20129,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:16.010738+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:08:16.010738+0000","last_peered":"2024-09-10T16:08:16.010738+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:08:16.010738+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3256803,"num_objects":796,"num_object_clones":0,"num_object_copies":1592,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":796,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17666,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-3508}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":90,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.837937+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:08:10.837937+0000","last_peered":"2024-09-10T16:08:10.837937+0000","last_clean":"2024-09-10T16:08:10.837937+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:08:10.837937+0000","last_undegraded":"2024-09-10T16:08:10.837937+0000","last_fullsized":"2024-09-10T16:08:10.837937+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'20391","reported_seq":29245,"reported_epoch":49,"state":"active+recovery_wait","last_fresh":"2024-09-10T16:08:15.993183+0000","last_change":"2024-09-10T16:08:10.929169+0000","last_active":"2024-09-10T16:08:15.993183+0000","last_peered":"2024-09-10T16:08:15.993183+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T16:08:10.860438+0000","last_became_peered":"2024-09-10T16:08:10.860438+0000","last_unstale":"2024-09-10T16:08:15.993183+0000","last_undegraded":"2024-09-10T16:08:15.993183+0000","last_fullsized":"2024-09-10T16:08:15.993183+0000","mapping_epoch":48,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3190784,"num_objects":779,"num_object_clones":0,"num_object_copies":1558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":779,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17945,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","0","1"],"object_location_counts":[{"shards":"1,2","objects":779}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20037","reported_seq":20182,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:15.723089+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:08:15.723089+0000","last_peered":"2024-09-10T16:08:15.723089+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:08:15.723089+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3280896,"num_objects":801,"num_object_clones":0,"num_object_copies":1602,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":801,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16668,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3503}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.838546+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:08:10.838546+0000","last_peered":"2024-09-10T16:08:10.838546+0000","last_clean":"2024-09-10T16:08:10.838546+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:08:10.838546+0000","last_undegraded":"2024-09-10T16:08:10.838546+0000","last_fullsized":"2024-09-10T16:08:10.838546+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19753","reported_seq":28329,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:14.743651+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:08:14.743651+0000","last_peered":"2024-09-10T16:08:14.743651+0000","last_clean":"2024-09-10T16:08:14.743651+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:08:14.743651+0000","last_undegraded":"2024-09-10T16:08:14.743651+0000","last_fullsized":"2024-09-10T16:08:14.743651+0000","mapping_epoch":46,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3461120,"num_objects":845,"num_object_clones":0,"num_object_copies":1690,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":845,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16143,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":104,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.838936+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:08:10.838936+0000","last_peered":"2024-09-10T16:08:10.838936+0000","last_clean":"2024-09-10T16:08:10.838936+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:08:10.838936+0000","last_undegraded":"2024-09-10T16:08:10.838936+0000","last_fullsized":"2024-09-10T16:08:10.838936+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20294","reported_seq":20446,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:15.784442+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:08:15.784442+0000","last_peered":"2024-09-10T16:08:15.784442+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:08:15.784442+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3088384,"num_objects":754,"num_object_clones":0,"num_object_copies":1508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16549,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3617}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.839114+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:08:10.839114+0000","last_peered":"2024-09-10T16:08:10.839114+0000","last_clean":"2024-09-10T16:08:10.839114+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:08:10.839114+0000","last_undegraded":"2024-09-10T16:08:10.839114+0000","last_fullsized":"2024-09-10T16:08:10.839114+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":85,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.838633+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:08:10.838633+0000","last_peered":"2024-09-10T16:08:10.838633+0000","last_clean":"2024-09-10T16:08:10.838633+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:08:10.838633+0000","last_undegraded":"2024-09-10T16:08:10.838633+0000","last_fullsized":"2024-09-10T16:08:10.838633+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'20377","reported_seq":20529,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:15.805584+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:08:15.805584+0000","last_peered":"2024-09-10T16:08:15.805584+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:08:15.805584+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3363782,"num_objects":823,"num_object_clones":0,"num_object_copies":1646,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":823,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16540,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3500},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":51688182,"num_objects":12629,"num_object_clones":0,"num_object_copies":25258,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38417,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12629,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":279477,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12973,"num_bytes_recovered":33095680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":314904576,"data_stored":314769977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160903,"ondisk_log_size":160903,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986299,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":519960,"kb_used_data":163392,"kb_used_omap":4,"kb_used_meta":356539,"kb_avail":93851880,"statfs":{"total":96636764160,"available":96104325120,"internally_reserved":0,"allocated":167313408,"data_stored":167000223,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":365096618},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659244,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":446944,"kb_used_data":40388,"kb_used_omap":4,"kb_used_meta":406523,"kb_avail":93924896,"statfs":{"total":96636764160,"available":96179093504,"internally_reserved":0,"allocated":41357312,"data_stored":41021985,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":416280238},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,2,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182527,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":571400,"kb_used_data":106608,"kb_used_omap":4,"kb_used_meta":464763,"kb_avail":93800440,"statfs":{"total":96636764160,"available":96051650560,"internally_reserved":0,"allocated":109166592,"data_stored":108853402,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":475917994},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,2,2,0,7,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":108384256,"data_stored":108348126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40349696,"data_stored":40291429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":166170624,"data_stored":166130422,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:08:18.365 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-10T16:08:18.365 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "49'19905", 'reported_seq': 19990, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:08:15.723842+0000', 'last_change': '2024-09-10T15:59:16.281558+0000', 'last_active': '2024-09-10T16:08:15.723842+0000', 'last_peered': '2024-09-10T16:08:15.723842+0000', 'last_clean': '2024-09-10T15:59:14.132456+0000', 'last_became_active': '2024-09-10T15:59:16.274407+0000', 'last_became_peered': '2024-09-10T15:59:16.274407+0000', 'last_unstale': '2024-09-10T16:08:15.723842+0000', 'last_undegraded': '2024-09-10T15:59:15.851279+0000', 'last_fullsized': '2024-09-10T15:59:15.774356+0000', 'mapping_epoch': 42, 'log_start': "45'9900", 'ondisk_log_start': "45'9900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'objects_scrubbed': 0, 'log_size': 10005, 'log_dups_size': 0, 'ondisk_log_size': 10005, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 3141632, 'num_objects': 767, 'num_object_clones': 0, 'num_object_copies': 1534, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4259, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 767, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 19905, 'num_write_kb': 41344, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3492}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:08:18.365 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-10T16:08:18.365 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "49'20296", 'reported_seq': 20444, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:08:15.772208+0000', 'last_change': '2024-09-10T15:59:16.308264+0000', 'last_active': '2024-09-10T16:08:15.772208+0000', 'last_peered': '2024-09-10T16:08:15.772208+0000', 'last_clean': '2024-09-10T15:59:14.111551+0000', 'last_became_active': '2024-09-10T15:59:16.300040+0000', 'last_became_peered': '2024-09-10T15:59:16.300040+0000', 'last_unstale': '2024-09-10T16:08:15.772208+0000', 'last_undegraded': '2024-09-10T15:59:15.878234+0000', 'last_fullsized': '2024-09-10T15:59:15.769328+0000', 'mapping_epoch': 42, 'log_start': "45'10200", 'ondisk_log_start': "45'10200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3442", 'last_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'objects_scrubbed': 3384, 'log_size': 10096, 'log_dups_size': 0, 'ondisk_log_size': 10096, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1292, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 3133923, 'num_objects': 766, 'num_object_clones': 0, 'num_object_copies': 1532, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4372, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 766, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 16908, 'num_write_kb': 28580, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3605}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:08:18.365 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-10T16:08:18.365 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "49'20295", 'reported_seq': 20442, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:08:15.720757+0000', 'last_change': '2024-09-10T15:59:16.331899+0000', 'last_active': '2024-09-10T16:08:15.720757+0000', 'last_peered': '2024-09-10T16:08:15.720757+0000', 'last_clean': '2024-09-10T15:59:14.139436+0000', 'last_became_active': '2024-09-10T15:59:16.318715+0000', 'last_became_peered': '2024-09-10T15:59:16.318715+0000', 'last_unstale': '2024-09-10T16:08:15.720757+0000', 'last_undegraded': '2024-09-10T15:59:15.877524+0000', 'last_fullsized': '2024-09-10T15:59:15.769042+0000', 'mapping_epoch': 42, 'log_start': "45'10200", 'ondisk_log_start': "45'10200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3537", 'last_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'objects_scrubbed': 3497, 'log_size': 10095, 'log_dups_size': 0, 'ondisk_log_size': 10095, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1224, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 3224518, 'num_objects': 789, 'num_object_clones': 0, 'num_object_copies': 1578, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4237, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 789, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 16794, 'num_write_kb': 28172, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3448}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:08:18.365 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-10T16:08:18.365 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "49'19660", 'reported_seq': 19810, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:08:15.847575+0000', 'last_change': '2024-09-10T15:59:16.175680+0000', 'last_active': '2024-09-10T16:08:15.847575+0000', 'last_peered': '2024-09-10T16:08:15.847575+0000', 'last_clean': '2024-09-10T15:59:14.088187+0000', 'last_became_active': '2024-09-10T15:59:16.165810+0000', 'last_became_peered': '2024-09-10T15:59:16.165810+0000', 'last_unstale': '2024-09-10T16:08:15.847575+0000', 'last_undegraded': '2024-09-10T15:59:15.822098+0000', 'last_fullsized': '2024-09-10T15:59:15.769123+0000', 'mapping_epoch': 42, 'log_start': "45'9600", 'ondisk_log_start': "45'9600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2218", 'last_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'objects_scrubbed': 2175, 'log_size': 10060, 'log_dups_size': 0, 'ondisk_log_size': 10060, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 825, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 3264995, 'num_objects': 798, 'num_object_clones': 0, 'num_object_copies': 1596, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4124, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 798, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 17479, 'num_write_kb': 32204, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': -3325}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:08:18.365 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-10T16:08:18.365 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "49'20130", 'reported_seq': 20551, 'reported_epoch': 49, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:08:15.810658+0000', 'last_change': '2024-09-10T16:08:09.280891+0000', 'last_active': '2024-09-10T16:08:15.810658+0000', 'last_peered': '2024-09-10T16:08:15.810658+0000', 'last_clean': '2024-09-10T15:59:14.129081+0000', 'last_became_active': '2024-09-10T15:59:16.269835+0000', 'last_became_peered': '2024-09-10T15:59:16.269835+0000', 'last_unstale': '2024-09-10T16:08:15.810658+0000', 'last_undegraded': '2024-09-10T15:59:15.860857+0000', 'last_fullsized': '2024-09-10T15:59:15.771410+0000', 'mapping_epoch': 42, 'log_start': "45'10100", 'ondisk_log_start': "45'10100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3557", 'last_scrub_stamp': '2024-09-10T15:59:02.493678+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:02.493678+0000', 'objects_scrubbed': 3488, 'log_size': 10030, 'log_dups_size': 0, 'ondisk_log_size': 10030, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1472, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 3194880, 'num_objects': 780, 'num_object_clones': 0, 'num_object_copies': 1560, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4117, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 780, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16634, 'num_write_kb': 27852, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 140, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3337}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:08:18.366 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-10T16:08:18.366 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "49'19986", 'reported_seq': 20129, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:08:16.010738+0000', 'last_change': '2024-09-10T15:59:16.284311+0000', 'last_active': '2024-09-10T16:08:16.010738+0000', 'last_peered': '2024-09-10T16:08:16.010738+0000', 'last_clean': '2024-09-10T15:59:14.116041+0000', 'last_became_active': '2024-09-10T15:59:16.274543+0000', 'last_became_peered': '2024-09-10T15:59:16.274543+0000', 'last_unstale': '2024-09-10T16:08:16.010738+0000', 'last_undegraded': '2024-09-10T15:59:15.812484+0000', 'last_fullsized': '2024-09-10T15:59:15.770885+0000', 'mapping_epoch': 42, 'log_start': "45'9900", 'ondisk_log_start': "45'9900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2356", 'last_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'objects_scrubbed': 2316, 'log_size': 10086, 'log_dups_size': 0, 'ondisk_log_size': 10086, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 783, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 3256803, 'num_objects': 796, 'num_object_clones': 0, 'num_object_copies': 1592, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 796, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 17666, 'num_write_kb': 32292, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': -3508}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:08:18.366 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-10T16:08:18.366 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "49'20391", 'reported_seq': 29245, 'reported_epoch': 49, 'state': 'active+recovery_wait', 'last_fresh': '2024-09-10T16:08:15.993183+0000', 'last_change': '2024-09-10T16:08:10.929169+0000', 'last_active': '2024-09-10T16:08:15.993183+0000', 'last_peered': '2024-09-10T16:08:15.993183+0000', 'last_clean': '2024-09-10T15:59:14.110654+0000', 'last_became_active': '2024-09-10T16:08:10.860438+0000', 'last_became_peered': '2024-09-10T16:08:10.860438+0000', 'last_unstale': '2024-09-10T16:08:15.993183+0000', 'last_undegraded': '2024-09-10T16:08:15.993183+0000', 'last_fullsized': '2024-09-10T16:08:15.993183+0000', 'mapping_epoch': 48, 'log_start': "45'10300", 'ondisk_log_start': "45'10300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2486", 'last_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'objects_scrubbed': 2440, 'log_size': 10091, 'log_dups_size': 0, 'ondisk_log_size': 10091, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 792, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 3190784, 'num_objects': 779, 'num_object_clones': 0, 'num_object_copies': 1558, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 779, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17945, 'num_write_kb': 32568, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 4355, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': ['2', '0', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 779}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:08:18.366 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-10T16:08:18.366 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "49'20037", 'reported_seq': 20182, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:08:15.723089+0000', 'last_change': '2024-09-10T15:59:16.315641+0000', 'last_active': '2024-09-10T16:08:15.723089+0000', 'last_peered': '2024-09-10T16:08:15.723089+0000', 'last_clean': '2024-09-10T15:59:14.117001+0000', 'last_became_active': '2024-09-10T15:59:16.302245+0000', 'last_became_peered': '2024-09-10T15:59:16.302245+0000', 'last_unstale': '2024-09-10T16:08:15.723089+0000', 'last_undegraded': '2024-09-10T15:59:15.845012+0000', 'last_fullsized': '2024-09-10T15:59:15.770337+0000', 'mapping_epoch': 42, 'log_start': "45'10000", 'ondisk_log_start': "45'10000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3404", 'last_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'objects_scrubbed': 3367, 'log_size': 10037, 'log_dups_size': 0, 'ondisk_log_size': 10037, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 983, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 3280896, 'num_objects': 801, 'num_object_clones': 0, 'num_object_copies': 1602, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 801, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16668, 'num_write_kb': 28204, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3503}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:08:18.366 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-10T16:08:18.366 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "49'20294", 'reported_seq': 20446, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:08:15.784442+0000', 'last_change': '2024-09-10T15:59:16.311291+0000', 'last_active': '2024-09-10T16:08:15.784442+0000', 'last_peered': '2024-09-10T16:08:15.784442+0000', 'last_clean': '2024-09-10T15:59:14.116363+0000', 'last_became_active': '2024-09-10T15:59:16.301244+0000', 'last_became_peered': '2024-09-10T15:59:16.301244+0000', 'last_unstale': '2024-09-10T16:08:15.784442+0000', 'last_undegraded': '2024-09-10T15:59:15.883434+0000', 'last_fullsized': '2024-09-10T15:59:15.772756+0000', 'mapping_epoch': 42, 'log_start': "45'10200", 'ondisk_log_start': "45'10200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3789", 'last_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'objects_scrubbed': 3741, 'log_size': 10094, 'log_dups_size': 0, 'ondisk_log_size': 10094, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1440, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 3088384, 'num_objects': 754, 'num_object_clones': 0, 'num_object_copies': 1508, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4373, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 754, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16549, 'num_write_kb': 27124, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3617}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:08:18.366 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-10T16:08:18.366 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "49'20377", 'reported_seq': 20529, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:08:15.805584+0000', 'last_change': '2024-09-10T15:59:16.313409+0000', 'last_active': '2024-09-10T16:08:15.805584+0000', 'last_peered': '2024-09-10T16:08:15.805584+0000', 'last_clean': '2024-09-10T15:59:14.129780+0000', 'last_became_active': '2024-09-10T15:59:16.300932+0000', 'last_became_peered': '2024-09-10T15:59:16.300932+0000', 'last_unstale': '2024-09-10T16:08:15.805584+0000', 'last_undegraded': '2024-09-10T15:59:15.887823+0000', 'last_fullsized': '2024-09-10T15:59:15.768513+0000', 'mapping_epoch': 42, 'log_start': "45'10300", 'ondisk_log_start': "45'10300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3880", 'last_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'objects_scrubbed': 3827, 'log_size': 10077, 'log_dups_size': 0, 'ondisk_log_size': 10077, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1397, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 3363782, 'num_objects': 823, 'num_object_clones': 0, 'num_object_copies': 1646, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4325, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 823, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16540, 'num_write_kb': 27072, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3500}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:08:18.367 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T16:08:19.051 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:08:19.052 INFO:teuthology.orchestra.run.smithi115.stdout:{"fsid":"7effee29-df05-4351-9741-ee203564ed81","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 38417/25262 objects degraded (152.074%), 9 pgs degraded, 9 pgs undersized","count":18},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":663,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":49,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983954,"num_in_osds":3,"osd_in_since":1725983947,"num_remapped_pgs":9},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":8},{"state_name":"active+recovery_wait","count":1},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":12631,"data_bytes":51688201,"bytes_used":1575223296,"bytes_avail":288335069184,"bytes_total":289910292480,"degraded_objects":38417,"degraded_total":25262,"degraded_ratio":1.5207426173699627,"recovering_objects_per_sec":31,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":255,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":509},"fsmap":{"epoch":1,"btime":"2024-09-10T15:57:14:967120+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T16:00:13.560776+0000","services":{}},"progress_events":{}} 2024-09-10T16:08:19.076 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-10T16:08:19.076 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:08:19.670 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:08:19.671 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:08:19.696 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":373,"stamp":"2024-09-10T16:08:17.754575+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":51688201,"num_objects":12631,"num_object_clones":0,"num_object_copies":25262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38417,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12631,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":279479,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12975,"num_bytes_recovered":33095699,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160906,"ondisk_log_size":160906,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1538304,"kb_used_data":310388,"kb_used_omap":12,"kb_used_meta":1227827,"kb_avail":281577216,"statfs":{"total":289910292480,"available":288335069184,"internally_reserved":0,"allocated":317837312,"data_stored":316875610,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1257294850},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,5,4,4,7,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-16711680,"num_objects":-4080,"num_object_clones":0,"num_object_copies":-8160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-254,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4080,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4080,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":255,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.004256"},"pg_stats":[{"pgid":"2.f","version":"49'19905","reported_seq":19990,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:15.723842+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:08:15.723842+0000","last_peered":"2024-09-10T16:08:15.723842+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:08:15.723842+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3141632,"num_objects":767,"num_object_clones":0,"num_object_copies":1534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":767,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19905,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3492}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20060","reported_seq":20183,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:15.964932+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:08:15.964932+0000","last_peered":"2024-09-10T16:08:15.964932+0000","last_clean":"2024-09-10T16:08:15.964932+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:08:15.964932+0000","last_undegraded":"2024-09-10T16:08:15.964932+0000","last_fullsized":"2024-09-10T16:08:15.964932+0000","mapping_epoch":35,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3212230,"num_objects":786,"num_object_clones":0,"num_object_copies":1572,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":786,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18165,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19985","reported_seq":28673,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:14.694212+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:08:14.694212+0000","last_peered":"2024-09-10T16:08:14.694212+0000","last_clean":"2024-09-10T16:08:14.694212+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:08:14.694212+0000","last_undegraded":"2024-09-10T16:08:14.694212+0000","last_fullsized":"2024-09-10T16:08:14.694212+0000","mapping_epoch":44,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3325976,"num_objects":813,"num_object_clones":0,"num_object_copies":1626,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":813,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17989,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19902","reported_seq":19993,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:15.997632+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:08:15.997632+0000","last_peered":"2024-09-10T16:08:15.997632+0000","last_clean":"2024-09-10T16:08:15.997632+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:08:15.997632+0000","last_undegraded":"2024-09-10T16:08:15.997632+0000","last_fullsized":"2024-09-10T16:08:15.997632+0000","mapping_epoch":35,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3104768,"num_objects":758,"num_object_clones":0,"num_object_copies":1516,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":758,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19902,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20103","reported_seq":20243,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:15.798072+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:08:15.798072+0000","last_peered":"2024-09-10T16:08:15.798072+0000","last_clean":"2024-09-10T16:08:15.798072+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:08:15.798072+0000","last_undegraded":"2024-09-10T16:08:15.798072+0000","last_fullsized":"2024-09-10T16:08:15.798072+0000","mapping_epoch":34,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3228131,"num_objects":789,"num_object_clones":0,"num_object_copies":1578,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":789,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16814,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20296","reported_seq":20444,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:15.772208+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:08:15.772208+0000","last_peered":"2024-09-10T16:08:15.772208+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:08:15.772208+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3133923,"num_objects":766,"num_object_clones":0,"num_object_copies":1532,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":766,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16908,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3605},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'20295","reported_seq":20442,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:15.720757+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:08:15.720757+0000","last_peered":"2024-09-10T16:08:15.720757+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:08:15.720757+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3224518,"num_objects":789,"num_object_clones":0,"num_object_copies":1578,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":789,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16794,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3448},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20529","reported_seq":20669,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:15.791179+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:08:15.791179+0000","last_peered":"2024-09-10T16:08:15.791179+0000","last_clean":"2024-09-10T16:08:15.791179+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:08:15.791179+0000","last_undegraded":"2024-09-10T16:08:15.791179+0000","last_fullsized":"2024-09-10T16:08:15.791179+0000","mapping_epoch":34,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3215360,"num_objects":785,"num_object_clones":0,"num_object_copies":1570,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":785,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17376,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.838857+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:08:10.838857+0000","last_peered":"2024-09-10T16:08:10.838857+0000","last_clean":"2024-09-10T16:08:10.838857+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:08:10.838857+0000","last_undegraded":"2024-09-10T16:08:10.838857+0000","last_fullsized":"2024-09-10T16:08:10.838857+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19660","reported_seq":19810,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:15.847575+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:08:15.847575+0000","last_peered":"2024-09-10T16:08:15.847575+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:08:15.847575+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3264995,"num_objects":798,"num_object_clones":0,"num_object_copies":1596,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":798,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17479,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-3325}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":104,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.838197+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:08:10.838197+0000","last_peered":"2024-09-10T16:08:10.838197+0000","last_clean":"2024-09-10T16:08:10.838197+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:08:10.838197+0000","last_undegraded":"2024-09-10T16:08:10.838197+0000","last_fullsized":"2024-09-10T16:08:10.838197+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20130","reported_seq":20551,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:15.810658+0000","last_change":"2024-09-10T16:08:09.280891+0000","last_active":"2024-09-10T16:08:15.810658+0000","last_peered":"2024-09-10T16:08:15.810658+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:08:15.810658+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3194880,"num_objects":780,"num_object_clones":0,"num_object_copies":1560,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4117,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":780,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16634,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":140,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3337}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":114,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.840262+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:08:10.840262+0000","last_peered":"2024-09-10T16:08:10.840262+0000","last_clean":"2024-09-10T16:08:10.840262+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:08:10.840262+0000","last_undegraded":"2024-09-10T16:08:10.840262+0000","last_fullsized":"2024-09-10T16:08:10.840262+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'19986","reported_seq":20129,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:16.010738+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:08:16.010738+0000","last_peered":"2024-09-10T16:08:16.010738+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:08:16.010738+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3256803,"num_objects":796,"num_object_clones":0,"num_object_copies":1592,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":796,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17666,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-3508}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":90,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.837937+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:08:10.837937+0000","last_peered":"2024-09-10T16:08:10.837937+0000","last_clean":"2024-09-10T16:08:10.837937+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:08:10.837937+0000","last_undegraded":"2024-09-10T16:08:10.837937+0000","last_fullsized":"2024-09-10T16:08:10.837937+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'20391","reported_seq":29245,"reported_epoch":49,"state":"active+recovery_wait","last_fresh":"2024-09-10T16:08:15.993183+0000","last_change":"2024-09-10T16:08:10.929169+0000","last_active":"2024-09-10T16:08:15.993183+0000","last_peered":"2024-09-10T16:08:15.993183+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T16:08:10.860438+0000","last_became_peered":"2024-09-10T16:08:10.860438+0000","last_unstale":"2024-09-10T16:08:15.993183+0000","last_undegraded":"2024-09-10T16:08:15.993183+0000","last_fullsized":"2024-09-10T16:08:15.993183+0000","mapping_epoch":48,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3190784,"num_objects":779,"num_object_clones":0,"num_object_copies":1558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":779,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17945,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","0","1"],"object_location_counts":[{"shards":"1,2","objects":779}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20037","reported_seq":20182,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:15.723089+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:08:15.723089+0000","last_peered":"2024-09-10T16:08:15.723089+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:08:15.723089+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3280896,"num_objects":801,"num_object_clones":0,"num_object_copies":1602,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":801,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16668,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3503}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.838546+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:08:10.838546+0000","last_peered":"2024-09-10T16:08:10.838546+0000","last_clean":"2024-09-10T16:08:10.838546+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:08:10.838546+0000","last_undegraded":"2024-09-10T16:08:10.838546+0000","last_fullsized":"2024-09-10T16:08:10.838546+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19753","reported_seq":28329,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:14.743651+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:08:14.743651+0000","last_peered":"2024-09-10T16:08:14.743651+0000","last_clean":"2024-09-10T16:08:14.743651+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:08:14.743651+0000","last_undegraded":"2024-09-10T16:08:14.743651+0000","last_fullsized":"2024-09-10T16:08:14.743651+0000","mapping_epoch":46,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3461120,"num_objects":845,"num_object_clones":0,"num_object_copies":1690,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":845,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16143,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":104,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.838936+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:08:10.838936+0000","last_peered":"2024-09-10T16:08:10.838936+0000","last_clean":"2024-09-10T16:08:10.838936+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:08:10.838936+0000","last_undegraded":"2024-09-10T16:08:10.838936+0000","last_fullsized":"2024-09-10T16:08:10.838936+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20294","reported_seq":20446,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:15.784442+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:08:15.784442+0000","last_peered":"2024-09-10T16:08:15.784442+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:08:15.784442+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3088384,"num_objects":754,"num_object_clones":0,"num_object_copies":1508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16549,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3617}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.839114+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:08:10.839114+0000","last_peered":"2024-09-10T16:08:10.839114+0000","last_clean":"2024-09-10T16:08:10.839114+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:08:10.839114+0000","last_undegraded":"2024-09-10T16:08:10.839114+0000","last_fullsized":"2024-09-10T16:08:10.839114+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":85,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.838633+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:08:10.838633+0000","last_peered":"2024-09-10T16:08:10.838633+0000","last_clean":"2024-09-10T16:08:10.838633+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:08:10.838633+0000","last_undegraded":"2024-09-10T16:08:10.838633+0000","last_fullsized":"2024-09-10T16:08:10.838633+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'20377","reported_seq":20529,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:15.805584+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:08:15.805584+0000","last_peered":"2024-09-10T16:08:15.805584+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:08:15.805584+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3363782,"num_objects":823,"num_object_clones":0,"num_object_copies":1646,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":823,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16540,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3500},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":51688182,"num_objects":12629,"num_object_clones":0,"num_object_copies":25258,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38417,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12629,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":279477,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12973,"num_bytes_recovered":33095680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":314904576,"data_stored":314769977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160903,"ondisk_log_size":160903,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986299,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":519960,"kb_used_data":163392,"kb_used_omap":4,"kb_used_meta":356539,"kb_avail":93851880,"statfs":{"total":96636764160,"available":96104325120,"internally_reserved":0,"allocated":167313408,"data_stored":167000223,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":365096618},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659244,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":446944,"kb_used_data":40388,"kb_used_omap":4,"kb_used_meta":406523,"kb_avail":93924896,"statfs":{"total":96636764160,"available":96179093504,"internally_reserved":0,"allocated":41357312,"data_stored":41021985,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":416280238},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,2,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182527,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":571400,"kb_used_data":106608,"kb_used_omap":4,"kb_used_meta":464763,"kb_avail":93800440,"statfs":{"total":96636764160,"available":96051650560,"internally_reserved":0,"allocated":109166592,"data_stored":108853402,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":475917994},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,2,2,0,7,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":108384256,"data_stored":108348126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40349696,"data_stored":40291429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":166170624,"data_stored":166130422,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:08:22.699 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:08:23.267 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:08:23.270 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:08:23.288 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":375,"stamp":"2024-09-10T16:08:21.758973+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":42005257,"num_objects":10267,"num_object_clones":0,"num_object_copies":20534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38361,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10267,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":281843,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13031,"num_bytes_recovered":33095699,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160770,"ondisk_log_size":160770,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1553948,"kb_used_data":291368,"kb_used_omap":12,"kb_used_meta":1262515,"kb_avail":281561572,"statfs":{"total":289910292480,"available":288319049728,"internally_reserved":0,"allocated":298360832,"data_stored":297399130,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1292815362},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,6,6,7,7,9],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-21622784,"num_objects":-5279,"num_object_clones":0,"num_object_copies":-10558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-242,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5279,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5279,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":243,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.922524"},"pg_stats":[{"pgid":"2.f","version":"49'20043","reported_seq":20128,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:20.824435+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:08:20.824435+0000","last_peered":"2024-09-10T16:08:20.824435+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:08:20.824435+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2576384,"num_objects":629,"num_object_clones":0,"num_object_copies":1258,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":629,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20043,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3630}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20206","reported_seq":20329,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:20.956434+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:08:20.956434+0000","last_peered":"2024-09-10T16:08:20.956434+0000","last_clean":"2024-09-10T16:08:20.956434+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:08:20.956434+0000","last_undegraded":"2024-09-10T16:08:20.956434+0000","last_fullsized":"2024-09-10T16:08:20.956434+0000","mapping_epoch":35,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2614214,"num_objects":640,"num_object_clones":0,"num_object_copies":1280,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":640,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18311,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20129","reported_seq":28817,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:19.805778+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:08:19.805778+0000","last_peered":"2024-09-10T16:08:19.805778+0000","last_clean":"2024-09-10T16:08:19.805778+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:08:19.805778+0000","last_undegraded":"2024-09-10T16:08:19.805778+0000","last_fullsized":"2024-09-10T16:08:19.805778+0000","mapping_epoch":44,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2736152,"num_objects":669,"num_object_clones":0,"num_object_copies":1338,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":669,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18133,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20053","reported_seq":20144,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:21.063454+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:08:21.063454+0000","last_peered":"2024-09-10T16:08:21.063454+0000","last_clean":"2024-09-10T16:08:21.063454+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:08:21.063454+0000","last_undegraded":"2024-09-10T16:08:21.063454+0000","last_fullsized":"2024-09-10T16:08:21.063454+0000","mapping_epoch":35,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2486272,"num_objects":607,"num_object_clones":0,"num_object_copies":1214,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":607,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20053,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20246","reported_seq":20386,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:20.722435+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:08:20.722435+0000","last_peered":"2024-09-10T16:08:20.722435+0000","last_clean":"2024-09-10T16:08:20.722435+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:08:20.722435+0000","last_undegraded":"2024-09-10T16:08:20.722435+0000","last_fullsized":"2024-09-10T16:08:20.722435+0000","mapping_epoch":34,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2642403,"num_objects":646,"num_object_clones":0,"num_object_copies":1292,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":646,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16957,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20440","reported_seq":20588,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:20.653037+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:08:20.653037+0000","last_peered":"2024-09-10T16:08:20.653037+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:08:20.653037+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2544099,"num_objects":622,"num_object_clones":0,"num_object_copies":1244,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":622,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17052,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3749},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'20445","reported_seq":20593,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:20.686425+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:08:20.686425+0000","last_peered":"2024-09-10T16:08:20.686425+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:08:20.686425+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2610118,"num_objects":639,"num_object_clones":0,"num_object_copies":1278,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":639,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16944,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3598},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20681","reported_seq":20821,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:20.795233+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:08:20.795233+0000","last_peered":"2024-09-10T16:08:20.795233+0000","last_clean":"2024-09-10T16:08:20.795233+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:08:20.795233+0000","last_undegraded":"2024-09-10T16:08:20.795233+0000","last_fullsized":"2024-09-10T16:08:20.795233+0000","mapping_epoch":34,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2592768,"num_objects":633,"num_object_clones":0,"num_object_copies":1266,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":633,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17528,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.838857+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:08:10.838857+0000","last_peered":"2024-09-10T16:08:10.838857+0000","last_clean":"2024-09-10T16:08:10.838857+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:08:10.838857+0000","last_undegraded":"2024-09-10T16:08:10.838857+0000","last_fullsized":"2024-09-10T16:08:10.838857+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19792","reported_seq":19942,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:20.978206+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:08:20.978206+0000","last_peered":"2024-09-10T16:08:20.978206+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:08:20.978206+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2724323,"num_objects":666,"num_object_clones":0,"num_object_copies":1332,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":666,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17611,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-3457}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":104,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.838197+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:08:10.838197+0000","last_peered":"2024-09-10T16:08:10.838197+0000","last_clean":"2024-09-10T16:08:10.838197+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:08:10.838197+0000","last_undegraded":"2024-09-10T16:08:10.838197+0000","last_fullsized":"2024-09-10T16:08:10.838197+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20294","reported_seq":20826,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:20.813231+0000","last_change":"2024-09-10T16:08:09.280891+0000","last_active":"2024-09-10T16:08:20.813231+0000","last_peered":"2024-09-10T16:08:20.813231+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:08:20.813231+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2523136,"num_objects":616,"num_object_clones":0,"num_object_copies":1232,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4061,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":616,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16798,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":196,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3445}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":114,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.840262+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:08:10.840262+0000","last_peered":"2024-09-10T16:08:10.840262+0000","last_clean":"2024-09-10T16:08:10.840262+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:08:10.840262+0000","last_undegraded":"2024-09-10T16:08:10.840262+0000","last_fullsized":"2024-09-10T16:08:10.840262+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20137","reported_seq":20282,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:21.050025+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:08:21.050025+0000","last_peered":"2024-09-10T16:08:21.050025+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:08:21.050025+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2638307,"num_objects":645,"num_object_clones":0,"num_object_copies":1290,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":645,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17817,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-3659}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":90,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.837937+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:08:10.837937+0000","last_peered":"2024-09-10T16:08:10.837937+0000","last_clean":"2024-09-10T16:08:10.837937+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:08:10.837937+0000","last_undegraded":"2024-09-10T16:08:10.837937+0000","last_fullsized":"2024-09-10T16:08:10.837937+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'20544","reported_seq":29399,"reported_epoch":49,"state":"active+recovery_wait","last_fresh":"2024-09-10T16:08:21.030947+0000","last_change":"2024-09-10T16:08:10.929169+0000","last_active":"2024-09-10T16:08:21.030947+0000","last_peered":"2024-09-10T16:08:21.030947+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T16:08:10.860438+0000","last_became_peered":"2024-09-10T16:08:10.860438+0000","last_unstale":"2024-09-10T16:08:21.030947+0000","last_undegraded":"2024-09-10T16:08:21.030947+0000","last_fullsized":"2024-09-10T16:08:21.030947+0000","mapping_epoch":48,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2564096,"num_objects":626,"num_object_clones":0,"num_object_copies":1252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":626,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18098,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","0","1"],"object_location_counts":[{"shards":"1,2","objects":626}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20190","reported_seq":20335,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:20.826779+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:08:20.826779+0000","last_peered":"2024-09-10T16:08:20.826779+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:08:20.826779+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2654208,"num_objects":648,"num_object_clones":0,"num_object_copies":1296,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":648,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16821,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3656}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.838546+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:08:10.838546+0000","last_peered":"2024-09-10T16:08:10.838546+0000","last_clean":"2024-09-10T16:08:10.838546+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:08:10.838546+0000","last_undegraded":"2024-09-10T16:08:10.838546+0000","last_fullsized":"2024-09-10T16:08:10.838546+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19916","reported_seq":28492,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:19.694053+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:08:19.694053+0000","last_peered":"2024-09-10T16:08:19.694053+0000","last_clean":"2024-09-10T16:08:19.694053+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:08:19.694053+0000","last_undegraded":"2024-09-10T16:08:19.694053+0000","last_fullsized":"2024-09-10T16:08:19.694053+0000","mapping_epoch":46,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2793472,"num_objects":682,"num_object_clones":0,"num_object_copies":1364,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":682,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16306,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":104,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.838936+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:08:10.838936+0000","last_peered":"2024-09-10T16:08:10.838936+0000","last_clean":"2024-09-10T16:08:10.838936+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:08:10.838936+0000","last_undegraded":"2024-09-10T16:08:10.838936+0000","last_fullsized":"2024-09-10T16:08:10.838936+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20415","reported_seq":20567,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:20.768543+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:08:20.768543+0000","last_peered":"2024-09-10T16:08:20.768543+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:08:20.768543+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2592768,"num_objects":633,"num_object_clones":0,"num_object_copies":1266,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":633,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16670,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3738}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.839114+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:08:10.839114+0000","last_peered":"2024-09-10T16:08:10.839114+0000","last_clean":"2024-09-10T16:08:10.839114+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:08:10.839114+0000","last_undegraded":"2024-09-10T16:08:10.839114+0000","last_fullsized":"2024-09-10T16:08:10.839114+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":85,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.838633+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:08:10.838633+0000","last_peered":"2024-09-10T16:08:10.838633+0000","last_clean":"2024-09-10T16:08:10.838633+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:08:10.838633+0000","last_undegraded":"2024-09-10T16:08:10.838633+0000","last_fullsized":"2024-09-10T16:08:10.838633+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'20536","reported_seq":20691,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:20.775547+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:08:20.775547+0000","last_peered":"2024-09-10T16:08:20.775547+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:08:20.775547+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2712518,"num_objects":664,"num_object_clones":0,"num_object_copies":1328,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":664,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16699,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3659},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":42005238,"num_objects":10265,"num_object_clones":0,"num_object_copies":20530,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38361,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10265,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":281841,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13029,"num_bytes_recovered":33095680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":295538688,"data_stored":295404089,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160767,"ondisk_log_size":160767,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986300,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":519456,"kb_used_data":158736,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93852384,"statfs":{"total":96636764160,"available":96104841216,"internally_reserved":0,"allocated":162545664,"data_stored":162232479,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369356458},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,0,2,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659245,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":462120,"kb_used_data":33368,"kb_used_omap":4,"kb_used_meta":428731,"kb_avail":93909720,"statfs":{"total":96636764160,"available":96163553280,"internally_reserved":0,"allocated":34168832,"data_stored":33833505,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":439021230},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,2,1,4,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182528,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":572372,"kb_used_data":99264,"kb_used_omap":4,"kb_used_meta":473083,"kb_avail":93799468,"statfs":{"total":96636764160,"available":96050655232,"internally_reserved":0,"allocated":101646336,"data_stored":101333146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":484437674},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,4,4,2,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":101060608,"data_stored":101024478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":33112064,"data_stored":33053797,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":161366016,"data_stored":161325814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:08:23.290 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:08:23.832 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:08:23.834 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:08:23.856 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":376,"stamp":"2024-09-10T16:08:23.759465+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":42005257,"num_objects":10267,"num_object_clones":0,"num_object_copies":20534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38361,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10267,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":281843,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13031,"num_bytes_recovered":33095699,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160770,"ondisk_log_size":160770,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1553948,"kb_used_data":291368,"kb_used_omap":12,"kb_used_meta":1262515,"kb_avail":281561572,"statfs":{"total":289910292480,"available":288319049728,"internally_reserved":0,"allocated":298360832,"data_stored":297399130,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1292815362},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,6,6,7,7,9],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18919424,"num_objects":-4619,"num_object_clones":0,"num_object_copies":-9238,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-128,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4619,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4619,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":128,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.008712"},"pg_stats":[{"pgid":"2.f","version":"49'20043","reported_seq":20128,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:20.824435+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:08:20.824435+0000","last_peered":"2024-09-10T16:08:20.824435+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:08:20.824435+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2576384,"num_objects":629,"num_object_clones":0,"num_object_copies":1258,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":629,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20043,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3630}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20206","reported_seq":20329,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:20.956434+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:08:20.956434+0000","last_peered":"2024-09-10T16:08:20.956434+0000","last_clean":"2024-09-10T16:08:20.956434+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:08:20.956434+0000","last_undegraded":"2024-09-10T16:08:20.956434+0000","last_fullsized":"2024-09-10T16:08:20.956434+0000","mapping_epoch":35,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2614214,"num_objects":640,"num_object_clones":0,"num_object_copies":1280,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":640,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18311,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20129","reported_seq":28817,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:19.805778+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:08:19.805778+0000","last_peered":"2024-09-10T16:08:19.805778+0000","last_clean":"2024-09-10T16:08:19.805778+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:08:19.805778+0000","last_undegraded":"2024-09-10T16:08:19.805778+0000","last_fullsized":"2024-09-10T16:08:19.805778+0000","mapping_epoch":44,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2736152,"num_objects":669,"num_object_clones":0,"num_object_copies":1338,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":669,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18133,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20053","reported_seq":20144,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:21.063454+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:08:21.063454+0000","last_peered":"2024-09-10T16:08:21.063454+0000","last_clean":"2024-09-10T16:08:21.063454+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:08:21.063454+0000","last_undegraded":"2024-09-10T16:08:21.063454+0000","last_fullsized":"2024-09-10T16:08:21.063454+0000","mapping_epoch":35,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2486272,"num_objects":607,"num_object_clones":0,"num_object_copies":1214,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":607,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20053,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20246","reported_seq":20386,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:20.722435+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:08:20.722435+0000","last_peered":"2024-09-10T16:08:20.722435+0000","last_clean":"2024-09-10T16:08:20.722435+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:08:20.722435+0000","last_undegraded":"2024-09-10T16:08:20.722435+0000","last_fullsized":"2024-09-10T16:08:20.722435+0000","mapping_epoch":34,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2642403,"num_objects":646,"num_object_clones":0,"num_object_copies":1292,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":646,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16957,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20440","reported_seq":20588,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:20.653037+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:08:20.653037+0000","last_peered":"2024-09-10T16:08:20.653037+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:08:20.653037+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2544099,"num_objects":622,"num_object_clones":0,"num_object_copies":1244,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":622,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17052,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3749},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'20445","reported_seq":20593,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:20.686425+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:08:20.686425+0000","last_peered":"2024-09-10T16:08:20.686425+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:08:20.686425+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2610118,"num_objects":639,"num_object_clones":0,"num_object_copies":1278,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":639,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16944,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3598},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20681","reported_seq":20821,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:20.795233+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:08:20.795233+0000","last_peered":"2024-09-10T16:08:20.795233+0000","last_clean":"2024-09-10T16:08:20.795233+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:08:20.795233+0000","last_undegraded":"2024-09-10T16:08:20.795233+0000","last_fullsized":"2024-09-10T16:08:20.795233+0000","mapping_epoch":34,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2592768,"num_objects":633,"num_object_clones":0,"num_object_copies":1266,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":633,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17528,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.838857+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:08:10.838857+0000","last_peered":"2024-09-10T16:08:10.838857+0000","last_clean":"2024-09-10T16:08:10.838857+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:08:10.838857+0000","last_undegraded":"2024-09-10T16:08:10.838857+0000","last_fullsized":"2024-09-10T16:08:10.838857+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19792","reported_seq":19942,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:20.978206+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:08:20.978206+0000","last_peered":"2024-09-10T16:08:20.978206+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:08:20.978206+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2724323,"num_objects":666,"num_object_clones":0,"num_object_copies":1332,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":666,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17611,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-3457}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":104,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.838197+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:08:10.838197+0000","last_peered":"2024-09-10T16:08:10.838197+0000","last_clean":"2024-09-10T16:08:10.838197+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:08:10.838197+0000","last_undegraded":"2024-09-10T16:08:10.838197+0000","last_fullsized":"2024-09-10T16:08:10.838197+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20294","reported_seq":20826,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:20.813231+0000","last_change":"2024-09-10T16:08:09.280891+0000","last_active":"2024-09-10T16:08:20.813231+0000","last_peered":"2024-09-10T16:08:20.813231+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:08:20.813231+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2523136,"num_objects":616,"num_object_clones":0,"num_object_copies":1232,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4061,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":616,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16798,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":196,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3445}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":114,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.840262+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:08:10.840262+0000","last_peered":"2024-09-10T16:08:10.840262+0000","last_clean":"2024-09-10T16:08:10.840262+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:08:10.840262+0000","last_undegraded":"2024-09-10T16:08:10.840262+0000","last_fullsized":"2024-09-10T16:08:10.840262+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20137","reported_seq":20282,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:21.050025+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:08:21.050025+0000","last_peered":"2024-09-10T16:08:21.050025+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:08:21.050025+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2638307,"num_objects":645,"num_object_clones":0,"num_object_copies":1290,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":645,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17817,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-3659}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":90,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.837937+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:08:10.837937+0000","last_peered":"2024-09-10T16:08:10.837937+0000","last_clean":"2024-09-10T16:08:10.837937+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:08:10.837937+0000","last_undegraded":"2024-09-10T16:08:10.837937+0000","last_fullsized":"2024-09-10T16:08:10.837937+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'20544","reported_seq":29399,"reported_epoch":49,"state":"active+recovery_wait","last_fresh":"2024-09-10T16:08:21.030947+0000","last_change":"2024-09-10T16:08:10.929169+0000","last_active":"2024-09-10T16:08:21.030947+0000","last_peered":"2024-09-10T16:08:21.030947+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T16:08:10.860438+0000","last_became_peered":"2024-09-10T16:08:10.860438+0000","last_unstale":"2024-09-10T16:08:21.030947+0000","last_undegraded":"2024-09-10T16:08:21.030947+0000","last_fullsized":"2024-09-10T16:08:21.030947+0000","mapping_epoch":48,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2564096,"num_objects":626,"num_object_clones":0,"num_object_copies":1252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":626,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18098,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","0","1"],"object_location_counts":[{"shards":"1,2","objects":626}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20190","reported_seq":20335,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:20.826779+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:08:20.826779+0000","last_peered":"2024-09-10T16:08:20.826779+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:08:20.826779+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2654208,"num_objects":648,"num_object_clones":0,"num_object_copies":1296,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":648,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16821,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3656}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.838546+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:08:10.838546+0000","last_peered":"2024-09-10T16:08:10.838546+0000","last_clean":"2024-09-10T16:08:10.838546+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:08:10.838546+0000","last_undegraded":"2024-09-10T16:08:10.838546+0000","last_fullsized":"2024-09-10T16:08:10.838546+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19916","reported_seq":28492,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:19.694053+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:08:19.694053+0000","last_peered":"2024-09-10T16:08:19.694053+0000","last_clean":"2024-09-10T16:08:19.694053+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:08:19.694053+0000","last_undegraded":"2024-09-10T16:08:19.694053+0000","last_fullsized":"2024-09-10T16:08:19.694053+0000","mapping_epoch":46,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2793472,"num_objects":682,"num_object_clones":0,"num_object_copies":1364,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":682,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16306,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":104,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.838936+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:08:10.838936+0000","last_peered":"2024-09-10T16:08:10.838936+0000","last_clean":"2024-09-10T16:08:10.838936+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:08:10.838936+0000","last_undegraded":"2024-09-10T16:08:10.838936+0000","last_fullsized":"2024-09-10T16:08:10.838936+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20415","reported_seq":20567,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:20.768543+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:08:20.768543+0000","last_peered":"2024-09-10T16:08:20.768543+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:08:20.768543+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2592768,"num_objects":633,"num_object_clones":0,"num_object_copies":1266,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":633,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16670,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3738}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.839114+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:08:10.839114+0000","last_peered":"2024-09-10T16:08:10.839114+0000","last_clean":"2024-09-10T16:08:10.839114+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:08:10.839114+0000","last_undegraded":"2024-09-10T16:08:10.839114+0000","last_fullsized":"2024-09-10T16:08:10.839114+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":85,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.838633+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:08:10.838633+0000","last_peered":"2024-09-10T16:08:10.838633+0000","last_clean":"2024-09-10T16:08:10.838633+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:08:10.838633+0000","last_undegraded":"2024-09-10T16:08:10.838633+0000","last_fullsized":"2024-09-10T16:08:10.838633+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'20536","reported_seq":20691,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:20.775547+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:08:20.775547+0000","last_peered":"2024-09-10T16:08:20.775547+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:08:20.775547+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2712518,"num_objects":664,"num_object_clones":0,"num_object_copies":1328,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":664,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16699,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3659},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":42005238,"num_objects":10265,"num_object_clones":0,"num_object_copies":20530,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38361,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10265,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":281841,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13029,"num_bytes_recovered":33095680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":295538688,"data_stored":295404089,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160767,"ondisk_log_size":160767,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986300,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":519456,"kb_used_data":158736,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93852384,"statfs":{"total":96636764160,"available":96104841216,"internally_reserved":0,"allocated":162545664,"data_stored":162232479,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369356458},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,0,2,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659245,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":462120,"kb_used_data":33368,"kb_used_omap":4,"kb_used_meta":428731,"kb_avail":93909720,"statfs":{"total":96636764160,"available":96163553280,"internally_reserved":0,"allocated":34168832,"data_stored":33833505,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":439021230},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,2,1,4,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182528,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":572372,"kb_used_data":99264,"kb_used_omap":4,"kb_used_meta":473083,"kb_avail":93799468,"statfs":{"total":96636764160,"available":96050655232,"internally_reserved":0,"allocated":101646336,"data_stored":101333146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":484437674},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,4,4,2,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":101060608,"data_stored":101024478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":33112064,"data_stored":33053797,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":161366016,"data_stored":161325814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:08:23.857 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-10T16:08:23.857 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "49'20043", 'reported_seq': 20128, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:08:20.824435+0000', 'last_change': '2024-09-10T15:59:16.281558+0000', 'last_active': '2024-09-10T16:08:20.824435+0000', 'last_peered': '2024-09-10T16:08:20.824435+0000', 'last_clean': '2024-09-10T15:59:14.132456+0000', 'last_became_active': '2024-09-10T15:59:16.274407+0000', 'last_became_peered': '2024-09-10T15:59:16.274407+0000', 'last_unstale': '2024-09-10T16:08:20.824435+0000', 'last_undegraded': '2024-09-10T15:59:15.851279+0000', 'last_fullsized': '2024-09-10T15:59:15.774356+0000', 'mapping_epoch': 42, 'log_start': "45'10000", 'ondisk_log_start': "45'10000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'objects_scrubbed': 0, 'log_size': 10043, 'log_dups_size': 0, 'ondisk_log_size': 10043, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 2576384, 'num_objects': 629, 'num_object_clones': 0, 'num_object_copies': 1258, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4259, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 629, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20043, 'num_write_kb': 41344, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3630}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:08:23.857 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-10T16:08:23.857 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "49'20440", 'reported_seq': 20588, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:08:20.653037+0000', 'last_change': '2024-09-10T15:59:16.308264+0000', 'last_active': '2024-09-10T16:08:20.653037+0000', 'last_peered': '2024-09-10T16:08:20.653037+0000', 'last_clean': '2024-09-10T15:59:14.111551+0000', 'last_became_active': '2024-09-10T15:59:16.300040+0000', 'last_became_peered': '2024-09-10T15:59:16.300040+0000', 'last_unstale': '2024-09-10T16:08:20.653037+0000', 'last_undegraded': '2024-09-10T15:59:15.878234+0000', 'last_fullsized': '2024-09-10T15:59:15.769328+0000', 'mapping_epoch': 42, 'log_start': "45'10400", 'ondisk_log_start': "45'10400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3442", 'last_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'objects_scrubbed': 3384, 'log_size': 10040, 'log_dups_size': 0, 'ondisk_log_size': 10040, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1292, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 2544099, 'num_objects': 622, 'num_object_clones': 0, 'num_object_copies': 1244, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4372, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 622, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 17052, 'num_write_kb': 28580, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3749}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:08:23.857 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-10T16:08:23.858 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "49'20445", 'reported_seq': 20593, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:08:20.686425+0000', 'last_change': '2024-09-10T15:59:16.331899+0000', 'last_active': '2024-09-10T16:08:20.686425+0000', 'last_peered': '2024-09-10T16:08:20.686425+0000', 'last_clean': '2024-09-10T15:59:14.139436+0000', 'last_became_active': '2024-09-10T15:59:16.318715+0000', 'last_became_peered': '2024-09-10T15:59:16.318715+0000', 'last_unstale': '2024-09-10T16:08:20.686425+0000', 'last_undegraded': '2024-09-10T15:59:15.877524+0000', 'last_fullsized': '2024-09-10T15:59:15.769042+0000', 'mapping_epoch': 42, 'log_start': "45'10400", 'ondisk_log_start': "45'10400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3537", 'last_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'objects_scrubbed': 3497, 'log_size': 10045, 'log_dups_size': 0, 'ondisk_log_size': 10045, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1224, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 2610118, 'num_objects': 639, 'num_object_clones': 0, 'num_object_copies': 1278, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4237, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 639, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 16944, 'num_write_kb': 28172, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3598}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:08:23.858 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-10T16:08:23.858 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "49'19792", 'reported_seq': 19942, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:08:20.978206+0000', 'last_change': '2024-09-10T15:59:16.175680+0000', 'last_active': '2024-09-10T16:08:20.978206+0000', 'last_peered': '2024-09-10T16:08:20.978206+0000', 'last_clean': '2024-09-10T15:59:14.088187+0000', 'last_became_active': '2024-09-10T15:59:16.165810+0000', 'last_became_peered': '2024-09-10T15:59:16.165810+0000', 'last_unstale': '2024-09-10T16:08:20.978206+0000', 'last_undegraded': '2024-09-10T15:59:15.822098+0000', 'last_fullsized': '2024-09-10T15:59:15.769123+0000', 'mapping_epoch': 42, 'log_start': "45'9700", 'ondisk_log_start': "45'9700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2218", 'last_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'objects_scrubbed': 2175, 'log_size': 10092, 'log_dups_size': 0, 'ondisk_log_size': 10092, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 825, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 2724323, 'num_objects': 666, 'num_object_clones': 0, 'num_object_copies': 1332, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4124, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 666, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 17611, 'num_write_kb': 32204, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': -3457}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:08:23.858 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-10T16:08:23.858 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "49'20294", 'reported_seq': 20826, 'reported_epoch': 49, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:08:20.813231+0000', 'last_change': '2024-09-10T16:08:09.280891+0000', 'last_active': '2024-09-10T16:08:20.813231+0000', 'last_peered': '2024-09-10T16:08:20.813231+0000', 'last_clean': '2024-09-10T15:59:14.129081+0000', 'last_became_active': '2024-09-10T15:59:16.269835+0000', 'last_became_peered': '2024-09-10T15:59:16.269835+0000', 'last_unstale': '2024-09-10T16:08:20.813231+0000', 'last_undegraded': '2024-09-10T15:59:15.860857+0000', 'last_fullsized': '2024-09-10T15:59:15.771410+0000', 'mapping_epoch': 42, 'log_start': "45'10200", 'ondisk_log_start': "45'10200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3557", 'last_scrub_stamp': '2024-09-10T15:59:02.493678+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:02.493678+0000', 'objects_scrubbed': 3488, 'log_size': 10094, 'log_dups_size': 0, 'ondisk_log_size': 10094, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1472, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 2523136, 'num_objects': 616, 'num_object_clones': 0, 'num_object_copies': 1232, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4061, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 616, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16798, 'num_write_kb': 27852, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 196, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3445}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:08:23.858 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-10T16:08:23.858 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "49'20137", 'reported_seq': 20282, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:08:21.050025+0000', 'last_change': '2024-09-10T15:59:16.284311+0000', 'last_active': '2024-09-10T16:08:21.050025+0000', 'last_peered': '2024-09-10T16:08:21.050025+0000', 'last_clean': '2024-09-10T15:59:14.116041+0000', 'last_became_active': '2024-09-10T15:59:16.274543+0000', 'last_became_peered': '2024-09-10T15:59:16.274543+0000', 'last_unstale': '2024-09-10T16:08:21.050025+0000', 'last_undegraded': '2024-09-10T15:59:15.812484+0000', 'last_fullsized': '2024-09-10T15:59:15.770885+0000', 'mapping_epoch': 42, 'log_start': "45'10100", 'ondisk_log_start': "45'10100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2356", 'last_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'objects_scrubbed': 2316, 'log_size': 10037, 'log_dups_size': 0, 'ondisk_log_size': 10037, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 783, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 2638307, 'num_objects': 645, 'num_object_clones': 0, 'num_object_copies': 1290, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 645, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 17817, 'num_write_kb': 32292, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': -3659}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:08:23.858 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-10T16:08:23.858 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "49'20544", 'reported_seq': 29399, 'reported_epoch': 49, 'state': 'active+recovery_wait', 'last_fresh': '2024-09-10T16:08:21.030947+0000', 'last_change': '2024-09-10T16:08:10.929169+0000', 'last_active': '2024-09-10T16:08:21.030947+0000', 'last_peered': '2024-09-10T16:08:21.030947+0000', 'last_clean': '2024-09-10T15:59:14.110654+0000', 'last_became_active': '2024-09-10T16:08:10.860438+0000', 'last_became_peered': '2024-09-10T16:08:10.860438+0000', 'last_unstale': '2024-09-10T16:08:21.030947+0000', 'last_undegraded': '2024-09-10T16:08:21.030947+0000', 'last_fullsized': '2024-09-10T16:08:21.030947+0000', 'mapping_epoch': 48, 'log_start': "45'10500", 'ondisk_log_start': "45'10500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2486", 'last_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'objects_scrubbed': 2440, 'log_size': 10044, 'log_dups_size': 0, 'ondisk_log_size': 10044, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 792, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 2564096, 'num_objects': 626, 'num_object_clones': 0, 'num_object_copies': 1252, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 626, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18098, 'num_write_kb': 32568, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 4355, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': ['2', '0', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 626}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:08:23.858 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-10T16:08:23.859 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "49'20190", 'reported_seq': 20335, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:08:20.826779+0000', 'last_change': '2024-09-10T15:59:16.315641+0000', 'last_active': '2024-09-10T16:08:20.826779+0000', 'last_peered': '2024-09-10T16:08:20.826779+0000', 'last_clean': '2024-09-10T15:59:14.117001+0000', 'last_became_active': '2024-09-10T15:59:16.302245+0000', 'last_became_peered': '2024-09-10T15:59:16.302245+0000', 'last_unstale': '2024-09-10T16:08:20.826779+0000', 'last_undegraded': '2024-09-10T15:59:15.845012+0000', 'last_fullsized': '2024-09-10T15:59:15.770337+0000', 'mapping_epoch': 42, 'log_start': "45'10100", 'ondisk_log_start': "45'10100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3404", 'last_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'objects_scrubbed': 3367, 'log_size': 10090, 'log_dups_size': 0, 'ondisk_log_size': 10090, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 983, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 2654208, 'num_objects': 648, 'num_object_clones': 0, 'num_object_copies': 1296, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 648, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16821, 'num_write_kb': 28204, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3656}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:08:23.859 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-10T16:08:23.859 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "49'20415", 'reported_seq': 20567, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:08:20.768543+0000', 'last_change': '2024-09-10T15:59:16.311291+0000', 'last_active': '2024-09-10T16:08:20.768543+0000', 'last_peered': '2024-09-10T16:08:20.768543+0000', 'last_clean': '2024-09-10T15:59:14.116363+0000', 'last_became_active': '2024-09-10T15:59:16.301244+0000', 'last_became_peered': '2024-09-10T15:59:16.301244+0000', 'last_unstale': '2024-09-10T16:08:20.768543+0000', 'last_undegraded': '2024-09-10T15:59:15.883434+0000', 'last_fullsized': '2024-09-10T15:59:15.772756+0000', 'mapping_epoch': 42, 'log_start': "45'10400", 'ondisk_log_start': "45'10400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3789", 'last_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'objects_scrubbed': 3741, 'log_size': 10015, 'log_dups_size': 0, 'ondisk_log_size': 10015, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1440, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 2592768, 'num_objects': 633, 'num_object_clones': 0, 'num_object_copies': 1266, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4373, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 633, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16670, 'num_write_kb': 27124, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3738}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:08:23.859 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-10T16:08:23.859 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "49'20536", 'reported_seq': 20691, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:08:20.775547+0000', 'last_change': '2024-09-10T15:59:16.313409+0000', 'last_active': '2024-09-10T16:08:20.775547+0000', 'last_peered': '2024-09-10T16:08:20.775547+0000', 'last_clean': '2024-09-10T15:59:14.129780+0000', 'last_became_active': '2024-09-10T15:59:16.300932+0000', 'last_became_peered': '2024-09-10T15:59:16.300932+0000', 'last_unstale': '2024-09-10T16:08:20.775547+0000', 'last_undegraded': '2024-09-10T15:59:15.887823+0000', 'last_fullsized': '2024-09-10T15:59:15.768513+0000', 'mapping_epoch': 42, 'log_start': "45'10500", 'ondisk_log_start': "45'10500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3880", 'last_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'objects_scrubbed': 3827, 'log_size': 10036, 'log_dups_size': 0, 'ondisk_log_size': 10036, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1397, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 2712518, 'num_objects': 664, 'num_object_clones': 0, 'num_object_copies': 1328, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4325, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 664, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16699, 'num_write_kb': 27072, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3659}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:08:23.859 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T16:08:24.572 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:08:24.572 INFO:teuthology.orchestra.run.smithi115.stdout:{"fsid":"7effee29-df05-4351-9741-ee203564ed81","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 38361/20534 objects degraded (186.817%), 9 pgs degraded, 9 pgs undersized","count":18},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":668,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":49,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983954,"num_in_osds":3,"osd_in_since":1725983947,"num_remapped_pgs":9},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":8},{"state_name":"active+recovery_wait","count":1},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":10267,"data_bytes":42005257,"bytes_used":1591242752,"bytes_avail":288319049728,"bytes_total":289910292480,"degraded_objects":38361,"degraded_total":20534,"degraded_ratio":1.8681698646147853,"recovering_objects_per_sec":10,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":128,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":384},"fsmap":{"epoch":1,"btime":"2024-09-10T15:57:14:967120+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T16:00:13.560776+0000","services":{}},"progress_events":{}} 2024-09-10T16:08:24.598 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-10T16:08:24.598 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:08:25.222 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:08:25.224 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:08:25.248 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":376,"stamp":"2024-09-10T16:08:23.759465+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":42005257,"num_objects":10267,"num_object_clones":0,"num_object_copies":20534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38361,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10267,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":281843,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13031,"num_bytes_recovered":33095699,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160770,"ondisk_log_size":160770,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1553948,"kb_used_data":291368,"kb_used_omap":12,"kb_used_meta":1262515,"kb_avail":281561572,"statfs":{"total":289910292480,"available":288319049728,"internally_reserved":0,"allocated":298360832,"data_stored":297399130,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1292815362},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,6,6,7,7,9],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18919424,"num_objects":-4619,"num_object_clones":0,"num_object_copies":-9238,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-128,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4619,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4619,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":128,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.008712"},"pg_stats":[{"pgid":"2.f","version":"49'20043","reported_seq":20128,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:20.824435+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:08:20.824435+0000","last_peered":"2024-09-10T16:08:20.824435+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:08:20.824435+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2576384,"num_objects":629,"num_object_clones":0,"num_object_copies":1258,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":629,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20043,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3630}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20206","reported_seq":20329,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:20.956434+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:08:20.956434+0000","last_peered":"2024-09-10T16:08:20.956434+0000","last_clean":"2024-09-10T16:08:20.956434+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:08:20.956434+0000","last_undegraded":"2024-09-10T16:08:20.956434+0000","last_fullsized":"2024-09-10T16:08:20.956434+0000","mapping_epoch":35,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2614214,"num_objects":640,"num_object_clones":0,"num_object_copies":1280,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":640,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18311,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20129","reported_seq":28817,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:19.805778+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:08:19.805778+0000","last_peered":"2024-09-10T16:08:19.805778+0000","last_clean":"2024-09-10T16:08:19.805778+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:08:19.805778+0000","last_undegraded":"2024-09-10T16:08:19.805778+0000","last_fullsized":"2024-09-10T16:08:19.805778+0000","mapping_epoch":44,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2736152,"num_objects":669,"num_object_clones":0,"num_object_copies":1338,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":669,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18133,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20053","reported_seq":20144,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:21.063454+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:08:21.063454+0000","last_peered":"2024-09-10T16:08:21.063454+0000","last_clean":"2024-09-10T16:08:21.063454+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:08:21.063454+0000","last_undegraded":"2024-09-10T16:08:21.063454+0000","last_fullsized":"2024-09-10T16:08:21.063454+0000","mapping_epoch":35,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2486272,"num_objects":607,"num_object_clones":0,"num_object_copies":1214,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":607,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20053,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20246","reported_seq":20386,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:20.722435+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:08:20.722435+0000","last_peered":"2024-09-10T16:08:20.722435+0000","last_clean":"2024-09-10T16:08:20.722435+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:08:20.722435+0000","last_undegraded":"2024-09-10T16:08:20.722435+0000","last_fullsized":"2024-09-10T16:08:20.722435+0000","mapping_epoch":34,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2642403,"num_objects":646,"num_object_clones":0,"num_object_copies":1292,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":646,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16957,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20440","reported_seq":20588,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:20.653037+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:08:20.653037+0000","last_peered":"2024-09-10T16:08:20.653037+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:08:20.653037+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2544099,"num_objects":622,"num_object_clones":0,"num_object_copies":1244,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":622,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17052,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3749},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'20445","reported_seq":20593,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:20.686425+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:08:20.686425+0000","last_peered":"2024-09-10T16:08:20.686425+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:08:20.686425+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2610118,"num_objects":639,"num_object_clones":0,"num_object_copies":1278,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":639,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16944,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3598},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20681","reported_seq":20821,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:20.795233+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:08:20.795233+0000","last_peered":"2024-09-10T16:08:20.795233+0000","last_clean":"2024-09-10T16:08:20.795233+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:08:20.795233+0000","last_undegraded":"2024-09-10T16:08:20.795233+0000","last_fullsized":"2024-09-10T16:08:20.795233+0000","mapping_epoch":34,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2592768,"num_objects":633,"num_object_clones":0,"num_object_copies":1266,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":633,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17528,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.838857+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:08:10.838857+0000","last_peered":"2024-09-10T16:08:10.838857+0000","last_clean":"2024-09-10T16:08:10.838857+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:08:10.838857+0000","last_undegraded":"2024-09-10T16:08:10.838857+0000","last_fullsized":"2024-09-10T16:08:10.838857+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19792","reported_seq":19942,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:20.978206+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:08:20.978206+0000","last_peered":"2024-09-10T16:08:20.978206+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:08:20.978206+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2724323,"num_objects":666,"num_object_clones":0,"num_object_copies":1332,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":666,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17611,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-3457}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":104,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.838197+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:08:10.838197+0000","last_peered":"2024-09-10T16:08:10.838197+0000","last_clean":"2024-09-10T16:08:10.838197+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:08:10.838197+0000","last_undegraded":"2024-09-10T16:08:10.838197+0000","last_fullsized":"2024-09-10T16:08:10.838197+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20294","reported_seq":20826,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:20.813231+0000","last_change":"2024-09-10T16:08:09.280891+0000","last_active":"2024-09-10T16:08:20.813231+0000","last_peered":"2024-09-10T16:08:20.813231+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:08:20.813231+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2523136,"num_objects":616,"num_object_clones":0,"num_object_copies":1232,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4061,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":616,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16798,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":196,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3445}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":114,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.840262+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:08:10.840262+0000","last_peered":"2024-09-10T16:08:10.840262+0000","last_clean":"2024-09-10T16:08:10.840262+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:08:10.840262+0000","last_undegraded":"2024-09-10T16:08:10.840262+0000","last_fullsized":"2024-09-10T16:08:10.840262+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20137","reported_seq":20282,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:21.050025+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:08:21.050025+0000","last_peered":"2024-09-10T16:08:21.050025+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:08:21.050025+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2638307,"num_objects":645,"num_object_clones":0,"num_object_copies":1290,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":645,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17817,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-3659}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":90,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.837937+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:08:10.837937+0000","last_peered":"2024-09-10T16:08:10.837937+0000","last_clean":"2024-09-10T16:08:10.837937+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:08:10.837937+0000","last_undegraded":"2024-09-10T16:08:10.837937+0000","last_fullsized":"2024-09-10T16:08:10.837937+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'20544","reported_seq":29399,"reported_epoch":49,"state":"active+recovery_wait","last_fresh":"2024-09-10T16:08:21.030947+0000","last_change":"2024-09-10T16:08:10.929169+0000","last_active":"2024-09-10T16:08:21.030947+0000","last_peered":"2024-09-10T16:08:21.030947+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T16:08:10.860438+0000","last_became_peered":"2024-09-10T16:08:10.860438+0000","last_unstale":"2024-09-10T16:08:21.030947+0000","last_undegraded":"2024-09-10T16:08:21.030947+0000","last_fullsized":"2024-09-10T16:08:21.030947+0000","mapping_epoch":48,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2564096,"num_objects":626,"num_object_clones":0,"num_object_copies":1252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":626,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18098,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","0","1"],"object_location_counts":[{"shards":"1,2","objects":626}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20190","reported_seq":20335,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:20.826779+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:08:20.826779+0000","last_peered":"2024-09-10T16:08:20.826779+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:08:20.826779+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2654208,"num_objects":648,"num_object_clones":0,"num_object_copies":1296,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":648,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16821,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3656}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.838546+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:08:10.838546+0000","last_peered":"2024-09-10T16:08:10.838546+0000","last_clean":"2024-09-10T16:08:10.838546+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:08:10.838546+0000","last_undegraded":"2024-09-10T16:08:10.838546+0000","last_fullsized":"2024-09-10T16:08:10.838546+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19916","reported_seq":28492,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:19.694053+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:08:19.694053+0000","last_peered":"2024-09-10T16:08:19.694053+0000","last_clean":"2024-09-10T16:08:19.694053+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:08:19.694053+0000","last_undegraded":"2024-09-10T16:08:19.694053+0000","last_fullsized":"2024-09-10T16:08:19.694053+0000","mapping_epoch":46,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2793472,"num_objects":682,"num_object_clones":0,"num_object_copies":1364,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":682,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16306,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":104,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.838936+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:08:10.838936+0000","last_peered":"2024-09-10T16:08:10.838936+0000","last_clean":"2024-09-10T16:08:10.838936+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:08:10.838936+0000","last_undegraded":"2024-09-10T16:08:10.838936+0000","last_fullsized":"2024-09-10T16:08:10.838936+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20415","reported_seq":20567,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:20.768543+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:08:20.768543+0000","last_peered":"2024-09-10T16:08:20.768543+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:08:20.768543+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2592768,"num_objects":633,"num_object_clones":0,"num_object_copies":1266,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":633,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16670,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3738}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.839114+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:08:10.839114+0000","last_peered":"2024-09-10T16:08:10.839114+0000","last_clean":"2024-09-10T16:08:10.839114+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:08:10.839114+0000","last_undegraded":"2024-09-10T16:08:10.839114+0000","last_fullsized":"2024-09-10T16:08:10.839114+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":85,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.838633+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:08:10.838633+0000","last_peered":"2024-09-10T16:08:10.838633+0000","last_clean":"2024-09-10T16:08:10.838633+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:08:10.838633+0000","last_undegraded":"2024-09-10T16:08:10.838633+0000","last_fullsized":"2024-09-10T16:08:10.838633+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'20536","reported_seq":20691,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:20.775547+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:08:20.775547+0000","last_peered":"2024-09-10T16:08:20.775547+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:08:20.775547+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2712518,"num_objects":664,"num_object_clones":0,"num_object_copies":1328,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":664,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16699,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3659},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":42005238,"num_objects":10265,"num_object_clones":0,"num_object_copies":20530,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38361,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10265,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":281841,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13029,"num_bytes_recovered":33095680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":295538688,"data_stored":295404089,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160767,"ondisk_log_size":160767,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986300,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":519456,"kb_used_data":158736,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93852384,"statfs":{"total":96636764160,"available":96104841216,"internally_reserved":0,"allocated":162545664,"data_stored":162232479,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369356458},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,0,2,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659245,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":462120,"kb_used_data":33368,"kb_used_omap":4,"kb_used_meta":428731,"kb_avail":93909720,"statfs":{"total":96636764160,"available":96163553280,"internally_reserved":0,"allocated":34168832,"data_stored":33833505,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":439021230},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,2,1,4,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182528,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":572372,"kb_used_data":99264,"kb_used_omap":4,"kb_used_meta":473083,"kb_avail":93799468,"statfs":{"total":96636764160,"available":96050655232,"internally_reserved":0,"allocated":101646336,"data_stored":101333146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":484437674},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,4,4,2,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":101060608,"data_stored":101024478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":33112064,"data_stored":33053797,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":161366016,"data_stored":161325814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:08:28.250 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:08:28.813 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:08:28.816 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:08:28.843 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":378,"stamp":"2024-09-10T16:08:27.760360+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":32240393,"num_objects":7883,"num_object_clones":0,"num_object_copies":15766,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38290,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7883,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":284227,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13102,"num_bytes_recovered":33095699,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161054,"ondisk_log_size":161054,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1552908,"kb_used_data":272280,"kb_used_omap":12,"kb_used_meta":1280563,"kb_avail":281562612,"statfs":{"total":289910292480,"available":288320114688,"internally_reserved":0,"allocated":278814720,"data_stored":277853018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1311296514},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[6,1,3,5,4,4,15,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-27602944,"num_objects":-6739,"num_object_clones":0,"num_object_copies":-13478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-199,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-6739,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6739,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":199,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.008777"},"pg_stats":[{"pgid":"2.f","version":"49'20200","reported_seq":20285,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:25.849767+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:08:25.849767+0000","last_peered":"2024-09-10T16:08:25.849767+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:08:25.849767+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1933312,"num_objects":472,"num_object_clones":0,"num_object_copies":944,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":472,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20200,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3787}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20341","reported_seq":20464,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:26.047086+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:08:26.047086+0000","last_peered":"2024-09-10T16:08:26.047086+0000","last_clean":"2024-09-10T16:08:26.047086+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:08:26.047086+0000","last_undegraded":"2024-09-10T16:08:26.047086+0000","last_fullsized":"2024-09-10T16:08:26.047086+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2061254,"num_objects":505,"num_object_clones":0,"num_object_copies":1010,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":505,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18446,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20287","reported_seq":28975,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:24.810711+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:08:24.810711+0000","last_peered":"2024-09-10T16:08:24.810711+0000","last_clean":"2024-09-10T16:08:24.810711+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:08:24.810711+0000","last_undegraded":"2024-09-10T16:08:24.810711+0000","last_fullsized":"2024-09-10T16:08:24.810711+0000","mapping_epoch":44,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2088984,"num_objects":511,"num_object_clones":0,"num_object_copies":1022,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":511,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18291,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20187","reported_seq":20278,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:26.029548+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:08:26.029548+0000","last_peered":"2024-09-10T16:08:26.029548+0000","last_clean":"2024-09-10T16:08:26.029548+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:08:26.029548+0000","last_undegraded":"2024-09-10T16:08:26.029548+0000","last_fullsized":"2024-09-10T16:08:26.029548+0000","mapping_epoch":35,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1937408,"num_objects":473,"num_object_clones":0,"num_object_copies":946,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":473,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20187,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20404","reported_seq":20544,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:25.878436+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:08:25.878436+0000","last_peered":"2024-09-10T16:08:25.878436+0000","last_clean":"2024-09-10T16:08:25.878436+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:08:25.878436+0000","last_undegraded":"2024-09-10T16:08:25.878436+0000","last_fullsized":"2024-09-10T16:08:25.878436+0000","mapping_epoch":34,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1995235,"num_objects":488,"num_object_clones":0,"num_object_copies":976,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":488,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17115,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20579","reported_seq":20727,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:25.837253+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:08:25.837253+0000","last_peered":"2024-09-10T16:08:25.837253+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:08:25.837253+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1974755,"num_objects":483,"num_object_clones":0,"num_object_copies":966,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":483,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17191,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3888},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'20599","reported_seq":20746,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:25.860026+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:08:25.860026+0000","last_peered":"2024-09-10T16:08:25.860026+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:08:25.860026+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1979334,"num_objects":485,"num_object_clones":0,"num_object_copies":970,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":485,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17098,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3752},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20837","reported_seq":20977,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:25.824261+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:08:25.824261+0000","last_peered":"2024-09-10T16:08:25.824261+0000","last_clean":"2024-09-10T16:08:25.824261+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:08:25.824261+0000","last_undegraded":"2024-09-10T16:08:25.824261+0000","last_fullsized":"2024-09-10T16:08:25.824261+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1953792,"num_objects":477,"num_object_clones":0,"num_object_copies":954,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":477,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17684,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.838857+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:08:10.838857+0000","last_peered":"2024-09-10T16:08:10.838857+0000","last_clean":"2024-09-10T16:08:10.838857+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:08:10.838857+0000","last_undegraded":"2024-09-10T16:08:10.838857+0000","last_fullsized":"2024-09-10T16:08:10.838857+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19938","reported_seq":20088,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:26.067262+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:08:26.067262+0000","last_peered":"2024-09-10T16:08:26.067262+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:08:26.067262+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2126307,"num_objects":520,"num_object_clones":0,"num_object_copies":1040,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":520,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17757,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-3603}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":104,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.838197+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:08:10.838197+0000","last_peered":"2024-09-10T16:08:10.838197+0000","last_clean":"2024-09-10T16:08:10.838197+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:08:10.838197+0000","last_undegraded":"2024-09-10T16:08:10.838197+0000","last_fullsized":"2024-09-10T16:08:10.838197+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20436","reported_seq":21112,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:25.863377+0000","last_change":"2024-09-10T16:08:09.280891+0000","last_active":"2024-09-10T16:08:25.863377+0000","last_peered":"2024-09-10T16:08:25.863377+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:08:25.863377+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1941504,"num_objects":474,"num_object_clones":0,"num_object_copies":948,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3990,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":474,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16940,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":267,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3516}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":114,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.840262+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:08:10.840262+0000","last_peered":"2024-09-10T16:08:10.840262+0000","last_clean":"2024-09-10T16:08:10.840262+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:08:10.840262+0000","last_undegraded":"2024-09-10T16:08:10.840262+0000","last_fullsized":"2024-09-10T16:08:10.840262+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20281","reported_seq":20426,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:26.031921+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:08:26.031921+0000","last_peered":"2024-09-10T16:08:26.031921+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:08:26.031921+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2048483,"num_objects":501,"num_object_clones":0,"num_object_copies":1002,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":501,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17961,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-3803}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":90,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.837937+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:08:10.837937+0000","last_peered":"2024-09-10T16:08:10.837937+0000","last_clean":"2024-09-10T16:08:10.837937+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:08:10.837937+0000","last_undegraded":"2024-09-10T16:08:10.837937+0000","last_fullsized":"2024-09-10T16:08:10.837937+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'20688","reported_seq":29543,"reported_epoch":49,"state":"active+recovery_wait","last_fresh":"2024-09-10T16:08:26.055161+0000","last_change":"2024-09-10T16:08:10.929169+0000","last_active":"2024-09-10T16:08:26.055161+0000","last_peered":"2024-09-10T16:08:26.055161+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T16:08:10.860438+0000","last_became_peered":"2024-09-10T16:08:10.860438+0000","last_unstale":"2024-09-10T16:08:26.055161+0000","last_undegraded":"2024-09-10T16:08:26.055161+0000","last_fullsized":"2024-09-10T16:08:26.055161+0000","mapping_epoch":48,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1974272,"num_objects":482,"num_object_clones":0,"num_object_copies":964,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":482,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18242,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","0","1"],"object_location_counts":[{"shards":"1,2","objects":482}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20342","reported_seq":20487,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:25.853483+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:08:25.853483+0000","last_peered":"2024-09-10T16:08:25.853483+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:08:25.853483+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2031616,"num_objects":496,"num_object_clones":0,"num_object_copies":992,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":496,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16973,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3808}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.838546+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:08:10.838546+0000","last_peered":"2024-09-10T16:08:10.838546+0000","last_clean":"2024-09-10T16:08:10.838546+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:08:10.838546+0000","last_undegraded":"2024-09-10T16:08:10.838546+0000","last_fullsized":"2024-09-10T16:08:10.838546+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20083","reported_seq":28659,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:24.782476+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:08:24.782476+0000","last_peered":"2024-09-10T16:08:24.782476+0000","last_clean":"2024-09-10T16:08:24.782476+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:08:24.782476+0000","last_undegraded":"2024-09-10T16:08:24.782476+0000","last_fullsized":"2024-09-10T16:08:24.782476+0000","mapping_epoch":46,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2109440,"num_objects":515,"num_object_clones":0,"num_object_copies":1030,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":515,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16473,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":104,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.838936+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:08:10.838936+0000","last_peered":"2024-09-10T16:08:10.838936+0000","last_clean":"2024-09-10T16:08:10.838936+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:08:10.838936+0000","last_undegraded":"2024-09-10T16:08:10.838936+0000","last_fullsized":"2024-09-10T16:08:10.838936+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20551","reported_seq":20703,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:25.812765+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:08:25.812765+0000","last_peered":"2024-09-10T16:08:25.812765+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:08:25.812765+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2035712,"num_objects":497,"num_object_clones":0,"num_object_copies":994,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":497,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16806,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3874}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.839114+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:08:10.839114+0000","last_peered":"2024-09-10T16:08:10.839114+0000","last_clean":"2024-09-10T16:08:10.839114+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:08:10.839114+0000","last_undegraded":"2024-09-10T16:08:10.839114+0000","last_fullsized":"2024-09-10T16:08:10.839114+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":85,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.838633+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:08:10.838633+0000","last_peered":"2024-09-10T16:08:10.838633+0000","last_clean":"2024-09-10T16:08:10.838633+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:08:10.838633+0000","last_undegraded":"2024-09-10T16:08:10.838633+0000","last_fullsized":"2024-09-10T16:08:10.838633+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'20698","reported_seq":20851,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:25.754988+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:08:25.754988+0000","last_peered":"2024-09-10T16:08:25.754988+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:08:25.754988+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2048966,"num_objects":502,"num_object_clones":0,"num_object_copies":1004,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":502,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16861,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3821},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":32240374,"num_objects":7881,"num_object_clones":0,"num_object_copies":15762,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38290,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7881,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":284225,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13100,"num_bytes_recovered":33095680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":275988480,"data_stored":275853881,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161051,"ondisk_log_size":161051,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986301,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518988,"kb_used_data":154108,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93852852,"statfs":{"total":96636764160,"available":96105320448,"internally_reserved":0,"allocated":157806592,"data_stored":157493407,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,0,0,2,0,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659246,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":481240,"kb_used_data":26116,"kb_used_omap":4,"kb_used_meta":455099,"kb_avail":93890600,"statfs":{"total":96636764160,"available":96143974400,"internally_reserved":0,"allocated":26742784,"data_stored":26407457,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":466022062},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,3,0,2,6,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182529,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":552680,"kb_used_data":92056,"kb_used_omap":4,"kb_used_meta":460603,"kb_avail":93819160,"statfs":{"total":96636764160,"available":96070819840,"internally_reserved":0,"allocated":94265344,"data_stored":93952154,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":471658154},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,3,2,2,2,6,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":93687808,"data_stored":93651678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":25694208,"data_stored":25635941,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":156606464,"data_stored":156566262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:08:28.844 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:08:29.431 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:08:29.432 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:08:29.452 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":378,"stamp":"2024-09-10T16:08:27.760360+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":32240393,"num_objects":7883,"num_object_clones":0,"num_object_copies":15766,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38290,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7883,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":284227,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13102,"num_bytes_recovered":33095699,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161054,"ondisk_log_size":161054,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1552908,"kb_used_data":272280,"kb_used_omap":12,"kb_used_meta":1280563,"kb_avail":281562612,"statfs":{"total":289910292480,"available":288320114688,"internally_reserved":0,"allocated":278814720,"data_stored":277853018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1311296514},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[6,1,3,5,4,4,15,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-27602944,"num_objects":-6739,"num_object_clones":0,"num_object_copies":-13478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-199,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-6739,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6739,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":199,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.008777"},"pg_stats":[{"pgid":"2.f","version":"49'20200","reported_seq":20285,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:25.849767+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:08:25.849767+0000","last_peered":"2024-09-10T16:08:25.849767+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:08:25.849767+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1933312,"num_objects":472,"num_object_clones":0,"num_object_copies":944,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":472,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20200,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3787}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20341","reported_seq":20464,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:26.047086+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:08:26.047086+0000","last_peered":"2024-09-10T16:08:26.047086+0000","last_clean":"2024-09-10T16:08:26.047086+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:08:26.047086+0000","last_undegraded":"2024-09-10T16:08:26.047086+0000","last_fullsized":"2024-09-10T16:08:26.047086+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2061254,"num_objects":505,"num_object_clones":0,"num_object_copies":1010,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":505,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18446,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20287","reported_seq":28975,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:24.810711+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:08:24.810711+0000","last_peered":"2024-09-10T16:08:24.810711+0000","last_clean":"2024-09-10T16:08:24.810711+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:08:24.810711+0000","last_undegraded":"2024-09-10T16:08:24.810711+0000","last_fullsized":"2024-09-10T16:08:24.810711+0000","mapping_epoch":44,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2088984,"num_objects":511,"num_object_clones":0,"num_object_copies":1022,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":511,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18291,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20187","reported_seq":20278,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:26.029548+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:08:26.029548+0000","last_peered":"2024-09-10T16:08:26.029548+0000","last_clean":"2024-09-10T16:08:26.029548+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:08:26.029548+0000","last_undegraded":"2024-09-10T16:08:26.029548+0000","last_fullsized":"2024-09-10T16:08:26.029548+0000","mapping_epoch":35,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1937408,"num_objects":473,"num_object_clones":0,"num_object_copies":946,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":473,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20187,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20404","reported_seq":20544,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:25.878436+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:08:25.878436+0000","last_peered":"2024-09-10T16:08:25.878436+0000","last_clean":"2024-09-10T16:08:25.878436+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:08:25.878436+0000","last_undegraded":"2024-09-10T16:08:25.878436+0000","last_fullsized":"2024-09-10T16:08:25.878436+0000","mapping_epoch":34,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1995235,"num_objects":488,"num_object_clones":0,"num_object_copies":976,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":488,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17115,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20579","reported_seq":20727,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:25.837253+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:08:25.837253+0000","last_peered":"2024-09-10T16:08:25.837253+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:08:25.837253+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1974755,"num_objects":483,"num_object_clones":0,"num_object_copies":966,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":483,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17191,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3888},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'20599","reported_seq":20746,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:25.860026+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:08:25.860026+0000","last_peered":"2024-09-10T16:08:25.860026+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:08:25.860026+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1979334,"num_objects":485,"num_object_clones":0,"num_object_copies":970,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":485,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17098,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3752},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20837","reported_seq":20977,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:25.824261+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:08:25.824261+0000","last_peered":"2024-09-10T16:08:25.824261+0000","last_clean":"2024-09-10T16:08:25.824261+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:08:25.824261+0000","last_undegraded":"2024-09-10T16:08:25.824261+0000","last_fullsized":"2024-09-10T16:08:25.824261+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1953792,"num_objects":477,"num_object_clones":0,"num_object_copies":954,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":477,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17684,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.838857+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:08:10.838857+0000","last_peered":"2024-09-10T16:08:10.838857+0000","last_clean":"2024-09-10T16:08:10.838857+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:08:10.838857+0000","last_undegraded":"2024-09-10T16:08:10.838857+0000","last_fullsized":"2024-09-10T16:08:10.838857+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19938","reported_seq":20088,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:26.067262+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:08:26.067262+0000","last_peered":"2024-09-10T16:08:26.067262+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:08:26.067262+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2126307,"num_objects":520,"num_object_clones":0,"num_object_copies":1040,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":520,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17757,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-3603}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":104,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.838197+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:08:10.838197+0000","last_peered":"2024-09-10T16:08:10.838197+0000","last_clean":"2024-09-10T16:08:10.838197+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:08:10.838197+0000","last_undegraded":"2024-09-10T16:08:10.838197+0000","last_fullsized":"2024-09-10T16:08:10.838197+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20436","reported_seq":21112,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:25.863377+0000","last_change":"2024-09-10T16:08:09.280891+0000","last_active":"2024-09-10T16:08:25.863377+0000","last_peered":"2024-09-10T16:08:25.863377+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:08:25.863377+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1941504,"num_objects":474,"num_object_clones":0,"num_object_copies":948,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3990,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":474,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16940,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":267,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3516}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":114,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.840262+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:08:10.840262+0000","last_peered":"2024-09-10T16:08:10.840262+0000","last_clean":"2024-09-10T16:08:10.840262+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:08:10.840262+0000","last_undegraded":"2024-09-10T16:08:10.840262+0000","last_fullsized":"2024-09-10T16:08:10.840262+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20281","reported_seq":20426,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:26.031921+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:08:26.031921+0000","last_peered":"2024-09-10T16:08:26.031921+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:08:26.031921+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2048483,"num_objects":501,"num_object_clones":0,"num_object_copies":1002,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":501,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17961,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-3803}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":90,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.837937+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:08:10.837937+0000","last_peered":"2024-09-10T16:08:10.837937+0000","last_clean":"2024-09-10T16:08:10.837937+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:08:10.837937+0000","last_undegraded":"2024-09-10T16:08:10.837937+0000","last_fullsized":"2024-09-10T16:08:10.837937+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'20688","reported_seq":29543,"reported_epoch":49,"state":"active+recovery_wait","last_fresh":"2024-09-10T16:08:26.055161+0000","last_change":"2024-09-10T16:08:10.929169+0000","last_active":"2024-09-10T16:08:26.055161+0000","last_peered":"2024-09-10T16:08:26.055161+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T16:08:10.860438+0000","last_became_peered":"2024-09-10T16:08:10.860438+0000","last_unstale":"2024-09-10T16:08:26.055161+0000","last_undegraded":"2024-09-10T16:08:26.055161+0000","last_fullsized":"2024-09-10T16:08:26.055161+0000","mapping_epoch":48,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1974272,"num_objects":482,"num_object_clones":0,"num_object_copies":964,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":482,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18242,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","0","1"],"object_location_counts":[{"shards":"1,2","objects":482}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20342","reported_seq":20487,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:25.853483+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:08:25.853483+0000","last_peered":"2024-09-10T16:08:25.853483+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:08:25.853483+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2031616,"num_objects":496,"num_object_clones":0,"num_object_copies":992,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":496,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16973,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3808}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.838546+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:08:10.838546+0000","last_peered":"2024-09-10T16:08:10.838546+0000","last_clean":"2024-09-10T16:08:10.838546+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:08:10.838546+0000","last_undegraded":"2024-09-10T16:08:10.838546+0000","last_fullsized":"2024-09-10T16:08:10.838546+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20083","reported_seq":28659,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:24.782476+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:08:24.782476+0000","last_peered":"2024-09-10T16:08:24.782476+0000","last_clean":"2024-09-10T16:08:24.782476+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:08:24.782476+0000","last_undegraded":"2024-09-10T16:08:24.782476+0000","last_fullsized":"2024-09-10T16:08:24.782476+0000","mapping_epoch":46,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2109440,"num_objects":515,"num_object_clones":0,"num_object_copies":1030,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":515,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16473,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":104,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.838936+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:08:10.838936+0000","last_peered":"2024-09-10T16:08:10.838936+0000","last_clean":"2024-09-10T16:08:10.838936+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:08:10.838936+0000","last_undegraded":"2024-09-10T16:08:10.838936+0000","last_fullsized":"2024-09-10T16:08:10.838936+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20551","reported_seq":20703,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:25.812765+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:08:25.812765+0000","last_peered":"2024-09-10T16:08:25.812765+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:08:25.812765+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2035712,"num_objects":497,"num_object_clones":0,"num_object_copies":994,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":497,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16806,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3874}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.839114+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:08:10.839114+0000","last_peered":"2024-09-10T16:08:10.839114+0000","last_clean":"2024-09-10T16:08:10.839114+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:08:10.839114+0000","last_undegraded":"2024-09-10T16:08:10.839114+0000","last_fullsized":"2024-09-10T16:08:10.839114+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":85,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.838633+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:08:10.838633+0000","last_peered":"2024-09-10T16:08:10.838633+0000","last_clean":"2024-09-10T16:08:10.838633+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:08:10.838633+0000","last_undegraded":"2024-09-10T16:08:10.838633+0000","last_fullsized":"2024-09-10T16:08:10.838633+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'20698","reported_seq":20851,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:25.754988+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:08:25.754988+0000","last_peered":"2024-09-10T16:08:25.754988+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:08:25.754988+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2048966,"num_objects":502,"num_object_clones":0,"num_object_copies":1004,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":502,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16861,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3821},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":32240374,"num_objects":7881,"num_object_clones":0,"num_object_copies":15762,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38290,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7881,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":284225,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13100,"num_bytes_recovered":33095680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":275988480,"data_stored":275853881,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161051,"ondisk_log_size":161051,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986301,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518988,"kb_used_data":154108,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93852852,"statfs":{"total":96636764160,"available":96105320448,"internally_reserved":0,"allocated":157806592,"data_stored":157493407,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,0,0,2,0,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659246,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":481240,"kb_used_data":26116,"kb_used_omap":4,"kb_used_meta":455099,"kb_avail":93890600,"statfs":{"total":96636764160,"available":96143974400,"internally_reserved":0,"allocated":26742784,"data_stored":26407457,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":466022062},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,3,0,2,6,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182529,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":552680,"kb_used_data":92056,"kb_used_omap":4,"kb_used_meta":460603,"kb_avail":93819160,"statfs":{"total":96636764160,"available":96070819840,"internally_reserved":0,"allocated":94265344,"data_stored":93952154,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":471658154},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,3,2,2,2,6,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":93687808,"data_stored":93651678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":25694208,"data_stored":25635941,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":156606464,"data_stored":156566262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:08:29.454 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-10T16:08:29.454 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "49'20200", 'reported_seq': 20285, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:08:25.849767+0000', 'last_change': '2024-09-10T15:59:16.281558+0000', 'last_active': '2024-09-10T16:08:25.849767+0000', 'last_peered': '2024-09-10T16:08:25.849767+0000', 'last_clean': '2024-09-10T15:59:14.132456+0000', 'last_became_active': '2024-09-10T15:59:16.274407+0000', 'last_became_peered': '2024-09-10T15:59:16.274407+0000', 'last_unstale': '2024-09-10T16:08:25.849767+0000', 'last_undegraded': '2024-09-10T15:59:15.851279+0000', 'last_fullsized': '2024-09-10T15:59:15.774356+0000', 'mapping_epoch': 42, 'log_start': "45'10100", 'ondisk_log_start': "45'10100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'objects_scrubbed': 0, 'log_size': 10100, 'log_dups_size': 0, 'ondisk_log_size': 10100, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 1933312, 'num_objects': 472, 'num_object_clones': 0, 'num_object_copies': 944, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4259, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 472, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20200, 'num_write_kb': 41344, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3787}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:08:29.454 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-10T16:08:29.454 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "49'20579", 'reported_seq': 20727, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:08:25.837253+0000', 'last_change': '2024-09-10T15:59:16.308264+0000', 'last_active': '2024-09-10T16:08:25.837253+0000', 'last_peered': '2024-09-10T16:08:25.837253+0000', 'last_clean': '2024-09-10T15:59:14.111551+0000', 'last_became_active': '2024-09-10T15:59:16.300040+0000', 'last_became_peered': '2024-09-10T15:59:16.300040+0000', 'last_unstale': '2024-09-10T16:08:25.837253+0000', 'last_undegraded': '2024-09-10T15:59:15.878234+0000', 'last_fullsized': '2024-09-10T15:59:15.769328+0000', 'mapping_epoch': 42, 'log_start': "45'10500", 'ondisk_log_start': "45'10500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3442", 'last_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'objects_scrubbed': 3384, 'log_size': 10079, 'log_dups_size': 0, 'ondisk_log_size': 10079, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1292, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 1974755, 'num_objects': 483, 'num_object_clones': 0, 'num_object_copies': 966, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4372, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 483, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 17191, 'num_write_kb': 28580, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3888}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:08:29.454 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-10T16:08:29.454 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "49'20599", 'reported_seq': 20746, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:08:25.860026+0000', 'last_change': '2024-09-10T15:59:16.331899+0000', 'last_active': '2024-09-10T16:08:25.860026+0000', 'last_peered': '2024-09-10T16:08:25.860026+0000', 'last_clean': '2024-09-10T15:59:14.139436+0000', 'last_became_active': '2024-09-10T15:59:16.318715+0000', 'last_became_peered': '2024-09-10T15:59:16.318715+0000', 'last_unstale': '2024-09-10T16:08:25.860026+0000', 'last_undegraded': '2024-09-10T15:59:15.877524+0000', 'last_fullsized': '2024-09-10T15:59:15.769042+0000', 'mapping_epoch': 42, 'log_start': "45'10500", 'ondisk_log_start': "45'10500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3537", 'last_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'objects_scrubbed': 3497, 'log_size': 10099, 'log_dups_size': 0, 'ondisk_log_size': 10099, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1224, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 1979334, 'num_objects': 485, 'num_object_clones': 0, 'num_object_copies': 970, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4237, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 485, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 17098, 'num_write_kb': 28172, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3752}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:08:29.454 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-10T16:08:29.454 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "49'19938", 'reported_seq': 20088, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:08:26.067262+0000', 'last_change': '2024-09-10T15:59:16.175680+0000', 'last_active': '2024-09-10T16:08:26.067262+0000', 'last_peered': '2024-09-10T16:08:26.067262+0000', 'last_clean': '2024-09-10T15:59:14.088187+0000', 'last_became_active': '2024-09-10T15:59:16.165810+0000', 'last_became_peered': '2024-09-10T15:59:16.165810+0000', 'last_unstale': '2024-09-10T16:08:26.067262+0000', 'last_undegraded': '2024-09-10T15:59:15.822098+0000', 'last_fullsized': '2024-09-10T15:59:15.769123+0000', 'mapping_epoch': 42, 'log_start': "45'9900", 'ondisk_log_start': "45'9900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2218", 'last_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'objects_scrubbed': 2175, 'log_size': 10038, 'log_dups_size': 0, 'ondisk_log_size': 10038, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 825, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 2126307, 'num_objects': 520, 'num_object_clones': 0, 'num_object_copies': 1040, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4124, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 520, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 17757, 'num_write_kb': 32204, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': -3603}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:08:29.455 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-10T16:08:29.455 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "49'20436", 'reported_seq': 21112, 'reported_epoch': 49, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:08:25.863377+0000', 'last_change': '2024-09-10T16:08:09.280891+0000', 'last_active': '2024-09-10T16:08:25.863377+0000', 'last_peered': '2024-09-10T16:08:25.863377+0000', 'last_clean': '2024-09-10T15:59:14.129081+0000', 'last_became_active': '2024-09-10T15:59:16.269835+0000', 'last_became_peered': '2024-09-10T15:59:16.269835+0000', 'last_unstale': '2024-09-10T16:08:25.863377+0000', 'last_undegraded': '2024-09-10T15:59:15.860857+0000', 'last_fullsized': '2024-09-10T15:59:15.771410+0000', 'mapping_epoch': 42, 'log_start': "45'10400", 'ondisk_log_start': "45'10400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3557", 'last_scrub_stamp': '2024-09-10T15:59:02.493678+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:02.493678+0000', 'objects_scrubbed': 3488, 'log_size': 10036, 'log_dups_size': 0, 'ondisk_log_size': 10036, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1472, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 1941504, 'num_objects': 474, 'num_object_clones': 0, 'num_object_copies': 948, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3990, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 474, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16940, 'num_write_kb': 27852, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 267, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3516}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:08:29.455 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-10T16:08:29.455 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "49'20281", 'reported_seq': 20426, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:08:26.031921+0000', 'last_change': '2024-09-10T15:59:16.284311+0000', 'last_active': '2024-09-10T16:08:26.031921+0000', 'last_peered': '2024-09-10T16:08:26.031921+0000', 'last_clean': '2024-09-10T15:59:14.116041+0000', 'last_became_active': '2024-09-10T15:59:16.274543+0000', 'last_became_peered': '2024-09-10T15:59:16.274543+0000', 'last_unstale': '2024-09-10T16:08:26.031921+0000', 'last_undegraded': '2024-09-10T15:59:15.812484+0000', 'last_fullsized': '2024-09-10T15:59:15.770885+0000', 'mapping_epoch': 42, 'log_start': "45'10200", 'ondisk_log_start': "45'10200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2356", 'last_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'objects_scrubbed': 2316, 'log_size': 10081, 'log_dups_size': 0, 'ondisk_log_size': 10081, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 783, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 2048483, 'num_objects': 501, 'num_object_clones': 0, 'num_object_copies': 1002, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 501, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 17961, 'num_write_kb': 32292, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': -3803}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:08:29.455 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-10T16:08:29.455 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "49'20688", 'reported_seq': 29543, 'reported_epoch': 49, 'state': 'active+recovery_wait', 'last_fresh': '2024-09-10T16:08:26.055161+0000', 'last_change': '2024-09-10T16:08:10.929169+0000', 'last_active': '2024-09-10T16:08:26.055161+0000', 'last_peered': '2024-09-10T16:08:26.055161+0000', 'last_clean': '2024-09-10T15:59:14.110654+0000', 'last_became_active': '2024-09-10T16:08:10.860438+0000', 'last_became_peered': '2024-09-10T16:08:10.860438+0000', 'last_unstale': '2024-09-10T16:08:26.055161+0000', 'last_undegraded': '2024-09-10T16:08:26.055161+0000', 'last_fullsized': '2024-09-10T16:08:26.055161+0000', 'mapping_epoch': 48, 'log_start': "45'10600", 'ondisk_log_start': "45'10600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2486", 'last_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'objects_scrubbed': 2440, 'log_size': 10088, 'log_dups_size': 0, 'ondisk_log_size': 10088, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 792, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 1974272, 'num_objects': 482, 'num_object_clones': 0, 'num_object_copies': 964, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 482, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18242, 'num_write_kb': 32568, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 4355, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': ['2', '0', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 482}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:08:29.455 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-10T16:08:29.455 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "49'20342", 'reported_seq': 20487, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:08:25.853483+0000', 'last_change': '2024-09-10T15:59:16.315641+0000', 'last_active': '2024-09-10T16:08:25.853483+0000', 'last_peered': '2024-09-10T16:08:25.853483+0000', 'last_clean': '2024-09-10T15:59:14.117001+0000', 'last_became_active': '2024-09-10T15:59:16.302245+0000', 'last_became_peered': '2024-09-10T15:59:16.302245+0000', 'last_unstale': '2024-09-10T16:08:25.853483+0000', 'last_undegraded': '2024-09-10T15:59:15.845012+0000', 'last_fullsized': '2024-09-10T15:59:15.770337+0000', 'mapping_epoch': 42, 'log_start': "45'10300", 'ondisk_log_start': "45'10300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3404", 'last_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'objects_scrubbed': 3367, 'log_size': 10042, 'log_dups_size': 0, 'ondisk_log_size': 10042, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 983, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 2031616, 'num_objects': 496, 'num_object_clones': 0, 'num_object_copies': 992, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 496, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16973, 'num_write_kb': 28204, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3808}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:08:29.455 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-10T16:08:29.455 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "49'20551", 'reported_seq': 20703, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:08:25.812765+0000', 'last_change': '2024-09-10T15:59:16.311291+0000', 'last_active': '2024-09-10T16:08:25.812765+0000', 'last_peered': '2024-09-10T16:08:25.812765+0000', 'last_clean': '2024-09-10T15:59:14.116363+0000', 'last_became_active': '2024-09-10T15:59:16.301244+0000', 'last_became_peered': '2024-09-10T15:59:16.301244+0000', 'last_unstale': '2024-09-10T16:08:25.812765+0000', 'last_undegraded': '2024-09-10T15:59:15.883434+0000', 'last_fullsized': '2024-09-10T15:59:15.772756+0000', 'mapping_epoch': 42, 'log_start': "45'10500", 'ondisk_log_start': "45'10500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3789", 'last_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'objects_scrubbed': 3741, 'log_size': 10051, 'log_dups_size': 0, 'ondisk_log_size': 10051, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1440, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 2035712, 'num_objects': 497, 'num_object_clones': 0, 'num_object_copies': 994, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4373, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 497, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16806, 'num_write_kb': 27124, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3874}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:08:29.455 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-10T16:08:29.455 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "49'20698", 'reported_seq': 20851, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:08:25.754988+0000', 'last_change': '2024-09-10T15:59:16.313409+0000', 'last_active': '2024-09-10T16:08:25.754988+0000', 'last_peered': '2024-09-10T16:08:25.754988+0000', 'last_clean': '2024-09-10T15:59:14.129780+0000', 'last_became_active': '2024-09-10T15:59:16.300932+0000', 'last_became_peered': '2024-09-10T15:59:16.300932+0000', 'last_unstale': '2024-09-10T16:08:25.754988+0000', 'last_undegraded': '2024-09-10T15:59:15.887823+0000', 'last_fullsized': '2024-09-10T15:59:15.768513+0000', 'mapping_epoch': 42, 'log_start': "45'10600", 'ondisk_log_start': "45'10600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3880", 'last_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'objects_scrubbed': 3827, 'log_size': 10098, 'log_dups_size': 0, 'ondisk_log_size': 10098, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1397, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 2048966, 'num_objects': 502, 'num_object_clones': 0, 'num_object_copies': 1004, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4325, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 502, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16861, 'num_write_kb': 27072, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3821}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:08:29.456 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T16:08:30.169 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:08:30.169 INFO:teuthology.orchestra.run.smithi115.stdout:{"fsid":"7effee29-df05-4351-9741-ee203564ed81","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 38290/15766 objects degraded (242.864%), 9 pgs degraded, 9 pgs undersized","count":18},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":674,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":49,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983954,"num_in_osds":3,"osd_in_since":1725983947,"num_remapped_pgs":9},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":8},{"state_name":"active+recovery_wait","count":1},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":7883,"data_bytes":32240393,"bytes_used":1590177792,"bytes_avail":288320114688,"bytes_total":289910292480,"degraded_objects":38290,"degraded_total":15766,"degraded_ratio":2.4286439172903718,"recovering_objects_per_sec":16,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":199,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":561},"fsmap":{"epoch":1,"btime":"2024-09-10T15:57:14:967120+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T16:00:13.560776+0000","services":{}},"progress_events":{}} 2024-09-10T16:08:30.200 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-10T16:08:30.201 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:08:30.730 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:08:30.732 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:08:30.756 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":379,"stamp":"2024-09-10T16:08:29.760830+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":32240393,"num_objects":7883,"num_object_clones":0,"num_object_copies":15766,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38290,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7883,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":284227,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13102,"num_bytes_recovered":33095699,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161054,"ondisk_log_size":161054,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1552908,"kb_used_data":272280,"kb_used_omap":12,"kb_used_meta":1280563,"kb_avail":281562612,"statfs":{"total":289910292480,"available":288320114688,"internally_reserved":0,"allocated":278814720,"data_stored":277853018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1311296514},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[6,1,3,5,4,4,15,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19447808,"num_objects":-4748,"num_object_clones":0,"num_object_copies":-9496,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-127,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4748,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4748,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":127,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006255"},"pg_stats":[{"pgid":"2.f","version":"49'20200","reported_seq":20285,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:25.849767+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:08:25.849767+0000","last_peered":"2024-09-10T16:08:25.849767+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:08:25.849767+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1933312,"num_objects":472,"num_object_clones":0,"num_object_copies":944,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":472,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20200,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3787}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20341","reported_seq":20464,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:26.047086+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:08:26.047086+0000","last_peered":"2024-09-10T16:08:26.047086+0000","last_clean":"2024-09-10T16:08:26.047086+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:08:26.047086+0000","last_undegraded":"2024-09-10T16:08:26.047086+0000","last_fullsized":"2024-09-10T16:08:26.047086+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2061254,"num_objects":505,"num_object_clones":0,"num_object_copies":1010,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":505,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18446,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20287","reported_seq":28975,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:24.810711+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:08:24.810711+0000","last_peered":"2024-09-10T16:08:24.810711+0000","last_clean":"2024-09-10T16:08:24.810711+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:08:24.810711+0000","last_undegraded":"2024-09-10T16:08:24.810711+0000","last_fullsized":"2024-09-10T16:08:24.810711+0000","mapping_epoch":44,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2088984,"num_objects":511,"num_object_clones":0,"num_object_copies":1022,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":511,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18291,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20187","reported_seq":20278,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:26.029548+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:08:26.029548+0000","last_peered":"2024-09-10T16:08:26.029548+0000","last_clean":"2024-09-10T16:08:26.029548+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:08:26.029548+0000","last_undegraded":"2024-09-10T16:08:26.029548+0000","last_fullsized":"2024-09-10T16:08:26.029548+0000","mapping_epoch":35,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1937408,"num_objects":473,"num_object_clones":0,"num_object_copies":946,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":473,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20187,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20404","reported_seq":20544,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:25.878436+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:08:25.878436+0000","last_peered":"2024-09-10T16:08:25.878436+0000","last_clean":"2024-09-10T16:08:25.878436+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:08:25.878436+0000","last_undegraded":"2024-09-10T16:08:25.878436+0000","last_fullsized":"2024-09-10T16:08:25.878436+0000","mapping_epoch":34,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1995235,"num_objects":488,"num_object_clones":0,"num_object_copies":976,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":488,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17115,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20579","reported_seq":20727,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:25.837253+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:08:25.837253+0000","last_peered":"2024-09-10T16:08:25.837253+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:08:25.837253+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1974755,"num_objects":483,"num_object_clones":0,"num_object_copies":966,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":483,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17191,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3888},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'20599","reported_seq":20746,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:25.860026+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:08:25.860026+0000","last_peered":"2024-09-10T16:08:25.860026+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:08:25.860026+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1979334,"num_objects":485,"num_object_clones":0,"num_object_copies":970,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":485,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17098,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3752},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20837","reported_seq":20977,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:25.824261+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:08:25.824261+0000","last_peered":"2024-09-10T16:08:25.824261+0000","last_clean":"2024-09-10T16:08:25.824261+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:08:25.824261+0000","last_undegraded":"2024-09-10T16:08:25.824261+0000","last_fullsized":"2024-09-10T16:08:25.824261+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1953792,"num_objects":477,"num_object_clones":0,"num_object_copies":954,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":477,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17684,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.838857+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:08:10.838857+0000","last_peered":"2024-09-10T16:08:10.838857+0000","last_clean":"2024-09-10T16:08:10.838857+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:08:10.838857+0000","last_undegraded":"2024-09-10T16:08:10.838857+0000","last_fullsized":"2024-09-10T16:08:10.838857+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19938","reported_seq":20088,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:26.067262+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:08:26.067262+0000","last_peered":"2024-09-10T16:08:26.067262+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:08:26.067262+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2126307,"num_objects":520,"num_object_clones":0,"num_object_copies":1040,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":520,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17757,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-3603}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":104,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.838197+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:08:10.838197+0000","last_peered":"2024-09-10T16:08:10.838197+0000","last_clean":"2024-09-10T16:08:10.838197+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:08:10.838197+0000","last_undegraded":"2024-09-10T16:08:10.838197+0000","last_fullsized":"2024-09-10T16:08:10.838197+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20436","reported_seq":21112,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:25.863377+0000","last_change":"2024-09-10T16:08:09.280891+0000","last_active":"2024-09-10T16:08:25.863377+0000","last_peered":"2024-09-10T16:08:25.863377+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:08:25.863377+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1941504,"num_objects":474,"num_object_clones":0,"num_object_copies":948,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3990,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":474,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16940,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":267,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3516}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":114,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.840262+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:08:10.840262+0000","last_peered":"2024-09-10T16:08:10.840262+0000","last_clean":"2024-09-10T16:08:10.840262+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:08:10.840262+0000","last_undegraded":"2024-09-10T16:08:10.840262+0000","last_fullsized":"2024-09-10T16:08:10.840262+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20281","reported_seq":20426,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:26.031921+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:08:26.031921+0000","last_peered":"2024-09-10T16:08:26.031921+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:08:26.031921+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2048483,"num_objects":501,"num_object_clones":0,"num_object_copies":1002,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":501,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17961,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-3803}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":90,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.837937+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:08:10.837937+0000","last_peered":"2024-09-10T16:08:10.837937+0000","last_clean":"2024-09-10T16:08:10.837937+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:08:10.837937+0000","last_undegraded":"2024-09-10T16:08:10.837937+0000","last_fullsized":"2024-09-10T16:08:10.837937+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'20688","reported_seq":29543,"reported_epoch":49,"state":"active+recovery_wait","last_fresh":"2024-09-10T16:08:26.055161+0000","last_change":"2024-09-10T16:08:10.929169+0000","last_active":"2024-09-10T16:08:26.055161+0000","last_peered":"2024-09-10T16:08:26.055161+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T16:08:10.860438+0000","last_became_peered":"2024-09-10T16:08:10.860438+0000","last_unstale":"2024-09-10T16:08:26.055161+0000","last_undegraded":"2024-09-10T16:08:26.055161+0000","last_fullsized":"2024-09-10T16:08:26.055161+0000","mapping_epoch":48,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1974272,"num_objects":482,"num_object_clones":0,"num_object_copies":964,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":482,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18242,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","0","1"],"object_location_counts":[{"shards":"1,2","objects":482}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20342","reported_seq":20487,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:25.853483+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:08:25.853483+0000","last_peered":"2024-09-10T16:08:25.853483+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:08:25.853483+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2031616,"num_objects":496,"num_object_clones":0,"num_object_copies":992,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":496,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16973,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3808}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.838546+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:08:10.838546+0000","last_peered":"2024-09-10T16:08:10.838546+0000","last_clean":"2024-09-10T16:08:10.838546+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:08:10.838546+0000","last_undegraded":"2024-09-10T16:08:10.838546+0000","last_fullsized":"2024-09-10T16:08:10.838546+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20083","reported_seq":28659,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:24.782476+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:08:24.782476+0000","last_peered":"2024-09-10T16:08:24.782476+0000","last_clean":"2024-09-10T16:08:24.782476+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:08:24.782476+0000","last_undegraded":"2024-09-10T16:08:24.782476+0000","last_fullsized":"2024-09-10T16:08:24.782476+0000","mapping_epoch":46,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2109440,"num_objects":515,"num_object_clones":0,"num_object_copies":1030,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":515,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16473,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":104,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.838936+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:08:10.838936+0000","last_peered":"2024-09-10T16:08:10.838936+0000","last_clean":"2024-09-10T16:08:10.838936+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:08:10.838936+0000","last_undegraded":"2024-09-10T16:08:10.838936+0000","last_fullsized":"2024-09-10T16:08:10.838936+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20551","reported_seq":20703,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:25.812765+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:08:25.812765+0000","last_peered":"2024-09-10T16:08:25.812765+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:08:25.812765+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2035712,"num_objects":497,"num_object_clones":0,"num_object_copies":994,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":497,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16806,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3874}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.839114+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:08:10.839114+0000","last_peered":"2024-09-10T16:08:10.839114+0000","last_clean":"2024-09-10T16:08:10.839114+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:08:10.839114+0000","last_undegraded":"2024-09-10T16:08:10.839114+0000","last_fullsized":"2024-09-10T16:08:10.839114+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":85,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.838633+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:08:10.838633+0000","last_peered":"2024-09-10T16:08:10.838633+0000","last_clean":"2024-09-10T16:08:10.838633+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:08:10.838633+0000","last_undegraded":"2024-09-10T16:08:10.838633+0000","last_fullsized":"2024-09-10T16:08:10.838633+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'20698","reported_seq":20851,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:25.754988+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:08:25.754988+0000","last_peered":"2024-09-10T16:08:25.754988+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:08:25.754988+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2048966,"num_objects":502,"num_object_clones":0,"num_object_copies":1004,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":502,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16861,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3821},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":32240374,"num_objects":7881,"num_object_clones":0,"num_object_copies":15762,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38290,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7881,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":284225,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13100,"num_bytes_recovered":33095680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":275988480,"data_stored":275853881,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161051,"ondisk_log_size":161051,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986301,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518988,"kb_used_data":154108,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93852852,"statfs":{"total":96636764160,"available":96105320448,"internally_reserved":0,"allocated":157806592,"data_stored":157493407,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,0,0,2,0,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659246,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":481240,"kb_used_data":26116,"kb_used_omap":4,"kb_used_meta":455099,"kb_avail":93890600,"statfs":{"total":96636764160,"available":96143974400,"internally_reserved":0,"allocated":26742784,"data_stored":26407457,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":466022062},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,3,0,2,6,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182529,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":552680,"kb_used_data":92056,"kb_used_omap":4,"kb_used_meta":460603,"kb_avail":93819160,"statfs":{"total":96636764160,"available":96070819840,"internally_reserved":0,"allocated":94265344,"data_stored":93952154,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":471658154},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,3,2,2,2,6,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":93687808,"data_stored":93651678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":25694208,"data_stored":25635941,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":156606464,"data_stored":156566262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:08:33.759 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:08:34.317 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:08:34.318 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:08:34.338 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":381,"stamp":"2024-09-10T16:08:33.761897+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":22471433,"num_objects":5498,"num_object_clones":0,"num_object_copies":10996,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38213,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5498,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":286612,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13179,"num_bytes_recovered":33095699,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160739,"ondisk_log_size":160739,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1513124,"kb_used_data":253284,"kb_used_omap":12,"kb_used_meta":1259763,"kb_avail":281602396,"statfs":{"total":289910292480,"available":288360853504,"internally_reserved":0,"allocated":259362816,"data_stored":258401114,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1289997314},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,5,6,10,4,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19533824,"num_objects":-4769,"num_object_clones":0,"num_object_copies":-9538,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-148,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4769,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4769,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":148,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002923"},"pg_stats":[{"pgid":"2.f","version":"49'20340","reported_seq":20425,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:30.923829+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:08:30.923829+0000","last_peered":"2024-09-10T16:08:30.923829+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:08:30.923829+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1359872,"num_objects":332,"num_object_clones":0,"num_object_copies":664,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":332,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20340,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3927}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20490","reported_seq":20613,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:31.059552+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:08:31.059552+0000","last_peered":"2024-09-10T16:08:31.059552+0000","last_clean":"2024-09-10T16:08:31.059552+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:08:31.059552+0000","last_undegraded":"2024-09-10T16:08:31.059552+0000","last_fullsized":"2024-09-10T16:08:31.059552+0000","mapping_epoch":35,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1450950,"num_objects":356,"num_object_clones":0,"num_object_copies":712,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":356,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18595,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20438","reported_seq":29126,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:29.928454+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:08:29.928454+0000","last_peered":"2024-09-10T16:08:29.928454+0000","last_clean":"2024-09-10T16:08:29.928454+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:08:29.928454+0000","last_undegraded":"2024-09-10T16:08:29.928454+0000","last_fullsized":"2024-09-10T16:08:29.928454+0000","mapping_epoch":44,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1470488,"num_objects":360,"num_object_clones":0,"num_object_copies":720,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":360,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18442,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20332","reported_seq":20423,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:31.061452+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:08:31.061452+0000","last_peered":"2024-09-10T16:08:31.061452+0000","last_clean":"2024-09-10T16:08:31.061452+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:08:31.061452+0000","last_undegraded":"2024-09-10T16:08:31.061452+0000","last_fullsized":"2024-09-10T16:08:31.061452+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1343488,"num_objects":328,"num_object_clones":0,"num_object_copies":656,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":328,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20332,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20534","reported_seq":20674,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:30.840521+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:08:30.840521+0000","last_peered":"2024-09-10T16:08:30.840521+0000","last_clean":"2024-09-10T16:08:30.840521+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:08:30.840521+0000","last_undegraded":"2024-09-10T16:08:30.840521+0000","last_fullsized":"2024-09-10T16:08:30.840521+0000","mapping_epoch":34,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1462755,"num_objects":358,"num_object_clones":0,"num_object_copies":716,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":358,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17245,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20716","reported_seq":20864,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:30.891501+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:08:30.891501+0000","last_peered":"2024-09-10T16:08:30.891501+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:08:30.891501+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1413603,"num_objects":346,"num_object_clones":0,"num_object_copies":692,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":346,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17328,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4025},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'20759","reported_seq":20907,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:30.829395+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:08:30.829395+0000","last_peered":"2024-09-10T16:08:30.829395+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:08:30.829395+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1323974,"num_objects":325,"num_object_clones":0,"num_object_copies":650,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":325,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17258,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3912},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20993","reported_seq":21133,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:30.735474+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:08:30.735474+0000","last_peered":"2024-09-10T16:08:30.735474+0000","last_clean":"2024-09-10T16:08:30.735474+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:08:30.735474+0000","last_undegraded":"2024-09-10T16:08:30.735474+0000","last_fullsized":"2024-09-10T16:08:30.735474+0000","mapping_epoch":34,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1314816,"num_objects":321,"num_object_clones":0,"num_object_copies":642,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":321,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17840,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.838857+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:08:10.838857+0000","last_peered":"2024-09-10T16:08:10.838857+0000","last_clean":"2024-09-10T16:08:10.838857+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:08:10.838857+0000","last_undegraded":"2024-09-10T16:08:10.838857+0000","last_fullsized":"2024-09-10T16:08:10.838857+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20096","reported_seq":20246,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:31.013377+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:08:31.013377+0000","last_peered":"2024-09-10T16:08:31.013377+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:08:31.013377+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1479139,"num_objects":362,"num_object_clones":0,"num_object_copies":724,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":362,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17915,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-3761}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":104,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.838197+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:08:10.838197+0000","last_peered":"2024-09-10T16:08:10.838197+0000","last_clean":"2024-09-10T16:08:10.838197+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:08:10.838197+0000","last_undegraded":"2024-09-10T16:08:10.838197+0000","last_fullsized":"2024-09-10T16:08:10.838197+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20564","reported_seq":21394,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:30.900290+0000","last_change":"2024-09-10T16:08:09.280891+0000","last_active":"2024-09-10T16:08:30.900290+0000","last_peered":"2024-09-10T16:08:30.900290+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:08:30.900290+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1417216,"num_objects":346,"num_object_clones":0,"num_object_copies":692,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3913,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":346,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17068,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":344,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3567}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":115,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.840312+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:08:10.840312+0000","last_peered":"2024-09-10T16:08:10.840312+0000","last_clean":"2024-09-10T16:08:10.840312+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:08:10.840312+0000","last_undegraded":"2024-09-10T16:08:10.840312+0000","last_fullsized":"2024-09-10T16:08:10.840312+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20435","reported_seq":20579,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:31.068056+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:08:31.068056+0000","last_peered":"2024-09-10T16:08:31.068056+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:08:31.068056+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1417699,"num_objects":347,"num_object_clones":0,"num_object_copies":694,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":347,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18115,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-3957}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":90,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.837937+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:08:10.837937+0000","last_peered":"2024-09-10T16:08:10.837937+0000","last_clean":"2024-09-10T16:08:10.837937+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:08:10.837937+0000","last_undegraded":"2024-09-10T16:08:10.837937+0000","last_fullsized":"2024-09-10T16:08:10.837937+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'20826","reported_seq":29681,"reported_epoch":49,"state":"active+recovery_wait","last_fresh":"2024-09-10T16:08:31.012194+0000","last_change":"2024-09-10T16:08:10.929169+0000","last_active":"2024-09-10T16:08:31.012194+0000","last_peered":"2024-09-10T16:08:31.012194+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T16:08:10.860438+0000","last_became_peered":"2024-09-10T16:08:10.860438+0000","last_unstale":"2024-09-10T16:08:31.012194+0000","last_undegraded":"2024-09-10T16:08:31.012194+0000","last_fullsized":"2024-09-10T16:08:31.012194+0000","mapping_epoch":48,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1409024,"num_objects":344,"num_object_clones":0,"num_object_copies":688,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":344,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18380,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","0","1"],"object_location_counts":[{"shards":"1,2","objects":344}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20502","reported_seq":20640,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:30.998672+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:08:30.998672+0000","last_peered":"2024-09-10T16:08:30.998672+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:08:30.998672+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1376256,"num_objects":336,"num_object_clones":0,"num_object_copies":672,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":336,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17133,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3968}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.838546+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:08:10.838546+0000","last_peered":"2024-09-10T16:08:10.838546+0000","last_clean":"2024-09-10T16:08:10.838546+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:08:10.838546+0000","last_undegraded":"2024-09-10T16:08:10.838546+0000","last_fullsized":"2024-09-10T16:08:10.838546+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20262","reported_seq":28838,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:29.825522+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:08:29.825522+0000","last_peered":"2024-09-10T16:08:29.825522+0000","last_clean":"2024-09-10T16:08:29.825522+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:08:29.825522+0000","last_undegraded":"2024-09-10T16:08:29.825522+0000","last_fullsized":"2024-09-10T16:08:29.825522+0000","mapping_epoch":46,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1376256,"num_objects":336,"num_object_clones":0,"num_object_copies":672,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":336,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16652,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":104,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.838936+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:08:10.838936+0000","last_peered":"2024-09-10T16:08:10.838936+0000","last_clean":"2024-09-10T16:08:10.838936+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:08:10.838936+0000","last_undegraded":"2024-09-10T16:08:10.838936+0000","last_fullsized":"2024-09-10T16:08:10.838936+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20705","reported_seq":20857,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:30.924657+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:08:30.924657+0000","last_peered":"2024-09-10T16:08:30.924657+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:08:30.924657+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1404928,"num_objects":343,"num_object_clones":0,"num_object_copies":686,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":343,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16960,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4028}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.839114+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:08:10.839114+0000","last_peered":"2024-09-10T16:08:10.839114+0000","last_clean":"2024-09-10T16:08:10.839114+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:08:10.839114+0000","last_undegraded":"2024-09-10T16:08:10.839114+0000","last_fullsized":"2024-09-10T16:08:10.839114+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":85,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.838633+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:08:10.838633+0000","last_peered":"2024-09-10T16:08:10.838633+0000","last_clean":"2024-09-10T16:08:10.838633+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:08:10.838633+0000","last_undegraded":"2024-09-10T16:08:10.838633+0000","last_fullsized":"2024-09-10T16:08:10.838633+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'20844","reported_seq":20999,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:30.899614+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:08:30.899614+0000","last_peered":"2024-09-10T16:08:30.899614+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:08:30.899614+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1450950,"num_objects":356,"num_object_clones":0,"num_object_copies":712,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":356,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17007,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3967},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":22471414,"num_objects":5496,"num_object_clones":0,"num_object_copies":10992,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38213,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5496,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":286610,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13177,"num_bytes_recovered":33095680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":256323584,"data_stored":256188985,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160736,"ondisk_log_size":160736,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986302,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":501776,"kb_used_data":149372,"kb_used_omap":4,"kb_used_meta":352379,"kb_avail":93870064,"statfs":{"total":96636764160,"available":96122945536,"internally_reserved":0,"allocated":152956928,"data_stored":152643743,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":360836778},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,1,0,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659247,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":461588,"kb_used_data":18936,"kb_used_omap":4,"kb_used_meta":442619,"kb_avail":93910252,"statfs":{"total":96636764160,"available":96164098048,"internally_reserved":0,"allocated":19390464,"data_stored":19055137,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":453242542},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,3,4,3,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182530,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":549760,"kb_used_data":84976,"kb_used_omap":4,"kb_used_meta":464763,"kb_avail":93822080,"statfs":{"total":96636764160,"available":96073809920,"internally_reserved":0,"allocated":87015424,"data_stored":86702234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":475917994},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,3,5,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":86302720,"data_stored":86266590,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":18210816,"data_stored":18152549,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":151810048,"data_stored":151769846,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:08:34.340 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:08:34.908 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:08:34.908 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:08:34.938 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":381,"stamp":"2024-09-10T16:08:33.761897+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":22471433,"num_objects":5498,"num_object_clones":0,"num_object_copies":10996,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38213,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5498,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":286612,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13179,"num_bytes_recovered":33095699,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160739,"ondisk_log_size":160739,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1513124,"kb_used_data":253284,"kb_used_omap":12,"kb_used_meta":1259763,"kb_avail":281602396,"statfs":{"total":289910292480,"available":288360853504,"internally_reserved":0,"allocated":259362816,"data_stored":258401114,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1289997314},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,5,6,10,4,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19533824,"num_objects":-4769,"num_object_clones":0,"num_object_copies":-9538,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-148,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4769,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4769,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":148,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002923"},"pg_stats":[{"pgid":"2.f","version":"49'20340","reported_seq":20425,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:30.923829+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:08:30.923829+0000","last_peered":"2024-09-10T16:08:30.923829+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:08:30.923829+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1359872,"num_objects":332,"num_object_clones":0,"num_object_copies":664,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":332,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20340,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3927}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20490","reported_seq":20613,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:31.059552+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:08:31.059552+0000","last_peered":"2024-09-10T16:08:31.059552+0000","last_clean":"2024-09-10T16:08:31.059552+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:08:31.059552+0000","last_undegraded":"2024-09-10T16:08:31.059552+0000","last_fullsized":"2024-09-10T16:08:31.059552+0000","mapping_epoch":35,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1450950,"num_objects":356,"num_object_clones":0,"num_object_copies":712,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":356,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18595,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20438","reported_seq":29126,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:29.928454+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:08:29.928454+0000","last_peered":"2024-09-10T16:08:29.928454+0000","last_clean":"2024-09-10T16:08:29.928454+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:08:29.928454+0000","last_undegraded":"2024-09-10T16:08:29.928454+0000","last_fullsized":"2024-09-10T16:08:29.928454+0000","mapping_epoch":44,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1470488,"num_objects":360,"num_object_clones":0,"num_object_copies":720,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":360,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18442,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20332","reported_seq":20423,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:31.061452+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:08:31.061452+0000","last_peered":"2024-09-10T16:08:31.061452+0000","last_clean":"2024-09-10T16:08:31.061452+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:08:31.061452+0000","last_undegraded":"2024-09-10T16:08:31.061452+0000","last_fullsized":"2024-09-10T16:08:31.061452+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1343488,"num_objects":328,"num_object_clones":0,"num_object_copies":656,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":328,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20332,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20534","reported_seq":20674,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:30.840521+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:08:30.840521+0000","last_peered":"2024-09-10T16:08:30.840521+0000","last_clean":"2024-09-10T16:08:30.840521+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:08:30.840521+0000","last_undegraded":"2024-09-10T16:08:30.840521+0000","last_fullsized":"2024-09-10T16:08:30.840521+0000","mapping_epoch":34,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1462755,"num_objects":358,"num_object_clones":0,"num_object_copies":716,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":358,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17245,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20716","reported_seq":20864,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:30.891501+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:08:30.891501+0000","last_peered":"2024-09-10T16:08:30.891501+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:08:30.891501+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1413603,"num_objects":346,"num_object_clones":0,"num_object_copies":692,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":346,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17328,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4025},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'20759","reported_seq":20907,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:30.829395+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:08:30.829395+0000","last_peered":"2024-09-10T16:08:30.829395+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:08:30.829395+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1323974,"num_objects":325,"num_object_clones":0,"num_object_copies":650,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":325,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17258,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3912},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20993","reported_seq":21133,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:30.735474+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:08:30.735474+0000","last_peered":"2024-09-10T16:08:30.735474+0000","last_clean":"2024-09-10T16:08:30.735474+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:08:30.735474+0000","last_undegraded":"2024-09-10T16:08:30.735474+0000","last_fullsized":"2024-09-10T16:08:30.735474+0000","mapping_epoch":34,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1314816,"num_objects":321,"num_object_clones":0,"num_object_copies":642,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":321,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17840,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.838857+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:08:10.838857+0000","last_peered":"2024-09-10T16:08:10.838857+0000","last_clean":"2024-09-10T16:08:10.838857+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:08:10.838857+0000","last_undegraded":"2024-09-10T16:08:10.838857+0000","last_fullsized":"2024-09-10T16:08:10.838857+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20096","reported_seq":20246,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:31.013377+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:08:31.013377+0000","last_peered":"2024-09-10T16:08:31.013377+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:08:31.013377+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1479139,"num_objects":362,"num_object_clones":0,"num_object_copies":724,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":362,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17915,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-3761}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":104,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.838197+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:08:10.838197+0000","last_peered":"2024-09-10T16:08:10.838197+0000","last_clean":"2024-09-10T16:08:10.838197+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:08:10.838197+0000","last_undegraded":"2024-09-10T16:08:10.838197+0000","last_fullsized":"2024-09-10T16:08:10.838197+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20564","reported_seq":21394,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:30.900290+0000","last_change":"2024-09-10T16:08:09.280891+0000","last_active":"2024-09-10T16:08:30.900290+0000","last_peered":"2024-09-10T16:08:30.900290+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:08:30.900290+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1417216,"num_objects":346,"num_object_clones":0,"num_object_copies":692,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3913,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":346,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17068,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":344,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3567}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":115,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.840312+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:08:10.840312+0000","last_peered":"2024-09-10T16:08:10.840312+0000","last_clean":"2024-09-10T16:08:10.840312+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:08:10.840312+0000","last_undegraded":"2024-09-10T16:08:10.840312+0000","last_fullsized":"2024-09-10T16:08:10.840312+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20435","reported_seq":20579,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:31.068056+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:08:31.068056+0000","last_peered":"2024-09-10T16:08:31.068056+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:08:31.068056+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1417699,"num_objects":347,"num_object_clones":0,"num_object_copies":694,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":347,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18115,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-3957}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":90,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.837937+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:08:10.837937+0000","last_peered":"2024-09-10T16:08:10.837937+0000","last_clean":"2024-09-10T16:08:10.837937+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:08:10.837937+0000","last_undegraded":"2024-09-10T16:08:10.837937+0000","last_fullsized":"2024-09-10T16:08:10.837937+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'20826","reported_seq":29681,"reported_epoch":49,"state":"active+recovery_wait","last_fresh":"2024-09-10T16:08:31.012194+0000","last_change":"2024-09-10T16:08:10.929169+0000","last_active":"2024-09-10T16:08:31.012194+0000","last_peered":"2024-09-10T16:08:31.012194+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T16:08:10.860438+0000","last_became_peered":"2024-09-10T16:08:10.860438+0000","last_unstale":"2024-09-10T16:08:31.012194+0000","last_undegraded":"2024-09-10T16:08:31.012194+0000","last_fullsized":"2024-09-10T16:08:31.012194+0000","mapping_epoch":48,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1409024,"num_objects":344,"num_object_clones":0,"num_object_copies":688,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":344,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18380,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","0","1"],"object_location_counts":[{"shards":"1,2","objects":344}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20502","reported_seq":20640,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:30.998672+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:08:30.998672+0000","last_peered":"2024-09-10T16:08:30.998672+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:08:30.998672+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1376256,"num_objects":336,"num_object_clones":0,"num_object_copies":672,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":336,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17133,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3968}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.838546+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:08:10.838546+0000","last_peered":"2024-09-10T16:08:10.838546+0000","last_clean":"2024-09-10T16:08:10.838546+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:08:10.838546+0000","last_undegraded":"2024-09-10T16:08:10.838546+0000","last_fullsized":"2024-09-10T16:08:10.838546+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20262","reported_seq":28838,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:29.825522+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:08:29.825522+0000","last_peered":"2024-09-10T16:08:29.825522+0000","last_clean":"2024-09-10T16:08:29.825522+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:08:29.825522+0000","last_undegraded":"2024-09-10T16:08:29.825522+0000","last_fullsized":"2024-09-10T16:08:29.825522+0000","mapping_epoch":46,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1376256,"num_objects":336,"num_object_clones":0,"num_object_copies":672,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":336,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16652,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":104,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.838936+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:08:10.838936+0000","last_peered":"2024-09-10T16:08:10.838936+0000","last_clean":"2024-09-10T16:08:10.838936+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:08:10.838936+0000","last_undegraded":"2024-09-10T16:08:10.838936+0000","last_fullsized":"2024-09-10T16:08:10.838936+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20705","reported_seq":20857,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:30.924657+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:08:30.924657+0000","last_peered":"2024-09-10T16:08:30.924657+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:08:30.924657+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1404928,"num_objects":343,"num_object_clones":0,"num_object_copies":686,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":343,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16960,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4028}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.839114+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:08:10.839114+0000","last_peered":"2024-09-10T16:08:10.839114+0000","last_clean":"2024-09-10T16:08:10.839114+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:08:10.839114+0000","last_undegraded":"2024-09-10T16:08:10.839114+0000","last_fullsized":"2024-09-10T16:08:10.839114+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":85,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.838633+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:08:10.838633+0000","last_peered":"2024-09-10T16:08:10.838633+0000","last_clean":"2024-09-10T16:08:10.838633+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:08:10.838633+0000","last_undegraded":"2024-09-10T16:08:10.838633+0000","last_fullsized":"2024-09-10T16:08:10.838633+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'20844","reported_seq":20999,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:30.899614+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:08:30.899614+0000","last_peered":"2024-09-10T16:08:30.899614+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:08:30.899614+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1450950,"num_objects":356,"num_object_clones":0,"num_object_copies":712,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":356,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17007,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3967},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":22471414,"num_objects":5496,"num_object_clones":0,"num_object_copies":10992,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38213,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5496,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":286610,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13177,"num_bytes_recovered":33095680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":256323584,"data_stored":256188985,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160736,"ondisk_log_size":160736,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986302,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":501776,"kb_used_data":149372,"kb_used_omap":4,"kb_used_meta":352379,"kb_avail":93870064,"statfs":{"total":96636764160,"available":96122945536,"internally_reserved":0,"allocated":152956928,"data_stored":152643743,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":360836778},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,1,0,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659247,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":461588,"kb_used_data":18936,"kb_used_omap":4,"kb_used_meta":442619,"kb_avail":93910252,"statfs":{"total":96636764160,"available":96164098048,"internally_reserved":0,"allocated":19390464,"data_stored":19055137,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":453242542},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,3,4,3,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182530,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":549760,"kb_used_data":84976,"kb_used_omap":4,"kb_used_meta":464763,"kb_avail":93822080,"statfs":{"total":96636764160,"available":96073809920,"internally_reserved":0,"allocated":87015424,"data_stored":86702234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":475917994},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,3,5,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":86302720,"data_stored":86266590,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":18210816,"data_stored":18152549,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":151810048,"data_stored":151769846,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:08:34.940 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-10T16:08:34.940 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "49'20340", 'reported_seq': 20425, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:08:30.923829+0000', 'last_change': '2024-09-10T15:59:16.281558+0000', 'last_active': '2024-09-10T16:08:30.923829+0000', 'last_peered': '2024-09-10T16:08:30.923829+0000', 'last_clean': '2024-09-10T15:59:14.132456+0000', 'last_became_active': '2024-09-10T15:59:16.274407+0000', 'last_became_peered': '2024-09-10T15:59:16.274407+0000', 'last_unstale': '2024-09-10T16:08:30.923829+0000', 'last_undegraded': '2024-09-10T15:59:15.851279+0000', 'last_fullsized': '2024-09-10T15:59:15.774356+0000', 'mapping_epoch': 42, 'log_start': "45'10300", 'ondisk_log_start': "45'10300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'objects_scrubbed': 0, 'log_size': 10040, 'log_dups_size': 0, 'ondisk_log_size': 10040, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 1359872, 'num_objects': 332, 'num_object_clones': 0, 'num_object_copies': 664, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4259, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 332, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20340, 'num_write_kb': 41344, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3927}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:08:34.940 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-10T16:08:34.940 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "49'20716", 'reported_seq': 20864, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:08:30.891501+0000', 'last_change': '2024-09-10T15:59:16.308264+0000', 'last_active': '2024-09-10T16:08:30.891501+0000', 'last_peered': '2024-09-10T16:08:30.891501+0000', 'last_clean': '2024-09-10T15:59:14.111551+0000', 'last_became_active': '2024-09-10T15:59:16.300040+0000', 'last_became_peered': '2024-09-10T15:59:16.300040+0000', 'last_unstale': '2024-09-10T16:08:30.891501+0000', 'last_undegraded': '2024-09-10T15:59:15.878234+0000', 'last_fullsized': '2024-09-10T15:59:15.769328+0000', 'mapping_epoch': 42, 'log_start': "45'10700", 'ondisk_log_start': "45'10700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3442", 'last_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'objects_scrubbed': 3384, 'log_size': 10016, 'log_dups_size': 0, 'ondisk_log_size': 10016, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1292, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 1413603, 'num_objects': 346, 'num_object_clones': 0, 'num_object_copies': 692, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4372, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 346, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 17328, 'num_write_kb': 28580, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4025}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:08:34.940 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-10T16:08:34.940 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "49'20759", 'reported_seq': 20907, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:08:30.829395+0000', 'last_change': '2024-09-10T15:59:16.331899+0000', 'last_active': '2024-09-10T16:08:30.829395+0000', 'last_peered': '2024-09-10T16:08:30.829395+0000', 'last_clean': '2024-09-10T15:59:14.139436+0000', 'last_became_active': '2024-09-10T15:59:16.318715+0000', 'last_became_peered': '2024-09-10T15:59:16.318715+0000', 'last_unstale': '2024-09-10T16:08:30.829395+0000', 'last_undegraded': '2024-09-10T15:59:15.877524+0000', 'last_fullsized': '2024-09-10T15:59:15.769042+0000', 'mapping_epoch': 42, 'log_start': "45'10700", 'ondisk_log_start': "45'10700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3537", 'last_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'objects_scrubbed': 3497, 'log_size': 10059, 'log_dups_size': 0, 'ondisk_log_size': 10059, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1224, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 1323974, 'num_objects': 325, 'num_object_clones': 0, 'num_object_copies': 650, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4237, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 325, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 17258, 'num_write_kb': 28172, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3912}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:08:34.940 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-10T16:08:34.940 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "49'20096", 'reported_seq': 20246, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:08:31.013377+0000', 'last_change': '2024-09-10T15:59:16.175680+0000', 'last_active': '2024-09-10T16:08:31.013377+0000', 'last_peered': '2024-09-10T16:08:31.013377+0000', 'last_clean': '2024-09-10T15:59:14.088187+0000', 'last_became_active': '2024-09-10T15:59:16.165810+0000', 'last_became_peered': '2024-09-10T15:59:16.165810+0000', 'last_unstale': '2024-09-10T16:08:31.013377+0000', 'last_undegraded': '2024-09-10T15:59:15.822098+0000', 'last_fullsized': '2024-09-10T15:59:15.769123+0000', 'mapping_epoch': 42, 'log_start': "45'10000", 'ondisk_log_start': "45'10000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2218", 'last_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'objects_scrubbed': 2175, 'log_size': 10096, 'log_dups_size': 0, 'ondisk_log_size': 10096, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 825, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 1479139, 'num_objects': 362, 'num_object_clones': 0, 'num_object_copies': 724, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4124, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 362, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 17915, 'num_write_kb': 32204, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': -3761}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:08:34.941 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-10T16:08:34.941 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "49'20564", 'reported_seq': 21394, 'reported_epoch': 49, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:08:30.900290+0000', 'last_change': '2024-09-10T16:08:09.280891+0000', 'last_active': '2024-09-10T16:08:30.900290+0000', 'last_peered': '2024-09-10T16:08:30.900290+0000', 'last_clean': '2024-09-10T15:59:14.129081+0000', 'last_became_active': '2024-09-10T15:59:16.269835+0000', 'last_became_peered': '2024-09-10T15:59:16.269835+0000', 'last_unstale': '2024-09-10T16:08:30.900290+0000', 'last_undegraded': '2024-09-10T15:59:15.860857+0000', 'last_fullsized': '2024-09-10T15:59:15.771410+0000', 'mapping_epoch': 42, 'log_start': "45'10500", 'ondisk_log_start': "45'10500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3557", 'last_scrub_stamp': '2024-09-10T15:59:02.493678+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:02.493678+0000', 'objects_scrubbed': 3488, 'log_size': 10064, 'log_dups_size': 0, 'ondisk_log_size': 10064, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1472, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 1417216, 'num_objects': 346, 'num_object_clones': 0, 'num_object_copies': 692, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3913, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 346, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17068, 'num_write_kb': 27852, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 344, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3567}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:08:34.941 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-10T16:08:34.941 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "49'20435", 'reported_seq': 20579, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:08:31.068056+0000', 'last_change': '2024-09-10T15:59:16.284311+0000', 'last_active': '2024-09-10T16:08:31.068056+0000', 'last_peered': '2024-09-10T16:08:31.068056+0000', 'last_clean': '2024-09-10T15:59:14.116041+0000', 'last_became_active': '2024-09-10T15:59:16.274543+0000', 'last_became_peered': '2024-09-10T15:59:16.274543+0000', 'last_unstale': '2024-09-10T16:08:31.068056+0000', 'last_undegraded': '2024-09-10T15:59:15.812484+0000', 'last_fullsized': '2024-09-10T15:59:15.770885+0000', 'mapping_epoch': 42, 'log_start': "45'10400", 'ondisk_log_start': "45'10400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2356", 'last_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'objects_scrubbed': 2316, 'log_size': 10035, 'log_dups_size': 0, 'ondisk_log_size': 10035, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 783, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 1417699, 'num_objects': 347, 'num_object_clones': 0, 'num_object_copies': 694, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 347, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 18115, 'num_write_kb': 32292, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': -3957}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:08:34.941 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-10T16:08:34.941 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "49'20826", 'reported_seq': 29681, 'reported_epoch': 49, 'state': 'active+recovery_wait', 'last_fresh': '2024-09-10T16:08:31.012194+0000', 'last_change': '2024-09-10T16:08:10.929169+0000', 'last_active': '2024-09-10T16:08:31.012194+0000', 'last_peered': '2024-09-10T16:08:31.012194+0000', 'last_clean': '2024-09-10T15:59:14.110654+0000', 'last_became_active': '2024-09-10T16:08:10.860438+0000', 'last_became_peered': '2024-09-10T16:08:10.860438+0000', 'last_unstale': '2024-09-10T16:08:31.012194+0000', 'last_undegraded': '2024-09-10T16:08:31.012194+0000', 'last_fullsized': '2024-09-10T16:08:31.012194+0000', 'mapping_epoch': 48, 'log_start': "45'10800", 'ondisk_log_start': "45'10800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2486", 'last_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'objects_scrubbed': 2440, 'log_size': 10026, 'log_dups_size': 0, 'ondisk_log_size': 10026, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 792, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 1409024, 'num_objects': 344, 'num_object_clones': 0, 'num_object_copies': 688, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 344, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18380, 'num_write_kb': 32568, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 4355, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': ['2', '0', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 344}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:08:34.941 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-10T16:08:34.941 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "49'20502", 'reported_seq': 20640, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:08:30.998672+0000', 'last_change': '2024-09-10T15:59:16.315641+0000', 'last_active': '2024-09-10T16:08:30.998672+0000', 'last_peered': '2024-09-10T16:08:30.998672+0000', 'last_clean': '2024-09-10T15:59:14.117001+0000', 'last_became_active': '2024-09-10T15:59:16.302245+0000', 'last_became_peered': '2024-09-10T15:59:16.302245+0000', 'last_unstale': '2024-09-10T16:08:30.998672+0000', 'last_undegraded': '2024-09-10T15:59:15.845012+0000', 'last_fullsized': '2024-09-10T15:59:15.770337+0000', 'mapping_epoch': 42, 'log_start': "45'10500", 'ondisk_log_start': "45'10500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3404", 'last_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'objects_scrubbed': 3367, 'log_size': 10002, 'log_dups_size': 0, 'ondisk_log_size': 10002, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 983, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 1376256, 'num_objects': 336, 'num_object_clones': 0, 'num_object_copies': 672, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 336, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17133, 'num_write_kb': 28204, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3968}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:08:34.941 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-10T16:08:34.941 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "49'20705", 'reported_seq': 20857, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:08:30.924657+0000', 'last_change': '2024-09-10T15:59:16.311291+0000', 'last_active': '2024-09-10T16:08:30.924657+0000', 'last_peered': '2024-09-10T16:08:30.924657+0000', 'last_clean': '2024-09-10T15:59:14.116363+0000', 'last_became_active': '2024-09-10T15:59:16.301244+0000', 'last_became_peered': '2024-09-10T15:59:16.301244+0000', 'last_unstale': '2024-09-10T16:08:30.924657+0000', 'last_undegraded': '2024-09-10T15:59:15.883434+0000', 'last_fullsized': '2024-09-10T15:59:15.772756+0000', 'mapping_epoch': 42, 'log_start': "45'10700", 'ondisk_log_start': "45'10700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3789", 'last_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'objects_scrubbed': 3741, 'log_size': 10005, 'log_dups_size': 0, 'ondisk_log_size': 10005, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1440, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 1404928, 'num_objects': 343, 'num_object_clones': 0, 'num_object_copies': 686, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4373, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 343, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16960, 'num_write_kb': 27124, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4028}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:08:34.942 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-10T16:08:34.942 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "49'20844", 'reported_seq': 20999, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:08:30.899614+0000', 'last_change': '2024-09-10T15:59:16.313409+0000', 'last_active': '2024-09-10T16:08:30.899614+0000', 'last_peered': '2024-09-10T16:08:30.899614+0000', 'last_clean': '2024-09-10T15:59:14.129780+0000', 'last_became_active': '2024-09-10T15:59:16.300932+0000', 'last_became_peered': '2024-09-10T15:59:16.300932+0000', 'last_unstale': '2024-09-10T16:08:30.899614+0000', 'last_undegraded': '2024-09-10T15:59:15.887823+0000', 'last_fullsized': '2024-09-10T15:59:15.768513+0000', 'mapping_epoch': 42, 'log_start': "45'10800", 'ondisk_log_start': "45'10800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3880", 'last_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'objects_scrubbed': 3827, 'log_size': 10044, 'log_dups_size': 0, 'ondisk_log_size': 10044, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1397, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 1450950, 'num_objects': 356, 'num_object_clones': 0, 'num_object_copies': 712, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4325, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 356, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17007, 'num_write_kb': 27072, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3967}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:08:34.942 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T16:08:35.625 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:08:35.626 INFO:teuthology.orchestra.run.smithi115.stdout:{"fsid":"7effee29-df05-4351-9741-ee203564ed81","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 38213/10996 objects degraded (347.517%), 9 pgs degraded, 9 pgs undersized","count":18},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":679,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":49,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983954,"num_in_osds":3,"osd_in_since":1725983947,"num_remapped_pgs":9},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":8},{"state_name":"active+recovery_wait","count":1},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":5498,"data_bytes":22471433,"bytes_used":1549438976,"bytes_avail":288360853504,"bytes_total":289910292480,"degraded_objects":38213,"degraded_total":10996,"degraded_ratio":3.4751727901054927,"recovering_objects_per_sec":12,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":148,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":397},"fsmap":{"epoch":1,"btime":"2024-09-10T15:57:14:967120+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T16:00:13.560776+0000","services":{}},"progress_events":{}} 2024-09-10T16:08:35.655 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-10T16:08:35.655 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:08:36.188 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:08:36.189 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:08:36.209 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":382,"stamp":"2024-09-10T16:08:35.762460+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":21308169,"num_objects":5214,"num_object_clones":0,"num_object_copies":10428,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38213,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5214,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":286896,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13179,"num_bytes_recovered":33095699,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160723,"ondisk_log_size":160723,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1510264,"kb_used_data":246276,"kb_used_omap":12,"kb_used_meta":1263923,"kb_avail":281605256,"statfs":{"total":289910292480,"available":288363782144,"internally_reserved":0,"allocated":252186624,"data_stored":251224922,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1294257154},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,4,5,13,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-20697088,"num_objects":-5053,"num_object_clones":0,"num_object_copies":-10106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-148,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5053,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5053,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":148,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002994"},"pg_stats":[{"pgid":"2.f","version":"49'20340","reported_seq":20425,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:30.923829+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:08:30.923829+0000","last_peered":"2024-09-10T16:08:30.923829+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:08:30.923829+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1359872,"num_objects":332,"num_object_clones":0,"num_object_copies":664,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":332,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20340,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3927}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20490","reported_seq":20613,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:31.059552+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:08:31.059552+0000","last_peered":"2024-09-10T16:08:31.059552+0000","last_clean":"2024-09-10T16:08:31.059552+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:08:31.059552+0000","last_undegraded":"2024-09-10T16:08:31.059552+0000","last_fullsized":"2024-09-10T16:08:31.059552+0000","mapping_epoch":35,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1450950,"num_objects":356,"num_object_clones":0,"num_object_copies":712,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":356,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18595,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20579","reported_seq":29267,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:34.868481+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:08:34.868481+0000","last_peered":"2024-09-10T16:08:34.868481+0000","last_clean":"2024-09-10T16:08:34.868481+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:08:34.868481+0000","last_undegraded":"2024-09-10T16:08:34.868481+0000","last_fullsized":"2024-09-10T16:08:34.868481+0000","mapping_epoch":44,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":892952,"num_objects":219,"num_object_clones":0,"num_object_copies":438,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":219,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18583,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20332","reported_seq":20423,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:31.061452+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:08:31.061452+0000","last_peered":"2024-09-10T16:08:31.061452+0000","last_clean":"2024-09-10T16:08:31.061452+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:08:31.061452+0000","last_undegraded":"2024-09-10T16:08:31.061452+0000","last_fullsized":"2024-09-10T16:08:31.061452+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1343488,"num_objects":328,"num_object_clones":0,"num_object_copies":656,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":328,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20332,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20534","reported_seq":20674,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:30.840521+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:08:30.840521+0000","last_peered":"2024-09-10T16:08:30.840521+0000","last_clean":"2024-09-10T16:08:30.840521+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:08:30.840521+0000","last_undegraded":"2024-09-10T16:08:30.840521+0000","last_fullsized":"2024-09-10T16:08:30.840521+0000","mapping_epoch":34,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1462755,"num_objects":358,"num_object_clones":0,"num_object_copies":716,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":358,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17245,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20716","reported_seq":20864,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:30.891501+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:08:30.891501+0000","last_peered":"2024-09-10T16:08:30.891501+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:08:30.891501+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1413603,"num_objects":346,"num_object_clones":0,"num_object_copies":692,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":346,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17328,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4025},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'20759","reported_seq":20907,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:30.829395+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:08:30.829395+0000","last_peered":"2024-09-10T16:08:30.829395+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:08:30.829395+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1323974,"num_objects":325,"num_object_clones":0,"num_object_copies":650,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":325,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17258,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3912},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20993","reported_seq":21133,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:30.735474+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:08:30.735474+0000","last_peered":"2024-09-10T16:08:30.735474+0000","last_clean":"2024-09-10T16:08:30.735474+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:08:30.735474+0000","last_undegraded":"2024-09-10T16:08:30.735474+0000","last_fullsized":"2024-09-10T16:08:30.735474+0000","mapping_epoch":34,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1314816,"num_objects":321,"num_object_clones":0,"num_object_copies":642,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":321,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17840,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.838857+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:08:10.838857+0000","last_peered":"2024-09-10T16:08:10.838857+0000","last_clean":"2024-09-10T16:08:10.838857+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:08:10.838857+0000","last_undegraded":"2024-09-10T16:08:10.838857+0000","last_fullsized":"2024-09-10T16:08:10.838857+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20096","reported_seq":20246,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:31.013377+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:08:31.013377+0000","last_peered":"2024-09-10T16:08:31.013377+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:08:31.013377+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1479139,"num_objects":362,"num_object_clones":0,"num_object_copies":724,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":362,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17915,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-3761}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":104,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.838197+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:08:10.838197+0000","last_peered":"2024-09-10T16:08:10.838197+0000","last_clean":"2024-09-10T16:08:10.838197+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:08:10.838197+0000","last_undegraded":"2024-09-10T16:08:10.838197+0000","last_fullsized":"2024-09-10T16:08:10.838197+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20564","reported_seq":21394,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:30.900290+0000","last_change":"2024-09-10T16:08:09.280891+0000","last_active":"2024-09-10T16:08:30.900290+0000","last_peered":"2024-09-10T16:08:30.900290+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:08:30.900290+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1417216,"num_objects":346,"num_object_clones":0,"num_object_copies":692,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3913,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":346,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17068,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":344,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3567}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":115,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.840312+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:08:10.840312+0000","last_peered":"2024-09-10T16:08:10.840312+0000","last_clean":"2024-09-10T16:08:10.840312+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:08:10.840312+0000","last_undegraded":"2024-09-10T16:08:10.840312+0000","last_fullsized":"2024-09-10T16:08:10.840312+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20435","reported_seq":20579,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:31.068056+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:08:31.068056+0000","last_peered":"2024-09-10T16:08:31.068056+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:08:31.068056+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1417699,"num_objects":347,"num_object_clones":0,"num_object_copies":694,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":347,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18115,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-3957}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":90,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.837937+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:08:10.837937+0000","last_peered":"2024-09-10T16:08:10.837937+0000","last_clean":"2024-09-10T16:08:10.837937+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:08:10.837937+0000","last_undegraded":"2024-09-10T16:08:10.837937+0000","last_fullsized":"2024-09-10T16:08:10.837937+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'20826","reported_seq":29681,"reported_epoch":49,"state":"active+recovery_wait","last_fresh":"2024-09-10T16:08:31.012194+0000","last_change":"2024-09-10T16:08:10.929169+0000","last_active":"2024-09-10T16:08:31.012194+0000","last_peered":"2024-09-10T16:08:31.012194+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T16:08:10.860438+0000","last_became_peered":"2024-09-10T16:08:10.860438+0000","last_unstale":"2024-09-10T16:08:31.012194+0000","last_undegraded":"2024-09-10T16:08:31.012194+0000","last_fullsized":"2024-09-10T16:08:31.012194+0000","mapping_epoch":48,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1409024,"num_objects":344,"num_object_clones":0,"num_object_copies":688,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":344,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18380,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","0","1"],"object_location_counts":[{"shards":"1,2","objects":344}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20502","reported_seq":20640,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:30.998672+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:08:30.998672+0000","last_peered":"2024-09-10T16:08:30.998672+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:08:30.998672+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1376256,"num_objects":336,"num_object_clones":0,"num_object_copies":672,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":336,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17133,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3968}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.838546+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:08:10.838546+0000","last_peered":"2024-09-10T16:08:10.838546+0000","last_clean":"2024-09-10T16:08:10.838546+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:08:10.838546+0000","last_undegraded":"2024-09-10T16:08:10.838546+0000","last_fullsized":"2024-09-10T16:08:10.838546+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20405","reported_seq":28981,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:34.911340+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:08:34.911340+0000","last_peered":"2024-09-10T16:08:34.911340+0000","last_clean":"2024-09-10T16:08:34.911340+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:08:34.911340+0000","last_undegraded":"2024-09-10T16:08:34.911340+0000","last_fullsized":"2024-09-10T16:08:34.911340+0000","mapping_epoch":46,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":790528,"num_objects":193,"num_object_clones":0,"num_object_copies":386,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":193,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16795,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":104,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.838936+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:08:10.838936+0000","last_peered":"2024-09-10T16:08:10.838936+0000","last_clean":"2024-09-10T16:08:10.838936+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:08:10.838936+0000","last_undegraded":"2024-09-10T16:08:10.838936+0000","last_fullsized":"2024-09-10T16:08:10.838936+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20705","reported_seq":20857,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:30.924657+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:08:30.924657+0000","last_peered":"2024-09-10T16:08:30.924657+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:08:30.924657+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1404928,"num_objects":343,"num_object_clones":0,"num_object_copies":686,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":343,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16960,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4028}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.839114+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:08:10.839114+0000","last_peered":"2024-09-10T16:08:10.839114+0000","last_clean":"2024-09-10T16:08:10.839114+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:08:10.839114+0000","last_undegraded":"2024-09-10T16:08:10.839114+0000","last_fullsized":"2024-09-10T16:08:10.839114+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":85,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.838633+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:08:10.838633+0000","last_peered":"2024-09-10T16:08:10.838633+0000","last_clean":"2024-09-10T16:08:10.838633+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:08:10.838633+0000","last_undegraded":"2024-09-10T16:08:10.838633+0000","last_fullsized":"2024-09-10T16:08:10.838633+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'20844","reported_seq":20999,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:30.899614+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:08:30.899614+0000","last_peered":"2024-09-10T16:08:30.899614+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:08:30.899614+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1450950,"num_objects":356,"num_object_clones":0,"num_object_copies":712,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":356,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17007,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3967},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":21308150,"num_objects":5212,"num_object_clones":0,"num_object_copies":10424,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38213,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5212,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":286894,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13177,"num_bytes_recovered":33095680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":248954880,"data_stored":248820281,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160720,"ondisk_log_size":160720,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986302,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":501776,"kb_used_data":149372,"kb_used_omap":4,"kb_used_meta":352379,"kb_avail":93870064,"statfs":{"total":96636764160,"available":96122945536,"internally_reserved":0,"allocated":152956928,"data_stored":152643743,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":360836778},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,1,0,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659248,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":458728,"kb_used_data":11928,"kb_used_omap":4,"kb_used_meta":446779,"kb_avail":93913112,"statfs":{"total":96636764160,"available":96167026688,"internally_reserved":0,"allocated":12214272,"data_stored":11878945,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":457502382},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,2,7],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182530,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":549760,"kb_used_data":84976,"kb_used_omap":4,"kb_used_meta":464763,"kb_avail":93822080,"statfs":{"total":96636764160,"available":96073809920,"internally_reserved":0,"allocated":87015424,"data_stored":86702234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":475917994},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,3,5,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":86302720,"data_stored":86266590,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":10842112,"data_stored":10783845,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":151810048,"data_stored":151769846,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:08:39.211 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:08:39.773 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:08:39.774 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:08:39.805 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":384,"stamp":"2024-09-10T16:08:39.763963+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":12645129,"num_objects":3099,"num_object_clones":0,"num_object_copies":6198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38138,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3099,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":289011,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13254,"num_bytes_recovered":33095699,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160938,"ondisk_log_size":160938,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1511000,"kb_used_data":234520,"kb_used_omap":12,"kb_used_meta":1276403,"kb_avail":281604520,"statfs":{"total":289910292480,"available":288363028480,"internally_reserved":0,"allocated":240148480,"data_stored":239186778,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1307036674},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,0,1,1,8,13,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19595264,"num_objects":-4784,"num_object_clones":0,"num_object_copies":-9568,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-152,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4784,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4784,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":152,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003603"},"pg_stats":[{"pgid":"2.f","version":"49'20492","reported_seq":20577,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:36.006576+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:08:36.006576+0000","last_peered":"2024-09-10T16:08:36.006576+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:08:36.006576+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":737280,"num_objects":180,"num_object_clones":0,"num_object_copies":360,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":180,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20492,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4079}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20644","reported_seq":20767,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:36.023310+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:08:36.023310+0000","last_peered":"2024-09-10T16:08:36.023310+0000","last_clean":"2024-09-10T16:08:36.023310+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:08:36.023310+0000","last_undegraded":"2024-09-10T16:08:36.023310+0000","last_fullsized":"2024-09-10T16:08:36.023310+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":820166,"num_objects":202,"num_object_clones":0,"num_object_copies":404,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":202,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18749,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20579","reported_seq":29267,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:34.868481+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:08:34.868481+0000","last_peered":"2024-09-10T16:08:34.868481+0000","last_clean":"2024-09-10T16:08:34.868481+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:08:34.868481+0000","last_undegraded":"2024-09-10T16:08:34.868481+0000","last_fullsized":"2024-09-10T16:08:34.868481+0000","mapping_epoch":44,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":892952,"num_objects":219,"num_object_clones":0,"num_object_copies":438,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":219,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18583,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20471","reported_seq":20562,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:36.006764+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:08:36.006764+0000","last_peered":"2024-09-10T16:08:36.006764+0000","last_clean":"2024-09-10T16:08:36.006764+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:08:36.006764+0000","last_undegraded":"2024-09-10T16:08:36.006764+0000","last_fullsized":"2024-09-10T16:08:36.006764+0000","mapping_epoch":35,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":774144,"num_objects":189,"num_object_clones":0,"num_object_copies":378,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":189,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20471,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20685","reported_seq":20825,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:36.050558+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:08:36.050558+0000","last_peered":"2024-09-10T16:08:36.050558+0000","last_clean":"2024-09-10T16:08:36.050558+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:08:36.050558+0000","last_undegraded":"2024-09-10T16:08:36.050558+0000","last_fullsized":"2024-09-10T16:08:36.050558+0000","mapping_epoch":34,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":844259,"num_objects":207,"num_object_clones":0,"num_object_copies":414,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":207,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17396,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20864","reported_seq":21012,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:36.051020+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:08:36.051020+0000","last_peered":"2024-09-10T16:08:36.051020+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:08:36.051020+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":807395,"num_objects":198,"num_object_clones":0,"num_object_copies":396,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":198,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17476,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4173},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'20907","reported_seq":21055,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:36.035583+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:08:36.035583+0000","last_peered":"2024-09-10T16:08:36.035583+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:08:36.035583+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":717766,"num_objects":177,"num_object_clones":0,"num_object_copies":354,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":177,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17406,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4060},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'21133","reported_seq":21273,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:36.080493+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:08:36.080493+0000","last_peered":"2024-09-10T16:08:36.080493+0000","last_clean":"2024-09-10T16:08:36.080493+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:08:36.080493+0000","last_undegraded":"2024-09-10T16:08:36.080493+0000","last_fullsized":"2024-09-10T16:08:36.080493+0000","mapping_epoch":34,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":741376,"num_objects":181,"num_object_clones":0,"num_object_copies":362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":181,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17980,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.838857+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:08:10.838857+0000","last_peered":"2024-09-10T16:08:10.838857+0000","last_clean":"2024-09-10T16:08:10.838857+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:08:10.838857+0000","last_undegraded":"2024-09-10T16:08:10.838857+0000","last_fullsized":"2024-09-10T16:08:10.838857+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20259","reported_seq":20409,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:36.104929+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:08:36.104929+0000","last_peered":"2024-09-10T16:08:36.104929+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:08:36.104929+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":811491,"num_objects":199,"num_object_clones":0,"num_object_copies":398,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":199,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18078,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-3924}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":104,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.838197+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:08:10.838197+0000","last_peered":"2024-09-10T16:08:10.838197+0000","last_clean":"2024-09-10T16:08:10.838197+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:08:10.838197+0000","last_undegraded":"2024-09-10T16:08:10.838197+0000","last_fullsized":"2024-09-10T16:08:10.838197+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20710","reported_seq":21689,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:36.114692+0000","last_change":"2024-09-10T16:08:09.280891+0000","last_active":"2024-09-10T16:08:36.114692+0000","last_peered":"2024-09-10T16:08:36.114692+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:08:36.114692+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":819200,"num_objects":200,"num_object_clones":0,"num_object_copies":400,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3838,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":200,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17214,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":419,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3638}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":115,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.840312+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:08:10.840312+0000","last_peered":"2024-09-10T16:08:10.840312+0000","last_clean":"2024-09-10T16:08:10.840312+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:08:10.840312+0000","last_undegraded":"2024-09-10T16:08:10.840312+0000","last_fullsized":"2024-09-10T16:08:10.840312+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20580","reported_seq":20725,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:36.029670+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:08:36.029670+0000","last_peered":"2024-09-10T16:08:36.029670+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:08:36.029670+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":823779,"num_objects":202,"num_object_clones":0,"num_object_copies":404,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":202,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18260,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-4102}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":90,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.837937+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:08:10.837937+0000","last_peered":"2024-09-10T16:08:10.837937+0000","last_clean":"2024-09-10T16:08:10.837937+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:08:10.837937+0000","last_undegraded":"2024-09-10T16:08:10.837937+0000","last_fullsized":"2024-09-10T16:08:10.837937+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'20992","reported_seq":29847,"reported_epoch":49,"state":"active+recovery_wait","last_fresh":"2024-09-10T16:08:36.111765+0000","last_change":"2024-09-10T16:08:10.929169+0000","last_active":"2024-09-10T16:08:36.111765+0000","last_peered":"2024-09-10T16:08:36.111765+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T16:08:10.860438+0000","last_became_peered":"2024-09-10T16:08:10.860438+0000","last_unstale":"2024-09-10T16:08:36.111765+0000","last_undegraded":"2024-09-10T16:08:36.111765+0000","last_fullsized":"2024-09-10T16:08:36.111765+0000","mapping_epoch":48,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":729088,"num_objects":178,"num_object_clones":0,"num_object_copies":356,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18546,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","0","1"],"object_location_counts":[{"shards":"1,2","objects":178}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20651","reported_seq":20796,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:36.122800+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:08:36.122800+0000","last_peered":"2024-09-10T16:08:36.122800+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:08:36.122800+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":765952,"num_objects":187,"num_object_clones":0,"num_object_copies":374,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":187,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17282,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4117}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.838546+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:08:10.838546+0000","last_peered":"2024-09-10T16:08:10.838546+0000","last_clean":"2024-09-10T16:08:10.838546+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:08:10.838546+0000","last_undegraded":"2024-09-10T16:08:10.838546+0000","last_fullsized":"2024-09-10T16:08:10.838546+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20405","reported_seq":28981,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:34.911340+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:08:34.911340+0000","last_peered":"2024-09-10T16:08:34.911340+0000","last_clean":"2024-09-10T16:08:34.911340+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:08:34.911340+0000","last_undegraded":"2024-09-10T16:08:34.911340+0000","last_fullsized":"2024-09-10T16:08:34.911340+0000","mapping_epoch":46,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":790528,"num_objects":193,"num_object_clones":0,"num_object_copies":386,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":193,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16795,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":104,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.838936+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:08:10.838936+0000","last_peered":"2024-09-10T16:08:10.838936+0000","last_clean":"2024-09-10T16:08:10.838936+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:08:10.838936+0000","last_undegraded":"2024-09-10T16:08:10.838936+0000","last_fullsized":"2024-09-10T16:08:10.838936+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20865","reported_seq":21017,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:36.116305+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:08:36.116305+0000","last_peered":"2024-09-10T16:08:36.116305+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:08:36.116305+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":749568,"num_objects":183,"num_object_clones":0,"num_object_copies":366,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":183,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17120,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4188}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.839114+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:08:10.839114+0000","last_peered":"2024-09-10T16:08:10.839114+0000","last_clean":"2024-09-10T16:08:10.839114+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:08:10.839114+0000","last_undegraded":"2024-09-10T16:08:10.839114+0000","last_fullsized":"2024-09-10T16:08:10.839114+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":85,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.838633+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:08:10.838633+0000","last_peered":"2024-09-10T16:08:10.838633+0000","last_clean":"2024-09-10T16:08:10.838633+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:08:10.838633+0000","last_undegraded":"2024-09-10T16:08:10.838633+0000","last_fullsized":"2024-09-10T16:08:10.838633+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'20998","reported_seq":21153,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:36.052751+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:08:36.052751+0000","last_peered":"2024-09-10T16:08:36.052751+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:08:36.052751+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":820166,"num_objects":202,"num_object_clones":0,"num_object_copies":404,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":202,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17161,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4121},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":12645110,"num_objects":3097,"num_object_clones":0,"num_object_copies":6194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38138,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3097,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":289009,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13252,"num_bytes_recovered":33095680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":236793856,"data_stored":236659257,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160935,"ondisk_log_size":160935,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986303,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":501156,"kb_used_data":144584,"kb_used_omap":4,"kb_used_meta":356539,"kb_avail":93870684,"statfs":{"total":96636764160,"available":96123580416,"internally_reserved":0,"allocated":148054016,"data_stored":147740831,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":365096618},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,0,2,3,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659248,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":458728,"kb_used_data":11928,"kb_used_omap":4,"kb_used_meta":446779,"kb_avail":93913112,"statfs":{"total":96636764160,"available":96167026688,"internally_reserved":0,"allocated":12214272,"data_stored":11878945,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":457502382},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,2,7],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182531,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":551116,"kb_used_data":78008,"kb_used_omap":4,"kb_used_meta":473083,"kb_avail":93820724,"statfs":{"total":96636764160,"available":96072421376,"internally_reserved":0,"allocated":79880192,"data_stored":79567002,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":484437674},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,0,1,4,3,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":79060992,"data_stored":79024862,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":10842112,"data_stored":10783845,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":146890752,"data_stored":146850550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:08:39.807 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:08:40.249 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:08:40.249 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:08:40.284 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":384,"stamp":"2024-09-10T16:08:39.763963+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":12645129,"num_objects":3099,"num_object_clones":0,"num_object_copies":6198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38138,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3099,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":289011,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13254,"num_bytes_recovered":33095699,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160938,"ondisk_log_size":160938,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1511000,"kb_used_data":234520,"kb_used_omap":12,"kb_used_meta":1276403,"kb_avail":281604520,"statfs":{"total":289910292480,"available":288363028480,"internally_reserved":0,"allocated":240148480,"data_stored":239186778,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1307036674},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,0,1,1,8,13,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19595264,"num_objects":-4784,"num_object_clones":0,"num_object_copies":-9568,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-152,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4784,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4784,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":152,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003603"},"pg_stats":[{"pgid":"2.f","version":"49'20492","reported_seq":20577,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:36.006576+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:08:36.006576+0000","last_peered":"2024-09-10T16:08:36.006576+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:08:36.006576+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":737280,"num_objects":180,"num_object_clones":0,"num_object_copies":360,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":180,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20492,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4079}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20644","reported_seq":20767,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:36.023310+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:08:36.023310+0000","last_peered":"2024-09-10T16:08:36.023310+0000","last_clean":"2024-09-10T16:08:36.023310+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:08:36.023310+0000","last_undegraded":"2024-09-10T16:08:36.023310+0000","last_fullsized":"2024-09-10T16:08:36.023310+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":820166,"num_objects":202,"num_object_clones":0,"num_object_copies":404,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":202,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18749,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20579","reported_seq":29267,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:34.868481+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:08:34.868481+0000","last_peered":"2024-09-10T16:08:34.868481+0000","last_clean":"2024-09-10T16:08:34.868481+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:08:34.868481+0000","last_undegraded":"2024-09-10T16:08:34.868481+0000","last_fullsized":"2024-09-10T16:08:34.868481+0000","mapping_epoch":44,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":892952,"num_objects":219,"num_object_clones":0,"num_object_copies":438,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":219,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18583,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20471","reported_seq":20562,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:36.006764+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:08:36.006764+0000","last_peered":"2024-09-10T16:08:36.006764+0000","last_clean":"2024-09-10T16:08:36.006764+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:08:36.006764+0000","last_undegraded":"2024-09-10T16:08:36.006764+0000","last_fullsized":"2024-09-10T16:08:36.006764+0000","mapping_epoch":35,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":774144,"num_objects":189,"num_object_clones":0,"num_object_copies":378,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":189,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20471,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20685","reported_seq":20825,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:36.050558+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:08:36.050558+0000","last_peered":"2024-09-10T16:08:36.050558+0000","last_clean":"2024-09-10T16:08:36.050558+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:08:36.050558+0000","last_undegraded":"2024-09-10T16:08:36.050558+0000","last_fullsized":"2024-09-10T16:08:36.050558+0000","mapping_epoch":34,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":844259,"num_objects":207,"num_object_clones":0,"num_object_copies":414,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":207,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17396,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20864","reported_seq":21012,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:36.051020+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:08:36.051020+0000","last_peered":"2024-09-10T16:08:36.051020+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:08:36.051020+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":807395,"num_objects":198,"num_object_clones":0,"num_object_copies":396,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":198,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17476,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4173},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'20907","reported_seq":21055,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:36.035583+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:08:36.035583+0000","last_peered":"2024-09-10T16:08:36.035583+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:08:36.035583+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":717766,"num_objects":177,"num_object_clones":0,"num_object_copies":354,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":177,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17406,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4060},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'21133","reported_seq":21273,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:36.080493+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:08:36.080493+0000","last_peered":"2024-09-10T16:08:36.080493+0000","last_clean":"2024-09-10T16:08:36.080493+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:08:36.080493+0000","last_undegraded":"2024-09-10T16:08:36.080493+0000","last_fullsized":"2024-09-10T16:08:36.080493+0000","mapping_epoch":34,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":741376,"num_objects":181,"num_object_clones":0,"num_object_copies":362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":181,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17980,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.838857+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:08:10.838857+0000","last_peered":"2024-09-10T16:08:10.838857+0000","last_clean":"2024-09-10T16:08:10.838857+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:08:10.838857+0000","last_undegraded":"2024-09-10T16:08:10.838857+0000","last_fullsized":"2024-09-10T16:08:10.838857+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20259","reported_seq":20409,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:36.104929+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:08:36.104929+0000","last_peered":"2024-09-10T16:08:36.104929+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:08:36.104929+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":811491,"num_objects":199,"num_object_clones":0,"num_object_copies":398,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":199,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18078,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-3924}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":104,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.838197+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:08:10.838197+0000","last_peered":"2024-09-10T16:08:10.838197+0000","last_clean":"2024-09-10T16:08:10.838197+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:08:10.838197+0000","last_undegraded":"2024-09-10T16:08:10.838197+0000","last_fullsized":"2024-09-10T16:08:10.838197+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20710","reported_seq":21689,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:36.114692+0000","last_change":"2024-09-10T16:08:09.280891+0000","last_active":"2024-09-10T16:08:36.114692+0000","last_peered":"2024-09-10T16:08:36.114692+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:08:36.114692+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":819200,"num_objects":200,"num_object_clones":0,"num_object_copies":400,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3838,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":200,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17214,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":419,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3638}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":115,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.840312+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:08:10.840312+0000","last_peered":"2024-09-10T16:08:10.840312+0000","last_clean":"2024-09-10T16:08:10.840312+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:08:10.840312+0000","last_undegraded":"2024-09-10T16:08:10.840312+0000","last_fullsized":"2024-09-10T16:08:10.840312+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20580","reported_seq":20725,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:36.029670+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:08:36.029670+0000","last_peered":"2024-09-10T16:08:36.029670+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:08:36.029670+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":823779,"num_objects":202,"num_object_clones":0,"num_object_copies":404,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":202,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18260,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-4102}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":90,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.837937+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:08:10.837937+0000","last_peered":"2024-09-10T16:08:10.837937+0000","last_clean":"2024-09-10T16:08:10.837937+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:08:10.837937+0000","last_undegraded":"2024-09-10T16:08:10.837937+0000","last_fullsized":"2024-09-10T16:08:10.837937+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'20992","reported_seq":29847,"reported_epoch":49,"state":"active+recovery_wait","last_fresh":"2024-09-10T16:08:36.111765+0000","last_change":"2024-09-10T16:08:10.929169+0000","last_active":"2024-09-10T16:08:36.111765+0000","last_peered":"2024-09-10T16:08:36.111765+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T16:08:10.860438+0000","last_became_peered":"2024-09-10T16:08:10.860438+0000","last_unstale":"2024-09-10T16:08:36.111765+0000","last_undegraded":"2024-09-10T16:08:36.111765+0000","last_fullsized":"2024-09-10T16:08:36.111765+0000","mapping_epoch":48,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":729088,"num_objects":178,"num_object_clones":0,"num_object_copies":356,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18546,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","0","1"],"object_location_counts":[{"shards":"1,2","objects":178}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20651","reported_seq":20796,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:36.122800+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:08:36.122800+0000","last_peered":"2024-09-10T16:08:36.122800+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:08:36.122800+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":765952,"num_objects":187,"num_object_clones":0,"num_object_copies":374,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":187,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17282,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4117}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.838546+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:08:10.838546+0000","last_peered":"2024-09-10T16:08:10.838546+0000","last_clean":"2024-09-10T16:08:10.838546+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:08:10.838546+0000","last_undegraded":"2024-09-10T16:08:10.838546+0000","last_fullsized":"2024-09-10T16:08:10.838546+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20405","reported_seq":28981,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:34.911340+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:08:34.911340+0000","last_peered":"2024-09-10T16:08:34.911340+0000","last_clean":"2024-09-10T16:08:34.911340+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:08:34.911340+0000","last_undegraded":"2024-09-10T16:08:34.911340+0000","last_fullsized":"2024-09-10T16:08:34.911340+0000","mapping_epoch":46,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":790528,"num_objects":193,"num_object_clones":0,"num_object_copies":386,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":193,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16795,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":104,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.838936+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:08:10.838936+0000","last_peered":"2024-09-10T16:08:10.838936+0000","last_clean":"2024-09-10T16:08:10.838936+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:08:10.838936+0000","last_undegraded":"2024-09-10T16:08:10.838936+0000","last_fullsized":"2024-09-10T16:08:10.838936+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20865","reported_seq":21017,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:36.116305+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:08:36.116305+0000","last_peered":"2024-09-10T16:08:36.116305+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:08:36.116305+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":749568,"num_objects":183,"num_object_clones":0,"num_object_copies":366,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":183,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17120,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4188}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.839114+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:08:10.839114+0000","last_peered":"2024-09-10T16:08:10.839114+0000","last_clean":"2024-09-10T16:08:10.839114+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:08:10.839114+0000","last_undegraded":"2024-09-10T16:08:10.839114+0000","last_fullsized":"2024-09-10T16:08:10.839114+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":85,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.838633+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:08:10.838633+0000","last_peered":"2024-09-10T16:08:10.838633+0000","last_clean":"2024-09-10T16:08:10.838633+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:08:10.838633+0000","last_undegraded":"2024-09-10T16:08:10.838633+0000","last_fullsized":"2024-09-10T16:08:10.838633+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'20998","reported_seq":21153,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:36.052751+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:08:36.052751+0000","last_peered":"2024-09-10T16:08:36.052751+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:08:36.052751+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":820166,"num_objects":202,"num_object_clones":0,"num_object_copies":404,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":202,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17161,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4121},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":12645110,"num_objects":3097,"num_object_clones":0,"num_object_copies":6194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38138,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3097,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":289009,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13252,"num_bytes_recovered":33095680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":236793856,"data_stored":236659257,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160935,"ondisk_log_size":160935,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986303,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":501156,"kb_used_data":144584,"kb_used_omap":4,"kb_used_meta":356539,"kb_avail":93870684,"statfs":{"total":96636764160,"available":96123580416,"internally_reserved":0,"allocated":148054016,"data_stored":147740831,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":365096618},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,0,2,3,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659248,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":458728,"kb_used_data":11928,"kb_used_omap":4,"kb_used_meta":446779,"kb_avail":93913112,"statfs":{"total":96636764160,"available":96167026688,"internally_reserved":0,"allocated":12214272,"data_stored":11878945,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":457502382},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,2,7],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182531,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":551116,"kb_used_data":78008,"kb_used_omap":4,"kb_used_meta":473083,"kb_avail":93820724,"statfs":{"total":96636764160,"available":96072421376,"internally_reserved":0,"allocated":79880192,"data_stored":79567002,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":484437674},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,0,1,4,3,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":79060992,"data_stored":79024862,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":10842112,"data_stored":10783845,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":146890752,"data_stored":146850550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:08:40.285 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-10T16:08:40.285 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "49'20492", 'reported_seq': 20577, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:08:36.006576+0000', 'last_change': '2024-09-10T15:59:16.281558+0000', 'last_active': '2024-09-10T16:08:36.006576+0000', 'last_peered': '2024-09-10T16:08:36.006576+0000', 'last_clean': '2024-09-10T15:59:14.132456+0000', 'last_became_active': '2024-09-10T15:59:16.274407+0000', 'last_became_peered': '2024-09-10T15:59:16.274407+0000', 'last_unstale': '2024-09-10T16:08:36.006576+0000', 'last_undegraded': '2024-09-10T15:59:15.851279+0000', 'last_fullsized': '2024-09-10T15:59:15.774356+0000', 'mapping_epoch': 42, 'log_start': "45'10400", 'ondisk_log_start': "45'10400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'objects_scrubbed': 0, 'log_size': 10092, 'log_dups_size': 0, 'ondisk_log_size': 10092, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 737280, 'num_objects': 180, 'num_object_clones': 0, 'num_object_copies': 360, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4259, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 180, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20492, 'num_write_kb': 41344, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4079}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:08:40.285 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-10T16:08:40.286 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "49'20864", 'reported_seq': 21012, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:08:36.051020+0000', 'last_change': '2024-09-10T15:59:16.308264+0000', 'last_active': '2024-09-10T16:08:36.051020+0000', 'last_peered': '2024-09-10T16:08:36.051020+0000', 'last_clean': '2024-09-10T15:59:14.111551+0000', 'last_became_active': '2024-09-10T15:59:16.300040+0000', 'last_became_peered': '2024-09-10T15:59:16.300040+0000', 'last_unstale': '2024-09-10T16:08:36.051020+0000', 'last_undegraded': '2024-09-10T15:59:15.878234+0000', 'last_fullsized': '2024-09-10T15:59:15.769328+0000', 'mapping_epoch': 42, 'log_start': "45'10800", 'ondisk_log_start': "45'10800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3442", 'last_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'objects_scrubbed': 3384, 'log_size': 10064, 'log_dups_size': 0, 'ondisk_log_size': 10064, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1292, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 807395, 'num_objects': 198, 'num_object_clones': 0, 'num_object_copies': 396, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4372, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 198, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 17476, 'num_write_kb': 28580, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4173}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:08:40.286 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-10T16:08:40.286 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "49'20907", 'reported_seq': 21055, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:08:36.035583+0000', 'last_change': '2024-09-10T15:59:16.331899+0000', 'last_active': '2024-09-10T16:08:36.035583+0000', 'last_peered': '2024-09-10T16:08:36.035583+0000', 'last_clean': '2024-09-10T15:59:14.139436+0000', 'last_became_active': '2024-09-10T15:59:16.318715+0000', 'last_became_peered': '2024-09-10T15:59:16.318715+0000', 'last_unstale': '2024-09-10T16:08:36.035583+0000', 'last_undegraded': '2024-09-10T15:59:15.877524+0000', 'last_fullsized': '2024-09-10T15:59:15.769042+0000', 'mapping_epoch': 42, 'log_start': "45'10900", 'ondisk_log_start': "45'10900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3537", 'last_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'objects_scrubbed': 3497, 'log_size': 10007, 'log_dups_size': 0, 'ondisk_log_size': 10007, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1224, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 717766, 'num_objects': 177, 'num_object_clones': 0, 'num_object_copies': 354, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4237, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 177, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 17406, 'num_write_kb': 28172, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4060}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:08:40.286 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-10T16:08:40.286 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "49'20259", 'reported_seq': 20409, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:08:36.104929+0000', 'last_change': '2024-09-10T15:59:16.175680+0000', 'last_active': '2024-09-10T16:08:36.104929+0000', 'last_peered': '2024-09-10T16:08:36.104929+0000', 'last_clean': '2024-09-10T15:59:14.088187+0000', 'last_became_active': '2024-09-10T15:59:16.165810+0000', 'last_became_peered': '2024-09-10T15:59:16.165810+0000', 'last_unstale': '2024-09-10T16:08:36.104929+0000', 'last_undegraded': '2024-09-10T15:59:15.822098+0000', 'last_fullsized': '2024-09-10T15:59:15.769123+0000', 'mapping_epoch': 42, 'log_start': "45'10200", 'ondisk_log_start': "45'10200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2218", 'last_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'objects_scrubbed': 2175, 'log_size': 10059, 'log_dups_size': 0, 'ondisk_log_size': 10059, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 825, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 811491, 'num_objects': 199, 'num_object_clones': 0, 'num_object_copies': 398, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4124, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 199, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 18078, 'num_write_kb': 32204, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': -3924}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:08:40.286 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-10T16:08:40.286 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "49'20710", 'reported_seq': 21689, 'reported_epoch': 49, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:08:36.114692+0000', 'last_change': '2024-09-10T16:08:09.280891+0000', 'last_active': '2024-09-10T16:08:36.114692+0000', 'last_peered': '2024-09-10T16:08:36.114692+0000', 'last_clean': '2024-09-10T15:59:14.129081+0000', 'last_became_active': '2024-09-10T15:59:16.269835+0000', 'last_became_peered': '2024-09-10T15:59:16.269835+0000', 'last_unstale': '2024-09-10T16:08:36.114692+0000', 'last_undegraded': '2024-09-10T15:59:15.860857+0000', 'last_fullsized': '2024-09-10T15:59:15.771410+0000', 'mapping_epoch': 42, 'log_start': "45'10700", 'ondisk_log_start': "45'10700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3557", 'last_scrub_stamp': '2024-09-10T15:59:02.493678+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:02.493678+0000', 'objects_scrubbed': 3488, 'log_size': 10010, 'log_dups_size': 0, 'ondisk_log_size': 10010, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1472, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 819200, 'num_objects': 200, 'num_object_clones': 0, 'num_object_copies': 400, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3838, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 200, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17214, 'num_write_kb': 27852, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 419, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3638}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:08:40.286 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-10T16:08:40.286 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "49'20580", 'reported_seq': 20725, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:08:36.029670+0000', 'last_change': '2024-09-10T15:59:16.284311+0000', 'last_active': '2024-09-10T16:08:36.029670+0000', 'last_peered': '2024-09-10T16:08:36.029670+0000', 'last_clean': '2024-09-10T15:59:14.116041+0000', 'last_became_active': '2024-09-10T15:59:16.274543+0000', 'last_became_peered': '2024-09-10T15:59:16.274543+0000', 'last_unstale': '2024-09-10T16:08:36.029670+0000', 'last_undegraded': '2024-09-10T15:59:15.812484+0000', 'last_fullsized': '2024-09-10T15:59:15.770885+0000', 'mapping_epoch': 42, 'log_start': "45'10500", 'ondisk_log_start': "45'10500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2356", 'last_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'objects_scrubbed': 2316, 'log_size': 10080, 'log_dups_size': 0, 'ondisk_log_size': 10080, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 783, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 823779, 'num_objects': 202, 'num_object_clones': 0, 'num_object_copies': 404, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 202, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 18260, 'num_write_kb': 32292, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': -4102}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:08:40.287 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-10T16:08:40.287 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "49'20992", 'reported_seq': 29847, 'reported_epoch': 49, 'state': 'active+recovery_wait', 'last_fresh': '2024-09-10T16:08:36.111765+0000', 'last_change': '2024-09-10T16:08:10.929169+0000', 'last_active': '2024-09-10T16:08:36.111765+0000', 'last_peered': '2024-09-10T16:08:36.111765+0000', 'last_clean': '2024-09-10T15:59:14.110654+0000', 'last_became_active': '2024-09-10T16:08:10.860438+0000', 'last_became_peered': '2024-09-10T16:08:10.860438+0000', 'last_unstale': '2024-09-10T16:08:36.111765+0000', 'last_undegraded': '2024-09-10T16:08:36.111765+0000', 'last_fullsized': '2024-09-10T16:08:36.111765+0000', 'mapping_epoch': 48, 'log_start': "45'10900", 'ondisk_log_start': "45'10900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2486", 'last_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'objects_scrubbed': 2440, 'log_size': 10092, 'log_dups_size': 0, 'ondisk_log_size': 10092, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 792, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 729088, 'num_objects': 178, 'num_object_clones': 0, 'num_object_copies': 356, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 178, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18546, 'num_write_kb': 32568, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 4355, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': ['2', '0', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 178}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:08:40.287 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-10T16:08:40.287 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "49'20651", 'reported_seq': 20796, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:08:36.122800+0000', 'last_change': '2024-09-10T15:59:16.315641+0000', 'last_active': '2024-09-10T16:08:36.122800+0000', 'last_peered': '2024-09-10T16:08:36.122800+0000', 'last_clean': '2024-09-10T15:59:14.117001+0000', 'last_became_active': '2024-09-10T15:59:16.302245+0000', 'last_became_peered': '2024-09-10T15:59:16.302245+0000', 'last_unstale': '2024-09-10T16:08:36.122800+0000', 'last_undegraded': '2024-09-10T15:59:15.845012+0000', 'last_fullsized': '2024-09-10T15:59:15.770337+0000', 'mapping_epoch': 42, 'log_start': "45'10600", 'ondisk_log_start': "45'10600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3404", 'last_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'objects_scrubbed': 3367, 'log_size': 10051, 'log_dups_size': 0, 'ondisk_log_size': 10051, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 983, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 765952, 'num_objects': 187, 'num_object_clones': 0, 'num_object_copies': 374, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 187, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17282, 'num_write_kb': 28204, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4117}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:08:40.287 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-10T16:08:40.287 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "49'20865", 'reported_seq': 21017, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:08:36.116305+0000', 'last_change': '2024-09-10T15:59:16.311291+0000', 'last_active': '2024-09-10T16:08:36.116305+0000', 'last_peered': '2024-09-10T16:08:36.116305+0000', 'last_clean': '2024-09-10T15:59:14.116363+0000', 'last_became_active': '2024-09-10T15:59:16.301244+0000', 'last_became_peered': '2024-09-10T15:59:16.301244+0000', 'last_unstale': '2024-09-10T16:08:36.116305+0000', 'last_undegraded': '2024-09-10T15:59:15.883434+0000', 'last_fullsized': '2024-09-10T15:59:15.772756+0000', 'mapping_epoch': 42, 'log_start': "45'10800", 'ondisk_log_start': "45'10800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3789", 'last_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'objects_scrubbed': 3741, 'log_size': 10065, 'log_dups_size': 0, 'ondisk_log_size': 10065, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1440, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 749568, 'num_objects': 183, 'num_object_clones': 0, 'num_object_copies': 366, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4373, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 183, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17120, 'num_write_kb': 27124, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4188}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:08:40.287 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-10T16:08:40.287 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "49'20998", 'reported_seq': 21153, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:08:36.052751+0000', 'last_change': '2024-09-10T15:59:16.313409+0000', 'last_active': '2024-09-10T16:08:36.052751+0000', 'last_peered': '2024-09-10T16:08:36.052751+0000', 'last_clean': '2024-09-10T15:59:14.129780+0000', 'last_became_active': '2024-09-10T15:59:16.300932+0000', 'last_became_peered': '2024-09-10T15:59:16.300932+0000', 'last_unstale': '2024-09-10T16:08:36.052751+0000', 'last_undegraded': '2024-09-10T15:59:15.887823+0000', 'last_fullsized': '2024-09-10T15:59:15.768513+0000', 'mapping_epoch': 42, 'log_start': "45'10900", 'ondisk_log_start': "45'10900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3880", 'last_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'objects_scrubbed': 3827, 'log_size': 10098, 'log_dups_size': 0, 'ondisk_log_size': 10098, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1397, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 820166, 'num_objects': 202, 'num_object_clones': 0, 'num_object_copies': 404, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4325, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 202, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17161, 'num_write_kb': 27072, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4121}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:08:40.287 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T16:08:40.976 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:08:40.976 INFO:teuthology.orchestra.run.smithi115.stdout:{"fsid":"7effee29-df05-4351-9741-ee203564ed81","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 38138/6198 objects degraded (615.328%), 9 pgs degraded, 9 pgs undersized","count":18},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":684,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":49,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983954,"num_in_osds":3,"osd_in_since":1725983947,"num_remapped_pgs":9},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":8},{"state_name":"active+recovery_wait","count":1},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3099,"data_bytes":12645129,"bytes_used":1547264000,"bytes_avail":288363028480,"bytes_total":289910292480,"degraded_objects":38138,"degraded_total":6198,"degraded_ratio":6.1532752500806716,"recovering_objects_per_sec":12,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":152,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":398},"fsmap":{"epoch":1,"btime":"2024-09-10T15:57:14:967120+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T16:00:13.560776+0000","services":{}},"progress_events":{}} 2024-09-10T16:08:41.001 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-10T16:08:41.002 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:08:41.566 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:08:41.568 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:08:41.594 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":384,"stamp":"2024-09-10T16:08:39.763963+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":12645129,"num_objects":3099,"num_object_clones":0,"num_object_copies":6198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38138,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3099,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":289011,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13254,"num_bytes_recovered":33095699,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160938,"ondisk_log_size":160938,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1511000,"kb_used_data":234520,"kb_used_omap":12,"kb_used_meta":1276403,"kb_avail":281604520,"statfs":{"total":289910292480,"available":288363028480,"internally_reserved":0,"allocated":240148480,"data_stored":239186778,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1307036674},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,0,1,1,8,13,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19595264,"num_objects":-4784,"num_object_clones":0,"num_object_copies":-9568,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-152,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4784,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4784,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":152,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003603"},"pg_stats":[{"pgid":"2.f","version":"49'20492","reported_seq":20577,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:36.006576+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:08:36.006576+0000","last_peered":"2024-09-10T16:08:36.006576+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:08:36.006576+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":737280,"num_objects":180,"num_object_clones":0,"num_object_copies":360,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":180,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20492,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4079}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20644","reported_seq":20767,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:36.023310+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:08:36.023310+0000","last_peered":"2024-09-10T16:08:36.023310+0000","last_clean":"2024-09-10T16:08:36.023310+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:08:36.023310+0000","last_undegraded":"2024-09-10T16:08:36.023310+0000","last_fullsized":"2024-09-10T16:08:36.023310+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":820166,"num_objects":202,"num_object_clones":0,"num_object_copies":404,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":202,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18749,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20579","reported_seq":29267,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:34.868481+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:08:34.868481+0000","last_peered":"2024-09-10T16:08:34.868481+0000","last_clean":"2024-09-10T16:08:34.868481+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:08:34.868481+0000","last_undegraded":"2024-09-10T16:08:34.868481+0000","last_fullsized":"2024-09-10T16:08:34.868481+0000","mapping_epoch":44,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":892952,"num_objects":219,"num_object_clones":0,"num_object_copies":438,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":219,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18583,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20471","reported_seq":20562,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:36.006764+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:08:36.006764+0000","last_peered":"2024-09-10T16:08:36.006764+0000","last_clean":"2024-09-10T16:08:36.006764+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:08:36.006764+0000","last_undegraded":"2024-09-10T16:08:36.006764+0000","last_fullsized":"2024-09-10T16:08:36.006764+0000","mapping_epoch":35,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":774144,"num_objects":189,"num_object_clones":0,"num_object_copies":378,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":189,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20471,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20685","reported_seq":20825,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:36.050558+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:08:36.050558+0000","last_peered":"2024-09-10T16:08:36.050558+0000","last_clean":"2024-09-10T16:08:36.050558+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:08:36.050558+0000","last_undegraded":"2024-09-10T16:08:36.050558+0000","last_fullsized":"2024-09-10T16:08:36.050558+0000","mapping_epoch":34,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":844259,"num_objects":207,"num_object_clones":0,"num_object_copies":414,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":207,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17396,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20864","reported_seq":21012,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:36.051020+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:08:36.051020+0000","last_peered":"2024-09-10T16:08:36.051020+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:08:36.051020+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":807395,"num_objects":198,"num_object_clones":0,"num_object_copies":396,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":198,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17476,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4173},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'20907","reported_seq":21055,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:36.035583+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:08:36.035583+0000","last_peered":"2024-09-10T16:08:36.035583+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:08:36.035583+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":717766,"num_objects":177,"num_object_clones":0,"num_object_copies":354,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":177,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17406,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4060},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'21133","reported_seq":21273,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:36.080493+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:08:36.080493+0000","last_peered":"2024-09-10T16:08:36.080493+0000","last_clean":"2024-09-10T16:08:36.080493+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:08:36.080493+0000","last_undegraded":"2024-09-10T16:08:36.080493+0000","last_fullsized":"2024-09-10T16:08:36.080493+0000","mapping_epoch":34,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":741376,"num_objects":181,"num_object_clones":0,"num_object_copies":362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":181,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17980,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.838857+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:08:10.838857+0000","last_peered":"2024-09-10T16:08:10.838857+0000","last_clean":"2024-09-10T16:08:10.838857+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:08:10.838857+0000","last_undegraded":"2024-09-10T16:08:10.838857+0000","last_fullsized":"2024-09-10T16:08:10.838857+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20259","reported_seq":20409,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:36.104929+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:08:36.104929+0000","last_peered":"2024-09-10T16:08:36.104929+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:08:36.104929+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":811491,"num_objects":199,"num_object_clones":0,"num_object_copies":398,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":199,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18078,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-3924}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":104,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.838197+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:08:10.838197+0000","last_peered":"2024-09-10T16:08:10.838197+0000","last_clean":"2024-09-10T16:08:10.838197+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:08:10.838197+0000","last_undegraded":"2024-09-10T16:08:10.838197+0000","last_fullsized":"2024-09-10T16:08:10.838197+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20710","reported_seq":21689,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:36.114692+0000","last_change":"2024-09-10T16:08:09.280891+0000","last_active":"2024-09-10T16:08:36.114692+0000","last_peered":"2024-09-10T16:08:36.114692+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:08:36.114692+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":819200,"num_objects":200,"num_object_clones":0,"num_object_copies":400,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3838,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":200,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17214,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":419,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3638}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":115,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.840312+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:08:10.840312+0000","last_peered":"2024-09-10T16:08:10.840312+0000","last_clean":"2024-09-10T16:08:10.840312+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:08:10.840312+0000","last_undegraded":"2024-09-10T16:08:10.840312+0000","last_fullsized":"2024-09-10T16:08:10.840312+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20580","reported_seq":20725,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:36.029670+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:08:36.029670+0000","last_peered":"2024-09-10T16:08:36.029670+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:08:36.029670+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":823779,"num_objects":202,"num_object_clones":0,"num_object_copies":404,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":202,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18260,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-4102}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":90,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.837937+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:08:10.837937+0000","last_peered":"2024-09-10T16:08:10.837937+0000","last_clean":"2024-09-10T16:08:10.837937+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:08:10.837937+0000","last_undegraded":"2024-09-10T16:08:10.837937+0000","last_fullsized":"2024-09-10T16:08:10.837937+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'20992","reported_seq":29847,"reported_epoch":49,"state":"active+recovery_wait","last_fresh":"2024-09-10T16:08:36.111765+0000","last_change":"2024-09-10T16:08:10.929169+0000","last_active":"2024-09-10T16:08:36.111765+0000","last_peered":"2024-09-10T16:08:36.111765+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T16:08:10.860438+0000","last_became_peered":"2024-09-10T16:08:10.860438+0000","last_unstale":"2024-09-10T16:08:36.111765+0000","last_undegraded":"2024-09-10T16:08:36.111765+0000","last_fullsized":"2024-09-10T16:08:36.111765+0000","mapping_epoch":48,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":729088,"num_objects":178,"num_object_clones":0,"num_object_copies":356,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18546,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","0","1"],"object_location_counts":[{"shards":"1,2","objects":178}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20651","reported_seq":20796,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:36.122800+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:08:36.122800+0000","last_peered":"2024-09-10T16:08:36.122800+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:08:36.122800+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":765952,"num_objects":187,"num_object_clones":0,"num_object_copies":374,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":187,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17282,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4117}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.838546+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:08:10.838546+0000","last_peered":"2024-09-10T16:08:10.838546+0000","last_clean":"2024-09-10T16:08:10.838546+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:08:10.838546+0000","last_undegraded":"2024-09-10T16:08:10.838546+0000","last_fullsized":"2024-09-10T16:08:10.838546+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20405","reported_seq":28981,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:34.911340+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:08:34.911340+0000","last_peered":"2024-09-10T16:08:34.911340+0000","last_clean":"2024-09-10T16:08:34.911340+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:08:34.911340+0000","last_undegraded":"2024-09-10T16:08:34.911340+0000","last_fullsized":"2024-09-10T16:08:34.911340+0000","mapping_epoch":46,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":790528,"num_objects":193,"num_object_clones":0,"num_object_copies":386,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":193,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16795,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":104,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.838936+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:08:10.838936+0000","last_peered":"2024-09-10T16:08:10.838936+0000","last_clean":"2024-09-10T16:08:10.838936+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:08:10.838936+0000","last_undegraded":"2024-09-10T16:08:10.838936+0000","last_fullsized":"2024-09-10T16:08:10.838936+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20865","reported_seq":21017,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:36.116305+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:08:36.116305+0000","last_peered":"2024-09-10T16:08:36.116305+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:08:36.116305+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":749568,"num_objects":183,"num_object_clones":0,"num_object_copies":366,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":183,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17120,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4188}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.839114+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:08:10.839114+0000","last_peered":"2024-09-10T16:08:10.839114+0000","last_clean":"2024-09-10T16:08:10.839114+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:08:10.839114+0000","last_undegraded":"2024-09-10T16:08:10.839114+0000","last_fullsized":"2024-09-10T16:08:10.839114+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":85,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.838633+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:08:10.838633+0000","last_peered":"2024-09-10T16:08:10.838633+0000","last_clean":"2024-09-10T16:08:10.838633+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:08:10.838633+0000","last_undegraded":"2024-09-10T16:08:10.838633+0000","last_fullsized":"2024-09-10T16:08:10.838633+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'20998","reported_seq":21153,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:36.052751+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:08:36.052751+0000","last_peered":"2024-09-10T16:08:36.052751+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:08:36.052751+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":820166,"num_objects":202,"num_object_clones":0,"num_object_copies":404,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":202,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17161,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4121},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":12645110,"num_objects":3097,"num_object_clones":0,"num_object_copies":6194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38138,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3097,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":289009,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13252,"num_bytes_recovered":33095680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":236793856,"data_stored":236659257,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160935,"ondisk_log_size":160935,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986303,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":501156,"kb_used_data":144584,"kb_used_omap":4,"kb_used_meta":356539,"kb_avail":93870684,"statfs":{"total":96636764160,"available":96123580416,"internally_reserved":0,"allocated":148054016,"data_stored":147740831,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":365096618},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,0,2,3,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659248,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":458728,"kb_used_data":11928,"kb_used_omap":4,"kb_used_meta":446779,"kb_avail":93913112,"statfs":{"total":96636764160,"available":96167026688,"internally_reserved":0,"allocated":12214272,"data_stored":11878945,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":457502382},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,2,7],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182531,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":551116,"kb_used_data":78008,"kb_used_omap":4,"kb_used_meta":473083,"kb_avail":93820724,"statfs":{"total":96636764160,"available":96072421376,"internally_reserved":0,"allocated":79880192,"data_stored":79567002,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":484437674},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,0,1,4,3,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":79060992,"data_stored":79024862,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":10842112,"data_stored":10783845,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":146890752,"data_stored":146850550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:08:42.677 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout:Removed 167119 objects 2024-09-10T16:08:42.677 INFO:tasks.lost_unfound.radosbench.client.admin.smithi115.stdout:Clean up completed and total clean up time :369.67 2024-09-10T16:08:44.596 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:08:44.893 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:08:44.894 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:08:44.906 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":386,"stamp":"2024-09-10T16:08:43.765030+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":3216137,"num_objects":797,"num_object_clones":0,"num_object_copies":1594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38071,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":797,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":291313,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13321,"num_bytes_recovered":33095699,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160740,"ondisk_log_size":160740,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1509500,"kb_used_data":214972,"kb_used_omap":12,"kb_used_meta":1294451,"kb_avail":281606020,"statfs":{"total":289910292480,"available":288364564480,"internally_reserved":0,"allocated":220131328,"data_stored":219169626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1325517826},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,1,2,11,5,14,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19255296,"num_objects":-4701,"num_object_clones":0,"num_object_copies":-9402,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4701,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4701,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":142,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003777"},"pg_stats":[{"pgid":"2.f","version":"49'20635","reported_seq":20720,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:41.188030+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:08:41.188030+0000","last_peered":"2024-09-10T16:08:41.188030+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:08:41.188030+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":151552,"num_objects":37,"num_object_clones":0,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20635,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4222}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20787","reported_seq":20910,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:41.002992+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:08:41.002992+0000","last_peered":"2024-09-10T16:08:41.002992+0000","last_clean":"2024-09-10T16:08:41.002992+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:08:41.002992+0000","last_undegraded":"2024-09-10T16:08:41.002992+0000","last_fullsized":"2024-09-10T16:08:41.002992+0000","mapping_epoch":35,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":234438,"num_objects":59,"num_object_clones":0,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18892,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20728","reported_seq":29416,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:40.223813+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:08:40.223813+0000","last_peered":"2024-09-10T16:08:40.223813+0000","last_clean":"2024-09-10T16:08:40.223813+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:08:40.223813+0000","last_undegraded":"2024-09-10T16:08:40.223813+0000","last_fullsized":"2024-09-10T16:08:40.223813+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":282648,"num_objects":70,"num_object_clones":0,"num_object_copies":140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18732,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20613","reported_seq":20704,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:40.884047+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:08:40.884047+0000","last_peered":"2024-09-10T16:08:40.884047+0000","last_clean":"2024-09-10T16:08:40.884047+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:08:40.884047+0000","last_undegraded":"2024-09-10T16:08:40.884047+0000","last_fullsized":"2024-09-10T16:08:40.884047+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":192512,"num_objects":47,"num_object_clones":0,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20613,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20842","reported_seq":20982,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:41.136021+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:08:41.136021+0000","last_peered":"2024-09-10T16:08:41.136021+0000","last_clean":"2024-09-10T16:08:41.136021+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:08:41.136021+0000","last_undegraded":"2024-09-10T16:08:41.136021+0000","last_fullsized":"2024-09-10T16:08:41.136021+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":201187,"num_objects":50,"num_object_clones":0,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17553,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'21010","reported_seq":21158,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:41.132237+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:08:41.132237+0000","last_peered":"2024-09-10T16:08:41.132237+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:08:41.132237+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":209379,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17622,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4319},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'21041","reported_seq":21189,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:41.145052+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:08:41.145052+0000","last_peered":"2024-09-10T16:08:41.145052+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:08:41.145052+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":168902,"num_objects":43,"num_object_clones":0,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17540,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4194},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'21273","reported_seq":21413,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:41.149656+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:08:41.149656+0000","last_peered":"2024-09-10T16:08:41.149656+0000","last_clean":"2024-09-10T16:08:41.149656+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:08:41.149656+0000","last_undegraded":"2024-09-10T16:08:41.149656+0000","last_fullsized":"2024-09-10T16:08:41.149656+0000","mapping_epoch":34,"log_start":"45'11200","ondisk_log_start":"45'11200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":167936,"num_objects":41,"num_object_clones":0,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18120,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.838857+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:08:10.838857+0000","last_peered":"2024-09-10T16:08:10.838857+0000","last_clean":"2024-09-10T16:08:10.838857+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:08:10.838857+0000","last_undegraded":"2024-09-10T16:08:10.838857+0000","last_fullsized":"2024-09-10T16:08:10.838857+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20410","reported_seq":20560,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:41.128326+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:08:41.128326+0000","last_peered":"2024-09-10T16:08:41.128326+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:08:41.128326+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":192995,"num_objects":48,"num_object_clones":0,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18229,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-4075}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":104,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.838197+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:08:10.838197+0000","last_peered":"2024-09-10T16:08:10.838197+0000","last_clean":"2024-09-10T16:08:10.838197+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:08:10.838197+0000","last_undegraded":"2024-09-10T16:08:10.838197+0000","last_fullsized":"2024-09-10T16:08:10.838197+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20857","reported_seq":21969,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:41.129625+0000","last_change":"2024-09-10T16:08:09.280891+0000","last_active":"2024-09-10T16:08:41.129625+0000","last_peered":"2024-09-10T16:08:41.129625+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:08:41.129625+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":217088,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3771,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17361,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":486,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3718}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":115,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.840312+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:08:10.840312+0000","last_peered":"2024-09-10T16:08:10.840312+0000","last_clean":"2024-09-10T16:08:10.840312+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:08:10.840312+0000","last_undegraded":"2024-09-10T16:08:10.840312+0000","last_fullsized":"2024-09-10T16:08:10.840312+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20736","reported_seq":20881,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:41.039501+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:08:41.039501+0000","last_peered":"2024-09-10T16:08:41.039501+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:08:41.039501+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":184803,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18416,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-4258}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":90,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.837937+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:08:10.837937+0000","last_peered":"2024-09-10T16:08:10.837937+0000","last_clean":"2024-09-10T16:08:10.837937+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:08:10.837937+0000","last_undegraded":"2024-09-10T16:08:10.837937+0000","last_fullsized":"2024-09-10T16:08:10.837937+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'21129","reported_seq":29984,"reported_epoch":49,"state":"active+recovery_wait","last_fresh":"2024-09-10T16:08:41.106117+0000","last_change":"2024-09-10T16:08:10.929169+0000","last_active":"2024-09-10T16:08:41.106117+0000","last_peered":"2024-09-10T16:08:41.106117+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T16:08:10.860438+0000","last_became_peered":"2024-09-10T16:08:10.860438+0000","last_unstale":"2024-09-10T16:08:41.106117+0000","last_undegraded":"2024-09-10T16:08:41.106117+0000","last_fullsized":"2024-09-10T16:08:41.106117+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":167936,"num_objects":41,"num_object_clones":0,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18683,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","0","1"],"object_location_counts":[{"shards":"1,2","objects":41}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20794","reported_seq":20939,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:40.940386+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:08:40.940386+0000","last_peered":"2024-09-10T16:08:40.940386+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:08:40.940386+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180224,"num_objects":44,"num_object_clones":0,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17425,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4260}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.838546+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:08:10.838546+0000","last_peered":"2024-09-10T16:08:10.838546+0000","last_clean":"2024-09-10T16:08:10.838546+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:08:10.838546+0000","last_undegraded":"2024-09-10T16:08:10.838546+0000","last_fullsized":"2024-09-10T16:08:10.838546+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20529","reported_seq":29105,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:40.212055+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:08:40.212055+0000","last_peered":"2024-09-10T16:08:40.212055+0000","last_clean":"2024-09-10T16:08:40.212055+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:08:40.212055+0000","last_undegraded":"2024-09-10T16:08:40.212055+0000","last_fullsized":"2024-09-10T16:08:40.212055+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":282624,"num_objects":69,"num_object_clones":0,"num_object_copies":138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":69,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16919,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":104,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.838936+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:08:10.838936+0000","last_peered":"2024-09-10T16:08:10.838936+0000","last_clean":"2024-09-10T16:08:10.838936+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:08:10.838936+0000","last_undegraded":"2024-09-10T16:08:10.838936+0000","last_fullsized":"2024-09-10T16:08:10.838936+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20997","reported_seq":21149,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:41.089473+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:08:41.089473+0000","last_peered":"2024-09-10T16:08:41.089473+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:08:41.089473+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":208896,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17252,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4320}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.839114+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:08:10.839114+0000","last_peered":"2024-09-10T16:08:10.839114+0000","last_clean":"2024-09-10T16:08:10.839114+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:08:10.839114+0000","last_undegraded":"2024-09-10T16:08:10.839114+0000","last_fullsized":"2024-09-10T16:08:10.839114+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":85,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.838633+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:08:10.838633+0000","last_peered":"2024-09-10T16:08:10.838633+0000","last_clean":"2024-09-10T16:08:10.838633+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:08:10.838633+0000","last_undegraded":"2024-09-10T16:08:10.838633+0000","last_fullsized":"2024-09-10T16:08:10.838633+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'21156","reported_seq":21310,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:41.143857+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:08:41.143857+0000","last_peered":"2024-09-10T16:08:41.143857+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:08:41.143857+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":172998,"num_objects":44,"num_object_clones":0,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17319,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4279},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":3216118,"num_objects":795,"num_object_clones":0,"num_object_copies":1590,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38071,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":795,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":291311,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13319,"num_bytes_recovered":33095680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":217694208,"data_stored":217559609,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160737,"ondisk_log_size":160737,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986304,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":500680,"kb_used_data":139960,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93871160,"statfs":{"total":96636764160,"available":96124067840,"internally_reserved":0,"allocated":143319040,"data_stored":143005855,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369356458},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,2,2,1,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659249,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":458956,"kb_used_data":3824,"kb_used_omap":4,"kb_used_meta":455099,"kb_avail":93912884,"statfs":{"total":96636764160,"available":96166793216,"internally_reserved":0,"allocated":3915776,"data_stored":3580449,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":466022062},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,0,2,2,7],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182532,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":549864,"kb_used_data":71188,"kb_used_omap":4,"kb_used_meta":478651,"kb_avail":93821976,"statfs":{"total":96636764160,"available":96073703424,"internally_reserved":0,"allocated":72896512,"data_stored":72583322,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":490139306},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,7,2,5,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":72056832,"data_stored":72020702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":3502080,"data_stored":3443813,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":142135296,"data_stored":142095094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:08:44.907 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:08:45.214 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:08:45.215 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:08:45.227 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":386,"stamp":"2024-09-10T16:08:43.765030+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":3216137,"num_objects":797,"num_object_clones":0,"num_object_copies":1594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38071,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":797,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":291313,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13321,"num_bytes_recovered":33095699,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160740,"ondisk_log_size":160740,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1509500,"kb_used_data":214972,"kb_used_omap":12,"kb_used_meta":1294451,"kb_avail":281606020,"statfs":{"total":289910292480,"available":288364564480,"internally_reserved":0,"allocated":220131328,"data_stored":219169626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1325517826},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,1,2,11,5,14,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19255296,"num_objects":-4701,"num_object_clones":0,"num_object_copies":-9402,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4701,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4701,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":142,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003777"},"pg_stats":[{"pgid":"2.f","version":"49'20635","reported_seq":20720,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:41.188030+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:08:41.188030+0000","last_peered":"2024-09-10T16:08:41.188030+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:08:41.188030+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":151552,"num_objects":37,"num_object_clones":0,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20635,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4222}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20787","reported_seq":20910,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:41.002992+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:08:41.002992+0000","last_peered":"2024-09-10T16:08:41.002992+0000","last_clean":"2024-09-10T16:08:41.002992+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:08:41.002992+0000","last_undegraded":"2024-09-10T16:08:41.002992+0000","last_fullsized":"2024-09-10T16:08:41.002992+0000","mapping_epoch":35,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":234438,"num_objects":59,"num_object_clones":0,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18892,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20728","reported_seq":29416,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:40.223813+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:08:40.223813+0000","last_peered":"2024-09-10T16:08:40.223813+0000","last_clean":"2024-09-10T16:08:40.223813+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:08:40.223813+0000","last_undegraded":"2024-09-10T16:08:40.223813+0000","last_fullsized":"2024-09-10T16:08:40.223813+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":282648,"num_objects":70,"num_object_clones":0,"num_object_copies":140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18732,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20613","reported_seq":20704,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:40.884047+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:08:40.884047+0000","last_peered":"2024-09-10T16:08:40.884047+0000","last_clean":"2024-09-10T16:08:40.884047+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:08:40.884047+0000","last_undegraded":"2024-09-10T16:08:40.884047+0000","last_fullsized":"2024-09-10T16:08:40.884047+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":192512,"num_objects":47,"num_object_clones":0,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20613,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20842","reported_seq":20982,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:41.136021+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:08:41.136021+0000","last_peered":"2024-09-10T16:08:41.136021+0000","last_clean":"2024-09-10T16:08:41.136021+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:08:41.136021+0000","last_undegraded":"2024-09-10T16:08:41.136021+0000","last_fullsized":"2024-09-10T16:08:41.136021+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":201187,"num_objects":50,"num_object_clones":0,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17553,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'21010","reported_seq":21158,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:41.132237+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:08:41.132237+0000","last_peered":"2024-09-10T16:08:41.132237+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:08:41.132237+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":209379,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17622,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4319},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'21041","reported_seq":21189,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:41.145052+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:08:41.145052+0000","last_peered":"2024-09-10T16:08:41.145052+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:08:41.145052+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":168902,"num_objects":43,"num_object_clones":0,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17540,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4194},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'21273","reported_seq":21413,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:41.149656+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:08:41.149656+0000","last_peered":"2024-09-10T16:08:41.149656+0000","last_clean":"2024-09-10T16:08:41.149656+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:08:41.149656+0000","last_undegraded":"2024-09-10T16:08:41.149656+0000","last_fullsized":"2024-09-10T16:08:41.149656+0000","mapping_epoch":34,"log_start":"45'11200","ondisk_log_start":"45'11200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":167936,"num_objects":41,"num_object_clones":0,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18120,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.838857+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:08:10.838857+0000","last_peered":"2024-09-10T16:08:10.838857+0000","last_clean":"2024-09-10T16:08:10.838857+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:08:10.838857+0000","last_undegraded":"2024-09-10T16:08:10.838857+0000","last_fullsized":"2024-09-10T16:08:10.838857+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20410","reported_seq":20560,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:41.128326+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:08:41.128326+0000","last_peered":"2024-09-10T16:08:41.128326+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:08:41.128326+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":192995,"num_objects":48,"num_object_clones":0,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18229,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-4075}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":104,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.838197+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:08:10.838197+0000","last_peered":"2024-09-10T16:08:10.838197+0000","last_clean":"2024-09-10T16:08:10.838197+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:08:10.838197+0000","last_undegraded":"2024-09-10T16:08:10.838197+0000","last_fullsized":"2024-09-10T16:08:10.838197+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20857","reported_seq":21969,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:41.129625+0000","last_change":"2024-09-10T16:08:09.280891+0000","last_active":"2024-09-10T16:08:41.129625+0000","last_peered":"2024-09-10T16:08:41.129625+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:08:41.129625+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":217088,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3771,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17361,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":486,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3718}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":115,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.840312+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:08:10.840312+0000","last_peered":"2024-09-10T16:08:10.840312+0000","last_clean":"2024-09-10T16:08:10.840312+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:08:10.840312+0000","last_undegraded":"2024-09-10T16:08:10.840312+0000","last_fullsized":"2024-09-10T16:08:10.840312+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20736","reported_seq":20881,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:41.039501+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:08:41.039501+0000","last_peered":"2024-09-10T16:08:41.039501+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:08:41.039501+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":184803,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18416,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-4258}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":90,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.837937+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:08:10.837937+0000","last_peered":"2024-09-10T16:08:10.837937+0000","last_clean":"2024-09-10T16:08:10.837937+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:08:10.837937+0000","last_undegraded":"2024-09-10T16:08:10.837937+0000","last_fullsized":"2024-09-10T16:08:10.837937+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'21129","reported_seq":29984,"reported_epoch":49,"state":"active+recovery_wait","last_fresh":"2024-09-10T16:08:41.106117+0000","last_change":"2024-09-10T16:08:10.929169+0000","last_active":"2024-09-10T16:08:41.106117+0000","last_peered":"2024-09-10T16:08:41.106117+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T16:08:10.860438+0000","last_became_peered":"2024-09-10T16:08:10.860438+0000","last_unstale":"2024-09-10T16:08:41.106117+0000","last_undegraded":"2024-09-10T16:08:41.106117+0000","last_fullsized":"2024-09-10T16:08:41.106117+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":167936,"num_objects":41,"num_object_clones":0,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18683,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","0","1"],"object_location_counts":[{"shards":"1,2","objects":41}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20794","reported_seq":20939,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:40.940386+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:08:40.940386+0000","last_peered":"2024-09-10T16:08:40.940386+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:08:40.940386+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180224,"num_objects":44,"num_object_clones":0,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17425,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4260}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.838546+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:08:10.838546+0000","last_peered":"2024-09-10T16:08:10.838546+0000","last_clean":"2024-09-10T16:08:10.838546+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:08:10.838546+0000","last_undegraded":"2024-09-10T16:08:10.838546+0000","last_fullsized":"2024-09-10T16:08:10.838546+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20529","reported_seq":29105,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:40.212055+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:08:40.212055+0000","last_peered":"2024-09-10T16:08:40.212055+0000","last_clean":"2024-09-10T16:08:40.212055+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:08:40.212055+0000","last_undegraded":"2024-09-10T16:08:40.212055+0000","last_fullsized":"2024-09-10T16:08:40.212055+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":282624,"num_objects":69,"num_object_clones":0,"num_object_copies":138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":69,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16919,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":104,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.838936+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:08:10.838936+0000","last_peered":"2024-09-10T16:08:10.838936+0000","last_clean":"2024-09-10T16:08:10.838936+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:08:10.838936+0000","last_undegraded":"2024-09-10T16:08:10.838936+0000","last_fullsized":"2024-09-10T16:08:10.838936+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20997","reported_seq":21149,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:41.089473+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:08:41.089473+0000","last_peered":"2024-09-10T16:08:41.089473+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:08:41.089473+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":208896,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17252,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4320}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.839114+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:08:10.839114+0000","last_peered":"2024-09-10T16:08:10.839114+0000","last_clean":"2024-09-10T16:08:10.839114+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:08:10.839114+0000","last_undegraded":"2024-09-10T16:08:10.839114+0000","last_fullsized":"2024-09-10T16:08:10.839114+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":85,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.838633+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:08:10.838633+0000","last_peered":"2024-09-10T16:08:10.838633+0000","last_clean":"2024-09-10T16:08:10.838633+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:08:10.838633+0000","last_undegraded":"2024-09-10T16:08:10.838633+0000","last_fullsized":"2024-09-10T16:08:10.838633+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'21156","reported_seq":21310,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:41.143857+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:08:41.143857+0000","last_peered":"2024-09-10T16:08:41.143857+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:08:41.143857+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":172998,"num_objects":44,"num_object_clones":0,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17319,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4279},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":3216118,"num_objects":795,"num_object_clones":0,"num_object_copies":1590,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38071,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":795,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":291311,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13319,"num_bytes_recovered":33095680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":217694208,"data_stored":217559609,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160737,"ondisk_log_size":160737,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986304,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":500680,"kb_used_data":139960,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93871160,"statfs":{"total":96636764160,"available":96124067840,"internally_reserved":0,"allocated":143319040,"data_stored":143005855,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369356458},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,2,2,1,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659249,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":458956,"kb_used_data":3824,"kb_used_omap":4,"kb_used_meta":455099,"kb_avail":93912884,"statfs":{"total":96636764160,"available":96166793216,"internally_reserved":0,"allocated":3915776,"data_stored":3580449,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":466022062},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,0,2,2,7],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182532,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":549864,"kb_used_data":71188,"kb_used_omap":4,"kb_used_meta":478651,"kb_avail":93821976,"statfs":{"total":96636764160,"available":96073703424,"internally_reserved":0,"allocated":72896512,"data_stored":72583322,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":490139306},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,7,2,5,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":72056832,"data_stored":72020702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":3502080,"data_stored":3443813,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":142135296,"data_stored":142095094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:08:45.228 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-10T16:08:45.228 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "49'20635", 'reported_seq': 20720, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:08:41.188030+0000', 'last_change': '2024-09-10T15:59:16.281558+0000', 'last_active': '2024-09-10T16:08:41.188030+0000', 'last_peered': '2024-09-10T16:08:41.188030+0000', 'last_clean': '2024-09-10T15:59:14.132456+0000', 'last_became_active': '2024-09-10T15:59:16.274407+0000', 'last_became_peered': '2024-09-10T15:59:16.274407+0000', 'last_unstale': '2024-09-10T16:08:41.188030+0000', 'last_undegraded': '2024-09-10T15:59:15.851279+0000', 'last_fullsized': '2024-09-10T15:59:15.774356+0000', 'mapping_epoch': 42, 'log_start': "45'10600", 'ondisk_log_start': "45'10600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'objects_scrubbed': 0, 'log_size': 10035, 'log_dups_size': 0, 'ondisk_log_size': 10035, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 151552, 'num_objects': 37, 'num_object_clones': 0, 'num_object_copies': 74, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4259, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 37, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20635, 'num_write_kb': 41344, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4222}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:08:45.229 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-10T16:08:45.229 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "49'21010", 'reported_seq': 21158, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:08:41.132237+0000', 'last_change': '2024-09-10T15:59:16.308264+0000', 'last_active': '2024-09-10T16:08:41.132237+0000', 'last_peered': '2024-09-10T16:08:41.132237+0000', 'last_clean': '2024-09-10T15:59:14.111551+0000', 'last_became_active': '2024-09-10T15:59:16.300040+0000', 'last_became_peered': '2024-09-10T15:59:16.300040+0000', 'last_unstale': '2024-09-10T16:08:41.132237+0000', 'last_undegraded': '2024-09-10T15:59:15.878234+0000', 'last_fullsized': '2024-09-10T15:59:15.769328+0000', 'mapping_epoch': 42, 'log_start': "45'11000", 'ondisk_log_start': "45'11000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3442", 'last_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'objects_scrubbed': 3384, 'log_size': 10010, 'log_dups_size': 0, 'ondisk_log_size': 10010, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1292, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 209379, 'num_objects': 52, 'num_object_clones': 0, 'num_object_copies': 104, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4372, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 52, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 17622, 'num_write_kb': 28580, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4319}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:08:45.229 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-10T16:08:45.229 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "49'21041", 'reported_seq': 21189, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:08:41.145052+0000', 'last_change': '2024-09-10T15:59:16.331899+0000', 'last_active': '2024-09-10T16:08:41.145052+0000', 'last_peered': '2024-09-10T16:08:41.145052+0000', 'last_clean': '2024-09-10T15:59:14.139436+0000', 'last_became_active': '2024-09-10T15:59:16.318715+0000', 'last_became_peered': '2024-09-10T15:59:16.318715+0000', 'last_unstale': '2024-09-10T16:08:41.145052+0000', 'last_undegraded': '2024-09-10T15:59:15.877524+0000', 'last_fullsized': '2024-09-10T15:59:15.769042+0000', 'mapping_epoch': 42, 'log_start': "45'11000", 'ondisk_log_start': "45'11000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3537", 'last_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'objects_scrubbed': 3497, 'log_size': 10041, 'log_dups_size': 0, 'ondisk_log_size': 10041, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1224, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 168902, 'num_objects': 43, 'num_object_clones': 0, 'num_object_copies': 86, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4237, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 43, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 17540, 'num_write_kb': 28172, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4194}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:08:45.229 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-10T16:08:45.229 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "49'20410", 'reported_seq': 20560, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:08:41.128326+0000', 'last_change': '2024-09-10T15:59:16.175680+0000', 'last_active': '2024-09-10T16:08:41.128326+0000', 'last_peered': '2024-09-10T16:08:41.128326+0000', 'last_clean': '2024-09-10T15:59:14.088187+0000', 'last_became_active': '2024-09-10T15:59:16.165810+0000', 'last_became_peered': '2024-09-10T15:59:16.165810+0000', 'last_unstale': '2024-09-10T16:08:41.128326+0000', 'last_undegraded': '2024-09-10T15:59:15.822098+0000', 'last_fullsized': '2024-09-10T15:59:15.769123+0000', 'mapping_epoch': 42, 'log_start': "45'10400", 'ondisk_log_start': "45'10400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2218", 'last_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'objects_scrubbed': 2175, 'log_size': 10010, 'log_dups_size': 0, 'ondisk_log_size': 10010, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 825, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 192995, 'num_objects': 48, 'num_object_clones': 0, 'num_object_copies': 96, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4124, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 48, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 18229, 'num_write_kb': 32204, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': -4075}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:08:45.229 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-10T16:08:45.229 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "49'20857", 'reported_seq': 21969, 'reported_epoch': 49, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:08:41.129625+0000', 'last_change': '2024-09-10T16:08:09.280891+0000', 'last_active': '2024-09-10T16:08:41.129625+0000', 'last_peered': '2024-09-10T16:08:41.129625+0000', 'last_clean': '2024-09-10T15:59:14.129081+0000', 'last_became_active': '2024-09-10T15:59:16.269835+0000', 'last_became_peered': '2024-09-10T15:59:16.269835+0000', 'last_unstale': '2024-09-10T16:08:41.129625+0000', 'last_undegraded': '2024-09-10T15:59:15.860857+0000', 'last_fullsized': '2024-09-10T15:59:15.771410+0000', 'mapping_epoch': 42, 'log_start': "45'10800", 'ondisk_log_start': "45'10800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3557", 'last_scrub_stamp': '2024-09-10T15:59:02.493678+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:02.493678+0000', 'objects_scrubbed': 3488, 'log_size': 10057, 'log_dups_size': 0, 'ondisk_log_size': 10057, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1472, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 217088, 'num_objects': 53, 'num_object_clones': 0, 'num_object_copies': 106, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3771, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 53, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17361, 'num_write_kb': 27852, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 486, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3718}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:08:45.229 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-10T16:08:45.229 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "49'20736", 'reported_seq': 20881, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:08:41.039501+0000', 'last_change': '2024-09-10T15:59:16.284311+0000', 'last_active': '2024-09-10T16:08:41.039501+0000', 'last_peered': '2024-09-10T16:08:41.039501+0000', 'last_clean': '2024-09-10T15:59:14.116041+0000', 'last_became_active': '2024-09-10T15:59:16.274543+0000', 'last_became_peered': '2024-09-10T15:59:16.274543+0000', 'last_unstale': '2024-09-10T16:08:41.039501+0000', 'last_undegraded': '2024-09-10T15:59:15.812484+0000', 'last_fullsized': '2024-09-10T15:59:15.770885+0000', 'mapping_epoch': 42, 'log_start': "45'10700", 'ondisk_log_start': "45'10700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2356", 'last_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'objects_scrubbed': 2316, 'log_size': 10036, 'log_dups_size': 0, 'ondisk_log_size': 10036, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 783, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 184803, 'num_objects': 46, 'num_object_clones': 0, 'num_object_copies': 92, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 46, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 18416, 'num_write_kb': 32292, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': -4258}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:08:45.230 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-10T16:08:45.230 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "49'21129", 'reported_seq': 29984, 'reported_epoch': 49, 'state': 'active+recovery_wait', 'last_fresh': '2024-09-10T16:08:41.106117+0000', 'last_change': '2024-09-10T16:08:10.929169+0000', 'last_active': '2024-09-10T16:08:41.106117+0000', 'last_peered': '2024-09-10T16:08:41.106117+0000', 'last_clean': '2024-09-10T15:59:14.110654+0000', 'last_became_active': '2024-09-10T16:08:10.860438+0000', 'last_became_peered': '2024-09-10T16:08:10.860438+0000', 'last_unstale': '2024-09-10T16:08:41.106117+0000', 'last_undegraded': '2024-09-10T16:08:41.106117+0000', 'last_fullsized': '2024-09-10T16:08:41.106117+0000', 'mapping_epoch': 48, 'log_start': "45'11100", 'ondisk_log_start': "45'11100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2486", 'last_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'objects_scrubbed': 2440, 'log_size': 10029, 'log_dups_size': 0, 'ondisk_log_size': 10029, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 792, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 167936, 'num_objects': 41, 'num_object_clones': 0, 'num_object_copies': 82, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 41, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18683, 'num_write_kb': 32568, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 4355, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': ['2', '0', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 41}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:08:45.230 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-10T16:08:45.230 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "49'20794", 'reported_seq': 20939, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:08:40.940386+0000', 'last_change': '2024-09-10T15:59:16.315641+0000', 'last_active': '2024-09-10T16:08:40.940386+0000', 'last_peered': '2024-09-10T16:08:40.940386+0000', 'last_clean': '2024-09-10T15:59:14.117001+0000', 'last_became_active': '2024-09-10T15:59:16.302245+0000', 'last_became_peered': '2024-09-10T15:59:16.302245+0000', 'last_unstale': '2024-09-10T16:08:40.940386+0000', 'last_undegraded': '2024-09-10T15:59:15.845012+0000', 'last_fullsized': '2024-09-10T15:59:15.770337+0000', 'mapping_epoch': 42, 'log_start': "45'10700", 'ondisk_log_start': "45'10700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3404", 'last_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'objects_scrubbed': 3367, 'log_size': 10094, 'log_dups_size': 0, 'ondisk_log_size': 10094, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 983, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 180224, 'num_objects': 44, 'num_object_clones': 0, 'num_object_copies': 88, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 44, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17425, 'num_write_kb': 28204, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4260}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:08:45.230 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-10T16:08:45.230 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "49'20997", 'reported_seq': 21149, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:08:41.089473+0000', 'last_change': '2024-09-10T15:59:16.311291+0000', 'last_active': '2024-09-10T16:08:41.089473+0000', 'last_peered': '2024-09-10T16:08:41.089473+0000', 'last_clean': '2024-09-10T15:59:14.116363+0000', 'last_became_active': '2024-09-10T15:59:16.301244+0000', 'last_became_peered': '2024-09-10T15:59:16.301244+0000', 'last_unstale': '2024-09-10T16:08:41.089473+0000', 'last_undegraded': '2024-09-10T15:59:15.883434+0000', 'last_fullsized': '2024-09-10T15:59:15.772756+0000', 'mapping_epoch': 42, 'log_start': "45'10900", 'ondisk_log_start': "45'10900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3789", 'last_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'objects_scrubbed': 3741, 'log_size': 10097, 'log_dups_size': 0, 'ondisk_log_size': 10097, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1440, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 208896, 'num_objects': 51, 'num_object_clones': 0, 'num_object_copies': 102, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4373, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 51, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17252, 'num_write_kb': 27124, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4320}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:08:45.230 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-10T16:08:45.230 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "49'21156", 'reported_seq': 21310, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:08:41.143857+0000', 'last_change': '2024-09-10T15:59:16.313409+0000', 'last_active': '2024-09-10T16:08:41.143857+0000', 'last_peered': '2024-09-10T16:08:41.143857+0000', 'last_clean': '2024-09-10T15:59:14.129780+0000', 'last_became_active': '2024-09-10T15:59:16.300932+0000', 'last_became_peered': '2024-09-10T15:59:16.300932+0000', 'last_unstale': '2024-09-10T16:08:41.143857+0000', 'last_undegraded': '2024-09-10T15:59:15.887823+0000', 'last_fullsized': '2024-09-10T15:59:15.768513+0000', 'mapping_epoch': 42, 'log_start': "45'11100", 'ondisk_log_start': "45'11100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3880", 'last_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'objects_scrubbed': 3827, 'log_size': 10056, 'log_dups_size': 0, 'ondisk_log_size': 10056, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1397, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 172998, 'num_objects': 44, 'num_object_clones': 0, 'num_object_copies': 88, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4325, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 44, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17319, 'num_write_kb': 27072, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4279}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:08:45.231 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T16:08:45.645 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:08:45.645 INFO:teuthology.orchestra.run.smithi115.stdout:{"fsid":"7effee29-df05-4351-9741-ee203564ed81","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 38071/1594 objects degraded (2388.394%), 9 pgs degraded, 9 pgs undersized","count":18},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":689,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":49,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983954,"num_in_osds":3,"osd_in_since":1725983947,"num_remapped_pgs":9},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":8},{"state_name":"active+recovery_wait","count":1},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":797,"data_bytes":3216137,"bytes_used":1545728000,"bytes_avail":288364564480,"bytes_total":289910292480,"degraded_objects":38071,"degraded_total":1594,"degraded_ratio":23.883939774153074,"recovering_objects_per_sec":11,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":142,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":391},"fsmap":{"epoch":1,"btime":"2024-09-10T15:57:14:967120+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T16:00:13.560776+0000","services":{}},"progress_events":{}} 2024-09-10T16:08:45.659 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-10T16:08:45.660 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:08:46.006 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:08:46.006 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:08:46.020 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":387,"stamp":"2024-09-10T16:08:45.765373+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2650865,"num_objects":658,"num_object_clones":0,"num_object_copies":1316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38071,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":658,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":291452,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13321,"num_bytes_recovered":33095699,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160879,"ondisk_log_size":160879,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1493648,"kb_used_data":211612,"kb_used_omap":12,"kb_used_meta":1281971,"kb_avail":281621872,"statfs":{"total":289910292480,"available":288380796928,"internally_reserved":0,"allocated":216690688,"data_stored":215733058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1312738306},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,2,9,3,7,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19820568,"num_objects":-4840,"num_object_clones":0,"num_object_copies":-9680,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4840,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4840,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":142,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003476"},"pg_stats":[{"pgid":"2.f","version":"49'20635","reported_seq":20720,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:41.188030+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:08:41.188030+0000","last_peered":"2024-09-10T16:08:41.188030+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:08:41.188030+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":151552,"num_objects":37,"num_object_clones":0,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20635,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4222}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20787","reported_seq":20910,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:41.002992+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:08:41.002992+0000","last_peered":"2024-09-10T16:08:41.002992+0000","last_clean":"2024-09-10T16:08:41.002992+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:08:41.002992+0000","last_undegraded":"2024-09-10T16:08:41.002992+0000","last_fullsized":"2024-09-10T16:08:41.002992+0000","mapping_epoch":35,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":234438,"num_objects":59,"num_object_clones":0,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18892,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20798","reported_seq":29486,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:42.665212+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:08:42.665212+0000","last_peered":"2024-09-10T16:08:42.665212+0000","last_clean":"2024-09-10T16:08:42.665212+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:08:42.665212+0000","last_undegraded":"2024-09-10T16:08:42.665212+0000","last_fullsized":"2024-09-10T16:08:42.665212+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18802,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20613","reported_seq":20704,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:40.884047+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:08:40.884047+0000","last_peered":"2024-09-10T16:08:40.884047+0000","last_clean":"2024-09-10T16:08:40.884047+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:08:40.884047+0000","last_undegraded":"2024-09-10T16:08:40.884047+0000","last_fullsized":"2024-09-10T16:08:40.884047+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":192512,"num_objects":47,"num_object_clones":0,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20613,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20842","reported_seq":20982,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:41.136021+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:08:41.136021+0000","last_peered":"2024-09-10T16:08:41.136021+0000","last_clean":"2024-09-10T16:08:41.136021+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:08:41.136021+0000","last_undegraded":"2024-09-10T16:08:41.136021+0000","last_fullsized":"2024-09-10T16:08:41.136021+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":201187,"num_objects":50,"num_object_clones":0,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17553,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'21010","reported_seq":21158,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:41.132237+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:08:41.132237+0000","last_peered":"2024-09-10T16:08:41.132237+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:08:41.132237+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":209379,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17622,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4319},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'21041","reported_seq":21189,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:41.145052+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:08:41.145052+0000","last_peered":"2024-09-10T16:08:41.145052+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:08:41.145052+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":168902,"num_objects":43,"num_object_clones":0,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17540,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4194},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'21273","reported_seq":21413,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:41.149656+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:08:41.149656+0000","last_peered":"2024-09-10T16:08:41.149656+0000","last_clean":"2024-09-10T16:08:41.149656+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:08:41.149656+0000","last_undegraded":"2024-09-10T16:08:41.149656+0000","last_fullsized":"2024-09-10T16:08:41.149656+0000","mapping_epoch":34,"log_start":"45'11200","ondisk_log_start":"45'11200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":167936,"num_objects":41,"num_object_clones":0,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18120,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.838857+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:08:10.838857+0000","last_peered":"2024-09-10T16:08:10.838857+0000","last_clean":"2024-09-10T16:08:10.838857+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:08:10.838857+0000","last_undegraded":"2024-09-10T16:08:10.838857+0000","last_fullsized":"2024-09-10T16:08:10.838857+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20410","reported_seq":20560,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:41.128326+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:08:41.128326+0000","last_peered":"2024-09-10T16:08:41.128326+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:08:41.128326+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":192995,"num_objects":48,"num_object_clones":0,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18229,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-4075}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":104,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.838197+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:08:10.838197+0000","last_peered":"2024-09-10T16:08:10.838197+0000","last_clean":"2024-09-10T16:08:10.838197+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:08:10.838197+0000","last_undegraded":"2024-09-10T16:08:10.838197+0000","last_fullsized":"2024-09-10T16:08:10.838197+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20857","reported_seq":21969,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:41.129625+0000","last_change":"2024-09-10T16:08:09.280891+0000","last_active":"2024-09-10T16:08:41.129625+0000","last_peered":"2024-09-10T16:08:41.129625+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:08:41.129625+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":217088,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3771,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17361,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":486,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3718}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":115,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.840312+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:08:10.840312+0000","last_peered":"2024-09-10T16:08:10.840312+0000","last_clean":"2024-09-10T16:08:10.840312+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:08:10.840312+0000","last_undegraded":"2024-09-10T16:08:10.840312+0000","last_fullsized":"2024-09-10T16:08:10.840312+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20736","reported_seq":20881,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:41.039501+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:08:41.039501+0000","last_peered":"2024-09-10T16:08:41.039501+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:08:41.039501+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":184803,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18416,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-4258}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":90,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.837937+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:08:10.837937+0000","last_peered":"2024-09-10T16:08:10.837937+0000","last_clean":"2024-09-10T16:08:10.837937+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:08:10.837937+0000","last_undegraded":"2024-09-10T16:08:10.837937+0000","last_fullsized":"2024-09-10T16:08:10.837937+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'21129","reported_seq":29984,"reported_epoch":49,"state":"active+recovery_wait","last_fresh":"2024-09-10T16:08:41.106117+0000","last_change":"2024-09-10T16:08:10.929169+0000","last_active":"2024-09-10T16:08:41.106117+0000","last_peered":"2024-09-10T16:08:41.106117+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T16:08:10.860438+0000","last_became_peered":"2024-09-10T16:08:10.860438+0000","last_unstale":"2024-09-10T16:08:41.106117+0000","last_undegraded":"2024-09-10T16:08:41.106117+0000","last_fullsized":"2024-09-10T16:08:41.106117+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":167936,"num_objects":41,"num_object_clones":0,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18683,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","0","1"],"object_location_counts":[{"shards":"1,2","objects":41}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20794","reported_seq":20939,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:40.940386+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:08:40.940386+0000","last_peered":"2024-09-10T16:08:40.940386+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:08:40.940386+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180224,"num_objects":44,"num_object_clones":0,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17425,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4260}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.838546+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:08:10.838546+0000","last_peered":"2024-09-10T16:08:10.838546+0000","last_clean":"2024-09-10T16:08:10.838546+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:08:10.838546+0000","last_undegraded":"2024-09-10T16:08:10.838546+0000","last_fullsized":"2024-09-10T16:08:10.838546+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20598","reported_seq":29174,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:42.620274+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:08:42.620274+0000","last_peered":"2024-09-10T16:08:42.620274+0000","last_clean":"2024-09-10T16:08:42.620274+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:08:42.620274+0000","last_undegraded":"2024-09-10T16:08:42.620274+0000","last_fullsized":"2024-09-10T16:08:42.620274+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16988,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":104,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.838936+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:08:10.838936+0000","last_peered":"2024-09-10T16:08:10.838936+0000","last_clean":"2024-09-10T16:08:10.838936+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:08:10.838936+0000","last_undegraded":"2024-09-10T16:08:10.838936+0000","last_fullsized":"2024-09-10T16:08:10.838936+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20997","reported_seq":21149,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:41.089473+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:08:41.089473+0000","last_peered":"2024-09-10T16:08:41.089473+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:08:41.089473+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":208896,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17252,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4320}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.839114+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:08:10.839114+0000","last_peered":"2024-09-10T16:08:10.839114+0000","last_clean":"2024-09-10T16:08:10.839114+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:08:10.839114+0000","last_undegraded":"2024-09-10T16:08:10.839114+0000","last_fullsized":"2024-09-10T16:08:10.839114+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":85,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.838633+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:08:10.838633+0000","last_peered":"2024-09-10T16:08:10.838633+0000","last_clean":"2024-09-10T16:08:10.838633+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:08:10.838633+0000","last_undegraded":"2024-09-10T16:08:10.838633+0000","last_fullsized":"2024-09-10T16:08:10.838633+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'21156","reported_seq":21310,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:41.143857+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:08:41.143857+0000","last_peered":"2024-09-10T16:08:41.143857+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:08:41.143857+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":172998,"num_objects":44,"num_object_clones":0,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17319,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4279},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":2650846,"num_objects":656,"num_object_clones":0,"num_object_copies":1312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38071,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":656,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":291450,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13319,"num_bytes_recovered":33095680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":214253568,"data_stored":214123041,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160876,"ondisk_log_size":160876,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986304,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":500680,"kb_used_data":139960,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93871160,"statfs":{"total":96636764160,"available":96124067840,"internally_reserved":0,"allocated":143319040,"data_stored":143005855,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369356458},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,2,2,1,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659250,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":443104,"kb_used_data":464,"kb_used_omap":4,"kb_used_meta":442619,"kb_avail":93928736,"statfs":{"total":96636764160,"available":96183025664,"internally_reserved":0,"allocated":475136,"data_stored":143881,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":453242542},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182532,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":549864,"kb_used_data":71188,"kb_used_omap":4,"kb_used_meta":478651,"kb_avail":93821976,"statfs":{"total":96636764160,"available":96073703424,"internally_reserved":0,"allocated":72896512,"data_stored":72583322,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":490139306},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,7,2,5,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":72056832,"data_stored":72020702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":7245,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":142135296,"data_stored":142095094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:08:49.023 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:08:49.388 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:08:49.388 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:08:49.402 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":388,"stamp":"2024-09-10T16:08:47.765751+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37677,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292098,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13715,"num_bytes_recovered":33095699,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161025,"ondisk_log_size":161025,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1492988,"kb_used_data":206796,"kb_used_omap":12,"kb_used_meta":1286131,"kb_avail":281622532,"statfs":{"total":289910292480,"available":288381472768,"internally_reserved":0,"allocated":211759104,"data_stored":210805546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1316998146},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-21303320,"num_objects":-5202,"num_object_clones":0,"num_object_copies":-10404,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-536,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5202,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5202,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":536,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003290"},"pg_stats":[{"pgid":"2.f","version":"49'20672","reported_seq":20757,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:42.659151+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:08:42.659151+0000","last_peered":"2024-09-10T16:08:42.659151+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:08:42.659151+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20672,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4259}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20844","reported_seq":20967,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:42.591606+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:08:42.591606+0000","last_peered":"2024-09-10T16:08:42.591606+0000","last_clean":"2024-09-10T16:08:42.591606+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:08:42.591606+0000","last_undegraded":"2024-09-10T16:08:42.591606+0000","last_fullsized":"2024-09-10T16:08:42.591606+0000","mapping_epoch":35,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18949,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20798","reported_seq":29486,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:42.665212+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:08:42.665212+0000","last_peered":"2024-09-10T16:08:42.665212+0000","last_clean":"2024-09-10T16:08:42.665212+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:08:42.665212+0000","last_undegraded":"2024-09-10T16:08:42.665212+0000","last_fullsized":"2024-09-10T16:08:42.665212+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18802,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20660","reported_seq":20751,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:42.649157+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:08:42.649157+0000","last_peered":"2024-09-10T16:08:42.649157+0000","last_clean":"2024-09-10T16:08:42.649157+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:08:42.649157+0000","last_undegraded":"2024-09-10T16:08:42.649157+0000","last_fullsized":"2024-09-10T16:08:42.649157+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20660,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20891","reported_seq":21031,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:42.618630+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:08:42.618630+0000","last_peered":"2024-09-10T16:08:42.618630+0000","last_clean":"2024-09-10T16:08:42.618630+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:08:42.618630+0000","last_undegraded":"2024-09-10T16:08:42.618630+0000","last_fullsized":"2024-09-10T16:08:42.618630+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17602,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'21061","reported_seq":21209,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:42.603655+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:08:42.603655+0000","last_peered":"2024-09-10T16:08:42.603655+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:08:42.603655+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17673,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4370},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'21082","reported_seq":21230,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:42.545975+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:08:42.545975+0000","last_peered":"2024-09-10T16:08:42.545975+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:08:42.545975+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17581,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4235},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'21314","reported_seq":21454,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:42.602682+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:08:42.602682+0000","last_peered":"2024-09-10T16:08:42.602682+0000","last_clean":"2024-09-10T16:08:42.602682+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:08:42.602682+0000","last_undegraded":"2024-09-10T16:08:42.602682+0000","last_fullsized":"2024-09-10T16:08:42.602682+0000","mapping_epoch":34,"log_start":"45'11300","ondisk_log_start":"45'11300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18161,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.838857+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:08:10.838857+0000","last_peered":"2024-09-10T16:08:10.838857+0000","last_clean":"2024-09-10T16:08:10.838857+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:08:10.838857+0000","last_undegraded":"2024-09-10T16:08:10.838857+0000","last_fullsized":"2024-09-10T16:08:10.838857+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20457","reported_seq":20607,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:42.632156+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:08:42.632156+0000","last_peered":"2024-09-10T16:08:42.632156+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:08:42.632156+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18276,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-4122}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":105,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.838234+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:08:10.838234+0000","last_peered":"2024-09-10T16:08:10.838234+0000","last_clean":"2024-09-10T16:08:10.838234+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:08:10.838234+0000","last_undegraded":"2024-09-10T16:08:10.838234+0000","last_fullsized":"2024-09-10T16:08:10.838234+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20910","reported_seq":22812,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:46.196001+0000","last_change":"2024-09-10T16:08:09.280891+0000","last_active":"2024-09-10T16:08:46.196001+0000","last_peered":"2024-09-10T16:08:46.196001+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:08:46.196001+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3377,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17414,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":880,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3377}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":115,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.840312+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:08:10.840312+0000","last_peered":"2024-09-10T16:08:10.840312+0000","last_clean":"2024-09-10T16:08:10.840312+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:08:10.840312+0000","last_undegraded":"2024-09-10T16:08:10.840312+0000","last_fullsized":"2024-09-10T16:08:10.840312+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20781","reported_seq":20927,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:42.672320+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:08:42.672320+0000","last_peered":"2024-09-10T16:08:42.672320+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:08:42.672320+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18461,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-4303}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":90,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.837937+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:08:10.837937+0000","last_peered":"2024-09-10T16:08:10.837937+0000","last_clean":"2024-09-10T16:08:10.837937+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:08:10.837937+0000","last_undegraded":"2024-09-10T16:08:10.837937+0000","last_fullsized":"2024-09-10T16:08:10.837937+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'21170","reported_seq":30025,"reported_epoch":49,"state":"active+recovery_wait","last_fresh":"2024-09-10T16:08:42.596968+0000","last_change":"2024-09-10T16:08:10.929169+0000","last_active":"2024-09-10T16:08:42.596968+0000","last_peered":"2024-09-10T16:08:42.596968+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T16:08:10.860438+0000","last_became_peered":"2024-09-10T16:08:10.860438+0000","last_unstale":"2024-09-10T16:08:42.596968+0000","last_undegraded":"2024-09-10T16:08:42.596968+0000","last_fullsized":"2024-09-10T16:08:42.596968+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18724,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","0","1"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20838","reported_seq":20983,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:42.624374+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:08:42.624374+0000","last_peered":"2024-09-10T16:08:42.624374+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:08:42.624374+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17469,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.838546+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:08:10.838546+0000","last_peered":"2024-09-10T16:08:10.838546+0000","last_clean":"2024-09-10T16:08:10.838546+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:08:10.838546+0000","last_undegraded":"2024-09-10T16:08:10.838546+0000","last_fullsized":"2024-09-10T16:08:10.838546+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20598","reported_seq":29174,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:42.620274+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:08:42.620274+0000","last_peered":"2024-09-10T16:08:42.620274+0000","last_clean":"2024-09-10T16:08:42.620274+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:08:42.620274+0000","last_undegraded":"2024-09-10T16:08:42.620274+0000","last_fullsized":"2024-09-10T16:08:42.620274+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16988,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":105,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.838971+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:08:10.838971+0000","last_peered":"2024-09-10T16:08:10.838971+0000","last_clean":"2024-09-10T16:08:10.838971+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:08:10.838971+0000","last_undegraded":"2024-09-10T16:08:10.838971+0000","last_fullsized":"2024-09-10T16:08:10.838971+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'21048","reported_seq":21201,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:42.674593+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:08:42.674593+0000","last_peered":"2024-09-10T16:08:42.674593+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:08:42.674593+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17303,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.839114+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:08:10.839114+0000","last_peered":"2024-09-10T16:08:10.839114+0000","last_clean":"2024-09-10T16:08:10.839114+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:08:10.839114+0000","last_undegraded":"2024-09-10T16:08:10.839114+0000","last_fullsized":"2024-09-10T16:08:10.839114+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":85,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.838633+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:08:10.838633+0000","last_peered":"2024-09-10T16:08:10.838633+0000","last_clean":"2024-09-10T16:08:10.838633+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:08:10.838633+0000","last_undegraded":"2024-09-10T16:08:10.838633+0000","last_fullsized":"2024-09-10T16:08:10.838633+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'21198","reported_seq":21353,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:42.668456+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:08:42.668456+0000","last_peered":"2024-09-10T16:08:42.668456+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:08:42.668456+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17361,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4321},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37677,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292096,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13713,"num_bytes_recovered":33095680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":210522112,"data_stored":210395657,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161022,"ondisk_log_size":161022,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986305,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":502592,"kb_used_data":137712,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93869248,"statfs":{"total":96636764160,"available":96122109952,"internally_reserved":0,"allocated":141017088,"data_stored":140707975,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659250,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":443104,"kb_used_data":464,"kb_used_omap":4,"kb_used_meta":442619,"kb_avail":93928736,"statfs":{"total":96636764160,"available":96183025664,"internally_reserved":0,"allocated":475136,"data_stored":143881,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":453242542},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182533,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":547292,"kb_used_data":68620,"kb_used_omap":4,"kb_used_meta":478651,"kb_avail":93824548,"statfs":{"total":96636764160,"available":96076337152,"internally_reserved":0,"allocated":70266880,"data_stored":69953690,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":490139306},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":69853184,"data_stored":69817054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":7245,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":140607488,"data_stored":140571358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:08:49.403 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:08:49.706 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:08:49.706 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:08:49.719 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":388,"stamp":"2024-09-10T16:08:47.765751+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37677,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292098,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13715,"num_bytes_recovered":33095699,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161025,"ondisk_log_size":161025,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1492988,"kb_used_data":206796,"kb_used_omap":12,"kb_used_meta":1286131,"kb_avail":281622532,"statfs":{"total":289910292480,"available":288381472768,"internally_reserved":0,"allocated":211759104,"data_stored":210805546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1316998146},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-21303320,"num_objects":-5202,"num_object_clones":0,"num_object_copies":-10404,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-536,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5202,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5202,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":536,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003290"},"pg_stats":[{"pgid":"2.f","version":"49'20672","reported_seq":20757,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:42.659151+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:08:42.659151+0000","last_peered":"2024-09-10T16:08:42.659151+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:08:42.659151+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20672,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4259}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20844","reported_seq":20967,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:42.591606+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:08:42.591606+0000","last_peered":"2024-09-10T16:08:42.591606+0000","last_clean":"2024-09-10T16:08:42.591606+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:08:42.591606+0000","last_undegraded":"2024-09-10T16:08:42.591606+0000","last_fullsized":"2024-09-10T16:08:42.591606+0000","mapping_epoch":35,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18949,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20798","reported_seq":29486,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:42.665212+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:08:42.665212+0000","last_peered":"2024-09-10T16:08:42.665212+0000","last_clean":"2024-09-10T16:08:42.665212+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:08:42.665212+0000","last_undegraded":"2024-09-10T16:08:42.665212+0000","last_fullsized":"2024-09-10T16:08:42.665212+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18802,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20660","reported_seq":20751,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:42.649157+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:08:42.649157+0000","last_peered":"2024-09-10T16:08:42.649157+0000","last_clean":"2024-09-10T16:08:42.649157+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:08:42.649157+0000","last_undegraded":"2024-09-10T16:08:42.649157+0000","last_fullsized":"2024-09-10T16:08:42.649157+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20660,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20891","reported_seq":21031,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:42.618630+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:08:42.618630+0000","last_peered":"2024-09-10T16:08:42.618630+0000","last_clean":"2024-09-10T16:08:42.618630+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:08:42.618630+0000","last_undegraded":"2024-09-10T16:08:42.618630+0000","last_fullsized":"2024-09-10T16:08:42.618630+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17602,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'21061","reported_seq":21209,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:42.603655+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:08:42.603655+0000","last_peered":"2024-09-10T16:08:42.603655+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:08:42.603655+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17673,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4370},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'21082","reported_seq":21230,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:42.545975+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:08:42.545975+0000","last_peered":"2024-09-10T16:08:42.545975+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:08:42.545975+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17581,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4235},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'21314","reported_seq":21454,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:42.602682+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:08:42.602682+0000","last_peered":"2024-09-10T16:08:42.602682+0000","last_clean":"2024-09-10T16:08:42.602682+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:08:42.602682+0000","last_undegraded":"2024-09-10T16:08:42.602682+0000","last_fullsized":"2024-09-10T16:08:42.602682+0000","mapping_epoch":34,"log_start":"45'11300","ondisk_log_start":"45'11300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18161,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.838857+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:08:10.838857+0000","last_peered":"2024-09-10T16:08:10.838857+0000","last_clean":"2024-09-10T16:08:10.838857+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:08:10.838857+0000","last_undegraded":"2024-09-10T16:08:10.838857+0000","last_fullsized":"2024-09-10T16:08:10.838857+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20457","reported_seq":20607,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:42.632156+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:08:42.632156+0000","last_peered":"2024-09-10T16:08:42.632156+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:08:42.632156+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18276,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-4122}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":105,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.838234+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:08:10.838234+0000","last_peered":"2024-09-10T16:08:10.838234+0000","last_clean":"2024-09-10T16:08:10.838234+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:08:10.838234+0000","last_undegraded":"2024-09-10T16:08:10.838234+0000","last_fullsized":"2024-09-10T16:08:10.838234+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20910","reported_seq":22812,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:46.196001+0000","last_change":"2024-09-10T16:08:09.280891+0000","last_active":"2024-09-10T16:08:46.196001+0000","last_peered":"2024-09-10T16:08:46.196001+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:08:46.196001+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3377,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17414,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":880,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3377}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":115,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.840312+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:08:10.840312+0000","last_peered":"2024-09-10T16:08:10.840312+0000","last_clean":"2024-09-10T16:08:10.840312+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:08:10.840312+0000","last_undegraded":"2024-09-10T16:08:10.840312+0000","last_fullsized":"2024-09-10T16:08:10.840312+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20781","reported_seq":20927,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:42.672320+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:08:42.672320+0000","last_peered":"2024-09-10T16:08:42.672320+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:08:42.672320+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18461,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-4303}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":90,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.837937+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:08:10.837937+0000","last_peered":"2024-09-10T16:08:10.837937+0000","last_clean":"2024-09-10T16:08:10.837937+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:08:10.837937+0000","last_undegraded":"2024-09-10T16:08:10.837937+0000","last_fullsized":"2024-09-10T16:08:10.837937+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'21170","reported_seq":30025,"reported_epoch":49,"state":"active+recovery_wait","last_fresh":"2024-09-10T16:08:42.596968+0000","last_change":"2024-09-10T16:08:10.929169+0000","last_active":"2024-09-10T16:08:42.596968+0000","last_peered":"2024-09-10T16:08:42.596968+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T16:08:10.860438+0000","last_became_peered":"2024-09-10T16:08:10.860438+0000","last_unstale":"2024-09-10T16:08:42.596968+0000","last_undegraded":"2024-09-10T16:08:42.596968+0000","last_fullsized":"2024-09-10T16:08:42.596968+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18724,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","0","1"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20838","reported_seq":20983,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:42.624374+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:08:42.624374+0000","last_peered":"2024-09-10T16:08:42.624374+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:08:42.624374+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17469,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.838546+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:08:10.838546+0000","last_peered":"2024-09-10T16:08:10.838546+0000","last_clean":"2024-09-10T16:08:10.838546+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:08:10.838546+0000","last_undegraded":"2024-09-10T16:08:10.838546+0000","last_fullsized":"2024-09-10T16:08:10.838546+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20598","reported_seq":29174,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:42.620274+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:08:42.620274+0000","last_peered":"2024-09-10T16:08:42.620274+0000","last_clean":"2024-09-10T16:08:42.620274+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:08:42.620274+0000","last_undegraded":"2024-09-10T16:08:42.620274+0000","last_fullsized":"2024-09-10T16:08:42.620274+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16988,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":105,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.838971+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:08:10.838971+0000","last_peered":"2024-09-10T16:08:10.838971+0000","last_clean":"2024-09-10T16:08:10.838971+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:08:10.838971+0000","last_undegraded":"2024-09-10T16:08:10.838971+0000","last_fullsized":"2024-09-10T16:08:10.838971+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'21048","reported_seq":21201,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:42.674593+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:08:42.674593+0000","last_peered":"2024-09-10T16:08:42.674593+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:08:42.674593+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17303,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.839114+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:08:10.839114+0000","last_peered":"2024-09-10T16:08:10.839114+0000","last_clean":"2024-09-10T16:08:10.839114+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:08:10.839114+0000","last_undegraded":"2024-09-10T16:08:10.839114+0000","last_fullsized":"2024-09-10T16:08:10.839114+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":85,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.838633+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:08:10.838633+0000","last_peered":"2024-09-10T16:08:10.838633+0000","last_clean":"2024-09-10T16:08:10.838633+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:08:10.838633+0000","last_undegraded":"2024-09-10T16:08:10.838633+0000","last_fullsized":"2024-09-10T16:08:10.838633+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'21198","reported_seq":21353,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:42.668456+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:08:42.668456+0000","last_peered":"2024-09-10T16:08:42.668456+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:08:42.668456+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17361,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4321},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37677,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292096,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13713,"num_bytes_recovered":33095680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":210522112,"data_stored":210395657,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161022,"ondisk_log_size":161022,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986305,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":502592,"kb_used_data":137712,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93869248,"statfs":{"total":96636764160,"available":96122109952,"internally_reserved":0,"allocated":141017088,"data_stored":140707975,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659250,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":443104,"kb_used_data":464,"kb_used_omap":4,"kb_used_meta":442619,"kb_avail":93928736,"statfs":{"total":96636764160,"available":96183025664,"internally_reserved":0,"allocated":475136,"data_stored":143881,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":453242542},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182533,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":547292,"kb_used_data":68620,"kb_used_omap":4,"kb_used_meta":478651,"kb_avail":93824548,"statfs":{"total":96636764160,"available":96076337152,"internally_reserved":0,"allocated":70266880,"data_stored":69953690,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":490139306},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":69853184,"data_stored":69817054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":7245,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":140607488,"data_stored":140571358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:08:49.720 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-10T16:08:49.720 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "49'20672", 'reported_seq': 20757, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:08:42.659151+0000', 'last_change': '2024-09-10T15:59:16.281558+0000', 'last_active': '2024-09-10T16:08:42.659151+0000', 'last_peered': '2024-09-10T16:08:42.659151+0000', 'last_clean': '2024-09-10T15:59:14.132456+0000', 'last_became_active': '2024-09-10T15:59:16.274407+0000', 'last_became_peered': '2024-09-10T15:59:16.274407+0000', 'last_unstale': '2024-09-10T16:08:42.659151+0000', 'last_undegraded': '2024-09-10T15:59:15.851279+0000', 'last_fullsized': '2024-09-10T15:59:15.774356+0000', 'mapping_epoch': 42, 'log_start': "45'10600", 'ondisk_log_start': "45'10600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'objects_scrubbed': 0, 'log_size': 10072, 'log_dups_size': 0, 'ondisk_log_size': 10072, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4259, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20672, 'num_write_kb': 41344, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4259}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:08:49.721 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-10T16:08:49.721 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "49'21061", 'reported_seq': 21209, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:08:42.603655+0000', 'last_change': '2024-09-10T15:59:16.308264+0000', 'last_active': '2024-09-10T16:08:42.603655+0000', 'last_peered': '2024-09-10T16:08:42.603655+0000', 'last_clean': '2024-09-10T15:59:14.111551+0000', 'last_became_active': '2024-09-10T15:59:16.300040+0000', 'last_became_peered': '2024-09-10T15:59:16.300040+0000', 'last_unstale': '2024-09-10T16:08:42.603655+0000', 'last_undegraded': '2024-09-10T15:59:15.878234+0000', 'last_fullsized': '2024-09-10T15:59:15.769328+0000', 'mapping_epoch': 42, 'log_start': "45'11000", 'ondisk_log_start': "45'11000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3442", 'last_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'objects_scrubbed': 3384, 'log_size': 10061, 'log_dups_size': 0, 'ondisk_log_size': 10061, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1292, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4372, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 17673, 'num_write_kb': 28580, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4370}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:08:49.721 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-10T16:08:49.721 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "49'21082", 'reported_seq': 21230, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:08:42.545975+0000', 'last_change': '2024-09-10T15:59:16.331899+0000', 'last_active': '2024-09-10T16:08:42.545975+0000', 'last_peered': '2024-09-10T16:08:42.545975+0000', 'last_clean': '2024-09-10T15:59:14.139436+0000', 'last_became_active': '2024-09-10T15:59:16.318715+0000', 'last_became_peered': '2024-09-10T15:59:16.318715+0000', 'last_unstale': '2024-09-10T16:08:42.545975+0000', 'last_undegraded': '2024-09-10T15:59:15.877524+0000', 'last_fullsized': '2024-09-10T15:59:15.769042+0000', 'mapping_epoch': 42, 'log_start': "45'11000", 'ondisk_log_start': "45'11000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3537", 'last_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'objects_scrubbed': 3497, 'log_size': 10082, 'log_dups_size': 0, 'ondisk_log_size': 10082, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1224, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4237, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 17581, 'num_write_kb': 28172, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4235}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:08:49.721 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-10T16:08:49.721 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "49'20457", 'reported_seq': 20607, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:08:42.632156+0000', 'last_change': '2024-09-10T15:59:16.175680+0000', 'last_active': '2024-09-10T16:08:42.632156+0000', 'last_peered': '2024-09-10T16:08:42.632156+0000', 'last_clean': '2024-09-10T15:59:14.088187+0000', 'last_became_active': '2024-09-10T15:59:16.165810+0000', 'last_became_peered': '2024-09-10T15:59:16.165810+0000', 'last_unstale': '2024-09-10T16:08:42.632156+0000', 'last_undegraded': '2024-09-10T15:59:15.822098+0000', 'last_fullsized': '2024-09-10T15:59:15.769123+0000', 'mapping_epoch': 42, 'log_start': "45'10400", 'ondisk_log_start': "45'10400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2218", 'last_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'objects_scrubbed': 2175, 'log_size': 10057, 'log_dups_size': 0, 'ondisk_log_size': 10057, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 825, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4124, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 18276, 'num_write_kb': 32204, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': -4122}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:08:49.721 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-10T16:08:49.721 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "49'20910", 'reported_seq': 22812, 'reported_epoch': 49, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:08:46.196001+0000', 'last_change': '2024-09-10T16:08:09.280891+0000', 'last_active': '2024-09-10T16:08:46.196001+0000', 'last_peered': '2024-09-10T16:08:46.196001+0000', 'last_clean': '2024-09-10T15:59:14.129081+0000', 'last_became_active': '2024-09-10T15:59:16.269835+0000', 'last_became_peered': '2024-09-10T15:59:16.269835+0000', 'last_unstale': '2024-09-10T16:08:46.196001+0000', 'last_undegraded': '2024-09-10T15:59:15.860857+0000', 'last_fullsized': '2024-09-10T15:59:15.771410+0000', 'mapping_epoch': 42, 'log_start': "45'10900", 'ondisk_log_start': "45'10900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3557", 'last_scrub_stamp': '2024-09-10T15:59:02.493678+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:02.493678+0000', 'objects_scrubbed': 3488, 'log_size': 10010, 'log_dups_size': 0, 'ondisk_log_size': 10010, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1472, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3377, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17414, 'num_write_kb': 27852, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 880, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3377}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:08:49.722 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-10T16:08:49.722 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "49'20781", 'reported_seq': 20927, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:08:42.672320+0000', 'last_change': '2024-09-10T15:59:16.284311+0000', 'last_active': '2024-09-10T16:08:42.672320+0000', 'last_peered': '2024-09-10T16:08:42.672320+0000', 'last_clean': '2024-09-10T15:59:14.116041+0000', 'last_became_active': '2024-09-10T15:59:16.274543+0000', 'last_became_peered': '2024-09-10T15:59:16.274543+0000', 'last_unstale': '2024-09-10T16:08:42.672320+0000', 'last_undegraded': '2024-09-10T15:59:15.812484+0000', 'last_fullsized': '2024-09-10T15:59:15.770885+0000', 'mapping_epoch': 42, 'log_start': "45'10700", 'ondisk_log_start': "45'10700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2356", 'last_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'objects_scrubbed': 2316, 'log_size': 10081, 'log_dups_size': 0, 'ondisk_log_size': 10081, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 783, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 18461, 'num_write_kb': 32292, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': -4303}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:08:49.722 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-10T16:08:49.722 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "49'21170", 'reported_seq': 30025, 'reported_epoch': 49, 'state': 'active+recovery_wait', 'last_fresh': '2024-09-10T16:08:42.596968+0000', 'last_change': '2024-09-10T16:08:10.929169+0000', 'last_active': '2024-09-10T16:08:42.596968+0000', 'last_peered': '2024-09-10T16:08:42.596968+0000', 'last_clean': '2024-09-10T15:59:14.110654+0000', 'last_became_active': '2024-09-10T16:08:10.860438+0000', 'last_became_peered': '2024-09-10T16:08:10.860438+0000', 'last_unstale': '2024-09-10T16:08:42.596968+0000', 'last_undegraded': '2024-09-10T16:08:42.596968+0000', 'last_fullsized': '2024-09-10T16:08:42.596968+0000', 'mapping_epoch': 48, 'log_start': "45'11100", 'ondisk_log_start': "45'11100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2486", 'last_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'objects_scrubbed': 2440, 'log_size': 10070, 'log_dups_size': 0, 'ondisk_log_size': 10070, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 792, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18724, 'num_write_kb': 32568, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 4355, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': ['2', '0', '1'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:08:49.722 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-10T16:08:49.722 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "49'20838", 'reported_seq': 20983, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:08:42.624374+0000', 'last_change': '2024-09-10T15:59:16.315641+0000', 'last_active': '2024-09-10T16:08:42.624374+0000', 'last_peered': '2024-09-10T16:08:42.624374+0000', 'last_clean': '2024-09-10T15:59:14.117001+0000', 'last_became_active': '2024-09-10T15:59:16.302245+0000', 'last_became_peered': '2024-09-10T15:59:16.302245+0000', 'last_unstale': '2024-09-10T16:08:42.624374+0000', 'last_undegraded': '2024-09-10T15:59:15.845012+0000', 'last_fullsized': '2024-09-10T15:59:15.770337+0000', 'mapping_epoch': 42, 'log_start': "45'10800", 'ondisk_log_start': "45'10800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3404", 'last_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'objects_scrubbed': 3367, 'log_size': 10038, 'log_dups_size': 0, 'ondisk_log_size': 10038, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 983, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17469, 'num_write_kb': 28204, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4304}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:08:49.722 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-10T16:08:49.722 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "49'21048", 'reported_seq': 21201, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:08:42.674593+0000', 'last_change': '2024-09-10T15:59:16.311291+0000', 'last_active': '2024-09-10T16:08:42.674593+0000', 'last_peered': '2024-09-10T16:08:42.674593+0000', 'last_clean': '2024-09-10T15:59:14.116363+0000', 'last_became_active': '2024-09-10T15:59:16.301244+0000', 'last_became_peered': '2024-09-10T15:59:16.301244+0000', 'last_unstale': '2024-09-10T16:08:42.674593+0000', 'last_undegraded': '2024-09-10T15:59:15.883434+0000', 'last_fullsized': '2024-09-10T15:59:15.772756+0000', 'mapping_epoch': 42, 'log_start': "45'11000", 'ondisk_log_start': "45'11000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3789", 'last_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'objects_scrubbed': 3741, 'log_size': 10048, 'log_dups_size': 0, 'ondisk_log_size': 10048, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1440, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4373, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17303, 'num_write_kb': 27124, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4371}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:08:49.722 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-10T16:08:49.722 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "49'21198", 'reported_seq': 21353, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:08:42.668456+0000', 'last_change': '2024-09-10T15:59:16.313409+0000', 'last_active': '2024-09-10T16:08:42.668456+0000', 'last_peered': '2024-09-10T16:08:42.668456+0000', 'last_clean': '2024-09-10T15:59:14.129780+0000', 'last_became_active': '2024-09-10T15:59:16.300932+0000', 'last_became_peered': '2024-09-10T15:59:16.300932+0000', 'last_unstale': '2024-09-10T16:08:42.668456+0000', 'last_undegraded': '2024-09-10T15:59:15.887823+0000', 'last_fullsized': '2024-09-10T15:59:15.768513+0000', 'mapping_epoch': 42, 'log_start': "45'11100", 'ondisk_log_start': "45'11100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3880", 'last_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'objects_scrubbed': 3827, 'log_size': 10098, 'log_dups_size': 0, 'ondisk_log_size': 10098, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1397, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4325, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17361, 'num_write_kb': 27072, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4321}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:08:49.723 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T16:08:50.163 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:08:50.164 INFO:teuthology.orchestra.run.smithi115.stdout:{"fsid":"7effee29-df05-4351-9741-ee203564ed81","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 37677/24 objects degraded (156987.500%), 9 pgs degraded, 9 pgs undersized","count":18},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":694,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":49,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983954,"num_in_osds":3,"osd_in_since":1725983947,"num_remapped_pgs":9},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":8},{"state_name":"active+recovery_wait","count":1},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":12,"data_bytes":4849,"bytes_used":1528819712,"bytes_avail":288381472768,"bytes_total":289910292480,"degraded_objects":37677,"degraded_total":24,"degraded_ratio":1569.875,"recovering_objects_per_sec":44,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":536,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":433},"fsmap":{"epoch":1,"btime":"2024-09-10T15:57:14:967120+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T16:00:13.560776+0000","services":{}},"progress_events":{}} 2024-09-10T16:08:50.177 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-10T16:08:50.178 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:08:50.481 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:08:50.482 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:08:50.494 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":389,"stamp":"2024-09-10T16:08:49.766164+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37677,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292098,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13715,"num_bytes_recovered":33095699,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161025,"ondisk_log_size":161025,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1492988,"kb_used_data":206796,"kb_used_omap":12,"kb_used_meta":1286131,"kb_avail":281622532,"statfs":{"total":289910292480,"available":288381472768,"internally_reserved":0,"allocated":211759104,"data_stored":210805546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1316998146},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-12640280,"num_objects":-3087,"num_object_clones":0,"num_object_copies":-6174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-461,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-3087,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3087,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":461,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002950"},"pg_stats":[{"pgid":"2.f","version":"49'20672","reported_seq":20757,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:42.659151+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:08:42.659151+0000","last_peered":"2024-09-10T16:08:42.659151+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:08:42.659151+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20672,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4259}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20844","reported_seq":20967,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:42.591606+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:08:42.591606+0000","last_peered":"2024-09-10T16:08:42.591606+0000","last_clean":"2024-09-10T16:08:42.591606+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:08:42.591606+0000","last_undegraded":"2024-09-10T16:08:42.591606+0000","last_fullsized":"2024-09-10T16:08:42.591606+0000","mapping_epoch":35,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18949,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20798","reported_seq":29486,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:42.665212+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:08:42.665212+0000","last_peered":"2024-09-10T16:08:42.665212+0000","last_clean":"2024-09-10T16:08:42.665212+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:08:42.665212+0000","last_undegraded":"2024-09-10T16:08:42.665212+0000","last_fullsized":"2024-09-10T16:08:42.665212+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18802,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20660","reported_seq":20751,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:42.649157+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:08:42.649157+0000","last_peered":"2024-09-10T16:08:42.649157+0000","last_clean":"2024-09-10T16:08:42.649157+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:08:42.649157+0000","last_undegraded":"2024-09-10T16:08:42.649157+0000","last_fullsized":"2024-09-10T16:08:42.649157+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20660,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20891","reported_seq":21031,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:42.618630+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:08:42.618630+0000","last_peered":"2024-09-10T16:08:42.618630+0000","last_clean":"2024-09-10T16:08:42.618630+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:08:42.618630+0000","last_undegraded":"2024-09-10T16:08:42.618630+0000","last_fullsized":"2024-09-10T16:08:42.618630+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17602,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'21061","reported_seq":21209,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:42.603655+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:08:42.603655+0000","last_peered":"2024-09-10T16:08:42.603655+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:08:42.603655+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17673,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4370},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'21082","reported_seq":21230,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:42.545975+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:08:42.545975+0000","last_peered":"2024-09-10T16:08:42.545975+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:08:42.545975+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17581,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4235},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'21314","reported_seq":21454,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:42.602682+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:08:42.602682+0000","last_peered":"2024-09-10T16:08:42.602682+0000","last_clean":"2024-09-10T16:08:42.602682+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:08:42.602682+0000","last_undegraded":"2024-09-10T16:08:42.602682+0000","last_fullsized":"2024-09-10T16:08:42.602682+0000","mapping_epoch":34,"log_start":"45'11300","ondisk_log_start":"45'11300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18161,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.838857+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:08:10.838857+0000","last_peered":"2024-09-10T16:08:10.838857+0000","last_clean":"2024-09-10T16:08:10.838857+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:08:10.838857+0000","last_undegraded":"2024-09-10T16:08:10.838857+0000","last_fullsized":"2024-09-10T16:08:10.838857+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20457","reported_seq":20607,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:42.632156+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:08:42.632156+0000","last_peered":"2024-09-10T16:08:42.632156+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:08:42.632156+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18276,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-4122}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":105,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.838234+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:08:10.838234+0000","last_peered":"2024-09-10T16:08:10.838234+0000","last_clean":"2024-09-10T16:08:10.838234+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:08:10.838234+0000","last_undegraded":"2024-09-10T16:08:10.838234+0000","last_fullsized":"2024-09-10T16:08:10.838234+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20910","reported_seq":22812,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:46.196001+0000","last_change":"2024-09-10T16:08:09.280891+0000","last_active":"2024-09-10T16:08:46.196001+0000","last_peered":"2024-09-10T16:08:46.196001+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:08:46.196001+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3377,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17414,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":880,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3377}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":115,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.840312+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:08:10.840312+0000","last_peered":"2024-09-10T16:08:10.840312+0000","last_clean":"2024-09-10T16:08:10.840312+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:08:10.840312+0000","last_undegraded":"2024-09-10T16:08:10.840312+0000","last_fullsized":"2024-09-10T16:08:10.840312+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20781","reported_seq":20927,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:42.672320+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:08:42.672320+0000","last_peered":"2024-09-10T16:08:42.672320+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:08:42.672320+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18461,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-4303}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":90,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.837937+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:08:10.837937+0000","last_peered":"2024-09-10T16:08:10.837937+0000","last_clean":"2024-09-10T16:08:10.837937+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:08:10.837937+0000","last_undegraded":"2024-09-10T16:08:10.837937+0000","last_fullsized":"2024-09-10T16:08:10.837937+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'21170","reported_seq":30025,"reported_epoch":49,"state":"active+recovery_wait","last_fresh":"2024-09-10T16:08:42.596968+0000","last_change":"2024-09-10T16:08:10.929169+0000","last_active":"2024-09-10T16:08:42.596968+0000","last_peered":"2024-09-10T16:08:42.596968+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T16:08:10.860438+0000","last_became_peered":"2024-09-10T16:08:10.860438+0000","last_unstale":"2024-09-10T16:08:42.596968+0000","last_undegraded":"2024-09-10T16:08:42.596968+0000","last_fullsized":"2024-09-10T16:08:42.596968+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18724,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","0","1"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20838","reported_seq":20983,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:42.624374+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:08:42.624374+0000","last_peered":"2024-09-10T16:08:42.624374+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:08:42.624374+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17469,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.838546+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:08:10.838546+0000","last_peered":"2024-09-10T16:08:10.838546+0000","last_clean":"2024-09-10T16:08:10.838546+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:08:10.838546+0000","last_undegraded":"2024-09-10T16:08:10.838546+0000","last_fullsized":"2024-09-10T16:08:10.838546+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20598","reported_seq":29174,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:42.620274+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:08:42.620274+0000","last_peered":"2024-09-10T16:08:42.620274+0000","last_clean":"2024-09-10T16:08:42.620274+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:08:42.620274+0000","last_undegraded":"2024-09-10T16:08:42.620274+0000","last_fullsized":"2024-09-10T16:08:42.620274+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16988,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":105,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.838971+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:08:10.838971+0000","last_peered":"2024-09-10T16:08:10.838971+0000","last_clean":"2024-09-10T16:08:10.838971+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:08:10.838971+0000","last_undegraded":"2024-09-10T16:08:10.838971+0000","last_fullsized":"2024-09-10T16:08:10.838971+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'21048","reported_seq":21201,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:42.674593+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:08:42.674593+0000","last_peered":"2024-09-10T16:08:42.674593+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:08:42.674593+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17303,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.839114+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:08:10.839114+0000","last_peered":"2024-09-10T16:08:10.839114+0000","last_clean":"2024-09-10T16:08:10.839114+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:08:10.839114+0000","last_undegraded":"2024-09-10T16:08:10.839114+0000","last_fullsized":"2024-09-10T16:08:10.839114+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":85,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.838633+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:08:10.838633+0000","last_peered":"2024-09-10T16:08:10.838633+0000","last_clean":"2024-09-10T16:08:10.838633+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:08:10.838633+0000","last_undegraded":"2024-09-10T16:08:10.838633+0000","last_fullsized":"2024-09-10T16:08:10.838633+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'21198","reported_seq":21353,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:42.668456+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:08:42.668456+0000","last_peered":"2024-09-10T16:08:42.668456+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:08:42.668456+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17361,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4321},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37677,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292096,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13713,"num_bytes_recovered":33095680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":210522112,"data_stored":210395657,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161022,"ondisk_log_size":161022,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986305,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":502592,"kb_used_data":137712,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93869248,"statfs":{"total":96636764160,"available":96122109952,"internally_reserved":0,"allocated":141017088,"data_stored":140707975,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659250,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":443104,"kb_used_data":464,"kb_used_omap":4,"kb_used_meta":442619,"kb_avail":93928736,"statfs":{"total":96636764160,"available":96183025664,"internally_reserved":0,"allocated":475136,"data_stored":143881,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":453242542},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182533,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":547292,"kb_used_data":68620,"kb_used_omap":4,"kb_used_meta":478651,"kb_avail":93824548,"statfs":{"total":96636764160,"available":96076337152,"internally_reserved":0,"allocated":70266880,"data_stored":69953690,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":490139306},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":69853184,"data_stored":69817054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":7245,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":140607488,"data_stored":140571358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:08:53.497 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:08:53.793 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:08:53.794 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:08:53.805 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":391,"stamp":"2024-09-10T16:08:53.766969+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37093,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292098,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14299,"num_bytes_recovered":33095699,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161025,"ondisk_log_size":161025,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1497148,"kb_used_data":206796,"kb_used_omap":12,"kb_used_meta":1290291,"kb_avail":281618372,"statfs":{"total":289910292480,"available":288377212928,"internally_reserved":0,"allocated":211759104,"data_stored":210805546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1321257986},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-3211288,"num_objects":-785,"num_object_clones":0,"num_object_copies":-1570,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-978,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-785,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":785,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":978,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002462"},"pg_stats":[{"pgid":"2.f","version":"49'20672","reported_seq":20757,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:42.659151+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:08:42.659151+0000","last_peered":"2024-09-10T16:08:42.659151+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:08:42.659151+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20672,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4259}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20844","reported_seq":20967,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:42.591606+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:08:42.591606+0000","last_peered":"2024-09-10T16:08:42.591606+0000","last_clean":"2024-09-10T16:08:42.591606+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:08:42.591606+0000","last_undegraded":"2024-09-10T16:08:42.591606+0000","last_fullsized":"2024-09-10T16:08:42.591606+0000","mapping_epoch":35,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18949,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20798","reported_seq":29486,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:42.665212+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:08:42.665212+0000","last_peered":"2024-09-10T16:08:42.665212+0000","last_clean":"2024-09-10T16:08:42.665212+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:08:42.665212+0000","last_undegraded":"2024-09-10T16:08:42.665212+0000","last_fullsized":"2024-09-10T16:08:42.665212+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18802,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20660","reported_seq":20751,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:42.649157+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:08:42.649157+0000","last_peered":"2024-09-10T16:08:42.649157+0000","last_clean":"2024-09-10T16:08:42.649157+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:08:42.649157+0000","last_undegraded":"2024-09-10T16:08:42.649157+0000","last_fullsized":"2024-09-10T16:08:42.649157+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20660,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20891","reported_seq":21031,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:42.618630+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:08:42.618630+0000","last_peered":"2024-09-10T16:08:42.618630+0000","last_clean":"2024-09-10T16:08:42.618630+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:08:42.618630+0000","last_undegraded":"2024-09-10T16:08:42.618630+0000","last_fullsized":"2024-09-10T16:08:42.618630+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17602,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'21061","reported_seq":21209,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:42.603655+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:08:42.603655+0000","last_peered":"2024-09-10T16:08:42.603655+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:08:42.603655+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17673,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4370},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'21082","reported_seq":21230,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:42.545975+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:08:42.545975+0000","last_peered":"2024-09-10T16:08:42.545975+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:08:42.545975+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17581,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4235},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'21314","reported_seq":21454,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:42.602682+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:08:42.602682+0000","last_peered":"2024-09-10T16:08:42.602682+0000","last_clean":"2024-09-10T16:08:42.602682+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:08:42.602682+0000","last_undegraded":"2024-09-10T16:08:42.602682+0000","last_fullsized":"2024-09-10T16:08:42.602682+0000","mapping_epoch":34,"log_start":"45'11300","ondisk_log_start":"45'11300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18161,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.838857+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:08:10.838857+0000","last_peered":"2024-09-10T16:08:10.838857+0000","last_clean":"2024-09-10T16:08:10.838857+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:08:10.838857+0000","last_undegraded":"2024-09-10T16:08:10.838857+0000","last_fullsized":"2024-09-10T16:08:10.838857+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20457","reported_seq":20607,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:42.632156+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:08:42.632156+0000","last_peered":"2024-09-10T16:08:42.632156+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:08:42.632156+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18276,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-4122}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":105,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.838234+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:08:10.838234+0000","last_peered":"2024-09-10T16:08:10.838234+0000","last_clean":"2024-09-10T16:08:10.838234+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:08:10.838234+0000","last_undegraded":"2024-09-10T16:08:10.838234+0000","last_fullsized":"2024-09-10T16:08:10.838234+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20910","reported_seq":23980,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:51.197674+0000","last_change":"2024-09-10T16:08:09.280891+0000","last_active":"2024-09-10T16:08:51.197674+0000","last_peered":"2024-09-10T16:08:51.197674+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:08:51.197674+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2793,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17414,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1464,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2793}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":115,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.840312+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:08:10.840312+0000","last_peered":"2024-09-10T16:08:10.840312+0000","last_clean":"2024-09-10T16:08:10.840312+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:08:10.840312+0000","last_undegraded":"2024-09-10T16:08:10.840312+0000","last_fullsized":"2024-09-10T16:08:10.840312+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20781","reported_seq":20927,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:42.672320+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:08:42.672320+0000","last_peered":"2024-09-10T16:08:42.672320+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:08:42.672320+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18461,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-4303}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":90,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.837937+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:08:10.837937+0000","last_peered":"2024-09-10T16:08:10.837937+0000","last_clean":"2024-09-10T16:08:10.837937+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:08:10.837937+0000","last_undegraded":"2024-09-10T16:08:10.837937+0000","last_fullsized":"2024-09-10T16:08:10.837937+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'21170","reported_seq":30025,"reported_epoch":49,"state":"active+recovery_wait","last_fresh":"2024-09-10T16:08:42.596968+0000","last_change":"2024-09-10T16:08:10.929169+0000","last_active":"2024-09-10T16:08:42.596968+0000","last_peered":"2024-09-10T16:08:42.596968+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T16:08:10.860438+0000","last_became_peered":"2024-09-10T16:08:10.860438+0000","last_unstale":"2024-09-10T16:08:42.596968+0000","last_undegraded":"2024-09-10T16:08:42.596968+0000","last_fullsized":"2024-09-10T16:08:42.596968+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18724,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","0","1"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20838","reported_seq":20983,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:42.624374+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:08:42.624374+0000","last_peered":"2024-09-10T16:08:42.624374+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:08:42.624374+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17469,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.838546+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:08:10.838546+0000","last_peered":"2024-09-10T16:08:10.838546+0000","last_clean":"2024-09-10T16:08:10.838546+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:08:10.838546+0000","last_undegraded":"2024-09-10T16:08:10.838546+0000","last_fullsized":"2024-09-10T16:08:10.838546+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20598","reported_seq":29174,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:42.620274+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:08:42.620274+0000","last_peered":"2024-09-10T16:08:42.620274+0000","last_clean":"2024-09-10T16:08:42.620274+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:08:42.620274+0000","last_undegraded":"2024-09-10T16:08:42.620274+0000","last_fullsized":"2024-09-10T16:08:42.620274+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16988,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":105,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.838971+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:08:10.838971+0000","last_peered":"2024-09-10T16:08:10.838971+0000","last_clean":"2024-09-10T16:08:10.838971+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:08:10.838971+0000","last_undegraded":"2024-09-10T16:08:10.838971+0000","last_fullsized":"2024-09-10T16:08:10.838971+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'21048","reported_seq":21201,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:42.674593+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:08:42.674593+0000","last_peered":"2024-09-10T16:08:42.674593+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:08:42.674593+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17303,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.839114+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:08:10.839114+0000","last_peered":"2024-09-10T16:08:10.839114+0000","last_clean":"2024-09-10T16:08:10.839114+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:08:10.839114+0000","last_undegraded":"2024-09-10T16:08:10.839114+0000","last_fullsized":"2024-09-10T16:08:10.839114+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":85,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.838633+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:08:10.838633+0000","last_peered":"2024-09-10T16:08:10.838633+0000","last_clean":"2024-09-10T16:08:10.838633+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:08:10.838633+0000","last_undegraded":"2024-09-10T16:08:10.838633+0000","last_fullsized":"2024-09-10T16:08:10.838633+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'21198","reported_seq":21353,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:42.668456+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:08:42.668456+0000","last_peered":"2024-09-10T16:08:42.668456+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:08:42.668456+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17361,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4321},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37093,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292096,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14297,"num_bytes_recovered":33095680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":210522112,"data_stored":210395657,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161022,"ondisk_log_size":161022,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986306,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":502592,"kb_used_data":137712,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93869248,"statfs":{"total":96636764160,"available":96122109952,"internally_reserved":0,"allocated":141017088,"data_stored":140707975,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659251,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":447264,"kb_used_data":464,"kb_used_omap":4,"kb_used_meta":446779,"kb_avail":93924576,"statfs":{"total":96636764160,"available":96178765824,"internally_reserved":0,"allocated":475136,"data_stored":143881,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":457502382},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182534,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":547292,"kb_used_data":68620,"kb_used_omap":4,"kb_used_meta":478651,"kb_avail":93824548,"statfs":{"total":96636764160,"available":96076337152,"internally_reserved":0,"allocated":70266880,"data_stored":69953690,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":490139306},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":69853184,"data_stored":69817054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":7245,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":140607488,"data_stored":140571358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:08:53.807 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:08:54.109 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:08:54.109 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:08:54.121 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":391,"stamp":"2024-09-10T16:08:53.766969+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37093,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292098,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14299,"num_bytes_recovered":33095699,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161025,"ondisk_log_size":161025,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1497148,"kb_used_data":206796,"kb_used_omap":12,"kb_used_meta":1290291,"kb_avail":281618372,"statfs":{"total":289910292480,"available":288377212928,"internally_reserved":0,"allocated":211759104,"data_stored":210805546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1321257986},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-3211288,"num_objects":-785,"num_object_clones":0,"num_object_copies":-1570,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-978,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-785,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":785,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":978,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002462"},"pg_stats":[{"pgid":"2.f","version":"49'20672","reported_seq":20757,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:42.659151+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:08:42.659151+0000","last_peered":"2024-09-10T16:08:42.659151+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:08:42.659151+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20672,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4259}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20844","reported_seq":20967,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:42.591606+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:08:42.591606+0000","last_peered":"2024-09-10T16:08:42.591606+0000","last_clean":"2024-09-10T16:08:42.591606+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:08:42.591606+0000","last_undegraded":"2024-09-10T16:08:42.591606+0000","last_fullsized":"2024-09-10T16:08:42.591606+0000","mapping_epoch":35,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18949,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20798","reported_seq":29486,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:42.665212+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:08:42.665212+0000","last_peered":"2024-09-10T16:08:42.665212+0000","last_clean":"2024-09-10T16:08:42.665212+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:08:42.665212+0000","last_undegraded":"2024-09-10T16:08:42.665212+0000","last_fullsized":"2024-09-10T16:08:42.665212+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18802,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20660","reported_seq":20751,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:42.649157+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:08:42.649157+0000","last_peered":"2024-09-10T16:08:42.649157+0000","last_clean":"2024-09-10T16:08:42.649157+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:08:42.649157+0000","last_undegraded":"2024-09-10T16:08:42.649157+0000","last_fullsized":"2024-09-10T16:08:42.649157+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20660,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20891","reported_seq":21031,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:42.618630+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:08:42.618630+0000","last_peered":"2024-09-10T16:08:42.618630+0000","last_clean":"2024-09-10T16:08:42.618630+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:08:42.618630+0000","last_undegraded":"2024-09-10T16:08:42.618630+0000","last_fullsized":"2024-09-10T16:08:42.618630+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17602,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'21061","reported_seq":21209,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:42.603655+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:08:42.603655+0000","last_peered":"2024-09-10T16:08:42.603655+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:08:42.603655+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17673,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4370},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'21082","reported_seq":21230,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:42.545975+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:08:42.545975+0000","last_peered":"2024-09-10T16:08:42.545975+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:08:42.545975+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17581,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4235},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'21314","reported_seq":21454,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:42.602682+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:08:42.602682+0000","last_peered":"2024-09-10T16:08:42.602682+0000","last_clean":"2024-09-10T16:08:42.602682+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:08:42.602682+0000","last_undegraded":"2024-09-10T16:08:42.602682+0000","last_fullsized":"2024-09-10T16:08:42.602682+0000","mapping_epoch":34,"log_start":"45'11300","ondisk_log_start":"45'11300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18161,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.838857+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:08:10.838857+0000","last_peered":"2024-09-10T16:08:10.838857+0000","last_clean":"2024-09-10T16:08:10.838857+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:08:10.838857+0000","last_undegraded":"2024-09-10T16:08:10.838857+0000","last_fullsized":"2024-09-10T16:08:10.838857+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20457","reported_seq":20607,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:42.632156+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:08:42.632156+0000","last_peered":"2024-09-10T16:08:42.632156+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:08:42.632156+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18276,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-4122}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":105,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.838234+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:08:10.838234+0000","last_peered":"2024-09-10T16:08:10.838234+0000","last_clean":"2024-09-10T16:08:10.838234+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:08:10.838234+0000","last_undegraded":"2024-09-10T16:08:10.838234+0000","last_fullsized":"2024-09-10T16:08:10.838234+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20910","reported_seq":23980,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:51.197674+0000","last_change":"2024-09-10T16:08:09.280891+0000","last_active":"2024-09-10T16:08:51.197674+0000","last_peered":"2024-09-10T16:08:51.197674+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:08:51.197674+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2793,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17414,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1464,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2793}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":115,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.840312+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:08:10.840312+0000","last_peered":"2024-09-10T16:08:10.840312+0000","last_clean":"2024-09-10T16:08:10.840312+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:08:10.840312+0000","last_undegraded":"2024-09-10T16:08:10.840312+0000","last_fullsized":"2024-09-10T16:08:10.840312+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20781","reported_seq":20927,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:42.672320+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:08:42.672320+0000","last_peered":"2024-09-10T16:08:42.672320+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:08:42.672320+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18461,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-4303}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":90,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.837937+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:08:10.837937+0000","last_peered":"2024-09-10T16:08:10.837937+0000","last_clean":"2024-09-10T16:08:10.837937+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:08:10.837937+0000","last_undegraded":"2024-09-10T16:08:10.837937+0000","last_fullsized":"2024-09-10T16:08:10.837937+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'21170","reported_seq":30025,"reported_epoch":49,"state":"active+recovery_wait","last_fresh":"2024-09-10T16:08:42.596968+0000","last_change":"2024-09-10T16:08:10.929169+0000","last_active":"2024-09-10T16:08:42.596968+0000","last_peered":"2024-09-10T16:08:42.596968+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T16:08:10.860438+0000","last_became_peered":"2024-09-10T16:08:10.860438+0000","last_unstale":"2024-09-10T16:08:42.596968+0000","last_undegraded":"2024-09-10T16:08:42.596968+0000","last_fullsized":"2024-09-10T16:08:42.596968+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18724,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","0","1"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20838","reported_seq":20983,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:42.624374+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:08:42.624374+0000","last_peered":"2024-09-10T16:08:42.624374+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:08:42.624374+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17469,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.838546+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:08:10.838546+0000","last_peered":"2024-09-10T16:08:10.838546+0000","last_clean":"2024-09-10T16:08:10.838546+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:08:10.838546+0000","last_undegraded":"2024-09-10T16:08:10.838546+0000","last_fullsized":"2024-09-10T16:08:10.838546+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20598","reported_seq":29174,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:42.620274+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:08:42.620274+0000","last_peered":"2024-09-10T16:08:42.620274+0000","last_clean":"2024-09-10T16:08:42.620274+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:08:42.620274+0000","last_undegraded":"2024-09-10T16:08:42.620274+0000","last_fullsized":"2024-09-10T16:08:42.620274+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16988,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":105,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.838971+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:08:10.838971+0000","last_peered":"2024-09-10T16:08:10.838971+0000","last_clean":"2024-09-10T16:08:10.838971+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:08:10.838971+0000","last_undegraded":"2024-09-10T16:08:10.838971+0000","last_fullsized":"2024-09-10T16:08:10.838971+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'21048","reported_seq":21201,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:42.674593+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:08:42.674593+0000","last_peered":"2024-09-10T16:08:42.674593+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:08:42.674593+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17303,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.839114+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:08:10.839114+0000","last_peered":"2024-09-10T16:08:10.839114+0000","last_clean":"2024-09-10T16:08:10.839114+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:08:10.839114+0000","last_undegraded":"2024-09-10T16:08:10.839114+0000","last_fullsized":"2024-09-10T16:08:10.839114+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":85,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.838633+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:08:10.838633+0000","last_peered":"2024-09-10T16:08:10.838633+0000","last_clean":"2024-09-10T16:08:10.838633+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:08:10.838633+0000","last_undegraded":"2024-09-10T16:08:10.838633+0000","last_fullsized":"2024-09-10T16:08:10.838633+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'21198","reported_seq":21353,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:42.668456+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:08:42.668456+0000","last_peered":"2024-09-10T16:08:42.668456+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:08:42.668456+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17361,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4321},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37093,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292096,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14297,"num_bytes_recovered":33095680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":210522112,"data_stored":210395657,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161022,"ondisk_log_size":161022,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986306,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":502592,"kb_used_data":137712,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93869248,"statfs":{"total":96636764160,"available":96122109952,"internally_reserved":0,"allocated":141017088,"data_stored":140707975,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659251,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":447264,"kb_used_data":464,"kb_used_omap":4,"kb_used_meta":446779,"kb_avail":93924576,"statfs":{"total":96636764160,"available":96178765824,"internally_reserved":0,"allocated":475136,"data_stored":143881,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":457502382},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182534,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":547292,"kb_used_data":68620,"kb_used_omap":4,"kb_used_meta":478651,"kb_avail":93824548,"statfs":{"total":96636764160,"available":96076337152,"internally_reserved":0,"allocated":70266880,"data_stored":69953690,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":490139306},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":69853184,"data_stored":69817054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":7245,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":140607488,"data_stored":140571358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:08:54.122 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-10T16:08:54.123 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "49'20672", 'reported_seq': 20757, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:08:42.659151+0000', 'last_change': '2024-09-10T15:59:16.281558+0000', 'last_active': '2024-09-10T16:08:42.659151+0000', 'last_peered': '2024-09-10T16:08:42.659151+0000', 'last_clean': '2024-09-10T15:59:14.132456+0000', 'last_became_active': '2024-09-10T15:59:16.274407+0000', 'last_became_peered': '2024-09-10T15:59:16.274407+0000', 'last_unstale': '2024-09-10T16:08:42.659151+0000', 'last_undegraded': '2024-09-10T15:59:15.851279+0000', 'last_fullsized': '2024-09-10T15:59:15.774356+0000', 'mapping_epoch': 42, 'log_start': "45'10600", 'ondisk_log_start': "45'10600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'objects_scrubbed': 0, 'log_size': 10072, 'log_dups_size': 0, 'ondisk_log_size': 10072, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4259, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20672, 'num_write_kb': 41344, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4259}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:08:54.123 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-10T16:08:54.123 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "49'21061", 'reported_seq': 21209, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:08:42.603655+0000', 'last_change': '2024-09-10T15:59:16.308264+0000', 'last_active': '2024-09-10T16:08:42.603655+0000', 'last_peered': '2024-09-10T16:08:42.603655+0000', 'last_clean': '2024-09-10T15:59:14.111551+0000', 'last_became_active': '2024-09-10T15:59:16.300040+0000', 'last_became_peered': '2024-09-10T15:59:16.300040+0000', 'last_unstale': '2024-09-10T16:08:42.603655+0000', 'last_undegraded': '2024-09-10T15:59:15.878234+0000', 'last_fullsized': '2024-09-10T15:59:15.769328+0000', 'mapping_epoch': 42, 'log_start': "45'11000", 'ondisk_log_start': "45'11000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3442", 'last_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'objects_scrubbed': 3384, 'log_size': 10061, 'log_dups_size': 0, 'ondisk_log_size': 10061, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1292, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4372, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 17673, 'num_write_kb': 28580, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4370}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:08:54.123 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-10T16:08:54.123 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "49'21082", 'reported_seq': 21230, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:08:42.545975+0000', 'last_change': '2024-09-10T15:59:16.331899+0000', 'last_active': '2024-09-10T16:08:42.545975+0000', 'last_peered': '2024-09-10T16:08:42.545975+0000', 'last_clean': '2024-09-10T15:59:14.139436+0000', 'last_became_active': '2024-09-10T15:59:16.318715+0000', 'last_became_peered': '2024-09-10T15:59:16.318715+0000', 'last_unstale': '2024-09-10T16:08:42.545975+0000', 'last_undegraded': '2024-09-10T15:59:15.877524+0000', 'last_fullsized': '2024-09-10T15:59:15.769042+0000', 'mapping_epoch': 42, 'log_start': "45'11000", 'ondisk_log_start': "45'11000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3537", 'last_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'objects_scrubbed': 3497, 'log_size': 10082, 'log_dups_size': 0, 'ondisk_log_size': 10082, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1224, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4237, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 17581, 'num_write_kb': 28172, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4235}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:08:54.123 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-10T16:08:54.123 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "49'20457", 'reported_seq': 20607, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:08:42.632156+0000', 'last_change': '2024-09-10T15:59:16.175680+0000', 'last_active': '2024-09-10T16:08:42.632156+0000', 'last_peered': '2024-09-10T16:08:42.632156+0000', 'last_clean': '2024-09-10T15:59:14.088187+0000', 'last_became_active': '2024-09-10T15:59:16.165810+0000', 'last_became_peered': '2024-09-10T15:59:16.165810+0000', 'last_unstale': '2024-09-10T16:08:42.632156+0000', 'last_undegraded': '2024-09-10T15:59:15.822098+0000', 'last_fullsized': '2024-09-10T15:59:15.769123+0000', 'mapping_epoch': 42, 'log_start': "45'10400", 'ondisk_log_start': "45'10400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2218", 'last_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'objects_scrubbed': 2175, 'log_size': 10057, 'log_dups_size': 0, 'ondisk_log_size': 10057, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 825, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4124, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 18276, 'num_write_kb': 32204, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': -4122}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:08:54.123 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-10T16:08:54.123 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "49'20910", 'reported_seq': 23980, 'reported_epoch': 49, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:08:51.197674+0000', 'last_change': '2024-09-10T16:08:09.280891+0000', 'last_active': '2024-09-10T16:08:51.197674+0000', 'last_peered': '2024-09-10T16:08:51.197674+0000', 'last_clean': '2024-09-10T15:59:14.129081+0000', 'last_became_active': '2024-09-10T15:59:16.269835+0000', 'last_became_peered': '2024-09-10T15:59:16.269835+0000', 'last_unstale': '2024-09-10T16:08:51.197674+0000', 'last_undegraded': '2024-09-10T15:59:15.860857+0000', 'last_fullsized': '2024-09-10T15:59:15.771410+0000', 'mapping_epoch': 42, 'log_start': "45'10900", 'ondisk_log_start': "45'10900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3557", 'last_scrub_stamp': '2024-09-10T15:59:02.493678+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:02.493678+0000', 'objects_scrubbed': 3488, 'log_size': 10010, 'log_dups_size': 0, 'ondisk_log_size': 10010, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1472, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2793, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17414, 'num_write_kb': 27852, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1464, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2793}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:08:54.124 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-10T16:08:54.124 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "49'20781", 'reported_seq': 20927, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:08:42.672320+0000', 'last_change': '2024-09-10T15:59:16.284311+0000', 'last_active': '2024-09-10T16:08:42.672320+0000', 'last_peered': '2024-09-10T16:08:42.672320+0000', 'last_clean': '2024-09-10T15:59:14.116041+0000', 'last_became_active': '2024-09-10T15:59:16.274543+0000', 'last_became_peered': '2024-09-10T15:59:16.274543+0000', 'last_unstale': '2024-09-10T16:08:42.672320+0000', 'last_undegraded': '2024-09-10T15:59:15.812484+0000', 'last_fullsized': '2024-09-10T15:59:15.770885+0000', 'mapping_epoch': 42, 'log_start': "45'10700", 'ondisk_log_start': "45'10700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2356", 'last_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'objects_scrubbed': 2316, 'log_size': 10081, 'log_dups_size': 0, 'ondisk_log_size': 10081, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 783, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 18461, 'num_write_kb': 32292, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': -4303}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:08:54.124 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-10T16:08:54.124 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "49'21170", 'reported_seq': 30025, 'reported_epoch': 49, 'state': 'active+recovery_wait', 'last_fresh': '2024-09-10T16:08:42.596968+0000', 'last_change': '2024-09-10T16:08:10.929169+0000', 'last_active': '2024-09-10T16:08:42.596968+0000', 'last_peered': '2024-09-10T16:08:42.596968+0000', 'last_clean': '2024-09-10T15:59:14.110654+0000', 'last_became_active': '2024-09-10T16:08:10.860438+0000', 'last_became_peered': '2024-09-10T16:08:10.860438+0000', 'last_unstale': '2024-09-10T16:08:42.596968+0000', 'last_undegraded': '2024-09-10T16:08:42.596968+0000', 'last_fullsized': '2024-09-10T16:08:42.596968+0000', 'mapping_epoch': 48, 'log_start': "45'11100", 'ondisk_log_start': "45'11100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2486", 'last_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'objects_scrubbed': 2440, 'log_size': 10070, 'log_dups_size': 0, 'ondisk_log_size': 10070, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 792, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18724, 'num_write_kb': 32568, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 4355, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': ['2', '0', '1'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:08:54.124 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-10T16:08:54.124 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "49'20838", 'reported_seq': 20983, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:08:42.624374+0000', 'last_change': '2024-09-10T15:59:16.315641+0000', 'last_active': '2024-09-10T16:08:42.624374+0000', 'last_peered': '2024-09-10T16:08:42.624374+0000', 'last_clean': '2024-09-10T15:59:14.117001+0000', 'last_became_active': '2024-09-10T15:59:16.302245+0000', 'last_became_peered': '2024-09-10T15:59:16.302245+0000', 'last_unstale': '2024-09-10T16:08:42.624374+0000', 'last_undegraded': '2024-09-10T15:59:15.845012+0000', 'last_fullsized': '2024-09-10T15:59:15.770337+0000', 'mapping_epoch': 42, 'log_start': "45'10800", 'ondisk_log_start': "45'10800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3404", 'last_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'objects_scrubbed': 3367, 'log_size': 10038, 'log_dups_size': 0, 'ondisk_log_size': 10038, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 983, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17469, 'num_write_kb': 28204, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4304}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:08:54.124 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-10T16:08:54.124 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "49'21048", 'reported_seq': 21201, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:08:42.674593+0000', 'last_change': '2024-09-10T15:59:16.311291+0000', 'last_active': '2024-09-10T16:08:42.674593+0000', 'last_peered': '2024-09-10T16:08:42.674593+0000', 'last_clean': '2024-09-10T15:59:14.116363+0000', 'last_became_active': '2024-09-10T15:59:16.301244+0000', 'last_became_peered': '2024-09-10T15:59:16.301244+0000', 'last_unstale': '2024-09-10T16:08:42.674593+0000', 'last_undegraded': '2024-09-10T15:59:15.883434+0000', 'last_fullsized': '2024-09-10T15:59:15.772756+0000', 'mapping_epoch': 42, 'log_start': "45'11000", 'ondisk_log_start': "45'11000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3789", 'last_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'objects_scrubbed': 3741, 'log_size': 10048, 'log_dups_size': 0, 'ondisk_log_size': 10048, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1440, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4373, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17303, 'num_write_kb': 27124, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4371}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:08:54.124 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-10T16:08:54.124 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "49'21198", 'reported_seq': 21353, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:08:42.668456+0000', 'last_change': '2024-09-10T15:59:16.313409+0000', 'last_active': '2024-09-10T16:08:42.668456+0000', 'last_peered': '2024-09-10T16:08:42.668456+0000', 'last_clean': '2024-09-10T15:59:14.129780+0000', 'last_became_active': '2024-09-10T15:59:16.300932+0000', 'last_became_peered': '2024-09-10T15:59:16.300932+0000', 'last_unstale': '2024-09-10T16:08:42.668456+0000', 'last_undegraded': '2024-09-10T15:59:15.887823+0000', 'last_fullsized': '2024-09-10T15:59:15.768513+0000', 'mapping_epoch': 42, 'log_start': "45'11100", 'ondisk_log_start': "45'11100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3880", 'last_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'objects_scrubbed': 3827, 'log_size': 10098, 'log_dups_size': 0, 'ondisk_log_size': 10098, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1397, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4325, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17361, 'num_write_kb': 27072, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4321}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:08:54.125 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T16:08:54.514 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:08:54.514 INFO:teuthology.orchestra.run.smithi115.stdout:{"fsid":"7effee29-df05-4351-9741-ee203564ed81","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 37093/24 objects degraded (154554.167%), 9 pgs degraded, 9 pgs undersized","count":18},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":698,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":49,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983954,"num_in_osds":3,"osd_in_since":1725983947,"num_remapped_pgs":9},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":8},{"state_name":"active+recovery_wait","count":1},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":12,"data_bytes":4849,"bytes_used":1533079552,"bytes_avail":288377212928,"bytes_total":289910292480,"degraded_objects":37093,"degraded_total":24,"degraded_ratio":1545.541666666667,"recovering_objects_per_sec":81,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":978,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":65},"fsmap":{"epoch":1,"btime":"2024-09-10T15:57:14:967120+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T16:00:13.560776+0000","services":{}},"progress_events":{}} 2024-09-10T16:08:54.529 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-10T16:08:54.529 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:08:54.834 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:08:54.835 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:08:54.849 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":391,"stamp":"2024-09-10T16:08:53.766969+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37093,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292098,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14299,"num_bytes_recovered":33095699,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161025,"ondisk_log_size":161025,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1497148,"kb_used_data":206796,"kb_used_omap":12,"kb_used_meta":1290291,"kb_avail":281618372,"statfs":{"total":289910292480,"available":288377212928,"internally_reserved":0,"allocated":211759104,"data_stored":210805546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1321257986},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-3211288,"num_objects":-785,"num_object_clones":0,"num_object_copies":-1570,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-978,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-785,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":785,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":978,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002462"},"pg_stats":[{"pgid":"2.f","version":"49'20672","reported_seq":20757,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:42.659151+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:08:42.659151+0000","last_peered":"2024-09-10T16:08:42.659151+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:08:42.659151+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20672,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4259}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20844","reported_seq":20967,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:42.591606+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:08:42.591606+0000","last_peered":"2024-09-10T16:08:42.591606+0000","last_clean":"2024-09-10T16:08:42.591606+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:08:42.591606+0000","last_undegraded":"2024-09-10T16:08:42.591606+0000","last_fullsized":"2024-09-10T16:08:42.591606+0000","mapping_epoch":35,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18949,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20798","reported_seq":29486,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:42.665212+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:08:42.665212+0000","last_peered":"2024-09-10T16:08:42.665212+0000","last_clean":"2024-09-10T16:08:42.665212+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:08:42.665212+0000","last_undegraded":"2024-09-10T16:08:42.665212+0000","last_fullsized":"2024-09-10T16:08:42.665212+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18802,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20660","reported_seq":20751,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:42.649157+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:08:42.649157+0000","last_peered":"2024-09-10T16:08:42.649157+0000","last_clean":"2024-09-10T16:08:42.649157+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:08:42.649157+0000","last_undegraded":"2024-09-10T16:08:42.649157+0000","last_fullsized":"2024-09-10T16:08:42.649157+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20660,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20891","reported_seq":21031,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:42.618630+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:08:42.618630+0000","last_peered":"2024-09-10T16:08:42.618630+0000","last_clean":"2024-09-10T16:08:42.618630+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:08:42.618630+0000","last_undegraded":"2024-09-10T16:08:42.618630+0000","last_fullsized":"2024-09-10T16:08:42.618630+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17602,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'21061","reported_seq":21209,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:42.603655+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:08:42.603655+0000","last_peered":"2024-09-10T16:08:42.603655+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:08:42.603655+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17673,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4370},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'21082","reported_seq":21230,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:42.545975+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:08:42.545975+0000","last_peered":"2024-09-10T16:08:42.545975+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:08:42.545975+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17581,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4235},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'21314","reported_seq":21454,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:42.602682+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:08:42.602682+0000","last_peered":"2024-09-10T16:08:42.602682+0000","last_clean":"2024-09-10T16:08:42.602682+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:08:42.602682+0000","last_undegraded":"2024-09-10T16:08:42.602682+0000","last_fullsized":"2024-09-10T16:08:42.602682+0000","mapping_epoch":34,"log_start":"45'11300","ondisk_log_start":"45'11300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18161,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.838857+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:08:10.838857+0000","last_peered":"2024-09-10T16:08:10.838857+0000","last_clean":"2024-09-10T16:08:10.838857+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:08:10.838857+0000","last_undegraded":"2024-09-10T16:08:10.838857+0000","last_fullsized":"2024-09-10T16:08:10.838857+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20457","reported_seq":20607,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:42.632156+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:08:42.632156+0000","last_peered":"2024-09-10T16:08:42.632156+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:08:42.632156+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18276,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-4122}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":105,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.838234+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:08:10.838234+0000","last_peered":"2024-09-10T16:08:10.838234+0000","last_clean":"2024-09-10T16:08:10.838234+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:08:10.838234+0000","last_undegraded":"2024-09-10T16:08:10.838234+0000","last_fullsized":"2024-09-10T16:08:10.838234+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20910","reported_seq":23980,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:51.197674+0000","last_change":"2024-09-10T16:08:09.280891+0000","last_active":"2024-09-10T16:08:51.197674+0000","last_peered":"2024-09-10T16:08:51.197674+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:08:51.197674+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2793,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17414,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1464,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2793}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":115,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.840312+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:08:10.840312+0000","last_peered":"2024-09-10T16:08:10.840312+0000","last_clean":"2024-09-10T16:08:10.840312+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:08:10.840312+0000","last_undegraded":"2024-09-10T16:08:10.840312+0000","last_fullsized":"2024-09-10T16:08:10.840312+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20781","reported_seq":20927,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:42.672320+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:08:42.672320+0000","last_peered":"2024-09-10T16:08:42.672320+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:08:42.672320+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18461,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-4303}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":90,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.837937+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:08:10.837937+0000","last_peered":"2024-09-10T16:08:10.837937+0000","last_clean":"2024-09-10T16:08:10.837937+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:08:10.837937+0000","last_undegraded":"2024-09-10T16:08:10.837937+0000","last_fullsized":"2024-09-10T16:08:10.837937+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'21170","reported_seq":30025,"reported_epoch":49,"state":"active+recovery_wait","last_fresh":"2024-09-10T16:08:42.596968+0000","last_change":"2024-09-10T16:08:10.929169+0000","last_active":"2024-09-10T16:08:42.596968+0000","last_peered":"2024-09-10T16:08:42.596968+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T16:08:10.860438+0000","last_became_peered":"2024-09-10T16:08:10.860438+0000","last_unstale":"2024-09-10T16:08:42.596968+0000","last_undegraded":"2024-09-10T16:08:42.596968+0000","last_fullsized":"2024-09-10T16:08:42.596968+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18724,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","0","1"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20838","reported_seq":20983,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:42.624374+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:08:42.624374+0000","last_peered":"2024-09-10T16:08:42.624374+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:08:42.624374+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17469,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.838546+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:08:10.838546+0000","last_peered":"2024-09-10T16:08:10.838546+0000","last_clean":"2024-09-10T16:08:10.838546+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:08:10.838546+0000","last_undegraded":"2024-09-10T16:08:10.838546+0000","last_fullsized":"2024-09-10T16:08:10.838546+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20598","reported_seq":29174,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:42.620274+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:08:42.620274+0000","last_peered":"2024-09-10T16:08:42.620274+0000","last_clean":"2024-09-10T16:08:42.620274+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:08:42.620274+0000","last_undegraded":"2024-09-10T16:08:42.620274+0000","last_fullsized":"2024-09-10T16:08:42.620274+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16988,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":105,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.838971+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:08:10.838971+0000","last_peered":"2024-09-10T16:08:10.838971+0000","last_clean":"2024-09-10T16:08:10.838971+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:08:10.838971+0000","last_undegraded":"2024-09-10T16:08:10.838971+0000","last_fullsized":"2024-09-10T16:08:10.838971+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'21048","reported_seq":21201,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:42.674593+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:08:42.674593+0000","last_peered":"2024-09-10T16:08:42.674593+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:08:42.674593+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17303,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.839114+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:08:10.839114+0000","last_peered":"2024-09-10T16:08:10.839114+0000","last_clean":"2024-09-10T16:08:10.839114+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:08:10.839114+0000","last_undegraded":"2024-09-10T16:08:10.839114+0000","last_fullsized":"2024-09-10T16:08:10.839114+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":85,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.838633+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:08:10.838633+0000","last_peered":"2024-09-10T16:08:10.838633+0000","last_clean":"2024-09-10T16:08:10.838633+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:08:10.838633+0000","last_undegraded":"2024-09-10T16:08:10.838633+0000","last_fullsized":"2024-09-10T16:08:10.838633+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'21198","reported_seq":21353,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:42.668456+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:08:42.668456+0000","last_peered":"2024-09-10T16:08:42.668456+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:08:42.668456+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17361,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4321},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37093,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292096,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14297,"num_bytes_recovered":33095680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":210522112,"data_stored":210395657,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161022,"ondisk_log_size":161022,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986306,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":502592,"kb_used_data":137712,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93869248,"statfs":{"total":96636764160,"available":96122109952,"internally_reserved":0,"allocated":141017088,"data_stored":140707975,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659251,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":447264,"kb_used_data":464,"kb_used_omap":4,"kb_used_meta":446779,"kb_avail":93924576,"statfs":{"total":96636764160,"available":96178765824,"internally_reserved":0,"allocated":475136,"data_stored":143881,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":457502382},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182534,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":547292,"kb_used_data":68620,"kb_used_omap":4,"kb_used_meta":478651,"kb_avail":93824548,"statfs":{"total":96636764160,"available":96076337152,"internally_reserved":0,"allocated":70266880,"data_stored":69953690,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":490139306},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":69853184,"data_stored":69817054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":7245,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":140607488,"data_stored":140571358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:08:57.851 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:08:58.160 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:08:58.161 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:08:58.173 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":393,"stamp":"2024-09-10T16:08:57.767764+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292098,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14972,"num_bytes_recovered":33095699,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161025,"ondisk_log_size":161025,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1501308,"kb_used_data":206796,"kb_used_omap":12,"kb_used_meta":1294451,"kb_avail":281614212,"statfs":{"total":289910292480,"available":288372953088,"internally_reserved":0,"allocated":211759104,"data_stored":210805546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1325517826},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-2646016,"num_objects":-646,"num_object_clones":0,"num_object_copies":-1292,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1651,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-646,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":646,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1651,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002390"},"pg_stats":[{"pgid":"2.f","version":"49'20672","reported_seq":20757,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:42.659151+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:08:42.659151+0000","last_peered":"2024-09-10T16:08:42.659151+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:08:42.659151+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20672,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4259}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20844","reported_seq":20967,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:42.591606+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:08:42.591606+0000","last_peered":"2024-09-10T16:08:42.591606+0000","last_clean":"2024-09-10T16:08:42.591606+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:08:42.591606+0000","last_undegraded":"2024-09-10T16:08:42.591606+0000","last_fullsized":"2024-09-10T16:08:42.591606+0000","mapping_epoch":35,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18949,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20798","reported_seq":29486,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:42.665212+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:08:42.665212+0000","last_peered":"2024-09-10T16:08:42.665212+0000","last_clean":"2024-09-10T16:08:42.665212+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:08:42.665212+0000","last_undegraded":"2024-09-10T16:08:42.665212+0000","last_fullsized":"2024-09-10T16:08:42.665212+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18802,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20660","reported_seq":20751,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:42.649157+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:08:42.649157+0000","last_peered":"2024-09-10T16:08:42.649157+0000","last_clean":"2024-09-10T16:08:42.649157+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:08:42.649157+0000","last_undegraded":"2024-09-10T16:08:42.649157+0000","last_fullsized":"2024-09-10T16:08:42.649157+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20660,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20891","reported_seq":21031,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:42.618630+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:08:42.618630+0000","last_peered":"2024-09-10T16:08:42.618630+0000","last_clean":"2024-09-10T16:08:42.618630+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:08:42.618630+0000","last_undegraded":"2024-09-10T16:08:42.618630+0000","last_fullsized":"2024-09-10T16:08:42.618630+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17602,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'21061","reported_seq":21209,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:42.603655+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:08:42.603655+0000","last_peered":"2024-09-10T16:08:42.603655+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:08:42.603655+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17673,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4370},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'21082","reported_seq":21230,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:42.545975+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:08:42.545975+0000","last_peered":"2024-09-10T16:08:42.545975+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:08:42.545975+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17581,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4235},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'21314","reported_seq":21455,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:42.623054+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:08:42.623054+0000","last_peered":"2024-09-10T16:08:42.623054+0000","last_clean":"2024-09-10T16:08:42.623054+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:08:42.623054+0000","last_undegraded":"2024-09-10T16:08:42.623054+0000","last_fullsized":"2024-09-10T16:08:42.623054+0000","mapping_epoch":34,"log_start":"45'11300","ondisk_log_start":"45'11300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18161,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.838857+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:08:10.838857+0000","last_peered":"2024-09-10T16:08:10.838857+0000","last_clean":"2024-09-10T16:08:10.838857+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:08:10.838857+0000","last_undegraded":"2024-09-10T16:08:10.838857+0000","last_fullsized":"2024-09-10T16:08:10.838857+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20457","reported_seq":20608,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:42.662113+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:08:42.662113+0000","last_peered":"2024-09-10T16:08:42.662113+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:08:42.662113+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18276,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-4122}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":105,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.838234+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:08:10.838234+0000","last_peered":"2024-09-10T16:08:10.838234+0000","last_clean":"2024-09-10T16:08:10.838234+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:08:10.838234+0000","last_undegraded":"2024-09-10T16:08:10.838234+0000","last_fullsized":"2024-09-10T16:08:10.838234+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20910","reported_seq":25326,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:56.196825+0000","last_change":"2024-09-10T16:08:09.280891+0000","last_active":"2024-09-10T16:08:56.196825+0000","last_peered":"2024-09-10T16:08:56.196825+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:08:56.196825+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2120,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17414,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2137,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2120}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":115,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.840312+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:08:10.840312+0000","last_peered":"2024-09-10T16:08:10.840312+0000","last_clean":"2024-09-10T16:08:10.840312+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:08:10.840312+0000","last_undegraded":"2024-09-10T16:08:10.840312+0000","last_fullsized":"2024-09-10T16:08:10.840312+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20781","reported_seq":20927,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:42.672320+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:08:42.672320+0000","last_peered":"2024-09-10T16:08:42.672320+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:08:42.672320+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18461,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-4303}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":90,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.837937+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:08:10.837937+0000","last_peered":"2024-09-10T16:08:10.837937+0000","last_clean":"2024-09-10T16:08:10.837937+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:08:10.837937+0000","last_undegraded":"2024-09-10T16:08:10.837937+0000","last_fullsized":"2024-09-10T16:08:10.837937+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'21170","reported_seq":30025,"reported_epoch":49,"state":"active+recovery_wait","last_fresh":"2024-09-10T16:08:42.596968+0000","last_change":"2024-09-10T16:08:10.929169+0000","last_active":"2024-09-10T16:08:42.596968+0000","last_peered":"2024-09-10T16:08:42.596968+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T16:08:10.860438+0000","last_became_peered":"2024-09-10T16:08:10.860438+0000","last_unstale":"2024-09-10T16:08:42.596968+0000","last_undegraded":"2024-09-10T16:08:42.596968+0000","last_fullsized":"2024-09-10T16:08:42.596968+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18724,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","0","1"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20838","reported_seq":20983,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:42.624374+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:08:42.624374+0000","last_peered":"2024-09-10T16:08:42.624374+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:08:42.624374+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17469,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.838546+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:08:10.838546+0000","last_peered":"2024-09-10T16:08:10.838546+0000","last_clean":"2024-09-10T16:08:10.838546+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:08:10.838546+0000","last_undegraded":"2024-09-10T16:08:10.838546+0000","last_fullsized":"2024-09-10T16:08:10.838546+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20598","reported_seq":29174,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:42.620274+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:08:42.620274+0000","last_peered":"2024-09-10T16:08:42.620274+0000","last_clean":"2024-09-10T16:08:42.620274+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:08:42.620274+0000","last_undegraded":"2024-09-10T16:08:42.620274+0000","last_fullsized":"2024-09-10T16:08:42.620274+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16988,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":105,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.838971+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:08:10.838971+0000","last_peered":"2024-09-10T16:08:10.838971+0000","last_clean":"2024-09-10T16:08:10.838971+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:08:10.838971+0000","last_undegraded":"2024-09-10T16:08:10.838971+0000","last_fullsized":"2024-09-10T16:08:10.838971+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'21048","reported_seq":21201,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:42.674593+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:08:42.674593+0000","last_peered":"2024-09-10T16:08:42.674593+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:08:42.674593+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17303,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.839114+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:08:10.839114+0000","last_peered":"2024-09-10T16:08:10.839114+0000","last_clean":"2024-09-10T16:08:10.839114+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:08:10.839114+0000","last_undegraded":"2024-09-10T16:08:10.839114+0000","last_fullsized":"2024-09-10T16:08:10.839114+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":85,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.838633+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:08:10.838633+0000","last_peered":"2024-09-10T16:08:10.838633+0000","last_clean":"2024-09-10T16:08:10.838633+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:08:10.838633+0000","last_undegraded":"2024-09-10T16:08:10.838633+0000","last_fullsized":"2024-09-10T16:08:10.838633+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'21198","reported_seq":21353,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:42.668456+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:08:42.668456+0000","last_peered":"2024-09-10T16:08:42.668456+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:08:42.668456+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17361,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4321},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292096,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14970,"num_bytes_recovered":33095680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":210522112,"data_stored":210395657,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161022,"ondisk_log_size":161022,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986307,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":502592,"kb_used_data":137712,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93869248,"statfs":{"total":96636764160,"available":96122109952,"internally_reserved":0,"allocated":141017088,"data_stored":140707975,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659252,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":451424,"kb_used_data":464,"kb_used_omap":4,"kb_used_meta":450939,"kb_avail":93920416,"statfs":{"total":96636764160,"available":96174505984,"internally_reserved":0,"allocated":475136,"data_stored":143881,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":461762222},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182535,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":547292,"kb_used_data":68620,"kb_used_omap":4,"kb_used_meta":478651,"kb_avail":93824548,"statfs":{"total":96636764160,"available":96076337152,"internally_reserved":0,"allocated":70266880,"data_stored":69953690,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":490139306},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":69853184,"data_stored":69817054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":7245,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":140607488,"data_stored":140571358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:08:58.174 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:08:58.529 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:08:58.529 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:08:58.541 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":393,"stamp":"2024-09-10T16:08:57.767764+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292098,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14972,"num_bytes_recovered":33095699,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161025,"ondisk_log_size":161025,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1501308,"kb_used_data":206796,"kb_used_omap":12,"kb_used_meta":1294451,"kb_avail":281614212,"statfs":{"total":289910292480,"available":288372953088,"internally_reserved":0,"allocated":211759104,"data_stored":210805546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1325517826},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-2646016,"num_objects":-646,"num_object_clones":0,"num_object_copies":-1292,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1651,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-646,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":646,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1651,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002390"},"pg_stats":[{"pgid":"2.f","version":"49'20672","reported_seq":20757,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:42.659151+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:08:42.659151+0000","last_peered":"2024-09-10T16:08:42.659151+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:08:42.659151+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20672,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4259}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20844","reported_seq":20967,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:42.591606+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:08:42.591606+0000","last_peered":"2024-09-10T16:08:42.591606+0000","last_clean":"2024-09-10T16:08:42.591606+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:08:42.591606+0000","last_undegraded":"2024-09-10T16:08:42.591606+0000","last_fullsized":"2024-09-10T16:08:42.591606+0000","mapping_epoch":35,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18949,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20798","reported_seq":29486,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:42.665212+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:08:42.665212+0000","last_peered":"2024-09-10T16:08:42.665212+0000","last_clean":"2024-09-10T16:08:42.665212+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:08:42.665212+0000","last_undegraded":"2024-09-10T16:08:42.665212+0000","last_fullsized":"2024-09-10T16:08:42.665212+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18802,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20660","reported_seq":20751,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:42.649157+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:08:42.649157+0000","last_peered":"2024-09-10T16:08:42.649157+0000","last_clean":"2024-09-10T16:08:42.649157+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:08:42.649157+0000","last_undegraded":"2024-09-10T16:08:42.649157+0000","last_fullsized":"2024-09-10T16:08:42.649157+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20660,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20891","reported_seq":21031,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:42.618630+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:08:42.618630+0000","last_peered":"2024-09-10T16:08:42.618630+0000","last_clean":"2024-09-10T16:08:42.618630+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:08:42.618630+0000","last_undegraded":"2024-09-10T16:08:42.618630+0000","last_fullsized":"2024-09-10T16:08:42.618630+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17602,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'21061","reported_seq":21209,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:42.603655+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:08:42.603655+0000","last_peered":"2024-09-10T16:08:42.603655+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:08:42.603655+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17673,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4370},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'21082","reported_seq":21230,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:42.545975+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:08:42.545975+0000","last_peered":"2024-09-10T16:08:42.545975+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:08:42.545975+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17581,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4235},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'21314","reported_seq":21455,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:42.623054+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:08:42.623054+0000","last_peered":"2024-09-10T16:08:42.623054+0000","last_clean":"2024-09-10T16:08:42.623054+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:08:42.623054+0000","last_undegraded":"2024-09-10T16:08:42.623054+0000","last_fullsized":"2024-09-10T16:08:42.623054+0000","mapping_epoch":34,"log_start":"45'11300","ondisk_log_start":"45'11300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18161,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.838857+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:08:10.838857+0000","last_peered":"2024-09-10T16:08:10.838857+0000","last_clean":"2024-09-10T16:08:10.838857+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:08:10.838857+0000","last_undegraded":"2024-09-10T16:08:10.838857+0000","last_fullsized":"2024-09-10T16:08:10.838857+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20457","reported_seq":20608,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:42.662113+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:08:42.662113+0000","last_peered":"2024-09-10T16:08:42.662113+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:08:42.662113+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18276,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-4122}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":105,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.838234+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:08:10.838234+0000","last_peered":"2024-09-10T16:08:10.838234+0000","last_clean":"2024-09-10T16:08:10.838234+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:08:10.838234+0000","last_undegraded":"2024-09-10T16:08:10.838234+0000","last_fullsized":"2024-09-10T16:08:10.838234+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20910","reported_seq":25326,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:56.196825+0000","last_change":"2024-09-10T16:08:09.280891+0000","last_active":"2024-09-10T16:08:56.196825+0000","last_peered":"2024-09-10T16:08:56.196825+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:08:56.196825+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2120,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17414,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2137,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2120}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":115,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.840312+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:08:10.840312+0000","last_peered":"2024-09-10T16:08:10.840312+0000","last_clean":"2024-09-10T16:08:10.840312+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:08:10.840312+0000","last_undegraded":"2024-09-10T16:08:10.840312+0000","last_fullsized":"2024-09-10T16:08:10.840312+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20781","reported_seq":20927,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:42.672320+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:08:42.672320+0000","last_peered":"2024-09-10T16:08:42.672320+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:08:42.672320+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18461,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-4303}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":90,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.837937+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:08:10.837937+0000","last_peered":"2024-09-10T16:08:10.837937+0000","last_clean":"2024-09-10T16:08:10.837937+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:08:10.837937+0000","last_undegraded":"2024-09-10T16:08:10.837937+0000","last_fullsized":"2024-09-10T16:08:10.837937+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'21170","reported_seq":30025,"reported_epoch":49,"state":"active+recovery_wait","last_fresh":"2024-09-10T16:08:42.596968+0000","last_change":"2024-09-10T16:08:10.929169+0000","last_active":"2024-09-10T16:08:42.596968+0000","last_peered":"2024-09-10T16:08:42.596968+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T16:08:10.860438+0000","last_became_peered":"2024-09-10T16:08:10.860438+0000","last_unstale":"2024-09-10T16:08:42.596968+0000","last_undegraded":"2024-09-10T16:08:42.596968+0000","last_fullsized":"2024-09-10T16:08:42.596968+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18724,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","0","1"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20838","reported_seq":20983,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:42.624374+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:08:42.624374+0000","last_peered":"2024-09-10T16:08:42.624374+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:08:42.624374+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17469,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.838546+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:08:10.838546+0000","last_peered":"2024-09-10T16:08:10.838546+0000","last_clean":"2024-09-10T16:08:10.838546+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:08:10.838546+0000","last_undegraded":"2024-09-10T16:08:10.838546+0000","last_fullsized":"2024-09-10T16:08:10.838546+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20598","reported_seq":29174,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:42.620274+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:08:42.620274+0000","last_peered":"2024-09-10T16:08:42.620274+0000","last_clean":"2024-09-10T16:08:42.620274+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:08:42.620274+0000","last_undegraded":"2024-09-10T16:08:42.620274+0000","last_fullsized":"2024-09-10T16:08:42.620274+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16988,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":105,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.838971+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:08:10.838971+0000","last_peered":"2024-09-10T16:08:10.838971+0000","last_clean":"2024-09-10T16:08:10.838971+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:08:10.838971+0000","last_undegraded":"2024-09-10T16:08:10.838971+0000","last_fullsized":"2024-09-10T16:08:10.838971+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'21048","reported_seq":21201,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:42.674593+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:08:42.674593+0000","last_peered":"2024-09-10T16:08:42.674593+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:08:42.674593+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17303,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.839114+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:08:10.839114+0000","last_peered":"2024-09-10T16:08:10.839114+0000","last_clean":"2024-09-10T16:08:10.839114+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:08:10.839114+0000","last_undegraded":"2024-09-10T16:08:10.839114+0000","last_fullsized":"2024-09-10T16:08:10.839114+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":85,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.838633+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:08:10.838633+0000","last_peered":"2024-09-10T16:08:10.838633+0000","last_clean":"2024-09-10T16:08:10.838633+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:08:10.838633+0000","last_undegraded":"2024-09-10T16:08:10.838633+0000","last_fullsized":"2024-09-10T16:08:10.838633+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'21198","reported_seq":21353,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:42.668456+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:08:42.668456+0000","last_peered":"2024-09-10T16:08:42.668456+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:08:42.668456+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17361,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4321},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292096,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14970,"num_bytes_recovered":33095680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":210522112,"data_stored":210395657,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161022,"ondisk_log_size":161022,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986307,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":502592,"kb_used_data":137712,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93869248,"statfs":{"total":96636764160,"available":96122109952,"internally_reserved":0,"allocated":141017088,"data_stored":140707975,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659252,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":451424,"kb_used_data":464,"kb_used_omap":4,"kb_used_meta":450939,"kb_avail":93920416,"statfs":{"total":96636764160,"available":96174505984,"internally_reserved":0,"allocated":475136,"data_stored":143881,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":461762222},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182535,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":547292,"kb_used_data":68620,"kb_used_omap":4,"kb_used_meta":478651,"kb_avail":93824548,"statfs":{"total":96636764160,"available":96076337152,"internally_reserved":0,"allocated":70266880,"data_stored":69953690,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":490139306},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":69853184,"data_stored":69817054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":7245,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":140607488,"data_stored":140571358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:08:58.543 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-10T16:08:58.543 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "49'20672", 'reported_seq': 20757, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:08:42.659151+0000', 'last_change': '2024-09-10T15:59:16.281558+0000', 'last_active': '2024-09-10T16:08:42.659151+0000', 'last_peered': '2024-09-10T16:08:42.659151+0000', 'last_clean': '2024-09-10T15:59:14.132456+0000', 'last_became_active': '2024-09-10T15:59:16.274407+0000', 'last_became_peered': '2024-09-10T15:59:16.274407+0000', 'last_unstale': '2024-09-10T16:08:42.659151+0000', 'last_undegraded': '2024-09-10T15:59:15.851279+0000', 'last_fullsized': '2024-09-10T15:59:15.774356+0000', 'mapping_epoch': 42, 'log_start': "45'10600", 'ondisk_log_start': "45'10600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'objects_scrubbed': 0, 'log_size': 10072, 'log_dups_size': 0, 'ondisk_log_size': 10072, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4259, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20672, 'num_write_kb': 41344, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4259}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:08:58.543 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-10T16:08:58.543 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "49'21061", 'reported_seq': 21209, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:08:42.603655+0000', 'last_change': '2024-09-10T15:59:16.308264+0000', 'last_active': '2024-09-10T16:08:42.603655+0000', 'last_peered': '2024-09-10T16:08:42.603655+0000', 'last_clean': '2024-09-10T15:59:14.111551+0000', 'last_became_active': '2024-09-10T15:59:16.300040+0000', 'last_became_peered': '2024-09-10T15:59:16.300040+0000', 'last_unstale': '2024-09-10T16:08:42.603655+0000', 'last_undegraded': '2024-09-10T15:59:15.878234+0000', 'last_fullsized': '2024-09-10T15:59:15.769328+0000', 'mapping_epoch': 42, 'log_start': "45'11000", 'ondisk_log_start': "45'11000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3442", 'last_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'objects_scrubbed': 3384, 'log_size': 10061, 'log_dups_size': 0, 'ondisk_log_size': 10061, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1292, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4372, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 17673, 'num_write_kb': 28580, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4370}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:08:58.543 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-10T16:08:58.543 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "49'21082", 'reported_seq': 21230, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:08:42.545975+0000', 'last_change': '2024-09-10T15:59:16.331899+0000', 'last_active': '2024-09-10T16:08:42.545975+0000', 'last_peered': '2024-09-10T16:08:42.545975+0000', 'last_clean': '2024-09-10T15:59:14.139436+0000', 'last_became_active': '2024-09-10T15:59:16.318715+0000', 'last_became_peered': '2024-09-10T15:59:16.318715+0000', 'last_unstale': '2024-09-10T16:08:42.545975+0000', 'last_undegraded': '2024-09-10T15:59:15.877524+0000', 'last_fullsized': '2024-09-10T15:59:15.769042+0000', 'mapping_epoch': 42, 'log_start': "45'11000", 'ondisk_log_start': "45'11000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3537", 'last_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'objects_scrubbed': 3497, 'log_size': 10082, 'log_dups_size': 0, 'ondisk_log_size': 10082, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1224, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4237, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 17581, 'num_write_kb': 28172, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4235}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:08:58.543 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-10T16:08:58.543 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "49'20457", 'reported_seq': 20608, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:08:42.662113+0000', 'last_change': '2024-09-10T15:59:16.175680+0000', 'last_active': '2024-09-10T16:08:42.662113+0000', 'last_peered': '2024-09-10T16:08:42.662113+0000', 'last_clean': '2024-09-10T15:59:14.088187+0000', 'last_became_active': '2024-09-10T15:59:16.165810+0000', 'last_became_peered': '2024-09-10T15:59:16.165810+0000', 'last_unstale': '2024-09-10T16:08:42.662113+0000', 'last_undegraded': '2024-09-10T15:59:15.822098+0000', 'last_fullsized': '2024-09-10T15:59:15.769123+0000', 'mapping_epoch': 42, 'log_start': "45'10400", 'ondisk_log_start': "45'10400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2218", 'last_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'objects_scrubbed': 2175, 'log_size': 10057, 'log_dups_size': 0, 'ondisk_log_size': 10057, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 825, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4124, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 18276, 'num_write_kb': 32204, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': -4122}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:08:58.543 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-10T16:08:58.544 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "49'20910", 'reported_seq': 25326, 'reported_epoch': 49, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:08:56.196825+0000', 'last_change': '2024-09-10T16:08:09.280891+0000', 'last_active': '2024-09-10T16:08:56.196825+0000', 'last_peered': '2024-09-10T16:08:56.196825+0000', 'last_clean': '2024-09-10T15:59:14.129081+0000', 'last_became_active': '2024-09-10T15:59:16.269835+0000', 'last_became_peered': '2024-09-10T15:59:16.269835+0000', 'last_unstale': '2024-09-10T16:08:56.196825+0000', 'last_undegraded': '2024-09-10T15:59:15.860857+0000', 'last_fullsized': '2024-09-10T15:59:15.771410+0000', 'mapping_epoch': 42, 'log_start': "45'10900", 'ondisk_log_start': "45'10900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3557", 'last_scrub_stamp': '2024-09-10T15:59:02.493678+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:02.493678+0000', 'objects_scrubbed': 3488, 'log_size': 10010, 'log_dups_size': 0, 'ondisk_log_size': 10010, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1472, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2120, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17414, 'num_write_kb': 27852, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2137, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2120}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:08:58.544 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-10T16:08:58.544 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "49'20781", 'reported_seq': 20927, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:08:42.672320+0000', 'last_change': '2024-09-10T15:59:16.284311+0000', 'last_active': '2024-09-10T16:08:42.672320+0000', 'last_peered': '2024-09-10T16:08:42.672320+0000', 'last_clean': '2024-09-10T15:59:14.116041+0000', 'last_became_active': '2024-09-10T15:59:16.274543+0000', 'last_became_peered': '2024-09-10T15:59:16.274543+0000', 'last_unstale': '2024-09-10T16:08:42.672320+0000', 'last_undegraded': '2024-09-10T15:59:15.812484+0000', 'last_fullsized': '2024-09-10T15:59:15.770885+0000', 'mapping_epoch': 42, 'log_start': "45'10700", 'ondisk_log_start': "45'10700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2356", 'last_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'objects_scrubbed': 2316, 'log_size': 10081, 'log_dups_size': 0, 'ondisk_log_size': 10081, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 783, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 18461, 'num_write_kb': 32292, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': -4303}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:08:58.544 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-10T16:08:58.544 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "49'21170", 'reported_seq': 30025, 'reported_epoch': 49, 'state': 'active+recovery_wait', 'last_fresh': '2024-09-10T16:08:42.596968+0000', 'last_change': '2024-09-10T16:08:10.929169+0000', 'last_active': '2024-09-10T16:08:42.596968+0000', 'last_peered': '2024-09-10T16:08:42.596968+0000', 'last_clean': '2024-09-10T15:59:14.110654+0000', 'last_became_active': '2024-09-10T16:08:10.860438+0000', 'last_became_peered': '2024-09-10T16:08:10.860438+0000', 'last_unstale': '2024-09-10T16:08:42.596968+0000', 'last_undegraded': '2024-09-10T16:08:42.596968+0000', 'last_fullsized': '2024-09-10T16:08:42.596968+0000', 'mapping_epoch': 48, 'log_start': "45'11100", 'ondisk_log_start': "45'11100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2486", 'last_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'objects_scrubbed': 2440, 'log_size': 10070, 'log_dups_size': 0, 'ondisk_log_size': 10070, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 792, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18724, 'num_write_kb': 32568, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 4355, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': ['2', '0', '1'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:08:58.544 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-10T16:08:58.544 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "49'20838", 'reported_seq': 20983, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:08:42.624374+0000', 'last_change': '2024-09-10T15:59:16.315641+0000', 'last_active': '2024-09-10T16:08:42.624374+0000', 'last_peered': '2024-09-10T16:08:42.624374+0000', 'last_clean': '2024-09-10T15:59:14.117001+0000', 'last_became_active': '2024-09-10T15:59:16.302245+0000', 'last_became_peered': '2024-09-10T15:59:16.302245+0000', 'last_unstale': '2024-09-10T16:08:42.624374+0000', 'last_undegraded': '2024-09-10T15:59:15.845012+0000', 'last_fullsized': '2024-09-10T15:59:15.770337+0000', 'mapping_epoch': 42, 'log_start': "45'10800", 'ondisk_log_start': "45'10800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3404", 'last_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'objects_scrubbed': 3367, 'log_size': 10038, 'log_dups_size': 0, 'ondisk_log_size': 10038, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 983, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17469, 'num_write_kb': 28204, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4304}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:08:58.544 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-10T16:08:58.544 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "49'21048", 'reported_seq': 21201, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:08:42.674593+0000', 'last_change': '2024-09-10T15:59:16.311291+0000', 'last_active': '2024-09-10T16:08:42.674593+0000', 'last_peered': '2024-09-10T16:08:42.674593+0000', 'last_clean': '2024-09-10T15:59:14.116363+0000', 'last_became_active': '2024-09-10T15:59:16.301244+0000', 'last_became_peered': '2024-09-10T15:59:16.301244+0000', 'last_unstale': '2024-09-10T16:08:42.674593+0000', 'last_undegraded': '2024-09-10T15:59:15.883434+0000', 'last_fullsized': '2024-09-10T15:59:15.772756+0000', 'mapping_epoch': 42, 'log_start': "45'11000", 'ondisk_log_start': "45'11000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3789", 'last_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'objects_scrubbed': 3741, 'log_size': 10048, 'log_dups_size': 0, 'ondisk_log_size': 10048, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1440, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4373, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17303, 'num_write_kb': 27124, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4371}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:08:58.544 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-10T16:08:58.544 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "49'21198", 'reported_seq': 21353, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:08:42.668456+0000', 'last_change': '2024-09-10T15:59:16.313409+0000', 'last_active': '2024-09-10T16:08:42.668456+0000', 'last_peered': '2024-09-10T16:08:42.668456+0000', 'last_clean': '2024-09-10T15:59:14.129780+0000', 'last_became_active': '2024-09-10T15:59:16.300932+0000', 'last_became_peered': '2024-09-10T15:59:16.300932+0000', 'last_unstale': '2024-09-10T16:08:42.668456+0000', 'last_undegraded': '2024-09-10T15:59:15.887823+0000', 'last_fullsized': '2024-09-10T15:59:15.768513+0000', 'mapping_epoch': 42, 'log_start': "45'11100", 'ondisk_log_start': "45'11100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3880", 'last_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'objects_scrubbed': 3827, 'log_size': 10098, 'log_dups_size': 0, 'ondisk_log_size': 10098, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1397, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4325, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17361, 'num_write_kb': 27072, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4321}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:08:58.545 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T16:08:58.991 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:08:58.992 INFO:teuthology.orchestra.run.smithi115.stdout:{"fsid":"7effee29-df05-4351-9741-ee203564ed81","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 36420/24 objects degraded (151750.000%), 9 pgs degraded, 9 pgs undersized","count":18},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":703,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":49,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983954,"num_in_osds":3,"osd_in_since":1725983947,"num_remapped_pgs":9},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":8},{"state_name":"active+recovery_wait","count":1},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":12,"data_bytes":4849,"bytes_used":1537339392,"bytes_avail":288372953088,"bytes_total":289910292480,"degraded_objects":36420,"degraded_total":24,"degraded_ratio":1517.5,"recovering_objects_per_sec":137,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1651,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":53},"fsmap":{"epoch":1,"btime":"2024-09-10T15:57:14:967120+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T16:00:13.560776+0000","services":{}},"progress_events":{}} 2024-09-10T16:08:59.005 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-10T16:08:59.005 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:08:59.347 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:08:59.348 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:08:59.360 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":393,"stamp":"2024-09-10T16:08:57.767764+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292098,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14972,"num_bytes_recovered":33095699,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161025,"ondisk_log_size":161025,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1501308,"kb_used_data":206796,"kb_used_omap":12,"kb_used_meta":1294451,"kb_avail":281614212,"statfs":{"total":289910292480,"available":288372953088,"internally_reserved":0,"allocated":211759104,"data_stored":210805546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1325517826},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-2646016,"num_objects":-646,"num_object_clones":0,"num_object_copies":-1292,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1651,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-646,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":646,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1651,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002390"},"pg_stats":[{"pgid":"2.f","version":"49'20672","reported_seq":20757,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:42.659151+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:08:42.659151+0000","last_peered":"2024-09-10T16:08:42.659151+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:08:42.659151+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20672,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4259}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20844","reported_seq":20967,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:42.591606+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:08:42.591606+0000","last_peered":"2024-09-10T16:08:42.591606+0000","last_clean":"2024-09-10T16:08:42.591606+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:08:42.591606+0000","last_undegraded":"2024-09-10T16:08:42.591606+0000","last_fullsized":"2024-09-10T16:08:42.591606+0000","mapping_epoch":35,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18949,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20798","reported_seq":29486,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:42.665212+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:08:42.665212+0000","last_peered":"2024-09-10T16:08:42.665212+0000","last_clean":"2024-09-10T16:08:42.665212+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:08:42.665212+0000","last_undegraded":"2024-09-10T16:08:42.665212+0000","last_fullsized":"2024-09-10T16:08:42.665212+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18802,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20660","reported_seq":20751,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:42.649157+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:08:42.649157+0000","last_peered":"2024-09-10T16:08:42.649157+0000","last_clean":"2024-09-10T16:08:42.649157+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:08:42.649157+0000","last_undegraded":"2024-09-10T16:08:42.649157+0000","last_fullsized":"2024-09-10T16:08:42.649157+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20660,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20891","reported_seq":21031,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:42.618630+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:08:42.618630+0000","last_peered":"2024-09-10T16:08:42.618630+0000","last_clean":"2024-09-10T16:08:42.618630+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:08:42.618630+0000","last_undegraded":"2024-09-10T16:08:42.618630+0000","last_fullsized":"2024-09-10T16:08:42.618630+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17602,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'21061","reported_seq":21209,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:42.603655+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:08:42.603655+0000","last_peered":"2024-09-10T16:08:42.603655+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:08:42.603655+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17673,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4370},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'21082","reported_seq":21230,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:42.545975+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:08:42.545975+0000","last_peered":"2024-09-10T16:08:42.545975+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:08:42.545975+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17581,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4235},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'21314","reported_seq":21455,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:42.623054+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:08:42.623054+0000","last_peered":"2024-09-10T16:08:42.623054+0000","last_clean":"2024-09-10T16:08:42.623054+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:08:42.623054+0000","last_undegraded":"2024-09-10T16:08:42.623054+0000","last_fullsized":"2024-09-10T16:08:42.623054+0000","mapping_epoch":34,"log_start":"45'11300","ondisk_log_start":"45'11300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18161,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.838857+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:08:10.838857+0000","last_peered":"2024-09-10T16:08:10.838857+0000","last_clean":"2024-09-10T16:08:10.838857+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:08:10.838857+0000","last_undegraded":"2024-09-10T16:08:10.838857+0000","last_fullsized":"2024-09-10T16:08:10.838857+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20457","reported_seq":20608,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:42.662113+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:08:42.662113+0000","last_peered":"2024-09-10T16:08:42.662113+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:08:42.662113+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18276,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-4122}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":105,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.838234+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:08:10.838234+0000","last_peered":"2024-09-10T16:08:10.838234+0000","last_clean":"2024-09-10T16:08:10.838234+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:08:10.838234+0000","last_undegraded":"2024-09-10T16:08:10.838234+0000","last_fullsized":"2024-09-10T16:08:10.838234+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20910","reported_seq":25326,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:56.196825+0000","last_change":"2024-09-10T16:08:09.280891+0000","last_active":"2024-09-10T16:08:56.196825+0000","last_peered":"2024-09-10T16:08:56.196825+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:08:56.196825+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2120,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17414,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2137,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2120}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":115,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.840312+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:08:10.840312+0000","last_peered":"2024-09-10T16:08:10.840312+0000","last_clean":"2024-09-10T16:08:10.840312+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:08:10.840312+0000","last_undegraded":"2024-09-10T16:08:10.840312+0000","last_fullsized":"2024-09-10T16:08:10.840312+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20781","reported_seq":20927,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:42.672320+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:08:42.672320+0000","last_peered":"2024-09-10T16:08:42.672320+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:08:42.672320+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18461,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-4303}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":90,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.837937+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:08:10.837937+0000","last_peered":"2024-09-10T16:08:10.837937+0000","last_clean":"2024-09-10T16:08:10.837937+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:08:10.837937+0000","last_undegraded":"2024-09-10T16:08:10.837937+0000","last_fullsized":"2024-09-10T16:08:10.837937+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'21170","reported_seq":30025,"reported_epoch":49,"state":"active+recovery_wait","last_fresh":"2024-09-10T16:08:42.596968+0000","last_change":"2024-09-10T16:08:10.929169+0000","last_active":"2024-09-10T16:08:42.596968+0000","last_peered":"2024-09-10T16:08:42.596968+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T16:08:10.860438+0000","last_became_peered":"2024-09-10T16:08:10.860438+0000","last_unstale":"2024-09-10T16:08:42.596968+0000","last_undegraded":"2024-09-10T16:08:42.596968+0000","last_fullsized":"2024-09-10T16:08:42.596968+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18724,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","0","1"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20838","reported_seq":20983,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:42.624374+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:08:42.624374+0000","last_peered":"2024-09-10T16:08:42.624374+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:08:42.624374+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17469,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.838546+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:08:10.838546+0000","last_peered":"2024-09-10T16:08:10.838546+0000","last_clean":"2024-09-10T16:08:10.838546+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:08:10.838546+0000","last_undegraded":"2024-09-10T16:08:10.838546+0000","last_fullsized":"2024-09-10T16:08:10.838546+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20598","reported_seq":29174,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:42.620274+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:08:42.620274+0000","last_peered":"2024-09-10T16:08:42.620274+0000","last_clean":"2024-09-10T16:08:42.620274+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:08:42.620274+0000","last_undegraded":"2024-09-10T16:08:42.620274+0000","last_fullsized":"2024-09-10T16:08:42.620274+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16988,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":105,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.838971+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:08:10.838971+0000","last_peered":"2024-09-10T16:08:10.838971+0000","last_clean":"2024-09-10T16:08:10.838971+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:08:10.838971+0000","last_undegraded":"2024-09-10T16:08:10.838971+0000","last_fullsized":"2024-09-10T16:08:10.838971+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'21048","reported_seq":21201,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:42.674593+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:08:42.674593+0000","last_peered":"2024-09-10T16:08:42.674593+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:08:42.674593+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17303,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.839114+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:08:10.839114+0000","last_peered":"2024-09-10T16:08:10.839114+0000","last_clean":"2024-09-10T16:08:10.839114+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:08:10.839114+0000","last_undegraded":"2024-09-10T16:08:10.839114+0000","last_fullsized":"2024-09-10T16:08:10.839114+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":85,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.838633+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:08:10.838633+0000","last_peered":"2024-09-10T16:08:10.838633+0000","last_clean":"2024-09-10T16:08:10.838633+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:08:10.838633+0000","last_undegraded":"2024-09-10T16:08:10.838633+0000","last_fullsized":"2024-09-10T16:08:10.838633+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'21198","reported_seq":21353,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:42.668456+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:08:42.668456+0000","last_peered":"2024-09-10T16:08:42.668456+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:08:42.668456+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17361,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4321},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292096,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14970,"num_bytes_recovered":33095680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":210522112,"data_stored":210395657,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161022,"ondisk_log_size":161022,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986307,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":502592,"kb_used_data":137712,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93869248,"statfs":{"total":96636764160,"available":96122109952,"internally_reserved":0,"allocated":141017088,"data_stored":140707975,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659252,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":451424,"kb_used_data":464,"kb_used_omap":4,"kb_used_meta":450939,"kb_avail":93920416,"statfs":{"total":96636764160,"available":96174505984,"internally_reserved":0,"allocated":475136,"data_stored":143881,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":461762222},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182535,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":547292,"kb_used_data":68620,"kb_used_omap":4,"kb_used_meta":478651,"kb_avail":93824548,"statfs":{"total":96636764160,"available":96076337152,"internally_reserved":0,"allocated":70266880,"data_stored":69953690,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":490139306},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":69853184,"data_stored":69817054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":7245,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":140607488,"data_stored":140571358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:09:02.362 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:09:02.681 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:09:02.681 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:09:02.693 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":395,"stamp":"2024-09-10T16:09:01.768501+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35636,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292098,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15756,"num_bytes_recovered":33095699,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161025,"ondisk_log_size":161025,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1492988,"kb_used_data":206796,"kb_used_omap":12,"kb_used_meta":1286131,"kb_avail":281622532,"statfs":{"total":289910292480,"available":288381472768,"internally_reserved":0,"allocated":211759104,"data_stored":210805546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1316998146},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2041,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2041,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002336"},"pg_stats":[{"pgid":"2.f","version":"49'20672","reported_seq":20757,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:42.659151+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:08:42.659151+0000","last_peered":"2024-09-10T16:08:42.659151+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:08:42.659151+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20672,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4259}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20844","reported_seq":20967,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:42.591606+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:08:42.591606+0000","last_peered":"2024-09-10T16:08:42.591606+0000","last_clean":"2024-09-10T16:08:42.591606+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:08:42.591606+0000","last_undegraded":"2024-09-10T16:08:42.591606+0000","last_fullsized":"2024-09-10T16:08:42.591606+0000","mapping_epoch":35,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18949,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20798","reported_seq":29487,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:42.681302+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:08:42.681302+0000","last_peered":"2024-09-10T16:08:42.681302+0000","last_clean":"2024-09-10T16:08:42.681302+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:08:42.681302+0000","last_undegraded":"2024-09-10T16:08:42.681302+0000","last_fullsized":"2024-09-10T16:08:42.681302+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18802,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20660","reported_seq":20751,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:42.649157+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:08:42.649157+0000","last_peered":"2024-09-10T16:08:42.649157+0000","last_clean":"2024-09-10T16:08:42.649157+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:08:42.649157+0000","last_undegraded":"2024-09-10T16:08:42.649157+0000","last_fullsized":"2024-09-10T16:08:42.649157+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20660,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20891","reported_seq":21032,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:42.663199+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:08:42.663199+0000","last_peered":"2024-09-10T16:08:42.663199+0000","last_clean":"2024-09-10T16:08:42.663199+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:08:42.663199+0000","last_undegraded":"2024-09-10T16:08:42.663199+0000","last_fullsized":"2024-09-10T16:08:42.663199+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17602,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'21061","reported_seq":21209,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:42.603655+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:08:42.603655+0000","last_peered":"2024-09-10T16:08:42.603655+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:08:42.603655+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17673,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4370},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'21082","reported_seq":21230,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:42.545975+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:08:42.545975+0000","last_peered":"2024-09-10T16:08:42.545975+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:08:42.545975+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17581,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4235},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'21314","reported_seq":21455,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:42.623054+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:08:42.623054+0000","last_peered":"2024-09-10T16:08:42.623054+0000","last_clean":"2024-09-10T16:08:42.623054+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:08:42.623054+0000","last_undegraded":"2024-09-10T16:08:42.623054+0000","last_fullsized":"2024-09-10T16:08:42.623054+0000","mapping_epoch":34,"log_start":"45'11300","ondisk_log_start":"45'11300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18161,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.838857+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:08:10.838857+0000","last_peered":"2024-09-10T16:08:10.838857+0000","last_clean":"2024-09-10T16:08:10.838857+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:08:10.838857+0000","last_undegraded":"2024-09-10T16:08:10.838857+0000","last_fullsized":"2024-09-10T16:08:10.838857+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20457","reported_seq":20608,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:42.662113+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:08:42.662113+0000","last_peered":"2024-09-10T16:08:42.662113+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:08:42.662113+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18276,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-4122}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":105,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.838234+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:08:10.838234+0000","last_peered":"2024-09-10T16:08:10.838234+0000","last_clean":"2024-09-10T16:08:10.838234+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:08:10.838234+0000","last_undegraded":"2024-09-10T16:08:10.838234+0000","last_fullsized":"2024-09-10T16:08:10.838234+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20910","reported_seq":26894,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:01.198107+0000","last_change":"2024-09-10T16:08:09.280891+0000","last_active":"2024-09-10T16:09:01.198107+0000","last_peered":"2024-09-10T16:09:01.198107+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:09:01.198107+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1336,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17414,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2921,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1336}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":115,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.840312+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:08:10.840312+0000","last_peered":"2024-09-10T16:08:10.840312+0000","last_clean":"2024-09-10T16:08:10.840312+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:08:10.840312+0000","last_undegraded":"2024-09-10T16:08:10.840312+0000","last_fullsized":"2024-09-10T16:08:10.840312+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20781","reported_seq":20927,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:42.672320+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:08:42.672320+0000","last_peered":"2024-09-10T16:08:42.672320+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:08:42.672320+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18461,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-4303}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":90,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.837937+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:08:10.837937+0000","last_peered":"2024-09-10T16:08:10.837937+0000","last_clean":"2024-09-10T16:08:10.837937+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:08:10.837937+0000","last_undegraded":"2024-09-10T16:08:10.837937+0000","last_fullsized":"2024-09-10T16:08:10.837937+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'21170","reported_seq":30025,"reported_epoch":49,"state":"active+recovery_wait","last_fresh":"2024-09-10T16:08:42.596968+0000","last_change":"2024-09-10T16:08:10.929169+0000","last_active":"2024-09-10T16:08:42.596968+0000","last_peered":"2024-09-10T16:08:42.596968+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T16:08:10.860438+0000","last_became_peered":"2024-09-10T16:08:10.860438+0000","last_unstale":"2024-09-10T16:08:42.596968+0000","last_undegraded":"2024-09-10T16:08:42.596968+0000","last_fullsized":"2024-09-10T16:08:42.596968+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18724,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","0","1"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20838","reported_seq":20983,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:42.624374+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:08:42.624374+0000","last_peered":"2024-09-10T16:08:42.624374+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:08:42.624374+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17469,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.838546+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:08:10.838546+0000","last_peered":"2024-09-10T16:08:10.838546+0000","last_clean":"2024-09-10T16:08:10.838546+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:08:10.838546+0000","last_undegraded":"2024-09-10T16:08:10.838546+0000","last_fullsized":"2024-09-10T16:08:10.838546+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20598","reported_seq":29174,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:42.620274+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:08:42.620274+0000","last_peered":"2024-09-10T16:08:42.620274+0000","last_clean":"2024-09-10T16:08:42.620274+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:08:42.620274+0000","last_undegraded":"2024-09-10T16:08:42.620274+0000","last_fullsized":"2024-09-10T16:08:42.620274+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16988,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":105,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.838971+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:08:10.838971+0000","last_peered":"2024-09-10T16:08:10.838971+0000","last_clean":"2024-09-10T16:08:10.838971+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:08:10.838971+0000","last_undegraded":"2024-09-10T16:08:10.838971+0000","last_fullsized":"2024-09-10T16:08:10.838971+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'21048","reported_seq":21201,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:42.674593+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:08:42.674593+0000","last_peered":"2024-09-10T16:08:42.674593+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:08:42.674593+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17303,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.839114+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:08:10.839114+0000","last_peered":"2024-09-10T16:08:10.839114+0000","last_clean":"2024-09-10T16:08:10.839114+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:08:10.839114+0000","last_undegraded":"2024-09-10T16:08:10.839114+0000","last_fullsized":"2024-09-10T16:08:10.839114+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":85,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.838633+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:08:10.838633+0000","last_peered":"2024-09-10T16:08:10.838633+0000","last_clean":"2024-09-10T16:08:10.838633+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:08:10.838633+0000","last_undegraded":"2024-09-10T16:08:10.838633+0000","last_fullsized":"2024-09-10T16:08:10.838633+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'21198","reported_seq":21353,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:42.668456+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:08:42.668456+0000","last_peered":"2024-09-10T16:08:42.668456+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:08:42.668456+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17361,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4321},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35636,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292096,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15754,"num_bytes_recovered":33095680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":210522112,"data_stored":210395657,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161022,"ondisk_log_size":161022,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986308,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":502592,"kb_used_data":137712,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93869248,"statfs":{"total":96636764160,"available":96122109952,"internally_reserved":0,"allocated":141017088,"data_stored":140707975,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659253,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":443104,"kb_used_data":464,"kb_used_omap":4,"kb_used_meta":442619,"kb_avail":93928736,"statfs":{"total":96636764160,"available":96183025664,"internally_reserved":0,"allocated":475136,"data_stored":143881,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":453242542},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182536,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":547292,"kb_used_data":68620,"kb_used_omap":4,"kb_used_meta":478651,"kb_avail":93824548,"statfs":{"total":96636764160,"available":96076337152,"internally_reserved":0,"allocated":70266880,"data_stored":69953690,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":490139306},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":69853184,"data_stored":69817054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":7245,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":140607488,"data_stored":140571358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:09:02.695 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:09:03.015 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:09:03.016 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:09:03.027 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":395,"stamp":"2024-09-10T16:09:01.768501+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35636,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292098,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15756,"num_bytes_recovered":33095699,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161025,"ondisk_log_size":161025,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1492988,"kb_used_data":206796,"kb_used_omap":12,"kb_used_meta":1286131,"kb_avail":281622532,"statfs":{"total":289910292480,"available":288381472768,"internally_reserved":0,"allocated":211759104,"data_stored":210805546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1316998146},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2041,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2041,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002336"},"pg_stats":[{"pgid":"2.f","version":"49'20672","reported_seq":20757,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:42.659151+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:08:42.659151+0000","last_peered":"2024-09-10T16:08:42.659151+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:08:42.659151+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20672,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4259}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20844","reported_seq":20967,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:42.591606+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:08:42.591606+0000","last_peered":"2024-09-10T16:08:42.591606+0000","last_clean":"2024-09-10T16:08:42.591606+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:08:42.591606+0000","last_undegraded":"2024-09-10T16:08:42.591606+0000","last_fullsized":"2024-09-10T16:08:42.591606+0000","mapping_epoch":35,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18949,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20798","reported_seq":29487,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:42.681302+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:08:42.681302+0000","last_peered":"2024-09-10T16:08:42.681302+0000","last_clean":"2024-09-10T16:08:42.681302+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:08:42.681302+0000","last_undegraded":"2024-09-10T16:08:42.681302+0000","last_fullsized":"2024-09-10T16:08:42.681302+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18802,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20660","reported_seq":20751,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:42.649157+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:08:42.649157+0000","last_peered":"2024-09-10T16:08:42.649157+0000","last_clean":"2024-09-10T16:08:42.649157+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:08:42.649157+0000","last_undegraded":"2024-09-10T16:08:42.649157+0000","last_fullsized":"2024-09-10T16:08:42.649157+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20660,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20891","reported_seq":21032,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:42.663199+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:08:42.663199+0000","last_peered":"2024-09-10T16:08:42.663199+0000","last_clean":"2024-09-10T16:08:42.663199+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:08:42.663199+0000","last_undegraded":"2024-09-10T16:08:42.663199+0000","last_fullsized":"2024-09-10T16:08:42.663199+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17602,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'21061","reported_seq":21209,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:42.603655+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:08:42.603655+0000","last_peered":"2024-09-10T16:08:42.603655+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:08:42.603655+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17673,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4370},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'21082","reported_seq":21230,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:42.545975+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:08:42.545975+0000","last_peered":"2024-09-10T16:08:42.545975+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:08:42.545975+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17581,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4235},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'21314","reported_seq":21455,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:42.623054+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:08:42.623054+0000","last_peered":"2024-09-10T16:08:42.623054+0000","last_clean":"2024-09-10T16:08:42.623054+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:08:42.623054+0000","last_undegraded":"2024-09-10T16:08:42.623054+0000","last_fullsized":"2024-09-10T16:08:42.623054+0000","mapping_epoch":34,"log_start":"45'11300","ondisk_log_start":"45'11300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18161,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.838857+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:08:10.838857+0000","last_peered":"2024-09-10T16:08:10.838857+0000","last_clean":"2024-09-10T16:08:10.838857+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:08:10.838857+0000","last_undegraded":"2024-09-10T16:08:10.838857+0000","last_fullsized":"2024-09-10T16:08:10.838857+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20457","reported_seq":20608,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:42.662113+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:08:42.662113+0000","last_peered":"2024-09-10T16:08:42.662113+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:08:42.662113+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18276,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-4122}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":105,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.838234+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:08:10.838234+0000","last_peered":"2024-09-10T16:08:10.838234+0000","last_clean":"2024-09-10T16:08:10.838234+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:08:10.838234+0000","last_undegraded":"2024-09-10T16:08:10.838234+0000","last_fullsized":"2024-09-10T16:08:10.838234+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20910","reported_seq":26894,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:01.198107+0000","last_change":"2024-09-10T16:08:09.280891+0000","last_active":"2024-09-10T16:09:01.198107+0000","last_peered":"2024-09-10T16:09:01.198107+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:09:01.198107+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1336,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17414,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2921,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1336}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":115,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.840312+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:08:10.840312+0000","last_peered":"2024-09-10T16:08:10.840312+0000","last_clean":"2024-09-10T16:08:10.840312+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:08:10.840312+0000","last_undegraded":"2024-09-10T16:08:10.840312+0000","last_fullsized":"2024-09-10T16:08:10.840312+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20781","reported_seq":20927,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:42.672320+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:08:42.672320+0000","last_peered":"2024-09-10T16:08:42.672320+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:08:42.672320+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18461,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-4303}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":90,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.837937+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:08:10.837937+0000","last_peered":"2024-09-10T16:08:10.837937+0000","last_clean":"2024-09-10T16:08:10.837937+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:08:10.837937+0000","last_undegraded":"2024-09-10T16:08:10.837937+0000","last_fullsized":"2024-09-10T16:08:10.837937+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'21170","reported_seq":30025,"reported_epoch":49,"state":"active+recovery_wait","last_fresh":"2024-09-10T16:08:42.596968+0000","last_change":"2024-09-10T16:08:10.929169+0000","last_active":"2024-09-10T16:08:42.596968+0000","last_peered":"2024-09-10T16:08:42.596968+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T16:08:10.860438+0000","last_became_peered":"2024-09-10T16:08:10.860438+0000","last_unstale":"2024-09-10T16:08:42.596968+0000","last_undegraded":"2024-09-10T16:08:42.596968+0000","last_fullsized":"2024-09-10T16:08:42.596968+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18724,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","0","1"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20838","reported_seq":20983,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:42.624374+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:08:42.624374+0000","last_peered":"2024-09-10T16:08:42.624374+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:08:42.624374+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17469,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.838546+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:08:10.838546+0000","last_peered":"2024-09-10T16:08:10.838546+0000","last_clean":"2024-09-10T16:08:10.838546+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:08:10.838546+0000","last_undegraded":"2024-09-10T16:08:10.838546+0000","last_fullsized":"2024-09-10T16:08:10.838546+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20598","reported_seq":29174,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:42.620274+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:08:42.620274+0000","last_peered":"2024-09-10T16:08:42.620274+0000","last_clean":"2024-09-10T16:08:42.620274+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:08:42.620274+0000","last_undegraded":"2024-09-10T16:08:42.620274+0000","last_fullsized":"2024-09-10T16:08:42.620274+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16988,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":105,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.838971+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:08:10.838971+0000","last_peered":"2024-09-10T16:08:10.838971+0000","last_clean":"2024-09-10T16:08:10.838971+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:08:10.838971+0000","last_undegraded":"2024-09-10T16:08:10.838971+0000","last_fullsized":"2024-09-10T16:08:10.838971+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'21048","reported_seq":21201,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:42.674593+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:08:42.674593+0000","last_peered":"2024-09-10T16:08:42.674593+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:08:42.674593+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17303,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.839114+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:08:10.839114+0000","last_peered":"2024-09-10T16:08:10.839114+0000","last_clean":"2024-09-10T16:08:10.839114+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:08:10.839114+0000","last_undegraded":"2024-09-10T16:08:10.839114+0000","last_fullsized":"2024-09-10T16:08:10.839114+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":85,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.838633+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:08:10.838633+0000","last_peered":"2024-09-10T16:08:10.838633+0000","last_clean":"2024-09-10T16:08:10.838633+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:08:10.838633+0000","last_undegraded":"2024-09-10T16:08:10.838633+0000","last_fullsized":"2024-09-10T16:08:10.838633+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'21198","reported_seq":21353,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:42.668456+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:08:42.668456+0000","last_peered":"2024-09-10T16:08:42.668456+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:08:42.668456+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17361,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4321},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35636,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292096,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15754,"num_bytes_recovered":33095680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":210522112,"data_stored":210395657,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161022,"ondisk_log_size":161022,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986308,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":502592,"kb_used_data":137712,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93869248,"statfs":{"total":96636764160,"available":96122109952,"internally_reserved":0,"allocated":141017088,"data_stored":140707975,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659253,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":443104,"kb_used_data":464,"kb_used_omap":4,"kb_used_meta":442619,"kb_avail":93928736,"statfs":{"total":96636764160,"available":96183025664,"internally_reserved":0,"allocated":475136,"data_stored":143881,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":453242542},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182536,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":547292,"kb_used_data":68620,"kb_used_omap":4,"kb_used_meta":478651,"kb_avail":93824548,"statfs":{"total":96636764160,"available":96076337152,"internally_reserved":0,"allocated":70266880,"data_stored":69953690,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":490139306},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":69853184,"data_stored":69817054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":7245,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":140607488,"data_stored":140571358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:09:03.029 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-10T16:09:03.029 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "49'20672", 'reported_seq': 20757, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:08:42.659151+0000', 'last_change': '2024-09-10T15:59:16.281558+0000', 'last_active': '2024-09-10T16:08:42.659151+0000', 'last_peered': '2024-09-10T16:08:42.659151+0000', 'last_clean': '2024-09-10T15:59:14.132456+0000', 'last_became_active': '2024-09-10T15:59:16.274407+0000', 'last_became_peered': '2024-09-10T15:59:16.274407+0000', 'last_unstale': '2024-09-10T16:08:42.659151+0000', 'last_undegraded': '2024-09-10T15:59:15.851279+0000', 'last_fullsized': '2024-09-10T15:59:15.774356+0000', 'mapping_epoch': 42, 'log_start': "45'10600", 'ondisk_log_start': "45'10600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'objects_scrubbed': 0, 'log_size': 10072, 'log_dups_size': 0, 'ondisk_log_size': 10072, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4259, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20672, 'num_write_kb': 41344, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4259}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:09:03.029 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-10T16:09:03.029 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "49'21061", 'reported_seq': 21209, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:08:42.603655+0000', 'last_change': '2024-09-10T15:59:16.308264+0000', 'last_active': '2024-09-10T16:08:42.603655+0000', 'last_peered': '2024-09-10T16:08:42.603655+0000', 'last_clean': '2024-09-10T15:59:14.111551+0000', 'last_became_active': '2024-09-10T15:59:16.300040+0000', 'last_became_peered': '2024-09-10T15:59:16.300040+0000', 'last_unstale': '2024-09-10T16:08:42.603655+0000', 'last_undegraded': '2024-09-10T15:59:15.878234+0000', 'last_fullsized': '2024-09-10T15:59:15.769328+0000', 'mapping_epoch': 42, 'log_start': "45'11000", 'ondisk_log_start': "45'11000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3442", 'last_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'objects_scrubbed': 3384, 'log_size': 10061, 'log_dups_size': 0, 'ondisk_log_size': 10061, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1292, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4372, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 17673, 'num_write_kb': 28580, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4370}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:09:03.029 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-10T16:09:03.029 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "49'21082", 'reported_seq': 21230, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:08:42.545975+0000', 'last_change': '2024-09-10T15:59:16.331899+0000', 'last_active': '2024-09-10T16:08:42.545975+0000', 'last_peered': '2024-09-10T16:08:42.545975+0000', 'last_clean': '2024-09-10T15:59:14.139436+0000', 'last_became_active': '2024-09-10T15:59:16.318715+0000', 'last_became_peered': '2024-09-10T15:59:16.318715+0000', 'last_unstale': '2024-09-10T16:08:42.545975+0000', 'last_undegraded': '2024-09-10T15:59:15.877524+0000', 'last_fullsized': '2024-09-10T15:59:15.769042+0000', 'mapping_epoch': 42, 'log_start': "45'11000", 'ondisk_log_start': "45'11000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3537", 'last_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'objects_scrubbed': 3497, 'log_size': 10082, 'log_dups_size': 0, 'ondisk_log_size': 10082, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1224, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4237, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 17581, 'num_write_kb': 28172, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4235}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:09:03.029 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-10T16:09:03.029 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "49'20457", 'reported_seq': 20608, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:08:42.662113+0000', 'last_change': '2024-09-10T15:59:16.175680+0000', 'last_active': '2024-09-10T16:08:42.662113+0000', 'last_peered': '2024-09-10T16:08:42.662113+0000', 'last_clean': '2024-09-10T15:59:14.088187+0000', 'last_became_active': '2024-09-10T15:59:16.165810+0000', 'last_became_peered': '2024-09-10T15:59:16.165810+0000', 'last_unstale': '2024-09-10T16:08:42.662113+0000', 'last_undegraded': '2024-09-10T15:59:15.822098+0000', 'last_fullsized': '2024-09-10T15:59:15.769123+0000', 'mapping_epoch': 42, 'log_start': "45'10400", 'ondisk_log_start': "45'10400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2218", 'last_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'objects_scrubbed': 2175, 'log_size': 10057, 'log_dups_size': 0, 'ondisk_log_size': 10057, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 825, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4124, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 18276, 'num_write_kb': 32204, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': -4122}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:09:03.030 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-10T16:09:03.030 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "49'20910", 'reported_seq': 26894, 'reported_epoch': 49, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:09:01.198107+0000', 'last_change': '2024-09-10T16:08:09.280891+0000', 'last_active': '2024-09-10T16:09:01.198107+0000', 'last_peered': '2024-09-10T16:09:01.198107+0000', 'last_clean': '2024-09-10T15:59:14.129081+0000', 'last_became_active': '2024-09-10T15:59:16.269835+0000', 'last_became_peered': '2024-09-10T15:59:16.269835+0000', 'last_unstale': '2024-09-10T16:09:01.198107+0000', 'last_undegraded': '2024-09-10T15:59:15.860857+0000', 'last_fullsized': '2024-09-10T15:59:15.771410+0000', 'mapping_epoch': 42, 'log_start': "45'10900", 'ondisk_log_start': "45'10900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3557", 'last_scrub_stamp': '2024-09-10T15:59:02.493678+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:02.493678+0000', 'objects_scrubbed': 3488, 'log_size': 10010, 'log_dups_size': 0, 'ondisk_log_size': 10010, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1472, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1336, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17414, 'num_write_kb': 27852, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2921, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1336}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:09:03.030 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-10T16:09:03.030 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "49'20781", 'reported_seq': 20927, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:08:42.672320+0000', 'last_change': '2024-09-10T15:59:16.284311+0000', 'last_active': '2024-09-10T16:08:42.672320+0000', 'last_peered': '2024-09-10T16:08:42.672320+0000', 'last_clean': '2024-09-10T15:59:14.116041+0000', 'last_became_active': '2024-09-10T15:59:16.274543+0000', 'last_became_peered': '2024-09-10T15:59:16.274543+0000', 'last_unstale': '2024-09-10T16:08:42.672320+0000', 'last_undegraded': '2024-09-10T15:59:15.812484+0000', 'last_fullsized': '2024-09-10T15:59:15.770885+0000', 'mapping_epoch': 42, 'log_start': "45'10700", 'ondisk_log_start': "45'10700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2356", 'last_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'objects_scrubbed': 2316, 'log_size': 10081, 'log_dups_size': 0, 'ondisk_log_size': 10081, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 783, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 18461, 'num_write_kb': 32292, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': -4303}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:09:03.030 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-10T16:09:03.030 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "49'21170", 'reported_seq': 30025, 'reported_epoch': 49, 'state': 'active+recovery_wait', 'last_fresh': '2024-09-10T16:08:42.596968+0000', 'last_change': '2024-09-10T16:08:10.929169+0000', 'last_active': '2024-09-10T16:08:42.596968+0000', 'last_peered': '2024-09-10T16:08:42.596968+0000', 'last_clean': '2024-09-10T15:59:14.110654+0000', 'last_became_active': '2024-09-10T16:08:10.860438+0000', 'last_became_peered': '2024-09-10T16:08:10.860438+0000', 'last_unstale': '2024-09-10T16:08:42.596968+0000', 'last_undegraded': '2024-09-10T16:08:42.596968+0000', 'last_fullsized': '2024-09-10T16:08:42.596968+0000', 'mapping_epoch': 48, 'log_start': "45'11100", 'ondisk_log_start': "45'11100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2486", 'last_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'objects_scrubbed': 2440, 'log_size': 10070, 'log_dups_size': 0, 'ondisk_log_size': 10070, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 792, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18724, 'num_write_kb': 32568, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 4355, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': ['2', '0', '1'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:09:03.030 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-10T16:09:03.030 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "49'20838", 'reported_seq': 20983, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:08:42.624374+0000', 'last_change': '2024-09-10T15:59:16.315641+0000', 'last_active': '2024-09-10T16:08:42.624374+0000', 'last_peered': '2024-09-10T16:08:42.624374+0000', 'last_clean': '2024-09-10T15:59:14.117001+0000', 'last_became_active': '2024-09-10T15:59:16.302245+0000', 'last_became_peered': '2024-09-10T15:59:16.302245+0000', 'last_unstale': '2024-09-10T16:08:42.624374+0000', 'last_undegraded': '2024-09-10T15:59:15.845012+0000', 'last_fullsized': '2024-09-10T15:59:15.770337+0000', 'mapping_epoch': 42, 'log_start': "45'10800", 'ondisk_log_start': "45'10800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3404", 'last_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'objects_scrubbed': 3367, 'log_size': 10038, 'log_dups_size': 0, 'ondisk_log_size': 10038, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 983, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17469, 'num_write_kb': 28204, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4304}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:09:03.030 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-10T16:09:03.030 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "49'21048", 'reported_seq': 21201, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:08:42.674593+0000', 'last_change': '2024-09-10T15:59:16.311291+0000', 'last_active': '2024-09-10T16:08:42.674593+0000', 'last_peered': '2024-09-10T16:08:42.674593+0000', 'last_clean': '2024-09-10T15:59:14.116363+0000', 'last_became_active': '2024-09-10T15:59:16.301244+0000', 'last_became_peered': '2024-09-10T15:59:16.301244+0000', 'last_unstale': '2024-09-10T16:08:42.674593+0000', 'last_undegraded': '2024-09-10T15:59:15.883434+0000', 'last_fullsized': '2024-09-10T15:59:15.772756+0000', 'mapping_epoch': 42, 'log_start': "45'11000", 'ondisk_log_start': "45'11000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3789", 'last_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'objects_scrubbed': 3741, 'log_size': 10048, 'log_dups_size': 0, 'ondisk_log_size': 10048, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1440, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4373, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17303, 'num_write_kb': 27124, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4371}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:09:03.031 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-10T16:09:03.031 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "49'21198", 'reported_seq': 21353, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:08:42.668456+0000', 'last_change': '2024-09-10T15:59:16.313409+0000', 'last_active': '2024-09-10T16:08:42.668456+0000', 'last_peered': '2024-09-10T16:08:42.668456+0000', 'last_clean': '2024-09-10T15:59:14.129780+0000', 'last_became_active': '2024-09-10T15:59:16.300932+0000', 'last_became_peered': '2024-09-10T15:59:16.300932+0000', 'last_unstale': '2024-09-10T16:08:42.668456+0000', 'last_undegraded': '2024-09-10T15:59:15.887823+0000', 'last_fullsized': '2024-09-10T15:59:15.768513+0000', 'mapping_epoch': 42, 'log_start': "45'11100", 'ondisk_log_start': "45'11100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3880", 'last_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'objects_scrubbed': 3827, 'log_size': 10098, 'log_dups_size': 0, 'ondisk_log_size': 10098, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1397, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4325, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17361, 'num_write_kb': 27072, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4321}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:09:03.031 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T16:09:03.422 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:09:03.422 INFO:teuthology.orchestra.run.smithi115.stdout:{"fsid":"7effee29-df05-4351-9741-ee203564ed81","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 35636/24 objects degraded (148483.333%), 9 pgs degraded, 9 pgs undersized","count":18},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":707,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":49,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983954,"num_in_osds":3,"osd_in_since":1725983947,"num_remapped_pgs":9},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":8},{"state_name":"active+recovery_wait","count":1},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":12,"data_bytes":4849,"bytes_used":1528819712,"bytes_avail":288381472768,"bytes_total":289910292480,"degraded_objects":35636,"degraded_total":24,"degraded_ratio":1484.833333333333,"recovering_objects_per_sec":170,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":2041,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-10T15:57:14:967120+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T16:00:13.560776+0000","services":{}},"progress_events":{}} 2024-09-10T16:09:03.435 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-10T16:09:03.436 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:09:03.793 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:09:03.794 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:09:03.807 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":396,"stamp":"2024-09-10T16:09:03.768921+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35636,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292098,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15756,"num_bytes_recovered":33095699,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161025,"ondisk_log_size":161025,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1492988,"kb_used_data":206796,"kb_used_omap":12,"kb_used_meta":1286131,"kb_avail":281622532,"statfs":{"total":289910292480,"available":288381472768,"internally_reserved":0,"allocated":211759104,"data_stored":210805546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1316998146},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1457,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1457,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002321"},"pg_stats":[{"pgid":"2.f","version":"49'20672","reported_seq":20757,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:42.659151+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:08:42.659151+0000","last_peered":"2024-09-10T16:08:42.659151+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:08:42.659151+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20672,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4259}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20844","reported_seq":20967,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:42.591606+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:08:42.591606+0000","last_peered":"2024-09-10T16:08:42.591606+0000","last_clean":"2024-09-10T16:08:42.591606+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:08:42.591606+0000","last_undegraded":"2024-09-10T16:08:42.591606+0000","last_fullsized":"2024-09-10T16:08:42.591606+0000","mapping_epoch":35,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18949,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20798","reported_seq":29487,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:42.681302+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:08:42.681302+0000","last_peered":"2024-09-10T16:08:42.681302+0000","last_clean":"2024-09-10T16:08:42.681302+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:08:42.681302+0000","last_undegraded":"2024-09-10T16:08:42.681302+0000","last_fullsized":"2024-09-10T16:08:42.681302+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18802,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20660","reported_seq":20751,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:42.649157+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:08:42.649157+0000","last_peered":"2024-09-10T16:08:42.649157+0000","last_clean":"2024-09-10T16:08:42.649157+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:08:42.649157+0000","last_undegraded":"2024-09-10T16:08:42.649157+0000","last_fullsized":"2024-09-10T16:08:42.649157+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20660,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20891","reported_seq":21032,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:42.663199+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:08:42.663199+0000","last_peered":"2024-09-10T16:08:42.663199+0000","last_clean":"2024-09-10T16:08:42.663199+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:08:42.663199+0000","last_undegraded":"2024-09-10T16:08:42.663199+0000","last_fullsized":"2024-09-10T16:08:42.663199+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17602,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'21061","reported_seq":21209,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:42.603655+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:08:42.603655+0000","last_peered":"2024-09-10T16:08:42.603655+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:08:42.603655+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17673,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4370},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'21082","reported_seq":21230,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:42.545975+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:08:42.545975+0000","last_peered":"2024-09-10T16:08:42.545975+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:08:42.545975+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17581,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4235},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'21314","reported_seq":21455,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:42.623054+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:08:42.623054+0000","last_peered":"2024-09-10T16:08:42.623054+0000","last_clean":"2024-09-10T16:08:42.623054+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:08:42.623054+0000","last_undegraded":"2024-09-10T16:08:42.623054+0000","last_fullsized":"2024-09-10T16:08:42.623054+0000","mapping_epoch":34,"log_start":"45'11300","ondisk_log_start":"45'11300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18161,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.838857+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:08:10.838857+0000","last_peered":"2024-09-10T16:08:10.838857+0000","last_clean":"2024-09-10T16:08:10.838857+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:08:10.838857+0000","last_undegraded":"2024-09-10T16:08:10.838857+0000","last_fullsized":"2024-09-10T16:08:10.838857+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20457","reported_seq":20608,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:42.662113+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:08:42.662113+0000","last_peered":"2024-09-10T16:08:42.662113+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:08:42.662113+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18276,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-4122}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":105,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.838234+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:08:10.838234+0000","last_peered":"2024-09-10T16:08:10.838234+0000","last_clean":"2024-09-10T16:08:10.838234+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:08:10.838234+0000","last_undegraded":"2024-09-10T16:08:10.838234+0000","last_fullsized":"2024-09-10T16:08:10.838234+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20910","reported_seq":26894,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:01.198107+0000","last_change":"2024-09-10T16:08:09.280891+0000","last_active":"2024-09-10T16:09:01.198107+0000","last_peered":"2024-09-10T16:09:01.198107+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:09:01.198107+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1336,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17414,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2921,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1336}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":115,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.840312+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:08:10.840312+0000","last_peered":"2024-09-10T16:08:10.840312+0000","last_clean":"2024-09-10T16:08:10.840312+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:08:10.840312+0000","last_undegraded":"2024-09-10T16:08:10.840312+0000","last_fullsized":"2024-09-10T16:08:10.840312+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20781","reported_seq":20927,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:42.672320+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:08:42.672320+0000","last_peered":"2024-09-10T16:08:42.672320+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:08:42.672320+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18461,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-4303}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":90,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.837937+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:08:10.837937+0000","last_peered":"2024-09-10T16:08:10.837937+0000","last_clean":"2024-09-10T16:08:10.837937+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:08:10.837937+0000","last_undegraded":"2024-09-10T16:08:10.837937+0000","last_fullsized":"2024-09-10T16:08:10.837937+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'21170","reported_seq":30025,"reported_epoch":49,"state":"active+recovery_wait","last_fresh":"2024-09-10T16:08:42.596968+0000","last_change":"2024-09-10T16:08:10.929169+0000","last_active":"2024-09-10T16:08:42.596968+0000","last_peered":"2024-09-10T16:08:42.596968+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T16:08:10.860438+0000","last_became_peered":"2024-09-10T16:08:10.860438+0000","last_unstale":"2024-09-10T16:08:42.596968+0000","last_undegraded":"2024-09-10T16:08:42.596968+0000","last_fullsized":"2024-09-10T16:08:42.596968+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18724,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","0","1"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20838","reported_seq":20983,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:42.624374+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:08:42.624374+0000","last_peered":"2024-09-10T16:08:42.624374+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:08:42.624374+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17469,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.838546+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:08:10.838546+0000","last_peered":"2024-09-10T16:08:10.838546+0000","last_clean":"2024-09-10T16:08:10.838546+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:08:10.838546+0000","last_undegraded":"2024-09-10T16:08:10.838546+0000","last_fullsized":"2024-09-10T16:08:10.838546+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20598","reported_seq":29174,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:42.620274+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:08:42.620274+0000","last_peered":"2024-09-10T16:08:42.620274+0000","last_clean":"2024-09-10T16:08:42.620274+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:08:42.620274+0000","last_undegraded":"2024-09-10T16:08:42.620274+0000","last_fullsized":"2024-09-10T16:08:42.620274+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16988,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":105,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.838971+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:08:10.838971+0000","last_peered":"2024-09-10T16:08:10.838971+0000","last_clean":"2024-09-10T16:08:10.838971+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:08:10.838971+0000","last_undegraded":"2024-09-10T16:08:10.838971+0000","last_fullsized":"2024-09-10T16:08:10.838971+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'21048","reported_seq":21201,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:42.674593+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:08:42.674593+0000","last_peered":"2024-09-10T16:08:42.674593+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:08:42.674593+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17303,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.839114+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:08:10.839114+0000","last_peered":"2024-09-10T16:08:10.839114+0000","last_clean":"2024-09-10T16:08:10.839114+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:08:10.839114+0000","last_undegraded":"2024-09-10T16:08:10.839114+0000","last_fullsized":"2024-09-10T16:08:10.839114+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":85,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.838633+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:08:10.838633+0000","last_peered":"2024-09-10T16:08:10.838633+0000","last_clean":"2024-09-10T16:08:10.838633+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:08:10.838633+0000","last_undegraded":"2024-09-10T16:08:10.838633+0000","last_fullsized":"2024-09-10T16:08:10.838633+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'21198","reported_seq":21353,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:42.668456+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:08:42.668456+0000","last_peered":"2024-09-10T16:08:42.668456+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:08:42.668456+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17361,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4321},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35636,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292096,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15754,"num_bytes_recovered":33095680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":210522112,"data_stored":210395657,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161022,"ondisk_log_size":161022,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986308,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":502592,"kb_used_data":137712,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93869248,"statfs":{"total":96636764160,"available":96122109952,"internally_reserved":0,"allocated":141017088,"data_stored":140707975,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659253,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":443104,"kb_used_data":464,"kb_used_omap":4,"kb_used_meta":442619,"kb_avail":93928736,"statfs":{"total":96636764160,"available":96183025664,"internally_reserved":0,"allocated":475136,"data_stored":143881,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":453242542},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182536,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":547292,"kb_used_data":68620,"kb_used_omap":4,"kb_used_meta":478651,"kb_avail":93824548,"statfs":{"total":96636764160,"available":96076337152,"internally_reserved":0,"allocated":70266880,"data_stored":69953690,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":490139306},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":69853184,"data_stored":69817054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":7245,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":140607488,"data_stored":140571358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:09:06.809 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:09:07.129 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:09:07.129 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:09:07.142 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":397,"stamp":"2024-09-10T16:09:05.769341+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35636,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292098,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15756,"num_bytes_recovered":33095699,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161025,"ondisk_log_size":161025,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1519356,"kb_used_data":206796,"kb_used_omap":12,"kb_used_meta":1312499,"kb_avail":281596164,"statfs":{"total":289910292480,"available":288354471936,"internally_reserved":0,"allocated":211759104,"data_stored":210805546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1343998978},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1457,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1457,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002372"},"pg_stats":[{"pgid":"2.f","version":"49'20672","reported_seq":20757,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:42.659151+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:08:42.659151+0000","last_peered":"2024-09-10T16:08:42.659151+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:08:42.659151+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20672,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4259}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20844","reported_seq":20967,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:42.591606+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:08:42.591606+0000","last_peered":"2024-09-10T16:08:42.591606+0000","last_clean":"2024-09-10T16:08:42.591606+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:08:42.591606+0000","last_undegraded":"2024-09-10T16:08:42.591606+0000","last_fullsized":"2024-09-10T16:08:42.591606+0000","mapping_epoch":35,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18949,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20798","reported_seq":29487,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:42.681302+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:08:42.681302+0000","last_peered":"2024-09-10T16:08:42.681302+0000","last_clean":"2024-09-10T16:08:42.681302+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:08:42.681302+0000","last_undegraded":"2024-09-10T16:08:42.681302+0000","last_fullsized":"2024-09-10T16:08:42.681302+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18802,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20660","reported_seq":20751,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:42.649157+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:08:42.649157+0000","last_peered":"2024-09-10T16:08:42.649157+0000","last_clean":"2024-09-10T16:08:42.649157+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:08:42.649157+0000","last_undegraded":"2024-09-10T16:08:42.649157+0000","last_fullsized":"2024-09-10T16:08:42.649157+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20660,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20891","reported_seq":21032,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:42.663199+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:08:42.663199+0000","last_peered":"2024-09-10T16:08:42.663199+0000","last_clean":"2024-09-10T16:08:42.663199+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:08:42.663199+0000","last_undegraded":"2024-09-10T16:08:42.663199+0000","last_fullsized":"2024-09-10T16:08:42.663199+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17602,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'21061","reported_seq":21209,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:42.603655+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:08:42.603655+0000","last_peered":"2024-09-10T16:08:42.603655+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:08:42.603655+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17673,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4370},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'21082","reported_seq":21230,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:42.545975+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:08:42.545975+0000","last_peered":"2024-09-10T16:08:42.545975+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:08:42.545975+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17581,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4235},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'21314","reported_seq":21455,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:42.623054+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:08:42.623054+0000","last_peered":"2024-09-10T16:08:42.623054+0000","last_clean":"2024-09-10T16:08:42.623054+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:08:42.623054+0000","last_undegraded":"2024-09-10T16:08:42.623054+0000","last_fullsized":"2024-09-10T16:08:42.623054+0000","mapping_epoch":34,"log_start":"45'11300","ondisk_log_start":"45'11300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18161,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.838857+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:08:10.838857+0000","last_peered":"2024-09-10T16:08:10.838857+0000","last_clean":"2024-09-10T16:08:10.838857+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:08:10.838857+0000","last_undegraded":"2024-09-10T16:08:10.838857+0000","last_fullsized":"2024-09-10T16:08:10.838857+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20457","reported_seq":20608,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:42.662113+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:08:42.662113+0000","last_peered":"2024-09-10T16:08:42.662113+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:08:42.662113+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18276,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-4122}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":105,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.838234+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:08:10.838234+0000","last_peered":"2024-09-10T16:08:10.838234+0000","last_clean":"2024-09-10T16:08:10.838234+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:08:10.838234+0000","last_undegraded":"2024-09-10T16:08:10.838234+0000","last_fullsized":"2024-09-10T16:08:10.838234+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20910","reported_seq":26894,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:01.198107+0000","last_change":"2024-09-10T16:08:09.280891+0000","last_active":"2024-09-10T16:09:01.198107+0000","last_peered":"2024-09-10T16:09:01.198107+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:09:01.198107+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1336,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17414,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2921,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1336}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":115,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.840312+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:08:10.840312+0000","last_peered":"2024-09-10T16:08:10.840312+0000","last_clean":"2024-09-10T16:08:10.840312+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:08:10.840312+0000","last_undegraded":"2024-09-10T16:08:10.840312+0000","last_fullsized":"2024-09-10T16:08:10.840312+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20781","reported_seq":20927,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:42.672320+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:08:42.672320+0000","last_peered":"2024-09-10T16:08:42.672320+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:08:42.672320+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18461,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-4303}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":90,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.837937+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:08:10.837937+0000","last_peered":"2024-09-10T16:08:10.837937+0000","last_clean":"2024-09-10T16:08:10.837937+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:08:10.837937+0000","last_undegraded":"2024-09-10T16:08:10.837937+0000","last_fullsized":"2024-09-10T16:08:10.837937+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'21170","reported_seq":30025,"reported_epoch":49,"state":"active+recovery_wait","last_fresh":"2024-09-10T16:08:42.596968+0000","last_change":"2024-09-10T16:08:10.929169+0000","last_active":"2024-09-10T16:08:42.596968+0000","last_peered":"2024-09-10T16:08:42.596968+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T16:08:10.860438+0000","last_became_peered":"2024-09-10T16:08:10.860438+0000","last_unstale":"2024-09-10T16:08:42.596968+0000","last_undegraded":"2024-09-10T16:08:42.596968+0000","last_fullsized":"2024-09-10T16:08:42.596968+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18724,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","0","1"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20838","reported_seq":20983,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:42.624374+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:08:42.624374+0000","last_peered":"2024-09-10T16:08:42.624374+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:08:42.624374+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17469,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.838546+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:08:10.838546+0000","last_peered":"2024-09-10T16:08:10.838546+0000","last_clean":"2024-09-10T16:08:10.838546+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:08:10.838546+0000","last_undegraded":"2024-09-10T16:08:10.838546+0000","last_fullsized":"2024-09-10T16:08:10.838546+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20598","reported_seq":29174,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:42.620274+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:08:42.620274+0000","last_peered":"2024-09-10T16:08:42.620274+0000","last_clean":"2024-09-10T16:08:42.620274+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:08:42.620274+0000","last_undegraded":"2024-09-10T16:08:42.620274+0000","last_fullsized":"2024-09-10T16:08:42.620274+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16988,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":105,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.838971+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:08:10.838971+0000","last_peered":"2024-09-10T16:08:10.838971+0000","last_clean":"2024-09-10T16:08:10.838971+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:08:10.838971+0000","last_undegraded":"2024-09-10T16:08:10.838971+0000","last_fullsized":"2024-09-10T16:08:10.838971+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'21048","reported_seq":21201,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:42.674593+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:08:42.674593+0000","last_peered":"2024-09-10T16:08:42.674593+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:08:42.674593+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17303,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.839114+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:08:10.839114+0000","last_peered":"2024-09-10T16:08:10.839114+0000","last_clean":"2024-09-10T16:08:10.839114+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:08:10.839114+0000","last_undegraded":"2024-09-10T16:08:10.839114+0000","last_fullsized":"2024-09-10T16:08:10.839114+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":85,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.838633+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:08:10.838633+0000","last_peered":"2024-09-10T16:08:10.838633+0000","last_clean":"2024-09-10T16:08:10.838633+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:08:10.838633+0000","last_undegraded":"2024-09-10T16:08:10.838633+0000","last_fullsized":"2024-09-10T16:08:10.838633+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'21198","reported_seq":21353,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:42.668456+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:08:42.668456+0000","last_peered":"2024-09-10T16:08:42.668456+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:08:42.668456+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17361,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4321},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35636,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292096,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15754,"num_bytes_recovered":33095680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":210522112,"data_stored":210395657,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161022,"ondisk_log_size":161022,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986308,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":502592,"kb_used_data":137712,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93869248,"statfs":{"total":96636764160,"available":96122109952,"internally_reserved":0,"allocated":141017088,"data_stored":140707975,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659254,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":469472,"kb_used_data":464,"kb_used_omap":4,"kb_used_meta":468987,"kb_avail":93902368,"statfs":{"total":96636764160,"available":96156024832,"internally_reserved":0,"allocated":475136,"data_stored":143881,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":480243374},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182536,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":547292,"kb_used_data":68620,"kb_used_omap":4,"kb_used_meta":478651,"kb_avail":93824548,"statfs":{"total":96636764160,"available":96076337152,"internally_reserved":0,"allocated":70266880,"data_stored":69953690,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":490139306},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":69853184,"data_stored":69817054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":7245,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":140607488,"data_stored":140571358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:09:07.143 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:09:07.445 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:09:07.445 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:09:07.457 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":397,"stamp":"2024-09-10T16:09:05.769341+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35636,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292098,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15756,"num_bytes_recovered":33095699,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161025,"ondisk_log_size":161025,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1519356,"kb_used_data":206796,"kb_used_omap":12,"kb_used_meta":1312499,"kb_avail":281596164,"statfs":{"total":289910292480,"available":288354471936,"internally_reserved":0,"allocated":211759104,"data_stored":210805546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1343998978},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1457,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1457,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002372"},"pg_stats":[{"pgid":"2.f","version":"49'20672","reported_seq":20757,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:42.659151+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:08:42.659151+0000","last_peered":"2024-09-10T16:08:42.659151+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:08:42.659151+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20672,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4259}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20844","reported_seq":20967,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:42.591606+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:08:42.591606+0000","last_peered":"2024-09-10T16:08:42.591606+0000","last_clean":"2024-09-10T16:08:42.591606+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:08:42.591606+0000","last_undegraded":"2024-09-10T16:08:42.591606+0000","last_fullsized":"2024-09-10T16:08:42.591606+0000","mapping_epoch":35,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18949,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20798","reported_seq":29487,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:42.681302+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:08:42.681302+0000","last_peered":"2024-09-10T16:08:42.681302+0000","last_clean":"2024-09-10T16:08:42.681302+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:08:42.681302+0000","last_undegraded":"2024-09-10T16:08:42.681302+0000","last_fullsized":"2024-09-10T16:08:42.681302+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18802,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20660","reported_seq":20751,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:42.649157+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:08:42.649157+0000","last_peered":"2024-09-10T16:08:42.649157+0000","last_clean":"2024-09-10T16:08:42.649157+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:08:42.649157+0000","last_undegraded":"2024-09-10T16:08:42.649157+0000","last_fullsized":"2024-09-10T16:08:42.649157+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20660,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20891","reported_seq":21032,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:42.663199+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:08:42.663199+0000","last_peered":"2024-09-10T16:08:42.663199+0000","last_clean":"2024-09-10T16:08:42.663199+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:08:42.663199+0000","last_undegraded":"2024-09-10T16:08:42.663199+0000","last_fullsized":"2024-09-10T16:08:42.663199+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17602,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'21061","reported_seq":21209,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:42.603655+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:08:42.603655+0000","last_peered":"2024-09-10T16:08:42.603655+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:08:42.603655+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17673,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4370},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'21082","reported_seq":21230,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:42.545975+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:08:42.545975+0000","last_peered":"2024-09-10T16:08:42.545975+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:08:42.545975+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17581,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4235},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'21314","reported_seq":21455,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:42.623054+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:08:42.623054+0000","last_peered":"2024-09-10T16:08:42.623054+0000","last_clean":"2024-09-10T16:08:42.623054+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:08:42.623054+0000","last_undegraded":"2024-09-10T16:08:42.623054+0000","last_fullsized":"2024-09-10T16:08:42.623054+0000","mapping_epoch":34,"log_start":"45'11300","ondisk_log_start":"45'11300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18161,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.838857+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:08:10.838857+0000","last_peered":"2024-09-10T16:08:10.838857+0000","last_clean":"2024-09-10T16:08:10.838857+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:08:10.838857+0000","last_undegraded":"2024-09-10T16:08:10.838857+0000","last_fullsized":"2024-09-10T16:08:10.838857+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20457","reported_seq":20608,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:42.662113+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:08:42.662113+0000","last_peered":"2024-09-10T16:08:42.662113+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:08:42.662113+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18276,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-4122}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":105,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.838234+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:08:10.838234+0000","last_peered":"2024-09-10T16:08:10.838234+0000","last_clean":"2024-09-10T16:08:10.838234+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:08:10.838234+0000","last_undegraded":"2024-09-10T16:08:10.838234+0000","last_fullsized":"2024-09-10T16:08:10.838234+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20910","reported_seq":26894,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:01.198107+0000","last_change":"2024-09-10T16:08:09.280891+0000","last_active":"2024-09-10T16:09:01.198107+0000","last_peered":"2024-09-10T16:09:01.198107+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:09:01.198107+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1336,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17414,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2921,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1336}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":115,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.840312+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:08:10.840312+0000","last_peered":"2024-09-10T16:08:10.840312+0000","last_clean":"2024-09-10T16:08:10.840312+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:08:10.840312+0000","last_undegraded":"2024-09-10T16:08:10.840312+0000","last_fullsized":"2024-09-10T16:08:10.840312+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20781","reported_seq":20927,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:42.672320+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:08:42.672320+0000","last_peered":"2024-09-10T16:08:42.672320+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:08:42.672320+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18461,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-4303}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":90,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.837937+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:08:10.837937+0000","last_peered":"2024-09-10T16:08:10.837937+0000","last_clean":"2024-09-10T16:08:10.837937+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:08:10.837937+0000","last_undegraded":"2024-09-10T16:08:10.837937+0000","last_fullsized":"2024-09-10T16:08:10.837937+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'21170","reported_seq":30025,"reported_epoch":49,"state":"active+recovery_wait","last_fresh":"2024-09-10T16:08:42.596968+0000","last_change":"2024-09-10T16:08:10.929169+0000","last_active":"2024-09-10T16:08:42.596968+0000","last_peered":"2024-09-10T16:08:42.596968+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T16:08:10.860438+0000","last_became_peered":"2024-09-10T16:08:10.860438+0000","last_unstale":"2024-09-10T16:08:42.596968+0000","last_undegraded":"2024-09-10T16:08:42.596968+0000","last_fullsized":"2024-09-10T16:08:42.596968+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18724,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","0","1"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20838","reported_seq":20983,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:42.624374+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:08:42.624374+0000","last_peered":"2024-09-10T16:08:42.624374+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:08:42.624374+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17469,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.838546+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:08:10.838546+0000","last_peered":"2024-09-10T16:08:10.838546+0000","last_clean":"2024-09-10T16:08:10.838546+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:08:10.838546+0000","last_undegraded":"2024-09-10T16:08:10.838546+0000","last_fullsized":"2024-09-10T16:08:10.838546+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20598","reported_seq":29174,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:42.620274+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:08:42.620274+0000","last_peered":"2024-09-10T16:08:42.620274+0000","last_clean":"2024-09-10T16:08:42.620274+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:08:42.620274+0000","last_undegraded":"2024-09-10T16:08:42.620274+0000","last_fullsized":"2024-09-10T16:08:42.620274+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16988,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":105,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.838971+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:08:10.838971+0000","last_peered":"2024-09-10T16:08:10.838971+0000","last_clean":"2024-09-10T16:08:10.838971+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:08:10.838971+0000","last_undegraded":"2024-09-10T16:08:10.838971+0000","last_fullsized":"2024-09-10T16:08:10.838971+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'21048","reported_seq":21201,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:42.674593+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:08:42.674593+0000","last_peered":"2024-09-10T16:08:42.674593+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:08:42.674593+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17303,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.839114+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:08:10.839114+0000","last_peered":"2024-09-10T16:08:10.839114+0000","last_clean":"2024-09-10T16:08:10.839114+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:08:10.839114+0000","last_undegraded":"2024-09-10T16:08:10.839114+0000","last_fullsized":"2024-09-10T16:08:10.839114+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":85,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.838633+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:08:10.838633+0000","last_peered":"2024-09-10T16:08:10.838633+0000","last_clean":"2024-09-10T16:08:10.838633+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:08:10.838633+0000","last_undegraded":"2024-09-10T16:08:10.838633+0000","last_fullsized":"2024-09-10T16:08:10.838633+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'21198","reported_seq":21353,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:42.668456+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:08:42.668456+0000","last_peered":"2024-09-10T16:08:42.668456+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:08:42.668456+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17361,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4321},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35636,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292096,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15754,"num_bytes_recovered":33095680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":210522112,"data_stored":210395657,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161022,"ondisk_log_size":161022,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986308,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":502592,"kb_used_data":137712,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93869248,"statfs":{"total":96636764160,"available":96122109952,"internally_reserved":0,"allocated":141017088,"data_stored":140707975,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659254,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":469472,"kb_used_data":464,"kb_used_omap":4,"kb_used_meta":468987,"kb_avail":93902368,"statfs":{"total":96636764160,"available":96156024832,"internally_reserved":0,"allocated":475136,"data_stored":143881,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":480243374},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182536,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":547292,"kb_used_data":68620,"kb_used_omap":4,"kb_used_meta":478651,"kb_avail":93824548,"statfs":{"total":96636764160,"available":96076337152,"internally_reserved":0,"allocated":70266880,"data_stored":69953690,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":490139306},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":69853184,"data_stored":69817054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":7245,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":140607488,"data_stored":140571358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:09:07.458 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-10T16:09:07.459 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "49'20672", 'reported_seq': 20757, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:08:42.659151+0000', 'last_change': '2024-09-10T15:59:16.281558+0000', 'last_active': '2024-09-10T16:08:42.659151+0000', 'last_peered': '2024-09-10T16:08:42.659151+0000', 'last_clean': '2024-09-10T15:59:14.132456+0000', 'last_became_active': '2024-09-10T15:59:16.274407+0000', 'last_became_peered': '2024-09-10T15:59:16.274407+0000', 'last_unstale': '2024-09-10T16:08:42.659151+0000', 'last_undegraded': '2024-09-10T15:59:15.851279+0000', 'last_fullsized': '2024-09-10T15:59:15.774356+0000', 'mapping_epoch': 42, 'log_start': "45'10600", 'ondisk_log_start': "45'10600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'objects_scrubbed': 0, 'log_size': 10072, 'log_dups_size': 0, 'ondisk_log_size': 10072, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4259, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20672, 'num_write_kb': 41344, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4259}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:09:07.459 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-10T16:09:07.459 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "49'21061", 'reported_seq': 21209, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:08:42.603655+0000', 'last_change': '2024-09-10T15:59:16.308264+0000', 'last_active': '2024-09-10T16:08:42.603655+0000', 'last_peered': '2024-09-10T16:08:42.603655+0000', 'last_clean': '2024-09-10T15:59:14.111551+0000', 'last_became_active': '2024-09-10T15:59:16.300040+0000', 'last_became_peered': '2024-09-10T15:59:16.300040+0000', 'last_unstale': '2024-09-10T16:08:42.603655+0000', 'last_undegraded': '2024-09-10T15:59:15.878234+0000', 'last_fullsized': '2024-09-10T15:59:15.769328+0000', 'mapping_epoch': 42, 'log_start': "45'11000", 'ondisk_log_start': "45'11000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3442", 'last_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'objects_scrubbed': 3384, 'log_size': 10061, 'log_dups_size': 0, 'ondisk_log_size': 10061, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1292, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4372, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 17673, 'num_write_kb': 28580, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4370}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:09:07.459 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-10T16:09:07.459 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "49'21082", 'reported_seq': 21230, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:08:42.545975+0000', 'last_change': '2024-09-10T15:59:16.331899+0000', 'last_active': '2024-09-10T16:08:42.545975+0000', 'last_peered': '2024-09-10T16:08:42.545975+0000', 'last_clean': '2024-09-10T15:59:14.139436+0000', 'last_became_active': '2024-09-10T15:59:16.318715+0000', 'last_became_peered': '2024-09-10T15:59:16.318715+0000', 'last_unstale': '2024-09-10T16:08:42.545975+0000', 'last_undegraded': '2024-09-10T15:59:15.877524+0000', 'last_fullsized': '2024-09-10T15:59:15.769042+0000', 'mapping_epoch': 42, 'log_start': "45'11000", 'ondisk_log_start': "45'11000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3537", 'last_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'objects_scrubbed': 3497, 'log_size': 10082, 'log_dups_size': 0, 'ondisk_log_size': 10082, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1224, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4237, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 17581, 'num_write_kb': 28172, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4235}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:09:07.459 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-10T16:09:07.459 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "49'20457", 'reported_seq': 20608, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:08:42.662113+0000', 'last_change': '2024-09-10T15:59:16.175680+0000', 'last_active': '2024-09-10T16:08:42.662113+0000', 'last_peered': '2024-09-10T16:08:42.662113+0000', 'last_clean': '2024-09-10T15:59:14.088187+0000', 'last_became_active': '2024-09-10T15:59:16.165810+0000', 'last_became_peered': '2024-09-10T15:59:16.165810+0000', 'last_unstale': '2024-09-10T16:08:42.662113+0000', 'last_undegraded': '2024-09-10T15:59:15.822098+0000', 'last_fullsized': '2024-09-10T15:59:15.769123+0000', 'mapping_epoch': 42, 'log_start': "45'10400", 'ondisk_log_start': "45'10400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2218", 'last_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'objects_scrubbed': 2175, 'log_size': 10057, 'log_dups_size': 0, 'ondisk_log_size': 10057, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 825, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4124, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 18276, 'num_write_kb': 32204, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': -4122}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:09:07.459 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-10T16:09:07.459 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "49'20910", 'reported_seq': 26894, 'reported_epoch': 49, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:09:01.198107+0000', 'last_change': '2024-09-10T16:08:09.280891+0000', 'last_active': '2024-09-10T16:09:01.198107+0000', 'last_peered': '2024-09-10T16:09:01.198107+0000', 'last_clean': '2024-09-10T15:59:14.129081+0000', 'last_became_active': '2024-09-10T15:59:16.269835+0000', 'last_became_peered': '2024-09-10T15:59:16.269835+0000', 'last_unstale': '2024-09-10T16:09:01.198107+0000', 'last_undegraded': '2024-09-10T15:59:15.860857+0000', 'last_fullsized': '2024-09-10T15:59:15.771410+0000', 'mapping_epoch': 42, 'log_start': "45'10900", 'ondisk_log_start': "45'10900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3557", 'last_scrub_stamp': '2024-09-10T15:59:02.493678+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:02.493678+0000', 'objects_scrubbed': 3488, 'log_size': 10010, 'log_dups_size': 0, 'ondisk_log_size': 10010, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1472, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1336, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17414, 'num_write_kb': 27852, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2921, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1336}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:09:07.459 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-10T16:09:07.460 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "49'20781", 'reported_seq': 20927, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:08:42.672320+0000', 'last_change': '2024-09-10T15:59:16.284311+0000', 'last_active': '2024-09-10T16:08:42.672320+0000', 'last_peered': '2024-09-10T16:08:42.672320+0000', 'last_clean': '2024-09-10T15:59:14.116041+0000', 'last_became_active': '2024-09-10T15:59:16.274543+0000', 'last_became_peered': '2024-09-10T15:59:16.274543+0000', 'last_unstale': '2024-09-10T16:08:42.672320+0000', 'last_undegraded': '2024-09-10T15:59:15.812484+0000', 'last_fullsized': '2024-09-10T15:59:15.770885+0000', 'mapping_epoch': 42, 'log_start': "45'10700", 'ondisk_log_start': "45'10700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2356", 'last_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'objects_scrubbed': 2316, 'log_size': 10081, 'log_dups_size': 0, 'ondisk_log_size': 10081, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 783, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 18461, 'num_write_kb': 32292, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': -4303}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:09:07.460 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-10T16:09:07.460 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "49'21170", 'reported_seq': 30025, 'reported_epoch': 49, 'state': 'active+recovery_wait', 'last_fresh': '2024-09-10T16:08:42.596968+0000', 'last_change': '2024-09-10T16:08:10.929169+0000', 'last_active': '2024-09-10T16:08:42.596968+0000', 'last_peered': '2024-09-10T16:08:42.596968+0000', 'last_clean': '2024-09-10T15:59:14.110654+0000', 'last_became_active': '2024-09-10T16:08:10.860438+0000', 'last_became_peered': '2024-09-10T16:08:10.860438+0000', 'last_unstale': '2024-09-10T16:08:42.596968+0000', 'last_undegraded': '2024-09-10T16:08:42.596968+0000', 'last_fullsized': '2024-09-10T16:08:42.596968+0000', 'mapping_epoch': 48, 'log_start': "45'11100", 'ondisk_log_start': "45'11100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2486", 'last_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'objects_scrubbed': 2440, 'log_size': 10070, 'log_dups_size': 0, 'ondisk_log_size': 10070, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 792, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18724, 'num_write_kb': 32568, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 4355, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': ['2', '0', '1'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:09:07.460 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-10T16:09:07.460 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "49'20838", 'reported_seq': 20983, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:08:42.624374+0000', 'last_change': '2024-09-10T15:59:16.315641+0000', 'last_active': '2024-09-10T16:08:42.624374+0000', 'last_peered': '2024-09-10T16:08:42.624374+0000', 'last_clean': '2024-09-10T15:59:14.117001+0000', 'last_became_active': '2024-09-10T15:59:16.302245+0000', 'last_became_peered': '2024-09-10T15:59:16.302245+0000', 'last_unstale': '2024-09-10T16:08:42.624374+0000', 'last_undegraded': '2024-09-10T15:59:15.845012+0000', 'last_fullsized': '2024-09-10T15:59:15.770337+0000', 'mapping_epoch': 42, 'log_start': "45'10800", 'ondisk_log_start': "45'10800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3404", 'last_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'objects_scrubbed': 3367, 'log_size': 10038, 'log_dups_size': 0, 'ondisk_log_size': 10038, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 983, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17469, 'num_write_kb': 28204, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4304}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:09:07.460 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-10T16:09:07.460 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "49'21048", 'reported_seq': 21201, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:08:42.674593+0000', 'last_change': '2024-09-10T15:59:16.311291+0000', 'last_active': '2024-09-10T16:08:42.674593+0000', 'last_peered': '2024-09-10T16:08:42.674593+0000', 'last_clean': '2024-09-10T15:59:14.116363+0000', 'last_became_active': '2024-09-10T15:59:16.301244+0000', 'last_became_peered': '2024-09-10T15:59:16.301244+0000', 'last_unstale': '2024-09-10T16:08:42.674593+0000', 'last_undegraded': '2024-09-10T15:59:15.883434+0000', 'last_fullsized': '2024-09-10T15:59:15.772756+0000', 'mapping_epoch': 42, 'log_start': "45'11000", 'ondisk_log_start': "45'11000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3789", 'last_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'objects_scrubbed': 3741, 'log_size': 10048, 'log_dups_size': 0, 'ondisk_log_size': 10048, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1440, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4373, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17303, 'num_write_kb': 27124, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4371}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:09:07.460 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-10T16:09:07.460 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "49'21198", 'reported_seq': 21353, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:08:42.668456+0000', 'last_change': '2024-09-10T15:59:16.313409+0000', 'last_active': '2024-09-10T16:08:42.668456+0000', 'last_peered': '2024-09-10T16:08:42.668456+0000', 'last_clean': '2024-09-10T15:59:14.129780+0000', 'last_became_active': '2024-09-10T15:59:16.300932+0000', 'last_became_peered': '2024-09-10T15:59:16.300932+0000', 'last_unstale': '2024-09-10T16:08:42.668456+0000', 'last_undegraded': '2024-09-10T15:59:15.887823+0000', 'last_fullsized': '2024-09-10T15:59:15.768513+0000', 'mapping_epoch': 42, 'log_start': "45'11100", 'ondisk_log_start': "45'11100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3880", 'last_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'objects_scrubbed': 3827, 'log_size': 10098, 'log_dups_size': 0, 'ondisk_log_size': 10098, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1397, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4325, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17361, 'num_write_kb': 27072, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4321}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:09:07.461 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T16:09:07.843 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:09:07.843 INFO:teuthology.orchestra.run.smithi115.stdout:{"fsid":"7effee29-df05-4351-9741-ee203564ed81","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 34592/24 objects degraded (144133.333%), 9 pgs degraded, 9 pgs undersized","count":18},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":711,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":49,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983954,"num_in_osds":3,"osd_in_since":1725983947,"num_remapped_pgs":9},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":8},{"state_name":"active+recovery_wait","count":1},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":12,"data_bytes":4849,"bytes_used":1555820544,"bytes_avail":288354471936,"bytes_total":289910292480,"degraded_objects":34592,"degraded_total":24,"degraded_ratio":1441.333333333333,"recovering_objects_per_sec":208,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":2501,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-10T15:57:14:967120+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T16:00:13.560776+0000","services":{}},"progress_events":{}} 2024-09-10T16:09:07.865 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-10T16:09:07.865 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:09:08.239 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:09:08.239 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:09:08.254 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":399,"stamp":"2024-09-10T16:09:07.844278+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34592,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292098,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16800,"num_bytes_recovered":33095699,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161025,"ondisk_log_size":161025,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1519356,"kb_used_data":206796,"kb_used_omap":12,"kb_used_meta":1312499,"kb_avail":281596164,"statfs":{"total":289910292480,"available":288354471936,"internally_reserved":0,"allocated":211759104,"data_stored":210805546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1343998978},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1828,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1828,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.076513"},"pg_stats":[{"pgid":"2.f","version":"49'20672","reported_seq":20757,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:42.659151+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:08:42.659151+0000","last_peered":"2024-09-10T16:08:42.659151+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:08:42.659151+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20672,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4259}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20844","reported_seq":20967,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:42.591606+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:08:42.591606+0000","last_peered":"2024-09-10T16:08:42.591606+0000","last_clean":"2024-09-10T16:08:42.591606+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:08:42.591606+0000","last_undegraded":"2024-09-10T16:08:42.591606+0000","last_fullsized":"2024-09-10T16:08:42.591606+0000","mapping_epoch":35,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18949,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20798","reported_seq":29487,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:42.681302+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:08:42.681302+0000","last_peered":"2024-09-10T16:08:42.681302+0000","last_clean":"2024-09-10T16:08:42.681302+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:08:42.681302+0000","last_undegraded":"2024-09-10T16:08:42.681302+0000","last_fullsized":"2024-09-10T16:08:42.681302+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18802,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20660","reported_seq":20751,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:42.649157+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:08:42.649157+0000","last_peered":"2024-09-10T16:08:42.649157+0000","last_clean":"2024-09-10T16:08:42.649157+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:08:42.649157+0000","last_undegraded":"2024-09-10T16:08:42.649157+0000","last_fullsized":"2024-09-10T16:08:42.649157+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20660,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20891","reported_seq":21032,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:42.663199+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:08:42.663199+0000","last_peered":"2024-09-10T16:08:42.663199+0000","last_clean":"2024-09-10T16:08:42.663199+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:08:42.663199+0000","last_undegraded":"2024-09-10T16:08:42.663199+0000","last_fullsized":"2024-09-10T16:08:42.663199+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17602,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'21061","reported_seq":21209,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:42.603655+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:08:42.603655+0000","last_peered":"2024-09-10T16:08:42.603655+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:08:42.603655+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17673,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4370},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'21082","reported_seq":21230,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:42.545975+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:08:42.545975+0000","last_peered":"2024-09-10T16:08:42.545975+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:08:42.545975+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17581,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4235},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'21314","reported_seq":21455,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:42.623054+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:08:42.623054+0000","last_peered":"2024-09-10T16:08:42.623054+0000","last_clean":"2024-09-10T16:08:42.623054+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:08:42.623054+0000","last_undegraded":"2024-09-10T16:08:42.623054+0000","last_fullsized":"2024-09-10T16:08:42.623054+0000","mapping_epoch":34,"log_start":"45'11300","ondisk_log_start":"45'11300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18161,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.838857+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:08:10.838857+0000","last_peered":"2024-09-10T16:08:10.838857+0000","last_clean":"2024-09-10T16:08:10.838857+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:08:10.838857+0000","last_undegraded":"2024-09-10T16:08:10.838857+0000","last_fullsized":"2024-09-10T16:08:10.838857+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20457","reported_seq":20608,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:42.662113+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:08:42.662113+0000","last_peered":"2024-09-10T16:08:42.662113+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:08:42.662113+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18276,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-4122}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":105,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.838234+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:08:10.838234+0000","last_peered":"2024-09-10T16:08:10.838234+0000","last_clean":"2024-09-10T16:08:10.838234+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:08:10.838234+0000","last_undegraded":"2024-09-10T16:08:10.838234+0000","last_fullsized":"2024-09-10T16:08:10.838234+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20910","reported_seq":28982,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:06.199847+0000","last_change":"2024-09-10T16:08:09.280891+0000","last_active":"2024-09-10T16:09:06.199847+0000","last_peered":"2024-09-10T16:09:06.199847+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:09:06.199847+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":292,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17414,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3965,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-292}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":115,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.840312+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:08:10.840312+0000","last_peered":"2024-09-10T16:08:10.840312+0000","last_clean":"2024-09-10T16:08:10.840312+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:08:10.840312+0000","last_undegraded":"2024-09-10T16:08:10.840312+0000","last_fullsized":"2024-09-10T16:08:10.840312+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20781","reported_seq":20927,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:42.672320+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:08:42.672320+0000","last_peered":"2024-09-10T16:08:42.672320+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:08:42.672320+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18461,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-4303}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":90,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.837937+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:08:10.837937+0000","last_peered":"2024-09-10T16:08:10.837937+0000","last_clean":"2024-09-10T16:08:10.837937+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:08:10.837937+0000","last_undegraded":"2024-09-10T16:08:10.837937+0000","last_fullsized":"2024-09-10T16:08:10.837937+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'21170","reported_seq":30025,"reported_epoch":49,"state":"active+recovery_wait","last_fresh":"2024-09-10T16:08:42.596968+0000","last_change":"2024-09-10T16:08:10.929169+0000","last_active":"2024-09-10T16:08:42.596968+0000","last_peered":"2024-09-10T16:08:42.596968+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T16:08:10.860438+0000","last_became_peered":"2024-09-10T16:08:10.860438+0000","last_unstale":"2024-09-10T16:08:42.596968+0000","last_undegraded":"2024-09-10T16:08:42.596968+0000","last_fullsized":"2024-09-10T16:08:42.596968+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18724,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","0","1"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20838","reported_seq":20983,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:42.624374+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:08:42.624374+0000","last_peered":"2024-09-10T16:08:42.624374+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:08:42.624374+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17469,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.838546+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:08:10.838546+0000","last_peered":"2024-09-10T16:08:10.838546+0000","last_clean":"2024-09-10T16:08:10.838546+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:08:10.838546+0000","last_undegraded":"2024-09-10T16:08:10.838546+0000","last_fullsized":"2024-09-10T16:08:10.838546+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20598","reported_seq":29174,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:42.620274+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:08:42.620274+0000","last_peered":"2024-09-10T16:08:42.620274+0000","last_clean":"2024-09-10T16:08:42.620274+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:08:42.620274+0000","last_undegraded":"2024-09-10T16:08:42.620274+0000","last_fullsized":"2024-09-10T16:08:42.620274+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16988,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":105,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.838971+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:08:10.838971+0000","last_peered":"2024-09-10T16:08:10.838971+0000","last_clean":"2024-09-10T16:08:10.838971+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:08:10.838971+0000","last_undegraded":"2024-09-10T16:08:10.838971+0000","last_fullsized":"2024-09-10T16:08:10.838971+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'21048","reported_seq":21201,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:42.674593+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:08:42.674593+0000","last_peered":"2024-09-10T16:08:42.674593+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:08:42.674593+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17303,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.839114+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:08:10.839114+0000","last_peered":"2024-09-10T16:08:10.839114+0000","last_clean":"2024-09-10T16:08:10.839114+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:08:10.839114+0000","last_undegraded":"2024-09-10T16:08:10.839114+0000","last_fullsized":"2024-09-10T16:08:10.839114+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":85,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.838633+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:08:10.838633+0000","last_peered":"2024-09-10T16:08:10.838633+0000","last_clean":"2024-09-10T16:08:10.838633+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:08:10.838633+0000","last_undegraded":"2024-09-10T16:08:10.838633+0000","last_fullsized":"2024-09-10T16:08:10.838633+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'21198","reported_seq":21353,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:42.668456+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:08:42.668456+0000","last_peered":"2024-09-10T16:08:42.668456+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:08:42.668456+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17361,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4321},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34592,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292096,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16798,"num_bytes_recovered":33095680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":210522112,"data_stored":210395657,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161022,"ondisk_log_size":161022,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986309,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":502592,"kb_used_data":137712,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93869248,"statfs":{"total":96636764160,"available":96122109952,"internally_reserved":0,"allocated":141017088,"data_stored":140707975,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659254,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":469472,"kb_used_data":464,"kb_used_omap":4,"kb_used_meta":468987,"kb_avail":93902368,"statfs":{"total":96636764160,"available":96156024832,"internally_reserved":0,"allocated":475136,"data_stored":143881,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":480243374},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182537,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":547292,"kb_used_data":68620,"kb_used_omap":4,"kb_used_meta":478651,"kb_avail":93824548,"statfs":{"total":96636764160,"available":96076337152,"internally_reserved":0,"allocated":70266880,"data_stored":69953690,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":490139306},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":69853184,"data_stored":69817054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":7245,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":140607488,"data_stored":140571358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:09:11.256 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:09:11.611 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:09:11.611 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:09:11.624 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":401,"stamp":"2024-09-10T16:09:09.770089+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34592,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292098,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16800,"num_bytes_recovered":33095699,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161025,"ondisk_log_size":161025,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1519356,"kb_used_data":206796,"kb_used_omap":12,"kb_used_meta":1312499,"kb_avail":281596164,"statfs":{"total":289910292480,"available":288354471936,"internally_reserved":0,"allocated":211759104,"data_stored":210805546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1343998978},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1044,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1044,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001587"},"pg_stats":[{"pgid":"2.f","version":"49'20672","reported_seq":20757,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:42.659151+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:08:42.659151+0000","last_peered":"2024-09-10T16:08:42.659151+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:08:42.659151+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20672,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4259}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20844","reported_seq":20967,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:42.591606+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:08:42.591606+0000","last_peered":"2024-09-10T16:08:42.591606+0000","last_clean":"2024-09-10T16:08:42.591606+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:08:42.591606+0000","last_undegraded":"2024-09-10T16:08:42.591606+0000","last_fullsized":"2024-09-10T16:08:42.591606+0000","mapping_epoch":35,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18949,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20798","reported_seq":29487,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:42.681302+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:08:42.681302+0000","last_peered":"2024-09-10T16:08:42.681302+0000","last_clean":"2024-09-10T16:08:42.681302+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:08:42.681302+0000","last_undegraded":"2024-09-10T16:08:42.681302+0000","last_fullsized":"2024-09-10T16:08:42.681302+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18802,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20660","reported_seq":20751,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:42.649157+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:08:42.649157+0000","last_peered":"2024-09-10T16:08:42.649157+0000","last_clean":"2024-09-10T16:08:42.649157+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:08:42.649157+0000","last_undegraded":"2024-09-10T16:08:42.649157+0000","last_fullsized":"2024-09-10T16:08:42.649157+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20660,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20891","reported_seq":21032,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:42.663199+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:08:42.663199+0000","last_peered":"2024-09-10T16:08:42.663199+0000","last_clean":"2024-09-10T16:08:42.663199+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:08:42.663199+0000","last_undegraded":"2024-09-10T16:08:42.663199+0000","last_fullsized":"2024-09-10T16:08:42.663199+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17602,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'21061","reported_seq":21209,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:42.603655+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:08:42.603655+0000","last_peered":"2024-09-10T16:08:42.603655+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:08:42.603655+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17673,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4370},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'21082","reported_seq":21230,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:42.545975+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:08:42.545975+0000","last_peered":"2024-09-10T16:08:42.545975+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:08:42.545975+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17581,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4235},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'21314","reported_seq":21455,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:42.623054+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:08:42.623054+0000","last_peered":"2024-09-10T16:08:42.623054+0000","last_clean":"2024-09-10T16:08:42.623054+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:08:42.623054+0000","last_undegraded":"2024-09-10T16:08:42.623054+0000","last_fullsized":"2024-09-10T16:08:42.623054+0000","mapping_epoch":34,"log_start":"45'11300","ondisk_log_start":"45'11300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18161,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.838857+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:08:10.838857+0000","last_peered":"2024-09-10T16:08:10.838857+0000","last_clean":"2024-09-10T16:08:10.838857+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:08:10.838857+0000","last_undegraded":"2024-09-10T16:08:10.838857+0000","last_fullsized":"2024-09-10T16:08:10.838857+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20457","reported_seq":20608,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:42.662113+0000","last_change":"2024-09-10T15:59:16.175680+0000","last_active":"2024-09-10T16:08:42.662113+0000","last_peered":"2024-09-10T16:08:42.662113+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:08:42.662113+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18276,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-4122}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":105,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.838234+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:08:10.838234+0000","last_peered":"2024-09-10T16:08:10.838234+0000","last_clean":"2024-09-10T16:08:10.838234+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:08:10.838234+0000","last_undegraded":"2024-09-10T16:08:10.838234+0000","last_fullsized":"2024-09-10T16:08:10.838234+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20910","reported_seq":28982,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:06.199847+0000","last_change":"2024-09-10T16:08:09.280891+0000","last_active":"2024-09-10T16:09:06.199847+0000","last_peered":"2024-09-10T16:09:06.199847+0000","last_clean":"2024-09-10T15:59:14.129081+0000","last_became_active":"2024-09-10T15:59:16.269835+0000","last_became_peered":"2024-09-10T15:59:16.269835+0000","last_unstale":"2024-09-10T16:09:06.199847+0000","last_undegraded":"2024-09-10T15:59:15.860857+0000","last_fullsized":"2024-09-10T15:59:15.771410+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":292,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17414,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3965,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-292}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":115,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.840312+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:08:10.840312+0000","last_peered":"2024-09-10T16:08:10.840312+0000","last_clean":"2024-09-10T16:08:10.840312+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:08:10.840312+0000","last_undegraded":"2024-09-10T16:08:10.840312+0000","last_fullsized":"2024-09-10T16:08:10.840312+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20781","reported_seq":20927,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:42.672320+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:08:42.672320+0000","last_peered":"2024-09-10T16:08:42.672320+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:08:42.672320+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18461,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-4303}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":90,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.837937+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:08:10.837937+0000","last_peered":"2024-09-10T16:08:10.837937+0000","last_clean":"2024-09-10T16:08:10.837937+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:08:10.837937+0000","last_undegraded":"2024-09-10T16:08:10.837937+0000","last_fullsized":"2024-09-10T16:08:10.837937+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'21170","reported_seq":30025,"reported_epoch":49,"state":"active+recovery_wait","last_fresh":"2024-09-10T16:08:42.596968+0000","last_change":"2024-09-10T16:08:10.929169+0000","last_active":"2024-09-10T16:08:42.596968+0000","last_peered":"2024-09-10T16:08:42.596968+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T16:08:10.860438+0000","last_became_peered":"2024-09-10T16:08:10.860438+0000","last_unstale":"2024-09-10T16:08:42.596968+0000","last_undegraded":"2024-09-10T16:08:42.596968+0000","last_fullsized":"2024-09-10T16:08:42.596968+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18724,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","0","1"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20838","reported_seq":20983,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:42.624374+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:08:42.624374+0000","last_peered":"2024-09-10T16:08:42.624374+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:08:42.624374+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17469,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.838546+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:08:10.838546+0000","last_peered":"2024-09-10T16:08:10.838546+0000","last_clean":"2024-09-10T16:08:10.838546+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:08:10.838546+0000","last_undegraded":"2024-09-10T16:08:10.838546+0000","last_fullsized":"2024-09-10T16:08:10.838546+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20598","reported_seq":29174,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:42.620274+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:08:42.620274+0000","last_peered":"2024-09-10T16:08:42.620274+0000","last_clean":"2024-09-10T16:08:42.620274+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:08:42.620274+0000","last_undegraded":"2024-09-10T16:08:42.620274+0000","last_fullsized":"2024-09-10T16:08:42.620274+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16988,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":105,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.838971+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:08:10.838971+0000","last_peered":"2024-09-10T16:08:10.838971+0000","last_clean":"2024-09-10T16:08:10.838971+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:08:10.838971+0000","last_undegraded":"2024-09-10T16:08:10.838971+0000","last_fullsized":"2024-09-10T16:08:10.838971+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'21048","reported_seq":21201,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:42.674593+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:08:42.674593+0000","last_peered":"2024-09-10T16:08:42.674593+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:08:42.674593+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17303,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.839114+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:08:10.839114+0000","last_peered":"2024-09-10T16:08:10.839114+0000","last_clean":"2024-09-10T16:08:10.839114+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:08:10.839114+0000","last_undegraded":"2024-09-10T16:08:10.839114+0000","last_fullsized":"2024-09-10T16:08:10.839114+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":85,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-10T16:08:10.838633+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:08:10.838633+0000","last_peered":"2024-09-10T16:08:10.838633+0000","last_clean":"2024-09-10T16:08:10.838633+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:08:10.838633+0000","last_undegraded":"2024-09-10T16:08:10.838633+0000","last_fullsized":"2024-09-10T16:08:10.838633+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'21198","reported_seq":21353,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:08:42.668456+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:08:42.668456+0000","last_peered":"2024-09-10T16:08:42.668456+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:08:42.668456+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17361,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4321},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34592,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292096,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16798,"num_bytes_recovered":33095680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":210522112,"data_stored":210395657,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161022,"ondisk_log_size":161022,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986309,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":502592,"kb_used_data":137712,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93869248,"statfs":{"total":96636764160,"available":96122109952,"internally_reserved":0,"allocated":141017088,"data_stored":140707975,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659254,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":469472,"kb_used_data":464,"kb_used_omap":4,"kb_used_meta":468987,"kb_avail":93902368,"statfs":{"total":96636764160,"available":96156024832,"internally_reserved":0,"allocated":475136,"data_stored":143881,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":480243374},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182537,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":547292,"kb_used_data":68620,"kb_used_omap":4,"kb_used_meta":478651,"kb_avail":93824548,"statfs":{"total":96636764160,"available":96076337152,"internally_reserved":0,"allocated":70266880,"data_stored":69953690,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":490139306},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":69853184,"data_stored":69817054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":7245,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":140607488,"data_stored":140571358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:09:11.625 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:09:12.002 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:09:12.002 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:09:12.014 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":402,"stamp":"2024-09-10T16:09:11.770440+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33683,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292098,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17709,"num_bytes_recovered":33096182,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161025,"ondisk_log_size":161025,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1511100,"kb_used_data":206840,"kb_used_omap":12,"kb_used_meta":1304179,"kb_avail":281604420,"statfs":{"total":289910292480,"available":288362926080,"internally_reserved":0,"allocated":211804160,"data_stored":210822562,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1335479298},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1953,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001518"},"pg_stats":[{"pgid":"2.f","version":"49'20672","reported_seq":20761,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:08.850557+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:09:08.850557+0000","last_peered":"2024-09-10T16:09:08.850557+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:09:08.850557+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20672,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4259}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20844","reported_seq":20971,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.853495+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:09:08.853495+0000","last_peered":"2024-09-10T16:09:08.853495+0000","last_clean":"2024-09-10T16:09:08.853495+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:09:08.853495+0000","last_undegraded":"2024-09-10T16:09:08.853495+0000","last_fullsized":"2024-09-10T16:09:08.853495+0000","mapping_epoch":35,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18949,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20798","reported_seq":29491,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.852143+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:09:08.852143+0000","last_peered":"2024-09-10T16:09:08.852143+0000","last_clean":"2024-09-10T16:09:08.852143+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:09:08.852143+0000","last_undegraded":"2024-09-10T16:09:08.852143+0000","last_fullsized":"2024-09-10T16:09:08.852143+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18802,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20660","reported_seq":20755,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.853148+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:09:08.853148+0000","last_peered":"2024-09-10T16:09:08.853148+0000","last_clean":"2024-09-10T16:09:08.853148+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:09:08.853148+0000","last_undegraded":"2024-09-10T16:09:08.853148+0000","last_fullsized":"2024-09-10T16:09:08.853148+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20660,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20891","reported_seq":21036,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.847836+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:09:08.847836+0000","last_peered":"2024-09-10T16:09:08.847836+0000","last_clean":"2024-09-10T16:09:08.847836+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:09:08.847836+0000","last_undegraded":"2024-09-10T16:09:08.847836+0000","last_fullsized":"2024-09-10T16:09:08.847836+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17602,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'21061","reported_seq":21214,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:08.849450+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:09:08.849450+0000","last_peered":"2024-09-10T16:09:08.849450+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:09:08.849450+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17673,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4370},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'21082","reported_seq":21235,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:08.848683+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:09:08.848683+0000","last_peered":"2024-09-10T16:09:08.848683+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:09:08.848683+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17581,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4235},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'21314","reported_seq":21459,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.847784+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:09:08.847784+0000","last_peered":"2024-09-10T16:09:08.847784+0000","last_clean":"2024-09-10T16:09:08.847784+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:09:08.847784+0000","last_undegraded":"2024-09-10T16:09:08.847784+0000","last_fullsized":"2024-09-10T16:09:08.847784+0000","mapping_epoch":34,"log_start":"45'11300","ondisk_log_start":"45'11300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18161,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.851493+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:09:08.851493+0000","last_peered":"2024-09-10T16:09:08.851493+0000","last_clean":"2024-09-10T16:09:08.851493+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:09:08.851493+0000","last_undegraded":"2024-09-10T16:09:08.851493+0000","last_fullsized":"2024-09-10T16:09:08.851493+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20457","reported_seq":21847,"reported_epoch":51,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:11.153034+0000","last_change":"2024-09-10T16:09:07.396124+0000","last_active":"2024-09-10T16:09:11.153034+0000","last_peered":"2024-09-10T16:09:11.153034+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:09:11.153034+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3507,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18276,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":618,"num_bytes_recovered":4579,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3506}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":109,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.852443+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:09:08.852443+0000","last_peered":"2024-09-10T16:09:08.852443+0000","last_clean":"2024-09-10T16:09:08.852443+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:09:08.852443+0000","last_undegraded":"2024-09-10T16:09:08.852443+0000","last_fullsized":"2024-09-10T16:09:08.852443+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20910","reported_seq":29577,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.909669+0000","last_change":"2024-09-10T16:09:08.909580+0000","last_active":"2024-09-10T16:09:08.909669+0000","last_peered":"2024-09-10T16:09:08.909669+0000","last_clean":"2024-09-10T16:09:08.909669+0000","last_became_active":"2024-09-10T16:09:08.854941+0000","last_became_peered":"2024-09-10T16:09:08.854941+0000","last_unstale":"2024-09-10T16:09:08.909669+0000","last_undegraded":"2024-09-10T16:09:08.909669+0000","last_fullsized":"2024-09-10T16:09:08.909669+0000","mapping_epoch":50,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:13:14.846691+0000","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17414,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4257,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":119,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.847624+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:09:08.847624+0000","last_peered":"2024-09-10T16:09:08.847624+0000","last_clean":"2024-09-10T16:09:08.847624+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:09:08.847624+0000","last_undegraded":"2024-09-10T16:09:08.847624+0000","last_fullsized":"2024-09-10T16:09:08.847624+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20781","reported_seq":20931,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:08.852538+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:09:08.852538+0000","last_peered":"2024-09-10T16:09:08.852538+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:09:08.852538+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18461,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-4303}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":94,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.852182+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:09:08.852182+0000","last_peered":"2024-09-10T16:09:08.852182+0000","last_clean":"2024-09-10T16:09:08.852182+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:09:08.852182+0000","last_undegraded":"2024-09-10T16:09:08.852182+0000","last_fullsized":"2024-09-10T16:09:08.852182+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'21170","reported_seq":30029,"reported_epoch":51,"state":"active+recovery_wait","last_fresh":"2024-09-10T16:09:08.852233+0000","last_change":"2024-09-10T16:08:10.929169+0000","last_active":"2024-09-10T16:09:08.852233+0000","last_peered":"2024-09-10T16:09:08.852233+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T16:08:10.860438+0000","last_became_peered":"2024-09-10T16:08:10.860438+0000","last_unstale":"2024-09-10T16:09:08.852233+0000","last_undegraded":"2024-09-10T16:09:08.852233+0000","last_fullsized":"2024-09-10T16:09:08.852233+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18724,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","0","1"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20838","reported_seq":20988,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:08.849672+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:09:08.849672+0000","last_peered":"2024-09-10T16:09:08.849672+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:09:08.849672+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17469,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.851774+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:09:08.851774+0000","last_peered":"2024-09-10T16:09:08.851774+0000","last_clean":"2024-09-10T16:09:08.851774+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:09:08.851774+0000","last_undegraded":"2024-09-10T16:09:08.851774+0000","last_fullsized":"2024-09-10T16:09:08.851774+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20598","reported_seq":29178,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.851713+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:09:08.851713+0000","last_peered":"2024-09-10T16:09:08.851713+0000","last_clean":"2024-09-10T16:09:08.851713+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:09:08.851713+0000","last_undegraded":"2024-09-10T16:09:08.851713+0000","last_fullsized":"2024-09-10T16:09:08.851713+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16988,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":109,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.853357+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:09:08.853357+0000","last_peered":"2024-09-10T16:09:08.853357+0000","last_clean":"2024-09-10T16:09:08.853357+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:09:08.853357+0000","last_undegraded":"2024-09-10T16:09:08.853357+0000","last_fullsized":"2024-09-10T16:09:08.853357+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'21048","reported_seq":21205,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:08.848232+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:09:08.848232+0000","last_peered":"2024-09-10T16:09:08.848232+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:09:08.848232+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17303,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.851634+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:09:08.851634+0000","last_peered":"2024-09-10T16:09:08.851634+0000","last_clean":"2024-09-10T16:09:08.851634+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:09:08.851634+0000","last_undegraded":"2024-09-10T16:09:08.851634+0000","last_fullsized":"2024-09-10T16:09:08.851634+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":89,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.851698+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:09:08.851698+0000","last_peered":"2024-09-10T16:09:08.851698+0000","last_clean":"2024-09-10T16:09:08.851698+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:09:08.851698+0000","last_undegraded":"2024-09-10T16:09:08.851698+0000","last_fullsized":"2024-09-10T16:09:08.851698+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'21198","reported_seq":21357,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:08.848736+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:09:08.848736+0000","last_peered":"2024-09-10T16:09:08.848736+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:09:08.848736+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17361,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4321},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33683,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292096,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17707,"num_bytes_recovered":33096163,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":210518016,"data_stored":210395174,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161022,"ondisk_log_size":161022,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986310,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":502616,"kb_used_data":137728,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93869224,"statfs":{"total":96636764160,"available":96122085376,"internally_reserved":0,"allocated":141033472,"data_stored":140713808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659255,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":461168,"kb_used_data":476,"kb_used_omap":4,"kb_used_meta":460667,"kb_avail":93910672,"statfs":{"total":96636764160,"available":96164528128,"internally_reserved":0,"allocated":487424,"data_stored":149231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":471723694},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182538,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":547316,"kb_used_data":68636,"kb_used_omap":4,"kb_used_meta":478651,"kb_avail":93824524,"statfs":{"total":96636764160,"available":96076312576,"internally_reserved":0,"allocated":70283264,"data_stored":69959523,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":490139306},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":69853184,"data_stored":69817054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":6762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":140607488,"data_stored":140571358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:09:12.015 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-10T16:09:12.015 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "49'20672", 'reported_seq': 20761, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:09:08.850557+0000', 'last_change': '2024-09-10T15:59:16.281558+0000', 'last_active': '2024-09-10T16:09:08.850557+0000', 'last_peered': '2024-09-10T16:09:08.850557+0000', 'last_clean': '2024-09-10T15:59:14.132456+0000', 'last_became_active': '2024-09-10T15:59:16.274407+0000', 'last_became_peered': '2024-09-10T15:59:16.274407+0000', 'last_unstale': '2024-09-10T16:09:08.850557+0000', 'last_undegraded': '2024-09-10T15:59:15.851279+0000', 'last_fullsized': '2024-09-10T15:59:15.774356+0000', 'mapping_epoch': 42, 'log_start': "45'10600", 'ondisk_log_start': "45'10600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'objects_scrubbed': 0, 'log_size': 10072, 'log_dups_size': 0, 'ondisk_log_size': 10072, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4259, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20672, 'num_write_kb': 41344, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4259}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:09:12.016 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-10T16:09:12.016 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "49'21061", 'reported_seq': 21214, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:09:08.849450+0000', 'last_change': '2024-09-10T15:59:16.308264+0000', 'last_active': '2024-09-10T16:09:08.849450+0000', 'last_peered': '2024-09-10T16:09:08.849450+0000', 'last_clean': '2024-09-10T15:59:14.111551+0000', 'last_became_active': '2024-09-10T15:59:16.300040+0000', 'last_became_peered': '2024-09-10T15:59:16.300040+0000', 'last_unstale': '2024-09-10T16:09:08.849450+0000', 'last_undegraded': '2024-09-10T15:59:15.878234+0000', 'last_fullsized': '2024-09-10T15:59:15.769328+0000', 'mapping_epoch': 42, 'log_start': "45'11000", 'ondisk_log_start': "45'11000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3442", 'last_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'objects_scrubbed': 3384, 'log_size': 10061, 'log_dups_size': 0, 'ondisk_log_size': 10061, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1292, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4372, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 17673, 'num_write_kb': 28580, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4370}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:09:12.016 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-10T16:09:12.016 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "49'21082", 'reported_seq': 21235, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:09:08.848683+0000', 'last_change': '2024-09-10T15:59:16.331899+0000', 'last_active': '2024-09-10T16:09:08.848683+0000', 'last_peered': '2024-09-10T16:09:08.848683+0000', 'last_clean': '2024-09-10T15:59:14.139436+0000', 'last_became_active': '2024-09-10T15:59:16.318715+0000', 'last_became_peered': '2024-09-10T15:59:16.318715+0000', 'last_unstale': '2024-09-10T16:09:08.848683+0000', 'last_undegraded': '2024-09-10T15:59:15.877524+0000', 'last_fullsized': '2024-09-10T15:59:15.769042+0000', 'mapping_epoch': 42, 'log_start': "45'11000", 'ondisk_log_start': "45'11000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3537", 'last_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'objects_scrubbed': 3497, 'log_size': 10082, 'log_dups_size': 0, 'ondisk_log_size': 10082, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1224, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4237, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 17581, 'num_write_kb': 28172, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4235}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:09:12.016 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-10T16:09:12.016 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "49'20457", 'reported_seq': 21847, 'reported_epoch': 51, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:09:11.153034+0000', 'last_change': '2024-09-10T16:09:07.396124+0000', 'last_active': '2024-09-10T16:09:11.153034+0000', 'last_peered': '2024-09-10T16:09:11.153034+0000', 'last_clean': '2024-09-10T15:59:14.088187+0000', 'last_became_active': '2024-09-10T15:59:16.165810+0000', 'last_became_peered': '2024-09-10T15:59:16.165810+0000', 'last_unstale': '2024-09-10T16:09:11.153034+0000', 'last_undegraded': '2024-09-10T15:59:15.822098+0000', 'last_fullsized': '2024-09-10T15:59:15.769123+0000', 'mapping_epoch': 42, 'log_start': "45'10400", 'ondisk_log_start': "45'10400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2218", 'last_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'objects_scrubbed': 2175, 'log_size': 10057, 'log_dups_size': 0, 'ondisk_log_size': 10057, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 825, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3507, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 18276, 'num_write_kb': 32204, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 618, 'num_bytes_recovered': 4579, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3506}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:09:12.016 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-10T16:09:12.016 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "49'20781", 'reported_seq': 20931, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:09:08.852538+0000', 'last_change': '2024-09-10T15:59:16.284311+0000', 'last_active': '2024-09-10T16:09:08.852538+0000', 'last_peered': '2024-09-10T16:09:08.852538+0000', 'last_clean': '2024-09-10T15:59:14.116041+0000', 'last_became_active': '2024-09-10T15:59:16.274543+0000', 'last_became_peered': '2024-09-10T15:59:16.274543+0000', 'last_unstale': '2024-09-10T16:09:08.852538+0000', 'last_undegraded': '2024-09-10T15:59:15.812484+0000', 'last_fullsized': '2024-09-10T15:59:15.770885+0000', 'mapping_epoch': 42, 'log_start': "45'10700", 'ondisk_log_start': "45'10700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2356", 'last_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'objects_scrubbed': 2316, 'log_size': 10081, 'log_dups_size': 0, 'ondisk_log_size': 10081, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 783, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 18461, 'num_write_kb': 32292, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': -4303}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:09:12.016 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-10T16:09:12.017 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "49'21170", 'reported_seq': 30029, 'reported_epoch': 51, 'state': 'active+recovery_wait', 'last_fresh': '2024-09-10T16:09:08.852233+0000', 'last_change': '2024-09-10T16:08:10.929169+0000', 'last_active': '2024-09-10T16:09:08.852233+0000', 'last_peered': '2024-09-10T16:09:08.852233+0000', 'last_clean': '2024-09-10T15:59:14.110654+0000', 'last_became_active': '2024-09-10T16:08:10.860438+0000', 'last_became_peered': '2024-09-10T16:08:10.860438+0000', 'last_unstale': '2024-09-10T16:09:08.852233+0000', 'last_undegraded': '2024-09-10T16:09:08.852233+0000', 'last_fullsized': '2024-09-10T16:09:08.852233+0000', 'mapping_epoch': 48, 'log_start': "45'11100", 'ondisk_log_start': "45'11100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2486", 'last_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'objects_scrubbed': 2440, 'log_size': 10070, 'log_dups_size': 0, 'ondisk_log_size': 10070, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 792, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18724, 'num_write_kb': 32568, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 4355, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': ['2', '0', '1'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:09:12.017 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-10T16:09:12.017 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "49'20838", 'reported_seq': 20988, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:09:08.849672+0000', 'last_change': '2024-09-10T15:59:16.315641+0000', 'last_active': '2024-09-10T16:09:08.849672+0000', 'last_peered': '2024-09-10T16:09:08.849672+0000', 'last_clean': '2024-09-10T15:59:14.117001+0000', 'last_became_active': '2024-09-10T15:59:16.302245+0000', 'last_became_peered': '2024-09-10T15:59:16.302245+0000', 'last_unstale': '2024-09-10T16:09:08.849672+0000', 'last_undegraded': '2024-09-10T15:59:15.845012+0000', 'last_fullsized': '2024-09-10T15:59:15.770337+0000', 'mapping_epoch': 42, 'log_start': "45'10800", 'ondisk_log_start': "45'10800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3404", 'last_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'objects_scrubbed': 3367, 'log_size': 10038, 'log_dups_size': 0, 'ondisk_log_size': 10038, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 983, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17469, 'num_write_kb': 28204, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4304}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:09:12.017 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-10T16:09:12.017 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "49'21048", 'reported_seq': 21205, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:09:08.848232+0000', 'last_change': '2024-09-10T15:59:16.311291+0000', 'last_active': '2024-09-10T16:09:08.848232+0000', 'last_peered': '2024-09-10T16:09:08.848232+0000', 'last_clean': '2024-09-10T15:59:14.116363+0000', 'last_became_active': '2024-09-10T15:59:16.301244+0000', 'last_became_peered': '2024-09-10T15:59:16.301244+0000', 'last_unstale': '2024-09-10T16:09:08.848232+0000', 'last_undegraded': '2024-09-10T15:59:15.883434+0000', 'last_fullsized': '2024-09-10T15:59:15.772756+0000', 'mapping_epoch': 42, 'log_start': "45'11000", 'ondisk_log_start': "45'11000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3789", 'last_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'objects_scrubbed': 3741, 'log_size': 10048, 'log_dups_size': 0, 'ondisk_log_size': 10048, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1440, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4373, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17303, 'num_write_kb': 27124, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4371}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:09:12.017 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-10T16:09:12.017 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "49'21198", 'reported_seq': 21357, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:09:08.848736+0000', 'last_change': '2024-09-10T15:59:16.313409+0000', 'last_active': '2024-09-10T16:09:08.848736+0000', 'last_peered': '2024-09-10T16:09:08.848736+0000', 'last_clean': '2024-09-10T15:59:14.129780+0000', 'last_became_active': '2024-09-10T15:59:16.300932+0000', 'last_became_peered': '2024-09-10T15:59:16.300932+0000', 'last_unstale': '2024-09-10T16:09:08.848736+0000', 'last_undegraded': '2024-09-10T15:59:15.887823+0000', 'last_fullsized': '2024-09-10T15:59:15.768513+0000', 'mapping_epoch': 42, 'log_start': "45'11100", 'ondisk_log_start': "45'11100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3880", 'last_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'objects_scrubbed': 3827, 'log_size': 10098, 'log_dups_size': 0, 'ondisk_log_size': 10098, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1397, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4325, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17361, 'num_write_kb': 27072, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4321}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:09:12.017 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T16:09:12.450 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:09:12.450 INFO:teuthology.orchestra.run.smithi115.stdout:{"fsid":"7effee29-df05-4351-9741-ee203564ed81","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 33683/24 objects degraded (140345.833%), 8 pgs degraded, 8 pgs undersized","count":16},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":716,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":51,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983954,"num_in_osds":3,"osd_in_since":1725983947,"num_remapped_pgs":8},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":15},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":7},{"state_name":"active+recovery_wait","count":1},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":12,"data_bytes":4849,"bytes_used":1547366400,"bytes_avail":288362926080,"bytes_total":289910292480,"degraded_objects":33683,"degraded_total":24,"degraded_ratio":1403.458333333333,"recovering_objects_per_sec":244,"recovering_bytes_per_sec":60,"recovering_keys_per_sec":0,"num_objects_recovered":1953,"num_bytes_recovered":483,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-10T15:57:14:967120+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T16:00:13.560776+0000","services":{}},"progress_events":{}} 2024-09-10T16:09:12.464 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-10T16:09:12.464 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:09:12.812 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:09:12.813 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:09:12.825 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":402,"stamp":"2024-09-10T16:09:11.770440+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33683,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292098,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17709,"num_bytes_recovered":33096182,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161025,"ondisk_log_size":161025,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1511100,"kb_used_data":206840,"kb_used_omap":12,"kb_used_meta":1304179,"kb_avail":281604420,"statfs":{"total":289910292480,"available":288362926080,"internally_reserved":0,"allocated":211804160,"data_stored":210822562,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1335479298},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1953,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001518"},"pg_stats":[{"pgid":"2.f","version":"49'20672","reported_seq":20761,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:08.850557+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:09:08.850557+0000","last_peered":"2024-09-10T16:09:08.850557+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:09:08.850557+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20672,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4259}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20844","reported_seq":20971,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.853495+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:09:08.853495+0000","last_peered":"2024-09-10T16:09:08.853495+0000","last_clean":"2024-09-10T16:09:08.853495+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:09:08.853495+0000","last_undegraded":"2024-09-10T16:09:08.853495+0000","last_fullsized":"2024-09-10T16:09:08.853495+0000","mapping_epoch":35,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18949,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20798","reported_seq":29491,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.852143+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:09:08.852143+0000","last_peered":"2024-09-10T16:09:08.852143+0000","last_clean":"2024-09-10T16:09:08.852143+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:09:08.852143+0000","last_undegraded":"2024-09-10T16:09:08.852143+0000","last_fullsized":"2024-09-10T16:09:08.852143+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18802,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20660","reported_seq":20755,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.853148+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:09:08.853148+0000","last_peered":"2024-09-10T16:09:08.853148+0000","last_clean":"2024-09-10T16:09:08.853148+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:09:08.853148+0000","last_undegraded":"2024-09-10T16:09:08.853148+0000","last_fullsized":"2024-09-10T16:09:08.853148+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20660,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20891","reported_seq":21036,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.847836+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:09:08.847836+0000","last_peered":"2024-09-10T16:09:08.847836+0000","last_clean":"2024-09-10T16:09:08.847836+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:09:08.847836+0000","last_undegraded":"2024-09-10T16:09:08.847836+0000","last_fullsized":"2024-09-10T16:09:08.847836+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17602,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'21061","reported_seq":21214,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:08.849450+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:09:08.849450+0000","last_peered":"2024-09-10T16:09:08.849450+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:09:08.849450+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17673,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4370},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'21082","reported_seq":21235,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:08.848683+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:09:08.848683+0000","last_peered":"2024-09-10T16:09:08.848683+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:09:08.848683+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17581,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4235},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'21314","reported_seq":21459,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.847784+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:09:08.847784+0000","last_peered":"2024-09-10T16:09:08.847784+0000","last_clean":"2024-09-10T16:09:08.847784+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:09:08.847784+0000","last_undegraded":"2024-09-10T16:09:08.847784+0000","last_fullsized":"2024-09-10T16:09:08.847784+0000","mapping_epoch":34,"log_start":"45'11300","ondisk_log_start":"45'11300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18161,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.851493+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:09:08.851493+0000","last_peered":"2024-09-10T16:09:08.851493+0000","last_clean":"2024-09-10T16:09:08.851493+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:09:08.851493+0000","last_undegraded":"2024-09-10T16:09:08.851493+0000","last_fullsized":"2024-09-10T16:09:08.851493+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20457","reported_seq":21847,"reported_epoch":51,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:11.153034+0000","last_change":"2024-09-10T16:09:07.396124+0000","last_active":"2024-09-10T16:09:11.153034+0000","last_peered":"2024-09-10T16:09:11.153034+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:09:11.153034+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3507,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18276,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":618,"num_bytes_recovered":4579,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3506}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":109,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.852443+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:09:08.852443+0000","last_peered":"2024-09-10T16:09:08.852443+0000","last_clean":"2024-09-10T16:09:08.852443+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:09:08.852443+0000","last_undegraded":"2024-09-10T16:09:08.852443+0000","last_fullsized":"2024-09-10T16:09:08.852443+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20910","reported_seq":29577,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.909669+0000","last_change":"2024-09-10T16:09:08.909580+0000","last_active":"2024-09-10T16:09:08.909669+0000","last_peered":"2024-09-10T16:09:08.909669+0000","last_clean":"2024-09-10T16:09:08.909669+0000","last_became_active":"2024-09-10T16:09:08.854941+0000","last_became_peered":"2024-09-10T16:09:08.854941+0000","last_unstale":"2024-09-10T16:09:08.909669+0000","last_undegraded":"2024-09-10T16:09:08.909669+0000","last_fullsized":"2024-09-10T16:09:08.909669+0000","mapping_epoch":50,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:13:14.846691+0000","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17414,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4257,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":119,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.847624+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:09:08.847624+0000","last_peered":"2024-09-10T16:09:08.847624+0000","last_clean":"2024-09-10T16:09:08.847624+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:09:08.847624+0000","last_undegraded":"2024-09-10T16:09:08.847624+0000","last_fullsized":"2024-09-10T16:09:08.847624+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20781","reported_seq":20931,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:08.852538+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:09:08.852538+0000","last_peered":"2024-09-10T16:09:08.852538+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:09:08.852538+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18461,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-4303}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":94,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.852182+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:09:08.852182+0000","last_peered":"2024-09-10T16:09:08.852182+0000","last_clean":"2024-09-10T16:09:08.852182+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:09:08.852182+0000","last_undegraded":"2024-09-10T16:09:08.852182+0000","last_fullsized":"2024-09-10T16:09:08.852182+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'21170","reported_seq":30029,"reported_epoch":51,"state":"active+recovery_wait","last_fresh":"2024-09-10T16:09:08.852233+0000","last_change":"2024-09-10T16:08:10.929169+0000","last_active":"2024-09-10T16:09:08.852233+0000","last_peered":"2024-09-10T16:09:08.852233+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T16:08:10.860438+0000","last_became_peered":"2024-09-10T16:08:10.860438+0000","last_unstale":"2024-09-10T16:09:08.852233+0000","last_undegraded":"2024-09-10T16:09:08.852233+0000","last_fullsized":"2024-09-10T16:09:08.852233+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18724,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","0","1"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20838","reported_seq":20988,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:08.849672+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:09:08.849672+0000","last_peered":"2024-09-10T16:09:08.849672+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:09:08.849672+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17469,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.851774+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:09:08.851774+0000","last_peered":"2024-09-10T16:09:08.851774+0000","last_clean":"2024-09-10T16:09:08.851774+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:09:08.851774+0000","last_undegraded":"2024-09-10T16:09:08.851774+0000","last_fullsized":"2024-09-10T16:09:08.851774+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20598","reported_seq":29178,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.851713+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:09:08.851713+0000","last_peered":"2024-09-10T16:09:08.851713+0000","last_clean":"2024-09-10T16:09:08.851713+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:09:08.851713+0000","last_undegraded":"2024-09-10T16:09:08.851713+0000","last_fullsized":"2024-09-10T16:09:08.851713+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16988,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":109,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.853357+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:09:08.853357+0000","last_peered":"2024-09-10T16:09:08.853357+0000","last_clean":"2024-09-10T16:09:08.853357+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:09:08.853357+0000","last_undegraded":"2024-09-10T16:09:08.853357+0000","last_fullsized":"2024-09-10T16:09:08.853357+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'21048","reported_seq":21205,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:08.848232+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:09:08.848232+0000","last_peered":"2024-09-10T16:09:08.848232+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:09:08.848232+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17303,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.851634+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:09:08.851634+0000","last_peered":"2024-09-10T16:09:08.851634+0000","last_clean":"2024-09-10T16:09:08.851634+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:09:08.851634+0000","last_undegraded":"2024-09-10T16:09:08.851634+0000","last_fullsized":"2024-09-10T16:09:08.851634+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":89,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.851698+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:09:08.851698+0000","last_peered":"2024-09-10T16:09:08.851698+0000","last_clean":"2024-09-10T16:09:08.851698+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:09:08.851698+0000","last_undegraded":"2024-09-10T16:09:08.851698+0000","last_fullsized":"2024-09-10T16:09:08.851698+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'21198","reported_seq":21357,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:08.848736+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:09:08.848736+0000","last_peered":"2024-09-10T16:09:08.848736+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:09:08.848736+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17361,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4321},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33683,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292096,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17707,"num_bytes_recovered":33096163,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":210518016,"data_stored":210395174,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161022,"ondisk_log_size":161022,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986310,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":502616,"kb_used_data":137728,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93869224,"statfs":{"total":96636764160,"available":96122085376,"internally_reserved":0,"allocated":141033472,"data_stored":140713808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659255,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":461168,"kb_used_data":476,"kb_used_omap":4,"kb_used_meta":460667,"kb_avail":93910672,"statfs":{"total":96636764160,"available":96164528128,"internally_reserved":0,"allocated":487424,"data_stored":149231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":471723694},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182538,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":547316,"kb_used_data":68636,"kb_used_omap":4,"kb_used_meta":478651,"kb_avail":93824524,"statfs":{"total":96636764160,"available":96076312576,"internally_reserved":0,"allocated":70283264,"data_stored":69959523,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":490139306},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":69853184,"data_stored":69817054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":6762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":140607488,"data_stored":140571358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:09:15.828 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:09:16.136 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:09:16.137 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:09:16.149 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":404,"stamp":"2024-09-10T16:09:15.771193+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33683,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292098,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17709,"num_bytes_recovered":33096182,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161025,"ondisk_log_size":161025,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1519416,"kb_used_data":206840,"kb_used_omap":12,"kb_used_meta":1312499,"kb_avail":281596104,"statfs":{"total":289910292480,"available":288354410496,"internally_reserved":0,"allocated":211804160,"data_stored":210822562,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1343998978},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-909,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":909,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001468"},"pg_stats":[{"pgid":"2.f","version":"49'20672","reported_seq":20761,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:08.850557+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:09:08.850557+0000","last_peered":"2024-09-10T16:09:08.850557+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:09:08.850557+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20672,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4259}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20844","reported_seq":20971,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.853495+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:09:08.853495+0000","last_peered":"2024-09-10T16:09:08.853495+0000","last_clean":"2024-09-10T16:09:08.853495+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:09:08.853495+0000","last_undegraded":"2024-09-10T16:09:08.853495+0000","last_fullsized":"2024-09-10T16:09:08.853495+0000","mapping_epoch":35,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18949,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20798","reported_seq":29491,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.852143+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:09:08.852143+0000","last_peered":"2024-09-10T16:09:08.852143+0000","last_clean":"2024-09-10T16:09:08.852143+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:09:08.852143+0000","last_undegraded":"2024-09-10T16:09:08.852143+0000","last_fullsized":"2024-09-10T16:09:08.852143+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18802,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20660","reported_seq":20755,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.853148+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:09:08.853148+0000","last_peered":"2024-09-10T16:09:08.853148+0000","last_clean":"2024-09-10T16:09:08.853148+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:09:08.853148+0000","last_undegraded":"2024-09-10T16:09:08.853148+0000","last_fullsized":"2024-09-10T16:09:08.853148+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20660,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20891","reported_seq":21036,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.847836+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:09:08.847836+0000","last_peered":"2024-09-10T16:09:08.847836+0000","last_clean":"2024-09-10T16:09:08.847836+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:09:08.847836+0000","last_undegraded":"2024-09-10T16:09:08.847836+0000","last_fullsized":"2024-09-10T16:09:08.847836+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17602,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'21061","reported_seq":21214,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:08.849450+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:09:08.849450+0000","last_peered":"2024-09-10T16:09:08.849450+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:09:08.849450+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17673,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4370},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'21082","reported_seq":21235,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:08.848683+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:09:08.848683+0000","last_peered":"2024-09-10T16:09:08.848683+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:09:08.848683+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17581,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4235},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'21314","reported_seq":21459,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.847784+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:09:08.847784+0000","last_peered":"2024-09-10T16:09:08.847784+0000","last_clean":"2024-09-10T16:09:08.847784+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:09:08.847784+0000","last_undegraded":"2024-09-10T16:09:08.847784+0000","last_fullsized":"2024-09-10T16:09:08.847784+0000","mapping_epoch":34,"log_start":"45'11300","ondisk_log_start":"45'11300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18161,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.851493+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:09:08.851493+0000","last_peered":"2024-09-10T16:09:08.851493+0000","last_clean":"2024-09-10T16:09:08.851493+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:09:08.851493+0000","last_undegraded":"2024-09-10T16:09:08.851493+0000","last_fullsized":"2024-09-10T16:09:08.851493+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20457","reported_seq":21847,"reported_epoch":51,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:11.153034+0000","last_change":"2024-09-10T16:09:07.396124+0000","last_active":"2024-09-10T16:09:11.153034+0000","last_peered":"2024-09-10T16:09:11.153034+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:09:11.153034+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3507,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18276,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":618,"num_bytes_recovered":4579,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3506}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":109,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.852443+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:09:08.852443+0000","last_peered":"2024-09-10T16:09:08.852443+0000","last_clean":"2024-09-10T16:09:08.852443+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:09:08.852443+0000","last_undegraded":"2024-09-10T16:09:08.852443+0000","last_fullsized":"2024-09-10T16:09:08.852443+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20910","reported_seq":29577,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.909669+0000","last_change":"2024-09-10T16:09:08.909580+0000","last_active":"2024-09-10T16:09:08.909669+0000","last_peered":"2024-09-10T16:09:08.909669+0000","last_clean":"2024-09-10T16:09:08.909669+0000","last_became_active":"2024-09-10T16:09:08.854941+0000","last_became_peered":"2024-09-10T16:09:08.854941+0000","last_unstale":"2024-09-10T16:09:08.909669+0000","last_undegraded":"2024-09-10T16:09:08.909669+0000","last_fullsized":"2024-09-10T16:09:08.909669+0000","mapping_epoch":50,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:13:14.846691+0000","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17414,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4257,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":119,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.847624+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:09:08.847624+0000","last_peered":"2024-09-10T16:09:08.847624+0000","last_clean":"2024-09-10T16:09:08.847624+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:09:08.847624+0000","last_undegraded":"2024-09-10T16:09:08.847624+0000","last_fullsized":"2024-09-10T16:09:08.847624+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20781","reported_seq":20931,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:08.852538+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:09:08.852538+0000","last_peered":"2024-09-10T16:09:08.852538+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:09:08.852538+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18461,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-4303}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":94,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.852182+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:09:08.852182+0000","last_peered":"2024-09-10T16:09:08.852182+0000","last_clean":"2024-09-10T16:09:08.852182+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:09:08.852182+0000","last_undegraded":"2024-09-10T16:09:08.852182+0000","last_fullsized":"2024-09-10T16:09:08.852182+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'21170","reported_seq":30029,"reported_epoch":51,"state":"active+recovery_wait","last_fresh":"2024-09-10T16:09:08.852233+0000","last_change":"2024-09-10T16:08:10.929169+0000","last_active":"2024-09-10T16:09:08.852233+0000","last_peered":"2024-09-10T16:09:08.852233+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T16:08:10.860438+0000","last_became_peered":"2024-09-10T16:08:10.860438+0000","last_unstale":"2024-09-10T16:09:08.852233+0000","last_undegraded":"2024-09-10T16:09:08.852233+0000","last_fullsized":"2024-09-10T16:09:08.852233+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18724,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","0","1"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20838","reported_seq":20988,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:08.849672+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:09:08.849672+0000","last_peered":"2024-09-10T16:09:08.849672+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:09:08.849672+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17469,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.851774+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:09:08.851774+0000","last_peered":"2024-09-10T16:09:08.851774+0000","last_clean":"2024-09-10T16:09:08.851774+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:09:08.851774+0000","last_undegraded":"2024-09-10T16:09:08.851774+0000","last_fullsized":"2024-09-10T16:09:08.851774+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20598","reported_seq":29178,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.851713+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:09:08.851713+0000","last_peered":"2024-09-10T16:09:08.851713+0000","last_clean":"2024-09-10T16:09:08.851713+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:09:08.851713+0000","last_undegraded":"2024-09-10T16:09:08.851713+0000","last_fullsized":"2024-09-10T16:09:08.851713+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16988,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":109,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.853357+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:09:08.853357+0000","last_peered":"2024-09-10T16:09:08.853357+0000","last_clean":"2024-09-10T16:09:08.853357+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:09:08.853357+0000","last_undegraded":"2024-09-10T16:09:08.853357+0000","last_fullsized":"2024-09-10T16:09:08.853357+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'21048","reported_seq":21205,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:08.848232+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:09:08.848232+0000","last_peered":"2024-09-10T16:09:08.848232+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:09:08.848232+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17303,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.851634+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:09:08.851634+0000","last_peered":"2024-09-10T16:09:08.851634+0000","last_clean":"2024-09-10T16:09:08.851634+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:09:08.851634+0000","last_undegraded":"2024-09-10T16:09:08.851634+0000","last_fullsized":"2024-09-10T16:09:08.851634+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":89,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.851698+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:09:08.851698+0000","last_peered":"2024-09-10T16:09:08.851698+0000","last_clean":"2024-09-10T16:09:08.851698+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:09:08.851698+0000","last_undegraded":"2024-09-10T16:09:08.851698+0000","last_fullsized":"2024-09-10T16:09:08.851698+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'21198","reported_seq":21357,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:08.848736+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:09:08.848736+0000","last_peered":"2024-09-10T16:09:08.848736+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:09:08.848736+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17361,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4321},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33683,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292096,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17707,"num_bytes_recovered":33096163,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":210518016,"data_stored":210395174,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161022,"ondisk_log_size":161022,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986310,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":502616,"kb_used_data":137728,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93869224,"statfs":{"total":96636764160,"available":96122085376,"internally_reserved":0,"allocated":141033472,"data_stored":140713808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659256,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":469484,"kb_used_data":476,"kb_used_omap":4,"kb_used_meta":468987,"kb_avail":93902356,"statfs":{"total":96636764160,"available":96156012544,"internally_reserved":0,"allocated":487424,"data_stored":149231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":480243374},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182538,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":547316,"kb_used_data":68636,"kb_used_omap":4,"kb_used_meta":478651,"kb_avail":93824524,"statfs":{"total":96636764160,"available":96076312576,"internally_reserved":0,"allocated":70283264,"data_stored":69959523,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":490139306},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":69853184,"data_stored":69817054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":6762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":140607488,"data_stored":140571358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:09:16.150 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:09:16.474 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:09:16.475 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:09:16.488 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":404,"stamp":"2024-09-10T16:09:15.771193+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33683,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292098,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17709,"num_bytes_recovered":33096182,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161025,"ondisk_log_size":161025,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1519416,"kb_used_data":206840,"kb_used_omap":12,"kb_used_meta":1312499,"kb_avail":281596104,"statfs":{"total":289910292480,"available":288354410496,"internally_reserved":0,"allocated":211804160,"data_stored":210822562,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1343998978},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-909,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":909,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001468"},"pg_stats":[{"pgid":"2.f","version":"49'20672","reported_seq":20761,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:08.850557+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:09:08.850557+0000","last_peered":"2024-09-10T16:09:08.850557+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:09:08.850557+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20672,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4259}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20844","reported_seq":20971,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.853495+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:09:08.853495+0000","last_peered":"2024-09-10T16:09:08.853495+0000","last_clean":"2024-09-10T16:09:08.853495+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:09:08.853495+0000","last_undegraded":"2024-09-10T16:09:08.853495+0000","last_fullsized":"2024-09-10T16:09:08.853495+0000","mapping_epoch":35,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18949,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20798","reported_seq":29491,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.852143+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:09:08.852143+0000","last_peered":"2024-09-10T16:09:08.852143+0000","last_clean":"2024-09-10T16:09:08.852143+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:09:08.852143+0000","last_undegraded":"2024-09-10T16:09:08.852143+0000","last_fullsized":"2024-09-10T16:09:08.852143+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18802,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20660","reported_seq":20755,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.853148+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:09:08.853148+0000","last_peered":"2024-09-10T16:09:08.853148+0000","last_clean":"2024-09-10T16:09:08.853148+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:09:08.853148+0000","last_undegraded":"2024-09-10T16:09:08.853148+0000","last_fullsized":"2024-09-10T16:09:08.853148+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20660,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20891","reported_seq":21036,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.847836+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:09:08.847836+0000","last_peered":"2024-09-10T16:09:08.847836+0000","last_clean":"2024-09-10T16:09:08.847836+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:09:08.847836+0000","last_undegraded":"2024-09-10T16:09:08.847836+0000","last_fullsized":"2024-09-10T16:09:08.847836+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17602,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'21061","reported_seq":21214,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:08.849450+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:09:08.849450+0000","last_peered":"2024-09-10T16:09:08.849450+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:09:08.849450+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17673,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4370},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'21082","reported_seq":21235,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:08.848683+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:09:08.848683+0000","last_peered":"2024-09-10T16:09:08.848683+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:09:08.848683+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17581,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4235},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'21314","reported_seq":21459,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.847784+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:09:08.847784+0000","last_peered":"2024-09-10T16:09:08.847784+0000","last_clean":"2024-09-10T16:09:08.847784+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:09:08.847784+0000","last_undegraded":"2024-09-10T16:09:08.847784+0000","last_fullsized":"2024-09-10T16:09:08.847784+0000","mapping_epoch":34,"log_start":"45'11300","ondisk_log_start":"45'11300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18161,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.851493+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:09:08.851493+0000","last_peered":"2024-09-10T16:09:08.851493+0000","last_clean":"2024-09-10T16:09:08.851493+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:09:08.851493+0000","last_undegraded":"2024-09-10T16:09:08.851493+0000","last_fullsized":"2024-09-10T16:09:08.851493+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20457","reported_seq":21847,"reported_epoch":51,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:11.153034+0000","last_change":"2024-09-10T16:09:07.396124+0000","last_active":"2024-09-10T16:09:11.153034+0000","last_peered":"2024-09-10T16:09:11.153034+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:09:11.153034+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3507,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18276,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":618,"num_bytes_recovered":4579,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3506}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":109,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.852443+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:09:08.852443+0000","last_peered":"2024-09-10T16:09:08.852443+0000","last_clean":"2024-09-10T16:09:08.852443+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:09:08.852443+0000","last_undegraded":"2024-09-10T16:09:08.852443+0000","last_fullsized":"2024-09-10T16:09:08.852443+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20910","reported_seq":29577,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.909669+0000","last_change":"2024-09-10T16:09:08.909580+0000","last_active":"2024-09-10T16:09:08.909669+0000","last_peered":"2024-09-10T16:09:08.909669+0000","last_clean":"2024-09-10T16:09:08.909669+0000","last_became_active":"2024-09-10T16:09:08.854941+0000","last_became_peered":"2024-09-10T16:09:08.854941+0000","last_unstale":"2024-09-10T16:09:08.909669+0000","last_undegraded":"2024-09-10T16:09:08.909669+0000","last_fullsized":"2024-09-10T16:09:08.909669+0000","mapping_epoch":50,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:13:14.846691+0000","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17414,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4257,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":119,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.847624+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:09:08.847624+0000","last_peered":"2024-09-10T16:09:08.847624+0000","last_clean":"2024-09-10T16:09:08.847624+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:09:08.847624+0000","last_undegraded":"2024-09-10T16:09:08.847624+0000","last_fullsized":"2024-09-10T16:09:08.847624+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20781","reported_seq":20931,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:08.852538+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:09:08.852538+0000","last_peered":"2024-09-10T16:09:08.852538+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:09:08.852538+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18461,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-4303}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":94,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.852182+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:09:08.852182+0000","last_peered":"2024-09-10T16:09:08.852182+0000","last_clean":"2024-09-10T16:09:08.852182+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:09:08.852182+0000","last_undegraded":"2024-09-10T16:09:08.852182+0000","last_fullsized":"2024-09-10T16:09:08.852182+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'21170","reported_seq":30029,"reported_epoch":51,"state":"active+recovery_wait","last_fresh":"2024-09-10T16:09:08.852233+0000","last_change":"2024-09-10T16:08:10.929169+0000","last_active":"2024-09-10T16:09:08.852233+0000","last_peered":"2024-09-10T16:09:08.852233+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T16:08:10.860438+0000","last_became_peered":"2024-09-10T16:08:10.860438+0000","last_unstale":"2024-09-10T16:09:08.852233+0000","last_undegraded":"2024-09-10T16:09:08.852233+0000","last_fullsized":"2024-09-10T16:09:08.852233+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18724,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","0","1"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20838","reported_seq":20988,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:08.849672+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:09:08.849672+0000","last_peered":"2024-09-10T16:09:08.849672+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:09:08.849672+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17469,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.851774+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:09:08.851774+0000","last_peered":"2024-09-10T16:09:08.851774+0000","last_clean":"2024-09-10T16:09:08.851774+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:09:08.851774+0000","last_undegraded":"2024-09-10T16:09:08.851774+0000","last_fullsized":"2024-09-10T16:09:08.851774+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20598","reported_seq":29178,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.851713+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:09:08.851713+0000","last_peered":"2024-09-10T16:09:08.851713+0000","last_clean":"2024-09-10T16:09:08.851713+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:09:08.851713+0000","last_undegraded":"2024-09-10T16:09:08.851713+0000","last_fullsized":"2024-09-10T16:09:08.851713+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16988,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":109,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.853357+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:09:08.853357+0000","last_peered":"2024-09-10T16:09:08.853357+0000","last_clean":"2024-09-10T16:09:08.853357+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:09:08.853357+0000","last_undegraded":"2024-09-10T16:09:08.853357+0000","last_fullsized":"2024-09-10T16:09:08.853357+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'21048","reported_seq":21205,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:08.848232+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:09:08.848232+0000","last_peered":"2024-09-10T16:09:08.848232+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:09:08.848232+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17303,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.851634+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:09:08.851634+0000","last_peered":"2024-09-10T16:09:08.851634+0000","last_clean":"2024-09-10T16:09:08.851634+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:09:08.851634+0000","last_undegraded":"2024-09-10T16:09:08.851634+0000","last_fullsized":"2024-09-10T16:09:08.851634+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":89,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.851698+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:09:08.851698+0000","last_peered":"2024-09-10T16:09:08.851698+0000","last_clean":"2024-09-10T16:09:08.851698+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:09:08.851698+0000","last_undegraded":"2024-09-10T16:09:08.851698+0000","last_fullsized":"2024-09-10T16:09:08.851698+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'21198","reported_seq":21357,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:08.848736+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:09:08.848736+0000","last_peered":"2024-09-10T16:09:08.848736+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:09:08.848736+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17361,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4321},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33683,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292096,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17707,"num_bytes_recovered":33096163,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":210518016,"data_stored":210395174,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161022,"ondisk_log_size":161022,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986310,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":502616,"kb_used_data":137728,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93869224,"statfs":{"total":96636764160,"available":96122085376,"internally_reserved":0,"allocated":141033472,"data_stored":140713808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659256,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":469484,"kb_used_data":476,"kb_used_omap":4,"kb_used_meta":468987,"kb_avail":93902356,"statfs":{"total":96636764160,"available":96156012544,"internally_reserved":0,"allocated":487424,"data_stored":149231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":480243374},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182538,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":547316,"kb_used_data":68636,"kb_used_omap":4,"kb_used_meta":478651,"kb_avail":93824524,"statfs":{"total":96636764160,"available":96076312576,"internally_reserved":0,"allocated":70283264,"data_stored":69959523,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":490139306},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":69853184,"data_stored":69817054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":6762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":140607488,"data_stored":140571358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:09:16.489 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-10T16:09:16.489 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "49'20672", 'reported_seq': 20761, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:09:08.850557+0000', 'last_change': '2024-09-10T15:59:16.281558+0000', 'last_active': '2024-09-10T16:09:08.850557+0000', 'last_peered': '2024-09-10T16:09:08.850557+0000', 'last_clean': '2024-09-10T15:59:14.132456+0000', 'last_became_active': '2024-09-10T15:59:16.274407+0000', 'last_became_peered': '2024-09-10T15:59:16.274407+0000', 'last_unstale': '2024-09-10T16:09:08.850557+0000', 'last_undegraded': '2024-09-10T15:59:15.851279+0000', 'last_fullsized': '2024-09-10T15:59:15.774356+0000', 'mapping_epoch': 42, 'log_start': "45'10600", 'ondisk_log_start': "45'10600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'objects_scrubbed': 0, 'log_size': 10072, 'log_dups_size': 0, 'ondisk_log_size': 10072, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4259, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20672, 'num_write_kb': 41344, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4259}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:09:16.490 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-10T16:09:16.490 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "49'21061", 'reported_seq': 21214, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:09:08.849450+0000', 'last_change': '2024-09-10T15:59:16.308264+0000', 'last_active': '2024-09-10T16:09:08.849450+0000', 'last_peered': '2024-09-10T16:09:08.849450+0000', 'last_clean': '2024-09-10T15:59:14.111551+0000', 'last_became_active': '2024-09-10T15:59:16.300040+0000', 'last_became_peered': '2024-09-10T15:59:16.300040+0000', 'last_unstale': '2024-09-10T16:09:08.849450+0000', 'last_undegraded': '2024-09-10T15:59:15.878234+0000', 'last_fullsized': '2024-09-10T15:59:15.769328+0000', 'mapping_epoch': 42, 'log_start': "45'11000", 'ondisk_log_start': "45'11000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3442", 'last_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'objects_scrubbed': 3384, 'log_size': 10061, 'log_dups_size': 0, 'ondisk_log_size': 10061, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1292, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4372, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 17673, 'num_write_kb': 28580, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4370}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:09:16.490 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-10T16:09:16.490 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "49'21082", 'reported_seq': 21235, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:09:08.848683+0000', 'last_change': '2024-09-10T15:59:16.331899+0000', 'last_active': '2024-09-10T16:09:08.848683+0000', 'last_peered': '2024-09-10T16:09:08.848683+0000', 'last_clean': '2024-09-10T15:59:14.139436+0000', 'last_became_active': '2024-09-10T15:59:16.318715+0000', 'last_became_peered': '2024-09-10T15:59:16.318715+0000', 'last_unstale': '2024-09-10T16:09:08.848683+0000', 'last_undegraded': '2024-09-10T15:59:15.877524+0000', 'last_fullsized': '2024-09-10T15:59:15.769042+0000', 'mapping_epoch': 42, 'log_start': "45'11000", 'ondisk_log_start': "45'11000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3537", 'last_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'objects_scrubbed': 3497, 'log_size': 10082, 'log_dups_size': 0, 'ondisk_log_size': 10082, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1224, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4237, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 17581, 'num_write_kb': 28172, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4235}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:09:16.490 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-10T16:09:16.490 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "49'20457", 'reported_seq': 21847, 'reported_epoch': 51, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:09:11.153034+0000', 'last_change': '2024-09-10T16:09:07.396124+0000', 'last_active': '2024-09-10T16:09:11.153034+0000', 'last_peered': '2024-09-10T16:09:11.153034+0000', 'last_clean': '2024-09-10T15:59:14.088187+0000', 'last_became_active': '2024-09-10T15:59:16.165810+0000', 'last_became_peered': '2024-09-10T15:59:16.165810+0000', 'last_unstale': '2024-09-10T16:09:11.153034+0000', 'last_undegraded': '2024-09-10T15:59:15.822098+0000', 'last_fullsized': '2024-09-10T15:59:15.769123+0000', 'mapping_epoch': 42, 'log_start': "45'10400", 'ondisk_log_start': "45'10400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2218", 'last_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'objects_scrubbed': 2175, 'log_size': 10057, 'log_dups_size': 0, 'ondisk_log_size': 10057, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 825, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3507, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 18276, 'num_write_kb': 32204, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 618, 'num_bytes_recovered': 4579, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3506}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:09:16.490 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-10T16:09:16.490 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "49'20781", 'reported_seq': 20931, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:09:08.852538+0000', 'last_change': '2024-09-10T15:59:16.284311+0000', 'last_active': '2024-09-10T16:09:08.852538+0000', 'last_peered': '2024-09-10T16:09:08.852538+0000', 'last_clean': '2024-09-10T15:59:14.116041+0000', 'last_became_active': '2024-09-10T15:59:16.274543+0000', 'last_became_peered': '2024-09-10T15:59:16.274543+0000', 'last_unstale': '2024-09-10T16:09:08.852538+0000', 'last_undegraded': '2024-09-10T15:59:15.812484+0000', 'last_fullsized': '2024-09-10T15:59:15.770885+0000', 'mapping_epoch': 42, 'log_start': "45'10700", 'ondisk_log_start': "45'10700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2356", 'last_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'objects_scrubbed': 2316, 'log_size': 10081, 'log_dups_size': 0, 'ondisk_log_size': 10081, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 783, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 18461, 'num_write_kb': 32292, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': -4303}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:09:16.490 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-10T16:09:16.491 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "49'21170", 'reported_seq': 30029, 'reported_epoch': 51, 'state': 'active+recovery_wait', 'last_fresh': '2024-09-10T16:09:08.852233+0000', 'last_change': '2024-09-10T16:08:10.929169+0000', 'last_active': '2024-09-10T16:09:08.852233+0000', 'last_peered': '2024-09-10T16:09:08.852233+0000', 'last_clean': '2024-09-10T15:59:14.110654+0000', 'last_became_active': '2024-09-10T16:08:10.860438+0000', 'last_became_peered': '2024-09-10T16:08:10.860438+0000', 'last_unstale': '2024-09-10T16:09:08.852233+0000', 'last_undegraded': '2024-09-10T16:09:08.852233+0000', 'last_fullsized': '2024-09-10T16:09:08.852233+0000', 'mapping_epoch': 48, 'log_start': "45'11100", 'ondisk_log_start': "45'11100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2486", 'last_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'objects_scrubbed': 2440, 'log_size': 10070, 'log_dups_size': 0, 'ondisk_log_size': 10070, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 792, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18724, 'num_write_kb': 32568, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 4355, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': ['2', '0', '1'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:09:16.491 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-10T16:09:16.491 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "49'20838", 'reported_seq': 20988, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:09:08.849672+0000', 'last_change': '2024-09-10T15:59:16.315641+0000', 'last_active': '2024-09-10T16:09:08.849672+0000', 'last_peered': '2024-09-10T16:09:08.849672+0000', 'last_clean': '2024-09-10T15:59:14.117001+0000', 'last_became_active': '2024-09-10T15:59:16.302245+0000', 'last_became_peered': '2024-09-10T15:59:16.302245+0000', 'last_unstale': '2024-09-10T16:09:08.849672+0000', 'last_undegraded': '2024-09-10T15:59:15.845012+0000', 'last_fullsized': '2024-09-10T15:59:15.770337+0000', 'mapping_epoch': 42, 'log_start': "45'10800", 'ondisk_log_start': "45'10800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3404", 'last_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'objects_scrubbed': 3367, 'log_size': 10038, 'log_dups_size': 0, 'ondisk_log_size': 10038, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 983, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17469, 'num_write_kb': 28204, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4304}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:09:16.491 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-10T16:09:16.491 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "49'21048", 'reported_seq': 21205, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:09:08.848232+0000', 'last_change': '2024-09-10T15:59:16.311291+0000', 'last_active': '2024-09-10T16:09:08.848232+0000', 'last_peered': '2024-09-10T16:09:08.848232+0000', 'last_clean': '2024-09-10T15:59:14.116363+0000', 'last_became_active': '2024-09-10T15:59:16.301244+0000', 'last_became_peered': '2024-09-10T15:59:16.301244+0000', 'last_unstale': '2024-09-10T16:09:08.848232+0000', 'last_undegraded': '2024-09-10T15:59:15.883434+0000', 'last_fullsized': '2024-09-10T15:59:15.772756+0000', 'mapping_epoch': 42, 'log_start': "45'11000", 'ondisk_log_start': "45'11000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3789", 'last_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'objects_scrubbed': 3741, 'log_size': 10048, 'log_dups_size': 0, 'ondisk_log_size': 10048, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1440, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4373, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17303, 'num_write_kb': 27124, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4371}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:09:16.491 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-10T16:09:16.491 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "49'21198", 'reported_seq': 21357, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:09:08.848736+0000', 'last_change': '2024-09-10T15:59:16.313409+0000', 'last_active': '2024-09-10T16:09:08.848736+0000', 'last_peered': '2024-09-10T16:09:08.848736+0000', 'last_clean': '2024-09-10T15:59:14.129780+0000', 'last_became_active': '2024-09-10T15:59:16.300932+0000', 'last_became_peered': '2024-09-10T15:59:16.300932+0000', 'last_unstale': '2024-09-10T16:09:08.848736+0000', 'last_undegraded': '2024-09-10T15:59:15.887823+0000', 'last_fullsized': '2024-09-10T15:59:15.768513+0000', 'mapping_epoch': 42, 'log_start': "45'11100", 'ondisk_log_start': "45'11100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3880", 'last_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'objects_scrubbed': 3827, 'log_size': 10098, 'log_dups_size': 0, 'ondisk_log_size': 10098, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1397, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4325, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17361, 'num_write_kb': 27072, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4321}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:09:16.492 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T16:09:16.880 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:09:16.880 INFO:teuthology.orchestra.run.smithi115.stdout:{"fsid":"7effee29-df05-4351-9741-ee203564ed81","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 33683/24 objects degraded (140345.833%), 8 pgs degraded, 8 pgs undersized","count":16},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":720,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":51,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983954,"num_in_osds":3,"osd_in_since":1725983947,"num_remapped_pgs":8},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":15},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":7},{"state_name":"active+recovery_wait","count":1},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":12,"data_bytes":4849,"bytes_used":1555881984,"bytes_avail":288354410496,"bytes_total":289910292480,"degraded_objects":33683,"degraded_total":24,"degraded_ratio":1403.458333333333,"recovering_objects_per_sec":113,"recovering_bytes_per_sec":60,"recovering_keys_per_sec":0,"num_objects_recovered":909,"num_bytes_recovered":483,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-10T15:57:14:967120+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T16:00:13.560776+0000","services":{}},"progress_events":{}} 2024-09-10T16:09:16.894 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-10T16:09:16.894 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:09:17.207 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:09:17.207 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:09:17.220 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":404,"stamp":"2024-09-10T16:09:15.771193+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33683,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292098,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17709,"num_bytes_recovered":33096182,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161025,"ondisk_log_size":161025,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1519416,"kb_used_data":206840,"kb_used_omap":12,"kb_used_meta":1312499,"kb_avail":281596104,"statfs":{"total":289910292480,"available":288354410496,"internally_reserved":0,"allocated":211804160,"data_stored":210822562,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1343998978},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-909,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":909,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001468"},"pg_stats":[{"pgid":"2.f","version":"49'20672","reported_seq":20761,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:08.850557+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:09:08.850557+0000","last_peered":"2024-09-10T16:09:08.850557+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:09:08.850557+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20672,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4259}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20844","reported_seq":20971,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.853495+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:09:08.853495+0000","last_peered":"2024-09-10T16:09:08.853495+0000","last_clean":"2024-09-10T16:09:08.853495+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:09:08.853495+0000","last_undegraded":"2024-09-10T16:09:08.853495+0000","last_fullsized":"2024-09-10T16:09:08.853495+0000","mapping_epoch":35,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18949,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20798","reported_seq":29491,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.852143+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:09:08.852143+0000","last_peered":"2024-09-10T16:09:08.852143+0000","last_clean":"2024-09-10T16:09:08.852143+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:09:08.852143+0000","last_undegraded":"2024-09-10T16:09:08.852143+0000","last_fullsized":"2024-09-10T16:09:08.852143+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18802,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20660","reported_seq":20755,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.853148+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:09:08.853148+0000","last_peered":"2024-09-10T16:09:08.853148+0000","last_clean":"2024-09-10T16:09:08.853148+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:09:08.853148+0000","last_undegraded":"2024-09-10T16:09:08.853148+0000","last_fullsized":"2024-09-10T16:09:08.853148+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20660,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20891","reported_seq":21036,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.847836+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:09:08.847836+0000","last_peered":"2024-09-10T16:09:08.847836+0000","last_clean":"2024-09-10T16:09:08.847836+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:09:08.847836+0000","last_undegraded":"2024-09-10T16:09:08.847836+0000","last_fullsized":"2024-09-10T16:09:08.847836+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17602,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'21061","reported_seq":21214,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:08.849450+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:09:08.849450+0000","last_peered":"2024-09-10T16:09:08.849450+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:09:08.849450+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17673,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4370},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'21082","reported_seq":21235,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:08.848683+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:09:08.848683+0000","last_peered":"2024-09-10T16:09:08.848683+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:09:08.848683+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17581,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4235},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'21314","reported_seq":21459,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.847784+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:09:08.847784+0000","last_peered":"2024-09-10T16:09:08.847784+0000","last_clean":"2024-09-10T16:09:08.847784+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:09:08.847784+0000","last_undegraded":"2024-09-10T16:09:08.847784+0000","last_fullsized":"2024-09-10T16:09:08.847784+0000","mapping_epoch":34,"log_start":"45'11300","ondisk_log_start":"45'11300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18161,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.851493+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:09:08.851493+0000","last_peered":"2024-09-10T16:09:08.851493+0000","last_clean":"2024-09-10T16:09:08.851493+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:09:08.851493+0000","last_undegraded":"2024-09-10T16:09:08.851493+0000","last_fullsized":"2024-09-10T16:09:08.851493+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20457","reported_seq":21847,"reported_epoch":51,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:11.153034+0000","last_change":"2024-09-10T16:09:07.396124+0000","last_active":"2024-09-10T16:09:11.153034+0000","last_peered":"2024-09-10T16:09:11.153034+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:09:11.153034+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3507,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18276,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":618,"num_bytes_recovered":4579,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3506}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":109,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.852443+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:09:08.852443+0000","last_peered":"2024-09-10T16:09:08.852443+0000","last_clean":"2024-09-10T16:09:08.852443+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:09:08.852443+0000","last_undegraded":"2024-09-10T16:09:08.852443+0000","last_fullsized":"2024-09-10T16:09:08.852443+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20910","reported_seq":29577,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.909669+0000","last_change":"2024-09-10T16:09:08.909580+0000","last_active":"2024-09-10T16:09:08.909669+0000","last_peered":"2024-09-10T16:09:08.909669+0000","last_clean":"2024-09-10T16:09:08.909669+0000","last_became_active":"2024-09-10T16:09:08.854941+0000","last_became_peered":"2024-09-10T16:09:08.854941+0000","last_unstale":"2024-09-10T16:09:08.909669+0000","last_undegraded":"2024-09-10T16:09:08.909669+0000","last_fullsized":"2024-09-10T16:09:08.909669+0000","mapping_epoch":50,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:13:14.846691+0000","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17414,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4257,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":119,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.847624+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:09:08.847624+0000","last_peered":"2024-09-10T16:09:08.847624+0000","last_clean":"2024-09-10T16:09:08.847624+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:09:08.847624+0000","last_undegraded":"2024-09-10T16:09:08.847624+0000","last_fullsized":"2024-09-10T16:09:08.847624+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20781","reported_seq":20931,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:08.852538+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:09:08.852538+0000","last_peered":"2024-09-10T16:09:08.852538+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:09:08.852538+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18461,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-4303}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":94,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.852182+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:09:08.852182+0000","last_peered":"2024-09-10T16:09:08.852182+0000","last_clean":"2024-09-10T16:09:08.852182+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:09:08.852182+0000","last_undegraded":"2024-09-10T16:09:08.852182+0000","last_fullsized":"2024-09-10T16:09:08.852182+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'21170","reported_seq":30029,"reported_epoch":51,"state":"active+recovery_wait","last_fresh":"2024-09-10T16:09:08.852233+0000","last_change":"2024-09-10T16:08:10.929169+0000","last_active":"2024-09-10T16:09:08.852233+0000","last_peered":"2024-09-10T16:09:08.852233+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T16:08:10.860438+0000","last_became_peered":"2024-09-10T16:08:10.860438+0000","last_unstale":"2024-09-10T16:09:08.852233+0000","last_undegraded":"2024-09-10T16:09:08.852233+0000","last_fullsized":"2024-09-10T16:09:08.852233+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18724,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","0","1"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20838","reported_seq":20988,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:08.849672+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:09:08.849672+0000","last_peered":"2024-09-10T16:09:08.849672+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:09:08.849672+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17469,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.851774+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:09:08.851774+0000","last_peered":"2024-09-10T16:09:08.851774+0000","last_clean":"2024-09-10T16:09:08.851774+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:09:08.851774+0000","last_undegraded":"2024-09-10T16:09:08.851774+0000","last_fullsized":"2024-09-10T16:09:08.851774+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20598","reported_seq":29178,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.851713+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:09:08.851713+0000","last_peered":"2024-09-10T16:09:08.851713+0000","last_clean":"2024-09-10T16:09:08.851713+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:09:08.851713+0000","last_undegraded":"2024-09-10T16:09:08.851713+0000","last_fullsized":"2024-09-10T16:09:08.851713+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16988,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":109,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.853357+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:09:08.853357+0000","last_peered":"2024-09-10T16:09:08.853357+0000","last_clean":"2024-09-10T16:09:08.853357+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:09:08.853357+0000","last_undegraded":"2024-09-10T16:09:08.853357+0000","last_fullsized":"2024-09-10T16:09:08.853357+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'21048","reported_seq":21205,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:08.848232+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:09:08.848232+0000","last_peered":"2024-09-10T16:09:08.848232+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:09:08.848232+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17303,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.851634+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:09:08.851634+0000","last_peered":"2024-09-10T16:09:08.851634+0000","last_clean":"2024-09-10T16:09:08.851634+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:09:08.851634+0000","last_undegraded":"2024-09-10T16:09:08.851634+0000","last_fullsized":"2024-09-10T16:09:08.851634+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":89,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.851698+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:09:08.851698+0000","last_peered":"2024-09-10T16:09:08.851698+0000","last_clean":"2024-09-10T16:09:08.851698+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:09:08.851698+0000","last_undegraded":"2024-09-10T16:09:08.851698+0000","last_fullsized":"2024-09-10T16:09:08.851698+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'21198","reported_seq":21357,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:08.848736+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:09:08.848736+0000","last_peered":"2024-09-10T16:09:08.848736+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:09:08.848736+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17361,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4321},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33683,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292096,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17707,"num_bytes_recovered":33096163,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":210518016,"data_stored":210395174,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161022,"ondisk_log_size":161022,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986310,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":502616,"kb_used_data":137728,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93869224,"statfs":{"total":96636764160,"available":96122085376,"internally_reserved":0,"allocated":141033472,"data_stored":140713808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659256,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":469484,"kb_used_data":476,"kb_used_omap":4,"kb_used_meta":468987,"kb_avail":93902356,"statfs":{"total":96636764160,"available":96156012544,"internally_reserved":0,"allocated":487424,"data_stored":149231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":480243374},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182538,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":547316,"kb_used_data":68636,"kb_used_omap":4,"kb_used_meta":478651,"kb_avail":93824524,"statfs":{"total":96636764160,"available":96076312576,"internally_reserved":0,"allocated":70283264,"data_stored":69959523,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":490139306},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":69853184,"data_stored":69817054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":6762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":140607488,"data_stored":140571358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:09:20.222 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:09:20.528 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:09:20.528 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:09:20.541 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":406,"stamp":"2024-09-10T16:09:19.771962+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32882,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292098,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18510,"num_bytes_recovered":33096182,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161025,"ondisk_log_size":161025,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1519416,"kb_used_data":206840,"kb_used_omap":12,"kb_used_meta":1312499,"kb_avail":281596104,"statfs":{"total":289910292480,"available":288354410496,"internally_reserved":0,"allocated":211804160,"data_stored":210822562,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1343998978},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1710,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1710,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.924659"},"pg_stats":[{"pgid":"2.f","version":"49'20672","reported_seq":20761,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:08.850557+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:09:08.850557+0000","last_peered":"2024-09-10T16:09:08.850557+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:09:08.850557+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20672,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4259}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20844","reported_seq":20971,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.853495+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:09:08.853495+0000","last_peered":"2024-09-10T16:09:08.853495+0000","last_clean":"2024-09-10T16:09:08.853495+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:09:08.853495+0000","last_undegraded":"2024-09-10T16:09:08.853495+0000","last_fullsized":"2024-09-10T16:09:08.853495+0000","mapping_epoch":35,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18949,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20798","reported_seq":29491,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.852143+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:09:08.852143+0000","last_peered":"2024-09-10T16:09:08.852143+0000","last_clean":"2024-09-10T16:09:08.852143+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:09:08.852143+0000","last_undegraded":"2024-09-10T16:09:08.852143+0000","last_fullsized":"2024-09-10T16:09:08.852143+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18802,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20660","reported_seq":20755,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.853148+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:09:08.853148+0000","last_peered":"2024-09-10T16:09:08.853148+0000","last_clean":"2024-09-10T16:09:08.853148+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:09:08.853148+0000","last_undegraded":"2024-09-10T16:09:08.853148+0000","last_fullsized":"2024-09-10T16:09:08.853148+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20660,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20891","reported_seq":21036,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.847836+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:09:08.847836+0000","last_peered":"2024-09-10T16:09:08.847836+0000","last_clean":"2024-09-10T16:09:08.847836+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:09:08.847836+0000","last_undegraded":"2024-09-10T16:09:08.847836+0000","last_fullsized":"2024-09-10T16:09:08.847836+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17602,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'21061","reported_seq":21214,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:08.849450+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:09:08.849450+0000","last_peered":"2024-09-10T16:09:08.849450+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:09:08.849450+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17673,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4370},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'21082","reported_seq":21235,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:08.848683+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:09:08.848683+0000","last_peered":"2024-09-10T16:09:08.848683+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:09:08.848683+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17581,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4235},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'21314","reported_seq":21459,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.847784+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:09:08.847784+0000","last_peered":"2024-09-10T16:09:08.847784+0000","last_clean":"2024-09-10T16:09:08.847784+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:09:08.847784+0000","last_undegraded":"2024-09-10T16:09:08.847784+0000","last_fullsized":"2024-09-10T16:09:08.847784+0000","mapping_epoch":34,"log_start":"45'11300","ondisk_log_start":"45'11300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18161,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.851493+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:09:08.851493+0000","last_peered":"2024-09-10T16:09:08.851493+0000","last_clean":"2024-09-10T16:09:08.851493+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:09:08.851493+0000","last_undegraded":"2024-09-10T16:09:08.851493+0000","last_fullsized":"2024-09-10T16:09:08.851493+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20457","reported_seq":23449,"reported_epoch":51,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:16.156769+0000","last_change":"2024-09-10T16:09:07.396124+0000","last_active":"2024-09-10T16:09:16.156769+0000","last_peered":"2024-09-10T16:09:16.156769+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:09:16.156769+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2706,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18276,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1419,"num_bytes_recovered":4579,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2705}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":109,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.852443+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:09:08.852443+0000","last_peered":"2024-09-10T16:09:08.852443+0000","last_clean":"2024-09-10T16:09:08.852443+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:09:08.852443+0000","last_undegraded":"2024-09-10T16:09:08.852443+0000","last_fullsized":"2024-09-10T16:09:08.852443+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20910","reported_seq":29577,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.909669+0000","last_change":"2024-09-10T16:09:08.909580+0000","last_active":"2024-09-10T16:09:08.909669+0000","last_peered":"2024-09-10T16:09:08.909669+0000","last_clean":"2024-09-10T16:09:08.909669+0000","last_became_active":"2024-09-10T16:09:08.854941+0000","last_became_peered":"2024-09-10T16:09:08.854941+0000","last_unstale":"2024-09-10T16:09:08.909669+0000","last_undegraded":"2024-09-10T16:09:08.909669+0000","last_fullsized":"2024-09-10T16:09:08.909669+0000","mapping_epoch":50,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:13:14.846691+0000","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17414,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4257,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":119,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.847624+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:09:08.847624+0000","last_peered":"2024-09-10T16:09:08.847624+0000","last_clean":"2024-09-10T16:09:08.847624+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:09:08.847624+0000","last_undegraded":"2024-09-10T16:09:08.847624+0000","last_fullsized":"2024-09-10T16:09:08.847624+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20781","reported_seq":20931,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:08.852538+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:09:08.852538+0000","last_peered":"2024-09-10T16:09:08.852538+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:09:08.852538+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18461,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-4303}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":94,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.852182+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:09:08.852182+0000","last_peered":"2024-09-10T16:09:08.852182+0000","last_clean":"2024-09-10T16:09:08.852182+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:09:08.852182+0000","last_undegraded":"2024-09-10T16:09:08.852182+0000","last_fullsized":"2024-09-10T16:09:08.852182+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'21170","reported_seq":30029,"reported_epoch":51,"state":"active+recovery_wait","last_fresh":"2024-09-10T16:09:08.852233+0000","last_change":"2024-09-10T16:08:10.929169+0000","last_active":"2024-09-10T16:09:08.852233+0000","last_peered":"2024-09-10T16:09:08.852233+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T16:08:10.860438+0000","last_became_peered":"2024-09-10T16:08:10.860438+0000","last_unstale":"2024-09-10T16:09:08.852233+0000","last_undegraded":"2024-09-10T16:09:08.852233+0000","last_fullsized":"2024-09-10T16:09:08.852233+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18724,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","0","1"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20838","reported_seq":20988,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:08.849672+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:09:08.849672+0000","last_peered":"2024-09-10T16:09:08.849672+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:09:08.849672+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17469,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.851774+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:09:08.851774+0000","last_peered":"2024-09-10T16:09:08.851774+0000","last_clean":"2024-09-10T16:09:08.851774+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:09:08.851774+0000","last_undegraded":"2024-09-10T16:09:08.851774+0000","last_fullsized":"2024-09-10T16:09:08.851774+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20598","reported_seq":29178,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.851713+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:09:08.851713+0000","last_peered":"2024-09-10T16:09:08.851713+0000","last_clean":"2024-09-10T16:09:08.851713+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:09:08.851713+0000","last_undegraded":"2024-09-10T16:09:08.851713+0000","last_fullsized":"2024-09-10T16:09:08.851713+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16988,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":109,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.853357+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:09:08.853357+0000","last_peered":"2024-09-10T16:09:08.853357+0000","last_clean":"2024-09-10T16:09:08.853357+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:09:08.853357+0000","last_undegraded":"2024-09-10T16:09:08.853357+0000","last_fullsized":"2024-09-10T16:09:08.853357+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'21048","reported_seq":21205,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:08.848232+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:09:08.848232+0000","last_peered":"2024-09-10T16:09:08.848232+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:09:08.848232+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17303,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.851634+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:09:08.851634+0000","last_peered":"2024-09-10T16:09:08.851634+0000","last_clean":"2024-09-10T16:09:08.851634+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:09:08.851634+0000","last_undegraded":"2024-09-10T16:09:08.851634+0000","last_fullsized":"2024-09-10T16:09:08.851634+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":89,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.851698+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:09:08.851698+0000","last_peered":"2024-09-10T16:09:08.851698+0000","last_clean":"2024-09-10T16:09:08.851698+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:09:08.851698+0000","last_undegraded":"2024-09-10T16:09:08.851698+0000","last_fullsized":"2024-09-10T16:09:08.851698+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'21198","reported_seq":21357,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:08.848736+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:09:08.848736+0000","last_peered":"2024-09-10T16:09:08.848736+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:09:08.848736+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17361,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4321},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32882,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292096,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18508,"num_bytes_recovered":33096163,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":210518016,"data_stored":210395174,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161022,"ondisk_log_size":161022,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986311,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":502616,"kb_used_data":137728,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93869224,"statfs":{"total":96636764160,"available":96122085376,"internally_reserved":0,"allocated":141033472,"data_stored":140713808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659256,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":469484,"kb_used_data":476,"kb_used_omap":4,"kb_used_meta":468987,"kb_avail":93902356,"statfs":{"total":96636764160,"available":96156012544,"internally_reserved":0,"allocated":487424,"data_stored":149231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":480243374},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182539,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":547316,"kb_used_data":68636,"kb_used_omap":4,"kb_used_meta":478651,"kb_avail":93824524,"statfs":{"total":96636764160,"available":96076312576,"internally_reserved":0,"allocated":70283264,"data_stored":69959523,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":490139306},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":69853184,"data_stored":69817054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":6762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":140607488,"data_stored":140571358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:09:20.542 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:09:20.850 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:09:20.851 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:09:20.863 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":406,"stamp":"2024-09-10T16:09:19.771962+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32882,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292098,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18510,"num_bytes_recovered":33096182,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161025,"ondisk_log_size":161025,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1519416,"kb_used_data":206840,"kb_used_omap":12,"kb_used_meta":1312499,"kb_avail":281596104,"statfs":{"total":289910292480,"available":288354410496,"internally_reserved":0,"allocated":211804160,"data_stored":210822562,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1343998978},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1710,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1710,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.924659"},"pg_stats":[{"pgid":"2.f","version":"49'20672","reported_seq":20761,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:08.850557+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:09:08.850557+0000","last_peered":"2024-09-10T16:09:08.850557+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:09:08.850557+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20672,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4259}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20844","reported_seq":20971,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.853495+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:09:08.853495+0000","last_peered":"2024-09-10T16:09:08.853495+0000","last_clean":"2024-09-10T16:09:08.853495+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:09:08.853495+0000","last_undegraded":"2024-09-10T16:09:08.853495+0000","last_fullsized":"2024-09-10T16:09:08.853495+0000","mapping_epoch":35,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18949,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20798","reported_seq":29491,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.852143+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:09:08.852143+0000","last_peered":"2024-09-10T16:09:08.852143+0000","last_clean":"2024-09-10T16:09:08.852143+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:09:08.852143+0000","last_undegraded":"2024-09-10T16:09:08.852143+0000","last_fullsized":"2024-09-10T16:09:08.852143+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18802,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20660","reported_seq":20755,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.853148+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:09:08.853148+0000","last_peered":"2024-09-10T16:09:08.853148+0000","last_clean":"2024-09-10T16:09:08.853148+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:09:08.853148+0000","last_undegraded":"2024-09-10T16:09:08.853148+0000","last_fullsized":"2024-09-10T16:09:08.853148+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20660,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20891","reported_seq":21036,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.847836+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:09:08.847836+0000","last_peered":"2024-09-10T16:09:08.847836+0000","last_clean":"2024-09-10T16:09:08.847836+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:09:08.847836+0000","last_undegraded":"2024-09-10T16:09:08.847836+0000","last_fullsized":"2024-09-10T16:09:08.847836+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17602,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'21061","reported_seq":21214,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:08.849450+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:09:08.849450+0000","last_peered":"2024-09-10T16:09:08.849450+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:09:08.849450+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17673,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4370},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'21082","reported_seq":21235,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:08.848683+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:09:08.848683+0000","last_peered":"2024-09-10T16:09:08.848683+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:09:08.848683+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17581,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4235},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'21314","reported_seq":21459,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.847784+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:09:08.847784+0000","last_peered":"2024-09-10T16:09:08.847784+0000","last_clean":"2024-09-10T16:09:08.847784+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:09:08.847784+0000","last_undegraded":"2024-09-10T16:09:08.847784+0000","last_fullsized":"2024-09-10T16:09:08.847784+0000","mapping_epoch":34,"log_start":"45'11300","ondisk_log_start":"45'11300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18161,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.851493+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:09:08.851493+0000","last_peered":"2024-09-10T16:09:08.851493+0000","last_clean":"2024-09-10T16:09:08.851493+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:09:08.851493+0000","last_undegraded":"2024-09-10T16:09:08.851493+0000","last_fullsized":"2024-09-10T16:09:08.851493+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20457","reported_seq":23449,"reported_epoch":51,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:16.156769+0000","last_change":"2024-09-10T16:09:07.396124+0000","last_active":"2024-09-10T16:09:16.156769+0000","last_peered":"2024-09-10T16:09:16.156769+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:09:16.156769+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2706,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18276,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1419,"num_bytes_recovered":4579,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2705}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":109,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.852443+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:09:08.852443+0000","last_peered":"2024-09-10T16:09:08.852443+0000","last_clean":"2024-09-10T16:09:08.852443+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:09:08.852443+0000","last_undegraded":"2024-09-10T16:09:08.852443+0000","last_fullsized":"2024-09-10T16:09:08.852443+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20910","reported_seq":29577,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.909669+0000","last_change":"2024-09-10T16:09:08.909580+0000","last_active":"2024-09-10T16:09:08.909669+0000","last_peered":"2024-09-10T16:09:08.909669+0000","last_clean":"2024-09-10T16:09:08.909669+0000","last_became_active":"2024-09-10T16:09:08.854941+0000","last_became_peered":"2024-09-10T16:09:08.854941+0000","last_unstale":"2024-09-10T16:09:08.909669+0000","last_undegraded":"2024-09-10T16:09:08.909669+0000","last_fullsized":"2024-09-10T16:09:08.909669+0000","mapping_epoch":50,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:13:14.846691+0000","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17414,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4257,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":119,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.847624+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:09:08.847624+0000","last_peered":"2024-09-10T16:09:08.847624+0000","last_clean":"2024-09-10T16:09:08.847624+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:09:08.847624+0000","last_undegraded":"2024-09-10T16:09:08.847624+0000","last_fullsized":"2024-09-10T16:09:08.847624+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20781","reported_seq":20931,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:08.852538+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:09:08.852538+0000","last_peered":"2024-09-10T16:09:08.852538+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:09:08.852538+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18461,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-4303}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":94,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.852182+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:09:08.852182+0000","last_peered":"2024-09-10T16:09:08.852182+0000","last_clean":"2024-09-10T16:09:08.852182+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:09:08.852182+0000","last_undegraded":"2024-09-10T16:09:08.852182+0000","last_fullsized":"2024-09-10T16:09:08.852182+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'21170","reported_seq":30029,"reported_epoch":51,"state":"active+recovery_wait","last_fresh":"2024-09-10T16:09:08.852233+0000","last_change":"2024-09-10T16:08:10.929169+0000","last_active":"2024-09-10T16:09:08.852233+0000","last_peered":"2024-09-10T16:09:08.852233+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T16:08:10.860438+0000","last_became_peered":"2024-09-10T16:08:10.860438+0000","last_unstale":"2024-09-10T16:09:08.852233+0000","last_undegraded":"2024-09-10T16:09:08.852233+0000","last_fullsized":"2024-09-10T16:09:08.852233+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18724,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","0","1"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20838","reported_seq":20988,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:08.849672+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:09:08.849672+0000","last_peered":"2024-09-10T16:09:08.849672+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:09:08.849672+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17469,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.851774+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:09:08.851774+0000","last_peered":"2024-09-10T16:09:08.851774+0000","last_clean":"2024-09-10T16:09:08.851774+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:09:08.851774+0000","last_undegraded":"2024-09-10T16:09:08.851774+0000","last_fullsized":"2024-09-10T16:09:08.851774+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20598","reported_seq":29178,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.851713+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:09:08.851713+0000","last_peered":"2024-09-10T16:09:08.851713+0000","last_clean":"2024-09-10T16:09:08.851713+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:09:08.851713+0000","last_undegraded":"2024-09-10T16:09:08.851713+0000","last_fullsized":"2024-09-10T16:09:08.851713+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16988,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":109,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.853357+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:09:08.853357+0000","last_peered":"2024-09-10T16:09:08.853357+0000","last_clean":"2024-09-10T16:09:08.853357+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:09:08.853357+0000","last_undegraded":"2024-09-10T16:09:08.853357+0000","last_fullsized":"2024-09-10T16:09:08.853357+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'21048","reported_seq":21205,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:08.848232+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:09:08.848232+0000","last_peered":"2024-09-10T16:09:08.848232+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:09:08.848232+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17303,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.851634+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:09:08.851634+0000","last_peered":"2024-09-10T16:09:08.851634+0000","last_clean":"2024-09-10T16:09:08.851634+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:09:08.851634+0000","last_undegraded":"2024-09-10T16:09:08.851634+0000","last_fullsized":"2024-09-10T16:09:08.851634+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":89,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.851698+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:09:08.851698+0000","last_peered":"2024-09-10T16:09:08.851698+0000","last_clean":"2024-09-10T16:09:08.851698+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:09:08.851698+0000","last_undegraded":"2024-09-10T16:09:08.851698+0000","last_fullsized":"2024-09-10T16:09:08.851698+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'21198","reported_seq":21357,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:08.848736+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:09:08.848736+0000","last_peered":"2024-09-10T16:09:08.848736+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:09:08.848736+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17361,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4321},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32882,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292096,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18508,"num_bytes_recovered":33096163,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":210518016,"data_stored":210395174,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161022,"ondisk_log_size":161022,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986311,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":502616,"kb_used_data":137728,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93869224,"statfs":{"total":96636764160,"available":96122085376,"internally_reserved":0,"allocated":141033472,"data_stored":140713808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659256,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":469484,"kb_used_data":476,"kb_used_omap":4,"kb_used_meta":468987,"kb_avail":93902356,"statfs":{"total":96636764160,"available":96156012544,"internally_reserved":0,"allocated":487424,"data_stored":149231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":480243374},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182539,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":547316,"kb_used_data":68636,"kb_used_omap":4,"kb_used_meta":478651,"kb_avail":93824524,"statfs":{"total":96636764160,"available":96076312576,"internally_reserved":0,"allocated":70283264,"data_stored":69959523,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":490139306},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":69853184,"data_stored":69817054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":6762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":140607488,"data_stored":140571358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:09:20.865 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-10T16:09:20.865 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "49'20672", 'reported_seq': 20761, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:09:08.850557+0000', 'last_change': '2024-09-10T15:59:16.281558+0000', 'last_active': '2024-09-10T16:09:08.850557+0000', 'last_peered': '2024-09-10T16:09:08.850557+0000', 'last_clean': '2024-09-10T15:59:14.132456+0000', 'last_became_active': '2024-09-10T15:59:16.274407+0000', 'last_became_peered': '2024-09-10T15:59:16.274407+0000', 'last_unstale': '2024-09-10T16:09:08.850557+0000', 'last_undegraded': '2024-09-10T15:59:15.851279+0000', 'last_fullsized': '2024-09-10T15:59:15.774356+0000', 'mapping_epoch': 42, 'log_start': "45'10600", 'ondisk_log_start': "45'10600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'objects_scrubbed': 0, 'log_size': 10072, 'log_dups_size': 0, 'ondisk_log_size': 10072, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4259, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20672, 'num_write_kb': 41344, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4259}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:09:20.865 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-10T16:09:20.865 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "49'21061", 'reported_seq': 21214, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:09:08.849450+0000', 'last_change': '2024-09-10T15:59:16.308264+0000', 'last_active': '2024-09-10T16:09:08.849450+0000', 'last_peered': '2024-09-10T16:09:08.849450+0000', 'last_clean': '2024-09-10T15:59:14.111551+0000', 'last_became_active': '2024-09-10T15:59:16.300040+0000', 'last_became_peered': '2024-09-10T15:59:16.300040+0000', 'last_unstale': '2024-09-10T16:09:08.849450+0000', 'last_undegraded': '2024-09-10T15:59:15.878234+0000', 'last_fullsized': '2024-09-10T15:59:15.769328+0000', 'mapping_epoch': 42, 'log_start': "45'11000", 'ondisk_log_start': "45'11000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3442", 'last_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'objects_scrubbed': 3384, 'log_size': 10061, 'log_dups_size': 0, 'ondisk_log_size': 10061, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1292, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4372, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 17673, 'num_write_kb': 28580, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4370}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:09:20.865 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-10T16:09:20.865 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "49'21082", 'reported_seq': 21235, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:09:08.848683+0000', 'last_change': '2024-09-10T15:59:16.331899+0000', 'last_active': '2024-09-10T16:09:08.848683+0000', 'last_peered': '2024-09-10T16:09:08.848683+0000', 'last_clean': '2024-09-10T15:59:14.139436+0000', 'last_became_active': '2024-09-10T15:59:16.318715+0000', 'last_became_peered': '2024-09-10T15:59:16.318715+0000', 'last_unstale': '2024-09-10T16:09:08.848683+0000', 'last_undegraded': '2024-09-10T15:59:15.877524+0000', 'last_fullsized': '2024-09-10T15:59:15.769042+0000', 'mapping_epoch': 42, 'log_start': "45'11000", 'ondisk_log_start': "45'11000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3537", 'last_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'objects_scrubbed': 3497, 'log_size': 10082, 'log_dups_size': 0, 'ondisk_log_size': 10082, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1224, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4237, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 17581, 'num_write_kb': 28172, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4235}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:09:20.865 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-10T16:09:20.865 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "49'20457", 'reported_seq': 23449, 'reported_epoch': 51, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:09:16.156769+0000', 'last_change': '2024-09-10T16:09:07.396124+0000', 'last_active': '2024-09-10T16:09:16.156769+0000', 'last_peered': '2024-09-10T16:09:16.156769+0000', 'last_clean': '2024-09-10T15:59:14.088187+0000', 'last_became_active': '2024-09-10T15:59:16.165810+0000', 'last_became_peered': '2024-09-10T15:59:16.165810+0000', 'last_unstale': '2024-09-10T16:09:16.156769+0000', 'last_undegraded': '2024-09-10T15:59:15.822098+0000', 'last_fullsized': '2024-09-10T15:59:15.769123+0000', 'mapping_epoch': 42, 'log_start': "45'10400", 'ondisk_log_start': "45'10400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2218", 'last_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'objects_scrubbed': 2175, 'log_size': 10057, 'log_dups_size': 0, 'ondisk_log_size': 10057, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 825, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2706, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 18276, 'num_write_kb': 32204, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1419, 'num_bytes_recovered': 4579, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -2705}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:09:20.865 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-10T16:09:20.866 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "49'20781", 'reported_seq': 20931, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:09:08.852538+0000', 'last_change': '2024-09-10T15:59:16.284311+0000', 'last_active': '2024-09-10T16:09:08.852538+0000', 'last_peered': '2024-09-10T16:09:08.852538+0000', 'last_clean': '2024-09-10T15:59:14.116041+0000', 'last_became_active': '2024-09-10T15:59:16.274543+0000', 'last_became_peered': '2024-09-10T15:59:16.274543+0000', 'last_unstale': '2024-09-10T16:09:08.852538+0000', 'last_undegraded': '2024-09-10T15:59:15.812484+0000', 'last_fullsized': '2024-09-10T15:59:15.770885+0000', 'mapping_epoch': 42, 'log_start': "45'10700", 'ondisk_log_start': "45'10700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2356", 'last_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'objects_scrubbed': 2316, 'log_size': 10081, 'log_dups_size': 0, 'ondisk_log_size': 10081, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 783, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 18461, 'num_write_kb': 32292, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': -4303}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:09:20.866 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-10T16:09:20.866 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "49'21170", 'reported_seq': 30029, 'reported_epoch': 51, 'state': 'active+recovery_wait', 'last_fresh': '2024-09-10T16:09:08.852233+0000', 'last_change': '2024-09-10T16:08:10.929169+0000', 'last_active': '2024-09-10T16:09:08.852233+0000', 'last_peered': '2024-09-10T16:09:08.852233+0000', 'last_clean': '2024-09-10T15:59:14.110654+0000', 'last_became_active': '2024-09-10T16:08:10.860438+0000', 'last_became_peered': '2024-09-10T16:08:10.860438+0000', 'last_unstale': '2024-09-10T16:09:08.852233+0000', 'last_undegraded': '2024-09-10T16:09:08.852233+0000', 'last_fullsized': '2024-09-10T16:09:08.852233+0000', 'mapping_epoch': 48, 'log_start': "45'11100", 'ondisk_log_start': "45'11100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2486", 'last_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'objects_scrubbed': 2440, 'log_size': 10070, 'log_dups_size': 0, 'ondisk_log_size': 10070, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 792, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18724, 'num_write_kb': 32568, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 4355, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': ['2', '0', '1'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:09:20.866 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-10T16:09:20.866 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "49'20838", 'reported_seq': 20988, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:09:08.849672+0000', 'last_change': '2024-09-10T15:59:16.315641+0000', 'last_active': '2024-09-10T16:09:08.849672+0000', 'last_peered': '2024-09-10T16:09:08.849672+0000', 'last_clean': '2024-09-10T15:59:14.117001+0000', 'last_became_active': '2024-09-10T15:59:16.302245+0000', 'last_became_peered': '2024-09-10T15:59:16.302245+0000', 'last_unstale': '2024-09-10T16:09:08.849672+0000', 'last_undegraded': '2024-09-10T15:59:15.845012+0000', 'last_fullsized': '2024-09-10T15:59:15.770337+0000', 'mapping_epoch': 42, 'log_start': "45'10800", 'ondisk_log_start': "45'10800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3404", 'last_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'objects_scrubbed': 3367, 'log_size': 10038, 'log_dups_size': 0, 'ondisk_log_size': 10038, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 983, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17469, 'num_write_kb': 28204, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4304}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:09:20.866 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-10T16:09:20.866 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "49'21048", 'reported_seq': 21205, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:09:08.848232+0000', 'last_change': '2024-09-10T15:59:16.311291+0000', 'last_active': '2024-09-10T16:09:08.848232+0000', 'last_peered': '2024-09-10T16:09:08.848232+0000', 'last_clean': '2024-09-10T15:59:14.116363+0000', 'last_became_active': '2024-09-10T15:59:16.301244+0000', 'last_became_peered': '2024-09-10T15:59:16.301244+0000', 'last_unstale': '2024-09-10T16:09:08.848232+0000', 'last_undegraded': '2024-09-10T15:59:15.883434+0000', 'last_fullsized': '2024-09-10T15:59:15.772756+0000', 'mapping_epoch': 42, 'log_start': "45'11000", 'ondisk_log_start': "45'11000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3789", 'last_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'objects_scrubbed': 3741, 'log_size': 10048, 'log_dups_size': 0, 'ondisk_log_size': 10048, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1440, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4373, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17303, 'num_write_kb': 27124, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4371}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:09:20.866 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-10T16:09:20.866 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "49'21198", 'reported_seq': 21357, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:09:08.848736+0000', 'last_change': '2024-09-10T15:59:16.313409+0000', 'last_active': '2024-09-10T16:09:08.848736+0000', 'last_peered': '2024-09-10T16:09:08.848736+0000', 'last_clean': '2024-09-10T15:59:14.129780+0000', 'last_became_active': '2024-09-10T15:59:16.300932+0000', 'last_became_peered': '2024-09-10T15:59:16.300932+0000', 'last_unstale': '2024-09-10T16:09:08.848736+0000', 'last_undegraded': '2024-09-10T15:59:15.887823+0000', 'last_fullsized': '2024-09-10T15:59:15.768513+0000', 'mapping_epoch': 42, 'log_start': "45'11100", 'ondisk_log_start': "45'11100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3880", 'last_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'objects_scrubbed': 3827, 'log_size': 10098, 'log_dups_size': 0, 'ondisk_log_size': 10098, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1397, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4325, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17361, 'num_write_kb': 27072, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4321}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:09:20.867 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T16:09:21.296 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:09:21.296 INFO:teuthology.orchestra.run.smithi115.stdout:{"fsid":"7effee29-df05-4351-9741-ee203564ed81","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 32882/24 objects degraded (137008.333%), 8 pgs degraded, 8 pgs undersized","count":16},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":725,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":51,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983954,"num_in_osds":3,"osd_in_since":1725983947,"num_remapped_pgs":8},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":15},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":7},{"state_name":"active+recovery_wait","count":1},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":12,"data_bytes":4849,"bytes_used":1555881984,"bytes_avail":288354410496,"bytes_total":289910292480,"degraded_objects":32882,"degraded_total":24,"degraded_ratio":1370.083333333333,"recovering_objects_per_sec":156,"recovering_bytes_per_sec":44,"recovering_keys_per_sec":0,"num_objects_recovered":1710,"num_bytes_recovered":483,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-10T15:57:14:967120+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T16:00:13.560776+0000","services":{}},"progress_events":{}} 2024-09-10T16:09:21.312 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-10T16:09:21.312 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:09:21.630 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:09:21.631 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:09:21.643 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":406,"stamp":"2024-09-10T16:09:19.771962+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32882,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292098,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18510,"num_bytes_recovered":33096182,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161025,"ondisk_log_size":161025,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1519416,"kb_used_data":206840,"kb_used_omap":12,"kb_used_meta":1312499,"kb_avail":281596104,"statfs":{"total":289910292480,"available":288354410496,"internally_reserved":0,"allocated":211804160,"data_stored":210822562,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1343998978},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1710,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1710,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.924659"},"pg_stats":[{"pgid":"2.f","version":"49'20672","reported_seq":20761,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:08.850557+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:09:08.850557+0000","last_peered":"2024-09-10T16:09:08.850557+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:09:08.850557+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20672,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4259}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20844","reported_seq":20971,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.853495+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:09:08.853495+0000","last_peered":"2024-09-10T16:09:08.853495+0000","last_clean":"2024-09-10T16:09:08.853495+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:09:08.853495+0000","last_undegraded":"2024-09-10T16:09:08.853495+0000","last_fullsized":"2024-09-10T16:09:08.853495+0000","mapping_epoch":35,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18949,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20798","reported_seq":29491,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.852143+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:09:08.852143+0000","last_peered":"2024-09-10T16:09:08.852143+0000","last_clean":"2024-09-10T16:09:08.852143+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:09:08.852143+0000","last_undegraded":"2024-09-10T16:09:08.852143+0000","last_fullsized":"2024-09-10T16:09:08.852143+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18802,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20660","reported_seq":20755,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.853148+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:09:08.853148+0000","last_peered":"2024-09-10T16:09:08.853148+0000","last_clean":"2024-09-10T16:09:08.853148+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:09:08.853148+0000","last_undegraded":"2024-09-10T16:09:08.853148+0000","last_fullsized":"2024-09-10T16:09:08.853148+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20660,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20891","reported_seq":21036,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.847836+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:09:08.847836+0000","last_peered":"2024-09-10T16:09:08.847836+0000","last_clean":"2024-09-10T16:09:08.847836+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:09:08.847836+0000","last_undegraded":"2024-09-10T16:09:08.847836+0000","last_fullsized":"2024-09-10T16:09:08.847836+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17602,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'21061","reported_seq":21214,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:08.849450+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:09:08.849450+0000","last_peered":"2024-09-10T16:09:08.849450+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:09:08.849450+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17673,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4370},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'21082","reported_seq":21235,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:08.848683+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:09:08.848683+0000","last_peered":"2024-09-10T16:09:08.848683+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:09:08.848683+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17581,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4235},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'21314","reported_seq":21459,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.847784+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:09:08.847784+0000","last_peered":"2024-09-10T16:09:08.847784+0000","last_clean":"2024-09-10T16:09:08.847784+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:09:08.847784+0000","last_undegraded":"2024-09-10T16:09:08.847784+0000","last_fullsized":"2024-09-10T16:09:08.847784+0000","mapping_epoch":34,"log_start":"45'11300","ondisk_log_start":"45'11300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18161,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.851493+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:09:08.851493+0000","last_peered":"2024-09-10T16:09:08.851493+0000","last_clean":"2024-09-10T16:09:08.851493+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:09:08.851493+0000","last_undegraded":"2024-09-10T16:09:08.851493+0000","last_fullsized":"2024-09-10T16:09:08.851493+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20457","reported_seq":23449,"reported_epoch":51,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:16.156769+0000","last_change":"2024-09-10T16:09:07.396124+0000","last_active":"2024-09-10T16:09:16.156769+0000","last_peered":"2024-09-10T16:09:16.156769+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:09:16.156769+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2706,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18276,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1419,"num_bytes_recovered":4579,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2705}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":109,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.852443+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:09:08.852443+0000","last_peered":"2024-09-10T16:09:08.852443+0000","last_clean":"2024-09-10T16:09:08.852443+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:09:08.852443+0000","last_undegraded":"2024-09-10T16:09:08.852443+0000","last_fullsized":"2024-09-10T16:09:08.852443+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20910","reported_seq":29577,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.909669+0000","last_change":"2024-09-10T16:09:08.909580+0000","last_active":"2024-09-10T16:09:08.909669+0000","last_peered":"2024-09-10T16:09:08.909669+0000","last_clean":"2024-09-10T16:09:08.909669+0000","last_became_active":"2024-09-10T16:09:08.854941+0000","last_became_peered":"2024-09-10T16:09:08.854941+0000","last_unstale":"2024-09-10T16:09:08.909669+0000","last_undegraded":"2024-09-10T16:09:08.909669+0000","last_fullsized":"2024-09-10T16:09:08.909669+0000","mapping_epoch":50,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:13:14.846691+0000","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17414,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4257,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":119,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.847624+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:09:08.847624+0000","last_peered":"2024-09-10T16:09:08.847624+0000","last_clean":"2024-09-10T16:09:08.847624+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:09:08.847624+0000","last_undegraded":"2024-09-10T16:09:08.847624+0000","last_fullsized":"2024-09-10T16:09:08.847624+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20781","reported_seq":20931,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:08.852538+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:09:08.852538+0000","last_peered":"2024-09-10T16:09:08.852538+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:09:08.852538+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18461,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-4303}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":94,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.852182+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:09:08.852182+0000","last_peered":"2024-09-10T16:09:08.852182+0000","last_clean":"2024-09-10T16:09:08.852182+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:09:08.852182+0000","last_undegraded":"2024-09-10T16:09:08.852182+0000","last_fullsized":"2024-09-10T16:09:08.852182+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'21170","reported_seq":30029,"reported_epoch":51,"state":"active+recovery_wait","last_fresh":"2024-09-10T16:09:08.852233+0000","last_change":"2024-09-10T16:08:10.929169+0000","last_active":"2024-09-10T16:09:08.852233+0000","last_peered":"2024-09-10T16:09:08.852233+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T16:08:10.860438+0000","last_became_peered":"2024-09-10T16:08:10.860438+0000","last_unstale":"2024-09-10T16:09:08.852233+0000","last_undegraded":"2024-09-10T16:09:08.852233+0000","last_fullsized":"2024-09-10T16:09:08.852233+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18724,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","0","1"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20838","reported_seq":20988,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:08.849672+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:09:08.849672+0000","last_peered":"2024-09-10T16:09:08.849672+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:09:08.849672+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17469,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.851774+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:09:08.851774+0000","last_peered":"2024-09-10T16:09:08.851774+0000","last_clean":"2024-09-10T16:09:08.851774+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:09:08.851774+0000","last_undegraded":"2024-09-10T16:09:08.851774+0000","last_fullsized":"2024-09-10T16:09:08.851774+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20598","reported_seq":29178,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.851713+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:09:08.851713+0000","last_peered":"2024-09-10T16:09:08.851713+0000","last_clean":"2024-09-10T16:09:08.851713+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:09:08.851713+0000","last_undegraded":"2024-09-10T16:09:08.851713+0000","last_fullsized":"2024-09-10T16:09:08.851713+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16988,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":109,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.853357+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:09:08.853357+0000","last_peered":"2024-09-10T16:09:08.853357+0000","last_clean":"2024-09-10T16:09:08.853357+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:09:08.853357+0000","last_undegraded":"2024-09-10T16:09:08.853357+0000","last_fullsized":"2024-09-10T16:09:08.853357+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'21048","reported_seq":21205,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:08.848232+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:09:08.848232+0000","last_peered":"2024-09-10T16:09:08.848232+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:09:08.848232+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17303,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.851634+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:09:08.851634+0000","last_peered":"2024-09-10T16:09:08.851634+0000","last_clean":"2024-09-10T16:09:08.851634+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:09:08.851634+0000","last_undegraded":"2024-09-10T16:09:08.851634+0000","last_fullsized":"2024-09-10T16:09:08.851634+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":89,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.851698+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:09:08.851698+0000","last_peered":"2024-09-10T16:09:08.851698+0000","last_clean":"2024-09-10T16:09:08.851698+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:09:08.851698+0000","last_undegraded":"2024-09-10T16:09:08.851698+0000","last_fullsized":"2024-09-10T16:09:08.851698+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'21198","reported_seq":21357,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:08.848736+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:09:08.848736+0000","last_peered":"2024-09-10T16:09:08.848736+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:09:08.848736+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17361,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4321},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32882,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292096,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18508,"num_bytes_recovered":33096163,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":210518016,"data_stored":210395174,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161022,"ondisk_log_size":161022,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986311,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":502616,"kb_used_data":137728,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93869224,"statfs":{"total":96636764160,"available":96122085376,"internally_reserved":0,"allocated":141033472,"data_stored":140713808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659256,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":469484,"kb_used_data":476,"kb_used_omap":4,"kb_used_meta":468987,"kb_avail":93902356,"statfs":{"total":96636764160,"available":96156012544,"internally_reserved":0,"allocated":487424,"data_stored":149231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":480243374},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182539,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":547316,"kb_used_data":68636,"kb_used_omap":4,"kb_used_meta":478651,"kb_avail":93824524,"statfs":{"total":96636764160,"available":96076312576,"internally_reserved":0,"allocated":70283264,"data_stored":69959523,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":490139306},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":69853184,"data_stored":69817054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":6762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":140607488,"data_stored":140571358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:09:24.645 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:09:24.970 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:09:24.971 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:09:24.982 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":408,"stamp":"2024-09-10T16:09:23.772660+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32213,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292098,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19179,"num_bytes_recovered":33096182,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161025,"ondisk_log_size":161025,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1523576,"kb_used_data":206840,"kb_used_omap":12,"kb_used_meta":1316659,"kb_avail":281591944,"statfs":{"total":289910292480,"available":288350150656,"internally_reserved":0,"allocated":211804160,"data_stored":210822562,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1348258818},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1470,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1470,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002219"},"pg_stats":[{"pgid":"2.f","version":"49'20672","reported_seq":20762,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:08.851657+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:09:08.851657+0000","last_peered":"2024-09-10T16:09:08.851657+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:09:08.851657+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20672,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4259}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20844","reported_seq":20971,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.853495+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:09:08.853495+0000","last_peered":"2024-09-10T16:09:08.853495+0000","last_clean":"2024-09-10T16:09:08.853495+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:09:08.853495+0000","last_undegraded":"2024-09-10T16:09:08.853495+0000","last_fullsized":"2024-09-10T16:09:08.853495+0000","mapping_epoch":35,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18949,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20798","reported_seq":29491,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.852143+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:09:08.852143+0000","last_peered":"2024-09-10T16:09:08.852143+0000","last_clean":"2024-09-10T16:09:08.852143+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:09:08.852143+0000","last_undegraded":"2024-09-10T16:09:08.852143+0000","last_fullsized":"2024-09-10T16:09:08.852143+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18802,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20660","reported_seq":20755,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.853148+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:09:08.853148+0000","last_peered":"2024-09-10T16:09:08.853148+0000","last_clean":"2024-09-10T16:09:08.853148+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:09:08.853148+0000","last_undegraded":"2024-09-10T16:09:08.853148+0000","last_fullsized":"2024-09-10T16:09:08.853148+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20660,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20891","reported_seq":21036,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.847836+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:09:08.847836+0000","last_peered":"2024-09-10T16:09:08.847836+0000","last_clean":"2024-09-10T16:09:08.847836+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:09:08.847836+0000","last_undegraded":"2024-09-10T16:09:08.847836+0000","last_fullsized":"2024-09-10T16:09:08.847836+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17602,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'21061","reported_seq":21214,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:08.849450+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:09:08.849450+0000","last_peered":"2024-09-10T16:09:08.849450+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:09:08.849450+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17673,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4370},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'21082","reported_seq":21235,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:08.848683+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:09:08.848683+0000","last_peered":"2024-09-10T16:09:08.848683+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:09:08.848683+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17581,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4235},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'21314","reported_seq":21459,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.847784+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:09:08.847784+0000","last_peered":"2024-09-10T16:09:08.847784+0000","last_clean":"2024-09-10T16:09:08.847784+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:09:08.847784+0000","last_undegraded":"2024-09-10T16:09:08.847784+0000","last_fullsized":"2024-09-10T16:09:08.847784+0000","mapping_epoch":34,"log_start":"45'11300","ondisk_log_start":"45'11300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18161,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":85,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.851620+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:09:08.851620+0000","last_peered":"2024-09-10T16:09:08.851620+0000","last_clean":"2024-09-10T16:09:08.851620+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:09:08.851620+0000","last_undegraded":"2024-09-10T16:09:08.851620+0000","last_fullsized":"2024-09-10T16:09:08.851620+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20457","reported_seq":24787,"reported_epoch":51,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:21.156341+0000","last_change":"2024-09-10T16:09:07.396124+0000","last_active":"2024-09-10T16:09:21.156341+0000","last_peered":"2024-09-10T16:09:21.156341+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:09:21.156341+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2037,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18276,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2088,"num_bytes_recovered":4579,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2036}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":109,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.852443+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:09:08.852443+0000","last_peered":"2024-09-10T16:09:08.852443+0000","last_clean":"2024-09-10T16:09:08.852443+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:09:08.852443+0000","last_undegraded":"2024-09-10T16:09:08.852443+0000","last_fullsized":"2024-09-10T16:09:08.852443+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20910","reported_seq":29577,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.909669+0000","last_change":"2024-09-10T16:09:08.909580+0000","last_active":"2024-09-10T16:09:08.909669+0000","last_peered":"2024-09-10T16:09:08.909669+0000","last_clean":"2024-09-10T16:09:08.909669+0000","last_became_active":"2024-09-10T16:09:08.854941+0000","last_became_peered":"2024-09-10T16:09:08.854941+0000","last_unstale":"2024-09-10T16:09:08.909669+0000","last_undegraded":"2024-09-10T16:09:08.909669+0000","last_fullsized":"2024-09-10T16:09:08.909669+0000","mapping_epoch":50,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:13:14.846691+0000","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17414,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4257,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":119,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.847624+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:09:08.847624+0000","last_peered":"2024-09-10T16:09:08.847624+0000","last_clean":"2024-09-10T16:09:08.847624+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:09:08.847624+0000","last_undegraded":"2024-09-10T16:09:08.847624+0000","last_fullsized":"2024-09-10T16:09:08.847624+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20781","reported_seq":20931,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:08.852538+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:09:08.852538+0000","last_peered":"2024-09-10T16:09:08.852538+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:09:08.852538+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18461,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-4303}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":94,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.852182+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:09:08.852182+0000","last_peered":"2024-09-10T16:09:08.852182+0000","last_clean":"2024-09-10T16:09:08.852182+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:09:08.852182+0000","last_undegraded":"2024-09-10T16:09:08.852182+0000","last_fullsized":"2024-09-10T16:09:08.852182+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'21170","reported_seq":30029,"reported_epoch":51,"state":"active+recovery_wait","last_fresh":"2024-09-10T16:09:08.852233+0000","last_change":"2024-09-10T16:08:10.929169+0000","last_active":"2024-09-10T16:09:08.852233+0000","last_peered":"2024-09-10T16:09:08.852233+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T16:08:10.860438+0000","last_became_peered":"2024-09-10T16:08:10.860438+0000","last_unstale":"2024-09-10T16:09:08.852233+0000","last_undegraded":"2024-09-10T16:09:08.852233+0000","last_fullsized":"2024-09-10T16:09:08.852233+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18724,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","0","1"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20838","reported_seq":20988,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:08.849672+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:09:08.849672+0000","last_peered":"2024-09-10T16:09:08.849672+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:09:08.849672+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17469,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.851774+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:09:08.851774+0000","last_peered":"2024-09-10T16:09:08.851774+0000","last_clean":"2024-09-10T16:09:08.851774+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:09:08.851774+0000","last_undegraded":"2024-09-10T16:09:08.851774+0000","last_fullsized":"2024-09-10T16:09:08.851774+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20598","reported_seq":29178,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.851713+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:09:08.851713+0000","last_peered":"2024-09-10T16:09:08.851713+0000","last_clean":"2024-09-10T16:09:08.851713+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:09:08.851713+0000","last_undegraded":"2024-09-10T16:09:08.851713+0000","last_fullsized":"2024-09-10T16:09:08.851713+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16988,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":109,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.853357+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:09:08.853357+0000","last_peered":"2024-09-10T16:09:08.853357+0000","last_clean":"2024-09-10T16:09:08.853357+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:09:08.853357+0000","last_undegraded":"2024-09-10T16:09:08.853357+0000","last_fullsized":"2024-09-10T16:09:08.853357+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'21048","reported_seq":21205,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:08.848232+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:09:08.848232+0000","last_peered":"2024-09-10T16:09:08.848232+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:09:08.848232+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17303,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.851634+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:09:08.851634+0000","last_peered":"2024-09-10T16:09:08.851634+0000","last_clean":"2024-09-10T16:09:08.851634+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:09:08.851634+0000","last_undegraded":"2024-09-10T16:09:08.851634+0000","last_fullsized":"2024-09-10T16:09:08.851634+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":89,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.851698+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:09:08.851698+0000","last_peered":"2024-09-10T16:09:08.851698+0000","last_clean":"2024-09-10T16:09:08.851698+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:09:08.851698+0000","last_undegraded":"2024-09-10T16:09:08.851698+0000","last_fullsized":"2024-09-10T16:09:08.851698+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'21198","reported_seq":21357,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:08.848736+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:09:08.848736+0000","last_peered":"2024-09-10T16:09:08.848736+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:09:08.848736+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17361,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4321},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32213,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292096,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19177,"num_bytes_recovered":33096163,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":210518016,"data_stored":210395174,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161022,"ondisk_log_size":161022,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986312,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":502616,"kb_used_data":137728,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93869224,"statfs":{"total":96636764160,"available":96122085376,"internally_reserved":0,"allocated":141033472,"data_stored":140713808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659257,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":473644,"kb_used_data":476,"kb_used_omap":4,"kb_used_meta":473147,"kb_avail":93898196,"statfs":{"total":96636764160,"available":96151752704,"internally_reserved":0,"allocated":487424,"data_stored":149231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":484503214},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182540,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":547316,"kb_used_data":68636,"kb_used_omap":4,"kb_used_meta":478651,"kb_avail":93824524,"statfs":{"total":96636764160,"available":96076312576,"internally_reserved":0,"allocated":70283264,"data_stored":69959523,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":490139306},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":69853184,"data_stored":69817054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":6762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":140607488,"data_stored":140571358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:09:24.984 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:09:25.290 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:09:25.291 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:09:25.303 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":408,"stamp":"2024-09-10T16:09:23.772660+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32213,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292098,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19179,"num_bytes_recovered":33096182,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161025,"ondisk_log_size":161025,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1523576,"kb_used_data":206840,"kb_used_omap":12,"kb_used_meta":1316659,"kb_avail":281591944,"statfs":{"total":289910292480,"available":288350150656,"internally_reserved":0,"allocated":211804160,"data_stored":210822562,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1348258818},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1470,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1470,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002219"},"pg_stats":[{"pgid":"2.f","version":"49'20672","reported_seq":20762,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:08.851657+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:09:08.851657+0000","last_peered":"2024-09-10T16:09:08.851657+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:09:08.851657+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20672,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4259}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20844","reported_seq":20971,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.853495+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:09:08.853495+0000","last_peered":"2024-09-10T16:09:08.853495+0000","last_clean":"2024-09-10T16:09:08.853495+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:09:08.853495+0000","last_undegraded":"2024-09-10T16:09:08.853495+0000","last_fullsized":"2024-09-10T16:09:08.853495+0000","mapping_epoch":35,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18949,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20798","reported_seq":29491,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.852143+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:09:08.852143+0000","last_peered":"2024-09-10T16:09:08.852143+0000","last_clean":"2024-09-10T16:09:08.852143+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:09:08.852143+0000","last_undegraded":"2024-09-10T16:09:08.852143+0000","last_fullsized":"2024-09-10T16:09:08.852143+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18802,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20660","reported_seq":20755,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.853148+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:09:08.853148+0000","last_peered":"2024-09-10T16:09:08.853148+0000","last_clean":"2024-09-10T16:09:08.853148+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:09:08.853148+0000","last_undegraded":"2024-09-10T16:09:08.853148+0000","last_fullsized":"2024-09-10T16:09:08.853148+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20660,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20891","reported_seq":21036,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.847836+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:09:08.847836+0000","last_peered":"2024-09-10T16:09:08.847836+0000","last_clean":"2024-09-10T16:09:08.847836+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:09:08.847836+0000","last_undegraded":"2024-09-10T16:09:08.847836+0000","last_fullsized":"2024-09-10T16:09:08.847836+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17602,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'21061","reported_seq":21214,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:08.849450+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:09:08.849450+0000","last_peered":"2024-09-10T16:09:08.849450+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:09:08.849450+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17673,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4370},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'21082","reported_seq":21235,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:08.848683+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:09:08.848683+0000","last_peered":"2024-09-10T16:09:08.848683+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:09:08.848683+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17581,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4235},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'21314","reported_seq":21459,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.847784+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:09:08.847784+0000","last_peered":"2024-09-10T16:09:08.847784+0000","last_clean":"2024-09-10T16:09:08.847784+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:09:08.847784+0000","last_undegraded":"2024-09-10T16:09:08.847784+0000","last_fullsized":"2024-09-10T16:09:08.847784+0000","mapping_epoch":34,"log_start":"45'11300","ondisk_log_start":"45'11300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18161,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":85,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.851620+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:09:08.851620+0000","last_peered":"2024-09-10T16:09:08.851620+0000","last_clean":"2024-09-10T16:09:08.851620+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:09:08.851620+0000","last_undegraded":"2024-09-10T16:09:08.851620+0000","last_fullsized":"2024-09-10T16:09:08.851620+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20457","reported_seq":24787,"reported_epoch":51,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:21.156341+0000","last_change":"2024-09-10T16:09:07.396124+0000","last_active":"2024-09-10T16:09:21.156341+0000","last_peered":"2024-09-10T16:09:21.156341+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:09:21.156341+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2037,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18276,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2088,"num_bytes_recovered":4579,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2036}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":109,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.852443+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:09:08.852443+0000","last_peered":"2024-09-10T16:09:08.852443+0000","last_clean":"2024-09-10T16:09:08.852443+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:09:08.852443+0000","last_undegraded":"2024-09-10T16:09:08.852443+0000","last_fullsized":"2024-09-10T16:09:08.852443+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20910","reported_seq":29577,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.909669+0000","last_change":"2024-09-10T16:09:08.909580+0000","last_active":"2024-09-10T16:09:08.909669+0000","last_peered":"2024-09-10T16:09:08.909669+0000","last_clean":"2024-09-10T16:09:08.909669+0000","last_became_active":"2024-09-10T16:09:08.854941+0000","last_became_peered":"2024-09-10T16:09:08.854941+0000","last_unstale":"2024-09-10T16:09:08.909669+0000","last_undegraded":"2024-09-10T16:09:08.909669+0000","last_fullsized":"2024-09-10T16:09:08.909669+0000","mapping_epoch":50,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:13:14.846691+0000","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17414,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4257,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":119,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.847624+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:09:08.847624+0000","last_peered":"2024-09-10T16:09:08.847624+0000","last_clean":"2024-09-10T16:09:08.847624+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:09:08.847624+0000","last_undegraded":"2024-09-10T16:09:08.847624+0000","last_fullsized":"2024-09-10T16:09:08.847624+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20781","reported_seq":20931,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:08.852538+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:09:08.852538+0000","last_peered":"2024-09-10T16:09:08.852538+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:09:08.852538+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18461,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-4303}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":94,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.852182+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:09:08.852182+0000","last_peered":"2024-09-10T16:09:08.852182+0000","last_clean":"2024-09-10T16:09:08.852182+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:09:08.852182+0000","last_undegraded":"2024-09-10T16:09:08.852182+0000","last_fullsized":"2024-09-10T16:09:08.852182+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'21170","reported_seq":30029,"reported_epoch":51,"state":"active+recovery_wait","last_fresh":"2024-09-10T16:09:08.852233+0000","last_change":"2024-09-10T16:08:10.929169+0000","last_active":"2024-09-10T16:09:08.852233+0000","last_peered":"2024-09-10T16:09:08.852233+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T16:08:10.860438+0000","last_became_peered":"2024-09-10T16:08:10.860438+0000","last_unstale":"2024-09-10T16:09:08.852233+0000","last_undegraded":"2024-09-10T16:09:08.852233+0000","last_fullsized":"2024-09-10T16:09:08.852233+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18724,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","0","1"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20838","reported_seq":20988,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:08.849672+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:09:08.849672+0000","last_peered":"2024-09-10T16:09:08.849672+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:09:08.849672+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17469,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.851774+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:09:08.851774+0000","last_peered":"2024-09-10T16:09:08.851774+0000","last_clean":"2024-09-10T16:09:08.851774+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:09:08.851774+0000","last_undegraded":"2024-09-10T16:09:08.851774+0000","last_fullsized":"2024-09-10T16:09:08.851774+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20598","reported_seq":29178,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.851713+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:09:08.851713+0000","last_peered":"2024-09-10T16:09:08.851713+0000","last_clean":"2024-09-10T16:09:08.851713+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:09:08.851713+0000","last_undegraded":"2024-09-10T16:09:08.851713+0000","last_fullsized":"2024-09-10T16:09:08.851713+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16988,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":109,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.853357+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:09:08.853357+0000","last_peered":"2024-09-10T16:09:08.853357+0000","last_clean":"2024-09-10T16:09:08.853357+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:09:08.853357+0000","last_undegraded":"2024-09-10T16:09:08.853357+0000","last_fullsized":"2024-09-10T16:09:08.853357+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'21048","reported_seq":21205,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:08.848232+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:09:08.848232+0000","last_peered":"2024-09-10T16:09:08.848232+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:09:08.848232+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17303,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.851634+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:09:08.851634+0000","last_peered":"2024-09-10T16:09:08.851634+0000","last_clean":"2024-09-10T16:09:08.851634+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:09:08.851634+0000","last_undegraded":"2024-09-10T16:09:08.851634+0000","last_fullsized":"2024-09-10T16:09:08.851634+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":89,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.851698+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:09:08.851698+0000","last_peered":"2024-09-10T16:09:08.851698+0000","last_clean":"2024-09-10T16:09:08.851698+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:09:08.851698+0000","last_undegraded":"2024-09-10T16:09:08.851698+0000","last_fullsized":"2024-09-10T16:09:08.851698+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'21198","reported_seq":21357,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:08.848736+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:09:08.848736+0000","last_peered":"2024-09-10T16:09:08.848736+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:09:08.848736+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17361,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4321},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32213,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292096,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19177,"num_bytes_recovered":33096163,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":210518016,"data_stored":210395174,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161022,"ondisk_log_size":161022,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986312,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":502616,"kb_used_data":137728,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93869224,"statfs":{"total":96636764160,"available":96122085376,"internally_reserved":0,"allocated":141033472,"data_stored":140713808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659257,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":473644,"kb_used_data":476,"kb_used_omap":4,"kb_used_meta":473147,"kb_avail":93898196,"statfs":{"total":96636764160,"available":96151752704,"internally_reserved":0,"allocated":487424,"data_stored":149231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":484503214},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182540,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":547316,"kb_used_data":68636,"kb_used_omap":4,"kb_used_meta":478651,"kb_avail":93824524,"statfs":{"total":96636764160,"available":96076312576,"internally_reserved":0,"allocated":70283264,"data_stored":69959523,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":490139306},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":69853184,"data_stored":69817054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":6762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":140607488,"data_stored":140571358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:09:25.305 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-10T16:09:25.305 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "49'20672", 'reported_seq': 20762, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:09:08.851657+0000', 'last_change': '2024-09-10T15:59:16.281558+0000', 'last_active': '2024-09-10T16:09:08.851657+0000', 'last_peered': '2024-09-10T16:09:08.851657+0000', 'last_clean': '2024-09-10T15:59:14.132456+0000', 'last_became_active': '2024-09-10T15:59:16.274407+0000', 'last_became_peered': '2024-09-10T15:59:16.274407+0000', 'last_unstale': '2024-09-10T16:09:08.851657+0000', 'last_undegraded': '2024-09-10T15:59:15.851279+0000', 'last_fullsized': '2024-09-10T15:59:15.774356+0000', 'mapping_epoch': 42, 'log_start': "45'10600", 'ondisk_log_start': "45'10600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'objects_scrubbed': 0, 'log_size': 10072, 'log_dups_size': 0, 'ondisk_log_size': 10072, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4259, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20672, 'num_write_kb': 41344, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4259}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:09:25.305 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-10T16:09:25.305 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "49'21061", 'reported_seq': 21214, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:09:08.849450+0000', 'last_change': '2024-09-10T15:59:16.308264+0000', 'last_active': '2024-09-10T16:09:08.849450+0000', 'last_peered': '2024-09-10T16:09:08.849450+0000', 'last_clean': '2024-09-10T15:59:14.111551+0000', 'last_became_active': '2024-09-10T15:59:16.300040+0000', 'last_became_peered': '2024-09-10T15:59:16.300040+0000', 'last_unstale': '2024-09-10T16:09:08.849450+0000', 'last_undegraded': '2024-09-10T15:59:15.878234+0000', 'last_fullsized': '2024-09-10T15:59:15.769328+0000', 'mapping_epoch': 42, 'log_start': "45'11000", 'ondisk_log_start': "45'11000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3442", 'last_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'objects_scrubbed': 3384, 'log_size': 10061, 'log_dups_size': 0, 'ondisk_log_size': 10061, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1292, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4372, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 17673, 'num_write_kb': 28580, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4370}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:09:25.305 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-10T16:09:25.305 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "49'21082", 'reported_seq': 21235, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:09:08.848683+0000', 'last_change': '2024-09-10T15:59:16.331899+0000', 'last_active': '2024-09-10T16:09:08.848683+0000', 'last_peered': '2024-09-10T16:09:08.848683+0000', 'last_clean': '2024-09-10T15:59:14.139436+0000', 'last_became_active': '2024-09-10T15:59:16.318715+0000', 'last_became_peered': '2024-09-10T15:59:16.318715+0000', 'last_unstale': '2024-09-10T16:09:08.848683+0000', 'last_undegraded': '2024-09-10T15:59:15.877524+0000', 'last_fullsized': '2024-09-10T15:59:15.769042+0000', 'mapping_epoch': 42, 'log_start': "45'11000", 'ondisk_log_start': "45'11000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3537", 'last_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'objects_scrubbed': 3497, 'log_size': 10082, 'log_dups_size': 0, 'ondisk_log_size': 10082, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1224, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4237, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 17581, 'num_write_kb': 28172, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4235}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:09:25.305 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-10T16:09:25.305 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "49'20457", 'reported_seq': 24787, 'reported_epoch': 51, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:09:21.156341+0000', 'last_change': '2024-09-10T16:09:07.396124+0000', 'last_active': '2024-09-10T16:09:21.156341+0000', 'last_peered': '2024-09-10T16:09:21.156341+0000', 'last_clean': '2024-09-10T15:59:14.088187+0000', 'last_became_active': '2024-09-10T15:59:16.165810+0000', 'last_became_peered': '2024-09-10T15:59:16.165810+0000', 'last_unstale': '2024-09-10T16:09:21.156341+0000', 'last_undegraded': '2024-09-10T15:59:15.822098+0000', 'last_fullsized': '2024-09-10T15:59:15.769123+0000', 'mapping_epoch': 42, 'log_start': "45'10400", 'ondisk_log_start': "45'10400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2218", 'last_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'objects_scrubbed': 2175, 'log_size': 10057, 'log_dups_size': 0, 'ondisk_log_size': 10057, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 825, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2037, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 18276, 'num_write_kb': 32204, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2088, 'num_bytes_recovered': 4579, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -2036}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:09:25.305 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-10T16:09:25.305 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "49'20781", 'reported_seq': 20931, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:09:08.852538+0000', 'last_change': '2024-09-10T15:59:16.284311+0000', 'last_active': '2024-09-10T16:09:08.852538+0000', 'last_peered': '2024-09-10T16:09:08.852538+0000', 'last_clean': '2024-09-10T15:59:14.116041+0000', 'last_became_active': '2024-09-10T15:59:16.274543+0000', 'last_became_peered': '2024-09-10T15:59:16.274543+0000', 'last_unstale': '2024-09-10T16:09:08.852538+0000', 'last_undegraded': '2024-09-10T15:59:15.812484+0000', 'last_fullsized': '2024-09-10T15:59:15.770885+0000', 'mapping_epoch': 42, 'log_start': "45'10700", 'ondisk_log_start': "45'10700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2356", 'last_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'objects_scrubbed': 2316, 'log_size': 10081, 'log_dups_size': 0, 'ondisk_log_size': 10081, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 783, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 18461, 'num_write_kb': 32292, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': -4303}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:09:25.306 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-10T16:09:25.306 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "49'21170", 'reported_seq': 30029, 'reported_epoch': 51, 'state': 'active+recovery_wait', 'last_fresh': '2024-09-10T16:09:08.852233+0000', 'last_change': '2024-09-10T16:08:10.929169+0000', 'last_active': '2024-09-10T16:09:08.852233+0000', 'last_peered': '2024-09-10T16:09:08.852233+0000', 'last_clean': '2024-09-10T15:59:14.110654+0000', 'last_became_active': '2024-09-10T16:08:10.860438+0000', 'last_became_peered': '2024-09-10T16:08:10.860438+0000', 'last_unstale': '2024-09-10T16:09:08.852233+0000', 'last_undegraded': '2024-09-10T16:09:08.852233+0000', 'last_fullsized': '2024-09-10T16:09:08.852233+0000', 'mapping_epoch': 48, 'log_start': "45'11100", 'ondisk_log_start': "45'11100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2486", 'last_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'objects_scrubbed': 2440, 'log_size': 10070, 'log_dups_size': 0, 'ondisk_log_size': 10070, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 792, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18724, 'num_write_kb': 32568, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 4355, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': ['2', '0', '1'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:09:25.306 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-10T16:09:25.306 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "49'20838", 'reported_seq': 20988, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:09:08.849672+0000', 'last_change': '2024-09-10T15:59:16.315641+0000', 'last_active': '2024-09-10T16:09:08.849672+0000', 'last_peered': '2024-09-10T16:09:08.849672+0000', 'last_clean': '2024-09-10T15:59:14.117001+0000', 'last_became_active': '2024-09-10T15:59:16.302245+0000', 'last_became_peered': '2024-09-10T15:59:16.302245+0000', 'last_unstale': '2024-09-10T16:09:08.849672+0000', 'last_undegraded': '2024-09-10T15:59:15.845012+0000', 'last_fullsized': '2024-09-10T15:59:15.770337+0000', 'mapping_epoch': 42, 'log_start': "45'10800", 'ondisk_log_start': "45'10800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3404", 'last_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'objects_scrubbed': 3367, 'log_size': 10038, 'log_dups_size': 0, 'ondisk_log_size': 10038, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 983, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17469, 'num_write_kb': 28204, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4304}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:09:25.306 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-10T16:09:25.306 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "49'21048", 'reported_seq': 21205, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:09:08.848232+0000', 'last_change': '2024-09-10T15:59:16.311291+0000', 'last_active': '2024-09-10T16:09:08.848232+0000', 'last_peered': '2024-09-10T16:09:08.848232+0000', 'last_clean': '2024-09-10T15:59:14.116363+0000', 'last_became_active': '2024-09-10T15:59:16.301244+0000', 'last_became_peered': '2024-09-10T15:59:16.301244+0000', 'last_unstale': '2024-09-10T16:09:08.848232+0000', 'last_undegraded': '2024-09-10T15:59:15.883434+0000', 'last_fullsized': '2024-09-10T15:59:15.772756+0000', 'mapping_epoch': 42, 'log_start': "45'11000", 'ondisk_log_start': "45'11000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3789", 'last_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'objects_scrubbed': 3741, 'log_size': 10048, 'log_dups_size': 0, 'ondisk_log_size': 10048, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1440, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4373, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17303, 'num_write_kb': 27124, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4371}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:09:25.306 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-10T16:09:25.306 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "49'21198", 'reported_seq': 21357, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:09:08.848736+0000', 'last_change': '2024-09-10T15:59:16.313409+0000', 'last_active': '2024-09-10T16:09:08.848736+0000', 'last_peered': '2024-09-10T16:09:08.848736+0000', 'last_clean': '2024-09-10T15:59:14.129780+0000', 'last_became_active': '2024-09-10T15:59:16.300932+0000', 'last_became_peered': '2024-09-10T15:59:16.300932+0000', 'last_unstale': '2024-09-10T16:09:08.848736+0000', 'last_undegraded': '2024-09-10T15:59:15.887823+0000', 'last_fullsized': '2024-09-10T15:59:15.768513+0000', 'mapping_epoch': 42, 'log_start': "45'11100", 'ondisk_log_start': "45'11100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3880", 'last_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'objects_scrubbed': 3827, 'log_size': 10098, 'log_dups_size': 0, 'ondisk_log_size': 10098, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1397, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4325, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17361, 'num_write_kb': 27072, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4321}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:09:25.306 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T16:09:25.687 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:09:25.687 INFO:teuthology.orchestra.run.smithi115.stdout:{"fsid":"7effee29-df05-4351-9741-ee203564ed81","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 32213/24 objects degraded (134220.833%), 8 pgs degraded, 8 pgs undersized","count":16},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":729,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":51,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983954,"num_in_osds":3,"osd_in_since":1725983947,"num_remapped_pgs":8},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":15},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":7},{"state_name":"active+recovery_wait","count":1},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":12,"data_bytes":4849,"bytes_used":1560141824,"bytes_avail":288350150656,"bytes_total":289910292480,"degraded_objects":32213,"degraded_total":24,"degraded_ratio":1342.208333333333,"recovering_objects_per_sec":122,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1470,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-10T15:57:14:967120+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T16:00:13.560776+0000","services":{}},"progress_events":{}} 2024-09-10T16:09:25.701 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-10T16:09:25.702 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:09:26.031 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:09:26.032 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:09:26.044 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":409,"stamp":"2024-09-10T16:09:25.772973+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32213,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292098,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19179,"num_bytes_recovered":33096182,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161025,"ondisk_log_size":161025,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1511096,"kb_used_data":206840,"kb_used_omap":12,"kb_used_meta":1304179,"kb_avail":281604424,"statfs":{"total":289910292480,"available":288362930176,"internally_reserved":0,"allocated":211804160,"data_stored":210822562,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1335479298},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1470,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1470,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002147"},"pg_stats":[{"pgid":"2.f","version":"49'20672","reported_seq":20762,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:08.851657+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:09:08.851657+0000","last_peered":"2024-09-10T16:09:08.851657+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:09:08.851657+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20672,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4259}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20844","reported_seq":20971,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.853495+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:09:08.853495+0000","last_peered":"2024-09-10T16:09:08.853495+0000","last_clean":"2024-09-10T16:09:08.853495+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:09:08.853495+0000","last_undegraded":"2024-09-10T16:09:08.853495+0000","last_fullsized":"2024-09-10T16:09:08.853495+0000","mapping_epoch":35,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18949,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20798","reported_seq":29491,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.852143+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:09:08.852143+0000","last_peered":"2024-09-10T16:09:08.852143+0000","last_clean":"2024-09-10T16:09:08.852143+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:09:08.852143+0000","last_undegraded":"2024-09-10T16:09:08.852143+0000","last_fullsized":"2024-09-10T16:09:08.852143+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18802,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20660","reported_seq":20755,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.853148+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:09:08.853148+0000","last_peered":"2024-09-10T16:09:08.853148+0000","last_clean":"2024-09-10T16:09:08.853148+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:09:08.853148+0000","last_undegraded":"2024-09-10T16:09:08.853148+0000","last_fullsized":"2024-09-10T16:09:08.853148+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20660,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20891","reported_seq":21036,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.847836+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:09:08.847836+0000","last_peered":"2024-09-10T16:09:08.847836+0000","last_clean":"2024-09-10T16:09:08.847836+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:09:08.847836+0000","last_undegraded":"2024-09-10T16:09:08.847836+0000","last_fullsized":"2024-09-10T16:09:08.847836+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17602,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'21061","reported_seq":21214,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:08.849450+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:09:08.849450+0000","last_peered":"2024-09-10T16:09:08.849450+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:09:08.849450+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17673,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4370},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'21082","reported_seq":21235,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:08.848683+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:09:08.848683+0000","last_peered":"2024-09-10T16:09:08.848683+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:09:08.848683+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17581,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4235},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'21314","reported_seq":21459,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.847784+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:09:08.847784+0000","last_peered":"2024-09-10T16:09:08.847784+0000","last_clean":"2024-09-10T16:09:08.847784+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:09:08.847784+0000","last_undegraded":"2024-09-10T16:09:08.847784+0000","last_fullsized":"2024-09-10T16:09:08.847784+0000","mapping_epoch":34,"log_start":"45'11300","ondisk_log_start":"45'11300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18161,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":85,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.851620+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:09:08.851620+0000","last_peered":"2024-09-10T16:09:08.851620+0000","last_clean":"2024-09-10T16:09:08.851620+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:09:08.851620+0000","last_undegraded":"2024-09-10T16:09:08.851620+0000","last_fullsized":"2024-09-10T16:09:08.851620+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20457","reported_seq":24787,"reported_epoch":51,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:21.156341+0000","last_change":"2024-09-10T16:09:07.396124+0000","last_active":"2024-09-10T16:09:21.156341+0000","last_peered":"2024-09-10T16:09:21.156341+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:09:21.156341+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2037,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18276,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2088,"num_bytes_recovered":4579,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2036}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":109,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.852443+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:09:08.852443+0000","last_peered":"2024-09-10T16:09:08.852443+0000","last_clean":"2024-09-10T16:09:08.852443+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:09:08.852443+0000","last_undegraded":"2024-09-10T16:09:08.852443+0000","last_fullsized":"2024-09-10T16:09:08.852443+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20910","reported_seq":29577,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.909669+0000","last_change":"2024-09-10T16:09:08.909580+0000","last_active":"2024-09-10T16:09:08.909669+0000","last_peered":"2024-09-10T16:09:08.909669+0000","last_clean":"2024-09-10T16:09:08.909669+0000","last_became_active":"2024-09-10T16:09:08.854941+0000","last_became_peered":"2024-09-10T16:09:08.854941+0000","last_unstale":"2024-09-10T16:09:08.909669+0000","last_undegraded":"2024-09-10T16:09:08.909669+0000","last_fullsized":"2024-09-10T16:09:08.909669+0000","mapping_epoch":50,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:13:14.846691+0000","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17414,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4257,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":119,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.847624+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:09:08.847624+0000","last_peered":"2024-09-10T16:09:08.847624+0000","last_clean":"2024-09-10T16:09:08.847624+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:09:08.847624+0000","last_undegraded":"2024-09-10T16:09:08.847624+0000","last_fullsized":"2024-09-10T16:09:08.847624+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20781","reported_seq":20931,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:08.852538+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:09:08.852538+0000","last_peered":"2024-09-10T16:09:08.852538+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:09:08.852538+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18461,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-4303}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":94,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.852182+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:09:08.852182+0000","last_peered":"2024-09-10T16:09:08.852182+0000","last_clean":"2024-09-10T16:09:08.852182+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:09:08.852182+0000","last_undegraded":"2024-09-10T16:09:08.852182+0000","last_fullsized":"2024-09-10T16:09:08.852182+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'21170","reported_seq":30029,"reported_epoch":51,"state":"active+recovery_wait","last_fresh":"2024-09-10T16:09:08.852233+0000","last_change":"2024-09-10T16:08:10.929169+0000","last_active":"2024-09-10T16:09:08.852233+0000","last_peered":"2024-09-10T16:09:08.852233+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T16:08:10.860438+0000","last_became_peered":"2024-09-10T16:08:10.860438+0000","last_unstale":"2024-09-10T16:09:08.852233+0000","last_undegraded":"2024-09-10T16:09:08.852233+0000","last_fullsized":"2024-09-10T16:09:08.852233+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18724,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","0","1"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20838","reported_seq":20988,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:08.849672+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:09:08.849672+0000","last_peered":"2024-09-10T16:09:08.849672+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:09:08.849672+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17469,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.851774+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:09:08.851774+0000","last_peered":"2024-09-10T16:09:08.851774+0000","last_clean":"2024-09-10T16:09:08.851774+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:09:08.851774+0000","last_undegraded":"2024-09-10T16:09:08.851774+0000","last_fullsized":"2024-09-10T16:09:08.851774+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20598","reported_seq":29178,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.851713+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:09:08.851713+0000","last_peered":"2024-09-10T16:09:08.851713+0000","last_clean":"2024-09-10T16:09:08.851713+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:09:08.851713+0000","last_undegraded":"2024-09-10T16:09:08.851713+0000","last_fullsized":"2024-09-10T16:09:08.851713+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16988,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":109,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.853357+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:09:08.853357+0000","last_peered":"2024-09-10T16:09:08.853357+0000","last_clean":"2024-09-10T16:09:08.853357+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:09:08.853357+0000","last_undegraded":"2024-09-10T16:09:08.853357+0000","last_fullsized":"2024-09-10T16:09:08.853357+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'21048","reported_seq":21205,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:08.848232+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:09:08.848232+0000","last_peered":"2024-09-10T16:09:08.848232+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:09:08.848232+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17303,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.851634+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:09:08.851634+0000","last_peered":"2024-09-10T16:09:08.851634+0000","last_clean":"2024-09-10T16:09:08.851634+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:09:08.851634+0000","last_undegraded":"2024-09-10T16:09:08.851634+0000","last_fullsized":"2024-09-10T16:09:08.851634+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":89,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.851698+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:09:08.851698+0000","last_peered":"2024-09-10T16:09:08.851698+0000","last_clean":"2024-09-10T16:09:08.851698+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:09:08.851698+0000","last_undegraded":"2024-09-10T16:09:08.851698+0000","last_fullsized":"2024-09-10T16:09:08.851698+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'21198","reported_seq":21357,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:08.848736+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:09:08.848736+0000","last_peered":"2024-09-10T16:09:08.848736+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:09:08.848736+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17361,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4321},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32213,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292096,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19177,"num_bytes_recovered":33096163,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":210518016,"data_stored":210395174,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161022,"ondisk_log_size":161022,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986312,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":502616,"kb_used_data":137728,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93869224,"statfs":{"total":96636764160,"available":96122085376,"internally_reserved":0,"allocated":141033472,"data_stored":140713808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659258,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":461164,"kb_used_data":476,"kb_used_omap":4,"kb_used_meta":460667,"kb_avail":93910676,"statfs":{"total":96636764160,"available":96164532224,"internally_reserved":0,"allocated":487424,"data_stored":149231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":471723694},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182540,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":547316,"kb_used_data":68636,"kb_used_omap":4,"kb_used_meta":478651,"kb_avail":93824524,"statfs":{"total":96636764160,"available":96076312576,"internally_reserved":0,"allocated":70283264,"data_stored":69959523,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":490139306},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":69853184,"data_stored":69817054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":6762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":140607488,"data_stored":140571358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:09:29.047 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:09:29.350 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:09:29.350 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:09:29.362 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":410,"stamp":"2024-09-10T16:09:27.773391+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31418,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292098,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19974,"num_bytes_recovered":33096182,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161025,"ondisk_log_size":161025,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1511096,"kb_used_data":206840,"kb_used_omap":12,"kb_used_meta":1304179,"kb_avail":281604424,"statfs":{"total":289910292480,"available":288362930176,"internally_reserved":0,"allocated":211804160,"data_stored":210822562,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1335479298},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2265,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2265,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002198"},"pg_stats":[{"pgid":"2.f","version":"49'20672","reported_seq":20762,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:08.851657+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:09:08.851657+0000","last_peered":"2024-09-10T16:09:08.851657+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:09:08.851657+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20672,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4259}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20844","reported_seq":20971,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.853495+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:09:08.853495+0000","last_peered":"2024-09-10T16:09:08.853495+0000","last_clean":"2024-09-10T16:09:08.853495+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:09:08.853495+0000","last_undegraded":"2024-09-10T16:09:08.853495+0000","last_fullsized":"2024-09-10T16:09:08.853495+0000","mapping_epoch":35,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18949,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20798","reported_seq":29491,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.852143+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:09:08.852143+0000","last_peered":"2024-09-10T16:09:08.852143+0000","last_clean":"2024-09-10T16:09:08.852143+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:09:08.852143+0000","last_undegraded":"2024-09-10T16:09:08.852143+0000","last_fullsized":"2024-09-10T16:09:08.852143+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18802,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20660","reported_seq":20755,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.853148+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:09:08.853148+0000","last_peered":"2024-09-10T16:09:08.853148+0000","last_clean":"2024-09-10T16:09:08.853148+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:09:08.853148+0000","last_undegraded":"2024-09-10T16:09:08.853148+0000","last_fullsized":"2024-09-10T16:09:08.853148+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20660,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20891","reported_seq":21036,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.847836+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:09:08.847836+0000","last_peered":"2024-09-10T16:09:08.847836+0000","last_clean":"2024-09-10T16:09:08.847836+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:09:08.847836+0000","last_undegraded":"2024-09-10T16:09:08.847836+0000","last_fullsized":"2024-09-10T16:09:08.847836+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17602,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'21061","reported_seq":21214,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:08.849450+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:09:08.849450+0000","last_peered":"2024-09-10T16:09:08.849450+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:09:08.849450+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17673,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4370},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'21082","reported_seq":21235,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:08.848683+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:09:08.848683+0000","last_peered":"2024-09-10T16:09:08.848683+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:09:08.848683+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17581,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4235},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'21314","reported_seq":21459,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.847784+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:09:08.847784+0000","last_peered":"2024-09-10T16:09:08.847784+0000","last_clean":"2024-09-10T16:09:08.847784+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:09:08.847784+0000","last_undegraded":"2024-09-10T16:09:08.847784+0000","last_fullsized":"2024-09-10T16:09:08.847784+0000","mapping_epoch":34,"log_start":"45'11300","ondisk_log_start":"45'11300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18161,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":85,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.851620+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:09:08.851620+0000","last_peered":"2024-09-10T16:09:08.851620+0000","last_clean":"2024-09-10T16:09:08.851620+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:09:08.851620+0000","last_undegraded":"2024-09-10T16:09:08.851620+0000","last_fullsized":"2024-09-10T16:09:08.851620+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20457","reported_seq":26377,"reported_epoch":51,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:26.157742+0000","last_change":"2024-09-10T16:09:07.396124+0000","last_active":"2024-09-10T16:09:26.157742+0000","last_peered":"2024-09-10T16:09:26.157742+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:09:26.157742+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1242,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18276,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2883,"num_bytes_recovered":4579,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1241}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":109,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.852443+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:09:08.852443+0000","last_peered":"2024-09-10T16:09:08.852443+0000","last_clean":"2024-09-10T16:09:08.852443+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:09:08.852443+0000","last_undegraded":"2024-09-10T16:09:08.852443+0000","last_fullsized":"2024-09-10T16:09:08.852443+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20910","reported_seq":29577,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.909669+0000","last_change":"2024-09-10T16:09:08.909580+0000","last_active":"2024-09-10T16:09:08.909669+0000","last_peered":"2024-09-10T16:09:08.909669+0000","last_clean":"2024-09-10T16:09:08.909669+0000","last_became_active":"2024-09-10T16:09:08.854941+0000","last_became_peered":"2024-09-10T16:09:08.854941+0000","last_unstale":"2024-09-10T16:09:08.909669+0000","last_undegraded":"2024-09-10T16:09:08.909669+0000","last_fullsized":"2024-09-10T16:09:08.909669+0000","mapping_epoch":50,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:13:14.846691+0000","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17414,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4257,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":119,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.847624+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:09:08.847624+0000","last_peered":"2024-09-10T16:09:08.847624+0000","last_clean":"2024-09-10T16:09:08.847624+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:09:08.847624+0000","last_undegraded":"2024-09-10T16:09:08.847624+0000","last_fullsized":"2024-09-10T16:09:08.847624+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20781","reported_seq":20931,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:08.852538+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:09:08.852538+0000","last_peered":"2024-09-10T16:09:08.852538+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:09:08.852538+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18461,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-4303}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":94,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.852182+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:09:08.852182+0000","last_peered":"2024-09-10T16:09:08.852182+0000","last_clean":"2024-09-10T16:09:08.852182+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:09:08.852182+0000","last_undegraded":"2024-09-10T16:09:08.852182+0000","last_fullsized":"2024-09-10T16:09:08.852182+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'21170","reported_seq":30029,"reported_epoch":51,"state":"active+recovery_wait","last_fresh":"2024-09-10T16:09:08.852233+0000","last_change":"2024-09-10T16:08:10.929169+0000","last_active":"2024-09-10T16:09:08.852233+0000","last_peered":"2024-09-10T16:09:08.852233+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T16:08:10.860438+0000","last_became_peered":"2024-09-10T16:08:10.860438+0000","last_unstale":"2024-09-10T16:09:08.852233+0000","last_undegraded":"2024-09-10T16:09:08.852233+0000","last_fullsized":"2024-09-10T16:09:08.852233+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18724,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","0","1"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20838","reported_seq":20988,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:08.849672+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:09:08.849672+0000","last_peered":"2024-09-10T16:09:08.849672+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:09:08.849672+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17469,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.851774+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:09:08.851774+0000","last_peered":"2024-09-10T16:09:08.851774+0000","last_clean":"2024-09-10T16:09:08.851774+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:09:08.851774+0000","last_undegraded":"2024-09-10T16:09:08.851774+0000","last_fullsized":"2024-09-10T16:09:08.851774+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20598","reported_seq":29178,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.851713+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:09:08.851713+0000","last_peered":"2024-09-10T16:09:08.851713+0000","last_clean":"2024-09-10T16:09:08.851713+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:09:08.851713+0000","last_undegraded":"2024-09-10T16:09:08.851713+0000","last_fullsized":"2024-09-10T16:09:08.851713+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16988,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":109,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.853357+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:09:08.853357+0000","last_peered":"2024-09-10T16:09:08.853357+0000","last_clean":"2024-09-10T16:09:08.853357+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:09:08.853357+0000","last_undegraded":"2024-09-10T16:09:08.853357+0000","last_fullsized":"2024-09-10T16:09:08.853357+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'21048","reported_seq":21205,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:08.848232+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:09:08.848232+0000","last_peered":"2024-09-10T16:09:08.848232+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:09:08.848232+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17303,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.851634+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:09:08.851634+0000","last_peered":"2024-09-10T16:09:08.851634+0000","last_clean":"2024-09-10T16:09:08.851634+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:09:08.851634+0000","last_undegraded":"2024-09-10T16:09:08.851634+0000","last_fullsized":"2024-09-10T16:09:08.851634+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":89,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.851698+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:09:08.851698+0000","last_peered":"2024-09-10T16:09:08.851698+0000","last_clean":"2024-09-10T16:09:08.851698+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:09:08.851698+0000","last_undegraded":"2024-09-10T16:09:08.851698+0000","last_fullsized":"2024-09-10T16:09:08.851698+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'21198","reported_seq":21357,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:08.848736+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:09:08.848736+0000","last_peered":"2024-09-10T16:09:08.848736+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:09:08.848736+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17361,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4321},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31418,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292096,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19972,"num_bytes_recovered":33096163,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":210518016,"data_stored":210395174,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161022,"ondisk_log_size":161022,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986313,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":502616,"kb_used_data":137728,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93869224,"statfs":{"total":96636764160,"available":96122085376,"internally_reserved":0,"allocated":141033472,"data_stored":140713808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659258,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":461164,"kb_used_data":476,"kb_used_omap":4,"kb_used_meta":460667,"kb_avail":93910676,"statfs":{"total":96636764160,"available":96164532224,"internally_reserved":0,"allocated":487424,"data_stored":149231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":471723694},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182541,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":547316,"kb_used_data":68636,"kb_used_omap":4,"kb_used_meta":478651,"kb_avail":93824524,"statfs":{"total":96636764160,"available":96076312576,"internally_reserved":0,"allocated":70283264,"data_stored":69959523,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":490139306},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":69853184,"data_stored":69817054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":6762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":140607488,"data_stored":140571358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:09:29.363 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:09:29.666 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:09:29.666 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:09:29.679 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":410,"stamp":"2024-09-10T16:09:27.773391+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31418,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292098,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19974,"num_bytes_recovered":33096182,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161025,"ondisk_log_size":161025,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1511096,"kb_used_data":206840,"kb_used_omap":12,"kb_used_meta":1304179,"kb_avail":281604424,"statfs":{"total":289910292480,"available":288362930176,"internally_reserved":0,"allocated":211804160,"data_stored":210822562,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1335479298},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2265,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2265,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002198"},"pg_stats":[{"pgid":"2.f","version":"49'20672","reported_seq":20762,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:08.851657+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:09:08.851657+0000","last_peered":"2024-09-10T16:09:08.851657+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:09:08.851657+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20672,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4259}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20844","reported_seq":20971,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.853495+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:09:08.853495+0000","last_peered":"2024-09-10T16:09:08.853495+0000","last_clean":"2024-09-10T16:09:08.853495+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:09:08.853495+0000","last_undegraded":"2024-09-10T16:09:08.853495+0000","last_fullsized":"2024-09-10T16:09:08.853495+0000","mapping_epoch":35,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18949,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20798","reported_seq":29491,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.852143+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:09:08.852143+0000","last_peered":"2024-09-10T16:09:08.852143+0000","last_clean":"2024-09-10T16:09:08.852143+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:09:08.852143+0000","last_undegraded":"2024-09-10T16:09:08.852143+0000","last_fullsized":"2024-09-10T16:09:08.852143+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18802,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20660","reported_seq":20755,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.853148+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:09:08.853148+0000","last_peered":"2024-09-10T16:09:08.853148+0000","last_clean":"2024-09-10T16:09:08.853148+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:09:08.853148+0000","last_undegraded":"2024-09-10T16:09:08.853148+0000","last_fullsized":"2024-09-10T16:09:08.853148+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20660,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20891","reported_seq":21036,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.847836+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:09:08.847836+0000","last_peered":"2024-09-10T16:09:08.847836+0000","last_clean":"2024-09-10T16:09:08.847836+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:09:08.847836+0000","last_undegraded":"2024-09-10T16:09:08.847836+0000","last_fullsized":"2024-09-10T16:09:08.847836+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17602,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'21061","reported_seq":21214,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:08.849450+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:09:08.849450+0000","last_peered":"2024-09-10T16:09:08.849450+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:09:08.849450+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17673,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4370},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'21082","reported_seq":21235,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:08.848683+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:09:08.848683+0000","last_peered":"2024-09-10T16:09:08.848683+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:09:08.848683+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17581,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4235},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'21314","reported_seq":21459,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.847784+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:09:08.847784+0000","last_peered":"2024-09-10T16:09:08.847784+0000","last_clean":"2024-09-10T16:09:08.847784+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:09:08.847784+0000","last_undegraded":"2024-09-10T16:09:08.847784+0000","last_fullsized":"2024-09-10T16:09:08.847784+0000","mapping_epoch":34,"log_start":"45'11300","ondisk_log_start":"45'11300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18161,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":85,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.851620+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:09:08.851620+0000","last_peered":"2024-09-10T16:09:08.851620+0000","last_clean":"2024-09-10T16:09:08.851620+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:09:08.851620+0000","last_undegraded":"2024-09-10T16:09:08.851620+0000","last_fullsized":"2024-09-10T16:09:08.851620+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20457","reported_seq":26377,"reported_epoch":51,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:26.157742+0000","last_change":"2024-09-10T16:09:07.396124+0000","last_active":"2024-09-10T16:09:26.157742+0000","last_peered":"2024-09-10T16:09:26.157742+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:09:26.157742+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1242,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18276,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2883,"num_bytes_recovered":4579,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1241}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":109,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.852443+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:09:08.852443+0000","last_peered":"2024-09-10T16:09:08.852443+0000","last_clean":"2024-09-10T16:09:08.852443+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:09:08.852443+0000","last_undegraded":"2024-09-10T16:09:08.852443+0000","last_fullsized":"2024-09-10T16:09:08.852443+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20910","reported_seq":29577,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.909669+0000","last_change":"2024-09-10T16:09:08.909580+0000","last_active":"2024-09-10T16:09:08.909669+0000","last_peered":"2024-09-10T16:09:08.909669+0000","last_clean":"2024-09-10T16:09:08.909669+0000","last_became_active":"2024-09-10T16:09:08.854941+0000","last_became_peered":"2024-09-10T16:09:08.854941+0000","last_unstale":"2024-09-10T16:09:08.909669+0000","last_undegraded":"2024-09-10T16:09:08.909669+0000","last_fullsized":"2024-09-10T16:09:08.909669+0000","mapping_epoch":50,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:13:14.846691+0000","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17414,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4257,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":119,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.847624+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:09:08.847624+0000","last_peered":"2024-09-10T16:09:08.847624+0000","last_clean":"2024-09-10T16:09:08.847624+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:09:08.847624+0000","last_undegraded":"2024-09-10T16:09:08.847624+0000","last_fullsized":"2024-09-10T16:09:08.847624+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20781","reported_seq":20931,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:08.852538+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:09:08.852538+0000","last_peered":"2024-09-10T16:09:08.852538+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:09:08.852538+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18461,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-4303}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":94,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.852182+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:09:08.852182+0000","last_peered":"2024-09-10T16:09:08.852182+0000","last_clean":"2024-09-10T16:09:08.852182+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:09:08.852182+0000","last_undegraded":"2024-09-10T16:09:08.852182+0000","last_fullsized":"2024-09-10T16:09:08.852182+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'21170","reported_seq":30029,"reported_epoch":51,"state":"active+recovery_wait","last_fresh":"2024-09-10T16:09:08.852233+0000","last_change":"2024-09-10T16:08:10.929169+0000","last_active":"2024-09-10T16:09:08.852233+0000","last_peered":"2024-09-10T16:09:08.852233+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T16:08:10.860438+0000","last_became_peered":"2024-09-10T16:08:10.860438+0000","last_unstale":"2024-09-10T16:09:08.852233+0000","last_undegraded":"2024-09-10T16:09:08.852233+0000","last_fullsized":"2024-09-10T16:09:08.852233+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18724,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","0","1"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20838","reported_seq":20988,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:08.849672+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:09:08.849672+0000","last_peered":"2024-09-10T16:09:08.849672+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:09:08.849672+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17469,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.851774+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:09:08.851774+0000","last_peered":"2024-09-10T16:09:08.851774+0000","last_clean":"2024-09-10T16:09:08.851774+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:09:08.851774+0000","last_undegraded":"2024-09-10T16:09:08.851774+0000","last_fullsized":"2024-09-10T16:09:08.851774+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20598","reported_seq":29178,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.851713+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:09:08.851713+0000","last_peered":"2024-09-10T16:09:08.851713+0000","last_clean":"2024-09-10T16:09:08.851713+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:09:08.851713+0000","last_undegraded":"2024-09-10T16:09:08.851713+0000","last_fullsized":"2024-09-10T16:09:08.851713+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16988,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":109,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.853357+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:09:08.853357+0000","last_peered":"2024-09-10T16:09:08.853357+0000","last_clean":"2024-09-10T16:09:08.853357+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:09:08.853357+0000","last_undegraded":"2024-09-10T16:09:08.853357+0000","last_fullsized":"2024-09-10T16:09:08.853357+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'21048","reported_seq":21205,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:08.848232+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:09:08.848232+0000","last_peered":"2024-09-10T16:09:08.848232+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:09:08.848232+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17303,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.851634+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:09:08.851634+0000","last_peered":"2024-09-10T16:09:08.851634+0000","last_clean":"2024-09-10T16:09:08.851634+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:09:08.851634+0000","last_undegraded":"2024-09-10T16:09:08.851634+0000","last_fullsized":"2024-09-10T16:09:08.851634+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":89,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.851698+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:09:08.851698+0000","last_peered":"2024-09-10T16:09:08.851698+0000","last_clean":"2024-09-10T16:09:08.851698+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:09:08.851698+0000","last_undegraded":"2024-09-10T16:09:08.851698+0000","last_fullsized":"2024-09-10T16:09:08.851698+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'21198","reported_seq":21357,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:08.848736+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:09:08.848736+0000","last_peered":"2024-09-10T16:09:08.848736+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:09:08.848736+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17361,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4321},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31418,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292096,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19972,"num_bytes_recovered":33096163,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":210518016,"data_stored":210395174,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161022,"ondisk_log_size":161022,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986313,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":502616,"kb_used_data":137728,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93869224,"statfs":{"total":96636764160,"available":96122085376,"internally_reserved":0,"allocated":141033472,"data_stored":140713808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659258,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":461164,"kb_used_data":476,"kb_used_omap":4,"kb_used_meta":460667,"kb_avail":93910676,"statfs":{"total":96636764160,"available":96164532224,"internally_reserved":0,"allocated":487424,"data_stored":149231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":471723694},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182541,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":547316,"kb_used_data":68636,"kb_used_omap":4,"kb_used_meta":478651,"kb_avail":93824524,"statfs":{"total":96636764160,"available":96076312576,"internally_reserved":0,"allocated":70283264,"data_stored":69959523,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":490139306},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":69853184,"data_stored":69817054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":6762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":140607488,"data_stored":140571358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:09:29.680 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-10T16:09:29.680 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "49'20672", 'reported_seq': 20762, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:09:08.851657+0000', 'last_change': '2024-09-10T15:59:16.281558+0000', 'last_active': '2024-09-10T16:09:08.851657+0000', 'last_peered': '2024-09-10T16:09:08.851657+0000', 'last_clean': '2024-09-10T15:59:14.132456+0000', 'last_became_active': '2024-09-10T15:59:16.274407+0000', 'last_became_peered': '2024-09-10T15:59:16.274407+0000', 'last_unstale': '2024-09-10T16:09:08.851657+0000', 'last_undegraded': '2024-09-10T15:59:15.851279+0000', 'last_fullsized': '2024-09-10T15:59:15.774356+0000', 'mapping_epoch': 42, 'log_start': "45'10600", 'ondisk_log_start': "45'10600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'objects_scrubbed': 0, 'log_size': 10072, 'log_dups_size': 0, 'ondisk_log_size': 10072, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4259, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20672, 'num_write_kb': 41344, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4259}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:09:29.681 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-10T16:09:29.681 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "49'21061", 'reported_seq': 21214, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:09:08.849450+0000', 'last_change': '2024-09-10T15:59:16.308264+0000', 'last_active': '2024-09-10T16:09:08.849450+0000', 'last_peered': '2024-09-10T16:09:08.849450+0000', 'last_clean': '2024-09-10T15:59:14.111551+0000', 'last_became_active': '2024-09-10T15:59:16.300040+0000', 'last_became_peered': '2024-09-10T15:59:16.300040+0000', 'last_unstale': '2024-09-10T16:09:08.849450+0000', 'last_undegraded': '2024-09-10T15:59:15.878234+0000', 'last_fullsized': '2024-09-10T15:59:15.769328+0000', 'mapping_epoch': 42, 'log_start': "45'11000", 'ondisk_log_start': "45'11000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3442", 'last_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'objects_scrubbed': 3384, 'log_size': 10061, 'log_dups_size': 0, 'ondisk_log_size': 10061, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1292, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4372, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 17673, 'num_write_kb': 28580, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4370}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:09:29.681 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-10T16:09:29.681 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "49'21082", 'reported_seq': 21235, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:09:08.848683+0000', 'last_change': '2024-09-10T15:59:16.331899+0000', 'last_active': '2024-09-10T16:09:08.848683+0000', 'last_peered': '2024-09-10T16:09:08.848683+0000', 'last_clean': '2024-09-10T15:59:14.139436+0000', 'last_became_active': '2024-09-10T15:59:16.318715+0000', 'last_became_peered': '2024-09-10T15:59:16.318715+0000', 'last_unstale': '2024-09-10T16:09:08.848683+0000', 'last_undegraded': '2024-09-10T15:59:15.877524+0000', 'last_fullsized': '2024-09-10T15:59:15.769042+0000', 'mapping_epoch': 42, 'log_start': "45'11000", 'ondisk_log_start': "45'11000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3537", 'last_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'objects_scrubbed': 3497, 'log_size': 10082, 'log_dups_size': 0, 'ondisk_log_size': 10082, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1224, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4237, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 17581, 'num_write_kb': 28172, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4235}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:09:29.681 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-10T16:09:29.681 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "49'20457", 'reported_seq': 26377, 'reported_epoch': 51, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:09:26.157742+0000', 'last_change': '2024-09-10T16:09:07.396124+0000', 'last_active': '2024-09-10T16:09:26.157742+0000', 'last_peered': '2024-09-10T16:09:26.157742+0000', 'last_clean': '2024-09-10T15:59:14.088187+0000', 'last_became_active': '2024-09-10T15:59:16.165810+0000', 'last_became_peered': '2024-09-10T15:59:16.165810+0000', 'last_unstale': '2024-09-10T16:09:26.157742+0000', 'last_undegraded': '2024-09-10T15:59:15.822098+0000', 'last_fullsized': '2024-09-10T15:59:15.769123+0000', 'mapping_epoch': 42, 'log_start': "45'10400", 'ondisk_log_start': "45'10400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2218", 'last_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'objects_scrubbed': 2175, 'log_size': 10057, 'log_dups_size': 0, 'ondisk_log_size': 10057, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 825, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1242, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 18276, 'num_write_kb': 32204, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2883, 'num_bytes_recovered': 4579, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -1241}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:09:29.682 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-10T16:09:29.682 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "49'20781", 'reported_seq': 20931, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:09:08.852538+0000', 'last_change': '2024-09-10T15:59:16.284311+0000', 'last_active': '2024-09-10T16:09:08.852538+0000', 'last_peered': '2024-09-10T16:09:08.852538+0000', 'last_clean': '2024-09-10T15:59:14.116041+0000', 'last_became_active': '2024-09-10T15:59:16.274543+0000', 'last_became_peered': '2024-09-10T15:59:16.274543+0000', 'last_unstale': '2024-09-10T16:09:08.852538+0000', 'last_undegraded': '2024-09-10T15:59:15.812484+0000', 'last_fullsized': '2024-09-10T15:59:15.770885+0000', 'mapping_epoch': 42, 'log_start': "45'10700", 'ondisk_log_start': "45'10700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2356", 'last_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'objects_scrubbed': 2316, 'log_size': 10081, 'log_dups_size': 0, 'ondisk_log_size': 10081, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 783, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 18461, 'num_write_kb': 32292, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': -4303}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:09:29.682 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-10T16:09:29.682 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "49'21170", 'reported_seq': 30029, 'reported_epoch': 51, 'state': 'active+recovery_wait', 'last_fresh': '2024-09-10T16:09:08.852233+0000', 'last_change': '2024-09-10T16:08:10.929169+0000', 'last_active': '2024-09-10T16:09:08.852233+0000', 'last_peered': '2024-09-10T16:09:08.852233+0000', 'last_clean': '2024-09-10T15:59:14.110654+0000', 'last_became_active': '2024-09-10T16:08:10.860438+0000', 'last_became_peered': '2024-09-10T16:08:10.860438+0000', 'last_unstale': '2024-09-10T16:09:08.852233+0000', 'last_undegraded': '2024-09-10T16:09:08.852233+0000', 'last_fullsized': '2024-09-10T16:09:08.852233+0000', 'mapping_epoch': 48, 'log_start': "45'11100", 'ondisk_log_start': "45'11100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2486", 'last_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'objects_scrubbed': 2440, 'log_size': 10070, 'log_dups_size': 0, 'ondisk_log_size': 10070, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 792, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18724, 'num_write_kb': 32568, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 4355, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': ['2', '0', '1'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:09:29.682 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-10T16:09:29.682 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "49'20838", 'reported_seq': 20988, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:09:08.849672+0000', 'last_change': '2024-09-10T15:59:16.315641+0000', 'last_active': '2024-09-10T16:09:08.849672+0000', 'last_peered': '2024-09-10T16:09:08.849672+0000', 'last_clean': '2024-09-10T15:59:14.117001+0000', 'last_became_active': '2024-09-10T15:59:16.302245+0000', 'last_became_peered': '2024-09-10T15:59:16.302245+0000', 'last_unstale': '2024-09-10T16:09:08.849672+0000', 'last_undegraded': '2024-09-10T15:59:15.845012+0000', 'last_fullsized': '2024-09-10T15:59:15.770337+0000', 'mapping_epoch': 42, 'log_start': "45'10800", 'ondisk_log_start': "45'10800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3404", 'last_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'objects_scrubbed': 3367, 'log_size': 10038, 'log_dups_size': 0, 'ondisk_log_size': 10038, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 983, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17469, 'num_write_kb': 28204, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4304}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:09:29.682 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-10T16:09:29.682 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "49'21048", 'reported_seq': 21205, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:09:08.848232+0000', 'last_change': '2024-09-10T15:59:16.311291+0000', 'last_active': '2024-09-10T16:09:08.848232+0000', 'last_peered': '2024-09-10T16:09:08.848232+0000', 'last_clean': '2024-09-10T15:59:14.116363+0000', 'last_became_active': '2024-09-10T15:59:16.301244+0000', 'last_became_peered': '2024-09-10T15:59:16.301244+0000', 'last_unstale': '2024-09-10T16:09:08.848232+0000', 'last_undegraded': '2024-09-10T15:59:15.883434+0000', 'last_fullsized': '2024-09-10T15:59:15.772756+0000', 'mapping_epoch': 42, 'log_start': "45'11000", 'ondisk_log_start': "45'11000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3789", 'last_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'objects_scrubbed': 3741, 'log_size': 10048, 'log_dups_size': 0, 'ondisk_log_size': 10048, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1440, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4373, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17303, 'num_write_kb': 27124, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4371}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:09:29.682 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-10T16:09:29.682 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "49'21198", 'reported_seq': 21357, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:09:08.848736+0000', 'last_change': '2024-09-10T15:59:16.313409+0000', 'last_active': '2024-09-10T16:09:08.848736+0000', 'last_peered': '2024-09-10T16:09:08.848736+0000', 'last_clean': '2024-09-10T15:59:14.129780+0000', 'last_became_active': '2024-09-10T15:59:16.300932+0000', 'last_became_peered': '2024-09-10T15:59:16.300932+0000', 'last_unstale': '2024-09-10T16:09:08.848736+0000', 'last_undegraded': '2024-09-10T15:59:15.887823+0000', 'last_fullsized': '2024-09-10T15:59:15.768513+0000', 'mapping_epoch': 42, 'log_start': "45'11100", 'ondisk_log_start': "45'11100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3880", 'last_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'objects_scrubbed': 3827, 'log_size': 10098, 'log_dups_size': 0, 'ondisk_log_size': 10098, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1397, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4325, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17361, 'num_write_kb': 27072, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4321}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:09:29.683 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T16:09:30.084 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:09:30.085 INFO:teuthology.orchestra.run.smithi115.stdout:{"fsid":"7effee29-df05-4351-9741-ee203564ed81","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 31418/24 objects degraded (130908.333%), 8 pgs degraded, 8 pgs undersized","count":16},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":734,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":51,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983954,"num_in_osds":3,"osd_in_since":1725983947,"num_remapped_pgs":8},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":15},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":7},{"state_name":"active+recovery_wait","count":1},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":12,"data_bytes":4849,"bytes_used":1547362304,"bytes_avail":288362930176,"bytes_total":289910292480,"degraded_objects":31418,"degraded_total":24,"degraded_ratio":1309.0833333333333,"recovering_objects_per_sec":121,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1464,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-10T15:57:14:967120+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T16:00:13.560776+0000","services":{}},"progress_events":{}} 2024-09-10T16:09:30.097 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-10T16:09:30.097 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:09:30.411 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:09:30.411 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:09:30.423 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":411,"stamp":"2024-09-10T16:09:29.773768+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31418,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292098,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19974,"num_bytes_recovered":33096182,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161025,"ondisk_log_size":161025,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1511096,"kb_used_data":206840,"kb_used_omap":12,"kb_used_meta":1304179,"kb_avail":281604424,"statfs":{"total":289910292480,"available":288362930176,"internally_reserved":0,"allocated":211804160,"data_stored":210822562,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1335479298},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1464,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1464,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002184"},"pg_stats":[{"pgid":"2.f","version":"49'20672","reported_seq":20762,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:08.851657+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:09:08.851657+0000","last_peered":"2024-09-10T16:09:08.851657+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:09:08.851657+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20672,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4259}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20844","reported_seq":20971,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.853495+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:09:08.853495+0000","last_peered":"2024-09-10T16:09:08.853495+0000","last_clean":"2024-09-10T16:09:08.853495+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:09:08.853495+0000","last_undegraded":"2024-09-10T16:09:08.853495+0000","last_fullsized":"2024-09-10T16:09:08.853495+0000","mapping_epoch":35,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18949,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20798","reported_seq":29491,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.852143+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:09:08.852143+0000","last_peered":"2024-09-10T16:09:08.852143+0000","last_clean":"2024-09-10T16:09:08.852143+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:09:08.852143+0000","last_undegraded":"2024-09-10T16:09:08.852143+0000","last_fullsized":"2024-09-10T16:09:08.852143+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18802,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20660","reported_seq":20755,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.853148+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:09:08.853148+0000","last_peered":"2024-09-10T16:09:08.853148+0000","last_clean":"2024-09-10T16:09:08.853148+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:09:08.853148+0000","last_undegraded":"2024-09-10T16:09:08.853148+0000","last_fullsized":"2024-09-10T16:09:08.853148+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20660,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20891","reported_seq":21036,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.847836+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:09:08.847836+0000","last_peered":"2024-09-10T16:09:08.847836+0000","last_clean":"2024-09-10T16:09:08.847836+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:09:08.847836+0000","last_undegraded":"2024-09-10T16:09:08.847836+0000","last_fullsized":"2024-09-10T16:09:08.847836+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17602,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'21061","reported_seq":21214,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:08.849450+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:09:08.849450+0000","last_peered":"2024-09-10T16:09:08.849450+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:09:08.849450+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17673,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4370},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'21082","reported_seq":21235,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:08.848683+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:09:08.848683+0000","last_peered":"2024-09-10T16:09:08.848683+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:09:08.848683+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17581,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4235},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'21314","reported_seq":21459,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.847784+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:09:08.847784+0000","last_peered":"2024-09-10T16:09:08.847784+0000","last_clean":"2024-09-10T16:09:08.847784+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:09:08.847784+0000","last_undegraded":"2024-09-10T16:09:08.847784+0000","last_fullsized":"2024-09-10T16:09:08.847784+0000","mapping_epoch":34,"log_start":"45'11300","ondisk_log_start":"45'11300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18161,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":85,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.851620+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:09:08.851620+0000","last_peered":"2024-09-10T16:09:08.851620+0000","last_clean":"2024-09-10T16:09:08.851620+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:09:08.851620+0000","last_undegraded":"2024-09-10T16:09:08.851620+0000","last_fullsized":"2024-09-10T16:09:08.851620+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20457","reported_seq":26377,"reported_epoch":51,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:26.157742+0000","last_change":"2024-09-10T16:09:07.396124+0000","last_active":"2024-09-10T16:09:26.157742+0000","last_peered":"2024-09-10T16:09:26.157742+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:09:26.157742+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1242,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18276,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2883,"num_bytes_recovered":4579,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1241}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":109,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.852443+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:09:08.852443+0000","last_peered":"2024-09-10T16:09:08.852443+0000","last_clean":"2024-09-10T16:09:08.852443+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:09:08.852443+0000","last_undegraded":"2024-09-10T16:09:08.852443+0000","last_fullsized":"2024-09-10T16:09:08.852443+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20910","reported_seq":29577,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.909669+0000","last_change":"2024-09-10T16:09:08.909580+0000","last_active":"2024-09-10T16:09:08.909669+0000","last_peered":"2024-09-10T16:09:08.909669+0000","last_clean":"2024-09-10T16:09:08.909669+0000","last_became_active":"2024-09-10T16:09:08.854941+0000","last_became_peered":"2024-09-10T16:09:08.854941+0000","last_unstale":"2024-09-10T16:09:08.909669+0000","last_undegraded":"2024-09-10T16:09:08.909669+0000","last_fullsized":"2024-09-10T16:09:08.909669+0000","mapping_epoch":50,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:13:14.846691+0000","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17414,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4257,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":119,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.847624+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:09:08.847624+0000","last_peered":"2024-09-10T16:09:08.847624+0000","last_clean":"2024-09-10T16:09:08.847624+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:09:08.847624+0000","last_undegraded":"2024-09-10T16:09:08.847624+0000","last_fullsized":"2024-09-10T16:09:08.847624+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20781","reported_seq":20931,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:08.852538+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:09:08.852538+0000","last_peered":"2024-09-10T16:09:08.852538+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:09:08.852538+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18461,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-4303}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":94,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.852182+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:09:08.852182+0000","last_peered":"2024-09-10T16:09:08.852182+0000","last_clean":"2024-09-10T16:09:08.852182+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:09:08.852182+0000","last_undegraded":"2024-09-10T16:09:08.852182+0000","last_fullsized":"2024-09-10T16:09:08.852182+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'21170","reported_seq":30029,"reported_epoch":51,"state":"active+recovery_wait","last_fresh":"2024-09-10T16:09:08.852233+0000","last_change":"2024-09-10T16:08:10.929169+0000","last_active":"2024-09-10T16:09:08.852233+0000","last_peered":"2024-09-10T16:09:08.852233+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T16:08:10.860438+0000","last_became_peered":"2024-09-10T16:08:10.860438+0000","last_unstale":"2024-09-10T16:09:08.852233+0000","last_undegraded":"2024-09-10T16:09:08.852233+0000","last_fullsized":"2024-09-10T16:09:08.852233+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18724,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","0","1"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20838","reported_seq":20988,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:08.849672+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:09:08.849672+0000","last_peered":"2024-09-10T16:09:08.849672+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:09:08.849672+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17469,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.851774+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:09:08.851774+0000","last_peered":"2024-09-10T16:09:08.851774+0000","last_clean":"2024-09-10T16:09:08.851774+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:09:08.851774+0000","last_undegraded":"2024-09-10T16:09:08.851774+0000","last_fullsized":"2024-09-10T16:09:08.851774+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20598","reported_seq":29178,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.851713+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:09:08.851713+0000","last_peered":"2024-09-10T16:09:08.851713+0000","last_clean":"2024-09-10T16:09:08.851713+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:09:08.851713+0000","last_undegraded":"2024-09-10T16:09:08.851713+0000","last_fullsized":"2024-09-10T16:09:08.851713+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16988,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":109,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.853357+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:09:08.853357+0000","last_peered":"2024-09-10T16:09:08.853357+0000","last_clean":"2024-09-10T16:09:08.853357+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:09:08.853357+0000","last_undegraded":"2024-09-10T16:09:08.853357+0000","last_fullsized":"2024-09-10T16:09:08.853357+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'21048","reported_seq":21205,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:08.848232+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:09:08.848232+0000","last_peered":"2024-09-10T16:09:08.848232+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:09:08.848232+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17303,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.851634+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:09:08.851634+0000","last_peered":"2024-09-10T16:09:08.851634+0000","last_clean":"2024-09-10T16:09:08.851634+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:09:08.851634+0000","last_undegraded":"2024-09-10T16:09:08.851634+0000","last_fullsized":"2024-09-10T16:09:08.851634+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":89,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.851698+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:09:08.851698+0000","last_peered":"2024-09-10T16:09:08.851698+0000","last_clean":"2024-09-10T16:09:08.851698+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:09:08.851698+0000","last_undegraded":"2024-09-10T16:09:08.851698+0000","last_fullsized":"2024-09-10T16:09:08.851698+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'21198","reported_seq":21357,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:08.848736+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:09:08.848736+0000","last_peered":"2024-09-10T16:09:08.848736+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:09:08.848736+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17361,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4321},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31418,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292096,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19972,"num_bytes_recovered":33096163,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":210518016,"data_stored":210395174,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161022,"ondisk_log_size":161022,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986313,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":502616,"kb_used_data":137728,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93869224,"statfs":{"total":96636764160,"available":96122085376,"internally_reserved":0,"allocated":141033472,"data_stored":140713808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659258,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":461164,"kb_used_data":476,"kb_used_omap":4,"kb_used_meta":460667,"kb_avail":93910676,"statfs":{"total":96636764160,"available":96164532224,"internally_reserved":0,"allocated":487424,"data_stored":149231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":471723694},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182541,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":547316,"kb_used_data":68636,"kb_used_omap":4,"kb_used_meta":478651,"kb_avail":93824524,"statfs":{"total":96636764160,"available":96076312576,"internally_reserved":0,"allocated":70283264,"data_stored":69959523,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":490139306},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":69853184,"data_stored":69817054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":6762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":140607488,"data_stored":140571358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:09:33.425 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:09:33.799 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:09:33.799 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:09:33.813 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":415,"stamp":"2024-09-10T16:09:33.774441+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30352,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292098,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21040,"num_bytes_recovered":33096182,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161025,"ondisk_log_size":161025,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1519416,"kb_used_data":206840,"kb_used_omap":12,"kb_used_meta":1312499,"kb_avail":281596104,"statfs":{"total":289910292480,"available":288354410496,"internally_reserved":0,"allocated":211804160,"data_stored":210822562,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1343998978},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1861,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1861,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001468"},"pg_stats":[{"pgid":"2.f","version":"49'20672","reported_seq":20762,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:08.851657+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:09:08.851657+0000","last_peered":"2024-09-10T16:09:08.851657+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:09:08.851657+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20672,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4259}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20844","reported_seq":20971,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.853495+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:09:08.853495+0000","last_peered":"2024-09-10T16:09:08.853495+0000","last_clean":"2024-09-10T16:09:08.853495+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:09:08.853495+0000","last_undegraded":"2024-09-10T16:09:08.853495+0000","last_fullsized":"2024-09-10T16:09:08.853495+0000","mapping_epoch":35,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18949,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20798","reported_seq":29491,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.852143+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:09:08.852143+0000","last_peered":"2024-09-10T16:09:08.852143+0000","last_clean":"2024-09-10T16:09:08.852143+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:09:08.852143+0000","last_undegraded":"2024-09-10T16:09:08.852143+0000","last_fullsized":"2024-09-10T16:09:08.852143+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18802,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20660","reported_seq":20755,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.853148+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:09:08.853148+0000","last_peered":"2024-09-10T16:09:08.853148+0000","last_clean":"2024-09-10T16:09:08.853148+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:09:08.853148+0000","last_undegraded":"2024-09-10T16:09:08.853148+0000","last_fullsized":"2024-09-10T16:09:08.853148+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20660,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20891","reported_seq":21036,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.847836+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:09:08.847836+0000","last_peered":"2024-09-10T16:09:08.847836+0000","last_clean":"2024-09-10T16:09:08.847836+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:09:08.847836+0000","last_undegraded":"2024-09-10T16:09:08.847836+0000","last_fullsized":"2024-09-10T16:09:08.847836+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17602,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'21061","reported_seq":21214,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:08.849450+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:09:08.849450+0000","last_peered":"2024-09-10T16:09:08.849450+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:09:08.849450+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17673,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4370},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'21082","reported_seq":21235,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:08.848683+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:09:08.848683+0000","last_peered":"2024-09-10T16:09:08.848683+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:09:08.848683+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17581,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4235},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'21314","reported_seq":21459,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.847784+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:09:08.847784+0000","last_peered":"2024-09-10T16:09:08.847784+0000","last_clean":"2024-09-10T16:09:08.847784+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:09:08.847784+0000","last_undegraded":"2024-09-10T16:09:08.847784+0000","last_fullsized":"2024-09-10T16:09:08.847784+0000","mapping_epoch":34,"log_start":"45'11300","ondisk_log_start":"45'11300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18161,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":85,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.851620+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:09:08.851620+0000","last_peered":"2024-09-10T16:09:08.851620+0000","last_clean":"2024-09-10T16:09:08.851620+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:09:08.851620+0000","last_undegraded":"2024-09-10T16:09:08.851620+0000","last_fullsized":"2024-09-10T16:09:08.851620+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20457","reported_seq":28509,"reported_epoch":51,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:31.159958+0000","last_change":"2024-09-10T16:09:07.396124+0000","last_active":"2024-09-10T16:09:31.159958+0000","last_peered":"2024-09-10T16:09:31.159958+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:09:31.159958+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":176,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18276,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3949,"num_bytes_recovered":4579,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-175}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":109,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.852443+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:09:08.852443+0000","last_peered":"2024-09-10T16:09:08.852443+0000","last_clean":"2024-09-10T16:09:08.852443+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:09:08.852443+0000","last_undegraded":"2024-09-10T16:09:08.852443+0000","last_fullsized":"2024-09-10T16:09:08.852443+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20910","reported_seq":29578,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.910137+0000","last_change":"2024-09-10T16:09:08.909580+0000","last_active":"2024-09-10T16:09:08.910137+0000","last_peered":"2024-09-10T16:09:08.910137+0000","last_clean":"2024-09-10T16:09:08.910137+0000","last_became_active":"2024-09-10T16:09:08.854941+0000","last_became_peered":"2024-09-10T16:09:08.854941+0000","last_unstale":"2024-09-10T16:09:08.910137+0000","last_undegraded":"2024-09-10T16:09:08.910137+0000","last_fullsized":"2024-09-10T16:09:08.910137+0000","mapping_epoch":50,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:13:14.846691+0000","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17414,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4257,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":119,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.847624+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:09:08.847624+0000","last_peered":"2024-09-10T16:09:08.847624+0000","last_clean":"2024-09-10T16:09:08.847624+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:09:08.847624+0000","last_undegraded":"2024-09-10T16:09:08.847624+0000","last_fullsized":"2024-09-10T16:09:08.847624+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20781","reported_seq":20931,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:08.852538+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:09:08.852538+0000","last_peered":"2024-09-10T16:09:08.852538+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:09:08.852538+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18461,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-4303}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":94,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.852182+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:09:08.852182+0000","last_peered":"2024-09-10T16:09:08.852182+0000","last_clean":"2024-09-10T16:09:08.852182+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:09:08.852182+0000","last_undegraded":"2024-09-10T16:09:08.852182+0000","last_fullsized":"2024-09-10T16:09:08.852182+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'21170","reported_seq":30029,"reported_epoch":51,"state":"active+recovery_wait","last_fresh":"2024-09-10T16:09:08.852233+0000","last_change":"2024-09-10T16:08:10.929169+0000","last_active":"2024-09-10T16:09:08.852233+0000","last_peered":"2024-09-10T16:09:08.852233+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T16:08:10.860438+0000","last_became_peered":"2024-09-10T16:08:10.860438+0000","last_unstale":"2024-09-10T16:09:08.852233+0000","last_undegraded":"2024-09-10T16:09:08.852233+0000","last_fullsized":"2024-09-10T16:09:08.852233+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18724,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","0","1"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20838","reported_seq":20988,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:08.849672+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:09:08.849672+0000","last_peered":"2024-09-10T16:09:08.849672+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:09:08.849672+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17469,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.851774+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:09:08.851774+0000","last_peered":"2024-09-10T16:09:08.851774+0000","last_clean":"2024-09-10T16:09:08.851774+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:09:08.851774+0000","last_undegraded":"2024-09-10T16:09:08.851774+0000","last_fullsized":"2024-09-10T16:09:08.851774+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20598","reported_seq":29178,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.851713+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:09:08.851713+0000","last_peered":"2024-09-10T16:09:08.851713+0000","last_clean":"2024-09-10T16:09:08.851713+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:09:08.851713+0000","last_undegraded":"2024-09-10T16:09:08.851713+0000","last_fullsized":"2024-09-10T16:09:08.851713+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16988,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":109,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.853357+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:09:08.853357+0000","last_peered":"2024-09-10T16:09:08.853357+0000","last_clean":"2024-09-10T16:09:08.853357+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:09:08.853357+0000","last_undegraded":"2024-09-10T16:09:08.853357+0000","last_fullsized":"2024-09-10T16:09:08.853357+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'21048","reported_seq":21205,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:08.848232+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:09:08.848232+0000","last_peered":"2024-09-10T16:09:08.848232+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:09:08.848232+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17303,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.851634+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:09:08.851634+0000","last_peered":"2024-09-10T16:09:08.851634+0000","last_clean":"2024-09-10T16:09:08.851634+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:09:08.851634+0000","last_undegraded":"2024-09-10T16:09:08.851634+0000","last_fullsized":"2024-09-10T16:09:08.851634+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":89,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.851698+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:09:08.851698+0000","last_peered":"2024-09-10T16:09:08.851698+0000","last_clean":"2024-09-10T16:09:08.851698+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:09:08.851698+0000","last_undegraded":"2024-09-10T16:09:08.851698+0000","last_fullsized":"2024-09-10T16:09:08.851698+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'21198","reported_seq":21357,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:08.848736+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:09:08.848736+0000","last_peered":"2024-09-10T16:09:08.848736+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:09:08.848736+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17361,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4321},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30352,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292096,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21038,"num_bytes_recovered":33096163,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":210518016,"data_stored":210395174,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161022,"ondisk_log_size":161022,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986314,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":502616,"kb_used_data":137728,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93869224,"statfs":{"total":96636764160,"available":96122085376,"internally_reserved":0,"allocated":141033472,"data_stored":140713808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659259,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":469484,"kb_used_data":476,"kb_used_omap":4,"kb_used_meta":468987,"kb_avail":93902356,"statfs":{"total":96636764160,"available":96156012544,"internally_reserved":0,"allocated":487424,"data_stored":149231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":480243374},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182542,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":547316,"kb_used_data":68636,"kb_used_omap":4,"kb_used_meta":478651,"kb_avail":93824524,"statfs":{"total":96636764160,"available":96076312576,"internally_reserved":0,"allocated":70283264,"data_stored":69959523,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":490139306},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":69853184,"data_stored":69817054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":6762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":140607488,"data_stored":140571358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:09:33.814 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:09:34.160 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:09:34.160 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:09:34.176 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":415,"stamp":"2024-09-10T16:09:33.774441+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30352,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292098,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21040,"num_bytes_recovered":33096182,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161025,"ondisk_log_size":161025,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1519416,"kb_used_data":206840,"kb_used_omap":12,"kb_used_meta":1312499,"kb_avail":281596104,"statfs":{"total":289910292480,"available":288354410496,"internally_reserved":0,"allocated":211804160,"data_stored":210822562,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1343998978},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1861,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1861,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001468"},"pg_stats":[{"pgid":"2.f","version":"49'20672","reported_seq":20762,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:08.851657+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:09:08.851657+0000","last_peered":"2024-09-10T16:09:08.851657+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:09:08.851657+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20672,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4259}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20844","reported_seq":20971,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.853495+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:09:08.853495+0000","last_peered":"2024-09-10T16:09:08.853495+0000","last_clean":"2024-09-10T16:09:08.853495+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:09:08.853495+0000","last_undegraded":"2024-09-10T16:09:08.853495+0000","last_fullsized":"2024-09-10T16:09:08.853495+0000","mapping_epoch":35,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18949,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20798","reported_seq":29491,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.852143+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:09:08.852143+0000","last_peered":"2024-09-10T16:09:08.852143+0000","last_clean":"2024-09-10T16:09:08.852143+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:09:08.852143+0000","last_undegraded":"2024-09-10T16:09:08.852143+0000","last_fullsized":"2024-09-10T16:09:08.852143+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18802,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20660","reported_seq":20755,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.853148+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:09:08.853148+0000","last_peered":"2024-09-10T16:09:08.853148+0000","last_clean":"2024-09-10T16:09:08.853148+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:09:08.853148+0000","last_undegraded":"2024-09-10T16:09:08.853148+0000","last_fullsized":"2024-09-10T16:09:08.853148+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20660,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20891","reported_seq":21036,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.847836+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:09:08.847836+0000","last_peered":"2024-09-10T16:09:08.847836+0000","last_clean":"2024-09-10T16:09:08.847836+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:09:08.847836+0000","last_undegraded":"2024-09-10T16:09:08.847836+0000","last_fullsized":"2024-09-10T16:09:08.847836+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17602,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'21061","reported_seq":21214,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:08.849450+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:09:08.849450+0000","last_peered":"2024-09-10T16:09:08.849450+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:09:08.849450+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17673,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4370},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'21082","reported_seq":21235,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:08.848683+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:09:08.848683+0000","last_peered":"2024-09-10T16:09:08.848683+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:09:08.848683+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17581,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4235},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'21314","reported_seq":21459,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.847784+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:09:08.847784+0000","last_peered":"2024-09-10T16:09:08.847784+0000","last_clean":"2024-09-10T16:09:08.847784+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:09:08.847784+0000","last_undegraded":"2024-09-10T16:09:08.847784+0000","last_fullsized":"2024-09-10T16:09:08.847784+0000","mapping_epoch":34,"log_start":"45'11300","ondisk_log_start":"45'11300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18161,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":85,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.851620+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:09:08.851620+0000","last_peered":"2024-09-10T16:09:08.851620+0000","last_clean":"2024-09-10T16:09:08.851620+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:09:08.851620+0000","last_undegraded":"2024-09-10T16:09:08.851620+0000","last_fullsized":"2024-09-10T16:09:08.851620+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20457","reported_seq":28509,"reported_epoch":51,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:31.159958+0000","last_change":"2024-09-10T16:09:07.396124+0000","last_active":"2024-09-10T16:09:31.159958+0000","last_peered":"2024-09-10T16:09:31.159958+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:09:31.159958+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":176,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18276,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3949,"num_bytes_recovered":4579,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-175}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":109,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.852443+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:09:08.852443+0000","last_peered":"2024-09-10T16:09:08.852443+0000","last_clean":"2024-09-10T16:09:08.852443+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:09:08.852443+0000","last_undegraded":"2024-09-10T16:09:08.852443+0000","last_fullsized":"2024-09-10T16:09:08.852443+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20910","reported_seq":29578,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.910137+0000","last_change":"2024-09-10T16:09:08.909580+0000","last_active":"2024-09-10T16:09:08.910137+0000","last_peered":"2024-09-10T16:09:08.910137+0000","last_clean":"2024-09-10T16:09:08.910137+0000","last_became_active":"2024-09-10T16:09:08.854941+0000","last_became_peered":"2024-09-10T16:09:08.854941+0000","last_unstale":"2024-09-10T16:09:08.910137+0000","last_undegraded":"2024-09-10T16:09:08.910137+0000","last_fullsized":"2024-09-10T16:09:08.910137+0000","mapping_epoch":50,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:13:14.846691+0000","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17414,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4257,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":119,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.847624+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:09:08.847624+0000","last_peered":"2024-09-10T16:09:08.847624+0000","last_clean":"2024-09-10T16:09:08.847624+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:09:08.847624+0000","last_undegraded":"2024-09-10T16:09:08.847624+0000","last_fullsized":"2024-09-10T16:09:08.847624+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20781","reported_seq":20931,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:08.852538+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:09:08.852538+0000","last_peered":"2024-09-10T16:09:08.852538+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:09:08.852538+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18461,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-4303}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":94,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.852182+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:09:08.852182+0000","last_peered":"2024-09-10T16:09:08.852182+0000","last_clean":"2024-09-10T16:09:08.852182+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:09:08.852182+0000","last_undegraded":"2024-09-10T16:09:08.852182+0000","last_fullsized":"2024-09-10T16:09:08.852182+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'21170","reported_seq":30029,"reported_epoch":51,"state":"active+recovery_wait","last_fresh":"2024-09-10T16:09:08.852233+0000","last_change":"2024-09-10T16:08:10.929169+0000","last_active":"2024-09-10T16:09:08.852233+0000","last_peered":"2024-09-10T16:09:08.852233+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T16:08:10.860438+0000","last_became_peered":"2024-09-10T16:08:10.860438+0000","last_unstale":"2024-09-10T16:09:08.852233+0000","last_undegraded":"2024-09-10T16:09:08.852233+0000","last_fullsized":"2024-09-10T16:09:08.852233+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18724,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","0","1"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20838","reported_seq":20988,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:08.849672+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:09:08.849672+0000","last_peered":"2024-09-10T16:09:08.849672+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:09:08.849672+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17469,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.851774+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:09:08.851774+0000","last_peered":"2024-09-10T16:09:08.851774+0000","last_clean":"2024-09-10T16:09:08.851774+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:09:08.851774+0000","last_undegraded":"2024-09-10T16:09:08.851774+0000","last_fullsized":"2024-09-10T16:09:08.851774+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20598","reported_seq":29178,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.851713+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:09:08.851713+0000","last_peered":"2024-09-10T16:09:08.851713+0000","last_clean":"2024-09-10T16:09:08.851713+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:09:08.851713+0000","last_undegraded":"2024-09-10T16:09:08.851713+0000","last_fullsized":"2024-09-10T16:09:08.851713+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16988,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":109,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.853357+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:09:08.853357+0000","last_peered":"2024-09-10T16:09:08.853357+0000","last_clean":"2024-09-10T16:09:08.853357+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:09:08.853357+0000","last_undegraded":"2024-09-10T16:09:08.853357+0000","last_fullsized":"2024-09-10T16:09:08.853357+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'21048","reported_seq":21205,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:08.848232+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:09:08.848232+0000","last_peered":"2024-09-10T16:09:08.848232+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:09:08.848232+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17303,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.851634+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:09:08.851634+0000","last_peered":"2024-09-10T16:09:08.851634+0000","last_clean":"2024-09-10T16:09:08.851634+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:09:08.851634+0000","last_undegraded":"2024-09-10T16:09:08.851634+0000","last_fullsized":"2024-09-10T16:09:08.851634+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":89,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.851698+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:09:08.851698+0000","last_peered":"2024-09-10T16:09:08.851698+0000","last_clean":"2024-09-10T16:09:08.851698+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:09:08.851698+0000","last_undegraded":"2024-09-10T16:09:08.851698+0000","last_fullsized":"2024-09-10T16:09:08.851698+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'21198","reported_seq":21357,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:08.848736+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:09:08.848736+0000","last_peered":"2024-09-10T16:09:08.848736+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:09:08.848736+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17361,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4321},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30352,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292096,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21038,"num_bytes_recovered":33096163,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":210518016,"data_stored":210395174,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161022,"ondisk_log_size":161022,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986314,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":502616,"kb_used_data":137728,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93869224,"statfs":{"total":96636764160,"available":96122085376,"internally_reserved":0,"allocated":141033472,"data_stored":140713808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659259,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":469484,"kb_used_data":476,"kb_used_omap":4,"kb_used_meta":468987,"kb_avail":93902356,"statfs":{"total":96636764160,"available":96156012544,"internally_reserved":0,"allocated":487424,"data_stored":149231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":480243374},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182542,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":547316,"kb_used_data":68636,"kb_used_omap":4,"kb_used_meta":478651,"kb_avail":93824524,"statfs":{"total":96636764160,"available":96076312576,"internally_reserved":0,"allocated":70283264,"data_stored":69959523,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":490139306},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":69853184,"data_stored":69817054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":6762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":140607488,"data_stored":140571358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:09:34.177 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-10T16:09:34.177 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "49'20672", 'reported_seq': 20762, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:09:08.851657+0000', 'last_change': '2024-09-10T15:59:16.281558+0000', 'last_active': '2024-09-10T16:09:08.851657+0000', 'last_peered': '2024-09-10T16:09:08.851657+0000', 'last_clean': '2024-09-10T15:59:14.132456+0000', 'last_became_active': '2024-09-10T15:59:16.274407+0000', 'last_became_peered': '2024-09-10T15:59:16.274407+0000', 'last_unstale': '2024-09-10T16:09:08.851657+0000', 'last_undegraded': '2024-09-10T15:59:15.851279+0000', 'last_fullsized': '2024-09-10T15:59:15.774356+0000', 'mapping_epoch': 42, 'log_start': "45'10600", 'ondisk_log_start': "45'10600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'objects_scrubbed': 0, 'log_size': 10072, 'log_dups_size': 0, 'ondisk_log_size': 10072, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4259, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20672, 'num_write_kb': 41344, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4259}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:09:34.177 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-10T16:09:34.178 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "49'21061", 'reported_seq': 21214, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:09:08.849450+0000', 'last_change': '2024-09-10T15:59:16.308264+0000', 'last_active': '2024-09-10T16:09:08.849450+0000', 'last_peered': '2024-09-10T16:09:08.849450+0000', 'last_clean': '2024-09-10T15:59:14.111551+0000', 'last_became_active': '2024-09-10T15:59:16.300040+0000', 'last_became_peered': '2024-09-10T15:59:16.300040+0000', 'last_unstale': '2024-09-10T16:09:08.849450+0000', 'last_undegraded': '2024-09-10T15:59:15.878234+0000', 'last_fullsized': '2024-09-10T15:59:15.769328+0000', 'mapping_epoch': 42, 'log_start': "45'11000", 'ondisk_log_start': "45'11000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3442", 'last_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'objects_scrubbed': 3384, 'log_size': 10061, 'log_dups_size': 0, 'ondisk_log_size': 10061, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1292, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4372, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 17673, 'num_write_kb': 28580, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4370}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:09:34.178 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-10T16:09:34.178 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "49'21082", 'reported_seq': 21235, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:09:08.848683+0000', 'last_change': '2024-09-10T15:59:16.331899+0000', 'last_active': '2024-09-10T16:09:08.848683+0000', 'last_peered': '2024-09-10T16:09:08.848683+0000', 'last_clean': '2024-09-10T15:59:14.139436+0000', 'last_became_active': '2024-09-10T15:59:16.318715+0000', 'last_became_peered': '2024-09-10T15:59:16.318715+0000', 'last_unstale': '2024-09-10T16:09:08.848683+0000', 'last_undegraded': '2024-09-10T15:59:15.877524+0000', 'last_fullsized': '2024-09-10T15:59:15.769042+0000', 'mapping_epoch': 42, 'log_start': "45'11000", 'ondisk_log_start': "45'11000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3537", 'last_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'objects_scrubbed': 3497, 'log_size': 10082, 'log_dups_size': 0, 'ondisk_log_size': 10082, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1224, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4237, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 17581, 'num_write_kb': 28172, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4235}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:09:34.178 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-10T16:09:34.178 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "49'20457", 'reported_seq': 28509, 'reported_epoch': 51, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:09:31.159958+0000', 'last_change': '2024-09-10T16:09:07.396124+0000', 'last_active': '2024-09-10T16:09:31.159958+0000', 'last_peered': '2024-09-10T16:09:31.159958+0000', 'last_clean': '2024-09-10T15:59:14.088187+0000', 'last_became_active': '2024-09-10T15:59:16.165810+0000', 'last_became_peered': '2024-09-10T15:59:16.165810+0000', 'last_unstale': '2024-09-10T16:09:31.159958+0000', 'last_undegraded': '2024-09-10T15:59:15.822098+0000', 'last_fullsized': '2024-09-10T15:59:15.769123+0000', 'mapping_epoch': 42, 'log_start': "45'10400", 'ondisk_log_start': "45'10400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2218", 'last_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:48.061852+0000', 'objects_scrubbed': 2175, 'log_size': 10057, 'log_dups_size': 0, 'ondisk_log_size': 10057, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 825, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 176, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 18276, 'num_write_kb': 32204, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3949, 'num_bytes_recovered': 4579, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -175}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:09:34.178 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-10T16:09:34.178 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "49'20781", 'reported_seq': 20931, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:09:08.852538+0000', 'last_change': '2024-09-10T15:59:16.284311+0000', 'last_active': '2024-09-10T16:09:08.852538+0000', 'last_peered': '2024-09-10T16:09:08.852538+0000', 'last_clean': '2024-09-10T15:59:14.116041+0000', 'last_became_active': '2024-09-10T15:59:16.274543+0000', 'last_became_peered': '2024-09-10T15:59:16.274543+0000', 'last_unstale': '2024-09-10T16:09:08.852538+0000', 'last_undegraded': '2024-09-10T15:59:15.812484+0000', 'last_fullsized': '2024-09-10T15:59:15.770885+0000', 'mapping_epoch': 42, 'log_start': "45'10700", 'ondisk_log_start': "45'10700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2356", 'last_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'objects_scrubbed': 2316, 'log_size': 10081, 'log_dups_size': 0, 'ondisk_log_size': 10081, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 783, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 18461, 'num_write_kb': 32292, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': -4303}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:09:34.178 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-10T16:09:34.178 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "49'21170", 'reported_seq': 30029, 'reported_epoch': 51, 'state': 'active+recovery_wait', 'last_fresh': '2024-09-10T16:09:08.852233+0000', 'last_change': '2024-09-10T16:08:10.929169+0000', 'last_active': '2024-09-10T16:09:08.852233+0000', 'last_peered': '2024-09-10T16:09:08.852233+0000', 'last_clean': '2024-09-10T15:59:14.110654+0000', 'last_became_active': '2024-09-10T16:08:10.860438+0000', 'last_became_peered': '2024-09-10T16:08:10.860438+0000', 'last_unstale': '2024-09-10T16:09:08.852233+0000', 'last_undegraded': '2024-09-10T16:09:08.852233+0000', 'last_fullsized': '2024-09-10T16:09:08.852233+0000', 'mapping_epoch': 48, 'log_start': "45'11100", 'ondisk_log_start': "45'11100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2486", 'last_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'objects_scrubbed': 2440, 'log_size': 10070, 'log_dups_size': 0, 'ondisk_log_size': 10070, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 792, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18724, 'num_write_kb': 32568, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 4355, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': ['2', '0', '1'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:09:34.178 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-10T16:09:34.178 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "49'20838", 'reported_seq': 20988, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:09:08.849672+0000', 'last_change': '2024-09-10T15:59:16.315641+0000', 'last_active': '2024-09-10T16:09:08.849672+0000', 'last_peered': '2024-09-10T16:09:08.849672+0000', 'last_clean': '2024-09-10T15:59:14.117001+0000', 'last_became_active': '2024-09-10T15:59:16.302245+0000', 'last_became_peered': '2024-09-10T15:59:16.302245+0000', 'last_unstale': '2024-09-10T16:09:08.849672+0000', 'last_undegraded': '2024-09-10T15:59:15.845012+0000', 'last_fullsized': '2024-09-10T15:59:15.770337+0000', 'mapping_epoch': 42, 'log_start': "45'10800", 'ondisk_log_start': "45'10800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3404", 'last_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'objects_scrubbed': 3367, 'log_size': 10038, 'log_dups_size': 0, 'ondisk_log_size': 10038, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 983, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17469, 'num_write_kb': 28204, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4304}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:09:34.179 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-10T16:09:34.179 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "49'21048", 'reported_seq': 21205, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:09:08.848232+0000', 'last_change': '2024-09-10T15:59:16.311291+0000', 'last_active': '2024-09-10T16:09:08.848232+0000', 'last_peered': '2024-09-10T16:09:08.848232+0000', 'last_clean': '2024-09-10T15:59:14.116363+0000', 'last_became_active': '2024-09-10T15:59:16.301244+0000', 'last_became_peered': '2024-09-10T15:59:16.301244+0000', 'last_unstale': '2024-09-10T16:09:08.848232+0000', 'last_undegraded': '2024-09-10T15:59:15.883434+0000', 'last_fullsized': '2024-09-10T15:59:15.772756+0000', 'mapping_epoch': 42, 'log_start': "45'11000", 'ondisk_log_start': "45'11000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3789", 'last_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'objects_scrubbed': 3741, 'log_size': 10048, 'log_dups_size': 0, 'ondisk_log_size': 10048, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1440, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4373, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17303, 'num_write_kb': 27124, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4371}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:09:34.179 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-10T16:09:34.179 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "49'21198", 'reported_seq': 21357, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:09:08.848736+0000', 'last_change': '2024-09-10T15:59:16.313409+0000', 'last_active': '2024-09-10T16:09:08.848736+0000', 'last_peered': '2024-09-10T16:09:08.848736+0000', 'last_clean': '2024-09-10T15:59:14.129780+0000', 'last_became_active': '2024-09-10T15:59:16.300932+0000', 'last_became_peered': '2024-09-10T15:59:16.300932+0000', 'last_unstale': '2024-09-10T16:09:08.848736+0000', 'last_undegraded': '2024-09-10T15:59:15.887823+0000', 'last_fullsized': '2024-09-10T15:59:15.768513+0000', 'mapping_epoch': 42, 'log_start': "45'11100", 'ondisk_log_start': "45'11100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3880", 'last_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'objects_scrubbed': 3827, 'log_size': 10098, 'log_dups_size': 0, 'ondisk_log_size': 10098, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1397, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4325, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17361, 'num_write_kb': 27072, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4321}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:09:34.179 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T16:09:34.572 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:09:34.572 INFO:teuthology.orchestra.run.smithi115.stdout:{"fsid":"7effee29-df05-4351-9741-ee203564ed81","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 30352/24 objects degraded (126466.667%), 8 pgs degraded, 8 pgs undersized","count":16},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":738,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":53,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983954,"num_in_osds":3,"osd_in_since":1725983947,"num_remapped_pgs":7},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":15},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":7},{"state_name":"active+recovery_wait","count":1},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":12,"data_bytes":4849,"bytes_used":1555881984,"bytes_avail":288354410496,"bytes_total":289910292480,"degraded_objects":30352,"degraded_total":24,"degraded_ratio":1264.6666666666667,"recovering_objects_per_sec":232,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1861,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-10T15:57:14:967120+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T16:00:13.560776+0000","services":{}},"progress_events":{}} 2024-09-10T16:09:34.588 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-10T16:09:34.588 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:09:34.904 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:09:34.904 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:09:34.917 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":415,"stamp":"2024-09-10T16:09:33.774441+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30352,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292098,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21040,"num_bytes_recovered":33096182,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161025,"ondisk_log_size":161025,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1519416,"kb_used_data":206840,"kb_used_omap":12,"kb_used_meta":1312499,"kb_avail":281596104,"statfs":{"total":289910292480,"available":288354410496,"internally_reserved":0,"allocated":211804160,"data_stored":210822562,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1343998978},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1861,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1861,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001468"},"pg_stats":[{"pgid":"2.f","version":"49'20672","reported_seq":20762,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:08.851657+0000","last_change":"2024-09-10T15:59:16.281558+0000","last_active":"2024-09-10T16:09:08.851657+0000","last_peered":"2024-09-10T16:09:08.851657+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:09:08.851657+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20672,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4259}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20844","reported_seq":20971,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.853495+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:09:08.853495+0000","last_peered":"2024-09-10T16:09:08.853495+0000","last_clean":"2024-09-10T16:09:08.853495+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:09:08.853495+0000","last_undegraded":"2024-09-10T16:09:08.853495+0000","last_fullsized":"2024-09-10T16:09:08.853495+0000","mapping_epoch":35,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18949,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20798","reported_seq":29491,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.852143+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:09:08.852143+0000","last_peered":"2024-09-10T16:09:08.852143+0000","last_clean":"2024-09-10T16:09:08.852143+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:09:08.852143+0000","last_undegraded":"2024-09-10T16:09:08.852143+0000","last_fullsized":"2024-09-10T16:09:08.852143+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18802,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20660","reported_seq":20755,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.853148+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:09:08.853148+0000","last_peered":"2024-09-10T16:09:08.853148+0000","last_clean":"2024-09-10T16:09:08.853148+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:09:08.853148+0000","last_undegraded":"2024-09-10T16:09:08.853148+0000","last_fullsized":"2024-09-10T16:09:08.853148+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20660,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20891","reported_seq":21036,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.847836+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:09:08.847836+0000","last_peered":"2024-09-10T16:09:08.847836+0000","last_clean":"2024-09-10T16:09:08.847836+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:09:08.847836+0000","last_undegraded":"2024-09-10T16:09:08.847836+0000","last_fullsized":"2024-09-10T16:09:08.847836+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17602,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'21061","reported_seq":21214,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:08.849450+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:09:08.849450+0000","last_peered":"2024-09-10T16:09:08.849450+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:09:08.849450+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17673,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4370},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'21082","reported_seq":21235,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:08.848683+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:09:08.848683+0000","last_peered":"2024-09-10T16:09:08.848683+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:09:08.848683+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17581,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4235},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'21314","reported_seq":21459,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.847784+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:09:08.847784+0000","last_peered":"2024-09-10T16:09:08.847784+0000","last_clean":"2024-09-10T16:09:08.847784+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:09:08.847784+0000","last_undegraded":"2024-09-10T16:09:08.847784+0000","last_fullsized":"2024-09-10T16:09:08.847784+0000","mapping_epoch":34,"log_start":"45'11300","ondisk_log_start":"45'11300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18161,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":85,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.851620+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:09:08.851620+0000","last_peered":"2024-09-10T16:09:08.851620+0000","last_clean":"2024-09-10T16:09:08.851620+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:09:08.851620+0000","last_undegraded":"2024-09-10T16:09:08.851620+0000","last_fullsized":"2024-09-10T16:09:08.851620+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20457","reported_seq":28509,"reported_epoch":51,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:31.159958+0000","last_change":"2024-09-10T16:09:07.396124+0000","last_active":"2024-09-10T16:09:31.159958+0000","last_peered":"2024-09-10T16:09:31.159958+0000","last_clean":"2024-09-10T15:59:14.088187+0000","last_became_active":"2024-09-10T15:59:16.165810+0000","last_became_peered":"2024-09-10T15:59:16.165810+0000","last_unstale":"2024-09-10T16:09:31.159958+0000","last_undegraded":"2024-09-10T15:59:15.822098+0000","last_fullsized":"2024-09-10T15:59:15.769123+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":176,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18276,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3949,"num_bytes_recovered":4579,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-175}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":109,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.852443+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:09:08.852443+0000","last_peered":"2024-09-10T16:09:08.852443+0000","last_clean":"2024-09-10T16:09:08.852443+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:09:08.852443+0000","last_undegraded":"2024-09-10T16:09:08.852443+0000","last_fullsized":"2024-09-10T16:09:08.852443+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20910","reported_seq":29578,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.910137+0000","last_change":"2024-09-10T16:09:08.909580+0000","last_active":"2024-09-10T16:09:08.910137+0000","last_peered":"2024-09-10T16:09:08.910137+0000","last_clean":"2024-09-10T16:09:08.910137+0000","last_became_active":"2024-09-10T16:09:08.854941+0000","last_became_peered":"2024-09-10T16:09:08.854941+0000","last_unstale":"2024-09-10T16:09:08.910137+0000","last_undegraded":"2024-09-10T16:09:08.910137+0000","last_fullsized":"2024-09-10T16:09:08.910137+0000","mapping_epoch":50,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:13:14.846691+0000","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17414,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4257,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":119,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.847624+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:09:08.847624+0000","last_peered":"2024-09-10T16:09:08.847624+0000","last_clean":"2024-09-10T16:09:08.847624+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:09:08.847624+0000","last_undegraded":"2024-09-10T16:09:08.847624+0000","last_fullsized":"2024-09-10T16:09:08.847624+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20781","reported_seq":20931,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:08.852538+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:09:08.852538+0000","last_peered":"2024-09-10T16:09:08.852538+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:09:08.852538+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18461,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-4303}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":94,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.852182+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:09:08.852182+0000","last_peered":"2024-09-10T16:09:08.852182+0000","last_clean":"2024-09-10T16:09:08.852182+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:09:08.852182+0000","last_undegraded":"2024-09-10T16:09:08.852182+0000","last_fullsized":"2024-09-10T16:09:08.852182+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'21170","reported_seq":30029,"reported_epoch":51,"state":"active+recovery_wait","last_fresh":"2024-09-10T16:09:08.852233+0000","last_change":"2024-09-10T16:08:10.929169+0000","last_active":"2024-09-10T16:09:08.852233+0000","last_peered":"2024-09-10T16:09:08.852233+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T16:08:10.860438+0000","last_became_peered":"2024-09-10T16:08:10.860438+0000","last_unstale":"2024-09-10T16:09:08.852233+0000","last_undegraded":"2024-09-10T16:09:08.852233+0000","last_fullsized":"2024-09-10T16:09:08.852233+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18724,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","0","1"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20838","reported_seq":20988,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:08.849672+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:09:08.849672+0000","last_peered":"2024-09-10T16:09:08.849672+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:09:08.849672+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17469,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.851774+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:09:08.851774+0000","last_peered":"2024-09-10T16:09:08.851774+0000","last_clean":"2024-09-10T16:09:08.851774+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:09:08.851774+0000","last_undegraded":"2024-09-10T16:09:08.851774+0000","last_fullsized":"2024-09-10T16:09:08.851774+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20598","reported_seq":29178,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.851713+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:09:08.851713+0000","last_peered":"2024-09-10T16:09:08.851713+0000","last_clean":"2024-09-10T16:09:08.851713+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:09:08.851713+0000","last_undegraded":"2024-09-10T16:09:08.851713+0000","last_fullsized":"2024-09-10T16:09:08.851713+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16988,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":109,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.853357+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:09:08.853357+0000","last_peered":"2024-09-10T16:09:08.853357+0000","last_clean":"2024-09-10T16:09:08.853357+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:09:08.853357+0000","last_undegraded":"2024-09-10T16:09:08.853357+0000","last_fullsized":"2024-09-10T16:09:08.853357+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'21048","reported_seq":21205,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:08.848232+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:09:08.848232+0000","last_peered":"2024-09-10T16:09:08.848232+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:09:08.848232+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17303,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.851634+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:09:08.851634+0000","last_peered":"2024-09-10T16:09:08.851634+0000","last_clean":"2024-09-10T16:09:08.851634+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:09:08.851634+0000","last_undegraded":"2024-09-10T16:09:08.851634+0000","last_fullsized":"2024-09-10T16:09:08.851634+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":89,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-10T16:09:08.851698+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:09:08.851698+0000","last_peered":"2024-09-10T16:09:08.851698+0000","last_clean":"2024-09-10T16:09:08.851698+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:09:08.851698+0000","last_undegraded":"2024-09-10T16:09:08.851698+0000","last_fullsized":"2024-09-10T16:09:08.851698+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'21198","reported_seq":21357,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:08.848736+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:09:08.848736+0000","last_peered":"2024-09-10T16:09:08.848736+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:09:08.848736+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17361,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4321},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30352,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292096,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21038,"num_bytes_recovered":33096163,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":210518016,"data_stored":210395174,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161022,"ondisk_log_size":161022,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986314,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":502616,"kb_used_data":137728,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93869224,"statfs":{"total":96636764160,"available":96122085376,"internally_reserved":0,"allocated":141033472,"data_stored":140713808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659259,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":469484,"kb_used_data":476,"kb_used_omap":4,"kb_used_meta":468987,"kb_avail":93902356,"statfs":{"total":96636764160,"available":96156012544,"internally_reserved":0,"allocated":487424,"data_stored":149231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":480243374},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182542,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":547316,"kb_used_data":68636,"kb_used_omap":4,"kb_used_meta":478651,"kb_avail":93824524,"statfs":{"total":96636764160,"available":96076312576,"internally_reserved":0,"allocated":70283264,"data_stored":69959523,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":490139306},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":69853184,"data_stored":69817054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":6762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":140607488,"data_stored":140571358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:09:37.919 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:09:38.271 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:09:38.271 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:09:38.286 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":417,"stamp":"2024-09-10T16:09:37.775203+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":29503,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292098,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21889,"num_bytes_recovered":33096182,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161025,"ondisk_log_size":161025,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1511148,"kb_used_data":206888,"kb_used_omap":12,"kb_used_meta":1304179,"kb_avail":281604372,"statfs":{"total":289910292480,"available":288362876928,"internally_reserved":0,"allocated":211853312,"data_stored":210839911,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1335479298},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1915,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1915,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001434"},"pg_stats":[{"pgid":"2.f","version":"49'20672","reported_seq":22113,"reported_epoch":53,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:36.208717+0000","last_change":"2024-09-10T16:09:31.772779+0000","last_active":"2024-09-10T16:09:36.208717+0000","last_peered":"2024-09-10T16:09:36.208717+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:09:36.208717+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3586,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20672,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":673,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3586}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20844","reported_seq":20976,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.933194+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:09:32.933194+0000","last_peered":"2024-09-10T16:09:32.933194+0000","last_clean":"2024-09-10T16:09:32.933194+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:09:32.933194+0000","last_undegraded":"2024-09-10T16:09:32.933194+0000","last_fullsized":"2024-09-10T16:09:32.933194+0000","mapping_epoch":35,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18949,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20798","reported_seq":29495,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.929401+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:09:32.929401+0000","last_peered":"2024-09-10T16:09:32.929401+0000","last_clean":"2024-09-10T16:09:32.929401+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:09:32.929401+0000","last_undegraded":"2024-09-10T16:09:32.929401+0000","last_fullsized":"2024-09-10T16:09:32.929401+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18802,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20660","reported_seq":20760,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.933021+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:09:32.933021+0000","last_peered":"2024-09-10T16:09:32.933021+0000","last_clean":"2024-09-10T16:09:32.933021+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:09:32.933021+0000","last_undegraded":"2024-09-10T16:09:32.933021+0000","last_fullsized":"2024-09-10T16:09:32.933021+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20660,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20891","reported_seq":21040,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.929877+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:09:32.929877+0000","last_peered":"2024-09-10T16:09:32.929877+0000","last_clean":"2024-09-10T16:09:32.929877+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:09:32.929877+0000","last_undegraded":"2024-09-10T16:09:32.929877+0000","last_fullsized":"2024-09-10T16:09:32.929877+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17602,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'21061","reported_seq":21218,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:32.929429+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:09:32.929429+0000","last_peered":"2024-09-10T16:09:32.929429+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:09:32.929429+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17673,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4370},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'21082","reported_seq":21239,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:32.928987+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:09:32.928987+0000","last_peered":"2024-09-10T16:09:32.928987+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:09:32.928987+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17581,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4235},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'21314","reported_seq":21463,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.928707+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:09:32.928707+0000","last_peered":"2024-09-10T16:09:32.928707+0000","last_clean":"2024-09-10T16:09:32.928707+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:09:32.928707+0000","last_undegraded":"2024-09-10T16:09:32.928707+0000","last_fullsized":"2024-09-10T16:09:32.928707+0000","mapping_epoch":34,"log_start":"45'11300","ondisk_log_start":"45'11300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18161,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.929057+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:09:32.929057+0000","last_peered":"2024-09-10T16:09:32.929057+0000","last_clean":"2024-09-10T16:09:32.929057+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:09:32.929057+0000","last_undegraded":"2024-09-10T16:09:32.929057+0000","last_fullsized":"2024-09-10T16:09:32.929057+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20457","reported_seq":20620,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:33.013394+0000","last_change":"2024-09-10T16:09:33.012840+0000","last_active":"2024-09-10T16:09:33.013394+0000","last_peered":"2024-09-10T16:09:33.013394+0000","last_clean":"2024-09-10T16:09:33.013394+0000","last_became_active":"2024-09-10T16:09:32.934056+0000","last_became_peered":"2024-09-10T16:09:32.934056+0000","last_unstale":"2024-09-10T16:09:33.013394+0000","last_undegraded":"2024-09-10T16:09:33.013394+0000","last_fullsized":"2024-09-10T16:09:33.013394+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:08:30.481235+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18276,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":4579,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":113,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.930929+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:09:32.930929+0000","last_peered":"2024-09-10T16:09:32.930929+0000","last_clean":"2024-09-10T16:09:32.930929+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:09:32.930929+0000","last_undegraded":"2024-09-10T16:09:32.930929+0000","last_fullsized":"2024-09-10T16:09:32.930929+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20910","reported_seq":29582,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.929808+0000","last_change":"2024-09-10T16:09:08.909580+0000","last_active":"2024-09-10T16:09:32.929808+0000","last_peered":"2024-09-10T16:09:32.929808+0000","last_clean":"2024-09-10T16:09:32.929808+0000","last_became_active":"2024-09-10T16:09:08.854941+0000","last_became_peered":"2024-09-10T16:09:08.854941+0000","last_unstale":"2024-09-10T16:09:32.929808+0000","last_undegraded":"2024-09-10T16:09:32.929808+0000","last_fullsized":"2024-09-10T16:09:32.929808+0000","mapping_epoch":50,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:13:14.846691+0000","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17414,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4257,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":123,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.928762+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:09:32.928762+0000","last_peered":"2024-09-10T16:09:32.928762+0000","last_clean":"2024-09-10T16:09:32.928762+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:09:32.928762+0000","last_undegraded":"2024-09-10T16:09:32.928762+0000","last_fullsized":"2024-09-10T16:09:32.928762+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20781","reported_seq":20935,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:32.931013+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:09:32.931013+0000","last_peered":"2024-09-10T16:09:32.931013+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:09:32.931013+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18461,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-4303}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":98,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.929298+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:09:32.929298+0000","last_peered":"2024-09-10T16:09:32.929298+0000","last_clean":"2024-09-10T16:09:32.929298+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:09:32.929298+0000","last_undegraded":"2024-09-10T16:09:32.929298+0000","last_fullsized":"2024-09-10T16:09:32.929298+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'21170","reported_seq":30033,"reported_epoch":53,"state":"active+recovery_wait","last_fresh":"2024-09-10T16:09:32.930698+0000","last_change":"2024-09-10T16:08:10.929169+0000","last_active":"2024-09-10T16:09:32.930698+0000","last_peered":"2024-09-10T16:09:32.930698+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T16:08:10.860438+0000","last_became_peered":"2024-09-10T16:08:10.860438+0000","last_unstale":"2024-09-10T16:09:32.930698+0000","last_undegraded":"2024-09-10T16:09:32.930698+0000","last_fullsized":"2024-09-10T16:09:32.930698+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18724,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","0","1"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20838","reported_seq":20992,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:32.930297+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:09:32.930297+0000","last_peered":"2024-09-10T16:09:32.930297+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:09:32.930297+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17469,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":88,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.929239+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:09:32.929239+0000","last_peered":"2024-09-10T16:09:32.929239+0000","last_clean":"2024-09-10T16:09:32.929239+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:09:32.929239+0000","last_undegraded":"2024-09-10T16:09:32.929239+0000","last_fullsized":"2024-09-10T16:09:32.929239+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20598","reported_seq":29183,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.929268+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:09:32.929268+0000","last_peered":"2024-09-10T16:09:32.929268+0000","last_clean":"2024-09-10T16:09:32.929268+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:09:32.929268+0000","last_undegraded":"2024-09-10T16:09:32.929268+0000","last_fullsized":"2024-09-10T16:09:32.929268+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16988,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":113,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.933167+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:09:32.933167+0000","last_peered":"2024-09-10T16:09:32.933167+0000","last_clean":"2024-09-10T16:09:32.933167+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:09:32.933167+0000","last_undegraded":"2024-09-10T16:09:32.933167+0000","last_fullsized":"2024-09-10T16:09:32.933167+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'21048","reported_seq":21209,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:32.931059+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:09:32.931059+0000","last_peered":"2024-09-10T16:09:32.931059+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:09:32.931059+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17303,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":88,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.929129+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:09:32.929129+0000","last_peered":"2024-09-10T16:09:32.929129+0000","last_clean":"2024-09-10T16:09:32.929129+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:09:32.929129+0000","last_undegraded":"2024-09-10T16:09:32.929129+0000","last_fullsized":"2024-09-10T16:09:32.929129+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":93,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.929492+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:09:32.929492+0000","last_peered":"2024-09-10T16:09:32.929492+0000","last_clean":"2024-09-10T16:09:32.929492+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:09:32.929492+0000","last_undegraded":"2024-09-10T16:09:32.929492+0000","last_fullsized":"2024-09-10T16:09:32.929492+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'21198","reported_seq":21362,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:32.931857+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:09:32.931857+0000","last_peered":"2024-09-10T16:09:32.931857+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:09:32.931857+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17361,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4321},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":29503,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292096,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21887,"num_bytes_recovered":33096163,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":210518016,"data_stored":210395174,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161022,"ondisk_log_size":161022,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986315,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":502632,"kb_used_data":137744,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93869208,"statfs":{"total":96636764160,"available":96122068992,"internally_reserved":0,"allocated":141049856,"data_stored":140719591,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659260,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":461184,"kb_used_data":492,"kb_used_omap":4,"kb_used_meta":460667,"kb_avail":93910656,"statfs":{"total":96636764160,"available":96164511744,"internally_reserved":0,"allocated":503808,"data_stored":155014,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":471723694},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182543,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":547332,"kb_used_data":68652,"kb_used_omap":4,"kb_used_meta":478651,"kb_avail":93824508,"statfs":{"total":96636764160,"available":96076296192,"internally_reserved":0,"allocated":70299648,"data_stored":69965306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":490139306},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":69853184,"data_stored":69817054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":6762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":140607488,"data_stored":140571358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:09:38.287 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:09:38.620 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:09:38.621 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:09:38.633 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":417,"stamp":"2024-09-10T16:09:37.775203+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":29503,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292098,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21889,"num_bytes_recovered":33096182,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161025,"ondisk_log_size":161025,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1511148,"kb_used_data":206888,"kb_used_omap":12,"kb_used_meta":1304179,"kb_avail":281604372,"statfs":{"total":289910292480,"available":288362876928,"internally_reserved":0,"allocated":211853312,"data_stored":210839911,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1335479298},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1915,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1915,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001434"},"pg_stats":[{"pgid":"2.f","version":"49'20672","reported_seq":22113,"reported_epoch":53,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:36.208717+0000","last_change":"2024-09-10T16:09:31.772779+0000","last_active":"2024-09-10T16:09:36.208717+0000","last_peered":"2024-09-10T16:09:36.208717+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:09:36.208717+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3586,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20672,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":673,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3586}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20844","reported_seq":20976,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.933194+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:09:32.933194+0000","last_peered":"2024-09-10T16:09:32.933194+0000","last_clean":"2024-09-10T16:09:32.933194+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:09:32.933194+0000","last_undegraded":"2024-09-10T16:09:32.933194+0000","last_fullsized":"2024-09-10T16:09:32.933194+0000","mapping_epoch":35,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18949,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20798","reported_seq":29495,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.929401+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:09:32.929401+0000","last_peered":"2024-09-10T16:09:32.929401+0000","last_clean":"2024-09-10T16:09:32.929401+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:09:32.929401+0000","last_undegraded":"2024-09-10T16:09:32.929401+0000","last_fullsized":"2024-09-10T16:09:32.929401+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18802,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20660","reported_seq":20760,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.933021+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:09:32.933021+0000","last_peered":"2024-09-10T16:09:32.933021+0000","last_clean":"2024-09-10T16:09:32.933021+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:09:32.933021+0000","last_undegraded":"2024-09-10T16:09:32.933021+0000","last_fullsized":"2024-09-10T16:09:32.933021+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20660,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20891","reported_seq":21040,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.929877+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:09:32.929877+0000","last_peered":"2024-09-10T16:09:32.929877+0000","last_clean":"2024-09-10T16:09:32.929877+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:09:32.929877+0000","last_undegraded":"2024-09-10T16:09:32.929877+0000","last_fullsized":"2024-09-10T16:09:32.929877+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17602,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'21061","reported_seq":21218,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:32.929429+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:09:32.929429+0000","last_peered":"2024-09-10T16:09:32.929429+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:09:32.929429+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17673,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4370},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'21082","reported_seq":21239,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:32.928987+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:09:32.928987+0000","last_peered":"2024-09-10T16:09:32.928987+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:09:32.928987+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17581,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4235},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'21314","reported_seq":21463,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.928707+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:09:32.928707+0000","last_peered":"2024-09-10T16:09:32.928707+0000","last_clean":"2024-09-10T16:09:32.928707+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:09:32.928707+0000","last_undegraded":"2024-09-10T16:09:32.928707+0000","last_fullsized":"2024-09-10T16:09:32.928707+0000","mapping_epoch":34,"log_start":"45'11300","ondisk_log_start":"45'11300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18161,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.929057+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:09:32.929057+0000","last_peered":"2024-09-10T16:09:32.929057+0000","last_clean":"2024-09-10T16:09:32.929057+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:09:32.929057+0000","last_undegraded":"2024-09-10T16:09:32.929057+0000","last_fullsized":"2024-09-10T16:09:32.929057+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20457","reported_seq":20620,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:33.013394+0000","last_change":"2024-09-10T16:09:33.012840+0000","last_active":"2024-09-10T16:09:33.013394+0000","last_peered":"2024-09-10T16:09:33.013394+0000","last_clean":"2024-09-10T16:09:33.013394+0000","last_became_active":"2024-09-10T16:09:32.934056+0000","last_became_peered":"2024-09-10T16:09:32.934056+0000","last_unstale":"2024-09-10T16:09:33.013394+0000","last_undegraded":"2024-09-10T16:09:33.013394+0000","last_fullsized":"2024-09-10T16:09:33.013394+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:08:30.481235+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18276,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":4579,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":113,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.930929+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:09:32.930929+0000","last_peered":"2024-09-10T16:09:32.930929+0000","last_clean":"2024-09-10T16:09:32.930929+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:09:32.930929+0000","last_undegraded":"2024-09-10T16:09:32.930929+0000","last_fullsized":"2024-09-10T16:09:32.930929+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20910","reported_seq":29582,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.929808+0000","last_change":"2024-09-10T16:09:08.909580+0000","last_active":"2024-09-10T16:09:32.929808+0000","last_peered":"2024-09-10T16:09:32.929808+0000","last_clean":"2024-09-10T16:09:32.929808+0000","last_became_active":"2024-09-10T16:09:08.854941+0000","last_became_peered":"2024-09-10T16:09:08.854941+0000","last_unstale":"2024-09-10T16:09:32.929808+0000","last_undegraded":"2024-09-10T16:09:32.929808+0000","last_fullsized":"2024-09-10T16:09:32.929808+0000","mapping_epoch":50,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:13:14.846691+0000","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17414,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4257,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":123,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.928762+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:09:32.928762+0000","last_peered":"2024-09-10T16:09:32.928762+0000","last_clean":"2024-09-10T16:09:32.928762+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:09:32.928762+0000","last_undegraded":"2024-09-10T16:09:32.928762+0000","last_fullsized":"2024-09-10T16:09:32.928762+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20781","reported_seq":20935,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:32.931013+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:09:32.931013+0000","last_peered":"2024-09-10T16:09:32.931013+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:09:32.931013+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18461,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-4303}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":98,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.929298+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:09:32.929298+0000","last_peered":"2024-09-10T16:09:32.929298+0000","last_clean":"2024-09-10T16:09:32.929298+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:09:32.929298+0000","last_undegraded":"2024-09-10T16:09:32.929298+0000","last_fullsized":"2024-09-10T16:09:32.929298+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'21170","reported_seq":30033,"reported_epoch":53,"state":"active+recovery_wait","last_fresh":"2024-09-10T16:09:32.930698+0000","last_change":"2024-09-10T16:08:10.929169+0000","last_active":"2024-09-10T16:09:32.930698+0000","last_peered":"2024-09-10T16:09:32.930698+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T16:08:10.860438+0000","last_became_peered":"2024-09-10T16:08:10.860438+0000","last_unstale":"2024-09-10T16:09:32.930698+0000","last_undegraded":"2024-09-10T16:09:32.930698+0000","last_fullsized":"2024-09-10T16:09:32.930698+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18724,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","0","1"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20838","reported_seq":20992,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:32.930297+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:09:32.930297+0000","last_peered":"2024-09-10T16:09:32.930297+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:09:32.930297+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17469,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":88,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.929239+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:09:32.929239+0000","last_peered":"2024-09-10T16:09:32.929239+0000","last_clean":"2024-09-10T16:09:32.929239+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:09:32.929239+0000","last_undegraded":"2024-09-10T16:09:32.929239+0000","last_fullsized":"2024-09-10T16:09:32.929239+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20598","reported_seq":29183,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.929268+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:09:32.929268+0000","last_peered":"2024-09-10T16:09:32.929268+0000","last_clean":"2024-09-10T16:09:32.929268+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:09:32.929268+0000","last_undegraded":"2024-09-10T16:09:32.929268+0000","last_fullsized":"2024-09-10T16:09:32.929268+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16988,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":113,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.933167+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:09:32.933167+0000","last_peered":"2024-09-10T16:09:32.933167+0000","last_clean":"2024-09-10T16:09:32.933167+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:09:32.933167+0000","last_undegraded":"2024-09-10T16:09:32.933167+0000","last_fullsized":"2024-09-10T16:09:32.933167+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'21048","reported_seq":21209,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:32.931059+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:09:32.931059+0000","last_peered":"2024-09-10T16:09:32.931059+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:09:32.931059+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17303,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":88,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.929129+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:09:32.929129+0000","last_peered":"2024-09-10T16:09:32.929129+0000","last_clean":"2024-09-10T16:09:32.929129+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:09:32.929129+0000","last_undegraded":"2024-09-10T16:09:32.929129+0000","last_fullsized":"2024-09-10T16:09:32.929129+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":93,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.929492+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:09:32.929492+0000","last_peered":"2024-09-10T16:09:32.929492+0000","last_clean":"2024-09-10T16:09:32.929492+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:09:32.929492+0000","last_undegraded":"2024-09-10T16:09:32.929492+0000","last_fullsized":"2024-09-10T16:09:32.929492+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'21198","reported_seq":21362,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:32.931857+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:09:32.931857+0000","last_peered":"2024-09-10T16:09:32.931857+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:09:32.931857+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17361,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4321},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":29503,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292096,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21887,"num_bytes_recovered":33096163,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":210518016,"data_stored":210395174,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161022,"ondisk_log_size":161022,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986315,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":502632,"kb_used_data":137744,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93869208,"statfs":{"total":96636764160,"available":96122068992,"internally_reserved":0,"allocated":141049856,"data_stored":140719591,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659260,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":461184,"kb_used_data":492,"kb_used_omap":4,"kb_used_meta":460667,"kb_avail":93910656,"statfs":{"total":96636764160,"available":96164511744,"internally_reserved":0,"allocated":503808,"data_stored":155014,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":471723694},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182543,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":547332,"kb_used_data":68652,"kb_used_omap":4,"kb_used_meta":478651,"kb_avail":93824508,"statfs":{"total":96636764160,"available":96076296192,"internally_reserved":0,"allocated":70299648,"data_stored":69965306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":490139306},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":69853184,"data_stored":69817054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":6762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":140607488,"data_stored":140571358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:09:38.634 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-10T16:09:38.635 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "49'20672", 'reported_seq': 22113, 'reported_epoch': 53, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:09:36.208717+0000', 'last_change': '2024-09-10T16:09:31.772779+0000', 'last_active': '2024-09-10T16:09:36.208717+0000', 'last_peered': '2024-09-10T16:09:36.208717+0000', 'last_clean': '2024-09-10T15:59:14.132456+0000', 'last_became_active': '2024-09-10T15:59:16.274407+0000', 'last_became_peered': '2024-09-10T15:59:16.274407+0000', 'last_unstale': '2024-09-10T16:09:36.208717+0000', 'last_undegraded': '2024-09-10T15:59:15.851279+0000', 'last_fullsized': '2024-09-10T15:59:15.774356+0000', 'mapping_epoch': 42, 'log_start': "45'10600", 'ondisk_log_start': "45'10600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'objects_scrubbed': 0, 'log_size': 10072, 'log_dups_size': 0, 'ondisk_log_size': 10072, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3586, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20672, 'num_write_kb': 41344, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 673, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3586}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:09:38.635 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-10T16:09:38.635 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "49'21061", 'reported_seq': 21218, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:09:32.929429+0000', 'last_change': '2024-09-10T15:59:16.308264+0000', 'last_active': '2024-09-10T16:09:32.929429+0000', 'last_peered': '2024-09-10T16:09:32.929429+0000', 'last_clean': '2024-09-10T15:59:14.111551+0000', 'last_became_active': '2024-09-10T15:59:16.300040+0000', 'last_became_peered': '2024-09-10T15:59:16.300040+0000', 'last_unstale': '2024-09-10T16:09:32.929429+0000', 'last_undegraded': '2024-09-10T15:59:15.878234+0000', 'last_fullsized': '2024-09-10T15:59:15.769328+0000', 'mapping_epoch': 42, 'log_start': "45'11000", 'ondisk_log_start': "45'11000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3442", 'last_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'objects_scrubbed': 3384, 'log_size': 10061, 'log_dups_size': 0, 'ondisk_log_size': 10061, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1292, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4372, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 17673, 'num_write_kb': 28580, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4370}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:09:38.635 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-10T16:09:38.635 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "49'21082", 'reported_seq': 21239, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:09:32.928987+0000', 'last_change': '2024-09-10T15:59:16.331899+0000', 'last_active': '2024-09-10T16:09:32.928987+0000', 'last_peered': '2024-09-10T16:09:32.928987+0000', 'last_clean': '2024-09-10T15:59:14.139436+0000', 'last_became_active': '2024-09-10T15:59:16.318715+0000', 'last_became_peered': '2024-09-10T15:59:16.318715+0000', 'last_unstale': '2024-09-10T16:09:32.928987+0000', 'last_undegraded': '2024-09-10T15:59:15.877524+0000', 'last_fullsized': '2024-09-10T15:59:15.769042+0000', 'mapping_epoch': 42, 'log_start': "45'11000", 'ondisk_log_start': "45'11000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3537", 'last_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'objects_scrubbed': 3497, 'log_size': 10082, 'log_dups_size': 0, 'ondisk_log_size': 10082, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1224, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4237, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 17581, 'num_write_kb': 28172, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4235}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:09:38.635 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-10T16:09:38.635 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "49'20781", 'reported_seq': 20935, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:09:32.931013+0000', 'last_change': '2024-09-10T15:59:16.284311+0000', 'last_active': '2024-09-10T16:09:32.931013+0000', 'last_peered': '2024-09-10T16:09:32.931013+0000', 'last_clean': '2024-09-10T15:59:14.116041+0000', 'last_became_active': '2024-09-10T15:59:16.274543+0000', 'last_became_peered': '2024-09-10T15:59:16.274543+0000', 'last_unstale': '2024-09-10T16:09:32.931013+0000', 'last_undegraded': '2024-09-10T15:59:15.812484+0000', 'last_fullsized': '2024-09-10T15:59:15.770885+0000', 'mapping_epoch': 42, 'log_start': "45'10700", 'ondisk_log_start': "45'10700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2356", 'last_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'objects_scrubbed': 2316, 'log_size': 10081, 'log_dups_size': 0, 'ondisk_log_size': 10081, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 783, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 18461, 'num_write_kb': 32292, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': -4303}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:09:38.635 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-10T16:09:38.635 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "49'21170", 'reported_seq': 30033, 'reported_epoch': 53, 'state': 'active+recovery_wait', 'last_fresh': '2024-09-10T16:09:32.930698+0000', 'last_change': '2024-09-10T16:08:10.929169+0000', 'last_active': '2024-09-10T16:09:32.930698+0000', 'last_peered': '2024-09-10T16:09:32.930698+0000', 'last_clean': '2024-09-10T15:59:14.110654+0000', 'last_became_active': '2024-09-10T16:08:10.860438+0000', 'last_became_peered': '2024-09-10T16:08:10.860438+0000', 'last_unstale': '2024-09-10T16:09:32.930698+0000', 'last_undegraded': '2024-09-10T16:09:32.930698+0000', 'last_fullsized': '2024-09-10T16:09:32.930698+0000', 'mapping_epoch': 48, 'log_start': "45'11100", 'ondisk_log_start': "45'11100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2486", 'last_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'objects_scrubbed': 2440, 'log_size': 10070, 'log_dups_size': 0, 'ondisk_log_size': 10070, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 792, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18724, 'num_write_kb': 32568, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 4355, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': ['2', '0', '1'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:09:38.636 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-10T16:09:38.636 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "49'20838", 'reported_seq': 20992, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:09:32.930297+0000', 'last_change': '2024-09-10T15:59:16.315641+0000', 'last_active': '2024-09-10T16:09:32.930297+0000', 'last_peered': '2024-09-10T16:09:32.930297+0000', 'last_clean': '2024-09-10T15:59:14.117001+0000', 'last_became_active': '2024-09-10T15:59:16.302245+0000', 'last_became_peered': '2024-09-10T15:59:16.302245+0000', 'last_unstale': '2024-09-10T16:09:32.930297+0000', 'last_undegraded': '2024-09-10T15:59:15.845012+0000', 'last_fullsized': '2024-09-10T15:59:15.770337+0000', 'mapping_epoch': 42, 'log_start': "45'10800", 'ondisk_log_start': "45'10800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3404", 'last_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'objects_scrubbed': 3367, 'log_size': 10038, 'log_dups_size': 0, 'ondisk_log_size': 10038, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 983, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17469, 'num_write_kb': 28204, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4304}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:09:38.636 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-10T16:09:38.636 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "49'21048", 'reported_seq': 21209, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:09:32.931059+0000', 'last_change': '2024-09-10T15:59:16.311291+0000', 'last_active': '2024-09-10T16:09:32.931059+0000', 'last_peered': '2024-09-10T16:09:32.931059+0000', 'last_clean': '2024-09-10T15:59:14.116363+0000', 'last_became_active': '2024-09-10T15:59:16.301244+0000', 'last_became_peered': '2024-09-10T15:59:16.301244+0000', 'last_unstale': '2024-09-10T16:09:32.931059+0000', 'last_undegraded': '2024-09-10T15:59:15.883434+0000', 'last_fullsized': '2024-09-10T15:59:15.772756+0000', 'mapping_epoch': 42, 'log_start': "45'11000", 'ondisk_log_start': "45'11000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3789", 'last_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'objects_scrubbed': 3741, 'log_size': 10048, 'log_dups_size': 0, 'ondisk_log_size': 10048, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1440, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4373, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17303, 'num_write_kb': 27124, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4371}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:09:38.636 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-10T16:09:38.636 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "49'21198", 'reported_seq': 21362, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:09:32.931857+0000', 'last_change': '2024-09-10T15:59:16.313409+0000', 'last_active': '2024-09-10T16:09:32.931857+0000', 'last_peered': '2024-09-10T16:09:32.931857+0000', 'last_clean': '2024-09-10T15:59:14.129780+0000', 'last_became_active': '2024-09-10T15:59:16.300932+0000', 'last_became_peered': '2024-09-10T15:59:16.300932+0000', 'last_unstale': '2024-09-10T16:09:32.931857+0000', 'last_undegraded': '2024-09-10T15:59:15.887823+0000', 'last_fullsized': '2024-09-10T15:59:15.768513+0000', 'mapping_epoch': 42, 'log_start': "45'11100", 'ondisk_log_start': "45'11100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3880", 'last_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'objects_scrubbed': 3827, 'log_size': 10098, 'log_dups_size': 0, 'ondisk_log_size': 10098, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1397, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4325, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17361, 'num_write_kb': 27072, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4321}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:09:38.636 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T16:09:39.106 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:09:39.106 INFO:teuthology.orchestra.run.smithi115.stdout:{"fsid":"7effee29-df05-4351-9741-ee203564ed81","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 29503/24 objects degraded (122929.167%), 7 pgs degraded, 7 pgs undersized","count":14},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":743,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":53,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983954,"num_in_osds":3,"osd_in_since":1725983947,"num_remapped_pgs":7},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":16},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":6},{"state_name":"active+recovery_wait","count":1},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":12,"data_bytes":4849,"bytes_used":1547415552,"bytes_avail":288362876928,"bytes_total":289910292480,"degraded_objects":29503,"degraded_total":24,"degraded_ratio":1229.2916666666667,"recovering_objects_per_sec":239,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1915,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-10T15:57:14:967120+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T16:00:13.560776+0000","services":{}},"progress_events":{}} 2024-09-10T16:09:39.122 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-10T16:09:39.123 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:09:39.462 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:09:39.462 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:09:39.478 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":417,"stamp":"2024-09-10T16:09:37.775203+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":29503,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292098,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21889,"num_bytes_recovered":33096182,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161025,"ondisk_log_size":161025,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1511148,"kb_used_data":206888,"kb_used_omap":12,"kb_used_meta":1304179,"kb_avail":281604372,"statfs":{"total":289910292480,"available":288362876928,"internally_reserved":0,"allocated":211853312,"data_stored":210839911,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1335479298},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1915,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1915,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001434"},"pg_stats":[{"pgid":"2.f","version":"49'20672","reported_seq":22113,"reported_epoch":53,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:36.208717+0000","last_change":"2024-09-10T16:09:31.772779+0000","last_active":"2024-09-10T16:09:36.208717+0000","last_peered":"2024-09-10T16:09:36.208717+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:09:36.208717+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3586,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20672,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":673,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3586}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20844","reported_seq":20976,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.933194+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:09:32.933194+0000","last_peered":"2024-09-10T16:09:32.933194+0000","last_clean":"2024-09-10T16:09:32.933194+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:09:32.933194+0000","last_undegraded":"2024-09-10T16:09:32.933194+0000","last_fullsized":"2024-09-10T16:09:32.933194+0000","mapping_epoch":35,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18949,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20798","reported_seq":29495,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.929401+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:09:32.929401+0000","last_peered":"2024-09-10T16:09:32.929401+0000","last_clean":"2024-09-10T16:09:32.929401+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:09:32.929401+0000","last_undegraded":"2024-09-10T16:09:32.929401+0000","last_fullsized":"2024-09-10T16:09:32.929401+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18802,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20660","reported_seq":20760,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.933021+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:09:32.933021+0000","last_peered":"2024-09-10T16:09:32.933021+0000","last_clean":"2024-09-10T16:09:32.933021+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:09:32.933021+0000","last_undegraded":"2024-09-10T16:09:32.933021+0000","last_fullsized":"2024-09-10T16:09:32.933021+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20660,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20891","reported_seq":21040,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.929877+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:09:32.929877+0000","last_peered":"2024-09-10T16:09:32.929877+0000","last_clean":"2024-09-10T16:09:32.929877+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:09:32.929877+0000","last_undegraded":"2024-09-10T16:09:32.929877+0000","last_fullsized":"2024-09-10T16:09:32.929877+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17602,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'21061","reported_seq":21218,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:32.929429+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:09:32.929429+0000","last_peered":"2024-09-10T16:09:32.929429+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:09:32.929429+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17673,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4370},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'21082","reported_seq":21239,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:32.928987+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:09:32.928987+0000","last_peered":"2024-09-10T16:09:32.928987+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:09:32.928987+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17581,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4235},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'21314","reported_seq":21463,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.928707+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:09:32.928707+0000","last_peered":"2024-09-10T16:09:32.928707+0000","last_clean":"2024-09-10T16:09:32.928707+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:09:32.928707+0000","last_undegraded":"2024-09-10T16:09:32.928707+0000","last_fullsized":"2024-09-10T16:09:32.928707+0000","mapping_epoch":34,"log_start":"45'11300","ondisk_log_start":"45'11300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18161,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.929057+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:09:32.929057+0000","last_peered":"2024-09-10T16:09:32.929057+0000","last_clean":"2024-09-10T16:09:32.929057+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:09:32.929057+0000","last_undegraded":"2024-09-10T16:09:32.929057+0000","last_fullsized":"2024-09-10T16:09:32.929057+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20457","reported_seq":20620,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:33.013394+0000","last_change":"2024-09-10T16:09:33.012840+0000","last_active":"2024-09-10T16:09:33.013394+0000","last_peered":"2024-09-10T16:09:33.013394+0000","last_clean":"2024-09-10T16:09:33.013394+0000","last_became_active":"2024-09-10T16:09:32.934056+0000","last_became_peered":"2024-09-10T16:09:32.934056+0000","last_unstale":"2024-09-10T16:09:33.013394+0000","last_undegraded":"2024-09-10T16:09:33.013394+0000","last_fullsized":"2024-09-10T16:09:33.013394+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:08:30.481235+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18276,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":4579,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":113,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.930929+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:09:32.930929+0000","last_peered":"2024-09-10T16:09:32.930929+0000","last_clean":"2024-09-10T16:09:32.930929+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:09:32.930929+0000","last_undegraded":"2024-09-10T16:09:32.930929+0000","last_fullsized":"2024-09-10T16:09:32.930929+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20910","reported_seq":29582,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.929808+0000","last_change":"2024-09-10T16:09:08.909580+0000","last_active":"2024-09-10T16:09:32.929808+0000","last_peered":"2024-09-10T16:09:32.929808+0000","last_clean":"2024-09-10T16:09:32.929808+0000","last_became_active":"2024-09-10T16:09:08.854941+0000","last_became_peered":"2024-09-10T16:09:08.854941+0000","last_unstale":"2024-09-10T16:09:32.929808+0000","last_undegraded":"2024-09-10T16:09:32.929808+0000","last_fullsized":"2024-09-10T16:09:32.929808+0000","mapping_epoch":50,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:13:14.846691+0000","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17414,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4257,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":123,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.928762+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:09:32.928762+0000","last_peered":"2024-09-10T16:09:32.928762+0000","last_clean":"2024-09-10T16:09:32.928762+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:09:32.928762+0000","last_undegraded":"2024-09-10T16:09:32.928762+0000","last_fullsized":"2024-09-10T16:09:32.928762+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20781","reported_seq":20935,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:32.931013+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:09:32.931013+0000","last_peered":"2024-09-10T16:09:32.931013+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:09:32.931013+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18461,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-4303}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":98,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.929298+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:09:32.929298+0000","last_peered":"2024-09-10T16:09:32.929298+0000","last_clean":"2024-09-10T16:09:32.929298+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:09:32.929298+0000","last_undegraded":"2024-09-10T16:09:32.929298+0000","last_fullsized":"2024-09-10T16:09:32.929298+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'21170","reported_seq":30033,"reported_epoch":53,"state":"active+recovery_wait","last_fresh":"2024-09-10T16:09:32.930698+0000","last_change":"2024-09-10T16:08:10.929169+0000","last_active":"2024-09-10T16:09:32.930698+0000","last_peered":"2024-09-10T16:09:32.930698+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T16:08:10.860438+0000","last_became_peered":"2024-09-10T16:08:10.860438+0000","last_unstale":"2024-09-10T16:09:32.930698+0000","last_undegraded":"2024-09-10T16:09:32.930698+0000","last_fullsized":"2024-09-10T16:09:32.930698+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18724,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","0","1"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20838","reported_seq":20992,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:32.930297+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:09:32.930297+0000","last_peered":"2024-09-10T16:09:32.930297+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:09:32.930297+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17469,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":88,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.929239+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:09:32.929239+0000","last_peered":"2024-09-10T16:09:32.929239+0000","last_clean":"2024-09-10T16:09:32.929239+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:09:32.929239+0000","last_undegraded":"2024-09-10T16:09:32.929239+0000","last_fullsized":"2024-09-10T16:09:32.929239+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20598","reported_seq":29183,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.929268+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:09:32.929268+0000","last_peered":"2024-09-10T16:09:32.929268+0000","last_clean":"2024-09-10T16:09:32.929268+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:09:32.929268+0000","last_undegraded":"2024-09-10T16:09:32.929268+0000","last_fullsized":"2024-09-10T16:09:32.929268+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16988,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":113,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.933167+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:09:32.933167+0000","last_peered":"2024-09-10T16:09:32.933167+0000","last_clean":"2024-09-10T16:09:32.933167+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:09:32.933167+0000","last_undegraded":"2024-09-10T16:09:32.933167+0000","last_fullsized":"2024-09-10T16:09:32.933167+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'21048","reported_seq":21209,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:32.931059+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:09:32.931059+0000","last_peered":"2024-09-10T16:09:32.931059+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:09:32.931059+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17303,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":88,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.929129+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:09:32.929129+0000","last_peered":"2024-09-10T16:09:32.929129+0000","last_clean":"2024-09-10T16:09:32.929129+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:09:32.929129+0000","last_undegraded":"2024-09-10T16:09:32.929129+0000","last_fullsized":"2024-09-10T16:09:32.929129+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":93,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.929492+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:09:32.929492+0000","last_peered":"2024-09-10T16:09:32.929492+0000","last_clean":"2024-09-10T16:09:32.929492+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:09:32.929492+0000","last_undegraded":"2024-09-10T16:09:32.929492+0000","last_fullsized":"2024-09-10T16:09:32.929492+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'21198","reported_seq":21362,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:32.931857+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:09:32.931857+0000","last_peered":"2024-09-10T16:09:32.931857+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:09:32.931857+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17361,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4321},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":29503,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292096,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21887,"num_bytes_recovered":33096163,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":210518016,"data_stored":210395174,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161022,"ondisk_log_size":161022,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986315,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":502632,"kb_used_data":137744,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93869208,"statfs":{"total":96636764160,"available":96122068992,"internally_reserved":0,"allocated":141049856,"data_stored":140719591,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659260,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":461184,"kb_used_data":492,"kb_used_omap":4,"kb_used_meta":460667,"kb_avail":93910656,"statfs":{"total":96636764160,"available":96164511744,"internally_reserved":0,"allocated":503808,"data_stored":155014,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":471723694},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182543,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":547332,"kb_used_data":68652,"kb_used_omap":4,"kb_used_meta":478651,"kb_avail":93824508,"statfs":{"total":96636764160,"available":96076296192,"internally_reserved":0,"allocated":70299648,"data_stored":69965306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":490139306},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":69853184,"data_stored":69817054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":6762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":140607488,"data_stored":140571358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:09:42.481 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:09:42.800 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:09:42.801 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:09:42.816 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":419,"stamp":"2024-09-10T16:09:41.776055+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":28567,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292098,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22825,"num_bytes_recovered":33096182,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161025,"ondisk_log_size":161025,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1519464,"kb_used_data":206888,"kb_used_omap":12,"kb_used_meta":1312499,"kb_avail":281596056,"statfs":{"total":289910292480,"available":288354361344,"internally_reserved":0,"allocated":211853312,"data_stored":210839911,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1343998978},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1785,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1785,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.848730"},"pg_stats":[{"pgid":"2.f","version":"49'20672","reported_seq":23985,"reported_epoch":53,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:41.207815+0000","last_change":"2024-09-10T16:09:31.772779+0000","last_active":"2024-09-10T16:09:41.207815+0000","last_peered":"2024-09-10T16:09:41.207815+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:09:41.207815+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2650,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20672,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1609,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2650}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20844","reported_seq":20976,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.933194+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:09:32.933194+0000","last_peered":"2024-09-10T16:09:32.933194+0000","last_clean":"2024-09-10T16:09:32.933194+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:09:32.933194+0000","last_undegraded":"2024-09-10T16:09:32.933194+0000","last_fullsized":"2024-09-10T16:09:32.933194+0000","mapping_epoch":35,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18949,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20798","reported_seq":29495,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.929401+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:09:32.929401+0000","last_peered":"2024-09-10T16:09:32.929401+0000","last_clean":"2024-09-10T16:09:32.929401+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:09:32.929401+0000","last_undegraded":"2024-09-10T16:09:32.929401+0000","last_fullsized":"2024-09-10T16:09:32.929401+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18802,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20660","reported_seq":20760,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.933021+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:09:32.933021+0000","last_peered":"2024-09-10T16:09:32.933021+0000","last_clean":"2024-09-10T16:09:32.933021+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:09:32.933021+0000","last_undegraded":"2024-09-10T16:09:32.933021+0000","last_fullsized":"2024-09-10T16:09:32.933021+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20660,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20891","reported_seq":21040,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.929877+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:09:32.929877+0000","last_peered":"2024-09-10T16:09:32.929877+0000","last_clean":"2024-09-10T16:09:32.929877+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:09:32.929877+0000","last_undegraded":"2024-09-10T16:09:32.929877+0000","last_fullsized":"2024-09-10T16:09:32.929877+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17602,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'21061","reported_seq":21218,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:32.929429+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:09:32.929429+0000","last_peered":"2024-09-10T16:09:32.929429+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:09:32.929429+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17673,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4370},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'21082","reported_seq":21239,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:32.928987+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:09:32.928987+0000","last_peered":"2024-09-10T16:09:32.928987+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:09:32.928987+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17581,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4235},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'21314","reported_seq":21463,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.928707+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:09:32.928707+0000","last_peered":"2024-09-10T16:09:32.928707+0000","last_clean":"2024-09-10T16:09:32.928707+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:09:32.928707+0000","last_undegraded":"2024-09-10T16:09:32.928707+0000","last_fullsized":"2024-09-10T16:09:32.928707+0000","mapping_epoch":34,"log_start":"45'11300","ondisk_log_start":"45'11300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18161,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.929057+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:09:32.929057+0000","last_peered":"2024-09-10T16:09:32.929057+0000","last_clean":"2024-09-10T16:09:32.929057+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:09:32.929057+0000","last_undegraded":"2024-09-10T16:09:32.929057+0000","last_fullsized":"2024-09-10T16:09:32.929057+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20457","reported_seq":20620,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:33.013394+0000","last_change":"2024-09-10T16:09:33.012840+0000","last_active":"2024-09-10T16:09:33.013394+0000","last_peered":"2024-09-10T16:09:33.013394+0000","last_clean":"2024-09-10T16:09:33.013394+0000","last_became_active":"2024-09-10T16:09:32.934056+0000","last_became_peered":"2024-09-10T16:09:32.934056+0000","last_unstale":"2024-09-10T16:09:33.013394+0000","last_undegraded":"2024-09-10T16:09:33.013394+0000","last_fullsized":"2024-09-10T16:09:33.013394+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:08:30.481235+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18276,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":4579,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":113,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.930929+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:09:32.930929+0000","last_peered":"2024-09-10T16:09:32.930929+0000","last_clean":"2024-09-10T16:09:32.930929+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:09:32.930929+0000","last_undegraded":"2024-09-10T16:09:32.930929+0000","last_fullsized":"2024-09-10T16:09:32.930929+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20910","reported_seq":29582,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.929808+0000","last_change":"2024-09-10T16:09:08.909580+0000","last_active":"2024-09-10T16:09:32.929808+0000","last_peered":"2024-09-10T16:09:32.929808+0000","last_clean":"2024-09-10T16:09:32.929808+0000","last_became_active":"2024-09-10T16:09:08.854941+0000","last_became_peered":"2024-09-10T16:09:08.854941+0000","last_unstale":"2024-09-10T16:09:32.929808+0000","last_undegraded":"2024-09-10T16:09:32.929808+0000","last_fullsized":"2024-09-10T16:09:32.929808+0000","mapping_epoch":50,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:13:14.846691+0000","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17414,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4257,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":123,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.928762+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:09:32.928762+0000","last_peered":"2024-09-10T16:09:32.928762+0000","last_clean":"2024-09-10T16:09:32.928762+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:09:32.928762+0000","last_undegraded":"2024-09-10T16:09:32.928762+0000","last_fullsized":"2024-09-10T16:09:32.928762+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20781","reported_seq":20935,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:32.931013+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:09:32.931013+0000","last_peered":"2024-09-10T16:09:32.931013+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:09:32.931013+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18461,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-4303}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":98,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.929298+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:09:32.929298+0000","last_peered":"2024-09-10T16:09:32.929298+0000","last_clean":"2024-09-10T16:09:32.929298+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:09:32.929298+0000","last_undegraded":"2024-09-10T16:09:32.929298+0000","last_fullsized":"2024-09-10T16:09:32.929298+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'21170","reported_seq":30033,"reported_epoch":53,"state":"active+recovery_wait","last_fresh":"2024-09-10T16:09:32.930698+0000","last_change":"2024-09-10T16:08:10.929169+0000","last_active":"2024-09-10T16:09:32.930698+0000","last_peered":"2024-09-10T16:09:32.930698+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T16:08:10.860438+0000","last_became_peered":"2024-09-10T16:08:10.860438+0000","last_unstale":"2024-09-10T16:09:32.930698+0000","last_undegraded":"2024-09-10T16:09:32.930698+0000","last_fullsized":"2024-09-10T16:09:32.930698+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18724,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","0","1"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20838","reported_seq":20992,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:32.930297+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:09:32.930297+0000","last_peered":"2024-09-10T16:09:32.930297+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:09:32.930297+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17469,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":88,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.929239+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:09:32.929239+0000","last_peered":"2024-09-10T16:09:32.929239+0000","last_clean":"2024-09-10T16:09:32.929239+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:09:32.929239+0000","last_undegraded":"2024-09-10T16:09:32.929239+0000","last_fullsized":"2024-09-10T16:09:32.929239+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20598","reported_seq":29183,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.929268+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:09:32.929268+0000","last_peered":"2024-09-10T16:09:32.929268+0000","last_clean":"2024-09-10T16:09:32.929268+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:09:32.929268+0000","last_undegraded":"2024-09-10T16:09:32.929268+0000","last_fullsized":"2024-09-10T16:09:32.929268+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16988,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":113,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.933167+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:09:32.933167+0000","last_peered":"2024-09-10T16:09:32.933167+0000","last_clean":"2024-09-10T16:09:32.933167+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:09:32.933167+0000","last_undegraded":"2024-09-10T16:09:32.933167+0000","last_fullsized":"2024-09-10T16:09:32.933167+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'21048","reported_seq":21209,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:32.931059+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:09:32.931059+0000","last_peered":"2024-09-10T16:09:32.931059+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:09:32.931059+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17303,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":88,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.929129+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:09:32.929129+0000","last_peered":"2024-09-10T16:09:32.929129+0000","last_clean":"2024-09-10T16:09:32.929129+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:09:32.929129+0000","last_undegraded":"2024-09-10T16:09:32.929129+0000","last_fullsized":"2024-09-10T16:09:32.929129+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":93,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.929492+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:09:32.929492+0000","last_peered":"2024-09-10T16:09:32.929492+0000","last_clean":"2024-09-10T16:09:32.929492+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:09:32.929492+0000","last_undegraded":"2024-09-10T16:09:32.929492+0000","last_fullsized":"2024-09-10T16:09:32.929492+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'21198","reported_seq":21362,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:32.931857+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:09:32.931857+0000","last_peered":"2024-09-10T16:09:32.931857+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:09:32.931857+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17361,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4321},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":28567,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292096,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22823,"num_bytes_recovered":33096163,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":210518016,"data_stored":210395174,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161022,"ondisk_log_size":161022,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986316,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":502632,"kb_used_data":137744,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93869208,"statfs":{"total":96636764160,"available":96122068992,"internally_reserved":0,"allocated":141049856,"data_stored":140719591,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659261,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":469500,"kb_used_data":492,"kb_used_omap":4,"kb_used_meta":468987,"kb_avail":93902340,"statfs":{"total":96636764160,"available":96155996160,"internally_reserved":0,"allocated":503808,"data_stored":155014,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":480243374},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182544,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":547332,"kb_used_data":68652,"kb_used_omap":4,"kb_used_meta":478651,"kb_avail":93824508,"statfs":{"total":96636764160,"available":96076296192,"internally_reserved":0,"allocated":70299648,"data_stored":69965306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":490139306},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":69853184,"data_stored":69817054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":6762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":140607488,"data_stored":140571358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:09:42.817 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:09:43.137 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:09:43.138 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:09:43.149 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":419,"stamp":"2024-09-10T16:09:41.776055+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":28567,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292098,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22825,"num_bytes_recovered":33096182,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161025,"ondisk_log_size":161025,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1519464,"kb_used_data":206888,"kb_used_omap":12,"kb_used_meta":1312499,"kb_avail":281596056,"statfs":{"total":289910292480,"available":288354361344,"internally_reserved":0,"allocated":211853312,"data_stored":210839911,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1343998978},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1785,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1785,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.848730"},"pg_stats":[{"pgid":"2.f","version":"49'20672","reported_seq":23985,"reported_epoch":53,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:41.207815+0000","last_change":"2024-09-10T16:09:31.772779+0000","last_active":"2024-09-10T16:09:41.207815+0000","last_peered":"2024-09-10T16:09:41.207815+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:09:41.207815+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2650,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20672,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1609,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2650}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20844","reported_seq":20976,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.933194+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:09:32.933194+0000","last_peered":"2024-09-10T16:09:32.933194+0000","last_clean":"2024-09-10T16:09:32.933194+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:09:32.933194+0000","last_undegraded":"2024-09-10T16:09:32.933194+0000","last_fullsized":"2024-09-10T16:09:32.933194+0000","mapping_epoch":35,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18949,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20798","reported_seq":29495,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.929401+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:09:32.929401+0000","last_peered":"2024-09-10T16:09:32.929401+0000","last_clean":"2024-09-10T16:09:32.929401+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:09:32.929401+0000","last_undegraded":"2024-09-10T16:09:32.929401+0000","last_fullsized":"2024-09-10T16:09:32.929401+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18802,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20660","reported_seq":20760,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.933021+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:09:32.933021+0000","last_peered":"2024-09-10T16:09:32.933021+0000","last_clean":"2024-09-10T16:09:32.933021+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:09:32.933021+0000","last_undegraded":"2024-09-10T16:09:32.933021+0000","last_fullsized":"2024-09-10T16:09:32.933021+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20660,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20891","reported_seq":21040,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.929877+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:09:32.929877+0000","last_peered":"2024-09-10T16:09:32.929877+0000","last_clean":"2024-09-10T16:09:32.929877+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:09:32.929877+0000","last_undegraded":"2024-09-10T16:09:32.929877+0000","last_fullsized":"2024-09-10T16:09:32.929877+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17602,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'21061","reported_seq":21218,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:32.929429+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:09:32.929429+0000","last_peered":"2024-09-10T16:09:32.929429+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:09:32.929429+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17673,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4370},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'21082","reported_seq":21239,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:32.928987+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:09:32.928987+0000","last_peered":"2024-09-10T16:09:32.928987+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:09:32.928987+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17581,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4235},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'21314","reported_seq":21463,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.928707+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:09:32.928707+0000","last_peered":"2024-09-10T16:09:32.928707+0000","last_clean":"2024-09-10T16:09:32.928707+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:09:32.928707+0000","last_undegraded":"2024-09-10T16:09:32.928707+0000","last_fullsized":"2024-09-10T16:09:32.928707+0000","mapping_epoch":34,"log_start":"45'11300","ondisk_log_start":"45'11300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18161,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.929057+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:09:32.929057+0000","last_peered":"2024-09-10T16:09:32.929057+0000","last_clean":"2024-09-10T16:09:32.929057+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:09:32.929057+0000","last_undegraded":"2024-09-10T16:09:32.929057+0000","last_fullsized":"2024-09-10T16:09:32.929057+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20457","reported_seq":20620,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:33.013394+0000","last_change":"2024-09-10T16:09:33.012840+0000","last_active":"2024-09-10T16:09:33.013394+0000","last_peered":"2024-09-10T16:09:33.013394+0000","last_clean":"2024-09-10T16:09:33.013394+0000","last_became_active":"2024-09-10T16:09:32.934056+0000","last_became_peered":"2024-09-10T16:09:32.934056+0000","last_unstale":"2024-09-10T16:09:33.013394+0000","last_undegraded":"2024-09-10T16:09:33.013394+0000","last_fullsized":"2024-09-10T16:09:33.013394+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:08:30.481235+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18276,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":4579,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":113,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.930929+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:09:32.930929+0000","last_peered":"2024-09-10T16:09:32.930929+0000","last_clean":"2024-09-10T16:09:32.930929+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:09:32.930929+0000","last_undegraded":"2024-09-10T16:09:32.930929+0000","last_fullsized":"2024-09-10T16:09:32.930929+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20910","reported_seq":29582,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.929808+0000","last_change":"2024-09-10T16:09:08.909580+0000","last_active":"2024-09-10T16:09:32.929808+0000","last_peered":"2024-09-10T16:09:32.929808+0000","last_clean":"2024-09-10T16:09:32.929808+0000","last_became_active":"2024-09-10T16:09:08.854941+0000","last_became_peered":"2024-09-10T16:09:08.854941+0000","last_unstale":"2024-09-10T16:09:32.929808+0000","last_undegraded":"2024-09-10T16:09:32.929808+0000","last_fullsized":"2024-09-10T16:09:32.929808+0000","mapping_epoch":50,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:13:14.846691+0000","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17414,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4257,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":123,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.928762+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:09:32.928762+0000","last_peered":"2024-09-10T16:09:32.928762+0000","last_clean":"2024-09-10T16:09:32.928762+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:09:32.928762+0000","last_undegraded":"2024-09-10T16:09:32.928762+0000","last_fullsized":"2024-09-10T16:09:32.928762+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20781","reported_seq":20935,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:32.931013+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:09:32.931013+0000","last_peered":"2024-09-10T16:09:32.931013+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:09:32.931013+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18461,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-4303}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":98,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.929298+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:09:32.929298+0000","last_peered":"2024-09-10T16:09:32.929298+0000","last_clean":"2024-09-10T16:09:32.929298+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:09:32.929298+0000","last_undegraded":"2024-09-10T16:09:32.929298+0000","last_fullsized":"2024-09-10T16:09:32.929298+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'21170","reported_seq":30033,"reported_epoch":53,"state":"active+recovery_wait","last_fresh":"2024-09-10T16:09:32.930698+0000","last_change":"2024-09-10T16:08:10.929169+0000","last_active":"2024-09-10T16:09:32.930698+0000","last_peered":"2024-09-10T16:09:32.930698+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T16:08:10.860438+0000","last_became_peered":"2024-09-10T16:08:10.860438+0000","last_unstale":"2024-09-10T16:09:32.930698+0000","last_undegraded":"2024-09-10T16:09:32.930698+0000","last_fullsized":"2024-09-10T16:09:32.930698+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18724,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","0","1"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20838","reported_seq":20992,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:32.930297+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:09:32.930297+0000","last_peered":"2024-09-10T16:09:32.930297+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:09:32.930297+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17469,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":88,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.929239+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:09:32.929239+0000","last_peered":"2024-09-10T16:09:32.929239+0000","last_clean":"2024-09-10T16:09:32.929239+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:09:32.929239+0000","last_undegraded":"2024-09-10T16:09:32.929239+0000","last_fullsized":"2024-09-10T16:09:32.929239+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20598","reported_seq":29183,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.929268+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:09:32.929268+0000","last_peered":"2024-09-10T16:09:32.929268+0000","last_clean":"2024-09-10T16:09:32.929268+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:09:32.929268+0000","last_undegraded":"2024-09-10T16:09:32.929268+0000","last_fullsized":"2024-09-10T16:09:32.929268+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16988,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":113,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.933167+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:09:32.933167+0000","last_peered":"2024-09-10T16:09:32.933167+0000","last_clean":"2024-09-10T16:09:32.933167+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:09:32.933167+0000","last_undegraded":"2024-09-10T16:09:32.933167+0000","last_fullsized":"2024-09-10T16:09:32.933167+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'21048","reported_seq":21209,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:32.931059+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:09:32.931059+0000","last_peered":"2024-09-10T16:09:32.931059+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:09:32.931059+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17303,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":88,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.929129+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:09:32.929129+0000","last_peered":"2024-09-10T16:09:32.929129+0000","last_clean":"2024-09-10T16:09:32.929129+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:09:32.929129+0000","last_undegraded":"2024-09-10T16:09:32.929129+0000","last_fullsized":"2024-09-10T16:09:32.929129+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":93,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.929492+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:09:32.929492+0000","last_peered":"2024-09-10T16:09:32.929492+0000","last_clean":"2024-09-10T16:09:32.929492+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:09:32.929492+0000","last_undegraded":"2024-09-10T16:09:32.929492+0000","last_fullsized":"2024-09-10T16:09:32.929492+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'21198","reported_seq":21362,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:32.931857+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:09:32.931857+0000","last_peered":"2024-09-10T16:09:32.931857+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:09:32.931857+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17361,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4321},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":28567,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292096,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22823,"num_bytes_recovered":33096163,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":210518016,"data_stored":210395174,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161022,"ondisk_log_size":161022,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986316,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":502632,"kb_used_data":137744,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93869208,"statfs":{"total":96636764160,"available":96122068992,"internally_reserved":0,"allocated":141049856,"data_stored":140719591,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659261,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":469500,"kb_used_data":492,"kb_used_omap":4,"kb_used_meta":468987,"kb_avail":93902340,"statfs":{"total":96636764160,"available":96155996160,"internally_reserved":0,"allocated":503808,"data_stored":155014,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":480243374},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182544,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":547332,"kb_used_data":68652,"kb_used_omap":4,"kb_used_meta":478651,"kb_avail":93824508,"statfs":{"total":96636764160,"available":96076296192,"internally_reserved":0,"allocated":70299648,"data_stored":69965306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":490139306},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":69853184,"data_stored":69817054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":6762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":140607488,"data_stored":140571358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:09:43.150 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-10T16:09:43.151 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "49'20672", 'reported_seq': 23985, 'reported_epoch': 53, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:09:41.207815+0000', 'last_change': '2024-09-10T16:09:31.772779+0000', 'last_active': '2024-09-10T16:09:41.207815+0000', 'last_peered': '2024-09-10T16:09:41.207815+0000', 'last_clean': '2024-09-10T15:59:14.132456+0000', 'last_became_active': '2024-09-10T15:59:16.274407+0000', 'last_became_peered': '2024-09-10T15:59:16.274407+0000', 'last_unstale': '2024-09-10T16:09:41.207815+0000', 'last_undegraded': '2024-09-10T15:59:15.851279+0000', 'last_fullsized': '2024-09-10T15:59:15.774356+0000', 'mapping_epoch': 42, 'log_start': "45'10600", 'ondisk_log_start': "45'10600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'objects_scrubbed': 0, 'log_size': 10072, 'log_dups_size': 0, 'ondisk_log_size': 10072, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2650, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20672, 'num_write_kb': 41344, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1609, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2650}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:09:43.151 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-10T16:09:43.151 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "49'21061", 'reported_seq': 21218, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:09:32.929429+0000', 'last_change': '2024-09-10T15:59:16.308264+0000', 'last_active': '2024-09-10T16:09:32.929429+0000', 'last_peered': '2024-09-10T16:09:32.929429+0000', 'last_clean': '2024-09-10T15:59:14.111551+0000', 'last_became_active': '2024-09-10T15:59:16.300040+0000', 'last_became_peered': '2024-09-10T15:59:16.300040+0000', 'last_unstale': '2024-09-10T16:09:32.929429+0000', 'last_undegraded': '2024-09-10T15:59:15.878234+0000', 'last_fullsized': '2024-09-10T15:59:15.769328+0000', 'mapping_epoch': 42, 'log_start': "45'11000", 'ondisk_log_start': "45'11000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3442", 'last_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'objects_scrubbed': 3384, 'log_size': 10061, 'log_dups_size': 0, 'ondisk_log_size': 10061, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1292, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4372, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 17673, 'num_write_kb': 28580, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4370}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:09:43.151 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-10T16:09:43.151 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "49'21082", 'reported_seq': 21239, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:09:32.928987+0000', 'last_change': '2024-09-10T15:59:16.331899+0000', 'last_active': '2024-09-10T16:09:32.928987+0000', 'last_peered': '2024-09-10T16:09:32.928987+0000', 'last_clean': '2024-09-10T15:59:14.139436+0000', 'last_became_active': '2024-09-10T15:59:16.318715+0000', 'last_became_peered': '2024-09-10T15:59:16.318715+0000', 'last_unstale': '2024-09-10T16:09:32.928987+0000', 'last_undegraded': '2024-09-10T15:59:15.877524+0000', 'last_fullsized': '2024-09-10T15:59:15.769042+0000', 'mapping_epoch': 42, 'log_start': "45'11000", 'ondisk_log_start': "45'11000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3537", 'last_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'objects_scrubbed': 3497, 'log_size': 10082, 'log_dups_size': 0, 'ondisk_log_size': 10082, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1224, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4237, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 17581, 'num_write_kb': 28172, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4235}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:09:43.151 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-10T16:09:43.151 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "49'20781", 'reported_seq': 20935, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:09:32.931013+0000', 'last_change': '2024-09-10T15:59:16.284311+0000', 'last_active': '2024-09-10T16:09:32.931013+0000', 'last_peered': '2024-09-10T16:09:32.931013+0000', 'last_clean': '2024-09-10T15:59:14.116041+0000', 'last_became_active': '2024-09-10T15:59:16.274543+0000', 'last_became_peered': '2024-09-10T15:59:16.274543+0000', 'last_unstale': '2024-09-10T16:09:32.931013+0000', 'last_undegraded': '2024-09-10T15:59:15.812484+0000', 'last_fullsized': '2024-09-10T15:59:15.770885+0000', 'mapping_epoch': 42, 'log_start': "45'10700", 'ondisk_log_start': "45'10700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2356", 'last_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'objects_scrubbed': 2316, 'log_size': 10081, 'log_dups_size': 0, 'ondisk_log_size': 10081, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 783, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 18461, 'num_write_kb': 32292, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': -4303}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:09:43.151 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-10T16:09:43.152 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "49'21170", 'reported_seq': 30033, 'reported_epoch': 53, 'state': 'active+recovery_wait', 'last_fresh': '2024-09-10T16:09:32.930698+0000', 'last_change': '2024-09-10T16:08:10.929169+0000', 'last_active': '2024-09-10T16:09:32.930698+0000', 'last_peered': '2024-09-10T16:09:32.930698+0000', 'last_clean': '2024-09-10T15:59:14.110654+0000', 'last_became_active': '2024-09-10T16:08:10.860438+0000', 'last_became_peered': '2024-09-10T16:08:10.860438+0000', 'last_unstale': '2024-09-10T16:09:32.930698+0000', 'last_undegraded': '2024-09-10T16:09:32.930698+0000', 'last_fullsized': '2024-09-10T16:09:32.930698+0000', 'mapping_epoch': 48, 'log_start': "45'11100", 'ondisk_log_start': "45'11100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2486", 'last_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'objects_scrubbed': 2440, 'log_size': 10070, 'log_dups_size': 0, 'ondisk_log_size': 10070, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 792, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18724, 'num_write_kb': 32568, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 4355, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': ['2', '0', '1'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:09:43.152 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-10T16:09:43.152 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "49'20838", 'reported_seq': 20992, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:09:32.930297+0000', 'last_change': '2024-09-10T15:59:16.315641+0000', 'last_active': '2024-09-10T16:09:32.930297+0000', 'last_peered': '2024-09-10T16:09:32.930297+0000', 'last_clean': '2024-09-10T15:59:14.117001+0000', 'last_became_active': '2024-09-10T15:59:16.302245+0000', 'last_became_peered': '2024-09-10T15:59:16.302245+0000', 'last_unstale': '2024-09-10T16:09:32.930297+0000', 'last_undegraded': '2024-09-10T15:59:15.845012+0000', 'last_fullsized': '2024-09-10T15:59:15.770337+0000', 'mapping_epoch': 42, 'log_start': "45'10800", 'ondisk_log_start': "45'10800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3404", 'last_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'objects_scrubbed': 3367, 'log_size': 10038, 'log_dups_size': 0, 'ondisk_log_size': 10038, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 983, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17469, 'num_write_kb': 28204, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4304}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:09:43.152 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-10T16:09:43.152 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "49'21048", 'reported_seq': 21209, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:09:32.931059+0000', 'last_change': '2024-09-10T15:59:16.311291+0000', 'last_active': '2024-09-10T16:09:32.931059+0000', 'last_peered': '2024-09-10T16:09:32.931059+0000', 'last_clean': '2024-09-10T15:59:14.116363+0000', 'last_became_active': '2024-09-10T15:59:16.301244+0000', 'last_became_peered': '2024-09-10T15:59:16.301244+0000', 'last_unstale': '2024-09-10T16:09:32.931059+0000', 'last_undegraded': '2024-09-10T15:59:15.883434+0000', 'last_fullsized': '2024-09-10T15:59:15.772756+0000', 'mapping_epoch': 42, 'log_start': "45'11000", 'ondisk_log_start': "45'11000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3789", 'last_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'objects_scrubbed': 3741, 'log_size': 10048, 'log_dups_size': 0, 'ondisk_log_size': 10048, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1440, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4373, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17303, 'num_write_kb': 27124, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4371}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:09:43.152 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-10T16:09:43.152 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "49'21198", 'reported_seq': 21362, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:09:32.931857+0000', 'last_change': '2024-09-10T15:59:16.313409+0000', 'last_active': '2024-09-10T16:09:32.931857+0000', 'last_peered': '2024-09-10T16:09:32.931857+0000', 'last_clean': '2024-09-10T15:59:14.129780+0000', 'last_became_active': '2024-09-10T15:59:16.300932+0000', 'last_became_peered': '2024-09-10T15:59:16.300932+0000', 'last_unstale': '2024-09-10T16:09:32.931857+0000', 'last_undegraded': '2024-09-10T15:59:15.887823+0000', 'last_fullsized': '2024-09-10T15:59:15.768513+0000', 'mapping_epoch': 42, 'log_start': "45'11100", 'ondisk_log_start': "45'11100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3880", 'last_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'objects_scrubbed': 3827, 'log_size': 10098, 'log_dups_size': 0, 'ondisk_log_size': 10098, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1397, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4325, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17361, 'num_write_kb': 27072, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4321}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:09:43.153 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T16:09:43.562 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:09:43.562 INFO:teuthology.orchestra.run.smithi115.stdout:{"fsid":"7effee29-df05-4351-9741-ee203564ed81","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 28567/24 objects degraded (119029.167%), 7 pgs degraded, 7 pgs undersized","count":14},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":747,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":53,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983954,"num_in_osds":3,"osd_in_since":1725983947,"num_remapped_pgs":7},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":16},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":6},{"state_name":"active+recovery_wait","count":1},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":12,"data_bytes":4849,"bytes_used":1555931136,"bytes_avail":288354361344,"bytes_total":289910292480,"degraded_objects":28567,"degraded_total":24,"degraded_ratio":1190.2916666666667,"recovering_objects_per_sec":181,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1785,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-10T15:57:14:967120+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T16:00:13.560776+0000","services":{}},"progress_events":{}} 2024-09-10T16:09:43.576 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-10T16:09:43.576 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:09:43.929 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:09:43.930 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:09:43.942 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":420,"stamp":"2024-09-10T16:09:43.776391+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":28567,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292098,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22825,"num_bytes_recovered":33096182,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161025,"ondisk_log_size":161025,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1519464,"kb_used_data":206888,"kb_used_omap":12,"kb_used_meta":1312499,"kb_avail":281596056,"statfs":{"total":289910292480,"available":288354361344,"internally_reserved":0,"allocated":211853312,"data_stored":210839911,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1343998978},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1785,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1785,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.849257"},"pg_stats":[{"pgid":"2.f","version":"49'20672","reported_seq":23985,"reported_epoch":53,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:41.207815+0000","last_change":"2024-09-10T16:09:31.772779+0000","last_active":"2024-09-10T16:09:41.207815+0000","last_peered":"2024-09-10T16:09:41.207815+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:09:41.207815+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2650,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20672,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1609,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2650}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20844","reported_seq":20976,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.933194+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:09:32.933194+0000","last_peered":"2024-09-10T16:09:32.933194+0000","last_clean":"2024-09-10T16:09:32.933194+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:09:32.933194+0000","last_undegraded":"2024-09-10T16:09:32.933194+0000","last_fullsized":"2024-09-10T16:09:32.933194+0000","mapping_epoch":35,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18949,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20798","reported_seq":29495,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.929401+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:09:32.929401+0000","last_peered":"2024-09-10T16:09:32.929401+0000","last_clean":"2024-09-10T16:09:32.929401+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:09:32.929401+0000","last_undegraded":"2024-09-10T16:09:32.929401+0000","last_fullsized":"2024-09-10T16:09:32.929401+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18802,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20660","reported_seq":20760,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.933021+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:09:32.933021+0000","last_peered":"2024-09-10T16:09:32.933021+0000","last_clean":"2024-09-10T16:09:32.933021+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:09:32.933021+0000","last_undegraded":"2024-09-10T16:09:32.933021+0000","last_fullsized":"2024-09-10T16:09:32.933021+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20660,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20891","reported_seq":21040,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.929877+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:09:32.929877+0000","last_peered":"2024-09-10T16:09:32.929877+0000","last_clean":"2024-09-10T16:09:32.929877+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:09:32.929877+0000","last_undegraded":"2024-09-10T16:09:32.929877+0000","last_fullsized":"2024-09-10T16:09:32.929877+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17602,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'21061","reported_seq":21218,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:32.929429+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:09:32.929429+0000","last_peered":"2024-09-10T16:09:32.929429+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:09:32.929429+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17673,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4370},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'21082","reported_seq":21239,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:32.928987+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:09:32.928987+0000","last_peered":"2024-09-10T16:09:32.928987+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:09:32.928987+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17581,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4235},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'21314","reported_seq":21463,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.928707+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:09:32.928707+0000","last_peered":"2024-09-10T16:09:32.928707+0000","last_clean":"2024-09-10T16:09:32.928707+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:09:32.928707+0000","last_undegraded":"2024-09-10T16:09:32.928707+0000","last_fullsized":"2024-09-10T16:09:32.928707+0000","mapping_epoch":34,"log_start":"45'11300","ondisk_log_start":"45'11300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18161,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.929057+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:09:32.929057+0000","last_peered":"2024-09-10T16:09:32.929057+0000","last_clean":"2024-09-10T16:09:32.929057+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:09:32.929057+0000","last_undegraded":"2024-09-10T16:09:32.929057+0000","last_fullsized":"2024-09-10T16:09:32.929057+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20457","reported_seq":20620,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:33.013394+0000","last_change":"2024-09-10T16:09:33.012840+0000","last_active":"2024-09-10T16:09:33.013394+0000","last_peered":"2024-09-10T16:09:33.013394+0000","last_clean":"2024-09-10T16:09:33.013394+0000","last_became_active":"2024-09-10T16:09:32.934056+0000","last_became_peered":"2024-09-10T16:09:32.934056+0000","last_unstale":"2024-09-10T16:09:33.013394+0000","last_undegraded":"2024-09-10T16:09:33.013394+0000","last_fullsized":"2024-09-10T16:09:33.013394+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:08:30.481235+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18276,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":4579,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":113,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.930929+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:09:32.930929+0000","last_peered":"2024-09-10T16:09:32.930929+0000","last_clean":"2024-09-10T16:09:32.930929+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:09:32.930929+0000","last_undegraded":"2024-09-10T16:09:32.930929+0000","last_fullsized":"2024-09-10T16:09:32.930929+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20910","reported_seq":29582,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.929808+0000","last_change":"2024-09-10T16:09:08.909580+0000","last_active":"2024-09-10T16:09:32.929808+0000","last_peered":"2024-09-10T16:09:32.929808+0000","last_clean":"2024-09-10T16:09:32.929808+0000","last_became_active":"2024-09-10T16:09:08.854941+0000","last_became_peered":"2024-09-10T16:09:08.854941+0000","last_unstale":"2024-09-10T16:09:32.929808+0000","last_undegraded":"2024-09-10T16:09:32.929808+0000","last_fullsized":"2024-09-10T16:09:32.929808+0000","mapping_epoch":50,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:13:14.846691+0000","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17414,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4257,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":123,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.928762+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:09:32.928762+0000","last_peered":"2024-09-10T16:09:32.928762+0000","last_clean":"2024-09-10T16:09:32.928762+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:09:32.928762+0000","last_undegraded":"2024-09-10T16:09:32.928762+0000","last_fullsized":"2024-09-10T16:09:32.928762+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20781","reported_seq":20935,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:32.931013+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:09:32.931013+0000","last_peered":"2024-09-10T16:09:32.931013+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:09:32.931013+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18461,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-4303}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":98,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.929298+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:09:32.929298+0000","last_peered":"2024-09-10T16:09:32.929298+0000","last_clean":"2024-09-10T16:09:32.929298+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:09:32.929298+0000","last_undegraded":"2024-09-10T16:09:32.929298+0000","last_fullsized":"2024-09-10T16:09:32.929298+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'21170","reported_seq":30033,"reported_epoch":53,"state":"active+recovery_wait","last_fresh":"2024-09-10T16:09:32.930698+0000","last_change":"2024-09-10T16:08:10.929169+0000","last_active":"2024-09-10T16:09:32.930698+0000","last_peered":"2024-09-10T16:09:32.930698+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T16:08:10.860438+0000","last_became_peered":"2024-09-10T16:08:10.860438+0000","last_unstale":"2024-09-10T16:09:32.930698+0000","last_undegraded":"2024-09-10T16:09:32.930698+0000","last_fullsized":"2024-09-10T16:09:32.930698+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18724,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","0","1"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20838","reported_seq":20992,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:32.930297+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:09:32.930297+0000","last_peered":"2024-09-10T16:09:32.930297+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:09:32.930297+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17469,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":88,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.929239+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:09:32.929239+0000","last_peered":"2024-09-10T16:09:32.929239+0000","last_clean":"2024-09-10T16:09:32.929239+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:09:32.929239+0000","last_undegraded":"2024-09-10T16:09:32.929239+0000","last_fullsized":"2024-09-10T16:09:32.929239+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20598","reported_seq":29183,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.929268+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:09:32.929268+0000","last_peered":"2024-09-10T16:09:32.929268+0000","last_clean":"2024-09-10T16:09:32.929268+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:09:32.929268+0000","last_undegraded":"2024-09-10T16:09:32.929268+0000","last_fullsized":"2024-09-10T16:09:32.929268+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16988,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":113,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.933167+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:09:32.933167+0000","last_peered":"2024-09-10T16:09:32.933167+0000","last_clean":"2024-09-10T16:09:32.933167+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:09:32.933167+0000","last_undegraded":"2024-09-10T16:09:32.933167+0000","last_fullsized":"2024-09-10T16:09:32.933167+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'21048","reported_seq":21209,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:32.931059+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:09:32.931059+0000","last_peered":"2024-09-10T16:09:32.931059+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:09:32.931059+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17303,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":88,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.929129+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:09:32.929129+0000","last_peered":"2024-09-10T16:09:32.929129+0000","last_clean":"2024-09-10T16:09:32.929129+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:09:32.929129+0000","last_undegraded":"2024-09-10T16:09:32.929129+0000","last_fullsized":"2024-09-10T16:09:32.929129+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":93,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.929492+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:09:32.929492+0000","last_peered":"2024-09-10T16:09:32.929492+0000","last_clean":"2024-09-10T16:09:32.929492+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:09:32.929492+0000","last_undegraded":"2024-09-10T16:09:32.929492+0000","last_fullsized":"2024-09-10T16:09:32.929492+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'21198","reported_seq":21362,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:32.931857+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:09:32.931857+0000","last_peered":"2024-09-10T16:09:32.931857+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:09:32.931857+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17361,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4321},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":28567,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292096,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22823,"num_bytes_recovered":33096163,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":210518016,"data_stored":210395174,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161022,"ondisk_log_size":161022,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986316,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":502632,"kb_used_data":137744,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93869208,"statfs":{"total":96636764160,"available":96122068992,"internally_reserved":0,"allocated":141049856,"data_stored":140719591,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659261,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":469500,"kb_used_data":492,"kb_used_omap":4,"kb_used_meta":468987,"kb_avail":93902340,"statfs":{"total":96636764160,"available":96155996160,"internally_reserved":0,"allocated":503808,"data_stored":155014,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":480243374},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182544,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":547332,"kb_used_data":68652,"kb_used_omap":4,"kb_used_meta":478651,"kb_avail":93824508,"statfs":{"total":96636764160,"available":96076296192,"internally_reserved":0,"allocated":70299648,"data_stored":69965306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":490139306},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":69853184,"data_stored":69817054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":6762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":140607488,"data_stored":140571358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:09:46.944 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:09:47.250 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:09:47.250 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:09:47.262 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":421,"stamp":"2024-09-10T16:09:45.776694+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":28567,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292098,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22825,"num_bytes_recovered":33096182,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161025,"ondisk_log_size":161025,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1523624,"kb_used_data":206888,"kb_used_omap":12,"kb_used_meta":1316659,"kb_avail":281591896,"statfs":{"total":289910292480,"available":288350101504,"internally_reserved":0,"allocated":211853312,"data_stored":210839911,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1348258818},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1785,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1785,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002252"},"pg_stats":[{"pgid":"2.f","version":"49'20672","reported_seq":23985,"reported_epoch":53,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:41.207815+0000","last_change":"2024-09-10T16:09:31.772779+0000","last_active":"2024-09-10T16:09:41.207815+0000","last_peered":"2024-09-10T16:09:41.207815+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:09:41.207815+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2650,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20672,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1609,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2650}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20844","reported_seq":20976,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.933194+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:09:32.933194+0000","last_peered":"2024-09-10T16:09:32.933194+0000","last_clean":"2024-09-10T16:09:32.933194+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:09:32.933194+0000","last_undegraded":"2024-09-10T16:09:32.933194+0000","last_fullsized":"2024-09-10T16:09:32.933194+0000","mapping_epoch":35,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18949,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20798","reported_seq":29495,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.929401+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:09:32.929401+0000","last_peered":"2024-09-10T16:09:32.929401+0000","last_clean":"2024-09-10T16:09:32.929401+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:09:32.929401+0000","last_undegraded":"2024-09-10T16:09:32.929401+0000","last_fullsized":"2024-09-10T16:09:32.929401+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18802,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20660","reported_seq":20760,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.933021+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:09:32.933021+0000","last_peered":"2024-09-10T16:09:32.933021+0000","last_clean":"2024-09-10T16:09:32.933021+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:09:32.933021+0000","last_undegraded":"2024-09-10T16:09:32.933021+0000","last_fullsized":"2024-09-10T16:09:32.933021+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20660,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20891","reported_seq":21040,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.929877+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:09:32.929877+0000","last_peered":"2024-09-10T16:09:32.929877+0000","last_clean":"2024-09-10T16:09:32.929877+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:09:32.929877+0000","last_undegraded":"2024-09-10T16:09:32.929877+0000","last_fullsized":"2024-09-10T16:09:32.929877+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17602,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'21061","reported_seq":21218,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:32.929429+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:09:32.929429+0000","last_peered":"2024-09-10T16:09:32.929429+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:09:32.929429+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17673,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4370},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'21082","reported_seq":21239,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:32.928987+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:09:32.928987+0000","last_peered":"2024-09-10T16:09:32.928987+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:09:32.928987+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17581,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4235},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'21314","reported_seq":21463,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.928707+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:09:32.928707+0000","last_peered":"2024-09-10T16:09:32.928707+0000","last_clean":"2024-09-10T16:09:32.928707+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:09:32.928707+0000","last_undegraded":"2024-09-10T16:09:32.928707+0000","last_fullsized":"2024-09-10T16:09:32.928707+0000","mapping_epoch":34,"log_start":"45'11300","ondisk_log_start":"45'11300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18161,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.929057+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:09:32.929057+0000","last_peered":"2024-09-10T16:09:32.929057+0000","last_clean":"2024-09-10T16:09:32.929057+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:09:32.929057+0000","last_undegraded":"2024-09-10T16:09:32.929057+0000","last_fullsized":"2024-09-10T16:09:32.929057+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20457","reported_seq":20620,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:33.013394+0000","last_change":"2024-09-10T16:09:33.012840+0000","last_active":"2024-09-10T16:09:33.013394+0000","last_peered":"2024-09-10T16:09:33.013394+0000","last_clean":"2024-09-10T16:09:33.013394+0000","last_became_active":"2024-09-10T16:09:32.934056+0000","last_became_peered":"2024-09-10T16:09:32.934056+0000","last_unstale":"2024-09-10T16:09:33.013394+0000","last_undegraded":"2024-09-10T16:09:33.013394+0000","last_fullsized":"2024-09-10T16:09:33.013394+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:08:30.481235+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18276,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":4579,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":113,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.930929+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:09:32.930929+0000","last_peered":"2024-09-10T16:09:32.930929+0000","last_clean":"2024-09-10T16:09:32.930929+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:09:32.930929+0000","last_undegraded":"2024-09-10T16:09:32.930929+0000","last_fullsized":"2024-09-10T16:09:32.930929+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20910","reported_seq":29582,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.929808+0000","last_change":"2024-09-10T16:09:08.909580+0000","last_active":"2024-09-10T16:09:32.929808+0000","last_peered":"2024-09-10T16:09:32.929808+0000","last_clean":"2024-09-10T16:09:32.929808+0000","last_became_active":"2024-09-10T16:09:08.854941+0000","last_became_peered":"2024-09-10T16:09:08.854941+0000","last_unstale":"2024-09-10T16:09:32.929808+0000","last_undegraded":"2024-09-10T16:09:32.929808+0000","last_fullsized":"2024-09-10T16:09:32.929808+0000","mapping_epoch":50,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:13:14.846691+0000","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17414,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4257,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":123,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.928762+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:09:32.928762+0000","last_peered":"2024-09-10T16:09:32.928762+0000","last_clean":"2024-09-10T16:09:32.928762+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:09:32.928762+0000","last_undegraded":"2024-09-10T16:09:32.928762+0000","last_fullsized":"2024-09-10T16:09:32.928762+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20781","reported_seq":20935,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:32.931013+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:09:32.931013+0000","last_peered":"2024-09-10T16:09:32.931013+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:09:32.931013+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18461,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-4303}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":99,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.929342+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:09:32.929342+0000","last_peered":"2024-09-10T16:09:32.929342+0000","last_clean":"2024-09-10T16:09:32.929342+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:09:32.929342+0000","last_undegraded":"2024-09-10T16:09:32.929342+0000","last_fullsized":"2024-09-10T16:09:32.929342+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'21170","reported_seq":30033,"reported_epoch":53,"state":"active+recovery_wait","last_fresh":"2024-09-10T16:09:32.930698+0000","last_change":"2024-09-10T16:08:10.929169+0000","last_active":"2024-09-10T16:09:32.930698+0000","last_peered":"2024-09-10T16:09:32.930698+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T16:08:10.860438+0000","last_became_peered":"2024-09-10T16:08:10.860438+0000","last_unstale":"2024-09-10T16:09:32.930698+0000","last_undegraded":"2024-09-10T16:09:32.930698+0000","last_fullsized":"2024-09-10T16:09:32.930698+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18724,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","0","1"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20838","reported_seq":20992,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:32.930297+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:09:32.930297+0000","last_peered":"2024-09-10T16:09:32.930297+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:09:32.930297+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17469,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.929280+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:09:32.929280+0000","last_peered":"2024-09-10T16:09:32.929280+0000","last_clean":"2024-09-10T16:09:32.929280+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:09:32.929280+0000","last_undegraded":"2024-09-10T16:09:32.929280+0000","last_fullsized":"2024-09-10T16:09:32.929280+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20598","reported_seq":29183,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.929268+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:09:32.929268+0000","last_peered":"2024-09-10T16:09:32.929268+0000","last_clean":"2024-09-10T16:09:32.929268+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:09:32.929268+0000","last_undegraded":"2024-09-10T16:09:32.929268+0000","last_fullsized":"2024-09-10T16:09:32.929268+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16988,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":113,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.933167+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:09:32.933167+0000","last_peered":"2024-09-10T16:09:32.933167+0000","last_clean":"2024-09-10T16:09:32.933167+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:09:32.933167+0000","last_undegraded":"2024-09-10T16:09:32.933167+0000","last_fullsized":"2024-09-10T16:09:32.933167+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'21048","reported_seq":21209,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:32.931059+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:09:32.931059+0000","last_peered":"2024-09-10T16:09:32.931059+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:09:32.931059+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17303,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":88,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.929129+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:09:32.929129+0000","last_peered":"2024-09-10T16:09:32.929129+0000","last_clean":"2024-09-10T16:09:32.929129+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:09:32.929129+0000","last_undegraded":"2024-09-10T16:09:32.929129+0000","last_fullsized":"2024-09-10T16:09:32.929129+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":93,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.929492+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:09:32.929492+0000","last_peered":"2024-09-10T16:09:32.929492+0000","last_clean":"2024-09-10T16:09:32.929492+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:09:32.929492+0000","last_undegraded":"2024-09-10T16:09:32.929492+0000","last_fullsized":"2024-09-10T16:09:32.929492+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'21198","reported_seq":21362,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:32.931857+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:09:32.931857+0000","last_peered":"2024-09-10T16:09:32.931857+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:09:32.931857+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17361,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4321},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":28567,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292096,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22823,"num_bytes_recovered":33096163,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":210518016,"data_stored":210395174,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161022,"ondisk_log_size":161022,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986316,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":502632,"kb_used_data":137744,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93869208,"statfs":{"total":96636764160,"available":96122068992,"internally_reserved":0,"allocated":141049856,"data_stored":140719591,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659262,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":473660,"kb_used_data":492,"kb_used_omap":4,"kb_used_meta":473147,"kb_avail":93898180,"statfs":{"total":96636764160,"available":96151736320,"internally_reserved":0,"allocated":503808,"data_stored":155014,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":484503214},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182544,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":547332,"kb_used_data":68652,"kb_used_omap":4,"kb_used_meta":478651,"kb_avail":93824508,"statfs":{"total":96636764160,"available":96076296192,"internally_reserved":0,"allocated":70299648,"data_stored":69965306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":490139306},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":69853184,"data_stored":69817054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":6762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":140607488,"data_stored":140571358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:09:47.263 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:09:47.564 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:09:47.564 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:09:47.576 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":421,"stamp":"2024-09-10T16:09:45.776694+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":28567,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292098,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22825,"num_bytes_recovered":33096182,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161025,"ondisk_log_size":161025,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1523624,"kb_used_data":206888,"kb_used_omap":12,"kb_used_meta":1316659,"kb_avail":281591896,"statfs":{"total":289910292480,"available":288350101504,"internally_reserved":0,"allocated":211853312,"data_stored":210839911,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1348258818},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1785,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1785,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002252"},"pg_stats":[{"pgid":"2.f","version":"49'20672","reported_seq":23985,"reported_epoch":53,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:41.207815+0000","last_change":"2024-09-10T16:09:31.772779+0000","last_active":"2024-09-10T16:09:41.207815+0000","last_peered":"2024-09-10T16:09:41.207815+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:09:41.207815+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2650,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20672,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1609,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2650}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20844","reported_seq":20976,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.933194+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:09:32.933194+0000","last_peered":"2024-09-10T16:09:32.933194+0000","last_clean":"2024-09-10T16:09:32.933194+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:09:32.933194+0000","last_undegraded":"2024-09-10T16:09:32.933194+0000","last_fullsized":"2024-09-10T16:09:32.933194+0000","mapping_epoch":35,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18949,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20798","reported_seq":29495,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.929401+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:09:32.929401+0000","last_peered":"2024-09-10T16:09:32.929401+0000","last_clean":"2024-09-10T16:09:32.929401+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:09:32.929401+0000","last_undegraded":"2024-09-10T16:09:32.929401+0000","last_fullsized":"2024-09-10T16:09:32.929401+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18802,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20660","reported_seq":20760,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.933021+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:09:32.933021+0000","last_peered":"2024-09-10T16:09:32.933021+0000","last_clean":"2024-09-10T16:09:32.933021+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:09:32.933021+0000","last_undegraded":"2024-09-10T16:09:32.933021+0000","last_fullsized":"2024-09-10T16:09:32.933021+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20660,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20891","reported_seq":21040,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.929877+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:09:32.929877+0000","last_peered":"2024-09-10T16:09:32.929877+0000","last_clean":"2024-09-10T16:09:32.929877+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:09:32.929877+0000","last_undegraded":"2024-09-10T16:09:32.929877+0000","last_fullsized":"2024-09-10T16:09:32.929877+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17602,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'21061","reported_seq":21218,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:32.929429+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:09:32.929429+0000","last_peered":"2024-09-10T16:09:32.929429+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:09:32.929429+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17673,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4370},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'21082","reported_seq":21239,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:32.928987+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:09:32.928987+0000","last_peered":"2024-09-10T16:09:32.928987+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:09:32.928987+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17581,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4235},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'21314","reported_seq":21463,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.928707+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:09:32.928707+0000","last_peered":"2024-09-10T16:09:32.928707+0000","last_clean":"2024-09-10T16:09:32.928707+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:09:32.928707+0000","last_undegraded":"2024-09-10T16:09:32.928707+0000","last_fullsized":"2024-09-10T16:09:32.928707+0000","mapping_epoch":34,"log_start":"45'11300","ondisk_log_start":"45'11300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18161,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.929057+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:09:32.929057+0000","last_peered":"2024-09-10T16:09:32.929057+0000","last_clean":"2024-09-10T16:09:32.929057+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:09:32.929057+0000","last_undegraded":"2024-09-10T16:09:32.929057+0000","last_fullsized":"2024-09-10T16:09:32.929057+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20457","reported_seq":20620,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:33.013394+0000","last_change":"2024-09-10T16:09:33.012840+0000","last_active":"2024-09-10T16:09:33.013394+0000","last_peered":"2024-09-10T16:09:33.013394+0000","last_clean":"2024-09-10T16:09:33.013394+0000","last_became_active":"2024-09-10T16:09:32.934056+0000","last_became_peered":"2024-09-10T16:09:32.934056+0000","last_unstale":"2024-09-10T16:09:33.013394+0000","last_undegraded":"2024-09-10T16:09:33.013394+0000","last_fullsized":"2024-09-10T16:09:33.013394+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:08:30.481235+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18276,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":4579,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":113,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.930929+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:09:32.930929+0000","last_peered":"2024-09-10T16:09:32.930929+0000","last_clean":"2024-09-10T16:09:32.930929+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:09:32.930929+0000","last_undegraded":"2024-09-10T16:09:32.930929+0000","last_fullsized":"2024-09-10T16:09:32.930929+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20910","reported_seq":29582,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.929808+0000","last_change":"2024-09-10T16:09:08.909580+0000","last_active":"2024-09-10T16:09:32.929808+0000","last_peered":"2024-09-10T16:09:32.929808+0000","last_clean":"2024-09-10T16:09:32.929808+0000","last_became_active":"2024-09-10T16:09:08.854941+0000","last_became_peered":"2024-09-10T16:09:08.854941+0000","last_unstale":"2024-09-10T16:09:32.929808+0000","last_undegraded":"2024-09-10T16:09:32.929808+0000","last_fullsized":"2024-09-10T16:09:32.929808+0000","mapping_epoch":50,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:13:14.846691+0000","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17414,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4257,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":123,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.928762+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:09:32.928762+0000","last_peered":"2024-09-10T16:09:32.928762+0000","last_clean":"2024-09-10T16:09:32.928762+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:09:32.928762+0000","last_undegraded":"2024-09-10T16:09:32.928762+0000","last_fullsized":"2024-09-10T16:09:32.928762+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20781","reported_seq":20935,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:32.931013+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:09:32.931013+0000","last_peered":"2024-09-10T16:09:32.931013+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:09:32.931013+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18461,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-4303}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":99,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.929342+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:09:32.929342+0000","last_peered":"2024-09-10T16:09:32.929342+0000","last_clean":"2024-09-10T16:09:32.929342+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:09:32.929342+0000","last_undegraded":"2024-09-10T16:09:32.929342+0000","last_fullsized":"2024-09-10T16:09:32.929342+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'21170","reported_seq":30033,"reported_epoch":53,"state":"active+recovery_wait","last_fresh":"2024-09-10T16:09:32.930698+0000","last_change":"2024-09-10T16:08:10.929169+0000","last_active":"2024-09-10T16:09:32.930698+0000","last_peered":"2024-09-10T16:09:32.930698+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T16:08:10.860438+0000","last_became_peered":"2024-09-10T16:08:10.860438+0000","last_unstale":"2024-09-10T16:09:32.930698+0000","last_undegraded":"2024-09-10T16:09:32.930698+0000","last_fullsized":"2024-09-10T16:09:32.930698+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18724,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","0","1"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20838","reported_seq":20992,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:32.930297+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:09:32.930297+0000","last_peered":"2024-09-10T16:09:32.930297+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:09:32.930297+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17469,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.929280+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:09:32.929280+0000","last_peered":"2024-09-10T16:09:32.929280+0000","last_clean":"2024-09-10T16:09:32.929280+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:09:32.929280+0000","last_undegraded":"2024-09-10T16:09:32.929280+0000","last_fullsized":"2024-09-10T16:09:32.929280+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20598","reported_seq":29183,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.929268+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:09:32.929268+0000","last_peered":"2024-09-10T16:09:32.929268+0000","last_clean":"2024-09-10T16:09:32.929268+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:09:32.929268+0000","last_undegraded":"2024-09-10T16:09:32.929268+0000","last_fullsized":"2024-09-10T16:09:32.929268+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16988,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":113,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.933167+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:09:32.933167+0000","last_peered":"2024-09-10T16:09:32.933167+0000","last_clean":"2024-09-10T16:09:32.933167+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:09:32.933167+0000","last_undegraded":"2024-09-10T16:09:32.933167+0000","last_fullsized":"2024-09-10T16:09:32.933167+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'21048","reported_seq":21209,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:32.931059+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:09:32.931059+0000","last_peered":"2024-09-10T16:09:32.931059+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:09:32.931059+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17303,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":88,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.929129+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:09:32.929129+0000","last_peered":"2024-09-10T16:09:32.929129+0000","last_clean":"2024-09-10T16:09:32.929129+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:09:32.929129+0000","last_undegraded":"2024-09-10T16:09:32.929129+0000","last_fullsized":"2024-09-10T16:09:32.929129+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":93,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.929492+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:09:32.929492+0000","last_peered":"2024-09-10T16:09:32.929492+0000","last_clean":"2024-09-10T16:09:32.929492+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:09:32.929492+0000","last_undegraded":"2024-09-10T16:09:32.929492+0000","last_fullsized":"2024-09-10T16:09:32.929492+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'21198","reported_seq":21362,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:32.931857+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:09:32.931857+0000","last_peered":"2024-09-10T16:09:32.931857+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:09:32.931857+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17361,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4321},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":28567,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292096,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22823,"num_bytes_recovered":33096163,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":210518016,"data_stored":210395174,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161022,"ondisk_log_size":161022,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986316,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":502632,"kb_used_data":137744,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93869208,"statfs":{"total":96636764160,"available":96122068992,"internally_reserved":0,"allocated":141049856,"data_stored":140719591,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659262,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":473660,"kb_used_data":492,"kb_used_omap":4,"kb_used_meta":473147,"kb_avail":93898180,"statfs":{"total":96636764160,"available":96151736320,"internally_reserved":0,"allocated":503808,"data_stored":155014,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":484503214},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182544,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":547332,"kb_used_data":68652,"kb_used_omap":4,"kb_used_meta":478651,"kb_avail":93824508,"statfs":{"total":96636764160,"available":96076296192,"internally_reserved":0,"allocated":70299648,"data_stored":69965306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":490139306},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":69853184,"data_stored":69817054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":6762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":140607488,"data_stored":140571358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:09:47.578 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-10T16:09:47.578 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "49'20672", 'reported_seq': 23985, 'reported_epoch': 53, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:09:41.207815+0000', 'last_change': '2024-09-10T16:09:31.772779+0000', 'last_active': '2024-09-10T16:09:41.207815+0000', 'last_peered': '2024-09-10T16:09:41.207815+0000', 'last_clean': '2024-09-10T15:59:14.132456+0000', 'last_became_active': '2024-09-10T15:59:16.274407+0000', 'last_became_peered': '2024-09-10T15:59:16.274407+0000', 'last_unstale': '2024-09-10T16:09:41.207815+0000', 'last_undegraded': '2024-09-10T15:59:15.851279+0000', 'last_fullsized': '2024-09-10T15:59:15.774356+0000', 'mapping_epoch': 42, 'log_start': "45'10600", 'ondisk_log_start': "45'10600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'objects_scrubbed': 0, 'log_size': 10072, 'log_dups_size': 0, 'ondisk_log_size': 10072, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2650, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20672, 'num_write_kb': 41344, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1609, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2650}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:09:47.578 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-10T16:09:47.578 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "49'21061", 'reported_seq': 21218, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:09:32.929429+0000', 'last_change': '2024-09-10T15:59:16.308264+0000', 'last_active': '2024-09-10T16:09:32.929429+0000', 'last_peered': '2024-09-10T16:09:32.929429+0000', 'last_clean': '2024-09-10T15:59:14.111551+0000', 'last_became_active': '2024-09-10T15:59:16.300040+0000', 'last_became_peered': '2024-09-10T15:59:16.300040+0000', 'last_unstale': '2024-09-10T16:09:32.929429+0000', 'last_undegraded': '2024-09-10T15:59:15.878234+0000', 'last_fullsized': '2024-09-10T15:59:15.769328+0000', 'mapping_epoch': 42, 'log_start': "45'11000", 'ondisk_log_start': "45'11000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3442", 'last_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'objects_scrubbed': 3384, 'log_size': 10061, 'log_dups_size': 0, 'ondisk_log_size': 10061, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1292, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4372, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 17673, 'num_write_kb': 28580, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4370}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:09:47.578 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-10T16:09:47.578 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "49'21082", 'reported_seq': 21239, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:09:32.928987+0000', 'last_change': '2024-09-10T15:59:16.331899+0000', 'last_active': '2024-09-10T16:09:32.928987+0000', 'last_peered': '2024-09-10T16:09:32.928987+0000', 'last_clean': '2024-09-10T15:59:14.139436+0000', 'last_became_active': '2024-09-10T15:59:16.318715+0000', 'last_became_peered': '2024-09-10T15:59:16.318715+0000', 'last_unstale': '2024-09-10T16:09:32.928987+0000', 'last_undegraded': '2024-09-10T15:59:15.877524+0000', 'last_fullsized': '2024-09-10T15:59:15.769042+0000', 'mapping_epoch': 42, 'log_start': "45'11000", 'ondisk_log_start': "45'11000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3537", 'last_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'objects_scrubbed': 3497, 'log_size': 10082, 'log_dups_size': 0, 'ondisk_log_size': 10082, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1224, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4237, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 17581, 'num_write_kb': 28172, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4235}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:09:47.578 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-10T16:09:47.578 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "49'20781", 'reported_seq': 20935, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:09:32.931013+0000', 'last_change': '2024-09-10T15:59:16.284311+0000', 'last_active': '2024-09-10T16:09:32.931013+0000', 'last_peered': '2024-09-10T16:09:32.931013+0000', 'last_clean': '2024-09-10T15:59:14.116041+0000', 'last_became_active': '2024-09-10T15:59:16.274543+0000', 'last_became_peered': '2024-09-10T15:59:16.274543+0000', 'last_unstale': '2024-09-10T16:09:32.931013+0000', 'last_undegraded': '2024-09-10T15:59:15.812484+0000', 'last_fullsized': '2024-09-10T15:59:15.770885+0000', 'mapping_epoch': 42, 'log_start': "45'10700", 'ondisk_log_start': "45'10700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2356", 'last_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'objects_scrubbed': 2316, 'log_size': 10081, 'log_dups_size': 0, 'ondisk_log_size': 10081, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 783, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 18461, 'num_write_kb': 32292, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': -4303}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:09:47.579 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-10T16:09:47.579 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "49'21170", 'reported_seq': 30033, 'reported_epoch': 53, 'state': 'active+recovery_wait', 'last_fresh': '2024-09-10T16:09:32.930698+0000', 'last_change': '2024-09-10T16:08:10.929169+0000', 'last_active': '2024-09-10T16:09:32.930698+0000', 'last_peered': '2024-09-10T16:09:32.930698+0000', 'last_clean': '2024-09-10T15:59:14.110654+0000', 'last_became_active': '2024-09-10T16:08:10.860438+0000', 'last_became_peered': '2024-09-10T16:08:10.860438+0000', 'last_unstale': '2024-09-10T16:09:32.930698+0000', 'last_undegraded': '2024-09-10T16:09:32.930698+0000', 'last_fullsized': '2024-09-10T16:09:32.930698+0000', 'mapping_epoch': 48, 'log_start': "45'11100", 'ondisk_log_start': "45'11100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2486", 'last_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'objects_scrubbed': 2440, 'log_size': 10070, 'log_dups_size': 0, 'ondisk_log_size': 10070, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 792, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18724, 'num_write_kb': 32568, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 4355, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': ['2', '0', '1'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:09:47.579 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-10T16:09:47.579 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "49'20838", 'reported_seq': 20992, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:09:32.930297+0000', 'last_change': '2024-09-10T15:59:16.315641+0000', 'last_active': '2024-09-10T16:09:32.930297+0000', 'last_peered': '2024-09-10T16:09:32.930297+0000', 'last_clean': '2024-09-10T15:59:14.117001+0000', 'last_became_active': '2024-09-10T15:59:16.302245+0000', 'last_became_peered': '2024-09-10T15:59:16.302245+0000', 'last_unstale': '2024-09-10T16:09:32.930297+0000', 'last_undegraded': '2024-09-10T15:59:15.845012+0000', 'last_fullsized': '2024-09-10T15:59:15.770337+0000', 'mapping_epoch': 42, 'log_start': "45'10800", 'ondisk_log_start': "45'10800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3404", 'last_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'objects_scrubbed': 3367, 'log_size': 10038, 'log_dups_size': 0, 'ondisk_log_size': 10038, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 983, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17469, 'num_write_kb': 28204, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4304}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:09:47.579 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-10T16:09:47.579 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "49'21048", 'reported_seq': 21209, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:09:32.931059+0000', 'last_change': '2024-09-10T15:59:16.311291+0000', 'last_active': '2024-09-10T16:09:32.931059+0000', 'last_peered': '2024-09-10T16:09:32.931059+0000', 'last_clean': '2024-09-10T15:59:14.116363+0000', 'last_became_active': '2024-09-10T15:59:16.301244+0000', 'last_became_peered': '2024-09-10T15:59:16.301244+0000', 'last_unstale': '2024-09-10T16:09:32.931059+0000', 'last_undegraded': '2024-09-10T15:59:15.883434+0000', 'last_fullsized': '2024-09-10T15:59:15.772756+0000', 'mapping_epoch': 42, 'log_start': "45'11000", 'ondisk_log_start': "45'11000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3789", 'last_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'objects_scrubbed': 3741, 'log_size': 10048, 'log_dups_size': 0, 'ondisk_log_size': 10048, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1440, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4373, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17303, 'num_write_kb': 27124, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4371}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:09:47.579 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-10T16:09:47.579 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "49'21198", 'reported_seq': 21362, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:09:32.931857+0000', 'last_change': '2024-09-10T15:59:16.313409+0000', 'last_active': '2024-09-10T16:09:32.931857+0000', 'last_peered': '2024-09-10T16:09:32.931857+0000', 'last_clean': '2024-09-10T15:59:14.129780+0000', 'last_became_active': '2024-09-10T15:59:16.300932+0000', 'last_became_peered': '2024-09-10T15:59:16.300932+0000', 'last_unstale': '2024-09-10T16:09:32.931857+0000', 'last_undegraded': '2024-09-10T15:59:15.887823+0000', 'last_fullsized': '2024-09-10T15:59:15.768513+0000', 'mapping_epoch': 42, 'log_start': "45'11100", 'ondisk_log_start': "45'11100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3880", 'last_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'objects_scrubbed': 3827, 'log_size': 10098, 'log_dups_size': 0, 'ondisk_log_size': 10098, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1397, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4325, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17361, 'num_write_kb': 27072, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4321}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:09:47.580 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T16:09:47.951 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:09:47.951 INFO:teuthology.orchestra.run.smithi115.stdout:{"fsid":"7effee29-df05-4351-9741-ee203564ed81","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 28567/24 objects degraded (119029.167%), 7 pgs degraded, 7 pgs undersized","count":14},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":751,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":53,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983954,"num_in_osds":3,"osd_in_since":1725983947,"num_remapped_pgs":7},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":16},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":6},{"state_name":"active+recovery_wait","count":1},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":12,"data_bytes":4849,"bytes_used":1560190976,"bytes_avail":288350101504,"bytes_total":289910292480,"degraded_objects":28567,"degraded_total":24,"degraded_ratio":1190.2916666666667,"recovering_objects_per_sec":148,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1785,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-10T15:57:14:967120+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T16:00:13.560776+0000","services":{}},"progress_events":{}} 2024-09-10T16:09:47.964 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-10T16:09:47.964 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:09:48.278 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:09:48.279 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:09:48.291 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":422,"stamp":"2024-09-10T16:09:47.776959+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":27812,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292098,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23580,"num_bytes_recovered":33096182,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161025,"ondisk_log_size":161025,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1523624,"kb_used_data":206888,"kb_used_omap":12,"kb_used_meta":1316659,"kb_avail":281591896,"statfs":{"total":289910292480,"available":288350101504,"internally_reserved":0,"allocated":211853312,"data_stored":210839911,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1348258818},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2364,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2364,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002099"},"pg_stats":[{"pgid":"2.f","version":"49'20672","reported_seq":25495,"reported_epoch":53,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:46.207166+0000","last_change":"2024-09-10T16:09:31.772779+0000","last_active":"2024-09-10T16:09:46.207166+0000","last_peered":"2024-09-10T16:09:46.207166+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:09:46.207166+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1895,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20672,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2364,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1895}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20844","reported_seq":20976,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.933194+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:09:32.933194+0000","last_peered":"2024-09-10T16:09:32.933194+0000","last_clean":"2024-09-10T16:09:32.933194+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:09:32.933194+0000","last_undegraded":"2024-09-10T16:09:32.933194+0000","last_fullsized":"2024-09-10T16:09:32.933194+0000","mapping_epoch":35,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18949,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20798","reported_seq":29495,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.929401+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:09:32.929401+0000","last_peered":"2024-09-10T16:09:32.929401+0000","last_clean":"2024-09-10T16:09:32.929401+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:09:32.929401+0000","last_undegraded":"2024-09-10T16:09:32.929401+0000","last_fullsized":"2024-09-10T16:09:32.929401+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18802,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20660","reported_seq":20760,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.933021+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:09:32.933021+0000","last_peered":"2024-09-10T16:09:32.933021+0000","last_clean":"2024-09-10T16:09:32.933021+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:09:32.933021+0000","last_undegraded":"2024-09-10T16:09:32.933021+0000","last_fullsized":"2024-09-10T16:09:32.933021+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20660,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20891","reported_seq":21040,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.929877+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:09:32.929877+0000","last_peered":"2024-09-10T16:09:32.929877+0000","last_clean":"2024-09-10T16:09:32.929877+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:09:32.929877+0000","last_undegraded":"2024-09-10T16:09:32.929877+0000","last_fullsized":"2024-09-10T16:09:32.929877+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17602,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'21061","reported_seq":21218,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:32.929429+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:09:32.929429+0000","last_peered":"2024-09-10T16:09:32.929429+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:09:32.929429+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17673,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4370},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'21082","reported_seq":21239,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:32.928987+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:09:32.928987+0000","last_peered":"2024-09-10T16:09:32.928987+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:09:32.928987+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17581,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4235},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'21314","reported_seq":21463,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.928707+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:09:32.928707+0000","last_peered":"2024-09-10T16:09:32.928707+0000","last_clean":"2024-09-10T16:09:32.928707+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:09:32.928707+0000","last_undegraded":"2024-09-10T16:09:32.928707+0000","last_fullsized":"2024-09-10T16:09:32.928707+0000","mapping_epoch":34,"log_start":"45'11300","ondisk_log_start":"45'11300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18161,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.929057+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:09:32.929057+0000","last_peered":"2024-09-10T16:09:32.929057+0000","last_clean":"2024-09-10T16:09:32.929057+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:09:32.929057+0000","last_undegraded":"2024-09-10T16:09:32.929057+0000","last_fullsized":"2024-09-10T16:09:32.929057+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20457","reported_seq":20620,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:33.013394+0000","last_change":"2024-09-10T16:09:33.012840+0000","last_active":"2024-09-10T16:09:33.013394+0000","last_peered":"2024-09-10T16:09:33.013394+0000","last_clean":"2024-09-10T16:09:33.013394+0000","last_became_active":"2024-09-10T16:09:32.934056+0000","last_became_peered":"2024-09-10T16:09:32.934056+0000","last_unstale":"2024-09-10T16:09:33.013394+0000","last_undegraded":"2024-09-10T16:09:33.013394+0000","last_fullsized":"2024-09-10T16:09:33.013394+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:08:30.481235+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18276,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":4579,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":113,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.930929+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:09:32.930929+0000","last_peered":"2024-09-10T16:09:32.930929+0000","last_clean":"2024-09-10T16:09:32.930929+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:09:32.930929+0000","last_undegraded":"2024-09-10T16:09:32.930929+0000","last_fullsized":"2024-09-10T16:09:32.930929+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20910","reported_seq":29582,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.929808+0000","last_change":"2024-09-10T16:09:08.909580+0000","last_active":"2024-09-10T16:09:32.929808+0000","last_peered":"2024-09-10T16:09:32.929808+0000","last_clean":"2024-09-10T16:09:32.929808+0000","last_became_active":"2024-09-10T16:09:08.854941+0000","last_became_peered":"2024-09-10T16:09:08.854941+0000","last_unstale":"2024-09-10T16:09:32.929808+0000","last_undegraded":"2024-09-10T16:09:32.929808+0000","last_fullsized":"2024-09-10T16:09:32.929808+0000","mapping_epoch":50,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:13:14.846691+0000","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17414,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4257,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":123,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.928762+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:09:32.928762+0000","last_peered":"2024-09-10T16:09:32.928762+0000","last_clean":"2024-09-10T16:09:32.928762+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:09:32.928762+0000","last_undegraded":"2024-09-10T16:09:32.928762+0000","last_fullsized":"2024-09-10T16:09:32.928762+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20781","reported_seq":20935,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:32.931013+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:09:32.931013+0000","last_peered":"2024-09-10T16:09:32.931013+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:09:32.931013+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18461,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-4303}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":99,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.929342+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:09:32.929342+0000","last_peered":"2024-09-10T16:09:32.929342+0000","last_clean":"2024-09-10T16:09:32.929342+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:09:32.929342+0000","last_undegraded":"2024-09-10T16:09:32.929342+0000","last_fullsized":"2024-09-10T16:09:32.929342+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'21170","reported_seq":30033,"reported_epoch":53,"state":"active+recovery_wait","last_fresh":"2024-09-10T16:09:32.930698+0000","last_change":"2024-09-10T16:08:10.929169+0000","last_active":"2024-09-10T16:09:32.930698+0000","last_peered":"2024-09-10T16:09:32.930698+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T16:08:10.860438+0000","last_became_peered":"2024-09-10T16:08:10.860438+0000","last_unstale":"2024-09-10T16:09:32.930698+0000","last_undegraded":"2024-09-10T16:09:32.930698+0000","last_fullsized":"2024-09-10T16:09:32.930698+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18724,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","0","1"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20838","reported_seq":20992,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:32.930297+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:09:32.930297+0000","last_peered":"2024-09-10T16:09:32.930297+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:09:32.930297+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17469,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.929280+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:09:32.929280+0000","last_peered":"2024-09-10T16:09:32.929280+0000","last_clean":"2024-09-10T16:09:32.929280+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:09:32.929280+0000","last_undegraded":"2024-09-10T16:09:32.929280+0000","last_fullsized":"2024-09-10T16:09:32.929280+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20598","reported_seq":29183,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.929268+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:09:32.929268+0000","last_peered":"2024-09-10T16:09:32.929268+0000","last_clean":"2024-09-10T16:09:32.929268+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:09:32.929268+0000","last_undegraded":"2024-09-10T16:09:32.929268+0000","last_fullsized":"2024-09-10T16:09:32.929268+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16988,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":113,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.933167+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:09:32.933167+0000","last_peered":"2024-09-10T16:09:32.933167+0000","last_clean":"2024-09-10T16:09:32.933167+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:09:32.933167+0000","last_undegraded":"2024-09-10T16:09:32.933167+0000","last_fullsized":"2024-09-10T16:09:32.933167+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'21048","reported_seq":21209,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:32.931059+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:09:32.931059+0000","last_peered":"2024-09-10T16:09:32.931059+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:09:32.931059+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17303,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":88,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.929129+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:09:32.929129+0000","last_peered":"2024-09-10T16:09:32.929129+0000","last_clean":"2024-09-10T16:09:32.929129+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:09:32.929129+0000","last_undegraded":"2024-09-10T16:09:32.929129+0000","last_fullsized":"2024-09-10T16:09:32.929129+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":93,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.929492+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:09:32.929492+0000","last_peered":"2024-09-10T16:09:32.929492+0000","last_clean":"2024-09-10T16:09:32.929492+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:09:32.929492+0000","last_undegraded":"2024-09-10T16:09:32.929492+0000","last_fullsized":"2024-09-10T16:09:32.929492+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'21198","reported_seq":21362,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:32.931857+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:09:32.931857+0000","last_peered":"2024-09-10T16:09:32.931857+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:09:32.931857+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17361,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4321},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":27812,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292096,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23578,"num_bytes_recovered":33096163,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":210518016,"data_stored":210395174,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161022,"ondisk_log_size":161022,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986317,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":502632,"kb_used_data":137744,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93869208,"statfs":{"total":96636764160,"available":96122068992,"internally_reserved":0,"allocated":141049856,"data_stored":140719591,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659262,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":473660,"kb_used_data":492,"kb_used_omap":4,"kb_used_meta":473147,"kb_avail":93898180,"statfs":{"total":96636764160,"available":96151736320,"internally_reserved":0,"allocated":503808,"data_stored":155014,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":484503214},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182545,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":547332,"kb_used_data":68652,"kb_used_omap":4,"kb_used_meta":478651,"kb_avail":93824508,"statfs":{"total":96636764160,"available":96076296192,"internally_reserved":0,"allocated":70299648,"data_stored":69965306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":490139306},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":69853184,"data_stored":69817054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":6762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":140607488,"data_stored":140571358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:09:51.294 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:09:51.640 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:09:51.641 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:09:51.653 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":423,"stamp":"2024-09-10T16:09:49.777295+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":27812,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292098,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23580,"num_bytes_recovered":33096182,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161025,"ondisk_log_size":161025,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1523624,"kb_used_data":206888,"kb_used_omap":12,"kb_used_meta":1316659,"kb_avail":281591896,"statfs":{"total":289910292480,"available":288350101504,"internally_reserved":0,"allocated":211853312,"data_stored":210839911,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1348258818},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1691,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1691,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002091"},"pg_stats":[{"pgid":"2.f","version":"49'20672","reported_seq":25495,"reported_epoch":53,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:46.207166+0000","last_change":"2024-09-10T16:09:31.772779+0000","last_active":"2024-09-10T16:09:46.207166+0000","last_peered":"2024-09-10T16:09:46.207166+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:09:46.207166+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1895,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20672,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2364,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1895}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20844","reported_seq":20976,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.933194+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:09:32.933194+0000","last_peered":"2024-09-10T16:09:32.933194+0000","last_clean":"2024-09-10T16:09:32.933194+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:09:32.933194+0000","last_undegraded":"2024-09-10T16:09:32.933194+0000","last_fullsized":"2024-09-10T16:09:32.933194+0000","mapping_epoch":35,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18949,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20798","reported_seq":29495,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.929401+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:09:32.929401+0000","last_peered":"2024-09-10T16:09:32.929401+0000","last_clean":"2024-09-10T16:09:32.929401+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:09:32.929401+0000","last_undegraded":"2024-09-10T16:09:32.929401+0000","last_fullsized":"2024-09-10T16:09:32.929401+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18802,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20660","reported_seq":20760,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.933021+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:09:32.933021+0000","last_peered":"2024-09-10T16:09:32.933021+0000","last_clean":"2024-09-10T16:09:32.933021+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:09:32.933021+0000","last_undegraded":"2024-09-10T16:09:32.933021+0000","last_fullsized":"2024-09-10T16:09:32.933021+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20660,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20891","reported_seq":21040,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.929877+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:09:32.929877+0000","last_peered":"2024-09-10T16:09:32.929877+0000","last_clean":"2024-09-10T16:09:32.929877+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:09:32.929877+0000","last_undegraded":"2024-09-10T16:09:32.929877+0000","last_fullsized":"2024-09-10T16:09:32.929877+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17602,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'21061","reported_seq":21218,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:32.929429+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:09:32.929429+0000","last_peered":"2024-09-10T16:09:32.929429+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:09:32.929429+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17673,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4370},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'21082","reported_seq":21239,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:32.928987+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:09:32.928987+0000","last_peered":"2024-09-10T16:09:32.928987+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:09:32.928987+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17581,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4235},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'21314","reported_seq":21463,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.928707+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:09:32.928707+0000","last_peered":"2024-09-10T16:09:32.928707+0000","last_clean":"2024-09-10T16:09:32.928707+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:09:32.928707+0000","last_undegraded":"2024-09-10T16:09:32.928707+0000","last_fullsized":"2024-09-10T16:09:32.928707+0000","mapping_epoch":34,"log_start":"45'11300","ondisk_log_start":"45'11300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18161,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.929057+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:09:32.929057+0000","last_peered":"2024-09-10T16:09:32.929057+0000","last_clean":"2024-09-10T16:09:32.929057+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:09:32.929057+0000","last_undegraded":"2024-09-10T16:09:32.929057+0000","last_fullsized":"2024-09-10T16:09:32.929057+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20457","reported_seq":20620,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:33.013394+0000","last_change":"2024-09-10T16:09:33.012840+0000","last_active":"2024-09-10T16:09:33.013394+0000","last_peered":"2024-09-10T16:09:33.013394+0000","last_clean":"2024-09-10T16:09:33.013394+0000","last_became_active":"2024-09-10T16:09:32.934056+0000","last_became_peered":"2024-09-10T16:09:32.934056+0000","last_unstale":"2024-09-10T16:09:33.013394+0000","last_undegraded":"2024-09-10T16:09:33.013394+0000","last_fullsized":"2024-09-10T16:09:33.013394+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:08:30.481235+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18276,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":4579,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":113,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.930929+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:09:32.930929+0000","last_peered":"2024-09-10T16:09:32.930929+0000","last_clean":"2024-09-10T16:09:32.930929+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:09:32.930929+0000","last_undegraded":"2024-09-10T16:09:32.930929+0000","last_fullsized":"2024-09-10T16:09:32.930929+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20910","reported_seq":29582,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.929808+0000","last_change":"2024-09-10T16:09:08.909580+0000","last_active":"2024-09-10T16:09:32.929808+0000","last_peered":"2024-09-10T16:09:32.929808+0000","last_clean":"2024-09-10T16:09:32.929808+0000","last_became_active":"2024-09-10T16:09:08.854941+0000","last_became_peered":"2024-09-10T16:09:08.854941+0000","last_unstale":"2024-09-10T16:09:32.929808+0000","last_undegraded":"2024-09-10T16:09:32.929808+0000","last_fullsized":"2024-09-10T16:09:32.929808+0000","mapping_epoch":50,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:13:14.846691+0000","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17414,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4257,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":123,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.928762+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:09:32.928762+0000","last_peered":"2024-09-10T16:09:32.928762+0000","last_clean":"2024-09-10T16:09:32.928762+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:09:32.928762+0000","last_undegraded":"2024-09-10T16:09:32.928762+0000","last_fullsized":"2024-09-10T16:09:32.928762+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20781","reported_seq":20935,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:32.931013+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:09:32.931013+0000","last_peered":"2024-09-10T16:09:32.931013+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:09:32.931013+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18461,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-4303}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":99,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.929342+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:09:32.929342+0000","last_peered":"2024-09-10T16:09:32.929342+0000","last_clean":"2024-09-10T16:09:32.929342+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:09:32.929342+0000","last_undegraded":"2024-09-10T16:09:32.929342+0000","last_fullsized":"2024-09-10T16:09:32.929342+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'21170","reported_seq":30033,"reported_epoch":53,"state":"active+recovery_wait","last_fresh":"2024-09-10T16:09:32.930698+0000","last_change":"2024-09-10T16:08:10.929169+0000","last_active":"2024-09-10T16:09:32.930698+0000","last_peered":"2024-09-10T16:09:32.930698+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T16:08:10.860438+0000","last_became_peered":"2024-09-10T16:08:10.860438+0000","last_unstale":"2024-09-10T16:09:32.930698+0000","last_undegraded":"2024-09-10T16:09:32.930698+0000","last_fullsized":"2024-09-10T16:09:32.930698+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18724,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","0","1"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20838","reported_seq":20992,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:32.930297+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:09:32.930297+0000","last_peered":"2024-09-10T16:09:32.930297+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:09:32.930297+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17469,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.929280+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:09:32.929280+0000","last_peered":"2024-09-10T16:09:32.929280+0000","last_clean":"2024-09-10T16:09:32.929280+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:09:32.929280+0000","last_undegraded":"2024-09-10T16:09:32.929280+0000","last_fullsized":"2024-09-10T16:09:32.929280+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20598","reported_seq":29183,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.929268+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:09:32.929268+0000","last_peered":"2024-09-10T16:09:32.929268+0000","last_clean":"2024-09-10T16:09:32.929268+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:09:32.929268+0000","last_undegraded":"2024-09-10T16:09:32.929268+0000","last_fullsized":"2024-09-10T16:09:32.929268+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16988,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":113,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.933167+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:09:32.933167+0000","last_peered":"2024-09-10T16:09:32.933167+0000","last_clean":"2024-09-10T16:09:32.933167+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:09:32.933167+0000","last_undegraded":"2024-09-10T16:09:32.933167+0000","last_fullsized":"2024-09-10T16:09:32.933167+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'21048","reported_seq":21209,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:32.931059+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:09:32.931059+0000","last_peered":"2024-09-10T16:09:32.931059+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:09:32.931059+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17303,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":88,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.929129+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:09:32.929129+0000","last_peered":"2024-09-10T16:09:32.929129+0000","last_clean":"2024-09-10T16:09:32.929129+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:09:32.929129+0000","last_undegraded":"2024-09-10T16:09:32.929129+0000","last_fullsized":"2024-09-10T16:09:32.929129+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":93,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.929492+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:09:32.929492+0000","last_peered":"2024-09-10T16:09:32.929492+0000","last_clean":"2024-09-10T16:09:32.929492+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:09:32.929492+0000","last_undegraded":"2024-09-10T16:09:32.929492+0000","last_fullsized":"2024-09-10T16:09:32.929492+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'21198","reported_seq":21362,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:32.931857+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:09:32.931857+0000","last_peered":"2024-09-10T16:09:32.931857+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:09:32.931857+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17361,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4321},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":27812,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292096,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23578,"num_bytes_recovered":33096163,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":210518016,"data_stored":210395174,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161022,"ondisk_log_size":161022,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986317,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":502632,"kb_used_data":137744,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93869208,"statfs":{"total":96636764160,"available":96122068992,"internally_reserved":0,"allocated":141049856,"data_stored":140719591,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659262,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":473660,"kb_used_data":492,"kb_used_omap":4,"kb_used_meta":473147,"kb_avail":93898180,"statfs":{"total":96636764160,"available":96151736320,"internally_reserved":0,"allocated":503808,"data_stored":155014,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":484503214},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182545,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":547332,"kb_used_data":68652,"kb_used_omap":4,"kb_used_meta":478651,"kb_avail":93824508,"statfs":{"total":96636764160,"available":96076296192,"internally_reserved":0,"allocated":70299648,"data_stored":69965306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":490139306},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":69853184,"data_stored":69817054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":6762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":140607488,"data_stored":140571358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:09:51.654 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:09:51.962 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:09:51.962 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:09:51.974 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":424,"stamp":"2024-09-10T16:09:51.777605+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":26944,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292098,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24448,"num_bytes_recovered":33096182,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161025,"ondisk_log_size":161025,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1515304,"kb_used_data":206888,"kb_used_omap":12,"kb_used_meta":1308339,"kb_avail":281600216,"statfs":{"total":289910292480,"available":288358621184,"internally_reserved":0,"allocated":211853312,"data_stored":210839911,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1339739138},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2559,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2559,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001961"},"pg_stats":[{"pgid":"2.f","version":"49'20672","reported_seq":27231,"reported_epoch":53,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:51.211039+0000","last_change":"2024-09-10T16:09:31.772779+0000","last_active":"2024-09-10T16:09:51.211039+0000","last_peered":"2024-09-10T16:09:51.211039+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:09:51.211039+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1027,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20672,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3232,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1027}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20844","reported_seq":20976,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.933194+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:09:32.933194+0000","last_peered":"2024-09-10T16:09:32.933194+0000","last_clean":"2024-09-10T16:09:32.933194+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:09:32.933194+0000","last_undegraded":"2024-09-10T16:09:32.933194+0000","last_fullsized":"2024-09-10T16:09:32.933194+0000","mapping_epoch":35,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18949,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20798","reported_seq":29495,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.929401+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:09:32.929401+0000","last_peered":"2024-09-10T16:09:32.929401+0000","last_clean":"2024-09-10T16:09:32.929401+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:09:32.929401+0000","last_undegraded":"2024-09-10T16:09:32.929401+0000","last_fullsized":"2024-09-10T16:09:32.929401+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18802,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20660","reported_seq":20760,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.933021+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:09:32.933021+0000","last_peered":"2024-09-10T16:09:32.933021+0000","last_clean":"2024-09-10T16:09:32.933021+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:09:32.933021+0000","last_undegraded":"2024-09-10T16:09:32.933021+0000","last_fullsized":"2024-09-10T16:09:32.933021+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20660,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20891","reported_seq":21040,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.929877+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:09:32.929877+0000","last_peered":"2024-09-10T16:09:32.929877+0000","last_clean":"2024-09-10T16:09:32.929877+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:09:32.929877+0000","last_undegraded":"2024-09-10T16:09:32.929877+0000","last_fullsized":"2024-09-10T16:09:32.929877+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17602,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'21061","reported_seq":21218,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:32.929429+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:09:32.929429+0000","last_peered":"2024-09-10T16:09:32.929429+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:09:32.929429+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17673,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4370},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'21082","reported_seq":21239,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:32.928987+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:09:32.928987+0000","last_peered":"2024-09-10T16:09:32.928987+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:09:32.928987+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17581,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4235},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'21314","reported_seq":21463,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.928707+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:09:32.928707+0000","last_peered":"2024-09-10T16:09:32.928707+0000","last_clean":"2024-09-10T16:09:32.928707+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:09:32.928707+0000","last_undegraded":"2024-09-10T16:09:32.928707+0000","last_fullsized":"2024-09-10T16:09:32.928707+0000","mapping_epoch":34,"log_start":"45'11300","ondisk_log_start":"45'11300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18161,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.929057+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:09:32.929057+0000","last_peered":"2024-09-10T16:09:32.929057+0000","last_clean":"2024-09-10T16:09:32.929057+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:09:32.929057+0000","last_undegraded":"2024-09-10T16:09:32.929057+0000","last_fullsized":"2024-09-10T16:09:32.929057+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20457","reported_seq":20620,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:33.013394+0000","last_change":"2024-09-10T16:09:33.012840+0000","last_active":"2024-09-10T16:09:33.013394+0000","last_peered":"2024-09-10T16:09:33.013394+0000","last_clean":"2024-09-10T16:09:33.013394+0000","last_became_active":"2024-09-10T16:09:32.934056+0000","last_became_peered":"2024-09-10T16:09:32.934056+0000","last_unstale":"2024-09-10T16:09:33.013394+0000","last_undegraded":"2024-09-10T16:09:33.013394+0000","last_fullsized":"2024-09-10T16:09:33.013394+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:08:30.481235+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18276,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":4579,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":113,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.930929+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:09:32.930929+0000","last_peered":"2024-09-10T16:09:32.930929+0000","last_clean":"2024-09-10T16:09:32.930929+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:09:32.930929+0000","last_undegraded":"2024-09-10T16:09:32.930929+0000","last_fullsized":"2024-09-10T16:09:32.930929+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20910","reported_seq":29582,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.929808+0000","last_change":"2024-09-10T16:09:08.909580+0000","last_active":"2024-09-10T16:09:32.929808+0000","last_peered":"2024-09-10T16:09:32.929808+0000","last_clean":"2024-09-10T16:09:32.929808+0000","last_became_active":"2024-09-10T16:09:08.854941+0000","last_became_peered":"2024-09-10T16:09:08.854941+0000","last_unstale":"2024-09-10T16:09:32.929808+0000","last_undegraded":"2024-09-10T16:09:32.929808+0000","last_fullsized":"2024-09-10T16:09:32.929808+0000","mapping_epoch":50,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:13:14.846691+0000","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17414,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4257,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":123,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.928762+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:09:32.928762+0000","last_peered":"2024-09-10T16:09:32.928762+0000","last_clean":"2024-09-10T16:09:32.928762+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:09:32.928762+0000","last_undegraded":"2024-09-10T16:09:32.928762+0000","last_fullsized":"2024-09-10T16:09:32.928762+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20781","reported_seq":20935,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:32.931013+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:09:32.931013+0000","last_peered":"2024-09-10T16:09:32.931013+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:09:32.931013+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18461,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-4303}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":99,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.929342+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:09:32.929342+0000","last_peered":"2024-09-10T16:09:32.929342+0000","last_clean":"2024-09-10T16:09:32.929342+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:09:32.929342+0000","last_undegraded":"2024-09-10T16:09:32.929342+0000","last_fullsized":"2024-09-10T16:09:32.929342+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'21170","reported_seq":30033,"reported_epoch":53,"state":"active+recovery_wait","last_fresh":"2024-09-10T16:09:32.930698+0000","last_change":"2024-09-10T16:08:10.929169+0000","last_active":"2024-09-10T16:09:32.930698+0000","last_peered":"2024-09-10T16:09:32.930698+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T16:08:10.860438+0000","last_became_peered":"2024-09-10T16:08:10.860438+0000","last_unstale":"2024-09-10T16:09:32.930698+0000","last_undegraded":"2024-09-10T16:09:32.930698+0000","last_fullsized":"2024-09-10T16:09:32.930698+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18724,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","0","1"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20838","reported_seq":20992,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:32.930297+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:09:32.930297+0000","last_peered":"2024-09-10T16:09:32.930297+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:09:32.930297+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17469,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.929280+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:09:32.929280+0000","last_peered":"2024-09-10T16:09:32.929280+0000","last_clean":"2024-09-10T16:09:32.929280+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:09:32.929280+0000","last_undegraded":"2024-09-10T16:09:32.929280+0000","last_fullsized":"2024-09-10T16:09:32.929280+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20598","reported_seq":29183,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.929268+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:09:32.929268+0000","last_peered":"2024-09-10T16:09:32.929268+0000","last_clean":"2024-09-10T16:09:32.929268+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:09:32.929268+0000","last_undegraded":"2024-09-10T16:09:32.929268+0000","last_fullsized":"2024-09-10T16:09:32.929268+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16988,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":113,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.933167+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:09:32.933167+0000","last_peered":"2024-09-10T16:09:32.933167+0000","last_clean":"2024-09-10T16:09:32.933167+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:09:32.933167+0000","last_undegraded":"2024-09-10T16:09:32.933167+0000","last_fullsized":"2024-09-10T16:09:32.933167+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'21048","reported_seq":21209,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:32.931059+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:09:32.931059+0000","last_peered":"2024-09-10T16:09:32.931059+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:09:32.931059+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17303,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.929172+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:09:32.929172+0000","last_peered":"2024-09-10T16:09:32.929172+0000","last_clean":"2024-09-10T16:09:32.929172+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:09:32.929172+0000","last_undegraded":"2024-09-10T16:09:32.929172+0000","last_fullsized":"2024-09-10T16:09:32.929172+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":93,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.929492+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:09:32.929492+0000","last_peered":"2024-09-10T16:09:32.929492+0000","last_clean":"2024-09-10T16:09:32.929492+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:09:32.929492+0000","last_undegraded":"2024-09-10T16:09:32.929492+0000","last_fullsized":"2024-09-10T16:09:32.929492+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'21198","reported_seq":21362,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:32.931857+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:09:32.931857+0000","last_peered":"2024-09-10T16:09:32.931857+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:09:32.931857+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17361,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4321},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":26944,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292096,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24446,"num_bytes_recovered":33096163,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":210518016,"data_stored":210395174,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161022,"ondisk_log_size":161022,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986318,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":502632,"kb_used_data":137744,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93869208,"statfs":{"total":96636764160,"available":96122068992,"internally_reserved":0,"allocated":141049856,"data_stored":140719591,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659263,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":465340,"kb_used_data":492,"kb_used_omap":4,"kb_used_meta":464827,"kb_avail":93906500,"statfs":{"total":96636764160,"available":96160256000,"internally_reserved":0,"allocated":503808,"data_stored":155014,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":475983534},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182546,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":547332,"kb_used_data":68652,"kb_used_omap":4,"kb_used_meta":478651,"kb_avail":93824508,"statfs":{"total":96636764160,"available":96076296192,"internally_reserved":0,"allocated":70299648,"data_stored":69965306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":490139306},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":69853184,"data_stored":69817054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":6762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":140607488,"data_stored":140571358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:09:51.976 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-10T16:09:51.976 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "49'20672", 'reported_seq': 27231, 'reported_epoch': 53, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:09:51.211039+0000', 'last_change': '2024-09-10T16:09:31.772779+0000', 'last_active': '2024-09-10T16:09:51.211039+0000', 'last_peered': '2024-09-10T16:09:51.211039+0000', 'last_clean': '2024-09-10T15:59:14.132456+0000', 'last_became_active': '2024-09-10T15:59:16.274407+0000', 'last_became_peered': '2024-09-10T15:59:16.274407+0000', 'last_unstale': '2024-09-10T16:09:51.211039+0000', 'last_undegraded': '2024-09-10T15:59:15.851279+0000', 'last_fullsized': '2024-09-10T15:59:15.774356+0000', 'mapping_epoch': 42, 'log_start': "45'10600", 'ondisk_log_start': "45'10600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'objects_scrubbed': 0, 'log_size': 10072, 'log_dups_size': 0, 'ondisk_log_size': 10072, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1027, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20672, 'num_write_kb': 41344, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3232, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1027}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:09:51.976 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-10T16:09:51.976 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "49'21061", 'reported_seq': 21218, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:09:32.929429+0000', 'last_change': '2024-09-10T15:59:16.308264+0000', 'last_active': '2024-09-10T16:09:32.929429+0000', 'last_peered': '2024-09-10T16:09:32.929429+0000', 'last_clean': '2024-09-10T15:59:14.111551+0000', 'last_became_active': '2024-09-10T15:59:16.300040+0000', 'last_became_peered': '2024-09-10T15:59:16.300040+0000', 'last_unstale': '2024-09-10T16:09:32.929429+0000', 'last_undegraded': '2024-09-10T15:59:15.878234+0000', 'last_fullsized': '2024-09-10T15:59:15.769328+0000', 'mapping_epoch': 42, 'log_start': "45'11000", 'ondisk_log_start': "45'11000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3442", 'last_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'objects_scrubbed': 3384, 'log_size': 10061, 'log_dups_size': 0, 'ondisk_log_size': 10061, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1292, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4372, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 17673, 'num_write_kb': 28580, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4370}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:09:51.976 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-10T16:09:51.976 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "49'21082", 'reported_seq': 21239, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:09:32.928987+0000', 'last_change': '2024-09-10T15:59:16.331899+0000', 'last_active': '2024-09-10T16:09:32.928987+0000', 'last_peered': '2024-09-10T16:09:32.928987+0000', 'last_clean': '2024-09-10T15:59:14.139436+0000', 'last_became_active': '2024-09-10T15:59:16.318715+0000', 'last_became_peered': '2024-09-10T15:59:16.318715+0000', 'last_unstale': '2024-09-10T16:09:32.928987+0000', 'last_undegraded': '2024-09-10T15:59:15.877524+0000', 'last_fullsized': '2024-09-10T15:59:15.769042+0000', 'mapping_epoch': 42, 'log_start': "45'11000", 'ondisk_log_start': "45'11000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3537", 'last_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'objects_scrubbed': 3497, 'log_size': 10082, 'log_dups_size': 0, 'ondisk_log_size': 10082, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1224, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4237, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 17581, 'num_write_kb': 28172, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4235}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:09:51.976 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-10T16:09:51.976 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "49'20781", 'reported_seq': 20935, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:09:32.931013+0000', 'last_change': '2024-09-10T15:59:16.284311+0000', 'last_active': '2024-09-10T16:09:32.931013+0000', 'last_peered': '2024-09-10T16:09:32.931013+0000', 'last_clean': '2024-09-10T15:59:14.116041+0000', 'last_became_active': '2024-09-10T15:59:16.274543+0000', 'last_became_peered': '2024-09-10T15:59:16.274543+0000', 'last_unstale': '2024-09-10T16:09:32.931013+0000', 'last_undegraded': '2024-09-10T15:59:15.812484+0000', 'last_fullsized': '2024-09-10T15:59:15.770885+0000', 'mapping_epoch': 42, 'log_start': "45'10700", 'ondisk_log_start': "45'10700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2356", 'last_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'objects_scrubbed': 2316, 'log_size': 10081, 'log_dups_size': 0, 'ondisk_log_size': 10081, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 783, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 18461, 'num_write_kb': 32292, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': -4303}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:09:51.977 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-10T16:09:51.977 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "49'21170", 'reported_seq': 30033, 'reported_epoch': 53, 'state': 'active+recovery_wait', 'last_fresh': '2024-09-10T16:09:32.930698+0000', 'last_change': '2024-09-10T16:08:10.929169+0000', 'last_active': '2024-09-10T16:09:32.930698+0000', 'last_peered': '2024-09-10T16:09:32.930698+0000', 'last_clean': '2024-09-10T15:59:14.110654+0000', 'last_became_active': '2024-09-10T16:08:10.860438+0000', 'last_became_peered': '2024-09-10T16:08:10.860438+0000', 'last_unstale': '2024-09-10T16:09:32.930698+0000', 'last_undegraded': '2024-09-10T16:09:32.930698+0000', 'last_fullsized': '2024-09-10T16:09:32.930698+0000', 'mapping_epoch': 48, 'log_start': "45'11100", 'ondisk_log_start': "45'11100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2486", 'last_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'objects_scrubbed': 2440, 'log_size': 10070, 'log_dups_size': 0, 'ondisk_log_size': 10070, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 792, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18724, 'num_write_kb': 32568, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 4355, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': ['2', '0', '1'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:09:51.977 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-10T16:09:51.977 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "49'20838", 'reported_seq': 20992, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:09:32.930297+0000', 'last_change': '2024-09-10T15:59:16.315641+0000', 'last_active': '2024-09-10T16:09:32.930297+0000', 'last_peered': '2024-09-10T16:09:32.930297+0000', 'last_clean': '2024-09-10T15:59:14.117001+0000', 'last_became_active': '2024-09-10T15:59:16.302245+0000', 'last_became_peered': '2024-09-10T15:59:16.302245+0000', 'last_unstale': '2024-09-10T16:09:32.930297+0000', 'last_undegraded': '2024-09-10T15:59:15.845012+0000', 'last_fullsized': '2024-09-10T15:59:15.770337+0000', 'mapping_epoch': 42, 'log_start': "45'10800", 'ondisk_log_start': "45'10800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3404", 'last_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'objects_scrubbed': 3367, 'log_size': 10038, 'log_dups_size': 0, 'ondisk_log_size': 10038, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 983, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17469, 'num_write_kb': 28204, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4304}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:09:51.977 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-10T16:09:51.977 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "49'21048", 'reported_seq': 21209, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:09:32.931059+0000', 'last_change': '2024-09-10T15:59:16.311291+0000', 'last_active': '2024-09-10T16:09:32.931059+0000', 'last_peered': '2024-09-10T16:09:32.931059+0000', 'last_clean': '2024-09-10T15:59:14.116363+0000', 'last_became_active': '2024-09-10T15:59:16.301244+0000', 'last_became_peered': '2024-09-10T15:59:16.301244+0000', 'last_unstale': '2024-09-10T16:09:32.931059+0000', 'last_undegraded': '2024-09-10T15:59:15.883434+0000', 'last_fullsized': '2024-09-10T15:59:15.772756+0000', 'mapping_epoch': 42, 'log_start': "45'11000", 'ondisk_log_start': "45'11000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3789", 'last_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'objects_scrubbed': 3741, 'log_size': 10048, 'log_dups_size': 0, 'ondisk_log_size': 10048, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1440, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4373, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17303, 'num_write_kb': 27124, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4371}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:09:51.977 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-10T16:09:51.977 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "49'21198", 'reported_seq': 21362, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:09:32.931857+0000', 'last_change': '2024-09-10T15:59:16.313409+0000', 'last_active': '2024-09-10T16:09:32.931857+0000', 'last_peered': '2024-09-10T16:09:32.931857+0000', 'last_clean': '2024-09-10T15:59:14.129780+0000', 'last_became_active': '2024-09-10T15:59:16.300932+0000', 'last_became_peered': '2024-09-10T15:59:16.300932+0000', 'last_unstale': '2024-09-10T16:09:32.931857+0000', 'last_undegraded': '2024-09-10T15:59:15.887823+0000', 'last_fullsized': '2024-09-10T15:59:15.768513+0000', 'mapping_epoch': 42, 'log_start': "45'11100", 'ondisk_log_start': "45'11100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3880", 'last_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'objects_scrubbed': 3827, 'log_size': 10098, 'log_dups_size': 0, 'ondisk_log_size': 10098, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1397, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4325, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17361, 'num_write_kb': 27072, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4321}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:09:51.978 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T16:09:52.352 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:09:52.352 INFO:teuthology.orchestra.run.smithi115.stdout:{"fsid":"7effee29-df05-4351-9741-ee203564ed81","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 26944/24 objects degraded (112266.667%), 7 pgs degraded, 7 pgs undersized","count":14},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":756,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":53,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983954,"num_in_osds":3,"osd_in_since":1725983947,"num_remapped_pgs":7},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":16},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":6},{"state_name":"active+recovery_wait","count":1},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":12,"data_bytes":4849,"bytes_used":1551671296,"bytes_avail":288358621184,"bytes_total":289910292480,"degraded_objects":26944,"degraded_total":24,"degraded_ratio":1122.6666666666667,"recovering_objects_per_sec":213,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":2559,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-10T15:57:14:967120+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T16:00:13.560776+0000","services":{}},"progress_events":{}} 2024-09-10T16:09:52.365 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-10T16:09:52.366 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:09:52.668 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:09:52.669 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:09:52.681 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":424,"stamp":"2024-09-10T16:09:51.777605+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":26944,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292098,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24448,"num_bytes_recovered":33096182,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161025,"ondisk_log_size":161025,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1515304,"kb_used_data":206888,"kb_used_omap":12,"kb_used_meta":1308339,"kb_avail":281600216,"statfs":{"total":289910292480,"available":288358621184,"internally_reserved":0,"allocated":211853312,"data_stored":210839911,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1339739138},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2559,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2559,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001961"},"pg_stats":[{"pgid":"2.f","version":"49'20672","reported_seq":27231,"reported_epoch":53,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:51.211039+0000","last_change":"2024-09-10T16:09:31.772779+0000","last_active":"2024-09-10T16:09:51.211039+0000","last_peered":"2024-09-10T16:09:51.211039+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:09:51.211039+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1027,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20672,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3232,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1027}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20844","reported_seq":20976,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.933194+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:09:32.933194+0000","last_peered":"2024-09-10T16:09:32.933194+0000","last_clean":"2024-09-10T16:09:32.933194+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:09:32.933194+0000","last_undegraded":"2024-09-10T16:09:32.933194+0000","last_fullsized":"2024-09-10T16:09:32.933194+0000","mapping_epoch":35,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18949,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20798","reported_seq":29495,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.929401+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:09:32.929401+0000","last_peered":"2024-09-10T16:09:32.929401+0000","last_clean":"2024-09-10T16:09:32.929401+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:09:32.929401+0000","last_undegraded":"2024-09-10T16:09:32.929401+0000","last_fullsized":"2024-09-10T16:09:32.929401+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18802,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20660","reported_seq":20760,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.933021+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:09:32.933021+0000","last_peered":"2024-09-10T16:09:32.933021+0000","last_clean":"2024-09-10T16:09:32.933021+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:09:32.933021+0000","last_undegraded":"2024-09-10T16:09:32.933021+0000","last_fullsized":"2024-09-10T16:09:32.933021+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20660,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20891","reported_seq":21040,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.929877+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:09:32.929877+0000","last_peered":"2024-09-10T16:09:32.929877+0000","last_clean":"2024-09-10T16:09:32.929877+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:09:32.929877+0000","last_undegraded":"2024-09-10T16:09:32.929877+0000","last_fullsized":"2024-09-10T16:09:32.929877+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17602,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'21061","reported_seq":21218,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:32.929429+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:09:32.929429+0000","last_peered":"2024-09-10T16:09:32.929429+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:09:32.929429+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17673,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4370},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'21082","reported_seq":21239,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:32.928987+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:09:32.928987+0000","last_peered":"2024-09-10T16:09:32.928987+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:09:32.928987+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17581,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4235},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'21314","reported_seq":21463,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.928707+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:09:32.928707+0000","last_peered":"2024-09-10T16:09:32.928707+0000","last_clean":"2024-09-10T16:09:32.928707+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:09:32.928707+0000","last_undegraded":"2024-09-10T16:09:32.928707+0000","last_fullsized":"2024-09-10T16:09:32.928707+0000","mapping_epoch":34,"log_start":"45'11300","ondisk_log_start":"45'11300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18161,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.929057+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:09:32.929057+0000","last_peered":"2024-09-10T16:09:32.929057+0000","last_clean":"2024-09-10T16:09:32.929057+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:09:32.929057+0000","last_undegraded":"2024-09-10T16:09:32.929057+0000","last_fullsized":"2024-09-10T16:09:32.929057+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20457","reported_seq":20620,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:33.013394+0000","last_change":"2024-09-10T16:09:33.012840+0000","last_active":"2024-09-10T16:09:33.013394+0000","last_peered":"2024-09-10T16:09:33.013394+0000","last_clean":"2024-09-10T16:09:33.013394+0000","last_became_active":"2024-09-10T16:09:32.934056+0000","last_became_peered":"2024-09-10T16:09:32.934056+0000","last_unstale":"2024-09-10T16:09:33.013394+0000","last_undegraded":"2024-09-10T16:09:33.013394+0000","last_fullsized":"2024-09-10T16:09:33.013394+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:08:30.481235+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18276,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":4579,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":113,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.930929+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:09:32.930929+0000","last_peered":"2024-09-10T16:09:32.930929+0000","last_clean":"2024-09-10T16:09:32.930929+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:09:32.930929+0000","last_undegraded":"2024-09-10T16:09:32.930929+0000","last_fullsized":"2024-09-10T16:09:32.930929+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20910","reported_seq":29582,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.929808+0000","last_change":"2024-09-10T16:09:08.909580+0000","last_active":"2024-09-10T16:09:32.929808+0000","last_peered":"2024-09-10T16:09:32.929808+0000","last_clean":"2024-09-10T16:09:32.929808+0000","last_became_active":"2024-09-10T16:09:08.854941+0000","last_became_peered":"2024-09-10T16:09:08.854941+0000","last_unstale":"2024-09-10T16:09:32.929808+0000","last_undegraded":"2024-09-10T16:09:32.929808+0000","last_fullsized":"2024-09-10T16:09:32.929808+0000","mapping_epoch":50,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:13:14.846691+0000","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17414,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4257,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":123,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.928762+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:09:32.928762+0000","last_peered":"2024-09-10T16:09:32.928762+0000","last_clean":"2024-09-10T16:09:32.928762+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:09:32.928762+0000","last_undegraded":"2024-09-10T16:09:32.928762+0000","last_fullsized":"2024-09-10T16:09:32.928762+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20781","reported_seq":20935,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:32.931013+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:09:32.931013+0000","last_peered":"2024-09-10T16:09:32.931013+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:09:32.931013+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18461,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-4303}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":99,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.929342+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:09:32.929342+0000","last_peered":"2024-09-10T16:09:32.929342+0000","last_clean":"2024-09-10T16:09:32.929342+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:09:32.929342+0000","last_undegraded":"2024-09-10T16:09:32.929342+0000","last_fullsized":"2024-09-10T16:09:32.929342+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'21170","reported_seq":30033,"reported_epoch":53,"state":"active+recovery_wait","last_fresh":"2024-09-10T16:09:32.930698+0000","last_change":"2024-09-10T16:08:10.929169+0000","last_active":"2024-09-10T16:09:32.930698+0000","last_peered":"2024-09-10T16:09:32.930698+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T16:08:10.860438+0000","last_became_peered":"2024-09-10T16:08:10.860438+0000","last_unstale":"2024-09-10T16:09:32.930698+0000","last_undegraded":"2024-09-10T16:09:32.930698+0000","last_fullsized":"2024-09-10T16:09:32.930698+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18724,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","0","1"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20838","reported_seq":20992,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:32.930297+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:09:32.930297+0000","last_peered":"2024-09-10T16:09:32.930297+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:09:32.930297+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17469,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.929280+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:09:32.929280+0000","last_peered":"2024-09-10T16:09:32.929280+0000","last_clean":"2024-09-10T16:09:32.929280+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:09:32.929280+0000","last_undegraded":"2024-09-10T16:09:32.929280+0000","last_fullsized":"2024-09-10T16:09:32.929280+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20598","reported_seq":29183,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.929268+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:09:32.929268+0000","last_peered":"2024-09-10T16:09:32.929268+0000","last_clean":"2024-09-10T16:09:32.929268+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:09:32.929268+0000","last_undegraded":"2024-09-10T16:09:32.929268+0000","last_fullsized":"2024-09-10T16:09:32.929268+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16988,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":113,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.933167+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:09:32.933167+0000","last_peered":"2024-09-10T16:09:32.933167+0000","last_clean":"2024-09-10T16:09:32.933167+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:09:32.933167+0000","last_undegraded":"2024-09-10T16:09:32.933167+0000","last_fullsized":"2024-09-10T16:09:32.933167+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'21048","reported_seq":21209,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:32.931059+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:09:32.931059+0000","last_peered":"2024-09-10T16:09:32.931059+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:09:32.931059+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17303,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.929172+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:09:32.929172+0000","last_peered":"2024-09-10T16:09:32.929172+0000","last_clean":"2024-09-10T16:09:32.929172+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:09:32.929172+0000","last_undegraded":"2024-09-10T16:09:32.929172+0000","last_fullsized":"2024-09-10T16:09:32.929172+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":93,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.929492+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:09:32.929492+0000","last_peered":"2024-09-10T16:09:32.929492+0000","last_clean":"2024-09-10T16:09:32.929492+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:09:32.929492+0000","last_undegraded":"2024-09-10T16:09:32.929492+0000","last_fullsized":"2024-09-10T16:09:32.929492+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'21198","reported_seq":21362,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:32.931857+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:09:32.931857+0000","last_peered":"2024-09-10T16:09:32.931857+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:09:32.931857+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17361,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4321},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":26944,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292096,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24446,"num_bytes_recovered":33096163,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":210518016,"data_stored":210395174,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161022,"ondisk_log_size":161022,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986318,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":502632,"kb_used_data":137744,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93869208,"statfs":{"total":96636764160,"available":96122068992,"internally_reserved":0,"allocated":141049856,"data_stored":140719591,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659263,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":465340,"kb_used_data":492,"kb_used_omap":4,"kb_used_meta":464827,"kb_avail":93906500,"statfs":{"total":96636764160,"available":96160256000,"internally_reserved":0,"allocated":503808,"data_stored":155014,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":475983534},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182546,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":547332,"kb_used_data":68652,"kb_used_omap":4,"kb_used_meta":478651,"kb_avail":93824508,"statfs":{"total":96636764160,"available":96076296192,"internally_reserved":0,"allocated":70299648,"data_stored":69965306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":490139306},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":69853184,"data_stored":69817054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":6762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":140607488,"data_stored":140571358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:09:55.683 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:09:56.021 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:09:56.021 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:09:56.033 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":427,"stamp":"2024-09-10T16:09:56.000972+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":26944,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292098,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24448,"num_bytes_recovered":33096182,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161025,"ondisk_log_size":161025,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1523624,"kb_used_data":206888,"kb_used_omap":12,"kb_used_meta":1316659,"kb_avail":281591896,"statfs":{"total":289910292480,"available":288350101504,"internally_reserved":0,"allocated":211853312,"data_stored":210839911,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1348258818},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1623,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1623,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.224277"},"pg_stats":[{"pgid":"2.f","version":"49'20672","reported_seq":27231,"reported_epoch":53,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:51.211039+0000","last_change":"2024-09-10T16:09:31.772779+0000","last_active":"2024-09-10T16:09:51.211039+0000","last_peered":"2024-09-10T16:09:51.211039+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:09:51.211039+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1027,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20672,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3232,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1027}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20844","reported_seq":20976,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.933194+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:09:32.933194+0000","last_peered":"2024-09-10T16:09:32.933194+0000","last_clean":"2024-09-10T16:09:32.933194+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:09:32.933194+0000","last_undegraded":"2024-09-10T16:09:32.933194+0000","last_fullsized":"2024-09-10T16:09:32.933194+0000","mapping_epoch":35,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18949,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20798","reported_seq":29495,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.929401+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:09:32.929401+0000","last_peered":"2024-09-10T16:09:32.929401+0000","last_clean":"2024-09-10T16:09:32.929401+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:09:32.929401+0000","last_undegraded":"2024-09-10T16:09:32.929401+0000","last_fullsized":"2024-09-10T16:09:32.929401+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18802,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20660","reported_seq":20760,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.933021+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:09:32.933021+0000","last_peered":"2024-09-10T16:09:32.933021+0000","last_clean":"2024-09-10T16:09:32.933021+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:09:32.933021+0000","last_undegraded":"2024-09-10T16:09:32.933021+0000","last_fullsized":"2024-09-10T16:09:32.933021+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20660,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20891","reported_seq":21040,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.929877+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:09:32.929877+0000","last_peered":"2024-09-10T16:09:32.929877+0000","last_clean":"2024-09-10T16:09:32.929877+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:09:32.929877+0000","last_undegraded":"2024-09-10T16:09:32.929877+0000","last_fullsized":"2024-09-10T16:09:32.929877+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17602,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'21061","reported_seq":21218,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:32.929429+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:09:32.929429+0000","last_peered":"2024-09-10T16:09:32.929429+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:09:32.929429+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17673,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4370},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'21082","reported_seq":21239,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:32.928987+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:09:32.928987+0000","last_peered":"2024-09-10T16:09:32.928987+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:09:32.928987+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17581,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4235},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'21314","reported_seq":21463,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.928707+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:09:32.928707+0000","last_peered":"2024-09-10T16:09:32.928707+0000","last_clean":"2024-09-10T16:09:32.928707+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:09:32.928707+0000","last_undegraded":"2024-09-10T16:09:32.928707+0000","last_fullsized":"2024-09-10T16:09:32.928707+0000","mapping_epoch":34,"log_start":"45'11300","ondisk_log_start":"45'11300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18161,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.929057+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:09:32.929057+0000","last_peered":"2024-09-10T16:09:32.929057+0000","last_clean":"2024-09-10T16:09:32.929057+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:09:32.929057+0000","last_undegraded":"2024-09-10T16:09:32.929057+0000","last_fullsized":"2024-09-10T16:09:32.929057+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20457","reported_seq":20620,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:33.013394+0000","last_change":"2024-09-10T16:09:33.012840+0000","last_active":"2024-09-10T16:09:33.013394+0000","last_peered":"2024-09-10T16:09:33.013394+0000","last_clean":"2024-09-10T16:09:33.013394+0000","last_became_active":"2024-09-10T16:09:32.934056+0000","last_became_peered":"2024-09-10T16:09:32.934056+0000","last_unstale":"2024-09-10T16:09:33.013394+0000","last_undegraded":"2024-09-10T16:09:33.013394+0000","last_fullsized":"2024-09-10T16:09:33.013394+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:08:30.481235+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18276,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":4579,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":113,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.930929+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:09:32.930929+0000","last_peered":"2024-09-10T16:09:32.930929+0000","last_clean":"2024-09-10T16:09:32.930929+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:09:32.930929+0000","last_undegraded":"2024-09-10T16:09:32.930929+0000","last_fullsized":"2024-09-10T16:09:32.930929+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20910","reported_seq":29582,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.929808+0000","last_change":"2024-09-10T16:09:08.909580+0000","last_active":"2024-09-10T16:09:32.929808+0000","last_peered":"2024-09-10T16:09:32.929808+0000","last_clean":"2024-09-10T16:09:32.929808+0000","last_became_active":"2024-09-10T16:09:08.854941+0000","last_became_peered":"2024-09-10T16:09:08.854941+0000","last_unstale":"2024-09-10T16:09:32.929808+0000","last_undegraded":"2024-09-10T16:09:32.929808+0000","last_fullsized":"2024-09-10T16:09:32.929808+0000","mapping_epoch":50,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:13:14.846691+0000","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17414,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4257,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":123,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.928762+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:09:32.928762+0000","last_peered":"2024-09-10T16:09:32.928762+0000","last_clean":"2024-09-10T16:09:32.928762+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:09:32.928762+0000","last_undegraded":"2024-09-10T16:09:32.928762+0000","last_fullsized":"2024-09-10T16:09:32.928762+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20781","reported_seq":20935,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:32.931013+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:09:32.931013+0000","last_peered":"2024-09-10T16:09:32.931013+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:09:32.931013+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18461,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-4303}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":99,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.929342+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:09:32.929342+0000","last_peered":"2024-09-10T16:09:32.929342+0000","last_clean":"2024-09-10T16:09:32.929342+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:09:32.929342+0000","last_undegraded":"2024-09-10T16:09:32.929342+0000","last_fullsized":"2024-09-10T16:09:32.929342+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'21170","reported_seq":30033,"reported_epoch":53,"state":"active+recovery_wait","last_fresh":"2024-09-10T16:09:32.930698+0000","last_change":"2024-09-10T16:08:10.929169+0000","last_active":"2024-09-10T16:09:32.930698+0000","last_peered":"2024-09-10T16:09:32.930698+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T16:08:10.860438+0000","last_became_peered":"2024-09-10T16:08:10.860438+0000","last_unstale":"2024-09-10T16:09:32.930698+0000","last_undegraded":"2024-09-10T16:09:32.930698+0000","last_fullsized":"2024-09-10T16:09:32.930698+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18724,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","0","1"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20838","reported_seq":20992,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:32.930297+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:09:32.930297+0000","last_peered":"2024-09-10T16:09:32.930297+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:09:32.930297+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17469,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.929280+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:09:32.929280+0000","last_peered":"2024-09-10T16:09:32.929280+0000","last_clean":"2024-09-10T16:09:32.929280+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:09:32.929280+0000","last_undegraded":"2024-09-10T16:09:32.929280+0000","last_fullsized":"2024-09-10T16:09:32.929280+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20598","reported_seq":29183,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.929268+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:09:32.929268+0000","last_peered":"2024-09-10T16:09:32.929268+0000","last_clean":"2024-09-10T16:09:32.929268+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:09:32.929268+0000","last_undegraded":"2024-09-10T16:09:32.929268+0000","last_fullsized":"2024-09-10T16:09:32.929268+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16988,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":113,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.933167+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:09:32.933167+0000","last_peered":"2024-09-10T16:09:32.933167+0000","last_clean":"2024-09-10T16:09:32.933167+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:09:32.933167+0000","last_undegraded":"2024-09-10T16:09:32.933167+0000","last_fullsized":"2024-09-10T16:09:32.933167+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'21048","reported_seq":21209,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:32.931059+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:09:32.931059+0000","last_peered":"2024-09-10T16:09:32.931059+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:09:32.931059+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17303,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.929172+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:09:32.929172+0000","last_peered":"2024-09-10T16:09:32.929172+0000","last_clean":"2024-09-10T16:09:32.929172+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:09:32.929172+0000","last_undegraded":"2024-09-10T16:09:32.929172+0000","last_fullsized":"2024-09-10T16:09:32.929172+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":93,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.929492+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:09:32.929492+0000","last_peered":"2024-09-10T16:09:32.929492+0000","last_clean":"2024-09-10T16:09:32.929492+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:09:32.929492+0000","last_undegraded":"2024-09-10T16:09:32.929492+0000","last_fullsized":"2024-09-10T16:09:32.929492+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'21198","reported_seq":21362,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:32.931857+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:09:32.931857+0000","last_peered":"2024-09-10T16:09:32.931857+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:09:32.931857+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17361,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4321},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":26944,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292096,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24446,"num_bytes_recovered":33096163,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":210518016,"data_stored":210395174,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161022,"ondisk_log_size":161022,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986318,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":502632,"kb_used_data":137744,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93869208,"statfs":{"total":96636764160,"available":96122068992,"internally_reserved":0,"allocated":141049856,"data_stored":140719591,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659264,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":473660,"kb_used_data":492,"kb_used_omap":4,"kb_used_meta":473147,"kb_avail":93898180,"statfs":{"total":96636764160,"available":96151736320,"internally_reserved":0,"allocated":503808,"data_stored":155014,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":484503214},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182546,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":547332,"kb_used_data":68652,"kb_used_omap":4,"kb_used_meta":478651,"kb_avail":93824508,"statfs":{"total":96636764160,"available":96076296192,"internally_reserved":0,"allocated":70299648,"data_stored":69965306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":490139306},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":69853184,"data_stored":69817054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":6762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":140607488,"data_stored":140571358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:09:56.035 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:09:56.378 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:09:56.379 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:09:56.394 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":427,"stamp":"2024-09-10T16:09:56.000972+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":26944,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292098,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24448,"num_bytes_recovered":33096182,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161025,"ondisk_log_size":161025,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1523624,"kb_used_data":206888,"kb_used_omap":12,"kb_used_meta":1316659,"kb_avail":281591896,"statfs":{"total":289910292480,"available":288350101504,"internally_reserved":0,"allocated":211853312,"data_stored":210839911,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1348258818},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1623,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1623,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.224277"},"pg_stats":[{"pgid":"2.f","version":"49'20672","reported_seq":27231,"reported_epoch":53,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:51.211039+0000","last_change":"2024-09-10T16:09:31.772779+0000","last_active":"2024-09-10T16:09:51.211039+0000","last_peered":"2024-09-10T16:09:51.211039+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:09:51.211039+0000","last_undegraded":"2024-09-10T15:59:15.851279+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1027,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20672,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3232,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1027}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20844","reported_seq":20976,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.933194+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:09:32.933194+0000","last_peered":"2024-09-10T16:09:32.933194+0000","last_clean":"2024-09-10T16:09:32.933194+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:09:32.933194+0000","last_undegraded":"2024-09-10T16:09:32.933194+0000","last_fullsized":"2024-09-10T16:09:32.933194+0000","mapping_epoch":35,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18949,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20798","reported_seq":29495,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.929401+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:09:32.929401+0000","last_peered":"2024-09-10T16:09:32.929401+0000","last_clean":"2024-09-10T16:09:32.929401+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:09:32.929401+0000","last_undegraded":"2024-09-10T16:09:32.929401+0000","last_fullsized":"2024-09-10T16:09:32.929401+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18802,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20660","reported_seq":20760,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.933021+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:09:32.933021+0000","last_peered":"2024-09-10T16:09:32.933021+0000","last_clean":"2024-09-10T16:09:32.933021+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:09:32.933021+0000","last_undegraded":"2024-09-10T16:09:32.933021+0000","last_fullsized":"2024-09-10T16:09:32.933021+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20660,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20891","reported_seq":21040,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.929877+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:09:32.929877+0000","last_peered":"2024-09-10T16:09:32.929877+0000","last_clean":"2024-09-10T16:09:32.929877+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:09:32.929877+0000","last_undegraded":"2024-09-10T16:09:32.929877+0000","last_fullsized":"2024-09-10T16:09:32.929877+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17602,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'21061","reported_seq":21218,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:32.929429+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:09:32.929429+0000","last_peered":"2024-09-10T16:09:32.929429+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:09:32.929429+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17673,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4370},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'21082","reported_seq":21239,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:32.928987+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:09:32.928987+0000","last_peered":"2024-09-10T16:09:32.928987+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:09:32.928987+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17581,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4235},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'21314","reported_seq":21463,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.928707+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:09:32.928707+0000","last_peered":"2024-09-10T16:09:32.928707+0000","last_clean":"2024-09-10T16:09:32.928707+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:09:32.928707+0000","last_undegraded":"2024-09-10T16:09:32.928707+0000","last_fullsized":"2024-09-10T16:09:32.928707+0000","mapping_epoch":34,"log_start":"45'11300","ondisk_log_start":"45'11300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18161,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.929057+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:09:32.929057+0000","last_peered":"2024-09-10T16:09:32.929057+0000","last_clean":"2024-09-10T16:09:32.929057+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:09:32.929057+0000","last_undegraded":"2024-09-10T16:09:32.929057+0000","last_fullsized":"2024-09-10T16:09:32.929057+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20457","reported_seq":20620,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:33.013394+0000","last_change":"2024-09-10T16:09:33.012840+0000","last_active":"2024-09-10T16:09:33.013394+0000","last_peered":"2024-09-10T16:09:33.013394+0000","last_clean":"2024-09-10T16:09:33.013394+0000","last_became_active":"2024-09-10T16:09:32.934056+0000","last_became_peered":"2024-09-10T16:09:32.934056+0000","last_unstale":"2024-09-10T16:09:33.013394+0000","last_undegraded":"2024-09-10T16:09:33.013394+0000","last_fullsized":"2024-09-10T16:09:33.013394+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:08:30.481235+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18276,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":4579,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":113,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.930929+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:09:32.930929+0000","last_peered":"2024-09-10T16:09:32.930929+0000","last_clean":"2024-09-10T16:09:32.930929+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:09:32.930929+0000","last_undegraded":"2024-09-10T16:09:32.930929+0000","last_fullsized":"2024-09-10T16:09:32.930929+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20910","reported_seq":29582,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.929808+0000","last_change":"2024-09-10T16:09:08.909580+0000","last_active":"2024-09-10T16:09:32.929808+0000","last_peered":"2024-09-10T16:09:32.929808+0000","last_clean":"2024-09-10T16:09:32.929808+0000","last_became_active":"2024-09-10T16:09:08.854941+0000","last_became_peered":"2024-09-10T16:09:08.854941+0000","last_unstale":"2024-09-10T16:09:32.929808+0000","last_undegraded":"2024-09-10T16:09:32.929808+0000","last_fullsized":"2024-09-10T16:09:32.929808+0000","mapping_epoch":50,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:13:14.846691+0000","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17414,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4257,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":123,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.928762+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:09:32.928762+0000","last_peered":"2024-09-10T16:09:32.928762+0000","last_clean":"2024-09-10T16:09:32.928762+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:09:32.928762+0000","last_undegraded":"2024-09-10T16:09:32.928762+0000","last_fullsized":"2024-09-10T16:09:32.928762+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20781","reported_seq":20935,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:32.931013+0000","last_change":"2024-09-10T15:59:16.284311+0000","last_active":"2024-09-10T16:09:32.931013+0000","last_peered":"2024-09-10T16:09:32.931013+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:09:32.931013+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18461,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-4303}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":99,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.929342+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:09:32.929342+0000","last_peered":"2024-09-10T16:09:32.929342+0000","last_clean":"2024-09-10T16:09:32.929342+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:09:32.929342+0000","last_undegraded":"2024-09-10T16:09:32.929342+0000","last_fullsized":"2024-09-10T16:09:32.929342+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'21170","reported_seq":30033,"reported_epoch":53,"state":"active+recovery_wait","last_fresh":"2024-09-10T16:09:32.930698+0000","last_change":"2024-09-10T16:08:10.929169+0000","last_active":"2024-09-10T16:09:32.930698+0000","last_peered":"2024-09-10T16:09:32.930698+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T16:08:10.860438+0000","last_became_peered":"2024-09-10T16:08:10.860438+0000","last_unstale":"2024-09-10T16:09:32.930698+0000","last_undegraded":"2024-09-10T16:09:32.930698+0000","last_fullsized":"2024-09-10T16:09:32.930698+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18724,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","0","1"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20838","reported_seq":20992,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:32.930297+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:09:32.930297+0000","last_peered":"2024-09-10T16:09:32.930297+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:09:32.930297+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17469,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.929280+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:09:32.929280+0000","last_peered":"2024-09-10T16:09:32.929280+0000","last_clean":"2024-09-10T16:09:32.929280+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:09:32.929280+0000","last_undegraded":"2024-09-10T16:09:32.929280+0000","last_fullsized":"2024-09-10T16:09:32.929280+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20598","reported_seq":29183,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.929268+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:09:32.929268+0000","last_peered":"2024-09-10T16:09:32.929268+0000","last_clean":"2024-09-10T16:09:32.929268+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:09:32.929268+0000","last_undegraded":"2024-09-10T16:09:32.929268+0000","last_fullsized":"2024-09-10T16:09:32.929268+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16988,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":113,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.933167+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:09:32.933167+0000","last_peered":"2024-09-10T16:09:32.933167+0000","last_clean":"2024-09-10T16:09:32.933167+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:09:32.933167+0000","last_undegraded":"2024-09-10T16:09:32.933167+0000","last_fullsized":"2024-09-10T16:09:32.933167+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'21048","reported_seq":21209,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:32.931059+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:09:32.931059+0000","last_peered":"2024-09-10T16:09:32.931059+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:09:32.931059+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17303,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.929172+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:09:32.929172+0000","last_peered":"2024-09-10T16:09:32.929172+0000","last_clean":"2024-09-10T16:09:32.929172+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:09:32.929172+0000","last_undegraded":"2024-09-10T16:09:32.929172+0000","last_fullsized":"2024-09-10T16:09:32.929172+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":93,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.929492+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:09:32.929492+0000","last_peered":"2024-09-10T16:09:32.929492+0000","last_clean":"2024-09-10T16:09:32.929492+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:09:32.929492+0000","last_undegraded":"2024-09-10T16:09:32.929492+0000","last_fullsized":"2024-09-10T16:09:32.929492+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'21198","reported_seq":21362,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:32.931857+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:09:32.931857+0000","last_peered":"2024-09-10T16:09:32.931857+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:09:32.931857+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17361,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4321},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":26944,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292096,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24446,"num_bytes_recovered":33096163,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":210518016,"data_stored":210395174,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161022,"ondisk_log_size":161022,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986318,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":502632,"kb_used_data":137744,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93869208,"statfs":{"total":96636764160,"available":96122068992,"internally_reserved":0,"allocated":141049856,"data_stored":140719591,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659264,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":473660,"kb_used_data":492,"kb_used_omap":4,"kb_used_meta":473147,"kb_avail":93898180,"statfs":{"total":96636764160,"available":96151736320,"internally_reserved":0,"allocated":503808,"data_stored":155014,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":484503214},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182546,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":547332,"kb_used_data":68652,"kb_used_omap":4,"kb_used_meta":478651,"kb_avail":93824508,"statfs":{"total":96636764160,"available":96076296192,"internally_reserved":0,"allocated":70299648,"data_stored":69965306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":490139306},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":69853184,"data_stored":69817054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":6762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":140607488,"data_stored":140571358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:09:56.395 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-10T16:09:56.395 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "49'20672", 'reported_seq': 27231, 'reported_epoch': 53, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:09:51.211039+0000', 'last_change': '2024-09-10T16:09:31.772779+0000', 'last_active': '2024-09-10T16:09:51.211039+0000', 'last_peered': '2024-09-10T16:09:51.211039+0000', 'last_clean': '2024-09-10T15:59:14.132456+0000', 'last_became_active': '2024-09-10T15:59:16.274407+0000', 'last_became_peered': '2024-09-10T15:59:16.274407+0000', 'last_unstale': '2024-09-10T16:09:51.211039+0000', 'last_undegraded': '2024-09-10T15:59:15.851279+0000', 'last_fullsized': '2024-09-10T15:59:15.774356+0000', 'mapping_epoch': 42, 'log_start': "45'10600", 'ondisk_log_start': "45'10600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'objects_scrubbed': 0, 'log_size': 10072, 'log_dups_size': 0, 'ondisk_log_size': 10072, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1027, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20672, 'num_write_kb': 41344, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3232, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1027}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:09:56.396 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-10T16:09:56.396 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "49'21061", 'reported_seq': 21218, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:09:32.929429+0000', 'last_change': '2024-09-10T15:59:16.308264+0000', 'last_active': '2024-09-10T16:09:32.929429+0000', 'last_peered': '2024-09-10T16:09:32.929429+0000', 'last_clean': '2024-09-10T15:59:14.111551+0000', 'last_became_active': '2024-09-10T15:59:16.300040+0000', 'last_became_peered': '2024-09-10T15:59:16.300040+0000', 'last_unstale': '2024-09-10T16:09:32.929429+0000', 'last_undegraded': '2024-09-10T15:59:15.878234+0000', 'last_fullsized': '2024-09-10T15:59:15.769328+0000', 'mapping_epoch': 42, 'log_start': "45'11000", 'ondisk_log_start': "45'11000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3442", 'last_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'objects_scrubbed': 3384, 'log_size': 10061, 'log_dups_size': 0, 'ondisk_log_size': 10061, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1292, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4372, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 17673, 'num_write_kb': 28580, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4370}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:09:56.396 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-10T16:09:56.396 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "49'21082", 'reported_seq': 21239, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:09:32.928987+0000', 'last_change': '2024-09-10T15:59:16.331899+0000', 'last_active': '2024-09-10T16:09:32.928987+0000', 'last_peered': '2024-09-10T16:09:32.928987+0000', 'last_clean': '2024-09-10T15:59:14.139436+0000', 'last_became_active': '2024-09-10T15:59:16.318715+0000', 'last_became_peered': '2024-09-10T15:59:16.318715+0000', 'last_unstale': '2024-09-10T16:09:32.928987+0000', 'last_undegraded': '2024-09-10T15:59:15.877524+0000', 'last_fullsized': '2024-09-10T15:59:15.769042+0000', 'mapping_epoch': 42, 'log_start': "45'11000", 'ondisk_log_start': "45'11000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3537", 'last_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'objects_scrubbed': 3497, 'log_size': 10082, 'log_dups_size': 0, 'ondisk_log_size': 10082, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1224, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4237, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 17581, 'num_write_kb': 28172, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4235}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:09:56.396 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-10T16:09:56.396 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "49'20781", 'reported_seq': 20935, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:09:32.931013+0000', 'last_change': '2024-09-10T15:59:16.284311+0000', 'last_active': '2024-09-10T16:09:32.931013+0000', 'last_peered': '2024-09-10T16:09:32.931013+0000', 'last_clean': '2024-09-10T15:59:14.116041+0000', 'last_became_active': '2024-09-10T15:59:16.274543+0000', 'last_became_peered': '2024-09-10T15:59:16.274543+0000', 'last_unstale': '2024-09-10T16:09:32.931013+0000', 'last_undegraded': '2024-09-10T15:59:15.812484+0000', 'last_fullsized': '2024-09-10T15:59:15.770885+0000', 'mapping_epoch': 42, 'log_start': "45'10700", 'ondisk_log_start': "45'10700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2356", 'last_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'objects_scrubbed': 2316, 'log_size': 10081, 'log_dups_size': 0, 'ondisk_log_size': 10081, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 783, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 18461, 'num_write_kb': 32292, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': -4303}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:09:56.396 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-10T16:09:56.396 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "49'21170", 'reported_seq': 30033, 'reported_epoch': 53, 'state': 'active+recovery_wait', 'last_fresh': '2024-09-10T16:09:32.930698+0000', 'last_change': '2024-09-10T16:08:10.929169+0000', 'last_active': '2024-09-10T16:09:32.930698+0000', 'last_peered': '2024-09-10T16:09:32.930698+0000', 'last_clean': '2024-09-10T15:59:14.110654+0000', 'last_became_active': '2024-09-10T16:08:10.860438+0000', 'last_became_peered': '2024-09-10T16:08:10.860438+0000', 'last_unstale': '2024-09-10T16:09:32.930698+0000', 'last_undegraded': '2024-09-10T16:09:32.930698+0000', 'last_fullsized': '2024-09-10T16:09:32.930698+0000', 'mapping_epoch': 48, 'log_start': "45'11100", 'ondisk_log_start': "45'11100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2486", 'last_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'objects_scrubbed': 2440, 'log_size': 10070, 'log_dups_size': 0, 'ondisk_log_size': 10070, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 792, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18724, 'num_write_kb': 32568, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 4355, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': ['2', '0', '1'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:09:56.397 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-10T16:09:56.397 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "49'20838", 'reported_seq': 20992, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:09:32.930297+0000', 'last_change': '2024-09-10T15:59:16.315641+0000', 'last_active': '2024-09-10T16:09:32.930297+0000', 'last_peered': '2024-09-10T16:09:32.930297+0000', 'last_clean': '2024-09-10T15:59:14.117001+0000', 'last_became_active': '2024-09-10T15:59:16.302245+0000', 'last_became_peered': '2024-09-10T15:59:16.302245+0000', 'last_unstale': '2024-09-10T16:09:32.930297+0000', 'last_undegraded': '2024-09-10T15:59:15.845012+0000', 'last_fullsized': '2024-09-10T15:59:15.770337+0000', 'mapping_epoch': 42, 'log_start': "45'10800", 'ondisk_log_start': "45'10800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3404", 'last_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'objects_scrubbed': 3367, 'log_size': 10038, 'log_dups_size': 0, 'ondisk_log_size': 10038, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 983, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17469, 'num_write_kb': 28204, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4304}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:09:56.397 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-10T16:09:56.397 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "49'21048", 'reported_seq': 21209, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:09:32.931059+0000', 'last_change': '2024-09-10T15:59:16.311291+0000', 'last_active': '2024-09-10T16:09:32.931059+0000', 'last_peered': '2024-09-10T16:09:32.931059+0000', 'last_clean': '2024-09-10T15:59:14.116363+0000', 'last_became_active': '2024-09-10T15:59:16.301244+0000', 'last_became_peered': '2024-09-10T15:59:16.301244+0000', 'last_unstale': '2024-09-10T16:09:32.931059+0000', 'last_undegraded': '2024-09-10T15:59:15.883434+0000', 'last_fullsized': '2024-09-10T15:59:15.772756+0000', 'mapping_epoch': 42, 'log_start': "45'11000", 'ondisk_log_start': "45'11000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3789", 'last_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'objects_scrubbed': 3741, 'log_size': 10048, 'log_dups_size': 0, 'ondisk_log_size': 10048, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1440, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4373, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17303, 'num_write_kb': 27124, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4371}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:09:56.397 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-10T16:09:56.397 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "49'21198", 'reported_seq': 21362, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:09:32.931857+0000', 'last_change': '2024-09-10T15:59:16.313409+0000', 'last_active': '2024-09-10T16:09:32.931857+0000', 'last_peered': '2024-09-10T16:09:32.931857+0000', 'last_clean': '2024-09-10T15:59:14.129780+0000', 'last_became_active': '2024-09-10T15:59:16.300932+0000', 'last_became_peered': '2024-09-10T15:59:16.300932+0000', 'last_unstale': '2024-09-10T16:09:32.931857+0000', 'last_undegraded': '2024-09-10T15:59:15.887823+0000', 'last_fullsized': '2024-09-10T15:59:15.768513+0000', 'mapping_epoch': 42, 'log_start': "45'11100", 'ondisk_log_start': "45'11100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3880", 'last_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'objects_scrubbed': 3827, 'log_size': 10098, 'log_dups_size': 0, 'ondisk_log_size': 10098, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1397, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4325, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17361, 'num_write_kb': 27072, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4321}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:09:56.398 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T16:09:56.815 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:09:56.816 INFO:teuthology.orchestra.run.smithi115.stdout:{"fsid":"7effee29-df05-4351-9741-ee203564ed81","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 26944/24 objects degraded (112266.667%), 7 pgs degraded, 7 pgs undersized","count":14},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":760,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":54,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983954,"num_in_osds":3,"osd_in_since":1725983947,"num_remapped_pgs":6},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":16},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":6},{"state_name":"active+recovery_wait","count":1},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":12,"data_bytes":4849,"bytes_used":1560190976,"bytes_avail":288350101504,"bytes_total":289910292480,"degraded_objects":26944,"degraded_total":24,"degraded_ratio":1122.6666666666667,"recovering_objects_per_sec":135,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1623,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-10T15:57:14:967120+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T16:00:13.560776+0000","services":{}},"progress_events":{}} 2024-09-10T16:09:56.829 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-10T16:09:56.829 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:09:57.160 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:09:57.160 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:09:57.176 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":428,"stamp":"2024-09-10T16:09:56.998708+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":25874,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292098,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25518,"num_bytes_recovered":33096665,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161025,"ondisk_log_size":161025,"up":48,"acting":42,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1523628,"kb_used_data":206896,"kb_used_omap":12,"kb_used_meta":1316659,"kb_avail":281591892,"statfs":{"total":289910292480,"available":288350097408,"internally_reserved":0,"allocated":211861504,"data_stored":210842788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1348258818},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1938,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1938,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.221749"},"pg_stats":[{"pgid":"2.f","version":"49'20672","reported_seq":29287,"reported_epoch":53,"state":"peering","last_fresh":"2024-09-10T16:09:55.745411+0000","last_change":"2024-09-10T16:09:56.009762+0000","last_active":"2024-09-10T16:09:56.009762+0000","last_peered":"2024-09-10T16:09:55.745411+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:09:55.745411+0000","last_undegraded":"2024-09-10T16:09:55.745411+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":54,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20672,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4259,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20844","reported_seq":20978,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:09:56.005082+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:09:56.005082+0000","last_peered":"2024-09-10T16:09:56.005082+0000","last_clean":"2024-09-10T16:09:56.005082+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:09:56.005082+0000","last_undegraded":"2024-09-10T16:09:56.005082+0000","last_fullsized":"2024-09-10T16:09:56.005082+0000","mapping_epoch":35,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18949,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20798","reported_seq":29495,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.929401+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:09:32.929401+0000","last_peered":"2024-09-10T16:09:32.929401+0000","last_clean":"2024-09-10T16:09:32.929401+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:09:32.929401+0000","last_undegraded":"2024-09-10T16:09:32.929401+0000","last_fullsized":"2024-09-10T16:09:32.929401+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18802,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20660","reported_seq":20762,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:09:56.005036+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:09:56.005036+0000","last_peered":"2024-09-10T16:09:56.005036+0000","last_clean":"2024-09-10T16:09:56.005036+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:09:56.005036+0000","last_undegraded":"2024-09-10T16:09:56.005036+0000","last_fullsized":"2024-09-10T16:09:56.005036+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20660,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20891","reported_seq":21042,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:09:56.000580+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:09:56.000580+0000","last_peered":"2024-09-10T16:09:56.000580+0000","last_clean":"2024-09-10T16:09:56.000580+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:09:56.000580+0000","last_undegraded":"2024-09-10T16:09:56.000580+0000","last_fullsized":"2024-09-10T16:09:56.000580+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17602,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'21061","reported_seq":21220,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:56.000563+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:09:56.000563+0000","last_peered":"2024-09-10T16:09:56.000563+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:09:56.000563+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17673,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4370},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'21082","reported_seq":21241,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:56.000545+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:09:56.000545+0000","last_peered":"2024-09-10T16:09:56.000545+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:09:56.000545+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17581,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4235},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'21314","reported_seq":21465,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:09:56.000471+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:09:56.000471+0000","last_peered":"2024-09-10T16:09:56.000471+0000","last_clean":"2024-09-10T16:09:56.000471+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:09:56.000471+0000","last_undegraded":"2024-09-10T16:09:56.000471+0000","last_fullsized":"2024-09-10T16:09:56.000471+0000","mapping_epoch":34,"log_start":"45'11300","ondisk_log_start":"45'11300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18161,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.929057+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:09:32.929057+0000","last_peered":"2024-09-10T16:09:32.929057+0000","last_clean":"2024-09-10T16:09:32.929057+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:09:32.929057+0000","last_undegraded":"2024-09-10T16:09:32.929057+0000","last_fullsized":"2024-09-10T16:09:32.929057+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20457","reported_seq":20620,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:33.013394+0000","last_change":"2024-09-10T16:09:33.012840+0000","last_active":"2024-09-10T16:09:33.013394+0000","last_peered":"2024-09-10T16:09:33.013394+0000","last_clean":"2024-09-10T16:09:33.013394+0000","last_became_active":"2024-09-10T16:09:32.934056+0000","last_became_peered":"2024-09-10T16:09:32.934056+0000","last_unstale":"2024-09-10T16:09:33.013394+0000","last_undegraded":"2024-09-10T16:09:33.013394+0000","last_fullsized":"2024-09-10T16:09:33.013394+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:08:30.481235+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18276,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":4579,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":115,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:09:56.004815+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:09:56.004815+0000","last_peered":"2024-09-10T16:09:56.004815+0000","last_clean":"2024-09-10T16:09:56.004815+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:09:56.004815+0000","last_undegraded":"2024-09-10T16:09:56.004815+0000","last_fullsized":"2024-09-10T16:09:56.004815+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20910","reported_seq":29584,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:09:56.002456+0000","last_change":"2024-09-10T16:09:08.909580+0000","last_active":"2024-09-10T16:09:56.002456+0000","last_peered":"2024-09-10T16:09:56.002456+0000","last_clean":"2024-09-10T16:09:56.002456+0000","last_became_active":"2024-09-10T16:09:08.854941+0000","last_became_peered":"2024-09-10T16:09:08.854941+0000","last_unstale":"2024-09-10T16:09:56.002456+0000","last_undegraded":"2024-09-10T16:09:56.002456+0000","last_fullsized":"2024-09-10T16:09:56.002456+0000","mapping_epoch":50,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:13:14.846691+0000","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17414,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4257,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":125,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:09:56.000454+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:09:56.000454+0000","last_peered":"2024-09-10T16:09:56.000454+0000","last_clean":"2024-09-10T16:09:56.000454+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:09:56.000454+0000","last_undegraded":"2024-09-10T16:09:56.000454+0000","last_fullsized":"2024-09-10T16:09:56.000454+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20781","reported_seq":21024,"reported_epoch":54,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:56.159000+0000","last_change":"2024-09-10T16:09:55.745709+0000","last_active":"2024-09-10T16:09:56.159000+0000","last_peered":"2024-09-10T16:09:56.159000+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:09:56.159000+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4262,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18461,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":45,"num_bytes_recovered":8675,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-4261}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":99,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.929342+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:09:32.929342+0000","last_peered":"2024-09-10T16:09:32.929342+0000","last_clean":"2024-09-10T16:09:32.929342+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:09:32.929342+0000","last_undegraded":"2024-09-10T16:09:32.929342+0000","last_fullsized":"2024-09-10T16:09:32.929342+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'21170","reported_seq":30035,"reported_epoch":54,"state":"active+recovery_wait","last_fresh":"2024-09-10T16:09:56.004450+0000","last_change":"2024-09-10T16:08:10.929169+0000","last_active":"2024-09-10T16:09:56.004450+0000","last_peered":"2024-09-10T16:09:56.004450+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T16:08:10.860438+0000","last_became_peered":"2024-09-10T16:08:10.860438+0000","last_unstale":"2024-09-10T16:09:56.004450+0000","last_undegraded":"2024-09-10T16:09:56.004450+0000","last_fullsized":"2024-09-10T16:09:56.004450+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18724,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","0","1"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20838","reported_seq":20994,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:56.000586+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:09:56.000586+0000","last_peered":"2024-09-10T16:09:56.000586+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:09:56.000586+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17469,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.929280+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:09:32.929280+0000","last_peered":"2024-09-10T16:09:32.929280+0000","last_clean":"2024-09-10T16:09:32.929280+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:09:32.929280+0000","last_undegraded":"2024-09-10T16:09:32.929280+0000","last_fullsized":"2024-09-10T16:09:32.929280+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20598","reported_seq":29183,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.929268+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:09:32.929268+0000","last_peered":"2024-09-10T16:09:32.929268+0000","last_clean":"2024-09-10T16:09:32.929268+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:09:32.929268+0000","last_undegraded":"2024-09-10T16:09:32.929268+0000","last_fullsized":"2024-09-10T16:09:32.929268+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16988,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":115,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:09:56.004946+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:09:56.004946+0000","last_peered":"2024-09-10T16:09:56.004946+0000","last_clean":"2024-09-10T16:09:56.004946+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:09:56.004946+0000","last_undegraded":"2024-09-10T16:09:56.004946+0000","last_fullsized":"2024-09-10T16:09:56.004946+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'21048","reported_seq":21211,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:56.001374+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:09:56.001374+0000","last_peered":"2024-09-10T16:09:56.001374+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:09:56.001374+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17303,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.929172+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:09:32.929172+0000","last_peered":"2024-09-10T16:09:32.929172+0000","last_clean":"2024-09-10T16:09:32.929172+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:09:32.929172+0000","last_undegraded":"2024-09-10T16:09:32.929172+0000","last_fullsized":"2024-09-10T16:09:32.929172+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":93,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.929492+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:09:32.929492+0000","last_peered":"2024-09-10T16:09:32.929492+0000","last_clean":"2024-09-10T16:09:32.929492+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:09:32.929492+0000","last_undegraded":"2024-09-10T16:09:32.929492+0000","last_fullsized":"2024-09-10T16:09:32.929492+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'21198","reported_seq":21364,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:56.000680+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:09:56.000680+0000","last_peered":"2024-09-10T16:09:56.000680+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:09:56.000680+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17361,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4321},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":25874,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292096,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25516,"num_bytes_recovered":33096646,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":210518016,"data_stored":210395174,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161022,"ondisk_log_size":161022,"up":32,"acting":26,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986319,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":502632,"kb_used_data":137744,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93869208,"statfs":{"total":96636764160,"available":96122068992,"internally_reserved":0,"allocated":141049856,"data_stored":140719591,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659264,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":473660,"kb_used_data":492,"kb_used_omap":4,"kb_used_meta":473147,"kb_avail":93898180,"statfs":{"total":96636764160,"available":96151736320,"internally_reserved":0,"allocated":503808,"data_stored":155014,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":484503214},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182547,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":547336,"kb_used_data":68660,"kb_used_omap":4,"kb_used_meta":478651,"kb_avail":93824504,"statfs":{"total":96636764160,"available":96076292096,"internally_reserved":0,"allocated":70307840,"data_stored":69968183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":490139306},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":69853184,"data_stored":69817054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":6762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":140607488,"data_stored":140571358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:10:00.178 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:10:00.533 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:10:00.533 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:10:00.546 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":430,"stamp":"2024-09-10T16:09:59.779083+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":25874,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292098,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25518,"num_bytes_recovered":33096665,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161025,"ondisk_log_size":161025,"up":48,"acting":42,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1523628,"kb_used_data":206896,"kb_used_omap":12,"kb_used_meta":1316659,"kb_avail":281591892,"statfs":{"total":289910292480,"available":288350097408,"internally_reserved":0,"allocated":211861504,"data_stored":210842788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1348258818},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1070,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1070,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001477"},"pg_stats":[{"pgid":"2.f","version":"49'20672","reported_seq":29287,"reported_epoch":53,"state":"peering","last_fresh":"2024-09-10T16:09:55.745411+0000","last_change":"2024-09-10T16:09:56.009762+0000","last_active":"2024-09-10T16:09:56.009762+0000","last_peered":"2024-09-10T16:09:55.745411+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:09:55.745411+0000","last_undegraded":"2024-09-10T16:09:55.745411+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":54,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20672,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4259,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20844","reported_seq":20978,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:09:56.005082+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:09:56.005082+0000","last_peered":"2024-09-10T16:09:56.005082+0000","last_clean":"2024-09-10T16:09:56.005082+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:09:56.005082+0000","last_undegraded":"2024-09-10T16:09:56.005082+0000","last_fullsized":"2024-09-10T16:09:56.005082+0000","mapping_epoch":35,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18949,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20798","reported_seq":29495,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.929401+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:09:32.929401+0000","last_peered":"2024-09-10T16:09:32.929401+0000","last_clean":"2024-09-10T16:09:32.929401+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:09:32.929401+0000","last_undegraded":"2024-09-10T16:09:32.929401+0000","last_fullsized":"2024-09-10T16:09:32.929401+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18802,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20660","reported_seq":20762,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:09:56.005036+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:09:56.005036+0000","last_peered":"2024-09-10T16:09:56.005036+0000","last_clean":"2024-09-10T16:09:56.005036+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:09:56.005036+0000","last_undegraded":"2024-09-10T16:09:56.005036+0000","last_fullsized":"2024-09-10T16:09:56.005036+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20660,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20891","reported_seq":21042,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:09:56.000580+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:09:56.000580+0000","last_peered":"2024-09-10T16:09:56.000580+0000","last_clean":"2024-09-10T16:09:56.000580+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:09:56.000580+0000","last_undegraded":"2024-09-10T16:09:56.000580+0000","last_fullsized":"2024-09-10T16:09:56.000580+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17602,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'21061","reported_seq":21220,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:56.000563+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:09:56.000563+0000","last_peered":"2024-09-10T16:09:56.000563+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:09:56.000563+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17673,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4370},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'21082","reported_seq":21241,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:56.000545+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:09:56.000545+0000","last_peered":"2024-09-10T16:09:56.000545+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:09:56.000545+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17581,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4235},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'21314","reported_seq":21465,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:09:56.000471+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:09:56.000471+0000","last_peered":"2024-09-10T16:09:56.000471+0000","last_clean":"2024-09-10T16:09:56.000471+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:09:56.000471+0000","last_undegraded":"2024-09-10T16:09:56.000471+0000","last_fullsized":"2024-09-10T16:09:56.000471+0000","mapping_epoch":34,"log_start":"45'11300","ondisk_log_start":"45'11300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18161,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.929057+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:09:32.929057+0000","last_peered":"2024-09-10T16:09:32.929057+0000","last_clean":"2024-09-10T16:09:32.929057+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:09:32.929057+0000","last_undegraded":"2024-09-10T16:09:32.929057+0000","last_fullsized":"2024-09-10T16:09:32.929057+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20457","reported_seq":20620,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:33.013394+0000","last_change":"2024-09-10T16:09:33.012840+0000","last_active":"2024-09-10T16:09:33.013394+0000","last_peered":"2024-09-10T16:09:33.013394+0000","last_clean":"2024-09-10T16:09:33.013394+0000","last_became_active":"2024-09-10T16:09:32.934056+0000","last_became_peered":"2024-09-10T16:09:32.934056+0000","last_unstale":"2024-09-10T16:09:33.013394+0000","last_undegraded":"2024-09-10T16:09:33.013394+0000","last_fullsized":"2024-09-10T16:09:33.013394+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:08:30.481235+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18276,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":4579,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":115,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:09:56.004815+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:09:56.004815+0000","last_peered":"2024-09-10T16:09:56.004815+0000","last_clean":"2024-09-10T16:09:56.004815+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:09:56.004815+0000","last_undegraded":"2024-09-10T16:09:56.004815+0000","last_fullsized":"2024-09-10T16:09:56.004815+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20910","reported_seq":29584,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:09:56.002456+0000","last_change":"2024-09-10T16:09:08.909580+0000","last_active":"2024-09-10T16:09:56.002456+0000","last_peered":"2024-09-10T16:09:56.002456+0000","last_clean":"2024-09-10T16:09:56.002456+0000","last_became_active":"2024-09-10T16:09:08.854941+0000","last_became_peered":"2024-09-10T16:09:08.854941+0000","last_unstale":"2024-09-10T16:09:56.002456+0000","last_undegraded":"2024-09-10T16:09:56.002456+0000","last_fullsized":"2024-09-10T16:09:56.002456+0000","mapping_epoch":50,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:13:14.846691+0000","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17414,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4257,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":125,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:09:56.000454+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:09:56.000454+0000","last_peered":"2024-09-10T16:09:56.000454+0000","last_clean":"2024-09-10T16:09:56.000454+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:09:56.000454+0000","last_undegraded":"2024-09-10T16:09:56.000454+0000","last_fullsized":"2024-09-10T16:09:56.000454+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20781","reported_seq":21024,"reported_epoch":54,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:56.159000+0000","last_change":"2024-09-10T16:09:55.745709+0000","last_active":"2024-09-10T16:09:56.159000+0000","last_peered":"2024-09-10T16:09:56.159000+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:09:56.159000+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4262,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18461,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":45,"num_bytes_recovered":8675,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-4261}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":99,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.929342+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:09:32.929342+0000","last_peered":"2024-09-10T16:09:32.929342+0000","last_clean":"2024-09-10T16:09:32.929342+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:09:32.929342+0000","last_undegraded":"2024-09-10T16:09:32.929342+0000","last_fullsized":"2024-09-10T16:09:32.929342+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'21170","reported_seq":30035,"reported_epoch":54,"state":"active+recovery_wait","last_fresh":"2024-09-10T16:09:56.004450+0000","last_change":"2024-09-10T16:08:10.929169+0000","last_active":"2024-09-10T16:09:56.004450+0000","last_peered":"2024-09-10T16:09:56.004450+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T16:08:10.860438+0000","last_became_peered":"2024-09-10T16:08:10.860438+0000","last_unstale":"2024-09-10T16:09:56.004450+0000","last_undegraded":"2024-09-10T16:09:56.004450+0000","last_fullsized":"2024-09-10T16:09:56.004450+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18724,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","0","1"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20838","reported_seq":20994,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:56.000586+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:09:56.000586+0000","last_peered":"2024-09-10T16:09:56.000586+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:09:56.000586+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17469,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.929280+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:09:32.929280+0000","last_peered":"2024-09-10T16:09:32.929280+0000","last_clean":"2024-09-10T16:09:32.929280+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:09:32.929280+0000","last_undegraded":"2024-09-10T16:09:32.929280+0000","last_fullsized":"2024-09-10T16:09:32.929280+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20598","reported_seq":29183,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.929268+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:09:32.929268+0000","last_peered":"2024-09-10T16:09:32.929268+0000","last_clean":"2024-09-10T16:09:32.929268+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:09:32.929268+0000","last_undegraded":"2024-09-10T16:09:32.929268+0000","last_fullsized":"2024-09-10T16:09:32.929268+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16988,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":115,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:09:56.004946+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:09:56.004946+0000","last_peered":"2024-09-10T16:09:56.004946+0000","last_clean":"2024-09-10T16:09:56.004946+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:09:56.004946+0000","last_undegraded":"2024-09-10T16:09:56.004946+0000","last_fullsized":"2024-09-10T16:09:56.004946+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'21048","reported_seq":21211,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:56.001374+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:09:56.001374+0000","last_peered":"2024-09-10T16:09:56.001374+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:09:56.001374+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17303,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.929172+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:09:32.929172+0000","last_peered":"2024-09-10T16:09:32.929172+0000","last_clean":"2024-09-10T16:09:32.929172+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:09:32.929172+0000","last_undegraded":"2024-09-10T16:09:32.929172+0000","last_fullsized":"2024-09-10T16:09:32.929172+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":93,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.929492+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:09:32.929492+0000","last_peered":"2024-09-10T16:09:32.929492+0000","last_clean":"2024-09-10T16:09:32.929492+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:09:32.929492+0000","last_undegraded":"2024-09-10T16:09:32.929492+0000","last_fullsized":"2024-09-10T16:09:32.929492+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'21198","reported_seq":21364,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:56.000680+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:09:56.000680+0000","last_peered":"2024-09-10T16:09:56.000680+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:09:56.000680+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17361,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4321},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":25874,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292096,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25516,"num_bytes_recovered":33096646,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":210518016,"data_stored":210395174,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161022,"ondisk_log_size":161022,"up":32,"acting":26,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986319,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":502632,"kb_used_data":137744,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93869208,"statfs":{"total":96636764160,"available":96122068992,"internally_reserved":0,"allocated":141049856,"data_stored":140719591,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659264,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":473660,"kb_used_data":492,"kb_used_omap":4,"kb_used_meta":473147,"kb_avail":93898180,"statfs":{"total":96636764160,"available":96151736320,"internally_reserved":0,"allocated":503808,"data_stored":155014,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":484503214},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182547,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":547336,"kb_used_data":68660,"kb_used_omap":4,"kb_used_meta":478651,"kb_avail":93824504,"statfs":{"total":96636764160,"available":96076292096,"internally_reserved":0,"allocated":70307840,"data_stored":69968183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":490139306},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":69853184,"data_stored":69817054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":6762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":140607488,"data_stored":140571358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:10:00.548 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:10:00.901 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:10:00.902 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:10:00.915 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":430,"stamp":"2024-09-10T16:09:59.779083+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":25874,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292098,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25518,"num_bytes_recovered":33096665,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161025,"ondisk_log_size":161025,"up":48,"acting":42,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1523628,"kb_used_data":206896,"kb_used_omap":12,"kb_used_meta":1316659,"kb_avail":281591892,"statfs":{"total":289910292480,"available":288350097408,"internally_reserved":0,"allocated":211861504,"data_stored":210842788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1348258818},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1070,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1070,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001477"},"pg_stats":[{"pgid":"2.f","version":"49'20672","reported_seq":29287,"reported_epoch":53,"state":"peering","last_fresh":"2024-09-10T16:09:55.745411+0000","last_change":"2024-09-10T16:09:56.009762+0000","last_active":"2024-09-10T16:09:56.009762+0000","last_peered":"2024-09-10T16:09:55.745411+0000","last_clean":"2024-09-10T15:59:14.132456+0000","last_became_active":"2024-09-10T15:59:16.274407+0000","last_became_peered":"2024-09-10T15:59:16.274407+0000","last_unstale":"2024-09-10T16:09:55.745411+0000","last_undegraded":"2024-09-10T16:09:55.745411+0000","last_fullsized":"2024-09-10T15:59:15.774356+0000","mapping_epoch":54,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20672,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4259,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20844","reported_seq":20978,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:09:56.005082+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:09:56.005082+0000","last_peered":"2024-09-10T16:09:56.005082+0000","last_clean":"2024-09-10T16:09:56.005082+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:09:56.005082+0000","last_undegraded":"2024-09-10T16:09:56.005082+0000","last_fullsized":"2024-09-10T16:09:56.005082+0000","mapping_epoch":35,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18949,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20798","reported_seq":29495,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.929401+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:09:32.929401+0000","last_peered":"2024-09-10T16:09:32.929401+0000","last_clean":"2024-09-10T16:09:32.929401+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:09:32.929401+0000","last_undegraded":"2024-09-10T16:09:32.929401+0000","last_fullsized":"2024-09-10T16:09:32.929401+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18802,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20660","reported_seq":20762,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:09:56.005036+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:09:56.005036+0000","last_peered":"2024-09-10T16:09:56.005036+0000","last_clean":"2024-09-10T16:09:56.005036+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:09:56.005036+0000","last_undegraded":"2024-09-10T16:09:56.005036+0000","last_fullsized":"2024-09-10T16:09:56.005036+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20660,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20891","reported_seq":21042,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:09:56.000580+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:09:56.000580+0000","last_peered":"2024-09-10T16:09:56.000580+0000","last_clean":"2024-09-10T16:09:56.000580+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:09:56.000580+0000","last_undegraded":"2024-09-10T16:09:56.000580+0000","last_fullsized":"2024-09-10T16:09:56.000580+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17602,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'21061","reported_seq":21220,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:56.000563+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:09:56.000563+0000","last_peered":"2024-09-10T16:09:56.000563+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:09:56.000563+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17673,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4370},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'21082","reported_seq":21241,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:56.000545+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:09:56.000545+0000","last_peered":"2024-09-10T16:09:56.000545+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:09:56.000545+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17581,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4235},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'21314","reported_seq":21465,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:09:56.000471+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:09:56.000471+0000","last_peered":"2024-09-10T16:09:56.000471+0000","last_clean":"2024-09-10T16:09:56.000471+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:09:56.000471+0000","last_undegraded":"2024-09-10T16:09:56.000471+0000","last_fullsized":"2024-09-10T16:09:56.000471+0000","mapping_epoch":34,"log_start":"45'11300","ondisk_log_start":"45'11300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18161,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.929057+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:09:32.929057+0000","last_peered":"2024-09-10T16:09:32.929057+0000","last_clean":"2024-09-10T16:09:32.929057+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:09:32.929057+0000","last_undegraded":"2024-09-10T16:09:32.929057+0000","last_fullsized":"2024-09-10T16:09:32.929057+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20457","reported_seq":20620,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:33.013394+0000","last_change":"2024-09-10T16:09:33.012840+0000","last_active":"2024-09-10T16:09:33.013394+0000","last_peered":"2024-09-10T16:09:33.013394+0000","last_clean":"2024-09-10T16:09:33.013394+0000","last_became_active":"2024-09-10T16:09:32.934056+0000","last_became_peered":"2024-09-10T16:09:32.934056+0000","last_unstale":"2024-09-10T16:09:33.013394+0000","last_undegraded":"2024-09-10T16:09:33.013394+0000","last_fullsized":"2024-09-10T16:09:33.013394+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:08:30.481235+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18276,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":4579,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":115,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:09:56.004815+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:09:56.004815+0000","last_peered":"2024-09-10T16:09:56.004815+0000","last_clean":"2024-09-10T16:09:56.004815+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:09:56.004815+0000","last_undegraded":"2024-09-10T16:09:56.004815+0000","last_fullsized":"2024-09-10T16:09:56.004815+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20910","reported_seq":29584,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:09:56.002456+0000","last_change":"2024-09-10T16:09:08.909580+0000","last_active":"2024-09-10T16:09:56.002456+0000","last_peered":"2024-09-10T16:09:56.002456+0000","last_clean":"2024-09-10T16:09:56.002456+0000","last_became_active":"2024-09-10T16:09:08.854941+0000","last_became_peered":"2024-09-10T16:09:08.854941+0000","last_unstale":"2024-09-10T16:09:56.002456+0000","last_undegraded":"2024-09-10T16:09:56.002456+0000","last_fullsized":"2024-09-10T16:09:56.002456+0000","mapping_epoch":50,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:13:14.846691+0000","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17414,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4257,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":125,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:09:56.000454+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:09:56.000454+0000","last_peered":"2024-09-10T16:09:56.000454+0000","last_clean":"2024-09-10T16:09:56.000454+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:09:56.000454+0000","last_undegraded":"2024-09-10T16:09:56.000454+0000","last_fullsized":"2024-09-10T16:09:56.000454+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20781","reported_seq":21024,"reported_epoch":54,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:56.159000+0000","last_change":"2024-09-10T16:09:55.745709+0000","last_active":"2024-09-10T16:09:56.159000+0000","last_peered":"2024-09-10T16:09:56.159000+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:09:56.159000+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4262,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18461,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":45,"num_bytes_recovered":8675,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-4261}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":99,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.929342+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:09:32.929342+0000","last_peered":"2024-09-10T16:09:32.929342+0000","last_clean":"2024-09-10T16:09:32.929342+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:09:32.929342+0000","last_undegraded":"2024-09-10T16:09:32.929342+0000","last_fullsized":"2024-09-10T16:09:32.929342+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'21170","reported_seq":30035,"reported_epoch":54,"state":"active+recovery_wait","last_fresh":"2024-09-10T16:09:56.004450+0000","last_change":"2024-09-10T16:08:10.929169+0000","last_active":"2024-09-10T16:09:56.004450+0000","last_peered":"2024-09-10T16:09:56.004450+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T16:08:10.860438+0000","last_became_peered":"2024-09-10T16:08:10.860438+0000","last_unstale":"2024-09-10T16:09:56.004450+0000","last_undegraded":"2024-09-10T16:09:56.004450+0000","last_fullsized":"2024-09-10T16:09:56.004450+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18724,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","0","1"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20838","reported_seq":20994,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:56.000586+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:09:56.000586+0000","last_peered":"2024-09-10T16:09:56.000586+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:09:56.000586+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17469,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.929280+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:09:32.929280+0000","last_peered":"2024-09-10T16:09:32.929280+0000","last_clean":"2024-09-10T16:09:32.929280+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:09:32.929280+0000","last_undegraded":"2024-09-10T16:09:32.929280+0000","last_fullsized":"2024-09-10T16:09:32.929280+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20598","reported_seq":29183,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.929268+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:09:32.929268+0000","last_peered":"2024-09-10T16:09:32.929268+0000","last_clean":"2024-09-10T16:09:32.929268+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:09:32.929268+0000","last_undegraded":"2024-09-10T16:09:32.929268+0000","last_fullsized":"2024-09-10T16:09:32.929268+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16988,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":115,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:09:56.004946+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:09:56.004946+0000","last_peered":"2024-09-10T16:09:56.004946+0000","last_clean":"2024-09-10T16:09:56.004946+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:09:56.004946+0000","last_undegraded":"2024-09-10T16:09:56.004946+0000","last_fullsized":"2024-09-10T16:09:56.004946+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'21048","reported_seq":21211,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:56.001374+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:09:56.001374+0000","last_peered":"2024-09-10T16:09:56.001374+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:09:56.001374+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17303,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.929172+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:09:32.929172+0000","last_peered":"2024-09-10T16:09:32.929172+0000","last_clean":"2024-09-10T16:09:32.929172+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:09:32.929172+0000","last_undegraded":"2024-09-10T16:09:32.929172+0000","last_fullsized":"2024-09-10T16:09:32.929172+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":93,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T16:09:32.929492+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:09:32.929492+0000","last_peered":"2024-09-10T16:09:32.929492+0000","last_clean":"2024-09-10T16:09:32.929492+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:09:32.929492+0000","last_undegraded":"2024-09-10T16:09:32.929492+0000","last_fullsized":"2024-09-10T16:09:32.929492+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'21198","reported_seq":21364,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:56.000680+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:09:56.000680+0000","last_peered":"2024-09-10T16:09:56.000680+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:09:56.000680+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17361,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4321},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":25874,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292096,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25516,"num_bytes_recovered":33096646,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":210518016,"data_stored":210395174,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161022,"ondisk_log_size":161022,"up":32,"acting":26,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986319,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":502632,"kb_used_data":137744,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93869208,"statfs":{"total":96636764160,"available":96122068992,"internally_reserved":0,"allocated":141049856,"data_stored":140719591,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659264,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":473660,"kb_used_data":492,"kb_used_omap":4,"kb_used_meta":473147,"kb_avail":93898180,"statfs":{"total":96636764160,"available":96151736320,"internally_reserved":0,"allocated":503808,"data_stored":155014,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":484503214},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182547,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":547336,"kb_used_data":68660,"kb_used_omap":4,"kb_used_meta":478651,"kb_avail":93824504,"statfs":{"total":96636764160,"available":96076292096,"internally_reserved":0,"allocated":70307840,"data_stored":69968183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":490139306},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":69853184,"data_stored":69817054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":6762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":140607488,"data_stored":140571358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:10:00.916 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-10T16:10:00.916 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "49'20672", 'reported_seq': 29287, 'reported_epoch': 53, 'state': 'peering', 'last_fresh': '2024-09-10T16:09:55.745411+0000', 'last_change': '2024-09-10T16:09:56.009762+0000', 'last_active': '2024-09-10T16:09:56.009762+0000', 'last_peered': '2024-09-10T16:09:55.745411+0000', 'last_clean': '2024-09-10T15:59:14.132456+0000', 'last_became_active': '2024-09-10T15:59:16.274407+0000', 'last_became_peered': '2024-09-10T15:59:16.274407+0000', 'last_unstale': '2024-09-10T16:09:55.745411+0000', 'last_undegraded': '2024-09-10T16:09:55.745411+0000', 'last_fullsized': '2024-09-10T15:59:15.774356+0000', 'mapping_epoch': 54, 'log_start': "45'10600", 'ondisk_log_start': "45'10600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'objects_scrubbed': 0, 'log_size': 10072, 'log_dups_size': 0, 'ondisk_log_size': 10072, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20672, 'num_write_kb': 41344, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 4259, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1, 2], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:10:00.916 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-10T16:10:00.916 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "49'21061", 'reported_seq': 21220, 'reported_epoch': 54, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:09:56.000563+0000', 'last_change': '2024-09-10T15:59:16.308264+0000', 'last_active': '2024-09-10T16:09:56.000563+0000', 'last_peered': '2024-09-10T16:09:56.000563+0000', 'last_clean': '2024-09-10T15:59:14.111551+0000', 'last_became_active': '2024-09-10T15:59:16.300040+0000', 'last_became_peered': '2024-09-10T15:59:16.300040+0000', 'last_unstale': '2024-09-10T16:09:56.000563+0000', 'last_undegraded': '2024-09-10T15:59:15.878234+0000', 'last_fullsized': '2024-09-10T15:59:15.769328+0000', 'mapping_epoch': 42, 'log_start': "45'11000", 'ondisk_log_start': "45'11000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3442", 'last_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'objects_scrubbed': 3384, 'log_size': 10061, 'log_dups_size': 0, 'ondisk_log_size': 10061, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1292, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4372, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 17673, 'num_write_kb': 28580, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4370}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:10:00.916 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-10T16:10:00.917 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "49'21082", 'reported_seq': 21241, 'reported_epoch': 54, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:09:56.000545+0000', 'last_change': '2024-09-10T15:59:16.331899+0000', 'last_active': '2024-09-10T16:09:56.000545+0000', 'last_peered': '2024-09-10T16:09:56.000545+0000', 'last_clean': '2024-09-10T15:59:14.139436+0000', 'last_became_active': '2024-09-10T15:59:16.318715+0000', 'last_became_peered': '2024-09-10T15:59:16.318715+0000', 'last_unstale': '2024-09-10T16:09:56.000545+0000', 'last_undegraded': '2024-09-10T15:59:15.877524+0000', 'last_fullsized': '2024-09-10T15:59:15.769042+0000', 'mapping_epoch': 42, 'log_start': "45'11000", 'ondisk_log_start': "45'11000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3537", 'last_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'objects_scrubbed': 3497, 'log_size': 10082, 'log_dups_size': 0, 'ondisk_log_size': 10082, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1224, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4237, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 17581, 'num_write_kb': 28172, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4235}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:10:00.917 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-10T16:10:00.917 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "49'20781", 'reported_seq': 21024, 'reported_epoch': 54, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:09:56.159000+0000', 'last_change': '2024-09-10T16:09:55.745709+0000', 'last_active': '2024-09-10T16:09:56.159000+0000', 'last_peered': '2024-09-10T16:09:56.159000+0000', 'last_clean': '2024-09-10T15:59:14.116041+0000', 'last_became_active': '2024-09-10T15:59:16.274543+0000', 'last_became_peered': '2024-09-10T15:59:16.274543+0000', 'last_unstale': '2024-09-10T16:09:56.159000+0000', 'last_undegraded': '2024-09-10T15:59:15.812484+0000', 'last_fullsized': '2024-09-10T15:59:15.770885+0000', 'mapping_epoch': 42, 'log_start': "45'10700", 'ondisk_log_start': "45'10700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2356", 'last_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'objects_scrubbed': 2316, 'log_size': 10081, 'log_dups_size': 0, 'ondisk_log_size': 10081, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 783, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4262, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 18461, 'num_write_kb': 32292, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 45, 'num_bytes_recovered': 8675, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -4261}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:10:00.917 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-10T16:10:00.917 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "49'21170", 'reported_seq': 30035, 'reported_epoch': 54, 'state': 'active+recovery_wait', 'last_fresh': '2024-09-10T16:09:56.004450+0000', 'last_change': '2024-09-10T16:08:10.929169+0000', 'last_active': '2024-09-10T16:09:56.004450+0000', 'last_peered': '2024-09-10T16:09:56.004450+0000', 'last_clean': '2024-09-10T15:59:14.110654+0000', 'last_became_active': '2024-09-10T16:08:10.860438+0000', 'last_became_peered': '2024-09-10T16:08:10.860438+0000', 'last_unstale': '2024-09-10T16:09:56.004450+0000', 'last_undegraded': '2024-09-10T16:09:56.004450+0000', 'last_fullsized': '2024-09-10T16:09:56.004450+0000', 'mapping_epoch': 48, 'log_start': "45'11100", 'ondisk_log_start': "45'11100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2486", 'last_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'objects_scrubbed': 2440, 'log_size': 10070, 'log_dups_size': 0, 'ondisk_log_size': 10070, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 792, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18724, 'num_write_kb': 32568, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 4355, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': ['2', '0', '1'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:10:00.917 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-10T16:10:00.917 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "49'20838", 'reported_seq': 20994, 'reported_epoch': 54, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:09:56.000586+0000', 'last_change': '2024-09-10T15:59:16.315641+0000', 'last_active': '2024-09-10T16:09:56.000586+0000', 'last_peered': '2024-09-10T16:09:56.000586+0000', 'last_clean': '2024-09-10T15:59:14.117001+0000', 'last_became_active': '2024-09-10T15:59:16.302245+0000', 'last_became_peered': '2024-09-10T15:59:16.302245+0000', 'last_unstale': '2024-09-10T16:09:56.000586+0000', 'last_undegraded': '2024-09-10T15:59:15.845012+0000', 'last_fullsized': '2024-09-10T15:59:15.770337+0000', 'mapping_epoch': 42, 'log_start': "45'10800", 'ondisk_log_start': "45'10800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3404", 'last_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'objects_scrubbed': 3367, 'log_size': 10038, 'log_dups_size': 0, 'ondisk_log_size': 10038, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 983, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17469, 'num_write_kb': 28204, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4304}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:10:00.917 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-10T16:10:00.917 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "49'21048", 'reported_seq': 21211, 'reported_epoch': 54, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:09:56.001374+0000', 'last_change': '2024-09-10T15:59:16.311291+0000', 'last_active': '2024-09-10T16:09:56.001374+0000', 'last_peered': '2024-09-10T16:09:56.001374+0000', 'last_clean': '2024-09-10T15:59:14.116363+0000', 'last_became_active': '2024-09-10T15:59:16.301244+0000', 'last_became_peered': '2024-09-10T15:59:16.301244+0000', 'last_unstale': '2024-09-10T16:09:56.001374+0000', 'last_undegraded': '2024-09-10T15:59:15.883434+0000', 'last_fullsized': '2024-09-10T15:59:15.772756+0000', 'mapping_epoch': 42, 'log_start': "45'11000", 'ondisk_log_start': "45'11000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3789", 'last_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'objects_scrubbed': 3741, 'log_size': 10048, 'log_dups_size': 0, 'ondisk_log_size': 10048, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1440, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4373, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17303, 'num_write_kb': 27124, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4371}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:10:00.917 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-10T16:10:00.918 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "49'21198", 'reported_seq': 21364, 'reported_epoch': 54, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:09:56.000680+0000', 'last_change': '2024-09-10T15:59:16.313409+0000', 'last_active': '2024-09-10T16:09:56.000680+0000', 'last_peered': '2024-09-10T16:09:56.000680+0000', 'last_clean': '2024-09-10T15:59:14.129780+0000', 'last_became_active': '2024-09-10T15:59:16.300932+0000', 'last_became_peered': '2024-09-10T15:59:16.300932+0000', 'last_unstale': '2024-09-10T16:09:56.000680+0000', 'last_undegraded': '2024-09-10T15:59:15.887823+0000', 'last_fullsized': '2024-09-10T15:59:15.768513+0000', 'mapping_epoch': 42, 'log_start': "45'11100", 'ondisk_log_start': "45'11100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3880", 'last_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'objects_scrubbed': 3827, 'log_size': 10098, 'log_dups_size': 0, 'ondisk_log_size': 10098, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1397, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4325, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17361, 'num_write_kb': 27072, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4321}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:10:00.918 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T16:10:01.433 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:10:01.433 INFO:teuthology.orchestra.run.smithi115.stdout:{"fsid":"7effee29-df05-4351-9741-ee203564ed81","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 25874/24 objects degraded (107808.333%), 6 pgs degraded, 6 pgs undersized","count":12},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":765,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":55,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983954,"num_in_osds":3,"osd_in_since":1725983947,"num_remapped_pgs":6},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":16},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":5},{"state_name":"active+recovering+undersized+degraded+remapped","count":1},{"state_name":"active+recovery_wait","count":1},{"state_name":"peering","count":1}],"num_pgs":24,"num_pools":2,"num_objects":12,"data_bytes":4849,"bytes_used":1560195072,"bytes_avail":288350097408,"bytes_total":289910292480,"inactive_pgs_ratio":0.041666667908430099,"degraded_objects":25874,"degraded_total":24,"degraded_ratio":1078.0833333333333,"recovering_objects_per_sec":133,"recovering_bytes_per_sec":60,"recovering_keys_per_sec":0,"num_objects_recovered":1070,"num_bytes_recovered":483,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-10T15:57:14:967120+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T16:00:13.560776+0000","services":{}},"progress_events":{}} 2024-09-10T16:10:01.446 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-10T16:10:01.447 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:10:01.799 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:10:01.800 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:10:01.813 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":431,"stamp":"2024-09-10T16:10:01.779385+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":25015,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292098,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26377,"num_bytes_recovered":33096665,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161025,"ondisk_log_size":161025,"up":48,"acting":42,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1515348,"kb_used_data":206932,"kb_used_omap":12,"kb_used_meta":1308339,"kb_avail":281600172,"statfs":{"total":289910292480,"available":288358576128,"internally_reserved":0,"allocated":211898368,"data_stored":210856627,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1339739138},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1929,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1929,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001289"},"pg_stats":[{"pgid":"2.f","version":"49'20672","reported_seq":29296,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T16:09:57.088647+0000","last_change":"2024-09-10T16:09:57.088531+0000","last_active":"2024-09-10T16:09:57.088647+0000","last_peered":"2024-09-10T16:09:57.088647+0000","last_clean":"2024-09-10T16:09:57.088647+0000","last_became_active":"2024-09-10T16:09:57.005755+0000","last_became_peered":"2024-09-10T16:09:57.005755+0000","last_unstale":"2024-09-10T16:09:57.088647+0000","last_undegraded":"2024-09-10T16:09:57.088647+0000","last_fullsized":"2024-09-10T16:09:57.088647+0000","mapping_epoch":54,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:09:19.329697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20672,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4259,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20844","reported_seq":20980,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T16:09:57.005461+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:09:57.005461+0000","last_peered":"2024-09-10T16:09:57.005461+0000","last_clean":"2024-09-10T16:09:57.005461+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:09:57.005461+0000","last_undegraded":"2024-09-10T16:09:57.005461+0000","last_fullsized":"2024-09-10T16:09:57.005461+0000","mapping_epoch":35,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18949,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20798","reported_seq":29499,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T16:09:57.004925+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:09:57.004925+0000","last_peered":"2024-09-10T16:09:57.004925+0000","last_clean":"2024-09-10T16:09:57.004925+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:09:57.004925+0000","last_undegraded":"2024-09-10T16:09:57.004925+0000","last_fullsized":"2024-09-10T16:09:57.004925+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18802,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20660","reported_seq":20764,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T16:09:57.005318+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:09:57.005318+0000","last_peered":"2024-09-10T16:09:57.005318+0000","last_clean":"2024-09-10T16:09:57.005318+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:09:57.005318+0000","last_undegraded":"2024-09-10T16:09:57.005318+0000","last_fullsized":"2024-09-10T16:09:57.005318+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20660,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20891","reported_seq":21044,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T16:09:57.001440+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:09:57.001440+0000","last_peered":"2024-09-10T16:09:57.001440+0000","last_clean":"2024-09-10T16:09:57.001440+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:09:57.001440+0000","last_undegraded":"2024-09-10T16:09:57.001440+0000","last_fullsized":"2024-09-10T16:09:57.001440+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17602,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'21061","reported_seq":21222,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:57.000888+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:09:57.000888+0000","last_peered":"2024-09-10T16:09:57.000888+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:09:57.000888+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17673,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4370},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'21082","reported_seq":21243,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:57.000962+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:09:57.000962+0000","last_peered":"2024-09-10T16:09:57.000962+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:09:57.000962+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17581,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4235},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'21314","reported_seq":21467,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T16:09:57.000718+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:09:57.000718+0000","last_peered":"2024-09-10T16:09:57.000718+0000","last_clean":"2024-09-10T16:09:57.000718+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:09:57.000718+0000","last_undegraded":"2024-09-10T16:09:57.000718+0000","last_fullsized":"2024-09-10T16:09:57.000718+0000","mapping_epoch":34,"log_start":"45'11300","ondisk_log_start":"45'11300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18161,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T16:09:57.003703+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:09:57.003703+0000","last_peered":"2024-09-10T16:09:57.003703+0000","last_clean":"2024-09-10T16:09:57.003703+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:09:57.003703+0000","last_undegraded":"2024-09-10T16:09:57.003703+0000","last_fullsized":"2024-09-10T16:09:57.003703+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20457","reported_seq":20624,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T16:09:57.004545+0000","last_change":"2024-09-10T16:09:33.012840+0000","last_active":"2024-09-10T16:09:57.004545+0000","last_peered":"2024-09-10T16:09:57.004545+0000","last_clean":"2024-09-10T16:09:57.004545+0000","last_became_active":"2024-09-10T16:09:32.934056+0000","last_became_peered":"2024-09-10T16:09:32.934056+0000","last_unstale":"2024-09-10T16:09:57.004545+0000","last_undegraded":"2024-09-10T16:09:57.004545+0000","last_fullsized":"2024-09-10T16:09:57.004545+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:08:30.481235+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18276,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":4579,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":117,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T16:09:57.004914+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:09:57.004914+0000","last_peered":"2024-09-10T16:09:57.004914+0000","last_clean":"2024-09-10T16:09:57.004914+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:09:57.004914+0000","last_undegraded":"2024-09-10T16:09:57.004914+0000","last_fullsized":"2024-09-10T16:09:57.004914+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20910","reported_seq":29586,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T16:09:57.001273+0000","last_change":"2024-09-10T16:09:08.909580+0000","last_active":"2024-09-10T16:09:57.001273+0000","last_peered":"2024-09-10T16:09:57.001273+0000","last_clean":"2024-09-10T16:09:57.001273+0000","last_became_active":"2024-09-10T16:09:08.854941+0000","last_became_peered":"2024-09-10T16:09:08.854941+0000","last_unstale":"2024-09-10T16:09:57.001273+0000","last_undegraded":"2024-09-10T16:09:57.001273+0000","last_fullsized":"2024-09-10T16:09:57.001273+0000","mapping_epoch":50,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:13:14.846691+0000","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17414,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4257,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":127,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T16:09:57.000790+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:09:57.000790+0000","last_peered":"2024-09-10T16:09:57.000790+0000","last_clean":"2024-09-10T16:09:57.000790+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:09:57.000790+0000","last_undegraded":"2024-09-10T16:09:57.000790+0000","last_fullsized":"2024-09-10T16:09:57.000790+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20781","reported_seq":22744,"reported_epoch":55,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:10:01.165825+0000","last_change":"2024-09-10T16:09:55.745709+0000","last_active":"2024-09-10T16:10:01.165825+0000","last_peered":"2024-09-10T16:10:01.165825+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:10:01.165825+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3403,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18461,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":904,"num_bytes_recovered":8675,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3402}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":103,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T16:09:57.004400+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:09:57.004400+0000","last_peered":"2024-09-10T16:09:57.004400+0000","last_clean":"2024-09-10T16:09:57.004400+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:09:57.004400+0000","last_undegraded":"2024-09-10T16:09:57.004400+0000","last_fullsized":"2024-09-10T16:09:57.004400+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'21170","reported_seq":30037,"reported_epoch":55,"state":"active+recovery_wait","last_fresh":"2024-09-10T16:09:57.004800+0000","last_change":"2024-09-10T16:08:10.929169+0000","last_active":"2024-09-10T16:09:57.004800+0000","last_peered":"2024-09-10T16:09:57.004800+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T16:08:10.860438+0000","last_became_peered":"2024-09-10T16:08:10.860438+0000","last_unstale":"2024-09-10T16:09:57.004800+0000","last_undegraded":"2024-09-10T16:09:57.004800+0000","last_fullsized":"2024-09-10T16:09:57.004800+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18724,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","0","1"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20838","reported_seq":20996,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:57.000718+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:09:57.000718+0000","last_peered":"2024-09-10T16:09:57.000718+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:09:57.000718+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17469,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T16:09:57.004632+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:09:57.004632+0000","last_peered":"2024-09-10T16:09:57.004632+0000","last_clean":"2024-09-10T16:09:57.004632+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:09:57.004632+0000","last_undegraded":"2024-09-10T16:09:57.004632+0000","last_fullsized":"2024-09-10T16:09:57.004632+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20598","reported_seq":29187,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T16:09:57.005160+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:09:57.005160+0000","last_peered":"2024-09-10T16:09:57.005160+0000","last_clean":"2024-09-10T16:09:57.005160+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:09:57.005160+0000","last_undegraded":"2024-09-10T16:09:57.005160+0000","last_fullsized":"2024-09-10T16:09:57.005160+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16988,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":117,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T16:09:57.005454+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:09:57.005454+0000","last_peered":"2024-09-10T16:09:57.005454+0000","last_clean":"2024-09-10T16:09:57.005454+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:09:57.005454+0000","last_undegraded":"2024-09-10T16:09:57.005454+0000","last_fullsized":"2024-09-10T16:09:57.005454+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'21048","reported_seq":21213,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:57.002350+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:09:57.002350+0000","last_peered":"2024-09-10T16:09:57.002350+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:09:57.002350+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17303,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T16:09:57.005070+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:09:57.005070+0000","last_peered":"2024-09-10T16:09:57.005070+0000","last_clean":"2024-09-10T16:09:57.005070+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:09:57.005070+0000","last_undegraded":"2024-09-10T16:09:57.005070+0000","last_fullsized":"2024-09-10T16:09:57.005070+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":97,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T16:09:57.005068+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:09:57.005068+0000","last_peered":"2024-09-10T16:09:57.005068+0000","last_clean":"2024-09-10T16:09:57.005068+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:09:57.005068+0000","last_undegraded":"2024-09-10T16:09:57.005068+0000","last_fullsized":"2024-09-10T16:09:57.005068+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'21198","reported_seq":21366,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:57.001544+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:09:57.001544+0000","last_peered":"2024-09-10T16:09:57.001544+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:09:57.001544+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17361,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4321},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":25015,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292096,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26375,"num_bytes_recovered":33096646,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":210513920,"data_stored":210394691,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161022,"ondisk_log_size":161022,"up":32,"acting":26,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986320,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":502648,"kb_used_data":137760,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93869192,"statfs":{"total":96636764160,"available":96122052608,"internally_reserved":0,"allocated":141066240,"data_stored":140725324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659265,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":465352,"kb_used_data":504,"kb_used_omap":4,"kb_used_meta":464827,"kb_avail":93906488,"statfs":{"total":96636764160,"available":96160243712,"internally_reserved":0,"allocated":516096,"data_stored":160264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":475983534},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182548,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":547348,"kb_used_data":68668,"kb_used_omap":4,"kb_used_meta":478651,"kb_avail":93824492,"statfs":{"total":96636764160,"available":96076279808,"internally_reserved":0,"allocated":70316032,"data_stored":69971039,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":490139306},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":69853184,"data_stored":69817054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":6279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":140607488,"data_stored":140571358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:10:04.814 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:10:05.217 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:10:05.218 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:10:05.231 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":432,"stamp":"2024-09-10T16:10:03.779754+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":25015,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292098,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26377,"num_bytes_recovered":33096665,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161025,"ondisk_log_size":161025,"up":48,"acting":42,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1515348,"kb_used_data":206932,"kb_used_omap":12,"kb_used_meta":1308339,"kb_avail":281600172,"statfs":{"total":289910292480,"available":288358576128,"internally_reserved":0,"allocated":211898368,"data_stored":210856627,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1339739138},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1929,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1929,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001346"},"pg_stats":[{"pgid":"2.f","version":"49'20672","reported_seq":29296,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T16:09:57.088647+0000","last_change":"2024-09-10T16:09:57.088531+0000","last_active":"2024-09-10T16:09:57.088647+0000","last_peered":"2024-09-10T16:09:57.088647+0000","last_clean":"2024-09-10T16:09:57.088647+0000","last_became_active":"2024-09-10T16:09:57.005755+0000","last_became_peered":"2024-09-10T16:09:57.005755+0000","last_unstale":"2024-09-10T16:09:57.088647+0000","last_undegraded":"2024-09-10T16:09:57.088647+0000","last_fullsized":"2024-09-10T16:09:57.088647+0000","mapping_epoch":54,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:09:19.329697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20672,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4259,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20844","reported_seq":20980,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T16:09:57.005461+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:09:57.005461+0000","last_peered":"2024-09-10T16:09:57.005461+0000","last_clean":"2024-09-10T16:09:57.005461+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:09:57.005461+0000","last_undegraded":"2024-09-10T16:09:57.005461+0000","last_fullsized":"2024-09-10T16:09:57.005461+0000","mapping_epoch":35,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18949,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20798","reported_seq":29499,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T16:09:57.004925+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:09:57.004925+0000","last_peered":"2024-09-10T16:09:57.004925+0000","last_clean":"2024-09-10T16:09:57.004925+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:09:57.004925+0000","last_undegraded":"2024-09-10T16:09:57.004925+0000","last_fullsized":"2024-09-10T16:09:57.004925+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18802,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20660","reported_seq":20764,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T16:09:57.005318+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:09:57.005318+0000","last_peered":"2024-09-10T16:09:57.005318+0000","last_clean":"2024-09-10T16:09:57.005318+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:09:57.005318+0000","last_undegraded":"2024-09-10T16:09:57.005318+0000","last_fullsized":"2024-09-10T16:09:57.005318+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20660,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20891","reported_seq":21044,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T16:09:57.001440+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:09:57.001440+0000","last_peered":"2024-09-10T16:09:57.001440+0000","last_clean":"2024-09-10T16:09:57.001440+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:09:57.001440+0000","last_undegraded":"2024-09-10T16:09:57.001440+0000","last_fullsized":"2024-09-10T16:09:57.001440+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17602,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'21061","reported_seq":21222,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:57.000888+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:09:57.000888+0000","last_peered":"2024-09-10T16:09:57.000888+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:09:57.000888+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17673,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4370},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'21082","reported_seq":21243,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:57.000962+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:09:57.000962+0000","last_peered":"2024-09-10T16:09:57.000962+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:09:57.000962+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17581,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4235},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'21314","reported_seq":21467,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T16:09:57.000718+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:09:57.000718+0000","last_peered":"2024-09-10T16:09:57.000718+0000","last_clean":"2024-09-10T16:09:57.000718+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:09:57.000718+0000","last_undegraded":"2024-09-10T16:09:57.000718+0000","last_fullsized":"2024-09-10T16:09:57.000718+0000","mapping_epoch":34,"log_start":"45'11300","ondisk_log_start":"45'11300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18161,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T16:09:57.003703+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:09:57.003703+0000","last_peered":"2024-09-10T16:09:57.003703+0000","last_clean":"2024-09-10T16:09:57.003703+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:09:57.003703+0000","last_undegraded":"2024-09-10T16:09:57.003703+0000","last_fullsized":"2024-09-10T16:09:57.003703+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20457","reported_seq":20624,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T16:09:57.004545+0000","last_change":"2024-09-10T16:09:33.012840+0000","last_active":"2024-09-10T16:09:57.004545+0000","last_peered":"2024-09-10T16:09:57.004545+0000","last_clean":"2024-09-10T16:09:57.004545+0000","last_became_active":"2024-09-10T16:09:32.934056+0000","last_became_peered":"2024-09-10T16:09:32.934056+0000","last_unstale":"2024-09-10T16:09:57.004545+0000","last_undegraded":"2024-09-10T16:09:57.004545+0000","last_fullsized":"2024-09-10T16:09:57.004545+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:08:30.481235+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18276,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":4579,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":117,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T16:09:57.004914+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:09:57.004914+0000","last_peered":"2024-09-10T16:09:57.004914+0000","last_clean":"2024-09-10T16:09:57.004914+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:09:57.004914+0000","last_undegraded":"2024-09-10T16:09:57.004914+0000","last_fullsized":"2024-09-10T16:09:57.004914+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20910","reported_seq":29586,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T16:09:57.001273+0000","last_change":"2024-09-10T16:09:08.909580+0000","last_active":"2024-09-10T16:09:57.001273+0000","last_peered":"2024-09-10T16:09:57.001273+0000","last_clean":"2024-09-10T16:09:57.001273+0000","last_became_active":"2024-09-10T16:09:08.854941+0000","last_became_peered":"2024-09-10T16:09:08.854941+0000","last_unstale":"2024-09-10T16:09:57.001273+0000","last_undegraded":"2024-09-10T16:09:57.001273+0000","last_fullsized":"2024-09-10T16:09:57.001273+0000","mapping_epoch":50,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:13:14.846691+0000","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17414,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4257,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":127,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T16:09:57.000790+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:09:57.000790+0000","last_peered":"2024-09-10T16:09:57.000790+0000","last_clean":"2024-09-10T16:09:57.000790+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:09:57.000790+0000","last_undegraded":"2024-09-10T16:09:57.000790+0000","last_fullsized":"2024-09-10T16:09:57.000790+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20781","reported_seq":22744,"reported_epoch":55,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:10:01.165825+0000","last_change":"2024-09-10T16:09:55.745709+0000","last_active":"2024-09-10T16:10:01.165825+0000","last_peered":"2024-09-10T16:10:01.165825+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:10:01.165825+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3403,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18461,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":904,"num_bytes_recovered":8675,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3402}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":103,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T16:09:57.004400+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:09:57.004400+0000","last_peered":"2024-09-10T16:09:57.004400+0000","last_clean":"2024-09-10T16:09:57.004400+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:09:57.004400+0000","last_undegraded":"2024-09-10T16:09:57.004400+0000","last_fullsized":"2024-09-10T16:09:57.004400+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'21170","reported_seq":30037,"reported_epoch":55,"state":"active+recovery_wait","last_fresh":"2024-09-10T16:09:57.004800+0000","last_change":"2024-09-10T16:08:10.929169+0000","last_active":"2024-09-10T16:09:57.004800+0000","last_peered":"2024-09-10T16:09:57.004800+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T16:08:10.860438+0000","last_became_peered":"2024-09-10T16:08:10.860438+0000","last_unstale":"2024-09-10T16:09:57.004800+0000","last_undegraded":"2024-09-10T16:09:57.004800+0000","last_fullsized":"2024-09-10T16:09:57.004800+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18724,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","0","1"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20838","reported_seq":20996,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:57.000718+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:09:57.000718+0000","last_peered":"2024-09-10T16:09:57.000718+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:09:57.000718+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17469,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T16:09:57.004632+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:09:57.004632+0000","last_peered":"2024-09-10T16:09:57.004632+0000","last_clean":"2024-09-10T16:09:57.004632+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:09:57.004632+0000","last_undegraded":"2024-09-10T16:09:57.004632+0000","last_fullsized":"2024-09-10T16:09:57.004632+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20598","reported_seq":29187,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T16:09:57.005160+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:09:57.005160+0000","last_peered":"2024-09-10T16:09:57.005160+0000","last_clean":"2024-09-10T16:09:57.005160+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:09:57.005160+0000","last_undegraded":"2024-09-10T16:09:57.005160+0000","last_fullsized":"2024-09-10T16:09:57.005160+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16988,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":117,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T16:09:57.005454+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:09:57.005454+0000","last_peered":"2024-09-10T16:09:57.005454+0000","last_clean":"2024-09-10T16:09:57.005454+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:09:57.005454+0000","last_undegraded":"2024-09-10T16:09:57.005454+0000","last_fullsized":"2024-09-10T16:09:57.005454+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'21048","reported_seq":21213,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:57.002350+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:09:57.002350+0000","last_peered":"2024-09-10T16:09:57.002350+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:09:57.002350+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17303,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T16:09:57.005070+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:09:57.005070+0000","last_peered":"2024-09-10T16:09:57.005070+0000","last_clean":"2024-09-10T16:09:57.005070+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:09:57.005070+0000","last_undegraded":"2024-09-10T16:09:57.005070+0000","last_fullsized":"2024-09-10T16:09:57.005070+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":97,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T16:09:57.005068+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:09:57.005068+0000","last_peered":"2024-09-10T16:09:57.005068+0000","last_clean":"2024-09-10T16:09:57.005068+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:09:57.005068+0000","last_undegraded":"2024-09-10T16:09:57.005068+0000","last_fullsized":"2024-09-10T16:09:57.005068+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'21198","reported_seq":21366,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:57.001544+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:09:57.001544+0000","last_peered":"2024-09-10T16:09:57.001544+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:09:57.001544+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17361,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4321},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":25015,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292096,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26375,"num_bytes_recovered":33096646,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":210513920,"data_stored":210394691,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161022,"ondisk_log_size":161022,"up":32,"acting":26,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986320,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":502648,"kb_used_data":137760,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93869192,"statfs":{"total":96636764160,"available":96122052608,"internally_reserved":0,"allocated":141066240,"data_stored":140725324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659265,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":465352,"kb_used_data":504,"kb_used_omap":4,"kb_used_meta":464827,"kb_avail":93906488,"statfs":{"total":96636764160,"available":96160243712,"internally_reserved":0,"allocated":516096,"data_stored":160264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":475983534},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182548,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":547348,"kb_used_data":68668,"kb_used_omap":4,"kb_used_meta":478651,"kb_avail":93824492,"statfs":{"total":96636764160,"available":96076279808,"internally_reserved":0,"allocated":70316032,"data_stored":69971039,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":490139306},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":69853184,"data_stored":69817054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":6279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":140607488,"data_stored":140571358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:10:05.232 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:10:05.635 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:10:05.636 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:10:05.648 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":432,"stamp":"2024-09-10T16:10:03.779754+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":25015,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292098,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26377,"num_bytes_recovered":33096665,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161025,"ondisk_log_size":161025,"up":48,"acting":42,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1515348,"kb_used_data":206932,"kb_used_omap":12,"kb_used_meta":1308339,"kb_avail":281600172,"statfs":{"total":289910292480,"available":288358576128,"internally_reserved":0,"allocated":211898368,"data_stored":210856627,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1339739138},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1929,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1929,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001346"},"pg_stats":[{"pgid":"2.f","version":"49'20672","reported_seq":29296,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T16:09:57.088647+0000","last_change":"2024-09-10T16:09:57.088531+0000","last_active":"2024-09-10T16:09:57.088647+0000","last_peered":"2024-09-10T16:09:57.088647+0000","last_clean":"2024-09-10T16:09:57.088647+0000","last_became_active":"2024-09-10T16:09:57.005755+0000","last_became_peered":"2024-09-10T16:09:57.005755+0000","last_unstale":"2024-09-10T16:09:57.088647+0000","last_undegraded":"2024-09-10T16:09:57.088647+0000","last_fullsized":"2024-09-10T16:09:57.088647+0000","mapping_epoch":54,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:09:19.329697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20672,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4259,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20844","reported_seq":20980,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T16:09:57.005461+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:09:57.005461+0000","last_peered":"2024-09-10T16:09:57.005461+0000","last_clean":"2024-09-10T16:09:57.005461+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:09:57.005461+0000","last_undegraded":"2024-09-10T16:09:57.005461+0000","last_fullsized":"2024-09-10T16:09:57.005461+0000","mapping_epoch":35,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18949,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20798","reported_seq":29499,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T16:09:57.004925+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:09:57.004925+0000","last_peered":"2024-09-10T16:09:57.004925+0000","last_clean":"2024-09-10T16:09:57.004925+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:09:57.004925+0000","last_undegraded":"2024-09-10T16:09:57.004925+0000","last_fullsized":"2024-09-10T16:09:57.004925+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18802,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20660","reported_seq":20764,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T16:09:57.005318+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:09:57.005318+0000","last_peered":"2024-09-10T16:09:57.005318+0000","last_clean":"2024-09-10T16:09:57.005318+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:09:57.005318+0000","last_undegraded":"2024-09-10T16:09:57.005318+0000","last_fullsized":"2024-09-10T16:09:57.005318+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20660,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20891","reported_seq":21044,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T16:09:57.001440+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:09:57.001440+0000","last_peered":"2024-09-10T16:09:57.001440+0000","last_clean":"2024-09-10T16:09:57.001440+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:09:57.001440+0000","last_undegraded":"2024-09-10T16:09:57.001440+0000","last_fullsized":"2024-09-10T16:09:57.001440+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17602,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'21061","reported_seq":21222,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:57.000888+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:09:57.000888+0000","last_peered":"2024-09-10T16:09:57.000888+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:09:57.000888+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17673,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4370},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'21082","reported_seq":21243,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:57.000962+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:09:57.000962+0000","last_peered":"2024-09-10T16:09:57.000962+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:09:57.000962+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17581,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4235},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'21314","reported_seq":21467,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T16:09:57.000718+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:09:57.000718+0000","last_peered":"2024-09-10T16:09:57.000718+0000","last_clean":"2024-09-10T16:09:57.000718+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:09:57.000718+0000","last_undegraded":"2024-09-10T16:09:57.000718+0000","last_fullsized":"2024-09-10T16:09:57.000718+0000","mapping_epoch":34,"log_start":"45'11300","ondisk_log_start":"45'11300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18161,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T16:09:57.003703+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:09:57.003703+0000","last_peered":"2024-09-10T16:09:57.003703+0000","last_clean":"2024-09-10T16:09:57.003703+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:09:57.003703+0000","last_undegraded":"2024-09-10T16:09:57.003703+0000","last_fullsized":"2024-09-10T16:09:57.003703+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20457","reported_seq":20624,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T16:09:57.004545+0000","last_change":"2024-09-10T16:09:33.012840+0000","last_active":"2024-09-10T16:09:57.004545+0000","last_peered":"2024-09-10T16:09:57.004545+0000","last_clean":"2024-09-10T16:09:57.004545+0000","last_became_active":"2024-09-10T16:09:32.934056+0000","last_became_peered":"2024-09-10T16:09:32.934056+0000","last_unstale":"2024-09-10T16:09:57.004545+0000","last_undegraded":"2024-09-10T16:09:57.004545+0000","last_fullsized":"2024-09-10T16:09:57.004545+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:08:30.481235+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18276,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":4579,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":117,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T16:09:57.004914+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:09:57.004914+0000","last_peered":"2024-09-10T16:09:57.004914+0000","last_clean":"2024-09-10T16:09:57.004914+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:09:57.004914+0000","last_undegraded":"2024-09-10T16:09:57.004914+0000","last_fullsized":"2024-09-10T16:09:57.004914+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20910","reported_seq":29586,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T16:09:57.001273+0000","last_change":"2024-09-10T16:09:08.909580+0000","last_active":"2024-09-10T16:09:57.001273+0000","last_peered":"2024-09-10T16:09:57.001273+0000","last_clean":"2024-09-10T16:09:57.001273+0000","last_became_active":"2024-09-10T16:09:08.854941+0000","last_became_peered":"2024-09-10T16:09:08.854941+0000","last_unstale":"2024-09-10T16:09:57.001273+0000","last_undegraded":"2024-09-10T16:09:57.001273+0000","last_fullsized":"2024-09-10T16:09:57.001273+0000","mapping_epoch":50,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:13:14.846691+0000","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17414,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4257,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":127,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T16:09:57.000790+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:09:57.000790+0000","last_peered":"2024-09-10T16:09:57.000790+0000","last_clean":"2024-09-10T16:09:57.000790+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:09:57.000790+0000","last_undegraded":"2024-09-10T16:09:57.000790+0000","last_fullsized":"2024-09-10T16:09:57.000790+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20781","reported_seq":22744,"reported_epoch":55,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:10:01.165825+0000","last_change":"2024-09-10T16:09:55.745709+0000","last_active":"2024-09-10T16:10:01.165825+0000","last_peered":"2024-09-10T16:10:01.165825+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:10:01.165825+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3403,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18461,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":904,"num_bytes_recovered":8675,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3402}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":103,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T16:09:57.004400+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:09:57.004400+0000","last_peered":"2024-09-10T16:09:57.004400+0000","last_clean":"2024-09-10T16:09:57.004400+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:09:57.004400+0000","last_undegraded":"2024-09-10T16:09:57.004400+0000","last_fullsized":"2024-09-10T16:09:57.004400+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'21170","reported_seq":30037,"reported_epoch":55,"state":"active+recovery_wait","last_fresh":"2024-09-10T16:09:57.004800+0000","last_change":"2024-09-10T16:08:10.929169+0000","last_active":"2024-09-10T16:09:57.004800+0000","last_peered":"2024-09-10T16:09:57.004800+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T16:08:10.860438+0000","last_became_peered":"2024-09-10T16:08:10.860438+0000","last_unstale":"2024-09-10T16:09:57.004800+0000","last_undegraded":"2024-09-10T16:09:57.004800+0000","last_fullsized":"2024-09-10T16:09:57.004800+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18724,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","0","1"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20838","reported_seq":20996,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:57.000718+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:09:57.000718+0000","last_peered":"2024-09-10T16:09:57.000718+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:09:57.000718+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17469,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T16:09:57.004632+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:09:57.004632+0000","last_peered":"2024-09-10T16:09:57.004632+0000","last_clean":"2024-09-10T16:09:57.004632+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:09:57.004632+0000","last_undegraded":"2024-09-10T16:09:57.004632+0000","last_fullsized":"2024-09-10T16:09:57.004632+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20598","reported_seq":29187,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T16:09:57.005160+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:09:57.005160+0000","last_peered":"2024-09-10T16:09:57.005160+0000","last_clean":"2024-09-10T16:09:57.005160+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:09:57.005160+0000","last_undegraded":"2024-09-10T16:09:57.005160+0000","last_fullsized":"2024-09-10T16:09:57.005160+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16988,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":117,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T16:09:57.005454+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:09:57.005454+0000","last_peered":"2024-09-10T16:09:57.005454+0000","last_clean":"2024-09-10T16:09:57.005454+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:09:57.005454+0000","last_undegraded":"2024-09-10T16:09:57.005454+0000","last_fullsized":"2024-09-10T16:09:57.005454+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'21048","reported_seq":21213,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:57.002350+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:09:57.002350+0000","last_peered":"2024-09-10T16:09:57.002350+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:09:57.002350+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17303,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T16:09:57.005070+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:09:57.005070+0000","last_peered":"2024-09-10T16:09:57.005070+0000","last_clean":"2024-09-10T16:09:57.005070+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:09:57.005070+0000","last_undegraded":"2024-09-10T16:09:57.005070+0000","last_fullsized":"2024-09-10T16:09:57.005070+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":97,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T16:09:57.005068+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:09:57.005068+0000","last_peered":"2024-09-10T16:09:57.005068+0000","last_clean":"2024-09-10T16:09:57.005068+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:09:57.005068+0000","last_undegraded":"2024-09-10T16:09:57.005068+0000","last_fullsized":"2024-09-10T16:09:57.005068+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'21198","reported_seq":21366,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:57.001544+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:09:57.001544+0000","last_peered":"2024-09-10T16:09:57.001544+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:09:57.001544+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17361,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4321},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":25015,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292096,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26375,"num_bytes_recovered":33096646,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":210513920,"data_stored":210394691,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161022,"ondisk_log_size":161022,"up":32,"acting":26,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986320,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":502648,"kb_used_data":137760,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93869192,"statfs":{"total":96636764160,"available":96122052608,"internally_reserved":0,"allocated":141066240,"data_stored":140725324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659265,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":465352,"kb_used_data":504,"kb_used_omap":4,"kb_used_meta":464827,"kb_avail":93906488,"statfs":{"total":96636764160,"available":96160243712,"internally_reserved":0,"allocated":516096,"data_stored":160264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":475983534},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182548,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":547348,"kb_used_data":68668,"kb_used_omap":4,"kb_used_meta":478651,"kb_avail":93824492,"statfs":{"total":96636764160,"available":96076279808,"internally_reserved":0,"allocated":70316032,"data_stored":69971039,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":490139306},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":69853184,"data_stored":69817054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":6279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":140607488,"data_stored":140571358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:10:05.650 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-10T16:10:05.650 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "49'21061", 'reported_seq': 21222, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:09:57.000888+0000', 'last_change': '2024-09-10T15:59:16.308264+0000', 'last_active': '2024-09-10T16:09:57.000888+0000', 'last_peered': '2024-09-10T16:09:57.000888+0000', 'last_clean': '2024-09-10T15:59:14.111551+0000', 'last_became_active': '2024-09-10T15:59:16.300040+0000', 'last_became_peered': '2024-09-10T15:59:16.300040+0000', 'last_unstale': '2024-09-10T16:09:57.000888+0000', 'last_undegraded': '2024-09-10T15:59:15.878234+0000', 'last_fullsized': '2024-09-10T15:59:15.769328+0000', 'mapping_epoch': 42, 'log_start': "45'11000", 'ondisk_log_start': "45'11000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3442", 'last_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'objects_scrubbed': 3384, 'log_size': 10061, 'log_dups_size': 0, 'ondisk_log_size': 10061, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1292, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4372, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 17673, 'num_write_kb': 28580, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4370}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:10:05.650 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-10T16:10:05.650 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "49'21082", 'reported_seq': 21243, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:09:57.000962+0000', 'last_change': '2024-09-10T15:59:16.331899+0000', 'last_active': '2024-09-10T16:09:57.000962+0000', 'last_peered': '2024-09-10T16:09:57.000962+0000', 'last_clean': '2024-09-10T15:59:14.139436+0000', 'last_became_active': '2024-09-10T15:59:16.318715+0000', 'last_became_peered': '2024-09-10T15:59:16.318715+0000', 'last_unstale': '2024-09-10T16:09:57.000962+0000', 'last_undegraded': '2024-09-10T15:59:15.877524+0000', 'last_fullsized': '2024-09-10T15:59:15.769042+0000', 'mapping_epoch': 42, 'log_start': "45'11000", 'ondisk_log_start': "45'11000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3537", 'last_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'objects_scrubbed': 3497, 'log_size': 10082, 'log_dups_size': 0, 'ondisk_log_size': 10082, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1224, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4237, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 17581, 'num_write_kb': 28172, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4235}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:10:05.650 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-10T16:10:05.650 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "49'20781", 'reported_seq': 22744, 'reported_epoch': 55, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:10:01.165825+0000', 'last_change': '2024-09-10T16:09:55.745709+0000', 'last_active': '2024-09-10T16:10:01.165825+0000', 'last_peered': '2024-09-10T16:10:01.165825+0000', 'last_clean': '2024-09-10T15:59:14.116041+0000', 'last_became_active': '2024-09-10T15:59:16.274543+0000', 'last_became_peered': '2024-09-10T15:59:16.274543+0000', 'last_unstale': '2024-09-10T16:10:01.165825+0000', 'last_undegraded': '2024-09-10T15:59:15.812484+0000', 'last_fullsized': '2024-09-10T15:59:15.770885+0000', 'mapping_epoch': 42, 'log_start': "45'10700", 'ondisk_log_start': "45'10700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2356", 'last_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'objects_scrubbed': 2316, 'log_size': 10081, 'log_dups_size': 0, 'ondisk_log_size': 10081, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 783, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3403, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 18461, 'num_write_kb': 32292, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 904, 'num_bytes_recovered': 8675, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3402}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:10:05.650 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-10T16:10:05.650 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "49'21170", 'reported_seq': 30037, 'reported_epoch': 55, 'state': 'active+recovery_wait', 'last_fresh': '2024-09-10T16:09:57.004800+0000', 'last_change': '2024-09-10T16:08:10.929169+0000', 'last_active': '2024-09-10T16:09:57.004800+0000', 'last_peered': '2024-09-10T16:09:57.004800+0000', 'last_clean': '2024-09-10T15:59:14.110654+0000', 'last_became_active': '2024-09-10T16:08:10.860438+0000', 'last_became_peered': '2024-09-10T16:08:10.860438+0000', 'last_unstale': '2024-09-10T16:09:57.004800+0000', 'last_undegraded': '2024-09-10T16:09:57.004800+0000', 'last_fullsized': '2024-09-10T16:09:57.004800+0000', 'mapping_epoch': 48, 'log_start': "45'11100", 'ondisk_log_start': "45'11100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2486", 'last_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'objects_scrubbed': 2440, 'log_size': 10070, 'log_dups_size': 0, 'ondisk_log_size': 10070, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 792, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18724, 'num_write_kb': 32568, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 4355, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': ['2', '0', '1'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:10:05.650 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-10T16:10:05.650 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "49'20838", 'reported_seq': 20996, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:09:57.000718+0000', 'last_change': '2024-09-10T15:59:16.315641+0000', 'last_active': '2024-09-10T16:09:57.000718+0000', 'last_peered': '2024-09-10T16:09:57.000718+0000', 'last_clean': '2024-09-10T15:59:14.117001+0000', 'last_became_active': '2024-09-10T15:59:16.302245+0000', 'last_became_peered': '2024-09-10T15:59:16.302245+0000', 'last_unstale': '2024-09-10T16:09:57.000718+0000', 'last_undegraded': '2024-09-10T15:59:15.845012+0000', 'last_fullsized': '2024-09-10T15:59:15.770337+0000', 'mapping_epoch': 42, 'log_start': "45'10800", 'ondisk_log_start': "45'10800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3404", 'last_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'objects_scrubbed': 3367, 'log_size': 10038, 'log_dups_size': 0, 'ondisk_log_size': 10038, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 983, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17469, 'num_write_kb': 28204, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4304}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:10:05.651 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-10T16:10:05.651 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "49'21048", 'reported_seq': 21213, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:09:57.002350+0000', 'last_change': '2024-09-10T15:59:16.311291+0000', 'last_active': '2024-09-10T16:09:57.002350+0000', 'last_peered': '2024-09-10T16:09:57.002350+0000', 'last_clean': '2024-09-10T15:59:14.116363+0000', 'last_became_active': '2024-09-10T15:59:16.301244+0000', 'last_became_peered': '2024-09-10T15:59:16.301244+0000', 'last_unstale': '2024-09-10T16:09:57.002350+0000', 'last_undegraded': '2024-09-10T15:59:15.883434+0000', 'last_fullsized': '2024-09-10T15:59:15.772756+0000', 'mapping_epoch': 42, 'log_start': "45'11000", 'ondisk_log_start': "45'11000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3789", 'last_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'objects_scrubbed': 3741, 'log_size': 10048, 'log_dups_size': 0, 'ondisk_log_size': 10048, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1440, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4373, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17303, 'num_write_kb': 27124, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4371}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:10:05.651 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-10T16:10:05.651 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "49'21198", 'reported_seq': 21366, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:09:57.001544+0000', 'last_change': '2024-09-10T15:59:16.313409+0000', 'last_active': '2024-09-10T16:09:57.001544+0000', 'last_peered': '2024-09-10T16:09:57.001544+0000', 'last_clean': '2024-09-10T15:59:14.129780+0000', 'last_became_active': '2024-09-10T15:59:16.300932+0000', 'last_became_peered': '2024-09-10T15:59:16.300932+0000', 'last_unstale': '2024-09-10T16:09:57.001544+0000', 'last_undegraded': '2024-09-10T15:59:15.887823+0000', 'last_fullsized': '2024-09-10T15:59:15.768513+0000', 'mapping_epoch': 42, 'log_start': "45'11100", 'ondisk_log_start': "45'11100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3880", 'last_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'objects_scrubbed': 3827, 'log_size': 10098, 'log_dups_size': 0, 'ondisk_log_size': 10098, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1397, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4325, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17361, 'num_write_kb': 27072, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4321}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:10:05.651 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T16:10:06.118 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:10:06.118 INFO:teuthology.orchestra.run.smithi115.stdout:{"fsid":"7effee29-df05-4351-9741-ee203564ed81","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 25015/24 objects degraded (104229.167%), 6 pgs degraded, 6 pgs undersized","count":12},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":770,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":55,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983954,"num_in_osds":3,"osd_in_since":1725983947,"num_remapped_pgs":6},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":17},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":5},{"state_name":"active+recovery_wait","count":1},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":12,"data_bytes":4849,"bytes_used":1560236032,"bytes_avail":288350056448,"bytes_total":289910292480,"degraded_objects":25015,"degraded_total":24,"degraded_ratio":1042.2916666666667,"recovering_objects_per_sec":197,"recovering_bytes_per_sec":49,"recovering_keys_per_sec":0,"num_objects_recovered":1929,"num_bytes_recovered":483,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-10T15:57:14:967120+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T16:00:13.560776+0000","services":{}},"progress_events":{}} 2024-09-10T16:10:06.132 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-10T16:10:06.132 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:10:06.743 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:10:06.744 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:10:06.756 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":433,"stamp":"2024-09-10T16:10:05.780054+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":25015,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292098,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26377,"num_bytes_recovered":33096665,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161025,"ondisk_log_size":161025,"up":48,"acting":42,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1523668,"kb_used_data":206932,"kb_used_omap":12,"kb_used_meta":1316659,"kb_avail":281591852,"statfs":{"total":289910292480,"available":288350056448,"internally_reserved":0,"allocated":211898368,"data_stored":210856627,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1348258818},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1929,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1929,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.779082"},"pg_stats":[{"pgid":"2.f","version":"49'20672","reported_seq":29296,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T16:09:57.088647+0000","last_change":"2024-09-10T16:09:57.088531+0000","last_active":"2024-09-10T16:09:57.088647+0000","last_peered":"2024-09-10T16:09:57.088647+0000","last_clean":"2024-09-10T16:09:57.088647+0000","last_became_active":"2024-09-10T16:09:57.005755+0000","last_became_peered":"2024-09-10T16:09:57.005755+0000","last_unstale":"2024-09-10T16:09:57.088647+0000","last_undegraded":"2024-09-10T16:09:57.088647+0000","last_fullsized":"2024-09-10T16:09:57.088647+0000","mapping_epoch":54,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:09:19.329697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20672,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4259,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20844","reported_seq":20980,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T16:09:57.005461+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:09:57.005461+0000","last_peered":"2024-09-10T16:09:57.005461+0000","last_clean":"2024-09-10T16:09:57.005461+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:09:57.005461+0000","last_undegraded":"2024-09-10T16:09:57.005461+0000","last_fullsized":"2024-09-10T16:09:57.005461+0000","mapping_epoch":35,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18949,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20798","reported_seq":29499,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T16:09:57.004925+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:09:57.004925+0000","last_peered":"2024-09-10T16:09:57.004925+0000","last_clean":"2024-09-10T16:09:57.004925+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:09:57.004925+0000","last_undegraded":"2024-09-10T16:09:57.004925+0000","last_fullsized":"2024-09-10T16:09:57.004925+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18802,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20660","reported_seq":20764,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T16:09:57.005318+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:09:57.005318+0000","last_peered":"2024-09-10T16:09:57.005318+0000","last_clean":"2024-09-10T16:09:57.005318+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:09:57.005318+0000","last_undegraded":"2024-09-10T16:09:57.005318+0000","last_fullsized":"2024-09-10T16:09:57.005318+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20660,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20891","reported_seq":21044,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T16:09:57.001440+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:09:57.001440+0000","last_peered":"2024-09-10T16:09:57.001440+0000","last_clean":"2024-09-10T16:09:57.001440+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:09:57.001440+0000","last_undegraded":"2024-09-10T16:09:57.001440+0000","last_fullsized":"2024-09-10T16:09:57.001440+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17602,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'21061","reported_seq":21222,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:57.000888+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:09:57.000888+0000","last_peered":"2024-09-10T16:09:57.000888+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:09:57.000888+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17673,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4370},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'21082","reported_seq":21243,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:57.000962+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:09:57.000962+0000","last_peered":"2024-09-10T16:09:57.000962+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:09:57.000962+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17581,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4235},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'21314","reported_seq":21467,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T16:09:57.000718+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:09:57.000718+0000","last_peered":"2024-09-10T16:09:57.000718+0000","last_clean":"2024-09-10T16:09:57.000718+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:09:57.000718+0000","last_undegraded":"2024-09-10T16:09:57.000718+0000","last_fullsized":"2024-09-10T16:09:57.000718+0000","mapping_epoch":34,"log_start":"45'11300","ondisk_log_start":"45'11300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18161,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T16:09:57.003703+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:09:57.003703+0000","last_peered":"2024-09-10T16:09:57.003703+0000","last_clean":"2024-09-10T16:09:57.003703+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:09:57.003703+0000","last_undegraded":"2024-09-10T16:09:57.003703+0000","last_fullsized":"2024-09-10T16:09:57.003703+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20457","reported_seq":20624,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T16:09:57.004545+0000","last_change":"2024-09-10T16:09:33.012840+0000","last_active":"2024-09-10T16:09:57.004545+0000","last_peered":"2024-09-10T16:09:57.004545+0000","last_clean":"2024-09-10T16:09:57.004545+0000","last_became_active":"2024-09-10T16:09:32.934056+0000","last_became_peered":"2024-09-10T16:09:32.934056+0000","last_unstale":"2024-09-10T16:09:57.004545+0000","last_undegraded":"2024-09-10T16:09:57.004545+0000","last_fullsized":"2024-09-10T16:09:57.004545+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:08:30.481235+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18276,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":4579,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":117,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T16:09:57.004914+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:09:57.004914+0000","last_peered":"2024-09-10T16:09:57.004914+0000","last_clean":"2024-09-10T16:09:57.004914+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:09:57.004914+0000","last_undegraded":"2024-09-10T16:09:57.004914+0000","last_fullsized":"2024-09-10T16:09:57.004914+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20910","reported_seq":29586,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T16:09:57.001273+0000","last_change":"2024-09-10T16:09:08.909580+0000","last_active":"2024-09-10T16:09:57.001273+0000","last_peered":"2024-09-10T16:09:57.001273+0000","last_clean":"2024-09-10T16:09:57.001273+0000","last_became_active":"2024-09-10T16:09:08.854941+0000","last_became_peered":"2024-09-10T16:09:08.854941+0000","last_unstale":"2024-09-10T16:09:57.001273+0000","last_undegraded":"2024-09-10T16:09:57.001273+0000","last_fullsized":"2024-09-10T16:09:57.001273+0000","mapping_epoch":50,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:13:14.846691+0000","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17414,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4257,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":127,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T16:09:57.000790+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:09:57.000790+0000","last_peered":"2024-09-10T16:09:57.000790+0000","last_clean":"2024-09-10T16:09:57.000790+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:09:57.000790+0000","last_undegraded":"2024-09-10T16:09:57.000790+0000","last_fullsized":"2024-09-10T16:09:57.000790+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20781","reported_seq":22744,"reported_epoch":55,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:10:01.165825+0000","last_change":"2024-09-10T16:09:55.745709+0000","last_active":"2024-09-10T16:10:01.165825+0000","last_peered":"2024-09-10T16:10:01.165825+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:10:01.165825+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3403,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18461,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":904,"num_bytes_recovered":8675,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3402}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":103,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T16:09:57.004400+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:09:57.004400+0000","last_peered":"2024-09-10T16:09:57.004400+0000","last_clean":"2024-09-10T16:09:57.004400+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:09:57.004400+0000","last_undegraded":"2024-09-10T16:09:57.004400+0000","last_fullsized":"2024-09-10T16:09:57.004400+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'21170","reported_seq":30037,"reported_epoch":55,"state":"active+recovery_wait","last_fresh":"2024-09-10T16:09:57.004800+0000","last_change":"2024-09-10T16:08:10.929169+0000","last_active":"2024-09-10T16:09:57.004800+0000","last_peered":"2024-09-10T16:09:57.004800+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T16:08:10.860438+0000","last_became_peered":"2024-09-10T16:08:10.860438+0000","last_unstale":"2024-09-10T16:09:57.004800+0000","last_undegraded":"2024-09-10T16:09:57.004800+0000","last_fullsized":"2024-09-10T16:09:57.004800+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18724,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","0","1"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20838","reported_seq":20996,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:57.000718+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:09:57.000718+0000","last_peered":"2024-09-10T16:09:57.000718+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:09:57.000718+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17469,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T16:09:57.004632+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:09:57.004632+0000","last_peered":"2024-09-10T16:09:57.004632+0000","last_clean":"2024-09-10T16:09:57.004632+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:09:57.004632+0000","last_undegraded":"2024-09-10T16:09:57.004632+0000","last_fullsized":"2024-09-10T16:09:57.004632+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20598","reported_seq":29187,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T16:09:57.005160+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:09:57.005160+0000","last_peered":"2024-09-10T16:09:57.005160+0000","last_clean":"2024-09-10T16:09:57.005160+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:09:57.005160+0000","last_undegraded":"2024-09-10T16:09:57.005160+0000","last_fullsized":"2024-09-10T16:09:57.005160+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16988,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":117,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T16:09:57.005454+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:09:57.005454+0000","last_peered":"2024-09-10T16:09:57.005454+0000","last_clean":"2024-09-10T16:09:57.005454+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:09:57.005454+0000","last_undegraded":"2024-09-10T16:09:57.005454+0000","last_fullsized":"2024-09-10T16:09:57.005454+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'21048","reported_seq":21213,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:57.002350+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:09:57.002350+0000","last_peered":"2024-09-10T16:09:57.002350+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:09:57.002350+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17303,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T16:09:57.005070+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:09:57.005070+0000","last_peered":"2024-09-10T16:09:57.005070+0000","last_clean":"2024-09-10T16:09:57.005070+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:09:57.005070+0000","last_undegraded":"2024-09-10T16:09:57.005070+0000","last_fullsized":"2024-09-10T16:09:57.005070+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":97,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T16:09:57.005068+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:09:57.005068+0000","last_peered":"2024-09-10T16:09:57.005068+0000","last_clean":"2024-09-10T16:09:57.005068+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:09:57.005068+0000","last_undegraded":"2024-09-10T16:09:57.005068+0000","last_fullsized":"2024-09-10T16:09:57.005068+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'21198","reported_seq":21366,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:57.001544+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:09:57.001544+0000","last_peered":"2024-09-10T16:09:57.001544+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:09:57.001544+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17361,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4321},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":25015,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292096,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26375,"num_bytes_recovered":33096646,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":210513920,"data_stored":210394691,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161022,"ondisk_log_size":161022,"up":32,"acting":26,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986320,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":502648,"kb_used_data":137760,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93869192,"statfs":{"total":96636764160,"available":96122052608,"internally_reserved":0,"allocated":141066240,"data_stored":140725324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659266,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":473672,"kb_used_data":504,"kb_used_omap":4,"kb_used_meta":473147,"kb_avail":93898168,"statfs":{"total":96636764160,"available":96151724032,"internally_reserved":0,"allocated":516096,"data_stored":160264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":484503214},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182548,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":547348,"kb_used_data":68668,"kb_used_omap":4,"kb_used_meta":478651,"kb_avail":93824492,"statfs":{"total":96636764160,"available":96076279808,"internally_reserved":0,"allocated":70316032,"data_stored":69971039,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":490139306},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":69853184,"data_stored":69817054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":6279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":140607488,"data_stored":140571358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:10:09.759 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:10:10.075 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:10:10.076 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:10:10.088 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":435,"stamp":"2024-09-10T16:10:09.780712+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":24066,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292098,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27326,"num_bytes_recovered":33096665,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161025,"ondisk_log_size":161025,"up":48,"acting":42,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1523668,"kb_used_data":206932,"kb_used_omap":12,"kb_used_meta":1316659,"kb_avail":281591852,"statfs":{"total":289910292480,"available":288350056448,"internally_reserved":0,"allocated":211898368,"data_stored":210856627,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1348258818},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1808,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1808,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001978"},"pg_stats":[{"pgid":"2.f","version":"49'20672","reported_seq":29296,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T16:09:57.088647+0000","last_change":"2024-09-10T16:09:57.088531+0000","last_active":"2024-09-10T16:09:57.088647+0000","last_peered":"2024-09-10T16:09:57.088647+0000","last_clean":"2024-09-10T16:09:57.088647+0000","last_became_active":"2024-09-10T16:09:57.005755+0000","last_became_peered":"2024-09-10T16:09:57.005755+0000","last_unstale":"2024-09-10T16:09:57.088647+0000","last_undegraded":"2024-09-10T16:09:57.088647+0000","last_fullsized":"2024-09-10T16:09:57.088647+0000","mapping_epoch":54,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:09:19.329697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20672,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4259,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20844","reported_seq":20980,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T16:09:57.005461+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:09:57.005461+0000","last_peered":"2024-09-10T16:09:57.005461+0000","last_clean":"2024-09-10T16:09:57.005461+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:09:57.005461+0000","last_undegraded":"2024-09-10T16:09:57.005461+0000","last_fullsized":"2024-09-10T16:09:57.005461+0000","mapping_epoch":35,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18949,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20798","reported_seq":29499,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T16:09:57.004925+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:09:57.004925+0000","last_peered":"2024-09-10T16:09:57.004925+0000","last_clean":"2024-09-10T16:09:57.004925+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:09:57.004925+0000","last_undegraded":"2024-09-10T16:09:57.004925+0000","last_fullsized":"2024-09-10T16:09:57.004925+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18802,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20660","reported_seq":20764,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T16:09:57.005318+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:09:57.005318+0000","last_peered":"2024-09-10T16:09:57.005318+0000","last_clean":"2024-09-10T16:09:57.005318+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:09:57.005318+0000","last_undegraded":"2024-09-10T16:09:57.005318+0000","last_fullsized":"2024-09-10T16:09:57.005318+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20660,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20891","reported_seq":21044,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T16:09:57.001440+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:09:57.001440+0000","last_peered":"2024-09-10T16:09:57.001440+0000","last_clean":"2024-09-10T16:09:57.001440+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:09:57.001440+0000","last_undegraded":"2024-09-10T16:09:57.001440+0000","last_fullsized":"2024-09-10T16:09:57.001440+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17602,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'21061","reported_seq":21222,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:57.000888+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:09:57.000888+0000","last_peered":"2024-09-10T16:09:57.000888+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:09:57.000888+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17673,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4370},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'21082","reported_seq":21243,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:57.000962+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:09:57.000962+0000","last_peered":"2024-09-10T16:09:57.000962+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:09:57.000962+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17581,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4235},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'21314","reported_seq":21467,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T16:09:57.000718+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:09:57.000718+0000","last_peered":"2024-09-10T16:09:57.000718+0000","last_clean":"2024-09-10T16:09:57.000718+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:09:57.000718+0000","last_undegraded":"2024-09-10T16:09:57.000718+0000","last_fullsized":"2024-09-10T16:09:57.000718+0000","mapping_epoch":34,"log_start":"45'11300","ondisk_log_start":"45'11300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18161,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T16:09:57.003703+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:09:57.003703+0000","last_peered":"2024-09-10T16:09:57.003703+0000","last_clean":"2024-09-10T16:09:57.003703+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:09:57.003703+0000","last_undegraded":"2024-09-10T16:09:57.003703+0000","last_fullsized":"2024-09-10T16:09:57.003703+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20457","reported_seq":20624,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T16:09:57.004545+0000","last_change":"2024-09-10T16:09:33.012840+0000","last_active":"2024-09-10T16:09:57.004545+0000","last_peered":"2024-09-10T16:09:57.004545+0000","last_clean":"2024-09-10T16:09:57.004545+0000","last_became_active":"2024-09-10T16:09:32.934056+0000","last_became_peered":"2024-09-10T16:09:32.934056+0000","last_unstale":"2024-09-10T16:09:57.004545+0000","last_undegraded":"2024-09-10T16:09:57.004545+0000","last_fullsized":"2024-09-10T16:09:57.004545+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:08:30.481235+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18276,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":4579,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":117,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T16:09:57.004914+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:09:57.004914+0000","last_peered":"2024-09-10T16:09:57.004914+0000","last_clean":"2024-09-10T16:09:57.004914+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:09:57.004914+0000","last_undegraded":"2024-09-10T16:09:57.004914+0000","last_fullsized":"2024-09-10T16:09:57.004914+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20910","reported_seq":29586,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T16:09:57.001273+0000","last_change":"2024-09-10T16:09:08.909580+0000","last_active":"2024-09-10T16:09:57.001273+0000","last_peered":"2024-09-10T16:09:57.001273+0000","last_clean":"2024-09-10T16:09:57.001273+0000","last_became_active":"2024-09-10T16:09:08.854941+0000","last_became_peered":"2024-09-10T16:09:08.854941+0000","last_unstale":"2024-09-10T16:09:57.001273+0000","last_undegraded":"2024-09-10T16:09:57.001273+0000","last_fullsized":"2024-09-10T16:09:57.001273+0000","mapping_epoch":50,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:13:14.846691+0000","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17414,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4257,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":127,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T16:09:57.000790+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:09:57.000790+0000","last_peered":"2024-09-10T16:09:57.000790+0000","last_clean":"2024-09-10T16:09:57.000790+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:09:57.000790+0000","last_undegraded":"2024-09-10T16:09:57.000790+0000","last_fullsized":"2024-09-10T16:09:57.000790+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20781","reported_seq":24642,"reported_epoch":55,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:10:06.172271+0000","last_change":"2024-09-10T16:09:55.745709+0000","last_active":"2024-09-10T16:10:06.172271+0000","last_peered":"2024-09-10T16:10:06.172271+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:10:06.172271+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2454,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18461,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1853,"num_bytes_recovered":8675,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2453}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":103,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T16:09:57.004400+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:09:57.004400+0000","last_peered":"2024-09-10T16:09:57.004400+0000","last_clean":"2024-09-10T16:09:57.004400+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:09:57.004400+0000","last_undegraded":"2024-09-10T16:09:57.004400+0000","last_fullsized":"2024-09-10T16:09:57.004400+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'21170","reported_seq":30037,"reported_epoch":55,"state":"active+recovery_wait","last_fresh":"2024-09-10T16:09:57.004800+0000","last_change":"2024-09-10T16:08:10.929169+0000","last_active":"2024-09-10T16:09:57.004800+0000","last_peered":"2024-09-10T16:09:57.004800+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T16:08:10.860438+0000","last_became_peered":"2024-09-10T16:08:10.860438+0000","last_unstale":"2024-09-10T16:09:57.004800+0000","last_undegraded":"2024-09-10T16:09:57.004800+0000","last_fullsized":"2024-09-10T16:09:57.004800+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18724,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","0","1"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20838","reported_seq":20996,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:57.000718+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:09:57.000718+0000","last_peered":"2024-09-10T16:09:57.000718+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:09:57.000718+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17469,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T16:09:57.004632+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:09:57.004632+0000","last_peered":"2024-09-10T16:09:57.004632+0000","last_clean":"2024-09-10T16:09:57.004632+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:09:57.004632+0000","last_undegraded":"2024-09-10T16:09:57.004632+0000","last_fullsized":"2024-09-10T16:09:57.004632+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20598","reported_seq":29187,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T16:09:57.005160+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:09:57.005160+0000","last_peered":"2024-09-10T16:09:57.005160+0000","last_clean":"2024-09-10T16:09:57.005160+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:09:57.005160+0000","last_undegraded":"2024-09-10T16:09:57.005160+0000","last_fullsized":"2024-09-10T16:09:57.005160+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16988,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":117,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T16:09:57.005454+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:09:57.005454+0000","last_peered":"2024-09-10T16:09:57.005454+0000","last_clean":"2024-09-10T16:09:57.005454+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:09:57.005454+0000","last_undegraded":"2024-09-10T16:09:57.005454+0000","last_fullsized":"2024-09-10T16:09:57.005454+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'21048","reported_seq":21213,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:57.002350+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:09:57.002350+0000","last_peered":"2024-09-10T16:09:57.002350+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:09:57.002350+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17303,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T16:09:57.005070+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:09:57.005070+0000","last_peered":"2024-09-10T16:09:57.005070+0000","last_clean":"2024-09-10T16:09:57.005070+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:09:57.005070+0000","last_undegraded":"2024-09-10T16:09:57.005070+0000","last_fullsized":"2024-09-10T16:09:57.005070+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":97,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T16:09:57.005068+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:09:57.005068+0000","last_peered":"2024-09-10T16:09:57.005068+0000","last_clean":"2024-09-10T16:09:57.005068+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:09:57.005068+0000","last_undegraded":"2024-09-10T16:09:57.005068+0000","last_fullsized":"2024-09-10T16:09:57.005068+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'21198","reported_seq":21366,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:57.001544+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:09:57.001544+0000","last_peered":"2024-09-10T16:09:57.001544+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:09:57.001544+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17361,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4321},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":24066,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292096,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27324,"num_bytes_recovered":33096646,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":210513920,"data_stored":210394691,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161022,"ondisk_log_size":161022,"up":32,"acting":26,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986321,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":502648,"kb_used_data":137760,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93869192,"statfs":{"total":96636764160,"available":96122052608,"internally_reserved":0,"allocated":141066240,"data_stored":140725324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659266,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":473672,"kb_used_data":504,"kb_used_omap":4,"kb_used_meta":473147,"kb_avail":93898168,"statfs":{"total":96636764160,"available":96151724032,"internally_reserved":0,"allocated":516096,"data_stored":160264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":484503214},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182549,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":547348,"kb_used_data":68668,"kb_used_omap":4,"kb_used_meta":478651,"kb_avail":93824492,"statfs":{"total":96636764160,"available":96076279808,"internally_reserved":0,"allocated":70316032,"data_stored":69971039,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":490139306},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":69853184,"data_stored":69817054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":6279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":140607488,"data_stored":140571358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:10:10.089 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:10:10.408 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:10:10.409 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:10:10.422 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":435,"stamp":"2024-09-10T16:10:09.780712+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":24066,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292098,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27326,"num_bytes_recovered":33096665,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161025,"ondisk_log_size":161025,"up":48,"acting":42,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1523668,"kb_used_data":206932,"kb_used_omap":12,"kb_used_meta":1316659,"kb_avail":281591852,"statfs":{"total":289910292480,"available":288350056448,"internally_reserved":0,"allocated":211898368,"data_stored":210856627,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1348258818},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1808,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1808,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001978"},"pg_stats":[{"pgid":"2.f","version":"49'20672","reported_seq":29296,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T16:09:57.088647+0000","last_change":"2024-09-10T16:09:57.088531+0000","last_active":"2024-09-10T16:09:57.088647+0000","last_peered":"2024-09-10T16:09:57.088647+0000","last_clean":"2024-09-10T16:09:57.088647+0000","last_became_active":"2024-09-10T16:09:57.005755+0000","last_became_peered":"2024-09-10T16:09:57.005755+0000","last_unstale":"2024-09-10T16:09:57.088647+0000","last_undegraded":"2024-09-10T16:09:57.088647+0000","last_fullsized":"2024-09-10T16:09:57.088647+0000","mapping_epoch":54,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:09:19.329697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20672,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4259,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20844","reported_seq":20980,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T16:09:57.005461+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:09:57.005461+0000","last_peered":"2024-09-10T16:09:57.005461+0000","last_clean":"2024-09-10T16:09:57.005461+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:09:57.005461+0000","last_undegraded":"2024-09-10T16:09:57.005461+0000","last_fullsized":"2024-09-10T16:09:57.005461+0000","mapping_epoch":35,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18949,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20798","reported_seq":29499,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T16:09:57.004925+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:09:57.004925+0000","last_peered":"2024-09-10T16:09:57.004925+0000","last_clean":"2024-09-10T16:09:57.004925+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:09:57.004925+0000","last_undegraded":"2024-09-10T16:09:57.004925+0000","last_fullsized":"2024-09-10T16:09:57.004925+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18802,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20660","reported_seq":20764,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T16:09:57.005318+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:09:57.005318+0000","last_peered":"2024-09-10T16:09:57.005318+0000","last_clean":"2024-09-10T16:09:57.005318+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:09:57.005318+0000","last_undegraded":"2024-09-10T16:09:57.005318+0000","last_fullsized":"2024-09-10T16:09:57.005318+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20660,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20891","reported_seq":21044,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T16:09:57.001440+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:09:57.001440+0000","last_peered":"2024-09-10T16:09:57.001440+0000","last_clean":"2024-09-10T16:09:57.001440+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:09:57.001440+0000","last_undegraded":"2024-09-10T16:09:57.001440+0000","last_fullsized":"2024-09-10T16:09:57.001440+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17602,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'21061","reported_seq":21222,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:57.000888+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:09:57.000888+0000","last_peered":"2024-09-10T16:09:57.000888+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:09:57.000888+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17673,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4370},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'21082","reported_seq":21243,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:57.000962+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:09:57.000962+0000","last_peered":"2024-09-10T16:09:57.000962+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:09:57.000962+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17581,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4235},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'21314","reported_seq":21467,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T16:09:57.000718+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:09:57.000718+0000","last_peered":"2024-09-10T16:09:57.000718+0000","last_clean":"2024-09-10T16:09:57.000718+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:09:57.000718+0000","last_undegraded":"2024-09-10T16:09:57.000718+0000","last_fullsized":"2024-09-10T16:09:57.000718+0000","mapping_epoch":34,"log_start":"45'11300","ondisk_log_start":"45'11300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18161,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T16:09:57.003703+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:09:57.003703+0000","last_peered":"2024-09-10T16:09:57.003703+0000","last_clean":"2024-09-10T16:09:57.003703+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:09:57.003703+0000","last_undegraded":"2024-09-10T16:09:57.003703+0000","last_fullsized":"2024-09-10T16:09:57.003703+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20457","reported_seq":20624,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T16:09:57.004545+0000","last_change":"2024-09-10T16:09:33.012840+0000","last_active":"2024-09-10T16:09:57.004545+0000","last_peered":"2024-09-10T16:09:57.004545+0000","last_clean":"2024-09-10T16:09:57.004545+0000","last_became_active":"2024-09-10T16:09:32.934056+0000","last_became_peered":"2024-09-10T16:09:32.934056+0000","last_unstale":"2024-09-10T16:09:57.004545+0000","last_undegraded":"2024-09-10T16:09:57.004545+0000","last_fullsized":"2024-09-10T16:09:57.004545+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:08:30.481235+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18276,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":4579,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":117,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T16:09:57.004914+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:09:57.004914+0000","last_peered":"2024-09-10T16:09:57.004914+0000","last_clean":"2024-09-10T16:09:57.004914+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:09:57.004914+0000","last_undegraded":"2024-09-10T16:09:57.004914+0000","last_fullsized":"2024-09-10T16:09:57.004914+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20910","reported_seq":29586,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T16:09:57.001273+0000","last_change":"2024-09-10T16:09:08.909580+0000","last_active":"2024-09-10T16:09:57.001273+0000","last_peered":"2024-09-10T16:09:57.001273+0000","last_clean":"2024-09-10T16:09:57.001273+0000","last_became_active":"2024-09-10T16:09:08.854941+0000","last_became_peered":"2024-09-10T16:09:08.854941+0000","last_unstale":"2024-09-10T16:09:57.001273+0000","last_undegraded":"2024-09-10T16:09:57.001273+0000","last_fullsized":"2024-09-10T16:09:57.001273+0000","mapping_epoch":50,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:13:14.846691+0000","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17414,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4257,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":127,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T16:09:57.000790+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:09:57.000790+0000","last_peered":"2024-09-10T16:09:57.000790+0000","last_clean":"2024-09-10T16:09:57.000790+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:09:57.000790+0000","last_undegraded":"2024-09-10T16:09:57.000790+0000","last_fullsized":"2024-09-10T16:09:57.000790+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20781","reported_seq":24642,"reported_epoch":55,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:10:06.172271+0000","last_change":"2024-09-10T16:09:55.745709+0000","last_active":"2024-09-10T16:10:06.172271+0000","last_peered":"2024-09-10T16:10:06.172271+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:10:06.172271+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2454,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18461,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1853,"num_bytes_recovered":8675,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2453}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":103,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T16:09:57.004400+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:09:57.004400+0000","last_peered":"2024-09-10T16:09:57.004400+0000","last_clean":"2024-09-10T16:09:57.004400+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:09:57.004400+0000","last_undegraded":"2024-09-10T16:09:57.004400+0000","last_fullsized":"2024-09-10T16:09:57.004400+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'21170","reported_seq":30037,"reported_epoch":55,"state":"active+recovery_wait","last_fresh":"2024-09-10T16:09:57.004800+0000","last_change":"2024-09-10T16:08:10.929169+0000","last_active":"2024-09-10T16:09:57.004800+0000","last_peered":"2024-09-10T16:09:57.004800+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T16:08:10.860438+0000","last_became_peered":"2024-09-10T16:08:10.860438+0000","last_unstale":"2024-09-10T16:09:57.004800+0000","last_undegraded":"2024-09-10T16:09:57.004800+0000","last_fullsized":"2024-09-10T16:09:57.004800+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18724,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","0","1"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20838","reported_seq":20996,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:57.000718+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:09:57.000718+0000","last_peered":"2024-09-10T16:09:57.000718+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:09:57.000718+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17469,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T16:09:57.004632+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:09:57.004632+0000","last_peered":"2024-09-10T16:09:57.004632+0000","last_clean":"2024-09-10T16:09:57.004632+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:09:57.004632+0000","last_undegraded":"2024-09-10T16:09:57.004632+0000","last_fullsized":"2024-09-10T16:09:57.004632+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20598","reported_seq":29187,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T16:09:57.005160+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:09:57.005160+0000","last_peered":"2024-09-10T16:09:57.005160+0000","last_clean":"2024-09-10T16:09:57.005160+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:09:57.005160+0000","last_undegraded":"2024-09-10T16:09:57.005160+0000","last_fullsized":"2024-09-10T16:09:57.005160+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16988,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":117,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T16:09:57.005454+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:09:57.005454+0000","last_peered":"2024-09-10T16:09:57.005454+0000","last_clean":"2024-09-10T16:09:57.005454+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:09:57.005454+0000","last_undegraded":"2024-09-10T16:09:57.005454+0000","last_fullsized":"2024-09-10T16:09:57.005454+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'21048","reported_seq":21213,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:57.002350+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:09:57.002350+0000","last_peered":"2024-09-10T16:09:57.002350+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:09:57.002350+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17303,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T16:09:57.005070+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:09:57.005070+0000","last_peered":"2024-09-10T16:09:57.005070+0000","last_clean":"2024-09-10T16:09:57.005070+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:09:57.005070+0000","last_undegraded":"2024-09-10T16:09:57.005070+0000","last_fullsized":"2024-09-10T16:09:57.005070+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":97,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T16:09:57.005068+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:09:57.005068+0000","last_peered":"2024-09-10T16:09:57.005068+0000","last_clean":"2024-09-10T16:09:57.005068+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:09:57.005068+0000","last_undegraded":"2024-09-10T16:09:57.005068+0000","last_fullsized":"2024-09-10T16:09:57.005068+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'21198","reported_seq":21366,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:57.001544+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:09:57.001544+0000","last_peered":"2024-09-10T16:09:57.001544+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:09:57.001544+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17361,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4321},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":24066,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292096,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27324,"num_bytes_recovered":33096646,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":210513920,"data_stored":210394691,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161022,"ondisk_log_size":161022,"up":32,"acting":26,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986321,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":502648,"kb_used_data":137760,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93869192,"statfs":{"total":96636764160,"available":96122052608,"internally_reserved":0,"allocated":141066240,"data_stored":140725324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659266,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":473672,"kb_used_data":504,"kb_used_omap":4,"kb_used_meta":473147,"kb_avail":93898168,"statfs":{"total":96636764160,"available":96151724032,"internally_reserved":0,"allocated":516096,"data_stored":160264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":484503214},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182549,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":547348,"kb_used_data":68668,"kb_used_omap":4,"kb_used_meta":478651,"kb_avail":93824492,"statfs":{"total":96636764160,"available":96076279808,"internally_reserved":0,"allocated":70316032,"data_stored":69971039,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":490139306},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":69853184,"data_stored":69817054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":6279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":140607488,"data_stored":140571358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:10:10.423 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-10T16:10:10.423 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "49'21061", 'reported_seq': 21222, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:09:57.000888+0000', 'last_change': '2024-09-10T15:59:16.308264+0000', 'last_active': '2024-09-10T16:09:57.000888+0000', 'last_peered': '2024-09-10T16:09:57.000888+0000', 'last_clean': '2024-09-10T15:59:14.111551+0000', 'last_became_active': '2024-09-10T15:59:16.300040+0000', 'last_became_peered': '2024-09-10T15:59:16.300040+0000', 'last_unstale': '2024-09-10T16:09:57.000888+0000', 'last_undegraded': '2024-09-10T15:59:15.878234+0000', 'last_fullsized': '2024-09-10T15:59:15.769328+0000', 'mapping_epoch': 42, 'log_start': "45'11000", 'ondisk_log_start': "45'11000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3442", 'last_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'objects_scrubbed': 3384, 'log_size': 10061, 'log_dups_size': 0, 'ondisk_log_size': 10061, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1292, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4372, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 17673, 'num_write_kb': 28580, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4370}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:10:10.423 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-10T16:10:10.423 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "49'21082", 'reported_seq': 21243, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:09:57.000962+0000', 'last_change': '2024-09-10T15:59:16.331899+0000', 'last_active': '2024-09-10T16:09:57.000962+0000', 'last_peered': '2024-09-10T16:09:57.000962+0000', 'last_clean': '2024-09-10T15:59:14.139436+0000', 'last_became_active': '2024-09-10T15:59:16.318715+0000', 'last_became_peered': '2024-09-10T15:59:16.318715+0000', 'last_unstale': '2024-09-10T16:09:57.000962+0000', 'last_undegraded': '2024-09-10T15:59:15.877524+0000', 'last_fullsized': '2024-09-10T15:59:15.769042+0000', 'mapping_epoch': 42, 'log_start': "45'11000", 'ondisk_log_start': "45'11000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3537", 'last_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'objects_scrubbed': 3497, 'log_size': 10082, 'log_dups_size': 0, 'ondisk_log_size': 10082, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1224, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4237, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 17581, 'num_write_kb': 28172, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4235}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:10:10.423 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-10T16:10:10.423 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "49'20781", 'reported_seq': 24642, 'reported_epoch': 55, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:10:06.172271+0000', 'last_change': '2024-09-10T16:09:55.745709+0000', 'last_active': '2024-09-10T16:10:06.172271+0000', 'last_peered': '2024-09-10T16:10:06.172271+0000', 'last_clean': '2024-09-10T15:59:14.116041+0000', 'last_became_active': '2024-09-10T15:59:16.274543+0000', 'last_became_peered': '2024-09-10T15:59:16.274543+0000', 'last_unstale': '2024-09-10T16:10:06.172271+0000', 'last_undegraded': '2024-09-10T15:59:15.812484+0000', 'last_fullsized': '2024-09-10T15:59:15.770885+0000', 'mapping_epoch': 42, 'log_start': "45'10700", 'ondisk_log_start': "45'10700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2356", 'last_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'objects_scrubbed': 2316, 'log_size': 10081, 'log_dups_size': 0, 'ondisk_log_size': 10081, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 783, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2454, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 18461, 'num_write_kb': 32292, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1853, 'num_bytes_recovered': 8675, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -2453}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:10:10.424 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-10T16:10:10.424 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "49'21170", 'reported_seq': 30037, 'reported_epoch': 55, 'state': 'active+recovery_wait', 'last_fresh': '2024-09-10T16:09:57.004800+0000', 'last_change': '2024-09-10T16:08:10.929169+0000', 'last_active': '2024-09-10T16:09:57.004800+0000', 'last_peered': '2024-09-10T16:09:57.004800+0000', 'last_clean': '2024-09-10T15:59:14.110654+0000', 'last_became_active': '2024-09-10T16:08:10.860438+0000', 'last_became_peered': '2024-09-10T16:08:10.860438+0000', 'last_unstale': '2024-09-10T16:09:57.004800+0000', 'last_undegraded': '2024-09-10T16:09:57.004800+0000', 'last_fullsized': '2024-09-10T16:09:57.004800+0000', 'mapping_epoch': 48, 'log_start': "45'11100", 'ondisk_log_start': "45'11100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2486", 'last_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'objects_scrubbed': 2440, 'log_size': 10070, 'log_dups_size': 0, 'ondisk_log_size': 10070, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 792, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18724, 'num_write_kb': 32568, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 4355, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': ['2', '0', '1'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:10:10.424 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-10T16:10:10.424 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "49'20838", 'reported_seq': 20996, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:09:57.000718+0000', 'last_change': '2024-09-10T15:59:16.315641+0000', 'last_active': '2024-09-10T16:09:57.000718+0000', 'last_peered': '2024-09-10T16:09:57.000718+0000', 'last_clean': '2024-09-10T15:59:14.117001+0000', 'last_became_active': '2024-09-10T15:59:16.302245+0000', 'last_became_peered': '2024-09-10T15:59:16.302245+0000', 'last_unstale': '2024-09-10T16:09:57.000718+0000', 'last_undegraded': '2024-09-10T15:59:15.845012+0000', 'last_fullsized': '2024-09-10T15:59:15.770337+0000', 'mapping_epoch': 42, 'log_start': "45'10800", 'ondisk_log_start': "45'10800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3404", 'last_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'objects_scrubbed': 3367, 'log_size': 10038, 'log_dups_size': 0, 'ondisk_log_size': 10038, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 983, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17469, 'num_write_kb': 28204, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4304}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:10:10.424 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-10T16:10:10.424 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "49'21048", 'reported_seq': 21213, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:09:57.002350+0000', 'last_change': '2024-09-10T15:59:16.311291+0000', 'last_active': '2024-09-10T16:09:57.002350+0000', 'last_peered': '2024-09-10T16:09:57.002350+0000', 'last_clean': '2024-09-10T15:59:14.116363+0000', 'last_became_active': '2024-09-10T15:59:16.301244+0000', 'last_became_peered': '2024-09-10T15:59:16.301244+0000', 'last_unstale': '2024-09-10T16:09:57.002350+0000', 'last_undegraded': '2024-09-10T15:59:15.883434+0000', 'last_fullsized': '2024-09-10T15:59:15.772756+0000', 'mapping_epoch': 42, 'log_start': "45'11000", 'ondisk_log_start': "45'11000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3789", 'last_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'objects_scrubbed': 3741, 'log_size': 10048, 'log_dups_size': 0, 'ondisk_log_size': 10048, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1440, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4373, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17303, 'num_write_kb': 27124, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4371}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:10:10.424 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-10T16:10:10.424 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "49'21198", 'reported_seq': 21366, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:09:57.001544+0000', 'last_change': '2024-09-10T15:59:16.313409+0000', 'last_active': '2024-09-10T16:09:57.001544+0000', 'last_peered': '2024-09-10T16:09:57.001544+0000', 'last_clean': '2024-09-10T15:59:14.129780+0000', 'last_became_active': '2024-09-10T15:59:16.300932+0000', 'last_became_peered': '2024-09-10T15:59:16.300932+0000', 'last_unstale': '2024-09-10T16:09:57.001544+0000', 'last_undegraded': '2024-09-10T15:59:15.887823+0000', 'last_fullsized': '2024-09-10T15:59:15.768513+0000', 'mapping_epoch': 42, 'log_start': "45'11100", 'ondisk_log_start': "45'11100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3880", 'last_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'objects_scrubbed': 3827, 'log_size': 10098, 'log_dups_size': 0, 'ondisk_log_size': 10098, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1397, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4325, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17361, 'num_write_kb': 27072, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4321}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:10:10.425 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T16:10:10.797 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:10:10.797 INFO:teuthology.orchestra.run.smithi115.stdout:{"fsid":"7effee29-df05-4351-9741-ee203564ed81","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 24066/24 objects degraded (100275.000%), 6 pgs degraded, 6 pgs undersized","count":12},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":774,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":55,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983954,"num_in_osds":3,"osd_in_since":1725983947,"num_remapped_pgs":6},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":17},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":5},{"state_name":"active+recovery_wait","count":1},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":12,"data_bytes":4849,"bytes_used":1560236032,"bytes_avail":288350056448,"bytes_total":289910292480,"degraded_objects":24066,"degraded_total":24,"degraded_ratio":1002.75,"recovering_objects_per_sec":150,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1808,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-10T15:57:14:967120+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T16:00:13.560776+0000","services":{}},"progress_events":{}} 2024-09-10T16:10:10.810 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-10T16:10:10.810 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:10:11.130 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:10:11.130 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:10:11.143 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":435,"stamp":"2024-09-10T16:10:09.780712+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":24066,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292098,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27326,"num_bytes_recovered":33096665,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161025,"ondisk_log_size":161025,"up":48,"acting":42,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1523668,"kb_used_data":206932,"kb_used_omap":12,"kb_used_meta":1316659,"kb_avail":281591852,"statfs":{"total":289910292480,"available":288350056448,"internally_reserved":0,"allocated":211898368,"data_stored":210856627,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1348258818},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1808,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1808,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001978"},"pg_stats":[{"pgid":"2.f","version":"49'20672","reported_seq":29296,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T16:09:57.088647+0000","last_change":"2024-09-10T16:09:57.088531+0000","last_active":"2024-09-10T16:09:57.088647+0000","last_peered":"2024-09-10T16:09:57.088647+0000","last_clean":"2024-09-10T16:09:57.088647+0000","last_became_active":"2024-09-10T16:09:57.005755+0000","last_became_peered":"2024-09-10T16:09:57.005755+0000","last_unstale":"2024-09-10T16:09:57.088647+0000","last_undegraded":"2024-09-10T16:09:57.088647+0000","last_fullsized":"2024-09-10T16:09:57.088647+0000","mapping_epoch":54,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:09:19.329697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20672,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4259,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20844","reported_seq":20980,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T16:09:57.005461+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:09:57.005461+0000","last_peered":"2024-09-10T16:09:57.005461+0000","last_clean":"2024-09-10T16:09:57.005461+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:09:57.005461+0000","last_undegraded":"2024-09-10T16:09:57.005461+0000","last_fullsized":"2024-09-10T16:09:57.005461+0000","mapping_epoch":35,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18949,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20798","reported_seq":29499,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T16:09:57.004925+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:09:57.004925+0000","last_peered":"2024-09-10T16:09:57.004925+0000","last_clean":"2024-09-10T16:09:57.004925+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:09:57.004925+0000","last_undegraded":"2024-09-10T16:09:57.004925+0000","last_fullsized":"2024-09-10T16:09:57.004925+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18802,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20660","reported_seq":20764,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T16:09:57.005318+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:09:57.005318+0000","last_peered":"2024-09-10T16:09:57.005318+0000","last_clean":"2024-09-10T16:09:57.005318+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:09:57.005318+0000","last_undegraded":"2024-09-10T16:09:57.005318+0000","last_fullsized":"2024-09-10T16:09:57.005318+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20660,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20891","reported_seq":21044,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T16:09:57.001440+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:09:57.001440+0000","last_peered":"2024-09-10T16:09:57.001440+0000","last_clean":"2024-09-10T16:09:57.001440+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:09:57.001440+0000","last_undegraded":"2024-09-10T16:09:57.001440+0000","last_fullsized":"2024-09-10T16:09:57.001440+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17602,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'21061","reported_seq":21222,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:57.000888+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:09:57.000888+0000","last_peered":"2024-09-10T16:09:57.000888+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:09:57.000888+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17673,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4370},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'21082","reported_seq":21243,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:57.000962+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:09:57.000962+0000","last_peered":"2024-09-10T16:09:57.000962+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:09:57.000962+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17581,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4235},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'21314","reported_seq":21467,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T16:09:57.000718+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:09:57.000718+0000","last_peered":"2024-09-10T16:09:57.000718+0000","last_clean":"2024-09-10T16:09:57.000718+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:09:57.000718+0000","last_undegraded":"2024-09-10T16:09:57.000718+0000","last_fullsized":"2024-09-10T16:09:57.000718+0000","mapping_epoch":34,"log_start":"45'11300","ondisk_log_start":"45'11300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18161,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T16:09:57.003703+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:09:57.003703+0000","last_peered":"2024-09-10T16:09:57.003703+0000","last_clean":"2024-09-10T16:09:57.003703+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:09:57.003703+0000","last_undegraded":"2024-09-10T16:09:57.003703+0000","last_fullsized":"2024-09-10T16:09:57.003703+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20457","reported_seq":20624,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T16:09:57.004545+0000","last_change":"2024-09-10T16:09:33.012840+0000","last_active":"2024-09-10T16:09:57.004545+0000","last_peered":"2024-09-10T16:09:57.004545+0000","last_clean":"2024-09-10T16:09:57.004545+0000","last_became_active":"2024-09-10T16:09:32.934056+0000","last_became_peered":"2024-09-10T16:09:32.934056+0000","last_unstale":"2024-09-10T16:09:57.004545+0000","last_undegraded":"2024-09-10T16:09:57.004545+0000","last_fullsized":"2024-09-10T16:09:57.004545+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:08:30.481235+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18276,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":4579,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":117,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T16:09:57.004914+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:09:57.004914+0000","last_peered":"2024-09-10T16:09:57.004914+0000","last_clean":"2024-09-10T16:09:57.004914+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:09:57.004914+0000","last_undegraded":"2024-09-10T16:09:57.004914+0000","last_fullsized":"2024-09-10T16:09:57.004914+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20910","reported_seq":29586,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T16:09:57.001273+0000","last_change":"2024-09-10T16:09:08.909580+0000","last_active":"2024-09-10T16:09:57.001273+0000","last_peered":"2024-09-10T16:09:57.001273+0000","last_clean":"2024-09-10T16:09:57.001273+0000","last_became_active":"2024-09-10T16:09:08.854941+0000","last_became_peered":"2024-09-10T16:09:08.854941+0000","last_unstale":"2024-09-10T16:09:57.001273+0000","last_undegraded":"2024-09-10T16:09:57.001273+0000","last_fullsized":"2024-09-10T16:09:57.001273+0000","mapping_epoch":50,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:13:14.846691+0000","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17414,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4257,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":127,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T16:09:57.000790+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:09:57.000790+0000","last_peered":"2024-09-10T16:09:57.000790+0000","last_clean":"2024-09-10T16:09:57.000790+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:09:57.000790+0000","last_undegraded":"2024-09-10T16:09:57.000790+0000","last_fullsized":"2024-09-10T16:09:57.000790+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20781","reported_seq":24642,"reported_epoch":55,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:10:06.172271+0000","last_change":"2024-09-10T16:09:55.745709+0000","last_active":"2024-09-10T16:10:06.172271+0000","last_peered":"2024-09-10T16:10:06.172271+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:10:06.172271+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2454,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18461,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1853,"num_bytes_recovered":8675,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2453}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":103,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T16:09:57.004400+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:09:57.004400+0000","last_peered":"2024-09-10T16:09:57.004400+0000","last_clean":"2024-09-10T16:09:57.004400+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:09:57.004400+0000","last_undegraded":"2024-09-10T16:09:57.004400+0000","last_fullsized":"2024-09-10T16:09:57.004400+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'21170","reported_seq":30037,"reported_epoch":55,"state":"active+recovery_wait","last_fresh":"2024-09-10T16:09:57.004800+0000","last_change":"2024-09-10T16:08:10.929169+0000","last_active":"2024-09-10T16:09:57.004800+0000","last_peered":"2024-09-10T16:09:57.004800+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T16:08:10.860438+0000","last_became_peered":"2024-09-10T16:08:10.860438+0000","last_unstale":"2024-09-10T16:09:57.004800+0000","last_undegraded":"2024-09-10T16:09:57.004800+0000","last_fullsized":"2024-09-10T16:09:57.004800+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18724,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","0","1"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20838","reported_seq":20996,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:57.000718+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:09:57.000718+0000","last_peered":"2024-09-10T16:09:57.000718+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:09:57.000718+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17469,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T16:09:57.004632+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:09:57.004632+0000","last_peered":"2024-09-10T16:09:57.004632+0000","last_clean":"2024-09-10T16:09:57.004632+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:09:57.004632+0000","last_undegraded":"2024-09-10T16:09:57.004632+0000","last_fullsized":"2024-09-10T16:09:57.004632+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20598","reported_seq":29187,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T16:09:57.005160+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:09:57.005160+0000","last_peered":"2024-09-10T16:09:57.005160+0000","last_clean":"2024-09-10T16:09:57.005160+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:09:57.005160+0000","last_undegraded":"2024-09-10T16:09:57.005160+0000","last_fullsized":"2024-09-10T16:09:57.005160+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16988,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":117,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T16:09:57.005454+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:09:57.005454+0000","last_peered":"2024-09-10T16:09:57.005454+0000","last_clean":"2024-09-10T16:09:57.005454+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:09:57.005454+0000","last_undegraded":"2024-09-10T16:09:57.005454+0000","last_fullsized":"2024-09-10T16:09:57.005454+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'21048","reported_seq":21213,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:57.002350+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:09:57.002350+0000","last_peered":"2024-09-10T16:09:57.002350+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:09:57.002350+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17303,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T16:09:57.005070+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:09:57.005070+0000","last_peered":"2024-09-10T16:09:57.005070+0000","last_clean":"2024-09-10T16:09:57.005070+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:09:57.005070+0000","last_undegraded":"2024-09-10T16:09:57.005070+0000","last_fullsized":"2024-09-10T16:09:57.005070+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":97,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T16:09:57.005068+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:09:57.005068+0000","last_peered":"2024-09-10T16:09:57.005068+0000","last_clean":"2024-09-10T16:09:57.005068+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:09:57.005068+0000","last_undegraded":"2024-09-10T16:09:57.005068+0000","last_fullsized":"2024-09-10T16:09:57.005068+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'21198","reported_seq":21366,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:57.001544+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:09:57.001544+0000","last_peered":"2024-09-10T16:09:57.001544+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:09:57.001544+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17361,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4321},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":24066,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292096,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27324,"num_bytes_recovered":33096646,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":210513920,"data_stored":210394691,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161022,"ondisk_log_size":161022,"up":32,"acting":26,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986321,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":502648,"kb_used_data":137760,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93869192,"statfs":{"total":96636764160,"available":96122052608,"internally_reserved":0,"allocated":141066240,"data_stored":140725324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659266,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":473672,"kb_used_data":504,"kb_used_omap":4,"kb_used_meta":473147,"kb_avail":93898168,"statfs":{"total":96636764160,"available":96151724032,"internally_reserved":0,"allocated":516096,"data_stored":160264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":484503214},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182549,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":547348,"kb_used_data":68668,"kb_used_omap":4,"kb_used_meta":478651,"kb_avail":93824492,"statfs":{"total":96636764160,"available":96076279808,"internally_reserved":0,"allocated":70316032,"data_stored":69971039,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":490139306},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":69853184,"data_stored":69817054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":6279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":140607488,"data_stored":140571358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:10:14.145 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:10:14.458 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:10:14.459 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:10:14.471 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":437,"stamp":"2024-09-10T16:10:13.781367+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":23329,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292098,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28063,"num_bytes_recovered":33096665,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161025,"ondisk_log_size":161025,"up":48,"acting":42,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1511188,"kb_used_data":206932,"kb_used_omap":12,"kb_used_meta":1304179,"kb_avail":281604332,"statfs":{"total":289910292480,"available":288362835968,"internally_reserved":0,"allocated":211898368,"data_stored":210856627,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1335479298},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1686,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1686,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001982"},"pg_stats":[{"pgid":"2.f","version":"49'20672","reported_seq":29296,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T16:09:57.088647+0000","last_change":"2024-09-10T16:09:57.088531+0000","last_active":"2024-09-10T16:09:57.088647+0000","last_peered":"2024-09-10T16:09:57.088647+0000","last_clean":"2024-09-10T16:09:57.088647+0000","last_became_active":"2024-09-10T16:09:57.005755+0000","last_became_peered":"2024-09-10T16:09:57.005755+0000","last_unstale":"2024-09-10T16:09:57.088647+0000","last_undegraded":"2024-09-10T16:09:57.088647+0000","last_fullsized":"2024-09-10T16:09:57.088647+0000","mapping_epoch":54,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:09:19.329697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20672,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4259,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20844","reported_seq":20980,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T16:09:57.005461+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:09:57.005461+0000","last_peered":"2024-09-10T16:09:57.005461+0000","last_clean":"2024-09-10T16:09:57.005461+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:09:57.005461+0000","last_undegraded":"2024-09-10T16:09:57.005461+0000","last_fullsized":"2024-09-10T16:09:57.005461+0000","mapping_epoch":35,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18949,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20798","reported_seq":29499,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T16:09:57.004925+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:09:57.004925+0000","last_peered":"2024-09-10T16:09:57.004925+0000","last_clean":"2024-09-10T16:09:57.004925+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:09:57.004925+0000","last_undegraded":"2024-09-10T16:09:57.004925+0000","last_fullsized":"2024-09-10T16:09:57.004925+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18802,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20660","reported_seq":20764,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T16:09:57.005318+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:09:57.005318+0000","last_peered":"2024-09-10T16:09:57.005318+0000","last_clean":"2024-09-10T16:09:57.005318+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:09:57.005318+0000","last_undegraded":"2024-09-10T16:09:57.005318+0000","last_fullsized":"2024-09-10T16:09:57.005318+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20660,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20891","reported_seq":21044,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T16:09:57.001440+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:09:57.001440+0000","last_peered":"2024-09-10T16:09:57.001440+0000","last_clean":"2024-09-10T16:09:57.001440+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:09:57.001440+0000","last_undegraded":"2024-09-10T16:09:57.001440+0000","last_fullsized":"2024-09-10T16:09:57.001440+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17602,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'21061","reported_seq":21222,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:57.000888+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:09:57.000888+0000","last_peered":"2024-09-10T16:09:57.000888+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:09:57.000888+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17673,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4370},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'21082","reported_seq":21243,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:57.000962+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:09:57.000962+0000","last_peered":"2024-09-10T16:09:57.000962+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:09:57.000962+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17581,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4235},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'21314","reported_seq":21467,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T16:09:57.000718+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:09:57.000718+0000","last_peered":"2024-09-10T16:09:57.000718+0000","last_clean":"2024-09-10T16:09:57.000718+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:09:57.000718+0000","last_undegraded":"2024-09-10T16:09:57.000718+0000","last_fullsized":"2024-09-10T16:09:57.000718+0000","mapping_epoch":34,"log_start":"45'11300","ondisk_log_start":"45'11300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18161,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T16:09:57.003703+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:09:57.003703+0000","last_peered":"2024-09-10T16:09:57.003703+0000","last_clean":"2024-09-10T16:09:57.003703+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:09:57.003703+0000","last_undegraded":"2024-09-10T16:09:57.003703+0000","last_fullsized":"2024-09-10T16:09:57.003703+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20457","reported_seq":20624,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T16:09:57.004545+0000","last_change":"2024-09-10T16:09:33.012840+0000","last_active":"2024-09-10T16:09:57.004545+0000","last_peered":"2024-09-10T16:09:57.004545+0000","last_clean":"2024-09-10T16:09:57.004545+0000","last_became_active":"2024-09-10T16:09:32.934056+0000","last_became_peered":"2024-09-10T16:09:32.934056+0000","last_unstale":"2024-09-10T16:09:57.004545+0000","last_undegraded":"2024-09-10T16:09:57.004545+0000","last_fullsized":"2024-09-10T16:09:57.004545+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:08:30.481235+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18276,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":4579,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":117,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T16:09:57.004914+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:09:57.004914+0000","last_peered":"2024-09-10T16:09:57.004914+0000","last_clean":"2024-09-10T16:09:57.004914+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:09:57.004914+0000","last_undegraded":"2024-09-10T16:09:57.004914+0000","last_fullsized":"2024-09-10T16:09:57.004914+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20910","reported_seq":29586,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T16:09:57.001273+0000","last_change":"2024-09-10T16:09:08.909580+0000","last_active":"2024-09-10T16:09:57.001273+0000","last_peered":"2024-09-10T16:09:57.001273+0000","last_clean":"2024-09-10T16:09:57.001273+0000","last_became_active":"2024-09-10T16:09:08.854941+0000","last_became_peered":"2024-09-10T16:09:08.854941+0000","last_unstale":"2024-09-10T16:09:57.001273+0000","last_undegraded":"2024-09-10T16:09:57.001273+0000","last_fullsized":"2024-09-10T16:09:57.001273+0000","mapping_epoch":50,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:13:14.846691+0000","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17414,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4257,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":127,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T16:09:57.000790+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:09:57.000790+0000","last_peered":"2024-09-10T16:09:57.000790+0000","last_clean":"2024-09-10T16:09:57.000790+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:09:57.000790+0000","last_undegraded":"2024-09-10T16:09:57.000790+0000","last_fullsized":"2024-09-10T16:09:57.000790+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20781","reported_seq":26116,"reported_epoch":55,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:10:11.172519+0000","last_change":"2024-09-10T16:09:55.745709+0000","last_active":"2024-09-10T16:10:11.172519+0000","last_peered":"2024-09-10T16:10:11.172519+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:10:11.172519+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1717,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18461,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2590,"num_bytes_recovered":8675,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1716}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":103,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T16:09:57.004400+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:09:57.004400+0000","last_peered":"2024-09-10T16:09:57.004400+0000","last_clean":"2024-09-10T16:09:57.004400+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:09:57.004400+0000","last_undegraded":"2024-09-10T16:09:57.004400+0000","last_fullsized":"2024-09-10T16:09:57.004400+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'21170","reported_seq":30037,"reported_epoch":55,"state":"active+recovery_wait","last_fresh":"2024-09-10T16:09:57.004800+0000","last_change":"2024-09-10T16:08:10.929169+0000","last_active":"2024-09-10T16:09:57.004800+0000","last_peered":"2024-09-10T16:09:57.004800+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T16:08:10.860438+0000","last_became_peered":"2024-09-10T16:08:10.860438+0000","last_unstale":"2024-09-10T16:09:57.004800+0000","last_undegraded":"2024-09-10T16:09:57.004800+0000","last_fullsized":"2024-09-10T16:09:57.004800+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18724,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","0","1"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20838","reported_seq":20996,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:57.000718+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:09:57.000718+0000","last_peered":"2024-09-10T16:09:57.000718+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:09:57.000718+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17469,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T16:09:57.004632+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:09:57.004632+0000","last_peered":"2024-09-10T16:09:57.004632+0000","last_clean":"2024-09-10T16:09:57.004632+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:09:57.004632+0000","last_undegraded":"2024-09-10T16:09:57.004632+0000","last_fullsized":"2024-09-10T16:09:57.004632+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20598","reported_seq":29187,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T16:09:57.005160+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:09:57.005160+0000","last_peered":"2024-09-10T16:09:57.005160+0000","last_clean":"2024-09-10T16:09:57.005160+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:09:57.005160+0000","last_undegraded":"2024-09-10T16:09:57.005160+0000","last_fullsized":"2024-09-10T16:09:57.005160+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16988,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":117,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T16:09:57.005454+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:09:57.005454+0000","last_peered":"2024-09-10T16:09:57.005454+0000","last_clean":"2024-09-10T16:09:57.005454+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:09:57.005454+0000","last_undegraded":"2024-09-10T16:09:57.005454+0000","last_fullsized":"2024-09-10T16:09:57.005454+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'21048","reported_seq":21213,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:57.002350+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:09:57.002350+0000","last_peered":"2024-09-10T16:09:57.002350+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:09:57.002350+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17303,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T16:09:57.005070+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:09:57.005070+0000","last_peered":"2024-09-10T16:09:57.005070+0000","last_clean":"2024-09-10T16:09:57.005070+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:09:57.005070+0000","last_undegraded":"2024-09-10T16:09:57.005070+0000","last_fullsized":"2024-09-10T16:09:57.005070+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":98,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T16:09:57.005104+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:09:57.005104+0000","last_peered":"2024-09-10T16:09:57.005104+0000","last_clean":"2024-09-10T16:09:57.005104+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:09:57.005104+0000","last_undegraded":"2024-09-10T16:09:57.005104+0000","last_fullsized":"2024-09-10T16:09:57.005104+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'21198","reported_seq":21366,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:57.001544+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:09:57.001544+0000","last_peered":"2024-09-10T16:09:57.001544+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:09:57.001544+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17361,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4321},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":23329,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292096,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28061,"num_bytes_recovered":33096646,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":210513920,"data_stored":210394691,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161022,"ondisk_log_size":161022,"up":32,"acting":26,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986322,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":502648,"kb_used_data":137760,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93869192,"statfs":{"total":96636764160,"available":96122052608,"internally_reserved":0,"allocated":141066240,"data_stored":140725324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659267,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":461192,"kb_used_data":504,"kb_used_omap":4,"kb_used_meta":460667,"kb_avail":93910648,"statfs":{"total":96636764160,"available":96164503552,"internally_reserved":0,"allocated":516096,"data_stored":160264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":471723694},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182550,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":547348,"kb_used_data":68668,"kb_used_omap":4,"kb_used_meta":478651,"kb_avail":93824492,"statfs":{"total":96636764160,"available":96076279808,"internally_reserved":0,"allocated":70316032,"data_stored":69971039,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":490139306},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":69853184,"data_stored":69817054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":6279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":140607488,"data_stored":140571358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:10:14.473 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:10:14.786 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:10:14.786 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:10:14.799 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":437,"stamp":"2024-09-10T16:10:13.781367+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":23329,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292098,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28063,"num_bytes_recovered":33096665,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161025,"ondisk_log_size":161025,"up":48,"acting":42,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1511188,"kb_used_data":206932,"kb_used_omap":12,"kb_used_meta":1304179,"kb_avail":281604332,"statfs":{"total":289910292480,"available":288362835968,"internally_reserved":0,"allocated":211898368,"data_stored":210856627,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1335479298},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1686,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1686,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001982"},"pg_stats":[{"pgid":"2.f","version":"49'20672","reported_seq":29296,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T16:09:57.088647+0000","last_change":"2024-09-10T16:09:57.088531+0000","last_active":"2024-09-10T16:09:57.088647+0000","last_peered":"2024-09-10T16:09:57.088647+0000","last_clean":"2024-09-10T16:09:57.088647+0000","last_became_active":"2024-09-10T16:09:57.005755+0000","last_became_peered":"2024-09-10T16:09:57.005755+0000","last_unstale":"2024-09-10T16:09:57.088647+0000","last_undegraded":"2024-09-10T16:09:57.088647+0000","last_fullsized":"2024-09-10T16:09:57.088647+0000","mapping_epoch":54,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:09:19.329697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20672,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4259,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20844","reported_seq":20980,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T16:09:57.005461+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:09:57.005461+0000","last_peered":"2024-09-10T16:09:57.005461+0000","last_clean":"2024-09-10T16:09:57.005461+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:09:57.005461+0000","last_undegraded":"2024-09-10T16:09:57.005461+0000","last_fullsized":"2024-09-10T16:09:57.005461+0000","mapping_epoch":35,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18949,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20798","reported_seq":29499,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T16:09:57.004925+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:09:57.004925+0000","last_peered":"2024-09-10T16:09:57.004925+0000","last_clean":"2024-09-10T16:09:57.004925+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:09:57.004925+0000","last_undegraded":"2024-09-10T16:09:57.004925+0000","last_fullsized":"2024-09-10T16:09:57.004925+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18802,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20660","reported_seq":20764,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T16:09:57.005318+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:09:57.005318+0000","last_peered":"2024-09-10T16:09:57.005318+0000","last_clean":"2024-09-10T16:09:57.005318+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:09:57.005318+0000","last_undegraded":"2024-09-10T16:09:57.005318+0000","last_fullsized":"2024-09-10T16:09:57.005318+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20660,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20891","reported_seq":21044,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T16:09:57.001440+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:09:57.001440+0000","last_peered":"2024-09-10T16:09:57.001440+0000","last_clean":"2024-09-10T16:09:57.001440+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:09:57.001440+0000","last_undegraded":"2024-09-10T16:09:57.001440+0000","last_fullsized":"2024-09-10T16:09:57.001440+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17602,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'21061","reported_seq":21222,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:57.000888+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:09:57.000888+0000","last_peered":"2024-09-10T16:09:57.000888+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:09:57.000888+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17673,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4370},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'21082","reported_seq":21243,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:57.000962+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:09:57.000962+0000","last_peered":"2024-09-10T16:09:57.000962+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:09:57.000962+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17581,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4235},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'21314","reported_seq":21467,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T16:09:57.000718+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:09:57.000718+0000","last_peered":"2024-09-10T16:09:57.000718+0000","last_clean":"2024-09-10T16:09:57.000718+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:09:57.000718+0000","last_undegraded":"2024-09-10T16:09:57.000718+0000","last_fullsized":"2024-09-10T16:09:57.000718+0000","mapping_epoch":34,"log_start":"45'11300","ondisk_log_start":"45'11300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18161,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T16:09:57.003703+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:09:57.003703+0000","last_peered":"2024-09-10T16:09:57.003703+0000","last_clean":"2024-09-10T16:09:57.003703+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:09:57.003703+0000","last_undegraded":"2024-09-10T16:09:57.003703+0000","last_fullsized":"2024-09-10T16:09:57.003703+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20457","reported_seq":20624,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T16:09:57.004545+0000","last_change":"2024-09-10T16:09:33.012840+0000","last_active":"2024-09-10T16:09:57.004545+0000","last_peered":"2024-09-10T16:09:57.004545+0000","last_clean":"2024-09-10T16:09:57.004545+0000","last_became_active":"2024-09-10T16:09:32.934056+0000","last_became_peered":"2024-09-10T16:09:32.934056+0000","last_unstale":"2024-09-10T16:09:57.004545+0000","last_undegraded":"2024-09-10T16:09:57.004545+0000","last_fullsized":"2024-09-10T16:09:57.004545+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:08:30.481235+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18276,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":4579,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":117,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T16:09:57.004914+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:09:57.004914+0000","last_peered":"2024-09-10T16:09:57.004914+0000","last_clean":"2024-09-10T16:09:57.004914+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:09:57.004914+0000","last_undegraded":"2024-09-10T16:09:57.004914+0000","last_fullsized":"2024-09-10T16:09:57.004914+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20910","reported_seq":29586,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T16:09:57.001273+0000","last_change":"2024-09-10T16:09:08.909580+0000","last_active":"2024-09-10T16:09:57.001273+0000","last_peered":"2024-09-10T16:09:57.001273+0000","last_clean":"2024-09-10T16:09:57.001273+0000","last_became_active":"2024-09-10T16:09:08.854941+0000","last_became_peered":"2024-09-10T16:09:08.854941+0000","last_unstale":"2024-09-10T16:09:57.001273+0000","last_undegraded":"2024-09-10T16:09:57.001273+0000","last_fullsized":"2024-09-10T16:09:57.001273+0000","mapping_epoch":50,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:13:14.846691+0000","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17414,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4257,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":127,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T16:09:57.000790+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:09:57.000790+0000","last_peered":"2024-09-10T16:09:57.000790+0000","last_clean":"2024-09-10T16:09:57.000790+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:09:57.000790+0000","last_undegraded":"2024-09-10T16:09:57.000790+0000","last_fullsized":"2024-09-10T16:09:57.000790+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20781","reported_seq":26116,"reported_epoch":55,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:10:11.172519+0000","last_change":"2024-09-10T16:09:55.745709+0000","last_active":"2024-09-10T16:10:11.172519+0000","last_peered":"2024-09-10T16:10:11.172519+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:10:11.172519+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1717,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18461,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2590,"num_bytes_recovered":8675,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1716}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":103,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T16:09:57.004400+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:09:57.004400+0000","last_peered":"2024-09-10T16:09:57.004400+0000","last_clean":"2024-09-10T16:09:57.004400+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:09:57.004400+0000","last_undegraded":"2024-09-10T16:09:57.004400+0000","last_fullsized":"2024-09-10T16:09:57.004400+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'21170","reported_seq":30037,"reported_epoch":55,"state":"active+recovery_wait","last_fresh":"2024-09-10T16:09:57.004800+0000","last_change":"2024-09-10T16:08:10.929169+0000","last_active":"2024-09-10T16:09:57.004800+0000","last_peered":"2024-09-10T16:09:57.004800+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T16:08:10.860438+0000","last_became_peered":"2024-09-10T16:08:10.860438+0000","last_unstale":"2024-09-10T16:09:57.004800+0000","last_undegraded":"2024-09-10T16:09:57.004800+0000","last_fullsized":"2024-09-10T16:09:57.004800+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18724,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","0","1"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20838","reported_seq":20996,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:57.000718+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:09:57.000718+0000","last_peered":"2024-09-10T16:09:57.000718+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:09:57.000718+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17469,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T16:09:57.004632+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:09:57.004632+0000","last_peered":"2024-09-10T16:09:57.004632+0000","last_clean":"2024-09-10T16:09:57.004632+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:09:57.004632+0000","last_undegraded":"2024-09-10T16:09:57.004632+0000","last_fullsized":"2024-09-10T16:09:57.004632+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20598","reported_seq":29187,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T16:09:57.005160+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:09:57.005160+0000","last_peered":"2024-09-10T16:09:57.005160+0000","last_clean":"2024-09-10T16:09:57.005160+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:09:57.005160+0000","last_undegraded":"2024-09-10T16:09:57.005160+0000","last_fullsized":"2024-09-10T16:09:57.005160+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16988,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":117,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T16:09:57.005454+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:09:57.005454+0000","last_peered":"2024-09-10T16:09:57.005454+0000","last_clean":"2024-09-10T16:09:57.005454+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:09:57.005454+0000","last_undegraded":"2024-09-10T16:09:57.005454+0000","last_fullsized":"2024-09-10T16:09:57.005454+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'21048","reported_seq":21213,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:57.002350+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:09:57.002350+0000","last_peered":"2024-09-10T16:09:57.002350+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:09:57.002350+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17303,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T16:09:57.005070+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:09:57.005070+0000","last_peered":"2024-09-10T16:09:57.005070+0000","last_clean":"2024-09-10T16:09:57.005070+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:09:57.005070+0000","last_undegraded":"2024-09-10T16:09:57.005070+0000","last_fullsized":"2024-09-10T16:09:57.005070+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":98,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T16:09:57.005104+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:09:57.005104+0000","last_peered":"2024-09-10T16:09:57.005104+0000","last_clean":"2024-09-10T16:09:57.005104+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:09:57.005104+0000","last_undegraded":"2024-09-10T16:09:57.005104+0000","last_fullsized":"2024-09-10T16:09:57.005104+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'21198","reported_seq":21366,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:57.001544+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:09:57.001544+0000","last_peered":"2024-09-10T16:09:57.001544+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:09:57.001544+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17361,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4321},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":23329,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292096,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28061,"num_bytes_recovered":33096646,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":210513920,"data_stored":210394691,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161022,"ondisk_log_size":161022,"up":32,"acting":26,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986322,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":502648,"kb_used_data":137760,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93869192,"statfs":{"total":96636764160,"available":96122052608,"internally_reserved":0,"allocated":141066240,"data_stored":140725324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659267,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":461192,"kb_used_data":504,"kb_used_omap":4,"kb_used_meta":460667,"kb_avail":93910648,"statfs":{"total":96636764160,"available":96164503552,"internally_reserved":0,"allocated":516096,"data_stored":160264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":471723694},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182550,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":547348,"kb_used_data":68668,"kb_used_omap":4,"kb_used_meta":478651,"kb_avail":93824492,"statfs":{"total":96636764160,"available":96076279808,"internally_reserved":0,"allocated":70316032,"data_stored":69971039,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":490139306},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":69853184,"data_stored":69817054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":6279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":140607488,"data_stored":140571358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:10:14.800 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-10T16:10:14.800 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "49'21061", 'reported_seq': 21222, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:09:57.000888+0000', 'last_change': '2024-09-10T15:59:16.308264+0000', 'last_active': '2024-09-10T16:09:57.000888+0000', 'last_peered': '2024-09-10T16:09:57.000888+0000', 'last_clean': '2024-09-10T15:59:14.111551+0000', 'last_became_active': '2024-09-10T15:59:16.300040+0000', 'last_became_peered': '2024-09-10T15:59:16.300040+0000', 'last_unstale': '2024-09-10T16:09:57.000888+0000', 'last_undegraded': '2024-09-10T15:59:15.878234+0000', 'last_fullsized': '2024-09-10T15:59:15.769328+0000', 'mapping_epoch': 42, 'log_start': "45'11000", 'ondisk_log_start': "45'11000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3442", 'last_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'objects_scrubbed': 3384, 'log_size': 10061, 'log_dups_size': 0, 'ondisk_log_size': 10061, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1292, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4372, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 17673, 'num_write_kb': 28580, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4370}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:10:14.800 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-10T16:10:14.800 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "49'21082", 'reported_seq': 21243, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:09:57.000962+0000', 'last_change': '2024-09-10T15:59:16.331899+0000', 'last_active': '2024-09-10T16:09:57.000962+0000', 'last_peered': '2024-09-10T16:09:57.000962+0000', 'last_clean': '2024-09-10T15:59:14.139436+0000', 'last_became_active': '2024-09-10T15:59:16.318715+0000', 'last_became_peered': '2024-09-10T15:59:16.318715+0000', 'last_unstale': '2024-09-10T16:09:57.000962+0000', 'last_undegraded': '2024-09-10T15:59:15.877524+0000', 'last_fullsized': '2024-09-10T15:59:15.769042+0000', 'mapping_epoch': 42, 'log_start': "45'11000", 'ondisk_log_start': "45'11000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3537", 'last_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'objects_scrubbed': 3497, 'log_size': 10082, 'log_dups_size': 0, 'ondisk_log_size': 10082, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1224, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4237, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 17581, 'num_write_kb': 28172, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4235}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:10:14.800 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-10T16:10:14.800 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "49'20781", 'reported_seq': 26116, 'reported_epoch': 55, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:10:11.172519+0000', 'last_change': '2024-09-10T16:09:55.745709+0000', 'last_active': '2024-09-10T16:10:11.172519+0000', 'last_peered': '2024-09-10T16:10:11.172519+0000', 'last_clean': '2024-09-10T15:59:14.116041+0000', 'last_became_active': '2024-09-10T15:59:16.274543+0000', 'last_became_peered': '2024-09-10T15:59:16.274543+0000', 'last_unstale': '2024-09-10T16:10:11.172519+0000', 'last_undegraded': '2024-09-10T15:59:15.812484+0000', 'last_fullsized': '2024-09-10T15:59:15.770885+0000', 'mapping_epoch': 42, 'log_start': "45'10700", 'ondisk_log_start': "45'10700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2356", 'last_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'objects_scrubbed': 2316, 'log_size': 10081, 'log_dups_size': 0, 'ondisk_log_size': 10081, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 783, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1717, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 18461, 'num_write_kb': 32292, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2590, 'num_bytes_recovered': 8675, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -1716}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:10:14.800 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-10T16:10:14.801 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "49'21170", 'reported_seq': 30037, 'reported_epoch': 55, 'state': 'active+recovery_wait', 'last_fresh': '2024-09-10T16:09:57.004800+0000', 'last_change': '2024-09-10T16:08:10.929169+0000', 'last_active': '2024-09-10T16:09:57.004800+0000', 'last_peered': '2024-09-10T16:09:57.004800+0000', 'last_clean': '2024-09-10T15:59:14.110654+0000', 'last_became_active': '2024-09-10T16:08:10.860438+0000', 'last_became_peered': '2024-09-10T16:08:10.860438+0000', 'last_unstale': '2024-09-10T16:09:57.004800+0000', 'last_undegraded': '2024-09-10T16:09:57.004800+0000', 'last_fullsized': '2024-09-10T16:09:57.004800+0000', 'mapping_epoch': 48, 'log_start': "45'11100", 'ondisk_log_start': "45'11100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2486", 'last_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'objects_scrubbed': 2440, 'log_size': 10070, 'log_dups_size': 0, 'ondisk_log_size': 10070, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 792, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18724, 'num_write_kb': 32568, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 4355, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': ['2', '0', '1'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:10:14.801 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-10T16:10:14.801 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "49'20838", 'reported_seq': 20996, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:09:57.000718+0000', 'last_change': '2024-09-10T15:59:16.315641+0000', 'last_active': '2024-09-10T16:09:57.000718+0000', 'last_peered': '2024-09-10T16:09:57.000718+0000', 'last_clean': '2024-09-10T15:59:14.117001+0000', 'last_became_active': '2024-09-10T15:59:16.302245+0000', 'last_became_peered': '2024-09-10T15:59:16.302245+0000', 'last_unstale': '2024-09-10T16:09:57.000718+0000', 'last_undegraded': '2024-09-10T15:59:15.845012+0000', 'last_fullsized': '2024-09-10T15:59:15.770337+0000', 'mapping_epoch': 42, 'log_start': "45'10800", 'ondisk_log_start': "45'10800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3404", 'last_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'objects_scrubbed': 3367, 'log_size': 10038, 'log_dups_size': 0, 'ondisk_log_size': 10038, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 983, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17469, 'num_write_kb': 28204, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4304}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:10:14.801 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-10T16:10:14.801 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "49'21048", 'reported_seq': 21213, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:09:57.002350+0000', 'last_change': '2024-09-10T15:59:16.311291+0000', 'last_active': '2024-09-10T16:09:57.002350+0000', 'last_peered': '2024-09-10T16:09:57.002350+0000', 'last_clean': '2024-09-10T15:59:14.116363+0000', 'last_became_active': '2024-09-10T15:59:16.301244+0000', 'last_became_peered': '2024-09-10T15:59:16.301244+0000', 'last_unstale': '2024-09-10T16:09:57.002350+0000', 'last_undegraded': '2024-09-10T15:59:15.883434+0000', 'last_fullsized': '2024-09-10T15:59:15.772756+0000', 'mapping_epoch': 42, 'log_start': "45'11000", 'ondisk_log_start': "45'11000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3789", 'last_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'objects_scrubbed': 3741, 'log_size': 10048, 'log_dups_size': 0, 'ondisk_log_size': 10048, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1440, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4373, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17303, 'num_write_kb': 27124, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4371}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:10:14.801 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-10T16:10:14.801 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "49'21198", 'reported_seq': 21366, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:09:57.001544+0000', 'last_change': '2024-09-10T15:59:16.313409+0000', 'last_active': '2024-09-10T16:09:57.001544+0000', 'last_peered': '2024-09-10T16:09:57.001544+0000', 'last_clean': '2024-09-10T15:59:14.129780+0000', 'last_became_active': '2024-09-10T15:59:16.300932+0000', 'last_became_peered': '2024-09-10T15:59:16.300932+0000', 'last_unstale': '2024-09-10T16:09:57.001544+0000', 'last_undegraded': '2024-09-10T15:59:15.887823+0000', 'last_fullsized': '2024-09-10T15:59:15.768513+0000', 'mapping_epoch': 42, 'log_start': "45'11100", 'ondisk_log_start': "45'11100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3880", 'last_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'objects_scrubbed': 3827, 'log_size': 10098, 'log_dups_size': 0, 'ondisk_log_size': 10098, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1397, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4325, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17361, 'num_write_kb': 27072, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4321}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:10:14.801 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T16:10:15.199 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:10:15.199 INFO:teuthology.orchestra.run.smithi115.stdout:{"fsid":"7effee29-df05-4351-9741-ee203564ed81","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 23329/24 objects degraded (97204.167%), 6 pgs degraded, 6 pgs undersized","count":12},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":779,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":55,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983954,"num_in_osds":3,"osd_in_since":1725983947,"num_remapped_pgs":6},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":17},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":5},{"state_name":"active+recovery_wait","count":1},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":12,"data_bytes":4849,"bytes_used":1547456512,"bytes_avail":288362835968,"bytes_total":289910292480,"degraded_objects":23329,"degraded_total":24,"degraded_ratio":972.04166666666652,"recovering_objects_per_sec":140,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1686,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-10T15:57:14:967120+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T16:00:13.560776+0000","services":{}},"progress_events":{}} 2024-09-10T16:10:15.213 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-10T16:10:15.213 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:10:15.573 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:10:15.573 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:10:15.587 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":437,"stamp":"2024-09-10T16:10:13.781367+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":23329,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292098,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28063,"num_bytes_recovered":33096665,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161025,"ondisk_log_size":161025,"up":48,"acting":42,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1511188,"kb_used_data":206932,"kb_used_omap":12,"kb_used_meta":1304179,"kb_avail":281604332,"statfs":{"total":289910292480,"available":288362835968,"internally_reserved":0,"allocated":211898368,"data_stored":210856627,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1335479298},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1686,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1686,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001982"},"pg_stats":[{"pgid":"2.f","version":"49'20672","reported_seq":29296,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T16:09:57.088647+0000","last_change":"2024-09-10T16:09:57.088531+0000","last_active":"2024-09-10T16:09:57.088647+0000","last_peered":"2024-09-10T16:09:57.088647+0000","last_clean":"2024-09-10T16:09:57.088647+0000","last_became_active":"2024-09-10T16:09:57.005755+0000","last_became_peered":"2024-09-10T16:09:57.005755+0000","last_unstale":"2024-09-10T16:09:57.088647+0000","last_undegraded":"2024-09-10T16:09:57.088647+0000","last_fullsized":"2024-09-10T16:09:57.088647+0000","mapping_epoch":54,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:09:19.329697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20672,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4259,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20844","reported_seq":20980,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T16:09:57.005461+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:09:57.005461+0000","last_peered":"2024-09-10T16:09:57.005461+0000","last_clean":"2024-09-10T16:09:57.005461+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:09:57.005461+0000","last_undegraded":"2024-09-10T16:09:57.005461+0000","last_fullsized":"2024-09-10T16:09:57.005461+0000","mapping_epoch":35,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18949,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20798","reported_seq":29499,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T16:09:57.004925+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:09:57.004925+0000","last_peered":"2024-09-10T16:09:57.004925+0000","last_clean":"2024-09-10T16:09:57.004925+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:09:57.004925+0000","last_undegraded":"2024-09-10T16:09:57.004925+0000","last_fullsized":"2024-09-10T16:09:57.004925+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18802,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20660","reported_seq":20764,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T16:09:57.005318+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:09:57.005318+0000","last_peered":"2024-09-10T16:09:57.005318+0000","last_clean":"2024-09-10T16:09:57.005318+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:09:57.005318+0000","last_undegraded":"2024-09-10T16:09:57.005318+0000","last_fullsized":"2024-09-10T16:09:57.005318+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20660,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20891","reported_seq":21044,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T16:09:57.001440+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:09:57.001440+0000","last_peered":"2024-09-10T16:09:57.001440+0000","last_clean":"2024-09-10T16:09:57.001440+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:09:57.001440+0000","last_undegraded":"2024-09-10T16:09:57.001440+0000","last_fullsized":"2024-09-10T16:09:57.001440+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17602,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'21061","reported_seq":21222,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:57.000888+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:09:57.000888+0000","last_peered":"2024-09-10T16:09:57.000888+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:09:57.000888+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17673,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4370},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'21082","reported_seq":21243,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:57.000962+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:09:57.000962+0000","last_peered":"2024-09-10T16:09:57.000962+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:09:57.000962+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17581,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4235},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'21314","reported_seq":21467,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T16:09:57.000718+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:09:57.000718+0000","last_peered":"2024-09-10T16:09:57.000718+0000","last_clean":"2024-09-10T16:09:57.000718+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:09:57.000718+0000","last_undegraded":"2024-09-10T16:09:57.000718+0000","last_fullsized":"2024-09-10T16:09:57.000718+0000","mapping_epoch":34,"log_start":"45'11300","ondisk_log_start":"45'11300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18161,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T16:09:57.003703+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:09:57.003703+0000","last_peered":"2024-09-10T16:09:57.003703+0000","last_clean":"2024-09-10T16:09:57.003703+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:09:57.003703+0000","last_undegraded":"2024-09-10T16:09:57.003703+0000","last_fullsized":"2024-09-10T16:09:57.003703+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20457","reported_seq":20624,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T16:09:57.004545+0000","last_change":"2024-09-10T16:09:33.012840+0000","last_active":"2024-09-10T16:09:57.004545+0000","last_peered":"2024-09-10T16:09:57.004545+0000","last_clean":"2024-09-10T16:09:57.004545+0000","last_became_active":"2024-09-10T16:09:32.934056+0000","last_became_peered":"2024-09-10T16:09:32.934056+0000","last_unstale":"2024-09-10T16:09:57.004545+0000","last_undegraded":"2024-09-10T16:09:57.004545+0000","last_fullsized":"2024-09-10T16:09:57.004545+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:08:30.481235+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18276,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":4579,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":117,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T16:09:57.004914+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:09:57.004914+0000","last_peered":"2024-09-10T16:09:57.004914+0000","last_clean":"2024-09-10T16:09:57.004914+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:09:57.004914+0000","last_undegraded":"2024-09-10T16:09:57.004914+0000","last_fullsized":"2024-09-10T16:09:57.004914+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20910","reported_seq":29586,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T16:09:57.001273+0000","last_change":"2024-09-10T16:09:08.909580+0000","last_active":"2024-09-10T16:09:57.001273+0000","last_peered":"2024-09-10T16:09:57.001273+0000","last_clean":"2024-09-10T16:09:57.001273+0000","last_became_active":"2024-09-10T16:09:08.854941+0000","last_became_peered":"2024-09-10T16:09:08.854941+0000","last_unstale":"2024-09-10T16:09:57.001273+0000","last_undegraded":"2024-09-10T16:09:57.001273+0000","last_fullsized":"2024-09-10T16:09:57.001273+0000","mapping_epoch":50,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:13:14.846691+0000","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17414,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4257,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":127,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T16:09:57.000790+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:09:57.000790+0000","last_peered":"2024-09-10T16:09:57.000790+0000","last_clean":"2024-09-10T16:09:57.000790+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:09:57.000790+0000","last_undegraded":"2024-09-10T16:09:57.000790+0000","last_fullsized":"2024-09-10T16:09:57.000790+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20781","reported_seq":26116,"reported_epoch":55,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:10:11.172519+0000","last_change":"2024-09-10T16:09:55.745709+0000","last_active":"2024-09-10T16:10:11.172519+0000","last_peered":"2024-09-10T16:10:11.172519+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:10:11.172519+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1717,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18461,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2590,"num_bytes_recovered":8675,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1716}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":103,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T16:09:57.004400+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:09:57.004400+0000","last_peered":"2024-09-10T16:09:57.004400+0000","last_clean":"2024-09-10T16:09:57.004400+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:09:57.004400+0000","last_undegraded":"2024-09-10T16:09:57.004400+0000","last_fullsized":"2024-09-10T16:09:57.004400+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'21170","reported_seq":30037,"reported_epoch":55,"state":"active+recovery_wait","last_fresh":"2024-09-10T16:09:57.004800+0000","last_change":"2024-09-10T16:08:10.929169+0000","last_active":"2024-09-10T16:09:57.004800+0000","last_peered":"2024-09-10T16:09:57.004800+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T16:08:10.860438+0000","last_became_peered":"2024-09-10T16:08:10.860438+0000","last_unstale":"2024-09-10T16:09:57.004800+0000","last_undegraded":"2024-09-10T16:09:57.004800+0000","last_fullsized":"2024-09-10T16:09:57.004800+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18724,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","0","1"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20838","reported_seq":20996,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:57.000718+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:09:57.000718+0000","last_peered":"2024-09-10T16:09:57.000718+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:09:57.000718+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17469,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T16:09:57.004632+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:09:57.004632+0000","last_peered":"2024-09-10T16:09:57.004632+0000","last_clean":"2024-09-10T16:09:57.004632+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:09:57.004632+0000","last_undegraded":"2024-09-10T16:09:57.004632+0000","last_fullsized":"2024-09-10T16:09:57.004632+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20598","reported_seq":29187,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T16:09:57.005160+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:09:57.005160+0000","last_peered":"2024-09-10T16:09:57.005160+0000","last_clean":"2024-09-10T16:09:57.005160+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:09:57.005160+0000","last_undegraded":"2024-09-10T16:09:57.005160+0000","last_fullsized":"2024-09-10T16:09:57.005160+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16988,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":117,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T16:09:57.005454+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:09:57.005454+0000","last_peered":"2024-09-10T16:09:57.005454+0000","last_clean":"2024-09-10T16:09:57.005454+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:09:57.005454+0000","last_undegraded":"2024-09-10T16:09:57.005454+0000","last_fullsized":"2024-09-10T16:09:57.005454+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'21048","reported_seq":21213,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:57.002350+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:09:57.002350+0000","last_peered":"2024-09-10T16:09:57.002350+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:09:57.002350+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17303,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T16:09:57.005070+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:09:57.005070+0000","last_peered":"2024-09-10T16:09:57.005070+0000","last_clean":"2024-09-10T16:09:57.005070+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:09:57.005070+0000","last_undegraded":"2024-09-10T16:09:57.005070+0000","last_fullsized":"2024-09-10T16:09:57.005070+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":98,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T16:09:57.005104+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:09:57.005104+0000","last_peered":"2024-09-10T16:09:57.005104+0000","last_clean":"2024-09-10T16:09:57.005104+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:09:57.005104+0000","last_undegraded":"2024-09-10T16:09:57.005104+0000","last_fullsized":"2024-09-10T16:09:57.005104+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'21198","reported_seq":21366,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:57.001544+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:09:57.001544+0000","last_peered":"2024-09-10T16:09:57.001544+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:09:57.001544+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17361,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4321},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":23329,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292096,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28061,"num_bytes_recovered":33096646,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":210513920,"data_stored":210394691,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161022,"ondisk_log_size":161022,"up":32,"acting":26,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986322,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":502648,"kb_used_data":137760,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93869192,"statfs":{"total":96636764160,"available":96122052608,"internally_reserved":0,"allocated":141066240,"data_stored":140725324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659267,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":461192,"kb_used_data":504,"kb_used_omap":4,"kb_used_meta":460667,"kb_avail":93910648,"statfs":{"total":96636764160,"available":96164503552,"internally_reserved":0,"allocated":516096,"data_stored":160264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":471723694},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182550,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":547348,"kb_used_data":68668,"kb_used_omap":4,"kb_used_meta":478651,"kb_avail":93824492,"statfs":{"total":96636764160,"available":96076279808,"internally_reserved":0,"allocated":70316032,"data_stored":69971039,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":490139306},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":69853184,"data_stored":69817054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":6279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":140607488,"data_stored":140571358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:10:18.589 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:10:18.910 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:10:18.911 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:10:18.923 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":439,"stamp":"2024-09-10T16:10:17.781962+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":22442,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292098,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28950,"num_bytes_recovered":33096665,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161025,"ondisk_log_size":161025,"up":48,"acting":42,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1537556,"kb_used_data":206932,"kb_used_omap":12,"kb_used_meta":1330547,"kb_avail":281577964,"statfs":{"total":289910292480,"available":288335835136,"internally_reserved":0,"allocated":211898368,"data_stored":210856627,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1362480130},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2573,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2573,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001907"},"pg_stats":[{"pgid":"2.f","version":"49'20672","reported_seq":29296,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T16:09:57.088647+0000","last_change":"2024-09-10T16:09:57.088531+0000","last_active":"2024-09-10T16:09:57.088647+0000","last_peered":"2024-09-10T16:09:57.088647+0000","last_clean":"2024-09-10T16:09:57.088647+0000","last_became_active":"2024-09-10T16:09:57.005755+0000","last_became_peered":"2024-09-10T16:09:57.005755+0000","last_unstale":"2024-09-10T16:09:57.088647+0000","last_undegraded":"2024-09-10T16:09:57.088647+0000","last_fullsized":"2024-09-10T16:09:57.088647+0000","mapping_epoch":54,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:09:19.329697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20672,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4259,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20844","reported_seq":20980,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T16:09:57.005461+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:09:57.005461+0000","last_peered":"2024-09-10T16:09:57.005461+0000","last_clean":"2024-09-10T16:09:57.005461+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:09:57.005461+0000","last_undegraded":"2024-09-10T16:09:57.005461+0000","last_fullsized":"2024-09-10T16:09:57.005461+0000","mapping_epoch":35,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18949,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20798","reported_seq":29499,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T16:09:57.004925+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:09:57.004925+0000","last_peered":"2024-09-10T16:09:57.004925+0000","last_clean":"2024-09-10T16:09:57.004925+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:09:57.004925+0000","last_undegraded":"2024-09-10T16:09:57.004925+0000","last_fullsized":"2024-09-10T16:09:57.004925+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18802,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20660","reported_seq":20764,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T16:09:57.005318+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:09:57.005318+0000","last_peered":"2024-09-10T16:09:57.005318+0000","last_clean":"2024-09-10T16:09:57.005318+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:09:57.005318+0000","last_undegraded":"2024-09-10T16:09:57.005318+0000","last_fullsized":"2024-09-10T16:09:57.005318+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20660,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20891","reported_seq":21044,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T16:09:57.001440+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:09:57.001440+0000","last_peered":"2024-09-10T16:09:57.001440+0000","last_clean":"2024-09-10T16:09:57.001440+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:09:57.001440+0000","last_undegraded":"2024-09-10T16:09:57.001440+0000","last_fullsized":"2024-09-10T16:09:57.001440+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17602,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'21061","reported_seq":21222,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:57.000888+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:09:57.000888+0000","last_peered":"2024-09-10T16:09:57.000888+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:09:57.000888+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17673,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4370},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'21082","reported_seq":21243,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:57.000962+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:09:57.000962+0000","last_peered":"2024-09-10T16:09:57.000962+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:09:57.000962+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17581,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4235},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'21314","reported_seq":21467,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T16:09:57.000718+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:09:57.000718+0000","last_peered":"2024-09-10T16:09:57.000718+0000","last_clean":"2024-09-10T16:09:57.000718+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:09:57.000718+0000","last_undegraded":"2024-09-10T16:09:57.000718+0000","last_fullsized":"2024-09-10T16:09:57.000718+0000","mapping_epoch":34,"log_start":"45'11300","ondisk_log_start":"45'11300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18161,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T16:09:57.003703+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:09:57.003703+0000","last_peered":"2024-09-10T16:09:57.003703+0000","last_clean":"2024-09-10T16:09:57.003703+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:09:57.003703+0000","last_undegraded":"2024-09-10T16:09:57.003703+0000","last_fullsized":"2024-09-10T16:09:57.003703+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20457","reported_seq":20624,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T16:09:57.004545+0000","last_change":"2024-09-10T16:09:33.012840+0000","last_active":"2024-09-10T16:09:57.004545+0000","last_peered":"2024-09-10T16:09:57.004545+0000","last_clean":"2024-09-10T16:09:57.004545+0000","last_became_active":"2024-09-10T16:09:32.934056+0000","last_became_peered":"2024-09-10T16:09:32.934056+0000","last_unstale":"2024-09-10T16:09:57.004545+0000","last_undegraded":"2024-09-10T16:09:57.004545+0000","last_fullsized":"2024-09-10T16:09:57.004545+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:08:30.481235+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18276,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":4579,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":117,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T16:09:57.004914+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:09:57.004914+0000","last_peered":"2024-09-10T16:09:57.004914+0000","last_clean":"2024-09-10T16:09:57.004914+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:09:57.004914+0000","last_undegraded":"2024-09-10T16:09:57.004914+0000","last_fullsized":"2024-09-10T16:09:57.004914+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20910","reported_seq":29586,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T16:09:57.001273+0000","last_change":"2024-09-10T16:09:08.909580+0000","last_active":"2024-09-10T16:09:57.001273+0000","last_peered":"2024-09-10T16:09:57.001273+0000","last_clean":"2024-09-10T16:09:57.001273+0000","last_became_active":"2024-09-10T16:09:08.854941+0000","last_became_peered":"2024-09-10T16:09:08.854941+0000","last_unstale":"2024-09-10T16:09:57.001273+0000","last_undegraded":"2024-09-10T16:09:57.001273+0000","last_fullsized":"2024-09-10T16:09:57.001273+0000","mapping_epoch":50,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:13:14.846691+0000","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17414,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4257,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":127,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T16:09:57.000790+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:09:57.000790+0000","last_peered":"2024-09-10T16:09:57.000790+0000","last_clean":"2024-09-10T16:09:57.000790+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:09:57.000790+0000","last_undegraded":"2024-09-10T16:09:57.000790+0000","last_fullsized":"2024-09-10T16:09:57.000790+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20781","reported_seq":27890,"reported_epoch":55,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:10:16.176740+0000","last_change":"2024-09-10T16:09:55.745709+0000","last_active":"2024-09-10T16:10:16.176740+0000","last_peered":"2024-09-10T16:10:16.176740+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:10:16.176740+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":830,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18461,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3477,"num_bytes_recovered":8675,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-829}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":103,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T16:09:57.004400+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:09:57.004400+0000","last_peered":"2024-09-10T16:09:57.004400+0000","last_clean":"2024-09-10T16:09:57.004400+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:09:57.004400+0000","last_undegraded":"2024-09-10T16:09:57.004400+0000","last_fullsized":"2024-09-10T16:09:57.004400+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'21170","reported_seq":30037,"reported_epoch":55,"state":"active+recovery_wait","last_fresh":"2024-09-10T16:09:57.004800+0000","last_change":"2024-09-10T16:08:10.929169+0000","last_active":"2024-09-10T16:09:57.004800+0000","last_peered":"2024-09-10T16:09:57.004800+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T16:08:10.860438+0000","last_became_peered":"2024-09-10T16:08:10.860438+0000","last_unstale":"2024-09-10T16:09:57.004800+0000","last_undegraded":"2024-09-10T16:09:57.004800+0000","last_fullsized":"2024-09-10T16:09:57.004800+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18724,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","0","1"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20838","reported_seq":20996,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:57.000718+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:09:57.000718+0000","last_peered":"2024-09-10T16:09:57.000718+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:09:57.000718+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17469,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T16:09:57.004632+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:09:57.004632+0000","last_peered":"2024-09-10T16:09:57.004632+0000","last_clean":"2024-09-10T16:09:57.004632+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:09:57.004632+0000","last_undegraded":"2024-09-10T16:09:57.004632+0000","last_fullsized":"2024-09-10T16:09:57.004632+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20598","reported_seq":29187,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T16:09:57.005160+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:09:57.005160+0000","last_peered":"2024-09-10T16:09:57.005160+0000","last_clean":"2024-09-10T16:09:57.005160+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:09:57.005160+0000","last_undegraded":"2024-09-10T16:09:57.005160+0000","last_fullsized":"2024-09-10T16:09:57.005160+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16988,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":117,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T16:09:57.005454+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:09:57.005454+0000","last_peered":"2024-09-10T16:09:57.005454+0000","last_clean":"2024-09-10T16:09:57.005454+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:09:57.005454+0000","last_undegraded":"2024-09-10T16:09:57.005454+0000","last_fullsized":"2024-09-10T16:09:57.005454+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'21048","reported_seq":21213,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:57.002350+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:09:57.002350+0000","last_peered":"2024-09-10T16:09:57.002350+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:09:57.002350+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17303,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T16:09:57.005070+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:09:57.005070+0000","last_peered":"2024-09-10T16:09:57.005070+0000","last_clean":"2024-09-10T16:09:57.005070+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:09:57.005070+0000","last_undegraded":"2024-09-10T16:09:57.005070+0000","last_fullsized":"2024-09-10T16:09:57.005070+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":98,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T16:09:57.005104+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:09:57.005104+0000","last_peered":"2024-09-10T16:09:57.005104+0000","last_clean":"2024-09-10T16:09:57.005104+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:09:57.005104+0000","last_undegraded":"2024-09-10T16:09:57.005104+0000","last_fullsized":"2024-09-10T16:09:57.005104+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'21198","reported_seq":21366,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:57.001544+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:09:57.001544+0000","last_peered":"2024-09-10T16:09:57.001544+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:09:57.001544+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17361,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4321},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":22442,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292096,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28948,"num_bytes_recovered":33096646,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":210513920,"data_stored":210394691,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161022,"ondisk_log_size":161022,"up":32,"acting":26,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986323,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":502648,"kb_used_data":137760,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93869192,"statfs":{"total":96636764160,"available":96122052608,"internally_reserved":0,"allocated":141066240,"data_stored":140725324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659268,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487560,"kb_used_data":504,"kb_used_omap":4,"kb_used_meta":487035,"kb_avail":93884280,"statfs":{"total":96636764160,"available":96137502720,"internally_reserved":0,"allocated":516096,"data_stored":160264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":498724526},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182551,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":547348,"kb_used_data":68668,"kb_used_omap":4,"kb_used_meta":478651,"kb_avail":93824492,"statfs":{"total":96636764160,"available":96076279808,"internally_reserved":0,"allocated":70316032,"data_stored":69971039,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":490139306},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":69853184,"data_stored":69817054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":6279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":140607488,"data_stored":140571358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:10:18.925 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:10:19.241 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:10:19.241 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:10:19.253 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":439,"stamp":"2024-09-10T16:10:17.781962+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":22442,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292098,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28950,"num_bytes_recovered":33096665,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161025,"ondisk_log_size":161025,"up":48,"acting":42,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1537556,"kb_used_data":206932,"kb_used_omap":12,"kb_used_meta":1330547,"kb_avail":281577964,"statfs":{"total":289910292480,"available":288335835136,"internally_reserved":0,"allocated":211898368,"data_stored":210856627,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1362480130},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2573,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2573,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001907"},"pg_stats":[{"pgid":"2.f","version":"49'20672","reported_seq":29296,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T16:09:57.088647+0000","last_change":"2024-09-10T16:09:57.088531+0000","last_active":"2024-09-10T16:09:57.088647+0000","last_peered":"2024-09-10T16:09:57.088647+0000","last_clean":"2024-09-10T16:09:57.088647+0000","last_became_active":"2024-09-10T16:09:57.005755+0000","last_became_peered":"2024-09-10T16:09:57.005755+0000","last_unstale":"2024-09-10T16:09:57.088647+0000","last_undegraded":"2024-09-10T16:09:57.088647+0000","last_fullsized":"2024-09-10T16:09:57.088647+0000","mapping_epoch":54,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:09:19.329697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20672,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4259,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20844","reported_seq":20980,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T16:09:57.005461+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:09:57.005461+0000","last_peered":"2024-09-10T16:09:57.005461+0000","last_clean":"2024-09-10T16:09:57.005461+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:09:57.005461+0000","last_undegraded":"2024-09-10T16:09:57.005461+0000","last_fullsized":"2024-09-10T16:09:57.005461+0000","mapping_epoch":35,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18949,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20798","reported_seq":29499,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T16:09:57.004925+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:09:57.004925+0000","last_peered":"2024-09-10T16:09:57.004925+0000","last_clean":"2024-09-10T16:09:57.004925+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:09:57.004925+0000","last_undegraded":"2024-09-10T16:09:57.004925+0000","last_fullsized":"2024-09-10T16:09:57.004925+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18802,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20660","reported_seq":20764,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T16:09:57.005318+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:09:57.005318+0000","last_peered":"2024-09-10T16:09:57.005318+0000","last_clean":"2024-09-10T16:09:57.005318+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:09:57.005318+0000","last_undegraded":"2024-09-10T16:09:57.005318+0000","last_fullsized":"2024-09-10T16:09:57.005318+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20660,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20891","reported_seq":21044,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T16:09:57.001440+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:09:57.001440+0000","last_peered":"2024-09-10T16:09:57.001440+0000","last_clean":"2024-09-10T16:09:57.001440+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:09:57.001440+0000","last_undegraded":"2024-09-10T16:09:57.001440+0000","last_fullsized":"2024-09-10T16:09:57.001440+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17602,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'21061","reported_seq":21222,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:57.000888+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:09:57.000888+0000","last_peered":"2024-09-10T16:09:57.000888+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:09:57.000888+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17673,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4370},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'21082","reported_seq":21243,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:57.000962+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:09:57.000962+0000","last_peered":"2024-09-10T16:09:57.000962+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:09:57.000962+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17581,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4235},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'21314","reported_seq":21467,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T16:09:57.000718+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:09:57.000718+0000","last_peered":"2024-09-10T16:09:57.000718+0000","last_clean":"2024-09-10T16:09:57.000718+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:09:57.000718+0000","last_undegraded":"2024-09-10T16:09:57.000718+0000","last_fullsized":"2024-09-10T16:09:57.000718+0000","mapping_epoch":34,"log_start":"45'11300","ondisk_log_start":"45'11300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18161,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T16:09:57.003703+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:09:57.003703+0000","last_peered":"2024-09-10T16:09:57.003703+0000","last_clean":"2024-09-10T16:09:57.003703+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:09:57.003703+0000","last_undegraded":"2024-09-10T16:09:57.003703+0000","last_fullsized":"2024-09-10T16:09:57.003703+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20457","reported_seq":20624,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T16:09:57.004545+0000","last_change":"2024-09-10T16:09:33.012840+0000","last_active":"2024-09-10T16:09:57.004545+0000","last_peered":"2024-09-10T16:09:57.004545+0000","last_clean":"2024-09-10T16:09:57.004545+0000","last_became_active":"2024-09-10T16:09:32.934056+0000","last_became_peered":"2024-09-10T16:09:32.934056+0000","last_unstale":"2024-09-10T16:09:57.004545+0000","last_undegraded":"2024-09-10T16:09:57.004545+0000","last_fullsized":"2024-09-10T16:09:57.004545+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:08:30.481235+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18276,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":4579,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":117,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T16:09:57.004914+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:09:57.004914+0000","last_peered":"2024-09-10T16:09:57.004914+0000","last_clean":"2024-09-10T16:09:57.004914+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:09:57.004914+0000","last_undegraded":"2024-09-10T16:09:57.004914+0000","last_fullsized":"2024-09-10T16:09:57.004914+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20910","reported_seq":29586,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T16:09:57.001273+0000","last_change":"2024-09-10T16:09:08.909580+0000","last_active":"2024-09-10T16:09:57.001273+0000","last_peered":"2024-09-10T16:09:57.001273+0000","last_clean":"2024-09-10T16:09:57.001273+0000","last_became_active":"2024-09-10T16:09:08.854941+0000","last_became_peered":"2024-09-10T16:09:08.854941+0000","last_unstale":"2024-09-10T16:09:57.001273+0000","last_undegraded":"2024-09-10T16:09:57.001273+0000","last_fullsized":"2024-09-10T16:09:57.001273+0000","mapping_epoch":50,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:13:14.846691+0000","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17414,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4257,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":127,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T16:09:57.000790+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:09:57.000790+0000","last_peered":"2024-09-10T16:09:57.000790+0000","last_clean":"2024-09-10T16:09:57.000790+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:09:57.000790+0000","last_undegraded":"2024-09-10T16:09:57.000790+0000","last_fullsized":"2024-09-10T16:09:57.000790+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20781","reported_seq":27890,"reported_epoch":55,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:10:16.176740+0000","last_change":"2024-09-10T16:09:55.745709+0000","last_active":"2024-09-10T16:10:16.176740+0000","last_peered":"2024-09-10T16:10:16.176740+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:10:16.176740+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":830,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18461,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3477,"num_bytes_recovered":8675,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-829}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":103,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T16:09:57.004400+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:09:57.004400+0000","last_peered":"2024-09-10T16:09:57.004400+0000","last_clean":"2024-09-10T16:09:57.004400+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:09:57.004400+0000","last_undegraded":"2024-09-10T16:09:57.004400+0000","last_fullsized":"2024-09-10T16:09:57.004400+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'21170","reported_seq":30037,"reported_epoch":55,"state":"active+recovery_wait","last_fresh":"2024-09-10T16:09:57.004800+0000","last_change":"2024-09-10T16:08:10.929169+0000","last_active":"2024-09-10T16:09:57.004800+0000","last_peered":"2024-09-10T16:09:57.004800+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T16:08:10.860438+0000","last_became_peered":"2024-09-10T16:08:10.860438+0000","last_unstale":"2024-09-10T16:09:57.004800+0000","last_undegraded":"2024-09-10T16:09:57.004800+0000","last_fullsized":"2024-09-10T16:09:57.004800+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18724,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","0","1"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20838","reported_seq":20996,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:57.000718+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:09:57.000718+0000","last_peered":"2024-09-10T16:09:57.000718+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:09:57.000718+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17469,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T16:09:57.004632+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:09:57.004632+0000","last_peered":"2024-09-10T16:09:57.004632+0000","last_clean":"2024-09-10T16:09:57.004632+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:09:57.004632+0000","last_undegraded":"2024-09-10T16:09:57.004632+0000","last_fullsized":"2024-09-10T16:09:57.004632+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20598","reported_seq":29187,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T16:09:57.005160+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:09:57.005160+0000","last_peered":"2024-09-10T16:09:57.005160+0000","last_clean":"2024-09-10T16:09:57.005160+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:09:57.005160+0000","last_undegraded":"2024-09-10T16:09:57.005160+0000","last_fullsized":"2024-09-10T16:09:57.005160+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16988,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":117,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T16:09:57.005454+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:09:57.005454+0000","last_peered":"2024-09-10T16:09:57.005454+0000","last_clean":"2024-09-10T16:09:57.005454+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:09:57.005454+0000","last_undegraded":"2024-09-10T16:09:57.005454+0000","last_fullsized":"2024-09-10T16:09:57.005454+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'21048","reported_seq":21213,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:57.002350+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:09:57.002350+0000","last_peered":"2024-09-10T16:09:57.002350+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:09:57.002350+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17303,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T16:09:57.005070+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:09:57.005070+0000","last_peered":"2024-09-10T16:09:57.005070+0000","last_clean":"2024-09-10T16:09:57.005070+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:09:57.005070+0000","last_undegraded":"2024-09-10T16:09:57.005070+0000","last_fullsized":"2024-09-10T16:09:57.005070+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":98,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T16:09:57.005104+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:09:57.005104+0000","last_peered":"2024-09-10T16:09:57.005104+0000","last_clean":"2024-09-10T16:09:57.005104+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:09:57.005104+0000","last_undegraded":"2024-09-10T16:09:57.005104+0000","last_fullsized":"2024-09-10T16:09:57.005104+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'21198","reported_seq":21366,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:57.001544+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:09:57.001544+0000","last_peered":"2024-09-10T16:09:57.001544+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:09:57.001544+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17361,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4321},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":22442,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292096,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28948,"num_bytes_recovered":33096646,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":210513920,"data_stored":210394691,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161022,"ondisk_log_size":161022,"up":32,"acting":26,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986323,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":502648,"kb_used_data":137760,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93869192,"statfs":{"total":96636764160,"available":96122052608,"internally_reserved":0,"allocated":141066240,"data_stored":140725324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659268,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487560,"kb_used_data":504,"kb_used_omap":4,"kb_used_meta":487035,"kb_avail":93884280,"statfs":{"total":96636764160,"available":96137502720,"internally_reserved":0,"allocated":516096,"data_stored":160264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":498724526},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182551,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":547348,"kb_used_data":68668,"kb_used_omap":4,"kb_used_meta":478651,"kb_avail":93824492,"statfs":{"total":96636764160,"available":96076279808,"internally_reserved":0,"allocated":70316032,"data_stored":69971039,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":490139306},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":69853184,"data_stored":69817054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":6279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":140607488,"data_stored":140571358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:10:19.254 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-10T16:10:19.254 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "49'21061", 'reported_seq': 21222, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:09:57.000888+0000', 'last_change': '2024-09-10T15:59:16.308264+0000', 'last_active': '2024-09-10T16:09:57.000888+0000', 'last_peered': '2024-09-10T16:09:57.000888+0000', 'last_clean': '2024-09-10T15:59:14.111551+0000', 'last_became_active': '2024-09-10T15:59:16.300040+0000', 'last_became_peered': '2024-09-10T15:59:16.300040+0000', 'last_unstale': '2024-09-10T16:09:57.000888+0000', 'last_undegraded': '2024-09-10T15:59:15.878234+0000', 'last_fullsized': '2024-09-10T15:59:15.769328+0000', 'mapping_epoch': 42, 'log_start': "45'11000", 'ondisk_log_start': "45'11000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3442", 'last_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'objects_scrubbed': 3384, 'log_size': 10061, 'log_dups_size': 0, 'ondisk_log_size': 10061, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1292, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4372, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 17673, 'num_write_kb': 28580, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4370}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:10:19.254 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-10T16:10:19.255 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "49'21082", 'reported_seq': 21243, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:09:57.000962+0000', 'last_change': '2024-09-10T15:59:16.331899+0000', 'last_active': '2024-09-10T16:09:57.000962+0000', 'last_peered': '2024-09-10T16:09:57.000962+0000', 'last_clean': '2024-09-10T15:59:14.139436+0000', 'last_became_active': '2024-09-10T15:59:16.318715+0000', 'last_became_peered': '2024-09-10T15:59:16.318715+0000', 'last_unstale': '2024-09-10T16:09:57.000962+0000', 'last_undegraded': '2024-09-10T15:59:15.877524+0000', 'last_fullsized': '2024-09-10T15:59:15.769042+0000', 'mapping_epoch': 42, 'log_start': "45'11000", 'ondisk_log_start': "45'11000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3537", 'last_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'objects_scrubbed': 3497, 'log_size': 10082, 'log_dups_size': 0, 'ondisk_log_size': 10082, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1224, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4237, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 17581, 'num_write_kb': 28172, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4235}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:10:19.255 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-10T16:10:19.255 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "49'20781", 'reported_seq': 27890, 'reported_epoch': 55, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:10:16.176740+0000', 'last_change': '2024-09-10T16:09:55.745709+0000', 'last_active': '2024-09-10T16:10:16.176740+0000', 'last_peered': '2024-09-10T16:10:16.176740+0000', 'last_clean': '2024-09-10T15:59:14.116041+0000', 'last_became_active': '2024-09-10T15:59:16.274543+0000', 'last_became_peered': '2024-09-10T15:59:16.274543+0000', 'last_unstale': '2024-09-10T16:10:16.176740+0000', 'last_undegraded': '2024-09-10T15:59:15.812484+0000', 'last_fullsized': '2024-09-10T15:59:15.770885+0000', 'mapping_epoch': 42, 'log_start': "45'10700", 'ondisk_log_start': "45'10700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2356", 'last_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:49.040618+0000', 'objects_scrubbed': 2316, 'log_size': 10081, 'log_dups_size': 0, 'ondisk_log_size': 10081, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 783, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 830, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 18461, 'num_write_kb': 32292, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3477, 'num_bytes_recovered': 8675, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -829}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:10:19.255 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-10T16:10:19.255 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "49'21170", 'reported_seq': 30037, 'reported_epoch': 55, 'state': 'active+recovery_wait', 'last_fresh': '2024-09-10T16:09:57.004800+0000', 'last_change': '2024-09-10T16:08:10.929169+0000', 'last_active': '2024-09-10T16:09:57.004800+0000', 'last_peered': '2024-09-10T16:09:57.004800+0000', 'last_clean': '2024-09-10T15:59:14.110654+0000', 'last_became_active': '2024-09-10T16:08:10.860438+0000', 'last_became_peered': '2024-09-10T16:08:10.860438+0000', 'last_unstale': '2024-09-10T16:09:57.004800+0000', 'last_undegraded': '2024-09-10T16:09:57.004800+0000', 'last_fullsized': '2024-09-10T16:09:57.004800+0000', 'mapping_epoch': 48, 'log_start': "45'11100", 'ondisk_log_start': "45'11100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2486", 'last_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'objects_scrubbed': 2440, 'log_size': 10070, 'log_dups_size': 0, 'ondisk_log_size': 10070, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 792, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18724, 'num_write_kb': 32568, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 4355, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': ['2', '0', '1'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:10:19.255 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-10T16:10:19.255 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "49'20838", 'reported_seq': 20996, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:09:57.000718+0000', 'last_change': '2024-09-10T15:59:16.315641+0000', 'last_active': '2024-09-10T16:09:57.000718+0000', 'last_peered': '2024-09-10T16:09:57.000718+0000', 'last_clean': '2024-09-10T15:59:14.117001+0000', 'last_became_active': '2024-09-10T15:59:16.302245+0000', 'last_became_peered': '2024-09-10T15:59:16.302245+0000', 'last_unstale': '2024-09-10T16:09:57.000718+0000', 'last_undegraded': '2024-09-10T15:59:15.845012+0000', 'last_fullsized': '2024-09-10T15:59:15.770337+0000', 'mapping_epoch': 42, 'log_start': "45'10800", 'ondisk_log_start': "45'10800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3404", 'last_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'objects_scrubbed': 3367, 'log_size': 10038, 'log_dups_size': 0, 'ondisk_log_size': 10038, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 983, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17469, 'num_write_kb': 28204, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4304}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:10:19.255 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-10T16:10:19.255 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "49'21048", 'reported_seq': 21213, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:09:57.002350+0000', 'last_change': '2024-09-10T15:59:16.311291+0000', 'last_active': '2024-09-10T16:09:57.002350+0000', 'last_peered': '2024-09-10T16:09:57.002350+0000', 'last_clean': '2024-09-10T15:59:14.116363+0000', 'last_became_active': '2024-09-10T15:59:16.301244+0000', 'last_became_peered': '2024-09-10T15:59:16.301244+0000', 'last_unstale': '2024-09-10T16:09:57.002350+0000', 'last_undegraded': '2024-09-10T15:59:15.883434+0000', 'last_fullsized': '2024-09-10T15:59:15.772756+0000', 'mapping_epoch': 42, 'log_start': "45'11000", 'ondisk_log_start': "45'11000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3789", 'last_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'objects_scrubbed': 3741, 'log_size': 10048, 'log_dups_size': 0, 'ondisk_log_size': 10048, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1440, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4373, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17303, 'num_write_kb': 27124, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4371}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:10:19.255 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-10T16:10:19.256 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "49'21198", 'reported_seq': 21366, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:09:57.001544+0000', 'last_change': '2024-09-10T15:59:16.313409+0000', 'last_active': '2024-09-10T16:09:57.001544+0000', 'last_peered': '2024-09-10T16:09:57.001544+0000', 'last_clean': '2024-09-10T15:59:14.129780+0000', 'last_became_active': '2024-09-10T15:59:16.300932+0000', 'last_became_peered': '2024-09-10T15:59:16.300932+0000', 'last_unstale': '2024-09-10T16:09:57.001544+0000', 'last_undegraded': '2024-09-10T15:59:15.887823+0000', 'last_fullsized': '2024-09-10T15:59:15.768513+0000', 'mapping_epoch': 42, 'log_start': "45'11100", 'ondisk_log_start': "45'11100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3880", 'last_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'objects_scrubbed': 3827, 'log_size': 10098, 'log_dups_size': 0, 'ondisk_log_size': 10098, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1397, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4325, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17361, 'num_write_kb': 27072, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4321}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:10:19.256 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T16:10:19.625 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:10:19.625 INFO:teuthology.orchestra.run.smithi115.stdout:{"fsid":"7effee29-df05-4351-9741-ee203564ed81","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 22442/24 objects degraded (93508.333%), 6 pgs degraded, 6 pgs undersized","count":12},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":783,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":55,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983954,"num_in_osds":3,"osd_in_since":1725983947,"num_remapped_pgs":6},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":17},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":5},{"state_name":"active+recovery_wait","count":1},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":12,"data_bytes":4849,"bytes_used":1574457344,"bytes_avail":288335835136,"bytes_total":289910292480,"degraded_objects":22442,"degraded_total":24,"degraded_ratio":935.08333333333348,"recovering_objects_per_sec":214,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":2573,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-10T15:57:14:967120+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T16:00:13.560776+0000","services":{}},"progress_events":{}} 2024-09-10T16:10:19.637 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-10T16:10:19.637 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:10:19.970 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:10:19.970 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:10:19.985 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":441,"stamp":"2024-09-10T16:10:19.848430+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":22442,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292098,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28950,"num_bytes_recovered":33096665,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161025,"ondisk_log_size":161025,"up":48,"acting":42,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1537556,"kb_used_data":206932,"kb_used_omap":12,"kb_used_meta":1330547,"kb_avail":281577964,"statfs":{"total":289910292480,"available":288335835136,"internally_reserved":0,"allocated":211898368,"data_stored":210856627,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1362480130},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1624,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.067718"},"pg_stats":[{"pgid":"2.f","version":"49'20672","reported_seq":29296,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T16:09:57.088647+0000","last_change":"2024-09-10T16:09:57.088531+0000","last_active":"2024-09-10T16:09:57.088647+0000","last_peered":"2024-09-10T16:09:57.088647+0000","last_clean":"2024-09-10T16:09:57.088647+0000","last_became_active":"2024-09-10T16:09:57.005755+0000","last_became_peered":"2024-09-10T16:09:57.005755+0000","last_unstale":"2024-09-10T16:09:57.088647+0000","last_undegraded":"2024-09-10T16:09:57.088647+0000","last_fullsized":"2024-09-10T16:09:57.088647+0000","mapping_epoch":54,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:09:19.329697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20672,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4259,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20844","reported_seq":20980,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T16:09:57.005461+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:09:57.005461+0000","last_peered":"2024-09-10T16:09:57.005461+0000","last_clean":"2024-09-10T16:09:57.005461+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:09:57.005461+0000","last_undegraded":"2024-09-10T16:09:57.005461+0000","last_fullsized":"2024-09-10T16:09:57.005461+0000","mapping_epoch":35,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18949,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20798","reported_seq":29499,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T16:09:57.004925+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:09:57.004925+0000","last_peered":"2024-09-10T16:09:57.004925+0000","last_clean":"2024-09-10T16:09:57.004925+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:09:57.004925+0000","last_undegraded":"2024-09-10T16:09:57.004925+0000","last_fullsized":"2024-09-10T16:09:57.004925+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18802,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20660","reported_seq":20764,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T16:09:57.005318+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:09:57.005318+0000","last_peered":"2024-09-10T16:09:57.005318+0000","last_clean":"2024-09-10T16:09:57.005318+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:09:57.005318+0000","last_undegraded":"2024-09-10T16:09:57.005318+0000","last_fullsized":"2024-09-10T16:09:57.005318+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20660,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20891","reported_seq":21044,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T16:09:57.001440+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:09:57.001440+0000","last_peered":"2024-09-10T16:09:57.001440+0000","last_clean":"2024-09-10T16:09:57.001440+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:09:57.001440+0000","last_undegraded":"2024-09-10T16:09:57.001440+0000","last_fullsized":"2024-09-10T16:09:57.001440+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17602,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'21061","reported_seq":21222,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:57.000888+0000","last_change":"2024-09-10T15:59:16.308264+0000","last_active":"2024-09-10T16:09:57.000888+0000","last_peered":"2024-09-10T16:09:57.000888+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:09:57.000888+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17673,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4370},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'21082","reported_seq":21243,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:57.000962+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:09:57.000962+0000","last_peered":"2024-09-10T16:09:57.000962+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:09:57.000962+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17581,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4235},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'21314","reported_seq":21467,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T16:09:57.000718+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:09:57.000718+0000","last_peered":"2024-09-10T16:09:57.000718+0000","last_clean":"2024-09-10T16:09:57.000718+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:09:57.000718+0000","last_undegraded":"2024-09-10T16:09:57.000718+0000","last_fullsized":"2024-09-10T16:09:57.000718+0000","mapping_epoch":34,"log_start":"45'11300","ondisk_log_start":"45'11300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18161,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T16:09:57.003703+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:09:57.003703+0000","last_peered":"2024-09-10T16:09:57.003703+0000","last_clean":"2024-09-10T16:09:57.003703+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:09:57.003703+0000","last_undegraded":"2024-09-10T16:09:57.003703+0000","last_fullsized":"2024-09-10T16:09:57.003703+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20457","reported_seq":20624,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T16:09:57.004545+0000","last_change":"2024-09-10T16:09:33.012840+0000","last_active":"2024-09-10T16:09:57.004545+0000","last_peered":"2024-09-10T16:09:57.004545+0000","last_clean":"2024-09-10T16:09:57.004545+0000","last_became_active":"2024-09-10T16:09:32.934056+0000","last_became_peered":"2024-09-10T16:09:32.934056+0000","last_unstale":"2024-09-10T16:09:57.004545+0000","last_undegraded":"2024-09-10T16:09:57.004545+0000","last_fullsized":"2024-09-10T16:09:57.004545+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:08:30.481235+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18276,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":4579,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":117,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T16:09:57.004914+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:09:57.004914+0000","last_peered":"2024-09-10T16:09:57.004914+0000","last_clean":"2024-09-10T16:09:57.004914+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:09:57.004914+0000","last_undegraded":"2024-09-10T16:09:57.004914+0000","last_fullsized":"2024-09-10T16:09:57.004914+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20910","reported_seq":29586,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T16:09:57.001273+0000","last_change":"2024-09-10T16:09:08.909580+0000","last_active":"2024-09-10T16:09:57.001273+0000","last_peered":"2024-09-10T16:09:57.001273+0000","last_clean":"2024-09-10T16:09:57.001273+0000","last_became_active":"2024-09-10T16:09:08.854941+0000","last_became_peered":"2024-09-10T16:09:08.854941+0000","last_unstale":"2024-09-10T16:09:57.001273+0000","last_undegraded":"2024-09-10T16:09:57.001273+0000","last_fullsized":"2024-09-10T16:09:57.001273+0000","mapping_epoch":50,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:13:14.846691+0000","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17414,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4257,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":127,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T16:09:57.000790+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:09:57.000790+0000","last_peered":"2024-09-10T16:09:57.000790+0000","last_clean":"2024-09-10T16:09:57.000790+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:09:57.000790+0000","last_undegraded":"2024-09-10T16:09:57.000790+0000","last_fullsized":"2024-09-10T16:09:57.000790+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20781","reported_seq":27890,"reported_epoch":55,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:10:16.176740+0000","last_change":"2024-09-10T16:09:55.745709+0000","last_active":"2024-09-10T16:10:16.176740+0000","last_peered":"2024-09-10T16:10:16.176740+0000","last_clean":"2024-09-10T15:59:14.116041+0000","last_became_active":"2024-09-10T15:59:16.274543+0000","last_became_peered":"2024-09-10T15:59:16.274543+0000","last_unstale":"2024-09-10T16:10:16.176740+0000","last_undegraded":"2024-09-10T15:59:15.812484+0000","last_fullsized":"2024-09-10T15:59:15.770885+0000","mapping_epoch":42,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":830,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18461,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3477,"num_bytes_recovered":8675,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-829}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":103,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T16:09:57.004400+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:09:57.004400+0000","last_peered":"2024-09-10T16:09:57.004400+0000","last_clean":"2024-09-10T16:09:57.004400+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:09:57.004400+0000","last_undegraded":"2024-09-10T16:09:57.004400+0000","last_fullsized":"2024-09-10T16:09:57.004400+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'21170","reported_seq":30037,"reported_epoch":55,"state":"active+recovery_wait","last_fresh":"2024-09-10T16:09:57.004800+0000","last_change":"2024-09-10T16:08:10.929169+0000","last_active":"2024-09-10T16:09:57.004800+0000","last_peered":"2024-09-10T16:09:57.004800+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T16:08:10.860438+0000","last_became_peered":"2024-09-10T16:08:10.860438+0000","last_unstale":"2024-09-10T16:09:57.004800+0000","last_undegraded":"2024-09-10T16:09:57.004800+0000","last_fullsized":"2024-09-10T16:09:57.004800+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18724,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","0","1"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20838","reported_seq":20996,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:57.000718+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:09:57.000718+0000","last_peered":"2024-09-10T16:09:57.000718+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:09:57.000718+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17469,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T16:09:57.004632+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:09:57.004632+0000","last_peered":"2024-09-10T16:09:57.004632+0000","last_clean":"2024-09-10T16:09:57.004632+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:09:57.004632+0000","last_undegraded":"2024-09-10T16:09:57.004632+0000","last_fullsized":"2024-09-10T16:09:57.004632+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20598","reported_seq":29187,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T16:09:57.005160+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:09:57.005160+0000","last_peered":"2024-09-10T16:09:57.005160+0000","last_clean":"2024-09-10T16:09:57.005160+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:09:57.005160+0000","last_undegraded":"2024-09-10T16:09:57.005160+0000","last_fullsized":"2024-09-10T16:09:57.005160+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16988,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":117,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T16:09:57.005454+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:09:57.005454+0000","last_peered":"2024-09-10T16:09:57.005454+0000","last_clean":"2024-09-10T16:09:57.005454+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:09:57.005454+0000","last_undegraded":"2024-09-10T16:09:57.005454+0000","last_fullsized":"2024-09-10T16:09:57.005454+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'21048","reported_seq":21213,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:57.002350+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:09:57.002350+0000","last_peered":"2024-09-10T16:09:57.002350+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:09:57.002350+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17303,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T16:09:57.005070+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:09:57.005070+0000","last_peered":"2024-09-10T16:09:57.005070+0000","last_clean":"2024-09-10T16:09:57.005070+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:09:57.005070+0000","last_undegraded":"2024-09-10T16:09:57.005070+0000","last_fullsized":"2024-09-10T16:09:57.005070+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":98,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T16:09:57.005104+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:09:57.005104+0000","last_peered":"2024-09-10T16:09:57.005104+0000","last_clean":"2024-09-10T16:09:57.005104+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:09:57.005104+0000","last_undegraded":"2024-09-10T16:09:57.005104+0000","last_fullsized":"2024-09-10T16:09:57.005104+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'21198","reported_seq":21366,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:09:57.001544+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:09:57.001544+0000","last_peered":"2024-09-10T16:09:57.001544+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:09:57.001544+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17361,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4321},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":22442,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292096,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28948,"num_bytes_recovered":33096646,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":210513920,"data_stored":210394691,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161022,"ondisk_log_size":161022,"up":32,"acting":26,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986323,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":502648,"kb_used_data":137760,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93869192,"statfs":{"total":96636764160,"available":96122052608,"internally_reserved":0,"allocated":141066240,"data_stored":140725324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659268,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487560,"kb_used_data":504,"kb_used_omap":4,"kb_used_meta":487035,"kb_avail":93884280,"statfs":{"total":96636764160,"available":96137502720,"internally_reserved":0,"allocated":516096,"data_stored":160264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":498724526},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182551,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":547348,"kb_used_data":68668,"kb_used_omap":4,"kb_used_meta":478651,"kb_avail":93824492,"statfs":{"total":96636764160,"available":96076279808,"internally_reserved":0,"allocated":70316032,"data_stored":69971039,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":490139306},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":69853184,"data_stored":69817054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":6279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":140607488,"data_stored":140571358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:10:22.987 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:10:23.316 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:10:23.317 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:10:23.328 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":443,"stamp":"2024-09-10T16:10:21.782700+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":21470,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292098,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29922,"num_bytes_recovered":33097148,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161025,"ondisk_log_size":161025,"up":48,"acting":43,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1529264,"kb_used_data":206960,"kb_used_omap":12,"kb_used_meta":1322227,"kb_avail":281586256,"statfs":{"total":289910292480,"available":288344326144,"internally_reserved":0,"allocated":211927040,"data_stored":210867531,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1353960450},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1859,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1859,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001332"},"pg_stats":[{"pgid":"2.f","version":"49'20672","reported_seq":29300,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.856536+0000","last_change":"2024-09-10T16:09:57.088531+0000","last_active":"2024-09-10T16:10:20.856536+0000","last_peered":"2024-09-10T16:10:20.856536+0000","last_clean":"2024-09-10T16:10:20.856536+0000","last_became_active":"2024-09-10T16:09:57.005755+0000","last_became_peered":"2024-09-10T16:09:57.005755+0000","last_unstale":"2024-09-10T16:10:20.856536+0000","last_undegraded":"2024-09-10T16:10:20.856536+0000","last_fullsized":"2024-09-10T16:10:20.856536+0000","mapping_epoch":54,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:09:19.329697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20672,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4259,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20844","reported_seq":20984,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.850591+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:10:20.850591+0000","last_peered":"2024-09-10T16:10:20.850591+0000","last_clean":"2024-09-10T16:10:20.850591+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:10:20.850591+0000","last_undegraded":"2024-09-10T16:10:20.850591+0000","last_fullsized":"2024-09-10T16:10:20.850591+0000","mapping_epoch":35,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18949,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20798","reported_seq":29501,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:10:19.852416+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:10:19.852416+0000","last_peered":"2024-09-10T16:10:19.852416+0000","last_clean":"2024-09-10T16:10:19.852416+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:10:19.852416+0000","last_undegraded":"2024-09-10T16:10:19.852416+0000","last_fullsized":"2024-09-10T16:10:19.852416+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18802,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20660","reported_seq":20768,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.850554+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:10:20.850554+0000","last_peered":"2024-09-10T16:10:20.850554+0000","last_clean":"2024-09-10T16:10:20.850554+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:10:20.850554+0000","last_undegraded":"2024-09-10T16:10:20.850554+0000","last_fullsized":"2024-09-10T16:10:20.850554+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20660,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20891","reported_seq":21048,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.854539+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:10:20.854539+0000","last_peered":"2024-09-10T16:10:20.854539+0000","last_clean":"2024-09-10T16:10:20.854539+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:10:20.854539+0000","last_undegraded":"2024-09-10T16:10:20.854539+0000","last_fullsized":"2024-09-10T16:10:20.854539+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17602,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'21061","reported_seq":21511,"reported_epoch":57,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:10:21.211567+0000","last_change":"2024-09-10T16:10:19.743410+0000","last_active":"2024-09-10T16:10:21.211567+0000","last_peered":"2024-09-10T16:10:21.211567+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:10:21.211567+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4230,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17673,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":142,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4229}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'21082","reported_seq":21247,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:10:20.854635+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:10:20.854635+0000","last_peered":"2024-09-10T16:10:20.854635+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:10:20.854635+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17581,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4235},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'21314","reported_seq":21471,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.854391+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:10:20.854391+0000","last_peered":"2024-09-10T16:10:20.854391+0000","last_clean":"2024-09-10T16:10:20.854391+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:10:20.854391+0000","last_undegraded":"2024-09-10T16:10:20.854391+0000","last_fullsized":"2024-09-10T16:10:20.854391+0000","mapping_epoch":34,"log_start":"45'11300","ondisk_log_start":"45'11300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18161,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":95,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:10:19.852001+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:10:19.852001+0000","last_peered":"2024-09-10T16:10:19.852001+0000","last_clean":"2024-09-10T16:10:19.852001+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:10:19.852001+0000","last_undegraded":"2024-09-10T16:10:19.852001+0000","last_fullsized":"2024-09-10T16:10:19.852001+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20457","reported_seq":20626,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:10:19.852214+0000","last_change":"2024-09-10T16:09:33.012840+0000","last_active":"2024-09-10T16:10:19.852214+0000","last_peered":"2024-09-10T16:10:19.852214+0000","last_clean":"2024-09-10T16:10:19.852214+0000","last_became_active":"2024-09-10T16:09:32.934056+0000","last_became_peered":"2024-09-10T16:09:32.934056+0000","last_unstale":"2024-09-10T16:10:19.852214+0000","last_undegraded":"2024-09-10T16:10:19.852214+0000","last_fullsized":"2024-09-10T16:10:19.852214+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:08:30.481235+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18276,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":4579,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":121,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.850337+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:10:20.850337+0000","last_peered":"2024-09-10T16:10:20.850337+0000","last_clean":"2024-09-10T16:10:20.850337+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:10:20.850337+0000","last_undegraded":"2024-09-10T16:10:20.850337+0000","last_fullsized":"2024-09-10T16:10:20.850337+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20910","reported_seq":29590,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.854908+0000","last_change":"2024-09-10T16:09:08.909580+0000","last_active":"2024-09-10T16:10:20.854908+0000","last_peered":"2024-09-10T16:10:20.854908+0000","last_clean":"2024-09-10T16:10:20.854908+0000","last_became_active":"2024-09-10T16:09:08.854941+0000","last_became_peered":"2024-09-10T16:09:08.854941+0000","last_unstale":"2024-09-10T16:10:20.854908+0000","last_undegraded":"2024-09-10T16:10:20.854908+0000","last_fullsized":"2024-09-10T16:10:20.854908+0000","mapping_epoch":50,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:13:14.846691+0000","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17414,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4257,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":131,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.854759+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:10:20.854759+0000","last_peered":"2024-09-10T16:10:20.854759+0000","last_clean":"2024-09-10T16:10:20.854759+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:10:20.854759+0000","last_undegraded":"2024-09-10T16:10:20.854759+0000","last_fullsized":"2024-09-10T16:10:20.854759+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20781","reported_seq":29561,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.907987+0000","last_change":"2024-09-10T16:10:20.907889+0000","last_active":"2024-09-10T16:10:20.907987+0000","last_peered":"2024-09-10T16:10:20.907987+0000","last_clean":"2024-09-10T16:10:20.907987+0000","last_became_active":"2024-09-10T16:10:20.852065+0000","last_became_peered":"2024-09-10T16:10:20.852065+0000","last_unstale":"2024-09-10T16:10:20.907987+0000","last_undegraded":"2024-09-10T16:10:20.907987+0000","last_fullsized":"2024-09-10T16:10:20.907987+0000","mapping_epoch":56,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:34.069549+0000","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18461,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4307,"num_bytes_recovered":8675,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":105,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:10:19.852141+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:10:19.852141+0000","last_peered":"2024-09-10T16:10:19.852141+0000","last_clean":"2024-09-10T16:10:19.852141+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:10:19.852141+0000","last_undegraded":"2024-09-10T16:10:19.852141+0000","last_fullsized":"2024-09-10T16:10:19.852141+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'21170","reported_seq":30042,"reported_epoch":57,"state":"active+recovery_wait","last_fresh":"2024-09-10T16:10:20.850377+0000","last_change":"2024-09-10T16:08:10.929169+0000","last_active":"2024-09-10T16:10:20.850377+0000","last_peered":"2024-09-10T16:10:20.850377+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T16:08:10.860438+0000","last_became_peered":"2024-09-10T16:08:10.860438+0000","last_unstale":"2024-09-10T16:10:20.850377+0000","last_undegraded":"2024-09-10T16:10:20.850377+0000","last_fullsized":"2024-09-10T16:10:20.850377+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18724,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","0","1"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20838","reported_seq":21000,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:10:20.854744+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:10:20.854744+0000","last_peered":"2024-09-10T16:10:20.854744+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:10:20.854744+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17469,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":95,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:10:19.852339+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:10:19.852339+0000","last_peered":"2024-09-10T16:10:19.852339+0000","last_clean":"2024-09-10T16:10:19.852339+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:10:19.852339+0000","last_undegraded":"2024-09-10T16:10:19.852339+0000","last_fullsized":"2024-09-10T16:10:19.852339+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20598","reported_seq":29189,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:10:19.852630+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:10:19.852630+0000","last_peered":"2024-09-10T16:10:19.852630+0000","last_clean":"2024-09-10T16:10:19.852630+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:10:19.852630+0000","last_undegraded":"2024-09-10T16:10:19.852630+0000","last_fullsized":"2024-09-10T16:10:19.852630+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16988,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":121,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.850614+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:10:20.850614+0000","last_peered":"2024-09-10T16:10:20.850614+0000","last_clean":"2024-09-10T16:10:20.850614+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:10:20.850614+0000","last_undegraded":"2024-09-10T16:10:20.850614+0000","last_fullsized":"2024-09-10T16:10:20.850614+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'21048","reported_seq":21217,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:10:20.855031+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:10:20.855031+0000","last_peered":"2024-09-10T16:10:20.855031+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:10:20.855031+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17303,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":95,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:10:19.852560+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:10:19.852560+0000","last_peered":"2024-09-10T16:10:19.852560+0000","last_clean":"2024-09-10T16:10:19.852560+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:10:19.852560+0000","last_undegraded":"2024-09-10T16:10:19.852560+0000","last_fullsized":"2024-09-10T16:10:19.852560+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":100,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:10:19.852018+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:10:19.852018+0000","last_peered":"2024-09-10T16:10:19.852018+0000","last_clean":"2024-09-10T16:10:19.852018+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:10:19.852018+0000","last_undegraded":"2024-09-10T16:10:19.852018+0000","last_fullsized":"2024-09-10T16:10:19.852018+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'21198","reported_seq":21370,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:10:20.854530+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:10:20.854530+0000","last_peered":"2024-09-10T16:10:20.854530+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:10:20.854530+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17361,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4321},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":21470,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292096,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29920,"num_bytes_recovered":33097129,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":210509824,"data_stored":210394208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161022,"ondisk_log_size":161022,"up":32,"acting":27,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986324,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":502652,"kb_used_data":137768,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93869188,"statfs":{"total":96636764160,"available":96122048512,"internally_reserved":0,"allocated":141074432,"data_stored":140728176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659269,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":479248,"kb_used_data":508,"kb_used_omap":4,"kb_used_meta":478715,"kb_avail":93892592,"statfs":{"total":96636764160,"available":96146014208,"internally_reserved":0,"allocated":520192,"data_stored":162633,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":490204846},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182552,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":547364,"kb_used_data":68684,"kb_used_omap":4,"kb_used_meta":478651,"kb_avail":93824476,"statfs":{"total":96636764160,"available":96076263424,"internally_reserved":0,"allocated":70332416,"data_stored":69976722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":490139306},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":69853184,"data_stored":69817054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":5796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":140607488,"data_stored":140571358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:10:23.330 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:10:23.657 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:10:23.658 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:10:23.670 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":443,"stamp":"2024-09-10T16:10:21.782700+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":21470,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292098,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29922,"num_bytes_recovered":33097148,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161025,"ondisk_log_size":161025,"up":48,"acting":43,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1529264,"kb_used_data":206960,"kb_used_omap":12,"kb_used_meta":1322227,"kb_avail":281586256,"statfs":{"total":289910292480,"available":288344326144,"internally_reserved":0,"allocated":211927040,"data_stored":210867531,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1353960450},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1859,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1859,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001332"},"pg_stats":[{"pgid":"2.f","version":"49'20672","reported_seq":29300,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.856536+0000","last_change":"2024-09-10T16:09:57.088531+0000","last_active":"2024-09-10T16:10:20.856536+0000","last_peered":"2024-09-10T16:10:20.856536+0000","last_clean":"2024-09-10T16:10:20.856536+0000","last_became_active":"2024-09-10T16:09:57.005755+0000","last_became_peered":"2024-09-10T16:09:57.005755+0000","last_unstale":"2024-09-10T16:10:20.856536+0000","last_undegraded":"2024-09-10T16:10:20.856536+0000","last_fullsized":"2024-09-10T16:10:20.856536+0000","mapping_epoch":54,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:09:19.329697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20672,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4259,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20844","reported_seq":20984,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.850591+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:10:20.850591+0000","last_peered":"2024-09-10T16:10:20.850591+0000","last_clean":"2024-09-10T16:10:20.850591+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:10:20.850591+0000","last_undegraded":"2024-09-10T16:10:20.850591+0000","last_fullsized":"2024-09-10T16:10:20.850591+0000","mapping_epoch":35,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18949,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20798","reported_seq":29501,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:10:19.852416+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:10:19.852416+0000","last_peered":"2024-09-10T16:10:19.852416+0000","last_clean":"2024-09-10T16:10:19.852416+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:10:19.852416+0000","last_undegraded":"2024-09-10T16:10:19.852416+0000","last_fullsized":"2024-09-10T16:10:19.852416+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18802,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20660","reported_seq":20768,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.850554+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:10:20.850554+0000","last_peered":"2024-09-10T16:10:20.850554+0000","last_clean":"2024-09-10T16:10:20.850554+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:10:20.850554+0000","last_undegraded":"2024-09-10T16:10:20.850554+0000","last_fullsized":"2024-09-10T16:10:20.850554+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20660,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20891","reported_seq":21048,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.854539+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:10:20.854539+0000","last_peered":"2024-09-10T16:10:20.854539+0000","last_clean":"2024-09-10T16:10:20.854539+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:10:20.854539+0000","last_undegraded":"2024-09-10T16:10:20.854539+0000","last_fullsized":"2024-09-10T16:10:20.854539+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17602,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'21061","reported_seq":21511,"reported_epoch":57,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:10:21.211567+0000","last_change":"2024-09-10T16:10:19.743410+0000","last_active":"2024-09-10T16:10:21.211567+0000","last_peered":"2024-09-10T16:10:21.211567+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:10:21.211567+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4230,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17673,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":142,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4229}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'21082","reported_seq":21247,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:10:20.854635+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:10:20.854635+0000","last_peered":"2024-09-10T16:10:20.854635+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:10:20.854635+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17581,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4235},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'21314","reported_seq":21471,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.854391+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:10:20.854391+0000","last_peered":"2024-09-10T16:10:20.854391+0000","last_clean":"2024-09-10T16:10:20.854391+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:10:20.854391+0000","last_undegraded":"2024-09-10T16:10:20.854391+0000","last_fullsized":"2024-09-10T16:10:20.854391+0000","mapping_epoch":34,"log_start":"45'11300","ondisk_log_start":"45'11300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18161,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":95,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:10:19.852001+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:10:19.852001+0000","last_peered":"2024-09-10T16:10:19.852001+0000","last_clean":"2024-09-10T16:10:19.852001+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:10:19.852001+0000","last_undegraded":"2024-09-10T16:10:19.852001+0000","last_fullsized":"2024-09-10T16:10:19.852001+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20457","reported_seq":20626,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:10:19.852214+0000","last_change":"2024-09-10T16:09:33.012840+0000","last_active":"2024-09-10T16:10:19.852214+0000","last_peered":"2024-09-10T16:10:19.852214+0000","last_clean":"2024-09-10T16:10:19.852214+0000","last_became_active":"2024-09-10T16:09:32.934056+0000","last_became_peered":"2024-09-10T16:09:32.934056+0000","last_unstale":"2024-09-10T16:10:19.852214+0000","last_undegraded":"2024-09-10T16:10:19.852214+0000","last_fullsized":"2024-09-10T16:10:19.852214+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:08:30.481235+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18276,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":4579,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":121,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.850337+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:10:20.850337+0000","last_peered":"2024-09-10T16:10:20.850337+0000","last_clean":"2024-09-10T16:10:20.850337+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:10:20.850337+0000","last_undegraded":"2024-09-10T16:10:20.850337+0000","last_fullsized":"2024-09-10T16:10:20.850337+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20910","reported_seq":29590,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.854908+0000","last_change":"2024-09-10T16:09:08.909580+0000","last_active":"2024-09-10T16:10:20.854908+0000","last_peered":"2024-09-10T16:10:20.854908+0000","last_clean":"2024-09-10T16:10:20.854908+0000","last_became_active":"2024-09-10T16:09:08.854941+0000","last_became_peered":"2024-09-10T16:09:08.854941+0000","last_unstale":"2024-09-10T16:10:20.854908+0000","last_undegraded":"2024-09-10T16:10:20.854908+0000","last_fullsized":"2024-09-10T16:10:20.854908+0000","mapping_epoch":50,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:13:14.846691+0000","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17414,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4257,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":131,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.854759+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:10:20.854759+0000","last_peered":"2024-09-10T16:10:20.854759+0000","last_clean":"2024-09-10T16:10:20.854759+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:10:20.854759+0000","last_undegraded":"2024-09-10T16:10:20.854759+0000","last_fullsized":"2024-09-10T16:10:20.854759+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20781","reported_seq":29561,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.907987+0000","last_change":"2024-09-10T16:10:20.907889+0000","last_active":"2024-09-10T16:10:20.907987+0000","last_peered":"2024-09-10T16:10:20.907987+0000","last_clean":"2024-09-10T16:10:20.907987+0000","last_became_active":"2024-09-10T16:10:20.852065+0000","last_became_peered":"2024-09-10T16:10:20.852065+0000","last_unstale":"2024-09-10T16:10:20.907987+0000","last_undegraded":"2024-09-10T16:10:20.907987+0000","last_fullsized":"2024-09-10T16:10:20.907987+0000","mapping_epoch":56,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:34.069549+0000","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18461,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4307,"num_bytes_recovered":8675,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":105,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:10:19.852141+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:10:19.852141+0000","last_peered":"2024-09-10T16:10:19.852141+0000","last_clean":"2024-09-10T16:10:19.852141+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:10:19.852141+0000","last_undegraded":"2024-09-10T16:10:19.852141+0000","last_fullsized":"2024-09-10T16:10:19.852141+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'21170","reported_seq":30042,"reported_epoch":57,"state":"active+recovery_wait","last_fresh":"2024-09-10T16:10:20.850377+0000","last_change":"2024-09-10T16:08:10.929169+0000","last_active":"2024-09-10T16:10:20.850377+0000","last_peered":"2024-09-10T16:10:20.850377+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T16:08:10.860438+0000","last_became_peered":"2024-09-10T16:08:10.860438+0000","last_unstale":"2024-09-10T16:10:20.850377+0000","last_undegraded":"2024-09-10T16:10:20.850377+0000","last_fullsized":"2024-09-10T16:10:20.850377+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18724,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","0","1"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20838","reported_seq":21000,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:10:20.854744+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:10:20.854744+0000","last_peered":"2024-09-10T16:10:20.854744+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:10:20.854744+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17469,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":95,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:10:19.852339+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:10:19.852339+0000","last_peered":"2024-09-10T16:10:19.852339+0000","last_clean":"2024-09-10T16:10:19.852339+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:10:19.852339+0000","last_undegraded":"2024-09-10T16:10:19.852339+0000","last_fullsized":"2024-09-10T16:10:19.852339+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20598","reported_seq":29189,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:10:19.852630+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:10:19.852630+0000","last_peered":"2024-09-10T16:10:19.852630+0000","last_clean":"2024-09-10T16:10:19.852630+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:10:19.852630+0000","last_undegraded":"2024-09-10T16:10:19.852630+0000","last_fullsized":"2024-09-10T16:10:19.852630+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16988,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":121,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.850614+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:10:20.850614+0000","last_peered":"2024-09-10T16:10:20.850614+0000","last_clean":"2024-09-10T16:10:20.850614+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:10:20.850614+0000","last_undegraded":"2024-09-10T16:10:20.850614+0000","last_fullsized":"2024-09-10T16:10:20.850614+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'21048","reported_seq":21217,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:10:20.855031+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:10:20.855031+0000","last_peered":"2024-09-10T16:10:20.855031+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:10:20.855031+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17303,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":95,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:10:19.852560+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:10:19.852560+0000","last_peered":"2024-09-10T16:10:19.852560+0000","last_clean":"2024-09-10T16:10:19.852560+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:10:19.852560+0000","last_undegraded":"2024-09-10T16:10:19.852560+0000","last_fullsized":"2024-09-10T16:10:19.852560+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":100,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:10:19.852018+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:10:19.852018+0000","last_peered":"2024-09-10T16:10:19.852018+0000","last_clean":"2024-09-10T16:10:19.852018+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:10:19.852018+0000","last_undegraded":"2024-09-10T16:10:19.852018+0000","last_fullsized":"2024-09-10T16:10:19.852018+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'21198","reported_seq":21370,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:10:20.854530+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:10:20.854530+0000","last_peered":"2024-09-10T16:10:20.854530+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:10:20.854530+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17361,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4321},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":21470,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292096,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29920,"num_bytes_recovered":33097129,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":210509824,"data_stored":210394208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161022,"ondisk_log_size":161022,"up":32,"acting":27,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986324,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":502652,"kb_used_data":137768,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93869188,"statfs":{"total":96636764160,"available":96122048512,"internally_reserved":0,"allocated":141074432,"data_stored":140728176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659269,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":479248,"kb_used_data":508,"kb_used_omap":4,"kb_used_meta":478715,"kb_avail":93892592,"statfs":{"total":96636764160,"available":96146014208,"internally_reserved":0,"allocated":520192,"data_stored":162633,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":490204846},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182552,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":547364,"kb_used_data":68684,"kb_used_omap":4,"kb_used_meta":478651,"kb_avail":93824476,"statfs":{"total":96636764160,"available":96076263424,"internally_reserved":0,"allocated":70332416,"data_stored":69976722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":490139306},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":69853184,"data_stored":69817054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":5796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":140607488,"data_stored":140571358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:10:23.671 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-10T16:10:23.671 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "49'21061", 'reported_seq': 21511, 'reported_epoch': 57, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:10:21.211567+0000', 'last_change': '2024-09-10T16:10:19.743410+0000', 'last_active': '2024-09-10T16:10:21.211567+0000', 'last_peered': '2024-09-10T16:10:21.211567+0000', 'last_clean': '2024-09-10T15:59:14.111551+0000', 'last_became_active': '2024-09-10T15:59:16.300040+0000', 'last_became_peered': '2024-09-10T15:59:16.300040+0000', 'last_unstale': '2024-09-10T16:10:21.211567+0000', 'last_undegraded': '2024-09-10T15:59:15.878234+0000', 'last_fullsized': '2024-09-10T15:59:15.769328+0000', 'mapping_epoch': 42, 'log_start': "45'11000", 'ondisk_log_start': "45'11000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3442", 'last_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'objects_scrubbed': 3384, 'log_size': 10061, 'log_dups_size': 0, 'ondisk_log_size': 10061, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1292, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4230, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 17673, 'num_write_kb': 28580, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 142, 'num_bytes_recovered': 483, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4229}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:10:23.671 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-10T16:10:23.671 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "49'21082", 'reported_seq': 21247, 'reported_epoch': 57, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:10:20.854635+0000', 'last_change': '2024-09-10T15:59:16.331899+0000', 'last_active': '2024-09-10T16:10:20.854635+0000', 'last_peered': '2024-09-10T16:10:20.854635+0000', 'last_clean': '2024-09-10T15:59:14.139436+0000', 'last_became_active': '2024-09-10T15:59:16.318715+0000', 'last_became_peered': '2024-09-10T15:59:16.318715+0000', 'last_unstale': '2024-09-10T16:10:20.854635+0000', 'last_undegraded': '2024-09-10T15:59:15.877524+0000', 'last_fullsized': '2024-09-10T15:59:15.769042+0000', 'mapping_epoch': 42, 'log_start': "45'11000", 'ondisk_log_start': "45'11000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3537", 'last_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'objects_scrubbed': 3497, 'log_size': 10082, 'log_dups_size': 0, 'ondisk_log_size': 10082, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1224, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4237, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 17581, 'num_write_kb': 28172, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4235}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:10:23.672 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-10T16:10:23.672 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "49'21170", 'reported_seq': 30042, 'reported_epoch': 57, 'state': 'active+recovery_wait', 'last_fresh': '2024-09-10T16:10:20.850377+0000', 'last_change': '2024-09-10T16:08:10.929169+0000', 'last_active': '2024-09-10T16:10:20.850377+0000', 'last_peered': '2024-09-10T16:10:20.850377+0000', 'last_clean': '2024-09-10T15:59:14.110654+0000', 'last_became_active': '2024-09-10T16:08:10.860438+0000', 'last_became_peered': '2024-09-10T16:08:10.860438+0000', 'last_unstale': '2024-09-10T16:10:20.850377+0000', 'last_undegraded': '2024-09-10T16:10:20.850377+0000', 'last_fullsized': '2024-09-10T16:10:20.850377+0000', 'mapping_epoch': 48, 'log_start': "45'11100", 'ondisk_log_start': "45'11100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2486", 'last_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'objects_scrubbed': 2440, 'log_size': 10070, 'log_dups_size': 0, 'ondisk_log_size': 10070, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 792, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18724, 'num_write_kb': 32568, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 4355, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': ['2', '0', '1'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:10:23.672 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-10T16:10:23.672 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "49'20838", 'reported_seq': 21000, 'reported_epoch': 57, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:10:20.854744+0000', 'last_change': '2024-09-10T15:59:16.315641+0000', 'last_active': '2024-09-10T16:10:20.854744+0000', 'last_peered': '2024-09-10T16:10:20.854744+0000', 'last_clean': '2024-09-10T15:59:14.117001+0000', 'last_became_active': '2024-09-10T15:59:16.302245+0000', 'last_became_peered': '2024-09-10T15:59:16.302245+0000', 'last_unstale': '2024-09-10T16:10:20.854744+0000', 'last_undegraded': '2024-09-10T15:59:15.845012+0000', 'last_fullsized': '2024-09-10T15:59:15.770337+0000', 'mapping_epoch': 42, 'log_start': "45'10800", 'ondisk_log_start': "45'10800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3404", 'last_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'objects_scrubbed': 3367, 'log_size': 10038, 'log_dups_size': 0, 'ondisk_log_size': 10038, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 983, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17469, 'num_write_kb': 28204, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4304}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:10:23.672 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-10T16:10:23.672 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "49'21048", 'reported_seq': 21217, 'reported_epoch': 57, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:10:20.855031+0000', 'last_change': '2024-09-10T15:59:16.311291+0000', 'last_active': '2024-09-10T16:10:20.855031+0000', 'last_peered': '2024-09-10T16:10:20.855031+0000', 'last_clean': '2024-09-10T15:59:14.116363+0000', 'last_became_active': '2024-09-10T15:59:16.301244+0000', 'last_became_peered': '2024-09-10T15:59:16.301244+0000', 'last_unstale': '2024-09-10T16:10:20.855031+0000', 'last_undegraded': '2024-09-10T15:59:15.883434+0000', 'last_fullsized': '2024-09-10T15:59:15.772756+0000', 'mapping_epoch': 42, 'log_start': "45'11000", 'ondisk_log_start': "45'11000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3789", 'last_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'objects_scrubbed': 3741, 'log_size': 10048, 'log_dups_size': 0, 'ondisk_log_size': 10048, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1440, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4373, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17303, 'num_write_kb': 27124, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4371}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:10:23.672 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-10T16:10:23.672 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "49'21198", 'reported_seq': 21370, 'reported_epoch': 57, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:10:20.854530+0000', 'last_change': '2024-09-10T15:59:16.313409+0000', 'last_active': '2024-09-10T16:10:20.854530+0000', 'last_peered': '2024-09-10T16:10:20.854530+0000', 'last_clean': '2024-09-10T15:59:14.129780+0000', 'last_became_active': '2024-09-10T15:59:16.300932+0000', 'last_became_peered': '2024-09-10T15:59:16.300932+0000', 'last_unstale': '2024-09-10T16:10:20.854530+0000', 'last_undegraded': '2024-09-10T15:59:15.887823+0000', 'last_fullsized': '2024-09-10T15:59:15.768513+0000', 'mapping_epoch': 42, 'log_start': "45'11100", 'ondisk_log_start': "45'11100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3880", 'last_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'objects_scrubbed': 3827, 'log_size': 10098, 'log_dups_size': 0, 'ondisk_log_size': 10098, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1397, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4325, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17361, 'num_write_kb': 27072, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4321}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:10:23.673 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T16:10:24.067 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:10:24.067 INFO:teuthology.orchestra.run.smithi115.stdout:{"fsid":"7effee29-df05-4351-9741-ee203564ed81","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 21470/24 objects degraded (89458.333%), 5 pgs degraded, 5 pgs undersized","count":10},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":788,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":57,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983954,"num_in_osds":3,"osd_in_since":1725983947,"num_remapped_pgs":5},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":18},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":4},{"state_name":"active+recovery_wait","count":1},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":12,"data_bytes":4849,"bytes_used":1565966336,"bytes_avail":288344326144,"bytes_total":289910292480,"degraded_objects":21470,"degraded_total":24,"degraded_ratio":894.58333333333348,"recovering_objects_per_sec":232,"recovering_bytes_per_sec":60,"recovering_keys_per_sec":0,"num_objects_recovered":1859,"num_bytes_recovered":483,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-10T15:57:14:967120+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T16:00:13.560776+0000","services":{}},"progress_events":{}} 2024-09-10T16:10:24.081 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-10T16:10:24.081 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:10:24.384 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:10:24.385 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:10:24.397 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":444,"stamp":"2024-09-10T16:10:23.783073+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":21470,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292098,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29922,"num_bytes_recovered":33097148,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161025,"ondisk_log_size":161025,"up":48,"acting":43,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1529264,"kb_used_data":206960,"kb_used_omap":12,"kb_used_meta":1322227,"kb_avail":281586256,"statfs":{"total":289910292480,"available":288344326144,"internally_reserved":0,"allocated":211927040,"data_stored":210867531,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1353960450},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1859,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1859,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001413"},"pg_stats":[{"pgid":"2.f","version":"49'20672","reported_seq":29300,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.856536+0000","last_change":"2024-09-10T16:09:57.088531+0000","last_active":"2024-09-10T16:10:20.856536+0000","last_peered":"2024-09-10T16:10:20.856536+0000","last_clean":"2024-09-10T16:10:20.856536+0000","last_became_active":"2024-09-10T16:09:57.005755+0000","last_became_peered":"2024-09-10T16:09:57.005755+0000","last_unstale":"2024-09-10T16:10:20.856536+0000","last_undegraded":"2024-09-10T16:10:20.856536+0000","last_fullsized":"2024-09-10T16:10:20.856536+0000","mapping_epoch":54,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:09:19.329697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20672,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4259,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20844","reported_seq":20984,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.850591+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:10:20.850591+0000","last_peered":"2024-09-10T16:10:20.850591+0000","last_clean":"2024-09-10T16:10:20.850591+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:10:20.850591+0000","last_undegraded":"2024-09-10T16:10:20.850591+0000","last_fullsized":"2024-09-10T16:10:20.850591+0000","mapping_epoch":35,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18949,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20798","reported_seq":29501,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:10:19.852416+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:10:19.852416+0000","last_peered":"2024-09-10T16:10:19.852416+0000","last_clean":"2024-09-10T16:10:19.852416+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:10:19.852416+0000","last_undegraded":"2024-09-10T16:10:19.852416+0000","last_fullsized":"2024-09-10T16:10:19.852416+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18802,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20660","reported_seq":20768,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.850554+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:10:20.850554+0000","last_peered":"2024-09-10T16:10:20.850554+0000","last_clean":"2024-09-10T16:10:20.850554+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:10:20.850554+0000","last_undegraded":"2024-09-10T16:10:20.850554+0000","last_fullsized":"2024-09-10T16:10:20.850554+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20660,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20891","reported_seq":21048,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.854539+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:10:20.854539+0000","last_peered":"2024-09-10T16:10:20.854539+0000","last_clean":"2024-09-10T16:10:20.854539+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:10:20.854539+0000","last_undegraded":"2024-09-10T16:10:20.854539+0000","last_fullsized":"2024-09-10T16:10:20.854539+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17602,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'21061","reported_seq":21511,"reported_epoch":57,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:10:21.211567+0000","last_change":"2024-09-10T16:10:19.743410+0000","last_active":"2024-09-10T16:10:21.211567+0000","last_peered":"2024-09-10T16:10:21.211567+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:10:21.211567+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4230,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17673,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":142,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4229}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'21082","reported_seq":21247,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:10:20.854635+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:10:20.854635+0000","last_peered":"2024-09-10T16:10:20.854635+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:10:20.854635+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17581,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4235},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'21314","reported_seq":21471,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.854391+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:10:20.854391+0000","last_peered":"2024-09-10T16:10:20.854391+0000","last_clean":"2024-09-10T16:10:20.854391+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:10:20.854391+0000","last_undegraded":"2024-09-10T16:10:20.854391+0000","last_fullsized":"2024-09-10T16:10:20.854391+0000","mapping_epoch":34,"log_start":"45'11300","ondisk_log_start":"45'11300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18161,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":95,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:10:19.852001+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:10:19.852001+0000","last_peered":"2024-09-10T16:10:19.852001+0000","last_clean":"2024-09-10T16:10:19.852001+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:10:19.852001+0000","last_undegraded":"2024-09-10T16:10:19.852001+0000","last_fullsized":"2024-09-10T16:10:19.852001+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20457","reported_seq":20626,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:10:19.852214+0000","last_change":"2024-09-10T16:09:33.012840+0000","last_active":"2024-09-10T16:10:19.852214+0000","last_peered":"2024-09-10T16:10:19.852214+0000","last_clean":"2024-09-10T16:10:19.852214+0000","last_became_active":"2024-09-10T16:09:32.934056+0000","last_became_peered":"2024-09-10T16:09:32.934056+0000","last_unstale":"2024-09-10T16:10:19.852214+0000","last_undegraded":"2024-09-10T16:10:19.852214+0000","last_fullsized":"2024-09-10T16:10:19.852214+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:08:30.481235+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18276,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":4579,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":121,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.850337+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:10:20.850337+0000","last_peered":"2024-09-10T16:10:20.850337+0000","last_clean":"2024-09-10T16:10:20.850337+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:10:20.850337+0000","last_undegraded":"2024-09-10T16:10:20.850337+0000","last_fullsized":"2024-09-10T16:10:20.850337+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20910","reported_seq":29590,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.854908+0000","last_change":"2024-09-10T16:09:08.909580+0000","last_active":"2024-09-10T16:10:20.854908+0000","last_peered":"2024-09-10T16:10:20.854908+0000","last_clean":"2024-09-10T16:10:20.854908+0000","last_became_active":"2024-09-10T16:09:08.854941+0000","last_became_peered":"2024-09-10T16:09:08.854941+0000","last_unstale":"2024-09-10T16:10:20.854908+0000","last_undegraded":"2024-09-10T16:10:20.854908+0000","last_fullsized":"2024-09-10T16:10:20.854908+0000","mapping_epoch":50,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:13:14.846691+0000","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17414,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4257,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":131,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.854759+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:10:20.854759+0000","last_peered":"2024-09-10T16:10:20.854759+0000","last_clean":"2024-09-10T16:10:20.854759+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:10:20.854759+0000","last_undegraded":"2024-09-10T16:10:20.854759+0000","last_fullsized":"2024-09-10T16:10:20.854759+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20781","reported_seq":29561,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.907987+0000","last_change":"2024-09-10T16:10:20.907889+0000","last_active":"2024-09-10T16:10:20.907987+0000","last_peered":"2024-09-10T16:10:20.907987+0000","last_clean":"2024-09-10T16:10:20.907987+0000","last_became_active":"2024-09-10T16:10:20.852065+0000","last_became_peered":"2024-09-10T16:10:20.852065+0000","last_unstale":"2024-09-10T16:10:20.907987+0000","last_undegraded":"2024-09-10T16:10:20.907987+0000","last_fullsized":"2024-09-10T16:10:20.907987+0000","mapping_epoch":56,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:34.069549+0000","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18461,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4307,"num_bytes_recovered":8675,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":105,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:10:19.852141+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:10:19.852141+0000","last_peered":"2024-09-10T16:10:19.852141+0000","last_clean":"2024-09-10T16:10:19.852141+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:10:19.852141+0000","last_undegraded":"2024-09-10T16:10:19.852141+0000","last_fullsized":"2024-09-10T16:10:19.852141+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'21170","reported_seq":30042,"reported_epoch":57,"state":"active+recovery_wait","last_fresh":"2024-09-10T16:10:20.850377+0000","last_change":"2024-09-10T16:08:10.929169+0000","last_active":"2024-09-10T16:10:20.850377+0000","last_peered":"2024-09-10T16:10:20.850377+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T16:08:10.860438+0000","last_became_peered":"2024-09-10T16:08:10.860438+0000","last_unstale":"2024-09-10T16:10:20.850377+0000","last_undegraded":"2024-09-10T16:10:20.850377+0000","last_fullsized":"2024-09-10T16:10:20.850377+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18724,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","0","1"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20838","reported_seq":21000,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:10:20.854744+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:10:20.854744+0000","last_peered":"2024-09-10T16:10:20.854744+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:10:20.854744+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17469,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":95,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:10:19.852339+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:10:19.852339+0000","last_peered":"2024-09-10T16:10:19.852339+0000","last_clean":"2024-09-10T16:10:19.852339+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:10:19.852339+0000","last_undegraded":"2024-09-10T16:10:19.852339+0000","last_fullsized":"2024-09-10T16:10:19.852339+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20598","reported_seq":29189,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:10:19.852630+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:10:19.852630+0000","last_peered":"2024-09-10T16:10:19.852630+0000","last_clean":"2024-09-10T16:10:19.852630+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:10:19.852630+0000","last_undegraded":"2024-09-10T16:10:19.852630+0000","last_fullsized":"2024-09-10T16:10:19.852630+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16988,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":121,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.850614+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:10:20.850614+0000","last_peered":"2024-09-10T16:10:20.850614+0000","last_clean":"2024-09-10T16:10:20.850614+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:10:20.850614+0000","last_undegraded":"2024-09-10T16:10:20.850614+0000","last_fullsized":"2024-09-10T16:10:20.850614+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'21048","reported_seq":21217,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:10:20.855031+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:10:20.855031+0000","last_peered":"2024-09-10T16:10:20.855031+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:10:20.855031+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17303,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":95,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:10:19.852560+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:10:19.852560+0000","last_peered":"2024-09-10T16:10:19.852560+0000","last_clean":"2024-09-10T16:10:19.852560+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:10:19.852560+0000","last_undegraded":"2024-09-10T16:10:19.852560+0000","last_fullsized":"2024-09-10T16:10:19.852560+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":100,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:10:19.852018+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:10:19.852018+0000","last_peered":"2024-09-10T16:10:19.852018+0000","last_clean":"2024-09-10T16:10:19.852018+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:10:19.852018+0000","last_undegraded":"2024-09-10T16:10:19.852018+0000","last_fullsized":"2024-09-10T16:10:19.852018+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'21198","reported_seq":21370,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:10:20.854530+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:10:20.854530+0000","last_peered":"2024-09-10T16:10:20.854530+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:10:20.854530+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17361,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4321},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":21470,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292096,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29920,"num_bytes_recovered":33097129,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":210509824,"data_stored":210394208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161022,"ondisk_log_size":161022,"up":32,"acting":27,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986324,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":502652,"kb_used_data":137768,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93869188,"statfs":{"total":96636764160,"available":96122048512,"internally_reserved":0,"allocated":141074432,"data_stored":140728176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659269,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":479248,"kb_used_data":508,"kb_used_omap":4,"kb_used_meta":478715,"kb_avail":93892592,"statfs":{"total":96636764160,"available":96146014208,"internally_reserved":0,"allocated":520192,"data_stored":162633,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":490204846},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182552,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":547364,"kb_used_data":68684,"kb_used_omap":4,"kb_used_meta":478651,"kb_avail":93824476,"statfs":{"total":96636764160,"available":96076263424,"internally_reserved":0,"allocated":70332416,"data_stored":69976722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":490139306},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":69853184,"data_stored":69817054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":5796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":140607488,"data_stored":140571358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:10:27.400 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:10:27.705 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:10:27.706 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:10:27.718 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":445,"stamp":"2024-09-10T16:10:25.783401+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":21470,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292098,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29922,"num_bytes_recovered":33097148,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161025,"ondisk_log_size":161025,"up":48,"acting":43,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1533428,"kb_used_data":206968,"kb_used_omap":12,"kb_used_meta":1326387,"kb_avail":281582092,"statfs":{"total":289910292480,"available":288340062208,"internally_reserved":0,"allocated":211935232,"data_stored":210870362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1358220290},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-972,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":972,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001439"},"pg_stats":[{"pgid":"2.f","version":"49'20672","reported_seq":29300,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.856536+0000","last_change":"2024-09-10T16:09:57.088531+0000","last_active":"2024-09-10T16:10:20.856536+0000","last_peered":"2024-09-10T16:10:20.856536+0000","last_clean":"2024-09-10T16:10:20.856536+0000","last_became_active":"2024-09-10T16:09:57.005755+0000","last_became_peered":"2024-09-10T16:09:57.005755+0000","last_unstale":"2024-09-10T16:10:20.856536+0000","last_undegraded":"2024-09-10T16:10:20.856536+0000","last_fullsized":"2024-09-10T16:10:20.856536+0000","mapping_epoch":54,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:09:19.329697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20672,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4259,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20844","reported_seq":20984,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.850591+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:10:20.850591+0000","last_peered":"2024-09-10T16:10:20.850591+0000","last_clean":"2024-09-10T16:10:20.850591+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:10:20.850591+0000","last_undegraded":"2024-09-10T16:10:20.850591+0000","last_fullsized":"2024-09-10T16:10:20.850591+0000","mapping_epoch":35,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18949,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20798","reported_seq":29503,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.851218+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:10:20.851218+0000","last_peered":"2024-09-10T16:10:20.851218+0000","last_clean":"2024-09-10T16:10:20.851218+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:10:20.851218+0000","last_undegraded":"2024-09-10T16:10:20.851218+0000","last_fullsized":"2024-09-10T16:10:20.851218+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18802,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20660","reported_seq":20768,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.850554+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:10:20.850554+0000","last_peered":"2024-09-10T16:10:20.850554+0000","last_clean":"2024-09-10T16:10:20.850554+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:10:20.850554+0000","last_undegraded":"2024-09-10T16:10:20.850554+0000","last_fullsized":"2024-09-10T16:10:20.850554+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20660,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20891","reported_seq":21048,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.854539+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:10:20.854539+0000","last_peered":"2024-09-10T16:10:20.854539+0000","last_clean":"2024-09-10T16:10:20.854539+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:10:20.854539+0000","last_undegraded":"2024-09-10T16:10:20.854539+0000","last_fullsized":"2024-09-10T16:10:20.854539+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17602,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'21061","reported_seq":21511,"reported_epoch":57,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:10:21.211567+0000","last_change":"2024-09-10T16:10:19.743410+0000","last_active":"2024-09-10T16:10:21.211567+0000","last_peered":"2024-09-10T16:10:21.211567+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:10:21.211567+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4230,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17673,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":142,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4229}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'21082","reported_seq":21247,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:10:20.854635+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:10:20.854635+0000","last_peered":"2024-09-10T16:10:20.854635+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:10:20.854635+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17581,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4235},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'21314","reported_seq":21471,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.854391+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:10:20.854391+0000","last_peered":"2024-09-10T16:10:20.854391+0000","last_clean":"2024-09-10T16:10:20.854391+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:10:20.854391+0000","last_undegraded":"2024-09-10T16:10:20.854391+0000","last_fullsized":"2024-09-10T16:10:20.854391+0000","mapping_epoch":34,"log_start":"45'11300","ondisk_log_start":"45'11300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18161,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.850980+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:10:20.850980+0000","last_peered":"2024-09-10T16:10:20.850980+0000","last_clean":"2024-09-10T16:10:20.850980+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:10:20.850980+0000","last_undegraded":"2024-09-10T16:10:20.850980+0000","last_fullsized":"2024-09-10T16:10:20.850980+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20457","reported_seq":20628,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.851257+0000","last_change":"2024-09-10T16:09:33.012840+0000","last_active":"2024-09-10T16:10:20.851257+0000","last_peered":"2024-09-10T16:10:20.851257+0000","last_clean":"2024-09-10T16:10:20.851257+0000","last_became_active":"2024-09-10T16:09:32.934056+0000","last_became_peered":"2024-09-10T16:09:32.934056+0000","last_unstale":"2024-09-10T16:10:20.851257+0000","last_undegraded":"2024-09-10T16:10:20.851257+0000","last_fullsized":"2024-09-10T16:10:20.851257+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:08:30.481235+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18276,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":4579,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":121,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.850337+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:10:20.850337+0000","last_peered":"2024-09-10T16:10:20.850337+0000","last_clean":"2024-09-10T16:10:20.850337+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:10:20.850337+0000","last_undegraded":"2024-09-10T16:10:20.850337+0000","last_fullsized":"2024-09-10T16:10:20.850337+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20910","reported_seq":29590,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.854908+0000","last_change":"2024-09-10T16:09:08.909580+0000","last_active":"2024-09-10T16:10:20.854908+0000","last_peered":"2024-09-10T16:10:20.854908+0000","last_clean":"2024-09-10T16:10:20.854908+0000","last_became_active":"2024-09-10T16:09:08.854941+0000","last_became_peered":"2024-09-10T16:09:08.854941+0000","last_unstale":"2024-09-10T16:10:20.854908+0000","last_undegraded":"2024-09-10T16:10:20.854908+0000","last_fullsized":"2024-09-10T16:10:20.854908+0000","mapping_epoch":50,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:13:14.846691+0000","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17414,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4257,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":131,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.854759+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:10:20.854759+0000","last_peered":"2024-09-10T16:10:20.854759+0000","last_clean":"2024-09-10T16:10:20.854759+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:10:20.854759+0000","last_undegraded":"2024-09-10T16:10:20.854759+0000","last_fullsized":"2024-09-10T16:10:20.854759+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20781","reported_seq":29561,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.907987+0000","last_change":"2024-09-10T16:10:20.907889+0000","last_active":"2024-09-10T16:10:20.907987+0000","last_peered":"2024-09-10T16:10:20.907987+0000","last_clean":"2024-09-10T16:10:20.907987+0000","last_became_active":"2024-09-10T16:10:20.852065+0000","last_became_peered":"2024-09-10T16:10:20.852065+0000","last_unstale":"2024-09-10T16:10:20.907987+0000","last_undegraded":"2024-09-10T16:10:20.907987+0000","last_fullsized":"2024-09-10T16:10:20.907987+0000","mapping_epoch":56,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:34.069549+0000","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18461,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4307,"num_bytes_recovered":8675,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":107,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.851171+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:10:20.851171+0000","last_peered":"2024-09-10T16:10:20.851171+0000","last_clean":"2024-09-10T16:10:20.851171+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:10:20.851171+0000","last_undegraded":"2024-09-10T16:10:20.851171+0000","last_fullsized":"2024-09-10T16:10:20.851171+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'21170","reported_seq":30042,"reported_epoch":57,"state":"active+recovery_wait","last_fresh":"2024-09-10T16:10:20.850377+0000","last_change":"2024-09-10T16:08:10.929169+0000","last_active":"2024-09-10T16:10:20.850377+0000","last_peered":"2024-09-10T16:10:20.850377+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T16:08:10.860438+0000","last_became_peered":"2024-09-10T16:08:10.860438+0000","last_unstale":"2024-09-10T16:10:20.850377+0000","last_undegraded":"2024-09-10T16:10:20.850377+0000","last_fullsized":"2024-09-10T16:10:20.850377+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18724,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","0","1"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20838","reported_seq":21000,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:10:20.854744+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:10:20.854744+0000","last_peered":"2024-09-10T16:10:20.854744+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:10:20.854744+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17469,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.851195+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:10:20.851195+0000","last_peered":"2024-09-10T16:10:20.851195+0000","last_clean":"2024-09-10T16:10:20.851195+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:10:20.851195+0000","last_undegraded":"2024-09-10T16:10:20.851195+0000","last_fullsized":"2024-09-10T16:10:20.851195+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20598","reported_seq":29191,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.851322+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:10:20.851322+0000","last_peered":"2024-09-10T16:10:20.851322+0000","last_clean":"2024-09-10T16:10:20.851322+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:10:20.851322+0000","last_undegraded":"2024-09-10T16:10:20.851322+0000","last_fullsized":"2024-09-10T16:10:20.851322+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16988,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":121,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.850614+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:10:20.850614+0000","last_peered":"2024-09-10T16:10:20.850614+0000","last_clean":"2024-09-10T16:10:20.850614+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:10:20.850614+0000","last_undegraded":"2024-09-10T16:10:20.850614+0000","last_fullsized":"2024-09-10T16:10:20.850614+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'21048","reported_seq":21217,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:10:20.855031+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:10:20.855031+0000","last_peered":"2024-09-10T16:10:20.855031+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:10:20.855031+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17303,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.851276+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:10:20.851276+0000","last_peered":"2024-09-10T16:10:20.851276+0000","last_clean":"2024-09-10T16:10:20.851276+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:10:20.851276+0000","last_undegraded":"2024-09-10T16:10:20.851276+0000","last_fullsized":"2024-09-10T16:10:20.851276+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":102,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.851340+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:10:20.851340+0000","last_peered":"2024-09-10T16:10:20.851340+0000","last_clean":"2024-09-10T16:10:20.851340+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:10:20.851340+0000","last_undegraded":"2024-09-10T16:10:20.851340+0000","last_fullsized":"2024-09-10T16:10:20.851340+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'21198","reported_seq":21370,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:10:20.854530+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:10:20.854530+0000","last_peered":"2024-09-10T16:10:20.854530+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:10:20.854530+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17361,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4321},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":21470,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292096,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29920,"num_bytes_recovered":33097129,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":210509824,"data_stored":210394208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161022,"ondisk_log_size":161022,"up":32,"acting":27,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986324,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":502652,"kb_used_data":137768,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93869188,"statfs":{"total":96636764160,"available":96122048512,"internally_reserved":0,"allocated":141074432,"data_stored":140728176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659270,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":483412,"kb_used_data":516,"kb_used_omap":4,"kb_used_meta":482875,"kb_avail":93888428,"statfs":{"total":96636764160,"available":96141750272,"internally_reserved":0,"allocated":528384,"data_stored":165464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":494464686},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182552,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":547364,"kb_used_data":68684,"kb_used_omap":4,"kb_used_meta":478651,"kb_avail":93824476,"statfs":{"total":96636764160,"available":96076263424,"internally_reserved":0,"allocated":70332416,"data_stored":69976722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":490139306},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":69853184,"data_stored":69817054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":5796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":140607488,"data_stored":140571358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:10:27.720 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:10:28.022 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:10:28.023 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:10:28.035 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":446,"stamp":"2024-09-10T16:10:27.783746+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":20956,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292098,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30436,"num_bytes_recovered":33097148,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161025,"ondisk_log_size":161025,"up":48,"acting":43,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1533440,"kb_used_data":206976,"kb_used_omap":12,"kb_used_meta":1326387,"kb_avail":281582080,"statfs":{"total":289910292480,"available":288340049920,"internally_reserved":0,"allocated":211943424,"data_stored":210873193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1358220290},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1486,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1486,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001411"},"pg_stats":[{"pgid":"2.f","version":"49'20672","reported_seq":29300,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.856536+0000","last_change":"2024-09-10T16:09:57.088531+0000","last_active":"2024-09-10T16:10:20.856536+0000","last_peered":"2024-09-10T16:10:20.856536+0000","last_clean":"2024-09-10T16:10:20.856536+0000","last_became_active":"2024-09-10T16:09:57.005755+0000","last_became_peered":"2024-09-10T16:09:57.005755+0000","last_unstale":"2024-09-10T16:10:20.856536+0000","last_undegraded":"2024-09-10T16:10:20.856536+0000","last_fullsized":"2024-09-10T16:10:20.856536+0000","mapping_epoch":54,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:09:19.329697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20672,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4259,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20844","reported_seq":20984,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.850591+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:10:20.850591+0000","last_peered":"2024-09-10T16:10:20.850591+0000","last_clean":"2024-09-10T16:10:20.850591+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:10:20.850591+0000","last_undegraded":"2024-09-10T16:10:20.850591+0000","last_fullsized":"2024-09-10T16:10:20.850591+0000","mapping_epoch":35,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18949,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20798","reported_seq":29503,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.851218+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:10:20.851218+0000","last_peered":"2024-09-10T16:10:20.851218+0000","last_clean":"2024-09-10T16:10:20.851218+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:10:20.851218+0000","last_undegraded":"2024-09-10T16:10:20.851218+0000","last_fullsized":"2024-09-10T16:10:20.851218+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18802,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20660","reported_seq":20768,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.850554+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:10:20.850554+0000","last_peered":"2024-09-10T16:10:20.850554+0000","last_clean":"2024-09-10T16:10:20.850554+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:10:20.850554+0000","last_undegraded":"2024-09-10T16:10:20.850554+0000","last_fullsized":"2024-09-10T16:10:20.850554+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20660,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20891","reported_seq":21048,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.854539+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:10:20.854539+0000","last_peered":"2024-09-10T16:10:20.854539+0000","last_clean":"2024-09-10T16:10:20.854539+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:10:20.854539+0000","last_undegraded":"2024-09-10T16:10:20.854539+0000","last_fullsized":"2024-09-10T16:10:20.854539+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17602,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'21061","reported_seq":22539,"reported_epoch":57,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:10:26.212402+0000","last_change":"2024-09-10T16:10:19.743410+0000","last_active":"2024-09-10T16:10:26.212402+0000","last_peered":"2024-09-10T16:10:26.212402+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:10:26.212402+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3716,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17673,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":656,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3715}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'21082","reported_seq":21247,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:10:20.854635+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:10:20.854635+0000","last_peered":"2024-09-10T16:10:20.854635+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:10:20.854635+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17581,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4235},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'21314","reported_seq":21471,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.854391+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:10:20.854391+0000","last_peered":"2024-09-10T16:10:20.854391+0000","last_clean":"2024-09-10T16:10:20.854391+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:10:20.854391+0000","last_undegraded":"2024-09-10T16:10:20.854391+0000","last_fullsized":"2024-09-10T16:10:20.854391+0000","mapping_epoch":34,"log_start":"45'11300","ondisk_log_start":"45'11300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18161,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.850980+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:10:20.850980+0000","last_peered":"2024-09-10T16:10:20.850980+0000","last_clean":"2024-09-10T16:10:20.850980+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:10:20.850980+0000","last_undegraded":"2024-09-10T16:10:20.850980+0000","last_fullsized":"2024-09-10T16:10:20.850980+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20457","reported_seq":20628,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.851257+0000","last_change":"2024-09-10T16:09:33.012840+0000","last_active":"2024-09-10T16:10:20.851257+0000","last_peered":"2024-09-10T16:10:20.851257+0000","last_clean":"2024-09-10T16:10:20.851257+0000","last_became_active":"2024-09-10T16:09:32.934056+0000","last_became_peered":"2024-09-10T16:09:32.934056+0000","last_unstale":"2024-09-10T16:10:20.851257+0000","last_undegraded":"2024-09-10T16:10:20.851257+0000","last_fullsized":"2024-09-10T16:10:20.851257+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:08:30.481235+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18276,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":4579,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":121,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.850337+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:10:20.850337+0000","last_peered":"2024-09-10T16:10:20.850337+0000","last_clean":"2024-09-10T16:10:20.850337+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:10:20.850337+0000","last_undegraded":"2024-09-10T16:10:20.850337+0000","last_fullsized":"2024-09-10T16:10:20.850337+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20910","reported_seq":29590,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.854908+0000","last_change":"2024-09-10T16:09:08.909580+0000","last_active":"2024-09-10T16:10:20.854908+0000","last_peered":"2024-09-10T16:10:20.854908+0000","last_clean":"2024-09-10T16:10:20.854908+0000","last_became_active":"2024-09-10T16:09:08.854941+0000","last_became_peered":"2024-09-10T16:09:08.854941+0000","last_unstale":"2024-09-10T16:10:20.854908+0000","last_undegraded":"2024-09-10T16:10:20.854908+0000","last_fullsized":"2024-09-10T16:10:20.854908+0000","mapping_epoch":50,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:13:14.846691+0000","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17414,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4257,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":131,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.854759+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:10:20.854759+0000","last_peered":"2024-09-10T16:10:20.854759+0000","last_clean":"2024-09-10T16:10:20.854759+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:10:20.854759+0000","last_undegraded":"2024-09-10T16:10:20.854759+0000","last_fullsized":"2024-09-10T16:10:20.854759+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20781","reported_seq":29561,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.907987+0000","last_change":"2024-09-10T16:10:20.907889+0000","last_active":"2024-09-10T16:10:20.907987+0000","last_peered":"2024-09-10T16:10:20.907987+0000","last_clean":"2024-09-10T16:10:20.907987+0000","last_became_active":"2024-09-10T16:10:20.852065+0000","last_became_peered":"2024-09-10T16:10:20.852065+0000","last_unstale":"2024-09-10T16:10:20.907987+0000","last_undegraded":"2024-09-10T16:10:20.907987+0000","last_fullsized":"2024-09-10T16:10:20.907987+0000","mapping_epoch":56,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:34.069549+0000","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18461,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4307,"num_bytes_recovered":8675,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":107,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.851171+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:10:20.851171+0000","last_peered":"2024-09-10T16:10:20.851171+0000","last_clean":"2024-09-10T16:10:20.851171+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:10:20.851171+0000","last_undegraded":"2024-09-10T16:10:20.851171+0000","last_fullsized":"2024-09-10T16:10:20.851171+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'21170","reported_seq":30042,"reported_epoch":57,"state":"active+recovery_wait","last_fresh":"2024-09-10T16:10:20.850377+0000","last_change":"2024-09-10T16:08:10.929169+0000","last_active":"2024-09-10T16:10:20.850377+0000","last_peered":"2024-09-10T16:10:20.850377+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T16:08:10.860438+0000","last_became_peered":"2024-09-10T16:08:10.860438+0000","last_unstale":"2024-09-10T16:10:20.850377+0000","last_undegraded":"2024-09-10T16:10:20.850377+0000","last_fullsized":"2024-09-10T16:10:20.850377+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18724,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","0","1"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20838","reported_seq":21000,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:10:20.854744+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:10:20.854744+0000","last_peered":"2024-09-10T16:10:20.854744+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:10:20.854744+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17469,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.851195+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:10:20.851195+0000","last_peered":"2024-09-10T16:10:20.851195+0000","last_clean":"2024-09-10T16:10:20.851195+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:10:20.851195+0000","last_undegraded":"2024-09-10T16:10:20.851195+0000","last_fullsized":"2024-09-10T16:10:20.851195+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20598","reported_seq":29191,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.851322+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:10:20.851322+0000","last_peered":"2024-09-10T16:10:20.851322+0000","last_clean":"2024-09-10T16:10:20.851322+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:10:20.851322+0000","last_undegraded":"2024-09-10T16:10:20.851322+0000","last_fullsized":"2024-09-10T16:10:20.851322+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16988,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":121,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.850614+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:10:20.850614+0000","last_peered":"2024-09-10T16:10:20.850614+0000","last_clean":"2024-09-10T16:10:20.850614+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:10:20.850614+0000","last_undegraded":"2024-09-10T16:10:20.850614+0000","last_fullsized":"2024-09-10T16:10:20.850614+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'21048","reported_seq":21217,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:10:20.855031+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:10:20.855031+0000","last_peered":"2024-09-10T16:10:20.855031+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:10:20.855031+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17303,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.851276+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:10:20.851276+0000","last_peered":"2024-09-10T16:10:20.851276+0000","last_clean":"2024-09-10T16:10:20.851276+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:10:20.851276+0000","last_undegraded":"2024-09-10T16:10:20.851276+0000","last_fullsized":"2024-09-10T16:10:20.851276+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":102,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.851340+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:10:20.851340+0000","last_peered":"2024-09-10T16:10:20.851340+0000","last_clean":"2024-09-10T16:10:20.851340+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:10:20.851340+0000","last_undegraded":"2024-09-10T16:10:20.851340+0000","last_fullsized":"2024-09-10T16:10:20.851340+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'21198","reported_seq":21370,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:10:20.854530+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:10:20.854530+0000","last_peered":"2024-09-10T16:10:20.854530+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:10:20.854530+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17361,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4321},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":20956,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292096,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30434,"num_bytes_recovered":33097129,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":210509824,"data_stored":210394208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161022,"ondisk_log_size":161022,"up":32,"acting":27,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986325,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":502664,"kb_used_data":137776,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93869176,"statfs":{"total":96636764160,"available":96122036224,"internally_reserved":0,"allocated":141082624,"data_stored":140731007,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659270,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":483412,"kb_used_data":516,"kb_used_omap":4,"kb_used_meta":482875,"kb_avail":93888428,"statfs":{"total":96636764160,"available":96141750272,"internally_reserved":0,"allocated":528384,"data_stored":165464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":494464686},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182553,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":547364,"kb_used_data":68684,"kb_used_omap":4,"kb_used_meta":478651,"kb_avail":93824476,"statfs":{"total":96636764160,"available":96076263424,"internally_reserved":0,"allocated":70332416,"data_stored":69976722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":490139306},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":69853184,"data_stored":69817054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":5796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":140607488,"data_stored":140571358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:10:28.036 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-10T16:10:28.036 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "49'21061", 'reported_seq': 22539, 'reported_epoch': 57, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:10:26.212402+0000', 'last_change': '2024-09-10T16:10:19.743410+0000', 'last_active': '2024-09-10T16:10:26.212402+0000', 'last_peered': '2024-09-10T16:10:26.212402+0000', 'last_clean': '2024-09-10T15:59:14.111551+0000', 'last_became_active': '2024-09-10T15:59:16.300040+0000', 'last_became_peered': '2024-09-10T15:59:16.300040+0000', 'last_unstale': '2024-09-10T16:10:26.212402+0000', 'last_undegraded': '2024-09-10T15:59:15.878234+0000', 'last_fullsized': '2024-09-10T15:59:15.769328+0000', 'mapping_epoch': 42, 'log_start': "45'11000", 'ondisk_log_start': "45'11000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3442", 'last_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'objects_scrubbed': 3384, 'log_size': 10061, 'log_dups_size': 0, 'ondisk_log_size': 10061, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1292, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3716, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 17673, 'num_write_kb': 28580, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 656, 'num_bytes_recovered': 483, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3715}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:10:28.036 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-10T16:10:28.037 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "49'21082", 'reported_seq': 21247, 'reported_epoch': 57, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:10:20.854635+0000', 'last_change': '2024-09-10T15:59:16.331899+0000', 'last_active': '2024-09-10T16:10:20.854635+0000', 'last_peered': '2024-09-10T16:10:20.854635+0000', 'last_clean': '2024-09-10T15:59:14.139436+0000', 'last_became_active': '2024-09-10T15:59:16.318715+0000', 'last_became_peered': '2024-09-10T15:59:16.318715+0000', 'last_unstale': '2024-09-10T16:10:20.854635+0000', 'last_undegraded': '2024-09-10T15:59:15.877524+0000', 'last_fullsized': '2024-09-10T15:59:15.769042+0000', 'mapping_epoch': 42, 'log_start': "45'11000", 'ondisk_log_start': "45'11000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3537", 'last_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'objects_scrubbed': 3497, 'log_size': 10082, 'log_dups_size': 0, 'ondisk_log_size': 10082, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1224, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4237, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 17581, 'num_write_kb': 28172, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4235}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:10:28.037 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-10T16:10:28.037 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "49'21170", 'reported_seq': 30042, 'reported_epoch': 57, 'state': 'active+recovery_wait', 'last_fresh': '2024-09-10T16:10:20.850377+0000', 'last_change': '2024-09-10T16:08:10.929169+0000', 'last_active': '2024-09-10T16:10:20.850377+0000', 'last_peered': '2024-09-10T16:10:20.850377+0000', 'last_clean': '2024-09-10T15:59:14.110654+0000', 'last_became_active': '2024-09-10T16:08:10.860438+0000', 'last_became_peered': '2024-09-10T16:08:10.860438+0000', 'last_unstale': '2024-09-10T16:10:20.850377+0000', 'last_undegraded': '2024-09-10T16:10:20.850377+0000', 'last_fullsized': '2024-09-10T16:10:20.850377+0000', 'mapping_epoch': 48, 'log_start': "45'11100", 'ondisk_log_start': "45'11100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2486", 'last_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'objects_scrubbed': 2440, 'log_size': 10070, 'log_dups_size': 0, 'ondisk_log_size': 10070, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 792, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18724, 'num_write_kb': 32568, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 4355, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': ['2', '0', '1'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:10:28.037 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-10T16:10:28.037 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "49'20838", 'reported_seq': 21000, 'reported_epoch': 57, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:10:20.854744+0000', 'last_change': '2024-09-10T15:59:16.315641+0000', 'last_active': '2024-09-10T16:10:20.854744+0000', 'last_peered': '2024-09-10T16:10:20.854744+0000', 'last_clean': '2024-09-10T15:59:14.117001+0000', 'last_became_active': '2024-09-10T15:59:16.302245+0000', 'last_became_peered': '2024-09-10T15:59:16.302245+0000', 'last_unstale': '2024-09-10T16:10:20.854744+0000', 'last_undegraded': '2024-09-10T15:59:15.845012+0000', 'last_fullsized': '2024-09-10T15:59:15.770337+0000', 'mapping_epoch': 42, 'log_start': "45'10800", 'ondisk_log_start': "45'10800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3404", 'last_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'objects_scrubbed': 3367, 'log_size': 10038, 'log_dups_size': 0, 'ondisk_log_size': 10038, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 983, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17469, 'num_write_kb': 28204, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4304}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:10:28.037 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-10T16:10:28.037 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "49'21048", 'reported_seq': 21217, 'reported_epoch': 57, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:10:20.855031+0000', 'last_change': '2024-09-10T15:59:16.311291+0000', 'last_active': '2024-09-10T16:10:20.855031+0000', 'last_peered': '2024-09-10T16:10:20.855031+0000', 'last_clean': '2024-09-10T15:59:14.116363+0000', 'last_became_active': '2024-09-10T15:59:16.301244+0000', 'last_became_peered': '2024-09-10T15:59:16.301244+0000', 'last_unstale': '2024-09-10T16:10:20.855031+0000', 'last_undegraded': '2024-09-10T15:59:15.883434+0000', 'last_fullsized': '2024-09-10T15:59:15.772756+0000', 'mapping_epoch': 42, 'log_start': "45'11000", 'ondisk_log_start': "45'11000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3789", 'last_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'objects_scrubbed': 3741, 'log_size': 10048, 'log_dups_size': 0, 'ondisk_log_size': 10048, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1440, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4373, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17303, 'num_write_kb': 27124, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4371}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:10:28.037 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-10T16:10:28.037 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "49'21198", 'reported_seq': 21370, 'reported_epoch': 57, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:10:20.854530+0000', 'last_change': '2024-09-10T15:59:16.313409+0000', 'last_active': '2024-09-10T16:10:20.854530+0000', 'last_peered': '2024-09-10T16:10:20.854530+0000', 'last_clean': '2024-09-10T15:59:14.129780+0000', 'last_became_active': '2024-09-10T15:59:16.300932+0000', 'last_became_peered': '2024-09-10T15:59:16.300932+0000', 'last_unstale': '2024-09-10T16:10:20.854530+0000', 'last_undegraded': '2024-09-10T15:59:15.887823+0000', 'last_fullsized': '2024-09-10T15:59:15.768513+0000', 'mapping_epoch': 42, 'log_start': "45'11100", 'ondisk_log_start': "45'11100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3880", 'last_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'objects_scrubbed': 3827, 'log_size': 10098, 'log_dups_size': 0, 'ondisk_log_size': 10098, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1397, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4325, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17361, 'num_write_kb': 27072, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4321}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:10:28.038 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T16:10:28.416 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:10:28.416 INFO:teuthology.orchestra.run.smithi115.stdout:{"fsid":"7effee29-df05-4351-9741-ee203564ed81","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 20956/24 objects degraded (87316.667%), 5 pgs degraded, 5 pgs undersized","count":10},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":792,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":57,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983954,"num_in_osds":3,"osd_in_since":1725983947,"num_remapped_pgs":5},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":18},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":4},{"state_name":"active+recovery_wait","count":1},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":12,"data_bytes":4849,"bytes_used":1570242560,"bytes_avail":288340049920,"bytes_total":289910292480,"degraded_objects":20956,"degraded_total":24,"degraded_ratio":873.16666666666652,"recovering_objects_per_sec":185,"recovering_bytes_per_sec":60,"recovering_keys_per_sec":0,"num_objects_recovered":1486,"num_bytes_recovered":483,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-10T15:57:14:967120+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T16:00:13.560776+0000","services":{}},"progress_events":{}} 2024-09-10T16:10:28.431 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-10T16:10:28.432 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:10:28.802 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:10:28.802 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:10:28.817 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":446,"stamp":"2024-09-10T16:10:27.783746+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":20956,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292098,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30436,"num_bytes_recovered":33097148,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161025,"ondisk_log_size":161025,"up":48,"acting":43,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1533440,"kb_used_data":206976,"kb_used_omap":12,"kb_used_meta":1326387,"kb_avail":281582080,"statfs":{"total":289910292480,"available":288340049920,"internally_reserved":0,"allocated":211943424,"data_stored":210873193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1358220290},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1486,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1486,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001411"},"pg_stats":[{"pgid":"2.f","version":"49'20672","reported_seq":29300,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.856536+0000","last_change":"2024-09-10T16:09:57.088531+0000","last_active":"2024-09-10T16:10:20.856536+0000","last_peered":"2024-09-10T16:10:20.856536+0000","last_clean":"2024-09-10T16:10:20.856536+0000","last_became_active":"2024-09-10T16:09:57.005755+0000","last_became_peered":"2024-09-10T16:09:57.005755+0000","last_unstale":"2024-09-10T16:10:20.856536+0000","last_undegraded":"2024-09-10T16:10:20.856536+0000","last_fullsized":"2024-09-10T16:10:20.856536+0000","mapping_epoch":54,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:09:19.329697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20672,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4259,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20844","reported_seq":20984,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.850591+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:10:20.850591+0000","last_peered":"2024-09-10T16:10:20.850591+0000","last_clean":"2024-09-10T16:10:20.850591+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:10:20.850591+0000","last_undegraded":"2024-09-10T16:10:20.850591+0000","last_fullsized":"2024-09-10T16:10:20.850591+0000","mapping_epoch":35,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18949,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20798","reported_seq":29503,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.851218+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:10:20.851218+0000","last_peered":"2024-09-10T16:10:20.851218+0000","last_clean":"2024-09-10T16:10:20.851218+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:10:20.851218+0000","last_undegraded":"2024-09-10T16:10:20.851218+0000","last_fullsized":"2024-09-10T16:10:20.851218+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18802,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20660","reported_seq":20768,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.850554+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:10:20.850554+0000","last_peered":"2024-09-10T16:10:20.850554+0000","last_clean":"2024-09-10T16:10:20.850554+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:10:20.850554+0000","last_undegraded":"2024-09-10T16:10:20.850554+0000","last_fullsized":"2024-09-10T16:10:20.850554+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20660,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20891","reported_seq":21048,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.854539+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:10:20.854539+0000","last_peered":"2024-09-10T16:10:20.854539+0000","last_clean":"2024-09-10T16:10:20.854539+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:10:20.854539+0000","last_undegraded":"2024-09-10T16:10:20.854539+0000","last_fullsized":"2024-09-10T16:10:20.854539+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17602,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'21061","reported_seq":22539,"reported_epoch":57,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:10:26.212402+0000","last_change":"2024-09-10T16:10:19.743410+0000","last_active":"2024-09-10T16:10:26.212402+0000","last_peered":"2024-09-10T16:10:26.212402+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:10:26.212402+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3716,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17673,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":656,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3715}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'21082","reported_seq":21247,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:10:20.854635+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:10:20.854635+0000","last_peered":"2024-09-10T16:10:20.854635+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:10:20.854635+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17581,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4235},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'21314","reported_seq":21471,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.854391+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:10:20.854391+0000","last_peered":"2024-09-10T16:10:20.854391+0000","last_clean":"2024-09-10T16:10:20.854391+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:10:20.854391+0000","last_undegraded":"2024-09-10T16:10:20.854391+0000","last_fullsized":"2024-09-10T16:10:20.854391+0000","mapping_epoch":34,"log_start":"45'11300","ondisk_log_start":"45'11300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18161,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.850980+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:10:20.850980+0000","last_peered":"2024-09-10T16:10:20.850980+0000","last_clean":"2024-09-10T16:10:20.850980+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:10:20.850980+0000","last_undegraded":"2024-09-10T16:10:20.850980+0000","last_fullsized":"2024-09-10T16:10:20.850980+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20457","reported_seq":20628,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.851257+0000","last_change":"2024-09-10T16:09:33.012840+0000","last_active":"2024-09-10T16:10:20.851257+0000","last_peered":"2024-09-10T16:10:20.851257+0000","last_clean":"2024-09-10T16:10:20.851257+0000","last_became_active":"2024-09-10T16:09:32.934056+0000","last_became_peered":"2024-09-10T16:09:32.934056+0000","last_unstale":"2024-09-10T16:10:20.851257+0000","last_undegraded":"2024-09-10T16:10:20.851257+0000","last_fullsized":"2024-09-10T16:10:20.851257+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:08:30.481235+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18276,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":4579,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":121,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.850337+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:10:20.850337+0000","last_peered":"2024-09-10T16:10:20.850337+0000","last_clean":"2024-09-10T16:10:20.850337+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:10:20.850337+0000","last_undegraded":"2024-09-10T16:10:20.850337+0000","last_fullsized":"2024-09-10T16:10:20.850337+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20910","reported_seq":29590,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.854908+0000","last_change":"2024-09-10T16:09:08.909580+0000","last_active":"2024-09-10T16:10:20.854908+0000","last_peered":"2024-09-10T16:10:20.854908+0000","last_clean":"2024-09-10T16:10:20.854908+0000","last_became_active":"2024-09-10T16:09:08.854941+0000","last_became_peered":"2024-09-10T16:09:08.854941+0000","last_unstale":"2024-09-10T16:10:20.854908+0000","last_undegraded":"2024-09-10T16:10:20.854908+0000","last_fullsized":"2024-09-10T16:10:20.854908+0000","mapping_epoch":50,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:13:14.846691+0000","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17414,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4257,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":131,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.854759+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:10:20.854759+0000","last_peered":"2024-09-10T16:10:20.854759+0000","last_clean":"2024-09-10T16:10:20.854759+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:10:20.854759+0000","last_undegraded":"2024-09-10T16:10:20.854759+0000","last_fullsized":"2024-09-10T16:10:20.854759+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20781","reported_seq":29561,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.907987+0000","last_change":"2024-09-10T16:10:20.907889+0000","last_active":"2024-09-10T16:10:20.907987+0000","last_peered":"2024-09-10T16:10:20.907987+0000","last_clean":"2024-09-10T16:10:20.907987+0000","last_became_active":"2024-09-10T16:10:20.852065+0000","last_became_peered":"2024-09-10T16:10:20.852065+0000","last_unstale":"2024-09-10T16:10:20.907987+0000","last_undegraded":"2024-09-10T16:10:20.907987+0000","last_fullsized":"2024-09-10T16:10:20.907987+0000","mapping_epoch":56,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:34.069549+0000","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18461,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4307,"num_bytes_recovered":8675,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":107,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.851171+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:10:20.851171+0000","last_peered":"2024-09-10T16:10:20.851171+0000","last_clean":"2024-09-10T16:10:20.851171+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:10:20.851171+0000","last_undegraded":"2024-09-10T16:10:20.851171+0000","last_fullsized":"2024-09-10T16:10:20.851171+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'21170","reported_seq":30042,"reported_epoch":57,"state":"active+recovery_wait","last_fresh":"2024-09-10T16:10:20.850377+0000","last_change":"2024-09-10T16:08:10.929169+0000","last_active":"2024-09-10T16:10:20.850377+0000","last_peered":"2024-09-10T16:10:20.850377+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T16:08:10.860438+0000","last_became_peered":"2024-09-10T16:08:10.860438+0000","last_unstale":"2024-09-10T16:10:20.850377+0000","last_undegraded":"2024-09-10T16:10:20.850377+0000","last_fullsized":"2024-09-10T16:10:20.850377+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18724,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","0","1"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20838","reported_seq":21000,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:10:20.854744+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:10:20.854744+0000","last_peered":"2024-09-10T16:10:20.854744+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:10:20.854744+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17469,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.851195+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:10:20.851195+0000","last_peered":"2024-09-10T16:10:20.851195+0000","last_clean":"2024-09-10T16:10:20.851195+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:10:20.851195+0000","last_undegraded":"2024-09-10T16:10:20.851195+0000","last_fullsized":"2024-09-10T16:10:20.851195+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20598","reported_seq":29191,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.851322+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:10:20.851322+0000","last_peered":"2024-09-10T16:10:20.851322+0000","last_clean":"2024-09-10T16:10:20.851322+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:10:20.851322+0000","last_undegraded":"2024-09-10T16:10:20.851322+0000","last_fullsized":"2024-09-10T16:10:20.851322+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16988,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":121,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.850614+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:10:20.850614+0000","last_peered":"2024-09-10T16:10:20.850614+0000","last_clean":"2024-09-10T16:10:20.850614+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:10:20.850614+0000","last_undegraded":"2024-09-10T16:10:20.850614+0000","last_fullsized":"2024-09-10T16:10:20.850614+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'21048","reported_seq":21217,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:10:20.855031+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:10:20.855031+0000","last_peered":"2024-09-10T16:10:20.855031+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:10:20.855031+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17303,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.851276+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:10:20.851276+0000","last_peered":"2024-09-10T16:10:20.851276+0000","last_clean":"2024-09-10T16:10:20.851276+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:10:20.851276+0000","last_undegraded":"2024-09-10T16:10:20.851276+0000","last_fullsized":"2024-09-10T16:10:20.851276+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":102,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.851340+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:10:20.851340+0000","last_peered":"2024-09-10T16:10:20.851340+0000","last_clean":"2024-09-10T16:10:20.851340+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:10:20.851340+0000","last_undegraded":"2024-09-10T16:10:20.851340+0000","last_fullsized":"2024-09-10T16:10:20.851340+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'21198","reported_seq":21370,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:10:20.854530+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:10:20.854530+0000","last_peered":"2024-09-10T16:10:20.854530+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:10:20.854530+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17361,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4321},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":20956,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292096,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30434,"num_bytes_recovered":33097129,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":210509824,"data_stored":210394208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161022,"ondisk_log_size":161022,"up":32,"acting":27,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986325,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":502664,"kb_used_data":137776,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93869176,"statfs":{"total":96636764160,"available":96122036224,"internally_reserved":0,"allocated":141082624,"data_stored":140731007,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659270,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":483412,"kb_used_data":516,"kb_used_omap":4,"kb_used_meta":482875,"kb_avail":93888428,"statfs":{"total":96636764160,"available":96141750272,"internally_reserved":0,"allocated":528384,"data_stored":165464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":494464686},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182553,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":547364,"kb_used_data":68684,"kb_used_omap":4,"kb_used_meta":478651,"kb_avail":93824476,"statfs":{"total":96636764160,"available":96076263424,"internally_reserved":0,"allocated":70332416,"data_stored":69976722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":490139306},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":69853184,"data_stored":69817054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":5796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":140607488,"data_stored":140571358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:10:31.819 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:10:32.163 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:10:32.163 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:10:32.178 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":448,"stamp":"2024-09-10T16:10:31.784474+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":20402,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292098,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30990,"num_bytes_recovered":33097148,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161025,"ondisk_log_size":161025,"up":48,"acting":43,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1537600,"kb_used_data":206976,"kb_used_omap":12,"kb_used_meta":1330547,"kb_avail":281577920,"statfs":{"total":289910292480,"available":288335790080,"internally_reserved":0,"allocated":211943424,"data_stored":210873193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1362480130},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2040,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2040,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.935117"},"pg_stats":[{"pgid":"2.f","version":"49'20672","reported_seq":29300,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.856536+0000","last_change":"2024-09-10T16:09:57.088531+0000","last_active":"2024-09-10T16:10:20.856536+0000","last_peered":"2024-09-10T16:10:20.856536+0000","last_clean":"2024-09-10T16:10:20.856536+0000","last_became_active":"2024-09-10T16:09:57.005755+0000","last_became_peered":"2024-09-10T16:09:57.005755+0000","last_unstale":"2024-09-10T16:10:20.856536+0000","last_undegraded":"2024-09-10T16:10:20.856536+0000","last_fullsized":"2024-09-10T16:10:20.856536+0000","mapping_epoch":54,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:09:19.329697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20672,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4259,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20844","reported_seq":20984,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.850591+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:10:20.850591+0000","last_peered":"2024-09-10T16:10:20.850591+0000","last_clean":"2024-09-10T16:10:20.850591+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:10:20.850591+0000","last_undegraded":"2024-09-10T16:10:20.850591+0000","last_fullsized":"2024-09-10T16:10:20.850591+0000","mapping_epoch":35,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18949,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20798","reported_seq":29503,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.851218+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:10:20.851218+0000","last_peered":"2024-09-10T16:10:20.851218+0000","last_clean":"2024-09-10T16:10:20.851218+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:10:20.851218+0000","last_undegraded":"2024-09-10T16:10:20.851218+0000","last_fullsized":"2024-09-10T16:10:20.851218+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18802,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20660","reported_seq":20768,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.850554+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:10:20.850554+0000","last_peered":"2024-09-10T16:10:20.850554+0000","last_clean":"2024-09-10T16:10:20.850554+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:10:20.850554+0000","last_undegraded":"2024-09-10T16:10:20.850554+0000","last_fullsized":"2024-09-10T16:10:20.850554+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20660,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20891","reported_seq":21048,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.854539+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:10:20.854539+0000","last_peered":"2024-09-10T16:10:20.854539+0000","last_clean":"2024-09-10T16:10:20.854539+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:10:20.854539+0000","last_undegraded":"2024-09-10T16:10:20.854539+0000","last_fullsized":"2024-09-10T16:10:20.854539+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17602,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'21061","reported_seq":23647,"reported_epoch":57,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:10:31.215480+0000","last_change":"2024-09-10T16:10:19.743410+0000","last_active":"2024-09-10T16:10:31.215480+0000","last_peered":"2024-09-10T16:10:31.215480+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:10:31.215480+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3162,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17673,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1210,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3161}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'21082","reported_seq":21247,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:10:20.854635+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:10:20.854635+0000","last_peered":"2024-09-10T16:10:20.854635+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:10:20.854635+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17581,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4235},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'21314","reported_seq":21471,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.854391+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:10:20.854391+0000","last_peered":"2024-09-10T16:10:20.854391+0000","last_clean":"2024-09-10T16:10:20.854391+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:10:20.854391+0000","last_undegraded":"2024-09-10T16:10:20.854391+0000","last_fullsized":"2024-09-10T16:10:20.854391+0000","mapping_epoch":34,"log_start":"45'11300","ondisk_log_start":"45'11300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18161,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.850980+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:10:20.850980+0000","last_peered":"2024-09-10T16:10:20.850980+0000","last_clean":"2024-09-10T16:10:20.850980+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:10:20.850980+0000","last_undegraded":"2024-09-10T16:10:20.850980+0000","last_fullsized":"2024-09-10T16:10:20.850980+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20457","reported_seq":20628,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.851257+0000","last_change":"2024-09-10T16:09:33.012840+0000","last_active":"2024-09-10T16:10:20.851257+0000","last_peered":"2024-09-10T16:10:20.851257+0000","last_clean":"2024-09-10T16:10:20.851257+0000","last_became_active":"2024-09-10T16:09:32.934056+0000","last_became_peered":"2024-09-10T16:09:32.934056+0000","last_unstale":"2024-09-10T16:10:20.851257+0000","last_undegraded":"2024-09-10T16:10:20.851257+0000","last_fullsized":"2024-09-10T16:10:20.851257+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:08:30.481235+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18276,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":4579,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":121,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.850337+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:10:20.850337+0000","last_peered":"2024-09-10T16:10:20.850337+0000","last_clean":"2024-09-10T16:10:20.850337+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:10:20.850337+0000","last_undegraded":"2024-09-10T16:10:20.850337+0000","last_fullsized":"2024-09-10T16:10:20.850337+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20910","reported_seq":29590,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.854908+0000","last_change":"2024-09-10T16:09:08.909580+0000","last_active":"2024-09-10T16:10:20.854908+0000","last_peered":"2024-09-10T16:10:20.854908+0000","last_clean":"2024-09-10T16:10:20.854908+0000","last_became_active":"2024-09-10T16:09:08.854941+0000","last_became_peered":"2024-09-10T16:09:08.854941+0000","last_unstale":"2024-09-10T16:10:20.854908+0000","last_undegraded":"2024-09-10T16:10:20.854908+0000","last_fullsized":"2024-09-10T16:10:20.854908+0000","mapping_epoch":50,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:13:14.846691+0000","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17414,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4257,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":131,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.854759+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:10:20.854759+0000","last_peered":"2024-09-10T16:10:20.854759+0000","last_clean":"2024-09-10T16:10:20.854759+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:10:20.854759+0000","last_undegraded":"2024-09-10T16:10:20.854759+0000","last_fullsized":"2024-09-10T16:10:20.854759+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20781","reported_seq":29561,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.907987+0000","last_change":"2024-09-10T16:10:20.907889+0000","last_active":"2024-09-10T16:10:20.907987+0000","last_peered":"2024-09-10T16:10:20.907987+0000","last_clean":"2024-09-10T16:10:20.907987+0000","last_became_active":"2024-09-10T16:10:20.852065+0000","last_became_peered":"2024-09-10T16:10:20.852065+0000","last_unstale":"2024-09-10T16:10:20.907987+0000","last_undegraded":"2024-09-10T16:10:20.907987+0000","last_fullsized":"2024-09-10T16:10:20.907987+0000","mapping_epoch":56,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:34.069549+0000","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18461,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4307,"num_bytes_recovered":8675,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":107,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.851171+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:10:20.851171+0000","last_peered":"2024-09-10T16:10:20.851171+0000","last_clean":"2024-09-10T16:10:20.851171+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:10:20.851171+0000","last_undegraded":"2024-09-10T16:10:20.851171+0000","last_fullsized":"2024-09-10T16:10:20.851171+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'21170","reported_seq":30042,"reported_epoch":57,"state":"active+recovery_wait","last_fresh":"2024-09-10T16:10:20.850377+0000","last_change":"2024-09-10T16:08:10.929169+0000","last_active":"2024-09-10T16:10:20.850377+0000","last_peered":"2024-09-10T16:10:20.850377+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T16:08:10.860438+0000","last_became_peered":"2024-09-10T16:08:10.860438+0000","last_unstale":"2024-09-10T16:10:20.850377+0000","last_undegraded":"2024-09-10T16:10:20.850377+0000","last_fullsized":"2024-09-10T16:10:20.850377+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18724,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","0","1"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20838","reported_seq":21000,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:10:20.854744+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:10:20.854744+0000","last_peered":"2024-09-10T16:10:20.854744+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:10:20.854744+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17469,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.851195+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:10:20.851195+0000","last_peered":"2024-09-10T16:10:20.851195+0000","last_clean":"2024-09-10T16:10:20.851195+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:10:20.851195+0000","last_undegraded":"2024-09-10T16:10:20.851195+0000","last_fullsized":"2024-09-10T16:10:20.851195+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20598","reported_seq":29191,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.851322+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:10:20.851322+0000","last_peered":"2024-09-10T16:10:20.851322+0000","last_clean":"2024-09-10T16:10:20.851322+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:10:20.851322+0000","last_undegraded":"2024-09-10T16:10:20.851322+0000","last_fullsized":"2024-09-10T16:10:20.851322+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16988,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":121,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.850614+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:10:20.850614+0000","last_peered":"2024-09-10T16:10:20.850614+0000","last_clean":"2024-09-10T16:10:20.850614+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:10:20.850614+0000","last_undegraded":"2024-09-10T16:10:20.850614+0000","last_fullsized":"2024-09-10T16:10:20.850614+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'21048","reported_seq":21217,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:10:20.855031+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:10:20.855031+0000","last_peered":"2024-09-10T16:10:20.855031+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:10:20.855031+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17303,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.851276+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:10:20.851276+0000","last_peered":"2024-09-10T16:10:20.851276+0000","last_clean":"2024-09-10T16:10:20.851276+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:10:20.851276+0000","last_undegraded":"2024-09-10T16:10:20.851276+0000","last_fullsized":"2024-09-10T16:10:20.851276+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":102,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.851340+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:10:20.851340+0000","last_peered":"2024-09-10T16:10:20.851340+0000","last_clean":"2024-09-10T16:10:20.851340+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:10:20.851340+0000","last_undegraded":"2024-09-10T16:10:20.851340+0000","last_fullsized":"2024-09-10T16:10:20.851340+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'21198","reported_seq":21370,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:10:20.854530+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:10:20.854530+0000","last_peered":"2024-09-10T16:10:20.854530+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:10:20.854530+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17361,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4321},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":20402,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292096,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30988,"num_bytes_recovered":33097129,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":210509824,"data_stored":210394208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161022,"ondisk_log_size":161022,"up":32,"acting":27,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986326,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":502664,"kb_used_data":137776,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93869176,"statfs":{"total":96636764160,"available":96122036224,"internally_reserved":0,"allocated":141082624,"data_stored":140731007,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659271,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487572,"kb_used_data":516,"kb_used_omap":4,"kb_used_meta":487035,"kb_avail":93884268,"statfs":{"total":96636764160,"available":96137490432,"internally_reserved":0,"allocated":528384,"data_stored":165464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":498724526},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182554,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":547364,"kb_used_data":68684,"kb_used_omap":4,"kb_used_meta":478651,"kb_avail":93824476,"statfs":{"total":96636764160,"available":96076263424,"internally_reserved":0,"allocated":70332416,"data_stored":69976722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":490139306},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":69853184,"data_stored":69817054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":5796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":140607488,"data_stored":140571358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:10:32.179 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:10:32.504 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:10:32.504 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:10:32.517 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":448,"stamp":"2024-09-10T16:10:31.784474+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":20402,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292098,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30990,"num_bytes_recovered":33097148,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161025,"ondisk_log_size":161025,"up":48,"acting":43,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1537600,"kb_used_data":206976,"kb_used_omap":12,"kb_used_meta":1330547,"kb_avail":281577920,"statfs":{"total":289910292480,"available":288335790080,"internally_reserved":0,"allocated":211943424,"data_stored":210873193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1362480130},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2040,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2040,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.935117"},"pg_stats":[{"pgid":"2.f","version":"49'20672","reported_seq":29300,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.856536+0000","last_change":"2024-09-10T16:09:57.088531+0000","last_active":"2024-09-10T16:10:20.856536+0000","last_peered":"2024-09-10T16:10:20.856536+0000","last_clean":"2024-09-10T16:10:20.856536+0000","last_became_active":"2024-09-10T16:09:57.005755+0000","last_became_peered":"2024-09-10T16:09:57.005755+0000","last_unstale":"2024-09-10T16:10:20.856536+0000","last_undegraded":"2024-09-10T16:10:20.856536+0000","last_fullsized":"2024-09-10T16:10:20.856536+0000","mapping_epoch":54,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:09:19.329697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20672,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4259,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20844","reported_seq":20984,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.850591+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:10:20.850591+0000","last_peered":"2024-09-10T16:10:20.850591+0000","last_clean":"2024-09-10T16:10:20.850591+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:10:20.850591+0000","last_undegraded":"2024-09-10T16:10:20.850591+0000","last_fullsized":"2024-09-10T16:10:20.850591+0000","mapping_epoch":35,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18949,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20798","reported_seq":29503,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.851218+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:10:20.851218+0000","last_peered":"2024-09-10T16:10:20.851218+0000","last_clean":"2024-09-10T16:10:20.851218+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:10:20.851218+0000","last_undegraded":"2024-09-10T16:10:20.851218+0000","last_fullsized":"2024-09-10T16:10:20.851218+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18802,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20660","reported_seq":20768,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.850554+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:10:20.850554+0000","last_peered":"2024-09-10T16:10:20.850554+0000","last_clean":"2024-09-10T16:10:20.850554+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:10:20.850554+0000","last_undegraded":"2024-09-10T16:10:20.850554+0000","last_fullsized":"2024-09-10T16:10:20.850554+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20660,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20891","reported_seq":21048,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.854539+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:10:20.854539+0000","last_peered":"2024-09-10T16:10:20.854539+0000","last_clean":"2024-09-10T16:10:20.854539+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:10:20.854539+0000","last_undegraded":"2024-09-10T16:10:20.854539+0000","last_fullsized":"2024-09-10T16:10:20.854539+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17602,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'21061","reported_seq":23647,"reported_epoch":57,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:10:31.215480+0000","last_change":"2024-09-10T16:10:19.743410+0000","last_active":"2024-09-10T16:10:31.215480+0000","last_peered":"2024-09-10T16:10:31.215480+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:10:31.215480+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3162,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17673,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1210,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3161}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'21082","reported_seq":21247,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:10:20.854635+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:10:20.854635+0000","last_peered":"2024-09-10T16:10:20.854635+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:10:20.854635+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17581,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4235},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'21314","reported_seq":21471,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.854391+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:10:20.854391+0000","last_peered":"2024-09-10T16:10:20.854391+0000","last_clean":"2024-09-10T16:10:20.854391+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:10:20.854391+0000","last_undegraded":"2024-09-10T16:10:20.854391+0000","last_fullsized":"2024-09-10T16:10:20.854391+0000","mapping_epoch":34,"log_start":"45'11300","ondisk_log_start":"45'11300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18161,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.850980+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:10:20.850980+0000","last_peered":"2024-09-10T16:10:20.850980+0000","last_clean":"2024-09-10T16:10:20.850980+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:10:20.850980+0000","last_undegraded":"2024-09-10T16:10:20.850980+0000","last_fullsized":"2024-09-10T16:10:20.850980+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20457","reported_seq":20628,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.851257+0000","last_change":"2024-09-10T16:09:33.012840+0000","last_active":"2024-09-10T16:10:20.851257+0000","last_peered":"2024-09-10T16:10:20.851257+0000","last_clean":"2024-09-10T16:10:20.851257+0000","last_became_active":"2024-09-10T16:09:32.934056+0000","last_became_peered":"2024-09-10T16:09:32.934056+0000","last_unstale":"2024-09-10T16:10:20.851257+0000","last_undegraded":"2024-09-10T16:10:20.851257+0000","last_fullsized":"2024-09-10T16:10:20.851257+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:08:30.481235+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18276,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":4579,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":121,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.850337+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:10:20.850337+0000","last_peered":"2024-09-10T16:10:20.850337+0000","last_clean":"2024-09-10T16:10:20.850337+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:10:20.850337+0000","last_undegraded":"2024-09-10T16:10:20.850337+0000","last_fullsized":"2024-09-10T16:10:20.850337+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20910","reported_seq":29590,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.854908+0000","last_change":"2024-09-10T16:09:08.909580+0000","last_active":"2024-09-10T16:10:20.854908+0000","last_peered":"2024-09-10T16:10:20.854908+0000","last_clean":"2024-09-10T16:10:20.854908+0000","last_became_active":"2024-09-10T16:09:08.854941+0000","last_became_peered":"2024-09-10T16:09:08.854941+0000","last_unstale":"2024-09-10T16:10:20.854908+0000","last_undegraded":"2024-09-10T16:10:20.854908+0000","last_fullsized":"2024-09-10T16:10:20.854908+0000","mapping_epoch":50,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:13:14.846691+0000","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17414,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4257,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":131,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.854759+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:10:20.854759+0000","last_peered":"2024-09-10T16:10:20.854759+0000","last_clean":"2024-09-10T16:10:20.854759+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:10:20.854759+0000","last_undegraded":"2024-09-10T16:10:20.854759+0000","last_fullsized":"2024-09-10T16:10:20.854759+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20781","reported_seq":29561,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.907987+0000","last_change":"2024-09-10T16:10:20.907889+0000","last_active":"2024-09-10T16:10:20.907987+0000","last_peered":"2024-09-10T16:10:20.907987+0000","last_clean":"2024-09-10T16:10:20.907987+0000","last_became_active":"2024-09-10T16:10:20.852065+0000","last_became_peered":"2024-09-10T16:10:20.852065+0000","last_unstale":"2024-09-10T16:10:20.907987+0000","last_undegraded":"2024-09-10T16:10:20.907987+0000","last_fullsized":"2024-09-10T16:10:20.907987+0000","mapping_epoch":56,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:34.069549+0000","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18461,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4307,"num_bytes_recovered":8675,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":107,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.851171+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:10:20.851171+0000","last_peered":"2024-09-10T16:10:20.851171+0000","last_clean":"2024-09-10T16:10:20.851171+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:10:20.851171+0000","last_undegraded":"2024-09-10T16:10:20.851171+0000","last_fullsized":"2024-09-10T16:10:20.851171+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'21170","reported_seq":30042,"reported_epoch":57,"state":"active+recovery_wait","last_fresh":"2024-09-10T16:10:20.850377+0000","last_change":"2024-09-10T16:08:10.929169+0000","last_active":"2024-09-10T16:10:20.850377+0000","last_peered":"2024-09-10T16:10:20.850377+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T16:08:10.860438+0000","last_became_peered":"2024-09-10T16:08:10.860438+0000","last_unstale":"2024-09-10T16:10:20.850377+0000","last_undegraded":"2024-09-10T16:10:20.850377+0000","last_fullsized":"2024-09-10T16:10:20.850377+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18724,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","0","1"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20838","reported_seq":21000,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:10:20.854744+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:10:20.854744+0000","last_peered":"2024-09-10T16:10:20.854744+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:10:20.854744+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17469,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.851195+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:10:20.851195+0000","last_peered":"2024-09-10T16:10:20.851195+0000","last_clean":"2024-09-10T16:10:20.851195+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:10:20.851195+0000","last_undegraded":"2024-09-10T16:10:20.851195+0000","last_fullsized":"2024-09-10T16:10:20.851195+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20598","reported_seq":29191,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.851322+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:10:20.851322+0000","last_peered":"2024-09-10T16:10:20.851322+0000","last_clean":"2024-09-10T16:10:20.851322+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:10:20.851322+0000","last_undegraded":"2024-09-10T16:10:20.851322+0000","last_fullsized":"2024-09-10T16:10:20.851322+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16988,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":121,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.850614+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:10:20.850614+0000","last_peered":"2024-09-10T16:10:20.850614+0000","last_clean":"2024-09-10T16:10:20.850614+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:10:20.850614+0000","last_undegraded":"2024-09-10T16:10:20.850614+0000","last_fullsized":"2024-09-10T16:10:20.850614+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'21048","reported_seq":21217,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:10:20.855031+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:10:20.855031+0000","last_peered":"2024-09-10T16:10:20.855031+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:10:20.855031+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17303,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.851276+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:10:20.851276+0000","last_peered":"2024-09-10T16:10:20.851276+0000","last_clean":"2024-09-10T16:10:20.851276+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:10:20.851276+0000","last_undegraded":"2024-09-10T16:10:20.851276+0000","last_fullsized":"2024-09-10T16:10:20.851276+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":102,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.851340+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:10:20.851340+0000","last_peered":"2024-09-10T16:10:20.851340+0000","last_clean":"2024-09-10T16:10:20.851340+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:10:20.851340+0000","last_undegraded":"2024-09-10T16:10:20.851340+0000","last_fullsized":"2024-09-10T16:10:20.851340+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'21198","reported_seq":21370,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:10:20.854530+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:10:20.854530+0000","last_peered":"2024-09-10T16:10:20.854530+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:10:20.854530+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17361,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4321},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":20402,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292096,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30988,"num_bytes_recovered":33097129,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":210509824,"data_stored":210394208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161022,"ondisk_log_size":161022,"up":32,"acting":27,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986326,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":502664,"kb_used_data":137776,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93869176,"statfs":{"total":96636764160,"available":96122036224,"internally_reserved":0,"allocated":141082624,"data_stored":140731007,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659271,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487572,"kb_used_data":516,"kb_used_omap":4,"kb_used_meta":487035,"kb_avail":93884268,"statfs":{"total":96636764160,"available":96137490432,"internally_reserved":0,"allocated":528384,"data_stored":165464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":498724526},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182554,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":547364,"kb_used_data":68684,"kb_used_omap":4,"kb_used_meta":478651,"kb_avail":93824476,"statfs":{"total":96636764160,"available":96076263424,"internally_reserved":0,"allocated":70332416,"data_stored":69976722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":490139306},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":69853184,"data_stored":69817054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":5796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":140607488,"data_stored":140571358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:10:32.518 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-10T16:10:32.518 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "49'21061", 'reported_seq': 23647, 'reported_epoch': 57, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:10:31.215480+0000', 'last_change': '2024-09-10T16:10:19.743410+0000', 'last_active': '2024-09-10T16:10:31.215480+0000', 'last_peered': '2024-09-10T16:10:31.215480+0000', 'last_clean': '2024-09-10T15:59:14.111551+0000', 'last_became_active': '2024-09-10T15:59:16.300040+0000', 'last_became_peered': '2024-09-10T15:59:16.300040+0000', 'last_unstale': '2024-09-10T16:10:31.215480+0000', 'last_undegraded': '2024-09-10T15:59:15.878234+0000', 'last_fullsized': '2024-09-10T15:59:15.769328+0000', 'mapping_epoch': 42, 'log_start': "45'11000", 'ondisk_log_start': "45'11000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3442", 'last_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'objects_scrubbed': 3384, 'log_size': 10061, 'log_dups_size': 0, 'ondisk_log_size': 10061, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1292, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3162, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 17673, 'num_write_kb': 28580, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1210, 'num_bytes_recovered': 483, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3161}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:10:32.518 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-10T16:10:32.518 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "49'21082", 'reported_seq': 21247, 'reported_epoch': 57, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:10:20.854635+0000', 'last_change': '2024-09-10T15:59:16.331899+0000', 'last_active': '2024-09-10T16:10:20.854635+0000', 'last_peered': '2024-09-10T16:10:20.854635+0000', 'last_clean': '2024-09-10T15:59:14.139436+0000', 'last_became_active': '2024-09-10T15:59:16.318715+0000', 'last_became_peered': '2024-09-10T15:59:16.318715+0000', 'last_unstale': '2024-09-10T16:10:20.854635+0000', 'last_undegraded': '2024-09-10T15:59:15.877524+0000', 'last_fullsized': '2024-09-10T15:59:15.769042+0000', 'mapping_epoch': 42, 'log_start': "45'11000", 'ondisk_log_start': "45'11000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3537", 'last_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'objects_scrubbed': 3497, 'log_size': 10082, 'log_dups_size': 0, 'ondisk_log_size': 10082, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1224, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4237, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 17581, 'num_write_kb': 28172, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4235}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:10:32.518 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-10T16:10:32.518 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "49'21170", 'reported_seq': 30042, 'reported_epoch': 57, 'state': 'active+recovery_wait', 'last_fresh': '2024-09-10T16:10:20.850377+0000', 'last_change': '2024-09-10T16:08:10.929169+0000', 'last_active': '2024-09-10T16:10:20.850377+0000', 'last_peered': '2024-09-10T16:10:20.850377+0000', 'last_clean': '2024-09-10T15:59:14.110654+0000', 'last_became_active': '2024-09-10T16:08:10.860438+0000', 'last_became_peered': '2024-09-10T16:08:10.860438+0000', 'last_unstale': '2024-09-10T16:10:20.850377+0000', 'last_undegraded': '2024-09-10T16:10:20.850377+0000', 'last_fullsized': '2024-09-10T16:10:20.850377+0000', 'mapping_epoch': 48, 'log_start': "45'11100", 'ondisk_log_start': "45'11100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2486", 'last_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'objects_scrubbed': 2440, 'log_size': 10070, 'log_dups_size': 0, 'ondisk_log_size': 10070, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 792, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18724, 'num_write_kb': 32568, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 4355, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': ['2', '0', '1'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:10:32.519 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-10T16:10:32.519 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "49'20838", 'reported_seq': 21000, 'reported_epoch': 57, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:10:20.854744+0000', 'last_change': '2024-09-10T15:59:16.315641+0000', 'last_active': '2024-09-10T16:10:20.854744+0000', 'last_peered': '2024-09-10T16:10:20.854744+0000', 'last_clean': '2024-09-10T15:59:14.117001+0000', 'last_became_active': '2024-09-10T15:59:16.302245+0000', 'last_became_peered': '2024-09-10T15:59:16.302245+0000', 'last_unstale': '2024-09-10T16:10:20.854744+0000', 'last_undegraded': '2024-09-10T15:59:15.845012+0000', 'last_fullsized': '2024-09-10T15:59:15.770337+0000', 'mapping_epoch': 42, 'log_start': "45'10800", 'ondisk_log_start': "45'10800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3404", 'last_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'objects_scrubbed': 3367, 'log_size': 10038, 'log_dups_size': 0, 'ondisk_log_size': 10038, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 983, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17469, 'num_write_kb': 28204, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4304}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:10:32.519 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-10T16:10:32.519 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "49'21048", 'reported_seq': 21217, 'reported_epoch': 57, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:10:20.855031+0000', 'last_change': '2024-09-10T15:59:16.311291+0000', 'last_active': '2024-09-10T16:10:20.855031+0000', 'last_peered': '2024-09-10T16:10:20.855031+0000', 'last_clean': '2024-09-10T15:59:14.116363+0000', 'last_became_active': '2024-09-10T15:59:16.301244+0000', 'last_became_peered': '2024-09-10T15:59:16.301244+0000', 'last_unstale': '2024-09-10T16:10:20.855031+0000', 'last_undegraded': '2024-09-10T15:59:15.883434+0000', 'last_fullsized': '2024-09-10T15:59:15.772756+0000', 'mapping_epoch': 42, 'log_start': "45'11000", 'ondisk_log_start': "45'11000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3789", 'last_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'objects_scrubbed': 3741, 'log_size': 10048, 'log_dups_size': 0, 'ondisk_log_size': 10048, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1440, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4373, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17303, 'num_write_kb': 27124, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4371}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:10:32.519 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-10T16:10:32.519 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "49'21198", 'reported_seq': 21370, 'reported_epoch': 57, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:10:20.854530+0000', 'last_change': '2024-09-10T15:59:16.313409+0000', 'last_active': '2024-09-10T16:10:20.854530+0000', 'last_peered': '2024-09-10T16:10:20.854530+0000', 'last_clean': '2024-09-10T15:59:14.129780+0000', 'last_became_active': '2024-09-10T15:59:16.300932+0000', 'last_became_peered': '2024-09-10T15:59:16.300932+0000', 'last_unstale': '2024-09-10T16:10:20.854530+0000', 'last_undegraded': '2024-09-10T15:59:15.887823+0000', 'last_fullsized': '2024-09-10T15:59:15.768513+0000', 'mapping_epoch': 42, 'log_start': "45'11100", 'ondisk_log_start': "45'11100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3880", 'last_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'objects_scrubbed': 3827, 'log_size': 10098, 'log_dups_size': 0, 'ondisk_log_size': 10098, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1397, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4325, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17361, 'num_write_kb': 27072, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4321}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:10:32.519 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T16:10:32.910 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:10:32.910 INFO:teuthology.orchestra.run.smithi115.stdout:{"fsid":"7effee29-df05-4351-9741-ee203564ed81","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 20402/24 objects degraded (85008.333%), 5 pgs degraded, 5 pgs undersized","count":10},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":796,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":57,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983954,"num_in_osds":3,"osd_in_since":1725983947,"num_remapped_pgs":5},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":18},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":4},{"state_name":"active+recovery_wait","count":1},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":12,"data_bytes":4849,"bytes_used":1574502400,"bytes_avail":288335790080,"bytes_total":289910292480,"degraded_objects":20402,"degraded_total":24,"degraded_ratio":850.08333333333348,"recovering_objects_per_sec":186,"recovering_bytes_per_sec":44,"recovering_keys_per_sec":0,"num_objects_recovered":2040,"num_bytes_recovered":483,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-10T15:57:14:967120+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T16:00:13.560776+0000","services":{}},"progress_events":{}} 2024-09-10T16:10:32.927 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-10T16:10:32.928 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:10:33.256 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:10:33.256 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:10:33.268 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":448,"stamp":"2024-09-10T16:10:31.784474+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":20402,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292098,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30990,"num_bytes_recovered":33097148,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161025,"ondisk_log_size":161025,"up":48,"acting":43,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1537600,"kb_used_data":206976,"kb_used_omap":12,"kb_used_meta":1330547,"kb_avail":281577920,"statfs":{"total":289910292480,"available":288335790080,"internally_reserved":0,"allocated":211943424,"data_stored":210873193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1362480130},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2040,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2040,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.935117"},"pg_stats":[{"pgid":"2.f","version":"49'20672","reported_seq":29300,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.856536+0000","last_change":"2024-09-10T16:09:57.088531+0000","last_active":"2024-09-10T16:10:20.856536+0000","last_peered":"2024-09-10T16:10:20.856536+0000","last_clean":"2024-09-10T16:10:20.856536+0000","last_became_active":"2024-09-10T16:09:57.005755+0000","last_became_peered":"2024-09-10T16:09:57.005755+0000","last_unstale":"2024-09-10T16:10:20.856536+0000","last_undegraded":"2024-09-10T16:10:20.856536+0000","last_fullsized":"2024-09-10T16:10:20.856536+0000","mapping_epoch":54,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:09:19.329697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20672,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4259,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20844","reported_seq":20984,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.850591+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:10:20.850591+0000","last_peered":"2024-09-10T16:10:20.850591+0000","last_clean":"2024-09-10T16:10:20.850591+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:10:20.850591+0000","last_undegraded":"2024-09-10T16:10:20.850591+0000","last_fullsized":"2024-09-10T16:10:20.850591+0000","mapping_epoch":35,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18949,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20798","reported_seq":29503,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.851218+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:10:20.851218+0000","last_peered":"2024-09-10T16:10:20.851218+0000","last_clean":"2024-09-10T16:10:20.851218+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:10:20.851218+0000","last_undegraded":"2024-09-10T16:10:20.851218+0000","last_fullsized":"2024-09-10T16:10:20.851218+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18802,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20660","reported_seq":20768,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.850554+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:10:20.850554+0000","last_peered":"2024-09-10T16:10:20.850554+0000","last_clean":"2024-09-10T16:10:20.850554+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:10:20.850554+0000","last_undegraded":"2024-09-10T16:10:20.850554+0000","last_fullsized":"2024-09-10T16:10:20.850554+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20660,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20891","reported_seq":21048,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.854539+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:10:20.854539+0000","last_peered":"2024-09-10T16:10:20.854539+0000","last_clean":"2024-09-10T16:10:20.854539+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:10:20.854539+0000","last_undegraded":"2024-09-10T16:10:20.854539+0000","last_fullsized":"2024-09-10T16:10:20.854539+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17602,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'21061","reported_seq":23647,"reported_epoch":57,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:10:31.215480+0000","last_change":"2024-09-10T16:10:19.743410+0000","last_active":"2024-09-10T16:10:31.215480+0000","last_peered":"2024-09-10T16:10:31.215480+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:10:31.215480+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3162,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17673,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1210,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3161}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'21082","reported_seq":21247,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:10:20.854635+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:10:20.854635+0000","last_peered":"2024-09-10T16:10:20.854635+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:10:20.854635+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17581,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4235},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'21314","reported_seq":21471,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.854391+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:10:20.854391+0000","last_peered":"2024-09-10T16:10:20.854391+0000","last_clean":"2024-09-10T16:10:20.854391+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:10:20.854391+0000","last_undegraded":"2024-09-10T16:10:20.854391+0000","last_fullsized":"2024-09-10T16:10:20.854391+0000","mapping_epoch":34,"log_start":"45'11300","ondisk_log_start":"45'11300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18161,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.850980+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:10:20.850980+0000","last_peered":"2024-09-10T16:10:20.850980+0000","last_clean":"2024-09-10T16:10:20.850980+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:10:20.850980+0000","last_undegraded":"2024-09-10T16:10:20.850980+0000","last_fullsized":"2024-09-10T16:10:20.850980+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20457","reported_seq":20628,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.851257+0000","last_change":"2024-09-10T16:09:33.012840+0000","last_active":"2024-09-10T16:10:20.851257+0000","last_peered":"2024-09-10T16:10:20.851257+0000","last_clean":"2024-09-10T16:10:20.851257+0000","last_became_active":"2024-09-10T16:09:32.934056+0000","last_became_peered":"2024-09-10T16:09:32.934056+0000","last_unstale":"2024-09-10T16:10:20.851257+0000","last_undegraded":"2024-09-10T16:10:20.851257+0000","last_fullsized":"2024-09-10T16:10:20.851257+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:08:30.481235+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18276,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":4579,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":121,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.850337+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:10:20.850337+0000","last_peered":"2024-09-10T16:10:20.850337+0000","last_clean":"2024-09-10T16:10:20.850337+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:10:20.850337+0000","last_undegraded":"2024-09-10T16:10:20.850337+0000","last_fullsized":"2024-09-10T16:10:20.850337+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20910","reported_seq":29590,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.854908+0000","last_change":"2024-09-10T16:09:08.909580+0000","last_active":"2024-09-10T16:10:20.854908+0000","last_peered":"2024-09-10T16:10:20.854908+0000","last_clean":"2024-09-10T16:10:20.854908+0000","last_became_active":"2024-09-10T16:09:08.854941+0000","last_became_peered":"2024-09-10T16:09:08.854941+0000","last_unstale":"2024-09-10T16:10:20.854908+0000","last_undegraded":"2024-09-10T16:10:20.854908+0000","last_fullsized":"2024-09-10T16:10:20.854908+0000","mapping_epoch":50,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:13:14.846691+0000","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17414,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4257,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":131,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.854759+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:10:20.854759+0000","last_peered":"2024-09-10T16:10:20.854759+0000","last_clean":"2024-09-10T16:10:20.854759+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:10:20.854759+0000","last_undegraded":"2024-09-10T16:10:20.854759+0000","last_fullsized":"2024-09-10T16:10:20.854759+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20781","reported_seq":29561,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.907987+0000","last_change":"2024-09-10T16:10:20.907889+0000","last_active":"2024-09-10T16:10:20.907987+0000","last_peered":"2024-09-10T16:10:20.907987+0000","last_clean":"2024-09-10T16:10:20.907987+0000","last_became_active":"2024-09-10T16:10:20.852065+0000","last_became_peered":"2024-09-10T16:10:20.852065+0000","last_unstale":"2024-09-10T16:10:20.907987+0000","last_undegraded":"2024-09-10T16:10:20.907987+0000","last_fullsized":"2024-09-10T16:10:20.907987+0000","mapping_epoch":56,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:34.069549+0000","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18461,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4307,"num_bytes_recovered":8675,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":107,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.851171+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:10:20.851171+0000","last_peered":"2024-09-10T16:10:20.851171+0000","last_clean":"2024-09-10T16:10:20.851171+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:10:20.851171+0000","last_undegraded":"2024-09-10T16:10:20.851171+0000","last_fullsized":"2024-09-10T16:10:20.851171+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'21170","reported_seq":30042,"reported_epoch":57,"state":"active+recovery_wait","last_fresh":"2024-09-10T16:10:20.850377+0000","last_change":"2024-09-10T16:08:10.929169+0000","last_active":"2024-09-10T16:10:20.850377+0000","last_peered":"2024-09-10T16:10:20.850377+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T16:08:10.860438+0000","last_became_peered":"2024-09-10T16:08:10.860438+0000","last_unstale":"2024-09-10T16:10:20.850377+0000","last_undegraded":"2024-09-10T16:10:20.850377+0000","last_fullsized":"2024-09-10T16:10:20.850377+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18724,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","0","1"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20838","reported_seq":21000,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:10:20.854744+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:10:20.854744+0000","last_peered":"2024-09-10T16:10:20.854744+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:10:20.854744+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17469,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.851195+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:10:20.851195+0000","last_peered":"2024-09-10T16:10:20.851195+0000","last_clean":"2024-09-10T16:10:20.851195+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:10:20.851195+0000","last_undegraded":"2024-09-10T16:10:20.851195+0000","last_fullsized":"2024-09-10T16:10:20.851195+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20598","reported_seq":29191,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.851322+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:10:20.851322+0000","last_peered":"2024-09-10T16:10:20.851322+0000","last_clean":"2024-09-10T16:10:20.851322+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:10:20.851322+0000","last_undegraded":"2024-09-10T16:10:20.851322+0000","last_fullsized":"2024-09-10T16:10:20.851322+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16988,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":121,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.850614+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:10:20.850614+0000","last_peered":"2024-09-10T16:10:20.850614+0000","last_clean":"2024-09-10T16:10:20.850614+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:10:20.850614+0000","last_undegraded":"2024-09-10T16:10:20.850614+0000","last_fullsized":"2024-09-10T16:10:20.850614+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'21048","reported_seq":21217,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:10:20.855031+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:10:20.855031+0000","last_peered":"2024-09-10T16:10:20.855031+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:10:20.855031+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17303,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.851276+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:10:20.851276+0000","last_peered":"2024-09-10T16:10:20.851276+0000","last_clean":"2024-09-10T16:10:20.851276+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:10:20.851276+0000","last_undegraded":"2024-09-10T16:10:20.851276+0000","last_fullsized":"2024-09-10T16:10:20.851276+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":102,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.851340+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:10:20.851340+0000","last_peered":"2024-09-10T16:10:20.851340+0000","last_clean":"2024-09-10T16:10:20.851340+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:10:20.851340+0000","last_undegraded":"2024-09-10T16:10:20.851340+0000","last_fullsized":"2024-09-10T16:10:20.851340+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'21198","reported_seq":21370,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:10:20.854530+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:10:20.854530+0000","last_peered":"2024-09-10T16:10:20.854530+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:10:20.854530+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17361,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4321},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":20402,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292096,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30988,"num_bytes_recovered":33097129,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":210509824,"data_stored":210394208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161022,"ondisk_log_size":161022,"up":32,"acting":27,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986326,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":502664,"kb_used_data":137776,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93869176,"statfs":{"total":96636764160,"available":96122036224,"internally_reserved":0,"allocated":141082624,"data_stored":140731007,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659271,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487572,"kb_used_data":516,"kb_used_omap":4,"kb_used_meta":487035,"kb_avail":93884268,"statfs":{"total":96636764160,"available":96137490432,"internally_reserved":0,"allocated":528384,"data_stored":165464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":498724526},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182554,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":547364,"kb_used_data":68684,"kb_used_omap":4,"kb_used_meta":478651,"kb_avail":93824476,"statfs":{"total":96636764160,"available":96076263424,"internally_reserved":0,"allocated":70332416,"data_stored":69976722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":490139306},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":69853184,"data_stored":69817054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":5796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":140607488,"data_stored":140571358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:10:36.270 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:10:36.568 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:10:36.568 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:10:36.580 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":450,"stamp":"2024-09-10T16:10:35.785136+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":20402,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292098,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30990,"num_bytes_recovered":33097148,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161025,"ondisk_log_size":161025,"up":48,"acting":43,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1541760,"kb_used_data":206976,"kb_used_omap":12,"kb_used_meta":1334707,"kb_avail":281573760,"statfs":{"total":289910292480,"available":288331530240,"internally_reserved":0,"allocated":211943424,"data_stored":210873193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1366739970},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1068,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1068,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002063"},"pg_stats":[{"pgid":"2.f","version":"49'20672","reported_seq":29300,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.856536+0000","last_change":"2024-09-10T16:09:57.088531+0000","last_active":"2024-09-10T16:10:20.856536+0000","last_peered":"2024-09-10T16:10:20.856536+0000","last_clean":"2024-09-10T16:10:20.856536+0000","last_became_active":"2024-09-10T16:09:57.005755+0000","last_became_peered":"2024-09-10T16:09:57.005755+0000","last_unstale":"2024-09-10T16:10:20.856536+0000","last_undegraded":"2024-09-10T16:10:20.856536+0000","last_fullsized":"2024-09-10T16:10:20.856536+0000","mapping_epoch":54,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:09:19.329697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20672,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4259,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20844","reported_seq":20984,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.850591+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:10:20.850591+0000","last_peered":"2024-09-10T16:10:20.850591+0000","last_clean":"2024-09-10T16:10:20.850591+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:10:20.850591+0000","last_undegraded":"2024-09-10T16:10:20.850591+0000","last_fullsized":"2024-09-10T16:10:20.850591+0000","mapping_epoch":35,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18949,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20798","reported_seq":29503,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.851218+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:10:20.851218+0000","last_peered":"2024-09-10T16:10:20.851218+0000","last_clean":"2024-09-10T16:10:20.851218+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:10:20.851218+0000","last_undegraded":"2024-09-10T16:10:20.851218+0000","last_fullsized":"2024-09-10T16:10:20.851218+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18802,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20660","reported_seq":20768,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.850554+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:10:20.850554+0000","last_peered":"2024-09-10T16:10:20.850554+0000","last_clean":"2024-09-10T16:10:20.850554+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:10:20.850554+0000","last_undegraded":"2024-09-10T16:10:20.850554+0000","last_fullsized":"2024-09-10T16:10:20.850554+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20660,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20891","reported_seq":21048,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.854539+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:10:20.854539+0000","last_peered":"2024-09-10T16:10:20.854539+0000","last_clean":"2024-09-10T16:10:20.854539+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:10:20.854539+0000","last_undegraded":"2024-09-10T16:10:20.854539+0000","last_fullsized":"2024-09-10T16:10:20.854539+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17602,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'21061","reported_seq":23647,"reported_epoch":57,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:10:31.215480+0000","last_change":"2024-09-10T16:10:19.743410+0000","last_active":"2024-09-10T16:10:31.215480+0000","last_peered":"2024-09-10T16:10:31.215480+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:10:31.215480+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3162,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17673,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1210,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3161}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'21082","reported_seq":21247,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:10:20.854635+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:10:20.854635+0000","last_peered":"2024-09-10T16:10:20.854635+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:10:20.854635+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17581,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4235},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'21314","reported_seq":21471,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.854391+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:10:20.854391+0000","last_peered":"2024-09-10T16:10:20.854391+0000","last_clean":"2024-09-10T16:10:20.854391+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:10:20.854391+0000","last_undegraded":"2024-09-10T16:10:20.854391+0000","last_fullsized":"2024-09-10T16:10:20.854391+0000","mapping_epoch":34,"log_start":"45'11300","ondisk_log_start":"45'11300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18161,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.850980+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:10:20.850980+0000","last_peered":"2024-09-10T16:10:20.850980+0000","last_clean":"2024-09-10T16:10:20.850980+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:10:20.850980+0000","last_undegraded":"2024-09-10T16:10:20.850980+0000","last_fullsized":"2024-09-10T16:10:20.850980+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20457","reported_seq":20628,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.851257+0000","last_change":"2024-09-10T16:09:33.012840+0000","last_active":"2024-09-10T16:10:20.851257+0000","last_peered":"2024-09-10T16:10:20.851257+0000","last_clean":"2024-09-10T16:10:20.851257+0000","last_became_active":"2024-09-10T16:09:32.934056+0000","last_became_peered":"2024-09-10T16:09:32.934056+0000","last_unstale":"2024-09-10T16:10:20.851257+0000","last_undegraded":"2024-09-10T16:10:20.851257+0000","last_fullsized":"2024-09-10T16:10:20.851257+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:08:30.481235+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18276,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":4579,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":121,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.850337+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:10:20.850337+0000","last_peered":"2024-09-10T16:10:20.850337+0000","last_clean":"2024-09-10T16:10:20.850337+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:10:20.850337+0000","last_undegraded":"2024-09-10T16:10:20.850337+0000","last_fullsized":"2024-09-10T16:10:20.850337+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20910","reported_seq":29590,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.854908+0000","last_change":"2024-09-10T16:09:08.909580+0000","last_active":"2024-09-10T16:10:20.854908+0000","last_peered":"2024-09-10T16:10:20.854908+0000","last_clean":"2024-09-10T16:10:20.854908+0000","last_became_active":"2024-09-10T16:09:08.854941+0000","last_became_peered":"2024-09-10T16:09:08.854941+0000","last_unstale":"2024-09-10T16:10:20.854908+0000","last_undegraded":"2024-09-10T16:10:20.854908+0000","last_fullsized":"2024-09-10T16:10:20.854908+0000","mapping_epoch":50,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:13:14.846691+0000","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17414,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4257,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":131,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.854759+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:10:20.854759+0000","last_peered":"2024-09-10T16:10:20.854759+0000","last_clean":"2024-09-10T16:10:20.854759+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:10:20.854759+0000","last_undegraded":"2024-09-10T16:10:20.854759+0000","last_fullsized":"2024-09-10T16:10:20.854759+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20781","reported_seq":29561,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.907987+0000","last_change":"2024-09-10T16:10:20.907889+0000","last_active":"2024-09-10T16:10:20.907987+0000","last_peered":"2024-09-10T16:10:20.907987+0000","last_clean":"2024-09-10T16:10:20.907987+0000","last_became_active":"2024-09-10T16:10:20.852065+0000","last_became_peered":"2024-09-10T16:10:20.852065+0000","last_unstale":"2024-09-10T16:10:20.907987+0000","last_undegraded":"2024-09-10T16:10:20.907987+0000","last_fullsized":"2024-09-10T16:10:20.907987+0000","mapping_epoch":56,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:34.069549+0000","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18461,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4307,"num_bytes_recovered":8675,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":107,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.851171+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:10:20.851171+0000","last_peered":"2024-09-10T16:10:20.851171+0000","last_clean":"2024-09-10T16:10:20.851171+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:10:20.851171+0000","last_undegraded":"2024-09-10T16:10:20.851171+0000","last_fullsized":"2024-09-10T16:10:20.851171+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'21170","reported_seq":30042,"reported_epoch":57,"state":"active+recovery_wait","last_fresh":"2024-09-10T16:10:20.850377+0000","last_change":"2024-09-10T16:08:10.929169+0000","last_active":"2024-09-10T16:10:20.850377+0000","last_peered":"2024-09-10T16:10:20.850377+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T16:08:10.860438+0000","last_became_peered":"2024-09-10T16:08:10.860438+0000","last_unstale":"2024-09-10T16:10:20.850377+0000","last_undegraded":"2024-09-10T16:10:20.850377+0000","last_fullsized":"2024-09-10T16:10:20.850377+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18724,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","0","1"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20838","reported_seq":21000,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:10:20.854744+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:10:20.854744+0000","last_peered":"2024-09-10T16:10:20.854744+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:10:20.854744+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17469,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.851195+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:10:20.851195+0000","last_peered":"2024-09-10T16:10:20.851195+0000","last_clean":"2024-09-10T16:10:20.851195+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:10:20.851195+0000","last_undegraded":"2024-09-10T16:10:20.851195+0000","last_fullsized":"2024-09-10T16:10:20.851195+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20598","reported_seq":29191,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.851322+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:10:20.851322+0000","last_peered":"2024-09-10T16:10:20.851322+0000","last_clean":"2024-09-10T16:10:20.851322+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:10:20.851322+0000","last_undegraded":"2024-09-10T16:10:20.851322+0000","last_fullsized":"2024-09-10T16:10:20.851322+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16988,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":121,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.850614+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:10:20.850614+0000","last_peered":"2024-09-10T16:10:20.850614+0000","last_clean":"2024-09-10T16:10:20.850614+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:10:20.850614+0000","last_undegraded":"2024-09-10T16:10:20.850614+0000","last_fullsized":"2024-09-10T16:10:20.850614+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'21048","reported_seq":21217,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:10:20.855031+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:10:20.855031+0000","last_peered":"2024-09-10T16:10:20.855031+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:10:20.855031+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17303,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.851276+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:10:20.851276+0000","last_peered":"2024-09-10T16:10:20.851276+0000","last_clean":"2024-09-10T16:10:20.851276+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:10:20.851276+0000","last_undegraded":"2024-09-10T16:10:20.851276+0000","last_fullsized":"2024-09-10T16:10:20.851276+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":102,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.851340+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:10:20.851340+0000","last_peered":"2024-09-10T16:10:20.851340+0000","last_clean":"2024-09-10T16:10:20.851340+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:10:20.851340+0000","last_undegraded":"2024-09-10T16:10:20.851340+0000","last_fullsized":"2024-09-10T16:10:20.851340+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'21198","reported_seq":21370,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:10:20.854530+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:10:20.854530+0000","last_peered":"2024-09-10T16:10:20.854530+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:10:20.854530+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17361,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4321},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":20402,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292096,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30988,"num_bytes_recovered":33097129,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":210509824,"data_stored":210394208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161022,"ondisk_log_size":161022,"up":32,"acting":27,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986326,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":502664,"kb_used_data":137776,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93869176,"statfs":{"total":96636764160,"available":96122036224,"internally_reserved":0,"allocated":141082624,"data_stored":140731007,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659272,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":491732,"kb_used_data":516,"kb_used_omap":4,"kb_used_meta":491195,"kb_avail":93880108,"statfs":{"total":96636764160,"available":96133230592,"internally_reserved":0,"allocated":528384,"data_stored":165464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":502984366},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182554,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":547364,"kb_used_data":68684,"kb_used_omap":4,"kb_used_meta":478651,"kb_avail":93824476,"statfs":{"total":96636764160,"available":96076263424,"internally_reserved":0,"allocated":70332416,"data_stored":69976722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":490139306},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":69853184,"data_stored":69817054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":5796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":140607488,"data_stored":140571358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:10:36.581 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:10:36.905 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:10:36.905 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:10:36.917 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":450,"stamp":"2024-09-10T16:10:35.785136+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":20402,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292098,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30990,"num_bytes_recovered":33097148,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161025,"ondisk_log_size":161025,"up":48,"acting":43,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1541760,"kb_used_data":206976,"kb_used_omap":12,"kb_used_meta":1334707,"kb_avail":281573760,"statfs":{"total":289910292480,"available":288331530240,"internally_reserved":0,"allocated":211943424,"data_stored":210873193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1366739970},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1068,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1068,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002063"},"pg_stats":[{"pgid":"2.f","version":"49'20672","reported_seq":29300,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.856536+0000","last_change":"2024-09-10T16:09:57.088531+0000","last_active":"2024-09-10T16:10:20.856536+0000","last_peered":"2024-09-10T16:10:20.856536+0000","last_clean":"2024-09-10T16:10:20.856536+0000","last_became_active":"2024-09-10T16:09:57.005755+0000","last_became_peered":"2024-09-10T16:09:57.005755+0000","last_unstale":"2024-09-10T16:10:20.856536+0000","last_undegraded":"2024-09-10T16:10:20.856536+0000","last_fullsized":"2024-09-10T16:10:20.856536+0000","mapping_epoch":54,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:09:19.329697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20672,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4259,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20844","reported_seq":20984,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.850591+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:10:20.850591+0000","last_peered":"2024-09-10T16:10:20.850591+0000","last_clean":"2024-09-10T16:10:20.850591+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:10:20.850591+0000","last_undegraded":"2024-09-10T16:10:20.850591+0000","last_fullsized":"2024-09-10T16:10:20.850591+0000","mapping_epoch":35,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18949,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20798","reported_seq":29503,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.851218+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:10:20.851218+0000","last_peered":"2024-09-10T16:10:20.851218+0000","last_clean":"2024-09-10T16:10:20.851218+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:10:20.851218+0000","last_undegraded":"2024-09-10T16:10:20.851218+0000","last_fullsized":"2024-09-10T16:10:20.851218+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18802,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20660","reported_seq":20768,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.850554+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:10:20.850554+0000","last_peered":"2024-09-10T16:10:20.850554+0000","last_clean":"2024-09-10T16:10:20.850554+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:10:20.850554+0000","last_undegraded":"2024-09-10T16:10:20.850554+0000","last_fullsized":"2024-09-10T16:10:20.850554+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20660,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20891","reported_seq":21048,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.854539+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:10:20.854539+0000","last_peered":"2024-09-10T16:10:20.854539+0000","last_clean":"2024-09-10T16:10:20.854539+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:10:20.854539+0000","last_undegraded":"2024-09-10T16:10:20.854539+0000","last_fullsized":"2024-09-10T16:10:20.854539+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17602,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'21061","reported_seq":23647,"reported_epoch":57,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:10:31.215480+0000","last_change":"2024-09-10T16:10:19.743410+0000","last_active":"2024-09-10T16:10:31.215480+0000","last_peered":"2024-09-10T16:10:31.215480+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:10:31.215480+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3162,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17673,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1210,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3161}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'21082","reported_seq":21247,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:10:20.854635+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:10:20.854635+0000","last_peered":"2024-09-10T16:10:20.854635+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:10:20.854635+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17581,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4235},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'21314","reported_seq":21471,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.854391+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:10:20.854391+0000","last_peered":"2024-09-10T16:10:20.854391+0000","last_clean":"2024-09-10T16:10:20.854391+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:10:20.854391+0000","last_undegraded":"2024-09-10T16:10:20.854391+0000","last_fullsized":"2024-09-10T16:10:20.854391+0000","mapping_epoch":34,"log_start":"45'11300","ondisk_log_start":"45'11300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18161,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.850980+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:10:20.850980+0000","last_peered":"2024-09-10T16:10:20.850980+0000","last_clean":"2024-09-10T16:10:20.850980+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:10:20.850980+0000","last_undegraded":"2024-09-10T16:10:20.850980+0000","last_fullsized":"2024-09-10T16:10:20.850980+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20457","reported_seq":20628,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.851257+0000","last_change":"2024-09-10T16:09:33.012840+0000","last_active":"2024-09-10T16:10:20.851257+0000","last_peered":"2024-09-10T16:10:20.851257+0000","last_clean":"2024-09-10T16:10:20.851257+0000","last_became_active":"2024-09-10T16:09:32.934056+0000","last_became_peered":"2024-09-10T16:09:32.934056+0000","last_unstale":"2024-09-10T16:10:20.851257+0000","last_undegraded":"2024-09-10T16:10:20.851257+0000","last_fullsized":"2024-09-10T16:10:20.851257+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:08:30.481235+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18276,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":4579,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":121,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.850337+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:10:20.850337+0000","last_peered":"2024-09-10T16:10:20.850337+0000","last_clean":"2024-09-10T16:10:20.850337+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:10:20.850337+0000","last_undegraded":"2024-09-10T16:10:20.850337+0000","last_fullsized":"2024-09-10T16:10:20.850337+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20910","reported_seq":29590,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.854908+0000","last_change":"2024-09-10T16:09:08.909580+0000","last_active":"2024-09-10T16:10:20.854908+0000","last_peered":"2024-09-10T16:10:20.854908+0000","last_clean":"2024-09-10T16:10:20.854908+0000","last_became_active":"2024-09-10T16:09:08.854941+0000","last_became_peered":"2024-09-10T16:09:08.854941+0000","last_unstale":"2024-09-10T16:10:20.854908+0000","last_undegraded":"2024-09-10T16:10:20.854908+0000","last_fullsized":"2024-09-10T16:10:20.854908+0000","mapping_epoch":50,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:13:14.846691+0000","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17414,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4257,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":131,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.854759+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:10:20.854759+0000","last_peered":"2024-09-10T16:10:20.854759+0000","last_clean":"2024-09-10T16:10:20.854759+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:10:20.854759+0000","last_undegraded":"2024-09-10T16:10:20.854759+0000","last_fullsized":"2024-09-10T16:10:20.854759+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20781","reported_seq":29561,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.907987+0000","last_change":"2024-09-10T16:10:20.907889+0000","last_active":"2024-09-10T16:10:20.907987+0000","last_peered":"2024-09-10T16:10:20.907987+0000","last_clean":"2024-09-10T16:10:20.907987+0000","last_became_active":"2024-09-10T16:10:20.852065+0000","last_became_peered":"2024-09-10T16:10:20.852065+0000","last_unstale":"2024-09-10T16:10:20.907987+0000","last_undegraded":"2024-09-10T16:10:20.907987+0000","last_fullsized":"2024-09-10T16:10:20.907987+0000","mapping_epoch":56,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:34.069549+0000","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18461,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4307,"num_bytes_recovered":8675,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":107,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.851171+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:10:20.851171+0000","last_peered":"2024-09-10T16:10:20.851171+0000","last_clean":"2024-09-10T16:10:20.851171+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:10:20.851171+0000","last_undegraded":"2024-09-10T16:10:20.851171+0000","last_fullsized":"2024-09-10T16:10:20.851171+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'21170","reported_seq":30042,"reported_epoch":57,"state":"active+recovery_wait","last_fresh":"2024-09-10T16:10:20.850377+0000","last_change":"2024-09-10T16:08:10.929169+0000","last_active":"2024-09-10T16:10:20.850377+0000","last_peered":"2024-09-10T16:10:20.850377+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T16:08:10.860438+0000","last_became_peered":"2024-09-10T16:08:10.860438+0000","last_unstale":"2024-09-10T16:10:20.850377+0000","last_undegraded":"2024-09-10T16:10:20.850377+0000","last_fullsized":"2024-09-10T16:10:20.850377+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18724,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","0","1"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20838","reported_seq":21000,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:10:20.854744+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:10:20.854744+0000","last_peered":"2024-09-10T16:10:20.854744+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:10:20.854744+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17469,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.851195+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:10:20.851195+0000","last_peered":"2024-09-10T16:10:20.851195+0000","last_clean":"2024-09-10T16:10:20.851195+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:10:20.851195+0000","last_undegraded":"2024-09-10T16:10:20.851195+0000","last_fullsized":"2024-09-10T16:10:20.851195+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20598","reported_seq":29191,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.851322+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:10:20.851322+0000","last_peered":"2024-09-10T16:10:20.851322+0000","last_clean":"2024-09-10T16:10:20.851322+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:10:20.851322+0000","last_undegraded":"2024-09-10T16:10:20.851322+0000","last_fullsized":"2024-09-10T16:10:20.851322+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16988,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":121,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.850614+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:10:20.850614+0000","last_peered":"2024-09-10T16:10:20.850614+0000","last_clean":"2024-09-10T16:10:20.850614+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:10:20.850614+0000","last_undegraded":"2024-09-10T16:10:20.850614+0000","last_fullsized":"2024-09-10T16:10:20.850614+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'21048","reported_seq":21217,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:10:20.855031+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:10:20.855031+0000","last_peered":"2024-09-10T16:10:20.855031+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:10:20.855031+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17303,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.851276+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:10:20.851276+0000","last_peered":"2024-09-10T16:10:20.851276+0000","last_clean":"2024-09-10T16:10:20.851276+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:10:20.851276+0000","last_undegraded":"2024-09-10T16:10:20.851276+0000","last_fullsized":"2024-09-10T16:10:20.851276+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":102,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.851340+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:10:20.851340+0000","last_peered":"2024-09-10T16:10:20.851340+0000","last_clean":"2024-09-10T16:10:20.851340+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:10:20.851340+0000","last_undegraded":"2024-09-10T16:10:20.851340+0000","last_fullsized":"2024-09-10T16:10:20.851340+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'21198","reported_seq":21370,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:10:20.854530+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:10:20.854530+0000","last_peered":"2024-09-10T16:10:20.854530+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:10:20.854530+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17361,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4321},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":20402,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292096,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30988,"num_bytes_recovered":33097129,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":210509824,"data_stored":210394208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161022,"ondisk_log_size":161022,"up":32,"acting":27,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986326,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":502664,"kb_used_data":137776,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93869176,"statfs":{"total":96636764160,"available":96122036224,"internally_reserved":0,"allocated":141082624,"data_stored":140731007,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659272,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":491732,"kb_used_data":516,"kb_used_omap":4,"kb_used_meta":491195,"kb_avail":93880108,"statfs":{"total":96636764160,"available":96133230592,"internally_reserved":0,"allocated":528384,"data_stored":165464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":502984366},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182554,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":547364,"kb_used_data":68684,"kb_used_omap":4,"kb_used_meta":478651,"kb_avail":93824476,"statfs":{"total":96636764160,"available":96076263424,"internally_reserved":0,"allocated":70332416,"data_stored":69976722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":490139306},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":69853184,"data_stored":69817054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":5796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":140607488,"data_stored":140571358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:10:36.918 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-10T16:10:36.919 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "49'21061", 'reported_seq': 23647, 'reported_epoch': 57, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:10:31.215480+0000', 'last_change': '2024-09-10T16:10:19.743410+0000', 'last_active': '2024-09-10T16:10:31.215480+0000', 'last_peered': '2024-09-10T16:10:31.215480+0000', 'last_clean': '2024-09-10T15:59:14.111551+0000', 'last_became_active': '2024-09-10T15:59:16.300040+0000', 'last_became_peered': '2024-09-10T15:59:16.300040+0000', 'last_unstale': '2024-09-10T16:10:31.215480+0000', 'last_undegraded': '2024-09-10T15:59:15.878234+0000', 'last_fullsized': '2024-09-10T15:59:15.769328+0000', 'mapping_epoch': 42, 'log_start': "45'11000", 'ondisk_log_start': "45'11000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3442", 'last_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'objects_scrubbed': 3384, 'log_size': 10061, 'log_dups_size': 0, 'ondisk_log_size': 10061, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1292, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3162, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 17673, 'num_write_kb': 28580, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1210, 'num_bytes_recovered': 483, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3161}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:10:36.919 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-10T16:10:36.919 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "49'21082", 'reported_seq': 21247, 'reported_epoch': 57, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:10:20.854635+0000', 'last_change': '2024-09-10T15:59:16.331899+0000', 'last_active': '2024-09-10T16:10:20.854635+0000', 'last_peered': '2024-09-10T16:10:20.854635+0000', 'last_clean': '2024-09-10T15:59:14.139436+0000', 'last_became_active': '2024-09-10T15:59:16.318715+0000', 'last_became_peered': '2024-09-10T15:59:16.318715+0000', 'last_unstale': '2024-09-10T16:10:20.854635+0000', 'last_undegraded': '2024-09-10T15:59:15.877524+0000', 'last_fullsized': '2024-09-10T15:59:15.769042+0000', 'mapping_epoch': 42, 'log_start': "45'11000", 'ondisk_log_start': "45'11000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3537", 'last_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'objects_scrubbed': 3497, 'log_size': 10082, 'log_dups_size': 0, 'ondisk_log_size': 10082, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1224, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4237, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 17581, 'num_write_kb': 28172, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4235}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:10:36.919 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-10T16:10:36.919 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "49'21170", 'reported_seq': 30042, 'reported_epoch': 57, 'state': 'active+recovery_wait', 'last_fresh': '2024-09-10T16:10:20.850377+0000', 'last_change': '2024-09-10T16:08:10.929169+0000', 'last_active': '2024-09-10T16:10:20.850377+0000', 'last_peered': '2024-09-10T16:10:20.850377+0000', 'last_clean': '2024-09-10T15:59:14.110654+0000', 'last_became_active': '2024-09-10T16:08:10.860438+0000', 'last_became_peered': '2024-09-10T16:08:10.860438+0000', 'last_unstale': '2024-09-10T16:10:20.850377+0000', 'last_undegraded': '2024-09-10T16:10:20.850377+0000', 'last_fullsized': '2024-09-10T16:10:20.850377+0000', 'mapping_epoch': 48, 'log_start': "45'11100", 'ondisk_log_start': "45'11100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2486", 'last_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'objects_scrubbed': 2440, 'log_size': 10070, 'log_dups_size': 0, 'ondisk_log_size': 10070, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 792, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18724, 'num_write_kb': 32568, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 4355, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': ['2', '0', '1'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:10:36.919 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-10T16:10:36.919 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "49'20838", 'reported_seq': 21000, 'reported_epoch': 57, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:10:20.854744+0000', 'last_change': '2024-09-10T15:59:16.315641+0000', 'last_active': '2024-09-10T16:10:20.854744+0000', 'last_peered': '2024-09-10T16:10:20.854744+0000', 'last_clean': '2024-09-10T15:59:14.117001+0000', 'last_became_active': '2024-09-10T15:59:16.302245+0000', 'last_became_peered': '2024-09-10T15:59:16.302245+0000', 'last_unstale': '2024-09-10T16:10:20.854744+0000', 'last_undegraded': '2024-09-10T15:59:15.845012+0000', 'last_fullsized': '2024-09-10T15:59:15.770337+0000', 'mapping_epoch': 42, 'log_start': "45'10800", 'ondisk_log_start': "45'10800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3404", 'last_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'objects_scrubbed': 3367, 'log_size': 10038, 'log_dups_size': 0, 'ondisk_log_size': 10038, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 983, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17469, 'num_write_kb': 28204, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4304}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:10:36.919 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-10T16:10:36.919 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "49'21048", 'reported_seq': 21217, 'reported_epoch': 57, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:10:20.855031+0000', 'last_change': '2024-09-10T15:59:16.311291+0000', 'last_active': '2024-09-10T16:10:20.855031+0000', 'last_peered': '2024-09-10T16:10:20.855031+0000', 'last_clean': '2024-09-10T15:59:14.116363+0000', 'last_became_active': '2024-09-10T15:59:16.301244+0000', 'last_became_peered': '2024-09-10T15:59:16.301244+0000', 'last_unstale': '2024-09-10T16:10:20.855031+0000', 'last_undegraded': '2024-09-10T15:59:15.883434+0000', 'last_fullsized': '2024-09-10T15:59:15.772756+0000', 'mapping_epoch': 42, 'log_start': "45'11000", 'ondisk_log_start': "45'11000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3789", 'last_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'objects_scrubbed': 3741, 'log_size': 10048, 'log_dups_size': 0, 'ondisk_log_size': 10048, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1440, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4373, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17303, 'num_write_kb': 27124, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4371}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:10:36.919 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-10T16:10:36.919 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "49'21198", 'reported_seq': 21370, 'reported_epoch': 57, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:10:20.854530+0000', 'last_change': '2024-09-10T15:59:16.313409+0000', 'last_active': '2024-09-10T16:10:20.854530+0000', 'last_peered': '2024-09-10T16:10:20.854530+0000', 'last_clean': '2024-09-10T15:59:14.129780+0000', 'last_became_active': '2024-09-10T15:59:16.300932+0000', 'last_became_peered': '2024-09-10T15:59:16.300932+0000', 'last_unstale': '2024-09-10T16:10:20.854530+0000', 'last_undegraded': '2024-09-10T15:59:15.887823+0000', 'last_fullsized': '2024-09-10T15:59:15.768513+0000', 'mapping_epoch': 42, 'log_start': "45'11100", 'ondisk_log_start': "45'11100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3880", 'last_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'objects_scrubbed': 3827, 'log_size': 10098, 'log_dups_size': 0, 'ondisk_log_size': 10098, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1397, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4325, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17361, 'num_write_kb': 27072, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4321}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:10:36.920 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T16:10:37.349 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:10:37.349 INFO:teuthology.orchestra.run.smithi115.stdout:{"fsid":"7effee29-df05-4351-9741-ee203564ed81","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 20402/24 objects degraded (85008.333%), 5 pgs degraded, 5 pgs undersized","count":10},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":801,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":57,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983954,"num_in_osds":3,"osd_in_since":1725983947,"num_remapped_pgs":5},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":18},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":4},{"state_name":"active+recovery_wait","count":1},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":12,"data_bytes":4849,"bytes_used":1578762240,"bytes_avail":288331530240,"bytes_total":289910292480,"degraded_objects":20402,"degraded_total":24,"degraded_ratio":850.08333333333348,"recovering_objects_per_sec":88,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1068,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-10T15:57:14:967120+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T16:00:13.560776+0000","services":{}},"progress_events":{}} 2024-09-10T16:10:37.363 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-10T16:10:37.363 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:10:37.668 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:10:37.668 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:10:37.680 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":450,"stamp":"2024-09-10T16:10:35.785136+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":20402,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292098,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30990,"num_bytes_recovered":33097148,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161025,"ondisk_log_size":161025,"up":48,"acting":43,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1541760,"kb_used_data":206976,"kb_used_omap":12,"kb_used_meta":1334707,"kb_avail":281573760,"statfs":{"total":289910292480,"available":288331530240,"internally_reserved":0,"allocated":211943424,"data_stored":210873193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1366739970},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1068,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1068,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002063"},"pg_stats":[{"pgid":"2.f","version":"49'20672","reported_seq":29300,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.856536+0000","last_change":"2024-09-10T16:09:57.088531+0000","last_active":"2024-09-10T16:10:20.856536+0000","last_peered":"2024-09-10T16:10:20.856536+0000","last_clean":"2024-09-10T16:10:20.856536+0000","last_became_active":"2024-09-10T16:09:57.005755+0000","last_became_peered":"2024-09-10T16:09:57.005755+0000","last_unstale":"2024-09-10T16:10:20.856536+0000","last_undegraded":"2024-09-10T16:10:20.856536+0000","last_fullsized":"2024-09-10T16:10:20.856536+0000","mapping_epoch":54,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:09:19.329697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20672,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4259,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20844","reported_seq":20984,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.850591+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:10:20.850591+0000","last_peered":"2024-09-10T16:10:20.850591+0000","last_clean":"2024-09-10T16:10:20.850591+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:10:20.850591+0000","last_undegraded":"2024-09-10T16:10:20.850591+0000","last_fullsized":"2024-09-10T16:10:20.850591+0000","mapping_epoch":35,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18949,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20798","reported_seq":29503,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.851218+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:10:20.851218+0000","last_peered":"2024-09-10T16:10:20.851218+0000","last_clean":"2024-09-10T16:10:20.851218+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:10:20.851218+0000","last_undegraded":"2024-09-10T16:10:20.851218+0000","last_fullsized":"2024-09-10T16:10:20.851218+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18802,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20660","reported_seq":20768,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.850554+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:10:20.850554+0000","last_peered":"2024-09-10T16:10:20.850554+0000","last_clean":"2024-09-10T16:10:20.850554+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:10:20.850554+0000","last_undegraded":"2024-09-10T16:10:20.850554+0000","last_fullsized":"2024-09-10T16:10:20.850554+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20660,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20891","reported_seq":21048,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.854539+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:10:20.854539+0000","last_peered":"2024-09-10T16:10:20.854539+0000","last_clean":"2024-09-10T16:10:20.854539+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:10:20.854539+0000","last_undegraded":"2024-09-10T16:10:20.854539+0000","last_fullsized":"2024-09-10T16:10:20.854539+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17602,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'21061","reported_seq":23647,"reported_epoch":57,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:10:31.215480+0000","last_change":"2024-09-10T16:10:19.743410+0000","last_active":"2024-09-10T16:10:31.215480+0000","last_peered":"2024-09-10T16:10:31.215480+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:10:31.215480+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3162,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17673,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1210,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3161}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'21082","reported_seq":21247,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:10:20.854635+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:10:20.854635+0000","last_peered":"2024-09-10T16:10:20.854635+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:10:20.854635+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17581,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4235},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'21314","reported_seq":21471,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.854391+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:10:20.854391+0000","last_peered":"2024-09-10T16:10:20.854391+0000","last_clean":"2024-09-10T16:10:20.854391+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:10:20.854391+0000","last_undegraded":"2024-09-10T16:10:20.854391+0000","last_fullsized":"2024-09-10T16:10:20.854391+0000","mapping_epoch":34,"log_start":"45'11300","ondisk_log_start":"45'11300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18161,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.850980+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:10:20.850980+0000","last_peered":"2024-09-10T16:10:20.850980+0000","last_clean":"2024-09-10T16:10:20.850980+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:10:20.850980+0000","last_undegraded":"2024-09-10T16:10:20.850980+0000","last_fullsized":"2024-09-10T16:10:20.850980+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20457","reported_seq":20628,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.851257+0000","last_change":"2024-09-10T16:09:33.012840+0000","last_active":"2024-09-10T16:10:20.851257+0000","last_peered":"2024-09-10T16:10:20.851257+0000","last_clean":"2024-09-10T16:10:20.851257+0000","last_became_active":"2024-09-10T16:09:32.934056+0000","last_became_peered":"2024-09-10T16:09:32.934056+0000","last_unstale":"2024-09-10T16:10:20.851257+0000","last_undegraded":"2024-09-10T16:10:20.851257+0000","last_fullsized":"2024-09-10T16:10:20.851257+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:08:30.481235+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18276,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":4579,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":121,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.850337+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:10:20.850337+0000","last_peered":"2024-09-10T16:10:20.850337+0000","last_clean":"2024-09-10T16:10:20.850337+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:10:20.850337+0000","last_undegraded":"2024-09-10T16:10:20.850337+0000","last_fullsized":"2024-09-10T16:10:20.850337+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20910","reported_seq":29590,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.854908+0000","last_change":"2024-09-10T16:09:08.909580+0000","last_active":"2024-09-10T16:10:20.854908+0000","last_peered":"2024-09-10T16:10:20.854908+0000","last_clean":"2024-09-10T16:10:20.854908+0000","last_became_active":"2024-09-10T16:09:08.854941+0000","last_became_peered":"2024-09-10T16:09:08.854941+0000","last_unstale":"2024-09-10T16:10:20.854908+0000","last_undegraded":"2024-09-10T16:10:20.854908+0000","last_fullsized":"2024-09-10T16:10:20.854908+0000","mapping_epoch":50,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:13:14.846691+0000","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17414,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4257,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":131,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.854759+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:10:20.854759+0000","last_peered":"2024-09-10T16:10:20.854759+0000","last_clean":"2024-09-10T16:10:20.854759+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:10:20.854759+0000","last_undegraded":"2024-09-10T16:10:20.854759+0000","last_fullsized":"2024-09-10T16:10:20.854759+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20781","reported_seq":29561,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.907987+0000","last_change":"2024-09-10T16:10:20.907889+0000","last_active":"2024-09-10T16:10:20.907987+0000","last_peered":"2024-09-10T16:10:20.907987+0000","last_clean":"2024-09-10T16:10:20.907987+0000","last_became_active":"2024-09-10T16:10:20.852065+0000","last_became_peered":"2024-09-10T16:10:20.852065+0000","last_unstale":"2024-09-10T16:10:20.907987+0000","last_undegraded":"2024-09-10T16:10:20.907987+0000","last_fullsized":"2024-09-10T16:10:20.907987+0000","mapping_epoch":56,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:34.069549+0000","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18461,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4307,"num_bytes_recovered":8675,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":107,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.851171+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:10:20.851171+0000","last_peered":"2024-09-10T16:10:20.851171+0000","last_clean":"2024-09-10T16:10:20.851171+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:10:20.851171+0000","last_undegraded":"2024-09-10T16:10:20.851171+0000","last_fullsized":"2024-09-10T16:10:20.851171+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'21170","reported_seq":30042,"reported_epoch":57,"state":"active+recovery_wait","last_fresh":"2024-09-10T16:10:20.850377+0000","last_change":"2024-09-10T16:08:10.929169+0000","last_active":"2024-09-10T16:10:20.850377+0000","last_peered":"2024-09-10T16:10:20.850377+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T16:08:10.860438+0000","last_became_peered":"2024-09-10T16:08:10.860438+0000","last_unstale":"2024-09-10T16:10:20.850377+0000","last_undegraded":"2024-09-10T16:10:20.850377+0000","last_fullsized":"2024-09-10T16:10:20.850377+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18724,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","0","1"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20838","reported_seq":21000,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:10:20.854744+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:10:20.854744+0000","last_peered":"2024-09-10T16:10:20.854744+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:10:20.854744+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17469,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.851195+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:10:20.851195+0000","last_peered":"2024-09-10T16:10:20.851195+0000","last_clean":"2024-09-10T16:10:20.851195+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:10:20.851195+0000","last_undegraded":"2024-09-10T16:10:20.851195+0000","last_fullsized":"2024-09-10T16:10:20.851195+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20598","reported_seq":29191,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.851322+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:10:20.851322+0000","last_peered":"2024-09-10T16:10:20.851322+0000","last_clean":"2024-09-10T16:10:20.851322+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:10:20.851322+0000","last_undegraded":"2024-09-10T16:10:20.851322+0000","last_fullsized":"2024-09-10T16:10:20.851322+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16988,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":121,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.850614+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:10:20.850614+0000","last_peered":"2024-09-10T16:10:20.850614+0000","last_clean":"2024-09-10T16:10:20.850614+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:10:20.850614+0000","last_undegraded":"2024-09-10T16:10:20.850614+0000","last_fullsized":"2024-09-10T16:10:20.850614+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'21048","reported_seq":21217,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:10:20.855031+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:10:20.855031+0000","last_peered":"2024-09-10T16:10:20.855031+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:10:20.855031+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17303,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.851276+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:10:20.851276+0000","last_peered":"2024-09-10T16:10:20.851276+0000","last_clean":"2024-09-10T16:10:20.851276+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:10:20.851276+0000","last_undegraded":"2024-09-10T16:10:20.851276+0000","last_fullsized":"2024-09-10T16:10:20.851276+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":102,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.851340+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:10:20.851340+0000","last_peered":"2024-09-10T16:10:20.851340+0000","last_clean":"2024-09-10T16:10:20.851340+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:10:20.851340+0000","last_undegraded":"2024-09-10T16:10:20.851340+0000","last_fullsized":"2024-09-10T16:10:20.851340+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'21198","reported_seq":21370,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:10:20.854530+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:10:20.854530+0000","last_peered":"2024-09-10T16:10:20.854530+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:10:20.854530+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17361,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4321},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":20402,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292096,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30988,"num_bytes_recovered":33097129,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":210509824,"data_stored":210394208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161022,"ondisk_log_size":161022,"up":32,"acting":27,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986326,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":502664,"kb_used_data":137776,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93869176,"statfs":{"total":96636764160,"available":96122036224,"internally_reserved":0,"allocated":141082624,"data_stored":140731007,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659272,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":491732,"kb_used_data":516,"kb_used_omap":4,"kb_used_meta":491195,"kb_avail":93880108,"statfs":{"total":96636764160,"available":96133230592,"internally_reserved":0,"allocated":528384,"data_stored":165464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":502984366},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182554,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":547364,"kb_used_data":68684,"kb_used_omap":4,"kb_used_meta":478651,"kb_avail":93824476,"statfs":{"total":96636764160,"available":96076263424,"internally_reserved":0,"allocated":70332416,"data_stored":69976722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":490139306},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":69853184,"data_stored":69817054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":5796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":140607488,"data_stored":140571358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:10:40.683 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:10:40.996 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:10:40.997 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:10:41.010 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":452,"stamp":"2024-09-10T16:10:39.785778+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":19808,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292098,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31584,"num_bytes_recovered":33097148,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161025,"ondisk_log_size":161025,"up":48,"acting":43,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1541760,"kb_used_data":206976,"kb_used_omap":12,"kb_used_meta":1334707,"kb_avail":281573760,"statfs":{"total":289910292480,"available":288331530240,"internally_reserved":0,"allocated":211943424,"data_stored":210873193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1366739970},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1148,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1148,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002031"},"pg_stats":[{"pgid":"2.f","version":"49'20672","reported_seq":29300,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.856536+0000","last_change":"2024-09-10T16:09:57.088531+0000","last_active":"2024-09-10T16:10:20.856536+0000","last_peered":"2024-09-10T16:10:20.856536+0000","last_clean":"2024-09-10T16:10:20.856536+0000","last_became_active":"2024-09-10T16:09:57.005755+0000","last_became_peered":"2024-09-10T16:09:57.005755+0000","last_unstale":"2024-09-10T16:10:20.856536+0000","last_undegraded":"2024-09-10T16:10:20.856536+0000","last_fullsized":"2024-09-10T16:10:20.856536+0000","mapping_epoch":54,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:09:19.329697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20672,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4259,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20844","reported_seq":20984,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.850591+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:10:20.850591+0000","last_peered":"2024-09-10T16:10:20.850591+0000","last_clean":"2024-09-10T16:10:20.850591+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:10:20.850591+0000","last_undegraded":"2024-09-10T16:10:20.850591+0000","last_fullsized":"2024-09-10T16:10:20.850591+0000","mapping_epoch":35,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18949,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20798","reported_seq":29503,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.851218+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:10:20.851218+0000","last_peered":"2024-09-10T16:10:20.851218+0000","last_clean":"2024-09-10T16:10:20.851218+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:10:20.851218+0000","last_undegraded":"2024-09-10T16:10:20.851218+0000","last_fullsized":"2024-09-10T16:10:20.851218+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18802,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20660","reported_seq":20768,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.850554+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:10:20.850554+0000","last_peered":"2024-09-10T16:10:20.850554+0000","last_clean":"2024-09-10T16:10:20.850554+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:10:20.850554+0000","last_undegraded":"2024-09-10T16:10:20.850554+0000","last_fullsized":"2024-09-10T16:10:20.850554+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20660,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20891","reported_seq":21048,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.854539+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:10:20.854539+0000","last_peered":"2024-09-10T16:10:20.854539+0000","last_clean":"2024-09-10T16:10:20.854539+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:10:20.854539+0000","last_undegraded":"2024-09-10T16:10:20.854539+0000","last_fullsized":"2024-09-10T16:10:20.854539+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17602,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'21061","reported_seq":24835,"reported_epoch":57,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:10:36.216454+0000","last_change":"2024-09-10T16:10:19.743410+0000","last_active":"2024-09-10T16:10:36.216454+0000","last_peered":"2024-09-10T16:10:36.216454+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:10:36.216454+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17673,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1804,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2567}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'21082","reported_seq":21247,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:10:20.854635+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:10:20.854635+0000","last_peered":"2024-09-10T16:10:20.854635+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:10:20.854635+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17581,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4235},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'21314","reported_seq":21471,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.854391+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:10:20.854391+0000","last_peered":"2024-09-10T16:10:20.854391+0000","last_clean":"2024-09-10T16:10:20.854391+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:10:20.854391+0000","last_undegraded":"2024-09-10T16:10:20.854391+0000","last_fullsized":"2024-09-10T16:10:20.854391+0000","mapping_epoch":34,"log_start":"45'11300","ondisk_log_start":"45'11300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18161,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.850980+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:10:20.850980+0000","last_peered":"2024-09-10T16:10:20.850980+0000","last_clean":"2024-09-10T16:10:20.850980+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:10:20.850980+0000","last_undegraded":"2024-09-10T16:10:20.850980+0000","last_fullsized":"2024-09-10T16:10:20.850980+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20457","reported_seq":20628,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.851257+0000","last_change":"2024-09-10T16:09:33.012840+0000","last_active":"2024-09-10T16:10:20.851257+0000","last_peered":"2024-09-10T16:10:20.851257+0000","last_clean":"2024-09-10T16:10:20.851257+0000","last_became_active":"2024-09-10T16:09:32.934056+0000","last_became_peered":"2024-09-10T16:09:32.934056+0000","last_unstale":"2024-09-10T16:10:20.851257+0000","last_undegraded":"2024-09-10T16:10:20.851257+0000","last_fullsized":"2024-09-10T16:10:20.851257+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:08:30.481235+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18276,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":4579,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":121,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.850337+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:10:20.850337+0000","last_peered":"2024-09-10T16:10:20.850337+0000","last_clean":"2024-09-10T16:10:20.850337+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:10:20.850337+0000","last_undegraded":"2024-09-10T16:10:20.850337+0000","last_fullsized":"2024-09-10T16:10:20.850337+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20910","reported_seq":29590,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.854908+0000","last_change":"2024-09-10T16:09:08.909580+0000","last_active":"2024-09-10T16:10:20.854908+0000","last_peered":"2024-09-10T16:10:20.854908+0000","last_clean":"2024-09-10T16:10:20.854908+0000","last_became_active":"2024-09-10T16:09:08.854941+0000","last_became_peered":"2024-09-10T16:09:08.854941+0000","last_unstale":"2024-09-10T16:10:20.854908+0000","last_undegraded":"2024-09-10T16:10:20.854908+0000","last_fullsized":"2024-09-10T16:10:20.854908+0000","mapping_epoch":50,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:13:14.846691+0000","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17414,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4257,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":132,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.854803+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:10:20.854803+0000","last_peered":"2024-09-10T16:10:20.854803+0000","last_clean":"2024-09-10T16:10:20.854803+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:10:20.854803+0000","last_undegraded":"2024-09-10T16:10:20.854803+0000","last_fullsized":"2024-09-10T16:10:20.854803+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20781","reported_seq":29561,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.907987+0000","last_change":"2024-09-10T16:10:20.907889+0000","last_active":"2024-09-10T16:10:20.907987+0000","last_peered":"2024-09-10T16:10:20.907987+0000","last_clean":"2024-09-10T16:10:20.907987+0000","last_became_active":"2024-09-10T16:10:20.852065+0000","last_became_peered":"2024-09-10T16:10:20.852065+0000","last_unstale":"2024-09-10T16:10:20.907987+0000","last_undegraded":"2024-09-10T16:10:20.907987+0000","last_fullsized":"2024-09-10T16:10:20.907987+0000","mapping_epoch":56,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:34.069549+0000","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18461,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4307,"num_bytes_recovered":8675,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":107,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.851171+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:10:20.851171+0000","last_peered":"2024-09-10T16:10:20.851171+0000","last_clean":"2024-09-10T16:10:20.851171+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:10:20.851171+0000","last_undegraded":"2024-09-10T16:10:20.851171+0000","last_fullsized":"2024-09-10T16:10:20.851171+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'21170","reported_seq":30042,"reported_epoch":57,"state":"active+recovery_wait","last_fresh":"2024-09-10T16:10:20.850377+0000","last_change":"2024-09-10T16:08:10.929169+0000","last_active":"2024-09-10T16:10:20.850377+0000","last_peered":"2024-09-10T16:10:20.850377+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T16:08:10.860438+0000","last_became_peered":"2024-09-10T16:08:10.860438+0000","last_unstale":"2024-09-10T16:10:20.850377+0000","last_undegraded":"2024-09-10T16:10:20.850377+0000","last_fullsized":"2024-09-10T16:10:20.850377+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18724,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","0","1"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20838","reported_seq":21000,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:10:20.854744+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:10:20.854744+0000","last_peered":"2024-09-10T16:10:20.854744+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:10:20.854744+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17469,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.851195+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:10:20.851195+0000","last_peered":"2024-09-10T16:10:20.851195+0000","last_clean":"2024-09-10T16:10:20.851195+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:10:20.851195+0000","last_undegraded":"2024-09-10T16:10:20.851195+0000","last_fullsized":"2024-09-10T16:10:20.851195+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20598","reported_seq":29191,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.851322+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:10:20.851322+0000","last_peered":"2024-09-10T16:10:20.851322+0000","last_clean":"2024-09-10T16:10:20.851322+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:10:20.851322+0000","last_undegraded":"2024-09-10T16:10:20.851322+0000","last_fullsized":"2024-09-10T16:10:20.851322+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16988,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":121,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.850614+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:10:20.850614+0000","last_peered":"2024-09-10T16:10:20.850614+0000","last_clean":"2024-09-10T16:10:20.850614+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:10:20.850614+0000","last_undegraded":"2024-09-10T16:10:20.850614+0000","last_fullsized":"2024-09-10T16:10:20.850614+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'21048","reported_seq":21217,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:10:20.855031+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:10:20.855031+0000","last_peered":"2024-09-10T16:10:20.855031+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:10:20.855031+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17303,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.851276+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:10:20.851276+0000","last_peered":"2024-09-10T16:10:20.851276+0000","last_clean":"2024-09-10T16:10:20.851276+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:10:20.851276+0000","last_undegraded":"2024-09-10T16:10:20.851276+0000","last_fullsized":"2024-09-10T16:10:20.851276+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":102,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.851340+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:10:20.851340+0000","last_peered":"2024-09-10T16:10:20.851340+0000","last_clean":"2024-09-10T16:10:20.851340+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:10:20.851340+0000","last_undegraded":"2024-09-10T16:10:20.851340+0000","last_fullsized":"2024-09-10T16:10:20.851340+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'21198","reported_seq":21370,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:10:20.854530+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:10:20.854530+0000","last_peered":"2024-09-10T16:10:20.854530+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:10:20.854530+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17361,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4321},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":19808,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292096,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31582,"num_bytes_recovered":33097129,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":210509824,"data_stored":210394208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161022,"ondisk_log_size":161022,"up":32,"acting":27,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986327,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":502664,"kb_used_data":137776,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93869176,"statfs":{"total":96636764160,"available":96122036224,"internally_reserved":0,"allocated":141082624,"data_stored":140731007,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659272,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":491732,"kb_used_data":516,"kb_used_omap":4,"kb_used_meta":491195,"kb_avail":93880108,"statfs":{"total":96636764160,"available":96133230592,"internally_reserved":0,"allocated":528384,"data_stored":165464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":502984366},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182555,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":547364,"kb_used_data":68684,"kb_used_omap":4,"kb_used_meta":478651,"kb_avail":93824476,"statfs":{"total":96636764160,"available":96076263424,"internally_reserved":0,"allocated":70332416,"data_stored":69976722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":490139306},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":69853184,"data_stored":69817054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":5796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":140607488,"data_stored":140571358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:10:41.011 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:10:41.323 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:10:41.324 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:10:41.336 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":452,"stamp":"2024-09-10T16:10:39.785778+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":19808,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292098,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31584,"num_bytes_recovered":33097148,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161025,"ondisk_log_size":161025,"up":48,"acting":43,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1541760,"kb_used_data":206976,"kb_used_omap":12,"kb_used_meta":1334707,"kb_avail":281573760,"statfs":{"total":289910292480,"available":288331530240,"internally_reserved":0,"allocated":211943424,"data_stored":210873193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1366739970},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1148,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1148,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002031"},"pg_stats":[{"pgid":"2.f","version":"49'20672","reported_seq":29300,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.856536+0000","last_change":"2024-09-10T16:09:57.088531+0000","last_active":"2024-09-10T16:10:20.856536+0000","last_peered":"2024-09-10T16:10:20.856536+0000","last_clean":"2024-09-10T16:10:20.856536+0000","last_became_active":"2024-09-10T16:09:57.005755+0000","last_became_peered":"2024-09-10T16:09:57.005755+0000","last_unstale":"2024-09-10T16:10:20.856536+0000","last_undegraded":"2024-09-10T16:10:20.856536+0000","last_fullsized":"2024-09-10T16:10:20.856536+0000","mapping_epoch":54,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:09:19.329697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20672,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4259,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20844","reported_seq":20984,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.850591+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:10:20.850591+0000","last_peered":"2024-09-10T16:10:20.850591+0000","last_clean":"2024-09-10T16:10:20.850591+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:10:20.850591+0000","last_undegraded":"2024-09-10T16:10:20.850591+0000","last_fullsized":"2024-09-10T16:10:20.850591+0000","mapping_epoch":35,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18949,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20798","reported_seq":29503,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.851218+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:10:20.851218+0000","last_peered":"2024-09-10T16:10:20.851218+0000","last_clean":"2024-09-10T16:10:20.851218+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:10:20.851218+0000","last_undegraded":"2024-09-10T16:10:20.851218+0000","last_fullsized":"2024-09-10T16:10:20.851218+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18802,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20660","reported_seq":20768,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.850554+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:10:20.850554+0000","last_peered":"2024-09-10T16:10:20.850554+0000","last_clean":"2024-09-10T16:10:20.850554+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:10:20.850554+0000","last_undegraded":"2024-09-10T16:10:20.850554+0000","last_fullsized":"2024-09-10T16:10:20.850554+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20660,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20891","reported_seq":21048,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.854539+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:10:20.854539+0000","last_peered":"2024-09-10T16:10:20.854539+0000","last_clean":"2024-09-10T16:10:20.854539+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:10:20.854539+0000","last_undegraded":"2024-09-10T16:10:20.854539+0000","last_fullsized":"2024-09-10T16:10:20.854539+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17602,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'21061","reported_seq":24835,"reported_epoch":57,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:10:36.216454+0000","last_change":"2024-09-10T16:10:19.743410+0000","last_active":"2024-09-10T16:10:36.216454+0000","last_peered":"2024-09-10T16:10:36.216454+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:10:36.216454+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17673,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1804,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2567}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'21082","reported_seq":21247,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:10:20.854635+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:10:20.854635+0000","last_peered":"2024-09-10T16:10:20.854635+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:10:20.854635+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17581,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4235},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'21314","reported_seq":21471,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.854391+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:10:20.854391+0000","last_peered":"2024-09-10T16:10:20.854391+0000","last_clean":"2024-09-10T16:10:20.854391+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:10:20.854391+0000","last_undegraded":"2024-09-10T16:10:20.854391+0000","last_fullsized":"2024-09-10T16:10:20.854391+0000","mapping_epoch":34,"log_start":"45'11300","ondisk_log_start":"45'11300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18161,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.850980+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:10:20.850980+0000","last_peered":"2024-09-10T16:10:20.850980+0000","last_clean":"2024-09-10T16:10:20.850980+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:10:20.850980+0000","last_undegraded":"2024-09-10T16:10:20.850980+0000","last_fullsized":"2024-09-10T16:10:20.850980+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20457","reported_seq":20628,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.851257+0000","last_change":"2024-09-10T16:09:33.012840+0000","last_active":"2024-09-10T16:10:20.851257+0000","last_peered":"2024-09-10T16:10:20.851257+0000","last_clean":"2024-09-10T16:10:20.851257+0000","last_became_active":"2024-09-10T16:09:32.934056+0000","last_became_peered":"2024-09-10T16:09:32.934056+0000","last_unstale":"2024-09-10T16:10:20.851257+0000","last_undegraded":"2024-09-10T16:10:20.851257+0000","last_fullsized":"2024-09-10T16:10:20.851257+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:08:30.481235+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18276,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":4579,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":121,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.850337+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:10:20.850337+0000","last_peered":"2024-09-10T16:10:20.850337+0000","last_clean":"2024-09-10T16:10:20.850337+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:10:20.850337+0000","last_undegraded":"2024-09-10T16:10:20.850337+0000","last_fullsized":"2024-09-10T16:10:20.850337+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20910","reported_seq":29590,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.854908+0000","last_change":"2024-09-10T16:09:08.909580+0000","last_active":"2024-09-10T16:10:20.854908+0000","last_peered":"2024-09-10T16:10:20.854908+0000","last_clean":"2024-09-10T16:10:20.854908+0000","last_became_active":"2024-09-10T16:09:08.854941+0000","last_became_peered":"2024-09-10T16:09:08.854941+0000","last_unstale":"2024-09-10T16:10:20.854908+0000","last_undegraded":"2024-09-10T16:10:20.854908+0000","last_fullsized":"2024-09-10T16:10:20.854908+0000","mapping_epoch":50,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:13:14.846691+0000","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17414,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4257,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":132,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.854803+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:10:20.854803+0000","last_peered":"2024-09-10T16:10:20.854803+0000","last_clean":"2024-09-10T16:10:20.854803+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:10:20.854803+0000","last_undegraded":"2024-09-10T16:10:20.854803+0000","last_fullsized":"2024-09-10T16:10:20.854803+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20781","reported_seq":29561,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.907987+0000","last_change":"2024-09-10T16:10:20.907889+0000","last_active":"2024-09-10T16:10:20.907987+0000","last_peered":"2024-09-10T16:10:20.907987+0000","last_clean":"2024-09-10T16:10:20.907987+0000","last_became_active":"2024-09-10T16:10:20.852065+0000","last_became_peered":"2024-09-10T16:10:20.852065+0000","last_unstale":"2024-09-10T16:10:20.907987+0000","last_undegraded":"2024-09-10T16:10:20.907987+0000","last_fullsized":"2024-09-10T16:10:20.907987+0000","mapping_epoch":56,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:34.069549+0000","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18461,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4307,"num_bytes_recovered":8675,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":107,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.851171+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:10:20.851171+0000","last_peered":"2024-09-10T16:10:20.851171+0000","last_clean":"2024-09-10T16:10:20.851171+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:10:20.851171+0000","last_undegraded":"2024-09-10T16:10:20.851171+0000","last_fullsized":"2024-09-10T16:10:20.851171+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'21170","reported_seq":30042,"reported_epoch":57,"state":"active+recovery_wait","last_fresh":"2024-09-10T16:10:20.850377+0000","last_change":"2024-09-10T16:08:10.929169+0000","last_active":"2024-09-10T16:10:20.850377+0000","last_peered":"2024-09-10T16:10:20.850377+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T16:08:10.860438+0000","last_became_peered":"2024-09-10T16:08:10.860438+0000","last_unstale":"2024-09-10T16:10:20.850377+0000","last_undegraded":"2024-09-10T16:10:20.850377+0000","last_fullsized":"2024-09-10T16:10:20.850377+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18724,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","0","1"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20838","reported_seq":21000,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:10:20.854744+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:10:20.854744+0000","last_peered":"2024-09-10T16:10:20.854744+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:10:20.854744+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17469,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.851195+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:10:20.851195+0000","last_peered":"2024-09-10T16:10:20.851195+0000","last_clean":"2024-09-10T16:10:20.851195+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:10:20.851195+0000","last_undegraded":"2024-09-10T16:10:20.851195+0000","last_fullsized":"2024-09-10T16:10:20.851195+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20598","reported_seq":29191,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.851322+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:10:20.851322+0000","last_peered":"2024-09-10T16:10:20.851322+0000","last_clean":"2024-09-10T16:10:20.851322+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:10:20.851322+0000","last_undegraded":"2024-09-10T16:10:20.851322+0000","last_fullsized":"2024-09-10T16:10:20.851322+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16988,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":121,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.850614+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:10:20.850614+0000","last_peered":"2024-09-10T16:10:20.850614+0000","last_clean":"2024-09-10T16:10:20.850614+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:10:20.850614+0000","last_undegraded":"2024-09-10T16:10:20.850614+0000","last_fullsized":"2024-09-10T16:10:20.850614+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'21048","reported_seq":21217,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:10:20.855031+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:10:20.855031+0000","last_peered":"2024-09-10T16:10:20.855031+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:10:20.855031+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17303,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.851276+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:10:20.851276+0000","last_peered":"2024-09-10T16:10:20.851276+0000","last_clean":"2024-09-10T16:10:20.851276+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:10:20.851276+0000","last_undegraded":"2024-09-10T16:10:20.851276+0000","last_fullsized":"2024-09-10T16:10:20.851276+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":102,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.851340+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:10:20.851340+0000","last_peered":"2024-09-10T16:10:20.851340+0000","last_clean":"2024-09-10T16:10:20.851340+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:10:20.851340+0000","last_undegraded":"2024-09-10T16:10:20.851340+0000","last_fullsized":"2024-09-10T16:10:20.851340+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'21198","reported_seq":21370,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:10:20.854530+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:10:20.854530+0000","last_peered":"2024-09-10T16:10:20.854530+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:10:20.854530+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17361,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4321},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":19808,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292096,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31582,"num_bytes_recovered":33097129,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":210509824,"data_stored":210394208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161022,"ondisk_log_size":161022,"up":32,"acting":27,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986327,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":502664,"kb_used_data":137776,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93869176,"statfs":{"total":96636764160,"available":96122036224,"internally_reserved":0,"allocated":141082624,"data_stored":140731007,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659272,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":491732,"kb_used_data":516,"kb_used_omap":4,"kb_used_meta":491195,"kb_avail":93880108,"statfs":{"total":96636764160,"available":96133230592,"internally_reserved":0,"allocated":528384,"data_stored":165464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":502984366},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182555,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":547364,"kb_used_data":68684,"kb_used_omap":4,"kb_used_meta":478651,"kb_avail":93824476,"statfs":{"total":96636764160,"available":96076263424,"internally_reserved":0,"allocated":70332416,"data_stored":69976722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":490139306},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":69853184,"data_stored":69817054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":5796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":140607488,"data_stored":140571358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:10:41.337 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-10T16:10:41.337 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "49'21061", 'reported_seq': 24835, 'reported_epoch': 57, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:10:36.216454+0000', 'last_change': '2024-09-10T16:10:19.743410+0000', 'last_active': '2024-09-10T16:10:36.216454+0000', 'last_peered': '2024-09-10T16:10:36.216454+0000', 'last_clean': '2024-09-10T15:59:14.111551+0000', 'last_became_active': '2024-09-10T15:59:16.300040+0000', 'last_became_peered': '2024-09-10T15:59:16.300040+0000', 'last_unstale': '2024-09-10T16:10:36.216454+0000', 'last_undegraded': '2024-09-10T15:59:15.878234+0000', 'last_fullsized': '2024-09-10T15:59:15.769328+0000', 'mapping_epoch': 42, 'log_start': "45'11000", 'ondisk_log_start': "45'11000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3442", 'last_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'objects_scrubbed': 3384, 'log_size': 10061, 'log_dups_size': 0, 'ondisk_log_size': 10061, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1292, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 17673, 'num_write_kb': 28580, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1804, 'num_bytes_recovered': 483, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2567}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:10:41.337 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-10T16:10:41.338 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "49'21082", 'reported_seq': 21247, 'reported_epoch': 57, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:10:20.854635+0000', 'last_change': '2024-09-10T15:59:16.331899+0000', 'last_active': '2024-09-10T16:10:20.854635+0000', 'last_peered': '2024-09-10T16:10:20.854635+0000', 'last_clean': '2024-09-10T15:59:14.139436+0000', 'last_became_active': '2024-09-10T15:59:16.318715+0000', 'last_became_peered': '2024-09-10T15:59:16.318715+0000', 'last_unstale': '2024-09-10T16:10:20.854635+0000', 'last_undegraded': '2024-09-10T15:59:15.877524+0000', 'last_fullsized': '2024-09-10T15:59:15.769042+0000', 'mapping_epoch': 42, 'log_start': "45'11000", 'ondisk_log_start': "45'11000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3537", 'last_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'objects_scrubbed': 3497, 'log_size': 10082, 'log_dups_size': 0, 'ondisk_log_size': 10082, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1224, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4237, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 17581, 'num_write_kb': 28172, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4235}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:10:41.338 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-10T16:10:41.338 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "49'21170", 'reported_seq': 30042, 'reported_epoch': 57, 'state': 'active+recovery_wait', 'last_fresh': '2024-09-10T16:10:20.850377+0000', 'last_change': '2024-09-10T16:08:10.929169+0000', 'last_active': '2024-09-10T16:10:20.850377+0000', 'last_peered': '2024-09-10T16:10:20.850377+0000', 'last_clean': '2024-09-10T15:59:14.110654+0000', 'last_became_active': '2024-09-10T16:08:10.860438+0000', 'last_became_peered': '2024-09-10T16:08:10.860438+0000', 'last_unstale': '2024-09-10T16:10:20.850377+0000', 'last_undegraded': '2024-09-10T16:10:20.850377+0000', 'last_fullsized': '2024-09-10T16:10:20.850377+0000', 'mapping_epoch': 48, 'log_start': "45'11100", 'ondisk_log_start': "45'11100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2486", 'last_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'objects_scrubbed': 2440, 'log_size': 10070, 'log_dups_size': 0, 'ondisk_log_size': 10070, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 792, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18724, 'num_write_kb': 32568, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 4355, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': ['2', '0', '1'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:10:41.338 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-10T16:10:41.338 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "49'20838", 'reported_seq': 21000, 'reported_epoch': 57, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:10:20.854744+0000', 'last_change': '2024-09-10T15:59:16.315641+0000', 'last_active': '2024-09-10T16:10:20.854744+0000', 'last_peered': '2024-09-10T16:10:20.854744+0000', 'last_clean': '2024-09-10T15:59:14.117001+0000', 'last_became_active': '2024-09-10T15:59:16.302245+0000', 'last_became_peered': '2024-09-10T15:59:16.302245+0000', 'last_unstale': '2024-09-10T16:10:20.854744+0000', 'last_undegraded': '2024-09-10T15:59:15.845012+0000', 'last_fullsized': '2024-09-10T15:59:15.770337+0000', 'mapping_epoch': 42, 'log_start': "45'10800", 'ondisk_log_start': "45'10800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3404", 'last_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'objects_scrubbed': 3367, 'log_size': 10038, 'log_dups_size': 0, 'ondisk_log_size': 10038, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 983, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17469, 'num_write_kb': 28204, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4304}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:10:41.338 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-10T16:10:41.338 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "49'21048", 'reported_seq': 21217, 'reported_epoch': 57, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:10:20.855031+0000', 'last_change': '2024-09-10T15:59:16.311291+0000', 'last_active': '2024-09-10T16:10:20.855031+0000', 'last_peered': '2024-09-10T16:10:20.855031+0000', 'last_clean': '2024-09-10T15:59:14.116363+0000', 'last_became_active': '2024-09-10T15:59:16.301244+0000', 'last_became_peered': '2024-09-10T15:59:16.301244+0000', 'last_unstale': '2024-09-10T16:10:20.855031+0000', 'last_undegraded': '2024-09-10T15:59:15.883434+0000', 'last_fullsized': '2024-09-10T15:59:15.772756+0000', 'mapping_epoch': 42, 'log_start': "45'11000", 'ondisk_log_start': "45'11000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3789", 'last_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'objects_scrubbed': 3741, 'log_size': 10048, 'log_dups_size': 0, 'ondisk_log_size': 10048, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1440, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4373, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17303, 'num_write_kb': 27124, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4371}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:10:41.338 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-10T16:10:41.338 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "49'21198", 'reported_seq': 21370, 'reported_epoch': 57, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:10:20.854530+0000', 'last_change': '2024-09-10T15:59:16.313409+0000', 'last_active': '2024-09-10T16:10:20.854530+0000', 'last_peered': '2024-09-10T16:10:20.854530+0000', 'last_clean': '2024-09-10T15:59:14.129780+0000', 'last_became_active': '2024-09-10T15:59:16.300932+0000', 'last_became_peered': '2024-09-10T15:59:16.300932+0000', 'last_unstale': '2024-09-10T16:10:20.854530+0000', 'last_undegraded': '2024-09-10T15:59:15.887823+0000', 'last_fullsized': '2024-09-10T15:59:15.768513+0000', 'mapping_epoch': 42, 'log_start': "45'11100", 'ondisk_log_start': "45'11100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3880", 'last_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'objects_scrubbed': 3827, 'log_size': 10098, 'log_dups_size': 0, 'ondisk_log_size': 10098, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1397, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4325, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17361, 'num_write_kb': 27072, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4321}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:10:41.339 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T16:10:41.784 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:10:41.784 INFO:teuthology.orchestra.run.smithi115.stdout:{"fsid":"7effee29-df05-4351-9741-ee203564ed81","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 19808/24 objects degraded (82533.333%), 5 pgs degraded, 5 pgs undersized","count":10},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":805,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":57,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983954,"num_in_osds":3,"osd_in_since":1725983947,"num_remapped_pgs":5},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":18},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":4},{"state_name":"active+recovery_wait","count":1},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":12,"data_bytes":4849,"bytes_used":1578762240,"bytes_avail":288331530240,"bytes_total":289910292480,"degraded_objects":19808,"degraded_total":24,"degraded_ratio":825.33333333333348,"recovering_objects_per_sec":95,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1148,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-10T15:57:14:967120+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T16:00:13.560776+0000","services":{}},"progress_events":{}} 2024-09-10T16:10:41.798 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-10T16:10:41.798 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:10:42.122 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:10:42.122 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:10:42.135 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":453,"stamp":"2024-09-10T16:10:41.786110+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":19106,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292098,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32286,"num_bytes_recovered":33097148,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161025,"ondisk_log_size":161025,"up":48,"acting":43,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1529280,"kb_used_data":206976,"kb_used_omap":12,"kb_used_meta":1322227,"kb_avail":281586240,"statfs":{"total":289910292480,"available":288344309760,"internally_reserved":0,"allocated":211943424,"data_stored":210873193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1353960450},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1850,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1850,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002019"},"pg_stats":[{"pgid":"2.f","version":"49'20672","reported_seq":29300,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.856536+0000","last_change":"2024-09-10T16:09:57.088531+0000","last_active":"2024-09-10T16:10:20.856536+0000","last_peered":"2024-09-10T16:10:20.856536+0000","last_clean":"2024-09-10T16:10:20.856536+0000","last_became_active":"2024-09-10T16:09:57.005755+0000","last_became_peered":"2024-09-10T16:09:57.005755+0000","last_unstale":"2024-09-10T16:10:20.856536+0000","last_undegraded":"2024-09-10T16:10:20.856536+0000","last_fullsized":"2024-09-10T16:10:20.856536+0000","mapping_epoch":54,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:09:19.329697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20672,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4259,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20844","reported_seq":20984,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.850591+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:10:20.850591+0000","last_peered":"2024-09-10T16:10:20.850591+0000","last_clean":"2024-09-10T16:10:20.850591+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:10:20.850591+0000","last_undegraded":"2024-09-10T16:10:20.850591+0000","last_fullsized":"2024-09-10T16:10:20.850591+0000","mapping_epoch":35,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18949,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20798","reported_seq":29503,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.851218+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:10:20.851218+0000","last_peered":"2024-09-10T16:10:20.851218+0000","last_clean":"2024-09-10T16:10:20.851218+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:10:20.851218+0000","last_undegraded":"2024-09-10T16:10:20.851218+0000","last_fullsized":"2024-09-10T16:10:20.851218+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18802,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20660","reported_seq":20768,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.850554+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:10:20.850554+0000","last_peered":"2024-09-10T16:10:20.850554+0000","last_clean":"2024-09-10T16:10:20.850554+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:10:20.850554+0000","last_undegraded":"2024-09-10T16:10:20.850554+0000","last_fullsized":"2024-09-10T16:10:20.850554+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20660,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20891","reported_seq":21048,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.854539+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:10:20.854539+0000","last_peered":"2024-09-10T16:10:20.854539+0000","last_clean":"2024-09-10T16:10:20.854539+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:10:20.854539+0000","last_undegraded":"2024-09-10T16:10:20.854539+0000","last_fullsized":"2024-09-10T16:10:20.854539+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17602,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'21061","reported_seq":26239,"reported_epoch":57,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:10:41.212418+0000","last_change":"2024-09-10T16:10:19.743410+0000","last_active":"2024-09-10T16:10:41.212418+0000","last_peered":"2024-09-10T16:10:41.212418+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:10:41.212418+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1866,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17673,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2506,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1865}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'21082","reported_seq":21247,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:10:20.854635+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:10:20.854635+0000","last_peered":"2024-09-10T16:10:20.854635+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:10:20.854635+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17581,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4235},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'21314","reported_seq":21471,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.854391+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:10:20.854391+0000","last_peered":"2024-09-10T16:10:20.854391+0000","last_clean":"2024-09-10T16:10:20.854391+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:10:20.854391+0000","last_undegraded":"2024-09-10T16:10:20.854391+0000","last_fullsized":"2024-09-10T16:10:20.854391+0000","mapping_epoch":34,"log_start":"45'11300","ondisk_log_start":"45'11300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18161,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.850980+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:10:20.850980+0000","last_peered":"2024-09-10T16:10:20.850980+0000","last_clean":"2024-09-10T16:10:20.850980+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:10:20.850980+0000","last_undegraded":"2024-09-10T16:10:20.850980+0000","last_fullsized":"2024-09-10T16:10:20.850980+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20457","reported_seq":20628,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.851257+0000","last_change":"2024-09-10T16:09:33.012840+0000","last_active":"2024-09-10T16:10:20.851257+0000","last_peered":"2024-09-10T16:10:20.851257+0000","last_clean":"2024-09-10T16:10:20.851257+0000","last_became_active":"2024-09-10T16:09:32.934056+0000","last_became_peered":"2024-09-10T16:09:32.934056+0000","last_unstale":"2024-09-10T16:10:20.851257+0000","last_undegraded":"2024-09-10T16:10:20.851257+0000","last_fullsized":"2024-09-10T16:10:20.851257+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:08:30.481235+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18276,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":4579,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":121,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.850337+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:10:20.850337+0000","last_peered":"2024-09-10T16:10:20.850337+0000","last_clean":"2024-09-10T16:10:20.850337+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:10:20.850337+0000","last_undegraded":"2024-09-10T16:10:20.850337+0000","last_fullsized":"2024-09-10T16:10:20.850337+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20910","reported_seq":29590,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.854908+0000","last_change":"2024-09-10T16:09:08.909580+0000","last_active":"2024-09-10T16:10:20.854908+0000","last_peered":"2024-09-10T16:10:20.854908+0000","last_clean":"2024-09-10T16:10:20.854908+0000","last_became_active":"2024-09-10T16:09:08.854941+0000","last_became_peered":"2024-09-10T16:09:08.854941+0000","last_unstale":"2024-09-10T16:10:20.854908+0000","last_undegraded":"2024-09-10T16:10:20.854908+0000","last_fullsized":"2024-09-10T16:10:20.854908+0000","mapping_epoch":50,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:13:14.846691+0000","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17414,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4257,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":132,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.854803+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:10:20.854803+0000","last_peered":"2024-09-10T16:10:20.854803+0000","last_clean":"2024-09-10T16:10:20.854803+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:10:20.854803+0000","last_undegraded":"2024-09-10T16:10:20.854803+0000","last_fullsized":"2024-09-10T16:10:20.854803+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20781","reported_seq":29561,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.907987+0000","last_change":"2024-09-10T16:10:20.907889+0000","last_active":"2024-09-10T16:10:20.907987+0000","last_peered":"2024-09-10T16:10:20.907987+0000","last_clean":"2024-09-10T16:10:20.907987+0000","last_became_active":"2024-09-10T16:10:20.852065+0000","last_became_peered":"2024-09-10T16:10:20.852065+0000","last_unstale":"2024-09-10T16:10:20.907987+0000","last_undegraded":"2024-09-10T16:10:20.907987+0000","last_fullsized":"2024-09-10T16:10:20.907987+0000","mapping_epoch":56,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:34.069549+0000","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18461,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4307,"num_bytes_recovered":8675,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":107,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.851171+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:10:20.851171+0000","last_peered":"2024-09-10T16:10:20.851171+0000","last_clean":"2024-09-10T16:10:20.851171+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:10:20.851171+0000","last_undegraded":"2024-09-10T16:10:20.851171+0000","last_fullsized":"2024-09-10T16:10:20.851171+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'21170","reported_seq":30042,"reported_epoch":57,"state":"active+recovery_wait","last_fresh":"2024-09-10T16:10:20.850377+0000","last_change":"2024-09-10T16:08:10.929169+0000","last_active":"2024-09-10T16:10:20.850377+0000","last_peered":"2024-09-10T16:10:20.850377+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T16:08:10.860438+0000","last_became_peered":"2024-09-10T16:08:10.860438+0000","last_unstale":"2024-09-10T16:10:20.850377+0000","last_undegraded":"2024-09-10T16:10:20.850377+0000","last_fullsized":"2024-09-10T16:10:20.850377+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18724,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","0","1"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20838","reported_seq":21000,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:10:20.854744+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:10:20.854744+0000","last_peered":"2024-09-10T16:10:20.854744+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:10:20.854744+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17469,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.851195+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:10:20.851195+0000","last_peered":"2024-09-10T16:10:20.851195+0000","last_clean":"2024-09-10T16:10:20.851195+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:10:20.851195+0000","last_undegraded":"2024-09-10T16:10:20.851195+0000","last_fullsized":"2024-09-10T16:10:20.851195+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20598","reported_seq":29191,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.851322+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:10:20.851322+0000","last_peered":"2024-09-10T16:10:20.851322+0000","last_clean":"2024-09-10T16:10:20.851322+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:10:20.851322+0000","last_undegraded":"2024-09-10T16:10:20.851322+0000","last_fullsized":"2024-09-10T16:10:20.851322+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16988,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":121,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.850614+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:10:20.850614+0000","last_peered":"2024-09-10T16:10:20.850614+0000","last_clean":"2024-09-10T16:10:20.850614+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:10:20.850614+0000","last_undegraded":"2024-09-10T16:10:20.850614+0000","last_fullsized":"2024-09-10T16:10:20.850614+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'21048","reported_seq":21217,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:10:20.855031+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:10:20.855031+0000","last_peered":"2024-09-10T16:10:20.855031+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:10:20.855031+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17303,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.851276+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:10:20.851276+0000","last_peered":"2024-09-10T16:10:20.851276+0000","last_clean":"2024-09-10T16:10:20.851276+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:10:20.851276+0000","last_undegraded":"2024-09-10T16:10:20.851276+0000","last_fullsized":"2024-09-10T16:10:20.851276+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":102,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.851340+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:10:20.851340+0000","last_peered":"2024-09-10T16:10:20.851340+0000","last_clean":"2024-09-10T16:10:20.851340+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:10:20.851340+0000","last_undegraded":"2024-09-10T16:10:20.851340+0000","last_fullsized":"2024-09-10T16:10:20.851340+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'21198","reported_seq":21370,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:10:20.854530+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:10:20.854530+0000","last_peered":"2024-09-10T16:10:20.854530+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:10:20.854530+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17361,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4321},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":19106,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292096,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32284,"num_bytes_recovered":33097129,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":210509824,"data_stored":210394208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161022,"ondisk_log_size":161022,"up":32,"acting":27,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986328,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":502664,"kb_used_data":137776,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93869176,"statfs":{"total":96636764160,"available":96122036224,"internally_reserved":0,"allocated":141082624,"data_stored":140731007,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659273,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":479252,"kb_used_data":516,"kb_used_omap":4,"kb_used_meta":478715,"kb_avail":93892588,"statfs":{"total":96636764160,"available":96146010112,"internally_reserved":0,"allocated":528384,"data_stored":165464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":490204846},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182556,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":547364,"kb_used_data":68684,"kb_used_omap":4,"kb_used_meta":478651,"kb_avail":93824476,"statfs":{"total":96636764160,"available":96076263424,"internally_reserved":0,"allocated":70332416,"data_stored":69976722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":490139306},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":69853184,"data_stored":69817054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":5796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":140607488,"data_stored":140571358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:10:45.137 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:10:45.449 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:10:45.449 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:10:45.462 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":454,"stamp":"2024-09-10T16:10:43.786460+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":19106,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292098,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32286,"num_bytes_recovered":33097148,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161025,"ondisk_log_size":161025,"up":48,"acting":43,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1529280,"kb_used_data":206976,"kb_used_omap":12,"kb_used_meta":1322227,"kb_avail":281586240,"statfs":{"total":289910292480,"available":288344309760,"internally_reserved":0,"allocated":211943424,"data_stored":210873193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1353960450},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1296,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1296,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001986"},"pg_stats":[{"pgid":"2.f","version":"49'20672","reported_seq":29300,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.856536+0000","last_change":"2024-09-10T16:09:57.088531+0000","last_active":"2024-09-10T16:10:20.856536+0000","last_peered":"2024-09-10T16:10:20.856536+0000","last_clean":"2024-09-10T16:10:20.856536+0000","last_became_active":"2024-09-10T16:09:57.005755+0000","last_became_peered":"2024-09-10T16:09:57.005755+0000","last_unstale":"2024-09-10T16:10:20.856536+0000","last_undegraded":"2024-09-10T16:10:20.856536+0000","last_fullsized":"2024-09-10T16:10:20.856536+0000","mapping_epoch":54,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:09:19.329697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20672,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4259,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20844","reported_seq":20984,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.850591+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:10:20.850591+0000","last_peered":"2024-09-10T16:10:20.850591+0000","last_clean":"2024-09-10T16:10:20.850591+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:10:20.850591+0000","last_undegraded":"2024-09-10T16:10:20.850591+0000","last_fullsized":"2024-09-10T16:10:20.850591+0000","mapping_epoch":35,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18949,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20798","reported_seq":29503,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.851218+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:10:20.851218+0000","last_peered":"2024-09-10T16:10:20.851218+0000","last_clean":"2024-09-10T16:10:20.851218+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:10:20.851218+0000","last_undegraded":"2024-09-10T16:10:20.851218+0000","last_fullsized":"2024-09-10T16:10:20.851218+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18802,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20660","reported_seq":20768,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.850554+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:10:20.850554+0000","last_peered":"2024-09-10T16:10:20.850554+0000","last_clean":"2024-09-10T16:10:20.850554+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:10:20.850554+0000","last_undegraded":"2024-09-10T16:10:20.850554+0000","last_fullsized":"2024-09-10T16:10:20.850554+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20660,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20891","reported_seq":21048,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.854539+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:10:20.854539+0000","last_peered":"2024-09-10T16:10:20.854539+0000","last_clean":"2024-09-10T16:10:20.854539+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:10:20.854539+0000","last_undegraded":"2024-09-10T16:10:20.854539+0000","last_fullsized":"2024-09-10T16:10:20.854539+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17602,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'21061","reported_seq":26239,"reported_epoch":57,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:10:41.212418+0000","last_change":"2024-09-10T16:10:19.743410+0000","last_active":"2024-09-10T16:10:41.212418+0000","last_peered":"2024-09-10T16:10:41.212418+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:10:41.212418+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1866,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17673,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2506,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1865}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'21082","reported_seq":21247,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:10:20.854635+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:10:20.854635+0000","last_peered":"2024-09-10T16:10:20.854635+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:10:20.854635+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17581,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4235},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'21314","reported_seq":21471,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.854391+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:10:20.854391+0000","last_peered":"2024-09-10T16:10:20.854391+0000","last_clean":"2024-09-10T16:10:20.854391+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:10:20.854391+0000","last_undegraded":"2024-09-10T16:10:20.854391+0000","last_fullsized":"2024-09-10T16:10:20.854391+0000","mapping_epoch":34,"log_start":"45'11300","ondisk_log_start":"45'11300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18161,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.850980+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:10:20.850980+0000","last_peered":"2024-09-10T16:10:20.850980+0000","last_clean":"2024-09-10T16:10:20.850980+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:10:20.850980+0000","last_undegraded":"2024-09-10T16:10:20.850980+0000","last_fullsized":"2024-09-10T16:10:20.850980+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20457","reported_seq":20628,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.851257+0000","last_change":"2024-09-10T16:09:33.012840+0000","last_active":"2024-09-10T16:10:20.851257+0000","last_peered":"2024-09-10T16:10:20.851257+0000","last_clean":"2024-09-10T16:10:20.851257+0000","last_became_active":"2024-09-10T16:09:32.934056+0000","last_became_peered":"2024-09-10T16:09:32.934056+0000","last_unstale":"2024-09-10T16:10:20.851257+0000","last_undegraded":"2024-09-10T16:10:20.851257+0000","last_fullsized":"2024-09-10T16:10:20.851257+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:08:30.481235+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18276,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":4579,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":121,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.850337+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:10:20.850337+0000","last_peered":"2024-09-10T16:10:20.850337+0000","last_clean":"2024-09-10T16:10:20.850337+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:10:20.850337+0000","last_undegraded":"2024-09-10T16:10:20.850337+0000","last_fullsized":"2024-09-10T16:10:20.850337+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20910","reported_seq":29590,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.854908+0000","last_change":"2024-09-10T16:09:08.909580+0000","last_active":"2024-09-10T16:10:20.854908+0000","last_peered":"2024-09-10T16:10:20.854908+0000","last_clean":"2024-09-10T16:10:20.854908+0000","last_became_active":"2024-09-10T16:09:08.854941+0000","last_became_peered":"2024-09-10T16:09:08.854941+0000","last_unstale":"2024-09-10T16:10:20.854908+0000","last_undegraded":"2024-09-10T16:10:20.854908+0000","last_fullsized":"2024-09-10T16:10:20.854908+0000","mapping_epoch":50,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:13:14.846691+0000","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17414,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4257,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":132,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.854803+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:10:20.854803+0000","last_peered":"2024-09-10T16:10:20.854803+0000","last_clean":"2024-09-10T16:10:20.854803+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:10:20.854803+0000","last_undegraded":"2024-09-10T16:10:20.854803+0000","last_fullsized":"2024-09-10T16:10:20.854803+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20781","reported_seq":29561,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.907987+0000","last_change":"2024-09-10T16:10:20.907889+0000","last_active":"2024-09-10T16:10:20.907987+0000","last_peered":"2024-09-10T16:10:20.907987+0000","last_clean":"2024-09-10T16:10:20.907987+0000","last_became_active":"2024-09-10T16:10:20.852065+0000","last_became_peered":"2024-09-10T16:10:20.852065+0000","last_unstale":"2024-09-10T16:10:20.907987+0000","last_undegraded":"2024-09-10T16:10:20.907987+0000","last_fullsized":"2024-09-10T16:10:20.907987+0000","mapping_epoch":56,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:34.069549+0000","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18461,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4307,"num_bytes_recovered":8675,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":107,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.851171+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:10:20.851171+0000","last_peered":"2024-09-10T16:10:20.851171+0000","last_clean":"2024-09-10T16:10:20.851171+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:10:20.851171+0000","last_undegraded":"2024-09-10T16:10:20.851171+0000","last_fullsized":"2024-09-10T16:10:20.851171+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'21170","reported_seq":30042,"reported_epoch":57,"state":"active+recovery_wait","last_fresh":"2024-09-10T16:10:20.850377+0000","last_change":"2024-09-10T16:08:10.929169+0000","last_active":"2024-09-10T16:10:20.850377+0000","last_peered":"2024-09-10T16:10:20.850377+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T16:08:10.860438+0000","last_became_peered":"2024-09-10T16:08:10.860438+0000","last_unstale":"2024-09-10T16:10:20.850377+0000","last_undegraded":"2024-09-10T16:10:20.850377+0000","last_fullsized":"2024-09-10T16:10:20.850377+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18724,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","0","1"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20838","reported_seq":21000,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:10:20.854744+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:10:20.854744+0000","last_peered":"2024-09-10T16:10:20.854744+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:10:20.854744+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17469,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.851195+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:10:20.851195+0000","last_peered":"2024-09-10T16:10:20.851195+0000","last_clean":"2024-09-10T16:10:20.851195+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:10:20.851195+0000","last_undegraded":"2024-09-10T16:10:20.851195+0000","last_fullsized":"2024-09-10T16:10:20.851195+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20598","reported_seq":29191,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.851322+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:10:20.851322+0000","last_peered":"2024-09-10T16:10:20.851322+0000","last_clean":"2024-09-10T16:10:20.851322+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:10:20.851322+0000","last_undegraded":"2024-09-10T16:10:20.851322+0000","last_fullsized":"2024-09-10T16:10:20.851322+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16988,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":121,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.850614+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:10:20.850614+0000","last_peered":"2024-09-10T16:10:20.850614+0000","last_clean":"2024-09-10T16:10:20.850614+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:10:20.850614+0000","last_undegraded":"2024-09-10T16:10:20.850614+0000","last_fullsized":"2024-09-10T16:10:20.850614+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'21048","reported_seq":21217,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:10:20.855031+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:10:20.855031+0000","last_peered":"2024-09-10T16:10:20.855031+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:10:20.855031+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17303,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.851276+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:10:20.851276+0000","last_peered":"2024-09-10T16:10:20.851276+0000","last_clean":"2024-09-10T16:10:20.851276+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:10:20.851276+0000","last_undegraded":"2024-09-10T16:10:20.851276+0000","last_fullsized":"2024-09-10T16:10:20.851276+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":102,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.851340+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:10:20.851340+0000","last_peered":"2024-09-10T16:10:20.851340+0000","last_clean":"2024-09-10T16:10:20.851340+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:10:20.851340+0000","last_undegraded":"2024-09-10T16:10:20.851340+0000","last_fullsized":"2024-09-10T16:10:20.851340+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'21198","reported_seq":21370,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:10:20.854530+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:10:20.854530+0000","last_peered":"2024-09-10T16:10:20.854530+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:10:20.854530+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17361,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4321},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":19106,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292096,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32284,"num_bytes_recovered":33097129,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":210509824,"data_stored":210394208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161022,"ondisk_log_size":161022,"up":32,"acting":27,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986328,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":502664,"kb_used_data":137776,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93869176,"statfs":{"total":96636764160,"available":96122036224,"internally_reserved":0,"allocated":141082624,"data_stored":140731007,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659273,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":479252,"kb_used_data":516,"kb_used_omap":4,"kb_used_meta":478715,"kb_avail":93892588,"statfs":{"total":96636764160,"available":96146010112,"internally_reserved":0,"allocated":528384,"data_stored":165464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":490204846},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182556,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":547364,"kb_used_data":68684,"kb_used_omap":4,"kb_used_meta":478651,"kb_avail":93824476,"statfs":{"total":96636764160,"available":96076263424,"internally_reserved":0,"allocated":70332416,"data_stored":69976722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":490139306},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":69853184,"data_stored":69817054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":5796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":140607488,"data_stored":140571358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:10:45.463 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:10:45.775 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:10:45.776 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:10:45.788 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":454,"stamp":"2024-09-10T16:10:43.786460+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":19106,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292098,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32286,"num_bytes_recovered":33097148,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161025,"ondisk_log_size":161025,"up":48,"acting":43,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1529280,"kb_used_data":206976,"kb_used_omap":12,"kb_used_meta":1322227,"kb_avail":281586240,"statfs":{"total":289910292480,"available":288344309760,"internally_reserved":0,"allocated":211943424,"data_stored":210873193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1353960450},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1296,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1296,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001986"},"pg_stats":[{"pgid":"2.f","version":"49'20672","reported_seq":29300,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.856536+0000","last_change":"2024-09-10T16:09:57.088531+0000","last_active":"2024-09-10T16:10:20.856536+0000","last_peered":"2024-09-10T16:10:20.856536+0000","last_clean":"2024-09-10T16:10:20.856536+0000","last_became_active":"2024-09-10T16:09:57.005755+0000","last_became_peered":"2024-09-10T16:09:57.005755+0000","last_unstale":"2024-09-10T16:10:20.856536+0000","last_undegraded":"2024-09-10T16:10:20.856536+0000","last_fullsized":"2024-09-10T16:10:20.856536+0000","mapping_epoch":54,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:09:19.329697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20672,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4259,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20844","reported_seq":20984,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.850591+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:10:20.850591+0000","last_peered":"2024-09-10T16:10:20.850591+0000","last_clean":"2024-09-10T16:10:20.850591+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:10:20.850591+0000","last_undegraded":"2024-09-10T16:10:20.850591+0000","last_fullsized":"2024-09-10T16:10:20.850591+0000","mapping_epoch":35,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18949,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20798","reported_seq":29503,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.851218+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:10:20.851218+0000","last_peered":"2024-09-10T16:10:20.851218+0000","last_clean":"2024-09-10T16:10:20.851218+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:10:20.851218+0000","last_undegraded":"2024-09-10T16:10:20.851218+0000","last_fullsized":"2024-09-10T16:10:20.851218+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18802,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20660","reported_seq":20768,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.850554+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:10:20.850554+0000","last_peered":"2024-09-10T16:10:20.850554+0000","last_clean":"2024-09-10T16:10:20.850554+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:10:20.850554+0000","last_undegraded":"2024-09-10T16:10:20.850554+0000","last_fullsized":"2024-09-10T16:10:20.850554+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20660,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20891","reported_seq":21048,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.854539+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:10:20.854539+0000","last_peered":"2024-09-10T16:10:20.854539+0000","last_clean":"2024-09-10T16:10:20.854539+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:10:20.854539+0000","last_undegraded":"2024-09-10T16:10:20.854539+0000","last_fullsized":"2024-09-10T16:10:20.854539+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17602,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'21061","reported_seq":26239,"reported_epoch":57,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:10:41.212418+0000","last_change":"2024-09-10T16:10:19.743410+0000","last_active":"2024-09-10T16:10:41.212418+0000","last_peered":"2024-09-10T16:10:41.212418+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:10:41.212418+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1866,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17673,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2506,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1865}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'21082","reported_seq":21247,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:10:20.854635+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:10:20.854635+0000","last_peered":"2024-09-10T16:10:20.854635+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:10:20.854635+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17581,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4235},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'21314","reported_seq":21471,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.854391+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:10:20.854391+0000","last_peered":"2024-09-10T16:10:20.854391+0000","last_clean":"2024-09-10T16:10:20.854391+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:10:20.854391+0000","last_undegraded":"2024-09-10T16:10:20.854391+0000","last_fullsized":"2024-09-10T16:10:20.854391+0000","mapping_epoch":34,"log_start":"45'11300","ondisk_log_start":"45'11300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18161,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.850980+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:10:20.850980+0000","last_peered":"2024-09-10T16:10:20.850980+0000","last_clean":"2024-09-10T16:10:20.850980+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:10:20.850980+0000","last_undegraded":"2024-09-10T16:10:20.850980+0000","last_fullsized":"2024-09-10T16:10:20.850980+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20457","reported_seq":20628,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.851257+0000","last_change":"2024-09-10T16:09:33.012840+0000","last_active":"2024-09-10T16:10:20.851257+0000","last_peered":"2024-09-10T16:10:20.851257+0000","last_clean":"2024-09-10T16:10:20.851257+0000","last_became_active":"2024-09-10T16:09:32.934056+0000","last_became_peered":"2024-09-10T16:09:32.934056+0000","last_unstale":"2024-09-10T16:10:20.851257+0000","last_undegraded":"2024-09-10T16:10:20.851257+0000","last_fullsized":"2024-09-10T16:10:20.851257+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:08:30.481235+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18276,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":4579,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":121,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.850337+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:10:20.850337+0000","last_peered":"2024-09-10T16:10:20.850337+0000","last_clean":"2024-09-10T16:10:20.850337+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:10:20.850337+0000","last_undegraded":"2024-09-10T16:10:20.850337+0000","last_fullsized":"2024-09-10T16:10:20.850337+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20910","reported_seq":29590,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.854908+0000","last_change":"2024-09-10T16:09:08.909580+0000","last_active":"2024-09-10T16:10:20.854908+0000","last_peered":"2024-09-10T16:10:20.854908+0000","last_clean":"2024-09-10T16:10:20.854908+0000","last_became_active":"2024-09-10T16:09:08.854941+0000","last_became_peered":"2024-09-10T16:09:08.854941+0000","last_unstale":"2024-09-10T16:10:20.854908+0000","last_undegraded":"2024-09-10T16:10:20.854908+0000","last_fullsized":"2024-09-10T16:10:20.854908+0000","mapping_epoch":50,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:13:14.846691+0000","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17414,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4257,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":132,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.854803+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:10:20.854803+0000","last_peered":"2024-09-10T16:10:20.854803+0000","last_clean":"2024-09-10T16:10:20.854803+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:10:20.854803+0000","last_undegraded":"2024-09-10T16:10:20.854803+0000","last_fullsized":"2024-09-10T16:10:20.854803+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20781","reported_seq":29561,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.907987+0000","last_change":"2024-09-10T16:10:20.907889+0000","last_active":"2024-09-10T16:10:20.907987+0000","last_peered":"2024-09-10T16:10:20.907987+0000","last_clean":"2024-09-10T16:10:20.907987+0000","last_became_active":"2024-09-10T16:10:20.852065+0000","last_became_peered":"2024-09-10T16:10:20.852065+0000","last_unstale":"2024-09-10T16:10:20.907987+0000","last_undegraded":"2024-09-10T16:10:20.907987+0000","last_fullsized":"2024-09-10T16:10:20.907987+0000","mapping_epoch":56,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:34.069549+0000","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18461,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4307,"num_bytes_recovered":8675,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":107,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.851171+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:10:20.851171+0000","last_peered":"2024-09-10T16:10:20.851171+0000","last_clean":"2024-09-10T16:10:20.851171+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:10:20.851171+0000","last_undegraded":"2024-09-10T16:10:20.851171+0000","last_fullsized":"2024-09-10T16:10:20.851171+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'21170","reported_seq":30042,"reported_epoch":57,"state":"active+recovery_wait","last_fresh":"2024-09-10T16:10:20.850377+0000","last_change":"2024-09-10T16:08:10.929169+0000","last_active":"2024-09-10T16:10:20.850377+0000","last_peered":"2024-09-10T16:10:20.850377+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T16:08:10.860438+0000","last_became_peered":"2024-09-10T16:08:10.860438+0000","last_unstale":"2024-09-10T16:10:20.850377+0000","last_undegraded":"2024-09-10T16:10:20.850377+0000","last_fullsized":"2024-09-10T16:10:20.850377+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18724,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","0","1"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20838","reported_seq":21000,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:10:20.854744+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:10:20.854744+0000","last_peered":"2024-09-10T16:10:20.854744+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:10:20.854744+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17469,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.851195+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:10:20.851195+0000","last_peered":"2024-09-10T16:10:20.851195+0000","last_clean":"2024-09-10T16:10:20.851195+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:10:20.851195+0000","last_undegraded":"2024-09-10T16:10:20.851195+0000","last_fullsized":"2024-09-10T16:10:20.851195+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20598","reported_seq":29191,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.851322+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:10:20.851322+0000","last_peered":"2024-09-10T16:10:20.851322+0000","last_clean":"2024-09-10T16:10:20.851322+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:10:20.851322+0000","last_undegraded":"2024-09-10T16:10:20.851322+0000","last_fullsized":"2024-09-10T16:10:20.851322+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16988,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":121,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.850614+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:10:20.850614+0000","last_peered":"2024-09-10T16:10:20.850614+0000","last_clean":"2024-09-10T16:10:20.850614+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:10:20.850614+0000","last_undegraded":"2024-09-10T16:10:20.850614+0000","last_fullsized":"2024-09-10T16:10:20.850614+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'21048","reported_seq":21217,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:10:20.855031+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:10:20.855031+0000","last_peered":"2024-09-10T16:10:20.855031+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:10:20.855031+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17303,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.851276+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:10:20.851276+0000","last_peered":"2024-09-10T16:10:20.851276+0000","last_clean":"2024-09-10T16:10:20.851276+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:10:20.851276+0000","last_undegraded":"2024-09-10T16:10:20.851276+0000","last_fullsized":"2024-09-10T16:10:20.851276+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":102,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.851340+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:10:20.851340+0000","last_peered":"2024-09-10T16:10:20.851340+0000","last_clean":"2024-09-10T16:10:20.851340+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:10:20.851340+0000","last_undegraded":"2024-09-10T16:10:20.851340+0000","last_fullsized":"2024-09-10T16:10:20.851340+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'21198","reported_seq":21370,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:10:20.854530+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:10:20.854530+0000","last_peered":"2024-09-10T16:10:20.854530+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:10:20.854530+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17361,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4321},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":19106,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292096,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32284,"num_bytes_recovered":33097129,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":210509824,"data_stored":210394208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161022,"ondisk_log_size":161022,"up":32,"acting":27,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986328,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":502664,"kb_used_data":137776,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93869176,"statfs":{"total":96636764160,"available":96122036224,"internally_reserved":0,"allocated":141082624,"data_stored":140731007,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659273,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":479252,"kb_used_data":516,"kb_used_omap":4,"kb_used_meta":478715,"kb_avail":93892588,"statfs":{"total":96636764160,"available":96146010112,"internally_reserved":0,"allocated":528384,"data_stored":165464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":490204846},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182556,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":547364,"kb_used_data":68684,"kb_used_omap":4,"kb_used_meta":478651,"kb_avail":93824476,"statfs":{"total":96636764160,"available":96076263424,"internally_reserved":0,"allocated":70332416,"data_stored":69976722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":490139306},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":69853184,"data_stored":69817054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":5796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":140607488,"data_stored":140571358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:10:45.789 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-10T16:10:45.789 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "49'21061", 'reported_seq': 26239, 'reported_epoch': 57, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:10:41.212418+0000', 'last_change': '2024-09-10T16:10:19.743410+0000', 'last_active': '2024-09-10T16:10:41.212418+0000', 'last_peered': '2024-09-10T16:10:41.212418+0000', 'last_clean': '2024-09-10T15:59:14.111551+0000', 'last_became_active': '2024-09-10T15:59:16.300040+0000', 'last_became_peered': '2024-09-10T15:59:16.300040+0000', 'last_unstale': '2024-09-10T16:10:41.212418+0000', 'last_undegraded': '2024-09-10T15:59:15.878234+0000', 'last_fullsized': '2024-09-10T15:59:15.769328+0000', 'mapping_epoch': 42, 'log_start': "45'11000", 'ondisk_log_start': "45'11000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3442", 'last_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'objects_scrubbed': 3384, 'log_size': 10061, 'log_dups_size': 0, 'ondisk_log_size': 10061, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1292, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1866, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 17673, 'num_write_kb': 28580, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2506, 'num_bytes_recovered': 483, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1865}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:10:45.789 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-10T16:10:45.789 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "49'21082", 'reported_seq': 21247, 'reported_epoch': 57, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:10:20.854635+0000', 'last_change': '2024-09-10T15:59:16.331899+0000', 'last_active': '2024-09-10T16:10:20.854635+0000', 'last_peered': '2024-09-10T16:10:20.854635+0000', 'last_clean': '2024-09-10T15:59:14.139436+0000', 'last_became_active': '2024-09-10T15:59:16.318715+0000', 'last_became_peered': '2024-09-10T15:59:16.318715+0000', 'last_unstale': '2024-09-10T16:10:20.854635+0000', 'last_undegraded': '2024-09-10T15:59:15.877524+0000', 'last_fullsized': '2024-09-10T15:59:15.769042+0000', 'mapping_epoch': 42, 'log_start': "45'11000", 'ondisk_log_start': "45'11000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3537", 'last_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'objects_scrubbed': 3497, 'log_size': 10082, 'log_dups_size': 0, 'ondisk_log_size': 10082, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1224, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4237, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 17581, 'num_write_kb': 28172, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4235}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:10:45.789 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-10T16:10:45.789 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "49'21170", 'reported_seq': 30042, 'reported_epoch': 57, 'state': 'active+recovery_wait', 'last_fresh': '2024-09-10T16:10:20.850377+0000', 'last_change': '2024-09-10T16:08:10.929169+0000', 'last_active': '2024-09-10T16:10:20.850377+0000', 'last_peered': '2024-09-10T16:10:20.850377+0000', 'last_clean': '2024-09-10T15:59:14.110654+0000', 'last_became_active': '2024-09-10T16:08:10.860438+0000', 'last_became_peered': '2024-09-10T16:08:10.860438+0000', 'last_unstale': '2024-09-10T16:10:20.850377+0000', 'last_undegraded': '2024-09-10T16:10:20.850377+0000', 'last_fullsized': '2024-09-10T16:10:20.850377+0000', 'mapping_epoch': 48, 'log_start': "45'11100", 'ondisk_log_start': "45'11100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2486", 'last_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'objects_scrubbed': 2440, 'log_size': 10070, 'log_dups_size': 0, 'ondisk_log_size': 10070, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 792, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18724, 'num_write_kb': 32568, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 4355, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': ['2', '0', '1'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:10:45.790 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-10T16:10:45.790 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "49'20838", 'reported_seq': 21000, 'reported_epoch': 57, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:10:20.854744+0000', 'last_change': '2024-09-10T15:59:16.315641+0000', 'last_active': '2024-09-10T16:10:20.854744+0000', 'last_peered': '2024-09-10T16:10:20.854744+0000', 'last_clean': '2024-09-10T15:59:14.117001+0000', 'last_became_active': '2024-09-10T15:59:16.302245+0000', 'last_became_peered': '2024-09-10T15:59:16.302245+0000', 'last_unstale': '2024-09-10T16:10:20.854744+0000', 'last_undegraded': '2024-09-10T15:59:15.845012+0000', 'last_fullsized': '2024-09-10T15:59:15.770337+0000', 'mapping_epoch': 42, 'log_start': "45'10800", 'ondisk_log_start': "45'10800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3404", 'last_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'objects_scrubbed': 3367, 'log_size': 10038, 'log_dups_size': 0, 'ondisk_log_size': 10038, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 983, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17469, 'num_write_kb': 28204, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4304}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:10:45.790 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-10T16:10:45.790 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "49'21048", 'reported_seq': 21217, 'reported_epoch': 57, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:10:20.855031+0000', 'last_change': '2024-09-10T15:59:16.311291+0000', 'last_active': '2024-09-10T16:10:20.855031+0000', 'last_peered': '2024-09-10T16:10:20.855031+0000', 'last_clean': '2024-09-10T15:59:14.116363+0000', 'last_became_active': '2024-09-10T15:59:16.301244+0000', 'last_became_peered': '2024-09-10T15:59:16.301244+0000', 'last_unstale': '2024-09-10T16:10:20.855031+0000', 'last_undegraded': '2024-09-10T15:59:15.883434+0000', 'last_fullsized': '2024-09-10T15:59:15.772756+0000', 'mapping_epoch': 42, 'log_start': "45'11000", 'ondisk_log_start': "45'11000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3789", 'last_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'objects_scrubbed': 3741, 'log_size': 10048, 'log_dups_size': 0, 'ondisk_log_size': 10048, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1440, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4373, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17303, 'num_write_kb': 27124, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4371}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:10:45.790 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-10T16:10:45.790 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "49'21198", 'reported_seq': 21370, 'reported_epoch': 57, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:10:20.854530+0000', 'last_change': '2024-09-10T15:59:16.313409+0000', 'last_active': '2024-09-10T16:10:20.854530+0000', 'last_peered': '2024-09-10T16:10:20.854530+0000', 'last_clean': '2024-09-10T15:59:14.129780+0000', 'last_became_active': '2024-09-10T15:59:16.300932+0000', 'last_became_peered': '2024-09-10T15:59:16.300932+0000', 'last_unstale': '2024-09-10T16:10:20.854530+0000', 'last_undegraded': '2024-09-10T15:59:15.887823+0000', 'last_fullsized': '2024-09-10T15:59:15.768513+0000', 'mapping_epoch': 42, 'log_start': "45'11100", 'ondisk_log_start': "45'11100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3880", 'last_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'objects_scrubbed': 3827, 'log_size': 10098, 'log_dups_size': 0, 'ondisk_log_size': 10098, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1397, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4325, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17361, 'num_write_kb': 27072, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4321}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:10:45.790 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T16:10:46.179 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:10:46.179 INFO:teuthology.orchestra.run.smithi115.stdout:{"fsid":"7effee29-df05-4351-9741-ee203564ed81","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 19106/24 objects degraded (79608.333%), 5 pgs degraded, 5 pgs undersized","count":10},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":810,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":57,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983954,"num_in_osds":3,"osd_in_since":1725983947,"num_remapped_pgs":5},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":18},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":4},{"state_name":"active+recovery_wait","count":1},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":12,"data_bytes":4849,"bytes_used":1574502400,"bytes_avail":288335790080,"bytes_total":289910292480,"degraded_objects":19106,"degraded_total":24,"degraded_ratio":796.08333333333348,"recovering_objects_per_sec":107,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1296,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-10T15:57:14:967120+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T16:00:13.560776+0000","services":{}},"progress_events":{}} 2024-09-10T16:10:46.192 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-10T16:10:46.192 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:10:46.499 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:10:46.500 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:10:46.512 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":455,"stamp":"2024-09-10T16:10:45.786768+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":19106,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292098,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32286,"num_bytes_recovered":33097148,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161025,"ondisk_log_size":161025,"up":48,"acting":43,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1537600,"kb_used_data":206976,"kb_used_omap":12,"kb_used_meta":1330547,"kb_avail":281577920,"statfs":{"total":289910292480,"available":288335790080,"internally_reserved":0,"allocated":211943424,"data_stored":210873193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1362480130},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1296,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1296,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001960"},"pg_stats":[{"pgid":"2.f","version":"49'20672","reported_seq":29300,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.856536+0000","last_change":"2024-09-10T16:09:57.088531+0000","last_active":"2024-09-10T16:10:20.856536+0000","last_peered":"2024-09-10T16:10:20.856536+0000","last_clean":"2024-09-10T16:10:20.856536+0000","last_became_active":"2024-09-10T16:09:57.005755+0000","last_became_peered":"2024-09-10T16:09:57.005755+0000","last_unstale":"2024-09-10T16:10:20.856536+0000","last_undegraded":"2024-09-10T16:10:20.856536+0000","last_fullsized":"2024-09-10T16:10:20.856536+0000","mapping_epoch":54,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:09:19.329697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20672,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4259,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20844","reported_seq":20984,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.850591+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:10:20.850591+0000","last_peered":"2024-09-10T16:10:20.850591+0000","last_clean":"2024-09-10T16:10:20.850591+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:10:20.850591+0000","last_undegraded":"2024-09-10T16:10:20.850591+0000","last_fullsized":"2024-09-10T16:10:20.850591+0000","mapping_epoch":35,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18949,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20798","reported_seq":29503,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.851218+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:10:20.851218+0000","last_peered":"2024-09-10T16:10:20.851218+0000","last_clean":"2024-09-10T16:10:20.851218+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:10:20.851218+0000","last_undegraded":"2024-09-10T16:10:20.851218+0000","last_fullsized":"2024-09-10T16:10:20.851218+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18802,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20660","reported_seq":20768,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.850554+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:10:20.850554+0000","last_peered":"2024-09-10T16:10:20.850554+0000","last_clean":"2024-09-10T16:10:20.850554+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:10:20.850554+0000","last_undegraded":"2024-09-10T16:10:20.850554+0000","last_fullsized":"2024-09-10T16:10:20.850554+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20660,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20891","reported_seq":21048,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.854539+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:10:20.854539+0000","last_peered":"2024-09-10T16:10:20.854539+0000","last_clean":"2024-09-10T16:10:20.854539+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:10:20.854539+0000","last_undegraded":"2024-09-10T16:10:20.854539+0000","last_fullsized":"2024-09-10T16:10:20.854539+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17602,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'21061","reported_seq":26239,"reported_epoch":57,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:10:41.212418+0000","last_change":"2024-09-10T16:10:19.743410+0000","last_active":"2024-09-10T16:10:41.212418+0000","last_peered":"2024-09-10T16:10:41.212418+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:10:41.212418+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1866,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17673,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2506,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1865}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'21082","reported_seq":21247,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:10:20.854635+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:10:20.854635+0000","last_peered":"2024-09-10T16:10:20.854635+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:10:20.854635+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17581,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4235},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'21314","reported_seq":21471,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.854391+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:10:20.854391+0000","last_peered":"2024-09-10T16:10:20.854391+0000","last_clean":"2024-09-10T16:10:20.854391+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:10:20.854391+0000","last_undegraded":"2024-09-10T16:10:20.854391+0000","last_fullsized":"2024-09-10T16:10:20.854391+0000","mapping_epoch":34,"log_start":"45'11300","ondisk_log_start":"45'11300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18161,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.850980+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:10:20.850980+0000","last_peered":"2024-09-10T16:10:20.850980+0000","last_clean":"2024-09-10T16:10:20.850980+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:10:20.850980+0000","last_undegraded":"2024-09-10T16:10:20.850980+0000","last_fullsized":"2024-09-10T16:10:20.850980+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20457","reported_seq":20628,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.851257+0000","last_change":"2024-09-10T16:09:33.012840+0000","last_active":"2024-09-10T16:10:20.851257+0000","last_peered":"2024-09-10T16:10:20.851257+0000","last_clean":"2024-09-10T16:10:20.851257+0000","last_became_active":"2024-09-10T16:09:32.934056+0000","last_became_peered":"2024-09-10T16:09:32.934056+0000","last_unstale":"2024-09-10T16:10:20.851257+0000","last_undegraded":"2024-09-10T16:10:20.851257+0000","last_fullsized":"2024-09-10T16:10:20.851257+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:08:30.481235+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18276,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":4579,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":121,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.850337+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:10:20.850337+0000","last_peered":"2024-09-10T16:10:20.850337+0000","last_clean":"2024-09-10T16:10:20.850337+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:10:20.850337+0000","last_undegraded":"2024-09-10T16:10:20.850337+0000","last_fullsized":"2024-09-10T16:10:20.850337+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20910","reported_seq":29590,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.854908+0000","last_change":"2024-09-10T16:09:08.909580+0000","last_active":"2024-09-10T16:10:20.854908+0000","last_peered":"2024-09-10T16:10:20.854908+0000","last_clean":"2024-09-10T16:10:20.854908+0000","last_became_active":"2024-09-10T16:09:08.854941+0000","last_became_peered":"2024-09-10T16:09:08.854941+0000","last_unstale":"2024-09-10T16:10:20.854908+0000","last_undegraded":"2024-09-10T16:10:20.854908+0000","last_fullsized":"2024-09-10T16:10:20.854908+0000","mapping_epoch":50,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:13:14.846691+0000","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17414,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4257,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":132,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.854803+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:10:20.854803+0000","last_peered":"2024-09-10T16:10:20.854803+0000","last_clean":"2024-09-10T16:10:20.854803+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:10:20.854803+0000","last_undegraded":"2024-09-10T16:10:20.854803+0000","last_fullsized":"2024-09-10T16:10:20.854803+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20781","reported_seq":29561,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.907987+0000","last_change":"2024-09-10T16:10:20.907889+0000","last_active":"2024-09-10T16:10:20.907987+0000","last_peered":"2024-09-10T16:10:20.907987+0000","last_clean":"2024-09-10T16:10:20.907987+0000","last_became_active":"2024-09-10T16:10:20.852065+0000","last_became_peered":"2024-09-10T16:10:20.852065+0000","last_unstale":"2024-09-10T16:10:20.907987+0000","last_undegraded":"2024-09-10T16:10:20.907987+0000","last_fullsized":"2024-09-10T16:10:20.907987+0000","mapping_epoch":56,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:34.069549+0000","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18461,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4307,"num_bytes_recovered":8675,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":107,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.851171+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:10:20.851171+0000","last_peered":"2024-09-10T16:10:20.851171+0000","last_clean":"2024-09-10T16:10:20.851171+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:10:20.851171+0000","last_undegraded":"2024-09-10T16:10:20.851171+0000","last_fullsized":"2024-09-10T16:10:20.851171+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'21170","reported_seq":30042,"reported_epoch":57,"state":"active+recovery_wait","last_fresh":"2024-09-10T16:10:20.850377+0000","last_change":"2024-09-10T16:08:10.929169+0000","last_active":"2024-09-10T16:10:20.850377+0000","last_peered":"2024-09-10T16:10:20.850377+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T16:08:10.860438+0000","last_became_peered":"2024-09-10T16:08:10.860438+0000","last_unstale":"2024-09-10T16:10:20.850377+0000","last_undegraded":"2024-09-10T16:10:20.850377+0000","last_fullsized":"2024-09-10T16:10:20.850377+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18724,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","0","1"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20838","reported_seq":21000,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:10:20.854744+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:10:20.854744+0000","last_peered":"2024-09-10T16:10:20.854744+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:10:20.854744+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17469,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.851195+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:10:20.851195+0000","last_peered":"2024-09-10T16:10:20.851195+0000","last_clean":"2024-09-10T16:10:20.851195+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:10:20.851195+0000","last_undegraded":"2024-09-10T16:10:20.851195+0000","last_fullsized":"2024-09-10T16:10:20.851195+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20598","reported_seq":29191,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.851322+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:10:20.851322+0000","last_peered":"2024-09-10T16:10:20.851322+0000","last_clean":"2024-09-10T16:10:20.851322+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:10:20.851322+0000","last_undegraded":"2024-09-10T16:10:20.851322+0000","last_fullsized":"2024-09-10T16:10:20.851322+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16988,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":121,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.850614+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:10:20.850614+0000","last_peered":"2024-09-10T16:10:20.850614+0000","last_clean":"2024-09-10T16:10:20.850614+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:10:20.850614+0000","last_undegraded":"2024-09-10T16:10:20.850614+0000","last_fullsized":"2024-09-10T16:10:20.850614+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'21048","reported_seq":21217,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:10:20.855031+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:10:20.855031+0000","last_peered":"2024-09-10T16:10:20.855031+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:10:20.855031+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17303,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.851276+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:10:20.851276+0000","last_peered":"2024-09-10T16:10:20.851276+0000","last_clean":"2024-09-10T16:10:20.851276+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:10:20.851276+0000","last_undegraded":"2024-09-10T16:10:20.851276+0000","last_fullsized":"2024-09-10T16:10:20.851276+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":102,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.851340+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:10:20.851340+0000","last_peered":"2024-09-10T16:10:20.851340+0000","last_clean":"2024-09-10T16:10:20.851340+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:10:20.851340+0000","last_undegraded":"2024-09-10T16:10:20.851340+0000","last_fullsized":"2024-09-10T16:10:20.851340+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'21198","reported_seq":21370,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:10:20.854530+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:10:20.854530+0000","last_peered":"2024-09-10T16:10:20.854530+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:10:20.854530+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17361,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4321},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":19106,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292096,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32284,"num_bytes_recovered":33097129,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":210509824,"data_stored":210394208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161022,"ondisk_log_size":161022,"up":32,"acting":27,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986328,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":502664,"kb_used_data":137776,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93869176,"statfs":{"total":96636764160,"available":96122036224,"internally_reserved":0,"allocated":141082624,"data_stored":140731007,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659274,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487572,"kb_used_data":516,"kb_used_omap":4,"kb_used_meta":487035,"kb_avail":93884268,"statfs":{"total":96636764160,"available":96137490432,"internally_reserved":0,"allocated":528384,"data_stored":165464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":498724526},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182556,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":547364,"kb_used_data":68684,"kb_used_omap":4,"kb_used_meta":478651,"kb_avail":93824476,"statfs":{"total":96636764160,"available":96076263424,"internally_reserved":0,"allocated":70332416,"data_stored":69976722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":490139306},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":69853184,"data_stored":69817054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":5796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":140607488,"data_stored":140571358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:10:49.515 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:10:49.833 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:10:49.834 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:10:49.845 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":457,"stamp":"2024-09-10T16:10:49.787459+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":18252,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292098,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33140,"num_bytes_recovered":33097148,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161025,"ondisk_log_size":161025,"up":48,"acting":43,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1537600,"kb_used_data":206976,"kb_used_omap":12,"kb_used_meta":1330547,"kb_avail":281577920,"statfs":{"total":289910292480,"available":288335790080,"internally_reserved":0,"allocated":211943424,"data_stored":210873193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1362480130},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1556,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1556,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002004"},"pg_stats":[{"pgid":"2.f","version":"49'20672","reported_seq":29300,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.856536+0000","last_change":"2024-09-10T16:09:57.088531+0000","last_active":"2024-09-10T16:10:20.856536+0000","last_peered":"2024-09-10T16:10:20.856536+0000","last_clean":"2024-09-10T16:10:20.856536+0000","last_became_active":"2024-09-10T16:09:57.005755+0000","last_became_peered":"2024-09-10T16:09:57.005755+0000","last_unstale":"2024-09-10T16:10:20.856536+0000","last_undegraded":"2024-09-10T16:10:20.856536+0000","last_fullsized":"2024-09-10T16:10:20.856536+0000","mapping_epoch":54,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:09:19.329697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20672,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4259,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20844","reported_seq":20984,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.850591+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:10:20.850591+0000","last_peered":"2024-09-10T16:10:20.850591+0000","last_clean":"2024-09-10T16:10:20.850591+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:10:20.850591+0000","last_undegraded":"2024-09-10T16:10:20.850591+0000","last_fullsized":"2024-09-10T16:10:20.850591+0000","mapping_epoch":35,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18949,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20798","reported_seq":29503,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.851218+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:10:20.851218+0000","last_peered":"2024-09-10T16:10:20.851218+0000","last_clean":"2024-09-10T16:10:20.851218+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:10:20.851218+0000","last_undegraded":"2024-09-10T16:10:20.851218+0000","last_fullsized":"2024-09-10T16:10:20.851218+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18802,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20660","reported_seq":20768,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.850554+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:10:20.850554+0000","last_peered":"2024-09-10T16:10:20.850554+0000","last_clean":"2024-09-10T16:10:20.850554+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:10:20.850554+0000","last_undegraded":"2024-09-10T16:10:20.850554+0000","last_fullsized":"2024-09-10T16:10:20.850554+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20660,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20891","reported_seq":21048,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.854539+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:10:20.854539+0000","last_peered":"2024-09-10T16:10:20.854539+0000","last_clean":"2024-09-10T16:10:20.854539+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:10:20.854539+0000","last_undegraded":"2024-09-10T16:10:20.854539+0000","last_fullsized":"2024-09-10T16:10:20.854539+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17602,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'21061","reported_seq":27947,"reported_epoch":57,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:10:46.214698+0000","last_change":"2024-09-10T16:10:19.743410+0000","last_active":"2024-09-10T16:10:46.214698+0000","last_peered":"2024-09-10T16:10:46.214698+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:10:46.214698+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1012,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17673,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3360,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1011}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'21082","reported_seq":21247,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:10:20.854635+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:10:20.854635+0000","last_peered":"2024-09-10T16:10:20.854635+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:10:20.854635+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17581,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4235},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'21314","reported_seq":21471,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.854391+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:10:20.854391+0000","last_peered":"2024-09-10T16:10:20.854391+0000","last_clean":"2024-09-10T16:10:20.854391+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:10:20.854391+0000","last_undegraded":"2024-09-10T16:10:20.854391+0000","last_fullsized":"2024-09-10T16:10:20.854391+0000","mapping_epoch":34,"log_start":"45'11300","ondisk_log_start":"45'11300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18161,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.850980+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:10:20.850980+0000","last_peered":"2024-09-10T16:10:20.850980+0000","last_clean":"2024-09-10T16:10:20.850980+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:10:20.850980+0000","last_undegraded":"2024-09-10T16:10:20.850980+0000","last_fullsized":"2024-09-10T16:10:20.850980+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20457","reported_seq":20628,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.851257+0000","last_change":"2024-09-10T16:09:33.012840+0000","last_active":"2024-09-10T16:10:20.851257+0000","last_peered":"2024-09-10T16:10:20.851257+0000","last_clean":"2024-09-10T16:10:20.851257+0000","last_became_active":"2024-09-10T16:09:32.934056+0000","last_became_peered":"2024-09-10T16:09:32.934056+0000","last_unstale":"2024-09-10T16:10:20.851257+0000","last_undegraded":"2024-09-10T16:10:20.851257+0000","last_fullsized":"2024-09-10T16:10:20.851257+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:08:30.481235+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18276,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":4579,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":121,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.850337+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:10:20.850337+0000","last_peered":"2024-09-10T16:10:20.850337+0000","last_clean":"2024-09-10T16:10:20.850337+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:10:20.850337+0000","last_undegraded":"2024-09-10T16:10:20.850337+0000","last_fullsized":"2024-09-10T16:10:20.850337+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20910","reported_seq":29590,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.854908+0000","last_change":"2024-09-10T16:09:08.909580+0000","last_active":"2024-09-10T16:10:20.854908+0000","last_peered":"2024-09-10T16:10:20.854908+0000","last_clean":"2024-09-10T16:10:20.854908+0000","last_became_active":"2024-09-10T16:09:08.854941+0000","last_became_peered":"2024-09-10T16:09:08.854941+0000","last_unstale":"2024-09-10T16:10:20.854908+0000","last_undegraded":"2024-09-10T16:10:20.854908+0000","last_fullsized":"2024-09-10T16:10:20.854908+0000","mapping_epoch":50,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:13:14.846691+0000","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17414,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4257,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":132,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.854803+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:10:20.854803+0000","last_peered":"2024-09-10T16:10:20.854803+0000","last_clean":"2024-09-10T16:10:20.854803+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:10:20.854803+0000","last_undegraded":"2024-09-10T16:10:20.854803+0000","last_fullsized":"2024-09-10T16:10:20.854803+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20781","reported_seq":29561,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.907987+0000","last_change":"2024-09-10T16:10:20.907889+0000","last_active":"2024-09-10T16:10:20.907987+0000","last_peered":"2024-09-10T16:10:20.907987+0000","last_clean":"2024-09-10T16:10:20.907987+0000","last_became_active":"2024-09-10T16:10:20.852065+0000","last_became_peered":"2024-09-10T16:10:20.852065+0000","last_unstale":"2024-09-10T16:10:20.907987+0000","last_undegraded":"2024-09-10T16:10:20.907987+0000","last_fullsized":"2024-09-10T16:10:20.907987+0000","mapping_epoch":56,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:34.069549+0000","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18461,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4307,"num_bytes_recovered":8675,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":107,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.851171+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:10:20.851171+0000","last_peered":"2024-09-10T16:10:20.851171+0000","last_clean":"2024-09-10T16:10:20.851171+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:10:20.851171+0000","last_undegraded":"2024-09-10T16:10:20.851171+0000","last_fullsized":"2024-09-10T16:10:20.851171+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'21170","reported_seq":30042,"reported_epoch":57,"state":"active+recovery_wait","last_fresh":"2024-09-10T16:10:20.850377+0000","last_change":"2024-09-10T16:08:10.929169+0000","last_active":"2024-09-10T16:10:20.850377+0000","last_peered":"2024-09-10T16:10:20.850377+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T16:08:10.860438+0000","last_became_peered":"2024-09-10T16:08:10.860438+0000","last_unstale":"2024-09-10T16:10:20.850377+0000","last_undegraded":"2024-09-10T16:10:20.850377+0000","last_fullsized":"2024-09-10T16:10:20.850377+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18724,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","0","1"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20838","reported_seq":21000,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:10:20.854744+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:10:20.854744+0000","last_peered":"2024-09-10T16:10:20.854744+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:10:20.854744+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17469,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.851195+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:10:20.851195+0000","last_peered":"2024-09-10T16:10:20.851195+0000","last_clean":"2024-09-10T16:10:20.851195+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:10:20.851195+0000","last_undegraded":"2024-09-10T16:10:20.851195+0000","last_fullsized":"2024-09-10T16:10:20.851195+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20598","reported_seq":29191,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.851322+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:10:20.851322+0000","last_peered":"2024-09-10T16:10:20.851322+0000","last_clean":"2024-09-10T16:10:20.851322+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:10:20.851322+0000","last_undegraded":"2024-09-10T16:10:20.851322+0000","last_fullsized":"2024-09-10T16:10:20.851322+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16988,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":121,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.850614+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:10:20.850614+0000","last_peered":"2024-09-10T16:10:20.850614+0000","last_clean":"2024-09-10T16:10:20.850614+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:10:20.850614+0000","last_undegraded":"2024-09-10T16:10:20.850614+0000","last_fullsized":"2024-09-10T16:10:20.850614+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'21048","reported_seq":21217,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:10:20.855031+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:10:20.855031+0000","last_peered":"2024-09-10T16:10:20.855031+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:10:20.855031+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17303,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.851276+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:10:20.851276+0000","last_peered":"2024-09-10T16:10:20.851276+0000","last_clean":"2024-09-10T16:10:20.851276+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:10:20.851276+0000","last_undegraded":"2024-09-10T16:10:20.851276+0000","last_fullsized":"2024-09-10T16:10:20.851276+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":102,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.851340+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:10:20.851340+0000","last_peered":"2024-09-10T16:10:20.851340+0000","last_clean":"2024-09-10T16:10:20.851340+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:10:20.851340+0000","last_undegraded":"2024-09-10T16:10:20.851340+0000","last_fullsized":"2024-09-10T16:10:20.851340+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'21198","reported_seq":21370,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:10:20.854530+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:10:20.854530+0000","last_peered":"2024-09-10T16:10:20.854530+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:10:20.854530+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17361,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4321},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":18252,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292096,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33138,"num_bytes_recovered":33097129,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":210509824,"data_stored":210394208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161022,"ondisk_log_size":161022,"up":32,"acting":27,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986329,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":502664,"kb_used_data":137776,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93869176,"statfs":{"total":96636764160,"available":96122036224,"internally_reserved":0,"allocated":141082624,"data_stored":140731007,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659274,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487572,"kb_used_data":516,"kb_used_omap":4,"kb_used_meta":487035,"kb_avail":93884268,"statfs":{"total":96636764160,"available":96137490432,"internally_reserved":0,"allocated":528384,"data_stored":165464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":498724526},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182557,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":547364,"kb_used_data":68684,"kb_used_omap":4,"kb_used_meta":478651,"kb_avail":93824476,"statfs":{"total":96636764160,"available":96076263424,"internally_reserved":0,"allocated":70332416,"data_stored":69976722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":490139306},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":69853184,"data_stored":69817054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":5796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":140607488,"data_stored":140571358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:10:49.847 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:10:50.145 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:10:50.145 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:10:50.160 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":457,"stamp":"2024-09-10T16:10:49.787459+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":18252,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292098,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33140,"num_bytes_recovered":33097148,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161025,"ondisk_log_size":161025,"up":48,"acting":43,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1537600,"kb_used_data":206976,"kb_used_omap":12,"kb_used_meta":1330547,"kb_avail":281577920,"statfs":{"total":289910292480,"available":288335790080,"internally_reserved":0,"allocated":211943424,"data_stored":210873193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1362480130},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1556,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1556,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002004"},"pg_stats":[{"pgid":"2.f","version":"49'20672","reported_seq":29300,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.856536+0000","last_change":"2024-09-10T16:09:57.088531+0000","last_active":"2024-09-10T16:10:20.856536+0000","last_peered":"2024-09-10T16:10:20.856536+0000","last_clean":"2024-09-10T16:10:20.856536+0000","last_became_active":"2024-09-10T16:09:57.005755+0000","last_became_peered":"2024-09-10T16:09:57.005755+0000","last_unstale":"2024-09-10T16:10:20.856536+0000","last_undegraded":"2024-09-10T16:10:20.856536+0000","last_fullsized":"2024-09-10T16:10:20.856536+0000","mapping_epoch":54,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:09:19.329697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20672,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4259,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20844","reported_seq":20984,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.850591+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:10:20.850591+0000","last_peered":"2024-09-10T16:10:20.850591+0000","last_clean":"2024-09-10T16:10:20.850591+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:10:20.850591+0000","last_undegraded":"2024-09-10T16:10:20.850591+0000","last_fullsized":"2024-09-10T16:10:20.850591+0000","mapping_epoch":35,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18949,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20798","reported_seq":29503,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.851218+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:10:20.851218+0000","last_peered":"2024-09-10T16:10:20.851218+0000","last_clean":"2024-09-10T16:10:20.851218+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:10:20.851218+0000","last_undegraded":"2024-09-10T16:10:20.851218+0000","last_fullsized":"2024-09-10T16:10:20.851218+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18802,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20660","reported_seq":20768,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.850554+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:10:20.850554+0000","last_peered":"2024-09-10T16:10:20.850554+0000","last_clean":"2024-09-10T16:10:20.850554+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:10:20.850554+0000","last_undegraded":"2024-09-10T16:10:20.850554+0000","last_fullsized":"2024-09-10T16:10:20.850554+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20660,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20891","reported_seq":21048,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.854539+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:10:20.854539+0000","last_peered":"2024-09-10T16:10:20.854539+0000","last_clean":"2024-09-10T16:10:20.854539+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:10:20.854539+0000","last_undegraded":"2024-09-10T16:10:20.854539+0000","last_fullsized":"2024-09-10T16:10:20.854539+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17602,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'21061","reported_seq":27947,"reported_epoch":57,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:10:46.214698+0000","last_change":"2024-09-10T16:10:19.743410+0000","last_active":"2024-09-10T16:10:46.214698+0000","last_peered":"2024-09-10T16:10:46.214698+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:10:46.214698+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1012,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17673,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3360,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1011}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'21082","reported_seq":21247,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:10:20.854635+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:10:20.854635+0000","last_peered":"2024-09-10T16:10:20.854635+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:10:20.854635+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17581,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4235},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'21314","reported_seq":21471,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.854391+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:10:20.854391+0000","last_peered":"2024-09-10T16:10:20.854391+0000","last_clean":"2024-09-10T16:10:20.854391+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:10:20.854391+0000","last_undegraded":"2024-09-10T16:10:20.854391+0000","last_fullsized":"2024-09-10T16:10:20.854391+0000","mapping_epoch":34,"log_start":"45'11300","ondisk_log_start":"45'11300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18161,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.850980+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:10:20.850980+0000","last_peered":"2024-09-10T16:10:20.850980+0000","last_clean":"2024-09-10T16:10:20.850980+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:10:20.850980+0000","last_undegraded":"2024-09-10T16:10:20.850980+0000","last_fullsized":"2024-09-10T16:10:20.850980+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20457","reported_seq":20628,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.851257+0000","last_change":"2024-09-10T16:09:33.012840+0000","last_active":"2024-09-10T16:10:20.851257+0000","last_peered":"2024-09-10T16:10:20.851257+0000","last_clean":"2024-09-10T16:10:20.851257+0000","last_became_active":"2024-09-10T16:09:32.934056+0000","last_became_peered":"2024-09-10T16:09:32.934056+0000","last_unstale":"2024-09-10T16:10:20.851257+0000","last_undegraded":"2024-09-10T16:10:20.851257+0000","last_fullsized":"2024-09-10T16:10:20.851257+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:08:30.481235+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18276,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":4579,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":121,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.850337+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:10:20.850337+0000","last_peered":"2024-09-10T16:10:20.850337+0000","last_clean":"2024-09-10T16:10:20.850337+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:10:20.850337+0000","last_undegraded":"2024-09-10T16:10:20.850337+0000","last_fullsized":"2024-09-10T16:10:20.850337+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20910","reported_seq":29590,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.854908+0000","last_change":"2024-09-10T16:09:08.909580+0000","last_active":"2024-09-10T16:10:20.854908+0000","last_peered":"2024-09-10T16:10:20.854908+0000","last_clean":"2024-09-10T16:10:20.854908+0000","last_became_active":"2024-09-10T16:09:08.854941+0000","last_became_peered":"2024-09-10T16:09:08.854941+0000","last_unstale":"2024-09-10T16:10:20.854908+0000","last_undegraded":"2024-09-10T16:10:20.854908+0000","last_fullsized":"2024-09-10T16:10:20.854908+0000","mapping_epoch":50,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:13:14.846691+0000","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17414,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4257,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":132,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.854803+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:10:20.854803+0000","last_peered":"2024-09-10T16:10:20.854803+0000","last_clean":"2024-09-10T16:10:20.854803+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:10:20.854803+0000","last_undegraded":"2024-09-10T16:10:20.854803+0000","last_fullsized":"2024-09-10T16:10:20.854803+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20781","reported_seq":29561,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.907987+0000","last_change":"2024-09-10T16:10:20.907889+0000","last_active":"2024-09-10T16:10:20.907987+0000","last_peered":"2024-09-10T16:10:20.907987+0000","last_clean":"2024-09-10T16:10:20.907987+0000","last_became_active":"2024-09-10T16:10:20.852065+0000","last_became_peered":"2024-09-10T16:10:20.852065+0000","last_unstale":"2024-09-10T16:10:20.907987+0000","last_undegraded":"2024-09-10T16:10:20.907987+0000","last_fullsized":"2024-09-10T16:10:20.907987+0000","mapping_epoch":56,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:34.069549+0000","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18461,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4307,"num_bytes_recovered":8675,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":107,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.851171+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:10:20.851171+0000","last_peered":"2024-09-10T16:10:20.851171+0000","last_clean":"2024-09-10T16:10:20.851171+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:10:20.851171+0000","last_undegraded":"2024-09-10T16:10:20.851171+0000","last_fullsized":"2024-09-10T16:10:20.851171+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'21170","reported_seq":30042,"reported_epoch":57,"state":"active+recovery_wait","last_fresh":"2024-09-10T16:10:20.850377+0000","last_change":"2024-09-10T16:08:10.929169+0000","last_active":"2024-09-10T16:10:20.850377+0000","last_peered":"2024-09-10T16:10:20.850377+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T16:08:10.860438+0000","last_became_peered":"2024-09-10T16:08:10.860438+0000","last_unstale":"2024-09-10T16:10:20.850377+0000","last_undegraded":"2024-09-10T16:10:20.850377+0000","last_fullsized":"2024-09-10T16:10:20.850377+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18724,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","0","1"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20838","reported_seq":21000,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:10:20.854744+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:10:20.854744+0000","last_peered":"2024-09-10T16:10:20.854744+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:10:20.854744+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17469,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.851195+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:10:20.851195+0000","last_peered":"2024-09-10T16:10:20.851195+0000","last_clean":"2024-09-10T16:10:20.851195+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:10:20.851195+0000","last_undegraded":"2024-09-10T16:10:20.851195+0000","last_fullsized":"2024-09-10T16:10:20.851195+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20598","reported_seq":29191,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.851322+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:10:20.851322+0000","last_peered":"2024-09-10T16:10:20.851322+0000","last_clean":"2024-09-10T16:10:20.851322+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:10:20.851322+0000","last_undegraded":"2024-09-10T16:10:20.851322+0000","last_fullsized":"2024-09-10T16:10:20.851322+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16988,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":121,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.850614+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:10:20.850614+0000","last_peered":"2024-09-10T16:10:20.850614+0000","last_clean":"2024-09-10T16:10:20.850614+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:10:20.850614+0000","last_undegraded":"2024-09-10T16:10:20.850614+0000","last_fullsized":"2024-09-10T16:10:20.850614+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'21048","reported_seq":21217,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:10:20.855031+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:10:20.855031+0000","last_peered":"2024-09-10T16:10:20.855031+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:10:20.855031+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17303,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.851276+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:10:20.851276+0000","last_peered":"2024-09-10T16:10:20.851276+0000","last_clean":"2024-09-10T16:10:20.851276+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:10:20.851276+0000","last_undegraded":"2024-09-10T16:10:20.851276+0000","last_fullsized":"2024-09-10T16:10:20.851276+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":102,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.851340+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:10:20.851340+0000","last_peered":"2024-09-10T16:10:20.851340+0000","last_clean":"2024-09-10T16:10:20.851340+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:10:20.851340+0000","last_undegraded":"2024-09-10T16:10:20.851340+0000","last_fullsized":"2024-09-10T16:10:20.851340+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'21198","reported_seq":21370,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:10:20.854530+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:10:20.854530+0000","last_peered":"2024-09-10T16:10:20.854530+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:10:20.854530+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17361,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4321},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":18252,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292096,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33138,"num_bytes_recovered":33097129,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":210509824,"data_stored":210394208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161022,"ondisk_log_size":161022,"up":32,"acting":27,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986329,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":502664,"kb_used_data":137776,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93869176,"statfs":{"total":96636764160,"available":96122036224,"internally_reserved":0,"allocated":141082624,"data_stored":140731007,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659274,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487572,"kb_used_data":516,"kb_used_omap":4,"kb_used_meta":487035,"kb_avail":93884268,"statfs":{"total":96636764160,"available":96137490432,"internally_reserved":0,"allocated":528384,"data_stored":165464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":498724526},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182557,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":547364,"kb_used_data":68684,"kb_used_omap":4,"kb_used_meta":478651,"kb_avail":93824476,"statfs":{"total":96636764160,"available":96076263424,"internally_reserved":0,"allocated":70332416,"data_stored":69976722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":490139306},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":69853184,"data_stored":69817054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":5796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":140607488,"data_stored":140571358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:10:50.161 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-10T16:10:50.161 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "49'21061", 'reported_seq': 27947, 'reported_epoch': 57, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:10:46.214698+0000', 'last_change': '2024-09-10T16:10:19.743410+0000', 'last_active': '2024-09-10T16:10:46.214698+0000', 'last_peered': '2024-09-10T16:10:46.214698+0000', 'last_clean': '2024-09-10T15:59:14.111551+0000', 'last_became_active': '2024-09-10T15:59:16.300040+0000', 'last_became_peered': '2024-09-10T15:59:16.300040+0000', 'last_unstale': '2024-09-10T16:10:46.214698+0000', 'last_undegraded': '2024-09-10T15:59:15.878234+0000', 'last_fullsized': '2024-09-10T15:59:15.769328+0000', 'mapping_epoch': 42, 'log_start': "45'11000", 'ondisk_log_start': "45'11000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3442", 'last_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'objects_scrubbed': 3384, 'log_size': 10061, 'log_dups_size': 0, 'ondisk_log_size': 10061, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1292, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1012, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 17673, 'num_write_kb': 28580, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3360, 'num_bytes_recovered': 483, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1011}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:10:50.161 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-10T16:10:50.162 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "49'21082", 'reported_seq': 21247, 'reported_epoch': 57, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:10:20.854635+0000', 'last_change': '2024-09-10T15:59:16.331899+0000', 'last_active': '2024-09-10T16:10:20.854635+0000', 'last_peered': '2024-09-10T16:10:20.854635+0000', 'last_clean': '2024-09-10T15:59:14.139436+0000', 'last_became_active': '2024-09-10T15:59:16.318715+0000', 'last_became_peered': '2024-09-10T15:59:16.318715+0000', 'last_unstale': '2024-09-10T16:10:20.854635+0000', 'last_undegraded': '2024-09-10T15:59:15.877524+0000', 'last_fullsized': '2024-09-10T15:59:15.769042+0000', 'mapping_epoch': 42, 'log_start': "45'11000", 'ondisk_log_start': "45'11000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3537", 'last_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'objects_scrubbed': 3497, 'log_size': 10082, 'log_dups_size': 0, 'ondisk_log_size': 10082, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1224, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4237, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 17581, 'num_write_kb': 28172, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4235}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:10:50.162 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-10T16:10:50.162 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "49'21170", 'reported_seq': 30042, 'reported_epoch': 57, 'state': 'active+recovery_wait', 'last_fresh': '2024-09-10T16:10:20.850377+0000', 'last_change': '2024-09-10T16:08:10.929169+0000', 'last_active': '2024-09-10T16:10:20.850377+0000', 'last_peered': '2024-09-10T16:10:20.850377+0000', 'last_clean': '2024-09-10T15:59:14.110654+0000', 'last_became_active': '2024-09-10T16:08:10.860438+0000', 'last_became_peered': '2024-09-10T16:08:10.860438+0000', 'last_unstale': '2024-09-10T16:10:20.850377+0000', 'last_undegraded': '2024-09-10T16:10:20.850377+0000', 'last_fullsized': '2024-09-10T16:10:20.850377+0000', 'mapping_epoch': 48, 'log_start': "45'11100", 'ondisk_log_start': "45'11100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2486", 'last_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:58:50.078953+0000', 'objects_scrubbed': 2440, 'log_size': 10070, 'log_dups_size': 0, 'ondisk_log_size': 10070, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 792, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18724, 'num_write_kb': 32568, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 4355, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': ['2', '0', '1'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:10:50.162 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-10T16:10:50.162 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "49'20838", 'reported_seq': 21000, 'reported_epoch': 57, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:10:20.854744+0000', 'last_change': '2024-09-10T15:59:16.315641+0000', 'last_active': '2024-09-10T16:10:20.854744+0000', 'last_peered': '2024-09-10T16:10:20.854744+0000', 'last_clean': '2024-09-10T15:59:14.117001+0000', 'last_became_active': '2024-09-10T15:59:16.302245+0000', 'last_became_peered': '2024-09-10T15:59:16.302245+0000', 'last_unstale': '2024-09-10T16:10:20.854744+0000', 'last_undegraded': '2024-09-10T15:59:15.845012+0000', 'last_fullsized': '2024-09-10T15:59:15.770337+0000', 'mapping_epoch': 42, 'log_start': "45'10800", 'ondisk_log_start': "45'10800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3404", 'last_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'objects_scrubbed': 3367, 'log_size': 10038, 'log_dups_size': 0, 'ondisk_log_size': 10038, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 983, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17469, 'num_write_kb': 28204, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4304}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:10:50.162 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-10T16:10:50.162 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "49'21048", 'reported_seq': 21217, 'reported_epoch': 57, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:10:20.855031+0000', 'last_change': '2024-09-10T15:59:16.311291+0000', 'last_active': '2024-09-10T16:10:20.855031+0000', 'last_peered': '2024-09-10T16:10:20.855031+0000', 'last_clean': '2024-09-10T15:59:14.116363+0000', 'last_became_active': '2024-09-10T15:59:16.301244+0000', 'last_became_peered': '2024-09-10T15:59:16.301244+0000', 'last_unstale': '2024-09-10T16:10:20.855031+0000', 'last_undegraded': '2024-09-10T15:59:15.883434+0000', 'last_fullsized': '2024-09-10T15:59:15.772756+0000', 'mapping_epoch': 42, 'log_start': "45'11000", 'ondisk_log_start': "45'11000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3789", 'last_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'objects_scrubbed': 3741, 'log_size': 10048, 'log_dups_size': 0, 'ondisk_log_size': 10048, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1440, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4373, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17303, 'num_write_kb': 27124, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4371}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:10:50.162 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-10T16:10:50.162 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "49'21198", 'reported_seq': 21370, 'reported_epoch': 57, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:10:20.854530+0000', 'last_change': '2024-09-10T15:59:16.313409+0000', 'last_active': '2024-09-10T16:10:20.854530+0000', 'last_peered': '2024-09-10T16:10:20.854530+0000', 'last_clean': '2024-09-10T15:59:14.129780+0000', 'last_became_active': '2024-09-10T15:59:16.300932+0000', 'last_became_peered': '2024-09-10T15:59:16.300932+0000', 'last_unstale': '2024-09-10T16:10:20.854530+0000', 'last_undegraded': '2024-09-10T15:59:15.887823+0000', 'last_fullsized': '2024-09-10T15:59:15.768513+0000', 'mapping_epoch': 42, 'log_start': "45'11100", 'ondisk_log_start': "45'11100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3880", 'last_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'objects_scrubbed': 3827, 'log_size': 10098, 'log_dups_size': 0, 'ondisk_log_size': 10098, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1397, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4325, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17361, 'num_write_kb': 27072, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4321}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:10:50.163 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T16:10:50.547 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:10:50.547 INFO:teuthology.orchestra.run.smithi115.stdout:{"fsid":"7effee29-df05-4351-9741-ee203564ed81","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 18252/24 objects degraded (76050.000%), 5 pgs degraded, 5 pgs undersized","count":10},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":814,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":57,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983954,"num_in_osds":3,"osd_in_since":1725983947,"num_remapped_pgs":5},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":18},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":4},{"state_name":"active+recovery_wait","count":1},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":12,"data_bytes":4849,"bytes_used":1574502400,"bytes_avail":288335790080,"bytes_total":289910292480,"degraded_objects":18252,"degraded_total":24,"degraded_ratio":760.5,"recovering_objects_per_sec":129,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1556,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-10T15:57:14:967120+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T16:00:13.560776+0000","services":{}},"progress_events":{}} 2024-09-10T16:10:50.559 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-10T16:10:50.559 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:10:50.866 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:10:50.867 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:10:50.879 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":457,"stamp":"2024-09-10T16:10:49.787459+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":18252,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292098,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33140,"num_bytes_recovered":33097148,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161025,"ondisk_log_size":161025,"up":48,"acting":43,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1537600,"kb_used_data":206976,"kb_used_omap":12,"kb_used_meta":1330547,"kb_avail":281577920,"statfs":{"total":289910292480,"available":288335790080,"internally_reserved":0,"allocated":211943424,"data_stored":210873193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1362480130},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1556,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1556,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002004"},"pg_stats":[{"pgid":"2.f","version":"49'20672","reported_seq":29300,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.856536+0000","last_change":"2024-09-10T16:09:57.088531+0000","last_active":"2024-09-10T16:10:20.856536+0000","last_peered":"2024-09-10T16:10:20.856536+0000","last_clean":"2024-09-10T16:10:20.856536+0000","last_became_active":"2024-09-10T16:09:57.005755+0000","last_became_peered":"2024-09-10T16:09:57.005755+0000","last_unstale":"2024-09-10T16:10:20.856536+0000","last_undegraded":"2024-09-10T16:10:20.856536+0000","last_fullsized":"2024-09-10T16:10:20.856536+0000","mapping_epoch":54,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:09:19.329697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20672,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4259,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20844","reported_seq":20984,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.850591+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:10:20.850591+0000","last_peered":"2024-09-10T16:10:20.850591+0000","last_clean":"2024-09-10T16:10:20.850591+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:10:20.850591+0000","last_undegraded":"2024-09-10T16:10:20.850591+0000","last_fullsized":"2024-09-10T16:10:20.850591+0000","mapping_epoch":35,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18949,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20798","reported_seq":29503,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.851218+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:10:20.851218+0000","last_peered":"2024-09-10T16:10:20.851218+0000","last_clean":"2024-09-10T16:10:20.851218+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:10:20.851218+0000","last_undegraded":"2024-09-10T16:10:20.851218+0000","last_fullsized":"2024-09-10T16:10:20.851218+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18802,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20660","reported_seq":20768,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.850554+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:10:20.850554+0000","last_peered":"2024-09-10T16:10:20.850554+0000","last_clean":"2024-09-10T16:10:20.850554+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:10:20.850554+0000","last_undegraded":"2024-09-10T16:10:20.850554+0000","last_fullsized":"2024-09-10T16:10:20.850554+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20660,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20891","reported_seq":21048,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.854539+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:10:20.854539+0000","last_peered":"2024-09-10T16:10:20.854539+0000","last_clean":"2024-09-10T16:10:20.854539+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:10:20.854539+0000","last_undegraded":"2024-09-10T16:10:20.854539+0000","last_fullsized":"2024-09-10T16:10:20.854539+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17602,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'21061","reported_seq":27947,"reported_epoch":57,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:10:46.214698+0000","last_change":"2024-09-10T16:10:19.743410+0000","last_active":"2024-09-10T16:10:46.214698+0000","last_peered":"2024-09-10T16:10:46.214698+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:10:46.214698+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1012,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17673,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3360,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1011}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'21082","reported_seq":21247,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:10:20.854635+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:10:20.854635+0000","last_peered":"2024-09-10T16:10:20.854635+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:10:20.854635+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17581,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4235},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'21314","reported_seq":21471,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.854391+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:10:20.854391+0000","last_peered":"2024-09-10T16:10:20.854391+0000","last_clean":"2024-09-10T16:10:20.854391+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:10:20.854391+0000","last_undegraded":"2024-09-10T16:10:20.854391+0000","last_fullsized":"2024-09-10T16:10:20.854391+0000","mapping_epoch":34,"log_start":"45'11300","ondisk_log_start":"45'11300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18161,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.850980+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:10:20.850980+0000","last_peered":"2024-09-10T16:10:20.850980+0000","last_clean":"2024-09-10T16:10:20.850980+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:10:20.850980+0000","last_undegraded":"2024-09-10T16:10:20.850980+0000","last_fullsized":"2024-09-10T16:10:20.850980+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20457","reported_seq":20628,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.851257+0000","last_change":"2024-09-10T16:09:33.012840+0000","last_active":"2024-09-10T16:10:20.851257+0000","last_peered":"2024-09-10T16:10:20.851257+0000","last_clean":"2024-09-10T16:10:20.851257+0000","last_became_active":"2024-09-10T16:09:32.934056+0000","last_became_peered":"2024-09-10T16:09:32.934056+0000","last_unstale":"2024-09-10T16:10:20.851257+0000","last_undegraded":"2024-09-10T16:10:20.851257+0000","last_fullsized":"2024-09-10T16:10:20.851257+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:08:30.481235+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18276,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":4579,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":121,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.850337+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:10:20.850337+0000","last_peered":"2024-09-10T16:10:20.850337+0000","last_clean":"2024-09-10T16:10:20.850337+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:10:20.850337+0000","last_undegraded":"2024-09-10T16:10:20.850337+0000","last_fullsized":"2024-09-10T16:10:20.850337+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20910","reported_seq":29590,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.854908+0000","last_change":"2024-09-10T16:09:08.909580+0000","last_active":"2024-09-10T16:10:20.854908+0000","last_peered":"2024-09-10T16:10:20.854908+0000","last_clean":"2024-09-10T16:10:20.854908+0000","last_became_active":"2024-09-10T16:09:08.854941+0000","last_became_peered":"2024-09-10T16:09:08.854941+0000","last_unstale":"2024-09-10T16:10:20.854908+0000","last_undegraded":"2024-09-10T16:10:20.854908+0000","last_fullsized":"2024-09-10T16:10:20.854908+0000","mapping_epoch":50,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:13:14.846691+0000","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17414,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4257,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":132,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.854803+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:10:20.854803+0000","last_peered":"2024-09-10T16:10:20.854803+0000","last_clean":"2024-09-10T16:10:20.854803+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:10:20.854803+0000","last_undegraded":"2024-09-10T16:10:20.854803+0000","last_fullsized":"2024-09-10T16:10:20.854803+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20781","reported_seq":29561,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.907987+0000","last_change":"2024-09-10T16:10:20.907889+0000","last_active":"2024-09-10T16:10:20.907987+0000","last_peered":"2024-09-10T16:10:20.907987+0000","last_clean":"2024-09-10T16:10:20.907987+0000","last_became_active":"2024-09-10T16:10:20.852065+0000","last_became_peered":"2024-09-10T16:10:20.852065+0000","last_unstale":"2024-09-10T16:10:20.907987+0000","last_undegraded":"2024-09-10T16:10:20.907987+0000","last_fullsized":"2024-09-10T16:10:20.907987+0000","mapping_epoch":56,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:34.069549+0000","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18461,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4307,"num_bytes_recovered":8675,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":107,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.851171+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:10:20.851171+0000","last_peered":"2024-09-10T16:10:20.851171+0000","last_clean":"2024-09-10T16:10:20.851171+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:10:20.851171+0000","last_undegraded":"2024-09-10T16:10:20.851171+0000","last_fullsized":"2024-09-10T16:10:20.851171+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'21170","reported_seq":30042,"reported_epoch":57,"state":"active+recovery_wait","last_fresh":"2024-09-10T16:10:20.850377+0000","last_change":"2024-09-10T16:08:10.929169+0000","last_active":"2024-09-10T16:10:20.850377+0000","last_peered":"2024-09-10T16:10:20.850377+0000","last_clean":"2024-09-10T15:59:14.110654+0000","last_became_active":"2024-09-10T16:08:10.860438+0000","last_became_peered":"2024-09-10T16:08:10.860438+0000","last_unstale":"2024-09-10T16:10:20.850377+0000","last_undegraded":"2024-09-10T16:10:20.850377+0000","last_fullsized":"2024-09-10T16:10:20.850377+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18724,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","0","1"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20838","reported_seq":21000,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:10:20.854744+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:10:20.854744+0000","last_peered":"2024-09-10T16:10:20.854744+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:10:20.854744+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17469,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.851195+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:10:20.851195+0000","last_peered":"2024-09-10T16:10:20.851195+0000","last_clean":"2024-09-10T16:10:20.851195+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:10:20.851195+0000","last_undegraded":"2024-09-10T16:10:20.851195+0000","last_fullsized":"2024-09-10T16:10:20.851195+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20598","reported_seq":29191,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.851322+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:10:20.851322+0000","last_peered":"2024-09-10T16:10:20.851322+0000","last_clean":"2024-09-10T16:10:20.851322+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:10:20.851322+0000","last_undegraded":"2024-09-10T16:10:20.851322+0000","last_fullsized":"2024-09-10T16:10:20.851322+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16988,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":121,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.850614+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:10:20.850614+0000","last_peered":"2024-09-10T16:10:20.850614+0000","last_clean":"2024-09-10T16:10:20.850614+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:10:20.850614+0000","last_undegraded":"2024-09-10T16:10:20.850614+0000","last_fullsized":"2024-09-10T16:10:20.850614+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'21048","reported_seq":21217,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:10:20.855031+0000","last_change":"2024-09-10T15:59:16.311291+0000","last_active":"2024-09-10T16:10:20.855031+0000","last_peered":"2024-09-10T16:10:20.855031+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:10:20.855031+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17303,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.851276+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:10:20.851276+0000","last_peered":"2024-09-10T16:10:20.851276+0000","last_clean":"2024-09-10T16:10:20.851276+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:10:20.851276+0000","last_undegraded":"2024-09-10T16:10:20.851276+0000","last_fullsized":"2024-09-10T16:10:20.851276+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":102,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.851340+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:10:20.851340+0000","last_peered":"2024-09-10T16:10:20.851340+0000","last_clean":"2024-09-10T16:10:20.851340+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:10:20.851340+0000","last_undegraded":"2024-09-10T16:10:20.851340+0000","last_fullsized":"2024-09-10T16:10:20.851340+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'21198","reported_seq":21370,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:10:20.854530+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:10:20.854530+0000","last_peered":"2024-09-10T16:10:20.854530+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:10:20.854530+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17361,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4321},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":18252,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292096,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33138,"num_bytes_recovered":33097129,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":210509824,"data_stored":210394208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161022,"ondisk_log_size":161022,"up":32,"acting":27,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986329,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":502664,"kb_used_data":137776,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93869176,"statfs":{"total":96636764160,"available":96122036224,"internally_reserved":0,"allocated":141082624,"data_stored":140731007,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659274,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487572,"kb_used_data":516,"kb_used_omap":4,"kb_used_meta":487035,"kb_avail":93884268,"statfs":{"total":96636764160,"available":96137490432,"internally_reserved":0,"allocated":528384,"data_stored":165464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":498724526},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182557,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":547364,"kb_used_data":68684,"kb_used_omap":4,"kb_used_meta":478651,"kb_avail":93824476,"statfs":{"total":96636764160,"available":96076263424,"internally_reserved":0,"allocated":70332416,"data_stored":69976722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":490139306},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":69853184,"data_stored":69817054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":5796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":140607488,"data_stored":140571358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:10:53.882 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:10:54.205 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:10:54.205 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:10:54.215 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":461,"stamp":"2024-09-10T16:10:53.788173+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":18200,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292098,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33192,"num_bytes_recovered":33097148,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161025,"ondisk_log_size":161025,"up":48,"acting":43,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1529280,"kb_used_data":206976,"kb_used_omap":12,"kb_used_meta":1322227,"kb_avail":281586240,"statfs":{"total":289910292480,"available":288344309760,"internally_reserved":0,"allocated":211943424,"data_stored":210873193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1353960450},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-906,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":906,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001404"},"pg_stats":[{"pgid":"2.f","version":"49'20672","reported_seq":29302,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:10:51.161692+0000","last_change":"2024-09-10T16:09:57.088531+0000","last_active":"2024-09-10T16:10:51.161692+0000","last_peered":"2024-09-10T16:10:51.161692+0000","last_clean":"2024-09-10T16:10:51.161692+0000","last_became_active":"2024-09-10T16:09:57.005755+0000","last_became_peered":"2024-09-10T16:09:57.005755+0000","last_unstale":"2024-09-10T16:10:51.161692+0000","last_undegraded":"2024-09-10T16:10:51.161692+0000","last_fullsized":"2024-09-10T16:10:51.161692+0000","mapping_epoch":54,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:09:19.329697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20672,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4259,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20844","reported_seq":20986,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:10:51.159480+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:10:51.159480+0000","last_peered":"2024-09-10T16:10:51.159480+0000","last_clean":"2024-09-10T16:10:51.159480+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:10:51.159480+0000","last_undegraded":"2024-09-10T16:10:51.159480+0000","last_fullsized":"2024-09-10T16:10:51.159480+0000","mapping_epoch":35,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18949,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20798","reported_seq":29503,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.851218+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:10:20.851218+0000","last_peered":"2024-09-10T16:10:20.851218+0000","last_clean":"2024-09-10T16:10:20.851218+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:10:20.851218+0000","last_undegraded":"2024-09-10T16:10:20.851218+0000","last_fullsized":"2024-09-10T16:10:20.851218+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18802,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20660","reported_seq":20770,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:10:51.159450+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:10:51.159450+0000","last_peered":"2024-09-10T16:10:51.159450+0000","last_clean":"2024-09-10T16:10:51.159450+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:10:51.159450+0000","last_undegraded":"2024-09-10T16:10:51.159450+0000","last_fullsized":"2024-09-10T16:10:51.159450+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20660,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20891","reported_seq":21050,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:10:51.161243+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:10:51.161243+0000","last_peered":"2024-09-10T16:10:51.161243+0000","last_clean":"2024-09-10T16:10:51.161243+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:10:51.161243+0000","last_undegraded":"2024-09-10T16:10:51.161243+0000","last_fullsized":"2024-09-10T16:10:51.161243+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17602,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'21061","reported_seq":27947,"reported_epoch":57,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:10:46.214698+0000","last_change":"2024-09-10T16:10:19.743410+0000","last_active":"2024-09-10T16:10:46.214698+0000","last_peered":"2024-09-10T16:10:46.214698+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:10:46.214698+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1012,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17673,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3360,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1011}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'21082","reported_seq":21249,"reported_epoch":58,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:10:51.160046+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:10:51.160046+0000","last_peered":"2024-09-10T16:10:51.160046+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:10:51.160046+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17581,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4235},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'21314","reported_seq":21473,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:10:51.159983+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:10:51.159983+0000","last_peered":"2024-09-10T16:10:51.159983+0000","last_clean":"2024-09-10T16:10:51.159983+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:10:51.159983+0000","last_undegraded":"2024-09-10T16:10:51.159983+0000","last_fullsized":"2024-09-10T16:10:51.159983+0000","mapping_epoch":34,"log_start":"45'11300","ondisk_log_start":"45'11300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18161,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.850980+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:10:20.850980+0000","last_peered":"2024-09-10T16:10:20.850980+0000","last_clean":"2024-09-10T16:10:20.850980+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:10:20.850980+0000","last_undegraded":"2024-09-10T16:10:20.850980+0000","last_fullsized":"2024-09-10T16:10:20.850980+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20457","reported_seq":20628,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.851257+0000","last_change":"2024-09-10T16:09:33.012840+0000","last_active":"2024-09-10T16:10:20.851257+0000","last_peered":"2024-09-10T16:10:20.851257+0000","last_clean":"2024-09-10T16:10:20.851257+0000","last_became_active":"2024-09-10T16:09:32.934056+0000","last_became_peered":"2024-09-10T16:09:32.934056+0000","last_unstale":"2024-09-10T16:10:20.851257+0000","last_undegraded":"2024-09-10T16:10:20.851257+0000","last_fullsized":"2024-09-10T16:10:20.851257+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:08:30.481235+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18276,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":4579,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":124,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:10:51.159327+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:10:51.159327+0000","last_peered":"2024-09-10T16:10:51.159327+0000","last_clean":"2024-09-10T16:10:51.159327+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:10:51.159327+0000","last_undegraded":"2024-09-10T16:10:51.159327+0000","last_fullsized":"2024-09-10T16:10:51.159327+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20910","reported_seq":29592,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:10:51.160977+0000","last_change":"2024-09-10T16:09:08.909580+0000","last_active":"2024-09-10T16:10:51.160977+0000","last_peered":"2024-09-10T16:10:51.160977+0000","last_clean":"2024-09-10T16:10:51.160977+0000","last_became_active":"2024-09-10T16:09:08.854941+0000","last_became_peered":"2024-09-10T16:09:08.854941+0000","last_unstale":"2024-09-10T16:10:51.160977+0000","last_undegraded":"2024-09-10T16:10:51.160977+0000","last_fullsized":"2024-09-10T16:10:51.160977+0000","mapping_epoch":50,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:13:14.846691+0000","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17414,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4257,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":134,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:10:51.160698+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:10:51.160698+0000","last_peered":"2024-09-10T16:10:51.160698+0000","last_clean":"2024-09-10T16:10:51.160698+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:10:51.160698+0000","last_undegraded":"2024-09-10T16:10:51.160698+0000","last_fullsized":"2024-09-10T16:10:51.160698+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20781","reported_seq":29564,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:10:51.159696+0000","last_change":"2024-09-10T16:10:20.907889+0000","last_active":"2024-09-10T16:10:51.159696+0000","last_peered":"2024-09-10T16:10:51.159696+0000","last_clean":"2024-09-10T16:10:51.159696+0000","last_became_active":"2024-09-10T16:10:20.852065+0000","last_became_peered":"2024-09-10T16:10:20.852065+0000","last_unstale":"2024-09-10T16:10:51.159696+0000","last_undegraded":"2024-09-10T16:10:51.159696+0000","last_fullsized":"2024-09-10T16:10:51.159696+0000","mapping_epoch":56,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:34.069549+0000","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18461,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4307,"num_bytes_recovered":8675,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":107,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.851171+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:10:20.851171+0000","last_peered":"2024-09-10T16:10:20.851171+0000","last_clean":"2024-09-10T16:10:20.851171+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:10:20.851171+0000","last_undegraded":"2024-09-10T16:10:20.851171+0000","last_fullsized":"2024-09-10T16:10:20.851171+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'21170","reported_seq":30049,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:10:51.159183+0000","last_change":"2024-09-10T16:10:50.728349+0000","last_active":"2024-09-10T16:10:51.159183+0000","last_peered":"2024-09-10T16:10:51.159183+0000","last_clean":"2024-09-10T16:10:51.159183+0000","last_became_active":"2024-09-10T16:08:10.860438+0000","last_became_peered":"2024-09-10T16:08:10.860438+0000","last_unstale":"2024-09-10T16:10:51.159183+0000","last_undegraded":"2024-09-10T16:10:51.159183+0000","last_fullsized":"2024-09-10T16:10:51.159183+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:38:13.781380+0000","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18724,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20838","reported_seq":21002,"reported_epoch":58,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:10:51.161052+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:10:51.161052+0000","last_peered":"2024-09-10T16:10:51.161052+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:10:51.161052+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17469,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.851195+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:10:20.851195+0000","last_peered":"2024-09-10T16:10:20.851195+0000","last_clean":"2024-09-10T16:10:20.851195+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:10:20.851195+0000","last_undegraded":"2024-09-10T16:10:20.851195+0000","last_fullsized":"2024-09-10T16:10:20.851195+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20598","reported_seq":29191,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.851322+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:10:20.851322+0000","last_peered":"2024-09-10T16:10:20.851322+0000","last_clean":"2024-09-10T16:10:20.851322+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:10:20.851322+0000","last_undegraded":"2024-09-10T16:10:20.851322+0000","last_fullsized":"2024-09-10T16:10:20.851322+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16988,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":124,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:10:51.159516+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:10:51.159516+0000","last_peered":"2024-09-10T16:10:51.159516+0000","last_clean":"2024-09-10T16:10:51.159516+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:10:51.159516+0000","last_undegraded":"2024-09-10T16:10:51.159516+0000","last_fullsized":"2024-09-10T16:10:51.159516+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'21048","reported_seq":21325,"reported_epoch":58,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:10:51.224441+0000","last_change":"2024-09-10T16:10:50.728566+0000","last_active":"2024-09-10T16:10:51.224441+0000","last_peered":"2024-09-10T16:10:51.224441+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:10:51.224441+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4321,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17303,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":52,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4321}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.851276+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:10:20.851276+0000","last_peered":"2024-09-10T16:10:20.851276+0000","last_clean":"2024-09-10T16:10:20.851276+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:10:20.851276+0000","last_undegraded":"2024-09-10T16:10:20.851276+0000","last_fullsized":"2024-09-10T16:10:20.851276+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":102,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.851340+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:10:20.851340+0000","last_peered":"2024-09-10T16:10:20.851340+0000","last_clean":"2024-09-10T16:10:20.851340+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:10:20.851340+0000","last_undegraded":"2024-09-10T16:10:20.851340+0000","last_fullsized":"2024-09-10T16:10:20.851340+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'21198","reported_seq":21372,"reported_epoch":58,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:10:51.161437+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:10:51.161437+0000","last_peered":"2024-09-10T16:10:51.161437+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:10:51.161437+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17361,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4321},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":18200,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292096,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33190,"num_bytes_recovered":33097129,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":210153472,"data_stored":210037856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161022,"ondisk_log_size":161022,"up":32,"acting":27,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986330,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":502664,"kb_used_data":137776,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93869176,"statfs":{"total":96636764160,"available":96122036224,"internally_reserved":0,"allocated":141082624,"data_stored":140731007,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659275,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":479252,"kb_used_data":516,"kb_used_omap":4,"kb_used_meta":478715,"kb_avail":93892588,"statfs":{"total":96636764160,"available":96146010112,"internally_reserved":0,"allocated":528384,"data_stored":165464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":490204846},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182558,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":547364,"kb_used_data":68684,"kb_used_omap":4,"kb_used_meta":478651,"kb_avail":93824476,"statfs":{"total":96636764160,"available":96076263424,"internally_reserved":0,"allocated":70332416,"data_stored":69976722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":490139306},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":69853184,"data_stored":69817054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":5796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":140251136,"data_stored":140215006,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:10:54.217 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:10:54.528 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:10:54.529 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:10:54.541 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":461,"stamp":"2024-09-10T16:10:53.788173+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":18200,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292098,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33192,"num_bytes_recovered":33097148,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161025,"ondisk_log_size":161025,"up":48,"acting":43,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1529280,"kb_used_data":206976,"kb_used_omap":12,"kb_used_meta":1322227,"kb_avail":281586240,"statfs":{"total":289910292480,"available":288344309760,"internally_reserved":0,"allocated":211943424,"data_stored":210873193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1353960450},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-906,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":906,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001404"},"pg_stats":[{"pgid":"2.f","version":"49'20672","reported_seq":29302,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:10:51.161692+0000","last_change":"2024-09-10T16:09:57.088531+0000","last_active":"2024-09-10T16:10:51.161692+0000","last_peered":"2024-09-10T16:10:51.161692+0000","last_clean":"2024-09-10T16:10:51.161692+0000","last_became_active":"2024-09-10T16:09:57.005755+0000","last_became_peered":"2024-09-10T16:09:57.005755+0000","last_unstale":"2024-09-10T16:10:51.161692+0000","last_undegraded":"2024-09-10T16:10:51.161692+0000","last_fullsized":"2024-09-10T16:10:51.161692+0000","mapping_epoch":54,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:09:19.329697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20672,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4259,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20844","reported_seq":20986,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:10:51.159480+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:10:51.159480+0000","last_peered":"2024-09-10T16:10:51.159480+0000","last_clean":"2024-09-10T16:10:51.159480+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:10:51.159480+0000","last_undegraded":"2024-09-10T16:10:51.159480+0000","last_fullsized":"2024-09-10T16:10:51.159480+0000","mapping_epoch":35,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18949,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20798","reported_seq":29503,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.851218+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:10:20.851218+0000","last_peered":"2024-09-10T16:10:20.851218+0000","last_clean":"2024-09-10T16:10:20.851218+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:10:20.851218+0000","last_undegraded":"2024-09-10T16:10:20.851218+0000","last_fullsized":"2024-09-10T16:10:20.851218+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18802,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20660","reported_seq":20770,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:10:51.159450+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:10:51.159450+0000","last_peered":"2024-09-10T16:10:51.159450+0000","last_clean":"2024-09-10T16:10:51.159450+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:10:51.159450+0000","last_undegraded":"2024-09-10T16:10:51.159450+0000","last_fullsized":"2024-09-10T16:10:51.159450+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20660,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20891","reported_seq":21050,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:10:51.161243+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:10:51.161243+0000","last_peered":"2024-09-10T16:10:51.161243+0000","last_clean":"2024-09-10T16:10:51.161243+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:10:51.161243+0000","last_undegraded":"2024-09-10T16:10:51.161243+0000","last_fullsized":"2024-09-10T16:10:51.161243+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17602,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'21061","reported_seq":27947,"reported_epoch":57,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:10:46.214698+0000","last_change":"2024-09-10T16:10:19.743410+0000","last_active":"2024-09-10T16:10:46.214698+0000","last_peered":"2024-09-10T16:10:46.214698+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:10:46.214698+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1012,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17673,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3360,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1011}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'21082","reported_seq":21249,"reported_epoch":58,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:10:51.160046+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:10:51.160046+0000","last_peered":"2024-09-10T16:10:51.160046+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:10:51.160046+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17581,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4235},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'21314","reported_seq":21473,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:10:51.159983+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:10:51.159983+0000","last_peered":"2024-09-10T16:10:51.159983+0000","last_clean":"2024-09-10T16:10:51.159983+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:10:51.159983+0000","last_undegraded":"2024-09-10T16:10:51.159983+0000","last_fullsized":"2024-09-10T16:10:51.159983+0000","mapping_epoch":34,"log_start":"45'11300","ondisk_log_start":"45'11300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18161,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.850980+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:10:20.850980+0000","last_peered":"2024-09-10T16:10:20.850980+0000","last_clean":"2024-09-10T16:10:20.850980+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:10:20.850980+0000","last_undegraded":"2024-09-10T16:10:20.850980+0000","last_fullsized":"2024-09-10T16:10:20.850980+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20457","reported_seq":20628,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.851257+0000","last_change":"2024-09-10T16:09:33.012840+0000","last_active":"2024-09-10T16:10:20.851257+0000","last_peered":"2024-09-10T16:10:20.851257+0000","last_clean":"2024-09-10T16:10:20.851257+0000","last_became_active":"2024-09-10T16:09:32.934056+0000","last_became_peered":"2024-09-10T16:09:32.934056+0000","last_unstale":"2024-09-10T16:10:20.851257+0000","last_undegraded":"2024-09-10T16:10:20.851257+0000","last_fullsized":"2024-09-10T16:10:20.851257+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:08:30.481235+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18276,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":4579,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":124,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:10:51.159327+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:10:51.159327+0000","last_peered":"2024-09-10T16:10:51.159327+0000","last_clean":"2024-09-10T16:10:51.159327+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:10:51.159327+0000","last_undegraded":"2024-09-10T16:10:51.159327+0000","last_fullsized":"2024-09-10T16:10:51.159327+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20910","reported_seq":29592,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:10:51.160977+0000","last_change":"2024-09-10T16:09:08.909580+0000","last_active":"2024-09-10T16:10:51.160977+0000","last_peered":"2024-09-10T16:10:51.160977+0000","last_clean":"2024-09-10T16:10:51.160977+0000","last_became_active":"2024-09-10T16:09:08.854941+0000","last_became_peered":"2024-09-10T16:09:08.854941+0000","last_unstale":"2024-09-10T16:10:51.160977+0000","last_undegraded":"2024-09-10T16:10:51.160977+0000","last_fullsized":"2024-09-10T16:10:51.160977+0000","mapping_epoch":50,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:13:14.846691+0000","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17414,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4257,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":134,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:10:51.160698+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:10:51.160698+0000","last_peered":"2024-09-10T16:10:51.160698+0000","last_clean":"2024-09-10T16:10:51.160698+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:10:51.160698+0000","last_undegraded":"2024-09-10T16:10:51.160698+0000","last_fullsized":"2024-09-10T16:10:51.160698+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20781","reported_seq":29564,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:10:51.159696+0000","last_change":"2024-09-10T16:10:20.907889+0000","last_active":"2024-09-10T16:10:51.159696+0000","last_peered":"2024-09-10T16:10:51.159696+0000","last_clean":"2024-09-10T16:10:51.159696+0000","last_became_active":"2024-09-10T16:10:20.852065+0000","last_became_peered":"2024-09-10T16:10:20.852065+0000","last_unstale":"2024-09-10T16:10:51.159696+0000","last_undegraded":"2024-09-10T16:10:51.159696+0000","last_fullsized":"2024-09-10T16:10:51.159696+0000","mapping_epoch":56,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:34.069549+0000","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18461,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4307,"num_bytes_recovered":8675,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":107,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.851171+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:10:20.851171+0000","last_peered":"2024-09-10T16:10:20.851171+0000","last_clean":"2024-09-10T16:10:20.851171+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:10:20.851171+0000","last_undegraded":"2024-09-10T16:10:20.851171+0000","last_fullsized":"2024-09-10T16:10:20.851171+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'21170","reported_seq":30049,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:10:51.159183+0000","last_change":"2024-09-10T16:10:50.728349+0000","last_active":"2024-09-10T16:10:51.159183+0000","last_peered":"2024-09-10T16:10:51.159183+0000","last_clean":"2024-09-10T16:10:51.159183+0000","last_became_active":"2024-09-10T16:08:10.860438+0000","last_became_peered":"2024-09-10T16:08:10.860438+0000","last_unstale":"2024-09-10T16:10:51.159183+0000","last_undegraded":"2024-09-10T16:10:51.159183+0000","last_fullsized":"2024-09-10T16:10:51.159183+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:38:13.781380+0000","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18724,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20838","reported_seq":21002,"reported_epoch":58,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:10:51.161052+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:10:51.161052+0000","last_peered":"2024-09-10T16:10:51.161052+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:10:51.161052+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17469,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.851195+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:10:20.851195+0000","last_peered":"2024-09-10T16:10:20.851195+0000","last_clean":"2024-09-10T16:10:20.851195+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:10:20.851195+0000","last_undegraded":"2024-09-10T16:10:20.851195+0000","last_fullsized":"2024-09-10T16:10:20.851195+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20598","reported_seq":29191,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.851322+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:10:20.851322+0000","last_peered":"2024-09-10T16:10:20.851322+0000","last_clean":"2024-09-10T16:10:20.851322+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:10:20.851322+0000","last_undegraded":"2024-09-10T16:10:20.851322+0000","last_fullsized":"2024-09-10T16:10:20.851322+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16988,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":124,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:10:51.159516+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:10:51.159516+0000","last_peered":"2024-09-10T16:10:51.159516+0000","last_clean":"2024-09-10T16:10:51.159516+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:10:51.159516+0000","last_undegraded":"2024-09-10T16:10:51.159516+0000","last_fullsized":"2024-09-10T16:10:51.159516+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'21048","reported_seq":21325,"reported_epoch":58,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:10:51.224441+0000","last_change":"2024-09-10T16:10:50.728566+0000","last_active":"2024-09-10T16:10:51.224441+0000","last_peered":"2024-09-10T16:10:51.224441+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:10:51.224441+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4321,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17303,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":52,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4321}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.851276+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:10:20.851276+0000","last_peered":"2024-09-10T16:10:20.851276+0000","last_clean":"2024-09-10T16:10:20.851276+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:10:20.851276+0000","last_undegraded":"2024-09-10T16:10:20.851276+0000","last_fullsized":"2024-09-10T16:10:20.851276+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":102,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.851340+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:10:20.851340+0000","last_peered":"2024-09-10T16:10:20.851340+0000","last_clean":"2024-09-10T16:10:20.851340+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:10:20.851340+0000","last_undegraded":"2024-09-10T16:10:20.851340+0000","last_fullsized":"2024-09-10T16:10:20.851340+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'21198","reported_seq":21372,"reported_epoch":58,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:10:51.161437+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:10:51.161437+0000","last_peered":"2024-09-10T16:10:51.161437+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:10:51.161437+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17361,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4321},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":18200,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292096,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33190,"num_bytes_recovered":33097129,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":210153472,"data_stored":210037856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161022,"ondisk_log_size":161022,"up":32,"acting":27,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986330,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":502664,"kb_used_data":137776,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93869176,"statfs":{"total":96636764160,"available":96122036224,"internally_reserved":0,"allocated":141082624,"data_stored":140731007,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659275,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":479252,"kb_used_data":516,"kb_used_omap":4,"kb_used_meta":478715,"kb_avail":93892588,"statfs":{"total":96636764160,"available":96146010112,"internally_reserved":0,"allocated":528384,"data_stored":165464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":490204846},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182558,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":547364,"kb_used_data":68684,"kb_used_omap":4,"kb_used_meta":478651,"kb_avail":93824476,"statfs":{"total":96636764160,"available":96076263424,"internally_reserved":0,"allocated":70332416,"data_stored":69976722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":490139306},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":69853184,"data_stored":69817054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":5796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":140251136,"data_stored":140215006,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:10:54.542 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-10T16:10:54.543 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "49'21061", 'reported_seq': 27947, 'reported_epoch': 57, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:10:46.214698+0000', 'last_change': '2024-09-10T16:10:19.743410+0000', 'last_active': '2024-09-10T16:10:46.214698+0000', 'last_peered': '2024-09-10T16:10:46.214698+0000', 'last_clean': '2024-09-10T15:59:14.111551+0000', 'last_became_active': '2024-09-10T15:59:16.300040+0000', 'last_became_peered': '2024-09-10T15:59:16.300040+0000', 'last_unstale': '2024-09-10T16:10:46.214698+0000', 'last_undegraded': '2024-09-10T15:59:15.878234+0000', 'last_fullsized': '2024-09-10T15:59:15.769328+0000', 'mapping_epoch': 42, 'log_start': "45'11000", 'ondisk_log_start': "45'11000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3442", 'last_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.194055+0000', 'objects_scrubbed': 3384, 'log_size': 10061, 'log_dups_size': 0, 'ondisk_log_size': 10061, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1292, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1012, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 17673, 'num_write_kb': 28580, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3360, 'num_bytes_recovered': 483, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1011}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:10:54.543 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-10T16:10:54.543 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "49'21082", 'reported_seq': 21249, 'reported_epoch': 58, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:10:51.160046+0000', 'last_change': '2024-09-10T15:59:16.331899+0000', 'last_active': '2024-09-10T16:10:51.160046+0000', 'last_peered': '2024-09-10T16:10:51.160046+0000', 'last_clean': '2024-09-10T15:59:14.139436+0000', 'last_became_active': '2024-09-10T15:59:16.318715+0000', 'last_became_peered': '2024-09-10T15:59:16.318715+0000', 'last_unstale': '2024-09-10T16:10:51.160046+0000', 'last_undegraded': '2024-09-10T15:59:15.877524+0000', 'last_fullsized': '2024-09-10T15:59:15.769042+0000', 'mapping_epoch': 42, 'log_start': "45'11000", 'ondisk_log_start': "45'11000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3537", 'last_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'objects_scrubbed': 3497, 'log_size': 10082, 'log_dups_size': 0, 'ondisk_log_size': 10082, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1224, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4237, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 17581, 'num_write_kb': 28172, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4235}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:10:54.543 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-10T16:10:54.543 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "49'20838", 'reported_seq': 21002, 'reported_epoch': 58, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:10:51.161052+0000', 'last_change': '2024-09-10T15:59:16.315641+0000', 'last_active': '2024-09-10T16:10:51.161052+0000', 'last_peered': '2024-09-10T16:10:51.161052+0000', 'last_clean': '2024-09-10T15:59:14.117001+0000', 'last_became_active': '2024-09-10T15:59:16.302245+0000', 'last_became_peered': '2024-09-10T15:59:16.302245+0000', 'last_unstale': '2024-09-10T16:10:51.161052+0000', 'last_undegraded': '2024-09-10T15:59:15.845012+0000', 'last_fullsized': '2024-09-10T15:59:15.770337+0000', 'mapping_epoch': 42, 'log_start': "45'10800", 'ondisk_log_start': "45'10800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3404", 'last_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'objects_scrubbed': 3367, 'log_size': 10038, 'log_dups_size': 0, 'ondisk_log_size': 10038, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 983, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17469, 'num_write_kb': 28204, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4304}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:10:54.543 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-10T16:10:54.543 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "49'21048", 'reported_seq': 21325, 'reported_epoch': 58, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:10:51.224441+0000', 'last_change': '2024-09-10T16:10:50.728566+0000', 'last_active': '2024-09-10T16:10:51.224441+0000', 'last_peered': '2024-09-10T16:10:51.224441+0000', 'last_clean': '2024-09-10T15:59:14.116363+0000', 'last_became_active': '2024-09-10T15:59:16.301244+0000', 'last_became_peered': '2024-09-10T15:59:16.301244+0000', 'last_unstale': '2024-09-10T16:10:51.224441+0000', 'last_undegraded': '2024-09-10T15:59:15.883434+0000', 'last_fullsized': '2024-09-10T15:59:15.772756+0000', 'mapping_epoch': 42, 'log_start': "45'11000", 'ondisk_log_start': "45'11000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3789", 'last_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'objects_scrubbed': 3741, 'log_size': 10048, 'log_dups_size': 0, 'ondisk_log_size': 10048, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1440, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4321, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17303, 'num_write_kb': 27124, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 52, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4321}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:10:54.543 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-10T16:10:54.543 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "49'21198", 'reported_seq': 21372, 'reported_epoch': 58, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:10:51.161437+0000', 'last_change': '2024-09-10T15:59:16.313409+0000', 'last_active': '2024-09-10T16:10:51.161437+0000', 'last_peered': '2024-09-10T16:10:51.161437+0000', 'last_clean': '2024-09-10T15:59:14.129780+0000', 'last_became_active': '2024-09-10T15:59:16.300932+0000', 'last_became_peered': '2024-09-10T15:59:16.300932+0000', 'last_unstale': '2024-09-10T16:10:51.161437+0000', 'last_undegraded': '2024-09-10T15:59:15.887823+0000', 'last_fullsized': '2024-09-10T15:59:15.768513+0000', 'mapping_epoch': 42, 'log_start': "45'11100", 'ondisk_log_start': "45'11100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3880", 'last_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'objects_scrubbed': 3827, 'log_size': 10098, 'log_dups_size': 0, 'ondisk_log_size': 10098, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1397, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4325, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17361, 'num_write_kb': 27072, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4321}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:10:54.544 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T16:10:55.012 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:10:55.013 INFO:teuthology.orchestra.run.smithi115.stdout:{"fsid":"7effee29-df05-4351-9741-ee203564ed81","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 18200/24 objects degraded (75833.333%), 5 pgs degraded, 5 pgs undersized","count":10},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":819,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":59,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983954,"num_in_osds":3,"osd_in_since":1725983947,"num_remapped_pgs":4},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":19},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":3},{"state_name":"active+recovering+undersized+degraded+remapped","count":2}],"num_pgs":24,"num_pools":2,"num_objects":12,"data_bytes":4849,"bytes_used":1565982720,"bytes_avail":288344309760,"bytes_total":289910292480,"degraded_objects":18200,"degraded_total":24,"degraded_ratio":758.33333333333348,"recovering_objects_per_sec":113,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":906,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-10T15:57:14:967120+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T16:00:13.560776+0000","services":{}},"progress_events":{}} 2024-09-10T16:10:55.028 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-10T16:10:55.029 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:10:55.393 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:10:55.393 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:10:55.406 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":461,"stamp":"2024-09-10T16:10:53.788173+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":18200,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292098,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33192,"num_bytes_recovered":33097148,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161025,"ondisk_log_size":161025,"up":48,"acting":43,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1529280,"kb_used_data":206976,"kb_used_omap":12,"kb_used_meta":1322227,"kb_avail":281586240,"statfs":{"total":289910292480,"available":288344309760,"internally_reserved":0,"allocated":211943424,"data_stored":210873193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1353960450},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-906,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":906,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001404"},"pg_stats":[{"pgid":"2.f","version":"49'20672","reported_seq":29302,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:10:51.161692+0000","last_change":"2024-09-10T16:09:57.088531+0000","last_active":"2024-09-10T16:10:51.161692+0000","last_peered":"2024-09-10T16:10:51.161692+0000","last_clean":"2024-09-10T16:10:51.161692+0000","last_became_active":"2024-09-10T16:09:57.005755+0000","last_became_peered":"2024-09-10T16:09:57.005755+0000","last_unstale":"2024-09-10T16:10:51.161692+0000","last_undegraded":"2024-09-10T16:10:51.161692+0000","last_fullsized":"2024-09-10T16:10:51.161692+0000","mapping_epoch":54,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:09:19.329697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20672,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4259,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20844","reported_seq":20986,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:10:51.159480+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:10:51.159480+0000","last_peered":"2024-09-10T16:10:51.159480+0000","last_clean":"2024-09-10T16:10:51.159480+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:10:51.159480+0000","last_undegraded":"2024-09-10T16:10:51.159480+0000","last_fullsized":"2024-09-10T16:10:51.159480+0000","mapping_epoch":35,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18949,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20798","reported_seq":29503,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.851218+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:10:20.851218+0000","last_peered":"2024-09-10T16:10:20.851218+0000","last_clean":"2024-09-10T16:10:20.851218+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:10:20.851218+0000","last_undegraded":"2024-09-10T16:10:20.851218+0000","last_fullsized":"2024-09-10T16:10:20.851218+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18802,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20660","reported_seq":20770,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:10:51.159450+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:10:51.159450+0000","last_peered":"2024-09-10T16:10:51.159450+0000","last_clean":"2024-09-10T16:10:51.159450+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:10:51.159450+0000","last_undegraded":"2024-09-10T16:10:51.159450+0000","last_fullsized":"2024-09-10T16:10:51.159450+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20660,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20891","reported_seq":21050,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:10:51.161243+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:10:51.161243+0000","last_peered":"2024-09-10T16:10:51.161243+0000","last_clean":"2024-09-10T16:10:51.161243+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:10:51.161243+0000","last_undegraded":"2024-09-10T16:10:51.161243+0000","last_fullsized":"2024-09-10T16:10:51.161243+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17602,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'21061","reported_seq":27947,"reported_epoch":57,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:10:46.214698+0000","last_change":"2024-09-10T16:10:19.743410+0000","last_active":"2024-09-10T16:10:46.214698+0000","last_peered":"2024-09-10T16:10:46.214698+0000","last_clean":"2024-09-10T15:59:14.111551+0000","last_became_active":"2024-09-10T15:59:16.300040+0000","last_became_peered":"2024-09-10T15:59:16.300040+0000","last_unstale":"2024-09-10T16:10:46.214698+0000","last_undegraded":"2024-09-10T15:59:15.878234+0000","last_fullsized":"2024-09-10T15:59:15.769328+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1012,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17673,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3360,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1011}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'21082","reported_seq":21249,"reported_epoch":58,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:10:51.160046+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:10:51.160046+0000","last_peered":"2024-09-10T16:10:51.160046+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:10:51.160046+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17581,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4235},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'21314","reported_seq":21473,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:10:51.159983+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:10:51.159983+0000","last_peered":"2024-09-10T16:10:51.159983+0000","last_clean":"2024-09-10T16:10:51.159983+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:10:51.159983+0000","last_undegraded":"2024-09-10T16:10:51.159983+0000","last_fullsized":"2024-09-10T16:10:51.159983+0000","mapping_epoch":34,"log_start":"45'11300","ondisk_log_start":"45'11300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18161,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.850980+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:10:20.850980+0000","last_peered":"2024-09-10T16:10:20.850980+0000","last_clean":"2024-09-10T16:10:20.850980+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:10:20.850980+0000","last_undegraded":"2024-09-10T16:10:20.850980+0000","last_fullsized":"2024-09-10T16:10:20.850980+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20457","reported_seq":20628,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.851257+0000","last_change":"2024-09-10T16:09:33.012840+0000","last_active":"2024-09-10T16:10:20.851257+0000","last_peered":"2024-09-10T16:10:20.851257+0000","last_clean":"2024-09-10T16:10:20.851257+0000","last_became_active":"2024-09-10T16:09:32.934056+0000","last_became_peered":"2024-09-10T16:09:32.934056+0000","last_unstale":"2024-09-10T16:10:20.851257+0000","last_undegraded":"2024-09-10T16:10:20.851257+0000","last_fullsized":"2024-09-10T16:10:20.851257+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:08:30.481235+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18276,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":4579,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":124,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:10:51.159327+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:10:51.159327+0000","last_peered":"2024-09-10T16:10:51.159327+0000","last_clean":"2024-09-10T16:10:51.159327+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:10:51.159327+0000","last_undegraded":"2024-09-10T16:10:51.159327+0000","last_fullsized":"2024-09-10T16:10:51.159327+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20910","reported_seq":29592,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:10:51.160977+0000","last_change":"2024-09-10T16:09:08.909580+0000","last_active":"2024-09-10T16:10:51.160977+0000","last_peered":"2024-09-10T16:10:51.160977+0000","last_clean":"2024-09-10T16:10:51.160977+0000","last_became_active":"2024-09-10T16:09:08.854941+0000","last_became_peered":"2024-09-10T16:09:08.854941+0000","last_unstale":"2024-09-10T16:10:51.160977+0000","last_undegraded":"2024-09-10T16:10:51.160977+0000","last_fullsized":"2024-09-10T16:10:51.160977+0000","mapping_epoch":50,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:13:14.846691+0000","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17414,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4257,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":134,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:10:51.160698+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:10:51.160698+0000","last_peered":"2024-09-10T16:10:51.160698+0000","last_clean":"2024-09-10T16:10:51.160698+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:10:51.160698+0000","last_undegraded":"2024-09-10T16:10:51.160698+0000","last_fullsized":"2024-09-10T16:10:51.160698+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20781","reported_seq":29564,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:10:51.159696+0000","last_change":"2024-09-10T16:10:20.907889+0000","last_active":"2024-09-10T16:10:51.159696+0000","last_peered":"2024-09-10T16:10:51.159696+0000","last_clean":"2024-09-10T16:10:51.159696+0000","last_became_active":"2024-09-10T16:10:20.852065+0000","last_became_peered":"2024-09-10T16:10:20.852065+0000","last_unstale":"2024-09-10T16:10:51.159696+0000","last_undegraded":"2024-09-10T16:10:51.159696+0000","last_fullsized":"2024-09-10T16:10:51.159696+0000","mapping_epoch":56,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:34.069549+0000","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18461,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4307,"num_bytes_recovered":8675,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":107,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.851171+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:10:20.851171+0000","last_peered":"2024-09-10T16:10:20.851171+0000","last_clean":"2024-09-10T16:10:20.851171+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:10:20.851171+0000","last_undegraded":"2024-09-10T16:10:20.851171+0000","last_fullsized":"2024-09-10T16:10:20.851171+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'21170","reported_seq":30049,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:10:51.159183+0000","last_change":"2024-09-10T16:10:50.728349+0000","last_active":"2024-09-10T16:10:51.159183+0000","last_peered":"2024-09-10T16:10:51.159183+0000","last_clean":"2024-09-10T16:10:51.159183+0000","last_became_active":"2024-09-10T16:08:10.860438+0000","last_became_peered":"2024-09-10T16:08:10.860438+0000","last_unstale":"2024-09-10T16:10:51.159183+0000","last_undegraded":"2024-09-10T16:10:51.159183+0000","last_fullsized":"2024-09-10T16:10:51.159183+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:38:13.781380+0000","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18724,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20838","reported_seq":21002,"reported_epoch":58,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:10:51.161052+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:10:51.161052+0000","last_peered":"2024-09-10T16:10:51.161052+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:10:51.161052+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17469,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.851195+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:10:20.851195+0000","last_peered":"2024-09-10T16:10:20.851195+0000","last_clean":"2024-09-10T16:10:20.851195+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:10:20.851195+0000","last_undegraded":"2024-09-10T16:10:20.851195+0000","last_fullsized":"2024-09-10T16:10:20.851195+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20598","reported_seq":29191,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.851322+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:10:20.851322+0000","last_peered":"2024-09-10T16:10:20.851322+0000","last_clean":"2024-09-10T16:10:20.851322+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:10:20.851322+0000","last_undegraded":"2024-09-10T16:10:20.851322+0000","last_fullsized":"2024-09-10T16:10:20.851322+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16988,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":124,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:10:51.159516+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:10:51.159516+0000","last_peered":"2024-09-10T16:10:51.159516+0000","last_clean":"2024-09-10T16:10:51.159516+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:10:51.159516+0000","last_undegraded":"2024-09-10T16:10:51.159516+0000","last_fullsized":"2024-09-10T16:10:51.159516+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'21048","reported_seq":21325,"reported_epoch":58,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:10:51.224441+0000","last_change":"2024-09-10T16:10:50.728566+0000","last_active":"2024-09-10T16:10:51.224441+0000","last_peered":"2024-09-10T16:10:51.224441+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:10:51.224441+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4321,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17303,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":52,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4321}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.851276+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:10:20.851276+0000","last_peered":"2024-09-10T16:10:20.851276+0000","last_clean":"2024-09-10T16:10:20.851276+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:10:20.851276+0000","last_undegraded":"2024-09-10T16:10:20.851276+0000","last_fullsized":"2024-09-10T16:10:20.851276+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":102,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:10:20.851340+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:10:20.851340+0000","last_peered":"2024-09-10T16:10:20.851340+0000","last_clean":"2024-09-10T16:10:20.851340+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:10:20.851340+0000","last_undegraded":"2024-09-10T16:10:20.851340+0000","last_fullsized":"2024-09-10T16:10:20.851340+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'21198","reported_seq":21372,"reported_epoch":58,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:10:51.161437+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:10:51.161437+0000","last_peered":"2024-09-10T16:10:51.161437+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:10:51.161437+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17361,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4321},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":18200,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292096,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33190,"num_bytes_recovered":33097129,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":210153472,"data_stored":210037856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161022,"ondisk_log_size":161022,"up":32,"acting":27,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986330,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":502664,"kb_used_data":137776,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93869176,"statfs":{"total":96636764160,"available":96122036224,"internally_reserved":0,"allocated":141082624,"data_stored":140731007,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659275,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":479252,"kb_used_data":516,"kb_used_omap":4,"kb_used_meta":478715,"kb_avail":93892588,"statfs":{"total":96636764160,"available":96146010112,"internally_reserved":0,"allocated":528384,"data_stored":165464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":490204846},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182558,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":547364,"kb_used_data":68684,"kb_used_omap":4,"kb_used_meta":478651,"kb_avail":93824476,"statfs":{"total":96636764160,"available":96076263424,"internally_reserved":0,"allocated":70332416,"data_stored":69976722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":490139306},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":69853184,"data_stored":69817054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":5796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":140251136,"data_stored":140215006,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:10:58.409 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:10:58.746 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:10:58.746 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:10:58.762 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":463,"stamp":"2024-09-10T16:10:57.788766+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":16326,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292098,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35066,"num_bytes_recovered":33097148,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161025,"ondisk_log_size":161025,"up":48,"acting":44,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1535364,"kb_used_data":204752,"kb_used_omap":12,"kb_used_meta":1330547,"kb_avail":281580156,"statfs":{"total":289910292480,"available":288338079744,"internally_reserved":0,"allocated":209666048,"data_stored":208570790,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1362480130},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1],"upper_bound":16},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1926,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1926,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001307"},"pg_stats":[{"pgid":"2.f","version":"49'20672","reported_seq":29304,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.169962+0000","last_change":"2024-09-10T16:09:57.088531+0000","last_active":"2024-09-10T16:10:52.169962+0000","last_peered":"2024-09-10T16:10:52.169962+0000","last_clean":"2024-09-10T16:10:52.169962+0000","last_became_active":"2024-09-10T16:09:57.005755+0000","last_became_peered":"2024-09-10T16:09:57.005755+0000","last_unstale":"2024-09-10T16:10:52.169962+0000","last_undegraded":"2024-09-10T16:10:52.169962+0000","last_fullsized":"2024-09-10T16:10:52.169962+0000","mapping_epoch":54,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:09:19.329697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20672,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4259,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20844","reported_seq":20988,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.168075+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:10:52.168075+0000","last_peered":"2024-09-10T16:10:52.168075+0000","last_clean":"2024-09-10T16:10:52.168075+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:10:52.168075+0000","last_undegraded":"2024-09-10T16:10:52.168075+0000","last_fullsized":"2024-09-10T16:10:52.168075+0000","mapping_epoch":35,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18949,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20798","reported_seq":29507,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.168961+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:10:52.168961+0000","last_peered":"2024-09-10T16:10:52.168961+0000","last_clean":"2024-09-10T16:10:52.168961+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:10:52.168961+0000","last_undegraded":"2024-09-10T16:10:52.168961+0000","last_fullsized":"2024-09-10T16:10:52.168961+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18802,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20660","reported_seq":20772,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.167700+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:10:52.167700+0000","last_peered":"2024-09-10T16:10:52.167700+0000","last_clean":"2024-09-10T16:10:52.167700+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:10:52.167700+0000","last_undegraded":"2024-09-10T16:10:52.167700+0000","last_fullsized":"2024-09-10T16:10:52.167700+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20660,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20891","reported_seq":21052,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.169809+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:10:52.169809+0000","last_peered":"2024-09-10T16:10:52.169809+0000","last_clean":"2024-09-10T16:10:52.169809+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:10:52.169809+0000","last_undegraded":"2024-09-10T16:10:52.169809+0000","last_fullsized":"2024-09-10T16:10:52.169809+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17602,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'21061","reported_seq":21222,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.242289+0000","last_change":"2024-09-10T16:10:52.241702+0000","last_active":"2024-09-10T16:10:52.242289+0000","last_peered":"2024-09-10T16:10:52.242289+0000","last_clean":"2024-09-10T16:10:52.242289+0000","last_became_active":"2024-09-10T16:10:52.171236+0000","last_became_peered":"2024-09-10T16:10:52.171236+0000","last_unstale":"2024-09-10T16:10:52.242289+0000","last_undegraded":"2024-09-10T16:10:52.242289+0000","last_fullsized":"2024-09-10T16:10:52.242289+0000","mapping_epoch":58,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:04:46.778280+0000","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17673,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4372,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'21082","reported_seq":21251,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:10:52.170050+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:10:52.170050+0000","last_peered":"2024-09-10T16:10:52.170050+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:10:52.170050+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17581,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4235},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'21314","reported_seq":21475,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.169708+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:10:52.169708+0000","last_peered":"2024-09-10T16:10:52.169708+0000","last_clean":"2024-09-10T16:10:52.169708+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:10:52.169708+0000","last_undegraded":"2024-09-10T16:10:52.169708+0000","last_fullsized":"2024-09-10T16:10:52.169708+0000","mapping_epoch":34,"log_start":"45'11300","ondisk_log_start":"45'11300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18161,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.168728+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:10:52.168728+0000","last_peered":"2024-09-10T16:10:52.168728+0000","last_clean":"2024-09-10T16:10:52.168728+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:10:52.168728+0000","last_undegraded":"2024-09-10T16:10:52.168728+0000","last_fullsized":"2024-09-10T16:10:52.168728+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20457","reported_seq":20632,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.168901+0000","last_change":"2024-09-10T16:09:33.012840+0000","last_active":"2024-09-10T16:10:52.168901+0000","last_peered":"2024-09-10T16:10:52.168901+0000","last_clean":"2024-09-10T16:10:52.168901+0000","last_became_active":"2024-09-10T16:09:32.934056+0000","last_became_peered":"2024-09-10T16:09:32.934056+0000","last_unstale":"2024-09-10T16:10:52.168901+0000","last_undegraded":"2024-09-10T16:10:52.168901+0000","last_fullsized":"2024-09-10T16:10:52.168901+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:08:30.481235+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18276,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":4579,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":126,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.168004+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:10:52.168004+0000","last_peered":"2024-09-10T16:10:52.168004+0000","last_clean":"2024-09-10T16:10:52.168004+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:10:52.168004+0000","last_undegraded":"2024-09-10T16:10:52.168004+0000","last_fullsized":"2024-09-10T16:10:52.168004+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20910","reported_seq":29594,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.169792+0000","last_change":"2024-09-10T16:09:08.909580+0000","last_active":"2024-09-10T16:10:52.169792+0000","last_peered":"2024-09-10T16:10:52.169792+0000","last_clean":"2024-09-10T16:10:52.169792+0000","last_became_active":"2024-09-10T16:09:08.854941+0000","last_became_peered":"2024-09-10T16:09:08.854941+0000","last_unstale":"2024-09-10T16:10:52.169792+0000","last_undegraded":"2024-09-10T16:10:52.169792+0000","last_fullsized":"2024-09-10T16:10:52.169792+0000","mapping_epoch":50,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:13:14.846691+0000","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17414,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4257,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":136,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.169714+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:10:52.169714+0000","last_peered":"2024-09-10T16:10:52.169714+0000","last_clean":"2024-09-10T16:10:52.169714+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:10:52.169714+0000","last_undegraded":"2024-09-10T16:10:52.169714+0000","last_fullsized":"2024-09-10T16:10:52.169714+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20781","reported_seq":29566,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.168064+0000","last_change":"2024-09-10T16:10:20.907889+0000","last_active":"2024-09-10T16:10:52.168064+0000","last_peered":"2024-09-10T16:10:52.168064+0000","last_clean":"2024-09-10T16:10:52.168064+0000","last_became_active":"2024-09-10T16:10:20.852065+0000","last_became_peered":"2024-09-10T16:10:20.852065+0000","last_unstale":"2024-09-10T16:10:52.168064+0000","last_undegraded":"2024-09-10T16:10:52.168064+0000","last_fullsized":"2024-09-10T16:10:52.168064+0000","mapping_epoch":56,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:34.069549+0000","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18461,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4307,"num_bytes_recovered":8675,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":111,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.168792+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:10:52.168792+0000","last_peered":"2024-09-10T16:10:52.168792+0000","last_clean":"2024-09-10T16:10:52.168792+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:10:52.168792+0000","last_undegraded":"2024-09-10T16:10:52.168792+0000","last_fullsized":"2024-09-10T16:10:52.168792+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'21170","reported_seq":30051,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.167373+0000","last_change":"2024-09-10T16:10:50.728349+0000","last_active":"2024-09-10T16:10:52.167373+0000","last_peered":"2024-09-10T16:10:52.167373+0000","last_clean":"2024-09-10T16:10:52.167373+0000","last_became_active":"2024-09-10T16:08:10.860438+0000","last_became_peered":"2024-09-10T16:08:10.860438+0000","last_unstale":"2024-09-10T16:10:52.167373+0000","last_undegraded":"2024-09-10T16:10:52.167373+0000","last_fullsized":"2024-09-10T16:10:52.167373+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:38:13.781380+0000","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18724,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20838","reported_seq":21004,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:10:52.169840+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:10:52.169840+0000","last_peered":"2024-09-10T16:10:52.169840+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:10:52.169840+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17469,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.168992+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:10:52.168992+0000","last_peered":"2024-09-10T16:10:52.168992+0000","last_clean":"2024-09-10T16:10:52.168992+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:10:52.168992+0000","last_undegraded":"2024-09-10T16:10:52.168992+0000","last_fullsized":"2024-09-10T16:10:52.168992+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20598","reported_seq":29195,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.169145+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:10:52.169145+0000","last_peered":"2024-09-10T16:10:52.169145+0000","last_clean":"2024-09-10T16:10:52.169145+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:10:52.169145+0000","last_undegraded":"2024-09-10T16:10:52.169145+0000","last_fullsized":"2024-09-10T16:10:52.169145+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16988,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":126,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.167911+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:10:52.167911+0000","last_peered":"2024-09-10T16:10:52.167911+0000","last_clean":"2024-09-10T16:10:52.167911+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:10:52.167911+0000","last_undegraded":"2024-09-10T16:10:52.167911+0000","last_fullsized":"2024-09-10T16:10:52.167911+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'21048","reported_seq":23051,"reported_epoch":59,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:10:56.223190+0000","last_change":"2024-09-10T16:10:50.728566+0000","last_active":"2024-09-10T16:10:56.223190+0000","last_peered":"2024-09-10T16:10:56.223190+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:10:56.223190+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3459,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17303,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":914,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3459}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.169062+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:10:52.169062+0000","last_peered":"2024-09-10T16:10:52.169062+0000","last_clean":"2024-09-10T16:10:52.169062+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:10:52.169062+0000","last_undegraded":"2024-09-10T16:10:52.169062+0000","last_fullsized":"2024-09-10T16:10:52.169062+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":106,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.169114+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:10:52.169114+0000","last_peered":"2024-09-10T16:10:52.169114+0000","last_clean":"2024-09-10T16:10:52.169114+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:10:52.169114+0000","last_undegraded":"2024-09-10T16:10:52.169114+0000","last_fullsized":"2024-09-10T16:10:52.169114+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'21198","reported_seq":21374,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:10:52.169851+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:10:52.169851+0000","last_peered":"2024-09-10T16:10:52.169851+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:10:52.169851+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17361,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4321},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":16326,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292096,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35064,"num_bytes_recovered":33097129,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":207814656,"data_stored":207706266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161022,"ondisk_log_size":161022,"up":32,"acting":28,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986331,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":500408,"kb_used_data":135528,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93871432,"statfs":{"total":96636764160,"available":96124346368,"internally_reserved":0,"allocated":138780672,"data_stored":138418304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659276,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487580,"kb_used_data":524,"kb_used_omap":4,"kb_used_meta":487035,"kb_avail":93884260,"statfs":{"total":96636764160,"available":96137482240,"internally_reserved":0,"allocated":536576,"data_stored":170131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":498724526},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1],"upper_bound":16},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182559,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":547376,"kb_used_data":68700,"kb_used_omap":4,"kb_used_meta":478651,"kb_avail":93824464,"statfs":{"total":96636764160,"available":96076251136,"internally_reserved":0,"allocated":70348800,"data_stored":69982355,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":490139306},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":69853184,"data_stored":69817054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":4830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":137920512,"data_stored":137884382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:10:58.763 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:10:59.091 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:10:59.092 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:10:59.105 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":463,"stamp":"2024-09-10T16:10:57.788766+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":16326,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292098,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35066,"num_bytes_recovered":33097148,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161025,"ondisk_log_size":161025,"up":48,"acting":44,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1535364,"kb_used_data":204752,"kb_used_omap":12,"kb_used_meta":1330547,"kb_avail":281580156,"statfs":{"total":289910292480,"available":288338079744,"internally_reserved":0,"allocated":209666048,"data_stored":208570790,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1362480130},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1],"upper_bound":16},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1926,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1926,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001307"},"pg_stats":[{"pgid":"2.f","version":"49'20672","reported_seq":29304,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.169962+0000","last_change":"2024-09-10T16:09:57.088531+0000","last_active":"2024-09-10T16:10:52.169962+0000","last_peered":"2024-09-10T16:10:52.169962+0000","last_clean":"2024-09-10T16:10:52.169962+0000","last_became_active":"2024-09-10T16:09:57.005755+0000","last_became_peered":"2024-09-10T16:09:57.005755+0000","last_unstale":"2024-09-10T16:10:52.169962+0000","last_undegraded":"2024-09-10T16:10:52.169962+0000","last_fullsized":"2024-09-10T16:10:52.169962+0000","mapping_epoch":54,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:09:19.329697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20672,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4259,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20844","reported_seq":20988,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.168075+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:10:52.168075+0000","last_peered":"2024-09-10T16:10:52.168075+0000","last_clean":"2024-09-10T16:10:52.168075+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:10:52.168075+0000","last_undegraded":"2024-09-10T16:10:52.168075+0000","last_fullsized":"2024-09-10T16:10:52.168075+0000","mapping_epoch":35,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18949,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20798","reported_seq":29507,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.168961+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:10:52.168961+0000","last_peered":"2024-09-10T16:10:52.168961+0000","last_clean":"2024-09-10T16:10:52.168961+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:10:52.168961+0000","last_undegraded":"2024-09-10T16:10:52.168961+0000","last_fullsized":"2024-09-10T16:10:52.168961+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18802,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20660","reported_seq":20772,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.167700+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:10:52.167700+0000","last_peered":"2024-09-10T16:10:52.167700+0000","last_clean":"2024-09-10T16:10:52.167700+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:10:52.167700+0000","last_undegraded":"2024-09-10T16:10:52.167700+0000","last_fullsized":"2024-09-10T16:10:52.167700+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20660,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20891","reported_seq":21052,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.169809+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:10:52.169809+0000","last_peered":"2024-09-10T16:10:52.169809+0000","last_clean":"2024-09-10T16:10:52.169809+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:10:52.169809+0000","last_undegraded":"2024-09-10T16:10:52.169809+0000","last_fullsized":"2024-09-10T16:10:52.169809+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17602,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'21061","reported_seq":21222,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.242289+0000","last_change":"2024-09-10T16:10:52.241702+0000","last_active":"2024-09-10T16:10:52.242289+0000","last_peered":"2024-09-10T16:10:52.242289+0000","last_clean":"2024-09-10T16:10:52.242289+0000","last_became_active":"2024-09-10T16:10:52.171236+0000","last_became_peered":"2024-09-10T16:10:52.171236+0000","last_unstale":"2024-09-10T16:10:52.242289+0000","last_undegraded":"2024-09-10T16:10:52.242289+0000","last_fullsized":"2024-09-10T16:10:52.242289+0000","mapping_epoch":58,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:04:46.778280+0000","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17673,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4372,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'21082","reported_seq":21251,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:10:52.170050+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:10:52.170050+0000","last_peered":"2024-09-10T16:10:52.170050+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:10:52.170050+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17581,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4235},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'21314","reported_seq":21475,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.169708+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:10:52.169708+0000","last_peered":"2024-09-10T16:10:52.169708+0000","last_clean":"2024-09-10T16:10:52.169708+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:10:52.169708+0000","last_undegraded":"2024-09-10T16:10:52.169708+0000","last_fullsized":"2024-09-10T16:10:52.169708+0000","mapping_epoch":34,"log_start":"45'11300","ondisk_log_start":"45'11300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18161,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.168728+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:10:52.168728+0000","last_peered":"2024-09-10T16:10:52.168728+0000","last_clean":"2024-09-10T16:10:52.168728+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:10:52.168728+0000","last_undegraded":"2024-09-10T16:10:52.168728+0000","last_fullsized":"2024-09-10T16:10:52.168728+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20457","reported_seq":20632,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.168901+0000","last_change":"2024-09-10T16:09:33.012840+0000","last_active":"2024-09-10T16:10:52.168901+0000","last_peered":"2024-09-10T16:10:52.168901+0000","last_clean":"2024-09-10T16:10:52.168901+0000","last_became_active":"2024-09-10T16:09:32.934056+0000","last_became_peered":"2024-09-10T16:09:32.934056+0000","last_unstale":"2024-09-10T16:10:52.168901+0000","last_undegraded":"2024-09-10T16:10:52.168901+0000","last_fullsized":"2024-09-10T16:10:52.168901+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:08:30.481235+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18276,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":4579,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":126,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.168004+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:10:52.168004+0000","last_peered":"2024-09-10T16:10:52.168004+0000","last_clean":"2024-09-10T16:10:52.168004+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:10:52.168004+0000","last_undegraded":"2024-09-10T16:10:52.168004+0000","last_fullsized":"2024-09-10T16:10:52.168004+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20910","reported_seq":29594,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.169792+0000","last_change":"2024-09-10T16:09:08.909580+0000","last_active":"2024-09-10T16:10:52.169792+0000","last_peered":"2024-09-10T16:10:52.169792+0000","last_clean":"2024-09-10T16:10:52.169792+0000","last_became_active":"2024-09-10T16:09:08.854941+0000","last_became_peered":"2024-09-10T16:09:08.854941+0000","last_unstale":"2024-09-10T16:10:52.169792+0000","last_undegraded":"2024-09-10T16:10:52.169792+0000","last_fullsized":"2024-09-10T16:10:52.169792+0000","mapping_epoch":50,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:13:14.846691+0000","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17414,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4257,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":136,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.169714+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:10:52.169714+0000","last_peered":"2024-09-10T16:10:52.169714+0000","last_clean":"2024-09-10T16:10:52.169714+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:10:52.169714+0000","last_undegraded":"2024-09-10T16:10:52.169714+0000","last_fullsized":"2024-09-10T16:10:52.169714+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20781","reported_seq":29566,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.168064+0000","last_change":"2024-09-10T16:10:20.907889+0000","last_active":"2024-09-10T16:10:52.168064+0000","last_peered":"2024-09-10T16:10:52.168064+0000","last_clean":"2024-09-10T16:10:52.168064+0000","last_became_active":"2024-09-10T16:10:20.852065+0000","last_became_peered":"2024-09-10T16:10:20.852065+0000","last_unstale":"2024-09-10T16:10:52.168064+0000","last_undegraded":"2024-09-10T16:10:52.168064+0000","last_fullsized":"2024-09-10T16:10:52.168064+0000","mapping_epoch":56,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:34.069549+0000","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18461,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4307,"num_bytes_recovered":8675,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":111,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.168792+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:10:52.168792+0000","last_peered":"2024-09-10T16:10:52.168792+0000","last_clean":"2024-09-10T16:10:52.168792+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:10:52.168792+0000","last_undegraded":"2024-09-10T16:10:52.168792+0000","last_fullsized":"2024-09-10T16:10:52.168792+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'21170","reported_seq":30051,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.167373+0000","last_change":"2024-09-10T16:10:50.728349+0000","last_active":"2024-09-10T16:10:52.167373+0000","last_peered":"2024-09-10T16:10:52.167373+0000","last_clean":"2024-09-10T16:10:52.167373+0000","last_became_active":"2024-09-10T16:08:10.860438+0000","last_became_peered":"2024-09-10T16:08:10.860438+0000","last_unstale":"2024-09-10T16:10:52.167373+0000","last_undegraded":"2024-09-10T16:10:52.167373+0000","last_fullsized":"2024-09-10T16:10:52.167373+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:38:13.781380+0000","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18724,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20838","reported_seq":21004,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:10:52.169840+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:10:52.169840+0000","last_peered":"2024-09-10T16:10:52.169840+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:10:52.169840+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17469,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.168992+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:10:52.168992+0000","last_peered":"2024-09-10T16:10:52.168992+0000","last_clean":"2024-09-10T16:10:52.168992+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:10:52.168992+0000","last_undegraded":"2024-09-10T16:10:52.168992+0000","last_fullsized":"2024-09-10T16:10:52.168992+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20598","reported_seq":29195,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.169145+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:10:52.169145+0000","last_peered":"2024-09-10T16:10:52.169145+0000","last_clean":"2024-09-10T16:10:52.169145+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:10:52.169145+0000","last_undegraded":"2024-09-10T16:10:52.169145+0000","last_fullsized":"2024-09-10T16:10:52.169145+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16988,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":126,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.167911+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:10:52.167911+0000","last_peered":"2024-09-10T16:10:52.167911+0000","last_clean":"2024-09-10T16:10:52.167911+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:10:52.167911+0000","last_undegraded":"2024-09-10T16:10:52.167911+0000","last_fullsized":"2024-09-10T16:10:52.167911+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'21048","reported_seq":23051,"reported_epoch":59,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:10:56.223190+0000","last_change":"2024-09-10T16:10:50.728566+0000","last_active":"2024-09-10T16:10:56.223190+0000","last_peered":"2024-09-10T16:10:56.223190+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:10:56.223190+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3459,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17303,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":914,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3459}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.169062+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:10:52.169062+0000","last_peered":"2024-09-10T16:10:52.169062+0000","last_clean":"2024-09-10T16:10:52.169062+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:10:52.169062+0000","last_undegraded":"2024-09-10T16:10:52.169062+0000","last_fullsized":"2024-09-10T16:10:52.169062+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":106,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.169114+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:10:52.169114+0000","last_peered":"2024-09-10T16:10:52.169114+0000","last_clean":"2024-09-10T16:10:52.169114+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:10:52.169114+0000","last_undegraded":"2024-09-10T16:10:52.169114+0000","last_fullsized":"2024-09-10T16:10:52.169114+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'21198","reported_seq":21374,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:10:52.169851+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:10:52.169851+0000","last_peered":"2024-09-10T16:10:52.169851+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:10:52.169851+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17361,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4321},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":16326,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292096,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35064,"num_bytes_recovered":33097129,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":207814656,"data_stored":207706266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161022,"ondisk_log_size":161022,"up":32,"acting":28,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986331,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":500408,"kb_used_data":135528,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93871432,"statfs":{"total":96636764160,"available":96124346368,"internally_reserved":0,"allocated":138780672,"data_stored":138418304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659276,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487580,"kb_used_data":524,"kb_used_omap":4,"kb_used_meta":487035,"kb_avail":93884260,"statfs":{"total":96636764160,"available":96137482240,"internally_reserved":0,"allocated":536576,"data_stored":170131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":498724526},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1],"upper_bound":16},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182559,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":547376,"kb_used_data":68700,"kb_used_omap":4,"kb_used_meta":478651,"kb_avail":93824464,"statfs":{"total":96636764160,"available":96076251136,"internally_reserved":0,"allocated":70348800,"data_stored":69982355,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":490139306},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":69853184,"data_stored":69817054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":4830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":137920512,"data_stored":137884382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:10:59.107 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-10T16:10:59.107 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "49'21082", 'reported_seq': 21251, 'reported_epoch': 59, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:10:52.170050+0000', 'last_change': '2024-09-10T15:59:16.331899+0000', 'last_active': '2024-09-10T16:10:52.170050+0000', 'last_peered': '2024-09-10T16:10:52.170050+0000', 'last_clean': '2024-09-10T15:59:14.139436+0000', 'last_became_active': '2024-09-10T15:59:16.318715+0000', 'last_became_peered': '2024-09-10T15:59:16.318715+0000', 'last_unstale': '2024-09-10T16:10:52.170050+0000', 'last_undegraded': '2024-09-10T15:59:15.877524+0000', 'last_fullsized': '2024-09-10T15:59:15.769042+0000', 'mapping_epoch': 42, 'log_start': "45'11000", 'ondisk_log_start': "45'11000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3537", 'last_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'objects_scrubbed': 3497, 'log_size': 10082, 'log_dups_size': 0, 'ondisk_log_size': 10082, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1224, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4237, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 17581, 'num_write_kb': 28172, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4235}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:10:59.107 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-10T16:10:59.107 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "49'20838", 'reported_seq': 21004, 'reported_epoch': 59, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:10:52.169840+0000', 'last_change': '2024-09-10T15:59:16.315641+0000', 'last_active': '2024-09-10T16:10:52.169840+0000', 'last_peered': '2024-09-10T16:10:52.169840+0000', 'last_clean': '2024-09-10T15:59:14.117001+0000', 'last_became_active': '2024-09-10T15:59:16.302245+0000', 'last_became_peered': '2024-09-10T15:59:16.302245+0000', 'last_unstale': '2024-09-10T16:10:52.169840+0000', 'last_undegraded': '2024-09-10T15:59:15.845012+0000', 'last_fullsized': '2024-09-10T15:59:15.770337+0000', 'mapping_epoch': 42, 'log_start': "45'10800", 'ondisk_log_start': "45'10800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3404", 'last_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'objects_scrubbed': 3367, 'log_size': 10038, 'log_dups_size': 0, 'ondisk_log_size': 10038, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 983, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17469, 'num_write_kb': 28204, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4304}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:10:59.107 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-10T16:10:59.107 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "49'21048", 'reported_seq': 23051, 'reported_epoch': 59, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:10:56.223190+0000', 'last_change': '2024-09-10T16:10:50.728566+0000', 'last_active': '2024-09-10T16:10:56.223190+0000', 'last_peered': '2024-09-10T16:10:56.223190+0000', 'last_clean': '2024-09-10T15:59:14.116363+0000', 'last_became_active': '2024-09-10T15:59:16.301244+0000', 'last_became_peered': '2024-09-10T15:59:16.301244+0000', 'last_unstale': '2024-09-10T16:10:56.223190+0000', 'last_undegraded': '2024-09-10T15:59:15.883434+0000', 'last_fullsized': '2024-09-10T15:59:15.772756+0000', 'mapping_epoch': 42, 'log_start': "45'11000", 'ondisk_log_start': "45'11000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3789", 'last_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'objects_scrubbed': 3741, 'log_size': 10048, 'log_dups_size': 0, 'ondisk_log_size': 10048, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1440, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3459, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17303, 'num_write_kb': 27124, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 914, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3459}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:10:59.107 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-10T16:10:59.107 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "49'21198", 'reported_seq': 21374, 'reported_epoch': 59, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:10:52.169851+0000', 'last_change': '2024-09-10T15:59:16.313409+0000', 'last_active': '2024-09-10T16:10:52.169851+0000', 'last_peered': '2024-09-10T16:10:52.169851+0000', 'last_clean': '2024-09-10T15:59:14.129780+0000', 'last_became_active': '2024-09-10T15:59:16.300932+0000', 'last_became_peered': '2024-09-10T15:59:16.300932+0000', 'last_unstale': '2024-09-10T16:10:52.169851+0000', 'last_undegraded': '2024-09-10T15:59:15.887823+0000', 'last_fullsized': '2024-09-10T15:59:15.768513+0000', 'mapping_epoch': 42, 'log_start': "45'11100", 'ondisk_log_start': "45'11100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3880", 'last_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'objects_scrubbed': 3827, 'log_size': 10098, 'log_dups_size': 0, 'ondisk_log_size': 10098, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1397, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4325, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17361, 'num_write_kb': 27072, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4321}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:10:59.108 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T16:10:59.489 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:10:59.490 INFO:teuthology.orchestra.run.smithi115.stdout:{"fsid":"7effee29-df05-4351-9741-ee203564ed81","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 16326/24 objects degraded (68025.000%), 4 pgs degraded, 4 pgs undersized","count":8},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":823,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":59,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983954,"num_in_osds":3,"osd_in_since":1725983947,"num_remapped_pgs":4},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":20},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":3},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":12,"data_bytes":4849,"bytes_used":1572212736,"bytes_avail":288338079744,"bytes_total":289910292480,"degraded_objects":16326,"degraded_total":24,"degraded_ratio":680.25,"recovering_objects_per_sec":240,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1926,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-10T15:57:14:967120+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T16:00:13.560776+0000","services":{}},"progress_events":{}} 2024-09-10T16:10:59.504 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-10T16:10:59.504 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:10:59.818 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:10:59.818 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:10:59.831 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":464,"stamp":"2024-09-10T16:10:59.789170+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":16326,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292098,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35066,"num_bytes_recovered":33097148,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161025,"ondisk_log_size":161025,"up":48,"acting":44,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1535364,"kb_used_data":204752,"kb_used_omap":12,"kb_used_meta":1330547,"kb_avail":281580156,"statfs":{"total":289910292480,"available":288338079744,"internally_reserved":0,"allocated":209666048,"data_stored":208570790,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1362480130},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1],"upper_bound":16},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1926,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1926,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.631295"},"pg_stats":[{"pgid":"2.f","version":"49'20672","reported_seq":29304,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.169962+0000","last_change":"2024-09-10T16:09:57.088531+0000","last_active":"2024-09-10T16:10:52.169962+0000","last_peered":"2024-09-10T16:10:52.169962+0000","last_clean":"2024-09-10T16:10:52.169962+0000","last_became_active":"2024-09-10T16:09:57.005755+0000","last_became_peered":"2024-09-10T16:09:57.005755+0000","last_unstale":"2024-09-10T16:10:52.169962+0000","last_undegraded":"2024-09-10T16:10:52.169962+0000","last_fullsized":"2024-09-10T16:10:52.169962+0000","mapping_epoch":54,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:09:19.329697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20672,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4259,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20844","reported_seq":20988,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.168075+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:10:52.168075+0000","last_peered":"2024-09-10T16:10:52.168075+0000","last_clean":"2024-09-10T16:10:52.168075+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:10:52.168075+0000","last_undegraded":"2024-09-10T16:10:52.168075+0000","last_fullsized":"2024-09-10T16:10:52.168075+0000","mapping_epoch":35,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18949,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20798","reported_seq":29507,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.168961+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:10:52.168961+0000","last_peered":"2024-09-10T16:10:52.168961+0000","last_clean":"2024-09-10T16:10:52.168961+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:10:52.168961+0000","last_undegraded":"2024-09-10T16:10:52.168961+0000","last_fullsized":"2024-09-10T16:10:52.168961+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18802,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20660","reported_seq":20772,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.167700+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:10:52.167700+0000","last_peered":"2024-09-10T16:10:52.167700+0000","last_clean":"2024-09-10T16:10:52.167700+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:10:52.167700+0000","last_undegraded":"2024-09-10T16:10:52.167700+0000","last_fullsized":"2024-09-10T16:10:52.167700+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20660,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20891","reported_seq":21052,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.169809+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:10:52.169809+0000","last_peered":"2024-09-10T16:10:52.169809+0000","last_clean":"2024-09-10T16:10:52.169809+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:10:52.169809+0000","last_undegraded":"2024-09-10T16:10:52.169809+0000","last_fullsized":"2024-09-10T16:10:52.169809+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17602,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'21061","reported_seq":21222,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.242289+0000","last_change":"2024-09-10T16:10:52.241702+0000","last_active":"2024-09-10T16:10:52.242289+0000","last_peered":"2024-09-10T16:10:52.242289+0000","last_clean":"2024-09-10T16:10:52.242289+0000","last_became_active":"2024-09-10T16:10:52.171236+0000","last_became_peered":"2024-09-10T16:10:52.171236+0000","last_unstale":"2024-09-10T16:10:52.242289+0000","last_undegraded":"2024-09-10T16:10:52.242289+0000","last_fullsized":"2024-09-10T16:10:52.242289+0000","mapping_epoch":58,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:04:46.778280+0000","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17673,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4372,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'21082","reported_seq":21251,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:10:52.170050+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:10:52.170050+0000","last_peered":"2024-09-10T16:10:52.170050+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:10:52.170050+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17581,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4235},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'21314","reported_seq":21475,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.169708+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:10:52.169708+0000","last_peered":"2024-09-10T16:10:52.169708+0000","last_clean":"2024-09-10T16:10:52.169708+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:10:52.169708+0000","last_undegraded":"2024-09-10T16:10:52.169708+0000","last_fullsized":"2024-09-10T16:10:52.169708+0000","mapping_epoch":34,"log_start":"45'11300","ondisk_log_start":"45'11300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18161,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.168728+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:10:52.168728+0000","last_peered":"2024-09-10T16:10:52.168728+0000","last_clean":"2024-09-10T16:10:52.168728+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:10:52.168728+0000","last_undegraded":"2024-09-10T16:10:52.168728+0000","last_fullsized":"2024-09-10T16:10:52.168728+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20457","reported_seq":20632,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.168901+0000","last_change":"2024-09-10T16:09:33.012840+0000","last_active":"2024-09-10T16:10:52.168901+0000","last_peered":"2024-09-10T16:10:52.168901+0000","last_clean":"2024-09-10T16:10:52.168901+0000","last_became_active":"2024-09-10T16:09:32.934056+0000","last_became_peered":"2024-09-10T16:09:32.934056+0000","last_unstale":"2024-09-10T16:10:52.168901+0000","last_undegraded":"2024-09-10T16:10:52.168901+0000","last_fullsized":"2024-09-10T16:10:52.168901+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:08:30.481235+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18276,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":4579,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":126,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.168004+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:10:52.168004+0000","last_peered":"2024-09-10T16:10:52.168004+0000","last_clean":"2024-09-10T16:10:52.168004+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:10:52.168004+0000","last_undegraded":"2024-09-10T16:10:52.168004+0000","last_fullsized":"2024-09-10T16:10:52.168004+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20910","reported_seq":29594,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.169792+0000","last_change":"2024-09-10T16:09:08.909580+0000","last_active":"2024-09-10T16:10:52.169792+0000","last_peered":"2024-09-10T16:10:52.169792+0000","last_clean":"2024-09-10T16:10:52.169792+0000","last_became_active":"2024-09-10T16:09:08.854941+0000","last_became_peered":"2024-09-10T16:09:08.854941+0000","last_unstale":"2024-09-10T16:10:52.169792+0000","last_undegraded":"2024-09-10T16:10:52.169792+0000","last_fullsized":"2024-09-10T16:10:52.169792+0000","mapping_epoch":50,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:13:14.846691+0000","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17414,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4257,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":136,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.169714+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:10:52.169714+0000","last_peered":"2024-09-10T16:10:52.169714+0000","last_clean":"2024-09-10T16:10:52.169714+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:10:52.169714+0000","last_undegraded":"2024-09-10T16:10:52.169714+0000","last_fullsized":"2024-09-10T16:10:52.169714+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20781","reported_seq":29566,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.168064+0000","last_change":"2024-09-10T16:10:20.907889+0000","last_active":"2024-09-10T16:10:52.168064+0000","last_peered":"2024-09-10T16:10:52.168064+0000","last_clean":"2024-09-10T16:10:52.168064+0000","last_became_active":"2024-09-10T16:10:20.852065+0000","last_became_peered":"2024-09-10T16:10:20.852065+0000","last_unstale":"2024-09-10T16:10:52.168064+0000","last_undegraded":"2024-09-10T16:10:52.168064+0000","last_fullsized":"2024-09-10T16:10:52.168064+0000","mapping_epoch":56,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:34.069549+0000","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18461,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4307,"num_bytes_recovered":8675,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":111,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.168792+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:10:52.168792+0000","last_peered":"2024-09-10T16:10:52.168792+0000","last_clean":"2024-09-10T16:10:52.168792+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:10:52.168792+0000","last_undegraded":"2024-09-10T16:10:52.168792+0000","last_fullsized":"2024-09-10T16:10:52.168792+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'21170","reported_seq":30051,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.167373+0000","last_change":"2024-09-10T16:10:50.728349+0000","last_active":"2024-09-10T16:10:52.167373+0000","last_peered":"2024-09-10T16:10:52.167373+0000","last_clean":"2024-09-10T16:10:52.167373+0000","last_became_active":"2024-09-10T16:08:10.860438+0000","last_became_peered":"2024-09-10T16:08:10.860438+0000","last_unstale":"2024-09-10T16:10:52.167373+0000","last_undegraded":"2024-09-10T16:10:52.167373+0000","last_fullsized":"2024-09-10T16:10:52.167373+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:38:13.781380+0000","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18724,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20838","reported_seq":21004,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:10:52.169840+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:10:52.169840+0000","last_peered":"2024-09-10T16:10:52.169840+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:10:52.169840+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17469,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.168992+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:10:52.168992+0000","last_peered":"2024-09-10T16:10:52.168992+0000","last_clean":"2024-09-10T16:10:52.168992+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:10:52.168992+0000","last_undegraded":"2024-09-10T16:10:52.168992+0000","last_fullsized":"2024-09-10T16:10:52.168992+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20598","reported_seq":29195,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.169145+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:10:52.169145+0000","last_peered":"2024-09-10T16:10:52.169145+0000","last_clean":"2024-09-10T16:10:52.169145+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:10:52.169145+0000","last_undegraded":"2024-09-10T16:10:52.169145+0000","last_fullsized":"2024-09-10T16:10:52.169145+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16988,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":126,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.167911+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:10:52.167911+0000","last_peered":"2024-09-10T16:10:52.167911+0000","last_clean":"2024-09-10T16:10:52.167911+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:10:52.167911+0000","last_undegraded":"2024-09-10T16:10:52.167911+0000","last_fullsized":"2024-09-10T16:10:52.167911+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'21048","reported_seq":23051,"reported_epoch":59,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:10:56.223190+0000","last_change":"2024-09-10T16:10:50.728566+0000","last_active":"2024-09-10T16:10:56.223190+0000","last_peered":"2024-09-10T16:10:56.223190+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:10:56.223190+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3459,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17303,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":914,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3459}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.169062+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:10:52.169062+0000","last_peered":"2024-09-10T16:10:52.169062+0000","last_clean":"2024-09-10T16:10:52.169062+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:10:52.169062+0000","last_undegraded":"2024-09-10T16:10:52.169062+0000","last_fullsized":"2024-09-10T16:10:52.169062+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":106,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.169114+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:10:52.169114+0000","last_peered":"2024-09-10T16:10:52.169114+0000","last_clean":"2024-09-10T16:10:52.169114+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:10:52.169114+0000","last_undegraded":"2024-09-10T16:10:52.169114+0000","last_fullsized":"2024-09-10T16:10:52.169114+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'21198","reported_seq":21374,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:10:52.169851+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:10:52.169851+0000","last_peered":"2024-09-10T16:10:52.169851+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:10:52.169851+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17361,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4321},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":16326,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292096,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35064,"num_bytes_recovered":33097129,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":207814656,"data_stored":207706266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161022,"ondisk_log_size":161022,"up":32,"acting":28,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986331,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":500408,"kb_used_data":135528,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93871432,"statfs":{"total":96636764160,"available":96124346368,"internally_reserved":0,"allocated":138780672,"data_stored":138418304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659276,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487580,"kb_used_data":524,"kb_used_omap":4,"kb_used_meta":487035,"kb_avail":93884260,"statfs":{"total":96636764160,"available":96137482240,"internally_reserved":0,"allocated":536576,"data_stored":170131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":498724526},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1],"upper_bound":16},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182559,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":547376,"kb_used_data":68700,"kb_used_omap":4,"kb_used_meta":478651,"kb_avail":93824464,"statfs":{"total":96636764160,"available":96076251136,"internally_reserved":0,"allocated":70348800,"data_stored":69982355,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":490139306},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":69853184,"data_stored":69817054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":4830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":137920512,"data_stored":137884382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:11:02.833 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:11:03.138 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:11:03.139 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:11:03.152 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":465,"stamp":"2024-09-10T16:11:01.789538+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":15534,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292098,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35858,"num_bytes_recovered":33097148,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161025,"ondisk_log_size":161025,"up":48,"acting":44,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1524644,"kb_used_data":202352,"kb_used_omap":12,"kb_used_meta":1322227,"kb_avail":281590876,"statfs":{"total":289910292480,"available":288349057024,"internally_reserved":0,"allocated":207208448,"data_stored":206113190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1353960450},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2666,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2666,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.001694"},"pg_stats":[{"pgid":"2.f","version":"49'20672","reported_seq":29304,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.169962+0000","last_change":"2024-09-10T16:09:57.088531+0000","last_active":"2024-09-10T16:10:52.169962+0000","last_peered":"2024-09-10T16:10:52.169962+0000","last_clean":"2024-09-10T16:10:52.169962+0000","last_became_active":"2024-09-10T16:09:57.005755+0000","last_became_peered":"2024-09-10T16:09:57.005755+0000","last_unstale":"2024-09-10T16:10:52.169962+0000","last_undegraded":"2024-09-10T16:10:52.169962+0000","last_fullsized":"2024-09-10T16:10:52.169962+0000","mapping_epoch":54,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:09:19.329697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20672,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4259,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20844","reported_seq":20988,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.168075+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:10:52.168075+0000","last_peered":"2024-09-10T16:10:52.168075+0000","last_clean":"2024-09-10T16:10:52.168075+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:10:52.168075+0000","last_undegraded":"2024-09-10T16:10:52.168075+0000","last_fullsized":"2024-09-10T16:10:52.168075+0000","mapping_epoch":35,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18949,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20798","reported_seq":29507,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.168961+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:10:52.168961+0000","last_peered":"2024-09-10T16:10:52.168961+0000","last_clean":"2024-09-10T16:10:52.168961+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:10:52.168961+0000","last_undegraded":"2024-09-10T16:10:52.168961+0000","last_fullsized":"2024-09-10T16:10:52.168961+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18802,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20660","reported_seq":20772,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.167700+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:10:52.167700+0000","last_peered":"2024-09-10T16:10:52.167700+0000","last_clean":"2024-09-10T16:10:52.167700+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:10:52.167700+0000","last_undegraded":"2024-09-10T16:10:52.167700+0000","last_fullsized":"2024-09-10T16:10:52.167700+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20660,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20891","reported_seq":21052,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.169809+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:10:52.169809+0000","last_peered":"2024-09-10T16:10:52.169809+0000","last_clean":"2024-09-10T16:10:52.169809+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:10:52.169809+0000","last_undegraded":"2024-09-10T16:10:52.169809+0000","last_fullsized":"2024-09-10T16:10:52.169809+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17602,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'21061","reported_seq":21222,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.242289+0000","last_change":"2024-09-10T16:10:52.241702+0000","last_active":"2024-09-10T16:10:52.242289+0000","last_peered":"2024-09-10T16:10:52.242289+0000","last_clean":"2024-09-10T16:10:52.242289+0000","last_became_active":"2024-09-10T16:10:52.171236+0000","last_became_peered":"2024-09-10T16:10:52.171236+0000","last_unstale":"2024-09-10T16:10:52.242289+0000","last_undegraded":"2024-09-10T16:10:52.242289+0000","last_fullsized":"2024-09-10T16:10:52.242289+0000","mapping_epoch":58,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:04:46.778280+0000","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17673,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4372,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'21082","reported_seq":21251,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:10:52.170050+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:10:52.170050+0000","last_peered":"2024-09-10T16:10:52.170050+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:10:52.170050+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17581,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4235},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'21314","reported_seq":21475,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.169708+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:10:52.169708+0000","last_peered":"2024-09-10T16:10:52.169708+0000","last_clean":"2024-09-10T16:10:52.169708+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:10:52.169708+0000","last_undegraded":"2024-09-10T16:10:52.169708+0000","last_fullsized":"2024-09-10T16:10:52.169708+0000","mapping_epoch":34,"log_start":"45'11300","ondisk_log_start":"45'11300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18161,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.168728+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:10:52.168728+0000","last_peered":"2024-09-10T16:10:52.168728+0000","last_clean":"2024-09-10T16:10:52.168728+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:10:52.168728+0000","last_undegraded":"2024-09-10T16:10:52.168728+0000","last_fullsized":"2024-09-10T16:10:52.168728+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20457","reported_seq":20632,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.168901+0000","last_change":"2024-09-10T16:09:33.012840+0000","last_active":"2024-09-10T16:10:52.168901+0000","last_peered":"2024-09-10T16:10:52.168901+0000","last_clean":"2024-09-10T16:10:52.168901+0000","last_became_active":"2024-09-10T16:09:32.934056+0000","last_became_peered":"2024-09-10T16:09:32.934056+0000","last_unstale":"2024-09-10T16:10:52.168901+0000","last_undegraded":"2024-09-10T16:10:52.168901+0000","last_fullsized":"2024-09-10T16:10:52.168901+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:08:30.481235+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18276,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":4579,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":126,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.168004+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:10:52.168004+0000","last_peered":"2024-09-10T16:10:52.168004+0000","last_clean":"2024-09-10T16:10:52.168004+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:10:52.168004+0000","last_undegraded":"2024-09-10T16:10:52.168004+0000","last_fullsized":"2024-09-10T16:10:52.168004+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20910","reported_seq":29594,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.169792+0000","last_change":"2024-09-10T16:09:08.909580+0000","last_active":"2024-09-10T16:10:52.169792+0000","last_peered":"2024-09-10T16:10:52.169792+0000","last_clean":"2024-09-10T16:10:52.169792+0000","last_became_active":"2024-09-10T16:09:08.854941+0000","last_became_peered":"2024-09-10T16:09:08.854941+0000","last_unstale":"2024-09-10T16:10:52.169792+0000","last_undegraded":"2024-09-10T16:10:52.169792+0000","last_fullsized":"2024-09-10T16:10:52.169792+0000","mapping_epoch":50,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:13:14.846691+0000","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17414,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4257,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":136,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.169714+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:10:52.169714+0000","last_peered":"2024-09-10T16:10:52.169714+0000","last_clean":"2024-09-10T16:10:52.169714+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:10:52.169714+0000","last_undegraded":"2024-09-10T16:10:52.169714+0000","last_fullsized":"2024-09-10T16:10:52.169714+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20781","reported_seq":29566,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.168064+0000","last_change":"2024-09-10T16:10:20.907889+0000","last_active":"2024-09-10T16:10:52.168064+0000","last_peered":"2024-09-10T16:10:52.168064+0000","last_clean":"2024-09-10T16:10:52.168064+0000","last_became_active":"2024-09-10T16:10:20.852065+0000","last_became_peered":"2024-09-10T16:10:20.852065+0000","last_unstale":"2024-09-10T16:10:52.168064+0000","last_undegraded":"2024-09-10T16:10:52.168064+0000","last_fullsized":"2024-09-10T16:10:52.168064+0000","mapping_epoch":56,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:34.069549+0000","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18461,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4307,"num_bytes_recovered":8675,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":111,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.168792+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:10:52.168792+0000","last_peered":"2024-09-10T16:10:52.168792+0000","last_clean":"2024-09-10T16:10:52.168792+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:10:52.168792+0000","last_undegraded":"2024-09-10T16:10:52.168792+0000","last_fullsized":"2024-09-10T16:10:52.168792+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'21170","reported_seq":30051,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.167373+0000","last_change":"2024-09-10T16:10:50.728349+0000","last_active":"2024-09-10T16:10:52.167373+0000","last_peered":"2024-09-10T16:10:52.167373+0000","last_clean":"2024-09-10T16:10:52.167373+0000","last_became_active":"2024-09-10T16:08:10.860438+0000","last_became_peered":"2024-09-10T16:08:10.860438+0000","last_unstale":"2024-09-10T16:10:52.167373+0000","last_undegraded":"2024-09-10T16:10:52.167373+0000","last_fullsized":"2024-09-10T16:10:52.167373+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:38:13.781380+0000","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18724,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20838","reported_seq":21004,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:10:52.169840+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:10:52.169840+0000","last_peered":"2024-09-10T16:10:52.169840+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:10:52.169840+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17469,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.168992+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:10:52.168992+0000","last_peered":"2024-09-10T16:10:52.168992+0000","last_clean":"2024-09-10T16:10:52.168992+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:10:52.168992+0000","last_undegraded":"2024-09-10T16:10:52.168992+0000","last_fullsized":"2024-09-10T16:10:52.168992+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20598","reported_seq":29195,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.169145+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:10:52.169145+0000","last_peered":"2024-09-10T16:10:52.169145+0000","last_clean":"2024-09-10T16:10:52.169145+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:10:52.169145+0000","last_undegraded":"2024-09-10T16:10:52.169145+0000","last_fullsized":"2024-09-10T16:10:52.169145+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16988,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":126,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.167911+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:10:52.167911+0000","last_peered":"2024-09-10T16:10:52.167911+0000","last_clean":"2024-09-10T16:10:52.167911+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:10:52.167911+0000","last_undegraded":"2024-09-10T16:10:52.167911+0000","last_fullsized":"2024-09-10T16:10:52.167911+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'21048","reported_seq":24635,"reported_epoch":59,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:11:01.221760+0000","last_change":"2024-09-10T16:10:50.728566+0000","last_active":"2024-09-10T16:11:01.221760+0000","last_peered":"2024-09-10T16:11:01.221760+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:11:01.221760+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2667,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17303,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1706,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2667}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.169062+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:10:52.169062+0000","last_peered":"2024-09-10T16:10:52.169062+0000","last_clean":"2024-09-10T16:10:52.169062+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:10:52.169062+0000","last_undegraded":"2024-09-10T16:10:52.169062+0000","last_fullsized":"2024-09-10T16:10:52.169062+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":106,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.169114+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:10:52.169114+0000","last_peered":"2024-09-10T16:10:52.169114+0000","last_clean":"2024-09-10T16:10:52.169114+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:10:52.169114+0000","last_undegraded":"2024-09-10T16:10:52.169114+0000","last_fullsized":"2024-09-10T16:10:52.169114+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'21198","reported_seq":21374,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:10:52.169851+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:10:52.169851+0000","last_peered":"2024-09-10T16:10:52.169851+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:10:52.169851+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17361,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4321},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":15534,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292096,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35856,"num_bytes_recovered":33097129,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":205357056,"data_stored":205248666,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161022,"ondisk_log_size":161022,"up":32,"acting":28,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986332,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":498008,"kb_used_data":133128,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93873832,"statfs":{"total":96636764160,"available":96126803968,"internally_reserved":0,"allocated":136323072,"data_stored":135960704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659277,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":479260,"kb_used_data":524,"kb_used_omap":4,"kb_used_meta":478715,"kb_avail":93892580,"statfs":{"total":96636764160,"available":96146001920,"internally_reserved":0,"allocated":536576,"data_stored":170131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":490204846},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182560,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":547376,"kb_used_data":68700,"kb_used_omap":4,"kb_used_meta":478651,"kb_avail":93824464,"statfs":{"total":96636764160,"available":96076251136,"internally_reserved":0,"allocated":70348800,"data_stored":69982355,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":490139306},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":69853184,"data_stored":69817054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":4830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":135462912,"data_stored":135426782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:11:03.154 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:11:03.469 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:11:03.470 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:11:03.482 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":465,"stamp":"2024-09-10T16:11:01.789538+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":15534,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292098,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35858,"num_bytes_recovered":33097148,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161025,"ondisk_log_size":161025,"up":48,"acting":44,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1524644,"kb_used_data":202352,"kb_used_omap":12,"kb_used_meta":1322227,"kb_avail":281590876,"statfs":{"total":289910292480,"available":288349057024,"internally_reserved":0,"allocated":207208448,"data_stored":206113190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1353960450},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2666,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2666,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.001694"},"pg_stats":[{"pgid":"2.f","version":"49'20672","reported_seq":29304,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.169962+0000","last_change":"2024-09-10T16:09:57.088531+0000","last_active":"2024-09-10T16:10:52.169962+0000","last_peered":"2024-09-10T16:10:52.169962+0000","last_clean":"2024-09-10T16:10:52.169962+0000","last_became_active":"2024-09-10T16:09:57.005755+0000","last_became_peered":"2024-09-10T16:09:57.005755+0000","last_unstale":"2024-09-10T16:10:52.169962+0000","last_undegraded":"2024-09-10T16:10:52.169962+0000","last_fullsized":"2024-09-10T16:10:52.169962+0000","mapping_epoch":54,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:09:19.329697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20672,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4259,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20844","reported_seq":20988,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.168075+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:10:52.168075+0000","last_peered":"2024-09-10T16:10:52.168075+0000","last_clean":"2024-09-10T16:10:52.168075+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:10:52.168075+0000","last_undegraded":"2024-09-10T16:10:52.168075+0000","last_fullsized":"2024-09-10T16:10:52.168075+0000","mapping_epoch":35,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18949,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20798","reported_seq":29507,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.168961+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:10:52.168961+0000","last_peered":"2024-09-10T16:10:52.168961+0000","last_clean":"2024-09-10T16:10:52.168961+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:10:52.168961+0000","last_undegraded":"2024-09-10T16:10:52.168961+0000","last_fullsized":"2024-09-10T16:10:52.168961+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18802,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20660","reported_seq":20772,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.167700+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:10:52.167700+0000","last_peered":"2024-09-10T16:10:52.167700+0000","last_clean":"2024-09-10T16:10:52.167700+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:10:52.167700+0000","last_undegraded":"2024-09-10T16:10:52.167700+0000","last_fullsized":"2024-09-10T16:10:52.167700+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20660,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20891","reported_seq":21052,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.169809+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:10:52.169809+0000","last_peered":"2024-09-10T16:10:52.169809+0000","last_clean":"2024-09-10T16:10:52.169809+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:10:52.169809+0000","last_undegraded":"2024-09-10T16:10:52.169809+0000","last_fullsized":"2024-09-10T16:10:52.169809+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17602,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'21061","reported_seq":21222,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.242289+0000","last_change":"2024-09-10T16:10:52.241702+0000","last_active":"2024-09-10T16:10:52.242289+0000","last_peered":"2024-09-10T16:10:52.242289+0000","last_clean":"2024-09-10T16:10:52.242289+0000","last_became_active":"2024-09-10T16:10:52.171236+0000","last_became_peered":"2024-09-10T16:10:52.171236+0000","last_unstale":"2024-09-10T16:10:52.242289+0000","last_undegraded":"2024-09-10T16:10:52.242289+0000","last_fullsized":"2024-09-10T16:10:52.242289+0000","mapping_epoch":58,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:04:46.778280+0000","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17673,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4372,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'21082","reported_seq":21251,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:10:52.170050+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:10:52.170050+0000","last_peered":"2024-09-10T16:10:52.170050+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:10:52.170050+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17581,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4235},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'21314","reported_seq":21475,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.169708+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:10:52.169708+0000","last_peered":"2024-09-10T16:10:52.169708+0000","last_clean":"2024-09-10T16:10:52.169708+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:10:52.169708+0000","last_undegraded":"2024-09-10T16:10:52.169708+0000","last_fullsized":"2024-09-10T16:10:52.169708+0000","mapping_epoch":34,"log_start":"45'11300","ondisk_log_start":"45'11300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18161,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.168728+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:10:52.168728+0000","last_peered":"2024-09-10T16:10:52.168728+0000","last_clean":"2024-09-10T16:10:52.168728+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:10:52.168728+0000","last_undegraded":"2024-09-10T16:10:52.168728+0000","last_fullsized":"2024-09-10T16:10:52.168728+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20457","reported_seq":20632,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.168901+0000","last_change":"2024-09-10T16:09:33.012840+0000","last_active":"2024-09-10T16:10:52.168901+0000","last_peered":"2024-09-10T16:10:52.168901+0000","last_clean":"2024-09-10T16:10:52.168901+0000","last_became_active":"2024-09-10T16:09:32.934056+0000","last_became_peered":"2024-09-10T16:09:32.934056+0000","last_unstale":"2024-09-10T16:10:52.168901+0000","last_undegraded":"2024-09-10T16:10:52.168901+0000","last_fullsized":"2024-09-10T16:10:52.168901+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:08:30.481235+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18276,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":4579,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":126,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.168004+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:10:52.168004+0000","last_peered":"2024-09-10T16:10:52.168004+0000","last_clean":"2024-09-10T16:10:52.168004+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:10:52.168004+0000","last_undegraded":"2024-09-10T16:10:52.168004+0000","last_fullsized":"2024-09-10T16:10:52.168004+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20910","reported_seq":29594,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.169792+0000","last_change":"2024-09-10T16:09:08.909580+0000","last_active":"2024-09-10T16:10:52.169792+0000","last_peered":"2024-09-10T16:10:52.169792+0000","last_clean":"2024-09-10T16:10:52.169792+0000","last_became_active":"2024-09-10T16:09:08.854941+0000","last_became_peered":"2024-09-10T16:09:08.854941+0000","last_unstale":"2024-09-10T16:10:52.169792+0000","last_undegraded":"2024-09-10T16:10:52.169792+0000","last_fullsized":"2024-09-10T16:10:52.169792+0000","mapping_epoch":50,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:13:14.846691+0000","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17414,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4257,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":136,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.169714+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:10:52.169714+0000","last_peered":"2024-09-10T16:10:52.169714+0000","last_clean":"2024-09-10T16:10:52.169714+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:10:52.169714+0000","last_undegraded":"2024-09-10T16:10:52.169714+0000","last_fullsized":"2024-09-10T16:10:52.169714+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20781","reported_seq":29566,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.168064+0000","last_change":"2024-09-10T16:10:20.907889+0000","last_active":"2024-09-10T16:10:52.168064+0000","last_peered":"2024-09-10T16:10:52.168064+0000","last_clean":"2024-09-10T16:10:52.168064+0000","last_became_active":"2024-09-10T16:10:20.852065+0000","last_became_peered":"2024-09-10T16:10:20.852065+0000","last_unstale":"2024-09-10T16:10:52.168064+0000","last_undegraded":"2024-09-10T16:10:52.168064+0000","last_fullsized":"2024-09-10T16:10:52.168064+0000","mapping_epoch":56,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:34.069549+0000","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18461,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4307,"num_bytes_recovered":8675,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":111,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.168792+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:10:52.168792+0000","last_peered":"2024-09-10T16:10:52.168792+0000","last_clean":"2024-09-10T16:10:52.168792+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:10:52.168792+0000","last_undegraded":"2024-09-10T16:10:52.168792+0000","last_fullsized":"2024-09-10T16:10:52.168792+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'21170","reported_seq":30051,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.167373+0000","last_change":"2024-09-10T16:10:50.728349+0000","last_active":"2024-09-10T16:10:52.167373+0000","last_peered":"2024-09-10T16:10:52.167373+0000","last_clean":"2024-09-10T16:10:52.167373+0000","last_became_active":"2024-09-10T16:08:10.860438+0000","last_became_peered":"2024-09-10T16:08:10.860438+0000","last_unstale":"2024-09-10T16:10:52.167373+0000","last_undegraded":"2024-09-10T16:10:52.167373+0000","last_fullsized":"2024-09-10T16:10:52.167373+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:38:13.781380+0000","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18724,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20838","reported_seq":21004,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:10:52.169840+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:10:52.169840+0000","last_peered":"2024-09-10T16:10:52.169840+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:10:52.169840+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17469,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.168992+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:10:52.168992+0000","last_peered":"2024-09-10T16:10:52.168992+0000","last_clean":"2024-09-10T16:10:52.168992+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:10:52.168992+0000","last_undegraded":"2024-09-10T16:10:52.168992+0000","last_fullsized":"2024-09-10T16:10:52.168992+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20598","reported_seq":29195,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.169145+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:10:52.169145+0000","last_peered":"2024-09-10T16:10:52.169145+0000","last_clean":"2024-09-10T16:10:52.169145+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:10:52.169145+0000","last_undegraded":"2024-09-10T16:10:52.169145+0000","last_fullsized":"2024-09-10T16:10:52.169145+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16988,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":126,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.167911+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:10:52.167911+0000","last_peered":"2024-09-10T16:10:52.167911+0000","last_clean":"2024-09-10T16:10:52.167911+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:10:52.167911+0000","last_undegraded":"2024-09-10T16:10:52.167911+0000","last_fullsized":"2024-09-10T16:10:52.167911+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'21048","reported_seq":24635,"reported_epoch":59,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:11:01.221760+0000","last_change":"2024-09-10T16:10:50.728566+0000","last_active":"2024-09-10T16:11:01.221760+0000","last_peered":"2024-09-10T16:11:01.221760+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:11:01.221760+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2667,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17303,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1706,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2667}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.169062+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:10:52.169062+0000","last_peered":"2024-09-10T16:10:52.169062+0000","last_clean":"2024-09-10T16:10:52.169062+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:10:52.169062+0000","last_undegraded":"2024-09-10T16:10:52.169062+0000","last_fullsized":"2024-09-10T16:10:52.169062+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":106,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.169114+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:10:52.169114+0000","last_peered":"2024-09-10T16:10:52.169114+0000","last_clean":"2024-09-10T16:10:52.169114+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:10:52.169114+0000","last_undegraded":"2024-09-10T16:10:52.169114+0000","last_fullsized":"2024-09-10T16:10:52.169114+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'21198","reported_seq":21374,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:10:52.169851+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:10:52.169851+0000","last_peered":"2024-09-10T16:10:52.169851+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:10:52.169851+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17361,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4321},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":15534,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292096,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35856,"num_bytes_recovered":33097129,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":205357056,"data_stored":205248666,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161022,"ondisk_log_size":161022,"up":32,"acting":28,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986332,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":498008,"kb_used_data":133128,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93873832,"statfs":{"total":96636764160,"available":96126803968,"internally_reserved":0,"allocated":136323072,"data_stored":135960704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659277,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":479260,"kb_used_data":524,"kb_used_omap":4,"kb_used_meta":478715,"kb_avail":93892580,"statfs":{"total":96636764160,"available":96146001920,"internally_reserved":0,"allocated":536576,"data_stored":170131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":490204846},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182560,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":547376,"kb_used_data":68700,"kb_used_omap":4,"kb_used_meta":478651,"kb_avail":93824464,"statfs":{"total":96636764160,"available":96076251136,"internally_reserved":0,"allocated":70348800,"data_stored":69982355,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":490139306},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":69853184,"data_stored":69817054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":4830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":135462912,"data_stored":135426782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:11:03.483 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-10T16:11:03.483 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "49'21082", 'reported_seq': 21251, 'reported_epoch': 59, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:10:52.170050+0000', 'last_change': '2024-09-10T15:59:16.331899+0000', 'last_active': '2024-09-10T16:10:52.170050+0000', 'last_peered': '2024-09-10T16:10:52.170050+0000', 'last_clean': '2024-09-10T15:59:14.139436+0000', 'last_became_active': '2024-09-10T15:59:16.318715+0000', 'last_became_peered': '2024-09-10T15:59:16.318715+0000', 'last_unstale': '2024-09-10T16:10:52.170050+0000', 'last_undegraded': '2024-09-10T15:59:15.877524+0000', 'last_fullsized': '2024-09-10T15:59:15.769042+0000', 'mapping_epoch': 42, 'log_start': "45'11000", 'ondisk_log_start': "45'11000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3537", 'last_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'objects_scrubbed': 3497, 'log_size': 10082, 'log_dups_size': 0, 'ondisk_log_size': 10082, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1224, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4237, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 17581, 'num_write_kb': 28172, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4235}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:11:03.483 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-10T16:11:03.483 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "49'20838", 'reported_seq': 21004, 'reported_epoch': 59, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:10:52.169840+0000', 'last_change': '2024-09-10T15:59:16.315641+0000', 'last_active': '2024-09-10T16:10:52.169840+0000', 'last_peered': '2024-09-10T16:10:52.169840+0000', 'last_clean': '2024-09-10T15:59:14.117001+0000', 'last_became_active': '2024-09-10T15:59:16.302245+0000', 'last_became_peered': '2024-09-10T15:59:16.302245+0000', 'last_unstale': '2024-09-10T16:10:52.169840+0000', 'last_undegraded': '2024-09-10T15:59:15.845012+0000', 'last_fullsized': '2024-09-10T15:59:15.770337+0000', 'mapping_epoch': 42, 'log_start': "45'10800", 'ondisk_log_start': "45'10800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3404", 'last_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'objects_scrubbed': 3367, 'log_size': 10038, 'log_dups_size': 0, 'ondisk_log_size': 10038, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 983, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17469, 'num_write_kb': 28204, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4304}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:11:03.483 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-10T16:11:03.483 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "49'21048", 'reported_seq': 24635, 'reported_epoch': 59, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:11:01.221760+0000', 'last_change': '2024-09-10T16:10:50.728566+0000', 'last_active': '2024-09-10T16:11:01.221760+0000', 'last_peered': '2024-09-10T16:11:01.221760+0000', 'last_clean': '2024-09-10T15:59:14.116363+0000', 'last_became_active': '2024-09-10T15:59:16.301244+0000', 'last_became_peered': '2024-09-10T15:59:16.301244+0000', 'last_unstale': '2024-09-10T16:11:01.221760+0000', 'last_undegraded': '2024-09-10T15:59:15.883434+0000', 'last_fullsized': '2024-09-10T15:59:15.772756+0000', 'mapping_epoch': 42, 'log_start': "45'11000", 'ondisk_log_start': "45'11000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3789", 'last_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'objects_scrubbed': 3741, 'log_size': 10048, 'log_dups_size': 0, 'ondisk_log_size': 10048, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1440, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2667, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17303, 'num_write_kb': 27124, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1706, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2667}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:11:03.484 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-10T16:11:03.484 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "49'21198", 'reported_seq': 21374, 'reported_epoch': 59, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:10:52.169851+0000', 'last_change': '2024-09-10T15:59:16.313409+0000', 'last_active': '2024-09-10T16:10:52.169851+0000', 'last_peered': '2024-09-10T16:10:52.169851+0000', 'last_clean': '2024-09-10T15:59:14.129780+0000', 'last_became_active': '2024-09-10T15:59:16.300932+0000', 'last_became_peered': '2024-09-10T15:59:16.300932+0000', 'last_unstale': '2024-09-10T16:10:52.169851+0000', 'last_undegraded': '2024-09-10T15:59:15.887823+0000', 'last_fullsized': '2024-09-10T15:59:15.768513+0000', 'mapping_epoch': 42, 'log_start': "45'11100", 'ondisk_log_start': "45'11100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3880", 'last_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'objects_scrubbed': 3827, 'log_size': 10098, 'log_dups_size': 0, 'ondisk_log_size': 10098, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1397, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4325, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17361, 'num_write_kb': 27072, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4321}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:11:03.484 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T16:11:03.865 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:11:03.865 INFO:teuthology.orchestra.run.smithi115.stdout:{"fsid":"7effee29-df05-4351-9741-ee203564ed81","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 15534/24 objects degraded (64725.000%), 4 pgs degraded, 4 pgs undersized","count":8},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":827,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":59,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983954,"num_in_osds":3,"osd_in_since":1725983947,"num_remapped_pgs":4},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":20},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":3},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":12,"data_bytes":4849,"bytes_used":1561235456,"bytes_avail":288349057024,"bytes_total":289910292480,"degraded_objects":15534,"degraded_total":24,"degraded_ratio":647.25,"recovering_objects_per_sec":266,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":2666,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-10T15:57:14:967120+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T16:00:13.560776+0000","services":{}},"progress_events":{}} 2024-09-10T16:11:03.879 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-10T16:11:03.879 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:11:04.181 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:11:04.181 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:11:04.194 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":466,"stamp":"2024-09-10T16:11:03.789904+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":15534,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292098,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35858,"num_bytes_recovered":33097148,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161025,"ondisk_log_size":161025,"up":48,"acting":44,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1524644,"kb_used_data":202352,"kb_used_omap":12,"kb_used_meta":1322227,"kb_avail":281590876,"statfs":{"total":289910292480,"available":288349057024,"internally_reserved":0,"allocated":207208448,"data_stored":206113190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1353960450},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2666,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2666,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"11.624099"},"pg_stats":[{"pgid":"2.f","version":"49'20672","reported_seq":29304,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.169962+0000","last_change":"2024-09-10T16:09:57.088531+0000","last_active":"2024-09-10T16:10:52.169962+0000","last_peered":"2024-09-10T16:10:52.169962+0000","last_clean":"2024-09-10T16:10:52.169962+0000","last_became_active":"2024-09-10T16:09:57.005755+0000","last_became_peered":"2024-09-10T16:09:57.005755+0000","last_unstale":"2024-09-10T16:10:52.169962+0000","last_undegraded":"2024-09-10T16:10:52.169962+0000","last_fullsized":"2024-09-10T16:10:52.169962+0000","mapping_epoch":54,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:09:19.329697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20672,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4259,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20844","reported_seq":20988,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.168075+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:10:52.168075+0000","last_peered":"2024-09-10T16:10:52.168075+0000","last_clean":"2024-09-10T16:10:52.168075+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:10:52.168075+0000","last_undegraded":"2024-09-10T16:10:52.168075+0000","last_fullsized":"2024-09-10T16:10:52.168075+0000","mapping_epoch":35,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18949,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20798","reported_seq":29507,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.168961+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:10:52.168961+0000","last_peered":"2024-09-10T16:10:52.168961+0000","last_clean":"2024-09-10T16:10:52.168961+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:10:52.168961+0000","last_undegraded":"2024-09-10T16:10:52.168961+0000","last_fullsized":"2024-09-10T16:10:52.168961+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18802,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20660","reported_seq":20772,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.167700+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:10:52.167700+0000","last_peered":"2024-09-10T16:10:52.167700+0000","last_clean":"2024-09-10T16:10:52.167700+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:10:52.167700+0000","last_undegraded":"2024-09-10T16:10:52.167700+0000","last_fullsized":"2024-09-10T16:10:52.167700+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20660,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20891","reported_seq":21052,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.169809+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:10:52.169809+0000","last_peered":"2024-09-10T16:10:52.169809+0000","last_clean":"2024-09-10T16:10:52.169809+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:10:52.169809+0000","last_undegraded":"2024-09-10T16:10:52.169809+0000","last_fullsized":"2024-09-10T16:10:52.169809+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17602,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'21061","reported_seq":21222,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.242289+0000","last_change":"2024-09-10T16:10:52.241702+0000","last_active":"2024-09-10T16:10:52.242289+0000","last_peered":"2024-09-10T16:10:52.242289+0000","last_clean":"2024-09-10T16:10:52.242289+0000","last_became_active":"2024-09-10T16:10:52.171236+0000","last_became_peered":"2024-09-10T16:10:52.171236+0000","last_unstale":"2024-09-10T16:10:52.242289+0000","last_undegraded":"2024-09-10T16:10:52.242289+0000","last_fullsized":"2024-09-10T16:10:52.242289+0000","mapping_epoch":58,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:04:46.778280+0000","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17673,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4372,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'21082","reported_seq":21251,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:10:52.170050+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:10:52.170050+0000","last_peered":"2024-09-10T16:10:52.170050+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:10:52.170050+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17581,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4235},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'21314","reported_seq":21475,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.169708+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:10:52.169708+0000","last_peered":"2024-09-10T16:10:52.169708+0000","last_clean":"2024-09-10T16:10:52.169708+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:10:52.169708+0000","last_undegraded":"2024-09-10T16:10:52.169708+0000","last_fullsized":"2024-09-10T16:10:52.169708+0000","mapping_epoch":34,"log_start":"45'11300","ondisk_log_start":"45'11300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18161,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.168728+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:10:52.168728+0000","last_peered":"2024-09-10T16:10:52.168728+0000","last_clean":"2024-09-10T16:10:52.168728+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:10:52.168728+0000","last_undegraded":"2024-09-10T16:10:52.168728+0000","last_fullsized":"2024-09-10T16:10:52.168728+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20457","reported_seq":20632,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.168901+0000","last_change":"2024-09-10T16:09:33.012840+0000","last_active":"2024-09-10T16:10:52.168901+0000","last_peered":"2024-09-10T16:10:52.168901+0000","last_clean":"2024-09-10T16:10:52.168901+0000","last_became_active":"2024-09-10T16:09:32.934056+0000","last_became_peered":"2024-09-10T16:09:32.934056+0000","last_unstale":"2024-09-10T16:10:52.168901+0000","last_undegraded":"2024-09-10T16:10:52.168901+0000","last_fullsized":"2024-09-10T16:10:52.168901+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:08:30.481235+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18276,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":4579,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":126,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.168004+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:10:52.168004+0000","last_peered":"2024-09-10T16:10:52.168004+0000","last_clean":"2024-09-10T16:10:52.168004+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:10:52.168004+0000","last_undegraded":"2024-09-10T16:10:52.168004+0000","last_fullsized":"2024-09-10T16:10:52.168004+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20910","reported_seq":29594,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.169792+0000","last_change":"2024-09-10T16:09:08.909580+0000","last_active":"2024-09-10T16:10:52.169792+0000","last_peered":"2024-09-10T16:10:52.169792+0000","last_clean":"2024-09-10T16:10:52.169792+0000","last_became_active":"2024-09-10T16:09:08.854941+0000","last_became_peered":"2024-09-10T16:09:08.854941+0000","last_unstale":"2024-09-10T16:10:52.169792+0000","last_undegraded":"2024-09-10T16:10:52.169792+0000","last_fullsized":"2024-09-10T16:10:52.169792+0000","mapping_epoch":50,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:13:14.846691+0000","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17414,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4257,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":136,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.169714+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:10:52.169714+0000","last_peered":"2024-09-10T16:10:52.169714+0000","last_clean":"2024-09-10T16:10:52.169714+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:10:52.169714+0000","last_undegraded":"2024-09-10T16:10:52.169714+0000","last_fullsized":"2024-09-10T16:10:52.169714+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20781","reported_seq":29566,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.168064+0000","last_change":"2024-09-10T16:10:20.907889+0000","last_active":"2024-09-10T16:10:52.168064+0000","last_peered":"2024-09-10T16:10:52.168064+0000","last_clean":"2024-09-10T16:10:52.168064+0000","last_became_active":"2024-09-10T16:10:20.852065+0000","last_became_peered":"2024-09-10T16:10:20.852065+0000","last_unstale":"2024-09-10T16:10:52.168064+0000","last_undegraded":"2024-09-10T16:10:52.168064+0000","last_fullsized":"2024-09-10T16:10:52.168064+0000","mapping_epoch":56,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:34.069549+0000","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18461,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4307,"num_bytes_recovered":8675,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":111,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.168792+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:10:52.168792+0000","last_peered":"2024-09-10T16:10:52.168792+0000","last_clean":"2024-09-10T16:10:52.168792+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:10:52.168792+0000","last_undegraded":"2024-09-10T16:10:52.168792+0000","last_fullsized":"2024-09-10T16:10:52.168792+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'21170","reported_seq":30051,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.167373+0000","last_change":"2024-09-10T16:10:50.728349+0000","last_active":"2024-09-10T16:10:52.167373+0000","last_peered":"2024-09-10T16:10:52.167373+0000","last_clean":"2024-09-10T16:10:52.167373+0000","last_became_active":"2024-09-10T16:08:10.860438+0000","last_became_peered":"2024-09-10T16:08:10.860438+0000","last_unstale":"2024-09-10T16:10:52.167373+0000","last_undegraded":"2024-09-10T16:10:52.167373+0000","last_fullsized":"2024-09-10T16:10:52.167373+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:38:13.781380+0000","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18724,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20838","reported_seq":21004,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:10:52.169840+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:10:52.169840+0000","last_peered":"2024-09-10T16:10:52.169840+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:10:52.169840+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17469,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.168992+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:10:52.168992+0000","last_peered":"2024-09-10T16:10:52.168992+0000","last_clean":"2024-09-10T16:10:52.168992+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:10:52.168992+0000","last_undegraded":"2024-09-10T16:10:52.168992+0000","last_fullsized":"2024-09-10T16:10:52.168992+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20598","reported_seq":29195,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.169145+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:10:52.169145+0000","last_peered":"2024-09-10T16:10:52.169145+0000","last_clean":"2024-09-10T16:10:52.169145+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:10:52.169145+0000","last_undegraded":"2024-09-10T16:10:52.169145+0000","last_fullsized":"2024-09-10T16:10:52.169145+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16988,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":126,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.167911+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:10:52.167911+0000","last_peered":"2024-09-10T16:10:52.167911+0000","last_clean":"2024-09-10T16:10:52.167911+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:10:52.167911+0000","last_undegraded":"2024-09-10T16:10:52.167911+0000","last_fullsized":"2024-09-10T16:10:52.167911+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'21048","reported_seq":24635,"reported_epoch":59,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:11:01.221760+0000","last_change":"2024-09-10T16:10:50.728566+0000","last_active":"2024-09-10T16:11:01.221760+0000","last_peered":"2024-09-10T16:11:01.221760+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:11:01.221760+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2667,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17303,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1706,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2667}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.169062+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:10:52.169062+0000","last_peered":"2024-09-10T16:10:52.169062+0000","last_clean":"2024-09-10T16:10:52.169062+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:10:52.169062+0000","last_undegraded":"2024-09-10T16:10:52.169062+0000","last_fullsized":"2024-09-10T16:10:52.169062+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":106,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.169114+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:10:52.169114+0000","last_peered":"2024-09-10T16:10:52.169114+0000","last_clean":"2024-09-10T16:10:52.169114+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:10:52.169114+0000","last_undegraded":"2024-09-10T16:10:52.169114+0000","last_fullsized":"2024-09-10T16:10:52.169114+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'21198","reported_seq":21374,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:10:52.169851+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:10:52.169851+0000","last_peered":"2024-09-10T16:10:52.169851+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:10:52.169851+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17361,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4321},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":15534,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292096,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35856,"num_bytes_recovered":33097129,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":205357056,"data_stored":205248666,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161022,"ondisk_log_size":161022,"up":32,"acting":28,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986332,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":498008,"kb_used_data":133128,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93873832,"statfs":{"total":96636764160,"available":96126803968,"internally_reserved":0,"allocated":136323072,"data_stored":135960704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659277,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":479260,"kb_used_data":524,"kb_used_omap":4,"kb_used_meta":478715,"kb_avail":93892580,"statfs":{"total":96636764160,"available":96146001920,"internally_reserved":0,"allocated":536576,"data_stored":170131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":490204846},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182560,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":547376,"kb_used_data":68700,"kb_used_omap":4,"kb_used_meta":478651,"kb_avail":93824464,"statfs":{"total":96636764160,"available":96076251136,"internally_reserved":0,"allocated":70348800,"data_stored":69982355,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":490139306},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":69853184,"data_stored":69817054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":4830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":135462912,"data_stored":135426782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:11:07.197 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:11:07.500 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:11:07.501 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:11:07.513 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":467,"stamp":"2024-09-10T16:11:05.790195+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":15534,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292098,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35858,"num_bytes_recovered":33097148,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161025,"ondisk_log_size":161025,"up":48,"acting":44,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1528804,"kb_used_data":202352,"kb_used_omap":12,"kb_used_meta":1326387,"kb_avail":281586716,"statfs":{"total":289910292480,"available":288344797184,"internally_reserved":0,"allocated":207208448,"data_stored":206113190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1358220290},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2666,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2666,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002021"},"pg_stats":[{"pgid":"2.f","version":"49'20672","reported_seq":29304,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.169962+0000","last_change":"2024-09-10T16:09:57.088531+0000","last_active":"2024-09-10T16:10:52.169962+0000","last_peered":"2024-09-10T16:10:52.169962+0000","last_clean":"2024-09-10T16:10:52.169962+0000","last_became_active":"2024-09-10T16:09:57.005755+0000","last_became_peered":"2024-09-10T16:09:57.005755+0000","last_unstale":"2024-09-10T16:10:52.169962+0000","last_undegraded":"2024-09-10T16:10:52.169962+0000","last_fullsized":"2024-09-10T16:10:52.169962+0000","mapping_epoch":54,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:09:19.329697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20672,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4259,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20844","reported_seq":20988,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.168075+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:10:52.168075+0000","last_peered":"2024-09-10T16:10:52.168075+0000","last_clean":"2024-09-10T16:10:52.168075+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:10:52.168075+0000","last_undegraded":"2024-09-10T16:10:52.168075+0000","last_fullsized":"2024-09-10T16:10:52.168075+0000","mapping_epoch":35,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18949,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20798","reported_seq":29507,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.168961+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:10:52.168961+0000","last_peered":"2024-09-10T16:10:52.168961+0000","last_clean":"2024-09-10T16:10:52.168961+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:10:52.168961+0000","last_undegraded":"2024-09-10T16:10:52.168961+0000","last_fullsized":"2024-09-10T16:10:52.168961+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18802,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20660","reported_seq":20772,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.167700+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:10:52.167700+0000","last_peered":"2024-09-10T16:10:52.167700+0000","last_clean":"2024-09-10T16:10:52.167700+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:10:52.167700+0000","last_undegraded":"2024-09-10T16:10:52.167700+0000","last_fullsized":"2024-09-10T16:10:52.167700+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20660,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20891","reported_seq":21052,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.169809+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:10:52.169809+0000","last_peered":"2024-09-10T16:10:52.169809+0000","last_clean":"2024-09-10T16:10:52.169809+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:10:52.169809+0000","last_undegraded":"2024-09-10T16:10:52.169809+0000","last_fullsized":"2024-09-10T16:10:52.169809+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17602,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'21061","reported_seq":21222,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.242289+0000","last_change":"2024-09-10T16:10:52.241702+0000","last_active":"2024-09-10T16:10:52.242289+0000","last_peered":"2024-09-10T16:10:52.242289+0000","last_clean":"2024-09-10T16:10:52.242289+0000","last_became_active":"2024-09-10T16:10:52.171236+0000","last_became_peered":"2024-09-10T16:10:52.171236+0000","last_unstale":"2024-09-10T16:10:52.242289+0000","last_undegraded":"2024-09-10T16:10:52.242289+0000","last_fullsized":"2024-09-10T16:10:52.242289+0000","mapping_epoch":58,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:04:46.778280+0000","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17673,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4372,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'21082","reported_seq":21251,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:10:52.170050+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:10:52.170050+0000","last_peered":"2024-09-10T16:10:52.170050+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:10:52.170050+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17581,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4235},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'21314","reported_seq":21475,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.169708+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:10:52.169708+0000","last_peered":"2024-09-10T16:10:52.169708+0000","last_clean":"2024-09-10T16:10:52.169708+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:10:52.169708+0000","last_undegraded":"2024-09-10T16:10:52.169708+0000","last_fullsized":"2024-09-10T16:10:52.169708+0000","mapping_epoch":34,"log_start":"45'11300","ondisk_log_start":"45'11300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18161,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.168728+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:10:52.168728+0000","last_peered":"2024-09-10T16:10:52.168728+0000","last_clean":"2024-09-10T16:10:52.168728+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:10:52.168728+0000","last_undegraded":"2024-09-10T16:10:52.168728+0000","last_fullsized":"2024-09-10T16:10:52.168728+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20457","reported_seq":20632,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.168901+0000","last_change":"2024-09-10T16:09:33.012840+0000","last_active":"2024-09-10T16:10:52.168901+0000","last_peered":"2024-09-10T16:10:52.168901+0000","last_clean":"2024-09-10T16:10:52.168901+0000","last_became_active":"2024-09-10T16:09:32.934056+0000","last_became_peered":"2024-09-10T16:09:32.934056+0000","last_unstale":"2024-09-10T16:10:52.168901+0000","last_undegraded":"2024-09-10T16:10:52.168901+0000","last_fullsized":"2024-09-10T16:10:52.168901+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:08:30.481235+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18276,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":4579,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":126,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.168004+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:10:52.168004+0000","last_peered":"2024-09-10T16:10:52.168004+0000","last_clean":"2024-09-10T16:10:52.168004+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:10:52.168004+0000","last_undegraded":"2024-09-10T16:10:52.168004+0000","last_fullsized":"2024-09-10T16:10:52.168004+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20910","reported_seq":29594,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.169792+0000","last_change":"2024-09-10T16:09:08.909580+0000","last_active":"2024-09-10T16:10:52.169792+0000","last_peered":"2024-09-10T16:10:52.169792+0000","last_clean":"2024-09-10T16:10:52.169792+0000","last_became_active":"2024-09-10T16:09:08.854941+0000","last_became_peered":"2024-09-10T16:09:08.854941+0000","last_unstale":"2024-09-10T16:10:52.169792+0000","last_undegraded":"2024-09-10T16:10:52.169792+0000","last_fullsized":"2024-09-10T16:10:52.169792+0000","mapping_epoch":50,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:13:14.846691+0000","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17414,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4257,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":136,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.169714+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:10:52.169714+0000","last_peered":"2024-09-10T16:10:52.169714+0000","last_clean":"2024-09-10T16:10:52.169714+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:10:52.169714+0000","last_undegraded":"2024-09-10T16:10:52.169714+0000","last_fullsized":"2024-09-10T16:10:52.169714+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20781","reported_seq":29566,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.168064+0000","last_change":"2024-09-10T16:10:20.907889+0000","last_active":"2024-09-10T16:10:52.168064+0000","last_peered":"2024-09-10T16:10:52.168064+0000","last_clean":"2024-09-10T16:10:52.168064+0000","last_became_active":"2024-09-10T16:10:20.852065+0000","last_became_peered":"2024-09-10T16:10:20.852065+0000","last_unstale":"2024-09-10T16:10:52.168064+0000","last_undegraded":"2024-09-10T16:10:52.168064+0000","last_fullsized":"2024-09-10T16:10:52.168064+0000","mapping_epoch":56,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:34.069549+0000","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18461,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4307,"num_bytes_recovered":8675,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":111,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.168792+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:10:52.168792+0000","last_peered":"2024-09-10T16:10:52.168792+0000","last_clean":"2024-09-10T16:10:52.168792+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:10:52.168792+0000","last_undegraded":"2024-09-10T16:10:52.168792+0000","last_fullsized":"2024-09-10T16:10:52.168792+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'21170","reported_seq":30051,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.167373+0000","last_change":"2024-09-10T16:10:50.728349+0000","last_active":"2024-09-10T16:10:52.167373+0000","last_peered":"2024-09-10T16:10:52.167373+0000","last_clean":"2024-09-10T16:10:52.167373+0000","last_became_active":"2024-09-10T16:08:10.860438+0000","last_became_peered":"2024-09-10T16:08:10.860438+0000","last_unstale":"2024-09-10T16:10:52.167373+0000","last_undegraded":"2024-09-10T16:10:52.167373+0000","last_fullsized":"2024-09-10T16:10:52.167373+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:38:13.781380+0000","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18724,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20838","reported_seq":21004,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:10:52.169840+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:10:52.169840+0000","last_peered":"2024-09-10T16:10:52.169840+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:10:52.169840+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17469,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.168992+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:10:52.168992+0000","last_peered":"2024-09-10T16:10:52.168992+0000","last_clean":"2024-09-10T16:10:52.168992+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:10:52.168992+0000","last_undegraded":"2024-09-10T16:10:52.168992+0000","last_fullsized":"2024-09-10T16:10:52.168992+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20598","reported_seq":29195,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.169145+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:10:52.169145+0000","last_peered":"2024-09-10T16:10:52.169145+0000","last_clean":"2024-09-10T16:10:52.169145+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:10:52.169145+0000","last_undegraded":"2024-09-10T16:10:52.169145+0000","last_fullsized":"2024-09-10T16:10:52.169145+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16988,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":126,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.167911+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:10:52.167911+0000","last_peered":"2024-09-10T16:10:52.167911+0000","last_clean":"2024-09-10T16:10:52.167911+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:10:52.167911+0000","last_undegraded":"2024-09-10T16:10:52.167911+0000","last_fullsized":"2024-09-10T16:10:52.167911+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'21048","reported_seq":24635,"reported_epoch":59,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:11:01.221760+0000","last_change":"2024-09-10T16:10:50.728566+0000","last_active":"2024-09-10T16:11:01.221760+0000","last_peered":"2024-09-10T16:11:01.221760+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:11:01.221760+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2667,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17303,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1706,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2667}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.169062+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:10:52.169062+0000","last_peered":"2024-09-10T16:10:52.169062+0000","last_clean":"2024-09-10T16:10:52.169062+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:10:52.169062+0000","last_undegraded":"2024-09-10T16:10:52.169062+0000","last_fullsized":"2024-09-10T16:10:52.169062+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":106,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.169114+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:10:52.169114+0000","last_peered":"2024-09-10T16:10:52.169114+0000","last_clean":"2024-09-10T16:10:52.169114+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:10:52.169114+0000","last_undegraded":"2024-09-10T16:10:52.169114+0000","last_fullsized":"2024-09-10T16:10:52.169114+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'21198","reported_seq":21374,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:10:52.169851+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:10:52.169851+0000","last_peered":"2024-09-10T16:10:52.169851+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:10:52.169851+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17361,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4321},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":15534,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292096,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35856,"num_bytes_recovered":33097129,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":205357056,"data_stored":205248666,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161022,"ondisk_log_size":161022,"up":32,"acting":28,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986332,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":498008,"kb_used_data":133128,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93873832,"statfs":{"total":96636764160,"available":96126803968,"internally_reserved":0,"allocated":136323072,"data_stored":135960704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659278,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":483420,"kb_used_data":524,"kb_used_omap":4,"kb_used_meta":482875,"kb_avail":93888420,"statfs":{"total":96636764160,"available":96141742080,"internally_reserved":0,"allocated":536576,"data_stored":170131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":494464686},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182560,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":547376,"kb_used_data":68700,"kb_used_omap":4,"kb_used_meta":478651,"kb_avail":93824464,"statfs":{"total":96636764160,"available":96076251136,"internally_reserved":0,"allocated":70348800,"data_stored":69982355,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":490139306},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":69853184,"data_stored":69817054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":4830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":135462912,"data_stored":135426782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:11:07.515 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:11:07.822 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:11:07.823 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:11:07.836 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":468,"stamp":"2024-09-10T16:11:07.790497+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":14831,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292098,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36561,"num_bytes_recovered":33097148,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161025,"ondisk_log_size":161025,"up":48,"acting":44,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1526404,"kb_used_data":199952,"kb_used_omap":12,"kb_used_meta":1326387,"kb_avail":281589116,"statfs":{"total":289910292480,"available":288347254784,"internally_reserved":0,"allocated":204750848,"data_stored":203655590,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1358220290},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2357,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2357,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002038"},"pg_stats":[{"pgid":"2.f","version":"49'20672","reported_seq":29304,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.169962+0000","last_change":"2024-09-10T16:09:57.088531+0000","last_active":"2024-09-10T16:10:52.169962+0000","last_peered":"2024-09-10T16:10:52.169962+0000","last_clean":"2024-09-10T16:10:52.169962+0000","last_became_active":"2024-09-10T16:09:57.005755+0000","last_became_peered":"2024-09-10T16:09:57.005755+0000","last_unstale":"2024-09-10T16:10:52.169962+0000","last_undegraded":"2024-09-10T16:10:52.169962+0000","last_fullsized":"2024-09-10T16:10:52.169962+0000","mapping_epoch":54,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:09:19.329697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20672,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4259,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20844","reported_seq":20988,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.168075+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:10:52.168075+0000","last_peered":"2024-09-10T16:10:52.168075+0000","last_clean":"2024-09-10T16:10:52.168075+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:10:52.168075+0000","last_undegraded":"2024-09-10T16:10:52.168075+0000","last_fullsized":"2024-09-10T16:10:52.168075+0000","mapping_epoch":35,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18949,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20798","reported_seq":29507,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.168961+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:10:52.168961+0000","last_peered":"2024-09-10T16:10:52.168961+0000","last_clean":"2024-09-10T16:10:52.168961+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:10:52.168961+0000","last_undegraded":"2024-09-10T16:10:52.168961+0000","last_fullsized":"2024-09-10T16:10:52.168961+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18802,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20660","reported_seq":20772,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.167700+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:10:52.167700+0000","last_peered":"2024-09-10T16:10:52.167700+0000","last_clean":"2024-09-10T16:10:52.167700+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:10:52.167700+0000","last_undegraded":"2024-09-10T16:10:52.167700+0000","last_fullsized":"2024-09-10T16:10:52.167700+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20660,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20891","reported_seq":21052,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.169809+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:10:52.169809+0000","last_peered":"2024-09-10T16:10:52.169809+0000","last_clean":"2024-09-10T16:10:52.169809+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:10:52.169809+0000","last_undegraded":"2024-09-10T16:10:52.169809+0000","last_fullsized":"2024-09-10T16:10:52.169809+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17602,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'21061","reported_seq":21222,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.242289+0000","last_change":"2024-09-10T16:10:52.241702+0000","last_active":"2024-09-10T16:10:52.242289+0000","last_peered":"2024-09-10T16:10:52.242289+0000","last_clean":"2024-09-10T16:10:52.242289+0000","last_became_active":"2024-09-10T16:10:52.171236+0000","last_became_peered":"2024-09-10T16:10:52.171236+0000","last_unstale":"2024-09-10T16:10:52.242289+0000","last_undegraded":"2024-09-10T16:10:52.242289+0000","last_fullsized":"2024-09-10T16:10:52.242289+0000","mapping_epoch":58,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:04:46.778280+0000","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17673,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4372,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'21082","reported_seq":21251,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:10:52.170050+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:10:52.170050+0000","last_peered":"2024-09-10T16:10:52.170050+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:10:52.170050+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17581,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4235},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'21314","reported_seq":21476,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.169765+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:10:52.169765+0000","last_peered":"2024-09-10T16:10:52.169765+0000","last_clean":"2024-09-10T16:10:52.169765+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:10:52.169765+0000","last_undegraded":"2024-09-10T16:10:52.169765+0000","last_fullsized":"2024-09-10T16:10:52.169765+0000","mapping_epoch":34,"log_start":"45'11300","ondisk_log_start":"45'11300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18161,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.168728+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:10:52.168728+0000","last_peered":"2024-09-10T16:10:52.168728+0000","last_clean":"2024-09-10T16:10:52.168728+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:10:52.168728+0000","last_undegraded":"2024-09-10T16:10:52.168728+0000","last_fullsized":"2024-09-10T16:10:52.168728+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20457","reported_seq":20632,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.168901+0000","last_change":"2024-09-10T16:09:33.012840+0000","last_active":"2024-09-10T16:10:52.168901+0000","last_peered":"2024-09-10T16:10:52.168901+0000","last_clean":"2024-09-10T16:10:52.168901+0000","last_became_active":"2024-09-10T16:09:32.934056+0000","last_became_peered":"2024-09-10T16:09:32.934056+0000","last_unstale":"2024-09-10T16:10:52.168901+0000","last_undegraded":"2024-09-10T16:10:52.168901+0000","last_fullsized":"2024-09-10T16:10:52.168901+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:08:30.481235+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18276,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":4579,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":126,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.168004+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:10:52.168004+0000","last_peered":"2024-09-10T16:10:52.168004+0000","last_clean":"2024-09-10T16:10:52.168004+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:10:52.168004+0000","last_undegraded":"2024-09-10T16:10:52.168004+0000","last_fullsized":"2024-09-10T16:10:52.168004+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20910","reported_seq":29594,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.169792+0000","last_change":"2024-09-10T16:09:08.909580+0000","last_active":"2024-09-10T16:10:52.169792+0000","last_peered":"2024-09-10T16:10:52.169792+0000","last_clean":"2024-09-10T16:10:52.169792+0000","last_became_active":"2024-09-10T16:09:08.854941+0000","last_became_peered":"2024-09-10T16:09:08.854941+0000","last_unstale":"2024-09-10T16:10:52.169792+0000","last_undegraded":"2024-09-10T16:10:52.169792+0000","last_fullsized":"2024-09-10T16:10:52.169792+0000","mapping_epoch":50,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:13:14.846691+0000","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17414,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4257,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":136,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.169714+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:10:52.169714+0000","last_peered":"2024-09-10T16:10:52.169714+0000","last_clean":"2024-09-10T16:10:52.169714+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:10:52.169714+0000","last_undegraded":"2024-09-10T16:10:52.169714+0000","last_fullsized":"2024-09-10T16:10:52.169714+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20781","reported_seq":29566,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.168064+0000","last_change":"2024-09-10T16:10:20.907889+0000","last_active":"2024-09-10T16:10:52.168064+0000","last_peered":"2024-09-10T16:10:52.168064+0000","last_clean":"2024-09-10T16:10:52.168064+0000","last_became_active":"2024-09-10T16:10:20.852065+0000","last_became_peered":"2024-09-10T16:10:20.852065+0000","last_unstale":"2024-09-10T16:10:52.168064+0000","last_undegraded":"2024-09-10T16:10:52.168064+0000","last_fullsized":"2024-09-10T16:10:52.168064+0000","mapping_epoch":56,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:34.069549+0000","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18461,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4307,"num_bytes_recovered":8675,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":111,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.168792+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:10:52.168792+0000","last_peered":"2024-09-10T16:10:52.168792+0000","last_clean":"2024-09-10T16:10:52.168792+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:10:52.168792+0000","last_undegraded":"2024-09-10T16:10:52.168792+0000","last_fullsized":"2024-09-10T16:10:52.168792+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'21170","reported_seq":30051,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.167373+0000","last_change":"2024-09-10T16:10:50.728349+0000","last_active":"2024-09-10T16:10:52.167373+0000","last_peered":"2024-09-10T16:10:52.167373+0000","last_clean":"2024-09-10T16:10:52.167373+0000","last_became_active":"2024-09-10T16:08:10.860438+0000","last_became_peered":"2024-09-10T16:08:10.860438+0000","last_unstale":"2024-09-10T16:10:52.167373+0000","last_undegraded":"2024-09-10T16:10:52.167373+0000","last_fullsized":"2024-09-10T16:10:52.167373+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:38:13.781380+0000","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18724,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20838","reported_seq":21004,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:10:52.169840+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:10:52.169840+0000","last_peered":"2024-09-10T16:10:52.169840+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:10:52.169840+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17469,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.168992+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:10:52.168992+0000","last_peered":"2024-09-10T16:10:52.168992+0000","last_clean":"2024-09-10T16:10:52.168992+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:10:52.168992+0000","last_undegraded":"2024-09-10T16:10:52.168992+0000","last_fullsized":"2024-09-10T16:10:52.168992+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20598","reported_seq":29195,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.169145+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:10:52.169145+0000","last_peered":"2024-09-10T16:10:52.169145+0000","last_clean":"2024-09-10T16:10:52.169145+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:10:52.169145+0000","last_undegraded":"2024-09-10T16:10:52.169145+0000","last_fullsized":"2024-09-10T16:10:52.169145+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16988,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":126,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.167911+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:10:52.167911+0000","last_peered":"2024-09-10T16:10:52.167911+0000","last_clean":"2024-09-10T16:10:52.167911+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:10:52.167911+0000","last_undegraded":"2024-09-10T16:10:52.167911+0000","last_fullsized":"2024-09-10T16:10:52.167911+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'21048","reported_seq":26041,"reported_epoch":59,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:11:06.224368+0000","last_change":"2024-09-10T16:10:50.728566+0000","last_active":"2024-09-10T16:11:06.224368+0000","last_peered":"2024-09-10T16:11:06.224368+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:11:06.224368+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1964,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17303,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2409,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1964}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.169062+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:10:52.169062+0000","last_peered":"2024-09-10T16:10:52.169062+0000","last_clean":"2024-09-10T16:10:52.169062+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:10:52.169062+0000","last_undegraded":"2024-09-10T16:10:52.169062+0000","last_fullsized":"2024-09-10T16:10:52.169062+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":106,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.169114+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:10:52.169114+0000","last_peered":"2024-09-10T16:10:52.169114+0000","last_clean":"2024-09-10T16:10:52.169114+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:10:52.169114+0000","last_undegraded":"2024-09-10T16:10:52.169114+0000","last_fullsized":"2024-09-10T16:10:52.169114+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'21198","reported_seq":21374,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:10:52.169851+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:10:52.169851+0000","last_peered":"2024-09-10T16:10:52.169851+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:10:52.169851+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17361,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4321},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":14831,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292096,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36559,"num_bytes_recovered":33097129,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":202899456,"data_stored":202791066,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161022,"ondisk_log_size":161022,"up":32,"acting":28,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986333,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":495608,"kb_used_data":130728,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93876232,"statfs":{"total":96636764160,"available":96129261568,"internally_reserved":0,"allocated":133865472,"data_stored":133503104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659278,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":483420,"kb_used_data":524,"kb_used_omap":4,"kb_used_meta":482875,"kb_avail":93888420,"statfs":{"total":96636764160,"available":96141742080,"internally_reserved":0,"allocated":536576,"data_stored":170131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":494464686},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182561,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":547376,"kb_used_data":68700,"kb_used_omap":4,"kb_used_meta":478651,"kb_avail":93824464,"statfs":{"total":96636764160,"available":96076251136,"internally_reserved":0,"allocated":70348800,"data_stored":69982355,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":490139306},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":69853184,"data_stored":69817054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":4830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":133005312,"data_stored":132969182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:11:07.837 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-10T16:11:07.837 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "49'21082", 'reported_seq': 21251, 'reported_epoch': 59, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:10:52.170050+0000', 'last_change': '2024-09-10T15:59:16.331899+0000', 'last_active': '2024-09-10T16:10:52.170050+0000', 'last_peered': '2024-09-10T16:10:52.170050+0000', 'last_clean': '2024-09-10T15:59:14.139436+0000', 'last_became_active': '2024-09-10T15:59:16.318715+0000', 'last_became_peered': '2024-09-10T15:59:16.318715+0000', 'last_unstale': '2024-09-10T16:10:52.170050+0000', 'last_undegraded': '2024-09-10T15:59:15.877524+0000', 'last_fullsized': '2024-09-10T15:59:15.769042+0000', 'mapping_epoch': 42, 'log_start': "45'11000", 'ondisk_log_start': "45'11000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3537", 'last_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'objects_scrubbed': 3497, 'log_size': 10082, 'log_dups_size': 0, 'ondisk_log_size': 10082, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1224, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4237, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 17581, 'num_write_kb': 28172, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4235}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:11:07.837 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-10T16:11:07.838 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "49'20838", 'reported_seq': 21004, 'reported_epoch': 59, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:10:52.169840+0000', 'last_change': '2024-09-10T15:59:16.315641+0000', 'last_active': '2024-09-10T16:10:52.169840+0000', 'last_peered': '2024-09-10T16:10:52.169840+0000', 'last_clean': '2024-09-10T15:59:14.117001+0000', 'last_became_active': '2024-09-10T15:59:16.302245+0000', 'last_became_peered': '2024-09-10T15:59:16.302245+0000', 'last_unstale': '2024-09-10T16:10:52.169840+0000', 'last_undegraded': '2024-09-10T15:59:15.845012+0000', 'last_fullsized': '2024-09-10T15:59:15.770337+0000', 'mapping_epoch': 42, 'log_start': "45'10800", 'ondisk_log_start': "45'10800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3404", 'last_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'objects_scrubbed': 3367, 'log_size': 10038, 'log_dups_size': 0, 'ondisk_log_size': 10038, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 983, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17469, 'num_write_kb': 28204, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4304}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:11:07.838 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-10T16:11:07.838 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "49'21048", 'reported_seq': 26041, 'reported_epoch': 59, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:11:06.224368+0000', 'last_change': '2024-09-10T16:10:50.728566+0000', 'last_active': '2024-09-10T16:11:06.224368+0000', 'last_peered': '2024-09-10T16:11:06.224368+0000', 'last_clean': '2024-09-10T15:59:14.116363+0000', 'last_became_active': '2024-09-10T15:59:16.301244+0000', 'last_became_peered': '2024-09-10T15:59:16.301244+0000', 'last_unstale': '2024-09-10T16:11:06.224368+0000', 'last_undegraded': '2024-09-10T15:59:15.883434+0000', 'last_fullsized': '2024-09-10T15:59:15.772756+0000', 'mapping_epoch': 42, 'log_start': "45'11000", 'ondisk_log_start': "45'11000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3789", 'last_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'objects_scrubbed': 3741, 'log_size': 10048, 'log_dups_size': 0, 'ondisk_log_size': 10048, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1440, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1964, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17303, 'num_write_kb': 27124, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2409, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1964}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:11:07.838 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-10T16:11:07.838 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "49'21198", 'reported_seq': 21374, 'reported_epoch': 59, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:10:52.169851+0000', 'last_change': '2024-09-10T15:59:16.313409+0000', 'last_active': '2024-09-10T16:10:52.169851+0000', 'last_peered': '2024-09-10T16:10:52.169851+0000', 'last_clean': '2024-09-10T15:59:14.129780+0000', 'last_became_active': '2024-09-10T15:59:16.300932+0000', 'last_became_peered': '2024-09-10T15:59:16.300932+0000', 'last_unstale': '2024-09-10T16:10:52.169851+0000', 'last_undegraded': '2024-09-10T15:59:15.887823+0000', 'last_fullsized': '2024-09-10T15:59:15.768513+0000', 'mapping_epoch': 42, 'log_start': "45'11100", 'ondisk_log_start': "45'11100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3880", 'last_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'objects_scrubbed': 3827, 'log_size': 10098, 'log_dups_size': 0, 'ondisk_log_size': 10098, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1397, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4325, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17361, 'num_write_kb': 27072, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4321}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:11:07.838 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T16:11:08.218 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:11:08.218 INFO:teuthology.orchestra.run.smithi115.stdout:{"fsid":"7effee29-df05-4351-9741-ee203564ed81","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 14831/24 objects degraded (61795.833%), 4 pgs degraded, 4 pgs undersized","count":8},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":832,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":59,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983954,"num_in_osds":3,"osd_in_since":1725983947,"num_remapped_pgs":4},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":20},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":3},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":12,"data_bytes":4849,"bytes_used":1563037696,"bytes_avail":288347254784,"bytes_total":289910292480,"degraded_objects":14831,"degraded_total":24,"degraded_ratio":617.95833333333337,"recovering_objects_per_sec":196,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":2357,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-10T15:57:14:967120+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T16:00:13.560776+0000","services":{}},"progress_events":{}} 2024-09-10T16:11:08.232 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-10T16:11:08.233 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:11:08.536 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:11:08.537 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:11:08.549 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":468,"stamp":"2024-09-10T16:11:07.790497+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":14831,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292098,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36561,"num_bytes_recovered":33097148,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161025,"ondisk_log_size":161025,"up":48,"acting":44,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1526404,"kb_used_data":199952,"kb_used_omap":12,"kb_used_meta":1326387,"kb_avail":281589116,"statfs":{"total":289910292480,"available":288347254784,"internally_reserved":0,"allocated":204750848,"data_stored":203655590,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1358220290},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2357,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2357,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002038"},"pg_stats":[{"pgid":"2.f","version":"49'20672","reported_seq":29304,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.169962+0000","last_change":"2024-09-10T16:09:57.088531+0000","last_active":"2024-09-10T16:10:52.169962+0000","last_peered":"2024-09-10T16:10:52.169962+0000","last_clean":"2024-09-10T16:10:52.169962+0000","last_became_active":"2024-09-10T16:09:57.005755+0000","last_became_peered":"2024-09-10T16:09:57.005755+0000","last_unstale":"2024-09-10T16:10:52.169962+0000","last_undegraded":"2024-09-10T16:10:52.169962+0000","last_fullsized":"2024-09-10T16:10:52.169962+0000","mapping_epoch":54,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:09:19.329697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20672,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4259,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20844","reported_seq":20988,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.168075+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:10:52.168075+0000","last_peered":"2024-09-10T16:10:52.168075+0000","last_clean":"2024-09-10T16:10:52.168075+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:10:52.168075+0000","last_undegraded":"2024-09-10T16:10:52.168075+0000","last_fullsized":"2024-09-10T16:10:52.168075+0000","mapping_epoch":35,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18949,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20798","reported_seq":29507,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.168961+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:10:52.168961+0000","last_peered":"2024-09-10T16:10:52.168961+0000","last_clean":"2024-09-10T16:10:52.168961+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:10:52.168961+0000","last_undegraded":"2024-09-10T16:10:52.168961+0000","last_fullsized":"2024-09-10T16:10:52.168961+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18802,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20660","reported_seq":20772,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.167700+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:10:52.167700+0000","last_peered":"2024-09-10T16:10:52.167700+0000","last_clean":"2024-09-10T16:10:52.167700+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:10:52.167700+0000","last_undegraded":"2024-09-10T16:10:52.167700+0000","last_fullsized":"2024-09-10T16:10:52.167700+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20660,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20891","reported_seq":21052,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.169809+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:10:52.169809+0000","last_peered":"2024-09-10T16:10:52.169809+0000","last_clean":"2024-09-10T16:10:52.169809+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:10:52.169809+0000","last_undegraded":"2024-09-10T16:10:52.169809+0000","last_fullsized":"2024-09-10T16:10:52.169809+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17602,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'21061","reported_seq":21222,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.242289+0000","last_change":"2024-09-10T16:10:52.241702+0000","last_active":"2024-09-10T16:10:52.242289+0000","last_peered":"2024-09-10T16:10:52.242289+0000","last_clean":"2024-09-10T16:10:52.242289+0000","last_became_active":"2024-09-10T16:10:52.171236+0000","last_became_peered":"2024-09-10T16:10:52.171236+0000","last_unstale":"2024-09-10T16:10:52.242289+0000","last_undegraded":"2024-09-10T16:10:52.242289+0000","last_fullsized":"2024-09-10T16:10:52.242289+0000","mapping_epoch":58,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:04:46.778280+0000","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17673,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4372,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'21082","reported_seq":21251,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:10:52.170050+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:10:52.170050+0000","last_peered":"2024-09-10T16:10:52.170050+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:10:52.170050+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17581,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4235},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'21314","reported_seq":21476,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.169765+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:10:52.169765+0000","last_peered":"2024-09-10T16:10:52.169765+0000","last_clean":"2024-09-10T16:10:52.169765+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:10:52.169765+0000","last_undegraded":"2024-09-10T16:10:52.169765+0000","last_fullsized":"2024-09-10T16:10:52.169765+0000","mapping_epoch":34,"log_start":"45'11300","ondisk_log_start":"45'11300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18161,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.168728+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:10:52.168728+0000","last_peered":"2024-09-10T16:10:52.168728+0000","last_clean":"2024-09-10T16:10:52.168728+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:10:52.168728+0000","last_undegraded":"2024-09-10T16:10:52.168728+0000","last_fullsized":"2024-09-10T16:10:52.168728+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20457","reported_seq":20632,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.168901+0000","last_change":"2024-09-10T16:09:33.012840+0000","last_active":"2024-09-10T16:10:52.168901+0000","last_peered":"2024-09-10T16:10:52.168901+0000","last_clean":"2024-09-10T16:10:52.168901+0000","last_became_active":"2024-09-10T16:09:32.934056+0000","last_became_peered":"2024-09-10T16:09:32.934056+0000","last_unstale":"2024-09-10T16:10:52.168901+0000","last_undegraded":"2024-09-10T16:10:52.168901+0000","last_fullsized":"2024-09-10T16:10:52.168901+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:08:30.481235+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18276,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":4579,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":126,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.168004+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:10:52.168004+0000","last_peered":"2024-09-10T16:10:52.168004+0000","last_clean":"2024-09-10T16:10:52.168004+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:10:52.168004+0000","last_undegraded":"2024-09-10T16:10:52.168004+0000","last_fullsized":"2024-09-10T16:10:52.168004+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20910","reported_seq":29594,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.169792+0000","last_change":"2024-09-10T16:09:08.909580+0000","last_active":"2024-09-10T16:10:52.169792+0000","last_peered":"2024-09-10T16:10:52.169792+0000","last_clean":"2024-09-10T16:10:52.169792+0000","last_became_active":"2024-09-10T16:09:08.854941+0000","last_became_peered":"2024-09-10T16:09:08.854941+0000","last_unstale":"2024-09-10T16:10:52.169792+0000","last_undegraded":"2024-09-10T16:10:52.169792+0000","last_fullsized":"2024-09-10T16:10:52.169792+0000","mapping_epoch":50,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:13:14.846691+0000","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17414,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4257,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":136,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.169714+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:10:52.169714+0000","last_peered":"2024-09-10T16:10:52.169714+0000","last_clean":"2024-09-10T16:10:52.169714+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:10:52.169714+0000","last_undegraded":"2024-09-10T16:10:52.169714+0000","last_fullsized":"2024-09-10T16:10:52.169714+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20781","reported_seq":29566,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.168064+0000","last_change":"2024-09-10T16:10:20.907889+0000","last_active":"2024-09-10T16:10:52.168064+0000","last_peered":"2024-09-10T16:10:52.168064+0000","last_clean":"2024-09-10T16:10:52.168064+0000","last_became_active":"2024-09-10T16:10:20.852065+0000","last_became_peered":"2024-09-10T16:10:20.852065+0000","last_unstale":"2024-09-10T16:10:52.168064+0000","last_undegraded":"2024-09-10T16:10:52.168064+0000","last_fullsized":"2024-09-10T16:10:52.168064+0000","mapping_epoch":56,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:34.069549+0000","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18461,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4307,"num_bytes_recovered":8675,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":111,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.168792+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:10:52.168792+0000","last_peered":"2024-09-10T16:10:52.168792+0000","last_clean":"2024-09-10T16:10:52.168792+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:10:52.168792+0000","last_undegraded":"2024-09-10T16:10:52.168792+0000","last_fullsized":"2024-09-10T16:10:52.168792+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'21170","reported_seq":30051,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.167373+0000","last_change":"2024-09-10T16:10:50.728349+0000","last_active":"2024-09-10T16:10:52.167373+0000","last_peered":"2024-09-10T16:10:52.167373+0000","last_clean":"2024-09-10T16:10:52.167373+0000","last_became_active":"2024-09-10T16:08:10.860438+0000","last_became_peered":"2024-09-10T16:08:10.860438+0000","last_unstale":"2024-09-10T16:10:52.167373+0000","last_undegraded":"2024-09-10T16:10:52.167373+0000","last_fullsized":"2024-09-10T16:10:52.167373+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:38:13.781380+0000","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18724,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20838","reported_seq":21004,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:10:52.169840+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:10:52.169840+0000","last_peered":"2024-09-10T16:10:52.169840+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:10:52.169840+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17469,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.168992+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:10:52.168992+0000","last_peered":"2024-09-10T16:10:52.168992+0000","last_clean":"2024-09-10T16:10:52.168992+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:10:52.168992+0000","last_undegraded":"2024-09-10T16:10:52.168992+0000","last_fullsized":"2024-09-10T16:10:52.168992+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20598","reported_seq":29195,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.169145+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:10:52.169145+0000","last_peered":"2024-09-10T16:10:52.169145+0000","last_clean":"2024-09-10T16:10:52.169145+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:10:52.169145+0000","last_undegraded":"2024-09-10T16:10:52.169145+0000","last_fullsized":"2024-09-10T16:10:52.169145+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16988,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":126,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.167911+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:10:52.167911+0000","last_peered":"2024-09-10T16:10:52.167911+0000","last_clean":"2024-09-10T16:10:52.167911+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:10:52.167911+0000","last_undegraded":"2024-09-10T16:10:52.167911+0000","last_fullsized":"2024-09-10T16:10:52.167911+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'21048","reported_seq":26041,"reported_epoch":59,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:11:06.224368+0000","last_change":"2024-09-10T16:10:50.728566+0000","last_active":"2024-09-10T16:11:06.224368+0000","last_peered":"2024-09-10T16:11:06.224368+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:11:06.224368+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1964,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17303,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2409,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1964}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.169062+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:10:52.169062+0000","last_peered":"2024-09-10T16:10:52.169062+0000","last_clean":"2024-09-10T16:10:52.169062+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:10:52.169062+0000","last_undegraded":"2024-09-10T16:10:52.169062+0000","last_fullsized":"2024-09-10T16:10:52.169062+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":106,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.169114+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:10:52.169114+0000","last_peered":"2024-09-10T16:10:52.169114+0000","last_clean":"2024-09-10T16:10:52.169114+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:10:52.169114+0000","last_undegraded":"2024-09-10T16:10:52.169114+0000","last_fullsized":"2024-09-10T16:10:52.169114+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'21198","reported_seq":21374,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:10:52.169851+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:10:52.169851+0000","last_peered":"2024-09-10T16:10:52.169851+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:10:52.169851+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17361,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4321},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":14831,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292096,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36559,"num_bytes_recovered":33097129,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":202899456,"data_stored":202791066,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161022,"ondisk_log_size":161022,"up":32,"acting":28,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986333,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":495608,"kb_used_data":130728,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93876232,"statfs":{"total":96636764160,"available":96129261568,"internally_reserved":0,"allocated":133865472,"data_stored":133503104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659278,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":483420,"kb_used_data":524,"kb_used_omap":4,"kb_used_meta":482875,"kb_avail":93888420,"statfs":{"total":96636764160,"available":96141742080,"internally_reserved":0,"allocated":536576,"data_stored":170131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":494464686},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182561,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":547376,"kb_used_data":68700,"kb_used_omap":4,"kb_used_meta":478651,"kb_avail":93824464,"statfs":{"total":96636764160,"available":96076251136,"internally_reserved":0,"allocated":70348800,"data_stored":69982355,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":490139306},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":69853184,"data_stored":69817054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":4830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":133005312,"data_stored":132969182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:11:11.551 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:11:11.859 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:11:11.859 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:11:11.872 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":470,"stamp":"2024-09-10T16:11:11.791123+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":13957,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292098,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37435,"num_bytes_recovered":33097148,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161025,"ondisk_log_size":161025,"up":48,"acting":44,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1532324,"kb_used_data":197552,"kb_used_omap":12,"kb_used_meta":1334707,"kb_avail":281583196,"statfs":{"total":289910292480,"available":288341192704,"internally_reserved":0,"allocated":202293248,"data_stored":201197990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1366739970},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2369,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2369,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001953"},"pg_stats":[{"pgid":"2.f","version":"49'20672","reported_seq":29304,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.169962+0000","last_change":"2024-09-10T16:09:57.088531+0000","last_active":"2024-09-10T16:10:52.169962+0000","last_peered":"2024-09-10T16:10:52.169962+0000","last_clean":"2024-09-10T16:10:52.169962+0000","last_became_active":"2024-09-10T16:09:57.005755+0000","last_became_peered":"2024-09-10T16:09:57.005755+0000","last_unstale":"2024-09-10T16:10:52.169962+0000","last_undegraded":"2024-09-10T16:10:52.169962+0000","last_fullsized":"2024-09-10T16:10:52.169962+0000","mapping_epoch":54,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:09:19.329697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20672,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4259,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20844","reported_seq":20988,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.168075+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:10:52.168075+0000","last_peered":"2024-09-10T16:10:52.168075+0000","last_clean":"2024-09-10T16:10:52.168075+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:10:52.168075+0000","last_undegraded":"2024-09-10T16:10:52.168075+0000","last_fullsized":"2024-09-10T16:10:52.168075+0000","mapping_epoch":35,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18949,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20798","reported_seq":29507,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.168961+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:10:52.168961+0000","last_peered":"2024-09-10T16:10:52.168961+0000","last_clean":"2024-09-10T16:10:52.168961+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:10:52.168961+0000","last_undegraded":"2024-09-10T16:10:52.168961+0000","last_fullsized":"2024-09-10T16:10:52.168961+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18802,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20660","reported_seq":20772,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.167700+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:10:52.167700+0000","last_peered":"2024-09-10T16:10:52.167700+0000","last_clean":"2024-09-10T16:10:52.167700+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:10:52.167700+0000","last_undegraded":"2024-09-10T16:10:52.167700+0000","last_fullsized":"2024-09-10T16:10:52.167700+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20660,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20891","reported_seq":21052,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.169809+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:10:52.169809+0000","last_peered":"2024-09-10T16:10:52.169809+0000","last_clean":"2024-09-10T16:10:52.169809+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:10:52.169809+0000","last_undegraded":"2024-09-10T16:10:52.169809+0000","last_fullsized":"2024-09-10T16:10:52.169809+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17602,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'21061","reported_seq":21222,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.242289+0000","last_change":"2024-09-10T16:10:52.241702+0000","last_active":"2024-09-10T16:10:52.242289+0000","last_peered":"2024-09-10T16:10:52.242289+0000","last_clean":"2024-09-10T16:10:52.242289+0000","last_became_active":"2024-09-10T16:10:52.171236+0000","last_became_peered":"2024-09-10T16:10:52.171236+0000","last_unstale":"2024-09-10T16:10:52.242289+0000","last_undegraded":"2024-09-10T16:10:52.242289+0000","last_fullsized":"2024-09-10T16:10:52.242289+0000","mapping_epoch":58,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:04:46.778280+0000","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17673,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4372,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'21082","reported_seq":21251,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:10:52.170050+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:10:52.170050+0000","last_peered":"2024-09-10T16:10:52.170050+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:10:52.170050+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17581,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4235},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'21314","reported_seq":21476,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.169765+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:10:52.169765+0000","last_peered":"2024-09-10T16:10:52.169765+0000","last_clean":"2024-09-10T16:10:52.169765+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:10:52.169765+0000","last_undegraded":"2024-09-10T16:10:52.169765+0000","last_fullsized":"2024-09-10T16:10:52.169765+0000","mapping_epoch":34,"log_start":"45'11300","ondisk_log_start":"45'11300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18161,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.168728+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:10:52.168728+0000","last_peered":"2024-09-10T16:10:52.168728+0000","last_clean":"2024-09-10T16:10:52.168728+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:10:52.168728+0000","last_undegraded":"2024-09-10T16:10:52.168728+0000","last_fullsized":"2024-09-10T16:10:52.168728+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20457","reported_seq":20632,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.168901+0000","last_change":"2024-09-10T16:09:33.012840+0000","last_active":"2024-09-10T16:10:52.168901+0000","last_peered":"2024-09-10T16:10:52.168901+0000","last_clean":"2024-09-10T16:10:52.168901+0000","last_became_active":"2024-09-10T16:09:32.934056+0000","last_became_peered":"2024-09-10T16:09:32.934056+0000","last_unstale":"2024-09-10T16:10:52.168901+0000","last_undegraded":"2024-09-10T16:10:52.168901+0000","last_fullsized":"2024-09-10T16:10:52.168901+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:08:30.481235+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18276,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":4579,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":126,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.168004+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:10:52.168004+0000","last_peered":"2024-09-10T16:10:52.168004+0000","last_clean":"2024-09-10T16:10:52.168004+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:10:52.168004+0000","last_undegraded":"2024-09-10T16:10:52.168004+0000","last_fullsized":"2024-09-10T16:10:52.168004+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20910","reported_seq":29594,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.169792+0000","last_change":"2024-09-10T16:09:08.909580+0000","last_active":"2024-09-10T16:10:52.169792+0000","last_peered":"2024-09-10T16:10:52.169792+0000","last_clean":"2024-09-10T16:10:52.169792+0000","last_became_active":"2024-09-10T16:09:08.854941+0000","last_became_peered":"2024-09-10T16:09:08.854941+0000","last_unstale":"2024-09-10T16:10:52.169792+0000","last_undegraded":"2024-09-10T16:10:52.169792+0000","last_fullsized":"2024-09-10T16:10:52.169792+0000","mapping_epoch":50,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:13:14.846691+0000","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17414,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4257,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":136,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.169714+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:10:52.169714+0000","last_peered":"2024-09-10T16:10:52.169714+0000","last_clean":"2024-09-10T16:10:52.169714+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:10:52.169714+0000","last_undegraded":"2024-09-10T16:10:52.169714+0000","last_fullsized":"2024-09-10T16:10:52.169714+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20781","reported_seq":29566,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.168064+0000","last_change":"2024-09-10T16:10:20.907889+0000","last_active":"2024-09-10T16:10:52.168064+0000","last_peered":"2024-09-10T16:10:52.168064+0000","last_clean":"2024-09-10T16:10:52.168064+0000","last_became_active":"2024-09-10T16:10:20.852065+0000","last_became_peered":"2024-09-10T16:10:20.852065+0000","last_unstale":"2024-09-10T16:10:52.168064+0000","last_undegraded":"2024-09-10T16:10:52.168064+0000","last_fullsized":"2024-09-10T16:10:52.168064+0000","mapping_epoch":56,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:34.069549+0000","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18461,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4307,"num_bytes_recovered":8675,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":111,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.168792+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:10:52.168792+0000","last_peered":"2024-09-10T16:10:52.168792+0000","last_clean":"2024-09-10T16:10:52.168792+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:10:52.168792+0000","last_undegraded":"2024-09-10T16:10:52.168792+0000","last_fullsized":"2024-09-10T16:10:52.168792+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'21170","reported_seq":30051,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.167373+0000","last_change":"2024-09-10T16:10:50.728349+0000","last_active":"2024-09-10T16:10:52.167373+0000","last_peered":"2024-09-10T16:10:52.167373+0000","last_clean":"2024-09-10T16:10:52.167373+0000","last_became_active":"2024-09-10T16:08:10.860438+0000","last_became_peered":"2024-09-10T16:08:10.860438+0000","last_unstale":"2024-09-10T16:10:52.167373+0000","last_undegraded":"2024-09-10T16:10:52.167373+0000","last_fullsized":"2024-09-10T16:10:52.167373+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:38:13.781380+0000","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18724,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20838","reported_seq":21004,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:10:52.169840+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:10:52.169840+0000","last_peered":"2024-09-10T16:10:52.169840+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:10:52.169840+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17469,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.168992+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:10:52.168992+0000","last_peered":"2024-09-10T16:10:52.168992+0000","last_clean":"2024-09-10T16:10:52.168992+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:10:52.168992+0000","last_undegraded":"2024-09-10T16:10:52.168992+0000","last_fullsized":"2024-09-10T16:10:52.168992+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20598","reported_seq":29195,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.169145+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:10:52.169145+0000","last_peered":"2024-09-10T16:10:52.169145+0000","last_clean":"2024-09-10T16:10:52.169145+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:10:52.169145+0000","last_undegraded":"2024-09-10T16:10:52.169145+0000","last_fullsized":"2024-09-10T16:10:52.169145+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16988,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":126,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.167911+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:10:52.167911+0000","last_peered":"2024-09-10T16:10:52.167911+0000","last_clean":"2024-09-10T16:10:52.167911+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:10:52.167911+0000","last_undegraded":"2024-09-10T16:10:52.167911+0000","last_fullsized":"2024-09-10T16:10:52.167911+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'21048","reported_seq":27789,"reported_epoch":59,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:11:11.223560+0000","last_change":"2024-09-10T16:10:50.728566+0000","last_active":"2024-09-10T16:11:11.223560+0000","last_peered":"2024-09-10T16:11:11.223560+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:11:11.223560+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1090,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17303,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3283,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1090}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.169062+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:10:52.169062+0000","last_peered":"2024-09-10T16:10:52.169062+0000","last_clean":"2024-09-10T16:10:52.169062+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:10:52.169062+0000","last_undegraded":"2024-09-10T16:10:52.169062+0000","last_fullsized":"2024-09-10T16:10:52.169062+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":106,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.169114+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:10:52.169114+0000","last_peered":"2024-09-10T16:10:52.169114+0000","last_clean":"2024-09-10T16:10:52.169114+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:10:52.169114+0000","last_undegraded":"2024-09-10T16:10:52.169114+0000","last_fullsized":"2024-09-10T16:10:52.169114+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'21198","reported_seq":21374,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:10:52.169851+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:10:52.169851+0000","last_peered":"2024-09-10T16:10:52.169851+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:10:52.169851+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17361,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4321},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":13957,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292096,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37433,"num_bytes_recovered":33097129,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":200441856,"data_stored":200333466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161022,"ondisk_log_size":161022,"up":32,"acting":28,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986334,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":493208,"kb_used_data":128328,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93878632,"statfs":{"total":96636764160,"available":96131719168,"internally_reserved":0,"allocated":131407872,"data_stored":131045504,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659279,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":491740,"kb_used_data":524,"kb_used_omap":4,"kb_used_meta":491195,"kb_avail":93880100,"statfs":{"total":96636764160,"available":96133222400,"internally_reserved":0,"allocated":536576,"data_stored":170131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":502984366},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182562,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":547376,"kb_used_data":68700,"kb_used_omap":4,"kb_used_meta":478651,"kb_avail":93824464,"statfs":{"total":96636764160,"available":96076251136,"internally_reserved":0,"allocated":70348800,"data_stored":69982355,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":490139306},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":69853184,"data_stored":69817054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":4830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":130547712,"data_stored":130511582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:11:11.874 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:11:12.176 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:11:12.176 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:11:12.190 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":470,"stamp":"2024-09-10T16:11:11.791123+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":13957,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292098,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37435,"num_bytes_recovered":33097148,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161025,"ondisk_log_size":161025,"up":48,"acting":44,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1532324,"kb_used_data":197552,"kb_used_omap":12,"kb_used_meta":1334707,"kb_avail":281583196,"statfs":{"total":289910292480,"available":288341192704,"internally_reserved":0,"allocated":202293248,"data_stored":201197990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1366739970},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2369,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2369,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001953"},"pg_stats":[{"pgid":"2.f","version":"49'20672","reported_seq":29304,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.169962+0000","last_change":"2024-09-10T16:09:57.088531+0000","last_active":"2024-09-10T16:10:52.169962+0000","last_peered":"2024-09-10T16:10:52.169962+0000","last_clean":"2024-09-10T16:10:52.169962+0000","last_became_active":"2024-09-10T16:09:57.005755+0000","last_became_peered":"2024-09-10T16:09:57.005755+0000","last_unstale":"2024-09-10T16:10:52.169962+0000","last_undegraded":"2024-09-10T16:10:52.169962+0000","last_fullsized":"2024-09-10T16:10:52.169962+0000","mapping_epoch":54,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:09:19.329697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20672,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4259,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20844","reported_seq":20988,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.168075+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:10:52.168075+0000","last_peered":"2024-09-10T16:10:52.168075+0000","last_clean":"2024-09-10T16:10:52.168075+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:10:52.168075+0000","last_undegraded":"2024-09-10T16:10:52.168075+0000","last_fullsized":"2024-09-10T16:10:52.168075+0000","mapping_epoch":35,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18949,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20798","reported_seq":29507,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.168961+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:10:52.168961+0000","last_peered":"2024-09-10T16:10:52.168961+0000","last_clean":"2024-09-10T16:10:52.168961+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:10:52.168961+0000","last_undegraded":"2024-09-10T16:10:52.168961+0000","last_fullsized":"2024-09-10T16:10:52.168961+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18802,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20660","reported_seq":20772,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.167700+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:10:52.167700+0000","last_peered":"2024-09-10T16:10:52.167700+0000","last_clean":"2024-09-10T16:10:52.167700+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:10:52.167700+0000","last_undegraded":"2024-09-10T16:10:52.167700+0000","last_fullsized":"2024-09-10T16:10:52.167700+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20660,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20891","reported_seq":21052,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.169809+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:10:52.169809+0000","last_peered":"2024-09-10T16:10:52.169809+0000","last_clean":"2024-09-10T16:10:52.169809+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:10:52.169809+0000","last_undegraded":"2024-09-10T16:10:52.169809+0000","last_fullsized":"2024-09-10T16:10:52.169809+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17602,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'21061","reported_seq":21222,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.242289+0000","last_change":"2024-09-10T16:10:52.241702+0000","last_active":"2024-09-10T16:10:52.242289+0000","last_peered":"2024-09-10T16:10:52.242289+0000","last_clean":"2024-09-10T16:10:52.242289+0000","last_became_active":"2024-09-10T16:10:52.171236+0000","last_became_peered":"2024-09-10T16:10:52.171236+0000","last_unstale":"2024-09-10T16:10:52.242289+0000","last_undegraded":"2024-09-10T16:10:52.242289+0000","last_fullsized":"2024-09-10T16:10:52.242289+0000","mapping_epoch":58,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:04:46.778280+0000","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17673,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4372,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'21082","reported_seq":21251,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:10:52.170050+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:10:52.170050+0000","last_peered":"2024-09-10T16:10:52.170050+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:10:52.170050+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17581,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4235},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'21314","reported_seq":21476,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.169765+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:10:52.169765+0000","last_peered":"2024-09-10T16:10:52.169765+0000","last_clean":"2024-09-10T16:10:52.169765+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:10:52.169765+0000","last_undegraded":"2024-09-10T16:10:52.169765+0000","last_fullsized":"2024-09-10T16:10:52.169765+0000","mapping_epoch":34,"log_start":"45'11300","ondisk_log_start":"45'11300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18161,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.168728+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:10:52.168728+0000","last_peered":"2024-09-10T16:10:52.168728+0000","last_clean":"2024-09-10T16:10:52.168728+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:10:52.168728+0000","last_undegraded":"2024-09-10T16:10:52.168728+0000","last_fullsized":"2024-09-10T16:10:52.168728+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20457","reported_seq":20632,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.168901+0000","last_change":"2024-09-10T16:09:33.012840+0000","last_active":"2024-09-10T16:10:52.168901+0000","last_peered":"2024-09-10T16:10:52.168901+0000","last_clean":"2024-09-10T16:10:52.168901+0000","last_became_active":"2024-09-10T16:09:32.934056+0000","last_became_peered":"2024-09-10T16:09:32.934056+0000","last_unstale":"2024-09-10T16:10:52.168901+0000","last_undegraded":"2024-09-10T16:10:52.168901+0000","last_fullsized":"2024-09-10T16:10:52.168901+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:08:30.481235+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18276,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":4579,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":126,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.168004+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:10:52.168004+0000","last_peered":"2024-09-10T16:10:52.168004+0000","last_clean":"2024-09-10T16:10:52.168004+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:10:52.168004+0000","last_undegraded":"2024-09-10T16:10:52.168004+0000","last_fullsized":"2024-09-10T16:10:52.168004+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20910","reported_seq":29594,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.169792+0000","last_change":"2024-09-10T16:09:08.909580+0000","last_active":"2024-09-10T16:10:52.169792+0000","last_peered":"2024-09-10T16:10:52.169792+0000","last_clean":"2024-09-10T16:10:52.169792+0000","last_became_active":"2024-09-10T16:09:08.854941+0000","last_became_peered":"2024-09-10T16:09:08.854941+0000","last_unstale":"2024-09-10T16:10:52.169792+0000","last_undegraded":"2024-09-10T16:10:52.169792+0000","last_fullsized":"2024-09-10T16:10:52.169792+0000","mapping_epoch":50,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:13:14.846691+0000","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17414,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4257,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":136,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.169714+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:10:52.169714+0000","last_peered":"2024-09-10T16:10:52.169714+0000","last_clean":"2024-09-10T16:10:52.169714+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:10:52.169714+0000","last_undegraded":"2024-09-10T16:10:52.169714+0000","last_fullsized":"2024-09-10T16:10:52.169714+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20781","reported_seq":29566,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.168064+0000","last_change":"2024-09-10T16:10:20.907889+0000","last_active":"2024-09-10T16:10:52.168064+0000","last_peered":"2024-09-10T16:10:52.168064+0000","last_clean":"2024-09-10T16:10:52.168064+0000","last_became_active":"2024-09-10T16:10:20.852065+0000","last_became_peered":"2024-09-10T16:10:20.852065+0000","last_unstale":"2024-09-10T16:10:52.168064+0000","last_undegraded":"2024-09-10T16:10:52.168064+0000","last_fullsized":"2024-09-10T16:10:52.168064+0000","mapping_epoch":56,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:34.069549+0000","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18461,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4307,"num_bytes_recovered":8675,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":111,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.168792+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:10:52.168792+0000","last_peered":"2024-09-10T16:10:52.168792+0000","last_clean":"2024-09-10T16:10:52.168792+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:10:52.168792+0000","last_undegraded":"2024-09-10T16:10:52.168792+0000","last_fullsized":"2024-09-10T16:10:52.168792+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'21170","reported_seq":30051,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.167373+0000","last_change":"2024-09-10T16:10:50.728349+0000","last_active":"2024-09-10T16:10:52.167373+0000","last_peered":"2024-09-10T16:10:52.167373+0000","last_clean":"2024-09-10T16:10:52.167373+0000","last_became_active":"2024-09-10T16:08:10.860438+0000","last_became_peered":"2024-09-10T16:08:10.860438+0000","last_unstale":"2024-09-10T16:10:52.167373+0000","last_undegraded":"2024-09-10T16:10:52.167373+0000","last_fullsized":"2024-09-10T16:10:52.167373+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:38:13.781380+0000","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18724,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20838","reported_seq":21004,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:10:52.169840+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:10:52.169840+0000","last_peered":"2024-09-10T16:10:52.169840+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:10:52.169840+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17469,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.168992+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:10:52.168992+0000","last_peered":"2024-09-10T16:10:52.168992+0000","last_clean":"2024-09-10T16:10:52.168992+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:10:52.168992+0000","last_undegraded":"2024-09-10T16:10:52.168992+0000","last_fullsized":"2024-09-10T16:10:52.168992+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20598","reported_seq":29195,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.169145+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:10:52.169145+0000","last_peered":"2024-09-10T16:10:52.169145+0000","last_clean":"2024-09-10T16:10:52.169145+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:10:52.169145+0000","last_undegraded":"2024-09-10T16:10:52.169145+0000","last_fullsized":"2024-09-10T16:10:52.169145+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16988,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":126,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.167911+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:10:52.167911+0000","last_peered":"2024-09-10T16:10:52.167911+0000","last_clean":"2024-09-10T16:10:52.167911+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:10:52.167911+0000","last_undegraded":"2024-09-10T16:10:52.167911+0000","last_fullsized":"2024-09-10T16:10:52.167911+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'21048","reported_seq":27789,"reported_epoch":59,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:11:11.223560+0000","last_change":"2024-09-10T16:10:50.728566+0000","last_active":"2024-09-10T16:11:11.223560+0000","last_peered":"2024-09-10T16:11:11.223560+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:11:11.223560+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1090,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17303,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3283,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1090}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.169062+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:10:52.169062+0000","last_peered":"2024-09-10T16:10:52.169062+0000","last_clean":"2024-09-10T16:10:52.169062+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:10:52.169062+0000","last_undegraded":"2024-09-10T16:10:52.169062+0000","last_fullsized":"2024-09-10T16:10:52.169062+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":106,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.169114+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:10:52.169114+0000","last_peered":"2024-09-10T16:10:52.169114+0000","last_clean":"2024-09-10T16:10:52.169114+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:10:52.169114+0000","last_undegraded":"2024-09-10T16:10:52.169114+0000","last_fullsized":"2024-09-10T16:10:52.169114+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'21198","reported_seq":21374,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:10:52.169851+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:10:52.169851+0000","last_peered":"2024-09-10T16:10:52.169851+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:10:52.169851+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17361,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4321},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":13957,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292096,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37433,"num_bytes_recovered":33097129,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":200441856,"data_stored":200333466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161022,"ondisk_log_size":161022,"up":32,"acting":28,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986334,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":493208,"kb_used_data":128328,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93878632,"statfs":{"total":96636764160,"available":96131719168,"internally_reserved":0,"allocated":131407872,"data_stored":131045504,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659279,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":491740,"kb_used_data":524,"kb_used_omap":4,"kb_used_meta":491195,"kb_avail":93880100,"statfs":{"total":96636764160,"available":96133222400,"internally_reserved":0,"allocated":536576,"data_stored":170131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":502984366},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182562,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":547376,"kb_used_data":68700,"kb_used_omap":4,"kb_used_meta":478651,"kb_avail":93824464,"statfs":{"total":96636764160,"available":96076251136,"internally_reserved":0,"allocated":70348800,"data_stored":69982355,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":490139306},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":69853184,"data_stored":69817054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":4830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":130547712,"data_stored":130511582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:11:12.192 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-10T16:11:12.192 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "49'21082", 'reported_seq': 21251, 'reported_epoch': 59, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:10:52.170050+0000', 'last_change': '2024-09-10T15:59:16.331899+0000', 'last_active': '2024-09-10T16:10:52.170050+0000', 'last_peered': '2024-09-10T16:10:52.170050+0000', 'last_clean': '2024-09-10T15:59:14.139436+0000', 'last_became_active': '2024-09-10T15:59:16.318715+0000', 'last_became_peered': '2024-09-10T15:59:16.318715+0000', 'last_unstale': '2024-09-10T16:10:52.170050+0000', 'last_undegraded': '2024-09-10T15:59:15.877524+0000', 'last_fullsized': '2024-09-10T15:59:15.769042+0000', 'mapping_epoch': 42, 'log_start': "45'11000", 'ondisk_log_start': "45'11000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3537", 'last_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'objects_scrubbed': 3497, 'log_size': 10082, 'log_dups_size': 0, 'ondisk_log_size': 10082, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1224, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4237, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 17581, 'num_write_kb': 28172, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4235}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:11:12.192 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-10T16:11:12.192 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "49'20838", 'reported_seq': 21004, 'reported_epoch': 59, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:10:52.169840+0000', 'last_change': '2024-09-10T15:59:16.315641+0000', 'last_active': '2024-09-10T16:10:52.169840+0000', 'last_peered': '2024-09-10T16:10:52.169840+0000', 'last_clean': '2024-09-10T15:59:14.117001+0000', 'last_became_active': '2024-09-10T15:59:16.302245+0000', 'last_became_peered': '2024-09-10T15:59:16.302245+0000', 'last_unstale': '2024-09-10T16:10:52.169840+0000', 'last_undegraded': '2024-09-10T15:59:15.845012+0000', 'last_fullsized': '2024-09-10T15:59:15.770337+0000', 'mapping_epoch': 42, 'log_start': "45'10800", 'ondisk_log_start': "45'10800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3404", 'last_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'objects_scrubbed': 3367, 'log_size': 10038, 'log_dups_size': 0, 'ondisk_log_size': 10038, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 983, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17469, 'num_write_kb': 28204, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4304}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:11:12.192 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-10T16:11:12.193 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "49'21048", 'reported_seq': 27789, 'reported_epoch': 59, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:11:11.223560+0000', 'last_change': '2024-09-10T16:10:50.728566+0000', 'last_active': '2024-09-10T16:11:11.223560+0000', 'last_peered': '2024-09-10T16:11:11.223560+0000', 'last_clean': '2024-09-10T15:59:14.116363+0000', 'last_became_active': '2024-09-10T15:59:16.301244+0000', 'last_became_peered': '2024-09-10T15:59:16.301244+0000', 'last_unstale': '2024-09-10T16:11:11.223560+0000', 'last_undegraded': '2024-09-10T15:59:15.883434+0000', 'last_fullsized': '2024-09-10T15:59:15.772756+0000', 'mapping_epoch': 42, 'log_start': "45'11000", 'ondisk_log_start': "45'11000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3789", 'last_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'objects_scrubbed': 3741, 'log_size': 10048, 'log_dups_size': 0, 'ondisk_log_size': 10048, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1440, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1090, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17303, 'num_write_kb': 27124, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3283, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1090}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:11:12.193 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-10T16:11:12.193 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "49'21198", 'reported_seq': 21374, 'reported_epoch': 59, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:10:52.169851+0000', 'last_change': '2024-09-10T15:59:16.313409+0000', 'last_active': '2024-09-10T16:10:52.169851+0000', 'last_peered': '2024-09-10T16:10:52.169851+0000', 'last_clean': '2024-09-10T15:59:14.129780+0000', 'last_became_active': '2024-09-10T15:59:16.300932+0000', 'last_became_peered': '2024-09-10T15:59:16.300932+0000', 'last_unstale': '2024-09-10T16:10:52.169851+0000', 'last_undegraded': '2024-09-10T15:59:15.887823+0000', 'last_fullsized': '2024-09-10T15:59:15.768513+0000', 'mapping_epoch': 42, 'log_start': "45'11100", 'ondisk_log_start': "45'11100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3880", 'last_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'objects_scrubbed': 3827, 'log_size': 10098, 'log_dups_size': 0, 'ondisk_log_size': 10098, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1397, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4325, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17361, 'num_write_kb': 27072, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4321}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:11:12.193 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T16:11:12.569 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:11:12.569 INFO:teuthology.orchestra.run.smithi115.stdout:{"fsid":"7effee29-df05-4351-9741-ee203564ed81","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 13957/24 objects degraded (58154.167%), 4 pgs degraded, 4 pgs undersized","count":8},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":836,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":59,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983954,"num_in_osds":3,"osd_in_since":1725983947,"num_remapped_pgs":4},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":20},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":3},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":12,"data_bytes":4849,"bytes_used":1569099776,"bytes_avail":288341192704,"bytes_total":289910292480,"degraded_objects":13957,"degraded_total":24,"degraded_ratio":581.54166666666663,"recovering_objects_per_sec":197,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":2369,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-10T15:57:14:967120+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T16:00:13.560776+0000","services":{}},"progress_events":{}} 2024-09-10T16:11:12.581 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-10T16:11:12.582 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:11:12.887 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:11:12.888 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:11:12.900 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":470,"stamp":"2024-09-10T16:11:11.791123+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":13957,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292098,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37435,"num_bytes_recovered":33097148,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161025,"ondisk_log_size":161025,"up":48,"acting":44,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1532324,"kb_used_data":197552,"kb_used_omap":12,"kb_used_meta":1334707,"kb_avail":281583196,"statfs":{"total":289910292480,"available":288341192704,"internally_reserved":0,"allocated":202293248,"data_stored":201197990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1366739970},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2369,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2369,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001953"},"pg_stats":[{"pgid":"2.f","version":"49'20672","reported_seq":29304,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.169962+0000","last_change":"2024-09-10T16:09:57.088531+0000","last_active":"2024-09-10T16:10:52.169962+0000","last_peered":"2024-09-10T16:10:52.169962+0000","last_clean":"2024-09-10T16:10:52.169962+0000","last_became_active":"2024-09-10T16:09:57.005755+0000","last_became_peered":"2024-09-10T16:09:57.005755+0000","last_unstale":"2024-09-10T16:10:52.169962+0000","last_undegraded":"2024-09-10T16:10:52.169962+0000","last_fullsized":"2024-09-10T16:10:52.169962+0000","mapping_epoch":54,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:09:19.329697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20672,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4259,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20844","reported_seq":20988,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.168075+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:10:52.168075+0000","last_peered":"2024-09-10T16:10:52.168075+0000","last_clean":"2024-09-10T16:10:52.168075+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:10:52.168075+0000","last_undegraded":"2024-09-10T16:10:52.168075+0000","last_fullsized":"2024-09-10T16:10:52.168075+0000","mapping_epoch":35,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18949,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20798","reported_seq":29507,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.168961+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:10:52.168961+0000","last_peered":"2024-09-10T16:10:52.168961+0000","last_clean":"2024-09-10T16:10:52.168961+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:10:52.168961+0000","last_undegraded":"2024-09-10T16:10:52.168961+0000","last_fullsized":"2024-09-10T16:10:52.168961+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18802,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20660","reported_seq":20772,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.167700+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:10:52.167700+0000","last_peered":"2024-09-10T16:10:52.167700+0000","last_clean":"2024-09-10T16:10:52.167700+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:10:52.167700+0000","last_undegraded":"2024-09-10T16:10:52.167700+0000","last_fullsized":"2024-09-10T16:10:52.167700+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20660,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20891","reported_seq":21052,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.169809+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:10:52.169809+0000","last_peered":"2024-09-10T16:10:52.169809+0000","last_clean":"2024-09-10T16:10:52.169809+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:10:52.169809+0000","last_undegraded":"2024-09-10T16:10:52.169809+0000","last_fullsized":"2024-09-10T16:10:52.169809+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17602,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'21061","reported_seq":21222,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.242289+0000","last_change":"2024-09-10T16:10:52.241702+0000","last_active":"2024-09-10T16:10:52.242289+0000","last_peered":"2024-09-10T16:10:52.242289+0000","last_clean":"2024-09-10T16:10:52.242289+0000","last_became_active":"2024-09-10T16:10:52.171236+0000","last_became_peered":"2024-09-10T16:10:52.171236+0000","last_unstale":"2024-09-10T16:10:52.242289+0000","last_undegraded":"2024-09-10T16:10:52.242289+0000","last_fullsized":"2024-09-10T16:10:52.242289+0000","mapping_epoch":58,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:04:46.778280+0000","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17673,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4372,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'21082","reported_seq":21251,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:10:52.170050+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:10:52.170050+0000","last_peered":"2024-09-10T16:10:52.170050+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:10:52.170050+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17581,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4235},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'21314","reported_seq":21476,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.169765+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:10:52.169765+0000","last_peered":"2024-09-10T16:10:52.169765+0000","last_clean":"2024-09-10T16:10:52.169765+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:10:52.169765+0000","last_undegraded":"2024-09-10T16:10:52.169765+0000","last_fullsized":"2024-09-10T16:10:52.169765+0000","mapping_epoch":34,"log_start":"45'11300","ondisk_log_start":"45'11300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18161,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.168728+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:10:52.168728+0000","last_peered":"2024-09-10T16:10:52.168728+0000","last_clean":"2024-09-10T16:10:52.168728+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:10:52.168728+0000","last_undegraded":"2024-09-10T16:10:52.168728+0000","last_fullsized":"2024-09-10T16:10:52.168728+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20457","reported_seq":20632,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.168901+0000","last_change":"2024-09-10T16:09:33.012840+0000","last_active":"2024-09-10T16:10:52.168901+0000","last_peered":"2024-09-10T16:10:52.168901+0000","last_clean":"2024-09-10T16:10:52.168901+0000","last_became_active":"2024-09-10T16:09:32.934056+0000","last_became_peered":"2024-09-10T16:09:32.934056+0000","last_unstale":"2024-09-10T16:10:52.168901+0000","last_undegraded":"2024-09-10T16:10:52.168901+0000","last_fullsized":"2024-09-10T16:10:52.168901+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:08:30.481235+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18276,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":4579,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":126,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.168004+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:10:52.168004+0000","last_peered":"2024-09-10T16:10:52.168004+0000","last_clean":"2024-09-10T16:10:52.168004+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:10:52.168004+0000","last_undegraded":"2024-09-10T16:10:52.168004+0000","last_fullsized":"2024-09-10T16:10:52.168004+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20910","reported_seq":29594,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.169792+0000","last_change":"2024-09-10T16:09:08.909580+0000","last_active":"2024-09-10T16:10:52.169792+0000","last_peered":"2024-09-10T16:10:52.169792+0000","last_clean":"2024-09-10T16:10:52.169792+0000","last_became_active":"2024-09-10T16:09:08.854941+0000","last_became_peered":"2024-09-10T16:09:08.854941+0000","last_unstale":"2024-09-10T16:10:52.169792+0000","last_undegraded":"2024-09-10T16:10:52.169792+0000","last_fullsized":"2024-09-10T16:10:52.169792+0000","mapping_epoch":50,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:13:14.846691+0000","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17414,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4257,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":136,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.169714+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:10:52.169714+0000","last_peered":"2024-09-10T16:10:52.169714+0000","last_clean":"2024-09-10T16:10:52.169714+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:10:52.169714+0000","last_undegraded":"2024-09-10T16:10:52.169714+0000","last_fullsized":"2024-09-10T16:10:52.169714+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20781","reported_seq":29566,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.168064+0000","last_change":"2024-09-10T16:10:20.907889+0000","last_active":"2024-09-10T16:10:52.168064+0000","last_peered":"2024-09-10T16:10:52.168064+0000","last_clean":"2024-09-10T16:10:52.168064+0000","last_became_active":"2024-09-10T16:10:20.852065+0000","last_became_peered":"2024-09-10T16:10:20.852065+0000","last_unstale":"2024-09-10T16:10:52.168064+0000","last_undegraded":"2024-09-10T16:10:52.168064+0000","last_fullsized":"2024-09-10T16:10:52.168064+0000","mapping_epoch":56,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:34.069549+0000","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18461,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4307,"num_bytes_recovered":8675,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":111,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.168792+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:10:52.168792+0000","last_peered":"2024-09-10T16:10:52.168792+0000","last_clean":"2024-09-10T16:10:52.168792+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:10:52.168792+0000","last_undegraded":"2024-09-10T16:10:52.168792+0000","last_fullsized":"2024-09-10T16:10:52.168792+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'21170","reported_seq":30051,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.167373+0000","last_change":"2024-09-10T16:10:50.728349+0000","last_active":"2024-09-10T16:10:52.167373+0000","last_peered":"2024-09-10T16:10:52.167373+0000","last_clean":"2024-09-10T16:10:52.167373+0000","last_became_active":"2024-09-10T16:08:10.860438+0000","last_became_peered":"2024-09-10T16:08:10.860438+0000","last_unstale":"2024-09-10T16:10:52.167373+0000","last_undegraded":"2024-09-10T16:10:52.167373+0000","last_fullsized":"2024-09-10T16:10:52.167373+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:38:13.781380+0000","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18724,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20838","reported_seq":21004,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:10:52.169840+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:10:52.169840+0000","last_peered":"2024-09-10T16:10:52.169840+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:10:52.169840+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17469,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.168992+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:10:52.168992+0000","last_peered":"2024-09-10T16:10:52.168992+0000","last_clean":"2024-09-10T16:10:52.168992+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:10:52.168992+0000","last_undegraded":"2024-09-10T16:10:52.168992+0000","last_fullsized":"2024-09-10T16:10:52.168992+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20598","reported_seq":29195,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.169145+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:10:52.169145+0000","last_peered":"2024-09-10T16:10:52.169145+0000","last_clean":"2024-09-10T16:10:52.169145+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:10:52.169145+0000","last_undegraded":"2024-09-10T16:10:52.169145+0000","last_fullsized":"2024-09-10T16:10:52.169145+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16988,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":126,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.167911+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:10:52.167911+0000","last_peered":"2024-09-10T16:10:52.167911+0000","last_clean":"2024-09-10T16:10:52.167911+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:10:52.167911+0000","last_undegraded":"2024-09-10T16:10:52.167911+0000","last_fullsized":"2024-09-10T16:10:52.167911+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'21048","reported_seq":27789,"reported_epoch":59,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:11:11.223560+0000","last_change":"2024-09-10T16:10:50.728566+0000","last_active":"2024-09-10T16:11:11.223560+0000","last_peered":"2024-09-10T16:11:11.223560+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:11:11.223560+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1090,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17303,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3283,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1090}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.169062+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:10:52.169062+0000","last_peered":"2024-09-10T16:10:52.169062+0000","last_clean":"2024-09-10T16:10:52.169062+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:10:52.169062+0000","last_undegraded":"2024-09-10T16:10:52.169062+0000","last_fullsized":"2024-09-10T16:10:52.169062+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":106,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.169114+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:10:52.169114+0000","last_peered":"2024-09-10T16:10:52.169114+0000","last_clean":"2024-09-10T16:10:52.169114+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:10:52.169114+0000","last_undegraded":"2024-09-10T16:10:52.169114+0000","last_fullsized":"2024-09-10T16:10:52.169114+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'21198","reported_seq":21374,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:10:52.169851+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:10:52.169851+0000","last_peered":"2024-09-10T16:10:52.169851+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:10:52.169851+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17361,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4321},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":13957,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292096,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37433,"num_bytes_recovered":33097129,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":200441856,"data_stored":200333466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161022,"ondisk_log_size":161022,"up":32,"acting":28,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986334,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":493208,"kb_used_data":128328,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93878632,"statfs":{"total":96636764160,"available":96131719168,"internally_reserved":0,"allocated":131407872,"data_stored":131045504,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659279,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":491740,"kb_used_data":524,"kb_used_omap":4,"kb_used_meta":491195,"kb_avail":93880100,"statfs":{"total":96636764160,"available":96133222400,"internally_reserved":0,"allocated":536576,"data_stored":170131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":502984366},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182562,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":547376,"kb_used_data":68700,"kb_used_omap":4,"kb_used_meta":478651,"kb_avail":93824464,"statfs":{"total":96636764160,"available":96076251136,"internally_reserved":0,"allocated":70348800,"data_stored":69982355,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":490139306},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":69853184,"data_stored":69817054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":4830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":130547712,"data_stored":130511582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:11:15.902 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:11:16.286 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:11:16.287 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:11:16.300 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":473,"stamp":"2024-09-10T16:11:16.221875+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":13957,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292098,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37435,"num_bytes_recovered":33097148,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161025,"ondisk_log_size":161025,"up":48,"acting":44,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1521604,"kb_used_data":195152,"kb_used_omap":12,"kb_used_meta":1326387,"kb_avail":281593916,"statfs":{"total":289910292480,"available":288352169984,"internally_reserved":0,"allocated":199835648,"data_stored":198740390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1358220290},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1577,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1577,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.431680"},"pg_stats":[{"pgid":"2.f","version":"49'20672","reported_seq":29304,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.169962+0000","last_change":"2024-09-10T16:09:57.088531+0000","last_active":"2024-09-10T16:10:52.169962+0000","last_peered":"2024-09-10T16:10:52.169962+0000","last_clean":"2024-09-10T16:10:52.169962+0000","last_became_active":"2024-09-10T16:09:57.005755+0000","last_became_peered":"2024-09-10T16:09:57.005755+0000","last_unstale":"2024-09-10T16:10:52.169962+0000","last_undegraded":"2024-09-10T16:10:52.169962+0000","last_fullsized":"2024-09-10T16:10:52.169962+0000","mapping_epoch":54,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:09:19.329697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20672,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4259,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20844","reported_seq":20988,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.168075+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:10:52.168075+0000","last_peered":"2024-09-10T16:10:52.168075+0000","last_clean":"2024-09-10T16:10:52.168075+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:10:52.168075+0000","last_undegraded":"2024-09-10T16:10:52.168075+0000","last_fullsized":"2024-09-10T16:10:52.168075+0000","mapping_epoch":35,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18949,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20798","reported_seq":29508,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.169011+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:10:52.169011+0000","last_peered":"2024-09-10T16:10:52.169011+0000","last_clean":"2024-09-10T16:10:52.169011+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:10:52.169011+0000","last_undegraded":"2024-09-10T16:10:52.169011+0000","last_fullsized":"2024-09-10T16:10:52.169011+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18802,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20660","reported_seq":20772,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.167700+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:10:52.167700+0000","last_peered":"2024-09-10T16:10:52.167700+0000","last_clean":"2024-09-10T16:10:52.167700+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:10:52.167700+0000","last_undegraded":"2024-09-10T16:10:52.167700+0000","last_fullsized":"2024-09-10T16:10:52.167700+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20660,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20891","reported_seq":21052,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.169809+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:10:52.169809+0000","last_peered":"2024-09-10T16:10:52.169809+0000","last_clean":"2024-09-10T16:10:52.169809+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:10:52.169809+0000","last_undegraded":"2024-09-10T16:10:52.169809+0000","last_fullsized":"2024-09-10T16:10:52.169809+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17602,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'21061","reported_seq":21222,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.242289+0000","last_change":"2024-09-10T16:10:52.241702+0000","last_active":"2024-09-10T16:10:52.242289+0000","last_peered":"2024-09-10T16:10:52.242289+0000","last_clean":"2024-09-10T16:10:52.242289+0000","last_became_active":"2024-09-10T16:10:52.171236+0000","last_became_peered":"2024-09-10T16:10:52.171236+0000","last_unstale":"2024-09-10T16:10:52.242289+0000","last_undegraded":"2024-09-10T16:10:52.242289+0000","last_fullsized":"2024-09-10T16:10:52.242289+0000","mapping_epoch":58,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:04:46.778280+0000","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17673,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4372,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'21082","reported_seq":21251,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:10:52.170050+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:10:52.170050+0000","last_peered":"2024-09-10T16:10:52.170050+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:10:52.170050+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17581,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4235},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'21314","reported_seq":21476,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.169765+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:10:52.169765+0000","last_peered":"2024-09-10T16:10:52.169765+0000","last_clean":"2024-09-10T16:10:52.169765+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:10:52.169765+0000","last_undegraded":"2024-09-10T16:10:52.169765+0000","last_fullsized":"2024-09-10T16:10:52.169765+0000","mapping_epoch":34,"log_start":"45'11300","ondisk_log_start":"45'11300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18161,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.168728+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:10:52.168728+0000","last_peered":"2024-09-10T16:10:52.168728+0000","last_clean":"2024-09-10T16:10:52.168728+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:10:52.168728+0000","last_undegraded":"2024-09-10T16:10:52.168728+0000","last_fullsized":"2024-09-10T16:10:52.168728+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20457","reported_seq":20632,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.168901+0000","last_change":"2024-09-10T16:09:33.012840+0000","last_active":"2024-09-10T16:10:52.168901+0000","last_peered":"2024-09-10T16:10:52.168901+0000","last_clean":"2024-09-10T16:10:52.168901+0000","last_became_active":"2024-09-10T16:09:32.934056+0000","last_became_peered":"2024-09-10T16:09:32.934056+0000","last_unstale":"2024-09-10T16:10:52.168901+0000","last_undegraded":"2024-09-10T16:10:52.168901+0000","last_fullsized":"2024-09-10T16:10:52.168901+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:08:30.481235+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18276,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":4579,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":126,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.168004+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:10:52.168004+0000","last_peered":"2024-09-10T16:10:52.168004+0000","last_clean":"2024-09-10T16:10:52.168004+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:10:52.168004+0000","last_undegraded":"2024-09-10T16:10:52.168004+0000","last_fullsized":"2024-09-10T16:10:52.168004+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20910","reported_seq":29594,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.169792+0000","last_change":"2024-09-10T16:09:08.909580+0000","last_active":"2024-09-10T16:10:52.169792+0000","last_peered":"2024-09-10T16:10:52.169792+0000","last_clean":"2024-09-10T16:10:52.169792+0000","last_became_active":"2024-09-10T16:09:08.854941+0000","last_became_peered":"2024-09-10T16:09:08.854941+0000","last_unstale":"2024-09-10T16:10:52.169792+0000","last_undegraded":"2024-09-10T16:10:52.169792+0000","last_fullsized":"2024-09-10T16:10:52.169792+0000","mapping_epoch":50,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:13:14.846691+0000","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17414,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4257,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":136,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.169714+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:10:52.169714+0000","last_peered":"2024-09-10T16:10:52.169714+0000","last_clean":"2024-09-10T16:10:52.169714+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:10:52.169714+0000","last_undegraded":"2024-09-10T16:10:52.169714+0000","last_fullsized":"2024-09-10T16:10:52.169714+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20781","reported_seq":29566,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.168064+0000","last_change":"2024-09-10T16:10:20.907889+0000","last_active":"2024-09-10T16:10:52.168064+0000","last_peered":"2024-09-10T16:10:52.168064+0000","last_clean":"2024-09-10T16:10:52.168064+0000","last_became_active":"2024-09-10T16:10:20.852065+0000","last_became_peered":"2024-09-10T16:10:20.852065+0000","last_unstale":"2024-09-10T16:10:52.168064+0000","last_undegraded":"2024-09-10T16:10:52.168064+0000","last_fullsized":"2024-09-10T16:10:52.168064+0000","mapping_epoch":56,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:34.069549+0000","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18461,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4307,"num_bytes_recovered":8675,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":111,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.168792+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:10:52.168792+0000","last_peered":"2024-09-10T16:10:52.168792+0000","last_clean":"2024-09-10T16:10:52.168792+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:10:52.168792+0000","last_undegraded":"2024-09-10T16:10:52.168792+0000","last_fullsized":"2024-09-10T16:10:52.168792+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'21170","reported_seq":30051,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.167373+0000","last_change":"2024-09-10T16:10:50.728349+0000","last_active":"2024-09-10T16:10:52.167373+0000","last_peered":"2024-09-10T16:10:52.167373+0000","last_clean":"2024-09-10T16:10:52.167373+0000","last_became_active":"2024-09-10T16:08:10.860438+0000","last_became_peered":"2024-09-10T16:08:10.860438+0000","last_unstale":"2024-09-10T16:10:52.167373+0000","last_undegraded":"2024-09-10T16:10:52.167373+0000","last_fullsized":"2024-09-10T16:10:52.167373+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:38:13.781380+0000","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18724,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20838","reported_seq":21004,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:10:52.169840+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:10:52.169840+0000","last_peered":"2024-09-10T16:10:52.169840+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:10:52.169840+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17469,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.168992+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:10:52.168992+0000","last_peered":"2024-09-10T16:10:52.168992+0000","last_clean":"2024-09-10T16:10:52.168992+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:10:52.168992+0000","last_undegraded":"2024-09-10T16:10:52.168992+0000","last_fullsized":"2024-09-10T16:10:52.168992+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20598","reported_seq":29195,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.169145+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:10:52.169145+0000","last_peered":"2024-09-10T16:10:52.169145+0000","last_clean":"2024-09-10T16:10:52.169145+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:10:52.169145+0000","last_undegraded":"2024-09-10T16:10:52.169145+0000","last_fullsized":"2024-09-10T16:10:52.169145+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16988,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":126,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.167911+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:10:52.167911+0000","last_peered":"2024-09-10T16:10:52.167911+0000","last_clean":"2024-09-10T16:10:52.167911+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:10:52.167911+0000","last_undegraded":"2024-09-10T16:10:52.167911+0000","last_fullsized":"2024-09-10T16:10:52.167911+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'21048","reported_seq":27789,"reported_epoch":59,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:11:11.223560+0000","last_change":"2024-09-10T16:10:50.728566+0000","last_active":"2024-09-10T16:11:11.223560+0000","last_peered":"2024-09-10T16:11:11.223560+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:11:11.223560+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1090,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17303,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3283,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1090}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.169062+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:10:52.169062+0000","last_peered":"2024-09-10T16:10:52.169062+0000","last_clean":"2024-09-10T16:10:52.169062+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:10:52.169062+0000","last_undegraded":"2024-09-10T16:10:52.169062+0000","last_fullsized":"2024-09-10T16:10:52.169062+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":106,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.169114+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:10:52.169114+0000","last_peered":"2024-09-10T16:10:52.169114+0000","last_clean":"2024-09-10T16:10:52.169114+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:10:52.169114+0000","last_undegraded":"2024-09-10T16:10:52.169114+0000","last_fullsized":"2024-09-10T16:10:52.169114+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'21198","reported_seq":21374,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:10:52.169851+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:10:52.169851+0000","last_peered":"2024-09-10T16:10:52.169851+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:10:52.169851+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17361,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4321},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":13957,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292096,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37433,"num_bytes_recovered":33097129,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":197984256,"data_stored":197875866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161022,"ondisk_log_size":161022,"up":32,"acting":28,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986335,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":490808,"kb_used_data":125928,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93881032,"statfs":{"total":96636764160,"available":96134176768,"internally_reserved":0,"allocated":128950272,"data_stored":128587904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659280,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":483420,"kb_used_data":524,"kb_used_omap":4,"kb_used_meta":482875,"kb_avail":93888420,"statfs":{"total":96636764160,"available":96141742080,"internally_reserved":0,"allocated":536576,"data_stored":170131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":494464686},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182562,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":547376,"kb_used_data":68700,"kb_used_omap":4,"kb_used_meta":478651,"kb_avail":93824464,"statfs":{"total":96636764160,"available":96076251136,"internally_reserved":0,"allocated":70348800,"data_stored":69982355,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":490139306},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":69853184,"data_stored":69817054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":4830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":128090112,"data_stored":128053982,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:11:16.302 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:11:16.641 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:11:16.642 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:11:16.654 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":473,"stamp":"2024-09-10T16:11:16.221875+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":13957,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292098,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37435,"num_bytes_recovered":33097148,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161025,"ondisk_log_size":161025,"up":48,"acting":44,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1521604,"kb_used_data":195152,"kb_used_omap":12,"kb_used_meta":1326387,"kb_avail":281593916,"statfs":{"total":289910292480,"available":288352169984,"internally_reserved":0,"allocated":199835648,"data_stored":198740390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1358220290},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1577,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1577,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.431680"},"pg_stats":[{"pgid":"2.f","version":"49'20672","reported_seq":29304,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.169962+0000","last_change":"2024-09-10T16:09:57.088531+0000","last_active":"2024-09-10T16:10:52.169962+0000","last_peered":"2024-09-10T16:10:52.169962+0000","last_clean":"2024-09-10T16:10:52.169962+0000","last_became_active":"2024-09-10T16:09:57.005755+0000","last_became_peered":"2024-09-10T16:09:57.005755+0000","last_unstale":"2024-09-10T16:10:52.169962+0000","last_undegraded":"2024-09-10T16:10:52.169962+0000","last_fullsized":"2024-09-10T16:10:52.169962+0000","mapping_epoch":54,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:09:19.329697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20672,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4259,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20844","reported_seq":20988,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.168075+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:10:52.168075+0000","last_peered":"2024-09-10T16:10:52.168075+0000","last_clean":"2024-09-10T16:10:52.168075+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:10:52.168075+0000","last_undegraded":"2024-09-10T16:10:52.168075+0000","last_fullsized":"2024-09-10T16:10:52.168075+0000","mapping_epoch":35,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18949,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20798","reported_seq":29508,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.169011+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:10:52.169011+0000","last_peered":"2024-09-10T16:10:52.169011+0000","last_clean":"2024-09-10T16:10:52.169011+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:10:52.169011+0000","last_undegraded":"2024-09-10T16:10:52.169011+0000","last_fullsized":"2024-09-10T16:10:52.169011+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18802,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20660","reported_seq":20772,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.167700+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:10:52.167700+0000","last_peered":"2024-09-10T16:10:52.167700+0000","last_clean":"2024-09-10T16:10:52.167700+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:10:52.167700+0000","last_undegraded":"2024-09-10T16:10:52.167700+0000","last_fullsized":"2024-09-10T16:10:52.167700+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20660,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20891","reported_seq":21052,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.169809+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:10:52.169809+0000","last_peered":"2024-09-10T16:10:52.169809+0000","last_clean":"2024-09-10T16:10:52.169809+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:10:52.169809+0000","last_undegraded":"2024-09-10T16:10:52.169809+0000","last_fullsized":"2024-09-10T16:10:52.169809+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17602,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'21061","reported_seq":21222,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.242289+0000","last_change":"2024-09-10T16:10:52.241702+0000","last_active":"2024-09-10T16:10:52.242289+0000","last_peered":"2024-09-10T16:10:52.242289+0000","last_clean":"2024-09-10T16:10:52.242289+0000","last_became_active":"2024-09-10T16:10:52.171236+0000","last_became_peered":"2024-09-10T16:10:52.171236+0000","last_unstale":"2024-09-10T16:10:52.242289+0000","last_undegraded":"2024-09-10T16:10:52.242289+0000","last_fullsized":"2024-09-10T16:10:52.242289+0000","mapping_epoch":58,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:04:46.778280+0000","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17673,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4372,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'21082","reported_seq":21251,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:10:52.170050+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:10:52.170050+0000","last_peered":"2024-09-10T16:10:52.170050+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:10:52.170050+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17581,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4235},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'21314","reported_seq":21476,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.169765+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:10:52.169765+0000","last_peered":"2024-09-10T16:10:52.169765+0000","last_clean":"2024-09-10T16:10:52.169765+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:10:52.169765+0000","last_undegraded":"2024-09-10T16:10:52.169765+0000","last_fullsized":"2024-09-10T16:10:52.169765+0000","mapping_epoch":34,"log_start":"45'11300","ondisk_log_start":"45'11300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18161,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.168728+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:10:52.168728+0000","last_peered":"2024-09-10T16:10:52.168728+0000","last_clean":"2024-09-10T16:10:52.168728+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:10:52.168728+0000","last_undegraded":"2024-09-10T16:10:52.168728+0000","last_fullsized":"2024-09-10T16:10:52.168728+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20457","reported_seq":20632,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.168901+0000","last_change":"2024-09-10T16:09:33.012840+0000","last_active":"2024-09-10T16:10:52.168901+0000","last_peered":"2024-09-10T16:10:52.168901+0000","last_clean":"2024-09-10T16:10:52.168901+0000","last_became_active":"2024-09-10T16:09:32.934056+0000","last_became_peered":"2024-09-10T16:09:32.934056+0000","last_unstale":"2024-09-10T16:10:52.168901+0000","last_undegraded":"2024-09-10T16:10:52.168901+0000","last_fullsized":"2024-09-10T16:10:52.168901+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:08:30.481235+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18276,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":4579,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":126,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.168004+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:10:52.168004+0000","last_peered":"2024-09-10T16:10:52.168004+0000","last_clean":"2024-09-10T16:10:52.168004+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:10:52.168004+0000","last_undegraded":"2024-09-10T16:10:52.168004+0000","last_fullsized":"2024-09-10T16:10:52.168004+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20910","reported_seq":29594,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.169792+0000","last_change":"2024-09-10T16:09:08.909580+0000","last_active":"2024-09-10T16:10:52.169792+0000","last_peered":"2024-09-10T16:10:52.169792+0000","last_clean":"2024-09-10T16:10:52.169792+0000","last_became_active":"2024-09-10T16:09:08.854941+0000","last_became_peered":"2024-09-10T16:09:08.854941+0000","last_unstale":"2024-09-10T16:10:52.169792+0000","last_undegraded":"2024-09-10T16:10:52.169792+0000","last_fullsized":"2024-09-10T16:10:52.169792+0000","mapping_epoch":50,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:13:14.846691+0000","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17414,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4257,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":136,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.169714+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:10:52.169714+0000","last_peered":"2024-09-10T16:10:52.169714+0000","last_clean":"2024-09-10T16:10:52.169714+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:10:52.169714+0000","last_undegraded":"2024-09-10T16:10:52.169714+0000","last_fullsized":"2024-09-10T16:10:52.169714+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20781","reported_seq":29566,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.168064+0000","last_change":"2024-09-10T16:10:20.907889+0000","last_active":"2024-09-10T16:10:52.168064+0000","last_peered":"2024-09-10T16:10:52.168064+0000","last_clean":"2024-09-10T16:10:52.168064+0000","last_became_active":"2024-09-10T16:10:20.852065+0000","last_became_peered":"2024-09-10T16:10:20.852065+0000","last_unstale":"2024-09-10T16:10:52.168064+0000","last_undegraded":"2024-09-10T16:10:52.168064+0000","last_fullsized":"2024-09-10T16:10:52.168064+0000","mapping_epoch":56,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:34.069549+0000","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18461,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4307,"num_bytes_recovered":8675,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":111,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.168792+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:10:52.168792+0000","last_peered":"2024-09-10T16:10:52.168792+0000","last_clean":"2024-09-10T16:10:52.168792+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:10:52.168792+0000","last_undegraded":"2024-09-10T16:10:52.168792+0000","last_fullsized":"2024-09-10T16:10:52.168792+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'21170","reported_seq":30051,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.167373+0000","last_change":"2024-09-10T16:10:50.728349+0000","last_active":"2024-09-10T16:10:52.167373+0000","last_peered":"2024-09-10T16:10:52.167373+0000","last_clean":"2024-09-10T16:10:52.167373+0000","last_became_active":"2024-09-10T16:08:10.860438+0000","last_became_peered":"2024-09-10T16:08:10.860438+0000","last_unstale":"2024-09-10T16:10:52.167373+0000","last_undegraded":"2024-09-10T16:10:52.167373+0000","last_fullsized":"2024-09-10T16:10:52.167373+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:38:13.781380+0000","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18724,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20838","reported_seq":21004,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:10:52.169840+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:10:52.169840+0000","last_peered":"2024-09-10T16:10:52.169840+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:10:52.169840+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17469,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.168992+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:10:52.168992+0000","last_peered":"2024-09-10T16:10:52.168992+0000","last_clean":"2024-09-10T16:10:52.168992+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:10:52.168992+0000","last_undegraded":"2024-09-10T16:10:52.168992+0000","last_fullsized":"2024-09-10T16:10:52.168992+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20598","reported_seq":29195,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.169145+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:10:52.169145+0000","last_peered":"2024-09-10T16:10:52.169145+0000","last_clean":"2024-09-10T16:10:52.169145+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:10:52.169145+0000","last_undegraded":"2024-09-10T16:10:52.169145+0000","last_fullsized":"2024-09-10T16:10:52.169145+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16988,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":126,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.167911+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:10:52.167911+0000","last_peered":"2024-09-10T16:10:52.167911+0000","last_clean":"2024-09-10T16:10:52.167911+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:10:52.167911+0000","last_undegraded":"2024-09-10T16:10:52.167911+0000","last_fullsized":"2024-09-10T16:10:52.167911+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'21048","reported_seq":27789,"reported_epoch":59,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:11:11.223560+0000","last_change":"2024-09-10T16:10:50.728566+0000","last_active":"2024-09-10T16:11:11.223560+0000","last_peered":"2024-09-10T16:11:11.223560+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:11:11.223560+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1090,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17303,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3283,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1090}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.169062+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:10:52.169062+0000","last_peered":"2024-09-10T16:10:52.169062+0000","last_clean":"2024-09-10T16:10:52.169062+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:10:52.169062+0000","last_undegraded":"2024-09-10T16:10:52.169062+0000","last_fullsized":"2024-09-10T16:10:52.169062+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":106,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.169114+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:10:52.169114+0000","last_peered":"2024-09-10T16:10:52.169114+0000","last_clean":"2024-09-10T16:10:52.169114+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:10:52.169114+0000","last_undegraded":"2024-09-10T16:10:52.169114+0000","last_fullsized":"2024-09-10T16:10:52.169114+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'21198","reported_seq":21374,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:10:52.169851+0000","last_change":"2024-09-10T15:59:16.313409+0000","last_active":"2024-09-10T16:10:52.169851+0000","last_peered":"2024-09-10T16:10:52.169851+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:10:52.169851+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17361,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4321},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":13957,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292096,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37433,"num_bytes_recovered":33097129,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":197984256,"data_stored":197875866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161022,"ondisk_log_size":161022,"up":32,"acting":28,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986335,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":490808,"kb_used_data":125928,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93881032,"statfs":{"total":96636764160,"available":96134176768,"internally_reserved":0,"allocated":128950272,"data_stored":128587904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659280,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":483420,"kb_used_data":524,"kb_used_omap":4,"kb_used_meta":482875,"kb_avail":93888420,"statfs":{"total":96636764160,"available":96141742080,"internally_reserved":0,"allocated":536576,"data_stored":170131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":494464686},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182562,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":547376,"kb_used_data":68700,"kb_used_omap":4,"kb_used_meta":478651,"kb_avail":93824464,"statfs":{"total":96636764160,"available":96076251136,"internally_reserved":0,"allocated":70348800,"data_stored":69982355,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":490139306},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":69853184,"data_stored":69817054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":4830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":128090112,"data_stored":128053982,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:11:16.656 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-10T16:11:16.656 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "49'21082", 'reported_seq': 21251, 'reported_epoch': 59, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:10:52.170050+0000', 'last_change': '2024-09-10T15:59:16.331899+0000', 'last_active': '2024-09-10T16:10:52.170050+0000', 'last_peered': '2024-09-10T16:10:52.170050+0000', 'last_clean': '2024-09-10T15:59:14.139436+0000', 'last_became_active': '2024-09-10T15:59:16.318715+0000', 'last_became_peered': '2024-09-10T15:59:16.318715+0000', 'last_unstale': '2024-09-10T16:10:52.170050+0000', 'last_undegraded': '2024-09-10T15:59:15.877524+0000', 'last_fullsized': '2024-09-10T15:59:15.769042+0000', 'mapping_epoch': 42, 'log_start': "45'11000", 'ondisk_log_start': "45'11000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3537", 'last_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'objects_scrubbed': 3497, 'log_size': 10082, 'log_dups_size': 0, 'ondisk_log_size': 10082, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1224, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4237, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 17581, 'num_write_kb': 28172, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4235}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:11:16.656 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-10T16:11:16.656 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "49'20838", 'reported_seq': 21004, 'reported_epoch': 59, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:10:52.169840+0000', 'last_change': '2024-09-10T15:59:16.315641+0000', 'last_active': '2024-09-10T16:10:52.169840+0000', 'last_peered': '2024-09-10T16:10:52.169840+0000', 'last_clean': '2024-09-10T15:59:14.117001+0000', 'last_became_active': '2024-09-10T15:59:16.302245+0000', 'last_became_peered': '2024-09-10T15:59:16.302245+0000', 'last_unstale': '2024-09-10T16:10:52.169840+0000', 'last_undegraded': '2024-09-10T15:59:15.845012+0000', 'last_fullsized': '2024-09-10T15:59:15.770337+0000', 'mapping_epoch': 42, 'log_start': "45'10800", 'ondisk_log_start': "45'10800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3404", 'last_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'objects_scrubbed': 3367, 'log_size': 10038, 'log_dups_size': 0, 'ondisk_log_size': 10038, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 983, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17469, 'num_write_kb': 28204, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4304}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:11:16.656 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-10T16:11:16.656 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "49'21048", 'reported_seq': 27789, 'reported_epoch': 59, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:11:11.223560+0000', 'last_change': '2024-09-10T16:10:50.728566+0000', 'last_active': '2024-09-10T16:11:11.223560+0000', 'last_peered': '2024-09-10T16:11:11.223560+0000', 'last_clean': '2024-09-10T15:59:14.116363+0000', 'last_became_active': '2024-09-10T15:59:16.301244+0000', 'last_became_peered': '2024-09-10T15:59:16.301244+0000', 'last_unstale': '2024-09-10T16:11:11.223560+0000', 'last_undegraded': '2024-09-10T15:59:15.883434+0000', 'last_fullsized': '2024-09-10T15:59:15.772756+0000', 'mapping_epoch': 42, 'log_start': "45'11000", 'ondisk_log_start': "45'11000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3789", 'last_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'objects_scrubbed': 3741, 'log_size': 10048, 'log_dups_size': 0, 'ondisk_log_size': 10048, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1440, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1090, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17303, 'num_write_kb': 27124, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3283, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1090}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:11:16.656 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-10T16:11:16.656 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "49'21198", 'reported_seq': 21374, 'reported_epoch': 59, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:10:52.169851+0000', 'last_change': '2024-09-10T15:59:16.313409+0000', 'last_active': '2024-09-10T16:10:52.169851+0000', 'last_peered': '2024-09-10T16:10:52.169851+0000', 'last_clean': '2024-09-10T15:59:14.129780+0000', 'last_became_active': '2024-09-10T15:59:16.300932+0000', 'last_became_peered': '2024-09-10T15:59:16.300932+0000', 'last_unstale': '2024-09-10T16:10:52.169851+0000', 'last_undegraded': '2024-09-10T15:59:15.887823+0000', 'last_fullsized': '2024-09-10T15:59:15.768513+0000', 'mapping_epoch': 42, 'log_start': "45'11100", 'ondisk_log_start': "45'11100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3880", 'last_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'objects_scrubbed': 3827, 'log_size': 10098, 'log_dups_size': 0, 'ondisk_log_size': 10098, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1397, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4325, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17361, 'num_write_kb': 27072, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4321}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:11:16.657 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T16:11:17.070 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:11:17.070 INFO:teuthology.orchestra.run.smithi115.stdout:{"fsid":"7effee29-df05-4351-9741-ee203564ed81","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 13957/24 objects degraded (58154.167%), 4 pgs degraded, 4 pgs undersized","count":8},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":841,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":60,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983954,"num_in_osds":3,"osd_in_since":1725983947,"num_remapped_pgs":3},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":20},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":3},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":12,"data_bytes":4849,"bytes_used":1560580096,"bytes_avail":288349712384,"bytes_total":289910292480,"degraded_objects":13957,"degraded_total":24,"degraded_ratio":581.54166666666663,"recovering_objects_per_sec":131,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1577,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-10T15:57:14:967120+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T16:00:13.560776+0000","services":{}},"progress_events":{}} 2024-09-10T16:11:17.084 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-10T16:11:17.084 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:11:17.437 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:11:17.437 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:11:17.452 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":474,"stamp":"2024-09-10T16:11:17.230361+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":13944,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292098,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37448,"num_bytes_recovered":33097631,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161025,"ondisk_log_size":161025,"up":48,"acting":44,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1521604,"kb_used_data":195152,"kb_used_omap":12,"kb_used_meta":1326387,"kb_avail":281593916,"statfs":{"total":289910292480,"available":288352169984,"internally_reserved":0,"allocated":199835648,"data_stored":198740390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1358220290},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-887,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":887,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.439863"},"pg_stats":[{"pgid":"2.f","version":"49'20672","reported_seq":29306,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:11:16.225228+0000","last_change":"2024-09-10T16:09:57.088531+0000","last_active":"2024-09-10T16:11:16.225228+0000","last_peered":"2024-09-10T16:11:16.225228+0000","last_clean":"2024-09-10T16:11:16.225228+0000","last_became_active":"2024-09-10T16:09:57.005755+0000","last_became_peered":"2024-09-10T16:09:57.005755+0000","last_unstale":"2024-09-10T16:11:16.225228+0000","last_undegraded":"2024-09-10T16:11:16.225228+0000","last_fullsized":"2024-09-10T16:11:16.225228+0000","mapping_epoch":54,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:09:19.329697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20672,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4259,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20844","reported_seq":20988,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.168075+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:10:52.168075+0000","last_peered":"2024-09-10T16:10:52.168075+0000","last_clean":"2024-09-10T16:10:52.168075+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:10:52.168075+0000","last_undegraded":"2024-09-10T16:10:52.168075+0000","last_fullsized":"2024-09-10T16:10:52.168075+0000","mapping_epoch":35,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18949,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20798","reported_seq":29508,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.169011+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:10:52.169011+0000","last_peered":"2024-09-10T16:10:52.169011+0000","last_clean":"2024-09-10T16:10:52.169011+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:10:52.169011+0000","last_undegraded":"2024-09-10T16:10:52.169011+0000","last_fullsized":"2024-09-10T16:10:52.169011+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18802,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20660","reported_seq":20772,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.167700+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:10:52.167700+0000","last_peered":"2024-09-10T16:10:52.167700+0000","last_clean":"2024-09-10T16:10:52.167700+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:10:52.167700+0000","last_undegraded":"2024-09-10T16:10:52.167700+0000","last_fullsized":"2024-09-10T16:10:52.167700+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20660,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20891","reported_seq":21055,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:11:16.223316+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:11:16.223316+0000","last_peered":"2024-09-10T16:11:16.223316+0000","last_clean":"2024-09-10T16:11:16.223316+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:11:16.223316+0000","last_undegraded":"2024-09-10T16:11:16.223316+0000","last_fullsized":"2024-09-10T16:11:16.223316+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17602,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'21061","reported_seq":21222,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.242289+0000","last_change":"2024-09-10T16:10:52.241702+0000","last_active":"2024-09-10T16:10:52.242289+0000","last_peered":"2024-09-10T16:10:52.242289+0000","last_clean":"2024-09-10T16:10:52.242289+0000","last_became_active":"2024-09-10T16:10:52.171236+0000","last_became_peered":"2024-09-10T16:10:52.171236+0000","last_unstale":"2024-09-10T16:10:52.242289+0000","last_undegraded":"2024-09-10T16:10:52.242289+0000","last_fullsized":"2024-09-10T16:10:52.242289+0000","mapping_epoch":58,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:04:46.778280+0000","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17673,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4372,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'21082","reported_seq":21253,"reported_epoch":60,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:11:16.222779+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:11:16.222779+0000","last_peered":"2024-09-10T16:11:16.222779+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:11:16.222779+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17581,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4235},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'21314","reported_seq":21478,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:11:16.222826+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:11:16.222826+0000","last_peered":"2024-09-10T16:11:16.222826+0000","last_clean":"2024-09-10T16:11:16.222826+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:11:16.222826+0000","last_undegraded":"2024-09-10T16:11:16.222826+0000","last_fullsized":"2024-09-10T16:11:16.222826+0000","mapping_epoch":34,"log_start":"45'11300","ondisk_log_start":"45'11300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18161,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.168728+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:10:52.168728+0000","last_peered":"2024-09-10T16:10:52.168728+0000","last_clean":"2024-09-10T16:10:52.168728+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:10:52.168728+0000","last_undegraded":"2024-09-10T16:10:52.168728+0000","last_fullsized":"2024-09-10T16:10:52.168728+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20457","reported_seq":20632,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.168901+0000","last_change":"2024-09-10T16:09:33.012840+0000","last_active":"2024-09-10T16:10:52.168901+0000","last_peered":"2024-09-10T16:10:52.168901+0000","last_clean":"2024-09-10T16:10:52.168901+0000","last_became_active":"2024-09-10T16:09:32.934056+0000","last_became_peered":"2024-09-10T16:09:32.934056+0000","last_unstale":"2024-09-10T16:10:52.168901+0000","last_undegraded":"2024-09-10T16:10:52.168901+0000","last_fullsized":"2024-09-10T16:10:52.168901+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:08:30.481235+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18276,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":4579,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":126,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.168004+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:10:52.168004+0000","last_peered":"2024-09-10T16:10:52.168004+0000","last_clean":"2024-09-10T16:10:52.168004+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:10:52.168004+0000","last_undegraded":"2024-09-10T16:10:52.168004+0000","last_fullsized":"2024-09-10T16:10:52.168004+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20910","reported_seq":29596,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:11:16.223112+0000","last_change":"2024-09-10T16:09:08.909580+0000","last_active":"2024-09-10T16:11:16.223112+0000","last_peered":"2024-09-10T16:11:16.223112+0000","last_clean":"2024-09-10T16:11:16.223112+0000","last_became_active":"2024-09-10T16:09:08.854941+0000","last_became_peered":"2024-09-10T16:09:08.854941+0000","last_unstale":"2024-09-10T16:11:16.223112+0000","last_undegraded":"2024-09-10T16:11:16.223112+0000","last_fullsized":"2024-09-10T16:11:16.223112+0000","mapping_epoch":50,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:13:14.846691+0000","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17414,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4257,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":138,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:11:16.222960+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:11:16.222960+0000","last_peered":"2024-09-10T16:11:16.222960+0000","last_clean":"2024-09-10T16:11:16.222960+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:11:16.222960+0000","last_undegraded":"2024-09-10T16:11:16.222960+0000","last_fullsized":"2024-09-10T16:11:16.222960+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20781","reported_seq":29566,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.168064+0000","last_change":"2024-09-10T16:10:20.907889+0000","last_active":"2024-09-10T16:10:52.168064+0000","last_peered":"2024-09-10T16:10:52.168064+0000","last_clean":"2024-09-10T16:10:52.168064+0000","last_became_active":"2024-09-10T16:10:20.852065+0000","last_became_peered":"2024-09-10T16:10:20.852065+0000","last_unstale":"2024-09-10T16:10:52.168064+0000","last_undegraded":"2024-09-10T16:10:52.168064+0000","last_fullsized":"2024-09-10T16:10:52.168064+0000","mapping_epoch":56,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:34.069549+0000","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18461,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4307,"num_bytes_recovered":8675,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":111,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.168792+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:10:52.168792+0000","last_peered":"2024-09-10T16:10:52.168792+0000","last_clean":"2024-09-10T16:10:52.168792+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:10:52.168792+0000","last_undegraded":"2024-09-10T16:10:52.168792+0000","last_fullsized":"2024-09-10T16:10:52.168792+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'21170","reported_seq":30051,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.167373+0000","last_change":"2024-09-10T16:10:50.728349+0000","last_active":"2024-09-10T16:10:52.167373+0000","last_peered":"2024-09-10T16:10:52.167373+0000","last_clean":"2024-09-10T16:10:52.167373+0000","last_became_active":"2024-09-10T16:08:10.860438+0000","last_became_peered":"2024-09-10T16:08:10.860438+0000","last_unstale":"2024-09-10T16:10:52.167373+0000","last_undegraded":"2024-09-10T16:10:52.167373+0000","last_fullsized":"2024-09-10T16:10:52.167373+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:38:13.781380+0000","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18724,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20838","reported_seq":21006,"reported_epoch":60,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:11:16.225225+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:11:16.225225+0000","last_peered":"2024-09-10T16:11:16.225225+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:11:16.225225+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17469,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.168992+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:10:52.168992+0000","last_peered":"2024-09-10T16:10:52.168992+0000","last_clean":"2024-09-10T16:10:52.168992+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:10:52.168992+0000","last_undegraded":"2024-09-10T16:10:52.168992+0000","last_fullsized":"2024-09-10T16:10:52.168992+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20598","reported_seq":29195,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.169145+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:10:52.169145+0000","last_peered":"2024-09-10T16:10:52.169145+0000","last_clean":"2024-09-10T16:10:52.169145+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:10:52.169145+0000","last_undegraded":"2024-09-10T16:10:52.169145+0000","last_fullsized":"2024-09-10T16:10:52.169145+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16988,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":126,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.167911+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:10:52.167911+0000","last_peered":"2024-09-10T16:10:52.167911+0000","last_clean":"2024-09-10T16:10:52.167911+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:10:52.167911+0000","last_undegraded":"2024-09-10T16:10:52.167911+0000","last_fullsized":"2024-09-10T16:10:52.167911+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'21048","reported_seq":27789,"reported_epoch":59,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:11:11.223560+0000","last_change":"2024-09-10T16:10:50.728566+0000","last_active":"2024-09-10T16:11:11.223560+0000","last_peered":"2024-09-10T16:11:11.223560+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:11:11.223560+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1090,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17303,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3283,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1090}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.169062+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:10:52.169062+0000","last_peered":"2024-09-10T16:10:52.169062+0000","last_clean":"2024-09-10T16:10:52.169062+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:10:52.169062+0000","last_undegraded":"2024-09-10T16:10:52.169062+0000","last_fullsized":"2024-09-10T16:10:52.169062+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":106,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.169114+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:10:52.169114+0000","last_peered":"2024-09-10T16:10:52.169114+0000","last_clean":"2024-09-10T16:10:52.169114+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:10:52.169114+0000","last_undegraded":"2024-09-10T16:10:52.169114+0000","last_fullsized":"2024-09-10T16:10:52.169114+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'21198","reported_seq":21403,"reported_epoch":60,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:11:16.230407+0000","last_change":"2024-09-10T16:11:16.084896+0000","last_active":"2024-09-10T16:11:16.230407+0000","last_peered":"2024-09-10T16:11:16.230407+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:11:16.230407+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4312,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17361,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4310}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":13944,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292096,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37446,"num_bytes_recovered":33097612,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":197984256,"data_stored":197875866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161022,"ondisk_log_size":161022,"up":32,"acting":28,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986335,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":490808,"kb_used_data":125928,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93881032,"statfs":{"total":96636764160,"available":96134176768,"internally_reserved":0,"allocated":128950272,"data_stored":128587904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659280,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":483420,"kb_used_data":524,"kb_used_omap":4,"kb_used_meta":482875,"kb_avail":93888420,"statfs":{"total":96636764160,"available":96141742080,"internally_reserved":0,"allocated":536576,"data_stored":170131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":494464686},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182563,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":547376,"kb_used_data":68700,"kb_used_omap":4,"kb_used_meta":478651,"kb_avail":93824464,"statfs":{"total":96636764160,"available":96076251136,"internally_reserved":0,"allocated":70348800,"data_stored":69982355,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":490139306},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":69853184,"data_stored":69817054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":4830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":128090112,"data_stored":128053982,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:11:20.454 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:11:20.804 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:11:20.804 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:11:20.817 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":476,"stamp":"2024-09-10T16:11:19.792338+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":13944,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292098,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37448,"num_bytes_recovered":33097631,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161025,"ondisk_log_size":161025,"up":48,"acting":44,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1521604,"kb_used_data":195152,"kb_used_omap":12,"kb_used_meta":1326387,"kb_avail":281593916,"statfs":{"total":289910292480,"available":288352169984,"internally_reserved":0,"allocated":199835648,"data_stored":198740390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1358220290},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-13,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001214"},"pg_stats":[{"pgid":"2.f","version":"49'20672","reported_seq":29306,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:11:16.225228+0000","last_change":"2024-09-10T16:09:57.088531+0000","last_active":"2024-09-10T16:11:16.225228+0000","last_peered":"2024-09-10T16:11:16.225228+0000","last_clean":"2024-09-10T16:11:16.225228+0000","last_became_active":"2024-09-10T16:09:57.005755+0000","last_became_peered":"2024-09-10T16:09:57.005755+0000","last_unstale":"2024-09-10T16:11:16.225228+0000","last_undegraded":"2024-09-10T16:11:16.225228+0000","last_fullsized":"2024-09-10T16:11:16.225228+0000","mapping_epoch":54,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:09:19.329697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20672,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4259,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20844","reported_seq":20988,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.168075+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:10:52.168075+0000","last_peered":"2024-09-10T16:10:52.168075+0000","last_clean":"2024-09-10T16:10:52.168075+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:10:52.168075+0000","last_undegraded":"2024-09-10T16:10:52.168075+0000","last_fullsized":"2024-09-10T16:10:52.168075+0000","mapping_epoch":35,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18949,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20798","reported_seq":29508,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.169011+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:10:52.169011+0000","last_peered":"2024-09-10T16:10:52.169011+0000","last_clean":"2024-09-10T16:10:52.169011+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:10:52.169011+0000","last_undegraded":"2024-09-10T16:10:52.169011+0000","last_fullsized":"2024-09-10T16:10:52.169011+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18802,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20660","reported_seq":20772,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.167700+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:10:52.167700+0000","last_peered":"2024-09-10T16:10:52.167700+0000","last_clean":"2024-09-10T16:10:52.167700+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:10:52.167700+0000","last_undegraded":"2024-09-10T16:10:52.167700+0000","last_fullsized":"2024-09-10T16:10:52.167700+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20660,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20891","reported_seq":21055,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:11:16.223316+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:11:16.223316+0000","last_peered":"2024-09-10T16:11:16.223316+0000","last_clean":"2024-09-10T16:11:16.223316+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:11:16.223316+0000","last_undegraded":"2024-09-10T16:11:16.223316+0000","last_fullsized":"2024-09-10T16:11:16.223316+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17602,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'21061","reported_seq":21222,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.242289+0000","last_change":"2024-09-10T16:10:52.241702+0000","last_active":"2024-09-10T16:10:52.242289+0000","last_peered":"2024-09-10T16:10:52.242289+0000","last_clean":"2024-09-10T16:10:52.242289+0000","last_became_active":"2024-09-10T16:10:52.171236+0000","last_became_peered":"2024-09-10T16:10:52.171236+0000","last_unstale":"2024-09-10T16:10:52.242289+0000","last_undegraded":"2024-09-10T16:10:52.242289+0000","last_fullsized":"2024-09-10T16:10:52.242289+0000","mapping_epoch":58,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:04:46.778280+0000","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17673,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4372,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'21082","reported_seq":21253,"reported_epoch":60,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:11:16.222779+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:11:16.222779+0000","last_peered":"2024-09-10T16:11:16.222779+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:11:16.222779+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17581,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4235},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'21314","reported_seq":21478,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:11:16.222826+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:11:16.222826+0000","last_peered":"2024-09-10T16:11:16.222826+0000","last_clean":"2024-09-10T16:11:16.222826+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:11:16.222826+0000","last_undegraded":"2024-09-10T16:11:16.222826+0000","last_fullsized":"2024-09-10T16:11:16.222826+0000","mapping_epoch":34,"log_start":"45'11300","ondisk_log_start":"45'11300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18161,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.168728+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:10:52.168728+0000","last_peered":"2024-09-10T16:10:52.168728+0000","last_clean":"2024-09-10T16:10:52.168728+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:10:52.168728+0000","last_undegraded":"2024-09-10T16:10:52.168728+0000","last_fullsized":"2024-09-10T16:10:52.168728+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20457","reported_seq":20632,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.168901+0000","last_change":"2024-09-10T16:09:33.012840+0000","last_active":"2024-09-10T16:10:52.168901+0000","last_peered":"2024-09-10T16:10:52.168901+0000","last_clean":"2024-09-10T16:10:52.168901+0000","last_became_active":"2024-09-10T16:09:32.934056+0000","last_became_peered":"2024-09-10T16:09:32.934056+0000","last_unstale":"2024-09-10T16:10:52.168901+0000","last_undegraded":"2024-09-10T16:10:52.168901+0000","last_fullsized":"2024-09-10T16:10:52.168901+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:08:30.481235+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18276,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":4579,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":126,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.168004+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:10:52.168004+0000","last_peered":"2024-09-10T16:10:52.168004+0000","last_clean":"2024-09-10T16:10:52.168004+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:10:52.168004+0000","last_undegraded":"2024-09-10T16:10:52.168004+0000","last_fullsized":"2024-09-10T16:10:52.168004+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20910","reported_seq":29596,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:11:16.223112+0000","last_change":"2024-09-10T16:09:08.909580+0000","last_active":"2024-09-10T16:11:16.223112+0000","last_peered":"2024-09-10T16:11:16.223112+0000","last_clean":"2024-09-10T16:11:16.223112+0000","last_became_active":"2024-09-10T16:09:08.854941+0000","last_became_peered":"2024-09-10T16:09:08.854941+0000","last_unstale":"2024-09-10T16:11:16.223112+0000","last_undegraded":"2024-09-10T16:11:16.223112+0000","last_fullsized":"2024-09-10T16:11:16.223112+0000","mapping_epoch":50,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:13:14.846691+0000","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17414,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4257,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":138,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:11:16.222960+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:11:16.222960+0000","last_peered":"2024-09-10T16:11:16.222960+0000","last_clean":"2024-09-10T16:11:16.222960+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:11:16.222960+0000","last_undegraded":"2024-09-10T16:11:16.222960+0000","last_fullsized":"2024-09-10T16:11:16.222960+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20781","reported_seq":29566,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.168064+0000","last_change":"2024-09-10T16:10:20.907889+0000","last_active":"2024-09-10T16:10:52.168064+0000","last_peered":"2024-09-10T16:10:52.168064+0000","last_clean":"2024-09-10T16:10:52.168064+0000","last_became_active":"2024-09-10T16:10:20.852065+0000","last_became_peered":"2024-09-10T16:10:20.852065+0000","last_unstale":"2024-09-10T16:10:52.168064+0000","last_undegraded":"2024-09-10T16:10:52.168064+0000","last_fullsized":"2024-09-10T16:10:52.168064+0000","mapping_epoch":56,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:34.069549+0000","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18461,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4307,"num_bytes_recovered":8675,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":111,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.168792+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:10:52.168792+0000","last_peered":"2024-09-10T16:10:52.168792+0000","last_clean":"2024-09-10T16:10:52.168792+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:10:52.168792+0000","last_undegraded":"2024-09-10T16:10:52.168792+0000","last_fullsized":"2024-09-10T16:10:52.168792+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'21170","reported_seq":30051,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.167373+0000","last_change":"2024-09-10T16:10:50.728349+0000","last_active":"2024-09-10T16:10:52.167373+0000","last_peered":"2024-09-10T16:10:52.167373+0000","last_clean":"2024-09-10T16:10:52.167373+0000","last_became_active":"2024-09-10T16:08:10.860438+0000","last_became_peered":"2024-09-10T16:08:10.860438+0000","last_unstale":"2024-09-10T16:10:52.167373+0000","last_undegraded":"2024-09-10T16:10:52.167373+0000","last_fullsized":"2024-09-10T16:10:52.167373+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:38:13.781380+0000","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18724,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20838","reported_seq":21006,"reported_epoch":60,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:11:16.225225+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:11:16.225225+0000","last_peered":"2024-09-10T16:11:16.225225+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:11:16.225225+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17469,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.168992+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:10:52.168992+0000","last_peered":"2024-09-10T16:10:52.168992+0000","last_clean":"2024-09-10T16:10:52.168992+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:10:52.168992+0000","last_undegraded":"2024-09-10T16:10:52.168992+0000","last_fullsized":"2024-09-10T16:10:52.168992+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20598","reported_seq":29195,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.169145+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:10:52.169145+0000","last_peered":"2024-09-10T16:10:52.169145+0000","last_clean":"2024-09-10T16:10:52.169145+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:10:52.169145+0000","last_undegraded":"2024-09-10T16:10:52.169145+0000","last_fullsized":"2024-09-10T16:10:52.169145+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16988,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":126,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.167911+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:10:52.167911+0000","last_peered":"2024-09-10T16:10:52.167911+0000","last_clean":"2024-09-10T16:10:52.167911+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:10:52.167911+0000","last_undegraded":"2024-09-10T16:10:52.167911+0000","last_fullsized":"2024-09-10T16:10:52.167911+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'21048","reported_seq":27789,"reported_epoch":59,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:11:11.223560+0000","last_change":"2024-09-10T16:10:50.728566+0000","last_active":"2024-09-10T16:11:11.223560+0000","last_peered":"2024-09-10T16:11:11.223560+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:11:11.223560+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1090,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17303,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3283,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1090}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.169062+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:10:52.169062+0000","last_peered":"2024-09-10T16:10:52.169062+0000","last_clean":"2024-09-10T16:10:52.169062+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:10:52.169062+0000","last_undegraded":"2024-09-10T16:10:52.169062+0000","last_fullsized":"2024-09-10T16:10:52.169062+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":106,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.169114+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:10:52.169114+0000","last_peered":"2024-09-10T16:10:52.169114+0000","last_clean":"2024-09-10T16:10:52.169114+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:10:52.169114+0000","last_undegraded":"2024-09-10T16:10:52.169114+0000","last_fullsized":"2024-09-10T16:10:52.169114+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'21198","reported_seq":21403,"reported_epoch":60,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:11:16.230407+0000","last_change":"2024-09-10T16:11:16.084896+0000","last_active":"2024-09-10T16:11:16.230407+0000","last_peered":"2024-09-10T16:11:16.230407+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:11:16.230407+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4312,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17361,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4310}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":13944,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292096,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37446,"num_bytes_recovered":33097612,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":197984256,"data_stored":197875866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161022,"ondisk_log_size":161022,"up":32,"acting":28,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986335,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":490808,"kb_used_data":125928,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93881032,"statfs":{"total":96636764160,"available":96134176768,"internally_reserved":0,"allocated":128950272,"data_stored":128587904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659280,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":483420,"kb_used_data":524,"kb_used_omap":4,"kb_used_meta":482875,"kb_avail":93888420,"statfs":{"total":96636764160,"available":96141742080,"internally_reserved":0,"allocated":536576,"data_stored":170131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":494464686},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182563,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":547376,"kb_used_data":68700,"kb_used_omap":4,"kb_used_meta":478651,"kb_avail":93824464,"statfs":{"total":96636764160,"available":96076251136,"internally_reserved":0,"allocated":70348800,"data_stored":69982355,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":490139306},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":69853184,"data_stored":69817054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":4830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":128090112,"data_stored":128053982,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:11:20.818 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:11:21.192 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:11:21.192 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:11:21.205 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":476,"stamp":"2024-09-10T16:11:19.792338+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":13944,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292098,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37448,"num_bytes_recovered":33097631,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161025,"ondisk_log_size":161025,"up":48,"acting":44,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1521604,"kb_used_data":195152,"kb_used_omap":12,"kb_used_meta":1326387,"kb_avail":281593916,"statfs":{"total":289910292480,"available":288352169984,"internally_reserved":0,"allocated":199835648,"data_stored":198740390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1358220290},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-13,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001214"},"pg_stats":[{"pgid":"2.f","version":"49'20672","reported_seq":29306,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:11:16.225228+0000","last_change":"2024-09-10T16:09:57.088531+0000","last_active":"2024-09-10T16:11:16.225228+0000","last_peered":"2024-09-10T16:11:16.225228+0000","last_clean":"2024-09-10T16:11:16.225228+0000","last_became_active":"2024-09-10T16:09:57.005755+0000","last_became_peered":"2024-09-10T16:09:57.005755+0000","last_unstale":"2024-09-10T16:11:16.225228+0000","last_undegraded":"2024-09-10T16:11:16.225228+0000","last_fullsized":"2024-09-10T16:11:16.225228+0000","mapping_epoch":54,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:09:19.329697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20672,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4259,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20844","reported_seq":20988,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.168075+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:10:52.168075+0000","last_peered":"2024-09-10T16:10:52.168075+0000","last_clean":"2024-09-10T16:10:52.168075+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:10:52.168075+0000","last_undegraded":"2024-09-10T16:10:52.168075+0000","last_fullsized":"2024-09-10T16:10:52.168075+0000","mapping_epoch":35,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18949,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20798","reported_seq":29508,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.169011+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:10:52.169011+0000","last_peered":"2024-09-10T16:10:52.169011+0000","last_clean":"2024-09-10T16:10:52.169011+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:10:52.169011+0000","last_undegraded":"2024-09-10T16:10:52.169011+0000","last_fullsized":"2024-09-10T16:10:52.169011+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18802,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20660","reported_seq":20772,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.167700+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:10:52.167700+0000","last_peered":"2024-09-10T16:10:52.167700+0000","last_clean":"2024-09-10T16:10:52.167700+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:10:52.167700+0000","last_undegraded":"2024-09-10T16:10:52.167700+0000","last_fullsized":"2024-09-10T16:10:52.167700+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20660,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20891","reported_seq":21055,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:11:16.223316+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:11:16.223316+0000","last_peered":"2024-09-10T16:11:16.223316+0000","last_clean":"2024-09-10T16:11:16.223316+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:11:16.223316+0000","last_undegraded":"2024-09-10T16:11:16.223316+0000","last_fullsized":"2024-09-10T16:11:16.223316+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17602,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'21061","reported_seq":21222,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.242289+0000","last_change":"2024-09-10T16:10:52.241702+0000","last_active":"2024-09-10T16:10:52.242289+0000","last_peered":"2024-09-10T16:10:52.242289+0000","last_clean":"2024-09-10T16:10:52.242289+0000","last_became_active":"2024-09-10T16:10:52.171236+0000","last_became_peered":"2024-09-10T16:10:52.171236+0000","last_unstale":"2024-09-10T16:10:52.242289+0000","last_undegraded":"2024-09-10T16:10:52.242289+0000","last_fullsized":"2024-09-10T16:10:52.242289+0000","mapping_epoch":58,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:04:46.778280+0000","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17673,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4372,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'21082","reported_seq":21253,"reported_epoch":60,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:11:16.222779+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:11:16.222779+0000","last_peered":"2024-09-10T16:11:16.222779+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:11:16.222779+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17581,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4235},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'21314","reported_seq":21478,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:11:16.222826+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:11:16.222826+0000","last_peered":"2024-09-10T16:11:16.222826+0000","last_clean":"2024-09-10T16:11:16.222826+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:11:16.222826+0000","last_undegraded":"2024-09-10T16:11:16.222826+0000","last_fullsized":"2024-09-10T16:11:16.222826+0000","mapping_epoch":34,"log_start":"45'11300","ondisk_log_start":"45'11300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18161,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.168728+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:10:52.168728+0000","last_peered":"2024-09-10T16:10:52.168728+0000","last_clean":"2024-09-10T16:10:52.168728+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:10:52.168728+0000","last_undegraded":"2024-09-10T16:10:52.168728+0000","last_fullsized":"2024-09-10T16:10:52.168728+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20457","reported_seq":20632,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.168901+0000","last_change":"2024-09-10T16:09:33.012840+0000","last_active":"2024-09-10T16:10:52.168901+0000","last_peered":"2024-09-10T16:10:52.168901+0000","last_clean":"2024-09-10T16:10:52.168901+0000","last_became_active":"2024-09-10T16:09:32.934056+0000","last_became_peered":"2024-09-10T16:09:32.934056+0000","last_unstale":"2024-09-10T16:10:52.168901+0000","last_undegraded":"2024-09-10T16:10:52.168901+0000","last_fullsized":"2024-09-10T16:10:52.168901+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:08:30.481235+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18276,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":4579,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":126,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.168004+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:10:52.168004+0000","last_peered":"2024-09-10T16:10:52.168004+0000","last_clean":"2024-09-10T16:10:52.168004+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:10:52.168004+0000","last_undegraded":"2024-09-10T16:10:52.168004+0000","last_fullsized":"2024-09-10T16:10:52.168004+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20910","reported_seq":29596,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:11:16.223112+0000","last_change":"2024-09-10T16:09:08.909580+0000","last_active":"2024-09-10T16:11:16.223112+0000","last_peered":"2024-09-10T16:11:16.223112+0000","last_clean":"2024-09-10T16:11:16.223112+0000","last_became_active":"2024-09-10T16:09:08.854941+0000","last_became_peered":"2024-09-10T16:09:08.854941+0000","last_unstale":"2024-09-10T16:11:16.223112+0000","last_undegraded":"2024-09-10T16:11:16.223112+0000","last_fullsized":"2024-09-10T16:11:16.223112+0000","mapping_epoch":50,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:13:14.846691+0000","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17414,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4257,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":138,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:11:16.222960+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:11:16.222960+0000","last_peered":"2024-09-10T16:11:16.222960+0000","last_clean":"2024-09-10T16:11:16.222960+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:11:16.222960+0000","last_undegraded":"2024-09-10T16:11:16.222960+0000","last_fullsized":"2024-09-10T16:11:16.222960+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20781","reported_seq":29566,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.168064+0000","last_change":"2024-09-10T16:10:20.907889+0000","last_active":"2024-09-10T16:10:52.168064+0000","last_peered":"2024-09-10T16:10:52.168064+0000","last_clean":"2024-09-10T16:10:52.168064+0000","last_became_active":"2024-09-10T16:10:20.852065+0000","last_became_peered":"2024-09-10T16:10:20.852065+0000","last_unstale":"2024-09-10T16:10:52.168064+0000","last_undegraded":"2024-09-10T16:10:52.168064+0000","last_fullsized":"2024-09-10T16:10:52.168064+0000","mapping_epoch":56,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:34.069549+0000","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18461,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4307,"num_bytes_recovered":8675,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":111,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.168792+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:10:52.168792+0000","last_peered":"2024-09-10T16:10:52.168792+0000","last_clean":"2024-09-10T16:10:52.168792+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:10:52.168792+0000","last_undegraded":"2024-09-10T16:10:52.168792+0000","last_fullsized":"2024-09-10T16:10:52.168792+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'21170","reported_seq":30051,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.167373+0000","last_change":"2024-09-10T16:10:50.728349+0000","last_active":"2024-09-10T16:10:52.167373+0000","last_peered":"2024-09-10T16:10:52.167373+0000","last_clean":"2024-09-10T16:10:52.167373+0000","last_became_active":"2024-09-10T16:08:10.860438+0000","last_became_peered":"2024-09-10T16:08:10.860438+0000","last_unstale":"2024-09-10T16:10:52.167373+0000","last_undegraded":"2024-09-10T16:10:52.167373+0000","last_fullsized":"2024-09-10T16:10:52.167373+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:38:13.781380+0000","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18724,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20838","reported_seq":21006,"reported_epoch":60,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:11:16.225225+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:11:16.225225+0000","last_peered":"2024-09-10T16:11:16.225225+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:11:16.225225+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17469,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.168992+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:10:52.168992+0000","last_peered":"2024-09-10T16:10:52.168992+0000","last_clean":"2024-09-10T16:10:52.168992+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:10:52.168992+0000","last_undegraded":"2024-09-10T16:10:52.168992+0000","last_fullsized":"2024-09-10T16:10:52.168992+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20598","reported_seq":29195,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.169145+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:10:52.169145+0000","last_peered":"2024-09-10T16:10:52.169145+0000","last_clean":"2024-09-10T16:10:52.169145+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:10:52.169145+0000","last_undegraded":"2024-09-10T16:10:52.169145+0000","last_fullsized":"2024-09-10T16:10:52.169145+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16988,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":126,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.167911+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:10:52.167911+0000","last_peered":"2024-09-10T16:10:52.167911+0000","last_clean":"2024-09-10T16:10:52.167911+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:10:52.167911+0000","last_undegraded":"2024-09-10T16:10:52.167911+0000","last_fullsized":"2024-09-10T16:10:52.167911+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'21048","reported_seq":27789,"reported_epoch":59,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:11:11.223560+0000","last_change":"2024-09-10T16:10:50.728566+0000","last_active":"2024-09-10T16:11:11.223560+0000","last_peered":"2024-09-10T16:11:11.223560+0000","last_clean":"2024-09-10T15:59:14.116363+0000","last_became_active":"2024-09-10T15:59:16.301244+0000","last_became_peered":"2024-09-10T15:59:16.301244+0000","last_unstale":"2024-09-10T16:11:11.223560+0000","last_undegraded":"2024-09-10T15:59:15.883434+0000","last_fullsized":"2024-09-10T15:59:15.772756+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1090,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17303,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3283,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1090}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.169062+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:10:52.169062+0000","last_peered":"2024-09-10T16:10:52.169062+0000","last_clean":"2024-09-10T16:10:52.169062+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:10:52.169062+0000","last_undegraded":"2024-09-10T16:10:52.169062+0000","last_fullsized":"2024-09-10T16:10:52.169062+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":106,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-10T16:10:52.169114+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:10:52.169114+0000","last_peered":"2024-09-10T16:10:52.169114+0000","last_clean":"2024-09-10T16:10:52.169114+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:10:52.169114+0000","last_undegraded":"2024-09-10T16:10:52.169114+0000","last_fullsized":"2024-09-10T16:10:52.169114+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'21198","reported_seq":21403,"reported_epoch":60,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:11:16.230407+0000","last_change":"2024-09-10T16:11:16.084896+0000","last_active":"2024-09-10T16:11:16.230407+0000","last_peered":"2024-09-10T16:11:16.230407+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:11:16.230407+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4312,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17361,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4310}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":13944,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292096,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37446,"num_bytes_recovered":33097612,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":197984256,"data_stored":197875866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161022,"ondisk_log_size":161022,"up":32,"acting":28,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986335,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":490808,"kb_used_data":125928,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93881032,"statfs":{"total":96636764160,"available":96134176768,"internally_reserved":0,"allocated":128950272,"data_stored":128587904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659280,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":483420,"kb_used_data":524,"kb_used_omap":4,"kb_used_meta":482875,"kb_avail":93888420,"statfs":{"total":96636764160,"available":96141742080,"internally_reserved":0,"allocated":536576,"data_stored":170131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":494464686},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182563,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":547376,"kb_used_data":68700,"kb_used_omap":4,"kb_used_meta":478651,"kb_avail":93824464,"statfs":{"total":96636764160,"available":96076251136,"internally_reserved":0,"allocated":70348800,"data_stored":69982355,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":490139306},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":69853184,"data_stored":69817054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":4830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":128090112,"data_stored":128053982,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:11:21.206 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-10T16:11:21.206 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "49'21082", 'reported_seq': 21253, 'reported_epoch': 60, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:11:16.222779+0000', 'last_change': '2024-09-10T15:59:16.331899+0000', 'last_active': '2024-09-10T16:11:16.222779+0000', 'last_peered': '2024-09-10T16:11:16.222779+0000', 'last_clean': '2024-09-10T15:59:14.139436+0000', 'last_became_active': '2024-09-10T15:59:16.318715+0000', 'last_became_peered': '2024-09-10T15:59:16.318715+0000', 'last_unstale': '2024-09-10T16:11:16.222779+0000', 'last_undegraded': '2024-09-10T15:59:15.877524+0000', 'last_fullsized': '2024-09-10T15:59:15.769042+0000', 'mapping_epoch': 42, 'log_start': "45'11000", 'ondisk_log_start': "45'11000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3537", 'last_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'objects_scrubbed': 3497, 'log_size': 10082, 'log_dups_size': 0, 'ondisk_log_size': 10082, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1224, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4237, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 17581, 'num_write_kb': 28172, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4235}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:11:21.207 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-10T16:11:21.207 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "49'20838", 'reported_seq': 21006, 'reported_epoch': 60, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:11:16.225225+0000', 'last_change': '2024-09-10T15:59:16.315641+0000', 'last_active': '2024-09-10T16:11:16.225225+0000', 'last_peered': '2024-09-10T16:11:16.225225+0000', 'last_clean': '2024-09-10T15:59:14.117001+0000', 'last_became_active': '2024-09-10T15:59:16.302245+0000', 'last_became_peered': '2024-09-10T15:59:16.302245+0000', 'last_unstale': '2024-09-10T16:11:16.225225+0000', 'last_undegraded': '2024-09-10T15:59:15.845012+0000', 'last_fullsized': '2024-09-10T15:59:15.770337+0000', 'mapping_epoch': 42, 'log_start': "45'10800", 'ondisk_log_start': "45'10800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3404", 'last_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'objects_scrubbed': 3367, 'log_size': 10038, 'log_dups_size': 0, 'ondisk_log_size': 10038, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 983, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17469, 'num_write_kb': 28204, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4304}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:11:21.207 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-10T16:11:21.207 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "49'21048", 'reported_seq': 27789, 'reported_epoch': 59, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:11:11.223560+0000', 'last_change': '2024-09-10T16:10:50.728566+0000', 'last_active': '2024-09-10T16:11:11.223560+0000', 'last_peered': '2024-09-10T16:11:11.223560+0000', 'last_clean': '2024-09-10T15:59:14.116363+0000', 'last_became_active': '2024-09-10T15:59:16.301244+0000', 'last_became_peered': '2024-09-10T15:59:16.301244+0000', 'last_unstale': '2024-09-10T16:11:11.223560+0000', 'last_undegraded': '2024-09-10T15:59:15.883434+0000', 'last_fullsized': '2024-09-10T15:59:15.772756+0000', 'mapping_epoch': 42, 'log_start': "45'11000", 'ondisk_log_start': "45'11000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3789", 'last_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:04.437170+0000', 'objects_scrubbed': 3741, 'log_size': 10048, 'log_dups_size': 0, 'ondisk_log_size': 10048, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1440, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1090, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17303, 'num_write_kb': 27124, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3283, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1090}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:11:21.207 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-10T16:11:21.207 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "49'21198", 'reported_seq': 21403, 'reported_epoch': 60, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:11:16.230407+0000', 'last_change': '2024-09-10T16:11:16.084896+0000', 'last_active': '2024-09-10T16:11:16.230407+0000', 'last_peered': '2024-09-10T16:11:16.230407+0000', 'last_clean': '2024-09-10T15:59:14.129780+0000', 'last_became_active': '2024-09-10T15:59:16.300932+0000', 'last_became_peered': '2024-09-10T15:59:16.300932+0000', 'last_unstale': '2024-09-10T16:11:16.230407+0000', 'last_undegraded': '2024-09-10T15:59:15.887823+0000', 'last_fullsized': '2024-09-10T15:59:15.768513+0000', 'mapping_epoch': 42, 'log_start': "45'11100", 'ondisk_log_start': "45'11100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3880", 'last_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'objects_scrubbed': 3827, 'log_size': 10098, 'log_dups_size': 0, 'ondisk_log_size': 10098, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1397, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4312, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17361, 'num_write_kb': 27072, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 13, 'num_bytes_recovered': 483, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4310}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:11:21.207 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T16:11:21.698 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:11:21.698 INFO:teuthology.orchestra.run.smithi115.stdout:{"fsid":"7effee29-df05-4351-9741-ee203564ed81","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 13944/24 objects degraded (58100.000%), 4 pgs degraded, 4 pgs undersized","count":8},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":845,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":61,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983954,"num_in_osds":3,"osd_in_since":1725983947,"num_remapped_pgs":3},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":20},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":2},{"state_name":"active+recovering+undersized+degraded+remapped","count":2}],"num_pgs":24,"num_pools":2,"num_objects":12,"data_bytes":4849,"bytes_used":1558122496,"bytes_avail":288352169984,"bytes_total":289910292480,"degraded_objects":13944,"degraded_total":24,"degraded_ratio":581,"recovering_objects_per_sec":1,"recovering_bytes_per_sec":60,"recovering_keys_per_sec":0,"num_objects_recovered":13,"num_bytes_recovered":483,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-10T15:57:14:967120+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T16:00:13.560776+0000","services":{}},"progress_events":{}} 2024-09-10T16:11:21.713 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-10T16:11:21.713 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:11:22.088 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:11:22.089 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:11:22.101 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":477,"stamp":"2024-09-10T16:11:21.792670+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":11989,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292098,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39403,"num_bytes_recovered":33097631,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161025,"ondisk_log_size":161025,"up":48,"acting":45,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1522932,"kb_used_data":192316,"kb_used_omap":12,"kb_used_meta":1330547,"kb_avail":281592588,"statfs":{"total":289910292480,"available":288350810112,"internally_reserved":0,"allocated":196931584,"data_stored":195811149,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1362480130},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1968,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1968,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001214"},"pg_stats":[{"pgid":"2.f","version":"49'20672","reported_seq":29308,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.232365+0000","last_change":"2024-09-10T16:09:57.088531+0000","last_active":"2024-09-10T16:11:17.232365+0000","last_peered":"2024-09-10T16:11:17.232365+0000","last_clean":"2024-09-10T16:11:17.232365+0000","last_became_active":"2024-09-10T16:09:57.005755+0000","last_became_peered":"2024-09-10T16:09:57.005755+0000","last_unstale":"2024-09-10T16:11:17.232365+0000","last_undegraded":"2024-09-10T16:11:17.232365+0000","last_fullsized":"2024-09-10T16:11:17.232365+0000","mapping_epoch":54,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:09:19.329697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20672,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4259,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20844","reported_seq":20992,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.232030+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:11:17.232030+0000","last_peered":"2024-09-10T16:11:17.232030+0000","last_clean":"2024-09-10T16:11:17.232030+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:11:17.232030+0000","last_undegraded":"2024-09-10T16:11:17.232030+0000","last_fullsized":"2024-09-10T16:11:17.232030+0000","mapping_epoch":35,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18949,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20798","reported_seq":29512,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.234237+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:11:17.234237+0000","last_peered":"2024-09-10T16:11:17.234237+0000","last_clean":"2024-09-10T16:11:17.234237+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:11:17.234237+0000","last_undegraded":"2024-09-10T16:11:17.234237+0000","last_fullsized":"2024-09-10T16:11:17.234237+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18802,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20660","reported_seq":20776,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.231774+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:11:17.231774+0000","last_peered":"2024-09-10T16:11:17.231774+0000","last_clean":"2024-09-10T16:11:17.231774+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:11:17.231774+0000","last_undegraded":"2024-09-10T16:11:17.231774+0000","last_fullsized":"2024-09-10T16:11:17.231774+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20660,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20891","reported_seq":21057,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.231216+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:11:17.231216+0000","last_peered":"2024-09-10T16:11:17.231216+0000","last_clean":"2024-09-10T16:11:17.231216+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:11:17.231216+0000","last_undegraded":"2024-09-10T16:11:17.231216+0000","last_fullsized":"2024-09-10T16:11:17.231216+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17602,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'21061","reported_seq":21226,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.233215+0000","last_change":"2024-09-10T16:10:52.241702+0000","last_active":"2024-09-10T16:11:17.233215+0000","last_peered":"2024-09-10T16:11:17.233215+0000","last_clean":"2024-09-10T16:11:17.233215+0000","last_became_active":"2024-09-10T16:10:52.171236+0000","last_became_peered":"2024-09-10T16:10:52.171236+0000","last_unstale":"2024-09-10T16:11:17.233215+0000","last_undegraded":"2024-09-10T16:11:17.233215+0000","last_fullsized":"2024-09-10T16:11:17.233215+0000","mapping_epoch":58,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:04:46.778280+0000","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17673,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4372,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'21082","reported_seq":21256,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:11:17.231941+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:11:17.231941+0000","last_peered":"2024-09-10T16:11:17.231941+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:11:17.231941+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17581,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4235},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'21314","reported_seq":21480,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.231025+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:11:17.231025+0000","last_peered":"2024-09-10T16:11:17.231025+0000","last_clean":"2024-09-10T16:11:17.231025+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:11:17.231025+0000","last_undegraded":"2024-09-10T16:11:17.231025+0000","last_fullsized":"2024-09-10T16:11:17.231025+0000","mapping_epoch":34,"log_start":"45'11300","ondisk_log_start":"45'11300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18161,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.232985+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:11:17.232985+0000","last_peered":"2024-09-10T16:11:17.232985+0000","last_clean":"2024-09-10T16:11:17.232985+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:11:17.232985+0000","last_undegraded":"2024-09-10T16:11:17.232985+0000","last_fullsized":"2024-09-10T16:11:17.232985+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20457","reported_seq":20636,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.233353+0000","last_change":"2024-09-10T16:09:33.012840+0000","last_active":"2024-09-10T16:11:17.233353+0000","last_peered":"2024-09-10T16:11:17.233353+0000","last_clean":"2024-09-10T16:11:17.233353+0000","last_became_active":"2024-09-10T16:09:32.934056+0000","last_became_peered":"2024-09-10T16:09:32.934056+0000","last_unstale":"2024-09-10T16:11:17.233353+0000","last_undegraded":"2024-09-10T16:11:17.233353+0000","last_fullsized":"2024-09-10T16:11:17.233353+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:08:30.481235+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18276,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":4579,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":130,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.232408+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:11:17.232408+0000","last_peered":"2024-09-10T16:11:17.232408+0000","last_clean":"2024-09-10T16:11:17.232408+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:11:17.232408+0000","last_undegraded":"2024-09-10T16:11:17.232408+0000","last_fullsized":"2024-09-10T16:11:17.232408+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20910","reported_seq":29598,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.230962+0000","last_change":"2024-09-10T16:09:08.909580+0000","last_active":"2024-09-10T16:11:17.230962+0000","last_peered":"2024-09-10T16:11:17.230962+0000","last_clean":"2024-09-10T16:11:17.230962+0000","last_became_active":"2024-09-10T16:09:08.854941+0000","last_became_peered":"2024-09-10T16:09:08.854941+0000","last_unstale":"2024-09-10T16:11:17.230962+0000","last_undegraded":"2024-09-10T16:11:17.230962+0000","last_fullsized":"2024-09-10T16:11:17.230962+0000","mapping_epoch":50,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:13:14.846691+0000","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17414,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4257,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":140,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.232615+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:11:17.232615+0000","last_peered":"2024-09-10T16:11:17.232615+0000","last_clean":"2024-09-10T16:11:17.232615+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:11:17.232615+0000","last_undegraded":"2024-09-10T16:11:17.232615+0000","last_fullsized":"2024-09-10T16:11:17.232615+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20781","reported_seq":29570,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.232496+0000","last_change":"2024-09-10T16:10:20.907889+0000","last_active":"2024-09-10T16:11:17.232496+0000","last_peered":"2024-09-10T16:11:17.232496+0000","last_clean":"2024-09-10T16:11:17.232496+0000","last_became_active":"2024-09-10T16:10:20.852065+0000","last_became_peered":"2024-09-10T16:10:20.852065+0000","last_unstale":"2024-09-10T16:11:17.232496+0000","last_undegraded":"2024-09-10T16:11:17.232496+0000","last_fullsized":"2024-09-10T16:11:17.232496+0000","mapping_epoch":56,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:34.069549+0000","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18461,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4307,"num_bytes_recovered":8675,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":115,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.233348+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:11:17.233348+0000","last_peered":"2024-09-10T16:11:17.233348+0000","last_clean":"2024-09-10T16:11:17.233348+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:11:17.233348+0000","last_undegraded":"2024-09-10T16:11:17.233348+0000","last_fullsized":"2024-09-10T16:11:17.233348+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'21170","reported_seq":30055,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.231608+0000","last_change":"2024-09-10T16:10:50.728349+0000","last_active":"2024-09-10T16:11:17.231608+0000","last_peered":"2024-09-10T16:11:17.231608+0000","last_clean":"2024-09-10T16:11:17.231608+0000","last_became_active":"2024-09-10T16:08:10.860438+0000","last_became_peered":"2024-09-10T16:08:10.860438+0000","last_unstale":"2024-09-10T16:11:17.231608+0000","last_undegraded":"2024-09-10T16:11:17.231608+0000","last_fullsized":"2024-09-10T16:11:17.231608+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:38:13.781380+0000","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18724,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20838","reported_seq":21009,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:11:17.232728+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:11:17.232728+0000","last_peered":"2024-09-10T16:11:17.232728+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:11:17.232728+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17469,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":105,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.233051+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:11:17.233051+0000","last_peered":"2024-09-10T16:11:17.233051+0000","last_clean":"2024-09-10T16:11:17.233051+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:11:17.233051+0000","last_undegraded":"2024-09-10T16:11:17.233051+0000","last_fullsized":"2024-09-10T16:11:17.233051+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20598","reported_seq":29199,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.233507+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:11:17.233507+0000","last_peered":"2024-09-10T16:11:17.233507+0000","last_clean":"2024-09-10T16:11:17.233507+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:11:17.233507+0000","last_undegraded":"2024-09-10T16:11:17.233507+0000","last_fullsized":"2024-09-10T16:11:17.233507+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16988,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":130,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.232065+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:11:17.232065+0000","last_peered":"2024-09-10T16:11:17.232065+0000","last_clean":"2024-09-10T16:11:17.232065+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:11:17.232065+0000","last_undegraded":"2024-09-10T16:11:17.232065+0000","last_fullsized":"2024-09-10T16:11:17.232065+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'21048","reported_seq":21210,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.319438+0000","last_change":"2024-09-10T16:11:17.315372+0000","last_active":"2024-09-10T16:11:17.319438+0000","last_peered":"2024-09-10T16:11:17.319438+0000","last_clean":"2024-09-10T16:11:17.319438+0000","last_became_active":"2024-09-10T16:11:17.234487+0000","last_became_peered":"2024-09-10T16:11:17.234487+0000","last_unstale":"2024-09-10T16:11:17.319438+0000","last_undegraded":"2024-09-10T16:11:17.319438+0000","last_fullsized":"2024-09-10T16:11:17.319438+0000","mapping_epoch":60,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:19:00.114196+0000","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17303,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4373,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":105,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.233467+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:11:17.233467+0000","last_peered":"2024-09-10T16:11:17.233467+0000","last_clean":"2024-09-10T16:11:17.233467+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:11:17.233467+0000","last_undegraded":"2024-09-10T16:11:17.233467+0000","last_fullsized":"2024-09-10T16:11:17.233467+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":110,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.233556+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:11:17.233556+0000","last_peered":"2024-09-10T16:11:17.233556+0000","last_clean":"2024-09-10T16:11:17.233556+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:11:17.233556+0000","last_undegraded":"2024-09-10T16:11:17.233556+0000","last_fullsized":"2024-09-10T16:11:17.233556+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'21198","reported_seq":23135,"reported_epoch":61,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:11:21.233487+0000","last_change":"2024-09-10T16:11:16.084896+0000","last_active":"2024-09-10T16:11:21.233487+0000","last_peered":"2024-09-10T16:11:21.233487+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:11:21.233487+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17361,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":878,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3445}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":11989,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292096,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39401,"num_bytes_recovered":33097612,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":195031040,"data_stored":194929876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161022,"ondisk_log_size":161022,"up":32,"acting":29,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986336,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487948,"kb_used_data":123068,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93883892,"statfs":{"total":96636764160,"available":96137105408,"internally_reserved":0,"allocated":126021632,"data_stored":125648463,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659281,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487588,"kb_used_data":532,"kb_used_omap":4,"kb_used_meta":487035,"kb_avail":93884252,"statfs":{"total":96636764160,"available":96137474048,"internally_reserved":0,"allocated":544768,"data_stored":174748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":498724526},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182564,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":547396,"kb_used_data":68716,"kb_used_omap":4,"kb_used_meta":478651,"kb_avail":93824444,"statfs":{"total":96636764160,"available":96076230656,"internally_reserved":0,"allocated":70365184,"data_stored":69987938,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":490139306},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":69853184,"data_stored":69817054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":125145088,"data_stored":125108958,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:11:25.104 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:11:25.413 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:11:25.414 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:11:25.426 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":478,"stamp":"2024-09-10T16:11:23.793086+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":11989,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292098,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39403,"num_bytes_recovered":33097631,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161025,"ondisk_log_size":161025,"up":48,"acting":45,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1522932,"kb_used_data":192316,"kb_used_omap":12,"kb_used_meta":1330547,"kb_avail":281592588,"statfs":{"total":289910292480,"available":288350810112,"internally_reserved":0,"allocated":196931584,"data_stored":195811149,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1362480130},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1968,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1968,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001356"},"pg_stats":[{"pgid":"2.f","version":"49'20672","reported_seq":29308,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.232365+0000","last_change":"2024-09-10T16:09:57.088531+0000","last_active":"2024-09-10T16:11:17.232365+0000","last_peered":"2024-09-10T16:11:17.232365+0000","last_clean":"2024-09-10T16:11:17.232365+0000","last_became_active":"2024-09-10T16:09:57.005755+0000","last_became_peered":"2024-09-10T16:09:57.005755+0000","last_unstale":"2024-09-10T16:11:17.232365+0000","last_undegraded":"2024-09-10T16:11:17.232365+0000","last_fullsized":"2024-09-10T16:11:17.232365+0000","mapping_epoch":54,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:09:19.329697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20672,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4259,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20844","reported_seq":20992,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.232030+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:11:17.232030+0000","last_peered":"2024-09-10T16:11:17.232030+0000","last_clean":"2024-09-10T16:11:17.232030+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:11:17.232030+0000","last_undegraded":"2024-09-10T16:11:17.232030+0000","last_fullsized":"2024-09-10T16:11:17.232030+0000","mapping_epoch":35,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18949,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20798","reported_seq":29512,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.234237+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:11:17.234237+0000","last_peered":"2024-09-10T16:11:17.234237+0000","last_clean":"2024-09-10T16:11:17.234237+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:11:17.234237+0000","last_undegraded":"2024-09-10T16:11:17.234237+0000","last_fullsized":"2024-09-10T16:11:17.234237+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18802,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20660","reported_seq":20776,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.231774+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:11:17.231774+0000","last_peered":"2024-09-10T16:11:17.231774+0000","last_clean":"2024-09-10T16:11:17.231774+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:11:17.231774+0000","last_undegraded":"2024-09-10T16:11:17.231774+0000","last_fullsized":"2024-09-10T16:11:17.231774+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20660,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20891","reported_seq":21057,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.231216+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:11:17.231216+0000","last_peered":"2024-09-10T16:11:17.231216+0000","last_clean":"2024-09-10T16:11:17.231216+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:11:17.231216+0000","last_undegraded":"2024-09-10T16:11:17.231216+0000","last_fullsized":"2024-09-10T16:11:17.231216+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17602,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'21061","reported_seq":21226,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.233215+0000","last_change":"2024-09-10T16:10:52.241702+0000","last_active":"2024-09-10T16:11:17.233215+0000","last_peered":"2024-09-10T16:11:17.233215+0000","last_clean":"2024-09-10T16:11:17.233215+0000","last_became_active":"2024-09-10T16:10:52.171236+0000","last_became_peered":"2024-09-10T16:10:52.171236+0000","last_unstale":"2024-09-10T16:11:17.233215+0000","last_undegraded":"2024-09-10T16:11:17.233215+0000","last_fullsized":"2024-09-10T16:11:17.233215+0000","mapping_epoch":58,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:04:46.778280+0000","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17673,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4372,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'21082","reported_seq":21256,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:11:17.231941+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:11:17.231941+0000","last_peered":"2024-09-10T16:11:17.231941+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:11:17.231941+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17581,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4235},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'21314","reported_seq":21480,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.231025+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:11:17.231025+0000","last_peered":"2024-09-10T16:11:17.231025+0000","last_clean":"2024-09-10T16:11:17.231025+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:11:17.231025+0000","last_undegraded":"2024-09-10T16:11:17.231025+0000","last_fullsized":"2024-09-10T16:11:17.231025+0000","mapping_epoch":34,"log_start":"45'11300","ondisk_log_start":"45'11300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18161,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.232985+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:11:17.232985+0000","last_peered":"2024-09-10T16:11:17.232985+0000","last_clean":"2024-09-10T16:11:17.232985+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:11:17.232985+0000","last_undegraded":"2024-09-10T16:11:17.232985+0000","last_fullsized":"2024-09-10T16:11:17.232985+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20457","reported_seq":20636,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.233353+0000","last_change":"2024-09-10T16:09:33.012840+0000","last_active":"2024-09-10T16:11:17.233353+0000","last_peered":"2024-09-10T16:11:17.233353+0000","last_clean":"2024-09-10T16:11:17.233353+0000","last_became_active":"2024-09-10T16:09:32.934056+0000","last_became_peered":"2024-09-10T16:09:32.934056+0000","last_unstale":"2024-09-10T16:11:17.233353+0000","last_undegraded":"2024-09-10T16:11:17.233353+0000","last_fullsized":"2024-09-10T16:11:17.233353+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:08:30.481235+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18276,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":4579,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":130,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.232408+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:11:17.232408+0000","last_peered":"2024-09-10T16:11:17.232408+0000","last_clean":"2024-09-10T16:11:17.232408+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:11:17.232408+0000","last_undegraded":"2024-09-10T16:11:17.232408+0000","last_fullsized":"2024-09-10T16:11:17.232408+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20910","reported_seq":29598,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.230962+0000","last_change":"2024-09-10T16:09:08.909580+0000","last_active":"2024-09-10T16:11:17.230962+0000","last_peered":"2024-09-10T16:11:17.230962+0000","last_clean":"2024-09-10T16:11:17.230962+0000","last_became_active":"2024-09-10T16:09:08.854941+0000","last_became_peered":"2024-09-10T16:09:08.854941+0000","last_unstale":"2024-09-10T16:11:17.230962+0000","last_undegraded":"2024-09-10T16:11:17.230962+0000","last_fullsized":"2024-09-10T16:11:17.230962+0000","mapping_epoch":50,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:13:14.846691+0000","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17414,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4257,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":140,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.232615+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:11:17.232615+0000","last_peered":"2024-09-10T16:11:17.232615+0000","last_clean":"2024-09-10T16:11:17.232615+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:11:17.232615+0000","last_undegraded":"2024-09-10T16:11:17.232615+0000","last_fullsized":"2024-09-10T16:11:17.232615+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20781","reported_seq":29570,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.232496+0000","last_change":"2024-09-10T16:10:20.907889+0000","last_active":"2024-09-10T16:11:17.232496+0000","last_peered":"2024-09-10T16:11:17.232496+0000","last_clean":"2024-09-10T16:11:17.232496+0000","last_became_active":"2024-09-10T16:10:20.852065+0000","last_became_peered":"2024-09-10T16:10:20.852065+0000","last_unstale":"2024-09-10T16:11:17.232496+0000","last_undegraded":"2024-09-10T16:11:17.232496+0000","last_fullsized":"2024-09-10T16:11:17.232496+0000","mapping_epoch":56,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:34.069549+0000","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18461,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4307,"num_bytes_recovered":8675,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":115,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.233348+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:11:17.233348+0000","last_peered":"2024-09-10T16:11:17.233348+0000","last_clean":"2024-09-10T16:11:17.233348+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:11:17.233348+0000","last_undegraded":"2024-09-10T16:11:17.233348+0000","last_fullsized":"2024-09-10T16:11:17.233348+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'21170","reported_seq":30055,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.231608+0000","last_change":"2024-09-10T16:10:50.728349+0000","last_active":"2024-09-10T16:11:17.231608+0000","last_peered":"2024-09-10T16:11:17.231608+0000","last_clean":"2024-09-10T16:11:17.231608+0000","last_became_active":"2024-09-10T16:08:10.860438+0000","last_became_peered":"2024-09-10T16:08:10.860438+0000","last_unstale":"2024-09-10T16:11:17.231608+0000","last_undegraded":"2024-09-10T16:11:17.231608+0000","last_fullsized":"2024-09-10T16:11:17.231608+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:38:13.781380+0000","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18724,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20838","reported_seq":21009,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:11:17.232728+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:11:17.232728+0000","last_peered":"2024-09-10T16:11:17.232728+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:11:17.232728+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17469,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":105,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.233051+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:11:17.233051+0000","last_peered":"2024-09-10T16:11:17.233051+0000","last_clean":"2024-09-10T16:11:17.233051+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:11:17.233051+0000","last_undegraded":"2024-09-10T16:11:17.233051+0000","last_fullsized":"2024-09-10T16:11:17.233051+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20598","reported_seq":29199,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.233507+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:11:17.233507+0000","last_peered":"2024-09-10T16:11:17.233507+0000","last_clean":"2024-09-10T16:11:17.233507+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:11:17.233507+0000","last_undegraded":"2024-09-10T16:11:17.233507+0000","last_fullsized":"2024-09-10T16:11:17.233507+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16988,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":130,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.232065+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:11:17.232065+0000","last_peered":"2024-09-10T16:11:17.232065+0000","last_clean":"2024-09-10T16:11:17.232065+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:11:17.232065+0000","last_undegraded":"2024-09-10T16:11:17.232065+0000","last_fullsized":"2024-09-10T16:11:17.232065+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'21048","reported_seq":21210,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.319438+0000","last_change":"2024-09-10T16:11:17.315372+0000","last_active":"2024-09-10T16:11:17.319438+0000","last_peered":"2024-09-10T16:11:17.319438+0000","last_clean":"2024-09-10T16:11:17.319438+0000","last_became_active":"2024-09-10T16:11:17.234487+0000","last_became_peered":"2024-09-10T16:11:17.234487+0000","last_unstale":"2024-09-10T16:11:17.319438+0000","last_undegraded":"2024-09-10T16:11:17.319438+0000","last_fullsized":"2024-09-10T16:11:17.319438+0000","mapping_epoch":60,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:19:00.114196+0000","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17303,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4373,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":105,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.233467+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:11:17.233467+0000","last_peered":"2024-09-10T16:11:17.233467+0000","last_clean":"2024-09-10T16:11:17.233467+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:11:17.233467+0000","last_undegraded":"2024-09-10T16:11:17.233467+0000","last_fullsized":"2024-09-10T16:11:17.233467+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":110,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.233556+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:11:17.233556+0000","last_peered":"2024-09-10T16:11:17.233556+0000","last_clean":"2024-09-10T16:11:17.233556+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:11:17.233556+0000","last_undegraded":"2024-09-10T16:11:17.233556+0000","last_fullsized":"2024-09-10T16:11:17.233556+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'21198","reported_seq":23135,"reported_epoch":61,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:11:21.233487+0000","last_change":"2024-09-10T16:11:16.084896+0000","last_active":"2024-09-10T16:11:21.233487+0000","last_peered":"2024-09-10T16:11:21.233487+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:11:21.233487+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17361,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":878,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3445}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":11989,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292096,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39401,"num_bytes_recovered":33097612,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":195031040,"data_stored":194929876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161022,"ondisk_log_size":161022,"up":32,"acting":29,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986336,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487948,"kb_used_data":123068,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93883892,"statfs":{"total":96636764160,"available":96137105408,"internally_reserved":0,"allocated":126021632,"data_stored":125648463,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659281,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487588,"kb_used_data":532,"kb_used_omap":4,"kb_used_meta":487035,"kb_avail":93884252,"statfs":{"total":96636764160,"available":96137474048,"internally_reserved":0,"allocated":544768,"data_stored":174748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":498724526},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182564,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":547396,"kb_used_data":68716,"kb_used_omap":4,"kb_used_meta":478651,"kb_avail":93824444,"statfs":{"total":96636764160,"available":96076230656,"internally_reserved":0,"allocated":70365184,"data_stored":69987938,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":490139306},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":69853184,"data_stored":69817054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":125145088,"data_stored":125108958,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:11:25.427 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:11:25.729 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:11:25.730 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:11:25.743 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":478,"stamp":"2024-09-10T16:11:23.793086+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":11989,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292098,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39403,"num_bytes_recovered":33097631,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161025,"ondisk_log_size":161025,"up":48,"acting":45,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1522932,"kb_used_data":192316,"kb_used_omap":12,"kb_used_meta":1330547,"kb_avail":281592588,"statfs":{"total":289910292480,"available":288350810112,"internally_reserved":0,"allocated":196931584,"data_stored":195811149,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1362480130},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1968,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1968,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001356"},"pg_stats":[{"pgid":"2.f","version":"49'20672","reported_seq":29308,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.232365+0000","last_change":"2024-09-10T16:09:57.088531+0000","last_active":"2024-09-10T16:11:17.232365+0000","last_peered":"2024-09-10T16:11:17.232365+0000","last_clean":"2024-09-10T16:11:17.232365+0000","last_became_active":"2024-09-10T16:09:57.005755+0000","last_became_peered":"2024-09-10T16:09:57.005755+0000","last_unstale":"2024-09-10T16:11:17.232365+0000","last_undegraded":"2024-09-10T16:11:17.232365+0000","last_fullsized":"2024-09-10T16:11:17.232365+0000","mapping_epoch":54,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:09:19.329697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20672,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4259,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20844","reported_seq":20992,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.232030+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:11:17.232030+0000","last_peered":"2024-09-10T16:11:17.232030+0000","last_clean":"2024-09-10T16:11:17.232030+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:11:17.232030+0000","last_undegraded":"2024-09-10T16:11:17.232030+0000","last_fullsized":"2024-09-10T16:11:17.232030+0000","mapping_epoch":35,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18949,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20798","reported_seq":29512,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.234237+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:11:17.234237+0000","last_peered":"2024-09-10T16:11:17.234237+0000","last_clean":"2024-09-10T16:11:17.234237+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:11:17.234237+0000","last_undegraded":"2024-09-10T16:11:17.234237+0000","last_fullsized":"2024-09-10T16:11:17.234237+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18802,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20660","reported_seq":20776,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.231774+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:11:17.231774+0000","last_peered":"2024-09-10T16:11:17.231774+0000","last_clean":"2024-09-10T16:11:17.231774+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:11:17.231774+0000","last_undegraded":"2024-09-10T16:11:17.231774+0000","last_fullsized":"2024-09-10T16:11:17.231774+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20660,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20891","reported_seq":21057,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.231216+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:11:17.231216+0000","last_peered":"2024-09-10T16:11:17.231216+0000","last_clean":"2024-09-10T16:11:17.231216+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:11:17.231216+0000","last_undegraded":"2024-09-10T16:11:17.231216+0000","last_fullsized":"2024-09-10T16:11:17.231216+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17602,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'21061","reported_seq":21226,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.233215+0000","last_change":"2024-09-10T16:10:52.241702+0000","last_active":"2024-09-10T16:11:17.233215+0000","last_peered":"2024-09-10T16:11:17.233215+0000","last_clean":"2024-09-10T16:11:17.233215+0000","last_became_active":"2024-09-10T16:10:52.171236+0000","last_became_peered":"2024-09-10T16:10:52.171236+0000","last_unstale":"2024-09-10T16:11:17.233215+0000","last_undegraded":"2024-09-10T16:11:17.233215+0000","last_fullsized":"2024-09-10T16:11:17.233215+0000","mapping_epoch":58,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:04:46.778280+0000","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17673,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4372,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'21082","reported_seq":21256,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:11:17.231941+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:11:17.231941+0000","last_peered":"2024-09-10T16:11:17.231941+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:11:17.231941+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17581,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4235},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'21314","reported_seq":21480,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.231025+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:11:17.231025+0000","last_peered":"2024-09-10T16:11:17.231025+0000","last_clean":"2024-09-10T16:11:17.231025+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:11:17.231025+0000","last_undegraded":"2024-09-10T16:11:17.231025+0000","last_fullsized":"2024-09-10T16:11:17.231025+0000","mapping_epoch":34,"log_start":"45'11300","ondisk_log_start":"45'11300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18161,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.232985+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:11:17.232985+0000","last_peered":"2024-09-10T16:11:17.232985+0000","last_clean":"2024-09-10T16:11:17.232985+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:11:17.232985+0000","last_undegraded":"2024-09-10T16:11:17.232985+0000","last_fullsized":"2024-09-10T16:11:17.232985+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20457","reported_seq":20636,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.233353+0000","last_change":"2024-09-10T16:09:33.012840+0000","last_active":"2024-09-10T16:11:17.233353+0000","last_peered":"2024-09-10T16:11:17.233353+0000","last_clean":"2024-09-10T16:11:17.233353+0000","last_became_active":"2024-09-10T16:09:32.934056+0000","last_became_peered":"2024-09-10T16:09:32.934056+0000","last_unstale":"2024-09-10T16:11:17.233353+0000","last_undegraded":"2024-09-10T16:11:17.233353+0000","last_fullsized":"2024-09-10T16:11:17.233353+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:08:30.481235+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18276,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":4579,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":130,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.232408+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:11:17.232408+0000","last_peered":"2024-09-10T16:11:17.232408+0000","last_clean":"2024-09-10T16:11:17.232408+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:11:17.232408+0000","last_undegraded":"2024-09-10T16:11:17.232408+0000","last_fullsized":"2024-09-10T16:11:17.232408+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20910","reported_seq":29598,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.230962+0000","last_change":"2024-09-10T16:09:08.909580+0000","last_active":"2024-09-10T16:11:17.230962+0000","last_peered":"2024-09-10T16:11:17.230962+0000","last_clean":"2024-09-10T16:11:17.230962+0000","last_became_active":"2024-09-10T16:09:08.854941+0000","last_became_peered":"2024-09-10T16:09:08.854941+0000","last_unstale":"2024-09-10T16:11:17.230962+0000","last_undegraded":"2024-09-10T16:11:17.230962+0000","last_fullsized":"2024-09-10T16:11:17.230962+0000","mapping_epoch":50,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:13:14.846691+0000","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17414,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4257,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":140,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.232615+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:11:17.232615+0000","last_peered":"2024-09-10T16:11:17.232615+0000","last_clean":"2024-09-10T16:11:17.232615+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:11:17.232615+0000","last_undegraded":"2024-09-10T16:11:17.232615+0000","last_fullsized":"2024-09-10T16:11:17.232615+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20781","reported_seq":29570,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.232496+0000","last_change":"2024-09-10T16:10:20.907889+0000","last_active":"2024-09-10T16:11:17.232496+0000","last_peered":"2024-09-10T16:11:17.232496+0000","last_clean":"2024-09-10T16:11:17.232496+0000","last_became_active":"2024-09-10T16:10:20.852065+0000","last_became_peered":"2024-09-10T16:10:20.852065+0000","last_unstale":"2024-09-10T16:11:17.232496+0000","last_undegraded":"2024-09-10T16:11:17.232496+0000","last_fullsized":"2024-09-10T16:11:17.232496+0000","mapping_epoch":56,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:34.069549+0000","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18461,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4307,"num_bytes_recovered":8675,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":115,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.233348+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:11:17.233348+0000","last_peered":"2024-09-10T16:11:17.233348+0000","last_clean":"2024-09-10T16:11:17.233348+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:11:17.233348+0000","last_undegraded":"2024-09-10T16:11:17.233348+0000","last_fullsized":"2024-09-10T16:11:17.233348+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'21170","reported_seq":30055,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.231608+0000","last_change":"2024-09-10T16:10:50.728349+0000","last_active":"2024-09-10T16:11:17.231608+0000","last_peered":"2024-09-10T16:11:17.231608+0000","last_clean":"2024-09-10T16:11:17.231608+0000","last_became_active":"2024-09-10T16:08:10.860438+0000","last_became_peered":"2024-09-10T16:08:10.860438+0000","last_unstale":"2024-09-10T16:11:17.231608+0000","last_undegraded":"2024-09-10T16:11:17.231608+0000","last_fullsized":"2024-09-10T16:11:17.231608+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:38:13.781380+0000","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18724,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20838","reported_seq":21009,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:11:17.232728+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:11:17.232728+0000","last_peered":"2024-09-10T16:11:17.232728+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:11:17.232728+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17469,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":105,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.233051+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:11:17.233051+0000","last_peered":"2024-09-10T16:11:17.233051+0000","last_clean":"2024-09-10T16:11:17.233051+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:11:17.233051+0000","last_undegraded":"2024-09-10T16:11:17.233051+0000","last_fullsized":"2024-09-10T16:11:17.233051+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20598","reported_seq":29199,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.233507+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:11:17.233507+0000","last_peered":"2024-09-10T16:11:17.233507+0000","last_clean":"2024-09-10T16:11:17.233507+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:11:17.233507+0000","last_undegraded":"2024-09-10T16:11:17.233507+0000","last_fullsized":"2024-09-10T16:11:17.233507+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16988,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":130,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.232065+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:11:17.232065+0000","last_peered":"2024-09-10T16:11:17.232065+0000","last_clean":"2024-09-10T16:11:17.232065+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:11:17.232065+0000","last_undegraded":"2024-09-10T16:11:17.232065+0000","last_fullsized":"2024-09-10T16:11:17.232065+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'21048","reported_seq":21210,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.319438+0000","last_change":"2024-09-10T16:11:17.315372+0000","last_active":"2024-09-10T16:11:17.319438+0000","last_peered":"2024-09-10T16:11:17.319438+0000","last_clean":"2024-09-10T16:11:17.319438+0000","last_became_active":"2024-09-10T16:11:17.234487+0000","last_became_peered":"2024-09-10T16:11:17.234487+0000","last_unstale":"2024-09-10T16:11:17.319438+0000","last_undegraded":"2024-09-10T16:11:17.319438+0000","last_fullsized":"2024-09-10T16:11:17.319438+0000","mapping_epoch":60,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:19:00.114196+0000","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17303,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4373,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":105,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.233467+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:11:17.233467+0000","last_peered":"2024-09-10T16:11:17.233467+0000","last_clean":"2024-09-10T16:11:17.233467+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:11:17.233467+0000","last_undegraded":"2024-09-10T16:11:17.233467+0000","last_fullsized":"2024-09-10T16:11:17.233467+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":110,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.233556+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:11:17.233556+0000","last_peered":"2024-09-10T16:11:17.233556+0000","last_clean":"2024-09-10T16:11:17.233556+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:11:17.233556+0000","last_undegraded":"2024-09-10T16:11:17.233556+0000","last_fullsized":"2024-09-10T16:11:17.233556+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'21198","reported_seq":23135,"reported_epoch":61,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:11:21.233487+0000","last_change":"2024-09-10T16:11:16.084896+0000","last_active":"2024-09-10T16:11:21.233487+0000","last_peered":"2024-09-10T16:11:21.233487+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:11:21.233487+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17361,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":878,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3445}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":11989,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292096,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39401,"num_bytes_recovered":33097612,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":195031040,"data_stored":194929876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161022,"ondisk_log_size":161022,"up":32,"acting":29,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986336,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487948,"kb_used_data":123068,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93883892,"statfs":{"total":96636764160,"available":96137105408,"internally_reserved":0,"allocated":126021632,"data_stored":125648463,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659281,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487588,"kb_used_data":532,"kb_used_omap":4,"kb_used_meta":487035,"kb_avail":93884252,"statfs":{"total":96636764160,"available":96137474048,"internally_reserved":0,"allocated":544768,"data_stored":174748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":498724526},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182564,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":547396,"kb_used_data":68716,"kb_used_omap":4,"kb_used_meta":478651,"kb_avail":93824444,"statfs":{"total":96636764160,"available":96076230656,"internally_reserved":0,"allocated":70365184,"data_stored":69987938,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":490139306},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":69853184,"data_stored":69817054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":125145088,"data_stored":125108958,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:11:25.744 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-10T16:11:25.744 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "49'21082", 'reported_seq': 21256, 'reported_epoch': 61, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:11:17.231941+0000', 'last_change': '2024-09-10T15:59:16.331899+0000', 'last_active': '2024-09-10T16:11:17.231941+0000', 'last_peered': '2024-09-10T16:11:17.231941+0000', 'last_clean': '2024-09-10T15:59:14.139436+0000', 'last_became_active': '2024-09-10T15:59:16.318715+0000', 'last_became_peered': '2024-09-10T15:59:16.318715+0000', 'last_unstale': '2024-09-10T16:11:17.231941+0000', 'last_undegraded': '2024-09-10T15:59:15.877524+0000', 'last_fullsized': '2024-09-10T15:59:15.769042+0000', 'mapping_epoch': 42, 'log_start': "45'11000", 'ondisk_log_start': "45'11000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3537", 'last_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'objects_scrubbed': 3497, 'log_size': 10082, 'log_dups_size': 0, 'ondisk_log_size': 10082, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1224, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4237, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 17581, 'num_write_kb': 28172, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4235}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:11:25.744 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-10T16:11:25.744 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "49'20838", 'reported_seq': 21009, 'reported_epoch': 61, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:11:17.232728+0000', 'last_change': '2024-09-10T15:59:16.315641+0000', 'last_active': '2024-09-10T16:11:17.232728+0000', 'last_peered': '2024-09-10T16:11:17.232728+0000', 'last_clean': '2024-09-10T15:59:14.117001+0000', 'last_became_active': '2024-09-10T15:59:16.302245+0000', 'last_became_peered': '2024-09-10T15:59:16.302245+0000', 'last_unstale': '2024-09-10T16:11:17.232728+0000', 'last_undegraded': '2024-09-10T15:59:15.845012+0000', 'last_fullsized': '2024-09-10T15:59:15.770337+0000', 'mapping_epoch': 42, 'log_start': "45'10800", 'ondisk_log_start': "45'10800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3404", 'last_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'objects_scrubbed': 3367, 'log_size': 10038, 'log_dups_size': 0, 'ondisk_log_size': 10038, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 983, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17469, 'num_write_kb': 28204, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4304}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:11:25.745 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-10T16:11:25.745 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "49'21198", 'reported_seq': 23135, 'reported_epoch': 61, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:11:21.233487+0000', 'last_change': '2024-09-10T16:11:16.084896+0000', 'last_active': '2024-09-10T16:11:21.233487+0000', 'last_peered': '2024-09-10T16:11:21.233487+0000', 'last_clean': '2024-09-10T15:59:14.129780+0000', 'last_became_active': '2024-09-10T15:59:16.300932+0000', 'last_became_peered': '2024-09-10T15:59:16.300932+0000', 'last_unstale': '2024-09-10T16:11:21.233487+0000', 'last_undegraded': '2024-09-10T15:59:15.887823+0000', 'last_fullsized': '2024-09-10T15:59:15.768513+0000', 'mapping_epoch': 42, 'log_start': "45'11100", 'ondisk_log_start': "45'11100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3880", 'last_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'objects_scrubbed': 3827, 'log_size': 10098, 'log_dups_size': 0, 'ondisk_log_size': 10098, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1397, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3447, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17361, 'num_write_kb': 27072, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 878, 'num_bytes_recovered': 483, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3445}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:11:25.745 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T16:11:26.128 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:11:26.128 INFO:teuthology.orchestra.run.smithi115.stdout:{"fsid":"7effee29-df05-4351-9741-ee203564ed81","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 11989/24 objects degraded (49954.167%), 3 pgs degraded, 3 pgs undersized","count":6},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":850,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":61,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983954,"num_in_osds":3,"osd_in_since":1725983947,"num_remapped_pgs":3},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":21},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":2},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":12,"data_bytes":4849,"bytes_used":1559482368,"bytes_avail":288350810112,"bytes_total":289910292480,"degraded_objects":11989,"degraded_total":24,"degraded_ratio":499.54166666666674,"recovering_objects_per_sec":245,"recovering_bytes_per_sec":60,"recovering_keys_per_sec":0,"num_objects_recovered":1968,"num_bytes_recovered":483,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-10T15:57:14:967120+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T16:00:13.560776+0000","services":{}},"progress_events":{}} 2024-09-10T16:11:26.142 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-10T16:11:26.143 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:11:26.454 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:11:26.455 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:11:26.467 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":479,"stamp":"2024-09-10T16:11:25.793428+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":11989,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292098,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39403,"num_bytes_recovered":33097631,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161025,"ondisk_log_size":161025,"up":48,"acting":45,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1514612,"kb_used_data":192316,"kb_used_omap":12,"kb_used_meta":1322227,"kb_avail":281600908,"statfs":{"total":289910292480,"available":288359329792,"internally_reserved":0,"allocated":196931584,"data_stored":195811149,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1353960450},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1968,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1968,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.571552"},"pg_stats":[{"pgid":"2.f","version":"49'20672","reported_seq":29308,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.232365+0000","last_change":"2024-09-10T16:09:57.088531+0000","last_active":"2024-09-10T16:11:17.232365+0000","last_peered":"2024-09-10T16:11:17.232365+0000","last_clean":"2024-09-10T16:11:17.232365+0000","last_became_active":"2024-09-10T16:09:57.005755+0000","last_became_peered":"2024-09-10T16:09:57.005755+0000","last_unstale":"2024-09-10T16:11:17.232365+0000","last_undegraded":"2024-09-10T16:11:17.232365+0000","last_fullsized":"2024-09-10T16:11:17.232365+0000","mapping_epoch":54,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:09:19.329697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20672,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4259,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20844","reported_seq":20992,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.232030+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:11:17.232030+0000","last_peered":"2024-09-10T16:11:17.232030+0000","last_clean":"2024-09-10T16:11:17.232030+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:11:17.232030+0000","last_undegraded":"2024-09-10T16:11:17.232030+0000","last_fullsized":"2024-09-10T16:11:17.232030+0000","mapping_epoch":35,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18949,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20798","reported_seq":29512,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.234237+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:11:17.234237+0000","last_peered":"2024-09-10T16:11:17.234237+0000","last_clean":"2024-09-10T16:11:17.234237+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:11:17.234237+0000","last_undegraded":"2024-09-10T16:11:17.234237+0000","last_fullsized":"2024-09-10T16:11:17.234237+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18802,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20660","reported_seq":20776,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.231774+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:11:17.231774+0000","last_peered":"2024-09-10T16:11:17.231774+0000","last_clean":"2024-09-10T16:11:17.231774+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:11:17.231774+0000","last_undegraded":"2024-09-10T16:11:17.231774+0000","last_fullsized":"2024-09-10T16:11:17.231774+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20660,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20891","reported_seq":21057,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.231216+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:11:17.231216+0000","last_peered":"2024-09-10T16:11:17.231216+0000","last_clean":"2024-09-10T16:11:17.231216+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:11:17.231216+0000","last_undegraded":"2024-09-10T16:11:17.231216+0000","last_fullsized":"2024-09-10T16:11:17.231216+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17602,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'21061","reported_seq":21226,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.233215+0000","last_change":"2024-09-10T16:10:52.241702+0000","last_active":"2024-09-10T16:11:17.233215+0000","last_peered":"2024-09-10T16:11:17.233215+0000","last_clean":"2024-09-10T16:11:17.233215+0000","last_became_active":"2024-09-10T16:10:52.171236+0000","last_became_peered":"2024-09-10T16:10:52.171236+0000","last_unstale":"2024-09-10T16:11:17.233215+0000","last_undegraded":"2024-09-10T16:11:17.233215+0000","last_fullsized":"2024-09-10T16:11:17.233215+0000","mapping_epoch":58,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:04:46.778280+0000","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17673,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4372,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'21082","reported_seq":21256,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:11:17.231941+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:11:17.231941+0000","last_peered":"2024-09-10T16:11:17.231941+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:11:17.231941+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17581,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4235},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'21314","reported_seq":21480,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.231025+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:11:17.231025+0000","last_peered":"2024-09-10T16:11:17.231025+0000","last_clean":"2024-09-10T16:11:17.231025+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:11:17.231025+0000","last_undegraded":"2024-09-10T16:11:17.231025+0000","last_fullsized":"2024-09-10T16:11:17.231025+0000","mapping_epoch":34,"log_start":"45'11300","ondisk_log_start":"45'11300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18161,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.232985+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:11:17.232985+0000","last_peered":"2024-09-10T16:11:17.232985+0000","last_clean":"2024-09-10T16:11:17.232985+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:11:17.232985+0000","last_undegraded":"2024-09-10T16:11:17.232985+0000","last_fullsized":"2024-09-10T16:11:17.232985+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20457","reported_seq":20636,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.233353+0000","last_change":"2024-09-10T16:09:33.012840+0000","last_active":"2024-09-10T16:11:17.233353+0000","last_peered":"2024-09-10T16:11:17.233353+0000","last_clean":"2024-09-10T16:11:17.233353+0000","last_became_active":"2024-09-10T16:09:32.934056+0000","last_became_peered":"2024-09-10T16:09:32.934056+0000","last_unstale":"2024-09-10T16:11:17.233353+0000","last_undegraded":"2024-09-10T16:11:17.233353+0000","last_fullsized":"2024-09-10T16:11:17.233353+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:08:30.481235+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18276,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":4579,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":130,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.232408+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:11:17.232408+0000","last_peered":"2024-09-10T16:11:17.232408+0000","last_clean":"2024-09-10T16:11:17.232408+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:11:17.232408+0000","last_undegraded":"2024-09-10T16:11:17.232408+0000","last_fullsized":"2024-09-10T16:11:17.232408+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20910","reported_seq":29598,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.230962+0000","last_change":"2024-09-10T16:09:08.909580+0000","last_active":"2024-09-10T16:11:17.230962+0000","last_peered":"2024-09-10T16:11:17.230962+0000","last_clean":"2024-09-10T16:11:17.230962+0000","last_became_active":"2024-09-10T16:09:08.854941+0000","last_became_peered":"2024-09-10T16:09:08.854941+0000","last_unstale":"2024-09-10T16:11:17.230962+0000","last_undegraded":"2024-09-10T16:11:17.230962+0000","last_fullsized":"2024-09-10T16:11:17.230962+0000","mapping_epoch":50,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:13:14.846691+0000","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17414,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4257,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":140,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.232615+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:11:17.232615+0000","last_peered":"2024-09-10T16:11:17.232615+0000","last_clean":"2024-09-10T16:11:17.232615+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:11:17.232615+0000","last_undegraded":"2024-09-10T16:11:17.232615+0000","last_fullsized":"2024-09-10T16:11:17.232615+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20781","reported_seq":29570,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.232496+0000","last_change":"2024-09-10T16:10:20.907889+0000","last_active":"2024-09-10T16:11:17.232496+0000","last_peered":"2024-09-10T16:11:17.232496+0000","last_clean":"2024-09-10T16:11:17.232496+0000","last_became_active":"2024-09-10T16:10:20.852065+0000","last_became_peered":"2024-09-10T16:10:20.852065+0000","last_unstale":"2024-09-10T16:11:17.232496+0000","last_undegraded":"2024-09-10T16:11:17.232496+0000","last_fullsized":"2024-09-10T16:11:17.232496+0000","mapping_epoch":56,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:34.069549+0000","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18461,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4307,"num_bytes_recovered":8675,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":115,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.233348+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:11:17.233348+0000","last_peered":"2024-09-10T16:11:17.233348+0000","last_clean":"2024-09-10T16:11:17.233348+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:11:17.233348+0000","last_undegraded":"2024-09-10T16:11:17.233348+0000","last_fullsized":"2024-09-10T16:11:17.233348+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'21170","reported_seq":30055,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.231608+0000","last_change":"2024-09-10T16:10:50.728349+0000","last_active":"2024-09-10T16:11:17.231608+0000","last_peered":"2024-09-10T16:11:17.231608+0000","last_clean":"2024-09-10T16:11:17.231608+0000","last_became_active":"2024-09-10T16:08:10.860438+0000","last_became_peered":"2024-09-10T16:08:10.860438+0000","last_unstale":"2024-09-10T16:11:17.231608+0000","last_undegraded":"2024-09-10T16:11:17.231608+0000","last_fullsized":"2024-09-10T16:11:17.231608+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:38:13.781380+0000","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18724,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20838","reported_seq":21009,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:11:17.232728+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:11:17.232728+0000","last_peered":"2024-09-10T16:11:17.232728+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:11:17.232728+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17469,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":105,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.233051+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:11:17.233051+0000","last_peered":"2024-09-10T16:11:17.233051+0000","last_clean":"2024-09-10T16:11:17.233051+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:11:17.233051+0000","last_undegraded":"2024-09-10T16:11:17.233051+0000","last_fullsized":"2024-09-10T16:11:17.233051+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20598","reported_seq":29199,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.233507+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:11:17.233507+0000","last_peered":"2024-09-10T16:11:17.233507+0000","last_clean":"2024-09-10T16:11:17.233507+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:11:17.233507+0000","last_undegraded":"2024-09-10T16:11:17.233507+0000","last_fullsized":"2024-09-10T16:11:17.233507+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16988,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":130,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.232065+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:11:17.232065+0000","last_peered":"2024-09-10T16:11:17.232065+0000","last_clean":"2024-09-10T16:11:17.232065+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:11:17.232065+0000","last_undegraded":"2024-09-10T16:11:17.232065+0000","last_fullsized":"2024-09-10T16:11:17.232065+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'21048","reported_seq":21210,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.319438+0000","last_change":"2024-09-10T16:11:17.315372+0000","last_active":"2024-09-10T16:11:17.319438+0000","last_peered":"2024-09-10T16:11:17.319438+0000","last_clean":"2024-09-10T16:11:17.319438+0000","last_became_active":"2024-09-10T16:11:17.234487+0000","last_became_peered":"2024-09-10T16:11:17.234487+0000","last_unstale":"2024-09-10T16:11:17.319438+0000","last_undegraded":"2024-09-10T16:11:17.319438+0000","last_fullsized":"2024-09-10T16:11:17.319438+0000","mapping_epoch":60,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:19:00.114196+0000","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17303,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4373,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":105,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.233467+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:11:17.233467+0000","last_peered":"2024-09-10T16:11:17.233467+0000","last_clean":"2024-09-10T16:11:17.233467+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:11:17.233467+0000","last_undegraded":"2024-09-10T16:11:17.233467+0000","last_fullsized":"2024-09-10T16:11:17.233467+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":110,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.233556+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:11:17.233556+0000","last_peered":"2024-09-10T16:11:17.233556+0000","last_clean":"2024-09-10T16:11:17.233556+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:11:17.233556+0000","last_undegraded":"2024-09-10T16:11:17.233556+0000","last_fullsized":"2024-09-10T16:11:17.233556+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'21198","reported_seq":23135,"reported_epoch":61,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:11:21.233487+0000","last_change":"2024-09-10T16:11:16.084896+0000","last_active":"2024-09-10T16:11:21.233487+0000","last_peered":"2024-09-10T16:11:21.233487+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:11:21.233487+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17361,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":878,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3445}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":11989,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292096,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39401,"num_bytes_recovered":33097612,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":195031040,"data_stored":194929876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161022,"ondisk_log_size":161022,"up":32,"acting":29,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986336,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487948,"kb_used_data":123068,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93883892,"statfs":{"total":96636764160,"available":96137105408,"internally_reserved":0,"allocated":126021632,"data_stored":125648463,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659282,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":479268,"kb_used_data":532,"kb_used_omap":4,"kb_used_meta":478715,"kb_avail":93892572,"statfs":{"total":96636764160,"available":96145993728,"internally_reserved":0,"allocated":544768,"data_stored":174748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":490204846},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182564,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":547396,"kb_used_data":68716,"kb_used_omap":4,"kb_used_meta":478651,"kb_avail":93824444,"statfs":{"total":96636764160,"available":96076230656,"internally_reserved":0,"allocated":70365184,"data_stored":69987938,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":490139306},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":69853184,"data_stored":69817054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":125145088,"data_stored":125108958,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:11:29.469 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:11:29.777 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:11:29.778 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:11:29.792 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":480,"stamp":"2024-09-10T16:11:27.793752+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":11319,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292098,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40073,"num_bytes_recovered":33097631,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161025,"ondisk_log_size":161025,"up":48,"acting":45,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1511612,"kb_used_data":189316,"kb_used_omap":12,"kb_used_meta":1322227,"kb_avail":281603908,"statfs":{"total":289910292480,"available":288362401792,"internally_reserved":0,"allocated":193859584,"data_stored":192742762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1353960450},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2625,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2625,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.563391"},"pg_stats":[{"pgid":"2.f","version":"49'20672","reported_seq":29308,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.232365+0000","last_change":"2024-09-10T16:09:57.088531+0000","last_active":"2024-09-10T16:11:17.232365+0000","last_peered":"2024-09-10T16:11:17.232365+0000","last_clean":"2024-09-10T16:11:17.232365+0000","last_became_active":"2024-09-10T16:09:57.005755+0000","last_became_peered":"2024-09-10T16:09:57.005755+0000","last_unstale":"2024-09-10T16:11:17.232365+0000","last_undegraded":"2024-09-10T16:11:17.232365+0000","last_fullsized":"2024-09-10T16:11:17.232365+0000","mapping_epoch":54,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:09:19.329697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20672,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4259,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20844","reported_seq":20992,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.232030+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:11:17.232030+0000","last_peered":"2024-09-10T16:11:17.232030+0000","last_clean":"2024-09-10T16:11:17.232030+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:11:17.232030+0000","last_undegraded":"2024-09-10T16:11:17.232030+0000","last_fullsized":"2024-09-10T16:11:17.232030+0000","mapping_epoch":35,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18949,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20798","reported_seq":29512,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.234237+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:11:17.234237+0000","last_peered":"2024-09-10T16:11:17.234237+0000","last_clean":"2024-09-10T16:11:17.234237+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:11:17.234237+0000","last_undegraded":"2024-09-10T16:11:17.234237+0000","last_fullsized":"2024-09-10T16:11:17.234237+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18802,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20660","reported_seq":20776,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.231774+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:11:17.231774+0000","last_peered":"2024-09-10T16:11:17.231774+0000","last_clean":"2024-09-10T16:11:17.231774+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:11:17.231774+0000","last_undegraded":"2024-09-10T16:11:17.231774+0000","last_fullsized":"2024-09-10T16:11:17.231774+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20660,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20891","reported_seq":21057,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.231216+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:11:17.231216+0000","last_peered":"2024-09-10T16:11:17.231216+0000","last_clean":"2024-09-10T16:11:17.231216+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:11:17.231216+0000","last_undegraded":"2024-09-10T16:11:17.231216+0000","last_fullsized":"2024-09-10T16:11:17.231216+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17602,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'21061","reported_seq":21226,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.233215+0000","last_change":"2024-09-10T16:10:52.241702+0000","last_active":"2024-09-10T16:11:17.233215+0000","last_peered":"2024-09-10T16:11:17.233215+0000","last_clean":"2024-09-10T16:11:17.233215+0000","last_became_active":"2024-09-10T16:10:52.171236+0000","last_became_peered":"2024-09-10T16:10:52.171236+0000","last_unstale":"2024-09-10T16:11:17.233215+0000","last_undegraded":"2024-09-10T16:11:17.233215+0000","last_fullsized":"2024-09-10T16:11:17.233215+0000","mapping_epoch":58,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:04:46.778280+0000","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17673,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4372,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'21082","reported_seq":21256,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:11:17.231941+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:11:17.231941+0000","last_peered":"2024-09-10T16:11:17.231941+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:11:17.231941+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17581,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4235},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'21314","reported_seq":21480,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.231025+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:11:17.231025+0000","last_peered":"2024-09-10T16:11:17.231025+0000","last_clean":"2024-09-10T16:11:17.231025+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:11:17.231025+0000","last_undegraded":"2024-09-10T16:11:17.231025+0000","last_fullsized":"2024-09-10T16:11:17.231025+0000","mapping_epoch":34,"log_start":"45'11300","ondisk_log_start":"45'11300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18161,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.232985+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:11:17.232985+0000","last_peered":"2024-09-10T16:11:17.232985+0000","last_clean":"2024-09-10T16:11:17.232985+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:11:17.232985+0000","last_undegraded":"2024-09-10T16:11:17.232985+0000","last_fullsized":"2024-09-10T16:11:17.232985+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20457","reported_seq":20636,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.233353+0000","last_change":"2024-09-10T16:09:33.012840+0000","last_active":"2024-09-10T16:11:17.233353+0000","last_peered":"2024-09-10T16:11:17.233353+0000","last_clean":"2024-09-10T16:11:17.233353+0000","last_became_active":"2024-09-10T16:09:32.934056+0000","last_became_peered":"2024-09-10T16:09:32.934056+0000","last_unstale":"2024-09-10T16:11:17.233353+0000","last_undegraded":"2024-09-10T16:11:17.233353+0000","last_fullsized":"2024-09-10T16:11:17.233353+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:08:30.481235+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18276,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":4579,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":130,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.232408+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:11:17.232408+0000","last_peered":"2024-09-10T16:11:17.232408+0000","last_clean":"2024-09-10T16:11:17.232408+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:11:17.232408+0000","last_undegraded":"2024-09-10T16:11:17.232408+0000","last_fullsized":"2024-09-10T16:11:17.232408+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20910","reported_seq":29598,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.230962+0000","last_change":"2024-09-10T16:09:08.909580+0000","last_active":"2024-09-10T16:11:17.230962+0000","last_peered":"2024-09-10T16:11:17.230962+0000","last_clean":"2024-09-10T16:11:17.230962+0000","last_became_active":"2024-09-10T16:09:08.854941+0000","last_became_peered":"2024-09-10T16:09:08.854941+0000","last_unstale":"2024-09-10T16:11:17.230962+0000","last_undegraded":"2024-09-10T16:11:17.230962+0000","last_fullsized":"2024-09-10T16:11:17.230962+0000","mapping_epoch":50,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:13:14.846691+0000","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17414,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4257,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":140,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.232615+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:11:17.232615+0000","last_peered":"2024-09-10T16:11:17.232615+0000","last_clean":"2024-09-10T16:11:17.232615+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:11:17.232615+0000","last_undegraded":"2024-09-10T16:11:17.232615+0000","last_fullsized":"2024-09-10T16:11:17.232615+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20781","reported_seq":29570,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.232496+0000","last_change":"2024-09-10T16:10:20.907889+0000","last_active":"2024-09-10T16:11:17.232496+0000","last_peered":"2024-09-10T16:11:17.232496+0000","last_clean":"2024-09-10T16:11:17.232496+0000","last_became_active":"2024-09-10T16:10:20.852065+0000","last_became_peered":"2024-09-10T16:10:20.852065+0000","last_unstale":"2024-09-10T16:11:17.232496+0000","last_undegraded":"2024-09-10T16:11:17.232496+0000","last_fullsized":"2024-09-10T16:11:17.232496+0000","mapping_epoch":56,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:34.069549+0000","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18461,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4307,"num_bytes_recovered":8675,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":115,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.233348+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:11:17.233348+0000","last_peered":"2024-09-10T16:11:17.233348+0000","last_clean":"2024-09-10T16:11:17.233348+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:11:17.233348+0000","last_undegraded":"2024-09-10T16:11:17.233348+0000","last_fullsized":"2024-09-10T16:11:17.233348+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'21170","reported_seq":30055,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.231608+0000","last_change":"2024-09-10T16:10:50.728349+0000","last_active":"2024-09-10T16:11:17.231608+0000","last_peered":"2024-09-10T16:11:17.231608+0000","last_clean":"2024-09-10T16:11:17.231608+0000","last_became_active":"2024-09-10T16:08:10.860438+0000","last_became_peered":"2024-09-10T16:08:10.860438+0000","last_unstale":"2024-09-10T16:11:17.231608+0000","last_undegraded":"2024-09-10T16:11:17.231608+0000","last_fullsized":"2024-09-10T16:11:17.231608+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:38:13.781380+0000","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18724,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20838","reported_seq":21009,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:11:17.232728+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:11:17.232728+0000","last_peered":"2024-09-10T16:11:17.232728+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:11:17.232728+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17469,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":105,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.233051+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:11:17.233051+0000","last_peered":"2024-09-10T16:11:17.233051+0000","last_clean":"2024-09-10T16:11:17.233051+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:11:17.233051+0000","last_undegraded":"2024-09-10T16:11:17.233051+0000","last_fullsized":"2024-09-10T16:11:17.233051+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20598","reported_seq":29199,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.233507+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:11:17.233507+0000","last_peered":"2024-09-10T16:11:17.233507+0000","last_clean":"2024-09-10T16:11:17.233507+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:11:17.233507+0000","last_undegraded":"2024-09-10T16:11:17.233507+0000","last_fullsized":"2024-09-10T16:11:17.233507+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16988,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":130,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.232065+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:11:17.232065+0000","last_peered":"2024-09-10T16:11:17.232065+0000","last_clean":"2024-09-10T16:11:17.232065+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:11:17.232065+0000","last_undegraded":"2024-09-10T16:11:17.232065+0000","last_fullsized":"2024-09-10T16:11:17.232065+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'21048","reported_seq":21210,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.319438+0000","last_change":"2024-09-10T16:11:17.315372+0000","last_active":"2024-09-10T16:11:17.319438+0000","last_peered":"2024-09-10T16:11:17.319438+0000","last_clean":"2024-09-10T16:11:17.319438+0000","last_became_active":"2024-09-10T16:11:17.234487+0000","last_became_peered":"2024-09-10T16:11:17.234487+0000","last_unstale":"2024-09-10T16:11:17.319438+0000","last_undegraded":"2024-09-10T16:11:17.319438+0000","last_fullsized":"2024-09-10T16:11:17.319438+0000","mapping_epoch":60,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:19:00.114196+0000","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17303,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4373,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":105,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.233467+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:11:17.233467+0000","last_peered":"2024-09-10T16:11:17.233467+0000","last_clean":"2024-09-10T16:11:17.233467+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:11:17.233467+0000","last_undegraded":"2024-09-10T16:11:17.233467+0000","last_fullsized":"2024-09-10T16:11:17.233467+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":110,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.233556+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:11:17.233556+0000","last_peered":"2024-09-10T16:11:17.233556+0000","last_clean":"2024-09-10T16:11:17.233556+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:11:17.233556+0000","last_undegraded":"2024-09-10T16:11:17.233556+0000","last_fullsized":"2024-09-10T16:11:17.233556+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'21198","reported_seq":24475,"reported_epoch":61,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:11:26.237211+0000","last_change":"2024-09-10T16:11:16.084896+0000","last_active":"2024-09-10T16:11:26.237211+0000","last_peered":"2024-09-10T16:11:26.237211+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:11:26.237211+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2777,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17361,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1548,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2775}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":11319,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292096,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40071,"num_bytes_recovered":33097612,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":191959040,"data_stored":191861489,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161022,"ondisk_log_size":161022,"up":32,"acting":29,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986337,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":484948,"kb_used_data":120068,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93886892,"statfs":{"total":96636764160,"available":96140177408,"internally_reserved":0,"allocated":122949632,"data_stored":122580076,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659282,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":479268,"kb_used_data":532,"kb_used_omap":4,"kb_used_meta":478715,"kb_avail":93892572,"statfs":{"total":96636764160,"available":96145993728,"internally_reserved":0,"allocated":544768,"data_stored":174748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":490204846},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182565,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":547396,"kb_used_data":68716,"kb_used_omap":4,"kb_used_meta":478651,"kb_avail":93824444,"statfs":{"total":96636764160,"available":96076230656,"internally_reserved":0,"allocated":70365184,"data_stored":69987938,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":490139306},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":69853184,"data_stored":69817054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":122073088,"data_stored":122040571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:11:29.793 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:11:30.095 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:11:30.096 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:11:30.108 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":481,"stamp":"2024-09-10T16:11:29.794078+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":11319,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292098,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40073,"num_bytes_recovered":33097631,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161025,"ondisk_log_size":161025,"up":48,"acting":45,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1511612,"kb_used_data":189316,"kb_used_omap":12,"kb_used_meta":1322227,"kb_avail":281603908,"statfs":{"total":289910292480,"available":288362401792,"internally_reserved":0,"allocated":193859584,"data_stored":192742762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1353960450},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2625,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2625,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002027"},"pg_stats":[{"pgid":"2.f","version":"49'20672","reported_seq":29308,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.232365+0000","last_change":"2024-09-10T16:09:57.088531+0000","last_active":"2024-09-10T16:11:17.232365+0000","last_peered":"2024-09-10T16:11:17.232365+0000","last_clean":"2024-09-10T16:11:17.232365+0000","last_became_active":"2024-09-10T16:09:57.005755+0000","last_became_peered":"2024-09-10T16:09:57.005755+0000","last_unstale":"2024-09-10T16:11:17.232365+0000","last_undegraded":"2024-09-10T16:11:17.232365+0000","last_fullsized":"2024-09-10T16:11:17.232365+0000","mapping_epoch":54,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:09:19.329697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20672,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4259,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20844","reported_seq":20992,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.232030+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:11:17.232030+0000","last_peered":"2024-09-10T16:11:17.232030+0000","last_clean":"2024-09-10T16:11:17.232030+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:11:17.232030+0000","last_undegraded":"2024-09-10T16:11:17.232030+0000","last_fullsized":"2024-09-10T16:11:17.232030+0000","mapping_epoch":35,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18949,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20798","reported_seq":29512,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.234237+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:11:17.234237+0000","last_peered":"2024-09-10T16:11:17.234237+0000","last_clean":"2024-09-10T16:11:17.234237+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:11:17.234237+0000","last_undegraded":"2024-09-10T16:11:17.234237+0000","last_fullsized":"2024-09-10T16:11:17.234237+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18802,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20660","reported_seq":20776,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.231774+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:11:17.231774+0000","last_peered":"2024-09-10T16:11:17.231774+0000","last_clean":"2024-09-10T16:11:17.231774+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:11:17.231774+0000","last_undegraded":"2024-09-10T16:11:17.231774+0000","last_fullsized":"2024-09-10T16:11:17.231774+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20660,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20891","reported_seq":21057,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.231216+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:11:17.231216+0000","last_peered":"2024-09-10T16:11:17.231216+0000","last_clean":"2024-09-10T16:11:17.231216+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:11:17.231216+0000","last_undegraded":"2024-09-10T16:11:17.231216+0000","last_fullsized":"2024-09-10T16:11:17.231216+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17602,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'21061","reported_seq":21226,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.233215+0000","last_change":"2024-09-10T16:10:52.241702+0000","last_active":"2024-09-10T16:11:17.233215+0000","last_peered":"2024-09-10T16:11:17.233215+0000","last_clean":"2024-09-10T16:11:17.233215+0000","last_became_active":"2024-09-10T16:10:52.171236+0000","last_became_peered":"2024-09-10T16:10:52.171236+0000","last_unstale":"2024-09-10T16:11:17.233215+0000","last_undegraded":"2024-09-10T16:11:17.233215+0000","last_fullsized":"2024-09-10T16:11:17.233215+0000","mapping_epoch":58,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:04:46.778280+0000","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17673,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4372,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'21082","reported_seq":21256,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:11:17.231941+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:11:17.231941+0000","last_peered":"2024-09-10T16:11:17.231941+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:11:17.231941+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17581,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4235},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'21314","reported_seq":21480,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.231025+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:11:17.231025+0000","last_peered":"2024-09-10T16:11:17.231025+0000","last_clean":"2024-09-10T16:11:17.231025+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:11:17.231025+0000","last_undegraded":"2024-09-10T16:11:17.231025+0000","last_fullsized":"2024-09-10T16:11:17.231025+0000","mapping_epoch":34,"log_start":"45'11300","ondisk_log_start":"45'11300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18161,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.232985+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:11:17.232985+0000","last_peered":"2024-09-10T16:11:17.232985+0000","last_clean":"2024-09-10T16:11:17.232985+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:11:17.232985+0000","last_undegraded":"2024-09-10T16:11:17.232985+0000","last_fullsized":"2024-09-10T16:11:17.232985+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20457","reported_seq":20636,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.233353+0000","last_change":"2024-09-10T16:09:33.012840+0000","last_active":"2024-09-10T16:11:17.233353+0000","last_peered":"2024-09-10T16:11:17.233353+0000","last_clean":"2024-09-10T16:11:17.233353+0000","last_became_active":"2024-09-10T16:09:32.934056+0000","last_became_peered":"2024-09-10T16:09:32.934056+0000","last_unstale":"2024-09-10T16:11:17.233353+0000","last_undegraded":"2024-09-10T16:11:17.233353+0000","last_fullsized":"2024-09-10T16:11:17.233353+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:08:30.481235+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18276,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":4579,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":130,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.232408+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:11:17.232408+0000","last_peered":"2024-09-10T16:11:17.232408+0000","last_clean":"2024-09-10T16:11:17.232408+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:11:17.232408+0000","last_undegraded":"2024-09-10T16:11:17.232408+0000","last_fullsized":"2024-09-10T16:11:17.232408+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20910","reported_seq":29598,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.230962+0000","last_change":"2024-09-10T16:09:08.909580+0000","last_active":"2024-09-10T16:11:17.230962+0000","last_peered":"2024-09-10T16:11:17.230962+0000","last_clean":"2024-09-10T16:11:17.230962+0000","last_became_active":"2024-09-10T16:09:08.854941+0000","last_became_peered":"2024-09-10T16:09:08.854941+0000","last_unstale":"2024-09-10T16:11:17.230962+0000","last_undegraded":"2024-09-10T16:11:17.230962+0000","last_fullsized":"2024-09-10T16:11:17.230962+0000","mapping_epoch":50,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:13:14.846691+0000","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17414,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4257,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":140,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.232615+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:11:17.232615+0000","last_peered":"2024-09-10T16:11:17.232615+0000","last_clean":"2024-09-10T16:11:17.232615+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:11:17.232615+0000","last_undegraded":"2024-09-10T16:11:17.232615+0000","last_fullsized":"2024-09-10T16:11:17.232615+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20781","reported_seq":29570,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.232496+0000","last_change":"2024-09-10T16:10:20.907889+0000","last_active":"2024-09-10T16:11:17.232496+0000","last_peered":"2024-09-10T16:11:17.232496+0000","last_clean":"2024-09-10T16:11:17.232496+0000","last_became_active":"2024-09-10T16:10:20.852065+0000","last_became_peered":"2024-09-10T16:10:20.852065+0000","last_unstale":"2024-09-10T16:11:17.232496+0000","last_undegraded":"2024-09-10T16:11:17.232496+0000","last_fullsized":"2024-09-10T16:11:17.232496+0000","mapping_epoch":56,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:34.069549+0000","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18461,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4307,"num_bytes_recovered":8675,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":115,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.233348+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:11:17.233348+0000","last_peered":"2024-09-10T16:11:17.233348+0000","last_clean":"2024-09-10T16:11:17.233348+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:11:17.233348+0000","last_undegraded":"2024-09-10T16:11:17.233348+0000","last_fullsized":"2024-09-10T16:11:17.233348+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'21170","reported_seq":30055,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.231608+0000","last_change":"2024-09-10T16:10:50.728349+0000","last_active":"2024-09-10T16:11:17.231608+0000","last_peered":"2024-09-10T16:11:17.231608+0000","last_clean":"2024-09-10T16:11:17.231608+0000","last_became_active":"2024-09-10T16:08:10.860438+0000","last_became_peered":"2024-09-10T16:08:10.860438+0000","last_unstale":"2024-09-10T16:11:17.231608+0000","last_undegraded":"2024-09-10T16:11:17.231608+0000","last_fullsized":"2024-09-10T16:11:17.231608+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:38:13.781380+0000","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18724,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20838","reported_seq":21009,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:11:17.232728+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:11:17.232728+0000","last_peered":"2024-09-10T16:11:17.232728+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:11:17.232728+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17469,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":105,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.233051+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:11:17.233051+0000","last_peered":"2024-09-10T16:11:17.233051+0000","last_clean":"2024-09-10T16:11:17.233051+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:11:17.233051+0000","last_undegraded":"2024-09-10T16:11:17.233051+0000","last_fullsized":"2024-09-10T16:11:17.233051+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20598","reported_seq":29199,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.233507+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:11:17.233507+0000","last_peered":"2024-09-10T16:11:17.233507+0000","last_clean":"2024-09-10T16:11:17.233507+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:11:17.233507+0000","last_undegraded":"2024-09-10T16:11:17.233507+0000","last_fullsized":"2024-09-10T16:11:17.233507+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16988,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":130,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.232065+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:11:17.232065+0000","last_peered":"2024-09-10T16:11:17.232065+0000","last_clean":"2024-09-10T16:11:17.232065+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:11:17.232065+0000","last_undegraded":"2024-09-10T16:11:17.232065+0000","last_fullsized":"2024-09-10T16:11:17.232065+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'21048","reported_seq":21210,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.319438+0000","last_change":"2024-09-10T16:11:17.315372+0000","last_active":"2024-09-10T16:11:17.319438+0000","last_peered":"2024-09-10T16:11:17.319438+0000","last_clean":"2024-09-10T16:11:17.319438+0000","last_became_active":"2024-09-10T16:11:17.234487+0000","last_became_peered":"2024-09-10T16:11:17.234487+0000","last_unstale":"2024-09-10T16:11:17.319438+0000","last_undegraded":"2024-09-10T16:11:17.319438+0000","last_fullsized":"2024-09-10T16:11:17.319438+0000","mapping_epoch":60,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:19:00.114196+0000","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17303,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4373,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":105,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.233467+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:11:17.233467+0000","last_peered":"2024-09-10T16:11:17.233467+0000","last_clean":"2024-09-10T16:11:17.233467+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:11:17.233467+0000","last_undegraded":"2024-09-10T16:11:17.233467+0000","last_fullsized":"2024-09-10T16:11:17.233467+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":110,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.233556+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:11:17.233556+0000","last_peered":"2024-09-10T16:11:17.233556+0000","last_clean":"2024-09-10T16:11:17.233556+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:11:17.233556+0000","last_undegraded":"2024-09-10T16:11:17.233556+0000","last_fullsized":"2024-09-10T16:11:17.233556+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'21198","reported_seq":24475,"reported_epoch":61,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:11:26.237211+0000","last_change":"2024-09-10T16:11:16.084896+0000","last_active":"2024-09-10T16:11:26.237211+0000","last_peered":"2024-09-10T16:11:26.237211+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:11:26.237211+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2777,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17361,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1548,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2775}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":11319,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292096,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40071,"num_bytes_recovered":33097612,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":191959040,"data_stored":191861489,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161022,"ondisk_log_size":161022,"up":32,"acting":29,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986337,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":484948,"kb_used_data":120068,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93886892,"statfs":{"total":96636764160,"available":96140177408,"internally_reserved":0,"allocated":122949632,"data_stored":122580076,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659282,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":479268,"kb_used_data":532,"kb_used_omap":4,"kb_used_meta":478715,"kb_avail":93892572,"statfs":{"total":96636764160,"available":96145993728,"internally_reserved":0,"allocated":544768,"data_stored":174748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":490204846},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182565,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":547396,"kb_used_data":68716,"kb_used_omap":4,"kb_used_meta":478651,"kb_avail":93824444,"statfs":{"total":96636764160,"available":96076230656,"internally_reserved":0,"allocated":70365184,"data_stored":69987938,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":490139306},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":69853184,"data_stored":69817054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":122073088,"data_stored":122040571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:11:30.109 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-10T16:11:30.109 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "49'21082", 'reported_seq': 21256, 'reported_epoch': 61, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:11:17.231941+0000', 'last_change': '2024-09-10T15:59:16.331899+0000', 'last_active': '2024-09-10T16:11:17.231941+0000', 'last_peered': '2024-09-10T16:11:17.231941+0000', 'last_clean': '2024-09-10T15:59:14.139436+0000', 'last_became_active': '2024-09-10T15:59:16.318715+0000', 'last_became_peered': '2024-09-10T15:59:16.318715+0000', 'last_unstale': '2024-09-10T16:11:17.231941+0000', 'last_undegraded': '2024-09-10T15:59:15.877524+0000', 'last_fullsized': '2024-09-10T15:59:15.769042+0000', 'mapping_epoch': 42, 'log_start': "45'11000", 'ondisk_log_start': "45'11000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3537", 'last_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'objects_scrubbed': 3497, 'log_size': 10082, 'log_dups_size': 0, 'ondisk_log_size': 10082, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1224, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4237, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 17581, 'num_write_kb': 28172, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4235}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:11:30.109 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-10T16:11:30.109 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "49'20838", 'reported_seq': 21009, 'reported_epoch': 61, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:11:17.232728+0000', 'last_change': '2024-09-10T15:59:16.315641+0000', 'last_active': '2024-09-10T16:11:17.232728+0000', 'last_peered': '2024-09-10T16:11:17.232728+0000', 'last_clean': '2024-09-10T15:59:14.117001+0000', 'last_became_active': '2024-09-10T15:59:16.302245+0000', 'last_became_peered': '2024-09-10T15:59:16.302245+0000', 'last_unstale': '2024-09-10T16:11:17.232728+0000', 'last_undegraded': '2024-09-10T15:59:15.845012+0000', 'last_fullsized': '2024-09-10T15:59:15.770337+0000', 'mapping_epoch': 42, 'log_start': "45'10800", 'ondisk_log_start': "45'10800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3404", 'last_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'objects_scrubbed': 3367, 'log_size': 10038, 'log_dups_size': 0, 'ondisk_log_size': 10038, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 983, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17469, 'num_write_kb': 28204, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4304}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:11:30.110 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-10T16:11:30.110 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "49'21198", 'reported_seq': 24475, 'reported_epoch': 61, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:11:26.237211+0000', 'last_change': '2024-09-10T16:11:16.084896+0000', 'last_active': '2024-09-10T16:11:26.237211+0000', 'last_peered': '2024-09-10T16:11:26.237211+0000', 'last_clean': '2024-09-10T15:59:14.129780+0000', 'last_became_active': '2024-09-10T15:59:16.300932+0000', 'last_became_peered': '2024-09-10T15:59:16.300932+0000', 'last_unstale': '2024-09-10T16:11:26.237211+0000', 'last_undegraded': '2024-09-10T15:59:15.887823+0000', 'last_fullsized': '2024-09-10T15:59:15.768513+0000', 'mapping_epoch': 42, 'log_start': "45'11100", 'ondisk_log_start': "45'11100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3880", 'last_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'objects_scrubbed': 3827, 'log_size': 10098, 'log_dups_size': 0, 'ondisk_log_size': 10098, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1397, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2777, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17361, 'num_write_kb': 27072, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1548, 'num_bytes_recovered': 483, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2775}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:11:30.110 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T16:11:30.480 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:11:30.481 INFO:teuthology.orchestra.run.smithi115.stdout:{"fsid":"7effee29-df05-4351-9741-ee203564ed81","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 11319/24 objects degraded (47162.500%), 3 pgs degraded, 3 pgs undersized","count":6},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":854,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":61,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983954,"num_in_osds":3,"osd_in_since":1725983947,"num_remapped_pgs":3},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":21},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":2},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":12,"data_bytes":4849,"bytes_used":1547890688,"bytes_avail":288362401792,"bytes_total":289910292480,"degraded_objects":11319,"degraded_total":24,"degraded_ratio":471.625,"recovering_objects_per_sec":218,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":2625,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-10T15:57:14:967120+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T16:00:13.560776+0000","services":{}},"progress_events":{}} 2024-09-10T16:11:30.495 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-10T16:11:30.495 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:11:30.819 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:11:30.819 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:11:30.831 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":481,"stamp":"2024-09-10T16:11:29.794078+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":11319,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292098,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40073,"num_bytes_recovered":33097631,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161025,"ondisk_log_size":161025,"up":48,"acting":45,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1511612,"kb_used_data":189316,"kb_used_omap":12,"kb_used_meta":1322227,"kb_avail":281603908,"statfs":{"total":289910292480,"available":288362401792,"internally_reserved":0,"allocated":193859584,"data_stored":192742762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1353960450},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2625,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2625,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002027"},"pg_stats":[{"pgid":"2.f","version":"49'20672","reported_seq":29308,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.232365+0000","last_change":"2024-09-10T16:09:57.088531+0000","last_active":"2024-09-10T16:11:17.232365+0000","last_peered":"2024-09-10T16:11:17.232365+0000","last_clean":"2024-09-10T16:11:17.232365+0000","last_became_active":"2024-09-10T16:09:57.005755+0000","last_became_peered":"2024-09-10T16:09:57.005755+0000","last_unstale":"2024-09-10T16:11:17.232365+0000","last_undegraded":"2024-09-10T16:11:17.232365+0000","last_fullsized":"2024-09-10T16:11:17.232365+0000","mapping_epoch":54,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:09:19.329697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20672,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4259,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20844","reported_seq":20992,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.232030+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:11:17.232030+0000","last_peered":"2024-09-10T16:11:17.232030+0000","last_clean":"2024-09-10T16:11:17.232030+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:11:17.232030+0000","last_undegraded":"2024-09-10T16:11:17.232030+0000","last_fullsized":"2024-09-10T16:11:17.232030+0000","mapping_epoch":35,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18949,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20798","reported_seq":29512,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.234237+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:11:17.234237+0000","last_peered":"2024-09-10T16:11:17.234237+0000","last_clean":"2024-09-10T16:11:17.234237+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:11:17.234237+0000","last_undegraded":"2024-09-10T16:11:17.234237+0000","last_fullsized":"2024-09-10T16:11:17.234237+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18802,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20660","reported_seq":20776,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.231774+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:11:17.231774+0000","last_peered":"2024-09-10T16:11:17.231774+0000","last_clean":"2024-09-10T16:11:17.231774+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:11:17.231774+0000","last_undegraded":"2024-09-10T16:11:17.231774+0000","last_fullsized":"2024-09-10T16:11:17.231774+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20660,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20891","reported_seq":21057,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.231216+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:11:17.231216+0000","last_peered":"2024-09-10T16:11:17.231216+0000","last_clean":"2024-09-10T16:11:17.231216+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:11:17.231216+0000","last_undegraded":"2024-09-10T16:11:17.231216+0000","last_fullsized":"2024-09-10T16:11:17.231216+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17602,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'21061","reported_seq":21226,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.233215+0000","last_change":"2024-09-10T16:10:52.241702+0000","last_active":"2024-09-10T16:11:17.233215+0000","last_peered":"2024-09-10T16:11:17.233215+0000","last_clean":"2024-09-10T16:11:17.233215+0000","last_became_active":"2024-09-10T16:10:52.171236+0000","last_became_peered":"2024-09-10T16:10:52.171236+0000","last_unstale":"2024-09-10T16:11:17.233215+0000","last_undegraded":"2024-09-10T16:11:17.233215+0000","last_fullsized":"2024-09-10T16:11:17.233215+0000","mapping_epoch":58,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:04:46.778280+0000","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17673,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4372,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'21082","reported_seq":21256,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:11:17.231941+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:11:17.231941+0000","last_peered":"2024-09-10T16:11:17.231941+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:11:17.231941+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17581,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4235},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'21314","reported_seq":21480,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.231025+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:11:17.231025+0000","last_peered":"2024-09-10T16:11:17.231025+0000","last_clean":"2024-09-10T16:11:17.231025+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:11:17.231025+0000","last_undegraded":"2024-09-10T16:11:17.231025+0000","last_fullsized":"2024-09-10T16:11:17.231025+0000","mapping_epoch":34,"log_start":"45'11300","ondisk_log_start":"45'11300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18161,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.232985+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:11:17.232985+0000","last_peered":"2024-09-10T16:11:17.232985+0000","last_clean":"2024-09-10T16:11:17.232985+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:11:17.232985+0000","last_undegraded":"2024-09-10T16:11:17.232985+0000","last_fullsized":"2024-09-10T16:11:17.232985+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20457","reported_seq":20636,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.233353+0000","last_change":"2024-09-10T16:09:33.012840+0000","last_active":"2024-09-10T16:11:17.233353+0000","last_peered":"2024-09-10T16:11:17.233353+0000","last_clean":"2024-09-10T16:11:17.233353+0000","last_became_active":"2024-09-10T16:09:32.934056+0000","last_became_peered":"2024-09-10T16:09:32.934056+0000","last_unstale":"2024-09-10T16:11:17.233353+0000","last_undegraded":"2024-09-10T16:11:17.233353+0000","last_fullsized":"2024-09-10T16:11:17.233353+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:08:30.481235+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18276,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":4579,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":130,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.232408+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:11:17.232408+0000","last_peered":"2024-09-10T16:11:17.232408+0000","last_clean":"2024-09-10T16:11:17.232408+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:11:17.232408+0000","last_undegraded":"2024-09-10T16:11:17.232408+0000","last_fullsized":"2024-09-10T16:11:17.232408+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20910","reported_seq":29598,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.230962+0000","last_change":"2024-09-10T16:09:08.909580+0000","last_active":"2024-09-10T16:11:17.230962+0000","last_peered":"2024-09-10T16:11:17.230962+0000","last_clean":"2024-09-10T16:11:17.230962+0000","last_became_active":"2024-09-10T16:09:08.854941+0000","last_became_peered":"2024-09-10T16:09:08.854941+0000","last_unstale":"2024-09-10T16:11:17.230962+0000","last_undegraded":"2024-09-10T16:11:17.230962+0000","last_fullsized":"2024-09-10T16:11:17.230962+0000","mapping_epoch":50,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:13:14.846691+0000","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17414,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4257,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":140,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.232615+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:11:17.232615+0000","last_peered":"2024-09-10T16:11:17.232615+0000","last_clean":"2024-09-10T16:11:17.232615+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:11:17.232615+0000","last_undegraded":"2024-09-10T16:11:17.232615+0000","last_fullsized":"2024-09-10T16:11:17.232615+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20781","reported_seq":29570,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.232496+0000","last_change":"2024-09-10T16:10:20.907889+0000","last_active":"2024-09-10T16:11:17.232496+0000","last_peered":"2024-09-10T16:11:17.232496+0000","last_clean":"2024-09-10T16:11:17.232496+0000","last_became_active":"2024-09-10T16:10:20.852065+0000","last_became_peered":"2024-09-10T16:10:20.852065+0000","last_unstale":"2024-09-10T16:11:17.232496+0000","last_undegraded":"2024-09-10T16:11:17.232496+0000","last_fullsized":"2024-09-10T16:11:17.232496+0000","mapping_epoch":56,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:34.069549+0000","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18461,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4307,"num_bytes_recovered":8675,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":115,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.233348+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:11:17.233348+0000","last_peered":"2024-09-10T16:11:17.233348+0000","last_clean":"2024-09-10T16:11:17.233348+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:11:17.233348+0000","last_undegraded":"2024-09-10T16:11:17.233348+0000","last_fullsized":"2024-09-10T16:11:17.233348+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'21170","reported_seq":30055,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.231608+0000","last_change":"2024-09-10T16:10:50.728349+0000","last_active":"2024-09-10T16:11:17.231608+0000","last_peered":"2024-09-10T16:11:17.231608+0000","last_clean":"2024-09-10T16:11:17.231608+0000","last_became_active":"2024-09-10T16:08:10.860438+0000","last_became_peered":"2024-09-10T16:08:10.860438+0000","last_unstale":"2024-09-10T16:11:17.231608+0000","last_undegraded":"2024-09-10T16:11:17.231608+0000","last_fullsized":"2024-09-10T16:11:17.231608+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:38:13.781380+0000","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18724,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20838","reported_seq":21009,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:11:17.232728+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:11:17.232728+0000","last_peered":"2024-09-10T16:11:17.232728+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:11:17.232728+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17469,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":105,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.233051+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:11:17.233051+0000","last_peered":"2024-09-10T16:11:17.233051+0000","last_clean":"2024-09-10T16:11:17.233051+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:11:17.233051+0000","last_undegraded":"2024-09-10T16:11:17.233051+0000","last_fullsized":"2024-09-10T16:11:17.233051+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20598","reported_seq":29199,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.233507+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:11:17.233507+0000","last_peered":"2024-09-10T16:11:17.233507+0000","last_clean":"2024-09-10T16:11:17.233507+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:11:17.233507+0000","last_undegraded":"2024-09-10T16:11:17.233507+0000","last_fullsized":"2024-09-10T16:11:17.233507+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16988,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":130,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.232065+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:11:17.232065+0000","last_peered":"2024-09-10T16:11:17.232065+0000","last_clean":"2024-09-10T16:11:17.232065+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:11:17.232065+0000","last_undegraded":"2024-09-10T16:11:17.232065+0000","last_fullsized":"2024-09-10T16:11:17.232065+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'21048","reported_seq":21210,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.319438+0000","last_change":"2024-09-10T16:11:17.315372+0000","last_active":"2024-09-10T16:11:17.319438+0000","last_peered":"2024-09-10T16:11:17.319438+0000","last_clean":"2024-09-10T16:11:17.319438+0000","last_became_active":"2024-09-10T16:11:17.234487+0000","last_became_peered":"2024-09-10T16:11:17.234487+0000","last_unstale":"2024-09-10T16:11:17.319438+0000","last_undegraded":"2024-09-10T16:11:17.319438+0000","last_fullsized":"2024-09-10T16:11:17.319438+0000","mapping_epoch":60,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:19:00.114196+0000","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17303,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4373,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":105,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.233467+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:11:17.233467+0000","last_peered":"2024-09-10T16:11:17.233467+0000","last_clean":"2024-09-10T16:11:17.233467+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:11:17.233467+0000","last_undegraded":"2024-09-10T16:11:17.233467+0000","last_fullsized":"2024-09-10T16:11:17.233467+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":110,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.233556+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:11:17.233556+0000","last_peered":"2024-09-10T16:11:17.233556+0000","last_clean":"2024-09-10T16:11:17.233556+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:11:17.233556+0000","last_undegraded":"2024-09-10T16:11:17.233556+0000","last_fullsized":"2024-09-10T16:11:17.233556+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'21198","reported_seq":24475,"reported_epoch":61,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:11:26.237211+0000","last_change":"2024-09-10T16:11:16.084896+0000","last_active":"2024-09-10T16:11:26.237211+0000","last_peered":"2024-09-10T16:11:26.237211+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:11:26.237211+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2777,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17361,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1548,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2775}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":11319,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292096,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40071,"num_bytes_recovered":33097612,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":191959040,"data_stored":191861489,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161022,"ondisk_log_size":161022,"up":32,"acting":29,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986337,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":484948,"kb_used_data":120068,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93886892,"statfs":{"total":96636764160,"available":96140177408,"internally_reserved":0,"allocated":122949632,"data_stored":122580076,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659282,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":479268,"kb_used_data":532,"kb_used_omap":4,"kb_used_meta":478715,"kb_avail":93892572,"statfs":{"total":96636764160,"available":96145993728,"internally_reserved":0,"allocated":544768,"data_stored":174748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":490204846},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182565,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":547396,"kb_used_data":68716,"kb_used_omap":4,"kb_used_meta":478651,"kb_avail":93824444,"statfs":{"total":96636764160,"available":96076230656,"internally_reserved":0,"allocated":70365184,"data_stored":69987938,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":490139306},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":69853184,"data_stored":69817054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":122073088,"data_stored":122040571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:11:33.833 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:11:34.140 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:11:34.140 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:11:34.154 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":483,"stamp":"2024-09-10T16:11:33.794815+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":10655,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292098,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40737,"num_bytes_recovered":33097631,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161025,"ondisk_log_size":161025,"up":48,"acting":45,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1512772,"kb_used_data":186316,"kb_used_omap":12,"kb_used_meta":1326387,"kb_avail":281602748,"statfs":{"total":289910292480,"available":288361213952,"internally_reserved":0,"allocated":190787584,"data_stored":189670762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1358220290},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1334,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1334,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002145"},"pg_stats":[{"pgid":"2.f","version":"49'20672","reported_seq":29308,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.232365+0000","last_change":"2024-09-10T16:09:57.088531+0000","last_active":"2024-09-10T16:11:17.232365+0000","last_peered":"2024-09-10T16:11:17.232365+0000","last_clean":"2024-09-10T16:11:17.232365+0000","last_became_active":"2024-09-10T16:09:57.005755+0000","last_became_peered":"2024-09-10T16:09:57.005755+0000","last_unstale":"2024-09-10T16:11:17.232365+0000","last_undegraded":"2024-09-10T16:11:17.232365+0000","last_fullsized":"2024-09-10T16:11:17.232365+0000","mapping_epoch":54,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:09:19.329697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20672,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4259,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20844","reported_seq":20992,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.232030+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:11:17.232030+0000","last_peered":"2024-09-10T16:11:17.232030+0000","last_clean":"2024-09-10T16:11:17.232030+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:11:17.232030+0000","last_undegraded":"2024-09-10T16:11:17.232030+0000","last_fullsized":"2024-09-10T16:11:17.232030+0000","mapping_epoch":35,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18949,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20798","reported_seq":29512,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.234237+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:11:17.234237+0000","last_peered":"2024-09-10T16:11:17.234237+0000","last_clean":"2024-09-10T16:11:17.234237+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:11:17.234237+0000","last_undegraded":"2024-09-10T16:11:17.234237+0000","last_fullsized":"2024-09-10T16:11:17.234237+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18802,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20660","reported_seq":20776,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.231774+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:11:17.231774+0000","last_peered":"2024-09-10T16:11:17.231774+0000","last_clean":"2024-09-10T16:11:17.231774+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:11:17.231774+0000","last_undegraded":"2024-09-10T16:11:17.231774+0000","last_fullsized":"2024-09-10T16:11:17.231774+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20660,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20891","reported_seq":21057,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.231216+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:11:17.231216+0000","last_peered":"2024-09-10T16:11:17.231216+0000","last_clean":"2024-09-10T16:11:17.231216+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:11:17.231216+0000","last_undegraded":"2024-09-10T16:11:17.231216+0000","last_fullsized":"2024-09-10T16:11:17.231216+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17602,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'21061","reported_seq":21226,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.233215+0000","last_change":"2024-09-10T16:10:52.241702+0000","last_active":"2024-09-10T16:11:17.233215+0000","last_peered":"2024-09-10T16:11:17.233215+0000","last_clean":"2024-09-10T16:11:17.233215+0000","last_became_active":"2024-09-10T16:10:52.171236+0000","last_became_peered":"2024-09-10T16:10:52.171236+0000","last_unstale":"2024-09-10T16:11:17.233215+0000","last_undegraded":"2024-09-10T16:11:17.233215+0000","last_fullsized":"2024-09-10T16:11:17.233215+0000","mapping_epoch":58,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:04:46.778280+0000","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17673,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4372,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'21082","reported_seq":21256,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:11:17.231941+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:11:17.231941+0000","last_peered":"2024-09-10T16:11:17.231941+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:11:17.231941+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17581,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4235},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'21314","reported_seq":21480,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.231025+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:11:17.231025+0000","last_peered":"2024-09-10T16:11:17.231025+0000","last_clean":"2024-09-10T16:11:17.231025+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:11:17.231025+0000","last_undegraded":"2024-09-10T16:11:17.231025+0000","last_fullsized":"2024-09-10T16:11:17.231025+0000","mapping_epoch":34,"log_start":"45'11300","ondisk_log_start":"45'11300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18161,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.232985+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:11:17.232985+0000","last_peered":"2024-09-10T16:11:17.232985+0000","last_clean":"2024-09-10T16:11:17.232985+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:11:17.232985+0000","last_undegraded":"2024-09-10T16:11:17.232985+0000","last_fullsized":"2024-09-10T16:11:17.232985+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20457","reported_seq":20636,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.233353+0000","last_change":"2024-09-10T16:09:33.012840+0000","last_active":"2024-09-10T16:11:17.233353+0000","last_peered":"2024-09-10T16:11:17.233353+0000","last_clean":"2024-09-10T16:11:17.233353+0000","last_became_active":"2024-09-10T16:09:32.934056+0000","last_became_peered":"2024-09-10T16:09:32.934056+0000","last_unstale":"2024-09-10T16:11:17.233353+0000","last_undegraded":"2024-09-10T16:11:17.233353+0000","last_fullsized":"2024-09-10T16:11:17.233353+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:08:30.481235+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18276,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":4579,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":130,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.232408+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:11:17.232408+0000","last_peered":"2024-09-10T16:11:17.232408+0000","last_clean":"2024-09-10T16:11:17.232408+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:11:17.232408+0000","last_undegraded":"2024-09-10T16:11:17.232408+0000","last_fullsized":"2024-09-10T16:11:17.232408+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20910","reported_seq":29598,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.230962+0000","last_change":"2024-09-10T16:09:08.909580+0000","last_active":"2024-09-10T16:11:17.230962+0000","last_peered":"2024-09-10T16:11:17.230962+0000","last_clean":"2024-09-10T16:11:17.230962+0000","last_became_active":"2024-09-10T16:09:08.854941+0000","last_became_peered":"2024-09-10T16:09:08.854941+0000","last_unstale":"2024-09-10T16:11:17.230962+0000","last_undegraded":"2024-09-10T16:11:17.230962+0000","last_fullsized":"2024-09-10T16:11:17.230962+0000","mapping_epoch":50,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:13:14.846691+0000","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17414,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4257,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":140,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.232615+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:11:17.232615+0000","last_peered":"2024-09-10T16:11:17.232615+0000","last_clean":"2024-09-10T16:11:17.232615+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:11:17.232615+0000","last_undegraded":"2024-09-10T16:11:17.232615+0000","last_fullsized":"2024-09-10T16:11:17.232615+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20781","reported_seq":29570,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.232496+0000","last_change":"2024-09-10T16:10:20.907889+0000","last_active":"2024-09-10T16:11:17.232496+0000","last_peered":"2024-09-10T16:11:17.232496+0000","last_clean":"2024-09-10T16:11:17.232496+0000","last_became_active":"2024-09-10T16:10:20.852065+0000","last_became_peered":"2024-09-10T16:10:20.852065+0000","last_unstale":"2024-09-10T16:11:17.232496+0000","last_undegraded":"2024-09-10T16:11:17.232496+0000","last_fullsized":"2024-09-10T16:11:17.232496+0000","mapping_epoch":56,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:34.069549+0000","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18461,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4307,"num_bytes_recovered":8675,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":115,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.233348+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:11:17.233348+0000","last_peered":"2024-09-10T16:11:17.233348+0000","last_clean":"2024-09-10T16:11:17.233348+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:11:17.233348+0000","last_undegraded":"2024-09-10T16:11:17.233348+0000","last_fullsized":"2024-09-10T16:11:17.233348+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'21170","reported_seq":30055,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.231608+0000","last_change":"2024-09-10T16:10:50.728349+0000","last_active":"2024-09-10T16:11:17.231608+0000","last_peered":"2024-09-10T16:11:17.231608+0000","last_clean":"2024-09-10T16:11:17.231608+0000","last_became_active":"2024-09-10T16:08:10.860438+0000","last_became_peered":"2024-09-10T16:08:10.860438+0000","last_unstale":"2024-09-10T16:11:17.231608+0000","last_undegraded":"2024-09-10T16:11:17.231608+0000","last_fullsized":"2024-09-10T16:11:17.231608+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:38:13.781380+0000","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18724,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20838","reported_seq":21009,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:11:17.232728+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:11:17.232728+0000","last_peered":"2024-09-10T16:11:17.232728+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:11:17.232728+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17469,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":105,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.233051+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:11:17.233051+0000","last_peered":"2024-09-10T16:11:17.233051+0000","last_clean":"2024-09-10T16:11:17.233051+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:11:17.233051+0000","last_undegraded":"2024-09-10T16:11:17.233051+0000","last_fullsized":"2024-09-10T16:11:17.233051+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20598","reported_seq":29199,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.233507+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:11:17.233507+0000","last_peered":"2024-09-10T16:11:17.233507+0000","last_clean":"2024-09-10T16:11:17.233507+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:11:17.233507+0000","last_undegraded":"2024-09-10T16:11:17.233507+0000","last_fullsized":"2024-09-10T16:11:17.233507+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16988,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":130,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.232065+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:11:17.232065+0000","last_peered":"2024-09-10T16:11:17.232065+0000","last_clean":"2024-09-10T16:11:17.232065+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:11:17.232065+0000","last_undegraded":"2024-09-10T16:11:17.232065+0000","last_fullsized":"2024-09-10T16:11:17.232065+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'21048","reported_seq":21210,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.319438+0000","last_change":"2024-09-10T16:11:17.315372+0000","last_active":"2024-09-10T16:11:17.319438+0000","last_peered":"2024-09-10T16:11:17.319438+0000","last_clean":"2024-09-10T16:11:17.319438+0000","last_became_active":"2024-09-10T16:11:17.234487+0000","last_became_peered":"2024-09-10T16:11:17.234487+0000","last_unstale":"2024-09-10T16:11:17.319438+0000","last_undegraded":"2024-09-10T16:11:17.319438+0000","last_fullsized":"2024-09-10T16:11:17.319438+0000","mapping_epoch":60,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:19:00.114196+0000","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17303,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4373,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":105,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.233467+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:11:17.233467+0000","last_peered":"2024-09-10T16:11:17.233467+0000","last_clean":"2024-09-10T16:11:17.233467+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:11:17.233467+0000","last_undegraded":"2024-09-10T16:11:17.233467+0000","last_fullsized":"2024-09-10T16:11:17.233467+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":110,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.233556+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:11:17.233556+0000","last_peered":"2024-09-10T16:11:17.233556+0000","last_clean":"2024-09-10T16:11:17.233556+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:11:17.233556+0000","last_undegraded":"2024-09-10T16:11:17.233556+0000","last_fullsized":"2024-09-10T16:11:17.233556+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'21198","reported_seq":25803,"reported_epoch":61,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:11:31.237745+0000","last_change":"2024-09-10T16:11:16.084896+0000","last_active":"2024-09-10T16:11:31.237745+0000","last_peered":"2024-09-10T16:11:31.237745+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:11:31.237745+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2113,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17361,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2212,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2111}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":10655,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292096,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40735,"num_bytes_recovered":33097612,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":188887040,"data_stored":188789489,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161022,"ondisk_log_size":161022,"up":32,"acting":29,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986338,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":481948,"kb_used_data":117068,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93889892,"statfs":{"total":96636764160,"available":96143249408,"internally_reserved":0,"allocated":119877632,"data_stored":119508076,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659283,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":483428,"kb_used_data":532,"kb_used_omap":4,"kb_used_meta":482875,"kb_avail":93888412,"statfs":{"total":96636764160,"available":96141733888,"internally_reserved":0,"allocated":544768,"data_stored":174748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":494464686},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182566,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":547396,"kb_used_data":68716,"kb_used_omap":4,"kb_used_meta":478651,"kb_avail":93824444,"statfs":{"total":96636764160,"available":96076230656,"internally_reserved":0,"allocated":70365184,"data_stored":69987938,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":490139306},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":69853184,"data_stored":69817054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":119001088,"data_stored":118968571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:11:34.155 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:11:34.479 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:11:34.480 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:11:34.493 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":483,"stamp":"2024-09-10T16:11:33.794815+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":10655,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292098,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40737,"num_bytes_recovered":33097631,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161025,"ondisk_log_size":161025,"up":48,"acting":45,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1512772,"kb_used_data":186316,"kb_used_omap":12,"kb_used_meta":1326387,"kb_avail":281602748,"statfs":{"total":289910292480,"available":288361213952,"internally_reserved":0,"allocated":190787584,"data_stored":189670762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1358220290},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1334,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1334,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002145"},"pg_stats":[{"pgid":"2.f","version":"49'20672","reported_seq":29308,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.232365+0000","last_change":"2024-09-10T16:09:57.088531+0000","last_active":"2024-09-10T16:11:17.232365+0000","last_peered":"2024-09-10T16:11:17.232365+0000","last_clean":"2024-09-10T16:11:17.232365+0000","last_became_active":"2024-09-10T16:09:57.005755+0000","last_became_peered":"2024-09-10T16:09:57.005755+0000","last_unstale":"2024-09-10T16:11:17.232365+0000","last_undegraded":"2024-09-10T16:11:17.232365+0000","last_fullsized":"2024-09-10T16:11:17.232365+0000","mapping_epoch":54,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:09:19.329697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20672,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4259,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20844","reported_seq":20992,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.232030+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:11:17.232030+0000","last_peered":"2024-09-10T16:11:17.232030+0000","last_clean":"2024-09-10T16:11:17.232030+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:11:17.232030+0000","last_undegraded":"2024-09-10T16:11:17.232030+0000","last_fullsized":"2024-09-10T16:11:17.232030+0000","mapping_epoch":35,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18949,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20798","reported_seq":29512,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.234237+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:11:17.234237+0000","last_peered":"2024-09-10T16:11:17.234237+0000","last_clean":"2024-09-10T16:11:17.234237+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:11:17.234237+0000","last_undegraded":"2024-09-10T16:11:17.234237+0000","last_fullsized":"2024-09-10T16:11:17.234237+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18802,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20660","reported_seq":20776,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.231774+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:11:17.231774+0000","last_peered":"2024-09-10T16:11:17.231774+0000","last_clean":"2024-09-10T16:11:17.231774+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:11:17.231774+0000","last_undegraded":"2024-09-10T16:11:17.231774+0000","last_fullsized":"2024-09-10T16:11:17.231774+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20660,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20891","reported_seq":21057,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.231216+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:11:17.231216+0000","last_peered":"2024-09-10T16:11:17.231216+0000","last_clean":"2024-09-10T16:11:17.231216+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:11:17.231216+0000","last_undegraded":"2024-09-10T16:11:17.231216+0000","last_fullsized":"2024-09-10T16:11:17.231216+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17602,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'21061","reported_seq":21226,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.233215+0000","last_change":"2024-09-10T16:10:52.241702+0000","last_active":"2024-09-10T16:11:17.233215+0000","last_peered":"2024-09-10T16:11:17.233215+0000","last_clean":"2024-09-10T16:11:17.233215+0000","last_became_active":"2024-09-10T16:10:52.171236+0000","last_became_peered":"2024-09-10T16:10:52.171236+0000","last_unstale":"2024-09-10T16:11:17.233215+0000","last_undegraded":"2024-09-10T16:11:17.233215+0000","last_fullsized":"2024-09-10T16:11:17.233215+0000","mapping_epoch":58,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:04:46.778280+0000","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17673,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4372,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'21082","reported_seq":21256,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:11:17.231941+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:11:17.231941+0000","last_peered":"2024-09-10T16:11:17.231941+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:11:17.231941+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17581,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4235},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'21314","reported_seq":21480,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.231025+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:11:17.231025+0000","last_peered":"2024-09-10T16:11:17.231025+0000","last_clean":"2024-09-10T16:11:17.231025+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:11:17.231025+0000","last_undegraded":"2024-09-10T16:11:17.231025+0000","last_fullsized":"2024-09-10T16:11:17.231025+0000","mapping_epoch":34,"log_start":"45'11300","ondisk_log_start":"45'11300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18161,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.232985+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:11:17.232985+0000","last_peered":"2024-09-10T16:11:17.232985+0000","last_clean":"2024-09-10T16:11:17.232985+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:11:17.232985+0000","last_undegraded":"2024-09-10T16:11:17.232985+0000","last_fullsized":"2024-09-10T16:11:17.232985+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20457","reported_seq":20636,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.233353+0000","last_change":"2024-09-10T16:09:33.012840+0000","last_active":"2024-09-10T16:11:17.233353+0000","last_peered":"2024-09-10T16:11:17.233353+0000","last_clean":"2024-09-10T16:11:17.233353+0000","last_became_active":"2024-09-10T16:09:32.934056+0000","last_became_peered":"2024-09-10T16:09:32.934056+0000","last_unstale":"2024-09-10T16:11:17.233353+0000","last_undegraded":"2024-09-10T16:11:17.233353+0000","last_fullsized":"2024-09-10T16:11:17.233353+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:08:30.481235+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18276,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":4579,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":130,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.232408+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:11:17.232408+0000","last_peered":"2024-09-10T16:11:17.232408+0000","last_clean":"2024-09-10T16:11:17.232408+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:11:17.232408+0000","last_undegraded":"2024-09-10T16:11:17.232408+0000","last_fullsized":"2024-09-10T16:11:17.232408+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20910","reported_seq":29598,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.230962+0000","last_change":"2024-09-10T16:09:08.909580+0000","last_active":"2024-09-10T16:11:17.230962+0000","last_peered":"2024-09-10T16:11:17.230962+0000","last_clean":"2024-09-10T16:11:17.230962+0000","last_became_active":"2024-09-10T16:09:08.854941+0000","last_became_peered":"2024-09-10T16:09:08.854941+0000","last_unstale":"2024-09-10T16:11:17.230962+0000","last_undegraded":"2024-09-10T16:11:17.230962+0000","last_fullsized":"2024-09-10T16:11:17.230962+0000","mapping_epoch":50,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:13:14.846691+0000","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17414,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4257,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":140,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.232615+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:11:17.232615+0000","last_peered":"2024-09-10T16:11:17.232615+0000","last_clean":"2024-09-10T16:11:17.232615+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:11:17.232615+0000","last_undegraded":"2024-09-10T16:11:17.232615+0000","last_fullsized":"2024-09-10T16:11:17.232615+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20781","reported_seq":29570,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.232496+0000","last_change":"2024-09-10T16:10:20.907889+0000","last_active":"2024-09-10T16:11:17.232496+0000","last_peered":"2024-09-10T16:11:17.232496+0000","last_clean":"2024-09-10T16:11:17.232496+0000","last_became_active":"2024-09-10T16:10:20.852065+0000","last_became_peered":"2024-09-10T16:10:20.852065+0000","last_unstale":"2024-09-10T16:11:17.232496+0000","last_undegraded":"2024-09-10T16:11:17.232496+0000","last_fullsized":"2024-09-10T16:11:17.232496+0000","mapping_epoch":56,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:34.069549+0000","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18461,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4307,"num_bytes_recovered":8675,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":115,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.233348+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:11:17.233348+0000","last_peered":"2024-09-10T16:11:17.233348+0000","last_clean":"2024-09-10T16:11:17.233348+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:11:17.233348+0000","last_undegraded":"2024-09-10T16:11:17.233348+0000","last_fullsized":"2024-09-10T16:11:17.233348+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'21170","reported_seq":30055,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.231608+0000","last_change":"2024-09-10T16:10:50.728349+0000","last_active":"2024-09-10T16:11:17.231608+0000","last_peered":"2024-09-10T16:11:17.231608+0000","last_clean":"2024-09-10T16:11:17.231608+0000","last_became_active":"2024-09-10T16:08:10.860438+0000","last_became_peered":"2024-09-10T16:08:10.860438+0000","last_unstale":"2024-09-10T16:11:17.231608+0000","last_undegraded":"2024-09-10T16:11:17.231608+0000","last_fullsized":"2024-09-10T16:11:17.231608+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:38:13.781380+0000","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18724,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20838","reported_seq":21009,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:11:17.232728+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:11:17.232728+0000","last_peered":"2024-09-10T16:11:17.232728+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:11:17.232728+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17469,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":105,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.233051+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:11:17.233051+0000","last_peered":"2024-09-10T16:11:17.233051+0000","last_clean":"2024-09-10T16:11:17.233051+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:11:17.233051+0000","last_undegraded":"2024-09-10T16:11:17.233051+0000","last_fullsized":"2024-09-10T16:11:17.233051+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20598","reported_seq":29199,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.233507+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:11:17.233507+0000","last_peered":"2024-09-10T16:11:17.233507+0000","last_clean":"2024-09-10T16:11:17.233507+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:11:17.233507+0000","last_undegraded":"2024-09-10T16:11:17.233507+0000","last_fullsized":"2024-09-10T16:11:17.233507+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16988,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":130,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.232065+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:11:17.232065+0000","last_peered":"2024-09-10T16:11:17.232065+0000","last_clean":"2024-09-10T16:11:17.232065+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:11:17.232065+0000","last_undegraded":"2024-09-10T16:11:17.232065+0000","last_fullsized":"2024-09-10T16:11:17.232065+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'21048","reported_seq":21210,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.319438+0000","last_change":"2024-09-10T16:11:17.315372+0000","last_active":"2024-09-10T16:11:17.319438+0000","last_peered":"2024-09-10T16:11:17.319438+0000","last_clean":"2024-09-10T16:11:17.319438+0000","last_became_active":"2024-09-10T16:11:17.234487+0000","last_became_peered":"2024-09-10T16:11:17.234487+0000","last_unstale":"2024-09-10T16:11:17.319438+0000","last_undegraded":"2024-09-10T16:11:17.319438+0000","last_fullsized":"2024-09-10T16:11:17.319438+0000","mapping_epoch":60,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:19:00.114196+0000","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17303,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4373,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":105,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.233467+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:11:17.233467+0000","last_peered":"2024-09-10T16:11:17.233467+0000","last_clean":"2024-09-10T16:11:17.233467+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:11:17.233467+0000","last_undegraded":"2024-09-10T16:11:17.233467+0000","last_fullsized":"2024-09-10T16:11:17.233467+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":110,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.233556+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:11:17.233556+0000","last_peered":"2024-09-10T16:11:17.233556+0000","last_clean":"2024-09-10T16:11:17.233556+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:11:17.233556+0000","last_undegraded":"2024-09-10T16:11:17.233556+0000","last_fullsized":"2024-09-10T16:11:17.233556+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'21198","reported_seq":25803,"reported_epoch":61,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:11:31.237745+0000","last_change":"2024-09-10T16:11:16.084896+0000","last_active":"2024-09-10T16:11:31.237745+0000","last_peered":"2024-09-10T16:11:31.237745+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:11:31.237745+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2113,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17361,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2212,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2111}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":10655,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292096,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40735,"num_bytes_recovered":33097612,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":188887040,"data_stored":188789489,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161022,"ondisk_log_size":161022,"up":32,"acting":29,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986338,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":481948,"kb_used_data":117068,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93889892,"statfs":{"total":96636764160,"available":96143249408,"internally_reserved":0,"allocated":119877632,"data_stored":119508076,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659283,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":483428,"kb_used_data":532,"kb_used_omap":4,"kb_used_meta":482875,"kb_avail":93888412,"statfs":{"total":96636764160,"available":96141733888,"internally_reserved":0,"allocated":544768,"data_stored":174748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":494464686},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182566,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":547396,"kb_used_data":68716,"kb_used_omap":4,"kb_used_meta":478651,"kb_avail":93824444,"statfs":{"total":96636764160,"available":96076230656,"internally_reserved":0,"allocated":70365184,"data_stored":69987938,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":490139306},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":69853184,"data_stored":69817054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":119001088,"data_stored":118968571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:11:34.494 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-10T16:11:34.494 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "49'21082", 'reported_seq': 21256, 'reported_epoch': 61, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:11:17.231941+0000', 'last_change': '2024-09-10T15:59:16.331899+0000', 'last_active': '2024-09-10T16:11:17.231941+0000', 'last_peered': '2024-09-10T16:11:17.231941+0000', 'last_clean': '2024-09-10T15:59:14.139436+0000', 'last_became_active': '2024-09-10T15:59:16.318715+0000', 'last_became_peered': '2024-09-10T15:59:16.318715+0000', 'last_unstale': '2024-09-10T16:11:17.231941+0000', 'last_undegraded': '2024-09-10T15:59:15.877524+0000', 'last_fullsized': '2024-09-10T15:59:15.769042+0000', 'mapping_epoch': 42, 'log_start': "45'11000", 'ondisk_log_start': "45'11000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3537", 'last_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'objects_scrubbed': 3497, 'log_size': 10082, 'log_dups_size': 0, 'ondisk_log_size': 10082, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1224, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4237, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 17581, 'num_write_kb': 28172, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4235}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:11:34.494 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-10T16:11:34.494 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "49'20838", 'reported_seq': 21009, 'reported_epoch': 61, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:11:17.232728+0000', 'last_change': '2024-09-10T15:59:16.315641+0000', 'last_active': '2024-09-10T16:11:17.232728+0000', 'last_peered': '2024-09-10T16:11:17.232728+0000', 'last_clean': '2024-09-10T15:59:14.117001+0000', 'last_became_active': '2024-09-10T15:59:16.302245+0000', 'last_became_peered': '2024-09-10T15:59:16.302245+0000', 'last_unstale': '2024-09-10T16:11:17.232728+0000', 'last_undegraded': '2024-09-10T15:59:15.845012+0000', 'last_fullsized': '2024-09-10T15:59:15.770337+0000', 'mapping_epoch': 42, 'log_start': "45'10800", 'ondisk_log_start': "45'10800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3404", 'last_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'objects_scrubbed': 3367, 'log_size': 10038, 'log_dups_size': 0, 'ondisk_log_size': 10038, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 983, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17469, 'num_write_kb': 28204, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4304}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:11:34.494 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-10T16:11:34.494 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "49'21198", 'reported_seq': 25803, 'reported_epoch': 61, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:11:31.237745+0000', 'last_change': '2024-09-10T16:11:16.084896+0000', 'last_active': '2024-09-10T16:11:31.237745+0000', 'last_peered': '2024-09-10T16:11:31.237745+0000', 'last_clean': '2024-09-10T15:59:14.129780+0000', 'last_became_active': '2024-09-10T15:59:16.300932+0000', 'last_became_peered': '2024-09-10T15:59:16.300932+0000', 'last_unstale': '2024-09-10T16:11:31.237745+0000', 'last_undegraded': '2024-09-10T15:59:15.887823+0000', 'last_fullsized': '2024-09-10T15:59:15.768513+0000', 'mapping_epoch': 42, 'log_start': "45'11100", 'ondisk_log_start': "45'11100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3880", 'last_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'objects_scrubbed': 3827, 'log_size': 10098, 'log_dups_size': 0, 'ondisk_log_size': 10098, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1397, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2113, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17361, 'num_write_kb': 27072, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2212, 'num_bytes_recovered': 483, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2111}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:11:34.495 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T16:11:34.890 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:11:34.890 INFO:teuthology.orchestra.run.smithi115.stdout:{"fsid":"7effee29-df05-4351-9741-ee203564ed81","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 10655/24 objects degraded (44395.833%), 3 pgs degraded, 3 pgs undersized","count":6},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":858,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":61,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983954,"num_in_osds":3,"osd_in_since":1725983947,"num_remapped_pgs":3},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":21},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":2},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":12,"data_bytes":4849,"bytes_used":1549078528,"bytes_avail":288361213952,"bytes_total":289910292480,"degraded_objects":10655,"degraded_total":24,"degraded_ratio":443.95833333333326,"recovering_objects_per_sec":111,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1334,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-10T15:57:14:967120+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T16:00:13.560776+0000","services":{}},"progress_events":{}} 2024-09-10T16:11:34.906 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-10T16:11:34.906 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:11:35.209 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:11:35.209 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:11:35.223 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":483,"stamp":"2024-09-10T16:11:33.794815+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":10655,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292098,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40737,"num_bytes_recovered":33097631,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161025,"ondisk_log_size":161025,"up":48,"acting":45,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1512772,"kb_used_data":186316,"kb_used_omap":12,"kb_used_meta":1326387,"kb_avail":281602748,"statfs":{"total":289910292480,"available":288361213952,"internally_reserved":0,"allocated":190787584,"data_stored":189670762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1358220290},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1334,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1334,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002145"},"pg_stats":[{"pgid":"2.f","version":"49'20672","reported_seq":29308,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.232365+0000","last_change":"2024-09-10T16:09:57.088531+0000","last_active":"2024-09-10T16:11:17.232365+0000","last_peered":"2024-09-10T16:11:17.232365+0000","last_clean":"2024-09-10T16:11:17.232365+0000","last_became_active":"2024-09-10T16:09:57.005755+0000","last_became_peered":"2024-09-10T16:09:57.005755+0000","last_unstale":"2024-09-10T16:11:17.232365+0000","last_undegraded":"2024-09-10T16:11:17.232365+0000","last_fullsized":"2024-09-10T16:11:17.232365+0000","mapping_epoch":54,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:09:19.329697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20672,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4259,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20844","reported_seq":20992,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.232030+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:11:17.232030+0000","last_peered":"2024-09-10T16:11:17.232030+0000","last_clean":"2024-09-10T16:11:17.232030+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:11:17.232030+0000","last_undegraded":"2024-09-10T16:11:17.232030+0000","last_fullsized":"2024-09-10T16:11:17.232030+0000","mapping_epoch":35,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18949,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20798","reported_seq":29512,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.234237+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:11:17.234237+0000","last_peered":"2024-09-10T16:11:17.234237+0000","last_clean":"2024-09-10T16:11:17.234237+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:11:17.234237+0000","last_undegraded":"2024-09-10T16:11:17.234237+0000","last_fullsized":"2024-09-10T16:11:17.234237+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18802,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20660","reported_seq":20776,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.231774+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:11:17.231774+0000","last_peered":"2024-09-10T16:11:17.231774+0000","last_clean":"2024-09-10T16:11:17.231774+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:11:17.231774+0000","last_undegraded":"2024-09-10T16:11:17.231774+0000","last_fullsized":"2024-09-10T16:11:17.231774+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20660,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20891","reported_seq":21057,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.231216+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:11:17.231216+0000","last_peered":"2024-09-10T16:11:17.231216+0000","last_clean":"2024-09-10T16:11:17.231216+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:11:17.231216+0000","last_undegraded":"2024-09-10T16:11:17.231216+0000","last_fullsized":"2024-09-10T16:11:17.231216+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17602,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'21061","reported_seq":21226,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.233215+0000","last_change":"2024-09-10T16:10:52.241702+0000","last_active":"2024-09-10T16:11:17.233215+0000","last_peered":"2024-09-10T16:11:17.233215+0000","last_clean":"2024-09-10T16:11:17.233215+0000","last_became_active":"2024-09-10T16:10:52.171236+0000","last_became_peered":"2024-09-10T16:10:52.171236+0000","last_unstale":"2024-09-10T16:11:17.233215+0000","last_undegraded":"2024-09-10T16:11:17.233215+0000","last_fullsized":"2024-09-10T16:11:17.233215+0000","mapping_epoch":58,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:04:46.778280+0000","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17673,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4372,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'21082","reported_seq":21256,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:11:17.231941+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:11:17.231941+0000","last_peered":"2024-09-10T16:11:17.231941+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:11:17.231941+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17581,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4235},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'21314","reported_seq":21480,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.231025+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:11:17.231025+0000","last_peered":"2024-09-10T16:11:17.231025+0000","last_clean":"2024-09-10T16:11:17.231025+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:11:17.231025+0000","last_undegraded":"2024-09-10T16:11:17.231025+0000","last_fullsized":"2024-09-10T16:11:17.231025+0000","mapping_epoch":34,"log_start":"45'11300","ondisk_log_start":"45'11300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18161,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.232985+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:11:17.232985+0000","last_peered":"2024-09-10T16:11:17.232985+0000","last_clean":"2024-09-10T16:11:17.232985+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:11:17.232985+0000","last_undegraded":"2024-09-10T16:11:17.232985+0000","last_fullsized":"2024-09-10T16:11:17.232985+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20457","reported_seq":20636,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.233353+0000","last_change":"2024-09-10T16:09:33.012840+0000","last_active":"2024-09-10T16:11:17.233353+0000","last_peered":"2024-09-10T16:11:17.233353+0000","last_clean":"2024-09-10T16:11:17.233353+0000","last_became_active":"2024-09-10T16:09:32.934056+0000","last_became_peered":"2024-09-10T16:09:32.934056+0000","last_unstale":"2024-09-10T16:11:17.233353+0000","last_undegraded":"2024-09-10T16:11:17.233353+0000","last_fullsized":"2024-09-10T16:11:17.233353+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:08:30.481235+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18276,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":4579,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":130,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.232408+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:11:17.232408+0000","last_peered":"2024-09-10T16:11:17.232408+0000","last_clean":"2024-09-10T16:11:17.232408+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:11:17.232408+0000","last_undegraded":"2024-09-10T16:11:17.232408+0000","last_fullsized":"2024-09-10T16:11:17.232408+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20910","reported_seq":29598,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.230962+0000","last_change":"2024-09-10T16:09:08.909580+0000","last_active":"2024-09-10T16:11:17.230962+0000","last_peered":"2024-09-10T16:11:17.230962+0000","last_clean":"2024-09-10T16:11:17.230962+0000","last_became_active":"2024-09-10T16:09:08.854941+0000","last_became_peered":"2024-09-10T16:09:08.854941+0000","last_unstale":"2024-09-10T16:11:17.230962+0000","last_undegraded":"2024-09-10T16:11:17.230962+0000","last_fullsized":"2024-09-10T16:11:17.230962+0000","mapping_epoch":50,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:13:14.846691+0000","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17414,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4257,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":140,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.232615+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:11:17.232615+0000","last_peered":"2024-09-10T16:11:17.232615+0000","last_clean":"2024-09-10T16:11:17.232615+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:11:17.232615+0000","last_undegraded":"2024-09-10T16:11:17.232615+0000","last_fullsized":"2024-09-10T16:11:17.232615+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20781","reported_seq":29570,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.232496+0000","last_change":"2024-09-10T16:10:20.907889+0000","last_active":"2024-09-10T16:11:17.232496+0000","last_peered":"2024-09-10T16:11:17.232496+0000","last_clean":"2024-09-10T16:11:17.232496+0000","last_became_active":"2024-09-10T16:10:20.852065+0000","last_became_peered":"2024-09-10T16:10:20.852065+0000","last_unstale":"2024-09-10T16:11:17.232496+0000","last_undegraded":"2024-09-10T16:11:17.232496+0000","last_fullsized":"2024-09-10T16:11:17.232496+0000","mapping_epoch":56,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:34.069549+0000","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18461,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4307,"num_bytes_recovered":8675,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":115,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.233348+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:11:17.233348+0000","last_peered":"2024-09-10T16:11:17.233348+0000","last_clean":"2024-09-10T16:11:17.233348+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:11:17.233348+0000","last_undegraded":"2024-09-10T16:11:17.233348+0000","last_fullsized":"2024-09-10T16:11:17.233348+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'21170","reported_seq":30055,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.231608+0000","last_change":"2024-09-10T16:10:50.728349+0000","last_active":"2024-09-10T16:11:17.231608+0000","last_peered":"2024-09-10T16:11:17.231608+0000","last_clean":"2024-09-10T16:11:17.231608+0000","last_became_active":"2024-09-10T16:08:10.860438+0000","last_became_peered":"2024-09-10T16:08:10.860438+0000","last_unstale":"2024-09-10T16:11:17.231608+0000","last_undegraded":"2024-09-10T16:11:17.231608+0000","last_fullsized":"2024-09-10T16:11:17.231608+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:38:13.781380+0000","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18724,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20838","reported_seq":21009,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:11:17.232728+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:11:17.232728+0000","last_peered":"2024-09-10T16:11:17.232728+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:11:17.232728+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17469,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":105,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.233051+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:11:17.233051+0000","last_peered":"2024-09-10T16:11:17.233051+0000","last_clean":"2024-09-10T16:11:17.233051+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:11:17.233051+0000","last_undegraded":"2024-09-10T16:11:17.233051+0000","last_fullsized":"2024-09-10T16:11:17.233051+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20598","reported_seq":29199,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.233507+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:11:17.233507+0000","last_peered":"2024-09-10T16:11:17.233507+0000","last_clean":"2024-09-10T16:11:17.233507+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:11:17.233507+0000","last_undegraded":"2024-09-10T16:11:17.233507+0000","last_fullsized":"2024-09-10T16:11:17.233507+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16988,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":130,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.232065+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:11:17.232065+0000","last_peered":"2024-09-10T16:11:17.232065+0000","last_clean":"2024-09-10T16:11:17.232065+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:11:17.232065+0000","last_undegraded":"2024-09-10T16:11:17.232065+0000","last_fullsized":"2024-09-10T16:11:17.232065+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'21048","reported_seq":21210,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.319438+0000","last_change":"2024-09-10T16:11:17.315372+0000","last_active":"2024-09-10T16:11:17.319438+0000","last_peered":"2024-09-10T16:11:17.319438+0000","last_clean":"2024-09-10T16:11:17.319438+0000","last_became_active":"2024-09-10T16:11:17.234487+0000","last_became_peered":"2024-09-10T16:11:17.234487+0000","last_unstale":"2024-09-10T16:11:17.319438+0000","last_undegraded":"2024-09-10T16:11:17.319438+0000","last_fullsized":"2024-09-10T16:11:17.319438+0000","mapping_epoch":60,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:19:00.114196+0000","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17303,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4373,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":105,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.233467+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:11:17.233467+0000","last_peered":"2024-09-10T16:11:17.233467+0000","last_clean":"2024-09-10T16:11:17.233467+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:11:17.233467+0000","last_undegraded":"2024-09-10T16:11:17.233467+0000","last_fullsized":"2024-09-10T16:11:17.233467+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":110,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.233556+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:11:17.233556+0000","last_peered":"2024-09-10T16:11:17.233556+0000","last_clean":"2024-09-10T16:11:17.233556+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:11:17.233556+0000","last_undegraded":"2024-09-10T16:11:17.233556+0000","last_fullsized":"2024-09-10T16:11:17.233556+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'21198","reported_seq":25803,"reported_epoch":61,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:11:31.237745+0000","last_change":"2024-09-10T16:11:16.084896+0000","last_active":"2024-09-10T16:11:31.237745+0000","last_peered":"2024-09-10T16:11:31.237745+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:11:31.237745+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2113,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17361,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2212,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2111}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":10655,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292096,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40735,"num_bytes_recovered":33097612,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":188887040,"data_stored":188789489,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161022,"ondisk_log_size":161022,"up":32,"acting":29,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986338,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":481948,"kb_used_data":117068,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93889892,"statfs":{"total":96636764160,"available":96143249408,"internally_reserved":0,"allocated":119877632,"data_stored":119508076,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659283,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":483428,"kb_used_data":532,"kb_used_omap":4,"kb_used_meta":482875,"kb_avail":93888412,"statfs":{"total":96636764160,"available":96141733888,"internally_reserved":0,"allocated":544768,"data_stored":174748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":494464686},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182566,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":547396,"kb_used_data":68716,"kb_used_omap":4,"kb_used_meta":478651,"kb_avail":93824444,"statfs":{"total":96636764160,"available":96076230656,"internally_reserved":0,"allocated":70365184,"data_stored":69987938,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":490139306},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":69853184,"data_stored":69817054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":119001088,"data_stored":118968571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:11:38.225 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:11:38.531 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:11:38.532 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:11:38.544 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":485,"stamp":"2024-09-10T16:11:37.795397+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":9877,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292098,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":41515,"num_bytes_recovered":33097631,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161025,"ondisk_log_size":161025,"up":48,"acting":45,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1513932,"kb_used_data":183316,"kb_used_omap":12,"kb_used_meta":1330547,"kb_avail":281601588,"statfs":{"total":289910292480,"available":288360026112,"internally_reserved":0,"allocated":187715584,"data_stored":186598762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1362480130},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2112,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2112,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001969"},"pg_stats":[{"pgid":"2.f","version":"49'20672","reported_seq":29309,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.232412+0000","last_change":"2024-09-10T16:09:57.088531+0000","last_active":"2024-09-10T16:11:17.232412+0000","last_peered":"2024-09-10T16:11:17.232412+0000","last_clean":"2024-09-10T16:11:17.232412+0000","last_became_active":"2024-09-10T16:09:57.005755+0000","last_became_peered":"2024-09-10T16:09:57.005755+0000","last_unstale":"2024-09-10T16:11:17.232412+0000","last_undegraded":"2024-09-10T16:11:17.232412+0000","last_fullsized":"2024-09-10T16:11:17.232412+0000","mapping_epoch":54,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:09:19.329697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20672,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4259,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20844","reported_seq":20992,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.232030+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:11:17.232030+0000","last_peered":"2024-09-10T16:11:17.232030+0000","last_clean":"2024-09-10T16:11:17.232030+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:11:17.232030+0000","last_undegraded":"2024-09-10T16:11:17.232030+0000","last_fullsized":"2024-09-10T16:11:17.232030+0000","mapping_epoch":35,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18949,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20798","reported_seq":29512,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.234237+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:11:17.234237+0000","last_peered":"2024-09-10T16:11:17.234237+0000","last_clean":"2024-09-10T16:11:17.234237+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:11:17.234237+0000","last_undegraded":"2024-09-10T16:11:17.234237+0000","last_fullsized":"2024-09-10T16:11:17.234237+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18802,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20660","reported_seq":20776,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.231774+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:11:17.231774+0000","last_peered":"2024-09-10T16:11:17.231774+0000","last_clean":"2024-09-10T16:11:17.231774+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:11:17.231774+0000","last_undegraded":"2024-09-10T16:11:17.231774+0000","last_fullsized":"2024-09-10T16:11:17.231774+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20660,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20891","reported_seq":21057,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.231216+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:11:17.231216+0000","last_peered":"2024-09-10T16:11:17.231216+0000","last_clean":"2024-09-10T16:11:17.231216+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:11:17.231216+0000","last_undegraded":"2024-09-10T16:11:17.231216+0000","last_fullsized":"2024-09-10T16:11:17.231216+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17602,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'21061","reported_seq":21226,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.233215+0000","last_change":"2024-09-10T16:10:52.241702+0000","last_active":"2024-09-10T16:11:17.233215+0000","last_peered":"2024-09-10T16:11:17.233215+0000","last_clean":"2024-09-10T16:11:17.233215+0000","last_became_active":"2024-09-10T16:10:52.171236+0000","last_became_peered":"2024-09-10T16:10:52.171236+0000","last_unstale":"2024-09-10T16:11:17.233215+0000","last_undegraded":"2024-09-10T16:11:17.233215+0000","last_fullsized":"2024-09-10T16:11:17.233215+0000","mapping_epoch":58,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:04:46.778280+0000","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17673,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4372,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'21082","reported_seq":21256,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:11:17.231941+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:11:17.231941+0000","last_peered":"2024-09-10T16:11:17.231941+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:11:17.231941+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17581,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4235},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'21314","reported_seq":21480,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.231025+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:11:17.231025+0000","last_peered":"2024-09-10T16:11:17.231025+0000","last_clean":"2024-09-10T16:11:17.231025+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:11:17.231025+0000","last_undegraded":"2024-09-10T16:11:17.231025+0000","last_fullsized":"2024-09-10T16:11:17.231025+0000","mapping_epoch":34,"log_start":"45'11300","ondisk_log_start":"45'11300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18161,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.232985+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:11:17.232985+0000","last_peered":"2024-09-10T16:11:17.232985+0000","last_clean":"2024-09-10T16:11:17.232985+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:11:17.232985+0000","last_undegraded":"2024-09-10T16:11:17.232985+0000","last_fullsized":"2024-09-10T16:11:17.232985+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20457","reported_seq":20636,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.233353+0000","last_change":"2024-09-10T16:09:33.012840+0000","last_active":"2024-09-10T16:11:17.233353+0000","last_peered":"2024-09-10T16:11:17.233353+0000","last_clean":"2024-09-10T16:11:17.233353+0000","last_became_active":"2024-09-10T16:09:32.934056+0000","last_became_peered":"2024-09-10T16:09:32.934056+0000","last_unstale":"2024-09-10T16:11:17.233353+0000","last_undegraded":"2024-09-10T16:11:17.233353+0000","last_fullsized":"2024-09-10T16:11:17.233353+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:08:30.481235+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18276,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":4579,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":130,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.232408+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:11:17.232408+0000","last_peered":"2024-09-10T16:11:17.232408+0000","last_clean":"2024-09-10T16:11:17.232408+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:11:17.232408+0000","last_undegraded":"2024-09-10T16:11:17.232408+0000","last_fullsized":"2024-09-10T16:11:17.232408+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20910","reported_seq":29599,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.231007+0000","last_change":"2024-09-10T16:09:08.909580+0000","last_active":"2024-09-10T16:11:17.231007+0000","last_peered":"2024-09-10T16:11:17.231007+0000","last_clean":"2024-09-10T16:11:17.231007+0000","last_became_active":"2024-09-10T16:09:08.854941+0000","last_became_peered":"2024-09-10T16:09:08.854941+0000","last_unstale":"2024-09-10T16:11:17.231007+0000","last_undegraded":"2024-09-10T16:11:17.231007+0000","last_fullsized":"2024-09-10T16:11:17.231007+0000","mapping_epoch":50,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:13:14.846691+0000","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17414,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4257,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":140,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.232615+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:11:17.232615+0000","last_peered":"2024-09-10T16:11:17.232615+0000","last_clean":"2024-09-10T16:11:17.232615+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:11:17.232615+0000","last_undegraded":"2024-09-10T16:11:17.232615+0000","last_fullsized":"2024-09-10T16:11:17.232615+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20781","reported_seq":29570,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.232496+0000","last_change":"2024-09-10T16:10:20.907889+0000","last_active":"2024-09-10T16:11:17.232496+0000","last_peered":"2024-09-10T16:11:17.232496+0000","last_clean":"2024-09-10T16:11:17.232496+0000","last_became_active":"2024-09-10T16:10:20.852065+0000","last_became_peered":"2024-09-10T16:10:20.852065+0000","last_unstale":"2024-09-10T16:11:17.232496+0000","last_undegraded":"2024-09-10T16:11:17.232496+0000","last_fullsized":"2024-09-10T16:11:17.232496+0000","mapping_epoch":56,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:34.069549+0000","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18461,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4307,"num_bytes_recovered":8675,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":115,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.233348+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:11:17.233348+0000","last_peered":"2024-09-10T16:11:17.233348+0000","last_clean":"2024-09-10T16:11:17.233348+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:11:17.233348+0000","last_undegraded":"2024-09-10T16:11:17.233348+0000","last_fullsized":"2024-09-10T16:11:17.233348+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'21170","reported_seq":30055,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.231608+0000","last_change":"2024-09-10T16:10:50.728349+0000","last_active":"2024-09-10T16:11:17.231608+0000","last_peered":"2024-09-10T16:11:17.231608+0000","last_clean":"2024-09-10T16:11:17.231608+0000","last_became_active":"2024-09-10T16:08:10.860438+0000","last_became_peered":"2024-09-10T16:08:10.860438+0000","last_unstale":"2024-09-10T16:11:17.231608+0000","last_undegraded":"2024-09-10T16:11:17.231608+0000","last_fullsized":"2024-09-10T16:11:17.231608+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:38:13.781380+0000","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18724,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20838","reported_seq":21009,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:11:17.232728+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:11:17.232728+0000","last_peered":"2024-09-10T16:11:17.232728+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:11:17.232728+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17469,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":105,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.233051+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:11:17.233051+0000","last_peered":"2024-09-10T16:11:17.233051+0000","last_clean":"2024-09-10T16:11:17.233051+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:11:17.233051+0000","last_undegraded":"2024-09-10T16:11:17.233051+0000","last_fullsized":"2024-09-10T16:11:17.233051+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20598","reported_seq":29199,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.233507+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:11:17.233507+0000","last_peered":"2024-09-10T16:11:17.233507+0000","last_clean":"2024-09-10T16:11:17.233507+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:11:17.233507+0000","last_undegraded":"2024-09-10T16:11:17.233507+0000","last_fullsized":"2024-09-10T16:11:17.233507+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16988,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":130,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.232065+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:11:17.232065+0000","last_peered":"2024-09-10T16:11:17.232065+0000","last_clean":"2024-09-10T16:11:17.232065+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:11:17.232065+0000","last_undegraded":"2024-09-10T16:11:17.232065+0000","last_fullsized":"2024-09-10T16:11:17.232065+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'21048","reported_seq":21210,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.319438+0000","last_change":"2024-09-10T16:11:17.315372+0000","last_active":"2024-09-10T16:11:17.319438+0000","last_peered":"2024-09-10T16:11:17.319438+0000","last_clean":"2024-09-10T16:11:17.319438+0000","last_became_active":"2024-09-10T16:11:17.234487+0000","last_became_peered":"2024-09-10T16:11:17.234487+0000","last_unstale":"2024-09-10T16:11:17.319438+0000","last_undegraded":"2024-09-10T16:11:17.319438+0000","last_fullsized":"2024-09-10T16:11:17.319438+0000","mapping_epoch":60,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:19:00.114196+0000","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17303,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4373,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":105,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.233467+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:11:17.233467+0000","last_peered":"2024-09-10T16:11:17.233467+0000","last_clean":"2024-09-10T16:11:17.233467+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:11:17.233467+0000","last_undegraded":"2024-09-10T16:11:17.233467+0000","last_fullsized":"2024-09-10T16:11:17.233467+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":110,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.233556+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:11:17.233556+0000","last_peered":"2024-09-10T16:11:17.233556+0000","last_clean":"2024-09-10T16:11:17.233556+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:11:17.233556+0000","last_undegraded":"2024-09-10T16:11:17.233556+0000","last_fullsized":"2024-09-10T16:11:17.233556+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'21198","reported_seq":27359,"reported_epoch":61,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:11:36.241544+0000","last_change":"2024-09-10T16:11:16.084896+0000","last_active":"2024-09-10T16:11:36.241544+0000","last_peered":"2024-09-10T16:11:36.241544+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:11:36.241544+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1335,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17361,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2990,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1333}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":9877,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292096,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":41513,"num_bytes_recovered":33097612,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":185815040,"data_stored":185717489,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161022,"ondisk_log_size":161022,"up":32,"acting":29,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986339,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":478948,"kb_used_data":114068,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93892892,"statfs":{"total":96636764160,"available":96146321408,"internally_reserved":0,"allocated":116805632,"data_stored":116436076,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659284,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487588,"kb_used_data":532,"kb_used_omap":4,"kb_used_meta":487035,"kb_avail":93884252,"statfs":{"total":96636764160,"available":96137474048,"internally_reserved":0,"allocated":544768,"data_stored":174748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":498724526},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182567,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":547396,"kb_used_data":68716,"kb_used_omap":4,"kb_used_meta":478651,"kb_avail":93824444,"statfs":{"total":96636764160,"available":96076230656,"internally_reserved":0,"allocated":70365184,"data_stored":69987938,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":490139306},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":69853184,"data_stored":69817054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":115929088,"data_stored":115896571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:11:38.545 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:11:38.850 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:11:38.850 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:11:38.863 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":485,"stamp":"2024-09-10T16:11:37.795397+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":9877,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292098,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":41515,"num_bytes_recovered":33097631,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161025,"ondisk_log_size":161025,"up":48,"acting":45,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1513932,"kb_used_data":183316,"kb_used_omap":12,"kb_used_meta":1330547,"kb_avail":281601588,"statfs":{"total":289910292480,"available":288360026112,"internally_reserved":0,"allocated":187715584,"data_stored":186598762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1362480130},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2112,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2112,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001969"},"pg_stats":[{"pgid":"2.f","version":"49'20672","reported_seq":29309,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.232412+0000","last_change":"2024-09-10T16:09:57.088531+0000","last_active":"2024-09-10T16:11:17.232412+0000","last_peered":"2024-09-10T16:11:17.232412+0000","last_clean":"2024-09-10T16:11:17.232412+0000","last_became_active":"2024-09-10T16:09:57.005755+0000","last_became_peered":"2024-09-10T16:09:57.005755+0000","last_unstale":"2024-09-10T16:11:17.232412+0000","last_undegraded":"2024-09-10T16:11:17.232412+0000","last_fullsized":"2024-09-10T16:11:17.232412+0000","mapping_epoch":54,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:09:19.329697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20672,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4259,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20844","reported_seq":20992,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.232030+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:11:17.232030+0000","last_peered":"2024-09-10T16:11:17.232030+0000","last_clean":"2024-09-10T16:11:17.232030+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:11:17.232030+0000","last_undegraded":"2024-09-10T16:11:17.232030+0000","last_fullsized":"2024-09-10T16:11:17.232030+0000","mapping_epoch":35,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18949,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20798","reported_seq":29512,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.234237+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:11:17.234237+0000","last_peered":"2024-09-10T16:11:17.234237+0000","last_clean":"2024-09-10T16:11:17.234237+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:11:17.234237+0000","last_undegraded":"2024-09-10T16:11:17.234237+0000","last_fullsized":"2024-09-10T16:11:17.234237+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18802,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20660","reported_seq":20776,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.231774+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:11:17.231774+0000","last_peered":"2024-09-10T16:11:17.231774+0000","last_clean":"2024-09-10T16:11:17.231774+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:11:17.231774+0000","last_undegraded":"2024-09-10T16:11:17.231774+0000","last_fullsized":"2024-09-10T16:11:17.231774+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20660,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20891","reported_seq":21057,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.231216+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:11:17.231216+0000","last_peered":"2024-09-10T16:11:17.231216+0000","last_clean":"2024-09-10T16:11:17.231216+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:11:17.231216+0000","last_undegraded":"2024-09-10T16:11:17.231216+0000","last_fullsized":"2024-09-10T16:11:17.231216+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17602,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'21061","reported_seq":21226,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.233215+0000","last_change":"2024-09-10T16:10:52.241702+0000","last_active":"2024-09-10T16:11:17.233215+0000","last_peered":"2024-09-10T16:11:17.233215+0000","last_clean":"2024-09-10T16:11:17.233215+0000","last_became_active":"2024-09-10T16:10:52.171236+0000","last_became_peered":"2024-09-10T16:10:52.171236+0000","last_unstale":"2024-09-10T16:11:17.233215+0000","last_undegraded":"2024-09-10T16:11:17.233215+0000","last_fullsized":"2024-09-10T16:11:17.233215+0000","mapping_epoch":58,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:04:46.778280+0000","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17673,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4372,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'21082","reported_seq":21256,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:11:17.231941+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:11:17.231941+0000","last_peered":"2024-09-10T16:11:17.231941+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:11:17.231941+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17581,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4235},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'21314","reported_seq":21480,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.231025+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:11:17.231025+0000","last_peered":"2024-09-10T16:11:17.231025+0000","last_clean":"2024-09-10T16:11:17.231025+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:11:17.231025+0000","last_undegraded":"2024-09-10T16:11:17.231025+0000","last_fullsized":"2024-09-10T16:11:17.231025+0000","mapping_epoch":34,"log_start":"45'11300","ondisk_log_start":"45'11300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18161,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.232985+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:11:17.232985+0000","last_peered":"2024-09-10T16:11:17.232985+0000","last_clean":"2024-09-10T16:11:17.232985+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:11:17.232985+0000","last_undegraded":"2024-09-10T16:11:17.232985+0000","last_fullsized":"2024-09-10T16:11:17.232985+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20457","reported_seq":20636,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.233353+0000","last_change":"2024-09-10T16:09:33.012840+0000","last_active":"2024-09-10T16:11:17.233353+0000","last_peered":"2024-09-10T16:11:17.233353+0000","last_clean":"2024-09-10T16:11:17.233353+0000","last_became_active":"2024-09-10T16:09:32.934056+0000","last_became_peered":"2024-09-10T16:09:32.934056+0000","last_unstale":"2024-09-10T16:11:17.233353+0000","last_undegraded":"2024-09-10T16:11:17.233353+0000","last_fullsized":"2024-09-10T16:11:17.233353+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:08:30.481235+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18276,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":4579,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":130,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.232408+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:11:17.232408+0000","last_peered":"2024-09-10T16:11:17.232408+0000","last_clean":"2024-09-10T16:11:17.232408+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:11:17.232408+0000","last_undegraded":"2024-09-10T16:11:17.232408+0000","last_fullsized":"2024-09-10T16:11:17.232408+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20910","reported_seq":29599,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.231007+0000","last_change":"2024-09-10T16:09:08.909580+0000","last_active":"2024-09-10T16:11:17.231007+0000","last_peered":"2024-09-10T16:11:17.231007+0000","last_clean":"2024-09-10T16:11:17.231007+0000","last_became_active":"2024-09-10T16:09:08.854941+0000","last_became_peered":"2024-09-10T16:09:08.854941+0000","last_unstale":"2024-09-10T16:11:17.231007+0000","last_undegraded":"2024-09-10T16:11:17.231007+0000","last_fullsized":"2024-09-10T16:11:17.231007+0000","mapping_epoch":50,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:13:14.846691+0000","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17414,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4257,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":140,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.232615+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:11:17.232615+0000","last_peered":"2024-09-10T16:11:17.232615+0000","last_clean":"2024-09-10T16:11:17.232615+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:11:17.232615+0000","last_undegraded":"2024-09-10T16:11:17.232615+0000","last_fullsized":"2024-09-10T16:11:17.232615+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20781","reported_seq":29570,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.232496+0000","last_change":"2024-09-10T16:10:20.907889+0000","last_active":"2024-09-10T16:11:17.232496+0000","last_peered":"2024-09-10T16:11:17.232496+0000","last_clean":"2024-09-10T16:11:17.232496+0000","last_became_active":"2024-09-10T16:10:20.852065+0000","last_became_peered":"2024-09-10T16:10:20.852065+0000","last_unstale":"2024-09-10T16:11:17.232496+0000","last_undegraded":"2024-09-10T16:11:17.232496+0000","last_fullsized":"2024-09-10T16:11:17.232496+0000","mapping_epoch":56,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:34.069549+0000","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18461,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4307,"num_bytes_recovered":8675,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":115,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.233348+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:11:17.233348+0000","last_peered":"2024-09-10T16:11:17.233348+0000","last_clean":"2024-09-10T16:11:17.233348+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:11:17.233348+0000","last_undegraded":"2024-09-10T16:11:17.233348+0000","last_fullsized":"2024-09-10T16:11:17.233348+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'21170","reported_seq":30055,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.231608+0000","last_change":"2024-09-10T16:10:50.728349+0000","last_active":"2024-09-10T16:11:17.231608+0000","last_peered":"2024-09-10T16:11:17.231608+0000","last_clean":"2024-09-10T16:11:17.231608+0000","last_became_active":"2024-09-10T16:08:10.860438+0000","last_became_peered":"2024-09-10T16:08:10.860438+0000","last_unstale":"2024-09-10T16:11:17.231608+0000","last_undegraded":"2024-09-10T16:11:17.231608+0000","last_fullsized":"2024-09-10T16:11:17.231608+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:38:13.781380+0000","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18724,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20838","reported_seq":21009,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:11:17.232728+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:11:17.232728+0000","last_peered":"2024-09-10T16:11:17.232728+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:11:17.232728+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17469,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":105,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.233051+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:11:17.233051+0000","last_peered":"2024-09-10T16:11:17.233051+0000","last_clean":"2024-09-10T16:11:17.233051+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:11:17.233051+0000","last_undegraded":"2024-09-10T16:11:17.233051+0000","last_fullsized":"2024-09-10T16:11:17.233051+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20598","reported_seq":29199,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.233507+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:11:17.233507+0000","last_peered":"2024-09-10T16:11:17.233507+0000","last_clean":"2024-09-10T16:11:17.233507+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:11:17.233507+0000","last_undegraded":"2024-09-10T16:11:17.233507+0000","last_fullsized":"2024-09-10T16:11:17.233507+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16988,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":130,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.232065+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:11:17.232065+0000","last_peered":"2024-09-10T16:11:17.232065+0000","last_clean":"2024-09-10T16:11:17.232065+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:11:17.232065+0000","last_undegraded":"2024-09-10T16:11:17.232065+0000","last_fullsized":"2024-09-10T16:11:17.232065+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'21048","reported_seq":21210,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.319438+0000","last_change":"2024-09-10T16:11:17.315372+0000","last_active":"2024-09-10T16:11:17.319438+0000","last_peered":"2024-09-10T16:11:17.319438+0000","last_clean":"2024-09-10T16:11:17.319438+0000","last_became_active":"2024-09-10T16:11:17.234487+0000","last_became_peered":"2024-09-10T16:11:17.234487+0000","last_unstale":"2024-09-10T16:11:17.319438+0000","last_undegraded":"2024-09-10T16:11:17.319438+0000","last_fullsized":"2024-09-10T16:11:17.319438+0000","mapping_epoch":60,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:19:00.114196+0000","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17303,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4373,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":105,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.233467+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:11:17.233467+0000","last_peered":"2024-09-10T16:11:17.233467+0000","last_clean":"2024-09-10T16:11:17.233467+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:11:17.233467+0000","last_undegraded":"2024-09-10T16:11:17.233467+0000","last_fullsized":"2024-09-10T16:11:17.233467+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":110,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.233556+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:11:17.233556+0000","last_peered":"2024-09-10T16:11:17.233556+0000","last_clean":"2024-09-10T16:11:17.233556+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:11:17.233556+0000","last_undegraded":"2024-09-10T16:11:17.233556+0000","last_fullsized":"2024-09-10T16:11:17.233556+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'21198","reported_seq":27359,"reported_epoch":61,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:11:36.241544+0000","last_change":"2024-09-10T16:11:16.084896+0000","last_active":"2024-09-10T16:11:36.241544+0000","last_peered":"2024-09-10T16:11:36.241544+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:11:36.241544+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1335,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17361,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2990,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1333}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":9877,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292096,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":41513,"num_bytes_recovered":33097612,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":185815040,"data_stored":185717489,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161022,"ondisk_log_size":161022,"up":32,"acting":29,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986339,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":478948,"kb_used_data":114068,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93892892,"statfs":{"total":96636764160,"available":96146321408,"internally_reserved":0,"allocated":116805632,"data_stored":116436076,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659284,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487588,"kb_used_data":532,"kb_used_omap":4,"kb_used_meta":487035,"kb_avail":93884252,"statfs":{"total":96636764160,"available":96137474048,"internally_reserved":0,"allocated":544768,"data_stored":174748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":498724526},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182567,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":547396,"kb_used_data":68716,"kb_used_omap":4,"kb_used_meta":478651,"kb_avail":93824444,"statfs":{"total":96636764160,"available":96076230656,"internally_reserved":0,"allocated":70365184,"data_stored":69987938,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":490139306},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":69853184,"data_stored":69817054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":115929088,"data_stored":115896571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:11:38.864 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-10T16:11:38.864 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "49'21082", 'reported_seq': 21256, 'reported_epoch': 61, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:11:17.231941+0000', 'last_change': '2024-09-10T15:59:16.331899+0000', 'last_active': '2024-09-10T16:11:17.231941+0000', 'last_peered': '2024-09-10T16:11:17.231941+0000', 'last_clean': '2024-09-10T15:59:14.139436+0000', 'last_became_active': '2024-09-10T15:59:16.318715+0000', 'last_became_peered': '2024-09-10T15:59:16.318715+0000', 'last_unstale': '2024-09-10T16:11:17.231941+0000', 'last_undegraded': '2024-09-10T15:59:15.877524+0000', 'last_fullsized': '2024-09-10T15:59:15.769042+0000', 'mapping_epoch': 42, 'log_start': "45'11000", 'ondisk_log_start': "45'11000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3537", 'last_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'objects_scrubbed': 3497, 'log_size': 10082, 'log_dups_size': 0, 'ondisk_log_size': 10082, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1224, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4237, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 17581, 'num_write_kb': 28172, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4235}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:11:38.864 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-10T16:11:38.864 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "49'20838", 'reported_seq': 21009, 'reported_epoch': 61, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:11:17.232728+0000', 'last_change': '2024-09-10T15:59:16.315641+0000', 'last_active': '2024-09-10T16:11:17.232728+0000', 'last_peered': '2024-09-10T16:11:17.232728+0000', 'last_clean': '2024-09-10T15:59:14.117001+0000', 'last_became_active': '2024-09-10T15:59:16.302245+0000', 'last_became_peered': '2024-09-10T15:59:16.302245+0000', 'last_unstale': '2024-09-10T16:11:17.232728+0000', 'last_undegraded': '2024-09-10T15:59:15.845012+0000', 'last_fullsized': '2024-09-10T15:59:15.770337+0000', 'mapping_epoch': 42, 'log_start': "45'10800", 'ondisk_log_start': "45'10800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3404", 'last_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'objects_scrubbed': 3367, 'log_size': 10038, 'log_dups_size': 0, 'ondisk_log_size': 10038, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 983, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17469, 'num_write_kb': 28204, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4304}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:11:38.864 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-10T16:11:38.864 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "49'21198", 'reported_seq': 27359, 'reported_epoch': 61, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:11:36.241544+0000', 'last_change': '2024-09-10T16:11:16.084896+0000', 'last_active': '2024-09-10T16:11:36.241544+0000', 'last_peered': '2024-09-10T16:11:36.241544+0000', 'last_clean': '2024-09-10T15:59:14.129780+0000', 'last_became_active': '2024-09-10T15:59:16.300932+0000', 'last_became_peered': '2024-09-10T15:59:16.300932+0000', 'last_unstale': '2024-09-10T16:11:36.241544+0000', 'last_undegraded': '2024-09-10T15:59:15.887823+0000', 'last_fullsized': '2024-09-10T15:59:15.768513+0000', 'mapping_epoch': 42, 'log_start': "45'11100", 'ondisk_log_start': "45'11100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3880", 'last_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'objects_scrubbed': 3827, 'log_size': 10098, 'log_dups_size': 0, 'ondisk_log_size': 10098, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1397, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1335, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17361, 'num_write_kb': 27072, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2990, 'num_bytes_recovered': 483, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1333}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:11:38.865 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T16:11:39.299 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:11:39.300 INFO:teuthology.orchestra.run.smithi115.stdout:{"fsid":"7effee29-df05-4351-9741-ee203564ed81","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 9877/24 objects degraded (41154.167%), 3 pgs degraded, 3 pgs undersized","count":6},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":863,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":61,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983954,"num_in_osds":3,"osd_in_since":1725983947,"num_remapped_pgs":3},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":21},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":2},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":12,"data_bytes":4849,"bytes_used":1550266368,"bytes_avail":288360026112,"bytes_total":289910292480,"degraded_objects":9877,"degraded_total":24,"degraded_ratio":411.54166666666674,"recovering_objects_per_sec":175,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":2112,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-10T15:57:14:967120+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T16:00:13.560776+0000","services":{}},"progress_events":{}} 2024-09-10T16:11:39.313 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-10T16:11:39.313 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:11:39.656 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:11:39.656 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:11:39.668 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":485,"stamp":"2024-09-10T16:11:37.795397+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":9877,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292098,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":41515,"num_bytes_recovered":33097631,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161025,"ondisk_log_size":161025,"up":48,"acting":45,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1513932,"kb_used_data":183316,"kb_used_omap":12,"kb_used_meta":1330547,"kb_avail":281601588,"statfs":{"total":289910292480,"available":288360026112,"internally_reserved":0,"allocated":187715584,"data_stored":186598762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1362480130},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2112,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2112,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001969"},"pg_stats":[{"pgid":"2.f","version":"49'20672","reported_seq":29309,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.232412+0000","last_change":"2024-09-10T16:09:57.088531+0000","last_active":"2024-09-10T16:11:17.232412+0000","last_peered":"2024-09-10T16:11:17.232412+0000","last_clean":"2024-09-10T16:11:17.232412+0000","last_became_active":"2024-09-10T16:09:57.005755+0000","last_became_peered":"2024-09-10T16:09:57.005755+0000","last_unstale":"2024-09-10T16:11:17.232412+0000","last_undegraded":"2024-09-10T16:11:17.232412+0000","last_fullsized":"2024-09-10T16:11:17.232412+0000","mapping_epoch":54,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:09:19.329697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20672,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4259,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20844","reported_seq":20992,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.232030+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:11:17.232030+0000","last_peered":"2024-09-10T16:11:17.232030+0000","last_clean":"2024-09-10T16:11:17.232030+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:11:17.232030+0000","last_undegraded":"2024-09-10T16:11:17.232030+0000","last_fullsized":"2024-09-10T16:11:17.232030+0000","mapping_epoch":35,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18949,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20798","reported_seq":29512,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.234237+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:11:17.234237+0000","last_peered":"2024-09-10T16:11:17.234237+0000","last_clean":"2024-09-10T16:11:17.234237+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:11:17.234237+0000","last_undegraded":"2024-09-10T16:11:17.234237+0000","last_fullsized":"2024-09-10T16:11:17.234237+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18802,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20660","reported_seq":20776,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.231774+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:11:17.231774+0000","last_peered":"2024-09-10T16:11:17.231774+0000","last_clean":"2024-09-10T16:11:17.231774+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:11:17.231774+0000","last_undegraded":"2024-09-10T16:11:17.231774+0000","last_fullsized":"2024-09-10T16:11:17.231774+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20660,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20891","reported_seq":21057,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.231216+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:11:17.231216+0000","last_peered":"2024-09-10T16:11:17.231216+0000","last_clean":"2024-09-10T16:11:17.231216+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:11:17.231216+0000","last_undegraded":"2024-09-10T16:11:17.231216+0000","last_fullsized":"2024-09-10T16:11:17.231216+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17602,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'21061","reported_seq":21226,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.233215+0000","last_change":"2024-09-10T16:10:52.241702+0000","last_active":"2024-09-10T16:11:17.233215+0000","last_peered":"2024-09-10T16:11:17.233215+0000","last_clean":"2024-09-10T16:11:17.233215+0000","last_became_active":"2024-09-10T16:10:52.171236+0000","last_became_peered":"2024-09-10T16:10:52.171236+0000","last_unstale":"2024-09-10T16:11:17.233215+0000","last_undegraded":"2024-09-10T16:11:17.233215+0000","last_fullsized":"2024-09-10T16:11:17.233215+0000","mapping_epoch":58,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:04:46.778280+0000","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17673,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4372,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'21082","reported_seq":21256,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:11:17.231941+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:11:17.231941+0000","last_peered":"2024-09-10T16:11:17.231941+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:11:17.231941+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17581,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4235},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'21314","reported_seq":21480,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.231025+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:11:17.231025+0000","last_peered":"2024-09-10T16:11:17.231025+0000","last_clean":"2024-09-10T16:11:17.231025+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:11:17.231025+0000","last_undegraded":"2024-09-10T16:11:17.231025+0000","last_fullsized":"2024-09-10T16:11:17.231025+0000","mapping_epoch":34,"log_start":"45'11300","ondisk_log_start":"45'11300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18161,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.232985+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:11:17.232985+0000","last_peered":"2024-09-10T16:11:17.232985+0000","last_clean":"2024-09-10T16:11:17.232985+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:11:17.232985+0000","last_undegraded":"2024-09-10T16:11:17.232985+0000","last_fullsized":"2024-09-10T16:11:17.232985+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20457","reported_seq":20636,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.233353+0000","last_change":"2024-09-10T16:09:33.012840+0000","last_active":"2024-09-10T16:11:17.233353+0000","last_peered":"2024-09-10T16:11:17.233353+0000","last_clean":"2024-09-10T16:11:17.233353+0000","last_became_active":"2024-09-10T16:09:32.934056+0000","last_became_peered":"2024-09-10T16:09:32.934056+0000","last_unstale":"2024-09-10T16:11:17.233353+0000","last_undegraded":"2024-09-10T16:11:17.233353+0000","last_fullsized":"2024-09-10T16:11:17.233353+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:08:30.481235+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18276,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":4579,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":130,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.232408+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:11:17.232408+0000","last_peered":"2024-09-10T16:11:17.232408+0000","last_clean":"2024-09-10T16:11:17.232408+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:11:17.232408+0000","last_undegraded":"2024-09-10T16:11:17.232408+0000","last_fullsized":"2024-09-10T16:11:17.232408+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20910","reported_seq":29599,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.231007+0000","last_change":"2024-09-10T16:09:08.909580+0000","last_active":"2024-09-10T16:11:17.231007+0000","last_peered":"2024-09-10T16:11:17.231007+0000","last_clean":"2024-09-10T16:11:17.231007+0000","last_became_active":"2024-09-10T16:09:08.854941+0000","last_became_peered":"2024-09-10T16:09:08.854941+0000","last_unstale":"2024-09-10T16:11:17.231007+0000","last_undegraded":"2024-09-10T16:11:17.231007+0000","last_fullsized":"2024-09-10T16:11:17.231007+0000","mapping_epoch":50,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:13:14.846691+0000","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17414,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4257,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":140,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.232615+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:11:17.232615+0000","last_peered":"2024-09-10T16:11:17.232615+0000","last_clean":"2024-09-10T16:11:17.232615+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:11:17.232615+0000","last_undegraded":"2024-09-10T16:11:17.232615+0000","last_fullsized":"2024-09-10T16:11:17.232615+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20781","reported_seq":29570,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.232496+0000","last_change":"2024-09-10T16:10:20.907889+0000","last_active":"2024-09-10T16:11:17.232496+0000","last_peered":"2024-09-10T16:11:17.232496+0000","last_clean":"2024-09-10T16:11:17.232496+0000","last_became_active":"2024-09-10T16:10:20.852065+0000","last_became_peered":"2024-09-10T16:10:20.852065+0000","last_unstale":"2024-09-10T16:11:17.232496+0000","last_undegraded":"2024-09-10T16:11:17.232496+0000","last_fullsized":"2024-09-10T16:11:17.232496+0000","mapping_epoch":56,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:34.069549+0000","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18461,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4307,"num_bytes_recovered":8675,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":115,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.233348+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:11:17.233348+0000","last_peered":"2024-09-10T16:11:17.233348+0000","last_clean":"2024-09-10T16:11:17.233348+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:11:17.233348+0000","last_undegraded":"2024-09-10T16:11:17.233348+0000","last_fullsized":"2024-09-10T16:11:17.233348+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'21170","reported_seq":30055,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.231608+0000","last_change":"2024-09-10T16:10:50.728349+0000","last_active":"2024-09-10T16:11:17.231608+0000","last_peered":"2024-09-10T16:11:17.231608+0000","last_clean":"2024-09-10T16:11:17.231608+0000","last_became_active":"2024-09-10T16:08:10.860438+0000","last_became_peered":"2024-09-10T16:08:10.860438+0000","last_unstale":"2024-09-10T16:11:17.231608+0000","last_undegraded":"2024-09-10T16:11:17.231608+0000","last_fullsized":"2024-09-10T16:11:17.231608+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:38:13.781380+0000","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18724,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20838","reported_seq":21009,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:11:17.232728+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:11:17.232728+0000","last_peered":"2024-09-10T16:11:17.232728+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:11:17.232728+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17469,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":105,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.233051+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:11:17.233051+0000","last_peered":"2024-09-10T16:11:17.233051+0000","last_clean":"2024-09-10T16:11:17.233051+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:11:17.233051+0000","last_undegraded":"2024-09-10T16:11:17.233051+0000","last_fullsized":"2024-09-10T16:11:17.233051+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20598","reported_seq":29199,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.233507+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:11:17.233507+0000","last_peered":"2024-09-10T16:11:17.233507+0000","last_clean":"2024-09-10T16:11:17.233507+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:11:17.233507+0000","last_undegraded":"2024-09-10T16:11:17.233507+0000","last_fullsized":"2024-09-10T16:11:17.233507+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16988,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":130,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.232065+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:11:17.232065+0000","last_peered":"2024-09-10T16:11:17.232065+0000","last_clean":"2024-09-10T16:11:17.232065+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:11:17.232065+0000","last_undegraded":"2024-09-10T16:11:17.232065+0000","last_fullsized":"2024-09-10T16:11:17.232065+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'21048","reported_seq":21210,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.319438+0000","last_change":"2024-09-10T16:11:17.315372+0000","last_active":"2024-09-10T16:11:17.319438+0000","last_peered":"2024-09-10T16:11:17.319438+0000","last_clean":"2024-09-10T16:11:17.319438+0000","last_became_active":"2024-09-10T16:11:17.234487+0000","last_became_peered":"2024-09-10T16:11:17.234487+0000","last_unstale":"2024-09-10T16:11:17.319438+0000","last_undegraded":"2024-09-10T16:11:17.319438+0000","last_fullsized":"2024-09-10T16:11:17.319438+0000","mapping_epoch":60,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:19:00.114196+0000","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17303,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4373,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":105,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.233467+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:11:17.233467+0000","last_peered":"2024-09-10T16:11:17.233467+0000","last_clean":"2024-09-10T16:11:17.233467+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:11:17.233467+0000","last_undegraded":"2024-09-10T16:11:17.233467+0000","last_fullsized":"2024-09-10T16:11:17.233467+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":110,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.233556+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:11:17.233556+0000","last_peered":"2024-09-10T16:11:17.233556+0000","last_clean":"2024-09-10T16:11:17.233556+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:11:17.233556+0000","last_undegraded":"2024-09-10T16:11:17.233556+0000","last_fullsized":"2024-09-10T16:11:17.233556+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'21198","reported_seq":27359,"reported_epoch":61,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:11:36.241544+0000","last_change":"2024-09-10T16:11:16.084896+0000","last_active":"2024-09-10T16:11:36.241544+0000","last_peered":"2024-09-10T16:11:36.241544+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:11:36.241544+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1335,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17361,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2990,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1333}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":9877,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292096,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":41513,"num_bytes_recovered":33097612,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":185815040,"data_stored":185717489,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161022,"ondisk_log_size":161022,"up":32,"acting":29,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986339,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":478948,"kb_used_data":114068,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93892892,"statfs":{"total":96636764160,"available":96146321408,"internally_reserved":0,"allocated":116805632,"data_stored":116436076,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659284,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487588,"kb_used_data":532,"kb_used_omap":4,"kb_used_meta":487035,"kb_avail":93884252,"statfs":{"total":96636764160,"available":96137474048,"internally_reserved":0,"allocated":544768,"data_stored":174748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":498724526},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182567,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":547396,"kb_used_data":68716,"kb_used_omap":4,"kb_used_meta":478651,"kb_avail":93824444,"statfs":{"total":96636764160,"available":96076230656,"internally_reserved":0,"allocated":70365184,"data_stored":69987938,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":490139306},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":69853184,"data_stored":69817054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":115929088,"data_stored":115896571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:11:42.671 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:11:43.000 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:11:43.000 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:11:43.013 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":487,"stamp":"2024-09-10T16:11:41.795963+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":8895,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292098,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":42497,"num_bytes_recovered":33097631,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161025,"ondisk_log_size":161025,"up":48,"acting":45,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1524820,"kb_used_data":180316,"kb_used_omap":12,"kb_used_meta":1344435,"kb_avail":281590700,"statfs":{"total":289910292480,"available":288348876800,"internally_reserved":0,"allocated":184643584,"data_stored":183526762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1376701442},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2424,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2424,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001884"},"pg_stats":[{"pgid":"2.f","version":"49'20672","reported_seq":29309,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.232412+0000","last_change":"2024-09-10T16:09:57.088531+0000","last_active":"2024-09-10T16:11:17.232412+0000","last_peered":"2024-09-10T16:11:17.232412+0000","last_clean":"2024-09-10T16:11:17.232412+0000","last_became_active":"2024-09-10T16:09:57.005755+0000","last_became_peered":"2024-09-10T16:09:57.005755+0000","last_unstale":"2024-09-10T16:11:17.232412+0000","last_undegraded":"2024-09-10T16:11:17.232412+0000","last_fullsized":"2024-09-10T16:11:17.232412+0000","mapping_epoch":54,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:09:19.329697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20672,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4259,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20844","reported_seq":20992,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.232030+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:11:17.232030+0000","last_peered":"2024-09-10T16:11:17.232030+0000","last_clean":"2024-09-10T16:11:17.232030+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:11:17.232030+0000","last_undegraded":"2024-09-10T16:11:17.232030+0000","last_fullsized":"2024-09-10T16:11:17.232030+0000","mapping_epoch":35,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18949,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20798","reported_seq":29512,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.234237+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:11:17.234237+0000","last_peered":"2024-09-10T16:11:17.234237+0000","last_clean":"2024-09-10T16:11:17.234237+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:11:17.234237+0000","last_undegraded":"2024-09-10T16:11:17.234237+0000","last_fullsized":"2024-09-10T16:11:17.234237+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18802,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20660","reported_seq":20776,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.231774+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:11:17.231774+0000","last_peered":"2024-09-10T16:11:17.231774+0000","last_clean":"2024-09-10T16:11:17.231774+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:11:17.231774+0000","last_undegraded":"2024-09-10T16:11:17.231774+0000","last_fullsized":"2024-09-10T16:11:17.231774+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20660,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20891","reported_seq":21057,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.231216+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:11:17.231216+0000","last_peered":"2024-09-10T16:11:17.231216+0000","last_clean":"2024-09-10T16:11:17.231216+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:11:17.231216+0000","last_undegraded":"2024-09-10T16:11:17.231216+0000","last_fullsized":"2024-09-10T16:11:17.231216+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17602,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'21061","reported_seq":21226,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.233215+0000","last_change":"2024-09-10T16:10:52.241702+0000","last_active":"2024-09-10T16:11:17.233215+0000","last_peered":"2024-09-10T16:11:17.233215+0000","last_clean":"2024-09-10T16:11:17.233215+0000","last_became_active":"2024-09-10T16:10:52.171236+0000","last_became_peered":"2024-09-10T16:10:52.171236+0000","last_unstale":"2024-09-10T16:11:17.233215+0000","last_undegraded":"2024-09-10T16:11:17.233215+0000","last_fullsized":"2024-09-10T16:11:17.233215+0000","mapping_epoch":58,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:04:46.778280+0000","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17673,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4372,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'21082","reported_seq":21256,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:11:17.231941+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:11:17.231941+0000","last_peered":"2024-09-10T16:11:17.231941+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:11:17.231941+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17581,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4235},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'21314","reported_seq":21480,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.231025+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:11:17.231025+0000","last_peered":"2024-09-10T16:11:17.231025+0000","last_clean":"2024-09-10T16:11:17.231025+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:11:17.231025+0000","last_undegraded":"2024-09-10T16:11:17.231025+0000","last_fullsized":"2024-09-10T16:11:17.231025+0000","mapping_epoch":34,"log_start":"45'11300","ondisk_log_start":"45'11300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18161,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.232985+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:11:17.232985+0000","last_peered":"2024-09-10T16:11:17.232985+0000","last_clean":"2024-09-10T16:11:17.232985+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:11:17.232985+0000","last_undegraded":"2024-09-10T16:11:17.232985+0000","last_fullsized":"2024-09-10T16:11:17.232985+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20457","reported_seq":20637,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.233390+0000","last_change":"2024-09-10T16:09:33.012840+0000","last_active":"2024-09-10T16:11:17.233390+0000","last_peered":"2024-09-10T16:11:17.233390+0000","last_clean":"2024-09-10T16:11:17.233390+0000","last_became_active":"2024-09-10T16:09:32.934056+0000","last_became_peered":"2024-09-10T16:09:32.934056+0000","last_unstale":"2024-09-10T16:11:17.233390+0000","last_undegraded":"2024-09-10T16:11:17.233390+0000","last_fullsized":"2024-09-10T16:11:17.233390+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:08:30.481235+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18276,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":4579,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":130,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.232408+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:11:17.232408+0000","last_peered":"2024-09-10T16:11:17.232408+0000","last_clean":"2024-09-10T16:11:17.232408+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:11:17.232408+0000","last_undegraded":"2024-09-10T16:11:17.232408+0000","last_fullsized":"2024-09-10T16:11:17.232408+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20910","reported_seq":29599,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.231007+0000","last_change":"2024-09-10T16:09:08.909580+0000","last_active":"2024-09-10T16:11:17.231007+0000","last_peered":"2024-09-10T16:11:17.231007+0000","last_clean":"2024-09-10T16:11:17.231007+0000","last_became_active":"2024-09-10T16:09:08.854941+0000","last_became_peered":"2024-09-10T16:09:08.854941+0000","last_unstale":"2024-09-10T16:11:17.231007+0000","last_undegraded":"2024-09-10T16:11:17.231007+0000","last_fullsized":"2024-09-10T16:11:17.231007+0000","mapping_epoch":50,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:13:14.846691+0000","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17414,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4257,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":140,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.232615+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:11:17.232615+0000","last_peered":"2024-09-10T16:11:17.232615+0000","last_clean":"2024-09-10T16:11:17.232615+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:11:17.232615+0000","last_undegraded":"2024-09-10T16:11:17.232615+0000","last_fullsized":"2024-09-10T16:11:17.232615+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20781","reported_seq":29570,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.232496+0000","last_change":"2024-09-10T16:10:20.907889+0000","last_active":"2024-09-10T16:11:17.232496+0000","last_peered":"2024-09-10T16:11:17.232496+0000","last_clean":"2024-09-10T16:11:17.232496+0000","last_became_active":"2024-09-10T16:10:20.852065+0000","last_became_peered":"2024-09-10T16:10:20.852065+0000","last_unstale":"2024-09-10T16:11:17.232496+0000","last_undegraded":"2024-09-10T16:11:17.232496+0000","last_fullsized":"2024-09-10T16:11:17.232496+0000","mapping_epoch":56,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:34.069549+0000","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18461,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4307,"num_bytes_recovered":8675,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":115,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.233348+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:11:17.233348+0000","last_peered":"2024-09-10T16:11:17.233348+0000","last_clean":"2024-09-10T16:11:17.233348+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:11:17.233348+0000","last_undegraded":"2024-09-10T16:11:17.233348+0000","last_fullsized":"2024-09-10T16:11:17.233348+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'21170","reported_seq":30055,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.231608+0000","last_change":"2024-09-10T16:10:50.728349+0000","last_active":"2024-09-10T16:11:17.231608+0000","last_peered":"2024-09-10T16:11:17.231608+0000","last_clean":"2024-09-10T16:11:17.231608+0000","last_became_active":"2024-09-10T16:08:10.860438+0000","last_became_peered":"2024-09-10T16:08:10.860438+0000","last_unstale":"2024-09-10T16:11:17.231608+0000","last_undegraded":"2024-09-10T16:11:17.231608+0000","last_fullsized":"2024-09-10T16:11:17.231608+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:38:13.781380+0000","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18724,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20838","reported_seq":21009,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:11:17.232728+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:11:17.232728+0000","last_peered":"2024-09-10T16:11:17.232728+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:11:17.232728+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17469,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":105,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.233051+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:11:17.233051+0000","last_peered":"2024-09-10T16:11:17.233051+0000","last_clean":"2024-09-10T16:11:17.233051+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:11:17.233051+0000","last_undegraded":"2024-09-10T16:11:17.233051+0000","last_fullsized":"2024-09-10T16:11:17.233051+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20598","reported_seq":29199,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.233507+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:11:17.233507+0000","last_peered":"2024-09-10T16:11:17.233507+0000","last_clean":"2024-09-10T16:11:17.233507+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:11:17.233507+0000","last_undegraded":"2024-09-10T16:11:17.233507+0000","last_fullsized":"2024-09-10T16:11:17.233507+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16988,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":130,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.232065+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:11:17.232065+0000","last_peered":"2024-09-10T16:11:17.232065+0000","last_clean":"2024-09-10T16:11:17.232065+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:11:17.232065+0000","last_undegraded":"2024-09-10T16:11:17.232065+0000","last_fullsized":"2024-09-10T16:11:17.232065+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'21048","reported_seq":21210,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.319438+0000","last_change":"2024-09-10T16:11:17.315372+0000","last_active":"2024-09-10T16:11:17.319438+0000","last_peered":"2024-09-10T16:11:17.319438+0000","last_clean":"2024-09-10T16:11:17.319438+0000","last_became_active":"2024-09-10T16:11:17.234487+0000","last_became_peered":"2024-09-10T16:11:17.234487+0000","last_unstale":"2024-09-10T16:11:17.319438+0000","last_undegraded":"2024-09-10T16:11:17.319438+0000","last_fullsized":"2024-09-10T16:11:17.319438+0000","mapping_epoch":60,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:19:00.114196+0000","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17303,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4373,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":105,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.233467+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:11:17.233467+0000","last_peered":"2024-09-10T16:11:17.233467+0000","last_clean":"2024-09-10T16:11:17.233467+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:11:17.233467+0000","last_undegraded":"2024-09-10T16:11:17.233467+0000","last_fullsized":"2024-09-10T16:11:17.233467+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":110,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.233556+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:11:17.233556+0000","last_peered":"2024-09-10T16:11:17.233556+0000","last_clean":"2024-09-10T16:11:17.233556+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:11:17.233556+0000","last_undegraded":"2024-09-10T16:11:17.233556+0000","last_fullsized":"2024-09-10T16:11:17.233556+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'21198","reported_seq":29324,"reported_epoch":61,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:11:41.242799+0000","last_change":"2024-09-10T16:11:16.084896+0000","last_active":"2024-09-10T16:11:41.242799+0000","last_peered":"2024-09-10T16:11:41.242799+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:11:41.242799+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":353,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17361,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3972,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-351}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":8895,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292096,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":42495,"num_bytes_recovered":33097612,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":182743040,"data_stored":182645489,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161022,"ondisk_log_size":161022,"up":32,"acting":29,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986340,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":475948,"kb_used_data":111068,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93895892,"statfs":{"total":96636764160,"available":96149393408,"internally_reserved":0,"allocated":113733632,"data_stored":113364076,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659285,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":501476,"kb_used_data":532,"kb_used_omap":4,"kb_used_meta":500923,"kb_avail":93870364,"statfs":{"total":96636764160,"available":96123252736,"internally_reserved":0,"allocated":544768,"data_stored":174748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":512945838},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182568,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":547396,"kb_used_data":68716,"kb_used_omap":4,"kb_used_meta":478651,"kb_avail":93824444,"statfs":{"total":96636764160,"available":96076230656,"internally_reserved":0,"allocated":70365184,"data_stored":69987938,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":490139306},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":69853184,"data_stored":69817054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":112857088,"data_stored":112824571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:11:43.014 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:11:43.324 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:11:43.324 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:11:43.340 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":488,"stamp":"2024-09-10T16:11:43.307546+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":8895,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292098,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":42497,"num_bytes_recovered":33097631,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161025,"ondisk_log_size":161025,"up":48,"acting":45,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1524820,"kb_used_data":180316,"kb_used_omap":12,"kb_used_meta":1344435,"kb_avail":281590700,"statfs":{"total":289910292480,"available":288348876800,"internally_reserved":0,"allocated":184643584,"data_stored":183526762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1376701442},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1760,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1760,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"11.513161"},"pg_stats":[{"pgid":"2.f","version":"49'20672","reported_seq":29309,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.232412+0000","last_change":"2024-09-10T16:09:57.088531+0000","last_active":"2024-09-10T16:11:17.232412+0000","last_peered":"2024-09-10T16:11:17.232412+0000","last_clean":"2024-09-10T16:11:17.232412+0000","last_became_active":"2024-09-10T16:09:57.005755+0000","last_became_peered":"2024-09-10T16:09:57.005755+0000","last_unstale":"2024-09-10T16:11:17.232412+0000","last_undegraded":"2024-09-10T16:11:17.232412+0000","last_fullsized":"2024-09-10T16:11:17.232412+0000","mapping_epoch":54,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:09:19.329697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20672,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4259,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20844","reported_seq":20992,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.232030+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:11:17.232030+0000","last_peered":"2024-09-10T16:11:17.232030+0000","last_clean":"2024-09-10T16:11:17.232030+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:11:17.232030+0000","last_undegraded":"2024-09-10T16:11:17.232030+0000","last_fullsized":"2024-09-10T16:11:17.232030+0000","mapping_epoch":35,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18949,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20798","reported_seq":29512,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.234237+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:11:17.234237+0000","last_peered":"2024-09-10T16:11:17.234237+0000","last_clean":"2024-09-10T16:11:17.234237+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:11:17.234237+0000","last_undegraded":"2024-09-10T16:11:17.234237+0000","last_fullsized":"2024-09-10T16:11:17.234237+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18802,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20660","reported_seq":20776,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.231774+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:11:17.231774+0000","last_peered":"2024-09-10T16:11:17.231774+0000","last_clean":"2024-09-10T16:11:17.231774+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:11:17.231774+0000","last_undegraded":"2024-09-10T16:11:17.231774+0000","last_fullsized":"2024-09-10T16:11:17.231774+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20660,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20891","reported_seq":21057,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.231216+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:11:17.231216+0000","last_peered":"2024-09-10T16:11:17.231216+0000","last_clean":"2024-09-10T16:11:17.231216+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:11:17.231216+0000","last_undegraded":"2024-09-10T16:11:17.231216+0000","last_fullsized":"2024-09-10T16:11:17.231216+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17602,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'21061","reported_seq":21226,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.233215+0000","last_change":"2024-09-10T16:10:52.241702+0000","last_active":"2024-09-10T16:11:17.233215+0000","last_peered":"2024-09-10T16:11:17.233215+0000","last_clean":"2024-09-10T16:11:17.233215+0000","last_became_active":"2024-09-10T16:10:52.171236+0000","last_became_peered":"2024-09-10T16:10:52.171236+0000","last_unstale":"2024-09-10T16:11:17.233215+0000","last_undegraded":"2024-09-10T16:11:17.233215+0000","last_fullsized":"2024-09-10T16:11:17.233215+0000","mapping_epoch":58,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:04:46.778280+0000","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17673,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4372,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'21082","reported_seq":21256,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:11:17.231941+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:11:17.231941+0000","last_peered":"2024-09-10T16:11:17.231941+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:11:17.231941+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17581,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4235},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'21314","reported_seq":21480,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.231025+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:11:17.231025+0000","last_peered":"2024-09-10T16:11:17.231025+0000","last_clean":"2024-09-10T16:11:17.231025+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:11:17.231025+0000","last_undegraded":"2024-09-10T16:11:17.231025+0000","last_fullsized":"2024-09-10T16:11:17.231025+0000","mapping_epoch":34,"log_start":"45'11300","ondisk_log_start":"45'11300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18161,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.232985+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:11:17.232985+0000","last_peered":"2024-09-10T16:11:17.232985+0000","last_clean":"2024-09-10T16:11:17.232985+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:11:17.232985+0000","last_undegraded":"2024-09-10T16:11:17.232985+0000","last_fullsized":"2024-09-10T16:11:17.232985+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20457","reported_seq":20637,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.233390+0000","last_change":"2024-09-10T16:09:33.012840+0000","last_active":"2024-09-10T16:11:17.233390+0000","last_peered":"2024-09-10T16:11:17.233390+0000","last_clean":"2024-09-10T16:11:17.233390+0000","last_became_active":"2024-09-10T16:09:32.934056+0000","last_became_peered":"2024-09-10T16:09:32.934056+0000","last_unstale":"2024-09-10T16:11:17.233390+0000","last_undegraded":"2024-09-10T16:11:17.233390+0000","last_fullsized":"2024-09-10T16:11:17.233390+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:08:30.481235+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18276,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":4579,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":130,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.232408+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:11:17.232408+0000","last_peered":"2024-09-10T16:11:17.232408+0000","last_clean":"2024-09-10T16:11:17.232408+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:11:17.232408+0000","last_undegraded":"2024-09-10T16:11:17.232408+0000","last_fullsized":"2024-09-10T16:11:17.232408+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20910","reported_seq":29599,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.231007+0000","last_change":"2024-09-10T16:09:08.909580+0000","last_active":"2024-09-10T16:11:17.231007+0000","last_peered":"2024-09-10T16:11:17.231007+0000","last_clean":"2024-09-10T16:11:17.231007+0000","last_became_active":"2024-09-10T16:09:08.854941+0000","last_became_peered":"2024-09-10T16:09:08.854941+0000","last_unstale":"2024-09-10T16:11:17.231007+0000","last_undegraded":"2024-09-10T16:11:17.231007+0000","last_fullsized":"2024-09-10T16:11:17.231007+0000","mapping_epoch":50,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:13:14.846691+0000","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17414,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4257,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":140,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.232615+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:11:17.232615+0000","last_peered":"2024-09-10T16:11:17.232615+0000","last_clean":"2024-09-10T16:11:17.232615+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:11:17.232615+0000","last_undegraded":"2024-09-10T16:11:17.232615+0000","last_fullsized":"2024-09-10T16:11:17.232615+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20781","reported_seq":29570,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.232496+0000","last_change":"2024-09-10T16:10:20.907889+0000","last_active":"2024-09-10T16:11:17.232496+0000","last_peered":"2024-09-10T16:11:17.232496+0000","last_clean":"2024-09-10T16:11:17.232496+0000","last_became_active":"2024-09-10T16:10:20.852065+0000","last_became_peered":"2024-09-10T16:10:20.852065+0000","last_unstale":"2024-09-10T16:11:17.232496+0000","last_undegraded":"2024-09-10T16:11:17.232496+0000","last_fullsized":"2024-09-10T16:11:17.232496+0000","mapping_epoch":56,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:34.069549+0000","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18461,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4307,"num_bytes_recovered":8675,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":115,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.233348+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:11:17.233348+0000","last_peered":"2024-09-10T16:11:17.233348+0000","last_clean":"2024-09-10T16:11:17.233348+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:11:17.233348+0000","last_undegraded":"2024-09-10T16:11:17.233348+0000","last_fullsized":"2024-09-10T16:11:17.233348+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'21170","reported_seq":30055,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.231608+0000","last_change":"2024-09-10T16:10:50.728349+0000","last_active":"2024-09-10T16:11:17.231608+0000","last_peered":"2024-09-10T16:11:17.231608+0000","last_clean":"2024-09-10T16:11:17.231608+0000","last_became_active":"2024-09-10T16:08:10.860438+0000","last_became_peered":"2024-09-10T16:08:10.860438+0000","last_unstale":"2024-09-10T16:11:17.231608+0000","last_undegraded":"2024-09-10T16:11:17.231608+0000","last_fullsized":"2024-09-10T16:11:17.231608+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:38:13.781380+0000","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18724,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20838","reported_seq":21009,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:11:17.232728+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:11:17.232728+0000","last_peered":"2024-09-10T16:11:17.232728+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:11:17.232728+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17469,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":105,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.233051+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:11:17.233051+0000","last_peered":"2024-09-10T16:11:17.233051+0000","last_clean":"2024-09-10T16:11:17.233051+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:11:17.233051+0000","last_undegraded":"2024-09-10T16:11:17.233051+0000","last_fullsized":"2024-09-10T16:11:17.233051+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20598","reported_seq":29199,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.233507+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:11:17.233507+0000","last_peered":"2024-09-10T16:11:17.233507+0000","last_clean":"2024-09-10T16:11:17.233507+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:11:17.233507+0000","last_undegraded":"2024-09-10T16:11:17.233507+0000","last_fullsized":"2024-09-10T16:11:17.233507+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16988,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":130,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.232065+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:11:17.232065+0000","last_peered":"2024-09-10T16:11:17.232065+0000","last_clean":"2024-09-10T16:11:17.232065+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:11:17.232065+0000","last_undegraded":"2024-09-10T16:11:17.232065+0000","last_fullsized":"2024-09-10T16:11:17.232065+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'21048","reported_seq":21210,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.319438+0000","last_change":"2024-09-10T16:11:17.315372+0000","last_active":"2024-09-10T16:11:17.319438+0000","last_peered":"2024-09-10T16:11:17.319438+0000","last_clean":"2024-09-10T16:11:17.319438+0000","last_became_active":"2024-09-10T16:11:17.234487+0000","last_became_peered":"2024-09-10T16:11:17.234487+0000","last_unstale":"2024-09-10T16:11:17.319438+0000","last_undegraded":"2024-09-10T16:11:17.319438+0000","last_fullsized":"2024-09-10T16:11:17.319438+0000","mapping_epoch":60,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:19:00.114196+0000","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17303,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4373,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":105,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.233467+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:11:17.233467+0000","last_peered":"2024-09-10T16:11:17.233467+0000","last_clean":"2024-09-10T16:11:17.233467+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:11:17.233467+0000","last_undegraded":"2024-09-10T16:11:17.233467+0000","last_fullsized":"2024-09-10T16:11:17.233467+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":110,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.233556+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:11:17.233556+0000","last_peered":"2024-09-10T16:11:17.233556+0000","last_clean":"2024-09-10T16:11:17.233556+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:11:17.233556+0000","last_undegraded":"2024-09-10T16:11:17.233556+0000","last_fullsized":"2024-09-10T16:11:17.233556+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'21198","reported_seq":29324,"reported_epoch":61,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:11:41.242799+0000","last_change":"2024-09-10T16:11:16.084896+0000","last_active":"2024-09-10T16:11:41.242799+0000","last_peered":"2024-09-10T16:11:41.242799+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:11:41.242799+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":353,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17361,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3972,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-351}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":8895,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292096,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":42495,"num_bytes_recovered":33097612,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":182743040,"data_stored":182645489,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161022,"ondisk_log_size":161022,"up":32,"acting":29,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986340,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":475948,"kb_used_data":111068,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93895892,"statfs":{"total":96636764160,"available":96149393408,"internally_reserved":0,"allocated":113733632,"data_stored":113364076,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659285,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":501476,"kb_used_data":532,"kb_used_omap":4,"kb_used_meta":500923,"kb_avail":93870364,"statfs":{"total":96636764160,"available":96123252736,"internally_reserved":0,"allocated":544768,"data_stored":174748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":512945838},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182568,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":547396,"kb_used_data":68716,"kb_used_omap":4,"kb_used_meta":478651,"kb_avail":93824444,"statfs":{"total":96636764160,"available":96076230656,"internally_reserved":0,"allocated":70365184,"data_stored":69987938,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":490139306},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":69853184,"data_stored":69817054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":112857088,"data_stored":112824571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:11:43.342 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-10T16:11:43.342 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "49'21082", 'reported_seq': 21256, 'reported_epoch': 61, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:11:17.231941+0000', 'last_change': '2024-09-10T15:59:16.331899+0000', 'last_active': '2024-09-10T16:11:17.231941+0000', 'last_peered': '2024-09-10T16:11:17.231941+0000', 'last_clean': '2024-09-10T15:59:14.139436+0000', 'last_became_active': '2024-09-10T15:59:16.318715+0000', 'last_became_peered': '2024-09-10T15:59:16.318715+0000', 'last_unstale': '2024-09-10T16:11:17.231941+0000', 'last_undegraded': '2024-09-10T15:59:15.877524+0000', 'last_fullsized': '2024-09-10T15:59:15.769042+0000', 'mapping_epoch': 42, 'log_start': "45'11000", 'ondisk_log_start': "45'11000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3537", 'last_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'objects_scrubbed': 3497, 'log_size': 10082, 'log_dups_size': 0, 'ondisk_log_size': 10082, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1224, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4237, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 17581, 'num_write_kb': 28172, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4235}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:11:43.342 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-10T16:11:43.342 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "49'20838", 'reported_seq': 21009, 'reported_epoch': 61, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:11:17.232728+0000', 'last_change': '2024-09-10T15:59:16.315641+0000', 'last_active': '2024-09-10T16:11:17.232728+0000', 'last_peered': '2024-09-10T16:11:17.232728+0000', 'last_clean': '2024-09-10T15:59:14.117001+0000', 'last_became_active': '2024-09-10T15:59:16.302245+0000', 'last_became_peered': '2024-09-10T15:59:16.302245+0000', 'last_unstale': '2024-09-10T16:11:17.232728+0000', 'last_undegraded': '2024-09-10T15:59:15.845012+0000', 'last_fullsized': '2024-09-10T15:59:15.770337+0000', 'mapping_epoch': 42, 'log_start': "45'10800", 'ondisk_log_start': "45'10800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3404", 'last_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'objects_scrubbed': 3367, 'log_size': 10038, 'log_dups_size': 0, 'ondisk_log_size': 10038, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 983, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17469, 'num_write_kb': 28204, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4304}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:11:43.342 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-10T16:11:43.342 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "49'21198", 'reported_seq': 29324, 'reported_epoch': 61, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:11:41.242799+0000', 'last_change': '2024-09-10T16:11:16.084896+0000', 'last_active': '2024-09-10T16:11:41.242799+0000', 'last_peered': '2024-09-10T16:11:41.242799+0000', 'last_clean': '2024-09-10T15:59:14.129780+0000', 'last_became_active': '2024-09-10T15:59:16.300932+0000', 'last_became_peered': '2024-09-10T15:59:16.300932+0000', 'last_unstale': '2024-09-10T16:11:41.242799+0000', 'last_undegraded': '2024-09-10T15:59:15.887823+0000', 'last_fullsized': '2024-09-10T15:59:15.768513+0000', 'mapping_epoch': 42, 'log_start': "45'11100", 'ondisk_log_start': "45'11100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3880", 'last_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:06.369334+0000', 'objects_scrubbed': 3827, 'log_size': 10098, 'log_dups_size': 0, 'ondisk_log_size': 10098, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1397, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 353, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17361, 'num_write_kb': 27072, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3972, 'num_bytes_recovered': 483, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -351}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:11:43.342 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T16:11:43.768 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:11:43.768 INFO:teuthology.orchestra.run.smithi115.stdout:{"fsid":"7effee29-df05-4351-9741-ee203564ed81","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 8895/24 objects degraded (37062.500%), 3 pgs degraded, 3 pgs undersized","count":6},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":867,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":62,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983954,"num_in_osds":3,"osd_in_since":1725983947,"num_remapped_pgs":2},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":21},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":2},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":12,"data_bytes":4849,"bytes_used":1561415680,"bytes_avail":288348876800,"bytes_total":289910292480,"degraded_objects":8895,"degraded_total":24,"degraded_ratio":370.625,"recovering_objects_per_sec":201,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":2424,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-10T15:57:14:967120+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T16:00:13.560776+0000","services":{}},"progress_events":{}} 2024-09-10T16:11:43.782 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-10T16:11:43.782 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:11:44.099 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:11:44.100 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:11:44.112 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":489,"stamp":"2024-09-10T16:11:43.796316+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":8895,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292098,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":42497,"num_bytes_recovered":33097631,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161025,"ondisk_log_size":161025,"up":48,"acting":45,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1524820,"kb_used_data":180316,"kb_used_omap":12,"kb_used_meta":1344435,"kb_avail":281590700,"statfs":{"total":289910292480,"available":288348876800,"internally_reserved":0,"allocated":184643584,"data_stored":183526762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1376701442},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1760,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1760,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.001500"},"pg_stats":[{"pgid":"2.f","version":"49'20672","reported_seq":29309,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.232412+0000","last_change":"2024-09-10T16:09:57.088531+0000","last_active":"2024-09-10T16:11:17.232412+0000","last_peered":"2024-09-10T16:11:17.232412+0000","last_clean":"2024-09-10T16:11:17.232412+0000","last_became_active":"2024-09-10T16:09:57.005755+0000","last_became_peered":"2024-09-10T16:09:57.005755+0000","last_unstale":"2024-09-10T16:11:17.232412+0000","last_undegraded":"2024-09-10T16:11:17.232412+0000","last_fullsized":"2024-09-10T16:11:17.232412+0000","mapping_epoch":54,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:09:19.329697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20672,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4259,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20844","reported_seq":20992,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.232030+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:11:17.232030+0000","last_peered":"2024-09-10T16:11:17.232030+0000","last_clean":"2024-09-10T16:11:17.232030+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:11:17.232030+0000","last_undegraded":"2024-09-10T16:11:17.232030+0000","last_fullsized":"2024-09-10T16:11:17.232030+0000","mapping_epoch":35,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18949,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20798","reported_seq":29512,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.234237+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:11:17.234237+0000","last_peered":"2024-09-10T16:11:17.234237+0000","last_clean":"2024-09-10T16:11:17.234237+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:11:17.234237+0000","last_undegraded":"2024-09-10T16:11:17.234237+0000","last_fullsized":"2024-09-10T16:11:17.234237+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18802,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20660","reported_seq":20776,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.231774+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:11:17.231774+0000","last_peered":"2024-09-10T16:11:17.231774+0000","last_clean":"2024-09-10T16:11:17.231774+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:11:17.231774+0000","last_undegraded":"2024-09-10T16:11:17.231774+0000","last_fullsized":"2024-09-10T16:11:17.231774+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20660,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20891","reported_seq":21057,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.231216+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:11:17.231216+0000","last_peered":"2024-09-10T16:11:17.231216+0000","last_clean":"2024-09-10T16:11:17.231216+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:11:17.231216+0000","last_undegraded":"2024-09-10T16:11:17.231216+0000","last_fullsized":"2024-09-10T16:11:17.231216+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17602,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'21061","reported_seq":21226,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.233215+0000","last_change":"2024-09-10T16:10:52.241702+0000","last_active":"2024-09-10T16:11:17.233215+0000","last_peered":"2024-09-10T16:11:17.233215+0000","last_clean":"2024-09-10T16:11:17.233215+0000","last_became_active":"2024-09-10T16:10:52.171236+0000","last_became_peered":"2024-09-10T16:10:52.171236+0000","last_unstale":"2024-09-10T16:11:17.233215+0000","last_undegraded":"2024-09-10T16:11:17.233215+0000","last_fullsized":"2024-09-10T16:11:17.233215+0000","mapping_epoch":58,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:04:46.778280+0000","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17673,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4372,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'21082","reported_seq":21256,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:11:17.231941+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:11:17.231941+0000","last_peered":"2024-09-10T16:11:17.231941+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:11:17.231941+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17581,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4235},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'21314","reported_seq":21480,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.231025+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:11:17.231025+0000","last_peered":"2024-09-10T16:11:17.231025+0000","last_clean":"2024-09-10T16:11:17.231025+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:11:17.231025+0000","last_undegraded":"2024-09-10T16:11:17.231025+0000","last_fullsized":"2024-09-10T16:11:17.231025+0000","mapping_epoch":34,"log_start":"45'11300","ondisk_log_start":"45'11300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18161,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.232985+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:11:17.232985+0000","last_peered":"2024-09-10T16:11:17.232985+0000","last_clean":"2024-09-10T16:11:17.232985+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:11:17.232985+0000","last_undegraded":"2024-09-10T16:11:17.232985+0000","last_fullsized":"2024-09-10T16:11:17.232985+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20457","reported_seq":20637,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.233390+0000","last_change":"2024-09-10T16:09:33.012840+0000","last_active":"2024-09-10T16:11:17.233390+0000","last_peered":"2024-09-10T16:11:17.233390+0000","last_clean":"2024-09-10T16:11:17.233390+0000","last_became_active":"2024-09-10T16:09:32.934056+0000","last_became_peered":"2024-09-10T16:09:32.934056+0000","last_unstale":"2024-09-10T16:11:17.233390+0000","last_undegraded":"2024-09-10T16:11:17.233390+0000","last_fullsized":"2024-09-10T16:11:17.233390+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:08:30.481235+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18276,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":4579,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":130,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.232408+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:11:17.232408+0000","last_peered":"2024-09-10T16:11:17.232408+0000","last_clean":"2024-09-10T16:11:17.232408+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:11:17.232408+0000","last_undegraded":"2024-09-10T16:11:17.232408+0000","last_fullsized":"2024-09-10T16:11:17.232408+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20910","reported_seq":29599,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.231007+0000","last_change":"2024-09-10T16:09:08.909580+0000","last_active":"2024-09-10T16:11:17.231007+0000","last_peered":"2024-09-10T16:11:17.231007+0000","last_clean":"2024-09-10T16:11:17.231007+0000","last_became_active":"2024-09-10T16:09:08.854941+0000","last_became_peered":"2024-09-10T16:09:08.854941+0000","last_unstale":"2024-09-10T16:11:17.231007+0000","last_undegraded":"2024-09-10T16:11:17.231007+0000","last_fullsized":"2024-09-10T16:11:17.231007+0000","mapping_epoch":50,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:13:14.846691+0000","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17414,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4257,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":140,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.232615+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:11:17.232615+0000","last_peered":"2024-09-10T16:11:17.232615+0000","last_clean":"2024-09-10T16:11:17.232615+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:11:17.232615+0000","last_undegraded":"2024-09-10T16:11:17.232615+0000","last_fullsized":"2024-09-10T16:11:17.232615+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20781","reported_seq":29570,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.232496+0000","last_change":"2024-09-10T16:10:20.907889+0000","last_active":"2024-09-10T16:11:17.232496+0000","last_peered":"2024-09-10T16:11:17.232496+0000","last_clean":"2024-09-10T16:11:17.232496+0000","last_became_active":"2024-09-10T16:10:20.852065+0000","last_became_peered":"2024-09-10T16:10:20.852065+0000","last_unstale":"2024-09-10T16:11:17.232496+0000","last_undegraded":"2024-09-10T16:11:17.232496+0000","last_fullsized":"2024-09-10T16:11:17.232496+0000","mapping_epoch":56,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:34.069549+0000","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18461,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4307,"num_bytes_recovered":8675,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":115,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.233348+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:11:17.233348+0000","last_peered":"2024-09-10T16:11:17.233348+0000","last_clean":"2024-09-10T16:11:17.233348+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:11:17.233348+0000","last_undegraded":"2024-09-10T16:11:17.233348+0000","last_fullsized":"2024-09-10T16:11:17.233348+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'21170","reported_seq":30055,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.231608+0000","last_change":"2024-09-10T16:10:50.728349+0000","last_active":"2024-09-10T16:11:17.231608+0000","last_peered":"2024-09-10T16:11:17.231608+0000","last_clean":"2024-09-10T16:11:17.231608+0000","last_became_active":"2024-09-10T16:08:10.860438+0000","last_became_peered":"2024-09-10T16:08:10.860438+0000","last_unstale":"2024-09-10T16:11:17.231608+0000","last_undegraded":"2024-09-10T16:11:17.231608+0000","last_fullsized":"2024-09-10T16:11:17.231608+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:38:13.781380+0000","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18724,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20838","reported_seq":21009,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:11:17.232728+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:11:17.232728+0000","last_peered":"2024-09-10T16:11:17.232728+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:11:17.232728+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17469,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":105,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.233051+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:11:17.233051+0000","last_peered":"2024-09-10T16:11:17.233051+0000","last_clean":"2024-09-10T16:11:17.233051+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:11:17.233051+0000","last_undegraded":"2024-09-10T16:11:17.233051+0000","last_fullsized":"2024-09-10T16:11:17.233051+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20598","reported_seq":29199,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.233507+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:11:17.233507+0000","last_peered":"2024-09-10T16:11:17.233507+0000","last_clean":"2024-09-10T16:11:17.233507+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:11:17.233507+0000","last_undegraded":"2024-09-10T16:11:17.233507+0000","last_fullsized":"2024-09-10T16:11:17.233507+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16988,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":130,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.232065+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:11:17.232065+0000","last_peered":"2024-09-10T16:11:17.232065+0000","last_clean":"2024-09-10T16:11:17.232065+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:11:17.232065+0000","last_undegraded":"2024-09-10T16:11:17.232065+0000","last_fullsized":"2024-09-10T16:11:17.232065+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'21048","reported_seq":21210,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.319438+0000","last_change":"2024-09-10T16:11:17.315372+0000","last_active":"2024-09-10T16:11:17.319438+0000","last_peered":"2024-09-10T16:11:17.319438+0000","last_clean":"2024-09-10T16:11:17.319438+0000","last_became_active":"2024-09-10T16:11:17.234487+0000","last_became_peered":"2024-09-10T16:11:17.234487+0000","last_unstale":"2024-09-10T16:11:17.319438+0000","last_undegraded":"2024-09-10T16:11:17.319438+0000","last_fullsized":"2024-09-10T16:11:17.319438+0000","mapping_epoch":60,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:19:00.114196+0000","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17303,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4373,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":105,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.233467+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:11:17.233467+0000","last_peered":"2024-09-10T16:11:17.233467+0000","last_clean":"2024-09-10T16:11:17.233467+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:11:17.233467+0000","last_undegraded":"2024-09-10T16:11:17.233467+0000","last_fullsized":"2024-09-10T16:11:17.233467+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":110,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.233556+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:11:17.233556+0000","last_peered":"2024-09-10T16:11:17.233556+0000","last_clean":"2024-09-10T16:11:17.233556+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:11:17.233556+0000","last_undegraded":"2024-09-10T16:11:17.233556+0000","last_fullsized":"2024-09-10T16:11:17.233556+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'21198","reported_seq":29324,"reported_epoch":61,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:11:41.242799+0000","last_change":"2024-09-10T16:11:16.084896+0000","last_active":"2024-09-10T16:11:41.242799+0000","last_peered":"2024-09-10T16:11:41.242799+0000","last_clean":"2024-09-10T15:59:14.129780+0000","last_became_active":"2024-09-10T15:59:16.300932+0000","last_became_peered":"2024-09-10T15:59:16.300932+0000","last_unstale":"2024-09-10T16:11:41.242799+0000","last_undegraded":"2024-09-10T15:59:15.887823+0000","last_fullsized":"2024-09-10T15:59:15.768513+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":353,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17361,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3972,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-351}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":8895,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292096,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":42495,"num_bytes_recovered":33097612,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":182743040,"data_stored":182645489,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161022,"ondisk_log_size":161022,"up":32,"acting":29,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986340,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":475948,"kb_used_data":111068,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93895892,"statfs":{"total":96636764160,"available":96149393408,"internally_reserved":0,"allocated":113733632,"data_stored":113364076,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659285,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":501476,"kb_used_data":532,"kb_used_omap":4,"kb_used_meta":500923,"kb_avail":93870364,"statfs":{"total":96636764160,"available":96123252736,"internally_reserved":0,"allocated":544768,"data_stored":174748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":512945838},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182568,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":547396,"kb_used_data":68716,"kb_used_omap":4,"kb_used_meta":478651,"kb_avail":93824444,"statfs":{"total":96636764160,"available":96076230656,"internally_reserved":0,"allocated":70365184,"data_stored":69987938,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":490139306},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":69853184,"data_stored":69817054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":112857088,"data_stored":112824571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:11:47.114 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:11:47.493 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:11:47.493 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:11:47.507 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":491,"stamp":"2024-09-10T16:11:45.796634+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":8542,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292098,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":42850,"num_bytes_recovered":33097631,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161025,"ondisk_log_size":161025,"up":48,"acting":46,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1533152,"kb_used_data":180328,"kb_used_omap":12,"kb_used_meta":1352755,"kb_avail":281582368,"statfs":{"total":289910292480,"available":288340344832,"internally_reserved":0,"allocated":184655872,"data_stored":183531812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1385221122},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1335,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1335,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001236"},"pg_stats":[{"pgid":"2.f","version":"49'20672","reported_seq":29309,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.232412+0000","last_change":"2024-09-10T16:09:57.088531+0000","last_active":"2024-09-10T16:11:17.232412+0000","last_peered":"2024-09-10T16:11:17.232412+0000","last_clean":"2024-09-10T16:11:17.232412+0000","last_became_active":"2024-09-10T16:09:57.005755+0000","last_became_peered":"2024-09-10T16:09:57.005755+0000","last_unstale":"2024-09-10T16:11:17.232412+0000","last_undegraded":"2024-09-10T16:11:17.232412+0000","last_fullsized":"2024-09-10T16:11:17.232412+0000","mapping_epoch":54,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:09:19.329697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20672,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4259,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20844","reported_seq":20992,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.232030+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:11:17.232030+0000","last_peered":"2024-09-10T16:11:17.232030+0000","last_clean":"2024-09-10T16:11:17.232030+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:11:17.232030+0000","last_undegraded":"2024-09-10T16:11:17.232030+0000","last_fullsized":"2024-09-10T16:11:17.232030+0000","mapping_epoch":35,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18949,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20798","reported_seq":29516,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310310+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:11:44.310310+0000","last_peered":"2024-09-10T16:11:44.310310+0000","last_clean":"2024-09-10T16:11:44.310310+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:11:44.310310+0000","last_undegraded":"2024-09-10T16:11:44.310310+0000","last_fullsized":"2024-09-10T16:11:44.310310+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18802,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20660","reported_seq":20776,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.231774+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:11:17.231774+0000","last_peered":"2024-09-10T16:11:17.231774+0000","last_clean":"2024-09-10T16:11:17.231774+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:11:17.231774+0000","last_undegraded":"2024-09-10T16:11:17.231774+0000","last_fullsized":"2024-09-10T16:11:17.231774+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20660,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20891","reported_seq":21057,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.231216+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:11:17.231216+0000","last_peered":"2024-09-10T16:11:17.231216+0000","last_clean":"2024-09-10T16:11:17.231216+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:11:17.231216+0000","last_undegraded":"2024-09-10T16:11:17.231216+0000","last_fullsized":"2024-09-10T16:11:17.231216+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17602,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'21061","reported_seq":21230,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310143+0000","last_change":"2024-09-10T16:10:52.241702+0000","last_active":"2024-09-10T16:11:44.310143+0000","last_peered":"2024-09-10T16:11:44.310143+0000","last_clean":"2024-09-10T16:11:44.310143+0000","last_became_active":"2024-09-10T16:10:52.171236+0000","last_became_peered":"2024-09-10T16:10:52.171236+0000","last_unstale":"2024-09-10T16:11:44.310143+0000","last_undegraded":"2024-09-10T16:11:44.310143+0000","last_fullsized":"2024-09-10T16:11:44.310143+0000","mapping_epoch":58,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:04:46.778280+0000","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17673,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4372,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'21082","reported_seq":21256,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:11:17.231941+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:11:17.231941+0000","last_peered":"2024-09-10T16:11:17.231941+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:11:17.231941+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17581,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4235},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'21314","reported_seq":21480,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.231025+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:11:17.231025+0000","last_peered":"2024-09-10T16:11:17.231025+0000","last_clean":"2024-09-10T16:11:17.231025+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:11:17.231025+0000","last_undegraded":"2024-09-10T16:11:17.231025+0000","last_fullsized":"2024-09-10T16:11:17.231025+0000","mapping_epoch":34,"log_start":"45'11300","ondisk_log_start":"45'11300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18161,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.309727+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:11:44.309727+0000","last_peered":"2024-09-10T16:11:44.309727+0000","last_clean":"2024-09-10T16:11:44.309727+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:11:44.309727+0000","last_undegraded":"2024-09-10T16:11:44.309727+0000","last_fullsized":"2024-09-10T16:11:44.309727+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20457","reported_seq":20641,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310911+0000","last_change":"2024-09-10T16:09:33.012840+0000","last_active":"2024-09-10T16:11:44.310911+0000","last_peered":"2024-09-10T16:11:44.310911+0000","last_clean":"2024-09-10T16:11:44.310911+0000","last_became_active":"2024-09-10T16:09:32.934056+0000","last_became_peered":"2024-09-10T16:09:32.934056+0000","last_unstale":"2024-09-10T16:11:44.310911+0000","last_undegraded":"2024-09-10T16:11:44.310911+0000","last_fullsized":"2024-09-10T16:11:44.310911+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:08:30.481235+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18276,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":4579,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":130,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.232408+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:11:17.232408+0000","last_peered":"2024-09-10T16:11:17.232408+0000","last_clean":"2024-09-10T16:11:17.232408+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:11:17.232408+0000","last_undegraded":"2024-09-10T16:11:17.232408+0000","last_fullsized":"2024-09-10T16:11:17.232408+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20910","reported_seq":29599,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.231007+0000","last_change":"2024-09-10T16:09:08.909580+0000","last_active":"2024-09-10T16:11:17.231007+0000","last_peered":"2024-09-10T16:11:17.231007+0000","last_clean":"2024-09-10T16:11:17.231007+0000","last_became_active":"2024-09-10T16:09:08.854941+0000","last_became_peered":"2024-09-10T16:09:08.854941+0000","last_unstale":"2024-09-10T16:11:17.231007+0000","last_undegraded":"2024-09-10T16:11:17.231007+0000","last_fullsized":"2024-09-10T16:11:17.231007+0000","mapping_epoch":50,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:13:14.846691+0000","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17414,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4257,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":140,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.232615+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:11:17.232615+0000","last_peered":"2024-09-10T16:11:17.232615+0000","last_clean":"2024-09-10T16:11:17.232615+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:11:17.232615+0000","last_undegraded":"2024-09-10T16:11:17.232615+0000","last_fullsized":"2024-09-10T16:11:17.232615+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20781","reported_seq":29570,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.232496+0000","last_change":"2024-09-10T16:10:20.907889+0000","last_active":"2024-09-10T16:11:17.232496+0000","last_peered":"2024-09-10T16:11:17.232496+0000","last_clean":"2024-09-10T16:11:17.232496+0000","last_became_active":"2024-09-10T16:10:20.852065+0000","last_became_peered":"2024-09-10T16:10:20.852065+0000","last_unstale":"2024-09-10T16:11:17.232496+0000","last_undegraded":"2024-09-10T16:11:17.232496+0000","last_fullsized":"2024-09-10T16:11:17.232496+0000","mapping_epoch":56,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:34.069549+0000","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18461,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4307,"num_bytes_recovered":8675,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":119,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310825+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:11:44.310825+0000","last_peered":"2024-09-10T16:11:44.310825+0000","last_clean":"2024-09-10T16:11:44.310825+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:11:44.310825+0000","last_undegraded":"2024-09-10T16:11:44.310825+0000","last_fullsized":"2024-09-10T16:11:44.310825+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'21170","reported_seq":30055,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.231608+0000","last_change":"2024-09-10T16:10:50.728349+0000","last_active":"2024-09-10T16:11:17.231608+0000","last_peered":"2024-09-10T16:11:17.231608+0000","last_clean":"2024-09-10T16:11:17.231608+0000","last_became_active":"2024-09-10T16:08:10.860438+0000","last_became_peered":"2024-09-10T16:08:10.860438+0000","last_unstale":"2024-09-10T16:11:17.231608+0000","last_undegraded":"2024-09-10T16:11:17.231608+0000","last_fullsized":"2024-09-10T16:11:17.231608+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:38:13.781380+0000","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18724,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20838","reported_seq":21009,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:11:17.232728+0000","last_change":"2024-09-10T15:59:16.315641+0000","last_active":"2024-09-10T16:11:17.232728+0000","last_peered":"2024-09-10T16:11:17.232728+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:11:17.232728+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17469,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":109,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310190+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:11:44.310190+0000","last_peered":"2024-09-10T16:11:44.310190+0000","last_clean":"2024-09-10T16:11:44.310190+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:11:44.310190+0000","last_undegraded":"2024-09-10T16:11:44.310190+0000","last_fullsized":"2024-09-10T16:11:44.310190+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20598","reported_seq":29204,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310762+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:11:44.310762+0000","last_peered":"2024-09-10T16:11:44.310762+0000","last_clean":"2024-09-10T16:11:44.310762+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:11:44.310762+0000","last_undegraded":"2024-09-10T16:11:44.310762+0000","last_fullsized":"2024-09-10T16:11:44.310762+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16988,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":130,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T16:11:17.232065+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:11:17.232065+0000","last_peered":"2024-09-10T16:11:17.232065+0000","last_clean":"2024-09-10T16:11:17.232065+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:11:17.232065+0000","last_undegraded":"2024-09-10T16:11:17.232065+0000","last_fullsized":"2024-09-10T16:11:17.232065+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'21048","reported_seq":21214,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310569+0000","last_change":"2024-09-10T16:11:17.315372+0000","last_active":"2024-09-10T16:11:44.310569+0000","last_peered":"2024-09-10T16:11:44.310569+0000","last_clean":"2024-09-10T16:11:44.310569+0000","last_became_active":"2024-09-10T16:11:17.234487+0000","last_became_peered":"2024-09-10T16:11:17.234487+0000","last_unstale":"2024-09-10T16:11:44.310569+0000","last_undegraded":"2024-09-10T16:11:44.310569+0000","last_fullsized":"2024-09-10T16:11:44.310569+0000","mapping_epoch":60,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:19:00.114196+0000","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17303,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4373,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":109,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310658+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:11:44.310658+0000","last_peered":"2024-09-10T16:11:44.310658+0000","last_clean":"2024-09-10T16:11:44.310658+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:11:44.310658+0000","last_undegraded":"2024-09-10T16:11:44.310658+0000","last_fullsized":"2024-09-10T16:11:44.310658+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":114,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310839+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:11:44.310839+0000","last_peered":"2024-09-10T16:11:44.310839+0000","last_clean":"2024-09-10T16:11:44.310839+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:11:44.310839+0000","last_undegraded":"2024-09-10T16:11:44.310839+0000","last_fullsized":"2024-09-10T16:11:44.310839+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'21198","reported_seq":21365,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.396057+0000","last_change":"2024-09-10T16:11:44.391021+0000","last_active":"2024-09-10T16:11:44.396057+0000","last_peered":"2024-09-10T16:11:44.396057+0000","last_clean":"2024-09-10T16:11:44.396057+0000","last_became_active":"2024-09-10T16:11:44.312647+0000","last_became_peered":"2024-09-10T16:11:44.312647+0000","last_unstale":"2024-09-10T16:11:44.396057+0000","last_undegraded":"2024-09-10T16:11:44.396057+0000","last_fullsized":"2024-09-10T16:11:44.396057+0000","mapping_epoch":62,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:33:28.041611+0000","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17361,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4325,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":8542,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292096,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":42848,"num_bytes_recovered":33097612,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":182738944,"data_stored":182645006,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161022,"ondisk_log_size":161022,"up":32,"acting":30,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986340,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":475948,"kb_used_data":111068,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93895892,"statfs":{"total":96636764160,"available":96149393408,"internally_reserved":0,"allocated":113733632,"data_stored":113364076,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659286,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":509808,"kb_used_data":544,"kb_used_omap":4,"kb_used_meta":509243,"kb_avail":93862032,"statfs":{"total":96636764160,"available":96114720768,"internally_reserved":0,"allocated":557056,"data_stored":179798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":521465518},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182568,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":547396,"kb_used_data":68716,"kb_used_omap":4,"kb_used_meta":478651,"kb_avail":93824444,"statfs":{"total":96636764160,"available":96076230656,"internally_reserved":0,"allocated":70365184,"data_stored":69987938,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":490139306},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":69853184,"data_stored":69817054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":112857088,"data_stored":112824571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:11:47.509 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:11:47.849 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:11:47.850 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:11:47.865 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":492,"stamp":"2024-09-10T16:11:47.796968+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7961,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292098,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43431,"num_bytes_recovered":33097631,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161025,"ondisk_log_size":161025,"up":48,"acting":46,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1529952,"kb_used_data":177124,"kb_used_omap":12,"kb_used_meta":1352755,"kb_avail":281585568,"statfs":{"total":289910292480,"available":288343621632,"internally_reserved":0,"allocated":181374976,"data_stored":180229214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1385221122},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1916,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1916,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001252"},"pg_stats":[{"pgid":"2.f","version":"49'20672","reported_seq":29313,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.314657+0000","last_change":"2024-09-10T16:09:57.088531+0000","last_active":"2024-09-10T16:11:44.314657+0000","last_peered":"2024-09-10T16:11:44.314657+0000","last_clean":"2024-09-10T16:11:44.314657+0000","last_became_active":"2024-09-10T16:09:57.005755+0000","last_became_peered":"2024-09-10T16:09:57.005755+0000","last_unstale":"2024-09-10T16:11:44.314657+0000","last_undegraded":"2024-09-10T16:11:44.314657+0000","last_fullsized":"2024-09-10T16:11:44.314657+0000","mapping_epoch":54,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:09:19.329697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20672,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4259,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20844","reported_seq":20996,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.311112+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:11:44.311112+0000","last_peered":"2024-09-10T16:11:44.311112+0000","last_clean":"2024-09-10T16:11:44.311112+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:11:44.311112+0000","last_undegraded":"2024-09-10T16:11:44.311112+0000","last_fullsized":"2024-09-10T16:11:44.311112+0000","mapping_epoch":35,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18949,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20798","reported_seq":29516,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310310+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:11:44.310310+0000","last_peered":"2024-09-10T16:11:44.310310+0000","last_clean":"2024-09-10T16:11:44.310310+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:11:44.310310+0000","last_undegraded":"2024-09-10T16:11:44.310310+0000","last_fullsized":"2024-09-10T16:11:44.310310+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18802,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20660","reported_seq":20780,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310730+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:11:44.310730+0000","last_peered":"2024-09-10T16:11:44.310730+0000","last_clean":"2024-09-10T16:11:44.310730+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:11:44.310730+0000","last_undegraded":"2024-09-10T16:11:44.310730+0000","last_fullsized":"2024-09-10T16:11:44.310730+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20660,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20891","reported_seq":21061,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.312021+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:11:44.312021+0000","last_peered":"2024-09-10T16:11:44.312021+0000","last_clean":"2024-09-10T16:11:44.312021+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:11:44.312021+0000","last_undegraded":"2024-09-10T16:11:44.312021+0000","last_fullsized":"2024-09-10T16:11:44.312021+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17602,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'21061","reported_seq":21230,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310143+0000","last_change":"2024-09-10T16:10:52.241702+0000","last_active":"2024-09-10T16:11:44.310143+0000","last_peered":"2024-09-10T16:11:44.310143+0000","last_clean":"2024-09-10T16:11:44.310143+0000","last_became_active":"2024-09-10T16:10:52.171236+0000","last_became_peered":"2024-09-10T16:10:52.171236+0000","last_unstale":"2024-09-10T16:11:44.310143+0000","last_undegraded":"2024-09-10T16:11:44.310143+0000","last_fullsized":"2024-09-10T16:11:44.310143+0000","mapping_epoch":58,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:04:46.778280+0000","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17673,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4372,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'21082","reported_seq":21260,"reported_epoch":63,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:11:44.312026+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:11:44.312026+0000","last_peered":"2024-09-10T16:11:44.312026+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:11:44.312026+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17581,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4235},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'21314","reported_seq":21484,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.311841+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:11:44.311841+0000","last_peered":"2024-09-10T16:11:44.311841+0000","last_clean":"2024-09-10T16:11:44.311841+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:11:44.311841+0000","last_undegraded":"2024-09-10T16:11:44.311841+0000","last_fullsized":"2024-09-10T16:11:44.311841+0000","mapping_epoch":34,"log_start":"45'11300","ondisk_log_start":"45'11300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18161,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.309727+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:11:44.309727+0000","last_peered":"2024-09-10T16:11:44.309727+0000","last_clean":"2024-09-10T16:11:44.309727+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:11:44.309727+0000","last_undegraded":"2024-09-10T16:11:44.309727+0000","last_fullsized":"2024-09-10T16:11:44.309727+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20457","reported_seq":20641,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310911+0000","last_change":"2024-09-10T16:09:33.012840+0000","last_active":"2024-09-10T16:11:44.310911+0000","last_peered":"2024-09-10T16:11:44.310911+0000","last_clean":"2024-09-10T16:11:44.310911+0000","last_became_active":"2024-09-10T16:09:32.934056+0000","last_became_peered":"2024-09-10T16:09:32.934056+0000","last_unstale":"2024-09-10T16:11:44.310911+0000","last_undegraded":"2024-09-10T16:11:44.310911+0000","last_fullsized":"2024-09-10T16:11:44.310911+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:08:30.481235+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18276,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":4579,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":134,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310369+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:11:44.310369+0000","last_peered":"2024-09-10T16:11:44.310369+0000","last_clean":"2024-09-10T16:11:44.310369+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:11:44.310369+0000","last_undegraded":"2024-09-10T16:11:44.310369+0000","last_fullsized":"2024-09-10T16:11:44.310369+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20910","reported_seq":29603,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.312044+0000","last_change":"2024-09-10T16:09:08.909580+0000","last_active":"2024-09-10T16:11:44.312044+0000","last_peered":"2024-09-10T16:11:44.312044+0000","last_clean":"2024-09-10T16:11:44.312044+0000","last_became_active":"2024-09-10T16:09:08.854941+0000","last_became_peered":"2024-09-10T16:09:08.854941+0000","last_unstale":"2024-09-10T16:11:44.312044+0000","last_undegraded":"2024-09-10T16:11:44.312044+0000","last_fullsized":"2024-09-10T16:11:44.312044+0000","mapping_epoch":50,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:13:14.846691+0000","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17414,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4257,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":144,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.311936+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:11:44.311936+0000","last_peered":"2024-09-10T16:11:44.311936+0000","last_clean":"2024-09-10T16:11:44.311936+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:11:44.311936+0000","last_undegraded":"2024-09-10T16:11:44.311936+0000","last_fullsized":"2024-09-10T16:11:44.311936+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20781","reported_seq":29574,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310450+0000","last_change":"2024-09-10T16:10:20.907889+0000","last_active":"2024-09-10T16:11:44.310450+0000","last_peered":"2024-09-10T16:11:44.310450+0000","last_clean":"2024-09-10T16:11:44.310450+0000","last_became_active":"2024-09-10T16:10:20.852065+0000","last_became_peered":"2024-09-10T16:10:20.852065+0000","last_unstale":"2024-09-10T16:11:44.310450+0000","last_undegraded":"2024-09-10T16:11:44.310450+0000","last_fullsized":"2024-09-10T16:11:44.310450+0000","mapping_epoch":56,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:34.069549+0000","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18461,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4307,"num_bytes_recovered":8675,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":119,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310825+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:11:44.310825+0000","last_peered":"2024-09-10T16:11:44.310825+0000","last_clean":"2024-09-10T16:11:44.310825+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:11:44.310825+0000","last_undegraded":"2024-09-10T16:11:44.310825+0000","last_fullsized":"2024-09-10T16:11:44.310825+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'21170","reported_seq":30059,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310126+0000","last_change":"2024-09-10T16:10:50.728349+0000","last_active":"2024-09-10T16:11:44.310126+0000","last_peered":"2024-09-10T16:11:44.310126+0000","last_clean":"2024-09-10T16:11:44.310126+0000","last_became_active":"2024-09-10T16:08:10.860438+0000","last_became_peered":"2024-09-10T16:08:10.860438+0000","last_unstale":"2024-09-10T16:11:44.310126+0000","last_undegraded":"2024-09-10T16:11:44.310126+0000","last_fullsized":"2024-09-10T16:11:44.310126+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:38:13.781380+0000","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18724,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20838","reported_seq":22176,"reported_epoch":63,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:11:46.242029+0000","last_change":"2024-09-10T16:11:42.563598+0000","last_active":"2024-09-10T16:11:46.242029+0000","last_peered":"2024-09-10T16:11:46.242029+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:11:46.242029+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3724,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17469,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":581,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3724}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":109,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310190+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:11:44.310190+0000","last_peered":"2024-09-10T16:11:44.310190+0000","last_clean":"2024-09-10T16:11:44.310190+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:11:44.310190+0000","last_undegraded":"2024-09-10T16:11:44.310190+0000","last_fullsized":"2024-09-10T16:11:44.310190+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20598","reported_seq":29204,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310762+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:11:44.310762+0000","last_peered":"2024-09-10T16:11:44.310762+0000","last_clean":"2024-09-10T16:11:44.310762+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:11:44.310762+0000","last_undegraded":"2024-09-10T16:11:44.310762+0000","last_fullsized":"2024-09-10T16:11:44.310762+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16988,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":134,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310946+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:11:44.310946+0000","last_peered":"2024-09-10T16:11:44.310946+0000","last_clean":"2024-09-10T16:11:44.310946+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:11:44.310946+0000","last_undegraded":"2024-09-10T16:11:44.310946+0000","last_fullsized":"2024-09-10T16:11:44.310946+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'21048","reported_seq":21214,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310569+0000","last_change":"2024-09-10T16:11:17.315372+0000","last_active":"2024-09-10T16:11:44.310569+0000","last_peered":"2024-09-10T16:11:44.310569+0000","last_clean":"2024-09-10T16:11:44.310569+0000","last_became_active":"2024-09-10T16:11:17.234487+0000","last_became_peered":"2024-09-10T16:11:17.234487+0000","last_unstale":"2024-09-10T16:11:44.310569+0000","last_undegraded":"2024-09-10T16:11:44.310569+0000","last_fullsized":"2024-09-10T16:11:44.310569+0000","mapping_epoch":60,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:19:00.114196+0000","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17303,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4373,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":109,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310658+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:11:44.310658+0000","last_peered":"2024-09-10T16:11:44.310658+0000","last_clean":"2024-09-10T16:11:44.310658+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:11:44.310658+0000","last_undegraded":"2024-09-10T16:11:44.310658+0000","last_fullsized":"2024-09-10T16:11:44.310658+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":114,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310839+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:11:44.310839+0000","last_peered":"2024-09-10T16:11:44.310839+0000","last_clean":"2024-09-10T16:11:44.310839+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:11:44.310839+0000","last_undegraded":"2024-09-10T16:11:44.310839+0000","last_fullsized":"2024-09-10T16:11:44.310839+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'21198","reported_seq":21365,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.396057+0000","last_change":"2024-09-10T16:11:44.391021+0000","last_active":"2024-09-10T16:11:44.396057+0000","last_peered":"2024-09-10T16:11:44.396057+0000","last_clean":"2024-09-10T16:11:44.396057+0000","last_became_active":"2024-09-10T16:11:44.312647+0000","last_became_peered":"2024-09-10T16:11:44.312647+0000","last_unstale":"2024-09-10T16:11:44.396057+0000","last_undegraded":"2024-09-10T16:11:44.396057+0000","last_fullsized":"2024-09-10T16:11:44.396057+0000","mapping_epoch":62,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:33:28.041611+0000","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17361,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4325,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7961,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292096,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43429,"num_bytes_recovered":33097612,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":179425280,"data_stored":179331342,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161022,"ondisk_log_size":161022,"up":32,"acting":30,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986341,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":472732,"kb_used_data":107848,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93899108,"statfs":{"total":96636764160,"available":96152686592,"internally_reserved":0,"allocated":110436352,"data_stored":110055945,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659286,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":509808,"kb_used_data":544,"kb_used_omap":4,"kb_used_meta":509243,"kb_avail":93862032,"statfs":{"total":96636764160,"available":96114720768,"internally_reserved":0,"allocated":557056,"data_stored":179798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":521465518},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182569,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":547412,"kb_used_data":68732,"kb_used_omap":4,"kb_used_meta":478651,"kb_avail":93824428,"statfs":{"total":96636764160,"available":96076214272,"internally_reserved":0,"allocated":70381568,"data_stored":69993471,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":490139306},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":69853184,"data_stored":69817054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":109543424,"data_stored":109510907,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:11:47.867 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-10T16:11:47.867 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "49'21082", 'reported_seq': 21260, 'reported_epoch': 63, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:11:44.312026+0000', 'last_change': '2024-09-10T15:59:16.331899+0000', 'last_active': '2024-09-10T16:11:44.312026+0000', 'last_peered': '2024-09-10T16:11:44.312026+0000', 'last_clean': '2024-09-10T15:59:14.139436+0000', 'last_became_active': '2024-09-10T15:59:16.318715+0000', 'last_became_peered': '2024-09-10T15:59:16.318715+0000', 'last_unstale': '2024-09-10T16:11:44.312026+0000', 'last_undegraded': '2024-09-10T15:59:15.877524+0000', 'last_fullsized': '2024-09-10T15:59:15.769042+0000', 'mapping_epoch': 42, 'log_start': "45'11000", 'ondisk_log_start': "45'11000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3537", 'last_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'objects_scrubbed': 3497, 'log_size': 10082, 'log_dups_size': 0, 'ondisk_log_size': 10082, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1224, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4237, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 17581, 'num_write_kb': 28172, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4235}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:11:47.868 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-10T16:11:47.868 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "49'20838", 'reported_seq': 22176, 'reported_epoch': 63, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:11:46.242029+0000', 'last_change': '2024-09-10T16:11:42.563598+0000', 'last_active': '2024-09-10T16:11:46.242029+0000', 'last_peered': '2024-09-10T16:11:46.242029+0000', 'last_clean': '2024-09-10T15:59:14.117001+0000', 'last_became_active': '2024-09-10T15:59:16.302245+0000', 'last_became_peered': '2024-09-10T15:59:16.302245+0000', 'last_unstale': '2024-09-10T16:11:46.242029+0000', 'last_undegraded': '2024-09-10T15:59:15.845012+0000', 'last_fullsized': '2024-09-10T15:59:15.770337+0000', 'mapping_epoch': 42, 'log_start': "45'10800", 'ondisk_log_start': "45'10800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3404", 'last_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'objects_scrubbed': 3367, 'log_size': 10038, 'log_dups_size': 0, 'ondisk_log_size': 10038, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 983, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3724, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17469, 'num_write_kb': 28204, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 581, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3724}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:11:47.869 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T16:11:48.347 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:11:48.348 INFO:teuthology.orchestra.run.smithi115.stdout:{"fsid":"7effee29-df05-4351-9741-ee203564ed81","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 7961/24 objects degraded (33170.833%), 2 pgs degraded, 2 pgs undersized","count":4},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":872,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":63,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983954,"num_in_osds":3,"osd_in_since":1725983947,"num_remapped_pgs":2},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":22},{"state_name":"active+recovering+undersized+degraded+remapped","count":1},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":12,"data_bytes":4849,"bytes_used":1566670848,"bytes_avail":288343621632,"bytes_total":289910292480,"degraded_objects":7961,"degraded_total":24,"degraded_ratio":331.70833333333326,"recovering_objects_per_sec":239,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1916,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-10T15:57:14:967120+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T16:00:13.560776+0000","services":{}},"progress_events":{}} 2024-09-10T16:11:48.361 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-10T16:11:48.361 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:11:48.668 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:11:48.669 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:11:48.680 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":492,"stamp":"2024-09-10T16:11:47.796968+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7961,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292098,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43431,"num_bytes_recovered":33097631,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161025,"ondisk_log_size":161025,"up":48,"acting":46,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1529952,"kb_used_data":177124,"kb_used_omap":12,"kb_used_meta":1352755,"kb_avail":281585568,"statfs":{"total":289910292480,"available":288343621632,"internally_reserved":0,"allocated":181374976,"data_stored":180229214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1385221122},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1916,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1916,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001252"},"pg_stats":[{"pgid":"2.f","version":"49'20672","reported_seq":29313,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.314657+0000","last_change":"2024-09-10T16:09:57.088531+0000","last_active":"2024-09-10T16:11:44.314657+0000","last_peered":"2024-09-10T16:11:44.314657+0000","last_clean":"2024-09-10T16:11:44.314657+0000","last_became_active":"2024-09-10T16:09:57.005755+0000","last_became_peered":"2024-09-10T16:09:57.005755+0000","last_unstale":"2024-09-10T16:11:44.314657+0000","last_undegraded":"2024-09-10T16:11:44.314657+0000","last_fullsized":"2024-09-10T16:11:44.314657+0000","mapping_epoch":54,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:09:19.329697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20672,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4259,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20844","reported_seq":20996,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.311112+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:11:44.311112+0000","last_peered":"2024-09-10T16:11:44.311112+0000","last_clean":"2024-09-10T16:11:44.311112+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:11:44.311112+0000","last_undegraded":"2024-09-10T16:11:44.311112+0000","last_fullsized":"2024-09-10T16:11:44.311112+0000","mapping_epoch":35,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18949,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20798","reported_seq":29516,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310310+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:11:44.310310+0000","last_peered":"2024-09-10T16:11:44.310310+0000","last_clean":"2024-09-10T16:11:44.310310+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:11:44.310310+0000","last_undegraded":"2024-09-10T16:11:44.310310+0000","last_fullsized":"2024-09-10T16:11:44.310310+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18802,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20660","reported_seq":20780,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310730+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:11:44.310730+0000","last_peered":"2024-09-10T16:11:44.310730+0000","last_clean":"2024-09-10T16:11:44.310730+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:11:44.310730+0000","last_undegraded":"2024-09-10T16:11:44.310730+0000","last_fullsized":"2024-09-10T16:11:44.310730+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20660,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20891","reported_seq":21061,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.312021+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:11:44.312021+0000","last_peered":"2024-09-10T16:11:44.312021+0000","last_clean":"2024-09-10T16:11:44.312021+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:11:44.312021+0000","last_undegraded":"2024-09-10T16:11:44.312021+0000","last_fullsized":"2024-09-10T16:11:44.312021+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17602,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'21061","reported_seq":21230,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310143+0000","last_change":"2024-09-10T16:10:52.241702+0000","last_active":"2024-09-10T16:11:44.310143+0000","last_peered":"2024-09-10T16:11:44.310143+0000","last_clean":"2024-09-10T16:11:44.310143+0000","last_became_active":"2024-09-10T16:10:52.171236+0000","last_became_peered":"2024-09-10T16:10:52.171236+0000","last_unstale":"2024-09-10T16:11:44.310143+0000","last_undegraded":"2024-09-10T16:11:44.310143+0000","last_fullsized":"2024-09-10T16:11:44.310143+0000","mapping_epoch":58,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:04:46.778280+0000","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17673,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4372,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'21082","reported_seq":21260,"reported_epoch":63,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:11:44.312026+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:11:44.312026+0000","last_peered":"2024-09-10T16:11:44.312026+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:11:44.312026+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17581,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4235},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'21314","reported_seq":21484,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.311841+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:11:44.311841+0000","last_peered":"2024-09-10T16:11:44.311841+0000","last_clean":"2024-09-10T16:11:44.311841+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:11:44.311841+0000","last_undegraded":"2024-09-10T16:11:44.311841+0000","last_fullsized":"2024-09-10T16:11:44.311841+0000","mapping_epoch":34,"log_start":"45'11300","ondisk_log_start":"45'11300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18161,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.309727+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:11:44.309727+0000","last_peered":"2024-09-10T16:11:44.309727+0000","last_clean":"2024-09-10T16:11:44.309727+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:11:44.309727+0000","last_undegraded":"2024-09-10T16:11:44.309727+0000","last_fullsized":"2024-09-10T16:11:44.309727+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20457","reported_seq":20641,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310911+0000","last_change":"2024-09-10T16:09:33.012840+0000","last_active":"2024-09-10T16:11:44.310911+0000","last_peered":"2024-09-10T16:11:44.310911+0000","last_clean":"2024-09-10T16:11:44.310911+0000","last_became_active":"2024-09-10T16:09:32.934056+0000","last_became_peered":"2024-09-10T16:09:32.934056+0000","last_unstale":"2024-09-10T16:11:44.310911+0000","last_undegraded":"2024-09-10T16:11:44.310911+0000","last_fullsized":"2024-09-10T16:11:44.310911+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:08:30.481235+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18276,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":4579,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":134,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310369+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:11:44.310369+0000","last_peered":"2024-09-10T16:11:44.310369+0000","last_clean":"2024-09-10T16:11:44.310369+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:11:44.310369+0000","last_undegraded":"2024-09-10T16:11:44.310369+0000","last_fullsized":"2024-09-10T16:11:44.310369+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20910","reported_seq":29603,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.312044+0000","last_change":"2024-09-10T16:09:08.909580+0000","last_active":"2024-09-10T16:11:44.312044+0000","last_peered":"2024-09-10T16:11:44.312044+0000","last_clean":"2024-09-10T16:11:44.312044+0000","last_became_active":"2024-09-10T16:09:08.854941+0000","last_became_peered":"2024-09-10T16:09:08.854941+0000","last_unstale":"2024-09-10T16:11:44.312044+0000","last_undegraded":"2024-09-10T16:11:44.312044+0000","last_fullsized":"2024-09-10T16:11:44.312044+0000","mapping_epoch":50,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:13:14.846691+0000","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17414,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4257,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":144,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.311936+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:11:44.311936+0000","last_peered":"2024-09-10T16:11:44.311936+0000","last_clean":"2024-09-10T16:11:44.311936+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:11:44.311936+0000","last_undegraded":"2024-09-10T16:11:44.311936+0000","last_fullsized":"2024-09-10T16:11:44.311936+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20781","reported_seq":29574,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310450+0000","last_change":"2024-09-10T16:10:20.907889+0000","last_active":"2024-09-10T16:11:44.310450+0000","last_peered":"2024-09-10T16:11:44.310450+0000","last_clean":"2024-09-10T16:11:44.310450+0000","last_became_active":"2024-09-10T16:10:20.852065+0000","last_became_peered":"2024-09-10T16:10:20.852065+0000","last_unstale":"2024-09-10T16:11:44.310450+0000","last_undegraded":"2024-09-10T16:11:44.310450+0000","last_fullsized":"2024-09-10T16:11:44.310450+0000","mapping_epoch":56,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:34.069549+0000","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18461,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4307,"num_bytes_recovered":8675,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":119,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310825+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:11:44.310825+0000","last_peered":"2024-09-10T16:11:44.310825+0000","last_clean":"2024-09-10T16:11:44.310825+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:11:44.310825+0000","last_undegraded":"2024-09-10T16:11:44.310825+0000","last_fullsized":"2024-09-10T16:11:44.310825+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'21170","reported_seq":30059,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310126+0000","last_change":"2024-09-10T16:10:50.728349+0000","last_active":"2024-09-10T16:11:44.310126+0000","last_peered":"2024-09-10T16:11:44.310126+0000","last_clean":"2024-09-10T16:11:44.310126+0000","last_became_active":"2024-09-10T16:08:10.860438+0000","last_became_peered":"2024-09-10T16:08:10.860438+0000","last_unstale":"2024-09-10T16:11:44.310126+0000","last_undegraded":"2024-09-10T16:11:44.310126+0000","last_fullsized":"2024-09-10T16:11:44.310126+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:38:13.781380+0000","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18724,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20838","reported_seq":22176,"reported_epoch":63,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:11:46.242029+0000","last_change":"2024-09-10T16:11:42.563598+0000","last_active":"2024-09-10T16:11:46.242029+0000","last_peered":"2024-09-10T16:11:46.242029+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:11:46.242029+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3724,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17469,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":581,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3724}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":109,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310190+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:11:44.310190+0000","last_peered":"2024-09-10T16:11:44.310190+0000","last_clean":"2024-09-10T16:11:44.310190+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:11:44.310190+0000","last_undegraded":"2024-09-10T16:11:44.310190+0000","last_fullsized":"2024-09-10T16:11:44.310190+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20598","reported_seq":29204,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310762+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:11:44.310762+0000","last_peered":"2024-09-10T16:11:44.310762+0000","last_clean":"2024-09-10T16:11:44.310762+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:11:44.310762+0000","last_undegraded":"2024-09-10T16:11:44.310762+0000","last_fullsized":"2024-09-10T16:11:44.310762+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16988,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":134,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310946+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:11:44.310946+0000","last_peered":"2024-09-10T16:11:44.310946+0000","last_clean":"2024-09-10T16:11:44.310946+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:11:44.310946+0000","last_undegraded":"2024-09-10T16:11:44.310946+0000","last_fullsized":"2024-09-10T16:11:44.310946+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'21048","reported_seq":21214,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310569+0000","last_change":"2024-09-10T16:11:17.315372+0000","last_active":"2024-09-10T16:11:44.310569+0000","last_peered":"2024-09-10T16:11:44.310569+0000","last_clean":"2024-09-10T16:11:44.310569+0000","last_became_active":"2024-09-10T16:11:17.234487+0000","last_became_peered":"2024-09-10T16:11:17.234487+0000","last_unstale":"2024-09-10T16:11:44.310569+0000","last_undegraded":"2024-09-10T16:11:44.310569+0000","last_fullsized":"2024-09-10T16:11:44.310569+0000","mapping_epoch":60,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:19:00.114196+0000","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17303,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4373,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":109,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310658+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:11:44.310658+0000","last_peered":"2024-09-10T16:11:44.310658+0000","last_clean":"2024-09-10T16:11:44.310658+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:11:44.310658+0000","last_undegraded":"2024-09-10T16:11:44.310658+0000","last_fullsized":"2024-09-10T16:11:44.310658+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":114,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310839+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:11:44.310839+0000","last_peered":"2024-09-10T16:11:44.310839+0000","last_clean":"2024-09-10T16:11:44.310839+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:11:44.310839+0000","last_undegraded":"2024-09-10T16:11:44.310839+0000","last_fullsized":"2024-09-10T16:11:44.310839+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'21198","reported_seq":21365,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.396057+0000","last_change":"2024-09-10T16:11:44.391021+0000","last_active":"2024-09-10T16:11:44.396057+0000","last_peered":"2024-09-10T16:11:44.396057+0000","last_clean":"2024-09-10T16:11:44.396057+0000","last_became_active":"2024-09-10T16:11:44.312647+0000","last_became_peered":"2024-09-10T16:11:44.312647+0000","last_unstale":"2024-09-10T16:11:44.396057+0000","last_undegraded":"2024-09-10T16:11:44.396057+0000","last_fullsized":"2024-09-10T16:11:44.396057+0000","mapping_epoch":62,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:33:28.041611+0000","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17361,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4325,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7961,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292096,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43429,"num_bytes_recovered":33097612,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":179425280,"data_stored":179331342,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161022,"ondisk_log_size":161022,"up":32,"acting":30,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986341,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":472732,"kb_used_data":107848,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93899108,"statfs":{"total":96636764160,"available":96152686592,"internally_reserved":0,"allocated":110436352,"data_stored":110055945,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659286,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":509808,"kb_used_data":544,"kb_used_omap":4,"kb_used_meta":509243,"kb_avail":93862032,"statfs":{"total":96636764160,"available":96114720768,"internally_reserved":0,"allocated":557056,"data_stored":179798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":521465518},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182569,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":547412,"kb_used_data":68732,"kb_used_omap":4,"kb_used_meta":478651,"kb_avail":93824428,"statfs":{"total":96636764160,"available":96076214272,"internally_reserved":0,"allocated":70381568,"data_stored":69993471,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":490139306},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":69853184,"data_stored":69817054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":109543424,"data_stored":109510907,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:11:51.683 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:11:51.989 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:11:51.990 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:11:52.001 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":494,"stamp":"2024-09-10T16:11:51.797539+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7260,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292098,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44132,"num_bytes_recovered":33097631,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161025,"ondisk_log_size":161025,"up":48,"acting":46,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1513868,"kb_used_data":173524,"kb_used_omap":12,"kb_used_meta":1340275,"kb_avail":281601652,"statfs":{"total":289910292480,"available":288360091648,"internally_reserved":0,"allocated":177688576,"data_stored":176542814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1372441602},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1635,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1635,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.489992"},"pg_stats":[{"pgid":"2.f","version":"49'20672","reported_seq":29313,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.314657+0000","last_change":"2024-09-10T16:09:57.088531+0000","last_active":"2024-09-10T16:11:44.314657+0000","last_peered":"2024-09-10T16:11:44.314657+0000","last_clean":"2024-09-10T16:11:44.314657+0000","last_became_active":"2024-09-10T16:09:57.005755+0000","last_became_peered":"2024-09-10T16:09:57.005755+0000","last_unstale":"2024-09-10T16:11:44.314657+0000","last_undegraded":"2024-09-10T16:11:44.314657+0000","last_fullsized":"2024-09-10T16:11:44.314657+0000","mapping_epoch":54,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:09:19.329697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20672,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4259,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20844","reported_seq":20997,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.311152+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:11:44.311152+0000","last_peered":"2024-09-10T16:11:44.311152+0000","last_clean":"2024-09-10T16:11:44.311152+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:11:44.311152+0000","last_undegraded":"2024-09-10T16:11:44.311152+0000","last_fullsized":"2024-09-10T16:11:44.311152+0000","mapping_epoch":35,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18949,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20798","reported_seq":29516,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310310+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:11:44.310310+0000","last_peered":"2024-09-10T16:11:44.310310+0000","last_clean":"2024-09-10T16:11:44.310310+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:11:44.310310+0000","last_undegraded":"2024-09-10T16:11:44.310310+0000","last_fullsized":"2024-09-10T16:11:44.310310+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18802,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20660","reported_seq":20781,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310770+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:11:44.310770+0000","last_peered":"2024-09-10T16:11:44.310770+0000","last_clean":"2024-09-10T16:11:44.310770+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:11:44.310770+0000","last_undegraded":"2024-09-10T16:11:44.310770+0000","last_fullsized":"2024-09-10T16:11:44.310770+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20660,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20891","reported_seq":21061,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.312021+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:11:44.312021+0000","last_peered":"2024-09-10T16:11:44.312021+0000","last_clean":"2024-09-10T16:11:44.312021+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:11:44.312021+0000","last_undegraded":"2024-09-10T16:11:44.312021+0000","last_fullsized":"2024-09-10T16:11:44.312021+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17602,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'21061","reported_seq":21230,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310143+0000","last_change":"2024-09-10T16:10:52.241702+0000","last_active":"2024-09-10T16:11:44.310143+0000","last_peered":"2024-09-10T16:11:44.310143+0000","last_clean":"2024-09-10T16:11:44.310143+0000","last_became_active":"2024-09-10T16:10:52.171236+0000","last_became_peered":"2024-09-10T16:10:52.171236+0000","last_unstale":"2024-09-10T16:11:44.310143+0000","last_undegraded":"2024-09-10T16:11:44.310143+0000","last_fullsized":"2024-09-10T16:11:44.310143+0000","mapping_epoch":58,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:04:46.778280+0000","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17673,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4372,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'21082","reported_seq":21260,"reported_epoch":63,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:11:44.312026+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:11:44.312026+0000","last_peered":"2024-09-10T16:11:44.312026+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:11:44.312026+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17581,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4235},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'21314","reported_seq":21484,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.311841+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:11:44.311841+0000","last_peered":"2024-09-10T16:11:44.311841+0000","last_clean":"2024-09-10T16:11:44.311841+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:11:44.311841+0000","last_undegraded":"2024-09-10T16:11:44.311841+0000","last_fullsized":"2024-09-10T16:11:44.311841+0000","mapping_epoch":34,"log_start":"45'11300","ondisk_log_start":"45'11300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18161,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.309727+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:11:44.309727+0000","last_peered":"2024-09-10T16:11:44.309727+0000","last_clean":"2024-09-10T16:11:44.309727+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:11:44.309727+0000","last_undegraded":"2024-09-10T16:11:44.309727+0000","last_fullsized":"2024-09-10T16:11:44.309727+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20457","reported_seq":20641,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310911+0000","last_change":"2024-09-10T16:09:33.012840+0000","last_active":"2024-09-10T16:11:44.310911+0000","last_peered":"2024-09-10T16:11:44.310911+0000","last_clean":"2024-09-10T16:11:44.310911+0000","last_became_active":"2024-09-10T16:09:32.934056+0000","last_became_peered":"2024-09-10T16:09:32.934056+0000","last_unstale":"2024-09-10T16:11:44.310911+0000","last_undegraded":"2024-09-10T16:11:44.310911+0000","last_fullsized":"2024-09-10T16:11:44.310911+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:08:30.481235+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18276,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":4579,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":134,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310369+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:11:44.310369+0000","last_peered":"2024-09-10T16:11:44.310369+0000","last_clean":"2024-09-10T16:11:44.310369+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:11:44.310369+0000","last_undegraded":"2024-09-10T16:11:44.310369+0000","last_fullsized":"2024-09-10T16:11:44.310369+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20910","reported_seq":29603,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.312044+0000","last_change":"2024-09-10T16:09:08.909580+0000","last_active":"2024-09-10T16:11:44.312044+0000","last_peered":"2024-09-10T16:11:44.312044+0000","last_clean":"2024-09-10T16:11:44.312044+0000","last_became_active":"2024-09-10T16:09:08.854941+0000","last_became_peered":"2024-09-10T16:09:08.854941+0000","last_unstale":"2024-09-10T16:11:44.312044+0000","last_undegraded":"2024-09-10T16:11:44.312044+0000","last_fullsized":"2024-09-10T16:11:44.312044+0000","mapping_epoch":50,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:13:14.846691+0000","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17414,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4257,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":144,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.311936+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:11:44.311936+0000","last_peered":"2024-09-10T16:11:44.311936+0000","last_clean":"2024-09-10T16:11:44.311936+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:11:44.311936+0000","last_undegraded":"2024-09-10T16:11:44.311936+0000","last_fullsized":"2024-09-10T16:11:44.311936+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20781","reported_seq":29574,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310450+0000","last_change":"2024-09-10T16:10:20.907889+0000","last_active":"2024-09-10T16:11:44.310450+0000","last_peered":"2024-09-10T16:11:44.310450+0000","last_clean":"2024-09-10T16:11:44.310450+0000","last_became_active":"2024-09-10T16:10:20.852065+0000","last_became_peered":"2024-09-10T16:10:20.852065+0000","last_unstale":"2024-09-10T16:11:44.310450+0000","last_undegraded":"2024-09-10T16:11:44.310450+0000","last_fullsized":"2024-09-10T16:11:44.310450+0000","mapping_epoch":56,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:34.069549+0000","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18461,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4307,"num_bytes_recovered":8675,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":120,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310865+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:11:44.310865+0000","last_peered":"2024-09-10T16:11:44.310865+0000","last_clean":"2024-09-10T16:11:44.310865+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:11:44.310865+0000","last_undegraded":"2024-09-10T16:11:44.310865+0000","last_fullsized":"2024-09-10T16:11:44.310865+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'21170","reported_seq":30059,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310126+0000","last_change":"2024-09-10T16:10:50.728349+0000","last_active":"2024-09-10T16:11:44.310126+0000","last_peered":"2024-09-10T16:11:44.310126+0000","last_clean":"2024-09-10T16:11:44.310126+0000","last_became_active":"2024-09-10T16:08:10.860438+0000","last_became_peered":"2024-09-10T16:08:10.860438+0000","last_unstale":"2024-09-10T16:11:44.310126+0000","last_undegraded":"2024-09-10T16:11:44.310126+0000","last_fullsized":"2024-09-10T16:11:44.310126+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:38:13.781380+0000","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18724,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20838","reported_seq":23578,"reported_epoch":63,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:11:51.245010+0000","last_change":"2024-09-10T16:11:42.563598+0000","last_active":"2024-09-10T16:11:51.245010+0000","last_peered":"2024-09-10T16:11:51.245010+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:11:51.245010+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3023,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17469,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1282,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3023}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310321+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:11:44.310321+0000","last_peered":"2024-09-10T16:11:44.310321+0000","last_clean":"2024-09-10T16:11:44.310321+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:11:44.310321+0000","last_undegraded":"2024-09-10T16:11:44.310321+0000","last_fullsized":"2024-09-10T16:11:44.310321+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20598","reported_seq":29204,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310762+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:11:44.310762+0000","last_peered":"2024-09-10T16:11:44.310762+0000","last_clean":"2024-09-10T16:11:44.310762+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:11:44.310762+0000","last_undegraded":"2024-09-10T16:11:44.310762+0000","last_fullsized":"2024-09-10T16:11:44.310762+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16988,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":134,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310946+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:11:44.310946+0000","last_peered":"2024-09-10T16:11:44.310946+0000","last_clean":"2024-09-10T16:11:44.310946+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:11:44.310946+0000","last_undegraded":"2024-09-10T16:11:44.310946+0000","last_fullsized":"2024-09-10T16:11:44.310946+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'21048","reported_seq":21214,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310569+0000","last_change":"2024-09-10T16:11:17.315372+0000","last_active":"2024-09-10T16:11:44.310569+0000","last_peered":"2024-09-10T16:11:44.310569+0000","last_clean":"2024-09-10T16:11:44.310569+0000","last_became_active":"2024-09-10T16:11:17.234487+0000","last_became_peered":"2024-09-10T16:11:17.234487+0000","last_unstale":"2024-09-10T16:11:44.310569+0000","last_undegraded":"2024-09-10T16:11:44.310569+0000","last_fullsized":"2024-09-10T16:11:44.310569+0000","mapping_epoch":60,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:19:00.114196+0000","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17303,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4373,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":109,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310658+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:11:44.310658+0000","last_peered":"2024-09-10T16:11:44.310658+0000","last_clean":"2024-09-10T16:11:44.310658+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:11:44.310658+0000","last_undegraded":"2024-09-10T16:11:44.310658+0000","last_fullsized":"2024-09-10T16:11:44.310658+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":114,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310839+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:11:44.310839+0000","last_peered":"2024-09-10T16:11:44.310839+0000","last_clean":"2024-09-10T16:11:44.310839+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:11:44.310839+0000","last_undegraded":"2024-09-10T16:11:44.310839+0000","last_fullsized":"2024-09-10T16:11:44.310839+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'21198","reported_seq":21365,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.396057+0000","last_change":"2024-09-10T16:11:44.391021+0000","last_active":"2024-09-10T16:11:44.396057+0000","last_peered":"2024-09-10T16:11:44.396057+0000","last_clean":"2024-09-10T16:11:44.396057+0000","last_became_active":"2024-09-10T16:11:44.312647+0000","last_became_peered":"2024-09-10T16:11:44.312647+0000","last_unstale":"2024-09-10T16:11:44.396057+0000","last_undegraded":"2024-09-10T16:11:44.396057+0000","last_fullsized":"2024-09-10T16:11:44.396057+0000","mapping_epoch":62,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:33:28.041611+0000","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17361,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4325,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7260,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292096,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44130,"num_bytes_recovered":33097612,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":175738880,"data_stored":175644942,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161022,"ondisk_log_size":161022,"up":32,"acting":30,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986342,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":469128,"kb_used_data":104248,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93902712,"statfs":{"total":96636764160,"available":96156377088,"internally_reserved":0,"allocated":106749952,"data_stored":106369545,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659287,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":497328,"kb_used_data":544,"kb_used_omap":4,"kb_used_meta":496763,"kb_avail":93874512,"statfs":{"total":96636764160,"available":96127500288,"internally_reserved":0,"allocated":557056,"data_stored":179798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":508685998},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182570,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":547412,"kb_used_data":68732,"kb_used_omap":4,"kb_used_meta":478651,"kb_avail":93824428,"statfs":{"total":96636764160,"available":96076214272,"internally_reserved":0,"allocated":70381568,"data_stored":69993471,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":490139306},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":69853184,"data_stored":69817054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":105857024,"data_stored":105824507,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:11:52.003 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:11:52.360 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:11:52.361 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:11:52.374 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":494,"stamp":"2024-09-10T16:11:51.797539+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7260,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292098,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44132,"num_bytes_recovered":33097631,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161025,"ondisk_log_size":161025,"up":48,"acting":46,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1513868,"kb_used_data":173524,"kb_used_omap":12,"kb_used_meta":1340275,"kb_avail":281601652,"statfs":{"total":289910292480,"available":288360091648,"internally_reserved":0,"allocated":177688576,"data_stored":176542814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1372441602},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1635,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1635,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.489992"},"pg_stats":[{"pgid":"2.f","version":"49'20672","reported_seq":29313,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.314657+0000","last_change":"2024-09-10T16:09:57.088531+0000","last_active":"2024-09-10T16:11:44.314657+0000","last_peered":"2024-09-10T16:11:44.314657+0000","last_clean":"2024-09-10T16:11:44.314657+0000","last_became_active":"2024-09-10T16:09:57.005755+0000","last_became_peered":"2024-09-10T16:09:57.005755+0000","last_unstale":"2024-09-10T16:11:44.314657+0000","last_undegraded":"2024-09-10T16:11:44.314657+0000","last_fullsized":"2024-09-10T16:11:44.314657+0000","mapping_epoch":54,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:09:19.329697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20672,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4259,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20844","reported_seq":20997,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.311152+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:11:44.311152+0000","last_peered":"2024-09-10T16:11:44.311152+0000","last_clean":"2024-09-10T16:11:44.311152+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:11:44.311152+0000","last_undegraded":"2024-09-10T16:11:44.311152+0000","last_fullsized":"2024-09-10T16:11:44.311152+0000","mapping_epoch":35,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18949,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20798","reported_seq":29516,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310310+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:11:44.310310+0000","last_peered":"2024-09-10T16:11:44.310310+0000","last_clean":"2024-09-10T16:11:44.310310+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:11:44.310310+0000","last_undegraded":"2024-09-10T16:11:44.310310+0000","last_fullsized":"2024-09-10T16:11:44.310310+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18802,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20660","reported_seq":20781,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310770+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:11:44.310770+0000","last_peered":"2024-09-10T16:11:44.310770+0000","last_clean":"2024-09-10T16:11:44.310770+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:11:44.310770+0000","last_undegraded":"2024-09-10T16:11:44.310770+0000","last_fullsized":"2024-09-10T16:11:44.310770+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20660,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20891","reported_seq":21061,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.312021+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:11:44.312021+0000","last_peered":"2024-09-10T16:11:44.312021+0000","last_clean":"2024-09-10T16:11:44.312021+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:11:44.312021+0000","last_undegraded":"2024-09-10T16:11:44.312021+0000","last_fullsized":"2024-09-10T16:11:44.312021+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17602,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'21061","reported_seq":21230,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310143+0000","last_change":"2024-09-10T16:10:52.241702+0000","last_active":"2024-09-10T16:11:44.310143+0000","last_peered":"2024-09-10T16:11:44.310143+0000","last_clean":"2024-09-10T16:11:44.310143+0000","last_became_active":"2024-09-10T16:10:52.171236+0000","last_became_peered":"2024-09-10T16:10:52.171236+0000","last_unstale":"2024-09-10T16:11:44.310143+0000","last_undegraded":"2024-09-10T16:11:44.310143+0000","last_fullsized":"2024-09-10T16:11:44.310143+0000","mapping_epoch":58,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:04:46.778280+0000","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17673,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4372,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'21082","reported_seq":21260,"reported_epoch":63,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:11:44.312026+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:11:44.312026+0000","last_peered":"2024-09-10T16:11:44.312026+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:11:44.312026+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17581,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4235},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'21314","reported_seq":21484,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.311841+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:11:44.311841+0000","last_peered":"2024-09-10T16:11:44.311841+0000","last_clean":"2024-09-10T16:11:44.311841+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:11:44.311841+0000","last_undegraded":"2024-09-10T16:11:44.311841+0000","last_fullsized":"2024-09-10T16:11:44.311841+0000","mapping_epoch":34,"log_start":"45'11300","ondisk_log_start":"45'11300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18161,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.309727+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:11:44.309727+0000","last_peered":"2024-09-10T16:11:44.309727+0000","last_clean":"2024-09-10T16:11:44.309727+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:11:44.309727+0000","last_undegraded":"2024-09-10T16:11:44.309727+0000","last_fullsized":"2024-09-10T16:11:44.309727+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20457","reported_seq":20641,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310911+0000","last_change":"2024-09-10T16:09:33.012840+0000","last_active":"2024-09-10T16:11:44.310911+0000","last_peered":"2024-09-10T16:11:44.310911+0000","last_clean":"2024-09-10T16:11:44.310911+0000","last_became_active":"2024-09-10T16:09:32.934056+0000","last_became_peered":"2024-09-10T16:09:32.934056+0000","last_unstale":"2024-09-10T16:11:44.310911+0000","last_undegraded":"2024-09-10T16:11:44.310911+0000","last_fullsized":"2024-09-10T16:11:44.310911+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:08:30.481235+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18276,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":4579,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":134,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310369+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:11:44.310369+0000","last_peered":"2024-09-10T16:11:44.310369+0000","last_clean":"2024-09-10T16:11:44.310369+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:11:44.310369+0000","last_undegraded":"2024-09-10T16:11:44.310369+0000","last_fullsized":"2024-09-10T16:11:44.310369+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20910","reported_seq":29603,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.312044+0000","last_change":"2024-09-10T16:09:08.909580+0000","last_active":"2024-09-10T16:11:44.312044+0000","last_peered":"2024-09-10T16:11:44.312044+0000","last_clean":"2024-09-10T16:11:44.312044+0000","last_became_active":"2024-09-10T16:09:08.854941+0000","last_became_peered":"2024-09-10T16:09:08.854941+0000","last_unstale":"2024-09-10T16:11:44.312044+0000","last_undegraded":"2024-09-10T16:11:44.312044+0000","last_fullsized":"2024-09-10T16:11:44.312044+0000","mapping_epoch":50,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:13:14.846691+0000","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17414,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4257,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":144,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.311936+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:11:44.311936+0000","last_peered":"2024-09-10T16:11:44.311936+0000","last_clean":"2024-09-10T16:11:44.311936+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:11:44.311936+0000","last_undegraded":"2024-09-10T16:11:44.311936+0000","last_fullsized":"2024-09-10T16:11:44.311936+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20781","reported_seq":29574,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310450+0000","last_change":"2024-09-10T16:10:20.907889+0000","last_active":"2024-09-10T16:11:44.310450+0000","last_peered":"2024-09-10T16:11:44.310450+0000","last_clean":"2024-09-10T16:11:44.310450+0000","last_became_active":"2024-09-10T16:10:20.852065+0000","last_became_peered":"2024-09-10T16:10:20.852065+0000","last_unstale":"2024-09-10T16:11:44.310450+0000","last_undegraded":"2024-09-10T16:11:44.310450+0000","last_fullsized":"2024-09-10T16:11:44.310450+0000","mapping_epoch":56,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:34.069549+0000","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18461,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4307,"num_bytes_recovered":8675,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":120,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310865+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:11:44.310865+0000","last_peered":"2024-09-10T16:11:44.310865+0000","last_clean":"2024-09-10T16:11:44.310865+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:11:44.310865+0000","last_undegraded":"2024-09-10T16:11:44.310865+0000","last_fullsized":"2024-09-10T16:11:44.310865+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'21170","reported_seq":30059,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310126+0000","last_change":"2024-09-10T16:10:50.728349+0000","last_active":"2024-09-10T16:11:44.310126+0000","last_peered":"2024-09-10T16:11:44.310126+0000","last_clean":"2024-09-10T16:11:44.310126+0000","last_became_active":"2024-09-10T16:08:10.860438+0000","last_became_peered":"2024-09-10T16:08:10.860438+0000","last_unstale":"2024-09-10T16:11:44.310126+0000","last_undegraded":"2024-09-10T16:11:44.310126+0000","last_fullsized":"2024-09-10T16:11:44.310126+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:38:13.781380+0000","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18724,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20838","reported_seq":23578,"reported_epoch":63,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:11:51.245010+0000","last_change":"2024-09-10T16:11:42.563598+0000","last_active":"2024-09-10T16:11:51.245010+0000","last_peered":"2024-09-10T16:11:51.245010+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:11:51.245010+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3023,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17469,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1282,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3023}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310321+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:11:44.310321+0000","last_peered":"2024-09-10T16:11:44.310321+0000","last_clean":"2024-09-10T16:11:44.310321+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:11:44.310321+0000","last_undegraded":"2024-09-10T16:11:44.310321+0000","last_fullsized":"2024-09-10T16:11:44.310321+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20598","reported_seq":29204,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310762+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:11:44.310762+0000","last_peered":"2024-09-10T16:11:44.310762+0000","last_clean":"2024-09-10T16:11:44.310762+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:11:44.310762+0000","last_undegraded":"2024-09-10T16:11:44.310762+0000","last_fullsized":"2024-09-10T16:11:44.310762+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16988,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":134,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310946+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:11:44.310946+0000","last_peered":"2024-09-10T16:11:44.310946+0000","last_clean":"2024-09-10T16:11:44.310946+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:11:44.310946+0000","last_undegraded":"2024-09-10T16:11:44.310946+0000","last_fullsized":"2024-09-10T16:11:44.310946+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'21048","reported_seq":21214,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310569+0000","last_change":"2024-09-10T16:11:17.315372+0000","last_active":"2024-09-10T16:11:44.310569+0000","last_peered":"2024-09-10T16:11:44.310569+0000","last_clean":"2024-09-10T16:11:44.310569+0000","last_became_active":"2024-09-10T16:11:17.234487+0000","last_became_peered":"2024-09-10T16:11:17.234487+0000","last_unstale":"2024-09-10T16:11:44.310569+0000","last_undegraded":"2024-09-10T16:11:44.310569+0000","last_fullsized":"2024-09-10T16:11:44.310569+0000","mapping_epoch":60,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:19:00.114196+0000","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17303,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4373,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":109,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310658+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:11:44.310658+0000","last_peered":"2024-09-10T16:11:44.310658+0000","last_clean":"2024-09-10T16:11:44.310658+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:11:44.310658+0000","last_undegraded":"2024-09-10T16:11:44.310658+0000","last_fullsized":"2024-09-10T16:11:44.310658+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":114,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310839+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:11:44.310839+0000","last_peered":"2024-09-10T16:11:44.310839+0000","last_clean":"2024-09-10T16:11:44.310839+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:11:44.310839+0000","last_undegraded":"2024-09-10T16:11:44.310839+0000","last_fullsized":"2024-09-10T16:11:44.310839+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'21198","reported_seq":21365,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.396057+0000","last_change":"2024-09-10T16:11:44.391021+0000","last_active":"2024-09-10T16:11:44.396057+0000","last_peered":"2024-09-10T16:11:44.396057+0000","last_clean":"2024-09-10T16:11:44.396057+0000","last_became_active":"2024-09-10T16:11:44.312647+0000","last_became_peered":"2024-09-10T16:11:44.312647+0000","last_unstale":"2024-09-10T16:11:44.396057+0000","last_undegraded":"2024-09-10T16:11:44.396057+0000","last_fullsized":"2024-09-10T16:11:44.396057+0000","mapping_epoch":62,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:33:28.041611+0000","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17361,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4325,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7260,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292096,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44130,"num_bytes_recovered":33097612,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":175738880,"data_stored":175644942,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161022,"ondisk_log_size":161022,"up":32,"acting":30,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986342,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":469128,"kb_used_data":104248,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93902712,"statfs":{"total":96636764160,"available":96156377088,"internally_reserved":0,"allocated":106749952,"data_stored":106369545,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659287,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":497328,"kb_used_data":544,"kb_used_omap":4,"kb_used_meta":496763,"kb_avail":93874512,"statfs":{"total":96636764160,"available":96127500288,"internally_reserved":0,"allocated":557056,"data_stored":179798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":508685998},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182570,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":547412,"kb_used_data":68732,"kb_used_omap":4,"kb_used_meta":478651,"kb_avail":93824428,"statfs":{"total":96636764160,"available":96076214272,"internally_reserved":0,"allocated":70381568,"data_stored":69993471,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":490139306},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":69853184,"data_stored":69817054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":105857024,"data_stored":105824507,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:11:52.375 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-10T16:11:52.376 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "49'21082", 'reported_seq': 21260, 'reported_epoch': 63, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:11:44.312026+0000', 'last_change': '2024-09-10T15:59:16.331899+0000', 'last_active': '2024-09-10T16:11:44.312026+0000', 'last_peered': '2024-09-10T16:11:44.312026+0000', 'last_clean': '2024-09-10T15:59:14.139436+0000', 'last_became_active': '2024-09-10T15:59:16.318715+0000', 'last_became_peered': '2024-09-10T15:59:16.318715+0000', 'last_unstale': '2024-09-10T16:11:44.312026+0000', 'last_undegraded': '2024-09-10T15:59:15.877524+0000', 'last_fullsized': '2024-09-10T15:59:15.769042+0000', 'mapping_epoch': 42, 'log_start': "45'11000", 'ondisk_log_start': "45'11000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3537", 'last_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'objects_scrubbed': 3497, 'log_size': 10082, 'log_dups_size': 0, 'ondisk_log_size': 10082, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1224, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4237, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 17581, 'num_write_kb': 28172, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4235}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:11:52.376 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-10T16:11:52.376 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "49'20838", 'reported_seq': 23578, 'reported_epoch': 63, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:11:51.245010+0000', 'last_change': '2024-09-10T16:11:42.563598+0000', 'last_active': '2024-09-10T16:11:51.245010+0000', 'last_peered': '2024-09-10T16:11:51.245010+0000', 'last_clean': '2024-09-10T15:59:14.117001+0000', 'last_became_active': '2024-09-10T15:59:16.302245+0000', 'last_became_peered': '2024-09-10T15:59:16.302245+0000', 'last_unstale': '2024-09-10T16:11:51.245010+0000', 'last_undegraded': '2024-09-10T15:59:15.845012+0000', 'last_fullsized': '2024-09-10T15:59:15.770337+0000', 'mapping_epoch': 42, 'log_start': "45'10800", 'ondisk_log_start': "45'10800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3404", 'last_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'objects_scrubbed': 3367, 'log_size': 10038, 'log_dups_size': 0, 'ondisk_log_size': 10038, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 983, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3023, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17469, 'num_write_kb': 28204, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1282, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3023}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:11:52.376 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T16:11:52.772 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:11:52.772 INFO:teuthology.orchestra.run.smithi115.stdout:{"fsid":"7effee29-df05-4351-9741-ee203564ed81","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 7260/24 objects degraded (30250.000%), 2 pgs degraded, 2 pgs undersized","count":4},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":876,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":63,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983954,"num_in_osds":3,"osd_in_since":1725983947,"num_remapped_pgs":2},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":22},{"state_name":"active+recovering+undersized+degraded+remapped","count":1},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":12,"data_bytes":4849,"bytes_used":1550200832,"bytes_avail":288360091648,"bytes_total":289910292480,"degraded_objects":7260,"degraded_total":24,"degraded_ratio":302.5,"recovering_objects_per_sec":192,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1635,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-10T15:57:14:967120+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T16:00:13.560776+0000","services":{}},"progress_events":{}} 2024-09-10T16:11:52.785 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-10T16:11:52.785 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:11:53.140 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:11:53.141 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:11:53.153 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":494,"stamp":"2024-09-10T16:11:51.797539+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7260,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292098,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44132,"num_bytes_recovered":33097631,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161025,"ondisk_log_size":161025,"up":48,"acting":46,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1513868,"kb_used_data":173524,"kb_used_omap":12,"kb_used_meta":1340275,"kb_avail":281601652,"statfs":{"total":289910292480,"available":288360091648,"internally_reserved":0,"allocated":177688576,"data_stored":176542814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1372441602},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1635,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1635,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.489992"},"pg_stats":[{"pgid":"2.f","version":"49'20672","reported_seq":29313,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.314657+0000","last_change":"2024-09-10T16:09:57.088531+0000","last_active":"2024-09-10T16:11:44.314657+0000","last_peered":"2024-09-10T16:11:44.314657+0000","last_clean":"2024-09-10T16:11:44.314657+0000","last_became_active":"2024-09-10T16:09:57.005755+0000","last_became_peered":"2024-09-10T16:09:57.005755+0000","last_unstale":"2024-09-10T16:11:44.314657+0000","last_undegraded":"2024-09-10T16:11:44.314657+0000","last_fullsized":"2024-09-10T16:11:44.314657+0000","mapping_epoch":54,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:09:19.329697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20672,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4259,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20844","reported_seq":20997,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.311152+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:11:44.311152+0000","last_peered":"2024-09-10T16:11:44.311152+0000","last_clean":"2024-09-10T16:11:44.311152+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:11:44.311152+0000","last_undegraded":"2024-09-10T16:11:44.311152+0000","last_fullsized":"2024-09-10T16:11:44.311152+0000","mapping_epoch":35,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18949,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20798","reported_seq":29516,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310310+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:11:44.310310+0000","last_peered":"2024-09-10T16:11:44.310310+0000","last_clean":"2024-09-10T16:11:44.310310+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:11:44.310310+0000","last_undegraded":"2024-09-10T16:11:44.310310+0000","last_fullsized":"2024-09-10T16:11:44.310310+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18802,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20660","reported_seq":20781,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310770+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:11:44.310770+0000","last_peered":"2024-09-10T16:11:44.310770+0000","last_clean":"2024-09-10T16:11:44.310770+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:11:44.310770+0000","last_undegraded":"2024-09-10T16:11:44.310770+0000","last_fullsized":"2024-09-10T16:11:44.310770+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20660,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20891","reported_seq":21061,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.312021+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:11:44.312021+0000","last_peered":"2024-09-10T16:11:44.312021+0000","last_clean":"2024-09-10T16:11:44.312021+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:11:44.312021+0000","last_undegraded":"2024-09-10T16:11:44.312021+0000","last_fullsized":"2024-09-10T16:11:44.312021+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17602,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'21061","reported_seq":21230,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310143+0000","last_change":"2024-09-10T16:10:52.241702+0000","last_active":"2024-09-10T16:11:44.310143+0000","last_peered":"2024-09-10T16:11:44.310143+0000","last_clean":"2024-09-10T16:11:44.310143+0000","last_became_active":"2024-09-10T16:10:52.171236+0000","last_became_peered":"2024-09-10T16:10:52.171236+0000","last_unstale":"2024-09-10T16:11:44.310143+0000","last_undegraded":"2024-09-10T16:11:44.310143+0000","last_fullsized":"2024-09-10T16:11:44.310143+0000","mapping_epoch":58,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:04:46.778280+0000","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17673,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4372,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'21082","reported_seq":21260,"reported_epoch":63,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:11:44.312026+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:11:44.312026+0000","last_peered":"2024-09-10T16:11:44.312026+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:11:44.312026+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17581,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4235},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'21314","reported_seq":21484,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.311841+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:11:44.311841+0000","last_peered":"2024-09-10T16:11:44.311841+0000","last_clean":"2024-09-10T16:11:44.311841+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:11:44.311841+0000","last_undegraded":"2024-09-10T16:11:44.311841+0000","last_fullsized":"2024-09-10T16:11:44.311841+0000","mapping_epoch":34,"log_start":"45'11300","ondisk_log_start":"45'11300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18161,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.309727+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:11:44.309727+0000","last_peered":"2024-09-10T16:11:44.309727+0000","last_clean":"2024-09-10T16:11:44.309727+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:11:44.309727+0000","last_undegraded":"2024-09-10T16:11:44.309727+0000","last_fullsized":"2024-09-10T16:11:44.309727+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20457","reported_seq":20641,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310911+0000","last_change":"2024-09-10T16:09:33.012840+0000","last_active":"2024-09-10T16:11:44.310911+0000","last_peered":"2024-09-10T16:11:44.310911+0000","last_clean":"2024-09-10T16:11:44.310911+0000","last_became_active":"2024-09-10T16:09:32.934056+0000","last_became_peered":"2024-09-10T16:09:32.934056+0000","last_unstale":"2024-09-10T16:11:44.310911+0000","last_undegraded":"2024-09-10T16:11:44.310911+0000","last_fullsized":"2024-09-10T16:11:44.310911+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:08:30.481235+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18276,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":4579,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":134,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310369+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:11:44.310369+0000","last_peered":"2024-09-10T16:11:44.310369+0000","last_clean":"2024-09-10T16:11:44.310369+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:11:44.310369+0000","last_undegraded":"2024-09-10T16:11:44.310369+0000","last_fullsized":"2024-09-10T16:11:44.310369+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20910","reported_seq":29603,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.312044+0000","last_change":"2024-09-10T16:09:08.909580+0000","last_active":"2024-09-10T16:11:44.312044+0000","last_peered":"2024-09-10T16:11:44.312044+0000","last_clean":"2024-09-10T16:11:44.312044+0000","last_became_active":"2024-09-10T16:09:08.854941+0000","last_became_peered":"2024-09-10T16:09:08.854941+0000","last_unstale":"2024-09-10T16:11:44.312044+0000","last_undegraded":"2024-09-10T16:11:44.312044+0000","last_fullsized":"2024-09-10T16:11:44.312044+0000","mapping_epoch":50,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:13:14.846691+0000","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17414,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4257,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":144,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.311936+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:11:44.311936+0000","last_peered":"2024-09-10T16:11:44.311936+0000","last_clean":"2024-09-10T16:11:44.311936+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:11:44.311936+0000","last_undegraded":"2024-09-10T16:11:44.311936+0000","last_fullsized":"2024-09-10T16:11:44.311936+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20781","reported_seq":29574,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310450+0000","last_change":"2024-09-10T16:10:20.907889+0000","last_active":"2024-09-10T16:11:44.310450+0000","last_peered":"2024-09-10T16:11:44.310450+0000","last_clean":"2024-09-10T16:11:44.310450+0000","last_became_active":"2024-09-10T16:10:20.852065+0000","last_became_peered":"2024-09-10T16:10:20.852065+0000","last_unstale":"2024-09-10T16:11:44.310450+0000","last_undegraded":"2024-09-10T16:11:44.310450+0000","last_fullsized":"2024-09-10T16:11:44.310450+0000","mapping_epoch":56,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:34.069549+0000","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18461,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4307,"num_bytes_recovered":8675,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":120,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310865+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:11:44.310865+0000","last_peered":"2024-09-10T16:11:44.310865+0000","last_clean":"2024-09-10T16:11:44.310865+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:11:44.310865+0000","last_undegraded":"2024-09-10T16:11:44.310865+0000","last_fullsized":"2024-09-10T16:11:44.310865+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'21170","reported_seq":30059,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310126+0000","last_change":"2024-09-10T16:10:50.728349+0000","last_active":"2024-09-10T16:11:44.310126+0000","last_peered":"2024-09-10T16:11:44.310126+0000","last_clean":"2024-09-10T16:11:44.310126+0000","last_became_active":"2024-09-10T16:08:10.860438+0000","last_became_peered":"2024-09-10T16:08:10.860438+0000","last_unstale":"2024-09-10T16:11:44.310126+0000","last_undegraded":"2024-09-10T16:11:44.310126+0000","last_fullsized":"2024-09-10T16:11:44.310126+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:38:13.781380+0000","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18724,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20838","reported_seq":23578,"reported_epoch":63,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:11:51.245010+0000","last_change":"2024-09-10T16:11:42.563598+0000","last_active":"2024-09-10T16:11:51.245010+0000","last_peered":"2024-09-10T16:11:51.245010+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:11:51.245010+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3023,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17469,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1282,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3023}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310321+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:11:44.310321+0000","last_peered":"2024-09-10T16:11:44.310321+0000","last_clean":"2024-09-10T16:11:44.310321+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:11:44.310321+0000","last_undegraded":"2024-09-10T16:11:44.310321+0000","last_fullsized":"2024-09-10T16:11:44.310321+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20598","reported_seq":29204,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310762+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:11:44.310762+0000","last_peered":"2024-09-10T16:11:44.310762+0000","last_clean":"2024-09-10T16:11:44.310762+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:11:44.310762+0000","last_undegraded":"2024-09-10T16:11:44.310762+0000","last_fullsized":"2024-09-10T16:11:44.310762+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16988,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":134,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310946+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:11:44.310946+0000","last_peered":"2024-09-10T16:11:44.310946+0000","last_clean":"2024-09-10T16:11:44.310946+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:11:44.310946+0000","last_undegraded":"2024-09-10T16:11:44.310946+0000","last_fullsized":"2024-09-10T16:11:44.310946+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'21048","reported_seq":21214,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310569+0000","last_change":"2024-09-10T16:11:17.315372+0000","last_active":"2024-09-10T16:11:44.310569+0000","last_peered":"2024-09-10T16:11:44.310569+0000","last_clean":"2024-09-10T16:11:44.310569+0000","last_became_active":"2024-09-10T16:11:17.234487+0000","last_became_peered":"2024-09-10T16:11:17.234487+0000","last_unstale":"2024-09-10T16:11:44.310569+0000","last_undegraded":"2024-09-10T16:11:44.310569+0000","last_fullsized":"2024-09-10T16:11:44.310569+0000","mapping_epoch":60,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:19:00.114196+0000","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17303,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4373,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":109,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310658+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:11:44.310658+0000","last_peered":"2024-09-10T16:11:44.310658+0000","last_clean":"2024-09-10T16:11:44.310658+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:11:44.310658+0000","last_undegraded":"2024-09-10T16:11:44.310658+0000","last_fullsized":"2024-09-10T16:11:44.310658+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":114,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310839+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:11:44.310839+0000","last_peered":"2024-09-10T16:11:44.310839+0000","last_clean":"2024-09-10T16:11:44.310839+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:11:44.310839+0000","last_undegraded":"2024-09-10T16:11:44.310839+0000","last_fullsized":"2024-09-10T16:11:44.310839+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'21198","reported_seq":21365,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.396057+0000","last_change":"2024-09-10T16:11:44.391021+0000","last_active":"2024-09-10T16:11:44.396057+0000","last_peered":"2024-09-10T16:11:44.396057+0000","last_clean":"2024-09-10T16:11:44.396057+0000","last_became_active":"2024-09-10T16:11:44.312647+0000","last_became_peered":"2024-09-10T16:11:44.312647+0000","last_unstale":"2024-09-10T16:11:44.396057+0000","last_undegraded":"2024-09-10T16:11:44.396057+0000","last_fullsized":"2024-09-10T16:11:44.396057+0000","mapping_epoch":62,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:33:28.041611+0000","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17361,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4325,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7260,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292096,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44130,"num_bytes_recovered":33097612,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":175738880,"data_stored":175644942,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161022,"ondisk_log_size":161022,"up":32,"acting":30,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986342,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":469128,"kb_used_data":104248,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93902712,"statfs":{"total":96636764160,"available":96156377088,"internally_reserved":0,"allocated":106749952,"data_stored":106369545,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659287,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":497328,"kb_used_data":544,"kb_used_omap":4,"kb_used_meta":496763,"kb_avail":93874512,"statfs":{"total":96636764160,"available":96127500288,"internally_reserved":0,"allocated":557056,"data_stored":179798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":508685998},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182570,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":547412,"kb_used_data":68732,"kb_used_omap":4,"kb_used_meta":478651,"kb_avail":93824428,"statfs":{"total":96636764160,"available":96076214272,"internally_reserved":0,"allocated":70381568,"data_stored":69993471,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":490139306},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":69853184,"data_stored":69817054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":105857024,"data_stored":105824507,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:11:56.155 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:11:56.459 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:11:56.459 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:11:56.471 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":496,"stamp":"2024-09-10T16:11:55.798165+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7260,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292098,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44132,"num_bytes_recovered":33097631,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161025,"ondisk_log_size":161025,"up":48,"acting":46,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1518028,"kb_used_data":173524,"kb_used_omap":12,"kb_used_meta":1344435,"kb_avail":281597492,"statfs":{"total":289910292480,"available":288355831808,"internally_reserved":0,"allocated":177688576,"data_stored":176542814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1376701442},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1635,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1635,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"11.489958"},"pg_stats":[{"pgid":"2.f","version":"49'20672","reported_seq":29313,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.314657+0000","last_change":"2024-09-10T16:09:57.088531+0000","last_active":"2024-09-10T16:11:44.314657+0000","last_peered":"2024-09-10T16:11:44.314657+0000","last_clean":"2024-09-10T16:11:44.314657+0000","last_became_active":"2024-09-10T16:09:57.005755+0000","last_became_peered":"2024-09-10T16:09:57.005755+0000","last_unstale":"2024-09-10T16:11:44.314657+0000","last_undegraded":"2024-09-10T16:11:44.314657+0000","last_fullsized":"2024-09-10T16:11:44.314657+0000","mapping_epoch":54,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:09:19.329697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20672,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4259,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20844","reported_seq":20997,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.311152+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:11:44.311152+0000","last_peered":"2024-09-10T16:11:44.311152+0000","last_clean":"2024-09-10T16:11:44.311152+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:11:44.311152+0000","last_undegraded":"2024-09-10T16:11:44.311152+0000","last_fullsized":"2024-09-10T16:11:44.311152+0000","mapping_epoch":35,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18949,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20798","reported_seq":29516,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310310+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:11:44.310310+0000","last_peered":"2024-09-10T16:11:44.310310+0000","last_clean":"2024-09-10T16:11:44.310310+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:11:44.310310+0000","last_undegraded":"2024-09-10T16:11:44.310310+0000","last_fullsized":"2024-09-10T16:11:44.310310+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18802,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20660","reported_seq":20781,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310770+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:11:44.310770+0000","last_peered":"2024-09-10T16:11:44.310770+0000","last_clean":"2024-09-10T16:11:44.310770+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:11:44.310770+0000","last_undegraded":"2024-09-10T16:11:44.310770+0000","last_fullsized":"2024-09-10T16:11:44.310770+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20660,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20891","reported_seq":21061,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.312021+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:11:44.312021+0000","last_peered":"2024-09-10T16:11:44.312021+0000","last_clean":"2024-09-10T16:11:44.312021+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:11:44.312021+0000","last_undegraded":"2024-09-10T16:11:44.312021+0000","last_fullsized":"2024-09-10T16:11:44.312021+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17602,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'21061","reported_seq":21230,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310143+0000","last_change":"2024-09-10T16:10:52.241702+0000","last_active":"2024-09-10T16:11:44.310143+0000","last_peered":"2024-09-10T16:11:44.310143+0000","last_clean":"2024-09-10T16:11:44.310143+0000","last_became_active":"2024-09-10T16:10:52.171236+0000","last_became_peered":"2024-09-10T16:10:52.171236+0000","last_unstale":"2024-09-10T16:11:44.310143+0000","last_undegraded":"2024-09-10T16:11:44.310143+0000","last_fullsized":"2024-09-10T16:11:44.310143+0000","mapping_epoch":58,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:04:46.778280+0000","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17673,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4372,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'21082","reported_seq":21260,"reported_epoch":63,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:11:44.312026+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:11:44.312026+0000","last_peered":"2024-09-10T16:11:44.312026+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:11:44.312026+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17581,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4235},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'21314","reported_seq":21484,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.311841+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:11:44.311841+0000","last_peered":"2024-09-10T16:11:44.311841+0000","last_clean":"2024-09-10T16:11:44.311841+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:11:44.311841+0000","last_undegraded":"2024-09-10T16:11:44.311841+0000","last_fullsized":"2024-09-10T16:11:44.311841+0000","mapping_epoch":34,"log_start":"45'11300","ondisk_log_start":"45'11300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18161,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.309727+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:11:44.309727+0000","last_peered":"2024-09-10T16:11:44.309727+0000","last_clean":"2024-09-10T16:11:44.309727+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:11:44.309727+0000","last_undegraded":"2024-09-10T16:11:44.309727+0000","last_fullsized":"2024-09-10T16:11:44.309727+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20457","reported_seq":20641,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310911+0000","last_change":"2024-09-10T16:09:33.012840+0000","last_active":"2024-09-10T16:11:44.310911+0000","last_peered":"2024-09-10T16:11:44.310911+0000","last_clean":"2024-09-10T16:11:44.310911+0000","last_became_active":"2024-09-10T16:09:32.934056+0000","last_became_peered":"2024-09-10T16:09:32.934056+0000","last_unstale":"2024-09-10T16:11:44.310911+0000","last_undegraded":"2024-09-10T16:11:44.310911+0000","last_fullsized":"2024-09-10T16:11:44.310911+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:08:30.481235+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18276,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":4579,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":134,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310369+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:11:44.310369+0000","last_peered":"2024-09-10T16:11:44.310369+0000","last_clean":"2024-09-10T16:11:44.310369+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:11:44.310369+0000","last_undegraded":"2024-09-10T16:11:44.310369+0000","last_fullsized":"2024-09-10T16:11:44.310369+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20910","reported_seq":29603,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.312044+0000","last_change":"2024-09-10T16:09:08.909580+0000","last_active":"2024-09-10T16:11:44.312044+0000","last_peered":"2024-09-10T16:11:44.312044+0000","last_clean":"2024-09-10T16:11:44.312044+0000","last_became_active":"2024-09-10T16:09:08.854941+0000","last_became_peered":"2024-09-10T16:09:08.854941+0000","last_unstale":"2024-09-10T16:11:44.312044+0000","last_undegraded":"2024-09-10T16:11:44.312044+0000","last_fullsized":"2024-09-10T16:11:44.312044+0000","mapping_epoch":50,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:13:14.846691+0000","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17414,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4257,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":144,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.311936+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:11:44.311936+0000","last_peered":"2024-09-10T16:11:44.311936+0000","last_clean":"2024-09-10T16:11:44.311936+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:11:44.311936+0000","last_undegraded":"2024-09-10T16:11:44.311936+0000","last_fullsized":"2024-09-10T16:11:44.311936+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20781","reported_seq":29574,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310450+0000","last_change":"2024-09-10T16:10:20.907889+0000","last_active":"2024-09-10T16:11:44.310450+0000","last_peered":"2024-09-10T16:11:44.310450+0000","last_clean":"2024-09-10T16:11:44.310450+0000","last_became_active":"2024-09-10T16:10:20.852065+0000","last_became_peered":"2024-09-10T16:10:20.852065+0000","last_unstale":"2024-09-10T16:11:44.310450+0000","last_undegraded":"2024-09-10T16:11:44.310450+0000","last_fullsized":"2024-09-10T16:11:44.310450+0000","mapping_epoch":56,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:34.069549+0000","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18461,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4307,"num_bytes_recovered":8675,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":120,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310865+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:11:44.310865+0000","last_peered":"2024-09-10T16:11:44.310865+0000","last_clean":"2024-09-10T16:11:44.310865+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:11:44.310865+0000","last_undegraded":"2024-09-10T16:11:44.310865+0000","last_fullsized":"2024-09-10T16:11:44.310865+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'21170","reported_seq":30059,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310126+0000","last_change":"2024-09-10T16:10:50.728349+0000","last_active":"2024-09-10T16:11:44.310126+0000","last_peered":"2024-09-10T16:11:44.310126+0000","last_clean":"2024-09-10T16:11:44.310126+0000","last_became_active":"2024-09-10T16:08:10.860438+0000","last_became_peered":"2024-09-10T16:08:10.860438+0000","last_unstale":"2024-09-10T16:11:44.310126+0000","last_undegraded":"2024-09-10T16:11:44.310126+0000","last_fullsized":"2024-09-10T16:11:44.310126+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:38:13.781380+0000","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18724,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20838","reported_seq":23578,"reported_epoch":63,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:11:51.245010+0000","last_change":"2024-09-10T16:11:42.563598+0000","last_active":"2024-09-10T16:11:51.245010+0000","last_peered":"2024-09-10T16:11:51.245010+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:11:51.245010+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3023,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17469,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1282,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3023}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310321+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:11:44.310321+0000","last_peered":"2024-09-10T16:11:44.310321+0000","last_clean":"2024-09-10T16:11:44.310321+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:11:44.310321+0000","last_undegraded":"2024-09-10T16:11:44.310321+0000","last_fullsized":"2024-09-10T16:11:44.310321+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20598","reported_seq":29204,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310762+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:11:44.310762+0000","last_peered":"2024-09-10T16:11:44.310762+0000","last_clean":"2024-09-10T16:11:44.310762+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:11:44.310762+0000","last_undegraded":"2024-09-10T16:11:44.310762+0000","last_fullsized":"2024-09-10T16:11:44.310762+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16988,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":134,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310946+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:11:44.310946+0000","last_peered":"2024-09-10T16:11:44.310946+0000","last_clean":"2024-09-10T16:11:44.310946+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:11:44.310946+0000","last_undegraded":"2024-09-10T16:11:44.310946+0000","last_fullsized":"2024-09-10T16:11:44.310946+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'21048","reported_seq":21214,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310569+0000","last_change":"2024-09-10T16:11:17.315372+0000","last_active":"2024-09-10T16:11:44.310569+0000","last_peered":"2024-09-10T16:11:44.310569+0000","last_clean":"2024-09-10T16:11:44.310569+0000","last_became_active":"2024-09-10T16:11:17.234487+0000","last_became_peered":"2024-09-10T16:11:17.234487+0000","last_unstale":"2024-09-10T16:11:44.310569+0000","last_undegraded":"2024-09-10T16:11:44.310569+0000","last_fullsized":"2024-09-10T16:11:44.310569+0000","mapping_epoch":60,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:19:00.114196+0000","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17303,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4373,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":109,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310658+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:11:44.310658+0000","last_peered":"2024-09-10T16:11:44.310658+0000","last_clean":"2024-09-10T16:11:44.310658+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:11:44.310658+0000","last_undegraded":"2024-09-10T16:11:44.310658+0000","last_fullsized":"2024-09-10T16:11:44.310658+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":114,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310839+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:11:44.310839+0000","last_peered":"2024-09-10T16:11:44.310839+0000","last_clean":"2024-09-10T16:11:44.310839+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:11:44.310839+0000","last_undegraded":"2024-09-10T16:11:44.310839+0000","last_fullsized":"2024-09-10T16:11:44.310839+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'21198","reported_seq":21365,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.396057+0000","last_change":"2024-09-10T16:11:44.391021+0000","last_active":"2024-09-10T16:11:44.396057+0000","last_peered":"2024-09-10T16:11:44.396057+0000","last_clean":"2024-09-10T16:11:44.396057+0000","last_became_active":"2024-09-10T16:11:44.312647+0000","last_became_peered":"2024-09-10T16:11:44.312647+0000","last_unstale":"2024-09-10T16:11:44.396057+0000","last_undegraded":"2024-09-10T16:11:44.396057+0000","last_fullsized":"2024-09-10T16:11:44.396057+0000","mapping_epoch":62,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:33:28.041611+0000","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17361,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4325,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7260,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292096,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44130,"num_bytes_recovered":33097612,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":175738880,"data_stored":175644942,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161022,"ondisk_log_size":161022,"up":32,"acting":30,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986342,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":469128,"kb_used_data":104248,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93902712,"statfs":{"total":96636764160,"available":96156377088,"internally_reserved":0,"allocated":106749952,"data_stored":106369545,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659288,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":501488,"kb_used_data":544,"kb_used_omap":4,"kb_used_meta":500923,"kb_avail":93870352,"statfs":{"total":96636764160,"available":96123240448,"internally_reserved":0,"allocated":557056,"data_stored":179798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":512945838},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182570,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":547412,"kb_used_data":68732,"kb_used_omap":4,"kb_used_meta":478651,"kb_avail":93824428,"statfs":{"total":96636764160,"available":96076214272,"internally_reserved":0,"allocated":70381568,"data_stored":69993471,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":490139306},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":69853184,"data_stored":69817054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":105857024,"data_stored":105824507,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:11:56.473 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:11:56.778 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:11:56.778 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:11:56.790 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":496,"stamp":"2024-09-10T16:11:55.798165+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7260,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292098,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44132,"num_bytes_recovered":33097631,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161025,"ondisk_log_size":161025,"up":48,"acting":46,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1518028,"kb_used_data":173524,"kb_used_omap":12,"kb_used_meta":1344435,"kb_avail":281597492,"statfs":{"total":289910292480,"available":288355831808,"internally_reserved":0,"allocated":177688576,"data_stored":176542814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1376701442},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1635,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1635,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"11.489958"},"pg_stats":[{"pgid":"2.f","version":"49'20672","reported_seq":29313,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.314657+0000","last_change":"2024-09-10T16:09:57.088531+0000","last_active":"2024-09-10T16:11:44.314657+0000","last_peered":"2024-09-10T16:11:44.314657+0000","last_clean":"2024-09-10T16:11:44.314657+0000","last_became_active":"2024-09-10T16:09:57.005755+0000","last_became_peered":"2024-09-10T16:09:57.005755+0000","last_unstale":"2024-09-10T16:11:44.314657+0000","last_undegraded":"2024-09-10T16:11:44.314657+0000","last_fullsized":"2024-09-10T16:11:44.314657+0000","mapping_epoch":54,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:09:19.329697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20672,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4259,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20844","reported_seq":20997,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.311152+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:11:44.311152+0000","last_peered":"2024-09-10T16:11:44.311152+0000","last_clean":"2024-09-10T16:11:44.311152+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:11:44.311152+0000","last_undegraded":"2024-09-10T16:11:44.311152+0000","last_fullsized":"2024-09-10T16:11:44.311152+0000","mapping_epoch":35,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18949,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20798","reported_seq":29516,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310310+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:11:44.310310+0000","last_peered":"2024-09-10T16:11:44.310310+0000","last_clean":"2024-09-10T16:11:44.310310+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:11:44.310310+0000","last_undegraded":"2024-09-10T16:11:44.310310+0000","last_fullsized":"2024-09-10T16:11:44.310310+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18802,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20660","reported_seq":20781,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310770+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:11:44.310770+0000","last_peered":"2024-09-10T16:11:44.310770+0000","last_clean":"2024-09-10T16:11:44.310770+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:11:44.310770+0000","last_undegraded":"2024-09-10T16:11:44.310770+0000","last_fullsized":"2024-09-10T16:11:44.310770+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20660,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20891","reported_seq":21061,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.312021+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:11:44.312021+0000","last_peered":"2024-09-10T16:11:44.312021+0000","last_clean":"2024-09-10T16:11:44.312021+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:11:44.312021+0000","last_undegraded":"2024-09-10T16:11:44.312021+0000","last_fullsized":"2024-09-10T16:11:44.312021+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17602,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'21061","reported_seq":21230,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310143+0000","last_change":"2024-09-10T16:10:52.241702+0000","last_active":"2024-09-10T16:11:44.310143+0000","last_peered":"2024-09-10T16:11:44.310143+0000","last_clean":"2024-09-10T16:11:44.310143+0000","last_became_active":"2024-09-10T16:10:52.171236+0000","last_became_peered":"2024-09-10T16:10:52.171236+0000","last_unstale":"2024-09-10T16:11:44.310143+0000","last_undegraded":"2024-09-10T16:11:44.310143+0000","last_fullsized":"2024-09-10T16:11:44.310143+0000","mapping_epoch":58,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:04:46.778280+0000","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17673,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4372,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'21082","reported_seq":21260,"reported_epoch":63,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:11:44.312026+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:11:44.312026+0000","last_peered":"2024-09-10T16:11:44.312026+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:11:44.312026+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17581,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4235},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'21314","reported_seq":21484,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.311841+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:11:44.311841+0000","last_peered":"2024-09-10T16:11:44.311841+0000","last_clean":"2024-09-10T16:11:44.311841+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:11:44.311841+0000","last_undegraded":"2024-09-10T16:11:44.311841+0000","last_fullsized":"2024-09-10T16:11:44.311841+0000","mapping_epoch":34,"log_start":"45'11300","ondisk_log_start":"45'11300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18161,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.309727+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:11:44.309727+0000","last_peered":"2024-09-10T16:11:44.309727+0000","last_clean":"2024-09-10T16:11:44.309727+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:11:44.309727+0000","last_undegraded":"2024-09-10T16:11:44.309727+0000","last_fullsized":"2024-09-10T16:11:44.309727+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20457","reported_seq":20641,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310911+0000","last_change":"2024-09-10T16:09:33.012840+0000","last_active":"2024-09-10T16:11:44.310911+0000","last_peered":"2024-09-10T16:11:44.310911+0000","last_clean":"2024-09-10T16:11:44.310911+0000","last_became_active":"2024-09-10T16:09:32.934056+0000","last_became_peered":"2024-09-10T16:09:32.934056+0000","last_unstale":"2024-09-10T16:11:44.310911+0000","last_undegraded":"2024-09-10T16:11:44.310911+0000","last_fullsized":"2024-09-10T16:11:44.310911+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:08:30.481235+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18276,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":4579,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":134,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310369+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:11:44.310369+0000","last_peered":"2024-09-10T16:11:44.310369+0000","last_clean":"2024-09-10T16:11:44.310369+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:11:44.310369+0000","last_undegraded":"2024-09-10T16:11:44.310369+0000","last_fullsized":"2024-09-10T16:11:44.310369+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20910","reported_seq":29603,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.312044+0000","last_change":"2024-09-10T16:09:08.909580+0000","last_active":"2024-09-10T16:11:44.312044+0000","last_peered":"2024-09-10T16:11:44.312044+0000","last_clean":"2024-09-10T16:11:44.312044+0000","last_became_active":"2024-09-10T16:09:08.854941+0000","last_became_peered":"2024-09-10T16:09:08.854941+0000","last_unstale":"2024-09-10T16:11:44.312044+0000","last_undegraded":"2024-09-10T16:11:44.312044+0000","last_fullsized":"2024-09-10T16:11:44.312044+0000","mapping_epoch":50,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:13:14.846691+0000","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17414,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4257,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":144,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.311936+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:11:44.311936+0000","last_peered":"2024-09-10T16:11:44.311936+0000","last_clean":"2024-09-10T16:11:44.311936+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:11:44.311936+0000","last_undegraded":"2024-09-10T16:11:44.311936+0000","last_fullsized":"2024-09-10T16:11:44.311936+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20781","reported_seq":29574,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310450+0000","last_change":"2024-09-10T16:10:20.907889+0000","last_active":"2024-09-10T16:11:44.310450+0000","last_peered":"2024-09-10T16:11:44.310450+0000","last_clean":"2024-09-10T16:11:44.310450+0000","last_became_active":"2024-09-10T16:10:20.852065+0000","last_became_peered":"2024-09-10T16:10:20.852065+0000","last_unstale":"2024-09-10T16:11:44.310450+0000","last_undegraded":"2024-09-10T16:11:44.310450+0000","last_fullsized":"2024-09-10T16:11:44.310450+0000","mapping_epoch":56,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:34.069549+0000","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18461,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4307,"num_bytes_recovered":8675,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":120,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310865+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:11:44.310865+0000","last_peered":"2024-09-10T16:11:44.310865+0000","last_clean":"2024-09-10T16:11:44.310865+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:11:44.310865+0000","last_undegraded":"2024-09-10T16:11:44.310865+0000","last_fullsized":"2024-09-10T16:11:44.310865+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'21170","reported_seq":30059,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310126+0000","last_change":"2024-09-10T16:10:50.728349+0000","last_active":"2024-09-10T16:11:44.310126+0000","last_peered":"2024-09-10T16:11:44.310126+0000","last_clean":"2024-09-10T16:11:44.310126+0000","last_became_active":"2024-09-10T16:08:10.860438+0000","last_became_peered":"2024-09-10T16:08:10.860438+0000","last_unstale":"2024-09-10T16:11:44.310126+0000","last_undegraded":"2024-09-10T16:11:44.310126+0000","last_fullsized":"2024-09-10T16:11:44.310126+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:38:13.781380+0000","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18724,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20838","reported_seq":23578,"reported_epoch":63,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:11:51.245010+0000","last_change":"2024-09-10T16:11:42.563598+0000","last_active":"2024-09-10T16:11:51.245010+0000","last_peered":"2024-09-10T16:11:51.245010+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:11:51.245010+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3023,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17469,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1282,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3023}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310321+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:11:44.310321+0000","last_peered":"2024-09-10T16:11:44.310321+0000","last_clean":"2024-09-10T16:11:44.310321+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:11:44.310321+0000","last_undegraded":"2024-09-10T16:11:44.310321+0000","last_fullsized":"2024-09-10T16:11:44.310321+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20598","reported_seq":29204,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310762+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:11:44.310762+0000","last_peered":"2024-09-10T16:11:44.310762+0000","last_clean":"2024-09-10T16:11:44.310762+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:11:44.310762+0000","last_undegraded":"2024-09-10T16:11:44.310762+0000","last_fullsized":"2024-09-10T16:11:44.310762+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16988,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":134,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310946+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:11:44.310946+0000","last_peered":"2024-09-10T16:11:44.310946+0000","last_clean":"2024-09-10T16:11:44.310946+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:11:44.310946+0000","last_undegraded":"2024-09-10T16:11:44.310946+0000","last_fullsized":"2024-09-10T16:11:44.310946+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'21048","reported_seq":21214,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310569+0000","last_change":"2024-09-10T16:11:17.315372+0000","last_active":"2024-09-10T16:11:44.310569+0000","last_peered":"2024-09-10T16:11:44.310569+0000","last_clean":"2024-09-10T16:11:44.310569+0000","last_became_active":"2024-09-10T16:11:17.234487+0000","last_became_peered":"2024-09-10T16:11:17.234487+0000","last_unstale":"2024-09-10T16:11:44.310569+0000","last_undegraded":"2024-09-10T16:11:44.310569+0000","last_fullsized":"2024-09-10T16:11:44.310569+0000","mapping_epoch":60,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:19:00.114196+0000","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17303,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4373,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":109,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310658+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:11:44.310658+0000","last_peered":"2024-09-10T16:11:44.310658+0000","last_clean":"2024-09-10T16:11:44.310658+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:11:44.310658+0000","last_undegraded":"2024-09-10T16:11:44.310658+0000","last_fullsized":"2024-09-10T16:11:44.310658+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":114,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310839+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:11:44.310839+0000","last_peered":"2024-09-10T16:11:44.310839+0000","last_clean":"2024-09-10T16:11:44.310839+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:11:44.310839+0000","last_undegraded":"2024-09-10T16:11:44.310839+0000","last_fullsized":"2024-09-10T16:11:44.310839+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'21198","reported_seq":21365,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.396057+0000","last_change":"2024-09-10T16:11:44.391021+0000","last_active":"2024-09-10T16:11:44.396057+0000","last_peered":"2024-09-10T16:11:44.396057+0000","last_clean":"2024-09-10T16:11:44.396057+0000","last_became_active":"2024-09-10T16:11:44.312647+0000","last_became_peered":"2024-09-10T16:11:44.312647+0000","last_unstale":"2024-09-10T16:11:44.396057+0000","last_undegraded":"2024-09-10T16:11:44.396057+0000","last_fullsized":"2024-09-10T16:11:44.396057+0000","mapping_epoch":62,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:33:28.041611+0000","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17361,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4325,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7260,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292096,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44130,"num_bytes_recovered":33097612,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":175738880,"data_stored":175644942,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161022,"ondisk_log_size":161022,"up":32,"acting":30,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986342,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":469128,"kb_used_data":104248,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93902712,"statfs":{"total":96636764160,"available":96156377088,"internally_reserved":0,"allocated":106749952,"data_stored":106369545,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659288,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":501488,"kb_used_data":544,"kb_used_omap":4,"kb_used_meta":500923,"kb_avail":93870352,"statfs":{"total":96636764160,"available":96123240448,"internally_reserved":0,"allocated":557056,"data_stored":179798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":512945838},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182570,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":547412,"kb_used_data":68732,"kb_used_omap":4,"kb_used_meta":478651,"kb_avail":93824428,"statfs":{"total":96636764160,"available":96076214272,"internally_reserved":0,"allocated":70381568,"data_stored":69993471,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":490139306},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":69853184,"data_stored":69817054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":105857024,"data_stored":105824507,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:11:56.792 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-10T16:11:56.792 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "49'21082", 'reported_seq': 21260, 'reported_epoch': 63, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:11:44.312026+0000', 'last_change': '2024-09-10T15:59:16.331899+0000', 'last_active': '2024-09-10T16:11:44.312026+0000', 'last_peered': '2024-09-10T16:11:44.312026+0000', 'last_clean': '2024-09-10T15:59:14.139436+0000', 'last_became_active': '2024-09-10T15:59:16.318715+0000', 'last_became_peered': '2024-09-10T15:59:16.318715+0000', 'last_unstale': '2024-09-10T16:11:44.312026+0000', 'last_undegraded': '2024-09-10T15:59:15.877524+0000', 'last_fullsized': '2024-09-10T15:59:15.769042+0000', 'mapping_epoch': 42, 'log_start': "45'11000", 'ondisk_log_start': "45'11000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3537", 'last_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'objects_scrubbed': 3497, 'log_size': 10082, 'log_dups_size': 0, 'ondisk_log_size': 10082, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1224, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4237, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 17581, 'num_write_kb': 28172, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4235}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:11:56.792 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-10T16:11:56.792 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "49'20838", 'reported_seq': 23578, 'reported_epoch': 63, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:11:51.245010+0000', 'last_change': '2024-09-10T16:11:42.563598+0000', 'last_active': '2024-09-10T16:11:51.245010+0000', 'last_peered': '2024-09-10T16:11:51.245010+0000', 'last_clean': '2024-09-10T15:59:14.117001+0000', 'last_became_active': '2024-09-10T15:59:16.302245+0000', 'last_became_peered': '2024-09-10T15:59:16.302245+0000', 'last_unstale': '2024-09-10T16:11:51.245010+0000', 'last_undegraded': '2024-09-10T15:59:15.845012+0000', 'last_fullsized': '2024-09-10T15:59:15.770337+0000', 'mapping_epoch': 42, 'log_start': "45'10800", 'ondisk_log_start': "45'10800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3404", 'last_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'objects_scrubbed': 3367, 'log_size': 10038, 'log_dups_size': 0, 'ondisk_log_size': 10038, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 983, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3023, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17469, 'num_write_kb': 28204, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1282, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3023}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:11:56.792 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T16:11:57.196 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:11:57.196 INFO:teuthology.orchestra.run.smithi115.stdout:{"fsid":"7effee29-df05-4351-9741-ee203564ed81","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 7260/24 objects degraded (30250.000%), 2 pgs degraded, 2 pgs undersized","count":4},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":881,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":63,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983954,"num_in_osds":3,"osd_in_since":1725983947,"num_remapped_pgs":2},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":22},{"state_name":"active+recovering+undersized+degraded+remapped","count":1},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":12,"data_bytes":4849,"bytes_used":1554460672,"bytes_avail":288355831808,"bytes_total":289910292480,"degraded_objects":7260,"degraded_total":24,"degraded_ratio":302.5,"recovering_objects_per_sec":142,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1635,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-10T15:57:14:967120+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T16:00:13.560776+0000","services":{}},"progress_events":{}} 2024-09-10T16:11:57.209 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-10T16:11:57.209 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:11:57.537 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:11:57.538 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:11:57.550 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":496,"stamp":"2024-09-10T16:11:55.798165+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7260,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292098,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44132,"num_bytes_recovered":33097631,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161025,"ondisk_log_size":161025,"up":48,"acting":46,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1518028,"kb_used_data":173524,"kb_used_omap":12,"kb_used_meta":1344435,"kb_avail":281597492,"statfs":{"total":289910292480,"available":288355831808,"internally_reserved":0,"allocated":177688576,"data_stored":176542814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1376701442},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1635,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1635,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"11.489958"},"pg_stats":[{"pgid":"2.f","version":"49'20672","reported_seq":29313,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.314657+0000","last_change":"2024-09-10T16:09:57.088531+0000","last_active":"2024-09-10T16:11:44.314657+0000","last_peered":"2024-09-10T16:11:44.314657+0000","last_clean":"2024-09-10T16:11:44.314657+0000","last_became_active":"2024-09-10T16:09:57.005755+0000","last_became_peered":"2024-09-10T16:09:57.005755+0000","last_unstale":"2024-09-10T16:11:44.314657+0000","last_undegraded":"2024-09-10T16:11:44.314657+0000","last_fullsized":"2024-09-10T16:11:44.314657+0000","mapping_epoch":54,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:09:19.329697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20672,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4259,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20844","reported_seq":20997,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.311152+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:11:44.311152+0000","last_peered":"2024-09-10T16:11:44.311152+0000","last_clean":"2024-09-10T16:11:44.311152+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:11:44.311152+0000","last_undegraded":"2024-09-10T16:11:44.311152+0000","last_fullsized":"2024-09-10T16:11:44.311152+0000","mapping_epoch":35,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18949,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20798","reported_seq":29516,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310310+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:11:44.310310+0000","last_peered":"2024-09-10T16:11:44.310310+0000","last_clean":"2024-09-10T16:11:44.310310+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:11:44.310310+0000","last_undegraded":"2024-09-10T16:11:44.310310+0000","last_fullsized":"2024-09-10T16:11:44.310310+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18802,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20660","reported_seq":20781,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310770+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:11:44.310770+0000","last_peered":"2024-09-10T16:11:44.310770+0000","last_clean":"2024-09-10T16:11:44.310770+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:11:44.310770+0000","last_undegraded":"2024-09-10T16:11:44.310770+0000","last_fullsized":"2024-09-10T16:11:44.310770+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20660,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20891","reported_seq":21061,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.312021+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:11:44.312021+0000","last_peered":"2024-09-10T16:11:44.312021+0000","last_clean":"2024-09-10T16:11:44.312021+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:11:44.312021+0000","last_undegraded":"2024-09-10T16:11:44.312021+0000","last_fullsized":"2024-09-10T16:11:44.312021+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17602,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'21061","reported_seq":21230,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310143+0000","last_change":"2024-09-10T16:10:52.241702+0000","last_active":"2024-09-10T16:11:44.310143+0000","last_peered":"2024-09-10T16:11:44.310143+0000","last_clean":"2024-09-10T16:11:44.310143+0000","last_became_active":"2024-09-10T16:10:52.171236+0000","last_became_peered":"2024-09-10T16:10:52.171236+0000","last_unstale":"2024-09-10T16:11:44.310143+0000","last_undegraded":"2024-09-10T16:11:44.310143+0000","last_fullsized":"2024-09-10T16:11:44.310143+0000","mapping_epoch":58,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:04:46.778280+0000","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17673,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4372,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'21082","reported_seq":21260,"reported_epoch":63,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:11:44.312026+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:11:44.312026+0000","last_peered":"2024-09-10T16:11:44.312026+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:11:44.312026+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17581,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4235},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'21314","reported_seq":21484,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.311841+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:11:44.311841+0000","last_peered":"2024-09-10T16:11:44.311841+0000","last_clean":"2024-09-10T16:11:44.311841+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:11:44.311841+0000","last_undegraded":"2024-09-10T16:11:44.311841+0000","last_fullsized":"2024-09-10T16:11:44.311841+0000","mapping_epoch":34,"log_start":"45'11300","ondisk_log_start":"45'11300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18161,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.309727+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:11:44.309727+0000","last_peered":"2024-09-10T16:11:44.309727+0000","last_clean":"2024-09-10T16:11:44.309727+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:11:44.309727+0000","last_undegraded":"2024-09-10T16:11:44.309727+0000","last_fullsized":"2024-09-10T16:11:44.309727+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20457","reported_seq":20641,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310911+0000","last_change":"2024-09-10T16:09:33.012840+0000","last_active":"2024-09-10T16:11:44.310911+0000","last_peered":"2024-09-10T16:11:44.310911+0000","last_clean":"2024-09-10T16:11:44.310911+0000","last_became_active":"2024-09-10T16:09:32.934056+0000","last_became_peered":"2024-09-10T16:09:32.934056+0000","last_unstale":"2024-09-10T16:11:44.310911+0000","last_undegraded":"2024-09-10T16:11:44.310911+0000","last_fullsized":"2024-09-10T16:11:44.310911+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:08:30.481235+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18276,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":4579,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":134,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310369+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:11:44.310369+0000","last_peered":"2024-09-10T16:11:44.310369+0000","last_clean":"2024-09-10T16:11:44.310369+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:11:44.310369+0000","last_undegraded":"2024-09-10T16:11:44.310369+0000","last_fullsized":"2024-09-10T16:11:44.310369+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20910","reported_seq":29603,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.312044+0000","last_change":"2024-09-10T16:09:08.909580+0000","last_active":"2024-09-10T16:11:44.312044+0000","last_peered":"2024-09-10T16:11:44.312044+0000","last_clean":"2024-09-10T16:11:44.312044+0000","last_became_active":"2024-09-10T16:09:08.854941+0000","last_became_peered":"2024-09-10T16:09:08.854941+0000","last_unstale":"2024-09-10T16:11:44.312044+0000","last_undegraded":"2024-09-10T16:11:44.312044+0000","last_fullsized":"2024-09-10T16:11:44.312044+0000","mapping_epoch":50,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:13:14.846691+0000","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17414,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4257,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":144,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.311936+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:11:44.311936+0000","last_peered":"2024-09-10T16:11:44.311936+0000","last_clean":"2024-09-10T16:11:44.311936+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:11:44.311936+0000","last_undegraded":"2024-09-10T16:11:44.311936+0000","last_fullsized":"2024-09-10T16:11:44.311936+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20781","reported_seq":29574,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310450+0000","last_change":"2024-09-10T16:10:20.907889+0000","last_active":"2024-09-10T16:11:44.310450+0000","last_peered":"2024-09-10T16:11:44.310450+0000","last_clean":"2024-09-10T16:11:44.310450+0000","last_became_active":"2024-09-10T16:10:20.852065+0000","last_became_peered":"2024-09-10T16:10:20.852065+0000","last_unstale":"2024-09-10T16:11:44.310450+0000","last_undegraded":"2024-09-10T16:11:44.310450+0000","last_fullsized":"2024-09-10T16:11:44.310450+0000","mapping_epoch":56,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:34.069549+0000","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18461,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4307,"num_bytes_recovered":8675,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":120,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310865+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:11:44.310865+0000","last_peered":"2024-09-10T16:11:44.310865+0000","last_clean":"2024-09-10T16:11:44.310865+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:11:44.310865+0000","last_undegraded":"2024-09-10T16:11:44.310865+0000","last_fullsized":"2024-09-10T16:11:44.310865+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'21170","reported_seq":30059,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310126+0000","last_change":"2024-09-10T16:10:50.728349+0000","last_active":"2024-09-10T16:11:44.310126+0000","last_peered":"2024-09-10T16:11:44.310126+0000","last_clean":"2024-09-10T16:11:44.310126+0000","last_became_active":"2024-09-10T16:08:10.860438+0000","last_became_peered":"2024-09-10T16:08:10.860438+0000","last_unstale":"2024-09-10T16:11:44.310126+0000","last_undegraded":"2024-09-10T16:11:44.310126+0000","last_fullsized":"2024-09-10T16:11:44.310126+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:38:13.781380+0000","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18724,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20838","reported_seq":23578,"reported_epoch":63,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:11:51.245010+0000","last_change":"2024-09-10T16:11:42.563598+0000","last_active":"2024-09-10T16:11:51.245010+0000","last_peered":"2024-09-10T16:11:51.245010+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:11:51.245010+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3023,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17469,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1282,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3023}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310321+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:11:44.310321+0000","last_peered":"2024-09-10T16:11:44.310321+0000","last_clean":"2024-09-10T16:11:44.310321+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:11:44.310321+0000","last_undegraded":"2024-09-10T16:11:44.310321+0000","last_fullsized":"2024-09-10T16:11:44.310321+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20598","reported_seq":29204,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310762+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:11:44.310762+0000","last_peered":"2024-09-10T16:11:44.310762+0000","last_clean":"2024-09-10T16:11:44.310762+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:11:44.310762+0000","last_undegraded":"2024-09-10T16:11:44.310762+0000","last_fullsized":"2024-09-10T16:11:44.310762+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16988,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":134,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310946+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:11:44.310946+0000","last_peered":"2024-09-10T16:11:44.310946+0000","last_clean":"2024-09-10T16:11:44.310946+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:11:44.310946+0000","last_undegraded":"2024-09-10T16:11:44.310946+0000","last_fullsized":"2024-09-10T16:11:44.310946+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'21048","reported_seq":21214,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310569+0000","last_change":"2024-09-10T16:11:17.315372+0000","last_active":"2024-09-10T16:11:44.310569+0000","last_peered":"2024-09-10T16:11:44.310569+0000","last_clean":"2024-09-10T16:11:44.310569+0000","last_became_active":"2024-09-10T16:11:17.234487+0000","last_became_peered":"2024-09-10T16:11:17.234487+0000","last_unstale":"2024-09-10T16:11:44.310569+0000","last_undegraded":"2024-09-10T16:11:44.310569+0000","last_fullsized":"2024-09-10T16:11:44.310569+0000","mapping_epoch":60,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:19:00.114196+0000","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17303,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4373,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":109,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310658+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:11:44.310658+0000","last_peered":"2024-09-10T16:11:44.310658+0000","last_clean":"2024-09-10T16:11:44.310658+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:11:44.310658+0000","last_undegraded":"2024-09-10T16:11:44.310658+0000","last_fullsized":"2024-09-10T16:11:44.310658+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":114,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310839+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:11:44.310839+0000","last_peered":"2024-09-10T16:11:44.310839+0000","last_clean":"2024-09-10T16:11:44.310839+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:11:44.310839+0000","last_undegraded":"2024-09-10T16:11:44.310839+0000","last_fullsized":"2024-09-10T16:11:44.310839+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'21198","reported_seq":21365,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.396057+0000","last_change":"2024-09-10T16:11:44.391021+0000","last_active":"2024-09-10T16:11:44.396057+0000","last_peered":"2024-09-10T16:11:44.396057+0000","last_clean":"2024-09-10T16:11:44.396057+0000","last_became_active":"2024-09-10T16:11:44.312647+0000","last_became_peered":"2024-09-10T16:11:44.312647+0000","last_unstale":"2024-09-10T16:11:44.396057+0000","last_undegraded":"2024-09-10T16:11:44.396057+0000","last_fullsized":"2024-09-10T16:11:44.396057+0000","mapping_epoch":62,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:33:28.041611+0000","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17361,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4325,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7260,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292096,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44130,"num_bytes_recovered":33097612,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":175738880,"data_stored":175644942,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161022,"ondisk_log_size":161022,"up":32,"acting":30,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986342,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":469128,"kb_used_data":104248,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93902712,"statfs":{"total":96636764160,"available":96156377088,"internally_reserved":0,"allocated":106749952,"data_stored":106369545,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659288,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":501488,"kb_used_data":544,"kb_used_omap":4,"kb_used_meta":500923,"kb_avail":93870352,"statfs":{"total":96636764160,"available":96123240448,"internally_reserved":0,"allocated":557056,"data_stored":179798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":512945838},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182570,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":547412,"kb_used_data":68732,"kb_used_omap":4,"kb_used_meta":478651,"kb_avail":93824428,"statfs":{"total":96636764160,"available":96076214272,"internally_reserved":0,"allocated":70381568,"data_stored":69993471,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":490139306},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":69853184,"data_stored":69817054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":105857024,"data_stored":105824507,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:12:00.554 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:12:00.877 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:12:00.877 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:12:00.889 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":498,"stamp":"2024-09-10T16:11:59.798797+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":6636,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292098,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44756,"num_bytes_recovered":33097631,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161025,"ondisk_log_size":161025,"up":48,"acting":46,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1514668,"kb_used_data":170164,"kb_used_omap":12,"kb_used_meta":1344435,"kb_avail":281600852,"statfs":{"total":289910292480,"available":288359272448,"internally_reserved":0,"allocated":174247936,"data_stored":173102174,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1376701442},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1325,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001829"},"pg_stats":[{"pgid":"2.f","version":"49'20672","reported_seq":29313,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.314657+0000","last_change":"2024-09-10T16:09:57.088531+0000","last_active":"2024-09-10T16:11:44.314657+0000","last_peered":"2024-09-10T16:11:44.314657+0000","last_clean":"2024-09-10T16:11:44.314657+0000","last_became_active":"2024-09-10T16:09:57.005755+0000","last_became_peered":"2024-09-10T16:09:57.005755+0000","last_unstale":"2024-09-10T16:11:44.314657+0000","last_undegraded":"2024-09-10T16:11:44.314657+0000","last_fullsized":"2024-09-10T16:11:44.314657+0000","mapping_epoch":54,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:09:19.329697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20672,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4259,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20844","reported_seq":20997,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.311152+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:11:44.311152+0000","last_peered":"2024-09-10T16:11:44.311152+0000","last_clean":"2024-09-10T16:11:44.311152+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:11:44.311152+0000","last_undegraded":"2024-09-10T16:11:44.311152+0000","last_fullsized":"2024-09-10T16:11:44.311152+0000","mapping_epoch":35,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18949,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20798","reported_seq":29516,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310310+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:11:44.310310+0000","last_peered":"2024-09-10T16:11:44.310310+0000","last_clean":"2024-09-10T16:11:44.310310+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:11:44.310310+0000","last_undegraded":"2024-09-10T16:11:44.310310+0000","last_fullsized":"2024-09-10T16:11:44.310310+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18802,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20660","reported_seq":20781,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310770+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:11:44.310770+0000","last_peered":"2024-09-10T16:11:44.310770+0000","last_clean":"2024-09-10T16:11:44.310770+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:11:44.310770+0000","last_undegraded":"2024-09-10T16:11:44.310770+0000","last_fullsized":"2024-09-10T16:11:44.310770+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20660,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20891","reported_seq":21061,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.312021+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:11:44.312021+0000","last_peered":"2024-09-10T16:11:44.312021+0000","last_clean":"2024-09-10T16:11:44.312021+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:11:44.312021+0000","last_undegraded":"2024-09-10T16:11:44.312021+0000","last_fullsized":"2024-09-10T16:11:44.312021+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17602,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'21061","reported_seq":21230,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310143+0000","last_change":"2024-09-10T16:10:52.241702+0000","last_active":"2024-09-10T16:11:44.310143+0000","last_peered":"2024-09-10T16:11:44.310143+0000","last_clean":"2024-09-10T16:11:44.310143+0000","last_became_active":"2024-09-10T16:10:52.171236+0000","last_became_peered":"2024-09-10T16:10:52.171236+0000","last_unstale":"2024-09-10T16:11:44.310143+0000","last_undegraded":"2024-09-10T16:11:44.310143+0000","last_fullsized":"2024-09-10T16:11:44.310143+0000","mapping_epoch":58,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:04:46.778280+0000","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17673,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4372,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'21082","reported_seq":21260,"reported_epoch":63,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:11:44.312026+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:11:44.312026+0000","last_peered":"2024-09-10T16:11:44.312026+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:11:44.312026+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17581,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4235},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'21314","reported_seq":21484,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.311841+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:11:44.311841+0000","last_peered":"2024-09-10T16:11:44.311841+0000","last_clean":"2024-09-10T16:11:44.311841+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:11:44.311841+0000","last_undegraded":"2024-09-10T16:11:44.311841+0000","last_fullsized":"2024-09-10T16:11:44.311841+0000","mapping_epoch":34,"log_start":"45'11300","ondisk_log_start":"45'11300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18161,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.309727+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:11:44.309727+0000","last_peered":"2024-09-10T16:11:44.309727+0000","last_clean":"2024-09-10T16:11:44.309727+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:11:44.309727+0000","last_undegraded":"2024-09-10T16:11:44.309727+0000","last_fullsized":"2024-09-10T16:11:44.309727+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20457","reported_seq":20641,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310911+0000","last_change":"2024-09-10T16:09:33.012840+0000","last_active":"2024-09-10T16:11:44.310911+0000","last_peered":"2024-09-10T16:11:44.310911+0000","last_clean":"2024-09-10T16:11:44.310911+0000","last_became_active":"2024-09-10T16:09:32.934056+0000","last_became_peered":"2024-09-10T16:09:32.934056+0000","last_unstale":"2024-09-10T16:11:44.310911+0000","last_undegraded":"2024-09-10T16:11:44.310911+0000","last_fullsized":"2024-09-10T16:11:44.310911+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:08:30.481235+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18276,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":4579,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":134,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310369+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:11:44.310369+0000","last_peered":"2024-09-10T16:11:44.310369+0000","last_clean":"2024-09-10T16:11:44.310369+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:11:44.310369+0000","last_undegraded":"2024-09-10T16:11:44.310369+0000","last_fullsized":"2024-09-10T16:11:44.310369+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20910","reported_seq":29603,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.312044+0000","last_change":"2024-09-10T16:09:08.909580+0000","last_active":"2024-09-10T16:11:44.312044+0000","last_peered":"2024-09-10T16:11:44.312044+0000","last_clean":"2024-09-10T16:11:44.312044+0000","last_became_active":"2024-09-10T16:09:08.854941+0000","last_became_peered":"2024-09-10T16:09:08.854941+0000","last_unstale":"2024-09-10T16:11:44.312044+0000","last_undegraded":"2024-09-10T16:11:44.312044+0000","last_fullsized":"2024-09-10T16:11:44.312044+0000","mapping_epoch":50,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:13:14.846691+0000","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17414,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4257,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":144,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.311936+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:11:44.311936+0000","last_peered":"2024-09-10T16:11:44.311936+0000","last_clean":"2024-09-10T16:11:44.311936+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:11:44.311936+0000","last_undegraded":"2024-09-10T16:11:44.311936+0000","last_fullsized":"2024-09-10T16:11:44.311936+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20781","reported_seq":29574,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310450+0000","last_change":"2024-09-10T16:10:20.907889+0000","last_active":"2024-09-10T16:11:44.310450+0000","last_peered":"2024-09-10T16:11:44.310450+0000","last_clean":"2024-09-10T16:11:44.310450+0000","last_became_active":"2024-09-10T16:10:20.852065+0000","last_became_peered":"2024-09-10T16:10:20.852065+0000","last_unstale":"2024-09-10T16:11:44.310450+0000","last_undegraded":"2024-09-10T16:11:44.310450+0000","last_fullsized":"2024-09-10T16:11:44.310450+0000","mapping_epoch":56,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:34.069549+0000","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18461,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4307,"num_bytes_recovered":8675,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":120,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310865+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:11:44.310865+0000","last_peered":"2024-09-10T16:11:44.310865+0000","last_clean":"2024-09-10T16:11:44.310865+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:11:44.310865+0000","last_undegraded":"2024-09-10T16:11:44.310865+0000","last_fullsized":"2024-09-10T16:11:44.310865+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'21170","reported_seq":30059,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310126+0000","last_change":"2024-09-10T16:10:50.728349+0000","last_active":"2024-09-10T16:11:44.310126+0000","last_peered":"2024-09-10T16:11:44.310126+0000","last_clean":"2024-09-10T16:11:44.310126+0000","last_became_active":"2024-09-10T16:08:10.860438+0000","last_became_peered":"2024-09-10T16:08:10.860438+0000","last_unstale":"2024-09-10T16:11:44.310126+0000","last_undegraded":"2024-09-10T16:11:44.310126+0000","last_fullsized":"2024-09-10T16:11:44.310126+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:38:13.781380+0000","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18724,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20838","reported_seq":24826,"reported_epoch":63,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:11:56.243591+0000","last_change":"2024-09-10T16:11:42.563598+0000","last_active":"2024-09-10T16:11:56.243591+0000","last_peered":"2024-09-10T16:11:56.243591+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:11:56.243591+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2399,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17469,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1906,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2399}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310321+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:11:44.310321+0000","last_peered":"2024-09-10T16:11:44.310321+0000","last_clean":"2024-09-10T16:11:44.310321+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:11:44.310321+0000","last_undegraded":"2024-09-10T16:11:44.310321+0000","last_fullsized":"2024-09-10T16:11:44.310321+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20598","reported_seq":29204,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310762+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:11:44.310762+0000","last_peered":"2024-09-10T16:11:44.310762+0000","last_clean":"2024-09-10T16:11:44.310762+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:11:44.310762+0000","last_undegraded":"2024-09-10T16:11:44.310762+0000","last_fullsized":"2024-09-10T16:11:44.310762+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16988,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":134,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310946+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:11:44.310946+0000","last_peered":"2024-09-10T16:11:44.310946+0000","last_clean":"2024-09-10T16:11:44.310946+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:11:44.310946+0000","last_undegraded":"2024-09-10T16:11:44.310946+0000","last_fullsized":"2024-09-10T16:11:44.310946+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'21048","reported_seq":21214,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310569+0000","last_change":"2024-09-10T16:11:17.315372+0000","last_active":"2024-09-10T16:11:44.310569+0000","last_peered":"2024-09-10T16:11:44.310569+0000","last_clean":"2024-09-10T16:11:44.310569+0000","last_became_active":"2024-09-10T16:11:17.234487+0000","last_became_peered":"2024-09-10T16:11:17.234487+0000","last_unstale":"2024-09-10T16:11:44.310569+0000","last_undegraded":"2024-09-10T16:11:44.310569+0000","last_fullsized":"2024-09-10T16:11:44.310569+0000","mapping_epoch":60,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:19:00.114196+0000","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17303,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4373,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":109,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310658+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:11:44.310658+0000","last_peered":"2024-09-10T16:11:44.310658+0000","last_clean":"2024-09-10T16:11:44.310658+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:11:44.310658+0000","last_undegraded":"2024-09-10T16:11:44.310658+0000","last_fullsized":"2024-09-10T16:11:44.310658+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":114,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310839+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:11:44.310839+0000","last_peered":"2024-09-10T16:11:44.310839+0000","last_clean":"2024-09-10T16:11:44.310839+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:11:44.310839+0000","last_undegraded":"2024-09-10T16:11:44.310839+0000","last_fullsized":"2024-09-10T16:11:44.310839+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'21198","reported_seq":21365,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.396057+0000","last_change":"2024-09-10T16:11:44.391021+0000","last_active":"2024-09-10T16:11:44.396057+0000","last_peered":"2024-09-10T16:11:44.396057+0000","last_clean":"2024-09-10T16:11:44.396057+0000","last_became_active":"2024-09-10T16:11:44.312647+0000","last_became_peered":"2024-09-10T16:11:44.312647+0000","last_unstale":"2024-09-10T16:11:44.396057+0000","last_undegraded":"2024-09-10T16:11:44.396057+0000","last_fullsized":"2024-09-10T16:11:44.396057+0000","mapping_epoch":62,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:33:28.041611+0000","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17361,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4325,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":6636,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292096,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44754,"num_bytes_recovered":33097612,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":172052480,"data_stored":171958542,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161022,"ondisk_log_size":161022,"up":32,"acting":30,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986343,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":465768,"kb_used_data":100888,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93906072,"statfs":{"total":96636764160,"available":96159817728,"internally_reserved":0,"allocated":103309312,"data_stored":102928905,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659288,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":501488,"kb_used_data":544,"kb_used_omap":4,"kb_used_meta":500923,"kb_avail":93870352,"statfs":{"total":96636764160,"available":96123240448,"internally_reserved":0,"allocated":557056,"data_stored":179798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":512945838},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182571,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":547412,"kb_used_data":68732,"kb_used_omap":4,"kb_used_meta":478651,"kb_avail":93824428,"statfs":{"total":96636764160,"available":96076214272,"internally_reserved":0,"allocated":70381568,"data_stored":69993471,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":490139306},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":69853184,"data_stored":69817054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102170624,"data_stored":102138107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:12:00.891 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:12:01.188 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:12:01.189 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:12:01.201 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":498,"stamp":"2024-09-10T16:11:59.798797+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":6636,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292098,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44756,"num_bytes_recovered":33097631,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161025,"ondisk_log_size":161025,"up":48,"acting":46,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1514668,"kb_used_data":170164,"kb_used_omap":12,"kb_used_meta":1344435,"kb_avail":281600852,"statfs":{"total":289910292480,"available":288359272448,"internally_reserved":0,"allocated":174247936,"data_stored":173102174,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1376701442},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1325,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001829"},"pg_stats":[{"pgid":"2.f","version":"49'20672","reported_seq":29313,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.314657+0000","last_change":"2024-09-10T16:09:57.088531+0000","last_active":"2024-09-10T16:11:44.314657+0000","last_peered":"2024-09-10T16:11:44.314657+0000","last_clean":"2024-09-10T16:11:44.314657+0000","last_became_active":"2024-09-10T16:09:57.005755+0000","last_became_peered":"2024-09-10T16:09:57.005755+0000","last_unstale":"2024-09-10T16:11:44.314657+0000","last_undegraded":"2024-09-10T16:11:44.314657+0000","last_fullsized":"2024-09-10T16:11:44.314657+0000","mapping_epoch":54,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:09:19.329697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20672,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4259,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20844","reported_seq":20997,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.311152+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:11:44.311152+0000","last_peered":"2024-09-10T16:11:44.311152+0000","last_clean":"2024-09-10T16:11:44.311152+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:11:44.311152+0000","last_undegraded":"2024-09-10T16:11:44.311152+0000","last_fullsized":"2024-09-10T16:11:44.311152+0000","mapping_epoch":35,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18949,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20798","reported_seq":29516,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310310+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:11:44.310310+0000","last_peered":"2024-09-10T16:11:44.310310+0000","last_clean":"2024-09-10T16:11:44.310310+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:11:44.310310+0000","last_undegraded":"2024-09-10T16:11:44.310310+0000","last_fullsized":"2024-09-10T16:11:44.310310+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18802,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20660","reported_seq":20781,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310770+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:11:44.310770+0000","last_peered":"2024-09-10T16:11:44.310770+0000","last_clean":"2024-09-10T16:11:44.310770+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:11:44.310770+0000","last_undegraded":"2024-09-10T16:11:44.310770+0000","last_fullsized":"2024-09-10T16:11:44.310770+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20660,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20891","reported_seq":21061,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.312021+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:11:44.312021+0000","last_peered":"2024-09-10T16:11:44.312021+0000","last_clean":"2024-09-10T16:11:44.312021+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:11:44.312021+0000","last_undegraded":"2024-09-10T16:11:44.312021+0000","last_fullsized":"2024-09-10T16:11:44.312021+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17602,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'21061","reported_seq":21230,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310143+0000","last_change":"2024-09-10T16:10:52.241702+0000","last_active":"2024-09-10T16:11:44.310143+0000","last_peered":"2024-09-10T16:11:44.310143+0000","last_clean":"2024-09-10T16:11:44.310143+0000","last_became_active":"2024-09-10T16:10:52.171236+0000","last_became_peered":"2024-09-10T16:10:52.171236+0000","last_unstale":"2024-09-10T16:11:44.310143+0000","last_undegraded":"2024-09-10T16:11:44.310143+0000","last_fullsized":"2024-09-10T16:11:44.310143+0000","mapping_epoch":58,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:04:46.778280+0000","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17673,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4372,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'21082","reported_seq":21260,"reported_epoch":63,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:11:44.312026+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:11:44.312026+0000","last_peered":"2024-09-10T16:11:44.312026+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:11:44.312026+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17581,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4235},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'21314","reported_seq":21484,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.311841+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:11:44.311841+0000","last_peered":"2024-09-10T16:11:44.311841+0000","last_clean":"2024-09-10T16:11:44.311841+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:11:44.311841+0000","last_undegraded":"2024-09-10T16:11:44.311841+0000","last_fullsized":"2024-09-10T16:11:44.311841+0000","mapping_epoch":34,"log_start":"45'11300","ondisk_log_start":"45'11300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18161,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.309727+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:11:44.309727+0000","last_peered":"2024-09-10T16:11:44.309727+0000","last_clean":"2024-09-10T16:11:44.309727+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:11:44.309727+0000","last_undegraded":"2024-09-10T16:11:44.309727+0000","last_fullsized":"2024-09-10T16:11:44.309727+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20457","reported_seq":20641,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310911+0000","last_change":"2024-09-10T16:09:33.012840+0000","last_active":"2024-09-10T16:11:44.310911+0000","last_peered":"2024-09-10T16:11:44.310911+0000","last_clean":"2024-09-10T16:11:44.310911+0000","last_became_active":"2024-09-10T16:09:32.934056+0000","last_became_peered":"2024-09-10T16:09:32.934056+0000","last_unstale":"2024-09-10T16:11:44.310911+0000","last_undegraded":"2024-09-10T16:11:44.310911+0000","last_fullsized":"2024-09-10T16:11:44.310911+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:08:30.481235+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18276,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":4579,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":134,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310369+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:11:44.310369+0000","last_peered":"2024-09-10T16:11:44.310369+0000","last_clean":"2024-09-10T16:11:44.310369+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:11:44.310369+0000","last_undegraded":"2024-09-10T16:11:44.310369+0000","last_fullsized":"2024-09-10T16:11:44.310369+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20910","reported_seq":29603,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.312044+0000","last_change":"2024-09-10T16:09:08.909580+0000","last_active":"2024-09-10T16:11:44.312044+0000","last_peered":"2024-09-10T16:11:44.312044+0000","last_clean":"2024-09-10T16:11:44.312044+0000","last_became_active":"2024-09-10T16:09:08.854941+0000","last_became_peered":"2024-09-10T16:09:08.854941+0000","last_unstale":"2024-09-10T16:11:44.312044+0000","last_undegraded":"2024-09-10T16:11:44.312044+0000","last_fullsized":"2024-09-10T16:11:44.312044+0000","mapping_epoch":50,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:13:14.846691+0000","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17414,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4257,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":144,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.311936+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:11:44.311936+0000","last_peered":"2024-09-10T16:11:44.311936+0000","last_clean":"2024-09-10T16:11:44.311936+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:11:44.311936+0000","last_undegraded":"2024-09-10T16:11:44.311936+0000","last_fullsized":"2024-09-10T16:11:44.311936+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20781","reported_seq":29574,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310450+0000","last_change":"2024-09-10T16:10:20.907889+0000","last_active":"2024-09-10T16:11:44.310450+0000","last_peered":"2024-09-10T16:11:44.310450+0000","last_clean":"2024-09-10T16:11:44.310450+0000","last_became_active":"2024-09-10T16:10:20.852065+0000","last_became_peered":"2024-09-10T16:10:20.852065+0000","last_unstale":"2024-09-10T16:11:44.310450+0000","last_undegraded":"2024-09-10T16:11:44.310450+0000","last_fullsized":"2024-09-10T16:11:44.310450+0000","mapping_epoch":56,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:34.069549+0000","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18461,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4307,"num_bytes_recovered":8675,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":120,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310865+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:11:44.310865+0000","last_peered":"2024-09-10T16:11:44.310865+0000","last_clean":"2024-09-10T16:11:44.310865+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:11:44.310865+0000","last_undegraded":"2024-09-10T16:11:44.310865+0000","last_fullsized":"2024-09-10T16:11:44.310865+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'21170","reported_seq":30059,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310126+0000","last_change":"2024-09-10T16:10:50.728349+0000","last_active":"2024-09-10T16:11:44.310126+0000","last_peered":"2024-09-10T16:11:44.310126+0000","last_clean":"2024-09-10T16:11:44.310126+0000","last_became_active":"2024-09-10T16:08:10.860438+0000","last_became_peered":"2024-09-10T16:08:10.860438+0000","last_unstale":"2024-09-10T16:11:44.310126+0000","last_undegraded":"2024-09-10T16:11:44.310126+0000","last_fullsized":"2024-09-10T16:11:44.310126+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:38:13.781380+0000","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18724,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20838","reported_seq":24826,"reported_epoch":63,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:11:56.243591+0000","last_change":"2024-09-10T16:11:42.563598+0000","last_active":"2024-09-10T16:11:56.243591+0000","last_peered":"2024-09-10T16:11:56.243591+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:11:56.243591+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2399,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17469,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1906,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2399}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310321+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:11:44.310321+0000","last_peered":"2024-09-10T16:11:44.310321+0000","last_clean":"2024-09-10T16:11:44.310321+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:11:44.310321+0000","last_undegraded":"2024-09-10T16:11:44.310321+0000","last_fullsized":"2024-09-10T16:11:44.310321+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20598","reported_seq":29204,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310762+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:11:44.310762+0000","last_peered":"2024-09-10T16:11:44.310762+0000","last_clean":"2024-09-10T16:11:44.310762+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:11:44.310762+0000","last_undegraded":"2024-09-10T16:11:44.310762+0000","last_fullsized":"2024-09-10T16:11:44.310762+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16988,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":134,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310946+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:11:44.310946+0000","last_peered":"2024-09-10T16:11:44.310946+0000","last_clean":"2024-09-10T16:11:44.310946+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:11:44.310946+0000","last_undegraded":"2024-09-10T16:11:44.310946+0000","last_fullsized":"2024-09-10T16:11:44.310946+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'21048","reported_seq":21214,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310569+0000","last_change":"2024-09-10T16:11:17.315372+0000","last_active":"2024-09-10T16:11:44.310569+0000","last_peered":"2024-09-10T16:11:44.310569+0000","last_clean":"2024-09-10T16:11:44.310569+0000","last_became_active":"2024-09-10T16:11:17.234487+0000","last_became_peered":"2024-09-10T16:11:17.234487+0000","last_unstale":"2024-09-10T16:11:44.310569+0000","last_undegraded":"2024-09-10T16:11:44.310569+0000","last_fullsized":"2024-09-10T16:11:44.310569+0000","mapping_epoch":60,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:19:00.114196+0000","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17303,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4373,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":109,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310658+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:11:44.310658+0000","last_peered":"2024-09-10T16:11:44.310658+0000","last_clean":"2024-09-10T16:11:44.310658+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:11:44.310658+0000","last_undegraded":"2024-09-10T16:11:44.310658+0000","last_fullsized":"2024-09-10T16:11:44.310658+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":114,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310839+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:11:44.310839+0000","last_peered":"2024-09-10T16:11:44.310839+0000","last_clean":"2024-09-10T16:11:44.310839+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:11:44.310839+0000","last_undegraded":"2024-09-10T16:11:44.310839+0000","last_fullsized":"2024-09-10T16:11:44.310839+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'21198","reported_seq":21365,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.396057+0000","last_change":"2024-09-10T16:11:44.391021+0000","last_active":"2024-09-10T16:11:44.396057+0000","last_peered":"2024-09-10T16:11:44.396057+0000","last_clean":"2024-09-10T16:11:44.396057+0000","last_became_active":"2024-09-10T16:11:44.312647+0000","last_became_peered":"2024-09-10T16:11:44.312647+0000","last_unstale":"2024-09-10T16:11:44.396057+0000","last_undegraded":"2024-09-10T16:11:44.396057+0000","last_fullsized":"2024-09-10T16:11:44.396057+0000","mapping_epoch":62,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:33:28.041611+0000","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17361,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4325,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":6636,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292096,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44754,"num_bytes_recovered":33097612,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":172052480,"data_stored":171958542,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161022,"ondisk_log_size":161022,"up":32,"acting":30,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986343,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":465768,"kb_used_data":100888,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93906072,"statfs":{"total":96636764160,"available":96159817728,"internally_reserved":0,"allocated":103309312,"data_stored":102928905,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659288,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":501488,"kb_used_data":544,"kb_used_omap":4,"kb_used_meta":500923,"kb_avail":93870352,"statfs":{"total":96636764160,"available":96123240448,"internally_reserved":0,"allocated":557056,"data_stored":179798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":512945838},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182571,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":547412,"kb_used_data":68732,"kb_used_omap":4,"kb_used_meta":478651,"kb_avail":93824428,"statfs":{"total":96636764160,"available":96076214272,"internally_reserved":0,"allocated":70381568,"data_stored":69993471,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":490139306},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":69853184,"data_stored":69817054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102170624,"data_stored":102138107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:12:01.202 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-10T16:12:01.202 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "49'21082", 'reported_seq': 21260, 'reported_epoch': 63, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:11:44.312026+0000', 'last_change': '2024-09-10T15:59:16.331899+0000', 'last_active': '2024-09-10T16:11:44.312026+0000', 'last_peered': '2024-09-10T16:11:44.312026+0000', 'last_clean': '2024-09-10T15:59:14.139436+0000', 'last_became_active': '2024-09-10T15:59:16.318715+0000', 'last_became_peered': '2024-09-10T15:59:16.318715+0000', 'last_unstale': '2024-09-10T16:11:44.312026+0000', 'last_undegraded': '2024-09-10T15:59:15.877524+0000', 'last_fullsized': '2024-09-10T15:59:15.769042+0000', 'mapping_epoch': 42, 'log_start': "45'11000", 'ondisk_log_start': "45'11000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3537", 'last_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'objects_scrubbed': 3497, 'log_size': 10082, 'log_dups_size': 0, 'ondisk_log_size': 10082, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1224, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4237, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 17581, 'num_write_kb': 28172, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4235}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:12:01.203 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-10T16:12:01.203 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "49'20838", 'reported_seq': 24826, 'reported_epoch': 63, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:11:56.243591+0000', 'last_change': '2024-09-10T16:11:42.563598+0000', 'last_active': '2024-09-10T16:11:56.243591+0000', 'last_peered': '2024-09-10T16:11:56.243591+0000', 'last_clean': '2024-09-10T15:59:14.117001+0000', 'last_became_active': '2024-09-10T15:59:16.302245+0000', 'last_became_peered': '2024-09-10T15:59:16.302245+0000', 'last_unstale': '2024-09-10T16:11:56.243591+0000', 'last_undegraded': '2024-09-10T15:59:15.845012+0000', 'last_fullsized': '2024-09-10T15:59:15.770337+0000', 'mapping_epoch': 42, 'log_start': "45'10800", 'ondisk_log_start': "45'10800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3404", 'last_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'objects_scrubbed': 3367, 'log_size': 10038, 'log_dups_size': 0, 'ondisk_log_size': 10038, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 983, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2399, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17469, 'num_write_kb': 28204, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1906, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2399}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:12:01.203 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T16:12:01.607 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:12:01.608 INFO:teuthology.orchestra.run.smithi115.stdout:{"fsid":"7effee29-df05-4351-9741-ee203564ed81","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 6636/24 objects degraded (27650.000%), 2 pgs degraded, 2 pgs undersized","count":4},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":885,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":63,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983954,"num_in_osds":3,"osd_in_since":1725983947,"num_remapped_pgs":2},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":22},{"state_name":"active+recovering+undersized+degraded+remapped","count":1},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":12,"data_bytes":4849,"bytes_used":1551020032,"bytes_avail":288359272448,"bytes_total":289910292480,"degraded_objects":6636,"degraded_total":24,"degraded_ratio":276.5,"recovering_objects_per_sec":110,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1325,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-10T15:57:14:967120+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T16:00:13.560776+0000","services":{}},"progress_events":{}} 2024-09-10T16:12:01.621 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-10T16:12:01.622 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:12:01.949 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:12:01.950 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:12:01.961 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":499,"stamp":"2024-09-10T16:12:01.799076+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5909,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292098,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":45483,"num_bytes_recovered":33097631,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161025,"ondisk_log_size":161025,"up":48,"acting":46,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1519388,"kb_used_data":166564,"kb_used_omap":12,"kb_used_meta":1352755,"kb_avail":281596132,"statfs":{"total":289910292480,"available":288354439168,"internally_reserved":0,"allocated":170561536,"data_stored":169415774,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1385221122},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2052,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2052,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001809"},"pg_stats":[{"pgid":"2.f","version":"49'20672","reported_seq":29313,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.314657+0000","last_change":"2024-09-10T16:09:57.088531+0000","last_active":"2024-09-10T16:11:44.314657+0000","last_peered":"2024-09-10T16:11:44.314657+0000","last_clean":"2024-09-10T16:11:44.314657+0000","last_became_active":"2024-09-10T16:09:57.005755+0000","last_became_peered":"2024-09-10T16:09:57.005755+0000","last_unstale":"2024-09-10T16:11:44.314657+0000","last_undegraded":"2024-09-10T16:11:44.314657+0000","last_fullsized":"2024-09-10T16:11:44.314657+0000","mapping_epoch":54,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:09:19.329697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20672,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4259,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20844","reported_seq":20997,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.311152+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:11:44.311152+0000","last_peered":"2024-09-10T16:11:44.311152+0000","last_clean":"2024-09-10T16:11:44.311152+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:11:44.311152+0000","last_undegraded":"2024-09-10T16:11:44.311152+0000","last_fullsized":"2024-09-10T16:11:44.311152+0000","mapping_epoch":35,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18949,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20798","reported_seq":29516,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310310+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:11:44.310310+0000","last_peered":"2024-09-10T16:11:44.310310+0000","last_clean":"2024-09-10T16:11:44.310310+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:11:44.310310+0000","last_undegraded":"2024-09-10T16:11:44.310310+0000","last_fullsized":"2024-09-10T16:11:44.310310+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18802,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20660","reported_seq":20781,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310770+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:11:44.310770+0000","last_peered":"2024-09-10T16:11:44.310770+0000","last_clean":"2024-09-10T16:11:44.310770+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:11:44.310770+0000","last_undegraded":"2024-09-10T16:11:44.310770+0000","last_fullsized":"2024-09-10T16:11:44.310770+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20660,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20891","reported_seq":21061,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.312021+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:11:44.312021+0000","last_peered":"2024-09-10T16:11:44.312021+0000","last_clean":"2024-09-10T16:11:44.312021+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:11:44.312021+0000","last_undegraded":"2024-09-10T16:11:44.312021+0000","last_fullsized":"2024-09-10T16:11:44.312021+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17602,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'21061","reported_seq":21230,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310143+0000","last_change":"2024-09-10T16:10:52.241702+0000","last_active":"2024-09-10T16:11:44.310143+0000","last_peered":"2024-09-10T16:11:44.310143+0000","last_clean":"2024-09-10T16:11:44.310143+0000","last_became_active":"2024-09-10T16:10:52.171236+0000","last_became_peered":"2024-09-10T16:10:52.171236+0000","last_unstale":"2024-09-10T16:11:44.310143+0000","last_undegraded":"2024-09-10T16:11:44.310143+0000","last_fullsized":"2024-09-10T16:11:44.310143+0000","mapping_epoch":58,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:04:46.778280+0000","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17673,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4372,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'21082","reported_seq":21260,"reported_epoch":63,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:11:44.312026+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:11:44.312026+0000","last_peered":"2024-09-10T16:11:44.312026+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:11:44.312026+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17581,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4235},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'21314","reported_seq":21484,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.311841+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:11:44.311841+0000","last_peered":"2024-09-10T16:11:44.311841+0000","last_clean":"2024-09-10T16:11:44.311841+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:11:44.311841+0000","last_undegraded":"2024-09-10T16:11:44.311841+0000","last_fullsized":"2024-09-10T16:11:44.311841+0000","mapping_epoch":34,"log_start":"45'11300","ondisk_log_start":"45'11300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18161,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.309727+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:11:44.309727+0000","last_peered":"2024-09-10T16:11:44.309727+0000","last_clean":"2024-09-10T16:11:44.309727+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:11:44.309727+0000","last_undegraded":"2024-09-10T16:11:44.309727+0000","last_fullsized":"2024-09-10T16:11:44.309727+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20457","reported_seq":20641,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310911+0000","last_change":"2024-09-10T16:09:33.012840+0000","last_active":"2024-09-10T16:11:44.310911+0000","last_peered":"2024-09-10T16:11:44.310911+0000","last_clean":"2024-09-10T16:11:44.310911+0000","last_became_active":"2024-09-10T16:09:32.934056+0000","last_became_peered":"2024-09-10T16:09:32.934056+0000","last_unstale":"2024-09-10T16:11:44.310911+0000","last_undegraded":"2024-09-10T16:11:44.310911+0000","last_fullsized":"2024-09-10T16:11:44.310911+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:08:30.481235+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18276,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":4579,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":134,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310369+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:11:44.310369+0000","last_peered":"2024-09-10T16:11:44.310369+0000","last_clean":"2024-09-10T16:11:44.310369+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:11:44.310369+0000","last_undegraded":"2024-09-10T16:11:44.310369+0000","last_fullsized":"2024-09-10T16:11:44.310369+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20910","reported_seq":29603,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.312044+0000","last_change":"2024-09-10T16:09:08.909580+0000","last_active":"2024-09-10T16:11:44.312044+0000","last_peered":"2024-09-10T16:11:44.312044+0000","last_clean":"2024-09-10T16:11:44.312044+0000","last_became_active":"2024-09-10T16:09:08.854941+0000","last_became_peered":"2024-09-10T16:09:08.854941+0000","last_unstale":"2024-09-10T16:11:44.312044+0000","last_undegraded":"2024-09-10T16:11:44.312044+0000","last_fullsized":"2024-09-10T16:11:44.312044+0000","mapping_epoch":50,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:13:14.846691+0000","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17414,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4257,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":144,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.311936+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:11:44.311936+0000","last_peered":"2024-09-10T16:11:44.311936+0000","last_clean":"2024-09-10T16:11:44.311936+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:11:44.311936+0000","last_undegraded":"2024-09-10T16:11:44.311936+0000","last_fullsized":"2024-09-10T16:11:44.311936+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20781","reported_seq":29574,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310450+0000","last_change":"2024-09-10T16:10:20.907889+0000","last_active":"2024-09-10T16:11:44.310450+0000","last_peered":"2024-09-10T16:11:44.310450+0000","last_clean":"2024-09-10T16:11:44.310450+0000","last_became_active":"2024-09-10T16:10:20.852065+0000","last_became_peered":"2024-09-10T16:10:20.852065+0000","last_unstale":"2024-09-10T16:11:44.310450+0000","last_undegraded":"2024-09-10T16:11:44.310450+0000","last_fullsized":"2024-09-10T16:11:44.310450+0000","mapping_epoch":56,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:34.069549+0000","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18461,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4307,"num_bytes_recovered":8675,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":120,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310865+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:11:44.310865+0000","last_peered":"2024-09-10T16:11:44.310865+0000","last_clean":"2024-09-10T16:11:44.310865+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:11:44.310865+0000","last_undegraded":"2024-09-10T16:11:44.310865+0000","last_fullsized":"2024-09-10T16:11:44.310865+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'21170","reported_seq":30059,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310126+0000","last_change":"2024-09-10T16:10:50.728349+0000","last_active":"2024-09-10T16:11:44.310126+0000","last_peered":"2024-09-10T16:11:44.310126+0000","last_clean":"2024-09-10T16:11:44.310126+0000","last_became_active":"2024-09-10T16:08:10.860438+0000","last_became_peered":"2024-09-10T16:08:10.860438+0000","last_unstale":"2024-09-10T16:11:44.310126+0000","last_undegraded":"2024-09-10T16:11:44.310126+0000","last_fullsized":"2024-09-10T16:11:44.310126+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:38:13.781380+0000","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18724,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20838","reported_seq":26280,"reported_epoch":63,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:12:01.241826+0000","last_change":"2024-09-10T16:11:42.563598+0000","last_active":"2024-09-10T16:12:01.241826+0000","last_peered":"2024-09-10T16:12:01.241826+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:12:01.241826+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1672,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17469,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2633,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1672}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310321+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:11:44.310321+0000","last_peered":"2024-09-10T16:11:44.310321+0000","last_clean":"2024-09-10T16:11:44.310321+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:11:44.310321+0000","last_undegraded":"2024-09-10T16:11:44.310321+0000","last_fullsized":"2024-09-10T16:11:44.310321+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20598","reported_seq":29204,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310762+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:11:44.310762+0000","last_peered":"2024-09-10T16:11:44.310762+0000","last_clean":"2024-09-10T16:11:44.310762+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:11:44.310762+0000","last_undegraded":"2024-09-10T16:11:44.310762+0000","last_fullsized":"2024-09-10T16:11:44.310762+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16988,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":134,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310946+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:11:44.310946+0000","last_peered":"2024-09-10T16:11:44.310946+0000","last_clean":"2024-09-10T16:11:44.310946+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:11:44.310946+0000","last_undegraded":"2024-09-10T16:11:44.310946+0000","last_fullsized":"2024-09-10T16:11:44.310946+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'21048","reported_seq":21214,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310569+0000","last_change":"2024-09-10T16:11:17.315372+0000","last_active":"2024-09-10T16:11:44.310569+0000","last_peered":"2024-09-10T16:11:44.310569+0000","last_clean":"2024-09-10T16:11:44.310569+0000","last_became_active":"2024-09-10T16:11:17.234487+0000","last_became_peered":"2024-09-10T16:11:17.234487+0000","last_unstale":"2024-09-10T16:11:44.310569+0000","last_undegraded":"2024-09-10T16:11:44.310569+0000","last_fullsized":"2024-09-10T16:11:44.310569+0000","mapping_epoch":60,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:19:00.114196+0000","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17303,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4373,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310689+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:11:44.310689+0000","last_peered":"2024-09-10T16:11:44.310689+0000","last_clean":"2024-09-10T16:11:44.310689+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:11:44.310689+0000","last_undegraded":"2024-09-10T16:11:44.310689+0000","last_fullsized":"2024-09-10T16:11:44.310689+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":114,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310839+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:11:44.310839+0000","last_peered":"2024-09-10T16:11:44.310839+0000","last_clean":"2024-09-10T16:11:44.310839+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:11:44.310839+0000","last_undegraded":"2024-09-10T16:11:44.310839+0000","last_fullsized":"2024-09-10T16:11:44.310839+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'21198","reported_seq":21365,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.396057+0000","last_change":"2024-09-10T16:11:44.391021+0000","last_active":"2024-09-10T16:11:44.396057+0000","last_peered":"2024-09-10T16:11:44.396057+0000","last_clean":"2024-09-10T16:11:44.396057+0000","last_became_active":"2024-09-10T16:11:44.312647+0000","last_became_peered":"2024-09-10T16:11:44.312647+0000","last_unstale":"2024-09-10T16:11:44.396057+0000","last_undegraded":"2024-09-10T16:11:44.396057+0000","last_fullsized":"2024-09-10T16:11:44.396057+0000","mapping_epoch":62,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:33:28.041611+0000","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17361,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4325,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5909,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292096,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":45481,"num_bytes_recovered":33097612,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":168366080,"data_stored":168272142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161022,"ondisk_log_size":161022,"up":32,"acting":30,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986344,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":462168,"kb_used_data":97288,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93909672,"statfs":{"total":96636764160,"available":96163504128,"internally_reserved":0,"allocated":99622912,"data_stored":99242505,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659289,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":509808,"kb_used_data":544,"kb_used_omap":4,"kb_used_meta":509243,"kb_avail":93862032,"statfs":{"total":96636764160,"available":96114720768,"internally_reserved":0,"allocated":557056,"data_stored":179798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":521465518},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182572,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":547412,"kb_used_data":68732,"kb_used_omap":4,"kb_used_meta":478651,"kb_avail":93824428,"statfs":{"total":96636764160,"available":96076214272,"internally_reserved":0,"allocated":70381568,"data_stored":69993471,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":490139306},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":69853184,"data_stored":69817054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98484224,"data_stored":98451707,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:12:04.964 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:12:05.314 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:12:05.315 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:12:05.326 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":500,"stamp":"2024-09-10T16:12:03.799403+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5909,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292098,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":45483,"num_bytes_recovered":33097631,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161025,"ondisk_log_size":161025,"up":48,"acting":46,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1519388,"kb_used_data":166564,"kb_used_omap":12,"kb_used_meta":1352755,"kb_avail":281596132,"statfs":{"total":289910292480,"available":288354439168,"internally_reserved":0,"allocated":170561536,"data_stored":169415774,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1385221122},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1351,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1351,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001864"},"pg_stats":[{"pgid":"2.f","version":"49'20672","reported_seq":29313,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.314657+0000","last_change":"2024-09-10T16:09:57.088531+0000","last_active":"2024-09-10T16:11:44.314657+0000","last_peered":"2024-09-10T16:11:44.314657+0000","last_clean":"2024-09-10T16:11:44.314657+0000","last_became_active":"2024-09-10T16:09:57.005755+0000","last_became_peered":"2024-09-10T16:09:57.005755+0000","last_unstale":"2024-09-10T16:11:44.314657+0000","last_undegraded":"2024-09-10T16:11:44.314657+0000","last_fullsized":"2024-09-10T16:11:44.314657+0000","mapping_epoch":54,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:09:19.329697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20672,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4259,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20844","reported_seq":20997,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.311152+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:11:44.311152+0000","last_peered":"2024-09-10T16:11:44.311152+0000","last_clean":"2024-09-10T16:11:44.311152+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:11:44.311152+0000","last_undegraded":"2024-09-10T16:11:44.311152+0000","last_fullsized":"2024-09-10T16:11:44.311152+0000","mapping_epoch":35,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18949,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20798","reported_seq":29516,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310310+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:11:44.310310+0000","last_peered":"2024-09-10T16:11:44.310310+0000","last_clean":"2024-09-10T16:11:44.310310+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:11:44.310310+0000","last_undegraded":"2024-09-10T16:11:44.310310+0000","last_fullsized":"2024-09-10T16:11:44.310310+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18802,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20660","reported_seq":20781,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310770+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:11:44.310770+0000","last_peered":"2024-09-10T16:11:44.310770+0000","last_clean":"2024-09-10T16:11:44.310770+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:11:44.310770+0000","last_undegraded":"2024-09-10T16:11:44.310770+0000","last_fullsized":"2024-09-10T16:11:44.310770+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20660,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20891","reported_seq":21061,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.312021+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:11:44.312021+0000","last_peered":"2024-09-10T16:11:44.312021+0000","last_clean":"2024-09-10T16:11:44.312021+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:11:44.312021+0000","last_undegraded":"2024-09-10T16:11:44.312021+0000","last_fullsized":"2024-09-10T16:11:44.312021+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17602,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'21061","reported_seq":21230,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310143+0000","last_change":"2024-09-10T16:10:52.241702+0000","last_active":"2024-09-10T16:11:44.310143+0000","last_peered":"2024-09-10T16:11:44.310143+0000","last_clean":"2024-09-10T16:11:44.310143+0000","last_became_active":"2024-09-10T16:10:52.171236+0000","last_became_peered":"2024-09-10T16:10:52.171236+0000","last_unstale":"2024-09-10T16:11:44.310143+0000","last_undegraded":"2024-09-10T16:11:44.310143+0000","last_fullsized":"2024-09-10T16:11:44.310143+0000","mapping_epoch":58,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:04:46.778280+0000","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17673,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4372,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'21082","reported_seq":21260,"reported_epoch":63,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:11:44.312026+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:11:44.312026+0000","last_peered":"2024-09-10T16:11:44.312026+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:11:44.312026+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17581,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4235},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'21314","reported_seq":21484,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.311841+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:11:44.311841+0000","last_peered":"2024-09-10T16:11:44.311841+0000","last_clean":"2024-09-10T16:11:44.311841+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:11:44.311841+0000","last_undegraded":"2024-09-10T16:11:44.311841+0000","last_fullsized":"2024-09-10T16:11:44.311841+0000","mapping_epoch":34,"log_start":"45'11300","ondisk_log_start":"45'11300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18161,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.309727+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:11:44.309727+0000","last_peered":"2024-09-10T16:11:44.309727+0000","last_clean":"2024-09-10T16:11:44.309727+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:11:44.309727+0000","last_undegraded":"2024-09-10T16:11:44.309727+0000","last_fullsized":"2024-09-10T16:11:44.309727+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20457","reported_seq":20641,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310911+0000","last_change":"2024-09-10T16:09:33.012840+0000","last_active":"2024-09-10T16:11:44.310911+0000","last_peered":"2024-09-10T16:11:44.310911+0000","last_clean":"2024-09-10T16:11:44.310911+0000","last_became_active":"2024-09-10T16:09:32.934056+0000","last_became_peered":"2024-09-10T16:09:32.934056+0000","last_unstale":"2024-09-10T16:11:44.310911+0000","last_undegraded":"2024-09-10T16:11:44.310911+0000","last_fullsized":"2024-09-10T16:11:44.310911+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:08:30.481235+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18276,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":4579,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":134,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310369+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:11:44.310369+0000","last_peered":"2024-09-10T16:11:44.310369+0000","last_clean":"2024-09-10T16:11:44.310369+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:11:44.310369+0000","last_undegraded":"2024-09-10T16:11:44.310369+0000","last_fullsized":"2024-09-10T16:11:44.310369+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20910","reported_seq":29603,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.312044+0000","last_change":"2024-09-10T16:09:08.909580+0000","last_active":"2024-09-10T16:11:44.312044+0000","last_peered":"2024-09-10T16:11:44.312044+0000","last_clean":"2024-09-10T16:11:44.312044+0000","last_became_active":"2024-09-10T16:09:08.854941+0000","last_became_peered":"2024-09-10T16:09:08.854941+0000","last_unstale":"2024-09-10T16:11:44.312044+0000","last_undegraded":"2024-09-10T16:11:44.312044+0000","last_fullsized":"2024-09-10T16:11:44.312044+0000","mapping_epoch":50,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:13:14.846691+0000","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17414,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4257,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":144,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.311936+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:11:44.311936+0000","last_peered":"2024-09-10T16:11:44.311936+0000","last_clean":"2024-09-10T16:11:44.311936+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:11:44.311936+0000","last_undegraded":"2024-09-10T16:11:44.311936+0000","last_fullsized":"2024-09-10T16:11:44.311936+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20781","reported_seq":29574,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310450+0000","last_change":"2024-09-10T16:10:20.907889+0000","last_active":"2024-09-10T16:11:44.310450+0000","last_peered":"2024-09-10T16:11:44.310450+0000","last_clean":"2024-09-10T16:11:44.310450+0000","last_became_active":"2024-09-10T16:10:20.852065+0000","last_became_peered":"2024-09-10T16:10:20.852065+0000","last_unstale":"2024-09-10T16:11:44.310450+0000","last_undegraded":"2024-09-10T16:11:44.310450+0000","last_fullsized":"2024-09-10T16:11:44.310450+0000","mapping_epoch":56,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:34.069549+0000","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18461,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4307,"num_bytes_recovered":8675,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":120,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310865+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:11:44.310865+0000","last_peered":"2024-09-10T16:11:44.310865+0000","last_clean":"2024-09-10T16:11:44.310865+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:11:44.310865+0000","last_undegraded":"2024-09-10T16:11:44.310865+0000","last_fullsized":"2024-09-10T16:11:44.310865+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'21170","reported_seq":30059,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310126+0000","last_change":"2024-09-10T16:10:50.728349+0000","last_active":"2024-09-10T16:11:44.310126+0000","last_peered":"2024-09-10T16:11:44.310126+0000","last_clean":"2024-09-10T16:11:44.310126+0000","last_became_active":"2024-09-10T16:08:10.860438+0000","last_became_peered":"2024-09-10T16:08:10.860438+0000","last_unstale":"2024-09-10T16:11:44.310126+0000","last_undegraded":"2024-09-10T16:11:44.310126+0000","last_fullsized":"2024-09-10T16:11:44.310126+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:38:13.781380+0000","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18724,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20838","reported_seq":26280,"reported_epoch":63,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:12:01.241826+0000","last_change":"2024-09-10T16:11:42.563598+0000","last_active":"2024-09-10T16:12:01.241826+0000","last_peered":"2024-09-10T16:12:01.241826+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:12:01.241826+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1672,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17469,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2633,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1672}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310321+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:11:44.310321+0000","last_peered":"2024-09-10T16:11:44.310321+0000","last_clean":"2024-09-10T16:11:44.310321+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:11:44.310321+0000","last_undegraded":"2024-09-10T16:11:44.310321+0000","last_fullsized":"2024-09-10T16:11:44.310321+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20598","reported_seq":29204,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310762+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:11:44.310762+0000","last_peered":"2024-09-10T16:11:44.310762+0000","last_clean":"2024-09-10T16:11:44.310762+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:11:44.310762+0000","last_undegraded":"2024-09-10T16:11:44.310762+0000","last_fullsized":"2024-09-10T16:11:44.310762+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16988,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":134,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310946+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:11:44.310946+0000","last_peered":"2024-09-10T16:11:44.310946+0000","last_clean":"2024-09-10T16:11:44.310946+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:11:44.310946+0000","last_undegraded":"2024-09-10T16:11:44.310946+0000","last_fullsized":"2024-09-10T16:11:44.310946+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'21048","reported_seq":21214,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310569+0000","last_change":"2024-09-10T16:11:17.315372+0000","last_active":"2024-09-10T16:11:44.310569+0000","last_peered":"2024-09-10T16:11:44.310569+0000","last_clean":"2024-09-10T16:11:44.310569+0000","last_became_active":"2024-09-10T16:11:17.234487+0000","last_became_peered":"2024-09-10T16:11:17.234487+0000","last_unstale":"2024-09-10T16:11:44.310569+0000","last_undegraded":"2024-09-10T16:11:44.310569+0000","last_fullsized":"2024-09-10T16:11:44.310569+0000","mapping_epoch":60,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:19:00.114196+0000","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17303,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4373,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310689+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:11:44.310689+0000","last_peered":"2024-09-10T16:11:44.310689+0000","last_clean":"2024-09-10T16:11:44.310689+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:11:44.310689+0000","last_undegraded":"2024-09-10T16:11:44.310689+0000","last_fullsized":"2024-09-10T16:11:44.310689+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":114,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310839+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:11:44.310839+0000","last_peered":"2024-09-10T16:11:44.310839+0000","last_clean":"2024-09-10T16:11:44.310839+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:11:44.310839+0000","last_undegraded":"2024-09-10T16:11:44.310839+0000","last_fullsized":"2024-09-10T16:11:44.310839+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'21198","reported_seq":21365,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.396057+0000","last_change":"2024-09-10T16:11:44.391021+0000","last_active":"2024-09-10T16:11:44.396057+0000","last_peered":"2024-09-10T16:11:44.396057+0000","last_clean":"2024-09-10T16:11:44.396057+0000","last_became_active":"2024-09-10T16:11:44.312647+0000","last_became_peered":"2024-09-10T16:11:44.312647+0000","last_unstale":"2024-09-10T16:11:44.396057+0000","last_undegraded":"2024-09-10T16:11:44.396057+0000","last_fullsized":"2024-09-10T16:11:44.396057+0000","mapping_epoch":62,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:33:28.041611+0000","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17361,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4325,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5909,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292096,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":45481,"num_bytes_recovered":33097612,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":168366080,"data_stored":168272142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161022,"ondisk_log_size":161022,"up":32,"acting":30,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986344,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":462168,"kb_used_data":97288,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93909672,"statfs":{"total":96636764160,"available":96163504128,"internally_reserved":0,"allocated":99622912,"data_stored":99242505,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659289,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":509808,"kb_used_data":544,"kb_used_omap":4,"kb_used_meta":509243,"kb_avail":93862032,"statfs":{"total":96636764160,"available":96114720768,"internally_reserved":0,"allocated":557056,"data_stored":179798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":521465518},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182572,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":547412,"kb_used_data":68732,"kb_used_omap":4,"kb_used_meta":478651,"kb_avail":93824428,"statfs":{"total":96636764160,"available":96076214272,"internally_reserved":0,"allocated":70381568,"data_stored":69993471,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":490139306},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":69853184,"data_stored":69817054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98484224,"data_stored":98451707,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:12:05.328 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:12:05.630 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:12:05.630 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:12:05.643 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":500,"stamp":"2024-09-10T16:12:03.799403+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5909,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292098,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":45483,"num_bytes_recovered":33097631,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161025,"ondisk_log_size":161025,"up":48,"acting":46,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1519388,"kb_used_data":166564,"kb_used_omap":12,"kb_used_meta":1352755,"kb_avail":281596132,"statfs":{"total":289910292480,"available":288354439168,"internally_reserved":0,"allocated":170561536,"data_stored":169415774,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1385221122},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1351,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1351,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001864"},"pg_stats":[{"pgid":"2.f","version":"49'20672","reported_seq":29313,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.314657+0000","last_change":"2024-09-10T16:09:57.088531+0000","last_active":"2024-09-10T16:11:44.314657+0000","last_peered":"2024-09-10T16:11:44.314657+0000","last_clean":"2024-09-10T16:11:44.314657+0000","last_became_active":"2024-09-10T16:09:57.005755+0000","last_became_peered":"2024-09-10T16:09:57.005755+0000","last_unstale":"2024-09-10T16:11:44.314657+0000","last_undegraded":"2024-09-10T16:11:44.314657+0000","last_fullsized":"2024-09-10T16:11:44.314657+0000","mapping_epoch":54,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:09:19.329697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20672,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4259,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20844","reported_seq":20997,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.311152+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:11:44.311152+0000","last_peered":"2024-09-10T16:11:44.311152+0000","last_clean":"2024-09-10T16:11:44.311152+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:11:44.311152+0000","last_undegraded":"2024-09-10T16:11:44.311152+0000","last_fullsized":"2024-09-10T16:11:44.311152+0000","mapping_epoch":35,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18949,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20798","reported_seq":29516,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310310+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:11:44.310310+0000","last_peered":"2024-09-10T16:11:44.310310+0000","last_clean":"2024-09-10T16:11:44.310310+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:11:44.310310+0000","last_undegraded":"2024-09-10T16:11:44.310310+0000","last_fullsized":"2024-09-10T16:11:44.310310+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18802,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20660","reported_seq":20781,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310770+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:11:44.310770+0000","last_peered":"2024-09-10T16:11:44.310770+0000","last_clean":"2024-09-10T16:11:44.310770+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:11:44.310770+0000","last_undegraded":"2024-09-10T16:11:44.310770+0000","last_fullsized":"2024-09-10T16:11:44.310770+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20660,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20891","reported_seq":21061,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.312021+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:11:44.312021+0000","last_peered":"2024-09-10T16:11:44.312021+0000","last_clean":"2024-09-10T16:11:44.312021+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:11:44.312021+0000","last_undegraded":"2024-09-10T16:11:44.312021+0000","last_fullsized":"2024-09-10T16:11:44.312021+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17602,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'21061","reported_seq":21230,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310143+0000","last_change":"2024-09-10T16:10:52.241702+0000","last_active":"2024-09-10T16:11:44.310143+0000","last_peered":"2024-09-10T16:11:44.310143+0000","last_clean":"2024-09-10T16:11:44.310143+0000","last_became_active":"2024-09-10T16:10:52.171236+0000","last_became_peered":"2024-09-10T16:10:52.171236+0000","last_unstale":"2024-09-10T16:11:44.310143+0000","last_undegraded":"2024-09-10T16:11:44.310143+0000","last_fullsized":"2024-09-10T16:11:44.310143+0000","mapping_epoch":58,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:04:46.778280+0000","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17673,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4372,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'21082","reported_seq":21260,"reported_epoch":63,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:11:44.312026+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:11:44.312026+0000","last_peered":"2024-09-10T16:11:44.312026+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:11:44.312026+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17581,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4235},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'21314","reported_seq":21484,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.311841+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:11:44.311841+0000","last_peered":"2024-09-10T16:11:44.311841+0000","last_clean":"2024-09-10T16:11:44.311841+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:11:44.311841+0000","last_undegraded":"2024-09-10T16:11:44.311841+0000","last_fullsized":"2024-09-10T16:11:44.311841+0000","mapping_epoch":34,"log_start":"45'11300","ondisk_log_start":"45'11300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18161,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.309727+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:11:44.309727+0000","last_peered":"2024-09-10T16:11:44.309727+0000","last_clean":"2024-09-10T16:11:44.309727+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:11:44.309727+0000","last_undegraded":"2024-09-10T16:11:44.309727+0000","last_fullsized":"2024-09-10T16:11:44.309727+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20457","reported_seq":20641,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310911+0000","last_change":"2024-09-10T16:09:33.012840+0000","last_active":"2024-09-10T16:11:44.310911+0000","last_peered":"2024-09-10T16:11:44.310911+0000","last_clean":"2024-09-10T16:11:44.310911+0000","last_became_active":"2024-09-10T16:09:32.934056+0000","last_became_peered":"2024-09-10T16:09:32.934056+0000","last_unstale":"2024-09-10T16:11:44.310911+0000","last_undegraded":"2024-09-10T16:11:44.310911+0000","last_fullsized":"2024-09-10T16:11:44.310911+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:08:30.481235+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18276,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":4579,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":134,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310369+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:11:44.310369+0000","last_peered":"2024-09-10T16:11:44.310369+0000","last_clean":"2024-09-10T16:11:44.310369+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:11:44.310369+0000","last_undegraded":"2024-09-10T16:11:44.310369+0000","last_fullsized":"2024-09-10T16:11:44.310369+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20910","reported_seq":29603,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.312044+0000","last_change":"2024-09-10T16:09:08.909580+0000","last_active":"2024-09-10T16:11:44.312044+0000","last_peered":"2024-09-10T16:11:44.312044+0000","last_clean":"2024-09-10T16:11:44.312044+0000","last_became_active":"2024-09-10T16:09:08.854941+0000","last_became_peered":"2024-09-10T16:09:08.854941+0000","last_unstale":"2024-09-10T16:11:44.312044+0000","last_undegraded":"2024-09-10T16:11:44.312044+0000","last_fullsized":"2024-09-10T16:11:44.312044+0000","mapping_epoch":50,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:13:14.846691+0000","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17414,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4257,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":144,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.311936+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:11:44.311936+0000","last_peered":"2024-09-10T16:11:44.311936+0000","last_clean":"2024-09-10T16:11:44.311936+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:11:44.311936+0000","last_undegraded":"2024-09-10T16:11:44.311936+0000","last_fullsized":"2024-09-10T16:11:44.311936+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20781","reported_seq":29574,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310450+0000","last_change":"2024-09-10T16:10:20.907889+0000","last_active":"2024-09-10T16:11:44.310450+0000","last_peered":"2024-09-10T16:11:44.310450+0000","last_clean":"2024-09-10T16:11:44.310450+0000","last_became_active":"2024-09-10T16:10:20.852065+0000","last_became_peered":"2024-09-10T16:10:20.852065+0000","last_unstale":"2024-09-10T16:11:44.310450+0000","last_undegraded":"2024-09-10T16:11:44.310450+0000","last_fullsized":"2024-09-10T16:11:44.310450+0000","mapping_epoch":56,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:34.069549+0000","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18461,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4307,"num_bytes_recovered":8675,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":120,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310865+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:11:44.310865+0000","last_peered":"2024-09-10T16:11:44.310865+0000","last_clean":"2024-09-10T16:11:44.310865+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:11:44.310865+0000","last_undegraded":"2024-09-10T16:11:44.310865+0000","last_fullsized":"2024-09-10T16:11:44.310865+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'21170","reported_seq":30059,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310126+0000","last_change":"2024-09-10T16:10:50.728349+0000","last_active":"2024-09-10T16:11:44.310126+0000","last_peered":"2024-09-10T16:11:44.310126+0000","last_clean":"2024-09-10T16:11:44.310126+0000","last_became_active":"2024-09-10T16:08:10.860438+0000","last_became_peered":"2024-09-10T16:08:10.860438+0000","last_unstale":"2024-09-10T16:11:44.310126+0000","last_undegraded":"2024-09-10T16:11:44.310126+0000","last_fullsized":"2024-09-10T16:11:44.310126+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:38:13.781380+0000","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18724,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20838","reported_seq":26280,"reported_epoch":63,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:12:01.241826+0000","last_change":"2024-09-10T16:11:42.563598+0000","last_active":"2024-09-10T16:12:01.241826+0000","last_peered":"2024-09-10T16:12:01.241826+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:12:01.241826+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1672,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17469,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2633,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1672}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310321+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:11:44.310321+0000","last_peered":"2024-09-10T16:11:44.310321+0000","last_clean":"2024-09-10T16:11:44.310321+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:11:44.310321+0000","last_undegraded":"2024-09-10T16:11:44.310321+0000","last_fullsized":"2024-09-10T16:11:44.310321+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20598","reported_seq":29204,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310762+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:11:44.310762+0000","last_peered":"2024-09-10T16:11:44.310762+0000","last_clean":"2024-09-10T16:11:44.310762+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:11:44.310762+0000","last_undegraded":"2024-09-10T16:11:44.310762+0000","last_fullsized":"2024-09-10T16:11:44.310762+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16988,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":134,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310946+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:11:44.310946+0000","last_peered":"2024-09-10T16:11:44.310946+0000","last_clean":"2024-09-10T16:11:44.310946+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:11:44.310946+0000","last_undegraded":"2024-09-10T16:11:44.310946+0000","last_fullsized":"2024-09-10T16:11:44.310946+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'21048","reported_seq":21214,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310569+0000","last_change":"2024-09-10T16:11:17.315372+0000","last_active":"2024-09-10T16:11:44.310569+0000","last_peered":"2024-09-10T16:11:44.310569+0000","last_clean":"2024-09-10T16:11:44.310569+0000","last_became_active":"2024-09-10T16:11:17.234487+0000","last_became_peered":"2024-09-10T16:11:17.234487+0000","last_unstale":"2024-09-10T16:11:44.310569+0000","last_undegraded":"2024-09-10T16:11:44.310569+0000","last_fullsized":"2024-09-10T16:11:44.310569+0000","mapping_epoch":60,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:19:00.114196+0000","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17303,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4373,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310689+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:11:44.310689+0000","last_peered":"2024-09-10T16:11:44.310689+0000","last_clean":"2024-09-10T16:11:44.310689+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:11:44.310689+0000","last_undegraded":"2024-09-10T16:11:44.310689+0000","last_fullsized":"2024-09-10T16:11:44.310689+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":114,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310839+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:11:44.310839+0000","last_peered":"2024-09-10T16:11:44.310839+0000","last_clean":"2024-09-10T16:11:44.310839+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:11:44.310839+0000","last_undegraded":"2024-09-10T16:11:44.310839+0000","last_fullsized":"2024-09-10T16:11:44.310839+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'21198","reported_seq":21365,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.396057+0000","last_change":"2024-09-10T16:11:44.391021+0000","last_active":"2024-09-10T16:11:44.396057+0000","last_peered":"2024-09-10T16:11:44.396057+0000","last_clean":"2024-09-10T16:11:44.396057+0000","last_became_active":"2024-09-10T16:11:44.312647+0000","last_became_peered":"2024-09-10T16:11:44.312647+0000","last_unstale":"2024-09-10T16:11:44.396057+0000","last_undegraded":"2024-09-10T16:11:44.396057+0000","last_fullsized":"2024-09-10T16:11:44.396057+0000","mapping_epoch":62,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:33:28.041611+0000","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17361,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4325,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5909,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292096,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":45481,"num_bytes_recovered":33097612,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":168366080,"data_stored":168272142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161022,"ondisk_log_size":161022,"up":32,"acting":30,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986344,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":462168,"kb_used_data":97288,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93909672,"statfs":{"total":96636764160,"available":96163504128,"internally_reserved":0,"allocated":99622912,"data_stored":99242505,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659289,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":509808,"kb_used_data":544,"kb_used_omap":4,"kb_used_meta":509243,"kb_avail":93862032,"statfs":{"total":96636764160,"available":96114720768,"internally_reserved":0,"allocated":557056,"data_stored":179798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":521465518},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182572,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":547412,"kb_used_data":68732,"kb_used_omap":4,"kb_used_meta":478651,"kb_avail":93824428,"statfs":{"total":96636764160,"available":96076214272,"internally_reserved":0,"allocated":70381568,"data_stored":69993471,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":490139306},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":69853184,"data_stored":69817054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98484224,"data_stored":98451707,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:12:05.644 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-10T16:12:05.644 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "49'21082", 'reported_seq': 21260, 'reported_epoch': 63, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:11:44.312026+0000', 'last_change': '2024-09-10T15:59:16.331899+0000', 'last_active': '2024-09-10T16:11:44.312026+0000', 'last_peered': '2024-09-10T16:11:44.312026+0000', 'last_clean': '2024-09-10T15:59:14.139436+0000', 'last_became_active': '2024-09-10T15:59:16.318715+0000', 'last_became_peered': '2024-09-10T15:59:16.318715+0000', 'last_unstale': '2024-09-10T16:11:44.312026+0000', 'last_undegraded': '2024-09-10T15:59:15.877524+0000', 'last_fullsized': '2024-09-10T15:59:15.769042+0000', 'mapping_epoch': 42, 'log_start': "45'11000", 'ondisk_log_start': "45'11000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3537", 'last_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'objects_scrubbed': 3497, 'log_size': 10082, 'log_dups_size': 0, 'ondisk_log_size': 10082, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1224, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4237, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 17581, 'num_write_kb': 28172, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4235}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:12:05.644 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-10T16:12:05.644 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "49'20838", 'reported_seq': 26280, 'reported_epoch': 63, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:12:01.241826+0000', 'last_change': '2024-09-10T16:11:42.563598+0000', 'last_active': '2024-09-10T16:12:01.241826+0000', 'last_peered': '2024-09-10T16:12:01.241826+0000', 'last_clean': '2024-09-10T15:59:14.117001+0000', 'last_became_active': '2024-09-10T15:59:16.302245+0000', 'last_became_peered': '2024-09-10T15:59:16.302245+0000', 'last_unstale': '2024-09-10T16:12:01.241826+0000', 'last_undegraded': '2024-09-10T15:59:15.845012+0000', 'last_fullsized': '2024-09-10T15:59:15.770337+0000', 'mapping_epoch': 42, 'log_start': "45'10800", 'ondisk_log_start': "45'10800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3404", 'last_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'objects_scrubbed': 3367, 'log_size': 10038, 'log_dups_size': 0, 'ondisk_log_size': 10038, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 983, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1672, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17469, 'num_write_kb': 28204, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2633, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1672}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:12:05.645 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T16:12:06.021 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:12:06.021 INFO:teuthology.orchestra.run.smithi115.stdout:{"fsid":"7effee29-df05-4351-9741-ee203564ed81","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 5909/24 objects degraded (24620.833%), 2 pgs degraded, 2 pgs undersized","count":4},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":890,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":63,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983954,"num_in_osds":3,"osd_in_since":1725983947,"num_remapped_pgs":2},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":22},{"state_name":"active+recovering+undersized+degraded+remapped","count":1},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":12,"data_bytes":4849,"bytes_used":1555853312,"bytes_avail":288354439168,"bytes_total":289910292480,"degraded_objects":5909,"degraded_total":24,"degraded_ratio":246.20833333333337,"recovering_objects_per_sec":112,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1351,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-10T15:57:14:967120+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T16:00:13.560776+0000","services":{}},"progress_events":{}} 2024-09-10T16:12:06.036 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-10T16:12:06.036 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:12:06.412 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:12:06.412 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:12:06.424 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":501,"stamp":"2024-09-10T16:12:05.799755+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5909,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292098,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":45483,"num_bytes_recovered":33097631,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161025,"ondisk_log_size":161025,"up":48,"acting":46,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1506908,"kb_used_data":166564,"kb_used_omap":12,"kb_used_meta":1340275,"kb_avail":281608612,"statfs":{"total":289910292480,"available":288367218688,"internally_reserved":0,"allocated":170561536,"data_stored":169415774,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1372441602},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1351,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1351,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001887"},"pg_stats":[{"pgid":"2.f","version":"49'20672","reported_seq":29313,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.314657+0000","last_change":"2024-09-10T16:09:57.088531+0000","last_active":"2024-09-10T16:11:44.314657+0000","last_peered":"2024-09-10T16:11:44.314657+0000","last_clean":"2024-09-10T16:11:44.314657+0000","last_became_active":"2024-09-10T16:09:57.005755+0000","last_became_peered":"2024-09-10T16:09:57.005755+0000","last_unstale":"2024-09-10T16:11:44.314657+0000","last_undegraded":"2024-09-10T16:11:44.314657+0000","last_fullsized":"2024-09-10T16:11:44.314657+0000","mapping_epoch":54,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:09:19.329697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20672,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4259,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20844","reported_seq":20997,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.311152+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:11:44.311152+0000","last_peered":"2024-09-10T16:11:44.311152+0000","last_clean":"2024-09-10T16:11:44.311152+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:11:44.311152+0000","last_undegraded":"2024-09-10T16:11:44.311152+0000","last_fullsized":"2024-09-10T16:11:44.311152+0000","mapping_epoch":35,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18949,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20798","reported_seq":29516,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310310+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:11:44.310310+0000","last_peered":"2024-09-10T16:11:44.310310+0000","last_clean":"2024-09-10T16:11:44.310310+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:11:44.310310+0000","last_undegraded":"2024-09-10T16:11:44.310310+0000","last_fullsized":"2024-09-10T16:11:44.310310+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18802,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20660","reported_seq":20781,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310770+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:11:44.310770+0000","last_peered":"2024-09-10T16:11:44.310770+0000","last_clean":"2024-09-10T16:11:44.310770+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:11:44.310770+0000","last_undegraded":"2024-09-10T16:11:44.310770+0000","last_fullsized":"2024-09-10T16:11:44.310770+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20660,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20891","reported_seq":21061,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.312021+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:11:44.312021+0000","last_peered":"2024-09-10T16:11:44.312021+0000","last_clean":"2024-09-10T16:11:44.312021+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:11:44.312021+0000","last_undegraded":"2024-09-10T16:11:44.312021+0000","last_fullsized":"2024-09-10T16:11:44.312021+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17602,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'21061","reported_seq":21230,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310143+0000","last_change":"2024-09-10T16:10:52.241702+0000","last_active":"2024-09-10T16:11:44.310143+0000","last_peered":"2024-09-10T16:11:44.310143+0000","last_clean":"2024-09-10T16:11:44.310143+0000","last_became_active":"2024-09-10T16:10:52.171236+0000","last_became_peered":"2024-09-10T16:10:52.171236+0000","last_unstale":"2024-09-10T16:11:44.310143+0000","last_undegraded":"2024-09-10T16:11:44.310143+0000","last_fullsized":"2024-09-10T16:11:44.310143+0000","mapping_epoch":58,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:04:46.778280+0000","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17673,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4372,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'21082","reported_seq":21260,"reported_epoch":63,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:11:44.312026+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:11:44.312026+0000","last_peered":"2024-09-10T16:11:44.312026+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:11:44.312026+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17581,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4235},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'21314","reported_seq":21484,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.311841+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:11:44.311841+0000","last_peered":"2024-09-10T16:11:44.311841+0000","last_clean":"2024-09-10T16:11:44.311841+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:11:44.311841+0000","last_undegraded":"2024-09-10T16:11:44.311841+0000","last_fullsized":"2024-09-10T16:11:44.311841+0000","mapping_epoch":34,"log_start":"45'11300","ondisk_log_start":"45'11300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18161,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.309727+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:11:44.309727+0000","last_peered":"2024-09-10T16:11:44.309727+0000","last_clean":"2024-09-10T16:11:44.309727+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:11:44.309727+0000","last_undegraded":"2024-09-10T16:11:44.309727+0000","last_fullsized":"2024-09-10T16:11:44.309727+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20457","reported_seq":20641,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310911+0000","last_change":"2024-09-10T16:09:33.012840+0000","last_active":"2024-09-10T16:11:44.310911+0000","last_peered":"2024-09-10T16:11:44.310911+0000","last_clean":"2024-09-10T16:11:44.310911+0000","last_became_active":"2024-09-10T16:09:32.934056+0000","last_became_peered":"2024-09-10T16:09:32.934056+0000","last_unstale":"2024-09-10T16:11:44.310911+0000","last_undegraded":"2024-09-10T16:11:44.310911+0000","last_fullsized":"2024-09-10T16:11:44.310911+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:08:30.481235+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18276,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":4579,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":134,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310369+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:11:44.310369+0000","last_peered":"2024-09-10T16:11:44.310369+0000","last_clean":"2024-09-10T16:11:44.310369+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:11:44.310369+0000","last_undegraded":"2024-09-10T16:11:44.310369+0000","last_fullsized":"2024-09-10T16:11:44.310369+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20910","reported_seq":29603,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.312044+0000","last_change":"2024-09-10T16:09:08.909580+0000","last_active":"2024-09-10T16:11:44.312044+0000","last_peered":"2024-09-10T16:11:44.312044+0000","last_clean":"2024-09-10T16:11:44.312044+0000","last_became_active":"2024-09-10T16:09:08.854941+0000","last_became_peered":"2024-09-10T16:09:08.854941+0000","last_unstale":"2024-09-10T16:11:44.312044+0000","last_undegraded":"2024-09-10T16:11:44.312044+0000","last_fullsized":"2024-09-10T16:11:44.312044+0000","mapping_epoch":50,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:13:14.846691+0000","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17414,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4257,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":144,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.311936+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:11:44.311936+0000","last_peered":"2024-09-10T16:11:44.311936+0000","last_clean":"2024-09-10T16:11:44.311936+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:11:44.311936+0000","last_undegraded":"2024-09-10T16:11:44.311936+0000","last_fullsized":"2024-09-10T16:11:44.311936+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20781","reported_seq":29574,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310450+0000","last_change":"2024-09-10T16:10:20.907889+0000","last_active":"2024-09-10T16:11:44.310450+0000","last_peered":"2024-09-10T16:11:44.310450+0000","last_clean":"2024-09-10T16:11:44.310450+0000","last_became_active":"2024-09-10T16:10:20.852065+0000","last_became_peered":"2024-09-10T16:10:20.852065+0000","last_unstale":"2024-09-10T16:11:44.310450+0000","last_undegraded":"2024-09-10T16:11:44.310450+0000","last_fullsized":"2024-09-10T16:11:44.310450+0000","mapping_epoch":56,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:34.069549+0000","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18461,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4307,"num_bytes_recovered":8675,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":120,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310865+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:11:44.310865+0000","last_peered":"2024-09-10T16:11:44.310865+0000","last_clean":"2024-09-10T16:11:44.310865+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:11:44.310865+0000","last_undegraded":"2024-09-10T16:11:44.310865+0000","last_fullsized":"2024-09-10T16:11:44.310865+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'21170","reported_seq":30059,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310126+0000","last_change":"2024-09-10T16:10:50.728349+0000","last_active":"2024-09-10T16:11:44.310126+0000","last_peered":"2024-09-10T16:11:44.310126+0000","last_clean":"2024-09-10T16:11:44.310126+0000","last_became_active":"2024-09-10T16:08:10.860438+0000","last_became_peered":"2024-09-10T16:08:10.860438+0000","last_unstale":"2024-09-10T16:11:44.310126+0000","last_undegraded":"2024-09-10T16:11:44.310126+0000","last_fullsized":"2024-09-10T16:11:44.310126+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:38:13.781380+0000","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18724,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20838","reported_seq":26280,"reported_epoch":63,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:12:01.241826+0000","last_change":"2024-09-10T16:11:42.563598+0000","last_active":"2024-09-10T16:12:01.241826+0000","last_peered":"2024-09-10T16:12:01.241826+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:12:01.241826+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1672,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17469,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2633,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1672}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310321+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:11:44.310321+0000","last_peered":"2024-09-10T16:11:44.310321+0000","last_clean":"2024-09-10T16:11:44.310321+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:11:44.310321+0000","last_undegraded":"2024-09-10T16:11:44.310321+0000","last_fullsized":"2024-09-10T16:11:44.310321+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20598","reported_seq":29204,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310762+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:11:44.310762+0000","last_peered":"2024-09-10T16:11:44.310762+0000","last_clean":"2024-09-10T16:11:44.310762+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:11:44.310762+0000","last_undegraded":"2024-09-10T16:11:44.310762+0000","last_fullsized":"2024-09-10T16:11:44.310762+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16988,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":134,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310946+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:11:44.310946+0000","last_peered":"2024-09-10T16:11:44.310946+0000","last_clean":"2024-09-10T16:11:44.310946+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:11:44.310946+0000","last_undegraded":"2024-09-10T16:11:44.310946+0000","last_fullsized":"2024-09-10T16:11:44.310946+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'21048","reported_seq":21214,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310569+0000","last_change":"2024-09-10T16:11:17.315372+0000","last_active":"2024-09-10T16:11:44.310569+0000","last_peered":"2024-09-10T16:11:44.310569+0000","last_clean":"2024-09-10T16:11:44.310569+0000","last_became_active":"2024-09-10T16:11:17.234487+0000","last_became_peered":"2024-09-10T16:11:17.234487+0000","last_unstale":"2024-09-10T16:11:44.310569+0000","last_undegraded":"2024-09-10T16:11:44.310569+0000","last_fullsized":"2024-09-10T16:11:44.310569+0000","mapping_epoch":60,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:19:00.114196+0000","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17303,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4373,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310689+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:11:44.310689+0000","last_peered":"2024-09-10T16:11:44.310689+0000","last_clean":"2024-09-10T16:11:44.310689+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:11:44.310689+0000","last_undegraded":"2024-09-10T16:11:44.310689+0000","last_fullsized":"2024-09-10T16:11:44.310689+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":114,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310839+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:11:44.310839+0000","last_peered":"2024-09-10T16:11:44.310839+0000","last_clean":"2024-09-10T16:11:44.310839+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:11:44.310839+0000","last_undegraded":"2024-09-10T16:11:44.310839+0000","last_fullsized":"2024-09-10T16:11:44.310839+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'21198","reported_seq":21365,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.396057+0000","last_change":"2024-09-10T16:11:44.391021+0000","last_active":"2024-09-10T16:11:44.396057+0000","last_peered":"2024-09-10T16:11:44.396057+0000","last_clean":"2024-09-10T16:11:44.396057+0000","last_became_active":"2024-09-10T16:11:44.312647+0000","last_became_peered":"2024-09-10T16:11:44.312647+0000","last_unstale":"2024-09-10T16:11:44.396057+0000","last_undegraded":"2024-09-10T16:11:44.396057+0000","last_fullsized":"2024-09-10T16:11:44.396057+0000","mapping_epoch":62,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:33:28.041611+0000","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17361,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4325,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5909,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292096,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":45481,"num_bytes_recovered":33097612,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":168366080,"data_stored":168272142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161022,"ondisk_log_size":161022,"up":32,"acting":30,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986344,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":462168,"kb_used_data":97288,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93909672,"statfs":{"total":96636764160,"available":96163504128,"internally_reserved":0,"allocated":99622912,"data_stored":99242505,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659290,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":497328,"kb_used_data":544,"kb_used_omap":4,"kb_used_meta":496763,"kb_avail":93874512,"statfs":{"total":96636764160,"available":96127500288,"internally_reserved":0,"allocated":557056,"data_stored":179798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":508685998},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182572,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":547412,"kb_used_data":68732,"kb_used_omap":4,"kb_used_meta":478651,"kb_avail":93824428,"statfs":{"total":96636764160,"available":96076214272,"internally_reserved":0,"allocated":70381568,"data_stored":69993471,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":490139306},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":69853184,"data_stored":69817054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98484224,"data_stored":98451707,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:12:09.426 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:12:09.758 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:12:09.758 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:12:09.770 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":502,"stamp":"2024-09-10T16:12:07.800130+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4997,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292098,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":46395,"num_bytes_recovered":33097631,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161025,"ondisk_log_size":161025,"up":48,"acting":46,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1503428,"kb_used_data":163084,"kb_used_omap":12,"kb_used_meta":1340275,"kb_avail":281612092,"statfs":{"total":289910292480,"available":288370782208,"internally_reserved":0,"allocated":166998016,"data_stored":165852254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1372441602},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2263,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2263,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001964"},"pg_stats":[{"pgid":"2.f","version":"49'20672","reported_seq":29313,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.314657+0000","last_change":"2024-09-10T16:09:57.088531+0000","last_active":"2024-09-10T16:11:44.314657+0000","last_peered":"2024-09-10T16:11:44.314657+0000","last_clean":"2024-09-10T16:11:44.314657+0000","last_became_active":"2024-09-10T16:09:57.005755+0000","last_became_peered":"2024-09-10T16:09:57.005755+0000","last_unstale":"2024-09-10T16:11:44.314657+0000","last_undegraded":"2024-09-10T16:11:44.314657+0000","last_fullsized":"2024-09-10T16:11:44.314657+0000","mapping_epoch":54,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:09:19.329697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20672,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4259,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20844","reported_seq":20997,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.311152+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:11:44.311152+0000","last_peered":"2024-09-10T16:11:44.311152+0000","last_clean":"2024-09-10T16:11:44.311152+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:11:44.311152+0000","last_undegraded":"2024-09-10T16:11:44.311152+0000","last_fullsized":"2024-09-10T16:11:44.311152+0000","mapping_epoch":35,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18949,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20798","reported_seq":29516,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310310+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:11:44.310310+0000","last_peered":"2024-09-10T16:11:44.310310+0000","last_clean":"2024-09-10T16:11:44.310310+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:11:44.310310+0000","last_undegraded":"2024-09-10T16:11:44.310310+0000","last_fullsized":"2024-09-10T16:11:44.310310+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18802,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20660","reported_seq":20781,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310770+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:11:44.310770+0000","last_peered":"2024-09-10T16:11:44.310770+0000","last_clean":"2024-09-10T16:11:44.310770+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:11:44.310770+0000","last_undegraded":"2024-09-10T16:11:44.310770+0000","last_fullsized":"2024-09-10T16:11:44.310770+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20660,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20891","reported_seq":21061,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.312021+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:11:44.312021+0000","last_peered":"2024-09-10T16:11:44.312021+0000","last_clean":"2024-09-10T16:11:44.312021+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:11:44.312021+0000","last_undegraded":"2024-09-10T16:11:44.312021+0000","last_fullsized":"2024-09-10T16:11:44.312021+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17602,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'21061","reported_seq":21230,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310143+0000","last_change":"2024-09-10T16:10:52.241702+0000","last_active":"2024-09-10T16:11:44.310143+0000","last_peered":"2024-09-10T16:11:44.310143+0000","last_clean":"2024-09-10T16:11:44.310143+0000","last_became_active":"2024-09-10T16:10:52.171236+0000","last_became_peered":"2024-09-10T16:10:52.171236+0000","last_unstale":"2024-09-10T16:11:44.310143+0000","last_undegraded":"2024-09-10T16:11:44.310143+0000","last_fullsized":"2024-09-10T16:11:44.310143+0000","mapping_epoch":58,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:04:46.778280+0000","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17673,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4372,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'21082","reported_seq":21260,"reported_epoch":63,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:11:44.312026+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:11:44.312026+0000","last_peered":"2024-09-10T16:11:44.312026+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:11:44.312026+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17581,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4235},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'21314","reported_seq":21484,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.311841+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:11:44.311841+0000","last_peered":"2024-09-10T16:11:44.311841+0000","last_clean":"2024-09-10T16:11:44.311841+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:11:44.311841+0000","last_undegraded":"2024-09-10T16:11:44.311841+0000","last_fullsized":"2024-09-10T16:11:44.311841+0000","mapping_epoch":34,"log_start":"45'11300","ondisk_log_start":"45'11300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18161,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.309727+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:11:44.309727+0000","last_peered":"2024-09-10T16:11:44.309727+0000","last_clean":"2024-09-10T16:11:44.309727+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:11:44.309727+0000","last_undegraded":"2024-09-10T16:11:44.309727+0000","last_fullsized":"2024-09-10T16:11:44.309727+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20457","reported_seq":20641,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310911+0000","last_change":"2024-09-10T16:09:33.012840+0000","last_active":"2024-09-10T16:11:44.310911+0000","last_peered":"2024-09-10T16:11:44.310911+0000","last_clean":"2024-09-10T16:11:44.310911+0000","last_became_active":"2024-09-10T16:09:32.934056+0000","last_became_peered":"2024-09-10T16:09:32.934056+0000","last_unstale":"2024-09-10T16:11:44.310911+0000","last_undegraded":"2024-09-10T16:11:44.310911+0000","last_fullsized":"2024-09-10T16:11:44.310911+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:08:30.481235+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18276,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":4579,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":134,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310369+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:11:44.310369+0000","last_peered":"2024-09-10T16:11:44.310369+0000","last_clean":"2024-09-10T16:11:44.310369+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:11:44.310369+0000","last_undegraded":"2024-09-10T16:11:44.310369+0000","last_fullsized":"2024-09-10T16:11:44.310369+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20910","reported_seq":29603,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.312044+0000","last_change":"2024-09-10T16:09:08.909580+0000","last_active":"2024-09-10T16:11:44.312044+0000","last_peered":"2024-09-10T16:11:44.312044+0000","last_clean":"2024-09-10T16:11:44.312044+0000","last_became_active":"2024-09-10T16:09:08.854941+0000","last_became_peered":"2024-09-10T16:09:08.854941+0000","last_unstale":"2024-09-10T16:11:44.312044+0000","last_undegraded":"2024-09-10T16:11:44.312044+0000","last_fullsized":"2024-09-10T16:11:44.312044+0000","mapping_epoch":50,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:13:14.846691+0000","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17414,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4257,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":144,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.311936+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:11:44.311936+0000","last_peered":"2024-09-10T16:11:44.311936+0000","last_clean":"2024-09-10T16:11:44.311936+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:11:44.311936+0000","last_undegraded":"2024-09-10T16:11:44.311936+0000","last_fullsized":"2024-09-10T16:11:44.311936+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20781","reported_seq":29574,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310450+0000","last_change":"2024-09-10T16:10:20.907889+0000","last_active":"2024-09-10T16:11:44.310450+0000","last_peered":"2024-09-10T16:11:44.310450+0000","last_clean":"2024-09-10T16:11:44.310450+0000","last_became_active":"2024-09-10T16:10:20.852065+0000","last_became_peered":"2024-09-10T16:10:20.852065+0000","last_unstale":"2024-09-10T16:11:44.310450+0000","last_undegraded":"2024-09-10T16:11:44.310450+0000","last_fullsized":"2024-09-10T16:11:44.310450+0000","mapping_epoch":56,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:34.069549+0000","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18461,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4307,"num_bytes_recovered":8675,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":120,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310865+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:11:44.310865+0000","last_peered":"2024-09-10T16:11:44.310865+0000","last_clean":"2024-09-10T16:11:44.310865+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:11:44.310865+0000","last_undegraded":"2024-09-10T16:11:44.310865+0000","last_fullsized":"2024-09-10T16:11:44.310865+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'21170","reported_seq":30059,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310126+0000","last_change":"2024-09-10T16:10:50.728349+0000","last_active":"2024-09-10T16:11:44.310126+0000","last_peered":"2024-09-10T16:11:44.310126+0000","last_clean":"2024-09-10T16:11:44.310126+0000","last_became_active":"2024-09-10T16:08:10.860438+0000","last_became_peered":"2024-09-10T16:08:10.860438+0000","last_unstale":"2024-09-10T16:11:44.310126+0000","last_undegraded":"2024-09-10T16:11:44.310126+0000","last_fullsized":"2024-09-10T16:11:44.310126+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:38:13.781380+0000","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18724,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20838","reported_seq":28104,"reported_epoch":63,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:12:06.248712+0000","last_change":"2024-09-10T16:11:42.563598+0000","last_active":"2024-09-10T16:12:06.248712+0000","last_peered":"2024-09-10T16:12:06.248712+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:12:06.248712+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":760,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17469,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3545,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-760}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310321+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:11:44.310321+0000","last_peered":"2024-09-10T16:11:44.310321+0000","last_clean":"2024-09-10T16:11:44.310321+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:11:44.310321+0000","last_undegraded":"2024-09-10T16:11:44.310321+0000","last_fullsized":"2024-09-10T16:11:44.310321+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20598","reported_seq":29204,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310762+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:11:44.310762+0000","last_peered":"2024-09-10T16:11:44.310762+0000","last_clean":"2024-09-10T16:11:44.310762+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:11:44.310762+0000","last_undegraded":"2024-09-10T16:11:44.310762+0000","last_fullsized":"2024-09-10T16:11:44.310762+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16988,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":134,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310946+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:11:44.310946+0000","last_peered":"2024-09-10T16:11:44.310946+0000","last_clean":"2024-09-10T16:11:44.310946+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:11:44.310946+0000","last_undegraded":"2024-09-10T16:11:44.310946+0000","last_fullsized":"2024-09-10T16:11:44.310946+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'21048","reported_seq":21214,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310569+0000","last_change":"2024-09-10T16:11:17.315372+0000","last_active":"2024-09-10T16:11:44.310569+0000","last_peered":"2024-09-10T16:11:44.310569+0000","last_clean":"2024-09-10T16:11:44.310569+0000","last_became_active":"2024-09-10T16:11:17.234487+0000","last_became_peered":"2024-09-10T16:11:17.234487+0000","last_unstale":"2024-09-10T16:11:44.310569+0000","last_undegraded":"2024-09-10T16:11:44.310569+0000","last_fullsized":"2024-09-10T16:11:44.310569+0000","mapping_epoch":60,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:19:00.114196+0000","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17303,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4373,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310689+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:11:44.310689+0000","last_peered":"2024-09-10T16:11:44.310689+0000","last_clean":"2024-09-10T16:11:44.310689+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:11:44.310689+0000","last_undegraded":"2024-09-10T16:11:44.310689+0000","last_fullsized":"2024-09-10T16:11:44.310689+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":114,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310839+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:11:44.310839+0000","last_peered":"2024-09-10T16:11:44.310839+0000","last_clean":"2024-09-10T16:11:44.310839+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:11:44.310839+0000","last_undegraded":"2024-09-10T16:11:44.310839+0000","last_fullsized":"2024-09-10T16:11:44.310839+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'21198","reported_seq":21365,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.396057+0000","last_change":"2024-09-10T16:11:44.391021+0000","last_active":"2024-09-10T16:11:44.396057+0000","last_peered":"2024-09-10T16:11:44.396057+0000","last_clean":"2024-09-10T16:11:44.396057+0000","last_became_active":"2024-09-10T16:11:44.312647+0000","last_became_peered":"2024-09-10T16:11:44.312647+0000","last_unstale":"2024-09-10T16:11:44.396057+0000","last_undegraded":"2024-09-10T16:11:44.396057+0000","last_fullsized":"2024-09-10T16:11:44.396057+0000","mapping_epoch":62,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:33:28.041611+0000","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17361,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4325,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4997,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292096,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":46393,"num_bytes_recovered":33097612,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":164679680,"data_stored":164585742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161022,"ondisk_log_size":161022,"up":32,"acting":30,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986345,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":458688,"kb_used_data":93808,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93913152,"statfs":{"total":96636764160,"available":96167067648,"internally_reserved":0,"allocated":96059392,"data_stored":95678985,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659290,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":497328,"kb_used_data":544,"kb_used_omap":4,"kb_used_meta":496763,"kb_avail":93874512,"statfs":{"total":96636764160,"available":96127500288,"internally_reserved":0,"allocated":557056,"data_stored":179798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":508685998},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182573,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":547412,"kb_used_data":68732,"kb_used_omap":4,"kb_used_meta":478651,"kb_avail":93824428,"statfs":{"total":96636764160,"available":96076214272,"internally_reserved":0,"allocated":70381568,"data_stored":69993471,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":490139306},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":69853184,"data_stored":69817054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94797824,"data_stored":94765307,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:12:09.772 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:12:10.106 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:12:10.107 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:12:10.121 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":504,"stamp":"2024-09-10T16:12:09.856063+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4997,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292098,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":46395,"num_bytes_recovered":33097631,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161025,"ondisk_log_size":161025,"up":48,"acting":46,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1503428,"kb_used_data":163084,"kb_used_omap":12,"kb_used_meta":1340275,"kb_avail":281612092,"statfs":{"total":289910292480,"available":288370782208,"internally_reserved":0,"allocated":166998016,"data_stored":165852254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1372441602},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1639,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1639,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.057265"},"pg_stats":[{"pgid":"2.f","version":"49'20672","reported_seq":29313,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.314657+0000","last_change":"2024-09-10T16:09:57.088531+0000","last_active":"2024-09-10T16:11:44.314657+0000","last_peered":"2024-09-10T16:11:44.314657+0000","last_clean":"2024-09-10T16:11:44.314657+0000","last_became_active":"2024-09-10T16:09:57.005755+0000","last_became_peered":"2024-09-10T16:09:57.005755+0000","last_unstale":"2024-09-10T16:11:44.314657+0000","last_undegraded":"2024-09-10T16:11:44.314657+0000","last_fullsized":"2024-09-10T16:11:44.314657+0000","mapping_epoch":54,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:09:19.329697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20672,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4259,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20844","reported_seq":20997,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.311152+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:11:44.311152+0000","last_peered":"2024-09-10T16:11:44.311152+0000","last_clean":"2024-09-10T16:11:44.311152+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:11:44.311152+0000","last_undegraded":"2024-09-10T16:11:44.311152+0000","last_fullsized":"2024-09-10T16:11:44.311152+0000","mapping_epoch":35,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18949,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20798","reported_seq":29516,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310310+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:11:44.310310+0000","last_peered":"2024-09-10T16:11:44.310310+0000","last_clean":"2024-09-10T16:11:44.310310+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:11:44.310310+0000","last_undegraded":"2024-09-10T16:11:44.310310+0000","last_fullsized":"2024-09-10T16:11:44.310310+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18802,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20660","reported_seq":20781,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310770+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:11:44.310770+0000","last_peered":"2024-09-10T16:11:44.310770+0000","last_clean":"2024-09-10T16:11:44.310770+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:11:44.310770+0000","last_undegraded":"2024-09-10T16:11:44.310770+0000","last_fullsized":"2024-09-10T16:11:44.310770+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20660,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20891","reported_seq":21061,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.312021+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:11:44.312021+0000","last_peered":"2024-09-10T16:11:44.312021+0000","last_clean":"2024-09-10T16:11:44.312021+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:11:44.312021+0000","last_undegraded":"2024-09-10T16:11:44.312021+0000","last_fullsized":"2024-09-10T16:11:44.312021+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17602,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'21061","reported_seq":21230,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310143+0000","last_change":"2024-09-10T16:10:52.241702+0000","last_active":"2024-09-10T16:11:44.310143+0000","last_peered":"2024-09-10T16:11:44.310143+0000","last_clean":"2024-09-10T16:11:44.310143+0000","last_became_active":"2024-09-10T16:10:52.171236+0000","last_became_peered":"2024-09-10T16:10:52.171236+0000","last_unstale":"2024-09-10T16:11:44.310143+0000","last_undegraded":"2024-09-10T16:11:44.310143+0000","last_fullsized":"2024-09-10T16:11:44.310143+0000","mapping_epoch":58,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:04:46.778280+0000","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17673,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4372,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'21082","reported_seq":21260,"reported_epoch":63,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:11:44.312026+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:11:44.312026+0000","last_peered":"2024-09-10T16:11:44.312026+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:11:44.312026+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17581,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4235},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'21314","reported_seq":21484,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.311841+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:11:44.311841+0000","last_peered":"2024-09-10T16:11:44.311841+0000","last_clean":"2024-09-10T16:11:44.311841+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:11:44.311841+0000","last_undegraded":"2024-09-10T16:11:44.311841+0000","last_fullsized":"2024-09-10T16:11:44.311841+0000","mapping_epoch":34,"log_start":"45'11300","ondisk_log_start":"45'11300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18161,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.309727+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:11:44.309727+0000","last_peered":"2024-09-10T16:11:44.309727+0000","last_clean":"2024-09-10T16:11:44.309727+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:11:44.309727+0000","last_undegraded":"2024-09-10T16:11:44.309727+0000","last_fullsized":"2024-09-10T16:11:44.309727+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20457","reported_seq":20641,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310911+0000","last_change":"2024-09-10T16:09:33.012840+0000","last_active":"2024-09-10T16:11:44.310911+0000","last_peered":"2024-09-10T16:11:44.310911+0000","last_clean":"2024-09-10T16:11:44.310911+0000","last_became_active":"2024-09-10T16:09:32.934056+0000","last_became_peered":"2024-09-10T16:09:32.934056+0000","last_unstale":"2024-09-10T16:11:44.310911+0000","last_undegraded":"2024-09-10T16:11:44.310911+0000","last_fullsized":"2024-09-10T16:11:44.310911+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:08:30.481235+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18276,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":4579,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":134,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310369+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:11:44.310369+0000","last_peered":"2024-09-10T16:11:44.310369+0000","last_clean":"2024-09-10T16:11:44.310369+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:11:44.310369+0000","last_undegraded":"2024-09-10T16:11:44.310369+0000","last_fullsized":"2024-09-10T16:11:44.310369+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20910","reported_seq":29603,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.312044+0000","last_change":"2024-09-10T16:09:08.909580+0000","last_active":"2024-09-10T16:11:44.312044+0000","last_peered":"2024-09-10T16:11:44.312044+0000","last_clean":"2024-09-10T16:11:44.312044+0000","last_became_active":"2024-09-10T16:09:08.854941+0000","last_became_peered":"2024-09-10T16:09:08.854941+0000","last_unstale":"2024-09-10T16:11:44.312044+0000","last_undegraded":"2024-09-10T16:11:44.312044+0000","last_fullsized":"2024-09-10T16:11:44.312044+0000","mapping_epoch":50,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:13:14.846691+0000","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17414,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4257,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":144,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.311936+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:11:44.311936+0000","last_peered":"2024-09-10T16:11:44.311936+0000","last_clean":"2024-09-10T16:11:44.311936+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:11:44.311936+0000","last_undegraded":"2024-09-10T16:11:44.311936+0000","last_fullsized":"2024-09-10T16:11:44.311936+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20781","reported_seq":29574,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310450+0000","last_change":"2024-09-10T16:10:20.907889+0000","last_active":"2024-09-10T16:11:44.310450+0000","last_peered":"2024-09-10T16:11:44.310450+0000","last_clean":"2024-09-10T16:11:44.310450+0000","last_became_active":"2024-09-10T16:10:20.852065+0000","last_became_peered":"2024-09-10T16:10:20.852065+0000","last_unstale":"2024-09-10T16:11:44.310450+0000","last_undegraded":"2024-09-10T16:11:44.310450+0000","last_fullsized":"2024-09-10T16:11:44.310450+0000","mapping_epoch":56,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:34.069549+0000","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18461,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4307,"num_bytes_recovered":8675,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":120,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310865+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:11:44.310865+0000","last_peered":"2024-09-10T16:11:44.310865+0000","last_clean":"2024-09-10T16:11:44.310865+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:11:44.310865+0000","last_undegraded":"2024-09-10T16:11:44.310865+0000","last_fullsized":"2024-09-10T16:11:44.310865+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'21170","reported_seq":30059,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310126+0000","last_change":"2024-09-10T16:10:50.728349+0000","last_active":"2024-09-10T16:11:44.310126+0000","last_peered":"2024-09-10T16:11:44.310126+0000","last_clean":"2024-09-10T16:11:44.310126+0000","last_became_active":"2024-09-10T16:08:10.860438+0000","last_became_peered":"2024-09-10T16:08:10.860438+0000","last_unstale":"2024-09-10T16:11:44.310126+0000","last_undegraded":"2024-09-10T16:11:44.310126+0000","last_fullsized":"2024-09-10T16:11:44.310126+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:38:13.781380+0000","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18724,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20838","reported_seq":28104,"reported_epoch":63,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:12:06.248712+0000","last_change":"2024-09-10T16:11:42.563598+0000","last_active":"2024-09-10T16:12:06.248712+0000","last_peered":"2024-09-10T16:12:06.248712+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:12:06.248712+0000","last_undegraded":"2024-09-10T15:59:15.845012+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":760,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17469,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3545,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-760}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310321+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:11:44.310321+0000","last_peered":"2024-09-10T16:11:44.310321+0000","last_clean":"2024-09-10T16:11:44.310321+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:11:44.310321+0000","last_undegraded":"2024-09-10T16:11:44.310321+0000","last_fullsized":"2024-09-10T16:11:44.310321+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20598","reported_seq":29204,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310762+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:11:44.310762+0000","last_peered":"2024-09-10T16:11:44.310762+0000","last_clean":"2024-09-10T16:11:44.310762+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:11:44.310762+0000","last_undegraded":"2024-09-10T16:11:44.310762+0000","last_fullsized":"2024-09-10T16:11:44.310762+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16988,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":134,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310946+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:11:44.310946+0000","last_peered":"2024-09-10T16:11:44.310946+0000","last_clean":"2024-09-10T16:11:44.310946+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:11:44.310946+0000","last_undegraded":"2024-09-10T16:11:44.310946+0000","last_fullsized":"2024-09-10T16:11:44.310946+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'21048","reported_seq":21214,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310569+0000","last_change":"2024-09-10T16:11:17.315372+0000","last_active":"2024-09-10T16:11:44.310569+0000","last_peered":"2024-09-10T16:11:44.310569+0000","last_clean":"2024-09-10T16:11:44.310569+0000","last_became_active":"2024-09-10T16:11:17.234487+0000","last_became_peered":"2024-09-10T16:11:17.234487+0000","last_unstale":"2024-09-10T16:11:44.310569+0000","last_undegraded":"2024-09-10T16:11:44.310569+0000","last_fullsized":"2024-09-10T16:11:44.310569+0000","mapping_epoch":60,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:19:00.114196+0000","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17303,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4373,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310689+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:11:44.310689+0000","last_peered":"2024-09-10T16:11:44.310689+0000","last_clean":"2024-09-10T16:11:44.310689+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:11:44.310689+0000","last_undegraded":"2024-09-10T16:11:44.310689+0000","last_fullsized":"2024-09-10T16:11:44.310689+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":114,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310839+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:11:44.310839+0000","last_peered":"2024-09-10T16:11:44.310839+0000","last_clean":"2024-09-10T16:11:44.310839+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:11:44.310839+0000","last_undegraded":"2024-09-10T16:11:44.310839+0000","last_fullsized":"2024-09-10T16:11:44.310839+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'21198","reported_seq":21365,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.396057+0000","last_change":"2024-09-10T16:11:44.391021+0000","last_active":"2024-09-10T16:11:44.396057+0000","last_peered":"2024-09-10T16:11:44.396057+0000","last_clean":"2024-09-10T16:11:44.396057+0000","last_became_active":"2024-09-10T16:11:44.312647+0000","last_became_peered":"2024-09-10T16:11:44.312647+0000","last_unstale":"2024-09-10T16:11:44.396057+0000","last_undegraded":"2024-09-10T16:11:44.396057+0000","last_fullsized":"2024-09-10T16:11:44.396057+0000","mapping_epoch":62,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:33:28.041611+0000","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17361,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4325,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4997,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292096,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":46393,"num_bytes_recovered":33097612,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":164679680,"data_stored":164585742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161022,"ondisk_log_size":161022,"up":32,"acting":30,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986345,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":458688,"kb_used_data":93808,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93913152,"statfs":{"total":96636764160,"available":96167067648,"internally_reserved":0,"allocated":96059392,"data_stored":95678985,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659290,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":497328,"kb_used_data":544,"kb_used_omap":4,"kb_used_meta":496763,"kb_avail":93874512,"statfs":{"total":96636764160,"available":96127500288,"internally_reserved":0,"allocated":557056,"data_stored":179798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":508685998},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182573,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":547412,"kb_used_data":68732,"kb_used_omap":4,"kb_used_meta":478651,"kb_avail":93824428,"statfs":{"total":96636764160,"available":96076214272,"internally_reserved":0,"allocated":70381568,"data_stored":69993471,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":490139306},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":69853184,"data_stored":69817054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94797824,"data_stored":94765307,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:12:10.122 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-10T16:12:10.122 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "49'21082", 'reported_seq': 21260, 'reported_epoch': 63, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:11:44.312026+0000', 'last_change': '2024-09-10T15:59:16.331899+0000', 'last_active': '2024-09-10T16:11:44.312026+0000', 'last_peered': '2024-09-10T16:11:44.312026+0000', 'last_clean': '2024-09-10T15:59:14.139436+0000', 'last_became_active': '2024-09-10T15:59:16.318715+0000', 'last_became_peered': '2024-09-10T15:59:16.318715+0000', 'last_unstale': '2024-09-10T16:11:44.312026+0000', 'last_undegraded': '2024-09-10T15:59:15.877524+0000', 'last_fullsized': '2024-09-10T15:59:15.769042+0000', 'mapping_epoch': 42, 'log_start': "45'11000", 'ondisk_log_start': "45'11000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3537", 'last_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'objects_scrubbed': 3497, 'log_size': 10082, 'log_dups_size': 0, 'ondisk_log_size': 10082, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1224, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4237, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 17581, 'num_write_kb': 28172, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4235}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:12:10.123 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-10T16:12:10.123 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "49'20838", 'reported_seq': 28104, 'reported_epoch': 63, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:12:06.248712+0000', 'last_change': '2024-09-10T16:11:42.563598+0000', 'last_active': '2024-09-10T16:12:06.248712+0000', 'last_peered': '2024-09-10T16:12:06.248712+0000', 'last_clean': '2024-09-10T15:59:14.117001+0000', 'last_became_active': '2024-09-10T15:59:16.302245+0000', 'last_became_peered': '2024-09-10T15:59:16.302245+0000', 'last_unstale': '2024-09-10T16:12:06.248712+0000', 'last_undegraded': '2024-09-10T15:59:15.845012+0000', 'last_fullsized': '2024-09-10T15:59:15.770337+0000', 'mapping_epoch': 42, 'log_start': "45'10800", 'ondisk_log_start': "45'10800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3404", 'last_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'objects_scrubbed': 3367, 'log_size': 10038, 'log_dups_size': 0, 'ondisk_log_size': 10038, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 983, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 760, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17469, 'num_write_kb': 28204, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3545, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -760}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:12:10.123 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T16:12:10.539 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:12:10.540 INFO:teuthology.orchestra.run.smithi115.stdout:{"fsid":"7effee29-df05-4351-9741-ee203564ed81","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 4997/24 objects degraded (20820.833%), 2 pgs degraded, 2 pgs undersized","count":4},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":894,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":64,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983954,"num_in_osds":3,"osd_in_since":1725983947,"num_remapped_pgs":1},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":22},{"state_name":"active+recovering+undersized+degraded+remapped","count":1},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":12,"data_bytes":4849,"bytes_used":1539510272,"bytes_avail":288370782208,"bytes_total":289910292480,"degraded_objects":4997,"degraded_total":24,"degraded_ratio":208.20833333333337,"recovering_objects_per_sec":136,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1639,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-10T15:57:14:967120+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T16:00:13.560776+0000","services":{}},"progress_events":{}} 2024-09-10T16:12:10.553 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-10T16:12:10.553 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:12:10.895 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:12:10.895 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:12:10.909 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":505,"stamp":"2024-09-10T16:12:10.861038+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292098,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":47155,"num_bytes_recovered":33097631,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161025,"ondisk_log_size":161025,"up":48,"acting":47,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1511760,"kb_used_data":163092,"kb_used_omap":12,"kb_used_meta":1348595,"kb_avail":281603760,"statfs":{"total":289910292480,"available":288362250240,"internally_reserved":0,"allocated":167006208,"data_stored":165855006,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1380961282},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1672,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1672,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.061961"},"pg_stats":[{"pgid":"2.f","version":"49'20672","reported_seq":29313,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.314657+0000","last_change":"2024-09-10T16:09:57.088531+0000","last_active":"2024-09-10T16:11:44.314657+0000","last_peered":"2024-09-10T16:11:44.314657+0000","last_clean":"2024-09-10T16:11:44.314657+0000","last_became_active":"2024-09-10T16:09:57.005755+0000","last_became_peered":"2024-09-10T16:09:57.005755+0000","last_unstale":"2024-09-10T16:11:44.314657+0000","last_undegraded":"2024-09-10T16:11:44.314657+0000","last_fullsized":"2024-09-10T16:11:44.314657+0000","mapping_epoch":54,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:09:19.329697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20672,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4259,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20844","reported_seq":20997,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.311152+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:11:44.311152+0000","last_peered":"2024-09-10T16:11:44.311152+0000","last_clean":"2024-09-10T16:11:44.311152+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:11:44.311152+0000","last_undegraded":"2024-09-10T16:11:44.311152+0000","last_fullsized":"2024-09-10T16:11:44.311152+0000","mapping_epoch":35,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18949,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20798","reported_seq":29518,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:12:09.858753+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:12:09.858753+0000","last_peered":"2024-09-10T16:12:09.858753+0000","last_clean":"2024-09-10T16:12:09.858753+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:12:09.858753+0000","last_undegraded":"2024-09-10T16:12:09.858753+0000","last_fullsized":"2024-09-10T16:12:09.858753+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18802,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20660","reported_seq":20781,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310770+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:11:44.310770+0000","last_peered":"2024-09-10T16:11:44.310770+0000","last_clean":"2024-09-10T16:11:44.310770+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:11:44.310770+0000","last_undegraded":"2024-09-10T16:11:44.310770+0000","last_fullsized":"2024-09-10T16:11:44.310770+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20660,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20891","reported_seq":21061,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.312021+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:11:44.312021+0000","last_peered":"2024-09-10T16:11:44.312021+0000","last_clean":"2024-09-10T16:11:44.312021+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:11:44.312021+0000","last_undegraded":"2024-09-10T16:11:44.312021+0000","last_fullsized":"2024-09-10T16:11:44.312021+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17602,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'21061","reported_seq":21232,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:12:09.860233+0000","last_change":"2024-09-10T16:10:52.241702+0000","last_active":"2024-09-10T16:12:09.860233+0000","last_peered":"2024-09-10T16:12:09.860233+0000","last_clean":"2024-09-10T16:12:09.860233+0000","last_became_active":"2024-09-10T16:10:52.171236+0000","last_became_peered":"2024-09-10T16:10:52.171236+0000","last_unstale":"2024-09-10T16:12:09.860233+0000","last_undegraded":"2024-09-10T16:12:09.860233+0000","last_fullsized":"2024-09-10T16:12:09.860233+0000","mapping_epoch":58,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:04:46.778280+0000","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17673,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4372,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'21082","reported_seq":21260,"reported_epoch":63,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:11:44.312026+0000","last_change":"2024-09-10T15:59:16.331899+0000","last_active":"2024-09-10T16:11:44.312026+0000","last_peered":"2024-09-10T16:11:44.312026+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:11:44.312026+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17581,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4235},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'21314","reported_seq":21484,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.311841+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:11:44.311841+0000","last_peered":"2024-09-10T16:11:44.311841+0000","last_clean":"2024-09-10T16:11:44.311841+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:11:44.311841+0000","last_undegraded":"2024-09-10T16:11:44.311841+0000","last_fullsized":"2024-09-10T16:11:44.311841+0000","mapping_epoch":34,"log_start":"45'11300","ondisk_log_start":"45'11300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18161,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":112,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:12:09.858069+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:12:09.858069+0000","last_peered":"2024-09-10T16:12:09.858069+0000","last_clean":"2024-09-10T16:12:09.858069+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:12:09.858069+0000","last_undegraded":"2024-09-10T16:12:09.858069+0000","last_fullsized":"2024-09-10T16:12:09.858069+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20457","reported_seq":20643,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:12:09.860256+0000","last_change":"2024-09-10T16:09:33.012840+0000","last_active":"2024-09-10T16:12:09.860256+0000","last_peered":"2024-09-10T16:12:09.860256+0000","last_clean":"2024-09-10T16:12:09.860256+0000","last_became_active":"2024-09-10T16:09:32.934056+0000","last_became_peered":"2024-09-10T16:09:32.934056+0000","last_unstale":"2024-09-10T16:12:09.860256+0000","last_undegraded":"2024-09-10T16:12:09.860256+0000","last_fullsized":"2024-09-10T16:12:09.860256+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:08:30.481235+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18276,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":4579,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":134,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310369+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:11:44.310369+0000","last_peered":"2024-09-10T16:11:44.310369+0000","last_clean":"2024-09-10T16:11:44.310369+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:11:44.310369+0000","last_undegraded":"2024-09-10T16:11:44.310369+0000","last_fullsized":"2024-09-10T16:11:44.310369+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20910","reported_seq":29603,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.312044+0000","last_change":"2024-09-10T16:09:08.909580+0000","last_active":"2024-09-10T16:11:44.312044+0000","last_peered":"2024-09-10T16:11:44.312044+0000","last_clean":"2024-09-10T16:11:44.312044+0000","last_became_active":"2024-09-10T16:09:08.854941+0000","last_became_peered":"2024-09-10T16:09:08.854941+0000","last_unstale":"2024-09-10T16:11:44.312044+0000","last_undegraded":"2024-09-10T16:11:44.312044+0000","last_fullsized":"2024-09-10T16:11:44.312044+0000","mapping_epoch":50,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:13:14.846691+0000","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17414,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4257,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":144,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.311936+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:11:44.311936+0000","last_peered":"2024-09-10T16:11:44.311936+0000","last_clean":"2024-09-10T16:11:44.311936+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:11:44.311936+0000","last_undegraded":"2024-09-10T16:11:44.311936+0000","last_fullsized":"2024-09-10T16:11:44.311936+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20781","reported_seq":29574,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310450+0000","last_change":"2024-09-10T16:10:20.907889+0000","last_active":"2024-09-10T16:11:44.310450+0000","last_peered":"2024-09-10T16:11:44.310450+0000","last_clean":"2024-09-10T16:11:44.310450+0000","last_became_active":"2024-09-10T16:10:20.852065+0000","last_became_peered":"2024-09-10T16:10:20.852065+0000","last_unstale":"2024-09-10T16:11:44.310450+0000","last_undegraded":"2024-09-10T16:11:44.310450+0000","last_fullsized":"2024-09-10T16:11:44.310450+0000","mapping_epoch":56,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:34.069549+0000","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18461,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4307,"num_bytes_recovered":8675,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":122,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:12:09.858220+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:12:09.858220+0000","last_peered":"2024-09-10T16:12:09.858220+0000","last_clean":"2024-09-10T16:12:09.858220+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:12:09.858220+0000","last_undegraded":"2024-09-10T16:12:09.858220+0000","last_fullsized":"2024-09-10T16:12:09.858220+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'21170","reported_seq":30059,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310126+0000","last_change":"2024-09-10T16:10:50.728349+0000","last_active":"2024-09-10T16:11:44.310126+0000","last_peered":"2024-09-10T16:11:44.310126+0000","last_clean":"2024-09-10T16:11:44.310126+0000","last_became_active":"2024-09-10T16:08:10.860438+0000","last_became_peered":"2024-09-10T16:08:10.860438+0000","last_unstale":"2024-09-10T16:11:44.310126+0000","last_undegraded":"2024-09-10T16:11:44.310126+0000","last_fullsized":"2024-09-10T16:11:44.310126+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:38:13.781380+0000","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18724,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20838","reported_seq":20986,"reported_epoch":64,"state":"peering","last_fresh":"2024-09-10T16:12:09.399058+0000","last_change":"2024-09-10T16:12:10.285851+0000","last_active":"2024-09-10T16:12:10.285851+0000","last_peered":"2024-09-10T16:12:09.399058+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:12:09.399058+0000","last_undegraded":"2024-09-10T16:12:09.399058+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":64,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17469,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4305,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":112,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:12:09.858658+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:12:09.858658+0000","last_peered":"2024-09-10T16:12:09.858658+0000","last_clean":"2024-09-10T16:12:09.858658+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:12:09.858658+0000","last_undegraded":"2024-09-10T16:12:09.858658+0000","last_fullsized":"2024-09-10T16:12:09.858658+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20598","reported_seq":29206,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:12:09.860146+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:12:09.860146+0000","last_peered":"2024-09-10T16:12:09.860146+0000","last_clean":"2024-09-10T16:12:09.860146+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:12:09.860146+0000","last_undegraded":"2024-09-10T16:12:09.860146+0000","last_fullsized":"2024-09-10T16:12:09.860146+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16988,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":134,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:11:44.310946+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:11:44.310946+0000","last_peered":"2024-09-10T16:11:44.310946+0000","last_clean":"2024-09-10T16:11:44.310946+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:11:44.310946+0000","last_undegraded":"2024-09-10T16:11:44.310946+0000","last_fullsized":"2024-09-10T16:11:44.310946+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'21048","reported_seq":21216,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:12:09.858839+0000","last_change":"2024-09-10T16:11:17.315372+0000","last_active":"2024-09-10T16:12:09.858839+0000","last_peered":"2024-09-10T16:12:09.858839+0000","last_clean":"2024-09-10T16:12:09.858839+0000","last_became_active":"2024-09-10T16:11:17.234487+0000","last_became_peered":"2024-09-10T16:11:17.234487+0000","last_unstale":"2024-09-10T16:12:09.858839+0000","last_undegraded":"2024-09-10T16:12:09.858839+0000","last_fullsized":"2024-09-10T16:12:09.858839+0000","mapping_epoch":60,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:19:00.114196+0000","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17303,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4373,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":112,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:12:09.859682+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:12:09.859682+0000","last_peered":"2024-09-10T16:12:09.859682+0000","last_clean":"2024-09-10T16:12:09.859682+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:12:09.859682+0000","last_undegraded":"2024-09-10T16:12:09.859682+0000","last_fullsized":"2024-09-10T16:12:09.859682+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":116,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:12:09.858231+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:12:09.858231+0000","last_peered":"2024-09-10T16:12:09.858231+0000","last_clean":"2024-09-10T16:12:09.858231+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:12:09.858231+0000","last_undegraded":"2024-09-10T16:12:09.858231+0000","last_fullsized":"2024-09-10T16:12:09.858231+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'21198","reported_seq":21367,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:12:09.858570+0000","last_change":"2024-09-10T16:11:44.391021+0000","last_active":"2024-09-10T16:12:09.858570+0000","last_peered":"2024-09-10T16:12:09.858570+0000","last_clean":"2024-09-10T16:12:09.858570+0000","last_became_active":"2024-09-10T16:11:44.312647+0000","last_became_peered":"2024-09-10T16:11:44.312647+0000","last_unstale":"2024-09-10T16:12:09.858570+0000","last_undegraded":"2024-09-10T16:12:09.858570+0000","last_fullsized":"2024-09-10T16:12:09.858570+0000","mapping_epoch":62,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:33:28.041611+0000","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17361,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4325,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292096,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":47153,"num_bytes_recovered":33097612,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":164679680,"data_stored":164585742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161022,"ondisk_log_size":161022,"up":32,"acting":31,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986345,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":458688,"kb_used_data":93808,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93913152,"statfs":{"total":96636764160,"available":96167067648,"internally_reserved":0,"allocated":96059392,"data_stored":95678985,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659291,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":505660,"kb_used_data":552,"kb_used_omap":4,"kb_used_meta":505083,"kb_avail":93866180,"statfs":{"total":96636764160,"available":96118968320,"internally_reserved":0,"allocated":565248,"data_stored":182550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":517205678},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182573,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":547412,"kb_used_data":68732,"kb_used_omap":4,"kb_used_meta":478651,"kb_avail":93824428,"statfs":{"total":96636764160,"available":96076214272,"internally_reserved":0,"allocated":70381568,"data_stored":69993471,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":490139306},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":69853184,"data_stored":69817054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94797824,"data_stored":94765307,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:12:13.911 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:12:14.295 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:12:14.296 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:12:14.309 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":507,"stamp":"2024-09-10T16:12:13.801121+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3961,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292098,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":47431,"num_bytes_recovered":33098597,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161025,"ondisk_log_size":161025,"up":48,"acting":47,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1508184,"kb_used_data":159516,"kb_used_omap":12,"kb_used_meta":1348595,"kb_avail":281607336,"statfs":{"total":289910292480,"available":288365912064,"internally_reserved":0,"allocated":163344384,"data_stored":162180454,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1380961282},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1948,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1948,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001366"},"pg_stats":[{"pgid":"2.f","version":"49'20672","reported_seq":29317,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.863984+0000","last_change":"2024-09-10T16:09:57.088531+0000","last_active":"2024-09-10T16:12:10.863984+0000","last_peered":"2024-09-10T16:12:10.863984+0000","last_clean":"2024-09-10T16:12:10.863984+0000","last_became_active":"2024-09-10T16:09:57.005755+0000","last_became_peered":"2024-09-10T16:09:57.005755+0000","last_unstale":"2024-09-10T16:12:10.863984+0000","last_undegraded":"2024-09-10T16:12:10.863984+0000","last_fullsized":"2024-09-10T16:12:10.863984+0000","mapping_epoch":54,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:09:19.329697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20672,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4259,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20844","reported_seq":21001,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.863381+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:12:10.863381+0000","last_peered":"2024-09-10T16:12:10.863381+0000","last_clean":"2024-09-10T16:12:10.863381+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:12:10.863381+0000","last_undegraded":"2024-09-10T16:12:10.863381+0000","last_fullsized":"2024-09-10T16:12:10.863381+0000","mapping_epoch":35,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18949,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20798","reported_seq":29518,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:12:09.858753+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:12:09.858753+0000","last_peered":"2024-09-10T16:12:09.858753+0000","last_clean":"2024-09-10T16:12:09.858753+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:12:09.858753+0000","last_undegraded":"2024-09-10T16:12:09.858753+0000","last_fullsized":"2024-09-10T16:12:09.858753+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18802,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20660","reported_seq":20785,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.863544+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:12:10.863544+0000","last_peered":"2024-09-10T16:12:10.863544+0000","last_clean":"2024-09-10T16:12:10.863544+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:12:10.863544+0000","last_undegraded":"2024-09-10T16:12:10.863544+0000","last_fullsized":"2024-09-10T16:12:10.863544+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20660,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20891","reported_seq":21065,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.864566+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:12:10.864566+0000","last_peered":"2024-09-10T16:12:10.864566+0000","last_clean":"2024-09-10T16:12:10.864566+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:12:10.864566+0000","last_undegraded":"2024-09-10T16:12:10.864566+0000","last_fullsized":"2024-09-10T16:12:10.864566+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17602,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'21061","reported_seq":21232,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:12:09.860233+0000","last_change":"2024-09-10T16:10:52.241702+0000","last_active":"2024-09-10T16:12:09.860233+0000","last_peered":"2024-09-10T16:12:09.860233+0000","last_clean":"2024-09-10T16:12:09.860233+0000","last_became_active":"2024-09-10T16:10:52.171236+0000","last_became_peered":"2024-09-10T16:10:52.171236+0000","last_unstale":"2024-09-10T16:12:09.860233+0000","last_undegraded":"2024-09-10T16:12:09.860233+0000","last_fullsized":"2024-09-10T16:12:09.860233+0000","mapping_epoch":58,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:04:46.778280+0000","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17673,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4372,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'21082","reported_seq":21817,"reported_epoch":65,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:12:11.254733+0000","last_change":"2024-09-10T16:12:09.399531+0000","last_active":"2024-09-10T16:12:11.254733+0000","last_peered":"2024-09-10T16:12:11.254733+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:12:11.254733+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3961,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17581,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":276,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3959}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'21314","reported_seq":21488,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.862825+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:12:10.862825+0000","last_peered":"2024-09-10T16:12:10.862825+0000","last_clean":"2024-09-10T16:12:10.862825+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:12:10.862825+0000","last_undegraded":"2024-09-10T16:12:10.862825+0000","last_fullsized":"2024-09-10T16:12:10.862825+0000","mapping_epoch":34,"log_start":"45'11300","ondisk_log_start":"45'11300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18161,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":112,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:12:09.858069+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:12:09.858069+0000","last_peered":"2024-09-10T16:12:09.858069+0000","last_clean":"2024-09-10T16:12:09.858069+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:12:09.858069+0000","last_undegraded":"2024-09-10T16:12:09.858069+0000","last_fullsized":"2024-09-10T16:12:09.858069+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20457","reported_seq":20643,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:12:09.860256+0000","last_change":"2024-09-10T16:09:33.012840+0000","last_active":"2024-09-10T16:12:09.860256+0000","last_peered":"2024-09-10T16:12:09.860256+0000","last_clean":"2024-09-10T16:12:09.860256+0000","last_became_active":"2024-09-10T16:09:32.934056+0000","last_became_peered":"2024-09-10T16:09:32.934056+0000","last_unstale":"2024-09-10T16:12:09.860256+0000","last_undegraded":"2024-09-10T16:12:09.860256+0000","last_fullsized":"2024-09-10T16:12:09.860256+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:08:30.481235+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18276,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":4579,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":138,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.862711+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:12:10.862711+0000","last_peered":"2024-09-10T16:12:10.862711+0000","last_clean":"2024-09-10T16:12:10.862711+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:12:10.862711+0000","last_undegraded":"2024-09-10T16:12:10.862711+0000","last_fullsized":"2024-09-10T16:12:10.862711+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20910","reported_seq":29607,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.864300+0000","last_change":"2024-09-10T16:09:08.909580+0000","last_active":"2024-09-10T16:12:10.864300+0000","last_peered":"2024-09-10T16:12:10.864300+0000","last_clean":"2024-09-10T16:12:10.864300+0000","last_became_active":"2024-09-10T16:09:08.854941+0000","last_became_peered":"2024-09-10T16:09:08.854941+0000","last_unstale":"2024-09-10T16:12:10.864300+0000","last_undegraded":"2024-09-10T16:12:10.864300+0000","last_fullsized":"2024-09-10T16:12:10.864300+0000","mapping_epoch":50,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:13:14.846691+0000","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17414,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4257,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":148,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.864168+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:12:10.864168+0000","last_peered":"2024-09-10T16:12:10.864168+0000","last_clean":"2024-09-10T16:12:10.864168+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:12:10.864168+0000","last_undegraded":"2024-09-10T16:12:10.864168+0000","last_fullsized":"2024-09-10T16:12:10.864168+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20781","reported_seq":29578,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.862793+0000","last_change":"2024-09-10T16:10:20.907889+0000","last_active":"2024-09-10T16:12:10.862793+0000","last_peered":"2024-09-10T16:12:10.862793+0000","last_clean":"2024-09-10T16:12:10.862793+0000","last_became_active":"2024-09-10T16:10:20.852065+0000","last_became_peered":"2024-09-10T16:10:20.852065+0000","last_unstale":"2024-09-10T16:12:10.862793+0000","last_undegraded":"2024-09-10T16:12:10.862793+0000","last_fullsized":"2024-09-10T16:12:10.862793+0000","mapping_epoch":56,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:34.069549+0000","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18461,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4307,"num_bytes_recovered":8675,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":122,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:12:09.858220+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:12:09.858220+0000","last_peered":"2024-09-10T16:12:09.858220+0000","last_clean":"2024-09-10T16:12:09.858220+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:12:09.858220+0000","last_undegraded":"2024-09-10T16:12:09.858220+0000","last_fullsized":"2024-09-10T16:12:09.858220+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'21170","reported_seq":30063,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.862468+0000","last_change":"2024-09-10T16:10:50.728349+0000","last_active":"2024-09-10T16:12:10.862468+0000","last_peered":"2024-09-10T16:12:10.862468+0000","last_clean":"2024-09-10T16:12:10.862468+0000","last_became_active":"2024-09-10T16:08:10.860438+0000","last_became_peered":"2024-09-10T16:08:10.860438+0000","last_unstale":"2024-09-10T16:12:10.862468+0000","last_undegraded":"2024-09-10T16:12:10.862468+0000","last_fullsized":"2024-09-10T16:12:10.862468+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:38:13.781380+0000","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18724,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20838","reported_seq":20986,"reported_epoch":64,"state":"peering","last_fresh":"2024-09-10T16:12:09.399058+0000","last_change":"2024-09-10T16:12:10.285851+0000","last_active":"2024-09-10T16:12:10.285851+0000","last_peered":"2024-09-10T16:12:09.399058+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:12:09.399058+0000","last_undegraded":"2024-09-10T16:12:09.399058+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":64,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17469,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4305,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":112,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:12:09.858658+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:12:09.858658+0000","last_peered":"2024-09-10T16:12:09.858658+0000","last_clean":"2024-09-10T16:12:09.858658+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:12:09.858658+0000","last_undegraded":"2024-09-10T16:12:09.858658+0000","last_fullsized":"2024-09-10T16:12:09.858658+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20598","reported_seq":29206,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:12:09.860146+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:12:09.860146+0000","last_peered":"2024-09-10T16:12:09.860146+0000","last_clean":"2024-09-10T16:12:09.860146+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:12:09.860146+0000","last_undegraded":"2024-09-10T16:12:09.860146+0000","last_fullsized":"2024-09-10T16:12:09.860146+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16988,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":138,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.863216+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:12:10.863216+0000","last_peered":"2024-09-10T16:12:10.863216+0000","last_clean":"2024-09-10T16:12:10.863216+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:12:10.863216+0000","last_undegraded":"2024-09-10T16:12:10.863216+0000","last_fullsized":"2024-09-10T16:12:10.863216+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'21048","reported_seq":21216,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:12:09.858839+0000","last_change":"2024-09-10T16:11:17.315372+0000","last_active":"2024-09-10T16:12:09.858839+0000","last_peered":"2024-09-10T16:12:09.858839+0000","last_clean":"2024-09-10T16:12:09.858839+0000","last_became_active":"2024-09-10T16:11:17.234487+0000","last_became_peered":"2024-09-10T16:11:17.234487+0000","last_unstale":"2024-09-10T16:12:09.858839+0000","last_undegraded":"2024-09-10T16:12:09.858839+0000","last_fullsized":"2024-09-10T16:12:09.858839+0000","mapping_epoch":60,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:19:00.114196+0000","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17303,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4373,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":112,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:12:09.859682+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:12:09.859682+0000","last_peered":"2024-09-10T16:12:09.859682+0000","last_clean":"2024-09-10T16:12:09.859682+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:12:09.859682+0000","last_undegraded":"2024-09-10T16:12:09.859682+0000","last_fullsized":"2024-09-10T16:12:09.859682+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":116,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:12:09.858231+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:12:09.858231+0000","last_peered":"2024-09-10T16:12:09.858231+0000","last_clean":"2024-09-10T16:12:09.858231+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:12:09.858231+0000","last_undegraded":"2024-09-10T16:12:09.858231+0000","last_fullsized":"2024-09-10T16:12:09.858231+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'21198","reported_seq":21367,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:12:09.858570+0000","last_change":"2024-09-10T16:11:44.391021+0000","last_active":"2024-09-10T16:12:09.858570+0000","last_peered":"2024-09-10T16:12:09.858570+0000","last_clean":"2024-09-10T16:12:09.858570+0000","last_became_active":"2024-09-10T16:11:44.312647+0000","last_became_peered":"2024-09-10T16:11:44.312647+0000","last_unstale":"2024-09-10T16:12:09.858570+0000","last_undegraded":"2024-09-10T16:12:09.858570+0000","last_fullsized":"2024-09-10T16:12:09.858570+0000","mapping_epoch":62,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:33:28.041611+0000","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17361,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4325,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3961,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292096,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":47429,"num_bytes_recovered":33098578,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":160874496,"data_stored":160784171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161022,"ondisk_log_size":161022,"up":32,"acting":31,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986346,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":455100,"kb_used_data":90216,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93916740,"statfs":{"total":96636764160,"available":96170741760,"internally_reserved":0,"allocated":92381184,"data_stored":91998950,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659291,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":505660,"kb_used_data":552,"kb_used_omap":4,"kb_used_meta":505083,"kb_avail":93866180,"statfs":{"total":96636764160,"available":96118968320,"internally_reserved":0,"allocated":565248,"data_stored":182550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":517205678},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182574,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":547424,"kb_used_data":68748,"kb_used_omap":4,"kb_used_meta":478651,"kb_avail":93824416,"statfs":{"total":96636764160,"available":96076201984,"internally_reserved":0,"allocated":70397952,"data_stored":69998954,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":490139306},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":69853184,"data_stored":69817054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90992640,"data_stored":90963736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:12:14.310 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:12:14.646 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:12:14.647 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:12:14.659 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":507,"stamp":"2024-09-10T16:12:13.801121+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3961,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292098,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":47431,"num_bytes_recovered":33098597,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161025,"ondisk_log_size":161025,"up":48,"acting":47,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1508184,"kb_used_data":159516,"kb_used_omap":12,"kb_used_meta":1348595,"kb_avail":281607336,"statfs":{"total":289910292480,"available":288365912064,"internally_reserved":0,"allocated":163344384,"data_stored":162180454,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1380961282},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1948,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1948,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001366"},"pg_stats":[{"pgid":"2.f","version":"49'20672","reported_seq":29317,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.863984+0000","last_change":"2024-09-10T16:09:57.088531+0000","last_active":"2024-09-10T16:12:10.863984+0000","last_peered":"2024-09-10T16:12:10.863984+0000","last_clean":"2024-09-10T16:12:10.863984+0000","last_became_active":"2024-09-10T16:09:57.005755+0000","last_became_peered":"2024-09-10T16:09:57.005755+0000","last_unstale":"2024-09-10T16:12:10.863984+0000","last_undegraded":"2024-09-10T16:12:10.863984+0000","last_fullsized":"2024-09-10T16:12:10.863984+0000","mapping_epoch":54,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:09:19.329697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20672,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4259,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20844","reported_seq":21001,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.863381+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:12:10.863381+0000","last_peered":"2024-09-10T16:12:10.863381+0000","last_clean":"2024-09-10T16:12:10.863381+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:12:10.863381+0000","last_undegraded":"2024-09-10T16:12:10.863381+0000","last_fullsized":"2024-09-10T16:12:10.863381+0000","mapping_epoch":35,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18949,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20798","reported_seq":29518,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:12:09.858753+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:12:09.858753+0000","last_peered":"2024-09-10T16:12:09.858753+0000","last_clean":"2024-09-10T16:12:09.858753+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:12:09.858753+0000","last_undegraded":"2024-09-10T16:12:09.858753+0000","last_fullsized":"2024-09-10T16:12:09.858753+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18802,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20660","reported_seq":20785,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.863544+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:12:10.863544+0000","last_peered":"2024-09-10T16:12:10.863544+0000","last_clean":"2024-09-10T16:12:10.863544+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:12:10.863544+0000","last_undegraded":"2024-09-10T16:12:10.863544+0000","last_fullsized":"2024-09-10T16:12:10.863544+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20660,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20891","reported_seq":21065,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.864566+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:12:10.864566+0000","last_peered":"2024-09-10T16:12:10.864566+0000","last_clean":"2024-09-10T16:12:10.864566+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:12:10.864566+0000","last_undegraded":"2024-09-10T16:12:10.864566+0000","last_fullsized":"2024-09-10T16:12:10.864566+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17602,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'21061","reported_seq":21232,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:12:09.860233+0000","last_change":"2024-09-10T16:10:52.241702+0000","last_active":"2024-09-10T16:12:09.860233+0000","last_peered":"2024-09-10T16:12:09.860233+0000","last_clean":"2024-09-10T16:12:09.860233+0000","last_became_active":"2024-09-10T16:10:52.171236+0000","last_became_peered":"2024-09-10T16:10:52.171236+0000","last_unstale":"2024-09-10T16:12:09.860233+0000","last_undegraded":"2024-09-10T16:12:09.860233+0000","last_fullsized":"2024-09-10T16:12:09.860233+0000","mapping_epoch":58,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:04:46.778280+0000","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17673,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4372,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'21082","reported_seq":21817,"reported_epoch":65,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:12:11.254733+0000","last_change":"2024-09-10T16:12:09.399531+0000","last_active":"2024-09-10T16:12:11.254733+0000","last_peered":"2024-09-10T16:12:11.254733+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:12:11.254733+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3961,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17581,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":276,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3959}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'21314","reported_seq":21488,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.862825+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:12:10.862825+0000","last_peered":"2024-09-10T16:12:10.862825+0000","last_clean":"2024-09-10T16:12:10.862825+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:12:10.862825+0000","last_undegraded":"2024-09-10T16:12:10.862825+0000","last_fullsized":"2024-09-10T16:12:10.862825+0000","mapping_epoch":34,"log_start":"45'11300","ondisk_log_start":"45'11300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18161,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":112,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:12:09.858069+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:12:09.858069+0000","last_peered":"2024-09-10T16:12:09.858069+0000","last_clean":"2024-09-10T16:12:09.858069+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:12:09.858069+0000","last_undegraded":"2024-09-10T16:12:09.858069+0000","last_fullsized":"2024-09-10T16:12:09.858069+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20457","reported_seq":20643,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:12:09.860256+0000","last_change":"2024-09-10T16:09:33.012840+0000","last_active":"2024-09-10T16:12:09.860256+0000","last_peered":"2024-09-10T16:12:09.860256+0000","last_clean":"2024-09-10T16:12:09.860256+0000","last_became_active":"2024-09-10T16:09:32.934056+0000","last_became_peered":"2024-09-10T16:09:32.934056+0000","last_unstale":"2024-09-10T16:12:09.860256+0000","last_undegraded":"2024-09-10T16:12:09.860256+0000","last_fullsized":"2024-09-10T16:12:09.860256+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:08:30.481235+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18276,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":4579,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":138,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.862711+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:12:10.862711+0000","last_peered":"2024-09-10T16:12:10.862711+0000","last_clean":"2024-09-10T16:12:10.862711+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:12:10.862711+0000","last_undegraded":"2024-09-10T16:12:10.862711+0000","last_fullsized":"2024-09-10T16:12:10.862711+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20910","reported_seq":29607,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.864300+0000","last_change":"2024-09-10T16:09:08.909580+0000","last_active":"2024-09-10T16:12:10.864300+0000","last_peered":"2024-09-10T16:12:10.864300+0000","last_clean":"2024-09-10T16:12:10.864300+0000","last_became_active":"2024-09-10T16:09:08.854941+0000","last_became_peered":"2024-09-10T16:09:08.854941+0000","last_unstale":"2024-09-10T16:12:10.864300+0000","last_undegraded":"2024-09-10T16:12:10.864300+0000","last_fullsized":"2024-09-10T16:12:10.864300+0000","mapping_epoch":50,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:13:14.846691+0000","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17414,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4257,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":148,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.864168+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:12:10.864168+0000","last_peered":"2024-09-10T16:12:10.864168+0000","last_clean":"2024-09-10T16:12:10.864168+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:12:10.864168+0000","last_undegraded":"2024-09-10T16:12:10.864168+0000","last_fullsized":"2024-09-10T16:12:10.864168+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20781","reported_seq":29578,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.862793+0000","last_change":"2024-09-10T16:10:20.907889+0000","last_active":"2024-09-10T16:12:10.862793+0000","last_peered":"2024-09-10T16:12:10.862793+0000","last_clean":"2024-09-10T16:12:10.862793+0000","last_became_active":"2024-09-10T16:10:20.852065+0000","last_became_peered":"2024-09-10T16:10:20.852065+0000","last_unstale":"2024-09-10T16:12:10.862793+0000","last_undegraded":"2024-09-10T16:12:10.862793+0000","last_fullsized":"2024-09-10T16:12:10.862793+0000","mapping_epoch":56,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:34.069549+0000","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18461,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4307,"num_bytes_recovered":8675,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":122,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:12:09.858220+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:12:09.858220+0000","last_peered":"2024-09-10T16:12:09.858220+0000","last_clean":"2024-09-10T16:12:09.858220+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:12:09.858220+0000","last_undegraded":"2024-09-10T16:12:09.858220+0000","last_fullsized":"2024-09-10T16:12:09.858220+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'21170","reported_seq":30063,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.862468+0000","last_change":"2024-09-10T16:10:50.728349+0000","last_active":"2024-09-10T16:12:10.862468+0000","last_peered":"2024-09-10T16:12:10.862468+0000","last_clean":"2024-09-10T16:12:10.862468+0000","last_became_active":"2024-09-10T16:08:10.860438+0000","last_became_peered":"2024-09-10T16:08:10.860438+0000","last_unstale":"2024-09-10T16:12:10.862468+0000","last_undegraded":"2024-09-10T16:12:10.862468+0000","last_fullsized":"2024-09-10T16:12:10.862468+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:38:13.781380+0000","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18724,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20838","reported_seq":20986,"reported_epoch":64,"state":"peering","last_fresh":"2024-09-10T16:12:09.399058+0000","last_change":"2024-09-10T16:12:10.285851+0000","last_active":"2024-09-10T16:12:10.285851+0000","last_peered":"2024-09-10T16:12:09.399058+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:12:09.399058+0000","last_undegraded":"2024-09-10T16:12:09.399058+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":64,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17469,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4305,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":112,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:12:09.858658+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:12:09.858658+0000","last_peered":"2024-09-10T16:12:09.858658+0000","last_clean":"2024-09-10T16:12:09.858658+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:12:09.858658+0000","last_undegraded":"2024-09-10T16:12:09.858658+0000","last_fullsized":"2024-09-10T16:12:09.858658+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20598","reported_seq":29206,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:12:09.860146+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:12:09.860146+0000","last_peered":"2024-09-10T16:12:09.860146+0000","last_clean":"2024-09-10T16:12:09.860146+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:12:09.860146+0000","last_undegraded":"2024-09-10T16:12:09.860146+0000","last_fullsized":"2024-09-10T16:12:09.860146+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16988,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":138,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.863216+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:12:10.863216+0000","last_peered":"2024-09-10T16:12:10.863216+0000","last_clean":"2024-09-10T16:12:10.863216+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:12:10.863216+0000","last_undegraded":"2024-09-10T16:12:10.863216+0000","last_fullsized":"2024-09-10T16:12:10.863216+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'21048","reported_seq":21216,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:12:09.858839+0000","last_change":"2024-09-10T16:11:17.315372+0000","last_active":"2024-09-10T16:12:09.858839+0000","last_peered":"2024-09-10T16:12:09.858839+0000","last_clean":"2024-09-10T16:12:09.858839+0000","last_became_active":"2024-09-10T16:11:17.234487+0000","last_became_peered":"2024-09-10T16:11:17.234487+0000","last_unstale":"2024-09-10T16:12:09.858839+0000","last_undegraded":"2024-09-10T16:12:09.858839+0000","last_fullsized":"2024-09-10T16:12:09.858839+0000","mapping_epoch":60,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:19:00.114196+0000","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17303,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4373,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":112,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:12:09.859682+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:12:09.859682+0000","last_peered":"2024-09-10T16:12:09.859682+0000","last_clean":"2024-09-10T16:12:09.859682+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:12:09.859682+0000","last_undegraded":"2024-09-10T16:12:09.859682+0000","last_fullsized":"2024-09-10T16:12:09.859682+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":116,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:12:09.858231+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:12:09.858231+0000","last_peered":"2024-09-10T16:12:09.858231+0000","last_clean":"2024-09-10T16:12:09.858231+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:12:09.858231+0000","last_undegraded":"2024-09-10T16:12:09.858231+0000","last_fullsized":"2024-09-10T16:12:09.858231+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'21198","reported_seq":21367,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:12:09.858570+0000","last_change":"2024-09-10T16:11:44.391021+0000","last_active":"2024-09-10T16:12:09.858570+0000","last_peered":"2024-09-10T16:12:09.858570+0000","last_clean":"2024-09-10T16:12:09.858570+0000","last_became_active":"2024-09-10T16:11:44.312647+0000","last_became_peered":"2024-09-10T16:11:44.312647+0000","last_unstale":"2024-09-10T16:12:09.858570+0000","last_undegraded":"2024-09-10T16:12:09.858570+0000","last_fullsized":"2024-09-10T16:12:09.858570+0000","mapping_epoch":62,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:33:28.041611+0000","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17361,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4325,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3961,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292096,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":47429,"num_bytes_recovered":33098578,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":160874496,"data_stored":160784171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161022,"ondisk_log_size":161022,"up":32,"acting":31,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986346,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":455100,"kb_used_data":90216,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93916740,"statfs":{"total":96636764160,"available":96170741760,"internally_reserved":0,"allocated":92381184,"data_stored":91998950,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659291,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":505660,"kb_used_data":552,"kb_used_omap":4,"kb_used_meta":505083,"kb_avail":93866180,"statfs":{"total":96636764160,"available":96118968320,"internally_reserved":0,"allocated":565248,"data_stored":182550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":517205678},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182574,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":547424,"kb_used_data":68748,"kb_used_omap":4,"kb_used_meta":478651,"kb_avail":93824416,"statfs":{"total":96636764160,"available":96076201984,"internally_reserved":0,"allocated":70397952,"data_stored":69998954,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":490139306},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":69853184,"data_stored":69817054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90992640,"data_stored":90963736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:12:14.661 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-10T16:12:14.661 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "49'21082", 'reported_seq': 21817, 'reported_epoch': 65, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:12:11.254733+0000', 'last_change': '2024-09-10T16:12:09.399531+0000', 'last_active': '2024-09-10T16:12:11.254733+0000', 'last_peered': '2024-09-10T16:12:11.254733+0000', 'last_clean': '2024-09-10T15:59:14.139436+0000', 'last_became_active': '2024-09-10T15:59:16.318715+0000', 'last_became_peered': '2024-09-10T15:59:16.318715+0000', 'last_unstale': '2024-09-10T16:12:11.254733+0000', 'last_undegraded': '2024-09-10T15:59:15.877524+0000', 'last_fullsized': '2024-09-10T15:59:15.769042+0000', 'mapping_epoch': 42, 'log_start': "45'11000", 'ondisk_log_start': "45'11000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3537", 'last_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'objects_scrubbed': 3497, 'log_size': 10082, 'log_dups_size': 0, 'ondisk_log_size': 10082, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1224, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3961, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 17581, 'num_write_kb': 28172, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 276, 'num_bytes_recovered': 966, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3959}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:12:14.661 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-10T16:12:14.661 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "49'20838", 'reported_seq': 20986, 'reported_epoch': 64, 'state': 'peering', 'last_fresh': '2024-09-10T16:12:09.399058+0000', 'last_change': '2024-09-10T16:12:10.285851+0000', 'last_active': '2024-09-10T16:12:10.285851+0000', 'last_peered': '2024-09-10T16:12:09.399058+0000', 'last_clean': '2024-09-10T15:59:14.117001+0000', 'last_became_active': '2024-09-10T15:59:16.302245+0000', 'last_became_peered': '2024-09-10T15:59:16.302245+0000', 'last_unstale': '2024-09-10T16:12:09.399058+0000', 'last_undegraded': '2024-09-10T16:12:09.399058+0000', 'last_fullsized': '2024-09-10T15:59:15.770337+0000', 'mapping_epoch': 64, 'log_start': "45'10800", 'ondisk_log_start': "45'10800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3404", 'last_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:00.927592+0000', 'objects_scrubbed': 3367, 'log_size': 10038, 'log_dups_size': 0, 'ondisk_log_size': 10038, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 983, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17469, 'num_write_kb': 28204, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 4305, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-10T16:12:14.661 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T16:12:15.064 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:12:15.064 INFO:teuthology.orchestra.run.smithi115.stdout:{"fsid":"7effee29-df05-4351-9741-ee203564ed81","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 3961/24 objects degraded (16504.167%), 1 pg degraded, 1 pg undersized","count":2},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":899,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":65,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983954,"num_in_osds":3,"osd_in_since":1725983947,"num_remapped_pgs":1},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":22},{"state_name":"peering","count":1},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":12,"data_bytes":4849,"bytes_used":1544380416,"bytes_avail":288365912064,"bytes_total":289910292480,"inactive_pgs_ratio":0.041666667908430099,"degraded_objects":3961,"degraded_total":24,"degraded_ratio":165.04166666666663,"recovering_objects_per_sec":243,"recovering_bytes_per_sec":120,"recovering_keys_per_sec":0,"num_objects_recovered":1948,"num_bytes_recovered":966,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-10T15:57:14:967120+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T16:00:13.560776+0000","services":{}},"progress_events":{}} 2024-09-10T16:12:15.079 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-10T16:12:15.079 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:12:15.392 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:12:15.392 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:12:15.405 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":507,"stamp":"2024-09-10T16:12:13.801121+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3961,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292098,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":47431,"num_bytes_recovered":33098597,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161025,"ondisk_log_size":161025,"up":48,"acting":47,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1508184,"kb_used_data":159516,"kb_used_omap":12,"kb_used_meta":1348595,"kb_avail":281607336,"statfs":{"total":289910292480,"available":288365912064,"internally_reserved":0,"allocated":163344384,"data_stored":162180454,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1380961282},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1948,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1948,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001366"},"pg_stats":[{"pgid":"2.f","version":"49'20672","reported_seq":29317,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.863984+0000","last_change":"2024-09-10T16:09:57.088531+0000","last_active":"2024-09-10T16:12:10.863984+0000","last_peered":"2024-09-10T16:12:10.863984+0000","last_clean":"2024-09-10T16:12:10.863984+0000","last_became_active":"2024-09-10T16:09:57.005755+0000","last_became_peered":"2024-09-10T16:09:57.005755+0000","last_unstale":"2024-09-10T16:12:10.863984+0000","last_undegraded":"2024-09-10T16:12:10.863984+0000","last_fullsized":"2024-09-10T16:12:10.863984+0000","mapping_epoch":54,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:09:19.329697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20672,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4259,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20844","reported_seq":21001,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.863381+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:12:10.863381+0000","last_peered":"2024-09-10T16:12:10.863381+0000","last_clean":"2024-09-10T16:12:10.863381+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:12:10.863381+0000","last_undegraded":"2024-09-10T16:12:10.863381+0000","last_fullsized":"2024-09-10T16:12:10.863381+0000","mapping_epoch":35,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18949,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20798","reported_seq":29518,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:12:09.858753+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:12:09.858753+0000","last_peered":"2024-09-10T16:12:09.858753+0000","last_clean":"2024-09-10T16:12:09.858753+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:12:09.858753+0000","last_undegraded":"2024-09-10T16:12:09.858753+0000","last_fullsized":"2024-09-10T16:12:09.858753+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18802,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20660","reported_seq":20785,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.863544+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:12:10.863544+0000","last_peered":"2024-09-10T16:12:10.863544+0000","last_clean":"2024-09-10T16:12:10.863544+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:12:10.863544+0000","last_undegraded":"2024-09-10T16:12:10.863544+0000","last_fullsized":"2024-09-10T16:12:10.863544+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20660,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20891","reported_seq":21065,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.864566+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:12:10.864566+0000","last_peered":"2024-09-10T16:12:10.864566+0000","last_clean":"2024-09-10T16:12:10.864566+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:12:10.864566+0000","last_undegraded":"2024-09-10T16:12:10.864566+0000","last_fullsized":"2024-09-10T16:12:10.864566+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17602,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'21061","reported_seq":21232,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:12:09.860233+0000","last_change":"2024-09-10T16:10:52.241702+0000","last_active":"2024-09-10T16:12:09.860233+0000","last_peered":"2024-09-10T16:12:09.860233+0000","last_clean":"2024-09-10T16:12:09.860233+0000","last_became_active":"2024-09-10T16:10:52.171236+0000","last_became_peered":"2024-09-10T16:10:52.171236+0000","last_unstale":"2024-09-10T16:12:09.860233+0000","last_undegraded":"2024-09-10T16:12:09.860233+0000","last_fullsized":"2024-09-10T16:12:09.860233+0000","mapping_epoch":58,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:04:46.778280+0000","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17673,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4372,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'21082","reported_seq":21817,"reported_epoch":65,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:12:11.254733+0000","last_change":"2024-09-10T16:12:09.399531+0000","last_active":"2024-09-10T16:12:11.254733+0000","last_peered":"2024-09-10T16:12:11.254733+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:12:11.254733+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3961,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17581,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":276,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3959}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'21314","reported_seq":21488,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.862825+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:12:10.862825+0000","last_peered":"2024-09-10T16:12:10.862825+0000","last_clean":"2024-09-10T16:12:10.862825+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:12:10.862825+0000","last_undegraded":"2024-09-10T16:12:10.862825+0000","last_fullsized":"2024-09-10T16:12:10.862825+0000","mapping_epoch":34,"log_start":"45'11300","ondisk_log_start":"45'11300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18161,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":112,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:12:09.858069+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:12:09.858069+0000","last_peered":"2024-09-10T16:12:09.858069+0000","last_clean":"2024-09-10T16:12:09.858069+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:12:09.858069+0000","last_undegraded":"2024-09-10T16:12:09.858069+0000","last_fullsized":"2024-09-10T16:12:09.858069+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20457","reported_seq":20643,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:12:09.860256+0000","last_change":"2024-09-10T16:09:33.012840+0000","last_active":"2024-09-10T16:12:09.860256+0000","last_peered":"2024-09-10T16:12:09.860256+0000","last_clean":"2024-09-10T16:12:09.860256+0000","last_became_active":"2024-09-10T16:09:32.934056+0000","last_became_peered":"2024-09-10T16:09:32.934056+0000","last_unstale":"2024-09-10T16:12:09.860256+0000","last_undegraded":"2024-09-10T16:12:09.860256+0000","last_fullsized":"2024-09-10T16:12:09.860256+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:08:30.481235+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18276,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":4579,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":138,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.862711+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:12:10.862711+0000","last_peered":"2024-09-10T16:12:10.862711+0000","last_clean":"2024-09-10T16:12:10.862711+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:12:10.862711+0000","last_undegraded":"2024-09-10T16:12:10.862711+0000","last_fullsized":"2024-09-10T16:12:10.862711+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20910","reported_seq":29607,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.864300+0000","last_change":"2024-09-10T16:09:08.909580+0000","last_active":"2024-09-10T16:12:10.864300+0000","last_peered":"2024-09-10T16:12:10.864300+0000","last_clean":"2024-09-10T16:12:10.864300+0000","last_became_active":"2024-09-10T16:09:08.854941+0000","last_became_peered":"2024-09-10T16:09:08.854941+0000","last_unstale":"2024-09-10T16:12:10.864300+0000","last_undegraded":"2024-09-10T16:12:10.864300+0000","last_fullsized":"2024-09-10T16:12:10.864300+0000","mapping_epoch":50,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:13:14.846691+0000","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17414,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4257,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":148,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.864168+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:12:10.864168+0000","last_peered":"2024-09-10T16:12:10.864168+0000","last_clean":"2024-09-10T16:12:10.864168+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:12:10.864168+0000","last_undegraded":"2024-09-10T16:12:10.864168+0000","last_fullsized":"2024-09-10T16:12:10.864168+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20781","reported_seq":29578,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.862793+0000","last_change":"2024-09-10T16:10:20.907889+0000","last_active":"2024-09-10T16:12:10.862793+0000","last_peered":"2024-09-10T16:12:10.862793+0000","last_clean":"2024-09-10T16:12:10.862793+0000","last_became_active":"2024-09-10T16:10:20.852065+0000","last_became_peered":"2024-09-10T16:10:20.852065+0000","last_unstale":"2024-09-10T16:12:10.862793+0000","last_undegraded":"2024-09-10T16:12:10.862793+0000","last_fullsized":"2024-09-10T16:12:10.862793+0000","mapping_epoch":56,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:34.069549+0000","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18461,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4307,"num_bytes_recovered":8675,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":122,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:12:09.858220+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:12:09.858220+0000","last_peered":"2024-09-10T16:12:09.858220+0000","last_clean":"2024-09-10T16:12:09.858220+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:12:09.858220+0000","last_undegraded":"2024-09-10T16:12:09.858220+0000","last_fullsized":"2024-09-10T16:12:09.858220+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'21170","reported_seq":30063,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.862468+0000","last_change":"2024-09-10T16:10:50.728349+0000","last_active":"2024-09-10T16:12:10.862468+0000","last_peered":"2024-09-10T16:12:10.862468+0000","last_clean":"2024-09-10T16:12:10.862468+0000","last_became_active":"2024-09-10T16:08:10.860438+0000","last_became_peered":"2024-09-10T16:08:10.860438+0000","last_unstale":"2024-09-10T16:12:10.862468+0000","last_undegraded":"2024-09-10T16:12:10.862468+0000","last_fullsized":"2024-09-10T16:12:10.862468+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:38:13.781380+0000","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18724,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20838","reported_seq":20986,"reported_epoch":64,"state":"peering","last_fresh":"2024-09-10T16:12:09.399058+0000","last_change":"2024-09-10T16:12:10.285851+0000","last_active":"2024-09-10T16:12:10.285851+0000","last_peered":"2024-09-10T16:12:09.399058+0000","last_clean":"2024-09-10T15:59:14.117001+0000","last_became_active":"2024-09-10T15:59:16.302245+0000","last_became_peered":"2024-09-10T15:59:16.302245+0000","last_unstale":"2024-09-10T16:12:09.399058+0000","last_undegraded":"2024-09-10T16:12:09.399058+0000","last_fullsized":"2024-09-10T15:59:15.770337+0000","mapping_epoch":64,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17469,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4305,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":112,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:12:09.858658+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:12:09.858658+0000","last_peered":"2024-09-10T16:12:09.858658+0000","last_clean":"2024-09-10T16:12:09.858658+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:12:09.858658+0000","last_undegraded":"2024-09-10T16:12:09.858658+0000","last_fullsized":"2024-09-10T16:12:09.858658+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20598","reported_seq":29206,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:12:09.860146+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:12:09.860146+0000","last_peered":"2024-09-10T16:12:09.860146+0000","last_clean":"2024-09-10T16:12:09.860146+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:12:09.860146+0000","last_undegraded":"2024-09-10T16:12:09.860146+0000","last_fullsized":"2024-09-10T16:12:09.860146+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16988,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":138,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.863216+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:12:10.863216+0000","last_peered":"2024-09-10T16:12:10.863216+0000","last_clean":"2024-09-10T16:12:10.863216+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:12:10.863216+0000","last_undegraded":"2024-09-10T16:12:10.863216+0000","last_fullsized":"2024-09-10T16:12:10.863216+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'21048","reported_seq":21216,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:12:09.858839+0000","last_change":"2024-09-10T16:11:17.315372+0000","last_active":"2024-09-10T16:12:09.858839+0000","last_peered":"2024-09-10T16:12:09.858839+0000","last_clean":"2024-09-10T16:12:09.858839+0000","last_became_active":"2024-09-10T16:11:17.234487+0000","last_became_peered":"2024-09-10T16:11:17.234487+0000","last_unstale":"2024-09-10T16:12:09.858839+0000","last_undegraded":"2024-09-10T16:12:09.858839+0000","last_fullsized":"2024-09-10T16:12:09.858839+0000","mapping_epoch":60,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:19:00.114196+0000","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17303,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4373,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":112,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:12:09.859682+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:12:09.859682+0000","last_peered":"2024-09-10T16:12:09.859682+0000","last_clean":"2024-09-10T16:12:09.859682+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:12:09.859682+0000","last_undegraded":"2024-09-10T16:12:09.859682+0000","last_fullsized":"2024-09-10T16:12:09.859682+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":116,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:12:09.858231+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:12:09.858231+0000","last_peered":"2024-09-10T16:12:09.858231+0000","last_clean":"2024-09-10T16:12:09.858231+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:12:09.858231+0000","last_undegraded":"2024-09-10T16:12:09.858231+0000","last_fullsized":"2024-09-10T16:12:09.858231+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'21198","reported_seq":21367,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:12:09.858570+0000","last_change":"2024-09-10T16:11:44.391021+0000","last_active":"2024-09-10T16:12:09.858570+0000","last_peered":"2024-09-10T16:12:09.858570+0000","last_clean":"2024-09-10T16:12:09.858570+0000","last_became_active":"2024-09-10T16:11:44.312647+0000","last_became_peered":"2024-09-10T16:11:44.312647+0000","last_unstale":"2024-09-10T16:12:09.858570+0000","last_undegraded":"2024-09-10T16:12:09.858570+0000","last_fullsized":"2024-09-10T16:12:09.858570+0000","mapping_epoch":62,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:33:28.041611+0000","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17361,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4325,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3961,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292096,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":47429,"num_bytes_recovered":33098578,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":160874496,"data_stored":160784171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161022,"ondisk_log_size":161022,"up":32,"acting":31,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986346,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":455100,"kb_used_data":90216,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93916740,"statfs":{"total":96636764160,"available":96170741760,"internally_reserved":0,"allocated":92381184,"data_stored":91998950,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659291,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":505660,"kb_used_data":552,"kb_used_omap":4,"kb_used_meta":505083,"kb_avail":93866180,"statfs":{"total":96636764160,"available":96118968320,"internally_reserved":0,"allocated":565248,"data_stored":182550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":517205678},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182574,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":547424,"kb_used_data":68748,"kb_used_omap":4,"kb_used_meta":478651,"kb_avail":93824416,"statfs":{"total":96636764160,"available":96076201984,"internally_reserved":0,"allocated":70397952,"data_stored":69998954,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":490139306},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":69853184,"data_stored":69817054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90992640,"data_stored":90963736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:12:18.407 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:12:18.773 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:12:18.774 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:12:18.786 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":509,"stamp":"2024-09-10T16:12:17.801773+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3198,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292098,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":48194,"num_bytes_recovered":33098597,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161025,"ondisk_log_size":161025,"up":48,"acting":47,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1495676,"kb_used_data":155336,"kb_used_omap":12,"kb_used_meta":1340275,"kb_avail":281619844,"statfs":{"total":289910292480,"available":288378720256,"internally_reserved":0,"allocated":159064064,"data_stored":157889212,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1372441602},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1799,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1799,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001324"},"pg_stats":[{"pgid":"2.f","version":"49'20672","reported_seq":29317,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.863984+0000","last_change":"2024-09-10T16:09:57.088531+0000","last_active":"2024-09-10T16:12:10.863984+0000","last_peered":"2024-09-10T16:12:10.863984+0000","last_clean":"2024-09-10T16:12:10.863984+0000","last_became_active":"2024-09-10T16:09:57.005755+0000","last_became_peered":"2024-09-10T16:09:57.005755+0000","last_unstale":"2024-09-10T16:12:10.863984+0000","last_undegraded":"2024-09-10T16:12:10.863984+0000","last_fullsized":"2024-09-10T16:12:10.863984+0000","mapping_epoch":54,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:09:19.329697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20672,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4259,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20844","reported_seq":21001,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.863381+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:12:10.863381+0000","last_peered":"2024-09-10T16:12:10.863381+0000","last_clean":"2024-09-10T16:12:10.863381+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:12:10.863381+0000","last_undegraded":"2024-09-10T16:12:10.863381+0000","last_fullsized":"2024-09-10T16:12:10.863381+0000","mapping_epoch":35,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18949,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20798","reported_seq":29520,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.865598+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:12:10.865598+0000","last_peered":"2024-09-10T16:12:10.865598+0000","last_clean":"2024-09-10T16:12:10.865598+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:12:10.865598+0000","last_undegraded":"2024-09-10T16:12:10.865598+0000","last_fullsized":"2024-09-10T16:12:10.865598+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18802,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20660","reported_seq":20785,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.863544+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:12:10.863544+0000","last_peered":"2024-09-10T16:12:10.863544+0000","last_clean":"2024-09-10T16:12:10.863544+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:12:10.863544+0000","last_undegraded":"2024-09-10T16:12:10.863544+0000","last_fullsized":"2024-09-10T16:12:10.863544+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20660,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20891","reported_seq":21065,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.864566+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:12:10.864566+0000","last_peered":"2024-09-10T16:12:10.864566+0000","last_clean":"2024-09-10T16:12:10.864566+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:12:10.864566+0000","last_undegraded":"2024-09-10T16:12:10.864566+0000","last_fullsized":"2024-09-10T16:12:10.864566+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17602,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'21061","reported_seq":21234,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.865406+0000","last_change":"2024-09-10T16:10:52.241702+0000","last_active":"2024-09-10T16:12:10.865406+0000","last_peered":"2024-09-10T16:12:10.865406+0000","last_clean":"2024-09-10T16:12:10.865406+0000","last_became_active":"2024-09-10T16:10:52.171236+0000","last_became_peered":"2024-09-10T16:10:52.171236+0000","last_unstale":"2024-09-10T16:12:10.865406+0000","last_undegraded":"2024-09-10T16:12:10.865406+0000","last_fullsized":"2024-09-10T16:12:10.865406+0000","mapping_epoch":58,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:04:46.778280+0000","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17673,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4372,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'21082","reported_seq":23343,"reported_epoch":65,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:12:16.251786+0000","last_change":"2024-09-10T16:12:09.399531+0000","last_active":"2024-09-10T16:12:16.251786+0000","last_peered":"2024-09-10T16:12:16.251786+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:12:16.251786+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3198,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17581,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1039,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3196}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'21314","reported_seq":21488,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.862825+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:12:10.862825+0000","last_peered":"2024-09-10T16:12:10.862825+0000","last_clean":"2024-09-10T16:12:10.862825+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:12:10.862825+0000","last_undegraded":"2024-09-10T16:12:10.862825+0000","last_fullsized":"2024-09-10T16:12:10.862825+0000","mapping_epoch":34,"log_start":"45'11300","ondisk_log_start":"45'11300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18161,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.865297+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:12:10.865297+0000","last_peered":"2024-09-10T16:12:10.865297+0000","last_clean":"2024-09-10T16:12:10.865297+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:12:10.865297+0000","last_undegraded":"2024-09-10T16:12:10.865297+0000","last_fullsized":"2024-09-10T16:12:10.865297+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20457","reported_seq":20645,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.865472+0000","last_change":"2024-09-10T16:09:33.012840+0000","last_active":"2024-09-10T16:12:10.865472+0000","last_peered":"2024-09-10T16:12:10.865472+0000","last_clean":"2024-09-10T16:12:10.865472+0000","last_became_active":"2024-09-10T16:09:32.934056+0000","last_became_peered":"2024-09-10T16:09:32.934056+0000","last_unstale":"2024-09-10T16:12:10.865472+0000","last_undegraded":"2024-09-10T16:12:10.865472+0000","last_fullsized":"2024-09-10T16:12:10.865472+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:08:30.481235+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18276,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":4579,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":138,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.862711+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:12:10.862711+0000","last_peered":"2024-09-10T16:12:10.862711+0000","last_clean":"2024-09-10T16:12:10.862711+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:12:10.862711+0000","last_undegraded":"2024-09-10T16:12:10.862711+0000","last_fullsized":"2024-09-10T16:12:10.862711+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20910","reported_seq":29607,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.864300+0000","last_change":"2024-09-10T16:09:08.909580+0000","last_active":"2024-09-10T16:12:10.864300+0000","last_peered":"2024-09-10T16:12:10.864300+0000","last_clean":"2024-09-10T16:12:10.864300+0000","last_became_active":"2024-09-10T16:09:08.854941+0000","last_became_peered":"2024-09-10T16:09:08.854941+0000","last_unstale":"2024-09-10T16:12:10.864300+0000","last_undegraded":"2024-09-10T16:12:10.864300+0000","last_fullsized":"2024-09-10T16:12:10.864300+0000","mapping_epoch":50,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:13:14.846691+0000","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17414,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4257,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":148,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.864168+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:12:10.864168+0000","last_peered":"2024-09-10T16:12:10.864168+0000","last_clean":"2024-09-10T16:12:10.864168+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:12:10.864168+0000","last_undegraded":"2024-09-10T16:12:10.864168+0000","last_fullsized":"2024-09-10T16:12:10.864168+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20781","reported_seq":29578,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.862793+0000","last_change":"2024-09-10T16:10:20.907889+0000","last_active":"2024-09-10T16:12:10.862793+0000","last_peered":"2024-09-10T16:12:10.862793+0000","last_clean":"2024-09-10T16:12:10.862793+0000","last_became_active":"2024-09-10T16:10:20.852065+0000","last_became_peered":"2024-09-10T16:10:20.852065+0000","last_unstale":"2024-09-10T16:12:10.862793+0000","last_undegraded":"2024-09-10T16:12:10.862793+0000","last_fullsized":"2024-09-10T16:12:10.862793+0000","mapping_epoch":56,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:34.069549+0000","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18461,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4307,"num_bytes_recovered":8675,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":124,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.865426+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:12:10.865426+0000","last_peered":"2024-09-10T16:12:10.865426+0000","last_clean":"2024-09-10T16:12:10.865426+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:12:10.865426+0000","last_undegraded":"2024-09-10T16:12:10.865426+0000","last_fullsized":"2024-09-10T16:12:10.865426+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'21170","reported_seq":30063,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.862468+0000","last_change":"2024-09-10T16:10:50.728349+0000","last_active":"2024-09-10T16:12:10.862468+0000","last_peered":"2024-09-10T16:12:10.862468+0000","last_clean":"2024-09-10T16:12:10.862468+0000","last_became_active":"2024-09-10T16:08:10.860438+0000","last_became_peered":"2024-09-10T16:08:10.860438+0000","last_unstale":"2024-09-10T16:12:10.862468+0000","last_undegraded":"2024-09-10T16:12:10.862468+0000","last_fullsized":"2024-09-10T16:12:10.862468+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:38:13.781380+0000","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18724,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20838","reported_seq":20995,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.956229+0000","last_change":"2024-09-10T16:12:10.956160+0000","last_active":"2024-09-10T16:12:10.956229+0000","last_peered":"2024-09-10T16:12:10.956229+0000","last_clean":"2024-09-10T16:12:10.956229+0000","last_became_active":"2024-09-10T16:12:10.866692+0000","last_became_peered":"2024-09-10T16:12:10.866692+0000","last_unstale":"2024-09-10T16:12:10.956229+0000","last_undegraded":"2024-09-10T16:12:10.956229+0000","last_fullsized":"2024-09-10T16:12:10.956229+0000","mapping_epoch":64,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:01:48.931853+0000","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17469,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4305,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.865568+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:12:10.865568+0000","last_peered":"2024-09-10T16:12:10.865568+0000","last_clean":"2024-09-10T16:12:10.865568+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:12:10.865568+0000","last_undegraded":"2024-09-10T16:12:10.865568+0000","last_fullsized":"2024-09-10T16:12:10.865568+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20598","reported_seq":29208,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.865708+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:12:10.865708+0000","last_peered":"2024-09-10T16:12:10.865708+0000","last_clean":"2024-09-10T16:12:10.865708+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:12:10.865708+0000","last_undegraded":"2024-09-10T16:12:10.865708+0000","last_fullsized":"2024-09-10T16:12:10.865708+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16988,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":138,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.863216+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:12:10.863216+0000","last_peered":"2024-09-10T16:12:10.863216+0000","last_clean":"2024-09-10T16:12:10.863216+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:12:10.863216+0000","last_undegraded":"2024-09-10T16:12:10.863216+0000","last_fullsized":"2024-09-10T16:12:10.863216+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'21048","reported_seq":21218,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.865639+0000","last_change":"2024-09-10T16:11:17.315372+0000","last_active":"2024-09-10T16:12:10.865639+0000","last_peered":"2024-09-10T16:12:10.865639+0000","last_clean":"2024-09-10T16:12:10.865639+0000","last_became_active":"2024-09-10T16:11:17.234487+0000","last_became_peered":"2024-09-10T16:11:17.234487+0000","last_unstale":"2024-09-10T16:12:10.865639+0000","last_undegraded":"2024-09-10T16:12:10.865639+0000","last_fullsized":"2024-09-10T16:12:10.865639+0000","mapping_epoch":60,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:19:00.114196+0000","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17303,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4373,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.865649+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:12:10.865649+0000","last_peered":"2024-09-10T16:12:10.865649+0000","last_clean":"2024-09-10T16:12:10.865649+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:12:10.865649+0000","last_undegraded":"2024-09-10T16:12:10.865649+0000","last_fullsized":"2024-09-10T16:12:10.865649+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":118,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.865717+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:12:10.865717+0000","last_peered":"2024-09-10T16:12:10.865717+0000","last_clean":"2024-09-10T16:12:10.865717+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:12:10.865717+0000","last_undegraded":"2024-09-10T16:12:10.865717+0000","last_fullsized":"2024-09-10T16:12:10.865717+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'21198","reported_seq":21369,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.865540+0000","last_change":"2024-09-10T16:11:44.391021+0000","last_active":"2024-09-10T16:12:10.865540+0000","last_peered":"2024-09-10T16:12:10.865540+0000","last_clean":"2024-09-10T16:12:10.865540+0000","last_became_active":"2024-09-10T16:11:44.312647+0000","last_became_peered":"2024-09-10T16:11:44.312647+0000","last_unstale":"2024-09-10T16:12:10.865540+0000","last_undegraded":"2024-09-10T16:12:10.865540+0000","last_fullsized":"2024-09-10T16:12:10.865540+0000","mapping_epoch":62,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:33:28.041611+0000","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17361,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4325,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3198,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292096,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":48192,"num_bytes_recovered":33098578,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":156573696,"data_stored":156483371,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161022,"ondisk_log_size":161022,"up":32,"acting":31,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986347,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":450908,"kb_used_data":86028,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93920932,"statfs":{"total":96636764160,"available":96175034368,"internally_reserved":0,"allocated":88092672,"data_stored":87704977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659292,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":497344,"kb_used_data":560,"kb_used_omap":4,"kb_used_meta":496763,"kb_avail":93874496,"statfs":{"total":96636764160,"available":96127483904,"internally_reserved":0,"allocated":573440,"data_stored":185281,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":508685998},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182575,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":547424,"kb_used_data":68748,"kb_used_omap":4,"kb_used_meta":478651,"kb_avail":93824416,"statfs":{"total":96636764160,"available":96076201984,"internally_reserved":0,"allocated":70397952,"data_stored":69998954,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":490139306},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":69853184,"data_stored":69817054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":86691840,"data_stored":86662936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:12:18.787 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:12:19.100 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:12:19.101 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:12:19.115 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":509,"stamp":"2024-09-10T16:12:17.801773+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3198,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292098,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":48194,"num_bytes_recovered":33098597,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161025,"ondisk_log_size":161025,"up":48,"acting":47,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1495676,"kb_used_data":155336,"kb_used_omap":12,"kb_used_meta":1340275,"kb_avail":281619844,"statfs":{"total":289910292480,"available":288378720256,"internally_reserved":0,"allocated":159064064,"data_stored":157889212,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1372441602},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1799,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1799,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001324"},"pg_stats":[{"pgid":"2.f","version":"49'20672","reported_seq":29317,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.863984+0000","last_change":"2024-09-10T16:09:57.088531+0000","last_active":"2024-09-10T16:12:10.863984+0000","last_peered":"2024-09-10T16:12:10.863984+0000","last_clean":"2024-09-10T16:12:10.863984+0000","last_became_active":"2024-09-10T16:09:57.005755+0000","last_became_peered":"2024-09-10T16:09:57.005755+0000","last_unstale":"2024-09-10T16:12:10.863984+0000","last_undegraded":"2024-09-10T16:12:10.863984+0000","last_fullsized":"2024-09-10T16:12:10.863984+0000","mapping_epoch":54,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:09:19.329697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20672,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4259,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20844","reported_seq":21001,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.863381+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:12:10.863381+0000","last_peered":"2024-09-10T16:12:10.863381+0000","last_clean":"2024-09-10T16:12:10.863381+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:12:10.863381+0000","last_undegraded":"2024-09-10T16:12:10.863381+0000","last_fullsized":"2024-09-10T16:12:10.863381+0000","mapping_epoch":35,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18949,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20798","reported_seq":29520,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.865598+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:12:10.865598+0000","last_peered":"2024-09-10T16:12:10.865598+0000","last_clean":"2024-09-10T16:12:10.865598+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:12:10.865598+0000","last_undegraded":"2024-09-10T16:12:10.865598+0000","last_fullsized":"2024-09-10T16:12:10.865598+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18802,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20660","reported_seq":20785,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.863544+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:12:10.863544+0000","last_peered":"2024-09-10T16:12:10.863544+0000","last_clean":"2024-09-10T16:12:10.863544+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:12:10.863544+0000","last_undegraded":"2024-09-10T16:12:10.863544+0000","last_fullsized":"2024-09-10T16:12:10.863544+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20660,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20891","reported_seq":21065,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.864566+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:12:10.864566+0000","last_peered":"2024-09-10T16:12:10.864566+0000","last_clean":"2024-09-10T16:12:10.864566+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:12:10.864566+0000","last_undegraded":"2024-09-10T16:12:10.864566+0000","last_fullsized":"2024-09-10T16:12:10.864566+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17602,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'21061","reported_seq":21234,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.865406+0000","last_change":"2024-09-10T16:10:52.241702+0000","last_active":"2024-09-10T16:12:10.865406+0000","last_peered":"2024-09-10T16:12:10.865406+0000","last_clean":"2024-09-10T16:12:10.865406+0000","last_became_active":"2024-09-10T16:10:52.171236+0000","last_became_peered":"2024-09-10T16:10:52.171236+0000","last_unstale":"2024-09-10T16:12:10.865406+0000","last_undegraded":"2024-09-10T16:12:10.865406+0000","last_fullsized":"2024-09-10T16:12:10.865406+0000","mapping_epoch":58,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:04:46.778280+0000","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17673,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4372,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'21082","reported_seq":23343,"reported_epoch":65,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:12:16.251786+0000","last_change":"2024-09-10T16:12:09.399531+0000","last_active":"2024-09-10T16:12:16.251786+0000","last_peered":"2024-09-10T16:12:16.251786+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:12:16.251786+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3198,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17581,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1039,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3196}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'21314","reported_seq":21488,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.862825+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:12:10.862825+0000","last_peered":"2024-09-10T16:12:10.862825+0000","last_clean":"2024-09-10T16:12:10.862825+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:12:10.862825+0000","last_undegraded":"2024-09-10T16:12:10.862825+0000","last_fullsized":"2024-09-10T16:12:10.862825+0000","mapping_epoch":34,"log_start":"45'11300","ondisk_log_start":"45'11300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18161,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.865297+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:12:10.865297+0000","last_peered":"2024-09-10T16:12:10.865297+0000","last_clean":"2024-09-10T16:12:10.865297+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:12:10.865297+0000","last_undegraded":"2024-09-10T16:12:10.865297+0000","last_fullsized":"2024-09-10T16:12:10.865297+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20457","reported_seq":20645,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.865472+0000","last_change":"2024-09-10T16:09:33.012840+0000","last_active":"2024-09-10T16:12:10.865472+0000","last_peered":"2024-09-10T16:12:10.865472+0000","last_clean":"2024-09-10T16:12:10.865472+0000","last_became_active":"2024-09-10T16:09:32.934056+0000","last_became_peered":"2024-09-10T16:09:32.934056+0000","last_unstale":"2024-09-10T16:12:10.865472+0000","last_undegraded":"2024-09-10T16:12:10.865472+0000","last_fullsized":"2024-09-10T16:12:10.865472+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:08:30.481235+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18276,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":4579,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":138,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.862711+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:12:10.862711+0000","last_peered":"2024-09-10T16:12:10.862711+0000","last_clean":"2024-09-10T16:12:10.862711+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:12:10.862711+0000","last_undegraded":"2024-09-10T16:12:10.862711+0000","last_fullsized":"2024-09-10T16:12:10.862711+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20910","reported_seq":29607,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.864300+0000","last_change":"2024-09-10T16:09:08.909580+0000","last_active":"2024-09-10T16:12:10.864300+0000","last_peered":"2024-09-10T16:12:10.864300+0000","last_clean":"2024-09-10T16:12:10.864300+0000","last_became_active":"2024-09-10T16:09:08.854941+0000","last_became_peered":"2024-09-10T16:09:08.854941+0000","last_unstale":"2024-09-10T16:12:10.864300+0000","last_undegraded":"2024-09-10T16:12:10.864300+0000","last_fullsized":"2024-09-10T16:12:10.864300+0000","mapping_epoch":50,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:13:14.846691+0000","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17414,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4257,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":148,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.864168+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:12:10.864168+0000","last_peered":"2024-09-10T16:12:10.864168+0000","last_clean":"2024-09-10T16:12:10.864168+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:12:10.864168+0000","last_undegraded":"2024-09-10T16:12:10.864168+0000","last_fullsized":"2024-09-10T16:12:10.864168+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20781","reported_seq":29578,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.862793+0000","last_change":"2024-09-10T16:10:20.907889+0000","last_active":"2024-09-10T16:12:10.862793+0000","last_peered":"2024-09-10T16:12:10.862793+0000","last_clean":"2024-09-10T16:12:10.862793+0000","last_became_active":"2024-09-10T16:10:20.852065+0000","last_became_peered":"2024-09-10T16:10:20.852065+0000","last_unstale":"2024-09-10T16:12:10.862793+0000","last_undegraded":"2024-09-10T16:12:10.862793+0000","last_fullsized":"2024-09-10T16:12:10.862793+0000","mapping_epoch":56,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:34.069549+0000","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18461,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4307,"num_bytes_recovered":8675,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":124,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.865426+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:12:10.865426+0000","last_peered":"2024-09-10T16:12:10.865426+0000","last_clean":"2024-09-10T16:12:10.865426+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:12:10.865426+0000","last_undegraded":"2024-09-10T16:12:10.865426+0000","last_fullsized":"2024-09-10T16:12:10.865426+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'21170","reported_seq":30063,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.862468+0000","last_change":"2024-09-10T16:10:50.728349+0000","last_active":"2024-09-10T16:12:10.862468+0000","last_peered":"2024-09-10T16:12:10.862468+0000","last_clean":"2024-09-10T16:12:10.862468+0000","last_became_active":"2024-09-10T16:08:10.860438+0000","last_became_peered":"2024-09-10T16:08:10.860438+0000","last_unstale":"2024-09-10T16:12:10.862468+0000","last_undegraded":"2024-09-10T16:12:10.862468+0000","last_fullsized":"2024-09-10T16:12:10.862468+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:38:13.781380+0000","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18724,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20838","reported_seq":20995,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.956229+0000","last_change":"2024-09-10T16:12:10.956160+0000","last_active":"2024-09-10T16:12:10.956229+0000","last_peered":"2024-09-10T16:12:10.956229+0000","last_clean":"2024-09-10T16:12:10.956229+0000","last_became_active":"2024-09-10T16:12:10.866692+0000","last_became_peered":"2024-09-10T16:12:10.866692+0000","last_unstale":"2024-09-10T16:12:10.956229+0000","last_undegraded":"2024-09-10T16:12:10.956229+0000","last_fullsized":"2024-09-10T16:12:10.956229+0000","mapping_epoch":64,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:01:48.931853+0000","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17469,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4305,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.865568+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:12:10.865568+0000","last_peered":"2024-09-10T16:12:10.865568+0000","last_clean":"2024-09-10T16:12:10.865568+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:12:10.865568+0000","last_undegraded":"2024-09-10T16:12:10.865568+0000","last_fullsized":"2024-09-10T16:12:10.865568+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20598","reported_seq":29208,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.865708+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:12:10.865708+0000","last_peered":"2024-09-10T16:12:10.865708+0000","last_clean":"2024-09-10T16:12:10.865708+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:12:10.865708+0000","last_undegraded":"2024-09-10T16:12:10.865708+0000","last_fullsized":"2024-09-10T16:12:10.865708+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16988,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":138,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.863216+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:12:10.863216+0000","last_peered":"2024-09-10T16:12:10.863216+0000","last_clean":"2024-09-10T16:12:10.863216+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:12:10.863216+0000","last_undegraded":"2024-09-10T16:12:10.863216+0000","last_fullsized":"2024-09-10T16:12:10.863216+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'21048","reported_seq":21218,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.865639+0000","last_change":"2024-09-10T16:11:17.315372+0000","last_active":"2024-09-10T16:12:10.865639+0000","last_peered":"2024-09-10T16:12:10.865639+0000","last_clean":"2024-09-10T16:12:10.865639+0000","last_became_active":"2024-09-10T16:11:17.234487+0000","last_became_peered":"2024-09-10T16:11:17.234487+0000","last_unstale":"2024-09-10T16:12:10.865639+0000","last_undegraded":"2024-09-10T16:12:10.865639+0000","last_fullsized":"2024-09-10T16:12:10.865639+0000","mapping_epoch":60,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:19:00.114196+0000","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17303,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4373,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.865649+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:12:10.865649+0000","last_peered":"2024-09-10T16:12:10.865649+0000","last_clean":"2024-09-10T16:12:10.865649+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:12:10.865649+0000","last_undegraded":"2024-09-10T16:12:10.865649+0000","last_fullsized":"2024-09-10T16:12:10.865649+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":118,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.865717+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:12:10.865717+0000","last_peered":"2024-09-10T16:12:10.865717+0000","last_clean":"2024-09-10T16:12:10.865717+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:12:10.865717+0000","last_undegraded":"2024-09-10T16:12:10.865717+0000","last_fullsized":"2024-09-10T16:12:10.865717+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'21198","reported_seq":21369,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.865540+0000","last_change":"2024-09-10T16:11:44.391021+0000","last_active":"2024-09-10T16:12:10.865540+0000","last_peered":"2024-09-10T16:12:10.865540+0000","last_clean":"2024-09-10T16:12:10.865540+0000","last_became_active":"2024-09-10T16:11:44.312647+0000","last_became_peered":"2024-09-10T16:11:44.312647+0000","last_unstale":"2024-09-10T16:12:10.865540+0000","last_undegraded":"2024-09-10T16:12:10.865540+0000","last_fullsized":"2024-09-10T16:12:10.865540+0000","mapping_epoch":62,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:33:28.041611+0000","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17361,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4325,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3198,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292096,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":48192,"num_bytes_recovered":33098578,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":156573696,"data_stored":156483371,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161022,"ondisk_log_size":161022,"up":32,"acting":31,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986347,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":450908,"kb_used_data":86028,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93920932,"statfs":{"total":96636764160,"available":96175034368,"internally_reserved":0,"allocated":88092672,"data_stored":87704977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659292,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":497344,"kb_used_data":560,"kb_used_omap":4,"kb_used_meta":496763,"kb_avail":93874496,"statfs":{"total":96636764160,"available":96127483904,"internally_reserved":0,"allocated":573440,"data_stored":185281,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":508685998},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182575,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":547424,"kb_used_data":68748,"kb_used_omap":4,"kb_used_meta":478651,"kb_avail":93824416,"statfs":{"total":96636764160,"available":96076201984,"internally_reserved":0,"allocated":70397952,"data_stored":69998954,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":490139306},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":69853184,"data_stored":69817054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":86691840,"data_stored":86662936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:12:19.117 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-10T16:12:19.118 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "49'21082", 'reported_seq': 23343, 'reported_epoch': 65, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:12:16.251786+0000', 'last_change': '2024-09-10T16:12:09.399531+0000', 'last_active': '2024-09-10T16:12:16.251786+0000', 'last_peered': '2024-09-10T16:12:16.251786+0000', 'last_clean': '2024-09-10T15:59:14.139436+0000', 'last_became_active': '2024-09-10T15:59:16.318715+0000', 'last_became_peered': '2024-09-10T15:59:16.318715+0000', 'last_unstale': '2024-09-10T16:12:16.251786+0000', 'last_undegraded': '2024-09-10T15:59:15.877524+0000', 'last_fullsized': '2024-09-10T15:59:15.769042+0000', 'mapping_epoch': 42, 'log_start': "45'11000", 'ondisk_log_start': "45'11000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3537", 'last_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'objects_scrubbed': 3497, 'log_size': 10082, 'log_dups_size': 0, 'ondisk_log_size': 10082, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1224, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3198, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 17581, 'num_write_kb': 28172, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1039, 'num_bytes_recovered': 966, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3196}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:12:19.118 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T16:12:19.507 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:12:19.507 INFO:teuthology.orchestra.run.smithi115.stdout:{"fsid":"7effee29-df05-4351-9741-ee203564ed81","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 3198/24 objects degraded (13325.000%), 1 pg degraded, 1 pg undersized","count":2},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":903,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":65,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983954,"num_in_osds":3,"osd_in_since":1725983947,"num_remapped_pgs":1},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":23},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":12,"data_bytes":4849,"bytes_used":1531572224,"bytes_avail":288378720256,"bytes_total":289910292480,"degraded_objects":3198,"degraded_total":24,"degraded_ratio":133.25,"recovering_objects_per_sec":224,"recovering_bytes_per_sec":120,"recovering_keys_per_sec":0,"num_objects_recovered":1799,"num_bytes_recovered":966,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-10T15:57:14:967120+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T16:00:13.560776+0000","services":{}},"progress_events":{}} 2024-09-10T16:12:19.520 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-10T16:12:19.521 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:12:19.824 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:12:19.825 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:12:19.838 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":510,"stamp":"2024-09-10T16:12:19.802087+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3198,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292098,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":48194,"num_bytes_recovered":33098597,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161025,"ondisk_log_size":161025,"up":48,"acting":47,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1495676,"kb_used_data":155336,"kb_used_omap":12,"kb_used_meta":1340275,"kb_avail":281619844,"statfs":{"total":289910292480,"available":288378720256,"internally_reserved":0,"allocated":159064064,"data_stored":157889212,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1372441602},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1799,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1799,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.946024"},"pg_stats":[{"pgid":"2.f","version":"49'20672","reported_seq":29317,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.863984+0000","last_change":"2024-09-10T16:09:57.088531+0000","last_active":"2024-09-10T16:12:10.863984+0000","last_peered":"2024-09-10T16:12:10.863984+0000","last_clean":"2024-09-10T16:12:10.863984+0000","last_became_active":"2024-09-10T16:09:57.005755+0000","last_became_peered":"2024-09-10T16:09:57.005755+0000","last_unstale":"2024-09-10T16:12:10.863984+0000","last_undegraded":"2024-09-10T16:12:10.863984+0000","last_fullsized":"2024-09-10T16:12:10.863984+0000","mapping_epoch":54,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:09:19.329697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20672,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4259,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20844","reported_seq":21001,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.863381+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:12:10.863381+0000","last_peered":"2024-09-10T16:12:10.863381+0000","last_clean":"2024-09-10T16:12:10.863381+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:12:10.863381+0000","last_undegraded":"2024-09-10T16:12:10.863381+0000","last_fullsized":"2024-09-10T16:12:10.863381+0000","mapping_epoch":35,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18949,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20798","reported_seq":29520,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.865598+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:12:10.865598+0000","last_peered":"2024-09-10T16:12:10.865598+0000","last_clean":"2024-09-10T16:12:10.865598+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:12:10.865598+0000","last_undegraded":"2024-09-10T16:12:10.865598+0000","last_fullsized":"2024-09-10T16:12:10.865598+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18802,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20660","reported_seq":20785,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.863544+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:12:10.863544+0000","last_peered":"2024-09-10T16:12:10.863544+0000","last_clean":"2024-09-10T16:12:10.863544+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:12:10.863544+0000","last_undegraded":"2024-09-10T16:12:10.863544+0000","last_fullsized":"2024-09-10T16:12:10.863544+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20660,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20891","reported_seq":21065,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.864566+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:12:10.864566+0000","last_peered":"2024-09-10T16:12:10.864566+0000","last_clean":"2024-09-10T16:12:10.864566+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:12:10.864566+0000","last_undegraded":"2024-09-10T16:12:10.864566+0000","last_fullsized":"2024-09-10T16:12:10.864566+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17602,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'21061","reported_seq":21234,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.865406+0000","last_change":"2024-09-10T16:10:52.241702+0000","last_active":"2024-09-10T16:12:10.865406+0000","last_peered":"2024-09-10T16:12:10.865406+0000","last_clean":"2024-09-10T16:12:10.865406+0000","last_became_active":"2024-09-10T16:10:52.171236+0000","last_became_peered":"2024-09-10T16:10:52.171236+0000","last_unstale":"2024-09-10T16:12:10.865406+0000","last_undegraded":"2024-09-10T16:12:10.865406+0000","last_fullsized":"2024-09-10T16:12:10.865406+0000","mapping_epoch":58,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:04:46.778280+0000","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17673,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4372,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'21082","reported_seq":23343,"reported_epoch":65,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:12:16.251786+0000","last_change":"2024-09-10T16:12:09.399531+0000","last_active":"2024-09-10T16:12:16.251786+0000","last_peered":"2024-09-10T16:12:16.251786+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:12:16.251786+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3198,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17581,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1039,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3196}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'21314","reported_seq":21488,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.862825+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:12:10.862825+0000","last_peered":"2024-09-10T16:12:10.862825+0000","last_clean":"2024-09-10T16:12:10.862825+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:12:10.862825+0000","last_undegraded":"2024-09-10T16:12:10.862825+0000","last_fullsized":"2024-09-10T16:12:10.862825+0000","mapping_epoch":34,"log_start":"45'11300","ondisk_log_start":"45'11300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18161,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.865297+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:12:10.865297+0000","last_peered":"2024-09-10T16:12:10.865297+0000","last_clean":"2024-09-10T16:12:10.865297+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:12:10.865297+0000","last_undegraded":"2024-09-10T16:12:10.865297+0000","last_fullsized":"2024-09-10T16:12:10.865297+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20457","reported_seq":20645,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.865472+0000","last_change":"2024-09-10T16:09:33.012840+0000","last_active":"2024-09-10T16:12:10.865472+0000","last_peered":"2024-09-10T16:12:10.865472+0000","last_clean":"2024-09-10T16:12:10.865472+0000","last_became_active":"2024-09-10T16:09:32.934056+0000","last_became_peered":"2024-09-10T16:09:32.934056+0000","last_unstale":"2024-09-10T16:12:10.865472+0000","last_undegraded":"2024-09-10T16:12:10.865472+0000","last_fullsized":"2024-09-10T16:12:10.865472+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:08:30.481235+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18276,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":4579,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":138,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.862711+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:12:10.862711+0000","last_peered":"2024-09-10T16:12:10.862711+0000","last_clean":"2024-09-10T16:12:10.862711+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:12:10.862711+0000","last_undegraded":"2024-09-10T16:12:10.862711+0000","last_fullsized":"2024-09-10T16:12:10.862711+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20910","reported_seq":29607,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.864300+0000","last_change":"2024-09-10T16:09:08.909580+0000","last_active":"2024-09-10T16:12:10.864300+0000","last_peered":"2024-09-10T16:12:10.864300+0000","last_clean":"2024-09-10T16:12:10.864300+0000","last_became_active":"2024-09-10T16:09:08.854941+0000","last_became_peered":"2024-09-10T16:09:08.854941+0000","last_unstale":"2024-09-10T16:12:10.864300+0000","last_undegraded":"2024-09-10T16:12:10.864300+0000","last_fullsized":"2024-09-10T16:12:10.864300+0000","mapping_epoch":50,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:13:14.846691+0000","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17414,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4257,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":148,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.864168+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:12:10.864168+0000","last_peered":"2024-09-10T16:12:10.864168+0000","last_clean":"2024-09-10T16:12:10.864168+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:12:10.864168+0000","last_undegraded":"2024-09-10T16:12:10.864168+0000","last_fullsized":"2024-09-10T16:12:10.864168+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20781","reported_seq":29578,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.862793+0000","last_change":"2024-09-10T16:10:20.907889+0000","last_active":"2024-09-10T16:12:10.862793+0000","last_peered":"2024-09-10T16:12:10.862793+0000","last_clean":"2024-09-10T16:12:10.862793+0000","last_became_active":"2024-09-10T16:10:20.852065+0000","last_became_peered":"2024-09-10T16:10:20.852065+0000","last_unstale":"2024-09-10T16:12:10.862793+0000","last_undegraded":"2024-09-10T16:12:10.862793+0000","last_fullsized":"2024-09-10T16:12:10.862793+0000","mapping_epoch":56,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:34.069549+0000","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18461,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4307,"num_bytes_recovered":8675,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":124,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.865426+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:12:10.865426+0000","last_peered":"2024-09-10T16:12:10.865426+0000","last_clean":"2024-09-10T16:12:10.865426+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:12:10.865426+0000","last_undegraded":"2024-09-10T16:12:10.865426+0000","last_fullsized":"2024-09-10T16:12:10.865426+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'21170","reported_seq":30063,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.862468+0000","last_change":"2024-09-10T16:10:50.728349+0000","last_active":"2024-09-10T16:12:10.862468+0000","last_peered":"2024-09-10T16:12:10.862468+0000","last_clean":"2024-09-10T16:12:10.862468+0000","last_became_active":"2024-09-10T16:08:10.860438+0000","last_became_peered":"2024-09-10T16:08:10.860438+0000","last_unstale":"2024-09-10T16:12:10.862468+0000","last_undegraded":"2024-09-10T16:12:10.862468+0000","last_fullsized":"2024-09-10T16:12:10.862468+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:38:13.781380+0000","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18724,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20838","reported_seq":20995,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.956229+0000","last_change":"2024-09-10T16:12:10.956160+0000","last_active":"2024-09-10T16:12:10.956229+0000","last_peered":"2024-09-10T16:12:10.956229+0000","last_clean":"2024-09-10T16:12:10.956229+0000","last_became_active":"2024-09-10T16:12:10.866692+0000","last_became_peered":"2024-09-10T16:12:10.866692+0000","last_unstale":"2024-09-10T16:12:10.956229+0000","last_undegraded":"2024-09-10T16:12:10.956229+0000","last_fullsized":"2024-09-10T16:12:10.956229+0000","mapping_epoch":64,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:01:48.931853+0000","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17469,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4305,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.865568+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:12:10.865568+0000","last_peered":"2024-09-10T16:12:10.865568+0000","last_clean":"2024-09-10T16:12:10.865568+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:12:10.865568+0000","last_undegraded":"2024-09-10T16:12:10.865568+0000","last_fullsized":"2024-09-10T16:12:10.865568+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20598","reported_seq":29208,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.865708+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:12:10.865708+0000","last_peered":"2024-09-10T16:12:10.865708+0000","last_clean":"2024-09-10T16:12:10.865708+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:12:10.865708+0000","last_undegraded":"2024-09-10T16:12:10.865708+0000","last_fullsized":"2024-09-10T16:12:10.865708+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16988,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":138,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.863216+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:12:10.863216+0000","last_peered":"2024-09-10T16:12:10.863216+0000","last_clean":"2024-09-10T16:12:10.863216+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:12:10.863216+0000","last_undegraded":"2024-09-10T16:12:10.863216+0000","last_fullsized":"2024-09-10T16:12:10.863216+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'21048","reported_seq":21218,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.865639+0000","last_change":"2024-09-10T16:11:17.315372+0000","last_active":"2024-09-10T16:12:10.865639+0000","last_peered":"2024-09-10T16:12:10.865639+0000","last_clean":"2024-09-10T16:12:10.865639+0000","last_became_active":"2024-09-10T16:11:17.234487+0000","last_became_peered":"2024-09-10T16:11:17.234487+0000","last_unstale":"2024-09-10T16:12:10.865639+0000","last_undegraded":"2024-09-10T16:12:10.865639+0000","last_fullsized":"2024-09-10T16:12:10.865639+0000","mapping_epoch":60,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:19:00.114196+0000","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17303,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4373,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.865649+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:12:10.865649+0000","last_peered":"2024-09-10T16:12:10.865649+0000","last_clean":"2024-09-10T16:12:10.865649+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:12:10.865649+0000","last_undegraded":"2024-09-10T16:12:10.865649+0000","last_fullsized":"2024-09-10T16:12:10.865649+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":118,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.865717+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:12:10.865717+0000","last_peered":"2024-09-10T16:12:10.865717+0000","last_clean":"2024-09-10T16:12:10.865717+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:12:10.865717+0000","last_undegraded":"2024-09-10T16:12:10.865717+0000","last_fullsized":"2024-09-10T16:12:10.865717+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'21198","reported_seq":21369,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.865540+0000","last_change":"2024-09-10T16:11:44.391021+0000","last_active":"2024-09-10T16:12:10.865540+0000","last_peered":"2024-09-10T16:12:10.865540+0000","last_clean":"2024-09-10T16:12:10.865540+0000","last_became_active":"2024-09-10T16:11:44.312647+0000","last_became_peered":"2024-09-10T16:11:44.312647+0000","last_unstale":"2024-09-10T16:12:10.865540+0000","last_undegraded":"2024-09-10T16:12:10.865540+0000","last_fullsized":"2024-09-10T16:12:10.865540+0000","mapping_epoch":62,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:33:28.041611+0000","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17361,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4325,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3198,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292096,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":48192,"num_bytes_recovered":33098578,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":156573696,"data_stored":156483371,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161022,"ondisk_log_size":161022,"up":32,"acting":31,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986347,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":450908,"kb_used_data":86028,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93920932,"statfs":{"total":96636764160,"available":96175034368,"internally_reserved":0,"allocated":88092672,"data_stored":87704977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659292,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":497344,"kb_used_data":560,"kb_used_omap":4,"kb_used_meta":496763,"kb_avail":93874496,"statfs":{"total":96636764160,"available":96127483904,"internally_reserved":0,"allocated":573440,"data_stored":185281,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":508685998},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182575,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":547424,"kb_used_data":68748,"kb_used_omap":4,"kb_used_meta":478651,"kb_avail":93824416,"statfs":{"total":96636764160,"available":96076201984,"internally_reserved":0,"allocated":70397952,"data_stored":69998954,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":490139306},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":69853184,"data_stored":69817054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":86691840,"data_stored":86662936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:12:22.840 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:12:23.147 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:12:23.147 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:12:23.159 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":511,"stamp":"2024-09-10T16:12:21.802393+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2592,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292098,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":48800,"num_bytes_recovered":33098597,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161025,"ondisk_log_size":161025,"up":48,"acting":47,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1495516,"kb_used_data":151016,"kb_used_omap":12,"kb_used_meta":1344435,"kb_avail":281620004,"statfs":{"total":289910292480,"available":288378884096,"internally_reserved":0,"allocated":154640384,"data_stored":153465532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1376701442},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1645,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1645,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.941355"},"pg_stats":[{"pgid":"2.f","version":"49'20672","reported_seq":29317,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.863984+0000","last_change":"2024-09-10T16:09:57.088531+0000","last_active":"2024-09-10T16:12:10.863984+0000","last_peered":"2024-09-10T16:12:10.863984+0000","last_clean":"2024-09-10T16:12:10.863984+0000","last_became_active":"2024-09-10T16:09:57.005755+0000","last_became_peered":"2024-09-10T16:09:57.005755+0000","last_unstale":"2024-09-10T16:12:10.863984+0000","last_undegraded":"2024-09-10T16:12:10.863984+0000","last_fullsized":"2024-09-10T16:12:10.863984+0000","mapping_epoch":54,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:09:19.329697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20672,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4259,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20844","reported_seq":21001,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.863381+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:12:10.863381+0000","last_peered":"2024-09-10T16:12:10.863381+0000","last_clean":"2024-09-10T16:12:10.863381+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:12:10.863381+0000","last_undegraded":"2024-09-10T16:12:10.863381+0000","last_fullsized":"2024-09-10T16:12:10.863381+0000","mapping_epoch":35,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18949,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20798","reported_seq":29520,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.865598+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:12:10.865598+0000","last_peered":"2024-09-10T16:12:10.865598+0000","last_clean":"2024-09-10T16:12:10.865598+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:12:10.865598+0000","last_undegraded":"2024-09-10T16:12:10.865598+0000","last_fullsized":"2024-09-10T16:12:10.865598+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18802,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20660","reported_seq":20785,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.863544+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:12:10.863544+0000","last_peered":"2024-09-10T16:12:10.863544+0000","last_clean":"2024-09-10T16:12:10.863544+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:12:10.863544+0000","last_undegraded":"2024-09-10T16:12:10.863544+0000","last_fullsized":"2024-09-10T16:12:10.863544+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20660,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20891","reported_seq":21065,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.864566+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:12:10.864566+0000","last_peered":"2024-09-10T16:12:10.864566+0000","last_clean":"2024-09-10T16:12:10.864566+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:12:10.864566+0000","last_undegraded":"2024-09-10T16:12:10.864566+0000","last_fullsized":"2024-09-10T16:12:10.864566+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17602,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'21061","reported_seq":21234,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.865406+0000","last_change":"2024-09-10T16:10:52.241702+0000","last_active":"2024-09-10T16:12:10.865406+0000","last_peered":"2024-09-10T16:12:10.865406+0000","last_clean":"2024-09-10T16:12:10.865406+0000","last_became_active":"2024-09-10T16:10:52.171236+0000","last_became_peered":"2024-09-10T16:10:52.171236+0000","last_unstale":"2024-09-10T16:12:10.865406+0000","last_undegraded":"2024-09-10T16:12:10.865406+0000","last_fullsized":"2024-09-10T16:12:10.865406+0000","mapping_epoch":58,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:04:46.778280+0000","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17673,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4372,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'21082","reported_seq":24555,"reported_epoch":65,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:12:21.253248+0000","last_change":"2024-09-10T16:12:09.399531+0000","last_active":"2024-09-10T16:12:21.253248+0000","last_peered":"2024-09-10T16:12:21.253248+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:12:21.253248+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2592,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17581,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1645,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2590}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'21314","reported_seq":21488,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.862825+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:12:10.862825+0000","last_peered":"2024-09-10T16:12:10.862825+0000","last_clean":"2024-09-10T16:12:10.862825+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:12:10.862825+0000","last_undegraded":"2024-09-10T16:12:10.862825+0000","last_fullsized":"2024-09-10T16:12:10.862825+0000","mapping_epoch":34,"log_start":"45'11300","ondisk_log_start":"45'11300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18161,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.865297+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:12:10.865297+0000","last_peered":"2024-09-10T16:12:10.865297+0000","last_clean":"2024-09-10T16:12:10.865297+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:12:10.865297+0000","last_undegraded":"2024-09-10T16:12:10.865297+0000","last_fullsized":"2024-09-10T16:12:10.865297+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20457","reported_seq":20645,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.865472+0000","last_change":"2024-09-10T16:09:33.012840+0000","last_active":"2024-09-10T16:12:10.865472+0000","last_peered":"2024-09-10T16:12:10.865472+0000","last_clean":"2024-09-10T16:12:10.865472+0000","last_became_active":"2024-09-10T16:09:32.934056+0000","last_became_peered":"2024-09-10T16:09:32.934056+0000","last_unstale":"2024-09-10T16:12:10.865472+0000","last_undegraded":"2024-09-10T16:12:10.865472+0000","last_fullsized":"2024-09-10T16:12:10.865472+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:08:30.481235+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18276,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":4579,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":138,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.862711+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:12:10.862711+0000","last_peered":"2024-09-10T16:12:10.862711+0000","last_clean":"2024-09-10T16:12:10.862711+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:12:10.862711+0000","last_undegraded":"2024-09-10T16:12:10.862711+0000","last_fullsized":"2024-09-10T16:12:10.862711+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20910","reported_seq":29607,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.864300+0000","last_change":"2024-09-10T16:09:08.909580+0000","last_active":"2024-09-10T16:12:10.864300+0000","last_peered":"2024-09-10T16:12:10.864300+0000","last_clean":"2024-09-10T16:12:10.864300+0000","last_became_active":"2024-09-10T16:09:08.854941+0000","last_became_peered":"2024-09-10T16:09:08.854941+0000","last_unstale":"2024-09-10T16:12:10.864300+0000","last_undegraded":"2024-09-10T16:12:10.864300+0000","last_fullsized":"2024-09-10T16:12:10.864300+0000","mapping_epoch":50,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:13:14.846691+0000","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17414,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4257,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":148,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.864168+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:12:10.864168+0000","last_peered":"2024-09-10T16:12:10.864168+0000","last_clean":"2024-09-10T16:12:10.864168+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:12:10.864168+0000","last_undegraded":"2024-09-10T16:12:10.864168+0000","last_fullsized":"2024-09-10T16:12:10.864168+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20781","reported_seq":29578,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.862793+0000","last_change":"2024-09-10T16:10:20.907889+0000","last_active":"2024-09-10T16:12:10.862793+0000","last_peered":"2024-09-10T16:12:10.862793+0000","last_clean":"2024-09-10T16:12:10.862793+0000","last_became_active":"2024-09-10T16:10:20.852065+0000","last_became_peered":"2024-09-10T16:10:20.852065+0000","last_unstale":"2024-09-10T16:12:10.862793+0000","last_undegraded":"2024-09-10T16:12:10.862793+0000","last_fullsized":"2024-09-10T16:12:10.862793+0000","mapping_epoch":56,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:34.069549+0000","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18461,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4307,"num_bytes_recovered":8675,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":124,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.865426+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:12:10.865426+0000","last_peered":"2024-09-10T16:12:10.865426+0000","last_clean":"2024-09-10T16:12:10.865426+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:12:10.865426+0000","last_undegraded":"2024-09-10T16:12:10.865426+0000","last_fullsized":"2024-09-10T16:12:10.865426+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'21170","reported_seq":30064,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.862524+0000","last_change":"2024-09-10T16:10:50.728349+0000","last_active":"2024-09-10T16:12:10.862524+0000","last_peered":"2024-09-10T16:12:10.862524+0000","last_clean":"2024-09-10T16:12:10.862524+0000","last_became_active":"2024-09-10T16:08:10.860438+0000","last_became_peered":"2024-09-10T16:08:10.860438+0000","last_unstale":"2024-09-10T16:12:10.862524+0000","last_undegraded":"2024-09-10T16:12:10.862524+0000","last_fullsized":"2024-09-10T16:12:10.862524+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:38:13.781380+0000","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18724,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20838","reported_seq":20995,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.956229+0000","last_change":"2024-09-10T16:12:10.956160+0000","last_active":"2024-09-10T16:12:10.956229+0000","last_peered":"2024-09-10T16:12:10.956229+0000","last_clean":"2024-09-10T16:12:10.956229+0000","last_became_active":"2024-09-10T16:12:10.866692+0000","last_became_peered":"2024-09-10T16:12:10.866692+0000","last_unstale":"2024-09-10T16:12:10.956229+0000","last_undegraded":"2024-09-10T16:12:10.956229+0000","last_fullsized":"2024-09-10T16:12:10.956229+0000","mapping_epoch":64,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:01:48.931853+0000","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17469,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4305,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.865568+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:12:10.865568+0000","last_peered":"2024-09-10T16:12:10.865568+0000","last_clean":"2024-09-10T16:12:10.865568+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:12:10.865568+0000","last_undegraded":"2024-09-10T16:12:10.865568+0000","last_fullsized":"2024-09-10T16:12:10.865568+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20598","reported_seq":29208,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.865708+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:12:10.865708+0000","last_peered":"2024-09-10T16:12:10.865708+0000","last_clean":"2024-09-10T16:12:10.865708+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:12:10.865708+0000","last_undegraded":"2024-09-10T16:12:10.865708+0000","last_fullsized":"2024-09-10T16:12:10.865708+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16988,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":138,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.863216+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:12:10.863216+0000","last_peered":"2024-09-10T16:12:10.863216+0000","last_clean":"2024-09-10T16:12:10.863216+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:12:10.863216+0000","last_undegraded":"2024-09-10T16:12:10.863216+0000","last_fullsized":"2024-09-10T16:12:10.863216+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'21048","reported_seq":21218,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.865639+0000","last_change":"2024-09-10T16:11:17.315372+0000","last_active":"2024-09-10T16:12:10.865639+0000","last_peered":"2024-09-10T16:12:10.865639+0000","last_clean":"2024-09-10T16:12:10.865639+0000","last_became_active":"2024-09-10T16:11:17.234487+0000","last_became_peered":"2024-09-10T16:11:17.234487+0000","last_unstale":"2024-09-10T16:12:10.865639+0000","last_undegraded":"2024-09-10T16:12:10.865639+0000","last_fullsized":"2024-09-10T16:12:10.865639+0000","mapping_epoch":60,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:19:00.114196+0000","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17303,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4373,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.865649+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:12:10.865649+0000","last_peered":"2024-09-10T16:12:10.865649+0000","last_clean":"2024-09-10T16:12:10.865649+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:12:10.865649+0000","last_undegraded":"2024-09-10T16:12:10.865649+0000","last_fullsized":"2024-09-10T16:12:10.865649+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":119,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.865758+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:12:10.865758+0000","last_peered":"2024-09-10T16:12:10.865758+0000","last_clean":"2024-09-10T16:12:10.865758+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:12:10.865758+0000","last_undegraded":"2024-09-10T16:12:10.865758+0000","last_fullsized":"2024-09-10T16:12:10.865758+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'21198","reported_seq":21369,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.865540+0000","last_change":"2024-09-10T16:11:44.391021+0000","last_active":"2024-09-10T16:12:10.865540+0000","last_peered":"2024-09-10T16:12:10.865540+0000","last_clean":"2024-09-10T16:12:10.865540+0000","last_became_active":"2024-09-10T16:11:44.312647+0000","last_became_peered":"2024-09-10T16:11:44.312647+0000","last_unstale":"2024-09-10T16:12:10.865540+0000","last_undegraded":"2024-09-10T16:12:10.865540+0000","last_fullsized":"2024-09-10T16:12:10.865540+0000","mapping_epoch":62,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:33:28.041611+0000","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17361,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4325,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2592,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292096,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":48798,"num_bytes_recovered":33098578,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":152272896,"data_stored":152182571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161022,"ondisk_log_size":161022,"up":32,"acting":31,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986348,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":446588,"kb_used_data":81708,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93925252,"statfs":{"total":96636764160,"available":96179458048,"internally_reserved":0,"allocated":83668992,"data_stored":83281297,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659293,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":501504,"kb_used_data":560,"kb_used_omap":4,"kb_used_meta":500923,"kb_avail":93870336,"statfs":{"total":96636764160,"available":96123224064,"internally_reserved":0,"allocated":573440,"data_stored":185281,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":512945838},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182576,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":547424,"kb_used_data":68748,"kb_used_omap":4,"kb_used_meta":478651,"kb_avail":93824416,"statfs":{"total":96636764160,"available":96076201984,"internally_reserved":0,"allocated":70397952,"data_stored":69998954,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":490139306},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":69853184,"data_stored":69817054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":82391040,"data_stored":82362136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:12:23.161 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:12:23.497 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:12:23.498 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:12:23.510 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":511,"stamp":"2024-09-10T16:12:21.802393+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2592,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292098,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":48800,"num_bytes_recovered":33098597,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161025,"ondisk_log_size":161025,"up":48,"acting":47,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1495516,"kb_used_data":151016,"kb_used_omap":12,"kb_used_meta":1344435,"kb_avail":281620004,"statfs":{"total":289910292480,"available":288378884096,"internally_reserved":0,"allocated":154640384,"data_stored":153465532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1376701442},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1645,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1645,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.941355"},"pg_stats":[{"pgid":"2.f","version":"49'20672","reported_seq":29317,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.863984+0000","last_change":"2024-09-10T16:09:57.088531+0000","last_active":"2024-09-10T16:12:10.863984+0000","last_peered":"2024-09-10T16:12:10.863984+0000","last_clean":"2024-09-10T16:12:10.863984+0000","last_became_active":"2024-09-10T16:09:57.005755+0000","last_became_peered":"2024-09-10T16:09:57.005755+0000","last_unstale":"2024-09-10T16:12:10.863984+0000","last_undegraded":"2024-09-10T16:12:10.863984+0000","last_fullsized":"2024-09-10T16:12:10.863984+0000","mapping_epoch":54,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:09:19.329697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20672,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4259,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20844","reported_seq":21001,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.863381+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:12:10.863381+0000","last_peered":"2024-09-10T16:12:10.863381+0000","last_clean":"2024-09-10T16:12:10.863381+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:12:10.863381+0000","last_undegraded":"2024-09-10T16:12:10.863381+0000","last_fullsized":"2024-09-10T16:12:10.863381+0000","mapping_epoch":35,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18949,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20798","reported_seq":29520,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.865598+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:12:10.865598+0000","last_peered":"2024-09-10T16:12:10.865598+0000","last_clean":"2024-09-10T16:12:10.865598+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:12:10.865598+0000","last_undegraded":"2024-09-10T16:12:10.865598+0000","last_fullsized":"2024-09-10T16:12:10.865598+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18802,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20660","reported_seq":20785,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.863544+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:12:10.863544+0000","last_peered":"2024-09-10T16:12:10.863544+0000","last_clean":"2024-09-10T16:12:10.863544+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:12:10.863544+0000","last_undegraded":"2024-09-10T16:12:10.863544+0000","last_fullsized":"2024-09-10T16:12:10.863544+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20660,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20891","reported_seq":21065,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.864566+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:12:10.864566+0000","last_peered":"2024-09-10T16:12:10.864566+0000","last_clean":"2024-09-10T16:12:10.864566+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:12:10.864566+0000","last_undegraded":"2024-09-10T16:12:10.864566+0000","last_fullsized":"2024-09-10T16:12:10.864566+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17602,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'21061","reported_seq":21234,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.865406+0000","last_change":"2024-09-10T16:10:52.241702+0000","last_active":"2024-09-10T16:12:10.865406+0000","last_peered":"2024-09-10T16:12:10.865406+0000","last_clean":"2024-09-10T16:12:10.865406+0000","last_became_active":"2024-09-10T16:10:52.171236+0000","last_became_peered":"2024-09-10T16:10:52.171236+0000","last_unstale":"2024-09-10T16:12:10.865406+0000","last_undegraded":"2024-09-10T16:12:10.865406+0000","last_fullsized":"2024-09-10T16:12:10.865406+0000","mapping_epoch":58,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:04:46.778280+0000","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17673,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4372,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'21082","reported_seq":24555,"reported_epoch":65,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:12:21.253248+0000","last_change":"2024-09-10T16:12:09.399531+0000","last_active":"2024-09-10T16:12:21.253248+0000","last_peered":"2024-09-10T16:12:21.253248+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:12:21.253248+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2592,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17581,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1645,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2590}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'21314","reported_seq":21488,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.862825+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:12:10.862825+0000","last_peered":"2024-09-10T16:12:10.862825+0000","last_clean":"2024-09-10T16:12:10.862825+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:12:10.862825+0000","last_undegraded":"2024-09-10T16:12:10.862825+0000","last_fullsized":"2024-09-10T16:12:10.862825+0000","mapping_epoch":34,"log_start":"45'11300","ondisk_log_start":"45'11300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18161,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.865297+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:12:10.865297+0000","last_peered":"2024-09-10T16:12:10.865297+0000","last_clean":"2024-09-10T16:12:10.865297+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:12:10.865297+0000","last_undegraded":"2024-09-10T16:12:10.865297+0000","last_fullsized":"2024-09-10T16:12:10.865297+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20457","reported_seq":20645,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.865472+0000","last_change":"2024-09-10T16:09:33.012840+0000","last_active":"2024-09-10T16:12:10.865472+0000","last_peered":"2024-09-10T16:12:10.865472+0000","last_clean":"2024-09-10T16:12:10.865472+0000","last_became_active":"2024-09-10T16:09:32.934056+0000","last_became_peered":"2024-09-10T16:09:32.934056+0000","last_unstale":"2024-09-10T16:12:10.865472+0000","last_undegraded":"2024-09-10T16:12:10.865472+0000","last_fullsized":"2024-09-10T16:12:10.865472+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:08:30.481235+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18276,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":4579,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":138,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.862711+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:12:10.862711+0000","last_peered":"2024-09-10T16:12:10.862711+0000","last_clean":"2024-09-10T16:12:10.862711+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:12:10.862711+0000","last_undegraded":"2024-09-10T16:12:10.862711+0000","last_fullsized":"2024-09-10T16:12:10.862711+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20910","reported_seq":29607,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.864300+0000","last_change":"2024-09-10T16:09:08.909580+0000","last_active":"2024-09-10T16:12:10.864300+0000","last_peered":"2024-09-10T16:12:10.864300+0000","last_clean":"2024-09-10T16:12:10.864300+0000","last_became_active":"2024-09-10T16:09:08.854941+0000","last_became_peered":"2024-09-10T16:09:08.854941+0000","last_unstale":"2024-09-10T16:12:10.864300+0000","last_undegraded":"2024-09-10T16:12:10.864300+0000","last_fullsized":"2024-09-10T16:12:10.864300+0000","mapping_epoch":50,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:13:14.846691+0000","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17414,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4257,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":148,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.864168+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:12:10.864168+0000","last_peered":"2024-09-10T16:12:10.864168+0000","last_clean":"2024-09-10T16:12:10.864168+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:12:10.864168+0000","last_undegraded":"2024-09-10T16:12:10.864168+0000","last_fullsized":"2024-09-10T16:12:10.864168+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20781","reported_seq":29578,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.862793+0000","last_change":"2024-09-10T16:10:20.907889+0000","last_active":"2024-09-10T16:12:10.862793+0000","last_peered":"2024-09-10T16:12:10.862793+0000","last_clean":"2024-09-10T16:12:10.862793+0000","last_became_active":"2024-09-10T16:10:20.852065+0000","last_became_peered":"2024-09-10T16:10:20.852065+0000","last_unstale":"2024-09-10T16:12:10.862793+0000","last_undegraded":"2024-09-10T16:12:10.862793+0000","last_fullsized":"2024-09-10T16:12:10.862793+0000","mapping_epoch":56,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:34.069549+0000","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18461,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4307,"num_bytes_recovered":8675,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":124,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.865426+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:12:10.865426+0000","last_peered":"2024-09-10T16:12:10.865426+0000","last_clean":"2024-09-10T16:12:10.865426+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:12:10.865426+0000","last_undegraded":"2024-09-10T16:12:10.865426+0000","last_fullsized":"2024-09-10T16:12:10.865426+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'21170","reported_seq":30064,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.862524+0000","last_change":"2024-09-10T16:10:50.728349+0000","last_active":"2024-09-10T16:12:10.862524+0000","last_peered":"2024-09-10T16:12:10.862524+0000","last_clean":"2024-09-10T16:12:10.862524+0000","last_became_active":"2024-09-10T16:08:10.860438+0000","last_became_peered":"2024-09-10T16:08:10.860438+0000","last_unstale":"2024-09-10T16:12:10.862524+0000","last_undegraded":"2024-09-10T16:12:10.862524+0000","last_fullsized":"2024-09-10T16:12:10.862524+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:38:13.781380+0000","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18724,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20838","reported_seq":20995,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.956229+0000","last_change":"2024-09-10T16:12:10.956160+0000","last_active":"2024-09-10T16:12:10.956229+0000","last_peered":"2024-09-10T16:12:10.956229+0000","last_clean":"2024-09-10T16:12:10.956229+0000","last_became_active":"2024-09-10T16:12:10.866692+0000","last_became_peered":"2024-09-10T16:12:10.866692+0000","last_unstale":"2024-09-10T16:12:10.956229+0000","last_undegraded":"2024-09-10T16:12:10.956229+0000","last_fullsized":"2024-09-10T16:12:10.956229+0000","mapping_epoch":64,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:01:48.931853+0000","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17469,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4305,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.865568+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:12:10.865568+0000","last_peered":"2024-09-10T16:12:10.865568+0000","last_clean":"2024-09-10T16:12:10.865568+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:12:10.865568+0000","last_undegraded":"2024-09-10T16:12:10.865568+0000","last_fullsized":"2024-09-10T16:12:10.865568+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20598","reported_seq":29208,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.865708+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:12:10.865708+0000","last_peered":"2024-09-10T16:12:10.865708+0000","last_clean":"2024-09-10T16:12:10.865708+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:12:10.865708+0000","last_undegraded":"2024-09-10T16:12:10.865708+0000","last_fullsized":"2024-09-10T16:12:10.865708+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16988,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":138,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.863216+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:12:10.863216+0000","last_peered":"2024-09-10T16:12:10.863216+0000","last_clean":"2024-09-10T16:12:10.863216+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:12:10.863216+0000","last_undegraded":"2024-09-10T16:12:10.863216+0000","last_fullsized":"2024-09-10T16:12:10.863216+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'21048","reported_seq":21218,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.865639+0000","last_change":"2024-09-10T16:11:17.315372+0000","last_active":"2024-09-10T16:12:10.865639+0000","last_peered":"2024-09-10T16:12:10.865639+0000","last_clean":"2024-09-10T16:12:10.865639+0000","last_became_active":"2024-09-10T16:11:17.234487+0000","last_became_peered":"2024-09-10T16:11:17.234487+0000","last_unstale":"2024-09-10T16:12:10.865639+0000","last_undegraded":"2024-09-10T16:12:10.865639+0000","last_fullsized":"2024-09-10T16:12:10.865639+0000","mapping_epoch":60,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:19:00.114196+0000","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17303,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4373,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.865649+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:12:10.865649+0000","last_peered":"2024-09-10T16:12:10.865649+0000","last_clean":"2024-09-10T16:12:10.865649+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:12:10.865649+0000","last_undegraded":"2024-09-10T16:12:10.865649+0000","last_fullsized":"2024-09-10T16:12:10.865649+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":119,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.865758+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:12:10.865758+0000","last_peered":"2024-09-10T16:12:10.865758+0000","last_clean":"2024-09-10T16:12:10.865758+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:12:10.865758+0000","last_undegraded":"2024-09-10T16:12:10.865758+0000","last_fullsized":"2024-09-10T16:12:10.865758+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'21198","reported_seq":21369,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.865540+0000","last_change":"2024-09-10T16:11:44.391021+0000","last_active":"2024-09-10T16:12:10.865540+0000","last_peered":"2024-09-10T16:12:10.865540+0000","last_clean":"2024-09-10T16:12:10.865540+0000","last_became_active":"2024-09-10T16:11:44.312647+0000","last_became_peered":"2024-09-10T16:11:44.312647+0000","last_unstale":"2024-09-10T16:12:10.865540+0000","last_undegraded":"2024-09-10T16:12:10.865540+0000","last_fullsized":"2024-09-10T16:12:10.865540+0000","mapping_epoch":62,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:33:28.041611+0000","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17361,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4325,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2592,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292096,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":48798,"num_bytes_recovered":33098578,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":152272896,"data_stored":152182571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161022,"ondisk_log_size":161022,"up":32,"acting":31,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986348,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":446588,"kb_used_data":81708,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93925252,"statfs":{"total":96636764160,"available":96179458048,"internally_reserved":0,"allocated":83668992,"data_stored":83281297,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659293,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":501504,"kb_used_data":560,"kb_used_omap":4,"kb_used_meta":500923,"kb_avail":93870336,"statfs":{"total":96636764160,"available":96123224064,"internally_reserved":0,"allocated":573440,"data_stored":185281,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":512945838},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182576,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":547424,"kb_used_data":68748,"kb_used_omap":4,"kb_used_meta":478651,"kb_avail":93824416,"statfs":{"total":96636764160,"available":96076201984,"internally_reserved":0,"allocated":70397952,"data_stored":69998954,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":490139306},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":69853184,"data_stored":69817054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":82391040,"data_stored":82362136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:12:23.512 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-10T16:12:23.512 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "49'21082", 'reported_seq': 24555, 'reported_epoch': 65, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:12:21.253248+0000', 'last_change': '2024-09-10T16:12:09.399531+0000', 'last_active': '2024-09-10T16:12:21.253248+0000', 'last_peered': '2024-09-10T16:12:21.253248+0000', 'last_clean': '2024-09-10T15:59:14.139436+0000', 'last_became_active': '2024-09-10T15:59:16.318715+0000', 'last_became_peered': '2024-09-10T15:59:16.318715+0000', 'last_unstale': '2024-09-10T16:12:21.253248+0000', 'last_undegraded': '2024-09-10T15:59:15.877524+0000', 'last_fullsized': '2024-09-10T15:59:15.769042+0000', 'mapping_epoch': 42, 'log_start': "45'11000", 'ondisk_log_start': "45'11000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3537", 'last_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'objects_scrubbed': 3497, 'log_size': 10082, 'log_dups_size': 0, 'ondisk_log_size': 10082, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1224, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2592, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 17581, 'num_write_kb': 28172, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1645, 'num_bytes_recovered': 966, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2590}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:12:23.512 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T16:12:23.900 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:12:23.900 INFO:teuthology.orchestra.run.smithi115.stdout:{"fsid":"7effee29-df05-4351-9741-ee203564ed81","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 2592/24 objects degraded (10800.000%), 1 pg degraded, 1 pg undersized","count":2},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":907,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":65,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983954,"num_in_osds":3,"osd_in_since":1725983947,"num_remapped_pgs":1},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":23},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":12,"data_bytes":4849,"bytes_used":1531408384,"bytes_avail":288378884096,"bytes_total":289910292480,"degraded_objects":2592,"degraded_total":24,"degraded_ratio":108,"recovering_objects_per_sec":150,"recovering_bytes_per_sec":88,"recovering_keys_per_sec":0,"num_objects_recovered":1645,"num_bytes_recovered":966,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-10T15:57:14:967120+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T16:00:13.560776+0000","services":{}},"progress_events":{}} 2024-09-10T16:12:23.914 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-10T16:12:23.915 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:12:24.244 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:12:24.245 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:12:24.257 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":512,"stamp":"2024-09-10T16:12:23.802789+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2592,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292098,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":48800,"num_bytes_recovered":33098597,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161025,"ondisk_log_size":161025,"up":48,"acting":47,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1495516,"kb_used_data":151016,"kb_used_omap":12,"kb_used_meta":1344435,"kb_avail":281620004,"statfs":{"total":289910292480,"available":288378884096,"internally_reserved":0,"allocated":154640384,"data_stored":153465532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1376701442},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1369,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1369,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002009"},"pg_stats":[{"pgid":"2.f","version":"49'20672","reported_seq":29317,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.863984+0000","last_change":"2024-09-10T16:09:57.088531+0000","last_active":"2024-09-10T16:12:10.863984+0000","last_peered":"2024-09-10T16:12:10.863984+0000","last_clean":"2024-09-10T16:12:10.863984+0000","last_became_active":"2024-09-10T16:09:57.005755+0000","last_became_peered":"2024-09-10T16:09:57.005755+0000","last_unstale":"2024-09-10T16:12:10.863984+0000","last_undegraded":"2024-09-10T16:12:10.863984+0000","last_fullsized":"2024-09-10T16:12:10.863984+0000","mapping_epoch":54,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:09:19.329697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20672,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4259,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20844","reported_seq":21001,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.863381+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:12:10.863381+0000","last_peered":"2024-09-10T16:12:10.863381+0000","last_clean":"2024-09-10T16:12:10.863381+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:12:10.863381+0000","last_undegraded":"2024-09-10T16:12:10.863381+0000","last_fullsized":"2024-09-10T16:12:10.863381+0000","mapping_epoch":35,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18949,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20798","reported_seq":29520,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.865598+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:12:10.865598+0000","last_peered":"2024-09-10T16:12:10.865598+0000","last_clean":"2024-09-10T16:12:10.865598+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:12:10.865598+0000","last_undegraded":"2024-09-10T16:12:10.865598+0000","last_fullsized":"2024-09-10T16:12:10.865598+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18802,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20660","reported_seq":20785,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.863544+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:12:10.863544+0000","last_peered":"2024-09-10T16:12:10.863544+0000","last_clean":"2024-09-10T16:12:10.863544+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:12:10.863544+0000","last_undegraded":"2024-09-10T16:12:10.863544+0000","last_fullsized":"2024-09-10T16:12:10.863544+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20660,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20891","reported_seq":21065,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.864566+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:12:10.864566+0000","last_peered":"2024-09-10T16:12:10.864566+0000","last_clean":"2024-09-10T16:12:10.864566+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:12:10.864566+0000","last_undegraded":"2024-09-10T16:12:10.864566+0000","last_fullsized":"2024-09-10T16:12:10.864566+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17602,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'21061","reported_seq":21234,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.865406+0000","last_change":"2024-09-10T16:10:52.241702+0000","last_active":"2024-09-10T16:12:10.865406+0000","last_peered":"2024-09-10T16:12:10.865406+0000","last_clean":"2024-09-10T16:12:10.865406+0000","last_became_active":"2024-09-10T16:10:52.171236+0000","last_became_peered":"2024-09-10T16:10:52.171236+0000","last_unstale":"2024-09-10T16:12:10.865406+0000","last_undegraded":"2024-09-10T16:12:10.865406+0000","last_fullsized":"2024-09-10T16:12:10.865406+0000","mapping_epoch":58,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:04:46.778280+0000","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17673,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4372,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'21082","reported_seq":24555,"reported_epoch":65,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:12:21.253248+0000","last_change":"2024-09-10T16:12:09.399531+0000","last_active":"2024-09-10T16:12:21.253248+0000","last_peered":"2024-09-10T16:12:21.253248+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:12:21.253248+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2592,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17581,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1645,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2590}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'21314","reported_seq":21488,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.862825+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:12:10.862825+0000","last_peered":"2024-09-10T16:12:10.862825+0000","last_clean":"2024-09-10T16:12:10.862825+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:12:10.862825+0000","last_undegraded":"2024-09-10T16:12:10.862825+0000","last_fullsized":"2024-09-10T16:12:10.862825+0000","mapping_epoch":34,"log_start":"45'11300","ondisk_log_start":"45'11300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18161,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.865297+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:12:10.865297+0000","last_peered":"2024-09-10T16:12:10.865297+0000","last_clean":"2024-09-10T16:12:10.865297+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:12:10.865297+0000","last_undegraded":"2024-09-10T16:12:10.865297+0000","last_fullsized":"2024-09-10T16:12:10.865297+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20457","reported_seq":20645,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.865472+0000","last_change":"2024-09-10T16:09:33.012840+0000","last_active":"2024-09-10T16:12:10.865472+0000","last_peered":"2024-09-10T16:12:10.865472+0000","last_clean":"2024-09-10T16:12:10.865472+0000","last_became_active":"2024-09-10T16:09:32.934056+0000","last_became_peered":"2024-09-10T16:09:32.934056+0000","last_unstale":"2024-09-10T16:12:10.865472+0000","last_undegraded":"2024-09-10T16:12:10.865472+0000","last_fullsized":"2024-09-10T16:12:10.865472+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:08:30.481235+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18276,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":4579,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":138,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.862711+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:12:10.862711+0000","last_peered":"2024-09-10T16:12:10.862711+0000","last_clean":"2024-09-10T16:12:10.862711+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:12:10.862711+0000","last_undegraded":"2024-09-10T16:12:10.862711+0000","last_fullsized":"2024-09-10T16:12:10.862711+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20910","reported_seq":29607,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.864300+0000","last_change":"2024-09-10T16:09:08.909580+0000","last_active":"2024-09-10T16:12:10.864300+0000","last_peered":"2024-09-10T16:12:10.864300+0000","last_clean":"2024-09-10T16:12:10.864300+0000","last_became_active":"2024-09-10T16:09:08.854941+0000","last_became_peered":"2024-09-10T16:09:08.854941+0000","last_unstale":"2024-09-10T16:12:10.864300+0000","last_undegraded":"2024-09-10T16:12:10.864300+0000","last_fullsized":"2024-09-10T16:12:10.864300+0000","mapping_epoch":50,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:13:14.846691+0000","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17414,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4257,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":148,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.864168+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:12:10.864168+0000","last_peered":"2024-09-10T16:12:10.864168+0000","last_clean":"2024-09-10T16:12:10.864168+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:12:10.864168+0000","last_undegraded":"2024-09-10T16:12:10.864168+0000","last_fullsized":"2024-09-10T16:12:10.864168+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20781","reported_seq":29578,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.862793+0000","last_change":"2024-09-10T16:10:20.907889+0000","last_active":"2024-09-10T16:12:10.862793+0000","last_peered":"2024-09-10T16:12:10.862793+0000","last_clean":"2024-09-10T16:12:10.862793+0000","last_became_active":"2024-09-10T16:10:20.852065+0000","last_became_peered":"2024-09-10T16:10:20.852065+0000","last_unstale":"2024-09-10T16:12:10.862793+0000","last_undegraded":"2024-09-10T16:12:10.862793+0000","last_fullsized":"2024-09-10T16:12:10.862793+0000","mapping_epoch":56,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:34.069549+0000","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18461,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4307,"num_bytes_recovered":8675,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":124,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.865426+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:12:10.865426+0000","last_peered":"2024-09-10T16:12:10.865426+0000","last_clean":"2024-09-10T16:12:10.865426+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:12:10.865426+0000","last_undegraded":"2024-09-10T16:12:10.865426+0000","last_fullsized":"2024-09-10T16:12:10.865426+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'21170","reported_seq":30064,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.862524+0000","last_change":"2024-09-10T16:10:50.728349+0000","last_active":"2024-09-10T16:12:10.862524+0000","last_peered":"2024-09-10T16:12:10.862524+0000","last_clean":"2024-09-10T16:12:10.862524+0000","last_became_active":"2024-09-10T16:08:10.860438+0000","last_became_peered":"2024-09-10T16:08:10.860438+0000","last_unstale":"2024-09-10T16:12:10.862524+0000","last_undegraded":"2024-09-10T16:12:10.862524+0000","last_fullsized":"2024-09-10T16:12:10.862524+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:38:13.781380+0000","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18724,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20838","reported_seq":20995,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.956229+0000","last_change":"2024-09-10T16:12:10.956160+0000","last_active":"2024-09-10T16:12:10.956229+0000","last_peered":"2024-09-10T16:12:10.956229+0000","last_clean":"2024-09-10T16:12:10.956229+0000","last_became_active":"2024-09-10T16:12:10.866692+0000","last_became_peered":"2024-09-10T16:12:10.866692+0000","last_unstale":"2024-09-10T16:12:10.956229+0000","last_undegraded":"2024-09-10T16:12:10.956229+0000","last_fullsized":"2024-09-10T16:12:10.956229+0000","mapping_epoch":64,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:01:48.931853+0000","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17469,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4305,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.865568+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:12:10.865568+0000","last_peered":"2024-09-10T16:12:10.865568+0000","last_clean":"2024-09-10T16:12:10.865568+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:12:10.865568+0000","last_undegraded":"2024-09-10T16:12:10.865568+0000","last_fullsized":"2024-09-10T16:12:10.865568+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20598","reported_seq":29208,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.865708+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:12:10.865708+0000","last_peered":"2024-09-10T16:12:10.865708+0000","last_clean":"2024-09-10T16:12:10.865708+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:12:10.865708+0000","last_undegraded":"2024-09-10T16:12:10.865708+0000","last_fullsized":"2024-09-10T16:12:10.865708+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16988,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":138,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.863216+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:12:10.863216+0000","last_peered":"2024-09-10T16:12:10.863216+0000","last_clean":"2024-09-10T16:12:10.863216+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:12:10.863216+0000","last_undegraded":"2024-09-10T16:12:10.863216+0000","last_fullsized":"2024-09-10T16:12:10.863216+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'21048","reported_seq":21218,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.865639+0000","last_change":"2024-09-10T16:11:17.315372+0000","last_active":"2024-09-10T16:12:10.865639+0000","last_peered":"2024-09-10T16:12:10.865639+0000","last_clean":"2024-09-10T16:12:10.865639+0000","last_became_active":"2024-09-10T16:11:17.234487+0000","last_became_peered":"2024-09-10T16:11:17.234487+0000","last_unstale":"2024-09-10T16:12:10.865639+0000","last_undegraded":"2024-09-10T16:12:10.865639+0000","last_fullsized":"2024-09-10T16:12:10.865639+0000","mapping_epoch":60,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:19:00.114196+0000","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17303,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4373,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.865649+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:12:10.865649+0000","last_peered":"2024-09-10T16:12:10.865649+0000","last_clean":"2024-09-10T16:12:10.865649+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:12:10.865649+0000","last_undegraded":"2024-09-10T16:12:10.865649+0000","last_fullsized":"2024-09-10T16:12:10.865649+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":119,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.865758+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:12:10.865758+0000","last_peered":"2024-09-10T16:12:10.865758+0000","last_clean":"2024-09-10T16:12:10.865758+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:12:10.865758+0000","last_undegraded":"2024-09-10T16:12:10.865758+0000","last_fullsized":"2024-09-10T16:12:10.865758+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'21198","reported_seq":21369,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.865540+0000","last_change":"2024-09-10T16:11:44.391021+0000","last_active":"2024-09-10T16:12:10.865540+0000","last_peered":"2024-09-10T16:12:10.865540+0000","last_clean":"2024-09-10T16:12:10.865540+0000","last_became_active":"2024-09-10T16:11:44.312647+0000","last_became_peered":"2024-09-10T16:11:44.312647+0000","last_unstale":"2024-09-10T16:12:10.865540+0000","last_undegraded":"2024-09-10T16:12:10.865540+0000","last_fullsized":"2024-09-10T16:12:10.865540+0000","mapping_epoch":62,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:33:28.041611+0000","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17361,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4325,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2592,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292096,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":48798,"num_bytes_recovered":33098578,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":152272896,"data_stored":152182571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161022,"ondisk_log_size":161022,"up":32,"acting":31,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986348,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":446588,"kb_used_data":81708,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93925252,"statfs":{"total":96636764160,"available":96179458048,"internally_reserved":0,"allocated":83668992,"data_stored":83281297,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659293,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":501504,"kb_used_data":560,"kb_used_omap":4,"kb_used_meta":500923,"kb_avail":93870336,"statfs":{"total":96636764160,"available":96123224064,"internally_reserved":0,"allocated":573440,"data_stored":185281,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":512945838},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182576,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":547424,"kb_used_data":68748,"kb_used_omap":4,"kb_used_meta":478651,"kb_avail":93824416,"statfs":{"total":96636764160,"available":96076201984,"internally_reserved":0,"allocated":70397952,"data_stored":69998954,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":490139306},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":69853184,"data_stored":69817054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":82391040,"data_stored":82362136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:12:27.259 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:12:27.579 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:12:27.580 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:12:27.592 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":513,"stamp":"2024-09-10T16:12:25.803069+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2592,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292098,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":48800,"num_bytes_recovered":33098597,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161025,"ondisk_log_size":161025,"up":48,"acting":47,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1499676,"kb_used_data":151016,"kb_used_omap":12,"kb_used_meta":1348595,"kb_avail":281615844,"statfs":{"total":289910292480,"available":288374624256,"internally_reserved":0,"allocated":154640384,"data_stored":153465532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1380961282},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1369,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1369,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001948"},"pg_stats":[{"pgid":"2.f","version":"49'20672","reported_seq":29317,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.863984+0000","last_change":"2024-09-10T16:09:57.088531+0000","last_active":"2024-09-10T16:12:10.863984+0000","last_peered":"2024-09-10T16:12:10.863984+0000","last_clean":"2024-09-10T16:12:10.863984+0000","last_became_active":"2024-09-10T16:09:57.005755+0000","last_became_peered":"2024-09-10T16:09:57.005755+0000","last_unstale":"2024-09-10T16:12:10.863984+0000","last_undegraded":"2024-09-10T16:12:10.863984+0000","last_fullsized":"2024-09-10T16:12:10.863984+0000","mapping_epoch":54,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:09:19.329697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20672,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4259,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20844","reported_seq":21001,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.863381+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:12:10.863381+0000","last_peered":"2024-09-10T16:12:10.863381+0000","last_clean":"2024-09-10T16:12:10.863381+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:12:10.863381+0000","last_undegraded":"2024-09-10T16:12:10.863381+0000","last_fullsized":"2024-09-10T16:12:10.863381+0000","mapping_epoch":35,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18949,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20798","reported_seq":29520,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.865598+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:12:10.865598+0000","last_peered":"2024-09-10T16:12:10.865598+0000","last_clean":"2024-09-10T16:12:10.865598+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:12:10.865598+0000","last_undegraded":"2024-09-10T16:12:10.865598+0000","last_fullsized":"2024-09-10T16:12:10.865598+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18802,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20660","reported_seq":20785,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.863544+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:12:10.863544+0000","last_peered":"2024-09-10T16:12:10.863544+0000","last_clean":"2024-09-10T16:12:10.863544+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:12:10.863544+0000","last_undegraded":"2024-09-10T16:12:10.863544+0000","last_fullsized":"2024-09-10T16:12:10.863544+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20660,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20891","reported_seq":21065,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.864566+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:12:10.864566+0000","last_peered":"2024-09-10T16:12:10.864566+0000","last_clean":"2024-09-10T16:12:10.864566+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:12:10.864566+0000","last_undegraded":"2024-09-10T16:12:10.864566+0000","last_fullsized":"2024-09-10T16:12:10.864566+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17602,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'21061","reported_seq":21234,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.865406+0000","last_change":"2024-09-10T16:10:52.241702+0000","last_active":"2024-09-10T16:12:10.865406+0000","last_peered":"2024-09-10T16:12:10.865406+0000","last_clean":"2024-09-10T16:12:10.865406+0000","last_became_active":"2024-09-10T16:10:52.171236+0000","last_became_peered":"2024-09-10T16:10:52.171236+0000","last_unstale":"2024-09-10T16:12:10.865406+0000","last_undegraded":"2024-09-10T16:12:10.865406+0000","last_fullsized":"2024-09-10T16:12:10.865406+0000","mapping_epoch":58,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:04:46.778280+0000","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17673,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4372,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'21082","reported_seq":24555,"reported_epoch":65,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:12:21.253248+0000","last_change":"2024-09-10T16:12:09.399531+0000","last_active":"2024-09-10T16:12:21.253248+0000","last_peered":"2024-09-10T16:12:21.253248+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:12:21.253248+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2592,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17581,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1645,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2590}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'21314","reported_seq":21488,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.862825+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:12:10.862825+0000","last_peered":"2024-09-10T16:12:10.862825+0000","last_clean":"2024-09-10T16:12:10.862825+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:12:10.862825+0000","last_undegraded":"2024-09-10T16:12:10.862825+0000","last_fullsized":"2024-09-10T16:12:10.862825+0000","mapping_epoch":34,"log_start":"45'11300","ondisk_log_start":"45'11300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18161,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.865297+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:12:10.865297+0000","last_peered":"2024-09-10T16:12:10.865297+0000","last_clean":"2024-09-10T16:12:10.865297+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:12:10.865297+0000","last_undegraded":"2024-09-10T16:12:10.865297+0000","last_fullsized":"2024-09-10T16:12:10.865297+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20457","reported_seq":20645,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.865472+0000","last_change":"2024-09-10T16:09:33.012840+0000","last_active":"2024-09-10T16:12:10.865472+0000","last_peered":"2024-09-10T16:12:10.865472+0000","last_clean":"2024-09-10T16:12:10.865472+0000","last_became_active":"2024-09-10T16:09:32.934056+0000","last_became_peered":"2024-09-10T16:09:32.934056+0000","last_unstale":"2024-09-10T16:12:10.865472+0000","last_undegraded":"2024-09-10T16:12:10.865472+0000","last_fullsized":"2024-09-10T16:12:10.865472+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:08:30.481235+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18276,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":4579,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":138,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.862711+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:12:10.862711+0000","last_peered":"2024-09-10T16:12:10.862711+0000","last_clean":"2024-09-10T16:12:10.862711+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:12:10.862711+0000","last_undegraded":"2024-09-10T16:12:10.862711+0000","last_fullsized":"2024-09-10T16:12:10.862711+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20910","reported_seq":29607,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.864300+0000","last_change":"2024-09-10T16:09:08.909580+0000","last_active":"2024-09-10T16:12:10.864300+0000","last_peered":"2024-09-10T16:12:10.864300+0000","last_clean":"2024-09-10T16:12:10.864300+0000","last_became_active":"2024-09-10T16:09:08.854941+0000","last_became_peered":"2024-09-10T16:09:08.854941+0000","last_unstale":"2024-09-10T16:12:10.864300+0000","last_undegraded":"2024-09-10T16:12:10.864300+0000","last_fullsized":"2024-09-10T16:12:10.864300+0000","mapping_epoch":50,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:13:14.846691+0000","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17414,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4257,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":148,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.864168+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:12:10.864168+0000","last_peered":"2024-09-10T16:12:10.864168+0000","last_clean":"2024-09-10T16:12:10.864168+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:12:10.864168+0000","last_undegraded":"2024-09-10T16:12:10.864168+0000","last_fullsized":"2024-09-10T16:12:10.864168+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20781","reported_seq":29578,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.862793+0000","last_change":"2024-09-10T16:10:20.907889+0000","last_active":"2024-09-10T16:12:10.862793+0000","last_peered":"2024-09-10T16:12:10.862793+0000","last_clean":"2024-09-10T16:12:10.862793+0000","last_became_active":"2024-09-10T16:10:20.852065+0000","last_became_peered":"2024-09-10T16:10:20.852065+0000","last_unstale":"2024-09-10T16:12:10.862793+0000","last_undegraded":"2024-09-10T16:12:10.862793+0000","last_fullsized":"2024-09-10T16:12:10.862793+0000","mapping_epoch":56,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:34.069549+0000","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18461,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4307,"num_bytes_recovered":8675,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":124,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.865426+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:12:10.865426+0000","last_peered":"2024-09-10T16:12:10.865426+0000","last_clean":"2024-09-10T16:12:10.865426+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:12:10.865426+0000","last_undegraded":"2024-09-10T16:12:10.865426+0000","last_fullsized":"2024-09-10T16:12:10.865426+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'21170","reported_seq":30064,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.862524+0000","last_change":"2024-09-10T16:10:50.728349+0000","last_active":"2024-09-10T16:12:10.862524+0000","last_peered":"2024-09-10T16:12:10.862524+0000","last_clean":"2024-09-10T16:12:10.862524+0000","last_became_active":"2024-09-10T16:08:10.860438+0000","last_became_peered":"2024-09-10T16:08:10.860438+0000","last_unstale":"2024-09-10T16:12:10.862524+0000","last_undegraded":"2024-09-10T16:12:10.862524+0000","last_fullsized":"2024-09-10T16:12:10.862524+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:38:13.781380+0000","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18724,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20838","reported_seq":20995,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.956229+0000","last_change":"2024-09-10T16:12:10.956160+0000","last_active":"2024-09-10T16:12:10.956229+0000","last_peered":"2024-09-10T16:12:10.956229+0000","last_clean":"2024-09-10T16:12:10.956229+0000","last_became_active":"2024-09-10T16:12:10.866692+0000","last_became_peered":"2024-09-10T16:12:10.866692+0000","last_unstale":"2024-09-10T16:12:10.956229+0000","last_undegraded":"2024-09-10T16:12:10.956229+0000","last_fullsized":"2024-09-10T16:12:10.956229+0000","mapping_epoch":64,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:01:48.931853+0000","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17469,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4305,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.865568+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:12:10.865568+0000","last_peered":"2024-09-10T16:12:10.865568+0000","last_clean":"2024-09-10T16:12:10.865568+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:12:10.865568+0000","last_undegraded":"2024-09-10T16:12:10.865568+0000","last_fullsized":"2024-09-10T16:12:10.865568+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20598","reported_seq":29208,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.865708+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:12:10.865708+0000","last_peered":"2024-09-10T16:12:10.865708+0000","last_clean":"2024-09-10T16:12:10.865708+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:12:10.865708+0000","last_undegraded":"2024-09-10T16:12:10.865708+0000","last_fullsized":"2024-09-10T16:12:10.865708+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16988,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":138,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.863216+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:12:10.863216+0000","last_peered":"2024-09-10T16:12:10.863216+0000","last_clean":"2024-09-10T16:12:10.863216+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:12:10.863216+0000","last_undegraded":"2024-09-10T16:12:10.863216+0000","last_fullsized":"2024-09-10T16:12:10.863216+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'21048","reported_seq":21218,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.865639+0000","last_change":"2024-09-10T16:11:17.315372+0000","last_active":"2024-09-10T16:12:10.865639+0000","last_peered":"2024-09-10T16:12:10.865639+0000","last_clean":"2024-09-10T16:12:10.865639+0000","last_became_active":"2024-09-10T16:11:17.234487+0000","last_became_peered":"2024-09-10T16:11:17.234487+0000","last_unstale":"2024-09-10T16:12:10.865639+0000","last_undegraded":"2024-09-10T16:12:10.865639+0000","last_fullsized":"2024-09-10T16:12:10.865639+0000","mapping_epoch":60,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:19:00.114196+0000","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17303,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4373,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.865649+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:12:10.865649+0000","last_peered":"2024-09-10T16:12:10.865649+0000","last_clean":"2024-09-10T16:12:10.865649+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:12:10.865649+0000","last_undegraded":"2024-09-10T16:12:10.865649+0000","last_fullsized":"2024-09-10T16:12:10.865649+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":119,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.865758+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:12:10.865758+0000","last_peered":"2024-09-10T16:12:10.865758+0000","last_clean":"2024-09-10T16:12:10.865758+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:12:10.865758+0000","last_undegraded":"2024-09-10T16:12:10.865758+0000","last_fullsized":"2024-09-10T16:12:10.865758+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'21198","reported_seq":21369,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.865540+0000","last_change":"2024-09-10T16:11:44.391021+0000","last_active":"2024-09-10T16:12:10.865540+0000","last_peered":"2024-09-10T16:12:10.865540+0000","last_clean":"2024-09-10T16:12:10.865540+0000","last_became_active":"2024-09-10T16:11:44.312647+0000","last_became_peered":"2024-09-10T16:11:44.312647+0000","last_unstale":"2024-09-10T16:12:10.865540+0000","last_undegraded":"2024-09-10T16:12:10.865540+0000","last_fullsized":"2024-09-10T16:12:10.865540+0000","mapping_epoch":62,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:33:28.041611+0000","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17361,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4325,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2592,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292096,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":48798,"num_bytes_recovered":33098578,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":152272896,"data_stored":152182571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161022,"ondisk_log_size":161022,"up":32,"acting":31,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986348,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":446588,"kb_used_data":81708,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93925252,"statfs":{"total":96636764160,"available":96179458048,"internally_reserved":0,"allocated":83668992,"data_stored":83281297,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659294,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":505664,"kb_used_data":560,"kb_used_omap":4,"kb_used_meta":505083,"kb_avail":93866176,"statfs":{"total":96636764160,"available":96118964224,"internally_reserved":0,"allocated":573440,"data_stored":185281,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":517205678},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182576,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":547424,"kb_used_data":68748,"kb_used_omap":4,"kb_used_meta":478651,"kb_avail":93824416,"statfs":{"total":96636764160,"available":96076201984,"internally_reserved":0,"allocated":70397952,"data_stored":69998954,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":490139306},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":69853184,"data_stored":69817054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":82391040,"data_stored":82362136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:12:27.594 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:12:27.894 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:12:27.895 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:12:27.906 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":514,"stamp":"2024-09-10T16:12:27.803456+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1899,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292098,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":49493,"num_bytes_recovered":33098597,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161025,"ondisk_log_size":161025,"up":48,"acting":47,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1495596,"kb_used_data":146936,"kb_used_omap":12,"kb_used_meta":1348595,"kb_avail":281619924,"statfs":{"total":289910292480,"available":288378802176,"internally_reserved":0,"allocated":150462464,"data_stored":149287612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1380961282},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2062,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2062,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002061"},"pg_stats":[{"pgid":"2.f","version":"49'20672","reported_seq":29317,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.863984+0000","last_change":"2024-09-10T16:09:57.088531+0000","last_active":"2024-09-10T16:12:10.863984+0000","last_peered":"2024-09-10T16:12:10.863984+0000","last_clean":"2024-09-10T16:12:10.863984+0000","last_became_active":"2024-09-10T16:09:57.005755+0000","last_became_peered":"2024-09-10T16:09:57.005755+0000","last_unstale":"2024-09-10T16:12:10.863984+0000","last_undegraded":"2024-09-10T16:12:10.863984+0000","last_fullsized":"2024-09-10T16:12:10.863984+0000","mapping_epoch":54,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:09:19.329697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20672,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4259,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20844","reported_seq":21001,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.863381+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:12:10.863381+0000","last_peered":"2024-09-10T16:12:10.863381+0000","last_clean":"2024-09-10T16:12:10.863381+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:12:10.863381+0000","last_undegraded":"2024-09-10T16:12:10.863381+0000","last_fullsized":"2024-09-10T16:12:10.863381+0000","mapping_epoch":35,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18949,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20798","reported_seq":29520,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.865598+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:12:10.865598+0000","last_peered":"2024-09-10T16:12:10.865598+0000","last_clean":"2024-09-10T16:12:10.865598+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:12:10.865598+0000","last_undegraded":"2024-09-10T16:12:10.865598+0000","last_fullsized":"2024-09-10T16:12:10.865598+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18802,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20660","reported_seq":20785,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.863544+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:12:10.863544+0000","last_peered":"2024-09-10T16:12:10.863544+0000","last_clean":"2024-09-10T16:12:10.863544+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:12:10.863544+0000","last_undegraded":"2024-09-10T16:12:10.863544+0000","last_fullsized":"2024-09-10T16:12:10.863544+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20660,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20891","reported_seq":21065,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.864566+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:12:10.864566+0000","last_peered":"2024-09-10T16:12:10.864566+0000","last_clean":"2024-09-10T16:12:10.864566+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:12:10.864566+0000","last_undegraded":"2024-09-10T16:12:10.864566+0000","last_fullsized":"2024-09-10T16:12:10.864566+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17602,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'21061","reported_seq":21234,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.865406+0000","last_change":"2024-09-10T16:10:52.241702+0000","last_active":"2024-09-10T16:12:10.865406+0000","last_peered":"2024-09-10T16:12:10.865406+0000","last_clean":"2024-09-10T16:12:10.865406+0000","last_became_active":"2024-09-10T16:10:52.171236+0000","last_became_peered":"2024-09-10T16:10:52.171236+0000","last_unstale":"2024-09-10T16:12:10.865406+0000","last_undegraded":"2024-09-10T16:12:10.865406+0000","last_fullsized":"2024-09-10T16:12:10.865406+0000","mapping_epoch":58,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:04:46.778280+0000","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17673,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4372,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'21082","reported_seq":25941,"reported_epoch":65,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:12:26.253143+0000","last_change":"2024-09-10T16:12:09.399531+0000","last_active":"2024-09-10T16:12:26.253143+0000","last_peered":"2024-09-10T16:12:26.253143+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:12:26.253143+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1899,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17581,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2338,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1897}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'21314","reported_seq":21488,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.862825+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:12:10.862825+0000","last_peered":"2024-09-10T16:12:10.862825+0000","last_clean":"2024-09-10T16:12:10.862825+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:12:10.862825+0000","last_undegraded":"2024-09-10T16:12:10.862825+0000","last_fullsized":"2024-09-10T16:12:10.862825+0000","mapping_epoch":34,"log_start":"45'11300","ondisk_log_start":"45'11300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18161,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.865297+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:12:10.865297+0000","last_peered":"2024-09-10T16:12:10.865297+0000","last_clean":"2024-09-10T16:12:10.865297+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:12:10.865297+0000","last_undegraded":"2024-09-10T16:12:10.865297+0000","last_fullsized":"2024-09-10T16:12:10.865297+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20457","reported_seq":20645,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.865472+0000","last_change":"2024-09-10T16:09:33.012840+0000","last_active":"2024-09-10T16:12:10.865472+0000","last_peered":"2024-09-10T16:12:10.865472+0000","last_clean":"2024-09-10T16:12:10.865472+0000","last_became_active":"2024-09-10T16:09:32.934056+0000","last_became_peered":"2024-09-10T16:09:32.934056+0000","last_unstale":"2024-09-10T16:12:10.865472+0000","last_undegraded":"2024-09-10T16:12:10.865472+0000","last_fullsized":"2024-09-10T16:12:10.865472+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:08:30.481235+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18276,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":4579,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":138,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.862711+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:12:10.862711+0000","last_peered":"2024-09-10T16:12:10.862711+0000","last_clean":"2024-09-10T16:12:10.862711+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:12:10.862711+0000","last_undegraded":"2024-09-10T16:12:10.862711+0000","last_fullsized":"2024-09-10T16:12:10.862711+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20910","reported_seq":29607,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.864300+0000","last_change":"2024-09-10T16:09:08.909580+0000","last_active":"2024-09-10T16:12:10.864300+0000","last_peered":"2024-09-10T16:12:10.864300+0000","last_clean":"2024-09-10T16:12:10.864300+0000","last_became_active":"2024-09-10T16:09:08.854941+0000","last_became_peered":"2024-09-10T16:09:08.854941+0000","last_unstale":"2024-09-10T16:12:10.864300+0000","last_undegraded":"2024-09-10T16:12:10.864300+0000","last_fullsized":"2024-09-10T16:12:10.864300+0000","mapping_epoch":50,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:13:14.846691+0000","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17414,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4257,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":148,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.864168+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:12:10.864168+0000","last_peered":"2024-09-10T16:12:10.864168+0000","last_clean":"2024-09-10T16:12:10.864168+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:12:10.864168+0000","last_undegraded":"2024-09-10T16:12:10.864168+0000","last_fullsized":"2024-09-10T16:12:10.864168+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20781","reported_seq":29578,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.862793+0000","last_change":"2024-09-10T16:10:20.907889+0000","last_active":"2024-09-10T16:12:10.862793+0000","last_peered":"2024-09-10T16:12:10.862793+0000","last_clean":"2024-09-10T16:12:10.862793+0000","last_became_active":"2024-09-10T16:10:20.852065+0000","last_became_peered":"2024-09-10T16:10:20.852065+0000","last_unstale":"2024-09-10T16:12:10.862793+0000","last_undegraded":"2024-09-10T16:12:10.862793+0000","last_fullsized":"2024-09-10T16:12:10.862793+0000","mapping_epoch":56,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:34.069549+0000","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18461,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4307,"num_bytes_recovered":8675,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":124,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.865426+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:12:10.865426+0000","last_peered":"2024-09-10T16:12:10.865426+0000","last_clean":"2024-09-10T16:12:10.865426+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:12:10.865426+0000","last_undegraded":"2024-09-10T16:12:10.865426+0000","last_fullsized":"2024-09-10T16:12:10.865426+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'21170","reported_seq":30064,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.862524+0000","last_change":"2024-09-10T16:10:50.728349+0000","last_active":"2024-09-10T16:12:10.862524+0000","last_peered":"2024-09-10T16:12:10.862524+0000","last_clean":"2024-09-10T16:12:10.862524+0000","last_became_active":"2024-09-10T16:08:10.860438+0000","last_became_peered":"2024-09-10T16:08:10.860438+0000","last_unstale":"2024-09-10T16:12:10.862524+0000","last_undegraded":"2024-09-10T16:12:10.862524+0000","last_fullsized":"2024-09-10T16:12:10.862524+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:38:13.781380+0000","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18724,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20838","reported_seq":20995,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.956229+0000","last_change":"2024-09-10T16:12:10.956160+0000","last_active":"2024-09-10T16:12:10.956229+0000","last_peered":"2024-09-10T16:12:10.956229+0000","last_clean":"2024-09-10T16:12:10.956229+0000","last_became_active":"2024-09-10T16:12:10.866692+0000","last_became_peered":"2024-09-10T16:12:10.866692+0000","last_unstale":"2024-09-10T16:12:10.956229+0000","last_undegraded":"2024-09-10T16:12:10.956229+0000","last_fullsized":"2024-09-10T16:12:10.956229+0000","mapping_epoch":64,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:01:48.931853+0000","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17469,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4305,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.865568+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:12:10.865568+0000","last_peered":"2024-09-10T16:12:10.865568+0000","last_clean":"2024-09-10T16:12:10.865568+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:12:10.865568+0000","last_undegraded":"2024-09-10T16:12:10.865568+0000","last_fullsized":"2024-09-10T16:12:10.865568+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20598","reported_seq":29208,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.865708+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:12:10.865708+0000","last_peered":"2024-09-10T16:12:10.865708+0000","last_clean":"2024-09-10T16:12:10.865708+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:12:10.865708+0000","last_undegraded":"2024-09-10T16:12:10.865708+0000","last_fullsized":"2024-09-10T16:12:10.865708+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16988,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":138,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.863216+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:12:10.863216+0000","last_peered":"2024-09-10T16:12:10.863216+0000","last_clean":"2024-09-10T16:12:10.863216+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:12:10.863216+0000","last_undegraded":"2024-09-10T16:12:10.863216+0000","last_fullsized":"2024-09-10T16:12:10.863216+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'21048","reported_seq":21218,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.865639+0000","last_change":"2024-09-10T16:11:17.315372+0000","last_active":"2024-09-10T16:12:10.865639+0000","last_peered":"2024-09-10T16:12:10.865639+0000","last_clean":"2024-09-10T16:12:10.865639+0000","last_became_active":"2024-09-10T16:11:17.234487+0000","last_became_peered":"2024-09-10T16:11:17.234487+0000","last_unstale":"2024-09-10T16:12:10.865639+0000","last_undegraded":"2024-09-10T16:12:10.865639+0000","last_fullsized":"2024-09-10T16:12:10.865639+0000","mapping_epoch":60,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:19:00.114196+0000","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17303,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4373,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.865649+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:12:10.865649+0000","last_peered":"2024-09-10T16:12:10.865649+0000","last_clean":"2024-09-10T16:12:10.865649+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:12:10.865649+0000","last_undegraded":"2024-09-10T16:12:10.865649+0000","last_fullsized":"2024-09-10T16:12:10.865649+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":119,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.865758+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:12:10.865758+0000","last_peered":"2024-09-10T16:12:10.865758+0000","last_clean":"2024-09-10T16:12:10.865758+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:12:10.865758+0000","last_undegraded":"2024-09-10T16:12:10.865758+0000","last_fullsized":"2024-09-10T16:12:10.865758+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'21198","reported_seq":21369,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.865540+0000","last_change":"2024-09-10T16:11:44.391021+0000","last_active":"2024-09-10T16:12:10.865540+0000","last_peered":"2024-09-10T16:12:10.865540+0000","last_clean":"2024-09-10T16:12:10.865540+0000","last_became_active":"2024-09-10T16:11:44.312647+0000","last_became_peered":"2024-09-10T16:11:44.312647+0000","last_unstale":"2024-09-10T16:12:10.865540+0000","last_undegraded":"2024-09-10T16:12:10.865540+0000","last_fullsized":"2024-09-10T16:12:10.865540+0000","mapping_epoch":62,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:33:28.041611+0000","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17361,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4325,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1899,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292096,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":49491,"num_bytes_recovered":33098578,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147972096,"data_stored":147881771,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161022,"ondisk_log_size":161022,"up":32,"acting":31,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986349,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":442508,"kb_used_data":77628,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93929332,"statfs":{"total":96636764160,"available":96183635968,"internally_reserved":0,"allocated":79491072,"data_stored":79103377,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659294,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":505664,"kb_used_data":560,"kb_used_omap":4,"kb_used_meta":505083,"kb_avail":93866176,"statfs":{"total":96636764160,"available":96118964224,"internally_reserved":0,"allocated":573440,"data_stored":185281,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":517205678},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182577,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":547424,"kb_used_data":68748,"kb_used_omap":4,"kb_used_meta":478651,"kb_avail":93824416,"statfs":{"total":96636764160,"available":96076201984,"internally_reserved":0,"allocated":70397952,"data_stored":69998954,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":490139306},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":69853184,"data_stored":69817054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":78090240,"data_stored":78061336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:12:27.908 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-10T16:12:27.908 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "49'21082", 'reported_seq': 25941, 'reported_epoch': 65, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:12:26.253143+0000', 'last_change': '2024-09-10T16:12:09.399531+0000', 'last_active': '2024-09-10T16:12:26.253143+0000', 'last_peered': '2024-09-10T16:12:26.253143+0000', 'last_clean': '2024-09-10T15:59:14.139436+0000', 'last_became_active': '2024-09-10T15:59:16.318715+0000', 'last_became_peered': '2024-09-10T15:59:16.318715+0000', 'last_unstale': '2024-09-10T16:12:26.253143+0000', 'last_undegraded': '2024-09-10T15:59:15.877524+0000', 'last_fullsized': '2024-09-10T15:59:15.769042+0000', 'mapping_epoch': 42, 'log_start': "45'11000", 'ondisk_log_start': "45'11000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3537", 'last_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'objects_scrubbed': 3497, 'log_size': 10082, 'log_dups_size': 0, 'ondisk_log_size': 10082, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1224, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1899, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 17581, 'num_write_kb': 28172, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2338, 'num_bytes_recovered': 966, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1897}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:12:27.908 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T16:12:28.328 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:12:28.328 INFO:teuthology.orchestra.run.smithi115.stdout:{"fsid":"7effee29-df05-4351-9741-ee203564ed81","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 2592/24 objects degraded (10800.000%), 1 pg degraded, 1 pg undersized","count":2},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":912,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":65,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983954,"num_in_osds":3,"osd_in_since":1725983947,"num_remapped_pgs":1},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":23},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":12,"data_bytes":4849,"bytes_used":1535668224,"bytes_avail":288374624256,"bytes_total":289910292480,"degraded_objects":2592,"degraded_total":24,"degraded_ratio":108,"recovering_objects_per_sec":114,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1369,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-10T15:57:14:967120+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T16:00:13.560776+0000","services":{}},"progress_events":{}} 2024-09-10T16:12:28.343 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-10T16:12:28.343 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:12:28.654 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:12:28.654 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:12:28.667 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":514,"stamp":"2024-09-10T16:12:27.803456+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1899,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292098,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":49493,"num_bytes_recovered":33098597,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161025,"ondisk_log_size":161025,"up":48,"acting":47,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1495596,"kb_used_data":146936,"kb_used_omap":12,"kb_used_meta":1348595,"kb_avail":281619924,"statfs":{"total":289910292480,"available":288378802176,"internally_reserved":0,"allocated":150462464,"data_stored":149287612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1380961282},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2062,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2062,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002061"},"pg_stats":[{"pgid":"2.f","version":"49'20672","reported_seq":29317,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.863984+0000","last_change":"2024-09-10T16:09:57.088531+0000","last_active":"2024-09-10T16:12:10.863984+0000","last_peered":"2024-09-10T16:12:10.863984+0000","last_clean":"2024-09-10T16:12:10.863984+0000","last_became_active":"2024-09-10T16:09:57.005755+0000","last_became_peered":"2024-09-10T16:09:57.005755+0000","last_unstale":"2024-09-10T16:12:10.863984+0000","last_undegraded":"2024-09-10T16:12:10.863984+0000","last_fullsized":"2024-09-10T16:12:10.863984+0000","mapping_epoch":54,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:09:19.329697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20672,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4259,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20844","reported_seq":21001,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.863381+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:12:10.863381+0000","last_peered":"2024-09-10T16:12:10.863381+0000","last_clean":"2024-09-10T16:12:10.863381+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:12:10.863381+0000","last_undegraded":"2024-09-10T16:12:10.863381+0000","last_fullsized":"2024-09-10T16:12:10.863381+0000","mapping_epoch":35,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18949,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20798","reported_seq":29520,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.865598+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:12:10.865598+0000","last_peered":"2024-09-10T16:12:10.865598+0000","last_clean":"2024-09-10T16:12:10.865598+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:12:10.865598+0000","last_undegraded":"2024-09-10T16:12:10.865598+0000","last_fullsized":"2024-09-10T16:12:10.865598+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18802,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20660","reported_seq":20785,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.863544+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:12:10.863544+0000","last_peered":"2024-09-10T16:12:10.863544+0000","last_clean":"2024-09-10T16:12:10.863544+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:12:10.863544+0000","last_undegraded":"2024-09-10T16:12:10.863544+0000","last_fullsized":"2024-09-10T16:12:10.863544+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20660,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20891","reported_seq":21065,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.864566+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:12:10.864566+0000","last_peered":"2024-09-10T16:12:10.864566+0000","last_clean":"2024-09-10T16:12:10.864566+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:12:10.864566+0000","last_undegraded":"2024-09-10T16:12:10.864566+0000","last_fullsized":"2024-09-10T16:12:10.864566+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17602,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'21061","reported_seq":21234,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.865406+0000","last_change":"2024-09-10T16:10:52.241702+0000","last_active":"2024-09-10T16:12:10.865406+0000","last_peered":"2024-09-10T16:12:10.865406+0000","last_clean":"2024-09-10T16:12:10.865406+0000","last_became_active":"2024-09-10T16:10:52.171236+0000","last_became_peered":"2024-09-10T16:10:52.171236+0000","last_unstale":"2024-09-10T16:12:10.865406+0000","last_undegraded":"2024-09-10T16:12:10.865406+0000","last_fullsized":"2024-09-10T16:12:10.865406+0000","mapping_epoch":58,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:04:46.778280+0000","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17673,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4372,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'21082","reported_seq":25941,"reported_epoch":65,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:12:26.253143+0000","last_change":"2024-09-10T16:12:09.399531+0000","last_active":"2024-09-10T16:12:26.253143+0000","last_peered":"2024-09-10T16:12:26.253143+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:12:26.253143+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1899,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17581,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2338,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1897}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'21314","reported_seq":21488,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.862825+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:12:10.862825+0000","last_peered":"2024-09-10T16:12:10.862825+0000","last_clean":"2024-09-10T16:12:10.862825+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:12:10.862825+0000","last_undegraded":"2024-09-10T16:12:10.862825+0000","last_fullsized":"2024-09-10T16:12:10.862825+0000","mapping_epoch":34,"log_start":"45'11300","ondisk_log_start":"45'11300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18161,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.865297+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:12:10.865297+0000","last_peered":"2024-09-10T16:12:10.865297+0000","last_clean":"2024-09-10T16:12:10.865297+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:12:10.865297+0000","last_undegraded":"2024-09-10T16:12:10.865297+0000","last_fullsized":"2024-09-10T16:12:10.865297+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20457","reported_seq":20645,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.865472+0000","last_change":"2024-09-10T16:09:33.012840+0000","last_active":"2024-09-10T16:12:10.865472+0000","last_peered":"2024-09-10T16:12:10.865472+0000","last_clean":"2024-09-10T16:12:10.865472+0000","last_became_active":"2024-09-10T16:09:32.934056+0000","last_became_peered":"2024-09-10T16:09:32.934056+0000","last_unstale":"2024-09-10T16:12:10.865472+0000","last_undegraded":"2024-09-10T16:12:10.865472+0000","last_fullsized":"2024-09-10T16:12:10.865472+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:08:30.481235+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18276,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":4579,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":138,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.862711+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:12:10.862711+0000","last_peered":"2024-09-10T16:12:10.862711+0000","last_clean":"2024-09-10T16:12:10.862711+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:12:10.862711+0000","last_undegraded":"2024-09-10T16:12:10.862711+0000","last_fullsized":"2024-09-10T16:12:10.862711+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20910","reported_seq":29607,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.864300+0000","last_change":"2024-09-10T16:09:08.909580+0000","last_active":"2024-09-10T16:12:10.864300+0000","last_peered":"2024-09-10T16:12:10.864300+0000","last_clean":"2024-09-10T16:12:10.864300+0000","last_became_active":"2024-09-10T16:09:08.854941+0000","last_became_peered":"2024-09-10T16:09:08.854941+0000","last_unstale":"2024-09-10T16:12:10.864300+0000","last_undegraded":"2024-09-10T16:12:10.864300+0000","last_fullsized":"2024-09-10T16:12:10.864300+0000","mapping_epoch":50,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:13:14.846691+0000","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17414,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4257,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":148,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.864168+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:12:10.864168+0000","last_peered":"2024-09-10T16:12:10.864168+0000","last_clean":"2024-09-10T16:12:10.864168+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:12:10.864168+0000","last_undegraded":"2024-09-10T16:12:10.864168+0000","last_fullsized":"2024-09-10T16:12:10.864168+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20781","reported_seq":29578,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.862793+0000","last_change":"2024-09-10T16:10:20.907889+0000","last_active":"2024-09-10T16:12:10.862793+0000","last_peered":"2024-09-10T16:12:10.862793+0000","last_clean":"2024-09-10T16:12:10.862793+0000","last_became_active":"2024-09-10T16:10:20.852065+0000","last_became_peered":"2024-09-10T16:10:20.852065+0000","last_unstale":"2024-09-10T16:12:10.862793+0000","last_undegraded":"2024-09-10T16:12:10.862793+0000","last_fullsized":"2024-09-10T16:12:10.862793+0000","mapping_epoch":56,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:34.069549+0000","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18461,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4307,"num_bytes_recovered":8675,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":124,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.865426+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:12:10.865426+0000","last_peered":"2024-09-10T16:12:10.865426+0000","last_clean":"2024-09-10T16:12:10.865426+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:12:10.865426+0000","last_undegraded":"2024-09-10T16:12:10.865426+0000","last_fullsized":"2024-09-10T16:12:10.865426+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'21170","reported_seq":30064,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.862524+0000","last_change":"2024-09-10T16:10:50.728349+0000","last_active":"2024-09-10T16:12:10.862524+0000","last_peered":"2024-09-10T16:12:10.862524+0000","last_clean":"2024-09-10T16:12:10.862524+0000","last_became_active":"2024-09-10T16:08:10.860438+0000","last_became_peered":"2024-09-10T16:08:10.860438+0000","last_unstale":"2024-09-10T16:12:10.862524+0000","last_undegraded":"2024-09-10T16:12:10.862524+0000","last_fullsized":"2024-09-10T16:12:10.862524+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:38:13.781380+0000","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18724,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20838","reported_seq":20995,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.956229+0000","last_change":"2024-09-10T16:12:10.956160+0000","last_active":"2024-09-10T16:12:10.956229+0000","last_peered":"2024-09-10T16:12:10.956229+0000","last_clean":"2024-09-10T16:12:10.956229+0000","last_became_active":"2024-09-10T16:12:10.866692+0000","last_became_peered":"2024-09-10T16:12:10.866692+0000","last_unstale":"2024-09-10T16:12:10.956229+0000","last_undegraded":"2024-09-10T16:12:10.956229+0000","last_fullsized":"2024-09-10T16:12:10.956229+0000","mapping_epoch":64,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:01:48.931853+0000","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17469,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4305,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.865568+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:12:10.865568+0000","last_peered":"2024-09-10T16:12:10.865568+0000","last_clean":"2024-09-10T16:12:10.865568+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:12:10.865568+0000","last_undegraded":"2024-09-10T16:12:10.865568+0000","last_fullsized":"2024-09-10T16:12:10.865568+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20598","reported_seq":29208,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.865708+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:12:10.865708+0000","last_peered":"2024-09-10T16:12:10.865708+0000","last_clean":"2024-09-10T16:12:10.865708+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:12:10.865708+0000","last_undegraded":"2024-09-10T16:12:10.865708+0000","last_fullsized":"2024-09-10T16:12:10.865708+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16988,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":138,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.863216+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:12:10.863216+0000","last_peered":"2024-09-10T16:12:10.863216+0000","last_clean":"2024-09-10T16:12:10.863216+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:12:10.863216+0000","last_undegraded":"2024-09-10T16:12:10.863216+0000","last_fullsized":"2024-09-10T16:12:10.863216+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'21048","reported_seq":21218,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.865639+0000","last_change":"2024-09-10T16:11:17.315372+0000","last_active":"2024-09-10T16:12:10.865639+0000","last_peered":"2024-09-10T16:12:10.865639+0000","last_clean":"2024-09-10T16:12:10.865639+0000","last_became_active":"2024-09-10T16:11:17.234487+0000","last_became_peered":"2024-09-10T16:11:17.234487+0000","last_unstale":"2024-09-10T16:12:10.865639+0000","last_undegraded":"2024-09-10T16:12:10.865639+0000","last_fullsized":"2024-09-10T16:12:10.865639+0000","mapping_epoch":60,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:19:00.114196+0000","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17303,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4373,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.865649+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:12:10.865649+0000","last_peered":"2024-09-10T16:12:10.865649+0000","last_clean":"2024-09-10T16:12:10.865649+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:12:10.865649+0000","last_undegraded":"2024-09-10T16:12:10.865649+0000","last_fullsized":"2024-09-10T16:12:10.865649+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":119,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.865758+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:12:10.865758+0000","last_peered":"2024-09-10T16:12:10.865758+0000","last_clean":"2024-09-10T16:12:10.865758+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:12:10.865758+0000","last_undegraded":"2024-09-10T16:12:10.865758+0000","last_fullsized":"2024-09-10T16:12:10.865758+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'21198","reported_seq":21369,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.865540+0000","last_change":"2024-09-10T16:11:44.391021+0000","last_active":"2024-09-10T16:12:10.865540+0000","last_peered":"2024-09-10T16:12:10.865540+0000","last_clean":"2024-09-10T16:12:10.865540+0000","last_became_active":"2024-09-10T16:11:44.312647+0000","last_became_peered":"2024-09-10T16:11:44.312647+0000","last_unstale":"2024-09-10T16:12:10.865540+0000","last_undegraded":"2024-09-10T16:12:10.865540+0000","last_fullsized":"2024-09-10T16:12:10.865540+0000","mapping_epoch":62,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:33:28.041611+0000","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17361,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4325,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1899,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292096,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":49491,"num_bytes_recovered":33098578,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147972096,"data_stored":147881771,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161022,"ondisk_log_size":161022,"up":32,"acting":31,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986349,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":442508,"kb_used_data":77628,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93929332,"statfs":{"total":96636764160,"available":96183635968,"internally_reserved":0,"allocated":79491072,"data_stored":79103377,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659294,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":505664,"kb_used_data":560,"kb_used_omap":4,"kb_used_meta":505083,"kb_avail":93866176,"statfs":{"total":96636764160,"available":96118964224,"internally_reserved":0,"allocated":573440,"data_stored":185281,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":517205678},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182577,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":547424,"kb_used_data":68748,"kb_used_omap":4,"kb_used_meta":478651,"kb_avail":93824416,"statfs":{"total":96636764160,"available":96076201984,"internally_reserved":0,"allocated":70397952,"data_stored":69998954,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":490139306},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":69853184,"data_stored":69817054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":78090240,"data_stored":78061336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:12:31.669 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:12:31.992 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:12:31.992 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:12:32.004 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":516,"stamp":"2024-09-10T16:12:31.804126+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1023,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292098,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":50369,"num_bytes_recovered":33098597,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161025,"ondisk_log_size":161025,"up":48,"acting":47,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1483076,"kb_used_data":142736,"kb_used_omap":12,"kb_used_meta":1340275,"kb_avail":281632444,"statfs":{"total":289910292480,"available":288391622656,"internally_reserved":0,"allocated":146161664,"data_stored":144986812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1372441602},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2175,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2175,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002039"},"pg_stats":[{"pgid":"2.f","version":"49'20672","reported_seq":29317,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.863984+0000","last_change":"2024-09-10T16:09:57.088531+0000","last_active":"2024-09-10T16:12:10.863984+0000","last_peered":"2024-09-10T16:12:10.863984+0000","last_clean":"2024-09-10T16:12:10.863984+0000","last_became_active":"2024-09-10T16:09:57.005755+0000","last_became_peered":"2024-09-10T16:09:57.005755+0000","last_unstale":"2024-09-10T16:12:10.863984+0000","last_undegraded":"2024-09-10T16:12:10.863984+0000","last_fullsized":"2024-09-10T16:12:10.863984+0000","mapping_epoch":54,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:09:19.329697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20672,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4259,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20844","reported_seq":21001,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.863381+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:12:10.863381+0000","last_peered":"2024-09-10T16:12:10.863381+0000","last_clean":"2024-09-10T16:12:10.863381+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:12:10.863381+0000","last_undegraded":"2024-09-10T16:12:10.863381+0000","last_fullsized":"2024-09-10T16:12:10.863381+0000","mapping_epoch":35,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18949,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20798","reported_seq":29520,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.865598+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:12:10.865598+0000","last_peered":"2024-09-10T16:12:10.865598+0000","last_clean":"2024-09-10T16:12:10.865598+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:12:10.865598+0000","last_undegraded":"2024-09-10T16:12:10.865598+0000","last_fullsized":"2024-09-10T16:12:10.865598+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18802,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20660","reported_seq":20785,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.863544+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:12:10.863544+0000","last_peered":"2024-09-10T16:12:10.863544+0000","last_clean":"2024-09-10T16:12:10.863544+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:12:10.863544+0000","last_undegraded":"2024-09-10T16:12:10.863544+0000","last_fullsized":"2024-09-10T16:12:10.863544+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20660,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20891","reported_seq":21065,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.864566+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:12:10.864566+0000","last_peered":"2024-09-10T16:12:10.864566+0000","last_clean":"2024-09-10T16:12:10.864566+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:12:10.864566+0000","last_undegraded":"2024-09-10T16:12:10.864566+0000","last_fullsized":"2024-09-10T16:12:10.864566+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17602,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'21061","reported_seq":21234,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.865406+0000","last_change":"2024-09-10T16:10:52.241702+0000","last_active":"2024-09-10T16:12:10.865406+0000","last_peered":"2024-09-10T16:12:10.865406+0000","last_clean":"2024-09-10T16:12:10.865406+0000","last_became_active":"2024-09-10T16:10:52.171236+0000","last_became_peered":"2024-09-10T16:10:52.171236+0000","last_unstale":"2024-09-10T16:12:10.865406+0000","last_undegraded":"2024-09-10T16:12:10.865406+0000","last_fullsized":"2024-09-10T16:12:10.865406+0000","mapping_epoch":58,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:04:46.778280+0000","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17673,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4372,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'21082","reported_seq":27693,"reported_epoch":65,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:12:31.255254+0000","last_change":"2024-09-10T16:12:09.399531+0000","last_active":"2024-09-10T16:12:31.255254+0000","last_peered":"2024-09-10T16:12:31.255254+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:12:31.255254+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1023,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17581,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3214,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1021}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'21314","reported_seq":21488,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.862825+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:12:10.862825+0000","last_peered":"2024-09-10T16:12:10.862825+0000","last_clean":"2024-09-10T16:12:10.862825+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:12:10.862825+0000","last_undegraded":"2024-09-10T16:12:10.862825+0000","last_fullsized":"2024-09-10T16:12:10.862825+0000","mapping_epoch":34,"log_start":"45'11300","ondisk_log_start":"45'11300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18161,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.865297+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:12:10.865297+0000","last_peered":"2024-09-10T16:12:10.865297+0000","last_clean":"2024-09-10T16:12:10.865297+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:12:10.865297+0000","last_undegraded":"2024-09-10T16:12:10.865297+0000","last_fullsized":"2024-09-10T16:12:10.865297+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20457","reported_seq":20645,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.865472+0000","last_change":"2024-09-10T16:09:33.012840+0000","last_active":"2024-09-10T16:12:10.865472+0000","last_peered":"2024-09-10T16:12:10.865472+0000","last_clean":"2024-09-10T16:12:10.865472+0000","last_became_active":"2024-09-10T16:09:32.934056+0000","last_became_peered":"2024-09-10T16:09:32.934056+0000","last_unstale":"2024-09-10T16:12:10.865472+0000","last_undegraded":"2024-09-10T16:12:10.865472+0000","last_fullsized":"2024-09-10T16:12:10.865472+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:08:30.481235+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18276,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":4579,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":138,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.862711+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:12:10.862711+0000","last_peered":"2024-09-10T16:12:10.862711+0000","last_clean":"2024-09-10T16:12:10.862711+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:12:10.862711+0000","last_undegraded":"2024-09-10T16:12:10.862711+0000","last_fullsized":"2024-09-10T16:12:10.862711+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20910","reported_seq":29607,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.864300+0000","last_change":"2024-09-10T16:09:08.909580+0000","last_active":"2024-09-10T16:12:10.864300+0000","last_peered":"2024-09-10T16:12:10.864300+0000","last_clean":"2024-09-10T16:12:10.864300+0000","last_became_active":"2024-09-10T16:09:08.854941+0000","last_became_peered":"2024-09-10T16:09:08.854941+0000","last_unstale":"2024-09-10T16:12:10.864300+0000","last_undegraded":"2024-09-10T16:12:10.864300+0000","last_fullsized":"2024-09-10T16:12:10.864300+0000","mapping_epoch":50,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:13:14.846691+0000","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17414,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4257,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":148,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.864168+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:12:10.864168+0000","last_peered":"2024-09-10T16:12:10.864168+0000","last_clean":"2024-09-10T16:12:10.864168+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:12:10.864168+0000","last_undegraded":"2024-09-10T16:12:10.864168+0000","last_fullsized":"2024-09-10T16:12:10.864168+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20781","reported_seq":29578,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.862793+0000","last_change":"2024-09-10T16:10:20.907889+0000","last_active":"2024-09-10T16:12:10.862793+0000","last_peered":"2024-09-10T16:12:10.862793+0000","last_clean":"2024-09-10T16:12:10.862793+0000","last_became_active":"2024-09-10T16:10:20.852065+0000","last_became_peered":"2024-09-10T16:10:20.852065+0000","last_unstale":"2024-09-10T16:12:10.862793+0000","last_undegraded":"2024-09-10T16:12:10.862793+0000","last_fullsized":"2024-09-10T16:12:10.862793+0000","mapping_epoch":56,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:34.069549+0000","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18461,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4307,"num_bytes_recovered":8675,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":124,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.865426+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:12:10.865426+0000","last_peered":"2024-09-10T16:12:10.865426+0000","last_clean":"2024-09-10T16:12:10.865426+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:12:10.865426+0000","last_undegraded":"2024-09-10T16:12:10.865426+0000","last_fullsized":"2024-09-10T16:12:10.865426+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'21170","reported_seq":30064,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.862524+0000","last_change":"2024-09-10T16:10:50.728349+0000","last_active":"2024-09-10T16:12:10.862524+0000","last_peered":"2024-09-10T16:12:10.862524+0000","last_clean":"2024-09-10T16:12:10.862524+0000","last_became_active":"2024-09-10T16:08:10.860438+0000","last_became_peered":"2024-09-10T16:08:10.860438+0000","last_unstale":"2024-09-10T16:12:10.862524+0000","last_undegraded":"2024-09-10T16:12:10.862524+0000","last_fullsized":"2024-09-10T16:12:10.862524+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:38:13.781380+0000","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18724,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20838","reported_seq":20995,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.956229+0000","last_change":"2024-09-10T16:12:10.956160+0000","last_active":"2024-09-10T16:12:10.956229+0000","last_peered":"2024-09-10T16:12:10.956229+0000","last_clean":"2024-09-10T16:12:10.956229+0000","last_became_active":"2024-09-10T16:12:10.866692+0000","last_became_peered":"2024-09-10T16:12:10.866692+0000","last_unstale":"2024-09-10T16:12:10.956229+0000","last_undegraded":"2024-09-10T16:12:10.956229+0000","last_fullsized":"2024-09-10T16:12:10.956229+0000","mapping_epoch":64,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:01:48.931853+0000","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17469,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4305,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.865568+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:12:10.865568+0000","last_peered":"2024-09-10T16:12:10.865568+0000","last_clean":"2024-09-10T16:12:10.865568+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:12:10.865568+0000","last_undegraded":"2024-09-10T16:12:10.865568+0000","last_fullsized":"2024-09-10T16:12:10.865568+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20598","reported_seq":29208,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.865708+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:12:10.865708+0000","last_peered":"2024-09-10T16:12:10.865708+0000","last_clean":"2024-09-10T16:12:10.865708+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:12:10.865708+0000","last_undegraded":"2024-09-10T16:12:10.865708+0000","last_fullsized":"2024-09-10T16:12:10.865708+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16988,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":138,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.863216+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:12:10.863216+0000","last_peered":"2024-09-10T16:12:10.863216+0000","last_clean":"2024-09-10T16:12:10.863216+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:12:10.863216+0000","last_undegraded":"2024-09-10T16:12:10.863216+0000","last_fullsized":"2024-09-10T16:12:10.863216+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'21048","reported_seq":21218,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.865639+0000","last_change":"2024-09-10T16:11:17.315372+0000","last_active":"2024-09-10T16:12:10.865639+0000","last_peered":"2024-09-10T16:12:10.865639+0000","last_clean":"2024-09-10T16:12:10.865639+0000","last_became_active":"2024-09-10T16:11:17.234487+0000","last_became_peered":"2024-09-10T16:11:17.234487+0000","last_unstale":"2024-09-10T16:12:10.865639+0000","last_undegraded":"2024-09-10T16:12:10.865639+0000","last_fullsized":"2024-09-10T16:12:10.865639+0000","mapping_epoch":60,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:19:00.114196+0000","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17303,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4373,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.865649+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:12:10.865649+0000","last_peered":"2024-09-10T16:12:10.865649+0000","last_clean":"2024-09-10T16:12:10.865649+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:12:10.865649+0000","last_undegraded":"2024-09-10T16:12:10.865649+0000","last_fullsized":"2024-09-10T16:12:10.865649+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":119,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.865758+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:12:10.865758+0000","last_peered":"2024-09-10T16:12:10.865758+0000","last_clean":"2024-09-10T16:12:10.865758+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:12:10.865758+0000","last_undegraded":"2024-09-10T16:12:10.865758+0000","last_fullsized":"2024-09-10T16:12:10.865758+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'21198","reported_seq":21369,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.865540+0000","last_change":"2024-09-10T16:11:44.391021+0000","last_active":"2024-09-10T16:12:10.865540+0000","last_peered":"2024-09-10T16:12:10.865540+0000","last_clean":"2024-09-10T16:12:10.865540+0000","last_became_active":"2024-09-10T16:11:44.312647+0000","last_became_peered":"2024-09-10T16:11:44.312647+0000","last_unstale":"2024-09-10T16:12:10.865540+0000","last_undegraded":"2024-09-10T16:12:10.865540+0000","last_fullsized":"2024-09-10T16:12:10.865540+0000","mapping_epoch":62,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:33:28.041611+0000","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17361,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4325,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1023,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292096,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":50367,"num_bytes_recovered":33098578,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":143671296,"data_stored":143580971,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161022,"ondisk_log_size":161022,"up":32,"acting":31,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986350,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":438308,"kb_used_data":73428,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93933532,"statfs":{"total":96636764160,"available":96187936768,"internally_reserved":0,"allocated":75190272,"data_stored":74802577,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659295,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":497344,"kb_used_data":560,"kb_used_omap":4,"kb_used_meta":496763,"kb_avail":93874496,"statfs":{"total":96636764160,"available":96127483904,"internally_reserved":0,"allocated":573440,"data_stored":185281,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":508685998},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182578,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":547424,"kb_used_data":68748,"kb_used_omap":4,"kb_used_meta":478651,"kb_avail":93824416,"statfs":{"total":96636764160,"available":96076201984,"internally_reserved":0,"allocated":70397952,"data_stored":69998954,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":490139306},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":69853184,"data_stored":69817054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":73789440,"data_stored":73760536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:12:32.005 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:12:32.315 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:12:32.315 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:12:32.327 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":516,"stamp":"2024-09-10T16:12:31.804126+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1023,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292098,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":50369,"num_bytes_recovered":33098597,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161025,"ondisk_log_size":161025,"up":48,"acting":47,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1483076,"kb_used_data":142736,"kb_used_omap":12,"kb_used_meta":1340275,"kb_avail":281632444,"statfs":{"total":289910292480,"available":288391622656,"internally_reserved":0,"allocated":146161664,"data_stored":144986812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1372441602},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2175,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2175,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002039"},"pg_stats":[{"pgid":"2.f","version":"49'20672","reported_seq":29317,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.863984+0000","last_change":"2024-09-10T16:09:57.088531+0000","last_active":"2024-09-10T16:12:10.863984+0000","last_peered":"2024-09-10T16:12:10.863984+0000","last_clean":"2024-09-10T16:12:10.863984+0000","last_became_active":"2024-09-10T16:09:57.005755+0000","last_became_peered":"2024-09-10T16:09:57.005755+0000","last_unstale":"2024-09-10T16:12:10.863984+0000","last_undegraded":"2024-09-10T16:12:10.863984+0000","last_fullsized":"2024-09-10T16:12:10.863984+0000","mapping_epoch":54,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:09:19.329697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20672,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4259,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20844","reported_seq":21001,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.863381+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:12:10.863381+0000","last_peered":"2024-09-10T16:12:10.863381+0000","last_clean":"2024-09-10T16:12:10.863381+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:12:10.863381+0000","last_undegraded":"2024-09-10T16:12:10.863381+0000","last_fullsized":"2024-09-10T16:12:10.863381+0000","mapping_epoch":35,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18949,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20798","reported_seq":29520,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.865598+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:12:10.865598+0000","last_peered":"2024-09-10T16:12:10.865598+0000","last_clean":"2024-09-10T16:12:10.865598+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:12:10.865598+0000","last_undegraded":"2024-09-10T16:12:10.865598+0000","last_fullsized":"2024-09-10T16:12:10.865598+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18802,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20660","reported_seq":20785,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.863544+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:12:10.863544+0000","last_peered":"2024-09-10T16:12:10.863544+0000","last_clean":"2024-09-10T16:12:10.863544+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:12:10.863544+0000","last_undegraded":"2024-09-10T16:12:10.863544+0000","last_fullsized":"2024-09-10T16:12:10.863544+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20660,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20891","reported_seq":21065,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.864566+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:12:10.864566+0000","last_peered":"2024-09-10T16:12:10.864566+0000","last_clean":"2024-09-10T16:12:10.864566+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:12:10.864566+0000","last_undegraded":"2024-09-10T16:12:10.864566+0000","last_fullsized":"2024-09-10T16:12:10.864566+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17602,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'21061","reported_seq":21234,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.865406+0000","last_change":"2024-09-10T16:10:52.241702+0000","last_active":"2024-09-10T16:12:10.865406+0000","last_peered":"2024-09-10T16:12:10.865406+0000","last_clean":"2024-09-10T16:12:10.865406+0000","last_became_active":"2024-09-10T16:10:52.171236+0000","last_became_peered":"2024-09-10T16:10:52.171236+0000","last_unstale":"2024-09-10T16:12:10.865406+0000","last_undegraded":"2024-09-10T16:12:10.865406+0000","last_fullsized":"2024-09-10T16:12:10.865406+0000","mapping_epoch":58,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:04:46.778280+0000","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17673,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4372,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'21082","reported_seq":27693,"reported_epoch":65,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:12:31.255254+0000","last_change":"2024-09-10T16:12:09.399531+0000","last_active":"2024-09-10T16:12:31.255254+0000","last_peered":"2024-09-10T16:12:31.255254+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:12:31.255254+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1023,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17581,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3214,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1021}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'21314","reported_seq":21488,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.862825+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:12:10.862825+0000","last_peered":"2024-09-10T16:12:10.862825+0000","last_clean":"2024-09-10T16:12:10.862825+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:12:10.862825+0000","last_undegraded":"2024-09-10T16:12:10.862825+0000","last_fullsized":"2024-09-10T16:12:10.862825+0000","mapping_epoch":34,"log_start":"45'11300","ondisk_log_start":"45'11300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18161,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.865297+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:12:10.865297+0000","last_peered":"2024-09-10T16:12:10.865297+0000","last_clean":"2024-09-10T16:12:10.865297+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:12:10.865297+0000","last_undegraded":"2024-09-10T16:12:10.865297+0000","last_fullsized":"2024-09-10T16:12:10.865297+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20457","reported_seq":20645,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.865472+0000","last_change":"2024-09-10T16:09:33.012840+0000","last_active":"2024-09-10T16:12:10.865472+0000","last_peered":"2024-09-10T16:12:10.865472+0000","last_clean":"2024-09-10T16:12:10.865472+0000","last_became_active":"2024-09-10T16:09:32.934056+0000","last_became_peered":"2024-09-10T16:09:32.934056+0000","last_unstale":"2024-09-10T16:12:10.865472+0000","last_undegraded":"2024-09-10T16:12:10.865472+0000","last_fullsized":"2024-09-10T16:12:10.865472+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:08:30.481235+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18276,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":4579,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":138,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.862711+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:12:10.862711+0000","last_peered":"2024-09-10T16:12:10.862711+0000","last_clean":"2024-09-10T16:12:10.862711+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:12:10.862711+0000","last_undegraded":"2024-09-10T16:12:10.862711+0000","last_fullsized":"2024-09-10T16:12:10.862711+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20910","reported_seq":29607,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.864300+0000","last_change":"2024-09-10T16:09:08.909580+0000","last_active":"2024-09-10T16:12:10.864300+0000","last_peered":"2024-09-10T16:12:10.864300+0000","last_clean":"2024-09-10T16:12:10.864300+0000","last_became_active":"2024-09-10T16:09:08.854941+0000","last_became_peered":"2024-09-10T16:09:08.854941+0000","last_unstale":"2024-09-10T16:12:10.864300+0000","last_undegraded":"2024-09-10T16:12:10.864300+0000","last_fullsized":"2024-09-10T16:12:10.864300+0000","mapping_epoch":50,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:13:14.846691+0000","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17414,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4257,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":148,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.864168+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:12:10.864168+0000","last_peered":"2024-09-10T16:12:10.864168+0000","last_clean":"2024-09-10T16:12:10.864168+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:12:10.864168+0000","last_undegraded":"2024-09-10T16:12:10.864168+0000","last_fullsized":"2024-09-10T16:12:10.864168+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20781","reported_seq":29578,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.862793+0000","last_change":"2024-09-10T16:10:20.907889+0000","last_active":"2024-09-10T16:12:10.862793+0000","last_peered":"2024-09-10T16:12:10.862793+0000","last_clean":"2024-09-10T16:12:10.862793+0000","last_became_active":"2024-09-10T16:10:20.852065+0000","last_became_peered":"2024-09-10T16:10:20.852065+0000","last_unstale":"2024-09-10T16:12:10.862793+0000","last_undegraded":"2024-09-10T16:12:10.862793+0000","last_fullsized":"2024-09-10T16:12:10.862793+0000","mapping_epoch":56,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:34.069549+0000","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18461,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4307,"num_bytes_recovered":8675,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":124,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.865426+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:12:10.865426+0000","last_peered":"2024-09-10T16:12:10.865426+0000","last_clean":"2024-09-10T16:12:10.865426+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:12:10.865426+0000","last_undegraded":"2024-09-10T16:12:10.865426+0000","last_fullsized":"2024-09-10T16:12:10.865426+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'21170","reported_seq":30064,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.862524+0000","last_change":"2024-09-10T16:10:50.728349+0000","last_active":"2024-09-10T16:12:10.862524+0000","last_peered":"2024-09-10T16:12:10.862524+0000","last_clean":"2024-09-10T16:12:10.862524+0000","last_became_active":"2024-09-10T16:08:10.860438+0000","last_became_peered":"2024-09-10T16:08:10.860438+0000","last_unstale":"2024-09-10T16:12:10.862524+0000","last_undegraded":"2024-09-10T16:12:10.862524+0000","last_fullsized":"2024-09-10T16:12:10.862524+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:38:13.781380+0000","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18724,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20838","reported_seq":20995,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.956229+0000","last_change":"2024-09-10T16:12:10.956160+0000","last_active":"2024-09-10T16:12:10.956229+0000","last_peered":"2024-09-10T16:12:10.956229+0000","last_clean":"2024-09-10T16:12:10.956229+0000","last_became_active":"2024-09-10T16:12:10.866692+0000","last_became_peered":"2024-09-10T16:12:10.866692+0000","last_unstale":"2024-09-10T16:12:10.956229+0000","last_undegraded":"2024-09-10T16:12:10.956229+0000","last_fullsized":"2024-09-10T16:12:10.956229+0000","mapping_epoch":64,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:01:48.931853+0000","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17469,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4305,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.865568+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:12:10.865568+0000","last_peered":"2024-09-10T16:12:10.865568+0000","last_clean":"2024-09-10T16:12:10.865568+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:12:10.865568+0000","last_undegraded":"2024-09-10T16:12:10.865568+0000","last_fullsized":"2024-09-10T16:12:10.865568+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20598","reported_seq":29208,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.865708+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:12:10.865708+0000","last_peered":"2024-09-10T16:12:10.865708+0000","last_clean":"2024-09-10T16:12:10.865708+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:12:10.865708+0000","last_undegraded":"2024-09-10T16:12:10.865708+0000","last_fullsized":"2024-09-10T16:12:10.865708+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16988,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":138,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.863216+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:12:10.863216+0000","last_peered":"2024-09-10T16:12:10.863216+0000","last_clean":"2024-09-10T16:12:10.863216+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:12:10.863216+0000","last_undegraded":"2024-09-10T16:12:10.863216+0000","last_fullsized":"2024-09-10T16:12:10.863216+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'21048","reported_seq":21218,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.865639+0000","last_change":"2024-09-10T16:11:17.315372+0000","last_active":"2024-09-10T16:12:10.865639+0000","last_peered":"2024-09-10T16:12:10.865639+0000","last_clean":"2024-09-10T16:12:10.865639+0000","last_became_active":"2024-09-10T16:11:17.234487+0000","last_became_peered":"2024-09-10T16:11:17.234487+0000","last_unstale":"2024-09-10T16:12:10.865639+0000","last_undegraded":"2024-09-10T16:12:10.865639+0000","last_fullsized":"2024-09-10T16:12:10.865639+0000","mapping_epoch":60,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:19:00.114196+0000","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17303,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4373,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.865649+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:12:10.865649+0000","last_peered":"2024-09-10T16:12:10.865649+0000","last_clean":"2024-09-10T16:12:10.865649+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:12:10.865649+0000","last_undegraded":"2024-09-10T16:12:10.865649+0000","last_fullsized":"2024-09-10T16:12:10.865649+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":119,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.865758+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:12:10.865758+0000","last_peered":"2024-09-10T16:12:10.865758+0000","last_clean":"2024-09-10T16:12:10.865758+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:12:10.865758+0000","last_undegraded":"2024-09-10T16:12:10.865758+0000","last_fullsized":"2024-09-10T16:12:10.865758+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'21198","reported_seq":21369,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.865540+0000","last_change":"2024-09-10T16:11:44.391021+0000","last_active":"2024-09-10T16:12:10.865540+0000","last_peered":"2024-09-10T16:12:10.865540+0000","last_clean":"2024-09-10T16:12:10.865540+0000","last_became_active":"2024-09-10T16:11:44.312647+0000","last_became_peered":"2024-09-10T16:11:44.312647+0000","last_unstale":"2024-09-10T16:12:10.865540+0000","last_undegraded":"2024-09-10T16:12:10.865540+0000","last_fullsized":"2024-09-10T16:12:10.865540+0000","mapping_epoch":62,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:33:28.041611+0000","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17361,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4325,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1023,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292096,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":50367,"num_bytes_recovered":33098578,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":143671296,"data_stored":143580971,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161022,"ondisk_log_size":161022,"up":32,"acting":31,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986350,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":438308,"kb_used_data":73428,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93933532,"statfs":{"total":96636764160,"available":96187936768,"internally_reserved":0,"allocated":75190272,"data_stored":74802577,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659295,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":497344,"kb_used_data":560,"kb_used_omap":4,"kb_used_meta":496763,"kb_avail":93874496,"statfs":{"total":96636764160,"available":96127483904,"internally_reserved":0,"allocated":573440,"data_stored":185281,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":508685998},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182578,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":547424,"kb_used_data":68748,"kb_used_omap":4,"kb_used_meta":478651,"kb_avail":93824416,"statfs":{"total":96636764160,"available":96076201984,"internally_reserved":0,"allocated":70397952,"data_stored":69998954,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":490139306},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":69853184,"data_stored":69817054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":73789440,"data_stored":73760536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:12:32.328 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-10T16:12:32.328 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "49'21082", 'reported_seq': 27693, 'reported_epoch': 65, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:12:31.255254+0000', 'last_change': '2024-09-10T16:12:09.399531+0000', 'last_active': '2024-09-10T16:12:31.255254+0000', 'last_peered': '2024-09-10T16:12:31.255254+0000', 'last_clean': '2024-09-10T15:59:14.139436+0000', 'last_became_active': '2024-09-10T15:59:16.318715+0000', 'last_became_peered': '2024-09-10T15:59:16.318715+0000', 'last_unstale': '2024-09-10T16:12:31.255254+0000', 'last_undegraded': '2024-09-10T15:59:15.877524+0000', 'last_fullsized': '2024-09-10T15:59:15.769042+0000', 'mapping_epoch': 42, 'log_start': "45'11000", 'ondisk_log_start': "45'11000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3537", 'last_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'objects_scrubbed': 3497, 'log_size': 10082, 'log_dups_size': 0, 'ondisk_log_size': 10082, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1224, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1023, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 17581, 'num_write_kb': 28172, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3214, 'num_bytes_recovered': 966, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1021}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:12:32.329 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T16:12:32.699 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:12:32.699 INFO:teuthology.orchestra.run.smithi115.stdout:{"fsid":"7effee29-df05-4351-9741-ee203564ed81","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 1023/24 objects degraded (4262.500%), 1 pg degraded, 1 pg undersized","count":2},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":916,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":65,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983954,"num_in_osds":3,"osd_in_since":1725983947,"num_remapped_pgs":1},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":23},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":12,"data_bytes":4849,"bytes_used":1518669824,"bytes_avail":288391622656,"bytes_total":289910292480,"degraded_objects":1023,"degraded_total":24,"degraded_ratio":42.625,"recovering_objects_per_sec":181,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":2175,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-10T15:57:14:967120+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T16:00:13.560776+0000","services":{}},"progress_events":{}} 2024-09-10T16:12:32.713 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-10T16:12:32.713 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:12:33.041 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:12:33.041 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:12:33.054 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":516,"stamp":"2024-09-10T16:12:31.804126+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1023,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292098,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":50369,"num_bytes_recovered":33098597,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161025,"ondisk_log_size":161025,"up":48,"acting":47,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1483076,"kb_used_data":142736,"kb_used_omap":12,"kb_used_meta":1340275,"kb_avail":281632444,"statfs":{"total":289910292480,"available":288391622656,"internally_reserved":0,"allocated":146161664,"data_stored":144986812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1372441602},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2175,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2175,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002039"},"pg_stats":[{"pgid":"2.f","version":"49'20672","reported_seq":29317,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.863984+0000","last_change":"2024-09-10T16:09:57.088531+0000","last_active":"2024-09-10T16:12:10.863984+0000","last_peered":"2024-09-10T16:12:10.863984+0000","last_clean":"2024-09-10T16:12:10.863984+0000","last_became_active":"2024-09-10T16:09:57.005755+0000","last_became_peered":"2024-09-10T16:09:57.005755+0000","last_unstale":"2024-09-10T16:12:10.863984+0000","last_undegraded":"2024-09-10T16:12:10.863984+0000","last_fullsized":"2024-09-10T16:12:10.863984+0000","mapping_epoch":54,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:09:19.329697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20672,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4259,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20844","reported_seq":21001,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.863381+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:12:10.863381+0000","last_peered":"2024-09-10T16:12:10.863381+0000","last_clean":"2024-09-10T16:12:10.863381+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:12:10.863381+0000","last_undegraded":"2024-09-10T16:12:10.863381+0000","last_fullsized":"2024-09-10T16:12:10.863381+0000","mapping_epoch":35,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18949,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20798","reported_seq":29520,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.865598+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:12:10.865598+0000","last_peered":"2024-09-10T16:12:10.865598+0000","last_clean":"2024-09-10T16:12:10.865598+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:12:10.865598+0000","last_undegraded":"2024-09-10T16:12:10.865598+0000","last_fullsized":"2024-09-10T16:12:10.865598+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18802,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20660","reported_seq":20785,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.863544+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:12:10.863544+0000","last_peered":"2024-09-10T16:12:10.863544+0000","last_clean":"2024-09-10T16:12:10.863544+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:12:10.863544+0000","last_undegraded":"2024-09-10T16:12:10.863544+0000","last_fullsized":"2024-09-10T16:12:10.863544+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20660,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20891","reported_seq":21065,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.864566+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:12:10.864566+0000","last_peered":"2024-09-10T16:12:10.864566+0000","last_clean":"2024-09-10T16:12:10.864566+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:12:10.864566+0000","last_undegraded":"2024-09-10T16:12:10.864566+0000","last_fullsized":"2024-09-10T16:12:10.864566+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17602,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'21061","reported_seq":21234,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.865406+0000","last_change":"2024-09-10T16:10:52.241702+0000","last_active":"2024-09-10T16:12:10.865406+0000","last_peered":"2024-09-10T16:12:10.865406+0000","last_clean":"2024-09-10T16:12:10.865406+0000","last_became_active":"2024-09-10T16:10:52.171236+0000","last_became_peered":"2024-09-10T16:10:52.171236+0000","last_unstale":"2024-09-10T16:12:10.865406+0000","last_undegraded":"2024-09-10T16:12:10.865406+0000","last_fullsized":"2024-09-10T16:12:10.865406+0000","mapping_epoch":58,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:04:46.778280+0000","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17673,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4372,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'21082","reported_seq":27693,"reported_epoch":65,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:12:31.255254+0000","last_change":"2024-09-10T16:12:09.399531+0000","last_active":"2024-09-10T16:12:31.255254+0000","last_peered":"2024-09-10T16:12:31.255254+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:12:31.255254+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1023,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17581,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3214,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1021}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'21314","reported_seq":21488,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.862825+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:12:10.862825+0000","last_peered":"2024-09-10T16:12:10.862825+0000","last_clean":"2024-09-10T16:12:10.862825+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:12:10.862825+0000","last_undegraded":"2024-09-10T16:12:10.862825+0000","last_fullsized":"2024-09-10T16:12:10.862825+0000","mapping_epoch":34,"log_start":"45'11300","ondisk_log_start":"45'11300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18161,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.865297+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:12:10.865297+0000","last_peered":"2024-09-10T16:12:10.865297+0000","last_clean":"2024-09-10T16:12:10.865297+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:12:10.865297+0000","last_undegraded":"2024-09-10T16:12:10.865297+0000","last_fullsized":"2024-09-10T16:12:10.865297+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20457","reported_seq":20645,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.865472+0000","last_change":"2024-09-10T16:09:33.012840+0000","last_active":"2024-09-10T16:12:10.865472+0000","last_peered":"2024-09-10T16:12:10.865472+0000","last_clean":"2024-09-10T16:12:10.865472+0000","last_became_active":"2024-09-10T16:09:32.934056+0000","last_became_peered":"2024-09-10T16:09:32.934056+0000","last_unstale":"2024-09-10T16:12:10.865472+0000","last_undegraded":"2024-09-10T16:12:10.865472+0000","last_fullsized":"2024-09-10T16:12:10.865472+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:08:30.481235+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18276,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":4579,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":138,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.862711+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:12:10.862711+0000","last_peered":"2024-09-10T16:12:10.862711+0000","last_clean":"2024-09-10T16:12:10.862711+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:12:10.862711+0000","last_undegraded":"2024-09-10T16:12:10.862711+0000","last_fullsized":"2024-09-10T16:12:10.862711+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20910","reported_seq":29607,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.864300+0000","last_change":"2024-09-10T16:09:08.909580+0000","last_active":"2024-09-10T16:12:10.864300+0000","last_peered":"2024-09-10T16:12:10.864300+0000","last_clean":"2024-09-10T16:12:10.864300+0000","last_became_active":"2024-09-10T16:09:08.854941+0000","last_became_peered":"2024-09-10T16:09:08.854941+0000","last_unstale":"2024-09-10T16:12:10.864300+0000","last_undegraded":"2024-09-10T16:12:10.864300+0000","last_fullsized":"2024-09-10T16:12:10.864300+0000","mapping_epoch":50,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:13:14.846691+0000","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17414,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4257,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":148,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.864168+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:12:10.864168+0000","last_peered":"2024-09-10T16:12:10.864168+0000","last_clean":"2024-09-10T16:12:10.864168+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:12:10.864168+0000","last_undegraded":"2024-09-10T16:12:10.864168+0000","last_fullsized":"2024-09-10T16:12:10.864168+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20781","reported_seq":29578,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.862793+0000","last_change":"2024-09-10T16:10:20.907889+0000","last_active":"2024-09-10T16:12:10.862793+0000","last_peered":"2024-09-10T16:12:10.862793+0000","last_clean":"2024-09-10T16:12:10.862793+0000","last_became_active":"2024-09-10T16:10:20.852065+0000","last_became_peered":"2024-09-10T16:10:20.852065+0000","last_unstale":"2024-09-10T16:12:10.862793+0000","last_undegraded":"2024-09-10T16:12:10.862793+0000","last_fullsized":"2024-09-10T16:12:10.862793+0000","mapping_epoch":56,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:34.069549+0000","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18461,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4307,"num_bytes_recovered":8675,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":124,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.865426+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:12:10.865426+0000","last_peered":"2024-09-10T16:12:10.865426+0000","last_clean":"2024-09-10T16:12:10.865426+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:12:10.865426+0000","last_undegraded":"2024-09-10T16:12:10.865426+0000","last_fullsized":"2024-09-10T16:12:10.865426+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'21170","reported_seq":30064,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.862524+0000","last_change":"2024-09-10T16:10:50.728349+0000","last_active":"2024-09-10T16:12:10.862524+0000","last_peered":"2024-09-10T16:12:10.862524+0000","last_clean":"2024-09-10T16:12:10.862524+0000","last_became_active":"2024-09-10T16:08:10.860438+0000","last_became_peered":"2024-09-10T16:08:10.860438+0000","last_unstale":"2024-09-10T16:12:10.862524+0000","last_undegraded":"2024-09-10T16:12:10.862524+0000","last_fullsized":"2024-09-10T16:12:10.862524+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:38:13.781380+0000","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18724,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20838","reported_seq":20995,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.956229+0000","last_change":"2024-09-10T16:12:10.956160+0000","last_active":"2024-09-10T16:12:10.956229+0000","last_peered":"2024-09-10T16:12:10.956229+0000","last_clean":"2024-09-10T16:12:10.956229+0000","last_became_active":"2024-09-10T16:12:10.866692+0000","last_became_peered":"2024-09-10T16:12:10.866692+0000","last_unstale":"2024-09-10T16:12:10.956229+0000","last_undegraded":"2024-09-10T16:12:10.956229+0000","last_fullsized":"2024-09-10T16:12:10.956229+0000","mapping_epoch":64,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:01:48.931853+0000","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17469,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4305,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.865568+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:12:10.865568+0000","last_peered":"2024-09-10T16:12:10.865568+0000","last_clean":"2024-09-10T16:12:10.865568+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:12:10.865568+0000","last_undegraded":"2024-09-10T16:12:10.865568+0000","last_fullsized":"2024-09-10T16:12:10.865568+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20598","reported_seq":29208,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.865708+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:12:10.865708+0000","last_peered":"2024-09-10T16:12:10.865708+0000","last_clean":"2024-09-10T16:12:10.865708+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:12:10.865708+0000","last_undegraded":"2024-09-10T16:12:10.865708+0000","last_fullsized":"2024-09-10T16:12:10.865708+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16988,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":138,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.863216+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:12:10.863216+0000","last_peered":"2024-09-10T16:12:10.863216+0000","last_clean":"2024-09-10T16:12:10.863216+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:12:10.863216+0000","last_undegraded":"2024-09-10T16:12:10.863216+0000","last_fullsized":"2024-09-10T16:12:10.863216+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'21048","reported_seq":21218,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.865639+0000","last_change":"2024-09-10T16:11:17.315372+0000","last_active":"2024-09-10T16:12:10.865639+0000","last_peered":"2024-09-10T16:12:10.865639+0000","last_clean":"2024-09-10T16:12:10.865639+0000","last_became_active":"2024-09-10T16:11:17.234487+0000","last_became_peered":"2024-09-10T16:11:17.234487+0000","last_unstale":"2024-09-10T16:12:10.865639+0000","last_undegraded":"2024-09-10T16:12:10.865639+0000","last_fullsized":"2024-09-10T16:12:10.865639+0000","mapping_epoch":60,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:19:00.114196+0000","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17303,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4373,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.865649+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:12:10.865649+0000","last_peered":"2024-09-10T16:12:10.865649+0000","last_clean":"2024-09-10T16:12:10.865649+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:12:10.865649+0000","last_undegraded":"2024-09-10T16:12:10.865649+0000","last_fullsized":"2024-09-10T16:12:10.865649+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":119,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.865758+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:12:10.865758+0000","last_peered":"2024-09-10T16:12:10.865758+0000","last_clean":"2024-09-10T16:12:10.865758+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:12:10.865758+0000","last_undegraded":"2024-09-10T16:12:10.865758+0000","last_fullsized":"2024-09-10T16:12:10.865758+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'21198","reported_seq":21369,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.865540+0000","last_change":"2024-09-10T16:11:44.391021+0000","last_active":"2024-09-10T16:12:10.865540+0000","last_peered":"2024-09-10T16:12:10.865540+0000","last_clean":"2024-09-10T16:12:10.865540+0000","last_became_active":"2024-09-10T16:11:44.312647+0000","last_became_peered":"2024-09-10T16:11:44.312647+0000","last_unstale":"2024-09-10T16:12:10.865540+0000","last_undegraded":"2024-09-10T16:12:10.865540+0000","last_fullsized":"2024-09-10T16:12:10.865540+0000","mapping_epoch":62,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:33:28.041611+0000","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17361,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4325,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1023,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292096,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":50367,"num_bytes_recovered":33098578,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":143671296,"data_stored":143580971,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161022,"ondisk_log_size":161022,"up":32,"acting":31,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986350,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":438308,"kb_used_data":73428,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93933532,"statfs":{"total":96636764160,"available":96187936768,"internally_reserved":0,"allocated":75190272,"data_stored":74802577,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659295,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":497344,"kb_used_data":560,"kb_used_omap":4,"kb_used_meta":496763,"kb_avail":93874496,"statfs":{"total":96636764160,"available":96127483904,"internally_reserved":0,"allocated":573440,"data_stored":185281,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":508685998},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182578,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":547424,"kb_used_data":68748,"kb_used_omap":4,"kb_used_meta":478651,"kb_avail":93824416,"statfs":{"total":96636764160,"available":96076201984,"internally_reserved":0,"allocated":70397952,"data_stored":69998954,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":490139306},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":69853184,"data_stored":69817054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":73789440,"data_stored":73760536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:12:36.056 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:12:36.340 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:12:36.341 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:12:36.352 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":518,"stamp":"2024-09-10T16:12:35.804799+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1023,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292098,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":50369,"num_bytes_recovered":33098597,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161025,"ondisk_log_size":161025,"up":48,"acting":47,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1491396,"kb_used_data":142736,"kb_used_omap":12,"kb_used_meta":1348595,"kb_avail":281624124,"statfs":{"total":289910292480,"available":288383102976,"internally_reserved":0,"allocated":146161664,"data_stored":144986812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1380961282},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1569,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1569,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002009"},"pg_stats":[{"pgid":"2.f","version":"49'20672","reported_seq":29317,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.863984+0000","last_change":"2024-09-10T16:09:57.088531+0000","last_active":"2024-09-10T16:12:10.863984+0000","last_peered":"2024-09-10T16:12:10.863984+0000","last_clean":"2024-09-10T16:12:10.863984+0000","last_became_active":"2024-09-10T16:09:57.005755+0000","last_became_peered":"2024-09-10T16:09:57.005755+0000","last_unstale":"2024-09-10T16:12:10.863984+0000","last_undegraded":"2024-09-10T16:12:10.863984+0000","last_fullsized":"2024-09-10T16:12:10.863984+0000","mapping_epoch":54,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:09:19.329697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20672,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4259,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20844","reported_seq":21001,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.863381+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:12:10.863381+0000","last_peered":"2024-09-10T16:12:10.863381+0000","last_clean":"2024-09-10T16:12:10.863381+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:12:10.863381+0000","last_undegraded":"2024-09-10T16:12:10.863381+0000","last_fullsized":"2024-09-10T16:12:10.863381+0000","mapping_epoch":35,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18949,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20798","reported_seq":29520,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.865598+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:12:10.865598+0000","last_peered":"2024-09-10T16:12:10.865598+0000","last_clean":"2024-09-10T16:12:10.865598+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:12:10.865598+0000","last_undegraded":"2024-09-10T16:12:10.865598+0000","last_fullsized":"2024-09-10T16:12:10.865598+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18802,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20660","reported_seq":20785,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.863544+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:12:10.863544+0000","last_peered":"2024-09-10T16:12:10.863544+0000","last_clean":"2024-09-10T16:12:10.863544+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:12:10.863544+0000","last_undegraded":"2024-09-10T16:12:10.863544+0000","last_fullsized":"2024-09-10T16:12:10.863544+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20660,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20891","reported_seq":21065,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.864566+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:12:10.864566+0000","last_peered":"2024-09-10T16:12:10.864566+0000","last_clean":"2024-09-10T16:12:10.864566+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:12:10.864566+0000","last_undegraded":"2024-09-10T16:12:10.864566+0000","last_fullsized":"2024-09-10T16:12:10.864566+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17602,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'21061","reported_seq":21234,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.865406+0000","last_change":"2024-09-10T16:10:52.241702+0000","last_active":"2024-09-10T16:12:10.865406+0000","last_peered":"2024-09-10T16:12:10.865406+0000","last_clean":"2024-09-10T16:12:10.865406+0000","last_became_active":"2024-09-10T16:10:52.171236+0000","last_became_peered":"2024-09-10T16:10:52.171236+0000","last_unstale":"2024-09-10T16:12:10.865406+0000","last_undegraded":"2024-09-10T16:12:10.865406+0000","last_fullsized":"2024-09-10T16:12:10.865406+0000","mapping_epoch":58,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:04:46.778280+0000","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17673,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4372,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'21082","reported_seq":27693,"reported_epoch":65,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:12:31.255254+0000","last_change":"2024-09-10T16:12:09.399531+0000","last_active":"2024-09-10T16:12:31.255254+0000","last_peered":"2024-09-10T16:12:31.255254+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:12:31.255254+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1023,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17581,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3214,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1021}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'21314","reported_seq":21488,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.862825+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:12:10.862825+0000","last_peered":"2024-09-10T16:12:10.862825+0000","last_clean":"2024-09-10T16:12:10.862825+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:12:10.862825+0000","last_undegraded":"2024-09-10T16:12:10.862825+0000","last_fullsized":"2024-09-10T16:12:10.862825+0000","mapping_epoch":34,"log_start":"45'11300","ondisk_log_start":"45'11300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18161,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.865297+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:12:10.865297+0000","last_peered":"2024-09-10T16:12:10.865297+0000","last_clean":"2024-09-10T16:12:10.865297+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:12:10.865297+0000","last_undegraded":"2024-09-10T16:12:10.865297+0000","last_fullsized":"2024-09-10T16:12:10.865297+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20457","reported_seq":20645,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.865472+0000","last_change":"2024-09-10T16:09:33.012840+0000","last_active":"2024-09-10T16:12:10.865472+0000","last_peered":"2024-09-10T16:12:10.865472+0000","last_clean":"2024-09-10T16:12:10.865472+0000","last_became_active":"2024-09-10T16:09:32.934056+0000","last_became_peered":"2024-09-10T16:09:32.934056+0000","last_unstale":"2024-09-10T16:12:10.865472+0000","last_undegraded":"2024-09-10T16:12:10.865472+0000","last_fullsized":"2024-09-10T16:12:10.865472+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:08:30.481235+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18276,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":4579,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":138,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.862711+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:12:10.862711+0000","last_peered":"2024-09-10T16:12:10.862711+0000","last_clean":"2024-09-10T16:12:10.862711+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:12:10.862711+0000","last_undegraded":"2024-09-10T16:12:10.862711+0000","last_fullsized":"2024-09-10T16:12:10.862711+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20910","reported_seq":29607,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.864300+0000","last_change":"2024-09-10T16:09:08.909580+0000","last_active":"2024-09-10T16:12:10.864300+0000","last_peered":"2024-09-10T16:12:10.864300+0000","last_clean":"2024-09-10T16:12:10.864300+0000","last_became_active":"2024-09-10T16:09:08.854941+0000","last_became_peered":"2024-09-10T16:09:08.854941+0000","last_unstale":"2024-09-10T16:12:10.864300+0000","last_undegraded":"2024-09-10T16:12:10.864300+0000","last_fullsized":"2024-09-10T16:12:10.864300+0000","mapping_epoch":50,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:13:14.846691+0000","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17414,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4257,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":148,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.864168+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:12:10.864168+0000","last_peered":"2024-09-10T16:12:10.864168+0000","last_clean":"2024-09-10T16:12:10.864168+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:12:10.864168+0000","last_undegraded":"2024-09-10T16:12:10.864168+0000","last_fullsized":"2024-09-10T16:12:10.864168+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20781","reported_seq":29578,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.862793+0000","last_change":"2024-09-10T16:10:20.907889+0000","last_active":"2024-09-10T16:12:10.862793+0000","last_peered":"2024-09-10T16:12:10.862793+0000","last_clean":"2024-09-10T16:12:10.862793+0000","last_became_active":"2024-09-10T16:10:20.852065+0000","last_became_peered":"2024-09-10T16:10:20.852065+0000","last_unstale":"2024-09-10T16:12:10.862793+0000","last_undegraded":"2024-09-10T16:12:10.862793+0000","last_fullsized":"2024-09-10T16:12:10.862793+0000","mapping_epoch":56,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:34.069549+0000","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18461,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4307,"num_bytes_recovered":8675,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":124,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.865426+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:12:10.865426+0000","last_peered":"2024-09-10T16:12:10.865426+0000","last_clean":"2024-09-10T16:12:10.865426+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:12:10.865426+0000","last_undegraded":"2024-09-10T16:12:10.865426+0000","last_fullsized":"2024-09-10T16:12:10.865426+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'21170","reported_seq":30064,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.862524+0000","last_change":"2024-09-10T16:10:50.728349+0000","last_active":"2024-09-10T16:12:10.862524+0000","last_peered":"2024-09-10T16:12:10.862524+0000","last_clean":"2024-09-10T16:12:10.862524+0000","last_became_active":"2024-09-10T16:08:10.860438+0000","last_became_peered":"2024-09-10T16:08:10.860438+0000","last_unstale":"2024-09-10T16:12:10.862524+0000","last_undegraded":"2024-09-10T16:12:10.862524+0000","last_fullsized":"2024-09-10T16:12:10.862524+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:38:13.781380+0000","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18724,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20838","reported_seq":20995,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.956229+0000","last_change":"2024-09-10T16:12:10.956160+0000","last_active":"2024-09-10T16:12:10.956229+0000","last_peered":"2024-09-10T16:12:10.956229+0000","last_clean":"2024-09-10T16:12:10.956229+0000","last_became_active":"2024-09-10T16:12:10.866692+0000","last_became_peered":"2024-09-10T16:12:10.866692+0000","last_unstale":"2024-09-10T16:12:10.956229+0000","last_undegraded":"2024-09-10T16:12:10.956229+0000","last_fullsized":"2024-09-10T16:12:10.956229+0000","mapping_epoch":64,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:01:48.931853+0000","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17469,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4305,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.865568+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:12:10.865568+0000","last_peered":"2024-09-10T16:12:10.865568+0000","last_clean":"2024-09-10T16:12:10.865568+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:12:10.865568+0000","last_undegraded":"2024-09-10T16:12:10.865568+0000","last_fullsized":"2024-09-10T16:12:10.865568+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20598","reported_seq":29208,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.865708+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:12:10.865708+0000","last_peered":"2024-09-10T16:12:10.865708+0000","last_clean":"2024-09-10T16:12:10.865708+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:12:10.865708+0000","last_undegraded":"2024-09-10T16:12:10.865708+0000","last_fullsized":"2024-09-10T16:12:10.865708+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16988,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":138,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.863216+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:12:10.863216+0000","last_peered":"2024-09-10T16:12:10.863216+0000","last_clean":"2024-09-10T16:12:10.863216+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:12:10.863216+0000","last_undegraded":"2024-09-10T16:12:10.863216+0000","last_fullsized":"2024-09-10T16:12:10.863216+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'21048","reported_seq":21218,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.865639+0000","last_change":"2024-09-10T16:11:17.315372+0000","last_active":"2024-09-10T16:12:10.865639+0000","last_peered":"2024-09-10T16:12:10.865639+0000","last_clean":"2024-09-10T16:12:10.865639+0000","last_became_active":"2024-09-10T16:11:17.234487+0000","last_became_peered":"2024-09-10T16:11:17.234487+0000","last_unstale":"2024-09-10T16:12:10.865639+0000","last_undegraded":"2024-09-10T16:12:10.865639+0000","last_fullsized":"2024-09-10T16:12:10.865639+0000","mapping_epoch":60,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:19:00.114196+0000","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17303,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4373,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.865649+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:12:10.865649+0000","last_peered":"2024-09-10T16:12:10.865649+0000","last_clean":"2024-09-10T16:12:10.865649+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:12:10.865649+0000","last_undegraded":"2024-09-10T16:12:10.865649+0000","last_fullsized":"2024-09-10T16:12:10.865649+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":119,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.865758+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:12:10.865758+0000","last_peered":"2024-09-10T16:12:10.865758+0000","last_clean":"2024-09-10T16:12:10.865758+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:12:10.865758+0000","last_undegraded":"2024-09-10T16:12:10.865758+0000","last_fullsized":"2024-09-10T16:12:10.865758+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'21198","reported_seq":21369,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.865540+0000","last_change":"2024-09-10T16:11:44.391021+0000","last_active":"2024-09-10T16:12:10.865540+0000","last_peered":"2024-09-10T16:12:10.865540+0000","last_clean":"2024-09-10T16:12:10.865540+0000","last_became_active":"2024-09-10T16:11:44.312647+0000","last_became_peered":"2024-09-10T16:11:44.312647+0000","last_unstale":"2024-09-10T16:12:10.865540+0000","last_undegraded":"2024-09-10T16:12:10.865540+0000","last_fullsized":"2024-09-10T16:12:10.865540+0000","mapping_epoch":62,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:33:28.041611+0000","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17361,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4325,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1023,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292096,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":50367,"num_bytes_recovered":33098578,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":143671296,"data_stored":143580971,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161022,"ondisk_log_size":161022,"up":32,"acting":31,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986350,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":438308,"kb_used_data":73428,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93933532,"statfs":{"total":96636764160,"available":96187936768,"internally_reserved":0,"allocated":75190272,"data_stored":74802577,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659296,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":505664,"kb_used_data":560,"kb_used_omap":4,"kb_used_meta":505083,"kb_avail":93866176,"statfs":{"total":96636764160,"available":96118964224,"internally_reserved":0,"allocated":573440,"data_stored":185281,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":517205678},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182578,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":547424,"kb_used_data":68748,"kb_used_omap":4,"kb_used_meta":478651,"kb_avail":93824416,"statfs":{"total":96636764160,"available":96076201984,"internally_reserved":0,"allocated":70397952,"data_stored":69998954,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":490139306},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":69853184,"data_stored":69817054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":73789440,"data_stored":73760536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:12:36.354 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:12:36.638 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:12:36.638 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:12:36.650 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":519,"stamp":"2024-09-10T16:12:36.464267+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292098,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51392,"num_bytes_recovered":33098597,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161025,"ondisk_log_size":161025,"up":48,"acting":47,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1486728,"kb_used_data":138072,"kb_used_omap":12,"kb_used_meta":1348595,"kb_avail":281628792,"statfs":{"total":289910292480,"available":288387883008,"internally_reserved":0,"allocated":141385728,"data_stored":140210876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1380961282},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2592,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2592,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.661197"},"pg_stats":[{"pgid":"2.f","version":"49'20672","reported_seq":29317,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.863984+0000","last_change":"2024-09-10T16:09:57.088531+0000","last_active":"2024-09-10T16:12:10.863984+0000","last_peered":"2024-09-10T16:12:10.863984+0000","last_clean":"2024-09-10T16:12:10.863984+0000","last_became_active":"2024-09-10T16:09:57.005755+0000","last_became_peered":"2024-09-10T16:09:57.005755+0000","last_unstale":"2024-09-10T16:12:10.863984+0000","last_undegraded":"2024-09-10T16:12:10.863984+0000","last_fullsized":"2024-09-10T16:12:10.863984+0000","mapping_epoch":54,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:09:19.329697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20672,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4259,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20844","reported_seq":21001,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.863381+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:12:10.863381+0000","last_peered":"2024-09-10T16:12:10.863381+0000","last_clean":"2024-09-10T16:12:10.863381+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:12:10.863381+0000","last_undegraded":"2024-09-10T16:12:10.863381+0000","last_fullsized":"2024-09-10T16:12:10.863381+0000","mapping_epoch":35,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18949,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20798","reported_seq":29520,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.865598+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:12:10.865598+0000","last_peered":"2024-09-10T16:12:10.865598+0000","last_clean":"2024-09-10T16:12:10.865598+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:12:10.865598+0000","last_undegraded":"2024-09-10T16:12:10.865598+0000","last_fullsized":"2024-09-10T16:12:10.865598+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18802,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20660","reported_seq":20785,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.863544+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:12:10.863544+0000","last_peered":"2024-09-10T16:12:10.863544+0000","last_clean":"2024-09-10T16:12:10.863544+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:12:10.863544+0000","last_undegraded":"2024-09-10T16:12:10.863544+0000","last_fullsized":"2024-09-10T16:12:10.863544+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20660,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20891","reported_seq":21065,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.864566+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:12:10.864566+0000","last_peered":"2024-09-10T16:12:10.864566+0000","last_clean":"2024-09-10T16:12:10.864566+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:12:10.864566+0000","last_undegraded":"2024-09-10T16:12:10.864566+0000","last_fullsized":"2024-09-10T16:12:10.864566+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17602,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'21061","reported_seq":21234,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.865406+0000","last_change":"2024-09-10T16:10:52.241702+0000","last_active":"2024-09-10T16:12:10.865406+0000","last_peered":"2024-09-10T16:12:10.865406+0000","last_clean":"2024-09-10T16:12:10.865406+0000","last_became_active":"2024-09-10T16:10:52.171236+0000","last_became_peered":"2024-09-10T16:10:52.171236+0000","last_unstale":"2024-09-10T16:12:10.865406+0000","last_undegraded":"2024-09-10T16:12:10.865406+0000","last_fullsized":"2024-09-10T16:12:10.865406+0000","mapping_epoch":58,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:04:46.778280+0000","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17673,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4372,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'21082","reported_seq":29740,"reported_epoch":65,"state":"active+undersized+remapped","last_fresh":"2024-09-10T16:12:35.687487+0000","last_change":"2024-09-10T16:12:35.687652+0000","last_active":"2024-09-10T16:12:35.687487+0000","last_peered":"2024-09-10T16:12:35.687487+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:12:35.687487+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17581,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4237,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","1","2"],"object_location_counts":[{"shards":"0","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'21314","reported_seq":21488,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.862825+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:12:10.862825+0000","last_peered":"2024-09-10T16:12:10.862825+0000","last_clean":"2024-09-10T16:12:10.862825+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:12:10.862825+0000","last_undegraded":"2024-09-10T16:12:10.862825+0000","last_fullsized":"2024-09-10T16:12:10.862825+0000","mapping_epoch":34,"log_start":"45'11300","ondisk_log_start":"45'11300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18161,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.865297+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:12:10.865297+0000","last_peered":"2024-09-10T16:12:10.865297+0000","last_clean":"2024-09-10T16:12:10.865297+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:12:10.865297+0000","last_undegraded":"2024-09-10T16:12:10.865297+0000","last_fullsized":"2024-09-10T16:12:10.865297+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20457","reported_seq":20645,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.865472+0000","last_change":"2024-09-10T16:09:33.012840+0000","last_active":"2024-09-10T16:12:10.865472+0000","last_peered":"2024-09-10T16:12:10.865472+0000","last_clean":"2024-09-10T16:12:10.865472+0000","last_became_active":"2024-09-10T16:09:32.934056+0000","last_became_peered":"2024-09-10T16:09:32.934056+0000","last_unstale":"2024-09-10T16:12:10.865472+0000","last_undegraded":"2024-09-10T16:12:10.865472+0000","last_fullsized":"2024-09-10T16:12:10.865472+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:08:30.481235+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18276,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":4579,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":138,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.862711+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:12:10.862711+0000","last_peered":"2024-09-10T16:12:10.862711+0000","last_clean":"2024-09-10T16:12:10.862711+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:12:10.862711+0000","last_undegraded":"2024-09-10T16:12:10.862711+0000","last_fullsized":"2024-09-10T16:12:10.862711+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20910","reported_seq":29607,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.864300+0000","last_change":"2024-09-10T16:09:08.909580+0000","last_active":"2024-09-10T16:12:10.864300+0000","last_peered":"2024-09-10T16:12:10.864300+0000","last_clean":"2024-09-10T16:12:10.864300+0000","last_became_active":"2024-09-10T16:09:08.854941+0000","last_became_peered":"2024-09-10T16:09:08.854941+0000","last_unstale":"2024-09-10T16:12:10.864300+0000","last_undegraded":"2024-09-10T16:12:10.864300+0000","last_fullsized":"2024-09-10T16:12:10.864300+0000","mapping_epoch":50,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:13:14.846691+0000","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17414,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4257,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":148,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.864168+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:12:10.864168+0000","last_peered":"2024-09-10T16:12:10.864168+0000","last_clean":"2024-09-10T16:12:10.864168+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:12:10.864168+0000","last_undegraded":"2024-09-10T16:12:10.864168+0000","last_fullsized":"2024-09-10T16:12:10.864168+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20781","reported_seq":29578,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.862793+0000","last_change":"2024-09-10T16:10:20.907889+0000","last_active":"2024-09-10T16:12:10.862793+0000","last_peered":"2024-09-10T16:12:10.862793+0000","last_clean":"2024-09-10T16:12:10.862793+0000","last_became_active":"2024-09-10T16:10:20.852065+0000","last_became_peered":"2024-09-10T16:10:20.852065+0000","last_unstale":"2024-09-10T16:12:10.862793+0000","last_undegraded":"2024-09-10T16:12:10.862793+0000","last_fullsized":"2024-09-10T16:12:10.862793+0000","mapping_epoch":56,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:34.069549+0000","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18461,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4307,"num_bytes_recovered":8675,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":124,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.865426+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:12:10.865426+0000","last_peered":"2024-09-10T16:12:10.865426+0000","last_clean":"2024-09-10T16:12:10.865426+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:12:10.865426+0000","last_undegraded":"2024-09-10T16:12:10.865426+0000","last_fullsized":"2024-09-10T16:12:10.865426+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'21170","reported_seq":30064,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.862524+0000","last_change":"2024-09-10T16:10:50.728349+0000","last_active":"2024-09-10T16:12:10.862524+0000","last_peered":"2024-09-10T16:12:10.862524+0000","last_clean":"2024-09-10T16:12:10.862524+0000","last_became_active":"2024-09-10T16:08:10.860438+0000","last_became_peered":"2024-09-10T16:08:10.860438+0000","last_unstale":"2024-09-10T16:12:10.862524+0000","last_undegraded":"2024-09-10T16:12:10.862524+0000","last_fullsized":"2024-09-10T16:12:10.862524+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:38:13.781380+0000","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18724,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20838","reported_seq":20995,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.956229+0000","last_change":"2024-09-10T16:12:10.956160+0000","last_active":"2024-09-10T16:12:10.956229+0000","last_peered":"2024-09-10T16:12:10.956229+0000","last_clean":"2024-09-10T16:12:10.956229+0000","last_became_active":"2024-09-10T16:12:10.866692+0000","last_became_peered":"2024-09-10T16:12:10.866692+0000","last_unstale":"2024-09-10T16:12:10.956229+0000","last_undegraded":"2024-09-10T16:12:10.956229+0000","last_fullsized":"2024-09-10T16:12:10.956229+0000","mapping_epoch":64,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:01:48.931853+0000","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17469,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4305,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.865568+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:12:10.865568+0000","last_peered":"2024-09-10T16:12:10.865568+0000","last_clean":"2024-09-10T16:12:10.865568+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:12:10.865568+0000","last_undegraded":"2024-09-10T16:12:10.865568+0000","last_fullsized":"2024-09-10T16:12:10.865568+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20598","reported_seq":29208,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.865708+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:12:10.865708+0000","last_peered":"2024-09-10T16:12:10.865708+0000","last_clean":"2024-09-10T16:12:10.865708+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:12:10.865708+0000","last_undegraded":"2024-09-10T16:12:10.865708+0000","last_fullsized":"2024-09-10T16:12:10.865708+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16988,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":138,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.863216+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:12:10.863216+0000","last_peered":"2024-09-10T16:12:10.863216+0000","last_clean":"2024-09-10T16:12:10.863216+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:12:10.863216+0000","last_undegraded":"2024-09-10T16:12:10.863216+0000","last_fullsized":"2024-09-10T16:12:10.863216+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'21048","reported_seq":21218,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.865639+0000","last_change":"2024-09-10T16:11:17.315372+0000","last_active":"2024-09-10T16:12:10.865639+0000","last_peered":"2024-09-10T16:12:10.865639+0000","last_clean":"2024-09-10T16:12:10.865639+0000","last_became_active":"2024-09-10T16:11:17.234487+0000","last_became_peered":"2024-09-10T16:11:17.234487+0000","last_unstale":"2024-09-10T16:12:10.865639+0000","last_undegraded":"2024-09-10T16:12:10.865639+0000","last_fullsized":"2024-09-10T16:12:10.865639+0000","mapping_epoch":60,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:19:00.114196+0000","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17303,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4373,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.865649+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:12:10.865649+0000","last_peered":"2024-09-10T16:12:10.865649+0000","last_clean":"2024-09-10T16:12:10.865649+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:12:10.865649+0000","last_undegraded":"2024-09-10T16:12:10.865649+0000","last_fullsized":"2024-09-10T16:12:10.865649+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":119,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.865758+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:12:10.865758+0000","last_peered":"2024-09-10T16:12:10.865758+0000","last_clean":"2024-09-10T16:12:10.865758+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:12:10.865758+0000","last_undegraded":"2024-09-10T16:12:10.865758+0000","last_fullsized":"2024-09-10T16:12:10.865758+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'21198","reported_seq":21369,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.865540+0000","last_change":"2024-09-10T16:11:44.391021+0000","last_active":"2024-09-10T16:12:10.865540+0000","last_peered":"2024-09-10T16:12:10.865540+0000","last_clean":"2024-09-10T16:12:10.865540+0000","last_became_active":"2024-09-10T16:11:44.312647+0000","last_became_peered":"2024-09-10T16:11:44.312647+0000","last_unstale":"2024-09-10T16:12:10.865540+0000","last_undegraded":"2024-09-10T16:12:10.865540+0000","last_fullsized":"2024-09-10T16:12:10.865540+0000","mapping_epoch":62,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:33:28.041611+0000","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17361,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4325,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292096,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51390,"num_bytes_recovered":33098578,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":138895360,"data_stored":138805035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161022,"ondisk_log_size":161022,"up":32,"acting":31,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986351,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":434108,"kb_used_data":69228,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93937732,"statfs":{"total":96636764160,"available":96192237568,"internally_reserved":0,"allocated":70889472,"data_stored":70501777,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659296,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":505664,"kb_used_data":560,"kb_used_omap":4,"kb_used_meta":505083,"kb_avail":93866176,"statfs":{"total":96636764160,"available":96118964224,"internally_reserved":0,"allocated":573440,"data_stored":185281,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":517205678},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182579,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":546956,"kb_used_data":68284,"kb_used_omap":4,"kb_used_meta":478651,"kb_avail":93824884,"statfs":{"total":96636764160,"available":96076681216,"internally_reserved":0,"allocated":69922816,"data_stored":69523818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":490139306},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":69378048,"data_stored":69341918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":69488640,"data_stored":69459736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:12:36.652 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-10T16:12:36.652 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "49'21082", 'reported_seq': 29740, 'reported_epoch': 65, 'state': 'active+undersized+remapped', 'last_fresh': '2024-09-10T16:12:35.687487+0000', 'last_change': '2024-09-10T16:12:35.687652+0000', 'last_active': '2024-09-10T16:12:35.687487+0000', 'last_peered': '2024-09-10T16:12:35.687487+0000', 'last_clean': '2024-09-10T15:59:14.139436+0000', 'last_became_active': '2024-09-10T15:59:16.318715+0000', 'last_became_peered': '2024-09-10T15:59:16.318715+0000', 'last_unstale': '2024-09-10T16:12:35.687487+0000', 'last_undegraded': '2024-09-10T15:59:15.877524+0000', 'last_fullsized': '2024-09-10T15:59:15.769042+0000', 'mapping_epoch': 42, 'log_start': "45'11000", 'ondisk_log_start': "45'11000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3537", 'last_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'objects_scrubbed': 3497, 'log_size': 10082, 'log_dups_size': 0, 'ondisk_log_size': 10082, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1224, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 17581, 'num_write_kb': 28172, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 4237, 'num_bytes_recovered': 966, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '1', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:12:36.652 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T16:12:37.004 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:12:37.005 INFO:teuthology.orchestra.run.smithi115.stdout:{"fsid":"7effee29-df05-4351-9741-ee203564ed81","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 1023/24 objects degraded (4262.500%), 1 pg degraded, 1 pg undersized","count":2},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":921,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":66,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983954,"num_in_osds":3,"osd_in_since":1725983947,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":23},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":12,"data_bytes":4849,"bytes_used":1527189504,"bytes_avail":288383102976,"bytes_total":289910292480,"degraded_objects":1023,"degraded_total":24,"degraded_ratio":42.625,"recovering_objects_per_sec":130,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1569,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-10T15:57:14:967120+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T16:00:13.560776+0000","services":{}},"progress_events":{}} 2024-09-10T16:12:37.017 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-10T16:12:37.017 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:12:37.298 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:12:37.298 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:12:37.310 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":519,"stamp":"2024-09-10T16:12:36.464267+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292098,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51392,"num_bytes_recovered":33098597,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161025,"ondisk_log_size":161025,"up":48,"acting":47,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1486728,"kb_used_data":138072,"kb_used_omap":12,"kb_used_meta":1348595,"kb_avail":281628792,"statfs":{"total":289910292480,"available":288387883008,"internally_reserved":0,"allocated":141385728,"data_stored":140210876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1380961282},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2592,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2592,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.661197"},"pg_stats":[{"pgid":"2.f","version":"49'20672","reported_seq":29317,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.863984+0000","last_change":"2024-09-10T16:09:57.088531+0000","last_active":"2024-09-10T16:12:10.863984+0000","last_peered":"2024-09-10T16:12:10.863984+0000","last_clean":"2024-09-10T16:12:10.863984+0000","last_became_active":"2024-09-10T16:09:57.005755+0000","last_became_peered":"2024-09-10T16:09:57.005755+0000","last_unstale":"2024-09-10T16:12:10.863984+0000","last_undegraded":"2024-09-10T16:12:10.863984+0000","last_fullsized":"2024-09-10T16:12:10.863984+0000","mapping_epoch":54,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:09:19.329697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20672,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4259,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20844","reported_seq":21001,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.863381+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:12:10.863381+0000","last_peered":"2024-09-10T16:12:10.863381+0000","last_clean":"2024-09-10T16:12:10.863381+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:12:10.863381+0000","last_undegraded":"2024-09-10T16:12:10.863381+0000","last_fullsized":"2024-09-10T16:12:10.863381+0000","mapping_epoch":35,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18949,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20798","reported_seq":29520,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.865598+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:12:10.865598+0000","last_peered":"2024-09-10T16:12:10.865598+0000","last_clean":"2024-09-10T16:12:10.865598+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:12:10.865598+0000","last_undegraded":"2024-09-10T16:12:10.865598+0000","last_fullsized":"2024-09-10T16:12:10.865598+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18802,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20660","reported_seq":20785,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.863544+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:12:10.863544+0000","last_peered":"2024-09-10T16:12:10.863544+0000","last_clean":"2024-09-10T16:12:10.863544+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:12:10.863544+0000","last_undegraded":"2024-09-10T16:12:10.863544+0000","last_fullsized":"2024-09-10T16:12:10.863544+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20660,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20891","reported_seq":21065,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.864566+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:12:10.864566+0000","last_peered":"2024-09-10T16:12:10.864566+0000","last_clean":"2024-09-10T16:12:10.864566+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:12:10.864566+0000","last_undegraded":"2024-09-10T16:12:10.864566+0000","last_fullsized":"2024-09-10T16:12:10.864566+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17602,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'21061","reported_seq":21234,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.865406+0000","last_change":"2024-09-10T16:10:52.241702+0000","last_active":"2024-09-10T16:12:10.865406+0000","last_peered":"2024-09-10T16:12:10.865406+0000","last_clean":"2024-09-10T16:12:10.865406+0000","last_became_active":"2024-09-10T16:10:52.171236+0000","last_became_peered":"2024-09-10T16:10:52.171236+0000","last_unstale":"2024-09-10T16:12:10.865406+0000","last_undegraded":"2024-09-10T16:12:10.865406+0000","last_fullsized":"2024-09-10T16:12:10.865406+0000","mapping_epoch":58,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:04:46.778280+0000","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17673,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4372,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'21082","reported_seq":29740,"reported_epoch":65,"state":"active+undersized+remapped","last_fresh":"2024-09-10T16:12:35.687487+0000","last_change":"2024-09-10T16:12:35.687652+0000","last_active":"2024-09-10T16:12:35.687487+0000","last_peered":"2024-09-10T16:12:35.687487+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:12:35.687487+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17581,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4237,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","1","2"],"object_location_counts":[{"shards":"0","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'21314","reported_seq":21488,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.862825+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:12:10.862825+0000","last_peered":"2024-09-10T16:12:10.862825+0000","last_clean":"2024-09-10T16:12:10.862825+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:12:10.862825+0000","last_undegraded":"2024-09-10T16:12:10.862825+0000","last_fullsized":"2024-09-10T16:12:10.862825+0000","mapping_epoch":34,"log_start":"45'11300","ondisk_log_start":"45'11300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18161,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.865297+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:12:10.865297+0000","last_peered":"2024-09-10T16:12:10.865297+0000","last_clean":"2024-09-10T16:12:10.865297+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:12:10.865297+0000","last_undegraded":"2024-09-10T16:12:10.865297+0000","last_fullsized":"2024-09-10T16:12:10.865297+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20457","reported_seq":20645,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.865472+0000","last_change":"2024-09-10T16:09:33.012840+0000","last_active":"2024-09-10T16:12:10.865472+0000","last_peered":"2024-09-10T16:12:10.865472+0000","last_clean":"2024-09-10T16:12:10.865472+0000","last_became_active":"2024-09-10T16:09:32.934056+0000","last_became_peered":"2024-09-10T16:09:32.934056+0000","last_unstale":"2024-09-10T16:12:10.865472+0000","last_undegraded":"2024-09-10T16:12:10.865472+0000","last_fullsized":"2024-09-10T16:12:10.865472+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:08:30.481235+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18276,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":4579,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":138,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.862711+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:12:10.862711+0000","last_peered":"2024-09-10T16:12:10.862711+0000","last_clean":"2024-09-10T16:12:10.862711+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:12:10.862711+0000","last_undegraded":"2024-09-10T16:12:10.862711+0000","last_fullsized":"2024-09-10T16:12:10.862711+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20910","reported_seq":29607,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.864300+0000","last_change":"2024-09-10T16:09:08.909580+0000","last_active":"2024-09-10T16:12:10.864300+0000","last_peered":"2024-09-10T16:12:10.864300+0000","last_clean":"2024-09-10T16:12:10.864300+0000","last_became_active":"2024-09-10T16:09:08.854941+0000","last_became_peered":"2024-09-10T16:09:08.854941+0000","last_unstale":"2024-09-10T16:12:10.864300+0000","last_undegraded":"2024-09-10T16:12:10.864300+0000","last_fullsized":"2024-09-10T16:12:10.864300+0000","mapping_epoch":50,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:13:14.846691+0000","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17414,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4257,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":148,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.864168+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:12:10.864168+0000","last_peered":"2024-09-10T16:12:10.864168+0000","last_clean":"2024-09-10T16:12:10.864168+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:12:10.864168+0000","last_undegraded":"2024-09-10T16:12:10.864168+0000","last_fullsized":"2024-09-10T16:12:10.864168+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20781","reported_seq":29578,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.862793+0000","last_change":"2024-09-10T16:10:20.907889+0000","last_active":"2024-09-10T16:12:10.862793+0000","last_peered":"2024-09-10T16:12:10.862793+0000","last_clean":"2024-09-10T16:12:10.862793+0000","last_became_active":"2024-09-10T16:10:20.852065+0000","last_became_peered":"2024-09-10T16:10:20.852065+0000","last_unstale":"2024-09-10T16:12:10.862793+0000","last_undegraded":"2024-09-10T16:12:10.862793+0000","last_fullsized":"2024-09-10T16:12:10.862793+0000","mapping_epoch":56,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:34.069549+0000","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18461,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4307,"num_bytes_recovered":8675,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":124,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.865426+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:12:10.865426+0000","last_peered":"2024-09-10T16:12:10.865426+0000","last_clean":"2024-09-10T16:12:10.865426+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:12:10.865426+0000","last_undegraded":"2024-09-10T16:12:10.865426+0000","last_fullsized":"2024-09-10T16:12:10.865426+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'21170","reported_seq":30064,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.862524+0000","last_change":"2024-09-10T16:10:50.728349+0000","last_active":"2024-09-10T16:12:10.862524+0000","last_peered":"2024-09-10T16:12:10.862524+0000","last_clean":"2024-09-10T16:12:10.862524+0000","last_became_active":"2024-09-10T16:08:10.860438+0000","last_became_peered":"2024-09-10T16:08:10.860438+0000","last_unstale":"2024-09-10T16:12:10.862524+0000","last_undegraded":"2024-09-10T16:12:10.862524+0000","last_fullsized":"2024-09-10T16:12:10.862524+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:38:13.781380+0000","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18724,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20838","reported_seq":20995,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.956229+0000","last_change":"2024-09-10T16:12:10.956160+0000","last_active":"2024-09-10T16:12:10.956229+0000","last_peered":"2024-09-10T16:12:10.956229+0000","last_clean":"2024-09-10T16:12:10.956229+0000","last_became_active":"2024-09-10T16:12:10.866692+0000","last_became_peered":"2024-09-10T16:12:10.866692+0000","last_unstale":"2024-09-10T16:12:10.956229+0000","last_undegraded":"2024-09-10T16:12:10.956229+0000","last_fullsized":"2024-09-10T16:12:10.956229+0000","mapping_epoch":64,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:01:48.931853+0000","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17469,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4305,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.865568+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:12:10.865568+0000","last_peered":"2024-09-10T16:12:10.865568+0000","last_clean":"2024-09-10T16:12:10.865568+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:12:10.865568+0000","last_undegraded":"2024-09-10T16:12:10.865568+0000","last_fullsized":"2024-09-10T16:12:10.865568+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20598","reported_seq":29208,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.865708+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:12:10.865708+0000","last_peered":"2024-09-10T16:12:10.865708+0000","last_clean":"2024-09-10T16:12:10.865708+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:12:10.865708+0000","last_undegraded":"2024-09-10T16:12:10.865708+0000","last_fullsized":"2024-09-10T16:12:10.865708+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16988,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":138,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.863216+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:12:10.863216+0000","last_peered":"2024-09-10T16:12:10.863216+0000","last_clean":"2024-09-10T16:12:10.863216+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:12:10.863216+0000","last_undegraded":"2024-09-10T16:12:10.863216+0000","last_fullsized":"2024-09-10T16:12:10.863216+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'21048","reported_seq":21218,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.865639+0000","last_change":"2024-09-10T16:11:17.315372+0000","last_active":"2024-09-10T16:12:10.865639+0000","last_peered":"2024-09-10T16:12:10.865639+0000","last_clean":"2024-09-10T16:12:10.865639+0000","last_became_active":"2024-09-10T16:11:17.234487+0000","last_became_peered":"2024-09-10T16:11:17.234487+0000","last_unstale":"2024-09-10T16:12:10.865639+0000","last_undegraded":"2024-09-10T16:12:10.865639+0000","last_fullsized":"2024-09-10T16:12:10.865639+0000","mapping_epoch":60,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:19:00.114196+0000","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17303,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4373,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.865649+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:12:10.865649+0000","last_peered":"2024-09-10T16:12:10.865649+0000","last_clean":"2024-09-10T16:12:10.865649+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:12:10.865649+0000","last_undegraded":"2024-09-10T16:12:10.865649+0000","last_fullsized":"2024-09-10T16:12:10.865649+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":119,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.865758+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:12:10.865758+0000","last_peered":"2024-09-10T16:12:10.865758+0000","last_clean":"2024-09-10T16:12:10.865758+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:12:10.865758+0000","last_undegraded":"2024-09-10T16:12:10.865758+0000","last_fullsized":"2024-09-10T16:12:10.865758+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'21198","reported_seq":21369,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.865540+0000","last_change":"2024-09-10T16:11:44.391021+0000","last_active":"2024-09-10T16:12:10.865540+0000","last_peered":"2024-09-10T16:12:10.865540+0000","last_clean":"2024-09-10T16:12:10.865540+0000","last_became_active":"2024-09-10T16:11:44.312647+0000","last_became_peered":"2024-09-10T16:11:44.312647+0000","last_unstale":"2024-09-10T16:12:10.865540+0000","last_undegraded":"2024-09-10T16:12:10.865540+0000","last_fullsized":"2024-09-10T16:12:10.865540+0000","mapping_epoch":62,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:33:28.041611+0000","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17361,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4325,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292096,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51390,"num_bytes_recovered":33098578,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":138895360,"data_stored":138805035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161022,"ondisk_log_size":161022,"up":32,"acting":31,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986351,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":434108,"kb_used_data":69228,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93937732,"statfs":{"total":96636764160,"available":96192237568,"internally_reserved":0,"allocated":70889472,"data_stored":70501777,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659296,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":505664,"kb_used_data":560,"kb_used_omap":4,"kb_used_meta":505083,"kb_avail":93866176,"statfs":{"total":96636764160,"available":96118964224,"internally_reserved":0,"allocated":573440,"data_stored":185281,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":517205678},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182579,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":546956,"kb_used_data":68284,"kb_used_omap":4,"kb_used_meta":478651,"kb_avail":93824884,"statfs":{"total":96636764160,"available":96076681216,"internally_reserved":0,"allocated":69922816,"data_stored":69523818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":490139306},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":69378048,"data_stored":69341918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":69488640,"data_stored":69459736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:12:40.313 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:12:40.612 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:12:40.612 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:12:40.624 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":522,"stamp":"2024-09-10T16:12:39.805606+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292098,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51392,"num_bytes_recovered":33098597,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161025,"ondisk_log_size":161025,"up":48,"acting":47,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1486728,"kb_used_data":138072,"kb_used_omap":12,"kb_used_meta":1348595,"kb_avail":281628792,"statfs":{"total":289910292480,"available":288387883008,"internally_reserved":0,"allocated":141385728,"data_stored":140210876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1380961282},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1023,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1023,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001479"},"pg_stats":[{"pgid":"2.f","version":"49'20672","reported_seq":29317,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.863984+0000","last_change":"2024-09-10T16:09:57.088531+0000","last_active":"2024-09-10T16:12:10.863984+0000","last_peered":"2024-09-10T16:12:10.863984+0000","last_clean":"2024-09-10T16:12:10.863984+0000","last_became_active":"2024-09-10T16:09:57.005755+0000","last_became_peered":"2024-09-10T16:09:57.005755+0000","last_unstale":"2024-09-10T16:12:10.863984+0000","last_undegraded":"2024-09-10T16:12:10.863984+0000","last_fullsized":"2024-09-10T16:12:10.863984+0000","mapping_epoch":54,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:09:19.329697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20672,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4259,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20844","reported_seq":21001,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.863381+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:12:10.863381+0000","last_peered":"2024-09-10T16:12:10.863381+0000","last_clean":"2024-09-10T16:12:10.863381+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:12:10.863381+0000","last_undegraded":"2024-09-10T16:12:10.863381+0000","last_fullsized":"2024-09-10T16:12:10.863381+0000","mapping_epoch":35,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18949,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20798","reported_seq":29520,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.865598+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:12:10.865598+0000","last_peered":"2024-09-10T16:12:10.865598+0000","last_clean":"2024-09-10T16:12:10.865598+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:12:10.865598+0000","last_undegraded":"2024-09-10T16:12:10.865598+0000","last_fullsized":"2024-09-10T16:12:10.865598+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18802,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20660","reported_seq":20785,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.863544+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:12:10.863544+0000","last_peered":"2024-09-10T16:12:10.863544+0000","last_clean":"2024-09-10T16:12:10.863544+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:12:10.863544+0000","last_undegraded":"2024-09-10T16:12:10.863544+0000","last_fullsized":"2024-09-10T16:12:10.863544+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20660,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20891","reported_seq":21065,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.864566+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:12:10.864566+0000","last_peered":"2024-09-10T16:12:10.864566+0000","last_clean":"2024-09-10T16:12:10.864566+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:12:10.864566+0000","last_undegraded":"2024-09-10T16:12:10.864566+0000","last_fullsized":"2024-09-10T16:12:10.864566+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17602,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'21061","reported_seq":21234,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.865406+0000","last_change":"2024-09-10T16:10:52.241702+0000","last_active":"2024-09-10T16:12:10.865406+0000","last_peered":"2024-09-10T16:12:10.865406+0000","last_clean":"2024-09-10T16:12:10.865406+0000","last_became_active":"2024-09-10T16:10:52.171236+0000","last_became_peered":"2024-09-10T16:10:52.171236+0000","last_unstale":"2024-09-10T16:12:10.865406+0000","last_undegraded":"2024-09-10T16:12:10.865406+0000","last_fullsized":"2024-09-10T16:12:10.865406+0000","mapping_epoch":58,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:04:46.778280+0000","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17673,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4372,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'21082","reported_seq":29740,"reported_epoch":65,"state":"active+undersized+remapped","last_fresh":"2024-09-10T16:12:35.687487+0000","last_change":"2024-09-10T16:12:35.687652+0000","last_active":"2024-09-10T16:12:35.687487+0000","last_peered":"2024-09-10T16:12:35.687487+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:12:35.687487+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17581,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4237,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","1","2"],"object_location_counts":[{"shards":"0","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'21314","reported_seq":21488,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.862825+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:12:10.862825+0000","last_peered":"2024-09-10T16:12:10.862825+0000","last_clean":"2024-09-10T16:12:10.862825+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:12:10.862825+0000","last_undegraded":"2024-09-10T16:12:10.862825+0000","last_fullsized":"2024-09-10T16:12:10.862825+0000","mapping_epoch":34,"log_start":"45'11300","ondisk_log_start":"45'11300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18161,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.865297+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:12:10.865297+0000","last_peered":"2024-09-10T16:12:10.865297+0000","last_clean":"2024-09-10T16:12:10.865297+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:12:10.865297+0000","last_undegraded":"2024-09-10T16:12:10.865297+0000","last_fullsized":"2024-09-10T16:12:10.865297+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20457","reported_seq":20645,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.865472+0000","last_change":"2024-09-10T16:09:33.012840+0000","last_active":"2024-09-10T16:12:10.865472+0000","last_peered":"2024-09-10T16:12:10.865472+0000","last_clean":"2024-09-10T16:12:10.865472+0000","last_became_active":"2024-09-10T16:09:32.934056+0000","last_became_peered":"2024-09-10T16:09:32.934056+0000","last_unstale":"2024-09-10T16:12:10.865472+0000","last_undegraded":"2024-09-10T16:12:10.865472+0000","last_fullsized":"2024-09-10T16:12:10.865472+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:08:30.481235+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18276,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":4579,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":138,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.862711+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:12:10.862711+0000","last_peered":"2024-09-10T16:12:10.862711+0000","last_clean":"2024-09-10T16:12:10.862711+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:12:10.862711+0000","last_undegraded":"2024-09-10T16:12:10.862711+0000","last_fullsized":"2024-09-10T16:12:10.862711+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20910","reported_seq":29607,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.864300+0000","last_change":"2024-09-10T16:09:08.909580+0000","last_active":"2024-09-10T16:12:10.864300+0000","last_peered":"2024-09-10T16:12:10.864300+0000","last_clean":"2024-09-10T16:12:10.864300+0000","last_became_active":"2024-09-10T16:09:08.854941+0000","last_became_peered":"2024-09-10T16:09:08.854941+0000","last_unstale":"2024-09-10T16:12:10.864300+0000","last_undegraded":"2024-09-10T16:12:10.864300+0000","last_fullsized":"2024-09-10T16:12:10.864300+0000","mapping_epoch":50,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:13:14.846691+0000","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17414,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4257,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":148,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.864168+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:12:10.864168+0000","last_peered":"2024-09-10T16:12:10.864168+0000","last_clean":"2024-09-10T16:12:10.864168+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:12:10.864168+0000","last_undegraded":"2024-09-10T16:12:10.864168+0000","last_fullsized":"2024-09-10T16:12:10.864168+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20781","reported_seq":29578,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.862793+0000","last_change":"2024-09-10T16:10:20.907889+0000","last_active":"2024-09-10T16:12:10.862793+0000","last_peered":"2024-09-10T16:12:10.862793+0000","last_clean":"2024-09-10T16:12:10.862793+0000","last_became_active":"2024-09-10T16:10:20.852065+0000","last_became_peered":"2024-09-10T16:10:20.852065+0000","last_unstale":"2024-09-10T16:12:10.862793+0000","last_undegraded":"2024-09-10T16:12:10.862793+0000","last_fullsized":"2024-09-10T16:12:10.862793+0000","mapping_epoch":56,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:34.069549+0000","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18461,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4307,"num_bytes_recovered":8675,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":124,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.865426+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:12:10.865426+0000","last_peered":"2024-09-10T16:12:10.865426+0000","last_clean":"2024-09-10T16:12:10.865426+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:12:10.865426+0000","last_undegraded":"2024-09-10T16:12:10.865426+0000","last_fullsized":"2024-09-10T16:12:10.865426+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'21170","reported_seq":30064,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.862524+0000","last_change":"2024-09-10T16:10:50.728349+0000","last_active":"2024-09-10T16:12:10.862524+0000","last_peered":"2024-09-10T16:12:10.862524+0000","last_clean":"2024-09-10T16:12:10.862524+0000","last_became_active":"2024-09-10T16:08:10.860438+0000","last_became_peered":"2024-09-10T16:08:10.860438+0000","last_unstale":"2024-09-10T16:12:10.862524+0000","last_undegraded":"2024-09-10T16:12:10.862524+0000","last_fullsized":"2024-09-10T16:12:10.862524+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:38:13.781380+0000","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18724,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20838","reported_seq":20995,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.956229+0000","last_change":"2024-09-10T16:12:10.956160+0000","last_active":"2024-09-10T16:12:10.956229+0000","last_peered":"2024-09-10T16:12:10.956229+0000","last_clean":"2024-09-10T16:12:10.956229+0000","last_became_active":"2024-09-10T16:12:10.866692+0000","last_became_peered":"2024-09-10T16:12:10.866692+0000","last_unstale":"2024-09-10T16:12:10.956229+0000","last_undegraded":"2024-09-10T16:12:10.956229+0000","last_fullsized":"2024-09-10T16:12:10.956229+0000","mapping_epoch":64,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:01:48.931853+0000","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17469,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4305,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.865568+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:12:10.865568+0000","last_peered":"2024-09-10T16:12:10.865568+0000","last_clean":"2024-09-10T16:12:10.865568+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:12:10.865568+0000","last_undegraded":"2024-09-10T16:12:10.865568+0000","last_fullsized":"2024-09-10T16:12:10.865568+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20598","reported_seq":29208,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.865708+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:12:10.865708+0000","last_peered":"2024-09-10T16:12:10.865708+0000","last_clean":"2024-09-10T16:12:10.865708+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:12:10.865708+0000","last_undegraded":"2024-09-10T16:12:10.865708+0000","last_fullsized":"2024-09-10T16:12:10.865708+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16988,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":138,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.863216+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:12:10.863216+0000","last_peered":"2024-09-10T16:12:10.863216+0000","last_clean":"2024-09-10T16:12:10.863216+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:12:10.863216+0000","last_undegraded":"2024-09-10T16:12:10.863216+0000","last_fullsized":"2024-09-10T16:12:10.863216+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'21048","reported_seq":21218,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.865639+0000","last_change":"2024-09-10T16:11:17.315372+0000","last_active":"2024-09-10T16:12:10.865639+0000","last_peered":"2024-09-10T16:12:10.865639+0000","last_clean":"2024-09-10T16:12:10.865639+0000","last_became_active":"2024-09-10T16:11:17.234487+0000","last_became_peered":"2024-09-10T16:11:17.234487+0000","last_unstale":"2024-09-10T16:12:10.865639+0000","last_undegraded":"2024-09-10T16:12:10.865639+0000","last_fullsized":"2024-09-10T16:12:10.865639+0000","mapping_epoch":60,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:19:00.114196+0000","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17303,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4373,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.865649+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:12:10.865649+0000","last_peered":"2024-09-10T16:12:10.865649+0000","last_clean":"2024-09-10T16:12:10.865649+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:12:10.865649+0000","last_undegraded":"2024-09-10T16:12:10.865649+0000","last_fullsized":"2024-09-10T16:12:10.865649+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":119,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.865758+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:12:10.865758+0000","last_peered":"2024-09-10T16:12:10.865758+0000","last_clean":"2024-09-10T16:12:10.865758+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:12:10.865758+0000","last_undegraded":"2024-09-10T16:12:10.865758+0000","last_fullsized":"2024-09-10T16:12:10.865758+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'21198","reported_seq":21369,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.865540+0000","last_change":"2024-09-10T16:11:44.391021+0000","last_active":"2024-09-10T16:12:10.865540+0000","last_peered":"2024-09-10T16:12:10.865540+0000","last_clean":"2024-09-10T16:12:10.865540+0000","last_became_active":"2024-09-10T16:11:44.312647+0000","last_became_peered":"2024-09-10T16:11:44.312647+0000","last_unstale":"2024-09-10T16:12:10.865540+0000","last_undegraded":"2024-09-10T16:12:10.865540+0000","last_fullsized":"2024-09-10T16:12:10.865540+0000","mapping_epoch":62,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:33:28.041611+0000","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17361,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4325,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292096,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51390,"num_bytes_recovered":33098578,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":138895360,"data_stored":138805035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161022,"ondisk_log_size":161022,"up":32,"acting":31,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986351,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":434108,"kb_used_data":69228,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93937732,"statfs":{"total":96636764160,"available":96192237568,"internally_reserved":0,"allocated":70889472,"data_stored":70501777,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659296,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":505664,"kb_used_data":560,"kb_used_omap":4,"kb_used_meta":505083,"kb_avail":93866176,"statfs":{"total":96636764160,"available":96118964224,"internally_reserved":0,"allocated":573440,"data_stored":185281,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":517205678},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182579,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":546956,"kb_used_data":68284,"kb_used_omap":4,"kb_used_meta":478651,"kb_avail":93824884,"statfs":{"total":96636764160,"available":96076681216,"internally_reserved":0,"allocated":69922816,"data_stored":69523818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":490139306},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":69378048,"data_stored":69341918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":69488640,"data_stored":69459736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:12:40.626 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:12:40.907 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:12:40.908 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:12:40.920 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":522,"stamp":"2024-09-10T16:12:39.805606+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292098,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51392,"num_bytes_recovered":33098597,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161025,"ondisk_log_size":161025,"up":48,"acting":47,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1486728,"kb_used_data":138072,"kb_used_omap":12,"kb_used_meta":1348595,"kb_avail":281628792,"statfs":{"total":289910292480,"available":288387883008,"internally_reserved":0,"allocated":141385728,"data_stored":140210876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1380961282},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1023,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1023,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001479"},"pg_stats":[{"pgid":"2.f","version":"49'20672","reported_seq":29317,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.863984+0000","last_change":"2024-09-10T16:09:57.088531+0000","last_active":"2024-09-10T16:12:10.863984+0000","last_peered":"2024-09-10T16:12:10.863984+0000","last_clean":"2024-09-10T16:12:10.863984+0000","last_became_active":"2024-09-10T16:09:57.005755+0000","last_became_peered":"2024-09-10T16:09:57.005755+0000","last_unstale":"2024-09-10T16:12:10.863984+0000","last_undegraded":"2024-09-10T16:12:10.863984+0000","last_fullsized":"2024-09-10T16:12:10.863984+0000","mapping_epoch":54,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:09:19.329697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20672,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4259,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20844","reported_seq":21001,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.863381+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:12:10.863381+0000","last_peered":"2024-09-10T16:12:10.863381+0000","last_clean":"2024-09-10T16:12:10.863381+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:12:10.863381+0000","last_undegraded":"2024-09-10T16:12:10.863381+0000","last_fullsized":"2024-09-10T16:12:10.863381+0000","mapping_epoch":35,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18949,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20798","reported_seq":29520,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.865598+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:12:10.865598+0000","last_peered":"2024-09-10T16:12:10.865598+0000","last_clean":"2024-09-10T16:12:10.865598+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:12:10.865598+0000","last_undegraded":"2024-09-10T16:12:10.865598+0000","last_fullsized":"2024-09-10T16:12:10.865598+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18802,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20660","reported_seq":20785,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.863544+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:12:10.863544+0000","last_peered":"2024-09-10T16:12:10.863544+0000","last_clean":"2024-09-10T16:12:10.863544+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:12:10.863544+0000","last_undegraded":"2024-09-10T16:12:10.863544+0000","last_fullsized":"2024-09-10T16:12:10.863544+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20660,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20891","reported_seq":21065,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.864566+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:12:10.864566+0000","last_peered":"2024-09-10T16:12:10.864566+0000","last_clean":"2024-09-10T16:12:10.864566+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:12:10.864566+0000","last_undegraded":"2024-09-10T16:12:10.864566+0000","last_fullsized":"2024-09-10T16:12:10.864566+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17602,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'21061","reported_seq":21234,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.865406+0000","last_change":"2024-09-10T16:10:52.241702+0000","last_active":"2024-09-10T16:12:10.865406+0000","last_peered":"2024-09-10T16:12:10.865406+0000","last_clean":"2024-09-10T16:12:10.865406+0000","last_became_active":"2024-09-10T16:10:52.171236+0000","last_became_peered":"2024-09-10T16:10:52.171236+0000","last_unstale":"2024-09-10T16:12:10.865406+0000","last_undegraded":"2024-09-10T16:12:10.865406+0000","last_fullsized":"2024-09-10T16:12:10.865406+0000","mapping_epoch":58,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:04:46.778280+0000","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17673,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4372,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'21082","reported_seq":29740,"reported_epoch":65,"state":"active+undersized+remapped","last_fresh":"2024-09-10T16:12:35.687487+0000","last_change":"2024-09-10T16:12:35.687652+0000","last_active":"2024-09-10T16:12:35.687487+0000","last_peered":"2024-09-10T16:12:35.687487+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:12:35.687487+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17581,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4237,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","1","2"],"object_location_counts":[{"shards":"0","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'21314","reported_seq":21488,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.862825+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:12:10.862825+0000","last_peered":"2024-09-10T16:12:10.862825+0000","last_clean":"2024-09-10T16:12:10.862825+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:12:10.862825+0000","last_undegraded":"2024-09-10T16:12:10.862825+0000","last_fullsized":"2024-09-10T16:12:10.862825+0000","mapping_epoch":34,"log_start":"45'11300","ondisk_log_start":"45'11300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18161,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.865297+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:12:10.865297+0000","last_peered":"2024-09-10T16:12:10.865297+0000","last_clean":"2024-09-10T16:12:10.865297+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:12:10.865297+0000","last_undegraded":"2024-09-10T16:12:10.865297+0000","last_fullsized":"2024-09-10T16:12:10.865297+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20457","reported_seq":20645,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.865472+0000","last_change":"2024-09-10T16:09:33.012840+0000","last_active":"2024-09-10T16:12:10.865472+0000","last_peered":"2024-09-10T16:12:10.865472+0000","last_clean":"2024-09-10T16:12:10.865472+0000","last_became_active":"2024-09-10T16:09:32.934056+0000","last_became_peered":"2024-09-10T16:09:32.934056+0000","last_unstale":"2024-09-10T16:12:10.865472+0000","last_undegraded":"2024-09-10T16:12:10.865472+0000","last_fullsized":"2024-09-10T16:12:10.865472+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:08:30.481235+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18276,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":4579,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":138,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.862711+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:12:10.862711+0000","last_peered":"2024-09-10T16:12:10.862711+0000","last_clean":"2024-09-10T16:12:10.862711+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:12:10.862711+0000","last_undegraded":"2024-09-10T16:12:10.862711+0000","last_fullsized":"2024-09-10T16:12:10.862711+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20910","reported_seq":29607,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.864300+0000","last_change":"2024-09-10T16:09:08.909580+0000","last_active":"2024-09-10T16:12:10.864300+0000","last_peered":"2024-09-10T16:12:10.864300+0000","last_clean":"2024-09-10T16:12:10.864300+0000","last_became_active":"2024-09-10T16:09:08.854941+0000","last_became_peered":"2024-09-10T16:09:08.854941+0000","last_unstale":"2024-09-10T16:12:10.864300+0000","last_undegraded":"2024-09-10T16:12:10.864300+0000","last_fullsized":"2024-09-10T16:12:10.864300+0000","mapping_epoch":50,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:13:14.846691+0000","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17414,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4257,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":148,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.864168+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:12:10.864168+0000","last_peered":"2024-09-10T16:12:10.864168+0000","last_clean":"2024-09-10T16:12:10.864168+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:12:10.864168+0000","last_undegraded":"2024-09-10T16:12:10.864168+0000","last_fullsized":"2024-09-10T16:12:10.864168+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20781","reported_seq":29578,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.862793+0000","last_change":"2024-09-10T16:10:20.907889+0000","last_active":"2024-09-10T16:12:10.862793+0000","last_peered":"2024-09-10T16:12:10.862793+0000","last_clean":"2024-09-10T16:12:10.862793+0000","last_became_active":"2024-09-10T16:10:20.852065+0000","last_became_peered":"2024-09-10T16:10:20.852065+0000","last_unstale":"2024-09-10T16:12:10.862793+0000","last_undegraded":"2024-09-10T16:12:10.862793+0000","last_fullsized":"2024-09-10T16:12:10.862793+0000","mapping_epoch":56,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:34.069549+0000","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18461,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4307,"num_bytes_recovered":8675,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":124,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.865426+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:12:10.865426+0000","last_peered":"2024-09-10T16:12:10.865426+0000","last_clean":"2024-09-10T16:12:10.865426+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:12:10.865426+0000","last_undegraded":"2024-09-10T16:12:10.865426+0000","last_fullsized":"2024-09-10T16:12:10.865426+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'21170","reported_seq":30064,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.862524+0000","last_change":"2024-09-10T16:10:50.728349+0000","last_active":"2024-09-10T16:12:10.862524+0000","last_peered":"2024-09-10T16:12:10.862524+0000","last_clean":"2024-09-10T16:12:10.862524+0000","last_became_active":"2024-09-10T16:08:10.860438+0000","last_became_peered":"2024-09-10T16:08:10.860438+0000","last_unstale":"2024-09-10T16:12:10.862524+0000","last_undegraded":"2024-09-10T16:12:10.862524+0000","last_fullsized":"2024-09-10T16:12:10.862524+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:38:13.781380+0000","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18724,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20838","reported_seq":20995,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.956229+0000","last_change":"2024-09-10T16:12:10.956160+0000","last_active":"2024-09-10T16:12:10.956229+0000","last_peered":"2024-09-10T16:12:10.956229+0000","last_clean":"2024-09-10T16:12:10.956229+0000","last_became_active":"2024-09-10T16:12:10.866692+0000","last_became_peered":"2024-09-10T16:12:10.866692+0000","last_unstale":"2024-09-10T16:12:10.956229+0000","last_undegraded":"2024-09-10T16:12:10.956229+0000","last_fullsized":"2024-09-10T16:12:10.956229+0000","mapping_epoch":64,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:01:48.931853+0000","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17469,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4305,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.865568+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:12:10.865568+0000","last_peered":"2024-09-10T16:12:10.865568+0000","last_clean":"2024-09-10T16:12:10.865568+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:12:10.865568+0000","last_undegraded":"2024-09-10T16:12:10.865568+0000","last_fullsized":"2024-09-10T16:12:10.865568+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20598","reported_seq":29208,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.865708+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:12:10.865708+0000","last_peered":"2024-09-10T16:12:10.865708+0000","last_clean":"2024-09-10T16:12:10.865708+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:12:10.865708+0000","last_undegraded":"2024-09-10T16:12:10.865708+0000","last_fullsized":"2024-09-10T16:12:10.865708+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16988,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":138,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.863216+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:12:10.863216+0000","last_peered":"2024-09-10T16:12:10.863216+0000","last_clean":"2024-09-10T16:12:10.863216+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:12:10.863216+0000","last_undegraded":"2024-09-10T16:12:10.863216+0000","last_fullsized":"2024-09-10T16:12:10.863216+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'21048","reported_seq":21218,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.865639+0000","last_change":"2024-09-10T16:11:17.315372+0000","last_active":"2024-09-10T16:12:10.865639+0000","last_peered":"2024-09-10T16:12:10.865639+0000","last_clean":"2024-09-10T16:12:10.865639+0000","last_became_active":"2024-09-10T16:11:17.234487+0000","last_became_peered":"2024-09-10T16:11:17.234487+0000","last_unstale":"2024-09-10T16:12:10.865639+0000","last_undegraded":"2024-09-10T16:12:10.865639+0000","last_fullsized":"2024-09-10T16:12:10.865639+0000","mapping_epoch":60,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:19:00.114196+0000","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17303,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4373,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.865649+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:12:10.865649+0000","last_peered":"2024-09-10T16:12:10.865649+0000","last_clean":"2024-09-10T16:12:10.865649+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:12:10.865649+0000","last_undegraded":"2024-09-10T16:12:10.865649+0000","last_fullsized":"2024-09-10T16:12:10.865649+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":119,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.865758+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:12:10.865758+0000","last_peered":"2024-09-10T16:12:10.865758+0000","last_clean":"2024-09-10T16:12:10.865758+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:12:10.865758+0000","last_undegraded":"2024-09-10T16:12:10.865758+0000","last_fullsized":"2024-09-10T16:12:10.865758+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'21198","reported_seq":21369,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.865540+0000","last_change":"2024-09-10T16:11:44.391021+0000","last_active":"2024-09-10T16:12:10.865540+0000","last_peered":"2024-09-10T16:12:10.865540+0000","last_clean":"2024-09-10T16:12:10.865540+0000","last_became_active":"2024-09-10T16:11:44.312647+0000","last_became_peered":"2024-09-10T16:11:44.312647+0000","last_unstale":"2024-09-10T16:12:10.865540+0000","last_undegraded":"2024-09-10T16:12:10.865540+0000","last_fullsized":"2024-09-10T16:12:10.865540+0000","mapping_epoch":62,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:33:28.041611+0000","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17361,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4325,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292096,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51390,"num_bytes_recovered":33098578,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":138895360,"data_stored":138805035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161022,"ondisk_log_size":161022,"up":32,"acting":31,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986351,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":434108,"kb_used_data":69228,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93937732,"statfs":{"total":96636764160,"available":96192237568,"internally_reserved":0,"allocated":70889472,"data_stored":70501777,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659296,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":505664,"kb_used_data":560,"kb_used_omap":4,"kb_used_meta":505083,"kb_avail":93866176,"statfs":{"total":96636764160,"available":96118964224,"internally_reserved":0,"allocated":573440,"data_stored":185281,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":517205678},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182579,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":546956,"kb_used_data":68284,"kb_used_omap":4,"kb_used_meta":478651,"kb_avail":93824884,"statfs":{"total":96636764160,"available":96076681216,"internally_reserved":0,"allocated":69922816,"data_stored":69523818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":490139306},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":69378048,"data_stored":69341918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":69488640,"data_stored":69459736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:12:40.921 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-10T16:12:40.921 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "49'21082", 'reported_seq': 29740, 'reported_epoch': 65, 'state': 'active+undersized+remapped', 'last_fresh': '2024-09-10T16:12:35.687487+0000', 'last_change': '2024-09-10T16:12:35.687652+0000', 'last_active': '2024-09-10T16:12:35.687487+0000', 'last_peered': '2024-09-10T16:12:35.687487+0000', 'last_clean': '2024-09-10T15:59:14.139436+0000', 'last_became_active': '2024-09-10T15:59:16.318715+0000', 'last_became_peered': '2024-09-10T15:59:16.318715+0000', 'last_unstale': '2024-09-10T16:12:35.687487+0000', 'last_undegraded': '2024-09-10T15:59:15.877524+0000', 'last_fullsized': '2024-09-10T15:59:15.769042+0000', 'mapping_epoch': 42, 'log_start': "45'11000", 'ondisk_log_start': "45'11000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3537", 'last_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:57:40.749557+0000', 'last_clean_scrub_stamp': '2024-09-10T15:59:03.205323+0000', 'objects_scrubbed': 3497, 'log_size': 10082, 'log_dups_size': 0, 'ondisk_log_size': 10082, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1224, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 17581, 'num_write_kb': 28172, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 4237, 'num_bytes_recovered': 966, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '1', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:12:40.924 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T16:12:41.277 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:12:41.277 INFO:teuthology.orchestra.run.smithi115.stdout:{"fsid":"7effee29-df05-4351-9741-ee203564ed81","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 1 pg undersized","count":1},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":925,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":67,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983954,"num_in_osds":3,"osd_in_since":1725983947,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":23},{"state_name":"active+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":12,"data_bytes":4849,"bytes_used":1522409472,"bytes_avail":288387883008,"bytes_total":289910292480,"recovering_objects_per_sec":127,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1023,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-10T15:57:14:967120+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T16:00:13.560776+0000","services":{}},"progress_events":{}} 2024-09-10T16:12:41.290 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-10T16:12:41.291 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:12:41.577 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:12:41.577 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:12:41.589 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":522,"stamp":"2024-09-10T16:12:39.805606+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292098,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51392,"num_bytes_recovered":33098597,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161025,"ondisk_log_size":161025,"up":48,"acting":47,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1486728,"kb_used_data":138072,"kb_used_omap":12,"kb_used_meta":1348595,"kb_avail":281628792,"statfs":{"total":289910292480,"available":288387883008,"internally_reserved":0,"allocated":141385728,"data_stored":140210876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1380961282},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1023,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1023,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001479"},"pg_stats":[{"pgid":"2.f","version":"49'20672","reported_seq":29317,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.863984+0000","last_change":"2024-09-10T16:09:57.088531+0000","last_active":"2024-09-10T16:12:10.863984+0000","last_peered":"2024-09-10T16:12:10.863984+0000","last_clean":"2024-09-10T16:12:10.863984+0000","last_became_active":"2024-09-10T16:09:57.005755+0000","last_became_peered":"2024-09-10T16:09:57.005755+0000","last_unstale":"2024-09-10T16:12:10.863984+0000","last_undegraded":"2024-09-10T16:12:10.863984+0000","last_fullsized":"2024-09-10T16:12:10.863984+0000","mapping_epoch":54,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:09:19.329697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20672,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4259,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20844","reported_seq":21001,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.863381+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:12:10.863381+0000","last_peered":"2024-09-10T16:12:10.863381+0000","last_clean":"2024-09-10T16:12:10.863381+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:12:10.863381+0000","last_undegraded":"2024-09-10T16:12:10.863381+0000","last_fullsized":"2024-09-10T16:12:10.863381+0000","mapping_epoch":35,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18949,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20798","reported_seq":29520,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.865598+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:12:10.865598+0000","last_peered":"2024-09-10T16:12:10.865598+0000","last_clean":"2024-09-10T16:12:10.865598+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:12:10.865598+0000","last_undegraded":"2024-09-10T16:12:10.865598+0000","last_fullsized":"2024-09-10T16:12:10.865598+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18802,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20660","reported_seq":20785,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.863544+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:12:10.863544+0000","last_peered":"2024-09-10T16:12:10.863544+0000","last_clean":"2024-09-10T16:12:10.863544+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:12:10.863544+0000","last_undegraded":"2024-09-10T16:12:10.863544+0000","last_fullsized":"2024-09-10T16:12:10.863544+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20660,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20891","reported_seq":21065,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.864566+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:12:10.864566+0000","last_peered":"2024-09-10T16:12:10.864566+0000","last_clean":"2024-09-10T16:12:10.864566+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:12:10.864566+0000","last_undegraded":"2024-09-10T16:12:10.864566+0000","last_fullsized":"2024-09-10T16:12:10.864566+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17602,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'21061","reported_seq":21234,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.865406+0000","last_change":"2024-09-10T16:10:52.241702+0000","last_active":"2024-09-10T16:12:10.865406+0000","last_peered":"2024-09-10T16:12:10.865406+0000","last_clean":"2024-09-10T16:12:10.865406+0000","last_became_active":"2024-09-10T16:10:52.171236+0000","last_became_peered":"2024-09-10T16:10:52.171236+0000","last_unstale":"2024-09-10T16:12:10.865406+0000","last_undegraded":"2024-09-10T16:12:10.865406+0000","last_fullsized":"2024-09-10T16:12:10.865406+0000","mapping_epoch":58,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:04:46.778280+0000","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17673,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4372,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'21082","reported_seq":29740,"reported_epoch":65,"state":"active+undersized+remapped","last_fresh":"2024-09-10T16:12:35.687487+0000","last_change":"2024-09-10T16:12:35.687652+0000","last_active":"2024-09-10T16:12:35.687487+0000","last_peered":"2024-09-10T16:12:35.687487+0000","last_clean":"2024-09-10T15:59:14.139436+0000","last_became_active":"2024-09-10T15:59:16.318715+0000","last_became_peered":"2024-09-10T15:59:16.318715+0000","last_unstale":"2024-09-10T16:12:35.687487+0000","last_undegraded":"2024-09-10T15:59:15.877524+0000","last_fullsized":"2024-09-10T15:59:15.769042+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17581,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4237,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","1","2"],"object_location_counts":[{"shards":"0","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'21314","reported_seq":21488,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.862825+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:12:10.862825+0000","last_peered":"2024-09-10T16:12:10.862825+0000","last_clean":"2024-09-10T16:12:10.862825+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:12:10.862825+0000","last_undegraded":"2024-09-10T16:12:10.862825+0000","last_fullsized":"2024-09-10T16:12:10.862825+0000","mapping_epoch":34,"log_start":"45'11300","ondisk_log_start":"45'11300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18161,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.865297+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:12:10.865297+0000","last_peered":"2024-09-10T16:12:10.865297+0000","last_clean":"2024-09-10T16:12:10.865297+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:12:10.865297+0000","last_undegraded":"2024-09-10T16:12:10.865297+0000","last_fullsized":"2024-09-10T16:12:10.865297+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20457","reported_seq":20645,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.865472+0000","last_change":"2024-09-10T16:09:33.012840+0000","last_active":"2024-09-10T16:12:10.865472+0000","last_peered":"2024-09-10T16:12:10.865472+0000","last_clean":"2024-09-10T16:12:10.865472+0000","last_became_active":"2024-09-10T16:09:32.934056+0000","last_became_peered":"2024-09-10T16:09:32.934056+0000","last_unstale":"2024-09-10T16:12:10.865472+0000","last_undegraded":"2024-09-10T16:12:10.865472+0000","last_fullsized":"2024-09-10T16:12:10.865472+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:08:30.481235+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18276,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":4579,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":138,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.862711+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:12:10.862711+0000","last_peered":"2024-09-10T16:12:10.862711+0000","last_clean":"2024-09-10T16:12:10.862711+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:12:10.862711+0000","last_undegraded":"2024-09-10T16:12:10.862711+0000","last_fullsized":"2024-09-10T16:12:10.862711+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20910","reported_seq":29607,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.864300+0000","last_change":"2024-09-10T16:09:08.909580+0000","last_active":"2024-09-10T16:12:10.864300+0000","last_peered":"2024-09-10T16:12:10.864300+0000","last_clean":"2024-09-10T16:12:10.864300+0000","last_became_active":"2024-09-10T16:09:08.854941+0000","last_became_peered":"2024-09-10T16:09:08.854941+0000","last_unstale":"2024-09-10T16:12:10.864300+0000","last_undegraded":"2024-09-10T16:12:10.864300+0000","last_fullsized":"2024-09-10T16:12:10.864300+0000","mapping_epoch":50,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:13:14.846691+0000","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17414,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4257,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":148,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.864168+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:12:10.864168+0000","last_peered":"2024-09-10T16:12:10.864168+0000","last_clean":"2024-09-10T16:12:10.864168+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:12:10.864168+0000","last_undegraded":"2024-09-10T16:12:10.864168+0000","last_fullsized":"2024-09-10T16:12:10.864168+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20781","reported_seq":29578,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.862793+0000","last_change":"2024-09-10T16:10:20.907889+0000","last_active":"2024-09-10T16:12:10.862793+0000","last_peered":"2024-09-10T16:12:10.862793+0000","last_clean":"2024-09-10T16:12:10.862793+0000","last_became_active":"2024-09-10T16:10:20.852065+0000","last_became_peered":"2024-09-10T16:10:20.852065+0000","last_unstale":"2024-09-10T16:12:10.862793+0000","last_undegraded":"2024-09-10T16:12:10.862793+0000","last_fullsized":"2024-09-10T16:12:10.862793+0000","mapping_epoch":56,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:34.069549+0000","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18461,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4307,"num_bytes_recovered":8675,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":124,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.865426+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:12:10.865426+0000","last_peered":"2024-09-10T16:12:10.865426+0000","last_clean":"2024-09-10T16:12:10.865426+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:12:10.865426+0000","last_undegraded":"2024-09-10T16:12:10.865426+0000","last_fullsized":"2024-09-10T16:12:10.865426+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'21170","reported_seq":30064,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.862524+0000","last_change":"2024-09-10T16:10:50.728349+0000","last_active":"2024-09-10T16:12:10.862524+0000","last_peered":"2024-09-10T16:12:10.862524+0000","last_clean":"2024-09-10T16:12:10.862524+0000","last_became_active":"2024-09-10T16:08:10.860438+0000","last_became_peered":"2024-09-10T16:08:10.860438+0000","last_unstale":"2024-09-10T16:12:10.862524+0000","last_undegraded":"2024-09-10T16:12:10.862524+0000","last_fullsized":"2024-09-10T16:12:10.862524+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:38:13.781380+0000","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18724,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20838","reported_seq":20995,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.956229+0000","last_change":"2024-09-10T16:12:10.956160+0000","last_active":"2024-09-10T16:12:10.956229+0000","last_peered":"2024-09-10T16:12:10.956229+0000","last_clean":"2024-09-10T16:12:10.956229+0000","last_became_active":"2024-09-10T16:12:10.866692+0000","last_became_peered":"2024-09-10T16:12:10.866692+0000","last_unstale":"2024-09-10T16:12:10.956229+0000","last_undegraded":"2024-09-10T16:12:10.956229+0000","last_fullsized":"2024-09-10T16:12:10.956229+0000","mapping_epoch":64,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:01:48.931853+0000","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17469,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4305,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.865568+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:12:10.865568+0000","last_peered":"2024-09-10T16:12:10.865568+0000","last_clean":"2024-09-10T16:12:10.865568+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:12:10.865568+0000","last_undegraded":"2024-09-10T16:12:10.865568+0000","last_fullsized":"2024-09-10T16:12:10.865568+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20598","reported_seq":29208,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.865708+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:12:10.865708+0000","last_peered":"2024-09-10T16:12:10.865708+0000","last_clean":"2024-09-10T16:12:10.865708+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:12:10.865708+0000","last_undegraded":"2024-09-10T16:12:10.865708+0000","last_fullsized":"2024-09-10T16:12:10.865708+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16988,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":138,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.863216+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:12:10.863216+0000","last_peered":"2024-09-10T16:12:10.863216+0000","last_clean":"2024-09-10T16:12:10.863216+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:12:10.863216+0000","last_undegraded":"2024-09-10T16:12:10.863216+0000","last_fullsized":"2024-09-10T16:12:10.863216+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'21048","reported_seq":21218,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.865639+0000","last_change":"2024-09-10T16:11:17.315372+0000","last_active":"2024-09-10T16:12:10.865639+0000","last_peered":"2024-09-10T16:12:10.865639+0000","last_clean":"2024-09-10T16:12:10.865639+0000","last_became_active":"2024-09-10T16:11:17.234487+0000","last_became_peered":"2024-09-10T16:11:17.234487+0000","last_unstale":"2024-09-10T16:12:10.865639+0000","last_undegraded":"2024-09-10T16:12:10.865639+0000","last_fullsized":"2024-09-10T16:12:10.865639+0000","mapping_epoch":60,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:19:00.114196+0000","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17303,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4373,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.865649+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:12:10.865649+0000","last_peered":"2024-09-10T16:12:10.865649+0000","last_clean":"2024-09-10T16:12:10.865649+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:12:10.865649+0000","last_undegraded":"2024-09-10T16:12:10.865649+0000","last_fullsized":"2024-09-10T16:12:10.865649+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":119,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.865758+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:12:10.865758+0000","last_peered":"2024-09-10T16:12:10.865758+0000","last_clean":"2024-09-10T16:12:10.865758+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:12:10.865758+0000","last_undegraded":"2024-09-10T16:12:10.865758+0000","last_fullsized":"2024-09-10T16:12:10.865758+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'21198","reported_seq":21369,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T16:12:10.865540+0000","last_change":"2024-09-10T16:11:44.391021+0000","last_active":"2024-09-10T16:12:10.865540+0000","last_peered":"2024-09-10T16:12:10.865540+0000","last_clean":"2024-09-10T16:12:10.865540+0000","last_became_active":"2024-09-10T16:11:44.312647+0000","last_became_peered":"2024-09-10T16:11:44.312647+0000","last_unstale":"2024-09-10T16:12:10.865540+0000","last_undegraded":"2024-09-10T16:12:10.865540+0000","last_fullsized":"2024-09-10T16:12:10.865540+0000","mapping_epoch":62,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:33:28.041611+0000","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17361,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4325,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292096,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51390,"num_bytes_recovered":33098578,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":138895360,"data_stored":138805035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161022,"ondisk_log_size":161022,"up":32,"acting":31,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986351,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":434108,"kb_used_data":69228,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93937732,"statfs":{"total":96636764160,"available":96192237568,"internally_reserved":0,"allocated":70889472,"data_stored":70501777,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373616298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659296,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":505664,"kb_used_data":560,"kb_used_omap":4,"kb_used_meta":505083,"kb_avail":93866176,"statfs":{"total":96636764160,"available":96118964224,"internally_reserved":0,"allocated":573440,"data_stored":185281,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":517205678},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182579,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":546956,"kb_used_data":68284,"kb_used_omap":4,"kb_used_meta":478651,"kb_avail":93824884,"statfs":{"total":96636764160,"available":96076681216,"internally_reserved":0,"allocated":69922816,"data_stored":69523818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":490139306},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":69378048,"data_stored":69341918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":69488640,"data_stored":69459736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:12:44.592 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:12:44.884 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:12:44.885 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:12:44.897 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":524,"stamp":"2024-09-10T16:12:43.806566+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292098,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51392,"num_bytes_recovered":33098597,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161025,"ondisk_log_size":161025,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1488148,"kb_used_data":131164,"kb_used_omap":12,"kb_used_meta":1356915,"kb_avail":281627372,"statfs":{"total":289910292480,"available":288386428928,"internally_reserved":0,"allocated":134311936,"data_stored":133104231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1389480962},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1023,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1023,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001767"},"pg_stats":[{"pgid":"2.f","version":"49'20672","reported_seq":29321,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:12:37.467107+0000","last_change":"2024-09-10T16:09:57.088531+0000","last_active":"2024-09-10T16:12:37.467107+0000","last_peered":"2024-09-10T16:12:37.467107+0000","last_clean":"2024-09-10T16:12:37.467107+0000","last_became_active":"2024-09-10T16:09:57.005755+0000","last_became_peered":"2024-09-10T16:09:57.005755+0000","last_unstale":"2024-09-10T16:12:37.467107+0000","last_undegraded":"2024-09-10T16:12:37.467107+0000","last_fullsized":"2024-09-10T16:12:37.467107+0000","mapping_epoch":54,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:09:19.329697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20672,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4259,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20844","reported_seq":21005,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:12:37.466623+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:12:37.466623+0000","last_peered":"2024-09-10T16:12:37.466623+0000","last_clean":"2024-09-10T16:12:37.466623+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:12:37.466623+0000","last_undegraded":"2024-09-10T16:12:37.466623+0000","last_fullsized":"2024-09-10T16:12:37.466623+0000","mapping_epoch":35,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18949,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20798","reported_seq":29524,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:12:37.468218+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:12:37.468218+0000","last_peered":"2024-09-10T16:12:37.468218+0000","last_clean":"2024-09-10T16:12:37.468218+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:12:37.468218+0000","last_undegraded":"2024-09-10T16:12:37.468218+0000","last_fullsized":"2024-09-10T16:12:37.468218+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18802,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20660","reported_seq":20789,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:12:37.466462+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:12:37.466462+0000","last_peered":"2024-09-10T16:12:37.466462+0000","last_clean":"2024-09-10T16:12:37.466462+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:12:37.466462+0000","last_undegraded":"2024-09-10T16:12:37.466462+0000","last_fullsized":"2024-09-10T16:12:37.466462+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20660,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20891","reported_seq":21069,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:12:37.467317+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:12:37.467317+0000","last_peered":"2024-09-10T16:12:37.467317+0000","last_clean":"2024-09-10T16:12:37.467317+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:12:37.467317+0000","last_undegraded":"2024-09-10T16:12:37.467317+0000","last_fullsized":"2024-09-10T16:12:37.467317+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17602,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'21061","reported_seq":21238,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:12:37.467825+0000","last_change":"2024-09-10T16:10:52.241702+0000","last_active":"2024-09-10T16:12:37.467825+0000","last_peered":"2024-09-10T16:12:37.467825+0000","last_clean":"2024-09-10T16:12:37.467825+0000","last_became_active":"2024-09-10T16:10:52.171236+0000","last_became_peered":"2024-09-10T16:10:52.171236+0000","last_unstale":"2024-09-10T16:12:37.467825+0000","last_undegraded":"2024-09-10T16:12:37.467825+0000","last_fullsized":"2024-09-10T16:12:37.467825+0000","mapping_epoch":58,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:04:46.778280+0000","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17673,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4372,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'21082","reported_seq":21243,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:12:37.536804+0000","last_change":"2024-09-10T16:12:37.536371+0000","last_active":"2024-09-10T16:12:37.536804+0000","last_peered":"2024-09-10T16:12:37.536804+0000","last_clean":"2024-09-10T16:12:37.536804+0000","last_became_active":"2024-09-10T16:12:37.469111+0000","last_became_peered":"2024-09-10T16:12:37.469111+0000","last_unstale":"2024-09-10T16:12:37.536804+0000","last_undegraded":"2024-09-10T16:12:37.536804+0000","last_fullsized":"2024-09-10T16:12:37.536804+0000","mapping_epoch":66,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:41:36.917715+0000","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17581,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4237,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'21314","reported_seq":21492,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:12:37.467167+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:12:37.467167+0000","last_peered":"2024-09-10T16:12:37.467167+0000","last_clean":"2024-09-10T16:12:37.467167+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:12:37.467167+0000","last_undegraded":"2024-09-10T16:12:37.467167+0000","last_fullsized":"2024-09-10T16:12:37.467167+0000","mapping_epoch":34,"log_start":"45'11300","ondisk_log_start":"45'11300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18161,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":118,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:12:37.467726+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:12:37.467726+0000","last_peered":"2024-09-10T16:12:37.467726+0000","last_clean":"2024-09-10T16:12:37.467726+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:12:37.467726+0000","last_undegraded":"2024-09-10T16:12:37.467726+0000","last_fullsized":"2024-09-10T16:12:37.467726+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20457","reported_seq":20649,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:12:37.468066+0000","last_change":"2024-09-10T16:09:33.012840+0000","last_active":"2024-09-10T16:12:37.468066+0000","last_peered":"2024-09-10T16:12:37.468066+0000","last_clean":"2024-09-10T16:12:37.468066+0000","last_became_active":"2024-09-10T16:09:32.934056+0000","last_became_peered":"2024-09-10T16:09:32.934056+0000","last_unstale":"2024-09-10T16:12:37.468066+0000","last_undegraded":"2024-09-10T16:12:37.468066+0000","last_fullsized":"2024-09-10T16:12:37.468066+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:08:30.481235+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18276,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":4579,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":142,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:12:37.466770+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:12:37.466770+0000","last_peered":"2024-09-10T16:12:37.466770+0000","last_clean":"2024-09-10T16:12:37.466770+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:12:37.466770+0000","last_undegraded":"2024-09-10T16:12:37.466770+0000","last_fullsized":"2024-09-10T16:12:37.466770+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20910","reported_seq":29611,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:12:37.466684+0000","last_change":"2024-09-10T16:09:08.909580+0000","last_active":"2024-09-10T16:12:37.466684+0000","last_peered":"2024-09-10T16:12:37.466684+0000","last_clean":"2024-09-10T16:12:37.466684+0000","last_became_active":"2024-09-10T16:09:08.854941+0000","last_became_peered":"2024-09-10T16:09:08.854941+0000","last_unstale":"2024-09-10T16:12:37.466684+0000","last_undegraded":"2024-09-10T16:12:37.466684+0000","last_fullsized":"2024-09-10T16:12:37.466684+0000","mapping_epoch":50,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:13:14.846691+0000","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17414,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4257,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":153,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:12:37.466565+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:12:37.466565+0000","last_peered":"2024-09-10T16:12:37.466565+0000","last_clean":"2024-09-10T16:12:37.466565+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:12:37.466565+0000","last_undegraded":"2024-09-10T16:12:37.466565+0000","last_fullsized":"2024-09-10T16:12:37.466565+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20781","reported_seq":29582,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:12:37.466797+0000","last_change":"2024-09-10T16:10:20.907889+0000","last_active":"2024-09-10T16:12:37.466797+0000","last_peered":"2024-09-10T16:12:37.466797+0000","last_clean":"2024-09-10T16:12:37.466797+0000","last_became_active":"2024-09-10T16:10:20.852065+0000","last_became_peered":"2024-09-10T16:10:20.852065+0000","last_unstale":"2024-09-10T16:12:37.466797+0000","last_undegraded":"2024-09-10T16:12:37.466797+0000","last_fullsized":"2024-09-10T16:12:37.466797+0000","mapping_epoch":56,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:34.069549+0000","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18461,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4307,"num_bytes_recovered":8675,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":128,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:12:37.467952+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:12:37.467952+0000","last_peered":"2024-09-10T16:12:37.467952+0000","last_clean":"2024-09-10T16:12:37.467952+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:12:37.467952+0000","last_undegraded":"2024-09-10T16:12:37.467952+0000","last_fullsized":"2024-09-10T16:12:37.467952+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'21170","reported_seq":30068,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:12:37.466337+0000","last_change":"2024-09-10T16:10:50.728349+0000","last_active":"2024-09-10T16:12:37.466337+0000","last_peered":"2024-09-10T16:12:37.466337+0000","last_clean":"2024-09-10T16:12:37.466337+0000","last_became_active":"2024-09-10T16:08:10.860438+0000","last_became_peered":"2024-09-10T16:08:10.860438+0000","last_unstale":"2024-09-10T16:12:37.466337+0000","last_undegraded":"2024-09-10T16:12:37.466337+0000","last_fullsized":"2024-09-10T16:12:37.466337+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:38:13.781380+0000","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18724,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20838","reported_seq":20999,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:12:37.468145+0000","last_change":"2024-09-10T16:12:10.956160+0000","last_active":"2024-09-10T16:12:37.468145+0000","last_peered":"2024-09-10T16:12:37.468145+0000","last_clean":"2024-09-10T16:12:37.468145+0000","last_became_active":"2024-09-10T16:12:10.866692+0000","last_became_peered":"2024-09-10T16:12:10.866692+0000","last_unstale":"2024-09-10T16:12:37.468145+0000","last_undegraded":"2024-09-10T16:12:37.468145+0000","last_fullsized":"2024-09-10T16:12:37.468145+0000","mapping_epoch":64,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:01:48.931853+0000","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17469,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4305,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":118,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:12:37.468138+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:12:37.468138+0000","last_peered":"2024-09-10T16:12:37.468138+0000","last_clean":"2024-09-10T16:12:37.468138+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:12:37.468138+0000","last_undegraded":"2024-09-10T16:12:37.468138+0000","last_fullsized":"2024-09-10T16:12:37.468138+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20598","reported_seq":29212,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:12:37.468033+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:12:37.468033+0000","last_peered":"2024-09-10T16:12:37.468033+0000","last_clean":"2024-09-10T16:12:37.468033+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:12:37.468033+0000","last_undegraded":"2024-09-10T16:12:37.468033+0000","last_fullsized":"2024-09-10T16:12:37.468033+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16988,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":142,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:12:37.466640+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:12:37.466640+0000","last_peered":"2024-09-10T16:12:37.466640+0000","last_clean":"2024-09-10T16:12:37.466640+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:12:37.466640+0000","last_undegraded":"2024-09-10T16:12:37.466640+0000","last_fullsized":"2024-09-10T16:12:37.466640+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'21048","reported_seq":21222,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:12:37.467973+0000","last_change":"2024-09-10T16:11:17.315372+0000","last_active":"2024-09-10T16:12:37.467973+0000","last_peered":"2024-09-10T16:12:37.467973+0000","last_clean":"2024-09-10T16:12:37.467973+0000","last_became_active":"2024-09-10T16:11:17.234487+0000","last_became_peered":"2024-09-10T16:11:17.234487+0000","last_unstale":"2024-09-10T16:12:37.467973+0000","last_undegraded":"2024-09-10T16:12:37.467973+0000","last_fullsized":"2024-09-10T16:12:37.467973+0000","mapping_epoch":60,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:19:00.114196+0000","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17303,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4373,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":118,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:12:37.468010+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:12:37.468010+0000","last_peered":"2024-09-10T16:12:37.468010+0000","last_clean":"2024-09-10T16:12:37.468010+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:12:37.468010+0000","last_undegraded":"2024-09-10T16:12:37.468010+0000","last_fullsized":"2024-09-10T16:12:37.468010+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":123,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:12:37.468110+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:12:37.468110+0000","last_peered":"2024-09-10T16:12:37.468110+0000","last_clean":"2024-09-10T16:12:37.468110+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:12:37.468110+0000","last_undegraded":"2024-09-10T16:12:37.468110+0000","last_fullsized":"2024-09-10T16:12:37.468110+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'21198","reported_seq":21373,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:12:37.467924+0000","last_change":"2024-09-10T16:11:44.391021+0000","last_active":"2024-09-10T16:12:37.467924+0000","last_peered":"2024-09-10T16:12:37.467924+0000","last_clean":"2024-09-10T16:12:37.467924+0000","last_became_active":"2024-09-10T16:11:44.312647+0000","last_became_peered":"2024-09-10T16:11:44.312647+0000","last_unstale":"2024-09-10T16:12:37.467924+0000","last_undegraded":"2024-09-10T16:12:37.467924+0000","last_fullsized":"2024-09-10T16:12:37.467924+0000","mapping_epoch":62,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:33:28.041611+0000","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17361,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4325,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292096,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51390,"num_bytes_recovered":33098578,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131649536,"data_stored":131559211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161022,"ondisk_log_size":161022,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986352,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":433728,"kb_used_data":64688,"kb_used_omap":4,"kb_used_meta":369019,"kb_avail":93938112,"statfs":{"total":96636764160,"available":96192626688,"internally_reserved":0,"allocated":66240512,"data_stored":65841866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":377876138},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659297,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":509848,"kb_used_data":576,"kb_used_omap":4,"kb_used_meta":509243,"kb_avail":93861992,"statfs":{"total":96636764160,"available":96114679808,"internally_reserved":0,"allocated":589824,"data_stored":190714,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":521465518},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182580,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":544572,"kb_used_data":65900,"kb_used_omap":4,"kb_used_meta":478651,"kb_avail":93827268,"statfs":{"total":96636764160,"available":96079122432,"internally_reserved":0,"allocated":67481600,"data_stored":67071651,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":490139306},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":66920448,"data_stored":66884318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":64700416,"data_stored":64671512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:12:44.898 INFO:tasks.lost_unfound.ceph_manager:clean! 2024-09-10T16:12:44.898 INFO:tasks.lost_unfound.ceph_manager:waiting for clean 2024-09-10T16:12:44.899 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:12:45.181 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:12:45.181 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:12:45.193 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":524,"stamp":"2024-09-10T16:12:43.806566+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292098,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51392,"num_bytes_recovered":33098597,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161025,"ondisk_log_size":161025,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1488148,"kb_used_data":131164,"kb_used_omap":12,"kb_used_meta":1356915,"kb_avail":281627372,"statfs":{"total":289910292480,"available":288386428928,"internally_reserved":0,"allocated":134311936,"data_stored":133104231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1389480962},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1023,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1023,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001767"},"pg_stats":[{"pgid":"2.f","version":"49'20672","reported_seq":29321,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:12:37.467107+0000","last_change":"2024-09-10T16:09:57.088531+0000","last_active":"2024-09-10T16:12:37.467107+0000","last_peered":"2024-09-10T16:12:37.467107+0000","last_clean":"2024-09-10T16:12:37.467107+0000","last_became_active":"2024-09-10T16:09:57.005755+0000","last_became_peered":"2024-09-10T16:09:57.005755+0000","last_unstale":"2024-09-10T16:12:37.467107+0000","last_undegraded":"2024-09-10T16:12:37.467107+0000","last_fullsized":"2024-09-10T16:12:37.467107+0000","mapping_epoch":54,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:09:19.329697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20672,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4259,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20844","reported_seq":21005,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:12:37.466623+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:12:37.466623+0000","last_peered":"2024-09-10T16:12:37.466623+0000","last_clean":"2024-09-10T16:12:37.466623+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:12:37.466623+0000","last_undegraded":"2024-09-10T16:12:37.466623+0000","last_fullsized":"2024-09-10T16:12:37.466623+0000","mapping_epoch":35,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18949,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20798","reported_seq":29524,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:12:37.468218+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:12:37.468218+0000","last_peered":"2024-09-10T16:12:37.468218+0000","last_clean":"2024-09-10T16:12:37.468218+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:12:37.468218+0000","last_undegraded":"2024-09-10T16:12:37.468218+0000","last_fullsized":"2024-09-10T16:12:37.468218+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18802,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20660","reported_seq":20789,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:12:37.466462+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:12:37.466462+0000","last_peered":"2024-09-10T16:12:37.466462+0000","last_clean":"2024-09-10T16:12:37.466462+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:12:37.466462+0000","last_undegraded":"2024-09-10T16:12:37.466462+0000","last_fullsized":"2024-09-10T16:12:37.466462+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20660,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20891","reported_seq":21069,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:12:37.467317+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:12:37.467317+0000","last_peered":"2024-09-10T16:12:37.467317+0000","last_clean":"2024-09-10T16:12:37.467317+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:12:37.467317+0000","last_undegraded":"2024-09-10T16:12:37.467317+0000","last_fullsized":"2024-09-10T16:12:37.467317+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17602,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'21061","reported_seq":21238,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:12:37.467825+0000","last_change":"2024-09-10T16:10:52.241702+0000","last_active":"2024-09-10T16:12:37.467825+0000","last_peered":"2024-09-10T16:12:37.467825+0000","last_clean":"2024-09-10T16:12:37.467825+0000","last_became_active":"2024-09-10T16:10:52.171236+0000","last_became_peered":"2024-09-10T16:10:52.171236+0000","last_unstale":"2024-09-10T16:12:37.467825+0000","last_undegraded":"2024-09-10T16:12:37.467825+0000","last_fullsized":"2024-09-10T16:12:37.467825+0000","mapping_epoch":58,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:04:46.778280+0000","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17673,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4372,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'21082","reported_seq":21243,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:12:37.536804+0000","last_change":"2024-09-10T16:12:37.536371+0000","last_active":"2024-09-10T16:12:37.536804+0000","last_peered":"2024-09-10T16:12:37.536804+0000","last_clean":"2024-09-10T16:12:37.536804+0000","last_became_active":"2024-09-10T16:12:37.469111+0000","last_became_peered":"2024-09-10T16:12:37.469111+0000","last_unstale":"2024-09-10T16:12:37.536804+0000","last_undegraded":"2024-09-10T16:12:37.536804+0000","last_fullsized":"2024-09-10T16:12:37.536804+0000","mapping_epoch":66,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:41:36.917715+0000","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17581,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4237,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'21314","reported_seq":21492,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:12:37.467167+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:12:37.467167+0000","last_peered":"2024-09-10T16:12:37.467167+0000","last_clean":"2024-09-10T16:12:37.467167+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:12:37.467167+0000","last_undegraded":"2024-09-10T16:12:37.467167+0000","last_fullsized":"2024-09-10T16:12:37.467167+0000","mapping_epoch":34,"log_start":"45'11300","ondisk_log_start":"45'11300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18161,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":118,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:12:37.467726+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:12:37.467726+0000","last_peered":"2024-09-10T16:12:37.467726+0000","last_clean":"2024-09-10T16:12:37.467726+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:12:37.467726+0000","last_undegraded":"2024-09-10T16:12:37.467726+0000","last_fullsized":"2024-09-10T16:12:37.467726+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20457","reported_seq":20649,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:12:37.468066+0000","last_change":"2024-09-10T16:09:33.012840+0000","last_active":"2024-09-10T16:12:37.468066+0000","last_peered":"2024-09-10T16:12:37.468066+0000","last_clean":"2024-09-10T16:12:37.468066+0000","last_became_active":"2024-09-10T16:09:32.934056+0000","last_became_peered":"2024-09-10T16:09:32.934056+0000","last_unstale":"2024-09-10T16:12:37.468066+0000","last_undegraded":"2024-09-10T16:12:37.468066+0000","last_fullsized":"2024-09-10T16:12:37.468066+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:08:30.481235+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18276,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":4579,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":142,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:12:37.466770+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:12:37.466770+0000","last_peered":"2024-09-10T16:12:37.466770+0000","last_clean":"2024-09-10T16:12:37.466770+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:12:37.466770+0000","last_undegraded":"2024-09-10T16:12:37.466770+0000","last_fullsized":"2024-09-10T16:12:37.466770+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20910","reported_seq":29611,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:12:37.466684+0000","last_change":"2024-09-10T16:09:08.909580+0000","last_active":"2024-09-10T16:12:37.466684+0000","last_peered":"2024-09-10T16:12:37.466684+0000","last_clean":"2024-09-10T16:12:37.466684+0000","last_became_active":"2024-09-10T16:09:08.854941+0000","last_became_peered":"2024-09-10T16:09:08.854941+0000","last_unstale":"2024-09-10T16:12:37.466684+0000","last_undegraded":"2024-09-10T16:12:37.466684+0000","last_fullsized":"2024-09-10T16:12:37.466684+0000","mapping_epoch":50,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:13:14.846691+0000","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17414,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4257,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":153,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:12:37.466565+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:12:37.466565+0000","last_peered":"2024-09-10T16:12:37.466565+0000","last_clean":"2024-09-10T16:12:37.466565+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:12:37.466565+0000","last_undegraded":"2024-09-10T16:12:37.466565+0000","last_fullsized":"2024-09-10T16:12:37.466565+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20781","reported_seq":29582,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:12:37.466797+0000","last_change":"2024-09-10T16:10:20.907889+0000","last_active":"2024-09-10T16:12:37.466797+0000","last_peered":"2024-09-10T16:12:37.466797+0000","last_clean":"2024-09-10T16:12:37.466797+0000","last_became_active":"2024-09-10T16:10:20.852065+0000","last_became_peered":"2024-09-10T16:10:20.852065+0000","last_unstale":"2024-09-10T16:12:37.466797+0000","last_undegraded":"2024-09-10T16:12:37.466797+0000","last_fullsized":"2024-09-10T16:12:37.466797+0000","mapping_epoch":56,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:34.069549+0000","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18461,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4307,"num_bytes_recovered":8675,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":128,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:12:37.467952+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:12:37.467952+0000","last_peered":"2024-09-10T16:12:37.467952+0000","last_clean":"2024-09-10T16:12:37.467952+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:12:37.467952+0000","last_undegraded":"2024-09-10T16:12:37.467952+0000","last_fullsized":"2024-09-10T16:12:37.467952+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'21170","reported_seq":30068,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:12:37.466337+0000","last_change":"2024-09-10T16:10:50.728349+0000","last_active":"2024-09-10T16:12:37.466337+0000","last_peered":"2024-09-10T16:12:37.466337+0000","last_clean":"2024-09-10T16:12:37.466337+0000","last_became_active":"2024-09-10T16:08:10.860438+0000","last_became_peered":"2024-09-10T16:08:10.860438+0000","last_unstale":"2024-09-10T16:12:37.466337+0000","last_undegraded":"2024-09-10T16:12:37.466337+0000","last_fullsized":"2024-09-10T16:12:37.466337+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:38:13.781380+0000","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18724,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20838","reported_seq":20999,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:12:37.468145+0000","last_change":"2024-09-10T16:12:10.956160+0000","last_active":"2024-09-10T16:12:37.468145+0000","last_peered":"2024-09-10T16:12:37.468145+0000","last_clean":"2024-09-10T16:12:37.468145+0000","last_became_active":"2024-09-10T16:12:10.866692+0000","last_became_peered":"2024-09-10T16:12:10.866692+0000","last_unstale":"2024-09-10T16:12:37.468145+0000","last_undegraded":"2024-09-10T16:12:37.468145+0000","last_fullsized":"2024-09-10T16:12:37.468145+0000","mapping_epoch":64,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:01:48.931853+0000","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17469,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4305,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":118,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:12:37.468138+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:12:37.468138+0000","last_peered":"2024-09-10T16:12:37.468138+0000","last_clean":"2024-09-10T16:12:37.468138+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:12:37.468138+0000","last_undegraded":"2024-09-10T16:12:37.468138+0000","last_fullsized":"2024-09-10T16:12:37.468138+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20598","reported_seq":29212,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:12:37.468033+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:12:37.468033+0000","last_peered":"2024-09-10T16:12:37.468033+0000","last_clean":"2024-09-10T16:12:37.468033+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:12:37.468033+0000","last_undegraded":"2024-09-10T16:12:37.468033+0000","last_fullsized":"2024-09-10T16:12:37.468033+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16988,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":142,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:12:37.466640+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:12:37.466640+0000","last_peered":"2024-09-10T16:12:37.466640+0000","last_clean":"2024-09-10T16:12:37.466640+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:12:37.466640+0000","last_undegraded":"2024-09-10T16:12:37.466640+0000","last_fullsized":"2024-09-10T16:12:37.466640+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'21048","reported_seq":21222,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:12:37.467973+0000","last_change":"2024-09-10T16:11:17.315372+0000","last_active":"2024-09-10T16:12:37.467973+0000","last_peered":"2024-09-10T16:12:37.467973+0000","last_clean":"2024-09-10T16:12:37.467973+0000","last_became_active":"2024-09-10T16:11:17.234487+0000","last_became_peered":"2024-09-10T16:11:17.234487+0000","last_unstale":"2024-09-10T16:12:37.467973+0000","last_undegraded":"2024-09-10T16:12:37.467973+0000","last_fullsized":"2024-09-10T16:12:37.467973+0000","mapping_epoch":60,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:19:00.114196+0000","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17303,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4373,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":118,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:12:37.468010+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:12:37.468010+0000","last_peered":"2024-09-10T16:12:37.468010+0000","last_clean":"2024-09-10T16:12:37.468010+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:12:37.468010+0000","last_undegraded":"2024-09-10T16:12:37.468010+0000","last_fullsized":"2024-09-10T16:12:37.468010+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":123,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:12:37.468110+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:12:37.468110+0000","last_peered":"2024-09-10T16:12:37.468110+0000","last_clean":"2024-09-10T16:12:37.468110+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:12:37.468110+0000","last_undegraded":"2024-09-10T16:12:37.468110+0000","last_fullsized":"2024-09-10T16:12:37.468110+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'21198","reported_seq":21373,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:12:37.467924+0000","last_change":"2024-09-10T16:11:44.391021+0000","last_active":"2024-09-10T16:12:37.467924+0000","last_peered":"2024-09-10T16:12:37.467924+0000","last_clean":"2024-09-10T16:12:37.467924+0000","last_became_active":"2024-09-10T16:11:44.312647+0000","last_became_peered":"2024-09-10T16:11:44.312647+0000","last_unstale":"2024-09-10T16:12:37.467924+0000","last_undegraded":"2024-09-10T16:12:37.467924+0000","last_fullsized":"2024-09-10T16:12:37.467924+0000","mapping_epoch":62,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:33:28.041611+0000","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17361,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4325,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292096,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51390,"num_bytes_recovered":33098578,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131649536,"data_stored":131559211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161022,"ondisk_log_size":161022,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986352,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":433728,"kb_used_data":64688,"kb_used_omap":4,"kb_used_meta":369019,"kb_avail":93938112,"statfs":{"total":96636764160,"available":96192626688,"internally_reserved":0,"allocated":66240512,"data_stored":65841866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":377876138},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659297,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":509848,"kb_used_data":576,"kb_used_omap":4,"kb_used_meta":509243,"kb_avail":93861992,"statfs":{"total":96636764160,"available":96114679808,"internally_reserved":0,"allocated":589824,"data_stored":190714,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":521465518},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182580,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":544572,"kb_used_data":65900,"kb_used_omap":4,"kb_used_meta":478651,"kb_avail":93827268,"statfs":{"total":96636764160,"available":96079122432,"internally_reserved":0,"allocated":67481600,"data_stored":67071651,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":490139306},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":66920448,"data_stored":66884318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":64700416,"data_stored":64671512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:12:45.195 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:12:45.475 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:12:45.475 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:12:45.487 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":524,"stamp":"2024-09-10T16:12:43.806566+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292098,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51392,"num_bytes_recovered":33098597,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161025,"ondisk_log_size":161025,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1488148,"kb_used_data":131164,"kb_used_omap":12,"kb_used_meta":1356915,"kb_avail":281627372,"statfs":{"total":289910292480,"available":288386428928,"internally_reserved":0,"allocated":134311936,"data_stored":133104231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1389480962},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1023,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1023,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001767"},"pg_stats":[{"pgid":"2.f","version":"49'20672","reported_seq":29321,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:12:37.467107+0000","last_change":"2024-09-10T16:09:57.088531+0000","last_active":"2024-09-10T16:12:37.467107+0000","last_peered":"2024-09-10T16:12:37.467107+0000","last_clean":"2024-09-10T16:12:37.467107+0000","last_became_active":"2024-09-10T16:09:57.005755+0000","last_became_peered":"2024-09-10T16:09:57.005755+0000","last_unstale":"2024-09-10T16:12:37.467107+0000","last_undegraded":"2024-09-10T16:12:37.467107+0000","last_fullsized":"2024-09-10T16:12:37.467107+0000","mapping_epoch":54,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:09:19.329697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20672,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4259,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20844","reported_seq":21005,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:12:37.466623+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:12:37.466623+0000","last_peered":"2024-09-10T16:12:37.466623+0000","last_clean":"2024-09-10T16:12:37.466623+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:12:37.466623+0000","last_undegraded":"2024-09-10T16:12:37.466623+0000","last_fullsized":"2024-09-10T16:12:37.466623+0000","mapping_epoch":35,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18949,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20798","reported_seq":29524,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:12:37.468218+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:12:37.468218+0000","last_peered":"2024-09-10T16:12:37.468218+0000","last_clean":"2024-09-10T16:12:37.468218+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:12:37.468218+0000","last_undegraded":"2024-09-10T16:12:37.468218+0000","last_fullsized":"2024-09-10T16:12:37.468218+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18802,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20660","reported_seq":20789,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:12:37.466462+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:12:37.466462+0000","last_peered":"2024-09-10T16:12:37.466462+0000","last_clean":"2024-09-10T16:12:37.466462+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:12:37.466462+0000","last_undegraded":"2024-09-10T16:12:37.466462+0000","last_fullsized":"2024-09-10T16:12:37.466462+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20660,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20891","reported_seq":21069,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:12:37.467317+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:12:37.467317+0000","last_peered":"2024-09-10T16:12:37.467317+0000","last_clean":"2024-09-10T16:12:37.467317+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:12:37.467317+0000","last_undegraded":"2024-09-10T16:12:37.467317+0000","last_fullsized":"2024-09-10T16:12:37.467317+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17602,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'21061","reported_seq":21238,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:12:37.467825+0000","last_change":"2024-09-10T16:10:52.241702+0000","last_active":"2024-09-10T16:12:37.467825+0000","last_peered":"2024-09-10T16:12:37.467825+0000","last_clean":"2024-09-10T16:12:37.467825+0000","last_became_active":"2024-09-10T16:10:52.171236+0000","last_became_peered":"2024-09-10T16:10:52.171236+0000","last_unstale":"2024-09-10T16:12:37.467825+0000","last_undegraded":"2024-09-10T16:12:37.467825+0000","last_fullsized":"2024-09-10T16:12:37.467825+0000","mapping_epoch":58,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:04:46.778280+0000","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17673,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4372,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'21082","reported_seq":21243,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:12:37.536804+0000","last_change":"2024-09-10T16:12:37.536371+0000","last_active":"2024-09-10T16:12:37.536804+0000","last_peered":"2024-09-10T16:12:37.536804+0000","last_clean":"2024-09-10T16:12:37.536804+0000","last_became_active":"2024-09-10T16:12:37.469111+0000","last_became_peered":"2024-09-10T16:12:37.469111+0000","last_unstale":"2024-09-10T16:12:37.536804+0000","last_undegraded":"2024-09-10T16:12:37.536804+0000","last_fullsized":"2024-09-10T16:12:37.536804+0000","mapping_epoch":66,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:41:36.917715+0000","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17581,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4237,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'21314","reported_seq":21492,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:12:37.467167+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:12:37.467167+0000","last_peered":"2024-09-10T16:12:37.467167+0000","last_clean":"2024-09-10T16:12:37.467167+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:12:37.467167+0000","last_undegraded":"2024-09-10T16:12:37.467167+0000","last_fullsized":"2024-09-10T16:12:37.467167+0000","mapping_epoch":34,"log_start":"45'11300","ondisk_log_start":"45'11300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18161,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":118,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:12:37.467726+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:12:37.467726+0000","last_peered":"2024-09-10T16:12:37.467726+0000","last_clean":"2024-09-10T16:12:37.467726+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:12:37.467726+0000","last_undegraded":"2024-09-10T16:12:37.467726+0000","last_fullsized":"2024-09-10T16:12:37.467726+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20457","reported_seq":20649,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:12:37.468066+0000","last_change":"2024-09-10T16:09:33.012840+0000","last_active":"2024-09-10T16:12:37.468066+0000","last_peered":"2024-09-10T16:12:37.468066+0000","last_clean":"2024-09-10T16:12:37.468066+0000","last_became_active":"2024-09-10T16:09:32.934056+0000","last_became_peered":"2024-09-10T16:09:32.934056+0000","last_unstale":"2024-09-10T16:12:37.468066+0000","last_undegraded":"2024-09-10T16:12:37.468066+0000","last_fullsized":"2024-09-10T16:12:37.468066+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:08:30.481235+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18276,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":4579,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":142,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:12:37.466770+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:12:37.466770+0000","last_peered":"2024-09-10T16:12:37.466770+0000","last_clean":"2024-09-10T16:12:37.466770+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:12:37.466770+0000","last_undegraded":"2024-09-10T16:12:37.466770+0000","last_fullsized":"2024-09-10T16:12:37.466770+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20910","reported_seq":29611,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:12:37.466684+0000","last_change":"2024-09-10T16:09:08.909580+0000","last_active":"2024-09-10T16:12:37.466684+0000","last_peered":"2024-09-10T16:12:37.466684+0000","last_clean":"2024-09-10T16:12:37.466684+0000","last_became_active":"2024-09-10T16:09:08.854941+0000","last_became_peered":"2024-09-10T16:09:08.854941+0000","last_unstale":"2024-09-10T16:12:37.466684+0000","last_undegraded":"2024-09-10T16:12:37.466684+0000","last_fullsized":"2024-09-10T16:12:37.466684+0000","mapping_epoch":50,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:13:14.846691+0000","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17414,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4257,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":153,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:12:37.466565+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:12:37.466565+0000","last_peered":"2024-09-10T16:12:37.466565+0000","last_clean":"2024-09-10T16:12:37.466565+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:12:37.466565+0000","last_undegraded":"2024-09-10T16:12:37.466565+0000","last_fullsized":"2024-09-10T16:12:37.466565+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20781","reported_seq":29582,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:12:37.466797+0000","last_change":"2024-09-10T16:10:20.907889+0000","last_active":"2024-09-10T16:12:37.466797+0000","last_peered":"2024-09-10T16:12:37.466797+0000","last_clean":"2024-09-10T16:12:37.466797+0000","last_became_active":"2024-09-10T16:10:20.852065+0000","last_became_peered":"2024-09-10T16:10:20.852065+0000","last_unstale":"2024-09-10T16:12:37.466797+0000","last_undegraded":"2024-09-10T16:12:37.466797+0000","last_fullsized":"2024-09-10T16:12:37.466797+0000","mapping_epoch":56,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:34.069549+0000","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18461,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4307,"num_bytes_recovered":8675,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":128,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:12:37.467952+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:12:37.467952+0000","last_peered":"2024-09-10T16:12:37.467952+0000","last_clean":"2024-09-10T16:12:37.467952+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:12:37.467952+0000","last_undegraded":"2024-09-10T16:12:37.467952+0000","last_fullsized":"2024-09-10T16:12:37.467952+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'21170","reported_seq":30068,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:12:37.466337+0000","last_change":"2024-09-10T16:10:50.728349+0000","last_active":"2024-09-10T16:12:37.466337+0000","last_peered":"2024-09-10T16:12:37.466337+0000","last_clean":"2024-09-10T16:12:37.466337+0000","last_became_active":"2024-09-10T16:08:10.860438+0000","last_became_peered":"2024-09-10T16:08:10.860438+0000","last_unstale":"2024-09-10T16:12:37.466337+0000","last_undegraded":"2024-09-10T16:12:37.466337+0000","last_fullsized":"2024-09-10T16:12:37.466337+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:38:13.781380+0000","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18724,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20838","reported_seq":20999,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:12:37.468145+0000","last_change":"2024-09-10T16:12:10.956160+0000","last_active":"2024-09-10T16:12:37.468145+0000","last_peered":"2024-09-10T16:12:37.468145+0000","last_clean":"2024-09-10T16:12:37.468145+0000","last_became_active":"2024-09-10T16:12:10.866692+0000","last_became_peered":"2024-09-10T16:12:10.866692+0000","last_unstale":"2024-09-10T16:12:37.468145+0000","last_undegraded":"2024-09-10T16:12:37.468145+0000","last_fullsized":"2024-09-10T16:12:37.468145+0000","mapping_epoch":64,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:01:48.931853+0000","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17469,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4305,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":118,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:12:37.468138+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:12:37.468138+0000","last_peered":"2024-09-10T16:12:37.468138+0000","last_clean":"2024-09-10T16:12:37.468138+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:12:37.468138+0000","last_undegraded":"2024-09-10T16:12:37.468138+0000","last_fullsized":"2024-09-10T16:12:37.468138+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20598","reported_seq":29212,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:12:37.468033+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:12:37.468033+0000","last_peered":"2024-09-10T16:12:37.468033+0000","last_clean":"2024-09-10T16:12:37.468033+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:12:37.468033+0000","last_undegraded":"2024-09-10T16:12:37.468033+0000","last_fullsized":"2024-09-10T16:12:37.468033+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16988,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":142,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:12:37.466640+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:12:37.466640+0000","last_peered":"2024-09-10T16:12:37.466640+0000","last_clean":"2024-09-10T16:12:37.466640+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:12:37.466640+0000","last_undegraded":"2024-09-10T16:12:37.466640+0000","last_fullsized":"2024-09-10T16:12:37.466640+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'21048","reported_seq":21222,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:12:37.467973+0000","last_change":"2024-09-10T16:11:17.315372+0000","last_active":"2024-09-10T16:12:37.467973+0000","last_peered":"2024-09-10T16:12:37.467973+0000","last_clean":"2024-09-10T16:12:37.467973+0000","last_became_active":"2024-09-10T16:11:17.234487+0000","last_became_peered":"2024-09-10T16:11:17.234487+0000","last_unstale":"2024-09-10T16:12:37.467973+0000","last_undegraded":"2024-09-10T16:12:37.467973+0000","last_fullsized":"2024-09-10T16:12:37.467973+0000","mapping_epoch":60,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:19:00.114196+0000","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17303,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4373,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":118,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:12:37.468010+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:12:37.468010+0000","last_peered":"2024-09-10T16:12:37.468010+0000","last_clean":"2024-09-10T16:12:37.468010+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:12:37.468010+0000","last_undegraded":"2024-09-10T16:12:37.468010+0000","last_fullsized":"2024-09-10T16:12:37.468010+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":123,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:12:37.468110+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:12:37.468110+0000","last_peered":"2024-09-10T16:12:37.468110+0000","last_clean":"2024-09-10T16:12:37.468110+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:12:37.468110+0000","last_undegraded":"2024-09-10T16:12:37.468110+0000","last_fullsized":"2024-09-10T16:12:37.468110+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'21198","reported_seq":21373,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:12:37.467924+0000","last_change":"2024-09-10T16:11:44.391021+0000","last_active":"2024-09-10T16:12:37.467924+0000","last_peered":"2024-09-10T16:12:37.467924+0000","last_clean":"2024-09-10T16:12:37.467924+0000","last_became_active":"2024-09-10T16:11:44.312647+0000","last_became_peered":"2024-09-10T16:11:44.312647+0000","last_unstale":"2024-09-10T16:12:37.467924+0000","last_undegraded":"2024-09-10T16:12:37.467924+0000","last_fullsized":"2024-09-10T16:12:37.467924+0000","mapping_epoch":62,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:33:28.041611+0000","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17361,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4325,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292096,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51390,"num_bytes_recovered":33098578,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131649536,"data_stored":131559211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161022,"ondisk_log_size":161022,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986352,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":433728,"kb_used_data":64688,"kb_used_omap":4,"kb_used_meta":369019,"kb_avail":93938112,"statfs":{"total":96636764160,"available":96192626688,"internally_reserved":0,"allocated":66240512,"data_stored":65841866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":377876138},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659297,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":509848,"kb_used_data":576,"kb_used_omap":4,"kb_used_meta":509243,"kb_avail":93861992,"statfs":{"total":96636764160,"available":96114679808,"internally_reserved":0,"allocated":589824,"data_stored":190714,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":521465518},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182580,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":544572,"kb_used_data":65900,"kb_used_omap":4,"kb_used_meta":478651,"kb_avail":93827268,"statfs":{"total":96636764160,"available":96079122432,"internally_reserved":0,"allocated":67481600,"data_stored":67071651,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":490139306},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":66920448,"data_stored":66884318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":64700416,"data_stored":64671512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:12:45.489 INFO:tasks.lost_unfound.ceph_manager:clean! 2024-09-10T16:12:45.489 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-09-10T16:12:45.499 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-10T16:12:45.499 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:12:45.780 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:12:45.780 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:12:45.793 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":524,"stamp":"2024-09-10T16:12:43.806566+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292098,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51392,"num_bytes_recovered":33098597,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161025,"ondisk_log_size":161025,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1488148,"kb_used_data":131164,"kb_used_omap":12,"kb_used_meta":1356915,"kb_avail":281627372,"statfs":{"total":289910292480,"available":288386428928,"internally_reserved":0,"allocated":134311936,"data_stored":133104231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1389480962},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1023,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1023,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001767"},"pg_stats":[{"pgid":"2.f","version":"49'20672","reported_seq":29321,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:12:37.467107+0000","last_change":"2024-09-10T16:09:57.088531+0000","last_active":"2024-09-10T16:12:37.467107+0000","last_peered":"2024-09-10T16:12:37.467107+0000","last_clean":"2024-09-10T16:12:37.467107+0000","last_became_active":"2024-09-10T16:09:57.005755+0000","last_became_peered":"2024-09-10T16:09:57.005755+0000","last_unstale":"2024-09-10T16:12:37.467107+0000","last_undegraded":"2024-09-10T16:12:37.467107+0000","last_fullsized":"2024-09-10T16:12:37.467107+0000","mapping_epoch":54,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:09:19.329697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20672,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4259,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20844","reported_seq":21005,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:12:37.466623+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:12:37.466623+0000","last_peered":"2024-09-10T16:12:37.466623+0000","last_clean":"2024-09-10T16:12:37.466623+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:12:37.466623+0000","last_undegraded":"2024-09-10T16:12:37.466623+0000","last_fullsized":"2024-09-10T16:12:37.466623+0000","mapping_epoch":35,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18949,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20798","reported_seq":29524,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:12:37.468218+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:12:37.468218+0000","last_peered":"2024-09-10T16:12:37.468218+0000","last_clean":"2024-09-10T16:12:37.468218+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:12:37.468218+0000","last_undegraded":"2024-09-10T16:12:37.468218+0000","last_fullsized":"2024-09-10T16:12:37.468218+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18802,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20660","reported_seq":20789,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:12:37.466462+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:12:37.466462+0000","last_peered":"2024-09-10T16:12:37.466462+0000","last_clean":"2024-09-10T16:12:37.466462+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:12:37.466462+0000","last_undegraded":"2024-09-10T16:12:37.466462+0000","last_fullsized":"2024-09-10T16:12:37.466462+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20660,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20891","reported_seq":21069,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:12:37.467317+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:12:37.467317+0000","last_peered":"2024-09-10T16:12:37.467317+0000","last_clean":"2024-09-10T16:12:37.467317+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:12:37.467317+0000","last_undegraded":"2024-09-10T16:12:37.467317+0000","last_fullsized":"2024-09-10T16:12:37.467317+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17602,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'21061","reported_seq":21238,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:12:37.467825+0000","last_change":"2024-09-10T16:10:52.241702+0000","last_active":"2024-09-10T16:12:37.467825+0000","last_peered":"2024-09-10T16:12:37.467825+0000","last_clean":"2024-09-10T16:12:37.467825+0000","last_became_active":"2024-09-10T16:10:52.171236+0000","last_became_peered":"2024-09-10T16:10:52.171236+0000","last_unstale":"2024-09-10T16:12:37.467825+0000","last_undegraded":"2024-09-10T16:12:37.467825+0000","last_fullsized":"2024-09-10T16:12:37.467825+0000","mapping_epoch":58,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:04:46.778280+0000","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17673,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4372,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'21082","reported_seq":21243,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:12:37.536804+0000","last_change":"2024-09-10T16:12:37.536371+0000","last_active":"2024-09-10T16:12:37.536804+0000","last_peered":"2024-09-10T16:12:37.536804+0000","last_clean":"2024-09-10T16:12:37.536804+0000","last_became_active":"2024-09-10T16:12:37.469111+0000","last_became_peered":"2024-09-10T16:12:37.469111+0000","last_unstale":"2024-09-10T16:12:37.536804+0000","last_undegraded":"2024-09-10T16:12:37.536804+0000","last_fullsized":"2024-09-10T16:12:37.536804+0000","mapping_epoch":66,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:41:36.917715+0000","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17581,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4237,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'21314","reported_seq":21492,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:12:37.467167+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:12:37.467167+0000","last_peered":"2024-09-10T16:12:37.467167+0000","last_clean":"2024-09-10T16:12:37.467167+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:12:37.467167+0000","last_undegraded":"2024-09-10T16:12:37.467167+0000","last_fullsized":"2024-09-10T16:12:37.467167+0000","mapping_epoch":34,"log_start":"45'11300","ondisk_log_start":"45'11300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18161,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":118,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:12:37.467726+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:12:37.467726+0000","last_peered":"2024-09-10T16:12:37.467726+0000","last_clean":"2024-09-10T16:12:37.467726+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:12:37.467726+0000","last_undegraded":"2024-09-10T16:12:37.467726+0000","last_fullsized":"2024-09-10T16:12:37.467726+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20457","reported_seq":20649,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:12:37.468066+0000","last_change":"2024-09-10T16:09:33.012840+0000","last_active":"2024-09-10T16:12:37.468066+0000","last_peered":"2024-09-10T16:12:37.468066+0000","last_clean":"2024-09-10T16:12:37.468066+0000","last_became_active":"2024-09-10T16:09:32.934056+0000","last_became_peered":"2024-09-10T16:09:32.934056+0000","last_unstale":"2024-09-10T16:12:37.468066+0000","last_undegraded":"2024-09-10T16:12:37.468066+0000","last_fullsized":"2024-09-10T16:12:37.468066+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:08:30.481235+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18276,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":4579,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":142,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:12:37.466770+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:12:37.466770+0000","last_peered":"2024-09-10T16:12:37.466770+0000","last_clean":"2024-09-10T16:12:37.466770+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:12:37.466770+0000","last_undegraded":"2024-09-10T16:12:37.466770+0000","last_fullsized":"2024-09-10T16:12:37.466770+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20910","reported_seq":29611,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:12:37.466684+0000","last_change":"2024-09-10T16:09:08.909580+0000","last_active":"2024-09-10T16:12:37.466684+0000","last_peered":"2024-09-10T16:12:37.466684+0000","last_clean":"2024-09-10T16:12:37.466684+0000","last_became_active":"2024-09-10T16:09:08.854941+0000","last_became_peered":"2024-09-10T16:09:08.854941+0000","last_unstale":"2024-09-10T16:12:37.466684+0000","last_undegraded":"2024-09-10T16:12:37.466684+0000","last_fullsized":"2024-09-10T16:12:37.466684+0000","mapping_epoch":50,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:13:14.846691+0000","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17414,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4257,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":153,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:12:37.466565+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:12:37.466565+0000","last_peered":"2024-09-10T16:12:37.466565+0000","last_clean":"2024-09-10T16:12:37.466565+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:12:37.466565+0000","last_undegraded":"2024-09-10T16:12:37.466565+0000","last_fullsized":"2024-09-10T16:12:37.466565+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20781","reported_seq":29582,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:12:37.466797+0000","last_change":"2024-09-10T16:10:20.907889+0000","last_active":"2024-09-10T16:12:37.466797+0000","last_peered":"2024-09-10T16:12:37.466797+0000","last_clean":"2024-09-10T16:12:37.466797+0000","last_became_active":"2024-09-10T16:10:20.852065+0000","last_became_peered":"2024-09-10T16:10:20.852065+0000","last_unstale":"2024-09-10T16:12:37.466797+0000","last_undegraded":"2024-09-10T16:12:37.466797+0000","last_fullsized":"2024-09-10T16:12:37.466797+0000","mapping_epoch":56,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:34.069549+0000","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18461,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4307,"num_bytes_recovered":8675,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":128,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:12:37.467952+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:12:37.467952+0000","last_peered":"2024-09-10T16:12:37.467952+0000","last_clean":"2024-09-10T16:12:37.467952+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:12:37.467952+0000","last_undegraded":"2024-09-10T16:12:37.467952+0000","last_fullsized":"2024-09-10T16:12:37.467952+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'21170","reported_seq":30068,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:12:37.466337+0000","last_change":"2024-09-10T16:10:50.728349+0000","last_active":"2024-09-10T16:12:37.466337+0000","last_peered":"2024-09-10T16:12:37.466337+0000","last_clean":"2024-09-10T16:12:37.466337+0000","last_became_active":"2024-09-10T16:08:10.860438+0000","last_became_peered":"2024-09-10T16:08:10.860438+0000","last_unstale":"2024-09-10T16:12:37.466337+0000","last_undegraded":"2024-09-10T16:12:37.466337+0000","last_fullsized":"2024-09-10T16:12:37.466337+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:38:13.781380+0000","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18724,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20838","reported_seq":20999,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:12:37.468145+0000","last_change":"2024-09-10T16:12:10.956160+0000","last_active":"2024-09-10T16:12:37.468145+0000","last_peered":"2024-09-10T16:12:37.468145+0000","last_clean":"2024-09-10T16:12:37.468145+0000","last_became_active":"2024-09-10T16:12:10.866692+0000","last_became_peered":"2024-09-10T16:12:10.866692+0000","last_unstale":"2024-09-10T16:12:37.468145+0000","last_undegraded":"2024-09-10T16:12:37.468145+0000","last_fullsized":"2024-09-10T16:12:37.468145+0000","mapping_epoch":64,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:01:48.931853+0000","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17469,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4305,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":118,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:12:37.468138+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:12:37.468138+0000","last_peered":"2024-09-10T16:12:37.468138+0000","last_clean":"2024-09-10T16:12:37.468138+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:12:37.468138+0000","last_undegraded":"2024-09-10T16:12:37.468138+0000","last_fullsized":"2024-09-10T16:12:37.468138+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20598","reported_seq":29212,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:12:37.468033+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:12:37.468033+0000","last_peered":"2024-09-10T16:12:37.468033+0000","last_clean":"2024-09-10T16:12:37.468033+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:12:37.468033+0000","last_undegraded":"2024-09-10T16:12:37.468033+0000","last_fullsized":"2024-09-10T16:12:37.468033+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16988,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":142,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:12:37.466640+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:12:37.466640+0000","last_peered":"2024-09-10T16:12:37.466640+0000","last_clean":"2024-09-10T16:12:37.466640+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:12:37.466640+0000","last_undegraded":"2024-09-10T16:12:37.466640+0000","last_fullsized":"2024-09-10T16:12:37.466640+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'21048","reported_seq":21222,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:12:37.467973+0000","last_change":"2024-09-10T16:11:17.315372+0000","last_active":"2024-09-10T16:12:37.467973+0000","last_peered":"2024-09-10T16:12:37.467973+0000","last_clean":"2024-09-10T16:12:37.467973+0000","last_became_active":"2024-09-10T16:11:17.234487+0000","last_became_peered":"2024-09-10T16:11:17.234487+0000","last_unstale":"2024-09-10T16:12:37.467973+0000","last_undegraded":"2024-09-10T16:12:37.467973+0000","last_fullsized":"2024-09-10T16:12:37.467973+0000","mapping_epoch":60,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:19:00.114196+0000","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17303,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4373,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":118,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:12:37.468010+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:12:37.468010+0000","last_peered":"2024-09-10T16:12:37.468010+0000","last_clean":"2024-09-10T16:12:37.468010+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:12:37.468010+0000","last_undegraded":"2024-09-10T16:12:37.468010+0000","last_fullsized":"2024-09-10T16:12:37.468010+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":123,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:12:37.468110+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:12:37.468110+0000","last_peered":"2024-09-10T16:12:37.468110+0000","last_clean":"2024-09-10T16:12:37.468110+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:12:37.468110+0000","last_undegraded":"2024-09-10T16:12:37.468110+0000","last_fullsized":"2024-09-10T16:12:37.468110+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'21198","reported_seq":21373,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:12:37.467924+0000","last_change":"2024-09-10T16:11:44.391021+0000","last_active":"2024-09-10T16:12:37.467924+0000","last_peered":"2024-09-10T16:12:37.467924+0000","last_clean":"2024-09-10T16:12:37.467924+0000","last_became_active":"2024-09-10T16:11:44.312647+0000","last_became_peered":"2024-09-10T16:11:44.312647+0000","last_unstale":"2024-09-10T16:12:37.467924+0000","last_undegraded":"2024-09-10T16:12:37.467924+0000","last_fullsized":"2024-09-10T16:12:37.467924+0000","mapping_epoch":62,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:33:28.041611+0000","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17361,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4325,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292096,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51390,"num_bytes_recovered":33098578,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131649536,"data_stored":131559211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161022,"ondisk_log_size":161022,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986352,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":433728,"kb_used_data":64688,"kb_used_omap":4,"kb_used_meta":369019,"kb_avail":93938112,"statfs":{"total":96636764160,"available":96192626688,"internally_reserved":0,"allocated":66240512,"data_stored":65841866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":377876138},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659297,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":509848,"kb_used_data":576,"kb_used_omap":4,"kb_used_meta":509243,"kb_avail":93861992,"statfs":{"total":96636764160,"available":96114679808,"internally_reserved":0,"allocated":589824,"data_stored":190714,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":521465518},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182580,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":544572,"kb_used_data":65900,"kb_used_omap":4,"kb_used_meta":478651,"kb_avail":93827268,"statfs":{"total":96636764160,"available":96079122432,"internally_reserved":0,"allocated":67481600,"data_stored":67071651,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":490139306},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":66920448,"data_stored":66884318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":64700416,"data_stored":64671512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:12:45.794 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:12:46.078 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:12:46.079 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:12:46.091 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":525,"stamp":"2024-09-10T16:12:45.806874+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292098,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51392,"num_bytes_recovered":33098597,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161025,"ondisk_log_size":161025,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1488148,"kb_used_data":131164,"kb_used_omap":12,"kb_used_meta":1356915,"kb_avail":281627372,"statfs":{"total":289910292480,"available":288386428928,"internally_reserved":0,"allocated":134311936,"data_stored":133104231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1389480962},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.342606"},"pg_stats":[{"pgid":"2.f","version":"49'20672","reported_seq":29321,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:12:37.467107+0000","last_change":"2024-09-10T16:09:57.088531+0000","last_active":"2024-09-10T16:12:37.467107+0000","last_peered":"2024-09-10T16:12:37.467107+0000","last_clean":"2024-09-10T16:12:37.467107+0000","last_became_active":"2024-09-10T16:09:57.005755+0000","last_became_peered":"2024-09-10T16:09:57.005755+0000","last_unstale":"2024-09-10T16:12:37.467107+0000","last_undegraded":"2024-09-10T16:12:37.467107+0000","last_fullsized":"2024-09-10T16:12:37.467107+0000","mapping_epoch":54,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:09:19.329697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20672,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4259,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20844","reported_seq":21005,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:12:37.466623+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:12:37.466623+0000","last_peered":"2024-09-10T16:12:37.466623+0000","last_clean":"2024-09-10T16:12:37.466623+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:12:37.466623+0000","last_undegraded":"2024-09-10T16:12:37.466623+0000","last_fullsized":"2024-09-10T16:12:37.466623+0000","mapping_epoch":35,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18949,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20798","reported_seq":29524,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:12:37.468218+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:12:37.468218+0000","last_peered":"2024-09-10T16:12:37.468218+0000","last_clean":"2024-09-10T16:12:37.468218+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:12:37.468218+0000","last_undegraded":"2024-09-10T16:12:37.468218+0000","last_fullsized":"2024-09-10T16:12:37.468218+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18802,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20660","reported_seq":20789,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:12:37.466462+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:12:37.466462+0000","last_peered":"2024-09-10T16:12:37.466462+0000","last_clean":"2024-09-10T16:12:37.466462+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:12:37.466462+0000","last_undegraded":"2024-09-10T16:12:37.466462+0000","last_fullsized":"2024-09-10T16:12:37.466462+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20660,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20891","reported_seq":21069,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:12:37.467317+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:12:37.467317+0000","last_peered":"2024-09-10T16:12:37.467317+0000","last_clean":"2024-09-10T16:12:37.467317+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:12:37.467317+0000","last_undegraded":"2024-09-10T16:12:37.467317+0000","last_fullsized":"2024-09-10T16:12:37.467317+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17602,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'21061","reported_seq":21238,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:12:37.467825+0000","last_change":"2024-09-10T16:10:52.241702+0000","last_active":"2024-09-10T16:12:37.467825+0000","last_peered":"2024-09-10T16:12:37.467825+0000","last_clean":"2024-09-10T16:12:37.467825+0000","last_became_active":"2024-09-10T16:10:52.171236+0000","last_became_peered":"2024-09-10T16:10:52.171236+0000","last_unstale":"2024-09-10T16:12:37.467825+0000","last_undegraded":"2024-09-10T16:12:37.467825+0000","last_fullsized":"2024-09-10T16:12:37.467825+0000","mapping_epoch":58,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:04:46.778280+0000","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17673,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4372,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'21082","reported_seq":21243,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:12:37.536804+0000","last_change":"2024-09-10T16:12:37.536371+0000","last_active":"2024-09-10T16:12:37.536804+0000","last_peered":"2024-09-10T16:12:37.536804+0000","last_clean":"2024-09-10T16:12:37.536804+0000","last_became_active":"2024-09-10T16:12:37.469111+0000","last_became_peered":"2024-09-10T16:12:37.469111+0000","last_unstale":"2024-09-10T16:12:37.536804+0000","last_undegraded":"2024-09-10T16:12:37.536804+0000","last_fullsized":"2024-09-10T16:12:37.536804+0000","mapping_epoch":66,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:41:36.917715+0000","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17581,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4237,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'21314","reported_seq":21492,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:12:37.467167+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:12:37.467167+0000","last_peered":"2024-09-10T16:12:37.467167+0000","last_clean":"2024-09-10T16:12:37.467167+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:12:37.467167+0000","last_undegraded":"2024-09-10T16:12:37.467167+0000","last_fullsized":"2024-09-10T16:12:37.467167+0000","mapping_epoch":34,"log_start":"45'11300","ondisk_log_start":"45'11300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18161,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":118,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:12:37.467726+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:12:37.467726+0000","last_peered":"2024-09-10T16:12:37.467726+0000","last_clean":"2024-09-10T16:12:37.467726+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:12:37.467726+0000","last_undegraded":"2024-09-10T16:12:37.467726+0000","last_fullsized":"2024-09-10T16:12:37.467726+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20457","reported_seq":20649,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:12:37.468066+0000","last_change":"2024-09-10T16:09:33.012840+0000","last_active":"2024-09-10T16:12:37.468066+0000","last_peered":"2024-09-10T16:12:37.468066+0000","last_clean":"2024-09-10T16:12:37.468066+0000","last_became_active":"2024-09-10T16:09:32.934056+0000","last_became_peered":"2024-09-10T16:09:32.934056+0000","last_unstale":"2024-09-10T16:12:37.468066+0000","last_undegraded":"2024-09-10T16:12:37.468066+0000","last_fullsized":"2024-09-10T16:12:37.468066+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:08:30.481235+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18276,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":4579,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":142,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:12:37.466770+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:12:37.466770+0000","last_peered":"2024-09-10T16:12:37.466770+0000","last_clean":"2024-09-10T16:12:37.466770+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:12:37.466770+0000","last_undegraded":"2024-09-10T16:12:37.466770+0000","last_fullsized":"2024-09-10T16:12:37.466770+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20910","reported_seq":29611,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:12:37.466684+0000","last_change":"2024-09-10T16:09:08.909580+0000","last_active":"2024-09-10T16:12:37.466684+0000","last_peered":"2024-09-10T16:12:37.466684+0000","last_clean":"2024-09-10T16:12:37.466684+0000","last_became_active":"2024-09-10T16:09:08.854941+0000","last_became_peered":"2024-09-10T16:09:08.854941+0000","last_unstale":"2024-09-10T16:12:37.466684+0000","last_undegraded":"2024-09-10T16:12:37.466684+0000","last_fullsized":"2024-09-10T16:12:37.466684+0000","mapping_epoch":50,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:13:14.846691+0000","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17414,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4257,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":153,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:12:37.466565+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:12:37.466565+0000","last_peered":"2024-09-10T16:12:37.466565+0000","last_clean":"2024-09-10T16:12:37.466565+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:12:37.466565+0000","last_undegraded":"2024-09-10T16:12:37.466565+0000","last_fullsized":"2024-09-10T16:12:37.466565+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20781","reported_seq":29582,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:12:37.466797+0000","last_change":"2024-09-10T16:10:20.907889+0000","last_active":"2024-09-10T16:12:37.466797+0000","last_peered":"2024-09-10T16:12:37.466797+0000","last_clean":"2024-09-10T16:12:37.466797+0000","last_became_active":"2024-09-10T16:10:20.852065+0000","last_became_peered":"2024-09-10T16:10:20.852065+0000","last_unstale":"2024-09-10T16:12:37.466797+0000","last_undegraded":"2024-09-10T16:12:37.466797+0000","last_fullsized":"2024-09-10T16:12:37.466797+0000","mapping_epoch":56,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:34.069549+0000","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18461,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4307,"num_bytes_recovered":8675,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":128,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:12:37.467952+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:12:37.467952+0000","last_peered":"2024-09-10T16:12:37.467952+0000","last_clean":"2024-09-10T16:12:37.467952+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:12:37.467952+0000","last_undegraded":"2024-09-10T16:12:37.467952+0000","last_fullsized":"2024-09-10T16:12:37.467952+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'21170","reported_seq":30068,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:12:37.466337+0000","last_change":"2024-09-10T16:10:50.728349+0000","last_active":"2024-09-10T16:12:37.466337+0000","last_peered":"2024-09-10T16:12:37.466337+0000","last_clean":"2024-09-10T16:12:37.466337+0000","last_became_active":"2024-09-10T16:08:10.860438+0000","last_became_peered":"2024-09-10T16:08:10.860438+0000","last_unstale":"2024-09-10T16:12:37.466337+0000","last_undegraded":"2024-09-10T16:12:37.466337+0000","last_fullsized":"2024-09-10T16:12:37.466337+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:38:13.781380+0000","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18724,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20838","reported_seq":20999,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:12:37.468145+0000","last_change":"2024-09-10T16:12:10.956160+0000","last_active":"2024-09-10T16:12:37.468145+0000","last_peered":"2024-09-10T16:12:37.468145+0000","last_clean":"2024-09-10T16:12:37.468145+0000","last_became_active":"2024-09-10T16:12:10.866692+0000","last_became_peered":"2024-09-10T16:12:10.866692+0000","last_unstale":"2024-09-10T16:12:37.468145+0000","last_undegraded":"2024-09-10T16:12:37.468145+0000","last_fullsized":"2024-09-10T16:12:37.468145+0000","mapping_epoch":64,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:01:48.931853+0000","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17469,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4305,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":118,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:12:37.468138+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:12:37.468138+0000","last_peered":"2024-09-10T16:12:37.468138+0000","last_clean":"2024-09-10T16:12:37.468138+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:12:37.468138+0000","last_undegraded":"2024-09-10T16:12:37.468138+0000","last_fullsized":"2024-09-10T16:12:37.468138+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20598","reported_seq":29212,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:12:37.468033+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:12:37.468033+0000","last_peered":"2024-09-10T16:12:37.468033+0000","last_clean":"2024-09-10T16:12:37.468033+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:12:37.468033+0000","last_undegraded":"2024-09-10T16:12:37.468033+0000","last_fullsized":"2024-09-10T16:12:37.468033+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16988,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":142,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:12:37.466640+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:12:37.466640+0000","last_peered":"2024-09-10T16:12:37.466640+0000","last_clean":"2024-09-10T16:12:37.466640+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:12:37.466640+0000","last_undegraded":"2024-09-10T16:12:37.466640+0000","last_fullsized":"2024-09-10T16:12:37.466640+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'21048","reported_seq":21222,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:12:37.467973+0000","last_change":"2024-09-10T16:11:17.315372+0000","last_active":"2024-09-10T16:12:37.467973+0000","last_peered":"2024-09-10T16:12:37.467973+0000","last_clean":"2024-09-10T16:12:37.467973+0000","last_became_active":"2024-09-10T16:11:17.234487+0000","last_became_peered":"2024-09-10T16:11:17.234487+0000","last_unstale":"2024-09-10T16:12:37.467973+0000","last_undegraded":"2024-09-10T16:12:37.467973+0000","last_fullsized":"2024-09-10T16:12:37.467973+0000","mapping_epoch":60,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:19:00.114196+0000","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17303,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4373,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":118,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:12:37.468010+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:12:37.468010+0000","last_peered":"2024-09-10T16:12:37.468010+0000","last_clean":"2024-09-10T16:12:37.468010+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:12:37.468010+0000","last_undegraded":"2024-09-10T16:12:37.468010+0000","last_fullsized":"2024-09-10T16:12:37.468010+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":123,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:12:37.468110+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:12:37.468110+0000","last_peered":"2024-09-10T16:12:37.468110+0000","last_clean":"2024-09-10T16:12:37.468110+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:12:37.468110+0000","last_undegraded":"2024-09-10T16:12:37.468110+0000","last_fullsized":"2024-09-10T16:12:37.468110+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'21198","reported_seq":21373,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:12:37.467924+0000","last_change":"2024-09-10T16:11:44.391021+0000","last_active":"2024-09-10T16:12:37.467924+0000","last_peered":"2024-09-10T16:12:37.467924+0000","last_clean":"2024-09-10T16:12:37.467924+0000","last_became_active":"2024-09-10T16:11:44.312647+0000","last_became_peered":"2024-09-10T16:11:44.312647+0000","last_unstale":"2024-09-10T16:12:37.467924+0000","last_undegraded":"2024-09-10T16:12:37.467924+0000","last_fullsized":"2024-09-10T16:12:37.467924+0000","mapping_epoch":62,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:33:28.041611+0000","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17361,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4325,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292096,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51390,"num_bytes_recovered":33098578,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131649536,"data_stored":131559211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161022,"ondisk_log_size":161022,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986352,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":433728,"kb_used_data":64688,"kb_used_omap":4,"kb_used_meta":369019,"kb_avail":93938112,"statfs":{"total":96636764160,"available":96192626688,"internally_reserved":0,"allocated":66240512,"data_stored":65841866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":377876138},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659298,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":509848,"kb_used_data":576,"kb_used_omap":4,"kb_used_meta":509243,"kb_avail":93861992,"statfs":{"total":96636764160,"available":96114679808,"internally_reserved":0,"allocated":589824,"data_stored":190714,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":521465518},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182580,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":544572,"kb_used_data":65900,"kb_used_omap":4,"kb_used_meta":478651,"kb_avail":93827268,"statfs":{"total":96636764160,"available":96079122432,"internally_reserved":0,"allocated":67481600,"data_stored":67071651,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":490139306},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":66920448,"data_stored":66884318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":64700416,"data_stored":64671512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:12:46.092 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-10T16:12:46.092 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:12:46.374 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:12:46.375 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:12:46.386 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":525,"stamp":"2024-09-10T16:12:45.806874+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292098,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51392,"num_bytes_recovered":33098597,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161025,"ondisk_log_size":161025,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1488148,"kb_used_data":131164,"kb_used_omap":12,"kb_used_meta":1356915,"kb_avail":281627372,"statfs":{"total":289910292480,"available":288386428928,"internally_reserved":0,"allocated":134311936,"data_stored":133104231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1389480962},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.342606"},"pg_stats":[{"pgid":"2.f","version":"49'20672","reported_seq":29321,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:12:37.467107+0000","last_change":"2024-09-10T16:09:57.088531+0000","last_active":"2024-09-10T16:12:37.467107+0000","last_peered":"2024-09-10T16:12:37.467107+0000","last_clean":"2024-09-10T16:12:37.467107+0000","last_became_active":"2024-09-10T16:09:57.005755+0000","last_became_peered":"2024-09-10T16:09:57.005755+0000","last_unstale":"2024-09-10T16:12:37.467107+0000","last_undegraded":"2024-09-10T16:12:37.467107+0000","last_fullsized":"2024-09-10T16:12:37.467107+0000","mapping_epoch":54,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:09:19.329697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20672,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4259,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20844","reported_seq":21005,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:12:37.466623+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:12:37.466623+0000","last_peered":"2024-09-10T16:12:37.466623+0000","last_clean":"2024-09-10T16:12:37.466623+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:12:37.466623+0000","last_undegraded":"2024-09-10T16:12:37.466623+0000","last_fullsized":"2024-09-10T16:12:37.466623+0000","mapping_epoch":35,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18949,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20798","reported_seq":29524,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:12:37.468218+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:12:37.468218+0000","last_peered":"2024-09-10T16:12:37.468218+0000","last_clean":"2024-09-10T16:12:37.468218+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:12:37.468218+0000","last_undegraded":"2024-09-10T16:12:37.468218+0000","last_fullsized":"2024-09-10T16:12:37.468218+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18802,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20660","reported_seq":20789,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:12:37.466462+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:12:37.466462+0000","last_peered":"2024-09-10T16:12:37.466462+0000","last_clean":"2024-09-10T16:12:37.466462+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:12:37.466462+0000","last_undegraded":"2024-09-10T16:12:37.466462+0000","last_fullsized":"2024-09-10T16:12:37.466462+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20660,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20891","reported_seq":21069,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:12:37.467317+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:12:37.467317+0000","last_peered":"2024-09-10T16:12:37.467317+0000","last_clean":"2024-09-10T16:12:37.467317+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:12:37.467317+0000","last_undegraded":"2024-09-10T16:12:37.467317+0000","last_fullsized":"2024-09-10T16:12:37.467317+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17602,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'21061","reported_seq":21238,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:12:37.467825+0000","last_change":"2024-09-10T16:10:52.241702+0000","last_active":"2024-09-10T16:12:37.467825+0000","last_peered":"2024-09-10T16:12:37.467825+0000","last_clean":"2024-09-10T16:12:37.467825+0000","last_became_active":"2024-09-10T16:10:52.171236+0000","last_became_peered":"2024-09-10T16:10:52.171236+0000","last_unstale":"2024-09-10T16:12:37.467825+0000","last_undegraded":"2024-09-10T16:12:37.467825+0000","last_fullsized":"2024-09-10T16:12:37.467825+0000","mapping_epoch":58,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:04:46.778280+0000","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17673,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4372,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'21082","reported_seq":21243,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:12:37.536804+0000","last_change":"2024-09-10T16:12:37.536371+0000","last_active":"2024-09-10T16:12:37.536804+0000","last_peered":"2024-09-10T16:12:37.536804+0000","last_clean":"2024-09-10T16:12:37.536804+0000","last_became_active":"2024-09-10T16:12:37.469111+0000","last_became_peered":"2024-09-10T16:12:37.469111+0000","last_unstale":"2024-09-10T16:12:37.536804+0000","last_undegraded":"2024-09-10T16:12:37.536804+0000","last_fullsized":"2024-09-10T16:12:37.536804+0000","mapping_epoch":66,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:41:36.917715+0000","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17581,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4237,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'21314","reported_seq":21492,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:12:37.467167+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:12:37.467167+0000","last_peered":"2024-09-10T16:12:37.467167+0000","last_clean":"2024-09-10T16:12:37.467167+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:12:37.467167+0000","last_undegraded":"2024-09-10T16:12:37.467167+0000","last_fullsized":"2024-09-10T16:12:37.467167+0000","mapping_epoch":34,"log_start":"45'11300","ondisk_log_start":"45'11300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18161,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":118,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:12:37.467726+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:12:37.467726+0000","last_peered":"2024-09-10T16:12:37.467726+0000","last_clean":"2024-09-10T16:12:37.467726+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:12:37.467726+0000","last_undegraded":"2024-09-10T16:12:37.467726+0000","last_fullsized":"2024-09-10T16:12:37.467726+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20457","reported_seq":20649,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:12:37.468066+0000","last_change":"2024-09-10T16:09:33.012840+0000","last_active":"2024-09-10T16:12:37.468066+0000","last_peered":"2024-09-10T16:12:37.468066+0000","last_clean":"2024-09-10T16:12:37.468066+0000","last_became_active":"2024-09-10T16:09:32.934056+0000","last_became_peered":"2024-09-10T16:09:32.934056+0000","last_unstale":"2024-09-10T16:12:37.468066+0000","last_undegraded":"2024-09-10T16:12:37.468066+0000","last_fullsized":"2024-09-10T16:12:37.468066+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:08:30.481235+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18276,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":4579,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":142,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:12:37.466770+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:12:37.466770+0000","last_peered":"2024-09-10T16:12:37.466770+0000","last_clean":"2024-09-10T16:12:37.466770+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:12:37.466770+0000","last_undegraded":"2024-09-10T16:12:37.466770+0000","last_fullsized":"2024-09-10T16:12:37.466770+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20910","reported_seq":29611,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:12:37.466684+0000","last_change":"2024-09-10T16:09:08.909580+0000","last_active":"2024-09-10T16:12:37.466684+0000","last_peered":"2024-09-10T16:12:37.466684+0000","last_clean":"2024-09-10T16:12:37.466684+0000","last_became_active":"2024-09-10T16:09:08.854941+0000","last_became_peered":"2024-09-10T16:09:08.854941+0000","last_unstale":"2024-09-10T16:12:37.466684+0000","last_undegraded":"2024-09-10T16:12:37.466684+0000","last_fullsized":"2024-09-10T16:12:37.466684+0000","mapping_epoch":50,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:13:14.846691+0000","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17414,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4257,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":153,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:12:37.466565+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:12:37.466565+0000","last_peered":"2024-09-10T16:12:37.466565+0000","last_clean":"2024-09-10T16:12:37.466565+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:12:37.466565+0000","last_undegraded":"2024-09-10T16:12:37.466565+0000","last_fullsized":"2024-09-10T16:12:37.466565+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20781","reported_seq":29582,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:12:37.466797+0000","last_change":"2024-09-10T16:10:20.907889+0000","last_active":"2024-09-10T16:12:37.466797+0000","last_peered":"2024-09-10T16:12:37.466797+0000","last_clean":"2024-09-10T16:12:37.466797+0000","last_became_active":"2024-09-10T16:10:20.852065+0000","last_became_peered":"2024-09-10T16:10:20.852065+0000","last_unstale":"2024-09-10T16:12:37.466797+0000","last_undegraded":"2024-09-10T16:12:37.466797+0000","last_fullsized":"2024-09-10T16:12:37.466797+0000","mapping_epoch":56,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:34.069549+0000","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18461,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4307,"num_bytes_recovered":8675,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":128,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:12:37.467952+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:12:37.467952+0000","last_peered":"2024-09-10T16:12:37.467952+0000","last_clean":"2024-09-10T16:12:37.467952+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:12:37.467952+0000","last_undegraded":"2024-09-10T16:12:37.467952+0000","last_fullsized":"2024-09-10T16:12:37.467952+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'21170","reported_seq":30068,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:12:37.466337+0000","last_change":"2024-09-10T16:10:50.728349+0000","last_active":"2024-09-10T16:12:37.466337+0000","last_peered":"2024-09-10T16:12:37.466337+0000","last_clean":"2024-09-10T16:12:37.466337+0000","last_became_active":"2024-09-10T16:08:10.860438+0000","last_became_peered":"2024-09-10T16:08:10.860438+0000","last_unstale":"2024-09-10T16:12:37.466337+0000","last_undegraded":"2024-09-10T16:12:37.466337+0000","last_fullsized":"2024-09-10T16:12:37.466337+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:38:13.781380+0000","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18724,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20838","reported_seq":20999,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:12:37.468145+0000","last_change":"2024-09-10T16:12:10.956160+0000","last_active":"2024-09-10T16:12:37.468145+0000","last_peered":"2024-09-10T16:12:37.468145+0000","last_clean":"2024-09-10T16:12:37.468145+0000","last_became_active":"2024-09-10T16:12:10.866692+0000","last_became_peered":"2024-09-10T16:12:10.866692+0000","last_unstale":"2024-09-10T16:12:37.468145+0000","last_undegraded":"2024-09-10T16:12:37.468145+0000","last_fullsized":"2024-09-10T16:12:37.468145+0000","mapping_epoch":64,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:01:48.931853+0000","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17469,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4305,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":118,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:12:37.468138+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:12:37.468138+0000","last_peered":"2024-09-10T16:12:37.468138+0000","last_clean":"2024-09-10T16:12:37.468138+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:12:37.468138+0000","last_undegraded":"2024-09-10T16:12:37.468138+0000","last_fullsized":"2024-09-10T16:12:37.468138+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20598","reported_seq":29212,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:12:37.468033+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:12:37.468033+0000","last_peered":"2024-09-10T16:12:37.468033+0000","last_clean":"2024-09-10T16:12:37.468033+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:12:37.468033+0000","last_undegraded":"2024-09-10T16:12:37.468033+0000","last_fullsized":"2024-09-10T16:12:37.468033+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16988,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":142,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:12:37.466640+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:12:37.466640+0000","last_peered":"2024-09-10T16:12:37.466640+0000","last_clean":"2024-09-10T16:12:37.466640+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:12:37.466640+0000","last_undegraded":"2024-09-10T16:12:37.466640+0000","last_fullsized":"2024-09-10T16:12:37.466640+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'21048","reported_seq":21222,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:12:37.467973+0000","last_change":"2024-09-10T16:11:17.315372+0000","last_active":"2024-09-10T16:12:37.467973+0000","last_peered":"2024-09-10T16:12:37.467973+0000","last_clean":"2024-09-10T16:12:37.467973+0000","last_became_active":"2024-09-10T16:11:17.234487+0000","last_became_peered":"2024-09-10T16:11:17.234487+0000","last_unstale":"2024-09-10T16:12:37.467973+0000","last_undegraded":"2024-09-10T16:12:37.467973+0000","last_fullsized":"2024-09-10T16:12:37.467973+0000","mapping_epoch":60,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:19:00.114196+0000","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17303,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4373,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":118,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:12:37.468010+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:12:37.468010+0000","last_peered":"2024-09-10T16:12:37.468010+0000","last_clean":"2024-09-10T16:12:37.468010+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:12:37.468010+0000","last_undegraded":"2024-09-10T16:12:37.468010+0000","last_fullsized":"2024-09-10T16:12:37.468010+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":123,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:12:37.468110+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:12:37.468110+0000","last_peered":"2024-09-10T16:12:37.468110+0000","last_clean":"2024-09-10T16:12:37.468110+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:12:37.468110+0000","last_undegraded":"2024-09-10T16:12:37.468110+0000","last_fullsized":"2024-09-10T16:12:37.468110+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'21198","reported_seq":21373,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:12:37.467924+0000","last_change":"2024-09-10T16:11:44.391021+0000","last_active":"2024-09-10T16:12:37.467924+0000","last_peered":"2024-09-10T16:12:37.467924+0000","last_clean":"2024-09-10T16:12:37.467924+0000","last_became_active":"2024-09-10T16:11:44.312647+0000","last_became_peered":"2024-09-10T16:11:44.312647+0000","last_unstale":"2024-09-10T16:12:37.467924+0000","last_undegraded":"2024-09-10T16:12:37.467924+0000","last_fullsized":"2024-09-10T16:12:37.467924+0000","mapping_epoch":62,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:33:28.041611+0000","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17361,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4325,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292096,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51390,"num_bytes_recovered":33098578,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131649536,"data_stored":131559211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161022,"ondisk_log_size":161022,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986352,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":433728,"kb_used_data":64688,"kb_used_omap":4,"kb_used_meta":369019,"kb_avail":93938112,"statfs":{"total":96636764160,"available":96192626688,"internally_reserved":0,"allocated":66240512,"data_stored":65841866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":377876138},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659298,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":509848,"kb_used_data":576,"kb_used_omap":4,"kb_used_meta":509243,"kb_avail":93861992,"statfs":{"total":96636764160,"available":96114679808,"internally_reserved":0,"allocated":589824,"data_stored":190714,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":521465518},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182580,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":544572,"kb_used_data":65900,"kb_used_omap":4,"kb_used_meta":478651,"kb_avail":93827268,"statfs":{"total":96636764160,"available":96079122432,"internally_reserved":0,"allocated":67481600,"data_stored":67071651,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":490139306},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":66920448,"data_stored":66884318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":64700416,"data_stored":64671512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:12:46.388 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T16:12:46.672 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:12:46.673 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":67,"fsid":"7effee29-df05-4351-9741-ee203564ed81","created":"2024-09-10T15:57:14.967498+0000","modified":"2024-09-10T16:12:37.463691+0000","last_up_change":"2024-09-10T15:59:14.117575+0000","last_in_change":"2024-09-10T15:59:07.057956+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":8,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2024-09-10T15:57:30.381324+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":14,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"unfound_pool","create_time":"2024-09-10T15:57:39.758755+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":16,"pg_placement_num":16,"pg_placement_num_target":16,"pg_num_target":16,"pg_num_pending":16,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rados":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"859508e0-e15d-4011-a39f-65e7aa04a129","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":21,"up_from":25,"up_thru":65,"down_at":22,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":1567796493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6801","nonce":1567796493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6803","nonce":1567796493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":1567796493}]},"public_addr":"172.21.15.115:6800/1567796493","cluster_addr":"172.21.15.115:6801/1567796493","heartbeat_back_addr":"172.21.15.115:6803/1567796493","heartbeat_front_addr":"172.21.15.115:6802/1567796493","state":["exists","up"]},{"osd":1,"uuid":"66daed2f-a693-4a13-8d8c-024794a032e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":26,"up_from":41,"up_thru":66,"down_at":27,"lost_at":27,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":257282173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6809","nonce":257282173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6811","nonce":257282173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":257282173}]},"public_addr":"172.21.15.115:6808/257282173","cluster_addr":"172.21.15.115:6809/257282173","heartbeat_back_addr":"172.21.15.115:6811/257282173","heartbeat_front_addr":"172.21.15.115:6810/257282173","state":["exists","up"]},{"osd":2,"uuid":"6a2ab153-d99e-4a5d-b79c-32b7f8ea9ffb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":18,"up_from":31,"up_thru":56,"down_at":19,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":2928349610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6805","nonce":2928349610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6807","nonce":2928349610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":2928349610}]},"public_addr":"172.21.15.115:6804/2928349610","cluster_addr":"172.21.15.115:6805/2928349610","heartbeat_back_addr":"172.21.15.115:6807/2928349610","heartbeat_front_addr":"172.21.15.115:6806/2928349610","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T15:57:23.007077+0000","dead_epoch":22},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T15:57:23.605294+0000","dead_epoch":27},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T15:57:22.852045+0000","dead_epoch":19}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T16:12:47.686 INFO:tasks.ceph:Scrubbing osd.0 2024-09-10T16:12:47.686 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-09-10T16:12:47.816 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-10T16:12:47.817 INFO:teuthology.orchestra.run.smithi115.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-10T16:12:47.817 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-10T16:12:47.827 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-09-10T16:12:48.113 INFO:teuthology.orchestra.run.smithi115.stderr:instructed osd(s) 0 to deep-scrub 2024-09-10T16:12:48.126 INFO:tasks.ceph:Scrubbing osd.1 2024-09-10T16:12:48.126 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-09-10T16:12:48.247 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-10T16:12:48.247 INFO:teuthology.orchestra.run.smithi115.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-10T16:12:48.247 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-10T16:12:48.257 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-09-10T16:12:48.543 INFO:teuthology.orchestra.run.smithi115.stderr:instructed osd(s) 1 to deep-scrub 2024-09-10T16:12:48.556 INFO:tasks.ceph:Scrubbing osd.2 2024-09-10T16:12:48.556 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-09-10T16:12:48.677 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-10T16:12:48.677 INFO:teuthology.orchestra.run.smithi115.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-10T16:12:48.678 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-10T16:12:48.688 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-09-10T16:12:48.971 INFO:teuthology.orchestra.run.smithi115.stderr:instructed osd(s) 2 to deep-scrub 2024-09-10T16:12:48.985 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:12:49.267 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:12:49.268 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:12:49.279 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":526,"stamp":"2024-09-10T16:12:47.807236+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292098,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51392,"num_bytes_recovered":33098597,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161025,"ondisk_log_size":161025,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1480948,"kb_used_data":123964,"kb_used_omap":12,"kb_used_meta":1356915,"kb_avail":281634572,"statfs":{"total":289910292480,"available":288393801728,"internally_reserved":0,"allocated":126939136,"data_stored":125731431,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1389480962},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.341995"},"pg_stats":[{"pgid":"2.f","version":"49'20672","reported_seq":29321,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:12:37.467107+0000","last_change":"2024-09-10T16:09:57.088531+0000","last_active":"2024-09-10T16:12:37.467107+0000","last_peered":"2024-09-10T16:12:37.467107+0000","last_clean":"2024-09-10T16:12:37.467107+0000","last_became_active":"2024-09-10T16:09:57.005755+0000","last_became_peered":"2024-09-10T16:09:57.005755+0000","last_unstale":"2024-09-10T16:12:37.467107+0000","last_undegraded":"2024-09-10T16:12:37.467107+0000","last_fullsized":"2024-09-10T16:12:37.467107+0000","mapping_epoch":54,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:09:19.329697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20672,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4259,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20844","reported_seq":21005,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:12:37.466623+0000","last_change":"2024-09-10T15:58:44.766551+0000","last_active":"2024-09-10T16:12:37.466623+0000","last_peered":"2024-09-10T16:12:37.466623+0000","last_clean":"2024-09-10T16:12:37.466623+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:12:37.466623+0000","last_undegraded":"2024-09-10T16:12:37.466623+0000","last_fullsized":"2024-09-10T16:12:37.466623+0000","mapping_epoch":35,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1918","last_scrub_stamp":"2024-09-10T15:58:44.766424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:44.766424+0000","objects_scrubbed":1891,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:04.477996+0000","scrub_duration":546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18949,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20798","reported_seq":29524,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:12:37.468218+0000","last_change":"2024-09-10T16:02:08.125987+0000","last_active":"2024-09-10T16:12:37.468218+0000","last_peered":"2024-09-10T16:12:37.468218+0000","last_clean":"2024-09-10T16:12:37.468218+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:12:37.468218+0000","last_undegraded":"2024-09-10T16:12:37.468218+0000","last_fullsized":"2024-09-10T16:12:37.468218+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2029","last_scrub_stamp":"2024-09-10T15:58:45.933789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:45.933789+0000","objects_scrubbed":1992,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:18.742248+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18802,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20660","reported_seq":20789,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:12:37.466462+0000","last_change":"2024-09-10T15:58:25.768359+0000","last_active":"2024-09-10T16:12:37.466462+0000","last_peered":"2024-09-10T16:12:37.466462+0000","last_clean":"2024-09-10T16:12:37.466462+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:12:37.466462+0000","last_undegraded":"2024-09-10T16:12:37.466462+0000","last_fullsized":"2024-09-10T16:12:37.466462+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:57:40.749557+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.397286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20660,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20891","reported_seq":21069,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:12:37.467317+0000","last_change":"2024-09-10T15:58:59.134342+0000","last_active":"2024-09-10T16:12:37.467317+0000","last_peered":"2024-09-10T16:12:37.467317+0000","last_clean":"2024-09-10T16:12:37.467317+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:12:37.467317+0000","last_undegraded":"2024-09-10T16:12:37.467317+0000","last_fullsized":"2024-09-10T16:12:37.467317+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3334","last_scrub_stamp":"2024-09-10T15:58:59.134086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:59.134086+0000","objects_scrubbed":3283,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:20.372290+0000","scrub_duration":1203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17602,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'21061","reported_seq":21238,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:12:37.467825+0000","last_change":"2024-09-10T16:10:52.241702+0000","last_active":"2024-09-10T16:12:37.467825+0000","last_peered":"2024-09-10T16:12:37.467825+0000","last_clean":"2024-09-10T16:12:37.467825+0000","last_became_active":"2024-09-10T16:10:52.171236+0000","last_became_peered":"2024-09-10T16:10:52.171236+0000","last_unstale":"2024-09-10T16:12:37.467825+0000","last_undegraded":"2024-09-10T16:12:37.467825+0000","last_fullsized":"2024-09-10T16:12:37.467825+0000","mapping_epoch":58,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3442","last_scrub_stamp":"2024-09-10T15:59:00.194055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.194055+0000","objects_scrubbed":3384,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:04:46.778280+0000","scrub_duration":1292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17673,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4372,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'21082","reported_seq":21243,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:12:37.536804+0000","last_change":"2024-09-10T16:12:37.536371+0000","last_active":"2024-09-10T16:12:37.536804+0000","last_peered":"2024-09-10T16:12:37.536804+0000","last_clean":"2024-09-10T16:12:37.536804+0000","last_became_active":"2024-09-10T16:12:37.469111+0000","last_became_peered":"2024-09-10T16:12:37.469111+0000","last_unstale":"2024-09-10T16:12:37.536804+0000","last_undegraded":"2024-09-10T16:12:37.536804+0000","last_fullsized":"2024-09-10T16:12:37.536804+0000","mapping_epoch":66,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3537","last_scrub_stamp":"2024-09-10T15:59:03.205323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:03.205323+0000","objects_scrubbed":3497,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:41:36.917715+0000","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17581,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4237,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'21314","reported_seq":21492,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:12:37.467167+0000","last_change":"2024-09-10T15:58:57.969832+0000","last_active":"2024-09-10T16:12:37.467167+0000","last_peered":"2024-09-10T16:12:37.467167+0000","last_clean":"2024-09-10T16:12:37.467167+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:12:37.467167+0000","last_undegraded":"2024-09-10T16:12:37.467167+0000","last_fullsized":"2024-09-10T16:12:37.467167+0000","mapping_epoch":34,"log_start":"45'11300","ondisk_log_start":"45'11300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3198","last_scrub_stamp":"2024-09-10T15:58:57.969758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:57.969758+0000","objects_scrubbed":3151,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:09.938843+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18161,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":118,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:12:37.467726+0000","last_change":"2024-09-10T15:59:15.775767+0000","last_active":"2024-09-10T16:12:37.467726+0000","last_peered":"2024-09-10T16:12:37.467726+0000","last_clean":"2024-09-10T16:12:37.467726+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:12:37.467726+0000","last_undegraded":"2024-09-10T16:12:37.467726+0000","last_fullsized":"2024-09-10T16:12:37.467726+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:18.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20457","reported_seq":20649,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:12:37.468066+0000","last_change":"2024-09-10T16:09:33.012840+0000","last_active":"2024-09-10T16:12:37.468066+0000","last_peered":"2024-09-10T16:12:37.468066+0000","last_clean":"2024-09-10T16:12:37.468066+0000","last_became_active":"2024-09-10T16:09:32.934056+0000","last_became_peered":"2024-09-10T16:09:32.934056+0000","last_unstale":"2024-09-10T16:12:37.468066+0000","last_undegraded":"2024-09-10T16:12:37.468066+0000","last_fullsized":"2024-09-10T16:12:37.468066+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2218","last_scrub_stamp":"2024-09-10T15:58:48.061852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:48.061852+0000","objects_scrubbed":2175,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:08:30.481235+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18276,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":4579,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":142,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:12:37.466770+0000","last_change":"2024-09-10T15:58:25.768587+0000","last_active":"2024-09-10T16:12:37.466770+0000","last_peered":"2024-09-10T16:12:37.466770+0000","last_clean":"2024-09-10T16:12:37.466770+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:12:37.466770+0000","last_undegraded":"2024-09-10T16:12:37.466770+0000","last_fullsized":"2024-09-10T16:12:37.466770+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:58.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20910","reported_seq":29611,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:12:37.466684+0000","last_change":"2024-09-10T16:09:08.909580+0000","last_active":"2024-09-10T16:12:37.466684+0000","last_peered":"2024-09-10T16:12:37.466684+0000","last_clean":"2024-09-10T16:12:37.466684+0000","last_became_active":"2024-09-10T16:09:08.854941+0000","last_became_peered":"2024-09-10T16:09:08.854941+0000","last_unstale":"2024-09-10T16:12:37.466684+0000","last_undegraded":"2024-09-10T16:12:37.466684+0000","last_fullsized":"2024-09-10T16:12:37.466684+0000","mapping_epoch":50,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3557","last_scrub_stamp":"2024-09-10T15:59:02.493678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:02.493678+0000","objects_scrubbed":3488,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:13:14.846691+0000","scrub_duration":1472,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17414,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4257,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":153,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:12:37.466565+0000","last_change":"2024-09-10T15:59:15.880461+0000","last_active":"2024-09-10T16:12:37.466565+0000","last_peered":"2024-09-10T16:12:37.466565+0000","last_clean":"2024-09-10T16:12:37.466565+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:12:37.466565+0000","last_undegraded":"2024-09-10T16:12:37.466565+0000","last_fullsized":"2024-09-10T16:12:37.466565+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:37.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20781","reported_seq":29582,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:12:37.466797+0000","last_change":"2024-09-10T16:10:20.907889+0000","last_active":"2024-09-10T16:12:37.466797+0000","last_peered":"2024-09-10T16:12:37.466797+0000","last_clean":"2024-09-10T16:12:37.466797+0000","last_became_active":"2024-09-10T16:10:20.852065+0000","last_became_peered":"2024-09-10T16:10:20.852065+0000","last_unstale":"2024-09-10T16:12:37.466797+0000","last_undegraded":"2024-09-10T16:12:37.466797+0000","last_fullsized":"2024-09-10T16:12:37.466797+0000","mapping_epoch":56,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2356","last_scrub_stamp":"2024-09-10T15:58:49.040618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:49.040618+0000","objects_scrubbed":2316,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:34.069549+0000","scrub_duration":783,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18461,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4307,"num_bytes_recovered":8675,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":128,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:12:37.467952+0000","last_change":"2024-09-10T15:59:15.837397+0000","last_active":"2024-09-10T16:12:37.467952+0000","last_peered":"2024-09-10T16:12:37.467952+0000","last_clean":"2024-09-10T16:12:37.467952+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:12:37.467952+0000","last_undegraded":"2024-09-10T16:12:37.467952+0000","last_fullsized":"2024-09-10T16:12:37.467952+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:36.565189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'21170","reported_seq":30068,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:12:37.466337+0000","last_change":"2024-09-10T16:10:50.728349+0000","last_active":"2024-09-10T16:12:37.466337+0000","last_peered":"2024-09-10T16:12:37.466337+0000","last_clean":"2024-09-10T16:12:37.466337+0000","last_became_active":"2024-09-10T16:08:10.860438+0000","last_became_peered":"2024-09-10T16:08:10.860438+0000","last_unstale":"2024-09-10T16:12:37.466337+0000","last_undegraded":"2024-09-10T16:12:37.466337+0000","last_fullsized":"2024-09-10T16:12:37.466337+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2486","last_scrub_stamp":"2024-09-10T15:58:50.078953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:58:50.078953+0000","objects_scrubbed":2440,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:38:13.781380+0000","scrub_duration":792,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18724,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20838","reported_seq":20999,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:12:37.468145+0000","last_change":"2024-09-10T16:12:10.956160+0000","last_active":"2024-09-10T16:12:37.468145+0000","last_peered":"2024-09-10T16:12:37.468145+0000","last_clean":"2024-09-10T16:12:37.468145+0000","last_became_active":"2024-09-10T16:12:10.866692+0000","last_became_peered":"2024-09-10T16:12:10.866692+0000","last_unstale":"2024-09-10T16:12:37.468145+0000","last_undegraded":"2024-09-10T16:12:37.468145+0000","last_fullsized":"2024-09-10T16:12:37.468145+0000","mapping_epoch":64,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3404","last_scrub_stamp":"2024-09-10T15:59:00.927592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:00.927592+0000","objects_scrubbed":3367,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:01:48.931853+0000","scrub_duration":983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17469,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4305,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":118,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:12:37.468138+0000","last_change":"2024-09-10T15:59:15.989317+0000","last_active":"2024-09-10T16:12:37.468138+0000","last_peered":"2024-09-10T16:12:37.468138+0000","last_clean":"2024-09-10T16:12:37.468138+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:12:37.468138+0000","last_undegraded":"2024-09-10T16:12:37.468138+0000","last_fullsized":"2024-09-10T16:12:37.468138+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:56.569020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20598","reported_seq":29212,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:12:37.468033+0000","last_change":"2024-09-10T16:05:19.929770+0000","last_active":"2024-09-10T16:12:37.468033+0000","last_peered":"2024-09-10T16:12:37.468033+0000","last_clean":"2024-09-10T16:12:37.468033+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:12:37.468033+0000","last_undegraded":"2024-09-10T16:12:37.468033+0000","last_fullsized":"2024-09-10T16:12:37.468033+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3653","last_scrub_stamp":"2024-09-10T15:59:05.298468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:05.298468+0000","objects_scrubbed":3602,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:33.017453+0000","scrub_duration":1289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16988,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":142,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:12:37.466640+0000","last_change":"2024-09-10T15:58:25.769717+0000","last_active":"2024-09-10T16:12:37.466640+0000","last_peered":"2024-09-10T16:12:37.466640+0000","last_clean":"2024-09-10T16:12:37.466640+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:12:37.466640+0000","last_undegraded":"2024-09-10T16:12:37.466640+0000","last_fullsized":"2024-09-10T16:12:37.466640+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:48.070108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'21048","reported_seq":21222,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:12:37.467973+0000","last_change":"2024-09-10T16:11:17.315372+0000","last_active":"2024-09-10T16:12:37.467973+0000","last_peered":"2024-09-10T16:12:37.467973+0000","last_clean":"2024-09-10T16:12:37.467973+0000","last_became_active":"2024-09-10T16:11:17.234487+0000","last_became_peered":"2024-09-10T16:11:17.234487+0000","last_unstale":"2024-09-10T16:12:37.467973+0000","last_undegraded":"2024-09-10T16:12:37.467973+0000","last_fullsized":"2024-09-10T16:12:37.467973+0000","mapping_epoch":60,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3789","last_scrub_stamp":"2024-09-10T15:59:04.437170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:04.437170+0000","objects_scrubbed":3741,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:19:00.114196+0000","scrub_duration":1440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17303,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4373,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":118,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:12:37.468010+0000","last_change":"2024-09-10T15:59:15.915914+0000","last_active":"2024-09-10T16:12:37.468010+0000","last_peered":"2024-09-10T16:12:37.468010+0000","last_clean":"2024-09-10T16:12:37.468010+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:12:37.468010+0000","last_undegraded":"2024-09-10T16:12:37.468010+0000","last_fullsized":"2024-09-10T16:12:37.468010+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.268885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":123,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:12:37.468110+0000","last_change":"2024-09-10T15:59:15.775948+0000","last_active":"2024-09-10T16:12:37.468110+0000","last_peered":"2024-09-10T16:12:37.468110+0000","last_clean":"2024-09-10T16:12:37.468110+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:12:37.468110+0000","last_undegraded":"2024-09-10T16:12:37.468110+0000","last_fullsized":"2024-09-10T16:12:37.468110+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:30.851910+0000","last_clean_scrub_stamp":"2024-09-10T15:57:30.851910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:22.309807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'21198","reported_seq":21373,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:12:37.467924+0000","last_change":"2024-09-10T16:11:44.391021+0000","last_active":"2024-09-10T16:12:37.467924+0000","last_peered":"2024-09-10T16:12:37.467924+0000","last_clean":"2024-09-10T16:12:37.467924+0000","last_became_active":"2024-09-10T16:11:44.312647+0000","last_became_peered":"2024-09-10T16:11:44.312647+0000","last_unstale":"2024-09-10T16:12:37.467924+0000","last_undegraded":"2024-09-10T16:12:37.467924+0000","last_fullsized":"2024-09-10T16:12:37.467924+0000","mapping_epoch":62,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3880","last_scrub_stamp":"2024-09-10T15:59:06.369334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:57:40.749557+0000","last_clean_scrub_stamp":"2024-09-10T15:59:06.369334+0000","objects_scrubbed":3827,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:33:28.041611+0000","scrub_duration":1397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17361,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4325,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292096,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51390,"num_bytes_recovered":33098578,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":124276736,"data_stored":124186411,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161022,"ondisk_log_size":161022,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986353,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":428928,"kb_used_data":59888,"kb_used_omap":4,"kb_used_meta":369019,"kb_avail":93942912,"statfs":{"total":96636764160,"available":96197541888,"internally_reserved":0,"allocated":61325312,"data_stored":60926666,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":377876138},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659298,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":509848,"kb_used_data":576,"kb_used_omap":4,"kb_used_meta":509243,"kb_avail":93861992,"statfs":{"total":96636764160,"available":96114679808,"internally_reserved":0,"allocated":589824,"data_stored":190714,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":521465518},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182581,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":542172,"kb_used_data":63500,"kb_used_omap":4,"kb_used_meta":478651,"kb_avail":93829668,"statfs":{"total":96636764160,"available":96081580032,"internally_reserved":0,"allocated":65024000,"data_stored":64614051,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":490139306},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":64462848,"data_stored":64426718,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":59785216,"data_stored":59756312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:12:49.285 INFO:tasks.ceph:pgid 2.f last_scrub_stamp 2024-09-10T15:57:40.749557+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=15, tm_min=57, tm_sec=40, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=16, tm_min=12, tm_sec=46, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T16:12:49.285 INFO:tasks.ceph:pgid 2.e last_scrub_stamp 2024-09-10T15:58:44.766424+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=15, tm_min=58, tm_sec=44, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=16, tm_min=12, tm_sec=46, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T16:12:49.285 INFO:tasks.ceph:pgid 2.d last_scrub_stamp 2024-09-10T15:58:45.933789+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=15, tm_min=58, tm_sec=45, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=16, tm_min=12, tm_sec=46, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T16:12:49.285 INFO:tasks.ceph:pgid 2.c last_scrub_stamp 2024-09-10T15:57:40.749557+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=15, tm_min=57, tm_sec=40, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=16, tm_min=12, tm_sec=46, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T16:12:49.285 INFO:tasks.ceph:pgid 2.b last_scrub_stamp 2024-09-10T15:58:59.134086+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=15, tm_min=58, tm_sec=59, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=16, tm_min=12, tm_sec=46, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T16:12:49.285 INFO:tasks.ceph:pgid 2.a last_scrub_stamp 2024-09-10T15:59:00.194055+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=15, tm_min=59, tm_sec=0, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=16, tm_min=12, tm_sec=46, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T16:12:49.285 INFO:tasks.ceph:pgid 2.9 last_scrub_stamp 2024-09-10T15:59:03.205323+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=15, tm_min=59, tm_sec=3, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=16, tm_min=12, tm_sec=46, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T16:12:49.286 INFO:tasks.ceph:pgid 2.8 last_scrub_stamp 2024-09-10T15:58:57.969758+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=15, tm_min=58, tm_sec=57, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=16, tm_min=12, tm_sec=46, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T16:12:49.286 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-09-10T15:57:30.851910+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=15, tm_min=57, tm_sec=30, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=16, tm_min=12, tm_sec=46, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T16:12:49.286 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2024-09-10T15:58:48.061852+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=15, tm_min=58, tm_sec=48, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=16, tm_min=12, tm_sec=46, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T16:12:49.286 INFO:tasks.ceph:pgid 1.1 last_scrub_stamp 2024-09-10T15:57:30.851910+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=15, tm_min=57, tm_sec=30, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=16, tm_min=12, tm_sec=46, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T16:12:49.286 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2024-09-10T15:59:02.493678+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=15, tm_min=59, tm_sec=2, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=16, tm_min=12, tm_sec=46, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T16:12:49.286 INFO:tasks.ceph:pgid 1.2 last_scrub_stamp 2024-09-10T15:57:30.851910+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=15, tm_min=57, tm_sec=30, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=16, tm_min=12, tm_sec=46, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T16:12:49.286 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2024-09-10T15:58:49.040618+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=15, tm_min=58, tm_sec=49, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=16, tm_min=12, tm_sec=46, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T16:12:49.286 INFO:tasks.ceph:pgid 1.3 last_scrub_stamp 2024-09-10T15:57:30.851910+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=15, tm_min=57, tm_sec=30, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=16, tm_min=12, tm_sec=46, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T16:12:49.287 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2024-09-10T15:58:50.078953+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=15, tm_min=58, tm_sec=50, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=16, tm_min=12, tm_sec=46, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T16:12:49.287 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2024-09-10T15:59:00.927592+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=15, tm_min=59, tm_sec=0, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=16, tm_min=12, tm_sec=46, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T16:12:49.287 INFO:tasks.ceph:pgid 1.4 last_scrub_stamp 2024-09-10T15:57:30.851910+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=15, tm_min=57, tm_sec=30, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=16, tm_min=12, tm_sec=46, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T16:12:49.287 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2024-09-10T15:59:05.298468+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=15, tm_min=59, tm_sec=5, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=16, tm_min=12, tm_sec=46, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T16:12:49.287 INFO:tasks.ceph:pgid 1.5 last_scrub_stamp 2024-09-10T15:57:30.851910+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=15, tm_min=57, tm_sec=30, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=16, tm_min=12, tm_sec=46, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T16:12:49.287 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2024-09-10T15:59:04.437170+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=15, tm_min=59, tm_sec=4, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=16, tm_min=12, tm_sec=46, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T16:12:49.287 INFO:tasks.ceph:pgid 1.6 last_scrub_stamp 2024-09-10T15:57:30.851910+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=15, tm_min=57, tm_sec=30, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=16, tm_min=12, tm_sec=46, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T16:12:49.287 INFO:tasks.ceph:pgid 1.7 last_scrub_stamp 2024-09-10T15:57:30.851910+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=15, tm_min=57, tm_sec=30, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=16, tm_min=12, tm_sec=46, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T16:12:49.288 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2024-09-10T15:59:06.369334+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=15, tm_min=59, tm_sec=6, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=16, tm_min=12, tm_sec=46, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T16:12:49.288 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-09-10T16:13:09.289 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:13:09.588 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T16:13:09.589 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T16:13:09.604 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":536,"stamp":"2024-09-10T16:13:07.811672+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292098,"num_write_kb":499903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51392,"num_bytes_recovered":33098597,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161037,"ondisk_log_size":161037,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1439540,"kb_used_data":95044,"kb_used_omap":12,"kb_used_meta":1344435,"kb_avail":281675980,"statfs":{"total":289910292480,"available":288436203520,"internally_reserved":0,"allocated":97325056,"data_stored":96124577,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13310,"internal_metadata":1376701442},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002700"},"pg_stats":[{"pgid":"2.f","version":"49'20672","reported_seq":29329,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:12:51.150592+0000","last_change":"2024-09-10T16:12:51.150592+0000","last_active":"2024-09-10T16:12:51.150592+0000","last_peered":"2024-09-10T16:12:51.150592+0000","last_clean":"2024-09-10T16:12:51.150592+0000","last_became_active":"2024-09-10T16:09:57.005755+0000","last_became_peered":"2024-09-10T16:09:57.005755+0000","last_unstale":"2024-09-10T16:12:51.150592+0000","last_undegraded":"2024-09-10T16:12:51.150592+0000","last_fullsized":"2024-09-10T16:12:51.150592+0000","mapping_epoch":54,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"49'20672","last_scrub_stamp":"2024-09-10T16:12:51.150483+0000","last_deep_scrub":"49'20672","last_deep_scrub_stamp":"2024-09-10T16:12:51.150483+0000","last_clean_scrub_stamp":"2024-09-10T16:12:51.150483+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:09:25.487876+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20672,"num_write_kb":41344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4259,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"67'20846","reported_seq":21015,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:12:54.384651+0000","last_change":"2024-09-10T16:12:54.384651+0000","last_active":"2024-09-10T16:12:54.384651+0000","last_peered":"2024-09-10T16:12:54.384651+0000","last_clean":"2024-09-10T16:12:54.384651+0000","last_became_active":"2024-09-10T15:58:25.769343+0000","last_became_peered":"2024-09-10T15:58:25.769343+0000","last_unstale":"2024-09-10T16:12:54.384651+0000","last_undegraded":"2024-09-10T16:12:54.384651+0000","last_fullsized":"2024-09-10T16:12:54.384651+0000","mapping_epoch":35,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'20846","last_scrub_stamp":"2024-09-10T16:12:54.384561+0000","last_deep_scrub":"67'20846","last_deep_scrub_stamp":"2024-09-10T16:12:54.384561+0000","last_clean_scrub_stamp":"2024-09-10T16:12:54.384561+0000","objects_scrubbed":2,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:20:16.323788+0000","scrub_duration":22,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18949,"num_write_kb":34120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20798","reported_seq":29532,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:12:56.817164+0000","last_change":"2024-09-10T16:12:56.817164+0000","last_active":"2024-09-10T16:12:56.817164+0000","last_peered":"2024-09-10T16:12:56.817164+0000","last_clean":"2024-09-10T16:12:56.817164+0000","last_became_active":"2024-09-10T16:02:08.110339+0000","last_became_peered":"2024-09-10T16:02:08.110339+0000","last_unstale":"2024-09-10T16:12:56.817164+0000","last_undegraded":"2024-09-10T16:12:56.817164+0000","last_fullsized":"2024-09-10T16:12:56.817164+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"49'20798","last_scrub_stamp":"2024-09-10T16:12:56.817072+0000","last_deep_scrub":"49'20798","last_deep_scrub_stamp":"2024-09-10T16:12:56.817072+0000","last_clean_scrub_stamp":"2024-09-10T16:12:56.817072+0000","objects_scrubbed":0,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:53:19.255266+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18802,"num_write_kb":33617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4269,"num_bytes_recovered":17481728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20660","reported_seq":20797,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:12:52.350966+0000","last_change":"2024-09-10T16:12:52.350966+0000","last_active":"2024-09-10T16:12:52.350966+0000","last_peered":"2024-09-10T16:12:52.350966+0000","last_clean":"2024-09-10T16:12:52.350966+0000","last_became_active":"2024-09-10T15:58:25.767789+0000","last_became_peered":"2024-09-10T15:58:25.767789+0000","last_unstale":"2024-09-10T16:12:52.350966+0000","last_undegraded":"2024-09-10T16:12:52.350966+0000","last_fullsized":"2024-09-10T16:12:52.350966+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"49'20660","last_scrub_stamp":"2024-09-10T16:12:52.350851+0000","last_deep_scrub":"49'20660","last_deep_scrub_stamp":"2024-09-10T16:12:52.350851+0000","last_clean_scrub_stamp":"2024-09-10T16:12:52.350851+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:24:25.699079+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20660,"num_write_kb":41320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"67'20892","reported_seq":21078,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:12:52.168832+0000","last_change":"2024-09-10T16:12:52.168832+0000","last_active":"2024-09-10T16:12:52.168832+0000","last_peered":"2024-09-10T16:12:52.168832+0000","last_clean":"2024-09-10T16:12:52.168832+0000","last_became_active":"2024-09-10T15:58:24.766631+0000","last_became_peered":"2024-09-10T15:58:24.766631+0000","last_unstale":"2024-09-10T16:12:52.168832+0000","last_undegraded":"2024-09-10T16:12:52.168832+0000","last_fullsized":"2024-09-10T16:12:52.168832+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'20892","last_scrub_stamp":"2024-09-10T16:12:52.168756+0000","last_deep_scrub":"67'20892","last_deep_scrub_stamp":"2024-09-10T16:12:52.168756+0000","last_clean_scrub_stamp":"2024-09-10T16:12:52.168756+0000","objects_scrubbed":1,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:12:15.757003+0000","scrub_duration":18,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17602,"num_write_kb":28640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"67'21062","reported_seq":21248,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:12:55.865422+0000","last_change":"2024-09-10T16:12:55.865363+0000","last_active":"2024-09-10T16:12:55.865422+0000","last_peered":"2024-09-10T16:12:55.865422+0000","last_clean":"2024-09-10T16:12:55.865422+0000","last_became_active":"2024-09-10T16:10:52.171236+0000","last_became_peered":"2024-09-10T16:10:52.171236+0000","last_unstale":"2024-09-10T16:12:55.865422+0000","last_undegraded":"2024-09-10T16:12:55.865422+0000","last_fullsized":"2024-09-10T16:12:55.865422+0000","mapping_epoch":58,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'21062","last_scrub_stamp":"2024-09-10T16:12:55.865266+0000","last_deep_scrub":"67'21062","last_deep_scrub_stamp":"2024-09-10T16:12:55.865266+0000","last_clean_scrub_stamp":"2024-09-10T16:12:55.865266+0000","objects_scrubbed":1,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:46:27.966467+0000","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17673,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4372,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"67'21084","reported_seq":21253,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:12:57.860128+0000","last_change":"2024-09-10T16:12:57.860128+0000","last_active":"2024-09-10T16:12:57.860128+0000","last_peered":"2024-09-10T16:12:57.860128+0000","last_clean":"2024-09-10T16:12:57.860128+0000","last_became_active":"2024-09-10T16:12:37.469111+0000","last_became_peered":"2024-09-10T16:12:37.469111+0000","last_unstale":"2024-09-10T16:12:57.860128+0000","last_undegraded":"2024-09-10T16:12:57.860128+0000","last_fullsized":"2024-09-10T16:12:57.860128+0000","mapping_epoch":66,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'21084","last_scrub_stamp":"2024-09-10T16:12:57.860052+0000","last_deep_scrub":"67'21084","last_deep_scrub_stamp":"2024-09-10T16:12:57.860052+0000","last_clean_scrub_stamp":"2024-09-10T16:12:57.860052+0000","objects_scrubbed":2,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:25:31.435057+0000","scrub_duration":20,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17581,"num_write_kb":28172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4237,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'21314","reported_seq":21500,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:12:50.180597+0000","last_change":"2024-09-10T16:12:50.180597+0000","last_active":"2024-09-10T16:12:50.180597+0000","last_peered":"2024-09-10T16:12:50.180597+0000","last_clean":"2024-09-10T16:12:50.180597+0000","last_became_active":"2024-09-10T15:58:24.766360+0000","last_became_peered":"2024-09-10T15:58:24.766360+0000","last_unstale":"2024-09-10T16:12:50.180597+0000","last_undegraded":"2024-09-10T16:12:50.180597+0000","last_fullsized":"2024-09-10T16:12:50.180597+0000","mapping_epoch":34,"log_start":"45'11300","ondisk_log_start":"45'11300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"49'21314","last_scrub_stamp":"2024-09-10T16:12:50.180460+0000","last_deep_scrub":"49'21314","last_deep_scrub_stamp":"2024-09-10T16:12:50.180460+0000","last_clean_scrub_stamp":"2024-09-10T16:12:50.180460+0000","objects_scrubbed":0,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:03:16.507941+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18161,"num_write_kb":30020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":126,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:12:49.908306+0000","last_change":"2024-09-10T16:12:49.908306+0000","last_active":"2024-09-10T16:12:49.908306+0000","last_peered":"2024-09-10T16:12:49.908306+0000","last_clean":"2024-09-10T16:12:49.908306+0000","last_became_active":"2024-09-10T15:59:15.775313+0000","last_became_peered":"2024-09-10T15:59:15.775313+0000","last_unstale":"2024-09-10T16:12:49.908306+0000","last_undegraded":"2024-09-10T16:12:49.908306+0000","last_fullsized":"2024-09-10T16:12:49.908306+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T16:12:49.908227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T16:12:49.908227+0000","last_clean_scrub_stamp":"2024-09-10T16:12:49.908227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:42:06.991412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114056,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"67'20458","reported_seq":20658,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:12:58.876210+0000","last_change":"2024-09-10T16:12:58.876210+0000","last_active":"2024-09-10T16:12:58.876210+0000","last_peered":"2024-09-10T16:12:58.876210+0000","last_clean":"2024-09-10T16:12:58.876210+0000","last_became_active":"2024-09-10T16:09:32.934056+0000","last_became_peered":"2024-09-10T16:09:32.934056+0000","last_unstale":"2024-09-10T16:12:58.876210+0000","last_undegraded":"2024-09-10T16:12:58.876210+0000","last_fullsized":"2024-09-10T16:12:58.876210+0000","mapping_epoch":52,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'20458","last_scrub_stamp":"2024-09-10T16:12:58.876122+0000","last_deep_scrub":"67'20458","last_deep_scrub_stamp":"2024-09-10T16:12:58.876122+0000","last_clean_scrub_stamp":"2024-09-10T16:12:58.876122+0000","objects_scrubbed":1,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:43:49.382174+0000","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18276,"num_write_kb":32204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":4579,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":151,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:12:49.370005+0000","last_change":"2024-09-10T16:12:49.369959+0000","last_active":"2024-09-10T16:12:49.370005+0000","last_peered":"2024-09-10T16:12:49.370005+0000","last_clean":"2024-09-10T16:12:49.370005+0000","last_became_active":"2024-09-10T15:58:25.767937+0000","last_became_peered":"2024-09-10T15:58:25.767937+0000","last_unstale":"2024-09-10T16:12:49.370005+0000","last_undegraded":"2024-09-10T16:12:49.370005+0000","last_fullsized":"2024-09-10T16:12:49.370005+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T16:12:49.369891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T16:12:49.369891+0000","last_clean_scrub_stamp":"2024-09-10T16:12:49.369891+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:21:56.126336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036926800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20910","reported_seq":29619,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:12:49.195756+0000","last_change":"2024-09-10T16:12:49.195756+0000","last_active":"2024-09-10T16:12:49.195756+0000","last_peered":"2024-09-10T16:12:49.195756+0000","last_clean":"2024-09-10T16:12:49.195756+0000","last_became_active":"2024-09-10T16:09:08.854941+0000","last_became_peered":"2024-09-10T16:09:08.854941+0000","last_unstale":"2024-09-10T16:12:49.195756+0000","last_undegraded":"2024-09-10T16:12:49.195756+0000","last_fullsized":"2024-09-10T16:12:49.195756+0000","mapping_epoch":50,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"49'20910","last_scrub_stamp":"2024-09-10T16:12:49.195682+0000","last_deep_scrub":"49'20910","last_deep_scrub_stamp":"2024-09-10T16:12:49.195682+0000","last_clean_scrub_stamp":"2024-09-10T16:12:49.195682+0000","objects_scrubbed":0,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:06:50.387854+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17414,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4257,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"67'3","reported_seq":162,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:12:48.220763+0000","last_change":"2024-09-10T16:12:48.220763+0000","last_active":"2024-09-10T16:12:48.220763+0000","last_peered":"2024-09-10T16:12:48.220763+0000","last_clean":"2024-09-10T16:12:48.220763+0000","last_became_active":"2024-09-10T15:59:15.880047+0000","last_became_peered":"2024-09-10T15:59:15.880047+0000","last_unstale":"2024-09-10T16:12:48.220763+0000","last_undegraded":"2024-09-10T16:12:48.220763+0000","last_fullsized":"2024-09-10T16:12:48.220763+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'3","last_scrub_stamp":"2024-09-10T16:12:48.220702+0000","last_deep_scrub":"67'3","last_deep_scrub_stamp":"2024-09-10T16:12:48.220702+0000","last_clean_scrub_stamp":"2024-09-10T16:12:48.220702+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:14:29.314419+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.001514811,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"67'20782","reported_seq":29592,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:12:50.418561+0000","last_change":"2024-09-10T16:12:50.418508+0000","last_active":"2024-09-10T16:12:50.418561+0000","last_peered":"2024-09-10T16:12:50.418561+0000","last_clean":"2024-09-10T16:12:50.418561+0000","last_became_active":"2024-09-10T16:10:20.852065+0000","last_became_peered":"2024-09-10T16:10:20.852065+0000","last_unstale":"2024-09-10T16:12:50.418561+0000","last_undegraded":"2024-09-10T16:12:50.418561+0000","last_fullsized":"2024-09-10T16:12:50.418561+0000","mapping_epoch":56,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'20782","last_scrub_stamp":"2024-09-10T16:12:50.418438+0000","last_deep_scrub":"67'20782","last_deep_scrub_stamp":"2024-09-10T16:12:50.418438+0000","last_clean_scrub_stamp":"2024-09-10T16:12:50.418438+0000","objects_scrubbed":1,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:01:04.608624+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18461,"num_write_kb":32292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4307,"num_bytes_recovered":8675,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"67'2","reported_seq":137,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:12:51.905192+0000","last_change":"2024-09-10T16:12:51.905192+0000","last_active":"2024-09-10T16:12:51.905192+0000","last_peered":"2024-09-10T16:12:51.905192+0000","last_clean":"2024-09-10T16:12:51.905192+0000","last_became_active":"2024-09-10T15:59:15.836787+0000","last_became_peered":"2024-09-10T15:59:15.836787+0000","last_unstale":"2024-09-10T16:12:51.905192+0000","last_undegraded":"2024-09-10T16:12:51.905192+0000","last_fullsized":"2024-09-10T16:12:51.905192+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'2","last_scrub_stamp":"2024-09-10T16:12:51.905113+0000","last_deep_scrub":"67'2","last_deep_scrub_stamp":"2024-09-10T16:12:51.905113+0000","last_clean_scrub_stamp":"2024-09-10T16:12:51.905113+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:44:49.828185+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0.001229843,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'21170","reported_seq":30076,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:12:53.330727+0000","last_change":"2024-09-10T16:12:53.330727+0000","last_active":"2024-09-10T16:12:53.330727+0000","last_peered":"2024-09-10T16:12:53.330727+0000","last_clean":"2024-09-10T16:12:53.330727+0000","last_became_active":"2024-09-10T16:08:10.860438+0000","last_became_peered":"2024-09-10T16:08:10.860438+0000","last_unstale":"2024-09-10T16:12:53.330727+0000","last_undegraded":"2024-09-10T16:12:53.330727+0000","last_fullsized":"2024-09-10T16:12:53.330727+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"49'21170","last_scrub_stamp":"2024-09-10T16:12:53.330625+0000","last_deep_scrub":"49'21170","last_deep_scrub_stamp":"2024-09-10T16:12:53.330625+0000","last_clean_scrub_stamp":"2024-09-10T16:12:53.330625+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:06:21.570671+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18724,"num_write_kb":32568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20838","reported_seq":21007,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:13:00.865405+0000","last_change":"2024-09-10T16:13:00.865405+0000","last_active":"2024-09-10T16:13:00.865405+0000","last_peered":"2024-09-10T16:13:00.865405+0000","last_clean":"2024-09-10T16:13:00.865405+0000","last_became_active":"2024-09-10T16:12:10.866692+0000","last_became_peered":"2024-09-10T16:12:10.866692+0000","last_unstale":"2024-09-10T16:13:00.865405+0000","last_undegraded":"2024-09-10T16:13:00.865405+0000","last_fullsized":"2024-09-10T16:13:00.865405+0000","mapping_epoch":64,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"49'20838","last_scrub_stamp":"2024-09-10T16:13:00.865282+0000","last_deep_scrub":"49'20838","last_deep_scrub_stamp":"2024-09-10T16:13:00.865282+0000","last_clean_scrub_stamp":"2024-09-10T16:13:00.865282+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:40:55.843881+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17469,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4305,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":126,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:12:48.865282+0000","last_change":"2024-09-10T16:12:48.865282+0000","last_active":"2024-09-10T16:12:48.865282+0000","last_peered":"2024-09-10T16:12:48.865282+0000","last_clean":"2024-09-10T16:12:48.865282+0000","last_became_active":"2024-09-10T15:59:15.988967+0000","last_became_peered":"2024-09-10T15:59:15.988967+0000","last_unstale":"2024-09-10T16:12:48.865282+0000","last_undegraded":"2024-09-10T16:12:48.865282+0000","last_fullsized":"2024-09-10T16:12:48.865282+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T16:12:48.865184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T16:12:48.865184+0000","last_clean_scrub_stamp":"2024-09-10T16:12:48.865184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:52:59.806750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064917299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20598","reported_seq":29220,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:12:53.911512+0000","last_change":"2024-09-10T16:12:53.911512+0000","last_active":"2024-09-10T16:12:53.911512+0000","last_peered":"2024-09-10T16:12:53.911512+0000","last_clean":"2024-09-10T16:12:53.911512+0000","last_became_active":"2024-09-10T16:05:19.878205+0000","last_became_peered":"2024-09-10T16:05:19.878205+0000","last_unstale":"2024-09-10T16:12:53.911512+0000","last_undegraded":"2024-09-10T16:12:53.911512+0000","last_fullsized":"2024-09-10T16:12:53.911512+0000","mapping_epoch":46,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"49'20598","last_scrub_stamp":"2024-09-10T16:12:53.911416+0000","last_deep_scrub":"49'20598","last_deep_scrub_stamp":"2024-09-10T16:12:53.911416+0000","last_clean_scrub_stamp":"2024-09-10T16:12:53.911416+0000","objects_scrubbed":0,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:58:29.385070+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16988,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":15601664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":151,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:12:51.358129+0000","last_change":"2024-09-10T16:12:51.358071+0000","last_active":"2024-09-10T16:12:51.358129+0000","last_peered":"2024-09-10T16:12:51.358129+0000","last_clean":"2024-09-10T16:12:51.358129+0000","last_became_active":"2024-09-10T15:58:25.769376+0000","last_became_peered":"2024-09-10T15:58:25.769376+0000","last_unstale":"2024-09-10T16:12:51.358129+0000","last_undegraded":"2024-09-10T16:12:51.358129+0000","last_fullsized":"2024-09-10T16:12:51.358129+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T16:12:51.357977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T16:12:51.357977+0000","last_clean_scrub_stamp":"2024-09-10T16:12:51.357977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:56:07.929278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034750299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'21048","reported_seq":21230,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:12:52.881717+0000","last_change":"2024-09-10T16:12:52.881717+0000","last_active":"2024-09-10T16:12:52.881717+0000","last_peered":"2024-09-10T16:12:52.881717+0000","last_clean":"2024-09-10T16:12:52.881717+0000","last_became_active":"2024-09-10T16:11:17.234487+0000","last_became_peered":"2024-09-10T16:11:17.234487+0000","last_unstale":"2024-09-10T16:12:52.881717+0000","last_undegraded":"2024-09-10T16:12:52.881717+0000","last_fullsized":"2024-09-10T16:12:52.881717+0000","mapping_epoch":60,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"49'21048","last_scrub_stamp":"2024-09-10T16:12:52.881603+0000","last_deep_scrub":"49'21048","last_deep_scrub_stamp":"2024-09-10T16:12:52.881603+0000","last_clean_scrub_stamp":"2024-09-10T16:12:52.881603+0000","objects_scrubbed":0,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:21:53.428819+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17303,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4373,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":126,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:12:50.898246+0000","last_change":"2024-09-10T16:12:50.898246+0000","last_active":"2024-09-10T16:12:50.898246+0000","last_peered":"2024-09-10T16:12:50.898246+0000","last_clean":"2024-09-10T16:12:50.898246+0000","last_became_active":"2024-09-10T15:59:15.915592+0000","last_became_peered":"2024-09-10T15:59:15.915592+0000","last_unstale":"2024-09-10T16:12:50.898246+0000","last_undegraded":"2024-09-10T16:12:50.898246+0000","last_fullsized":"2024-09-10T16:12:50.898246+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T16:12:50.898166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T16:12:50.898166+0000","last_clean_scrub_stamp":"2024-09-10T16:12:50.898166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:55:49.711258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088275500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":131,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:12:59.879054+0000","last_change":"2024-09-10T16:12:59.879054+0000","last_active":"2024-09-10T16:12:59.879054+0000","last_peered":"2024-09-10T16:12:59.879054+0000","last_clean":"2024-09-10T16:12:59.879054+0000","last_became_active":"2024-09-10T15:59:15.775617+0000","last_became_peered":"2024-09-10T15:59:15.775617+0000","last_unstale":"2024-09-10T16:12:59.879054+0000","last_undegraded":"2024-09-10T16:12:59.879054+0000","last_fullsized":"2024-09-10T16:12:59.879054+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T16:12:59.878974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T16:12:59.878974+0000","last_clean_scrub_stamp":"2024-09-10T16:12:59.878974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:26:24.097398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001267357,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"67'21200","reported_seq":21383,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:12:54.889677+0000","last_change":"2024-09-10T16:12:54.889677+0000","last_active":"2024-09-10T16:12:54.889677+0000","last_peered":"2024-09-10T16:12:54.889677+0000","last_clean":"2024-09-10T16:12:54.889677+0000","last_became_active":"2024-09-10T16:11:44.312647+0000","last_became_peered":"2024-09-10T16:11:44.312647+0000","last_unstale":"2024-09-10T16:12:54.889677+0000","last_undegraded":"2024-09-10T16:12:54.889677+0000","last_fullsized":"2024-09-10T16:12:54.889677+0000","mapping_epoch":62,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'21200","last_scrub_stamp":"2024-09-10T16:12:54.889603+0000","last_deep_scrub":"67'21200","last_deep_scrub_stamp":"2024-09-10T16:12:54.889603+0000","last_clean_scrub_stamp":"2024-09-10T16:12:54.889603+0000","objects_scrubbed":2,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:30:49.300252+0000","scrub_duration":21,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17361,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4325,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292096,"num_write_kb":499901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51390,"num_bytes_recovered":33098578,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":94785536,"data_stored":94702437,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161032,"ondisk_log_size":161032,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986357,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":392968,"kb_used_data":40568,"kb_used_omap":4,"kb_used_meta":352379,"kb_avail":93978872,"statfs":{"total":96636764160,"available":96234364928,"internally_reserved":0,"allocated":41541632,"data_stored":41146599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":360836778},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659302,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":509840,"kb_used_data":576,"kb_used_omap":4,"kb_used_meta":509243,"kb_avail":93862000,"statfs":{"total":96636764160,"available":96114688000,"internally_reserved":0,"allocated":589824,"data_stored":190714,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":521465518},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182585,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":536732,"kb_used_data":53900,"kb_used_omap":4,"kb_used_meta":482811,"kb_avail":93835108,"statfs":{"total":96636764160,"available":96087150592,"internally_reserved":0,"allocated":55193600,"data_stored":54787264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":494399146},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":54632448,"data_stored":54599931,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":40124416,"data_stored":40099125,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:13:09.607 DEBUG:teuthology.orchestra.run.smithi115:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-09-10T16:13:09.919 INFO:teuthology.misc:Shutting down mds daemons... 2024-09-10T16:13:09.920 INFO:teuthology.misc:Shutting down osd daemons... 2024-09-10T16:13:09.920 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-09-10T16:13:09.920 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T16:13:14.379 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~0s 2024-09-10T16:13:16.022 INFO:tasks.ceph.osd.0:Stopped 2024-09-10T16:13:16.023 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-09-10T16:13:16.023 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T16:13:16.105 INFO:tasks.ceph.osd.1:Stopped 2024-09-10T16:13:16.105 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-09-10T16:13:16.105 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T16:13:19.782 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~0s 2024-09-10T16:13:19.782 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 has been restored 2024-09-10T16:13:22.207 INFO:tasks.ceph.osd.2:Stopped 2024-09-10T16:13:22.207 INFO:teuthology.misc:Shutting down mgr daemons... 2024-09-10T16:13:22.208 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-09-10T16:13:22.208 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T16:13:22.240 INFO:tasks.ceph.mgr.x:Stopped 2024-09-10T16:13:22.240 INFO:teuthology.misc:Shutting down mon daemons... 2024-09-10T16:13:22.241 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-09-10T16:13:22.241 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T16:13:22.295 INFO:tasks.ceph.mon.a:Stopped 2024-09-10T16:13:22.295 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-09-10T16:13:22.295 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T16:13:22.347 INFO:tasks.ceph.mon.b:Stopped 2024-09-10T16:13:22.348 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-09-10T16:13:22.348 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T16:13:22.400 INFO:tasks.ceph.mon.c:Stopped 2024-09-10T16:13:22.400 INFO:tasks.ceph:Checking cluster log for badness... 2024-09-10T16:13:22.401 DEBUG:teuthology.orchestra.run.smithi115:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v 'objects unfound and apparently lost' | egrep -v 'overall HEALTH_' | egrep -v '\(OSDMAP_FLAGS\)' | egrep -v '\(OSD_' | egrep -v '\(PG_' | egrep -v '\(OBJECT_' | egrep -v '\(SLOW_OPS\)' | egrep -v 'slow request' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | head -n 1 2024-09-10T16:13:22.541 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi115.front.sepia.ceph.com 2024-09-10T16:13:22.541 DEBUG:teuthology.orchestra.run.smithi115:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-09-10T16:13:22.890 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi115.front.sepia.ceph.com 2024-09-10T16:13:22.890 DEBUG:teuthology.orchestra.run.smithi115:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-09-10T16:13:23.063 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi115.front.sepia.ceph.com 2024-09-10T16:13:23.063 DEBUG:teuthology.orchestra.run.smithi115:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-09-10T16:13:23.124 INFO:tasks.ceph:Archiving mon data... 2024-09-10T16:13:23.125 DEBUG:teuthology.misc:Transferring archived files from smithi115:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/skanta-2024-09-10_15:30:27-rados-wip-bharath7-testing-2024-09-10-1409-distro-default-smithi/7899267/data/mon.a.tgz 2024-09-10T16:13:23.125 DEBUG:teuthology.orchestra.run.smithi115:> mktemp 2024-09-10T16:13:23.156 INFO:teuthology.orchestra.run.smithi115.stdout:/tmp/tmp.SeijpOSJkr 2024-09-10T16:13:23.157 DEBUG:teuthology.orchestra.run.smithi115:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.SeijpOSJkr 2024-09-10T16:13:23.286 DEBUG:teuthology.orchestra.run.smithi115:> sudo chmod 0666 /tmp/tmp.SeijpOSJkr 2024-09-10T16:13:23.407 DEBUG:teuthology.orchestra.remote:smithi115:/tmp/tmp.SeijpOSJkr is 297KB 2024-09-10T16:13:23.503 DEBUG:teuthology.orchestra.run.smithi115:> rm -fr /tmp/tmp.SeijpOSJkr 2024-09-10T16:13:23.518 DEBUG:teuthology.misc:Transferring archived files from smithi115:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/skanta-2024-09-10_15:30:27-rados-wip-bharath7-testing-2024-09-10-1409-distro-default-smithi/7899267/data/mon.b.tgz 2024-09-10T16:13:23.518 DEBUG:teuthology.orchestra.run.smithi115:> mktemp 2024-09-10T16:13:23.573 INFO:teuthology.orchestra.run.smithi115.stdout:/tmp/tmp.nPcSGtqlTK 2024-09-10T16:13:23.573 DEBUG:teuthology.orchestra.run.smithi115:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.nPcSGtqlTK 2024-09-10T16:13:23.689 DEBUG:teuthology.orchestra.run.smithi115:> sudo chmod 0666 /tmp/tmp.nPcSGtqlTK 2024-09-10T16:13:23.773 DEBUG:teuthology.orchestra.remote:smithi115:/tmp/tmp.nPcSGtqlTK is 300KB 2024-09-10T16:13:23.868 DEBUG:teuthology.orchestra.run.smithi115:> rm -fr /tmp/tmp.nPcSGtqlTK 2024-09-10T16:13:23.883 DEBUG:teuthology.misc:Transferring archived files from smithi115:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/skanta-2024-09-10_15:30:27-rados-wip-bharath7-testing-2024-09-10-1409-distro-default-smithi/7899267/data/mon.c.tgz 2024-09-10T16:13:23.883 DEBUG:teuthology.orchestra.run.smithi115:> mktemp 2024-09-10T16:13:23.937 INFO:teuthology.orchestra.run.smithi115.stdout:/tmp/tmp.B4EiBqgFrm 2024-09-10T16:13:23.940 DEBUG:teuthology.orchestra.run.smithi115:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.B4EiBqgFrm 2024-09-10T16:13:24.053 DEBUG:teuthology.orchestra.run.smithi115:> sudo chmod 0666 /tmp/tmp.B4EiBqgFrm 2024-09-10T16:13:24.136 DEBUG:teuthology.orchestra.remote:smithi115:/tmp/tmp.B4EiBqgFrm is 300KB 2024-09-10T16:13:24.202 DEBUG:teuthology.orchestra.run.smithi115:> rm -fr /tmp/tmp.B4EiBqgFrm 2024-09-10T16:13:24.217 INFO:tasks.ceph:Cleaning ceph cluster... 2024-09-10T16:13:24.217 DEBUG:teuthology.orchestra.run.smithi115:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-10T16:13:24.342 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-09-10T16:13:24.342 INFO:tasks.ceph:Archiving crash dumps... 2024-09-10T16:13:24.344 DEBUG:teuthology.misc:Transferring archived files from smithi115:/var/lib/ceph/crash to /home/teuthworker/archive/skanta-2024-09-10_15:30:27-rados-wip-bharath7-testing-2024-09-10-1409-distro-default-smithi/7899267/remote/smithi115/crash 2024-09-10T16:13:24.345 DEBUG:teuthology.orchestra.run.smithi115:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-10T16:13:24.406 INFO:tasks.ceph:Compressing logs... 2024-09-10T16:13:24.407 DEBUG:teuthology.orchestra.run.smithi115:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-10T16:13:24.480 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-09-10T16:13:24.757 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70768.log 2024-09-10T16:13:24.757 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38038.log 2024-09-10T16:13:24.757 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.70768.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58142.log 2024-09-10T16:13:24.757 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.70768.log.gz 2024-09-10T16:13:24.757 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.38038.log: gzip 0.0% -5 -- replaced with /var/log/ceph/ceph-client.admin.38038.log.gz --verbose 2024-09-10T16:13:24.757 INFO:teuthology.orchestra.run.smithi115.stderr: -- /var/log/ceph/ceph-client.admin.43630.log 2024-09-10T16:13:24.757 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.58142.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49516.log 2024-09-10T16:13:24.757 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58142.log.gz 2024-09-10T16:13:24.757 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.43630.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66299.log 2024-09-10T16:13:24.757 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43630.log.gz 2024-09-10T16:13:24.757 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.49516.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59617.log 2024-09-10T16:13:24.758 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49516.log.gz 2024-09-10T16:13:24.758 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.66299.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73750.log 2024-09-10T16:13:24.758 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66299.log.gz 2024-09-10T16:13:24.758 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.59617.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52429.log 2024-09-10T16:13:24.758 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59617.log.gz 2024-09-10T16:13:24.758 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.73750.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59869.log 2024-09-10T16:13:24.758 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.73750.log.gz 2024-09-10T16:13:24.758 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.52429.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66005.log 2024-09-10T16:13:24.758 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52429.log.gz 2024-09-10T16:13:24.758 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.59869.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70600.log 2024-09-10T16:13:24.758 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59869.log.gz 2024-09-10T16:13:24.758 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.66005.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39079.log 2024-09-10T16:13:24.758 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66005.log.gz 2024-09-10T16:13:24.759 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.70600.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45058.log 2024-09-10T16:13:24.759 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.70600.log.gz 2024-09-10T16:13:24.759 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.39079.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69424.log 2024-09-10T16:13:24.759 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39079.log.gz 2024-09-10T16:13:24.759 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.45058.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59701.log 2024-09-10T16:13:24.759 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45058.log.gz 2024-09-10T16:13:24.759 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.69424.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48760.log 2024-09-10T16:13:24.759 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69424.log.gz 2024-09-10T16:13:24.759 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.59701.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47415.log 2024-09-10T16:13:24.759 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59701.log.gz 2024-09-10T16:13:24.759 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.48760.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58016.log 2024-09-10T16:13:24.759 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48760.log.gz 2024-09-10T16:13:24.759 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.47415.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55454.log 2024-09-10T16:13:24.759 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47415.log.gz 2024-09-10T16:13:24.760 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.58016.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47625.log 2024-09-10T16:13:24.760 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58016.log.gz 2024-09-10T16:13:24.760 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.55454.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58689.log 2024-09-10T16:13:24.760 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55454.log.gz 2024-09-10T16:13:24.760 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.47625.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65963.log 2024-09-10T16:13:24.760 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47625.log.gz 2024-09-10T16:13:24.760 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.58689.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43107.log 2024-09-10T16:13:24.760 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58689.log.gz 2024-09-10T16:13:24.760 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.65963.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39810.log 2024-09-10T16:13:24.760 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65963.log.gz 2024-09-10T16:13:24.760 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.43107.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43672.log 2024-09-10T16:13:24.760 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43107.log.gz 2024-09-10T16:13:24.760 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.39810.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70726.log 2024-09-10T16:13:24.760 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39810.log.gz 2024-09-10T16:13:24.761 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.43672.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49348.log 2024-09-10T16:13:24.761 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43672.log.gz 2024-09-10T16:13:24.761 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.70726.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72658.log 2024-09-10T16:13:24.761 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.70726.log.gz 2024-09-10T16:13:24.761 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.49348.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49978.log 2024-09-10T16:13:24.761 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49348.log.gz 2024-09-10T16:13:24.761 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48255.log 2024-09-10T16:13:24.761 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.72658.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.72658.log.gz 2024-09-10T16:13:24.761 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.49978.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55370.log 2024-09-10T16:13:24.761 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49978.log.gz 2024-09-10T16:13:24.761 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.48255.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56630.log 2024-09-10T16:13:24.761 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48255.log.gz 2024-09-10T16:13:24.762 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73666.log 2024-09-10T16:13:24.762 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.55370.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55370.log.gz 2024-09-10T16:13:24.762 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.56630.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63442.log 2024-09-10T16:13:24.762 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56630.log.gz 2024-09-10T16:13:24.762 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.73666.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74294.log 2024-09-10T16:13:24.762 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.73666.log.gz 2024-09-10T16:13:24.762 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.63442.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38246.log 2024-09-10T16:13:24.762 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63442.log.gz 2024-09-10T16:13:24.762 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.74294.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66677.log 2024-09-10T16:13:24.762 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.74294.log.gz 2024-09-10T16:13:24.762 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.38246.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57638.log 2024-09-10T16:13:24.762 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38246.log.gz 2024-09-10T16:13:24.762 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.66677.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49306.log 2024-09-10T16:13:24.762 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66677.log.gz 2024-09-10T16:13:24.763 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.57638.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61465.log 2024-09-10T16:13:24.763 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57638.log.gz 2024-09-10T16:13:24.763 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64702.log 2024-09-10T16:13:24.763 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.49306.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49306.log.gz 2024-09-10T16:13:24.763 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.61465.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49474.log 2024-09-10T16:13:24.763 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61465.log.gz 2024-09-10T16:13:24.763 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.64702.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53942.log 2024-09-10T16:13:24.763 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64702.log.gz 2024-09-10T16:13:24.763 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.49474.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60373.log 2024-09-10T16:13:24.763 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49474.log.gz 2024-09-10T16:13:24.763 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.53942.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47373.log 2024-09-10T16:13:24.763 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53942.log.gz 2024-09-10T16:13:24.763 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68109.log 2024-09-10T16:13:24.764 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.60373.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60373.log.gz 2024-09-10T16:13:24.764 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.47373.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49096.log 2024-09-10T16:13:24.764 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47373.log.gz 2024-09-10T16:13:24.764 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.68109.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69045.log 2024-09-10T16:13:24.764 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68109.log.gz 2024-09-10T16:13:24.764 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.49096.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68403.log 2024-09-10T16:13:24.764 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49096.log.gz 2024-09-10T16:13:24.764 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.69045.log: gzip 0.0% -5 -- replaced with /var/log/ceph/ceph-client.admin.69045.log.gz --verbose 2024-09-10T16:13:24.764 INFO:teuthology.orchestra.run.smithi115.stderr: -- /var/log/ceph/ceph-client.admin.69382.log 2024-09-10T16:13:24.764 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.68403.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48087.log 2024-09-10T16:13:24.764 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68403.log.gz 2024-09-10T16:13:24.764 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.69382.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36777.log 2024-09-10T16:13:24.765 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69382.log.gz 2024-09-10T16:13:24.765 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.48087.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65080.log 2024-09-10T16:13:24.765 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48087.log.gz 2024-09-10T16:13:24.765 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.36777.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63610.log 2024-09-10T16:13:24.765 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36777.log.gz 2024-09-10T16:13:24.765 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.65080.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47541.log 2024-09-10T16:13:24.765 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65080.log.gz 2024-09-10T16:13:24.765 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.63610.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38135.log 2024-09-10T16:13:24.765 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63610.log.gz 2024-09-10T16:13:24.765 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.47541.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38431.log 2024-09-10T16:13:24.765 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47541.log.gz 2024-09-10T16:13:24.765 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.38135.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55538.log 2024-09-10T16:13:24.766 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38135.log.gz 2024-09-10T16:13:24.766 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.38431.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69928.log 2024-09-10T16:13:24.766 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38431.log.gz 2024-09-10T16:13:24.766 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.55538.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55538.log.gz 2024-09-10T16:13:24.766 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44344.log 2024-09-10T16:13:24.766 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.69928.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37763.log 2024-09-10T16:13:24.766 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69928.log.gz 2024-09-10T16:13:24.766 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.44344.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63862.log 2024-09-10T16:13:24.766 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44344.log.gz 2024-09-10T16:13:24.766 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.37763.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63526.log 2024-09-10T16:13:24.766 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37763.log.gz 2024-09-10T16:13:24.766 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.63862.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61213.log 2024-09-10T16:13:24.766 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63862.log.gz 2024-09-10T16:13:24.766 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.63526.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66845.log 2024-09-10T16:13:24.767 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63526.log.gz 2024-09-10T16:13:24.767 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.61213.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67436.log 2024-09-10T16:13:24.767 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61213.log.gz 2024-09-10T16:13:24.767 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.66845.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62979.log 2024-09-10T16:13:24.767 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66845.log.gz 2024-09-10T16:13:24.767 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.67436.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60877.log 2024-09-10T16:13:24.767 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67436.log.gz 2024-09-10T16:13:24.767 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.62979.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71524.log 2024-09-10T16:13:24.767 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62979.log.gz 2024-09-10T16:13:24.767 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.60877.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53312.log 2024-09-10T16:13:24.767 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60877.log.gz 2024-09-10T16:13:24.767 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.71524.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73876.log 2024-09-10T16:13:24.767 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.71524.log.gz 2024-09-10T16:13:24.767 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.53312.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38949.log 2024-09-10T16:13:24.768 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53312.log.gz 2024-09-10T16:13:24.768 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.73876.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44742.log 2024-09-10T16:13:24.768 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.73876.log.gz 2024-09-10T16:13:24.768 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.38949.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52345.log 2024-09-10T16:13:24.768 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38949.log.gz 2024-09-10T16:13:24.768 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.44742.log: gzip 0.0% -5 -- replaced with /var/log/ceph/ceph-client.admin.44742.log.gz --verbose 2024-09-10T16:13:24.768 INFO:teuthology.orchestra.run.smithi115.stderr: -- /var/log/ceph/ceph-client.admin.42626.log 2024-09-10T16:13:24.768 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.52345.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48129.log 2024-09-10T16:13:24.768 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52345.log.gz 2024-09-10T16:13:24.768 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.42626.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38172.log 2024-09-10T16:13:24.768 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42626.log.gz 2024-09-10T16:13:24.768 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49810.log 2024-09-10T16:13:24.768 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.48129.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48129.log.gz 2024-09-10T16:13:24.769 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.38172.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65753.log 2024-09-10T16:13:24.769 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38172.log.gz 2024-09-10T16:13:24.769 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56882.log 2024-09-10T16:13:24.769 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.49810.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49810.log.gz 2024-09-10T16:13:24.769 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.65753.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42584.log 2024-09-10T16:13:24.769 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65753.log.gz 2024-09-10T16:13:24.769 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.56882.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71566.log 2024-09-10T16:13:24.769 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56882.log.gz 2024-09-10T16:13:24.769 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.42584.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62264.log 2024-09-10T16:13:24.769 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42584.log.gz 2024-09-10T16:13:24.769 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.71566.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.71566.log.gz 2024-09-10T16:13:24.769 INFO:teuthology.orchestra.run.smithi115.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.47919.log 2024-09-10T16:13:24.769 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.62264.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41162.log 2024-09-10T16:13:24.770 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62264.log.gz 2024-09-10T16:13:24.770 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.47919.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60415.log 2024-09-10T16:13:24.770 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47919.log.gz 2024-09-10T16:13:24.770 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.41162.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64786.log 2024-09-10T16:13:24.770 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41162.log.gz 2024-09-10T16:13:24.770 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.60415.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42771.log 2024-09-10T16:13:24.770 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60415.log.gz 2024-09-10T16:13:24.770 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.64786.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54572.log 2024-09-10T16:13:24.770 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64786.log.gz 2024-09-10T16:13:24.770 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.42771.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51421.log 2024-09-10T16:13:24.770 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42771.log.gz 2024-09-10T16:13:24.770 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.54572.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44134.log 2024-09-10T16:13:24.771 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54572.log.gz 2024-09-10T16:13:24.771 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.51421.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71398.log 2024-09-10T16:13:24.771 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51421.log.gz 2024-09-10T16:13:24.771 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.44134.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43798.log 2024-09-10T16:13:24.771 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44134.log.gz 2024-09-10T16:13:24.771 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.71398.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58984.log 2024-09-10T16:13:24.771 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.71398.log.gz 2024-09-10T16:13:24.771 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.43798.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73078.log 2024-09-10T16:13:24.771 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43798.log.gz 2024-09-10T16:13:24.771 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.58984.log: gzip 0.0% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.58984.log.gz -- 2024-09-10T16:13:24.771 INFO:teuthology.orchestra.run.smithi115.stderr: /var/log/ceph/ceph-client.admin.68877.log 2024-09-10T16:13:24.771 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.73078.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43504.log 2024-09-10T16:13:24.771 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.73078.log.gz 2024-09-10T16:13:24.772 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.68877.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67478.log 2024-09-10T16:13:24.772 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68877.log.gz 2024-09-10T16:13:24.772 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.43504.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37848.log 2024-09-10T16:13:24.772 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43504.log.gz 2024-09-10T16:13:24.772 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.67478.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48381.log 2024-09-10T16:13:24.772 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67478.log.gz 2024-09-10T16:13:24.772 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.37848.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71902.log 2024-09-10T16:13:24.772 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37848.log.gz 2024-09-10T16:13:24.772 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.48381.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37200.log 2024-09-10T16:13:24.772 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48381.log.gz 2024-09-10T16:13:24.772 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.71902.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58646.log 2024-09-10T16:13:24.772 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.71902.log.gz 2024-09-10T16:13:24.772 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.37200.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55958.log 2024-09-10T16:13:24.773 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37200.log.gz 2024-09-10T16:13:24.773 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.58646.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41478.log 2024-09-10T16:13:24.773 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58646.log.gz 2024-09-10T16:13:24.773 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.55958.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67391.log 2024-09-10T16:13:24.773 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55958.log.gz 2024-09-10T16:13:24.773 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.41478.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48634.log 2024-09-10T16:13:24.773 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41478.log.gz 2024-09-10T16:13:24.773 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.67391.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53144.log 2024-09-10T16:13:24.773 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67391.log.gz 2024-09-10T16:13:24.773 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.48634.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48634.log.gz 2024-09-10T16:13:24.773 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45836.log 2024-09-10T16:13:24.773 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.53144.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71020.log 2024-09-10T16:13:24.774 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53144.log.gz 2024-09-10T16:13:24.774 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.45836.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-09-10T16:13:24.774 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45836.log.gz 2024-09-10T16:13:24.774 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.71020.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68656.log 2024-09-10T16:13:24.774 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.71020.log.gz 2024-09-10T16:13:24.774 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56042.log 2024-09-10T16:13:24.774 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.68656.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63568.log 2024-09-10T16:13:24.774 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68656.log.gz 2024-09-10T16:13:24.774 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.56042.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37604.log 2024-09-10T16:13:24.774 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56042.log.gz 2024-09-10T16:13:24.774 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.63568.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50959.log 2024-09-10T16:13:24.774 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63568.log.gz 2024-09-10T16:13:24.774 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.37604.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69130.log 2024-09-10T16:13:24.775 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37604.log.gz 2024-09-10T16:13:24.775 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.50959.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63106.log 2024-09-10T16:13:24.775 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50959.log.gz 2024-09-10T16:13:24.775 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.69130.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48592.log 2024-09-10T16:13:24.775 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69130.log.gz 2024-09-10T16:13:24.775 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.63106.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63106.log.gz 2024-09-10T16:13:24.775 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45466.log 2024-09-10T16:13:24.775 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.48592.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58478.log 2024-09-10T16:13:24.775 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48592.log.gz 2024-09-10T16:13:24.775 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.45466.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44008.log 2024-09-10T16:13:24.775 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45466.log.gz 2024-09-10T16:13:24.775 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.58478.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44680.log 2024-09-10T16:13:24.775 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58478.log.gz 2024-09-10T16:13:24.775 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.44008.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69760.log 2024-09-10T16:13:24.776 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44008.log.gz 2024-09-10T16:13:24.776 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.44680.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69550.log 2024-09-10T16:13:24.776 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44680.log.gz 2024-09-10T16:13:24.776 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.69760.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45725.log 2024-09-10T16:13:24.776 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69760.log.gz 2024-09-10T16:13:24.776 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.69550.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51253.log 2024-09-10T16:13:24.776 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69550.log.gz 2024-09-10T16:13:24.776 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.45725.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72112.log 2024-09-10T16:13:24.776 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45725.log.gz 2024-09-10T16:13:24.776 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.51253.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51253.log.gz 2024-09-10T16:13:24.776 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60079.log 2024-09-10T16:13:24.776 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.72112.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56966.log 2024-09-10T16:13:24.776 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.72112.log.gz 2024-09-10T16:13:24.777 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.60079.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72532.log 2024-09-10T16:13:24.777 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60079.log.gz 2024-09-10T16:13:24.777 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.56966.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67773.log 2024-09-10T16:13:24.777 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56966.log.gz 2024-09-10T16:13:24.777 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.72532.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54236.log 2024-09-10T16:13:24.777 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.72532.log.gz 2024-09-10T16:13:24.777 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.67773.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72490.log 2024-09-10T16:13:24.777 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67773.log.gz 2024-09-10T16:13:24.777 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.54236.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44596.log 2024-09-10T16:13:24.777 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54236.log.gz 2024-09-10T16:13:24.777 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.72490.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59491.log 2024-09-10T16:13:24.777 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.72490.log.gz 2024-09-10T16:13:24.777 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.44596.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.44596.log.gz 2024-09-10T16:13:24.778 INFO:teuthology.orchestra.run.smithi115.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.52849.log 2024-09-10T16:13:24.778 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.59491.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45100.log 2024-09-10T16:13:24.778 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59491.log.gz 2024-09-10T16:13:24.778 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.52849.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59362.log 2024-09-10T16:13:24.778 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52849.log.gz 2024-09-10T16:13:24.778 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.45100.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67899.log 2024-09-10T16:13:24.778 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45100.log.gz 2024-09-10T16:13:24.778 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.59362.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59362.log.gz 2024-09-10T16:13:24.778 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71860.log 2024-09-10T16:13:24.778 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.67899.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64534.log 2024-09-10T16:13:24.778 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67899.log.gz 2024-09-10T16:13:24.778 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.71860.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54194.log 2024-09-10T16:13:24.778 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.71860.log.gz 2024-09-10T16:13:24.779 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.64534.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65206.log 2024-09-10T16:13:24.779 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64534.log.gz 2024-09-10T16:13:24.779 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.54194.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72994.log 2024-09-10T16:13:24.779 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54194.log.gz 2024-09-10T16:13:24.779 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.65206.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43276.log 2024-09-10T16:13:24.779 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65206.log.gz 2024-09-10T16:13:24.779 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.72994.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70054.log 2024-09-10T16:13:24.779 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.72994.log.gz 2024-09-10T16:13:24.779 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.43276.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61507.log 2024-09-10T16:13:24.779 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43276.log.gz 2024-09-10T16:13:24.779 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.70054.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65795.log 2024-09-10T16:13:24.779 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.70054.log.gz 2024-09-10T16:13:24.779 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.61507.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47667.log 2024-09-10T16:13:24.780 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61507.log.gz 2024-09-10T16:13:24.780 INFO:teuthology.orchestra.run.smithi115.stderr:gzip/var/log/ceph/ceph-client.admin.65795.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.37954.log 2024-09-10T16:13:24.780 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65795.log.gz 2024-09-10T16:13:24.780 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.47667.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51169.log 2024-09-10T16:13:24.780 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47667.log.gz 2024-09-10T16:13:24.780 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.37954.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56210.log 2024-09-10T16:13:24.780 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37954.log.gz 2024-09-10T16:13:24.780 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.51169.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49936.log 2024-09-10T16:13:24.780 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51169.log.gz 2024-09-10T16:13:24.780 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.56210.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51379.log 2024-09-10T16:13:24.780 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56210.log.gz 2024-09-10T16:13:24.780 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.49936.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37368.log 2024-09-10T16:13:24.781 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49936.log.gz 2024-09-10T16:13:24.781 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.51379.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53606.log 2024-09-10T16:13:24.781 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51379.log.gz 2024-09-10T16:13:24.781 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.37368.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65458.log 2024-09-10T16:13:24.781 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37368.log.gz 2024-09-10T16:13:24.781 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.53606.log: gzip -5 --verbose 0.0% -- -- replaced with /var/log/ceph/ceph-client.admin.53606.log.gz 2024-09-10T16:13:24.781 INFO:teuthology.orchestra.run.smithi115.stderr: /var/log/ceph/ceph-client.admin.73330.log 2024-09-10T16:13:24.781 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.65458.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50539.log 2024-09-10T16:13:24.781 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65458.log.gz 2024-09-10T16:13:24.781 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.73330.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73036.log 2024-09-10T16:13:24.781 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.73330.log.gz 2024-09-10T16:13:24.781 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.50539.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65416.log 2024-09-10T16:13:24.782 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50539.log.gz 2024-09-10T16:13:24.782 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.73036.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.73036.log.gz 2024-09-10T16:13:24.782 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61591.log 2024-09-10T16:13:24.782 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.65416.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66887.log 2024-09-10T16:13:24.782 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65416.log.gz 2024-09-10T16:13:24.782 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.61591.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56588.log 2024-09-10T16:13:24.782 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61591.log.gz 2024-09-10T16:13:24.782 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.66887.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49222.log 2024-09-10T16:13:24.782 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66887.log.gz 2024-09-10T16:13:24.782 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.56588.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53102.log 2024-09-10T16:13:24.782 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56588.log.gz 2024-09-10T16:13:24.782 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.49222.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59407.log 2024-09-10T16:13:24.782 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49222.log.gz 2024-09-10T16:13:24.783 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.53102.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68067.log 2024-09-10T16:13:24.783 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53102.log.gz 2024-09-10T16:13:24.783 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.59407.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38357.log 2024-09-10T16:13:24.783 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59407.log.gz 2024-09-10T16:13:24.783 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.68067.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58900.log 2024-09-10T16:13:24.783 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68067.log.gz 2024-09-10T16:13:24.783 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.38357.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64870.log 2024-09-10T16:13:24.783 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38357.log.gz 2024-09-10T16:13:24.783 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.58900.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32154.log 2024-09-10T16:13:24.783 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58900.log.gz 2024-09-10T16:13:24.783 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.64870.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56336.log 2024-09-10T16:13:24.783 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64870.log.gz 2024-09-10T16:13:24.783 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.32154.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32154.log.gz 2024-09-10T16:13:24.784 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48003.log 2024-09-10T16:13:24.784 INFO:teuthology.orchestra.run.smithi115.stderr:gzip/var/log/ceph/ceph-client.admin.56336.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.32194.log 2024-09-10T16:13:24.784 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56336.log.gz 2024-09-10T16:13:24.784 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.48003.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36776.log 2024-09-10T16:13:24.784 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48003.log.gz 2024-09-10T16:13:24.784 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.32194.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57428.log 2024-09-10T16:13:24.784 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32194.log.gz 2024-09-10T16:13:24.784 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72280.log 2024-09-10T16:13:24.784 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.36776.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36776.log.gz 2024-09-10T16:13:24.784 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.57428.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67265.log 2024-09-10T16:13:24.784 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57428.log.gz 2024-09-10T16:13:24.784 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.72280.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43192.log 2024-09-10T16:13:24.784 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.72280.log.gz 2024-09-10T16:13:24.785 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69214.log 2024-09-10T16:13:24.785 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.67265.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67265.log.gz 2024-09-10T16:13:24.785 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.43192.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36519.log 2024-09-10T16:13:24.785 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43192.log.gz 2024-09-10T16:13:24.785 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.69214.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72826.log 2024-09-10T16:13:24.785 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69214.log.gz 2024-09-10T16:13:24.785 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52093.log 2024-09-10T16:13:24.785 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.36519.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36519.log.gz 2024-09-10T16:13:24.785 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.72826.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61129.log 2024-09-10T16:13:24.785 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.72826.log.gz 2024-09-10T16:13:24.785 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.52093.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59026.log 2024-09-10T16:13:24.785 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52093.log.gz 2024-09-10T16:13:24.785 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52009.log 2024-09-10T16:13:24.786 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.61129.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61129.log.gz 2024-09-10T16:13:24.786 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.59026.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46021.log 2024-09-10T16:13:24.786 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59026.log.gz 2024-09-10T16:13:24.786 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.52009.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49684.log 2024-09-10T16:13:24.786 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52009.log.gz 2024-09-10T16:13:24.786 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57848.log 2024-09-10T16:13:24.786 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.46021.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46021.log.gz 2024-09-10T16:13:24.786 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.49684.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64618.log 2024-09-10T16:13:24.786 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49684.log.gz 2024-09-10T16:13:24.786 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.57848.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56420.log 2024-09-10T16:13:24.786 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57848.log.gz 2024-09-10T16:13:24.786 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.64618.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39514.log 2024-09-10T16:13:24.787 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64618.log.gz 2024-09-10T16:13:24.787 INFO:teuthology.orchestra.run.smithi115.stderr:gzip/var/log/ceph/ceph-client.admin.56420.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.73456.log 2024-09-10T16:13:24.787 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56420.log.gz 2024-09-10T16:13:24.787 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.39514.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50398.log 2024-09-10T16:13:24.787 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39514.log.gz 2024-09-10T16:13:24.787 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.73456.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64324.log 2024-09-10T16:13:24.787 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.73456.log.gz 2024-09-10T16:13:24.787 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.50398.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39121.log 2024-09-10T16:13:24.787 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50398.log.gz 2024-09-10T16:13:24.787 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.64324.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39218.log 2024-09-10T16:13:24.787 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64324.log.gz 2024-09-10T16:13:24.787 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.39121.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71272.log 2024-09-10T16:13:24.787 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39121.log.gz 2024-09-10T16:13:24.788 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.39218.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71314.log 2024-09-10T16:13:24.788 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39218.log.gz 2024-09-10T16:13:24.788 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.71272.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.71272.log.gz 2024-09-10T16:13:24.788 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68740.log 2024-09-10T16:13:24.788 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.71314.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52681.log 2024-09-10T16:13:24.788 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.71314.log.gz 2024-09-10T16:13:24.788 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.68740.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66131.log 2024-09-10T16:13:24.788 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68740.log.gz 2024-09-10T16:13:24.788 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.52681.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45984.log 2024-09-10T16:13:24.788 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52681.log.gz 2024-09-10T16:13:24.788 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.66131.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66761.log 2024-09-10T16:13:24.788 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66131.log.gz 2024-09-10T16:13:24.789 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.45984.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59995.log 2024-09-10T16:13:24.789 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45984.log.gz 2024-09-10T16:13:24.789 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.66761.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65837.log 2024-09-10T16:13:24.789 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66761.log.gz 2024-09-10T16:13:24.789 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.59995.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38838.log 2024-09-10T16:13:24.789 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59995.log.gz 2024-09-10T16:13:24.789 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.65837.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58058.log 2024-09-10T16:13:24.789 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65837.log.gz 2024-09-10T16:13:24.789 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.38838.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50833.log 2024-09-10T16:13:24.789 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38838.log.gz 2024-09-10T16:13:24.789 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.58058.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58058.log.gz 2024-09-10T16:13:24.789 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55580.log 2024-09-10T16:13:24.789 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.50833.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55244.log 2024-09-10T16:13:24.789 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50833.log.gz 2024-09-10T16:13:24.790 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.55580.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62095.log 2024-09-10T16:13:24.790 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55580.log.gz 2024-09-10T16:13:24.790 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.55244.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45614.log 2024-09-10T16:13:24.790 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55244.log.gz 2024-09-10T16:13:24.790 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.62095.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55874.log 2024-09-10T16:13:24.790 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62095.log.gz 2024-09-10T16:13:24.790 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.45614.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53774.log 2024-09-10T16:13:24.790 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45614.log.gz 2024-09-10T16:13:24.790 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39884.log 2024-09-10T16:13:24.790 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.55874.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55874.log.gz 2024-09-10T16:13:24.790 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.53774.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43004.log 2024-09-10T16:13:24.790 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53774.log.gz 2024-09-10T16:13:24.790 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.39884.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57764.log 2024-09-10T16:13:24.791 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39884.log.gz 2024-09-10T16:13:24.791 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50917.log 2024-09-10T16:13:24.791 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.43004.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43004.log.gz 2024-09-10T16:13:24.791 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.57764.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58352.log 2024-09-10T16:13:24.791 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57764.log.gz 2024-09-10T16:13:24.791 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.50917.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36864.log 2024-09-10T16:13:24.791 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50917.log.gz 2024-09-10T16:13:24.791 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70180.log 2024-09-10T16:13:24.791 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.58352.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58352.log.gz 2024-09-10T16:13:24.791 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.36864.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58562.log 2024-09-10T16:13:24.791 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36864.log.gz 2024-09-10T16:13:24.791 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.70180.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51925.log 2024-09-10T16:13:24.791 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.70180.log.gz 2024-09-10T16:13:24.791 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50623.log 2024-09-10T16:13:24.792 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.58562.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58562.log.gz 2024-09-10T16:13:24.792 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.51925.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38505.log 2024-09-10T16:13:24.792 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51925.log.gz 2024-09-10T16:13:24.792 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.50623.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65248.log 2024-09-10T16:13:24.792 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50623.log.gz 2024-09-10T16:13:24.792 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65627.log 2024-09-10T16:13:24.792 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.38505.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38505.log.gz 2024-09-10T16:13:24.792 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.65248.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53984.log 2024-09-10T16:13:24.792 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65248.log.gz 2024-09-10T16:13:24.792 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.65627.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41562.log 2024-09-10T16:13:24.792 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65627.log.gz 2024-09-10T16:13:24.792 INFO:teuthology.orchestra.run.smithi115.stderr:gzip/var/log/ceph/ceph-client.admin.53984.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.64954.log 2024-09-10T16:13:24.792 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53984.log.gz 2024-09-10T16:13:24.793 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.41562.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59785.log 2024-09-10T16:13:24.793 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41562.log.gz 2024-09-10T16:13:24.793 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.64954.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48928.log 2024-09-10T16:13:24.793 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64954.log.gz 2024-09-10T16:13:24.793 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.59785.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53732.log 2024-09-10T16:13:24.793 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59785.log.gz 2024-09-10T16:13:24.793 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.48928.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71482.log 2024-09-10T16:13:24.793 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48928.log.gz 2024-09-10T16:13:24.793 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.53732.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48465.log 2024-09-10T16:13:24.793 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53732.log.gz 2024-09-10T16:13:24.793 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.71482.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72070.log 2024-09-10T16:13:24.793 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.71482.log.gz 2024-09-10T16:13:24.794 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.48465.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48465.log.gz 2024-09-10T16:13:24.794 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41161.log 2024-09-10T16:13:24.794 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.72070.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61633.log 2024-09-10T16:13:24.794 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.72070.log.gz 2024-09-10T16:13:24.794 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.41161.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64660.log 2024-09-10T16:13:24.794 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41161.log.gz 2024-09-10T16:13:24.794 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.61633.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50497.log 2024-09-10T16:13:24.794 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61633.log.gz 2024-09-10T16:13:24.794 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66341.log 2024-09-10T16:13:24.794 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.64660.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64660.log.gz 2024-09-10T16:13:24.794 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.50497.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60793.log 2024-09-10T16:13:24.794 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50497.log.gz 2024-09-10T16:13:24.794 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.66341.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71692.log 2024-09-10T16:13:24.795 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66341.log.gz 2024-09-10T16:13:24.795 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.60793.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69003.log 2024-09-10T16:13:24.795 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60793.log.gz 2024-09-10T16:13:24.795 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.71692.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72028.log 2024-09-10T16:13:24.795 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.71692.log.gz 2024-09-10T16:13:24.795 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.69003.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39736.log 2024-09-10T16:13:24.795 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69003.log.gz 2024-09-10T16:13:24.795 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.72028.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67520.log 2024-09-10T16:13:24.795 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.72028.log.gz 2024-09-10T16:13:24.795 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.39736.log: gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.27381.log 2024-09-10T16:13:24.795 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39736.log.gz 2024-09-10T16:13:24.795 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.67520.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61969.log 2024-09-10T16:13:24.795 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67520.log.gz 2024-09-10T16:13:24.796 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph.tmp-client.admin.27381.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56924.log 2024-09-10T16:13:24.796 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.27381.log.gz 2024-09-10T16:13:24.796 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.61969.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64828.log 2024-09-10T16:13:24.796 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61969.log.gz 2024-09-10T16:13:24.796 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.56924.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51589.log 2024-09-10T16:13:24.796 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56924.log.gz 2024-09-10T16:13:24.796 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.64828.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58268.log 2024-09-10T16:13:24.796 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64828.log.gz 2024-09-10T16:13:24.796 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.51589.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59449.log 2024-09-10T16:13:24.796 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51589.log.gz 2024-09-10T16:13:24.796 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.58268.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66173.log 2024-09-10T16:13:24.796 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58268.log.gz 2024-09-10T16:13:24.796 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.59449.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74254.log 2024-09-10T16:13:24.797 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59449.log.gz 2024-09-10T16:13:24.797 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53858.log 2024-09-10T16:13:24.797 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.66173.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66173.log.gz 2024-09-10T16:13:24.797 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.74254.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50314.log 2024-09-10T16:13:24.797 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.74254.log.gz 2024-09-10T16:13:24.797 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.53858.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53438.log 2024-09-10T16:13:24.797 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53858.log.gz 2024-09-10T16:13:24.797 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70390.log 2024-09-10T16:13:24.797 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.50314.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50314.log.gz 2024-09-10T16:13:24.797 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.53438.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38653.log 2024-09-10T16:13:24.797 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53438.log.gz 2024-09-10T16:13:24.797 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.70390.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39403.log 2024-09-10T16:13:24.797 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.70390.log.gz 2024-09-10T16:13:24.798 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.38653.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56546.log 2024-09-10T16:13:24.798 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38653.log.gz 2024-09-10T16:13:24.798 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.39403.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41246.log 2024-09-10T16:13:24.798 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39403.log.gz 2024-09-10T16:13:24.798 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.56546.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65879.log 2024-09-10T16:13:24.798 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56546.log.gz 2024-09-10T16:13:24.798 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.41246.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60961.log 2024-09-10T16:13:24.798 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41246.log.gz 2024-09-10T16:13:24.798 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.65879.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64282.log 2024-09-10T16:13:24.798 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65879.log.gz 2024-09-10T16:13:24.798 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.60961.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55328.log 2024-09-10T16:13:24.798 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60961.log.gz 2024-09-10T16:13:24.798 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.64282.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65122.log 2024-09-10T16:13:24.799 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64282.log.gz 2024-09-10T16:13:24.799 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.55328.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65585.log 2024-09-10T16:13:24.799 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55328.log.gz 2024-09-10T16:13:24.799 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.65122.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53018.log 2024-09-10T16:13:24.799 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65122.log.gz 2024-09-10T16:13:24.799 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.65585.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60037.log 2024-09-10T16:13:24.799 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65585.log.gz 2024-09-10T16:13:24.799 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.53018.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46095.log 2024-09-10T16:13:24.799 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53018.log.gz 2024-09-10T16:13:24.799 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.60037.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44470.log 2024-09-10T16:13:24.799 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60037.log.gz 2024-09-10T16:13:24.799 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.46095.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71146.log 2024-09-10T16:13:24.799 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46095.log.gz 2024-09-10T16:13:24.800 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.44470.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63400.log 2024-09-10T16:13:24.800 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44470.log.gz 2024-09-10T16:13:24.800 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.71146.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-09-10T16:13:24.800 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.71146.log.gz 2024-09-10T16:13:24.800 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.63400.log: -- /var/log/ceph/ceph-client.admin.38764.log 2024-09-10T16:13:24.800 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63400.log.gz 2024-09-10T16:13:24.800 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37679.log 2024-09-10T16:13:24.800 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.38764.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52177.log 2024-09-10T16:13:24.800 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38764.log.gz 2024-09-10T16:13:24.800 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.37679.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56378.log 2024-09-10T16:13:24.800 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37679.log.gz 2024-09-10T16:13:24.800 INFO:teuthology.orchestra.run.smithi115.stderr:gzip/var/log/ceph/ceph-client.admin.52177.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.61843.log 2024-09-10T16:13:24.801 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52177.log.gz 2024-09-10T16:13:24.801 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.56378.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42855.log 2024-09-10T16:13:24.801 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56378.log.gz 2024-09-10T16:13:24.801 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.61843.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38912.log 2024-09-10T16:13:24.801 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61843.log.gz 2024-09-10T16:13:24.801 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.42855.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63820.log 2024-09-10T16:13:24.801 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42855.log.gz 2024-09-10T16:13:24.801 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.38912.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36438.log 2024-09-10T16:13:24.801 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38912.log.gz 2024-09-10T16:13:24.801 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.63820.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57176.log 2024-09-10T16:13:24.801 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63820.log.gz 2024-09-10T16:13:24.801 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.36438.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56672.log 2024-09-10T16:13:24.802 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36438.log.gz 2024-09-10T16:13:24.802 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.57176.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51715.log 2024-09-10T16:13:24.802 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57176.log.gz 2024-09-10T16:13:24.802 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.56672.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55160.log 2024-09-10T16:13:24.802 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56672.log.gz 2024-09-10T16:13:24.802 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.51715.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61045.log 2024-09-10T16:13:24.802 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51715.log.gz 2024-09-10T16:13:24.802 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.55160.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58774.log 2024-09-10T16:13:24.802 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55160.log.gz 2024-09-10T16:13:24.802 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.61045.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38394.log 2024-09-10T16:13:24.802 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61045.log.gz 2024-09-10T16:13:24.802 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.58774.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55832.log 2024-09-10T16:13:24.802 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58774.log.gz 2024-09-10T16:13:24.802 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.38394.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44260.log 2024-09-10T16:13:24.803 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38394.log.gz 2024-09-10T16:13:24.803 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.55832.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49894.log 2024-09-10T16:13:24.803 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55832.log.gz 2024-09-10T16:13:24.803 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.44260.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44910.log 2024-09-10T16:13:24.803 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44260.log.gz 2024-09-10T16:13:24.803 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.49894.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51127.log 2024-09-10T16:13:24.803 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49894.log.gz 2024-09-10T16:13:24.803 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.44910.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40069.log 2024-09-10T16:13:24.803 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44910.log.gz 2024-09-10T16:13:24.803 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.51127.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54656.log 2024-09-10T16:13:24.803 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51127.log.gz 2024-09-10T16:13:24.803 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.40069.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47961.log 2024-09-10T16:13:24.803 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40069.log.gz 2024-09-10T16:13:24.804 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.54656.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48718.log 2024-09-10T16:13:24.804 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54656.log.gz 2024-09-10T16:13:24.804 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.47961.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37050.log 2024-09-10T16:13:24.804 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47961.log.gz 2024-09-10T16:13:24.804 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.48718.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62138.log 2024-09-10T16:13:24.804 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48718.log.gz 2024-09-10T16:13:24.804 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.37050.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68487.log 2024-09-10T16:13:24.804 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37050.log.gz 2024-09-10T16:13:24.804 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71188.log 2024-09-10T16:13:24.804 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.62138.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62138.log.gz 2024-09-10T16:13:24.804 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64996.log 2024-09-10T16:13:24.804 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.68487.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68487.log.gz 2024-09-10T16:13:24.804 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.71188.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36654.log 2024-09-10T16:13:24.805 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.71188.log.gz 2024-09-10T16:13:24.805 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.64996.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37497.log 2024-09-10T16:13:24.805 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64996.log.gz 2024-09-10T16:13:24.805 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.36654.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41329.log 2024-09-10T16:13:24.805 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36654.log.gz 2024-09-10T16:13:24.805 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.37497.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54614.log 2024-09-10T16:13:24.805 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37497.log.gz 2024-09-10T16:13:24.805 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.41329.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67307.log 2024-09-10T16:13:24.805 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41329.log.gz 2024-09-10T16:13:24.805 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.54614.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65332.log 2024-09-10T16:13:24.805 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54614.log.gz 2024-09-10T16:13:24.805 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.67307.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48423.log 2024-09-10T16:13:24.805 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67307.log.gz 2024-09-10T16:13:24.805 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.65332.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43966.log 2024-09-10T16:13:24.806 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65332.log.gz 2024-09-10T16:13:24.806 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66719.log 2024-09-10T16:13:24.806 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.48423.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48423.log.gz 2024-09-10T16:13:24.806 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.43966.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45762.log 2024-09-10T16:13:24.806 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43966.log.gz 2024-09-10T16:13:24.806 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.66719.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71062.log 2024-09-10T16:13:24.806 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66719.log.gz 2024-09-10T16:13:24.806 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.45762.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52135.log 2024-09-10T16:13:24.806 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45762.log.gz 2024-09-10T16:13:24.806 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.71062.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62011.log 2024-09-10T16:13:24.806 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.71062.log.gz 2024-09-10T16:13:24.806 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.52135.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38690.log 2024-09-10T16:13:24.806 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52135.log.gz 2024-09-10T16:13:24.807 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.62011.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68613.log 2024-09-10T16:13:24.807 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62011.log.gz 2024-09-10T16:13:24.807 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37452.log 2024-09-10T16:13:24.807 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.38690.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38690.log.gz 2024-09-10T16:13:24.807 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.68613.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41414.log 2024-09-10T16:13:24.807 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68613.log.gz 2024-09-10T16:13:24.807 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.37452.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39440.log 2024-09-10T16:13:24.807 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37452.log.gz 2024-09-10T16:13:24.807 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36778.log 2024-09-10T16:13:24.807 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.41414.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41414.log.gz 2024-09-10T16:13:24.807 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.39440.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52976.log 2024-09-10T16:13:24.807 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39440.log.gz 2024-09-10T16:13:24.807 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.36778.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57596.log 2024-09-10T16:13:24.808 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36778.log.gz 2024-09-10T16:13:24.808 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38616.log 2024-09-10T16:13:24.808 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.52976.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52976.log.gz 2024-09-10T16:13:24.808 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.57596.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51967.log 2024-09-10T16:13:24.808 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57596.log.gz 2024-09-10T16:13:24.808 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.38616.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61927.log 2024-09-10T16:13:24.808 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38616.log.gz 2024-09-10T16:13:24.808 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.51967.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64072.log 2024-09-10T16:13:24.808 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51967.log.gz 2024-09-10T16:13:24.808 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.61927.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72238.log 2024-09-10T16:13:24.808 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61927.log.gz 2024-09-10T16:13:24.808 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.64072.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32355.log 2024-09-10T16:13:24.808 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64072.log.gz 2024-09-10T16:13:24.809 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.72238.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67223.log 2024-09-10T16:13:24.809 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.72238.log.gz 2024-09-10T16:13:24.809 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.32355.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32355.log.gz 2024-09-10T16:13:24.809 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41604.log 2024-09-10T16:13:24.809 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.67223.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51547.log 2024-09-10T16:13:24.809 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67223.log.gz 2024-09-10T16:13:24.809 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.41604.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44176.log 2024-09-10T16:13:24.809 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41604.log.gz 2024-09-10T16:13:24.809 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.51547.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38579.log 2024-09-10T16:13:24.809 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51547.log.gz 2024-09-10T16:13:24.809 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.44176.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51757.log 2024-09-10T16:13:24.809 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44176.log.gz 2024-09-10T16:13:24.809 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.38579.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62685.log 2024-09-10T16:13:24.810 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38579.log.gz 2024-09-10T16:13:24.810 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.51757.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47331.log 2024-09-10T16:13:24.810 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51757.log.gz 2024-09-10T16:13:24.810 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.62685.log: -- /var/log/ceph/ceph-client.admin.42962.log 2024-09-10T16:13:24.810 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62685.log.gz 2024-09-10T16:13:24.810 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.47331.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64408.log 2024-09-10T16:13:24.810 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47331.log.gz 2024-09-10T16:13:24.810 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.42962.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36612.log 2024-09-10T16:13:24.810 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42962.log.gz 2024-09-10T16:13:24.810 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.64408.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74170.log 2024-09-10T16:13:24.810 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64408.log.gz 2024-09-10T16:13:24.810 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.36612.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59827.log 2024-09-10T16:13:24.810 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36612.log.gz 2024-09-10T16:13:24.810 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.74170.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56000.log 2024-09-10T16:13:24.811 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.74170.log.gz 2024-09-10T16:13:24.811 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.59827.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43340.log 2024-09-10T16:13:24.811 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59827.log.gz 2024-09-10T16:13:24.811 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.56000.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63316.log 2024-09-10T16:13:24.811 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56000.log.gz 2024-09-10T16:13:24.811 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.43340.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37539.log 2024-09-10T16:13:24.811 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43340.log.gz 2024-09-10T16:13:24.811 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.63316.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52891.log 2024-09-10T16:13:24.811 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63316.log.gz 2024-09-10T16:13:24.811 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.37539.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70684.log 2024-09-10T16:13:24.811 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37539.log.gz 2024-09-10T16:13:24.811 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.52891.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72868.log 2024-09-10T16:13:24.811 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52891.log.gz 2024-09-10T16:13:24.811 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43924.log 2024-09-10T16:13:24.812 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.70684.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.70684.log.gz 2024-09-10T16:13:24.812 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62432.log 2024-09-10T16:13:24.812 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.72868.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.72868.log.gz 2024-09-10T16:13:24.812 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.43924.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51505.log 2024-09-10T16:13:24.812 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43924.log.gz 2024-09-10T16:13:24.812 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.62432.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72574.log 2024-09-10T16:13:24.812 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62432.log.gz 2024-09-10T16:13:24.812 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.51505.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64366.log 2024-09-10T16:13:24.812 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51505.log.gz 2024-09-10T16:13:24.812 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.72574.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71608.log 2024-09-10T16:13:24.812 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.72574.log.gz 2024-09-10T16:13:24.812 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.64366.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37581.log 2024-09-10T16:13:24.812 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64366.log.gz 2024-09-10T16:13:24.813 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.71608.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60835.log 2024-09-10T16:13:24.813 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.71608.log.gz 2024-09-10T16:13:24.813 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.37581.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53648.log 2024-09-10T16:13:24.813 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37581.log.gz 2024-09-10T16:13:24.813 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.60835.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48676.log 2024-09-10T16:13:24.813 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60835.log.gz 2024-09-10T16:13:24.813 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.53648.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55202.log 2024-09-10T16:13:24.813 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53648.log.gz 2024-09-10T16:13:24.813 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.48676.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69844.log 2024-09-10T16:13:24.813 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48676.log.gz 2024-09-10T16:13:24.813 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.55202.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56084.log 2024-09-10T16:13:24.813 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55202.log.gz 2024-09-10T16:13:24.813 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.69844.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39144.log 2024-09-10T16:13:24.813 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69844.log.gz 2024-09-10T16:13:24.814 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.56084.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71986.log 2024-09-10T16:13:24.814 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56084.log.gz 2024-09-10T16:13:24.814 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.39144.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51883.log 2024-09-10T16:13:24.814 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39144.log.gz 2024-09-10T16:13:24.814 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.71986.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43546.log 2024-09-10T16:13:24.814 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.71986.log.gz 2024-09-10T16:13:24.814 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.51883.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44050.log 2024-09-10T16:13:24.814 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51883.log.gz 2024-09-10T16:13:24.814 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53480.log 2024-09-10T16:13:24.814 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.43546.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43546.log.gz 2024-09-10T16:13:24.814 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.44050.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54110.log 2024-09-10T16:13:24.814 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44050.log.gz 2024-09-10T16:13:24.814 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.53480.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57260.log 2024-09-10T16:13:24.814 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53480.log.gz 2024-09-10T16:13:24.815 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72154.log 2024-09-10T16:13:24.815 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.54110.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54110.log.gz 2024-09-10T16:13:24.815 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.57260.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63778.log 2024-09-10T16:13:24.815 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57260.log.gz 2024-09-10T16:13:24.815 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.72154.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70936.log 2024-09-10T16:13:24.815 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.72154.log.gz 2024-09-10T16:13:24.815 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.63778.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53396.log 2024-09-10T16:13:24.815 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63778.log.gz 2024-09-10T16:13:24.815 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.70936.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45429.log 2024-09-10T16:13:24.815 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.70936.log.gz 2024-09-10T16:13:24.815 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.53396.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67605.log 2024-09-10T16:13:24.815 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53396.log.gz 2024-09-10T16:13:24.815 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.45429.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54446.log 2024-09-10T16:13:24.816 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45429.log.gz 2024-09-10T16:13:24.816 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.67605.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37242.log 2024-09-10T16:13:24.816 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67605.log.gz 2024-09-10T16:13:24.816 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.54446.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52219.log 2024-09-10T16:13:24.816 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54446.log.gz 2024-09-10T16:13:24.816 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.37242.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37242.log.gz 2024-09-10T16:13:24.816 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58942.log 2024-09-10T16:13:24.816 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.52219.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47499.log 2024-09-10T16:13:24.816 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52219.log.gz 2024-09-10T16:13:24.816 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.58942.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60751.log 2024-09-10T16:13:24.816 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58942.log.gz 2024-09-10T16:13:24.816 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.47499.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53228.log 2024-09-10T16:13:24.816 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47499.log.gz 2024-09-10T16:13:24.816 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.60751.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69970.log 2024-09-10T16:13:24.817 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60751.log.gz 2024-09-10T16:13:24.817 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.53228.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65500.log 2024-09-10T16:13:24.817 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53228.log.gz 2024-09-10T16:13:24.817 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.69970.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52555.log 2024-09-10T16:13:24.817 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69970.log.gz 2024-09-10T16:13:24.817 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.65500.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-09-10T16:13:24.817 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65500.log.gz 2024-09-10T16:13:24.817 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.52555.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53564.log 2024-09-10T16:13:24.817 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52555.log.gz 2024-09-10T16:13:24.817 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51631.log 2024-09-10T16:13:24.817 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.53564.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67055.log 2024-09-10T16:13:24.817 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53564.log.gz 2024-09-10T16:13:24.817 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.51631.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39366.log 2024-09-10T16:13:24.818 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51631.log.gz 2024-09-10T16:13:24.818 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.67055.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73708.log 2024-09-10T16:13:24.818 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67055.log.gz 2024-09-10T16:13:24.818 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.39366.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59278.log 2024-09-10T16:13:24.818 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39366.log.gz 2024-09-10T16:13:24.818 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.73708.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66593.log 2024-09-10T16:13:24.818 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.73708.log.gz 2024-09-10T16:13:24.818 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.59278.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37010.log 2024-09-10T16:13:24.818 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59278.log.gz 2024-09-10T16:13:24.818 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.66593.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48339.log 2024-09-10T16:13:24.818 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66593.log.gz 2024-09-10T16:13:24.818 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.37010.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49180.log 2024-09-10T16:13:24.818 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37010.log.gz 2024-09-10T16:13:24.818 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.48339.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36990.log 2024-09-10T16:13:24.819 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48339.log.gz 2024-09-10T16:13:24.819 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.49180.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45244.log 2024-09-10T16:13:24.819 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49180.log.gz 2024-09-10T16:13:24.819 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.36990.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44428.log 2024-09-10T16:13:24.819 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36990.log.gz 2024-09-10T16:13:24.819 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.45244.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58310.log 2024-09-10T16:13:24.819 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45244.log.gz 2024-09-10T16:13:24.819 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.44428.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51001.log 2024-09-10T16:13:24.819 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44428.log.gz 2024-09-10T16:13:24.819 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.58310.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68698.log 2024-09-10T16:13:24.819 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58310.log.gz 2024-09-10T16:13:24.819 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.51001.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71734.log 2024-09-10T16:13:24.819 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51001.log.gz 2024-09-10T16:13:24.819 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.68698.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60289.log 2024-09-10T16:13:24.820 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68698.log.gz 2024-09-10T16:13:24.820 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.71734.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57554.log 2024-09-10T16:13:24.820 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.71734.log.gz 2024-09-10T16:13:24.820 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.60289.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50020.log 2024-09-10T16:13:24.820 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60289.log.gz 2024-09-10T16:13:24.820 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.57554.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57974.log 2024-09-10T16:13:24.820 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57554.log.gz 2024-09-10T16:13:24.820 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.50020.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71104.log 2024-09-10T16:13:24.820 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50020.log.gz 2024-09-10T16:13:24.820 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.57974.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68361.log 2024-09-10T16:13:24.820 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57974.log.gz 2024-09-10T16:13:24.820 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.71104.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69718.log 2024-09-10T16:13:24.820 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.71104.log.gz 2024-09-10T16:13:24.821 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.68361.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43882.log 2024-09-10T16:13:24.821 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68361.log.gz 2024-09-10T16:13:24.821 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.69718.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50707.log 2024-09-10T16:13:24.821 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69718.log.gz 2024-09-10T16:13:24.821 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.43882.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72910.log 2024-09-10T16:13:24.821 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43882.log.gz 2024-09-10T16:13:24.821 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.50707.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66383.log 2024-09-10T16:13:24.821 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50707.log.gz 2024-09-10T16:13:24.821 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.72910.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59743.log 2024-09-10T16:13:24.821 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.72910.log.gz 2024-09-10T16:13:24.821 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.66383.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63274.log 2024-09-10T16:13:24.821 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66383.log.gz 2024-09-10T16:13:24.821 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.59743.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61885.log 2024-09-10T16:13:24.821 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59743.log.gz 2024-09-10T16:13:24.822 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.63274.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44890.log 2024-09-10T16:13:24.822 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63274.log.gz 2024-09-10T16:13:24.822 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.61885.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47163.log 2024-09-10T16:13:24.822 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61885.log.gz 2024-09-10T16:13:24.822 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.44890.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53522.log 2024-09-10T16:13:24.822 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44890.log.gz 2024-09-10T16:13:24.822 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.47163.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58184.log 2024-09-10T16:13:24.822 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47163.log.gz 2024-09-10T16:13:24.822 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.53522.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43275.log 2024-09-10T16:13:24.822 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53522.log.gz 2024-09-10T16:13:24.822 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.58184.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49390.log 2024-09-10T16:13:24.822 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58184.log.gz 2024-09-10T16:13:24.822 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.43275.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55748.log 2024-09-10T16:13:24.823 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43275.log.gz 2024-09-10T16:13:24.823 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.49390.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69298.log 2024-09-10T16:13:24.823 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49390.log.gz 2024-09-10T16:13:24.823 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.55748.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61423.log 2024-09-10T16:13:24.823 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55748.log.gz 2024-09-10T16:13:24.823 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.69298.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56462.log 2024-09-10T16:13:24.823 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69298.log.gz 2024-09-10T16:13:24.823 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.61423.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70432.log 2024-09-10T16:13:24.823 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61423.log.gz 2024-09-10T16:13:24.823 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.56462.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48507.log 2024-09-10T16:13:24.823 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56462.log.gz 2024-09-10T16:13:24.823 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.70432.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48171.log 2024-09-10T16:13:24.823 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.70432.log.gz 2024-09-10T16:13:24.824 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.48507.log: gzip -5 0.0% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.48507.log.gz 2024-09-10T16:13:24.824 INFO:teuthology.orchestra.run.smithi115.stderr: /var/log/ceph/ceph-client.admin.63736.log 2024-09-10T16:13:24.824 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.48171.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47247.log 2024-09-10T16:13:24.824 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48171.log.gz 2024-09-10T16:13:24.824 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.63736.log: /var/log/ceph/ceph-client.admin.54068.log 2024-09-10T16:13:24.824 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63736.log.gz 2024-09-10T16:13:24.824 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.47247.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74086.log 2024-09-10T16:13:24.824 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47247.log.gz 2024-09-10T16:13:24.824 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.54068.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64450.log 2024-09-10T16:13:24.824 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54068.log.gz 2024-09-10T16:13:24.824 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.74086.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68445.log 2024-09-10T16:13:24.824 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.74086.log.gz 2024-09-10T16:13:24.824 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.64450.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42687.log 2024-09-10T16:13:24.825 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64450.log.gz 2024-09-10T16:13:24.825 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.68445.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57722.log 2024-09-10T16:13:24.825 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68445.log.gz 2024-09-10T16:13:24.825 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.42687.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53816.log 2024-09-10T16:13:24.825 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42687.log.gz 2024-09-10T16:13:24.825 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.57722.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49768.log 2024-09-10T16:13:24.825 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57722.log.gz 2024-09-10T16:13:24.825 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.53816.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73834.log 2024-09-10T16:13:24.825 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53816.log.gz 2024-09-10T16:13:24.825 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.49768.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42688.log 2024-09-10T16:13:24.825 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49768.log.gz 2024-09-10T16:13:24.825 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.73834.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43714.log 2024-09-10T16:13:24.825 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.73834.log.gz 2024-09-10T16:13:24.825 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.42688.log: /var/log/ceph/ceph-client.admin.60667.log 2024-09-10T16:13:24.826 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42688.log.gz 2024-09-10T16:13:24.826 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.43714.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51085.log 2024-09-10T16:13:24.826 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43714.log.gz 2024-09-10T16:13:24.826 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.60667.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49852.log 2024-09-10T16:13:24.826 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60667.log.gz 2024-09-10T16:13:24.826 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.51085.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70474.log 2024-09-10T16:13:24.826 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51085.log.gz 2024-09-10T16:13:24.826 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.49852.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49852.log.gz 2024-09-10T16:13:24.826 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57806.log 2024-09-10T16:13:24.826 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.70474.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61255.log 2024-09-10T16:13:24.826 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.70474.log.gz 2024-09-10T16:13:24.826 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.57806.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70642.log 2024-09-10T16:13:24.826 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57806.log.gz 2024-09-10T16:13:24.827 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.61255.log: gzip 0.0% -5 -- replaced with /var/log/ceph/ceph-client.admin.61255.log.gz --verbose 2024-09-10T16:13:24.827 INFO:teuthology.orchestra.run.smithi115.stderr: -- /var/log/ceph/ceph-client.admin.70348.log 2024-09-10T16:13:24.827 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.70642.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44092.log 2024-09-10T16:13:24.827 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.70642.log.gz 2024-09-10T16:13:24.827 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.70348.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55916.log 2024-09-10T16:13:24.827 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.70348.log.gz 2024-09-10T16:13:24.827 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.44092.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60709.log 2024-09-10T16:13:24.827 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44092.log.gz 2024-09-10T16:13:24.827 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62306.log 2024-09-10T16:13:24.827 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.55916.log: /var/log/ceph/ceph-client.admin.60709.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55916.log.gz 2024-09-10T16:13:24.827 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50791.log 2024-09-10T16:13:24.827 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60709.log.gz 2024-09-10T16:13:24.827 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.62306.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45207.log 2024-09-10T16:13:24.828 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62306.log.gz 2024-09-10T16:13:24.828 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.50791.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66089.log 2024-09-10T16:13:24.828 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50791.log.gz 2024-09-10T16:13:24.828 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.45207.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45688.log 2024-09-10T16:13:24.828 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45207.log.gz 2024-09-10T16:13:24.828 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.66089.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62222.log 2024-09-10T16:13:24.828 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66089.log.gz 2024-09-10T16:13:24.828 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.45688.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57092.log 2024-09-10T16:13:24.828 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45688.log.gz 2024-09-10T16:13:24.828 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46225.log 2024-09-10T16:13:24.828 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.62222.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62222.log.gz 2024-09-10T16:13:24.828 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.57092.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41058.log 2024-09-10T16:13:24.828 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57092.log.gz 2024-09-10T16:13:24.829 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62474.log/var/log/ceph/ceph-client.admin.46225.log: 2024-09-10T16:13:24.829 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46225.log.gz 2024-09-10T16:13:24.829 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.41058.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64030.log 2024-09-10T16:13:24.829 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41058.log.gz 2024-09-10T16:13:24.829 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.62474.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-09-10T16:13:24.829 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62474.log.gz 2024-09-10T16:13:24.829 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.64030.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37912.log 2024-09-10T16:13:24.829 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64030.log.gz 2024-09-10T16:13:24.829 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62601.log 2024-09-10T16:13:24.829 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.37912.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66257.log 2024-09-10T16:13:24.829 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37912.log.gz 2024-09-10T16:13:24.829 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.62601.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44386.log 2024-09-10T16:13:24.829 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62601.log.gz 2024-09-10T16:13:24.829 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.66257.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66257.log.gz 2024-09-10T16:13:24.830 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50104.log 2024-09-10T16:13:24.830 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.44386.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48844.log 2024-09-10T16:13:24.830 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44386.log.gz 2024-09-10T16:13:24.830 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.50104.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67983.log 2024-09-10T16:13:24.830 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50104.log.gz 2024-09-10T16:13:24.830 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.48844.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39292.log 2024-09-10T16:13:24.830 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48844.log.gz 2024-09-10T16:13:24.830 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.67983.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71944.log 2024-09-10T16:13:24.830 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67983.log.gz 2024-09-10T16:13:24.830 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.39292.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39699.log 2024-09-10T16:13:24.830 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39292.log.gz 2024-09-10T16:13:24.830 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.71944.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69256.log 2024-09-10T16:13:24.830 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.71944.log.gz 2024-09-10T16:13:24.831 INFO:teuthology.orchestra.run.smithi115.stderr: 96.2% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-09-10T16:13:24.831 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.39699.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49558.log 2024-09-10T16:13:24.831 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39699.log.gz 2024-09-10T16:13:24.831 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.69256.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38986.log 2024-09-10T16:13:24.831 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69256.log.gz 2024-09-10T16:13:24.831 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.49558.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64240.log 2024-09-10T16:13:24.831 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49558.log.gz 2024-09-10T16:13:24.831 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.38986.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39773.log 2024-09-10T16:13:24.831 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38986.log.gz 2024-09-10T16:13:24.831 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.64240.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64156.log 2024-09-10T16:13:24.831 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64240.log.gz 2024-09-10T16:13:24.831 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.39773.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57386.log 2024-09-10T16:13:24.831 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39773.log.gz 2024-09-10T16:13:24.831 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.64156.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70516.log 2024-09-10T16:13:24.832 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64156.log.gz 2024-09-10T16:13:24.832 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.57386.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50455.log 2024-09-10T16:13:24.832 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57386.log.gz 2024-09-10T16:13:24.832 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.70516.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52051.log 2024-09-10T16:13:24.832 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.70516.log.gz 2024-09-10T16:13:24.832 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.50455.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56756.log 2024-09-10T16:13:24.832 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50455.log.gz 2024-09-10T16:13:24.832 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.52051.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45540.log 2024-09-10T16:13:24.832 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52051.log.gz 2024-09-10T16:13:24.832 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.56756.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45016.log 2024-09-10T16:13:24.832 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56756.log.gz 2024-09-10T16:13:24.832 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.45540.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65711.log 2024-09-10T16:13:24.832 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45540.log.gz 2024-09-10T16:13:24.833 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.45016.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41330.log 2024-09-10T16:13:24.833 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45016.log.gz 2024-09-10T16:13:24.833 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.65711.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57050.log 2024-09-10T16:13:24.833 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65711.log.gz 2024-09-10T16:13:24.833 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.41330.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59152.log 2024-09-10T16:13:24.833 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41330.log.gz 2024-09-10T16:13:24.833 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.57050.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69676.log 2024-09-10T16:13:24.833 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57050.log.gz 2024-09-10T16:13:24.833 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.59152.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56126.log 2024-09-10T16:13:24.833 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59152.log.gz 2024-09-10T16:13:24.833 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.69676.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64576.log 2024-09-10T16:13:24.833 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69676.log.gz 2024-09-10T16:13:24.833 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.56126.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38061.log 2024-09-10T16:13:24.834 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56126.log.gz 2024-09-10T16:13:24.834 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.64576.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69886.log 2024-09-10T16:13:24.834 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64576.log.gz 2024-09-10T16:13:24.834 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.38061.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69340.log 2024-09-10T16:13:24.834 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38061.log.gz 2024-09-10T16:13:24.834 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.69886.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70222.log 2024-09-10T16:13:24.834 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69886.log.gz 2024-09-10T16:13:24.834 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.69340.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44554.log 2024-09-10T16:13:24.834 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69340.log.gz 2024-09-10T16:13:24.834 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.70222.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36904.log 2024-09-10T16:13:24.834 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.70222.log.gz 2024-09-10T16:13:24.834 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.44554.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57008.log 2024-09-10T16:13:24.834 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44554.log.gz 2024-09-10T16:13:24.835 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.36904.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39477.log 2024-09-10T16:13:24.835 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36904.log.gz 2024-09-10T16:13:24.835 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.57008.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60121.log 2024-09-10T16:13:24.835 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57008.log.gz 2024-09-10T16:13:24.835 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.39477.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39255.log 2024-09-10T16:13:24.835 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39477.log.gz 2024-09-10T16:13:24.835 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.60121.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73918.log 2024-09-10T16:13:24.835 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60121.log.gz 2024-09-10T16:13:24.835 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51337.log 2024-09-10T16:13:24.835 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.39255.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39255.log.gz 2024-09-10T16:13:24.835 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.73918.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38542.log 2024-09-10T16:13:24.835 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.73918.log.gz 2024-09-10T16:13:24.835 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.51337.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44638.log 2024-09-10T16:13:24.836 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51337.log.gz 2024-09-10T16:13:24.836 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.38542.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44218.log 2024-09-10T16:13:24.836 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38542.log.gz 2024-09-10T16:13:24.836 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.44638.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36696.log 2024-09-10T16:13:24.836 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44638.log.gz 2024-09-10T16:13:24.836 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.44218.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62727.log 2024-09-10T16:13:24.836 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44218.log.gz 2024-09-10T16:13:24.836 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.36696.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-09-10T16:13:24.836 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36696.log.gz 2024-09-10T16:13:24.836 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.62727.log: gzip 0.0% -5 -- replaced with /var/log/ceph/ceph-client.admin.62727.log.gz 2024-09-10T16:13:24.836 INFO:teuthology.orchestra.run.smithi115.stderr: --verbose -- /var/log/ceph/ceph-client.admin.63190.log 2024-09-10T16:13:24.836 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74002.log 2024-09-10T16:13:24.837 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.63190.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37680.log 2024-09-10T16:13:24.837 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63190.log.gz 2024-09-10T16:13:24.837 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.74002.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59236.log 2024-09-10T16:13:24.837 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.74002.log.gz 2024-09-10T16:13:24.837 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.37680.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43840.log 2024-09-10T16:13:24.837 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37680.log.gz 2024-09-10T16:13:24.837 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.59236.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60205.log 2024-09-10T16:13:24.837 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59236.log.gz 2024-09-10T16:13:24.837 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.43840.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66047.log 2024-09-10T16:13:24.837 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43840.log.gz 2024-09-10T16:13:24.837 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.60205.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57470.log 2024-09-10T16:13:24.837 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60205.log.gz 2024-09-10T16:13:24.837 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.66047.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64492.log 2024-09-10T16:13:24.838 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66047.log.gz 2024-09-10T16:13:24.838 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.57470.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65290.log 2024-09-10T16:13:24.838 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57470.log.gz 2024-09-10T16:13:24.838 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.64492.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68961.log 2024-09-10T16:13:24.838 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64492.log.gz 2024-09-10T16:13:24.838 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.65290.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32114.log 2024-09-10T16:13:24.838 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65290.log.gz 2024-09-10T16:13:24.838 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.68961.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60163.log 2024-09-10T16:13:24.838 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68961.log.gz 2024-09-10T16:13:24.838 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.32114.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69508.log 2024-09-10T16:13:24.838 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32114.log.gz 2024-09-10T16:13:24.838 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49012.log 2024-09-10T16:13:24.838 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.60163.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60163.log.gz 2024-09-10T16:13:24.839 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.69508.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52387.log 2024-09-10T16:13:24.839 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69508.log.gz 2024-09-10T16:13:24.839 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.49012.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49012.log.gz 2024-09-10T16:13:24.839 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66971.log 2024-09-10T16:13:24.839 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.52387.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58732.log 2024-09-10T16:13:24.839 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52387.log.gz 2024-09-10T16:13:24.839 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.66971.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62348.log 2024-09-10T16:13:24.839 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66971.log.gz 2024-09-10T16:13:24.839 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.58732.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67181.log 2024-09-10T16:13:24.839 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58732.log.gz 2024-09-10T16:13:24.839 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54866.log 2024-09-10T16:13:24.839 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.62348.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62348.log.gz 2024-09-10T16:13:24.839 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.67181.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54278.log 2024-09-10T16:13:24.839 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67181.log.gz 2024-09-10T16:13:24.840 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.54866.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60247.log 2024-09-10T16:13:24.840 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54866.log.gz 2024-09-10T16:13:24.840 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61675.log 2024-09-10T16:13:24.840 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.54278.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54278.log.gz 2024-09-10T16:13:24.840 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.60247.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68235.log 2024-09-10T16:13:24.840 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60247.log.gz 2024-09-10T16:13:24.840 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55622.log 2024-09-10T16:13:24.840 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.61675.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61675.log.gz 2024-09-10T16:13:24.840 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.68235.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42772.log 2024-09-10T16:13:24.840 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68235.log.gz 2024-09-10T16:13:24.840 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.55622.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55622.log.gz 2024-09-10T16:13:24.840 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65374.log 2024-09-10T16:13:24.840 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.42772.log: /var/log/ceph/ceph-client.admin.39847.log 2024-09-10T16:13:24.841 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42772.log.gz 2024-09-10T16:13:24.841 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.65374.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57512.log 2024-09-10T16:13:24.841 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65374.log.gz 2024-09-10T16:13:24.841 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.39847.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47793.log 2024-09-10T16:13:24.841 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39847.log.gz 2024-09-10T16:13:24.841 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.57512.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61381.log 2024-09-10T16:13:24.841 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57512.log.gz 2024-09-10T16:13:24.841 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.47793.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44512.log 2024-09-10T16:13:24.841 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47793.log.gz 2024-09-10T16:13:24.841 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.61381.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43756.log 2024-09-10T16:13:24.841 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61381.log.gz 2024-09-10T16:13:24.841 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.44512.log: gzip -5 --verbose 0.0% -- -- replaced with /var/log/ceph/ceph-client.admin.44512.log.gz /var/log/ceph/ceph-client.admin.43108.log 2024-09-10T16:13:24.841 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-09-10T16:13:24.842 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.43756.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62516.log 2024-09-10T16:13:24.842 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43756.log.gz 2024-09-10T16:13:24.842 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.43108.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70096.log 2024-09-10T16:13:24.842 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43108.log.gz 2024-09-10T16:13:24.842 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.62516.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72616.log 2024-09-10T16:13:24.842 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62516.log.gz 2024-09-10T16:13:24.842 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.70096.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38468.log 2024-09-10T16:13:24.842 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.70096.log.gz 2024-09-10T16:13:24.842 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.72616.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52639.log 2024-09-10T16:13:24.842 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.72616.log.gz 2024-09-10T16:13:24.842 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.38468.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69634.log 2024-09-10T16:13:24.842 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38468.log.gz 2024-09-10T16:13:24.842 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.52639.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53900.log 2024-09-10T16:13:24.843 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52639.log.gz 2024-09-10T16:13:24.843 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.69634.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44974.log 2024-09-10T16:13:24.843 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69634.log.gz 2024-09-10T16:13:24.843 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.53900.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53900.log.gz 2024-09-10T16:13:24.843 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41520.log 2024-09-10T16:13:24.843 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.44974.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37764.log 2024-09-10T16:13:24.843 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44974.log.gz 2024-09-10T16:13:24.843 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.41520.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58226.log 2024-09-10T16:13:24.843 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41520.log.gz 2024-09-10T16:13:24.843 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.37764.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37764.log.gz 2024-09-10T16:13:24.843 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46132.log 2024-09-10T16:13:24.843 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.58226.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64114.log 2024-09-10T16:13:24.843 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58226.log.gz 2024-09-10T16:13:24.844 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.46132.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73960.log 2024-09-10T16:13:24.844 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46132.log.gz 2024-09-10T16:13:24.844 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.64114.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63484.log 2024-09-10T16:13:24.844 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64114.log.gz 2024-09-10T16:13:24.844 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.73960.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.73960.log.gzgzip 2024-09-10T16:13:24.844 INFO:teuthology.orchestra.run.smithi115.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.73540.log 2024-09-10T16:13:24.844 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.63484.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60541.log 2024-09-10T16:13:24.844 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63484.log.gz 2024-09-10T16:13:24.844 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.73540.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65921.log 2024-09-10T16:13:24.844 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.73540.log.gz 2024-09-10T16:13:24.844 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.60541.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62180.log 2024-09-10T16:13:24.844 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60541.log.gz 2024-09-10T16:13:24.844 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.65921.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52723.log 2024-09-10T16:13:24.845 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65921.log.gz 2024-09-10T16:13:24.845 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.62180.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62180.log.gzgzip 2024-09-10T16:13:24.845 INFO:teuthology.orchestra.run.smithi115.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.46169.log 2024-09-10T16:13:24.845 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.52723.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51841.log 2024-09-10T16:13:24.845 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52723.log.gz 2024-09-10T16:13:24.845 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.46169.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74044.log 2024-09-10T16:13:24.845 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46169.log.gz 2024-09-10T16:13:24.845 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.51841.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50146.log 2024-09-10T16:13:24.845 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51841.log.gz 2024-09-10T16:13:24.845 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.74044.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52807.log 2024-09-10T16:13:24.845 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.74044.log.gz 2024-09-10T16:13:24.845 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47835.log 2024-09-10T16:13:24.845 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.50146.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50146.log.gz 2024-09-10T16:13:24.845 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.52807.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61297.log 2024-09-10T16:13:24.846 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52807.log.gz 2024-09-10T16:13:24.846 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.47835.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73624.log 2024-09-10T16:13:24.846 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47835.log.gz 2024-09-10T16:13:24.846 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.61297.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72700.log 2024-09-10T16:13:24.846 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61297.log.gz 2024-09-10T16:13:24.846 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.73624.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62895.log 2024-09-10T16:13:24.846 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.73624.log.gz 2024-09-10T16:13:24.846 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.72700.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-09-10T16:13:24.846 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.72700.log.gz 2024-09-10T16:13:24.846 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.62895.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34921.log 2024-09-10T16:13:24.846 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62895.log.gz 2024-09-10T16:13:24.846 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72742.log 2024-09-10T16:13:24.846 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.34921.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73498.log 2024-09-10T16:13:24.846 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34921.log.gz 2024-09-10T16:13:24.847 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.72742.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.72742.log.gz 2024-09-10T16:13:24.847 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41100.log 2024-09-10T16:13:24.847 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.73498.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59068.log 2024-09-10T16:13:24.847 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.73498.log.gz 2024-09-10T16:13:24.847 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.41100.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41100.log.gz 2024-09-10T16:13:24.847 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67647.log 2024-09-10T16:13:24.847 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.59068.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44740.log 2024-09-10T16:13:24.847 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59068.log.gz 2024-09-10T16:13:24.847 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.67647.log: -- /var/log/ceph/ceph-client.admin.62559.log 2024-09-10T16:13:24.847 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67647.log.gz 2024-09-10T16:13:24.847 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.44740.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44740.log.gz 2024-09-10T16:13:24.847 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56294.log 2024-09-10T16:13:24.847 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.62559.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55076.log 2024-09-10T16:13:24.847 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62559.log.gz 2024-09-10T16:13:24.848 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.56294.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54026.log 2024-09-10T16:13:24.848 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56294.log.gz 2024-09-10T16:13:24.848 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.55076.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41646.log 2024-09-10T16:13:24.848 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55076.log.gz 2024-09-10T16:13:24.848 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.54026.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55286.log 2024-09-10T16:13:24.848 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54026.log.gz 2024-09-10T16:13:24.848 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.41646.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41646.log.gz 2024-09-10T16:13:24.848 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62390.log 2024-09-10T16:13:24.848 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.55286.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63694.log 2024-09-10T16:13:24.848 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55286.log.gz 2024-09-10T16:13:24.848 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.62390.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48970.log 2024-09-10T16:13:24.848 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62390.log.gz 2024-09-10T16:13:24.848 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.63694.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72322.log 2024-09-10T16:13:24.849 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63694.log.gz 2024-09-10T16:13:24.849 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.48970.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54950.log 2024-09-10T16:13:24.849 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48970.log.gz 2024-09-10T16:13:24.849 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.72322.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67349.log 2024-09-10T16:13:24.849 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.72322.log.gz 2024-09-10T16:13:24.849 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.54950.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47583.log 2024-09-10T16:13:24.849 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54950.log.gz 2024-09-10T16:13:24.849 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.67349.log: gzip 0.0% -5 -- replaced with /var/log/ceph/ceph-client.admin.67349.log.gz 2024-09-10T16:13:24.849 INFO:teuthology.orchestra.run.smithi115.stderr: --verbose -- /var/log/ceph/ceph-client.admin.67562.log 2024-09-10T16:13:24.849 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.47583.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73414.log 2024-09-10T16:13:24.849 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47583.log.gz 2024-09-10T16:13:24.849 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.67562.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62053.log 2024-09-10T16:13:24.849 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67562.log.gz 2024-09-10T16:13:24.850 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.73414.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67731.log 2024-09-10T16:13:24.850 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.73414.log.gz 2024-09-10T16:13:24.850 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.62053.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49138.log 2024-09-10T16:13:24.850 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62053.log.gz 2024-09-10T16:13:24.850 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.67731.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66425.log 2024-09-10T16:13:24.850 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67731.log.gz 2024-09-10T16:13:24.850 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.49138.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64912.log 2024-09-10T16:13:24.850 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49138.log.gz 2024-09-10T16:13:24.850 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.66425.log: gzip 0.0% -5 -- replaced with /var/log/ceph/ceph-client.admin.66425.log.gz --verbose 2024-09-10T16:13:24.850 INFO:teuthology.orchestra.run.smithi115.stderr: -- /var/log/ceph/ceph-client.admin.64744.log 2024-09-10T16:13:24.850 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.64912.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67941.log 2024-09-10T16:13:24.850 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64912.log.gz 2024-09-10T16:13:24.850 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.64744.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67689.log 2024-09-10T16:13:24.851 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64744.log.gz 2024-09-10T16:13:24.851 INFO:teuthology.orchestra.run.smithi115.stderr: 95.5% -- replaced with /var/log/ceph/ceph.log.gz 2024-09-10T16:13:24.851 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.67941.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68193.log 2024-09-10T16:13:24.851 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67941.log.gz 2024-09-10T16:13:24.851 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.67689.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37326.log 2024-09-10T16:13:24.851 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67689.log.gz 2024-09-10T16:13:24.851 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.68193.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61717.log 2024-09-10T16:13:24.851 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68193.log.gz 2024-09-10T16:13:24.851 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50665.log 2024-09-10T16:13:24.851 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.37326.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37326.log.gz 2024-09-10T16:13:24.851 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.61717.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37116.log 2024-09-10T16:13:24.851 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61717.log.gz 2024-09-10T16:13:24.851 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.50665.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-09-10T16:13:24.851 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50665.log.gz 2024-09-10T16:13:24.852 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.37116.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37116.log.gz 2024-09-10T16:13:24.852 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70012.log 2024-09-10T16:13:24.852 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42856.log 2024-09-10T16:13:24.852 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.70012.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39958.log 2024-09-10T16:13:24.852 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.70012.log.gz 2024-09-10T16:13:24.852 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.42856.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51673.log 2024-09-10T16:13:24.852 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42856.log.gz 2024-09-10T16:13:24.852 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.39958.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50581.log 2024-09-10T16:13:24.852 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39958.log.gz 2024-09-10T16:13:24.852 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.51673.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49642.log 2024-09-10T16:13:24.852 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51673.log.gz 2024-09-10T16:13:24.852 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.50581.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66467.log 2024-09-10T16:13:24.852 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50581.log.gz 2024-09-10T16:13:24.853 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58520.log 2024-09-10T16:13:24.853 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.49642.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49642.log.gz 2024-09-10T16:13:24.853 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.66467.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49726.log 2024-09-10T16:13:24.853 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66467.log.gz 2024-09-10T16:13:24.853 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.58520.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65164.log 2024-09-10T16:13:24.853 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58520.log.gz 2024-09-10T16:13:24.853 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.49726.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62643.log 2024-09-10T16:13:24.853 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49726.log.gz 2024-09-10T16:13:24.853 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.65164.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61759.log 2024-09-10T16:13:24.853 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65164.log.gz 2024-09-10T16:13:24.853 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.62643.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48549.log 2024-09-10T16:13:24.853 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62643.log.gz 2024-09-10T16:13:24.853 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.61759.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63652.log 2024-09-10T16:13:24.853 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61759.log.gz 2024-09-10T16:13:24.854 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.48549.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39921.log 2024-09-10T16:13:24.854 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48549.log.gz 2024-09-10T16:13:24.854 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.63652.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63652.log.gz 2024-09-10T16:13:24.854 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70852.log 2024-09-10T16:13:24.854 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.39921.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50875.log 2024-09-10T16:13:24.854 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39921.log.gz 2024-09-10T16:13:24.854 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.70852.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.70852.log.gz 2024-09-10T16:13:24.854 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31818.log 2024-09-10T16:13:24.854 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.50875.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39181.log 2024-09-10T16:13:24.854 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50875.log.gz 2024-09-10T16:13:24.854 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.31818.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40106.log 2024-09-10T16:13:24.854 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31818.log.gz 2024-09-10T16:13:24.855 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.39181.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68835.log 2024-09-10T16:13:24.855 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39181.log.gz 2024-09-10T16:13:24.855 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.40106.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40106.log.gz 2024-09-10T16:13:24.855 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53186.log 2024-09-10T16:13:24.855 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.68835.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39551.log 2024-09-10T16:13:24.855 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68835.log.gz 2024-09-10T16:13:24.855 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.53186.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50272.log 2024-09-10T16:13:24.855 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53186.log.gz 2024-09-10T16:13:24.855 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.39551.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48886.log 2024-09-10T16:13:24.855 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39551.log.gz 2024-09-10T16:13:24.855 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.50272.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51463.log 2024-09-10T16:13:24.855 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50272.log.gz 2024-09-10T16:13:24.855 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.48886.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54992.log 2024-09-10T16:13:24.856 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48886.log.gz 2024-09-10T16:13:24.856 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.51463.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51211.log 2024-09-10T16:13:24.856 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51463.log.gz 2024-09-10T16:13:24.856 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.54992.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45184.log 2024-09-10T16:13:24.856 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54992.log.gz 2024-09-10T16:13:24.856 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.51211.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49264.log 2024-09-10T16:13:24.856 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51211.log.gz 2024-09-10T16:13:24.856 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.45184.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36494.log 2024-09-10T16:13:24.856 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45184.log.gz 2024-09-10T16:13:24.856 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44826.log 2024-09-10T16:13:24.856 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.49264.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49264.log.gz 2024-09-10T16:13:24.856 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.36494.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45577.log 2024-09-10T16:13:24.856 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36494.log.gz 2024-09-10T16:13:24.857 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.44826.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44826.log.gz 2024-09-10T16:13:24.857 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57134.log 2024-09-10T16:13:24.857 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63064.log 2024-09-10T16:13:24.857 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.45577.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45577.log.gz 2024-09-10T16:13:24.857 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.57134.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70558.log 2024-09-10T16:13:24.857 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57134.log.gz 2024-09-10T16:13:24.857 INFO:teuthology.orchestra.run.smithi115.stderr:gzip/var/log/ceph/ceph-client.admin.63064.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.49600.log 2024-09-10T16:13:24.857 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63064.log.gz 2024-09-10T16:13:24.857 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.70558.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44825.log 2024-09-10T16:13:24.857 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.70558.log.gz 2024-09-10T16:13:24.857 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.49600.log: gzip -5 --verbose -- 0.0% /var/log/ceph/ceph-client.admin.70810.log -- replaced with /var/log/ceph/ceph-client.admin.49600.log.gz 2024-09-10T16:13:24.857 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-09-10T16:13:24.858 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.44825.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69802.log 2024-09-10T16:13:24.858 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44825.log.gz 2024-09-10T16:13:24.858 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.70810.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55790.log 2024-09-10T16:13:24.858 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.70810.log.gz 2024-09-10T16:13:24.858 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.69802.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44302.log 2024-09-10T16:13:24.858 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69802.log.gz 2024-09-10T16:13:24.858 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.55790.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45392.log 2024-09-10T16:13:24.858 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55790.log.gz 2024-09-10T16:13:24.858 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.44302.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39625.log 2024-09-10T16:13:24.858 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44302.log.gz 2024-09-10T16:13:24.858 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.45392.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55412.log 2024-09-10T16:13:24.858 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45392.log.gz 2024-09-10T16:13:24.858 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.39625.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53060.log 2024-09-10T16:13:24.858 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39625.log.gz 2024-09-10T16:13:24.859 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.55412.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56840.log 2024-09-10T16:13:24.859 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55412.log.gz 2024-09-10T16:13:24.859 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.53060.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71440.log 2024-09-10T16:13:24.859 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53060.log.gz 2024-09-10T16:13:24.859 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.56840.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67013.log 2024-09-10T16:13:24.859 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56840.log.gz 2024-09-10T16:13:24.859 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.71440.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47751.log 2024-09-10T16:13:24.859 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.71440.log.gz 2024-09-10T16:13:24.859 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.67013.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52513.log 2024-09-10T16:13:24.859 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67013.log.gz 2024-09-10T16:13:24.859 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.47751.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60499.log 2024-09-10T16:13:24.859 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47751.log.gz 2024-09-10T16:13:24.859 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.52513.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59320.log 2024-09-10T16:13:24.860 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52513.log.gz 2024-09-10T16:13:24.860 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.60499.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73582.log 2024-09-10T16:13:24.860 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60499.log.gz 2024-09-10T16:13:24.860 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.59320.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66551.log 2024-09-10T16:13:24.860 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59320.log.gz 2024-09-10T16:13:24.860 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.73582.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40032.log 2024-09-10T16:13:24.860 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.73582.log.gz 2024-09-10T16:13:24.860 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.66551.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37847.log 2024-09-10T16:13:24.860 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66551.log.gz 2024-09-10T16:13:24.860 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.40032.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73204.log 2024-09-10T16:13:24.860 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40032.log.gz 2024-09-10T16:13:24.860 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.37847.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62769.log 2024-09-10T16:13:24.860 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37847.log.gz 2024-09-10T16:13:24.861 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.73204.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45318.log 2024-09-10T16:13:24.861 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.73204.log.gz 2024-09-10T16:13:24.861 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.62769.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43191.log 2024-09-10T16:13:24.861 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62769.log.gz 2024-09-10T16:13:24.861 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.45318.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43420.log 2024-09-10T16:13:24.861 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45318.log.gz 2024-09-10T16:13:24.861 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.43191.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52261.log 2024-09-10T16:13:24.861 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43191.log.gz 2024-09-10T16:13:24.861 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.43420.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38875.log 2024-09-10T16:13:24.861 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43420.log.gz 2024-09-10T16:13:24.861 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.52261.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37158.log 2024-09-10T16:13:24.861 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52261.log.gz 2024-09-10T16:13:24.861 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.38875.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63021.log 2024-09-10T16:13:24.862 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38875.log.gz 2024-09-10T16:13:24.862 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.37158.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56168.log 2024-09-10T16:13:24.862 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37158.log.gz 2024-09-10T16:13:24.862 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.63021.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63021.log.gz 2024-09-10T16:13:24.862 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59911.log 2024-09-10T16:13:24.862 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.56168.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43588.log 2024-09-10T16:13:24.862 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56168.log.gz 2024-09-10T16:13:24.862 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.59911.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68793.log 2024-09-10T16:13:24.862 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59911.log.gz 2024-09-10T16:13:24.862 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.43588.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60331.log 2024-09-10T16:13:24.862 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43588.log.gz 2024-09-10T16:13:24.862 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.68793.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58100.log 2024-09-10T16:13:24.862 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68793.log.gz 2024-09-10T16:13:24.862 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.60331.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45142.log 2024-09-10T16:13:24.863 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60331.log.gz 2024-09-10T16:13:24.863 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.58100.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56504.log 2024-09-10T16:13:24.863 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58100.log.gz 2024-09-10T16:13:24.863 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.45142.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52597.log 2024-09-10T16:13:24.863 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45142.log.gz 2024-09-10T16:13:24.863 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.56504.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56252.log 2024-09-10T16:13:24.863 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56504.log.gz 2024-09-10T16:13:24.863 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.52597.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70978.log 2024-09-10T16:13:24.863 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52597.log.gz 2024-09-10T16:13:24.863 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.56252.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45947.log 2024-09-10T16:13:24.863 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56252.log.gz 2024-09-10T16:13:24.863 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.70978.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68319.log 2024-09-10T16:13:24.864 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.70978.log.gz 2024-09-10T16:13:24.864 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.45947.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61003.log 2024-09-10T16:13:24.864 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45947.log.gz 2024-09-10T16:13:24.864 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.68319.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73372.log 2024-09-10T16:13:24.864 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68319.log.gz 2024-09-10T16:13:24.864 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.61003.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52765.log 2024-09-10T16:13:24.864 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61003.log.gz 2024-09-10T16:13:24.864 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57680.log 2024-09-10T16:13:24.864 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.73372.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.73372.log.gz 2024-09-10T16:13:24.864 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.52765.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54320.log 2024-09-10T16:13:24.864 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52765.log.gz 2024-09-10T16:13:24.864 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.57680.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62937.log 2024-09-10T16:13:24.864 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57680.log.gz 2024-09-10T16:13:24.865 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.54320.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59953.log 2024-09-10T16:13:24.865 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54320.log.gz 2024-09-10T16:13:24.865 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.62937.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57344.log 2024-09-10T16:13:24.865 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62937.log.gz 2024-09-10T16:13:24.865 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.59953.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47877.log 2024-09-10T16:13:24.865 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59953.log.gz 2024-09-10T16:13:24.865 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.57344.log: gzip -5 0.0% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.57344.log.gz /var/log/ceph/ceph-client.admin.55034.log 2024-09-10T16:13:24.865 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-09-10T16:13:24.865 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.47877.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63946.log 2024-09-10T16:13:24.865 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47877.log.gz 2024-09-10T16:13:24.865 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.55034.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55034.log.gz 2024-09-10T16:13:24.865 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-09-10T16:13:24.865 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.63946.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57932.log 2024-09-10T16:13:24.866 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63946.log.gz 2024-09-10T16:13:24.866 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43363.log 2024-09-10T16:13:24.866 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.57932.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45503.log 2024-09-10T16:13:24.866 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57932.log.gz 2024-09-10T16:13:24.866 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43462.log 2024-09-10T16:13:24.866 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.43363.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43363.log.gz 2024-09-10T16:13:24.866 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.45503.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63988.log 2024-09-10T16:13:24.866 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45503.log.gz 2024-09-10T16:13:24.866 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.43462.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38727.log 2024-09-10T16:13:24.866 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43462.log.gz 2024-09-10T16:13:24.866 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68025.log 2024-09-10T16:13:24.866 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.63988.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63988.log.gz 2024-09-10T16:13:24.866 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.38727.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50230.log 2024-09-10T16:13:24.867 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38727.log.gz 2024-09-10T16:13:24.867 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38283.log 2024-09-10T16:13:24.867 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.68025.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68025.log.gz 2024-09-10T16:13:24.867 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60919.log 2024-09-10T16:13:24.867 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.50230.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50230.log.gz 2024-09-10T16:13:24.867 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.38283.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73246.log 2024-09-10T16:13:24.867 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38283.log.gz 2024-09-10T16:13:24.867 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50062.log 2024-09-10T16:13:24.867 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.60919.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60919.log.gz 2024-09-10T16:13:24.867 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.73246.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72364.log 2024-09-10T16:13:24.867 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.73246.log.gz 2024-09-10T16:13:24.867 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.50062.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60625.log 2024-09-10T16:13:24.867 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50062.log.gz 2024-09-10T16:13:24.868 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.72364.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.72364.log.gz 2024-09-10T16:13:24.868 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43046.log 2024-09-10T16:13:24.868 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.60625.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47709.log 2024-09-10T16:13:24.868 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60625.log.gz 2024-09-10T16:13:24.868 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.43046.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52471.log 2024-09-10T16:13:24.868 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43046.log.gz 2024-09-10T16:13:24.868 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.47709.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61801.log 2024-09-10T16:13:24.868 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47709.log.gz 2024-09-10T16:13:24.868 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.52471.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56798.log 2024-09-10T16:13:24.868 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52471.log.gz 2024-09-10T16:13:24.868 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.61801.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39023.log 2024-09-10T16:13:24.868 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61801.log.gz 2024-09-10T16:13:24.868 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.56798.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70306.log 2024-09-10T16:13:24.869 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56798.log.gz 2024-09-10T16:13:24.869 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.39023.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66215.log 2024-09-10T16:13:24.869 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39023.log.gz 2024-09-10T16:13:24.869 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.70306.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54530.log 2024-09-10T16:13:24.869 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.70306.log.gz 2024-09-10T16:13:24.869 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37284.log 2024-09-10T16:13:24.869 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.66215.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66215.log.gz 2024-09-10T16:13:24.869 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.54530.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49054.log 2024-09-10T16:13:24.869 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54530.log.gz 2024-09-10T16:13:24.869 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.37284.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67857.log 2024-09-10T16:13:24.869 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37284.log.gz 2024-09-10T16:13:24.869 INFO:teuthology.orchestra.run.smithi115.stderr:gzip/var/log/ceph/ceph-client.admin.49054.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.69592.log 2024-09-10T16:13:24.869 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49054.log.gz 2024-09-10T16:13:24.870 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.67857.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72952.log 2024-09-10T16:13:24.870 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67857.log.gz 2024-09-10T16:13:24.870 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.69592.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45873.log 2024-09-10T16:13:24.870 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69592.log.gz 2024-09-10T16:13:24.870 INFO:teuthology.orchestra.run.smithi115.stderr:gzip/var/log/ceph/ceph-client.admin.72952.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.64198.log 2024-09-10T16:13:24.870 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.72952.log.gz 2024-09-10T16:13:24.870 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.45873.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47289.log 2024-09-10T16:13:24.870 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45873.log.gz 2024-09-10T16:13:24.870 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.64198.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64198.log.gz 2024-09-10T16:13:24.870 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39662.log 2024-09-10T16:13:24.870 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.47289.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53354.log 2024-09-10T16:13:24.870 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47289.log.gz 2024-09-10T16:13:24.870 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.39662.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66803.log 2024-09-10T16:13:24.871 INFO:teuthology.orchestra.run.smithi115.stderr: -- replaced with /var/log/ceph/ceph-client.admin.39662.log.gz 2024-09-10T16:13:24.871 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.53354.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60457.log 2024-09-10T16:13:24.871 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53354.log.gz 2024-09-10T16:13:24.871 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.66803.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66635.log 2024-09-10T16:13:24.871 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66803.log.gz 2024-09-10T16:13:24.871 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.60457.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60457.log.gz 2024-09-10T16:13:24.871 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57890.log 2024-09-10T16:13:24.871 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54404.log 2024-09-10T16:13:24.871 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.66635.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66635.log.gz 2024-09-10T16:13:24.871 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.57890.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57890.log.gz 2024-09-10T16:13:24.871 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61087.log 2024-09-10T16:13:24.871 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.54404.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68919.log 2024-09-10T16:13:24.872 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54404.log.gz 2024-09-10T16:13:24.872 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.61087.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73120.log 2024-09-10T16:13:24.872 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61087.log.gz 2024-09-10T16:13:24.872 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.68919.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66509.log 2024-09-10T16:13:24.872 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68919.log.gz 2024-09-10T16:13:24.872 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.73120.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58604.log 2024-09-10T16:13:24.872 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.73120.log.gz 2024-09-10T16:13:24.872 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.66509.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47457.log 2024-09-10T16:13:24.872 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66509.log.gz 2024-09-10T16:13:24.872 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.58604.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31995.log 2024-09-10T16:13:24.872 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58604.log.gz 2024-09-10T16:13:24.872 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.47457.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45355.log 2024-09-10T16:13:24.872 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47457.log.gz 2024-09-10T16:13:24.872 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.31995.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45651.log 2024-09-10T16:13:24.873 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31995.log.gz 2024-09-10T16:13:24.873 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.45355.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72196.log 2024-09-10T16:13:24.873 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45355.log.gz 2024-09-10T16:13:24.873 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.45651.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61171.log 2024-09-10T16:13:24.873 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45651.log.gz 2024-09-10T16:13:24.873 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.72196.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50188.log 2024-09-10T16:13:24.873 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.72196.log.gz 2024-09-10T16:13:24.873 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.61171.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36570.log 2024-09-10T16:13:24.873 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61171.log.gz 2024-09-10T16:13:24.873 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.50188.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50188.log.gz 2024-09-10T16:13:24.873 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54152.log 2024-09-10T16:13:24.873 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.36570.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45281.log 2024-09-10T16:13:24.873 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36570.log.gz 2024-09-10T16:13:24.874 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.54152.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41413.log 2024-09-10T16:13:24.874 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54152.log.gz 2024-09-10T16:13:24.874 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.45281.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54362.log 2024-09-10T16:13:24.874 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45281.log.gz 2024-09-10T16:13:24.874 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.41413.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58858.log 2024-09-10T16:13:24.874 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41413.log.gz 2024-09-10T16:13:24.874 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.54362.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39329.log 2024-09-10T16:13:24.874 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54362.log.gz 2024-09-10T16:13:24.874 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.58858.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60583.log 2024-09-10T16:13:24.874 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58858.log.gz 2024-09-10T16:13:24.874 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.39329.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58816.log 2024-09-10T16:13:24.874 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39329.log.gz 2024-09-10T16:13:24.874 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.60583.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59575.log 2024-09-10T16:13:24.875 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60583.log.gz 2024-09-10T16:13:24.875 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.58816.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68151.log 2024-09-10T16:13:24.875 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58816.log.gz 2024-09-10T16:13:24.875 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.59575.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72448.log 2024-09-10T16:13:24.875 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59575.log.gz 2024-09-10T16:13:24.875 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.68151.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39995.log 2024-09-10T16:13:24.875 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68151.log.gz 2024-09-10T16:13:24.875 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.72448.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51043.log 2024-09-10T16:13:24.875 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.72448.log.gz 2024-09-10T16:13:24.875 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.39995.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38098.log 2024-09-10T16:13:24.875 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39995.log.gz 2024-09-10T16:13:24.875 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.51043.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51043.log.gz 2024-09-10T16:13:24.875 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41245.log 2024-09-10T16:13:24.876 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.38098.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51799.log 2024-09-10T16:13:24.876 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38098.log.gz 2024-09-10T16:13:24.876 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.41245.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67097.log 2024-09-10T16:13:24.876 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41245.log.gz 2024-09-10T16:13:24.876 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.51799.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58436.log 2024-09-10T16:13:24.876 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51799.log.gz 2024-09-10T16:13:24.876 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.67097.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63148.log 2024-09-10T16:13:24.876 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67097.log.gz 2024-09-10T16:13:24.876 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.58436.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45910.log 2024-09-10T16:13:24.876 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58436.log.gz 2024-09-10T16:13:24.876 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.63148.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55118.log 2024-09-10T16:13:24.876 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63148.log.gz 2024-09-10T16:13:24.876 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.45910.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38801.log 2024-09-10T16:13:24.877 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45910.log.gz 2024-09-10T16:13:24.877 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.55118.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52303.log 2024-09-10T16:13:24.877 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55118.log.gz 2024-09-10T16:13:24.877 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.38801.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71356.log 2024-09-10T16:13:24.877 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38801.log.gz 2024-09-10T16:13:24.877 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46058.log 2024-09-10T16:13:24.877 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.52303.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52303.log.gz 2024-09-10T16:13:24.877 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.71356.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55664.log 2024-09-10T16:13:24.877 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.71356.log.gz 2024-09-10T16:13:24.877 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.46058.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54824.log 2024-09-10T16:13:24.877 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46058.log.gz 2024-09-10T16:13:24.877 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.55664.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50356.log 2024-09-10T16:13:24.877 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55664.log.gz 2024-09-10T16:13:24.878 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.54824.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59110.log 2024-09-10T16:13:24.878 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54824.log.gz 2024-09-10T16:13:24.878 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.50356.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55706.log 2024-09-10T16:13:24.878 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50356.log.gz 2024-09-10T16:13:24.878 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.59110.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61549.log 2024-09-10T16:13:24.878 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59110.log.gz 2024-09-10T16:13:24.878 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.55706.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37410.log 2024-09-10T16:13:24.878 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55706.log.gz 2024-09-10T16:13:24.878 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.61549.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65542.log 2024-09-10T16:13:24.878 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61549.log.gz 2024-09-10T16:13:24.878 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.37410.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70264.log 2024-09-10T16:13:24.878 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37410.log.gz 2024-09-10T16:13:24.878 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.65542.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62811.log 2024-09-10T16:13:24.879 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65542.log.gz 2024-09-10T16:13:24.879 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49432.log 2024-09-10T16:13:24.879 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.70264.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.70264.log.gz 2024-09-10T16:13:24.879 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.62811.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48045.log 2024-09-10T16:13:24.879 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62811.log.gz 2024-09-10T16:13:24.879 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.49432.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50749.log 2024-09-10T16:13:24.879 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49432.log.gz 2024-09-10T16:13:24.879 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62853.log 2024-09-10T16:13:24.879 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.48045.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48045.log.gz 2024-09-10T16:13:24.879 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.50749.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73162.log 2024-09-10T16:13:24.879 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50749.log.gz 2024-09-10T16:13:24.879 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.62853.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70894.log 2024-09-10T16:13:24.879 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62853.log.gz 2024-09-10T16:13:24.880 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72784.log 2024-09-10T16:13:24.880 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.73162.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.73162.log.gz 2024-09-10T16:13:24.880 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.70894.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70138.log 2024-09-10T16:13:24.880 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.70894.log.gz 2024-09-10T16:13:24.880 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.72784.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59533.log 2024-09-10T16:13:24.880 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.72784.log.gz 2024-09-10T16:13:24.880 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67815.log 2024-09-10T16:13:24.880 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.70138.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.70138.log.gz 2024-09-10T16:13:24.880 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5/var/log/ceph/ceph-client.admin.59533.log: --verbose -- /var/log/ceph/ceph-client.admin.72406.log 2024-09-10T16:13:24.880 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59533.log.gz 2024-09-10T16:13:24.880 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.67815.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74128.log 2024-09-10T16:13:24.880 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67815.log.gz 2024-09-10T16:13:24.880 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69172.log 2024-09-10T16:13:24.880 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.72406.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.72406.log.gz 2024-09-10T16:13:24.881 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.74128.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61339.log 2024-09-10T16:13:24.881 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.74128.log.gz 2024-09-10T16:13:24.881 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.69172.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39588.log 2024-09-10T16:13:24.881 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69172.log.gz 2024-09-10T16:13:24.881 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.61339.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59194.log 2024-09-10T16:13:24.881 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61339.log.gz 2024-09-10T16:13:24.881 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.39588.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67139.log 2024-09-10T16:13:24.881 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39588.log.gz 2024-09-10T16:13:24.881 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.59194.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71776.log 2024-09-10T16:13:24.881 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59194.log.gz 2024-09-10T16:13:24.881 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65038.log 2024-09-10T16:13:24.881 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.67139.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67139.log.gz 2024-09-10T16:13:24.881 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.71776.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54698.log 2024-09-10T16:13:24.881 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.71776.log.gz 2024-09-10T16:13:24.882 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.65038.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48802.log 2024-09-10T16:13:24.882 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65038.log.gz 2024-09-10T16:13:24.882 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.54698.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37996.log 2024-09-10T16:13:24.882 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54698.log.gz 2024-09-10T16:13:24.882 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.48802.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69466.log 2024-09-10T16:13:24.882 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48802.log.gz 2024-09-10T16:13:24.882 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.37996.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42920.log 2024-09-10T16:13:24.882 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37996.log.gz 2024-09-10T16:13:24.882 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.69466.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59659.log 2024-09-10T16:13:24.882 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69466.log.gz 2024-09-10T16:13:24.882 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.42920.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47205.log 2024-09-10T16:13:24.882 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42920.log.gz 2024-09-10T16:13:24.882 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.59659.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73288.log 2024-09-10T16:13:24.883 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59659.log.gz 2024-09-10T16:13:24.883 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.47205.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73792.log 2024-09-10T16:13:24.883 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47205.log.gz 2024-09-10T16:13:24.883 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.73288.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45799.log 2024-09-10T16:13:24.883 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.73288.log.gz 2024-09-10T16:13:24.883 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.73792.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68571.log 2024-09-10T16:13:24.883 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.73792.log.gz 2024-09-10T16:13:24.883 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.45799.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57302.log 2024-09-10T16:13:24.883 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45799.log.gz 2024-09-10T16:13:24.883 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.68571.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38320.log 2024-09-10T16:13:24.883 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68571.log.gz 2024-09-10T16:13:24.883 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.57302.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69087.log 2024-09-10T16:13:24.883 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57302.log.gz 2024-09-10T16:13:24.884 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.38320.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52933.log 2024-09-10T16:13:24.884 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38320.log.gz 2024-09-10T16:13:24.884 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.69087.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74212.log 2024-09-10T16:13:24.884 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69087.log.gz 2024-09-10T16:13:24.884 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.52933.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65669.log 2024-09-10T16:13:24.884 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52933.log.gz 2024-09-10T16:13:24.884 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.74212.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71650.log 2024-09-10T16:13:24.884 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.74212.log.gz 2024-09-10T16:13:24.884 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.65669.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54488.log 2024-09-10T16:13:24.884 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65669.log.gz 2024-09-10T16:13:24.884 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.71650.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48297.log 2024-09-10T16:13:24.884 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.71650.log.gz 2024-09-10T16:13:24.884 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.54488.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53690.log 2024-09-10T16:13:24.885 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54488.log.gz 2024-09-10T16:13:24.885 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.48297.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58394.log 2024-09-10T16:13:24.885 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48297.log.gz 2024-09-10T16:13:24.885 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.53690.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56714.log 2024-09-10T16:13:24.885 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53690.log.gz 2024-09-10T16:13:24.885 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.58394.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36903.log 2024-09-10T16:13:24.885 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58394.log.gz 2024-09-10T16:13:24.885 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.56714.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54782.log 2024-09-10T16:13:24.885 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56714.log.gz 2024-09-10T16:13:24.885 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.36903.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54908.log 2024-09-10T16:13:24.885 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36903.log.gz 2024-09-10T16:13:24.885 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.54782.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63904.log 2024-09-10T16:13:24.885 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54782.log.gz 2024-09-10T16:13:24.886 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.54908.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53270.log 2024-09-10T16:13:24.886 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54908.log.gz 2024-09-10T16:13:24.886 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.63904.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48213.log 2024-09-10T16:13:24.886 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63904.log.gz 2024-09-10T16:13:24.886 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.53270.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51295.log 2024-09-10T16:13:24.886 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53270.log.gz 2024-09-10T16:13:24.886 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.48213.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55496.log 2024-09-10T16:13:24.886 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48213.log.gz 2024-09-10T16:13:24.886 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.51295.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71230.log 2024-09-10T16:13:24.886 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51295.log.gz 2024-09-10T16:13:24.886 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.55496.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38209.log 2024-09-10T16:13:24.886 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55496.log.gz 2024-09-10T16:13:24.886 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.71230.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54740.log 2024-09-10T16:13:24.886 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.71230.log.gz 2024-09-10T16:13:24.887 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.38209.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57218.log 2024-09-10T16:13:24.887 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38209.log.gz 2024-09-10T16:13:24.887 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.54740.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71818.log 2024-09-10T16:13:24.887 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54740.log.gz 2024-09-10T16:13:24.887 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.57218.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63358.log 2024-09-10T16:13:24.887 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57218.log.gz 2024-09-10T16:13:24.887 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.71818.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66929.log 2024-09-10T16:13:24.887 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.71818.log.gz 2024-09-10T16:13:24.887 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.63358.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63232.log 2024-09-10T16:13:24.887 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63358.log.gz 2024-09-10T16:13:24.887 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.66929.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68277.log 2024-09-10T16:13:24.887 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66929.log.gz 2024-09-10T16:13:24.887 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.63232.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68529.log 2024-09-10T16:13:24.888 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63232.log.gz 2024-09-10T16:13:24.888 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.68277.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32293.log 2024-09-10T16:13:24.888 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68277.log.gz 2024-09-10T16:13:24.888 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.68529.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68529.log.gz 2024-09-10T16:13:24.888 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.32293.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32293.log.gz 2024-09-10T16:13:24.888 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 has been restored 2024-09-10T16:13:25.676 INFO:teuthology.orchestra.run.smithi115.stderr: 91.4% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-09-10T16:13:26.493 INFO:teuthology.orchestra.run.smithi115.stderr: 91.5% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-09-10T16:13:27.463 INFO:teuthology.orchestra.run.smithi115.stderr: 89.9% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-09-10T16:13:27.629 INFO:teuthology.orchestra.run.smithi115.stderr: 91.3% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-09-10T16:20:32.957 INFO:teuthology.orchestra.run.smithi115.stderr: 91.8% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-09-10T16:26:24.992 INFO:teuthology.orchestra.run.smithi115.stderr: 90.9% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-09-10T16:27:51.404 INFO:teuthology.orchestra.run.smithi115.stderr: 91.1% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-09-10T16:27:51.406 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-09-10T16:27:51.406 INFO:teuthology.orchestra.run.smithi115.stderr:real 14m26.944s 2024-09-10T16:27:51.406 INFO:teuthology.orchestra.run.smithi115.stderr:user 6m0.966s 2024-09-10T16:27:51.406 INFO:teuthology.orchestra.run.smithi115.stderr:sys 0m16.782s 2024-09-10T16:27:51.407 INFO:tasks.ceph:Archiving logs... 2024-09-10T16:27:51.408 DEBUG:teuthology.misc:Transferring archived files from smithi115:/var/log/ceph to /home/teuthworker/archive/skanta-2024-09-10_15:30:27-rados-wip-bharath7-testing-2024-09-10-1409-distro-default-smithi/7899267/remote/smithi115/log 2024-09-10T16:27:51.412 DEBUG:teuthology.orchestra.run.smithi115:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-10T16:28:57.708 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-10T16:28:57.718 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-10T16:28:57.719 DEBUG:teuthology.orchestra.run.smithi115:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-10T16:28:57.842 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-10T16:28:57.850 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-10T16:28:57.850 DEBUG:teuthology.orchestra.run.smithi115:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-10T16:28:57.867 INFO:teuthology.orchestra.run.smithi115.stderr:bash: line 1: ntpq: command not found 2024-09-10T16:28:57.910 INFO:teuthology.orchestra.run.smithi115.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-10T16:28:57.911 INFO:teuthology.orchestra.run.smithi115.stdout:=============================================================================== 2024-09-10T16:28:57.911 INFO:teuthology.orchestra.run.smithi115.stdout:^* hv01.front.sepia.ceph.com 3 6 377 53 +439us[ +444us] +/- 39ms 2024-09-10T16:28:57.911 INFO:teuthology.orchestra.run.smithi115.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 54 -103us[ -98us] +/- 51ms 2024-09-10T16:28:57.911 INFO:teuthology.orchestra.run.smithi115.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 52 -739us[ -739us] +/- 83ms 2024-09-10T16:28:57.911 INFO:teuthology.orchestra.run.smithi115.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T16:28:57.911 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-10T16:28:57.920 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-10T16:28:57.920 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-10T16:28:57.957 DEBUG:teuthology.orchestra.run.smithi115:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-10T16:28:58.101 DEBUG:teuthology.orchestra.run.smithi115:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-10T16:28:58.133 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T16:28:58.133 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-10T16:28:58.142 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-10T16:28:58.207 INFO:teuthology.task.internal:Duration was 2287.168306 seconds 2024-09-10T16:28:58.208 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-10T16:28:58.257 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-10T16:28:58.258 DEBUG:teuthology.orchestra.run.smithi115:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-10T16:28:58.359 INFO:teuthology.orchestra.run.smithi115.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-10T16:28:58.725 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-10T16:28:58.725 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi115.front.sepia.ceph.com 2024-09-10T16:28:58.726 DEBUG:teuthology.orchestra.run.smithi115:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-10T16:28:58.749 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-10T16:28:58.749 DEBUG:teuthology.orchestra.run.smithi115:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-10T16:28:58.818 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-10T16:28:58.819 DEBUG:teuthology.orchestra.run.smithi115:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-10T16:28:58.996 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-10T16:28:59.006 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-10T16:28:59.006 DEBUG:teuthology.orchestra.run.smithi115:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-10T16:28:59.031 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-10T16:28:59.050 DEBUG:teuthology.orchestra.run.smithi115:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-10T16:28:59.115 INFO:teuthology.orchestra.run.smithi115.stdout:kernel.core_pattern = core 2024-09-10T16:28:59.196 DEBUG:teuthology.orchestra.run.smithi115:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-10T16:28:59.250 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T16:28:59.251 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-10T16:28:59.261 INFO:teuthology.task.internal:Transferring archived files... 2024-09-10T16:28:59.262 DEBUG:teuthology.misc:Transferring archived files from smithi115:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/skanta-2024-09-10_15:30:27-rados-wip-bharath7-testing-2024-09-10-1409-distro-default-smithi/7899267/remote/smithi115 2024-09-10T16:28:59.262 DEBUG:teuthology.orchestra.run.smithi115:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-10T16:28:59.326 INFO:teuthology.task.internal:Removing archive directory... 2024-09-10T16:28:59.326 DEBUG:teuthology.orchestra.run.smithi115:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-10T16:28:59.376 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-10T16:28:59.385 INFO:teuthology.task.internal:Not uploading archives. 2024-09-10T16:28:59.386 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-10T16:28:59.425 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-10T16:28:59.427 DEBUG:teuthology.orchestra.run.smithi115:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-10T16:28:59.442 INFO:teuthology.orchestra.run.smithi115.stdout: 265340 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 10 16:28 /home/ubuntu/cephtest 2024-09-10T16:28:59.443 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-10T16:28:59.475 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-10T16:28:59.577 INFO:teuthology.run:Summary data: description: rados/singleton/{all/lost-unfound mon_election/classic msgr-failures/none msgr/async-v2only objectstore/{bluestore-options/write$/{write_v1} bluestore/bluestore-stupid} rados supported-random-distro$/{centos_latest}} duration: 2287.168305873871 flavor: default owner: scheduled_skanta@teuthology success: true 2024-09-10T16:28:59.577 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-10T16:28:59.656 INFO:teuthology.run:pass